2024-09-17T04:12:11.646 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-17T04:12:11.668 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T04:12:11.758 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908519 branch: wip-pdonnell-testing-20240916.200549-debug description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}} email: pdonnell@redhat.com first_in_suite: false job_id: '7908519' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240916.200549-debug ceph: cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: client: pid file: '' global: ms die on skipped message: false mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf kclient: syntax: v1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240916.200549-debug sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 476 sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf sleep_before_teardown: 0 subset: 320/512 suite: fs suite_branch: wip-pdonnell-testing-20240916.200549-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi100.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMyWBlyrgUGcQVVd5EecuSrhbQiyprjA+Ap91cVKUAQIKdwv+8mg/vWvNw/b/2QfwpFf1C7CytUYz9j+XKLTVhQ= smithi184.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMo7DdF79+dnZHSEM9/GUw5Xd3VJT4TNL3oUbwqu1c0jsRXZWKRkiZeLh4OdNrUsD3ID85c4vC6jpfP0H/KcnL8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestMisc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-17_02:09:51 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-17T04:12:11.758 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa; will attempt to use it 2024-09-17T04:12:11.759 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa/tasks 2024-09-17T04:12:11.759 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-17T04:12:11.761 INFO:teuthology.task.internal:Checking packages... 2024-09-17T04:12:11.782 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf' 2024-09-17T04:12:11.782 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-17T04:12:11.782 INFO:teuthology.packaging:ref: None 2024-09-17T04:12:11.782 INFO:teuthology.packaging:tag: None 2024-09-17T04:12:11.782 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240916.200549-debug 2024-09-17T04:12:11.782 INFO:teuthology.packaging:sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T04:12:11.783 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240916.200549-debug 2024-09-17T04:12:11.927 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4989.gce1b342b 2024-09-17T04:12:11.928 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-17T04:12:11.935 INFO:teuthology.task.internal:no buildpackages task found 2024-09-17T04:12:11.935 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-17T04:12:11.975 INFO:teuthology.task.internal:Saving configuration 2024-09-17T04:12:11.992 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-17T04:12:12.000 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-17T04:12:12.026 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908519', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 04:07:22.555884', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMyWBlyrgUGcQVVd5EecuSrhbQiyprjA+Ap91cVKUAQIKdwv+8mg/vWvNw/b/2QfwpFf1C7CytUYz9j+XKLTVhQ='} 2024-09-17T04:12:12.049 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908519', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 04:07:22.554057', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMo7DdF79+dnZHSEM9/GUw5Xd3VJT4TNL3oUbwqu1c0jsRXZWKRkiZeLh4OdNrUsD3ID85c4vC6jpfP0H/KcnL8='} 2024-09-17T04:12:12.049 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-17T04:12:12.056 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-17T04:12:12.056 INFO:teuthology.task.internal:roles: ubuntu@smithi184.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-17T04:12:12.056 INFO:teuthology.run_tasks:Running task console_log... 2024-09-17T04:12:12.133 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f606e6ffac0>, signals=[15]) 2024-09-17T04:12:12.133 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-17T04:12:12.165 INFO:teuthology.task.internal:Opening connections... 2024-09-17T04:12:12.165 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi100.front.sepia.ceph.com 2024-09-17T04:12:12.167 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:12:12.242 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi184.front.sepia.ceph.com 2024-09-17T04:12:12.242 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:12:12.316 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-17T04:12:12.322 DEBUG:teuthology.orchestra.run.smithi100:> uname -m 2024-09-17T04:12:12.341 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2024-09-17T04:12:12.341 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2024-09-17T04:12:12.399 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="CentOS Stream" 2024-09-17T04:12:12.399 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="9" 2024-09-17T04:12:12.399 INFO:teuthology.orchestra.run.smithi100.stdout:ID="centos" 2024-09-17T04:12:12.399 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="rhel fedora" 2024-09-17T04:12:12.399 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="9" 2024-09-17T04:12:12.399 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el9" 2024-09-17T04:12:12.399 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T04:12:12.399 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31" 2024-09-17T04:12:12.399 INFO:teuthology.orchestra.run.smithi100.stdout:LOGO="fedora-logo-icon" 2024-09-17T04:12:12.399 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T04:12:12.399 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://centos.org/" 2024-09-17T04:12:12.400 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T04:12:12.400 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T04:12:12.400 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T04:12:12.400 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2024-09-17T04:12:12.420 DEBUG:teuthology.orchestra.run.smithi184:> uname -m 2024-09-17T04:12:12.439 INFO:teuthology.orchestra.run.smithi184.stdout:x86_64 2024-09-17T04:12:12.439 DEBUG:teuthology.orchestra.run.smithi184:> cat /etc/os-release 2024-09-17T04:12:12.497 INFO:teuthology.orchestra.run.smithi184.stdout:NAME="CentOS Stream" 2024-09-17T04:12:12.497 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION="9" 2024-09-17T04:12:12.497 INFO:teuthology.orchestra.run.smithi184.stdout:ID="centos" 2024-09-17T04:12:12.497 INFO:teuthology.orchestra.run.smithi184.stdout:ID_LIKE="rhel fedora" 2024-09-17T04:12:12.497 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_ID="9" 2024-09-17T04:12:12.497 INFO:teuthology.orchestra.run.smithi184.stdout:PLATFORM_ID="platform:el9" 2024-09-17T04:12:12.497 INFO:teuthology.orchestra.run.smithi184.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T04:12:12.497 INFO:teuthology.orchestra.run.smithi184.stdout:ANSI_COLOR="0;31" 2024-09-17T04:12:12.497 INFO:teuthology.orchestra.run.smithi184.stdout:LOGO="fedora-logo-icon" 2024-09-17T04:12:12.498 INFO:teuthology.orchestra.run.smithi184.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T04:12:12.498 INFO:teuthology.orchestra.run.smithi184.stdout:HOME_URL="https://centos.org/" 2024-09-17T04:12:12.498 INFO:teuthology.orchestra.run.smithi184.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T04:12:12.498 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T04:12:12.498 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T04:12:12.498 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server 2024-09-17T04:12:12.520 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-17T04:12:12.529 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-17T04:12:12.567 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-17T04:12:12.567 DEBUG:teuthology.orchestra.run.smithi100:> test '!' -e /home/ubuntu/cephtest 2024-09-17T04:12:12.570 DEBUG:teuthology.orchestra.run.smithi184:> test '!' -e /home/ubuntu/cephtest 2024-09-17T04:12:12.586 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-17T04:12:12.653 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-17T04:12:12.653 DEBUG:teuthology.orchestra.run.smithi100:> test -z $(ls -A /var/lib/ceph) 2024-09-17T04:12:12.656 DEBUG:teuthology.orchestra.run.smithi184:> test -z $(ls -A /var/lib/ceph) 2024-09-17T04:12:12.755 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-17T04:12:12.811 INFO:teuthology.run_tasks:Running task kernel... 2024-09-17T04:12:12.825 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-17T04:12:12.825 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-09-17T04:12:12.825 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-09-17T04:12:12.826 DEBUG:teuthology.orchestra.run.smithi100:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T04:12:12.826 DEBUG:teuthology.orchestra.run.smithi184:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T04:12:12.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T04:12:12.843 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-17T04:12:12.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T04:12:12.846 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-17T04:12:13.023 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T04:12:13.024 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-17T04:12:13.024 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-09-17T04:12:13.026 DEBUG:teuthology.task.kernel:sha1 for client.1 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T04:12:13.026 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-17T04:12:13.026 DEBUG:teuthology.orchestra.run.smithi184:> uname -r 2024-09-17T04:12:13.041 INFO:teuthology.orchestra.run.smithi100.stdout:5.14.0-503.el9.x86_64 2024-09-17T04:12:13.042 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T04:12:13.042 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-17T04:12:13.042 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum upgrade -y linux-firmware 2024-09-17T04:12:13.044 INFO:teuthology.orchestra.run.smithi184.stdout:5.14.0-503.el9.x86_64 2024-09-17T04:12:13.044 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T04:12:13.044 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-17T04:12:13.044 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum upgrade -y linux-firmware 2024-09-17T04:12:16.482 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.3 MB 00:01 2024-09-17T04:12:17.358 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - BaseOS 3.3 MB/s | 8.3 MB 00:02 2024-09-17T04:12:19.526 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:01 2024-09-17T04:12:19.681 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - AppStream 16 MB/s | 20 MB 00:01 2024-09-17T04:12:24.032 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-17T04:12:24.201 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 6.5 MB 00:00 2024-09-17T04:12:25.824 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-17T04:12:25.843 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 19 kB 00:00 2024-09-17T04:12:27.005 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-17T04:12:27.137 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 19 MB/s | 23 MB 00:01 2024-09-17T04:12:32.453 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-17T04:12:32.559 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-17T04:12:34.404 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-09-17T04:12:34.412 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-09-17T04:12:34.412 INFO:teuthology.orchestra.run.smithi100.stdout: Package Architecture Version Repository Size 2024-09-17T04:12:34.412 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-09-17T04:12:34.412 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrading: 2024-09-17T04:12:34.412 INFO:teuthology.orchestra.run.smithi100.stdout: linux-firmware noarch 20240905-146.el9 baseos 388 M 2024-09-17T04:12:34.412 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:12:34.413 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-09-17T04:12:34.413 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-09-17T04:12:34.413 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrade 1 Package 2024-09-17T04:12:34.413 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:12:34.413 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 388 M 2024-09-17T04:12:34.413 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-09-17T04:12:34.554 INFO:teuthology.orchestra.run.smithi100.stdout:[MIRROR] linux-firmware-20240905-146.el9.noarch.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/linux-firmware-20240905-146.el9.noarch.rpm (IP: 206.82.17.213) 2024-09-17T04:12:34.603 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-17T04:12:34.612 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-17T04:12:34.612 INFO:teuthology.orchestra.run.smithi184.stdout: Package Architecture Version Repository Size 2024-09-17T04:12:34.612 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-17T04:12:34.612 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrading: 2024-09-17T04:12:34.613 INFO:teuthology.orchestra.run.smithi184.stdout: linux-firmware noarch 20240905-146.el9 baseos 388 M 2024-09-17T04:12:34.613 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:12:34.613 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-17T04:12:34.613 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-17T04:12:34.613 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrade 1 Package 2024-09-17T04:12:34.613 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:12:34.613 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 388 M 2024-09-17T04:12:34.613 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-17T04:12:34.633 INFO:teuthology.orchestra.run.smithi100.stdout:[MIRROR] linux-firmware-20240905-146.el9.noarch.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/linux-firmware-20240905-146.el9.noarch.rpm (IP: 206.82.17.213) 2024-09-17T04:12:34.829 INFO:teuthology.orchestra.run.smithi100.stdout:[MIRROR] linux-firmware-20240905-146.el9.noarch.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/linux-firmware-20240905-146.el9.noarch.rpm (IP: 172.65.90.23) 2024-09-17T04:12:35.050 INFO:teuthology.orchestra.run.smithi100.stdout:[MIRROR] linux-firmware-20240905-146.el9.noarch.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/linux-firmware-20240905-146.el9.noarch.rpm (IP: 172.65.90.20) 2024-09-17T04:12:43.301 INFO:teuthology.orchestra.run.smithi184.stdout:linux-firmware-20240905-146.el9.noarch.rpm 45 MB/s | 388 MB 00:08 2024-09-17T04:12:43.302 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-17T04:12:43.302 INFO:teuthology.orchestra.run.smithi184.stdout:Total 45 MB/s | 388 MB 00:08 2024-09-17T04:12:46.429 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-17T04:12:46.611 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-17T04:12:46.612 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-17T04:12:49.918 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-17T04:12:49.919 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-17T04:12:59.801 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-17T04:13:00.186 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-17T04:13:00.266 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-17T04:13:00.860 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: linux-firmware-20240716-145.el9.noarch 2/2 2024-09-17T04:13:00.861 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-17T04:13:01.658 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-17T04:13:01.659 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:13:01.659 INFO:teuthology.orchestra.run.smithi184.stdout:Upgraded: 2024-09-17T04:13:01.659 INFO:teuthology.orchestra.run.smithi184.stdout: linux-firmware-20240905-146.el9.noarch 2024-09-17T04:13:01.659 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:13:01.659 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-17T04:13:01.984 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-17T04:13:01.985 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T04:13:02.138 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-17T04:13:02.138 DEBUG:teuthology.orchestra.run.smithi184:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-17T04:13:02.141 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi184.front.sepia.ceph.com to complete... 2024-09-17T04:13:04.042 INFO:teuthology.orchestra.run.smithi100.stdout:linux-firmware-20240905-146.el9.noarch.rpm 13 MB/s | 388 MB 00:29 2024-09-17T04:13:04.045 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-09-17T04:13:04.045 INFO:teuthology.orchestra.run.smithi100.stdout:Total 13 MB/s | 388 MB 00:29 2024-09-17T04:13:07.158 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-09-17T04:13:07.249 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-09-17T04:13:07.249 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-09-17T04:13:08.443 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17 04:13:08 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-17T04:13:08.443 INFO:teuthology.orchestra.run.smithi184.stderr:FINISHED --2024-09-17 04:13:08-- 2024-09-17T04:13:08.443 INFO:teuthology.orchestra.run.smithi184.stderr:Total wall clock time: 6.3s 2024-09-17T04:13:08.443 INFO:teuthology.orchestra.run.smithi184.stderr:Downloaded: 1 files, 343M in 6.0s (57.1 MB/s) 2024-09-17T04:13:08.447 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-17T04:13:08.447 DEBUG:teuthology.orchestra.run.smithi184:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-17T04:13:10.504 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-09-17T04:13:10.505 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-09-17T04:13:10.512 INFO:teuthology.orchestra.run.smithi184.stdout:Verifying... ######################################## 2024-09-17T04:13:10.657 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing... ######################################## 2024-09-17T04:13:10.717 INFO:teuthology.orchestra.run.smithi184.stdout:Updating / installing... 2024-09-17T04:13:18.764 INFO:teuthology.orchestra.run.smithi184.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-17T04:13:19.792 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-09-17T04:13:20.544 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-17T04:13:20.625 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-17T04:13:21.123 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: linux-firmware-20240716-145.el9.noarch 2/2 2024-09-17T04:13:21.123 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-17T04:13:21.863 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-17T04:13:21.863 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:13:21.863 INFO:teuthology.orchestra.run.smithi100.stdout:Upgraded: 2024-09-17T04:13:21.863 INFO:teuthology.orchestra.run.smithi100.stdout: linux-firmware-20240905-146.el9.noarch 2024-09-17T04:13:21.863 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:13:21.864 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-09-17T04:13:22.181 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-17T04:13:22.181 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T04:13:22.336 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-17T04:13:22.337 DEBUG:teuthology.orchestra.run.smithi100:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-17T04:13:22.339 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi100.front.sepia.ceph.com to complete... 2024-09-17T04:13:28.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17 04:13:28 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-17T04:13:28.642 INFO:teuthology.orchestra.run.smithi100.stderr:FINISHED --2024-09-17 04:13:28-- 2024-09-17T04:13:28.642 INFO:teuthology.orchestra.run.smithi100.stderr:Total wall clock time: 6.3s 2024-09-17T04:13:28.642 INFO:teuthology.orchestra.run.smithi100.stderr:Downloaded: 1 files, 343M in 6.0s (57.1 MB/s) 2024-09-17T04:13:28.645 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-17T04:13:28.645 DEBUG:teuthology.orchestra.run.smithi100:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-17T04:13:30.705 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying... ######################################## 2024-09-17T04:13:30.834 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing... ######################################## 2024-09-17T04:13:30.894 INFO:teuthology.orchestra.run.smithi100.stdout:Updating / installing... 2024-09-17T04:13:38.869 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-17T04:13:59.147 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi184.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-17T04:13:59.147 DEBUG:teuthology.orchestra.run.smithi184:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-17T04:13:59.250 INFO:teuthology.orchestra.run.smithi184.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:13:59.250 INFO:teuthology.orchestra.run.smithi184.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:13:59.250 INFO:teuthology.orchestra.run.smithi184.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:13:59.250 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:13:59.251 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-17T04:13:59.251 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-17T04:13:59.251 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-17T04:13:59.251 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-17T04:13:59.251 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-17T04:13:59.251 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-17T04:13:59.251 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-17T04:13:59.251 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-17T04:13:59.251 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-17T04:13:59.251 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-17T04:13:59.251 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-17T04:13:59.252 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-17T04:13:59.252 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-17T04:13:59.252 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-17T04:13:59.252 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-17T04:13:59.252 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-17T04:13:59.252 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-17T04:13:59.252 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-17T04:13:59.252 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-17T04:13:59.252 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-17T04:13:59.252 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-17T04:13:59.252 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-17T04:13:59.252 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-17T04:13:59.253 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-17T04:13:59.253 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-17T04:13:59.253 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-17T04:13:59.253 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-17T04:13:59.253 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-17T04:13:59.253 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-17T04:13:59.253 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-17T04:13:59.253 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-17T04:13:59.253 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-17T04:13:59.253 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-17T04:13:59.253 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-17T04:13:59.254 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-17T04:13:59.254 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-17T04:13:59.254 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-17T04:13:59.254 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-17T04:13:59.254 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-17T04:13:59.254 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-17T04:13:59.254 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-17T04:13:59.254 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-17T04:13:59.254 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-17T04:13:59.254 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-17T04:13:59.254 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-17T04:13:59.254 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-17T04:13:59.255 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-17T04:13:59.255 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-17T04:13:59.255 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-17T04:13:59.255 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-17T04:13:59.255 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-17T04:13:59.255 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-17T04:13:59.255 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-17T04:13:59.255 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-17T04:13:59.255 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-17T04:13:59.255 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-17T04:13:59.255 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-17T04:13:59.255 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-17T04:13:59.256 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-17T04:13:59.256 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-17T04:13:59.256 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-17T04:13:59.256 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-17T04:13:59.256 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-17T04:13:59.256 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-17T04:13:59.256 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-17T04:13:59.256 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-17T04:13:59.256 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-17T04:13:59.256 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-17T04:13:59.256 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-17T04:13:59.256 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-17T04:13:59.257 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-17T04:13:59.257 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-17T04:13:59.257 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-17T04:13:59.257 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-17T04:13:59.257 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-17T04:13:59.257 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-17T04:13:59.257 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-17T04:13:59.257 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-17T04:13:59.257 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-17T04:13:59.257 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-17T04:13:59.257 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-17T04:13:59.257 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-17T04:13:59.258 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-17T04:13:59.258 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-17T04:13:59.258 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-17T04:13:59.258 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-17T04:13:59.258 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-17T04:13:59.258 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-17T04:13:59.258 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-17T04:13:59.258 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-17T04:13:59.258 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-17T04:13:59.258 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-17T04:13:59.259 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-17T04:13:59.259 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-17T04:13:59.259 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-17T04:13:59.259 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-17T04:13:59.259 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-17T04:13:59.259 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-17T04:13:59.259 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-17T04:13:59.259 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-17T04:13:59.259 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-17T04:13:59.259 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-17T04:13:59.259 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-17T04:13:59.260 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-17T04:13:59.260 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-17T04:13:59.260 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-17T04:13:59.260 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-17T04:13:59.260 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-17T04:13:59.260 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-17T04:13:59.260 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-17T04:13:59.260 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-17T04:13:59.260 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-17T04:13:59.260 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-17T04:13:59.260 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-17T04:13:59.261 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-17T04:13:59.261 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-17T04:13:59.261 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-17T04:13:59.261 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-17T04:13:59.261 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-17T04:13:59.261 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-17T04:13:59.261 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-17T04:13:59.261 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-17T04:13:59.261 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-17T04:13:59.261 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-17T04:13:59.261 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-17T04:13:59.261 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-17T04:13:59.262 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-17T04:13:59.262 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-17T04:13:59.262 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-17T04:13:59.262 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-17T04:13:59.262 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-17T04:13:59.262 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-17T04:13:59.262 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-17T04:13:59.262 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-17T04:13:59.262 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-17T04:13:59.263 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-17T04:13:59.263 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-17T04:13:59.263 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-17T04:13:59.263 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-17T04:13:59.263 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-17T04:13:59.263 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-17T04:13:59.263 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-17T04:13:59.263 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-17T04:13:59.263 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-17T04:13:59.263 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-17T04:13:59.263 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-17T04:13:59.263 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-17T04:13:59.263 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-17T04:13:59.264 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-17T04:13:59.264 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-17T04:13:59.264 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-17T04:13:59.264 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-17T04:13:59.264 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-17T04:13:59.264 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-17T04:13:59.264 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-17T04:13:59.264 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-17T04:13:59.264 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-17T04:13:59.264 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-17T04:13:59.264 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-17T04:13:59.265 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-17T04:13:59.265 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-17T04:13:59.265 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-17T04:13:59.265 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-17T04:13:59.265 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-17T04:13:59.265 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-17T04:13:59.265 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-17T04:13:59.265 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-17T04:13:59.265 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-17T04:13:59.265 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-17T04:13:59.265 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-17T04:13:59.265 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-17T04:13:59.266 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-17T04:13:59.266 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-17T04:13:59.266 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-17T04:13:59.266 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-17T04:13:59.266 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-17T04:13:59.266 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-17T04:13:59.266 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-17T04:13:59.266 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-17T04:13:59.266 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-17T04:13:59.266 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-17T04:13:59.266 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-17T04:13:59.266 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-17T04:13:59.266 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-17T04:13:59.267 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-17T04:13:59.267 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-17T04:13:59.267 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-17T04:13:59.267 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-17T04:13:59.267 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-17T04:13:59.267 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-17T04:13:59.267 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-17T04:13:59.267 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-17T04:13:59.267 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-17T04:13:59.267 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-17T04:13:59.267 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-17T04:13:59.267 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-17T04:13:59.267 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-17T04:13:59.267 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-17T04:13:59.268 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-17T04:13:59.268 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-17T04:13:59.268 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-17T04:13:59.268 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-17T04:13:59.268 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-17T04:13:59.268 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-17T04:13:59.268 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-17T04:13:59.268 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-17T04:13:59.268 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-17T04:13:59.268 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-17T04:13:59.268 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-17T04:13:59.268 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-17T04:13:59.269 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-17T04:13:59.269 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-17T04:13:59.269 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-17T04:13:59.269 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-17T04:13:59.269 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-17T04:13:59.269 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-17T04:13:59.269 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-17T04:13:59.269 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-17T04:13:59.269 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-17T04:13:59.270 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-17T04:13:59.270 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-17T04:13:59.270 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-17T04:13:59.270 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-17T04:13:59.270 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-17T04:13:59.270 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-17T04:13:59.270 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-17T04:13:59.270 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-17T04:13:59.270 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-17T04:13:59.270 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-17T04:13:59.270 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-17T04:13:59.270 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-17T04:13:59.270 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-17T04:13:59.271 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-17T04:13:59.271 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-17T04:13:59.271 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-17T04:13:59.271 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-17T04:13:59.271 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-17T04:13:59.271 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-17T04:13:59.271 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-17T04:13:59.271 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-17T04:13:59.271 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-17T04:13:59.271 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-17T04:13:59.271 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-17T04:13:59.271 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-17T04:13:59.271 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-17T04:13:59.272 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-17T04:13:59.272 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-17T04:13:59.272 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-17T04:13:59.272 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-17T04:13:59.272 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-17T04:13:59.272 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-17T04:13:59.272 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-17T04:13:59.272 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-17T04:13:59.272 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-17T04:13:59.272 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-17T04:13:59.272 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-17T04:13:59.272 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-17T04:13:59.272 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-17T04:13:59.273 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-17T04:13:59.273 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-17T04:13:59.273 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-17T04:13:59.273 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-17T04:13:59.273 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-17T04:13:59.273 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-17T04:13:59.273 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-17T04:13:59.273 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-17T04:13:59.273 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-17T04:13:59.273 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-17T04:13:59.273 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-17T04:13:59.274 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-17T04:13:59.274 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-17T04:13:59.274 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-17T04:13:59.274 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-17T04:13:59.274 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-17T04:13:59.274 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-17T04:13:59.274 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-17T04:13:59.274 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-17T04:13:59.274 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-17T04:13:59.274 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-17T04:13:59.274 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-17T04:13:59.274 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-17T04:13:59.275 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-17T04:13:59.275 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-17T04:13:59.275 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-17T04:13:59.275 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-17T04:13:59.275 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-17T04:13:59.275 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-17T04:13:59.275 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-17T04:13:59.275 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-17T04:13:59.275 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-17T04:13:59.275 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-17T04:13:59.275 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-17T04:13:59.275 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-17T04:13:59.276 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-17T04:13:59.276 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-17T04:13:59.276 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-17T04:13:59.276 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-17T04:13:59.276 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-17T04:13:59.276 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-17T04:13:59.276 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-17T04:13:59.276 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-17T04:13:59.276 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-17T04:13:59.276 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-17T04:13:59.276 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-17T04:13:59.276 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-17T04:13:59.276 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-17T04:13:59.277 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-17T04:13:59.277 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-17T04:13:59.277 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-17T04:13:59.277 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-17T04:13:59.277 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-17T04:13:59.277 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-17T04:13:59.278 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-17T04:13:59.278 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-17T04:13:59.278 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-17T04:13:59.278 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-17T04:13:59.278 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-17T04:13:59.278 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-17T04:13:59.278 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-17T04:13:59.278 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-17T04:13:59.278 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-17T04:13:59.278 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-17T04:13:59.278 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-17T04:13:59.278 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-17T04:13:59.278 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-17T04:13:59.279 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-17T04:13:59.279 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-17T04:13:59.279 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-17T04:13:59.279 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-17T04:13:59.279 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-17T04:13:59.279 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-17T04:13:59.279 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-17T04:13:59.279 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-17T04:13:59.279 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-17T04:13:59.279 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-17T04:13:59.279 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-17T04:13:59.280 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-17T04:13:59.280 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-17T04:13:59.280 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-17T04:13:59.280 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-17T04:13:59.280 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-17T04:13:59.280 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-17T04:13:59.280 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-17T04:13:59.280 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-17T04:13:59.280 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-17T04:13:59.280 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-17T04:13:59.280 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-17T04:13:59.281 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-17T04:13:59.281 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-17T04:13:59.281 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-17T04:13:59.281 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-17T04:13:59.281 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-17T04:13:59.281 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-17T04:13:59.281 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-17T04:13:59.281 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-17T04:13:59.281 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-17T04:13:59.281 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-17T04:13:59.281 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-17T04:13:59.281 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-17T04:13:59.281 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-17T04:13:59.282 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-17T04:13:59.282 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-17T04:13:59.282 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-17T04:13:59.282 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-17T04:13:59.282 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-17T04:13:59.282 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-17T04:13:59.282 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-17T04:13:59.282 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-17T04:13:59.282 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-17T04:13:59.282 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-17T04:13:59.283 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-17T04:13:59.283 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-17T04:13:59.283 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-17T04:13:59.283 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-17T04:13:59.283 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-17T04:13:59.283 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-17T04:13:59.283 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-17T04:13:59.283 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-17T04:13:59.283 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-17T04:13:59.283 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-17T04:13:59.283 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-17T04:13:59.283 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-17T04:13:59.283 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-17T04:13:59.283 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-17T04:13:59.284 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-17T04:13:59.284 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-17T04:13:59.284 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-17T04:13:59.284 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-17T04:13:59.284 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-17T04:13:59.284 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-17T04:13:59.284 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-17T04:13:59.284 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-17T04:13:59.284 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-17T04:13:59.284 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-17T04:13:59.285 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-17T04:13:59.285 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-17T04:13:59.285 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-17T04:13:59.285 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-17T04:13:59.285 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-17T04:13:59.285 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-17T04:13:59.285 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-17T04:13:59.285 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-17T04:13:59.285 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-17T04:13:59.285 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-17T04:13:59.285 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-17T04:13:59.285 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-17T04:13:59.285 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-17T04:13:59.286 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-17T04:13:59.286 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-17T04:13:59.286 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-17T04:13:59.286 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-17T04:13:59.286 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-17T04:13:59.286 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-17T04:13:59.286 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-17T04:13:59.286 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-17T04:13:59.286 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-17T04:13:59.286 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-17T04:13:59.286 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-17T04:13:59.286 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-17T04:13:59.287 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-17T04:13:59.287 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-17T04:13:59.287 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-17T04:13:59.287 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-17T04:13:59.287 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-17T04:13:59.287 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-17T04:13:59.287 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-17T04:13:59.287 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-17T04:13:59.287 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-17T04:13:59.287 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-17T04:13:59.287 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-17T04:13:59.287 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-17T04:13:59.288 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-17T04:13:59.288 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-17T04:13:59.288 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-17T04:13:59.288 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-17T04:13:59.288 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-17T04:13:59.288 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-17T04:13:59.288 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-17T04:13:59.288 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-17T04:13:59.288 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-17T04:13:59.288 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-17T04:13:59.288 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-17T04:13:59.288 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-17T04:13:59.288 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-17T04:13:59.289 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-17T04:13:59.289 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-17T04:13:59.289 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-17T04:13:59.289 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-17T04:13:59.289 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-17T04:13:59.289 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-17T04:13:59.289 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-17T04:13:59.289 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-17T04:13:59.289 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-17T04:13:59.289 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-17T04:13:59.289 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-17T04:13:59.289 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-17T04:13:59.289 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-17T04:13:59.290 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-17T04:13:59.290 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-17T04:13:59.290 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-17T04:13:59.290 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-17T04:13:59.290 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-17T04:13:59.290 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-17T04:13:59.290 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-17T04:13:59.290 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-17T04:13:59.290 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-17T04:13:59.291 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-17T04:13:59.291 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-17T04:13:59.291 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-17T04:13:59.291 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-17T04:13:59.291 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-17T04:13:59.291 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-17T04:13:59.291 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-17T04:13:59.291 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-17T04:13:59.291 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-17T04:13:59.291 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-17T04:13:59.291 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-17T04:13:59.291 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-17T04:13:59.291 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-17T04:13:59.292 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-17T04:13:59.292 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-17T04:13:59.292 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-17T04:13:59.292 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-17T04:13:59.292 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-17T04:13:59.292 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-17T04:13:59.292 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-17T04:13:59.292 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-17T04:13:59.292 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-17T04:13:59.292 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-17T04:13:59.292 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-17T04:13:59.292 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-17T04:13:59.292 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-17T04:13:59.293 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-17T04:13:59.293 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-17T04:13:59.293 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-17T04:13:59.293 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-17T04:13:59.293 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-17T04:13:59.293 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-17T04:13:59.293 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-17T04:13:59.293 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-17T04:13:59.293 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-17T04:13:59.293 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-17T04:13:59.293 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-17T04:13:59.293 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-17T04:13:59.294 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-17T04:13:59.294 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-17T04:13:59.294 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-17T04:13:59.294 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-17T04:13:59.294 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-17T04:13:59.294 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-17T04:13:59.294 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-17T04:13:59.294 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-17T04:13:59.294 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-17T04:13:59.294 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-17T04:13:59.294 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-17T04:13:59.294 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-17T04:13:59.294 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-17T04:13:59.294 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-17T04:13:59.295 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-17T04:13:59.295 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-17T04:13:59.295 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-17T04:13:59.295 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-17T04:13:59.295 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-17T04:13:59.295 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-17T04:13:59.295 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-17T04:13:59.295 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-17T04:13:59.295 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-17T04:13:59.295 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-17T04:13:59.295 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-17T04:13:59.295 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-17T04:13:59.295 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-17T04:13:59.296 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-17T04:13:59.296 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-17T04:13:59.296 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-17T04:13:59.296 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-17T04:13:59.296 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-17T04:13:59.296 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-17T04:13:59.296 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-17T04:13:59.296 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-17T04:13:59.296 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-17T04:13:59.296 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-17T04:13:59.296 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-17T04:13:59.296 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-17T04:13:59.296 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-17T04:13:59.297 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-17T04:13:59.297 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-17T04:13:59.297 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-17T04:13:59.297 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-17T04:13:59.297 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-17T04:13:59.297 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-17T04:13:59.297 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-17T04:13:59.297 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-17T04:13:59.297 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-17T04:13:59.297 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-17T04:13:59.297 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-17T04:13:59.297 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-17T04:13:59.298 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-17T04:13:59.298 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-17T04:13:59.298 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-17T04:13:59.298 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-17T04:13:59.298 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-17T04:13:59.298 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-17T04:13:59.298 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-17T04:13:59.298 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-17T04:13:59.298 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-17T04:13:59.298 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-17T04:13:59.298 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-17T04:13:59.298 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-17T04:13:59.299 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-17T04:13:59.299 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-17T04:13:59.299 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-17T04:13:59.299 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-17T04:13:59.299 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-17T04:13:59.299 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-17T04:13:59.299 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-17T04:13:59.300 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-17T04:13:59.300 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-17T04:13:59.300 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-17T04:13:59.300 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-17T04:13:59.301 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-17T04:13:59.301 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-17T04:13:59.301 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-17T04:13:59.301 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-17T04:13:59.301 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-17T04:13:59.301 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-17T04:13:59.301 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-17T04:13:59.301 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-17T04:13:59.301 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-17T04:13:59.301 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-17T04:13:59.302 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-17T04:13:59.302 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-17T04:13:59.302 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-17T04:13:59.302 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-17T04:13:59.302 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-17T04:13:59.302 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-17T04:13:59.302 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-17T04:13:59.302 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-17T04:13:59.302 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-17T04:13:59.302 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-17T04:13:59.303 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-17T04:13:59.303 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-17T04:13:59.303 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-17T04:13:59.303 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-17T04:13:59.303 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-17T04:13:59.303 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-17T04:13:59.303 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-17T04:13:59.303 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-17T04:13:59.303 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-17T04:13:59.303 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-17T04:13:59.303 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-17T04:13:59.304 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-17T04:13:59.304 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-17T04:13:59.304 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-17T04:13:59.304 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-17T04:13:59.304 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-17T04:13:59.304 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-17T04:13:59.304 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-17T04:13:59.304 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-17T04:13:59.304 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-17T04:13:59.304 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-17T04:13:59.304 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-17T04:13:59.304 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-17T04:13:59.305 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-17T04:13:59.305 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-17T04:13:59.305 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-17T04:13:59.305 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-17T04:13:59.305 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-17T04:13:59.305 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-17T04:13:59.305 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-17T04:13:59.305 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-17T04:13:59.305 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-17T04:13:59.305 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-17T04:13:59.305 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-17T04:13:59.305 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-17T04:13:59.306 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-17T04:13:59.306 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-17T04:13:59.306 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-17T04:13:59.306 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-17T04:13:59.306 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-17T04:13:59.306 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-17T04:13:59.306 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-17T04:13:59.306 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-17T04:13:59.306 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-17T04:13:59.307 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-17T04:13:59.307 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-17T04:13:59.307 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-17T04:13:59.307 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-17T04:13:59.307 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-17T04:13:59.307 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-17T04:13:59.307 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-17T04:13:59.308 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-17T04:13:59.308 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-17T04:13:59.308 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-17T04:13:59.308 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-17T04:13:59.308 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-17T04:13:59.308 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-17T04:13:59.308 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-17T04:13:59.308 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-17T04:13:59.308 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-17T04:13:59.308 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-17T04:13:59.309 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-17T04:13:59.309 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-17T04:13:59.309 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-17T04:13:59.309 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-17T04:13:59.309 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-17T04:13:59.309 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-17T04:13:59.309 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-17T04:13:59.309 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-17T04:13:59.309 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-17T04:13:59.309 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-17T04:13:59.309 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-17T04:13:59.309 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-17T04:13:59.309 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-17T04:13:59.310 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-17T04:13:59.310 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-17T04:13:59.310 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-17T04:13:59.310 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-17T04:13:59.310 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-17T04:13:59.310 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-17T04:13:59.310 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-17T04:13:59.310 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-17T04:13:59.310 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-17T04:13:59.310 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-17T04:13:59.310 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-17T04:13:59.310 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-17T04:13:59.310 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-17T04:13:59.311 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-17T04:13:59.311 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-17T04:13:59.311 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-17T04:13:59.311 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-17T04:13:59.311 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-17T04:13:59.311 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-17T04:13:59.311 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-17T04:13:59.311 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-17T04:13:59.311 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-17T04:13:59.311 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-17T04:13:59.311 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-17T04:13:59.311 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-17T04:13:59.312 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-17T04:13:59.312 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-17T04:13:59.312 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-17T04:13:59.312 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-17T04:13:59.312 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-17T04:13:59.312 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-17T04:13:59.312 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-17T04:13:59.312 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-17T04:13:59.312 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-17T04:13:59.312 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-17T04:13:59.312 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-17T04:13:59.312 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-17T04:13:59.313 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-17T04:13:59.313 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-17T04:13:59.313 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-17T04:13:59.313 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-17T04:13:59.313 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-17T04:13:59.313 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-17T04:13:59.313 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-17T04:13:59.313 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-17T04:13:59.313 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-17T04:13:59.313 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-17T04:13:59.313 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-17T04:13:59.313 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-17T04:13:59.313 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-17T04:13:59.314 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-17T04:13:59.314 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-17T04:13:59.314 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-17T04:13:59.314 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-17T04:13:59.314 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-17T04:13:59.314 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-17T04:13:59.314 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-17T04:13:59.314 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-17T04:13:59.314 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-17T04:13:59.314 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-17T04:13:59.314 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-17T04:13:59.314 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-17T04:13:59.314 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-17T04:13:59.314 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-17T04:13:59.315 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-17T04:13:59.315 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-17T04:13:59.315 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-17T04:13:59.315 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-17T04:13:59.315 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-17T04:13:59.315 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-17T04:13:59.315 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-17T04:13:59.315 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-17T04:13:59.315 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-17T04:13:59.315 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-17T04:13:59.315 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-17T04:13:59.315 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-17T04:13:59.315 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-17T04:13:59.315 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-17T04:13:59.316 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-17T04:13:59.316 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-17T04:13:59.316 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-17T04:13:59.316 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-17T04:13:59.316 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-17T04:13:59.316 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-17T04:13:59.316 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-17T04:13:59.316 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-17T04:13:59.316 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-17T04:13:59.316 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-17T04:13:59.316 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-17T04:13:59.316 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-17T04:13:59.317 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-17T04:13:59.317 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-17T04:13:59.317 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-17T04:13:59.317 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-17T04:13:59.317 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-17T04:13:59.317 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-17T04:13:59.317 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-17T04:13:59.317 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-17T04:13:59.317 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-17T04:13:59.317 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-17T04:13:59.317 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-17T04:13:59.317 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-17T04:13:59.318 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-17T04:13:59.318 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-17T04:13:59.318 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-17T04:13:59.318 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-17T04:13:59.318 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-17T04:13:59.318 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-17T04:13:59.318 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-17T04:13:59.318 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-17T04:13:59.318 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-17T04:13:59.318 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-17T04:13:59.318 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-17T04:13:59.318 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-17T04:13:59.318 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-17T04:13:59.319 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-17T04:13:59.319 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-17T04:13:59.319 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-17T04:13:59.319 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-17T04:13:59.319 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-17T04:13:59.319 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-17T04:13:59.319 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-17T04:13:59.319 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-17T04:13:59.319 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-17T04:13:59.319 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-17T04:13:59.319 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-17T04:13:59.319 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-17T04:13:59.319 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-17T04:13:59.319 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-17T04:13:59.320 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-17T04:13:59.320 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-17T04:13:59.320 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-17T04:13:59.320 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-17T04:13:59.320 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-17T04:13:59.320 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-17T04:13:59.320 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-17T04:13:59.320 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-17T04:13:59.321 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-17T04:13:59.321 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-17T04:13:59.321 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-17T04:13:59.321 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-17T04:13:59.321 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-17T04:13:59.321 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-17T04:13:59.321 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-17T04:13:59.321 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-17T04:13:59.321 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-17T04:13:59.322 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-17T04:13:59.322 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-17T04:13:59.322 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-17T04:13:59.322 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-17T04:13:59.322 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-17T04:13:59.322 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-17T04:13:59.322 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-17T04:13:59.322 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-17T04:13:59.322 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-17T04:13:59.322 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-17T04:13:59.322 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-17T04:13:59.322 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-17T04:13:59.322 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-17T04:13:59.322 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-17T04:13:59.323 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-17T04:13:59.323 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-17T04:13:59.323 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-17T04:13:59.323 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-17T04:13:59.323 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-17T04:13:59.323 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-17T04:13:59.323 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-17T04:13:59.323 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-17T04:13:59.323 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-17T04:13:59.323 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-17T04:13:59.323 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-17T04:13:59.323 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-17T04:13:59.324 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-17T04:13:59.324 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-17T04:13:59.324 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-17T04:13:59.324 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-17T04:13:59.324 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-17T04:13:59.324 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-17T04:13:59.324 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-17T04:13:59.324 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-17T04:13:59.324 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-17T04:13:59.324 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-17T04:13:59.324 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-17T04:13:59.324 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-17T04:13:59.325 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-17T04:13:59.325 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-17T04:13:59.325 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-17T04:13:59.325 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-17T04:13:59.325 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-17T04:13:59.325 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-17T04:13:59.325 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-17T04:13:59.325 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-17T04:13:59.325 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-17T04:13:59.325 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-17T04:13:59.325 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-17T04:13:59.325 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-17T04:13:59.325 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-17T04:13:59.325 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-17T04:13:59.326 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-17T04:13:59.326 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-17T04:13:59.326 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-17T04:13:59.326 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-17T04:13:59.326 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-17T04:13:59.326 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-17T04:13:59.326 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-17T04:13:59.326 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-17T04:13:59.326 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-17T04:13:59.326 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-17T04:13:59.326 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-17T04:13:59.327 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-17T04:13:59.327 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-17T04:13:59.327 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-17T04:13:59.327 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-17T04:13:59.327 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-17T04:13:59.327 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-17T04:13:59.327 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-17T04:13:59.327 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-17T04:13:59.327 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-17T04:13:59.327 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-17T04:13:59.327 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-17T04:13:59.327 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-17T04:13:59.328 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-17T04:13:59.328 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-17T04:13:59.328 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-17T04:13:59.328 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-17T04:13:59.328 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-17T04:13:59.328 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-17T04:13:59.328 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-17T04:13:59.328 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-17T04:13:59.328 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-17T04:13:59.328 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-17T04:13:59.328 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-17T04:13:59.328 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-17T04:13:59.328 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-17T04:13:59.328 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-17T04:13:59.329 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-17T04:13:59.329 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-17T04:13:59.329 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-17T04:13:59.329 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-17T04:13:59.329 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-17T04:13:59.329 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-17T04:13:59.330 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-17T04:13:59.330 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-17T04:13:59.330 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-17T04:13:59.330 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-17T04:13:59.330 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-17T04:13:59.330 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-17T04:13:59.331 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-17T04:13:59.331 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-17T04:13:59.331 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-17T04:13:59.331 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-17T04:13:59.331 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-17T04:13:59.331 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-17T04:13:59.332 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-17T04:13:59.332 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-17T04:13:59.332 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-17T04:13:59.332 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-17T04:13:59.332 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-17T04:13:59.332 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-17T04:13:59.332 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-17T04:13:59.333 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-17T04:13:59.333 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-17T04:13:59.333 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-17T04:13:59.333 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-17T04:13:59.333 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-17T04:13:59.333 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-17T04:13:59.334 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-17T04:13:59.334 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-17T04:13:59.334 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-17T04:13:59.334 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-17T04:13:59.334 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-17T04:13:59.334 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-17T04:13:59.335 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-17T04:13:59.335 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-17T04:13:59.335 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-17T04:13:59.335 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-17T04:13:59.335 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-17T04:13:59.335 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-17T04:13:59.335 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-17T04:13:59.336 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-17T04:13:59.336 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-17T04:13:59.336 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-17T04:13:59.336 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-17T04:13:59.336 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-17T04:13:59.337 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-17T04:13:59.337 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-17T04:13:59.337 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-17T04:13:59.337 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-17T04:13:59.337 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-17T04:13:59.337 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-17T04:13:59.338 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-17T04:13:59.338 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-17T04:13:59.338 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-17T04:13:59.338 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-17T04:13:59.338 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-17T04:13:59.338 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-17T04:13:59.339 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-17T04:13:59.339 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-17T04:13:59.339 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-17T04:13:59.339 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-17T04:13:59.339 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-17T04:13:59.339 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-17T04:13:59.339 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-17T04:13:59.339 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-17T04:13:59.340 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-17T04:13:59.340 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-17T04:13:59.340 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-17T04:13:59.340 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-17T04:13:59.340 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-17T04:13:59.340 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-17T04:13:59.340 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-17T04:13:59.340 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-17T04:13:59.340 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-17T04:13:59.340 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-17T04:13:59.340 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-17T04:13:59.340 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-17T04:13:59.341 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-17T04:13:59.341 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-17T04:13:59.341 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-17T04:13:59.341 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-17T04:13:59.341 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-17T04:13:59.341 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-17T04:13:59.341 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-17T04:13:59.341 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-17T04:13:59.341 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-17T04:13:59.341 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-17T04:13:59.341 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-17T04:13:59.341 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-17T04:13:59.341 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-17T04:13:59.342 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-17T04:13:59.342 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-17T04:13:59.342 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-17T04:13:59.342 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-17T04:13:59.342 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-17T04:13:59.342 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-17T04:13:59.342 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-17T04:13:59.342 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-17T04:13:59.342 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-17T04:13:59.342 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-17T04:13:59.342 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-17T04:13:59.342 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-17T04:13:59.342 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-17T04:13:59.343 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-17T04:13:59.343 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-17T04:13:59.343 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-17T04:13:59.343 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-17T04:13:59.343 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-17T04:13:59.343 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-17T04:13:59.343 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-17T04:13:59.343 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-17T04:13:59.343 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-17T04:13:59.343 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-17T04:13:59.343 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-17T04:13:59.343 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-17T04:13:59.343 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-17T04:13:59.343 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-17T04:13:59.344 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-17T04:13:59.344 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-17T04:13:59.344 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-17T04:13:59.344 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-17T04:13:59.344 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-17T04:13:59.344 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-17T04:13:59.344 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-17T04:13:59.344 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-17T04:13:59.344 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-17T04:13:59.344 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-17T04:13:59.344 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-17T04:13:59.344 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-17T04:13:59.344 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-17T04:13:59.345 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-17T04:13:59.345 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-17T04:13:59.345 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-17T04:13:59.345 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-17T04:13:59.345 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-17T04:13:59.345 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-17T04:13:59.345 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-17T04:13:59.345 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-17T04:13:59.345 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-17T04:13:59.345 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-17T04:13:59.345 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-17T04:13:59.345 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-17T04:13:59.345 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-17T04:13:59.345 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-17T04:13:59.346 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-17T04:13:59.346 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-17T04:13:59.346 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-17T04:13:59.346 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-17T04:13:59.346 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-17T04:13:59.346 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-17T04:13:59.346 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-17T04:13:59.346 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-17T04:13:59.346 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-17T04:13:59.346 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-17T04:13:59.346 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-17T04:13:59.346 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-17T04:13:59.346 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-17T04:13:59.347 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-17T04:13:59.347 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-17T04:13:59.347 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-17T04:13:59.347 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-17T04:13:59.347 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-17T04:13:59.347 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-17T04:13:59.347 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-17T04:13:59.347 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-17T04:13:59.347 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-17T04:13:59.347 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-17T04:13:59.347 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-17T04:13:59.348 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-17T04:13:59.348 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-17T04:13:59.348 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-17T04:13:59.348 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-17T04:13:59.348 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-17T04:13:59.348 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-17T04:13:59.348 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-17T04:13:59.348 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-17T04:13:59.348 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-17T04:13:59.348 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-17T04:13:59.348 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-17T04:13:59.348 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-17T04:13:59.349 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-17T04:13:59.349 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-17T04:13:59.349 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-17T04:13:59.349 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-17T04:13:59.349 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-17T04:13:59.349 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-17T04:13:59.349 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-17T04:13:59.349 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-17T04:13:59.349 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-17T04:13:59.349 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-17T04:13:59.349 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-17T04:13:59.349 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-17T04:13:59.349 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-17T04:13:59.350 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-17T04:13:59.350 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-17T04:13:59.350 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-17T04:13:59.350 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-17T04:13:59.350 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-17T04:13:59.350 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-17T04:13:59.350 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-17T04:13:59.350 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-17T04:13:59.350 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-17T04:13:59.350 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-17T04:13:59.350 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-17T04:13:59.350 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-17T04:13:59.350 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-17T04:13:59.351 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-17T04:13:59.351 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-17T04:13:59.351 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-17T04:13:59.351 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-17T04:13:59.351 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-17T04:13:59.351 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-17T04:13:59.351 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-17T04:13:59.351 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-17T04:13:59.351 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-17T04:13:59.351 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-17T04:13:59.351 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-17T04:13:59.352 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-17T04:13:59.352 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-17T04:13:59.352 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-17T04:13:59.352 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-17T04:13:59.352 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-17T04:13:59.352 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-17T04:13:59.352 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-17T04:13:59.352 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-17T04:13:59.352 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-17T04:13:59.352 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-17T04:13:59.352 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-17T04:13:59.352 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-17T04:13:59.353 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-17T04:13:59.353 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-17T04:13:59.353 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-17T04:13:59.353 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-17T04:13:59.353 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-17T04:13:59.353 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-17T04:13:59.353 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-17T04:13:59.353 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-17T04:13:59.353 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-17T04:13:59.353 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-17T04:13:59.353 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-17T04:13:59.353 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-17T04:13:59.354 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-17T04:13:59.354 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-17T04:13:59.354 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-17T04:13:59.354 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-17T04:13:59.354 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-17T04:13:59.354 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-17T04:13:59.354 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-17T04:13:59.354 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-17T04:13:59.354 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-17T04:13:59.354 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-17T04:13:59.354 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-17T04:13:59.354 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-17T04:13:59.354 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-17T04:13:59.354 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-17T04:13:59.355 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-17T04:13:59.355 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-17T04:13:59.355 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-17T04:13:59.355 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-17T04:13:59.355 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-17T04:13:59.355 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-17T04:13:59.355 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-17T04:13:59.355 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-17T04:13:59.355 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-17T04:13:59.355 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-17T04:13:59.355 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-17T04:13:59.355 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-17T04:13:59.355 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-17T04:13:59.355 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-17T04:13:59.356 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-17T04:13:59.356 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-17T04:13:59.356 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-17T04:13:59.356 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-17T04:13:59.356 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-17T04:13:59.356 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-17T04:13:59.356 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-17T04:13:59.356 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-17T04:13:59.356 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-17T04:13:59.356 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-17T04:13:59.356 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-17T04:13:59.356 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-17T04:13:59.356 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-17T04:13:59.357 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-17T04:13:59.357 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-17T04:13:59.357 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-17T04:13:59.357 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-17T04:13:59.357 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-17T04:13:59.357 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-17T04:13:59.357 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-17T04:13:59.357 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-17T04:13:59.357 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-17T04:13:59.357 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-17T04:13:59.357 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-17T04:13:59.357 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-17T04:13:59.357 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-17T04:13:59.358 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-17T04:13:59.358 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-17T04:13:59.358 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-17T04:13:59.358 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-17T04:13:59.358 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-17T04:13:59.358 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-17T04:13:59.358 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-17T04:13:59.358 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-17T04:13:59.358 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-17T04:13:59.358 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-17T04:13:59.358 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-17T04:13:59.358 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-17T04:13:59.358 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-17T04:13:59.358 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-17T04:13:59.359 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-17T04:13:59.359 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-17T04:13:59.359 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-17T04:13:59.359 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-17T04:13:59.359 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-17T04:13:59.359 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-17T04:13:59.359 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-17T04:13:59.359 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-17T04:13:59.359 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-17T04:13:59.359 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-17T04:13:59.359 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-17T04:13:59.360 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-17T04:13:59.360 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-17T04:13:59.360 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-17T04:13:59.360 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-17T04:13:59.360 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-17T04:13:59.360 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-17T04:13:59.360 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-17T04:13:59.360 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-17T04:13:59.360 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-17T04:13:59.360 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-17T04:13:59.360 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-17T04:13:59.360 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-17T04:13:59.360 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-17T04:13:59.361 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-17T04:13:59.361 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-17T04:13:59.361 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-17T04:13:59.361 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-17T04:13:59.361 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-17T04:13:59.361 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-17T04:13:59.361 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-17T04:13:59.361 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-17T04:13:59.361 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-17T04:13:59.361 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-17T04:13:59.361 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-17T04:13:59.361 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-17T04:13:59.361 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-17T04:13:59.362 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-17T04:13:59.362 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-17T04:13:59.362 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-17T04:13:59.362 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-17T04:13:59.362 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-17T04:13:59.362 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-17T04:13:59.362 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-17T04:13:59.362 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-17T04:13:59.362 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-17T04:13:59.362 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-17T04:13:59.362 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-17T04:13:59.362 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-17T04:13:59.363 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-17T04:13:59.363 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-17T04:13:59.363 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-17T04:13:59.363 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-17T04:13:59.363 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-17T04:13:59.363 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-17T04:13:59.363 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-17T04:13:59.363 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-17T04:13:59.363 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-17T04:13:59.363 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-17T04:13:59.363 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-17T04:13:59.363 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-17T04:13:59.364 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-17T04:13:59.364 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-17T04:13:59.364 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-17T04:13:59.364 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-17T04:13:59.364 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-17T04:13:59.364 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-17T04:13:59.364 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-17T04:13:59.364 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-17T04:13:59.364 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-17T04:13:59.364 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-17T04:13:59.364 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-17T04:13:59.364 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-17T04:13:59.365 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-17T04:13:59.365 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-17T04:13:59.365 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-17T04:13:59.365 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-17T04:13:59.365 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-17T04:13:59.365 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-17T04:13:59.365 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-17T04:13:59.366 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-17T04:13:59.366 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-17T04:13:59.366 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-17T04:13:59.366 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-17T04:13:59.366 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-17T04:13:59.366 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-17T04:13:59.366 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-17T04:13:59.366 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-17T04:13:59.366 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-17T04:13:59.367 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-17T04:13:59.367 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-17T04:13:59.367 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-17T04:13:59.367 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-17T04:13:59.367 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-17T04:13:59.367 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-17T04:13:59.367 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-17T04:13:59.367 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-17T04:13:59.368 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-17T04:13:59.368 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-17T04:13:59.368 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-17T04:13:59.368 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-17T04:13:59.368 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-17T04:13:59.368 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-17T04:13:59.368 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-17T04:13:59.368 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-17T04:13:59.369 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-17T04:13:59.369 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-17T04:13:59.369 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-17T04:13:59.369 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-17T04:13:59.369 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-17T04:13:59.369 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-17T04:13:59.369 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-17T04:13:59.369 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-17T04:13:59.369 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-17T04:13:59.370 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-17T04:13:59.370 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-17T04:13:59.370 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-17T04:13:59.370 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-17T04:13:59.370 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-17T04:13:59.370 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-17T04:13:59.370 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-17T04:13:59.370 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-17T04:13:59.371 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-17T04:13:59.371 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-17T04:13:59.371 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-17T04:13:59.371 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-17T04:13:59.371 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-17T04:13:59.371 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-17T04:13:59.371 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-17T04:13:59.371 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-17T04:13:59.371 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-17T04:13:59.372 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-17T04:13:59.372 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-17T04:13:59.372 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-17T04:13:59.372 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-17T04:13:59.372 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-17T04:13:59.372 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-17T04:13:59.372 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-17T04:13:59.372 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-17T04:13:59.373 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-17T04:13:59.373 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-17T04:13:59.373 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-17T04:13:59.373 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-17T04:13:59.373 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-17T04:13:59.373 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-17T04:13:59.373 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-17T04:13:59.373 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-17T04:13:59.374 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-17T04:13:59.374 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-17T04:13:59.374 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-17T04:13:59.374 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-17T04:13:59.374 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-17T04:13:59.374 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-17T04:13:59.374 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-17T04:13:59.375 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-17T04:13:59.375 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-17T04:13:59.375 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-17T04:13:59.375 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-17T04:13:59.375 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-17T04:13:59.375 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-17T04:13:59.375 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-17T04:13:59.375 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-17T04:13:59.376 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-17T04:13:59.376 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-17T04:13:59.376 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-17T04:13:59.376 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-17T04:13:59.376 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-17T04:13:59.376 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-17T04:13:59.376 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-17T04:13:59.376 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-17T04:13:59.377 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-17T04:13:59.377 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-17T04:13:59.377 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-17T04:13:59.377 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-17T04:13:59.377 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-17T04:13:59.377 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-17T04:13:59.377 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-17T04:13:59.377 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-17T04:13:59.378 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-17T04:13:59.378 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-17T04:13:59.378 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-17T04:13:59.378 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-17T04:13:59.378 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-17T04:13:59.378 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-17T04:13:59.378 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-17T04:13:59.379 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-17T04:13:59.379 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-17T04:13:59.379 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-17T04:13:59.379 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-17T04:13:59.379 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-17T04:13:59.379 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-17T04:13:59.379 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-17T04:13:59.380 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-17T04:13:59.380 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-17T04:13:59.380 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-17T04:13:59.380 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-17T04:13:59.380 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-17T04:13:59.380 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-17T04:13:59.380 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-17T04:13:59.380 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-17T04:13:59.380 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-17T04:13:59.380 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-17T04:13:59.380 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-17T04:13:59.380 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-17T04:13:59.380 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-17T04:13:59.381 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-17T04:13:59.381 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-17T04:13:59.381 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-17T04:13:59.381 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-17T04:13:59.381 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-17T04:13:59.381 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-17T04:13:59.381 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-17T04:13:59.381 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-17T04:13:59.381 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-17T04:13:59.381 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-17T04:13:59.381 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-17T04:13:59.381 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-17T04:13:59.382 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-17T04:13:59.382 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-17T04:13:59.382 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-17T04:13:59.382 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-17T04:13:59.382 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-17T04:13:59.382 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-17T04:13:59.382 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-17T04:13:59.382 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-17T04:13:59.382 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-17T04:13:59.382 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-17T04:13:59.382 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-17T04:13:59.382 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-17T04:13:59.382 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-17T04:13:59.383 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-17T04:13:59.383 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-17T04:13:59.383 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-17T04:13:59.383 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-17T04:13:59.383 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-17T04:13:59.383 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-17T04:13:59.383 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-17T04:13:59.383 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-17T04:13:59.383 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-17T04:13:59.383 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-17T04:13:59.383 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-17T04:13:59.384 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-17T04:13:59.384 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-17T04:13:59.384 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-17T04:13:59.384 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-17T04:13:59.384 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-17T04:13:59.384 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-17T04:13:59.384 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-17T04:13:59.384 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-17T04:13:59.384 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-17T04:13:59.384 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-17T04:13:59.384 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-17T04:13:59.384 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-17T04:13:59.384 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-17T04:13:59.385 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-17T04:13:59.385 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-17T04:13:59.385 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-17T04:13:59.385 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-17T04:13:59.385 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-17T04:13:59.385 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-17T04:13:59.385 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-17T04:13:59.385 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-17T04:13:59.385 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-17T04:13:59.385 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-17T04:13:59.385 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-17T04:13:59.385 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-17T04:13:59.385 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-17T04:13:59.385 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-17T04:13:59.386 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-17T04:13:59.386 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-17T04:13:59.386 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-17T04:13:59.386 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-17T04:13:59.386 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-17T04:13:59.386 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-17T04:13:59.386 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-17T04:13:59.386 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-17T04:13:59.386 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-17T04:13:59.386 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-17T04:13:59.386 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-17T04:13:59.386 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-17T04:13:59.386 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-17T04:13:59.386 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-17T04:13:59.387 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-17T04:13:59.387 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-17T04:13:59.387 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-17T04:13:59.387 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-17T04:13:59.387 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-17T04:13:59.387 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-17T04:13:59.387 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-17T04:13:59.387 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-17T04:13:59.387 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-17T04:13:59.387 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-17T04:13:59.387 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-17T04:13:59.387 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-17T04:13:59.387 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-17T04:13:59.388 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-17T04:13:59.388 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-17T04:13:59.388 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-17T04:13:59.388 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-17T04:13:59.388 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-17T04:13:59.388 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-17T04:13:59.388 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-17T04:13:59.388 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-17T04:13:59.388 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-17T04:13:59.388 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-17T04:13:59.388 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-17T04:13:59.388 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-17T04:13:59.388 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-17T04:13:59.388 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-17T04:13:59.389 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-17T04:13:59.389 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-17T04:13:59.389 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-17T04:13:59.389 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-17T04:13:59.389 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-17T04:13:59.389 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-17T04:13:59.389 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-17T04:13:59.389 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-17T04:13:59.389 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-17T04:13:59.389 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-17T04:13:59.389 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-17T04:13:59.389 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-17T04:13:59.389 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-17T04:13:59.390 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-17T04:13:59.390 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-17T04:13:59.390 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-17T04:13:59.390 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-17T04:13:59.390 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-17T04:13:59.390 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-17T04:13:59.390 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-17T04:13:59.390 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-17T04:13:59.390 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-17T04:13:59.390 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-17T04:13:59.390 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-17T04:13:59.390 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-17T04:13:59.390 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-17T04:13:59.390 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-17T04:13:59.391 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-17T04:13:59.391 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-17T04:13:59.391 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-17T04:13:59.391 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-17T04:13:59.391 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-17T04:13:59.391 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-17T04:13:59.391 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-17T04:13:59.391 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-17T04:13:59.391 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-17T04:13:59.391 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-17T04:13:59.391 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-17T04:13:59.391 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-17T04:13:59.391 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-17T04:13:59.391 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-17T04:13:59.392 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-17T04:13:59.392 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-17T04:13:59.392 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-17T04:13:59.392 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-17T04:13:59.392 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-17T04:13:59.392 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-17T04:13:59.392 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-17T04:13:59.392 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-17T04:13:59.392 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-17T04:13:59.392 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-17T04:13:59.392 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-17T04:13:59.392 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-17T04:13:59.392 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-17T04:13:59.393 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-17T04:13:59.393 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-17T04:13:59.393 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-17T04:13:59.393 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-17T04:13:59.393 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-17T04:13:59.393 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-17T04:13:59.393 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-17T04:13:59.393 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-17T04:13:59.393 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-17T04:13:59.393 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-17T04:13:59.393 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-17T04:13:59.393 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-17T04:13:59.393 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-17T04:13:59.393 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-17T04:13:59.394 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-17T04:13:59.394 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-17T04:13:59.394 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-17T04:13:59.394 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-17T04:13:59.394 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-17T04:13:59.394 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-17T04:13:59.394 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-17T04:13:59.394 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-17T04:13:59.394 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-17T04:13:59.394 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-17T04:13:59.394 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-17T04:13:59.394 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-17T04:13:59.394 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-17T04:13:59.395 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-17T04:13:59.395 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-17T04:13:59.395 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-17T04:13:59.395 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-17T04:13:59.395 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-17T04:13:59.395 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-17T04:13:59.395 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-17T04:13:59.395 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-17T04:13:59.395 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-17T04:13:59.395 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-17T04:13:59.395 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-17T04:13:59.395 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-17T04:13:59.395 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-17T04:13:59.396 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-17T04:13:59.396 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-17T04:13:59.396 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-17T04:13:59.396 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-17T04:13:59.396 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-17T04:13:59.396 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-17T04:13:59.396 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-17T04:13:59.396 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-17T04:13:59.396 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-17T04:13:59.396 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-17T04:13:59.396 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-17T04:13:59.396 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-17T04:13:59.396 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-17T04:13:59.396 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-17T04:13:59.397 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-17T04:13:59.397 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-17T04:13:59.397 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-17T04:13:59.397 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-17T04:13:59.397 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-17T04:13:59.397 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-17T04:13:59.397 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-17T04:13:59.397 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-17T04:13:59.397 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-17T04:13:59.397 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-17T04:13:59.397 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-17T04:13:59.397 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-17T04:13:59.397 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-17T04:13:59.398 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-17T04:13:59.398 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-17T04:13:59.398 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-17T04:13:59.398 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-17T04:13:59.398 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-17T04:13:59.398 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-17T04:13:59.398 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-17T04:13:59.398 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-17T04:13:59.398 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-17T04:13:59.398 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-17T04:13:59.398 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-17T04:13:59.398 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-17T04:13:59.398 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-17T04:13:59.399 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-17T04:13:59.399 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-17T04:13:59.399 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-17T04:13:59.399 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-17T04:13:59.399 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-17T04:13:59.399 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-17T04:13:59.399 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-17T04:13:59.399 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-17T04:13:59.399 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-17T04:13:59.399 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-17T04:13:59.399 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-17T04:13:59.399 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-17T04:13:59.399 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-17T04:13:59.399 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-17T04:13:59.400 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-17T04:13:59.400 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-17T04:13:59.400 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-17T04:13:59.400 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-17T04:13:59.400 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-17T04:13:59.400 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-17T04:13:59.400 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-17T04:13:59.400 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-17T04:13:59.401 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-17T04:13:59.401 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-17T04:13:59.401 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-17T04:13:59.401 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-17T04:13:59.401 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-17T04:13:59.402 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-17T04:13:59.402 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-17T04:13:59.402 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-17T04:13:59.402 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-17T04:13:59.402 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-17T04:13:59.402 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-17T04:13:59.402 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-17T04:13:59.402 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-17T04:13:59.402 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-17T04:13:59.402 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-17T04:13:59.403 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-17T04:13:59.403 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-17T04:13:59.403 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-17T04:13:59.403 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-17T04:13:59.403 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-17T04:13:59.403 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-17T04:13:59.403 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-17T04:13:59.403 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-17T04:13:59.403 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-17T04:13:59.403 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-17T04:13:59.403 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-17T04:13:59.404 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-17T04:13:59.404 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-17T04:13:59.404 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-17T04:13:59.404 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-17T04:13:59.404 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-17T04:13:59.404 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-17T04:13:59.404 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-17T04:13:59.404 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-17T04:13:59.404 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-17T04:13:59.404 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-17T04:13:59.404 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-17T04:13:59.404 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-17T04:13:59.405 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-17T04:13:59.405 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-17T04:13:59.405 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-17T04:13:59.405 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-17T04:13:59.405 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-17T04:13:59.405 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-17T04:13:59.405 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-17T04:13:59.405 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-17T04:13:59.405 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-17T04:13:59.405 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-17T04:13:59.405 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-17T04:13:59.405 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-17T04:13:59.405 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-17T04:13:59.406 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-17T04:13:59.406 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-17T04:13:59.406 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-17T04:13:59.406 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-17T04:13:59.406 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-17T04:13:59.406 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-17T04:13:59.406 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-17T04:13:59.406 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-17T04:13:59.406 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-17T04:13:59.406 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-17T04:13:59.406 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-17T04:13:59.406 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-17T04:13:59.406 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-17T04:13:59.406 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-17T04:13:59.407 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-17T04:13:59.407 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-17T04:13:59.407 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-17T04:13:59.407 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-17T04:13:59.407 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-17T04:13:59.407 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-17T04:13:59.407 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-17T04:13:59.407 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-17T04:13:59.407 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-17T04:13:59.407 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-17T04:13:59.407 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-17T04:13:59.407 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-17T04:13:59.408 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-17T04:13:59.408 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-17T04:13:59.408 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-17T04:13:59.408 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-17T04:13:59.408 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-17T04:13:59.408 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-17T04:13:59.408 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-17T04:13:59.408 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-17T04:13:59.408 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-17T04:13:59.408 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-17T04:13:59.408 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-17T04:13:59.408 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-17T04:13:59.408 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-17T04:13:59.408 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-17T04:13:59.409 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-17T04:13:59.409 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-17T04:13:59.409 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-17T04:13:59.409 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-17T04:13:59.409 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-17T04:13:59.409 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-17T04:13:59.409 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-17T04:13:59.409 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-17T04:13:59.409 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-17T04:13:59.409 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-17T04:13:59.409 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-17T04:13:59.409 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-17T04:13:59.409 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-17T04:13:59.409 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-17T04:13:59.410 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-17T04:13:59.410 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-17T04:13:59.410 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-17T04:13:59.410 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-17T04:13:59.410 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-17T04:13:59.410 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-17T04:13:59.410 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-17T04:13:59.410 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-17T04:13:59.410 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-17T04:13:59.410 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-17T04:13:59.410 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-17T04:13:59.410 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-17T04:13:59.410 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-17T04:13:59.410 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-17T04:13:59.410 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-17T04:13:59.411 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-17T04:13:59.411 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-17T04:13:59.411 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-17T04:13:59.411 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-17T04:13:59.411 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-17T04:13:59.411 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-17T04:13:59.411 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-17T04:13:59.411 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-17T04:13:59.411 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-17T04:13:59.411 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-17T04:13:59.411 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-17T04:13:59.411 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-17T04:13:59.411 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-17T04:13:59.412 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-17T04:13:59.412 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-17T04:13:59.412 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-17T04:13:59.412 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-17T04:13:59.412 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-17T04:13:59.412 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-17T04:13:59.412 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-17T04:13:59.412 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-17T04:13:59.412 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-17T04:13:59.412 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-17T04:13:59.412 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-17T04:13:59.412 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-17T04:13:59.412 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-17T04:13:59.413 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-17T04:13:59.413 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-17T04:13:59.413 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-17T04:13:59.413 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-17T04:13:59.413 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-17T04:13:59.413 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-17T04:13:59.413 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-17T04:13:59.413 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-17T04:13:59.413 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-17T04:13:59.413 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-17T04:13:59.413 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-17T04:13:59.413 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-17T04:13:59.413 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-17T04:13:59.413 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-17T04:13:59.414 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-17T04:13:59.414 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-17T04:13:59.414 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-17T04:13:59.414 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-17T04:13:59.414 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-17T04:13:59.414 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-17T04:13:59.414 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-17T04:13:59.414 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-17T04:13:59.414 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-17T04:13:59.414 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-17T04:13:59.414 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-17T04:13:59.414 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-17T04:13:59.414 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-17T04:13:59.414 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-17T04:13:59.415 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-17T04:13:59.415 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-17T04:13:59.415 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-17T04:13:59.415 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-17T04:13:59.415 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-17T04:13:59.415 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-17T04:13:59.415 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-17T04:13:59.415 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-17T04:13:59.415 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-17T04:13:59.415 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-17T04:13:59.415 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-17T04:13:59.415 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-17T04:13:59.415 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-17T04:13:59.416 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-17T04:13:59.416 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-17T04:13:59.416 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-17T04:13:59.416 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-17T04:13:59.416 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-17T04:13:59.416 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-17T04:13:59.416 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-17T04:13:59.416 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-17T04:13:59.416 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-17T04:13:59.416 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-17T04:13:59.416 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-17T04:13:59.416 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-17T04:13:59.417 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-17T04:13:59.417 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-17T04:13:59.417 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-17T04:13:59.417 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-17T04:13:59.417 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-17T04:13:59.417 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-17T04:13:59.417 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-17T04:13:59.417 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-17T04:13:59.417 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-17T04:13:59.417 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-17T04:13:59.417 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-17T04:13:59.417 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-17T04:13:59.418 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-17T04:13:59.418 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-17T04:13:59.418 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-17T04:13:59.418 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-17T04:13:59.418 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-17T04:13:59.418 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-17T04:13:59.418 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-17T04:13:59.418 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-17T04:13:59.418 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-17T04:13:59.418 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-17T04:13:59.418 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-17T04:13:59.418 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-17T04:13:59.418 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-17T04:13:59.419 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-17T04:13:59.419 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-17T04:13:59.419 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-17T04:13:59.419 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-17T04:13:59.419 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-17T04:13:59.419 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-17T04:13:59.419 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-17T04:13:59.419 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-17T04:13:59.419 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-17T04:13:59.419 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-17T04:13:59.419 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-17T04:13:59.419 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-17T04:13:59.419 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-17T04:13:59.419 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-17T04:13:59.420 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-17T04:13:59.420 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-17T04:13:59.420 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-17T04:13:59.420 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-17T04:13:59.420 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-17T04:13:59.420 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-17T04:13:59.420 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-17T04:13:59.420 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-17T04:13:59.420 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-17T04:13:59.420 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-17T04:13:59.420 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-17T04:13:59.420 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-17T04:13:59.420 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-17T04:13:59.421 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-17T04:13:59.421 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-17T04:13:59.421 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-17T04:13:59.421 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-17T04:13:59.421 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-17T04:13:59.421 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-17T04:13:59.421 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-17T04:13:59.421 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-17T04:13:59.421 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-17T04:13:59.421 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-17T04:13:59.421 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-17T04:13:59.422 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-17T04:13:59.422 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-17T04:13:59.422 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-17T04:13:59.426 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:13:59.426 DEBUG:teuthology.orchestra.run.smithi184:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-17T04:13:59.449 INFO:teuthology.orchestra.run.smithi184.stdout:postinstall scriptlet (using /bin/sh): 2024-09-17T04:13:59.450 INFO:teuthology.orchestra.run.smithi184.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-17T04:13:59.450 INFO:teuthology.orchestra.run.smithi184.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-17T04:13:59.450 INFO:teuthology.orchestra.run.smithi184.stdout:fi 2024-09-17T04:13:59.450 INFO:teuthology.orchestra.run.smithi184.stdout:for file in vmlinuz System.map config; do 2024-09-17T04:13:59.450 INFO:teuthology.orchestra.run.smithi184.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-17T04:13:59.450 INFO:teuthology.orchestra.run.smithi184.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-17T04:13:59.450 INFO:teuthology.orchestra.run.smithi184.stdout: fi 2024-09-17T04:13:59.450 INFO:teuthology.orchestra.run.smithi184.stdout:done 2024-09-17T04:13:59.450 INFO:teuthology.orchestra.run.smithi184.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-17T04:13:59.450 INFO:teuthology.orchestra.run.smithi184.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-17T04:13:59.450 INFO:teuthology.orchestra.run.smithi184.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-17T04:13:59.450 INFO:teuthology.orchestra.run.smithi184.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-17T04:13:59.450 INFO:teuthology.orchestra.run.smithi184.stdout:fi 2024-09-17T04:13:59.450 INFO:teuthology.orchestra.run.smithi184.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-17T04:13:59.451 INFO:teuthology.orchestra.run.smithi184.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:13:59.451 INFO:teuthology.orchestra.run.smithi184.stdout:fi 2024-09-17T04:13:59.451 INFO:teuthology.orchestra.run.smithi184.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-17T04:13:59.451 INFO:teuthology.orchestra.run.smithi184.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-17T04:13:59.451 INFO:teuthology.orchestra.run.smithi184.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:13:59.451 INFO:teuthology.orchestra.run.smithi184.stdout:fi 2024-09-17T04:13:59.452 DEBUG:teuthology.orchestra.run.smithi184:> sudo rpm -qi grub2-tools 2024-09-17T04:13:59.526 INFO:teuthology.orchestra.run.smithi184.stdout:Name : grub2-tools 2024-09-17T04:13:59.526 INFO:teuthology.orchestra.run.smithi184.stdout:Epoch : 1 2024-09-17T04:13:59.526 INFO:teuthology.orchestra.run.smithi184.stdout:Version : 2.06 2024-09-17T04:13:59.526 INFO:teuthology.orchestra.run.smithi184.stdout:Release : 82.el9 2024-09-17T04:13:59.526 INFO:teuthology.orchestra.run.smithi184.stdout:Architecture: x86_64 2024-09-17T04:13:59.526 INFO:teuthology.orchestra.run.smithi184.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T04:13:59.526 INFO:teuthology.orchestra.run.smithi184.stdout:Group : Unspecified 2024-09-17T04:13:59.526 INFO:teuthology.orchestra.run.smithi184.stdout:Size : 8274670 2024-09-17T04:13:59.526 INFO:teuthology.orchestra.run.smithi184.stdout:License : GPLv3+ 2024-09-17T04:13:59.526 INFO:teuthology.orchestra.run.smithi184.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T04:13:59.526 INFO:teuthology.orchestra.run.smithi184.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T04:13:59.527 INFO:teuthology.orchestra.run.smithi184.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T04:13:59.527 INFO:teuthology.orchestra.run.smithi184.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T04:13:59.527 INFO:teuthology.orchestra.run.smithi184.stdout:Packager : builder@centos.org 2024-09-17T04:13:59.527 INFO:teuthology.orchestra.run.smithi184.stdout:Vendor : CentOS 2024-09-17T04:13:59.527 INFO:teuthology.orchestra.run.smithi184.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T04:13:59.527 INFO:teuthology.orchestra.run.smithi184.stdout:Summary : Support tools for GRUB. 2024-09-17T04:13:59.527 INFO:teuthology.orchestra.run.smithi184.stdout:Description : 2024-09-17T04:13:59.527 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:13:59.527 INFO:teuthology.orchestra.run.smithi184.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T04:13:59.527 INFO:teuthology.orchestra.run.smithi184.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T04:13:59.527 INFO:teuthology.orchestra.run.smithi184.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T04:13:59.528 INFO:teuthology.orchestra.run.smithi184.stdout:hardware devices. 2024-09-17T04:13:59.528 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:13:59.528 INFO:teuthology.orchestra.run.smithi184.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T04:13:59.529 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T04:13:59.529 INFO:teuthology.task.kernel:Updating grub on smithi184 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:13:59.529 DEBUG:teuthology.orchestra.run.smithi184:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T04:14:00.129 INFO:teuthology.orchestra.run.smithi184.stderr:Generating grub configuration file ... 2024-09-17T04:14:01.539 INFO:teuthology.orchestra.run.smithi184.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T04:14:01.569 INFO:teuthology.orchestra.run.smithi184.stderr:done 2024-09-17T04:14:01.571 DEBUG:teuthology.orchestra.run.smithi184:> mktemp 2024-09-17T04:14:01.586 INFO:teuthology.orchestra.run.smithi184.stdout:/tmp/tmp.fiPNkJTLsB 2024-09-17T04:14:01.587 DEBUG:teuthology.orchestra.run.smithi184:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.fiPNkJTLsB 2024-09-17T04:14:01.655 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0666 /tmp/tmp.fiPNkJTLsB 2024-09-17T04:14:01.791 DEBUG:teuthology.orchestra.remote:smithi184:/tmp/tmp.fiPNkJTLsB is 6KB 2024-09-17T04:14:01.840 DEBUG:teuthology.orchestra.run.smithi184:> rm -fr /tmp/tmp.fiPNkJTLsB 2024-09-17T04:14:01.856 DEBUG:teuthology.orchestra.run.smithi184:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T04:14:01.924 INFO:teuthology.orchestra.run.smithi184.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T04:14:01.924 INFO:teuthology.orchestra.run.smithi184.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T04:14:01.925 INFO:teuthology.orchestra.run.smithi184.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T04:14:01.925 INFO:teuthology.orchestra.run.smithi184.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-17T04:14:01.926 DEBUG:teuthology.orchestra.run.smithi184:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:14:02.087 DEBUG:teuthology.orchestra.run.smithi184:> sudo shutdown -r now 2024-09-17T04:14:19.174 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi100.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-17T04:14:19.175 DEBUG:teuthology.orchestra.run.smithi100:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-17T04:14:19.261 INFO:teuthology.orchestra.run.smithi100.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:14:19.262 INFO:teuthology.orchestra.run.smithi100.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:14:19.262 INFO:teuthology.orchestra.run.smithi100.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:14:19.262 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:14:19.262 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-17T04:14:19.262 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-17T04:14:19.262 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-17T04:14:19.262 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-17T04:14:19.262 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-17T04:14:19.262 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-17T04:14:19.262 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-17T04:14:19.262 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-17T04:14:19.262 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-17T04:14:19.263 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-17T04:14:19.263 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-17T04:14:19.263 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-17T04:14:19.263 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-17T04:14:19.263 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-17T04:14:19.263 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-17T04:14:19.263 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-17T04:14:19.263 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-17T04:14:19.263 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-17T04:14:19.263 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-17T04:14:19.263 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-17T04:14:19.263 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-17T04:14:19.263 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-17T04:14:19.264 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-17T04:14:19.264 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-17T04:14:19.264 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-17T04:14:19.264 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-17T04:14:19.264 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-17T04:14:19.264 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-17T04:14:19.264 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-17T04:14:19.264 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-17T04:14:19.264 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-17T04:14:19.264 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-17T04:14:19.264 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-17T04:14:19.264 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-17T04:14:19.264 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-17T04:14:19.265 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-17T04:14:19.265 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-17T04:14:19.265 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-17T04:14:19.265 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-17T04:14:19.265 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-17T04:14:19.265 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-17T04:14:19.267 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-17T04:14:19.267 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-17T04:14:19.267 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-17T04:14:19.268 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-17T04:14:19.268 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-17T04:14:19.268 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-17T04:14:19.268 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-17T04:14:19.268 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-17T04:14:19.268 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-17T04:14:19.268 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-17T04:14:19.268 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-17T04:14:19.268 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-17T04:14:19.268 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-17T04:14:19.268 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-17T04:14:19.268 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-17T04:14:19.268 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-17T04:14:19.268 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-17T04:14:19.269 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-17T04:14:19.269 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-17T04:14:19.269 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-17T04:14:19.269 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-17T04:14:19.269 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-17T04:14:19.269 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-17T04:14:19.269 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-17T04:14:19.269 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-17T04:14:19.269 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-17T04:14:19.269 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-17T04:14:19.269 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-17T04:14:19.269 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-17T04:14:19.270 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-17T04:14:19.270 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-17T04:14:19.270 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-17T04:14:19.270 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-17T04:14:19.270 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-17T04:14:19.270 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-17T04:14:19.270 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-17T04:14:19.270 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-17T04:14:19.270 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-17T04:14:19.270 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-17T04:14:19.270 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-17T04:14:19.270 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-17T04:14:19.270 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-17T04:14:19.271 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-17T04:14:19.271 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-17T04:14:19.271 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-17T04:14:19.271 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-17T04:14:19.271 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-17T04:14:19.271 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-17T04:14:19.271 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-17T04:14:19.271 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-17T04:14:19.271 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-17T04:14:19.271 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-17T04:14:19.271 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-17T04:14:19.271 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-17T04:14:19.271 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-17T04:14:19.272 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-17T04:14:19.272 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-17T04:14:19.272 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-17T04:14:19.272 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-17T04:14:19.272 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-17T04:14:19.272 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-17T04:14:19.272 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-17T04:14:19.272 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-17T04:14:19.272 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-17T04:14:19.272 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-17T04:14:19.272 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-17T04:14:19.272 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-17T04:14:19.272 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-17T04:14:19.272 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-17T04:14:19.273 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-17T04:14:19.273 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-17T04:14:19.273 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-17T04:14:19.273 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-17T04:14:19.273 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-17T04:14:19.273 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-17T04:14:19.273 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-17T04:14:19.273 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-17T04:14:19.273 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-17T04:14:19.273 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-17T04:14:19.273 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-17T04:14:19.273 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-17T04:14:19.273 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-17T04:14:19.274 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-17T04:14:19.274 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-17T04:14:19.274 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-17T04:14:19.274 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-17T04:14:19.274 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-17T04:14:19.274 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-17T04:14:19.274 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-17T04:14:19.274 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-17T04:14:19.274 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-17T04:14:19.274 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-17T04:14:19.274 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-17T04:14:19.274 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-17T04:14:19.274 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-17T04:14:19.275 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-17T04:14:19.275 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-17T04:14:19.275 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-17T04:14:19.275 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-17T04:14:19.275 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-17T04:14:19.275 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-17T04:14:19.275 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-17T04:14:19.275 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-17T04:14:19.275 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-17T04:14:19.275 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-17T04:14:19.275 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-17T04:14:19.275 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-17T04:14:19.275 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-17T04:14:19.275 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-17T04:14:19.276 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-17T04:14:19.276 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-17T04:14:19.276 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-17T04:14:19.276 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-17T04:14:19.276 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-17T04:14:19.276 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-17T04:14:19.276 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-17T04:14:19.276 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-17T04:14:19.276 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-17T04:14:19.276 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-17T04:14:19.276 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-17T04:14:19.276 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-17T04:14:19.276 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-17T04:14:19.277 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-17T04:14:19.277 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-17T04:14:19.277 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-17T04:14:19.277 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-17T04:14:19.277 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-17T04:14:19.277 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-17T04:14:19.277 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-17T04:14:19.277 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-17T04:14:19.277 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-17T04:14:19.277 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-17T04:14:19.278 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-17T04:14:19.278 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-17T04:14:19.278 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-17T04:14:19.278 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-17T04:14:19.278 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-17T04:14:19.278 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-17T04:14:19.278 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-17T04:14:19.278 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-17T04:14:19.278 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-17T04:14:19.278 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-17T04:14:19.278 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-17T04:14:19.278 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-17T04:14:19.278 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-17T04:14:19.279 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-17T04:14:19.279 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-17T04:14:19.279 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-17T04:14:19.279 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-17T04:14:19.279 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-17T04:14:19.279 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-17T04:14:19.279 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-17T04:14:19.279 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-17T04:14:19.279 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-17T04:14:19.279 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-17T04:14:19.279 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-17T04:14:19.279 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-17T04:14:19.279 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-17T04:14:19.279 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-17T04:14:19.280 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-17T04:14:19.280 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-17T04:14:19.280 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-17T04:14:19.280 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-17T04:14:19.280 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-17T04:14:19.280 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-17T04:14:19.280 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-17T04:14:19.280 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-17T04:14:19.280 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-17T04:14:19.280 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-17T04:14:19.280 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-17T04:14:19.281 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-17T04:14:19.281 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-17T04:14:19.281 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-17T04:14:19.281 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-17T04:14:19.281 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-17T04:14:19.281 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-17T04:14:19.281 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-17T04:14:19.281 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-17T04:14:19.281 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-17T04:14:19.281 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-17T04:14:19.281 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-17T04:14:19.281 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-17T04:14:19.281 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-17T04:14:19.282 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-17T04:14:19.282 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-17T04:14:19.282 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-17T04:14:19.282 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-17T04:14:19.282 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-17T04:14:19.282 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-17T04:14:19.282 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-17T04:14:19.282 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-17T04:14:19.282 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-17T04:14:19.282 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-17T04:14:19.282 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-17T04:14:19.282 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-17T04:14:19.282 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-17T04:14:19.283 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-17T04:14:19.283 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-17T04:14:19.283 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-17T04:14:19.283 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-17T04:14:19.283 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-17T04:14:19.283 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-17T04:14:19.283 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-17T04:14:19.283 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-17T04:14:19.283 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-17T04:14:19.283 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-17T04:14:19.283 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-17T04:14:19.283 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-17T04:14:19.283 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-17T04:14:19.283 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-17T04:14:19.284 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-17T04:14:19.284 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-17T04:14:19.284 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-17T04:14:19.284 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-17T04:14:19.284 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-17T04:14:19.284 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-17T04:14:19.284 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-17T04:14:19.284 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-17T04:14:19.284 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-17T04:14:19.285 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-17T04:14:19.285 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-17T04:14:19.285 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-17T04:14:19.285 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-17T04:14:19.285 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-17T04:14:19.285 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-17T04:14:19.285 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-17T04:14:19.285 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-17T04:14:19.285 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-17T04:14:19.285 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-17T04:14:19.285 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-17T04:14:19.285 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-17T04:14:19.285 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-17T04:14:19.286 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-17T04:14:19.286 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-17T04:14:19.286 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-17T04:14:19.286 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-17T04:14:19.286 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-17T04:14:19.286 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-17T04:14:19.286 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-17T04:14:19.286 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-17T04:14:19.286 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-17T04:14:19.286 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-17T04:14:19.286 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-17T04:14:19.286 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-17T04:14:19.286 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-17T04:14:19.286 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-17T04:14:19.287 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-17T04:14:19.287 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-17T04:14:19.287 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-17T04:14:19.287 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-17T04:14:19.287 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-17T04:14:19.287 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-17T04:14:19.287 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-17T04:14:19.287 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-17T04:14:19.287 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-17T04:14:19.287 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-17T04:14:19.287 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-17T04:14:19.287 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-17T04:14:19.287 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-17T04:14:19.287 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-17T04:14:19.288 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-17T04:14:19.288 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-17T04:14:19.288 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-17T04:14:19.288 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-17T04:14:19.288 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-17T04:14:19.288 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-17T04:14:19.288 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-17T04:14:19.288 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-17T04:14:19.288 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-17T04:14:19.288 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-17T04:14:19.288 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-17T04:14:19.288 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-17T04:14:19.288 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-17T04:14:19.289 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-17T04:14:19.289 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-17T04:14:19.289 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-17T04:14:19.289 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-17T04:14:19.289 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-17T04:14:19.289 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-17T04:14:19.289 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-17T04:14:19.289 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-17T04:14:19.289 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-17T04:14:19.289 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-17T04:14:19.289 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-17T04:14:19.289 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-17T04:14:19.289 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-17T04:14:19.289 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-17T04:14:19.290 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-17T04:14:19.290 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-17T04:14:19.290 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-17T04:14:19.290 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-17T04:14:19.290 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-17T04:14:19.290 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-17T04:14:19.290 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-17T04:14:19.290 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-17T04:14:19.290 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-17T04:14:19.290 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-17T04:14:19.290 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-17T04:14:19.290 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-17T04:14:19.290 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-17T04:14:19.291 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-17T04:14:19.291 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-17T04:14:19.291 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-17T04:14:19.291 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-17T04:14:19.291 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-17T04:14:19.291 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-17T04:14:19.291 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-17T04:14:19.291 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-17T04:14:19.291 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-17T04:14:19.291 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-17T04:14:19.291 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-17T04:14:19.291 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-17T04:14:19.291 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-17T04:14:19.292 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-17T04:14:19.292 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-17T04:14:19.292 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-17T04:14:19.292 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-17T04:14:19.292 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-17T04:14:19.292 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-17T04:14:19.292 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-17T04:14:19.292 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-17T04:14:19.292 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-17T04:14:19.292 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-17T04:14:19.292 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-17T04:14:19.292 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-17T04:14:19.292 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-17T04:14:19.292 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-17T04:14:19.293 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-17T04:14:19.293 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-17T04:14:19.293 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-17T04:14:19.293 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-17T04:14:19.293 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-17T04:14:19.293 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-17T04:14:19.293 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-17T04:14:19.293 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-17T04:14:19.293 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-17T04:14:19.293 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-17T04:14:19.293 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-17T04:14:19.293 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-17T04:14:19.293 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-17T04:14:19.294 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-17T04:14:19.294 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-17T04:14:19.294 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-17T04:14:19.294 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-17T04:14:19.294 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-17T04:14:19.294 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-17T04:14:19.294 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-17T04:14:19.294 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-17T04:14:19.294 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-17T04:14:19.294 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-17T04:14:19.294 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-17T04:14:19.294 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-17T04:14:19.294 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-17T04:14:19.294 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-17T04:14:19.295 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-17T04:14:19.295 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-17T04:14:19.295 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-17T04:14:19.295 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-17T04:14:19.295 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-17T04:14:19.295 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-17T04:14:19.295 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-17T04:14:19.295 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-17T04:14:19.295 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-17T04:14:19.295 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-17T04:14:19.295 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-17T04:14:19.295 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-17T04:14:19.295 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-17T04:14:19.296 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-17T04:14:19.296 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-17T04:14:19.296 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-17T04:14:19.296 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-17T04:14:19.296 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-17T04:14:19.296 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-17T04:14:19.296 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-17T04:14:19.296 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-17T04:14:19.296 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-17T04:14:19.296 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-17T04:14:19.296 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-17T04:14:19.296 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-17T04:14:19.296 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-17T04:14:19.296 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-17T04:14:19.297 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-17T04:14:19.297 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-17T04:14:19.297 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-17T04:14:19.297 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-17T04:14:19.297 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-17T04:14:19.297 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-17T04:14:19.297 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-17T04:14:19.297 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-17T04:14:19.297 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-17T04:14:19.297 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-17T04:14:19.297 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-17T04:14:19.297 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-17T04:14:19.297 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-17T04:14:19.298 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-17T04:14:19.298 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-17T04:14:19.298 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-17T04:14:19.298 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-17T04:14:19.298 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-17T04:14:19.298 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-17T04:14:19.298 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-17T04:14:19.298 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-17T04:14:19.298 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-17T04:14:19.298 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-17T04:14:19.298 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-17T04:14:19.298 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-17T04:14:19.298 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-17T04:14:19.299 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-17T04:14:19.299 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-17T04:14:19.299 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-17T04:14:19.299 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-17T04:14:19.299 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-17T04:14:19.299 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-17T04:14:19.299 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-17T04:14:19.299 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-17T04:14:19.299 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-17T04:14:19.299 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-17T04:14:19.299 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-17T04:14:19.299 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-17T04:14:19.299 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-17T04:14:19.299 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-17T04:14:19.300 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-17T04:14:19.300 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-17T04:14:19.300 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-17T04:14:19.300 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-17T04:14:19.300 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-17T04:14:19.300 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-17T04:14:19.300 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-17T04:14:19.300 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-17T04:14:19.300 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-17T04:14:19.300 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-17T04:14:19.300 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-17T04:14:19.300 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-17T04:14:19.301 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-17T04:14:19.301 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-17T04:14:19.301 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-17T04:14:19.301 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-17T04:14:19.301 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-17T04:14:19.301 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-17T04:14:19.301 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-17T04:14:19.301 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-17T04:14:19.301 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-17T04:14:19.301 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-17T04:14:19.301 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-17T04:14:19.301 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-17T04:14:19.301 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-17T04:14:19.302 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-17T04:14:19.302 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-17T04:14:19.302 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-17T04:14:19.302 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-17T04:14:19.302 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-17T04:14:19.302 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-17T04:14:19.302 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-17T04:14:19.302 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-17T04:14:19.302 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-17T04:14:19.302 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-17T04:14:19.302 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-17T04:14:19.302 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-17T04:14:19.302 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-17T04:14:19.303 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-17T04:14:19.303 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-17T04:14:19.303 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-17T04:14:19.303 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-17T04:14:19.303 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-17T04:14:19.303 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-17T04:14:19.303 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-17T04:14:19.303 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-17T04:14:19.303 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-17T04:14:19.303 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-17T04:14:19.303 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-17T04:14:19.303 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-17T04:14:19.303 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-17T04:14:19.303 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-17T04:14:19.304 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-17T04:14:19.304 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-17T04:14:19.304 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-17T04:14:19.304 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-17T04:14:19.304 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-17T04:14:19.304 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-17T04:14:19.304 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-17T04:14:19.304 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-17T04:14:19.304 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-17T04:14:19.304 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-17T04:14:19.304 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-17T04:14:19.304 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-17T04:14:19.304 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-17T04:14:19.305 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-17T04:14:19.305 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-17T04:14:19.305 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-17T04:14:19.305 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-17T04:14:19.305 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-17T04:14:19.305 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-17T04:14:19.305 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-17T04:14:19.305 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-17T04:14:19.305 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-17T04:14:19.305 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-17T04:14:19.305 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-17T04:14:19.305 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-17T04:14:19.305 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-17T04:14:19.305 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-17T04:14:19.306 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-17T04:14:19.306 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-17T04:14:19.306 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-17T04:14:19.306 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-17T04:14:19.306 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-17T04:14:19.306 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-17T04:14:19.306 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-17T04:14:19.306 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-17T04:14:19.306 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-17T04:14:19.306 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-17T04:14:19.306 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-17T04:14:19.306 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-17T04:14:19.306 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-17T04:14:19.307 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-17T04:14:19.307 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-17T04:14:19.307 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-17T04:14:19.307 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-17T04:14:19.307 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-17T04:14:19.307 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-17T04:14:19.307 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-17T04:14:19.307 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-17T04:14:19.307 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-17T04:14:19.307 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-17T04:14:19.307 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-17T04:14:19.307 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-17T04:14:19.307 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-17T04:14:19.307 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-17T04:14:19.308 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-17T04:14:19.308 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-17T04:14:19.308 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-17T04:14:19.308 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-17T04:14:19.308 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-17T04:14:19.308 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-17T04:14:19.308 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-17T04:14:19.308 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-17T04:14:19.308 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-17T04:14:19.308 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-17T04:14:19.308 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-17T04:14:19.308 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-17T04:14:19.308 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-17T04:14:19.309 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-17T04:14:19.309 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-17T04:14:19.309 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-17T04:14:19.309 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-17T04:14:19.309 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-17T04:14:19.309 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-17T04:14:19.309 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-17T04:14:19.309 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-17T04:14:19.309 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-17T04:14:19.309 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-17T04:14:19.309 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-17T04:14:19.309 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-17T04:14:19.309 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-17T04:14:19.309 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-17T04:14:19.310 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-17T04:14:19.310 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-17T04:14:19.310 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-17T04:14:19.310 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-17T04:14:19.310 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-17T04:14:19.310 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-17T04:14:19.310 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-17T04:14:19.310 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-17T04:14:19.310 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-17T04:14:19.310 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-17T04:14:19.310 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-17T04:14:19.310 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-17T04:14:19.310 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-17T04:14:19.310 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-17T04:14:19.311 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-17T04:14:19.311 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-17T04:14:19.311 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-17T04:14:19.311 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-17T04:14:19.311 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-17T04:14:19.311 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-17T04:14:19.311 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-17T04:14:19.311 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-17T04:14:19.311 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-17T04:14:19.311 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-17T04:14:19.311 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-17T04:14:19.311 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-17T04:14:19.311 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-17T04:14:19.312 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-17T04:14:19.312 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-17T04:14:19.312 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-17T04:14:19.312 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-17T04:14:19.312 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-17T04:14:19.312 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-17T04:14:19.312 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-17T04:14:19.312 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-17T04:14:19.312 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-17T04:14:19.312 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-17T04:14:19.312 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-17T04:14:19.312 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-17T04:14:19.312 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-17T04:14:19.313 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-17T04:14:19.313 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-17T04:14:19.313 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-17T04:14:19.313 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-17T04:14:19.313 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-17T04:14:19.313 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-17T04:14:19.313 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-17T04:14:19.313 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-17T04:14:19.313 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-17T04:14:19.313 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-17T04:14:19.313 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-17T04:14:19.313 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-17T04:14:19.313 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-17T04:14:19.313 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-17T04:14:19.314 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-17T04:14:19.314 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-17T04:14:19.314 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-17T04:14:19.314 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-17T04:14:19.314 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-17T04:14:19.314 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-17T04:14:19.314 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-17T04:14:19.314 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-17T04:14:19.314 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-17T04:14:19.314 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-17T04:14:19.314 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-17T04:14:19.314 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-17T04:14:19.315 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-17T04:14:19.315 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-17T04:14:19.315 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-17T04:14:19.315 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-17T04:14:19.315 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-17T04:14:19.315 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-17T04:14:19.315 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-17T04:14:19.315 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-17T04:14:19.315 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-17T04:14:19.315 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-17T04:14:19.315 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-17T04:14:19.315 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-17T04:14:19.315 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-17T04:14:19.315 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-17T04:14:19.316 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-17T04:14:19.316 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-17T04:14:19.316 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-17T04:14:19.316 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-17T04:14:19.316 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-17T04:14:19.316 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-17T04:14:19.316 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-17T04:14:19.316 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-17T04:14:19.316 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-17T04:14:19.316 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-17T04:14:19.316 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-17T04:14:19.316 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-17T04:14:19.316 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-17T04:14:19.317 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-17T04:14:19.317 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-17T04:14:19.317 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-17T04:14:19.317 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-17T04:14:19.317 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-17T04:14:19.317 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-17T04:14:19.317 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-17T04:14:19.317 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-17T04:14:19.317 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-17T04:14:19.317 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-17T04:14:19.317 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-17T04:14:19.317 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-17T04:14:19.317 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-17T04:14:19.317 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-17T04:14:19.318 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-17T04:14:19.318 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-17T04:14:19.318 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-17T04:14:19.318 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-17T04:14:19.318 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-17T04:14:19.318 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-17T04:14:19.318 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-17T04:14:19.318 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-17T04:14:19.318 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-17T04:14:19.318 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-17T04:14:19.318 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-17T04:14:19.318 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-17T04:14:19.318 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-17T04:14:19.319 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-17T04:14:19.319 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-17T04:14:19.319 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-17T04:14:19.319 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-17T04:14:19.319 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-17T04:14:19.319 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-17T04:14:19.319 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-17T04:14:19.319 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-17T04:14:19.319 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-17T04:14:19.319 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-17T04:14:19.319 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-17T04:14:19.319 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-17T04:14:19.319 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-17T04:14:19.319 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-17T04:14:19.320 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-17T04:14:19.320 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-17T04:14:19.320 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-17T04:14:19.320 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-17T04:14:19.320 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-17T04:14:19.320 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-17T04:14:19.320 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-17T04:14:19.320 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-17T04:14:19.320 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-17T04:14:19.320 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-17T04:14:19.320 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-17T04:14:19.320 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-17T04:14:19.320 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-17T04:14:19.321 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-17T04:14:19.321 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-17T04:14:19.321 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-17T04:14:19.321 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-17T04:14:19.321 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-17T04:14:19.321 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-17T04:14:19.321 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-17T04:14:19.321 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-17T04:14:19.321 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-17T04:14:19.321 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-17T04:14:19.321 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-17T04:14:19.321 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-17T04:14:19.321 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-17T04:14:19.321 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-17T04:14:19.322 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-17T04:14:19.322 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-17T04:14:19.322 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-17T04:14:19.322 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-17T04:14:19.322 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-17T04:14:19.322 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-17T04:14:19.322 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-17T04:14:19.322 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-17T04:14:19.322 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-17T04:14:19.322 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-17T04:14:19.322 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-17T04:14:19.322 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-17T04:14:19.322 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-17T04:14:19.322 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-17T04:14:19.323 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-17T04:14:19.323 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-17T04:14:19.323 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-17T04:14:19.323 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-17T04:14:19.323 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-17T04:14:19.323 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-17T04:14:19.323 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-17T04:14:19.323 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-17T04:14:19.323 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-17T04:14:19.323 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-17T04:14:19.323 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-17T04:14:19.323 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-17T04:14:19.323 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-17T04:14:19.323 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-17T04:14:19.324 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-17T04:14:19.324 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-17T04:14:19.324 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-17T04:14:19.324 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-17T04:14:19.324 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-17T04:14:19.324 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-17T04:14:19.324 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-17T04:14:19.324 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-17T04:14:19.324 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-17T04:14:19.324 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-17T04:14:19.324 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-17T04:14:19.324 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-17T04:14:19.324 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-17T04:14:19.325 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-17T04:14:19.325 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-17T04:14:19.325 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-17T04:14:19.325 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-17T04:14:19.325 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-17T04:14:19.325 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-17T04:14:19.325 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-17T04:14:19.325 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-17T04:14:19.325 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-17T04:14:19.325 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-17T04:14:19.325 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-17T04:14:19.325 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-17T04:14:19.325 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-17T04:14:19.325 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-17T04:14:19.326 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-17T04:14:19.326 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-17T04:14:19.326 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-17T04:14:19.326 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-17T04:14:19.326 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-17T04:14:19.326 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-17T04:14:19.326 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-17T04:14:19.326 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-17T04:14:19.326 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-17T04:14:19.326 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-17T04:14:19.326 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-17T04:14:19.326 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-17T04:14:19.326 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-17T04:14:19.327 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-17T04:14:19.327 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-17T04:14:19.327 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-17T04:14:19.327 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-17T04:14:19.327 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-17T04:14:19.327 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-17T04:14:19.327 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-17T04:14:19.327 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-17T04:14:19.327 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-17T04:14:19.327 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-17T04:14:19.327 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-17T04:14:19.327 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-17T04:14:19.327 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-17T04:14:19.327 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-17T04:14:19.328 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-17T04:14:19.328 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-17T04:14:19.328 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-17T04:14:19.328 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-17T04:14:19.328 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-17T04:14:19.328 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-17T04:14:19.328 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-17T04:14:19.328 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-17T04:14:19.328 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-17T04:14:19.328 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-17T04:14:19.328 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-17T04:14:19.328 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-17T04:14:19.328 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-17T04:14:19.329 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-17T04:14:19.329 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-17T04:14:19.329 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-17T04:14:19.329 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-17T04:14:19.329 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-17T04:14:19.329 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-17T04:14:19.329 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-17T04:14:19.329 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-17T04:14:19.329 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-17T04:14:19.329 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-17T04:14:19.329 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-17T04:14:19.329 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-17T04:14:19.330 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-17T04:14:19.330 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-17T04:14:19.330 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-17T04:14:19.330 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-17T04:14:19.330 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-17T04:14:19.330 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-17T04:14:19.330 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-17T04:14:19.330 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-17T04:14:19.330 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-17T04:14:19.330 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-17T04:14:19.330 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-17T04:14:19.330 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-17T04:14:19.330 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-17T04:14:19.330 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-17T04:14:19.331 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-17T04:14:19.331 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-17T04:14:19.331 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-17T04:14:19.331 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-17T04:14:19.331 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-17T04:14:19.331 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-17T04:14:19.331 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-17T04:14:19.331 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-17T04:14:19.331 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-17T04:14:19.331 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-17T04:14:19.331 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-17T04:14:19.331 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-17T04:14:19.331 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-17T04:14:19.331 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-17T04:14:19.332 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-17T04:14:19.332 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-17T04:14:19.332 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-17T04:14:19.332 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-17T04:14:19.332 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-17T04:14:19.332 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-17T04:14:19.332 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-17T04:14:19.332 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-17T04:14:19.332 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-17T04:14:19.332 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-17T04:14:19.332 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-17T04:14:19.332 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-17T04:14:19.332 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-17T04:14:19.333 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-17T04:14:19.333 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-17T04:14:19.333 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-17T04:14:19.333 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-17T04:14:19.333 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-17T04:14:19.333 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-17T04:14:19.333 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-17T04:14:19.333 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-17T04:14:19.333 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-17T04:14:19.333 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-17T04:14:19.333 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-17T04:14:19.333 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-17T04:14:19.333 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-17T04:14:19.333 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-17T04:14:19.334 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-17T04:14:19.334 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-17T04:14:19.334 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-17T04:14:19.334 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-17T04:14:19.334 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-17T04:14:19.334 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-17T04:14:19.334 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-17T04:14:19.334 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-17T04:14:19.334 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-17T04:14:19.334 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-17T04:14:19.334 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-17T04:14:19.334 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-17T04:14:19.334 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-17T04:14:19.334 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-17T04:14:19.335 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-17T04:14:19.335 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-17T04:14:19.335 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-17T04:14:19.335 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-17T04:14:19.335 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-17T04:14:19.335 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-17T04:14:19.335 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-17T04:14:19.335 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-17T04:14:19.335 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-17T04:14:19.335 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-17T04:14:19.335 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-17T04:14:19.335 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-17T04:14:19.335 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-17T04:14:19.335 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-17T04:14:19.336 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-17T04:14:19.336 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-17T04:14:19.336 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-17T04:14:19.336 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-17T04:14:19.336 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-17T04:14:19.336 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-17T04:14:19.336 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-17T04:14:19.336 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-17T04:14:19.336 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-17T04:14:19.336 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-17T04:14:19.336 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-17T04:14:19.336 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-17T04:14:19.336 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-17T04:14:19.336 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-17T04:14:19.337 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-17T04:14:19.337 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-17T04:14:19.337 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-17T04:14:19.337 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-17T04:14:19.337 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-17T04:14:19.337 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-17T04:14:19.337 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-17T04:14:19.337 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-17T04:14:19.337 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-17T04:14:19.337 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-17T04:14:19.337 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-17T04:14:19.337 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-17T04:14:19.337 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-17T04:14:19.338 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-17T04:14:19.338 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-17T04:14:19.338 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-17T04:14:19.338 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-17T04:14:19.338 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-17T04:14:19.338 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-17T04:14:19.338 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-17T04:14:19.338 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-17T04:14:19.338 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-17T04:14:19.338 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-17T04:14:19.338 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-17T04:14:19.338 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-17T04:14:19.338 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-17T04:14:19.338 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-17T04:14:19.338 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-17T04:14:19.339 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-17T04:14:19.339 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-17T04:14:19.339 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-17T04:14:19.339 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-17T04:14:19.339 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-17T04:14:19.339 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-17T04:14:19.339 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-17T04:14:19.339 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-17T04:14:19.339 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-17T04:14:19.339 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-17T04:14:19.339 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-17T04:14:19.339 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-17T04:14:19.339 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-17T04:14:19.339 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-17T04:14:19.340 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-17T04:14:19.340 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-17T04:14:19.340 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-17T04:14:19.340 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-17T04:14:19.340 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-17T04:14:19.340 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-17T04:14:19.340 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-17T04:14:19.340 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-17T04:14:19.340 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-17T04:14:19.340 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-17T04:14:19.340 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-17T04:14:19.340 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-17T04:14:19.340 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-17T04:14:19.341 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-17T04:14:19.341 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-17T04:14:19.341 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-17T04:14:19.341 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-17T04:14:19.341 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-17T04:14:19.341 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-17T04:14:19.341 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-17T04:14:19.341 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-17T04:14:19.341 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-17T04:14:19.341 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-17T04:14:19.341 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-17T04:14:19.341 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-17T04:14:19.341 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-17T04:14:19.342 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-17T04:14:19.342 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-17T04:14:19.342 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-17T04:14:19.342 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-17T04:14:19.342 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-17T04:14:19.342 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-17T04:14:19.342 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-17T04:14:19.342 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-17T04:14:19.342 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-17T04:14:19.342 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-17T04:14:19.342 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-17T04:14:19.342 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-17T04:14:19.342 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-17T04:14:19.342 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-17T04:14:19.343 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-17T04:14:19.343 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-17T04:14:19.343 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-17T04:14:19.343 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-17T04:14:19.343 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-17T04:14:19.343 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-17T04:14:19.343 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-17T04:14:19.343 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-17T04:14:19.343 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-17T04:14:19.343 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-17T04:14:19.343 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-17T04:14:19.343 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-17T04:14:19.343 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-17T04:14:19.343 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-17T04:14:19.344 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-17T04:14:19.344 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-17T04:14:19.344 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-17T04:14:19.344 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-17T04:14:19.344 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-17T04:14:19.344 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-17T04:14:19.344 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-17T04:14:19.344 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-17T04:14:19.344 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-17T04:14:19.344 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-17T04:14:19.344 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-17T04:14:19.344 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-17T04:14:19.344 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-17T04:14:19.344 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-17T04:14:19.345 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-17T04:14:19.345 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-17T04:14:19.345 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-17T04:14:19.345 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-17T04:14:19.345 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-17T04:14:19.345 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-17T04:14:19.345 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-17T04:14:19.345 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-17T04:14:19.345 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-17T04:14:19.345 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-17T04:14:19.345 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-17T04:14:19.345 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-17T04:14:19.345 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-17T04:14:19.345 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-17T04:14:19.346 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-17T04:14:19.346 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-17T04:14:19.346 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-17T04:14:19.346 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-17T04:14:19.346 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-17T04:14:19.346 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-17T04:14:19.346 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-17T04:14:19.346 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-17T04:14:19.346 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-17T04:14:19.346 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-17T04:14:19.346 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-17T04:14:19.346 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-17T04:14:19.346 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-17T04:14:19.346 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-17T04:14:19.347 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-17T04:14:19.347 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-17T04:14:19.347 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-17T04:14:19.347 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-17T04:14:19.347 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-17T04:14:19.347 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-17T04:14:19.347 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-17T04:14:19.347 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-17T04:14:19.347 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-17T04:14:19.347 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-17T04:14:19.347 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-17T04:14:19.347 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-17T04:14:19.347 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-17T04:14:19.348 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-17T04:14:19.348 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-17T04:14:19.348 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-17T04:14:19.348 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-17T04:14:19.348 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-17T04:14:19.348 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-17T04:14:19.348 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-17T04:14:19.348 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-17T04:14:19.348 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-17T04:14:19.348 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-17T04:14:19.348 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-17T04:14:19.348 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-17T04:14:19.348 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-17T04:14:19.349 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-17T04:14:19.349 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-17T04:14:19.349 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-17T04:14:19.349 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-17T04:14:19.349 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-17T04:14:19.349 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-17T04:14:19.349 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-17T04:14:19.349 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-17T04:14:19.349 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-17T04:14:19.349 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-17T04:14:19.349 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-17T04:14:19.349 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-17T04:14:19.349 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-17T04:14:19.350 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-17T04:14:19.350 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-17T04:14:19.350 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-17T04:14:19.350 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-17T04:14:19.350 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-17T04:14:19.350 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-17T04:14:19.350 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-17T04:14:19.350 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-17T04:14:19.350 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-17T04:14:19.350 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-17T04:14:19.350 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-17T04:14:19.350 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-17T04:14:19.350 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-17T04:14:19.350 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-17T04:14:19.351 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-17T04:14:19.351 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-17T04:14:19.351 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-17T04:14:19.351 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-17T04:14:19.351 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-17T04:14:19.351 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-17T04:14:19.351 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-17T04:14:19.351 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-17T04:14:19.351 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-17T04:14:19.351 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-17T04:14:19.351 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-17T04:14:19.351 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-17T04:14:19.351 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-17T04:14:19.351 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-17T04:14:19.352 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-17T04:14:19.352 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-17T04:14:19.352 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-17T04:14:19.352 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-17T04:14:19.352 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-17T04:14:19.352 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-17T04:14:19.352 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-17T04:14:19.352 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-17T04:14:19.352 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-17T04:14:19.352 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-17T04:14:19.352 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-17T04:14:19.352 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-17T04:14:19.352 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-17T04:14:19.353 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-17T04:14:19.353 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-17T04:14:19.353 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-17T04:14:19.353 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-17T04:14:19.353 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-17T04:14:19.353 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-17T04:14:19.353 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-17T04:14:19.353 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-17T04:14:19.353 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-17T04:14:19.353 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-17T04:14:19.353 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-17T04:14:19.353 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-17T04:14:19.353 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-17T04:14:19.354 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-17T04:14:19.354 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-17T04:14:19.354 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-17T04:14:19.354 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-17T04:14:19.354 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-17T04:14:19.354 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-17T04:14:19.354 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-17T04:14:19.354 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-17T04:14:19.354 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-17T04:14:19.354 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-17T04:14:19.354 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-17T04:14:19.354 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-17T04:14:19.354 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-17T04:14:19.355 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-17T04:14:19.355 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-17T04:14:19.355 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-17T04:14:19.355 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-17T04:14:19.355 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-17T04:14:19.355 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-17T04:14:19.355 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-17T04:14:19.355 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-17T04:14:19.355 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-17T04:14:19.355 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-17T04:14:19.355 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-17T04:14:19.355 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-17T04:14:19.355 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-17T04:14:19.356 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-17T04:14:19.356 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-17T04:14:19.356 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-17T04:14:19.356 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-17T04:14:19.356 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-17T04:14:19.356 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-17T04:14:19.356 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-17T04:14:19.356 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-17T04:14:19.356 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-17T04:14:19.356 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-17T04:14:19.356 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-17T04:14:19.357 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-17T04:14:19.357 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-17T04:14:19.357 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-17T04:14:19.357 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-17T04:14:19.357 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-17T04:14:19.357 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-17T04:14:19.357 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-17T04:14:19.357 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-17T04:14:19.357 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-17T04:14:19.357 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-17T04:14:19.357 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-17T04:14:19.357 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-17T04:14:19.358 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-17T04:14:19.358 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-17T04:14:19.358 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-17T04:14:19.358 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-17T04:14:19.358 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-17T04:14:19.358 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-17T04:14:19.358 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-17T04:14:19.358 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-17T04:14:19.358 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-17T04:14:19.358 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-17T04:14:19.358 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-17T04:14:19.358 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-17T04:14:19.358 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-17T04:14:19.359 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-17T04:14:19.359 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-17T04:14:19.359 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-17T04:14:19.359 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-17T04:14:19.359 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-17T04:14:19.359 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-17T04:14:19.359 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-17T04:14:19.359 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-17T04:14:19.359 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-17T04:14:19.359 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-17T04:14:19.359 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-17T04:14:19.359 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-17T04:14:19.360 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-17T04:14:19.360 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-17T04:14:19.360 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-17T04:14:19.360 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-17T04:14:19.360 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-17T04:14:19.360 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-17T04:14:19.360 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-17T04:14:19.360 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-17T04:14:19.360 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-17T04:14:19.360 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-17T04:14:19.360 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-17T04:14:19.360 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-17T04:14:19.361 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-17T04:14:19.361 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-17T04:14:19.361 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-17T04:14:19.361 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-17T04:14:19.361 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-17T04:14:19.361 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-17T04:14:19.361 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-17T04:14:19.361 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-17T04:14:19.361 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-17T04:14:19.361 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-17T04:14:19.361 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-17T04:14:19.361 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-17T04:14:19.361 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-17T04:14:19.362 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-17T04:14:19.362 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-17T04:14:19.362 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-17T04:14:19.362 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-17T04:14:19.362 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-17T04:14:19.362 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-17T04:14:19.362 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-17T04:14:19.362 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-17T04:14:19.362 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-17T04:14:19.362 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-17T04:14:19.362 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-17T04:14:19.362 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-17T04:14:19.362 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-17T04:14:19.362 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-17T04:14:19.363 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-17T04:14:19.363 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-17T04:14:19.363 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-17T04:14:19.363 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-17T04:14:19.363 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-17T04:14:19.363 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-17T04:14:19.363 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-17T04:14:19.363 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-17T04:14:19.363 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-17T04:14:19.363 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-17T04:14:19.363 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-17T04:14:19.363 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-17T04:14:19.363 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-17T04:14:19.364 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-17T04:14:19.364 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-17T04:14:19.364 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-17T04:14:19.364 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-17T04:14:19.364 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-17T04:14:19.364 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-17T04:14:19.364 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-17T04:14:19.364 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-17T04:14:19.364 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-17T04:14:19.364 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-17T04:14:19.364 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-17T04:14:19.364 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-17T04:14:19.364 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-17T04:14:19.365 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-17T04:14:19.365 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-17T04:14:19.365 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-17T04:14:19.365 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-17T04:14:19.365 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-17T04:14:19.365 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-17T04:14:19.365 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-17T04:14:19.365 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-17T04:14:19.365 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-17T04:14:19.365 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-17T04:14:19.365 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-17T04:14:19.365 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-17T04:14:19.366 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-17T04:14:19.366 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-17T04:14:19.366 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-17T04:14:19.366 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-17T04:14:19.366 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-17T04:14:19.366 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-17T04:14:19.366 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-17T04:14:19.366 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-17T04:14:19.366 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-17T04:14:19.366 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-17T04:14:19.366 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-17T04:14:19.366 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-17T04:14:19.366 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-17T04:14:19.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-17T04:14:19.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-17T04:14:19.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-17T04:14:19.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-17T04:14:19.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-17T04:14:19.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-17T04:14:19.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-17T04:14:19.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-17T04:14:19.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-17T04:14:19.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-17T04:14:19.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-17T04:14:19.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-17T04:14:19.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-17T04:14:19.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-17T04:14:19.368 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-17T04:14:19.368 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-17T04:14:19.368 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-17T04:14:19.368 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-17T04:14:19.368 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-17T04:14:19.368 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-17T04:14:19.368 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-17T04:14:19.368 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-17T04:14:19.368 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-17T04:14:19.368 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-17T04:14:19.368 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-17T04:14:19.368 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-17T04:14:19.369 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-17T04:14:19.369 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-17T04:14:19.369 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-17T04:14:19.369 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-17T04:14:19.369 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-17T04:14:19.369 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-17T04:14:19.369 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-17T04:14:19.369 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-17T04:14:19.369 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-17T04:14:19.369 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-17T04:14:19.369 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-17T04:14:19.369 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-17T04:14:19.369 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-17T04:14:19.370 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-17T04:14:19.370 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-17T04:14:19.370 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-17T04:14:19.370 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-17T04:14:19.370 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-17T04:14:19.370 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-17T04:14:19.370 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-17T04:14:19.370 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-17T04:14:19.370 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-17T04:14:19.370 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-17T04:14:19.370 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-17T04:14:19.370 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-17T04:14:19.370 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-17T04:14:19.370 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-17T04:14:19.371 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-17T04:14:19.371 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-17T04:14:19.371 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-17T04:14:19.371 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-17T04:14:19.371 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-17T04:14:19.371 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-17T04:14:19.371 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-17T04:14:19.371 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-17T04:14:19.371 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-17T04:14:19.371 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-17T04:14:19.371 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-17T04:14:19.371 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-17T04:14:19.372 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-17T04:14:19.372 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-17T04:14:19.372 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-17T04:14:19.372 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-17T04:14:19.372 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-17T04:14:19.372 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-17T04:14:19.372 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-17T04:14:19.372 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-17T04:14:19.372 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-17T04:14:19.372 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-17T04:14:19.372 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-17T04:14:19.372 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-17T04:14:19.372 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-17T04:14:19.373 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-17T04:14:19.373 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-17T04:14:19.373 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-17T04:14:19.373 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-17T04:14:19.373 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-17T04:14:19.373 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-17T04:14:19.373 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-17T04:14:19.373 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-17T04:14:19.373 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-17T04:14:19.373 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-17T04:14:19.373 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-17T04:14:19.373 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-17T04:14:19.373 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-17T04:14:19.373 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-17T04:14:19.374 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-17T04:14:19.374 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-17T04:14:19.374 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-17T04:14:19.374 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-17T04:14:19.374 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-17T04:14:19.374 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-17T04:14:19.374 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-17T04:14:19.374 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-17T04:14:19.374 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-17T04:14:19.374 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-17T04:14:19.374 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-17T04:14:19.374 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-17T04:14:19.374 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-17T04:14:19.374 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-17T04:14:19.375 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-17T04:14:19.375 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-17T04:14:19.375 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-17T04:14:19.375 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-17T04:14:19.375 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-17T04:14:19.375 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-17T04:14:19.375 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-17T04:14:19.375 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-17T04:14:19.375 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-17T04:14:19.375 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-17T04:14:19.375 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-17T04:14:19.375 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-17T04:14:19.375 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-17T04:14:19.376 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-17T04:14:19.376 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-17T04:14:19.376 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-17T04:14:19.376 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-17T04:14:19.376 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-17T04:14:19.376 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-17T04:14:19.376 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-17T04:14:19.376 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-17T04:14:19.376 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-17T04:14:19.376 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-17T04:14:19.376 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-17T04:14:19.376 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-17T04:14:19.376 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-17T04:14:19.376 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-17T04:14:19.377 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-17T04:14:19.377 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-17T04:14:19.377 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-17T04:14:19.377 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-17T04:14:19.377 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-17T04:14:19.377 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-17T04:14:19.377 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-17T04:14:19.377 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-17T04:14:19.377 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-17T04:14:19.377 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-17T04:14:19.377 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-17T04:14:19.377 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-17T04:14:19.377 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-17T04:14:19.378 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-17T04:14:19.378 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-17T04:14:19.378 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-17T04:14:19.378 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-17T04:14:19.378 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-17T04:14:19.378 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-17T04:14:19.378 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-17T04:14:19.378 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-17T04:14:19.378 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-17T04:14:19.378 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-17T04:14:19.378 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-17T04:14:19.378 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-17T04:14:19.378 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-17T04:14:19.378 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-17T04:14:19.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-17T04:14:19.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-17T04:14:19.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-17T04:14:19.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-17T04:14:19.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-17T04:14:19.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-17T04:14:19.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-17T04:14:19.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-17T04:14:19.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-17T04:14:19.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-17T04:14:19.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-17T04:14:19.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-17T04:14:19.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-17T04:14:19.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-17T04:14:19.380 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-17T04:14:19.380 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-17T04:14:19.380 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-17T04:14:19.380 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-17T04:14:19.380 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-17T04:14:19.380 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-17T04:14:19.380 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-17T04:14:19.380 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-17T04:14:19.380 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-17T04:14:19.380 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-17T04:14:19.380 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-17T04:14:19.380 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-17T04:14:19.380 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-17T04:14:19.380 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-17T04:14:19.381 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-17T04:14:19.381 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-17T04:14:19.381 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-17T04:14:19.381 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-17T04:14:19.381 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-17T04:14:19.381 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-17T04:14:19.381 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-17T04:14:19.381 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-17T04:14:19.381 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-17T04:14:19.381 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-17T04:14:19.381 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-17T04:14:19.381 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-17T04:14:19.381 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-17T04:14:19.382 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-17T04:14:19.382 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-17T04:14:19.382 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-17T04:14:19.382 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-17T04:14:19.382 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-17T04:14:19.382 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-17T04:14:19.382 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-17T04:14:19.382 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-17T04:14:19.382 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-17T04:14:19.382 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-17T04:14:19.382 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-17T04:14:19.382 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-17T04:14:19.382 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-17T04:14:19.382 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-17T04:14:19.383 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-17T04:14:19.383 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-17T04:14:19.383 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-17T04:14:19.383 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-17T04:14:19.383 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-17T04:14:19.383 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-17T04:14:19.383 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-17T04:14:19.383 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-17T04:14:19.383 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-17T04:14:19.383 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-17T04:14:19.383 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-17T04:14:19.383 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-17T04:14:19.383 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-17T04:14:19.383 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-17T04:14:19.384 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-17T04:14:19.384 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-17T04:14:19.384 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-17T04:14:19.384 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-17T04:14:19.384 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-17T04:14:19.384 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-17T04:14:19.384 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-17T04:14:19.384 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-17T04:14:19.384 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-17T04:14:19.384 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-17T04:14:19.384 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-17T04:14:19.384 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-17T04:14:19.385 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-17T04:14:19.385 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-17T04:14:19.385 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-17T04:14:19.385 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-17T04:14:19.385 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-17T04:14:19.385 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-17T04:14:19.385 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-17T04:14:19.385 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-17T04:14:19.385 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-17T04:14:19.385 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-17T04:14:19.385 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-17T04:14:19.385 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-17T04:14:19.385 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-17T04:14:19.386 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-17T04:14:19.386 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-17T04:14:19.386 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-17T04:14:19.386 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-17T04:14:19.386 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-17T04:14:19.386 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-17T04:14:19.386 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-17T04:14:19.386 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-17T04:14:19.386 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-17T04:14:19.386 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-17T04:14:19.386 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-17T04:14:19.386 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-17T04:14:19.386 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-17T04:14:19.386 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-17T04:14:19.387 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-17T04:14:19.387 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-17T04:14:19.387 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-17T04:14:19.387 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-17T04:14:19.387 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-17T04:14:19.387 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-17T04:14:19.387 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-17T04:14:19.387 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-17T04:14:19.387 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-17T04:14:19.387 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-17T04:14:19.387 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-17T04:14:19.387 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-17T04:14:19.387 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-17T04:14:19.388 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-17T04:14:19.388 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-17T04:14:19.388 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-17T04:14:19.388 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-17T04:14:19.388 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-17T04:14:19.388 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-17T04:14:19.388 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-17T04:14:19.388 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-17T04:14:19.388 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-17T04:14:19.388 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-17T04:14:19.388 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-17T04:14:19.388 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-17T04:14:19.388 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-17T04:14:19.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-17T04:14:19.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-17T04:14:19.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-17T04:14:19.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-17T04:14:19.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-17T04:14:19.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-17T04:14:19.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-17T04:14:19.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-17T04:14:19.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-17T04:14:19.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-17T04:14:19.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-17T04:14:19.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-17T04:14:19.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-17T04:14:19.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-17T04:14:19.390 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-17T04:14:19.390 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-17T04:14:19.390 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-17T04:14:19.390 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-17T04:14:19.390 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-17T04:14:19.390 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-17T04:14:19.390 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-17T04:14:19.390 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-17T04:14:19.390 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-17T04:14:19.390 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-17T04:14:19.390 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-17T04:14:19.390 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-17T04:14:19.390 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-17T04:14:19.391 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-17T04:14:19.391 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-17T04:14:19.391 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-17T04:14:19.391 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-17T04:14:19.391 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-17T04:14:19.391 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-17T04:14:19.391 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-17T04:14:19.391 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-17T04:14:19.391 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-17T04:14:19.391 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-17T04:14:19.391 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-17T04:14:19.391 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-17T04:14:19.391 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-17T04:14:19.392 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-17T04:14:19.392 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-17T04:14:19.392 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-17T04:14:19.392 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-17T04:14:19.392 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-17T04:14:19.392 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-17T04:14:19.392 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-17T04:14:19.392 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-17T04:14:19.392 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-17T04:14:19.392 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-17T04:14:19.392 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-17T04:14:19.392 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-17T04:14:19.393 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-17T04:14:19.393 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-17T04:14:19.393 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-17T04:14:19.393 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-17T04:14:19.393 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-17T04:14:19.393 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-17T04:14:19.393 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-17T04:14:19.393 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-17T04:14:19.393 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-17T04:14:19.393 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-17T04:14:19.393 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-17T04:14:19.393 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-17T04:14:19.393 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-17T04:14:19.393 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-17T04:14:19.394 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-17T04:14:19.394 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-17T04:14:19.394 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-17T04:14:19.394 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-17T04:14:19.394 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-17T04:14:19.394 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-17T04:14:19.394 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-17T04:14:19.394 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-17T04:14:19.394 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-17T04:14:19.394 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-17T04:14:19.394 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-17T04:14:19.394 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-17T04:14:19.394 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-17T04:14:19.395 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-17T04:14:19.395 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-17T04:14:19.395 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-17T04:14:19.395 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-17T04:14:19.395 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-17T04:14:19.395 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-17T04:14:19.395 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-17T04:14:19.395 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-17T04:14:19.395 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-17T04:14:19.395 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-17T04:14:19.395 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-17T04:14:19.395 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-17T04:14:19.395 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-17T04:14:19.395 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-17T04:14:19.396 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-17T04:14:19.396 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-17T04:14:19.396 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-17T04:14:19.396 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-17T04:14:19.396 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-17T04:14:19.396 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-17T04:14:19.396 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-17T04:14:19.396 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-17T04:14:19.396 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-17T04:14:19.396 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-17T04:14:19.396 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-17T04:14:19.396 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-17T04:14:19.396 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-17T04:14:19.396 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-17T04:14:19.397 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-17T04:14:19.397 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-17T04:14:19.397 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-17T04:14:19.397 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-17T04:14:19.397 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-17T04:14:19.397 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-17T04:14:19.397 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-17T04:14:19.397 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-17T04:14:19.397 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-17T04:14:19.397 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-17T04:14:19.397 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-17T04:14:19.397 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-17T04:14:19.397 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-17T04:14:19.397 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-17T04:14:19.398 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-17T04:14:19.398 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-17T04:14:19.398 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-17T04:14:19.398 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-17T04:14:19.398 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-17T04:14:19.398 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-17T04:14:19.398 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-17T04:14:19.398 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-17T04:14:19.398 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-17T04:14:19.398 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-17T04:14:19.398 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-17T04:14:19.398 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-17T04:14:19.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-17T04:14:19.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-17T04:14:19.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-17T04:14:19.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-17T04:14:19.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-17T04:14:19.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-17T04:14:19.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-17T04:14:19.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-17T04:14:19.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-17T04:14:19.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-17T04:14:19.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-17T04:14:19.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-17T04:14:19.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-17T04:14:19.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-17T04:14:19.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-17T04:14:19.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-17T04:14:19.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-17T04:14:19.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-17T04:14:19.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-17T04:14:19.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-17T04:14:19.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-17T04:14:19.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-17T04:14:19.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-17T04:14:19.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-17T04:14:19.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-17T04:14:19.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-17T04:14:19.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-17T04:14:19.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-17T04:14:19.401 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-17T04:14:19.401 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-17T04:14:19.401 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-17T04:14:19.401 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-17T04:14:19.401 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-17T04:14:19.401 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-17T04:14:19.401 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-17T04:14:19.401 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-17T04:14:19.401 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-17T04:14:19.401 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-17T04:14:19.401 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-17T04:14:19.401 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-17T04:14:19.401 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-17T04:14:19.402 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-17T04:14:19.402 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-17T04:14:19.402 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-17T04:14:19.402 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-17T04:14:19.402 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-17T04:14:19.402 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-17T04:14:19.402 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-17T04:14:19.402 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-17T04:14:19.402 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-17T04:14:19.402 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-17T04:14:19.402 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-17T04:14:19.402 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-17T04:14:19.402 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-17T04:14:19.402 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-17T04:14:19.403 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-17T04:14:19.403 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-17T04:14:19.403 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-17T04:14:19.403 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-17T04:14:19.403 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-17T04:14:19.403 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-17T04:14:19.403 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-17T04:14:19.403 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-17T04:14:19.403 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-17T04:14:19.403 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-17T04:14:19.403 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-17T04:14:19.403 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-17T04:14:19.403 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-17T04:14:19.403 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-17T04:14:19.404 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-17T04:14:19.404 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-17T04:14:19.404 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-17T04:14:19.404 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-17T04:14:19.404 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-17T04:14:19.404 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-17T04:14:19.404 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-17T04:14:19.404 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-17T04:14:19.404 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-17T04:14:19.404 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-17T04:14:19.404 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-17T04:14:19.404 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-17T04:14:19.404 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-17T04:14:19.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-17T04:14:19.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-17T04:14:19.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-17T04:14:19.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-17T04:14:19.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-17T04:14:19.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-17T04:14:19.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-17T04:14:19.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-17T04:14:19.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-17T04:14:19.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-17T04:14:19.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-17T04:14:19.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-17T04:14:19.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-17T04:14:19.406 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-17T04:14:19.406 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-17T04:14:19.406 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-17T04:14:19.406 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-17T04:14:19.406 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-17T04:14:19.406 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-17T04:14:19.406 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-17T04:14:19.406 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-17T04:14:19.406 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-17T04:14:19.406 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-17T04:14:19.406 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-17T04:14:19.406 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-17T04:14:19.406 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-17T04:14:19.406 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-17T04:14:19.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-17T04:14:19.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-17T04:14:19.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-17T04:14:19.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-17T04:14:19.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-17T04:14:19.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-17T04:14:19.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-17T04:14:19.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-17T04:14:19.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-17T04:14:19.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-17T04:14:19.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-17T04:14:19.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-17T04:14:19.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-17T04:14:19.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-17T04:14:19.408 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-17T04:14:19.408 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-17T04:14:19.408 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-17T04:14:19.408 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-17T04:14:19.408 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-17T04:14:19.408 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-17T04:14:19.408 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-17T04:14:19.408 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-17T04:14:19.408 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-17T04:14:19.408 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-17T04:14:19.408 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-17T04:14:19.408 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-17T04:14:19.408 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-17T04:14:19.409 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-17T04:14:19.409 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-17T04:14:19.409 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-17T04:14:19.409 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-17T04:14:19.409 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-17T04:14:19.409 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-17T04:14:19.409 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-17T04:14:19.409 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-17T04:14:19.409 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-17T04:14:19.409 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-17T04:14:19.409 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-17T04:14:19.409 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-17T04:14:19.409 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-17T04:14:19.409 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-17T04:14:19.410 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-17T04:14:19.410 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-17T04:14:19.410 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-17T04:14:19.410 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-17T04:14:19.410 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-17T04:14:19.410 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-17T04:14:19.410 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-17T04:14:19.410 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-17T04:14:19.410 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-17T04:14:19.410 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-17T04:14:19.410 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-17T04:14:19.410 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-17T04:14:19.410 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-17T04:14:19.410 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-17T04:14:19.411 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-17T04:14:19.411 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-17T04:14:19.411 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-17T04:14:19.411 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-17T04:14:19.411 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-17T04:14:19.414 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:14:19.415 DEBUG:teuthology.orchestra.run.smithi100:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-17T04:14:19.440 INFO:teuthology.orchestra.run.smithi100.stdout:postinstall scriptlet (using /bin/sh): 2024-09-17T04:14:19.441 INFO:teuthology.orchestra.run.smithi100.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-17T04:14:19.441 INFO:teuthology.orchestra.run.smithi100.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-17T04:14:19.441 INFO:teuthology.orchestra.run.smithi100.stdout:fi 2024-09-17T04:14:19.441 INFO:teuthology.orchestra.run.smithi100.stdout:for file in vmlinuz System.map config; do 2024-09-17T04:14:19.441 INFO:teuthology.orchestra.run.smithi100.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-17T04:14:19.441 INFO:teuthology.orchestra.run.smithi100.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-17T04:14:19.441 INFO:teuthology.orchestra.run.smithi100.stdout: fi 2024-09-17T04:14:19.441 INFO:teuthology.orchestra.run.smithi100.stdout:done 2024-09-17T04:14:19.441 INFO:teuthology.orchestra.run.smithi100.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-17T04:14:19.441 INFO:teuthology.orchestra.run.smithi100.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-17T04:14:19.441 INFO:teuthology.orchestra.run.smithi100.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-17T04:14:19.442 INFO:teuthology.orchestra.run.smithi100.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-17T04:14:19.442 INFO:teuthology.orchestra.run.smithi100.stdout:fi 2024-09-17T04:14:19.442 INFO:teuthology.orchestra.run.smithi100.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-17T04:14:19.442 INFO:teuthology.orchestra.run.smithi100.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:14:19.442 INFO:teuthology.orchestra.run.smithi100.stdout:fi 2024-09-17T04:14:19.442 INFO:teuthology.orchestra.run.smithi100.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-17T04:14:19.442 INFO:teuthology.orchestra.run.smithi100.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-17T04:14:19.442 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:14:19.442 INFO:teuthology.orchestra.run.smithi100.stdout:fi 2024-09-17T04:14:19.443 DEBUG:teuthology.orchestra.run.smithi100:> sudo rpm -qi grub2-tools 2024-09-17T04:14:19.520 INFO:teuthology.orchestra.run.smithi100.stdout:Name : grub2-tools 2024-09-17T04:14:19.521 INFO:teuthology.orchestra.run.smithi100.stdout:Epoch : 1 2024-09-17T04:14:19.521 INFO:teuthology.orchestra.run.smithi100.stdout:Version : 2.06 2024-09-17T04:14:19.521 INFO:teuthology.orchestra.run.smithi100.stdout:Release : 82.el9 2024-09-17T04:14:19.521 INFO:teuthology.orchestra.run.smithi100.stdout:Architecture: x86_64 2024-09-17T04:14:19.521 INFO:teuthology.orchestra.run.smithi100.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T04:14:19.521 INFO:teuthology.orchestra.run.smithi100.stdout:Group : Unspecified 2024-09-17T04:14:19.521 INFO:teuthology.orchestra.run.smithi100.stdout:Size : 8274670 2024-09-17T04:14:19.521 INFO:teuthology.orchestra.run.smithi100.stdout:License : GPLv3+ 2024-09-17T04:14:19.521 INFO:teuthology.orchestra.run.smithi100.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T04:14:19.521 INFO:teuthology.orchestra.run.smithi100.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T04:14:19.521 INFO:teuthology.orchestra.run.smithi100.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T04:14:19.522 INFO:teuthology.orchestra.run.smithi100.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T04:14:19.522 INFO:teuthology.orchestra.run.smithi100.stdout:Packager : builder@centos.org 2024-09-17T04:14:19.522 INFO:teuthology.orchestra.run.smithi100.stdout:Vendor : CentOS 2024-09-17T04:14:19.522 INFO:teuthology.orchestra.run.smithi100.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T04:14:19.522 INFO:teuthology.orchestra.run.smithi100.stdout:Summary : Support tools for GRUB. 2024-09-17T04:14:19.522 INFO:teuthology.orchestra.run.smithi100.stdout:Description : 2024-09-17T04:14:19.522 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:14:19.522 INFO:teuthology.orchestra.run.smithi100.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T04:14:19.522 INFO:teuthology.orchestra.run.smithi100.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T04:14:19.522 INFO:teuthology.orchestra.run.smithi100.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T04:14:19.522 INFO:teuthology.orchestra.run.smithi100.stdout:hardware devices. 2024-09-17T04:14:19.522 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:14:19.522 INFO:teuthology.orchestra.run.smithi100.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T04:14:19.523 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T04:14:19.523 INFO:teuthology.task.kernel:Updating grub on smithi100 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:14:19.523 DEBUG:teuthology.orchestra.run.smithi100:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T04:14:20.164 INFO:teuthology.orchestra.run.smithi100.stderr:Generating grub configuration file ... 2024-09-17T04:14:21.523 INFO:teuthology.orchestra.run.smithi100.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T04:14:21.555 INFO:teuthology.orchestra.run.smithi100.stderr:done 2024-09-17T04:14:21.564 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-09-17T04:14:21.579 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.1MpcrgcCUZ 2024-09-17T04:14:21.579 DEBUG:teuthology.orchestra.run.smithi100:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.1MpcrgcCUZ 2024-09-17T04:14:21.647 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.1MpcrgcCUZ 2024-09-17T04:14:21.790 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.1MpcrgcCUZ is 6KB 2024-09-17T04:14:21.838 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.1MpcrgcCUZ 2024-09-17T04:14:21.853 DEBUG:teuthology.orchestra.run.smithi100:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T04:14:21.916 INFO:teuthology.orchestra.run.smithi100.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T04:14:21.916 INFO:teuthology.orchestra.run.smithi100.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T04:14:21.916 INFO:teuthology.orchestra.run.smithi100.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T04:14:21.916 INFO:teuthology.orchestra.run.smithi100.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-17T04:14:21.917 DEBUG:teuthology.orchestra.run.smithi100:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:14:22.071 DEBUG:teuthology.orchestra.run.smithi100:> sudo shutdown -r now 2024-09-17T04:14:32.093 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-17T04:14:32.093 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-09-17T04:14:32.094 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:14:50.429 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.184 2024-09-17T04:14:52.075 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-17T04:14:52.076 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-17T04:14:52.076 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:14:59.433 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-09-17T04:14:59.433 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:15:02.493 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.184 2024-09-17T04:15:10.401 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-09-17T04:15:14.497 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-09-17T04:15:14.497 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:15:19.405 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-17T04:15:19.405 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:15:22.461 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-09-17T04:15:34.473 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-17T04:15:34.473 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:15:46.503 DEBUG:teuthology.orchestra.run.smithi184:> true 2024-09-17T04:15:47.027 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-09-17T04:15:47.027 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-17T04:15:47.027 DEBUG:teuthology.orchestra.run.smithi184:> uname -r 2024-09-17T04:15:47.061 INFO:teuthology.orchestra.run.smithi184.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:15:47.062 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:15:47.062 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T04:15:47.062 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T04:15:48.063 DEBUG:teuthology.parallel:result is None 2024-09-17T04:16:06.374 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-09-17T04:16:06.848 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-17T04:16:06.848 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-17T04:16:06.849 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-09-17T04:16:06.885 INFO:teuthology.orchestra.run.smithi100.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:16:06.886 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:16:06.886 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T04:16:06.886 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T04:16:07.889 DEBUG:teuthology.parallel:result is None 2024-09-17T04:16:07.889 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-17T04:16:07.899 INFO:teuthology.task.internal:Creating test directory... 2024-09-17T04:16:07.899 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T04:16:07.903 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T04:16:07.943 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-17T04:16:07.950 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-17T04:16:07.999 INFO:teuthology.task.internal:Creating archive directory... 2024-09-17T04:16:07.999 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T04:16:08.003 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T04:16:08.052 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-17T04:16:08.059 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-17T04:16:08.059 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T04:16:08.084 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T04:16:08.208 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T04:16:08.218 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T04:16:08.232 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T04:16:08.280 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T04:16:08.283 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-17T04:16:08.290 INFO:teuthology.task.internal:Configuring sudo... 2024-09-17T04:16:08.290 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T04:16:08.294 DEBUG:teuthology.orchestra.run.smithi184:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T04:16:08.350 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-17T04:16:08.359 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-17T04:16:08.359 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T04:16:08.363 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T04:16:08.397 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T04:16:08.500 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T04:16:08.582 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T04:16:08.648 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T04:16:08.718 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:16:08.719 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T04:16:08.774 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T04:16:08.837 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T04:16:08.921 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T04:16:08.984 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T04:16:09.053 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:16:09.054 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T04:16:09.107 DEBUG:teuthology.orchestra.run.smithi100:> sudo service rsyslog restart 2024-09-17T04:16:09.112 DEBUG:teuthology.orchestra.run.smithi184:> sudo service rsyslog restart 2024-09-17T04:16:09.194 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T04:16:09.232 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T04:16:09.585 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-17T04:16:09.595 INFO:teuthology.task.internal:Starting timer... 2024-09-17T04:16:09.595 INFO:teuthology.run_tasks:Running task pcp... 2024-09-17T04:16:09.651 INFO:teuthology.run_tasks:Running task selinux... 2024-09-17T04:16:09.701 DEBUG:teuthology.orchestra.run.smithi100:> sudo service auditd rotate 2024-09-17T04:16:09.780 INFO:teuthology.orchestra.run.smithi100.stdout:Rotating logs: 2024-09-17T04:16:09.784 DEBUG:teuthology.orchestra.run.smithi184:> sudo service auditd rotate 2024-09-17T04:16:09.871 INFO:teuthology.orchestra.run.smithi184.stdout:Rotating logs: 2024-09-17T04:16:09.874 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-17T04:16:09.875 DEBUG:teuthology.orchestra.run.smithi100:> /usr/sbin/getenforce 2024-09-17T04:16:09.914 INFO:teuthology.orchestra.run.smithi100.stdout:Permissive 2024-09-17T04:16:09.914 DEBUG:teuthology.orchestra.run.smithi184:> /usr/sbin/getenforce 2024-09-17T04:16:09.960 INFO:teuthology.orchestra.run.smithi184.stdout:Permissive 2024-09-17T04:16:09.960 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi100.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi184.front.sepia.ceph.com': 'permissive'} 2024-09-17T04:16:09.961 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T04:16:10.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T04:16:10.019 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T04:16:10.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T04:16:10.077 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-17T04:16:10.077 DEBUG:teuthology.orchestra.run.smithi100:> sudo /usr/sbin/setenforce permissive 2024-09-17T04:16:10.159 DEBUG:teuthology.orchestra.run.smithi184:> sudo /usr/sbin/setenforce permissive 2024-09-17T04:16:10.224 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-17T04:16:10.233 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-17T04:16:10.244 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-17T04:16:10.245 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi100.front.sepia.ceph.com,smithi184.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-17T04:20:54.374 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi100.front.sepia.ceph.com'), Remote(name='ubuntu@smithi184.front.sepia.ceph.com')] 2024-09-17T04:20:54.376 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-17T04:20:54.377 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:20:54.465 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-09-17T04:20:54.575 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-17T04:20:54.575 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-09-17T04:20:54.576 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:20:54.659 DEBUG:teuthology.orchestra.run.smithi184:> true 2024-09-17T04:20:54.763 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-09-17T04:20:54.763 INFO:teuthology.run_tasks:Running task clock... 2024-09-17T04:20:54.779 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-17T04:20:54.779 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T04:20:54.780 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T04:20:54.784 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T04:20:54.784 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T04:20:54.840 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T04:20:54.863 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T04:20:54.869 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T04:20:54.893 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T04:20:54.921 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: ntpd: command not found 2024-09-17T04:20:54.944 INFO:teuthology.orchestra.run.smithi100.stdout:506 Cannot talk to daemon 2024-09-17T04:20:54.944 INFO:teuthology.orchestra.run.smithi184.stderr:sudo: ntpd: command not found 2024-09-17T04:20:54.968 INFO:teuthology.orchestra.run.smithi184.stdout:506 Cannot talk to daemon 2024-09-17T04:20:54.974 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T04:20:54.998 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T04:20:55.003 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T04:20:55.029 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T04:20:55.094 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-09-17T04:20:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T04:20:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-09-17T04:20:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T04:20:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T04:20:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T04:20:55.101 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T04:20:55.129 INFO:teuthology.orchestra.run.smithi184.stderr:bash: line 1: ntpq: command not found 2024-09-17T04:20:55.135 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T04:20:55.136 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2024-09-17T04:20:55.136 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T04:20:55.136 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T04:20:55.136 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T04:20:55.136 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T04:20:55.137 INFO:teuthology.run_tasks:Running task install... 2024-09-17T04:20:55.145 DEBUG:teuthology.task.install:project ceph 2024-09-17T04:20:55.145 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}} 2024-09-17T04:20:55.145 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'} 2024-09-17T04:20:55.145 INFO:teuthology.task.install:Using flavor: default 2024-09-17T04:20:55.152 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-17T04:20:55.153 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-17T04:20:55.153 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf', 'tag': None, 'wait_for_package': False} 2024-09-17T04:20:55.154 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T04:20:55.156 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf', 'tag': None, 'wait_for_package': False} 2024-09-17T04:20:55.156 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T04:20:55.299 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/centos/9/flavors/default/ 2024-09-17T04:20:55.300 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4989.gce1b342b 2024-09-17T04:20:55.310 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/centos/9/flavors/default/ 2024-09-17T04:20:55.310 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4989.gce1b342b 2024-09-17T04:20:55.412 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T04:20:55.412 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:20:55.412 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T04:20:55.422 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T04:20:55.422 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:20:55.422 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T04:20:55.469 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-17T04:20:55.469 DEBUG:teuthology.orchestra.run.smithi100:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T04:20:55.479 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-17T04:20:55.479 DEBUG:teuthology.orchestra.run.smithi184:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T04:20:55.565 DEBUG:teuthology.orchestra.run.smithi100:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T04:20:55.576 DEBUG:teuthology.orchestra.run.smithi184:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T04:20:55.643 DEBUG:teuthology.orchestra.run.smithi100:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T04:20:55.655 DEBUG:teuthology.orchestra.run.smithi184:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T04:20:55.699 INFO:teuthology.orchestra.run.smithi100.stdout:check_obsoletes = 1 2024-09-17T04:20:55.702 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum clean all 2024-09-17T04:20:55.709 INFO:teuthology.orchestra.run.smithi184.stdout:check_obsoletes = 1 2024-09-17T04:20:55.713 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum clean all 2024-09-17T04:20:55.971 INFO:teuthology.orchestra.run.smithi100.stdout:45 files removed 2024-09-17T04:20:55.988 INFO:teuthology.orchestra.run.smithi184.stdout:45 files removed 2024-09-17T04:20:56.004 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-17T04:20:56.021 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-17T04:20:56.702 INFO:teuthology.orchestra.run.smithi100.stdout:ceph packages for x86_64 278 kB/s | 84 kB 00:00 2024-09-17T04:20:56.717 INFO:teuthology.orchestra.run.smithi184.stdout:ceph packages for x86_64 277 kB/s | 84 kB 00:00 2024-09-17T04:20:57.002 INFO:teuthology.orchestra.run.smithi100.stdout:ceph noarch packages 95 kB/s | 24 kB 00:00 2024-09-17T04:20:57.020 INFO:teuthology.orchestra.run.smithi184.stdout:ceph noarch packages 93 kB/s | 24 kB 00:00 2024-09-17T04:20:57.303 INFO:teuthology.orchestra.run.smithi100.stdout:ceph source packages 7.4 kB/s | 1.9 kB 00:00 2024-09-17T04:20:57.303 INFO:teuthology.orchestra.run.smithi184.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-17T04:20:58.161 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.3 MB 00:00 2024-09-17T04:20:58.384 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 7.9 MB/s | 8.3 MB 00:01 2024-09-17T04:21:12.745 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - AppStream 1.5 MB/s | 20 MB 00:13 2024-09-17T04:21:13.713 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 1.4 MB/s | 20 MB 00:14 2024-09-17T04:21:18.045 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - CRB 5.2 MB/s | 6.5 MB 00:01 2024-09-17T04:21:18.845 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 6.1 MB/s | 6.5 MB 00:01 2024-09-17T04:21:19.629 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 19 kB 00:00 2024-09-17T04:21:20.817 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-17T04:21:21.379 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 19 kB 00:01 2024-09-17T04:21:22.533 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-17T04:21:27.239 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-09-17T04:21:28.797 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-09-17T04:21:29.649 INFO:teuthology.orchestra.run.smithi184.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-17T04:21:29.650 INFO:teuthology.orchestra.run.smithi184.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-17T04:21:29.664 INFO:teuthology.orchestra.run.smithi184.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-17T04:21:29.665 INFO:teuthology.orchestra.run.smithi184.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-17T04:21:29.666 INFO:teuthology.orchestra.run.smithi184.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-17T04:21:29.668 INFO:teuthology.orchestra.run.smithi184.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-17T04:21:29.670 INFO:teuthology.orchestra.run.smithi184.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-17T04:21:29.673 INFO:teuthology.orchestra.run.smithi184.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-17T04:21:29.674 INFO:teuthology.orchestra.run.smithi184.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-17T04:21:29.740 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-17T04:21:29.757 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================= 2024-09-17T04:21:29.757 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-09-17T04:21:29.757 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================= 2024-09-17T04:21:29.757 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-17T04:21:29.757 INFO:teuthology.orchestra.run.smithi184.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-17T04:21:29.757 INFO:teuthology.orchestra.run.smithi184.stdout: ceph x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 6.6 k 2024-09-17T04:21:29.757 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 5.5 M 2024-09-17T04:21:29.757 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 873 k 2024-09-17T04:21:29.757 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 148 k 2024-09-17T04:21:29.757 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 1.5 M 2024-09-17T04:21:29.757 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 154 k 2024-09-17T04:21:29.758 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 14 M 2024-09-17T04:21:29.758 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 7.4 M 2024-09-17T04:21:29.758 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 49 k 2024-09-17T04:21:29.758 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 10 M 2024-09-17T04:21:29.758 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 40 M 2024-09-17T04:21:29.758 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 285 k 2024-09-17T04:21:29.758 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 784 k 2024-09-17T04:21:29.758 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-mirror x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 225 k 2024-09-17T04:21:29.758 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-shell x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 27 k 2024-09-17T04:21:29.758 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-top noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 18 k 2024-09-17T04:21:29.758 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-17T04:21:29.758 INFO:teuthology.orchestra.run.smithi184.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-17T04:21:29.759 INFO:teuthology.orchestra.run.smithi184.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-17T04:21:29.759 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 32 k 2024-09-17T04:21:29.759 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 766 k 2024-09-17T04:21:29.759 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 127 k 2024-09-17T04:21:29.759 INFO:teuthology.orchestra.run.smithi184.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-17T04:21:29.759 INFO:teuthology.orchestra.run.smithi184.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-17T04:21:29.759 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-17T04:21:29.759 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-17T04:21:29.759 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-17T04:21:29.759 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-17T04:21:29.759 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 162 k 2024-09-17T04:21:29.759 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 321 k 2024-09-17T04:21:29.760 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 305 k 2024-09-17T04:21:29.760 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 99 k 2024-09-17T04:21:29.760 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 86 k 2024-09-17T04:21:29.760 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 3.1 M 2024-09-17T04:21:29.760 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 170 k 2024-09-17T04:21:29.760 INFO:teuthology.orchestra.run.smithi184.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-17T04:21:29.760 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrading: 2024-09-17T04:21:29.760 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-17T04:21:29.760 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-17T04:21:29.760 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-17T04:21:29.760 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-17T04:21:29.760 INFO:teuthology.orchestra.run.smithi184.stdout: librados2 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 3.4 M 2024-09-17T04:21:29.761 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 3.2 M 2024-09-17T04:21:29.761 INFO:teuthology.orchestra.run.smithi184.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-17T04:21:29.761 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-17T04:21:29.761 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-09-17T04:21:29.761 INFO:teuthology.orchestra.run.smithi184.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-17T04:21:29.761 INFO:teuthology.orchestra.run.smithi184.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-17T04:21:29.761 INFO:teuthology.orchestra.run.smithi184.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T04:21:29.761 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 20 M 2024-09-17T04:21:29.761 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 34 k 2024-09-17T04:21:29.761 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 2.3 M 2024-09-17T04:21:29.762 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 291 k 2024-09-17T04:21:29.762 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 5.0 M 2024-09-17T04:21:29.762 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 18 M 2024-09-17T04:21:29.762 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 16 k 2024-09-17T04:21:29.762 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 25 k 2024-09-17T04:21:29.762 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T04:21:29.762 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T04:21:29.762 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T04:21:29.762 INFO:teuthology.orchestra.run.smithi184.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T04:21:29.762 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T04:21:29.762 INFO:teuthology.orchestra.run.smithi184.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-17T04:21:29.762 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T04:21:29.763 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T04:21:29.763 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 159 k 2024-09-17T04:21:29.763 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T04:21:29.763 INFO:teuthology.orchestra.run.smithi184.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-17T04:21:29.763 INFO:teuthology.orchestra.run.smithi184.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T04:21:29.763 INFO:teuthology.orchestra.run.smithi184.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T04:21:29.763 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T04:21:29.763 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T04:21:29.763 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 507 k 2024-09-17T04:21:29.763 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T04:21:29.764 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 5.3 M 2024-09-17T04:21:29.764 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T04:21:29.764 INFO:teuthology.orchestra.run.smithi184.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-17T04:21:29.764 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T04:21:29.764 INFO:teuthology.orchestra.run.smithi184.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-17T04:21:29.764 INFO:teuthology.orchestra.run.smithi184.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-17T04:21:29.764 INFO:teuthology.orchestra.run.smithi184.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-17T04:21:29.764 INFO:teuthology.orchestra.run.smithi184.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T04:21:29.764 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T04:21:29.764 INFO:teuthology.orchestra.run.smithi184.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T04:21:29.765 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-17T04:21:29.765 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-17T04:21:29.765 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-17T04:21:29.765 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T04:21:29.765 INFO:teuthology.orchestra.run.smithi184.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-17T04:21:29.765 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T04:21:29.765 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T04:21:29.765 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T04:21:29.765 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 45 k 2024-09-17T04:21:29.765 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 143 k 2024-09-17T04:21:29.765 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T04:21:29.765 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T04:21:29.766 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T04:21:29.766 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T04:21:29.766 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-17T04:21:29.766 INFO:teuthology.orchestra.run.smithi184.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-17T04:21:29.766 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T04:21:29.766 INFO:teuthology.orchestra.run.smithi184.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T04:21:29.766 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T04:21:29.766 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-17T04:21:29.766 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-17T04:21:29.766 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T04:21:29.766 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T04:21:29.766 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-17T04:21:29.767 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T04:21:29.767 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T04:21:29.767 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T04:21:29.767 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T04:21:29.767 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T04:21:29.767 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T04:21:29.767 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T04:21:29.767 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T04:21:29.767 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T04:21:29.767 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T04:21:29.767 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T04:21:29.767 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T04:21:29.767 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T04:21:29.767 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T04:21:29.768 INFO:teuthology.orchestra.run.smithi184.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T04:21:29.768 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T04:21:29.768 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T04:21:29.768 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T04:21:29.768 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T04:21:29.768 INFO:teuthology.orchestra.run.smithi184.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-17T04:21:29.768 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T04:21:29.768 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T04:21:29.768 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T04:21:29.768 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T04:21:29.769 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-17T04:21:29.769 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T04:21:29.769 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T04:21:29.769 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T04:21:29.769 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T04:21:29.769 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T04:21:29.769 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T04:21:29.769 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T04:21:29.769 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T04:21:29.769 INFO:teuthology.orchestra.run.smithi184.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T04:21:29.769 INFO:teuthology.orchestra.run.smithi184.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T04:21:29.769 INFO:teuthology.orchestra.run.smithi184.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T04:21:29.770 INFO:teuthology.orchestra.run.smithi184.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-17T04:21:29.770 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T04:21:29.770 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T04:21:29.770 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T04:21:29.770 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T04:21:29.770 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-17T04:21:29.770 INFO:teuthology.orchestra.run.smithi184.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-17T04:21:29.770 INFO:teuthology.orchestra.run.smithi184.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T04:21:29.770 INFO:teuthology.orchestra.run.smithi184.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T04:21:29.770 INFO:teuthology.orchestra.run.smithi184.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T04:21:29.770 INFO:teuthology.orchestra.run.smithi184.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-17T04:21:29.770 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T04:21:29.770 INFO:teuthology.orchestra.run.smithi184.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-17T04:21:29.771 INFO:teuthology.orchestra.run.smithi184.stdout:Installing weak dependencies: 2024-09-17T04:21:29.771 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-17T04:21:29.771 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:21:29.771 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-17T04:21:29.771 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================= 2024-09-17T04:21:29.771 INFO:teuthology.orchestra.run.smithi184.stdout:Install 149 Packages 2024-09-17T04:21:29.771 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrade 8 Packages 2024-09-17T04:21:29.771 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:21:29.772 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 228 M 2024-09-17T04:21:29.772 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-17T04:21:30.490 INFO:teuthology.orchestra.run.smithi184.stdout:(1/157): ceph-19.3.0-4989.gce1b342b.el9.x86_64. 34 kB/s | 6.6 kB 00:00 2024-09-17T04:21:30.865 INFO:teuthology.orchestra.run.smithi184.stdout:(2/157): ceph-fuse-19.3.0-4989.gce1b342b.el9.x8 2.3 MB/s | 873 kB 00:00 2024-09-17T04:21:30.941 INFO:teuthology.orchestra.run.smithi184.stdout:(3/157): ceph-base-19.3.0-4989.gce1b342b.el9.x8 8.6 MB/s | 5.5 MB 00:00 2024-09-17T04:21:30.966 INFO:teuthology.orchestra.run.smithi184.stdout:(4/157): ceph-immutable-object-cache-19.3.0-498 1.5 MB/s | 148 kB 00:00 2024-09-17T04:21:31.124 INFO:teuthology.orchestra.run.smithi184.stdout:(5/157): ceph-mds-19.3.0-4989.gce1b342b.el9.x86 12 MB/s | 2.3 MB 00:00 2024-09-17T04:21:31.129 INFO:teuthology.orchestra.run.smithi100.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-17T04:21:31.130 INFO:teuthology.orchestra.run.smithi100.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-17T04:21:31.144 INFO:teuthology.orchestra.run.smithi100.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-17T04:21:31.145 INFO:teuthology.orchestra.run.smithi100.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-17T04:21:31.146 INFO:teuthology.orchestra.run.smithi100.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-17T04:21:31.147 INFO:teuthology.orchestra.run.smithi100.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-17T04:21:31.149 INFO:teuthology.orchestra.run.smithi100.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-17T04:21:31.152 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-17T04:21:31.153 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-17T04:21:31.221 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-09-17T04:21:31.251 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================= 2024-09-17T04:21:31.252 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-09-17T04:21:31.252 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================= 2024-09-17T04:21:31.252 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-09-17T04:21:31.252 INFO:teuthology.orchestra.run.smithi100.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-17T04:21:31.252 INFO:teuthology.orchestra.run.smithi100.stdout: ceph x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 6.6 k 2024-09-17T04:21:31.252 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 5.5 M 2024-09-17T04:21:31.252 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 873 k 2024-09-17T04:21:31.252 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 148 k 2024-09-17T04:21:31.252 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 1.5 M 2024-09-17T04:21:31.252 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 154 k 2024-09-17T04:21:31.252 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 14 M 2024-09-17T04:21:31.252 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 7.4 M 2024-09-17T04:21:31.253 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 49 k 2024-09-17T04:21:31.253 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 10 M 2024-09-17T04:21:31.253 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 40 M 2024-09-17T04:21:31.253 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 285 k 2024-09-17T04:21:31.253 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 784 k 2024-09-17T04:21:31.253 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 225 k 2024-09-17T04:21:31.253 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-shell x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 27 k 2024-09-17T04:21:31.253 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 18 k 2024-09-17T04:21:31.253 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-17T04:21:31.253 INFO:teuthology.orchestra.run.smithi100.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-17T04:21:31.253 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-17T04:21:31.253 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 32 k 2024-09-17T04:21:31.254 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 766 k 2024-09-17T04:21:31.254 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 127 k 2024-09-17T04:21:31.254 INFO:teuthology.orchestra.run.smithi100.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-17T04:21:31.254 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-17T04:21:31.254 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-17T04:21:31.254 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-17T04:21:31.254 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-17T04:21:31.254 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-17T04:21:31.254 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 162 k 2024-09-17T04:21:31.254 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 321 k 2024-09-17T04:21:31.254 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 305 k 2024-09-17T04:21:31.254 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 99 k 2024-09-17T04:21:31.254 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 86 k 2024-09-17T04:21:31.255 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 3.1 M 2024-09-17T04:21:31.255 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 170 k 2024-09-17T04:21:31.255 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-17T04:21:31.255 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrading: 2024-09-17T04:21:31.255 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-17T04:21:31.255 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-17T04:21:31.255 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-17T04:21:31.255 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-17T04:21:31.255 INFO:teuthology.orchestra.run.smithi100.stdout: librados2 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 3.4 M 2024-09-17T04:21:31.255 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 3.2 M 2024-09-17T04:21:31.255 INFO:teuthology.orchestra.run.smithi100.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-17T04:21:31.255 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-17T04:21:31.255 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-09-17T04:21:31.256 INFO:teuthology.orchestra.run.smithi100.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-17T04:21:31.256 INFO:teuthology.orchestra.run.smithi100.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-17T04:21:31.256 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T04:21:31.256 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 20 M 2024-09-17T04:21:31.256 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 34 k 2024-09-17T04:21:31.256 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 2.3 M 2024-09-17T04:21:31.256 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 291 k 2024-09-17T04:21:31.256 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 5.0 M 2024-09-17T04:21:31.256 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 18 M 2024-09-17T04:21:31.256 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 16 k 2024-09-17T04:21:31.256 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 25 k 2024-09-17T04:21:31.256 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T04:21:31.257 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T04:21:31.257 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T04:21:31.257 INFO:teuthology.orchestra.run.smithi100.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T04:21:31.257 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T04:21:31.257 INFO:teuthology.orchestra.run.smithi100.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-17T04:21:31.257 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T04:21:31.257 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T04:21:31.257 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 159 k 2024-09-17T04:21:31.257 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T04:21:31.257 INFO:teuthology.orchestra.run.smithi100.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-17T04:21:31.257 INFO:teuthology.orchestra.run.smithi100.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T04:21:31.258 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T04:21:31.258 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T04:21:31.258 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T04:21:31.258 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 507 k 2024-09-17T04:21:31.258 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T04:21:31.258 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 5.3 M 2024-09-17T04:21:31.258 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T04:21:31.258 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-17T04:21:31.258 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T04:21:31.258 INFO:teuthology.orchestra.run.smithi100.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-17T04:21:31.258 INFO:teuthology.orchestra.run.smithi100.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-17T04:21:31.259 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-17T04:21:31.259 INFO:teuthology.orchestra.run.smithi100.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T04:21:31.259 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T04:21:31.259 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T04:21:31.259 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-17T04:21:31.259 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-17T04:21:31.259 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-17T04:21:31.259 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T04:21:31.259 INFO:teuthology.orchestra.run.smithi100.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-17T04:21:31.259 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T04:21:31.259 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T04:21:31.260 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T04:21:31.260 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 45 k 2024-09-17T04:21:31.260 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 143 k 2024-09-17T04:21:31.260 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T04:21:31.260 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T04:21:31.260 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T04:21:31.260 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T04:21:31.260 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-17T04:21:31.260 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-17T04:21:31.260 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T04:21:31.260 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T04:21:31.260 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T04:21:31.260 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-17T04:21:31.261 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-17T04:21:31.261 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T04:21:31.261 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T04:21:31.261 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-17T04:21:31.261 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T04:21:31.261 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T04:21:31.261 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T04:21:31.261 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T04:21:31.261 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T04:21:31.261 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T04:21:31.261 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T04:21:31.261 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T04:21:31.262 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T04:21:31.262 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T04:21:31.262 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T04:21:31.262 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T04:21:31.262 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T04:21:31.262 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T04:21:31.262 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T04:21:31.262 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T04:21:31.262 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T04:21:31.262 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T04:21:31.262 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T04:21:31.262 INFO:teuthology.orchestra.run.smithi100.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-17T04:21:31.263 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T04:21:31.263 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T04:21:31.263 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T04:21:31.263 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T04:21:31.263 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-17T04:21:31.263 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T04:21:31.263 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T04:21:31.263 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T04:21:31.263 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T04:21:31.263 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T04:21:31.263 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T04:21:31.264 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T04:21:31.264 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T04:21:31.264 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T04:21:31.264 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T04:21:31.264 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T04:21:31.264 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-17T04:21:31.264 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T04:21:31.264 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T04:21:31.264 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T04:21:31.264 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T04:21:31.264 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-17T04:21:31.264 INFO:teuthology.orchestra.run.smithi100.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-17T04:21:31.264 INFO:teuthology.orchestra.run.smithi100.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T04:21:31.265 INFO:teuthology.orchestra.run.smithi100.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T04:21:31.265 INFO:teuthology.orchestra.run.smithi100.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T04:21:31.265 INFO:teuthology.orchestra.run.smithi100.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-17T04:21:31.265 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T04:21:31.265 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-17T04:21:31.265 INFO:teuthology.orchestra.run.smithi100.stdout:Installing weak dependencies: 2024-09-17T04:21:31.265 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-17T04:21:31.265 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:21:31.265 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-09-17T04:21:31.265 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================= 2024-09-17T04:21:31.265 INFO:teuthology.orchestra.run.smithi100.stdout:Install 149 Packages 2024-09-17T04:21:31.265 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrade 8 Packages 2024-09-17T04:21:31.266 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:21:31.266 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 228 M 2024-09-17T04:21:31.266 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-09-17T04:21:31.516 INFO:teuthology.orchestra.run.smithi184.stdout:(6/157): ceph-mon-19.3.0-4989.gce1b342b.el9.x86 13 MB/s | 5.0 MB 00:00 2024-09-17T04:21:31.550 INFO:teuthology.orchestra.run.smithi184.stdout:(7/157): ceph-mgr-19.3.0-4989.gce1b342b.el9.x86 2.6 MB/s | 1.5 MB 00:00 2024-09-17T04:21:31.632 INFO:teuthology.orchestra.run.smithi100.stdout:(1/157): ceph-19.3.0-4989.gce1b342b.el9.x86_64. 35 kB/s | 6.6 kB 00:00 2024-09-17T04:21:31.834 INFO:teuthology.orchestra.run.smithi184.stdout:(8/157): ceph-common-19.3.0-4989.gce1b342b.el9. 13 MB/s | 20 MB 00:01 2024-09-17T04:21:31.900 INFO:teuthology.orchestra.run.smithi184.stdout:(9/157): ceph-selinux-19.3.0-4989.gce1b342b.el9 380 kB/s | 25 kB 00:00 2024-09-17T04:21:32.360 INFO:teuthology.orchestra.run.smithi100.stdout:(2/157): ceph-fuse-19.3.0-4989.gce1b342b.el9.x8 1.2 MB/s | 873 kB 00:00 2024-09-17T04:21:32.533 INFO:teuthology.orchestra.run.smithi100.stdout:(3/157): ceph-immutable-object-cache-19.3.0-498 858 kB/s | 148 kB 00:00 2024-09-17T04:21:32.801 INFO:teuthology.orchestra.run.smithi184.stdout:(10/157): ceph-osd-19.3.0-4989.gce1b342b.el9.x8 14 MB/s | 18 MB 00:01 2024-09-17T04:21:32.884 INFO:teuthology.orchestra.run.smithi184.stdout:(11/157): cephfs-mirror-19.3.0-4989.gce1b342b.e 2.6 MB/s | 225 kB 00:00 2024-09-17T04:21:32.968 INFO:teuthology.orchestra.run.smithi184.stdout:(12/157): cephfs-shell-19.3.0-4989.gce1b342b.el 326 kB/s | 27 kB 00:00 2024-09-17T04:21:33.051 INFO:teuthology.orchestra.run.smithi184.stdout:(13/157): libcephfs-devel-19.3.0-4989.gce1b342b 382 kB/s | 32 kB 00:00 2024-09-17T04:21:33.151 INFO:teuthology.orchestra.run.smithi184.stdout:(14/157): libcephfs2-19.3.0-4989.gce1b342b.el9. 7.5 MB/s | 766 kB 00:00 2024-09-17T04:21:33.260 INFO:teuthology.orchestra.run.smithi184.stdout:(15/157): libcephsqlite-19.3.0-4989.gce1b342b.e 1.4 MB/s | 159 kB 00:00 2024-09-17T04:21:33.335 INFO:teuthology.orchestra.run.smithi184.stdout:(16/157): librados-devel-19.3.0-4989.gce1b342b. 1.7 MB/s | 127 kB 00:00 2024-09-17T04:21:33.427 INFO:teuthology.orchestra.run.smithi184.stdout:(17/157): libradosstriper1-19.3.0-4989.gce1b342 5.4 MB/s | 507 kB 00:00 2024-09-17T04:21:33.454 INFO:teuthology.orchestra.run.smithi100.stdout:(4/157): ceph-mds-19.3.0-4989.gce1b342b.el9.x86 2.5 MB/s | 2.3 MB 00:00 2024-09-17T04:21:33.670 INFO:teuthology.orchestra.run.smithi184.stdout:(18/157): ceph-radosgw-19.3.0-4989.gce1b342b.el 4.9 MB/s | 10 MB 00:02 2024-09-17T04:21:33.761 INFO:teuthology.orchestra.run.smithi184.stdout:(19/157): python3-ceph-argparse-19.3.0-4989.gce 494 kB/s | 45 kB 00:00 2024-09-17T04:21:33.862 INFO:teuthology.orchestra.run.smithi184.stdout:(20/157): librgw2-19.3.0-4989.gce1b342b.el9.x86 12 MB/s | 5.3 MB 00:00 2024-09-17T04:21:33.887 INFO:teuthology.orchestra.run.smithi184.stdout:(21/157): python3-ceph-common-19.3.0-4989.gce1b 1.1 MB/s | 143 kB 00:00 2024-09-17T04:21:33.929 INFO:teuthology.orchestra.run.smithi184.stdout:(22/157): python3-cephfs-19.3.0-4989.gce1b342b. 2.4 MB/s | 162 kB 00:00 2024-09-17T04:21:33.933 INFO:teuthology.orchestra.run.smithi100.stdout:(5/157): ceph-mgr-19.3.0-4989.gce1b342b.el9.x86 3.2 MB/s | 1.5 MB 00:00 2024-09-17T04:21:33.962 INFO:teuthology.orchestra.run.smithi184.stdout:(23/157): python3-rados-19.3.0-4989.gce1b342b.e 4.2 MB/s | 321 kB 00:00 2024-09-17T04:21:34.004 INFO:teuthology.orchestra.run.smithi184.stdout:(24/157): python3-rbd-19.3.0-4989.gce1b342b.el9 4.0 MB/s | 305 kB 00:00 2024-09-17T04:21:34.063 INFO:teuthology.orchestra.run.smithi184.stdout:(25/157): python3-rgw-19.3.0-4989.gce1b342b.el9 993 kB/s | 99 kB 00:00 2024-09-17T04:21:34.088 INFO:teuthology.orchestra.run.smithi184.stdout:(26/157): rbd-fuse-19.3.0-4989.gce1b342b.el9.x8 1.0 MB/s | 86 kB 00:00 2024-09-17T04:21:34.172 INFO:teuthology.orchestra.run.smithi184.stdout:(27/157): rbd-nbd-19.3.0-4989.gce1b342b.el9.x86 2.0 MB/s | 170 kB 00:00 2024-09-17T04:21:34.255 INFO:teuthology.orchestra.run.smithi184.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4989.g 410 kB/s | 34 kB 00:00 2024-09-17T04:21:34.325 INFO:teuthology.orchestra.run.smithi100.stdout:(6/157): ceph-base-19.3.0-4989.gce1b342b.el9.x8 1.9 MB/s | 5.5 MB 00:02 2024-09-17T04:21:34.592 INFO:teuthology.orchestra.run.smithi100.stdout:(7/157): ceph-mon-19.3.0-4989.gce1b342b.el9.x86 7.5 MB/s | 5.0 MB 00:00 2024-09-17T04:21:34.664 INFO:teuthology.orchestra.run.smithi184.stdout:(29/157): ceph-test-19.3.0-4989.gce1b342b.el9.x 14 MB/s | 40 MB 00:02 2024-09-17T04:21:34.691 INFO:teuthology.orchestra.run.smithi184.stdout:(30/157): ceph-mgr-cephadm-19.3.0-4989.gce1b342 356 kB/s | 154 kB 00:00 2024-09-17T04:21:34.756 INFO:teuthology.orchestra.run.smithi184.stdout:(31/157): rbd-mirror-19.3.0-4989.gce1b342b.el9. 4.5 MB/s | 3.1 MB 00:00 2024-09-17T04:21:34.834 INFO:teuthology.orchestra.run.smithi100.stdout:(8/157): ceph-common-19.3.0-4989.gce1b342b.el9. 5.9 MB/s | 20 MB 00:03 2024-09-17T04:21:34.841 INFO:teuthology.orchestra.run.smithi184.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4989.gce 3.4 MB/s | 291 kB 00:00 2024-09-17T04:21:34.909 INFO:teuthology.orchestra.run.smithi100.stdout:(9/157): ceph-selinux-19.3.0-4989.gce1b342b.el9 338 kB/s | 25 kB 00:00 2024-09-17T04:21:34.931 INFO:teuthology.orchestra.run.smithi184.stdout:(33/157): ceph-mgr-rook-19.3.0-4989.gce1b342b.e 540 kB/s | 49 kB 00:00 2024-09-17T04:21:35.040 INFO:teuthology.orchestra.run.smithi184.stdout:(34/157): ceph-prometheus-alerts-19.3.0-4989.gc 152 kB/s | 16 kB 00:00 2024-09-17T04:21:35.257 INFO:teuthology.orchestra.run.smithi184.stdout:(35/157): ceph-mgr-diskprediction-local-19.3.0- 13 MB/s | 7.4 MB 00:00 2024-09-17T04:21:35.349 INFO:teuthology.orchestra.run.smithi184.stdout:(36/157): cephadm-19.3.0-4989.gce1b342b.el9.noa 8.4 MB/s | 784 kB 00:00 2024-09-17T04:21:35.424 INFO:teuthology.orchestra.run.smithi184.stdout:(37/157): cephfs-top-19.3.0-4989.gce1b342b.el9. 244 kB/s | 18 kB 00:00 2024-09-17T04:21:35.550 INFO:teuthology.orchestra.run.smithi184.stdout:(38/157): ceph-volume-19.3.0-4989.gce1b342b.el9 560 kB/s | 285 kB 00:00 2024-09-17T04:21:35.700 INFO:teuthology.orchestra.run.smithi184.stdout:(39/157): ceph-mgr-dashboard-19.3.0-4989.gce1b3 14 MB/s | 14 MB 00:01 2024-09-17T04:21:35.718 INFO:teuthology.orchestra.run.smithi100.stdout:(10/157): ceph-radosgw-19.3.0-4989.gce1b342b.el 9.3 MB/s | 10 MB 00:01 2024-09-17T04:21:35.725 INFO:teuthology.orchestra.run.smithi184.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 266 kB/s | 80 kB 00:00 2024-09-17T04:21:35.784 INFO:teuthology.orchestra.run.smithi100.stdout:(11/157): cephfs-mirror-19.3.0-4989.gce1b342b.e 3.3 MB/s | 225 kB 00:00 2024-09-17T04:21:35.817 INFO:teuthology.orchestra.run.smithi184.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.6 MB/s | 191 kB 00:00 2024-09-17T04:21:35.826 INFO:teuthology.orchestra.run.smithi100.stdout:(12/157): cephfs-shell-19.3.0-4989.gce1b342b.el 654 kB/s | 27 kB 00:00 2024-09-17T04:21:35.885 INFO:teuthology.orchestra.run.smithi100.stdout:(13/157): libcephfs-devel-19.3.0-4989.gce1b342b 543 kB/s | 32 kB 00:00 2024-09-17T04:21:35.934 INFO:teuthology.orchestra.run.smithi184.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-09-17T04:21:36.001 INFO:teuthology.orchestra.run.smithi184.stdout:(43/157): libgfortran-11.5.0-2.el9.x86_64.rpm 1.7 MB/s | 801 kB 00:00 2024-09-17T04:21:36.019 INFO:teuthology.orchestra.run.smithi100.stdout:(14/157): libcephfs2-19.3.0-4989.gce1b342b.el9. 5.6 MB/s | 766 kB 00:00 2024-09-17T04:21:36.026 INFO:teuthology.orchestra.run.smithi184.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-09-17T04:21:36.068 INFO:teuthology.orchestra.run.smithi184.stdout:(45/157): python3-cryptography-36.0.1-4.el9.x86 4.9 MB/s | 1.2 MB 00:00 2024-09-17T04:21:36.093 INFO:teuthology.orchestra.run.smithi184.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-09-17T04:21:36.094 INFO:teuthology.orchestra.run.smithi100.stdout:(15/157): libcephsqlite-19.3.0-4989.gce1b342b.e 2.1 MB/s | 159 kB 00:00 2024-09-17T04:21:36.119 INFO:teuthology.orchestra.run.smithi184.stdout:(47/157): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-09-17T04:21:36.144 INFO:teuthology.orchestra.run.smithi184.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-09-17T04:21:36.153 INFO:teuthology.orchestra.run.smithi100.stdout:(16/157): librados-devel-19.3.0-4989.gce1b342b. 2.1 MB/s | 127 kB 00:00 2024-09-17T04:21:36.211 INFO:teuthology.orchestra.run.smithi100.stdout:(17/157): libradosstriper1-19.3.0-4989.gce1b342 8.5 MB/s | 507 kB 00:00 2024-09-17T04:21:36.253 INFO:teuthology.orchestra.run.smithi184.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 987 kB/s | 107 kB 00:00 2024-09-17T04:21:36.303 INFO:teuthology.orchestra.run.smithi184.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 437 kB/s | 22 kB 00:00 2024-09-17T04:21:36.362 INFO:teuthology.orchestra.run.smithi184.stdout:(51/157): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 943 kB 00:00 2024-09-17T04:21:36.387 INFO:teuthology.orchestra.run.smithi184.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 313 kB 00:00 2024-09-17T04:21:36.412 INFO:teuthology.orchestra.run.smithi184.stdout:(53/157): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-09-17T04:21:36.438 INFO:teuthology.orchestra.run.smithi184.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 433 kB/s | 33 kB 00:00 2024-09-17T04:21:36.463 INFO:teuthology.orchestra.run.smithi184.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-17T04:21:36.488 INFO:teuthology.orchestra.run.smithi184.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-17T04:21:36.505 INFO:teuthology.orchestra.run.smithi184.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 4.0 MB/s | 171 kB 00:00 2024-09-17T04:21:36.538 INFO:teuthology.orchestra.run.smithi184.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-17T04:21:36.589 INFO:teuthology.orchestra.run.smithi184.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 543 kB/s | 45 kB 00:00 2024-09-17T04:21:36.637 INFO:teuthology.orchestra.run.smithi100.stdout:(18/157): librgw2-19.3.0-4989.gce1b342b.el9.x86 12 MB/s | 5.3 MB 00:00 2024-09-17T04:21:36.672 INFO:teuthology.orchestra.run.smithi184.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 4.8 MB/s | 662 kB 00:00 2024-09-17T04:21:36.711 INFO:teuthology.orchestra.run.smithi100.stdout:(19/157): python3-ceph-argparse-19.3.0-4989.gce 613 kB/s | 45 kB 00:00 2024-09-17T04:21:36.714 INFO:teuthology.orchestra.run.smithi184.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 4.5 MB/s | 581 kB 00:00 2024-09-17T04:21:36.739 INFO:teuthology.orchestra.run.smithi184.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 367 kB/s | 24 kB 00:00 2024-09-17T04:21:36.765 INFO:teuthology.orchestra.run.smithi184.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-09-17T04:21:36.771 INFO:teuthology.orchestra.run.smithi100.stdout:(20/157): python3-ceph-common-19.3.0-4989.gce1b 2.3 MB/s | 143 kB 00:00 2024-09-17T04:21:36.838 INFO:teuthology.orchestra.run.smithi100.stdout:(21/157): python3-cephfs-19.3.0-4989.gce1b342b. 2.4 MB/s | 162 kB 00:00 2024-09-17T04:21:36.882 INFO:teuthology.orchestra.run.smithi184.stdout:(64/157): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-17T04:21:36.907 INFO:teuthology.orchestra.run.smithi184.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 292 kB 00:00 2024-09-17T04:21:36.932 INFO:teuthology.orchestra.run.smithi184.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 764 kB/s | 38 kB 00:00 2024-09-17T04:21:37.246 INFO:teuthology.orchestra.run.smithi100.stdout:(22/157): ceph-test-19.3.0-4989.gce1b342b.el9.x 17 MB/s | 40 MB 00:02 2024-09-17T04:21:37.280 INFO:teuthology.orchestra.run.smithi100.stdout:(23/157): python3-rados-19.3.0-4989.gce1b342b.e 726 kB/s | 321 kB 00:00 2024-09-17T04:21:37.305 INFO:teuthology.orchestra.run.smithi100.stdout:(24/157): python3-rbd-19.3.0-4989.gce1b342b.el9 5.1 MB/s | 305 kB 00:00 2024-09-17T04:21:37.330 INFO:teuthology.orchestra.run.smithi100.stdout:(25/157): python3-rgw-19.3.0-4989.gce1b342b.el9 1.9 MB/s | 99 kB 00:00 2024-09-17T04:21:37.356 INFO:teuthology.orchestra.run.smithi100.stdout:(26/157): rbd-fuse-19.3.0-4989.gce1b342b.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-17T04:21:37.406 INFO:teuthology.orchestra.run.smithi100.stdout:(27/157): rbd-nbd-19.3.0-4989.gce1b342b.el9.x86 3.3 MB/s | 170 kB 00:00 2024-09-17T04:21:37.457 INFO:teuthology.orchestra.run.smithi100.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4989.g 688 kB/s | 34 kB 00:00 2024-09-17T04:21:37.491 INFO:teuthology.orchestra.run.smithi184.stdout:(67/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 7.9 MB/s | 4.4 MB 00:00 2024-09-17T04:21:37.507 INFO:teuthology.orchestra.run.smithi100.stdout:(29/157): ceph-mgr-cephadm-19.3.0-4989.gce1b342 3.0 MB/s | 154 kB 00:00 2024-09-17T04:21:37.549 INFO:teuthology.orchestra.run.smithi184.stdout:(68/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.6 MB/s | 3.0 MB 00:01 2024-09-17T04:21:37.557 INFO:teuthology.orchestra.run.smithi100.stdout:(30/157): rbd-mirror-19.3.0-4989.gce1b342b.el9. 14 MB/s | 3.1 MB 00:00 2024-09-17T04:21:37.700 INFO:teuthology.orchestra.run.smithi184.stdout:(69/157): openblas-openmp-0.3.26-2.el9.x86_64.r 6.2 MB/s | 4.9 MB 00:00 2024-09-17T04:21:37.758 INFO:teuthology.orchestra.run.smithi184.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-17T04:21:37.825 INFO:teuthology.orchestra.run.smithi184.stdout:(71/157): postgresql-13.16-1.el9.x86_64.rpm 4.8 MB/s | 1.6 MB 00:00 2024-09-17T04:21:37.953 INFO:teuthology.orchestra.run.smithi184.stdout:(72/157): postgresql-contrib-13.16-1.el9.x86_64 2.2 MB/s | 910 kB 00:00 2024-09-17T04:21:38.001 INFO:teuthology.orchestra.run.smithi184.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-09-17T04:21:38.157 INFO:teuthology.orchestra.run.smithi100.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 12 MB/s | 7.4 MB 00:00 2024-09-17T04:21:38.159 INFO:teuthology.orchestra.run.smithi184.stdout:(74/157): protobuf-3.14.0-13.el9.x86_64.rpm 3.0 MB/s | 1.0 MB 00:00 2024-09-17T04:21:38.226 INFO:teuthology.orchestra.run.smithi184.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 917 kB/s | 61 kB 00:00 2024-09-17T04:21:38.309 INFO:teuthology.orchestra.run.smithi184.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-17T04:21:38.316 INFO:teuthology.orchestra.run.smithi100.stdout:(32/157): ceph-mgr-dashboard-19.3.0-4989.gce1b3 18 MB/s | 14 MB 00:00 2024-09-17T04:21:38.341 INFO:teuthology.orchestra.run.smithi100.stdout:(33/157): ceph-mgr-modules-core-19.3.0-4989.gce 1.5 MB/s | 291 kB 00:00 2024-09-17T04:21:38.367 INFO:teuthology.orchestra.run.smithi100.stdout:(34/157): ceph-mgr-rook-19.3.0-4989.gce1b342b.e 985 kB/s | 49 kB 00:00 2024-09-17T04:21:38.392 INFO:teuthology.orchestra.run.smithi100.stdout:(35/157): ceph-prometheus-alerts-19.3.0-4989.gc 329 kB/s | 16 kB 00:00 2024-09-17T04:21:38.417 INFO:teuthology.orchestra.run.smithi100.stdout:(36/157): ceph-volume-19.3.0-4989.gce1b342b.el9 5.6 MB/s | 285 kB 00:00 2024-09-17T04:21:38.459 INFO:teuthology.orchestra.run.smithi100.stdout:(37/157): cephfs-top-19.3.0-4989.gce1b342b.el9. 439 kB/s | 18 kB 00:00 2024-09-17T04:21:38.493 INFO:teuthology.orchestra.run.smithi100.stdout:(38/157): cephadm-19.3.0-4989.gce1b342b.el9.noa 7.6 MB/s | 784 kB 00:00 2024-09-17T04:21:38.527 INFO:teuthology.orchestra.run.smithi184.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 1.1 MB/s | 248 kB 00:00 2024-09-17T04:21:38.601 INFO:teuthology.orchestra.run.smithi100.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 562 kB/s | 80 kB 00:00 2024-09-17T04:21:38.627 INFO:teuthology.orchestra.run.smithi184.stdout:(78/157): postgresql-server-13.16-1.el9.x86_64. 6.7 MB/s | 5.8 MB 00:00 2024-09-17T04:21:38.652 INFO:teuthology.orchestra.run.smithi184.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 204 kB/s | 26 kB 00:00 2024-09-17T04:21:38.677 INFO:teuthology.orchestra.run.smithi184.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-17T04:21:38.728 INFO:teuthology.orchestra.run.smithi184.stdout:(81/157): python3-markupsafe-1.1.1-12.el9.x86_6 696 kB/s | 35 kB 00:00 2024-09-17T04:21:38.753 INFO:teuthology.orchestra.run.smithi184.stdout:(82/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-09-17T04:21:38.802 INFO:teuthology.orchestra.run.smithi100.stdout:(40/157): ceph-osd-19.3.0-4989.gce1b342b.el9.x8 4.0 MB/s | 18 MB 00:04 2024-09-17T04:21:38.827 INFO:teuthology.orchestra.run.smithi100.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 849 kB/s | 191 kB 00:00 2024-09-17T04:21:38.837 INFO:teuthology.orchestra.run.smithi184.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-09-17T04:21:38.892 INFO:teuthology.orchestra.run.smithi100.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.0 MB/s | 801 kB 00:00 2024-09-17T04:21:38.953 INFO:teuthology.orchestra.run.smithi100.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 1.7 MB/s | 106 kB 00:00 2024-09-17T04:21:38.995 INFO:teuthology.orchestra.run.smithi100.stdout:(44/157): python3-cryptography-36.0.1-4.el9.x86 7.3 MB/s | 1.2 MB 00:00 2024-09-17T04:21:39.020 INFO:teuthology.orchestra.run.smithi100.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-17T04:21:39.045 INFO:teuthology.orchestra.run.smithi100.stdout:(46/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 253 kB 00:00 2024-09-17T04:21:39.062 INFO:teuthology.orchestra.run.smithi184.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 987 kB/s | 222 kB 00:00 2024-09-17T04:21:39.071 INFO:teuthology.orchestra.run.smithi100.stdout:(47/157): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-09-17T04:21:39.120 INFO:teuthology.orchestra.run.smithi184.stdout:(85/157): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-17T04:21:39.181 INFO:teuthology.orchestra.run.smithi184.stdout:(86/157): python3-prettytable-0.7.2-27.el9.noar 726 kB/s | 42 kB 00:00 2024-09-17T04:21:39.246 INFO:teuthology.orchestra.run.smithi100.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 952 kB/s | 215 kB 00:00 2024-09-17T04:21:39.296 INFO:teuthology.orchestra.run.smithi184.stdout:(87/157): python3-protobuf-3.14.0-13.el9.noarch 2.3 MB/s | 269 kB 00:00 2024-09-17T04:21:39.372 INFO:teuthology.orchestra.run.smithi184.stdout:(88/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.9 MB/s | 5.1 MB 00:00 2024-09-17T04:21:39.397 INFO:teuthology.orchestra.run.smithi184.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-09-17T04:21:39.447 INFO:teuthology.orchestra.run.smithi184.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-09-17T04:21:39.522 INFO:teuthology.orchestra.run.smithi184.stdout:(91/157): python3-babel-2.9.1-2.el9.noarch.rpm 3.9 MB/s | 6.0 MB 00:01 2024-09-17T04:21:39.548 INFO:teuthology.orchestra.run.smithi184.stdout:(92/157): python3-pytz-2021.1-5.el9.noarch.rpm 339 kB/s | 51 kB 00:00 2024-09-17T04:21:39.573 INFO:teuthology.orchestra.run.smithi184.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 428 kB/s | 54 kB 00:00 2024-09-17T04:21:39.598 INFO:teuthology.orchestra.run.smithi184.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 828 kB/s | 42 kB 00:00 2024-09-17T04:21:39.623 INFO:teuthology.orchestra.run.smithi184.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 885 kB/s | 44 kB 00:00 2024-09-17T04:21:39.674 INFO:teuthology.orchestra.run.smithi184.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-09-17T04:21:39.679 INFO:teuthology.orchestra.run.smithi100.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 247 kB/s | 107 kB 00:00 2024-09-17T04:21:39.707 INFO:teuthology.orchestra.run.smithi184.stdout:(97/157): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-09-17T04:21:39.730 INFO:teuthology.orchestra.run.smithi100.stdout:(50/157): automake-1.16.2-8.el9.noarch.rpm 1.0 MB/s | 689 kB 00:00 2024-09-17T04:21:39.732 INFO:teuthology.orchestra.run.smithi184.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 814 kB/s | 47 kB 00:00 2024-09-17T04:21:39.764 INFO:teuthology.orchestra.run.smithi100.stdout:(51/157): bison-3.7.4-5.el9.x86_64.rpm 1.3 MB/s | 943 kB 00:00 2024-09-17T04:21:39.783 INFO:teuthology.orchestra.run.smithi184.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-17T04:21:39.842 INFO:teuthology.orchestra.run.smithi184.stdout:(100/157): socat-1.7.4.1-6.el9.x86_64.rpm 2.2 MB/s | 304 kB 00:00 2024-09-17T04:21:39.868 INFO:teuthology.orchestra.run.smithi184.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.1 MB/s | 96 kB 00:00 2024-09-17T04:21:39.936 INFO:teuthology.orchestra.run.smithi184.stdout:(102/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 675 kB/s | 45 kB 00:00 2024-09-17T04:21:39.979 INFO:teuthology.orchestra.run.smithi184.stdout:(103/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 476 kB/s | 64 kB 00:00 2024-09-17T04:21:39.980 INFO:teuthology.orchestra.run.smithi100.stdout:(52/157): elfutils-libelf-devel-0.191-4.el9.x86 73 kB/s | 22 kB 00:00 2024-09-17T04:21:40.002 INFO:teuthology.orchestra.run.smithi184.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 334 kB/s | 22 kB 00:00 2024-09-17T04:21:40.051 INFO:teuthology.orchestra.run.smithi100.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 122 kB/s | 33 kB 00:00 2024-09-17T04:21:40.126 INFO:teuthology.orchestra.run.smithi184.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 4.3 MB/s | 548 kB 00:00 2024-09-17T04:21:40.160 INFO:teuthology.orchestra.run.smithi184.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-17T04:21:40.185 INFO:teuthology.orchestra.run.smithi184.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 790 kB/s | 19 kB 00:00 2024-09-17T04:21:40.244 INFO:teuthology.orchestra.run.smithi184.stdout:(108/157): protobuf-compiler-3.14.0-13.el9.x86_ 3.2 MB/s | 863 kB 00:00 2024-09-17T04:21:40.248 INFO:teuthology.orchestra.run.smithi100.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 606 kB/s | 313 kB 00:00 2024-09-17T04:21:40.298 INFO:teuthology.orchestra.run.smithi100.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 68 kB/s | 18 kB 00:00 2024-09-17T04:21:40.302 INFO:teuthology.orchestra.run.smithi184.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 446 kB/s | 26 kB 00:00 2024-09-17T04:21:40.336 INFO:teuthology.orchestra.run.smithi184.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-17T04:21:40.370 INFO:teuthology.orchestra.run.smithi184.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-17T04:21:40.411 INFO:teuthology.orchestra.run.smithi184.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 3.6 MB/s | 151 kB 00:00 2024-09-17T04:21:40.448 INFO:teuthology.orchestra.run.smithi100.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 56 kB/s | 11 kB 00:00 2024-09-17T04:21:40.495 INFO:teuthology.orchestra.run.smithi184.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 839 kB 00:00 2024-09-17T04:21:40.545 INFO:teuthology.orchestra.run.smithi184.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-17T04:21:40.604 INFO:teuthology.orchestra.run.smithi184.stdout:(115/157): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-17T04:21:40.647 INFO:teuthology.orchestra.run.smithi184.stdout:(116/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 430 kB/s | 43 kB 00:00 2024-09-17T04:21:40.680 INFO:teuthology.orchestra.run.smithi184.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 428 kB/s | 32 kB 00:00 2024-09-17T04:21:40.705 INFO:teuthology.orchestra.run.smithi184.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 244 kB/s | 14 kB 00:00 2024-09-17T04:21:40.730 INFO:teuthology.orchestra.run.smithi184.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-09-17T04:21:40.740 INFO:teuthology.orchestra.run.smithi100.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 386 kB/s | 171 kB 00:00 2024-09-17T04:21:40.764 INFO:teuthology.orchestra.run.smithi184.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-17T04:21:40.789 INFO:teuthology.orchestra.run.smithi184.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-17T04:21:40.814 INFO:teuthology.orchestra.run.smithi184.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-17T04:21:40.848 INFO:teuthology.orchestra.run.smithi184.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-17T04:21:40.865 INFO:teuthology.orchestra.run.smithi184.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 652 kB/s | 11 kB 00:00 2024-09-17T04:21:40.882 INFO:teuthology.orchestra.run.smithi184.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-17T04:21:40.890 INFO:teuthology.orchestra.run.smithi100.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 362 kB/s | 160 kB 00:00 2024-09-17T04:21:40.899 INFO:teuthology.orchestra.run.smithi184.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-17T04:21:40.916 INFO:teuthology.orchestra.run.smithi184.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 1.2 MB/s | 19 kB 00:00 2024-09-17T04:21:40.933 INFO:teuthology.orchestra.run.smithi184.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 1.2 MB/s | 20 kB 00:00 2024-09-17T04:21:40.949 INFO:teuthology.orchestra.run.smithi100.stdout:(59/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.1 MB/s | 3.0 MB 00:00 2024-09-17T04:21:40.991 INFO:teuthology.orchestra.run.smithi184.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 10 MB/s | 2.0 MB 00:00 2024-09-17T04:21:41.016 INFO:teuthology.orchestra.run.smithi184.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-17T04:21:41.075 INFO:teuthology.orchestra.run.smithi184.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-09-17T04:21:41.099 INFO:teuthology.orchestra.run.smithi100.stdout:(60/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 126 kB/s | 45 kB 00:00 2024-09-17T04:21:41.109 INFO:teuthology.orchestra.run.smithi184.stdout:(132/157): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-09-17T04:21:41.134 INFO:teuthology.orchestra.run.smithi184.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 986 kB/s | 58 kB 00:00 2024-09-17T04:21:41.159 INFO:teuthology.orchestra.run.smithi184.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 663 kB/s | 16 kB 00:00 2024-09-17T04:21:41.185 INFO:teuthology.orchestra.run.smithi184.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-17T04:21:41.210 INFO:teuthology.orchestra.run.smithi184.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-17T04:21:41.235 INFO:teuthology.orchestra.run.smithi184.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-17T04:21:41.269 INFO:teuthology.orchestra.run.smithi184.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 520 kB/s | 31 kB 00:00 2024-09-17T04:21:41.294 INFO:teuthology.orchestra.run.smithi184.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-17T04:21:41.320 INFO:teuthology.orchestra.run.smithi184.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-09-17T04:21:41.345 INFO:teuthology.orchestra.run.smithi184.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 710 kB/s | 36 kB 00:00 2024-09-17T04:21:41.371 INFO:teuthology.orchestra.run.smithi184.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-17T04:21:41.396 INFO:teuthology.orchestra.run.smithi184.stdout:(143/157): python3-webob-1.8.8-2.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-09-17T04:21:41.421 INFO:teuthology.orchestra.run.smithi184.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-17T04:21:41.471 INFO:teuthology.orchestra.run.smithi184.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 404 kB/s | 20 kB 00:00 2024-09-17T04:21:41.505 INFO:teuthology.orchestra.run.smithi184.stdout:(146/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-09-17T04:21:41.530 INFO:teuthology.orchestra.run.smithi184.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-17T04:21:41.549 INFO:teuthology.orchestra.run.smithi100.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 1.0 MB/s | 662 kB 00:00 2024-09-17T04:21:41.634 INFO:teuthology.orchestra.run.smithi100.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 46 kB/s | 24 kB 00:00 2024-09-17T04:21:41.714 INFO:teuthology.orchestra.run.smithi184.stdout:(148/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.4 MB/s | 16 MB 00:02 2024-09-17T04:21:41.764 INFO:teuthology.orchestra.run.smithi184.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 6.1 MB/s | 1.6 MB 00:00 2024-09-17T04:21:41.933 INFO:teuthology.orchestra.run.smithi100.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 124 kB/s | 47 kB 00:00 2024-09-17T04:21:41.981 INFO:teuthology.orchestra.run.smithi184.stdout:(150/157): librbd1-19.3.0-4989.gce1b342b.el9.x8 12 MB/s | 3.2 MB 00:00 2024-09-17T04:21:42.065 INFO:teuthology.orchestra.run.smithi184.stdout:(151/157): librados2-19.3.0-4989.gce1b342b.el9. 6.3 MB/s | 3.4 MB 00:00 2024-09-17T04:21:42.075 INFO:teuthology.orchestra.run.smithi100.stdout:(64/157): libtool-2.4.6-46.el9.x86_64.rpm 516 kB/s | 581 kB 00:01 2024-09-17T04:21:42.192 INFO:teuthology.orchestra.run.smithi100.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 523 kB/s | 292 kB 00:00 2024-09-17T04:21:42.315 INFO:teuthology.orchestra.run.smithi184.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 164 kB/s | 41 kB 00:00 2024-09-17T04:21:42.359 INFO:teuthology.orchestra.run.smithi100.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 136 kB/s | 38 kB 00:00 2024-09-17T04:21:42.384 INFO:teuthology.orchestra.run.smithi100.stdout:(67/157): lua-5.4.4-4.el9.x86_64.rpm 418 kB/s | 188 kB 00:00 2024-09-17T04:21:42.465 INFO:teuthology.orchestra.run.smithi184.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 1.6 MB/s | 249 kB 00:00 2024-09-17T04:21:42.690 INFO:teuthology.orchestra.run.smithi184.stdout:(154/157): openssl-3.2.2-6.el9.x86_64.rpm 6.0 MB/s | 1.4 MB 00:00 2024-09-17T04:21:42.733 INFO:teuthology.orchestra.run.smithi184.stdout:(155/157): NetworkManager-libnm-1.51.0-1.el9.x8 2.4 MB/s | 1.8 MB 00:00 2024-09-17T04:21:42.874 INFO:teuthology.orchestra.run.smithi184.stdout:(156/157): NetworkManager-1.51.0-1.el9.x86_64.r 2.1 MB/s | 2.3 MB 00:01 2024-09-17T04:21:43.052 INFO:teuthology.orchestra.run.smithi100.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 5.8 MB/s | 4.9 MB 00:00 2024-09-17T04:21:43.244 INFO:teuthology.orchestra.run.smithi100.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 1.8 MB/s | 1.6 MB 00:00 2024-09-17T04:21:43.319 INFO:teuthology.orchestra.run.smithi100.stdout:(70/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 4.6 MB/s | 4.4 MB 00:00 2024-09-17T04:21:43.333 INFO:teuthology.orchestra.run.smithi184.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.7 MB/s | 2.4 MB 00:00 2024-09-17T04:21:43.338 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-17T04:21:43.338 INFO:teuthology.orchestra.run.smithi184.stdout:Total 17 MB/s | 228 MB 00:13 2024-09-17T04:21:43.677 INFO:teuthology.orchestra.run.smithi100.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 318 kB/s | 137 kB 00:00 2024-09-17T04:21:43.761 INFO:teuthology.orchestra.run.smithi100.stdout:(72/157): postgresql-contrib-13.16-1.el9.x86_64 1.3 MB/s | 910 kB 00:00 2024-09-17T04:21:44.111 INFO:teuthology.orchestra.run.smithi100.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 250 kB/s | 87 kB 00:00 2024-09-17T04:21:44.203 INFO:teuthology.orchestra.run.smithi100.stdout:(74/157): postgresql-server-13.16-1.el9.x86_64. 6.5 MB/s | 5.8 MB 00:00 2024-09-17T04:21:44.487 INFO:teuthology.orchestra.run.smithi100.stdout:(75/157): protobuf-3.14.0-13.el9.x86_64.rpm 1.2 MB/s | 1.0 MB 00:00 2024-09-17T04:21:44.545 INFO:teuthology.orchestra.run.smithi100.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 179 kB/s | 61 kB 00:00 2024-09-17T04:21:44.996 INFO:teuthology.orchestra.run.smithi100.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 481 kB/s | 245 kB 00:00 2024-09-17T04:21:45.128 INFO:teuthology.orchestra.run.smithi100.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 426 kB/s | 248 kB 00:00 2024-09-17T04:21:45.213 INFO:teuthology.orchestra.run.smithi100.stdout:(79/157): python3-babel-2.9.1-2.el9.noarch.rpm 5.4 MB/s | 6.0 MB 00:01 2024-09-17T04:21:45.313 INFO:teuthology.orchestra.run.smithi100.stdout:(80/157): python3-jsonpatch-1.21-16.el9.noarch. 81 kB/s | 26 kB 00:00 2024-09-17T04:21:45.401 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-17T04:21:45.421 INFO:teuthology.orchestra.run.smithi100.stdout:(81/157): python3-jsonpointer-2.0-4.el9.noarch. 65 kB/s | 19 kB 00:00 2024-09-17T04:21:45.583 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-17T04:21:45.584 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-17T04:21:45.622 INFO:teuthology.orchestra.run.smithi100.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 113 kB/s | 35 kB 00:00 2024-09-17T04:21:45.672 INFO:teuthology.orchestra.run.smithi100.stdout:(83/157): python3-mako-1.1.4-6.el9.noarch.rpm 375 kB/s | 172 kB 00:00 2024-09-17T04:21:46.172 INFO:teuthology.orchestra.run.smithi100.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 444 kB/s | 222 kB 00:00 2024-09-17T04:21:46.356 INFO:teuthology.orchestra.run.smithi100.stdout:(85/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 341 kB/s | 250 kB 00:00 2024-09-17T04:21:46.531 INFO:teuthology.orchestra.run.smithi100.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 215 kB/s | 77 kB 00:00 2024-09-17T04:21:46.765 INFO:teuthology.orchestra.run.smithi100.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 103 kB/s | 42 kB 00:00 2024-09-17T04:21:46.831 INFO:teuthology.orchestra.run.smithi100.stdout:(88/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.6 MB/s | 5.1 MB 00:01 2024-09-17T04:21:47.057 INFO:teuthology.orchestra.run.smithi100.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 512 kB/s | 269 kB 00:00 2024-09-17T04:21:47.215 INFO:teuthology.orchestra.run.smithi100.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 353 kB/s | 159 kB 00:00 2024-09-17T04:21:47.349 INFO:teuthology.orchestra.run.smithi100.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 539 kB/s | 279 kB 00:00 2024-09-17T04:21:47.408 INFO:teuthology.orchestra.run.smithi100.stdout:(92/157): python3-pytz-2021.1-5.el9.noarch.rpm 145 kB/s | 51 kB 00:00 2024-09-17T04:21:47.558 INFO:teuthology.orchestra.run.smithi100.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 157 kB/s | 54 kB 00:00 2024-09-17T04:21:47.683 INFO:teuthology.orchestra.run.smithi100.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 151 kB/s | 42 kB 00:00 2024-09-17T04:21:47.909 INFO:teuthology.orchestra.run.smithi100.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 126 kB/s | 44 kB 00:00 2024-09-17T04:21:47.944 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-17T04:21:47.945 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-17T04:21:48.176 INFO:teuthology.orchestra.run.smithi100.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 135 kB/s | 36 kB 00:00 2024-09-17T04:21:48.201 INFO:teuthology.orchestra.run.smithi100.stdout:(97/157): qatlib-24.02.0-1.el9.x86_64.rpm 426 kB/s | 221 kB 00:00 2024-09-17T04:21:48.526 INFO:teuthology.orchestra.run.smithi100.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 135 kB/s | 47 kB 00:00 2024-09-17T04:21:48.752 INFO:teuthology.orchestra.run.smithi100.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 553 kB/s | 304 kB 00:00 2024-09-17T04:21:48.902 INFO:teuthology.orchestra.run.smithi100.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 154 kB/s | 58 kB 00:00 2024-09-17T04:21:49.219 INFO:teuthology.orchestra.run.smithi100.stdout:(101/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 8.7 MB/s | 16 MB 00:01 2024-09-17T04:21:49.244 INFO:teuthology.orchestra.run.smithi100.stdout:(102/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 195 kB/s | 96 kB 00:00 2024-09-17T04:21:49.270 INFO:teuthology.orchestra.run.smithi100.stdout:(103/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 173 kB/s | 64 kB 00:00 2024-09-17T04:21:49.294 INFO:teuthology.orchestra.run.smithi100.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-09-17T04:21:49.361 INFO:teuthology.orchestra.run.smithi100.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 8.1 MB/s | 548 kB 00:00 2024-09-17T04:21:49.395 INFO:teuthology.orchestra.run.smithi100.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-17T04:21:49.420 INFO:teuthology.orchestra.run.smithi100.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 784 kB/s | 19 kB 00:00 2024-09-17T04:21:49.462 INFO:teuthology.orchestra.run.smithi100.stdout:(108/157): protobuf-compiler-3.14.0-13.el9.x86_ 4.4 MB/s | 863 kB 00:00 2024-09-17T04:21:49.513 INFO:teuthology.orchestra.run.smithi100.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 522 kB/s | 26 kB 00:00 2024-09-17T04:21:49.537 INFO:teuthology.orchestra.run.smithi100.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-17T04:21:49.571 INFO:teuthology.orchestra.run.smithi100.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-17T04:21:49.597 INFO:teuthology.orchestra.run.smithi100.stdout:(112/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 119 kB/s | 45 kB 00:00 2024-09-17T04:21:49.639 INFO:teuthology.orchestra.run.smithi100.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-09-17T04:21:49.802 INFO:teuthology.orchestra.run.smithi100.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 3.3 MB/s | 548 kB 00:00 2024-09-17T04:21:49.875 INFO:teuthology.orchestra.run.smithi100.stdout:(115/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.0 MB/s | 839 kB 00:00 2024-09-17T04:21:49.933 INFO:teuthology.orchestra.run.smithi100.stdout:(116/157): libarrow-9.0.0-11.el9.x86_64.rpm 8.6 MB/s | 4.4 MB 00:00 2024-09-17T04:21:49.959 INFO:teuthology.orchestra.run.smithi100.stdout:(117/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 277 kB/s | 43 kB 00:00 2024-09-17T04:21:49.984 INFO:teuthology.orchestra.run.smithi100.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 282 kB/s | 14 kB 00:00 2024-09-17T04:21:50.009 INFO:teuthology.orchestra.run.smithi100.stdout:(119/157): python3-cachetools-4.2.4-1.el9.noarc 240 kB/s | 32 kB 00:00 2024-09-17T04:21:50.034 INFO:teuthology.orchestra.run.smithi100.stdout:(120/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-09-17T04:21:50.076 INFO:teuthology.orchestra.run.smithi100.stdout:(121/157): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-09-17T04:21:50.102 INFO:teuthology.orchestra.run.smithi100.stdout:(122/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-09-17T04:21:50.135 INFO:teuthology.orchestra.run.smithi100.stdout:(123/157): python3-google-auth-2.34.0-1.el9.noa 2.1 MB/s | 220 kB 00:00 2024-09-17T04:21:50.161 INFO:teuthology.orchestra.run.smithi100.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 432 kB/s | 11 kB 00:00 2024-09-17T04:21:50.218 INFO:teuthology.orchestra.run.smithi100.stdout:(125/157): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-09-17T04:21:50.245 INFO:teuthology.orchestra.run.smithi100.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 211 kB/s | 18 kB 00:00 2024-09-17T04:21:50.270 INFO:teuthology.orchestra.run.smithi100.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 454 kB/s | 23 kB 00:00 2024-09-17T04:21:50.295 INFO:teuthology.orchestra.run.smithi100.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 389 kB/s | 19 kB 00:00 2024-09-17T04:21:50.320 INFO:teuthology.orchestra.run.smithi100.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-09-17T04:21:50.346 INFO:teuthology.orchestra.run.smithi100.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-17T04:21:50.379 INFO:teuthology.orchestra.run.smithi100.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-17T04:21:50.404 INFO:teuthology.orchestra.run.smithi100.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-17T04:21:50.446 INFO:teuthology.orchestra.run.smithi100.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-09-17T04:21:50.472 INFO:teuthology.orchestra.run.smithi100.stdout:(134/157): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-09-17T04:21:50.497 INFO:teuthology.orchestra.run.smithi100.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-09-17T04:21:50.522 INFO:teuthology.orchestra.run.smithi100.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-17T04:21:50.548 INFO:teuthology.orchestra.run.smithi100.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 566 kB/s | 28 kB 00:00 2024-09-17T04:21:50.573 INFO:teuthology.orchestra.run.smithi100.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 615 kB/s | 31 kB 00:00 2024-09-17T04:21:50.598 INFO:teuthology.orchestra.run.smithi100.stdout:(139/157): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-09-17T04:21:50.624 INFO:teuthology.orchestra.run.smithi100.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-17T04:21:50.649 INFO:teuthology.orchestra.run.smithi100.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-17T04:21:50.691 INFO:teuthology.orchestra.run.smithi100.stdout:(142/157): python3-grpcio-1.46.7-10.el9.x86_64. 3.3 MB/s | 2.0 MB 00:00 2024-09-17T04:21:50.716 INFO:teuthology.orchestra.run.smithi100.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 810 kB/s | 74 kB 00:00 2024-09-17T04:21:50.741 INFO:teuthology.orchestra.run.smithi100.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-09-17T04:21:50.766 INFO:teuthology.orchestra.run.smithi100.stdout:(145/157): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-09-17T04:21:50.792 INFO:teuthology.orchestra.run.smithi100.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 401 kB/s | 20 kB 00:00 2024-09-17T04:21:50.817 INFO:teuthology.orchestra.run.smithi100.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-17T04:21:50.851 INFO:teuthology.orchestra.run.smithi100.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-17T04:21:50.934 INFO:teuthology.orchestra.run.smithi100.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-17T04:21:51.143 INFO:teuthology.orchestra.run.smithi100.stdout:(150/157): librados2-19.3.0-4989.gce1b342b.el9. 10 MB/s | 3.4 MB 00:00 2024-09-17T04:21:51.311 INFO:teuthology.orchestra.run.smithi100.stdout:(151/157): NetworkManager-1.51.0-1.el9.x86_64.r 6.2 MB/s | 2.3 MB 00:00 2024-09-17T04:21:51.377 INFO:teuthology.orchestra.run.smithi100.stdout:(152/157): librbd1-19.3.0-4989.gce1b342b.el9.x8 6.0 MB/s | 3.2 MB 00:00 2024-09-17T04:21:51.402 INFO:teuthology.orchestra.run.smithi100.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 446 kB/s | 41 kB 00:00 2024-09-17T04:21:51.494 INFO:teuthology.orchestra.run.smithi100.stdout:(154/157): NetworkManager-libnm-1.51.0-1.el9.x8 5.2 MB/s | 1.8 MB 00:00 2024-09-17T04:21:51.520 INFO:teuthology.orchestra.run.smithi100.stdout:(155/157): NetworkManager-tui-1.51.0-1.el9.x86_ 1.7 MB/s | 249 kB 00:00 2024-09-17T04:21:51.653 INFO:teuthology.orchestra.run.smithi100.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 5.4 MB/s | 1.4 MB 00:00 2024-09-17T04:21:51.720 INFO:teuthology.orchestra.run.smithi100.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-09-17T04:21:51.725 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-09-17T04:21:51.725 INFO:teuthology.orchestra.run.smithi100.stdout:Total 11 MB/s | 228 MB 00:20 2024-09-17T04:21:53.848 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-09-17T04:21:54.033 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-09-17T04:21:54.033 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-09-17T04:21:54.892 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-17T04:21:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-17T04:21:55.081 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-17T04:21:55.145 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-17T04:21:55.190 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-17T04:21:55.221 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-17T04:21:55.301 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-17T04:21:55.701 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-17T04:21:55.814 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librados2-2:19.3.0-4989.gce1b342b.el9.x86_64 8/165 2024-09-17T04:21:55.950 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:19.3.0-4989.gce1b342b.el9.x86_64 8/165 2024-09-17T04:21:55.990 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs2-2:19.3.0-4989.gce1b342b.el9.x86_64 9/165 2024-09-17T04:21:56.116 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephfs2-2:19.3.0-4989.gce1b342b.el9.x86_64 9/165 2024-09-17T04:21:56.146 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rados-2:19.3.0-4989.gce1b342b.el9.x86_64 10/165 2024-09-17T04:21:56.185 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-17T04:21:56.374 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-09-17T04:21:56.374 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-09-17T04:21:56.395 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-17T04:21:56.461 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-17T04:21:56.511 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephsqlite-2:19.3.0-4989.gce1b342b.el9.x86_64 14/165 2024-09-17T04:21:56.624 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4989.gce1b342b.el9.x86_64 14/165 2024-09-17T04:21:56.654 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libradosstriper1-2:19.3.0-4989.gce1b342b.el9.x86 15/165 2024-09-17T04:21:56.744 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4989.gce1b342b.el9.x86 15/165 2024-09-17T04:21:56.772 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-17T04:21:56.813 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-17T04:21:57.082 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-17T04:21:57.286 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-17T04:21:57.529 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-17T04:21:57.644 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-17T04:21:57.771 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-17T04:21:58.055 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-17T04:21:58.215 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-17T04:21:58.244 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-17T04:21:58.288 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-17T04:21:58.378 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-17T04:21:58.448 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-17T04:21:58.490 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-17T04:21:58.558 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-17T04:21:58.597 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-17T04:21:58.631 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-17T04:21:58.718 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-17T04:21:58.783 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-17T04:21:58.819 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-17T04:21:58.864 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-17T04:21:58.965 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-17T04:21:59.005 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-common-2:19.3.0-4989.gce1b342b.el9. 34/165 2024-09-17T04:21:59.049 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-argparse-2:19.3.0-4989.gce1b342b.el 35/165 2024-09-17T04:21:59.077 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cephfs-2:19.3.0-4989.gce1b342b.el9.x86_6 36/165 2024-09-17T04:21:59.169 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-17T04:21:59.221 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-17T04:21:59.311 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-17T04:21:59.495 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-17T04:21:59.558 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-17T04:22:00.191 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-17T04:22:00.309 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-17T04:22:00.357 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-17T04:22:01.003 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-17T04:22:01.053 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librados-devel-2:19.3.0-4989.gce1b342b.el9.x86_6 46/165 2024-09-17T04:22:01.107 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-17T04:22:01.302 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-17T04:22:01.396 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-17T04:22:01.500 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-17T04:22:01.600 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-17T04:22:01.683 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-17T04:22:01.723 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-17T04:22:01.759 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-17T04:22:01.826 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-17T04:22:01.866 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-17T04:22:01.912 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-17T04:22:01.971 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-17T04:22:02.150 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-17T04:22:02.199 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-17T04:22:02.241 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-17T04:22:02.360 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-17T04:22:02.407 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-17T04:22:02.449 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-17T04:22:02.479 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-17T04:22:02.504 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-17T04:22:02.624 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-17T04:22:02.782 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-17T04:22:02.829 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-17T04:22:02.865 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-17T04:22:02.900 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-17T04:22:02.900 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-09-17T04:22:02.940 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-17T04:22:02.979 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-17T04:22:03.016 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-17T04:22:03.051 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-17T04:22:03.067 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-17T04:22:03.084 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-17T04:22:03.108 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-17T04:22:03.146 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-17T04:22:03.173 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-17T04:22:03.185 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-17T04:22:03.210 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-17T04:22:03.243 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-17T04:22:03.293 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-17T04:22:03.308 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-17T04:22:03.352 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-17T04:22:03.388 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-17T04:22:03.512 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-17T04:22:03.584 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-17T04:22:03.683 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-17T04:22:03.801 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librados2-2:19.3.0-4989.gce1b342b.el9.x86_64 8/165 2024-09-17T04:22:03.935 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:19.3.0-4989.gce1b342b.el9.x86_64 8/165 2024-09-17T04:22:03.986 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs2-2:19.3.0-4989.gce1b342b.el9.x86_64 9/165 2024-09-17T04:22:04.082 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephfs2-2:19.3.0-4989.gce1b342b.el9.x86_64 9/165 2024-09-17T04:22:04.116 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rados-2:19.3.0-4989.gce1b342b.el9.x86_64 10/165 2024-09-17T04:22:04.156 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-17T04:22:04.368 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-17T04:22:04.431 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-17T04:22:04.458 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephsqlite-2:19.3.0-4989.gce1b342b.el9.x86_64 14/165 2024-09-17T04:22:04.470 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-17T04:22:04.521 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-17T04:22:04.555 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-17T04:22:04.578 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4989.gce1b342b.el9.x86_64 14/165 2024-09-17T04:22:04.609 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libradosstriper1-2:19.3.0-4989.gce1b342b.el9.x86 15/165 2024-09-17T04:22:04.669 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-17T04:22:04.690 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4989.gce1b342b.el9.x86 15/165 2024-09-17T04:22:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-17T04:22:04.737 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-17T04:22:04.760 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-17T04:22:04.798 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-17T04:22:04.895 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-17T04:22:04.940 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-17T04:22:04.945 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-17T04:22:05.135 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-17T04:22:05.174 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-17T04:22:05.211 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-17T04:22:05.305 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-17T04:22:05.341 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-17T04:22:05.459 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-17T04:22:05.576 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-17T04:22:05.853 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-17T04:22:05.998 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-17T04:22:06.032 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-17T04:22:06.076 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-17T04:22:06.125 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-17T04:22:06.158 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-17T04:22:06.221 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-17T04:22:06.259 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-17T04:22:06.269 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-17T04:22:06.314 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-17T04:22:06.352 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-17T04:22:06.395 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-17T04:22:06.482 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-17T04:22:06.555 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-17T04:22:06.591 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-17T04:22:06.637 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-17T04:22:06.727 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-17T04:22:06.761 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-common-2:19.3.0-4989.gce1b342b.el9. 34/165 2024-09-17T04:22:06.805 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-argparse-2:19.3.0-4989.gce1b342b.el 35/165 2024-09-17T04:22:06.833 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cephfs-2:19.3.0-4989.gce1b342b.el9.x86_6 36/165 2024-09-17T04:22:06.923 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-17T04:22:06.977 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-17T04:22:07.068 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-17T04:22:07.227 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-17T04:22:07.288 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-17T04:22:07.862 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-17T04:22:07.921 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-17T04:22:07.993 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-17T04:22:08.022 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-17T04:22:08.029 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-17T04:22:08.062 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-17T04:22:08.067 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-17T04:22:08.111 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-17T04:22:08.150 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librados-devel-2:19.3.0-4989.gce1b342b.el9.x86_6 46/165 2024-09-17T04:22:08.203 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-17T04:22:08.376 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-17T04:22:08.423 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-17T04:22:08.444 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-17T04:22:08.465 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librbd1-2:19.3.0-4989.gce1b342b.el9.x86_64 102/165 2024-09-17T04:22:08.549 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-17T04:22:08.578 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:19.3.0-4989.gce1b342b.el9.x86_64 102/165 2024-09-17T04:22:08.613 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rbd-2:19.3.0-4989.gce1b342b.el9.x86_64 103/165 2024-09-17T04:22:08.617 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-17T04:22:08.661 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-17T04:22:08.700 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-17T04:22:08.741 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-17T04:22:08.778 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-17T04:22:08.836 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-17T04:22:08.877 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-17T04:22:08.922 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-17T04:22:08.972 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-17T04:22:09.142 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-17T04:22:09.197 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-17T04:22:09.206 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-17T04:22:09.240 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-17T04:22:09.243 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librgw2-2:19.3.0-4989.gce1b342b.el9.x86_64 106/165 2024-09-17T04:22:09.317 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librgw2-2:19.3.0-4989.gce1b342b.el9.x86_64 106/165 2024-09-17T04:22:09.344 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rgw-2:19.3.0-4989.gce1b342b.el9.x86_64 107/165 2024-09-17T04:22:09.357 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-17T04:22:09.419 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-17T04:22:09.465 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-17T04:22:09.496 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-17T04:22:09.519 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-17T04:22:09.630 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-17T04:22:09.786 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-17T04:22:09.828 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-17T04:22:09.865 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-17T04:22:09.899 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-17T04:22:09.938 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-17T04:22:09.989 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-17T04:22:10.026 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-17T04:22:10.061 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-17T04:22:10.094 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-17T04:22:10.137 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-17T04:22:10.170 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-17T04:22:10.269 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-17T04:22:10.319 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-17T04:22:10.357 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-17T04:22:10.479 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-17T04:22:10.554 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-17T04:22:11.446 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-17T04:22:11.505 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-17T04:22:11.539 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-17T04:22:11.679 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-17T04:22:11.739 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-17T04:22:11.792 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-17T04:22:11.835 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 108/165 2024-09-17T04:22:11.874 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 108/165 2024-09-17T04:22:11.890 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-17T04:22:11.933 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-17T04:22:12.168 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-17T04:22:12.197 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-17T04:22:12.289 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-17T04:22:12.527 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 108/165 2024-09-17T04:22:12.569 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-base-2:19.3.0-4989.gce1b342b.el9.x86_64 109/165 2024-09-17T04:22:12.702 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-base-2:19.3.0-4989.gce1b342b.el9.x86_64 109/165 2024-09-17T04:22:12.703 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T04:22:12.703 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T04:22:12.703 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:22:12.745 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-selinux-2:19.3.0-4989.gce1b342b.el9.x86_64 110/165 2024-09-17T04:22:13.088 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-17T04:22:13.226 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-17T04:22:14.776 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-17T04:22:14.888 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-17T04:22:14.924 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-17T04:22:14.957 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-17T04:22:15.310 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-17T04:22:15.360 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librbd1-2:19.3.0-4989.gce1b342b.el9.x86_64 102/165 2024-09-17T04:22:15.472 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:19.3.0-4989.gce1b342b.el9.x86_64 102/165 2024-09-17T04:22:15.507 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rbd-2:19.3.0-4989.gce1b342b.el9.x86_64 103/165 2024-09-17T04:22:15.555 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-17T04:22:16.091 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-17T04:22:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librgw2-2:19.3.0-4989.gce1b342b.el9.x86_64 106/165 2024-09-17T04:22:16.212 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librgw2-2:19.3.0-4989.gce1b342b.el9.x86_64 106/165 2024-09-17T04:22:16.248 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rgw-2:19.3.0-4989.gce1b342b.el9.x86_64 107/165 2024-09-17T04:22:18.710 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 108/165 2024-09-17T04:22:18.752 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 108/165 2024-09-17T04:22:19.407 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 108/165 2024-09-17T04:22:19.464 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-base-2:19.3.0-4989.gce1b342b.el9.x86_64 109/165 2024-09-17T04:22:19.578 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-base-2:19.3.0-4989.gce1b342b.el9.x86_64 109/165 2024-09-17T04:22:19.578 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T04:22:19.578 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T04:22:19.578 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:22:19.616 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-selinux-2:19.3.0-4989.gce1b342b.el9.x86_64 110/165 2024-09-17T04:22:25.372 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4989.gce1b342b.el9.x86_64 110/165 2024-09-17T04:22:25.373 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /sys 2024-09-17T04:22:25.373 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /proc 2024-09-17T04:22:25.373 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /mnt 2024-09-17T04:22:25.373 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /var/tmp 2024-09-17T04:22:25.373 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /home 2024-09-17T04:22:25.373 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /root 2024-09-17T04:22:25.373 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /tmp 2024-09-17T04:22:25.373 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:22:25.582 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-volume-2:19.3.0-4989.gce1b342b.el9.noarch 111/165 2024-09-17T04:22:25.617 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-volume-2:19.3.0-4989.gce1b342b.el9.noarch 111/165 2024-09-17T04:22:25.618 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:22:25.618 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T04:22:25.618 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:22:27.301 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-osd-2:19.3.0-4989.gce1b342b.el9.x86_64 112/165 2024-09-17T04:22:27.359 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-osd-2:19.3.0-4989.gce1b342b.el9.x86_64 112/165 2024-09-17T04:22:27.360 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:22:27.360 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T04:22:27.360 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T04:22:27.360 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T04:22:27.360 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:22:27.610 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mds-2:19.3.0-4989.gce1b342b.el9.x86_64 113/165 2024-09-17T04:22:27.661 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mds-2:19.3.0-4989.gce1b342b.el9.x86_64 113/165 2024-09-17T04:22:27.662 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:22:27.662 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T04:22:27.662 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T04:22:27.662 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T04:22:27.662 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:22:28.177 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mon-2:19.3.0-4989.gce1b342b.el9.x86_64 114/165 2024-09-17T04:22:28.228 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mon-2:19.3.0-4989.gce1b342b.el9.x86_64 114/165 2024-09-17T04:22:28.228 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:22:28.228 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T04:22:28.229 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T04:22:28.229 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T04:22:28.229 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:22:28.369 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-17T04:22:28.430 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-17T04:22:28.493 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-17T04:22:28.721 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-17T04:22:28.789 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-17T04:22:28.875 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-17T04:22:28.984 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-17T04:22:29.156 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-17T04:22:29.217 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-17T04:22:29.254 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-17T04:22:29.288 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-17T04:22:29.863 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 126/165 2024-09-17T04:22:29.895 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 126/165 2024-09-17T04:22:29.929 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4989.gce1b342b.e 127/165 2024-09-17T04:22:30.032 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4989.gce1b342b. 128/165 2024-09-17T04:22:30.076 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4989.gce1b342b.el9.noa 129/165 2024-09-17T04:22:32.140 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4989.gce1b342b.el9.x86_64 110/165 2024-09-17T04:22:32.141 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /sys 2024-09-17T04:22:32.141 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /proc 2024-09-17T04:22:32.141 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /mnt 2024-09-17T04:22:32.141 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /var/tmp 2024-09-17T04:22:32.141 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /home 2024-09-17T04:22:32.141 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /root 2024-09-17T04:22:32.141 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /tmp 2024-09-17T04:22:32.141 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:22:32.345 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-volume-2:19.3.0-4989.gce1b342b.el9.noarch 111/165 2024-09-17T04:22:32.382 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-volume-2:19.3.0-4989.gce1b342b.el9.noarch 111/165 2024-09-17T04:22:32.383 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:22:32.383 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T04:22:32.383 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:22:33.714 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4989.gce1b342b.el9.noa 129/165 2024-09-17T04:22:33.782 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4989.gce1b342b.el9.n 130/165 2024-09-17T04:22:34.101 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-osd-2:19.3.0-4989.gce1b342b.el9.x86_64 112/165 2024-09-17T04:22:34.156 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-osd-2:19.3.0-4989.gce1b342b.el9.x86_64 112/165 2024-09-17T04:22:34.157 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:22:34.157 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T04:22:34.157 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T04:22:34.157 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T04:22:34.157 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:22:34.407 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mds-2:19.3.0-4989.gce1b342b.el9.x86_64 113/165 2024-09-17T04:22:34.458 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mds-2:19.3.0-4989.gce1b342b.el9.x86_64 113/165 2024-09-17T04:22:34.458 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:22:34.458 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T04:22:34.458 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T04:22:34.458 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T04:22:34.458 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:22:34.862 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4989.gce1b342b.el9.n 130/165 2024-09-17T04:22:34.890 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4989.gce1 131/165 2024-09-17T04:22:34.974 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mon-2:19.3.0-4989.gce1b342b.el9.x86_64 114/165 2024-09-17T04:22:35.025 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mon-2:19.3.0-4989.gce1b342b.el9.x86_64 114/165 2024-09-17T04:22:35.026 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:22:35.026 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T04:22:35.026 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T04:22:35.026 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T04:22:35.026 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:22:35.043 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4989.gce1 131/165 2024-09-17T04:22:35.158 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-17T04:22:35.218 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-17T04:22:35.235 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4989.gce1b342b.el 132/165 2024-09-17T04:22:35.269 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-2:19.3.0-4989.gce1b342b.el9.x86_64 133/165 2024-09-17T04:22:35.308 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-17T04:22:35.319 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4989.gce1b342b.el9.x86_64 133/165 2024-09-17T04:22:35.319 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:22:35.320 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T04:22:35.320 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T04:22:35.320 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T04:22:35.320 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:22:35.402 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-rook-2:19.3.0-4989.gce1b342b.el9.noarch 134/165 2024-09-17T04:22:35.428 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4989.gce1b342b.el9.noarch 134/165 2024-09-17T04:22:35.535 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-17T04:22:35.537 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-2:19.3.0-4989.gce1b342b.el9.x86_64 135/165 2024-09-17T04:22:35.596 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-17T04:22:35.601 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 136/165 2024-09-17T04:22:35.682 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-17T04:22:35.790 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-17T04:22:35.967 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-17T04:22:36.039 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-17T04:22:36.077 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-17T04:22:36.110 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-17T04:22:36.617 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 126/165 2024-09-17T04:22:36.660 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 126/165 2024-09-17T04:22:36.674 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-17T04:22:36.702 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4989.gce1b342b.e 127/165 2024-09-17T04:22:36.721 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-radosgw-2:19.3.0-4989.gce1b342b.el9.x86_64 138/165 2024-09-17T04:22:36.773 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4989.gce1b342b.el9.x86_64 138/165 2024-09-17T04:22:36.774 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:22:36.774 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T04:22:36.774 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T04:22:36.774 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T04:22:36.774 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:22:36.783 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4989.gce1b342b. 128/165 2024-09-17T04:22:36.816 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4989.gce1b342b.el9.noa 129/165 2024-09-17T04:22:36.838 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4989.gce1b3 139/165 2024-09-17T04:22:36.888 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4989.gce1b3 139/165 2024-09-17T04:22:36.889 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:22:36.889 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T04:22:36.889 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:22:36.947 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 140/165 2024-09-17T04:22:36.998 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 140/165 2024-09-17T04:22:36.999 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:22:36.999 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-17T04:22:36.999 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-17T04:22:36.999 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-17T04:22:36.999 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:22:37.348 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 141/165 2024-09-17T04:22:37.399 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 141/165 2024-09-17T04:22:37.399 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:22:37.399 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T04:22:37.399 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T04:22:37.399 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T04:22:37.400 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:22:40.495 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4989.gce1b342b.el9.noa 129/165 2024-09-17T04:22:40.555 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4989.gce1b342b.el9.n 130/165 2024-09-17T04:22:41.639 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4989.gce1b342b.el9.n 130/165 2024-09-17T04:22:41.663 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4989.gce1 131/165 2024-09-17T04:22:41.753 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-test-2:19.3.0-4989.gce1b342b.el9.x86_64 142/165 2024-09-17T04:22:41.794 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 143/165 2024-09-17T04:22:41.817 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4989.gce1 131/165 2024-09-17T04:22:41.837 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-nbd-2:19.3.0-4989.gce1b342b.el9.x86_64 144/165 2024-09-17T04:22:41.867 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-17T04:22:41.902 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-17T04:22:42.000 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4989.gce1b342b.el 132/165 2024-09-17T04:22:42.032 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-2:19.3.0-4989.gce1b342b.el9.x86_64 133/165 2024-09-17T04:22:42.084 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4989.gce1b342b.el9.x86_64 133/165 2024-09-17T04:22:42.084 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:22:42.084 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T04:22:42.084 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T04:22:42.084 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T04:22:42.084 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:22:42.141 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-rook-2:19.3.0-4989.gce1b342b.el9.noarch 134/165 2024-09-17T04:22:42.151 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-shell-2:19.3.0-4989.gce1b342b.el9.x86_64 147/165 2024-09-17T04:22:42.167 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4989.gce1b342b.el9.noarch 134/165 2024-09-17T04:22:42.207 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-17T04:22:42.236 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs-devel-2:19.3.0-4989.gce1b342b.el9.x86_ 149/165 2024-09-17T04:22:42.268 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-17T04:22:42.277 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-2:19.3.0-4989.gce1b342b.el9.x86_64 135/165 2024-09-17T04:22:42.303 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-17T04:22:42.345 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 136/165 2024-09-17T04:22:43.075 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-17T04:22:43.126 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-17T04:22:43.168 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-17T04:22:43.400 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-17T04:22:43.427 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-radosgw-2:19.3.0-4989.gce1b342b.el9.x86_64 138/165 2024-09-17T04:22:43.478 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4989.gce1b342b.el9.x86_64 138/165 2024-09-17T04:22:43.479 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:22:43.479 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T04:22:43.479 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T04:22:43.479 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T04:22:43.479 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:22:43.528 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4989.gce1b3 139/165 2024-09-17T04:22:43.578 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4989.gce1b3 139/165 2024-09-17T04:22:43.578 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:22:43.579 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T04:22:43.579 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:22:43.629 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 140/165 2024-09-17T04:22:43.679 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 140/165 2024-09-17T04:22:43.679 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:22:43.680 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-17T04:22:43.680 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-17T04:22:43.680 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-17T04:22:43.680 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:22:44.038 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 141/165 2024-09-17T04:22:44.088 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 141/165 2024-09-17T04:22:44.089 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:22:44.089 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T04:22:44.089 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T04:22:44.089 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T04:22:44.089 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:22:44.458 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-top-2:19.3.0-4989.gce1b342b.el9.noarch 153/165 2024-09-17T04:22:44.575 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-17T04:22:44.628 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-17T04:22:44.737 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-17T04:22:44.777 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-17T04:22:44.805 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-17T04:22:44.864 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-17T04:22:44.864 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-17T04:22:44.979 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-17T04:22:44.979 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-17T04:22:45.088 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-17T04:22:45.113 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-17T04:22:45.128 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-17T04:22:45.129 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-17T04:22:45.382 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-17T04:22:45.382 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-17T04:22:45.491 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-17T04:22:45.565 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-17T04:22:45.803 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-17T04:22:45.804 INFO:teuthology.orchestra.run.smithi184.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-17T04:22:45.804 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:22:48.485 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-test-2:19.3.0-4989.gce1b342b.el9.x86_64 142/165 2024-09-17T04:22:48.526 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 143/165 2024-09-17T04:22:48.569 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-nbd-2:19.3.0-4989.gce1b342b.el9.x86_64 144/165 2024-09-17T04:22:48.598 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-17T04:22:48.642 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-17T04:22:48.893 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-shell-2:19.3.0-4989.gce1b342b.el9.x86_64 147/165 2024-09-17T04:22:48.947 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-17T04:22:48.976 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs-devel-2:19.3.0-4989.gce1b342b.el9.x86_ 149/165 2024-09-17T04:22:49.016 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-17T04:22:49.094 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-17T04:22:49.835 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-17T04:22:49.884 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-17T04:22:49.925 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-17T04:22:51.219 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-top-2:19.3.0-4989.gce1b342b.el9.noarch 153/165 2024-09-17T04:22:51.324 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-17T04:22:51.376 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-17T04:22:51.480 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-17T04:22:51.518 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-17T04:22:51.543 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-17T04:22:51.605 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-17T04:22:51.605 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-17T04:22:51.712 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-17T04:22:51.713 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-17T04:22:51.830 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-17T04:22:51.854 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-17T04:22:51.870 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-17T04:22:51.870 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-17T04:22:52.107 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-17T04:22:52.107 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-17T04:22:52.207 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-2:19.3.0-4989.gce1b342b.el9.x86_64 1/165 2024-09-17T04:22:52.207 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-base-2:19.3.0-4989.gce1b342b.el9.x86_64 2/165 2024-09-17T04:22:52.207 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 3/165 2024-09-17T04:22:52.207 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 4/165 2024-09-17T04:22:52.207 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4989.gce1b3 5/165 2024-09-17T04:22:52.207 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mds-2:19.3.0-4989.gce1b342b.el9.x86_64 6/165 2024-09-17T04:22:52.207 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-2:19.3.0-4989.gce1b342b.el9.x86_64 7/165 2024-09-17T04:22:52.208 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mon-2:19.3.0-4989.gce1b342b.el9.x86_64 8/165 2024-09-17T04:22:52.208 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-osd-2:19.3.0-4989.gce1b342b.el9.x86_64 9/165 2024-09-17T04:22:52.208 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-radosgw-2:19.3.0-4989.gce1b342b.el9.x86_64 10/165 2024-09-17T04:22:52.208 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-selinux-2:19.3.0-4989.gce1b342b.el9.x86_64 11/165 2024-09-17T04:22:52.208 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-test-2:19.3.0-4989.gce1b342b.el9.x86_64 12/165 2024-09-17T04:22:52.208 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 13/165 2024-09-17T04:22:52.208 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-shell-2:19.3.0-4989.gce1b342b.el9.x86_64 14/165 2024-09-17T04:22:52.208 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs-devel-2:19.3.0-4989.gce1b342b.el9.x86_ 15/165 2024-09-17T04:22:52.208 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs2-2:19.3.0-4989.gce1b342b.el9.x86_64 16/165 2024-09-17T04:22:52.208 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephsqlite-2:19.3.0-4989.gce1b342b.el9.x86_64 17/165 2024-09-17T04:22:52.208 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados-devel-2:19.3.0-4989.gce1b342b.el9.x86_6 18/165 2024-09-17T04:22:52.208 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libradosstriper1-2:19.3.0-4989.gce1b342b.el9.x86 19/165 2024-09-17T04:22:52.209 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librgw2-2:19.3.0-4989.gce1b342b.el9.x86_64 20/165 2024-09-17T04:22:52.209 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4989.gce1b342b.el 21/165 2024-09-17T04:22:52.209 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-common-2:19.3.0-4989.gce1b342b.el9. 22/165 2024-09-17T04:22:52.209 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cephfs-2:19.3.0-4989.gce1b342b.el9.x86_6 23/165 2024-09-17T04:22:52.209 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rados-2:19.3.0-4989.gce1b342b.el9.x86_64 24/165 2024-09-17T04:22:52.209 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rbd-2:19.3.0-4989.gce1b342b.el9.x86_64 25/165 2024-09-17T04:22:52.209 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rgw-2:19.3.0-4989.gce1b342b.el9.x86_64 26/165 2024-09-17T04:22:52.209 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 27/165 2024-09-17T04:22:52.225 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-17T04:22:52.230 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 28/165 2024-09-17T04:22:52.230 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-nbd-2:19.3.0-4989.gce1b342b.el9.x86_64 29/165 2024-09-17T04:22:52.230 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4989.gce1b342b. 30/165 2024-09-17T04:22:52.230 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4989.gce1b342b.el9.noa 31/165 2024-09-17T04:22:52.230 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4989.gce1b342b.el9.n 32/165 2024-09-17T04:22:52.231 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4989.gce1 33/165 2024-09-17T04:22:52.231 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4989.gce1b342b.el 34/165 2024-09-17T04:22:52.231 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4989.gce1b342b.el9.noarch 35/165 2024-09-17T04:22:52.231 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4989.gce1b342b.e 36/165 2024-09-17T04:22:52.231 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-volume-2:19.3.0-4989.gce1b342b.el9.noarch 37/165 2024-09-17T04:22:52.231 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 38/165 2024-09-17T04:22:52.231 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-top-2:19.3.0-4989.gce1b342b.el9.noarch 39/165 2024-09-17T04:22:52.231 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-17T04:22:52.231 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-17T04:22:52.231 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-17T04:22:52.231 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-17T04:22:52.231 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-17T04:22:52.231 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-17T04:22:52.231 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-17T04:22:52.232 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-17T04:22:52.232 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-17T04:22:52.232 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-17T04:22:52.232 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-17T04:22:52.232 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-17T04:22:52.232 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-17T04:22:52.232 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-17T04:22:52.232 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-17T04:22:52.232 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-17T04:22:52.232 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-17T04:22:52.232 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-17T04:22:52.232 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-17T04:22:52.232 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-17T04:22:52.233 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-17T04:22:52.233 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-17T04:22:52.233 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-17T04:22:52.233 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-17T04:22:52.233 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-17T04:22:52.233 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-17T04:22:52.233 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-17T04:22:52.233 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-17T04:22:52.233 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-17T04:22:52.233 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-17T04:22:52.233 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-17T04:22:52.233 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-17T04:22:52.233 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-17T04:22:52.234 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-17T04:22:52.234 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-17T04:22:52.234 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-17T04:22:52.234 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-17T04:22:52.234 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-17T04:22:52.234 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-17T04:22:52.234 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-17T04:22:52.234 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-17T04:22:52.234 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-17T04:22:52.234 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-17T04:22:52.234 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-17T04:22:52.234 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-17T04:22:52.234 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-17T04:22:52.235 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-17T04:22:52.235 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-17T04:22:52.235 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-17T04:22:52.235 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-17T04:22:52.235 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-17T04:22:52.235 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-17T04:22:52.235 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-17T04:22:52.235 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-17T04:22:52.235 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-17T04:22:52.235 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-17T04:22:52.235 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-17T04:22:52.235 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-17T04:22:52.235 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-17T04:22:52.236 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-17T04:22:52.236 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-17T04:22:52.236 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-17T04:22:52.236 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-17T04:22:52.236 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-17T04:22:52.236 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-17T04:22:52.236 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-17T04:22:52.236 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-17T04:22:52.236 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-17T04:22:52.236 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-17T04:22:52.236 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-17T04:22:52.236 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-17T04:22:52.236 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-17T04:22:52.237 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-17T04:22:52.237 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-17T04:22:52.237 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-17T04:22:52.237 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-17T04:22:52.237 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-17T04:22:52.237 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-17T04:22:52.237 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-17T04:22:52.237 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-17T04:22:52.237 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-17T04:22:52.237 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-17T04:22:52.237 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-17T04:22:52.237 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-17T04:22:52.237 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-17T04:22:52.238 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-17T04:22:52.238 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-17T04:22:52.238 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-17T04:22:52.238 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-17T04:22:52.238 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-17T04:22:52.238 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-17T04:22:52.238 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-17T04:22:52.238 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-17T04:22:52.238 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-17T04:22:52.238 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-17T04:22:52.238 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-17T04:22:52.238 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-17T04:22:52.238 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-17T04:22:52.238 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-17T04:22:52.239 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-17T04:22:52.239 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-17T04:22:52.239 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-17T04:22:52.239 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-17T04:22:52.239 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-17T04:22:52.239 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-17T04:22:52.239 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-17T04:22:52.239 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-17T04:22:52.239 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-17T04:22:52.239 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-17T04:22:52.239 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-17T04:22:52.239 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:19.3.0-4989.gce1b342b.el9.x86_64 150/165 2024-09-17T04:22:52.239 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-17T04:22:52.239 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:19.3.0-4989.gce1b342b.el9.x86_64 152/165 2024-09-17T04:22:52.240 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-17T04:22:52.240 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-17T04:22:52.240 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-17T04:22:52.240 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-17T04:22:52.240 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-17T04:22:52.240 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-17T04:22:52.240 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-17T04:22:52.240 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-17T04:22:52.240 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-17T04:22:52.240 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-17T04:22:52.240 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-17T04:22:52.240 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-17T04:22:52.299 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-17T04:22:52.502 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-17T04:22:52.503 INFO:teuthology.orchestra.run.smithi100.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-17T04:22:52.503 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:22:54.546 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-17T04:22:54.546 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:22:54.546 INFO:teuthology.orchestra.run.smithi184.stdout:Upgraded: 2024-09-17T04:22:54.546 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-17T04:22:54.546 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-17T04:22:54.546 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-17T04:22:54.546 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-17T04:22:54.546 INFO:teuthology.orchestra.run.smithi184.stdout: librados2-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.546 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.547 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-17T04:22:54.547 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-17T04:22:54.547 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-17T04:22:54.547 INFO:teuthology.orchestra.run.smithi184.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-17T04:22:54.547 INFO:teuthology.orchestra.run.smithi184.stdout: automake-1.16.2-8.el9.noarch 2024-09-17T04:22:54.547 INFO:teuthology.orchestra.run.smithi184.stdout: bison-3.7.4-5.el9.x86_64 2024-09-17T04:22:54.547 INFO:teuthology.orchestra.run.smithi184.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T04:22:54.547 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.547 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.547 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.547 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.547 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:22:54.547 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.548 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.548 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.548 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:22:54.548 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:22:54.548 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:22:54.548 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:22:54.548 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:22:54.548 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.548 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.548 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:22:54.548 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.548 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.549 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.549 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:22:54.549 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:22:54.549 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.549 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-shell-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.549 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-top-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:22:54.549 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-17T04:22:54.549 INFO:teuthology.orchestra.run.smithi184.stdout: flex-2.6.4-9.el9.x86_64 2024-09-17T04:22:54.549 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T04:22:54.549 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T04:22:54.549 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T04:22:54.549 INFO:teuthology.orchestra.run.smithi184.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-17T04:22:54.549 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T04:22:54.550 INFO:teuthology.orchestra.run.smithi184.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-17T04:22:54.550 INFO:teuthology.orchestra.run.smithi184.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-17T04:22:54.550 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T04:22:54.550 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T04:22:54.550 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.550 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.550 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.550 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T04:22:54.550 INFO:teuthology.orchestra.run.smithi184.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-17T04:22:54.550 INFO:teuthology.orchestra.run.smithi184.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T04:22:54.550 INFO:teuthology.orchestra.run.smithi184.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T04:22:54.550 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T04:22:54.550 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T04:22:54.550 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.551 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.551 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T04:22:54.551 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.551 INFO:teuthology.orchestra.run.smithi184.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-17T04:22:54.551 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T04:22:54.551 INFO:teuthology.orchestra.run.smithi184.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-17T04:22:54.551 INFO:teuthology.orchestra.run.smithi184.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-17T04:22:54.551 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T04:22:54.551 INFO:teuthology.orchestra.run.smithi184.stdout: lua-5.4.4-4.el9.x86_64 2024-09-17T04:22:54.551 INFO:teuthology.orchestra.run.smithi184.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-17T04:22:54.551 INFO:teuthology.orchestra.run.smithi184.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-17T04:22:54.551 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T04:22:54.551 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T04:22:54.552 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-17T04:22:54.552 INFO:teuthology.orchestra.run.smithi184.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T04:22:54.552 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-13.16-1.el9.x86_64 2024-09-17T04:22:54.552 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-17T04:22:54.552 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-17T04:22:54.552 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-17T04:22:54.552 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-17T04:22:54.552 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-17T04:22:54.552 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T04:22:54.552 INFO:teuthology.orchestra.run.smithi184.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-17T04:22:54.552 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T04:22:54.552 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T04:22:54.552 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T04:22:54.552 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.553 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.553 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.553 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T04:22:54.553 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T04:22:54.553 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T04:22:54.553 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T04:22:54.553 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-17T04:22:54.553 INFO:teuthology.orchestra.run.smithi184.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-17T04:22:54.553 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T04:22:54.553 INFO:teuthology.orchestra.run.smithi184.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T04:22:54.553 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T04:22:54.553 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-17T04:22:54.553 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-17T04:22:54.554 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T04:22:54.554 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T04:22:54.554 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T04:22:54.554 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T04:22:54.554 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T04:22:54.554 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T04:22:54.554 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T04:22:54.554 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T04:22:54.554 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T04:22:54.554 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T04:22:54.554 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T04:22:54.554 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T04:22:54.554 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T04:22:54.555 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T04:22:54.555 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T04:22:54.555 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T04:22:54.555 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T04:22:54.555 INFO:teuthology.orchestra.run.smithi184.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T04:22:54.555 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T04:22:54.555 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T04:22:54.555 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T04:22:54.555 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T04:22:54.555 INFO:teuthology.orchestra.run.smithi184.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-17T04:22:54.555 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T04:22:54.555 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T04:22:54.555 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T04:22:54.556 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T04:22:54.556 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-17T04:22:54.556 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T04:22:54.556 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.556 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.556 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T04:22:54.556 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T04:22:54.556 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T04:22:54.556 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.556 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T04:22:54.556 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T04:22:54.556 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T04:22:54.557 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T04:22:54.557 INFO:teuthology.orchestra.run.smithi184.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T04:22:54.557 INFO:teuthology.orchestra.run.smithi184.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T04:22:54.557 INFO:teuthology.orchestra.run.smithi184.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T04:22:54.557 INFO:teuthology.orchestra.run.smithi184.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-17T04:22:54.557 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T04:22:54.557 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T04:22:54.557 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T04:22:54.557 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T04:22:54.557 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-17T04:22:54.557 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-17T04:22:54.557 INFO:teuthology.orchestra.run.smithi184.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-17T04:22:54.557 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.558 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.558 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:22:54.558 INFO:teuthology.orchestra.run.smithi184.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T04:22:54.558 INFO:teuthology.orchestra.run.smithi184.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T04:22:54.558 INFO:teuthology.orchestra.run.smithi184.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T04:22:54.558 INFO:teuthology.orchestra.run.smithi184.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-17T04:22:54.558 INFO:teuthology.orchestra.run.smithi184.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-17T04:22:54.558 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T04:22:54.558 INFO:teuthology.orchestra.run.smithi184.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-17T04:22:54.559 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:22:54.559 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-17T04:22:55.201 DEBUG:teuthology.parallel:result is None 2024-09-17T04:22:58.685 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-2:19.3.0-4989.gce1b342b.el9.x86_64 1/165 2024-09-17T04:22:58.685 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-base-2:19.3.0-4989.gce1b342b.el9.x86_64 2/165 2024-09-17T04:22:58.685 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 3/165 2024-09-17T04:22:58.686 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 4/165 2024-09-17T04:22:58.686 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4989.gce1b3 5/165 2024-09-17T04:22:58.686 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mds-2:19.3.0-4989.gce1b342b.el9.x86_64 6/165 2024-09-17T04:22:58.686 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-2:19.3.0-4989.gce1b342b.el9.x86_64 7/165 2024-09-17T04:22:58.686 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mon-2:19.3.0-4989.gce1b342b.el9.x86_64 8/165 2024-09-17T04:22:58.686 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-osd-2:19.3.0-4989.gce1b342b.el9.x86_64 9/165 2024-09-17T04:22:58.686 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-radosgw-2:19.3.0-4989.gce1b342b.el9.x86_64 10/165 2024-09-17T04:22:58.686 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-selinux-2:19.3.0-4989.gce1b342b.el9.x86_64 11/165 2024-09-17T04:22:58.686 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-test-2:19.3.0-4989.gce1b342b.el9.x86_64 12/165 2024-09-17T04:22:58.686 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 13/165 2024-09-17T04:22:58.686 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-shell-2:19.3.0-4989.gce1b342b.el9.x86_64 14/165 2024-09-17T04:22:58.686 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs-devel-2:19.3.0-4989.gce1b342b.el9.x86_ 15/165 2024-09-17T04:22:58.686 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs2-2:19.3.0-4989.gce1b342b.el9.x86_64 16/165 2024-09-17T04:22:58.686 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephsqlite-2:19.3.0-4989.gce1b342b.el9.x86_64 17/165 2024-09-17T04:22:58.687 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados-devel-2:19.3.0-4989.gce1b342b.el9.x86_6 18/165 2024-09-17T04:22:58.687 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libradosstriper1-2:19.3.0-4989.gce1b342b.el9.x86 19/165 2024-09-17T04:22:58.687 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librgw2-2:19.3.0-4989.gce1b342b.el9.x86_64 20/165 2024-09-17T04:22:58.687 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4989.gce1b342b.el 21/165 2024-09-17T04:22:58.687 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-common-2:19.3.0-4989.gce1b342b.el9. 22/165 2024-09-17T04:22:58.687 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cephfs-2:19.3.0-4989.gce1b342b.el9.x86_6 23/165 2024-09-17T04:22:58.687 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rados-2:19.3.0-4989.gce1b342b.el9.x86_64 24/165 2024-09-17T04:22:58.687 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rbd-2:19.3.0-4989.gce1b342b.el9.x86_64 25/165 2024-09-17T04:22:58.699 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rgw-2:19.3.0-4989.gce1b342b.el9.x86_64 26/165 2024-09-17T04:22:58.700 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 27/165 2024-09-17T04:22:58.700 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 28/165 2024-09-17T04:22:58.700 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-nbd-2:19.3.0-4989.gce1b342b.el9.x86_64 29/165 2024-09-17T04:22:58.700 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4989.gce1b342b. 30/165 2024-09-17T04:22:58.700 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4989.gce1b342b.el9.noa 31/165 2024-09-17T04:22:58.700 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4989.gce1b342b.el9.n 32/165 2024-09-17T04:22:58.700 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4989.gce1 33/165 2024-09-17T04:22:58.700 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4989.gce1b342b.el 34/165 2024-09-17T04:22:58.700 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4989.gce1b342b.el9.noarch 35/165 2024-09-17T04:22:58.700 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4989.gce1b342b.e 36/165 2024-09-17T04:22:58.700 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-volume-2:19.3.0-4989.gce1b342b.el9.noarch 37/165 2024-09-17T04:22:58.701 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 38/165 2024-09-17T04:22:58.701 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-top-2:19.3.0-4989.gce1b342b.el9.noarch 39/165 2024-09-17T04:22:58.701 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-17T04:22:58.701 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-17T04:22:58.701 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-17T04:22:58.701 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-17T04:22:58.701 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-17T04:22:58.701 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-17T04:22:58.701 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-17T04:22:58.701 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-17T04:22:58.701 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-17T04:22:58.701 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-17T04:22:58.701 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-17T04:22:58.701 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-17T04:22:58.702 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-17T04:22:58.702 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-17T04:22:58.702 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-17T04:22:58.702 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-17T04:22:58.702 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-17T04:22:58.702 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-17T04:22:58.702 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-17T04:22:58.702 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-17T04:22:58.702 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-17T04:22:58.702 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-17T04:22:58.702 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-17T04:22:58.702 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-17T04:22:58.703 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-17T04:22:58.703 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-17T04:22:58.703 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-17T04:22:58.703 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-17T04:22:58.703 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-17T04:22:58.703 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-17T04:22:58.703 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-17T04:22:58.703 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-17T04:22:58.703 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-17T04:22:58.703 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-17T04:22:58.703 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-17T04:22:58.703 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-17T04:22:58.703 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-17T04:22:58.704 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-17T04:22:58.704 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-17T04:22:58.704 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-17T04:22:58.704 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-17T04:22:58.704 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-17T04:22:58.704 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-17T04:22:58.704 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-17T04:22:58.704 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-17T04:22:58.704 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-17T04:22:58.704 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-17T04:22:58.704 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-17T04:22:58.704 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-17T04:22:58.705 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-17T04:22:58.705 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-17T04:22:58.705 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-17T04:22:58.705 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-17T04:22:58.705 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-17T04:22:58.705 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-17T04:22:58.705 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-17T04:22:58.705 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-17T04:22:58.705 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-17T04:22:58.705 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-17T04:22:58.705 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-17T04:22:58.705 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-17T04:22:58.706 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-17T04:22:58.706 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-17T04:22:58.706 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-17T04:22:58.706 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-17T04:22:58.706 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-17T04:22:58.706 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-17T04:22:58.706 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-17T04:22:58.706 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-17T04:22:58.706 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-17T04:22:58.706 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-17T04:22:58.706 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-17T04:22:58.707 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-17T04:22:58.707 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-17T04:22:58.707 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-17T04:22:58.707 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-17T04:22:58.707 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-17T04:22:58.707 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-17T04:22:58.707 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-17T04:22:58.707 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-17T04:22:58.707 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-17T04:22:58.707 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-17T04:22:58.707 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-17T04:22:58.707 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-17T04:22:58.708 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-17T04:22:58.708 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-17T04:22:58.708 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-17T04:22:58.708 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-17T04:22:58.708 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-17T04:22:58.708 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-17T04:22:58.708 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-17T04:22:58.708 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-17T04:22:58.708 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-17T04:22:58.708 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-17T04:22:58.708 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-17T04:22:58.709 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-17T04:22:58.709 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-17T04:22:58.709 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-17T04:22:58.709 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-17T04:22:58.709 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-17T04:22:58.709 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-17T04:22:58.709 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-17T04:22:58.709 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-17T04:22:58.709 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-17T04:22:58.709 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-17T04:22:58.709 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-17T04:22:58.709 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-17T04:22:58.710 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-17T04:22:58.710 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-17T04:22:58.710 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-17T04:22:58.710 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:19.3.0-4989.gce1b342b.el9.x86_64 150/165 2024-09-17T04:22:58.710 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-17T04:22:58.710 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:19.3.0-4989.gce1b342b.el9.x86_64 152/165 2024-09-17T04:22:58.710 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-17T04:22:58.710 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-17T04:22:58.710 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-17T04:22:58.710 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-17T04:22:58.710 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-17T04:22:58.710 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-17T04:22:58.710 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-17T04:22:58.710 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-17T04:22:58.710 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-17T04:22:58.711 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-17T04:22:58.711 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-17T04:22:58.711 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-17T04:23:00.840 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-17T04:23:00.841 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:00.841 INFO:teuthology.orchestra.run.smithi100.stdout:Upgraded: 2024-09-17T04:23:00.841 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-17T04:23:00.841 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-17T04:23:00.841 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-17T04:23:00.841 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-17T04:23:00.841 INFO:teuthology.orchestra.run.smithi100.stdout: librados2-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.841 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.841 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-17T04:23:00.841 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-17T04:23:00.841 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-09-17T04:23:00.842 INFO:teuthology.orchestra.run.smithi100.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-17T04:23:00.842 INFO:teuthology.orchestra.run.smithi100.stdout: automake-1.16.2-8.el9.noarch 2024-09-17T04:23:00.842 INFO:teuthology.orchestra.run.smithi100.stdout: bison-3.7.4-5.el9.x86_64 2024-09-17T04:23:00.842 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T04:23:00.842 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.842 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.842 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.842 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.842 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:23:00.842 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.842 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.842 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.843 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:23:00.843 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:23:00.843 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:23:00.843 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:23:00.843 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:23:00.843 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.843 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.843 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:23:00.843 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.843 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.843 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.843 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:23:00.843 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:23:00.844 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.844 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-shell-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.844 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:23:00.844 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-17T04:23:00.844 INFO:teuthology.orchestra.run.smithi100.stdout: flex-2.6.4-9.el9.x86_64 2024-09-17T04:23:00.844 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T04:23:00.844 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T04:23:00.844 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T04:23:00.844 INFO:teuthology.orchestra.run.smithi100.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-17T04:23:00.844 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T04:23:00.844 INFO:teuthology.orchestra.run.smithi100.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-17T04:23:00.844 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-17T04:23:00.844 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T04:23:00.845 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T04:23:00.845 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.845 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.845 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.845 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T04:23:00.845 INFO:teuthology.orchestra.run.smithi100.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-17T04:23:00.845 INFO:teuthology.orchestra.run.smithi100.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T04:23:00.845 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T04:23:00.845 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T04:23:00.845 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T04:23:00.845 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.845 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.845 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T04:23:00.846 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.846 INFO:teuthology.orchestra.run.smithi100.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-17T04:23:00.846 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T04:23:00.846 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-17T04:23:00.846 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-17T04:23:00.846 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T04:23:00.846 INFO:teuthology.orchestra.run.smithi100.stdout: lua-5.4.4-4.el9.x86_64 2024-09-17T04:23:00.846 INFO:teuthology.orchestra.run.smithi100.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-17T04:23:00.846 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-17T04:23:00.846 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T04:23:00.846 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T04:23:00.846 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-17T04:23:00.846 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T04:23:00.846 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-13.16-1.el9.x86_64 2024-09-17T04:23:00.847 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-17T04:23:00.847 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-17T04:23:00.847 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-17T04:23:00.847 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-17T04:23:00.847 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-17T04:23:00.847 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T04:23:00.847 INFO:teuthology.orchestra.run.smithi100.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-17T04:23:00.847 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T04:23:00.847 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T04:23:00.847 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T04:23:00.847 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.847 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.847 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.848 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T04:23:00.848 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T04:23:00.848 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T04:23:00.848 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T04:23:00.848 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-17T04:23:00.848 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-17T04:23:00.848 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T04:23:00.848 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T04:23:00.848 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T04:23:00.848 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-17T04:23:00.848 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-17T04:23:00.848 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T04:23:00.848 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T04:23:00.848 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T04:23:00.849 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T04:23:00.849 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T04:23:00.849 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T04:23:00.849 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T04:23:00.849 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T04:23:00.849 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T04:23:00.849 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T04:23:00.849 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T04:23:00.849 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T04:23:00.849 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T04:23:00.849 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T04:23:00.849 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T04:23:00.849 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T04:23:00.850 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T04:23:00.850 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T04:23:00.850 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T04:23:00.850 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T04:23:00.850 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T04:23:00.850 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T04:23:00.850 INFO:teuthology.orchestra.run.smithi100.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-17T04:23:00.850 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T04:23:00.850 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T04:23:00.850 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T04:23:00.850 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T04:23:00.850 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-17T04:23:00.850 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T04:23:00.850 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.851 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.851 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T04:23:00.851 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T04:23:00.851 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T04:23:00.851 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.851 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T04:23:00.851 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T04:23:00.851 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T04:23:00.851 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T04:23:00.851 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T04:23:00.851 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T04:23:00.851 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T04:23:00.851 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-17T04:23:00.851 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T04:23:00.852 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T04:23:00.852 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T04:23:00.852 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T04:23:00.852 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-17T04:23:00.852 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-17T04:23:00.852 INFO:teuthology.orchestra.run.smithi100.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-17T04:23:00.852 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.852 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.852 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:23:00.852 INFO:teuthology.orchestra.run.smithi100.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T04:23:00.852 INFO:teuthology.orchestra.run.smithi100.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T04:23:00.852 INFO:teuthology.orchestra.run.smithi100.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T04:23:00.852 INFO:teuthology.orchestra.run.smithi100.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-17T04:23:00.853 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-17T04:23:00.853 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T04:23:00.853 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-17T04:23:00.853 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:00.853 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-09-17T04:23:01.505 DEBUG:teuthology.parallel:result is None 2024-09-17T04:23:01.507 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T04:23:01.655 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T04:23:01.700 INFO:teuthology.orchestra.run.smithi100.stdout:19.3.0-4989.gce1b342b.el9 2024-09-17T04:23:01.701 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4989.gce1b342b.el9 2024-09-17T04:23:01.701 INFO:teuthology.task.install:The correct ceph version 19.3.0-4989.gce1b342b is installed. 2024-09-17T04:23:01.703 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T04:23:01.844 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T04:23:01.894 INFO:teuthology.orchestra.run.smithi184.stdout:19.3.0-4989.gce1b342b.el9 2024-09-17T04:23:01.895 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4989.gce1b342b.el9 2024-09-17T04:23:01.895 INFO:teuthology.task.install:The correct ceph version 19.3.0-4989.gce1b342b is installed. 2024-09-17T04:23:01.897 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-17T04:23:01.897 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:23:01.897 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T04:23:01.959 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:23:01.959 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T04:23:02.021 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-17T04:23:02.021 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:23:02.021 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T04:23:02.079 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T04:23:02.182 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:23:02.182 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T04:23:02.240 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T04:23:02.337 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-17T04:23:02.337 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:23:02.337 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T04:23:02.400 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T04:23:02.497 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:23:02.509 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T04:23:02.564 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T04:23:02.660 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-17T04:23:02.660 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:23:02.660 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T04:23:02.716 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T04:23:02.812 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:23:02.813 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T04:23:02.867 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T04:23:02.962 INFO:teuthology.run_tasks:Running task ceph... 2024-09-17T04:23:03.047 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-17T04:23:03.047 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 777 /var/log/ceph 2024-09-17T04:23:03.051 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 777 /var/log/ceph 2024-09-17T04:23:03.106 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-17T04:23:03.106 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T04:23:03.110 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T04:23:03.164 INFO:tasks.ceph:Creating extra log directories... 2024-09-17T04:23:03.164 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T04:23:03.168 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T04:23:03.229 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-17T04:23:03.229 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-17T04:23:03.229 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-17T04:23:03.230 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:23:03.230 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-17T04:23:03.264 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-17T04:23:03.435 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-17T04:23:03.490 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:23:03.490 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-17T04:23:03.526 DEBUG:teuthology.orchestra.run.smithi184:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-17T04:23:03.689 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-17T04:23:03.751 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-17T04:23:03.751 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-17T04:23:03.751 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908519', 'branch': 'wip-pdonnell-testing-20240916.200549-debug', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7908519', 'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240916.200549-debug'}, 'ceph': {'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}}, 'kclient': {'syntax': 'v1'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240916.200549-debug', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 90, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 476, 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf', 'sleep_before_teardown': 0, 'subset': '320/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240916.200549-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi100.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMyWBlyrgUGcQVVd5EecuSrhbQiyprjA+Ap91cVKUAQIKdwv+8mg/vWvNw/b/2QfwpFf1C7CytUYz9j+XKLTVhQ=', 'smithi184.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMo7DdF79+dnZHSEM9/GUw5Xd3VJT4TNL3oUbwqu1c0jsRXZWKRkiZeLh4OdNrUsD3ID85c4vC6jpfP0H/KcnL8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}}, {'ceph': {'create_rbd_pool': False, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestMisc']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-17_02:09:51', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-17T04:23:03.752 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T04:23:03.757 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T04:23:03.792 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T04:23:03.831 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T04:23:03.887 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:23:03.887 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T04:23:03.923 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T04:23:03.923 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_1 2024-09-17T04:23:03.998 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T04:23:03.998 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T04:23:03.999 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 6h/6d Inode: 637 Links: 1 2024-09-17T04:23:03.999 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T04:23:03.999 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T04:23:03.999 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-09-17 04:22:52.855227563 +0000 2024-09-17T04:23:03.999 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-17 04:20:27.432522196 +0000 2024-09-17T04:23:03.999 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-17 04:20:27.432522196 +0000 2024-09-17T04:23:03.999 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-09-17 04:20:27.432522196 +0000 2024-09-17T04:23:03.999 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T04:23:04.092 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-17T04:23:04.092 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-17T04:23:04.092 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000537474 s, 953 kB/s 2024-09-17T04:23:04.094 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T04:23:04.131 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_2 2024-09-17T04:23:04.207 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T04:23:04.207 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T04:23:04.207 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 6h/6d Inode: 658 Links: 1 2024-09-17T04:23:04.207 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T04:23:04.207 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T04:23:04.207 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-09-17 04:22:52.855227563 +0000 2024-09-17T04:23:04.207 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-17 04:20:27.879510112 +0000 2024-09-17T04:23:04.207 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-17 04:20:27.879510112 +0000 2024-09-17T04:23:04.207 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-09-17 04:20:27.879510112 +0000 2024-09-17T04:23:04.208 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T04:23:04.301 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-17T04:23:04.301 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-17T04:23:04.301 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000295278 s, 1.7 MB/s 2024-09-17T04:23:04.304 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T04:23:04.342 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_3 2024-09-17T04:23:04.418 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T04:23:04.418 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T04:23:04.418 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 6h/6d Inode: 677 Links: 1 2024-09-17T04:23:04.418 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T04:23:04.418 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T04:23:04.418 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-09-17 04:22:52.856227536 +0000 2024-09-17T04:23:04.419 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-17 04:20:28.356497216 +0000 2024-09-17T04:23:04.419 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-17 04:20:28.356497216 +0000 2024-09-17T04:23:04.419 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-09-17 04:20:28.356497216 +0000 2024-09-17T04:23:04.419 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T04:23:04.511 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-17T04:23:04.511 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-17T04:23:04.511 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.00025771 s, 2.0 MB/s 2024-09-17T04:23:04.514 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T04:23:04.551 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_4 2024-09-17T04:23:04.626 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T04:23:04.627 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T04:23:04.627 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 6h/6d Inode: 694 Links: 1 2024-09-17T04:23:04.627 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T04:23:04.627 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T04:23:04.627 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-09-17 04:22:52.857227509 +0000 2024-09-17T04:23:04.627 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-17 04:20:28.800485213 +0000 2024-09-17T04:23:04.627 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-17 04:20:28.800485213 +0000 2024-09-17T04:23:04.627 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-09-17 04:20:28.800485213 +0000 2024-09-17T04:23:04.627 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T04:23:04.719 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-17T04:23:04.720 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-17T04:23:04.720 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000271407 s, 1.9 MB/s 2024-09-17T04:23:04.722 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T04:23:04.759 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T04:23:04.759 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:23:04.759 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T04:23:04.800 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T04:23:04.800 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_1 2024-09-17T04:23:04.879 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T04:23:04.879 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T04:23:04.880 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-09-17T04:23:04.880 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T04:23:04.880 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T04:23:04.880 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-09-17 04:22:46.149771796 +0000 2024-09-17T04:23:04.880 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-09-17 04:20:27.457090911 +0000 2024-09-17T04:23:04.880 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-09-17 04:20:27.457090911 +0000 2024-09-17T04:23:04.880 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-09-17 04:20:27.457090911 +0000 2024-09-17T04:23:04.880 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T04:23:04.976 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-09-17T04:23:04.976 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-09-17T04:23:04.976 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000277079 s, 1.8 MB/s 2024-09-17T04:23:04.979 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T04:23:05.017 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_2 2024-09-17T04:23:05.093 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T04:23:05.093 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T04:23:05.093 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-09-17T04:23:05.093 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T04:23:05.093 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T04:23:05.094 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-09-17 04:22:46.150771775 +0000 2024-09-17T04:23:05.094 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-09-17 04:20:27.947080706 +0000 2024-09-17T04:23:05.094 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-09-17 04:20:27.947080706 +0000 2024-09-17T04:23:05.094 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-09-17 04:20:27.947080706 +0000 2024-09-17T04:23:05.094 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T04:23:05.186 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-09-17T04:23:05.186 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-09-17T04:23:05.186 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000466969 s, 1.1 MB/s 2024-09-17T04:23:05.189 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T04:23:05.226 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_3 2024-09-17T04:23:05.301 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T04:23:05.301 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T04:23:05.301 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-09-17T04:23:05.301 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T04:23:05.301 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T04:23:05.301 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-09-17 04:22:46.150771775 +0000 2024-09-17T04:23:05.301 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-09-17 04:20:28.428070688 +0000 2024-09-17T04:23:05.301 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-09-17 04:20:28.428070688 +0000 2024-09-17T04:23:05.302 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-09-17 04:20:28.428070688 +0000 2024-09-17T04:23:05.302 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T04:23:05.394 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-09-17T04:23:05.394 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-09-17T04:23:05.394 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000287692 s, 1.8 MB/s 2024-09-17T04:23:05.396 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T04:23:05.433 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_4 2024-09-17T04:23:05.508 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T04:23:05.508 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T04:23:05.508 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-09-17T04:23:05.509 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T04:23:05.509 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T04:23:05.509 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-09-17 04:22:46.151771754 +0000 2024-09-17T04:23:05.509 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-09-17 04:20:28.935060129 +0000 2024-09-17T04:23:05.509 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-09-17 04:20:28.935060129 +0000 2024-09-17T04:23:05.509 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-09-17 04:20:28.935060129 +0000 2024-09-17T04:23:05.509 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T04:23:05.601 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-09-17T04:23:05.601 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-09-17T04:23:05.601 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000289763 s, 1.8 MB/s 2024-09-17T04:23:05.604 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T04:23:05.640 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T04:23:05.640 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi100.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi184.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-17T04:23:05.640 INFO:tasks.ceph:Generating config... 2024-09-17T04:23:05.642 INFO:tasks.ceph:[client] pid file = 2024-09-17T04:23:05.642 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-17T04:23:05.642 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-17T04:23:05.642 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-09-17T04:23:05.642 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-17T04:23:05.642 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-17T04:23:05.642 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-17T04:23:05.642 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-17T04:23:05.642 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-17T04:23:05.642 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-17T04:23:05.642 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-17T04:23:05.642 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-17T04:23:05.643 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-17T04:23:05.643 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-17T04:23:05.643 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-17T04:23:05.643 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-17T04:23:05.643 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-17T04:23:05.643 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-17T04:23:05.643 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-17T04:23:05.643 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-17T04:23:05.643 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-17T04:23:05.643 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-17T04:23:05.643 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-17T04:23:05.643 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-17T04:23:05.644 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-17T04:23:05.644 INFO:tasks.ceph:Setting up mon.a... 2024-09-17T04:23:05.644 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-17T04:23:05.804 INFO:teuthology.orchestra.run.smithi100.stdout:creating /etc/ceph/ceph.keyring 2024-09-17T04:23:05.816 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-17T04:23:05.969 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T04:23:06.024 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.100'), ('mon.b', '172.21.15.184'), ('mon.c', '[v2:172.21.15.184:3301,v1:172.21.15.184:6790]')] 2024-09-17T04:23:06.025 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.100,172.21.15.184,[v2:172.21.15.184:3301,v1:172.21.15.184:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-17T04:23:06.025 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:23:06.026 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T04:23:06.060 DEBUG:teuthology.orchestra.run.smithi100:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.100 --add b 172.21.15.184 --addv c '[v2:172.21.15.184:3301,v1:172.21.15.184:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-17T04:23:06.239 INFO:teuthology.orchestra.run.smithi100.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-17T04:23:06.239 INFO:teuthology.orchestra.run.smithi100.stdout:monmaptool: generated fsid 8a399842-3a5c-4ea2-9afb-b2704332e59f 2024-09-17T04:23:06.239 INFO:teuthology.orchestra.run.smithi100.stdout:setting min_mon_release = quincy 2024-09-17T04:23:06.239 INFO:teuthology.orchestra.run.smithi100.stdout:epoch 0 2024-09-17T04:23:06.239 INFO:teuthology.orchestra.run.smithi100.stdout:fsid 8a399842-3a5c-4ea2-9afb-b2704332e59f 2024-09-17T04:23:06.239 INFO:teuthology.orchestra.run.smithi100.stdout:last_changed 2024-09-17T04:23:06.240015+0000 2024-09-17T04:23:06.239 INFO:teuthology.orchestra.run.smithi100.stdout:created 2024-09-17T04:23:06.240015+0000 2024-09-17T04:23:06.239 INFO:teuthology.orchestra.run.smithi100.stdout:min_mon_release 17 (quincy) 2024-09-17T04:23:06.239 INFO:teuthology.orchestra.run.smithi100.stdout:election_strategy: 1 2024-09-17T04:23:06.239 INFO:teuthology.orchestra.run.smithi100.stdout:0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-09-17T04:23:06.239 INFO:teuthology.orchestra.run.smithi100.stdout:1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.b 2024-09-17T04:23:06.239 INFO:teuthology.orchestra.run.smithi100.stdout:2: [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] mon.c 2024-09-17T04:23:06.240 INFO:teuthology.orchestra.run.smithi100.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-17T04:23:06.247 DEBUG:teuthology.orchestra.run.smithi100:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T04:23:06.282 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 8a399842-3a5c-4ea2-9afb-b2704332e59f... 2024-09-17T04:23:06.283 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T04:23:06.326 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T04:23:06.419 INFO:teuthology.orchestra.run.smithi100.stdout:[global] 2024-09-17T04:23:06.419 INFO:teuthology.orchestra.run.smithi100.stdout: chdir = "" 2024-09-17T04:23:06.419 INFO:teuthology.orchestra.run.smithi100.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T04:23:06.419 INFO:teuthology.orchestra.run.smithi100.stdout: auth supported = cephx 2024-09-17T04:23:06.419 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.419 INFO:teuthology.orchestra.run.smithi100.stdout: filestore xattr use omap = true 2024-09-17T04:23:06.419 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.419 INFO:teuthology.orchestra.run.smithi100.stdout: mon clock drift allowed = 1.000 2024-09-17T04:23:06.419 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.420 INFO:teuthology.orchestra.run.smithi100.stdout: osd crush chooseleaf type = 0 2024-09-17T04:23:06.420 INFO:teuthology.orchestra.run.smithi100.stdout: auth debug = true 2024-09-17T04:23:06.420 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.420 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on old message = true 2024-09-17T04:23:06.420 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on bug = true 2024-09-17T04:23:06.420 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.420 INFO:teuthology.orchestra.run.smithi100.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T04:23:06.420 INFO:teuthology.orchestra.run.smithi100.stdout: mon pg warn max object skew = 0 2024-09-17T04:23:06.420 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.420 INFO:teuthology.orchestra.run.smithi100.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T04:23:06.420 INFO:teuthology.orchestra.run.smithi100.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T04:23:06.420 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.420 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default size = 2 2024-09-17T04:23:06.420 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.421 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow primary affinity = true 2024-09-17T04:23:06.421 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow pg remap = true 2024-09-17T04:23:06.421 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on legacy crush tunables = false 2024-09-17T04:23:06.421 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on crush straw calc version zero = false 2024-09-17T04:23:06.421 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on no sortbitwise = false 2024-09-17T04:23:06.421 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on osd down out interval zero = false 2024-09-17T04:23:06.421 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on too few osds = false 2024-09-17T04:23:06.421 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T04:23:06.421 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T04:23:06.421 INFO:teuthology.orchestra.run.smithi100.stdout: mon_allow_pool_size_one = true 2024-09-17T04:23:06.421 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.421 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T04:23:06.421 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.421 INFO:teuthology.orchestra.run.smithi100.stdout: osd default data pool replay window = 5 2024-09-17T04:23:06.422 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.422 INFO:teuthology.orchestra.run.smithi100.stdout: mon allow pool delete = true 2024-09-17T04:23:06.422 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.422 INFO:teuthology.orchestra.run.smithi100.stdout: mon cluster log file level = debug 2024-09-17T04:23:06.422 INFO:teuthology.orchestra.run.smithi100.stdout: debug asserts on shutdown = true 2024-09-17T04:23:06.422 INFO:teuthology.orchestra.run.smithi100.stdout: mon health detail to clog = false 2024-09-17T04:23:06.422 INFO:teuthology.orchestra.run.smithi100.stdout: mon host = "172.21.15.100,172.21.15.184,[v2:172.21.15.184:3301,v1:172.21.15.184:6790]" 2024-09-17T04:23:06.422 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on skipped message = False 2024-09-17T04:23:06.422 INFO:teuthology.orchestra.run.smithi100.stdout: fsid = 8a399842-3a5c-4ea2-9afb-b2704332e59f 2024-09-17T04:23:06.422 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.422 INFO:teuthology.orchestra.run.smithi100.stdout:[osd] 2024-09-17T04:23:06.422 INFO:teuthology.orchestra.run.smithi100.stdout: osd journal size = 100 2024-09-17T04:23:06.422 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.422 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub load threshold = 5.0 2024-09-17T04:23:06.423 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub max interval = 600 2024-09-17T04:23:06.423 INFO:teuthology.orchestra.run.smithi100.stdout: osd mclock profile = high_recovery_ops 2024-09-17T04:23:06.423 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.423 INFO:teuthology.orchestra.run.smithi100.stdout: osd recover clone overlap = true 2024-09-17T04:23:06.423 INFO:teuthology.orchestra.run.smithi100.stdout: osd recovery max chunk = 1048576 2024-09-17T04:23:06.423 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.423 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug shutdown = true 2024-09-17T04:23:06.423 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug op order = true 2024-09-17T04:23:06.423 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug verify stray on activate = true 2024-09-17T04:23:06.423 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug trim objects = true 2024-09-17T04:23:06.423 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.423 INFO:teuthology.orchestra.run.smithi100.stdout: osd open classes on start = true 2024-09-17T04:23:06.423 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug pg log writeout = true 2024-09-17T04:23:06.423 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.423 INFO:teuthology.orchestra.run.smithi100.stdout: osd deep scrub update digest min age = 30 2024-09-17T04:23:06.424 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.424 INFO:teuthology.orchestra.run.smithi100.stdout: osd map max advance = 10 2024-09-17T04:23:06.424 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.424 INFO:teuthology.orchestra.run.smithi100.stdout: journal zero on create = true 2024-09-17T04:23:06.424 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.424 INFO:teuthology.orchestra.run.smithi100.stdout: filestore ondisk finisher threads = 3 2024-09-17T04:23:06.424 INFO:teuthology.orchestra.run.smithi100.stdout: filestore apply finisher threads = 3 2024-09-17T04:23:06.424 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.424 INFO:teuthology.orchestra.run.smithi100.stdout: bdev debug aio = true 2024-09-17T04:23:06.424 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug misdirected ops = true 2024-09-17T04:23:06.424 INFO:teuthology.orchestra.run.smithi100.stdout: bdev async discard = True 2024-09-17T04:23:06.424 INFO:teuthology.orchestra.run.smithi100.stdout: bdev enable discard = True 2024-09-17T04:23:06.424 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore allocator = bitmap 2024-09-17T04:23:06.425 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore block size = 96636764160 2024-09-17T04:23:06.425 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore fsck on mount = True 2024-09-17T04:23:06.425 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluefs = 1/20 2024-09-17T04:23:06.425 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluestore = 1/20 2024-09-17T04:23:06.425 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-09-17T04:23:06.425 INFO:teuthology.orchestra.run.smithi100.stdout: debug osd = 20 2024-09-17T04:23:06.425 INFO:teuthology.orchestra.run.smithi100.stdout: debug rocksdb = 4/10 2024-09-17T04:23:06.425 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T04:23:06.425 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd full ratio = 0.9 2024-09-17T04:23:06.425 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd nearfull ratio = 0.8 2024-09-17T04:23:06.425 INFO:teuthology.orchestra.run.smithi100.stdout: osd failsafe full ratio = 0.95 2024-09-17T04:23:06.425 INFO:teuthology.orchestra.run.smithi100.stdout: osd objectstore = bluestore 2024-09-17T04:23:06.425 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.425 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr] 2024-09-17T04:23:06.426 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-09-17T04:23:06.426 INFO:teuthology.orchestra.run.smithi100.stdout: debug mgr = 20 2024-09-17T04:23:06.426 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2024-09-17T04:23:06.426 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2024-09-17T04:23:06.426 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2024-09-17T04:23:06.426 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2024-09-17T04:23:06.426 INFO:teuthology.orchestra.run.smithi100.stdout: mgr/telemetry/nag = false 2024-09-17T04:23:06.426 INFO:teuthology.orchestra.run.smithi100.stdout: debug client = 20 2024-09-17T04:23:06.426 INFO:teuthology.orchestra.run.smithi100.stdout: debug finisher = 20 2024-09-17T04:23:06.426 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.426 INFO:teuthology.orchestra.run.smithi100.stdout:[mon] 2024-09-17T04:23:06.426 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-09-17T04:23:06.426 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2024-09-17T04:23:06.426 INFO:teuthology.orchestra.run.smithi100.stdout: debug paxos = 20 2024-09-17T04:23:06.426 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2024-09-17T04:23:06.427 INFO:teuthology.orchestra.run.smithi100.stdout: mon data avail warn = 5 2024-09-17T04:23:06.427 INFO:teuthology.orchestra.run.smithi100.stdout: mon mgr mkfs grace = 240 2024-09-17T04:23:06.427 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2024-09-17T04:23:06.427 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd reporter subtree level = osd 2024-09-17T04:23:06.427 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd prime pg temp = true 2024-09-17T04:23:06.427 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2024-09-17T04:23:06.427 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.427 INFO:teuthology.orchestra.run.smithi100.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T04:23:06.427 INFO:teuthology.orchestra.run.smithi100.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T04:23:06.427 INFO:teuthology.orchestra.run.smithi100.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T04:23:06.427 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.427 INFO:teuthology.orchestra.run.smithi100.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T04:23:06.427 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T04:23:06.427 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T04:23:06.427 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.428 INFO:teuthology.orchestra.run.smithi100.stdout: # 1m isn't quite enough 2024-09-17T04:23:06.428 INFO:teuthology.orchestra.run.smithi100.stdout: mon_down_mkfs_grace = 2m 2024-09-17T04:23:06.428 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.428 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_filestore_osds = false 2024-09-17T04:23:06.428 INFO:teuthology.orchestra.run.smithi100.stdout: mon down mkfs grace = 300 2024-09-17T04:23:06.428 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:06.428 INFO:teuthology.orchestra.run.smithi100.stdout:[client] 2024-09-17T04:23:06.428 INFO:teuthology.orchestra.run.smithi100.stdout: rgw cache enabled = true 2024-09-17T04:23:06.428 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable ops log = true 2024-09-17T04:23:06.428 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable usage log = true 2024-09-17T04:23:06.428 INFO:teuthology.orchestra.run.smithi100.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T04:23:06.428 INFO:teuthology.orchestra.run.smithi100.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T04:23:06.428 INFO:teuthology.orchestra.run.smithi100.stdout: pid file = "" 2024-09-17T04:23:06.428 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.a] 2024-09-17T04:23:06.429 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.b] 2024-09-17T04:23:06.429 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.c] 2024-09-17T04:23:06.429 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.a] 2024-09-17T04:23:06.429 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.c] 2024-09-17T04:23:06.429 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.b] 2024-09-17T04:23:06.429 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.d] 2024-09-17T04:23:06.435 INFO:teuthology.orchestra.run.smithi184.stdout:[global] 2024-09-17T04:23:06.435 INFO:teuthology.orchestra.run.smithi184.stdout: chdir = "" 2024-09-17T04:23:06.435 INFO:teuthology.orchestra.run.smithi184.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T04:23:06.435 INFO:teuthology.orchestra.run.smithi184.stdout: auth supported = cephx 2024-09-17T04:23:06.435 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.435 INFO:teuthology.orchestra.run.smithi184.stdout: filestore xattr use omap = true 2024-09-17T04:23:06.435 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.435 INFO:teuthology.orchestra.run.smithi184.stdout: mon clock drift allowed = 1.000 2024-09-17T04:23:06.436 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.436 INFO:teuthology.orchestra.run.smithi184.stdout: osd crush chooseleaf type = 0 2024-09-17T04:23:06.436 INFO:teuthology.orchestra.run.smithi184.stdout: auth debug = true 2024-09-17T04:23:06.436 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.436 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on old message = true 2024-09-17T04:23:06.436 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on bug = true 2024-09-17T04:23:06.436 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.436 INFO:teuthology.orchestra.run.smithi184.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T04:23:06.436 INFO:teuthology.orchestra.run.smithi184.stdout: mon pg warn max object skew = 0 2024-09-17T04:23:06.436 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.436 INFO:teuthology.orchestra.run.smithi184.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T04:23:06.436 INFO:teuthology.orchestra.run.smithi184.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T04:23:06.436 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.436 INFO:teuthology.orchestra.run.smithi184.stdout: osd pool default size = 2 2024-09-17T04:23:06.436 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.437 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd allow primary affinity = true 2024-09-17T04:23:06.437 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd allow pg remap = true 2024-09-17T04:23:06.437 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on legacy crush tunables = false 2024-09-17T04:23:06.437 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on crush straw calc version zero = false 2024-09-17T04:23:06.437 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on no sortbitwise = false 2024-09-17T04:23:06.437 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on osd down out interval zero = false 2024-09-17T04:23:06.437 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on too few osds = false 2024-09-17T04:23:06.437 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T04:23:06.437 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T04:23:06.437 INFO:teuthology.orchestra.run.smithi184.stdout: mon_allow_pool_size_one = true 2024-09-17T04:23:06.437 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.437 INFO:teuthology.orchestra.run.smithi184.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T04:23:06.437 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.437 INFO:teuthology.orchestra.run.smithi184.stdout: osd default data pool replay window = 5 2024-09-17T04:23:06.437 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.438 INFO:teuthology.orchestra.run.smithi184.stdout: mon allow pool delete = true 2024-09-17T04:23:06.438 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.438 INFO:teuthology.orchestra.run.smithi184.stdout: mon cluster log file level = debug 2024-09-17T04:23:06.438 INFO:teuthology.orchestra.run.smithi184.stdout: debug asserts on shutdown = true 2024-09-17T04:23:06.438 INFO:teuthology.orchestra.run.smithi184.stdout: mon health detail to clog = false 2024-09-17T04:23:06.438 INFO:teuthology.orchestra.run.smithi184.stdout: mon host = "172.21.15.100,172.21.15.184,[v2:172.21.15.184:3301,v1:172.21.15.184:6790]" 2024-09-17T04:23:06.438 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on skipped message = False 2024-09-17T04:23:06.438 INFO:teuthology.orchestra.run.smithi184.stdout: fsid = 8a399842-3a5c-4ea2-9afb-b2704332e59f 2024-09-17T04:23:06.438 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.438 INFO:teuthology.orchestra.run.smithi184.stdout:[osd] 2024-09-17T04:23:06.438 INFO:teuthology.orchestra.run.smithi184.stdout: osd journal size = 100 2024-09-17T04:23:06.438 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.438 INFO:teuthology.orchestra.run.smithi184.stdout: osd scrub load threshold = 5.0 2024-09-17T04:23:06.439 INFO:teuthology.orchestra.run.smithi184.stdout: osd scrub max interval = 600 2024-09-17T04:23:06.439 INFO:teuthology.orchestra.run.smithi184.stdout: osd mclock profile = high_recovery_ops 2024-09-17T04:23:06.439 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.439 INFO:teuthology.orchestra.run.smithi184.stdout: osd recover clone overlap = true 2024-09-17T04:23:06.439 INFO:teuthology.orchestra.run.smithi184.stdout: osd recovery max chunk = 1048576 2024-09-17T04:23:06.439 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.439 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug shutdown = true 2024-09-17T04:23:06.439 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug op order = true 2024-09-17T04:23:06.439 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug verify stray on activate = true 2024-09-17T04:23:06.439 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug trim objects = true 2024-09-17T04:23:06.439 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.439 INFO:teuthology.orchestra.run.smithi184.stdout: osd open classes on start = true 2024-09-17T04:23:06.439 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug pg log writeout = true 2024-09-17T04:23:06.439 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.440 INFO:teuthology.orchestra.run.smithi184.stdout: osd deep scrub update digest min age = 30 2024-09-17T04:23:06.440 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.440 INFO:teuthology.orchestra.run.smithi184.stdout: osd map max advance = 10 2024-09-17T04:23:06.440 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.440 INFO:teuthology.orchestra.run.smithi184.stdout: journal zero on create = true 2024-09-17T04:23:06.440 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.440 INFO:teuthology.orchestra.run.smithi184.stdout: filestore ondisk finisher threads = 3 2024-09-17T04:23:06.440 INFO:teuthology.orchestra.run.smithi184.stdout: filestore apply finisher threads = 3 2024-09-17T04:23:06.440 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.440 INFO:teuthology.orchestra.run.smithi184.stdout: bdev debug aio = true 2024-09-17T04:23:06.440 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug misdirected ops = true 2024-09-17T04:23:06.440 INFO:teuthology.orchestra.run.smithi184.stdout: bdev async discard = True 2024-09-17T04:23:06.440 INFO:teuthology.orchestra.run.smithi184.stdout: bdev enable discard = True 2024-09-17T04:23:06.441 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore allocator = bitmap 2024-09-17T04:23:06.441 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore block size = 96636764160 2024-09-17T04:23:06.441 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore fsck on mount = True 2024-09-17T04:23:06.441 INFO:teuthology.orchestra.run.smithi184.stdout: debug bluefs = 1/20 2024-09-17T04:23:06.441 INFO:teuthology.orchestra.run.smithi184.stdout: debug bluestore = 1/20 2024-09-17T04:23:06.441 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2024-09-17T04:23:06.441 INFO:teuthology.orchestra.run.smithi184.stdout: debug osd = 20 2024-09-17T04:23:06.441 INFO:teuthology.orchestra.run.smithi184.stdout: debug rocksdb = 4/10 2024-09-17T04:23:06.441 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T04:23:06.441 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd full ratio = 0.9 2024-09-17T04:23:06.441 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd nearfull ratio = 0.8 2024-09-17T04:23:06.441 INFO:teuthology.orchestra.run.smithi184.stdout: osd failsafe full ratio = 0.95 2024-09-17T04:23:06.442 INFO:teuthology.orchestra.run.smithi184.stdout: osd objectstore = bluestore 2024-09-17T04:23:06.442 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.442 INFO:teuthology.orchestra.run.smithi184.stdout:[mgr] 2024-09-17T04:23:06.442 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2024-09-17T04:23:06.442 INFO:teuthology.orchestra.run.smithi184.stdout: debug mgr = 20 2024-09-17T04:23:06.442 INFO:teuthology.orchestra.run.smithi184.stdout: debug mon = 20 2024-09-17T04:23:06.442 INFO:teuthology.orchestra.run.smithi184.stdout: debug auth = 20 2024-09-17T04:23:06.442 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min pgs per osd = 4 2024-09-17T04:23:06.442 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min bytes per osd = 10 2024-09-17T04:23:06.442 INFO:teuthology.orchestra.run.smithi184.stdout: mgr/telemetry/nag = false 2024-09-17T04:23:06.442 INFO:teuthology.orchestra.run.smithi184.stdout: debug client = 20 2024-09-17T04:23:06.442 INFO:teuthology.orchestra.run.smithi184.stdout: debug finisher = 20 2024-09-17T04:23:06.442 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.442 INFO:teuthology.orchestra.run.smithi184.stdout:[mon] 2024-09-17T04:23:06.442 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2024-09-17T04:23:06.442 INFO:teuthology.orchestra.run.smithi184.stdout: debug mon = 20 2024-09-17T04:23:06.443 INFO:teuthology.orchestra.run.smithi184.stdout: debug paxos = 20 2024-09-17T04:23:06.443 INFO:teuthology.orchestra.run.smithi184.stdout: debug auth = 20 2024-09-17T04:23:06.443 INFO:teuthology.orchestra.run.smithi184.stdout: mon data avail warn = 5 2024-09-17T04:23:06.443 INFO:teuthology.orchestra.run.smithi184.stdout: mon mgr mkfs grace = 240 2024-09-17T04:23:06.443 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min pgs per osd = 4 2024-09-17T04:23:06.443 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd reporter subtree level = osd 2024-09-17T04:23:06.443 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd prime pg temp = true 2024-09-17T04:23:06.443 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min bytes per osd = 10 2024-09-17T04:23:06.443 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.443 INFO:teuthology.orchestra.run.smithi184.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T04:23:06.443 INFO:teuthology.orchestra.run.smithi184.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T04:23:06.443 INFO:teuthology.orchestra.run.smithi184.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T04:23:06.443 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.443 INFO:teuthology.orchestra.run.smithi184.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T04:23:06.443 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T04:23:06.444 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T04:23:06.444 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.444 INFO:teuthology.orchestra.run.smithi184.stdout: # 1m isn't quite enough 2024-09-17T04:23:06.444 INFO:teuthology.orchestra.run.smithi184.stdout: mon_down_mkfs_grace = 2m 2024-09-17T04:23:06.444 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.444 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_filestore_osds = false 2024-09-17T04:23:06.444 INFO:teuthology.orchestra.run.smithi184.stdout: mon down mkfs grace = 300 2024-09-17T04:23:06.444 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-17T04:23:06.444 INFO:teuthology.orchestra.run.smithi184.stdout:[client] 2024-09-17T04:23:06.444 INFO:teuthology.orchestra.run.smithi184.stdout: rgw cache enabled = true 2024-09-17T04:23:06.444 INFO:teuthology.orchestra.run.smithi184.stdout: rgw enable ops log = true 2024-09-17T04:23:06.444 INFO:teuthology.orchestra.run.smithi184.stdout: rgw enable usage log = true 2024-09-17T04:23:06.444 INFO:teuthology.orchestra.run.smithi184.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T04:23:06.444 INFO:teuthology.orchestra.run.smithi184.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T04:23:06.445 INFO:teuthology.orchestra.run.smithi184.stdout: pid file = "" 2024-09-17T04:23:06.445 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.a] 2024-09-17T04:23:06.445 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.b] 2024-09-17T04:23:06.445 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.c] 2024-09-17T04:23:06.445 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.a] 2024-09-17T04:23:06.445 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.c] 2024-09-17T04:23:06.445 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.b] 2024-09-17T04:23:06.445 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.d] 2024-09-17T04:23:06.461 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-17T04:23:06.462 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-17T04:23:06.627 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-17T04:23:06.627 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:23:06.627 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-17T04:23:06.662 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:23:06.662 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-17T04:23:06.736 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi100.front.sepia.ceph.com 2024-09-17T04:23:06.736 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:23:06.737 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T04:23:06.737 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T04:23:06.853 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:23:06.853 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T04:23:06.929 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi184.front.sepia.ceph.com 2024-09-17T04:23:06.929 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:23:06.929 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T04:23:06.929 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T04:23:07.010 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:23:07.010 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T04:23:07.086 INFO:tasks.ceph:Setting up mon nodes... 2024-09-17T04:23:07.086 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-17T04:23:07.086 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-17T04:23:07.249 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-17T04:23:07.262 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T04:23:07.451 INFO:teuthology.orchestra.run.smithi184.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T04:23:07.465 INFO:tasks.ceph:Setting up mds nodes... 2024-09-17T04:23:07.466 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-17T04:23:07.630 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-17T04:23:07.642 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-17T04:23:07.698 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-17T04:23:07.863 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-17T04:23:07.874 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-17T04:23:07.929 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-17T04:23:08.097 INFO:teuthology.orchestra.run.smithi184.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-17T04:23:08.109 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-17T04:23:08.165 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-17T04:23:08.330 INFO:teuthology.orchestra.run.smithi184.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-17T04:23:08.341 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-17T04:23:08.397 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-17T04:23:08.397 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-17T04:23:08.539 INFO:teuthology.orchestra.run.smithi100.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-17T04:23:08.572 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-17T04:23:08.715 INFO:teuthology.orchestra.run.smithi100.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-17T04:23:08.748 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-17T04:23:08.894 INFO:teuthology.orchestra.run.smithi184.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-17T04:23:08.928 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-17T04:23:09.069 INFO:teuthology.orchestra.run.smithi184.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-17T04:23:09.103 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-17T04:23:09.104 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi100.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi184.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-17T04:23:09.104 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-17T04:23:09.160 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T04:23:09.160 INFO:tasks.ceph:role: osd.0 2024-09-17T04:23:09.160 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-17T04:23:09.160 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T04:23:09.222 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T04:23:09.222 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T04:23:09.222 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T04:23:09.223 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T04:23:09.223 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T04:23:09.223 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-09-17T04:23:09.223 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T04:23:09.223 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T04:23:09.223 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T04:23:09.223 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T04:23:09.630 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-09-17T04:23:09.634 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-09-17T04:23:09.634 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-17T04:23:09.765 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-17T04:23:09.828 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-17T04:23:09.884 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T04:23:09.884 INFO:tasks.ceph:role: osd.1 2024-09-17T04:23:09.885 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-17T04:23:09.885 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T04:23:09.948 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T04:23:09.948 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T04:23:09.948 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T04:23:09.948 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T04:23:09.948 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T04:23:09.948 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-09-17T04:23:09.948 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T04:23:09.949 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T04:23:09.949 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T04:23:09.949 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T04:23:10.306 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-09-17T04:23:10.310 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-09-17T04:23:10.311 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-17T04:23:10.396 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-17T04:23:10.457 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-17T04:23:10.512 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T04:23:10.512 INFO:tasks.ceph:role: osd.2 2024-09-17T04:23:10.512 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-17T04:23:10.513 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T04:23:10.575 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T04:23:10.575 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T04:23:10.575 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T04:23:10.575 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T04:23:10.575 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T04:23:10.575 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-09-17T04:23:10.575 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T04:23:10.575 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T04:23:10.576 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T04:23:10.576 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T04:23:10.976 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-09-17T04:23:10.980 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-09-17T04:23:10.980 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-17T04:23:11.085 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-17T04:23:11.147 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-17T04:23:11.202 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T04:23:11.202 INFO:tasks.ceph:role: osd.3 2024-09-17T04:23:11.202 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-17T04:23:11.202 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-17T04:23:11.265 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T04:23:11.265 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T04:23:11.265 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T04:23:11.265 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T04:23:11.266 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T04:23:11.266 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-09-17T04:23:11.266 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T04:23:11.266 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T04:23:11.266 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T04:23:11.266 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T04:23:11.685 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-09-17T04:23:11.689 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-09-17T04:23:11.689 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-17T04:23:11.829 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-17T04:23:11.893 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T04:23:12.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:12.055+0000 7fd8c70e6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-17T04:23:12.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:12.061+0000 7fd8c70e6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-17T04:23:12.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:12.061+0000 7fd8c70e6640 -1 bdev(0x55a255775900 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-17T04:23:12.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:12.061+0000 7fd8c70e6640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-17T04:23:15.117 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-17T04:23:15.174 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T04:23:15.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:15.327+0000 7fdea4129640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-17T04:23:15.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:15.328+0000 7fdea4129640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-17T04:23:15.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:15.328+0000 7fdea4129640 -1 bdev(0x5616932cd900 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-17T04:23:15.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:15.328+0000 7fdea4129640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-17T04:23:18.402 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-17T04:23:18.458 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T04:23:18.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:18.611+0000 7fdca401f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-17T04:23:18.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:18.611+0000 7fdca401f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-17T04:23:18.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:18.611+0000 7fdca401f640 -1 bdev(0x56351c101900 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-17T04:23:18.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:18.611+0000 7fdca401f640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-17T04:23:21.856 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-17T04:23:21.914 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T04:23:22.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:22.067+0000 7f395ae0b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-17T04:23:22.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:22.068+0000 7f395ae0b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-17T04:23:22.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:22.068+0000 7f395ae0b640 -1 bdev(0x55aef38dd900 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-17T04:23:22.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:22.068+0000 7f395ae0b640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-17T04:23:25.070 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-17T04:23:25.126 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-17T04:23:25.192 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T04:23:25.192 INFO:tasks.ceph:role: osd.4 2024-09-17T04:23:25.192 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi184.front.sepia.ceph.com 2024-09-17T04:23:25.192 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T04:23:25.262 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T04:23:25.263 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T04:23:25.263 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T04:23:25.263 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T04:23:25.263 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T04:23:25.263 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2024-09-17T04:23:25.263 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T04:23:25.263 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T04:23:25.263 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T04:23:25.263 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T04:23:25.672 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2024-09-17T04:23:25.676 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2024-09-17T04:23:25.677 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-17T04:23:25.814 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-17T04:23:25.881 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-17T04:23:25.937 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T04:23:25.937 INFO:tasks.ceph:role: osd.5 2024-09-17T04:23:25.938 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi184.front.sepia.ceph.com 2024-09-17T04:23:25.938 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T04:23:25.999 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T04:23:26.000 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T04:23:26.000 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T04:23:26.000 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T04:23:26.000 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T04:23:26.000 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2024-09-17T04:23:26.000 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T04:23:26.000 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T04:23:26.000 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T04:23:26.000 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T04:23:26.356 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2024-09-17T04:23:26.360 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2024-09-17T04:23:26.360 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-17T04:23:26.448 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-17T04:23:26.510 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-17T04:23:26.566 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T04:23:26.566 INFO:tasks.ceph:role: osd.6 2024-09-17T04:23:26.566 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi184.front.sepia.ceph.com 2024-09-17T04:23:26.566 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T04:23:26.628 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T04:23:26.628 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T04:23:26.628 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T04:23:26.628 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T04:23:26.629 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T04:23:26.629 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2024-09-17T04:23:26.629 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T04:23:26.629 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T04:23:26.629 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T04:23:26.629 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T04:23:27.021 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2024-09-17T04:23:27.024 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2024-09-17T04:23:27.025 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-17T04:23:27.123 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-17T04:23:27.186 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-17T04:23:27.241 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T04:23:27.241 INFO:tasks.ceph:role: osd.7 2024-09-17T04:23:27.241 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi184.front.sepia.ceph.com 2024-09-17T04:23:27.241 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-17T04:23:27.304 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T04:23:27.304 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T04:23:27.304 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T04:23:27.304 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T04:23:27.305 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T04:23:27.305 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2024-09-17T04:23:27.305 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T04:23:27.305 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T04:23:27.305 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T04:23:27.305 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T04:23:27.731 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2024-09-17T04:23:27.735 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2024-09-17T04:23:27.735 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-17T04:23:27.878 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-17T04:23:27.942 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T04:23:28.111 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:28.110+0000 7f2c1ee0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-17T04:23:28.112 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:28.112+0000 7f2c1ee0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-17T04:23:28.112 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:28.112+0000 7f2c1ee0d640 -1 bdev(0x55eb735b1900 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-17T04:23:28.113 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:28.112+0000 7f2c1ee0d640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-17T04:23:31.168 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-17T04:23:31.225 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T04:23:31.380 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:31.379+0000 7f50f2a18640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-17T04:23:31.380 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:31.379+0000 7f50f2a18640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-17T04:23:31.380 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:31.379+0000 7f50f2a18640 -1 bdev(0x55f849959900 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-17T04:23:31.380 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:31.380+0000 7f50f2a18640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-17T04:23:33.753 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:23:33.758 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:23:34.429 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-17T04:23:34.484 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T04:23:34.640 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:34.639+0000 7f3654f32640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-17T04:23:34.640 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:34.639+0000 7f3654f32640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-17T04:23:34.640 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:34.639+0000 7f3654f32640 -1 bdev(0x562bafe19900 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-17T04:23:34.640 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:34.640+0000 7f3654f32640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-17T04:23:37.673 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-17T04:23:37.730 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T04:23:37.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:37.884+0000 7f8830224640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-17T04:23:37.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:37.885+0000 7f8830224640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-17T04:23:37.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:37.885+0000 7f8830224640 -1 bdev(0x5592d6fa9900 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-17T04:23:37.886 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:37.885+0000 7f8830224640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-17T04:23:40.937 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-17T04:23:40.993 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-17T04:23:40.994 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:23:40.994 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-17T04:23:41.056 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:23:41.056 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-17T04:23:41.153 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:23:41.154 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-17T04:23:41.250 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:23:41.250 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-17T04:23:41.345 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:23:41.345 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-17T04:23:41.441 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:23:41.441 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-17T04:23:41.537 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:23:41.537 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-17T04:23:41.632 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:23:41.632 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-17T04:23:41.690 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:23:41.691 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-17T04:23:41.786 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:23:41.786 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-17T04:23:41.882 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:23:41.883 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-17T04:23:41.978 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:23:41.978 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-17T04:23:42.074 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:23:42.074 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-17T04:23:42.168 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:23:42.169 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-17T04:23:42.263 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:23:42.263 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-17T04:23:42.300 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:23:42.301 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-17T04:23:42.377 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:23:42.377 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-17T04:23:42.412 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:23:42.412 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-17T04:23:42.487 INFO:tasks.ceph:Adding keys to all mons... 2024-09-17T04:23:42.487 DEBUG:teuthology.orchestra.run.smithi100:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T04:23:42.491 DEBUG:teuthology.orchestra.run.smithi184:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T04:23:42.540 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr.y] 2024-09-17T04:23:42.540 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQArBOlmM4D1DhAA5uUWgRVOMujWqbYzOjFp9A== 2024-09-17T04:23:42.541 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.a] 2024-09-17T04:23:42.541 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQArBOlmV8asJRAAkSgIWqGc1OeSRbs9PEOT8w== 2024-09-17T04:23:42.541 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.c] 2024-09-17T04:23:42.541 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQArBOlmXaOGMxAAT+96g/5fAzPW8Cc2HtQUyA== 2024-09-17T04:23:42.541 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.0] 2024-09-17T04:23:42.541 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAwBOlmofFmAxAARerYkU9B2a9MM9vHa3o4LQ== 2024-09-17T04:23:42.541 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.1] 2024-09-17T04:23:42.541 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAzBOlmbz6VExAAoVqSqXTc1wArKhdA6+NxnA== 2024-09-17T04:23:42.541 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.2] 2024-09-17T04:23:42.541 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQA2BOlmfeSAJBAAchY6XhyDRusc+zqwQ6v6wQ== 2024-09-17T04:23:42.541 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.3] 2024-09-17T04:23:42.541 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQA6BOlm/TQiBBAAHhbs2+RCjAenGR1frhbTmQ== 2024-09-17T04:23:42.542 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr.x] 2024-09-17T04:23:42.542 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQArBOlmfaL7GhAArqLH66RkbBsz+TNOKtPHOA== 2024-09-17T04:23:42.542 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.b] 2024-09-17T04:23:42.542 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAsBOlmpM7ZBRAA1Og785tJwa6NA9zBdmNNQA== 2024-09-17T04:23:42.542 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.d] 2024-09-17T04:23:42.542 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAsBOlmRqC9ExAA23kZ0Ks1lGN2k4GXL8ASrg== 2024-09-17T04:23:42.542 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.4] 2024-09-17T04:23:42.542 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBABOlm7OKvBhAALkaAKEUYN5ChwGejoSF8yA== 2024-09-17T04:23:42.542 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.5] 2024-09-17T04:23:42.542 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBDBOlmQpi1FhAAbA47dg6ckK1PFoCZL0SBEQ== 2024-09-17T04:23:42.542 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.6] 2024-09-17T04:23:42.542 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBGBOlmx/w0JhAAWcPvZkMYtXD/EMKrg/W/4Q== 2024-09-17T04:23:42.542 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.7] 2024-09-17T04:23:42.543 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBJBOlmWHjWNBAAgdul2UKY9g5p7EVi0ZdrFw== 2024-09-17T04:23:42.543 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-09-17T04:23:42.543 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAsBOlmfYg9IBAABP1uRKGOugrA47VxwRLkjA== 2024-09-17T04:23:42.543 INFO:teuthology.orchestra.run.smithi100.stdout:[client.2] 2024-09-17T04:23:42.543 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAsBOlmgcm0KhAAGF06VlHjBM/AGxafN6/3aA== 2024-09-17T04:23:42.543 INFO:teuthology.orchestra.run.smithi100.stdout:[client.1] 2024-09-17T04:23:42.543 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAsBOlmByhhNRAAPGaxFCDd8ywFVwwhprrPbQ== 2024-09-17T04:23:42.543 INFO:teuthology.orchestra.run.smithi100.stdout:[client.3] 2024-09-17T04:23:42.543 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAtBOlm9VQ6BBAAXcahu35/uDm8Z+f+3SF0Gg== 2024-09-17T04:23:42.579 INFO:teuthology.orchestra.run.smithi184.stdout:[mgr.y] 2024-09-17T04:23:42.579 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQArBOlmM4D1DhAA5uUWgRVOMujWqbYzOjFp9A== 2024-09-17T04:23:42.579 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.a] 2024-09-17T04:23:42.580 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQArBOlmV8asJRAAkSgIWqGc1OeSRbs9PEOT8w== 2024-09-17T04:23:42.580 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.c] 2024-09-17T04:23:42.580 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQArBOlmXaOGMxAAT+96g/5fAzPW8Cc2HtQUyA== 2024-09-17T04:23:42.580 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.0] 2024-09-17T04:23:42.580 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQAwBOlmofFmAxAARerYkU9B2a9MM9vHa3o4LQ== 2024-09-17T04:23:42.580 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.1] 2024-09-17T04:23:42.580 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQAzBOlmbz6VExAAoVqSqXTc1wArKhdA6+NxnA== 2024-09-17T04:23:42.580 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.2] 2024-09-17T04:23:42.580 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQA2BOlmfeSAJBAAchY6XhyDRusc+zqwQ6v6wQ== 2024-09-17T04:23:42.580 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.3] 2024-09-17T04:23:42.580 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQA6BOlm/TQiBBAAHhbs2+RCjAenGR1frhbTmQ== 2024-09-17T04:23:42.580 INFO:teuthology.orchestra.run.smithi184.stdout:[mgr.x] 2024-09-17T04:23:42.581 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQArBOlmfaL7GhAArqLH66RkbBsz+TNOKtPHOA== 2024-09-17T04:23:42.581 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.b] 2024-09-17T04:23:42.581 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQAsBOlmpM7ZBRAA1Og785tJwa6NA9zBdmNNQA== 2024-09-17T04:23:42.581 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.d] 2024-09-17T04:23:42.581 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQAsBOlmRqC9ExAA23kZ0Ks1lGN2k4GXL8ASrg== 2024-09-17T04:23:42.581 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.4] 2024-09-17T04:23:42.581 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBABOlm7OKvBhAALkaAKEUYN5ChwGejoSF8yA== 2024-09-17T04:23:42.581 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.5] 2024-09-17T04:23:42.581 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBDBOlmQpi1FhAAbA47dg6ckK1PFoCZL0SBEQ== 2024-09-17T04:23:42.581 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.6] 2024-09-17T04:23:42.581 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBGBOlmx/w0JhAAWcPvZkMYtXD/EMKrg/W/4Q== 2024-09-17T04:23:42.581 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.7] 2024-09-17T04:23:42.581 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBJBOlmWHjWNBAAgdul2UKY9g5p7EVi0ZdrFw== 2024-09-17T04:23:42.581 INFO:teuthology.orchestra.run.smithi184.stdout:[client.0] 2024-09-17T04:23:42.582 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQAsBOlmfYg9IBAABP1uRKGOugrA47VxwRLkjA== 2024-09-17T04:23:42.582 INFO:teuthology.orchestra.run.smithi184.stdout:[client.2] 2024-09-17T04:23:42.582 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQAsBOlmgcm0KhAAGF06VlHjBM/AGxafN6/3aA== 2024-09-17T04:23:42.582 INFO:teuthology.orchestra.run.smithi184.stdout:[client.1] 2024-09-17T04:23:42.582 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQAsBOlmByhhNRAAPGaxFCDd8ywFVwwhprrPbQ== 2024-09-17T04:23:42.582 INFO:teuthology.orchestra.run.smithi184.stdout:[client.3] 2024-09-17T04:23:42.582 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQAtBOlm9VQ6BBAAXcahu35/uDm8Z+f+3SF0Gg== 2024-09-17T04:23:42.583 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T04:23:42.588 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T04:23:42.788 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T04:23:42.793 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T04:23:42.957 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T04:23:42.961 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T04:23:43.128 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:23:43.132 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:23:43.294 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:23:43.298 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:23:43.460 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:23:43.464 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:23:43.628 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:23:43.632 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:23:43.796 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T04:23:43.800 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T04:23:43.962 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T04:23:43.965 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T04:23:44.129 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T04:23:44.133 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T04:23:44.296 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:23:44.300 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:23:44.461 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:23:44.465 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:23:44.624 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:23:44.628 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:23:44.787 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:23:44.792 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:23:44.954 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T04:23:44.958 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T04:23:45.122 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T04:23:45.126 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T04:23:45.292 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T04:23:45.296 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T04:23:45.458 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T04:23:45.462 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T04:23:45.626 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-17T04:23:45.626 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-17T04:23:45.682 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T04:23:45.858 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-17T04:23:45.914 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-17T04:23:45.970 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T04:23:46.147 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-17T04:23:46.202 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-17T04:23:46.257 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T04:23:46.432 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-17T04:23:46.488 DEBUG:teuthology.orchestra.run.smithi100:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T04:23:46.492 DEBUG:teuthology.orchestra.run.smithi184:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T04:23:46.527 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-17T04:23:46.527 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-17T04:23:46.527 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-17T04:23:46.569 INFO:tasks.ceph.mon.a:Started 2024-09-17T04:23:46.569 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-17T04:23:46.570 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-17T04:23:46.573 INFO:tasks.ceph.mon.b:Started 2024-09-17T04:23:46.574 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-17T04:23:46.574 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-17T04:23:46.577 INFO:tasks.ceph.mon.c:Started 2024-09-17T04:23:46.577 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-17T04:23:46.578 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-17T04:23:46.578 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-17T04:23:46.581 INFO:tasks.ceph.mgr.y:Started 2024-09-17T04:23:46.581 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-17T04:23:46.582 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-17T04:23:46.585 INFO:tasks.ceph.mgr.x:Started 2024-09-17T04:23:46.586 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-17T04:23:46.586 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-17T04:23:46.590 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-17T04:23:46.590 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-17T04:23:46.635 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-17T04:23:46.636 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-17T04:23:46.683 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-17T04:23:46.683 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-17T04:23:46.730 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-17T04:23:46.730 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-17T04:23:46.776 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-17T04:23:46.776 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-17T04:23:46.825 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-17T04:23:46.826 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-17T04:23:46.872 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-17T04:23:46.872 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-17T04:23:46.929 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-17T04:23:46.929 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-17T04:23:46.980 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-17T04:23:46.980 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-17T04:23:47.030 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-17T04:23:47.031 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-17T04:23:47.089 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-17T04:23:47.089 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-17T04:23:47.132 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-17T04:23:47.132 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-17T04:23:47.183 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-17T04:23:47.183 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-17T04:23:47.237 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-17T04:23:47.237 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-17T04:23:47.288 INFO:tasks.ceph:setting config [mgr] client_mount_timeout = 30 2024-09-17T04:23:47.289 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph config set mgr client_mount_timeout 30 2024-09-17T04:23:47.354 INFO:tasks.ceph:setting config [mgr] debug_client = 20 2024-09-17T04:23:47.354 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph config set mgr debug_client 20 2024-09-17T04:23:47.411 INFO:tasks.ceph:setting config [mgr] debug_mgr = 20 2024-09-17T04:23:47.411 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph config set mgr debug_mgr 20 2024-09-17T04:23:47.471 INFO:tasks.ceph:setting config [mgr] debug_ms = 1 2024-09-17T04:23:47.471 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph config set mgr debug_ms 1 2024-09-17T04:23:47.496 INFO:tasks.ceph:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-17T04:23:47.496 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph config set mgr mon_warn_on_pool_no_app False 2024-09-17T04:23:47.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.508+0000 7f2c24400640 1 -- 172.21.15.100:0/1869551247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c1c13b6a0 msgr2=0x7f2c1c13e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:47.513 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-17T04:23:47.513 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-17T04:23:47.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.508+0000 7f2c24400640 1 --2- 172.21.15.100:0/1869551247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c1c13b6a0 0x7f2c1c13e1a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400da90 tx=0x7f2c1400df50 comp rx=0 tx=0).stop 2024-09-17T04:23:47.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.522+0000 7f2c24400640 1 -- 172.21.15.100:0/1869551247 shutdown_connections 2024-09-17T04:23:47.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.522+0000 7f2c24400640 1 --2- 172.21.15.100:0/1869551247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c1c13b6a0 0x7f2c1c13e1a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.522+0000 7f2c24400640 1 --2- 172.21.15.100:0/1869551247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c1c0b3480 0x7f2c1c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.522+0000 7f2c24400640 1 --2- 172.21.15.100:0/1869551247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c1c073840 0x7f2c1c15e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.522+0000 7f2c24400640 1 -- 172.21.15.100:0/1869551247 >> 172.21.15.100:0/1869551247 conn(0x7f2c1c06cda0 msgr2=0x7f2c1c161450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:47.532 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-17T04:23:47.532 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-17T04:23:47.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.522+0000 7f2c24400640 1 -- 172.21.15.100:0/1869551247 shutdown_connections 2024-09-17T04:23:47.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.522+0000 7f2c24400640 1 -- 172.21.15.100:0/1869551247 wait complete. 2024-09-17T04:23:47.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.530+0000 7f776be00640 1 -- 172.21.15.100:0/2409707440 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f7738005310 con 0x7f77640d5150 2024-09-17T04:23:47.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.529+0000 7f18f3400640 1 -- 172.21.15.100:0/1404014784 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f19000a3080 con 0x7f18f8037f50 2024-09-17T04:23:47.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.536+0000 7f85aaa00640 1 -- 172.21.15.100:0/3660163129 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac006310 con 0x7f85b40ccb50 2024-09-17T04:23:47.555 DEBUG:tasks.ceph:set 22 configs 2024-09-17T04:23:47.555 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi100') 2024-09-17T04:23:47.555 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi100') 2024-09-17T04:23:47.556 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi100') 2024-09-17T04:23:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.544+0000 7f85a3e00640 1 -- 172.21.15.100:0/3660163129 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b40b49d0 con 0x7f85b40ccb50 2024-09-17T04:23:47.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.557+0000 7f85aaa00640 1 -- 172.21.15.100:0/3660163129 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f85ac018020 con 0x7f85b40ccb50 2024-09-17T04:23:47.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.557+0000 7f85aaa00640 1 -- 172.21.15.100:0/3660163129 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f85ac01a610 con 0x7f85b40ccb50 2024-09-17T04:23:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.564+0000 7f7761600640 1 -- 172.21.15.100:0/2409707440 <== mon.2 v2:172.21.15.184:3301/0 10 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f775803a850 con 0x7f77640d5150 2024-09-17T04:23:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.564+0000 7f85aaa00640 1 -- 172.21.15.100:0/3660163129 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f85ac01a9d0 con 0x7f85b40ccb50 2024-09-17T04:23:47.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.576+0000 7f7761600640 1 -- 172.21.15.100:0/2409707440 <== mon.2 v2:172.21.15.184:3301/0 11 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v6)=0 v6) ==== 123+0+0 (secure 0 0 0) 0x7f775803aa90 con 0x7f77640d5150 2024-09-17T04:23:47.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.583+0000 7f18f3400640 1 -- 172.21.15.100:0/1404014784 <== mon.0 v2:172.21.15.100:3300/0 7 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1900094ca0 con 0x7f18f8037f50 2024-09-17T04:23:47.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.581+0000 7f776be00640 1 -- 172.21.15.100:0/2409707440 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77640d5150 msgr2=0x7f77640b1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:47.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.581+0000 7f776be00640 1 --2- 172.21.15.100:0/2409707440 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77640d5150 0x7f77640b1240 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f775802a5c0 tx=0x7f77580046c0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.585+0000 7f776be00640 1 -- 172.21.15.100:0/2409707440 shutdown_connections 2024-09-17T04:23:47.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.585+0000 7f776be00640 1 --2- 172.21.15.100:0/2409707440 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77640d5150 0x7f77640b1240 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.585+0000 7f776be00640 1 --2- 172.21.15.100:0/2409707440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77640d0730 0x7f7764156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.585+0000 7f776be00640 1 --2- 172.21.15.100:0/2409707440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f776406dea0 0x7f776415ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.585+0000 7f776be00640 1 -- 172.21.15.100:0/2409707440 >> 172.21.15.100:0/2409707440 conn(0x7f776408bfe0 msgr2=0x7f7764163080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:47.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.586+0000 7f776be00640 1 -- 172.21.15.100:0/2409707440 shutdown_connections 2024-09-17T04:23:47.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.586+0000 7f776be00640 1 -- 172.21.15.100:0/2409707440 wait complete. 2024-09-17T04:23:47.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.566+0000 7f45f9600640 1 -- 172.21.15.100:0/2315321773 <== mon.2 v2:172.21.15.184:3301/0 9 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45e800da10 con 0x7f45fc0cec20 2024-09-17T04:23:47.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.594+0000 7f85aaa00640 1 -- 172.21.15.100:0/3660163129 <== mon.0 v2:172.21.15.100:3300/0 7 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85ac01abb0 con 0x7f85b40ccb50 2024-09-17T04:23:47.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.596+0000 7fefe3e00640 1 -- 172.21.15.100:0/2993432536 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefe4074d40 msgr2=0x7fefe415f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:47.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.600+0000 7fefe1600640 1 -- 172.21.15.100:0/2993432536 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd0035020 con 0x7fefe4074d40 2024-09-17T04:23:47.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.600+0000 7fefe3e00640 1 --2- 172.21.15.100:0/2993432536 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefe4074d40 0x7fefe415f200 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fefd0005fd0 tx=0x7fefd002a5d0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.600+0000 7fefe3e00640 1 -- 172.21.15.100:0/2993432536 shutdown_connections 2024-09-17T04:23:47.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.600+0000 7fefe3e00640 1 --2- 172.21.15.100:0/2993432536 >> v2:172.21.15.100:3300/0 conn(0x7fefe415fb50 0x7fefe4162c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.600+0000 7fefe3e00640 1 --2- 172.21.15.100:0/2993432536 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefe4074d40 0x7fefe415f200 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.600+0000 7fefe3e00640 1 -- 172.21.15.100:0/2993432536 >> 172.21.15.100:0/2993432536 conn(0x7fefe40b1d60 msgr2=0x7fefe40b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:47.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.604+0000 7fefe3e00640 1 -- 172.21.15.100:0/2993432536 shutdown_connections 2024-09-17T04:23:47.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.607+0000 7fefe3e00640 1 -- 172.21.15.100:0/2993432536 wait complete. 2024-09-17T04:23:47.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.621+0000 7fefe3e00640 1 Processor -- start 2024-09-17T04:23:47.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.621+0000 7fefe3e00640 1 -- start start 2024-09-17T04:23:47.639 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi100') 2024-09-17T04:23:47.640 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi100') 2024-09-17T04:23:47.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.641+0000 7fefe3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefe4074d40 0x7fefe4148070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:47.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.641+0000 7fefe3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefe415f740 0x7fefe4157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:47.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.641+0000 7fefe3e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefe415fb50 0x7fefe4153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:47.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.641+0000 7fefe3e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fefe4160aa0 con 0x7fefe4074d40 2024-09-17T04:23:47.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.641+0000 7fefe3e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fefe4160920 con 0x7fefe415f740 2024-09-17T04:23:47.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.641+0000 7fefe3e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fefe4160c20 con 0x7fefe415fb50 2024-09-17T04:23:47.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.645+0000 7fefe3400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefe415fb50 0x7fefe4153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:47.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.645+0000 7fefe3400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefe415fb50 0x7fefe4153a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.100:35368/0 (socket says 172.21.15.100:35368) 2024-09-17T04:23:47.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.645+0000 7fefe3400640 1 -- 172.21.15.100:0/3316011418 learned_addr learned my addr 172.21.15.100:0/3316011418 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:23:47.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.649+0000 7fefe2a00640 1 --2- 172.21.15.100:0/3316011418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefe4074d40 0x7fefe4148070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:47.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.649+0000 7fefe2a00640 1 -- 172.21.15.100:0/3316011418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefe415fb50 msgr2=0x7fefe4153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:47.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.650+0000 7fefe2a00640 1 --2- 172.21.15.100:0/3316011418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefe415fb50 0x7fefe4153a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.650+0000 7fefe2a00640 1 -- 172.21.15.100:0/3316011418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefe415f740 msgr2=0x7fefe4157600 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T04:23:47.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.650+0000 7fefe2a00640 1 --2- 172.21.15.100:0/3316011418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefe415f740 0x7fefe4157600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.650+0000 7fefe2a00640 1 -- 172.21.15.100:0/3316011418 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe414e500 con 0x7fefe4074d40 2024-09-17T04:23:47.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.652+0000 7fefe2a00640 1 --2- 172.21.15.100:0/3316011418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefe4074d40 0x7fefe4148070 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fefd0031000 tx=0x7fefd00039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:23:47.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.653+0000 7fefdaa00640 1 -- 172.21.15.100:0/3316011418 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd003e070 con 0x7fefe4074d40 2024-09-17T04:23:47.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.653+0000 7fefdaa00640 1 -- 172.21.15.100:0/3316011418 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefd0004890 con 0x7fefe4074d40 2024-09-17T04:23:47.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.653+0000 7fefdaa00640 1 -- 172.21.15.100:0/3316011418 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd0035040 con 0x7fefe4074d40 2024-09-17T04:23:47.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.653+0000 7fefe3e00640 1 -- 172.21.15.100:0/3316011418 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe41fd4f0 con 0x7fefe4074d40 2024-09-17T04:23:47.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.653+0000 7fefe3e00640 1 -- 172.21.15.100:0/3316011418 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe41fdab0 con 0x7fefe4074d40 2024-09-17T04:23:47.664 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi100') 2024-09-17T04:23:47.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.660+0000 7fefcfe00640 1 -- 172.21.15.100:0/3316011418 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefb8005290 con 0x7fefe4074d40 2024-09-17T04:23:47.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.663+0000 7fefdaa00640 1 -- 172.21.15.100:0/3316011418 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fefd0041070 con 0x7fefe4074d40 2024-09-17T04:23:47.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.664+0000 7fefdaa00640 1 -- 172.21.15.100:0/3316011418 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fefd0043ba0 con 0x7fefe4074d40 2024-09-17T04:23:47.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.675+0000 7fefdaa00640 1 -- 172.21.15.100:0/3316011418 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fefd0004ce0 con 0x7fefe4074d40 2024-09-17T04:23:47.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.763+0000 7f4604600640 1 -- 172.21.15.100:0/2315321773 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f45d00052f0 con 0x7f45fc0cec20 2024-09-17T04:23:47.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.777+0000 7fe195e00640 1 -- 172.21.15.100:0/1678687687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe174004800 msgr2=0x7fe174004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:47.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.777+0000 7fe195e00640 1 --2- 172.21.15.100:0/1678687687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe174004800 0x7fe174004c10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00c310 tx=0x7fe174005830 comp rx=0 tx=0).stop 2024-09-17T04:23:47.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.777+0000 7fe18f400640 1 -- 172.21.15.100:0/1678687687 reap_dead start 2024-09-17T04:23:47.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.781+0000 7fe195e00640 1 -- 172.21.15.100:0/1678687687 shutdown_connections 2024-09-17T04:23:47.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.781+0000 7fe195e00640 1 -- 172.21.15.100:0/1678687687 >> 172.21.15.100:0/1678687687 conn(0x7fe1900d46e0 msgr2=0x7fe1900d4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:47.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.781+0000 7fe195e00640 1 -- 172.21.15.100:0/1678687687 shutdown_connections 2024-09-17T04:23:47.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.781+0000 7fe195e00640 1 -- 172.21.15.100:0/1678687687 wait complete. 2024-09-17T04:23:47.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.790+0000 7fe195e00640 1 Processor -- start 2024-09-17T04:23:47.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.791+0000 7f45f9600640 1 -- 172.21.15.100:0/2315321773 <== mon.2 v2:172.21.15.184:3301/0 10 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v7)=0 v7) ==== 108+0+0 (secure 0 0 0) 0x7f45e8013030 con 0x7f45fc0cec20 2024-09-17T04:23:47.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.791+0000 7fe195e00640 1 -- start start 2024-09-17T04:23:47.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.795+0000 7f18f0c00640 1 -- 172.21.15.100:0/1404014784 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f18d4005180 con 0x7f18f8037f50 2024-09-17T04:23:47.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.795+0000 7f45eea00640 1 -- 172.21.15.100:0/2315321773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45fc0cec20 msgr2=0x7f45fc1577d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:47.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.795+0000 7f45eea00640 1 --2- 172.21.15.100:0/2315321773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45fc0cec20 0x7f45fc1577d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f45e800b590 tx=0x7f45e800ba50 comp rx=0 tx=0).stop 2024-09-17T04:23:47.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.796+0000 7f45eea00640 1 -- 172.21.15.100:0/2315321773 shutdown_connections 2024-09-17T04:23:47.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.796+0000 7f45eea00640 1 --2- 172.21.15.100:0/2315321773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45fc0d0480 0x7f45fc0c8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.796+0000 7f45eea00640 1 --2- 172.21.15.100:0/2315321773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45fc0cff60 0x7f45fc0c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.796+0000 7f45eea00640 1 --2- 172.21.15.100:0/2315321773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45fc0cec20 0x7f45fc1577d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.796+0000 7f45eea00640 1 -- 172.21.15.100:0/2315321773 >> 172.21.15.100:0/2315321773 conn(0x7f45fc08dd30 msgr2=0x7f45fc0d4680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:47.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.796+0000 7f45eea00640 1 -- 172.21.15.100:0/2315321773 shutdown_connections 2024-09-17T04:23:47.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.797+0000 7f45eea00640 1 -- 172.21.15.100:0/2315321773 wait complete. 2024-09-17T04:23:47.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.798+0000 7fe195e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe174004800 0x7fe19015b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:47.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.802+0000 7fe195e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1900cc130 0x7fe19015fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:47.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.802+0000 7fe18ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe174004800 0x7fe19015b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:47.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.802+0000 7fe18ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe174004800 0x7fe19015b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.100:39262/0 (socket says 172.21.15.100:39262) 2024-09-17T04:23:47.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.802+0000 7fe18ea00640 1 -- 172.21.15.100:0/4183399031 learned_addr learned my addr 172.21.15.100:0/4183399031 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:23:47.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.802+0000 7fe195e00640 1 --2- 172.21.15.100:0/4183399031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1900ce740 0x7fe1901602e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:47.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.802+0000 7fe18e000640 1 --2- 172.21.15.100:0/4183399031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1900cc130 0x7fe19015fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:47.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.803+0000 7fe195e00640 1 -- 172.21.15.100:0/4183399031 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe1900ba170 con 0x7fe1900ce740 2024-09-17T04:23:47.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.803+0000 7fe195e00640 1 -- 172.21.15.100:0/4183399031 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe1900b9ff0 con 0x7fe174004800 2024-09-17T04:23:47.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.803+0000 7fe195e00640 1 -- 172.21.15.100:0/4183399031 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe1900cf850 con 0x7fe1900cc130 2024-09-17T04:23:47.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.803+0000 7fe18f400640 1 --2- 172.21.15.100:0/4183399031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1900ce740 0x7fe1901602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:47.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.804+0000 7fe18f400640 1 -- 172.21.15.100:0/4183399031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1900cc130 msgr2=0x7fe19015fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:47.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.804+0000 7fe18f400640 1 --2- 172.21.15.100:0/4183399031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1900cc130 0x7fe19015fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.804+0000 7fe18f400640 1 -- 172.21.15.100:0/4183399031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe174004800 msgr2=0x7fe19015b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:47.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.804+0000 7fe18f400640 1 --2- 172.21.15.100:0/4183399031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe174004800 0x7fe19015b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.804+0000 7fe18f400640 1 -- 172.21.15.100:0/4183399031 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1900a2380 con 0x7fe1900ce740 2024-09-17T04:23:47.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.805+0000 7fe18f400640 1 --2- 172.21.15.100:0/4183399031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1900ce740 0x7fe1901602e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe1900d0b00 tx=0x7fe18800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:23:47.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.806+0000 7fe187400640 1 -- 172.21.15.100:0/4183399031 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe188003b70 con 0x7fe1900ce740 2024-09-17T04:23:47.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.806+0000 7fe18ea00640 1 --2- 172.21.15.100:0/4183399031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe174004800 0x7fe19015b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:23:47.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.807+0000 7fe18e000640 1 --2- 172.21.15.100:0/4183399031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1900cc130 0x7fe19015fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:23:47.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.807+0000 7fe195e00640 1 -- 172.21.15.100:0/4183399031 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1900a20a0 con 0x7fe1900ce740 2024-09-17T04:23:47.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.807+0000 7fe187400640 1 -- 172.21.15.100:0/4183399031 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe188003d10 con 0x7fe1900ce740 2024-09-17T04:23:47.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.807+0000 7fe187400640 1 -- 172.21.15.100:0/4183399031 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe188003b70 con 0x7fe1900ce740 2024-09-17T04:23:47.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.812+0000 7fe195e00640 1 -- 172.21.15.100:0/4183399031 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1901f9ef0 con 0x7fe1900ce740 2024-09-17T04:23:47.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.814+0000 7f18f3400640 1 -- 172.21.15.100:0/1404014784 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v8)=0 v8) ==== 117+0+0 (secure 0 0 0) 0x7f190009e3f0 con 0x7f18f8037f50 2024-09-17T04:23:47.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.815+0000 7f18f0c00640 1 -- 172.21.15.100:0/1404014784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18f8037f50 msgr2=0x7f18f80e89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:47.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.815+0000 7f18f0c00640 1 --2- 172.21.15.100:0/1404014784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18f8037f50 0x7f18f80e89d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1900077030 tx=0x7f190006a030 comp rx=0 tx=0).stop 2024-09-17T04:23:47.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.816+0000 7f18f0c00640 1 -- 172.21.15.100:0/1404014784 shutdown_connections 2024-09-17T04:23:47.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.816+0000 7f18f0c00640 1 --2- 172.21.15.100:0/1404014784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18f8037f50 0x7f18f80e89d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.816+0000 7f18f0c00640 1 --2- 172.21.15.100:0/1404014784 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18f8007350 0x7f18f8051830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.816+0000 7f18f0c00640 1 --2- 172.21.15.100:0/1404014784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18f8006f40 0x7f18f804c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.816+0000 7f18f0c00640 1 -- 172.21.15.100:0/1404014784 >> 172.21.15.100:0/1404014784 conn(0x7f18f800e220 msgr2=0x7f18f803cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:47.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.816+0000 7f18f0c00640 1 -- 172.21.15.100:0/1404014784 shutdown_connections 2024-09-17T04:23:47.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.816+0000 7f18f0c00640 1 -- 172.21.15.100:0/1404014784 wait complete. 2024-09-17T04:23:47.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.818+0000 7fe187400640 1 -- 172.21.15.100:0/4183399031 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe188004b20 con 0x7fe1900ce740 2024-09-17T04:23:47.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.818+0000 7fe187400640 1 -- 172.21.15.100:0/4183399031 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe188004100 con 0x7fe1900ce740 2024-09-17T04:23:47.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.821+0000 7fe195e00640 1 -- 172.21.15.100:0/4183399031 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1900a1e10 con 0x7fe1900ce740 2024-09-17T04:23:47.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.827+0000 7f7a8be00640 1 -- 172.21.15.100:0/736575686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a8c15a530 msgr2=0x7f7a8c15a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:47.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.827+0000 7f7a8be00640 1 --2- 172.21.15.100:0/736575686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a8c15a530 0x7f7a8c15a940 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c005fd0 tx=0x7f7a7c02a4f0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.827+0000 7f7a8be00640 1 -- 172.21.15.100:0/736575686 shutdown_connections 2024-09-17T04:23:47.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.827+0000 7f7a8be00640 1 --2- 172.21.15.100:0/736575686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a8c15a530 0x7f7a8c15a940 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.827+0000 7f7a8be00640 1 --2- 172.21.15.100:0/736575686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a8c0b0940 0x7f7a8c159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.827+0000 7f7a8be00640 1 -- 172.21.15.100:0/736575686 >> 172.21.15.100:0/736575686 conn(0x7f7a8c0742a0 msgr2=0x7f7a8c0746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:47.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.831+0000 7fe187400640 1 -- 172.21.15.100:0/4183399031 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fe188027070 con 0x7fe1900ce740 2024-09-17T04:23:47.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.850+0000 7f85a3e00640 1 -- 172.21.15.100:0/3660163129 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f85b40008d0 con 0x7f85b40ccb50 2024-09-17T04:23:47.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.858+0000 7f7a8be00640 1 -- 172.21.15.100:0/736575686 shutdown_connections 2024-09-17T04:23:47.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.875+0000 7f85aaa00640 1 -- 172.21.15.100:0/3660163129 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v9)=0 v9) ==== 106+0+0 (secure 0 0 0) 0x7f85ac01ad80 con 0x7f85b40ccb50 2024-09-17T04:23:47.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.864+0000 7f7a8be00640 1 -- 172.21.15.100:0/736575686 wait complete. 2024-09-17T04:23:47.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.886+0000 7f85b8a00640 1 -- 172.21.15.100:0/3660163129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b40ccb50 msgr2=0x7f85b409cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:47.889 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi100') 2024-09-17T04:23:47.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.893+0000 7f85b8a00640 1 --2- 172.21.15.100:0/3660163129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b40ccb50 0x7f85b409cf40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00c520 tx=0x7f85ac00c9e0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.897+0000 7f7a8be00640 1 Processor -- start 2024-09-17T04:23:47.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.901+0000 7f7a8be00640 1 -- start start 2024-09-17T04:23:47.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.912+0000 7f85b8a00640 1 -- 172.21.15.100:0/3660163129 shutdown_connections 2024-09-17T04:23:47.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.912+0000 7f85b8a00640 1 --2- 172.21.15.100:0/3660163129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b40ccb50 0x7f85b409cf40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.912+0000 7f85b8a00640 1 --2- 172.21.15.100:0/3660163129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85b40b3900 0x7f85b4209890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.912+0000 7f85b8a00640 1 --2- 172.21.15.100:0/3660163129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85b4074dc0 0x7f85b4213ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.912+0000 7f85b8a00640 1 -- 172.21.15.100:0/3660163129 >> 172.21.15.100:0/3660163129 conn(0x7f85b40742a0 msgr2=0x7f85b4070790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:47.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.916+0000 7f85b8a00640 1 -- 172.21.15.100:0/3660163129 shutdown_connections 2024-09-17T04:23:47.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.916+0000 7f85b8a00640 1 -- 172.21.15.100:0/3660163129 wait complete. 2024-09-17T04:23:47.922 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi100') 2024-09-17T04:23:47.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.922+0000 7f7a8be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a8c0ad0b0 0x7f7a8c099910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:47.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.922+0000 7f7a8aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a8c0ad0b0 0x7f7a8c099910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:47.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.923+0000 7f7a8aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a8c0ad0b0 0x7f7a8c099910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36948/0 (socket says 172.21.15.100:36948) 2024-09-17T04:23:47.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.929+0000 7f7a8aa00640 1 -- 172.21.15.100:0/3737930925 learned_addr learned my addr 172.21.15.100:0/3737930925 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:23:47.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.929+0000 7f7a8be00640 1 --2- 172.21.15.100:0/3737930925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a8c0b0940 0x7f7a8c0d0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:47.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.929+0000 7f7a8be00640 1 --2- 172.21.15.100:0/3737930925 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a8c15a530 0x7f7a8c0d0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:47.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.930+0000 7f7a8be00640 1 -- 172.21.15.100:0/3737930925 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c0ccce0 con 0x7f7a8c0ad0b0 2024-09-17T04:23:47.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.930+0000 7f7a8a000640 1 --2- 172.21.15.100:0/3737930925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a8c0b0940 0x7f7a8c0d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:47.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.930+0000 7f7a8be00640 1 -- 172.21.15.100:0/3737930925 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c0adf70 con 0x7f7a8c0b0940 2024-09-17T04:23:47.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.930+0000 7f7a8b400640 1 --2- 172.21.15.100:0/3737930925 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a8c15a530 0x7f7a8c0d0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:47.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.932+0000 7f7a8be00640 1 -- 172.21.15.100:0/3737930925 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7a8c0cce60 con 0x7f7a8c15a530 2024-09-17T04:23:47.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.932+0000 7f7a8a000640 1 -- 172.21.15.100:0/3737930925 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a8c15a530 msgr2=0x7f7a8c0d0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:47.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.932+0000 7f7a8a000640 1 --2- 172.21.15.100:0/3737930925 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a8c15a530 0x7f7a8c0d0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.933+0000 7f7a8a000640 1 -- 172.21.15.100:0/3737930925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a8c0ad0b0 msgr2=0x7f7a8c099910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:47.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.933+0000 7f7a8a000640 1 --2- 172.21.15.100:0/3737930925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a8c0ad0b0 0x7f7a8c099910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.933+0000 7f7a8a000640 1 -- 172.21.15.100:0/3737930925 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c14ff00 con 0x7f7a8c0b0940 2024-09-17T04:23:47.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.933+0000 7f7a8b400640 1 --2- 172.21.15.100:0/3737930925 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a8c15a530 0x7f7a8c0d0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:23:47.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.933+0000 7f7a8aa00640 1 --2- 172.21.15.100:0/3737930925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a8c0ad0b0 0x7f7a8c099910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:23:47.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.934+0000 7f7a8a000640 1 --2- 172.21.15.100:0/3737930925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a8c0b0940 0x7f7a8c0d0320 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c004000 tx=0x7f7a7c004260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:23:47.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.948+0000 7f7a83400640 1 -- 172.21.15.100:0/3737930925 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c041070 con 0x7f7a8c0b0940 2024-09-17T04:23:47.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.948+0000 7f7a83400640 1 -- 172.21.15.100:0/3737930925 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a7c031a00 con 0x7f7a8c0b0940 2024-09-17T04:23:47.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.948+0000 7f7a8be00640 1 -- 172.21.15.100:0/3737930925 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a8c13eba0 con 0x7f7a8c0b0940 2024-09-17T04:23:47.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.948+0000 7f7a83400640 1 -- 172.21.15.100:0/3737930925 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c031e50 con 0x7f7a8c0b0940 2024-09-17T04:23:47.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.950+0000 7f7a8be00640 1 -- 172.21.15.100:0/3737930925 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a8c151a10 con 0x7f7a8c0b0940 2024-09-17T04:23:47.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.962+0000 7f7a83400640 1 -- 172.21.15.100:0/3737930925 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7a7c044070 con 0x7f7a8c0b0940 2024-09-17T04:23:47.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.962+0000 7f7a8be00640 1 -- 172.21.15.100:0/3737930925 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a8c1501a0 con 0x7f7a8c0b0940 2024-09-17T04:23:47.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.966+0000 7f7a83400640 1 -- 172.21.15.100:0/3737930925 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7a7c03b770 con 0x7f7a8c0b0940 2024-09-17T04:23:47.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.969+0000 7f7a83400640 1 -- 172.21.15.100:0/3737930925 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f7a7c0094e0 con 0x7f7a8c0b0940 2024-09-17T04:23:47.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.970+0000 7f627d800640 1 -- 172.21.15.100:0/1950681146 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62780b9660 msgr2=0x7f627806dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:47.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.970+0000 7f627d800640 1 --2- 172.21.15.100:0/1950681146 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62780b9660 0x7f627806dec0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6268009940 tx=0x7f626802a5d0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.980 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi100') 2024-09-17T04:23:47.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.982+0000 7f627d800640 1 -- 172.21.15.100:0/1950681146 shutdown_connections 2024-09-17T04:23:47.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.982+0000 7f627d800640 1 --2- 172.21.15.100:0/1950681146 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62780b9660 0x7f627806dec0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.982+0000 7f627d800640 1 -- 172.21.15.100:0/1950681146 >> 172.21.15.100:0/1950681146 conn(0x7f62780d4550 msgr2=0x7f62780d4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:47.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.983+0000 7f627d800640 1 -- 172.21.15.100:0/1950681146 shutdown_connections 2024-09-17T04:23:47.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.984+0000 7f627d800640 1 -- 172.21.15.100:0/1950681146 wait complete. 2024-09-17T04:23:47.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.991+0000 7f627d800640 1 Processor -- start 2024-09-17T04:23:47.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.992+0000 7f627d800640 1 -- start start 2024-09-17T04:23:47.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.995+0000 7f627d800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62780b3870 0x7f627815a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:47.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.995+0000 7f627d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62780b9660 0x7f627815a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:47.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.996+0000 7f6277e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62780b3870 0x7f627815a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:47.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.996+0000 7f627d800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62780d3570 0x7f627815f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:47.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.996+0000 7f627d800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f62780d3b00 con 0x7f62780b9660 2024-09-17T04:23:47.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.996+0000 7f627d800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f62780d3980 con 0x7f62780b3870 2024-09-17T04:23:47.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.996+0000 7f627d800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f62780b66e0 con 0x7f62780d3570 2024-09-17T04:23:47.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.996+0000 7f6277400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62780b9660 0x7f627815a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:47.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.996+0000 7f6277400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62780b9660 0x7f627815a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36986/0 (socket says 172.21.15.100:36986) 2024-09-17T04:23:47.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.996+0000 7f6277400640 1 -- 172.21.15.100:0/1050177097 learned_addr learned my addr 172.21.15.100:0/1050177097 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:23:47.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.996+0000 7f627ce00640 1 --2- 172.21.15.100:0/1050177097 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62780d3570 0x7f627815f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:47.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.998+0000 7f6277400640 1 -- 172.21.15.100:0/1050177097 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62780d3570 msgr2=0x7f627815f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:47.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.998+0000 7f6277400640 1 --2- 172.21.15.100:0/1050177097 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62780d3570 0x7f627815f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.998+0000 7f6277400640 1 -- 172.21.15.100:0/1050177097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62780b3870 msgr2=0x7f627815a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:47.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.998+0000 7f6277400640 1 --2- 172.21.15.100:0/1050177097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62780b3870 0x7f627815a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:47.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.998+0000 7f6277400640 1 -- 172.21.15.100:0/1050177097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62780cd540 con 0x7f62780b9660 2024-09-17T04:23:47.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:47.998+0000 7f6277e00640 1 --2- 172.21.15.100:0/1050177097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62780b3870 0x7f627815a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:23:48.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.001+0000 7fcc6e000640 1 -- 172.21.15.100:0/584324422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc50003a50 msgr2=0x7fcc50003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.001+0000 7fcc6e000640 1 --2- 172.21.15.100:0/584324422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc50003a50 0x7fcc50003ec0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcc5400c260 tx=0x7fcc50005830 comp rx=0 tx=0).stop 2024-09-17T04:23:48.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.005+0000 7fcc6e000640 1 -- 172.21.15.100:0/584324422 shutdown_connections 2024-09-17T04:23:48.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.005+0000 7fcc6e000640 1 --2- 172.21.15.100:0/584324422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc50003a50 0x7fcc50003ec0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.005+0000 7fcc6e000640 1 -- 172.21.15.100:0/584324422 >> 172.21.15.100:0/584324422 conn(0x7fcc680742a0 msgr2=0x7fcc680746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:48.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.009+0000 7f8540a00640 1 -- 172.21.15.100:0/1180332263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f853c15a940 msgr2=0x7f853c0ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.009+0000 7f8540a00640 1 --2- 172.21.15.100:0/1180332263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f853c15a940 0x7f853c0ad7c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8528009830 tx=0x7f852802a360 comp rx=0 tx=0).stop 2024-09-17T04:23:48.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.010+0000 7f8540a00640 1 -- 172.21.15.100:0/1180332263 shutdown_connections 2024-09-17T04:23:48.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.010+0000 7f8540a00640 1 --2- 172.21.15.100:0/1180332263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f853c15a940 0x7f853c0ad7c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.010+0000 7f8540a00640 1 --2- 172.21.15.100:0/1180332263 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f853c0b0940 0x7f853c159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.010+0000 7f8540a00640 1 -- 172.21.15.100:0/1180332263 >> 172.21.15.100:0/1180332263 conn(0x7f853c0742a0 msgr2=0x7f853c0746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:48.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.010+0000 7f8540a00640 1 -- 172.21.15.100:0/1180332263 shutdown_connections 2024-09-17T04:23:48.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.014+0000 7f8540a00640 1 -- 172.21.15.100:0/1180332263 wait complete. 2024-09-17T04:23:48.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.024+0000 7f6277400640 1 --2- 172.21.15.100:0/1050177097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62780b9660 0x7f627815a5a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f626400b410 tx=0x7f626400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:23:48.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.027+0000 7f626fe00640 1 -- 172.21.15.100:0/1050177097 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6264015020 con 0x7f62780b9660 2024-09-17T04:23:48.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.027+0000 7f626fe00640 1 -- 172.21.15.100:0/1050177097 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6264011940 con 0x7f62780b9660 2024-09-17T04:23:48.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.029+0000 7f8540a00640 1 Processor -- start 2024-09-17T04:23:48.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.029+0000 7f8540a00640 1 -- start start 2024-09-17T04:23:48.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.033+0000 7f626fe00640 1 -- 172.21.15.100:0/1050177097 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6264011d90 con 0x7f62780b9660 2024-09-17T04:23:48.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.033+0000 7f627d800640 1 -- 172.21.15.100:0/1050177097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62780cd820 con 0x7f62780b9660 2024-09-17T04:23:48.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.033+0000 7f627d800640 1 -- 172.21.15.100:0/1050177097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62780cdde0 con 0x7f62780b9660 2024-09-17T04:23:48.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.037+0000 7fefcfe00640 1 -- 172.21.15.100:0/3316011418 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7fefb8005850 con 0x7fefe4074d40 2024-09-17T04:23:48.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.039+0000 7f626fe00640 1 -- 172.21.15.100:0/1050177097 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6264018020 con 0x7f62780b9660 2024-09-17T04:23:48.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.039+0000 7f626fe00640 1 -- 172.21.15.100:0/1050177097 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f626401a5d0 con 0x7f62780b9660 2024-09-17T04:23:48.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.039+0000 7fcc6e000640 1 -- 172.21.15.100:0/584324422 shutdown_connections 2024-09-17T04:23:48.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.039+0000 7fcc6e000640 1 -- 172.21.15.100:0/584324422 wait complete. 2024-09-17T04:23:48.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.043+0000 7f8540a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f853c0b0940 0x7f853c153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.043+0000 7f8540a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f853c15a530 0x7f853c1570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.043+0000 7f8540a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f853c15a940 0x7f853c2000e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.039+0000 7f627d800640 1 -- 172.21.15.100:0/1050177097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62780b6c00 con 0x7f62780b9660 2024-09-17T04:23:48.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.046+0000 7f8539e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f853c15a530 0x7f853c1570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:48.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.047+0000 7f8539e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f853c15a530 0x7f853c1570d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.100:35474/0 (socket says 172.21.15.100:35474) 2024-09-17T04:23:48.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.047+0000 7f8539e00640 1 -- 172.21.15.100:0/173645097 learned_addr learned my addr 172.21.15.100:0/173645097 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:23:48.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.047+0000 7f853a800640 1 --2- 172.21.15.100:0/173645097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f853c0b0940 0x7f853c153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:48.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.048+0000 7f8540a00640 1 -- 172.21.15.100:0/173645097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f853c0cce60 con 0x7f853c15a940 2024-09-17T04:23:48.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.048+0000 7f8540a00640 1 -- 172.21.15.100:0/173645097 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f853c0ccce0 con 0x7f853c0b0940 2024-09-17T04:23:48.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.048+0000 7f8540a00640 1 -- 172.21.15.100:0/173645097 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f853c0ccfe0 con 0x7f853c15a530 2024-09-17T04:23:48.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.048+0000 7f853b200640 1 --2- 172.21.15.100:0/173645097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f853c15a940 0x7f853c2000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:48.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.049+0000 7f626fe00640 1 -- 172.21.15.100:0/1050177097 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f626401f070 con 0x7f62780b9660 2024-09-17T04:23:48.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.049+0000 7fcc6e000640 1 Processor -- start 2024-09-17T04:23:48.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.048+0000 7f853a800640 1 -- 172.21.15.100:0/173645097 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f853c15a530 msgr2=0x7f853c1570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.048+0000 7f853a800640 1 --2- 172.21.15.100:0/173645097 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f853c15a530 0x7f853c1570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.048+0000 7f853a800640 1 -- 172.21.15.100:0/173645097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f853c15a940 msgr2=0x7f853c2000e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.048+0000 7f853a800640 1 --2- 172.21.15.100:0/173645097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f853c15a940 0x7f853c2000e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.048+0000 7f853a800640 1 -- 172.21.15.100:0/173645097 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853c2062e0 con 0x7f853c0b0940 2024-09-17T04:23:48.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.052+0000 7f853a800640 1 --2- 172.21.15.100:0/173645097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f853c0b0940 0x7f853c153940 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f853400ee40 tx=0x7f853400c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:23:48.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.053+0000 7fcc6e000640 1 -- start start 2024-09-17T04:23:48.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.055+0000 7f8532a00640 1 -- 172.21.15.100:0/173645097 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853401b070 con 0x7f853c0b0940 2024-09-17T04:23:48.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.055+0000 7f8532a00640 1 -- 172.21.15.100:0/173645097 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f853400b570 con 0x7f853c0b0940 2024-09-17T04:23:48.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.055+0000 7f8532a00640 1 -- 172.21.15.100:0/173645097 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853400b9c0 con 0x7f853c0b0940 2024-09-17T04:23:48.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.055+0000 7f8540a00640 1 -- 172.21.15.100:0/173645097 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f853c1556c0 con 0x7f853c0b0940 2024-09-17T04:23:48.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.055+0000 7f8540a00640 1 -- 172.21.15.100:0/173645097 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f853c206830 con 0x7f853c0b0940 2024-09-17T04:23:48.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.062+0000 7fefdaa00640 1 -- 172.21.15.100:0/3316011418 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v10)=0 v10) ==== 115+0+0 (secure 0 0 0) 0x7fefd0004ec0 con 0x7fefe4074d40 2024-09-17T04:23:48.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.060+0000 7f8532a00640 1 -- 172.21.15.100:0/173645097 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f853401e070 con 0x7f853c0b0940 2024-09-17T04:23:48.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.060+0000 7f8532a00640 1 -- 172.21.15.100:0/173645097 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8534020cb0 con 0x7f853c0b0940 2024-09-17T04:23:48.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.064+0000 7f8527e00640 1 -- 172.21.15.100:0/173645097 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f853c155890 con 0x7f853c0b0940 2024-09-17T04:23:48.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.067+0000 7fcc6e000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc6815ccf0 0x7fcc68154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.067+0000 7fcc6e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc68158630 0x7fcc68158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.067+0000 7fcc6e000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc6806faa0 0x7fcc6806feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.068+0000 7f8532a00640 1 -- 172.21.15.100:0/173645097 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f853400bbc0 con 0x7f853c0b0940 2024-09-17T04:23:48.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.067+0000 7fcc6e000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcc680cd040 con 0x7fcc68158630 2024-09-17T04:23:48.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.067+0000 7fcc6e000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcc680ccec0 con 0x7fcc6806faa0 2024-09-17T04:23:48.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.067+0000 7fcc6e000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcc680cd1c0 con 0x7fcc6815ccf0 2024-09-17T04:23:48.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.067+0000 7fcc6cc00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc6815ccf0 0x7fcc68154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:48.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.067+0000 7fcc6cc00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc6815ccf0 0x7fcc68154ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.100:35488/0 (socket says 172.21.15.100:35488) 2024-09-17T04:23:48.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.067+0000 7fcc6cc00640 1 -- 172.21.15.100:0/3903388111 learned_addr learned my addr 172.21.15.100:0/3903388111 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:23:48.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.068+0000 7fcc6cc00640 1 -- 172.21.15.100:0/3903388111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc6806faa0 msgr2=0x7fcc6806feb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T04:23:48.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.068+0000 7fcc6cc00640 1 --2- 172.21.15.100:0/3903388111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc6806faa0 0x7fcc6806feb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.068+0000 7fcc6cc00640 1 -- 172.21.15.100:0/3903388111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc68158630 msgr2=0x7fcc68158a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T04:23:48.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.069+0000 7fcc67e00640 1 --2- 172.21.15.100:0/3903388111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc68158630 0x7fcc68158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:48.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.072+0000 7fefcfe00640 1 -- 172.21.15.100:0/3316011418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefe4074d40 msgr2=0x7fefe4148070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.072+0000 7fefcfe00640 1 --2- 172.21.15.100:0/3316011418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefe4074d40 0x7fefe4148070 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fefd0031000 tx=0x7fefd00039e0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.069+0000 7fcc6cc00640 1 --2- 172.21.15.100:0/3903388111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc68158630 0x7fcc68158a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.069+0000 7fcc6cc00640 1 -- 172.21.15.100:0/3903388111 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc680703f0 con 0x7fcc6815ccf0 2024-09-17T04:23:48.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.072+0000 7fefcfe00640 1 -- 172.21.15.100:0/3316011418 shutdown_connections 2024-09-17T04:23:48.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.072+0000 7fefcfe00640 1 --2- 172.21.15.100:0/3316011418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefe415fb50 0x7fefe4153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.072+0000 7fefcfe00640 1 --2- 172.21.15.100:0/3316011418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefe415f740 0x7fefe4157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.072+0000 7fefcfe00640 1 --2- 172.21.15.100:0/3316011418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefe4074d40 0x7fefe4148070 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.072+0000 7fefcfe00640 1 -- 172.21.15.100:0/3316011418 >> 172.21.15.100:0/3316011418 conn(0x7fefe40b1d60 msgr2=0x7fefe40cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:48.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.072+0000 7fcc67e00640 1 --2- 172.21.15.100:0/3903388111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc68158630 0x7fcc68158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:23:48.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.073+0000 7fefcfe00640 1 -- 172.21.15.100:0/3316011418 shutdown_connections 2024-09-17T04:23:48.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.073+0000 7fefcfe00640 1 -- 172.21.15.100:0/3316011418 wait complete. 2024-09-17T04:23:48.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.104+0000 7fcc6cc00640 1 --2- 172.21.15.100:0/3903388111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc6815ccf0 0x7fcc68154ad0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcc54000c00 tx=0x7fcc54001720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:23:48.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.104+0000 7fcc64c00640 1 -- 172.21.15.100:0/3903388111 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc54031040 con 0x7fcc6815ccf0 2024-09-17T04:23:48.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.104+0000 7fcc64c00640 1 -- 172.21.15.100:0/3903388111 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc54003800 con 0x7fcc6815ccf0 2024-09-17T04:23:48.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.104+0000 7fcc64c00640 1 -- 172.21.15.100:0/3903388111 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc54031040 con 0x7fcc6815ccf0 2024-09-17T04:23:48.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.105+0000 7fcc6e000640 1 -- 172.21.15.100:0/3903388111 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc681f5060 con 0x7fcc6815ccf0 2024-09-17T04:23:48.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.106+0000 7fcc6e000640 1 -- 172.21.15.100:0/3903388111 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc681f5620 con 0x7fcc6815ccf0 2024-09-17T04:23:48.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.111+0000 7fcc64c00640 1 -- 172.21.15.100:0/3903388111 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcc54034070 con 0x7fcc6815ccf0 2024-09-17T04:23:48.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.111+0000 7fcc6e000640 1 -- 172.21.15.100:0/3903388111 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc680cd720 con 0x7fcc6815ccf0 2024-09-17T04:23:48.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.111+0000 7fcc64c00640 1 -- 172.21.15.100:0/3903388111 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcc54036c80 con 0x7fcc6815ccf0 2024-09-17T04:23:48.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.114+0000 7fcc64c00640 1 -- 172.21.15.100:0/3903388111 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fcc5404f080 con 0x7fcc6815ccf0 2024-09-17T04:23:48.162 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi100') 2024-09-17T04:23:48.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.167+0000 7fbdbd800640 1 -- 172.21.15.100:0/365792345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdb80cf160 msgr2=0x7fbdb80cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.167+0000 7fbdbd800640 1 --2- 172.21.15.100:0/365792345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdb80cf160 0x7fbdb80cf570 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80ba260 tx=0x7fbda402a560 comp rx=0 tx=0).stop 2024-09-17T04:23:48.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.167+0000 7fbdbd800640 1 -- 172.21.15.100:0/365792345 shutdown_connections 2024-09-17T04:23:48.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.167+0000 7fbdbd800640 1 --2- 172.21.15.100:0/365792345 >> v2:172.21.15.184:3300/0 conn(0x7fbdb80d0bf0 0x7fbdb80d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.167+0000 7fbdbd800640 1 --2- 172.21.15.100:0/365792345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdb80cf160 0x7fbdb80cf570 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.167+0000 7fbdbd800640 1 -- 172.21.15.100:0/365792345 >> 172.21.15.100:0/365792345 conn(0x7fbdb808bfe0 msgr2=0x7fbdb80d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:48.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.167+0000 7f3d78a00640 1 -- 172.21.15.100:0/1942030366 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d68031600 con 0x7f3d7c1610b0 2024-09-17T04:23:48.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.167+0000 7f3d7b200640 1 -- 172.21.15.100:0/1942030366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d7c1610b0 msgr2=0x7f3d7c0cd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.167+0000 7f3d7b200640 1 --2- 172.21.15.100:0/1942030366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d7c1610b0 0x7f3d7c0cd340 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c073040 tx=0x7f3d6802a4a0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.166+0000 7f5cf5600640 1 -- 172.21.15.100:0/3269437856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cd4004800 msgr2=0x7f5cd4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.166+0000 7f5cf5600640 1 --2- 172.21.15.100:0/3269437856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cd4004800 0x7f5cd4004c10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00c260 tx=0x7f5cd4005830 comp rx=0 tx=0).stop 2024-09-17T04:23:48.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.170+0000 7f5cf5600640 1 -- 172.21.15.100:0/3269437856 shutdown_connections 2024-09-17T04:23:48.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.170+0000 7f5cf5600640 1 --2- 172.21.15.100:0/3269437856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cd4004800 0x7f5cd4004c10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.170+0000 7f5cf5600640 1 -- 172.21.15.100:0/3269437856 >> 172.21.15.100:0/3269437856 conn(0x7f5cf00742a0 msgr2=0x7f5cf00746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:48.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.172+0000 7fbdbd800640 1 -- 172.21.15.100:0/365792345 shutdown_connections 2024-09-17T04:23:48.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.173+0000 7fbdbd800640 1 -- 172.21.15.100:0/365792345 wait complete. 2024-09-17T04:23:48.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.171+0000 7f3d7b200640 1 -- 172.21.15.100:0/1942030366 shutdown_connections 2024-09-17T04:23:48.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.171+0000 7f3d7b200640 1 --2- 172.21.15.100:0/1942030366 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d7c1604b0 0x7f3d7c1608c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.171+0000 7f3d7b200640 1 --2- 172.21.15.100:0/1942030366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d7c1610b0 0x7f3d7c0cd340 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.171+0000 7f3d7b200640 1 --2- 172.21.15.100:0/1942030366 >> v2:172.21.15.184:3300/0 conn(0x7f3d7c0cefb0 0x7f3d7c0d0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.171+0000 7f3d7b200640 1 -- 172.21.15.100:0/1942030366 >> 172.21.15.100:0/1942030366 conn(0x7f3d7c0742a0 msgr2=0x7f3d7c0746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:48.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.173+0000 7f3d7b200640 1 -- 172.21.15.100:0/1942030366 shutdown_connections 2024-09-17T04:23:48.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.175+0000 7f3d7b200640 1 -- 172.21.15.100:0/1942030366 wait complete. 2024-09-17T04:23:48.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.175+0000 7f5cf5600640 1 -- 172.21.15.100:0/3269437856 shutdown_connections 2024-09-17T04:23:48.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.176+0000 7f5cf5600640 1 -- 172.21.15.100:0/3269437856 wait complete. 2024-09-17T04:23:48.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.184+0000 7f3d7b200640 1 Processor -- start 2024-09-17T04:23:48.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.182+0000 7f5cf5600640 1 Processor -- start 2024-09-17T04:23:48.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.186+0000 7f761fe00640 1 -- 172.21.15.100:0/3011282665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76200ce7b0 msgr2=0x7f76200b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.186+0000 7f761fe00640 1 --2- 172.21.15.100:0/3011282665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76200ce7b0 0x7f76200b3dd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f76200cdf50 tx=0x7f7618032b90 comp rx=0 tx=0).stop 2024-09-17T04:23:48.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.190+0000 7f3d7b200640 1 -- start start 2024-09-17T04:23:48.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.190+0000 7f5cf5600640 1 -- start start 2024-09-17T04:23:48.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.193+0000 7f761fe00640 1 -- 172.21.15.100:0/3011282665 shutdown_connections 2024-09-17T04:23:48.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.193+0000 7f761fe00640 1 --2- 172.21.15.100:0/3011282665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76200ce7b0 0x7f76200b3dd0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.193+0000 7f761fe00640 1 --2- 172.21.15.100:0/3011282665 >> v2:172.21.15.100:3300/0 conn(0x7f7620074dc0 0x7f762015b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.193+0000 7f761fe00640 1 -- 172.21.15.100:0/3011282665 >> 172.21.15.100:0/3011282665 conn(0x7f76200742a0 msgr2=0x7f76200746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:48.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.193+0000 7f761fe00640 1 -- 172.21.15.100:0/3011282665 shutdown_connections 2024-09-17T04:23:48.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.193+0000 7f761fe00640 1 -- 172.21.15.100:0/3011282665 wait complete. 2024-09-17T04:23:48.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.190+0000 7fbdbd800640 1 Processor -- start 2024-09-17T04:23:48.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.197+0000 7fe195e00640 1 -- 172.21.15.100:0/4183399031 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7fe1900d52d0 con 0x7fe1900ce740 2024-09-17T04:23:48.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.194+0000 7fbdbd800640 1 -- start start 2024-09-17T04:23:48.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.199+0000 7f5cf5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cd4004800 0x7f5cf0158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.199+0000 7f761fe00640 1 Processor -- start 2024-09-17T04:23:48.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.200+0000 7f5cf5600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cf01f02e0 0x7f5cf01f06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.201+0000 7fbdbd800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbdb80b98b0 0x7fbdb815ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.201+0000 7fbdbd800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdb80cf160 0x7fbdb81f5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.202+0000 7f3d7b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d7c0cefb0 0x7f3d7c09b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.202+0000 7f7a8be00640 1 -- 172.21.15.100:0/3737930925 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f7a8c1541a0 con 0x7f7a8c0b0940 2024-09-17T04:23:48.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.202+0000 7fbdbd800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdb80d0bf0 0x7fbdb8168070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.202+0000 7f3d7b200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d7c1604b0 0x7f3d7c15fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.202+0000 7f3d7b200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d7c1610b0 0x7f3d7c158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.202+0000 7f3d7b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c15d310 con 0x7f3d7c0cefb0 2024-09-17T04:23:48.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.203+0000 7f3d79400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d7c1604b0 0x7f3d7c15fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:48.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.203+0000 7f3d7b200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c15d190 con 0x7f3d7c1610b0 2024-09-17T04:23:48.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.203+0000 7f3d7b200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3d7c15d490 con 0x7f3d7c1604b0 2024-09-17T04:23:48.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.203+0000 7f3d7a800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d7c1610b0 0x7f3d7c158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:48.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.203+0000 7f3d79400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d7c1604b0 0x7f3d7c15fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.100:35510/0 (socket says 172.21.15.100:35510) 2024-09-17T04:23:48.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.203+0000 7f3d79400640 1 -- 172.21.15.100:0/3941692903 learned_addr learned my addr 172.21.15.100:0/3941692903 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:23:48.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.204+0000 7f3d79e00640 1 --2- 172.21.15.100:0/3941692903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d7c0cefb0 0x7f3d7c09b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:48.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.205+0000 7f3d79e00640 1 -- 172.21.15.100:0/3941692903 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d7c1604b0 msgr2=0x7f3d7c15fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.205+0000 7f3d79e00640 1 --2- 172.21.15.100:0/3941692903 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d7c1604b0 0x7f3d7c15fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.205+0000 7f3d79e00640 1 -- 172.21.15.100:0/3941692903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d7c1610b0 msgr2=0x7f3d7c158450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.207+0000 7f3d79e00640 1 --2- 172.21.15.100:0/3941692903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d7c1610b0 0x7f3d7c158450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.207+0000 7f3d79e00640 1 -- 172.21.15.100:0/3941692903 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d7c15a500 con 0x7f3d7c0cefb0 2024-09-17T04:23:48.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.207+0000 7f3d7a800640 1 --2- 172.21.15.100:0/3941692903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d7c1610b0 0x7f3d7c158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:23:48.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.209+0000 7f3d79e00640 1 --2- 172.21.15.100:0/3941692903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d7c0cefb0 0x7f3d7c09b770 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400ee40 tx=0x7f3d7400c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:23:48.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.206+0000 7f5cef400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cf01f02e0 0x7f5cf01f06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:48.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.206+0000 7f5cef400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cf01f02e0 0x7f5cf01f06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.100:35498/0 (socket says 172.21.15.100:35498) 2024-09-17T04:23:48.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.206+0000 7f5cef400640 1 -- 172.21.15.100:0/3247542418 learned_addr learned my addr 172.21.15.100:0/3247542418 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:23:48.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.206+0000 7f5cf5600640 1 --2- 172.21.15.100:0/3247542418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cf01f4430 0x7f5cf01f4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.206+0000 7f5cf4c00640 1 --2- 172.21.15.100:0/3247542418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cf01f4430 0x7f5cf01f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:48.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.209+0000 7f5cf5600640 1 -- 172.21.15.100:0/3247542418 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf00ccca0 con 0x7f5cd4004800 2024-09-17T04:23:48.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.209+0000 7f5cefe00640 1 --2- 172.21.15.100:0/3247542418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cd4004800 0x7f5cf0158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:48.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.202+0000 7fbdbd800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb80d1540 con 0x7fbdb80cf160 2024-09-17T04:23:48.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.202+0000 7fbdbd800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb80d1f50 con 0x7fbdb80d0bf0 2024-09-17T04:23:48.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.202+0000 7fbdbd800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbdb80d16c0 con 0x7fbdb80b98b0 2024-09-17T04:23:48.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.204+0000 7fbdb7e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbdb80b98b0 0x7fbdb815ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:48.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.204+0000 7fbdb7e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbdb80b98b0 0x7fbdb815ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.100:35508/0 (socket says 172.21.15.100:35508) 2024-09-17T04:23:48.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.204+0000 7fbdb7e00640 1 -- 172.21.15.100:0/261407260 learned_addr learned my addr 172.21.15.100:0/261407260 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:23:48.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.204+0000 7fbdbce00640 1 --2- 172.21.15.100:0/261407260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdb80d0bf0 0x7fbdb8168070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:48.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.205+0000 7fbdb7400640 1 --2- 172.21.15.100:0/261407260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdb80cf160 0x7fbdb81f5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:48.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.203+0000 7f761fe00640 1 -- start start 2024-09-17T04:23:48.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.210+0000 7fbdb7e00640 1 -- 172.21.15.100:0/261407260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdb80d0bf0 msgr2=0x7fbdb8168070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.210+0000 7fbdb7e00640 1 --2- 172.21.15.100:0/261407260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdb80d0bf0 0x7fbdb8168070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.210+0000 7fbdb7e00640 1 -- 172.21.15.100:0/261407260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdb80cf160 msgr2=0x7fbdb81f5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.210+0000 7fbdb7e00640 1 --2- 172.21.15.100:0/261407260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdb80cf160 0x7fbdb81f5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.210+0000 7fbdb7e00640 1 -- 172.21.15.100:0/261407260 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb809ea40 con 0x7fbdb80b98b0 2024-09-17T04:23:48.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.210+0000 7fbdbce00640 1 --2- 172.21.15.100:0/261407260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdb80d0bf0 0x7fbdb8168070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:23:48.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.210+0000 7fbdb7400640 1 --2- 172.21.15.100:0/261407260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdb80cf160 0x7fbdb81f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:23:48.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.210+0000 7f5cf5600640 1 -- 172.21.15.100:0/3247542418 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf00ccb20 con 0x7f5cf01f4430 2024-09-17T04:23:48.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.211+0000 7f5cf5600640 1 -- 172.21.15.100:0/3247542418 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5cf00cce20 con 0x7f5cf01f02e0 2024-09-17T04:23:48.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.211+0000 7fe187400640 1 -- 172.21.15.100:0/4183399031 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v11)=0 v11) ==== 122+0+0 (secure 0 0 0) 0x7fe188004460 con 0x7fe1900ce740 2024-09-17T04:23:48.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.211+0000 7fbdb7e00640 1 --2- 172.21.15.100:0/261407260 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbdb80b98b0 0x7fbdb815ba10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbda800c3c0 tx=0x7fbda800c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:23:48.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.212+0000 7f3d72000640 1 -- 172.21.15.100:0/3941692903 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7401b070 con 0x7f3d7c0cefb0 2024-09-17T04:23:48.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.212+0000 7fbdafe00640 1 -- 172.21.15.100:0/261407260 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda8011790 con 0x7fbdb80b98b0 2024-09-17T04:23:48.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.212+0000 7fbdbd800640 1 -- 172.21.15.100:0/261407260 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb81f65c0 con 0x7fbdb80b98b0 2024-09-17T04:23:48.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.212+0000 7fbdbd800640 1 -- 172.21.15.100:0/261407260 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdb80a0ba0 con 0x7fbdb80b98b0 2024-09-17T04:23:48.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.213+0000 7f5cf4c00640 1 -- 172.21.15.100:0/3247542418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cf01f02e0 msgr2=0x7f5cf01f06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.213+0000 7f5cf4c00640 1 --2- 172.21.15.100:0/3247542418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cf01f02e0 0x7f5cf01f06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.213+0000 7f5cf4c00640 1 -- 172.21.15.100:0/3247542418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cd4004800 msgr2=0x7f5cf0158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.213+0000 7f5cf4c00640 1 --2- 172.21.15.100:0/3247542418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cd4004800 0x7f5cf0158010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.213+0000 7f5cf4c00640 1 -- 172.21.15.100:0/3247542418 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf0158980 con 0x7f5cf01f4430 2024-09-17T04:23:48.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.213+0000 7f5cf4c00640 1 --2- 172.21.15.100:0/3247542418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cf01f4430 0x7f5cf01f4840 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00718c0 tx=0x7f5ce800dd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:23:48.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.213+0000 7f5ce7e00640 1 -- 172.21.15.100:0/3247542418 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8004bd0 con 0x7f5cf01f4430 2024-09-17T04:23:48.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.214+0000 7f5ce7e00640 1 -- 172.21.15.100:0/3247542418 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ce8016070 con 0x7f5cf01f4430 2024-09-17T04:23:48.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.214+0000 7f5ce7e00640 1 -- 172.21.15.100:0/3247542418 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce80043c0 con 0x7f5cf01f4430 2024-09-17T04:23:48.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.214+0000 7fe195e00640 1 -- 172.21.15.100:0/4183399031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1900ce740 msgr2=0x7fe1901602e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.214+0000 7fe195e00640 1 --2- 172.21.15.100:0/4183399031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1900ce740 0x7fe1901602e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe1900d0b00 tx=0x7fe18800f790 comp rx=0 tx=0).stop 2024-09-17T04:23:48.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.215+0000 7f761fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7620074dc0 0x7f7620202b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.215+0000 7f5cf5600640 1 -- 172.21.15.100:0/3247542418 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf0158c60 con 0x7f5cf01f4430 2024-09-17T04:23:48.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.215+0000 7f5cf5600640 1 -- 172.21.15.100:0/3247542418 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf0159220 con 0x7f5cf01f4430 2024-09-17T04:23:48.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.215+0000 7fbdafe00640 1 -- 172.21.15.100:0/261407260 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbda8017070 con 0x7fbdb80b98b0 2024-09-17T04:23:48.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.215+0000 7fbdafe00640 1 -- 172.21.15.100:0/261407260 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda8011790 con 0x7fbdb80b98b0 2024-09-17T04:23:48.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.216+0000 7f761fe00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76200af490 0x7f762015da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.216+0000 7f761fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7620157070 0x7f762015df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.216+0000 7fe195e00640 1 -- 172.21.15.100:0/4183399031 shutdown_connections 2024-09-17T04:23:48.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.216+0000 7fe195e00640 1 --2- 172.21.15.100:0/4183399031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1900ce740 0x7fe1901602e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.216+0000 7fe195e00640 1 --2- 172.21.15.100:0/4183399031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1900cc130 0x7fe19015fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.216+0000 7fe195e00640 1 --2- 172.21.15.100:0/4183399031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe174004800 0x7fe19015b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.216+0000 7fe195e00640 1 -- 172.21.15.100:0/4183399031 >> 172.21.15.100:0/4183399031 conn(0x7fe1900d46e0 msgr2=0x7fe19006e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:48.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.216+0000 7f761fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f76200b47b0 con 0x7f7620074dc0 2024-09-17T04:23:48.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.216+0000 7f761fe00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f76200b4630 con 0x7f7620157070 2024-09-17T04:23:48.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.216+0000 7f761f400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7620157070 0x7f762015df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:48.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.216+0000 7f761f400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7620157070 0x7f762015df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.100:39378/0 (socket says 172.21.15.100:39378) 2024-09-17T04:23:48.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.216+0000 7f761f400640 1 -- 172.21.15.100:0/2235159705 learned_addr learned my addr 172.21.15.100:0/2235159705 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:23:48.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.216+0000 7f761fe00640 1 -- 172.21.15.100:0/2235159705 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f76200b4930 con 0x7f76200af490 2024-09-17T04:23:48.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.216+0000 7fe195e00640 1 -- 172.21.15.100:0/4183399031 shutdown_connections 2024-09-17T04:23:48.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.217+0000 7fe195e00640 1 -- 172.21.15.100:0/4183399031 wait complete. 2024-09-17T04:23:48.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.216+0000 7f761ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7620074dc0 0x7f7620202b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:48.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.217+0000 7f761f400640 1 -- 172.21.15.100:0/2235159705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76200af490 msgr2=0x7f762015da40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T04:23:48.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.217+0000 7f761f400640 1 --2- 172.21.15.100:0/2235159705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76200af490 0x7f762015da40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.217+0000 7f761f400640 1 -- 172.21.15.100:0/2235159705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7620074dc0 msgr2=0x7f7620202b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.217+0000 7f761f400640 1 --2- 172.21.15.100:0/2235159705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7620074dc0 0x7f7620202b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.217+0000 7f761f400640 1 -- 172.21.15.100:0/2235159705 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762015bef0 con 0x7f7620157070 2024-09-17T04:23:48.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.217+0000 7f761f400640 1 --2- 172.21.15.100:0/2235159705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7620157070 0x7f762015df80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f761800bcc0 tx=0x7f7618003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:23:48.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.217+0000 7f761ea00640 1 --2- 172.21.15.100:0/2235159705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7620074dc0 0x7f7620202b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:23:48.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.217+0000 7f3d72000640 1 -- 172.21.15.100:0/3941692903 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d7400b570 con 0x7f3d7c0cefb0 2024-09-17T04:23:48.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.218+0000 7f7a83400640 1 -- 172.21.15.100:0/3737930925 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v12)=0 v12) ==== 121+0+0 (secure 0 0 0) 0x7f7a7c0094e0 con 0x7f7a8c0b0940 2024-09-17T04:23:48.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.220+0000 7f5ce7e00640 1 -- 172.21.15.100:0/3247542418 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5ce8004560 con 0x7f5cf01f4430 2024-09-17T04:23:48.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.220+0000 7f5ce7e00640 1 -- 172.21.15.100:0/3247542418 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5ce801b6c0 con 0x7f5cf01f4430 2024-09-17T04:23:48.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.218+0000 7fbdafe00640 1 -- 172.21.15.100:0/261407260 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbda8011b40 con 0x7fbdb80b98b0 2024-09-17T04:23:48.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.218+0000 7fbdafe00640 1 -- 172.21.15.100:0/261407260 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbda8012420 con 0x7fbdb80b98b0 2024-09-17T04:23:48.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.220+0000 7f5ce5600640 1 -- 172.21.15.100:0/3247542418 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cc4005290 con 0x7f5cf01f4430 2024-09-17T04:23:48.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.218+0000 7f3d7b200640 1 -- 172.21.15.100:0/3941692903 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d7c15a7e0 con 0x7f3d7c0cefb0 2024-09-17T04:23:48.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.222+0000 7f3d7b200640 1 -- 172.21.15.100:0/3941692903 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d7c09be80 con 0x7f3d7c0cefb0 2024-09-17T04:23:48.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.222+0000 7f3d72000640 1 -- 172.21.15.100:0/3941692903 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7400b9c0 con 0x7f3d7c0cefb0 2024-09-17T04:23:48.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.222+0000 7f7617400640 1 -- 172.21.15.100:0/2235159705 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7618042070 con 0x7f7620157070 2024-09-17T04:23:48.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.222+0000 7f7617400640 1 -- 172.21.15.100:0/2235159705 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76180193d0 con 0x7f7620157070 2024-09-17T04:23:48.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.225+0000 7f7a8be00640 1 -- 172.21.15.100:0/3737930925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a8c0b0940 msgr2=0x7f7a8c0d0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.225+0000 7f7a8be00640 1 --2- 172.21.15.100:0/3737930925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a8c0b0940 0x7f7a8c0d0320 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c004000 tx=0x7f7a7c004260 comp rx=0 tx=0).stop 2024-09-17T04:23:48.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.225+0000 7f7a8be00640 1 -- 172.21.15.100:0/3737930925 shutdown_connections 2024-09-17T04:23:48.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.225+0000 7f7a8be00640 1 --2- 172.21.15.100:0/3737930925 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a8c15a530 0x7f7a8c0d0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.225+0000 7f7a8be00640 1 --2- 172.21.15.100:0/3737930925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a8c0b0940 0x7f7a8c0d0320 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.225+0000 7f7a8be00640 1 --2- 172.21.15.100:0/3737930925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a8c0ad0b0 0x7f7a8c099910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.225+0000 7f7a8be00640 1 -- 172.21.15.100:0/3737930925 >> 172.21.15.100:0/3737930925 conn(0x7f7a8c0742a0 msgr2=0x7f7a8c0ae760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:48.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.225+0000 7f7a8be00640 1 -- 172.21.15.100:0/3737930925 shutdown_connections 2024-09-17T04:23:48.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.225+0000 7f7a8be00640 1 -- 172.21.15.100:0/3737930925 wait complete. 2024-09-17T04:23:48.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.226+0000 7fbdbd800640 1 -- 172.21.15.100:0/261407260 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd90005290 con 0x7fbdb80b98b0 2024-09-17T04:23:48.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.224+0000 7f3d72000640 1 -- 172.21.15.100:0/3941692903 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3d7401e070 con 0x7f3d7c0cefb0 2024-09-17T04:23:48.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.224+0000 7f3d72000640 1 -- 172.21.15.100:0/3941692903 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3d74020d40 con 0x7f3d7c0cefb0 2024-09-17T04:23:48.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.229+0000 7f761fe00640 1 -- 172.21.15.100:0/2235159705 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f762015c1b0 con 0x7f7620157070 2024-09-17T04:23:48.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.229+0000 7f761fe00640 1 -- 172.21.15.100:0/2235159705 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f762015c770 con 0x7f7620157070 2024-09-17T04:23:48.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.230+0000 7fbdafe00640 1 -- 172.21.15.100:0/261407260 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fbda801c460 con 0x7fbdb80b98b0 2024-09-17T04:23:48.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.231+0000 7f7617400640 1 -- 172.21.15.100:0/2235159705 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7618034b30 con 0x7f7620157070 2024-09-17T04:23:48.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.231+0000 7f7617400640 1 -- 172.21.15.100:0/2235159705 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7618034cd0 con 0x7f7620157070 2024-09-17T04:23:48.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.231+0000 7f7617400640 1 -- 172.21.15.100:0/2235159705 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f761803d790 con 0x7f7620157070 2024-09-17T04:23:48.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.231+0000 7f3d7b200640 1 -- 172.21.15.100:0/3941692903 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d54005290 con 0x7f3d7c0cefb0 2024-09-17T04:23:48.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.235+0000 7f627d800640 1 -- 172.21.15.100:0/1050177097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f62780d5290 con 0x7f62780b9660 2024-09-17T04:23:48.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.239+0000 7f7614c00640 1 -- 172.21.15.100:0/2235159705 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76200b4e50 con 0x7f7620157070 2024-09-17T04:23:48.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.236+0000 7f5ce7e00640 1 -- 172.21.15.100:0/3247542418 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f5ce8006070 con 0x7f5cf01f4430 2024-09-17T04:23:48.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.245+0000 7f3d72000640 1 -- 172.21.15.100:0/3941692903 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f3d7400bbb0 con 0x7f3d7c0cefb0 2024-09-17T04:23:48.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.252+0000 7f626fe00640 1 -- 172.21.15.100:0/1050177097 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v13)=0 v13) ==== 119+0+0 (secure 0 0 0) 0x7f626401a820 con 0x7f62780b9660 2024-09-17T04:23:48.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.248+0000 7f7617400640 1 -- 172.21.15.100:0/2235159705 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f7618004040 con 0x7f7620157070 2024-09-17T04:23:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.259+0000 7f627d800640 1 -- 172.21.15.100:0/1050177097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62780b9660 msgr2=0x7f627815a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.259+0000 7f627d800640 1 --2- 172.21.15.100:0/1050177097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62780b9660 0x7f627815a5a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f626400b410 tx=0x7f626400b8d0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.260+0000 7f627d800640 1 -- 172.21.15.100:0/1050177097 shutdown_connections 2024-09-17T04:23:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.260+0000 7f627d800640 1 --2- 172.21.15.100:0/1050177097 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62780d3570 0x7f627815f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.260+0000 7f627d800640 1 --2- 172.21.15.100:0/1050177097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62780b9660 0x7f627815a5a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.260+0000 7f627d800640 1 --2- 172.21.15.100:0/1050177097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62780b3870 0x7f627815a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.260+0000 7f627d800640 1 -- 172.21.15.100:0/1050177097 >> 172.21.15.100:0/1050177097 conn(0x7f62780d4550 msgr2=0x7f62780ceba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.260+0000 7f627d800640 1 -- 172.21.15.100:0/1050177097 shutdown_connections 2024-09-17T04:23:48.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.260+0000 7f627d800640 1 -- 172.21.15.100:0/1050177097 wait complete. 2024-09-17T04:23:48.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.275+0000 7fcc6e000640 1 -- 172.21.15.100:0/3903388111 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fcc680aef40 con 0x7fcc6815ccf0 2024-09-17T04:23:48.284 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi100') 2024-09-17T04:23:48.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.281+0000 7fcb72e00640 1 -- 172.21.15.100:0/4214017471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb6c0b0ff0 msgr2=0x7fcb6c0cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.281+0000 7fcb72e00640 1 --2- 172.21.15.100:0/4214017471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb6c0b0ff0 0x7fcb6c0cccb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcb64005fd0 tx=0x7fcb6402a9a0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.281+0000 7fcb72e00640 1 -- 172.21.15.100:0/4214017471 shutdown_connections 2024-09-17T04:23:48.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.281+0000 7fcb72e00640 1 --2- 172.21.15.100:0/4214017471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb6c0ae4a0 0x7fcb6c1685c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.281+0000 7fcb72e00640 1 --2- 172.21.15.100:0/4214017471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb6c0b0ff0 0x7fcb6c0cccb0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.281+0000 7fcb72e00640 1 -- 172.21.15.100:0/4214017471 >> 172.21.15.100:0/4214017471 conn(0x7fcb6c0742a0 msgr2=0x7fcb6c0746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:48.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.281+0000 7fcb72e00640 1 -- 172.21.15.100:0/4214017471 shutdown_connections 2024-09-17T04:23:48.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.281+0000 7fcb72e00640 1 -- 172.21.15.100:0/4214017471 wait complete. 2024-09-17T04:23:48.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.288+0000 7f0f31e00640 1 -- 172.21.15.100:0/3774433600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f2c0b0940 msgr2=0x7f0f2c159ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.288+0000 7f0f31e00640 1 --2- 172.21.15.100:0/3774433600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f2c0b0940 0x7f0f2c159ff0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0f24005fd0 tx=0x7f0f2402a6c0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.288+0000 7f0f31e00640 1 -- 172.21.15.100:0/3774433600 shutdown_connections 2024-09-17T04:23:48.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.288+0000 7f0f31e00640 1 --2- 172.21.15.100:0/3774433600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f2c0b0940 0x7f0f2c159ff0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.288+0000 7f0f31e00640 1 -- 172.21.15.100:0/3774433600 >> 172.21.15.100:0/3774433600 conn(0x7f0f2c0742a0 msgr2=0x7f0f2c0746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:48.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.288+0000 7f0f31e00640 1 -- 172.21.15.100:0/3774433600 shutdown_connections 2024-09-17T04:23:48.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.289+0000 7f0f31e00640 1 -- 172.21.15.100:0/3774433600 wait complete. 2024-09-17T04:23:48.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.290+0000 7fcb72e00640 1 Processor -- start 2024-09-17T04:23:48.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.290+0000 7fcb72e00640 1 -- start start 2024-09-17T04:23:48.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.294+0000 7f0f31e00640 1 Processor -- start 2024-09-17T04:23:48.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.295+0000 7f0f31e00640 1 -- start start 2024-09-17T04:23:48.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.295+0000 7fcc64c00640 1 -- 172.21.15.100:0/3903388111 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v14)=0 v14) ==== 120+0+0 (secure 0 0 0) 0x7fcc5404f080 con 0x7fcc6815ccf0 2024-09-17T04:23:48.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.295+0000 7fcb72e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb6c0ae4a0 0x7fcb6c14f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.295+0000 7fcb72e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb6c0b0ff0 0x7fcb6c0b4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.295+0000 7fcb72e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb6c0cd1f0 0x7fcb6c0b4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.295+0000 7fcb72e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6c168c80 con 0x7fcb6c0b0ff0 2024-09-17T04:23:48.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.295+0000 7fcb72e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6c168b00 con 0x7fcb6c0cd1f0 2024-09-17T04:23:48.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.295+0000 7fcb72e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcb6c168e00 con 0x7fcb6c0ae4a0 2024-09-17T04:23:48.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.296+0000 7fcb71000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb6c0b0ff0 0x7fcb6c0b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:48.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.297+0000 7fcb71000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb6c0b0ff0 0x7fcb6c0b4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37078/0 (socket says 172.21.15.100:37078) 2024-09-17T04:23:48.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.297+0000 7fcb71000640 1 -- 172.21.15.100:0/1285903643 learned_addr learned my addr 172.21.15.100:0/1285903643 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:23:48.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.300+0000 7f0f31e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f2c0b0940 0x7f0f2c09d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.301+0000 7f0f31e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f2c15a530 0x7f0f2c14c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.301+0000 7f0f31e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f2c15a940 0x7f0f2c15c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.301+0000 7f0f31e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2c15d2d0 con 0x7f0f2c15a940 2024-09-17T04:23:48.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.301+0000 7f0f31e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2c0cdd40 con 0x7f0f2c15a530 2024-09-17T04:23:48.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.301+0000 7f0f31e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0f2c15d450 con 0x7f0f2c0b0940 2024-09-17T04:23:48.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.301+0000 7f0f30a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f2c0b0940 0x7f0f2c09d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:48.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.301+0000 7f0f30a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f2c0b0940 0x7f0f2c09d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.100:35552/0 (socket says 172.21.15.100:35552) 2024-09-17T04:23:48.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.301+0000 7f0f30a00640 1 -- 172.21.15.100:0/2163007695 learned_addr learned my addr 172.21.15.100:0/2163007695 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:23:48.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.298+0000 7fcc6e000640 1 -- 172.21.15.100:0/3903388111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc6815ccf0 msgr2=0x7fcc68154ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.298+0000 7fcc6e000640 1 --2- 172.21.15.100:0/3903388111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc6815ccf0 0x7fcc68154ad0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcc54000c00 tx=0x7fcc54001720 comp rx=0 tx=0).stop 2024-09-17T04:23:48.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.298+0000 7fcc6e000640 1 -- 172.21.15.100:0/3903388111 shutdown_connections 2024-09-17T04:23:48.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.298+0000 7fcc6e000640 1 --2- 172.21.15.100:0/3903388111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc6806faa0 0x7fcc6806feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.298+0000 7fcc6e000640 1 --2- 172.21.15.100:0/3903388111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc68158630 0x7fcc68158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.298+0000 7fcc6e000640 1 --2- 172.21.15.100:0/3903388111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc6815ccf0 0x7fcc68154ad0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.298+0000 7fcc6e000640 1 -- 172.21.15.100:0/3903388111 >> 172.21.15.100:0/3903388111 conn(0x7fcc680742a0 msgr2=0x7fcc680b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:48.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.298+0000 7fcc6e000640 1 -- 172.21.15.100:0/3903388111 shutdown_connections 2024-09-17T04:23:48.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.298+0000 7fcc6e000640 1 -- 172.21.15.100:0/3903388111 wait complete. 2024-09-17T04:23:48.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.298+0000 7fcb71000640 1 -- 172.21.15.100:0/1285903643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb6c0ae4a0 msgr2=0x7fcb6c14f7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T04:23:48.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.298+0000 7fcb71000640 1 --2- 172.21.15.100:0/1285903643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb6c0ae4a0 0x7fcb6c14f7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.298+0000 7fcb71000640 1 -- 172.21.15.100:0/1285903643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb6c0cd1f0 msgr2=0x7fcb6c0b4660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T04:23:48.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.298+0000 7fcb71000640 1 --2- 172.21.15.100:0/1285903643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb6c0cd1f0 0x7fcb6c0b4660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.300+0000 7fcb71000640 1 -- 172.21.15.100:0/1285903643 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6c14fd00 con 0x7fcb6c0b0ff0 2024-09-17T04:23:48.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.300+0000 7fcb71000640 1 --2- 172.21.15.100:0/1285903643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb6c0b0ff0 0x7fcb6c0b4120 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800b410 tx=0x7fcb5800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:23:48.306 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi100') 2024-09-17T04:23:48.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.302+0000 7f0f2be00640 1 --2- 172.21.15.100:0/2163007695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f2c15a530 0x7f0f2c14c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:48.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.302+0000 7f0f31400640 1 --2- 172.21.15.100:0/2163007695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f2c15a940 0x7f0f2c15c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:48.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.302+0000 7f0f30a00640 1 -- 172.21.15.100:0/2163007695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f2c15a530 msgr2=0x7f0f2c14c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.302+0000 7f0f30a00640 1 --2- 172.21.15.100:0/2163007695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f2c15a530 0x7f0f2c14c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.302+0000 7f0f30a00640 1 -- 172.21.15.100:0/2163007695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f2c15a940 msgr2=0x7f0f2c15c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.303+0000 7f0f30a00640 1 --2- 172.21.15.100:0/2163007695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f2c15a940 0x7f0f2c15c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.303+0000 7f0f30a00640 1 -- 172.21.15.100:0/2163007695 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2c1fe870 con 0x7f0f2c0b0940 2024-09-17T04:23:48.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.303+0000 7f0f31400640 1 --2- 172.21.15.100:0/2163007695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f2c15a940 0x7f0f2c15c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:23:48.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.303+0000 7f0f30a00640 1 --2- 172.21.15.100:0/2163007695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f2c0b0940 0x7f0f2c09d780 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0f24000f80 tx=0x7f0f24032e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:23:48.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.303+0000 7f0f28c00640 1 -- 172.21.15.100:0/2163007695 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f24041020 con 0x7f0f2c0b0940 2024-09-17T04:23:48.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.304+0000 7f0f28c00640 1 -- 172.21.15.100:0/2163007695 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f24037040 con 0x7f0f2c0b0940 2024-09-17T04:23:48.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.303+0000 7fcb69600640 1 -- 172.21.15.100:0/1285903643 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb58015020 con 0x7fcb6c0b0ff0 2024-09-17T04:23:48.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.304+0000 7f0f28c00640 1 -- 172.21.15.100:0/2163007695 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f24036070 con 0x7f0f2c0b0940 2024-09-17T04:23:48.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.304+0000 7fcb69600640 1 -- 172.21.15.100:0/1285903643 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb58011920 con 0x7fcb6c0b0ff0 2024-09-17T04:23:48.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.304+0000 7fcb69600640 1 -- 172.21.15.100:0/1285903643 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb58011d70 con 0x7fcb6c0b0ff0 2024-09-17T04:23:48.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.304+0000 7fcb72e00640 1 -- 172.21.15.100:0/1285903643 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb6c1553f0 con 0x7fcb6c0b0ff0 2024-09-17T04:23:48.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.304+0000 7fcb72e00640 1 -- 172.21.15.100:0/1285903643 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb6c152490 con 0x7fcb6c0b0ff0 2024-09-17T04:23:48.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.307+0000 7f0f31e00640 1 -- 172.21.15.100:0/2163007695 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f2c1fdaf0 con 0x7f0f2c0b0940 2024-09-17T04:23:48.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.307+0000 7f0f31e00640 1 -- 172.21.15.100:0/2163007695 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f2c1fe070 con 0x7f0f2c0b0940 2024-09-17T04:23:48.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.307+0000 7f0f28c00640 1 -- 172.21.15.100:0/2163007695 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0f24044020 con 0x7f0f2c0b0940 2024-09-17T04:23:48.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.307+0000 7f0f28c00640 1 -- 172.21.15.100:0/2163007695 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0f24046750 con 0x7f0f2c0b0940 2024-09-17T04:23:48.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.311+0000 7f0f31e00640 1 -- 172.21.15.100:0/2163007695 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f2c159ff0 con 0x7f0f2c0b0940 2024-09-17T04:23:48.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.308+0000 7fcb72e00640 1 -- 172.21.15.100:0/1285903643 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb48005290 con 0x7fcb6c0b0ff0 2024-09-17T04:23:48.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.309+0000 7fcb69600640 1 -- 172.21.15.100:0/1285903643 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcb58018020 con 0x7fcb6c0b0ff0 2024-09-17T04:23:48.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.309+0000 7fcb69600640 1 -- 172.21.15.100:0/1285903643 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcb580198f0 con 0x7fcb6c0b0ff0 2024-09-17T04:23:48.312 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi100') 2024-09-17T04:23:48.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.314+0000 7f0f28c00640 1 -- 172.21.15.100:0/2163007695 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f0f2404b030 con 0x7f0f2c0b0940 2024-09-17T04:23:48.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.315+0000 7f0f2be00640 1 --2- 172.21.15.100:0/2163007695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f2c15a530 0x7f0f2c14c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:23:48.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.320+0000 7fcb69600640 1 -- 172.21.15.100:0/1285903643 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fcb5801f020 con 0x7fcb6c0b0ff0 2024-09-17T04:23:48.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.321+0000 7f8527e00640 1 -- 172.21.15.100:0/173645097 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f853c0cc750 con 0x7f853c0b0940 2024-09-17T04:23:48.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.335+0000 7f8532a00640 1 -- 172.21.15.100:0/173645097 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v15)=0 v15) ==== 121+0+0 (secure 0 0 0) 0x7f853c0cc750 con 0x7f853c0b0940 2024-09-17T04:23:48.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.336+0000 7f734b200640 1 -- 172.21.15.100:0/1670061344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73440ccc50 msgr2=0x7f73440cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.336+0000 7f734b200640 1 --2- 172.21.15.100:0/1670061344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73440ccc50 0x7f73440cf570 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7330009830 tx=0x7f733002e8e0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.337+0000 7f734b200640 1 -- 172.21.15.100:0/1670061344 shutdown_connections 2024-09-17T04:23:48.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.337+0000 7f734b200640 1 --2- 172.21.15.100:0/1670061344 >> v2:172.21.15.184:3300/0 conn(0x7f73440cfab0 0x7f7344073630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.340+0000 7f8527e00640 1 -- 172.21.15.100:0/173645097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f853c0b0940 msgr2=0x7f853c153940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.340+0000 7f8527e00640 1 --2- 172.21.15.100:0/173645097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f853c0b0940 0x7f853c153940 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f853400ee40 tx=0x7f853400c740 comp rx=0 tx=0).stop 2024-09-17T04:23:48.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.340+0000 7f8527e00640 1 -- 172.21.15.100:0/173645097 shutdown_connections 2024-09-17T04:23:48.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.340+0000 7f8527e00640 1 --2- 172.21.15.100:0/173645097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f853c15a940 0x7f853c2000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.340+0000 7f8527e00640 1 --2- 172.21.15.100:0/173645097 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f853c15a530 0x7f853c1570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.340+0000 7f8527e00640 1 --2- 172.21.15.100:0/173645097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f853c0b0940 0x7f853c153940 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.340+0000 7f8527e00640 1 -- 172.21.15.100:0/173645097 >> 172.21.15.100:0/173645097 conn(0x7f853c0742a0 msgr2=0x7f853c06f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:48.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.337+0000 7f734b200640 1 --2- 172.21.15.100:0/1670061344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73440ccc50 0x7f73440cf570 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.341+0000 7f734b200640 1 -- 172.21.15.100:0/1670061344 >> 172.21.15.100:0/1670061344 conn(0x7f734406d1c0 msgr2=0x7f734406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:48.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.341+0000 7f734b200640 1 -- 172.21.15.100:0/1670061344 shutdown_connections 2024-09-17T04:23:48.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.341+0000 7f734b200640 1 -- 172.21.15.100:0/1670061344 wait complete. 2024-09-17T04:23:48.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.341+0000 7f8527e00640 1 -- 172.21.15.100:0/173645097 shutdown_connections 2024-09-17T04:23:48.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.341+0000 7f8527e00640 1 -- 172.21.15.100:0/173645097 wait complete. 2024-09-17T04:23:48.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.350+0000 7f734b200640 1 Processor -- start 2024-09-17T04:23:48.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.350+0000 7f734b200640 1 -- start start 2024-09-17T04:23:48.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.353+0000 7f734b200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7344073b70 0x7f7344143ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.353+0000 7f734b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73440ccc50 0x7f7344152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.354+0000 7f7343e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7344073b70 0x7f7344143ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:48.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.354+0000 7f7343e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7344073b70 0x7f7344143ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.100:39414/0 (socket says 172.21.15.100:39414) 2024-09-17T04:23:48.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.355+0000 7f7343400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73440ccc50 0x7f7344152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:48.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.355+0000 7f7343400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73440ccc50 0x7f7344152da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37094/0 (socket says 172.21.15.100:37094) 2024-09-17T04:23:48.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.355+0000 7f7343e00640 1 -- 172.21.15.100:0/2926025940 learned_addr learned my addr 172.21.15.100:0/2926025940 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:23:48.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.355+0000 7f734b200640 1 --2- 172.21.15.100:0/2926025940 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73440cfab0 0x7f73441532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.355+0000 7f734b200640 1 -- 172.21.15.100:0/2926025940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f734415fa00 con 0x7f73440ccc50 2024-09-17T04:23:48.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.356+0000 7f7349200640 1 --2- 172.21.15.100:0/2926025940 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73440cfab0 0x7f73441532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:48.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.356+0000 7f734b200640 1 -- 172.21.15.100:0/2926025940 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f734415f880 con 0x7f7344073b70 2024-09-17T04:23:48.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.356+0000 7f734b200640 1 -- 172.21.15.100:0/2926025940 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f734415fb80 con 0x7f73440cfab0 2024-09-17T04:23:48.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.357+0000 7f7343400640 1 -- 172.21.15.100:0/2926025940 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73440cfab0 msgr2=0x7f73441532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.358+0000 7f7343400640 1 --2- 172.21.15.100:0/2926025940 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73440cfab0 0x7f73441532e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.358+0000 7f7343400640 1 -- 172.21.15.100:0/2926025940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7344073b70 msgr2=0x7f7344143ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.358+0000 7f7343400640 1 --2- 172.21.15.100:0/2926025940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7344073b70 0x7f7344143ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.358+0000 7f7343400640 1 -- 172.21.15.100:0/2926025940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73441f5080 con 0x7f73440ccc50 2024-09-17T04:23:48.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.359+0000 7f7343400640 1 --2- 172.21.15.100:0/2926025940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73440ccc50 0x7f7344152da0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f733400ac10 tx=0x7f7334007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:23:48.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.359+0000 7f7343e00640 1 --2- 172.21.15.100:0/2926025940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7344073b70 0x7f7344143ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:23:48.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.363+0000 7f733be00640 1 -- 172.21.15.100:0/2926025940 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733400ad00 con 0x7f73440ccc50 2024-09-17T04:23:48.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.363+0000 7f734b200640 1 -- 172.21.15.100:0/2926025940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73441f5360 con 0x7f73440ccc50 2024-09-17T04:23:48.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.363+0000 7f734b200640 1 -- 172.21.15.100:0/2926025940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73441f5920 con 0x7f73440ccc50 2024-09-17T04:23:48.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.364+0000 7f733be00640 1 -- 172.21.15.100:0/2926025940 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7334007c40 con 0x7f73440ccc50 2024-09-17T04:23:48.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.368+0000 7f733be00640 1 -- 172.21.15.100:0/2926025940 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733400ad00 con 0x7f73440ccc50 2024-09-17T04:23:48.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.373+0000 7f734b200640 1 -- 172.21.15.100:0/2926025940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73440cf570 con 0x7f73440ccc50 2024-09-17T04:23:48.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.373+0000 7f733be00640 1 -- 172.21.15.100:0/2926025940 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7334005020 con 0x7f73440ccc50 2024-09-17T04:23:48.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.373+0000 7f733be00640 1 -- 172.21.15.100:0/2926025940 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7334004dc0 con 0x7f73440ccc50 2024-09-17T04:23:48.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.380+0000 7ff9c2c00640 1 -- 172.21.15.100:0/2053617228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9bc0cf870 msgr2=0x7ff9bc0d0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.380+0000 7ff9c2c00640 1 --2- 172.21.15.100:0/2053617228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9bc0cf870 0x7ff9bc0d0bf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4005fd0 tx=0x7ff9b402ab30 comp rx=0 tx=0).stop 2024-09-17T04:23:48.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.382+0000 7f733be00640 1 -- 172.21.15.100:0/2926025940 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f7334007df0 con 0x7f73440ccc50 2024-09-17T04:23:48.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.382+0000 7ff9c2c00640 1 -- 172.21.15.100:0/2053617228 shutdown_connections 2024-09-17T04:23:48.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.382+0000 7ff9c2c00640 1 --2- 172.21.15.100:0/2053617228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9bc0cf870 0x7ff9bc0d0bf0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.382+0000 7ff9c2c00640 1 -- 172.21.15.100:0/2053617228 >> 172.21.15.100:0/2053617228 conn(0x7ff9bc08bfe0 msgr2=0x7ff9bc0d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:48.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.383+0000 7ff9c2c00640 1 -- 172.21.15.100:0/2053617228 shutdown_connections 2024-09-17T04:23:48.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.387+0000 7ff9c2c00640 1 -- 172.21.15.100:0/2053617228 wait complete. 2024-09-17T04:23:48.392 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi100') 2024-09-17T04:23:48.393 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi100') 2024-09-17T04:23:48.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.410+0000 7f7614c00640 1 -- 172.21.15.100:0/2235159705 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f76201542c0 con 0x7f7620157070 2024-09-17T04:23:48.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.392+0000 7ff9c2c00640 1 Processor -- start 2024-09-17T04:23:48.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.393+0000 7ff9c2c00640 1 -- start start 2024-09-17T04:23:48.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.395+0000 7ff9c2c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9bc0b3690 0x7ff9bc15eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.395+0000 7ff9c2c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9bc0b9930 0x7ff9bc161820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.395+0000 7ff9c2c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9bc0cf870 0x7ff9bc0a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.395+0000 7ff9c2c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff9bc0d12b0 con 0x7ff9bc0cf870 2024-09-17T04:23:48.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.395+0000 7ff9c2c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff9bc0d1130 con 0x7ff9bc0b3690 2024-09-17T04:23:48.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.395+0000 7ff9c2c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff9bc0d1430 con 0x7ff9bc0b9930 2024-09-17T04:23:48.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.396+0000 7ff9c1800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9bc0b3690 0x7ff9bc15eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:48.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.396+0000 7ff9c1800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9bc0b3690 0x7ff9bc15eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.100:39428/0 (socket says 172.21.15.100:39428) 2024-09-17T04:23:48.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.396+0000 7ff9c1800640 1 -- 172.21.15.100:0/1920132094 learned_addr learned my addr 172.21.15.100:0/1920132094 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:23:48.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.396+0000 7ff9c2200640 1 --2- 172.21.15.100:0/1920132094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9bc0cf870 0x7ff9bc0a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:48.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.397+0000 7ff9c2200640 1 -- 172.21.15.100:0/1920132094 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9bc0b9930 msgr2=0x7ff9bc161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.397+0000 7ff9c2200640 1 --2- 172.21.15.100:0/1920132094 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9bc0b9930 0x7ff9bc161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.397+0000 7ff9c2200640 1 -- 172.21.15.100:0/1920132094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9bc0b3690 msgr2=0x7ff9bc15eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.397+0000 7ff9c2200640 1 --2- 172.21.15.100:0/1920132094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9bc0b3690 0x7ff9bc15eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.397+0000 7ff9c2200640 1 -- 172.21.15.100:0/1920132094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9bc1605c0 con 0x7ff9bc0cf870 2024-09-17T04:23:48.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.398+0000 7ff9c2200640 1 --2- 172.21.15.100:0/1920132094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9bc0cf870 0x7ff9bc0a1890 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4032000 tx=0x7ff9b402cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:23:48.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.399+0000 7ff9b9600640 1 -- 172.21.15.100:0/1920132094 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4044070 con 0x7ff9bc0cf870 2024-09-17T04:23:48.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.399+0000 7ff9b9600640 1 -- 172.21.15.100:0/1920132094 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9b4004040 con 0x7ff9bc0cf870 2024-09-17T04:23:48.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.399+0000 7ff9c2c00640 1 -- 172.21.15.100:0/1920132094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9bc15f8c0 con 0x7ff9bc0cf870 2024-09-17T04:23:48.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.399+0000 7ff9c2c00640 1 -- 172.21.15.100:0/1920132094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9bc15e5a0 con 0x7ff9bc0cf870 2024-09-17T04:23:48.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.399+0000 7ff9b9600640 1 -- 172.21.15.100:0/1920132094 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4004450 con 0x7ff9bc0cf870 2024-09-17T04:23:48.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.402+0000 7ff9c2c00640 1 -- 172.21.15.100:0/1920132094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff998005290 con 0x7ff9bc0cf870 2024-09-17T04:23:48.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.403+0000 7ff9b9600640 1 -- 172.21.15.100:0/1920132094 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff9b4047070 con 0x7ff9bc0cf870 2024-09-17T04:23:48.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.403+0000 7ff9b9600640 1 -- 172.21.15.100:0/1920132094 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff9b402da70 con 0x7ff9bc0cf870 2024-09-17T04:23:48.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.407+0000 7ff9b9600640 1 -- 172.21.15.100:0/1920132094 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7ff9bc1605c0 con 0x7ff9bc0cf870 2024-09-17T04:23:48.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.418+0000 7f5ce5600640 1 -- 172.21.15.100:0/3247542418 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f5cc4005580 con 0x7f5cf01f4430 2024-09-17T04:23:48.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.422+0000 7f7617400640 1 -- 172.21.15.100:0/2235159705 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v16)=0 v16) ==== 121+0+0 (secure 0 0 0) 0x7f761803d9e0 con 0x7f7620157070 2024-09-17T04:23:48.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.423+0000 7f3d7b200640 1 -- 172.21.15.100:0/3941692903 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f3d54005580 con 0x7f3d7c0cefb0 2024-09-17T04:23:48.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.426+0000 7f5ce7e00640 1 -- 172.21.15.100:0/3247542418 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v17)=0 v17) ==== 119+0+0 (secure 0 0 0) 0x7f5ce801b910 con 0x7f5cf01f4430 2024-09-17T04:23:48.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.426+0000 7f7614c00640 1 -- 172.21.15.100:0/2235159705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7620157070 msgr2=0x7f762015df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.426+0000 7f7614c00640 1 --2- 172.21.15.100:0/2235159705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7620157070 0x7f762015df80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f761800bcc0 tx=0x7f7618003f10 comp rx=0 tx=0).stop 2024-09-17T04:23:48.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.427+0000 7f5ce5600640 1 -- 172.21.15.100:0/3247542418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cf01f4430 msgr2=0x7f5cf01f4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.427+0000 7f5ce5600640 1 --2- 172.21.15.100:0/3247542418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cf01f4430 0x7f5cf01f4840 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00718c0 tx=0x7f5ce800dd80 comp rx=0 tx=0).stop 2024-09-17T04:23:48.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.428+0000 7f5ce5600640 1 -- 172.21.15.100:0/3247542418 shutdown_connections 2024-09-17T04:23:48.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.427+0000 7f7614c00640 1 -- 172.21.15.100:0/2235159705 shutdown_connections 2024-09-17T04:23:48.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.428+0000 7f5ce5600640 1 --2- 172.21.15.100:0/3247542418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cf01f4430 0x7f5cf01f4840 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.428+0000 7f5ce5600640 1 --2- 172.21.15.100:0/3247542418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cf01f02e0 0x7f5cf01f06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.428+0000 7f5ce5600640 1 --2- 172.21.15.100:0/3247542418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cd4004800 0x7f5cf0158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.428+0000 7f5ce5600640 1 -- 172.21.15.100:0/3247542418 >> 172.21.15.100:0/3247542418 conn(0x7f5cf00742a0 msgr2=0x7f5cf006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:48.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.428+0000 7f5ce5600640 1 -- 172.21.15.100:0/3247542418 shutdown_connections 2024-09-17T04:23:48.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.428+0000 7f5ce5600640 1 -- 172.21.15.100:0/3247542418 wait complete. 2024-09-17T04:23:48.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.427+0000 7f7614c00640 1 --2- 172.21.15.100:0/2235159705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7620157070 0x7f762015df80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.427+0000 7f7614c00640 1 --2- 172.21.15.100:0/2235159705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76200af490 0x7f762015da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.427+0000 7f7614c00640 1 --2- 172.21.15.100:0/2235159705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7620074dc0 0x7f7620202b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.427+0000 7f7614c00640 1 -- 172.21.15.100:0/2235159705 >> 172.21.15.100:0/2235159705 conn(0x7f76200742a0 msgr2=0x7f76200ae400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:48.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.428+0000 7f7614c00640 1 -- 172.21.15.100:0/2235159705 shutdown_connections 2024-09-17T04:23:48.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.428+0000 7f7614c00640 1 -- 172.21.15.100:0/2235159705 wait complete. 2024-09-17T04:23:48.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.434+0000 7f3d72000640 1 -- 172.21.15.100:0/3941692903 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v18)=0 v18) ==== 120+0+0 (secure 0 0 0) 0x7f3d7400bdf0 con 0x7f3d7c0cefb0 2024-09-17T04:23:48.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.437+0000 7f3d7b200640 1 -- 172.21.15.100:0/3941692903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d7c0cefb0 msgr2=0x7f3d7c09b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.437+0000 7f3d7b200640 1 --2- 172.21.15.100:0/3941692903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d7c0cefb0 0x7f3d7c09b770 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400ee40 tx=0x7f3d7400c740 comp rx=0 tx=0).stop 2024-09-17T04:23:48.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.437+0000 7f3d7b200640 1 -- 172.21.15.100:0/3941692903 shutdown_connections 2024-09-17T04:23:48.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.437+0000 7f3d7b200640 1 --2- 172.21.15.100:0/3941692903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d7c1610b0 0x7f3d7c158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.437+0000 7f3d7b200640 1 --2- 172.21.15.100:0/3941692903 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d7c1604b0 0x7f3d7c15fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.437+0000 7f3d7b200640 1 --2- 172.21.15.100:0/3941692903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d7c0cefb0 0x7f3d7c09b770 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.437+0000 7f3d7b200640 1 -- 172.21.15.100:0/3941692903 >> 172.21.15.100:0/3941692903 conn(0x7f3d7c0742a0 msgr2=0x7f3d7c0cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:48.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.439+0000 7f3d7b200640 1 -- 172.21.15.100:0/3941692903 shutdown_connections 2024-09-17T04:23:48.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.439+0000 7f3d7b200640 1 -- 172.21.15.100:0/3941692903 wait complete. 2024-09-17T04:23:48.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.441+0000 7fbdbd800640 1 -- 172.21.15.100:0/261407260 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7fbd90005850 con 0x7fbdb80b98b0 2024-09-17T04:23:48.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.452+0000 7fbdafe00640 1 -- 172.21.15.100:0/261407260 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v19)=0 v19) ==== 111+0+0 (secure 0 0 0) 0x7fbda801c640 con 0x7fbdb80b98b0 2024-09-17T04:23:48.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.458+0000 7fbdbd800640 1 -- 172.21.15.100:0/261407260 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbdb80b98b0 msgr2=0x7fbdb815ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.458+0000 7fbdbd800640 1 --2- 172.21.15.100:0/261407260 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbdb80b98b0 0x7fbdb815ba10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbda800c3c0 tx=0x7fbda800c880 comp rx=0 tx=0).stop 2024-09-17T04:23:48.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.458+0000 7fbdbd800640 1 -- 172.21.15.100:0/261407260 shutdown_connections 2024-09-17T04:23:48.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.458+0000 7fbdbd800640 1 --2- 172.21.15.100:0/261407260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdb80d0bf0 0x7fbdb8168070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.458+0000 7fbdbd800640 1 --2- 172.21.15.100:0/261407260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdb80cf160 0x7fbdb81f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.458+0000 7fbdbd800640 1 --2- 172.21.15.100:0/261407260 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbdb80b98b0 0x7fbdb815ba10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.458+0000 7fbdbd800640 1 -- 172.21.15.100:0/261407260 >> 172.21.15.100:0/261407260 conn(0x7fbdb808bfe0 msgr2=0x7fbdb80d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:48.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.458+0000 7fbdbd800640 1 -- 172.21.15.100:0/261407260 shutdown_connections 2024-09-17T04:23:48.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.458+0000 7fbdbd800640 1 -- 172.21.15.100:0/261407260 wait complete. 2024-09-17T04:23:48.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.475+0000 7fcb72e00640 1 -- 172.21.15.100:0/1285903643 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7fcb48005850 con 0x7fcb6c0b0ff0 2024-09-17T04:23:48.479 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi100') 2024-09-17T04:23:48.480 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi100') 2024-09-17T04:23:48.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.480+0000 7fcb69600640 1 -- 172.21.15.100:0/1285903643 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v20)=0 v20) ==== 106+0+0 (secure 0 0 0) 0x7fcb6c14fd00 con 0x7fcb6c0b0ff0 2024-09-17T04:23:48.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.482+0000 7fcb72e00640 1 -- 172.21.15.100:0/1285903643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb6c0b0ff0 msgr2=0x7fcb6c0b4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.482+0000 7fcb72e00640 1 --2- 172.21.15.100:0/1285903643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb6c0b0ff0 0x7fcb6c0b4120 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800b410 tx=0x7fcb5800b8d0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.482+0000 7fcb72e00640 1 -- 172.21.15.100:0/1285903643 shutdown_connections 2024-09-17T04:23:48.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.482+0000 7fcb72e00640 1 --2- 172.21.15.100:0/1285903643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb6c0cd1f0 0x7fcb6c0b4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.482+0000 7fcb72e00640 1 --2- 172.21.15.100:0/1285903643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb6c0b0ff0 0x7fcb6c0b4120 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.482+0000 7fcb72e00640 1 --2- 172.21.15.100:0/1285903643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb6c0ae4a0 0x7fcb6c14f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.482+0000 7fcb72e00640 1 -- 172.21.15.100:0/1285903643 >> 172.21.15.100:0/1285903643 conn(0x7fcb6c0742a0 msgr2=0x7fcb6c06f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:48.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.482+0000 7fcb72e00640 1 -- 172.21.15.100:0/1285903643 shutdown_connections 2024-09-17T04:23:48.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.482+0000 7fcb72e00640 1 -- 172.21.15.100:0/1285903643 wait complete. 2024-09-17T04:23:48.496 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi100') 2024-09-17T04:23:48.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.504+0000 7f0f31e00640 1 -- 172.21.15.100:0/2163007695 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f0f2c15d0b0 con 0x7f0f2c0b0940 2024-09-17T04:23:48.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.509+0000 7f0f28c00640 1 -- 172.21.15.100:0/2163007695 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v21)=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7f0f240047c0 con 0x7f0f2c0b0940 2024-09-17T04:23:48.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.510+0000 7f0f31e00640 1 -- 172.21.15.100:0/2163007695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f2c0b0940 msgr2=0x7f0f2c09d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.510+0000 7f0f31e00640 1 --2- 172.21.15.100:0/2163007695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f2c0b0940 0x7f0f2c09d780 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0f24000f80 tx=0x7f0f24032e90 comp rx=0 tx=0).stop 2024-09-17T04:23:48.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.510+0000 7f0f31e00640 1 -- 172.21.15.100:0/2163007695 shutdown_connections 2024-09-17T04:23:48.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.510+0000 7f0f31e00640 1 --2- 172.21.15.100:0/2163007695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f2c15a940 0x7f0f2c15c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.510+0000 7f0f31e00640 1 --2- 172.21.15.100:0/2163007695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f2c15a530 0x7f0f2c14c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.510+0000 7f0f31e00640 1 --2- 172.21.15.100:0/2163007695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f2c0b0940 0x7f0f2c09d780 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.510+0000 7f0f31e00640 1 -- 172.21.15.100:0/2163007695 >> 172.21.15.100:0/2163007695 conn(0x7f0f2c0742a0 msgr2=0x7f0f2c161c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:48.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.510+0000 7f0f31e00640 1 -- 172.21.15.100:0/2163007695 shutdown_connections 2024-09-17T04:23:48.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.510+0000 7f0f31e00640 1 -- 172.21.15.100:0/2163007695 wait complete. 2024-09-17T04:23:48.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.523+0000 7ff9c2c00640 1 -- 172.21.15.100:0/1920132094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7ff998005580 con 0x7ff9bc0cf870 2024-09-17T04:23:48.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.526+0000 7ff9b9600640 1 -- 172.21.15.100:0/1920132094 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v22)=0 v22) ==== 125+0+0 (secure 0 0 0) 0x7ff9b4032030 con 0x7ff9bc0cf870 2024-09-17T04:23:48.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.527+0000 7ff9c2c00640 1 -- 172.21.15.100:0/1920132094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9bc0cf870 msgr2=0x7ff9bc0a1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.527+0000 7ff9c2c00640 1 --2- 172.21.15.100:0/1920132094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9bc0cf870 0x7ff9bc0a1890 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4032000 tx=0x7ff9b402cf00 comp rx=0 tx=0).stop 2024-09-17T04:23:48.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.527+0000 7ff9c2c00640 1 -- 172.21.15.100:0/1920132094 shutdown_connections 2024-09-17T04:23:48.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.527+0000 7ff9c2c00640 1 --2- 172.21.15.100:0/1920132094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9bc0cf870 0x7ff9bc0a1890 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.527+0000 7ff9c2c00640 1 --2- 172.21.15.100:0/1920132094 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9bc0b9930 0x7ff9bc161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.527+0000 7ff9c2c00640 1 --2- 172.21.15.100:0/1920132094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9bc0b3690 0x7ff9bc15eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.527+0000 7ff9c2c00640 1 -- 172.21.15.100:0/1920132094 >> 172.21.15.100:0/1920132094 conn(0x7ff9bc08bfe0 msgr2=0x7ff9bc162590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:48.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.528+0000 7ff9c2c00640 1 -- 172.21.15.100:0/1920132094 shutdown_connections 2024-09-17T04:23:48.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.528+0000 7ff9c2c00640 1 -- 172.21.15.100:0/1920132094 wait complete. 2024-09-17T04:23:48.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.541+0000 7f734b200640 1 -- 172.21.15.100:0/2926025940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7f7344155a70 con 0x7f73440ccc50 2024-09-17T04:23:48.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.544+0000 7f733be00640 1 -- 172.21.15.100:0/2926025940 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v23)=0 v23) ==== 108+0+0 (secure 0 0 0) 0x7f73340047a0 con 0x7f73440ccc50 2024-09-17T04:23:48.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.546+0000 7f734b200640 1 -- 172.21.15.100:0/2926025940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73440ccc50 msgr2=0x7f7344152da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.546+0000 7f734b200640 1 --2- 172.21.15.100:0/2926025940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73440ccc50 0x7f7344152da0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f733400ac10 tx=0x7f7334007590 comp rx=0 tx=0).stop 2024-09-17T04:23:48.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.546+0000 7f734b200640 1 -- 172.21.15.100:0/2926025940 shutdown_connections 2024-09-17T04:23:48.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.546+0000 7f734b200640 1 --2- 172.21.15.100:0/2926025940 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73440cfab0 0x7f73441532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.546+0000 7f734b200640 1 --2- 172.21.15.100:0/2926025940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73440ccc50 0x7f7344152da0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.546+0000 7f734b200640 1 --2- 172.21.15.100:0/2926025940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7344073b70 0x7f7344143ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.546+0000 7f734b200640 1 -- 172.21.15.100:0/2926025940 >> 172.21.15.100:0/2926025940 conn(0x7f734406d1c0 msgr2=0x7f734415cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:48.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.546+0000 7f734b200640 1 -- 172.21.15.100:0/2926025940 shutdown_connections 2024-09-17T04:23:48.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.546+0000 7f734b200640 1 -- 172.21.15.100:0/2926025940 wait complete. 2024-09-17T04:23:48.596 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi100') 2024-09-17T04:23:48.596 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi100') 2024-09-17T04:23:48.596 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi100') 2024-09-17T04:23:48.596 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi100') 2024-09-17T04:23:48.597 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph config dump 2024-09-17T04:23:48.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.848+0000 7f8ac2600640 1 -- 172.21.15.100:0/3972370117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8abc152920 msgr2=0x7f8abc15f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.848+0000 7f8ac2600640 1 --2- 172.21.15.100:0/3972370117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8abc152920 0x7f8abc15f3d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8009830 tx=0x7f8aa802a530 comp rx=0 tx=0).stop 2024-09-17T04:23:48.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.848+0000 7f8ac2600640 1 -- 172.21.15.100:0/3972370117 shutdown_connections 2024-09-17T04:23:48.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.848+0000 7f8ac2600640 1 --2- 172.21.15.100:0/3972370117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8abc152920 0x7f8abc15f3d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.848+0000 7f8ac2600640 1 -- 172.21.15.100:0/3972370117 >> 172.21.15.100:0/3972370117 conn(0x7f8abc06d730 msgr2=0x7f8abc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:48.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.849+0000 7f8ac2600640 1 -- 172.21.15.100:0/3972370117 shutdown_connections 2024-09-17T04:23:48.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.849+0000 7f8ac2600640 1 -- 172.21.15.100:0/3972370117 wait complete. 2024-09-17T04:23:48.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.851+0000 7f8ac2600640 1 Processor -- start 2024-09-17T04:23:48.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.851+0000 7f8ac2600640 1 -- start start 2024-09-17T04:23:48.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.852+0000 7f8ac2600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8abc0aebc0 0x7f8abc0b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.852+0000 7f8ac2600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8abc152920 0x7f8abc151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.852+0000 7f8ac2600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8abc15f910 0x7f8abc0b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:48.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.852+0000 7f8ac2600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc0b4390 con 0x7f8abc15f910 2024-09-17T04:23:48.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.852+0000 7f8ac2600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc0b4210 con 0x7f8abc0aebc0 2024-09-17T04:23:48.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.852+0000 7f8ac2600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8abc0b4510 con 0x7f8abc152920 2024-09-17T04:23:48.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.852+0000 7f8abbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8abc15f910 0x7f8abc0b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:48.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.852+0000 7f8abb400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8abc0aebc0 0x7f8abc0b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:48.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.853+0000 7f8abbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8abc15f910 0x7f8abc0b0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37108/0 (socket says 172.21.15.100:37108) 2024-09-17T04:23:48.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.853+0000 7f8abbe00640 1 -- 172.21.15.100:0/94535849 learned_addr learned my addr 172.21.15.100:0/94535849 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:23:48.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.853+0000 7f8abaa00640 1 --2- 172.21.15.100:0/94535849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8abc152920 0x7f8abc151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:48.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.853+0000 7f8abbe00640 1 -- 172.21.15.100:0/94535849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8abc152920 msgr2=0x7f8abc151570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.853+0000 7f8abbe00640 1 --2- 172.21.15.100:0/94535849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8abc152920 0x7f8abc151570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.853+0000 7f8abbe00640 1 -- 172.21.15.100:0/94535849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8abc0aebc0 msgr2=0x7f8abc0b1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.853+0000 7f8abbe00640 1 --2- 172.21.15.100:0/94535849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8abc0aebc0 0x7f8abc0b1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.853+0000 7f8abbe00640 1 -- 172.21.15.100:0/94535849 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8abc0b2340 con 0x7f8abc15f910 2024-09-17T04:23:48.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.853+0000 7f8abb400640 1 --2- 172.21.15.100:0/94535849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8abc0aebc0 0x7f8abc0b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:23:48.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.853+0000 7f8abbe00640 1 --2- 172.21.15.100:0/94535849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8abc15f910 0x7f8abc0b0cc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8ab400cb40 tx=0x7f8ab400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:23:48.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.853+0000 7f8ab3400640 1 -- 172.21.15.100:0/94535849 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4016020 con 0x7f8abc15f910 2024-09-17T04:23:48.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.854+0000 7f8ab3400640 1 -- 172.21.15.100:0/94535849 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ab400adb0 con 0x7f8abc15f910 2024-09-17T04:23:48.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.854+0000 7f8ab3400640 1 -- 172.21.15.100:0/94535849 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4012680 con 0x7f8abc15f910 2024-09-17T04:23:48.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.854+0000 7f8ac2600640 1 -- 172.21.15.100:0/94535849 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8abc154be0 con 0x7f8abc15f910 2024-09-17T04:23:48.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.854+0000 7f8ac2600640 1 -- 172.21.15.100:0/94535849 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8abc155160 con 0x7f8abc15f910 2024-09-17T04:23:48.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.854+0000 7f8ab3400640 1 -- 172.21.15.100:0/94535849 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8ab4019020 con 0x7f8abc15f910 2024-09-17T04:23:48.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.854+0000 7f8ab3400640 1 -- 172.21.15.100:0/94535849 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8ab401bd00 con 0x7f8abc15f910 2024-09-17T04:23:48.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.855+0000 7f8ac2600640 1 -- 172.21.15.100:0/94535849 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8abc0b49c0 con 0x7f8abc15f910 2024-09-17T04:23:48.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.857+0000 7f8ab3400640 1 -- 172.21.15.100:0/94535849 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f8ab4020020 con 0x7f8abc15f910 2024-09-17T04:23:48.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.920+0000 7f8ac2600640 1 -- 172.21.15.100:0/94535849 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f8abc1eb700 con 0x7f8abc15f910 2024-09-17T04:23:48.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.921+0000 7f8ab3400640 1 -- 172.21.15.100:0/94535849 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v23) ==== 59+0+1472 (secure 0 0 0) 0x7f8abc154be0 con 0x7f8abc15f910 2024-09-17T04:23:48.921 INFO:teuthology.orchestra.run.smithi100.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T04:23:48.921 INFO:teuthology.orchestra.run.smithi100.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T04:23:48.921 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced client_mount_timeout 30 2024-09-17T04:23:48.921 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_client 20/20 2024-09-17T04:23:48.921 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_mgr 20/20 2024-09-17T04:23:48.921 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_ms 1/1 2024-09-17T04:23:48.921 INFO:teuthology.orchestra.run.smithi100.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T04:23:48.922 INFO:teuthology.orchestra.run.smithi100.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T04:23:48.922 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_mds 20/20 2024-09-17T04:23:48.922 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T04:23:48.922 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_ms 2/2 2024-09-17T04:23:48.922 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_debug_frag true 2024-09-17T04:23:48.922 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_debug_scatterstat true 2024-09-17T04:23:48.922 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T04:23:48.922 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_verify_scatter true 2024-09-17T04:23:48.922 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T04:23:48.922 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T04:23:48.923 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T04:23:48.923 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced client_mount_timeout 600 2024-09-17T04:23:48.923 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced debug_client 20/20 2024-09-17T04:23:48.923 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced debug_ms 1/1 2024-09-17T04:23:48.923 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T04:23:48.923 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T04:23:48.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.921+0000 7f8ac2600640 1 -- 172.21.15.100:0/94535849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8abc15f910 msgr2=0x7f8abc0b0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:48.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.921+0000 7f8ac2600640 1 --2- 172.21.15.100:0/94535849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8abc15f910 0x7f8abc0b0cc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8ab400cb40 tx=0x7f8ab400a060 comp rx=0 tx=0).stop 2024-09-17T04:23:48.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.921+0000 7f8ac2600640 1 -- 172.21.15.100:0/94535849 shutdown_connections 2024-09-17T04:23:48.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.921+0000 7f8ac2600640 1 --2- 172.21.15.100:0/94535849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8abc15f910 0x7f8abc0b0cc0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.921+0000 7f8ac2600640 1 --2- 172.21.15.100:0/94535849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8abc152920 0x7f8abc151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.921+0000 7f8ac2600640 1 --2- 172.21.15.100:0/94535849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8abc0aebc0 0x7f8abc0b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:48.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.921+0000 7f8ac2600640 1 -- 172.21.15.100:0/94535849 >> 172.21.15.100:0/94535849 conn(0x7f8abc06d730 msgr2=0x7f8abc0700a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:48.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.922+0000 7f8ac2600640 1 -- 172.21.15.100:0/94535849 shutdown_connections 2024-09-17T04:23:48.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:48.922+0000 7f8ac2600640 1 -- 172.21.15.100:0/94535849 wait complete. 2024-09-17T04:23:48.942 INFO:tasks.ceph:Setting crush tunables to default 2024-09-17T04:23:48.942 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd crush tunables default 2024-09-17T04:23:49.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.154+0000 7f93c3e00640 1 -- 172.21.15.100:0/1713172185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93c40b98b0 msgr2=0x7f93c40b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:49.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.154+0000 7f93c3e00640 1 --2- 172.21.15.100:0/1713172185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93c40b98b0 0x7f93c40b4770 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f93c40d3d40 tx=0x7f93b402a540 comp rx=0 tx=0).stop 2024-09-17T04:23:49.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.154+0000 7f93c3e00640 1 -- 172.21.15.100:0/1713172185 shutdown_connections 2024-09-17T04:23:49.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.154+0000 7f93c3e00640 1 --2- 172.21.15.100:0/1713172185 >> v2:172.21.15.184:3300/0 conn(0x7f93c40cf6a0 0x7f93c40d0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:49.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.154+0000 7f93c3e00640 1 --2- 172.21.15.100:0/1713172185 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93c40b4cb0 0x7f93c40cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:49.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.154+0000 7f93c3e00640 1 --2- 172.21.15.100:0/1713172185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93c40b98b0 0x7f93c40b4770 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:49.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.154+0000 7f93c3e00640 1 -- 172.21.15.100:0/1713172185 >> 172.21.15.100:0/1713172185 conn(0x7f93c408bfe0 msgr2=0x7f93c40d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:49.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.154+0000 7f93c3e00640 1 -- 172.21.15.100:0/1713172185 shutdown_connections 2024-09-17T04:23:49.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.154+0000 7f93c3e00640 1 -- 172.21.15.100:0/1713172185 wait complete. 2024-09-17T04:23:49.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.156+0000 7f93c3e00640 1 Processor -- start 2024-09-17T04:23:49.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.156+0000 7f93c3e00640 1 -- start start 2024-09-17T04:23:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.158+0000 7f93c3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93c40b4cb0 0x7f93c40d25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.158+0000 7f93c3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93c40b98b0 0x7f93c41f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.158+0000 7f93c3e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93c40cf6a0 0x7f93c40b7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.158+0000 7f93c3e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f93c40d14c0 con 0x7f93c40b98b0 2024-09-17T04:23:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.158+0000 7f93c3e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f93c40d1340 con 0x7f93c40b4cb0 2024-09-17T04:23:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.158+0000 7f93c3e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f93c40d1640 con 0x7f93c40cf6a0 2024-09-17T04:23:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.158+0000 7f93bbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93c40b98b0 0x7f93c41f4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.158+0000 7f93c2a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93c40b4cb0 0x7f93c40d25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.158+0000 7f93bbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93c40b98b0 0x7f93c41f4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37118/0 (socket says 172.21.15.100:37118) 2024-09-17T04:23:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.158+0000 7f93bbe00640 1 -- 172.21.15.100:0/1321814800 learned_addr learned my addr 172.21.15.100:0/1321814800 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:23:49.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.158+0000 7f93c3400640 1 --2- 172.21.15.100:0/1321814800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93c40cf6a0 0x7f93c40b7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.159+0000 7f93c2a00640 1 -- 172.21.15.100:0/1321814800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93c40cf6a0 msgr2=0x7f93c40b7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.159+0000 7f93c2a00640 1 --2- 172.21.15.100:0/1321814800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93c40cf6a0 0x7f93c40b7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.159+0000 7f93c2a00640 1 -- 172.21.15.100:0/1321814800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93c40b98b0 msgr2=0x7f93c41f4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.159+0000 7f93c2a00640 1 --2- 172.21.15.100:0/1321814800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93c40b98b0 0x7f93c41f4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.159+0000 7f93c2a00640 1 -- 172.21.15.100:0/1321814800 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c41f54e0 con 0x7f93c40b4cb0 2024-09-17T04:23:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.159+0000 7f93bbe00640 1 --2- 172.21.15.100:0/1321814800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93c40b98b0 0x7f93c41f4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:23:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.159+0000 7f93c2a00640 1 --2- 172.21.15.100:0/1321814800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93c40b4cb0 0x7f93c40d25a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f93c40b7130 tx=0x7f93b40052f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:23:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.159+0000 7f93baa00640 1 -- 172.21.15.100:0/1321814800 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4005cf0 con 0x7f93c40b4cb0 2024-09-17T04:23:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.159+0000 7f93baa00640 1 -- 172.21.15.100:0/1321814800 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93b4031c60 con 0x7f93c40b4cb0 2024-09-17T04:23:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.159+0000 7f93baa00640 1 -- 172.21.15.100:0/1321814800 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4004a00 con 0x7f93c40b4cb0 2024-09-17T04:23:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.159+0000 7f93c3e00640 1 -- 172.21.15.100:0/1321814800 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c40b2560 con 0x7f93c40b4cb0 2024-09-17T04:23:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.159+0000 7f93c3e00640 1 -- 172.21.15.100:0/1321814800 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c40b2b20 con 0x7f93c40b4cb0 2024-09-17T04:23:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.160+0000 7f93baa00640 1 -- 172.21.15.100:0/1321814800 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f93b4004ba0 con 0x7f93c40b4cb0 2024-09-17T04:23:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.160+0000 7f93baa00640 1 -- 172.21.15.100:0/1321814800 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f93b403b410 con 0x7f93c40b4cb0 2024-09-17T04:23:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.160+0000 7f93c3e00640 1 -- 172.21.15.100:0/1321814800 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93c40b4770 con 0x7f93c40b4cb0 2024-09-17T04:23:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.163+0000 7f93baa00640 1 -- 172.21.15.100:0/1321814800 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f93b404a030 con 0x7f93c40b4cb0 2024-09-17T04:23:49.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.230+0000 7f93c3e00640 1 -- 172.21.15.100:0/1321814800 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f93c40b77d0 con 0x7f93c40b4cb0 2024-09-17T04:23:49.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.234+0000 7f93baa00640 1 -- 172.21.15.100:0/1321814800 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f93b403f030 con 0x7f93c40b4cb0 2024-09-17T04:23:49.235 INFO:teuthology.orchestra.run.smithi100.stderr:adjusted tunables profile to default 2024-09-17T04:23:49.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.236+0000 7f93c3e00640 1 -- 172.21.15.100:0/1321814800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93c40b4cb0 msgr2=0x7f93c40d25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:49.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.236+0000 7f93c3e00640 1 --2- 172.21.15.100:0/1321814800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93c40b4cb0 0x7f93c40d25a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f93c40b7130 tx=0x7f93b40052f0 comp rx=0 tx=0).stop 2024-09-17T04:23:49.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.236+0000 7f93c3e00640 1 -- 172.21.15.100:0/1321814800 shutdown_connections 2024-09-17T04:23:49.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.236+0000 7f93c3e00640 1 --2- 172.21.15.100:0/1321814800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93c40cf6a0 0x7f93c40b7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:49.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.236+0000 7f93c3e00640 1 --2- 172.21.15.100:0/1321814800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93c40b98b0 0x7f93c41f4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:49.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.236+0000 7f93c3e00640 1 --2- 172.21.15.100:0/1321814800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93c40b4cb0 0x7f93c40d25a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:49.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.236+0000 7f93c3e00640 1 -- 172.21.15.100:0/1321814800 >> 172.21.15.100:0/1321814800 conn(0x7f93c408bfe0 msgr2=0x7f93c40d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:49.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.236+0000 7f93c3e00640 1 -- 172.21.15.100:0/1321814800 shutdown_connections 2024-09-17T04:23:49.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:49.236+0000 7f93c3e00640 1 -- 172.21.15.100:0/1321814800 wait complete. 2024-09-17T04:23:49.258 INFO:tasks.ceph:check_enable_crimson: False 2024-09-17T04:23:49.258 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-17T04:23:49.258 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:23:49.258 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-17T04:23:49.315 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:23:49.315 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-17T04:23:49.411 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:23:49.411 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-17T04:23:49.506 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:23:49.506 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-17T04:23:49.603 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:23:49.603 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-17T04:23:49.659 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:23:49.659 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-17T04:23:49.755 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:23:49.755 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-17T04:23:49.855 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:23:49.855 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-17T04:23:49.949 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 30913101-9146-4dd0-8dd1-fcf99fc3489f 0 2024-09-17T04:23:50.209 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.209+0000 7f1c3f000640 1 -- 172.21.15.184:0/1503518208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c380b7960 msgr2=0x7f1c380af1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:50.209 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.209+0000 7f1c3f000640 1 --2- 172.21.15.184:0/1503518208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c380b7960 0x7f1c380af1f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1c38074db0 tx=0x7f1c2802a420 comp rx=0 tx=0).stop 2024-09-17T04:23:50.209 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.209+0000 7f1c3f000640 1 -- 172.21.15.184:0/1503518208 shutdown_connections 2024-09-17T04:23:50.209 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.209+0000 7f1c3f000640 1 --2- 172.21.15.184:0/1503518208 >> v2:172.21.15.184:3300/0 conn(0x7f1c380af730 0x7f1c380d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:50.209 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.209+0000 7f1c3f000640 1 --2- 172.21.15.184:0/1503518208 >> v2:172.21.15.100:3300/0 conn(0x7f1c380b7d70 0x7f1c380d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:50.210 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.209+0000 7f1c3f000640 1 --2- 172.21.15.184:0/1503518208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c380b7960 0x7f1c380af1f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:50.210 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.209+0000 7f1c3f000640 1 -- 172.21.15.184:0/1503518208 >> 172.21.15.184:0/1503518208 conn(0x7f1c380cef20 msgr2=0x7f1c380cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:50.210 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.209+0000 7f1c3f000640 1 -- 172.21.15.184:0/1503518208 shutdown_connections 2024-09-17T04:23:50.210 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.209+0000 7f1c3f000640 1 -- 172.21.15.184:0/1503518208 wait complete. 2024-09-17T04:23:50.211 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.211+0000 7f1c3f000640 1 Processor -- start 2024-09-17T04:23:50.211 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.212+0000 7f1c3f000640 1 -- start start 2024-09-17T04:23:50.213 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.213+0000 7f1c3f000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c380af730 0x7f1c38153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:50.214 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.213+0000 7f1c3f000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c380b7960 0x7f1c38155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:50.214 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.213+0000 7f1c3f000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c380b7d70 0x7f1c38156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:50.214 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.213+0000 7f1c3f000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1c380ceb90 con 0x7f1c380af730 2024-09-17T04:23:50.214 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.213+0000 7f1c3f000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1c380b8310 con 0x7f1c380b7960 2024-09-17T04:23:50.214 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.213+0000 7f1c3f000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1c380ced10 con 0x7f1c380b7d70 2024-09-17T04:23:50.214 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.213+0000 7f1c2fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c380b7960 0x7f1c38155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:50.214 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.213+0000 7f1c3d000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c380b7d70 0x7f1c38156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:50.214 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.213+0000 7f1c2fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c380b7960 0x7f1c38155af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:42964/0 (socket says 172.21.15.184:42964) 2024-09-17T04:23:50.215 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.213+0000 7f1c2fe00640 1 -- 172.21.15.184:0/3062066359 learned_addr learned my addr 172.21.15.184:0/3062066359 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-09-17T04:23:50.215 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.213+0000 7f1c37e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c380af730 0x7f1c38153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:50.215 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.214+0000 7f1c2fe00640 1 -- 172.21.15.184:0/3062066359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c380b7d70 msgr2=0x7f1c38156030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:50.215 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.214+0000 7f1c2fe00640 1 --2- 172.21.15.184:0/3062066359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c380b7d70 0x7f1c38156030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:50.215 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.214+0000 7f1c2fe00640 1 -- 172.21.15.184:0/3062066359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c380af730 msgr2=0x7f1c38153ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:50.215 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.214+0000 7f1c2fe00640 1 --2- 172.21.15.184:0/3062066359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c380af730 0x7f1c38153ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:50.216 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.214+0000 7f1c2fe00640 1 -- 172.21.15.184:0/3062066359 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c38154220 con 0x7f1c380b7960 2024-09-17T04:23:50.216 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.214+0000 7f1c2fe00640 1 --2- 172.21.15.184:0/3062066359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c380b7960 0x7f1c38155af0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1c2000d6d0 tx=0x7f1c2000dad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:23:50.218 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.214+0000 7f1c34c00640 1 -- 172.21.15.184:0/3062066359 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c20017070 con 0x7f1c380b7960 2024-09-17T04:23:50.218 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.215+0000 7f1c3f000640 1 -- 172.21.15.184:0/3062066359 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c38156800 con 0x7f1c380b7960 2024-09-17T04:23:50.218 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.214+0000 7f1c34c00640 1 -- 172.21.15.184:0/3062066359 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c20004310 con 0x7f1c380b7960 2024-09-17T04:23:50.218 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.215+0000 7f1c34c00640 1 -- 172.21.15.184:0/3062066359 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c20005280 con 0x7f1c380b7960 2024-09-17T04:23:50.218 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.215+0000 7f1c3f000640 1 -- 172.21.15.184:0/3062066359 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c3806e030 con 0x7f1c380b7960 2024-09-17T04:23:50.219 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.215+0000 7f1c34c00640 1 -- 172.21.15.184:0/3062066359 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1c200047a0 con 0x7f1c380b7960 2024-09-17T04:23:50.219 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.215+0000 7f1c34c00640 1 -- 172.21.15.184:0/3062066359 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f1c200035f0 con 0x7f1c380b7960 2024-09-17T04:23:50.219 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.216+0000 7f1c3f000640 1 -- 172.21.15.184:0/3062066359 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c38156570 con 0x7f1c380b7960 2024-09-17T04:23:50.219 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.220+0000 7f1c34c00640 1 -- 172.21.15.184:0/3062066359 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f1c20020020 con 0x7f1c380b7960 2024-09-17T04:23:50.293 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.294+0000 7f1c3f000640 1 -- 172.21.15.184:0/3062066359 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd new", "uuid": "30913101-9146-4dd0-8dd1-fcf99fc3489f", "id": 0} v 0) -- 0x7f1c380cd620 con 0x7f1c380b7960 2024-09-17T04:23:50.299 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.300+0000 7f1c34c00640 1 -- 172.21.15.184:0/3062066359 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "30913101-9146-4dd0-8dd1-fcf99fc3489f", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f1c20009d90 con 0x7f1c380b7960 2024-09-17T04:23:50.299 INFO:teuthology.orchestra.run.smithi184.stdout:0 2024-09-17T04:23:50.301 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.301+0000 7f1c3f000640 1 -- 172.21.15.184:0/3062066359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c380b7960 msgr2=0x7f1c38155af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:50.301 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.301+0000 7f1c3f000640 1 --2- 172.21.15.184:0/3062066359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c380b7960 0x7f1c38155af0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1c2000d6d0 tx=0x7f1c2000dad0 comp rx=0 tx=0).stop 2024-09-17T04:23:50.301 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.301+0000 7f1c3f000640 1 -- 172.21.15.184:0/3062066359 shutdown_connections 2024-09-17T04:23:50.301 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.301+0000 7f1c3f000640 1 --2- 172.21.15.184:0/3062066359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c380b7d70 0x7f1c38156030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:50.301 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.301+0000 7f1c3f000640 1 --2- 172.21.15.184:0/3062066359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c380b7960 0x7f1c38155af0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:50.301 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.301+0000 7f1c3f000640 1 --2- 172.21.15.184:0/3062066359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c380af730 0x7f1c38153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:50.302 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.301+0000 7f1c3f000640 1 -- 172.21.15.184:0/3062066359 >> 172.21.15.184:0/3062066359 conn(0x7f1c380cef20 msgr2=0x7f1c380b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:50.302 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.301+0000 7f1c3f000640 1 -- 172.21.15.184:0/3062066359 shutdown_connections 2024-09-17T04:23:50.302 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.301+0000 7f1c3f000640 1 -- 172.21.15.184:0/3062066359 wait complete. 2024-09-17T04:23:50.324 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 26225b84-4d11-4e1d-bf54-c91e7899f413 1 2024-09-17T04:23:50.545 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.545+0000 7efe09e00640 1 -- 172.21.15.184:0/1045375852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe040b66b0 msgr2=0x7efe0406e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:50.545 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.545+0000 7efe09e00640 1 --2- 172.21.15.184:0/1045375852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe040b66b0 0x7efe0406e5d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009860 tx=0x7efdf002a510 comp rx=0 tx=0).stop 2024-09-17T04:23:50.545 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.545+0000 7efe09e00640 1 -- 172.21.15.184:0/1045375852 shutdown_connections 2024-09-17T04:23:50.545 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.545+0000 7efe09e00640 1 --2- 172.21.15.184:0/1045375852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe040b66b0 0x7efe0406e5d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:50.545 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.545+0000 7efe09e00640 1 -- 172.21.15.184:0/1045375852 >> 172.21.15.184:0/1045375852 conn(0x7efe040d4520 msgr2=0x7efe040d4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:50.545 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.545+0000 7efe09e00640 1 -- 172.21.15.184:0/1045375852 shutdown_connections 2024-09-17T04:23:50.545 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.545+0000 7efe09e00640 1 -- 172.21.15.184:0/1045375852 wait complete. 2024-09-17T04:23:50.546 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.547+0000 7efe09e00640 1 Processor -- start 2024-09-17T04:23:50.547 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.547+0000 7efe09e00640 1 -- start start 2024-09-17T04:23:50.549 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.549+0000 7efe09e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe040b98e0 0x7efe0415d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:50.549 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.549+0000 7efe09e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe0415d830 0x7efe040b5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:50.549 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.549+0000 7efe09e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe040b5ab0 0x7efe041f8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:50.549 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.549+0000 7efe09e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efe0406f3a0 con 0x7efe040b98e0 2024-09-17T04:23:50.549 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.549+0000 7efe09e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efe0406f220 con 0x7efe040b5ab0 2024-09-17T04:23:50.550 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.549+0000 7efe09e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efe0406f520 con 0x7efe0415d830 2024-09-17T04:23:50.550 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.549+0000 7efe03e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe0415d830 0x7efe040b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:50.550 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.549+0000 7efe03e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe0415d830 0x7efe040b5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:48702/0 (socket says 172.21.15.184:48702) 2024-09-17T04:23:50.550 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.549+0000 7efe03e00640 1 -- 172.21.15.184:0/2429421501 learned_addr learned my addr 172.21.15.184:0/2429421501 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-09-17T04:23:50.550 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.549+0000 7efe09400640 1 --2- 172.21.15.184:0/2429421501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe040b5ab0 0x7efe041f8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:50.550 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.549+0000 7efe08a00640 1 --2- 172.21.15.184:0/2429421501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe040b98e0 0x7efe0415d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:50.550 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.550+0000 7efe09400640 1 -- 172.21.15.184:0/2429421501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe0415d830 msgr2=0x7efe040b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:50.550 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.550+0000 7efe09400640 1 --2- 172.21.15.184:0/2429421501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe0415d830 0x7efe040b5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:50.550 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.550+0000 7efe09400640 1 -- 172.21.15.184:0/2429421501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe040b98e0 msgr2=0x7efe0415d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:50.550 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.550+0000 7efe09400640 1 --2- 172.21.15.184:0/2429421501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe040b98e0 0x7efe0415d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:50.550 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.550+0000 7efe09400640 1 -- 172.21.15.184:0/2429421501 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0415d640 con 0x7efe040b5ab0 2024-09-17T04:23:50.550 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.550+0000 7efe08a00640 1 --2- 172.21.15.184:0/2429421501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe040b98e0 0x7efe0415d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:23:50.550 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.550+0000 7efe03e00640 1 --2- 172.21.15.184:0/2429421501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe0415d830 0x7efe040b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:23:50.551 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.550+0000 7efe09400640 1 --2- 172.21.15.184:0/2429421501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe040b5ab0 0x7efe041f8cd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00ca50 tx=0x7efdfc00cf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:23:50.551 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.550+0000 7efe00c00640 1 -- 172.21.15.184:0/2429421501 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc016020 con 0x7efe040b5ab0 2024-09-17T04:23:50.551 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.550+0000 7efe00c00640 1 -- 172.21.15.184:0/2429421501 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efdfc00aca0 con 0x7efe040b5ab0 2024-09-17T04:23:50.552 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.550+0000 7efe00c00640 1 -- 172.21.15.184:0/2429421501 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc012680 con 0x7efe040b5ab0 2024-09-17T04:23:50.552 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.550+0000 7efe09e00640 1 -- 172.21.15.184:0/2429421501 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe040b5ec0 con 0x7efe040b5ab0 2024-09-17T04:23:50.552 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.550+0000 7efe09e00640 1 -- 172.21.15.184:0/2429421501 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe040cb970 con 0x7efe040b5ab0 2024-09-17T04:23:50.552 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.551+0000 7efe00c00640 1 -- 172.21.15.184:0/2429421501 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efdfc019020 con 0x7efe040b5ab0 2024-09-17T04:23:50.552 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.551+0000 7efe00c00640 1 -- 172.21.15.184:0/2429421501 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7efdfc01bc20 con 0x7efe040b5ab0 2024-09-17T04:23:50.552 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.551+0000 7efe09e00640 1 -- 172.21.15.184:0/2429421501 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe040d4100 con 0x7efe040b5ab0 2024-09-17T04:23:50.553 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.554+0000 7efe00c00640 1 -- 172.21.15.184:0/2429421501 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7efdfc020020 con 0x7efe040b5ab0 2024-09-17T04:23:50.626 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.627+0000 7efe09e00640 1 -- 172.21.15.184:0/2429421501 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd new", "uuid": "26225b84-4d11-4e1d-bf54-c91e7899f413", "id": 1} v 0) -- 0x7efe0406feb0 con 0x7efe040b5ab0 2024-09-17T04:23:50.633 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.633+0000 7efe00c00640 1 -- 172.21.15.184:0/2429421501 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "26225b84-4d11-4e1d-bf54-c91e7899f413", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7efdfc01be20 con 0x7efe040b5ab0 2024-09-17T04:23:50.633 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-09-17T04:23:50.634 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.634+0000 7efe09e00640 1 -- 172.21.15.184:0/2429421501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe040b5ab0 msgr2=0x7efe041f8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:50.634 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.634+0000 7efe09e00640 1 --2- 172.21.15.184:0/2429421501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe040b5ab0 0x7efe041f8cd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00ca50 tx=0x7efdfc00cf10 comp rx=0 tx=0).stop 2024-09-17T04:23:50.634 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.634+0000 7efe09e00640 1 -- 172.21.15.184:0/2429421501 shutdown_connections 2024-09-17T04:23:50.634 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.634+0000 7efe09e00640 1 --2- 172.21.15.184:0/2429421501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe040b5ab0 0x7efe041f8cd0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:50.634 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.634+0000 7efe09e00640 1 --2- 172.21.15.184:0/2429421501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe0415d830 0x7efe040b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:50.634 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.634+0000 7efe09e00640 1 --2- 172.21.15.184:0/2429421501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe040b98e0 0x7efe0415d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:50.635 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.634+0000 7efe09e00640 1 -- 172.21.15.184:0/2429421501 >> 172.21.15.184:0/2429421501 conn(0x7efe040d4520 msgr2=0x7efe040b2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:50.635 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.634+0000 7efe09e00640 1 -- 172.21.15.184:0/2429421501 shutdown_connections 2024-09-17T04:23:50.635 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.635+0000 7efe09e00640 1 -- 172.21.15.184:0/2429421501 wait complete. 2024-09-17T04:23:50.655 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new f12f76eb-7482-4c5b-847d-99b9746807c5 2 2024-09-17T04:23:50.876 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.877+0000 7f4041e00640 1 -- 172.21.15.184:0/511619906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403c0741f0 msgr2=0x7f403c074600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:50.877 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.877+0000 7f4041e00640 1 --2- 172.21.15.184:0/511619906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403c0741f0 0x7f403c074600 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4028005fd0 tx=0x7f402802a5d0 comp rx=0 tx=0).stop 2024-09-17T04:23:50.877 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.877+0000 7f4041e00640 1 -- 172.21.15.184:0/511619906 shutdown_connections 2024-09-17T04:23:50.877 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.877+0000 7f4041e00640 1 --2- 172.21.15.184:0/511619906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403c0741f0 0x7f403c074600 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:50.877 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.877+0000 7f4041e00640 1 --2- 172.21.15.184:0/511619906 >> v2:172.21.15.184:3300/0 conn(0x7f403c06fb20 0x7f403c1534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:50.877 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.877+0000 7f4041e00640 1 -- 172.21.15.184:0/511619906 >> 172.21.15.184:0/511619906 conn(0x7f403c0b1cd0 msgr2=0x7f403c0b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:50.877 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.877+0000 7f4041e00640 1 -- 172.21.15.184:0/511619906 shutdown_connections 2024-09-17T04:23:50.877 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.877+0000 7f4041e00640 1 -- 172.21.15.184:0/511619906 wait complete. 2024-09-17T04:23:50.878 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.879+0000 7f4041e00640 1 Processor -- start 2024-09-17T04:23:50.878 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.879+0000 7f4041e00640 1 -- start start 2024-09-17T04:23:50.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.881+0000 7f4041e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f403c06fb20 0x7f403c155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:50.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.881+0000 7f4041e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f403c0741f0 0x7f403c15f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:50.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.881+0000 7f4041e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403c0aefd0 0x7f403c15f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:50.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.881+0000 7f4041e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f403c15cfd0 con 0x7f403c0741f0 2024-09-17T04:23:50.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.881+0000 7f4041e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f403c15ce50 con 0x7f403c06fb20 2024-09-17T04:23:50.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.881+0000 7f4041e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f403c15d150 con 0x7f403c0aefd0 2024-09-17T04:23:50.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.881+0000 7f4040a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f403c06fb20 0x7f403c155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:50.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.881+0000 7f4041400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403c0aefd0 0x7f403c15f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:50.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.881+0000 7f4040a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f403c06fb20 0x7f403c155a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:43002/0 (socket says 172.21.15.184:43002) 2024-09-17T04:23:50.882 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.881+0000 7f4040a00640 1 -- 172.21.15.184:0/769807166 learned_addr learned my addr 172.21.15.184:0/769807166 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-09-17T04:23:50.882 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.881+0000 7f403be00640 1 --2- 172.21.15.184:0/769807166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f403c0741f0 0x7f403c15f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:50.882 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.881+0000 7f4041400640 1 -- 172.21.15.184:0/769807166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f403c06fb20 msgr2=0x7f403c155a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:50.882 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.881+0000 7f4041400640 1 --2- 172.21.15.184:0/769807166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f403c06fb20 0x7f403c155a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:50.882 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.881+0000 7f4041400640 1 -- 172.21.15.184:0/769807166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f403c0741f0 msgr2=0x7f403c15f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:50.882 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.881+0000 7f4041400640 1 --2- 172.21.15.184:0/769807166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f403c0741f0 0x7f403c15f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:50.882 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.881+0000 7f4041400640 1 -- 172.21.15.184:0/769807166 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403c157410 con 0x7f403c0aefd0 2024-09-17T04:23:50.882 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.882+0000 7f4041400640 1 --2- 172.21.15.184:0/769807166 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403c0aefd0 0x7f403c15f900 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f403400c730 tx=0x7f403400cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:23:50.883 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.882+0000 7f4038c00640 1 -- 172.21.15.184:0/769807166 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034003b70 con 0x7f403c0aefd0 2024-09-17T04:23:50.883 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.882+0000 7f4038c00640 1 -- 172.21.15.184:0/769807166 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4034003d10 con 0x7f403c0aefd0 2024-09-17T04:23:50.883 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.882+0000 7f4041e00640 1 -- 172.21.15.184:0/769807166 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f403c1562a0 con 0x7f403c0aefd0 2024-09-17T04:23:50.883 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.882+0000 7f4038c00640 1 -- 172.21.15.184:0/769807166 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034005330 con 0x7f403c0aefd0 2024-09-17T04:23:50.883 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.882+0000 7f4041e00640 1 -- 172.21.15.184:0/769807166 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f403c156860 con 0x7f403c0aefd0 2024-09-17T04:23:50.883 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.882+0000 7f4038c00640 1 -- 172.21.15.184:0/769807166 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f40340056e0 con 0x7f403c0aefd0 2024-09-17T04:23:50.883 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.882+0000 7f4038c00640 1 -- 172.21.15.184:0/769807166 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f403400aa60 con 0x7f403c0aefd0 2024-09-17T04:23:50.883 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.883+0000 7f4041e00640 1 -- 172.21.15.184:0/769807166 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f403c156010 con 0x7f403c0aefd0 2024-09-17T04:23:50.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.886+0000 7f4038c00640 1 -- 172.21.15.184:0/769807166 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f403400ad60 con 0x7f403c0aefd0 2024-09-17T04:23:50.960 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.961+0000 7f4041e00640 1 -- 172.21.15.184:0/769807166 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f12f76eb-7482-4c5b-847d-99b9746807c5", "id": 2} v 0) -- 0x7f403c15d670 con 0x7f403c0aefd0 2024-09-17T04:23:50.965 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.966+0000 7f4038c00640 1 -- 172.21.15.184:0/769807166 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f12f76eb-7482-4c5b-847d-99b9746807c5", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f403400a640 con 0x7f403c0aefd0 2024-09-17T04:23:50.965 INFO:teuthology.orchestra.run.smithi184.stdout:2 2024-09-17T04:23:50.966 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.967+0000 7f4041e00640 1 -- 172.21.15.184:0/769807166 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403c0aefd0 msgr2=0x7f403c15f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:50.966 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.967+0000 7f4041e00640 1 --2- 172.21.15.184:0/769807166 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403c0aefd0 0x7f403c15f900 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f403400c730 tx=0x7f403400cbf0 comp rx=0 tx=0).stop 2024-09-17T04:23:50.967 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.967+0000 7f4041e00640 1 -- 172.21.15.184:0/769807166 shutdown_connections 2024-09-17T04:23:50.967 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.967+0000 7f4041e00640 1 --2- 172.21.15.184:0/769807166 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403c0aefd0 0x7f403c15f900 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:50.967 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.967+0000 7f4041e00640 1 --2- 172.21.15.184:0/769807166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f403c0741f0 0x7f403c15f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:50.967 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.967+0000 7f4041e00640 1 --2- 172.21.15.184:0/769807166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f403c06fb20 0x7f403c155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:50.967 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.967+0000 7f4041e00640 1 -- 172.21.15.184:0/769807166 >> 172.21.15.184:0/769807166 conn(0x7f403c0b1cd0 msgr2=0x7f403c0afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:50.967 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.967+0000 7f4041e00640 1 -- 172.21.15.184:0/769807166 shutdown_connections 2024-09-17T04:23:50.967 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:50.967+0000 7f4041e00640 1 -- 172.21.15.184:0/769807166 wait complete. 2024-09-17T04:23:50.987 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new c2694cdd-d620-4025-96d9-9aebbb2cedce 3 2024-09-17T04:23:51.210 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.210+0000 7fec7a200640 1 -- 172.21.15.184:0/3509800269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec740cf730 msgr2=0x7fec740cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:51.210 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.210+0000 7fec7a200640 1 --2- 172.21.15.184:0/3509800269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec740cf730 0x7fec740cfb40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fec60005fd0 tx=0x7fec6002a5d0 comp rx=0 tx=0).stop 2024-09-17T04:23:51.210 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.210+0000 7fec7a200640 1 -- 172.21.15.184:0/3509800269 shutdown_connections 2024-09-17T04:23:51.210 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.210+0000 7fec7a200640 1 --2- 172.21.15.184:0/3509800269 >> v2:172.21.15.100:3300/0 conn(0x7fec7406df10 0x7fec7406e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:51.210 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.210+0000 7fec7a200640 1 --2- 172.21.15.184:0/3509800269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec740cf730 0x7fec740cfb40 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:51.211 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.210+0000 7fec7a200640 1 -- 172.21.15.184:0/3509800269 >> 172.21.15.184:0/3509800269 conn(0x7fec740d4510 msgr2=0x7fec740d4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:51.211 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.210+0000 7fec7a200640 1 -- 172.21.15.184:0/3509800269 shutdown_connections 2024-09-17T04:23:51.211 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.210+0000 7fec7a200640 1 -- 172.21.15.184:0/3509800269 wait complete. 2024-09-17T04:23:51.213 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.213+0000 7fec7a200640 1 Processor -- start 2024-09-17T04:23:51.213 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.214+0000 7fec7a200640 1 -- start start 2024-09-17T04:23:51.216 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.216+0000 7fec7a200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec7406df10 0x7fec741619f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:51.217 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.216+0000 7fec7a200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec740cf730 0x7fec74165260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:51.217 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.216+0000 7fec7a200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec740b2240 0x7fec740b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:51.217 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.216+0000 7fec7a200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fec7406fbc0 con 0x7fec7406df10 2024-09-17T04:23:51.217 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.216+0000 7fec7a200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fec7408cff0 con 0x7fec740cf730 2024-09-17T04:23:51.217 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.216+0000 7fec7a200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fec74165960 con 0x7fec740b2240 2024-09-17T04:23:51.217 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.216+0000 7fec73e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec740cf730 0x7fec74165260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:51.218 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.216+0000 7fec73e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec740cf730 0x7fec74165260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:43018/0 (socket says 172.21.15.184:43018) 2024-09-17T04:23:51.218 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.216+0000 7fec73e00640 1 -- 172.21.15.184:0/17768573 learned_addr learned my addr 172.21.15.184:0/17768573 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-09-17T04:23:51.218 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.216+0000 7fec78e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec7406df10 0x7fec741619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:51.218 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.216+0000 7fec79800640 1 --2- 172.21.15.184:0/17768573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec740b2240 0x7fec740b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:51.218 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.217+0000 7fec78e00640 1 -- 172.21.15.184:0/17768573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec740b2240 msgr2=0x7fec740b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:51.218 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.217+0000 7fec78e00640 1 --2- 172.21.15.184:0/17768573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec740b2240 0x7fec740b2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:51.218 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.217+0000 7fec78e00640 1 -- 172.21.15.184:0/17768573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec740cf730 msgr2=0x7fec74165260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:51.218 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.217+0000 7fec78e00640 1 --2- 172.21.15.184:0/17768573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec740cf730 0x7fec74165260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:51.218 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.217+0000 7fec78e00640 1 -- 172.21.15.184:0/17768573 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec740b2b90 con 0x7fec7406df10 2024-09-17T04:23:51.218 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.217+0000 7fec73e00640 1 --2- 172.21.15.184:0/17768573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec740cf730 0x7fec74165260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:23:51.218 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.217+0000 7fec78e00640 1 --2- 172.21.15.184:0/17768573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec7406df10 0x7fec741619f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fec6800c390 tx=0x7fec6800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:23:51.220 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.217+0000 7fec70c00640 1 -- 172.21.15.184:0/17768573 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec680052d0 con 0x7fec7406df10 2024-09-17T04:23:51.220 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.218+0000 7fec70c00640 1 -- 172.21.15.184:0/17768573 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec68007240 con 0x7fec7406df10 2024-09-17T04:23:51.220 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.218+0000 7fec70c00640 1 -- 172.21.15.184:0/17768573 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68007690 con 0x7fec7406df10 2024-09-17T04:23:51.220 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.218+0000 7fec7a200640 1 -- 172.21.15.184:0/17768573 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec741f8b60 con 0x7fec7406df10 2024-09-17T04:23:51.220 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.218+0000 7fec7a200640 1 -- 172.21.15.184:0/17768573 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec741f90e0 con 0x7fec7406df10 2024-09-17T04:23:51.220 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.218+0000 7fec70c00640 1 -- 172.21.15.184:0/17768573 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fec680067b0 con 0x7fec7406df10 2024-09-17T04:23:51.220 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.218+0000 7fec70c00640 1 -- 172.21.15.184:0/17768573 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fec68003d20 con 0x7fec7406df10 2024-09-17T04:23:51.221 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.219+0000 7fec7a200640 1 -- 172.21.15.184:0/17768573 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec7406a060 con 0x7fec7406df10 2024-09-17T04:23:51.221 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.221+0000 7fec70c00640 1 -- 172.21.15.184:0/17768573 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fec7406a060 con 0x7fec7406df10 2024-09-17T04:23:51.293 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.293+0000 7fec7a200640 1 -- 172.21.15.184:0/17768573 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c2694cdd-d620-4025-96d9-9aebbb2cedce", "id": 3} v 0) -- 0x7fec740b9e20 con 0x7fec7406df10 2024-09-17T04:23:51.295 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.296+0000 7fec70c00640 1 -- 172.21.15.184:0/17768573 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c2694cdd-d620-4025-96d9-9aebbb2cedce", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fec68023080 con 0x7fec7406df10 2024-09-17T04:23:51.296 INFO:teuthology.orchestra.run.smithi184.stdout:3 2024-09-17T04:23:51.297 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.297+0000 7fec7a200640 1 -- 172.21.15.184:0/17768573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec7406df10 msgr2=0x7fec741619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:51.297 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.297+0000 7fec7a200640 1 --2- 172.21.15.184:0/17768573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec7406df10 0x7fec741619f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fec6800c390 tx=0x7fec6800c850 comp rx=0 tx=0).stop 2024-09-17T04:23:51.297 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.298+0000 7fec7a200640 1 -- 172.21.15.184:0/17768573 shutdown_connections 2024-09-17T04:23:51.298 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.298+0000 7fec7a200640 1 --2- 172.21.15.184:0/17768573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec740b2240 0x7fec740b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:51.298 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.298+0000 7fec7a200640 1 --2- 172.21.15.184:0/17768573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec740cf730 0x7fec74165260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:51.298 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.298+0000 7fec7a200640 1 --2- 172.21.15.184:0/17768573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec7406df10 0x7fec741619f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:51.298 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.298+0000 7fec7a200640 1 -- 172.21.15.184:0/17768573 >> 172.21.15.184:0/17768573 conn(0x7fec740d4510 msgr2=0x7fec740ceb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:51.298 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.298+0000 7fec7a200640 1 -- 172.21.15.184:0/17768573 shutdown_connections 2024-09-17T04:23:51.298 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.298+0000 7fec7a200640 1 -- 172.21.15.184:0/17768573 wait complete. 2024-09-17T04:23:51.320 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new e5b4d175-f95f-4dc1-b00e-75bd8604cbd3 4 2024-09-17T04:23:51.541 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.542+0000 7f5dea000640 1 -- 172.21.15.184:0/2508102321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5de4070160 msgr2=0x7f5de40ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:51.542 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.542+0000 7f5dea000640 1 --2- 172.21.15.184:0/2508102321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5de4070160 0x7f5de40ccb50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc005fd0 tx=0x7f5ddc02aa90 comp rx=0 tx=0).stop 2024-09-17T04:23:51.542 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.542+0000 7f5dea000640 1 -- 172.21.15.184:0/2508102321 shutdown_connections 2024-09-17T04:23:51.542 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.542+0000 7f5dea000640 1 --2- 172.21.15.184:0/2508102321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5de4070160 0x7f5de40ccb50 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:51.542 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.542+0000 7f5dea000640 1 --2- 172.21.15.184:0/2508102321 >> v2:172.21.15.184:3300/0 conn(0x7f5de415f290 0x7f5de415ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:51.542 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.542+0000 7f5dea000640 1 -- 172.21.15.184:0/2508102321 >> 172.21.15.184:0/2508102321 conn(0x7f5de406d620 msgr2=0x7f5de40b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:51.542 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.542+0000 7f5dea000640 1 -- 172.21.15.184:0/2508102321 shutdown_connections 2024-09-17T04:23:51.542 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.542+0000 7f5dea000640 1 -- 172.21.15.184:0/2508102321 wait complete. 2024-09-17T04:23:51.543 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.544+0000 7f5dea000640 1 Processor -- start 2024-09-17T04:23:51.543 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.544+0000 7f5dea000640 1 -- start start 2024-09-17T04:23:51.546 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.546+0000 7f5dea000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5de4070160 0x7f5de40aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:51.546 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.546+0000 7f5dea000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5de40b30d0 0x7f5de40af120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:51.546 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.546+0000 7f5dea000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de415f290 0x7f5de414e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:51.546 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.546+0000 7f5dea000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5de4070780 con 0x7f5de415f290 2024-09-17T04:23:51.546 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.546+0000 7f5dea000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5de40b3680 con 0x7f5de40b30d0 2024-09-17T04:23:51.546 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.546+0000 7f5dea000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5de415e020 con 0x7f5de4070160 2024-09-17T04:23:51.546 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.546+0000 7f5de8c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5de4070160 0x7f5de40aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:51.546 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.546+0000 7f5de8c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5de4070160 0x7f5de40aebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:48768/0 (socket says 172.21.15.184:48768) 2024-09-17T04:23:51.547 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.546+0000 7f5de8c00640 1 -- 172.21.15.184:0/194346604 learned_addr learned my addr 172.21.15.184:0/194346604 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-09-17T04:23:51.547 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.546+0000 7f5de3e00640 1 --2- 172.21.15.184:0/194346604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5de40b30d0 0x7f5de40af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:51.547 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.546+0000 7f5de9600640 1 --2- 172.21.15.184:0/194346604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de415f290 0x7f5de414e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:51.547 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.546+0000 7f5de8c00640 1 -- 172.21.15.184:0/194346604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5de40b30d0 msgr2=0x7f5de40af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:51.547 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.546+0000 7f5de8c00640 1 --2- 172.21.15.184:0/194346604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5de40b30d0 0x7f5de40af120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:51.547 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.546+0000 7f5de8c00640 1 -- 172.21.15.184:0/194346604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de415f290 msgr2=0x7f5de414e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:51.547 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.546+0000 7f5de8c00640 1 --2- 172.21.15.184:0/194346604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de415f290 0x7f5de414e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:51.547 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.546+0000 7f5de8c00640 1 -- 172.21.15.184:0/194346604 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de415b4d0 con 0x7f5de4070160 2024-09-17T04:23:51.547 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.546+0000 7f5de3e00640 1 --2- 172.21.15.184:0/194346604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5de40b30d0 0x7f5de40af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:23:51.547 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.547+0000 7f5de8c00640 1 --2- 172.21.15.184:0/194346604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5de4070160 0x7f5de40aebe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800eac0 tx=0x7f5dd800ef80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:23:51.548 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.547+0000 7f5de0c00640 1 -- 172.21.15.184:0/194346604 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd801b070 con 0x7f5de4070160 2024-09-17T04:23:51.548 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.547+0000 7f5de0c00640 1 -- 172.21.15.184:0/194346604 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5dd800cd70 con 0x7f5de4070160 2024-09-17T04:23:51.548 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.547+0000 7f5de0c00640 1 -- 172.21.15.184:0/194346604 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8016670 con 0x7f5de4070160 2024-09-17T04:23:51.549 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.547+0000 7f5dea000640 1 -- 172.21.15.184:0/194346604 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de41f8d30 con 0x7f5de4070160 2024-09-17T04:23:51.549 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.547+0000 7f5dea000640 1 -- 172.21.15.184:0/194346604 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de41f92f0 con 0x7f5de4070160 2024-09-17T04:23:51.549 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.547+0000 7f5de0c00640 1 -- 172.21.15.184:0/194346604 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5dd801e070 con 0x7f5de4070160 2024-09-17T04:23:51.549 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.548+0000 7f5de0c00640 1 -- 172.21.15.184:0/194346604 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f5dd8020440 con 0x7f5de4070160 2024-09-17T04:23:51.549 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.548+0000 7f5dea000640 1 -- 172.21.15.184:0/194346604 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5de40cdd60 con 0x7f5de4070160 2024-09-17T04:23:51.550 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.551+0000 7f5de0c00640 1 -- 172.21.15.184:0/194346604 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f5de40cdd60 con 0x7f5de4070160 2024-09-17T04:23:51.622 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.623+0000 7f5dea000640 1 -- 172.21.15.184:0/194346604 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e5b4d175-f95f-4dc1-b00e-75bd8604cbd3", "id": 4} v 0) -- 0x7f5de40cd430 con 0x7f5de4070160 2024-09-17T04:23:51.628 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.628+0000 7f5de0c00640 1 -- 172.21.15.184:0/194346604 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e5b4d175-f95f-4dc1-b00e-75bd8604cbd3", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f5dd80208a0 con 0x7f5de4070160 2024-09-17T04:23:51.628 INFO:teuthology.orchestra.run.smithi184.stdout:4 2024-09-17T04:23:51.629 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.630+0000 7f5dea000640 1 -- 172.21.15.184:0/194346604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5de4070160 msgr2=0x7f5de40aebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:51.629 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.630+0000 7f5dea000640 1 --2- 172.21.15.184:0/194346604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5de4070160 0x7f5de40aebe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800eac0 tx=0x7f5dd800ef80 comp rx=0 tx=0).stop 2024-09-17T04:23:51.630 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.630+0000 7f5dea000640 1 -- 172.21.15.184:0/194346604 shutdown_connections 2024-09-17T04:23:51.630 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.630+0000 7f5dea000640 1 --2- 172.21.15.184:0/194346604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de415f290 0x7f5de414e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:51.630 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.630+0000 7f5dea000640 1 --2- 172.21.15.184:0/194346604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5de40b30d0 0x7f5de40af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:51.630 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.630+0000 7f5dea000640 1 --2- 172.21.15.184:0/194346604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5de4070160 0x7f5de40aebe0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:51.630 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.630+0000 7f5dea000640 1 -- 172.21.15.184:0/194346604 >> 172.21.15.184:0/194346604 conn(0x7f5de406d620 msgr2=0x7f5de4156940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:51.630 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.630+0000 7f5dea000640 1 -- 172.21.15.184:0/194346604 shutdown_connections 2024-09-17T04:23:51.630 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.630+0000 7f5dea000640 1 -- 172.21.15.184:0/194346604 wait complete. 2024-09-17T04:23:51.649 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new c38f68c8-5099-440f-981c-01e5710475ba 5 2024-09-17T04:23:51.872 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.871+0000 7f5699e00640 1 -- 172.21.15.184:0/2092206166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56940d4c70 msgr2=0x7f56940cf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:51.872 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.871+0000 7f5699e00640 1 --2- 172.21.15.184:0/2092206166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56940d4c70 0x7f56940cf830 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f568c005fd0 tx=0x7f568c02aaa0 comp rx=0 tx=0).stop 2024-09-17T04:23:51.872 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.871+0000 7f5699e00640 1 -- 172.21.15.184:0/2092206166 shutdown_connections 2024-09-17T04:23:51.872 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.871+0000 7f5699e00640 1 --2- 172.21.15.184:0/2092206166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56940d4c70 0x7f56940cf830 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:51.872 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.871+0000 7f5699e00640 1 --2- 172.21.15.184:0/2092206166 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56940d24f0 0x7f56940d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:51.873 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.871+0000 7f5699e00640 1 -- 172.21.15.184:0/2092206166 >> 172.21.15.184:0/2092206166 conn(0x7f569408dd50 msgr2=0x7f569409d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:51.873 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.871+0000 7f5699e00640 1 -- 172.21.15.184:0/2092206166 shutdown_connections 2024-09-17T04:23:51.873 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.871+0000 7f5699e00640 1 -- 172.21.15.184:0/2092206166 wait complete. 2024-09-17T04:23:51.874 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.874+0000 7f5699e00640 1 Processor -- start 2024-09-17T04:23:51.874 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.875+0000 7f5699e00640 1 -- start start 2024-09-17T04:23:51.877 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.877+0000 7f5699e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56940d24f0 0x7f56940a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:51.878 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.877+0000 7f5699e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56940d2f00 0x7f56940cde70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:51.878 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.877+0000 7f5699e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56941f5530 0x7f56940c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:51.878 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.877+0000 7f5699e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f56940cfef0 con 0x7f56940d2f00 2024-09-17T04:23:51.878 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.877+0000 7f5699e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f56940cfd70 con 0x7f56940d24f0 2024-09-17T04:23:51.878 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.877+0000 7f5699e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f56940d0070 con 0x7f56941f5530 2024-09-17T04:23:51.878 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.877+0000 7f5698a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56940d24f0 0x7f56940a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:51.878 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.878+0000 7f5699400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56941f5530 0x7f56940c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:51.878 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.878+0000 7f5698a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56940d24f0 0x7f56940a0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:43044/0 (socket says 172.21.15.184:43044) 2024-09-17T04:23:51.878 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.878+0000 7f5698a00640 1 -- 172.21.15.184:0/1390827522 learned_addr learned my addr 172.21.15.184:0/1390827522 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-09-17T04:23:51.878 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.878+0000 7f5693e00640 1 --2- 172.21.15.184:0/1390827522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56940d2f00 0x7f56940cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:51.878 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.878+0000 7f5698a00640 1 -- 172.21.15.184:0/1390827522 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56941f5530 msgr2=0x7f56940c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:51.878 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.878+0000 7f5698a00640 1 --2- 172.21.15.184:0/1390827522 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56941f5530 0x7f56940c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:51.879 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.878+0000 7f5698a00640 1 -- 172.21.15.184:0/1390827522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56940d2f00 msgr2=0x7f56940cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:51.879 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.878+0000 7f5698a00640 1 --2- 172.21.15.184:0/1390827522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56940d2f00 0x7f56940cde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:51.879 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.878+0000 7f5698a00640 1 -- 172.21.15.184:0/1390827522 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56940a1320 con 0x7f56940d24f0 2024-09-17T04:23:51.879 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.878+0000 7f5699400640 1 --2- 172.21.15.184:0/1390827522 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56941f5530 0x7f56940c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:23:51.879 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.878+0000 7f5698a00640 1 --2- 172.21.15.184:0/1390827522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56940d24f0 0x7f56940a0ad0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f568800e7c0 tx=0x7f568800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:23:51.880 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.879+0000 7f5690c00640 1 -- 172.21.15.184:0/1390827522 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568800ccc0 con 0x7f56940d24f0 2024-09-17T04:23:51.880 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.879+0000 7f5690c00640 1 -- 172.21.15.184:0/1390827522 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5688010430 con 0x7f56940d24f0 2024-09-17T04:23:51.880 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.879+0000 7f5690c00640 1 -- 172.21.15.184:0/1390827522 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5688010840 con 0x7f56940d24f0 2024-09-17T04:23:51.880 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.879+0000 7f5699e00640 1 -- 172.21.15.184:0/1390827522 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56940a1600 con 0x7f56940d24f0 2024-09-17T04:23:51.880 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.879+0000 7f5699e00640 1 -- 172.21.15.184:0/1390827522 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56940a1bc0 con 0x7f56940d24f0 2024-09-17T04:23:51.880 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.879+0000 7f5690c00640 1 -- 172.21.15.184:0/1390827522 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f56880109e0 con 0x7f56940d24f0 2024-09-17T04:23:51.880 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.879+0000 7f5690c00640 1 -- 172.21.15.184:0/1390827522 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f5688010cd0 con 0x7f56940d24f0 2024-09-17T04:23:51.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.880+0000 7f5699e00640 1 -- 172.21.15.184:0/1390827522 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56940d5740 con 0x7f56940d24f0 2024-09-17T04:23:51.882 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.883+0000 7f5690c00640 1 -- 172.21.15.184:0/1390827522 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f56940d5740 con 0x7f56940d24f0 2024-09-17T04:23:51.957 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.957+0000 7f5690c00640 1 -- 172.21.15.184:0/1390827522 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mgrmap(e 2) ==== 50742+0+0 (secure 0 0 0) 0x7f5688018c40 con 0x7f56940d24f0 2024-09-17T04:23:51.963 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.963+0000 7f5699e00640 1 -- 172.21.15.184:0/1390827522 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c38f68c8-5099-440f-981c-01e5710475ba", "id": 5} v 0) -- 0x7f56940d0520 con 0x7f56940d24f0 2024-09-17T04:23:51.968 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.968+0000 7f5690c00640 1 -- 172.21.15.184:0/1390827522 <== mon.1 v2:172.21.15.184:3300/0 8 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c38f68c8-5099-440f-981c-01e5710475ba", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f56880071c0 con 0x7f56940d24f0 2024-09-17T04:23:51.968 INFO:teuthology.orchestra.run.smithi184.stdout:5 2024-09-17T04:23:51.969 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.970+0000 7f5699e00640 1 -- 172.21.15.184:0/1390827522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56940d24f0 msgr2=0x7f56940a0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:51.970 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.970+0000 7f5699e00640 1 --2- 172.21.15.184:0/1390827522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56940d24f0 0x7f56940a0ad0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f568800e7c0 tx=0x7f568800ec80 comp rx=0 tx=0).stop 2024-09-17T04:23:51.970 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.970+0000 7f5699e00640 1 -- 172.21.15.184:0/1390827522 shutdown_connections 2024-09-17T04:23:51.970 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.970+0000 7f5699e00640 1 --2- 172.21.15.184:0/1390827522 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56941f5530 0x7f56940c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:51.970 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.970+0000 7f5699e00640 1 --2- 172.21.15.184:0/1390827522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56940d2f00 0x7f56940cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:51.970 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.970+0000 7f5699e00640 1 --2- 172.21.15.184:0/1390827522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56940d24f0 0x7f56940a0ad0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:51.970 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.970+0000 7f5699e00640 1 -- 172.21.15.184:0/1390827522 >> 172.21.15.184:0/1390827522 conn(0x7f569408dd50 msgr2=0x7f56940b8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:51.970 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.970+0000 7f5699e00640 1 -- 172.21.15.184:0/1390827522 shutdown_connections 2024-09-17T04:23:51.970 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:51.970+0000 7f5699e00640 1 -- 172.21.15.184:0/1390827522 wait complete. 2024-09-17T04:23:51.992 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new d7791716-3d17-43ea-a427-31dffe31ecd8 6 2024-09-17T04:23:52.207 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.207+0000 7f949a400640 1 -- 172.21.15.184:0/2198558062 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9494073a80 msgr2=0x7f94940cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:52.207 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.207+0000 7f949a400640 1 --2- 172.21.15.184:0/2198558062 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9494073a80 0x7f94940cd610 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f948c005fd0 tx=0x7f948c02ab20 comp rx=0 tx=0).stop 2024-09-17T04:23:52.207 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.207+0000 7f949a400640 1 -- 172.21.15.184:0/2198558062 shutdown_connections 2024-09-17T04:23:52.207 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.207+0000 7f949a400640 1 --2- 172.21.15.184:0/2198558062 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9494073a80 0x7f94940cd610 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:52.208 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.207+0000 7f949a400640 1 --2- 172.21.15.184:0/2198558062 >> v2:172.21.15.100:3300/0 conn(0x7f9494145a40 0x7f9494073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:52.208 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.207+0000 7f949a400640 1 -- 172.21.15.184:0/2198558062 >> 172.21.15.184:0/2198558062 conn(0x7f949406d1c0 msgr2=0x7f949406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:52.208 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.207+0000 7f949a400640 1 -- 172.21.15.184:0/2198558062 shutdown_connections 2024-09-17T04:23:52.208 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.207+0000 7f949a400640 1 -- 172.21.15.184:0/2198558062 wait complete. 2024-09-17T04:23:52.209 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.210+0000 7f949a400640 1 Processor -- start 2024-09-17T04:23:52.209 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.210+0000 7f949a400640 1 -- start start 2024-09-17T04:23:52.211 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.211+0000 7f949a400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9494073a80 0x7f94941563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:52.212 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.211+0000 7f949a400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9494145550 0x7f949413e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:52.212 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.211+0000 7f949a400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9494145a40 0x7f949413e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:52.212 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.211+0000 7f949a400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f94940ce190 con 0x7f9494145550 2024-09-17T04:23:52.212 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.211+0000 7f949a400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f94940ce010 con 0x7f9494073a80 2024-09-17T04:23:52.212 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.211+0000 7f949a400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f94940ce310 con 0x7f9494145a40 2024-09-17T04:23:52.212 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.211+0000 7f9493400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9494073a80 0x7f94941563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:52.212 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.212+0000 7f9493e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9494145a40 0x7f949413e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:52.212 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.212+0000 7f948aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9494145550 0x7f949413e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:52.212 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.212+0000 7f9493e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9494145a40 0x7f949413e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:48846/0 (socket says 172.21.15.184:48846) 2024-09-17T04:23:52.212 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.212+0000 7f9493e00640 1 -- 172.21.15.184:0/1004846274 learned_addr learned my addr 172.21.15.184:0/1004846274 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-09-17T04:23:52.212 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.212+0000 7f9493e00640 1 -- 172.21.15.184:0/1004846274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9494073a80 msgr2=0x7f94941563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:52.212 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.212+0000 7f9493e00640 1 --2- 172.21.15.184:0/1004846274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9494073a80 0x7f94941563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:52.213 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.212+0000 7f9493e00640 1 -- 172.21.15.184:0/1004846274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9494145550 msgr2=0x7f949413e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:52.213 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.212+0000 7f9493e00640 1 --2- 172.21.15.184:0/1004846274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9494145550 0x7f949413e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:52.213 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.212+0000 7f9493e00640 1 -- 172.21.15.184:0/1004846274 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94940ae4d0 con 0x7f9494145a40 2024-09-17T04:23:52.213 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.212+0000 7f948aa00640 1 --2- 172.21.15.184:0/1004846274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9494145550 0x7f949413e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:23:52.213 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.212+0000 7f9493e00640 1 --2- 172.21.15.184:0/1004846274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9494145a40 0x7f949413e8f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f948c006010 tx=0x7f948c003ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:23:52.214 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.212+0000 7f948be00640 1 -- 172.21.15.184:0/1004846274 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c044070 con 0x7f9494145a40 2024-09-17T04:23:52.214 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.213+0000 7f948be00640 1 -- 172.21.15.184:0/1004846274 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f948c004040 con 0x7f9494145a40 2024-09-17T04:23:52.214 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.213+0000 7f949a400640 1 -- 172.21.15.184:0/1004846274 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94940ae7b0 con 0x7f9494145a40 2024-09-17T04:23:52.214 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.213+0000 7f948be00640 1 -- 172.21.15.184:0/1004846274 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c004430 con 0x7f9494145a40 2024-09-17T04:23:52.214 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.213+0000 7f949a400640 1 -- 172.21.15.184:0/1004846274 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f94940aed70 con 0x7f9494145a40 2024-09-17T04:23:52.214 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.214+0000 7f949a400640 1 -- 172.21.15.184:0/1004846274 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94940ce7c0 con 0x7f9494145a40 2024-09-17T04:23:52.214 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.214+0000 7f948be00640 1 -- 172.21.15.184:0/1004846274 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 2) ==== 50742+0+0 (secure 0 0 0) 0x7f948c0045d0 con 0x7f9494145a40 2024-09-17T04:23:52.215 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.214+0000 7f948be00640 1 -- 172.21.15.184:0/1004846274 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f948c072890 con 0x7f9494145a40 2024-09-17T04:23:52.216 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.216+0000 7f948be00640 1 -- 172.21.15.184:0/1004846274 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f948c032030 con 0x7f9494145a40 2024-09-17T04:23:52.287 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.288+0000 7f949a400640 1 -- 172.21.15.184:0/1004846274 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d7791716-3d17-43ea-a427-31dffe31ecd8", "id": 6} v 0) -- 0x7f94940b0470 con 0x7f9494145a40 2024-09-17T04:23:52.297 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.298+0000 7f948be00640 1 -- 172.21.15.184:0/1004846274 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d7791716-3d17-43ea-a427-31dffe31ecd8", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f948c074cd0 con 0x7f9494145a40 2024-09-17T04:23:52.297 INFO:teuthology.orchestra.run.smithi184.stdout:6 2024-09-17T04:23:52.299 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.299+0000 7f949a400640 1 -- 172.21.15.184:0/1004846274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9494145a40 msgr2=0x7f949413e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:52.299 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.299+0000 7f949a400640 1 --2- 172.21.15.184:0/1004846274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9494145a40 0x7f949413e8f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f948c006010 tx=0x7f948c003ce0 comp rx=0 tx=0).stop 2024-09-17T04:23:52.299 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.299+0000 7f949a400640 1 -- 172.21.15.184:0/1004846274 shutdown_connections 2024-09-17T04:23:52.299 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.299+0000 7f949a400640 1 --2- 172.21.15.184:0/1004846274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9494145a40 0x7f949413e8f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:52.299 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.299+0000 7f949a400640 1 --2- 172.21.15.184:0/1004846274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9494145550 0x7f949413e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:52.299 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.299+0000 7f949a400640 1 --2- 172.21.15.184:0/1004846274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9494073a80 0x7f94941563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:52.299 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.299+0000 7f949a400640 1 -- 172.21.15.184:0/1004846274 >> 172.21.15.184:0/1004846274 conn(0x7f949406d1c0 msgr2=0x7f949413f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:52.299 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.299+0000 7f949a400640 1 -- 172.21.15.184:0/1004846274 shutdown_connections 2024-09-17T04:23:52.300 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.299+0000 7f949a400640 1 -- 172.21.15.184:0/1004846274 wait complete. 2024-09-17T04:23:52.318 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new fb888ad0-76e4-4352-8067-37ec89059f36 7 2024-09-17T04:23:52.530 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.530+0000 7f47aa200640 1 -- 172.21.15.184:0/803571033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47a40ad3d0 msgr2=0x7f47a40ad7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:52.530 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.530+0000 7f47aa200640 1 --2- 172.21.15.184:0/803571033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47a40ad3d0 0x7f47a40ad7e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f479c005fd0 tx=0x7f479c02aaa0 comp rx=0 tx=0).stop 2024-09-17T04:23:52.530 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.530+0000 7f47aa200640 1 -- 172.21.15.184:0/803571033 shutdown_connections 2024-09-17T04:23:52.530 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.530+0000 7f47aa200640 1 --2- 172.21.15.184:0/803571033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47a40ad3d0 0x7f47a40ad7e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:52.530 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.530+0000 7f47aa200640 1 --2- 172.21.15.184:0/803571033 >> v2:172.21.15.184:3300/0 conn(0x7f47a40b4f00 0x7f47a40704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:52.530 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.530+0000 7f47aa200640 1 -- 172.21.15.184:0/803571033 >> 172.21.15.184:0/803571033 conn(0x7f47a406d710 msgr2=0x7f47a40b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:52.530 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.530+0000 7f47aa200640 1 -- 172.21.15.184:0/803571033 shutdown_connections 2024-09-17T04:23:52.530 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.530+0000 7f47aa200640 1 -- 172.21.15.184:0/803571033 wait complete. 2024-09-17T04:23:52.531 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.532+0000 7f47aa200640 1 Processor -- start 2024-09-17T04:23:52.531 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.532+0000 7f47aa200640 1 -- start start 2024-09-17T04:23:52.535 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.533+0000 7f47aa200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47a40ad3d0 0x7f47a4144ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:52.535 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.533+0000 7f47aa200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47a40b4f00 0x7f47a414c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:52.535 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.534+0000 7f47aa200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47a4160490 0x7f47a414cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:52.535 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.534+0000 7f47aa200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f47a406f0f0 con 0x7f47a40b4f00 2024-09-17T04:23:52.535 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.534+0000 7f47aa200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f47a406ef70 con 0x7f47a4160490 2024-09-17T04:23:52.535 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.534+0000 7f47aa200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f47a406f270 con 0x7f47a40ad3d0 2024-09-17T04:23:52.536 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.534+0000 7f47a3400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47a40ad3d0 0x7f47a4144ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:52.536 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.534+0000 7f47a3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47a4160490 0x7f47a414cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:52.536 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.534+0000 7f47a3400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47a40ad3d0 0x7f47a4144ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:48872/0 (socket says 172.21.15.184:48872) 2024-09-17T04:23:52.536 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.534+0000 7f47a3400640 1 -- 172.21.15.184:0/3261027088 learned_addr learned my addr 172.21.15.184:0/3261027088 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-09-17T04:23:52.536 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.534+0000 7f47a2a00640 1 --2- 172.21.15.184:0/3261027088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47a40b4f00 0x7f47a414c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:52.536 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.534+0000 7f47a3400640 1 -- 172.21.15.184:0/3261027088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47a4160490 msgr2=0x7f47a414cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:52.537 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.534+0000 7f47a3400640 1 --2- 172.21.15.184:0/3261027088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47a4160490 0x7f47a414cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:52.537 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.534+0000 7f47a3400640 1 -- 172.21.15.184:0/3261027088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47a40b4f00 msgr2=0x7f47a414c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:52.537 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.534+0000 7f47a3400640 1 --2- 172.21.15.184:0/3261027088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47a40b4f00 0x7f47a414c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:52.537 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.534+0000 7f47a3400640 1 -- 172.21.15.184:0/3261027088 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a4154b50 con 0x7f47a40ad3d0 2024-09-17T04:23:52.537 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.534+0000 7f47a3400640 1 --2- 172.21.15.184:0/3261027088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47a40ad3d0 0x7f47a4144ed0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f479400e9f0 tx=0x7f479400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:23:52.537 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.535+0000 7f479b400640 1 -- 172.21.15.184:0/3261027088 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479400cd80 con 0x7f47a40ad3d0 2024-09-17T04:23:52.538 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.535+0000 7f479b400640 1 -- 172.21.15.184:0/3261027088 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4794004020 con 0x7f47a40ad3d0 2024-09-17T04:23:52.538 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.535+0000 7f479b400640 1 -- 172.21.15.184:0/3261027088 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4794004470 con 0x7f47a40ad3d0 2024-09-17T04:23:52.538 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.535+0000 7f47aa200640 1 -- 172.21.15.184:0/3261027088 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a4154e30 con 0x7f47a40ad3d0 2024-09-17T04:23:52.538 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.535+0000 7f47aa200640 1 -- 172.21.15.184:0/3261027088 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a41553f0 con 0x7f47a40ad3d0 2024-09-17T04:23:52.539 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.536+0000 7f47aa200640 1 -- 172.21.15.184:0/3261027088 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a4157240 con 0x7f47a40ad3d0 2024-09-17T04:23:52.539 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.536+0000 7f479b400640 1 -- 172.21.15.184:0/3261027088 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 2) ==== 50742+0+0 (secure 0 0 0) 0x7f47940035f0 con 0x7f47a40ad3d0 2024-09-17T04:23:52.539 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.537+0000 7f479b400640 1 -- 172.21.15.184:0/3261027088 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f4794052e80 con 0x7f47a40ad3d0 2024-09-17T04:23:52.539 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.538+0000 7f479b400640 1 -- 172.21.15.184:0/3261027088 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f47a4157240 con 0x7f47a40ad3d0 2024-09-17T04:23:52.608 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.609+0000 7f47aa200640 1 -- 172.21.15.184:0/3261027088 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd new", "uuid": "fb888ad0-76e4-4352-8067-37ec89059f36", "id": 7} v 0) -- 0x7f47a4069840 con 0x7f47a40ad3d0 2024-09-17T04:23:52.618 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.618+0000 7f479b400640 1 -- 172.21.15.184:0/3261027088 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fb888ad0-76e4-4352-8067-37ec89059f36", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f47a4069840 con 0x7f47a40ad3d0 2024-09-17T04:23:52.618 INFO:teuthology.orchestra.run.smithi184.stdout:7 2024-09-17T04:23:52.620 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.620+0000 7f47aa200640 1 -- 172.21.15.184:0/3261027088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47a40ad3d0 msgr2=0x7f47a4144ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:52.620 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.620+0000 7f47aa200640 1 --2- 172.21.15.184:0/3261027088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47a40ad3d0 0x7f47a4144ed0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f479400e9f0 tx=0x7f479400eeb0 comp rx=0 tx=0).stop 2024-09-17T04:23:52.620 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.620+0000 7f47aa200640 1 -- 172.21.15.184:0/3261027088 shutdown_connections 2024-09-17T04:23:52.620 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.620+0000 7f47aa200640 1 --2- 172.21.15.184:0/3261027088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47a4160490 0x7f47a414cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:52.620 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.620+0000 7f47aa200640 1 --2- 172.21.15.184:0/3261027088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47a40b4f00 0x7f47a414c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:52.620 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.620+0000 7f47aa200640 1 --2- 172.21.15.184:0/3261027088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47a40ad3d0 0x7f47a4144ed0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:52.620 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.620+0000 7f47aa200640 1 -- 172.21.15.184:0/3261027088 >> 172.21.15.184:0/3261027088 conn(0x7f47a406d710 msgr2=0x7f47a415c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:52.620 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.620+0000 7f47aa200640 1 -- 172.21.15.184:0/3261027088 shutdown_connections 2024-09-17T04:23:52.620 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:23:52.620+0000 7f47aa200640 1 -- 172.21.15.184:0/3261027088 wait complete. 2024-09-17T04:23:52.642 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-17T04:23:52.642 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-17T04:23:52.646 INFO:tasks.ceph.osd.0:Started 2024-09-17T04:23:52.646 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-17T04:23:52.646 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-17T04:23:52.650 INFO:tasks.ceph.osd.1:Started 2024-09-17T04:23:52.650 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-17T04:23:52.650 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-17T04:23:52.655 INFO:tasks.ceph.osd.2:Started 2024-09-17T04:23:52.655 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-17T04:23:52.655 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-17T04:23:52.661 INFO:tasks.ceph.osd.3:Started 2024-09-17T04:23:52.661 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-17T04:23:52.661 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-17T04:23:52.665 INFO:tasks.ceph.osd.4:Started 2024-09-17T04:23:52.665 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-17T04:23:52.665 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-17T04:23:52.669 INFO:tasks.ceph.osd.5:Started 2024-09-17T04:23:52.669 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-17T04:23:52.669 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-17T04:23:52.673 INFO:tasks.ceph.osd.6:Started 2024-09-17T04:23:52.673 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-17T04:23:52.673 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-17T04:23:52.677 INFO:tasks.ceph.osd.7:Started 2024-09-17T04:23:52.677 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:23:52.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.935+0000 7f69e3000640 1 -- 172.21.15.100:0/3127598294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69dc15d760 msgr2=0x7f69dc1535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:52.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.935+0000 7f69e3000640 1 --2- 172.21.15.100:0/3127598294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69dc15d760 0x7f69dc1535c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f69cc005fd0 tx=0x7f69cc02a540 comp rx=0 tx=0).stop 2024-09-17T04:23:52.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.935+0000 7f69e3000640 1 -- 172.21.15.100:0/3127598294 shutdown_connections 2024-09-17T04:23:52.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.935+0000 7f69e3000640 1 --2- 172.21.15.100:0/3127598294 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69dc0b3550 0x7f69dc0b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:52.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.935+0000 7f69e3000640 1 --2- 172.21.15.100:0/3127598294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69dc15d760 0x7f69dc1535c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:52.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.935+0000 7f69e3000640 1 -- 172.21.15.100:0/3127598294 >> 172.21.15.100:0/3127598294 conn(0x7f69dc06d730 msgr2=0x7f69dc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:52.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.935+0000 7f69e3000640 1 -- 172.21.15.100:0/3127598294 shutdown_connections 2024-09-17T04:23:52.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.935+0000 7f69e3000640 1 -- 172.21.15.100:0/3127598294 wait complete. 2024-09-17T04:23:52.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.938+0000 7f69e3000640 1 Processor -- start 2024-09-17T04:23:52.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.938+0000 7f69e3000640 1 -- start start 2024-09-17T04:23:52.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.939+0000 7f69e3000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69dc0ad0e0 0x7f69dc13f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:52.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.939+0000 7f69e3000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69dc0b3550 0x7f69dc14c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:52.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.939+0000 7f69e3000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69dc15d760 0x7f69dc068e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:52.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.939+0000 7f69e3000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc06f240 con 0x7f69dc15d760 2024-09-17T04:23:52.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.939+0000 7f69e3000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc0b4e30 con 0x7f69dc0ad0e0 2024-09-17T04:23:52.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.939+0000 7f69e3000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f69dc0b3ea0 con 0x7f69dc0b3550 2024-09-17T04:23:52.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.940+0000 7f69e1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69dc15d760 0x7f69dc068e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:52.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.940+0000 7f69e1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69dc15d760 0x7f69dc068e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37150/0 (socket says 172.21.15.100:37150) 2024-09-17T04:23:52.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.940+0000 7f69e1000640 1 -- 172.21.15.100:0/693310197 learned_addr learned my addr 172.21.15.100:0/693310197 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:23:52.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.940+0000 7f69dbe00640 1 --2- 172.21.15.100:0/693310197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69dc0ad0e0 0x7f69dc13f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:52.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.940+0000 7f69db400640 1 --2- 172.21.15.100:0/693310197 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69dc0b3550 0x7f69dc14c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:52.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.940+0000 7f69e1000640 1 -- 172.21.15.100:0/693310197 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69dc0b3550 msgr2=0x7f69dc14c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:52.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.940+0000 7f69e1000640 1 --2- 172.21.15.100:0/693310197 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69dc0b3550 0x7f69dc14c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:52.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.940+0000 7f69e1000640 1 -- 172.21.15.100:0/693310197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69dc0ad0e0 msgr2=0x7f69dc13f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:52.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.940+0000 7f69e1000640 1 --2- 172.21.15.100:0/693310197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69dc0ad0e0 0x7f69dc13f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:52.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.940+0000 7f69e1000640 1 -- 172.21.15.100:0/693310197 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69dc1f3320 con 0x7f69dc15d760 2024-09-17T04:23:52.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.940+0000 7f69db400640 1 --2- 172.21.15.100:0/693310197 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69dc0b3550 0x7f69dc14c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:23:52.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.940+0000 7f69e1000640 1 --2- 172.21.15.100:0/693310197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69dc15d760 0x7f69dc068e80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f69d400ec50 tx=0x7f69d400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:23:52.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.941+0000 7f69d3e00640 1 -- 172.21.15.100:0/693310197 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d4003b70 con 0x7f69dc15d760 2024-09-17T04:23:52.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.941+0000 7f69d3e00640 1 -- 172.21.15.100:0/693310197 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69d4003d10 con 0x7f69dc15d760 2024-09-17T04:23:52.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.941+0000 7f69d3e00640 1 -- 172.21.15.100:0/693310197 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d4004280 con 0x7f69dc15d760 2024-09-17T04:23:52.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.941+0000 7f69e3000640 1 -- 172.21.15.100:0/693310197 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69dc1f35d0 con 0x7f69dc15d760 2024-09-17T04:23:52.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.941+0000 7f69e3000640 1 -- 172.21.15.100:0/693310197 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69dc1f3b90 con 0x7f69dc15d760 2024-09-17T04:23:52.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.942+0000 7f69e3000640 1 -- 172.21.15.100:0/693310197 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69dc0b42f0 con 0x7f69dc15d760 2024-09-17T04:23:52.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.942+0000 7f69d3e00640 1 -- 172.21.15.100:0/693310197 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 2) ==== 50742+0+0 (secure 0 0 0) 0x7f69d4004420 con 0x7f69dc15d760 2024-09-17T04:23:52.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.943+0000 7f69d3e00640 1 -- 172.21.15.100:0/693310197 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f69d4053160 con 0x7f69dc15d760 2024-09-17T04:23:52.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.944+0000 7f69d3e00640 1 -- 172.21.15.100:0/693310197 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f69d4018dd0 con 0x7f69dc15d760 2024-09-17T04:23:52.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.976+0000 7f69d3e00640 1 -- 172.21.15.100:0/693310197 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 3) ==== 101279+0+0 (secure 0 0 0) 0x7f69d4018990 con 0x7f69dc15d760 2024-09-17T04:23:52.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.977+0000 7f69d3e00640 1 --2- 172.21.15.100:0/693310197 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f69bc078ea0 0x7f69bc07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:52.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.977+0000 7f69dbe00640 1 --2- 172.21.15.100:0/693310197 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f69bc078ea0 0x7f69bc07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:52.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:52.978+0000 7f69dbe00640 1 --2- 172.21.15.100:0/693310197 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f69bc078ea0 0x7f69bc07b2c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f69cc033000 tx=0x7f69cc004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:23:52.986 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:23:52.987+0000 7fa190600640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-17T04:23:52.986 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:23:52.987+0000 7fa190600640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-17T04:23:53.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:53.014+0000 7f69e3000640 1 -- 172.21.15.100:0/693310197 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69dc06e060 con 0x7f69dc15d760 2024-09-17T04:23:53.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:53.015+0000 7f69d3e00640 1 -- 172.21.15.100:0/693310197 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f69d402c560 con 0x7f69dc15d760 2024-09-17T04:23:53.015 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:23:53.015 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":10,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:23:52.610465+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:23:53.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:53.016+0000 7f69e3000640 1 -- 172.21.15.100:0/693310197 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f69bc078ea0 msgr2=0x7f69bc07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:53.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:53.016+0000 7f69e3000640 1 --2- 172.21.15.100:0/693310197 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f69bc078ea0 0x7f69bc07b2c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f69cc033000 tx=0x7f69cc004730 comp rx=0 tx=0).stop 2024-09-17T04:23:53.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:53.016+0000 7f69e3000640 1 -- 172.21.15.100:0/693310197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69dc15d760 msgr2=0x7f69dc068e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:53.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:53.016+0000 7f69e3000640 1 --2- 172.21.15.100:0/693310197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69dc15d760 0x7f69dc068e80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f69d400ec50 tx=0x7f69d400c6a0 comp rx=0 tx=0).stop 2024-09-17T04:23:53.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:53.016+0000 7f69e3000640 1 -- 172.21.15.100:0/693310197 shutdown_connections 2024-09-17T04:23:53.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:53.016+0000 7f69e3000640 1 --2- 172.21.15.100:0/693310197 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f69bc078ea0 0x7f69bc07b2c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:53.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:53.016+0000 7f69e3000640 1 --2- 172.21.15.100:0/693310197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69dc15d760 0x7f69dc068e80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:53.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:53.016+0000 7f69e3000640 1 --2- 172.21.15.100:0/693310197 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69dc0b3550 0x7f69dc14c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:53.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:53.016+0000 7f69e3000640 1 --2- 172.21.15.100:0/693310197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69dc0ad0e0 0x7f69dc13f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:53.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:53.016+0000 7f69e3000640 1 -- 172.21.15.100:0/693310197 >> 172.21.15.100:0/693310197 conn(0x7f69dc06d730 msgr2=0x7f69dc15e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:53.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:53.016+0000 7f69e3000640 1 -- 172.21.15.100:0/693310197 shutdown_connections 2024-09-17T04:23:53.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:23:53.016+0000 7f69e3000640 1 -- 172.21.15.100:0/693310197 wait complete. 2024-09-17T04:23:53.036 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-17T04:23:53.037 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-17T04:23:53.843 DEBUG:teuthology.orchestra.run.smithi100:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T04:23:54.049 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.048+0000 7ff1fee00640 1 -- 172.21.15.100:0/2876897351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1dc004800 msgr2=0x7ff1dc004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:54.049 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.048+0000 7ff1fee00640 1 --2- 172.21.15.100:0/2876897351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1dc004800 0x7ff1dc004c10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff1f80c73e0 tx=0x7ff1dc005830 comp rx=0 tx=0).stop 2024-09-17T04:23:54.049 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.048+0000 7ff1fee00640 1 -- 172.21.15.100:0/2876897351 shutdown_connections 2024-09-17T04:23:54.049 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.048+0000 7ff1fee00640 1 --2- 172.21.15.100:0/2876897351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1dc004800 0x7ff1dc004c10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:54.049 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.048+0000 7ff1fee00640 1 -- 172.21.15.100:0/2876897351 >> 172.21.15.100:0/2876897351 conn(0x7ff1f80d0b50 msgr2=0x7ff1f80b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:54.049 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.048+0000 7ff1fee00640 1 -- 172.21.15.100:0/2876897351 shutdown_connections 2024-09-17T04:23:54.049 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.048+0000 7ff1fee00640 1 -- 172.21.15.100:0/2876897351 wait complete. 2024-09-17T04:23:54.050 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.050+0000 7ff1fee00640 1 Processor -- start 2024-09-17T04:23:54.051 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.050+0000 7ff1fee00640 1 -- start start 2024-09-17T04:23:54.054 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.052+0000 7ff1fee00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1dc004800 0x7ff1f809d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:54.054 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.052+0000 7ff1fee00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1f8074c40 0x7ff1f8073f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:54.054 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.052+0000 7ff1fee00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1f8075050 0x7ff1f8074460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:54.054 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.052+0000 7ff1fee00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8156c60 con 0x7ff1dc004800 2024-09-17T04:23:54.054 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.052+0000 7ff1fee00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8156ae0 con 0x7ff1f8074c40 2024-09-17T04:23:54.054 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.052+0000 7ff1fee00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f8156de0 con 0x7ff1f8075050 2024-09-17T04:23:54.054 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.052+0000 7ff1f7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1dc004800 0x7ff1f809d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:54.054 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.052+0000 7ff1f7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1dc004800 0x7ff1f809d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37176/0 (socket says 172.21.15.100:37176) 2024-09-17T04:23:54.054 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.052+0000 7ff1f7e00640 1 -- 172.21.15.100:0/528105082 learned_addr learned my addr 172.21.15.100:0/528105082 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:23:54.054 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.052+0000 7ff1f7400640 1 --2- 172.21.15.100:0/528105082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1f8074c40 0x7ff1f8073f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:54.054 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.052+0000 7ff1fce00640 1 --2- 172.21.15.100:0/528105082 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1f8075050 0x7ff1f8074460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:54.054 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.052+0000 7ff1f7e00640 1 -- 172.21.15.100:0/528105082 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1f8075050 msgr2=0x7ff1f8074460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:54.055 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.052+0000 7ff1f7e00640 1 --2- 172.21.15.100:0/528105082 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1f8075050 0x7ff1f8074460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:54.055 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.052+0000 7ff1f7e00640 1 -- 172.21.15.100:0/528105082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1f8074c40 msgr2=0x7ff1f8073f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:54.055 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.052+0000 7ff1f7e00640 1 --2- 172.21.15.100:0/528105082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1f8074c40 0x7ff1f8073f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:54.055 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.052+0000 7ff1f7e00640 1 -- 172.21.15.100:0/528105082 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f809dcd0 con 0x7ff1dc004800 2024-09-17T04:23:54.055 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.053+0000 7ff1f7e00640 1 --2- 172.21.15.100:0/528105082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1dc004800 0x7ff1f809d790 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8011930 tx=0x7ff1e8011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:23:54.055 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.053+0000 7ff1efe00640 1 -- 172.21.15.100:0/528105082 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e800fe10 con 0x7ff1dc004800 2024-09-17T04:23:54.055 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.053+0000 7ff1efe00640 1 -- 172.21.15.100:0/528105082 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1e8004020 con 0x7ff1dc004800 2024-09-17T04:23:54.055 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.053+0000 7ff1fee00640 1 -- 172.21.15.100:0/528105082 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f81f7560 con 0x7ff1dc004800 2024-09-17T04:23:54.055 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.053+0000 7ff1efe00640 1 -- 172.21.15.100:0/528105082 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8005020 con 0x7ff1dc004800 2024-09-17T04:23:54.055 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.053+0000 7ff1fee00640 1 -- 172.21.15.100:0/528105082 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f81f7b20 con 0x7ff1dc004800 2024-09-17T04:23:54.056 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.054+0000 7ff1fee00640 1 -- 172.21.15.100:0/528105082 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f8157340 con 0x7ff1dc004800 2024-09-17T04:23:54.056 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.055+0000 7ff1efe00640 1 -- 172.21.15.100:0/528105082 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 4) ==== 101385+0+0 (secure 0 0 0) 0x7ff1e8021020 con 0x7ff1dc004800 2024-09-17T04:23:54.056 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.056+0000 7ff1efe00640 1 --2- 172.21.15.100:0/528105082 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff1d8078b00 0x7ff1d807b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:23:54.056 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.056+0000 7ff1efe00640 1 -- 172.21.15.100:0/528105082 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7ff1e809a810 con 0x7ff1dc004800 2024-09-17T04:23:54.056 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.056+0000 7ff1f7400640 1 --2- 172.21.15.100:0/528105082 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff1d8078b00 0x7ff1d807b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:23:54.057 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.057+0000 7ff1f7400640 1 --2- 172.21.15.100:0/528105082 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff1d8078b00 0x7ff1d807b010 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4033000 tx=0x7ff1f815d690 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:23:54.061 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.061+0000 7ff1efe00640 1 -- 172.21.15.100:0/528105082 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff1e8063c70 con 0x7ff1dc004800 2024-09-17T04:23:54.076 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-17T04:23:54.076+0000 7fa10d816640 -1 Falling back to public interface 2024-09-17T04:23:54.114 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-17T04:23:54.114+0000 7f4d9a81b640 -1 Falling back to public interface 2024-09-17T04:23:54.247 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.246+0000 7ff1fee00640 1 -- 172.21.15.100:0/528105082 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1f806d7a0 con 0x7ff1dc004800 2024-09-17T04:23:54.247 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.247+0000 7ff1efe00640 1 -- 172.21.15.100:0/528105082 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7ff1e8063570 con 0x7ff1dc004800 2024-09-17T04:23:54.248 INFO:teuthology.misc.health.smithi100.stdout: 2024-09-17T04:23:54.248 INFO:teuthology.misc.health.smithi100.stdout:{"epoch":10,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:23:52.610465+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:23:54.250 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.249+0000 7ff1fee00640 1 -- 172.21.15.100:0/528105082 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff1d8078b00 msgr2=0x7ff1d807b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:54.250 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.249+0000 7ff1fee00640 1 --2- 172.21.15.100:0/528105082 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff1d8078b00 0x7ff1d807b010 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4033000 tx=0x7ff1f815d690 comp rx=0 tx=0).stop 2024-09-17T04:23:54.250 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.249+0000 7ff1fee00640 1 -- 172.21.15.100:0/528105082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1dc004800 msgr2=0x7ff1f809d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:23:54.250 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.249+0000 7ff1fee00640 1 --2- 172.21.15.100:0/528105082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1dc004800 0x7ff1f809d790 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8011930 tx=0x7ff1e8011df0 comp rx=0 tx=0).stop 2024-09-17T04:23:54.250 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.249+0000 7ff1fee00640 1 -- 172.21.15.100:0/528105082 shutdown_connections 2024-09-17T04:23:54.250 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.249+0000 7ff1fee00640 1 --2- 172.21.15.100:0/528105082 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff1d8078b00 0x7ff1d807b010 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:54.250 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.249+0000 7ff1fee00640 1 --2- 172.21.15.100:0/528105082 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1f8075050 0x7ff1f8074460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:54.250 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.249+0000 7ff1fee00640 1 --2- 172.21.15.100:0/528105082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1f8074c40 0x7ff1f8073f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:54.250 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.249+0000 7ff1fee00640 1 --2- 172.21.15.100:0/528105082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1dc004800 0x7ff1f809d790 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:23:54.251 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.249+0000 7ff1fee00640 1 -- 172.21.15.100:0/528105082 >> 172.21.15.100:0/528105082 conn(0x7ff1f80d0b50 msgr2=0x7ff1f815dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:23:54.251 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.249+0000 7ff1fee00640 1 -- 172.21.15.100:0/528105082 shutdown_connections 2024-09-17T04:23:54.251 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:23:54.250+0000 7ff1fee00640 1 -- 172.21.15.100:0/528105082 wait complete. 2024-09-17T04:23:54.268 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-17T04:23:54.303 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-17T04:23:54.303+0000 7f7744e2f640 -1 Falling back to public interface 2024-09-17T04:23:54.309 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-17T04:23:54.309+0000 7f1740dae640 -1 Falling back to public interface 2024-09-17T04:23:54.315 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-17T04:23:54.315+0000 7fcb2ed2c640 -1 Falling back to public interface 2024-09-17T04:23:54.515 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-17T04:23:54.515+0000 7f16b5fc5640 -1 Falling back to public interface 2024-09-17T04:23:54.522 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-17T04:23:54.523+0000 7f9c92558640 -1 Falling back to public interface 2024-09-17T04:23:54.524 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-17T04:23:54.524+0000 7f118641e640 -1 Falling back to public interface 2024-09-17T04:23:58.061 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-17T04:23:58.061+0000 7f4d9a81b640 -1 osd.3 0 log_to_monitors true 2024-09-17T04:23:58.251 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-17T04:23:58.252+0000 7fa10d816640 -1 osd.5 0 log_to_monitors true 2024-09-17T04:23:58.283 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-17T04:23:58.283+0000 7f7744e2f640 -1 osd.1 0 log_to_monitors true 2024-09-17T04:23:58.285 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-17T04:23:58.285+0000 7fcb2ed2c640 -1 osd.0 0 log_to_monitors true 2024-09-17T04:23:58.433 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-17T04:23:58.433+0000 7f16b5fc5640 -1 osd.7 0 log_to_monitors true 2024-09-17T04:23:58.661 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-17T04:23:58.660+0000 7f118641e640 -1 osd.2 0 log_to_monitors true 2024-09-17T04:23:58.713 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-17T04:23:58.714+0000 7f1740dae640 -1 osd.4 0 log_to_monitors true 2024-09-17T04:23:58.733 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-17T04:23:58.734+0000 7f9c92558640 -1 osd.6 0 log_to_monitors true 2024-09-17T04:24:00.248 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-17T04:24:00.247+0000 7f4d95000640 -1 osd.3 0 waiting for initial osdmap 2024-09-17T04:24:00.503 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-17T04:24:00.503+0000 7f4d8b000640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T04:24:01.073 DEBUG:teuthology.orchestra.run.smithi100:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T04:24:01.339 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.338+0000 7f8385000640 1 -- 172.21.15.100:0/79793995 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83800b5f80 msgr2=0x7f83800b66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:01.339 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.338+0000 7f8385000640 1 --2- 172.21.15.100:0/79793995 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83800b5f80 0x7f83800b66f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f83800d2330 tx=0x7f837002a480 comp rx=0 tx=0).stop 2024-09-17T04:24:01.339 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.338+0000 7f8385000640 1 -- 172.21.15.100:0/79793995 shutdown_connections 2024-09-17T04:24:01.339 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.338+0000 7f8385000640 1 --2- 172.21.15.100:0/79793995 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83800cbb60 0x7f83800cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:01.340 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.338+0000 7f8385000640 1 --2- 172.21.15.100:0/79793995 >> v2:172.21.15.100:3300/0 conn(0x7f83800cdbd0 0x7f83800cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:01.340 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.338+0000 7f8385000640 1 --2- 172.21.15.100:0/79793995 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83800b5f80 0x7f83800b66f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:01.340 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.339+0000 7f8385000640 1 -- 172.21.15.100:0/79793995 >> 172.21.15.100:0/79793995 conn(0x7f83800b2e10 msgr2=0x7f83800b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:01.340 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.339+0000 7f8385000640 1 -- 172.21.15.100:0/79793995 shutdown_connections 2024-09-17T04:24:01.340 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.339+0000 7f8385000640 1 -- 172.21.15.100:0/79793995 wait complete. 2024-09-17T04:24:01.342 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.342+0000 7f8385000640 1 Processor -- start 2024-09-17T04:24:01.345 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.342+0000 7f8385000640 1 -- start start 2024-09-17T04:24:01.345 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.345+0000 7f8385000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83800b5f80 0x7f83800d1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:01.346 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.345+0000 7f8385000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83800cbb60 0x7f83801fd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:01.346 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.345+0000 7f8385000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83800cdbd0 0x7f83801531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:01.348 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.345+0000 7f8385000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f83800cc630 con 0x7f83800cdbd0 2024-09-17T04:24:01.349 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.345+0000 7f8385000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f83800cc4b0 con 0x7f83800cbb60 2024-09-17T04:24:01.349 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.345+0000 7f8385000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f83800cc7b0 con 0x7f83800b5f80 2024-09-17T04:24:01.349 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.345+0000 7f837fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83800cdbd0 0x7f83801531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:01.349 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.345+0000 7f837fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83800cdbd0 0x7f83801531c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45224/0 (socket says 172.21.15.100:45224) 2024-09-17T04:24:01.349 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.345+0000 7f837fe00640 1 -- 172.21.15.100:0/1600198835 learned_addr learned my addr 172.21.15.100:0/1600198835 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:01.349 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.346+0000 7f837ea00640 1 --2- 172.21.15.100:0/1600198835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83800cbb60 0x7f83801fd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:01.349 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.346+0000 7f837f400640 1 --2- 172.21.15.100:0/1600198835 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83800b5f80 0x7f83800d1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:01.350 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.346+0000 7f837fe00640 1 -- 172.21.15.100:0/1600198835 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83800b5f80 msgr2=0x7f83800d1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:01.350 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.346+0000 7f837fe00640 1 --2- 172.21.15.100:0/1600198835 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83800b5f80 0x7f83800d1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:01.350 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.346+0000 7f837fe00640 1 -- 172.21.15.100:0/1600198835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83800cbb60 msgr2=0x7f83801fd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:01.350 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.346+0000 7f837fe00640 1 --2- 172.21.15.100:0/1600198835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83800cbb60 0x7f83801fd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:01.350 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.346+0000 7f837fe00640 1 -- 172.21.15.100:0/1600198835 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8380156e70 con 0x7f83800cdbd0 2024-09-17T04:24:01.350 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.347+0000 7f837fe00640 1 --2- 172.21.15.100:0/1600198835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83800cdbd0 0x7f83801531c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f837800b990 tx=0x7f837800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:01.350 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.348+0000 7f8377400640 1 -- 172.21.15.100:0/1600198835 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8378019070 con 0x7f83800cdbd0 2024-09-17T04:24:01.351 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.348+0000 7f8385000640 1 -- 172.21.15.100:0/1600198835 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83801560b0 con 0x7f83800cdbd0 2024-09-17T04:24:01.351 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.349+0000 7f8385000640 1 -- 172.21.15.100:0/1600198835 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8380156670 con 0x7f83800cdbd0 2024-09-17T04:24:01.351 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.349+0000 7f8377400640 1 -- 172.21.15.100:0/1600198835 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f837800cc90 con 0x7f83800cdbd0 2024-09-17T04:24:01.351 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.349+0000 7f8377400640 1 -- 172.21.15.100:0/1600198835 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83780146a0 con 0x7f83800cdbd0 2024-09-17T04:24:01.352 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.350+0000 7f8385000640 1 -- 172.21.15.100:0/1600198835 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8358005290 con 0x7f83800cdbd0 2024-09-17T04:24:01.358 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.354+0000 7f8377400640 1 -- 172.21.15.100:0/1600198835 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 4) ==== 101385+0+0 (secure 0 0 0) 0x7f837801c070 con 0x7f83800cdbd0 2024-09-17T04:24:01.358 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.355+0000 7f8377400640 1 --2- 172.21.15.100:0/1600198835 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f835c078b30 0x7f835c07b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:01.358 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.356+0000 7f8377400640 1 -- 172.21.15.100:0/1600198835 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2231+0+0 (secure 0 0 0) 0x7f83780999b0 con 0x7f83800cdbd0 2024-09-17T04:24:01.358 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.356+0000 7f837f400640 1 --2- 172.21.15.100:0/1600198835 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f835c078b30 0x7f835c07b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:01.358 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.357+0000 7f837f400640 1 --2- 172.21.15.100:0/1600198835 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f835c078b30 0x7f835c07b040 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8370033000 tx=0x7f8370004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:01.366 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.366+0000 7f8377400640 1 -- 172.21.15.100:0/1600198835 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f83780626a0 con 0x7f83800cdbd0 2024-09-17T04:24:01.687 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.687+0000 7f8385000640 1 -- 172.21.15.100:0/1600198835 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8358005580 con 0x7f83800cdbd0 2024-09-17T04:24:01.688 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.688+0000 7f8377400640 1 -- 172.21.15.100:0/1600198835 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+7567 (secure 0 0 0) 0x7f83780670d0 con 0x7f83800cdbd0 2024-09-17T04:24:01.689 INFO:teuthology.misc.health.smithi100.stdout: 2024-09-17T04:24:01.689 INFO:teuthology.misc.health.smithi100.stdout:{"epoch":14,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:24:01.417069+0000","last_up_change":"2024-09-17T04:24:01.417069+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:24:01.692 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.691+0000 7f8385000640 1 -- 172.21.15.100:0/1600198835 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f835c078b30 msgr2=0x7f835c07b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:01.692 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.691+0000 7f8385000640 1 --2- 172.21.15.100:0/1600198835 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f835c078b30 0x7f835c07b040 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8370033000 tx=0x7f8370004730 comp rx=0 tx=0).stop 2024-09-17T04:24:01.692 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.691+0000 7f8385000640 1 -- 172.21.15.100:0/1600198835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83800cdbd0 msgr2=0x7f83801531c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:01.692 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.691+0000 7f8385000640 1 --2- 172.21.15.100:0/1600198835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83800cdbd0 0x7f83801531c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f837800b990 tx=0x7f837800be50 comp rx=0 tx=0).stop 2024-09-17T04:24:01.692 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.692+0000 7f8385000640 1 -- 172.21.15.100:0/1600198835 shutdown_connections 2024-09-17T04:24:01.693 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.692+0000 7f8385000640 1 --2- 172.21.15.100:0/1600198835 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f835c078b30 0x7f835c07b040 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:01.693 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.692+0000 7f8385000640 1 --2- 172.21.15.100:0/1600198835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83800cdbd0 0x7f83801531c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:01.693 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.692+0000 7f8385000640 1 --2- 172.21.15.100:0/1600198835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83800cbb60 0x7f83801fd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:01.693 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.692+0000 7f8385000640 1 --2- 172.21.15.100:0/1600198835 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83800b5f80 0x7f83800d1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:01.693 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.692+0000 7f8385000640 1 -- 172.21.15.100:0/1600198835 >> 172.21.15.100:0/1600198835 conn(0x7f83800b2e10 msgr2=0x7f8380070a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:01.693 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.692+0000 7f8385000640 1 -- 172.21.15.100:0/1600198835 shutdown_connections 2024-09-17T04:24:01.693 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:01.692+0000 7f8385000640 1 -- 172.21.15.100:0/1600198835 wait complete. 2024-09-17T04:24:01.720 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-09-17T04:24:02.084 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-17T04:24:02.084+0000 7f773fc00640 -1 osd.1 0 waiting for initial osdmap 2024-09-17T04:24:02.085 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-17T04:24:02.085+0000 7f1181800640 -1 osd.2 0 waiting for initial osdmap 2024-09-17T04:24:02.091 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-17T04:24:02.091+0000 7f7735200640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T04:24:02.095 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-17T04:24:02.095+0000 7f1176e00640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T04:24:02.126 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-17T04:24:02.126+0000 7fcb2a600640 -1 osd.0 0 waiting for initial osdmap 2024-09-17T04:24:02.132 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-17T04:24:02.132+0000 7fcb1fc00640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T04:24:02.483 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-17T04:24:02.484+0000 7fa108a00640 -1 osd.5 0 waiting for initial osdmap 2024-09-17T04:24:02.492 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-17T04:24:02.492+0000 7f173c600640 -1 osd.4 0 waiting for initial osdmap 2024-09-17T04:24:02.496 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-17T04:24:02.494+0000 7fa0fe000640 -1 osd.5 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T04:24:02.502 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-17T04:24:02.503+0000 7f1731c00640 -1 osd.4 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T04:24:02.512 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-17T04:24:02.512+0000 7f9c8d400640 -1 osd.6 0 waiting for initial osdmap 2024-09-17T04:24:02.525 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-17T04:24:02.526+0000 7f9c83400640 -1 osd.6 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T04:24:02.549 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-17T04:24:02.549+0000 7f16b0e00640 -1 osd.7 0 waiting for initial osdmap 2024-09-17T04:24:02.557 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-17T04:24:02.557+0000 7f16a6e00640 -1 osd.7 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T04:24:03.867 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:24:03.872 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:24:08.527 DEBUG:teuthology.orchestra.run.smithi100:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T04:24:08.731 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.731+0000 7fb951400640 1 -- 172.21.15.100:0/1277324366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb94c0b63c0 msgr2=0x7fb94c0ccd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:08.731 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.731+0000 7fb951400640 1 --2- 172.21.15.100:0/1277324366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb94c0b63c0 0x7fb94c0ccd80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb94c072f20 tx=0x7fb93402a5f0 comp rx=0 tx=0).stop 2024-09-17T04:24:08.731 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.731+0000 7fb951400640 1 -- 172.21.15.100:0/1277324366 shutdown_connections 2024-09-17T04:24:08.731 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.731+0000 7fb951400640 1 --2- 172.21.15.100:0/1277324366 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb94c0cd2c0 0x7fb94c0cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:08.731 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.731+0000 7fb951400640 1 --2- 172.21.15.100:0/1277324366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb94c0b63c0 0x7fb94c0ccd80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:08.732 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.731+0000 7fb951400640 1 --2- 172.21.15.100:0/1277324366 >> v2:172.21.15.184:3300/0 conn(0x7fb94c0c4930 0x7fb94c15c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:08.732 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.731+0000 7fb951400640 1 -- 172.21.15.100:0/1277324366 >> 172.21.15.100:0/1277324366 conn(0x7fb94c0b2e10 msgr2=0x7fb94c0b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:08.732 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.731+0000 7fb951400640 1 -- 172.21.15.100:0/1277324366 shutdown_connections 2024-09-17T04:24:08.732 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.731+0000 7fb951400640 1 -- 172.21.15.100:0/1277324366 wait complete. 2024-09-17T04:24:08.733 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.734+0000 7fb951400640 1 Processor -- start 2024-09-17T04:24:08.733 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.734+0000 7fb951400640 1 -- start start 2024-09-17T04:24:08.736 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.735+0000 7fb951400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb94c0b63c0 0x7fb94c154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:08.737 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.735+0000 7fb951400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb94c0c4930 0x7fb94c0af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:08.737 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.735+0000 7fb951400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb94c0cd2c0 0x7fb94c0afc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:08.737 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.735+0000 7fb951400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c06dc50 con 0x7fb94c0c4930 2024-09-17T04:24:08.737 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.735+0000 7fb951400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c06dad0 con 0x7fb94c0b63c0 2024-09-17T04:24:08.737 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.735+0000 7fb951400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb94c0ce1f0 con 0x7fb94c0cd2c0 2024-09-17T04:24:08.737 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.735+0000 7fb943e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb94c0c4930 0x7fb94c0af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:08.737 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.735+0000 7fb943e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb94c0c4930 0x7fb94c0af720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45024/0 (socket says 172.21.15.100:45024) 2024-09-17T04:24:08.737 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.735+0000 7fb943e00640 1 -- 172.21.15.100:0/2831561660 learned_addr learned my addr 172.21.15.100:0/2831561660 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:08.737 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.736+0000 7fb94be00640 1 --2- 172.21.15.100:0/2831561660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb94c0b63c0 0x7fb94c154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:08.737 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.736+0000 7fb950a00640 1 --2- 172.21.15.100:0/2831561660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb94c0cd2c0 0x7fb94c0afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:08.737 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.736+0000 7fb943e00640 1 -- 172.21.15.100:0/2831561660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb94c0cd2c0 msgr2=0x7fb94c0afc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:08.737 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.736+0000 7fb943e00640 1 --2- 172.21.15.100:0/2831561660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb94c0cd2c0 0x7fb94c0afc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:08.738 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.736+0000 7fb943e00640 1 -- 172.21.15.100:0/2831561660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb94c0b63c0 msgr2=0x7fb94c154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:08.738 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.736+0000 7fb943e00640 1 --2- 172.21.15.100:0/2831561660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb94c0b63c0 0x7fb94c154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:08.738 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.736+0000 7fb943e00640 1 -- 172.21.15.100:0/2831561660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb94c0cf250 con 0x7fb94c0c4930 2024-09-17T04:24:08.738 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.736+0000 7fb950a00640 1 --2- 172.21.15.100:0/2831561660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb94c0cd2c0 0x7fb94c0afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:24:08.738 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.736+0000 7fb943e00640 1 --2- 172.21.15.100:0/2831561660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb94c0c4930 0x7fb94c0af720 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb934034040 tx=0x7fb934003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:08.738 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.736+0000 7fb948c00640 1 -- 172.21.15.100:0/2831561660 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93403e030 con 0x7fb94c0c4930 2024-09-17T04:24:08.738 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.736+0000 7fb948c00640 1 -- 172.21.15.100:0/2831561660 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb934003e00 con 0x7fb94c0c4930 2024-09-17T04:24:08.738 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.737+0000 7fb951400640 1 -- 172.21.15.100:0/2831561660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb94c0cf530 con 0x7fb94c0c4930 2024-09-17T04:24:08.738 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.737+0000 7fb948c00640 1 -- 172.21.15.100:0/2831561660 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb934035040 con 0x7fb94c0c4930 2024-09-17T04:24:08.738 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.737+0000 7fb951400640 1 -- 172.21.15.100:0/2831561660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb94c0cfaf0 con 0x7fb94c0c4930 2024-09-17T04:24:08.739 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.738+0000 7fb951400640 1 -- 172.21.15.100:0/2831561660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb94c0b3750 con 0x7fb94c0c4930 2024-09-17T04:24:08.739 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.739+0000 7fb948c00640 1 -- 172.21.15.100:0/2831561660 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb934041030 con 0x7fb94c0c4930 2024-09-17T04:24:08.739 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.740+0000 7fb948c00640 1 --2- 172.21.15.100:0/2831561660 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb928078b90 0x7fb92807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:08.739 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.740+0000 7fb948c00640 1 -- 172.21.15.100:0/2831561660 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fb9340bb160 con 0x7fb94c0c4930 2024-09-17T04:24:08.740 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.740+0000 7fb94be00640 1 --2- 172.21.15.100:0/2831561660 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb928078b90 0x7fb92807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:08.740 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.741+0000 7fb94be00640 1 --2- 172.21.15.100:0/2831561660 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb928078b90 0x7fb92807b0a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb93c005fd0 tx=0x7fb93c00b260 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:08.744 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.745+0000 7fb948c00640 1 -- 172.21.15.100:0/2831561660 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb94c0b3750 con 0x7fb94c0c4930 2024-09-17T04:24:08.929 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.930+0000 7fb951400640 1 -- 172.21.15.100:0/2831561660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb94c15c880 con 0x7fb94c0c4930 2024-09-17T04:24:08.930 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.931+0000 7fb948c00640 1 -- 172.21.15.100:0/2831561660 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12944 (secure 0 0 0) 0x7fb934083200 con 0x7fb94c0c4930 2024-09-17T04:24:08.930 INFO:teuthology.misc.health.smithi100.stdout: 2024-09-17T04:24:08.930 INFO:teuthology.misc.health.smithi100.stdout:{"epoch":19,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:24:06.515652+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:24:08.933 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.933+0000 7fb951400640 1 -- 172.21.15.100:0/2831561660 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb928078b90 msgr2=0x7fb92807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:08.933 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.933+0000 7fb951400640 1 --2- 172.21.15.100:0/2831561660 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb928078b90 0x7fb92807b0a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb93c005fd0 tx=0x7fb93c00b260 comp rx=0 tx=0).stop 2024-09-17T04:24:08.933 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.933+0000 7fb951400640 1 -- 172.21.15.100:0/2831561660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb94c0c4930 msgr2=0x7fb94c0af720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:08.933 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.933+0000 7fb951400640 1 --2- 172.21.15.100:0/2831561660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb94c0c4930 0x7fb94c0af720 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb934034040 tx=0x7fb934003910 comp rx=0 tx=0).stop 2024-09-17T04:24:08.934 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.933+0000 7fb951400640 1 -- 172.21.15.100:0/2831561660 shutdown_connections 2024-09-17T04:24:08.934 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.933+0000 7fb951400640 1 --2- 172.21.15.100:0/2831561660 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb928078b90 0x7fb92807b0a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:08.934 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.933+0000 7fb951400640 1 --2- 172.21.15.100:0/2831561660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb94c0cd2c0 0x7fb94c0afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:08.934 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.933+0000 7fb951400640 1 --2- 172.21.15.100:0/2831561660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb94c0c4930 0x7fb94c0af720 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:08.934 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.933+0000 7fb951400640 1 --2- 172.21.15.100:0/2831561660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb94c0b63c0 0x7fb94c154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:08.934 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.933+0000 7fb951400640 1 -- 172.21.15.100:0/2831561660 >> 172.21.15.100:0/2831561660 conn(0x7fb94c0b2e10 msgr2=0x7fb94c071780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:08.934 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.934+0000 7fb951400640 1 -- 172.21.15.100:0/2831561660 shutdown_connections 2024-09-17T04:24:08.934 INFO:teuthology.misc.health.smithi100.stderr:2024-09-17T04:24:08.934+0000 7fb951400640 1 -- 172.21.15.100:0/2831561660 wait complete. 2024-09-17T04:24:08.953 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-17T04:24:08.953 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-17T04:24:08.954 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T04:24:08.954 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T04:24:08.958 INFO:tasks.ceph.mds.a:Started 2024-09-17T04:24:08.958 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T04:24:08.958 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T04:24:08.962 INFO:tasks.ceph.mds.c:Started 2024-09-17T04:24:08.962 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T04:24:08.962 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T04:24:08.966 INFO:tasks.ceph.mds.b:Started 2024-09-17T04:24:08.966 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T04:24:08.966 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T04:24:08.970 INFO:tasks.ceph.mds.d:Started 2024-09-17T04:24:08.971 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-17T04:24:08.971 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:24:09.155 INFO:tasks.ceph.mds.a.smithi100.stdout:starting mds.a at 2024-09-17T04:24:09.161 INFO:tasks.ceph.mds.c.smithi100.stdout:starting mds.c at 2024-09-17T04:24:09.174 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-09-17T04:24:09.174 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-09-17T04:24:09.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.204+0000 7fb1a4e00640 1 -- 172.21.15.100:0/3564603314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1a00b2f60 msgr2=0x7fb1a015a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:09.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.204+0000 7fb1a4e00640 1 --2- 172.21.15.100:0/3564603314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1a00b2f60 0x7fb1a015a0f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb188009830 tx=0x7fb18802a480 comp rx=0 tx=0).stop 2024-09-17T04:24:09.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.204+0000 7fb1a4e00640 1 -- 172.21.15.100:0/3564603314 shutdown_connections 2024-09-17T04:24:09.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.204+0000 7fb1a4e00640 1 --2- 172.21.15.100:0/3564603314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1a00b2f60 0x7fb1a015a0f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:09.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.204+0000 7fb1a4e00640 1 --2- 172.21.15.100:0/3564603314 >> v2:172.21.15.184:3300/0 conn(0x7fb1a00ad9c0 0x7fb1a00af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:09.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.204+0000 7fb1a4e00640 1 -- 172.21.15.100:0/3564603314 >> 172.21.15.100:0/3564603314 conn(0x7fb1a00b1e40 msgr2=0x7fb1a00b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:09.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.204+0000 7fb1a4e00640 1 -- 172.21.15.100:0/3564603314 shutdown_connections 2024-09-17T04:24:09.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.204+0000 7fb1a4e00640 1 -- 172.21.15.100:0/3564603314 wait complete. 2024-09-17T04:24:09.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.206+0000 7fb1a4e00640 1 Processor -- start 2024-09-17T04:24:09.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.206+0000 7fb1a4e00640 1 -- start start 2024-09-17T04:24:09.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.208+0000 7fb1a4e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1a00ad9c0 0x7fb1a01e2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:09.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.208+0000 7fb1a4e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1a00b2f60 0x7fb1a0145810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:09.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.208+0000 7fb1a4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1a015d670 0x7fb1a01f6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:09.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.208+0000 7fb1a4e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a006f410 con 0x7fb1a015d670 2024-09-17T04:24:09.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.208+0000 7fb1a4e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a006f290 con 0x7fb1a00b2f60 2024-09-17T04:24:09.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.208+0000 7fb1a4e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb1a006f590 con 0x7fb1a00ad9c0 2024-09-17T04:24:09.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.208+0000 7fb19fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1a015d670 0x7fb1a01f6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:09.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.208+0000 7fb19fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1a015d670 0x7fb1a01f6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45058/0 (socket says 172.21.15.100:45058) 2024-09-17T04:24:09.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.208+0000 7fb19fe00640 1 -- 172.21.15.100:0/656350958 learned_addr learned my addr 172.21.15.100:0/656350958 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:09.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.208+0000 7fb19f400640 1 --2- 172.21.15.100:0/656350958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1a00ad9c0 0x7fb1a01e2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:09.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.208+0000 7fb196a00640 1 --2- 172.21.15.100:0/656350958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1a00b2f60 0x7fb1a0145810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:09.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.208+0000 7fb19fe00640 1 -- 172.21.15.100:0/656350958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1a00ad9c0 msgr2=0x7fb1a01e2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:09.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.208+0000 7fb19fe00640 1 --2- 172.21.15.100:0/656350958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1a00ad9c0 0x7fb1a01e2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:09.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.208+0000 7fb19fe00640 1 -- 172.21.15.100:0/656350958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1a00b2f60 msgr2=0x7fb1a0145810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:09.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.208+0000 7fb19fe00640 1 --2- 172.21.15.100:0/656350958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1a00b2f60 0x7fb1a0145810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:09.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.208+0000 7fb19fe00640 1 -- 172.21.15.100:0/656350958 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a0158770 con 0x7fb1a015d670 2024-09-17T04:24:09.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.209+0000 7fb19fe00640 1 --2- 172.21.15.100:0/656350958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1a015d670 0x7fb1a01f6970 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb19800cae0 tx=0x7fb19800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:09.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.209+0000 7fb197e00640 1 -- 172.21.15.100:0/656350958 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb198016020 con 0x7fb1a015d670 2024-09-17T04:24:09.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.209+0000 7fb197e00640 1 -- 172.21.15.100:0/656350958 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb19800ad30 con 0x7fb1a015d670 2024-09-17T04:24:09.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.209+0000 7fb1a4e00640 1 -- 172.21.15.100:0/656350958 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a0158a50 con 0x7fb1a015d670 2024-09-17T04:24:09.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.209+0000 7fb197e00640 1 -- 172.21.15.100:0/656350958 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb198012680 con 0x7fb1a015d670 2024-09-17T04:24:09.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.209+0000 7fb1a4e00640 1 -- 172.21.15.100:0/656350958 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a0159010 con 0x7fb1a015d670 2024-09-17T04:24:09.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.210+0000 7fb1a4e00640 1 -- 172.21.15.100:0/656350958 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1a006fa20 con 0x7fb1a015d670 2024-09-17T04:24:09.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.211+0000 7fb197e00640 1 -- 172.21.15.100:0/656350958 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb198019020 con 0x7fb1a015d670 2024-09-17T04:24:09.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.212+0000 7fb197e00640 1 --2- 172.21.15.100:0/656350958 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb17c078b90 0x7fb17c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:09.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.212+0000 7fb197e00640 1 -- 172.21.15.100:0/656350958 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fb19809ad00 con 0x7fb1a015d670 2024-09-17T04:24:09.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.212+0000 7fb19f400640 1 --2- 172.21.15.100:0/656350958 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb17c078b90 0x7fb17c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:09.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.213+0000 7fb19f400640 1 --2- 172.21.15.100:0/656350958 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb17c078b90 0x7fb17c07b0a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb190009b30 tx=0x7fb190005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:09.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.217+0000 7fb197e00640 1 -- 172.21.15.100:0/656350958 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb198020030 con 0x7fb1a015d670 2024-09-17T04:24:09.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.436+0000 7fb1a4e00640 1 -- 172.21.15.100:0/656350958 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1a00705d0 con 0x7fb1a015d670 2024-09-17T04:24:09.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.437+0000 7fb197e00640 1 -- 172.21.15.100:0/656350958 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1437 (secure 0 0 0) 0x7fb198062d20 con 0x7fb1a015d670 2024-09-17T04:24:09.436 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:09.437 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":2,"btime":"2024-09-17T04:24:09:162886+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/1644264480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1644264480},{"type":"v1","addr":"172.21.15.100:6834","nonce":1644264480}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-17T04:24:09.437 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 2 2024-09-17T04:24:09.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.439+0000 7fb1a4e00640 1 -- 172.21.15.100:0/656350958 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb17c078b90 msgr2=0x7fb17c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:09.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.439+0000 7fb1a4e00640 1 --2- 172.21.15.100:0/656350958 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb17c078b90 0x7fb17c07b0a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb190009b30 tx=0x7fb190005f90 comp rx=0 tx=0).stop 2024-09-17T04:24:09.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.439+0000 7fb1a4e00640 1 -- 172.21.15.100:0/656350958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1a015d670 msgr2=0x7fb1a01f6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:09.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.439+0000 7fb1a4e00640 1 --2- 172.21.15.100:0/656350958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1a015d670 0x7fb1a01f6970 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb19800cae0 tx=0x7fb19800a060 comp rx=0 tx=0).stop 2024-09-17T04:24:09.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.440+0000 7fb1a4e00640 1 -- 172.21.15.100:0/656350958 shutdown_connections 2024-09-17T04:24:09.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.440+0000 7fb1a4e00640 1 --2- 172.21.15.100:0/656350958 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb17c078b90 0x7fb17c07b0a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:09.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.440+0000 7fb1a4e00640 1 --2- 172.21.15.100:0/656350958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1a015d670 0x7fb1a01f6970 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:09.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.440+0000 7fb1a4e00640 1 --2- 172.21.15.100:0/656350958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1a00b2f60 0x7fb1a0145810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:09.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.440+0000 7fb1a4e00640 1 --2- 172.21.15.100:0/656350958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1a00ad9c0 0x7fb1a01e2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:09.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.440+0000 7fb1a4e00640 1 -- 172.21.15.100:0/656350958 >> 172.21.15.100:0/656350958 conn(0x7fb1a00b1e40 msgr2=0x7fb1a015bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:09.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.440+0000 7fb1a4e00640 1 -- 172.21.15.100:0/656350958 shutdown_connections 2024-09-17T04:24:09.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:09.440+0000 7fb1a4e00640 1 -- 172.21.15.100:0/656350958 wait complete. 2024-09-17T04:24:11.461 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:24:11.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.692+0000 7fac77200640 1 -- 172.21.15.100:0/3736286208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac780b3160 msgr2=0x7fac780d24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:11.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.692+0000 7fac77200640 1 --2- 172.21.15.100:0/3736286208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac780b3160 0x7fac780d24a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fac7808d440 tx=0x7fac6002a4d0 comp rx=0 tx=0).stop 2024-09-17T04:24:11.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.692+0000 7fac77200640 1 -- 172.21.15.100:0/3736286208 shutdown_connections 2024-09-17T04:24:11.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.692+0000 7fac77200640 1 --2- 172.21.15.100:0/3736286208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac780d29e0 0x7fac780d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:11.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.692+0000 7fac77200640 1 --2- 172.21.15.100:0/3736286208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac780b3160 0x7fac780d24a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:11.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.692+0000 7fac77200640 1 --2- 172.21.15.100:0/3736286208 >> v2:172.21.15.184:3300/0 conn(0x7fac780b9e00 0x7fac78162a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:11.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.692+0000 7fac77200640 1 -- 172.21.15.100:0/3736286208 >> 172.21.15.100:0/3736286208 conn(0x7fac7808bfe0 msgr2=0x7fac7809d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:11.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.692+0000 7fac77200640 1 -- 172.21.15.100:0/3736286208 shutdown_connections 2024-09-17T04:24:11.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.692+0000 7fac77200640 1 -- 172.21.15.100:0/3736286208 wait complete. 2024-09-17T04:24:11.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.694+0000 7fac77200640 1 Processor -- start 2024-09-17T04:24:11.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.694+0000 7fac77200640 1 -- start start 2024-09-17T04:24:11.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.696+0000 7fac77200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac780b3160 0x7fac7815ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:11.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.696+0000 7fac77200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac780b9e00 0x7fac781ea230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:11.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.696+0000 7fac77200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac780d29e0 0x7fac781ea770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:11.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.696+0000 7fac77200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fac780d4330 con 0x7fac780b9e00 2024-09-17T04:24:11.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.696+0000 7fac77200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fac780d1e10 con 0x7fac780b3160 2024-09-17T04:24:11.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.696+0000 7fac77200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fac780d44b0 con 0x7fac780d29e0 2024-09-17T04:24:11.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.696+0000 7fac6d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac780b9e00 0x7fac781ea230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:11.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.696+0000 7fac6d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac780b9e00 0x7fac781ea230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45078/0 (socket says 172.21.15.100:45078) 2024-09-17T04:24:11.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.696+0000 7fac6d400640 1 -- 172.21.15.100:0/3635848033 learned_addr learned my addr 172.21.15.100:0/3635848033 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:11.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.696+0000 7fac75e00640 1 --2- 172.21.15.100:0/3635848033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac780b3160 0x7fac7815ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:11.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.696+0000 7fac76800640 1 --2- 172.21.15.100:0/3635848033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac780d29e0 0x7fac781ea770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:11.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.697+0000 7fac6d400640 1 -- 172.21.15.100:0/3635848033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac780d29e0 msgr2=0x7fac781ea770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:11.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.697+0000 7fac6d400640 1 --2- 172.21.15.100:0/3635848033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac780d29e0 0x7fac781ea770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:11.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.697+0000 7fac6d400640 1 -- 172.21.15.100:0/3635848033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac780b3160 msgr2=0x7fac7815ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:11.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.697+0000 7fac6d400640 1 --2- 172.21.15.100:0/3635848033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac780b3160 0x7fac7815ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:11.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.697+0000 7fac6d400640 1 -- 172.21.15.100:0/3635848033 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac780a1c10 con 0x7fac780b9e00 2024-09-17T04:24:11.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.697+0000 7fac6d400640 1 --2- 172.21.15.100:0/3635848033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac780b9e00 0x7fac781ea230 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fac60034040 tx=0x7fac600038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:11.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.697+0000 7fac6ea00640 1 -- 172.21.15.100:0/3635848033 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6003e030 con 0x7fac780b9e00 2024-09-17T04:24:11.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.697+0000 7fac6ea00640 1 -- 172.21.15.100:0/3635848033 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac60003ca0 con 0x7fac780b9e00 2024-09-17T04:24:11.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.697+0000 7fac6ea00640 1 -- 172.21.15.100:0/3635848033 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60035040 con 0x7fac780b9e00 2024-09-17T04:24:11.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.697+0000 7fac77200640 1 -- 172.21.15.100:0/3635848033 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac780a2050 con 0x7fac780b9e00 2024-09-17T04:24:11.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.697+0000 7fac77200640 1 -- 172.21.15.100:0/3635848033 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac781e92f0 con 0x7fac780b9e00 2024-09-17T04:24:11.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.698+0000 7fac77200640 1 -- 172.21.15.100:0/3635848033 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac780d49d0 con 0x7fac780b9e00 2024-09-17T04:24:11.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.700+0000 7fac6ea00640 1 -- 172.21.15.100:0/3635848033 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fac60041030 con 0x7fac780b9e00 2024-09-17T04:24:11.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.701+0000 7fac6ea00640 1 --2- 172.21.15.100:0/3635848033 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fac54078b90 0x7fac5407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:11.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.701+0000 7fac6ea00640 1 -- 172.21.15.100:0/3635848033 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fac600bb080 con 0x7fac780b9e00 2024-09-17T04:24:11.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.701+0000 7fac75e00640 1 --2- 172.21.15.100:0/3635848033 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fac54078b90 0x7fac5407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:11.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.702+0000 7fac75e00640 1 --2- 172.21.15.100:0/3635848033 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fac54078b90 0x7fac5407b0a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fac68009b30 tx=0x7fac68005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:11.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.705+0000 7fac6ea00640 1 -- 172.21.15.100:0/3635848033 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fac600831b0 con 0x7fac780b9e00 2024-09-17T04:24:11.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.925+0000 7fac77200640 1 -- 172.21.15.100:0/3635848033 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac780a1de0 con 0x7fac780b9e00 2024-09-17T04:24:11.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.926+0000 7fac6ea00640 1 -- 172.21.15.100:0/3635848033 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3912 (secure 0 0 0) 0x7fac60087be0 con 0x7fac780b9e00 2024-09-17T04:24:11.925 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:11.926 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":3,"btime":"2024-09-17T04:24:10:164383+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4328,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2129046042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":2129046042},{"type":"v1","addr":"172.21.15.100:6835","nonce":2129046042}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4334,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/44255118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":44255118},{"type":"v1","addr":"172.21.15.184:6836","nonce":44255118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/1644264480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1644264480},{"type":"v1","addr":"172.21.15.100:6834","nonce":1644264480}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4368,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2211103925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2211103925},{"type":"v1","addr":"172.21.15.184:6837","nonce":2211103925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-17T04:24:11.926 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 3 2024-09-17T04:24:11.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.928+0000 7fac77200640 1 -- 172.21.15.100:0/3635848033 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fac54078b90 msgr2=0x7fac5407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:11.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.928+0000 7fac77200640 1 --2- 172.21.15.100:0/3635848033 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fac54078b90 0x7fac5407b0a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fac68009b30 tx=0x7fac68005f90 comp rx=0 tx=0).stop 2024-09-17T04:24:11.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.928+0000 7fac77200640 1 -- 172.21.15.100:0/3635848033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac780b9e00 msgr2=0x7fac781ea230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:11.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.928+0000 7fac77200640 1 --2- 172.21.15.100:0/3635848033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac780b9e00 0x7fac781ea230 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fac60034040 tx=0x7fac600038a0 comp rx=0 tx=0).stop 2024-09-17T04:24:11.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.928+0000 7fac77200640 1 -- 172.21.15.100:0/3635848033 shutdown_connections 2024-09-17T04:24:11.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.928+0000 7fac77200640 1 --2- 172.21.15.100:0/3635848033 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fac54078b90 0x7fac5407b0a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:11.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.928+0000 7fac77200640 1 --2- 172.21.15.100:0/3635848033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac780d29e0 0x7fac781ea770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:11.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.928+0000 7fac77200640 1 --2- 172.21.15.100:0/3635848033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac780b9e00 0x7fac781ea230 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:11.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.928+0000 7fac77200640 1 --2- 172.21.15.100:0/3635848033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac780b3160 0x7fac7815ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:11.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.928+0000 7fac77200640 1 -- 172.21.15.100:0/3635848033 >> 172.21.15.100:0/3635848033 conn(0x7fac7808bfe0 msgr2=0x7fac780b5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:11.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.928+0000 7fac77200640 1 -- 172.21.15.100:0/3635848033 shutdown_connections 2024-09-17T04:24:11.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:11.928+0000 7fac77200640 1 -- 172.21.15.100:0/3635848033 wait complete. 2024-09-17T04:24:11.949 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T04:24:12.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.169+0000 7f7b2d200640 1 -- 172.21.15.100:0/3981244576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b2809ba60 msgr2=0x7f7b280b7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:12.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.169+0000 7f7b2d200640 1 --2- 172.21.15.100:0/3981244576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b2809ba60 0x7f7b280b7330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7b280cc9c0 tx=0x7f7b1802a540 comp rx=0 tx=0).stop 2024-09-17T04:24:12.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.169+0000 7f7b2d200640 1 -- 172.21.15.100:0/3981244576 shutdown_connections 2024-09-17T04:24:12.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.169+0000 7f7b2d200640 1 --2- 172.21.15.100:0/3981244576 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b280cffe0 0x7f7b280d03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:12.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.169+0000 7f7b2d200640 1 --2- 172.21.15.100:0/3981244576 >> v2:172.21.15.184:3300/0 conn(0x7f7b280cd440 0x7f7b280cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:12.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.169+0000 7f7b2d200640 1 --2- 172.21.15.100:0/3981244576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b2809ba60 0x7f7b280b7330 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:12.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.169+0000 7f7b2d200640 1 -- 172.21.15.100:0/3981244576 >> 172.21.15.100:0/3981244576 conn(0x7f7b280cee90 msgr2=0x7f7b280cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:12.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.170+0000 7f7b2d200640 1 -- 172.21.15.100:0/3981244576 shutdown_connections 2024-09-17T04:24:12.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.170+0000 7f7b2d200640 1 -- 172.21.15.100:0/3981244576 wait complete. 2024-09-17T04:24:12.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.172+0000 7f7b2d200640 1 Processor -- start 2024-09-17T04:24:12.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.172+0000 7f7b2d200640 1 -- start start 2024-09-17T04:24:12.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.173+0000 7f7b2d200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b2809ba60 0x7f7b281f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:12.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.173+0000 7f7b2d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b280cd440 0x7f7b281df6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:12.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.173+0000 7f7b2d200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b280cffe0 0x7f7b280cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:12.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.173+0000 7f7b2d200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7b280d0cc0 con 0x7f7b280cd440 2024-09-17T04:24:12.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.173+0000 7f7b2d200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7b280d0b40 con 0x7f7b2809ba60 2024-09-17T04:24:12.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.173+0000 7f7b2d200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7b280d0e40 con 0x7f7b280cffe0 2024-09-17T04:24:12.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.174+0000 7f7b25400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b280cd440 0x7f7b281df6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:12.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.174+0000 7f7b25400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b280cd440 0x7f7b281df6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45102/0 (socket says 172.21.15.100:45102) 2024-09-17T04:24:12.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.174+0000 7f7b25400640 1 -- 172.21.15.100:0/588517692 learned_addr learned my addr 172.21.15.100:0/588517692 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:12.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.174+0000 7f7b25e00640 1 --2- 172.21.15.100:0/588517692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b2809ba60 0x7f7b281f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:12.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.174+0000 7f7b26800640 1 --2- 172.21.15.100:0/588517692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b280cffe0 0x7f7b280cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:12.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.174+0000 7f7b25400640 1 -- 172.21.15.100:0/588517692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b280cffe0 msgr2=0x7f7b280cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:12.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.174+0000 7f7b25400640 1 --2- 172.21.15.100:0/588517692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b280cffe0 0x7f7b280cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:12.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.174+0000 7f7b25400640 1 -- 172.21.15.100:0/588517692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b2809ba60 msgr2=0x7f7b281f5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:12.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.174+0000 7f7b25400640 1 --2- 172.21.15.100:0/588517692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b2809ba60 0x7f7b281f5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:12.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.174+0000 7f7b25400640 1 -- 172.21.15.100:0/588517692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b281f64d0 con 0x7f7b280cd440 2024-09-17T04:24:12.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.174+0000 7f7b25400640 1 --2- 172.21.15.100:0/588517692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b280cd440 0x7f7b281df6d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400e710 tx=0x7f7b1400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:12.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.174+0000 7f7b1e000640 1 -- 172.21.15.100:0/588517692 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1400cab0 con 0x7f7b280cd440 2024-09-17T04:24:12.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.175+0000 7f7b1e000640 1 -- 172.21.15.100:0/588517692 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b1400cd40 con 0x7f7b280cd440 2024-09-17T04:24:12.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.175+0000 7f7b2d200640 1 -- 172.21.15.100:0/588517692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b281f67b0 con 0x7f7b280cd440 2024-09-17T04:24:12.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.175+0000 7f7b1e000640 1 -- 172.21.15.100:0/588517692 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b14010690 con 0x7f7b280cd440 2024-09-17T04:24:12.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.175+0000 7f7b2d200640 1 -- 172.21.15.100:0/588517692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b281e25f0 con 0x7f7b280cd440 2024-09-17T04:24:12.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.176+0000 7f7b2d200640 1 -- 172.21.15.100:0/588517692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b281f6240 con 0x7f7b280cd440 2024-09-17T04:24:12.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.177+0000 7f7b1e000640 1 -- 172.21.15.100:0/588517692 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7b14010a40 con 0x7f7b280cd440 2024-09-17T04:24:12.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.178+0000 7f7b1e000640 1 --2- 172.21.15.100:0/588517692 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7b08078b90 0x7f7b0807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:12.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.178+0000 7f7b1e000640 1 -- 172.21.15.100:0/588517692 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f7b1409b660 con 0x7f7b280cd440 2024-09-17T04:24:12.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.179+0000 7f7b25e00640 1 --2- 172.21.15.100:0/588517692 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7b08078b90 0x7f7b0807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:12.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.179+0000 7f7b25e00640 1 --2- 172.21.15.100:0/588517692 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7b08078b90 0x7f7b0807b0a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7b18034000 tx=0x7f7b18004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:12.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.183+0000 7f7b1e000640 1 -- 172.21.15.100:0/588517692 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f7b14063d50 con 0x7f7b280cd440 2024-09-17T04:24:12.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.369+0000 7f7b2d200640 1 -- 172.21.15.100:0/588517692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7b281e0e40 con 0x7f7b280cd440 2024-09-17T04:24:12.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.369+0000 7f7b1e000640 1 -- 172.21.15.100:0/588517692 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) ==== 84+0+65 (secure 0 0 0) 0x7f7b14063650 con 0x7f7b280cd440 2024-09-17T04:24:12.369 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:12.369 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-09-17T04:24:12.369 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:24:12.369 INFO:teuthology.orchestra.run.smithi100.stdout: "poolnum": 1, 2024-09-17T04:24:12.369 INFO:teuthology.orchestra.run.smithi100.stdout: "poolname": ".mgr" 2024-09-17T04:24:12.369 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:24:12.369 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-09-17T04:24:12.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.371+0000 7f7b2d200640 1 -- 172.21.15.100:0/588517692 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7b08078b90 msgr2=0x7f7b0807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:12.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.371+0000 7f7b2d200640 1 --2- 172.21.15.100:0/588517692 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7b08078b90 0x7f7b0807b0a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7b18034000 tx=0x7f7b18004730 comp rx=0 tx=0).stop 2024-09-17T04:24:12.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.371+0000 7f7b2d200640 1 -- 172.21.15.100:0/588517692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b280cd440 msgr2=0x7f7b281df6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:12.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.371+0000 7f7b2d200640 1 --2- 172.21.15.100:0/588517692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b280cd440 0x7f7b281df6d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400e710 tx=0x7f7b1400ebd0 comp rx=0 tx=0).stop 2024-09-17T04:24:12.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.371+0000 7f7b2d200640 1 -- 172.21.15.100:0/588517692 shutdown_connections 2024-09-17T04:24:12.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.371+0000 7f7b2d200640 1 --2- 172.21.15.100:0/588517692 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7b08078b90 0x7f7b0807b0a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:12.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.371+0000 7f7b2d200640 1 --2- 172.21.15.100:0/588517692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b280cffe0 0x7f7b280cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:12.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.371+0000 7f7b2d200640 1 --2- 172.21.15.100:0/588517692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b280cd440 0x7f7b281df6d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:12.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.371+0000 7f7b2d200640 1 --2- 172.21.15.100:0/588517692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b2809ba60 0x7f7b281f5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:12.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.371+0000 7f7b2d200640 1 -- 172.21.15.100:0/588517692 >> 172.21.15.100:0/588517692 conn(0x7f7b280cee90 msgr2=0x7f7b281e8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:12.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.371+0000 7f7b2d200640 1 -- 172.21.15.100:0/588517692 shutdown_connections 2024-09-17T04:24:12.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.372+0000 7f7b2d200640 1 -- 172.21.15.100:0/588517692 wait complete. 2024-09-17T04:24:12.392 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T04:24:12.392 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T04:24:12.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.617+0000 7f7a4e200640 1 -- 172.21.15.100:0/1651539518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a48162ff0 msgr2=0x7f7a48158e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:12.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.617+0000 7f7a4e200640 1 --2- 172.21.15.100:0/1651539518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a48162ff0 0x7f7a48158e70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7a480710a0 tx=0x7f7a3002a5f0 comp rx=0 tx=0).stop 2024-09-17T04:24:12.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.617+0000 7f7a4e200640 1 -- 172.21.15.100:0/1651539518 shutdown_connections 2024-09-17T04:24:12.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.617+0000 7f7a4e200640 1 --2- 172.21.15.100:0/1651539518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a48162ff0 0x7f7a48158e70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:12.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.617+0000 7f7a4e200640 1 --2- 172.21.15.100:0/1651539518 >> v2:172.21.15.184:3300/0 conn(0x7f7a4815fd30 0x7f7a480b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:12.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.617+0000 7f7a4e200640 1 -- 172.21.15.100:0/1651539518 >> 172.21.15.100:0/1651539518 conn(0x7f7a480b2540 msgr2=0x7f7a480d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:12.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.617+0000 7f7a4e200640 1 -- 172.21.15.100:0/1651539518 shutdown_connections 2024-09-17T04:24:12.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.617+0000 7f7a4e200640 1 -- 172.21.15.100:0/1651539518 wait complete. 2024-09-17T04:24:12.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.619+0000 7f7a4e200640 1 Processor -- start 2024-09-17T04:24:12.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.619+0000 7f7a4e200640 1 -- start start 2024-09-17T04:24:12.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.620+0000 7f7a4e200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a4806cda0 0x7f7a481f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:12.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.620+0000 7f7a4e200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a4815fd30 0x7f7a481f8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:12.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.620+0000 7f7a4e200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a48162ff0 0x7f7a480742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:12.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.620+0000 7f7a4e200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7a480af490 con 0x7f7a4815fd30 2024-09-17T04:24:12.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.620+0000 7f7a4e200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7a480af310 con 0x7f7a48162ff0 2024-09-17T04:24:12.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.620+0000 7f7a4e200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7a480af610 con 0x7f7a4806cda0 2024-09-17T04:24:12.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.621+0000 7f7a3ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a4815fd30 0x7f7a481f8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:12.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.621+0000 7f7a3ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a4815fd30 0x7f7a481f8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45120/0 (socket says 172.21.15.100:45120) 2024-09-17T04:24:12.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.621+0000 7f7a3ea00640 1 -- 172.21.15.100:0/129206920 learned_addr learned my addr 172.21.15.100:0/129206920 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:12.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.621+0000 7f7a47400640 1 --2- 172.21.15.100:0/129206920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a4806cda0 0x7f7a481f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:12.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.621+0000 7f7a47e00640 1 --2- 172.21.15.100:0/129206920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a48162ff0 0x7f7a480742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:12.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.621+0000 7f7a3ea00640 1 -- 172.21.15.100:0/129206920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a4806cda0 msgr2=0x7f7a481f64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:12.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.621+0000 7f7a3ea00640 1 --2- 172.21.15.100:0/129206920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a4806cda0 0x7f7a481f64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:12.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.621+0000 7f7a3ea00640 1 -- 172.21.15.100:0/129206920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a48162ff0 msgr2=0x7f7a480742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:12.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.621+0000 7f7a3ea00640 1 --2- 172.21.15.100:0/129206920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a48162ff0 0x7f7a480742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:12.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.621+0000 7f7a3ea00640 1 -- 172.21.15.100:0/129206920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a481f99e0 con 0x7f7a4815fd30 2024-09-17T04:24:12.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.621+0000 7f7a47400640 1 --2- 172.21.15.100:0/129206920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a4806cda0 0x7f7a481f64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:24:12.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.621+0000 7f7a47e00640 1 --2- 172.21.15.100:0/129206920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a48162ff0 0x7f7a480742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:24:12.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.621+0000 7f7a3ea00640 1 --2- 172.21.15.100:0/129206920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a4815fd30 0x7f7a481f8f60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7a30034040 tx=0x7f7a30003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:12.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.622+0000 7f7a3fe00640 1 -- 172.21.15.100:0/129206920 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3003e030 con 0x7f7a4815fd30 2024-09-17T04:24:12.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.622+0000 7f7a3fe00640 1 -- 172.21.15.100:0/129206920 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a30003d10 con 0x7f7a4815fd30 2024-09-17T04:24:12.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.622+0000 7f7a3fe00640 1 -- 172.21.15.100:0/129206920 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a30035040 con 0x7f7a4815fd30 2024-09-17T04:24:12.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.622+0000 7f7a4e200640 1 -- 172.21.15.100:0/129206920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a481f9cc0 con 0x7f7a4815fd30 2024-09-17T04:24:12.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.622+0000 7f7a4e200640 1 -- 172.21.15.100:0/129206920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a481f5fa0 con 0x7f7a4815fd30 2024-09-17T04:24:12.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.623+0000 7f7a4e200640 1 -- 172.21.15.100:0/129206920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a481f9750 con 0x7f7a4815fd30 2024-09-17T04:24:12.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.624+0000 7f7a3fe00640 1 -- 172.21.15.100:0/129206920 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7a30041030 con 0x7f7a4815fd30 2024-09-17T04:24:12.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.625+0000 7f7a3fe00640 1 --2- 172.21.15.100:0/129206920 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7a24078bc0 0x7f7a2407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:12.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.625+0000 7f7a3fe00640 1 -- 172.21.15.100:0/129206920 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f7a300bb140 con 0x7f7a4815fd30 2024-09-17T04:24:12.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.625+0000 7f7a47400640 1 --2- 172.21.15.100:0/129206920 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7a24078bc0 0x7f7a2407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:12.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.626+0000 7f7a47400640 1 --2- 172.21.15.100:0/129206920 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7a24078bc0 0x7f7a2407b0d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7a38009ab0 tx=0x7f7a38005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:12.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.629+0000 7f7a3fe00640 1 -- 172.21.15.100:0/129206920 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f7a300838c0 con 0x7f7a4815fd30 2024-09-17T04:24:12.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:12.814+0000 7f7a4e200640 1 -- 172.21.15.100:0/129206920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f7a48073b90 con 0x7f7a4815fd30 2024-09-17T04:24:13.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.017+0000 7f7a3fe00640 1 -- 172.21.15.100:0/129206920 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) ==== 138+0+0 (secure 0 0 0) 0x7f7a300831c0 con 0x7f7a4815fd30 2024-09-17T04:24:13.017 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' created 2024-09-17T04:24:13.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.020+0000 7f7a4e200640 1 -- 172.21.15.100:0/129206920 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7a24078bc0 msgr2=0x7f7a2407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:13.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.020+0000 7f7a4e200640 1 --2- 172.21.15.100:0/129206920 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7a24078bc0 0x7f7a2407b0d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7a38009ab0 tx=0x7f7a38005f90 comp rx=0 tx=0).stop 2024-09-17T04:24:13.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.020+0000 7f7a4e200640 1 -- 172.21.15.100:0/129206920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a4815fd30 msgr2=0x7f7a481f8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:13.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.020+0000 7f7a4e200640 1 --2- 172.21.15.100:0/129206920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a4815fd30 0x7f7a481f8f60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7a30034040 tx=0x7f7a30003820 comp rx=0 tx=0).stop 2024-09-17T04:24:13.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.020+0000 7f7a4e200640 1 -- 172.21.15.100:0/129206920 shutdown_connections 2024-09-17T04:24:13.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.020+0000 7f7a4e200640 1 --2- 172.21.15.100:0/129206920 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7a24078bc0 0x7f7a2407b0d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:13.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.020+0000 7f7a4e200640 1 --2- 172.21.15.100:0/129206920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a48162ff0 0x7f7a480742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:13.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.020+0000 7f7a4e200640 1 --2- 172.21.15.100:0/129206920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a4815fd30 0x7f7a481f8f60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:13.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.020+0000 7f7a4e200640 1 --2- 172.21.15.100:0/129206920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a4806cda0 0x7f7a481f64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:13.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.020+0000 7f7a4e200640 1 -- 172.21.15.100:0/129206920 >> 172.21.15.100:0/129206920 conn(0x7f7a480b2540 msgr2=0x7f7a480ad0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:13.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.020+0000 7f7a4e200640 1 -- 172.21.15.100:0/129206920 shutdown_connections 2024-09-17T04:24:13.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.020+0000 7f7a4e200640 1 -- 172.21.15.100:0/129206920 wait complete. 2024-09-17T04:24:13.045 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T04:24:13.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.275+0000 7f3550200640 1 -- 172.21.15.100:0/2519471578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3530005150 msgr2=0x7f353000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:13.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.275+0000 7f3550200640 1 --2- 172.21.15.100:0/2519471578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3530005150 0x7f353000b9c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3540005fd0 tx=0x7f3530005830 comp rx=0 tx=0).stop 2024-09-17T04:24:13.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.275+0000 7f354e200640 1 -- 172.21.15.100:0/2519471578 reap_dead start 2024-09-17T04:24:13.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.275+0000 7f3550200640 1 -- 172.21.15.100:0/2519471578 shutdown_connections 2024-09-17T04:24:13.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.275+0000 7f3550200640 1 -- 172.21.15.100:0/2519471578 >> 172.21.15.100:0/2519471578 conn(0x7f354806d1c0 msgr2=0x7f354806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:13.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.275+0000 7f3550200640 1 -- 172.21.15.100:0/2519471578 shutdown_connections 2024-09-17T04:24:13.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.275+0000 7f3550200640 1 -- 172.21.15.100:0/2519471578 wait complete. 2024-09-17T04:24:13.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.278+0000 7f3550200640 1 Processor -- start 2024-09-17T04:24:13.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.278+0000 7f3550200640 1 -- start start 2024-09-17T04:24:13.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.279+0000 7f3550200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3530005150 0x7f354815bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:13.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.279+0000 7f3550200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f354813e310 0x7f35480b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:13.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.279+0000 7f3550200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35480b49f0 0x7f354813ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:13.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.279+0000 7f3550200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f35480cdcd0 con 0x7f35480b49f0 2024-09-17T04:24:13.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.279+0000 7f3550200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f35480cdb50 con 0x7f3530005150 2024-09-17T04:24:13.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.279+0000 7f3550200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f35480cde50 con 0x7f354813e310 2024-09-17T04:24:13.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.280+0000 7f354e200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35480b49f0 0x7f354813ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:13.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.280+0000 7f354e200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35480b49f0 0x7f354813ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45124/0 (socket says 172.21.15.100:45124) 2024-09-17T04:24:13.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.280+0000 7f354e200640 1 -- 172.21.15.100:0/2028745296 learned_addr learned my addr 172.21.15.100:0/2028745296 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:13.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.280+0000 7f354d800640 1 --2- 172.21.15.100:0/2028745296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3530005150 0x7f354815bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:13.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.280+0000 7f354ce00640 1 --2- 172.21.15.100:0/2028745296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f354813e310 0x7f35480b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:13.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.280+0000 7f354e200640 1 -- 172.21.15.100:0/2028745296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f354813e310 msgr2=0x7f35480b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:13.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.280+0000 7f354e200640 1 --2- 172.21.15.100:0/2028745296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f354813e310 0x7f35480b44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:13.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.280+0000 7f354e200640 1 -- 172.21.15.100:0/2028745296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3530005150 msgr2=0x7f354815bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:13.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.280+0000 7f354e200640 1 --2- 172.21.15.100:0/2028745296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3530005150 0x7f354815bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:13.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.280+0000 7f354e200640 1 -- 172.21.15.100:0/2028745296 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354813c670 con 0x7f35480b49f0 2024-09-17T04:24:13.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.280+0000 7f354e200640 1 --2- 172.21.15.100:0/2028745296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35480b49f0 0x7f354813ce10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f354002ba00 tx=0x7f3540032bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:13.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.280+0000 7f3545600640 1 -- 172.21.15.100:0/2028745296 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3540036030 con 0x7f35480b49f0 2024-09-17T04:24:13.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.281+0000 7f3545600640 1 -- 172.21.15.100:0/2028745296 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3540043070 con 0x7f35480b49f0 2024-09-17T04:24:13.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.281+0000 7f3550200640 1 -- 172.21.15.100:0/2028745296 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35480b4e00 con 0x7f35480b49f0 2024-09-17T04:24:13.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.281+0000 7f3545600640 1 -- 172.21.15.100:0/2028745296 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354003c8d0 con 0x7f35480b49f0 2024-09-17T04:24:13.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.281+0000 7f3550200640 1 -- 172.21.15.100:0/2028745296 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f354815c400 con 0x7f35480b49f0 2024-09-17T04:24:13.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.282+0000 7f3550200640 1 -- 172.21.15.100:0/2028745296 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35480cc820 con 0x7f35480b49f0 2024-09-17T04:24:13.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.283+0000 7f3545600640 1 -- 172.21.15.100:0/2028745296 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f354003ca70 con 0x7f35480b49f0 2024-09-17T04:24:13.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.285+0000 7f3545600640 1 --2- 172.21.15.100:0/2028745296 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3528078bc0 0x7f352807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:13.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.285+0000 7f3545600640 1 -- 172.21.15.100:0/2028745296 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5227+0+0 (secure 0 0 0) 0x7f35400bc5c0 con 0x7f35480b49f0 2024-09-17T04:24:13.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.285+0000 7f354d800640 1 --2- 172.21.15.100:0/2028745296 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3528078bc0 0x7f352807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:13.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.286+0000 7f354d800640 1 --2- 172.21.15.100:0/2028745296 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3528078bc0 0x7f352807b0d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f353c000fb0 tx=0x7f353c0011c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:13.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.290+0000 7f3545600640 1 -- 172.21.15.100:0/2028745296 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f35480cc820 con 0x7f35480b49f0 2024-09-17T04:24:13.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:13.474+0000 7f3550200640 1 -- 172.21.15.100:0/2028745296 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f35480cd610 con 0x7f35480b49f0 2024-09-17T04:24:14.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.026+0000 7f3545600640 1 -- 172.21.15.100:0/2028745296 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) ==== 170+0+0 (secure 0 0 0) 0x7f35480b4e00 con 0x7f35480b49f0 2024-09-17T04:24:14.026 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' created 2024-09-17T04:24:14.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.029+0000 7f3550200640 1 -- 172.21.15.100:0/2028745296 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3528078bc0 msgr2=0x7f352807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:14.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.029+0000 7f3550200640 1 --2- 172.21.15.100:0/2028745296 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3528078bc0 0x7f352807b0d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f353c000fb0 tx=0x7f353c0011c0 comp rx=0 tx=0).stop 2024-09-17T04:24:14.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.029+0000 7f3550200640 1 -- 172.21.15.100:0/2028745296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35480b49f0 msgr2=0x7f354813ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:14.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.029+0000 7f3550200640 1 --2- 172.21.15.100:0/2028745296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35480b49f0 0x7f354813ce10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f354002ba00 tx=0x7f3540032bd0 comp rx=0 tx=0).stop 2024-09-17T04:24:14.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.029+0000 7f3550200640 1 -- 172.21.15.100:0/2028745296 shutdown_connections 2024-09-17T04:24:14.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.029+0000 7f3550200640 1 --2- 172.21.15.100:0/2028745296 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3528078bc0 0x7f352807b0d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:14.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.029+0000 7f3550200640 1 --2- 172.21.15.100:0/2028745296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35480b49f0 0x7f354813ce10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:14.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.029+0000 7f3550200640 1 --2- 172.21.15.100:0/2028745296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f354813e310 0x7f35480b44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:14.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.029+0000 7f3550200640 1 --2- 172.21.15.100:0/2028745296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3530005150 0x7f354815bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:14.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.029+0000 7f3550200640 1 -- 172.21.15.100:0/2028745296 >> 172.21.15.100:0/2028745296 conn(0x7f354806d1c0 msgr2=0x7f35480b2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:14.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.029+0000 7f3550200640 1 -- 172.21.15.100:0/2028745296 shutdown_connections 2024-09-17T04:24:14.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.029+0000 7f3550200640 1 -- 172.21.15.100:0/2028745296 wait complete. 2024-09-17T04:24:14.073 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T04:24:14.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.296+0000 7fe632a00640 1 -- 172.21.15.100:0/3854315671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe62c073c50 msgr2=0x7fe62c1674a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:14.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.296+0000 7fe632a00640 1 --2- 172.21.15.100:0/3854315671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe62c073c50 0x7fe62c1674a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe624005fd0 tx=0x7fe62402abc0 comp rx=0 tx=0).stop 2024-09-17T04:24:14.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.296+0000 7fe632a00640 1 -- 172.21.15.100:0/3854315671 shutdown_connections 2024-09-17T04:24:14.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.296+0000 7fe632a00640 1 --2- 172.21.15.100:0/3854315671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe62c073c50 0x7fe62c1674a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:14.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.296+0000 7fe632a00640 1 -- 172.21.15.100:0/3854315671 >> 172.21.15.100:0/3854315671 conn(0x7fe62c06d1c0 msgr2=0x7fe62c06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:14.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.296+0000 7fe632a00640 1 -- 172.21.15.100:0/3854315671 shutdown_connections 2024-09-17T04:24:14.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.296+0000 7fe632a00640 1 -- 172.21.15.100:0/3854315671 wait complete. 2024-09-17T04:24:14.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.299+0000 7fe632a00640 1 Processor -- start 2024-09-17T04:24:14.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.299+0000 7fe632a00640 1 -- start start 2024-09-17T04:24:14.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.300+0000 7fe632a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe62c145550 0x7fe62c15d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:14.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.300+0000 7fe632a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe62c145a40 0x7fe62c15f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:14.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.300+0000 7fe632a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe62c1f4070 0x7fe62c156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:14.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.300+0000 7fe632a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c167d70 con 0x7fe62c1f4070 2024-09-17T04:24:14.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.300+0000 7fe632a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c167bf0 con 0x7fe62c145a40 2024-09-17T04:24:14.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.300+0000 7fe632a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe62c167ef0 con 0x7fe62c145550 2024-09-17T04:24:14.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.300+0000 7fe630a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe62c1f4070 0x7fe62c156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:14.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.300+0000 7fe630a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe62c1f4070 0x7fe62c156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45136/0 (socket says 172.21.15.100:45136) 2024-09-17T04:24:14.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.300+0000 7fe630a00640 1 -- 172.21.15.100:0/4060813011 learned_addr learned my addr 172.21.15.100:0/4060813011 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:14.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.301+0000 7fe62be00640 1 --2- 172.21.15.100:0/4060813011 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe62c145550 0x7fe62c15d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:14.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.301+0000 7fe62b400640 1 --2- 172.21.15.100:0/4060813011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe62c145a40 0x7fe62c15f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:14.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.301+0000 7fe630a00640 1 -- 172.21.15.100:0/4060813011 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe62c145550 msgr2=0x7fe62c15d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:14.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.301+0000 7fe630a00640 1 --2- 172.21.15.100:0/4060813011 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe62c145550 0x7fe62c15d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:14.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.301+0000 7fe630a00640 1 -- 172.21.15.100:0/4060813011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe62c145a40 msgr2=0x7fe62c15f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:14.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.301+0000 7fe630a00640 1 --2- 172.21.15.100:0/4060813011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe62c145a40 0x7fe62c15f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:14.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.301+0000 7fe630a00640 1 -- 172.21.15.100:0/4060813011 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62c15cd60 con 0x7fe62c1f4070 2024-09-17T04:24:14.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.301+0000 7fe630a00640 1 --2- 172.21.15.100:0/4060813011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe62c1f4070 0x7fe62c156680 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe624009d90 tx=0x7fe6240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:14.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.301+0000 7fe623e00640 1 -- 172.21.15.100:0/4060813011 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6240046f0 con 0x7fe62c1f4070 2024-09-17T04:24:14.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.301+0000 7fe623e00640 1 -- 172.21.15.100:0/4060813011 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe624045070 con 0x7fe62c1f4070 2024-09-17T04:24:14.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.301+0000 7fe632a00640 1 -- 172.21.15.100:0/4060813011 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe62c13f030 con 0x7fe62c1f4070 2024-09-17T04:24:14.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.301+0000 7fe623e00640 1 -- 172.21.15.100:0/4060813011 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62403e6b0 con 0x7fe62c1f4070 2024-09-17T04:24:14.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.301+0000 7fe632a00640 1 -- 172.21.15.100:0/4060813011 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe62c15d300 con 0x7fe62c1f4070 2024-09-17T04:24:14.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.302+0000 7fe632a00640 1 -- 172.21.15.100:0/4060813011 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe62c143240 con 0x7fe62c1f4070 2024-09-17T04:24:14.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.304+0000 7fe623e00640 1 -- 172.21.15.100:0/4060813011 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe62403e850 con 0x7fe62c1f4070 2024-09-17T04:24:14.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.305+0000 7fe623e00640 1 --2- 172.21.15.100:0/4060813011 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe60c078bc0 0x7fe60c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:14.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.305+0000 7fe623e00640 1 -- 172.21.15.100:0/4060813011 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5609+0+0 (secure 0 0 0) 0x7fe6240bd2f0 con 0x7fe62c1f4070 2024-09-17T04:24:14.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.305+0000 7fe62be00640 1 --2- 172.21.15.100:0/4060813011 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe60c078bc0 0x7fe60c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:14.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.306+0000 7fe62be00640 1 --2- 172.21.15.100:0/4060813011 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe60c078bc0 0x7fe60c07b0d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe61c000f60 tx=0x7fe61c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:14.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.308+0000 7fe623e00640 1 -- 172.21.15.100:0/4060813011 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe62c143240 con 0x7fe62c1f4070 2024-09-17T04:24:14.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:14.526+0000 7fe632a00640 1 -- 172.21.15.100:0/4060813011 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe62c13fff0 con 0x7fe62c1f4070 2024-09-17T04:24:15.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.028+0000 7fe623e00640 1 -- 172.21.15.100:0/4060813011 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fe624085770 con 0x7fe62c1f4070 2024-09-17T04:24:15.027 INFO:teuthology.orchestra.run.smithi100.stderr:new fs with metadata pool 2 and data pool 3 2024-09-17T04:24:15.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.031+0000 7fe632a00640 1 -- 172.21.15.100:0/4060813011 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe60c078bc0 msgr2=0x7fe60c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:15.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.031+0000 7fe632a00640 1 --2- 172.21.15.100:0/4060813011 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe60c078bc0 0x7fe60c07b0d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe61c000f60 tx=0x7fe61c009040 comp rx=0 tx=0).stop 2024-09-17T04:24:15.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.031+0000 7fe632a00640 1 -- 172.21.15.100:0/4060813011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe62c1f4070 msgr2=0x7fe62c156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:15.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.031+0000 7fe632a00640 1 --2- 172.21.15.100:0/4060813011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe62c1f4070 0x7fe62c156680 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe624009d90 tx=0x7fe6240042f0 comp rx=0 tx=0).stop 2024-09-17T04:24:15.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.031+0000 7fe632a00640 1 -- 172.21.15.100:0/4060813011 shutdown_connections 2024-09-17T04:24:15.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.031+0000 7fe632a00640 1 --2- 172.21.15.100:0/4060813011 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe60c078bc0 0x7fe60c07b0d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:15.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.032+0000 7fe632a00640 1 --2- 172.21.15.100:0/4060813011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe62c1f4070 0x7fe62c156680 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:15.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.032+0000 7fe632a00640 1 --2- 172.21.15.100:0/4060813011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe62c145a40 0x7fe62c15f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:15.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.032+0000 7fe632a00640 1 --2- 172.21.15.100:0/4060813011 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe62c145550 0x7fe62c15d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:15.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.032+0000 7fe632a00640 1 -- 172.21.15.100:0/4060813011 >> 172.21.15.100:0/4060813011 conn(0x7fe62c06d1c0 msgr2=0x7fe62c141aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:15.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.032+0000 7fe632a00640 1 -- 172.21.15.100:0/4060813011 shutdown_connections 2024-09-17T04:24:15.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.032+0000 7fe632a00640 1 -- 172.21.15.100:0/4060813011 wait complete. 2024-09-17T04:24:15.073 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:24:15.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.294+0000 7f846ca00640 1 -- 172.21.15.100:0/586738237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84680aebc0 msgr2=0x7f8468152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:15.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.294+0000 7f846ca00640 1 --2- 172.21.15.100:0/586738237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84680aebc0 0x7f8468152920 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8458009830 tx=0x7f845802a480 comp rx=0 tx=0).stop 2024-09-17T04:24:15.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.294+0000 7f846ca00640 1 -- 172.21.15.100:0/586738237 shutdown_connections 2024-09-17T04:24:15.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.294+0000 7f846ca00640 1 --2- 172.21.15.100:0/586738237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f846815fae0 0x7f84680b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:15.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.294+0000 7f846ca00640 1 --2- 172.21.15.100:0/586738237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84680aebc0 0x7f8468152920 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:15.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.294+0000 7f846ca00640 1 -- 172.21.15.100:0/586738237 >> 172.21.15.100:0/586738237 conn(0x7f846806d730 msgr2=0x7f84680b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:15.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.294+0000 7f846ca00640 1 -- 172.21.15.100:0/586738237 shutdown_connections 2024-09-17T04:24:15.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.294+0000 7f846ca00640 1 -- 172.21.15.100:0/586738237 wait complete. 2024-09-17T04:24:15.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.297+0000 7f846ca00640 1 Processor -- start 2024-09-17T04:24:15.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.297+0000 7f846ca00640 1 -- start start 2024-09-17T04:24:15.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.298+0000 7f846ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84680aebc0 0x7f84681f2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:15.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.298+0000 7f846ca00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f84680b2750 0x7f84681f5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:15.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.298+0000 7f846ca00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8468158c20 0x7f8468159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:15.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.298+0000 7f846ca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f84680b4130 con 0x7f84680aebc0 2024-09-17T04:24:15.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.298+0000 7f846ca00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f84680b3fb0 con 0x7f8468158c20 2024-09-17T04:24:15.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.298+0000 7f846ca00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f84680b42b0 con 0x7f84680b2750 2024-09-17T04:24:15.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.298+0000 7f8465e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84680aebc0 0x7f84681f2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:15.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.298+0000 7f8465e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84680aebc0 0x7f84681f2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41458/0 (socket says 172.21.15.100:41458) 2024-09-17T04:24:15.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.298+0000 7f8465e00640 1 -- 172.21.15.100:0/742416888 learned_addr learned my addr 172.21.15.100:0/742416888 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:15.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.298+0000 7f8465400640 1 --2- 172.21.15.100:0/742416888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f84680b2750 0x7f84681f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:15.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.298+0000 7f8466800640 1 --2- 172.21.15.100:0/742416888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8468158c20 0x7f8468159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:15.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.299+0000 7f8465e00640 1 -- 172.21.15.100:0/742416888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f84680b2750 msgr2=0x7f84681f5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:15.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.299+0000 7f8465e00640 1 --2- 172.21.15.100:0/742416888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f84680b2750 0x7f84681f5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:15.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.299+0000 7f8465e00640 1 -- 172.21.15.100:0/742416888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8468158c20 msgr2=0x7f8468159030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:15.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.299+0000 7f8465e00640 1 --2- 172.21.15.100:0/742416888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8468158c20 0x7f8468159030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:15.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.299+0000 7f8465e00640 1 -- 172.21.15.100:0/742416888 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8468159570 con 0x7f84680aebc0 2024-09-17T04:24:15.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.299+0000 7f8466800640 1 --2- 172.21.15.100:0/742416888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8468158c20 0x7f8468159030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:24:15.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.299+0000 7f8465e00640 1 --2- 172.21.15.100:0/742416888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84680aebc0 0x7f84681f2a10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8458000c00 tx=0x7f8458033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:15.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.299+0000 7f845e000640 1 -- 172.21.15.100:0/742416888 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8458005c90 con 0x7f84680aebc0 2024-09-17T04:24:15.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.299+0000 7f845e000640 1 -- 172.21.15.100:0/742416888 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8458031e50 con 0x7f84680aebc0 2024-09-17T04:24:15.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.299+0000 7f845e000640 1 -- 172.21.15.100:0/742416888 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84580332f0 con 0x7f84680aebc0 2024-09-17T04:24:15.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.299+0000 7f846ca00640 1 -- 172.21.15.100:0/742416888 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84681f64e0 con 0x7f84680aebc0 2024-09-17T04:24:15.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.299+0000 7f846ca00640 1 -- 172.21.15.100:0/742416888 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8468073660 con 0x7f84680aebc0 2024-09-17T04:24:15.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.300+0000 7f846ca00640 1 -- 172.21.15.100:0/742416888 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84680b4760 con 0x7f84680aebc0 2024-09-17T04:24:15.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.302+0000 7f845e000640 1 -- 172.21.15.100:0/742416888 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8458033490 con 0x7f84680aebc0 2024-09-17T04:24:15.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.303+0000 7f845e000640 1 --2- 172.21.15.100:0/742416888 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8448078bc0 0x7f844807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:15.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.303+0000 7f845e000640 1 -- 172.21.15.100:0/742416888 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f84580bc1a0 con 0x7f84680aebc0 2024-09-17T04:24:15.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.303+0000 7f8465400640 1 --2- 172.21.15.100:0/742416888 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8448078bc0 0x7f844807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:15.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.304+0000 7f8465400640 1 --2- 172.21.15.100:0/742416888 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8448078bc0 0x7f844807b0d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8454000f60 tx=0x7f8454009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:15.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.307+0000 7f845e000640 1 -- 172.21.15.100:0/742416888 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8468159570 con 0x7f84680aebc0 2024-09-17T04:24:15.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.490+0000 7f846ca00640 1 -- 172.21.15.100:0/742416888 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f84680719e0 con 0x7f84680aebc0 2024-09-17T04:24:15.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.491+0000 7f845e000640 1 -- 172.21.15.100:0/742416888 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16726 (secure 0 0 0) 0x7f845803f030 con 0x7f84680aebc0 2024-09-17T04:24:15.491 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:15.491 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":22,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:24:14.527327+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:24:12.826665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T04:24:13.485446+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:24:15.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.494+0000 7f846ca00640 1 -- 172.21.15.100:0/742416888 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8448078bc0 msgr2=0x7f844807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:15.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.494+0000 7f846ca00640 1 --2- 172.21.15.100:0/742416888 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8448078bc0 0x7f844807b0d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8454000f60 tx=0x7f8454009040 comp rx=0 tx=0).stop 2024-09-17T04:24:15.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.494+0000 7f846ca00640 1 -- 172.21.15.100:0/742416888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84680aebc0 msgr2=0x7f84681f2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:15.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.494+0000 7f846ca00640 1 --2- 172.21.15.100:0/742416888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84680aebc0 0x7f84681f2a10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8458000c00 tx=0x7f8458033c90 comp rx=0 tx=0).stop 2024-09-17T04:24:15.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.494+0000 7f846ca00640 1 -- 172.21.15.100:0/742416888 shutdown_connections 2024-09-17T04:24:15.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.494+0000 7f846ca00640 1 --2- 172.21.15.100:0/742416888 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8448078bc0 0x7f844807b0d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:15.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.494+0000 7f846ca00640 1 --2- 172.21.15.100:0/742416888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8468158c20 0x7f8468159030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:15.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.494+0000 7f846ca00640 1 --2- 172.21.15.100:0/742416888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f84680b2750 0x7f84681f5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:15.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.494+0000 7f846ca00640 1 --2- 172.21.15.100:0/742416888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84680aebc0 0x7f84681f2a10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:15.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.494+0000 7f846ca00640 1 -- 172.21.15.100:0/742416888 >> 172.21.15.100:0/742416888 conn(0x7f846806d730 msgr2=0x7f846806f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:15.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.494+0000 7f846ca00640 1 -- 172.21.15.100:0/742416888 shutdown_connections 2024-09-17T04:24:15.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.494+0000 7f846ca00640 1 -- 172.21.15.100:0/742416888 wait complete. 2024-09-17T04:24:15.517 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:24:15.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.737+0000 7f6da5200640 1 -- 172.21.15.100:0/2613301799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6da00ad0e0 msgr2=0x7f6da00ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:15.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.737+0000 7f6da5200640 1 --2- 172.21.15.100:0/2613301799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6da00ad0e0 0x7f6da00ad4f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6d88005fd0 tx=0x7f6d8802a540 comp rx=0 tx=0).stop 2024-09-17T04:24:15.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.737+0000 7f6da5200640 1 -- 172.21.15.100:0/2613301799 shutdown_connections 2024-09-17T04:24:15.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.737+0000 7f6da5200640 1 --2- 172.21.15.100:0/2613301799 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6da00b3550 0x7f6da00b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:15.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.737+0000 7f6da5200640 1 --2- 172.21.15.100:0/2613301799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6da00ad0e0 0x7f6da00ad4f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:15.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.737+0000 7f6da5200640 1 -- 172.21.15.100:0/2613301799 >> 172.21.15.100:0/2613301799 conn(0x7f6da006d730 msgr2=0x7f6da00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:15.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.737+0000 7f6da5200640 1 -- 172.21.15.100:0/2613301799 shutdown_connections 2024-09-17T04:24:15.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.737+0000 7f6da5200640 1 -- 172.21.15.100:0/2613301799 wait complete. 2024-09-17T04:24:15.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.739+0000 7f6da5200640 1 Processor -- start 2024-09-17T04:24:15.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.739+0000 7f6da5200640 1 -- start start 2024-09-17T04:24:15.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.741+0000 7f6da5200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6da00ad0e0 0x7f6da0155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:15.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.741+0000 7f6da5200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6da00b3550 0x7f6da0151450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:15.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.741+0000 7f6da5200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6da015d760 0x7f6da01f8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:15.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.741+0000 7f6da5200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6da00b4da0 con 0x7f6da00b3550 2024-09-17T04:24:15.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.741+0000 7f6da5200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6da00b4c20 con 0x7f6da00ad0e0 2024-09-17T04:24:15.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.741+0000 7f6da5200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6da00b3ea0 con 0x7f6da015d760 2024-09-17T04:24:15.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.741+0000 7f6d95400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6da00b3550 0x7f6da0151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:15.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.741+0000 7f6d95400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6da00b3550 0x7f6da0151450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41480/0 (socket says 172.21.15.100:41480) 2024-09-17T04:24:15.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.741+0000 7f6d95400640 1 -- 172.21.15.100:0/676552533 learned_addr learned my addr 172.21.15.100:0/676552533 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:15.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.741+0000 7f6d9de00640 1 --2- 172.21.15.100:0/676552533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6da00ad0e0 0x7f6da0155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:15.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.741+0000 7f6d9e800640 1 --2- 172.21.15.100:0/676552533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6da015d760 0x7f6da01f8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:15.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.742+0000 7f6d95400640 1 -- 172.21.15.100:0/676552533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6da015d760 msgr2=0x7f6da01f8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:15.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.742+0000 7f6d95400640 1 --2- 172.21.15.100:0/676552533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6da015d760 0x7f6da01f8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:15.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.742+0000 7f6d95400640 1 -- 172.21.15.100:0/676552533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6da00ad0e0 msgr2=0x7f6da0155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:15.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.742+0000 7f6d95400640 1 --2- 172.21.15.100:0/676552533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6da00ad0e0 0x7f6da0155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:15.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.742+0000 7f6d95400640 1 -- 172.21.15.100:0/676552533 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da009a450 con 0x7f6da00b3550 2024-09-17T04:24:15.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.742+0000 7f6d9de00640 1 --2- 172.21.15.100:0/676552533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6da00ad0e0 0x7f6da0155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:24:15.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.742+0000 7f6d95400640 1 --2- 172.21.15.100:0/676552533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6da00b3550 0x7f6da0151450 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6d88034040 tx=0x7f6d88003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:15.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.742+0000 7f6d96a00640 1 -- 172.21.15.100:0/676552533 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8803e030 con 0x7f6da00b3550 2024-09-17T04:24:15.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.742+0000 7f6d96a00640 1 -- 172.21.15.100:0/676552533 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d88003e00 con 0x7f6da00b3550 2024-09-17T04:24:15.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.742+0000 7f6d96a00640 1 -- 172.21.15.100:0/676552533 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d88035040 con 0x7f6da00b3550 2024-09-17T04:24:15.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.742+0000 7f6da5200640 1 -- 172.21.15.100:0/676552533 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6da01563b0 con 0x7f6da00b3550 2024-09-17T04:24:15.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.742+0000 7f6da5200640 1 -- 172.21.15.100:0/676552533 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6da009a710 con 0x7f6da00b3550 2024-09-17T04:24:15.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.744+0000 7f6da5200640 1 -- 172.21.15.100:0/676552533 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6da009dd70 con 0x7f6da00b3550 2024-09-17T04:24:15.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.745+0000 7f6d96a00640 1 -- 172.21.15.100:0/676552533 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6d88041030 con 0x7f6da00b3550 2024-09-17T04:24:15.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.746+0000 7f6d96a00640 1 --2- 172.21.15.100:0/676552533 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6d7c078b90 0x7f6d7c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:15.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.746+0000 7f6d96a00640 1 -- 172.21.15.100:0/676552533 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6d880bb500 con 0x7f6da00b3550 2024-09-17T04:24:15.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.746+0000 7f6d9de00640 1 --2- 172.21.15.100:0/676552533 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6d7c078b90 0x7f6d7c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:15.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.747+0000 7f6d9de00640 1 --2- 172.21.15.100:0/676552533 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6d7c078b90 0x7f6d7c07b0a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6d90000f60 tx=0x7f6d90009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:15.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.751+0000 7f6d96a00640 1 -- 172.21.15.100:0/676552533 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6d880832e0 con 0x7f6da00b3550 2024-09-17T04:24:15.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.935+0000 7f6da5200640 1 -- 172.21.15.100:0/676552533 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6da00b4370 con 0x7f6da00b3550 2024-09-17T04:24:15.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.936+0000 7f6d96a00640 1 -- 172.21.15.100:0/676552533 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16726 (secure 0 0 0) 0x7f6d88087d10 con 0x7f6da00b3550 2024-09-17T04:24:15.937 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:15.937 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":22,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:24:14.527327+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:24:12.826665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T04:24:13.485446+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:24:15.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.938+0000 7f6da5200640 1 -- 172.21.15.100:0/676552533 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6d7c078b90 msgr2=0x7f6d7c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:15.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.938+0000 7f6da5200640 1 --2- 172.21.15.100:0/676552533 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6d7c078b90 0x7f6d7c07b0a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6d90000f60 tx=0x7f6d90009040 comp rx=0 tx=0).stop 2024-09-17T04:24:15.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.938+0000 7f6da5200640 1 -- 172.21.15.100:0/676552533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6da00b3550 msgr2=0x7f6da0151450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:15.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.938+0000 7f6da5200640 1 --2- 172.21.15.100:0/676552533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6da00b3550 0x7f6da0151450 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6d88034040 tx=0x7f6d88003910 comp rx=0 tx=0).stop 2024-09-17T04:24:15.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.939+0000 7f6da5200640 1 -- 172.21.15.100:0/676552533 shutdown_connections 2024-09-17T04:24:15.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.939+0000 7f6da5200640 1 --2- 172.21.15.100:0/676552533 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6d7c078b90 0x7f6d7c07b0a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:15.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.939+0000 7f6da5200640 1 --2- 172.21.15.100:0/676552533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6da015d760 0x7f6da01f8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:15.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.939+0000 7f6da5200640 1 --2- 172.21.15.100:0/676552533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6da00b3550 0x7f6da0151450 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:15.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.939+0000 7f6da5200640 1 --2- 172.21.15.100:0/676552533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6da00ad0e0 0x7f6da0155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:15.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.939+0000 7f6da5200640 1 -- 172.21.15.100:0/676552533 >> 172.21.15.100:0/676552533 conn(0x7f6da006d730 msgr2=0x7f6da01535c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:15.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.939+0000 7f6da5200640 1 -- 172.21.15.100:0/676552533 shutdown_connections 2024-09-17T04:24:15.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:15.939+0000 7f6da5200640 1 -- 172.21.15.100:0/676552533 wait complete. 2024-09-17T04:24:15.962 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T04:24:16.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.190+0000 7f429b200640 1 -- 172.21.15.100:0/1806514742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f429c0b3a50 msgr2=0x7f429c0cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:16.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.190+0000 7f429b200640 1 --2- 172.21.15.100:0/1806514742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f429c0b3a50 0x7f429c0cedb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4288009830 tx=0x7f428802a510 comp rx=0 tx=0).stop 2024-09-17T04:24:16.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.190+0000 7f429b200640 1 -- 172.21.15.100:0/1806514742 shutdown_connections 2024-09-17T04:24:16.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.190+0000 7f429b200640 1 --2- 172.21.15.100:0/1806514742 >> v2:172.21.15.184:3300/0 conn(0x7f429c0cf2f0 0x7f429c0d4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:16.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.190+0000 7f429b200640 1 --2- 172.21.15.100:0/1806514742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f429c0b3a50 0x7f429c0cedb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:16.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.190+0000 7f429b200640 1 -- 172.21.15.100:0/1806514742 >> 172.21.15.100:0/1806514742 conn(0x7f429c09d940 msgr2=0x7f429c08bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:16.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.190+0000 7f429b200640 1 -- 172.21.15.100:0/1806514742 shutdown_connections 2024-09-17T04:24:16.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.190+0000 7f429b200640 1 -- 172.21.15.100:0/1806514742 wait complete. 2024-09-17T04:24:16.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.192+0000 7f429b200640 1 Processor -- start 2024-09-17T04:24:16.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.192+0000 7f429b200640 1 -- start start 2024-09-17T04:24:16.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.194+0000 7f429b200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f429c0b3a50 0x7f429c0b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:16.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.194+0000 7f429b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f429c0b9660 0x7f429c0b7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:16.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.194+0000 7f429b200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f429c0cf2f0 0x7f429c1f4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:16.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.194+0000 7f429b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f429c06f3a0 con 0x7f429c0b9660 2024-09-17T04:24:16.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.194+0000 7f429b200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f429c06f220 con 0x7f429c0cf2f0 2024-09-17T04:24:16.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.194+0000 7f429b200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f429c06f520 con 0x7f429c0b3a50 2024-09-17T04:24:16.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.194+0000 7f4299400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f429c0b9660 0x7f429c0b7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:16.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.194+0000 7f4299400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f429c0b9660 0x7f429c0b7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41486/0 (socket says 172.21.15.100:41486) 2024-09-17T04:24:16.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.194+0000 7f4299400640 1 -- 172.21.15.100:0/522040831 learned_addr learned my addr 172.21.15.100:0/522040831 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:16.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.194+0000 7f4299e00640 1 --2- 172.21.15.100:0/522040831 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f429c0b3a50 0x7f429c0b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:16.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.194+0000 7f429a800640 1 --2- 172.21.15.100:0/522040831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f429c0cf2f0 0x7f429c1f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:16.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.195+0000 7f4299400640 1 -- 172.21.15.100:0/522040831 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f429c0b3a50 msgr2=0x7f429c0b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:16.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.195+0000 7f4299400640 1 --2- 172.21.15.100:0/522040831 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f429c0b3a50 0x7f429c0b50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:16.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.195+0000 7f4299400640 1 -- 172.21.15.100:0/522040831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f429c0cf2f0 msgr2=0x7f429c1f4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:16.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.195+0000 7f4299400640 1 --2- 172.21.15.100:0/522040831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f429c0cf2f0 0x7f429c1f4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:16.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.195+0000 7f4299400640 1 -- 172.21.15.100:0/522040831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f429c0b58d0 con 0x7f429c0b9660 2024-09-17T04:24:16.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.195+0000 7f4299e00640 1 --2- 172.21.15.100:0/522040831 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f429c0b3a50 0x7f429c0b50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:24:16.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.195+0000 7f4299400640 1 --2- 172.21.15.100:0/522040831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f429c0b9660 0x7f429c0b7bc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4288000c00 tx=0x7f4288033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:16.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.195+0000 7f4292000640 1 -- 172.21.15.100:0/522040831 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4288031d60 con 0x7f429c0b9660 2024-09-17T04:24:16.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.195+0000 7f4292000640 1 -- 172.21.15.100:0/522040831 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4288033e10 con 0x7f429c0b9660 2024-09-17T04:24:16.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.195+0000 7f4292000640 1 -- 172.21.15.100:0/522040831 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4288033310 con 0x7f429c0b9660 2024-09-17T04:24:16.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.195+0000 7f429b200640 1 -- 172.21.15.100:0/522040831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f429c0b5bb0 con 0x7f429c0b9660 2024-09-17T04:24:16.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.195+0000 7f429b200640 1 -- 172.21.15.100:0/522040831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f429c14a660 con 0x7f429c0b9660 2024-09-17T04:24:16.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.196+0000 7f429b200640 1 -- 172.21.15.100:0/522040831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f429c06f9f0 con 0x7f429c0b9660 2024-09-17T04:24:16.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.198+0000 7f4292000640 1 -- 172.21.15.100:0/522040831 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f42880334b0 con 0x7f429c0b9660 2024-09-17T04:24:16.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.199+0000 7f4292000640 1 --2- 172.21.15.100:0/522040831 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f427c078b90 0x7f427c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:16.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.199+0000 7f4292000640 1 -- 172.21.15.100:0/522040831 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f42880bc070 con 0x7f429c0b9660 2024-09-17T04:24:16.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.199+0000 7f4299e00640 1 --2- 172.21.15.100:0/522040831 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f427c078b90 0x7f427c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:16.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.200+0000 7f4299e00640 1 --2- 172.21.15.100:0/522040831 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f427c078b90 0x7f427c07b0a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f428c007fd0 tx=0x7f428c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:16.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.203+0000 7f4292000640 1 -- 172.21.15.100:0/522040831 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f428803f030 con 0x7f429c0b9660 2024-09-17T04:24:16.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:16.424+0000 7f429b200640 1 -- 172.21.15.100:0/522040831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f429c0b5660 con 0x7f429c0b9660 2024-09-17T04:24:17.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.057+0000 7f4292000640 1 -- 172.21.15.100:0/522040831 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f4288083e50 con 0x7f429c0b9660 2024-09-17T04:24:17.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.060+0000 7f429b200640 1 -- 172.21.15.100:0/522040831 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f427c078b90 msgr2=0x7f427c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:17.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.061+0000 7f429b200640 1 --2- 172.21.15.100:0/522040831 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f427c078b90 0x7f427c07b0a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f428c007fd0 tx=0x7f428c009040 comp rx=0 tx=0).stop 2024-09-17T04:24:17.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.061+0000 7f429b200640 1 -- 172.21.15.100:0/522040831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f429c0b9660 msgr2=0x7f429c0b7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:17.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.061+0000 7f429b200640 1 --2- 172.21.15.100:0/522040831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f429c0b9660 0x7f429c0b7bc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4288000c00 tx=0x7f4288033c20 comp rx=0 tx=0).stop 2024-09-17T04:24:17.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.061+0000 7f429b200640 1 -- 172.21.15.100:0/522040831 shutdown_connections 2024-09-17T04:24:17.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.061+0000 7f429b200640 1 --2- 172.21.15.100:0/522040831 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f427c078b90 0x7f427c07b0a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:17.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.061+0000 7f429b200640 1 --2- 172.21.15.100:0/522040831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f429c0cf2f0 0x7f429c1f4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:17.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.061+0000 7f429b200640 1 --2- 172.21.15.100:0/522040831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f429c0b9660 0x7f429c0b7bc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:17.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.061+0000 7f429b200640 1 --2- 172.21.15.100:0/522040831 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f429c0b3a50 0x7f429c0b50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:17.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.061+0000 7f429b200640 1 -- 172.21.15.100:0/522040831 >> 172.21.15.100:0/522040831 conn(0x7f429c09d940 msgr2=0x7f429c1587d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:17.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.061+0000 7f429b200640 1 -- 172.21.15.100:0/522040831 shutdown_connections 2024-09-17T04:24:17.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.061+0000 7f429b200640 1 -- 172.21.15.100:0/522040831 wait complete. 2024-09-17T04:24:17.083 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T04:24:17.083 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T04:24:17.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.308+0000 7fb5db800640 1 -- 172.21.15.100:0/1351418082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5d415f9e0 msgr2=0x7fb5d4147b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:17.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.308+0000 7fb5db800640 1 --2- 172.21.15.100:0/1351418082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5d415f9e0 0x7fb5d4147b90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4072ef0 tx=0x7fb5c802a4d0 comp rx=0 tx=0).stop 2024-09-17T04:24:17.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.308+0000 7fb5db800640 1 -- 172.21.15.100:0/1351418082 shutdown_connections 2024-09-17T04:24:17.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.308+0000 7fb5db800640 1 --2- 172.21.15.100:0/1351418082 >> v2:172.21.15.184:3300/0 conn(0x7fb5d4073c50 0x7fb5d40af210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:17.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.308+0000 7fb5db800640 1 --2- 172.21.15.100:0/1351418082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5d415f9e0 0x7fb5d4147b90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:17.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.308+0000 7fb5db800640 1 -- 172.21.15.100:0/1351418082 >> 172.21.15.100:0/1351418082 conn(0x7fb5d406d1c0 msgr2=0x7fb5d406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:17.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.309+0000 7fb5db800640 1 -- 172.21.15.100:0/1351418082 shutdown_connections 2024-09-17T04:24:17.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.309+0000 7fb5db800640 1 -- 172.21.15.100:0/1351418082 wait complete. 2024-09-17T04:24:17.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.311+0000 7fb5db800640 1 Processor -- start 2024-09-17T04:24:17.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.311+0000 7fb5db800640 1 -- start start 2024-09-17T04:24:17.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.312+0000 7fb5db800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5d40aee00 0x7fb5d41f6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:17.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.312+0000 7fb5db800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5d415f9e0 0x7fb5d413edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:17.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.312+0000 7fb5db800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5d40c5a70 0x7fb5d4144f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:17.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.312+0000 7fb5db800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d406f9e0 con 0x7fb5d40c5a70 2024-09-17T04:24:17.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.312+0000 7fb5db800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d41484e0 con 0x7fb5d415f9e0 2024-09-17T04:24:17.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.312+0000 7fb5db800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb5d406fb60 con 0x7fb5d40aee00 2024-09-17T04:24:17.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.313+0000 7fb5d9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5d40c5a70 0x7fb5d4144f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:17.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.313+0000 7fb5d9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5d40c5a70 0x7fb5d4144f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41502/0 (socket says 172.21.15.100:41502) 2024-09-17T04:24:17.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.313+0000 7fb5d9800640 1 -- 172.21.15.100:0/151307186 learned_addr learned my addr 172.21.15.100:0/151307186 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:17.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.313+0000 7fb5d8e00640 1 --2- 172.21.15.100:0/151307186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5d40aee00 0x7fb5d41f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:17.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.313+0000 7fb5d3e00640 1 --2- 172.21.15.100:0/151307186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5d415f9e0 0x7fb5d413edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:17.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.313+0000 7fb5d9800640 1 -- 172.21.15.100:0/151307186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5d40aee00 msgr2=0x7fb5d41f6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:17.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.313+0000 7fb5d9800640 1 --2- 172.21.15.100:0/151307186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5d40aee00 0x7fb5d41f6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:17.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.313+0000 7fb5d9800640 1 -- 172.21.15.100:0/151307186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5d415f9e0 msgr2=0x7fb5d413edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:17.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.313+0000 7fb5d9800640 1 --2- 172.21.15.100:0/151307186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5d415f9e0 0x7fb5d413edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:17.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.313+0000 7fb5d9800640 1 -- 172.21.15.100:0/151307186 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d41f8920 con 0x7fb5d40c5a70 2024-09-17T04:24:17.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.313+0000 7fb5d8e00640 1 --2- 172.21.15.100:0/151307186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5d40aee00 0x7fb5d41f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:24:17.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.313+0000 7fb5d9800640 1 --2- 172.21.15.100:0/151307186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5d40c5a70 0x7fb5d4144f80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc00e960 tx=0x7fb5cc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:17.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.313+0000 7fb5d0c00640 1 -- 172.21.15.100:0/151307186 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc004d20 con 0x7fb5d40c5a70 2024-09-17T04:24:17.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.313+0000 7fb5d0c00640 1 -- 172.21.15.100:0/151307186 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5cc00cb20 con 0x7fb5d40c5a70 2024-09-17T04:24:17.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.313+0000 7fb5d0c00640 1 -- 172.21.15.100:0/151307186 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc005020 con 0x7fb5d40c5a70 2024-09-17T04:24:17.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.313+0000 7fb5db800640 1 -- 172.21.15.100:0/151307186 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d41f6800 con 0x7fb5d40c5a70 2024-09-17T04:24:17.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.314+0000 7fb5db800640 1 -- 172.21.15.100:0/151307186 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d41f6dc0 con 0x7fb5d40c5a70 2024-09-17T04:24:17.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.315+0000 7fb5db800640 1 -- 172.21.15.100:0/151307186 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5d4070050 con 0x7fb5d40c5a70 2024-09-17T04:24:17.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.316+0000 7fb5d0c00640 1 -- 172.21.15.100:0/151307186 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb5cc0051c0 con 0x7fb5d40c5a70 2024-09-17T04:24:17.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.317+0000 7fb5d0c00640 1 --2- 172.21.15.100:0/151307186 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb5b4078bc0 0x7fb5b407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:17.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.317+0000 7fb5d0c00640 1 -- 172.21.15.100:0/151307186 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb5cc09b700 con 0x7fb5d40c5a70 2024-09-17T04:24:17.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.317+0000 7fb5d8e00640 1 --2- 172.21.15.100:0/151307186 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb5b4078bc0 0x7fb5b407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:17.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.318+0000 7fb5d8e00640 1 --2- 172.21.15.100:0/151307186 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb5b4078bc0 0x7fb5b407b0d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8033000 tx=0x7fb5c8004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:17.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.321+0000 7fb5d0c00640 1 -- 172.21.15.100:0/151307186 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb5cc016030 con 0x7fb5d40c5a70 2024-09-17T04:24:17.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:17.541+0000 7fb5db800640 1 -- 172.21.15.100:0/151307186 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb5d41f6570 con 0x7fb5d40c5a70 2024-09-17T04:24:18.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.064+0000 7fb5d0c00640 1 -- 172.21.15.100:0/151307186 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7fb5cc063aa0 con 0x7fb5d40c5a70 2024-09-17T04:24:18.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.067+0000 7fb5db800640 1 -- 172.21.15.100:0/151307186 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb5b4078bc0 msgr2=0x7fb5b407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:18.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.067+0000 7fb5db800640 1 --2- 172.21.15.100:0/151307186 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb5b4078bc0 0x7fb5b407b0d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8033000 tx=0x7fb5c8004730 comp rx=0 tx=0).stop 2024-09-17T04:24:18.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.067+0000 7fb5db800640 1 -- 172.21.15.100:0/151307186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5d40c5a70 msgr2=0x7fb5d4144f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:18.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.067+0000 7fb5db800640 1 --2- 172.21.15.100:0/151307186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5d40c5a70 0x7fb5d4144f80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc00e960 tx=0x7fb5cc00ee20 comp rx=0 tx=0).stop 2024-09-17T04:24:18.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.067+0000 7fb5db800640 1 -- 172.21.15.100:0/151307186 shutdown_connections 2024-09-17T04:24:18.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.067+0000 7fb5db800640 1 --2- 172.21.15.100:0/151307186 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb5b4078bc0 0x7fb5b407b0d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:18.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.067+0000 7fb5db800640 1 --2- 172.21.15.100:0/151307186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5d40c5a70 0x7fb5d4144f80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:18.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.067+0000 7fb5db800640 1 --2- 172.21.15.100:0/151307186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5d415f9e0 0x7fb5d413edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:18.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.067+0000 7fb5db800640 1 --2- 172.21.15.100:0/151307186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5d40aee00 0x7fb5d41f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:18.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.067+0000 7fb5db800640 1 -- 172.21.15.100:0/151307186 >> 172.21.15.100:0/151307186 conn(0x7fb5d406d1c0 msgr2=0x7fb5d4068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:18.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.067+0000 7fb5db800640 1 -- 172.21.15.100:0/151307186 shutdown_connections 2024-09-17T04:24:18.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.067+0000 7fb5db800640 1 -- 172.21.15.100:0/151307186 wait complete. 2024-09-17T04:24:18.092 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T04:24:18.092 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:24:18.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.310+0000 7fe6bd400640 1 -- 172.21.15.100:0/2244740094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe69c004d40 msgr2=0x7fe69c005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:18.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.310+0000 7fe6bd400640 1 --2- 172.21.15.100:0/2244740094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe69c004d40 0x7fe69c005170 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0009b50 tx=0x7fe69c005830 comp rx=0 tx=0).stop 2024-09-17T04:24:18.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.310+0000 7fe6b7200640 1 -- 172.21.15.100:0/2244740094 reap_dead start 2024-09-17T04:24:18.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.310+0000 7fe6bd400640 1 -- 172.21.15.100:0/2244740094 shutdown_connections 2024-09-17T04:24:18.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.310+0000 7fe6bd400640 1 -- 172.21.15.100:0/2244740094 >> 172.21.15.100:0/2244740094 conn(0x7fe6b806d730 msgr2=0x7fe6b80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:18.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.310+0000 7fe6bd400640 1 -- 172.21.15.100:0/2244740094 shutdown_connections 2024-09-17T04:24:18.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.310+0000 7fe6bd400640 1 -- 172.21.15.100:0/2244740094 wait complete. 2024-09-17T04:24:18.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.312+0000 7fe6bd400640 1 Processor -- start 2024-09-17T04:24:18.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.312+0000 7fe6bd400640 1 -- start start 2024-09-17T04:24:18.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.314+0000 7fe6bd400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe69c004d40 0x7fe6b80999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:18.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.314+0000 7fe6bd400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6b8145b90 0x7fe6b8154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:18.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.314+0000 7fe6bd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6b8152570 0x7fe6b80b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:18.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.314+0000 7fe6bd400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b806f230 con 0x7fe6b8152570 2024-09-17T04:24:18.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.314+0000 7fe6bd400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b80b4e30 con 0x7fe6b8145b90 2024-09-17T04:24:18.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.314+0000 7fe6bd400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe6b8165890 con 0x7fe69c004d40 2024-09-17T04:24:18.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.314+0000 7fe6b7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6b8152570 0x7fe6b80b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:18.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.314+0000 7fe6b7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6b8152570 0x7fe6b80b1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41526/0 (socket says 172.21.15.100:41526) 2024-09-17T04:24:18.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.314+0000 7fe6b6800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe69c004d40 0x7fe6b80999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:18.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.314+0000 7fe6b7200640 1 -- 172.21.15.100:0/3611513273 learned_addr learned my addr 172.21.15.100:0/3611513273 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:18.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.314+0000 7fe6b5e00640 1 --2- 172.21.15.100:0/3611513273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6b8145b90 0x7fe6b8154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:18.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.314+0000 7fe6b7200640 1 -- 172.21.15.100:0/3611513273 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe69c004d40 msgr2=0x7fe6b80999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:18.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.314+0000 7fe6b7200640 1 --2- 172.21.15.100:0/3611513273 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe69c004d40 0x7fe6b80999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:18.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.314+0000 7fe6b7200640 1 -- 172.21.15.100:0/3611513273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6b8145b90 msgr2=0x7fe6b8154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:18.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.314+0000 7fe6b7200640 1 --2- 172.21.15.100:0/3611513273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6b8145b90 0x7fe6b8154ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:18.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.314+0000 7fe6b7200640 1 -- 172.21.15.100:0/3611513273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b8099ee0 con 0x7fe6b8152570 2024-09-17T04:24:18.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.315+0000 7fe6b7200640 1 --2- 172.21.15.100:0/3611513273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6b8152570 0x7fe6b80b1250 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0036040 tx=0x7fe6b0033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:18.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.315+0000 7fe6aea00640 1 -- 172.21.15.100:0/3611513273 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0038040 con 0x7fe6b8152570 2024-09-17T04:24:18.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.315+0000 7fe6aea00640 1 -- 172.21.15.100:0/3611513273 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6b0004110 con 0x7fe6b8152570 2024-09-17T04:24:18.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.315+0000 7fe6bd400640 1 -- 172.21.15.100:0/3611513273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6b8155420 con 0x7fe6b8152570 2024-09-17T04:24:18.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.315+0000 7fe6aea00640 1 -- 172.21.15.100:0/3611513273 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b00336e0 con 0x7fe6b8152570 2024-09-17T04:24:18.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.315+0000 7fe6bd400640 1 -- 172.21.15.100:0/3611513273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6b80b1f10 con 0x7fe6b8152570 2024-09-17T04:24:18.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.316+0000 7fe6bd400640 1 -- 172.21.15.100:0/3611513273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6b81535c0 con 0x7fe6b8152570 2024-09-17T04:24:18.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.317+0000 7fe6aea00640 1 -- 172.21.15.100:0/3611513273 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe6b0044020 con 0x7fe6b8152570 2024-09-17T04:24:18.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.318+0000 7fe6aea00640 1 --2- 172.21.15.100:0/3611513273 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe698078b90 0x7fe69807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:18.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.318+0000 7fe6aea00640 1 -- 172.21.15.100:0/3611513273 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe6b00bab50 con 0x7fe6b8152570 2024-09-17T04:24:18.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.319+0000 7fe6b6800640 1 --2- 172.21.15.100:0/3611513273 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe698078b90 0x7fe69807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:18.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.319+0000 7fe6b6800640 1 --2- 172.21.15.100:0/3611513273 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe698078b90 0x7fe69807b0a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8001520 tx=0x7fe6a8004a00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:18.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.322+0000 7fe6aea00640 1 -- 172.21.15.100:0/3611513273 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe6b8099ee0 con 0x7fe6b8152570 2024-09-17T04:24:18.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.536+0000 7fe6bd400640 1 -- 172.21.15.100:0/3611513273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6b814faa0 con 0x7fe6b8152570 2024-09-17T04:24:18.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.537+0000 7fe6aea00640 1 -- 172.21.15.100:0/3611513273 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7fe6b0082930 con 0x7fe6b8152570 2024-09-17T04:24:18.538 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:18.538 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":8,"btime":"2024-09-17T04:24:18:058994+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4328,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2129046042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":2129046042},{"type":"v1","addr":"172.21.15.100:6835","nonce":2129046042}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4334,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/44255118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":44255118},{"type":"v1","addr":"172.21.15.184:6836","nonce":44255118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/1644264480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1644264480},{"type":"v1","addr":"172.21.15.100:6834","nonce":1644264480}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:24:14.527272+0000","modified":"2024-09-17T04:24:17.543043+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4368},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4368":{"gid":4368,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/2211103925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2211103925},{"type":"v1","addr":"172.21.15.184:6837","nonce":2211103925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4368,"qdb_cluster":[4368]},"id":1}]} 2024-09-17T04:24:18.538 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 8 2024-09-17T04:24:18.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.539+0000 7fe6bd400640 1 -- 172.21.15.100:0/3611513273 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe698078b90 msgr2=0x7fe69807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:18.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.539+0000 7fe6bd400640 1 --2- 172.21.15.100:0/3611513273 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe698078b90 0x7fe69807b0a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8001520 tx=0x7fe6a8004a00 comp rx=0 tx=0).stop 2024-09-17T04:24:18.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.539+0000 7fe6bd400640 1 -- 172.21.15.100:0/3611513273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6b8152570 msgr2=0x7fe6b80b1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:18.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.539+0000 7fe6bd400640 1 --2- 172.21.15.100:0/3611513273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6b8152570 0x7fe6b80b1250 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0036040 tx=0x7fe6b0033670 comp rx=0 tx=0).stop 2024-09-17T04:24:18.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.539+0000 7fe6bd400640 1 -- 172.21.15.100:0/3611513273 shutdown_connections 2024-09-17T04:24:18.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.540+0000 7fe6bd400640 1 --2- 172.21.15.100:0/3611513273 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe698078b90 0x7fe69807b0a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:18.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.540+0000 7fe6bd400640 1 --2- 172.21.15.100:0/3611513273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6b8152570 0x7fe6b80b1250 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:18.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.540+0000 7fe6bd400640 1 --2- 172.21.15.100:0/3611513273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6b8145b90 0x7fe6b8154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:18.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.540+0000 7fe6bd400640 1 --2- 172.21.15.100:0/3611513273 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe69c004d40 0x7fe6b80999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:18.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.540+0000 7fe6bd400640 1 -- 172.21.15.100:0/3611513273 >> 172.21.15.100:0/3611513273 conn(0x7fe6b806d730 msgr2=0x7fe6b81463c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:18.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.540+0000 7fe6bd400640 1 -- 172.21.15.100:0/3611513273 shutdown_connections 2024-09-17T04:24:18.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.540+0000 7fe6bd400640 1 -- 172.21.15.100:0/3611513273 wait complete. 2024-09-17T04:24:18.563 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:24:18.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.782+0000 7fba1be00640 1 -- 172.21.15.100:0/203803001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba140ad3c0 msgr2=0x7fba140ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:18.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.782+0000 7fba1be00640 1 --2- 172.21.15.100:0/203803001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba140ad3c0 0x7fba140ad7d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fba14072810 tx=0x7fba0002b150 comp rx=0 tx=0).stop 2024-09-17T04:24:18.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.782+0000 7fba1be00640 1 -- 172.21.15.100:0/203803001 shutdown_connections 2024-09-17T04:24:18.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.782+0000 7fba1be00640 1 --2- 172.21.15.100:0/203803001 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba140add10 0x7fba140b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:18.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.782+0000 7fba1be00640 1 --2- 172.21.15.100:0/203803001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba140ad3c0 0x7fba140ad7d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:18.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.782+0000 7fba1be00640 1 --2- 172.21.15.100:0/203803001 >> v2:172.21.15.184:3300/0 conn(0x7fba140704b0 0x7fba14073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:18.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.782+0000 7fba1be00640 1 -- 172.21.15.100:0/203803001 >> 172.21.15.100:0/203803001 conn(0x7fba1406d730 msgr2=0x7fba140b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:18.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.782+0000 7fba1be00640 1 -- 172.21.15.100:0/203803001 shutdown_connections 2024-09-17T04:24:18.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.782+0000 7fba1be00640 1 -- 172.21.15.100:0/203803001 wait complete. 2024-09-17T04:24:18.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.785+0000 7fba1be00640 1 Processor -- start 2024-09-17T04:24:18.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.785+0000 7fba1be00640 1 -- start start 2024-09-17T04:24:18.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.786+0000 7fba1be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba140704b0 0x7fba141f4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:18.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.786+0000 7fba1be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba140ad3c0 0x7fba14154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:18.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.786+0000 7fba1be00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba140add10 0x7fba141e8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:18.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.786+0000 7fba1be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fba140712b0 con 0x7fba140704b0 2024-09-17T04:24:18.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.786+0000 7fba1be00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fba14071130 con 0x7fba140ad3c0 2024-09-17T04:24:18.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.786+0000 7fba1be00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fba140b4350 con 0x7fba140add10 2024-09-17T04:24:18.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.787+0000 7fba19400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba140704b0 0x7fba141f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:18.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.787+0000 7fba19400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba140704b0 0x7fba141f4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41552/0 (socket says 172.21.15.100:41552) 2024-09-17T04:24:18.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.787+0000 7fba19400640 1 -- 172.21.15.100:0/1934538844 learned_addr learned my addr 172.21.15.100:0/1934538844 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:18.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.787+0000 7fba18a00640 1 --2- 172.21.15.100:0/1934538844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba140ad3c0 0x7fba14154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:18.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.787+0000 7fba19e00640 1 --2- 172.21.15.100:0/1934538844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba140add10 0x7fba141e8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:18.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.787+0000 7fba19400640 1 -- 172.21.15.100:0/1934538844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba140add10 msgr2=0x7fba141e8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:18.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.787+0000 7fba19400640 1 --2- 172.21.15.100:0/1934538844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba140add10 0x7fba141e8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:18.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.787+0000 7fba19400640 1 -- 172.21.15.100:0/1934538844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba140ad3c0 msgr2=0x7fba14154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:18.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.787+0000 7fba19400640 1 --2- 172.21.15.100:0/1934538844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba140ad3c0 0x7fba14154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:18.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.787+0000 7fba19400640 1 -- 172.21.15.100:0/1934538844 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1415dba0 con 0x7fba140704b0 2024-09-17T04:24:18.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.787+0000 7fba19400640 1 --2- 172.21.15.100:0/1934538844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba140704b0 0x7fba141f4250 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fba0800d750 tx=0x7fba0800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:18.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.788+0000 7fba11600640 1 -- 172.21.15.100:0/1934538844 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba08017070 con 0x7fba140704b0 2024-09-17T04:24:18.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.788+0000 7fba11600640 1 -- 172.21.15.100:0/1934538844 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba08009cc0 con 0x7fba140704b0 2024-09-17T04:24:18.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.788+0000 7fba1be00640 1 -- 172.21.15.100:0/1934538844 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba1415de80 con 0x7fba140704b0 2024-09-17T04:24:18.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.788+0000 7fba11600640 1 -- 172.21.15.100:0/1934538844 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba080106c0 con 0x7fba140704b0 2024-09-17T04:24:18.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.788+0000 7fba1be00640 1 -- 172.21.15.100:0/1934538844 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba1415bfc0 con 0x7fba140704b0 2024-09-17T04:24:18.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.789+0000 7fba1be00640 1 -- 172.21.15.100:0/1934538844 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba141f49e0 con 0x7fba140704b0 2024-09-17T04:24:18.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.790+0000 7fba11600640 1 -- 172.21.15.100:0/1934538844 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fba0801a070 con 0x7fba140704b0 2024-09-17T04:24:18.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.792+0000 7fba11600640 1 --2- 172.21.15.100:0/1934538844 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb9f4078b90 0x7fb9f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:18.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.792+0000 7fba11600640 1 -- 172.21.15.100:0/1934538844 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fba0809ae30 con 0x7fba140704b0 2024-09-17T04:24:18.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.792+0000 7fba18a00640 1 --2- 172.21.15.100:0/1934538844 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb9f4078b90 0x7fb9f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:18.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.793+0000 7fba18a00640 1 --2- 172.21.15.100:0/1934538844 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb9f4078b90 0x7fb9f407b0a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fba00033000 tx=0x7fba00035040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:18.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.796+0000 7fba11600640 1 -- 172.21.15.100:0/1934538844 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fba0800de60 con 0x7fba140704b0 2024-09-17T04:24:18.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.979+0000 7fba1be00640 1 -- 172.21.15.100:0/1934538844 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fba140afb60 con 0x7fba140704b0 2024-09-17T04:24:18.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.981+0000 7fba11600640 1 -- 172.21.15.100:0/1934538844 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16713 (secure 0 0 0) 0x7fba140afb60 con 0x7fba140704b0 2024-09-17T04:24:18.981 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:18.981 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":23,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:24:16.038948+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:24:12.826665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T04:24:13.485446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:24:18.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.983+0000 7fba1be00640 1 -- 172.21.15.100:0/1934538844 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb9f4078b90 msgr2=0x7fb9f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:18.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.983+0000 7fba1be00640 1 --2- 172.21.15.100:0/1934538844 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb9f4078b90 0x7fb9f407b0a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fba00033000 tx=0x7fba00035040 comp rx=0 tx=0).stop 2024-09-17T04:24:18.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.983+0000 7fba1be00640 1 -- 172.21.15.100:0/1934538844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba140704b0 msgr2=0x7fba141f4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:18.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.983+0000 7fba1be00640 1 --2- 172.21.15.100:0/1934538844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba140704b0 0x7fba141f4250 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fba0800d750 tx=0x7fba0800dc10 comp rx=0 tx=0).stop 2024-09-17T04:24:18.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.983+0000 7fba1be00640 1 -- 172.21.15.100:0/1934538844 shutdown_connections 2024-09-17T04:24:18.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.983+0000 7fba1be00640 1 --2- 172.21.15.100:0/1934538844 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb9f4078b90 0x7fb9f407b0a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:18.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.983+0000 7fba1be00640 1 --2- 172.21.15.100:0/1934538844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba140add10 0x7fba141e8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:18.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.983+0000 7fba1be00640 1 --2- 172.21.15.100:0/1934538844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba140ad3c0 0x7fba14154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:18.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.983+0000 7fba1be00640 1 --2- 172.21.15.100:0/1934538844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba140704b0 0x7fba141f4250 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:18.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.983+0000 7fba1be00640 1 -- 172.21.15.100:0/1934538844 >> 172.21.15.100:0/1934538844 conn(0x7fba1406d730 msgr2=0x7fba140b0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:18.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.983+0000 7fba1be00640 1 -- 172.21.15.100:0/1934538844 shutdown_connections 2024-09-17T04:24:18.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:18.983+0000 7fba1be00640 1 -- 172.21.15.100:0/1934538844 wait complete. 2024-09-17T04:24:19.007 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T04:24:19.008 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:24:19.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.230+0000 7f283b200640 1 -- 172.21.15.100:0/3242769944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28200053c0 msgr2=0x7f282000ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:19.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.230+0000 7f283b200640 1 --2- 172.21.15.100:0/3242769944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28200053c0 0x7f282000ba70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2834005fd0 tx=0x7f2820005830 comp rx=0 tx=0).stop 2024-09-17T04:24:19.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.230+0000 7f283b200640 1 -- 172.21.15.100:0/3242769944 shutdown_connections 2024-09-17T04:24:19.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.230+0000 7f283b200640 1 --2- 172.21.15.100:0/3242769944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28200053c0 0x7f282000ba70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:19.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.230+0000 7f283b200640 1 -- 172.21.15.100:0/3242769944 >> 172.21.15.100:0/3242769944 conn(0x7f283c0b1e40 msgr2=0x7f283c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:19.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.230+0000 7f283b200640 1 -- 172.21.15.100:0/3242769944 shutdown_connections 2024-09-17T04:24:19.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.230+0000 7f283b200640 1 -- 172.21.15.100:0/3242769944 wait complete. 2024-09-17T04:24:19.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.232+0000 7f283b200640 1 Processor -- start 2024-09-17T04:24:19.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.232+0000 7f283b200640 1 -- start start 2024-09-17T04:24:19.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.234+0000 7f283b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28200053c0 0x7f283c1f5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:19.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.234+0000 7f283b200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f283c1f6de0 0x7f283c156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:19.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.234+0000 7f283b200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f283c0cdf90 0x7f283c0ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:19.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.234+0000 7f283b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f283c15a0f0 con 0x7f28200053c0 2024-09-17T04:24:19.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.234+0000 7f283b200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f283c0b3440 con 0x7f283c1f6de0 2024-09-17T04:24:19.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.234+0000 7f283b200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f283c15a270 con 0x7f283c0cdf90 2024-09-17T04:24:19.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.234+0000 7f2839e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28200053c0 0x7f283c1f5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:19.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.234+0000 7f2839e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28200053c0 0x7f283c1f5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41574/0 (socket says 172.21.15.100:41574) 2024-09-17T04:24:19.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.234+0000 7f2839e00640 1 -- 172.21.15.100:0/158213480 learned_addr learned my addr 172.21.15.100:0/158213480 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:19.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.234+0000 7f283a800640 1 --2- 172.21.15.100:0/158213480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f283c0cdf90 0x7f283c0ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:19.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.234+0000 7f2831400640 1 --2- 172.21.15.100:0/158213480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f283c1f6de0 0x7f283c156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:19.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.234+0000 7f2839e00640 1 -- 172.21.15.100:0/158213480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f283c0cdf90 msgr2=0x7f283c0ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:19.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.234+0000 7f2839e00640 1 --2- 172.21.15.100:0/158213480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f283c0cdf90 0x7f283c0ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:19.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.234+0000 7f2839e00640 1 -- 172.21.15.100:0/158213480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f283c1f6de0 msgr2=0x7f283c156e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:19.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.234+0000 7f2839e00640 1 --2- 172.21.15.100:0/158213480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f283c1f6de0 0x7f283c156e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:19.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.234+0000 7f2839e00640 1 -- 172.21.15.100:0/158213480 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283c1f2b70 con 0x7f28200053c0 2024-09-17T04:24:19.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.234+0000 7f283a800640 1 --2- 172.21.15.100:0/158213480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f283c0cdf90 0x7f283c0ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:24:19.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.235+0000 7f2839e00640 1 --2- 172.21.15.100:0/158213480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28200053c0 0x7f283c1f5ae0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f282c003cd0 tx=0x7f282c001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:19.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.235+0000 7f2832a00640 1 -- 172.21.15.100:0/158213480 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c015070 con 0x7f28200053c0 2024-09-17T04:24:19.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.235+0000 7f2832a00640 1 -- 172.21.15.100:0/158213480 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f282c011860 con 0x7f28200053c0 2024-09-17T04:24:19.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.235+0000 7f283b200640 1 -- 172.21.15.100:0/158213480 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f283c0ce8e0 con 0x7f28200053c0 2024-09-17T04:24:19.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.235+0000 7f2832a00640 1 -- 172.21.15.100:0/158213480 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c011cb0 con 0x7f28200053c0 2024-09-17T04:24:19.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.235+0000 7f283b200640 1 -- 172.21.15.100:0/158213480 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f283c1f30d0 con 0x7f28200053c0 2024-09-17T04:24:19.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.236+0000 7f283b200640 1 -- 172.21.15.100:0/158213480 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f283c15d6c0 con 0x7f28200053c0 2024-09-17T04:24:19.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.237+0000 7f2832a00640 1 -- 172.21.15.100:0/158213480 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f282c018070 con 0x7f28200053c0 2024-09-17T04:24:19.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.238+0000 7f2832a00640 1 --2- 172.21.15.100:0/158213480 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2818078b90 0x7f281807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:19.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.238+0000 7f2832a00640 1 -- 172.21.15.100:0/158213480 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f282c09b990 con 0x7f28200053c0 2024-09-17T04:24:19.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.238+0000 7f2831400640 1 --2- 172.21.15.100:0/158213480 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2818078b90 0x7f281807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:19.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.239+0000 7f2831400640 1 --2- 172.21.15.100:0/158213480 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2818078b90 0x7f281807b0a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2824001860 tx=0x7f2824001750 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:19.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.243+0000 7f2832a00640 1 -- 172.21.15.100:0/158213480 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f283c0ce8e0 con 0x7f28200053c0 2024-09-17T04:24:19.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.423+0000 7f283b200640 1 -- 172.21.15.100:0/158213480 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f283c06f190 con 0x7f2818078b90 2024-09-17T04:24:19.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.429+0000 7f2832a00640 1 -- 172.21.15.100:0/158213480 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245292 (secure 0 0 0) 0x7f283c06f190 con 0x7f2818078b90 2024-09-17T04:24:19.433 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:19.434 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:24:19.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.433+0000 7f283b200640 1 -- 172.21.15.100:0/158213480 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2818078b90 msgr2=0x7f281807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:19.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.433+0000 7f283b200640 1 --2- 172.21.15.100:0/158213480 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2818078b90 0x7f281807b0a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2824001860 tx=0x7f2824001750 comp rx=0 tx=0).stop 2024-09-17T04:24:19.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.433+0000 7f283b200640 1 -- 172.21.15.100:0/158213480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28200053c0 msgr2=0x7f283c1f5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:19.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.433+0000 7f283b200640 1 --2- 172.21.15.100:0/158213480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28200053c0 0x7f283c1f5ae0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f282c003cd0 tx=0x7f282c001800 comp rx=0 tx=0).stop 2024-09-17T04:24:19.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.434+0000 7f283b200640 1 -- 172.21.15.100:0/158213480 shutdown_connections 2024-09-17T04:24:19.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.434+0000 7f283b200640 1 --2- 172.21.15.100:0/158213480 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2818078b90 0x7f281807b0a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:19.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.434+0000 7f283b200640 1 --2- 172.21.15.100:0/158213480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f283c0cdf90 0x7f283c0ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:19.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.434+0000 7f283b200640 1 --2- 172.21.15.100:0/158213480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f283c1f6de0 0x7f283c156e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:19.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.434+0000 7f283b200640 1 --2- 172.21.15.100:0/158213480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28200053c0 0x7f283c1f5ae0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:19.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.434+0000 7f283b200640 1 -- 172.21.15.100:0/158213480 >> 172.21.15.100:0/158213480 conn(0x7f283c0b1e40 msgr2=0x7f283c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:19.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.434+0000 7f283b200640 1 -- 172.21.15.100:0/158213480 shutdown_connections 2024-09-17T04:24:19.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.434+0000 7f283b200640 1 -- 172.21.15.100:0/158213480 wait complete. 2024-09-17T04:24:19.455 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-17T04:24:17.964480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":158,"acting":158,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219156,"kb_used_data":2676,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755564,"statfs":{"total":773094113280,"available":772869697536,"internally_reserved":0,"allocated":2740224,"data_stored":1668127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.944839"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.041403+0000","last_change":"2024-09-17T04:24:14.041403+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.041403+0000","last_undegraded":"2024-09-17T04:24:14.041403+0000","last_fullsized":"2024-09-17T04:24:14.041403+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.049136+0000","last_change":"2024-09-17T04:24:14.049136+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.049136+0000","last_undegraded":"2024-09-17T04:24:14.049136+0000","last_fullsized":"2024-09-17T04:24:14.049136+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.039709+0000","last_change":"2024-09-17T04:24:14.039709+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.039709+0000","last_undegraded":"2024-09-17T04:24:14.039709+0000","last_fullsized":"2024-09-17T04:24:14.039709+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.048608+0000","last_change":"2024-09-17T04:24:14.048608+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.048608+0000","last_undegraded":"2024-09-17T04:24:14.048608+0000","last_fullsized":"2024-09-17T04:24:14.048608+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.044340+0000","last_change":"2024-09-17T04:24:14.044340+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.044340+0000","last_undegraded":"2024-09-17T04:24:14.044340+0000","last_fullsized":"2024-09-17T04:24:14.044340+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224308+0000","last_change":"2024-09-17T04:24:15.062993+0000","last_active":"2024-09-17T04:24:16.224308+0000","last_peered":"2024-09-17T04:24:16.224308+0000","last_clean":"2024-09-17T04:24:16.224308+0000","last_became_active":"2024-09-17T04:24:15.062311+0000","last_became_peered":"2024-09-17T04:24:15.062311+0000","last_unstale":"2024-09-17T04:24:16.224308+0000","last_undegraded":"2024-09-17T04:24:16.224308+0000","last_fullsized":"2024-09-17T04:24:16.224308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:26:42.504856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.040807+0000","last_change":"2024-09-17T04:24:14.040807+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.040807+0000","last_undegraded":"2024-09-17T04:24:14.040807+0000","last_fullsized":"2024-09-17T04:24:14.040807+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.045923+0000","last_change":"2024-09-17T04:24:14.045923+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.045923+0000","last_undegraded":"2024-09-17T04:24:14.045923+0000","last_fullsized":"2024-09-17T04:24:14.045923+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.039794+0000","last_change":"2024-09-17T04:24:14.039794+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.039794+0000","last_undegraded":"2024-09-17T04:24:14.039794+0000","last_fullsized":"2024-09-17T04:24:14.039794+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.034825+0000","last_change":"2024-09-17T04:24:14.034825+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.034825+0000","last_undegraded":"2024-09-17T04:24:14.034825+0000","last_fullsized":"2024-09-17T04:24:14.034825+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225225+0000","last_change":"2024-09-17T04:24:15.047951+0000","last_active":"2024-09-17T04:24:16.225225+0000","last_peered":"2024-09-17T04:24:16.225225+0000","last_clean":"2024-09-17T04:24:16.225225+0000","last_became_active":"2024-09-17T04:24:15.047536+0000","last_became_peered":"2024-09-17T04:24:15.047536+0000","last_unstale":"2024-09-17T04:24:16.225225+0000","last_undegraded":"2024-09-17T04:24:16.225225+0000","last_fullsized":"2024-09-17T04:24:16.225225+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:38:48.386032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.047511+0000","last_change":"2024-09-17T04:24:14.047511+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.047511+0000","last_undegraded":"2024-09-17T04:24:14.047511+0000","last_fullsized":"2024-09-17T04:24:14.047511+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.045559+0000","last_change":"2024-09-17T04:24:14.045559+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.045559+0000","last_undegraded":"2024-09-17T04:24:14.045559+0000","last_fullsized":"2024-09-17T04:24:14.045559+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.042563+0000","last_change":"2024-09-17T04:24:14.042563+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.042563+0000","last_undegraded":"2024-09-17T04:24:14.042563+0000","last_fullsized":"2024-09-17T04:24:14.042563+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.041977+0000","last_change":"2024-09-17T04:24:14.041977+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.041977+0000","last_undegraded":"2024-09-17T04:24:14.041977+0000","last_fullsized":"2024-09-17T04:24:14.041977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.044137+0000","last_change":"2024-09-17T04:24:14.044137+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.044137+0000","last_undegraded":"2024-09-17T04:24:14.044137+0000","last_fullsized":"2024-09-17T04:24:14.044137+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224382+0000","last_change":"2024-09-17T04:24:15.070200+0000","last_active":"2024-09-17T04:24:16.224382+0000","last_peered":"2024-09-17T04:24:16.224382+0000","last_clean":"2024-09-17T04:24:16.224382+0000","last_became_active":"2024-09-17T04:24:15.069872+0000","last_became_peered":"2024-09-17T04:24:15.069872+0000","last_unstale":"2024-09-17T04:24:16.224382+0000","last_undegraded":"2024-09-17T04:24:16.224382+0000","last_fullsized":"2024-09-17T04:24:16.224382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:22.880022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.046960+0000","last_change":"2024-09-17T04:24:14.046960+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.046960+0000","last_undegraded":"2024-09-17T04:24:14.046960+0000","last_fullsized":"2024-09-17T04:24:14.046960+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225400+0000","last_change":"2024-09-17T04:24:15.074863+0000","last_active":"2024-09-17T04:24:16.225400+0000","last_peered":"2024-09-17T04:24:16.225400+0000","last_clean":"2024-09-17T04:24:16.225400+0000","last_became_active":"2024-09-17T04:24:15.074467+0000","last_became_peered":"2024-09-17T04:24:15.074467+0000","last_unstale":"2024-09-17T04:24:16.225400+0000","last_undegraded":"2024-09-17T04:24:16.225400+0000","last_fullsized":"2024-09-17T04:24:16.225400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:36:59.215343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.044399+0000","last_change":"2024-09-17T04:24:14.044399+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.044399+0000","last_undegraded":"2024-09-17T04:24:14.044399+0000","last_fullsized":"2024-09-17T04:24:14.044399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225430+0000","last_change":"2024-09-17T04:24:15.075902+0000","last_active":"2024-09-17T04:24:16.225430+0000","last_peered":"2024-09-17T04:24:16.225430+0000","last_clean":"2024-09-17T04:24:16.225430+0000","last_became_active":"2024-09-17T04:24:15.075625+0000","last_became_peered":"2024-09-17T04:24:15.075625+0000","last_unstale":"2024-09-17T04:24:16.225430+0000","last_undegraded":"2024-09-17T04:24:16.225430+0000","last_fullsized":"2024-09-17T04:24:16.225430+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:34.202614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.039484+0000","last_change":"2024-09-17T04:24:14.039484+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.039484+0000","last_undegraded":"2024-09-17T04:24:14.039484+0000","last_fullsized":"2024-09-17T04:24:14.039484+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.042670+0000","last_change":"2024-09-17T04:24:14.042670+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.042670+0000","last_undegraded":"2024-09-17T04:24:14.042670+0000","last_fullsized":"2024-09-17T04:24:14.042670+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.034151+0000","last_change":"2024-09-17T04:24:14.034151+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.034151+0000","last_undegraded":"2024-09-17T04:24:14.034151+0000","last_fullsized":"2024-09-17T04:24:14.034151+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225374+0000","last_change":"2024-09-17T04:24:14.062806+0000","last_active":"2024-09-17T04:24:16.225374+0000","last_peered":"2024-09-17T04:24:16.225374+0000","last_clean":"2024-09-17T04:24:16.225374+0000","last_became_active":"2024-09-17T04:24:14.062282+0000","last_became_peered":"2024-09-17T04:24:14.062282+0000","last_unstale":"2024-09-17T04:24:16.225374+0000","last_undegraded":"2024-09-17T04:24:16.225374+0000","last_fullsized":"2024-09-17T04:24:16.225374+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:23.698652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.058753+0000","last_change":"2024-09-17T04:24:14.056222+0000","last_active":"2024-09-17T04:24:14.058753+0000","last_peered":"2024-09-17T04:24:14.058753+0000","last_clean":"2024-09-17T04:24:14.058753+0000","last_became_active":"2024-09-17T04:24:14.055874+0000","last_became_peered":"2024-09-17T04:24:14.055874+0000","last_unstale":"2024-09-17T04:24:14.058753+0000","last_undegraded":"2024-09-17T04:24:14.058753+0000","last_fullsized":"2024-09-17T04:24:14.058753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:22.510592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224568+0000","last_change":"2024-09-17T04:24:14.054189+0000","last_active":"2024-09-17T04:24:16.224568+0000","last_peered":"2024-09-17T04:24:16.224568+0000","last_clean":"2024-09-17T04:24:16.224568+0000","last_became_active":"2024-09-17T04:24:14.053870+0000","last_became_peered":"2024-09-17T04:24:14.053870+0000","last_unstale":"2024-09-17T04:24:16.224568+0000","last_undegraded":"2024-09-17T04:24:16.224568+0000","last_fullsized":"2024-09-17T04:24:16.224568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:12.776180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224542+0000","last_change":"2024-09-17T04:24:15.059936+0000","last_active":"2024-09-17T04:24:16.224542+0000","last_peered":"2024-09-17T04:24:16.224542+0000","last_clean":"2024-09-17T04:24:16.224542+0000","last_became_active":"2024-09-17T04:24:15.059645+0000","last_became_peered":"2024-09-17T04:24:15.059645+0000","last_unstale":"2024-09-17T04:24:16.224542+0000","last_undegraded":"2024-09-17T04:24:16.224542+0000","last_fullsized":"2024-09-17T04:24:16.224542+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:08:11.703658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.064116+0000","last_change":"2024-09-17T04:24:14.060082+0000","last_active":"2024-09-17T04:24:14.064116+0000","last_peered":"2024-09-17T04:24:14.064116+0000","last_clean":"2024-09-17T04:24:14.064116+0000","last_became_active":"2024-09-17T04:24:14.058795+0000","last_became_peered":"2024-09-17T04:24:14.058795+0000","last_unstale":"2024-09-17T04:24:14.064116+0000","last_undegraded":"2024-09-17T04:24:14.064116+0000","last_fullsized":"2024-09-17T04:24:14.064116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:44.734015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.056597+0000","last_change":"2024-09-17T04:24:14.053595+0000","last_active":"2024-09-17T04:24:14.056597+0000","last_peered":"2024-09-17T04:24:14.056597+0000","last_clean":"2024-09-17T04:24:14.056597+0000","last_became_active":"2024-09-17T04:24:14.053260+0000","last_became_peered":"2024-09-17T04:24:14.053260+0000","last_unstale":"2024-09-17T04:24:14.056597+0000","last_undegraded":"2024-09-17T04:24:14.056597+0000","last_fullsized":"2024-09-17T04:24:14.056597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:29.564993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.043231+0000","last_change":"2024-09-17T04:24:14.043231+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.043231+0000","last_undegraded":"2024-09-17T04:24:14.043231+0000","last_fullsized":"2024-09-17T04:24:14.043231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.064090+0000","last_change":"2024-09-17T04:24:14.059203+0000","last_active":"2024-09-17T04:24:14.064090+0000","last_peered":"2024-09-17T04:24:14.064090+0000","last_clean":"2024-09-17T04:24:14.064090+0000","last_became_active":"2024-09-17T04:24:14.058623+0000","last_became_peered":"2024-09-17T04:24:14.058623+0000","last_unstale":"2024-09-17T04:24:14.064090+0000","last_undegraded":"2024-09-17T04:24:14.064090+0000","last_fullsized":"2024-09-17T04:24:14.064090+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:28.196011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T04:24:13.026983+0000","last_change":"2024-09-17T04:24:13.026983+0000","last_active":"2024-09-17T04:24:13.015626+0000","last_peered":"2024-09-17T04:24:13.015626+0000","last_clean":"2024-09-17T04:24:13.015626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:13.026983+0000","last_undegraded":"2024-09-17T04:24:13.026983+0000","last_fullsized":"2024-09-17T04:24:13.026983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.033988+0000","last_change":"2024-09-17T04:24:14.033988+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.033988+0000","last_undegraded":"2024-09-17T04:24:14.033988+0000","last_fullsized":"2024-09-17T04:24:14.033988+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.060560+0000","last_change":"2024-09-17T04:24:14.058729+0000","last_active":"2024-09-17T04:24:14.060560+0000","last_peered":"2024-09-17T04:24:14.060560+0000","last_clean":"2024-09-17T04:24:14.060560+0000","last_became_active":"2024-09-17T04:24:14.057616+0000","last_became_peered":"2024-09-17T04:24:14.057616+0000","last_unstale":"2024-09-17T04:24:14.060560+0000","last_undegraded":"2024-09-17T04:24:14.060560+0000","last_fullsized":"2024-09-17T04:24:14.060560+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:23:31.947281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.043536+0000","last_change":"2024-09-17T04:24:14.043536+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.043536+0000","last_undegraded":"2024-09-17T04:24:14.043536+0000","last_fullsized":"2024-09-17T04:24:14.043536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T04:24:13.029149+0000","last_change":"2024-09-17T04:24:13.029149+0000","last_active":"2024-09-17T04:24:13.015626+0000","last_peered":"2024-09-17T04:24:13.015626+0000","last_clean":"2024-09-17T04:24:13.015626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:13.029149+0000","last_undegraded":"2024-09-17T04:24:13.029149+0000","last_fullsized":"2024-09-17T04:24:13.029149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.064018+0000","last_change":"2024-09-17T04:24:14.058127+0000","last_active":"2024-09-17T04:24:14.064018+0000","last_peered":"2024-09-17T04:24:14.064018+0000","last_clean":"2024-09-17T04:24:14.064018+0000","last_became_active":"2024-09-17T04:24:14.055990+0000","last_became_peered":"2024-09-17T04:24:14.055990+0000","last_unstale":"2024-09-17T04:24:14.064018+0000","last_undegraded":"2024-09-17T04:24:14.064018+0000","last_fullsized":"2024-09-17T04:24:14.064018+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:23:31.947281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.037775+0000","last_change":"2024-09-17T04:24:14.037775+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.037775+0000","last_undegraded":"2024-09-17T04:24:14.037775+0000","last_fullsized":"2024-09-17T04:24:14.037775+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.053546+0000","last_change":"2024-09-17T04:24:14.051949+0000","last_active":"2024-09-17T04:24:14.053546+0000","last_peered":"2024-09-17T04:24:14.053546+0000","last_clean":"2024-09-17T04:24:14.053546+0000","last_became_active":"2024-09-17T04:24:14.051548+0000","last_became_peered":"2024-09-17T04:24:14.051548+0000","last_unstale":"2024-09-17T04:24:14.053546+0000","last_undegraded":"2024-09-17T04:24:14.053546+0000","last_fullsized":"2024-09-17T04:24:14.053546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:19:57.759163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.039461+0000","last_change":"2024-09-17T04:24:14.039461+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.039461+0000","last_undegraded":"2024-09-17T04:24:14.039461+0000","last_fullsized":"2024-09-17T04:24:14.039461+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T04:24:13.021867+0000","last_change":"2024-09-17T04:24:13.021867+0000","last_active":"2024-09-17T04:24:13.015626+0000","last_peered":"2024-09-17T04:24:13.015626+0000","last_clean":"2024-09-17T04:24:13.015626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:13.021867+0000","last_undegraded":"2024-09-17T04:24:13.021867+0000","last_fullsized":"2024-09-17T04:24:13.021867+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.037601+0000","last_change":"2024-09-17T04:24:14.037601+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.037601+0000","last_undegraded":"2024-09-17T04:24:14.037601+0000","last_fullsized":"2024-09-17T04:24:14.037601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.063831+0000","last_change":"2024-09-17T04:24:14.059673+0000","last_active":"2024-09-17T04:24:14.063831+0000","last_peered":"2024-09-17T04:24:14.063831+0000","last_clean":"2024-09-17T04:24:14.063831+0000","last_became_active":"2024-09-17T04:24:14.057668+0000","last_became_peered":"2024-09-17T04:24:14.057668+0000","last_unstale":"2024-09-17T04:24:14.063831+0000","last_undegraded":"2024-09-17T04:24:14.063831+0000","last_fullsized":"2024-09-17T04:24:14.063831+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:18:36.484153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.039797+0000","last_change":"2024-09-17T04:24:14.039797+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.039797+0000","last_undegraded":"2024-09-17T04:24:14.039797+0000","last_fullsized":"2024-09-17T04:24:14.039797+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T04:24:13.021874+0000","last_change":"2024-09-17T04:24:13.021874+0000","last_active":"2024-09-17T04:24:13.015626+0000","last_peered":"2024-09-17T04:24:13.015626+0000","last_clean":"2024-09-17T04:24:13.015626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:13.021874+0000","last_undegraded":"2024-09-17T04:24:13.021874+0000","last_fullsized":"2024-09-17T04:24:13.021874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.039784+0000","last_change":"2024-09-17T04:24:14.039784+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.039784+0000","last_undegraded":"2024-09-17T04:24:14.039784+0000","last_fullsized":"2024-09-17T04:24:14.039784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T04:24:13.024980+0000","last_change":"2024-09-17T04:24:13.024980+0000","last_active":"2024-09-17T04:24:13.015626+0000","last_peered":"2024-09-17T04:24:13.015626+0000","last_clean":"2024-09-17T04:24:13.015626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:13.024980+0000","last_undegraded":"2024-09-17T04:24:13.024980+0000","last_fullsized":"2024-09-17T04:24:13.024980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.046138+0000","last_change":"2024-09-17T04:24:14.046138+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.046138+0000","last_undegraded":"2024-09-17T04:24:14.046138+0000","last_fullsized":"2024-09-17T04:24:14.046138+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T04:24:13.021850+0000","last_change":"2024-09-17T04:24:13.021850+0000","last_active":"2024-09-17T04:24:13.015626+0000","last_peered":"2024-09-17T04:24:13.015626+0000","last_clean":"2024-09-17T04:24:13.015626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:13.021850+0000","last_undegraded":"2024-09-17T04:24:13.021850+0000","last_fullsized":"2024-09-17T04:24:13.021850+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.043031+0000","last_change":"2024-09-17T04:24:14.043031+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.043031+0000","last_undegraded":"2024-09-17T04:24:14.043031+0000","last_fullsized":"2024-09-17T04:24:14.043031+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":107,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.050263+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:24:14.050263+0000","last_peered":"2024-09-17T04:24:14.050263+0000","last_clean":"2024-09-17T04:24:14.050263+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:24:14.050263+0000","last_undegraded":"2024-09-17T04:24:14.050263+0000","last_fullsized":"2024-09-17T04:24:14.050263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.053358+0000","last_change":"2024-09-17T04:24:14.049088+0000","last_active":"2024-09-17T04:24:14.053358+0000","last_peered":"2024-09-17T04:24:14.053358+0000","last_clean":"2024-09-17T04:24:14.053358+0000","last_became_active":"2024-09-17T04:24:14.048529+0000","last_became_peered":"2024-09-17T04:24:14.048529+0000","last_unstale":"2024-09-17T04:24:14.053358+0000","last_undegraded":"2024-09-17T04:24:14.053358+0000","last_fullsized":"2024-09-17T04:24:14.053358+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:29.564993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225083+0000","last_change":"2024-09-17T04:24:15.065699+0000","last_active":"2024-09-17T04:24:16.225083+0000","last_peered":"2024-09-17T04:24:16.225083+0000","last_clean":"2024-09-17T04:24:16.225083+0000","last_became_active":"2024-09-17T04:24:15.061569+0000","last_became_peered":"2024-09-17T04:24:15.061569+0000","last_unstale":"2024-09-17T04:24:16.225083+0000","last_undegraded":"2024-09-17T04:24:16.225083+0000","last_fullsized":"2024-09-17T04:24:16.225083+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:01:19.496265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.064651+0000","last_change":"2024-09-17T04:24:14.061081+0000","last_active":"2024-09-17T04:24:14.064651+0000","last_peered":"2024-09-17T04:24:14.064651+0000","last_clean":"2024-09-17T04:24:14.064651+0000","last_became_active":"2024-09-17T04:24:14.060394+0000","last_became_peered":"2024-09-17T04:24:14.060394+0000","last_unstale":"2024-09-17T04:24:14.064651+0000","last_undegraded":"2024-09-17T04:24:14.064651+0000","last_fullsized":"2024-09-17T04:24:14.064651+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:44.734015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T04:24:13.021863+0000","last_change":"2024-09-17T04:24:13.021863+0000","last_active":"2024-09-17T04:24:13.015626+0000","last_peered":"2024-09-17T04:24:13.015626+0000","last_clean":"2024-09-17T04:24:13.015626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:13.021863+0000","last_undegraded":"2024-09-17T04:24:13.021863+0000","last_fullsized":"2024-09-17T04:24:13.021863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.061385+0000","last_change":"2024-09-17T04:24:14.053751+0000","last_active":"2024-09-17T04:24:14.061385+0000","last_peered":"2024-09-17T04:24:14.061385+0000","last_clean":"2024-09-17T04:24:14.061385+0000","last_became_active":"2024-09-17T04:24:14.051943+0000","last_became_peered":"2024-09-17T04:24:14.051943+0000","last_unstale":"2024-09-17T04:24:14.061385+0000","last_undegraded":"2024-09-17T04:24:14.061385+0000","last_fullsized":"2024-09-17T04:24:14.061385+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:22.510592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.033873+0000","last_change":"2024-09-17T04:24:14.033873+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.033873+0000","last_undegraded":"2024-09-17T04:24:14.033873+0000","last_fullsized":"2024-09-17T04:24:14.033873+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T04:24:13.024681+0000","last_change":"2024-09-17T04:24:13.024681+0000","last_active":"2024-09-17T04:24:13.015626+0000","last_peered":"2024-09-17T04:24:13.015626+0000","last_clean":"2024-09-17T04:24:13.015626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:13.024681+0000","last_undegraded":"2024-09-17T04:24:13.024681+0000","last_fullsized":"2024-09-17T04:24:13.024681+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.044390+0000","last_change":"2024-09-17T04:24:14.044390+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.044390+0000","last_undegraded":"2024-09-17T04:24:14.044390+0000","last_fullsized":"2024-09-17T04:24:14.044390+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.067724+0000","last_change":"2024-09-17T04:24:14.063354+0000","last_active":"2024-09-17T04:24:14.067724+0000","last_peered":"2024-09-17T04:24:14.067724+0000","last_clean":"2024-09-17T04:24:14.067724+0000","last_became_active":"2024-09-17T04:24:14.062231+0000","last_became_peered":"2024-09-17T04:24:14.062231+0000","last_unstale":"2024-09-17T04:24:14.067724+0000","last_undegraded":"2024-09-17T04:24:14.067724+0000","last_fullsized":"2024-09-17T04:24:14.067724+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:21.535057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T04:24:13.024741+0000","last_change":"2024-09-17T04:24:13.024741+0000","last_active":"2024-09-17T04:24:13.015626+0000","last_peered":"2024-09-17T04:24:13.015626+0000","last_clean":"2024-09-17T04:24:13.015626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:13.024741+0000","last_undegraded":"2024-09-17T04:24:13.024741+0000","last_fullsized":"2024-09-17T04:24:13.024741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225582+0000","last_change":"2024-09-17T04:24:15.050480+0000","last_active":"2024-09-17T04:24:16.225582+0000","last_peered":"2024-09-17T04:24:16.225582+0000","last_clean":"2024-09-17T04:24:16.225582+0000","last_became_active":"2024-09-17T04:24:15.050166+0000","last_became_peered":"2024-09-17T04:24:15.050166+0000","last_unstale":"2024-09-17T04:24:16.225582+0000","last_undegraded":"2024-09-17T04:24:16.225582+0000","last_fullsized":"2024-09-17T04:24:16.225582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:02.125884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.067446+0000","last_change":"2024-09-17T04:24:14.063222+0000","last_active":"2024-09-17T04:24:14.067446+0000","last_peered":"2024-09-17T04:24:14.067446+0000","last_clean":"2024-09-17T04:24:14.067446+0000","last_became_active":"2024-09-17T04:24:14.062750+0000","last_became_peered":"2024-09-17T04:24:14.062750+0000","last_unstale":"2024-09-17T04:24:14.067446+0000","last_undegraded":"2024-09-17T04:24:14.067446+0000","last_fullsized":"2024-09-17T04:24:14.067446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:28.196011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.063634+0000","last_change":"2024-09-17T04:24:14.059487+0000","last_active":"2024-09-17T04:24:14.063634+0000","last_peered":"2024-09-17T04:24:14.063634+0000","last_clean":"2024-09-17T04:24:14.063634+0000","last_became_active":"2024-09-17T04:24:14.059186+0000","last_became_peered":"2024-09-17T04:24:14.059186+0000","last_unstale":"2024-09-17T04:24:14.063634+0000","last_undegraded":"2024-09-17T04:24:14.063634+0000","last_fullsized":"2024-09-17T04:24:14.063634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:23:31.947281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.033932+0000","last_change":"2024-09-17T04:24:14.033932+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.033932+0000","last_undegraded":"2024-09-17T04:24:14.033932+0000","last_fullsized":"2024-09-17T04:24:14.033932+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T04:24:13.024464+0000","last_change":"2024-09-17T04:24:13.024464+0000","last_active":"2024-09-17T04:24:13.015626+0000","last_peered":"2024-09-17T04:24:13.015626+0000","last_clean":"2024-09-17T04:24:13.015626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:13.024464+0000","last_undegraded":"2024-09-17T04:24:13.024464+0000","last_fullsized":"2024-09-17T04:24:13.024464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224778+0000","last_change":"2024-09-17T04:24:15.064498+0000","last_active":"2024-09-17T04:24:16.224778+0000","last_peered":"2024-09-17T04:24:16.224778+0000","last_clean":"2024-09-17T04:24:16.224778+0000","last_became_active":"2024-09-17T04:24:15.062700+0000","last_became_peered":"2024-09-17T04:24:15.062700+0000","last_unstale":"2024-09-17T04:24:16.224778+0000","last_undegraded":"2024-09-17T04:24:16.224778+0000","last_fullsized":"2024-09-17T04:24:16.224778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:31.021313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T04:24:13.024450+0000","last_change":"2024-09-17T04:24:13.024450+0000","last_active":"2024-09-17T04:24:13.015626+0000","last_peered":"2024-09-17T04:24:13.015626+0000","last_clean":"2024-09-17T04:24:13.015626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:13.024450+0000","last_undegraded":"2024-09-17T04:24:13.024450+0000","last_fullsized":"2024-09-17T04:24:13.024450+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225575+0000","last_change":"2024-09-17T04:24:15.060939+0000","last_active":"2024-09-17T04:24:16.225575+0000","last_peered":"2024-09-17T04:24:16.225575+0000","last_clean":"2024-09-17T04:24:16.225575+0000","last_became_active":"2024-09-17T04:24:15.060600+0000","last_became_peered":"2024-09-17T04:24:15.060600+0000","last_unstale":"2024-09-17T04:24:16.225575+0000","last_undegraded":"2024-09-17T04:24:16.225575+0000","last_fullsized":"2024-09-17T04:24:16.225575+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:05.895665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.067686+0000","last_change":"2024-09-17T04:24:14.062616+0000","last_active":"2024-09-17T04:24:14.067686+0000","last_peered":"2024-09-17T04:24:14.067686+0000","last_clean":"2024-09-17T04:24:14.067686+0000","last_became_active":"2024-09-17T04:24:14.062182+0000","last_became_peered":"2024-09-17T04:24:14.062182+0000","last_unstale":"2024-09-17T04:24:14.067686+0000","last_undegraded":"2024-09-17T04:24:14.067686+0000","last_fullsized":"2024-09-17T04:24:14.067686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:42.267414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.043034+0000","last_change":"2024-09-17T04:24:14.043034+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.043034+0000","last_undegraded":"2024-09-17T04:24:14.043034+0000","last_fullsized":"2024-09-17T04:24:14.043034+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.045081+0000","last_change":"2024-09-17T04:24:14.045081+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.045081+0000","last_undegraded":"2024-09-17T04:24:14.045081+0000","last_fullsized":"2024-09-17T04:24:14.045081+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.068177+0000","last_change":"2024-09-17T04:24:14.065850+0000","last_active":"2024-09-17T04:24:14.068177+0000","last_peered":"2024-09-17T04:24:14.068177+0000","last_clean":"2024-09-17T04:24:14.068177+0000","last_became_active":"2024-09-17T04:24:14.065444+0000","last_became_peered":"2024-09-17T04:24:14.065444+0000","last_unstale":"2024-09-17T04:24:14.068177+0000","last_undegraded":"2024-09-17T04:24:14.068177+0000","last_fullsized":"2024-09-17T04:24:14.068177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:51.429181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.046655+0000","last_change":"2024-09-17T04:24:14.046655+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.046655+0000","last_undegraded":"2024-09-17T04:24:14.046655+0000","last_fullsized":"2024-09-17T04:24:14.046655+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.062937+0000","last_change":"2024-09-17T04:24:14.060131+0000","last_active":"2024-09-17T04:24:14.062937+0000","last_peered":"2024-09-17T04:24:14.062937+0000","last_clean":"2024-09-17T04:24:14.062937+0000","last_became_active":"2024-09-17T04:24:14.059707+0000","last_became_peered":"2024-09-17T04:24:14.059707+0000","last_unstale":"2024-09-17T04:24:14.062937+0000","last_undegraded":"2024-09-17T04:24:14.062937+0000","last_fullsized":"2024-09-17T04:24:14.062937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:28.196011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225369+0000","last_change":"2024-09-17T04:24:14.056735+0000","last_active":"2024-09-17T04:24:16.225369+0000","last_peered":"2024-09-17T04:24:16.225369+0000","last_clean":"2024-09-17T04:24:16.225369+0000","last_became_active":"2024-09-17T04:24:14.056433+0000","last_became_peered":"2024-09-17T04:24:14.056433+0000","last_unstale":"2024-09-17T04:24:16.225369+0000","last_undegraded":"2024-09-17T04:24:16.225369+0000","last_fullsized":"2024-09-17T04:24:16.225369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:03.618384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225403+0000","last_change":"2024-09-17T04:24:15.054773+0000","last_active":"2024-09-17T04:24:16.225403+0000","last_peered":"2024-09-17T04:24:16.225403+0000","last_clean":"2024-09-17T04:24:16.225403+0000","last_became_active":"2024-09-17T04:24:15.054453+0000","last_became_peered":"2024-09-17T04:24:15.054453+0000","last_unstale":"2024-09-17T04:24:16.225403+0000","last_undegraded":"2024-09-17T04:24:16.225403+0000","last_fullsized":"2024-09-17T04:24:16.225403+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:58.942093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T04:24:13.029115+0000","last_change":"2024-09-17T04:24:13.029115+0000","last_active":"2024-09-17T04:24:13.015626+0000","last_peered":"2024-09-17T04:24:13.015626+0000","last_clean":"2024-09-17T04:24:13.015626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:13.029115+0000","last_undegraded":"2024-09-17T04:24:13.029115+0000","last_fullsized":"2024-09-17T04:24:13.029115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":92,"acting":92,"num_store_stats":5},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476742,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":786432,"data_stored":650853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476741,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":60485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476742,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":60485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476742,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":60485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":180224,"data_stored":54668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":54668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":774144,"data_stored":645036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":81447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:24:19.459 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:24:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.714+0000 7f23acc00640 1 -- 172.21.15.100:0/2352482987 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23a80aebc0 msgr2=0x7f23a8152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.714+0000 7f23acc00640 1 --2- 172.21.15.100:0/2352482987 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23a80aebc0 0x7f23a8152920 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2398005fd0 tx=0x7f239802a540 comp rx=0 tx=0).stop 2024-09-17T04:24:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.714+0000 7f23acc00640 1 -- 172.21.15.100:0/2352482987 shutdown_connections 2024-09-17T04:24:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.714+0000 7f23acc00640 1 --2- 172.21.15.100:0/2352482987 >> v2:172.21.15.184:3300/0 conn(0x7f23a80b2750 0x7f23a815f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.714+0000 7f23acc00640 1 --2- 172.21.15.100:0/2352482987 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23a80aebc0 0x7f23a8152920 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.714+0000 7f23acc00640 1 -- 172.21.15.100:0/2352482987 >> 172.21.15.100:0/2352482987 conn(0x7f23a806d730 msgr2=0x7f23a80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:19.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.714+0000 7f23acc00640 1 -- 172.21.15.100:0/2352482987 shutdown_connections 2024-09-17T04:24:19.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.714+0000 7f23acc00640 1 -- 172.21.15.100:0/2352482987 wait complete. 2024-09-17T04:24:19.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.717+0000 7f23acc00640 1 Processor -- start 2024-09-17T04:24:19.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.717+0000 7f23acc00640 1 -- start start 2024-09-17T04:24:19.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.718+0000 7f23acc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23a80aebc0 0x7f23a81f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:19.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.718+0000 7f23acc00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23a80b2750 0x7f23a8074180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:19.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.718+0000 7f23acc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23a815f910 0x7f23a8074970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:19.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.718+0000 7f23acc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f23a815c4a0 con 0x7f23a815f910 2024-09-17T04:24:19.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.718+0000 7f23acc00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f23a80b4a20 con 0x7f23a80aebc0 2024-09-17T04:24:19.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.718+0000 7f23acc00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f23a815c620 con 0x7f23a80b2750 2024-09-17T04:24:19.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.718+0000 7f23a6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23a815f910 0x7f23a8074970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:19.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.718+0000 7f23a6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23a815f910 0x7f23a8074970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41582/0 (socket says 172.21.15.100:41582) 2024-09-17T04:24:19.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.718+0000 7f23a6800640 1 -- 172.21.15.100:0/3102719357 learned_addr learned my addr 172.21.15.100:0/3102719357 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:19.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.719+0000 7f23a5e00640 1 --2- 172.21.15.100:0/3102719357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23a80aebc0 0x7f23a81f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:19.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.719+0000 7f239d400640 1 --2- 172.21.15.100:0/3102719357 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23a80b2750 0x7f23a8074180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:19.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.719+0000 7f23a5e00640 1 -- 172.21.15.100:0/3102719357 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23a80b2750 msgr2=0x7f23a8074180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:19.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.719+0000 7f23a5e00640 1 --2- 172.21.15.100:0/3102719357 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23a80b2750 0x7f23a8074180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:19.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.719+0000 7f23a5e00640 1 -- 172.21.15.100:0/3102719357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23a815f910 msgr2=0x7f23a8074970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:19.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.719+0000 7f23a5e00640 1 --2- 172.21.15.100:0/3102719357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23a815f910 0x7f23a8074970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:19.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.719+0000 7f23a5e00640 1 -- 172.21.15.100:0/3102719357 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a80730e0 con 0x7f23a80aebc0 2024-09-17T04:24:19.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.719+0000 7f23a5e00640 1 --2- 172.21.15.100:0/3102719357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23a80aebc0 0x7f23a81f61c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2398031000 tx=0x7f2398004c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:19.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.720+0000 7f239ea00640 1 -- 172.21.15.100:0/3102719357 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239803e030 con 0x7f23a80aebc0 2024-09-17T04:24:19.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.720+0000 7f239ea00640 1 -- 172.21.15.100:0/3102719357 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2398005040 con 0x7f23a80aebc0 2024-09-17T04:24:19.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.720+0000 7f239ea00640 1 -- 172.21.15.100:0/3102719357 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2398035040 con 0x7f23a80aebc0 2024-09-17T04:24:19.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.720+0000 7f23acc00640 1 -- 172.21.15.100:0/3102719357 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23a80733c0 con 0x7f23a80aebc0 2024-09-17T04:24:19.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.720+0000 7f23acc00640 1 -- 172.21.15.100:0/3102719357 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23a8073980 con 0x7f23a80aebc0 2024-09-17T04:24:19.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.721+0000 7f23acc00640 1 -- 172.21.15.100:0/3102719357 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23a815cb40 con 0x7f23a80aebc0 2024-09-17T04:24:19.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.722+0000 7f239ea00640 1 -- 172.21.15.100:0/3102719357 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2398041030 con 0x7f23a80aebc0 2024-09-17T04:24:19.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.723+0000 7f239ea00640 1 --2- 172.21.15.100:0/3102719357 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2384078bc0 0x7f238407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:19.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.723+0000 7f239ea00640 1 -- 172.21.15.100:0/3102719357 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f23980bb610 con 0x7f23a80aebc0 2024-09-17T04:24:19.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.724+0000 7f239d400640 1 --2- 172.21.15.100:0/3102719357 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2384078bc0 0x7f238407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:19.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.724+0000 7f239d400640 1 --2- 172.21.15.100:0/3102719357 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2384078bc0 0x7f238407b0d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2390009b30 tx=0x7f2390005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:19.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.728+0000 7f239ea00640 1 -- 172.21.15.100:0/3102719357 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f2398083570 con 0x7f23a80aebc0 2024-09-17T04:24:19.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.909+0000 7f23acc00640 1 -- 172.21.15.100:0/3102719357 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f23a8160110 con 0x7f2384078bc0 2024-09-17T04:24:19.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.915+0000 7f239ea00640 1 -- 172.21.15.100:0/3102719357 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245292 (secure 0 0 0) 0x7f23a8160110 con 0x7f2384078bc0 2024-09-17T04:24:19.918 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:19.919 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:24:19.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.919+0000 7f23acc00640 1 -- 172.21.15.100:0/3102719357 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2384078bc0 msgr2=0x7f238407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:19.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.919+0000 7f23acc00640 1 --2- 172.21.15.100:0/3102719357 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2384078bc0 0x7f238407b0d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2390009b30 tx=0x7f2390005f90 comp rx=0 tx=0).stop 2024-09-17T04:24:19.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.919+0000 7f23acc00640 1 -- 172.21.15.100:0/3102719357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23a80aebc0 msgr2=0x7f23a81f61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:19.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.919+0000 7f23acc00640 1 --2- 172.21.15.100:0/3102719357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23a80aebc0 0x7f23a81f61c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2398031000 tx=0x7f2398004c40 comp rx=0 tx=0).stop 2024-09-17T04:24:19.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.919+0000 7f23acc00640 1 -- 172.21.15.100:0/3102719357 shutdown_connections 2024-09-17T04:24:19.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.919+0000 7f23acc00640 1 --2- 172.21.15.100:0/3102719357 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2384078bc0 0x7f238407b0d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:19.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.919+0000 7f23acc00640 1 --2- 172.21.15.100:0/3102719357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23a815f910 0x7f23a8074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:19.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.919+0000 7f23acc00640 1 --2- 172.21.15.100:0/3102719357 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23a80b2750 0x7f23a8074180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:19.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.919+0000 7f23acc00640 1 --2- 172.21.15.100:0/3102719357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23a80aebc0 0x7f23a81f61c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:19.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.919+0000 7f23acc00640 1 -- 172.21.15.100:0/3102719357 >> 172.21.15.100:0/3102719357 conn(0x7f23a806d730 msgr2=0x7f23a815bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:19.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.919+0000 7f23acc00640 1 -- 172.21.15.100:0/3102719357 shutdown_connections 2024-09-17T04:24:19.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:19.919+0000 7f23acc00640 1 -- 172.21.15.100:0/3102719357 wait complete. 2024-09-17T04:24:19.943 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-17T04:24:17.964480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":158,"acting":158,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219156,"kb_used_data":2676,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755564,"statfs":{"total":773094113280,"available":772869697536,"internally_reserved":0,"allocated":2740224,"data_stored":1668127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.944839"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.041403+0000","last_change":"2024-09-17T04:24:14.041403+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.041403+0000","last_undegraded":"2024-09-17T04:24:14.041403+0000","last_fullsized":"2024-09-17T04:24:14.041403+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.049136+0000","last_change":"2024-09-17T04:24:14.049136+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.049136+0000","last_undegraded":"2024-09-17T04:24:14.049136+0000","last_fullsized":"2024-09-17T04:24:14.049136+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.039709+0000","last_change":"2024-09-17T04:24:14.039709+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.039709+0000","last_undegraded":"2024-09-17T04:24:14.039709+0000","last_fullsized":"2024-09-17T04:24:14.039709+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.048608+0000","last_change":"2024-09-17T04:24:14.048608+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.048608+0000","last_undegraded":"2024-09-17T04:24:14.048608+0000","last_fullsized":"2024-09-17T04:24:14.048608+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.044340+0000","last_change":"2024-09-17T04:24:14.044340+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.044340+0000","last_undegraded":"2024-09-17T04:24:14.044340+0000","last_fullsized":"2024-09-17T04:24:14.044340+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224308+0000","last_change":"2024-09-17T04:24:15.062993+0000","last_active":"2024-09-17T04:24:16.224308+0000","last_peered":"2024-09-17T04:24:16.224308+0000","last_clean":"2024-09-17T04:24:16.224308+0000","last_became_active":"2024-09-17T04:24:15.062311+0000","last_became_peered":"2024-09-17T04:24:15.062311+0000","last_unstale":"2024-09-17T04:24:16.224308+0000","last_undegraded":"2024-09-17T04:24:16.224308+0000","last_fullsized":"2024-09-17T04:24:16.224308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:26:42.504856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.040807+0000","last_change":"2024-09-17T04:24:14.040807+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.040807+0000","last_undegraded":"2024-09-17T04:24:14.040807+0000","last_fullsized":"2024-09-17T04:24:14.040807+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.045923+0000","last_change":"2024-09-17T04:24:14.045923+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.045923+0000","last_undegraded":"2024-09-17T04:24:14.045923+0000","last_fullsized":"2024-09-17T04:24:14.045923+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.039794+0000","last_change":"2024-09-17T04:24:14.039794+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.039794+0000","last_undegraded":"2024-09-17T04:24:14.039794+0000","last_fullsized":"2024-09-17T04:24:14.039794+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.034825+0000","last_change":"2024-09-17T04:24:14.034825+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.034825+0000","last_undegraded":"2024-09-17T04:24:14.034825+0000","last_fullsized":"2024-09-17T04:24:14.034825+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225225+0000","last_change":"2024-09-17T04:24:15.047951+0000","last_active":"2024-09-17T04:24:16.225225+0000","last_peered":"2024-09-17T04:24:16.225225+0000","last_clean":"2024-09-17T04:24:16.225225+0000","last_became_active":"2024-09-17T04:24:15.047536+0000","last_became_peered":"2024-09-17T04:24:15.047536+0000","last_unstale":"2024-09-17T04:24:16.225225+0000","last_undegraded":"2024-09-17T04:24:16.225225+0000","last_fullsized":"2024-09-17T04:24:16.225225+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:38:48.386032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.047511+0000","last_change":"2024-09-17T04:24:14.047511+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.047511+0000","last_undegraded":"2024-09-17T04:24:14.047511+0000","last_fullsized":"2024-09-17T04:24:14.047511+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.045559+0000","last_change":"2024-09-17T04:24:14.045559+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.045559+0000","last_undegraded":"2024-09-17T04:24:14.045559+0000","last_fullsized":"2024-09-17T04:24:14.045559+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.042563+0000","last_change":"2024-09-17T04:24:14.042563+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.042563+0000","last_undegraded":"2024-09-17T04:24:14.042563+0000","last_fullsized":"2024-09-17T04:24:14.042563+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.041977+0000","last_change":"2024-09-17T04:24:14.041977+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.041977+0000","last_undegraded":"2024-09-17T04:24:14.041977+0000","last_fullsized":"2024-09-17T04:24:14.041977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.044137+0000","last_change":"2024-09-17T04:24:14.044137+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.044137+0000","last_undegraded":"2024-09-17T04:24:14.044137+0000","last_fullsized":"2024-09-17T04:24:14.044137+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224382+0000","last_change":"2024-09-17T04:24:15.070200+0000","last_active":"2024-09-17T04:24:16.224382+0000","last_peered":"2024-09-17T04:24:16.224382+0000","last_clean":"2024-09-17T04:24:16.224382+0000","last_became_active":"2024-09-17T04:24:15.069872+0000","last_became_peered":"2024-09-17T04:24:15.069872+0000","last_unstale":"2024-09-17T04:24:16.224382+0000","last_undegraded":"2024-09-17T04:24:16.224382+0000","last_fullsized":"2024-09-17T04:24:16.224382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:22.880022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.046960+0000","last_change":"2024-09-17T04:24:14.046960+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.046960+0000","last_undegraded":"2024-09-17T04:24:14.046960+0000","last_fullsized":"2024-09-17T04:24:14.046960+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225400+0000","last_change":"2024-09-17T04:24:15.074863+0000","last_active":"2024-09-17T04:24:16.225400+0000","last_peered":"2024-09-17T04:24:16.225400+0000","last_clean":"2024-09-17T04:24:16.225400+0000","last_became_active":"2024-09-17T04:24:15.074467+0000","last_became_peered":"2024-09-17T04:24:15.074467+0000","last_unstale":"2024-09-17T04:24:16.225400+0000","last_undegraded":"2024-09-17T04:24:16.225400+0000","last_fullsized":"2024-09-17T04:24:16.225400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:36:59.215343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.044399+0000","last_change":"2024-09-17T04:24:14.044399+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.044399+0000","last_undegraded":"2024-09-17T04:24:14.044399+0000","last_fullsized":"2024-09-17T04:24:14.044399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225430+0000","last_change":"2024-09-17T04:24:15.075902+0000","last_active":"2024-09-17T04:24:16.225430+0000","last_peered":"2024-09-17T04:24:16.225430+0000","last_clean":"2024-09-17T04:24:16.225430+0000","last_became_active":"2024-09-17T04:24:15.075625+0000","last_became_peered":"2024-09-17T04:24:15.075625+0000","last_unstale":"2024-09-17T04:24:16.225430+0000","last_undegraded":"2024-09-17T04:24:16.225430+0000","last_fullsized":"2024-09-17T04:24:16.225430+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:34.202614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.039484+0000","last_change":"2024-09-17T04:24:14.039484+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.039484+0000","last_undegraded":"2024-09-17T04:24:14.039484+0000","last_fullsized":"2024-09-17T04:24:14.039484+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.042670+0000","last_change":"2024-09-17T04:24:14.042670+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.042670+0000","last_undegraded":"2024-09-17T04:24:14.042670+0000","last_fullsized":"2024-09-17T04:24:14.042670+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.034151+0000","last_change":"2024-09-17T04:24:14.034151+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.034151+0000","last_undegraded":"2024-09-17T04:24:14.034151+0000","last_fullsized":"2024-09-17T04:24:14.034151+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225374+0000","last_change":"2024-09-17T04:24:14.062806+0000","last_active":"2024-09-17T04:24:16.225374+0000","last_peered":"2024-09-17T04:24:16.225374+0000","last_clean":"2024-09-17T04:24:16.225374+0000","last_became_active":"2024-09-17T04:24:14.062282+0000","last_became_peered":"2024-09-17T04:24:14.062282+0000","last_unstale":"2024-09-17T04:24:16.225374+0000","last_undegraded":"2024-09-17T04:24:16.225374+0000","last_fullsized":"2024-09-17T04:24:16.225374+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:23.698652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.058753+0000","last_change":"2024-09-17T04:24:14.056222+0000","last_active":"2024-09-17T04:24:14.058753+0000","last_peered":"2024-09-17T04:24:14.058753+0000","last_clean":"2024-09-17T04:24:14.058753+0000","last_became_active":"2024-09-17T04:24:14.055874+0000","last_became_peered":"2024-09-17T04:24:14.055874+0000","last_unstale":"2024-09-17T04:24:14.058753+0000","last_undegraded":"2024-09-17T04:24:14.058753+0000","last_fullsized":"2024-09-17T04:24:14.058753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:22.510592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224568+0000","last_change":"2024-09-17T04:24:14.054189+0000","last_active":"2024-09-17T04:24:16.224568+0000","last_peered":"2024-09-17T04:24:16.224568+0000","last_clean":"2024-09-17T04:24:16.224568+0000","last_became_active":"2024-09-17T04:24:14.053870+0000","last_became_peered":"2024-09-17T04:24:14.053870+0000","last_unstale":"2024-09-17T04:24:16.224568+0000","last_undegraded":"2024-09-17T04:24:16.224568+0000","last_fullsized":"2024-09-17T04:24:16.224568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:12.776180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224542+0000","last_change":"2024-09-17T04:24:15.059936+0000","last_active":"2024-09-17T04:24:16.224542+0000","last_peered":"2024-09-17T04:24:16.224542+0000","last_clean":"2024-09-17T04:24:16.224542+0000","last_became_active":"2024-09-17T04:24:15.059645+0000","last_became_peered":"2024-09-17T04:24:15.059645+0000","last_unstale":"2024-09-17T04:24:16.224542+0000","last_undegraded":"2024-09-17T04:24:16.224542+0000","last_fullsized":"2024-09-17T04:24:16.224542+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:08:11.703658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.064116+0000","last_change":"2024-09-17T04:24:14.060082+0000","last_active":"2024-09-17T04:24:14.064116+0000","last_peered":"2024-09-17T04:24:14.064116+0000","last_clean":"2024-09-17T04:24:14.064116+0000","last_became_active":"2024-09-17T04:24:14.058795+0000","last_became_peered":"2024-09-17T04:24:14.058795+0000","last_unstale":"2024-09-17T04:24:14.064116+0000","last_undegraded":"2024-09-17T04:24:14.064116+0000","last_fullsized":"2024-09-17T04:24:14.064116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:44.734015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.056597+0000","last_change":"2024-09-17T04:24:14.053595+0000","last_active":"2024-09-17T04:24:14.056597+0000","last_peered":"2024-09-17T04:24:14.056597+0000","last_clean":"2024-09-17T04:24:14.056597+0000","last_became_active":"2024-09-17T04:24:14.053260+0000","last_became_peered":"2024-09-17T04:24:14.053260+0000","last_unstale":"2024-09-17T04:24:14.056597+0000","last_undegraded":"2024-09-17T04:24:14.056597+0000","last_fullsized":"2024-09-17T04:24:14.056597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:29.564993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.043231+0000","last_change":"2024-09-17T04:24:14.043231+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.043231+0000","last_undegraded":"2024-09-17T04:24:14.043231+0000","last_fullsized":"2024-09-17T04:24:14.043231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.064090+0000","last_change":"2024-09-17T04:24:14.059203+0000","last_active":"2024-09-17T04:24:14.064090+0000","last_peered":"2024-09-17T04:24:14.064090+0000","last_clean":"2024-09-17T04:24:14.064090+0000","last_became_active":"2024-09-17T04:24:14.058623+0000","last_became_peered":"2024-09-17T04:24:14.058623+0000","last_unstale":"2024-09-17T04:24:14.064090+0000","last_undegraded":"2024-09-17T04:24:14.064090+0000","last_fullsized":"2024-09-17T04:24:14.064090+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:28.196011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T04:24:13.026983+0000","last_change":"2024-09-17T04:24:13.026983+0000","last_active":"2024-09-17T04:24:13.015626+0000","last_peered":"2024-09-17T04:24:13.015626+0000","last_clean":"2024-09-17T04:24:13.015626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:13.026983+0000","last_undegraded":"2024-09-17T04:24:13.026983+0000","last_fullsized":"2024-09-17T04:24:13.026983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.033988+0000","last_change":"2024-09-17T04:24:14.033988+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.033988+0000","last_undegraded":"2024-09-17T04:24:14.033988+0000","last_fullsized":"2024-09-17T04:24:14.033988+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.060560+0000","last_change":"2024-09-17T04:24:14.058729+0000","last_active":"2024-09-17T04:24:14.060560+0000","last_peered":"2024-09-17T04:24:14.060560+0000","last_clean":"2024-09-17T04:24:14.060560+0000","last_became_active":"2024-09-17T04:24:14.057616+0000","last_became_peered":"2024-09-17T04:24:14.057616+0000","last_unstale":"2024-09-17T04:24:14.060560+0000","last_undegraded":"2024-09-17T04:24:14.060560+0000","last_fullsized":"2024-09-17T04:24:14.060560+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:23:31.947281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.043536+0000","last_change":"2024-09-17T04:24:14.043536+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.043536+0000","last_undegraded":"2024-09-17T04:24:14.043536+0000","last_fullsized":"2024-09-17T04:24:14.043536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T04:24:13.029149+0000","last_change":"2024-09-17T04:24:13.029149+0000","last_active":"2024-09-17T04:24:13.015626+0000","last_peered":"2024-09-17T04:24:13.015626+0000","last_clean":"2024-09-17T04:24:13.015626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:13.029149+0000","last_undegraded":"2024-09-17T04:24:13.029149+0000","last_fullsized":"2024-09-17T04:24:13.029149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.064018+0000","last_change":"2024-09-17T04:24:14.058127+0000","last_active":"2024-09-17T04:24:14.064018+0000","last_peered":"2024-09-17T04:24:14.064018+0000","last_clean":"2024-09-17T04:24:14.064018+0000","last_became_active":"2024-09-17T04:24:14.055990+0000","last_became_peered":"2024-09-17T04:24:14.055990+0000","last_unstale":"2024-09-17T04:24:14.064018+0000","last_undegraded":"2024-09-17T04:24:14.064018+0000","last_fullsized":"2024-09-17T04:24:14.064018+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:23:31.947281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.037775+0000","last_change":"2024-09-17T04:24:14.037775+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.037775+0000","last_undegraded":"2024-09-17T04:24:14.037775+0000","last_fullsized":"2024-09-17T04:24:14.037775+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.053546+0000","last_change":"2024-09-17T04:24:14.051949+0000","last_active":"2024-09-17T04:24:14.053546+0000","last_peered":"2024-09-17T04:24:14.053546+0000","last_clean":"2024-09-17T04:24:14.053546+0000","last_became_active":"2024-09-17T04:24:14.051548+0000","last_became_peered":"2024-09-17T04:24:14.051548+0000","last_unstale":"2024-09-17T04:24:14.053546+0000","last_undegraded":"2024-09-17T04:24:14.053546+0000","last_fullsized":"2024-09-17T04:24:14.053546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:19:57.759163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.039461+0000","last_change":"2024-09-17T04:24:14.039461+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.039461+0000","last_undegraded":"2024-09-17T04:24:14.039461+0000","last_fullsized":"2024-09-17T04:24:14.039461+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T04:24:13.021867+0000","last_change":"2024-09-17T04:24:13.021867+0000","last_active":"2024-09-17T04:24:13.015626+0000","last_peered":"2024-09-17T04:24:13.015626+0000","last_clean":"2024-09-17T04:24:13.015626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:13.021867+0000","last_undegraded":"2024-09-17T04:24:13.021867+0000","last_fullsized":"2024-09-17T04:24:13.021867+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.037601+0000","last_change":"2024-09-17T04:24:14.037601+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.037601+0000","last_undegraded":"2024-09-17T04:24:14.037601+0000","last_fullsized":"2024-09-17T04:24:14.037601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.063831+0000","last_change":"2024-09-17T04:24:14.059673+0000","last_active":"2024-09-17T04:24:14.063831+0000","last_peered":"2024-09-17T04:24:14.063831+0000","last_clean":"2024-09-17T04:24:14.063831+0000","last_became_active":"2024-09-17T04:24:14.057668+0000","last_became_peered":"2024-09-17T04:24:14.057668+0000","last_unstale":"2024-09-17T04:24:14.063831+0000","last_undegraded":"2024-09-17T04:24:14.063831+0000","last_fullsized":"2024-09-17T04:24:14.063831+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:18:36.484153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.039797+0000","last_change":"2024-09-17T04:24:14.039797+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.039797+0000","last_undegraded":"2024-09-17T04:24:14.039797+0000","last_fullsized":"2024-09-17T04:24:14.039797+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T04:24:13.021874+0000","last_change":"2024-09-17T04:24:13.021874+0000","last_active":"2024-09-17T04:24:13.015626+0000","last_peered":"2024-09-17T04:24:13.015626+0000","last_clean":"2024-09-17T04:24:13.015626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:13.021874+0000","last_undegraded":"2024-09-17T04:24:13.021874+0000","last_fullsized":"2024-09-17T04:24:13.021874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.039784+0000","last_change":"2024-09-17T04:24:14.039784+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.039784+0000","last_undegraded":"2024-09-17T04:24:14.039784+0000","last_fullsized":"2024-09-17T04:24:14.039784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T04:24:13.024980+0000","last_change":"2024-09-17T04:24:13.024980+0000","last_active":"2024-09-17T04:24:13.015626+0000","last_peered":"2024-09-17T04:24:13.015626+0000","last_clean":"2024-09-17T04:24:13.015626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:13.024980+0000","last_undegraded":"2024-09-17T04:24:13.024980+0000","last_fullsized":"2024-09-17T04:24:13.024980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.046138+0000","last_change":"2024-09-17T04:24:14.046138+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.046138+0000","last_undegraded":"2024-09-17T04:24:14.046138+0000","last_fullsized":"2024-09-17T04:24:14.046138+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T04:24:13.021850+0000","last_change":"2024-09-17T04:24:13.021850+0000","last_active":"2024-09-17T04:24:13.015626+0000","last_peered":"2024-09-17T04:24:13.015626+0000","last_clean":"2024-09-17T04:24:13.015626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:13.021850+0000","last_undegraded":"2024-09-17T04:24:13.021850+0000","last_fullsized":"2024-09-17T04:24:13.021850+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.043031+0000","last_change":"2024-09-17T04:24:14.043031+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.043031+0000","last_undegraded":"2024-09-17T04:24:14.043031+0000","last_fullsized":"2024-09-17T04:24:14.043031+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":107,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.050263+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:24:14.050263+0000","last_peered":"2024-09-17T04:24:14.050263+0000","last_clean":"2024-09-17T04:24:14.050263+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:24:14.050263+0000","last_undegraded":"2024-09-17T04:24:14.050263+0000","last_fullsized":"2024-09-17T04:24:14.050263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.053358+0000","last_change":"2024-09-17T04:24:14.049088+0000","last_active":"2024-09-17T04:24:14.053358+0000","last_peered":"2024-09-17T04:24:14.053358+0000","last_clean":"2024-09-17T04:24:14.053358+0000","last_became_active":"2024-09-17T04:24:14.048529+0000","last_became_peered":"2024-09-17T04:24:14.048529+0000","last_unstale":"2024-09-17T04:24:14.053358+0000","last_undegraded":"2024-09-17T04:24:14.053358+0000","last_fullsized":"2024-09-17T04:24:14.053358+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:29.564993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225083+0000","last_change":"2024-09-17T04:24:15.065699+0000","last_active":"2024-09-17T04:24:16.225083+0000","last_peered":"2024-09-17T04:24:16.225083+0000","last_clean":"2024-09-17T04:24:16.225083+0000","last_became_active":"2024-09-17T04:24:15.061569+0000","last_became_peered":"2024-09-17T04:24:15.061569+0000","last_unstale":"2024-09-17T04:24:16.225083+0000","last_undegraded":"2024-09-17T04:24:16.225083+0000","last_fullsized":"2024-09-17T04:24:16.225083+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:01:19.496265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.064651+0000","last_change":"2024-09-17T04:24:14.061081+0000","last_active":"2024-09-17T04:24:14.064651+0000","last_peered":"2024-09-17T04:24:14.064651+0000","last_clean":"2024-09-17T04:24:14.064651+0000","last_became_active":"2024-09-17T04:24:14.060394+0000","last_became_peered":"2024-09-17T04:24:14.060394+0000","last_unstale":"2024-09-17T04:24:14.064651+0000","last_undegraded":"2024-09-17T04:24:14.064651+0000","last_fullsized":"2024-09-17T04:24:14.064651+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:44.734015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T04:24:13.021863+0000","last_change":"2024-09-17T04:24:13.021863+0000","last_active":"2024-09-17T04:24:13.015626+0000","last_peered":"2024-09-17T04:24:13.015626+0000","last_clean":"2024-09-17T04:24:13.015626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:13.021863+0000","last_undegraded":"2024-09-17T04:24:13.021863+0000","last_fullsized":"2024-09-17T04:24:13.021863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.061385+0000","last_change":"2024-09-17T04:24:14.053751+0000","last_active":"2024-09-17T04:24:14.061385+0000","last_peered":"2024-09-17T04:24:14.061385+0000","last_clean":"2024-09-17T04:24:14.061385+0000","last_became_active":"2024-09-17T04:24:14.051943+0000","last_became_peered":"2024-09-17T04:24:14.051943+0000","last_unstale":"2024-09-17T04:24:14.061385+0000","last_undegraded":"2024-09-17T04:24:14.061385+0000","last_fullsized":"2024-09-17T04:24:14.061385+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:22.510592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.033873+0000","last_change":"2024-09-17T04:24:14.033873+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.033873+0000","last_undegraded":"2024-09-17T04:24:14.033873+0000","last_fullsized":"2024-09-17T04:24:14.033873+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T04:24:13.024681+0000","last_change":"2024-09-17T04:24:13.024681+0000","last_active":"2024-09-17T04:24:13.015626+0000","last_peered":"2024-09-17T04:24:13.015626+0000","last_clean":"2024-09-17T04:24:13.015626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:13.024681+0000","last_undegraded":"2024-09-17T04:24:13.024681+0000","last_fullsized":"2024-09-17T04:24:13.024681+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.044390+0000","last_change":"2024-09-17T04:24:14.044390+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.044390+0000","last_undegraded":"2024-09-17T04:24:14.044390+0000","last_fullsized":"2024-09-17T04:24:14.044390+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.067724+0000","last_change":"2024-09-17T04:24:14.063354+0000","last_active":"2024-09-17T04:24:14.067724+0000","last_peered":"2024-09-17T04:24:14.067724+0000","last_clean":"2024-09-17T04:24:14.067724+0000","last_became_active":"2024-09-17T04:24:14.062231+0000","last_became_peered":"2024-09-17T04:24:14.062231+0000","last_unstale":"2024-09-17T04:24:14.067724+0000","last_undegraded":"2024-09-17T04:24:14.067724+0000","last_fullsized":"2024-09-17T04:24:14.067724+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:21.535057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T04:24:13.024741+0000","last_change":"2024-09-17T04:24:13.024741+0000","last_active":"2024-09-17T04:24:13.015626+0000","last_peered":"2024-09-17T04:24:13.015626+0000","last_clean":"2024-09-17T04:24:13.015626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:13.024741+0000","last_undegraded":"2024-09-17T04:24:13.024741+0000","last_fullsized":"2024-09-17T04:24:13.024741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225582+0000","last_change":"2024-09-17T04:24:15.050480+0000","last_active":"2024-09-17T04:24:16.225582+0000","last_peered":"2024-09-17T04:24:16.225582+0000","last_clean":"2024-09-17T04:24:16.225582+0000","last_became_active":"2024-09-17T04:24:15.050166+0000","last_became_peered":"2024-09-17T04:24:15.050166+0000","last_unstale":"2024-09-17T04:24:16.225582+0000","last_undegraded":"2024-09-17T04:24:16.225582+0000","last_fullsized":"2024-09-17T04:24:16.225582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:02.125884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.067446+0000","last_change":"2024-09-17T04:24:14.063222+0000","last_active":"2024-09-17T04:24:14.067446+0000","last_peered":"2024-09-17T04:24:14.067446+0000","last_clean":"2024-09-17T04:24:14.067446+0000","last_became_active":"2024-09-17T04:24:14.062750+0000","last_became_peered":"2024-09-17T04:24:14.062750+0000","last_unstale":"2024-09-17T04:24:14.067446+0000","last_undegraded":"2024-09-17T04:24:14.067446+0000","last_fullsized":"2024-09-17T04:24:14.067446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:28.196011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.063634+0000","last_change":"2024-09-17T04:24:14.059487+0000","last_active":"2024-09-17T04:24:14.063634+0000","last_peered":"2024-09-17T04:24:14.063634+0000","last_clean":"2024-09-17T04:24:14.063634+0000","last_became_active":"2024-09-17T04:24:14.059186+0000","last_became_peered":"2024-09-17T04:24:14.059186+0000","last_unstale":"2024-09-17T04:24:14.063634+0000","last_undegraded":"2024-09-17T04:24:14.063634+0000","last_fullsized":"2024-09-17T04:24:14.063634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:23:31.947281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.033932+0000","last_change":"2024-09-17T04:24:14.033932+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.033932+0000","last_undegraded":"2024-09-17T04:24:14.033932+0000","last_fullsized":"2024-09-17T04:24:14.033932+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T04:24:13.024464+0000","last_change":"2024-09-17T04:24:13.024464+0000","last_active":"2024-09-17T04:24:13.015626+0000","last_peered":"2024-09-17T04:24:13.015626+0000","last_clean":"2024-09-17T04:24:13.015626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:13.024464+0000","last_undegraded":"2024-09-17T04:24:13.024464+0000","last_fullsized":"2024-09-17T04:24:13.024464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224778+0000","last_change":"2024-09-17T04:24:15.064498+0000","last_active":"2024-09-17T04:24:16.224778+0000","last_peered":"2024-09-17T04:24:16.224778+0000","last_clean":"2024-09-17T04:24:16.224778+0000","last_became_active":"2024-09-17T04:24:15.062700+0000","last_became_peered":"2024-09-17T04:24:15.062700+0000","last_unstale":"2024-09-17T04:24:16.224778+0000","last_undegraded":"2024-09-17T04:24:16.224778+0000","last_fullsized":"2024-09-17T04:24:16.224778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:31.021313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T04:24:13.024450+0000","last_change":"2024-09-17T04:24:13.024450+0000","last_active":"2024-09-17T04:24:13.015626+0000","last_peered":"2024-09-17T04:24:13.015626+0000","last_clean":"2024-09-17T04:24:13.015626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:13.024450+0000","last_undegraded":"2024-09-17T04:24:13.024450+0000","last_fullsized":"2024-09-17T04:24:13.024450+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225575+0000","last_change":"2024-09-17T04:24:15.060939+0000","last_active":"2024-09-17T04:24:16.225575+0000","last_peered":"2024-09-17T04:24:16.225575+0000","last_clean":"2024-09-17T04:24:16.225575+0000","last_became_active":"2024-09-17T04:24:15.060600+0000","last_became_peered":"2024-09-17T04:24:15.060600+0000","last_unstale":"2024-09-17T04:24:16.225575+0000","last_undegraded":"2024-09-17T04:24:16.225575+0000","last_fullsized":"2024-09-17T04:24:16.225575+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:05.895665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.067686+0000","last_change":"2024-09-17T04:24:14.062616+0000","last_active":"2024-09-17T04:24:14.067686+0000","last_peered":"2024-09-17T04:24:14.067686+0000","last_clean":"2024-09-17T04:24:14.067686+0000","last_became_active":"2024-09-17T04:24:14.062182+0000","last_became_peered":"2024-09-17T04:24:14.062182+0000","last_unstale":"2024-09-17T04:24:14.067686+0000","last_undegraded":"2024-09-17T04:24:14.067686+0000","last_fullsized":"2024-09-17T04:24:14.067686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:42.267414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.043034+0000","last_change":"2024-09-17T04:24:14.043034+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.043034+0000","last_undegraded":"2024-09-17T04:24:14.043034+0000","last_fullsized":"2024-09-17T04:24:14.043034+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.045081+0000","last_change":"2024-09-17T04:24:14.045081+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.045081+0000","last_undegraded":"2024-09-17T04:24:14.045081+0000","last_fullsized":"2024-09-17T04:24:14.045081+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.068177+0000","last_change":"2024-09-17T04:24:14.065850+0000","last_active":"2024-09-17T04:24:14.068177+0000","last_peered":"2024-09-17T04:24:14.068177+0000","last_clean":"2024-09-17T04:24:14.068177+0000","last_became_active":"2024-09-17T04:24:14.065444+0000","last_became_peered":"2024-09-17T04:24:14.065444+0000","last_unstale":"2024-09-17T04:24:14.068177+0000","last_undegraded":"2024-09-17T04:24:14.068177+0000","last_fullsized":"2024-09-17T04:24:14.068177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:51.429181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-17T04:24:14.046655+0000","last_change":"2024-09-17T04:24:14.046655+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.046655+0000","last_undegraded":"2024-09-17T04:24:14.046655+0000","last_fullsized":"2024-09-17T04:24:14.046655+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T04:24:14.062937+0000","last_change":"2024-09-17T04:24:14.060131+0000","last_active":"2024-09-17T04:24:14.062937+0000","last_peered":"2024-09-17T04:24:14.062937+0000","last_clean":"2024-09-17T04:24:14.062937+0000","last_became_active":"2024-09-17T04:24:14.059707+0000","last_became_peered":"2024-09-17T04:24:14.059707+0000","last_unstale":"2024-09-17T04:24:14.062937+0000","last_undegraded":"2024-09-17T04:24:14.062937+0000","last_fullsized":"2024-09-17T04:24:14.062937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:28.196011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:24:14.018386+0000","last_change":"2024-09-17T04:24:14.018386+0000","last_active":"2024-09-17T04:24:14.018386+0000","last_peered":"2024-09-17T04:24:14.018386+0000","last_clean":"2024-09-17T04:24:14.018386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:14.018386+0000","last_undegraded":"2024-09-17T04:24:14.018386+0000","last_fullsized":"2024-09-17T04:24:14.018386+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225369+0000","last_change":"2024-09-17T04:24:14.056735+0000","last_active":"2024-09-17T04:24:16.225369+0000","last_peered":"2024-09-17T04:24:16.225369+0000","last_clean":"2024-09-17T04:24:16.225369+0000","last_became_active":"2024-09-17T04:24:14.056433+0000","last_became_peered":"2024-09-17T04:24:14.056433+0000","last_unstale":"2024-09-17T04:24:16.225369+0000","last_undegraded":"2024-09-17T04:24:16.225369+0000","last_fullsized":"2024-09-17T04:24:16.225369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:03.618384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225403+0000","last_change":"2024-09-17T04:24:15.054773+0000","last_active":"2024-09-17T04:24:16.225403+0000","last_peered":"2024-09-17T04:24:16.225403+0000","last_clean":"2024-09-17T04:24:16.225403+0000","last_became_active":"2024-09-17T04:24:15.054453+0000","last_became_peered":"2024-09-17T04:24:15.054453+0000","last_unstale":"2024-09-17T04:24:16.225403+0000","last_undegraded":"2024-09-17T04:24:16.225403+0000","last_fullsized":"2024-09-17T04:24:16.225403+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:58.942093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T04:24:13.029115+0000","last_change":"2024-09-17T04:24:13.029115+0000","last_active":"2024-09-17T04:24:13.015626+0000","last_peered":"2024-09-17T04:24:13.015626+0000","last_clean":"2024-09-17T04:24:13.015626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:24:13.029115+0000","last_undegraded":"2024-09-17T04:24:13.029115+0000","last_fullsized":"2024-09-17T04:24:13.029115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":92,"acting":92,"num_store_stats":5},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476742,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":786432,"data_stored":650853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476741,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":60485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476742,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":60485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476742,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":60485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":180224,"data_stored":54668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":54668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":774144,"data_stored":645036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":81447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:24:19.947 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:24:20.173 INFO:tasks.ceph.mds.d.smithi184.stderr:2024-09-17T04:24:20.173+0000 7f782e000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T04:24:20.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.204+0000 7ff9bac00640 1 -- 172.21.15.100:0/2590119981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9b40b5410 msgr2=0x7ff9b40b5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:20.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.204+0000 7ff9bac00640 1 --2- 172.21.15.100:0/2590119981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9b40b5410 0x7ff9b40b5820 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac005fd0 tx=0x7ff9ac02a5d0 comp rx=0 tx=0).stop 2024-09-17T04:24:20.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.204+0000 7ff9bac00640 1 -- 172.21.15.100:0/2590119981 shutdown_connections 2024-09-17T04:24:20.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.204+0000 7ff9bac00640 1 --2- 172.21.15.100:0/2590119981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9b40b5410 0x7ff9b40b5820 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:20.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.204+0000 7ff9bac00640 1 --2- 172.21.15.100:0/2590119981 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9b415ee30 0x7ff9b40b2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:20.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.204+0000 7ff9bac00640 1 -- 172.21.15.100:0/2590119981 >> 172.21.15.100:0/2590119981 conn(0x7ff9b40b2fd0 msgr2=0x7ff9b40b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:20.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.204+0000 7ff9bac00640 1 -- 172.21.15.100:0/2590119981 shutdown_connections 2024-09-17T04:24:20.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.204+0000 7ff9bac00640 1 -- 172.21.15.100:0/2590119981 wait complete. 2024-09-17T04:24:20.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.206+0000 7ff9bac00640 1 Processor -- start 2024-09-17T04:24:20.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.206+0000 7ff9bac00640 1 -- start start 2024-09-17T04:24:20.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.208+0000 7ff9bac00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9b40b5410 0x7ff9b40ae400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:20.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.208+0000 7ff9bac00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9b40cf7c0 0x7ff9b4159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:20.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.208+0000 7ff9bac00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9b415ee30 0x7ff9b41f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:20.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.208+0000 7ff9bac00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b40b6170 con 0x7ff9b415ee30 2024-09-17T04:24:20.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.208+0000 7ff9bac00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b40b5ff0 con 0x7ff9b40cf7c0 2024-09-17T04:24:20.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.208+0000 7ff9bac00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff9b40b62f0 con 0x7ff9b40b5410 2024-09-17T04:24:20.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.208+0000 7ff9b8c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9b415ee30 0x7ff9b41f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:20.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.208+0000 7ff9b8c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9b415ee30 0x7ff9b41f5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41600/0 (socket says 172.21.15.100:41600) 2024-09-17T04:24:20.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.208+0000 7ff9b8c00640 1 -- 172.21.15.100:0/3077813324 learned_addr learned my addr 172.21.15.100:0/3077813324 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:20.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.208+0000 7ff9b3e00640 1 --2- 172.21.15.100:0/3077813324 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9b40b5410 0x7ff9b40ae400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:20.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.208+0000 7ff9b3400640 1 --2- 172.21.15.100:0/3077813324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9b40cf7c0 0x7ff9b4159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:20.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.208+0000 7ff9b8c00640 1 -- 172.21.15.100:0/3077813324 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9b40b5410 msgr2=0x7ff9b40ae400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:20.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.208+0000 7ff9b8c00640 1 --2- 172.21.15.100:0/3077813324 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9b40b5410 0x7ff9b40ae400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:20.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.208+0000 7ff9b8c00640 1 -- 172.21.15.100:0/3077813324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9b40cf7c0 msgr2=0x7ff9b4159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:20.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.208+0000 7ff9b8c00640 1 --2- 172.21.15.100:0/3077813324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9b40cf7c0 0x7ff9b4159760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:20.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.208+0000 7ff9b8c00640 1 -- 172.21.15.100:0/3077813324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b4159ca0 con 0x7ff9b415ee30 2024-09-17T04:24:20.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.209+0000 7ff9b8c00640 1 --2- 172.21.15.100:0/3077813324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9b415ee30 0x7ff9b41f5f10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac006010 tx=0x7ff9ac003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:20.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.209+0000 7ff9abe00640 1 -- 172.21.15.100:0/3077813324 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac03e030 con 0x7ff9b415ee30 2024-09-17T04:24:20.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.209+0000 7ff9abe00640 1 -- 172.21.15.100:0/3077813324 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9ac003c20 con 0x7ff9b415ee30 2024-09-17T04:24:20.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.209+0000 7ff9abe00640 1 -- 172.21.15.100:0/3077813324 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac035040 con 0x7ff9b415ee30 2024-09-17T04:24:20.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.209+0000 7ff9bac00640 1 -- 172.21.15.100:0/3077813324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b41f6e90 con 0x7ff9b415ee30 2024-09-17T04:24:20.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.209+0000 7ff9bac00640 1 -- 172.21.15.100:0/3077813324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b40b13c0 con 0x7ff9b415ee30 2024-09-17T04:24:20.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.210+0000 7ff9bac00640 1 -- 172.21.15.100:0/3077813324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9b40b5820 con 0x7ff9b415ee30 2024-09-17T04:24:20.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.211+0000 7ff9abe00640 1 -- 172.21.15.100:0/3077813324 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff9ac041030 con 0x7ff9b415ee30 2024-09-17T04:24:20.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.212+0000 7ff9abe00640 1 --2- 172.21.15.100:0/3077813324 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff994078b90 0x7ff99407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:20.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.213+0000 7ff9abe00640 1 -- 172.21.15.100:0/3077813324 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff9ac0bb2b0 con 0x7ff9b415ee30 2024-09-17T04:24:20.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.213+0000 7ff9b3e00640 1 --2- 172.21.15.100:0/3077813324 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff994078b90 0x7ff99407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:20.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.213+0000 7ff9b3e00640 1 --2- 172.21.15.100:0/3077813324 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff994078b90 0x7ff99407b0a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40d0de0 tx=0x7ff9a4009600 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:20.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.217+0000 7ff9abe00640 1 -- 172.21.15.100:0/3077813324 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff9ac083090 con 0x7ff9b415ee30 2024-09-17T04:24:20.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.399+0000 7ff9bac00640 1 -- 172.21.15.100:0/3077813324 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff9b406d9d0 con 0x7ff994078b90 2024-09-17T04:24:20.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.405+0000 7ff9abe00640 1 -- 172.21.15.100:0/3077813324 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251937 (secure 0 0 0) 0x7ff9b406d9d0 con 0x7ff994078b90 2024-09-17T04:24:20.408 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:20.409 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:24:20.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.408+0000 7ff9bac00640 1 -- 172.21.15.100:0/3077813324 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff994078b90 msgr2=0x7ff99407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:20.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.408+0000 7ff9bac00640 1 --2- 172.21.15.100:0/3077813324 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff994078b90 0x7ff99407b0a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40d0de0 tx=0x7ff9a4009600 comp rx=0 tx=0).stop 2024-09-17T04:24:20.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.408+0000 7ff9bac00640 1 -- 172.21.15.100:0/3077813324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9b415ee30 msgr2=0x7ff9b41f5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:20.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.408+0000 7ff9bac00640 1 --2- 172.21.15.100:0/3077813324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9b415ee30 0x7ff9b41f5f10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac006010 tx=0x7ff9ac003820 comp rx=0 tx=0).stop 2024-09-17T04:24:20.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.409+0000 7ff9bac00640 1 -- 172.21.15.100:0/3077813324 shutdown_connections 2024-09-17T04:24:20.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.409+0000 7ff9bac00640 1 --2- 172.21.15.100:0/3077813324 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff994078b90 0x7ff99407b0a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:20.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.409+0000 7ff9bac00640 1 --2- 172.21.15.100:0/3077813324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9b415ee30 0x7ff9b41f5f10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:20.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.409+0000 7ff9bac00640 1 --2- 172.21.15.100:0/3077813324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9b40cf7c0 0x7ff9b4159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:20.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.409+0000 7ff9bac00640 1 --2- 172.21.15.100:0/3077813324 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9b40b5410 0x7ff9b40ae400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:20.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.409+0000 7ff9bac00640 1 -- 172.21.15.100:0/3077813324 >> 172.21.15.100:0/3077813324 conn(0x7ff9b40b2fd0 msgr2=0x7ff9b4070630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:20.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.409+0000 7ff9bac00640 1 -- 172.21.15.100:0/3077813324 shutdown_connections 2024-09-17T04:24:20.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.409+0000 7ff9bac00640 1 -- 172.21.15.100:0/3077813324 wait complete. 2024-09-17T04:24:20.430 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-17T04:24:19.965152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219276,"kb_used_data":3016,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755444,"statfs":{"total":773094113280,"available":772869574656,"internally_reserved":0,"allocated":3088384,"data_stored":1824236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001746"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224232+0000","last_change":"2024-09-17T04:24:15.051472+0000","last_active":"2024-09-17T04:24:16.224232+0000","last_peered":"2024-09-17T04:24:16.224232+0000","last_clean":"2024-09-17T04:24:16.224232+0000","last_became_active":"2024-09-17T04:24:15.051162+0000","last_became_peered":"2024-09-17T04:24:15.051162+0000","last_unstale":"2024-09-17T04:24:16.224232+0000","last_undegraded":"2024-09-17T04:24:16.224232+0000","last_fullsized":"2024-09-17T04:24:16.224232+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:35:10.792366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223431+0000","last_change":"2024-09-17T04:24:15.069649+0000","last_active":"2024-09-17T04:24:16.223431+0000","last_peered":"2024-09-17T04:24:16.223431+0000","last_clean":"2024-09-17T04:24:16.223431+0000","last_became_active":"2024-09-17T04:24:15.068975+0000","last_became_peered":"2024-09-17T04:24:15.068975+0000","last_unstale":"2024-09-17T04:24:16.223431+0000","last_undegraded":"2024-09-17T04:24:16.223431+0000","last_fullsized":"2024-09-17T04:24:16.223431+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:47.639252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223497+0000","last_change":"2024-09-17T04:24:15.052608+0000","last_active":"2024-09-17T04:24:16.223497+0000","last_peered":"2024-09-17T04:24:16.223497+0000","last_clean":"2024-09-17T04:24:16.223497+0000","last_became_active":"2024-09-17T04:24:15.052252+0000","last_became_peered":"2024-09-17T04:24:15.052252+0000","last_unstale":"2024-09-17T04:24:16.223497+0000","last_undegraded":"2024-09-17T04:24:16.223497+0000","last_fullsized":"2024-09-17T04:24:16.223497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:34:23.523300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055891+0000","last_change":"2024-09-17T04:24:15.059215+0000","last_active":"2024-09-17T04:24:16.055891+0000","last_peered":"2024-09-17T04:24:16.055891+0000","last_clean":"2024-09-17T04:24:16.055891+0000","last_became_active":"2024-09-17T04:24:15.058921+0000","last_became_peered":"2024-09-17T04:24:15.058921+0000","last_unstale":"2024-09-17T04:24:16.055891+0000","last_undegraded":"2024-09-17T04:24:16.055891+0000","last_fullsized":"2024-09-17T04:24:16.055891+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:34.142688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224122+0000","last_change":"2024-09-17T04:24:15.042545+0000","last_active":"2024-09-17T04:24:16.224122+0000","last_peered":"2024-09-17T04:24:16.224122+0000","last_clean":"2024-09-17T04:24:16.224122+0000","last_became_active":"2024-09-17T04:24:15.042235+0000","last_became_peered":"2024-09-17T04:24:15.042235+0000","last_unstale":"2024-09-17T04:24:16.224122+0000","last_undegraded":"2024-09-17T04:24:16.224122+0000","last_fullsized":"2024-09-17T04:24:16.224122+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:06:32.518189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056013+0000","last_change":"2024-09-17T04:24:15.057624+0000","last_active":"2024-09-17T04:24:16.056013+0000","last_peered":"2024-09-17T04:24:16.056013+0000","last_clean":"2024-09-17T04:24:16.056013+0000","last_became_active":"2024-09-17T04:24:15.057329+0000","last_became_peered":"2024-09-17T04:24:15.057329+0000","last_unstale":"2024-09-17T04:24:16.056013+0000","last_undegraded":"2024-09-17T04:24:16.056013+0000","last_fullsized":"2024-09-17T04:24:16.056013+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:44.355705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223880+0000","last_change":"2024-09-17T04:24:15.062645+0000","last_active":"2024-09-17T04:24:16.223880+0000","last_peered":"2024-09-17T04:24:16.223880+0000","last_clean":"2024-09-17T04:24:16.223880+0000","last_became_active":"2024-09-17T04:24:15.062227+0000","last_became_peered":"2024-09-17T04:24:15.062227+0000","last_unstale":"2024-09-17T04:24:16.223880+0000","last_undegraded":"2024-09-17T04:24:16.223880+0000","last_fullsized":"2024-09-17T04:24:16.223880+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:13.159492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224308+0000","last_change":"2024-09-17T04:24:15.062993+0000","last_active":"2024-09-17T04:24:16.224308+0000","last_peered":"2024-09-17T04:24:16.224308+0000","last_clean":"2024-09-17T04:24:16.224308+0000","last_became_active":"2024-09-17T04:24:15.062311+0000","last_became_peered":"2024-09-17T04:24:15.062311+0000","last_unstale":"2024-09-17T04:24:16.224308+0000","last_undegraded":"2024-09-17T04:24:16.224308+0000","last_fullsized":"2024-09-17T04:24:16.224308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:26:42.504856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224742+0000","last_change":"2024-09-17T04:24:15.070288+0000","last_active":"2024-09-17T04:24:16.224742+0000","last_peered":"2024-09-17T04:24:16.224742+0000","last_clean":"2024-09-17T04:24:16.224742+0000","last_became_active":"2024-09-17T04:24:15.068589+0000","last_became_peered":"2024-09-17T04:24:15.068589+0000","last_unstale":"2024-09-17T04:24:16.224742+0000","last_undegraded":"2024-09-17T04:24:16.224742+0000","last_fullsized":"2024-09-17T04:24:16.224742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:33.110035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225192+0000","last_change":"2024-09-17T04:24:15.062446+0000","last_active":"2024-09-17T04:24:16.225192+0000","last_peered":"2024-09-17T04:24:16.225192+0000","last_clean":"2024-09-17T04:24:16.225192+0000","last_became_active":"2024-09-17T04:24:15.060522+0000","last_became_peered":"2024-09-17T04:24:15.060522+0000","last_unstale":"2024-09-17T04:24:16.225192+0000","last_undegraded":"2024-09-17T04:24:16.225192+0000","last_fullsized":"2024-09-17T04:24:16.225192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:09.349585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224815+0000","last_change":"2024-09-17T04:24:15.062048+0000","last_active":"2024-09-17T04:24:16.224815+0000","last_peered":"2024-09-17T04:24:16.224815+0000","last_clean":"2024-09-17T04:24:16.224815+0000","last_became_active":"2024-09-17T04:24:15.060887+0000","last_became_peered":"2024-09-17T04:24:15.060887+0000","last_unstale":"2024-09-17T04:24:16.224815+0000","last_undegraded":"2024-09-17T04:24:16.224815+0000","last_fullsized":"2024-09-17T04:24:16.224815+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:22.452641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054679+0000","last_change":"2024-09-17T04:24:15.068320+0000","last_active":"2024-09-17T04:24:16.054679+0000","last_peered":"2024-09-17T04:24:16.054679+0000","last_clean":"2024-09-17T04:24:16.054679+0000","last_became_active":"2024-09-17T04:24:15.067973+0000","last_became_peered":"2024-09-17T04:24:15.067973+0000","last_unstale":"2024-09-17T04:24:16.054679+0000","last_undegraded":"2024-09-17T04:24:16.054679+0000","last_fullsized":"2024-09-17T04:24:16.054679+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:04.323459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225225+0000","last_change":"2024-09-17T04:24:15.047951+0000","last_active":"2024-09-17T04:24:16.225225+0000","last_peered":"2024-09-17T04:24:16.225225+0000","last_clean":"2024-09-17T04:24:16.225225+0000","last_became_active":"2024-09-17T04:24:15.047536+0000","last_became_peered":"2024-09-17T04:24:15.047536+0000","last_unstale":"2024-09-17T04:24:16.225225+0000","last_undegraded":"2024-09-17T04:24:16.225225+0000","last_fullsized":"2024-09-17T04:24:16.225225+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:38:48.386032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224034+0000","last_change":"2024-09-17T04:24:15.070607+0000","last_active":"2024-09-17T04:24:16.224034+0000","last_peered":"2024-09-17T04:24:16.224034+0000","last_clean":"2024-09-17T04:24:16.224034+0000","last_became_active":"2024-09-17T04:24:15.067087+0000","last_became_peered":"2024-09-17T04:24:15.067087+0000","last_unstale":"2024-09-17T04:24:16.224034+0000","last_undegraded":"2024-09-17T04:24:16.224034+0000","last_fullsized":"2024-09-17T04:24:16.224034+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:09.349585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224091+0000","last_change":"2024-09-17T04:24:15.076095+0000","last_active":"2024-09-17T04:24:16.224091+0000","last_peered":"2024-09-17T04:24:16.224091+0000","last_clean":"2024-09-17T04:24:16.224091+0000","last_became_active":"2024-09-17T04:24:15.075803+0000","last_became_peered":"2024-09-17T04:24:15.075803+0000","last_unstale":"2024-09-17T04:24:16.224091+0000","last_undegraded":"2024-09-17T04:24:16.224091+0000","last_fullsized":"2024-09-17T04:24:16.224091+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:28.104736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055352+0000","last_change":"2024-09-17T04:24:15.053797+0000","last_active":"2024-09-17T04:24:16.055352+0000","last_peered":"2024-09-17T04:24:16.055352+0000","last_clean":"2024-09-17T04:24:16.055352+0000","last_became_active":"2024-09-17T04:24:15.053449+0000","last_became_peered":"2024-09-17T04:24:15.053449+0000","last_unstale":"2024-09-17T04:24:16.055352+0000","last_undegraded":"2024-09-17T04:24:16.055352+0000","last_fullsized":"2024-09-17T04:24:16.055352+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:49:03.743620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056725+0000","last_change":"2024-09-17T04:24:15.055632+0000","last_active":"2024-09-17T04:24:16.056725+0000","last_peered":"2024-09-17T04:24:16.056725+0000","last_clean":"2024-09-17T04:24:16.056725+0000","last_became_active":"2024-09-17T04:24:15.055156+0000","last_became_peered":"2024-09-17T04:24:15.055156+0000","last_unstale":"2024-09-17T04:24:16.056725+0000","last_undegraded":"2024-09-17T04:24:16.056725+0000","last_fullsized":"2024-09-17T04:24:16.056725+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:16.120591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.052124+0000","last_change":"2024-09-17T04:24:15.057788+0000","last_active":"2024-09-17T04:24:16.052124+0000","last_peered":"2024-09-17T04:24:16.052124+0000","last_clean":"2024-09-17T04:24:16.052124+0000","last_became_active":"2024-09-17T04:24:15.057141+0000","last_became_peered":"2024-09-17T04:24:15.057141+0000","last_unstale":"2024-09-17T04:24:16.052124+0000","last_undegraded":"2024-09-17T04:24:16.052124+0000","last_fullsized":"2024-09-17T04:24:16.052124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:10.704647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224426+0000","last_change":"2024-09-17T04:24:15.067312+0000","last_active":"2024-09-17T04:24:16.224426+0000","last_peered":"2024-09-17T04:24:16.224426+0000","last_clean":"2024-09-17T04:24:16.224426+0000","last_became_active":"2024-09-17T04:24:15.059516+0000","last_became_peered":"2024-09-17T04:24:15.059516+0000","last_unstale":"2024-09-17T04:24:16.224426+0000","last_undegraded":"2024-09-17T04:24:16.224426+0000","last_fullsized":"2024-09-17T04:24:16.224426+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:16.120591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224948+0000","last_change":"2024-09-17T04:24:15.076912+0000","last_active":"2024-09-17T04:24:16.224948+0000","last_peered":"2024-09-17T04:24:16.224948+0000","last_clean":"2024-09-17T04:24:16.224948+0000","last_became_active":"2024-09-17T04:24:15.076617+0000","last_became_peered":"2024-09-17T04:24:15.076617+0000","last_unstale":"2024-09-17T04:24:16.224948+0000","last_undegraded":"2024-09-17T04:24:16.224948+0000","last_fullsized":"2024-09-17T04:24:16.224948+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:47.639252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223809+0000","last_change":"2024-09-17T04:24:15.070958+0000","last_active":"2024-09-17T04:24:16.223809+0000","last_peered":"2024-09-17T04:24:16.223809+0000","last_clean":"2024-09-17T04:24:16.223809+0000","last_became_active":"2024-09-17T04:24:15.070624+0000","last_became_peered":"2024-09-17T04:24:15.070624+0000","last_unstale":"2024-09-17T04:24:16.223809+0000","last_undegraded":"2024-09-17T04:24:16.223809+0000","last_fullsized":"2024-09-17T04:24:16.223809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:13.752351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055648+0000","last_change":"2024-09-17T04:24:15.058815+0000","last_active":"2024-09-17T04:24:16.055648+0000","last_peered":"2024-09-17T04:24:16.055648+0000","last_clean":"2024-09-17T04:24:16.055648+0000","last_became_active":"2024-09-17T04:24:15.058482+0000","last_became_peered":"2024-09-17T04:24:15.058482+0000","last_unstale":"2024-09-17T04:24:16.055648+0000","last_undegraded":"2024-09-17T04:24:16.055648+0000","last_fullsized":"2024-09-17T04:24:16.055648+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:18.409788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224382+0000","last_change":"2024-09-17T04:24:15.070200+0000","last_active":"2024-09-17T04:24:16.224382+0000","last_peered":"2024-09-17T04:24:16.224382+0000","last_clean":"2024-09-17T04:24:16.224382+0000","last_became_active":"2024-09-17T04:24:15.069872+0000","last_became_peered":"2024-09-17T04:24:15.069872+0000","last_unstale":"2024-09-17T04:24:16.224382+0000","last_undegraded":"2024-09-17T04:24:16.224382+0000","last_fullsized":"2024-09-17T04:24:16.224382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:22.880022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055404+0000","last_change":"2024-09-17T04:24:15.050935+0000","last_active":"2024-09-17T04:24:16.055404+0000","last_peered":"2024-09-17T04:24:16.055404+0000","last_clean":"2024-09-17T04:24:16.055404+0000","last_became_active":"2024-09-17T04:24:15.048946+0000","last_became_peered":"2024-09-17T04:24:15.048946+0000","last_unstale":"2024-09-17T04:24:16.055404+0000","last_undegraded":"2024-09-17T04:24:16.055404+0000","last_fullsized":"2024-09-17T04:24:16.055404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:11:09.116226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225400+0000","last_change":"2024-09-17T04:24:15.074863+0000","last_active":"2024-09-17T04:24:16.225400+0000","last_peered":"2024-09-17T04:24:16.225400+0000","last_clean":"2024-09-17T04:24:16.225400+0000","last_became_active":"2024-09-17T04:24:15.074467+0000","last_became_peered":"2024-09-17T04:24:15.074467+0000","last_unstale":"2024-09-17T04:24:16.225400+0000","last_undegraded":"2024-09-17T04:24:16.225400+0000","last_fullsized":"2024-09-17T04:24:16.225400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:36:59.215343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225206+0000","last_change":"2024-09-17T04:24:15.069430+0000","last_active":"2024-09-17T04:24:16.225206+0000","last_peered":"2024-09-17T04:24:16.225206+0000","last_clean":"2024-09-17T04:24:16.225206+0000","last_became_active":"2024-09-17T04:24:15.069117+0000","last_became_peered":"2024-09-17T04:24:15.069117+0000","last_unstale":"2024-09-17T04:24:16.225206+0000","last_undegraded":"2024-09-17T04:24:16.225206+0000","last_fullsized":"2024-09-17T04:24:16.225206+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:49:03.743620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224018+0000","last_change":"2024-09-17T04:24:15.057539+0000","last_active":"2024-09-17T04:24:16.224018+0000","last_peered":"2024-09-17T04:24:16.224018+0000","last_clean":"2024-09-17T04:24:16.224018+0000","last_became_active":"2024-09-17T04:24:15.057194+0000","last_became_peered":"2024-09-17T04:24:15.057194+0000","last_unstale":"2024-09-17T04:24:16.224018+0000","last_undegraded":"2024-09-17T04:24:16.224018+0000","last_fullsized":"2024-09-17T04:24:16.224018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:09.349585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225430+0000","last_change":"2024-09-17T04:24:15.075902+0000","last_active":"2024-09-17T04:24:16.225430+0000","last_peered":"2024-09-17T04:24:16.225430+0000","last_clean":"2024-09-17T04:24:16.225430+0000","last_became_active":"2024-09-17T04:24:15.075625+0000","last_became_peered":"2024-09-17T04:24:15.075625+0000","last_unstale":"2024-09-17T04:24:16.225430+0000","last_undegraded":"2024-09-17T04:24:16.225430+0000","last_fullsized":"2024-09-17T04:24:16.225430+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:34.202614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223788+0000","last_change":"2024-09-17T04:24:15.052108+0000","last_active":"2024-09-17T04:24:16.223788+0000","last_peered":"2024-09-17T04:24:16.223788+0000","last_clean":"2024-09-17T04:24:16.223788+0000","last_became_active":"2024-09-17T04:24:15.051794+0000","last_became_peered":"2024-09-17T04:24:15.051794+0000","last_unstale":"2024-09-17T04:24:16.223788+0000","last_undegraded":"2024-09-17T04:24:16.223788+0000","last_fullsized":"2024-09-17T04:24:16.223788+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:13:56.486773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055909+0000","last_change":"2024-09-17T04:24:15.057335+0000","last_active":"2024-09-17T04:24:16.055909+0000","last_peered":"2024-09-17T04:24:16.055909+0000","last_clean":"2024-09-17T04:24:16.055909+0000","last_became_active":"2024-09-17T04:24:15.056924+0000","last_became_peered":"2024-09-17T04:24:15.056924+0000","last_unstale":"2024-09-17T04:24:16.055909+0000","last_undegraded":"2024-09-17T04:24:16.055909+0000","last_fullsized":"2024-09-17T04:24:16.055909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:29:56.515791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.052052+0000","last_change":"2024-09-17T04:24:15.054070+0000","last_active":"2024-09-17T04:24:16.052052+0000","last_peered":"2024-09-17T04:24:16.052052+0000","last_clean":"2024-09-17T04:24:16.052052+0000","last_became_active":"2024-09-17T04:24:15.052619+0000","last_became_peered":"2024-09-17T04:24:15.052619+0000","last_unstale":"2024-09-17T04:24:16.052052+0000","last_undegraded":"2024-09-17T04:24:16.052052+0000","last_fullsized":"2024-09-17T04:24:16.052052+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:49.409374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223562+0000","last_change":"2024-09-17T04:24:15.061084+0000","last_active":"2024-09-17T04:24:16.223562+0000","last_peered":"2024-09-17T04:24:16.223562+0000","last_clean":"2024-09-17T04:24:16.223562+0000","last_became_active":"2024-09-17T04:24:15.060769+0000","last_became_peered":"2024-09-17T04:24:15.060769+0000","last_unstale":"2024-09-17T04:24:16.223562+0000","last_undegraded":"2024-09-17T04:24:16.223562+0000","last_fullsized":"2024-09-17T04:24:16.223562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:13:56.486773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054106+0000","last_change":"2024-09-17T04:24:15.068899+0000","last_active":"2024-09-17T04:24:16.054106+0000","last_peered":"2024-09-17T04:24:16.054106+0000","last_clean":"2024-09-17T04:24:16.054106+0000","last_became_active":"2024-09-17T04:24:15.068595+0000","last_became_peered":"2024-09-17T04:24:15.068595+0000","last_unstale":"2024-09-17T04:24:16.054106+0000","last_undegraded":"2024-09-17T04:24:16.054106+0000","last_fullsized":"2024-09-17T04:24:16.054106+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:13.785027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225374+0000","last_change":"2024-09-17T04:24:14.062806+0000","last_active":"2024-09-17T04:24:16.225374+0000","last_peered":"2024-09-17T04:24:16.225374+0000","last_clean":"2024-09-17T04:24:16.225374+0000","last_became_active":"2024-09-17T04:24:14.062282+0000","last_became_peered":"2024-09-17T04:24:14.062282+0000","last_unstale":"2024-09-17T04:24:16.225374+0000","last_undegraded":"2024-09-17T04:24:16.225374+0000","last_fullsized":"2024-09-17T04:24:16.225374+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:23.698652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224005+0000","last_change":"2024-09-17T04:24:15.075318+0000","last_active":"2024-09-17T04:24:16.224005+0000","last_peered":"2024-09-17T04:24:16.224005+0000","last_clean":"2024-09-17T04:24:16.224005+0000","last_became_active":"2024-09-17T04:24:15.074876+0000","last_became_peered":"2024-09-17T04:24:15.074876+0000","last_unstale":"2024-09-17T04:24:16.224005+0000","last_undegraded":"2024-09-17T04:24:16.224005+0000","last_fullsized":"2024-09-17T04:24:16.224005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:47.639252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224710+0000","last_change":"2024-09-17T04:24:14.056222+0000","last_active":"2024-09-17T04:24:16.224710+0000","last_peered":"2024-09-17T04:24:16.224710+0000","last_clean":"2024-09-17T04:24:16.224710+0000","last_became_active":"2024-09-17T04:24:14.055874+0000","last_became_peered":"2024-09-17T04:24:14.055874+0000","last_unstale":"2024-09-17T04:24:16.224710+0000","last_undegraded":"2024-09-17T04:24:16.224710+0000","last_fullsized":"2024-09-17T04:24:16.224710+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:44.734015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224568+0000","last_change":"2024-09-17T04:24:14.054189+0000","last_active":"2024-09-17T04:24:16.224568+0000","last_peered":"2024-09-17T04:24:16.224568+0000","last_clean":"2024-09-17T04:24:16.224568+0000","last_became_active":"2024-09-17T04:24:14.053870+0000","last_became_peered":"2024-09-17T04:24:14.053870+0000","last_unstale":"2024-09-17T04:24:16.224568+0000","last_undegraded":"2024-09-17T04:24:16.224568+0000","last_fullsized":"2024-09-17T04:24:16.224568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:12.776180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224542+0000","last_change":"2024-09-17T04:24:15.059936+0000","last_active":"2024-09-17T04:24:16.224542+0000","last_peered":"2024-09-17T04:24:16.224542+0000","last_clean":"2024-09-17T04:24:16.224542+0000","last_became_active":"2024-09-17T04:24:15.059645+0000","last_became_peered":"2024-09-17T04:24:15.059645+0000","last_unstale":"2024-09-17T04:24:16.224542+0000","last_undegraded":"2024-09-17T04:24:16.224542+0000","last_fullsized":"2024-09-17T04:24:16.224542+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:08:11.703658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055748+0000","last_change":"2024-09-17T04:24:14.060082+0000","last_active":"2024-09-17T04:24:16.055748+0000","last_peered":"2024-09-17T04:24:16.055748+0000","last_clean":"2024-09-17T04:24:16.055748+0000","last_became_active":"2024-09-17T04:24:14.058795+0000","last_became_peered":"2024-09-17T04:24:14.058795+0000","last_unstale":"2024-09-17T04:24:16.055748+0000","last_undegraded":"2024-09-17T04:24:16.055748+0000","last_fullsized":"2024-09-17T04:24:16.055748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:21.535057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.222826+0000","last_change":"2024-09-17T04:24:15.061788+0000","last_active":"2024-09-17T04:24:16.222826+0000","last_peered":"2024-09-17T04:24:16.222826+0000","last_clean":"2024-09-17T04:24:16.222826+0000","last_became_active":"2024-09-17T04:24:15.055072+0000","last_became_peered":"2024-09-17T04:24:15.055072+0000","last_unstale":"2024-09-17T04:24:16.222826+0000","last_undegraded":"2024-09-17T04:24:16.222826+0000","last_fullsized":"2024-09-17T04:24:16.222826+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:13.752351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225245+0000","last_change":"2024-09-17T04:24:14.053595+0000","last_active":"2024-09-17T04:24:16.225245+0000","last_peered":"2024-09-17T04:24:16.225245+0000","last_clean":"2024-09-17T04:24:16.225245+0000","last_became_active":"2024-09-17T04:24:14.053260+0000","last_became_peered":"2024-09-17T04:24:14.053260+0000","last_unstale":"2024-09-17T04:24:16.225245+0000","last_undegraded":"2024-09-17T04:24:16.225245+0000","last_fullsized":"2024-09-17T04:24:16.225245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:51.429181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056660+0000","last_change":"2024-09-17T04:24:15.053812+0000","last_active":"2024-09-17T04:24:16.056660+0000","last_peered":"2024-09-17T04:24:16.056660+0000","last_clean":"2024-09-17T04:24:16.056660+0000","last_became_active":"2024-09-17T04:24:15.053501+0000","last_became_peered":"2024-09-17T04:24:15.053501+0000","last_unstale":"2024-09-17T04:24:16.056660+0000","last_undegraded":"2024-09-17T04:24:16.056660+0000","last_fullsized":"2024-09-17T04:24:16.056660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:47.959956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056115+0000","last_change":"2024-09-17T04:24:14.059203+0000","last_active":"2024-09-17T04:24:16.056115+0000","last_peered":"2024-09-17T04:24:16.056115+0000","last_clean":"2024-09-17T04:24:16.056115+0000","last_became_active":"2024-09-17T04:24:14.058623+0000","last_became_peered":"2024-09-17T04:24:14.058623+0000","last_unstale":"2024-09-17T04:24:16.056115+0000","last_undegraded":"2024-09-17T04:24:16.056115+0000","last_fullsized":"2024-09-17T04:24:16.056115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:20.372651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223819+0000","last_change":"2024-09-17T04:24:15.068669+0000","last_active":"2024-09-17T04:24:16.223819+0000","last_peered":"2024-09-17T04:24:16.223819+0000","last_clean":"2024-09-17T04:24:16.223819+0000","last_became_active":"2024-09-17T04:24:15.066578+0000","last_became_peered":"2024-09-17T04:24:15.066578+0000","last_unstale":"2024-09-17T04:24:16.223819+0000","last_undegraded":"2024-09-17T04:24:16.223819+0000","last_fullsized":"2024-09-17T04:24:16.223819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:01.706191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223044+0000","last_change":"2024-09-17T04:24:14.061969+0000","last_active":"2024-09-17T04:24:16.223044+0000","last_peered":"2024-09-17T04:24:16.223044+0000","last_clean":"2024-09-17T04:24:16.223044+0000","last_became_active":"2024-09-17T04:24:14.061593+0000","last_became_peered":"2024-09-17T04:24:14.061593+0000","last_unstale":"2024-09-17T04:24:16.223044+0000","last_undegraded":"2024-09-17T04:24:16.223044+0000","last_fullsized":"2024-09-17T04:24:16.223044+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:48.406614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054926+0000","last_change":"2024-09-17T04:24:15.071779+0000","last_active":"2024-09-17T04:24:16.054926+0000","last_peered":"2024-09-17T04:24:16.054926+0000","last_clean":"2024-09-17T04:24:16.054926+0000","last_became_active":"2024-09-17T04:24:15.071431+0000","last_became_peered":"2024-09-17T04:24:15.071431+0000","last_unstale":"2024-09-17T04:24:16.054926+0000","last_undegraded":"2024-09-17T04:24:16.054926+0000","last_fullsized":"2024-09-17T04:24:16.054926+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:10.704647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224939+0000","last_change":"2024-09-17T04:24:14.058729+0000","last_active":"2024-09-17T04:24:16.224939+0000","last_peered":"2024-09-17T04:24:16.224939+0000","last_clean":"2024-09-17T04:24:16.224939+0000","last_became_active":"2024-09-17T04:24:14.057616+0000","last_became_peered":"2024-09-17T04:24:14.057616+0000","last_unstale":"2024-09-17T04:24:16.224939+0000","last_undegraded":"2024-09-17T04:24:16.224939+0000","last_fullsized":"2024-09-17T04:24:16.224939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:42.267414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223701+0000","last_change":"2024-09-17T04:24:15.056613+0000","last_active":"2024-09-17T04:24:16.223701+0000","last_peered":"2024-09-17T04:24:16.223701+0000","last_clean":"2024-09-17T04:24:16.223701+0000","last_became_active":"2024-09-17T04:24:15.056300+0000","last_became_peered":"2024-09-17T04:24:15.056300+0000","last_unstale":"2024-09-17T04:24:16.223701+0000","last_undegraded":"2024-09-17T04:24:16.223701+0000","last_fullsized":"2024-09-17T04:24:16.223701+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:34.142688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223885+0000","last_change":"2024-09-17T04:24:14.054359+0000","last_active":"2024-09-17T04:24:16.223885+0000","last_peered":"2024-09-17T04:24:16.223885+0000","last_clean":"2024-09-17T04:24:16.223885+0000","last_became_active":"2024-09-17T04:24:14.053935+0000","last_became_peered":"2024-09-17T04:24:14.053935+0000","last_unstale":"2024-09-17T04:24:16.223885+0000","last_undegraded":"2024-09-17T04:24:16.223885+0000","last_fullsized":"2024-09-17T04:24:16.223885+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:01.211818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223752+0000","last_change":"2024-09-17T04:24:15.068236+0000","last_active":"2024-09-17T04:24:16.223752+0000","last_peered":"2024-09-17T04:24:16.223752+0000","last_clean":"2024-09-17T04:24:16.223752+0000","last_became_active":"2024-09-17T04:24:15.067460+0000","last_became_peered":"2024-09-17T04:24:15.067460+0000","last_unstale":"2024-09-17T04:24:16.223752+0000","last_undegraded":"2024-09-17T04:24:16.223752+0000","last_fullsized":"2024-09-17T04:24:16.223752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:44:58.329654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056099+0000","last_change":"2024-09-17T04:24:14.058127+0000","last_active":"2024-09-17T04:24:16.056099+0000","last_peered":"2024-09-17T04:24:16.056099+0000","last_clean":"2024-09-17T04:24:16.056099+0000","last_became_active":"2024-09-17T04:24:14.055990+0000","last_became_peered":"2024-09-17T04:24:14.055990+0000","last_unstale":"2024-09-17T04:24:16.056099+0000","last_undegraded":"2024-09-17T04:24:16.056099+0000","last_fullsized":"2024-09-17T04:24:16.056099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:51.429181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223887+0000","last_change":"2024-09-17T04:24:15.076359+0000","last_active":"2024-09-17T04:24:16.223887+0000","last_peered":"2024-09-17T04:24:16.223887+0000","last_clean":"2024-09-17T04:24:16.223887+0000","last_became_active":"2024-09-17T04:24:15.075235+0000","last_became_peered":"2024-09-17T04:24:15.075235+0000","last_unstale":"2024-09-17T04:24:16.223887+0000","last_undegraded":"2024-09-17T04:24:16.223887+0000","last_fullsized":"2024-09-17T04:24:16.223887+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:44.355705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224496+0000","last_change":"2024-09-17T04:24:14.051949+0000","last_active":"2024-09-17T04:24:16.224496+0000","last_peered":"2024-09-17T04:24:16.224496+0000","last_clean":"2024-09-17T04:24:16.224496+0000","last_became_active":"2024-09-17T04:24:14.051548+0000","last_became_peered":"2024-09-17T04:24:14.051548+0000","last_unstale":"2024-09-17T04:24:16.224496+0000","last_undegraded":"2024-09-17T04:24:16.224496+0000","last_fullsized":"2024-09-17T04:24:16.224496+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:21.535057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223899+0000","last_change":"2024-09-17T04:24:15.060676+0000","last_active":"2024-09-17T04:24:16.223899+0000","last_peered":"2024-09-17T04:24:16.223899+0000","last_clean":"2024-09-17T04:24:16.223899+0000","last_became_active":"2024-09-17T04:24:15.060268+0000","last_became_peered":"2024-09-17T04:24:15.060268+0000","last_unstale":"2024-09-17T04:24:16.223899+0000","last_undegraded":"2024-09-17T04:24:16.223899+0000","last_fullsized":"2024-09-17T04:24:16.223899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:13.752351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.051789+0000","last_change":"2024-09-17T04:24:14.043539+0000","last_active":"2024-09-17T04:24:16.051789+0000","last_peered":"2024-09-17T04:24:16.051789+0000","last_clean":"2024-09-17T04:24:16.051789+0000","last_became_active":"2024-09-17T04:24:14.043214+0000","last_became_peered":"2024-09-17T04:24:14.043214+0000","last_unstale":"2024-09-17T04:24:16.051789+0000","last_undegraded":"2024-09-17T04:24:16.051789+0000","last_fullsized":"2024-09-17T04:24:16.051789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:42.267414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223990+0000","last_change":"2024-09-17T04:24:15.068886+0000","last_active":"2024-09-17T04:24:16.223990+0000","last_peered":"2024-09-17T04:24:16.223990+0000","last_clean":"2024-09-17T04:24:16.223990+0000","last_became_active":"2024-09-17T04:24:15.068532+0000","last_became_peered":"2024-09-17T04:24:15.068532+0000","last_unstale":"2024-09-17T04:24:16.223990+0000","last_undegraded":"2024-09-17T04:24:16.223990+0000","last_fullsized":"2024-09-17T04:24:16.223990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:58.719218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054761+0000","last_change":"2024-09-17T04:24:14.059673+0000","last_active":"2024-09-17T04:24:16.054761+0000","last_peered":"2024-09-17T04:24:16.054761+0000","last_clean":"2024-09-17T04:24:16.054761+0000","last_became_active":"2024-09-17T04:24:14.057668+0000","last_became_peered":"2024-09-17T04:24:14.057668+0000","last_unstale":"2024-09-17T04:24:16.054761+0000","last_undegraded":"2024-09-17T04:24:16.054761+0000","last_fullsized":"2024-09-17T04:24:16.054761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:01.211818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224143+0000","last_change":"2024-09-17T04:24:15.069931+0000","last_active":"2024-09-17T04:24:16.224143+0000","last_peered":"2024-09-17T04:24:16.224143+0000","last_clean":"2024-09-17T04:24:16.224143+0000","last_became_active":"2024-09-17T04:24:15.069568+0000","last_became_peered":"2024-09-17T04:24:15.069568+0000","last_unstale":"2024-09-17T04:24:16.224143+0000","last_undegraded":"2024-09-17T04:24:16.224143+0000","last_fullsized":"2024-09-17T04:24:16.224143+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:35.584338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.052093+0000","last_change":"2024-09-17T04:24:14.054912+0000","last_active":"2024-09-17T04:24:16.052093+0000","last_peered":"2024-09-17T04:24:16.052093+0000","last_clean":"2024-09-17T04:24:16.052093+0000","last_became_active":"2024-09-17T04:24:14.054105+0000","last_became_peered":"2024-09-17T04:24:14.054105+0000","last_unstale":"2024-09-17T04:24:16.052093+0000","last_undegraded":"2024-09-17T04:24:16.052093+0000","last_fullsized":"2024-09-17T04:24:16.052093+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:44.734015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224717+0000","last_change":"2024-09-17T04:24:15.075676+0000","last_active":"2024-09-17T04:24:16.224717+0000","last_peered":"2024-09-17T04:24:16.224717+0000","last_clean":"2024-09-17T04:24:16.224717+0000","last_became_active":"2024-09-17T04:24:15.075114+0000","last_became_peered":"2024-09-17T04:24:15.075114+0000","last_unstale":"2024-09-17T04:24:16.224717+0000","last_undegraded":"2024-09-17T04:24:16.224717+0000","last_fullsized":"2024-09-17T04:24:16.224717+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:51:30.987569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223146+0000","last_change":"2024-09-17T04:24:14.039417+0000","last_active":"2024-09-17T04:24:16.223146+0000","last_peered":"2024-09-17T04:24:16.223146+0000","last_clean":"2024-09-17T04:24:16.223146+0000","last_became_active":"2024-09-17T04:24:14.039025+0000","last_became_peered":"2024-09-17T04:24:14.039025+0000","last_unstale":"2024-09-17T04:24:16.223146+0000","last_undegraded":"2024-09-17T04:24:16.223146+0000","last_fullsized":"2024-09-17T04:24:16.223146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:03.320698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056433+0000","last_change":"2024-09-17T04:24:15.061023+0000","last_active":"2024-09-17T04:24:16.056433+0000","last_peered":"2024-09-17T04:24:16.056433+0000","last_clean":"2024-09-17T04:24:16.056433+0000","last_became_active":"2024-09-17T04:24:15.060544+0000","last_became_peered":"2024-09-17T04:24:15.060544+0000","last_unstale":"2024-09-17T04:24:16.056433+0000","last_undegraded":"2024-09-17T04:24:16.056433+0000","last_fullsized":"2024-09-17T04:24:16.056433+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:35:10.792366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.051950+0000","last_change":"2024-09-17T04:24:14.052529+0000","last_active":"2024-09-17T04:24:16.051950+0000","last_peered":"2024-09-17T04:24:16.051950+0000","last_clean":"2024-09-17T04:24:16.051950+0000","last_became_active":"2024-09-17T04:24:14.052135+0000","last_became_peered":"2024-09-17T04:24:14.052135+0000","last_unstale":"2024-09-17T04:24:16.051950+0000","last_undegraded":"2024-09-17T04:24:16.051950+0000","last_fullsized":"2024-09-17T04:24:16.051950+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:28.196011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224039+0000","last_change":"2024-09-17T04:24:15.064435+0000","last_active":"2024-09-17T04:24:16.224039+0000","last_peered":"2024-09-17T04:24:16.224039+0000","last_clean":"2024-09-17T04:24:16.224039+0000","last_became_active":"2024-09-17T04:24:15.064195+0000","last_became_peered":"2024-09-17T04:24:15.064195+0000","last_unstale":"2024-09-17T04:24:16.224039+0000","last_undegraded":"2024-09-17T04:24:16.224039+0000","last_fullsized":"2024-09-17T04:24:16.224039+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:47.959956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223547+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:24:16.223547+0000","last_peered":"2024-09-17T04:24:16.223547+0000","last_clean":"2024-09-17T04:24:16.223547+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:24:16.223547+0000","last_undegraded":"2024-09-17T04:24:16.223547+0000","last_fullsized":"2024-09-17T04:24:16.223547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055535+0000","last_change":"2024-09-17T04:24:14.049088+0000","last_active":"2024-09-17T04:24:16.055535+0000","last_peered":"2024-09-17T04:24:16.055535+0000","last_clean":"2024-09-17T04:24:16.055535+0000","last_became_active":"2024-09-17T04:24:14.048529+0000","last_became_peered":"2024-09-17T04:24:14.048529+0000","last_unstale":"2024-09-17T04:24:16.055535+0000","last_undegraded":"2024-09-17T04:24:16.055535+0000","last_fullsized":"2024-09-17T04:24:16.055535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:42.267414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225083+0000","last_change":"2024-09-17T04:24:15.065699+0000","last_active":"2024-09-17T04:24:16.225083+0000","last_peered":"2024-09-17T04:24:16.225083+0000","last_clean":"2024-09-17T04:24:16.225083+0000","last_became_active":"2024-09-17T04:24:15.061569+0000","last_became_peered":"2024-09-17T04:24:15.061569+0000","last_unstale":"2024-09-17T04:24:16.225083+0000","last_undegraded":"2024-09-17T04:24:16.225083+0000","last_fullsized":"2024-09-17T04:24:16.225083+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:01:19.496265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223384+0000","last_change":"2024-09-17T04:24:14.061081+0000","last_active":"2024-09-17T04:24:16.223384+0000","last_peered":"2024-09-17T04:24:16.223384+0000","last_clean":"2024-09-17T04:24:16.223384+0000","last_became_active":"2024-09-17T04:24:14.060394+0000","last_became_peered":"2024-09-17T04:24:14.060394+0000","last_unstale":"2024-09-17T04:24:16.223384+0000","last_undegraded":"2024-09-17T04:24:16.223384+0000","last_fullsized":"2024-09-17T04:24:16.223384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:08.023686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223901+0000","last_change":"2024-09-17T04:24:15.070960+0000","last_active":"2024-09-17T04:24:16.223901+0000","last_peered":"2024-09-17T04:24:16.223901+0000","last_clean":"2024-09-17T04:24:16.223901+0000","last_became_active":"2024-09-17T04:24:15.070528+0000","last_became_peered":"2024-09-17T04:24:15.070528+0000","last_unstale":"2024-09-17T04:24:16.223901+0000","last_undegraded":"2024-09-17T04:24:16.223901+0000","last_fullsized":"2024-09-17T04:24:16.223901+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:22.452641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.052102+0000","last_change":"2024-09-17T04:24:14.051042+0000","last_active":"2024-09-17T04:24:16.052102+0000","last_peered":"2024-09-17T04:24:16.052102+0000","last_clean":"2024-09-17T04:24:16.052102+0000","last_became_active":"2024-09-17T04:24:14.050513+0000","last_became_peered":"2024-09-17T04:24:14.050513+0000","last_unstale":"2024-09-17T04:24:16.052102+0000","last_undegraded":"2024-09-17T04:24:16.052102+0000","last_fullsized":"2024-09-17T04:24:16.052102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:21.535057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.222913+0000","last_change":"2024-09-17T04:24:15.061656+0000","last_active":"2024-09-17T04:24:16.222913+0000","last_peered":"2024-09-17T04:24:16.222913+0000","last_clean":"2024-09-17T04:24:16.222913+0000","last_became_active":"2024-09-17T04:24:15.061365+0000","last_became_peered":"2024-09-17T04:24:15.061365+0000","last_unstale":"2024-09-17T04:24:16.222913+0000","last_undegraded":"2024-09-17T04:24:16.222913+0000","last_fullsized":"2024-09-17T04:24:16.222913+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:22.452641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056245+0000","last_change":"2024-09-17T04:24:14.053751+0000","last_active":"2024-09-17T04:24:16.056245+0000","last_peered":"2024-09-17T04:24:16.056245+0000","last_clean":"2024-09-17T04:24:16.056245+0000","last_became_active":"2024-09-17T04:24:14.051943+0000","last_became_peered":"2024-09-17T04:24:14.051943+0000","last_unstale":"2024-09-17T04:24:16.056245+0000","last_undegraded":"2024-09-17T04:24:16.056245+0000","last_fullsized":"2024-09-17T04:24:16.056245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:08.023686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054263+0000","last_change":"2024-09-17T04:24:15.074736+0000","last_active":"2024-09-17T04:24:16.054263+0000","last_peered":"2024-09-17T04:24:16.054263+0000","last_clean":"2024-09-17T04:24:16.054263+0000","last_became_active":"2024-09-17T04:24:15.074374+0000","last_became_peered":"2024-09-17T04:24:15.074374+0000","last_unstale":"2024-09-17T04:24:16.054263+0000","last_undegraded":"2024-09-17T04:24:16.054263+0000","last_fullsized":"2024-09-17T04:24:16.054263+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:49.409374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.222921+0000","last_change":"2024-09-17T04:24:14.057105+0000","last_active":"2024-09-17T04:24:16.222921+0000","last_peered":"2024-09-17T04:24:16.222921+0000","last_clean":"2024-09-17T04:24:16.222921+0000","last_became_active":"2024-09-17T04:24:14.056908+0000","last_became_peered":"2024-09-17T04:24:14.056908+0000","last_unstale":"2024-09-17T04:24:16.222921+0000","last_undegraded":"2024-09-17T04:24:16.222921+0000","last_fullsized":"2024-09-17T04:24:16.222921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:12.782267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056301+0000","last_change":"2024-09-17T04:24:15.058663+0000","last_active":"2024-09-17T04:24:16.056301+0000","last_peered":"2024-09-17T04:24:16.056301+0000","last_clean":"2024-09-17T04:24:16.056301+0000","last_became_active":"2024-09-17T04:24:15.057807+0000","last_became_peered":"2024-09-17T04:24:15.057807+0000","last_unstale":"2024-09-17T04:24:16.056301+0000","last_undegraded":"2024-09-17T04:24:16.056301+0000","last_fullsized":"2024-09-17T04:24:16.056301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:35.584338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224488+0000","last_change":"2024-09-17T04:24:14.063354+0000","last_active":"2024-09-17T04:24:16.224488+0000","last_peered":"2024-09-17T04:24:16.224488+0000","last_clean":"2024-09-17T04:24:16.224488+0000","last_became_active":"2024-09-17T04:24:14.062231+0000","last_became_peered":"2024-09-17T04:24:14.062231+0000","last_unstale":"2024-09-17T04:24:16.224488+0000","last_undegraded":"2024-09-17T04:24:16.224488+0000","last_fullsized":"2024-09-17T04:24:16.224488+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:40.777526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.222850+0000","last_change":"2024-09-17T04:24:15.058346+0000","last_active":"2024-09-17T04:24:16.222850+0000","last_peered":"2024-09-17T04:24:16.222850+0000","last_clean":"2024-09-17T04:24:16.222850+0000","last_became_active":"2024-09-17T04:24:15.057977+0000","last_became_peered":"2024-09-17T04:24:15.057977+0000","last_unstale":"2024-09-17T04:24:16.222850+0000","last_undegraded":"2024-09-17T04:24:16.222850+0000","last_fullsized":"2024-09-17T04:24:16.222850+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:01.706191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223545+0000","last_change":"2024-09-17T04:24:14.049298+0000","last_active":"2024-09-17T04:24:16.223545+0000","last_peered":"2024-09-17T04:24:16.223545+0000","last_clean":"2024-09-17T04:24:16.223545+0000","last_became_active":"2024-09-17T04:24:14.049070+0000","last_became_peered":"2024-09-17T04:24:14.049070+0000","last_unstale":"2024-09-17T04:24:16.223545+0000","last_undegraded":"2024-09-17T04:24:16.223545+0000","last_fullsized":"2024-09-17T04:24:16.223545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:52:59.361386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225582+0000","last_change":"2024-09-17T04:24:15.050480+0000","last_active":"2024-09-17T04:24:16.225582+0000","last_peered":"2024-09-17T04:24:16.225582+0000","last_clean":"2024-09-17T04:24:16.225582+0000","last_became_active":"2024-09-17T04:24:15.050166+0000","last_became_peered":"2024-09-17T04:24:15.050166+0000","last_unstale":"2024-09-17T04:24:16.225582+0000","last_undegraded":"2024-09-17T04:24:16.225582+0000","last_fullsized":"2024-09-17T04:24:16.225582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:02.125884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225264+0000","last_change":"2024-09-17T04:24:14.063222+0000","last_active":"2024-09-17T04:24:16.225264+0000","last_peered":"2024-09-17T04:24:16.225264+0000","last_clean":"2024-09-17T04:24:16.225264+0000","last_became_active":"2024-09-17T04:24:14.062750+0000","last_became_peered":"2024-09-17T04:24:14.062750+0000","last_unstale":"2024-09-17T04:24:16.225264+0000","last_undegraded":"2024-09-17T04:24:16.225264+0000","last_fullsized":"2024-09-17T04:24:16.225264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:08.023686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224491+0000","last_change":"2024-09-17T04:24:15.067950+0000","last_active":"2024-09-17T04:24:16.224491+0000","last_peered":"2024-09-17T04:24:16.224491+0000","last_clean":"2024-09-17T04:24:16.224491+0000","last_became_active":"2024-09-17T04:24:15.066567+0000","last_became_peered":"2024-09-17T04:24:15.066567+0000","last_unstale":"2024-09-17T04:24:16.224491+0000","last_undegraded":"2024-09-17T04:24:16.224491+0000","last_fullsized":"2024-09-17T04:24:16.224491+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:28.104736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223381+0000","last_change":"2024-09-17T04:24:14.059487+0000","last_active":"2024-09-17T04:24:16.223381+0000","last_peered":"2024-09-17T04:24:16.223381+0000","last_clean":"2024-09-17T04:24:16.223381+0000","last_became_active":"2024-09-17T04:24:14.059186+0000","last_became_peered":"2024-09-17T04:24:14.059186+0000","last_unstale":"2024-09-17T04:24:16.223381+0000","last_undegraded":"2024-09-17T04:24:16.223381+0000","last_fullsized":"2024-09-17T04:24:16.223381+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:03.320698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054147+0000","last_change":"2024-09-17T04:24:15.065941+0000","last_active":"2024-09-17T04:24:16.054147+0000","last_peered":"2024-09-17T04:24:16.054147+0000","last_clean":"2024-09-17T04:24:16.054147+0000","last_became_active":"2024-09-17T04:24:15.065582+0000","last_became_peered":"2024-09-17T04:24:15.065582+0000","last_unstale":"2024-09-17T04:24:16.054147+0000","last_undegraded":"2024-09-17T04:24:16.054147+0000","last_fullsized":"2024-09-17T04:24:16.054147+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:30.122754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.222894+0000","last_change":"2024-09-17T04:24:14.062678+0000","last_active":"2024-09-17T04:24:16.222894+0000","last_peered":"2024-09-17T04:24:16.222894+0000","last_clean":"2024-09-17T04:24:16.222894+0000","last_became_active":"2024-09-17T04:24:14.062281+0000","last_became_peered":"2024-09-17T04:24:14.062281+0000","last_unstale":"2024-09-17T04:24:16.222894+0000","last_undegraded":"2024-09-17T04:24:16.222894+0000","last_fullsized":"2024-09-17T04:24:16.222894+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:40.254206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224778+0000","last_change":"2024-09-17T04:24:15.064498+0000","last_active":"2024-09-17T04:24:16.224778+0000","last_peered":"2024-09-17T04:24:16.224778+0000","last_clean":"2024-09-17T04:24:16.224778+0000","last_became_active":"2024-09-17T04:24:15.062700+0000","last_became_peered":"2024-09-17T04:24:15.062700+0000","last_unstale":"2024-09-17T04:24:16.224778+0000","last_undegraded":"2024-09-17T04:24:16.224778+0000","last_fullsized":"2024-09-17T04:24:16.224778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:31.021313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223265+0000","last_change":"2024-09-17T04:24:14.047923+0000","last_active":"2024-09-17T04:24:16.223265+0000","last_peered":"2024-09-17T04:24:16.223265+0000","last_clean":"2024-09-17T04:24:16.223265+0000","last_became_active":"2024-09-17T04:24:14.047596+0000","last_became_peered":"2024-09-17T04:24:14.047596+0000","last_unstale":"2024-09-17T04:24:16.223265+0000","last_undegraded":"2024-09-17T04:24:16.223265+0000","last_fullsized":"2024-09-17T04:24:16.223265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:29.119993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225575+0000","last_change":"2024-09-17T04:24:15.060939+0000","last_active":"2024-09-17T04:24:16.225575+0000","last_peered":"2024-09-17T04:24:16.225575+0000","last_clean":"2024-09-17T04:24:16.225575+0000","last_became_active":"2024-09-17T04:24:15.060600+0000","last_became_peered":"2024-09-17T04:24:15.060600+0000","last_unstale":"2024-09-17T04:24:16.225575+0000","last_undegraded":"2024-09-17T04:24:16.225575+0000","last_fullsized":"2024-09-17T04:24:16.225575+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:05.895665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224176+0000","last_change":"2024-09-17T04:24:14.062616+0000","last_active":"2024-09-17T04:24:16.224176+0000","last_peered":"2024-09-17T04:24:16.224176+0000","last_clean":"2024-09-17T04:24:16.224176+0000","last_became_active":"2024-09-17T04:24:14.062182+0000","last_became_peered":"2024-09-17T04:24:14.062182+0000","last_unstale":"2024-09-17T04:24:16.224176+0000","last_undegraded":"2024-09-17T04:24:16.224176+0000","last_fullsized":"2024-09-17T04:24:16.224176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:51.246396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224637+0000","last_change":"2024-09-17T04:24:15.079589+0000","last_active":"2024-09-17T04:24:16.224637+0000","last_peered":"2024-09-17T04:24:16.224637+0000","last_clean":"2024-09-17T04:24:16.224637+0000","last_became_active":"2024-09-17T04:24:15.079019+0000","last_became_peered":"2024-09-17T04:24:15.079019+0000","last_unstale":"2024-09-17T04:24:16.224637+0000","last_undegraded":"2024-09-17T04:24:16.224637+0000","last_fullsized":"2024-09-17T04:24:16.224637+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:01.706191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056070+0000","last_change":"2024-09-17T04:24:15.080533+0000","last_active":"2024-09-17T04:24:16.056070+0000","last_peered":"2024-09-17T04:24:16.056070+0000","last_clean":"2024-09-17T04:24:16.056070+0000","last_became_active":"2024-09-17T04:24:15.080179+0000","last_became_peered":"2024-09-17T04:24:15.080179+0000","last_unstale":"2024-09-17T04:24:16.056070+0000","last_undegraded":"2024-09-17T04:24:16.056070+0000","last_fullsized":"2024-09-17T04:24:16.056070+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:58.719218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224154+0000","last_change":"2024-09-17T04:24:14.065850+0000","last_active":"2024-09-17T04:24:16.224154+0000","last_peered":"2024-09-17T04:24:16.224154+0000","last_clean":"2024-09-17T04:24:16.224154+0000","last_became_active":"2024-09-17T04:24:14.065444+0000","last_became_peered":"2024-09-17T04:24:14.065444+0000","last_unstale":"2024-09-17T04:24:16.224154+0000","last_undegraded":"2024-09-17T04:24:16.224154+0000","last_fullsized":"2024-09-17T04:24:16.224154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:51.121630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056174+0000","last_change":"2024-09-17T04:24:15.076511+0000","last_active":"2024-09-17T04:24:16.056174+0000","last_peered":"2024-09-17T04:24:16.056174+0000","last_clean":"2024-09-17T04:24:16.056174+0000","last_became_active":"2024-09-17T04:24:15.076037+0000","last_became_peered":"2024-09-17T04:24:15.076037+0000","last_unstale":"2024-09-17T04:24:16.056174+0000","last_undegraded":"2024-09-17T04:24:16.056174+0000","last_fullsized":"2024-09-17T04:24:16.056174+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:06:32.518189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224180+0000","last_change":"2024-09-17T04:24:14.060131+0000","last_active":"2024-09-17T04:24:16.224180+0000","last_peered":"2024-09-17T04:24:16.224180+0000","last_clean":"2024-09-17T04:24:16.224180+0000","last_became_active":"2024-09-17T04:24:14.059707+0000","last_became_peered":"2024-09-17T04:24:14.059707+0000","last_unstale":"2024-09-17T04:24:16.224180+0000","last_undegraded":"2024-09-17T04:24:16.224180+0000","last_fullsized":"2024-09-17T04:24:16.224180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:22.869429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223586+0000","last_change":"2024-09-17T04:24:15.068641+0000","last_active":"2024-09-17T04:24:16.223586+0000","last_peered":"2024-09-17T04:24:16.223586+0000","last_clean":"2024-09-17T04:24:16.223586+0000","last_became_active":"2024-09-17T04:24:15.065609+0000","last_became_peered":"2024-09-17T04:24:15.065609+0000","last_unstale":"2024-09-17T04:24:16.223586+0000","last_undegraded":"2024-09-17T04:24:16.223586+0000","last_fullsized":"2024-09-17T04:24:16.223586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:43:02.004482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225369+0000","last_change":"2024-09-17T04:24:14.056735+0000","last_active":"2024-09-17T04:24:16.225369+0000","last_peered":"2024-09-17T04:24:16.225369+0000","last_clean":"2024-09-17T04:24:16.225369+0000","last_became_active":"2024-09-17T04:24:14.056433+0000","last_became_peered":"2024-09-17T04:24:14.056433+0000","last_unstale":"2024-09-17T04:24:16.225369+0000","last_undegraded":"2024-09-17T04:24:16.225369+0000","last_fullsized":"2024-09-17T04:24:16.225369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:03.618384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225403+0000","last_change":"2024-09-17T04:24:15.054773+0000","last_active":"2024-09-17T04:24:16.225403+0000","last_peered":"2024-09-17T04:24:16.225403+0000","last_clean":"2024-09-17T04:24:16.225403+0000","last_became_active":"2024-09-17T04:24:15.054453+0000","last_became_peered":"2024-09-17T04:24:15.054453+0000","last_unstale":"2024-09-17T04:24:16.225403+0000","last_undegraded":"2024-09-17T04:24:16.225403+0000","last_fullsized":"2024-09-17T04:24:16.225403+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:58.942093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223981+0000","last_change":"2024-09-17T04:24:14.062995+0000","last_active":"2024-09-17T04:24:16.223981+0000","last_peered":"2024-09-17T04:24:16.223981+0000","last_clean":"2024-09-17T04:24:16.223981+0000","last_became_active":"2024-09-17T04:24:14.062615+0000","last_became_peered":"2024-09-17T04:24:14.062615+0000","last_unstale":"2024-09-17T04:24:16.223981+0000","last_undegraded":"2024-09-17T04:24:16.223981+0000","last_fullsized":"2024-09-17T04:24:16.223981+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:20:07.263105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476743,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27844,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343996,"statfs":{"total":96636764160,"available":96608251904,"internally_reserved":0,"allocated":827392,"data_stored":670785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476742,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":81447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476743,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476743,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":79961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":79835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":237568,"data_stored":79995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":835584,"data_stored":670841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":81447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:24:20.435 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T04:24:20.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.689+0000 7f5205000640 1 -- 172.21.15.100:0/4116766271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52000ad0e0 msgr2=0x7f52000ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:20.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.689+0000 7f5205000640 1 --2- 172.21.15.100:0/4116766271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52000ad0e0 0x7f52000ad4f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5200075250 tx=0x7f51e802a560 comp rx=0 tx=0).stop 2024-09-17T04:24:20.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.689+0000 7f5205000640 1 -- 172.21.15.100:0/4116766271 shutdown_connections 2024-09-17T04:24:20.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.689+0000 7f5205000640 1 --2- 172.21.15.100:0/4116766271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52000ad0e0 0x7f52000ad4f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:20.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.689+0000 7f5205000640 1 --2- 172.21.15.100:0/4116766271 >> v2:172.21.15.184:3300/0 conn(0x7f520015d760 0x7f52001535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:20.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.689+0000 7f5205000640 1 -- 172.21.15.100:0/4116766271 >> 172.21.15.100:0/4116766271 conn(0x7f520006d730 msgr2=0x7f52000b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:20.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.689+0000 7f5205000640 1 -- 172.21.15.100:0/4116766271 shutdown_connections 2024-09-17T04:24:20.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.689+0000 7f5205000640 1 -- 172.21.15.100:0/4116766271 wait complete. 2024-09-17T04:24:20.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.691+0000 7f5205000640 1 Processor -- start 2024-09-17T04:24:20.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.691+0000 7f5205000640 1 -- start start 2024-09-17T04:24:20.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.693+0000 7f5205000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52000ad0e0 0x7f5200146760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:20.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.693+0000 7f5205000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f520015d760 0x7f52000b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:20.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.693+0000 7f5205000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5200160310 0x7f52000b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:20.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.693+0000 7f5205000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f52000b4c20 con 0x7f520015d760 2024-09-17T04:24:20.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.693+0000 7f5205000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5200153c70 con 0x7f52000ad0e0 2024-09-17T04:24:20.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.693+0000 7f5205000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f52000b4da0 con 0x7f5200160310 2024-09-17T04:24:20.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.693+0000 7f51f5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f520015d760 0x7f52000b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:20.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.693+0000 7f51f5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f520015d760 0x7f52000b1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41610/0 (socket says 172.21.15.100:41610) 2024-09-17T04:24:20.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.693+0000 7f51f5400640 1 -- 172.21.15.100:0/1145331653 learned_addr learned my addr 172.21.15.100:0/1145331653 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:20.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.693+0000 7f51fde00640 1 --2- 172.21.15.100:0/1145331653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52000ad0e0 0x7f5200146760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:20.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.693+0000 7f51fe800640 1 --2- 172.21.15.100:0/1145331653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5200160310 0x7f52000b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:20.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.694+0000 7f51f5400640 1 -- 172.21.15.100:0/1145331653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5200160310 msgr2=0x7f52000b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:20.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.694+0000 7f51f5400640 1 --2- 172.21.15.100:0/1145331653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5200160310 0x7f52000b16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:20.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.694+0000 7f51f5400640 1 -- 172.21.15.100:0/1145331653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52000ad0e0 msgr2=0x7f5200146760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:20.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.694+0000 7f51f5400640 1 --2- 172.21.15.100:0/1145331653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52000ad0e0 0x7f5200146760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:20.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.694+0000 7f51f5400640 1 -- 172.21.15.100:0/1145331653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52001f6b50 con 0x7f520015d760 2024-09-17T04:24:20.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.694+0000 7f51f5400640 1 --2- 172.21.15.100:0/1145331653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f520015d760 0x7f52000b1170 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f51e8000c00 tx=0x7f51e8003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:20.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.694+0000 7f51f6a00640 1 -- 172.21.15.100:0/1145331653 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e803e030 con 0x7f520015d760 2024-09-17T04:24:20.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.694+0000 7f51f6a00640 1 -- 172.21.15.100:0/1145331653 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51e8003d10 con 0x7f520015d760 2024-09-17T04:24:20.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.694+0000 7f5205000640 1 -- 172.21.15.100:0/1145331653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52001f5fd0 con 0x7f520015d760 2024-09-17T04:24:20.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.694+0000 7f51f6a00640 1 -- 172.21.15.100:0/1145331653 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e8035040 con 0x7f520015d760 2024-09-17T04:24:20.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.694+0000 7f5205000640 1 -- 172.21.15.100:0/1145331653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52001f6590 con 0x7f520015d760 2024-09-17T04:24:20.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.695+0000 7f5205000640 1 -- 172.21.15.100:0/1145331653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5200160ff0 con 0x7f520015d760 2024-09-17T04:24:20.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.697+0000 7f51f6a00640 1 -- 172.21.15.100:0/1145331653 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f51e8041030 con 0x7f520015d760 2024-09-17T04:24:20.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.697+0000 7f51f6a00640 1 --2- 172.21.15.100:0/1145331653 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f51dc078b90 0x7f51dc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:20.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.698+0000 7f51f6a00640 1 -- 172.21.15.100:0/1145331653 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f51e80bb360 con 0x7f520015d760 2024-09-17T04:24:20.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.698+0000 7f51fde00640 1 --2- 172.21.15.100:0/1145331653 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f51dc078b90 0x7f51dc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:20.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.699+0000 7f51fde00640 1 --2- 172.21.15.100:0/1145331653 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f51dc078b90 0x7f51dc07b0a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009b30 tx=0x7f51f0005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:20.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.702+0000 7f51f6a00640 1 -- 172.21.15.100:0/1145331653 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f51e80831d0 con 0x7f520015d760 2024-09-17T04:24:20.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.956+0000 7f5205000640 1 -- 172.21.15.100:0/1145331653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f520015cfa0 con 0x7f520015d760 2024-09-17T04:24:20.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.956+0000 7f51f6a00640 1 -- 172.21.15.100:0/1145331653 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1053 (secure 0 0 0) 0x7f51e8087c00 con 0x7f520015d760 2024-09-17T04:24:20.957 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:20.957 INFO:teuthology.orchestra.run.smithi100.stdout:{"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":34,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":23,"num_osds":8,"num_up_osds":8,"osd_up_since":1726547043,"num_in_osds":8,"osd_in_since":1726547032,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":224538624,"bytes_avail":772869574656,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":8,"btime":"2024-09-17T04:24:18:058994+0000","id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4368}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-17T04:23:46.838483+0000","services":{}},"progress_events":{}} 2024-09-17T04:24:20.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.958+0000 7f5205000640 1 -- 172.21.15.100:0/1145331653 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f51dc078b90 msgr2=0x7f51dc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:20.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.958+0000 7f5205000640 1 --2- 172.21.15.100:0/1145331653 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f51dc078b90 0x7f51dc07b0a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009b30 tx=0x7f51f0005f90 comp rx=0 tx=0).stop 2024-09-17T04:24:20.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.958+0000 7f5205000640 1 -- 172.21.15.100:0/1145331653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f520015d760 msgr2=0x7f52000b1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:20.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.958+0000 7f5205000640 1 --2- 172.21.15.100:0/1145331653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f520015d760 0x7f52000b1170 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f51e8000c00 tx=0x7f51e8003820 comp rx=0 tx=0).stop 2024-09-17T04:24:20.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.959+0000 7f5205000640 1 -- 172.21.15.100:0/1145331653 shutdown_connections 2024-09-17T04:24:20.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.959+0000 7f5205000640 1 --2- 172.21.15.100:0/1145331653 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f51dc078b90 0x7f51dc07b0a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:20.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.959+0000 7f5205000640 1 --2- 172.21.15.100:0/1145331653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5200160310 0x7f52000b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:20.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.959+0000 7f5205000640 1 --2- 172.21.15.100:0/1145331653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f520015d760 0x7f52000b1170 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:20.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.959+0000 7f5205000640 1 --2- 172.21.15.100:0/1145331653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52000ad0e0 0x7f5200146760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:20.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.959+0000 7f5205000640 1 -- 172.21.15.100:0/1145331653 >> 172.21.15.100:0/1145331653 conn(0x7f520006d730 msgr2=0x7f52000d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:20.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.959+0000 7f5205000640 1 -- 172.21.15.100:0/1145331653 shutdown_connections 2024-09-17T04:24:20.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:20.959+0000 7f5205000640 1 -- 172.21.15.100:0/1145331653 wait complete. 2024-09-17T04:24:20.979 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T04:24:20.980 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:24:21.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.201+0000 7f62e1600640 1 -- 172.21.15.100:0/3755703118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62c0004000 msgr2=0x7f62c0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:21.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.201+0000 7f62e1600640 1 --2- 172.21.15.100:0/3755703118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62c0004000 0x7f62c0004800 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f62c400c2e0 tx=0x7f62c0005830 comp rx=0 tx=0).stop 2024-09-17T04:24:21.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.201+0000 7f62db400640 1 -- 172.21.15.100:0/3755703118 reap_dead start 2024-09-17T04:24:21.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.201+0000 7f62e1600640 1 -- 172.21.15.100:0/3755703118 shutdown_connections 2024-09-17T04:24:21.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.201+0000 7f62e1600640 1 -- 172.21.15.100:0/3755703118 >> 172.21.15.100:0/3755703118 conn(0x7f62dc06d730 msgr2=0x7f62dc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:21.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.201+0000 7f62e1600640 1 -- 172.21.15.100:0/3755703118 shutdown_connections 2024-09-17T04:24:21.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.201+0000 7f62e1600640 1 -- 172.21.15.100:0/3755703118 wait complete. 2024-09-17T04:24:21.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.203+0000 7f62e1600640 1 Processor -- start 2024-09-17T04:24:21.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.203+0000 7f62e1600640 1 -- start start 2024-09-17T04:24:21.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.205+0000 7f62e1600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62c0004000 0x7f62dc0994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:21.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.205+0000 7f62e1600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62dc1f49f0 0x7f62dc1f4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:21.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.205+0000 7f62e1600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62dc1f5340 0x7f62dc1f3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:21.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.205+0000 7f62e1600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f62dc0b4070 con 0x7f62dc1f49f0 2024-09-17T04:24:21.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.205+0000 7f62e1600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f62dc0b4e90 con 0x7f62c0004000 2024-09-17T04:24:21.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.205+0000 7f62e1600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f62dc0b41f0 con 0x7f62dc1f5340 2024-09-17T04:24:21.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.205+0000 7f62d2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62dc1f49f0 0x7f62dc1f4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:21.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.205+0000 7f62d2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62dc1f49f0 0x7f62dc1f4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41626/0 (socket says 172.21.15.100:41626) 2024-09-17T04:24:21.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.205+0000 7f62d2000640 1 -- 172.21.15.100:0/3222166766 learned_addr learned my addr 172.21.15.100:0/3222166766 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:21.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.205+0000 7f62db400640 1 --2- 172.21.15.100:0/3222166766 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62dc1f5340 0x7f62dc1f3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:21.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.205+0000 7f62daa00640 1 --2- 172.21.15.100:0/3222166766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62c0004000 0x7f62dc0994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:21.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.205+0000 7f62d2000640 1 -- 172.21.15.100:0/3222166766 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62dc1f5340 msgr2=0x7f62dc1f3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:21.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.205+0000 7f62d2000640 1 --2- 172.21.15.100:0/3222166766 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62dc1f5340 0x7f62dc1f3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:21.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.205+0000 7f62d2000640 1 -- 172.21.15.100:0/3222166766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62c0004000 msgr2=0x7f62dc0994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:21.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.205+0000 7f62d2000640 1 --2- 172.21.15.100:0/3222166766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62c0004000 0x7f62dc0994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:21.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.205+0000 7f62d2000640 1 -- 172.21.15.100:0/3222166766 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62dc14b1f0 con 0x7f62dc1f49f0 2024-09-17T04:24:21.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.206+0000 7f62d2000640 1 --2- 172.21.15.100:0/3222166766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62dc1f49f0 0x7f62dc1f4e00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f62c400c3d0 tx=0x7f62c4004bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:21.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.206+0000 7f62d3400640 1 -- 172.21.15.100:0/3222166766 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c4031040 con 0x7f62dc1f49f0 2024-09-17T04:24:21.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.206+0000 7f62d3400640 1 -- 172.21.15.100:0/3222166766 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62c4006420 con 0x7f62dc1f49f0 2024-09-17T04:24:21.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.206+0000 7f62e1600640 1 -- 172.21.15.100:0/3222166766 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62dc14b4b0 con 0x7f62dc1f49f0 2024-09-17T04:24:21.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.206+0000 7f62d3400640 1 -- 172.21.15.100:0/3222166766 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c4006c90 con 0x7f62dc1f49f0 2024-09-17T04:24:21.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.206+0000 7f62e1600640 1 -- 172.21.15.100:0/3222166766 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62dc14bf90 con 0x7f62dc1f49f0 2024-09-17T04:24:21.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.207+0000 7f62e1600640 1 -- 172.21.15.100:0/3222166766 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62dc09a740 con 0x7f62dc1f49f0 2024-09-17T04:24:21.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.208+0000 7f62d3400640 1 -- 172.21.15.100:0/3222166766 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f62c4034070 con 0x7f62dc1f49f0 2024-09-17T04:24:21.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.209+0000 7f62d3400640 1 --2- 172.21.15.100:0/3222166766 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f62b8078bc0 0x7f62b807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:21.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.209+0000 7f62d3400640 1 -- 172.21.15.100:0/3222166766 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f62c40bb090 con 0x7f62dc1f49f0 2024-09-17T04:24:21.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.210+0000 7f62daa00640 1 --2- 172.21.15.100:0/3222166766 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f62b8078bc0 0x7f62b807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:21.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.211+0000 7f62daa00640 1 --2- 172.21.15.100:0/3222166766 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f62b8078bc0 0x7f62b807b0d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009c50 tx=0x7f62cc005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:21.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.214+0000 7f62d3400640 1 -- 172.21.15.100:0/3222166766 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f62c4083550 con 0x7f62dc1f49f0 2024-09-17T04:24:21.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.392+0000 7f62e1600640 1 -- 172.21.15.100:0/3222166766 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f62dc09a9d0 con 0x7f62b8078bc0 2024-09-17T04:24:21.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.397+0000 7f62d3400640 1 -- 172.21.15.100:0/3222166766 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251937 (secure 0 0 0) 0x7f62cc05e840 con 0x7f62b8078bc0 2024-09-17T04:24:21.401 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:21.402 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:24:21.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.401+0000 7f62e1600640 1 -- 172.21.15.100:0/3222166766 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f62b8078bc0 msgr2=0x7f62b807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:21.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.401+0000 7f62e1600640 1 --2- 172.21.15.100:0/3222166766 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f62b8078bc0 0x7f62b807b0d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009c50 tx=0x7f62cc005f90 comp rx=0 tx=0).stop 2024-09-17T04:24:21.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.401+0000 7f62e1600640 1 -- 172.21.15.100:0/3222166766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62dc1f49f0 msgr2=0x7f62dc1f4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:21.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.401+0000 7f62e1600640 1 --2- 172.21.15.100:0/3222166766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62dc1f49f0 0x7f62dc1f4e00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f62c400c3d0 tx=0x7f62c4004bc0 comp rx=0 tx=0).stop 2024-09-17T04:24:21.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.401+0000 7f62e1600640 1 -- 172.21.15.100:0/3222166766 shutdown_connections 2024-09-17T04:24:21.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.401+0000 7f62e1600640 1 --2- 172.21.15.100:0/3222166766 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f62b8078bc0 0x7f62b807b0d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:21.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.401+0000 7f62e1600640 1 --2- 172.21.15.100:0/3222166766 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62dc1f5340 0x7f62dc1f3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:21.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.401+0000 7f62e1600640 1 --2- 172.21.15.100:0/3222166766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62dc1f49f0 0x7f62dc1f4e00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:21.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.401+0000 7f62e1600640 1 --2- 172.21.15.100:0/3222166766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62c0004000 0x7f62dc0994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:21.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.401+0000 7f62e1600640 1 -- 172.21.15.100:0/3222166766 >> 172.21.15.100:0/3222166766 conn(0x7f62dc06d730 msgr2=0x7f62dc15e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:21.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.401+0000 7f62e1600640 1 -- 172.21.15.100:0/3222166766 shutdown_connections 2024-09-17T04:24:21.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:21.401+0000 7f62e1600640 1 -- 172.21.15.100:0/3222166766 wait complete. 2024-09-17T04:24:21.422 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-17T04:24:19.965152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219276,"kb_used_data":3016,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755444,"statfs":{"total":773094113280,"available":772869574656,"internally_reserved":0,"allocated":3088384,"data_stored":1824236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001746"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224232+0000","last_change":"2024-09-17T04:24:15.051472+0000","last_active":"2024-09-17T04:24:16.224232+0000","last_peered":"2024-09-17T04:24:16.224232+0000","last_clean":"2024-09-17T04:24:16.224232+0000","last_became_active":"2024-09-17T04:24:15.051162+0000","last_became_peered":"2024-09-17T04:24:15.051162+0000","last_unstale":"2024-09-17T04:24:16.224232+0000","last_undegraded":"2024-09-17T04:24:16.224232+0000","last_fullsized":"2024-09-17T04:24:16.224232+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:35:10.792366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223431+0000","last_change":"2024-09-17T04:24:15.069649+0000","last_active":"2024-09-17T04:24:16.223431+0000","last_peered":"2024-09-17T04:24:16.223431+0000","last_clean":"2024-09-17T04:24:16.223431+0000","last_became_active":"2024-09-17T04:24:15.068975+0000","last_became_peered":"2024-09-17T04:24:15.068975+0000","last_unstale":"2024-09-17T04:24:16.223431+0000","last_undegraded":"2024-09-17T04:24:16.223431+0000","last_fullsized":"2024-09-17T04:24:16.223431+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:47.639252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223497+0000","last_change":"2024-09-17T04:24:15.052608+0000","last_active":"2024-09-17T04:24:16.223497+0000","last_peered":"2024-09-17T04:24:16.223497+0000","last_clean":"2024-09-17T04:24:16.223497+0000","last_became_active":"2024-09-17T04:24:15.052252+0000","last_became_peered":"2024-09-17T04:24:15.052252+0000","last_unstale":"2024-09-17T04:24:16.223497+0000","last_undegraded":"2024-09-17T04:24:16.223497+0000","last_fullsized":"2024-09-17T04:24:16.223497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:34:23.523300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055891+0000","last_change":"2024-09-17T04:24:15.059215+0000","last_active":"2024-09-17T04:24:16.055891+0000","last_peered":"2024-09-17T04:24:16.055891+0000","last_clean":"2024-09-17T04:24:16.055891+0000","last_became_active":"2024-09-17T04:24:15.058921+0000","last_became_peered":"2024-09-17T04:24:15.058921+0000","last_unstale":"2024-09-17T04:24:16.055891+0000","last_undegraded":"2024-09-17T04:24:16.055891+0000","last_fullsized":"2024-09-17T04:24:16.055891+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:34.142688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224122+0000","last_change":"2024-09-17T04:24:15.042545+0000","last_active":"2024-09-17T04:24:16.224122+0000","last_peered":"2024-09-17T04:24:16.224122+0000","last_clean":"2024-09-17T04:24:16.224122+0000","last_became_active":"2024-09-17T04:24:15.042235+0000","last_became_peered":"2024-09-17T04:24:15.042235+0000","last_unstale":"2024-09-17T04:24:16.224122+0000","last_undegraded":"2024-09-17T04:24:16.224122+0000","last_fullsized":"2024-09-17T04:24:16.224122+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:06:32.518189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056013+0000","last_change":"2024-09-17T04:24:15.057624+0000","last_active":"2024-09-17T04:24:16.056013+0000","last_peered":"2024-09-17T04:24:16.056013+0000","last_clean":"2024-09-17T04:24:16.056013+0000","last_became_active":"2024-09-17T04:24:15.057329+0000","last_became_peered":"2024-09-17T04:24:15.057329+0000","last_unstale":"2024-09-17T04:24:16.056013+0000","last_undegraded":"2024-09-17T04:24:16.056013+0000","last_fullsized":"2024-09-17T04:24:16.056013+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:44.355705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223880+0000","last_change":"2024-09-17T04:24:15.062645+0000","last_active":"2024-09-17T04:24:16.223880+0000","last_peered":"2024-09-17T04:24:16.223880+0000","last_clean":"2024-09-17T04:24:16.223880+0000","last_became_active":"2024-09-17T04:24:15.062227+0000","last_became_peered":"2024-09-17T04:24:15.062227+0000","last_unstale":"2024-09-17T04:24:16.223880+0000","last_undegraded":"2024-09-17T04:24:16.223880+0000","last_fullsized":"2024-09-17T04:24:16.223880+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:13.159492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224308+0000","last_change":"2024-09-17T04:24:15.062993+0000","last_active":"2024-09-17T04:24:16.224308+0000","last_peered":"2024-09-17T04:24:16.224308+0000","last_clean":"2024-09-17T04:24:16.224308+0000","last_became_active":"2024-09-17T04:24:15.062311+0000","last_became_peered":"2024-09-17T04:24:15.062311+0000","last_unstale":"2024-09-17T04:24:16.224308+0000","last_undegraded":"2024-09-17T04:24:16.224308+0000","last_fullsized":"2024-09-17T04:24:16.224308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:26:42.504856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224742+0000","last_change":"2024-09-17T04:24:15.070288+0000","last_active":"2024-09-17T04:24:16.224742+0000","last_peered":"2024-09-17T04:24:16.224742+0000","last_clean":"2024-09-17T04:24:16.224742+0000","last_became_active":"2024-09-17T04:24:15.068589+0000","last_became_peered":"2024-09-17T04:24:15.068589+0000","last_unstale":"2024-09-17T04:24:16.224742+0000","last_undegraded":"2024-09-17T04:24:16.224742+0000","last_fullsized":"2024-09-17T04:24:16.224742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:33.110035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225192+0000","last_change":"2024-09-17T04:24:15.062446+0000","last_active":"2024-09-17T04:24:16.225192+0000","last_peered":"2024-09-17T04:24:16.225192+0000","last_clean":"2024-09-17T04:24:16.225192+0000","last_became_active":"2024-09-17T04:24:15.060522+0000","last_became_peered":"2024-09-17T04:24:15.060522+0000","last_unstale":"2024-09-17T04:24:16.225192+0000","last_undegraded":"2024-09-17T04:24:16.225192+0000","last_fullsized":"2024-09-17T04:24:16.225192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:09.349585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224815+0000","last_change":"2024-09-17T04:24:15.062048+0000","last_active":"2024-09-17T04:24:16.224815+0000","last_peered":"2024-09-17T04:24:16.224815+0000","last_clean":"2024-09-17T04:24:16.224815+0000","last_became_active":"2024-09-17T04:24:15.060887+0000","last_became_peered":"2024-09-17T04:24:15.060887+0000","last_unstale":"2024-09-17T04:24:16.224815+0000","last_undegraded":"2024-09-17T04:24:16.224815+0000","last_fullsized":"2024-09-17T04:24:16.224815+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:22.452641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054679+0000","last_change":"2024-09-17T04:24:15.068320+0000","last_active":"2024-09-17T04:24:16.054679+0000","last_peered":"2024-09-17T04:24:16.054679+0000","last_clean":"2024-09-17T04:24:16.054679+0000","last_became_active":"2024-09-17T04:24:15.067973+0000","last_became_peered":"2024-09-17T04:24:15.067973+0000","last_unstale":"2024-09-17T04:24:16.054679+0000","last_undegraded":"2024-09-17T04:24:16.054679+0000","last_fullsized":"2024-09-17T04:24:16.054679+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:04.323459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225225+0000","last_change":"2024-09-17T04:24:15.047951+0000","last_active":"2024-09-17T04:24:16.225225+0000","last_peered":"2024-09-17T04:24:16.225225+0000","last_clean":"2024-09-17T04:24:16.225225+0000","last_became_active":"2024-09-17T04:24:15.047536+0000","last_became_peered":"2024-09-17T04:24:15.047536+0000","last_unstale":"2024-09-17T04:24:16.225225+0000","last_undegraded":"2024-09-17T04:24:16.225225+0000","last_fullsized":"2024-09-17T04:24:16.225225+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:38:48.386032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224034+0000","last_change":"2024-09-17T04:24:15.070607+0000","last_active":"2024-09-17T04:24:16.224034+0000","last_peered":"2024-09-17T04:24:16.224034+0000","last_clean":"2024-09-17T04:24:16.224034+0000","last_became_active":"2024-09-17T04:24:15.067087+0000","last_became_peered":"2024-09-17T04:24:15.067087+0000","last_unstale":"2024-09-17T04:24:16.224034+0000","last_undegraded":"2024-09-17T04:24:16.224034+0000","last_fullsized":"2024-09-17T04:24:16.224034+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:09.349585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224091+0000","last_change":"2024-09-17T04:24:15.076095+0000","last_active":"2024-09-17T04:24:16.224091+0000","last_peered":"2024-09-17T04:24:16.224091+0000","last_clean":"2024-09-17T04:24:16.224091+0000","last_became_active":"2024-09-17T04:24:15.075803+0000","last_became_peered":"2024-09-17T04:24:15.075803+0000","last_unstale":"2024-09-17T04:24:16.224091+0000","last_undegraded":"2024-09-17T04:24:16.224091+0000","last_fullsized":"2024-09-17T04:24:16.224091+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:28.104736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055352+0000","last_change":"2024-09-17T04:24:15.053797+0000","last_active":"2024-09-17T04:24:16.055352+0000","last_peered":"2024-09-17T04:24:16.055352+0000","last_clean":"2024-09-17T04:24:16.055352+0000","last_became_active":"2024-09-17T04:24:15.053449+0000","last_became_peered":"2024-09-17T04:24:15.053449+0000","last_unstale":"2024-09-17T04:24:16.055352+0000","last_undegraded":"2024-09-17T04:24:16.055352+0000","last_fullsized":"2024-09-17T04:24:16.055352+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:49:03.743620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056725+0000","last_change":"2024-09-17T04:24:15.055632+0000","last_active":"2024-09-17T04:24:16.056725+0000","last_peered":"2024-09-17T04:24:16.056725+0000","last_clean":"2024-09-17T04:24:16.056725+0000","last_became_active":"2024-09-17T04:24:15.055156+0000","last_became_peered":"2024-09-17T04:24:15.055156+0000","last_unstale":"2024-09-17T04:24:16.056725+0000","last_undegraded":"2024-09-17T04:24:16.056725+0000","last_fullsized":"2024-09-17T04:24:16.056725+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:16.120591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.052124+0000","last_change":"2024-09-17T04:24:15.057788+0000","last_active":"2024-09-17T04:24:16.052124+0000","last_peered":"2024-09-17T04:24:16.052124+0000","last_clean":"2024-09-17T04:24:16.052124+0000","last_became_active":"2024-09-17T04:24:15.057141+0000","last_became_peered":"2024-09-17T04:24:15.057141+0000","last_unstale":"2024-09-17T04:24:16.052124+0000","last_undegraded":"2024-09-17T04:24:16.052124+0000","last_fullsized":"2024-09-17T04:24:16.052124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:10.704647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224426+0000","last_change":"2024-09-17T04:24:15.067312+0000","last_active":"2024-09-17T04:24:16.224426+0000","last_peered":"2024-09-17T04:24:16.224426+0000","last_clean":"2024-09-17T04:24:16.224426+0000","last_became_active":"2024-09-17T04:24:15.059516+0000","last_became_peered":"2024-09-17T04:24:15.059516+0000","last_unstale":"2024-09-17T04:24:16.224426+0000","last_undegraded":"2024-09-17T04:24:16.224426+0000","last_fullsized":"2024-09-17T04:24:16.224426+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:16.120591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224948+0000","last_change":"2024-09-17T04:24:15.076912+0000","last_active":"2024-09-17T04:24:16.224948+0000","last_peered":"2024-09-17T04:24:16.224948+0000","last_clean":"2024-09-17T04:24:16.224948+0000","last_became_active":"2024-09-17T04:24:15.076617+0000","last_became_peered":"2024-09-17T04:24:15.076617+0000","last_unstale":"2024-09-17T04:24:16.224948+0000","last_undegraded":"2024-09-17T04:24:16.224948+0000","last_fullsized":"2024-09-17T04:24:16.224948+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:47.639252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223809+0000","last_change":"2024-09-17T04:24:15.070958+0000","last_active":"2024-09-17T04:24:16.223809+0000","last_peered":"2024-09-17T04:24:16.223809+0000","last_clean":"2024-09-17T04:24:16.223809+0000","last_became_active":"2024-09-17T04:24:15.070624+0000","last_became_peered":"2024-09-17T04:24:15.070624+0000","last_unstale":"2024-09-17T04:24:16.223809+0000","last_undegraded":"2024-09-17T04:24:16.223809+0000","last_fullsized":"2024-09-17T04:24:16.223809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:13.752351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055648+0000","last_change":"2024-09-17T04:24:15.058815+0000","last_active":"2024-09-17T04:24:16.055648+0000","last_peered":"2024-09-17T04:24:16.055648+0000","last_clean":"2024-09-17T04:24:16.055648+0000","last_became_active":"2024-09-17T04:24:15.058482+0000","last_became_peered":"2024-09-17T04:24:15.058482+0000","last_unstale":"2024-09-17T04:24:16.055648+0000","last_undegraded":"2024-09-17T04:24:16.055648+0000","last_fullsized":"2024-09-17T04:24:16.055648+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:18.409788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224382+0000","last_change":"2024-09-17T04:24:15.070200+0000","last_active":"2024-09-17T04:24:16.224382+0000","last_peered":"2024-09-17T04:24:16.224382+0000","last_clean":"2024-09-17T04:24:16.224382+0000","last_became_active":"2024-09-17T04:24:15.069872+0000","last_became_peered":"2024-09-17T04:24:15.069872+0000","last_unstale":"2024-09-17T04:24:16.224382+0000","last_undegraded":"2024-09-17T04:24:16.224382+0000","last_fullsized":"2024-09-17T04:24:16.224382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:22.880022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055404+0000","last_change":"2024-09-17T04:24:15.050935+0000","last_active":"2024-09-17T04:24:16.055404+0000","last_peered":"2024-09-17T04:24:16.055404+0000","last_clean":"2024-09-17T04:24:16.055404+0000","last_became_active":"2024-09-17T04:24:15.048946+0000","last_became_peered":"2024-09-17T04:24:15.048946+0000","last_unstale":"2024-09-17T04:24:16.055404+0000","last_undegraded":"2024-09-17T04:24:16.055404+0000","last_fullsized":"2024-09-17T04:24:16.055404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:11:09.116226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225400+0000","last_change":"2024-09-17T04:24:15.074863+0000","last_active":"2024-09-17T04:24:16.225400+0000","last_peered":"2024-09-17T04:24:16.225400+0000","last_clean":"2024-09-17T04:24:16.225400+0000","last_became_active":"2024-09-17T04:24:15.074467+0000","last_became_peered":"2024-09-17T04:24:15.074467+0000","last_unstale":"2024-09-17T04:24:16.225400+0000","last_undegraded":"2024-09-17T04:24:16.225400+0000","last_fullsized":"2024-09-17T04:24:16.225400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:36:59.215343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225206+0000","last_change":"2024-09-17T04:24:15.069430+0000","last_active":"2024-09-17T04:24:16.225206+0000","last_peered":"2024-09-17T04:24:16.225206+0000","last_clean":"2024-09-17T04:24:16.225206+0000","last_became_active":"2024-09-17T04:24:15.069117+0000","last_became_peered":"2024-09-17T04:24:15.069117+0000","last_unstale":"2024-09-17T04:24:16.225206+0000","last_undegraded":"2024-09-17T04:24:16.225206+0000","last_fullsized":"2024-09-17T04:24:16.225206+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:49:03.743620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224018+0000","last_change":"2024-09-17T04:24:15.057539+0000","last_active":"2024-09-17T04:24:16.224018+0000","last_peered":"2024-09-17T04:24:16.224018+0000","last_clean":"2024-09-17T04:24:16.224018+0000","last_became_active":"2024-09-17T04:24:15.057194+0000","last_became_peered":"2024-09-17T04:24:15.057194+0000","last_unstale":"2024-09-17T04:24:16.224018+0000","last_undegraded":"2024-09-17T04:24:16.224018+0000","last_fullsized":"2024-09-17T04:24:16.224018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:09.349585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225430+0000","last_change":"2024-09-17T04:24:15.075902+0000","last_active":"2024-09-17T04:24:16.225430+0000","last_peered":"2024-09-17T04:24:16.225430+0000","last_clean":"2024-09-17T04:24:16.225430+0000","last_became_active":"2024-09-17T04:24:15.075625+0000","last_became_peered":"2024-09-17T04:24:15.075625+0000","last_unstale":"2024-09-17T04:24:16.225430+0000","last_undegraded":"2024-09-17T04:24:16.225430+0000","last_fullsized":"2024-09-17T04:24:16.225430+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:34.202614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223788+0000","last_change":"2024-09-17T04:24:15.052108+0000","last_active":"2024-09-17T04:24:16.223788+0000","last_peered":"2024-09-17T04:24:16.223788+0000","last_clean":"2024-09-17T04:24:16.223788+0000","last_became_active":"2024-09-17T04:24:15.051794+0000","last_became_peered":"2024-09-17T04:24:15.051794+0000","last_unstale":"2024-09-17T04:24:16.223788+0000","last_undegraded":"2024-09-17T04:24:16.223788+0000","last_fullsized":"2024-09-17T04:24:16.223788+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:13:56.486773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055909+0000","last_change":"2024-09-17T04:24:15.057335+0000","last_active":"2024-09-17T04:24:16.055909+0000","last_peered":"2024-09-17T04:24:16.055909+0000","last_clean":"2024-09-17T04:24:16.055909+0000","last_became_active":"2024-09-17T04:24:15.056924+0000","last_became_peered":"2024-09-17T04:24:15.056924+0000","last_unstale":"2024-09-17T04:24:16.055909+0000","last_undegraded":"2024-09-17T04:24:16.055909+0000","last_fullsized":"2024-09-17T04:24:16.055909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:29:56.515791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.052052+0000","last_change":"2024-09-17T04:24:15.054070+0000","last_active":"2024-09-17T04:24:16.052052+0000","last_peered":"2024-09-17T04:24:16.052052+0000","last_clean":"2024-09-17T04:24:16.052052+0000","last_became_active":"2024-09-17T04:24:15.052619+0000","last_became_peered":"2024-09-17T04:24:15.052619+0000","last_unstale":"2024-09-17T04:24:16.052052+0000","last_undegraded":"2024-09-17T04:24:16.052052+0000","last_fullsized":"2024-09-17T04:24:16.052052+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:49.409374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223562+0000","last_change":"2024-09-17T04:24:15.061084+0000","last_active":"2024-09-17T04:24:16.223562+0000","last_peered":"2024-09-17T04:24:16.223562+0000","last_clean":"2024-09-17T04:24:16.223562+0000","last_became_active":"2024-09-17T04:24:15.060769+0000","last_became_peered":"2024-09-17T04:24:15.060769+0000","last_unstale":"2024-09-17T04:24:16.223562+0000","last_undegraded":"2024-09-17T04:24:16.223562+0000","last_fullsized":"2024-09-17T04:24:16.223562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:13:56.486773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054106+0000","last_change":"2024-09-17T04:24:15.068899+0000","last_active":"2024-09-17T04:24:16.054106+0000","last_peered":"2024-09-17T04:24:16.054106+0000","last_clean":"2024-09-17T04:24:16.054106+0000","last_became_active":"2024-09-17T04:24:15.068595+0000","last_became_peered":"2024-09-17T04:24:15.068595+0000","last_unstale":"2024-09-17T04:24:16.054106+0000","last_undegraded":"2024-09-17T04:24:16.054106+0000","last_fullsized":"2024-09-17T04:24:16.054106+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:13.785027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225374+0000","last_change":"2024-09-17T04:24:14.062806+0000","last_active":"2024-09-17T04:24:16.225374+0000","last_peered":"2024-09-17T04:24:16.225374+0000","last_clean":"2024-09-17T04:24:16.225374+0000","last_became_active":"2024-09-17T04:24:14.062282+0000","last_became_peered":"2024-09-17T04:24:14.062282+0000","last_unstale":"2024-09-17T04:24:16.225374+0000","last_undegraded":"2024-09-17T04:24:16.225374+0000","last_fullsized":"2024-09-17T04:24:16.225374+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:23.698652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224005+0000","last_change":"2024-09-17T04:24:15.075318+0000","last_active":"2024-09-17T04:24:16.224005+0000","last_peered":"2024-09-17T04:24:16.224005+0000","last_clean":"2024-09-17T04:24:16.224005+0000","last_became_active":"2024-09-17T04:24:15.074876+0000","last_became_peered":"2024-09-17T04:24:15.074876+0000","last_unstale":"2024-09-17T04:24:16.224005+0000","last_undegraded":"2024-09-17T04:24:16.224005+0000","last_fullsized":"2024-09-17T04:24:16.224005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:47.639252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224710+0000","last_change":"2024-09-17T04:24:14.056222+0000","last_active":"2024-09-17T04:24:16.224710+0000","last_peered":"2024-09-17T04:24:16.224710+0000","last_clean":"2024-09-17T04:24:16.224710+0000","last_became_active":"2024-09-17T04:24:14.055874+0000","last_became_peered":"2024-09-17T04:24:14.055874+0000","last_unstale":"2024-09-17T04:24:16.224710+0000","last_undegraded":"2024-09-17T04:24:16.224710+0000","last_fullsized":"2024-09-17T04:24:16.224710+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:44.734015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224568+0000","last_change":"2024-09-17T04:24:14.054189+0000","last_active":"2024-09-17T04:24:16.224568+0000","last_peered":"2024-09-17T04:24:16.224568+0000","last_clean":"2024-09-17T04:24:16.224568+0000","last_became_active":"2024-09-17T04:24:14.053870+0000","last_became_peered":"2024-09-17T04:24:14.053870+0000","last_unstale":"2024-09-17T04:24:16.224568+0000","last_undegraded":"2024-09-17T04:24:16.224568+0000","last_fullsized":"2024-09-17T04:24:16.224568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:12.776180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224542+0000","last_change":"2024-09-17T04:24:15.059936+0000","last_active":"2024-09-17T04:24:16.224542+0000","last_peered":"2024-09-17T04:24:16.224542+0000","last_clean":"2024-09-17T04:24:16.224542+0000","last_became_active":"2024-09-17T04:24:15.059645+0000","last_became_peered":"2024-09-17T04:24:15.059645+0000","last_unstale":"2024-09-17T04:24:16.224542+0000","last_undegraded":"2024-09-17T04:24:16.224542+0000","last_fullsized":"2024-09-17T04:24:16.224542+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:08:11.703658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055748+0000","last_change":"2024-09-17T04:24:14.060082+0000","last_active":"2024-09-17T04:24:16.055748+0000","last_peered":"2024-09-17T04:24:16.055748+0000","last_clean":"2024-09-17T04:24:16.055748+0000","last_became_active":"2024-09-17T04:24:14.058795+0000","last_became_peered":"2024-09-17T04:24:14.058795+0000","last_unstale":"2024-09-17T04:24:16.055748+0000","last_undegraded":"2024-09-17T04:24:16.055748+0000","last_fullsized":"2024-09-17T04:24:16.055748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:21.535057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.222826+0000","last_change":"2024-09-17T04:24:15.061788+0000","last_active":"2024-09-17T04:24:16.222826+0000","last_peered":"2024-09-17T04:24:16.222826+0000","last_clean":"2024-09-17T04:24:16.222826+0000","last_became_active":"2024-09-17T04:24:15.055072+0000","last_became_peered":"2024-09-17T04:24:15.055072+0000","last_unstale":"2024-09-17T04:24:16.222826+0000","last_undegraded":"2024-09-17T04:24:16.222826+0000","last_fullsized":"2024-09-17T04:24:16.222826+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:13.752351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225245+0000","last_change":"2024-09-17T04:24:14.053595+0000","last_active":"2024-09-17T04:24:16.225245+0000","last_peered":"2024-09-17T04:24:16.225245+0000","last_clean":"2024-09-17T04:24:16.225245+0000","last_became_active":"2024-09-17T04:24:14.053260+0000","last_became_peered":"2024-09-17T04:24:14.053260+0000","last_unstale":"2024-09-17T04:24:16.225245+0000","last_undegraded":"2024-09-17T04:24:16.225245+0000","last_fullsized":"2024-09-17T04:24:16.225245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:51.429181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056660+0000","last_change":"2024-09-17T04:24:15.053812+0000","last_active":"2024-09-17T04:24:16.056660+0000","last_peered":"2024-09-17T04:24:16.056660+0000","last_clean":"2024-09-17T04:24:16.056660+0000","last_became_active":"2024-09-17T04:24:15.053501+0000","last_became_peered":"2024-09-17T04:24:15.053501+0000","last_unstale":"2024-09-17T04:24:16.056660+0000","last_undegraded":"2024-09-17T04:24:16.056660+0000","last_fullsized":"2024-09-17T04:24:16.056660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:47.959956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056115+0000","last_change":"2024-09-17T04:24:14.059203+0000","last_active":"2024-09-17T04:24:16.056115+0000","last_peered":"2024-09-17T04:24:16.056115+0000","last_clean":"2024-09-17T04:24:16.056115+0000","last_became_active":"2024-09-17T04:24:14.058623+0000","last_became_peered":"2024-09-17T04:24:14.058623+0000","last_unstale":"2024-09-17T04:24:16.056115+0000","last_undegraded":"2024-09-17T04:24:16.056115+0000","last_fullsized":"2024-09-17T04:24:16.056115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:20.372651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223819+0000","last_change":"2024-09-17T04:24:15.068669+0000","last_active":"2024-09-17T04:24:16.223819+0000","last_peered":"2024-09-17T04:24:16.223819+0000","last_clean":"2024-09-17T04:24:16.223819+0000","last_became_active":"2024-09-17T04:24:15.066578+0000","last_became_peered":"2024-09-17T04:24:15.066578+0000","last_unstale":"2024-09-17T04:24:16.223819+0000","last_undegraded":"2024-09-17T04:24:16.223819+0000","last_fullsized":"2024-09-17T04:24:16.223819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:01.706191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223044+0000","last_change":"2024-09-17T04:24:14.061969+0000","last_active":"2024-09-17T04:24:16.223044+0000","last_peered":"2024-09-17T04:24:16.223044+0000","last_clean":"2024-09-17T04:24:16.223044+0000","last_became_active":"2024-09-17T04:24:14.061593+0000","last_became_peered":"2024-09-17T04:24:14.061593+0000","last_unstale":"2024-09-17T04:24:16.223044+0000","last_undegraded":"2024-09-17T04:24:16.223044+0000","last_fullsized":"2024-09-17T04:24:16.223044+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:48.406614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054926+0000","last_change":"2024-09-17T04:24:15.071779+0000","last_active":"2024-09-17T04:24:16.054926+0000","last_peered":"2024-09-17T04:24:16.054926+0000","last_clean":"2024-09-17T04:24:16.054926+0000","last_became_active":"2024-09-17T04:24:15.071431+0000","last_became_peered":"2024-09-17T04:24:15.071431+0000","last_unstale":"2024-09-17T04:24:16.054926+0000","last_undegraded":"2024-09-17T04:24:16.054926+0000","last_fullsized":"2024-09-17T04:24:16.054926+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:10.704647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224939+0000","last_change":"2024-09-17T04:24:14.058729+0000","last_active":"2024-09-17T04:24:16.224939+0000","last_peered":"2024-09-17T04:24:16.224939+0000","last_clean":"2024-09-17T04:24:16.224939+0000","last_became_active":"2024-09-17T04:24:14.057616+0000","last_became_peered":"2024-09-17T04:24:14.057616+0000","last_unstale":"2024-09-17T04:24:16.224939+0000","last_undegraded":"2024-09-17T04:24:16.224939+0000","last_fullsized":"2024-09-17T04:24:16.224939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:42.267414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223701+0000","last_change":"2024-09-17T04:24:15.056613+0000","last_active":"2024-09-17T04:24:16.223701+0000","last_peered":"2024-09-17T04:24:16.223701+0000","last_clean":"2024-09-17T04:24:16.223701+0000","last_became_active":"2024-09-17T04:24:15.056300+0000","last_became_peered":"2024-09-17T04:24:15.056300+0000","last_unstale":"2024-09-17T04:24:16.223701+0000","last_undegraded":"2024-09-17T04:24:16.223701+0000","last_fullsized":"2024-09-17T04:24:16.223701+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:34.142688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223885+0000","last_change":"2024-09-17T04:24:14.054359+0000","last_active":"2024-09-17T04:24:16.223885+0000","last_peered":"2024-09-17T04:24:16.223885+0000","last_clean":"2024-09-17T04:24:16.223885+0000","last_became_active":"2024-09-17T04:24:14.053935+0000","last_became_peered":"2024-09-17T04:24:14.053935+0000","last_unstale":"2024-09-17T04:24:16.223885+0000","last_undegraded":"2024-09-17T04:24:16.223885+0000","last_fullsized":"2024-09-17T04:24:16.223885+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:01.211818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223752+0000","last_change":"2024-09-17T04:24:15.068236+0000","last_active":"2024-09-17T04:24:16.223752+0000","last_peered":"2024-09-17T04:24:16.223752+0000","last_clean":"2024-09-17T04:24:16.223752+0000","last_became_active":"2024-09-17T04:24:15.067460+0000","last_became_peered":"2024-09-17T04:24:15.067460+0000","last_unstale":"2024-09-17T04:24:16.223752+0000","last_undegraded":"2024-09-17T04:24:16.223752+0000","last_fullsized":"2024-09-17T04:24:16.223752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:44:58.329654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056099+0000","last_change":"2024-09-17T04:24:14.058127+0000","last_active":"2024-09-17T04:24:16.056099+0000","last_peered":"2024-09-17T04:24:16.056099+0000","last_clean":"2024-09-17T04:24:16.056099+0000","last_became_active":"2024-09-17T04:24:14.055990+0000","last_became_peered":"2024-09-17T04:24:14.055990+0000","last_unstale":"2024-09-17T04:24:16.056099+0000","last_undegraded":"2024-09-17T04:24:16.056099+0000","last_fullsized":"2024-09-17T04:24:16.056099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:51.429181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223887+0000","last_change":"2024-09-17T04:24:15.076359+0000","last_active":"2024-09-17T04:24:16.223887+0000","last_peered":"2024-09-17T04:24:16.223887+0000","last_clean":"2024-09-17T04:24:16.223887+0000","last_became_active":"2024-09-17T04:24:15.075235+0000","last_became_peered":"2024-09-17T04:24:15.075235+0000","last_unstale":"2024-09-17T04:24:16.223887+0000","last_undegraded":"2024-09-17T04:24:16.223887+0000","last_fullsized":"2024-09-17T04:24:16.223887+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:44.355705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224496+0000","last_change":"2024-09-17T04:24:14.051949+0000","last_active":"2024-09-17T04:24:16.224496+0000","last_peered":"2024-09-17T04:24:16.224496+0000","last_clean":"2024-09-17T04:24:16.224496+0000","last_became_active":"2024-09-17T04:24:14.051548+0000","last_became_peered":"2024-09-17T04:24:14.051548+0000","last_unstale":"2024-09-17T04:24:16.224496+0000","last_undegraded":"2024-09-17T04:24:16.224496+0000","last_fullsized":"2024-09-17T04:24:16.224496+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:21.535057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223899+0000","last_change":"2024-09-17T04:24:15.060676+0000","last_active":"2024-09-17T04:24:16.223899+0000","last_peered":"2024-09-17T04:24:16.223899+0000","last_clean":"2024-09-17T04:24:16.223899+0000","last_became_active":"2024-09-17T04:24:15.060268+0000","last_became_peered":"2024-09-17T04:24:15.060268+0000","last_unstale":"2024-09-17T04:24:16.223899+0000","last_undegraded":"2024-09-17T04:24:16.223899+0000","last_fullsized":"2024-09-17T04:24:16.223899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:13.752351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.051789+0000","last_change":"2024-09-17T04:24:14.043539+0000","last_active":"2024-09-17T04:24:16.051789+0000","last_peered":"2024-09-17T04:24:16.051789+0000","last_clean":"2024-09-17T04:24:16.051789+0000","last_became_active":"2024-09-17T04:24:14.043214+0000","last_became_peered":"2024-09-17T04:24:14.043214+0000","last_unstale":"2024-09-17T04:24:16.051789+0000","last_undegraded":"2024-09-17T04:24:16.051789+0000","last_fullsized":"2024-09-17T04:24:16.051789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:42.267414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223990+0000","last_change":"2024-09-17T04:24:15.068886+0000","last_active":"2024-09-17T04:24:16.223990+0000","last_peered":"2024-09-17T04:24:16.223990+0000","last_clean":"2024-09-17T04:24:16.223990+0000","last_became_active":"2024-09-17T04:24:15.068532+0000","last_became_peered":"2024-09-17T04:24:15.068532+0000","last_unstale":"2024-09-17T04:24:16.223990+0000","last_undegraded":"2024-09-17T04:24:16.223990+0000","last_fullsized":"2024-09-17T04:24:16.223990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:58.719218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054761+0000","last_change":"2024-09-17T04:24:14.059673+0000","last_active":"2024-09-17T04:24:16.054761+0000","last_peered":"2024-09-17T04:24:16.054761+0000","last_clean":"2024-09-17T04:24:16.054761+0000","last_became_active":"2024-09-17T04:24:14.057668+0000","last_became_peered":"2024-09-17T04:24:14.057668+0000","last_unstale":"2024-09-17T04:24:16.054761+0000","last_undegraded":"2024-09-17T04:24:16.054761+0000","last_fullsized":"2024-09-17T04:24:16.054761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:01.211818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224143+0000","last_change":"2024-09-17T04:24:15.069931+0000","last_active":"2024-09-17T04:24:16.224143+0000","last_peered":"2024-09-17T04:24:16.224143+0000","last_clean":"2024-09-17T04:24:16.224143+0000","last_became_active":"2024-09-17T04:24:15.069568+0000","last_became_peered":"2024-09-17T04:24:15.069568+0000","last_unstale":"2024-09-17T04:24:16.224143+0000","last_undegraded":"2024-09-17T04:24:16.224143+0000","last_fullsized":"2024-09-17T04:24:16.224143+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:35.584338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.052093+0000","last_change":"2024-09-17T04:24:14.054912+0000","last_active":"2024-09-17T04:24:16.052093+0000","last_peered":"2024-09-17T04:24:16.052093+0000","last_clean":"2024-09-17T04:24:16.052093+0000","last_became_active":"2024-09-17T04:24:14.054105+0000","last_became_peered":"2024-09-17T04:24:14.054105+0000","last_unstale":"2024-09-17T04:24:16.052093+0000","last_undegraded":"2024-09-17T04:24:16.052093+0000","last_fullsized":"2024-09-17T04:24:16.052093+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:44.734015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224717+0000","last_change":"2024-09-17T04:24:15.075676+0000","last_active":"2024-09-17T04:24:16.224717+0000","last_peered":"2024-09-17T04:24:16.224717+0000","last_clean":"2024-09-17T04:24:16.224717+0000","last_became_active":"2024-09-17T04:24:15.075114+0000","last_became_peered":"2024-09-17T04:24:15.075114+0000","last_unstale":"2024-09-17T04:24:16.224717+0000","last_undegraded":"2024-09-17T04:24:16.224717+0000","last_fullsized":"2024-09-17T04:24:16.224717+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:51:30.987569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223146+0000","last_change":"2024-09-17T04:24:14.039417+0000","last_active":"2024-09-17T04:24:16.223146+0000","last_peered":"2024-09-17T04:24:16.223146+0000","last_clean":"2024-09-17T04:24:16.223146+0000","last_became_active":"2024-09-17T04:24:14.039025+0000","last_became_peered":"2024-09-17T04:24:14.039025+0000","last_unstale":"2024-09-17T04:24:16.223146+0000","last_undegraded":"2024-09-17T04:24:16.223146+0000","last_fullsized":"2024-09-17T04:24:16.223146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:03.320698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056433+0000","last_change":"2024-09-17T04:24:15.061023+0000","last_active":"2024-09-17T04:24:16.056433+0000","last_peered":"2024-09-17T04:24:16.056433+0000","last_clean":"2024-09-17T04:24:16.056433+0000","last_became_active":"2024-09-17T04:24:15.060544+0000","last_became_peered":"2024-09-17T04:24:15.060544+0000","last_unstale":"2024-09-17T04:24:16.056433+0000","last_undegraded":"2024-09-17T04:24:16.056433+0000","last_fullsized":"2024-09-17T04:24:16.056433+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:35:10.792366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.051950+0000","last_change":"2024-09-17T04:24:14.052529+0000","last_active":"2024-09-17T04:24:16.051950+0000","last_peered":"2024-09-17T04:24:16.051950+0000","last_clean":"2024-09-17T04:24:16.051950+0000","last_became_active":"2024-09-17T04:24:14.052135+0000","last_became_peered":"2024-09-17T04:24:14.052135+0000","last_unstale":"2024-09-17T04:24:16.051950+0000","last_undegraded":"2024-09-17T04:24:16.051950+0000","last_fullsized":"2024-09-17T04:24:16.051950+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:28.196011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224039+0000","last_change":"2024-09-17T04:24:15.064435+0000","last_active":"2024-09-17T04:24:16.224039+0000","last_peered":"2024-09-17T04:24:16.224039+0000","last_clean":"2024-09-17T04:24:16.224039+0000","last_became_active":"2024-09-17T04:24:15.064195+0000","last_became_peered":"2024-09-17T04:24:15.064195+0000","last_unstale":"2024-09-17T04:24:16.224039+0000","last_undegraded":"2024-09-17T04:24:16.224039+0000","last_fullsized":"2024-09-17T04:24:16.224039+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:47.959956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223547+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:24:16.223547+0000","last_peered":"2024-09-17T04:24:16.223547+0000","last_clean":"2024-09-17T04:24:16.223547+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:24:16.223547+0000","last_undegraded":"2024-09-17T04:24:16.223547+0000","last_fullsized":"2024-09-17T04:24:16.223547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055535+0000","last_change":"2024-09-17T04:24:14.049088+0000","last_active":"2024-09-17T04:24:16.055535+0000","last_peered":"2024-09-17T04:24:16.055535+0000","last_clean":"2024-09-17T04:24:16.055535+0000","last_became_active":"2024-09-17T04:24:14.048529+0000","last_became_peered":"2024-09-17T04:24:14.048529+0000","last_unstale":"2024-09-17T04:24:16.055535+0000","last_undegraded":"2024-09-17T04:24:16.055535+0000","last_fullsized":"2024-09-17T04:24:16.055535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:42.267414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225083+0000","last_change":"2024-09-17T04:24:15.065699+0000","last_active":"2024-09-17T04:24:16.225083+0000","last_peered":"2024-09-17T04:24:16.225083+0000","last_clean":"2024-09-17T04:24:16.225083+0000","last_became_active":"2024-09-17T04:24:15.061569+0000","last_became_peered":"2024-09-17T04:24:15.061569+0000","last_unstale":"2024-09-17T04:24:16.225083+0000","last_undegraded":"2024-09-17T04:24:16.225083+0000","last_fullsized":"2024-09-17T04:24:16.225083+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:01:19.496265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223384+0000","last_change":"2024-09-17T04:24:14.061081+0000","last_active":"2024-09-17T04:24:16.223384+0000","last_peered":"2024-09-17T04:24:16.223384+0000","last_clean":"2024-09-17T04:24:16.223384+0000","last_became_active":"2024-09-17T04:24:14.060394+0000","last_became_peered":"2024-09-17T04:24:14.060394+0000","last_unstale":"2024-09-17T04:24:16.223384+0000","last_undegraded":"2024-09-17T04:24:16.223384+0000","last_fullsized":"2024-09-17T04:24:16.223384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:08.023686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223901+0000","last_change":"2024-09-17T04:24:15.070960+0000","last_active":"2024-09-17T04:24:16.223901+0000","last_peered":"2024-09-17T04:24:16.223901+0000","last_clean":"2024-09-17T04:24:16.223901+0000","last_became_active":"2024-09-17T04:24:15.070528+0000","last_became_peered":"2024-09-17T04:24:15.070528+0000","last_unstale":"2024-09-17T04:24:16.223901+0000","last_undegraded":"2024-09-17T04:24:16.223901+0000","last_fullsized":"2024-09-17T04:24:16.223901+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:22.452641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.052102+0000","last_change":"2024-09-17T04:24:14.051042+0000","last_active":"2024-09-17T04:24:16.052102+0000","last_peered":"2024-09-17T04:24:16.052102+0000","last_clean":"2024-09-17T04:24:16.052102+0000","last_became_active":"2024-09-17T04:24:14.050513+0000","last_became_peered":"2024-09-17T04:24:14.050513+0000","last_unstale":"2024-09-17T04:24:16.052102+0000","last_undegraded":"2024-09-17T04:24:16.052102+0000","last_fullsized":"2024-09-17T04:24:16.052102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:21.535057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.222913+0000","last_change":"2024-09-17T04:24:15.061656+0000","last_active":"2024-09-17T04:24:16.222913+0000","last_peered":"2024-09-17T04:24:16.222913+0000","last_clean":"2024-09-17T04:24:16.222913+0000","last_became_active":"2024-09-17T04:24:15.061365+0000","last_became_peered":"2024-09-17T04:24:15.061365+0000","last_unstale":"2024-09-17T04:24:16.222913+0000","last_undegraded":"2024-09-17T04:24:16.222913+0000","last_fullsized":"2024-09-17T04:24:16.222913+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:22.452641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056245+0000","last_change":"2024-09-17T04:24:14.053751+0000","last_active":"2024-09-17T04:24:16.056245+0000","last_peered":"2024-09-17T04:24:16.056245+0000","last_clean":"2024-09-17T04:24:16.056245+0000","last_became_active":"2024-09-17T04:24:14.051943+0000","last_became_peered":"2024-09-17T04:24:14.051943+0000","last_unstale":"2024-09-17T04:24:16.056245+0000","last_undegraded":"2024-09-17T04:24:16.056245+0000","last_fullsized":"2024-09-17T04:24:16.056245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:08.023686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054263+0000","last_change":"2024-09-17T04:24:15.074736+0000","last_active":"2024-09-17T04:24:16.054263+0000","last_peered":"2024-09-17T04:24:16.054263+0000","last_clean":"2024-09-17T04:24:16.054263+0000","last_became_active":"2024-09-17T04:24:15.074374+0000","last_became_peered":"2024-09-17T04:24:15.074374+0000","last_unstale":"2024-09-17T04:24:16.054263+0000","last_undegraded":"2024-09-17T04:24:16.054263+0000","last_fullsized":"2024-09-17T04:24:16.054263+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:49.409374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.222921+0000","last_change":"2024-09-17T04:24:14.057105+0000","last_active":"2024-09-17T04:24:16.222921+0000","last_peered":"2024-09-17T04:24:16.222921+0000","last_clean":"2024-09-17T04:24:16.222921+0000","last_became_active":"2024-09-17T04:24:14.056908+0000","last_became_peered":"2024-09-17T04:24:14.056908+0000","last_unstale":"2024-09-17T04:24:16.222921+0000","last_undegraded":"2024-09-17T04:24:16.222921+0000","last_fullsized":"2024-09-17T04:24:16.222921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:12.782267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056301+0000","last_change":"2024-09-17T04:24:15.058663+0000","last_active":"2024-09-17T04:24:16.056301+0000","last_peered":"2024-09-17T04:24:16.056301+0000","last_clean":"2024-09-17T04:24:16.056301+0000","last_became_active":"2024-09-17T04:24:15.057807+0000","last_became_peered":"2024-09-17T04:24:15.057807+0000","last_unstale":"2024-09-17T04:24:16.056301+0000","last_undegraded":"2024-09-17T04:24:16.056301+0000","last_fullsized":"2024-09-17T04:24:16.056301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:35.584338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224488+0000","last_change":"2024-09-17T04:24:14.063354+0000","last_active":"2024-09-17T04:24:16.224488+0000","last_peered":"2024-09-17T04:24:16.224488+0000","last_clean":"2024-09-17T04:24:16.224488+0000","last_became_active":"2024-09-17T04:24:14.062231+0000","last_became_peered":"2024-09-17T04:24:14.062231+0000","last_unstale":"2024-09-17T04:24:16.224488+0000","last_undegraded":"2024-09-17T04:24:16.224488+0000","last_fullsized":"2024-09-17T04:24:16.224488+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:40.777526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.222850+0000","last_change":"2024-09-17T04:24:15.058346+0000","last_active":"2024-09-17T04:24:16.222850+0000","last_peered":"2024-09-17T04:24:16.222850+0000","last_clean":"2024-09-17T04:24:16.222850+0000","last_became_active":"2024-09-17T04:24:15.057977+0000","last_became_peered":"2024-09-17T04:24:15.057977+0000","last_unstale":"2024-09-17T04:24:16.222850+0000","last_undegraded":"2024-09-17T04:24:16.222850+0000","last_fullsized":"2024-09-17T04:24:16.222850+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:01.706191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223545+0000","last_change":"2024-09-17T04:24:14.049298+0000","last_active":"2024-09-17T04:24:16.223545+0000","last_peered":"2024-09-17T04:24:16.223545+0000","last_clean":"2024-09-17T04:24:16.223545+0000","last_became_active":"2024-09-17T04:24:14.049070+0000","last_became_peered":"2024-09-17T04:24:14.049070+0000","last_unstale":"2024-09-17T04:24:16.223545+0000","last_undegraded":"2024-09-17T04:24:16.223545+0000","last_fullsized":"2024-09-17T04:24:16.223545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:52:59.361386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225582+0000","last_change":"2024-09-17T04:24:15.050480+0000","last_active":"2024-09-17T04:24:16.225582+0000","last_peered":"2024-09-17T04:24:16.225582+0000","last_clean":"2024-09-17T04:24:16.225582+0000","last_became_active":"2024-09-17T04:24:15.050166+0000","last_became_peered":"2024-09-17T04:24:15.050166+0000","last_unstale":"2024-09-17T04:24:16.225582+0000","last_undegraded":"2024-09-17T04:24:16.225582+0000","last_fullsized":"2024-09-17T04:24:16.225582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:02.125884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225264+0000","last_change":"2024-09-17T04:24:14.063222+0000","last_active":"2024-09-17T04:24:16.225264+0000","last_peered":"2024-09-17T04:24:16.225264+0000","last_clean":"2024-09-17T04:24:16.225264+0000","last_became_active":"2024-09-17T04:24:14.062750+0000","last_became_peered":"2024-09-17T04:24:14.062750+0000","last_unstale":"2024-09-17T04:24:16.225264+0000","last_undegraded":"2024-09-17T04:24:16.225264+0000","last_fullsized":"2024-09-17T04:24:16.225264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:08.023686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224491+0000","last_change":"2024-09-17T04:24:15.067950+0000","last_active":"2024-09-17T04:24:16.224491+0000","last_peered":"2024-09-17T04:24:16.224491+0000","last_clean":"2024-09-17T04:24:16.224491+0000","last_became_active":"2024-09-17T04:24:15.066567+0000","last_became_peered":"2024-09-17T04:24:15.066567+0000","last_unstale":"2024-09-17T04:24:16.224491+0000","last_undegraded":"2024-09-17T04:24:16.224491+0000","last_fullsized":"2024-09-17T04:24:16.224491+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:28.104736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223381+0000","last_change":"2024-09-17T04:24:14.059487+0000","last_active":"2024-09-17T04:24:16.223381+0000","last_peered":"2024-09-17T04:24:16.223381+0000","last_clean":"2024-09-17T04:24:16.223381+0000","last_became_active":"2024-09-17T04:24:14.059186+0000","last_became_peered":"2024-09-17T04:24:14.059186+0000","last_unstale":"2024-09-17T04:24:16.223381+0000","last_undegraded":"2024-09-17T04:24:16.223381+0000","last_fullsized":"2024-09-17T04:24:16.223381+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:03.320698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054147+0000","last_change":"2024-09-17T04:24:15.065941+0000","last_active":"2024-09-17T04:24:16.054147+0000","last_peered":"2024-09-17T04:24:16.054147+0000","last_clean":"2024-09-17T04:24:16.054147+0000","last_became_active":"2024-09-17T04:24:15.065582+0000","last_became_peered":"2024-09-17T04:24:15.065582+0000","last_unstale":"2024-09-17T04:24:16.054147+0000","last_undegraded":"2024-09-17T04:24:16.054147+0000","last_fullsized":"2024-09-17T04:24:16.054147+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:30.122754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.222894+0000","last_change":"2024-09-17T04:24:14.062678+0000","last_active":"2024-09-17T04:24:16.222894+0000","last_peered":"2024-09-17T04:24:16.222894+0000","last_clean":"2024-09-17T04:24:16.222894+0000","last_became_active":"2024-09-17T04:24:14.062281+0000","last_became_peered":"2024-09-17T04:24:14.062281+0000","last_unstale":"2024-09-17T04:24:16.222894+0000","last_undegraded":"2024-09-17T04:24:16.222894+0000","last_fullsized":"2024-09-17T04:24:16.222894+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:40.254206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224778+0000","last_change":"2024-09-17T04:24:15.064498+0000","last_active":"2024-09-17T04:24:16.224778+0000","last_peered":"2024-09-17T04:24:16.224778+0000","last_clean":"2024-09-17T04:24:16.224778+0000","last_became_active":"2024-09-17T04:24:15.062700+0000","last_became_peered":"2024-09-17T04:24:15.062700+0000","last_unstale":"2024-09-17T04:24:16.224778+0000","last_undegraded":"2024-09-17T04:24:16.224778+0000","last_fullsized":"2024-09-17T04:24:16.224778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:31.021313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223265+0000","last_change":"2024-09-17T04:24:14.047923+0000","last_active":"2024-09-17T04:24:16.223265+0000","last_peered":"2024-09-17T04:24:16.223265+0000","last_clean":"2024-09-17T04:24:16.223265+0000","last_became_active":"2024-09-17T04:24:14.047596+0000","last_became_peered":"2024-09-17T04:24:14.047596+0000","last_unstale":"2024-09-17T04:24:16.223265+0000","last_undegraded":"2024-09-17T04:24:16.223265+0000","last_fullsized":"2024-09-17T04:24:16.223265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:29.119993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225575+0000","last_change":"2024-09-17T04:24:15.060939+0000","last_active":"2024-09-17T04:24:16.225575+0000","last_peered":"2024-09-17T04:24:16.225575+0000","last_clean":"2024-09-17T04:24:16.225575+0000","last_became_active":"2024-09-17T04:24:15.060600+0000","last_became_peered":"2024-09-17T04:24:15.060600+0000","last_unstale":"2024-09-17T04:24:16.225575+0000","last_undegraded":"2024-09-17T04:24:16.225575+0000","last_fullsized":"2024-09-17T04:24:16.225575+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:05.895665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224176+0000","last_change":"2024-09-17T04:24:14.062616+0000","last_active":"2024-09-17T04:24:16.224176+0000","last_peered":"2024-09-17T04:24:16.224176+0000","last_clean":"2024-09-17T04:24:16.224176+0000","last_became_active":"2024-09-17T04:24:14.062182+0000","last_became_peered":"2024-09-17T04:24:14.062182+0000","last_unstale":"2024-09-17T04:24:16.224176+0000","last_undegraded":"2024-09-17T04:24:16.224176+0000","last_fullsized":"2024-09-17T04:24:16.224176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:51.246396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224637+0000","last_change":"2024-09-17T04:24:15.079589+0000","last_active":"2024-09-17T04:24:16.224637+0000","last_peered":"2024-09-17T04:24:16.224637+0000","last_clean":"2024-09-17T04:24:16.224637+0000","last_became_active":"2024-09-17T04:24:15.079019+0000","last_became_peered":"2024-09-17T04:24:15.079019+0000","last_unstale":"2024-09-17T04:24:16.224637+0000","last_undegraded":"2024-09-17T04:24:16.224637+0000","last_fullsized":"2024-09-17T04:24:16.224637+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:01.706191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056070+0000","last_change":"2024-09-17T04:24:15.080533+0000","last_active":"2024-09-17T04:24:16.056070+0000","last_peered":"2024-09-17T04:24:16.056070+0000","last_clean":"2024-09-17T04:24:16.056070+0000","last_became_active":"2024-09-17T04:24:15.080179+0000","last_became_peered":"2024-09-17T04:24:15.080179+0000","last_unstale":"2024-09-17T04:24:16.056070+0000","last_undegraded":"2024-09-17T04:24:16.056070+0000","last_fullsized":"2024-09-17T04:24:16.056070+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:58.719218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224154+0000","last_change":"2024-09-17T04:24:14.065850+0000","last_active":"2024-09-17T04:24:16.224154+0000","last_peered":"2024-09-17T04:24:16.224154+0000","last_clean":"2024-09-17T04:24:16.224154+0000","last_became_active":"2024-09-17T04:24:14.065444+0000","last_became_peered":"2024-09-17T04:24:14.065444+0000","last_unstale":"2024-09-17T04:24:16.224154+0000","last_undegraded":"2024-09-17T04:24:16.224154+0000","last_fullsized":"2024-09-17T04:24:16.224154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:51.121630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056174+0000","last_change":"2024-09-17T04:24:15.076511+0000","last_active":"2024-09-17T04:24:16.056174+0000","last_peered":"2024-09-17T04:24:16.056174+0000","last_clean":"2024-09-17T04:24:16.056174+0000","last_became_active":"2024-09-17T04:24:15.076037+0000","last_became_peered":"2024-09-17T04:24:15.076037+0000","last_unstale":"2024-09-17T04:24:16.056174+0000","last_undegraded":"2024-09-17T04:24:16.056174+0000","last_fullsized":"2024-09-17T04:24:16.056174+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:06:32.518189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224180+0000","last_change":"2024-09-17T04:24:14.060131+0000","last_active":"2024-09-17T04:24:16.224180+0000","last_peered":"2024-09-17T04:24:16.224180+0000","last_clean":"2024-09-17T04:24:16.224180+0000","last_became_active":"2024-09-17T04:24:14.059707+0000","last_became_peered":"2024-09-17T04:24:14.059707+0000","last_unstale":"2024-09-17T04:24:16.224180+0000","last_undegraded":"2024-09-17T04:24:16.224180+0000","last_fullsized":"2024-09-17T04:24:16.224180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:22.869429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223586+0000","last_change":"2024-09-17T04:24:15.068641+0000","last_active":"2024-09-17T04:24:16.223586+0000","last_peered":"2024-09-17T04:24:16.223586+0000","last_clean":"2024-09-17T04:24:16.223586+0000","last_became_active":"2024-09-17T04:24:15.065609+0000","last_became_peered":"2024-09-17T04:24:15.065609+0000","last_unstale":"2024-09-17T04:24:16.223586+0000","last_undegraded":"2024-09-17T04:24:16.223586+0000","last_fullsized":"2024-09-17T04:24:16.223586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:43:02.004482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225369+0000","last_change":"2024-09-17T04:24:14.056735+0000","last_active":"2024-09-17T04:24:16.225369+0000","last_peered":"2024-09-17T04:24:16.225369+0000","last_clean":"2024-09-17T04:24:16.225369+0000","last_became_active":"2024-09-17T04:24:14.056433+0000","last_became_peered":"2024-09-17T04:24:14.056433+0000","last_unstale":"2024-09-17T04:24:16.225369+0000","last_undegraded":"2024-09-17T04:24:16.225369+0000","last_fullsized":"2024-09-17T04:24:16.225369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:03.618384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225403+0000","last_change":"2024-09-17T04:24:15.054773+0000","last_active":"2024-09-17T04:24:16.225403+0000","last_peered":"2024-09-17T04:24:16.225403+0000","last_clean":"2024-09-17T04:24:16.225403+0000","last_became_active":"2024-09-17T04:24:15.054453+0000","last_became_peered":"2024-09-17T04:24:15.054453+0000","last_unstale":"2024-09-17T04:24:16.225403+0000","last_undegraded":"2024-09-17T04:24:16.225403+0000","last_fullsized":"2024-09-17T04:24:16.225403+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:58.942093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223981+0000","last_change":"2024-09-17T04:24:14.062995+0000","last_active":"2024-09-17T04:24:16.223981+0000","last_peered":"2024-09-17T04:24:16.223981+0000","last_clean":"2024-09-17T04:24:16.223981+0000","last_became_active":"2024-09-17T04:24:14.062615+0000","last_became_peered":"2024-09-17T04:24:14.062615+0000","last_unstale":"2024-09-17T04:24:16.223981+0000","last_undegraded":"2024-09-17T04:24:16.223981+0000","last_fullsized":"2024-09-17T04:24:16.223981+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:20:07.263105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476743,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27844,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343996,"statfs":{"total":96636764160,"available":96608251904,"internally_reserved":0,"allocated":827392,"data_stored":670785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476742,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":81447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476743,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476743,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":79961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":79835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":237568,"data_stored":79995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":835584,"data_stored":670841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":81447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:24:24.426 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:24:24.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.655+0000 7f644ae00640 1 -- 172.21.15.100:0/559420356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6444070710 msgr2=0x7f6444073270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:24.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.655+0000 7f644ae00640 1 --2- 172.21.15.100:0/559420356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6444070710 0x7f6444073270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f643c005fd0 tx=0x7f643c02a5d0 comp rx=0 tx=0).stop 2024-09-17T04:24:24.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.655+0000 7f644ae00640 1 -- 172.21.15.100:0/559420356 shutdown_connections 2024-09-17T04:24:24.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.655+0000 7f644ae00640 1 --2- 172.21.15.100:0/559420356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6444070710 0x7f6444073270 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:24.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.655+0000 7f644ae00640 1 --2- 172.21.15.100:0/559420356 >> v2:172.21.15.184:3300/0 conn(0x7f64440af050 0x7f64440ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:24.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.655+0000 7f644ae00640 1 -- 172.21.15.100:0/559420356 >> 172.21.15.100:0/559420356 conn(0x7f64440b1e40 msgr2=0x7f64440b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:24.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.655+0000 7f644ae00640 1 -- 172.21.15.100:0/559420356 shutdown_connections 2024-09-17T04:24:24.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.655+0000 7f644ae00640 1 -- 172.21.15.100:0/559420356 wait complete. 2024-09-17T04:24:24.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.657+0000 7f644ae00640 1 Processor -- start 2024-09-17T04:24:24.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.658+0000 7f644ae00640 1 -- start start 2024-09-17T04:24:24.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.659+0000 7f644ae00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6444070710 0x7f644409aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:24.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.659+0000 7f644ae00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6444072e60 0x7f64441f73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:24.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.659+0000 7f644ae00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64440af050 0x7f6444151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:24.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.659+0000 7f644ae00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6444165870 con 0x7f64440af050 2024-09-17T04:24:24.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.659+0000 7f644ae00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f64441656f0 con 0x7f6444070710 2024-09-17T04:24:24.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.659+0000 7f644ae00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f64441659f0 con 0x7f6444072e60 2024-09-17T04:24:24.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.659+0000 7f6449a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6444070710 0x7f644409aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:24.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.659+0000 7f644a400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64440af050 0x7f6444151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:24.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.659+0000 7f6449a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6444070710 0x7f644409aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.100:46896/0 (socket says 172.21.15.100:46896) 2024-09-17T04:24:24.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.659+0000 7f6449a00640 1 -- 172.21.15.100:0/2251857668 learned_addr learned my addr 172.21.15.100:0/2251857668 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:24.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.659+0000 7f6449000640 1 --2- 172.21.15.100:0/2251857668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6444072e60 0x7f64441f73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:24.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.660+0000 7f6449a00640 1 -- 172.21.15.100:0/2251857668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6444072e60 msgr2=0x7f64441f73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:24.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.660+0000 7f6449a00640 1 --2- 172.21.15.100:0/2251857668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6444072e60 0x7f64441f73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:24.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.660+0000 7f6449a00640 1 -- 172.21.15.100:0/2251857668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64440af050 msgr2=0x7f6444151800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:24.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.660+0000 7f6449a00640 1 --2- 172.21.15.100:0/2251857668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64440af050 0x7f6444151800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:24.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.660+0000 7f6449a00640 1 -- 172.21.15.100:0/2251857668 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644409c880 con 0x7f6444070710 2024-09-17T04:24:24.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.660+0000 7f6449a00640 1 --2- 172.21.15.100:0/2251857668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6444070710 0x7f644409aa50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f643800ece0 tx=0x7f643800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:24.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.660+0000 7f6441600640 1 -- 172.21.15.100:0/2251857668 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438003b70 con 0x7f6444070710 2024-09-17T04:24:24.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.660+0000 7f6441600640 1 -- 172.21.15.100:0/2251857668 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6438003d10 con 0x7f6444070710 2024-09-17T04:24:24.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.660+0000 7f644ae00640 1 -- 172.21.15.100:0/2251857668 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644409af90 con 0x7f6444070710 2024-09-17T04:24:24.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.661+0000 7f6441600640 1 -- 172.21.15.100:0/2251857668 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438004280 con 0x7f6444070710 2024-09-17T04:24:24.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.661+0000 7f644ae00640 1 -- 172.21.15.100:0/2251857668 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6444131be0 con 0x7f6444070710 2024-09-17T04:24:24.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.662+0000 7f644ae00640 1 -- 172.21.15.100:0/2251857668 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f644409ca50 con 0x7f6444070710 2024-09-17T04:24:24.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.663+0000 7f6441600640 1 -- 172.21.15.100:0/2251857668 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6438004420 con 0x7f6444070710 2024-09-17T04:24:24.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.664+0000 7f6441600640 1 --2- 172.21.15.100:0/2251857668 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6424078bc0 0x7f642407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:24.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.664+0000 7f6441600640 1 -- 172.21.15.100:0/2251857668 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f643809be10 con 0x7f6444070710 2024-09-17T04:24:24.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.664+0000 7f6449000640 1 --2- 172.21.15.100:0/2251857668 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6424078bc0 0x7f642407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:24.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.665+0000 7f6449000640 1 --2- 172.21.15.100:0/2251857668 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6424078bc0 0x7f642407b0d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6444151120 tx=0x7f6430009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:24.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.669+0000 7f6441600640 1 -- 172.21.15.100:0/2251857668 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6438063be0 con 0x7f6444070710 2024-09-17T04:24:24.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.852+0000 7f644ae00640 1 -- 172.21.15.100:0/2251857668 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6444073270 con 0x7f6424078bc0 2024-09-17T04:24:24.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.857+0000 7f6441600640 1 -- 172.21.15.100:0/2251857668 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251889 (secure 0 0 0) 0x7f6444073270 con 0x7f6424078bc0 2024-09-17T04:24:24.861 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:24.862 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:24:24.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.861+0000 7f644ae00640 1 -- 172.21.15.100:0/2251857668 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6424078bc0 msgr2=0x7f642407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:24.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.861+0000 7f644ae00640 1 --2- 172.21.15.100:0/2251857668 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6424078bc0 0x7f642407b0d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6444151120 tx=0x7f6430009040 comp rx=0 tx=0).stop 2024-09-17T04:24:24.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.861+0000 7f644ae00640 1 -- 172.21.15.100:0/2251857668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6444070710 msgr2=0x7f644409aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:24.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.861+0000 7f644ae00640 1 --2- 172.21.15.100:0/2251857668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6444070710 0x7f644409aa50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f643800ece0 tx=0x7f643800c6a0 comp rx=0 tx=0).stop 2024-09-17T04:24:24.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.861+0000 7f644ae00640 1 -- 172.21.15.100:0/2251857668 shutdown_connections 2024-09-17T04:24:24.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.862+0000 7f644ae00640 1 --2- 172.21.15.100:0/2251857668 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6424078bc0 0x7f642407b0d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:24.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.862+0000 7f644ae00640 1 --2- 172.21.15.100:0/2251857668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64440af050 0x7f6444151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:24.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.862+0000 7f644ae00640 1 --2- 172.21.15.100:0/2251857668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6444072e60 0x7f64441f73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:24.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.862+0000 7f644ae00640 1 --2- 172.21.15.100:0/2251857668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6444070710 0x7f644409aa50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:24.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.862+0000 7f644ae00640 1 -- 172.21.15.100:0/2251857668 >> 172.21.15.100:0/2251857668 conn(0x7f64440b1e40 msgr2=0x7f64440b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:24.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.862+0000 7f644ae00640 1 -- 172.21.15.100:0/2251857668 shutdown_connections 2024-09-17T04:24:24.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:24.862+0000 7f644ae00640 1 -- 172.21.15.100:0/2251857668 wait complete. 2024-09-17T04:24:24.886 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-09-17T04:24:23.966453+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219276,"kb_used_data":3016,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755444,"statfs":{"total":773094113280,"available":772869574656,"internally_reserved":0,"allocated":3088384,"data_stored":1824236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.935000"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224232+0000","last_change":"2024-09-17T04:24:15.051472+0000","last_active":"2024-09-17T04:24:16.224232+0000","last_peered":"2024-09-17T04:24:16.224232+0000","last_clean":"2024-09-17T04:24:16.224232+0000","last_became_active":"2024-09-17T04:24:15.051162+0000","last_became_peered":"2024-09-17T04:24:15.051162+0000","last_unstale":"2024-09-17T04:24:16.224232+0000","last_undegraded":"2024-09-17T04:24:16.224232+0000","last_fullsized":"2024-09-17T04:24:16.224232+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:35:10.792366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223431+0000","last_change":"2024-09-17T04:24:15.069649+0000","last_active":"2024-09-17T04:24:16.223431+0000","last_peered":"2024-09-17T04:24:16.223431+0000","last_clean":"2024-09-17T04:24:16.223431+0000","last_became_active":"2024-09-17T04:24:15.068975+0000","last_became_peered":"2024-09-17T04:24:15.068975+0000","last_unstale":"2024-09-17T04:24:16.223431+0000","last_undegraded":"2024-09-17T04:24:16.223431+0000","last_fullsized":"2024-09-17T04:24:16.223431+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:47.639252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223497+0000","last_change":"2024-09-17T04:24:15.052608+0000","last_active":"2024-09-17T04:24:16.223497+0000","last_peered":"2024-09-17T04:24:16.223497+0000","last_clean":"2024-09-17T04:24:16.223497+0000","last_became_active":"2024-09-17T04:24:15.052252+0000","last_became_peered":"2024-09-17T04:24:15.052252+0000","last_unstale":"2024-09-17T04:24:16.223497+0000","last_undegraded":"2024-09-17T04:24:16.223497+0000","last_fullsized":"2024-09-17T04:24:16.223497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:34:23.523300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055891+0000","last_change":"2024-09-17T04:24:15.059215+0000","last_active":"2024-09-17T04:24:16.055891+0000","last_peered":"2024-09-17T04:24:16.055891+0000","last_clean":"2024-09-17T04:24:16.055891+0000","last_became_active":"2024-09-17T04:24:15.058921+0000","last_became_peered":"2024-09-17T04:24:15.058921+0000","last_unstale":"2024-09-17T04:24:16.055891+0000","last_undegraded":"2024-09-17T04:24:16.055891+0000","last_fullsized":"2024-09-17T04:24:16.055891+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:34.142688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224122+0000","last_change":"2024-09-17T04:24:15.042545+0000","last_active":"2024-09-17T04:24:16.224122+0000","last_peered":"2024-09-17T04:24:16.224122+0000","last_clean":"2024-09-17T04:24:16.224122+0000","last_became_active":"2024-09-17T04:24:15.042235+0000","last_became_peered":"2024-09-17T04:24:15.042235+0000","last_unstale":"2024-09-17T04:24:16.224122+0000","last_undegraded":"2024-09-17T04:24:16.224122+0000","last_fullsized":"2024-09-17T04:24:16.224122+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:06:32.518189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056013+0000","last_change":"2024-09-17T04:24:15.057624+0000","last_active":"2024-09-17T04:24:16.056013+0000","last_peered":"2024-09-17T04:24:16.056013+0000","last_clean":"2024-09-17T04:24:16.056013+0000","last_became_active":"2024-09-17T04:24:15.057329+0000","last_became_peered":"2024-09-17T04:24:15.057329+0000","last_unstale":"2024-09-17T04:24:16.056013+0000","last_undegraded":"2024-09-17T04:24:16.056013+0000","last_fullsized":"2024-09-17T04:24:16.056013+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:44.355705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223880+0000","last_change":"2024-09-17T04:24:15.062645+0000","last_active":"2024-09-17T04:24:16.223880+0000","last_peered":"2024-09-17T04:24:16.223880+0000","last_clean":"2024-09-17T04:24:16.223880+0000","last_became_active":"2024-09-17T04:24:15.062227+0000","last_became_peered":"2024-09-17T04:24:15.062227+0000","last_unstale":"2024-09-17T04:24:16.223880+0000","last_undegraded":"2024-09-17T04:24:16.223880+0000","last_fullsized":"2024-09-17T04:24:16.223880+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:13.159492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224308+0000","last_change":"2024-09-17T04:24:15.062993+0000","last_active":"2024-09-17T04:24:16.224308+0000","last_peered":"2024-09-17T04:24:16.224308+0000","last_clean":"2024-09-17T04:24:16.224308+0000","last_became_active":"2024-09-17T04:24:15.062311+0000","last_became_peered":"2024-09-17T04:24:15.062311+0000","last_unstale":"2024-09-17T04:24:16.224308+0000","last_undegraded":"2024-09-17T04:24:16.224308+0000","last_fullsized":"2024-09-17T04:24:16.224308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:26:42.504856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224742+0000","last_change":"2024-09-17T04:24:15.070288+0000","last_active":"2024-09-17T04:24:16.224742+0000","last_peered":"2024-09-17T04:24:16.224742+0000","last_clean":"2024-09-17T04:24:16.224742+0000","last_became_active":"2024-09-17T04:24:15.068589+0000","last_became_peered":"2024-09-17T04:24:15.068589+0000","last_unstale":"2024-09-17T04:24:16.224742+0000","last_undegraded":"2024-09-17T04:24:16.224742+0000","last_fullsized":"2024-09-17T04:24:16.224742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:33.110035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225192+0000","last_change":"2024-09-17T04:24:15.062446+0000","last_active":"2024-09-17T04:24:16.225192+0000","last_peered":"2024-09-17T04:24:16.225192+0000","last_clean":"2024-09-17T04:24:16.225192+0000","last_became_active":"2024-09-17T04:24:15.060522+0000","last_became_peered":"2024-09-17T04:24:15.060522+0000","last_unstale":"2024-09-17T04:24:16.225192+0000","last_undegraded":"2024-09-17T04:24:16.225192+0000","last_fullsized":"2024-09-17T04:24:16.225192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:09.349585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224815+0000","last_change":"2024-09-17T04:24:15.062048+0000","last_active":"2024-09-17T04:24:16.224815+0000","last_peered":"2024-09-17T04:24:16.224815+0000","last_clean":"2024-09-17T04:24:16.224815+0000","last_became_active":"2024-09-17T04:24:15.060887+0000","last_became_peered":"2024-09-17T04:24:15.060887+0000","last_unstale":"2024-09-17T04:24:16.224815+0000","last_undegraded":"2024-09-17T04:24:16.224815+0000","last_fullsized":"2024-09-17T04:24:16.224815+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:22.452641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054679+0000","last_change":"2024-09-17T04:24:15.068320+0000","last_active":"2024-09-17T04:24:16.054679+0000","last_peered":"2024-09-17T04:24:16.054679+0000","last_clean":"2024-09-17T04:24:16.054679+0000","last_became_active":"2024-09-17T04:24:15.067973+0000","last_became_peered":"2024-09-17T04:24:15.067973+0000","last_unstale":"2024-09-17T04:24:16.054679+0000","last_undegraded":"2024-09-17T04:24:16.054679+0000","last_fullsized":"2024-09-17T04:24:16.054679+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:04.323459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225225+0000","last_change":"2024-09-17T04:24:15.047951+0000","last_active":"2024-09-17T04:24:16.225225+0000","last_peered":"2024-09-17T04:24:16.225225+0000","last_clean":"2024-09-17T04:24:16.225225+0000","last_became_active":"2024-09-17T04:24:15.047536+0000","last_became_peered":"2024-09-17T04:24:15.047536+0000","last_unstale":"2024-09-17T04:24:16.225225+0000","last_undegraded":"2024-09-17T04:24:16.225225+0000","last_fullsized":"2024-09-17T04:24:16.225225+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:38:48.386032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224034+0000","last_change":"2024-09-17T04:24:15.070607+0000","last_active":"2024-09-17T04:24:16.224034+0000","last_peered":"2024-09-17T04:24:16.224034+0000","last_clean":"2024-09-17T04:24:16.224034+0000","last_became_active":"2024-09-17T04:24:15.067087+0000","last_became_peered":"2024-09-17T04:24:15.067087+0000","last_unstale":"2024-09-17T04:24:16.224034+0000","last_undegraded":"2024-09-17T04:24:16.224034+0000","last_fullsized":"2024-09-17T04:24:16.224034+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:09.349585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224091+0000","last_change":"2024-09-17T04:24:15.076095+0000","last_active":"2024-09-17T04:24:16.224091+0000","last_peered":"2024-09-17T04:24:16.224091+0000","last_clean":"2024-09-17T04:24:16.224091+0000","last_became_active":"2024-09-17T04:24:15.075803+0000","last_became_peered":"2024-09-17T04:24:15.075803+0000","last_unstale":"2024-09-17T04:24:16.224091+0000","last_undegraded":"2024-09-17T04:24:16.224091+0000","last_fullsized":"2024-09-17T04:24:16.224091+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:28.104736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055352+0000","last_change":"2024-09-17T04:24:15.053797+0000","last_active":"2024-09-17T04:24:16.055352+0000","last_peered":"2024-09-17T04:24:16.055352+0000","last_clean":"2024-09-17T04:24:16.055352+0000","last_became_active":"2024-09-17T04:24:15.053449+0000","last_became_peered":"2024-09-17T04:24:15.053449+0000","last_unstale":"2024-09-17T04:24:16.055352+0000","last_undegraded":"2024-09-17T04:24:16.055352+0000","last_fullsized":"2024-09-17T04:24:16.055352+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:49:03.743620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056725+0000","last_change":"2024-09-17T04:24:15.055632+0000","last_active":"2024-09-17T04:24:16.056725+0000","last_peered":"2024-09-17T04:24:16.056725+0000","last_clean":"2024-09-17T04:24:16.056725+0000","last_became_active":"2024-09-17T04:24:15.055156+0000","last_became_peered":"2024-09-17T04:24:15.055156+0000","last_unstale":"2024-09-17T04:24:16.056725+0000","last_undegraded":"2024-09-17T04:24:16.056725+0000","last_fullsized":"2024-09-17T04:24:16.056725+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:16.120591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.052124+0000","last_change":"2024-09-17T04:24:15.057788+0000","last_active":"2024-09-17T04:24:16.052124+0000","last_peered":"2024-09-17T04:24:16.052124+0000","last_clean":"2024-09-17T04:24:16.052124+0000","last_became_active":"2024-09-17T04:24:15.057141+0000","last_became_peered":"2024-09-17T04:24:15.057141+0000","last_unstale":"2024-09-17T04:24:16.052124+0000","last_undegraded":"2024-09-17T04:24:16.052124+0000","last_fullsized":"2024-09-17T04:24:16.052124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:10.704647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224426+0000","last_change":"2024-09-17T04:24:15.067312+0000","last_active":"2024-09-17T04:24:16.224426+0000","last_peered":"2024-09-17T04:24:16.224426+0000","last_clean":"2024-09-17T04:24:16.224426+0000","last_became_active":"2024-09-17T04:24:15.059516+0000","last_became_peered":"2024-09-17T04:24:15.059516+0000","last_unstale":"2024-09-17T04:24:16.224426+0000","last_undegraded":"2024-09-17T04:24:16.224426+0000","last_fullsized":"2024-09-17T04:24:16.224426+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:16.120591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224948+0000","last_change":"2024-09-17T04:24:15.076912+0000","last_active":"2024-09-17T04:24:16.224948+0000","last_peered":"2024-09-17T04:24:16.224948+0000","last_clean":"2024-09-17T04:24:16.224948+0000","last_became_active":"2024-09-17T04:24:15.076617+0000","last_became_peered":"2024-09-17T04:24:15.076617+0000","last_unstale":"2024-09-17T04:24:16.224948+0000","last_undegraded":"2024-09-17T04:24:16.224948+0000","last_fullsized":"2024-09-17T04:24:16.224948+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:47.639252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223809+0000","last_change":"2024-09-17T04:24:15.070958+0000","last_active":"2024-09-17T04:24:16.223809+0000","last_peered":"2024-09-17T04:24:16.223809+0000","last_clean":"2024-09-17T04:24:16.223809+0000","last_became_active":"2024-09-17T04:24:15.070624+0000","last_became_peered":"2024-09-17T04:24:15.070624+0000","last_unstale":"2024-09-17T04:24:16.223809+0000","last_undegraded":"2024-09-17T04:24:16.223809+0000","last_fullsized":"2024-09-17T04:24:16.223809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:13.752351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055648+0000","last_change":"2024-09-17T04:24:15.058815+0000","last_active":"2024-09-17T04:24:16.055648+0000","last_peered":"2024-09-17T04:24:16.055648+0000","last_clean":"2024-09-17T04:24:16.055648+0000","last_became_active":"2024-09-17T04:24:15.058482+0000","last_became_peered":"2024-09-17T04:24:15.058482+0000","last_unstale":"2024-09-17T04:24:16.055648+0000","last_undegraded":"2024-09-17T04:24:16.055648+0000","last_fullsized":"2024-09-17T04:24:16.055648+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:18.409788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224382+0000","last_change":"2024-09-17T04:24:15.070200+0000","last_active":"2024-09-17T04:24:16.224382+0000","last_peered":"2024-09-17T04:24:16.224382+0000","last_clean":"2024-09-17T04:24:16.224382+0000","last_became_active":"2024-09-17T04:24:15.069872+0000","last_became_peered":"2024-09-17T04:24:15.069872+0000","last_unstale":"2024-09-17T04:24:16.224382+0000","last_undegraded":"2024-09-17T04:24:16.224382+0000","last_fullsized":"2024-09-17T04:24:16.224382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:22.880022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055404+0000","last_change":"2024-09-17T04:24:15.050935+0000","last_active":"2024-09-17T04:24:16.055404+0000","last_peered":"2024-09-17T04:24:16.055404+0000","last_clean":"2024-09-17T04:24:16.055404+0000","last_became_active":"2024-09-17T04:24:15.048946+0000","last_became_peered":"2024-09-17T04:24:15.048946+0000","last_unstale":"2024-09-17T04:24:16.055404+0000","last_undegraded":"2024-09-17T04:24:16.055404+0000","last_fullsized":"2024-09-17T04:24:16.055404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:11:09.116226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225400+0000","last_change":"2024-09-17T04:24:15.074863+0000","last_active":"2024-09-17T04:24:16.225400+0000","last_peered":"2024-09-17T04:24:16.225400+0000","last_clean":"2024-09-17T04:24:16.225400+0000","last_became_active":"2024-09-17T04:24:15.074467+0000","last_became_peered":"2024-09-17T04:24:15.074467+0000","last_unstale":"2024-09-17T04:24:16.225400+0000","last_undegraded":"2024-09-17T04:24:16.225400+0000","last_fullsized":"2024-09-17T04:24:16.225400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:36:59.215343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225206+0000","last_change":"2024-09-17T04:24:15.069430+0000","last_active":"2024-09-17T04:24:16.225206+0000","last_peered":"2024-09-17T04:24:16.225206+0000","last_clean":"2024-09-17T04:24:16.225206+0000","last_became_active":"2024-09-17T04:24:15.069117+0000","last_became_peered":"2024-09-17T04:24:15.069117+0000","last_unstale":"2024-09-17T04:24:16.225206+0000","last_undegraded":"2024-09-17T04:24:16.225206+0000","last_fullsized":"2024-09-17T04:24:16.225206+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:49:03.743620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224018+0000","last_change":"2024-09-17T04:24:15.057539+0000","last_active":"2024-09-17T04:24:16.224018+0000","last_peered":"2024-09-17T04:24:16.224018+0000","last_clean":"2024-09-17T04:24:16.224018+0000","last_became_active":"2024-09-17T04:24:15.057194+0000","last_became_peered":"2024-09-17T04:24:15.057194+0000","last_unstale":"2024-09-17T04:24:16.224018+0000","last_undegraded":"2024-09-17T04:24:16.224018+0000","last_fullsized":"2024-09-17T04:24:16.224018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:09.349585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225430+0000","last_change":"2024-09-17T04:24:15.075902+0000","last_active":"2024-09-17T04:24:16.225430+0000","last_peered":"2024-09-17T04:24:16.225430+0000","last_clean":"2024-09-17T04:24:16.225430+0000","last_became_active":"2024-09-17T04:24:15.075625+0000","last_became_peered":"2024-09-17T04:24:15.075625+0000","last_unstale":"2024-09-17T04:24:16.225430+0000","last_undegraded":"2024-09-17T04:24:16.225430+0000","last_fullsized":"2024-09-17T04:24:16.225430+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:34.202614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223788+0000","last_change":"2024-09-17T04:24:15.052108+0000","last_active":"2024-09-17T04:24:16.223788+0000","last_peered":"2024-09-17T04:24:16.223788+0000","last_clean":"2024-09-17T04:24:16.223788+0000","last_became_active":"2024-09-17T04:24:15.051794+0000","last_became_peered":"2024-09-17T04:24:15.051794+0000","last_unstale":"2024-09-17T04:24:16.223788+0000","last_undegraded":"2024-09-17T04:24:16.223788+0000","last_fullsized":"2024-09-17T04:24:16.223788+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:13:56.486773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055909+0000","last_change":"2024-09-17T04:24:15.057335+0000","last_active":"2024-09-17T04:24:16.055909+0000","last_peered":"2024-09-17T04:24:16.055909+0000","last_clean":"2024-09-17T04:24:16.055909+0000","last_became_active":"2024-09-17T04:24:15.056924+0000","last_became_peered":"2024-09-17T04:24:15.056924+0000","last_unstale":"2024-09-17T04:24:16.055909+0000","last_undegraded":"2024-09-17T04:24:16.055909+0000","last_fullsized":"2024-09-17T04:24:16.055909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:29:56.515791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.052052+0000","last_change":"2024-09-17T04:24:15.054070+0000","last_active":"2024-09-17T04:24:16.052052+0000","last_peered":"2024-09-17T04:24:16.052052+0000","last_clean":"2024-09-17T04:24:16.052052+0000","last_became_active":"2024-09-17T04:24:15.052619+0000","last_became_peered":"2024-09-17T04:24:15.052619+0000","last_unstale":"2024-09-17T04:24:16.052052+0000","last_undegraded":"2024-09-17T04:24:16.052052+0000","last_fullsized":"2024-09-17T04:24:16.052052+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:49.409374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223562+0000","last_change":"2024-09-17T04:24:15.061084+0000","last_active":"2024-09-17T04:24:16.223562+0000","last_peered":"2024-09-17T04:24:16.223562+0000","last_clean":"2024-09-17T04:24:16.223562+0000","last_became_active":"2024-09-17T04:24:15.060769+0000","last_became_peered":"2024-09-17T04:24:15.060769+0000","last_unstale":"2024-09-17T04:24:16.223562+0000","last_undegraded":"2024-09-17T04:24:16.223562+0000","last_fullsized":"2024-09-17T04:24:16.223562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:13:56.486773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054106+0000","last_change":"2024-09-17T04:24:15.068899+0000","last_active":"2024-09-17T04:24:16.054106+0000","last_peered":"2024-09-17T04:24:16.054106+0000","last_clean":"2024-09-17T04:24:16.054106+0000","last_became_active":"2024-09-17T04:24:15.068595+0000","last_became_peered":"2024-09-17T04:24:15.068595+0000","last_unstale":"2024-09-17T04:24:16.054106+0000","last_undegraded":"2024-09-17T04:24:16.054106+0000","last_fullsized":"2024-09-17T04:24:16.054106+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:13.785027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225374+0000","last_change":"2024-09-17T04:24:14.062806+0000","last_active":"2024-09-17T04:24:16.225374+0000","last_peered":"2024-09-17T04:24:16.225374+0000","last_clean":"2024-09-17T04:24:16.225374+0000","last_became_active":"2024-09-17T04:24:14.062282+0000","last_became_peered":"2024-09-17T04:24:14.062282+0000","last_unstale":"2024-09-17T04:24:16.225374+0000","last_undegraded":"2024-09-17T04:24:16.225374+0000","last_fullsized":"2024-09-17T04:24:16.225374+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:23.698652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224005+0000","last_change":"2024-09-17T04:24:15.075318+0000","last_active":"2024-09-17T04:24:16.224005+0000","last_peered":"2024-09-17T04:24:16.224005+0000","last_clean":"2024-09-17T04:24:16.224005+0000","last_became_active":"2024-09-17T04:24:15.074876+0000","last_became_peered":"2024-09-17T04:24:15.074876+0000","last_unstale":"2024-09-17T04:24:16.224005+0000","last_undegraded":"2024-09-17T04:24:16.224005+0000","last_fullsized":"2024-09-17T04:24:16.224005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:47.639252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224710+0000","last_change":"2024-09-17T04:24:14.056222+0000","last_active":"2024-09-17T04:24:16.224710+0000","last_peered":"2024-09-17T04:24:16.224710+0000","last_clean":"2024-09-17T04:24:16.224710+0000","last_became_active":"2024-09-17T04:24:14.055874+0000","last_became_peered":"2024-09-17T04:24:14.055874+0000","last_unstale":"2024-09-17T04:24:16.224710+0000","last_undegraded":"2024-09-17T04:24:16.224710+0000","last_fullsized":"2024-09-17T04:24:16.224710+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:44.734015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224568+0000","last_change":"2024-09-17T04:24:14.054189+0000","last_active":"2024-09-17T04:24:16.224568+0000","last_peered":"2024-09-17T04:24:16.224568+0000","last_clean":"2024-09-17T04:24:16.224568+0000","last_became_active":"2024-09-17T04:24:14.053870+0000","last_became_peered":"2024-09-17T04:24:14.053870+0000","last_unstale":"2024-09-17T04:24:16.224568+0000","last_undegraded":"2024-09-17T04:24:16.224568+0000","last_fullsized":"2024-09-17T04:24:16.224568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:12.776180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224542+0000","last_change":"2024-09-17T04:24:15.059936+0000","last_active":"2024-09-17T04:24:16.224542+0000","last_peered":"2024-09-17T04:24:16.224542+0000","last_clean":"2024-09-17T04:24:16.224542+0000","last_became_active":"2024-09-17T04:24:15.059645+0000","last_became_peered":"2024-09-17T04:24:15.059645+0000","last_unstale":"2024-09-17T04:24:16.224542+0000","last_undegraded":"2024-09-17T04:24:16.224542+0000","last_fullsized":"2024-09-17T04:24:16.224542+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:08:11.703658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055748+0000","last_change":"2024-09-17T04:24:14.060082+0000","last_active":"2024-09-17T04:24:16.055748+0000","last_peered":"2024-09-17T04:24:16.055748+0000","last_clean":"2024-09-17T04:24:16.055748+0000","last_became_active":"2024-09-17T04:24:14.058795+0000","last_became_peered":"2024-09-17T04:24:14.058795+0000","last_unstale":"2024-09-17T04:24:16.055748+0000","last_undegraded":"2024-09-17T04:24:16.055748+0000","last_fullsized":"2024-09-17T04:24:16.055748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:21.535057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.222826+0000","last_change":"2024-09-17T04:24:15.061788+0000","last_active":"2024-09-17T04:24:16.222826+0000","last_peered":"2024-09-17T04:24:16.222826+0000","last_clean":"2024-09-17T04:24:16.222826+0000","last_became_active":"2024-09-17T04:24:15.055072+0000","last_became_peered":"2024-09-17T04:24:15.055072+0000","last_unstale":"2024-09-17T04:24:16.222826+0000","last_undegraded":"2024-09-17T04:24:16.222826+0000","last_fullsized":"2024-09-17T04:24:16.222826+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:13.752351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225245+0000","last_change":"2024-09-17T04:24:14.053595+0000","last_active":"2024-09-17T04:24:16.225245+0000","last_peered":"2024-09-17T04:24:16.225245+0000","last_clean":"2024-09-17T04:24:16.225245+0000","last_became_active":"2024-09-17T04:24:14.053260+0000","last_became_peered":"2024-09-17T04:24:14.053260+0000","last_unstale":"2024-09-17T04:24:16.225245+0000","last_undegraded":"2024-09-17T04:24:16.225245+0000","last_fullsized":"2024-09-17T04:24:16.225245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:51.429181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056660+0000","last_change":"2024-09-17T04:24:15.053812+0000","last_active":"2024-09-17T04:24:16.056660+0000","last_peered":"2024-09-17T04:24:16.056660+0000","last_clean":"2024-09-17T04:24:16.056660+0000","last_became_active":"2024-09-17T04:24:15.053501+0000","last_became_peered":"2024-09-17T04:24:15.053501+0000","last_unstale":"2024-09-17T04:24:16.056660+0000","last_undegraded":"2024-09-17T04:24:16.056660+0000","last_fullsized":"2024-09-17T04:24:16.056660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:47.959956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056115+0000","last_change":"2024-09-17T04:24:14.059203+0000","last_active":"2024-09-17T04:24:16.056115+0000","last_peered":"2024-09-17T04:24:16.056115+0000","last_clean":"2024-09-17T04:24:16.056115+0000","last_became_active":"2024-09-17T04:24:14.058623+0000","last_became_peered":"2024-09-17T04:24:14.058623+0000","last_unstale":"2024-09-17T04:24:16.056115+0000","last_undegraded":"2024-09-17T04:24:16.056115+0000","last_fullsized":"2024-09-17T04:24:16.056115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:20.372651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223819+0000","last_change":"2024-09-17T04:24:15.068669+0000","last_active":"2024-09-17T04:24:16.223819+0000","last_peered":"2024-09-17T04:24:16.223819+0000","last_clean":"2024-09-17T04:24:16.223819+0000","last_became_active":"2024-09-17T04:24:15.066578+0000","last_became_peered":"2024-09-17T04:24:15.066578+0000","last_unstale":"2024-09-17T04:24:16.223819+0000","last_undegraded":"2024-09-17T04:24:16.223819+0000","last_fullsized":"2024-09-17T04:24:16.223819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:01.706191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223044+0000","last_change":"2024-09-17T04:24:14.061969+0000","last_active":"2024-09-17T04:24:16.223044+0000","last_peered":"2024-09-17T04:24:16.223044+0000","last_clean":"2024-09-17T04:24:16.223044+0000","last_became_active":"2024-09-17T04:24:14.061593+0000","last_became_peered":"2024-09-17T04:24:14.061593+0000","last_unstale":"2024-09-17T04:24:16.223044+0000","last_undegraded":"2024-09-17T04:24:16.223044+0000","last_fullsized":"2024-09-17T04:24:16.223044+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:48.406614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054926+0000","last_change":"2024-09-17T04:24:15.071779+0000","last_active":"2024-09-17T04:24:16.054926+0000","last_peered":"2024-09-17T04:24:16.054926+0000","last_clean":"2024-09-17T04:24:16.054926+0000","last_became_active":"2024-09-17T04:24:15.071431+0000","last_became_peered":"2024-09-17T04:24:15.071431+0000","last_unstale":"2024-09-17T04:24:16.054926+0000","last_undegraded":"2024-09-17T04:24:16.054926+0000","last_fullsized":"2024-09-17T04:24:16.054926+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:10.704647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224939+0000","last_change":"2024-09-17T04:24:14.058729+0000","last_active":"2024-09-17T04:24:16.224939+0000","last_peered":"2024-09-17T04:24:16.224939+0000","last_clean":"2024-09-17T04:24:16.224939+0000","last_became_active":"2024-09-17T04:24:14.057616+0000","last_became_peered":"2024-09-17T04:24:14.057616+0000","last_unstale":"2024-09-17T04:24:16.224939+0000","last_undegraded":"2024-09-17T04:24:16.224939+0000","last_fullsized":"2024-09-17T04:24:16.224939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:42.267414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223701+0000","last_change":"2024-09-17T04:24:15.056613+0000","last_active":"2024-09-17T04:24:16.223701+0000","last_peered":"2024-09-17T04:24:16.223701+0000","last_clean":"2024-09-17T04:24:16.223701+0000","last_became_active":"2024-09-17T04:24:15.056300+0000","last_became_peered":"2024-09-17T04:24:15.056300+0000","last_unstale":"2024-09-17T04:24:16.223701+0000","last_undegraded":"2024-09-17T04:24:16.223701+0000","last_fullsized":"2024-09-17T04:24:16.223701+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:34.142688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223885+0000","last_change":"2024-09-17T04:24:14.054359+0000","last_active":"2024-09-17T04:24:16.223885+0000","last_peered":"2024-09-17T04:24:16.223885+0000","last_clean":"2024-09-17T04:24:16.223885+0000","last_became_active":"2024-09-17T04:24:14.053935+0000","last_became_peered":"2024-09-17T04:24:14.053935+0000","last_unstale":"2024-09-17T04:24:16.223885+0000","last_undegraded":"2024-09-17T04:24:16.223885+0000","last_fullsized":"2024-09-17T04:24:16.223885+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:01.211818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223752+0000","last_change":"2024-09-17T04:24:15.068236+0000","last_active":"2024-09-17T04:24:16.223752+0000","last_peered":"2024-09-17T04:24:16.223752+0000","last_clean":"2024-09-17T04:24:16.223752+0000","last_became_active":"2024-09-17T04:24:15.067460+0000","last_became_peered":"2024-09-17T04:24:15.067460+0000","last_unstale":"2024-09-17T04:24:16.223752+0000","last_undegraded":"2024-09-17T04:24:16.223752+0000","last_fullsized":"2024-09-17T04:24:16.223752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:44:58.329654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056099+0000","last_change":"2024-09-17T04:24:14.058127+0000","last_active":"2024-09-17T04:24:16.056099+0000","last_peered":"2024-09-17T04:24:16.056099+0000","last_clean":"2024-09-17T04:24:16.056099+0000","last_became_active":"2024-09-17T04:24:14.055990+0000","last_became_peered":"2024-09-17T04:24:14.055990+0000","last_unstale":"2024-09-17T04:24:16.056099+0000","last_undegraded":"2024-09-17T04:24:16.056099+0000","last_fullsized":"2024-09-17T04:24:16.056099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:51.429181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223887+0000","last_change":"2024-09-17T04:24:15.076359+0000","last_active":"2024-09-17T04:24:16.223887+0000","last_peered":"2024-09-17T04:24:16.223887+0000","last_clean":"2024-09-17T04:24:16.223887+0000","last_became_active":"2024-09-17T04:24:15.075235+0000","last_became_peered":"2024-09-17T04:24:15.075235+0000","last_unstale":"2024-09-17T04:24:16.223887+0000","last_undegraded":"2024-09-17T04:24:16.223887+0000","last_fullsized":"2024-09-17T04:24:16.223887+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:44.355705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224496+0000","last_change":"2024-09-17T04:24:14.051949+0000","last_active":"2024-09-17T04:24:16.224496+0000","last_peered":"2024-09-17T04:24:16.224496+0000","last_clean":"2024-09-17T04:24:16.224496+0000","last_became_active":"2024-09-17T04:24:14.051548+0000","last_became_peered":"2024-09-17T04:24:14.051548+0000","last_unstale":"2024-09-17T04:24:16.224496+0000","last_undegraded":"2024-09-17T04:24:16.224496+0000","last_fullsized":"2024-09-17T04:24:16.224496+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:21.535057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223899+0000","last_change":"2024-09-17T04:24:15.060676+0000","last_active":"2024-09-17T04:24:16.223899+0000","last_peered":"2024-09-17T04:24:16.223899+0000","last_clean":"2024-09-17T04:24:16.223899+0000","last_became_active":"2024-09-17T04:24:15.060268+0000","last_became_peered":"2024-09-17T04:24:15.060268+0000","last_unstale":"2024-09-17T04:24:16.223899+0000","last_undegraded":"2024-09-17T04:24:16.223899+0000","last_fullsized":"2024-09-17T04:24:16.223899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:13.752351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.051789+0000","last_change":"2024-09-17T04:24:14.043539+0000","last_active":"2024-09-17T04:24:16.051789+0000","last_peered":"2024-09-17T04:24:16.051789+0000","last_clean":"2024-09-17T04:24:16.051789+0000","last_became_active":"2024-09-17T04:24:14.043214+0000","last_became_peered":"2024-09-17T04:24:14.043214+0000","last_unstale":"2024-09-17T04:24:16.051789+0000","last_undegraded":"2024-09-17T04:24:16.051789+0000","last_fullsized":"2024-09-17T04:24:16.051789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:42.267414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223990+0000","last_change":"2024-09-17T04:24:15.068886+0000","last_active":"2024-09-17T04:24:16.223990+0000","last_peered":"2024-09-17T04:24:16.223990+0000","last_clean":"2024-09-17T04:24:16.223990+0000","last_became_active":"2024-09-17T04:24:15.068532+0000","last_became_peered":"2024-09-17T04:24:15.068532+0000","last_unstale":"2024-09-17T04:24:16.223990+0000","last_undegraded":"2024-09-17T04:24:16.223990+0000","last_fullsized":"2024-09-17T04:24:16.223990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:58.719218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054761+0000","last_change":"2024-09-17T04:24:14.059673+0000","last_active":"2024-09-17T04:24:16.054761+0000","last_peered":"2024-09-17T04:24:16.054761+0000","last_clean":"2024-09-17T04:24:16.054761+0000","last_became_active":"2024-09-17T04:24:14.057668+0000","last_became_peered":"2024-09-17T04:24:14.057668+0000","last_unstale":"2024-09-17T04:24:16.054761+0000","last_undegraded":"2024-09-17T04:24:16.054761+0000","last_fullsized":"2024-09-17T04:24:16.054761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:01.211818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224143+0000","last_change":"2024-09-17T04:24:15.069931+0000","last_active":"2024-09-17T04:24:16.224143+0000","last_peered":"2024-09-17T04:24:16.224143+0000","last_clean":"2024-09-17T04:24:16.224143+0000","last_became_active":"2024-09-17T04:24:15.069568+0000","last_became_peered":"2024-09-17T04:24:15.069568+0000","last_unstale":"2024-09-17T04:24:16.224143+0000","last_undegraded":"2024-09-17T04:24:16.224143+0000","last_fullsized":"2024-09-17T04:24:16.224143+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:35.584338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.052093+0000","last_change":"2024-09-17T04:24:14.054912+0000","last_active":"2024-09-17T04:24:16.052093+0000","last_peered":"2024-09-17T04:24:16.052093+0000","last_clean":"2024-09-17T04:24:16.052093+0000","last_became_active":"2024-09-17T04:24:14.054105+0000","last_became_peered":"2024-09-17T04:24:14.054105+0000","last_unstale":"2024-09-17T04:24:16.052093+0000","last_undegraded":"2024-09-17T04:24:16.052093+0000","last_fullsized":"2024-09-17T04:24:16.052093+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:44.734015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224717+0000","last_change":"2024-09-17T04:24:15.075676+0000","last_active":"2024-09-17T04:24:16.224717+0000","last_peered":"2024-09-17T04:24:16.224717+0000","last_clean":"2024-09-17T04:24:16.224717+0000","last_became_active":"2024-09-17T04:24:15.075114+0000","last_became_peered":"2024-09-17T04:24:15.075114+0000","last_unstale":"2024-09-17T04:24:16.224717+0000","last_undegraded":"2024-09-17T04:24:16.224717+0000","last_fullsized":"2024-09-17T04:24:16.224717+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:51:30.987569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223146+0000","last_change":"2024-09-17T04:24:14.039417+0000","last_active":"2024-09-17T04:24:16.223146+0000","last_peered":"2024-09-17T04:24:16.223146+0000","last_clean":"2024-09-17T04:24:16.223146+0000","last_became_active":"2024-09-17T04:24:14.039025+0000","last_became_peered":"2024-09-17T04:24:14.039025+0000","last_unstale":"2024-09-17T04:24:16.223146+0000","last_undegraded":"2024-09-17T04:24:16.223146+0000","last_fullsized":"2024-09-17T04:24:16.223146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:03.320698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056433+0000","last_change":"2024-09-17T04:24:15.061023+0000","last_active":"2024-09-17T04:24:16.056433+0000","last_peered":"2024-09-17T04:24:16.056433+0000","last_clean":"2024-09-17T04:24:16.056433+0000","last_became_active":"2024-09-17T04:24:15.060544+0000","last_became_peered":"2024-09-17T04:24:15.060544+0000","last_unstale":"2024-09-17T04:24:16.056433+0000","last_undegraded":"2024-09-17T04:24:16.056433+0000","last_fullsized":"2024-09-17T04:24:16.056433+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:35:10.792366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.051950+0000","last_change":"2024-09-17T04:24:14.052529+0000","last_active":"2024-09-17T04:24:16.051950+0000","last_peered":"2024-09-17T04:24:16.051950+0000","last_clean":"2024-09-17T04:24:16.051950+0000","last_became_active":"2024-09-17T04:24:14.052135+0000","last_became_peered":"2024-09-17T04:24:14.052135+0000","last_unstale":"2024-09-17T04:24:16.051950+0000","last_undegraded":"2024-09-17T04:24:16.051950+0000","last_fullsized":"2024-09-17T04:24:16.051950+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:28.196011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224039+0000","last_change":"2024-09-17T04:24:15.064435+0000","last_active":"2024-09-17T04:24:16.224039+0000","last_peered":"2024-09-17T04:24:16.224039+0000","last_clean":"2024-09-17T04:24:16.224039+0000","last_became_active":"2024-09-17T04:24:15.064195+0000","last_became_peered":"2024-09-17T04:24:15.064195+0000","last_unstale":"2024-09-17T04:24:16.224039+0000","last_undegraded":"2024-09-17T04:24:16.224039+0000","last_fullsized":"2024-09-17T04:24:16.224039+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:47.959956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223547+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:24:16.223547+0000","last_peered":"2024-09-17T04:24:16.223547+0000","last_clean":"2024-09-17T04:24:16.223547+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:24:16.223547+0000","last_undegraded":"2024-09-17T04:24:16.223547+0000","last_fullsized":"2024-09-17T04:24:16.223547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055535+0000","last_change":"2024-09-17T04:24:14.049088+0000","last_active":"2024-09-17T04:24:16.055535+0000","last_peered":"2024-09-17T04:24:16.055535+0000","last_clean":"2024-09-17T04:24:16.055535+0000","last_became_active":"2024-09-17T04:24:14.048529+0000","last_became_peered":"2024-09-17T04:24:14.048529+0000","last_unstale":"2024-09-17T04:24:16.055535+0000","last_undegraded":"2024-09-17T04:24:16.055535+0000","last_fullsized":"2024-09-17T04:24:16.055535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:42.267414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225083+0000","last_change":"2024-09-17T04:24:15.065699+0000","last_active":"2024-09-17T04:24:16.225083+0000","last_peered":"2024-09-17T04:24:16.225083+0000","last_clean":"2024-09-17T04:24:16.225083+0000","last_became_active":"2024-09-17T04:24:15.061569+0000","last_became_peered":"2024-09-17T04:24:15.061569+0000","last_unstale":"2024-09-17T04:24:16.225083+0000","last_undegraded":"2024-09-17T04:24:16.225083+0000","last_fullsized":"2024-09-17T04:24:16.225083+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:01:19.496265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223384+0000","last_change":"2024-09-17T04:24:14.061081+0000","last_active":"2024-09-17T04:24:16.223384+0000","last_peered":"2024-09-17T04:24:16.223384+0000","last_clean":"2024-09-17T04:24:16.223384+0000","last_became_active":"2024-09-17T04:24:14.060394+0000","last_became_peered":"2024-09-17T04:24:14.060394+0000","last_unstale":"2024-09-17T04:24:16.223384+0000","last_undegraded":"2024-09-17T04:24:16.223384+0000","last_fullsized":"2024-09-17T04:24:16.223384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:08.023686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223901+0000","last_change":"2024-09-17T04:24:15.070960+0000","last_active":"2024-09-17T04:24:16.223901+0000","last_peered":"2024-09-17T04:24:16.223901+0000","last_clean":"2024-09-17T04:24:16.223901+0000","last_became_active":"2024-09-17T04:24:15.070528+0000","last_became_peered":"2024-09-17T04:24:15.070528+0000","last_unstale":"2024-09-17T04:24:16.223901+0000","last_undegraded":"2024-09-17T04:24:16.223901+0000","last_fullsized":"2024-09-17T04:24:16.223901+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:22.452641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.052102+0000","last_change":"2024-09-17T04:24:14.051042+0000","last_active":"2024-09-17T04:24:16.052102+0000","last_peered":"2024-09-17T04:24:16.052102+0000","last_clean":"2024-09-17T04:24:16.052102+0000","last_became_active":"2024-09-17T04:24:14.050513+0000","last_became_peered":"2024-09-17T04:24:14.050513+0000","last_unstale":"2024-09-17T04:24:16.052102+0000","last_undegraded":"2024-09-17T04:24:16.052102+0000","last_fullsized":"2024-09-17T04:24:16.052102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:21.535057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.222913+0000","last_change":"2024-09-17T04:24:15.061656+0000","last_active":"2024-09-17T04:24:16.222913+0000","last_peered":"2024-09-17T04:24:16.222913+0000","last_clean":"2024-09-17T04:24:16.222913+0000","last_became_active":"2024-09-17T04:24:15.061365+0000","last_became_peered":"2024-09-17T04:24:15.061365+0000","last_unstale":"2024-09-17T04:24:16.222913+0000","last_undegraded":"2024-09-17T04:24:16.222913+0000","last_fullsized":"2024-09-17T04:24:16.222913+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:22.452641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056245+0000","last_change":"2024-09-17T04:24:14.053751+0000","last_active":"2024-09-17T04:24:16.056245+0000","last_peered":"2024-09-17T04:24:16.056245+0000","last_clean":"2024-09-17T04:24:16.056245+0000","last_became_active":"2024-09-17T04:24:14.051943+0000","last_became_peered":"2024-09-17T04:24:14.051943+0000","last_unstale":"2024-09-17T04:24:16.056245+0000","last_undegraded":"2024-09-17T04:24:16.056245+0000","last_fullsized":"2024-09-17T04:24:16.056245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:08.023686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054263+0000","last_change":"2024-09-17T04:24:15.074736+0000","last_active":"2024-09-17T04:24:16.054263+0000","last_peered":"2024-09-17T04:24:16.054263+0000","last_clean":"2024-09-17T04:24:16.054263+0000","last_became_active":"2024-09-17T04:24:15.074374+0000","last_became_peered":"2024-09-17T04:24:15.074374+0000","last_unstale":"2024-09-17T04:24:16.054263+0000","last_undegraded":"2024-09-17T04:24:16.054263+0000","last_fullsized":"2024-09-17T04:24:16.054263+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:49.409374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.222921+0000","last_change":"2024-09-17T04:24:14.057105+0000","last_active":"2024-09-17T04:24:16.222921+0000","last_peered":"2024-09-17T04:24:16.222921+0000","last_clean":"2024-09-17T04:24:16.222921+0000","last_became_active":"2024-09-17T04:24:14.056908+0000","last_became_peered":"2024-09-17T04:24:14.056908+0000","last_unstale":"2024-09-17T04:24:16.222921+0000","last_undegraded":"2024-09-17T04:24:16.222921+0000","last_fullsized":"2024-09-17T04:24:16.222921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:12.782267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056301+0000","last_change":"2024-09-17T04:24:15.058663+0000","last_active":"2024-09-17T04:24:16.056301+0000","last_peered":"2024-09-17T04:24:16.056301+0000","last_clean":"2024-09-17T04:24:16.056301+0000","last_became_active":"2024-09-17T04:24:15.057807+0000","last_became_peered":"2024-09-17T04:24:15.057807+0000","last_unstale":"2024-09-17T04:24:16.056301+0000","last_undegraded":"2024-09-17T04:24:16.056301+0000","last_fullsized":"2024-09-17T04:24:16.056301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:35.584338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224488+0000","last_change":"2024-09-17T04:24:14.063354+0000","last_active":"2024-09-17T04:24:16.224488+0000","last_peered":"2024-09-17T04:24:16.224488+0000","last_clean":"2024-09-17T04:24:16.224488+0000","last_became_active":"2024-09-17T04:24:14.062231+0000","last_became_peered":"2024-09-17T04:24:14.062231+0000","last_unstale":"2024-09-17T04:24:16.224488+0000","last_undegraded":"2024-09-17T04:24:16.224488+0000","last_fullsized":"2024-09-17T04:24:16.224488+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:40.777526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.222850+0000","last_change":"2024-09-17T04:24:15.058346+0000","last_active":"2024-09-17T04:24:16.222850+0000","last_peered":"2024-09-17T04:24:16.222850+0000","last_clean":"2024-09-17T04:24:16.222850+0000","last_became_active":"2024-09-17T04:24:15.057977+0000","last_became_peered":"2024-09-17T04:24:15.057977+0000","last_unstale":"2024-09-17T04:24:16.222850+0000","last_undegraded":"2024-09-17T04:24:16.222850+0000","last_fullsized":"2024-09-17T04:24:16.222850+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:01.706191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223545+0000","last_change":"2024-09-17T04:24:14.049298+0000","last_active":"2024-09-17T04:24:16.223545+0000","last_peered":"2024-09-17T04:24:16.223545+0000","last_clean":"2024-09-17T04:24:16.223545+0000","last_became_active":"2024-09-17T04:24:14.049070+0000","last_became_peered":"2024-09-17T04:24:14.049070+0000","last_unstale":"2024-09-17T04:24:16.223545+0000","last_undegraded":"2024-09-17T04:24:16.223545+0000","last_fullsized":"2024-09-17T04:24:16.223545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:52:59.361386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225582+0000","last_change":"2024-09-17T04:24:15.050480+0000","last_active":"2024-09-17T04:24:16.225582+0000","last_peered":"2024-09-17T04:24:16.225582+0000","last_clean":"2024-09-17T04:24:16.225582+0000","last_became_active":"2024-09-17T04:24:15.050166+0000","last_became_peered":"2024-09-17T04:24:15.050166+0000","last_unstale":"2024-09-17T04:24:16.225582+0000","last_undegraded":"2024-09-17T04:24:16.225582+0000","last_fullsized":"2024-09-17T04:24:16.225582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:02.125884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225264+0000","last_change":"2024-09-17T04:24:14.063222+0000","last_active":"2024-09-17T04:24:16.225264+0000","last_peered":"2024-09-17T04:24:16.225264+0000","last_clean":"2024-09-17T04:24:16.225264+0000","last_became_active":"2024-09-17T04:24:14.062750+0000","last_became_peered":"2024-09-17T04:24:14.062750+0000","last_unstale":"2024-09-17T04:24:16.225264+0000","last_undegraded":"2024-09-17T04:24:16.225264+0000","last_fullsized":"2024-09-17T04:24:16.225264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:08.023686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224491+0000","last_change":"2024-09-17T04:24:15.067950+0000","last_active":"2024-09-17T04:24:16.224491+0000","last_peered":"2024-09-17T04:24:16.224491+0000","last_clean":"2024-09-17T04:24:16.224491+0000","last_became_active":"2024-09-17T04:24:15.066567+0000","last_became_peered":"2024-09-17T04:24:15.066567+0000","last_unstale":"2024-09-17T04:24:16.224491+0000","last_undegraded":"2024-09-17T04:24:16.224491+0000","last_fullsized":"2024-09-17T04:24:16.224491+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:28.104736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223381+0000","last_change":"2024-09-17T04:24:14.059487+0000","last_active":"2024-09-17T04:24:16.223381+0000","last_peered":"2024-09-17T04:24:16.223381+0000","last_clean":"2024-09-17T04:24:16.223381+0000","last_became_active":"2024-09-17T04:24:14.059186+0000","last_became_peered":"2024-09-17T04:24:14.059186+0000","last_unstale":"2024-09-17T04:24:16.223381+0000","last_undegraded":"2024-09-17T04:24:16.223381+0000","last_fullsized":"2024-09-17T04:24:16.223381+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:03.320698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054147+0000","last_change":"2024-09-17T04:24:15.065941+0000","last_active":"2024-09-17T04:24:16.054147+0000","last_peered":"2024-09-17T04:24:16.054147+0000","last_clean":"2024-09-17T04:24:16.054147+0000","last_became_active":"2024-09-17T04:24:15.065582+0000","last_became_peered":"2024-09-17T04:24:15.065582+0000","last_unstale":"2024-09-17T04:24:16.054147+0000","last_undegraded":"2024-09-17T04:24:16.054147+0000","last_fullsized":"2024-09-17T04:24:16.054147+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:30.122754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.222894+0000","last_change":"2024-09-17T04:24:14.062678+0000","last_active":"2024-09-17T04:24:16.222894+0000","last_peered":"2024-09-17T04:24:16.222894+0000","last_clean":"2024-09-17T04:24:16.222894+0000","last_became_active":"2024-09-17T04:24:14.062281+0000","last_became_peered":"2024-09-17T04:24:14.062281+0000","last_unstale":"2024-09-17T04:24:16.222894+0000","last_undegraded":"2024-09-17T04:24:16.222894+0000","last_fullsized":"2024-09-17T04:24:16.222894+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:40.254206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224778+0000","last_change":"2024-09-17T04:24:15.064498+0000","last_active":"2024-09-17T04:24:16.224778+0000","last_peered":"2024-09-17T04:24:16.224778+0000","last_clean":"2024-09-17T04:24:16.224778+0000","last_became_active":"2024-09-17T04:24:15.062700+0000","last_became_peered":"2024-09-17T04:24:15.062700+0000","last_unstale":"2024-09-17T04:24:16.224778+0000","last_undegraded":"2024-09-17T04:24:16.224778+0000","last_fullsized":"2024-09-17T04:24:16.224778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:31.021313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223265+0000","last_change":"2024-09-17T04:24:14.047923+0000","last_active":"2024-09-17T04:24:16.223265+0000","last_peered":"2024-09-17T04:24:16.223265+0000","last_clean":"2024-09-17T04:24:16.223265+0000","last_became_active":"2024-09-17T04:24:14.047596+0000","last_became_peered":"2024-09-17T04:24:14.047596+0000","last_unstale":"2024-09-17T04:24:16.223265+0000","last_undegraded":"2024-09-17T04:24:16.223265+0000","last_fullsized":"2024-09-17T04:24:16.223265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:29.119993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225575+0000","last_change":"2024-09-17T04:24:15.060939+0000","last_active":"2024-09-17T04:24:16.225575+0000","last_peered":"2024-09-17T04:24:16.225575+0000","last_clean":"2024-09-17T04:24:16.225575+0000","last_became_active":"2024-09-17T04:24:15.060600+0000","last_became_peered":"2024-09-17T04:24:15.060600+0000","last_unstale":"2024-09-17T04:24:16.225575+0000","last_undegraded":"2024-09-17T04:24:16.225575+0000","last_fullsized":"2024-09-17T04:24:16.225575+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:05.895665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224176+0000","last_change":"2024-09-17T04:24:14.062616+0000","last_active":"2024-09-17T04:24:16.224176+0000","last_peered":"2024-09-17T04:24:16.224176+0000","last_clean":"2024-09-17T04:24:16.224176+0000","last_became_active":"2024-09-17T04:24:14.062182+0000","last_became_peered":"2024-09-17T04:24:14.062182+0000","last_unstale":"2024-09-17T04:24:16.224176+0000","last_undegraded":"2024-09-17T04:24:16.224176+0000","last_fullsized":"2024-09-17T04:24:16.224176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:51.246396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224637+0000","last_change":"2024-09-17T04:24:15.079589+0000","last_active":"2024-09-17T04:24:16.224637+0000","last_peered":"2024-09-17T04:24:16.224637+0000","last_clean":"2024-09-17T04:24:16.224637+0000","last_became_active":"2024-09-17T04:24:15.079019+0000","last_became_peered":"2024-09-17T04:24:15.079019+0000","last_unstale":"2024-09-17T04:24:16.224637+0000","last_undegraded":"2024-09-17T04:24:16.224637+0000","last_fullsized":"2024-09-17T04:24:16.224637+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:01.706191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056070+0000","last_change":"2024-09-17T04:24:15.080533+0000","last_active":"2024-09-17T04:24:16.056070+0000","last_peered":"2024-09-17T04:24:16.056070+0000","last_clean":"2024-09-17T04:24:16.056070+0000","last_became_active":"2024-09-17T04:24:15.080179+0000","last_became_peered":"2024-09-17T04:24:15.080179+0000","last_unstale":"2024-09-17T04:24:16.056070+0000","last_undegraded":"2024-09-17T04:24:16.056070+0000","last_fullsized":"2024-09-17T04:24:16.056070+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:58.719218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224154+0000","last_change":"2024-09-17T04:24:14.065850+0000","last_active":"2024-09-17T04:24:16.224154+0000","last_peered":"2024-09-17T04:24:16.224154+0000","last_clean":"2024-09-17T04:24:16.224154+0000","last_became_active":"2024-09-17T04:24:14.065444+0000","last_became_peered":"2024-09-17T04:24:14.065444+0000","last_unstale":"2024-09-17T04:24:16.224154+0000","last_undegraded":"2024-09-17T04:24:16.224154+0000","last_fullsized":"2024-09-17T04:24:16.224154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:51.121630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056174+0000","last_change":"2024-09-17T04:24:15.076511+0000","last_active":"2024-09-17T04:24:16.056174+0000","last_peered":"2024-09-17T04:24:16.056174+0000","last_clean":"2024-09-17T04:24:16.056174+0000","last_became_active":"2024-09-17T04:24:15.076037+0000","last_became_peered":"2024-09-17T04:24:15.076037+0000","last_unstale":"2024-09-17T04:24:16.056174+0000","last_undegraded":"2024-09-17T04:24:16.056174+0000","last_fullsized":"2024-09-17T04:24:16.056174+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:06:32.518189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224180+0000","last_change":"2024-09-17T04:24:14.060131+0000","last_active":"2024-09-17T04:24:16.224180+0000","last_peered":"2024-09-17T04:24:16.224180+0000","last_clean":"2024-09-17T04:24:16.224180+0000","last_became_active":"2024-09-17T04:24:14.059707+0000","last_became_peered":"2024-09-17T04:24:14.059707+0000","last_unstale":"2024-09-17T04:24:16.224180+0000","last_undegraded":"2024-09-17T04:24:16.224180+0000","last_fullsized":"2024-09-17T04:24:16.224180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:22.869429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223586+0000","last_change":"2024-09-17T04:24:15.068641+0000","last_active":"2024-09-17T04:24:16.223586+0000","last_peered":"2024-09-17T04:24:16.223586+0000","last_clean":"2024-09-17T04:24:16.223586+0000","last_became_active":"2024-09-17T04:24:15.065609+0000","last_became_peered":"2024-09-17T04:24:15.065609+0000","last_unstale":"2024-09-17T04:24:16.223586+0000","last_undegraded":"2024-09-17T04:24:16.223586+0000","last_fullsized":"2024-09-17T04:24:16.223586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:43:02.004482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225369+0000","last_change":"2024-09-17T04:24:14.056735+0000","last_active":"2024-09-17T04:24:16.225369+0000","last_peered":"2024-09-17T04:24:16.225369+0000","last_clean":"2024-09-17T04:24:16.225369+0000","last_became_active":"2024-09-17T04:24:14.056433+0000","last_became_peered":"2024-09-17T04:24:14.056433+0000","last_unstale":"2024-09-17T04:24:16.225369+0000","last_undegraded":"2024-09-17T04:24:16.225369+0000","last_fullsized":"2024-09-17T04:24:16.225369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:03.618384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225403+0000","last_change":"2024-09-17T04:24:15.054773+0000","last_active":"2024-09-17T04:24:16.225403+0000","last_peered":"2024-09-17T04:24:16.225403+0000","last_clean":"2024-09-17T04:24:16.225403+0000","last_became_active":"2024-09-17T04:24:15.054453+0000","last_became_peered":"2024-09-17T04:24:15.054453+0000","last_unstale":"2024-09-17T04:24:16.225403+0000","last_undegraded":"2024-09-17T04:24:16.225403+0000","last_fullsized":"2024-09-17T04:24:16.225403+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:58.942093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223981+0000","last_change":"2024-09-17T04:24:14.062995+0000","last_active":"2024-09-17T04:24:16.223981+0000","last_peered":"2024-09-17T04:24:16.223981+0000","last_clean":"2024-09-17T04:24:16.223981+0000","last_became_active":"2024-09-17T04:24:14.062615+0000","last_became_peered":"2024-09-17T04:24:14.062615+0000","last_unstale":"2024-09-17T04:24:16.223981+0000","last_undegraded":"2024-09-17T04:24:16.223981+0000","last_fullsized":"2024-09-17T04:24:16.223981+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:20:07.263105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476743,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27844,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343996,"statfs":{"total":96636764160,"available":96608251904,"internally_reserved":0,"allocated":827392,"data_stored":670785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476742,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":81447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476743,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476743,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":79961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":79835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":237568,"data_stored":79995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":835584,"data_stored":670841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":81447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:24:24.890 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T04:24:24.891 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-17T04:24:24.891 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-17T04:24:25.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.148+0000 7fdc89a00640 1 -- 172.21.15.100:0/3018938190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc840b3720 msgr2=0x7fdc840d1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:25.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.148+0000 7fdc89a00640 1 --2- 172.21.15.100:0/3018938190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc840b3720 0x7fdc840d1090 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdc70005fd0 tx=0x7fdc7002a5f0 comp rx=0 tx=0).stop 2024-09-17T04:24:25.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.148+0000 7fdc89a00640 1 -- 172.21.15.100:0/3018938190 shutdown_connections 2024-09-17T04:24:25.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.148+0000 7fdc89a00640 1 --2- 172.21.15.100:0/3018938190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc840b3720 0x7fdc840d1090 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:25.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.148+0000 7fdc89a00640 1 -- 172.21.15.100:0/3018938190 >> 172.21.15.100:0/3018938190 conn(0x7fdc840d4680 msgr2=0x7fdc840d4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:25.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.149+0000 7fdc89a00640 1 -- 172.21.15.100:0/3018938190 shutdown_connections 2024-09-17T04:24:25.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.149+0000 7fdc89a00640 1 -- 172.21.15.100:0/3018938190 wait complete. 2024-09-17T04:24:25.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.151+0000 7fdc89a00640 1 Processor -- start 2024-09-17T04:24:25.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.151+0000 7fdc89a00640 1 -- start start 2024-09-17T04:24:25.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.152+0000 7fdc89a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc840b3720 0x7fdc841f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:25.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.152+0000 7fdc89a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc840b9780 0x7fdc841f99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:25.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.152+0000 7fdc89a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc840d15d0 0x7fdc8414a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:25.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.152+0000 7fdc89a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdc840d0020 con 0x7fdc840d15d0 2024-09-17T04:24:25.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.152+0000 7fdc89a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdc840cfea0 con 0x7fdc840b3720 2024-09-17T04:24:25.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.152+0000 7fdc89a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdc840d01a0 con 0x7fdc840b9780 2024-09-17T04:24:25.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.152+0000 7fdc83400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc840d15d0 0x7fdc8414a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:25.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.152+0000 7fdc83400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc840d15d0 0x7fdc8414a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47900/0 (socket says 172.21.15.100:47900) 2024-09-17T04:24:25.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.152+0000 7fdc83400640 1 -- 172.21.15.100:0/1020107289 learned_addr learned my addr 172.21.15.100:0/1020107289 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:25.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.153+0000 7fdc82a00640 1 --2- 172.21.15.100:0/1020107289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc840b3720 0x7fdc841f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:25.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.153+0000 7fdc82000640 1 --2- 172.21.15.100:0/1020107289 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc840b9780 0x7fdc841f99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:25.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.153+0000 7fdc83400640 1 -- 172.21.15.100:0/1020107289 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc840b9780 msgr2=0x7fdc841f99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:25.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.153+0000 7fdc83400640 1 --2- 172.21.15.100:0/1020107289 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc840b9780 0x7fdc841f99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:25.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.153+0000 7fdc83400640 1 -- 172.21.15.100:0/1020107289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc840b3720 msgr2=0x7fdc841f61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:25.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.153+0000 7fdc83400640 1 --2- 172.21.15.100:0/1020107289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc840b3720 0x7fdc841f61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:25.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.153+0000 7fdc83400640 1 -- 172.21.15.100:0/1020107289 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc8414b060 con 0x7fdc840d15d0 2024-09-17T04:24:25.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.153+0000 7fdc82a00640 1 --2- 172.21.15.100:0/1020107289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc840b3720 0x7fdc841f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:24:25.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.153+0000 7fdc83400640 1 --2- 172.21.15.100:0/1020107289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc840d15d0 0x7fdc8414a3c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c00c3c0 tx=0x7fdc7c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:25.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.153+0000 7fdc7aa00640 1 -- 172.21.15.100:0/1020107289 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c017070 con 0x7fdc840d15d0 2024-09-17T04:24:25.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.153+0000 7fdc7aa00640 1 -- 172.21.15.100:0/1020107289 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc7c0053c0 con 0x7fdc840d15d0 2024-09-17T04:24:25.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.153+0000 7fdc7aa00640 1 -- 172.21.15.100:0/1020107289 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c003850 con 0x7fdc840d15d0 2024-09-17T04:24:25.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.153+0000 7fdc89a00640 1 -- 172.21.15.100:0/1020107289 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc840b5210 con 0x7fdc840d15d0 2024-09-17T04:24:25.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.154+0000 7fdc89a00640 1 -- 172.21.15.100:0/1020107289 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc841f6cb0 con 0x7fdc840d15d0 2024-09-17T04:24:25.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.155+0000 7fdc89a00640 1 -- 172.21.15.100:0/1020107289 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc841f6a20 con 0x7fdc840d15d0 2024-09-17T04:24:25.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.156+0000 7fdc7aa00640 1 -- 172.21.15.100:0/1020107289 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdc7c01a070 con 0x7fdc840d15d0 2024-09-17T04:24:25.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.157+0000 7fdc7aa00640 1 --2- 172.21.15.100:0/1020107289 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdc64078bc0 0x7fdc6407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:25.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.157+0000 7fdc7aa00640 1 -- 172.21.15.100:0/1020107289 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fdc7c09a0e0 con 0x7fdc840d15d0 2024-09-17T04:24:25.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.157+0000 7fdc82a00640 1 --2- 172.21.15.100:0/1020107289 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdc64078bc0 0x7fdc6407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:25.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.158+0000 7fdc82a00640 1 --2- 172.21.15.100:0/1020107289 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdc64078bc0 0x7fdc6407b0d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdc74001350 tx=0x7fdc74009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:25.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.161+0000 7fdc7aa00640 1 -- 172.21.15.100:0/1020107289 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fdc7c061ec0 con 0x7fdc840d15d0 2024-09-17T04:24:25.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.347+0000 7fdc89a00640 1 -- 172.21.15.100:0/1020107289 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fdc8408d100 con 0x7fdc840d15d0 2024-09-17T04:24:25.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.348+0000 7fdc7aa00640 1 -- 172.21.15.100:0/1020107289 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v31) ==== 86+0+160 (secure 0 0 0) 0x7fdc8408d100 con 0x7fdc840d15d0 2024-09-17T04:24:25.349 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:25.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.350+0000 7fdc89a00640 1 -- 172.21.15.100:0/1020107289 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdc64078bc0 msgr2=0x7fdc6407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:25.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.350+0000 7fdc89a00640 1 --2- 172.21.15.100:0/1020107289 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdc64078bc0 0x7fdc6407b0d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdc74001350 tx=0x7fdc74009040 comp rx=0 tx=0).stop 2024-09-17T04:24:25.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.350+0000 7fdc89a00640 1 -- 172.21.15.100:0/1020107289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc840d15d0 msgr2=0x7fdc8414a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:25.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.350+0000 7fdc89a00640 1 --2- 172.21.15.100:0/1020107289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc840d15d0 0x7fdc8414a3c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c00c3c0 tx=0x7fdc7c00c880 comp rx=0 tx=0).stop 2024-09-17T04:24:25.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.351+0000 7fdc89a00640 1 -- 172.21.15.100:0/1020107289 shutdown_connections 2024-09-17T04:24:25.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.351+0000 7fdc89a00640 1 --2- 172.21.15.100:0/1020107289 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdc64078bc0 0x7fdc6407b0d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:25.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.351+0000 7fdc89a00640 1 --2- 172.21.15.100:0/1020107289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc840d15d0 0x7fdc8414a3c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:25.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.351+0000 7fdc89a00640 1 --2- 172.21.15.100:0/1020107289 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc840b9780 0x7fdc841f99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:25.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.351+0000 7fdc89a00640 1 --2- 172.21.15.100:0/1020107289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc840b3720 0x7fdc841f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:25.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.351+0000 7fdc89a00640 1 -- 172.21.15.100:0/1020107289 >> 172.21.15.100:0/1020107289 conn(0x7fdc840d4680 msgr2=0x7fdc8415ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:25.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.351+0000 7fdc89a00640 1 -- 172.21.15.100:0/1020107289 shutdown_connections 2024-09-17T04:24:25.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.351+0000 7fdc89a00640 1 -- 172.21.15.100:0/1020107289 wait complete. 2024-09-17T04:24:25.372 INFO:teuthology.orchestra.run.smithi100.stdout:[{"version":31,"timestamp":"2024-09-17T04:24:02.551210+0000","name":"","changes":[{"name":"osd.7/osd_mclock_max_capacity_iops_ssd","new_value":"3467.457298"}]}] 2024-09-17T04:24:25.372 INFO:tasks.ceph_manager:config epoch is 31 2024-09-17T04:24:25.372 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-17T04:24:25.373 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-17T04:24:25.373 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-17T04:24:25.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.590+0000 7fe9e3a00640 1 -- 172.21.15.100:0/1292038996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9dc15d760 msgr2=0x7fe9dc1535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:25.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.590+0000 7fe9e3a00640 1 --2- 172.21.15.100:0/1292038996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9dc15d760 0x7fe9dc1535c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc0b3100 tx=0x7fe9cc02a560 comp rx=0 tx=0).stop 2024-09-17T04:24:25.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.590+0000 7fe9e3a00640 1 -- 172.21.15.100:0/1292038996 shutdown_connections 2024-09-17T04:24:25.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.590+0000 7fe9e3a00640 1 --2- 172.21.15.100:0/1292038996 >> v2:172.21.15.184:3300/0 conn(0x7fe9dc0b3550 0x7fe9dc0b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:25.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.590+0000 7fe9e3a00640 1 --2- 172.21.15.100:0/1292038996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9dc15d760 0x7fe9dc1535c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:25.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.590+0000 7fe9e3a00640 1 -- 172.21.15.100:0/1292038996 >> 172.21.15.100:0/1292038996 conn(0x7fe9dc06d730 msgr2=0x7fe9dc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:25.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.590+0000 7fe9e3a00640 1 -- 172.21.15.100:0/1292038996 shutdown_connections 2024-09-17T04:24:25.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.590+0000 7fe9e3a00640 1 -- 172.21.15.100:0/1292038996 wait complete. 2024-09-17T04:24:25.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.592+0000 7fe9e3a00640 1 Processor -- start 2024-09-17T04:24:25.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.593+0000 7fe9e3a00640 1 -- start start 2024-09-17T04:24:25.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.594+0000 7fe9e3a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9dc0ad0e0 0x7fe9dc154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:25.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.594+0000 7fe9e3a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9dc0b3550 0x7fe9dc14c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:25.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.594+0000 7fe9e3a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9dc15d760 0x7fe9dc1f7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:25.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.594+0000 7fe9e3a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe9dc0b4da0 con 0x7fe9dc15d760 2024-09-17T04:24:25.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.594+0000 7fe9e3a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe9dc0b4c20 con 0x7fe9dc0ad0e0 2024-09-17T04:24:25.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.594+0000 7fe9e3a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe9dc0b3ea0 con 0x7fe9dc0b3550 2024-09-17T04:24:25.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.594+0000 7fe9e1a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9dc15d760 0x7fe9dc1f7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:25.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.594+0000 7fe9e1a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9dc15d760 0x7fe9dc1f7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47912/0 (socket says 172.21.15.100:47912) 2024-09-17T04:24:25.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.594+0000 7fe9e1a00640 1 -- 172.21.15.100:0/2577067103 learned_addr learned my addr 172.21.15.100:0/2577067103 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:25.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.594+0000 7fe9e1000640 1 --2- 172.21.15.100:0/2577067103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9dc0ad0e0 0x7fe9dc154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:25.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.594+0000 7fe9d3e00640 1 --2- 172.21.15.100:0/2577067103 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9dc0b3550 0x7fe9dc14c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:25.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.595+0000 7fe9e1a00640 1 -- 172.21.15.100:0/2577067103 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9dc0b3550 msgr2=0x7fe9dc14c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:25.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.595+0000 7fe9e1a00640 1 --2- 172.21.15.100:0/2577067103 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9dc0b3550 0x7fe9dc14c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:25.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.595+0000 7fe9e1a00640 1 -- 172.21.15.100:0/2577067103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9dc0ad0e0 msgr2=0x7fe9dc154e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:25.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.595+0000 7fe9e1a00640 1 --2- 172.21.15.100:0/2577067103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9dc0ad0e0 0x7fe9dc154e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:25.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.595+0000 7fe9e1a00640 1 -- 172.21.15.100:0/2577067103 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9dc14b340 con 0x7fe9dc15d760 2024-09-17T04:24:25.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.595+0000 7fe9d3e00640 1 --2- 172.21.15.100:0/2577067103 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9dc0b3550 0x7fe9dc14c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:24:25.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.595+0000 7fe9e1a00640 1 --2- 172.21.15.100:0/2577067103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9dc15d760 0x7fe9dc1f7780 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400dac0 tx=0x7fe9d400df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:25.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.595+0000 7fe9d9600640 1 -- 172.21.15.100:0/2577067103 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4017070 con 0x7fe9dc15d760 2024-09-17T04:24:25.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.595+0000 7fe9d9600640 1 -- 172.21.15.100:0/2577067103 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9d4003970 con 0x7fe9dc15d760 2024-09-17T04:24:25.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.595+0000 7fe9d9600640 1 -- 172.21.15.100:0/2577067103 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d400be60 con 0x7fe9dc15d760 2024-09-17T04:24:25.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.595+0000 7fe9e3a00640 1 -- 172.21.15.100:0/2577067103 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9dc14b620 con 0x7fe9dc15d760 2024-09-17T04:24:25.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.595+0000 7fe9e3a00640 1 -- 172.21.15.100:0/2577067103 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9dc14bbe0 con 0x7fe9dc15d760 2024-09-17T04:24:25.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.596+0000 7fe9e3a00640 1 -- 172.21.15.100:0/2577067103 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9dc0b4360 con 0x7fe9dc15d760 2024-09-17T04:24:25.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.597+0000 7fe9d9600640 1 -- 172.21.15.100:0/2577067103 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe9d4010460 con 0x7fe9dc15d760 2024-09-17T04:24:25.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.598+0000 7fe9d9600640 1 --2- 172.21.15.100:0/2577067103 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe9b8078b90 0x7fe9b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:25.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.599+0000 7fe9d9600640 1 -- 172.21.15.100:0/2577067103 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe9d409aff0 con 0x7fe9dc15d760 2024-09-17T04:24:25.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.599+0000 7fe9e1000640 1 --2- 172.21.15.100:0/2577067103 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe9b8078b90 0x7fe9b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:25.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.600+0000 7fe9e1000640 1 --2- 172.21.15.100:0/2577067103 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe9b8078b90 0x7fe9b807b0a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc033000 tx=0x7fe9cc004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:25.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.603+0000 7fe9d9600640 1 -- 172.21.15.100:0/2577067103 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe9d4062e30 con 0x7fe9dc15d760 2024-09-17T04:24:25.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.835+0000 7fe9e3a00640 1 -- 172.21.15.100:0/2577067103 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fe9dc152f90 con 0x7fe9dc15d760 2024-09-17T04:24:25.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.838+0000 7fe9d9600640 1 -- 172.21.15.100:0/2577067103 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+194722 (secure 0 0 0) 0x7fe9d4067860 con 0x7fe9dc15d760 2024-09-17T04:24:25.841 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:25.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.842+0000 7fe9e3a00640 1 -- 172.21.15.100:0/2577067103 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe9b8078b90 msgr2=0x7fe9b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:25.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.842+0000 7fe9e3a00640 1 --2- 172.21.15.100:0/2577067103 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe9b8078b90 0x7fe9b807b0a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc033000 tx=0x7fe9cc004730 comp rx=0 tx=0).stop 2024-09-17T04:24:25.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.842+0000 7fe9e3a00640 1 -- 172.21.15.100:0/2577067103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9dc15d760 msgr2=0x7fe9dc1f7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:25.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.842+0000 7fe9e3a00640 1 --2- 172.21.15.100:0/2577067103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9dc15d760 0x7fe9dc1f7780 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400dac0 tx=0x7fe9d400df80 comp rx=0 tx=0).stop 2024-09-17T04:24:25.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.842+0000 7fe9e3a00640 1 -- 172.21.15.100:0/2577067103 shutdown_connections 2024-09-17T04:24:25.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.842+0000 7fe9e3a00640 1 --2- 172.21.15.100:0/2577067103 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe9b8078b90 0x7fe9b807b0a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:25.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.842+0000 7fe9e3a00640 1 --2- 172.21.15.100:0/2577067103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9dc15d760 0x7fe9dc1f7780 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:25.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.842+0000 7fe9e3a00640 1 --2- 172.21.15.100:0/2577067103 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9dc0b3550 0x7fe9dc14c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:25.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.842+0000 7fe9e3a00640 1 --2- 172.21.15.100:0/2577067103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9dc0ad0e0 0x7fe9dc154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:25.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.842+0000 7fe9e3a00640 1 -- 172.21.15.100:0/2577067103 >> 172.21.15.100:0/2577067103 conn(0x7fe9dc06d730 msgr2=0x7fe9dc0b13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:25.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.842+0000 7fe9e3a00640 1 -- 172.21.15.100:0/2577067103 shutdown_connections 2024-09-17T04:24:25.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:25.842+0000 7fe9e3a00640 1 -- 172.21.15.100:0/2577067103 wait complete. 2024-09-17T04:24:25.866 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":5,"flags":0,"active_gid":4103,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":4096933233},{"type":"v1","addr":"172.21.15.184:6801","nonce":4096933233}]},"active_addr":"172.21.15.184:6801/4096933233","active_change":"2024-09-17T04:23:51.951168+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4108,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.184:0","nonce":2967819727}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.184:0","nonce":3031882921}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.184:0","nonce":4274574405}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.184:0","nonce":1536258390}]}]} 2024-09-17T04:24:25.870 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-17T04:24:25.871 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-17T04:24:25.871 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:24:26.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.091+0000 7faf4c200640 1 -- 172.21.15.100:0/2928607099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf4415d760 msgr2=0x7faf441535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:26.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.091+0000 7faf4c200640 1 --2- 172.21.15.100:0/2928607099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf4415d760 0x7faf441535c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7faf38005fd0 tx=0x7faf3802a540 comp rx=0 tx=0).stop 2024-09-17T04:24:26.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.091+0000 7faf4c200640 1 -- 172.21.15.100:0/2928607099 shutdown_connections 2024-09-17T04:24:26.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.091+0000 7faf4c200640 1 --2- 172.21.15.100:0/2928607099 >> v2:172.21.15.184:3300/0 conn(0x7faf440ad0e0 0x7faf440ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:26.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.091+0000 7faf4c200640 1 --2- 172.21.15.100:0/2928607099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf4415d760 0x7faf441535c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:26.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.092+0000 7faf4c200640 1 -- 172.21.15.100:0/2928607099 >> 172.21.15.100:0/2928607099 conn(0x7faf4406d730 msgr2=0x7faf440b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:26.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.092+0000 7faf4c200640 1 -- 172.21.15.100:0/2928607099 shutdown_connections 2024-09-17T04:24:26.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.092+0000 7faf4c200640 1 -- 172.21.15.100:0/2928607099 wait complete. 2024-09-17T04:24:26.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.094+0000 7faf4c200640 1 Processor -- start 2024-09-17T04:24:26.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.094+0000 7faf4c200640 1 -- start start 2024-09-17T04:24:26.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.095+0000 7faf4c200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf440ad0e0 0x7faf44099630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:26.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.095+0000 7faf4c200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf4415d760 0x7faf441f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:26.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.095+0000 7faf4c200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf44160310 0x7faf441f7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:26.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.095+0000 7faf4c200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faf4406f220 con 0x7faf4415d760 2024-09-17T04:24:26.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.095+0000 7faf4c200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faf4406f0a0 con 0x7faf44160310 2024-09-17T04:24:26.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.095+0000 7faf4c200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faf440b4c20 con 0x7faf440ad0e0 2024-09-17T04:24:26.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.096+0000 7faf48e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf4415d760 0x7faf441f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:26.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.096+0000 7faf48e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf4415d760 0x7faf441f6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47926/0 (socket says 172.21.15.100:47926) 2024-09-17T04:24:26.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.096+0000 7faf48e00640 1 -- 172.21.15.100:0/2386597212 learned_addr learned my addr 172.21.15.100:0/2386597212 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:26.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.096+0000 7faf49800640 1 --2- 172.21.15.100:0/2386597212 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf440ad0e0 0x7faf44099630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:26.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.096+0000 7faf4a200640 1 --2- 172.21.15.100:0/2386597212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf44160310 0x7faf441f7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:26.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.096+0000 7faf48e00640 1 -- 172.21.15.100:0/2386597212 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf440ad0e0 msgr2=0x7faf44099630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:26.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.096+0000 7faf48e00640 1 --2- 172.21.15.100:0/2386597212 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf440ad0e0 0x7faf44099630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:26.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.096+0000 7faf48e00640 1 -- 172.21.15.100:0/2386597212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf44160310 msgr2=0x7faf441f7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:26.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.096+0000 7faf48e00640 1 --2- 172.21.15.100:0/2386597212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf44160310 0x7faf441f7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:26.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.096+0000 7faf48e00640 1 -- 172.21.15.100:0/2386597212 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf44142440 con 0x7faf4415d760 2024-09-17T04:24:26.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.096+0000 7faf48e00640 1 --2- 172.21.15.100:0/2386597212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf4415d760 0x7faf441f6ec0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7faf30009fd0 tx=0x7faf3000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:26.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.097+0000 7faf41600640 1 -- 172.21.15.100:0/2386597212 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3000ccc0 con 0x7faf4415d760 2024-09-17T04:24:26.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.097+0000 7faf41600640 1 -- 172.21.15.100:0/2386597212 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf3000ee70 con 0x7faf4415d760 2024-09-17T04:24:26.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.097+0000 7faf4c200640 1 -- 172.21.15.100:0/2386597212 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf44142720 con 0x7faf4415d760 2024-09-17T04:24:26.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.097+0000 7faf41600640 1 -- 172.21.15.100:0/2386597212 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3000ce60 con 0x7faf4415d760 2024-09-17T04:24:26.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.097+0000 7faf4c200640 1 -- 172.21.15.100:0/2386597212 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf44142ce0 con 0x7faf4415d760 2024-09-17T04:24:26.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.098+0000 7faf4c200640 1 -- 172.21.15.100:0/2386597212 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf441f7cd0 con 0x7faf4415d760 2024-09-17T04:24:26.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.099+0000 7faf41600640 1 -- 172.21.15.100:0/2386597212 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faf30004280 con 0x7faf4415d760 2024-09-17T04:24:26.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.100+0000 7faf41600640 1 --2- 172.21.15.100:0/2386597212 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faf24078bc0 0x7faf2407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:26.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.100+0000 7faf41600640 1 -- 172.21.15.100:0/2386597212 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7faf3009b520 con 0x7faf4415d760 2024-09-17T04:24:26.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.100+0000 7faf49800640 1 --2- 172.21.15.100:0/2386597212 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faf24078bc0 0x7faf2407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:26.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.101+0000 7faf49800640 1 --2- 172.21.15.100:0/2386597212 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faf24078bc0 0x7faf2407b0d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faf38033000 tx=0x7faf38004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:26.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.104+0000 7faf41600640 1 -- 172.21.15.100:0/2386597212 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7faf30063300 con 0x7faf4415d760 2024-09-17T04:24:26.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.288+0000 7faf4c200640 1 -- 172.21.15.100:0/2386597212 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faf44160fa0 con 0x7faf4415d760 2024-09-17T04:24:26.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.289+0000 7faf41600640 1 -- 172.21.15.100:0/2386597212 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16713 (secure 0 0 0) 0x7faf30067d30 con 0x7faf4415d760 2024-09-17T04:24:26.290 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:26.290 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":23,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:24:16.038948+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:24:12.826665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T04:24:13.485446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:24:26.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.292+0000 7faf4c200640 1 -- 172.21.15.100:0/2386597212 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faf24078bc0 msgr2=0x7faf2407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:26.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.292+0000 7faf4c200640 1 --2- 172.21.15.100:0/2386597212 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faf24078bc0 0x7faf2407b0d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faf38033000 tx=0x7faf38004730 comp rx=0 tx=0).stop 2024-09-17T04:24:26.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.292+0000 7faf4c200640 1 -- 172.21.15.100:0/2386597212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf4415d760 msgr2=0x7faf441f6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:26.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.292+0000 7faf4c200640 1 --2- 172.21.15.100:0/2386597212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf4415d760 0x7faf441f6ec0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7faf30009fd0 tx=0x7faf3000ec20 comp rx=0 tx=0).stop 2024-09-17T04:24:26.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.292+0000 7faf4c200640 1 -- 172.21.15.100:0/2386597212 shutdown_connections 2024-09-17T04:24:26.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.292+0000 7faf4c200640 1 --2- 172.21.15.100:0/2386597212 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faf24078bc0 0x7faf2407b0d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:26.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.292+0000 7faf4c200640 1 --2- 172.21.15.100:0/2386597212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf44160310 0x7faf441f7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:26.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.292+0000 7faf4c200640 1 --2- 172.21.15.100:0/2386597212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf4415d760 0x7faf441f6ec0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:26.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.292+0000 7faf4c200640 1 --2- 172.21.15.100:0/2386597212 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf440ad0e0 0x7faf44099630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:26.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.292+0000 7faf4c200640 1 -- 172.21.15.100:0/2386597212 >> 172.21.15.100:0/2386597212 conn(0x7faf4406d730 msgr2=0x7faf4415f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:26.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.292+0000 7faf4c200640 1 -- 172.21.15.100:0/2386597212 shutdown_connections 2024-09-17T04:24:26.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.292+0000 7faf4c200640 1 -- 172.21.15.100:0/2386597212 wait complete. 2024-09-17T04:24:26.315 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-17T04:24:26.315 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:24:26.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.535+0000 7f6501600640 1 -- 172.21.15.100:0/2100281526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64fc0aebc0 msgr2=0x7f64fc152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:26.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.535+0000 7f6501600640 1 --2- 172.21.15.100:0/2100281526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64fc0aebc0 0x7f64fc152920 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f64fc160420 tx=0x7f64ec02a4a0 comp rx=0 tx=0).stop 2024-09-17T04:24:26.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.535+0000 7f6501600640 1 -- 172.21.15.100:0/2100281526 shutdown_connections 2024-09-17T04:24:26.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.535+0000 7f6501600640 1 --2- 172.21.15.100:0/2100281526 >> v2:172.21.15.184:3300/0 conn(0x7f64fc15fae0 0x7f64fc0b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:26.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.535+0000 7f6501600640 1 --2- 172.21.15.100:0/2100281526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64fc0aebc0 0x7f64fc152920 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:26.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.535+0000 7f6501600640 1 -- 172.21.15.100:0/2100281526 >> 172.21.15.100:0/2100281526 conn(0x7f64fc06d730 msgr2=0x7f64fc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:26.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.535+0000 7f6501600640 1 -- 172.21.15.100:0/2100281526 shutdown_connections 2024-09-17T04:24:26.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.535+0000 7f6501600640 1 -- 172.21.15.100:0/2100281526 wait complete. 2024-09-17T04:24:26.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.537+0000 7f6501600640 1 Processor -- start 2024-09-17T04:24:26.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.537+0000 7f6501600640 1 -- start start 2024-09-17T04:24:26.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.539+0000 7f6501600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64fc0aebc0 0x7f64fc0ced00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:26.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.539+0000 7f6501600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64fc0b2750 0x7f64fc0cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:26.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.539+0000 7f6501600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64fc145800 0x7f64fc145c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:26.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.539+0000 7f6501600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc0b40d0 con 0x7f64fc0b2750 2024-09-17T04:24:26.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.539+0000 7f6501600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc0b3f50 con 0x7f64fc145800 2024-09-17T04:24:26.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.539+0000 7f6501600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f64fc0b4250 con 0x7f64fc0aebc0 2024-09-17T04:24:26.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.539+0000 7f64f2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64fc0b2750 0x7f64fc0cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:26.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.539+0000 7f64f2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64fc0b2750 0x7f64fc0cd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47948/0 (socket says 172.21.15.100:47948) 2024-09-17T04:24:26.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.539+0000 7f64f2000640 1 -- 172.21.15.100:0/1856003131 learned_addr learned my addr 172.21.15.100:0/1856003131 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:26.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.539+0000 7f64fb400640 1 --2- 172.21.15.100:0/1856003131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64fc145800 0x7f64fc145c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:26.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.539+0000 7f64faa00640 1 --2- 172.21.15.100:0/1856003131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64fc0aebc0 0x7f64fc0ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:26.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.540+0000 7f64f2000640 1 -- 172.21.15.100:0/1856003131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64fc0aebc0 msgr2=0x7f64fc0ced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:26.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.540+0000 7f64f2000640 1 --2- 172.21.15.100:0/1856003131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64fc0aebc0 0x7f64fc0ced00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:26.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.540+0000 7f64f2000640 1 -- 172.21.15.100:0/1856003131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64fc145800 msgr2=0x7f64fc145c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:26.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.540+0000 7f64f2000640 1 --2- 172.21.15.100:0/1856003131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64fc145800 0x7f64fc145c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:26.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.540+0000 7f64f2000640 1 -- 172.21.15.100:0/1856003131 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc146150 con 0x7f64fc0b2750 2024-09-17T04:24:26.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.540+0000 7f64f2000640 1 --2- 172.21.15.100:0/1856003131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64fc0b2750 0x7f64fc0cd930 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f64e400c3c0 tx=0x7f64e400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:26.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.540+0000 7f64f3400640 1 -- 172.21.15.100:0/1856003131 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4017070 con 0x7f64fc0b2750 2024-09-17T04:24:26.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.540+0000 7f64f3400640 1 -- 172.21.15.100:0/1856003131 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64e40035f0 con 0x7f64fc0b2750 2024-09-17T04:24:26.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.540+0000 7f64f3400640 1 -- 172.21.15.100:0/1856003131 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4003a40 con 0x7f64fc0b2750 2024-09-17T04:24:26.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.540+0000 7f6501600640 1 -- 172.21.15.100:0/1856003131 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64fc1f56d0 con 0x7f64fc0b2750 2024-09-17T04:24:26.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.540+0000 7f6501600640 1 -- 172.21.15.100:0/1856003131 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64fc0cdf40 con 0x7f64fc0b2750 2024-09-17T04:24:26.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.541+0000 7f6501600640 1 -- 172.21.15.100:0/1856003131 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64fc0b4770 con 0x7f64fc0b2750 2024-09-17T04:24:26.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.543+0000 7f64f3400640 1 -- 172.21.15.100:0/1856003131 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f64e401a070 con 0x7f64fc0b2750 2024-09-17T04:24:26.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.543+0000 7f64f3400640 1 --2- 172.21.15.100:0/1856003131 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f64d8078b90 0x7f64d807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:26.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.544+0000 7f64f3400640 1 -- 172.21.15.100:0/1856003131 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f64e409a060 con 0x7f64fc0b2750 2024-09-17T04:24:26.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.544+0000 7f64faa00640 1 --2- 172.21.15.100:0/1856003131 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f64d8078b90 0x7f64d807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:26.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.545+0000 7f64faa00640 1 --2- 172.21.15.100:0/1856003131 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f64d8078b90 0x7f64d807b0a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f64ec034000 tx=0x7f64ec004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:26.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.548+0000 7f64f3400640 1 -- 172.21.15.100:0/1856003131 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f64e4061db0 con 0x7f64fc0b2750 2024-09-17T04:24:26.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.732+0000 7f6501600640 1 -- 172.21.15.100:0/1856003131 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64fc15ec00 con 0x7f64fc0b2750 2024-09-17T04:24:26.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.733+0000 7f64f3400640 1 -- 172.21.15.100:0/1856003131 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16713 (secure 0 0 0) 0x7f64e40667e0 con 0x7f64fc0b2750 2024-09-17T04:24:26.734 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:26.734 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":23,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:24:16.038948+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:24:12.826665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T04:24:13.485446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:24:26.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.735+0000 7f6501600640 1 -- 172.21.15.100:0/1856003131 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f64d8078b90 msgr2=0x7f64d807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:26.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.735+0000 7f6501600640 1 --2- 172.21.15.100:0/1856003131 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f64d8078b90 0x7f64d807b0a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f64ec034000 tx=0x7f64ec004730 comp rx=0 tx=0).stop 2024-09-17T04:24:26.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.736+0000 7f6501600640 1 -- 172.21.15.100:0/1856003131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64fc0b2750 msgr2=0x7f64fc0cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:26.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.736+0000 7f6501600640 1 --2- 172.21.15.100:0/1856003131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64fc0b2750 0x7f64fc0cd930 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f64e400c3c0 tx=0x7f64e400c880 comp rx=0 tx=0).stop 2024-09-17T04:24:26.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.736+0000 7f6501600640 1 -- 172.21.15.100:0/1856003131 shutdown_connections 2024-09-17T04:24:26.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.736+0000 7f6501600640 1 --2- 172.21.15.100:0/1856003131 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f64d8078b90 0x7f64d807b0a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:26.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.736+0000 7f6501600640 1 --2- 172.21.15.100:0/1856003131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64fc145800 0x7f64fc145c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:26.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.736+0000 7f6501600640 1 --2- 172.21.15.100:0/1856003131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64fc0b2750 0x7f64fc0cd930 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:26.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.736+0000 7f6501600640 1 --2- 172.21.15.100:0/1856003131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64fc0aebc0 0x7f64fc0ced00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:26.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.736+0000 7f6501600640 1 -- 172.21.15.100:0/1856003131 >> 172.21.15.100:0/1856003131 conn(0x7f64fc06d730 msgr2=0x7f64fc0d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:26.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.736+0000 7f6501600640 1 -- 172.21.15.100:0/1856003131 shutdown_connections 2024-09-17T04:24:26.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:26.736+0000 7f6501600640 1 -- 172.21.15.100:0/1856003131 wait complete. 2024-09-17T04:24:26.757 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T04:24:26.758 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-17T04:24:26.758 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T04:24:26.759 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-17T04:24:26.759 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-17T04:24:26.760 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-17T04:24:26.760 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-17T04:24:26.760 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-17T04:24:27.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.154+0000 7f8b4f200640 1 -- 172.21.15.100:0/4098175861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b5006ee00 msgr2=0x7f8b501623a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.154+0000 7f8b4f200640 1 --2- 172.21.15.100:0/4098175861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b5006ee00 0x7f8b501623a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009830 tx=0x7f8b3802a520 comp rx=0 tx=0).stop 2024-09-17T04:24:27.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.154+0000 7f188a800640 1 -- 172.21.15.100:0/2466886319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18840b4060 msgr2=0x7f18840b4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.154+0000 7f188a800640 1 --2- 172.21.15.100:0/2466886319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18840b4060 0x7f18840b4470 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1874005fd0 tx=0x7f187402a5b0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.155+0000 7f8b4f200640 1 -- 172.21.15.100:0/4098175861 shutdown_connections 2024-09-17T04:24:27.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.155+0000 7f8b4f200640 1 --2- 172.21.15.100:0/4098175861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b5006ee00 0x7f8b501623a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.155+0000 7f8b4f200640 1 -- 172.21.15.100:0/4098175861 >> 172.21.15.100:0/4098175861 conn(0x7f8b500b1c60 msgr2=0x7f8b500b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:27.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.155+0000 7f188a800640 1 -- 172.21.15.100:0/2466886319 shutdown_connections 2024-09-17T04:24:27.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.155+0000 7f188a800640 1 --2- 172.21.15.100:0/2466886319 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18840ae030 0x7f18840ae440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.155+0000 7f188a800640 1 --2- 172.21.15.100:0/2466886319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18840b4060 0x7f18840b4470 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.155+0000 7f188a800640 1 -- 172.21.15.100:0/2466886319 >> 172.21.15.100:0/2466886319 conn(0x7f18840b2650 msgr2=0x7f18840b2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:27.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.155+0000 7f8b4f200640 1 -- 172.21.15.100:0/4098175861 shutdown_connections 2024-09-17T04:24:27.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.156+0000 7f188a800640 1 -- 172.21.15.100:0/2466886319 shutdown_connections 2024-09-17T04:24:27.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.155+0000 7f8b4f200640 1 -- 172.21.15.100:0/4098175861 wait complete. 2024-09-17T04:24:27.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.156+0000 7f188a800640 1 -- 172.21.15.100:0/2466886319 wait complete. 2024-09-17T04:24:27.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.160+0000 7f8b4f200640 1 Processor -- start 2024-09-17T04:24:27.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.160+0000 7f188a800640 1 Processor -- start 2024-09-17T04:24:27.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.160+0000 7f8b4f200640 1 -- start start 2024-09-17T04:24:27.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.160+0000 7fdb97e00640 1 -- 172.21.15.100:0/3312629277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb90005c60 msgr2=0x7fdb90036430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.160+0000 7fdb97e00640 1 --2- 172.21.15.100:0/3312629277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb90005c60 0x7fdb90036430 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdb90034180 tx=0x7fdb8802a020 comp rx=0 tx=0).stop 2024-09-17T04:24:27.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.160+0000 7f188a800640 1 -- start start 2024-09-17T04:24:27.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.161+0000 7fdb97e00640 1 -- 172.21.15.100:0/3312629277 shutdown_connections 2024-09-17T04:24:27.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.161+0000 7fdb97e00640 1 --2- 172.21.15.100:0/3312629277 >> v2:172.21.15.184:3300/0 conn(0x7fdb90036970 0x7fdb9001c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.161+0000 7fdb97e00640 1 --2- 172.21.15.100:0/3312629277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb90005c60 0x7fdb90036430 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.161+0000 7fdb97e00640 1 --2- 172.21.15.100:0/3312629277 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb90009ba0 0x7fdb900df180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.161+0000 7fdb97e00640 1 -- 172.21.15.100:0/3312629277 >> 172.21.15.100:0/3312629277 conn(0x7fdb90051420 msgr2=0x7fdb90051860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:27.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.162+0000 7fdb97e00640 1 -- 172.21.15.100:0/3312629277 shutdown_connections 2024-09-17T04:24:27.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.162+0000 7fdb97e00640 1 -- 172.21.15.100:0/3312629277 wait complete. 2024-09-17T04:24:27.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.162+0000 7f188a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18840ae030 0x7f1884145d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.163+0000 7f188a800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18840b4060 0x7f18841f9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.163+0000 7f188a800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1884155a80 0x7f18841f95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.163+0000 7f188a800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f18840ae980 con 0x7f18840ae030 2024-09-17T04:24:27.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.163+0000 7f188a800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f18840af8c0 con 0x7f18840b4060 2024-09-17T04:24:27.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.163+0000 7f188a800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f18840aeb00 con 0x7f1884155a80 2024-09-17T04:24:27.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.163+0000 7f1883400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18840ae030 0x7f1884145d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.163+0000 7f1883400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18840ae030 0x7f1884145d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47998/0 (socket says 172.21.15.100:47998) 2024-09-17T04:24:27.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.163+0000 7f1883400640 1 -- 172.21.15.100:0/2808994528 learned_addr learned my addr 172.21.15.100:0/2808994528 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:27.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.163+0000 7f1882a00640 1 --2- 172.21.15.100:0/2808994528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18840b4060 0x7f18841f9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.163+0000 7f1883e00640 1 --2- 172.21.15.100:0/2808994528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1884155a80 0x7f18841f95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.164+0000 7f1882a00640 1 -- 172.21.15.100:0/2808994528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1884155a80 msgr2=0x7f18841f95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.165+0000 7f8b4f200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b5006ee00 0x7f8b50159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.165+0000 7f8b4f200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b5006f210 0x7f8b50148f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.165+0000 7f8b4f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b501628e0 0x7f8b5014e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.166+0000 7f8b4de00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b5006ee00 0x7f8b50159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.166+0000 7f8b4de00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b5006ee00 0x7f8b50159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.100:36272/0 (socket says 172.21.15.100:36272) 2024-09-17T04:24:27.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.166+0000 7f8b45400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b5006f210 0x7f8b50148f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.166+0000 7f8b45400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b5006f210 0x7f8b50148f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.100:53388/0 (socket says 172.21.15.100:53388) 2024-09-17T04:24:27.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.166+0000 7f8b4e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b501628e0 0x7f8b5014e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.166+0000 7f8b4f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8b50154350 con 0x7f8b501628e0 2024-09-17T04:24:27.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.164+0000 7f1882a00640 1 --2- 172.21.15.100:0/2808994528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1884155a80 0x7f18841f95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.164+0000 7f1882a00640 1 -- 172.21.15.100:0/2808994528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18840ae030 msgr2=0x7f1884145d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.164+0000 7f1882a00640 1 --2- 172.21.15.100:0/2808994528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18840ae030 0x7f1884145d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.164+0000 7f1882a00640 1 -- 172.21.15.100:0/2808994528 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1884156ac0 con 0x7f18840b4060 2024-09-17T04:24:27.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.165+0000 7f1883e00640 1 --2- 172.21.15.100:0/2808994528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1884155a80 0x7f18841f95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:24:27.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.165+0000 7f1883400640 1 --2- 172.21.15.100:0/2808994528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18840ae030 0x7f1884145d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:24:27.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.166+0000 7f1882a00640 1 --2- 172.21.15.100:0/2808994528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18840b4060 0x7f18841f9060 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1884069da0 tx=0x7f1874031690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.167+0000 7f187b400640 1 -- 172.21.15.100:0/2808994528 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1874041070 con 0x7f18840b4060 2024-09-17T04:24:27.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.167+0000 7f187b400640 1 -- 172.21.15.100:0/2808994528 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1874031cb0 con 0x7f18840b4060 2024-09-17T04:24:27.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.167+0000 7f187b400640 1 -- 172.21.15.100:0/2808994528 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187403b6c0 con 0x7f18840b4060 2024-09-17T04:24:27.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.167+0000 7f188a800640 1 -- 172.21.15.100:0/2808994528 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1884158e50 con 0x7f18840b4060 2024-09-17T04:24:27.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.167+0000 7f8074a00640 1 -- 172.21.15.100:0/2733067115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8070073b00 msgr2=0x7f80700b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.166+0000 7f8b4f200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8b501544d0 con 0x7f8b5006ee00 2024-09-17T04:24:27.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.166+0000 7f8b4f200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8b50154650 con 0x7f8b5006f210 2024-09-17T04:24:27.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.166+0000 7f8b4de00640 1 -- 172.21.15.100:0/2946428688 learned_addr learned my addr 172.21.15.100:0/2946428688 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:27.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.167+0000 7f8b4de00640 1 -- 172.21.15.100:0/2946428688 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b5006f210 msgr2=0x7f8b50148f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.167+0000 7f8b4de00640 1 --2- 172.21.15.100:0/2946428688 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b5006f210 0x7f8b50148f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.167+0000 7f188a800640 1 -- 172.21.15.100:0/2808994528 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1884157060 con 0x7f18840b4060 2024-09-17T04:24:27.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.167+0000 7f8b4de00640 1 -- 172.21.15.100:0/2946428688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b501628e0 msgr2=0x7f8b5014e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.167+0000 7f8074a00640 1 --2- 172.21.15.100:0/2733067115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8070073b00 0x7f80700b3520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f807006f940 tx=0x7f806002a540 comp rx=0 tx=0).stop 2024-09-17T04:24:27.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.168+0000 7f8074a00640 1 -- 172.21.15.100:0/2733067115 shutdown_connections 2024-09-17T04:24:27.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.168+0000 7f8074a00640 1 --2- 172.21.15.100:0/2733067115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8070073b00 0x7f80700b3520 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.168+0000 7f8074a00640 1 --2- 172.21.15.100:0/2733067115 >> v2:172.21.15.100:3300/0 conn(0x7f80701518d0 0x7f80700735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.168+0000 7f8074a00640 1 -- 172.21.15.100:0/2733067115 >> 172.21.15.100:0/2733067115 conn(0x7f80700b1e40 msgr2=0x7f80700b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:27.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.169+0000 7f8074a00640 1 -- 172.21.15.100:0/2733067115 shutdown_connections 2024-09-17T04:24:27.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.168+0000 7f8b4de00640 1 --2- 172.21.15.100:0/2946428688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b501628e0 0x7f8b5014e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.168+0000 7f8b4de00640 1 -- 172.21.15.100:0/2946428688 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b50149ae0 con 0x7f8b5006ee00 2024-09-17T04:24:27.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.169+0000 7f8b4e800640 1 --2- 172.21.15.100:0/2946428688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b501628e0 0x7f8b5014e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:24:27.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.169+0000 7f8074a00640 1 -- 172.21.15.100:0/2733067115 wait complete. 2024-09-17T04:24:27.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.169+0000 7f8b4de00640 1 --2- 172.21.15.100:0/2946428688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b5006ee00 0x7f8b50159470 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8b38000c00 tx=0x7f8b38033ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.169+0000 7fdb97e00640 1 Processor -- start 2024-09-17T04:24:27.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.169+0000 7fc887c00640 1 -- 172.21.15.100:0/994440194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8800aff10 msgr2=0x7fc8800b0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.169+0000 7f8b46a00640 1 -- 172.21.15.100:0/2946428688 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b38031b50 con 0x7f8b5006ee00 2024-09-17T04:24:27.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.170+0000 7f8b46a00640 1 -- 172.21.15.100:0/2946428688 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b38031de0 con 0x7f8b5006ee00 2024-09-17T04:24:27.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.170+0000 7f8b46a00640 1 -- 172.21.15.100:0/2946428688 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b38033330 con 0x7f8b5006ee00 2024-09-17T04:24:27.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.169+0000 7fdb97e00640 1 -- start start 2024-09-17T04:24:27.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.169+0000 7fc887c00640 1 --2- 172.21.15.100:0/994440194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8800aff10 0x7fc8800b0320 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc88006e9f0 tx=0x7fc87402a580 comp rx=0 tx=0).stop 2024-09-17T04:24:27.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.170+0000 7fc887c00640 1 -- 172.21.15.100:0/994440194 shutdown_connections 2024-09-17T04:24:27.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.170+0000 7fc887c00640 1 --2- 172.21.15.100:0/994440194 >> v2:172.21.15.100:3300/0 conn(0x7fc8800b31e0 0x7fc8800b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.170+0000 7fc887c00640 1 --2- 172.21.15.100:0/994440194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8800aff10 0x7fc8800b0320 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.170+0000 7fc887c00640 1 -- 172.21.15.100:0/994440194 >> 172.21.15.100:0/994440194 conn(0x7fc8800d02f0 msgr2=0x7fc8800d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:27.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.170+0000 7fc887c00640 1 -- 172.21.15.100:0/994440194 shutdown_connections 2024-09-17T04:24:27.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.170+0000 7f8b4f200640 1 -- 172.21.15.100:0/2946428688 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b5014dc20 con 0x7f8b5006ee00 2024-09-17T04:24:27.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.170+0000 7f8b4f200640 1 -- 172.21.15.100:0/2946428688 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b5014e1e0 con 0x7f8b5006ee00 2024-09-17T04:24:27.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.171+0000 7f188a800640 1 -- 172.21.15.100:0/2808994528 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f18840af010 con 0x7f18840b4060 2024-09-17T04:24:27.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.170+0000 7fc887c00640 1 -- 172.21.15.100:0/994440194 wait complete. 2024-09-17T04:24:27.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.171+0000 7f187b400640 1 -- 172.21.15.100:0/2808994528 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1874044070 con 0x7f18840b4060 2024-09-17T04:24:27.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.172+0000 7f7642600640 1 -- 172.21.15.100:0/620140335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f763c0711c0 msgr2=0x7f763c0b3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.172+0000 7f9725000640 1 -- 172.21.15.100:0/2578647612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97200af3f0 msgr2=0x7f972015eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.172+0000 7f9725000640 1 --2- 172.21.15.100:0/2578647612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97200af3f0 0x7f972015eca0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f97200b5400 tx=0x7f970c02c2a0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.172+0000 7fdb97e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb90005c60 0x7fdb900dd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.172+0000 7fdb97e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb90009ba0 0x7fdb9016a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.172+0000 7fdb97e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb90036970 0x7fdb9016add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.172+0000 7fdb97e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9000bdb0 con 0x7fdb90005c60 2024-09-17T04:24:27.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.172+0000 7fdb97e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9000bc30 con 0x7fdb90009ba0 2024-09-17T04:24:27.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.172+0000 7fdb97e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdb9001caf0 con 0x7fdb90036970 2024-09-17T04:24:27.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.173+0000 7fdb96a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb90005c60 0x7fdb900dd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.173+0000 7f8074a00640 1 Processor -- start 2024-09-17T04:24:27.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.173+0000 7f9725000640 1 -- 172.21.15.100:0/2578647612 shutdown_connections 2024-09-17T04:24:27.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.173+0000 7f9725000640 1 --2- 172.21.15.100:0/2578647612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97200af3f0 0x7f972015eca0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.173+0000 7f9725000640 1 --2- 172.21.15.100:0/2578647612 >> v2:172.21.15.100:3300/0 conn(0x7f972015add0 0x7f972015e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.173+0000 7f9725000640 1 -- 172.21.15.100:0/2578647612 >> 172.21.15.100:0/2578647612 conn(0x7f97200742a0 msgr2=0x7f97200746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:27.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.173+0000 7f8074a00640 1 -- start start 2024-09-17T04:24:27.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.172+0000 7f7642600640 1 --2- 172.21.15.100:0/620140335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f763c0711c0 0x7f763c0b3d60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f763c0d13a0 tx=0x7f762c02a490 comp rx=0 tx=0).stop 2024-09-17T04:24:27.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.172+0000 7f7642600640 1 -- 172.21.15.100:0/620140335 shutdown_connections 2024-09-17T04:24:27.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.172+0000 7f7642600640 1 --2- 172.21.15.100:0/620140335 >> v2:172.21.15.100:3300/0 conn(0x7f763c0af920 0x7f763c0cfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.172+0000 7f7642600640 1 --2- 172.21.15.100:0/620140335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f763c0af510 0x7f763c0cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.172+0000 7f7642600640 1 --2- 172.21.15.100:0/620140335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f763c0711c0 0x7f763c0b3d60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.172+0000 7f7642600640 1 -- 172.21.15.100:0/620140335 >> 172.21.15.100:0/620140335 conn(0x7f763c06d3d0 msgr2=0x7f763c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:27.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.173+0000 7f8b4f200640 1 -- 172.21.15.100:0/2946428688 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8b501623a0 con 0x7f8b5006ee00 2024-09-17T04:24:27.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.173+0000 7fdb96a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb90005c60 0x7fdb900dd7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48044/0 (socket says 172.21.15.100:48044) 2024-09-17T04:24:27.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.173+0000 7fdb96a00640 1 -- 172.21.15.100:0/3969895035 learned_addr learned my addr 172.21.15.100:0/3969895035 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:27.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.173+0000 7fdb96000640 1 --2- 172.21.15.100:0/3969895035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb90009ba0 0x7fdb9016a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.173+0000 7fdb97400640 1 --2- 172.21.15.100:0/3969895035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb90036970 0x7fdb9016add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.173+0000 7f187b400640 1 --2- 172.21.15.100:0/2808994528 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1864078be0 0x7f186407b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.173+0000 7f187b400640 1 -- 172.21.15.100:0/2808994528 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f18740bb8d0 con 0x7f18840b4060 2024-09-17T04:24:27.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.174+0000 7f9725000640 1 -- 172.21.15.100:0/2578647612 shutdown_connections 2024-09-17T04:24:27.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.173+0000 7f7642600640 1 -- 172.21.15.100:0/620140335 shutdown_connections 2024-09-17T04:24:27.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.174+0000 7f187b400640 1 --2- 172.21.15.100:0/2808994528 >> [v2:172.21.15.100:6808/1254254975,v1:172.21.15.100:6809/1254254975] conn(0x7f1864080e50 0x7f18640832a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.174+0000 7f187b400640 1 -- 172.21.15.100:0/2808994528 --> [v2:172.21.15.100:6808/1254254975,v1:172.21.15.100:6809/1254254975] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1874004470 con 0x7f1864080e50 2024-09-17T04:24:27.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.174+0000 7f187b400640 1 -- 172.21.15.100:0/2808994528 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f18740c02a0 con 0x7f18840b4060 2024-09-17T04:24:27.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.174+0000 7f1883400640 1 --2- 172.21.15.100:0/2808994528 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1864078be0 0x7f186407b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.174+0000 7f1883e00640 1 --2- 172.21.15.100:0/2808994528 >> [v2:172.21.15.100:6808/1254254975,v1:172.21.15.100:6809/1254254975] conn(0x7f1864080e50 0x7f18640832a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.174+0000 7f7642600640 1 -- 172.21.15.100:0/620140335 wait complete. 2024-09-17T04:24:27.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.175+0000 7f1883400640 1 --2- 172.21.15.100:0/2808994528 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1864078be0 0x7f186407b0f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f18700018a0 tx=0x7f1870009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.174+0000 7fdb96a00640 1 -- 172.21.15.100:0/3969895035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb90036970 msgr2=0x7fdb9016add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.175+0000 7fdb96a00640 1 --2- 172.21.15.100:0/3969895035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb90036970 0x7fdb9016add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.175+0000 7fdb96a00640 1 -- 172.21.15.100:0/3969895035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb90009ba0 msgr2=0x7fdb9016a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.175+0000 7fdb96a00640 1 --2- 172.21.15.100:0/3969895035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb90009ba0 0x7fdb9016a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.175+0000 7fdb96a00640 1 -- 172.21.15.100:0/3969895035 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb900222d0 con 0x7fdb90005c60 2024-09-17T04:24:27.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.175+0000 7fdb97400640 1 --2- 172.21.15.100:0/3969895035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb90036970 0x7fdb9016add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:24:27.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.175+0000 7f8074a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8070070f50 0x7f807009baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.175+0000 7f8074a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8070073b00 0x7f80700b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.175+0000 7fc887c00640 1 Processor -- start 2024-09-17T04:24:27.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.175+0000 7f8074a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80701518d0 0x7f80701f48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.176+0000 7fc887c00640 1 -- start start 2024-09-17T04:24:27.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.176+0000 7f8074a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f80700b3be0 con 0x7f80701518d0 2024-09-17T04:24:27.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.176+0000 7f8074a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f80700b3a60 con 0x7f8070073b00 2024-09-17T04:24:27.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.176+0000 7f8074a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f80700b3d60 con 0x7f8070070f50 2024-09-17T04:24:27.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.176+0000 7f1883e00640 1 --2- 172.21.15.100:0/2808994528 >> [v2:172.21.15.100:6808/1254254975,v1:172.21.15.100:6809/1254254975] conn(0x7f1864080e50 0x7f18640832a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.176+0000 7f806f400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8070070f50 0x7f807009baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.176+0000 7fdb96000640 1 --2- 172.21.15.100:0/3969895035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb90009ba0 0x7fdb9016a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:24:27.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.176+0000 7f806ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8070073b00 0x7f80700b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.176+0000 7f806fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80701518d0 0x7f80701f48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.176+0000 7f806ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8070073b00 0x7f80700b11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.100:36290/0 (socket says 172.21.15.100:36290) 2024-09-17T04:24:27.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.176+0000 7f806ea00640 1 -- 172.21.15.100:0/1976844114 learned_addr learned my addr 172.21.15.100:0/1976844114 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:27.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.175+0000 7f8b46a00640 1 -- 172.21.15.100:0/2946428688 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8b380334d0 con 0x7f8b5006ee00 2024-09-17T04:24:27.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.177+0000 7f8b46a00640 1 --2- 172.21.15.100:0/2946428688 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8b2c078be0 0x7f8b2c07b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.176+0000 7f806ea00640 1 -- 172.21.15.100:0/1976844114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8070070f50 msgr2=0x7f807009baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.176+0000 7f806ea00640 1 --2- 172.21.15.100:0/1976844114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8070070f50 0x7f807009baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.177+0000 7f806ea00640 1 -- 172.21.15.100:0/1976844114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80701518d0 msgr2=0x7f80701f48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.177+0000 7f806ea00640 1 --2- 172.21.15.100:0/1976844114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80701518d0 0x7f80701f48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.177+0000 7f8b46a00640 1 -- 172.21.15.100:0/2946428688 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f8b380bbee0 con 0x7f8b5006ee00 2024-09-17T04:24:27.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.177+0000 7f8b45400640 1 --2- 172.21.15.100:0/2946428688 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8b2c078be0 0x7f8b2c07b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.177+0000 7f8b46a00640 1 --2- 172.21.15.100:0/2946428688 >> [v2:172.21.15.100:6816/1292500299,v1:172.21.15.100:6817/1292500299] conn(0x7f8b2c080eb0 0x7f8b2c083300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.177+0000 7f8b46a00640 1 -- 172.21.15.100:0/2946428688 --> [v2:172.21.15.100:6816/1292500299,v1:172.21.15.100:6817/1292500299] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8b38001d30 con 0x7f8b2c080eb0 2024-09-17T04:24:27.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.177+0000 7f8b46a00640 1 -- 172.21.15.100:0/2946428688 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8b380be630 con 0x7f8b5006ee00 2024-09-17T04:24:27.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.178+0000 7f187b400640 1 -- 172.21.15.100:0/2808994528 <== osd.1 v2:172.21.15.100:6808/1254254975 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1874004470 con 0x7f1864080e50 2024-09-17T04:24:27.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.178+0000 7fdb96a00640 1 --2- 172.21.15.100:0/3969895035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb90005c60 0x7fdb900dd7d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400be00 tx=0x7fdb84007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.177+0000 7f806ea00640 1 -- 172.21.15.100:0/1976844114 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f807009bfe0 con 0x7f8070073b00 2024-09-17T04:24:27.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.177+0000 7f806fe00640 1 --2- 172.21.15.100:0/1976844114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80701518d0 0x7f80701f48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:24:27.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.175+0000 7f9725000640 1 -- 172.21.15.100:0/2578647612 wait complete. 2024-09-17T04:24:27.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.179+0000 7fc887c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8800aff10 0x7fc88009dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.179+0000 7f806ea00640 1 --2- 172.21.15.100:0/1976844114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8070073b00 0x7f80700b11a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f805c00c970 tx=0x7f805c00ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.179+0000 7fc887c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8800b31e0 0x7fc8801571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.178+0000 7f8b4e800640 1 --2- 172.21.15.100:0/2946428688 >> [v2:172.21.15.100:6816/1292500299,v1:172.21.15.100:6817/1292500299] conn(0x7f8b2c080eb0 0x7f8b2c083300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.179+0000 7fc887c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8800b3b30 0x7fc8801f8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.179+0000 7fc887c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc8800b1510 con 0x7fc8800b3b30 2024-09-17T04:24:27.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.179+0000 7fdb8ea00640 1 -- 172.21.15.100:0/3969895035 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8401d620 con 0x7fdb90005c60 2024-09-17T04:24:27.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.179+0000 7fc887c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc8800b1390 con 0x7fc8800b31e0 2024-09-17T04:24:27.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.179+0000 7fc887c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc8800b5270 con 0x7fc8800aff10 2024-09-17T04:24:27.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.179+0000 7fdb8ea00640 1 -- 172.21.15.100:0/3969895035 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb84022070 con 0x7fdb90005c60 2024-09-17T04:24:27.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.179+0000 7fdb8ea00640 1 -- 172.21.15.100:0/3969895035 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8401da40 con 0x7fdb90005c60 2024-09-17T04:24:27.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.178+0000 7f8b45400640 1 --2- 172.21.15.100:0/2946428688 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8b2c078be0 0x7f8b2c07b0f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8b40007fd0 tx=0x7f8b40009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.179+0000 7fc885200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8800aff10 0x7fc88009dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.179+0000 7fc885200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8800aff10 0x7fc88009dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.100:53414/0 (socket says 172.21.15.100:53414) 2024-09-17T04:24:27.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.179+0000 7f8067400640 1 -- 172.21.15.100:0/1976844114 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c007bb0 con 0x7f8070073b00 2024-09-17T04:24:27.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.179+0000 7f8067400640 1 -- 172.21.15.100:0/1976844114 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f805c007e40 con 0x7f8070073b00 2024-09-17T04:24:27.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.179+0000 7f8067400640 1 -- 172.21.15.100:0/1976844114 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c00f6b0 con 0x7f8070073b00 2024-09-17T04:24:27.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.179+0000 7f8074a00640 1 -- 172.21.15.100:0/1976844114 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80700b16e0 con 0x7f8070073b00 2024-09-17T04:24:27.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.179+0000 7fc885200640 1 -- 172.21.15.100:0/1411818929 learned_addr learned my addr 172.21.15.100:0/1411818929 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:27.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.180+0000 7f9725000640 1 Processor -- start 2024-09-17T04:24:27.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.180+0000 7fc885200640 1 -- 172.21.15.100:0/1411818929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8800b31e0 msgr2=0x7fc8801571f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T04:24:27.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.180+0000 7fc885200640 1 --2- 172.21.15.100:0/1411818929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8800b31e0 0x7fc8801571f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.180+0000 7fc885200640 1 -- 172.21.15.100:0/1411818929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8800b3b30 msgr2=0x7fc8801f8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.180+0000 7fc885200640 1 --2- 172.21.15.100:0/1411818929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8800b3b30 0x7fc8801f8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.180+0000 7fc885200640 1 -- 172.21.15.100:0/1411818929 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc880160d30 con 0x7fc8800aff10 2024-09-17T04:24:27.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.180+0000 7fc885200640 1 --2- 172.21.15.100:0/1411818929 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8800aff10 0x7fc88009dcb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc874034040 tx=0x7fc874003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.180+0000 7f8b4e800640 1 --2- 172.21.15.100:0/2946428688 >> [v2:172.21.15.100:6816/1292500299,v1:172.21.15.100:6817/1292500299] conn(0x7f8b2c080eb0 0x7f8b2c083300 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.181+0000 7f7642600640 1 Processor -- start 2024-09-17T04:24:27.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.181+0000 7f7642600640 1 -- start start 2024-09-17T04:24:27.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.181+0000 7f8b46a00640 1 -- 172.21.15.100:0/2946428688 <== osd.0 v2:172.21.15.100:6816/1292500299 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8b38001d30 con 0x7f8b2c080eb0 2024-09-17T04:24:27.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.182+0000 7fdb97e00640 1 -- 172.21.15.100:0/3969895035 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb90168a60 con 0x7fdb90005c60 2024-09-17T04:24:27.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.181+0000 7fc87cc00640 1 -- 172.21.15.100:0/1411818929 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87403e030 con 0x7fc8800aff10 2024-09-17T04:24:27.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.181+0000 7fc87cc00640 1 -- 172.21.15.100:0/1411818929 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8740047a0 con 0x7fc8800aff10 2024-09-17T04:24:27.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.181+0000 7fc87cc00640 1 -- 172.21.15.100:0/1411818929 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc874035040 con 0x7fc8800aff10 2024-09-17T04:24:27.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.181+0000 7fc887c00640 1 -- 172.21.15.100:0/1411818929 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc880161010 con 0x7fc8800aff10 2024-09-17T04:24:27.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.181+0000 7fc887c00640 1 -- 172.21.15.100:0/1411818929 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8801f8130 con 0x7fc8800aff10 2024-09-17T04:24:27.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.180+0000 7f8074a00640 1 -- 172.21.15.100:0/1976844114 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f807009c7d0 con 0x7f8070073b00 2024-09-17T04:24:27.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.180+0000 7f8074a00640 1 -- 172.21.15.100:0/1976844114 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f80700b3520 con 0x7f8070073b00 2024-09-17T04:24:27.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.182+0000 7f9725000640 1 -- start start 2024-09-17T04:24:27.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.182+0000 7fc887c00640 1 -- 172.21.15.100:0/1411818929 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc85c001030 con 0x7fc8800aff10 2024-09-17T04:24:27.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.183+0000 7f7642600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f763c0711c0 0x7f763c0c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.183+0000 7f7642600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f763c0af510 0x7f763c151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.183+0000 7f7642600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f763c0af920 0x7f763c0cae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.183+0000 7f7642600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f763c070360 con 0x7f763c0af510 2024-09-17T04:24:27.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.183+0000 7f7642600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f763c0701e0 con 0x7f763c0af920 2024-09-17T04:24:27.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.183+0000 7f7642600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f763c0704e0 con 0x7f763c0711c0 2024-09-17T04:24:27.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.184+0000 7f8067400640 1 -- 172.21.15.100:0/1976844114 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f805c018400 con 0x7f8070073b00 2024-09-17T04:24:27.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.185+0000 7f763b400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f763c0711c0 0x7f763c0c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.185+0000 7f763b400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f763c0711c0 0x7f763c0c3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.100:53424/0 (socket says 172.21.15.100:53424) 2024-09-17T04:24:27.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.185+0000 7f763b400640 1 -- 172.21.15.100:0/215132568 learned_addr learned my addr 172.21.15.100:0/215132568 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:27.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.185+0000 7f763be00640 1 --2- 172.21.15.100:0/215132568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f763c0af920 0x7f763c0cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.185+0000 7f763aa00640 1 --2- 172.21.15.100:0/215132568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f763c0af510 0x7f763c151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.185+0000 7f763be00640 1 -- 172.21.15.100:0/215132568 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f763c0711c0 msgr2=0x7f763c0c3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.185+0000 7f763be00640 1 --2- 172.21.15.100:0/215132568 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f763c0711c0 0x7f763c0c3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.185+0000 7f763be00640 1 -- 172.21.15.100:0/215132568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f763c0af510 msgr2=0x7f763c151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.186+0000 7f763be00640 1 --2- 172.21.15.100:0/215132568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f763c0af510 0x7f763c151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.186+0000 7f763be00640 1 -- 172.21.15.100:0/215132568 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f763c1570d0 con 0x7f763c0af920 2024-09-17T04:24:27.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.184+0000 7fdb97e00640 1 -- 172.21.15.100:0/3969895035 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb90168d00 con 0x7fdb90005c60 2024-09-17T04:24:27.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.186+0000 7f8067400640 1 --2- 172.21.15.100:0/1976844114 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8050078c60 0x7f805007b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.186+0000 7f763aa00640 1 --2- 172.21.15.100:0/215132568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f763c0af510 0x7f763c151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:24:27.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.186+0000 7f763be00640 1 --2- 172.21.15.100:0/215132568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f763c0af920 0x7f763c0cae90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7634012f60 tx=0x7f7634012f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.186+0000 7f8067400640 1 -- 172.21.15.100:0/1976844114 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f805c09b680 con 0x7f8070073b00 2024-09-17T04:24:27.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.186+0000 7f8067400640 1 --2- 172.21.15.100:0/1976844114 >> [v2:172.21.15.184:6802/3708129583,v1:172.21.15.184:6803/3708129583] conn(0x7f8050080f30 0x7f8050083380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.186+0000 7f8067400640 1 -- 172.21.15.100:0/1976844114 --> [v2:172.21.15.184:6802/3708129583,v1:172.21.15.184:6803/3708129583] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f805c00de50 con 0x7f8050080f30 2024-09-17T04:24:27.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.186+0000 7f8067400640 1 -- 172.21.15.100:0/1976844114 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f805c09ba70 con 0x7f8070073b00 2024-09-17T04:24:27.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.186+0000 7f806f400640 1 --2- 172.21.15.100:0/1976844114 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8050078c60 0x7f805007b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.186+0000 7f425ee00640 1 -- 172.21.15.100:0/2668926763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f425815e770 msgr2=0x7f42580b7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.186+0000 7f425ee00640 1 --2- 172.21.15.100:0/2668926763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f425815e770 0x7f42580b7a00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f42580ba1e0 tx=0x7f424402a4a0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.187+0000 7fc87cc00640 1 -- 172.21.15.100:0/1411818929 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc874041030 con 0x7fc8800aff10 2024-09-17T04:24:27.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.187+0000 7f806f400640 1 --2- 172.21.15.100:0/1976844114 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8050078c60 0x7f805007b170 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8068004a30 tx=0x7f80680049a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.186+0000 7f9725000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97200af3f0 0x7f972014c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.186+0000 7f7633400640 1 -- 172.21.15.100:0/215132568 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634003970 con 0x7f763c0af920 2024-09-17T04:24:27.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.187+0000 7f9725000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f972015add0 0x7f97200ad0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.187+0000 7f9725000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f972015f1e0 0x7f97200ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.187+0000 7f9725000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9720074dc0 con 0x7f972015add0 2024-09-17T04:24:27.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.187+0000 7f9725000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f97200b42f0 con 0x7f97200af3f0 2024-09-17T04:24:27.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.187+0000 7f9725000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9720074f40 con 0x7f972015f1e0 2024-09-17T04:24:27.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.187+0000 7f971de00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97200af3f0 0x7f972014c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.187+0000 7f971de00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97200af3f0 0x7f972014c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.100:36320/0 (socket says 172.21.15.100:36320) 2024-09-17T04:24:27.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.187+0000 7f971de00640 1 -- 172.21.15.100:0/1294992513 learned_addr learned my addr 172.21.15.100:0/1294992513 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:27.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.187+0000 7f971e800640 1 --2- 172.21.15.100:0/1294992513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f972015f1e0 0x7f97200ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.187+0000 7f7633400640 1 -- 172.21.15.100:0/215132568 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7634003c00 con 0x7f763c0af920 2024-09-17T04:24:27.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.187+0000 7f7633400640 1 -- 172.21.15.100:0/215132568 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634004020 con 0x7f763c0af920 2024-09-17T04:24:27.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.188+0000 7f971de00640 1 -- 172.21.15.100:0/1294992513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f972015f1e0 msgr2=0x7f97200ad5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.188+0000 7f971de00640 1 --2- 172.21.15.100:0/1294992513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f972015f1e0 0x7f97200ad5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.188+0000 7f971de00640 1 -- 172.21.15.100:0/1294992513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f972015add0 msgr2=0x7f97200ad0b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T04:24:27.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.188+0000 7f971de00640 1 --2- 172.21.15.100:0/1294992513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f972015add0 0x7f97200ad0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.188+0000 7f7642600640 1 -- 172.21.15.100:0/215132568 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f763c1f3dd0 con 0x7f763c0af920 2024-09-17T04:24:27.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.188+0000 7f7642600640 1 -- 172.21.15.100:0/215132568 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f763c1f42d0 con 0x7f763c0af920 2024-09-17T04:24:27.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.188+0000 7f971de00640 1 -- 172.21.15.100:0/1294992513 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f972014d2f0 con 0x7f97200af3f0 2024-09-17T04:24:27.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.186+0000 7fdb97e00640 1 -- 172.21.15.100:0/3969895035 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdb90053cc0 con 0x7fdb90005c60 2024-09-17T04:24:27.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.189+0000 7f971de00640 1 --2- 172.21.15.100:0/1294992513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97200af3f0 0x7f972014c8a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9710007830 tx=0x7f9710007cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.188+0000 7fc87cc00640 1 --2- 172.21.15.100:0/1411818929 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc860078c60 0x7fc86007b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.189+0000 7fc87cc00640 1 -- 172.21.15.100:0/1411818929 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc8740bb920 con 0x7fc8800aff10 2024-09-17T04:24:27.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.189+0000 7fc87cc00640 1 --2- 172.21.15.100:0/1411818929 >> [v2:172.21.15.184:6810/985001108,v1:172.21.15.184:6811/985001108] conn(0x7fc860080ed0 0x7fc860083320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.189+0000 7fc87cc00640 1 -- 172.21.15.100:0/1411818929 --> [v2:172.21.15.184:6810/985001108,v1:172.21.15.184:6811/985001108] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc8600839f0 con 0x7fc860080ed0 2024-09-17T04:24:27.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.189+0000 7fc87cc00640 1 -- 172.21.15.100:0/1411818929 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc8740bbd10 con 0x7fc8800aff10 2024-09-17T04:24:27.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.189+0000 7f425ee00640 1 -- 172.21.15.100:0/2668926763 shutdown_connections 2024-09-17T04:24:27.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.189+0000 7f425ee00640 1 --2- 172.21.15.100:0/2668926763 >> v2:172.21.15.100:3300/0 conn(0x7f42580b7f40 0x7f42580b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.189+0000 7f7642600640 1 -- 172.21.15.100:0/215132568 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f763c0b3d60 con 0x7f763c0af920 2024-09-17T04:24:27.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.189+0000 7f806fe00640 1 --2- 172.21.15.100:0/1976844114 >> [v2:172.21.15.184:6802/3708129583,v1:172.21.15.184:6803/3708129583] conn(0x7f8050080f30 0x7f8050083380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.189+0000 7f425ee00640 1 --2- 172.21.15.100:0/2668926763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f425815e770 0x7f42580b7a00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.189+0000 7f9716000640 1 -- 172.21.15.100:0/1294992513 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9710018070 con 0x7f97200af3f0 2024-09-17T04:24:27.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.189+0000 7f425ee00640 1 -- 172.21.15.100:0/2668926763 >> 172.21.15.100:0/2668926763 conn(0x7f42580d44f0 msgr2=0x7f42580d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:27.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.190+0000 7f9716000640 1 -- 172.21.15.100:0/1294992513 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f971000ec10 con 0x7f97200af3f0 2024-09-17T04:24:27.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.190+0000 7f9716000640 1 -- 172.21.15.100:0/1294992513 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9710005020 con 0x7f97200af3f0 2024-09-17T04:24:27.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.190+0000 7f425ee00640 1 -- 172.21.15.100:0/2668926763 shutdown_connections 2024-09-17T04:24:27.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.189+0000 7fc87fe00640 1 --2- 172.21.15.100:0/1411818929 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc860078c60 0x7fc86007b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.190+0000 7fc885c00640 1 --2- 172.21.15.100:0/1411818929 >> [v2:172.21.15.184:6810/985001108,v1:172.21.15.184:6811/985001108] conn(0x7fc860080ed0 0x7fc860083320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.190+0000 7f425ee00640 1 -- 172.21.15.100:0/2668926763 wait complete. 2024-09-17T04:24:27.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.190+0000 7fdb8ea00640 1 -- 172.21.15.100:0/3969895035 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdb8401dbe0 con 0x7fdb90005c60 2024-09-17T04:24:27.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.190+0000 7f806fe00640 1 --2- 172.21.15.100:0/1976844114 >> [v2:172.21.15.184:6802/3708129583,v1:172.21.15.184:6803/3708129583] conn(0x7f8050080f30 0x7f8050083380 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.191+0000 7f7633400640 1 -- 172.21.15.100:0/215132568 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7634021020 con 0x7f763c0af920 2024-09-17T04:24:27.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.191+0000 7fc885c00640 1 --2- 172.21.15.100:0/1411818929 >> [v2:172.21.15.184:6810/985001108,v1:172.21.15.184:6811/985001108] conn(0x7fc860080ed0 0x7fc860083320 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.191+0000 7f9725000640 1 -- 172.21.15.100:0/1294992513 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97201e50f0 con 0x7f97200af3f0 2024-09-17T04:24:27.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.191+0000 7f9725000640 1 -- 172.21.15.100:0/1294992513 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97200adb30 con 0x7f97200af3f0 2024-09-17T04:24:27.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.191+0000 7f8067400640 1 -- 172.21.15.100:0/1976844114 <== osd.5 v2:172.21.15.184:6802/3708129583 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f805c00de50 con 0x7f8050080f30 2024-09-17T04:24:27.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.192+0000 7f7633400640 1 --2- 172.21.15.100:0/215132568 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f761c078c10 0x7f761c07b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.192+0000 7f7633400640 1 -- 172.21.15.100:0/215132568 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f763409f110 con 0x7f763c0af920 2024-09-17T04:24:27.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.192+0000 7fc87cc00640 1 -- 172.21.15.100:0/1411818929 <== osd.4 v2:172.21.15.184:6810/985001108 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc8600839f0 con 0x7fc860080ed0 2024-09-17T04:24:27.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.192+0000 7f763b400640 1 --2- 172.21.15.100:0/215132568 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f761c078c10 0x7f761c07b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.192+0000 7f7633400640 1 --2- 172.21.15.100:0/215132568 >> [v2:172.21.15.184:6825/3908254181,v1:172.21.15.184:6827/3908254181] conn(0x7f761c080e80 0x7f761c0832d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.192+0000 7f7633400640 1 -- 172.21.15.100:0/215132568 --> [v2:172.21.15.184:6825/3908254181,v1:172.21.15.184:6827/3908254181] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7634017e50 con 0x7f761c080e80 2024-09-17T04:24:27.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.192+0000 7f7633400640 1 -- 172.21.15.100:0/215132568 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f76340a6050 con 0x7f763c0af920 2024-09-17T04:24:27.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.192+0000 7fdb8ea00640 1 --2- 172.21.15.100:0/3969895035 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdb78078c60 0x7fdb7807b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.192+0000 7fdb8ea00640 1 -- 172.21.15.100:0/3969895035 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fdb840a3980 con 0x7fdb90005c60 2024-09-17T04:24:27.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.192+0000 7fdb8ea00640 1 --2- 172.21.15.100:0/3969895035 >> [v2:172.21.15.100:6824/2894635016,v1:172.21.15.100:6825/2894635016] conn(0x7fdb78080f30 0x7fdb78083380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.192+0000 7fc87fe00640 1 --2- 172.21.15.100:0/1411818929 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc860078c60 0x7fc86007b170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc880161ec0 tx=0x7fc86c006b80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.193+0000 7fdb8ea00640 1 -- 172.21.15.100:0/3969895035 --> [v2:172.21.15.100:6824/2894635016,v1:172.21.15.100:6825/2894635016] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdb8401eec0 con 0x7fdb78080f30 2024-09-17T04:24:27.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.193+0000 7fdb8ea00640 1 -- 172.21.15.100:0/3969895035 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fdb840a82a0 con 0x7fdb90005c60 2024-09-17T04:24:27.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.193+0000 7f970b400640 1 -- 172.21.15.100:0/1294992513 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f972014d0a0 con 0x7f97200af3f0 2024-09-17T04:24:27.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.193+0000 7fdb96000640 1 --2- 172.21.15.100:0/3969895035 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdb78078c60 0x7fdb7807b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.193+0000 7f763aa00640 1 --2- 172.21.15.100:0/215132568 >> [v2:172.21.15.184:6825/3908254181,v1:172.21.15.184:6827/3908254181] conn(0x7f761c080e80 0x7f761c0832d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.193+0000 7f763b400640 1 --2- 172.21.15.100:0/215132568 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f761c078c10 0x7f761c07b120 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f762c0095f0 tx=0x7f762c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.193+0000 7fdb97400640 1 --2- 172.21.15.100:0/3969895035 >> [v2:172.21.15.100:6824/2894635016,v1:172.21.15.100:6825/2894635016] conn(0x7fdb78080f30 0x7fdb78083380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.194+0000 7f763aa00640 1 --2- 172.21.15.100:0/215132568 >> [v2:172.21.15.184:6825/3908254181,v1:172.21.15.184:6827/3908254181] conn(0x7f761c080e80 0x7f761c0832d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.194+0000 7fdb97400640 1 --2- 172.21.15.100:0/3969895035 >> [v2:172.21.15.100:6824/2894635016,v1:172.21.15.100:6825/2894635016] conn(0x7fdb78080f30 0x7fdb78083380 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.195+0000 7fdb96000640 1 --2- 172.21.15.100:0/3969895035 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdb78078c60 0x7fdb7807b170 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdb88033000 tx=0x7fdb8802c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.195+0000 7f7633400640 1 -- 172.21.15.100:0/215132568 <== osd.6 v2:172.21.15.184:6825/3908254181 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7634017e50 con 0x7f761c080e80 2024-09-17T04:24:27.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.195+0000 7f425ee00640 1 Processor -- start 2024-09-17T04:24:27.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.196+0000 7fdb8ea00640 1 -- 172.21.15.100:0/3969895035 <== osd.2 v2:172.21.15.100:6824/2894635016 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdb8401eec0 con 0x7fdb78080f30 2024-09-17T04:24:27.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.196+0000 7f9716000640 1 -- 172.21.15.100:0/1294992513 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f97100051c0 con 0x7f97200af3f0 2024-09-17T04:24:27.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.197+0000 7f9716000640 1 --2- 172.21.15.100:0/1294992513 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9700078c10 0x7f970007b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.198+0000 7f9716000640 1 -- 172.21.15.100:0/1294992513 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f971005f620 con 0x7f97200af3f0 2024-09-17T04:24:27.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.198+0000 7f971d400640 1 --2- 172.21.15.100:0/1294992513 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9700078c10 0x7f970007b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.198+0000 7f425ee00640 1 -- start start 2024-09-17T04:24:27.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.199+0000 7f9716000640 1 --2- 172.21.15.100:0/1294992513 >> [v2:172.21.15.100:6800/2935840419,v1:172.21.15.100:6801/2935840419] conn(0x7f9700081a80 0x7f9700083f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.199+0000 7f9716000640 1 -- 172.21.15.100:0/1294992513 --> [v2:172.21.15.100:6800/2935840419,v1:172.21.15.100:6801/2935840419] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9700084640 con 0x7f9700081a80 2024-09-17T04:24:27.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.199+0000 7f9716000640 1 -- 172.21.15.100:0/1294992513 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f971005fba0 con 0x7f97200af3f0 2024-09-17T04:24:27.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.199+0000 7f971d400640 1 --2- 172.21.15.100:0/1294992513 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9700078c10 0x7f970007b120 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f970c036000 tx=0x7f970c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.201+0000 7f425ee00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42580b7f40 0x7f42580ce420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.201+0000 7f425ee00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f425815e770 0x7f42580c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.201+0000 7f425ee00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42580ca440 0x7f42582053f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.201+0000 7f425ee00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f42580d39a0 con 0x7f42580ca440 2024-09-17T04:24:27.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.201+0000 7f425ee00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f42580d3820 con 0x7f425815e770 2024-09-17T04:24:27.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.201+0000 7f425ee00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f42580d3b20 con 0x7f42580b7f40 2024-09-17T04:24:27.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.201+0000 7f425d000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f425815e770 0x7f42580c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.201+0000 7f425d000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f425815e770 0x7f42580c9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.100:36336/0 (socket says 172.21.15.100:36336) 2024-09-17T04:24:27.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.201+0000 7f425d000640 1 -- 172.21.15.100:0/296202748 learned_addr learned my addr 172.21.15.100:0/296202748 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:27.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.201+0000 7f425da00640 1 --2- 172.21.15.100:0/296202748 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42580b7f40 0x7f42580ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.202+0000 7f425e400640 1 --2- 172.21.15.100:0/296202748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42580ca440 0x7f42582053f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.202+0000 7f971e800640 1 --2- 172.21.15.100:0/1294992513 >> [v2:172.21.15.100:6800/2935840419,v1:172.21.15.100:6801/2935840419] conn(0x7f9700081a80 0x7f9700083f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.202+0000 7f425d000640 1 -- 172.21.15.100:0/296202748 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42580b7f40 msgr2=0x7f42580ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.202+0000 7f425d000640 1 --2- 172.21.15.100:0/296202748 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42580b7f40 0x7f42580ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.203+0000 7f425d000640 1 -- 172.21.15.100:0/296202748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42580ca440 msgr2=0x7f42582053f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.203+0000 7f425d000640 1 --2- 172.21.15.100:0/296202748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42580ca440 0x7f42582053f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.203+0000 7f425d000640 1 -- 172.21.15.100:0/296202748 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42580cc0f0 con 0x7f425815e770 2024-09-17T04:24:27.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.203+0000 7f971e800640 1 --2- 172.21.15.100:0/1294992513 >> [v2:172.21.15.100:6800/2935840419,v1:172.21.15.100:6801/2935840419] conn(0x7f9700081a80 0x7f9700083f70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.203+0000 7f425e400640 1 --2- 172.21.15.100:0/296202748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42580ca440 0x7f42582053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:24:27.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.203+0000 7f425d000640 1 --2- 172.21.15.100:0/296202748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f425815e770 0x7f42580c9cc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4244009780 tx=0x7f4244033d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.204+0000 7f4255600640 1 -- 172.21.15.100:0/296202748 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4244031d60 con 0x7f425815e770 2024-09-17T04:24:27.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.204+0000 7f4255600640 1 -- 172.21.15.100:0/296202748 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4244005020 con 0x7f425815e770 2024-09-17T04:24:27.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.204+0000 7f4255600640 1 -- 172.21.15.100:0/296202748 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4244005470 con 0x7f425815e770 2024-09-17T04:24:27.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.204+0000 7f9716000640 1 -- 172.21.15.100:0/1294992513 <== osd.3 v2:172.21.15.100:6800/2935840419 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9700084640 con 0x7f9700081a80 2024-09-17T04:24:27.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.205+0000 7f425ee00640 1 -- 172.21.15.100:0/296202748 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42582062e0 con 0x7f425815e770 2024-09-17T04:24:27.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.205+0000 7f425ee00640 1 -- 172.21.15.100:0/296202748 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42581f4cd0 con 0x7f425815e770 2024-09-17T04:24:27.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.206+0000 7f425ee00640 1 -- 172.21.15.100:0/296202748 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f42580cbcd0 con 0x7f425815e770 2024-09-17T04:24:27.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.209+0000 7f4255600640 1 -- 172.21.15.100:0/296202748 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f42440047a0 con 0x7f425815e770 2024-09-17T04:24:27.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.211+0000 7f4255600640 1 --2- 172.21.15.100:0/296202748 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4238078b90 0x7f423807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.211+0000 7f4255600640 1 -- 172.21.15.100:0/296202748 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f42440bc3e0 con 0x7f425815e770 2024-09-17T04:24:27.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.211+0000 7f425da00640 1 --2- 172.21.15.100:0/296202748 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4238078b90 0x7f423807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.211+0000 7f4255600640 1 --2- 172.21.15.100:0/296202748 >> [v2:172.21.15.184:6818/366777084,v1:172.21.15.184:6819/366777084] conn(0x7f4238080de0 0x7f4238083230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.211+0000 7f4255600640 1 -- 172.21.15.100:0/296202748 --> [v2:172.21.15.184:6818/366777084,v1:172.21.15.184:6819/366777084] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4244045e50 con 0x7f4238080de0 2024-09-17T04:24:27.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.211+0000 7f4255600640 1 -- 172.21.15.100:0/296202748 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f42440bc790 con 0x7f425815e770 2024-09-17T04:24:27.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.212+0000 7f425e400640 1 --2- 172.21.15.100:0/296202748 >> [v2:172.21.15.184:6818/366777084,v1:172.21.15.184:6819/366777084] conn(0x7f4238080de0 0x7f4238083230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.212+0000 7f425da00640 1 --2- 172.21.15.100:0/296202748 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4238078b90 0x7f423807b0a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f424c001720 tx=0x7f424c007e90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.213+0000 7f425e400640 1 --2- 172.21.15.100:0/296202748 >> [v2:172.21.15.184:6818/366777084,v1:172.21.15.184:6819/366777084] conn(0x7f4238080de0 0x7f4238083230 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.214+0000 7f4255600640 1 -- 172.21.15.100:0/296202748 <== osd.7 v2:172.21.15.184:6818/366777084 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4244045e50 con 0x7f4238080de0 2024-09-17T04:24:27.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.213+0000 7f8b4f200640 1 -- 172.21.15.100:0/2946428688 --> [v2:172.21.15.100:6816/1292500299,v1:172.21.15.100:6817/1292500299] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8b50070d00 con 0x7f8b2c080eb0 2024-09-17T04:24:27.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.215+0000 7fc887c00640 1 -- 172.21.15.100:0/1411818929 --> [v2:172.21.15.184:6810/985001108,v1:172.21.15.184:6811/985001108] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc85c002dd0 con 0x7fc860080ed0 2024-09-17T04:24:27.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.215+0000 7f8b46a00640 1 -- 172.21.15.100:0/2946428688 <== osd.0 v2:172.21.15.100:6816/1292500299 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8b50070d00 con 0x7f8b2c080eb0 2024-09-17T04:24:27.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.216+0000 7f188a800640 1 -- 172.21.15.100:0/2808994528 --> [v2:172.21.15.100:6808/1254254975,v1:172.21.15.100:6809/1254254975] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f18841574a0 con 0x7f1864080e50 2024-09-17T04:24:27.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.216+0000 7f8b4f200640 1 -- 172.21.15.100:0/2946428688 >> [v2:172.21.15.100:6816/1292500299,v1:172.21.15.100:6817/1292500299] conn(0x7f8b2c080eb0 msgr2=0x7f8b2c083300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.216+0000 7f8b4f200640 1 --2- 172.21.15.100:0/2946428688 >> [v2:172.21.15.100:6816/1292500299,v1:172.21.15.100:6817/1292500299] conn(0x7f8b2c080eb0 0x7f8b2c083300 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.216+0000 7fc87cc00640 1 -- 172.21.15.100:0/1411818929 <== osd.4 v2:172.21.15.184:6810/985001108 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc85c002dd0 con 0x7fc860080ed0 2024-09-17T04:24:27.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.217+0000 7f8b4f200640 1 -- 172.21.15.100:0/2946428688 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8b2c078be0 msgr2=0x7f8b2c07b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.217+0000 7f8b4f200640 1 --2- 172.21.15.100:0/2946428688 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8b2c078be0 0x7f8b2c07b0f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8b40007fd0 tx=0x7f8b40009040 comp rx=0 tx=0).stop 2024-09-17T04:24:27.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.217+0000 7fc887c00640 1 -- 172.21.15.100:0/1411818929 >> [v2:172.21.15.184:6810/985001108,v1:172.21.15.184:6811/985001108] conn(0x7fc860080ed0 msgr2=0x7fc860083320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.217+0000 7fc887c00640 1 --2- 172.21.15.100:0/1411818929 >> [v2:172.21.15.184:6810/985001108,v1:172.21.15.184:6811/985001108] conn(0x7fc860080ed0 0x7fc860083320 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.218+0000 7f8b4f200640 1 -- 172.21.15.100:0/2946428688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b5006ee00 msgr2=0x7f8b50159470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.217+0000 7f187b400640 1 -- 172.21.15.100:0/2808994528 <== osd.1 v2:172.21.15.100:6808/1254254975 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f18841574a0 con 0x7f1864080e50 2024-09-17T04:24:27.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.218+0000 7f8b4f200640 1 --2- 172.21.15.100:0/2946428688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b5006ee00 0x7f8b50159470 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8b38000c00 tx=0x7f8b38033ba0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.218+0000 7f8b4e800640 1 -- 172.21.15.100:0/2946428688 reap_dead start 2024-09-17T04:24:27.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.218+0000 7f8b4f200640 1 -- 172.21.15.100:0/2946428688 shutdown_connections 2024-09-17T04:24:27.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.218+0000 7f8b4f200640 1 -- 172.21.15.100:0/2946428688 >> 172.21.15.100:0/2946428688 conn(0x7f8b500b1c60 msgr2=0x7f8b50068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:27.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.218+0000 7fc887c00640 1 -- 172.21.15.100:0/1411818929 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc860078c60 msgr2=0x7fc86007b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.218+0000 7fc887c00640 1 --2- 172.21.15.100:0/1411818929 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc860078c60 0x7fc86007b170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc880161ec0 tx=0x7fc86c006b80 comp rx=0 tx=0).stop 2024-09-17T04:24:27.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.218+0000 7fc887c00640 1 -- 172.21.15.100:0/1411818929 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8800aff10 msgr2=0x7fc88009dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.218+0000 7fc887c00640 1 --2- 172.21.15.100:0/1411818929 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8800aff10 0x7fc88009dcb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc874034040 tx=0x7fc874003b90 comp rx=0 tx=0).stop 2024-09-17T04:24:27.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.218+0000 7fc885c00640 1 -- 172.21.15.100:0/1411818929 reap_dead start 2024-09-17T04:24:27.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.218+0000 7fc887c00640 1 -- 172.21.15.100:0/1411818929 shutdown_connections 2024-09-17T04:24:27.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.218+0000 7fc887c00640 1 -- 172.21.15.100:0/1411818929 >> 172.21.15.100:0/1411818929 conn(0x7fc8800d02f0 msgr2=0x7fc8800ae490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:27.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.218+0000 7f188a800640 1 -- 172.21.15.100:0/2808994528 >> [v2:172.21.15.100:6808/1254254975,v1:172.21.15.100:6809/1254254975] conn(0x7f1864080e50 msgr2=0x7f18640832a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.218+0000 7f188a800640 1 --2- 172.21.15.100:0/2808994528 >> [v2:172.21.15.100:6808/1254254975,v1:172.21.15.100:6809/1254254975] conn(0x7f1864080e50 0x7f18640832a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.218+0000 7f188a800640 1 -- 172.21.15.100:0/2808994528 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1864078be0 msgr2=0x7f186407b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.218+0000 7f188a800640 1 --2- 172.21.15.100:0/2808994528 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1864078be0 0x7f186407b0f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f18700018a0 tx=0x7f1870009040 comp rx=0 tx=0).stop 2024-09-17T04:24:27.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.218+0000 7f188a800640 1 -- 172.21.15.100:0/2808994528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18840b4060 msgr2=0x7f18841f9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.218+0000 7f188a800640 1 --2- 172.21.15.100:0/2808994528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18840b4060 0x7f18841f9060 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1884069da0 tx=0x7f1874031690 comp rx=0 tx=0).stop 2024-09-17T04:24:27.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.219+0000 7f8b4f200640 1 -- 172.21.15.100:0/2946428688 shutdown_connections 2024-09-17T04:24:27.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.219+0000 7f1883e00640 1 -- 172.21.15.100:0/2808994528 reap_dead start 2024-09-17T04:24:27.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.219+0000 7f8b4f200640 1 -- 172.21.15.100:0/2946428688 wait complete. 2024-09-17T04:24:27.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.219+0000 7f188a800640 1 -- 172.21.15.100:0/2808994528 shutdown_connections 2024-09-17T04:24:27.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.219+0000 7f188a800640 1 -- 172.21.15.100:0/2808994528 >> 172.21.15.100:0/2808994528 conn(0x7f18840b2650 msgr2=0x7f188414f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:27.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.219+0000 7fc887c00640 1 -- 172.21.15.100:0/1411818929 shutdown_connections 2024-09-17T04:24:27.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.219+0000 7f188a800640 1 -- 172.21.15.100:0/2808994528 shutdown_connections 2024-09-17T04:24:27.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.219+0000 7f188a800640 1 -- 172.21.15.100:0/2808994528 wait complete. 2024-09-17T04:24:27.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.222+0000 7fdb97e00640 1 -- 172.21.15.100:0/3969895035 --> [v2:172.21.15.100:6824/2894635016,v1:172.21.15.100:6825/2894635016] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdb900df180 con 0x7fdb78080f30 2024-09-17T04:24:27.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.219+0000 7fc887c00640 1 -- 172.21.15.100:0/1411818929 wait complete. 2024-09-17T04:24:27.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.225+0000 7f7642600640 1 -- 172.21.15.100:0/215132568 --> [v2:172.21.15.184:6825/3908254181,v1:172.21.15.184:6827/3908254181] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f763c1f4080 con 0x7f761c080e80 2024-09-17T04:24:27.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.225+0000 7fdb8ea00640 1 -- 172.21.15.100:0/3969895035 <== osd.2 v2:172.21.15.100:6824/2894635016 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdb900df180 con 0x7fdb78080f30 2024-09-17T04:24:27.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.225+0000 7f7633400640 1 -- 172.21.15.100:0/215132568 <== osd.6 v2:172.21.15.184:6825/3908254181 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f763c1f4080 con 0x7f761c080e80 2024-09-17T04:24:27.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.226+0000 7fdb97e00640 1 -- 172.21.15.100:0/3969895035 >> [v2:172.21.15.100:6824/2894635016,v1:172.21.15.100:6825/2894635016] conn(0x7fdb78080f30 msgr2=0x7fdb78083380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.226+0000 7fdb97e00640 1 --2- 172.21.15.100:0/3969895035 >> [v2:172.21.15.100:6824/2894635016,v1:172.21.15.100:6825/2894635016] conn(0x7fdb78080f30 0x7fdb78083380 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.226+0000 7f8074a00640 1 -- 172.21.15.100:0/1976844114 --> [v2:172.21.15.184:6802/3708129583,v1:172.21.15.184:6803/3708129583] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f807013b340 con 0x7f8050080f30 2024-09-17T04:24:27.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.226+0000 7f7642600640 1 -- 172.21.15.100:0/215132568 >> [v2:172.21.15.184:6825/3908254181,v1:172.21.15.184:6827/3908254181] conn(0x7f761c080e80 msgr2=0x7f761c0832d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.226+0000 7f7642600640 1 --2- 172.21.15.100:0/215132568 >> [v2:172.21.15.184:6825/3908254181,v1:172.21.15.184:6827/3908254181] conn(0x7f761c080e80 0x7f761c0832d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.226+0000 7fdb97e00640 1 -- 172.21.15.100:0/3969895035 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdb78078c60 msgr2=0x7fdb7807b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.226+0000 7fdb97e00640 1 --2- 172.21.15.100:0/3969895035 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdb78078c60 0x7fdb7807b170 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdb88033000 tx=0x7fdb8802c040 comp rx=0 tx=0).stop 2024-09-17T04:24:27.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.226+0000 7fdb97e00640 1 -- 172.21.15.100:0/3969895035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb90005c60 msgr2=0x7fdb900dd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.226+0000 7fdb97e00640 1 --2- 172.21.15.100:0/3969895035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb90005c60 0x7fdb900dd7d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400be00 tx=0x7fdb84007590 comp rx=0 tx=0).stop 2024-09-17T04:24:27.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.227+0000 7fdb97400640 1 -- 172.21.15.100:0/3969895035 reap_dead start 2024-09-17T04:24:27.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.227+0000 7fdb97e00640 1 -- 172.21.15.100:0/3969895035 shutdown_connections 2024-09-17T04:24:27.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.227+0000 7fdb97e00640 1 -- 172.21.15.100:0/3969895035 >> 172.21.15.100:0/3969895035 conn(0x7fdb90051420 msgr2=0x7fdb900e1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:27.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.227+0000 7f8067400640 1 -- 172.21.15.100:0/1976844114 <== osd.5 v2:172.21.15.184:6802/3708129583 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8060005020 con 0x7f8050080f30 2024-09-17T04:24:27.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.227+0000 7fdb97e00640 1 -- 172.21.15.100:0/3969895035 shutdown_connections 2024-09-17T04:24:27.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.228+0000 7fdb97e00640 1 -- 172.21.15.100:0/3969895035 wait complete. 2024-09-17T04:24:27.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.227+0000 7f7642600640 1 -- 172.21.15.100:0/215132568 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f761c078c10 msgr2=0x7f761c07b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.227+0000 7f7642600640 1 --2- 172.21.15.100:0/215132568 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f761c078c10 0x7f761c07b120 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f762c0095f0 tx=0x7f762c004730 comp rx=0 tx=0).stop 2024-09-17T04:24:27.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.227+0000 7f7642600640 1 -- 172.21.15.100:0/215132568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f763c0af920 msgr2=0x7f763c0cae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.227+0000 7f7642600640 1 --2- 172.21.15.100:0/215132568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f763c0af920 0x7f763c0cae90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7634012f60 tx=0x7f7634012f90 comp rx=0 tx=0).stop 2024-09-17T04:24:27.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.227+0000 7f763be00640 1 -- 172.21.15.100:0/215132568 reap_dead start 2024-09-17T04:24:27.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.228+0000 7f8074a00640 1 -- 172.21.15.100:0/1976844114 >> [v2:172.21.15.184:6802/3708129583,v1:172.21.15.184:6803/3708129583] conn(0x7f8050080f30 msgr2=0x7f8050083380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.228+0000 7f8074a00640 1 --2- 172.21.15.100:0/1976844114 >> [v2:172.21.15.184:6802/3708129583,v1:172.21.15.184:6803/3708129583] conn(0x7f8050080f30 0x7f8050083380 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.228+0000 7f8074a00640 1 -- 172.21.15.100:0/1976844114 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8050078c60 msgr2=0x7f805007b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.228+0000 7f8074a00640 1 --2- 172.21.15.100:0/1976844114 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8050078c60 0x7f805007b170 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8068004a30 tx=0x7f80680049a0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.228+0000 7f8074a00640 1 -- 172.21.15.100:0/1976844114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8070073b00 msgr2=0x7f80700b11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.228+0000 7f8074a00640 1 --2- 172.21.15.100:0/1976844114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8070073b00 0x7f80700b11a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f805c00c970 tx=0x7f805c00ce30 comp rx=0 tx=0).stop 2024-09-17T04:24:27.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.228+0000 7f806fe00640 1 -- 172.21.15.100:0/1976844114 reap_dead start 2024-09-17T04:24:27.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.229+0000 7f8074a00640 1 -- 172.21.15.100:0/1976844114 shutdown_connections 2024-09-17T04:24:27.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.229+0000 7f8074a00640 1 -- 172.21.15.100:0/1976844114 >> 172.21.15.100:0/1976844114 conn(0x7f80700b1e40 msgr2=0x7f80701534f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:27.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.230+0000 7f8074a00640 1 -- 172.21.15.100:0/1976844114 shutdown_connections 2024-09-17T04:24:27.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.230+0000 7f8074a00640 1 -- 172.21.15.100:0/1976844114 wait complete. 2024-09-17T04:24:27.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.230+0000 7f7642600640 1 -- 172.21.15.100:0/215132568 shutdown_connections 2024-09-17T04:24:27.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.230+0000 7f7642600640 1 -- 172.21.15.100:0/215132568 >> 172.21.15.100:0/215132568 conn(0x7f763c06d3d0 msgr2=0x7f763c0730b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:27.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.231+0000 7f7642600640 1 -- 172.21.15.100:0/215132568 shutdown_connections 2024-09-17T04:24:27.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.231+0000 7f970b400640 1 -- 172.21.15.100:0/1294992513 --> [v2:172.21.15.100:6800/2935840419,v1:172.21.15.100:6801/2935840419] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f97200d00b0 con 0x7f9700081a80 2024-09-17T04:24:27.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.231+0000 7f7642600640 1 -- 172.21.15.100:0/215132568 wait complete. 2024-09-17T04:24:27.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.232+0000 7f9716000640 1 -- 172.21.15.100:0/1294992513 <== osd.3 v2:172.21.15.100:6800/2935840419 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f97200d00b0 con 0x7f9700081a80 2024-09-17T04:24:27.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.233+0000 7f970b400640 1 -- 172.21.15.100:0/1294992513 >> [v2:172.21.15.100:6800/2935840419,v1:172.21.15.100:6801/2935840419] conn(0x7f9700081a80 msgr2=0x7f9700083f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.233+0000 7f970b400640 1 --2- 172.21.15.100:0/1294992513 >> [v2:172.21.15.100:6800/2935840419,v1:172.21.15.100:6801/2935840419] conn(0x7f9700081a80 0x7f9700083f70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.234+0000 7f970b400640 1 -- 172.21.15.100:0/1294992513 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9700078c10 msgr2=0x7f970007b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.234+0000 7f970b400640 1 --2- 172.21.15.100:0/1294992513 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9700078c10 0x7f970007b120 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f970c036000 tx=0x7f970c004730 comp rx=0 tx=0).stop 2024-09-17T04:24:27.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.234+0000 7f970b400640 1 -- 172.21.15.100:0/1294992513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97200af3f0 msgr2=0x7f972014c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.234+0000 7f970b400640 1 --2- 172.21.15.100:0/1294992513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97200af3f0 0x7f972014c8a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9710007830 tx=0x7f9710007cf0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.234+0000 7f971e800640 1 -- 172.21.15.100:0/1294992513 reap_dead start 2024-09-17T04:24:27.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.234+0000 7f970b400640 1 -- 172.21.15.100:0/1294992513 shutdown_connections 2024-09-17T04:24:27.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.234+0000 7f970b400640 1 -- 172.21.15.100:0/1294992513 >> 172.21.15.100:0/1294992513 conn(0x7f97200742a0 msgr2=0x7f972006efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:27.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.235+0000 7f970b400640 1 -- 172.21.15.100:0/1294992513 shutdown_connections 2024-09-17T04:24:27.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.236+0000 7f970b400640 1 -- 172.21.15.100:0/1294992513 wait complete. 2024-09-17T04:24:27.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.245+0000 7f425ee00640 1 -- 172.21.15.100:0/296202748 --> [v2:172.21.15.184:6818/366777084,v1:172.21.15.184:6819/366777084] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f42581f4ed0 con 0x7f4238080de0 2024-09-17T04:24:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.246+0000 7f4255600640 1 -- 172.21.15.100:0/296202748 <== osd.7 v2:172.21.15.184:6818/366777084 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f42581f4ed0 con 0x7f4238080de0 2024-09-17T04:24:27.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.248+0000 7f425ee00640 1 -- 172.21.15.100:0/296202748 >> [v2:172.21.15.184:6818/366777084,v1:172.21.15.184:6819/366777084] conn(0x7f4238080de0 msgr2=0x7f4238083230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.248+0000 7f425ee00640 1 --2- 172.21.15.100:0/296202748 >> [v2:172.21.15.184:6818/366777084,v1:172.21.15.184:6819/366777084] conn(0x7f4238080de0 0x7f4238083230 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.249+0000 7f425ee00640 1 -- 172.21.15.100:0/296202748 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4238078b90 msgr2=0x7f423807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.249+0000 7f425ee00640 1 --2- 172.21.15.100:0/296202748 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4238078b90 0x7f423807b0a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f424c001720 tx=0x7f424c007e90 comp rx=0 tx=0).stop 2024-09-17T04:24:27.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.249+0000 7f425ee00640 1 -- 172.21.15.100:0/296202748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f425815e770 msgr2=0x7f42580c9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.249+0000 7f425ee00640 1 --2- 172.21.15.100:0/296202748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f425815e770 0x7f42580c9cc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4244009780 tx=0x7f4244033d60 comp rx=0 tx=0).stop 2024-09-17T04:24:27.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.249+0000 7f425e400640 1 -- 172.21.15.100:0/296202748 reap_dead start 2024-09-17T04:24:27.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.249+0000 7f425ee00640 1 -- 172.21.15.100:0/296202748 shutdown_connections 2024-09-17T04:24:27.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.249+0000 7f425ee00640 1 -- 172.21.15.100:0/296202748 >> 172.21.15.100:0/296202748 conn(0x7f42580d44f0 msgr2=0x7f42580d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:27.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.250+0000 7f425ee00640 1 -- 172.21.15.100:0/296202748 shutdown_connections 2024-09-17T04:24:27.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.250+0000 7f425ee00640 1 -- 172.21.15.100:0/296202748 wait complete. 2024-09-17T04:24:27.255 INFO:teuthology.orchestra.run.smithi100.stdout:64424509447 2024-09-17T04:24:27.255 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T04:24:27.257 INFO:teuthology.orchestra.run.smithi100.stdout:64424509447 2024-09-17T04:24:27.258 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T04:24:27.272 INFO:teuthology.orchestra.run.smithi100.stdout:64424509448 2024-09-17T04:24:27.272 INFO:teuthology.orchestra.run.smithi100.stdout:68719476745 2024-09-17T04:24:27.273 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T04:24:27.273 INFO:teuthology.orchestra.run.smithi100.stdout:68719476745 2024-09-17T04:24:27.274 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T04:24:27.274 INFO:teuthology.orchestra.run.smithi100.stdout:68719476744 2024-09-17T04:24:27.275 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T04:24:27.275 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T04:24:27.276 INFO:teuthology.orchestra.run.smithi100.stdout:60129542153 2024-09-17T04:24:27.276 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T04:24:27.282 INFO:teuthology.orchestra.run.smithi100.stdout:68719476745 2024-09-17T04:24:27.283 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T04:24:27.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.606+0000 7f0f6aa00640 1 -- 172.21.15.100:0/4142392728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f5c054180 msgr2=0x7f0f5c007540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.606+0000 7f0f6aa00640 1 --2- 172.21.15.100:0/4142392728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f5c054180 0x7f0f5c007540 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c011a00 tx=0x7f0f5802a4f0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.606+0000 7f0f6aa00640 1 -- 172.21.15.100:0/4142392728 shutdown_connections 2024-09-17T04:24:27.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.606+0000 7f0f6aa00640 1 --2- 172.21.15.100:0/4142392728 >> v2:172.21.15.100:3300/0 conn(0x7f0f5c053680 0x7f0f5c053a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.606+0000 7f0f6aa00640 1 --2- 172.21.15.100:0/4142392728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f5c0e31f0 0x7f0f5c053140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.606+0000 7f0f6aa00640 1 --2- 172.21.15.100:0/4142392728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f5c054180 0x7f0f5c007540 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.606+0000 7f0f6aa00640 1 -- 172.21.15.100:0/4142392728 >> 172.21.15.100:0/4142392728 conn(0x7f0f5c03a670 msgr2=0x7f0f5c03aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:27.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.607+0000 7f0f6aa00640 1 -- 172.21.15.100:0/4142392728 shutdown_connections 2024-09-17T04:24:27.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.607+0000 7f0f6aa00640 1 -- 172.21.15.100:0/4142392728 wait complete. 2024-09-17T04:24:27.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.611+0000 7f0f6aa00640 1 Processor -- start 2024-09-17T04:24:27.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.611+0000 7f0f6aa00640 1 -- start start 2024-09-17T04:24:27.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.614+0000 7f0f6aa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f5c053680 0x7f0f5c052750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.614+0000 7f0f6aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c054180 0x7f0f5c0cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.615+0000 7f0f63e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f5c053680 0x7f0f5c052750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.615+0000 7f0f63e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f5c053680 0x7f0f5c052750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.100:36360/0 (socket says 172.21.15.100:36360) 2024-09-17T04:24:27.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.615+0000 7f0f63400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c054180 0x7f0f5c0cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.615+0000 7f0f6aa00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f5c0e31f0 0x7f0f5c0ce310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.615+0000 7f0f6aa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c0e71c0 con 0x7f0f5c054180 2024-09-17T04:24:27.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.615+0000 7f0f63e00640 1 -- 172.21.15.100:0/1309031702 learned_addr learned my addr 172.21.15.100:0/1309031702 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:27.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.615+0000 7f0f6aa00640 1 -- 172.21.15.100:0/1309031702 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c0e7040 con 0x7f0f5c053680 2024-09-17T04:24:27.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.615+0000 7f0f6aa00640 1 -- 172.21.15.100:0/1309031702 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0f5c0e7340 con 0x7f0f5c0e31f0 2024-09-17T04:24:27.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.616+0000 7f0f68a00640 1 --2- 172.21.15.100:0/1309031702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f5c0e31f0 0x7f0f5c0ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.617+0000 7f0f63e00640 1 -- 172.21.15.100:0/1309031702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f5c0e31f0 msgr2=0x7f0f5c0ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.617+0000 7f0f63e00640 1 --2- 172.21.15.100:0/1309031702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f5c0e31f0 0x7f0f5c0ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.617+0000 7f0f63e00640 1 -- 172.21.15.100:0/1309031702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c054180 msgr2=0x7f0f5c0cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.617+0000 7f0f63e00640 1 --2- 172.21.15.100:0/1309031702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c054180 0x7f0f5c0cddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.618+0000 7fdf0c800640 1 -- 172.21.15.100:0/136822692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf040af3f0 msgr2=0x7fdf0415ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.618+0000 7fdf0c800640 1 --2- 172.21.15.100:0/136822692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf040af3f0 0x7fdf0415ee70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdef4005fd0 tx=0x7fdef402a530 comp rx=0 tx=0).stop 2024-09-17T04:24:27.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.618+0000 7f0f63e00640 1 -- 172.21.15.100:0/1309031702 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5c0e6020 con 0x7f0f5c053680 2024-09-17T04:24:27.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.618+0000 7fdf0c800640 1 -- 172.21.15.100:0/136822692 shutdown_connections 2024-09-17T04:24:27.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.618+0000 7fdf0c800640 1 --2- 172.21.15.100:0/136822692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf040b30d0 0x7fdf040b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.618+0000 7fdf0c800640 1 --2- 172.21.15.100:0/136822692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf040af3f0 0x7fdf0415ee70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.618+0000 7fdf0c800640 1 -- 172.21.15.100:0/136822692 >> 172.21.15.100:0/136822692 conn(0x7fdf040742a0 msgr2=0x7fdf040746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:27.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.618+0000 7fdf0c800640 1 -- 172.21.15.100:0/136822692 shutdown_connections 2024-09-17T04:24:27.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.619+0000 7fdf0c800640 1 -- 172.21.15.100:0/136822692 wait complete. 2024-09-17T04:24:27.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.618+0000 7f0f68a00640 1 --2- 172.21.15.100:0/1309031702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f5c0e31f0 0x7f0f5c0ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:24:27.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.618+0000 7f0f63400640 1 --2- 172.21.15.100:0/1309031702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c054180 0x7f0f5c0cddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:24:27.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.619+0000 7f0f63e00640 1 --2- 172.21.15.100:0/1309031702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f5c053680 0x7f0f5c052750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0f58000c00 tx=0x7f0f58003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.619+0000 7f0f57e00640 1 -- 172.21.15.100:0/1309031702 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5803e030 con 0x7f0f5c053680 2024-09-17T04:24:27.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.620+0000 7f0f57e00640 1 -- 172.21.15.100:0/1309031702 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f58003c20 con 0x7f0f5c053680 2024-09-17T04:24:27.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.620+0000 7f0f57e00640 1 -- 172.21.15.100:0/1309031702 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f58035040 con 0x7f0f5c053680 2024-09-17T04:24:27.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.620+0000 7f0f6aa00640 1 -- 172.21.15.100:0/1309031702 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f5c0e6300 con 0x7f0f5c053680 2024-09-17T04:24:27.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.620+0000 7f0f6aa00640 1 -- 172.21.15.100:0/1309031702 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f5c03d7e0 con 0x7f0f5c053680 2024-09-17T04:24:27.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.624+0000 7fdf0c800640 1 Processor -- start 2024-09-17T04:24:27.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.624+0000 7fdf0c800640 1 -- start start 2024-09-17T04:24:27.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.624+0000 7f0f6aa00640 1 -- 172.21.15.100:0/1309031702 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f40005290 con 0x7f0f5c053680 2024-09-17T04:24:27.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.627+0000 7fdf0c800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf040af3f0 0x7fdf040cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.626+0000 7f0f57e00640 1 -- 172.21.15.100:0/1309031702 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0f58041030 con 0x7f0f5c053680 2024-09-17T04:24:27.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.628+0000 7fdf0c800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf040b30d0 0x7fdf04159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.628+0000 7fdf0c800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf0415add0 0x7fdf04155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.628+0000 7f0f57e00640 1 --2- 172.21.15.100:0/1309031702 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0f44078b90 0x7f0f4407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.628+0000 7fdf09e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf040af3f0 0x7fdf040cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.628+0000 7fdf09e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf040af3f0 0x7fdf040cb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.100:36362/0 (socket says 172.21.15.100:36362) 2024-09-17T04:24:27.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.628+0000 7fdf09e00640 1 -- 172.21.15.100:0/3637359028 learned_addr learned my addr 172.21.15.100:0/3637359028 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:27.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.629+0000 7fdf0c800640 1 -- 172.21.15.100:0/3637359028 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdf040b3db0 con 0x7fdf0415add0 2024-09-17T04:24:27.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.629+0000 7f0f57e00640 1 -- 172.21.15.100:0/1309031702 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f0f580bb280 con 0x7f0f5c053680 2024-09-17T04:24:27.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.629+0000 7fdf09400640 1 --2- 172.21.15.100:0/3637359028 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf040b30d0 0x7fdf04159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.629+0000 7f0f63400640 1 --2- 172.21.15.100:0/1309031702 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0f44078b90 0x7f0f4407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.629+0000 7fdf0c800640 1 -- 172.21.15.100:0/3637359028 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdf040b3c30 con 0x7fdf040af3f0 2024-09-17T04:24:27.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.629+0000 7fdf0c800640 1 -- 172.21.15.100:0/3637359028 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdf040b3f30 con 0x7fdf040b30d0 2024-09-17T04:24:27.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.629+0000 7fdf0a800640 1 --2- 172.21.15.100:0/3637359028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf0415add0 0x7fdf04155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.630+0000 7f0f63400640 1 --2- 172.21.15.100:0/1309031702 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0f44078b90 0x7f0f4407b0a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c0d2450 tx=0x7f0f50005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.630+0000 7fdf09e00640 1 -- 172.21.15.100:0/3637359028 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf040b30d0 msgr2=0x7fdf04159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.630+0000 7fdf09e00640 1 --2- 172.21.15.100:0/3637359028 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf040b30d0 0x7fdf04159750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.630+0000 7fdf09e00640 1 -- 172.21.15.100:0/3637359028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf0415add0 msgr2=0x7fdf04155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.630+0000 7fdf09e00640 1 --2- 172.21.15.100:0/3637359028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf0415add0 0x7fdf04155f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.630+0000 7fdf09e00640 1 -- 172.21.15.100:0/3637359028 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf04159c90 con 0x7fdf040af3f0 2024-09-17T04:24:27.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.631+0000 7fdf09e00640 1 --2- 172.21.15.100:0/3637359028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf040af3f0 0x7fdf040cb9f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000ba50 tx=0x7fdf0000bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.632+0000 7fdefa000640 1 -- 172.21.15.100:0/3637359028 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf00019070 con 0x7fdf040af3f0 2024-09-17T04:24:27.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.632+0000 7fdefa000640 1 -- 172.21.15.100:0/3637359028 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf0000cd00 con 0x7fdf040af3f0 2024-09-17T04:24:27.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.632+0000 7fdefa000640 1 -- 172.21.15.100:0/3637359028 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf000146a0 con 0x7fdf040af3f0 2024-09-17T04:24:27.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.632+0000 7f0f57e00640 1 -- 172.21.15.100:0/1309031702 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0f58083060 con 0x7f0f5c053680 2024-09-17T04:24:27.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.632+0000 7fdf0c800640 1 -- 172.21.15.100:0/3637359028 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf04158450 con 0x7fdf040af3f0 2024-09-17T04:24:27.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.633+0000 7fdf0c800640 1 -- 172.21.15.100:0/3637359028 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf04158710 con 0x7fdf040af3f0 2024-09-17T04:24:27.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.638+0000 7fdefa000640 1 -- 172.21.15.100:0/3637359028 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdf0001d3f0 con 0x7fdf040af3f0 2024-09-17T04:24:27.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.638+0000 7fdeef400640 1 -- 172.21.15.100:0/3637359028 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf040b34e0 con 0x7fdf040af3f0 2024-09-17T04:24:27.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.639+0000 7fdefa000640 1 --2- 172.21.15.100:0/3637359028 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdee8078bc0 0x7fdee807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.640+0000 7fdefa000640 1 -- 172.21.15.100:0/3637359028 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fdf0009a9f0 con 0x7fdf040af3f0 2024-09-17T04:24:27.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.640+0000 7fdf09400640 1 --2- 172.21.15.100:0/3637359028 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdee8078bc0 0x7fdee807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.641+0000 7fdf09400640 1 --2- 172.21.15.100:0/3637359028 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdee8078bc0 0x7fdee807b0d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdef4033000 tx=0x7fdef4004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.645+0000 7fdefa000640 1 -- 172.21.15.100:0/3637359028 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fdf000627d0 con 0x7fdf040af3f0 2024-09-17T04:24:27.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.712+0000 7fa650400640 1 -- 172.21.15.100:0/2435525710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa630003a00 msgr2=0x7fa630003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.712+0000 7fa650400640 1 --2- 172.21.15.100:0/2435525710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa630003a00 0x7fa630003e50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa63400c4a0 tx=0x7fa630005830 comp rx=0 tx=0).stop 2024-09-17T04:24:27.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.713+0000 7fa650400640 1 -- 172.21.15.100:0/2435525710 shutdown_connections 2024-09-17T04:24:27.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.713+0000 7fa650400640 1 --2- 172.21.15.100:0/2435525710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa630003a00 0x7fa630003e50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.713+0000 7fa650400640 1 -- 172.21.15.100:0/2435525710 >> 172.21.15.100:0/2435525710 conn(0x7fa64003a5e0 msgr2=0x7fa64003a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:27.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.713+0000 7fa650400640 1 -- 172.21.15.100:0/2435525710 shutdown_connections 2024-09-17T04:24:27.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.713+0000 7fa650400640 1 -- 172.21.15.100:0/2435525710 wait complete. 2024-09-17T04:24:27.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.714+0000 7f922b000640 1 -- 172.21.15.100:0/1960517645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92240cf6c0 msgr2=0x7f9224162e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.714+0000 7f922b000640 1 --2- 172.21.15.100:0/1960517645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92240cf6c0 0x7f9224162e20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f922408d3b0 tx=0x7f921802a9a0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.715+0000 7f922b000640 1 -- 172.21.15.100:0/1960517645 shutdown_connections 2024-09-17T04:24:27.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.715+0000 7f922b000640 1 --2- 172.21.15.100:0/1960517645 >> v2:172.21.15.100:3300/0 conn(0x7f92240b7f40 0x7f92240b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.715+0000 7f922b000640 1 --2- 172.21.15.100:0/1960517645 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f922415e770 0x7f92240b7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.715+0000 7f922b000640 1 --2- 172.21.15.100:0/1960517645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92240cf6c0 0x7f9224162e20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.715+0000 7f922b000640 1 -- 172.21.15.100:0/1960517645 >> 172.21.15.100:0/1960517645 conn(0x7f92240d44f0 msgr2=0x7f92240d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:27.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.715+0000 7f922b000640 1 -- 172.21.15.100:0/1960517645 shutdown_connections 2024-09-17T04:24:27.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.716+0000 7f922b000640 1 -- 172.21.15.100:0/1960517645 wait complete. 2024-09-17T04:24:27.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.717+0000 7fa650400640 1 Processor -- start 2024-09-17T04:24:27.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.718+0000 7fa650400640 1 -- start start 2024-09-17T04:24:27.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.719+0000 7f922b000640 1 Processor -- start 2024-09-17T04:24:27.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.719+0000 7f922b000640 1 -- start start 2024-09-17T04:24:27.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.721+0000 7fa650400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa630003a00 0x7fa64017dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.721+0000 7fa650400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa640179ce0 0x7fa64017a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.721+0000 7fa650400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6400ddc50 0x7fa6400de060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.721+0000 7fa650400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa640053bb0 con 0x7fa640179ce0 2024-09-17T04:24:27.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.721+0000 7fa650400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa640054630 con 0x7fa6400ddc50 2024-09-17T04:24:27.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.721+0000 7fa650400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa640053d30 con 0x7fa630003a00 2024-09-17T04:24:27.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.722+0000 7ff683e00640 1 -- 172.21.15.100:0/760315718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff67c007520 msgr2=0x7ff67c0e31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.722+0000 7ff683e00640 1 --2- 172.21.15.100:0/760315718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff67c007520 0x7ff67c0e31d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff67c007210 tx=0x7ff67402a580 comp rx=0 tx=0).stop 2024-09-17T04:24:27.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.722+0000 7f922b000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92240b7f40 0x7f922414bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.722+0000 7f922b000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f922415e770 0x7f92241f4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.722+0000 7fa64e400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6400ddc50 0x7fa6400de060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.722+0000 7fa64e400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6400ddc50 0x7fa6400de060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.100:36410/0 (socket says 172.21.15.100:36410) 2024-09-17T04:24:27.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.722+0000 7fa64e400640 1 -- 172.21.15.100:0/931237704 learned_addr learned my addr 172.21.15.100:0/931237704 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:27.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.722+0000 7f922b000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9224160d90 0x7f92241611a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.722+0000 7f922b000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f92240d3820 con 0x7f922415e770 2024-09-17T04:24:27.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.722+0000 7f922b000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f92240b68c0 con 0x7f9224160d90 2024-09-17T04:24:27.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.722+0000 7f922b000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f92240d39a0 con 0x7f92240b7f40 2024-09-17T04:24:27.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.722+0000 7f922a600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9224160d90 0x7f92241611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.723+0000 7f922a600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9224160d90 0x7f92241611a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.100:36414/0 (socket says 172.21.15.100:36414) 2024-09-17T04:24:27.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.723+0000 7f922a600640 1 -- 172.21.15.100:0/1090093263 learned_addr learned my addr 172.21.15.100:0/1090093263 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:27.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.723+0000 7f9229c00640 1 --2- 172.21.15.100:0/1090093263 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92240b7f40 0x7f922414bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.723+0000 7fa64da00640 1 --2- 172.21.15.100:0/931237704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa630003a00 0x7fa64017dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.723+0000 7ff683e00640 1 -- 172.21.15.100:0/760315718 shutdown_connections 2024-09-17T04:24:27.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.723+0000 7ff683e00640 1 --2- 172.21.15.100:0/760315718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff67c007520 0x7ff67c0e31d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.723+0000 7ff683e00640 1 -- 172.21.15.100:0/760315718 >> 172.21.15.100:0/760315718 conn(0x7ff67c03a650 msgr2=0x7ff67c03aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:27.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.723+0000 7fa64d000640 1 --2- 172.21.15.100:0/931237704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa640179ce0 0x7fa64017a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.723+0000 7ff683e00640 1 -- 172.21.15.100:0/760315718 shutdown_connections 2024-09-17T04:24:27.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.723+0000 7f9229200640 1 --2- 172.21.15.100:0/1090093263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f922415e770 0x7f92241f4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.723+0000 7fa64e400640 1 -- 172.21.15.100:0/931237704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa630003a00 msgr2=0x7fa64017dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.724+0000 7f9229200640 1 -- 172.21.15.100:0/1090093263 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92240b7f40 msgr2=0x7f922414bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.724+0000 7f9229200640 1 --2- 172.21.15.100:0/1090093263 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92240b7f40 0x7f922414bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.724+0000 7f9229200640 1 -- 172.21.15.100:0/1090093263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9224160d90 msgr2=0x7f92241611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.724+0000 7f9229200640 1 --2- 172.21.15.100:0/1090093263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9224160d90 0x7f92241611a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.724+0000 7f9229200640 1 -- 172.21.15.100:0/1090093263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92241616e0 con 0x7f922415e770 2024-09-17T04:24:27.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.723+0000 7fa64e400640 1 --2- 172.21.15.100:0/931237704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa630003a00 0x7fa64017dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.723+0000 7fa64e400640 1 -- 172.21.15.100:0/931237704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa640179ce0 msgr2=0x7fa64017a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.723+0000 7fa64e400640 1 --2- 172.21.15.100:0/931237704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa640179ce0 0x7fa64017a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.723+0000 7fa64e400640 1 -- 172.21.15.100:0/931237704 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64017ac60 con 0x7fa6400ddc50 2024-09-17T04:24:27.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.723+0000 7fa64da00640 1 --2- 172.21.15.100:0/931237704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa630003a00 0x7fa64017dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:24:27.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.724+0000 7fa64e400640 1 --2- 172.21.15.100:0/931237704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6400ddc50 0x7fa6400de060 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa64003b450 tx=0x7fa63c001710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.724+0000 7fa645600640 1 -- 172.21.15.100:0/931237704 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c011660 con 0x7fa6400ddc50 2024-09-17T04:24:27.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.724+0000 7f9229200640 1 --2- 172.21.15.100:0/1090093263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f922415e770 0x7f92241f4f90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f921000c940 tx=0x7f921000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.724+0000 7fa645600640 1 -- 172.21.15.100:0/931237704 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa63c017070 con 0x7fa6400ddc50 2024-09-17T04:24:27.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.724+0000 7fa645600640 1 -- 172.21.15.100:0/931237704 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c011660 con 0x7fa6400ddc50 2024-09-17T04:24:27.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.725+0000 7fa650400640 1 -- 172.21.15.100:0/931237704 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6400de5a0 con 0x7fa6400ddc50 2024-09-17T04:24:27.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.725+0000 7f9221600640 1 -- 172.21.15.100:0/1090093263 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210007bb0 con 0x7f922415e770 2024-09-17T04:24:27.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.725+0000 7f9221600640 1 -- 172.21.15.100:0/1090093263 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9210007e40 con 0x7f922415e770 2024-09-17T04:24:27.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.725+0000 7f9221600640 1 -- 172.21.15.100:0/1090093263 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921000f6b0 con 0x7f922415e770 2024-09-17T04:24:27.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.725+0000 7fa650400640 1 -- 172.21.15.100:0/931237704 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa64017a980 con 0x7fa6400ddc50 2024-09-17T04:24:27.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.725+0000 7f922b000640 1 -- 172.21.15.100:0/1090093263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f922414c2b0 con 0x7f922415e770 2024-09-17T04:24:27.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.725+0000 7f922b000640 1 -- 172.21.15.100:0/1090093263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92241f9390 con 0x7f922415e770 2024-09-17T04:24:27.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.727+0000 7f922b000640 1 -- 172.21.15.100:0/1090093263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92241ea480 con 0x7f922415e770 2024-09-17T04:24:27.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.727+0000 7fa650400640 1 -- 172.21.15.100:0/931237704 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa64017a6f0 con 0x7fa6400ddc50 2024-09-17T04:24:27.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.724+0000 7ff683e00640 1 -- 172.21.15.100:0/760315718 wait complete. 2024-09-17T04:24:27.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.728+0000 7ff6d1a00640 1 -- 172.21.15.100:0/91900939 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6cc166820 msgr2=0x7ff6cc166c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.728+0000 7ff6d1a00640 1 --2- 172.21.15.100:0/91900939 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6cc166820 0x7ff6cc166c30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4009830 tx=0x7ff6b402a520 comp rx=0 tx=0).stop 2024-09-17T04:24:27.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.729+0000 7ff6d1a00640 1 -- 172.21.15.100:0/91900939 shutdown_connections 2024-09-17T04:24:27.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.729+0000 7ff6d1a00640 1 --2- 172.21.15.100:0/91900939 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6cc166820 0x7ff6cc166c30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.729+0000 7ff6d1a00640 1 --2- 172.21.15.100:0/91900939 >> v2:172.21.15.100:3300/0 conn(0x7ff6cc06fc90 0x7ff6cc0700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.729+0000 7ff6d1a00640 1 -- 172.21.15.100:0/91900939 >> 172.21.15.100:0/91900939 conn(0x7ff6cc0b1e40 msgr2=0x7ff6cc0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:27.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.729+0000 7ff6d1a00640 1 -- 172.21.15.100:0/91900939 shutdown_connections 2024-09-17T04:24:27.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.729+0000 7ff6d1a00640 1 -- 172.21.15.100:0/91900939 wait complete. 2024-09-17T04:24:27.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.730+0000 7f9221600640 1 -- 172.21.15.100:0/1090093263 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9210018400 con 0x7f922415e770 2024-09-17T04:24:27.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.730+0000 7ff683e00640 1 Processor -- start 2024-09-17T04:24:27.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.731+0000 7f9221600640 1 --2- 172.21.15.100:0/1090093263 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9204078be0 0x7f920407b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.732+0000 7f9221600640 1 -- 172.21.15.100:0/1090093263 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f921009b8b0 con 0x7f922415e770 2024-09-17T04:24:27.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.732+0000 7ff6d1a00640 1 Processor -- start 2024-09-17T04:24:27.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.732+0000 7ff6d1a00640 1 -- start start 2024-09-17T04:24:27.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.730+0000 7ff683e00640 1 -- start start 2024-09-17T04:24:27.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.733+0000 7ff683e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff67c007520 0x7ff67c0dea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.733+0000 7ff683e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff67c053830 0x7ff67c17c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.733+0000 7ff683e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff67c054160 0x7ff67c17c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.733+0000 7ff682a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff67c007520 0x7ff67c0dea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.734+0000 7ff682a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff67c007520 0x7ff67c0dea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.100:53528/0 (socket says 172.21.15.100:53528) 2024-09-17T04:24:27.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.734+0000 7ff682a00640 1 -- 172.21.15.100:0/3380945489 learned_addr learned my addr 172.21.15.100:0/3380945489 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:27.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.733+0000 7fa645600640 1 -- 172.21.15.100:0/931237704 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa63c011930 con 0x7fa6400ddc50 2024-09-17T04:24:27.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.734+0000 7ff683e00640 1 -- 172.21.15.100:0/3380945489 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c03ccb0 con 0x7ff67c054160 2024-09-17T04:24:27.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.734+0000 7ff683e00640 1 -- 172.21.15.100:0/3380945489 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c03cb30 con 0x7ff67c053830 2024-09-17T04:24:27.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.734+0000 7ff6d1a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6cc06fc90 0x7ff6cc156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.734+0000 7ff6d1a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6cc15cb70 0x7ff6cc0b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.734+0000 7ff6d1a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6cc166820 0x7ff6cc0b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.734+0000 7ff683e00640 1 -- 172.21.15.100:0/3380945489 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff67c03ce30 con 0x7ff67c007520 2024-09-17T04:24:27.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.735+0000 7ff682000640 1 --2- 172.21.15.100:0/3380945489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff67c053830 0x7ff67c17c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.734+0000 7ff6d1a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc164570 con 0x7ff6cc166820 2024-09-17T04:24:27.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.735+0000 7ff6d1a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc1643f0 con 0x7ff6cc06fc90 2024-09-17T04:24:27.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.735+0000 7ff6cbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6cc06fc90 0x7ff6cc156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.735+0000 7ff683400640 1 --2- 172.21.15.100:0/3380945489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff67c054160 0x7ff67c17c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.735+0000 7ff6d1a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff6cc0b3ae0 con 0x7ff6cc15cb70 2024-09-17T04:24:27.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.735+0000 7ff6cbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6cc06fc90 0x7ff6cc156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.100:36426/0 (socket says 172.21.15.100:36426) 2024-09-17T04:24:27.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.735+0000 7ff6cbe00640 1 -- 172.21.15.100:0/2555528862 learned_addr learned my addr 172.21.15.100:0/2555528862 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:27.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.735+0000 7ff6c3e00640 1 --2- 172.21.15.100:0/2555528862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6cc15cb70 0x7ff6cc0b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.735+0000 7ff6d1000640 1 --2- 172.21.15.100:0/2555528862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6cc166820 0x7ff6cc0b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.732+0000 7fcc4ea00640 1 -- 172.21.15.100:0/1120943636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc480b3720 msgr2=0x7fcc480b3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.732+0000 7fcc4ea00640 1 --2- 172.21.15.100:0/1120943636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc480b3720 0x7fcc480b3b30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcc38009830 tx=0x7fcc3802a520 comp rx=0 tx=0).stop 2024-09-17T04:24:27.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.736+0000 7fa645600640 1 --2- 172.21.15.100:0/931237704 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa628078d90 0x7fa62807b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.736+0000 7f9229c00640 1 --2- 172.21.15.100:0/1090093263 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9204078be0 0x7f920407b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.736+0000 7ff683400640 1 -- 172.21.15.100:0/3380945489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff67c007520 msgr2=0x7ff67c0dea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.736+0000 7ff683400640 1 --2- 172.21.15.100:0/3380945489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff67c007520 0x7ff67c0dea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.736+0000 7ff683400640 1 -- 172.21.15.100:0/3380945489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff67c053830 msgr2=0x7ff67c17c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.736+0000 7ff683400640 1 --2- 172.21.15.100:0/3380945489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff67c053830 0x7ff67c17c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.736+0000 7ff683400640 1 -- 172.21.15.100:0/3380945489 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff67c187650 con 0x7ff67c054160 2024-09-17T04:24:27.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.736+0000 7ff6c3e00640 1 -- 172.21.15.100:0/2555528862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6cc06fc90 msgr2=0x7ff6cc156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.737+0000 7fcc4ea00640 1 -- 172.21.15.100:0/1120943636 shutdown_connections 2024-09-17T04:24:27.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.737+0000 7fa645600640 1 -- 172.21.15.100:0/931237704 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa63c09afb0 con 0x7fa6400ddc50 2024-09-17T04:24:27.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.737+0000 7fa64da00640 1 --2- 172.21.15.100:0/931237704 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa628078d90 0x7fa62807b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.737+0000 7fa645600640 1 -- 172.21.15.100:0/931237704 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa6400de5a0 con 0x7fa6400ddc50 2024-09-17T04:24:27.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.737+0000 7fcc4ea00640 1 --2- 172.21.15.100:0/1120943636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc480b3720 0x7fcc480b3b30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.737+0000 7fcc4ea00640 1 --2- 172.21.15.100:0/1120943636 >> v2:172.21.15.100:3300/0 conn(0x7fcc48170690 0x7fcc480b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.737+0000 7ff6c3e00640 1 --2- 172.21.15.100:0/2555528862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6cc06fc90 0x7ff6cc156ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.737+0000 7ff6c3e00640 1 -- 172.21.15.100:0/2555528862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6cc166820 msgr2=0x7ff6cc0b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.737+0000 7ff6c3e00640 1 --2- 172.21.15.100:0/2555528862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6cc166820 0x7ff6cc0b14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.737+0000 7ff6c3e00640 1 -- 172.21.15.100:0/2555528862 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6cc1570e0 con 0x7ff6cc15cb70 2024-09-17T04:24:27.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.737+0000 7fcc4ea00640 1 -- 172.21.15.100:0/1120943636 >> 172.21.15.100:0/1120943636 conn(0x7fcc480d02f0 msgr2=0x7fcc480d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:27.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.737+0000 7ff6d1000640 1 --2- 172.21.15.100:0/2555528862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6cc166820 0x7ff6cc0b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:24:27.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.737+0000 7ff6cbe00640 1 --2- 172.21.15.100:0/2555528862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6cc06fc90 0x7ff6cc156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:24:27.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.737+0000 7ff683400640 1 --2- 172.21.15.100:0/3380945489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff67c054160 0x7ff67c17c8c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00c4d0 tx=0x7ff66c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.737+0000 7ff6c3e00640 1 --2- 172.21.15.100:0/2555528862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6cc15cb70 0x7ff6cc0b0f60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0b4c70 tx=0x7ff6b4033ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.737+0000 7fcc4ea00640 1 -- 172.21.15.100:0/1120943636 shutdown_connections 2024-09-17T04:24:27.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.737+0000 7ff682000640 1 --2- 172.21.15.100:0/3380945489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff67c053830 0x7ff67c17c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:24:27.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.738+0000 7f9221600640 1 -- 172.21.15.100:0/1090093263 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f92241ea480 con 0x7f922415e770 2024-09-17T04:24:27.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.738+0000 7fcc4ea00640 1 -- 172.21.15.100:0/1120943636 wait complete. 2024-09-17T04:24:27.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.738+0000 7f9229c00640 1 --2- 172.21.15.100:0/1090093263 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9204078be0 0x7f920407b0f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9218035000 tx=0x7f9218004ab0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.739+0000 7fa64da00640 1 --2- 172.21.15.100:0/931237704 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa628078d90 0x7fa62807b2a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa63400c470 tx=0x7fa63402d070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.738+0000 7ff67aa00640 1 -- 172.21.15.100:0/3380945489 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c017070 con 0x7ff67c054160 2024-09-17T04:24:27.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.738+0000 7ff67aa00640 1 -- 172.21.15.100:0/3380945489 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff66c0052d0 con 0x7ff67c054160 2024-09-17T04:24:27.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.738+0000 7ff67aa00640 1 -- 172.21.15.100:0/3380945489 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c003850 con 0x7ff67c054160 2024-09-17T04:24:27.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.738+0000 7ff683e00640 1 -- 172.21.15.100:0/3380945489 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff67c187930 con 0x7ff67c054160 2024-09-17T04:24:27.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.738+0000 7ff683e00640 1 -- 172.21.15.100:0/3380945489 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff67c187ef0 con 0x7ff67c054160 2024-09-17T04:24:27.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.740+0000 7ff683e00640 1 -- 172.21.15.100:0/3380945489 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff67c0e31d0 con 0x7ff67c054160 2024-09-17T04:24:27.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.738+0000 7ff6c8c00640 1 -- 172.21.15.100:0/2555528862 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4031d60 con 0x7ff6cc15cb70 2024-09-17T04:24:27.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.738+0000 7ff6c8c00640 1 -- 172.21.15.100:0/2555528862 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6b4033e50 con 0x7ff6cc15cb70 2024-09-17T04:24:27.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.738+0000 7ff6c8c00640 1 -- 172.21.15.100:0/2555528862 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4033330 con 0x7ff6cc15cb70 2024-09-17T04:24:27.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.738+0000 7ff6d1a00640 1 -- 172.21.15.100:0/2555528862 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6cc1f9ec0 con 0x7ff6cc15cb70 2024-09-17T04:24:27.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.739+0000 7ff6d1a00640 1 -- 172.21.15.100:0/2555528862 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6cc1fa480 con 0x7ff6cc15cb70 2024-09-17T04:24:27.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.742+0000 7fcc4ea00640 1 Processor -- start 2024-09-17T04:24:27.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.744+0000 7ff67aa00640 1 -- 172.21.15.100:0/3380945489 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff66c01a070 con 0x7ff67c054160 2024-09-17T04:24:27.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.742+0000 7fcc4ea00640 1 -- start start 2024-09-17T04:24:27.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.744+0000 7fcc4ea00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc480b3720 0x7fcc480b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.744+0000 7fcc4ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc480b5ab0 0x7fcc4809c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.745+0000 7fcc4ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc48170690 0x7fcc4820e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.745+0000 7fcc4ea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcc480995c0 con 0x7fcc480b5ab0 2024-09-17T04:24:27.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.745+0000 7fcc4ea00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcc48099440 con 0x7fcc48170690 2024-09-17T04:24:27.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.745+0000 7fcc4ea00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcc480cfbc0 con 0x7fcc480b3720 2024-09-17T04:24:27.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.745+0000 7fcc47e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc480b3720 0x7fcc480b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.745+0000 7fcc47e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc480b3720 0x7fcc480b0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.100:53538/0 (socket says 172.21.15.100:53538) 2024-09-17T04:24:27.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.745+0000 7fcc47e00640 1 -- 172.21.15.100:0/1425451415 learned_addr learned my addr 172.21.15.100:0/1425451415 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:27.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.746+0000 7fcc4ca00640 1 --2- 172.21.15.100:0/1425451415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc48170690 0x7fcc4820e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.746+0000 7ff67aa00640 1 --2- 172.21.15.100:0/3380945489 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff660078c60 0x7ff66007b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.746+0000 7fcc47e00640 1 -- 172.21.15.100:0/1425451415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc48170690 msgr2=0x7fcc4820e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.746+0000 7ff67aa00640 1 -- 172.21.15.100:0/3380945489 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff66c09a280 con 0x7ff67c054160 2024-09-17T04:24:27.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.746+0000 7fcc47e00640 1 --2- 172.21.15.100:0/1425451415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc48170690 0x7fcc4820e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.746+0000 7fcc47e00640 1 -- 172.21.15.100:0/1425451415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc480b5ab0 msgr2=0x7fcc4809c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.746+0000 7fcc47e00640 1 --2- 172.21.15.100:0/1425451415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc480b5ab0 0x7fcc4809c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.746+0000 7fcc47e00640 1 -- 172.21.15.100:0/1425451415 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4820add0 con 0x7fcc480b3720 2024-09-17T04:24:27.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.747+0000 7fcc47e00640 1 --2- 172.21.15.100:0/1425451415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc480b3720 0x7fcc480b0bb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400c3a0 tx=0x7fcc3400c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.747+0000 7fcc3fe00640 1 -- 172.21.15.100:0/1425451415 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34017070 con 0x7fcc480b3720 2024-09-17T04:24:27.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.747+0000 7fcc3fe00640 1 -- 172.21.15.100:0/1425451415 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc340053c0 con 0x7fcc480b3720 2024-09-17T04:24:27.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.747+0000 7fcc3fe00640 1 -- 172.21.15.100:0/1425451415 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34003850 con 0x7fcc480b3720 2024-09-17T04:24:27.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.747+0000 7fcc4ea00640 1 -- 172.21.15.100:0/1425451415 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc4820b0b0 con 0x7fcc480b3720 2024-09-17T04:24:27.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.748+0000 7fcc4ea00640 1 -- 172.21.15.100:0/1425451415 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc4820b670 con 0x7fcc480b3720 2024-09-17T04:24:27.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.744+0000 7ff6c8c00640 1 -- 172.21.15.100:0/2555528862 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff6b40334d0 con 0x7ff6cc15cb70 2024-09-17T04:24:27.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.744+0000 7ff6d1a00640 1 -- 172.21.15.100:0/2555528862 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6a4005290 con 0x7ff6cc15cb70 2024-09-17T04:24:27.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.745+0000 7ff6c8c00640 1 --2- 172.21.15.100:0/2555528862 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff6a8078bc0 0x7ff6a807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.745+0000 7ff6c8c00640 1 -- 172.21.15.100:0/2555528862 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff6b40bbc10 con 0x7ff6cc15cb70 2024-09-17T04:24:27.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.749+0000 7ff682a00640 1 --2- 172.21.15.100:0/3380945489 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff660078c60 0x7ff66007b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.749+0000 7ff6cbe00640 1 --2- 172.21.15.100:0/2555528862 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff6a8078bc0 0x7ff6a807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.749+0000 7fcc4ea00640 1 -- 172.21.15.100:0/1425451415 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc480b31e0 con 0x7fcc480b3720 2024-09-17T04:24:27.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.750+0000 7ff6cbe00640 1 --2- 172.21.15.100:0/2555528862 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff6a8078bc0 0x7ff6a807b0d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc0058c0 tx=0x7ff6bc005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.750+0000 7ff682a00640 1 --2- 172.21.15.100:0/3380945489 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff660078c60 0x7ff66007b170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff68406c570 tx=0x7ff684068d00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.750+0000 7ff67aa00640 1 -- 172.21.15.100:0/3380945489 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff66c021020 con 0x7ff67c054160 2024-09-17T04:24:27.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.751+0000 7fcc3fe00640 1 -- 172.21.15.100:0/1425451415 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcc3401a070 con 0x7fcc480b3720 2024-09-17T04:24:27.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.752+0000 7fcc3fe00640 1 --2- 172.21.15.100:0/1425451415 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fcc28078c10 0x7fcc2807b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.752+0000 7fcc3fe00640 1 -- 172.21.15.100:0/1425451415 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fcc3405e0d0 con 0x7fcc480b3720 2024-09-17T04:24:27.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.752+0000 7fcc47400640 1 --2- 172.21.15.100:0/1425451415 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fcc28078c10 0x7fcc2807b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.754+0000 7fcc47400640 1 --2- 172.21.15.100:0/1425451415 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fcc28078c10 0x7fcc2807b120 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009330 tx=0x7fcc40009280 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.756+0000 7ff6c8c00640 1 -- 172.21.15.100:0/2555528862 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff6b404a030 con 0x7ff6cc15cb70 2024-09-17T04:24:27.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.761+0000 7fcc3fe00640 1 -- 172.21.15.100:0/1425451415 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fcc480b5ec0 con 0x7fcc480b3720 2024-09-17T04:24:27.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.771+0000 7f2332a00640 1 -- 172.21.15.100:0/2883558048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f232c0d13a0 msgr2=0x7f232c15e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.771+0000 7f232be00640 1 -- 172.21.15.100:0/2883558048 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23180047a0 con 0x7f232c0d13a0 2024-09-17T04:24:27.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.771+0000 7f2332a00640 1 --2- 172.21.15.100:0/2883558048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f232c0d13a0 0x7f232c15e9c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2318009900 tx=0x7f231802a580 comp rx=0 tx=0).stop 2024-09-17T04:24:27.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.771+0000 7f2332a00640 1 -- 172.21.15.100:0/2883558048 shutdown_connections 2024-09-17T04:24:27.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.771+0000 7f2332a00640 1 --2- 172.21.15.100:0/2883558048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f232c0d13a0 0x7f232c15e9c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.771+0000 7f2332a00640 1 --2- 172.21.15.100:0/2883558048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f232c0d2590 0x7f232c0b3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.771+0000 7f2332a00640 1 -- 172.21.15.100:0/2883558048 >> 172.21.15.100:0/2883558048 conn(0x7f232c06dd60 msgr2=0x7f232c06e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:27.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.772+0000 7f2332a00640 1 -- 172.21.15.100:0/2883558048 shutdown_connections 2024-09-17T04:24:27.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.772+0000 7f2332a00640 1 -- 172.21.15.100:0/2883558048 wait complete. 2024-09-17T04:24:27.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.776+0000 7f2332a00640 1 Processor -- start 2024-09-17T04:24:27.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.776+0000 7f2332a00640 1 -- start start 2024-09-17T04:24:27.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.778+0000 7f2332a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f232c0d13a0 0x7f232c14f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.778+0000 7f2332a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f232c0d2590 0x7f232c070aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.779+0000 7f2332a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f232c163150 0x7f232c160da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.779+0000 7f2330c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f232c0d2590 0x7f232c070aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.779+0000 7f2330c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f232c0d2590 0x7f232c070aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.100:53564/0 (socket says 172.21.15.100:53564) 2024-09-17T04:24:27.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.779+0000 7f2332000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f232c163150 0x7f232c160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.779+0000 7f2332a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f232c0cf6a0 con 0x7f232c163150 2024-09-17T04:24:27.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.779+0000 7f2332a00640 1 -- 172.21.15.100:0/1369845105 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f232c0cf520 con 0x7f232c0d13a0 2024-09-17T04:24:27.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.779+0000 7f2330c00640 1 -- 172.21.15.100:0/1369845105 learned_addr learned my addr 172.21.15.100:0/1369845105 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:27.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.779+0000 7f2332a00640 1 -- 172.21.15.100:0/1369845105 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f232c0cf820 con 0x7f232c0d2590 2024-09-17T04:24:27.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.780+0000 7f2331600640 1 --2- 172.21.15.100:0/1369845105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f232c0d13a0 0x7f232c14f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.783+0000 7f2330c00640 1 -- 172.21.15.100:0/1369845105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f232c0d13a0 msgr2=0x7f232c14f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.783+0000 7f2330c00640 1 --2- 172.21.15.100:0/1369845105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f232c0d13a0 0x7f232c14f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.783+0000 7f2330c00640 1 -- 172.21.15.100:0/1369845105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f232c163150 msgr2=0x7f232c160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:27.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.783+0000 7f2330c00640 1 --2- 172.21.15.100:0/1369845105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f232c163150 0x7f232c160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:27.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.784+0000 7f2330c00640 1 -- 172.21.15.100:0/1369845105 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232c14fb60 con 0x7f232c0d2590 2024-09-17T04:24:27.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.784+0000 7f2331600640 1 --2- 172.21.15.100:0/1369845105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f232c0d13a0 0x7f232c14f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:24:27.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.784+0000 7f2332000640 1 --2- 172.21.15.100:0/1369845105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f232c163150 0x7f232c160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:24:27.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.784+0000 7f2330c00640 1 --2- 172.21.15.100:0/1369845105 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f232c0d2590 0x7f232c070aa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f232000e7c0 tx=0x7f232000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:27.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.785+0000 7f2329600640 1 -- 172.21.15.100:0/1369845105 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232000cc60 con 0x7f232c0d2590 2024-09-17T04:24:27.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.785+0000 7f2329600640 1 -- 172.21.15.100:0/1369845105 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f232000ee70 con 0x7f232c0d2590 2024-09-17T04:24:27.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.785+0000 7f2329600640 1 -- 172.21.15.100:0/1369845105 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2320010690 con 0x7f232c0d2590 2024-09-17T04:24:27.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.785+0000 7f2332a00640 1 -- 172.21.15.100:0/1369845105 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f232c14fe40 con 0x7f232c0d2590 2024-09-17T04:24:27.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.785+0000 7f2332a00640 1 -- 172.21.15.100:0/1369845105 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f232c150400 con 0x7f232c0d2590 2024-09-17T04:24:27.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.789+0000 7f2332a00640 1 -- 172.21.15.100:0/1369845105 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f232c069660 con 0x7f232c0d2590 2024-09-17T04:24:27.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.793+0000 7f2329600640 1 -- 172.21.15.100:0/1369845105 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2320010830 con 0x7f232c0d2590 2024-09-17T04:24:27.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.794+0000 7f2329600640 1 --2- 172.21.15.100:0/1369845105 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f230c078b90 0x7f230c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:27.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.794+0000 7f2329600640 1 -- 172.21.15.100:0/1369845105 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f232009bf10 con 0x7f232c0d2590 2024-09-17T04:24:27.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.795+0000 7f2331600640 1 --2- 172.21.15.100:0/1369845105 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f230c078b90 0x7f230c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:27.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.795+0000 7f2329600640 1 -- 172.21.15.100:0/1369845105 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f2320063ce0 con 0x7f232c0d2590 2024-09-17T04:24:27.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:27.796+0000 7f2331600640 1 --2- 172.21.15.100:0/1369845105 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f230c078b90 0x7f230c07b0a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f232c0d0340 tx=0x7f2324009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:28.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.066+0000 7f0f6aa00640 1 -- 172.21.15.100:0/1309031702 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f0f40005580 con 0x7f0f5c053680 2024-09-17T04:24:28.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.067+0000 7f0f57e00640 1 -- 172.21.15.100:0/1309031702 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0f58087a90 con 0x7f0f5c053680 2024-09-17T04:24:28.067 INFO:teuthology.orchestra.run.smithi100.stdout:64424509446 2024-09-17T04:24:28.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.071+0000 7f0f6aa00640 1 -- 172.21.15.100:0/1309031702 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0f44078b90 msgr2=0x7f0f4407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:28.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.071+0000 7f0f6aa00640 1 --2- 172.21.15.100:0/1309031702 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0f44078b90 0x7f0f4407b0a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c0d2450 tx=0x7f0f50005f90 comp rx=0 tx=0).stop 2024-09-17T04:24:28.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.072+0000 7f0f6aa00640 1 -- 172.21.15.100:0/1309031702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f5c053680 msgr2=0x7f0f5c052750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:28.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.072+0000 7f0f6aa00640 1 --2- 172.21.15.100:0/1309031702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f5c053680 0x7f0f5c052750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0f58000c00 tx=0x7f0f58003820 comp rx=0 tx=0).stop 2024-09-17T04:24:28.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.072+0000 7f0f6aa00640 1 -- 172.21.15.100:0/1309031702 shutdown_connections 2024-09-17T04:24:28.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.072+0000 7f0f6aa00640 1 --2- 172.21.15.100:0/1309031702 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0f44078b90 0x7f0f4407b0a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.072+0000 7f0f6aa00640 1 --2- 172.21.15.100:0/1309031702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f5c0e31f0 0x7f0f5c0ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.072+0000 7f0f6aa00640 1 --2- 172.21.15.100:0/1309031702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c054180 0x7f0f5c0cddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.072+0000 7f0f6aa00640 1 --2- 172.21.15.100:0/1309031702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f5c053680 0x7f0f5c052750 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.072+0000 7f0f6aa00640 1 -- 172.21.15.100:0/1309031702 >> 172.21.15.100:0/1309031702 conn(0x7f0f5c03a670 msgr2=0x7f0f5c04d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:28.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.073+0000 7f0f6aa00640 1 -- 172.21.15.100:0/1309031702 shutdown_connections 2024-09-17T04:24:28.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.073+0000 7f0f6aa00640 1 -- 172.21.15.100:0/1309031702 wait complete. 2024-09-17T04:24:28.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.087+0000 7fdeef400640 1 -- 172.21.15.100:0/3637359028 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fdf04154060 con 0x7fdf040af3f0 2024-09-17T04:24:28.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.088+0000 7fdefa000640 1 -- 172.21.15.100:0/3637359028 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdf00067200 con 0x7fdf040af3f0 2024-09-17T04:24:28.091 INFO:teuthology.orchestra.run.smithi100.stdout:64424509447 2024-09-17T04:24:28.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.097+0000 7fdeef400640 1 -- 172.21.15.100:0/3637359028 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdee8078bc0 msgr2=0x7fdee807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:28.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.097+0000 7fdeef400640 1 --2- 172.21.15.100:0/3637359028 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdee8078bc0 0x7fdee807b0d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdef4033000 tx=0x7fdef4004730 comp rx=0 tx=0).stop 2024-09-17T04:24:28.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.097+0000 7fdeef400640 1 -- 172.21.15.100:0/3637359028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf040af3f0 msgr2=0x7fdf040cb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:28.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.097+0000 7fdeef400640 1 --2- 172.21.15.100:0/3637359028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf040af3f0 0x7fdf040cb9f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000ba50 tx=0x7fdf0000bf10 comp rx=0 tx=0).stop 2024-09-17T04:24:28.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.097+0000 7fdeef400640 1 -- 172.21.15.100:0/3637359028 shutdown_connections 2024-09-17T04:24:28.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.097+0000 7fdeef400640 1 --2- 172.21.15.100:0/3637359028 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdee8078bc0 0x7fdee807b0d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.097+0000 7fdeef400640 1 --2- 172.21.15.100:0/3637359028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf0415add0 0x7fdf04155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.097+0000 7fdeef400640 1 --2- 172.21.15.100:0/3637359028 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf040b30d0 0x7fdf04159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.097+0000 7fdeef400640 1 --2- 172.21.15.100:0/3637359028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf040af3f0 0x7fdf040cb9f0 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000ba50 tx=0x7fdf0000bf10 comp rx=0 tx=0).stop 2024-09-17T04:24:28.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.097+0000 7fdeef400640 1 -- 172.21.15.100:0/3637359028 >> 172.21.15.100:0/3637359028 conn(0x7fdf040742a0 msgr2=0x7fdf04068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:28.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.099+0000 7fdeef400640 1 -- 172.21.15.100:0/3637359028 shutdown_connections 2024-09-17T04:24:28.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.099+0000 7fdeef400640 1 -- 172.21.15.100:0/3637359028 wait complete. 2024-09-17T04:24:28.111 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.0 2024-09-17T04:24:28.125 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.1 2024-09-17T04:24:28.125 DEBUG:teuthology.parallel:result is None 2024-09-17T04:24:28.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.178+0000 7ff6d1a00640 1 -- 172.21.15.100:0/2555528862 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7ff6a4005580 con 0x7ff6cc15cb70 2024-09-17T04:24:28.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.179+0000 7ff6c8c00640 1 -- 172.21.15.100:0/2555528862 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff6b403f030 con 0x7ff6cc15cb70 2024-09-17T04:24:28.180 INFO:teuthology.orchestra.run.smithi100.stdout:60129542153 2024-09-17T04:24:28.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.180+0000 7ff683e00640 1 -- 172.21.15.100:0/3380945489 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ff67c1873e0 con 0x7ff67c054160 2024-09-17T04:24:28.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.180+0000 7ff67aa00640 1 -- 172.21.15.100:0/3380945489 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff66c0626a0 con 0x7ff67c054160 2024-09-17T04:24:28.181 INFO:teuthology.orchestra.run.smithi100.stdout:68719476744 2024-09-17T04:24:28.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.183+0000 7ff6d1a00640 1 -- 172.21.15.100:0/2555528862 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff6a8078bc0 msgr2=0x7ff6a807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:28.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.183+0000 7ff6d1a00640 1 --2- 172.21.15.100:0/2555528862 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff6a8078bc0 0x7ff6a807b0d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc0058c0 tx=0x7ff6bc005f90 comp rx=0 tx=0).stop 2024-09-17T04:24:28.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.183+0000 7ff6d1a00640 1 -- 172.21.15.100:0/2555528862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6cc15cb70 msgr2=0x7ff6cc0b0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:28.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.183+0000 7ff6d1a00640 1 --2- 172.21.15.100:0/2555528862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6cc15cb70 0x7ff6cc0b0f60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0b4c70 tx=0x7ff6b4033ba0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.183+0000 7ff6d1a00640 1 -- 172.21.15.100:0/2555528862 shutdown_connections 2024-09-17T04:24:28.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.183+0000 7ff6d1a00640 1 --2- 172.21.15.100:0/2555528862 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff6a8078bc0 0x7ff6a807b0d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.183+0000 7ff6d1a00640 1 --2- 172.21.15.100:0/2555528862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6cc166820 0x7ff6cc0b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.183+0000 7ff6d1a00640 1 --2- 172.21.15.100:0/2555528862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6cc15cb70 0x7ff6cc0b0f60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.183+0000 7ff6d1a00640 1 --2- 172.21.15.100:0/2555528862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6cc06fc90 0x7ff6cc156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.183+0000 7ff6d1a00640 1 -- 172.21.15.100:0/2555528862 >> 172.21.15.100:0/2555528862 conn(0x7ff6cc0b1e40 msgr2=0x7ff6cc0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:28.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.183+0000 7ff6d1a00640 1 -- 172.21.15.100:0/2555528862 shutdown_connections 2024-09-17T04:24:28.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.183+0000 7ff6d1a00640 1 -- 172.21.15.100:0/2555528862 wait complete. 2024-09-17T04:24:28.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.184+0000 7ff683e00640 1 -- 172.21.15.100:0/3380945489 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff660078c60 msgr2=0x7ff66007b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:28.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.184+0000 7ff683e00640 1 --2- 172.21.15.100:0/3380945489 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff660078c60 0x7ff66007b170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff68406c570 tx=0x7ff684068d00 comp rx=0 tx=0).stop 2024-09-17T04:24:28.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.184+0000 7ff683e00640 1 -- 172.21.15.100:0/3380945489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff67c054160 msgr2=0x7ff67c17c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:28.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.184+0000 7ff683e00640 1 --2- 172.21.15.100:0/3380945489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff67c054160 0x7ff67c17c8c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00c4d0 tx=0x7ff66c00c8d0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.184+0000 7ff683e00640 1 -- 172.21.15.100:0/3380945489 shutdown_connections 2024-09-17T04:24:28.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.184+0000 7ff683e00640 1 --2- 172.21.15.100:0/3380945489 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff660078c60 0x7ff66007b170 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.184+0000 7ff683e00640 1 --2- 172.21.15.100:0/3380945489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff67c054160 0x7ff67c17c8c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.184+0000 7ff683e00640 1 --2- 172.21.15.100:0/3380945489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff67c053830 0x7ff67c17c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.184+0000 7ff683e00640 1 --2- 172.21.15.100:0/3380945489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff67c007520 0x7ff67c0dea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.184+0000 7ff683e00640 1 -- 172.21.15.100:0/3380945489 >> 172.21.15.100:0/3380945489 conn(0x7ff67c03a650 msgr2=0x7ff67c04e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:28.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.184+0000 7ff683e00640 1 -- 172.21.15.100:0/3380945489 shutdown_connections 2024-09-17T04:24:28.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.184+0000 7ff683e00640 1 -- 172.21.15.100:0/3380945489 wait complete. 2024-09-17T04:24:28.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.186+0000 7fa650400640 1 -- 172.21.15.100:0/931237704 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fa64017e180 con 0x7fa6400ddc50 2024-09-17T04:24:28.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.186+0000 7fa645600640 1 -- 172.21.15.100:0/931237704 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa64017ac60 con 0x7fa6400ddc50 2024-09-17T04:24:28.188 INFO:teuthology.orchestra.run.smithi100.stdout:68719476745 2024-09-17T04:24:28.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.190+0000 7fcc4ea00640 1 -- 172.21.15.100:0/1425451415 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fcc480b1860 con 0x7fcc480b3720 2024-09-17T04:24:28.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.190+0000 7fa650400640 1 -- 172.21.15.100:0/931237704 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa628078d90 msgr2=0x7fa62807b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:28.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.190+0000 7fa650400640 1 --2- 172.21.15.100:0/931237704 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa628078d90 0x7fa62807b2a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa63400c470 tx=0x7fa63402d070 comp rx=0 tx=0).stop 2024-09-17T04:24:28.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.190+0000 7fa650400640 1 -- 172.21.15.100:0/931237704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6400ddc50 msgr2=0x7fa6400de060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:28.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.190+0000 7fa650400640 1 --2- 172.21.15.100:0/931237704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6400ddc50 0x7fa6400de060 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa64003b450 tx=0x7fa63c001710 comp rx=0 tx=0).stop 2024-09-17T04:24:28.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.191+0000 7fa650400640 1 -- 172.21.15.100:0/931237704 shutdown_connections 2024-09-17T04:24:28.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.191+0000 7fa650400640 1 --2- 172.21.15.100:0/931237704 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa628078d90 0x7fa62807b2a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.191+0000 7fa650400640 1 --2- 172.21.15.100:0/931237704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6400ddc50 0x7fa6400de060 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.191+0000 7fa650400640 1 --2- 172.21.15.100:0/931237704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa640179ce0 0x7fa64017a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.191+0000 7fa650400640 1 --2- 172.21.15.100:0/931237704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa630003a00 0x7fa64017dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.191+0000 7fa650400640 1 -- 172.21.15.100:0/931237704 >> 172.21.15.100:0/931237704 conn(0x7fa64003a5e0 msgr2=0x7fa64003c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:28.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.191+0000 7fa650400640 1 -- 172.21.15.100:0/931237704 shutdown_connections 2024-09-17T04:24:28.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.191+0000 7fcc3fe00640 1 -- 172.21.15.100:0/1425451415 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcc480b5ec0 con 0x7fcc480b3720 2024-09-17T04:24:28.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.191+0000 7fa650400640 1 -- 172.21.15.100:0/931237704 wait complete. 2024-09-17T04:24:28.193 INFO:teuthology.orchestra.run.smithi100.stdout:68719476745 2024-09-17T04:24:28.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.194+0000 7fcc4ea00640 1 -- 172.21.15.100:0/1425451415 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fcc28078c10 msgr2=0x7fcc2807b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:28.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.194+0000 7fcc4ea00640 1 --2- 172.21.15.100:0/1425451415 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fcc28078c10 0x7fcc2807b120 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009330 tx=0x7fcc40009280 comp rx=0 tx=0).stop 2024-09-17T04:24:28.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.194+0000 7fcc4ea00640 1 -- 172.21.15.100:0/1425451415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc480b3720 msgr2=0x7fcc480b0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:28.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.194+0000 7fcc4ea00640 1 --2- 172.21.15.100:0/1425451415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc480b3720 0x7fcc480b0bb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400c3a0 tx=0x7fcc3400c860 comp rx=0 tx=0).stop 2024-09-17T04:24:28.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.195+0000 7fcc4ea00640 1 -- 172.21.15.100:0/1425451415 shutdown_connections 2024-09-17T04:24:28.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.195+0000 7fcc4ea00640 1 --2- 172.21.15.100:0/1425451415 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fcc28078c10 0x7fcc2807b120 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.195+0000 7f922b000640 1 -- 172.21.15.100:0/1090093263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f9224051080 con 0x7f922415e770 2024-09-17T04:24:28.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.195+0000 7fcc4ea00640 1 --2- 172.21.15.100:0/1425451415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc48170690 0x7fcc4820e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.195+0000 7fcc4ea00640 1 --2- 172.21.15.100:0/1425451415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc480b5ab0 0x7fcc4809c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.195+0000 7fcc4ea00640 1 --2- 172.21.15.100:0/1425451415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc480b3720 0x7fcc480b0bb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.195+0000 7fcc4ea00640 1 -- 172.21.15.100:0/1425451415 >> 172.21.15.100:0/1425451415 conn(0x7fcc480d02f0 msgr2=0x7fcc480b3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:28.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.195+0000 7fcc4ea00640 1 -- 172.21.15.100:0/1425451415 shutdown_connections 2024-09-17T04:24:28.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.195+0000 7fcc4ea00640 1 -- 172.21.15.100:0/1425451415 wait complete. 2024-09-17T04:24:28.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.197+0000 7f9221600640 1 -- 172.21.15.100:0/1090093263 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9224051080 con 0x7f922415e770 2024-09-17T04:24:28.197 INFO:teuthology.orchestra.run.smithi100.stdout:64424509448 2024-09-17T04:24:28.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.201+0000 7f922b000640 1 -- 172.21.15.100:0/1090093263 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9204078be0 msgr2=0x7f920407b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:28.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.201+0000 7f922b000640 1 --2- 172.21.15.100:0/1090093263 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9204078be0 0x7f920407b0f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9218035000 tx=0x7f9218004ab0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.201+0000 7f922b000640 1 -- 172.21.15.100:0/1090093263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f922415e770 msgr2=0x7f92241f4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:28.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.201+0000 7f922b000640 1 --2- 172.21.15.100:0/1090093263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f922415e770 0x7f92241f4f90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f921000c940 tx=0x7f921000ce00 comp rx=0 tx=0).stop 2024-09-17T04:24:28.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.201+0000 7f922b000640 1 -- 172.21.15.100:0/1090093263 shutdown_connections 2024-09-17T04:24:28.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.201+0000 7f922b000640 1 --2- 172.21.15.100:0/1090093263 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9204078be0 0x7f920407b0f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.201+0000 7f922b000640 1 --2- 172.21.15.100:0/1090093263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9224160d90 0x7f92241611a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.201+0000 7f922b000640 1 --2- 172.21.15.100:0/1090093263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f922415e770 0x7f92241f4f90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.201+0000 7f922b000640 1 --2- 172.21.15.100:0/1090093263 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92240b7f40 0x7f922414bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.201+0000 7f922b000640 1 -- 172.21.15.100:0/1090093263 >> 172.21.15.100:0/1090093263 conn(0x7f92240d44f0 msgr2=0x7f92240d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:28.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.201+0000 7f922b000640 1 -- 172.21.15.100:0/1090093263 shutdown_connections 2024-09-17T04:24:28.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.202+0000 7f922b000640 1 -- 172.21.15.100:0/1090093263 wait complete. 2024-09-17T04:24:28.221 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476745 got 68719476745 for osd.4 2024-09-17T04:24:28.221 DEBUG:teuthology.parallel:result is None 2024-09-17T04:24:28.223 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542153 for osd.3 2024-09-17T04:24:28.223 DEBUG:teuthology.parallel:result is None 2024-09-17T04:24:28.224 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476744 got 68719476744 for osd.6 2024-09-17T04:24:28.224 DEBUG:teuthology.parallel:result is None 2024-09-17T04:24:28.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.230+0000 7f2332a00640 1 -- 172.21.15.100:0/1369845105 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f232c06cda0 con 0x7f232c0d2590 2024-09-17T04:24:28.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.231+0000 7f2329600640 1 -- 172.21.15.100:0/1369845105 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2320068710 con 0x7f232c0d2590 2024-09-17T04:24:28.231 INFO:teuthology.orchestra.run.smithi100.stdout:68719476745 2024-09-17T04:24:28.233 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.2 2024-09-17T04:24:28.233 DEBUG:teuthology.parallel:result is None 2024-09-17T04:24:28.234 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476745 got 68719476745 for osd.5 2024-09-17T04:24:28.235 DEBUG:teuthology.parallel:result is None 2024-09-17T04:24:28.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.234+0000 7f2332a00640 1 -- 172.21.15.100:0/1369845105 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f230c078b90 msgr2=0x7f230c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:28.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.234+0000 7f2332a00640 1 --2- 172.21.15.100:0/1369845105 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f230c078b90 0x7f230c07b0a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f232c0d0340 tx=0x7f2324009040 comp rx=0 tx=0).stop 2024-09-17T04:24:28.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.234+0000 7f2332a00640 1 -- 172.21.15.100:0/1369845105 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f232c0d2590 msgr2=0x7f232c070aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:28.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.234+0000 7f2332a00640 1 --2- 172.21.15.100:0/1369845105 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f232c0d2590 0x7f232c070aa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f232000e7c0 tx=0x7f232000ec80 comp rx=0 tx=0).stop 2024-09-17T04:24:28.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.235+0000 7f2332a00640 1 -- 172.21.15.100:0/1369845105 shutdown_connections 2024-09-17T04:24:28.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.235+0000 7f2332a00640 1 --2- 172.21.15.100:0/1369845105 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f230c078b90 0x7f230c07b0a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.235+0000 7f2332a00640 1 --2- 172.21.15.100:0/1369845105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f232c163150 0x7f232c160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.235+0000 7f2332a00640 1 --2- 172.21.15.100:0/1369845105 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f232c0d2590 0x7f232c070aa0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.235+0000 7f2332a00640 1 --2- 172.21.15.100:0/1369845105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f232c0d13a0 0x7f232c14f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:28.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.235+0000 7f2332a00640 1 -- 172.21.15.100:0/1369845105 >> 172.21.15.100:0/1369845105 conn(0x7f232c06dd60 msgr2=0x7f232c15b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:28.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.235+0000 7f2332a00640 1 -- 172.21.15.100:0/1369845105 shutdown_connections 2024-09-17T04:24:28.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:28.235+0000 7f2332a00640 1 -- 172.21.15.100:0/1369845105 wait complete. 2024-09-17T04:24:28.262 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476745 got 68719476745 for osd.7 2024-09-17T04:24:28.262 DEBUG:teuthology.parallel:result is None 2024-09-17T04:24:29.112 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T04:24:29.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.341+0000 7f06db200640 1 -- 172.21.15.100:0/1037946440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06d4070cc0 msgr2=0x7f06d4156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:29.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.341+0000 7f06db200640 1 --2- 172.21.15.100:0/1037946440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06d4070cc0 0x7f06d4156e40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f06c0005fd0 tx=0x7f06c002a5a0 comp rx=0 tx=0).stop 2024-09-17T04:24:29.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.341+0000 7f06db200640 1 -- 172.21.15.100:0/1037946440 shutdown_connections 2024-09-17T04:24:29.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.341+0000 7f06db200640 1 --2- 172.21.15.100:0/1037946440 >> v2:172.21.15.184:3300/0 conn(0x7f06d415ce30 0x7f06d415d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:29.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.341+0000 7f06db200640 1 --2- 172.21.15.100:0/1037946440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06d4070cc0 0x7f06d4156e40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:29.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.341+0000 7f06db200640 1 -- 172.21.15.100:0/1037946440 >> 172.21.15.100:0/1037946440 conn(0x7f06d406d3d0 msgr2=0x7f06d40b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:29.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.341+0000 7f06db200640 1 -- 172.21.15.100:0/1037946440 shutdown_connections 2024-09-17T04:24:29.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.341+0000 7f06db200640 1 -- 172.21.15.100:0/1037946440 wait complete. 2024-09-17T04:24:29.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.344+0000 7f06db200640 1 Processor -- start 2024-09-17T04:24:29.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.344+0000 7f06db200640 1 -- start start 2024-09-17T04:24:29.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.345+0000 7f06db200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06d40cfc80 0x7f06d40b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:29.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.345+0000 7f06db200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06d415ce30 0x7f06d4156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:29.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.345+0000 7f06db200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06d4073f70 0x7f06d4074380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:29.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.345+0000 7f06db200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f06d40cd050 con 0x7f06d4073f70 2024-09-17T04:24:29.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.345+0000 7f06db200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f06d40cced0 con 0x7f06d415ce30 2024-09-17T04:24:29.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.345+0000 7f06db200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f06d40cd1d0 con 0x7f06d40cfc80 2024-09-17T04:24:29.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.345+0000 7f06d9200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06d4073f70 0x7f06d4074380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:29.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.345+0000 7f06d9200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06d4073f70 0x7f06d4074380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48224/0 (socket says 172.21.15.100:48224) 2024-09-17T04:24:29.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.345+0000 7f06d9200640 1 -- 172.21.15.100:0/1775760501 learned_addr learned my addr 172.21.15.100:0/1775760501 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:29.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.345+0000 7f06d3e00640 1 --2- 172.21.15.100:0/1775760501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06d40cfc80 0x7f06d40b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:29.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.345+0000 7f06d3400640 1 --2- 172.21.15.100:0/1775760501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06d415ce30 0x7f06d4156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:29.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.346+0000 7f06d9200640 1 -- 172.21.15.100:0/1775760501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06d40cfc80 msgr2=0x7f06d40b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:29.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.346+0000 7f06d9200640 1 --2- 172.21.15.100:0/1775760501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06d40cfc80 0x7f06d40b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:29.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.346+0000 7f06d9200640 1 -- 172.21.15.100:0/1775760501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06d415ce30 msgr2=0x7f06d4156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:29.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.346+0000 7f06d9200640 1 --2- 172.21.15.100:0/1775760501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06d415ce30 0x7f06d4156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:29.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.346+0000 7f06d9200640 1 -- 172.21.15.100:0/1775760501 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d40748c0 con 0x7f06d4073f70 2024-09-17T04:24:29.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.346+0000 7f06d9200640 1 --2- 172.21.15.100:0/1775760501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06d4073f70 0x7f06d4074380 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f06cc00e9c0 tx=0x7f06cc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:29.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.346+0000 7f06cbe00640 1 -- 172.21.15.100:0/1775760501 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc004d20 con 0x7f06d4073f70 2024-09-17T04:24:29.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.346+0000 7f06cbe00640 1 -- 172.21.15.100:0/1775760501 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06cc00cb20 con 0x7f06d4073f70 2024-09-17T04:24:29.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.346+0000 7f06db200640 1 -- 172.21.15.100:0/1775760501 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06d40afe70 con 0x7f06d4073f70 2024-09-17T04:24:29.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.346+0000 7f06cbe00640 1 -- 172.21.15.100:0/1775760501 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc005020 con 0x7f06d4073f70 2024-09-17T04:24:29.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.346+0000 7f06db200640 1 -- 172.21.15.100:0/1775760501 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06d40b0430 con 0x7f06d4073f70 2024-09-17T04:24:29.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.348+0000 7f06db200640 1 -- 172.21.15.100:0/1775760501 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06d40c7720 con 0x7f06d4073f70 2024-09-17T04:24:29.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.349+0000 7f06cbe00640 1 -- 172.21.15.100:0/1775760501 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f06cc0051c0 con 0x7f06d4073f70 2024-09-17T04:24:29.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.350+0000 7f06cbe00640 1 --2- 172.21.15.100:0/1775760501 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f06b4078b90 0x7f06b407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:29.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.350+0000 7f06cbe00640 1 -- 172.21.15.100:0/1775760501 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f06cc09b6a0 con 0x7f06d4073f70 2024-09-17T04:24:29.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.351+0000 7f06d3e00640 1 --2- 172.21.15.100:0/1775760501 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f06b4078b90 0x7f06b407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:29.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.351+0000 7f06d3e00640 1 --2- 172.21.15.100:0/1775760501 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f06b4078b90 0x7f06b407b0a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f06c4000f60 tx=0x7f06c4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:29.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.355+0000 7f06cbe00640 1 -- 172.21.15.100:0/1775760501 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f06d40748c0 con 0x7f06d4073f70 2024-09-17T04:24:29.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.538+0000 7f06db200640 1 -- 172.21.15.100:0/1775760501 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f06d40cd710 con 0x7f06d4073f70 2024-09-17T04:24:29.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.538+0000 7f06cbe00640 1 -- 172.21.15.100:0/1775760501 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f06cc016030 con 0x7f06d4073f70 2024-09-17T04:24:29.538 INFO:teuthology.orchestra.run.smithi100.stdout:64424509447 2024-09-17T04:24:29.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.540+0000 7f06db200640 1 -- 172.21.15.100:0/1775760501 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f06b4078b90 msgr2=0x7f06b407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:29.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.540+0000 7f06db200640 1 --2- 172.21.15.100:0/1775760501 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f06b4078b90 0x7f06b407b0a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f06c4000f60 tx=0x7f06c4009040 comp rx=0 tx=0).stop 2024-09-17T04:24:29.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.540+0000 7f06db200640 1 -- 172.21.15.100:0/1775760501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06d4073f70 msgr2=0x7f06d4074380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:29.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.540+0000 7f06db200640 1 --2- 172.21.15.100:0/1775760501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06d4073f70 0x7f06d4074380 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f06cc00e9c0 tx=0x7f06cc00ee80 comp rx=0 tx=0).stop 2024-09-17T04:24:29.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.541+0000 7f06db200640 1 -- 172.21.15.100:0/1775760501 shutdown_connections 2024-09-17T04:24:29.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.541+0000 7f06db200640 1 --2- 172.21.15.100:0/1775760501 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f06b4078b90 0x7f06b407b0a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:29.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.541+0000 7f06db200640 1 --2- 172.21.15.100:0/1775760501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06d4073f70 0x7f06d4074380 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:29.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.541+0000 7f06db200640 1 --2- 172.21.15.100:0/1775760501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06d415ce30 0x7f06d4156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:29.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.541+0000 7f06db200640 1 --2- 172.21.15.100:0/1775760501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06d40cfc80 0x7f06d40b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:29.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.541+0000 7f06db200640 1 -- 172.21.15.100:0/1775760501 >> 172.21.15.100:0/1775760501 conn(0x7f06d406d3d0 msgr2=0x7f06d415f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:29.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.541+0000 7f06db200640 1 -- 172.21.15.100:0/1775760501 shutdown_connections 2024-09-17T04:24:29.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.541+0000 7f06db200640 1 -- 172.21.15.100:0/1775760501 wait complete. 2024-09-17T04:24:29.562 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.0 2024-09-17T04:24:29.562 DEBUG:teuthology.parallel:result is None 2024-09-17T04:24:29.562 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T04:24:29.562 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:24:29.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.781+0000 7f9546e00640 1 -- 172.21.15.100:0/640305504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95400707b0 msgr2=0x7f954006f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:29.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.781+0000 7f9546e00640 1 --2- 172.21.15.100:0/640305504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95400707b0 0x7f954006f930 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9530005fd0 tx=0x7f953002a540 comp rx=0 tx=0).stop 2024-09-17T04:24:29.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.781+0000 7f9546e00640 1 -- 172.21.15.100:0/640305504 shutdown_connections 2024-09-17T04:24:29.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.781+0000 7f9546e00640 1 --2- 172.21.15.100:0/640305504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95400b4110 0x7f954006fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:29.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.781+0000 7f9546e00640 1 --2- 172.21.15.100:0/640305504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95400707b0 0x7f954006f930 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:29.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.781+0000 7f9546e00640 1 -- 172.21.15.100:0/640305504 >> 172.21.15.100:0/640305504 conn(0x7f954006d730 msgr2=0x7f95400b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:29.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.781+0000 7f9546e00640 1 -- 172.21.15.100:0/640305504 shutdown_connections 2024-09-17T04:24:29.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.781+0000 7f9546e00640 1 -- 172.21.15.100:0/640305504 wait complete. 2024-09-17T04:24:29.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.783+0000 7f9546e00640 1 Processor -- start 2024-09-17T04:24:29.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.784+0000 7f9546e00640 1 -- start start 2024-09-17T04:24:29.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.785+0000 7f9546e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95400707b0 0x7f95401f6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:29.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.785+0000 7f9546e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95400730b0 0x7f954014af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:29.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.785+0000 7f9546e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95400b4110 0x7f954015d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:29.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.785+0000 7f9546e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f954006f0a0 con 0x7f95400b4110 2024-09-17T04:24:29.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.785+0000 7f9546e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f954015b2b0 con 0x7f95400707b0 2024-09-17T04:24:29.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.785+0000 7f9546e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f954006f220 con 0x7f95400730b0 2024-09-17T04:24:29.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.785+0000 7f9544e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95400b4110 0x7f954015d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:29.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.785+0000 7f9544e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95400b4110 0x7f954015d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48238/0 (socket says 172.21.15.100:48238) 2024-09-17T04:24:29.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.785+0000 7f9544e00640 1 -- 172.21.15.100:0/4026846511 learned_addr learned my addr 172.21.15.100:0/4026846511 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:29.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.785+0000 7f953fe00640 1 --2- 172.21.15.100:0/4026846511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95400707b0 0x7f95401f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:29.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.785+0000 7f953f400640 1 --2- 172.21.15.100:0/4026846511 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95400730b0 0x7f954014af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:29.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.786+0000 7f9544e00640 1 -- 172.21.15.100:0/4026846511 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95400730b0 msgr2=0x7f954014af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:29.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.786+0000 7f9544e00640 1 --2- 172.21.15.100:0/4026846511 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95400730b0 0x7f954014af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:29.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.786+0000 7f9544e00640 1 -- 172.21.15.100:0/4026846511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95400707b0 msgr2=0x7f95401f6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:29.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.786+0000 7f9544e00640 1 --2- 172.21.15.100:0/4026846511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95400707b0 0x7f95401f6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:29.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.786+0000 7f9544e00640 1 -- 172.21.15.100:0/4026846511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95401f6a10 con 0x7f95400b4110 2024-09-17T04:24:29.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.786+0000 7f9544e00640 1 --2- 172.21.15.100:0/4026846511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95400b4110 0x7f954015d970 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f953800ebe0 tx=0x7f953800c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:29.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.786+0000 7f9537e00640 1 -- 172.21.15.100:0/4026846511 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9538004d20 con 0x7f95400b4110 2024-09-17T04:24:29.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.786+0000 7f9537e00640 1 -- 172.21.15.100:0/4026846511 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f953800b570 con 0x7f95400b4110 2024-09-17T04:24:29.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.786+0000 7f9546e00640 1 -- 172.21.15.100:0/4026846511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95401f6cf0 con 0x7f95400b4110 2024-09-17T04:24:29.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.786+0000 7f9537e00640 1 -- 172.21.15.100:0/4026846511 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953800b9c0 con 0x7f95400b4110 2024-09-17T04:24:29.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.786+0000 7f9546e00640 1 -- 172.21.15.100:0/4026846511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9540151f80 con 0x7f95400b4110 2024-09-17T04:24:29.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.788+0000 7f9546e00640 1 -- 172.21.15.100:0/4026846511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f954006f6d0 con 0x7f95400b4110 2024-09-17T04:24:29.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.789+0000 7f9537e00640 1 -- 172.21.15.100:0/4026846511 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f953800bb60 con 0x7f95400b4110 2024-09-17T04:24:29.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.790+0000 7f9537e00640 1 --2- 172.21.15.100:0/4026846511 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9520078bc0 0x7f952007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:29.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.790+0000 7f9537e00640 1 -- 172.21.15.100:0/4026846511 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f953809b730 con 0x7f95400b4110 2024-09-17T04:24:29.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.791+0000 7f953fe00640 1 --2- 172.21.15.100:0/4026846511 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9520078bc0 0x7f952007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:29.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.791+0000 7f953fe00640 1 --2- 172.21.15.100:0/4026846511 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9520078bc0 0x7f952007b0d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9530033000 tx=0x7f9530004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:29.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.795+0000 7f9537e00640 1 -- 172.21.15.100:0/4026846511 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9538063640 con 0x7f95400b4110 2024-09-17T04:24:29.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.980+0000 7f9546e00640 1 -- 172.21.15.100:0/4026846511 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f95401f67a0 con 0x7f9520078bc0 2024-09-17T04:24:29.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.985+0000 7f9537e00640 1 -- 172.21.15.100:0/4026846511 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251826 (secure 0 0 0) 0x7f95401f67a0 con 0x7f9520078bc0 2024-09-17T04:24:29.988 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:29.989 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:24:29.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.988+0000 7f9546e00640 1 -- 172.21.15.100:0/4026846511 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9520078bc0 msgr2=0x7f952007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:29.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.988+0000 7f9546e00640 1 --2- 172.21.15.100:0/4026846511 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9520078bc0 0x7f952007b0d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9530033000 tx=0x7f9530004730 comp rx=0 tx=0).stop 2024-09-17T04:24:29.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.988+0000 7f9546e00640 1 -- 172.21.15.100:0/4026846511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95400b4110 msgr2=0x7f954015d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:29.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.988+0000 7f9546e00640 1 --2- 172.21.15.100:0/4026846511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95400b4110 0x7f954015d970 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f953800ebe0 tx=0x7f953800c7a0 comp rx=0 tx=0).stop 2024-09-17T04:24:29.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.989+0000 7f9546e00640 1 -- 172.21.15.100:0/4026846511 shutdown_connections 2024-09-17T04:24:29.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.989+0000 7f9546e00640 1 --2- 172.21.15.100:0/4026846511 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9520078bc0 0x7f952007b0d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:29.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.989+0000 7f9546e00640 1 --2- 172.21.15.100:0/4026846511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95400b4110 0x7f954015d970 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:29.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.989+0000 7f9546e00640 1 --2- 172.21.15.100:0/4026846511 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95400730b0 0x7f954014af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:29.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.989+0000 7f9546e00640 1 --2- 172.21.15.100:0/4026846511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95400707b0 0x7f95401f6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:29.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.989+0000 7f9546e00640 1 -- 172.21.15.100:0/4026846511 >> 172.21.15.100:0/4026846511 conn(0x7f954006d730 msgr2=0x7f95400af3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:29.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.989+0000 7f9546e00640 1 -- 172.21.15.100:0/4026846511 shutdown_connections 2024-09-17T04:24:29.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:29.989+0000 7f9546e00640 1 -- 172.21.15.100:0/4026846511 wait complete. 2024-09-17T04:24:30.010 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-09-17T04:24:29.968443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219276,"kb_used_data":3016,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755444,"statfs":{"total":773094113280,"available":772869574656,"internally_reserved":0,"allocated":3088384,"data_stored":1824236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003962"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224232+0000","last_change":"2024-09-17T04:24:15.051472+0000","last_active":"2024-09-17T04:24:16.224232+0000","last_peered":"2024-09-17T04:24:16.224232+0000","last_clean":"2024-09-17T04:24:16.224232+0000","last_became_active":"2024-09-17T04:24:15.051162+0000","last_became_peered":"2024-09-17T04:24:15.051162+0000","last_unstale":"2024-09-17T04:24:16.224232+0000","last_undegraded":"2024-09-17T04:24:16.224232+0000","last_fullsized":"2024-09-17T04:24:16.224232+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:35:10.792366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223431+0000","last_change":"2024-09-17T04:24:15.069649+0000","last_active":"2024-09-17T04:24:16.223431+0000","last_peered":"2024-09-17T04:24:16.223431+0000","last_clean":"2024-09-17T04:24:16.223431+0000","last_became_active":"2024-09-17T04:24:15.068975+0000","last_became_peered":"2024-09-17T04:24:15.068975+0000","last_unstale":"2024-09-17T04:24:16.223431+0000","last_undegraded":"2024-09-17T04:24:16.223431+0000","last_fullsized":"2024-09-17T04:24:16.223431+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:47.639252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223497+0000","last_change":"2024-09-17T04:24:15.052608+0000","last_active":"2024-09-17T04:24:16.223497+0000","last_peered":"2024-09-17T04:24:16.223497+0000","last_clean":"2024-09-17T04:24:16.223497+0000","last_became_active":"2024-09-17T04:24:15.052252+0000","last_became_peered":"2024-09-17T04:24:15.052252+0000","last_unstale":"2024-09-17T04:24:16.223497+0000","last_undegraded":"2024-09-17T04:24:16.223497+0000","last_fullsized":"2024-09-17T04:24:16.223497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:34:23.523300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055891+0000","last_change":"2024-09-17T04:24:15.059215+0000","last_active":"2024-09-17T04:24:16.055891+0000","last_peered":"2024-09-17T04:24:16.055891+0000","last_clean":"2024-09-17T04:24:16.055891+0000","last_became_active":"2024-09-17T04:24:15.058921+0000","last_became_peered":"2024-09-17T04:24:15.058921+0000","last_unstale":"2024-09-17T04:24:16.055891+0000","last_undegraded":"2024-09-17T04:24:16.055891+0000","last_fullsized":"2024-09-17T04:24:16.055891+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:34.142688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224122+0000","last_change":"2024-09-17T04:24:15.042545+0000","last_active":"2024-09-17T04:24:16.224122+0000","last_peered":"2024-09-17T04:24:16.224122+0000","last_clean":"2024-09-17T04:24:16.224122+0000","last_became_active":"2024-09-17T04:24:15.042235+0000","last_became_peered":"2024-09-17T04:24:15.042235+0000","last_unstale":"2024-09-17T04:24:16.224122+0000","last_undegraded":"2024-09-17T04:24:16.224122+0000","last_fullsized":"2024-09-17T04:24:16.224122+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:06:32.518189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056013+0000","last_change":"2024-09-17T04:24:15.057624+0000","last_active":"2024-09-17T04:24:16.056013+0000","last_peered":"2024-09-17T04:24:16.056013+0000","last_clean":"2024-09-17T04:24:16.056013+0000","last_became_active":"2024-09-17T04:24:15.057329+0000","last_became_peered":"2024-09-17T04:24:15.057329+0000","last_unstale":"2024-09-17T04:24:16.056013+0000","last_undegraded":"2024-09-17T04:24:16.056013+0000","last_fullsized":"2024-09-17T04:24:16.056013+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:44.355705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223880+0000","last_change":"2024-09-17T04:24:15.062645+0000","last_active":"2024-09-17T04:24:16.223880+0000","last_peered":"2024-09-17T04:24:16.223880+0000","last_clean":"2024-09-17T04:24:16.223880+0000","last_became_active":"2024-09-17T04:24:15.062227+0000","last_became_peered":"2024-09-17T04:24:15.062227+0000","last_unstale":"2024-09-17T04:24:16.223880+0000","last_undegraded":"2024-09-17T04:24:16.223880+0000","last_fullsized":"2024-09-17T04:24:16.223880+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:13.159492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224308+0000","last_change":"2024-09-17T04:24:15.062993+0000","last_active":"2024-09-17T04:24:16.224308+0000","last_peered":"2024-09-17T04:24:16.224308+0000","last_clean":"2024-09-17T04:24:16.224308+0000","last_became_active":"2024-09-17T04:24:15.062311+0000","last_became_peered":"2024-09-17T04:24:15.062311+0000","last_unstale":"2024-09-17T04:24:16.224308+0000","last_undegraded":"2024-09-17T04:24:16.224308+0000","last_fullsized":"2024-09-17T04:24:16.224308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:26:42.504856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224742+0000","last_change":"2024-09-17T04:24:15.070288+0000","last_active":"2024-09-17T04:24:16.224742+0000","last_peered":"2024-09-17T04:24:16.224742+0000","last_clean":"2024-09-17T04:24:16.224742+0000","last_became_active":"2024-09-17T04:24:15.068589+0000","last_became_peered":"2024-09-17T04:24:15.068589+0000","last_unstale":"2024-09-17T04:24:16.224742+0000","last_undegraded":"2024-09-17T04:24:16.224742+0000","last_fullsized":"2024-09-17T04:24:16.224742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:33.110035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225192+0000","last_change":"2024-09-17T04:24:15.062446+0000","last_active":"2024-09-17T04:24:16.225192+0000","last_peered":"2024-09-17T04:24:16.225192+0000","last_clean":"2024-09-17T04:24:16.225192+0000","last_became_active":"2024-09-17T04:24:15.060522+0000","last_became_peered":"2024-09-17T04:24:15.060522+0000","last_unstale":"2024-09-17T04:24:16.225192+0000","last_undegraded":"2024-09-17T04:24:16.225192+0000","last_fullsized":"2024-09-17T04:24:16.225192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:09.349585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224815+0000","last_change":"2024-09-17T04:24:15.062048+0000","last_active":"2024-09-17T04:24:16.224815+0000","last_peered":"2024-09-17T04:24:16.224815+0000","last_clean":"2024-09-17T04:24:16.224815+0000","last_became_active":"2024-09-17T04:24:15.060887+0000","last_became_peered":"2024-09-17T04:24:15.060887+0000","last_unstale":"2024-09-17T04:24:16.224815+0000","last_undegraded":"2024-09-17T04:24:16.224815+0000","last_fullsized":"2024-09-17T04:24:16.224815+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:22.452641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054679+0000","last_change":"2024-09-17T04:24:15.068320+0000","last_active":"2024-09-17T04:24:16.054679+0000","last_peered":"2024-09-17T04:24:16.054679+0000","last_clean":"2024-09-17T04:24:16.054679+0000","last_became_active":"2024-09-17T04:24:15.067973+0000","last_became_peered":"2024-09-17T04:24:15.067973+0000","last_unstale":"2024-09-17T04:24:16.054679+0000","last_undegraded":"2024-09-17T04:24:16.054679+0000","last_fullsized":"2024-09-17T04:24:16.054679+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:04.323459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225225+0000","last_change":"2024-09-17T04:24:15.047951+0000","last_active":"2024-09-17T04:24:16.225225+0000","last_peered":"2024-09-17T04:24:16.225225+0000","last_clean":"2024-09-17T04:24:16.225225+0000","last_became_active":"2024-09-17T04:24:15.047536+0000","last_became_peered":"2024-09-17T04:24:15.047536+0000","last_unstale":"2024-09-17T04:24:16.225225+0000","last_undegraded":"2024-09-17T04:24:16.225225+0000","last_fullsized":"2024-09-17T04:24:16.225225+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:38:48.386032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224034+0000","last_change":"2024-09-17T04:24:15.070607+0000","last_active":"2024-09-17T04:24:16.224034+0000","last_peered":"2024-09-17T04:24:16.224034+0000","last_clean":"2024-09-17T04:24:16.224034+0000","last_became_active":"2024-09-17T04:24:15.067087+0000","last_became_peered":"2024-09-17T04:24:15.067087+0000","last_unstale":"2024-09-17T04:24:16.224034+0000","last_undegraded":"2024-09-17T04:24:16.224034+0000","last_fullsized":"2024-09-17T04:24:16.224034+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:09.349585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224091+0000","last_change":"2024-09-17T04:24:15.076095+0000","last_active":"2024-09-17T04:24:16.224091+0000","last_peered":"2024-09-17T04:24:16.224091+0000","last_clean":"2024-09-17T04:24:16.224091+0000","last_became_active":"2024-09-17T04:24:15.075803+0000","last_became_peered":"2024-09-17T04:24:15.075803+0000","last_unstale":"2024-09-17T04:24:16.224091+0000","last_undegraded":"2024-09-17T04:24:16.224091+0000","last_fullsized":"2024-09-17T04:24:16.224091+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:28.104736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055352+0000","last_change":"2024-09-17T04:24:15.053797+0000","last_active":"2024-09-17T04:24:16.055352+0000","last_peered":"2024-09-17T04:24:16.055352+0000","last_clean":"2024-09-17T04:24:16.055352+0000","last_became_active":"2024-09-17T04:24:15.053449+0000","last_became_peered":"2024-09-17T04:24:15.053449+0000","last_unstale":"2024-09-17T04:24:16.055352+0000","last_undegraded":"2024-09-17T04:24:16.055352+0000","last_fullsized":"2024-09-17T04:24:16.055352+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:49:03.743620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056725+0000","last_change":"2024-09-17T04:24:15.055632+0000","last_active":"2024-09-17T04:24:16.056725+0000","last_peered":"2024-09-17T04:24:16.056725+0000","last_clean":"2024-09-17T04:24:16.056725+0000","last_became_active":"2024-09-17T04:24:15.055156+0000","last_became_peered":"2024-09-17T04:24:15.055156+0000","last_unstale":"2024-09-17T04:24:16.056725+0000","last_undegraded":"2024-09-17T04:24:16.056725+0000","last_fullsized":"2024-09-17T04:24:16.056725+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:16.120591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.052124+0000","last_change":"2024-09-17T04:24:15.057788+0000","last_active":"2024-09-17T04:24:16.052124+0000","last_peered":"2024-09-17T04:24:16.052124+0000","last_clean":"2024-09-17T04:24:16.052124+0000","last_became_active":"2024-09-17T04:24:15.057141+0000","last_became_peered":"2024-09-17T04:24:15.057141+0000","last_unstale":"2024-09-17T04:24:16.052124+0000","last_undegraded":"2024-09-17T04:24:16.052124+0000","last_fullsized":"2024-09-17T04:24:16.052124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:10.704647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224426+0000","last_change":"2024-09-17T04:24:15.067312+0000","last_active":"2024-09-17T04:24:16.224426+0000","last_peered":"2024-09-17T04:24:16.224426+0000","last_clean":"2024-09-17T04:24:16.224426+0000","last_became_active":"2024-09-17T04:24:15.059516+0000","last_became_peered":"2024-09-17T04:24:15.059516+0000","last_unstale":"2024-09-17T04:24:16.224426+0000","last_undegraded":"2024-09-17T04:24:16.224426+0000","last_fullsized":"2024-09-17T04:24:16.224426+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:16.120591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224948+0000","last_change":"2024-09-17T04:24:15.076912+0000","last_active":"2024-09-17T04:24:16.224948+0000","last_peered":"2024-09-17T04:24:16.224948+0000","last_clean":"2024-09-17T04:24:16.224948+0000","last_became_active":"2024-09-17T04:24:15.076617+0000","last_became_peered":"2024-09-17T04:24:15.076617+0000","last_unstale":"2024-09-17T04:24:16.224948+0000","last_undegraded":"2024-09-17T04:24:16.224948+0000","last_fullsized":"2024-09-17T04:24:16.224948+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:47.639252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223809+0000","last_change":"2024-09-17T04:24:15.070958+0000","last_active":"2024-09-17T04:24:16.223809+0000","last_peered":"2024-09-17T04:24:16.223809+0000","last_clean":"2024-09-17T04:24:16.223809+0000","last_became_active":"2024-09-17T04:24:15.070624+0000","last_became_peered":"2024-09-17T04:24:15.070624+0000","last_unstale":"2024-09-17T04:24:16.223809+0000","last_undegraded":"2024-09-17T04:24:16.223809+0000","last_fullsized":"2024-09-17T04:24:16.223809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:13.752351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055648+0000","last_change":"2024-09-17T04:24:15.058815+0000","last_active":"2024-09-17T04:24:16.055648+0000","last_peered":"2024-09-17T04:24:16.055648+0000","last_clean":"2024-09-17T04:24:16.055648+0000","last_became_active":"2024-09-17T04:24:15.058482+0000","last_became_peered":"2024-09-17T04:24:15.058482+0000","last_unstale":"2024-09-17T04:24:16.055648+0000","last_undegraded":"2024-09-17T04:24:16.055648+0000","last_fullsized":"2024-09-17T04:24:16.055648+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:18.409788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224382+0000","last_change":"2024-09-17T04:24:15.070200+0000","last_active":"2024-09-17T04:24:16.224382+0000","last_peered":"2024-09-17T04:24:16.224382+0000","last_clean":"2024-09-17T04:24:16.224382+0000","last_became_active":"2024-09-17T04:24:15.069872+0000","last_became_peered":"2024-09-17T04:24:15.069872+0000","last_unstale":"2024-09-17T04:24:16.224382+0000","last_undegraded":"2024-09-17T04:24:16.224382+0000","last_fullsized":"2024-09-17T04:24:16.224382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:22.880022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055404+0000","last_change":"2024-09-17T04:24:15.050935+0000","last_active":"2024-09-17T04:24:16.055404+0000","last_peered":"2024-09-17T04:24:16.055404+0000","last_clean":"2024-09-17T04:24:16.055404+0000","last_became_active":"2024-09-17T04:24:15.048946+0000","last_became_peered":"2024-09-17T04:24:15.048946+0000","last_unstale":"2024-09-17T04:24:16.055404+0000","last_undegraded":"2024-09-17T04:24:16.055404+0000","last_fullsized":"2024-09-17T04:24:16.055404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:11:09.116226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225400+0000","last_change":"2024-09-17T04:24:15.074863+0000","last_active":"2024-09-17T04:24:16.225400+0000","last_peered":"2024-09-17T04:24:16.225400+0000","last_clean":"2024-09-17T04:24:16.225400+0000","last_became_active":"2024-09-17T04:24:15.074467+0000","last_became_peered":"2024-09-17T04:24:15.074467+0000","last_unstale":"2024-09-17T04:24:16.225400+0000","last_undegraded":"2024-09-17T04:24:16.225400+0000","last_fullsized":"2024-09-17T04:24:16.225400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:36:59.215343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225206+0000","last_change":"2024-09-17T04:24:15.069430+0000","last_active":"2024-09-17T04:24:16.225206+0000","last_peered":"2024-09-17T04:24:16.225206+0000","last_clean":"2024-09-17T04:24:16.225206+0000","last_became_active":"2024-09-17T04:24:15.069117+0000","last_became_peered":"2024-09-17T04:24:15.069117+0000","last_unstale":"2024-09-17T04:24:16.225206+0000","last_undegraded":"2024-09-17T04:24:16.225206+0000","last_fullsized":"2024-09-17T04:24:16.225206+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:49:03.743620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224018+0000","last_change":"2024-09-17T04:24:15.057539+0000","last_active":"2024-09-17T04:24:16.224018+0000","last_peered":"2024-09-17T04:24:16.224018+0000","last_clean":"2024-09-17T04:24:16.224018+0000","last_became_active":"2024-09-17T04:24:15.057194+0000","last_became_peered":"2024-09-17T04:24:15.057194+0000","last_unstale":"2024-09-17T04:24:16.224018+0000","last_undegraded":"2024-09-17T04:24:16.224018+0000","last_fullsized":"2024-09-17T04:24:16.224018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:09.349585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225430+0000","last_change":"2024-09-17T04:24:15.075902+0000","last_active":"2024-09-17T04:24:16.225430+0000","last_peered":"2024-09-17T04:24:16.225430+0000","last_clean":"2024-09-17T04:24:16.225430+0000","last_became_active":"2024-09-17T04:24:15.075625+0000","last_became_peered":"2024-09-17T04:24:15.075625+0000","last_unstale":"2024-09-17T04:24:16.225430+0000","last_undegraded":"2024-09-17T04:24:16.225430+0000","last_fullsized":"2024-09-17T04:24:16.225430+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:34.202614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223788+0000","last_change":"2024-09-17T04:24:15.052108+0000","last_active":"2024-09-17T04:24:16.223788+0000","last_peered":"2024-09-17T04:24:16.223788+0000","last_clean":"2024-09-17T04:24:16.223788+0000","last_became_active":"2024-09-17T04:24:15.051794+0000","last_became_peered":"2024-09-17T04:24:15.051794+0000","last_unstale":"2024-09-17T04:24:16.223788+0000","last_undegraded":"2024-09-17T04:24:16.223788+0000","last_fullsized":"2024-09-17T04:24:16.223788+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:13:56.486773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055909+0000","last_change":"2024-09-17T04:24:15.057335+0000","last_active":"2024-09-17T04:24:16.055909+0000","last_peered":"2024-09-17T04:24:16.055909+0000","last_clean":"2024-09-17T04:24:16.055909+0000","last_became_active":"2024-09-17T04:24:15.056924+0000","last_became_peered":"2024-09-17T04:24:15.056924+0000","last_unstale":"2024-09-17T04:24:16.055909+0000","last_undegraded":"2024-09-17T04:24:16.055909+0000","last_fullsized":"2024-09-17T04:24:16.055909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:29:56.515791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.052052+0000","last_change":"2024-09-17T04:24:15.054070+0000","last_active":"2024-09-17T04:24:16.052052+0000","last_peered":"2024-09-17T04:24:16.052052+0000","last_clean":"2024-09-17T04:24:16.052052+0000","last_became_active":"2024-09-17T04:24:15.052619+0000","last_became_peered":"2024-09-17T04:24:15.052619+0000","last_unstale":"2024-09-17T04:24:16.052052+0000","last_undegraded":"2024-09-17T04:24:16.052052+0000","last_fullsized":"2024-09-17T04:24:16.052052+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:49.409374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223562+0000","last_change":"2024-09-17T04:24:15.061084+0000","last_active":"2024-09-17T04:24:16.223562+0000","last_peered":"2024-09-17T04:24:16.223562+0000","last_clean":"2024-09-17T04:24:16.223562+0000","last_became_active":"2024-09-17T04:24:15.060769+0000","last_became_peered":"2024-09-17T04:24:15.060769+0000","last_unstale":"2024-09-17T04:24:16.223562+0000","last_undegraded":"2024-09-17T04:24:16.223562+0000","last_fullsized":"2024-09-17T04:24:16.223562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:13:56.486773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054106+0000","last_change":"2024-09-17T04:24:15.068899+0000","last_active":"2024-09-17T04:24:16.054106+0000","last_peered":"2024-09-17T04:24:16.054106+0000","last_clean":"2024-09-17T04:24:16.054106+0000","last_became_active":"2024-09-17T04:24:15.068595+0000","last_became_peered":"2024-09-17T04:24:15.068595+0000","last_unstale":"2024-09-17T04:24:16.054106+0000","last_undegraded":"2024-09-17T04:24:16.054106+0000","last_fullsized":"2024-09-17T04:24:16.054106+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:13.785027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225374+0000","last_change":"2024-09-17T04:24:14.062806+0000","last_active":"2024-09-17T04:24:16.225374+0000","last_peered":"2024-09-17T04:24:16.225374+0000","last_clean":"2024-09-17T04:24:16.225374+0000","last_became_active":"2024-09-17T04:24:14.062282+0000","last_became_peered":"2024-09-17T04:24:14.062282+0000","last_unstale":"2024-09-17T04:24:16.225374+0000","last_undegraded":"2024-09-17T04:24:16.225374+0000","last_fullsized":"2024-09-17T04:24:16.225374+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:23.698652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224005+0000","last_change":"2024-09-17T04:24:15.075318+0000","last_active":"2024-09-17T04:24:16.224005+0000","last_peered":"2024-09-17T04:24:16.224005+0000","last_clean":"2024-09-17T04:24:16.224005+0000","last_became_active":"2024-09-17T04:24:15.074876+0000","last_became_peered":"2024-09-17T04:24:15.074876+0000","last_unstale":"2024-09-17T04:24:16.224005+0000","last_undegraded":"2024-09-17T04:24:16.224005+0000","last_fullsized":"2024-09-17T04:24:16.224005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:47.639252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224710+0000","last_change":"2024-09-17T04:24:14.056222+0000","last_active":"2024-09-17T04:24:16.224710+0000","last_peered":"2024-09-17T04:24:16.224710+0000","last_clean":"2024-09-17T04:24:16.224710+0000","last_became_active":"2024-09-17T04:24:14.055874+0000","last_became_peered":"2024-09-17T04:24:14.055874+0000","last_unstale":"2024-09-17T04:24:16.224710+0000","last_undegraded":"2024-09-17T04:24:16.224710+0000","last_fullsized":"2024-09-17T04:24:16.224710+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:44.734015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224568+0000","last_change":"2024-09-17T04:24:14.054189+0000","last_active":"2024-09-17T04:24:16.224568+0000","last_peered":"2024-09-17T04:24:16.224568+0000","last_clean":"2024-09-17T04:24:16.224568+0000","last_became_active":"2024-09-17T04:24:14.053870+0000","last_became_peered":"2024-09-17T04:24:14.053870+0000","last_unstale":"2024-09-17T04:24:16.224568+0000","last_undegraded":"2024-09-17T04:24:16.224568+0000","last_fullsized":"2024-09-17T04:24:16.224568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:12.776180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224542+0000","last_change":"2024-09-17T04:24:15.059936+0000","last_active":"2024-09-17T04:24:16.224542+0000","last_peered":"2024-09-17T04:24:16.224542+0000","last_clean":"2024-09-17T04:24:16.224542+0000","last_became_active":"2024-09-17T04:24:15.059645+0000","last_became_peered":"2024-09-17T04:24:15.059645+0000","last_unstale":"2024-09-17T04:24:16.224542+0000","last_undegraded":"2024-09-17T04:24:16.224542+0000","last_fullsized":"2024-09-17T04:24:16.224542+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:08:11.703658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055748+0000","last_change":"2024-09-17T04:24:14.060082+0000","last_active":"2024-09-17T04:24:16.055748+0000","last_peered":"2024-09-17T04:24:16.055748+0000","last_clean":"2024-09-17T04:24:16.055748+0000","last_became_active":"2024-09-17T04:24:14.058795+0000","last_became_peered":"2024-09-17T04:24:14.058795+0000","last_unstale":"2024-09-17T04:24:16.055748+0000","last_undegraded":"2024-09-17T04:24:16.055748+0000","last_fullsized":"2024-09-17T04:24:16.055748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:21.535057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.222826+0000","last_change":"2024-09-17T04:24:15.061788+0000","last_active":"2024-09-17T04:24:16.222826+0000","last_peered":"2024-09-17T04:24:16.222826+0000","last_clean":"2024-09-17T04:24:16.222826+0000","last_became_active":"2024-09-17T04:24:15.055072+0000","last_became_peered":"2024-09-17T04:24:15.055072+0000","last_unstale":"2024-09-17T04:24:16.222826+0000","last_undegraded":"2024-09-17T04:24:16.222826+0000","last_fullsized":"2024-09-17T04:24:16.222826+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:13.752351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225245+0000","last_change":"2024-09-17T04:24:14.053595+0000","last_active":"2024-09-17T04:24:16.225245+0000","last_peered":"2024-09-17T04:24:16.225245+0000","last_clean":"2024-09-17T04:24:16.225245+0000","last_became_active":"2024-09-17T04:24:14.053260+0000","last_became_peered":"2024-09-17T04:24:14.053260+0000","last_unstale":"2024-09-17T04:24:16.225245+0000","last_undegraded":"2024-09-17T04:24:16.225245+0000","last_fullsized":"2024-09-17T04:24:16.225245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:51.429181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056660+0000","last_change":"2024-09-17T04:24:15.053812+0000","last_active":"2024-09-17T04:24:16.056660+0000","last_peered":"2024-09-17T04:24:16.056660+0000","last_clean":"2024-09-17T04:24:16.056660+0000","last_became_active":"2024-09-17T04:24:15.053501+0000","last_became_peered":"2024-09-17T04:24:15.053501+0000","last_unstale":"2024-09-17T04:24:16.056660+0000","last_undegraded":"2024-09-17T04:24:16.056660+0000","last_fullsized":"2024-09-17T04:24:16.056660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:47.959956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056115+0000","last_change":"2024-09-17T04:24:14.059203+0000","last_active":"2024-09-17T04:24:16.056115+0000","last_peered":"2024-09-17T04:24:16.056115+0000","last_clean":"2024-09-17T04:24:16.056115+0000","last_became_active":"2024-09-17T04:24:14.058623+0000","last_became_peered":"2024-09-17T04:24:14.058623+0000","last_unstale":"2024-09-17T04:24:16.056115+0000","last_undegraded":"2024-09-17T04:24:16.056115+0000","last_fullsized":"2024-09-17T04:24:16.056115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:20.372651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223819+0000","last_change":"2024-09-17T04:24:15.068669+0000","last_active":"2024-09-17T04:24:16.223819+0000","last_peered":"2024-09-17T04:24:16.223819+0000","last_clean":"2024-09-17T04:24:16.223819+0000","last_became_active":"2024-09-17T04:24:15.066578+0000","last_became_peered":"2024-09-17T04:24:15.066578+0000","last_unstale":"2024-09-17T04:24:16.223819+0000","last_undegraded":"2024-09-17T04:24:16.223819+0000","last_fullsized":"2024-09-17T04:24:16.223819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:01.706191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223044+0000","last_change":"2024-09-17T04:24:14.061969+0000","last_active":"2024-09-17T04:24:16.223044+0000","last_peered":"2024-09-17T04:24:16.223044+0000","last_clean":"2024-09-17T04:24:16.223044+0000","last_became_active":"2024-09-17T04:24:14.061593+0000","last_became_peered":"2024-09-17T04:24:14.061593+0000","last_unstale":"2024-09-17T04:24:16.223044+0000","last_undegraded":"2024-09-17T04:24:16.223044+0000","last_fullsized":"2024-09-17T04:24:16.223044+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:48.406614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054926+0000","last_change":"2024-09-17T04:24:15.071779+0000","last_active":"2024-09-17T04:24:16.054926+0000","last_peered":"2024-09-17T04:24:16.054926+0000","last_clean":"2024-09-17T04:24:16.054926+0000","last_became_active":"2024-09-17T04:24:15.071431+0000","last_became_peered":"2024-09-17T04:24:15.071431+0000","last_unstale":"2024-09-17T04:24:16.054926+0000","last_undegraded":"2024-09-17T04:24:16.054926+0000","last_fullsized":"2024-09-17T04:24:16.054926+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:10.704647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224939+0000","last_change":"2024-09-17T04:24:14.058729+0000","last_active":"2024-09-17T04:24:16.224939+0000","last_peered":"2024-09-17T04:24:16.224939+0000","last_clean":"2024-09-17T04:24:16.224939+0000","last_became_active":"2024-09-17T04:24:14.057616+0000","last_became_peered":"2024-09-17T04:24:14.057616+0000","last_unstale":"2024-09-17T04:24:16.224939+0000","last_undegraded":"2024-09-17T04:24:16.224939+0000","last_fullsized":"2024-09-17T04:24:16.224939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:42.267414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223701+0000","last_change":"2024-09-17T04:24:15.056613+0000","last_active":"2024-09-17T04:24:16.223701+0000","last_peered":"2024-09-17T04:24:16.223701+0000","last_clean":"2024-09-17T04:24:16.223701+0000","last_became_active":"2024-09-17T04:24:15.056300+0000","last_became_peered":"2024-09-17T04:24:15.056300+0000","last_unstale":"2024-09-17T04:24:16.223701+0000","last_undegraded":"2024-09-17T04:24:16.223701+0000","last_fullsized":"2024-09-17T04:24:16.223701+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:34.142688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223885+0000","last_change":"2024-09-17T04:24:14.054359+0000","last_active":"2024-09-17T04:24:16.223885+0000","last_peered":"2024-09-17T04:24:16.223885+0000","last_clean":"2024-09-17T04:24:16.223885+0000","last_became_active":"2024-09-17T04:24:14.053935+0000","last_became_peered":"2024-09-17T04:24:14.053935+0000","last_unstale":"2024-09-17T04:24:16.223885+0000","last_undegraded":"2024-09-17T04:24:16.223885+0000","last_fullsized":"2024-09-17T04:24:16.223885+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:01.211818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223752+0000","last_change":"2024-09-17T04:24:15.068236+0000","last_active":"2024-09-17T04:24:16.223752+0000","last_peered":"2024-09-17T04:24:16.223752+0000","last_clean":"2024-09-17T04:24:16.223752+0000","last_became_active":"2024-09-17T04:24:15.067460+0000","last_became_peered":"2024-09-17T04:24:15.067460+0000","last_unstale":"2024-09-17T04:24:16.223752+0000","last_undegraded":"2024-09-17T04:24:16.223752+0000","last_fullsized":"2024-09-17T04:24:16.223752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:44:58.329654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056099+0000","last_change":"2024-09-17T04:24:14.058127+0000","last_active":"2024-09-17T04:24:16.056099+0000","last_peered":"2024-09-17T04:24:16.056099+0000","last_clean":"2024-09-17T04:24:16.056099+0000","last_became_active":"2024-09-17T04:24:14.055990+0000","last_became_peered":"2024-09-17T04:24:14.055990+0000","last_unstale":"2024-09-17T04:24:16.056099+0000","last_undegraded":"2024-09-17T04:24:16.056099+0000","last_fullsized":"2024-09-17T04:24:16.056099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:51.429181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223887+0000","last_change":"2024-09-17T04:24:15.076359+0000","last_active":"2024-09-17T04:24:16.223887+0000","last_peered":"2024-09-17T04:24:16.223887+0000","last_clean":"2024-09-17T04:24:16.223887+0000","last_became_active":"2024-09-17T04:24:15.075235+0000","last_became_peered":"2024-09-17T04:24:15.075235+0000","last_unstale":"2024-09-17T04:24:16.223887+0000","last_undegraded":"2024-09-17T04:24:16.223887+0000","last_fullsized":"2024-09-17T04:24:16.223887+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:44.355705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224496+0000","last_change":"2024-09-17T04:24:14.051949+0000","last_active":"2024-09-17T04:24:16.224496+0000","last_peered":"2024-09-17T04:24:16.224496+0000","last_clean":"2024-09-17T04:24:16.224496+0000","last_became_active":"2024-09-17T04:24:14.051548+0000","last_became_peered":"2024-09-17T04:24:14.051548+0000","last_unstale":"2024-09-17T04:24:16.224496+0000","last_undegraded":"2024-09-17T04:24:16.224496+0000","last_fullsized":"2024-09-17T04:24:16.224496+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:21.535057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223899+0000","last_change":"2024-09-17T04:24:15.060676+0000","last_active":"2024-09-17T04:24:16.223899+0000","last_peered":"2024-09-17T04:24:16.223899+0000","last_clean":"2024-09-17T04:24:16.223899+0000","last_became_active":"2024-09-17T04:24:15.060268+0000","last_became_peered":"2024-09-17T04:24:15.060268+0000","last_unstale":"2024-09-17T04:24:16.223899+0000","last_undegraded":"2024-09-17T04:24:16.223899+0000","last_fullsized":"2024-09-17T04:24:16.223899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:13.752351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.051789+0000","last_change":"2024-09-17T04:24:14.043539+0000","last_active":"2024-09-17T04:24:16.051789+0000","last_peered":"2024-09-17T04:24:16.051789+0000","last_clean":"2024-09-17T04:24:16.051789+0000","last_became_active":"2024-09-17T04:24:14.043214+0000","last_became_peered":"2024-09-17T04:24:14.043214+0000","last_unstale":"2024-09-17T04:24:16.051789+0000","last_undegraded":"2024-09-17T04:24:16.051789+0000","last_fullsized":"2024-09-17T04:24:16.051789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:42.267414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223990+0000","last_change":"2024-09-17T04:24:15.068886+0000","last_active":"2024-09-17T04:24:16.223990+0000","last_peered":"2024-09-17T04:24:16.223990+0000","last_clean":"2024-09-17T04:24:16.223990+0000","last_became_active":"2024-09-17T04:24:15.068532+0000","last_became_peered":"2024-09-17T04:24:15.068532+0000","last_unstale":"2024-09-17T04:24:16.223990+0000","last_undegraded":"2024-09-17T04:24:16.223990+0000","last_fullsized":"2024-09-17T04:24:16.223990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:58.719218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054761+0000","last_change":"2024-09-17T04:24:14.059673+0000","last_active":"2024-09-17T04:24:16.054761+0000","last_peered":"2024-09-17T04:24:16.054761+0000","last_clean":"2024-09-17T04:24:16.054761+0000","last_became_active":"2024-09-17T04:24:14.057668+0000","last_became_peered":"2024-09-17T04:24:14.057668+0000","last_unstale":"2024-09-17T04:24:16.054761+0000","last_undegraded":"2024-09-17T04:24:16.054761+0000","last_fullsized":"2024-09-17T04:24:16.054761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:01.211818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224143+0000","last_change":"2024-09-17T04:24:15.069931+0000","last_active":"2024-09-17T04:24:16.224143+0000","last_peered":"2024-09-17T04:24:16.224143+0000","last_clean":"2024-09-17T04:24:16.224143+0000","last_became_active":"2024-09-17T04:24:15.069568+0000","last_became_peered":"2024-09-17T04:24:15.069568+0000","last_unstale":"2024-09-17T04:24:16.224143+0000","last_undegraded":"2024-09-17T04:24:16.224143+0000","last_fullsized":"2024-09-17T04:24:16.224143+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:35.584338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.052093+0000","last_change":"2024-09-17T04:24:14.054912+0000","last_active":"2024-09-17T04:24:16.052093+0000","last_peered":"2024-09-17T04:24:16.052093+0000","last_clean":"2024-09-17T04:24:16.052093+0000","last_became_active":"2024-09-17T04:24:14.054105+0000","last_became_peered":"2024-09-17T04:24:14.054105+0000","last_unstale":"2024-09-17T04:24:16.052093+0000","last_undegraded":"2024-09-17T04:24:16.052093+0000","last_fullsized":"2024-09-17T04:24:16.052093+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:44.734015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224717+0000","last_change":"2024-09-17T04:24:15.075676+0000","last_active":"2024-09-17T04:24:16.224717+0000","last_peered":"2024-09-17T04:24:16.224717+0000","last_clean":"2024-09-17T04:24:16.224717+0000","last_became_active":"2024-09-17T04:24:15.075114+0000","last_became_peered":"2024-09-17T04:24:15.075114+0000","last_unstale":"2024-09-17T04:24:16.224717+0000","last_undegraded":"2024-09-17T04:24:16.224717+0000","last_fullsized":"2024-09-17T04:24:16.224717+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:51:30.987569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223146+0000","last_change":"2024-09-17T04:24:14.039417+0000","last_active":"2024-09-17T04:24:16.223146+0000","last_peered":"2024-09-17T04:24:16.223146+0000","last_clean":"2024-09-17T04:24:16.223146+0000","last_became_active":"2024-09-17T04:24:14.039025+0000","last_became_peered":"2024-09-17T04:24:14.039025+0000","last_unstale":"2024-09-17T04:24:16.223146+0000","last_undegraded":"2024-09-17T04:24:16.223146+0000","last_fullsized":"2024-09-17T04:24:16.223146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:03.320698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056433+0000","last_change":"2024-09-17T04:24:15.061023+0000","last_active":"2024-09-17T04:24:16.056433+0000","last_peered":"2024-09-17T04:24:16.056433+0000","last_clean":"2024-09-17T04:24:16.056433+0000","last_became_active":"2024-09-17T04:24:15.060544+0000","last_became_peered":"2024-09-17T04:24:15.060544+0000","last_unstale":"2024-09-17T04:24:16.056433+0000","last_undegraded":"2024-09-17T04:24:16.056433+0000","last_fullsized":"2024-09-17T04:24:16.056433+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:35:10.792366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.051950+0000","last_change":"2024-09-17T04:24:14.052529+0000","last_active":"2024-09-17T04:24:16.051950+0000","last_peered":"2024-09-17T04:24:16.051950+0000","last_clean":"2024-09-17T04:24:16.051950+0000","last_became_active":"2024-09-17T04:24:14.052135+0000","last_became_peered":"2024-09-17T04:24:14.052135+0000","last_unstale":"2024-09-17T04:24:16.051950+0000","last_undegraded":"2024-09-17T04:24:16.051950+0000","last_fullsized":"2024-09-17T04:24:16.051950+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:28.196011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224039+0000","last_change":"2024-09-17T04:24:15.064435+0000","last_active":"2024-09-17T04:24:16.224039+0000","last_peered":"2024-09-17T04:24:16.224039+0000","last_clean":"2024-09-17T04:24:16.224039+0000","last_became_active":"2024-09-17T04:24:15.064195+0000","last_became_peered":"2024-09-17T04:24:15.064195+0000","last_unstale":"2024-09-17T04:24:16.224039+0000","last_undegraded":"2024-09-17T04:24:16.224039+0000","last_fullsized":"2024-09-17T04:24:16.224039+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:47.959956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223547+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:24:16.223547+0000","last_peered":"2024-09-17T04:24:16.223547+0000","last_clean":"2024-09-17T04:24:16.223547+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:24:16.223547+0000","last_undegraded":"2024-09-17T04:24:16.223547+0000","last_fullsized":"2024-09-17T04:24:16.223547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055535+0000","last_change":"2024-09-17T04:24:14.049088+0000","last_active":"2024-09-17T04:24:16.055535+0000","last_peered":"2024-09-17T04:24:16.055535+0000","last_clean":"2024-09-17T04:24:16.055535+0000","last_became_active":"2024-09-17T04:24:14.048529+0000","last_became_peered":"2024-09-17T04:24:14.048529+0000","last_unstale":"2024-09-17T04:24:16.055535+0000","last_undegraded":"2024-09-17T04:24:16.055535+0000","last_fullsized":"2024-09-17T04:24:16.055535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:42.267414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225083+0000","last_change":"2024-09-17T04:24:15.065699+0000","last_active":"2024-09-17T04:24:16.225083+0000","last_peered":"2024-09-17T04:24:16.225083+0000","last_clean":"2024-09-17T04:24:16.225083+0000","last_became_active":"2024-09-17T04:24:15.061569+0000","last_became_peered":"2024-09-17T04:24:15.061569+0000","last_unstale":"2024-09-17T04:24:16.225083+0000","last_undegraded":"2024-09-17T04:24:16.225083+0000","last_fullsized":"2024-09-17T04:24:16.225083+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:01:19.496265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223384+0000","last_change":"2024-09-17T04:24:14.061081+0000","last_active":"2024-09-17T04:24:16.223384+0000","last_peered":"2024-09-17T04:24:16.223384+0000","last_clean":"2024-09-17T04:24:16.223384+0000","last_became_active":"2024-09-17T04:24:14.060394+0000","last_became_peered":"2024-09-17T04:24:14.060394+0000","last_unstale":"2024-09-17T04:24:16.223384+0000","last_undegraded":"2024-09-17T04:24:16.223384+0000","last_fullsized":"2024-09-17T04:24:16.223384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:08.023686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223901+0000","last_change":"2024-09-17T04:24:15.070960+0000","last_active":"2024-09-17T04:24:16.223901+0000","last_peered":"2024-09-17T04:24:16.223901+0000","last_clean":"2024-09-17T04:24:16.223901+0000","last_became_active":"2024-09-17T04:24:15.070528+0000","last_became_peered":"2024-09-17T04:24:15.070528+0000","last_unstale":"2024-09-17T04:24:16.223901+0000","last_undegraded":"2024-09-17T04:24:16.223901+0000","last_fullsized":"2024-09-17T04:24:16.223901+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:22.452641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.052102+0000","last_change":"2024-09-17T04:24:14.051042+0000","last_active":"2024-09-17T04:24:16.052102+0000","last_peered":"2024-09-17T04:24:16.052102+0000","last_clean":"2024-09-17T04:24:16.052102+0000","last_became_active":"2024-09-17T04:24:14.050513+0000","last_became_peered":"2024-09-17T04:24:14.050513+0000","last_unstale":"2024-09-17T04:24:16.052102+0000","last_undegraded":"2024-09-17T04:24:16.052102+0000","last_fullsized":"2024-09-17T04:24:16.052102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:21.535057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.222913+0000","last_change":"2024-09-17T04:24:15.061656+0000","last_active":"2024-09-17T04:24:16.222913+0000","last_peered":"2024-09-17T04:24:16.222913+0000","last_clean":"2024-09-17T04:24:16.222913+0000","last_became_active":"2024-09-17T04:24:15.061365+0000","last_became_peered":"2024-09-17T04:24:15.061365+0000","last_unstale":"2024-09-17T04:24:16.222913+0000","last_undegraded":"2024-09-17T04:24:16.222913+0000","last_fullsized":"2024-09-17T04:24:16.222913+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:22.452641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056245+0000","last_change":"2024-09-17T04:24:14.053751+0000","last_active":"2024-09-17T04:24:16.056245+0000","last_peered":"2024-09-17T04:24:16.056245+0000","last_clean":"2024-09-17T04:24:16.056245+0000","last_became_active":"2024-09-17T04:24:14.051943+0000","last_became_peered":"2024-09-17T04:24:14.051943+0000","last_unstale":"2024-09-17T04:24:16.056245+0000","last_undegraded":"2024-09-17T04:24:16.056245+0000","last_fullsized":"2024-09-17T04:24:16.056245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:08.023686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054263+0000","last_change":"2024-09-17T04:24:15.074736+0000","last_active":"2024-09-17T04:24:16.054263+0000","last_peered":"2024-09-17T04:24:16.054263+0000","last_clean":"2024-09-17T04:24:16.054263+0000","last_became_active":"2024-09-17T04:24:15.074374+0000","last_became_peered":"2024-09-17T04:24:15.074374+0000","last_unstale":"2024-09-17T04:24:16.054263+0000","last_undegraded":"2024-09-17T04:24:16.054263+0000","last_fullsized":"2024-09-17T04:24:16.054263+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:49.409374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.222921+0000","last_change":"2024-09-17T04:24:14.057105+0000","last_active":"2024-09-17T04:24:16.222921+0000","last_peered":"2024-09-17T04:24:16.222921+0000","last_clean":"2024-09-17T04:24:16.222921+0000","last_became_active":"2024-09-17T04:24:14.056908+0000","last_became_peered":"2024-09-17T04:24:14.056908+0000","last_unstale":"2024-09-17T04:24:16.222921+0000","last_undegraded":"2024-09-17T04:24:16.222921+0000","last_fullsized":"2024-09-17T04:24:16.222921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:12.782267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056301+0000","last_change":"2024-09-17T04:24:15.058663+0000","last_active":"2024-09-17T04:24:16.056301+0000","last_peered":"2024-09-17T04:24:16.056301+0000","last_clean":"2024-09-17T04:24:16.056301+0000","last_became_active":"2024-09-17T04:24:15.057807+0000","last_became_peered":"2024-09-17T04:24:15.057807+0000","last_unstale":"2024-09-17T04:24:16.056301+0000","last_undegraded":"2024-09-17T04:24:16.056301+0000","last_fullsized":"2024-09-17T04:24:16.056301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:35.584338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224488+0000","last_change":"2024-09-17T04:24:14.063354+0000","last_active":"2024-09-17T04:24:16.224488+0000","last_peered":"2024-09-17T04:24:16.224488+0000","last_clean":"2024-09-17T04:24:16.224488+0000","last_became_active":"2024-09-17T04:24:14.062231+0000","last_became_peered":"2024-09-17T04:24:14.062231+0000","last_unstale":"2024-09-17T04:24:16.224488+0000","last_undegraded":"2024-09-17T04:24:16.224488+0000","last_fullsized":"2024-09-17T04:24:16.224488+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:40.777526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.222850+0000","last_change":"2024-09-17T04:24:15.058346+0000","last_active":"2024-09-17T04:24:16.222850+0000","last_peered":"2024-09-17T04:24:16.222850+0000","last_clean":"2024-09-17T04:24:16.222850+0000","last_became_active":"2024-09-17T04:24:15.057977+0000","last_became_peered":"2024-09-17T04:24:15.057977+0000","last_unstale":"2024-09-17T04:24:16.222850+0000","last_undegraded":"2024-09-17T04:24:16.222850+0000","last_fullsized":"2024-09-17T04:24:16.222850+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:01.706191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223545+0000","last_change":"2024-09-17T04:24:14.049298+0000","last_active":"2024-09-17T04:24:16.223545+0000","last_peered":"2024-09-17T04:24:16.223545+0000","last_clean":"2024-09-17T04:24:16.223545+0000","last_became_active":"2024-09-17T04:24:14.049070+0000","last_became_peered":"2024-09-17T04:24:14.049070+0000","last_unstale":"2024-09-17T04:24:16.223545+0000","last_undegraded":"2024-09-17T04:24:16.223545+0000","last_fullsized":"2024-09-17T04:24:16.223545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:52:59.361386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225582+0000","last_change":"2024-09-17T04:24:15.050480+0000","last_active":"2024-09-17T04:24:16.225582+0000","last_peered":"2024-09-17T04:24:16.225582+0000","last_clean":"2024-09-17T04:24:16.225582+0000","last_became_active":"2024-09-17T04:24:15.050166+0000","last_became_peered":"2024-09-17T04:24:15.050166+0000","last_unstale":"2024-09-17T04:24:16.225582+0000","last_undegraded":"2024-09-17T04:24:16.225582+0000","last_fullsized":"2024-09-17T04:24:16.225582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:02.125884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225264+0000","last_change":"2024-09-17T04:24:14.063222+0000","last_active":"2024-09-17T04:24:16.225264+0000","last_peered":"2024-09-17T04:24:16.225264+0000","last_clean":"2024-09-17T04:24:16.225264+0000","last_became_active":"2024-09-17T04:24:14.062750+0000","last_became_peered":"2024-09-17T04:24:14.062750+0000","last_unstale":"2024-09-17T04:24:16.225264+0000","last_undegraded":"2024-09-17T04:24:16.225264+0000","last_fullsized":"2024-09-17T04:24:16.225264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:08.023686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224491+0000","last_change":"2024-09-17T04:24:15.067950+0000","last_active":"2024-09-17T04:24:16.224491+0000","last_peered":"2024-09-17T04:24:16.224491+0000","last_clean":"2024-09-17T04:24:16.224491+0000","last_became_active":"2024-09-17T04:24:15.066567+0000","last_became_peered":"2024-09-17T04:24:15.066567+0000","last_unstale":"2024-09-17T04:24:16.224491+0000","last_undegraded":"2024-09-17T04:24:16.224491+0000","last_fullsized":"2024-09-17T04:24:16.224491+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:28.104736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223381+0000","last_change":"2024-09-17T04:24:14.059487+0000","last_active":"2024-09-17T04:24:16.223381+0000","last_peered":"2024-09-17T04:24:16.223381+0000","last_clean":"2024-09-17T04:24:16.223381+0000","last_became_active":"2024-09-17T04:24:14.059186+0000","last_became_peered":"2024-09-17T04:24:14.059186+0000","last_unstale":"2024-09-17T04:24:16.223381+0000","last_undegraded":"2024-09-17T04:24:16.223381+0000","last_fullsized":"2024-09-17T04:24:16.223381+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:03.320698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054147+0000","last_change":"2024-09-17T04:24:15.065941+0000","last_active":"2024-09-17T04:24:16.054147+0000","last_peered":"2024-09-17T04:24:16.054147+0000","last_clean":"2024-09-17T04:24:16.054147+0000","last_became_active":"2024-09-17T04:24:15.065582+0000","last_became_peered":"2024-09-17T04:24:15.065582+0000","last_unstale":"2024-09-17T04:24:16.054147+0000","last_undegraded":"2024-09-17T04:24:16.054147+0000","last_fullsized":"2024-09-17T04:24:16.054147+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:30.122754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.222894+0000","last_change":"2024-09-17T04:24:14.062678+0000","last_active":"2024-09-17T04:24:16.222894+0000","last_peered":"2024-09-17T04:24:16.222894+0000","last_clean":"2024-09-17T04:24:16.222894+0000","last_became_active":"2024-09-17T04:24:14.062281+0000","last_became_peered":"2024-09-17T04:24:14.062281+0000","last_unstale":"2024-09-17T04:24:16.222894+0000","last_undegraded":"2024-09-17T04:24:16.222894+0000","last_fullsized":"2024-09-17T04:24:16.222894+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:40.254206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224778+0000","last_change":"2024-09-17T04:24:15.064498+0000","last_active":"2024-09-17T04:24:16.224778+0000","last_peered":"2024-09-17T04:24:16.224778+0000","last_clean":"2024-09-17T04:24:16.224778+0000","last_became_active":"2024-09-17T04:24:15.062700+0000","last_became_peered":"2024-09-17T04:24:15.062700+0000","last_unstale":"2024-09-17T04:24:16.224778+0000","last_undegraded":"2024-09-17T04:24:16.224778+0000","last_fullsized":"2024-09-17T04:24:16.224778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:31.021313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223265+0000","last_change":"2024-09-17T04:24:14.047923+0000","last_active":"2024-09-17T04:24:16.223265+0000","last_peered":"2024-09-17T04:24:16.223265+0000","last_clean":"2024-09-17T04:24:16.223265+0000","last_became_active":"2024-09-17T04:24:14.047596+0000","last_became_peered":"2024-09-17T04:24:14.047596+0000","last_unstale":"2024-09-17T04:24:16.223265+0000","last_undegraded":"2024-09-17T04:24:16.223265+0000","last_fullsized":"2024-09-17T04:24:16.223265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:29.119993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225575+0000","last_change":"2024-09-17T04:24:15.060939+0000","last_active":"2024-09-17T04:24:16.225575+0000","last_peered":"2024-09-17T04:24:16.225575+0000","last_clean":"2024-09-17T04:24:16.225575+0000","last_became_active":"2024-09-17T04:24:15.060600+0000","last_became_peered":"2024-09-17T04:24:15.060600+0000","last_unstale":"2024-09-17T04:24:16.225575+0000","last_undegraded":"2024-09-17T04:24:16.225575+0000","last_fullsized":"2024-09-17T04:24:16.225575+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:05.895665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224176+0000","last_change":"2024-09-17T04:24:14.062616+0000","last_active":"2024-09-17T04:24:16.224176+0000","last_peered":"2024-09-17T04:24:16.224176+0000","last_clean":"2024-09-17T04:24:16.224176+0000","last_became_active":"2024-09-17T04:24:14.062182+0000","last_became_peered":"2024-09-17T04:24:14.062182+0000","last_unstale":"2024-09-17T04:24:16.224176+0000","last_undegraded":"2024-09-17T04:24:16.224176+0000","last_fullsized":"2024-09-17T04:24:16.224176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:51.246396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224637+0000","last_change":"2024-09-17T04:24:15.079589+0000","last_active":"2024-09-17T04:24:16.224637+0000","last_peered":"2024-09-17T04:24:16.224637+0000","last_clean":"2024-09-17T04:24:16.224637+0000","last_became_active":"2024-09-17T04:24:15.079019+0000","last_became_peered":"2024-09-17T04:24:15.079019+0000","last_unstale":"2024-09-17T04:24:16.224637+0000","last_undegraded":"2024-09-17T04:24:16.224637+0000","last_fullsized":"2024-09-17T04:24:16.224637+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:01.706191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056070+0000","last_change":"2024-09-17T04:24:15.080533+0000","last_active":"2024-09-17T04:24:16.056070+0000","last_peered":"2024-09-17T04:24:16.056070+0000","last_clean":"2024-09-17T04:24:16.056070+0000","last_became_active":"2024-09-17T04:24:15.080179+0000","last_became_peered":"2024-09-17T04:24:15.080179+0000","last_unstale":"2024-09-17T04:24:16.056070+0000","last_undegraded":"2024-09-17T04:24:16.056070+0000","last_fullsized":"2024-09-17T04:24:16.056070+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:58.719218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224154+0000","last_change":"2024-09-17T04:24:14.065850+0000","last_active":"2024-09-17T04:24:16.224154+0000","last_peered":"2024-09-17T04:24:16.224154+0000","last_clean":"2024-09-17T04:24:16.224154+0000","last_became_active":"2024-09-17T04:24:14.065444+0000","last_became_peered":"2024-09-17T04:24:14.065444+0000","last_unstale":"2024-09-17T04:24:16.224154+0000","last_undegraded":"2024-09-17T04:24:16.224154+0000","last_fullsized":"2024-09-17T04:24:16.224154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:51.121630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056174+0000","last_change":"2024-09-17T04:24:15.076511+0000","last_active":"2024-09-17T04:24:16.056174+0000","last_peered":"2024-09-17T04:24:16.056174+0000","last_clean":"2024-09-17T04:24:16.056174+0000","last_became_active":"2024-09-17T04:24:15.076037+0000","last_became_peered":"2024-09-17T04:24:15.076037+0000","last_unstale":"2024-09-17T04:24:16.056174+0000","last_undegraded":"2024-09-17T04:24:16.056174+0000","last_fullsized":"2024-09-17T04:24:16.056174+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:06:32.518189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224180+0000","last_change":"2024-09-17T04:24:14.060131+0000","last_active":"2024-09-17T04:24:16.224180+0000","last_peered":"2024-09-17T04:24:16.224180+0000","last_clean":"2024-09-17T04:24:16.224180+0000","last_became_active":"2024-09-17T04:24:14.059707+0000","last_became_peered":"2024-09-17T04:24:14.059707+0000","last_unstale":"2024-09-17T04:24:16.224180+0000","last_undegraded":"2024-09-17T04:24:16.224180+0000","last_fullsized":"2024-09-17T04:24:16.224180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:22.869429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223586+0000","last_change":"2024-09-17T04:24:15.068641+0000","last_active":"2024-09-17T04:24:16.223586+0000","last_peered":"2024-09-17T04:24:16.223586+0000","last_clean":"2024-09-17T04:24:16.223586+0000","last_became_active":"2024-09-17T04:24:15.065609+0000","last_became_peered":"2024-09-17T04:24:15.065609+0000","last_unstale":"2024-09-17T04:24:16.223586+0000","last_undegraded":"2024-09-17T04:24:16.223586+0000","last_fullsized":"2024-09-17T04:24:16.223586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:43:02.004482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225369+0000","last_change":"2024-09-17T04:24:14.056735+0000","last_active":"2024-09-17T04:24:16.225369+0000","last_peered":"2024-09-17T04:24:16.225369+0000","last_clean":"2024-09-17T04:24:16.225369+0000","last_became_active":"2024-09-17T04:24:14.056433+0000","last_became_peered":"2024-09-17T04:24:14.056433+0000","last_unstale":"2024-09-17T04:24:16.225369+0000","last_undegraded":"2024-09-17T04:24:16.225369+0000","last_fullsized":"2024-09-17T04:24:16.225369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:03.618384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225403+0000","last_change":"2024-09-17T04:24:15.054773+0000","last_active":"2024-09-17T04:24:16.225403+0000","last_peered":"2024-09-17T04:24:16.225403+0000","last_clean":"2024-09-17T04:24:16.225403+0000","last_became_active":"2024-09-17T04:24:15.054453+0000","last_became_peered":"2024-09-17T04:24:15.054453+0000","last_unstale":"2024-09-17T04:24:16.225403+0000","last_undegraded":"2024-09-17T04:24:16.225403+0000","last_fullsized":"2024-09-17T04:24:16.225403+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:58.942093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223981+0000","last_change":"2024-09-17T04:24:14.062995+0000","last_active":"2024-09-17T04:24:16.223981+0000","last_peered":"2024-09-17T04:24:16.223981+0000","last_clean":"2024-09-17T04:24:16.223981+0000","last_became_active":"2024-09-17T04:24:14.062615+0000","last_became_peered":"2024-09-17T04:24:14.062615+0000","last_unstale":"2024-09-17T04:24:16.223981+0000","last_undegraded":"2024-09-17T04:24:16.223981+0000","last_fullsized":"2024-09-17T04:24:16.223981+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:20:07.263105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476746,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27844,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343996,"statfs":{"total":96636764160,"available":96608251904,"internally_reserved":0,"allocated":827392,"data_stored":670785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476745,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":81447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476746,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476746,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":79961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":79835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":237568,"data_stored":79995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":835584,"data_stored":670841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542153,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":81447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:24:30.015 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:24:30.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.274+0000 7f9fcba00640 1 -- 172.21.15.100:0/3370554897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc40ad0e0 msgr2=0x7f9fc40ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:30.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.274+0000 7f9fcba00640 1 --2- 172.21.15.100:0/3370554897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc40ad0e0 0x7f9fc40ad4f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9fac005fd0 tx=0x7f9fac02a540 comp rx=0 tx=0).stop 2024-09-17T04:24:30.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.274+0000 7f9fcba00640 1 -- 172.21.15.100:0/3370554897 shutdown_connections 2024-09-17T04:24:30.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.274+0000 7f9fcba00640 1 --2- 172.21.15.100:0/3370554897 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fc40b3550 0x7f9fc40b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:30.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.274+0000 7f9fcba00640 1 --2- 172.21.15.100:0/3370554897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc40ad0e0 0x7f9fc40ad4f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:30.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.274+0000 7f9fcba00640 1 -- 172.21.15.100:0/3370554897 >> 172.21.15.100:0/3370554897 conn(0x7f9fc406d730 msgr2=0x7f9fc40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:30.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.274+0000 7f9fcba00640 1 -- 172.21.15.100:0/3370554897 shutdown_connections 2024-09-17T04:24:30.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.274+0000 7f9fcba00640 1 -- 172.21.15.100:0/3370554897 wait complete. 2024-09-17T04:24:30.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.276+0000 7f9fcba00640 1 Processor -- start 2024-09-17T04:24:30.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.277+0000 7f9fcba00640 1 -- start start 2024-09-17T04:24:30.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.278+0000 7f9fcba00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fc40ad0e0 0x7f9fc4156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:30.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.278+0000 7f9fcba00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc40b3550 0x7f9fc41f5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:30.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.278+0000 7f9fcba00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fc415d760 0x7f9fc415b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:30.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.278+0000 7f9fcba00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc40b4da0 con 0x7f9fc40b3550 2024-09-17T04:24:30.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.278+0000 7f9fcba00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc40b4c20 con 0x7f9fc40ad0e0 2024-09-17T04:24:30.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.278+0000 7f9fcba00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc40b3ea0 con 0x7f9fc415d760 2024-09-17T04:24:30.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.278+0000 7f9fbbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc40b3550 0x7f9fc41f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:30.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.278+0000 7f9fbbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc40b3550 0x7f9fc41f5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48256/0 (socket says 172.21.15.100:48256) 2024-09-17T04:24:30.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.278+0000 7f9fbbe00640 1 -- 172.21.15.100:0/3033366597 learned_addr learned my addr 172.21.15.100:0/3033366597 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:30.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.278+0000 7f9fc9000640 1 --2- 172.21.15.100:0/3033366597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fc40ad0e0 0x7f9fc4156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:30.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.278+0000 7f9fc9a00640 1 --2- 172.21.15.100:0/3033366597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fc415d760 0x7f9fc415b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:30.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.279+0000 7f9fbbe00640 1 -- 172.21.15.100:0/3033366597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fc415d760 msgr2=0x7f9fc415b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:30.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.279+0000 7f9fbbe00640 1 --2- 172.21.15.100:0/3033366597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fc415d760 0x7f9fc415b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:30.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.279+0000 7f9fbbe00640 1 -- 172.21.15.100:0/3033366597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fc40ad0e0 msgr2=0x7f9fc4156700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:30.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.279+0000 7f9fbbe00640 1 --2- 172.21.15.100:0/3033366597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fc40ad0e0 0x7f9fc4156700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:30.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.279+0000 7f9fbbe00640 1 -- 172.21.15.100:0/3033366597 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc414f4b0 con 0x7f9fc40b3550 2024-09-17T04:24:30.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.279+0000 7f9fbbe00640 1 --2- 172.21.15.100:0/3033366597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc40b3550 0x7f9fc41f5070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9fac034040 tx=0x7f9fac0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:30.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.279+0000 7f9fc1600640 1 -- 172.21.15.100:0/3033366597 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac03e030 con 0x7f9fc40b3550 2024-09-17T04:24:30.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.279+0000 7f9fc1600640 1 -- 172.21.15.100:0/3033366597 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fac003d90 con 0x7f9fc40b3550 2024-09-17T04:24:30.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.279+0000 7f9fc1600640 1 -- 172.21.15.100:0/3033366597 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac035040 con 0x7f9fc40b3550 2024-09-17T04:24:30.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.279+0000 7f9fcba00640 1 -- 172.21.15.100:0/3033366597 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc414f790 con 0x7f9fc40b3550 2024-09-17T04:24:30.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.279+0000 7f9fcba00640 1 -- 172.21.15.100:0/3033366597 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc414fd50 con 0x7f9fc40b3550 2024-09-17T04:24:30.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.281+0000 7f9fcba00640 1 -- 172.21.15.100:0/3033366597 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fc40b4370 con 0x7f9fc40b3550 2024-09-17T04:24:30.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.281+0000 7f9fc1600640 1 -- 172.21.15.100:0/3033366597 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9fac041030 con 0x7f9fc40b3550 2024-09-17T04:24:30.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.282+0000 7f9fc1600640 1 --2- 172.21.15.100:0/3033366597 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9fa0078bc0 0x7f9fa007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:30.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.282+0000 7f9fc1600640 1 -- 172.21.15.100:0/3033366597 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f9fac0bb450 con 0x7f9fc40b3550 2024-09-17T04:24:30.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.283+0000 7f9fc9000640 1 --2- 172.21.15.100:0/3033366597 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9fa0078bc0 0x7f9fa007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:30.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.283+0000 7f9fc9000640 1 --2- 172.21.15.100:0/3033366597 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9fa0078bc0 0x7f9fa007b0d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40015e0 tx=0x7f9fb4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:30.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.287+0000 7f9fc1600640 1 -- 172.21.15.100:0/3033366597 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9fac083880 con 0x7f9fc40b3550 2024-09-17T04:24:30.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.467+0000 7f9fcba00640 1 -- 172.21.15.100:0/3033366597 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9fc4069700 con 0x7f9fa0078bc0 2024-09-17T04:24:30.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.472+0000 7f9fc1600640 1 -- 172.21.15.100:0/3033366597 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251826 (secure 0 0 0) 0x7f9fc4069700 con 0x7f9fa0078bc0 2024-09-17T04:24:30.475 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:30.476 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:24:30.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.476+0000 7f9fcba00640 1 -- 172.21.15.100:0/3033366597 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9fa0078bc0 msgr2=0x7f9fa007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:30.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.476+0000 7f9fcba00640 1 --2- 172.21.15.100:0/3033366597 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9fa0078bc0 0x7f9fa007b0d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40015e0 tx=0x7f9fb4009040 comp rx=0 tx=0).stop 2024-09-17T04:24:30.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.476+0000 7f9fcba00640 1 -- 172.21.15.100:0/3033366597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc40b3550 msgr2=0x7f9fc41f5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:30.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.476+0000 7f9fcba00640 1 --2- 172.21.15.100:0/3033366597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc40b3550 0x7f9fc41f5070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9fac034040 tx=0x7f9fac0038a0 comp rx=0 tx=0).stop 2024-09-17T04:24:30.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.476+0000 7f9fcba00640 1 -- 172.21.15.100:0/3033366597 shutdown_connections 2024-09-17T04:24:30.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.476+0000 7f9fcba00640 1 --2- 172.21.15.100:0/3033366597 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9fa0078bc0 0x7f9fa007b0d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:30.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.476+0000 7f9fcba00640 1 --2- 172.21.15.100:0/3033366597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fc415d760 0x7f9fc415b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:30.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.476+0000 7f9fcba00640 1 --2- 172.21.15.100:0/3033366597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc40b3550 0x7f9fc41f5070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:30.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.476+0000 7f9fcba00640 1 --2- 172.21.15.100:0/3033366597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fc40ad0e0 0x7f9fc4156700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:30.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.476+0000 7f9fcba00640 1 -- 172.21.15.100:0/3033366597 >> 172.21.15.100:0/3033366597 conn(0x7f9fc406d730 msgr2=0x7f9fc415dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:30.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.476+0000 7f9fcba00640 1 -- 172.21.15.100:0/3033366597 shutdown_connections 2024-09-17T04:24:30.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.476+0000 7f9fcba00640 1 -- 172.21.15.100:0/3033366597 wait complete. 2024-09-17T04:24:30.498 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-09-17T04:24:29.968443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219276,"kb_used_data":3016,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755444,"statfs":{"total":773094113280,"available":772869574656,"internally_reserved":0,"allocated":3088384,"data_stored":1824236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003962"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224232+0000","last_change":"2024-09-17T04:24:15.051472+0000","last_active":"2024-09-17T04:24:16.224232+0000","last_peered":"2024-09-17T04:24:16.224232+0000","last_clean":"2024-09-17T04:24:16.224232+0000","last_became_active":"2024-09-17T04:24:15.051162+0000","last_became_peered":"2024-09-17T04:24:15.051162+0000","last_unstale":"2024-09-17T04:24:16.224232+0000","last_undegraded":"2024-09-17T04:24:16.224232+0000","last_fullsized":"2024-09-17T04:24:16.224232+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:35:10.792366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223431+0000","last_change":"2024-09-17T04:24:15.069649+0000","last_active":"2024-09-17T04:24:16.223431+0000","last_peered":"2024-09-17T04:24:16.223431+0000","last_clean":"2024-09-17T04:24:16.223431+0000","last_became_active":"2024-09-17T04:24:15.068975+0000","last_became_peered":"2024-09-17T04:24:15.068975+0000","last_unstale":"2024-09-17T04:24:16.223431+0000","last_undegraded":"2024-09-17T04:24:16.223431+0000","last_fullsized":"2024-09-17T04:24:16.223431+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:47.639252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223497+0000","last_change":"2024-09-17T04:24:15.052608+0000","last_active":"2024-09-17T04:24:16.223497+0000","last_peered":"2024-09-17T04:24:16.223497+0000","last_clean":"2024-09-17T04:24:16.223497+0000","last_became_active":"2024-09-17T04:24:15.052252+0000","last_became_peered":"2024-09-17T04:24:15.052252+0000","last_unstale":"2024-09-17T04:24:16.223497+0000","last_undegraded":"2024-09-17T04:24:16.223497+0000","last_fullsized":"2024-09-17T04:24:16.223497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:34:23.523300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055891+0000","last_change":"2024-09-17T04:24:15.059215+0000","last_active":"2024-09-17T04:24:16.055891+0000","last_peered":"2024-09-17T04:24:16.055891+0000","last_clean":"2024-09-17T04:24:16.055891+0000","last_became_active":"2024-09-17T04:24:15.058921+0000","last_became_peered":"2024-09-17T04:24:15.058921+0000","last_unstale":"2024-09-17T04:24:16.055891+0000","last_undegraded":"2024-09-17T04:24:16.055891+0000","last_fullsized":"2024-09-17T04:24:16.055891+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:34.142688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224122+0000","last_change":"2024-09-17T04:24:15.042545+0000","last_active":"2024-09-17T04:24:16.224122+0000","last_peered":"2024-09-17T04:24:16.224122+0000","last_clean":"2024-09-17T04:24:16.224122+0000","last_became_active":"2024-09-17T04:24:15.042235+0000","last_became_peered":"2024-09-17T04:24:15.042235+0000","last_unstale":"2024-09-17T04:24:16.224122+0000","last_undegraded":"2024-09-17T04:24:16.224122+0000","last_fullsized":"2024-09-17T04:24:16.224122+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:06:32.518189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056013+0000","last_change":"2024-09-17T04:24:15.057624+0000","last_active":"2024-09-17T04:24:16.056013+0000","last_peered":"2024-09-17T04:24:16.056013+0000","last_clean":"2024-09-17T04:24:16.056013+0000","last_became_active":"2024-09-17T04:24:15.057329+0000","last_became_peered":"2024-09-17T04:24:15.057329+0000","last_unstale":"2024-09-17T04:24:16.056013+0000","last_undegraded":"2024-09-17T04:24:16.056013+0000","last_fullsized":"2024-09-17T04:24:16.056013+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:44.355705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223880+0000","last_change":"2024-09-17T04:24:15.062645+0000","last_active":"2024-09-17T04:24:16.223880+0000","last_peered":"2024-09-17T04:24:16.223880+0000","last_clean":"2024-09-17T04:24:16.223880+0000","last_became_active":"2024-09-17T04:24:15.062227+0000","last_became_peered":"2024-09-17T04:24:15.062227+0000","last_unstale":"2024-09-17T04:24:16.223880+0000","last_undegraded":"2024-09-17T04:24:16.223880+0000","last_fullsized":"2024-09-17T04:24:16.223880+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:13.159492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224308+0000","last_change":"2024-09-17T04:24:15.062993+0000","last_active":"2024-09-17T04:24:16.224308+0000","last_peered":"2024-09-17T04:24:16.224308+0000","last_clean":"2024-09-17T04:24:16.224308+0000","last_became_active":"2024-09-17T04:24:15.062311+0000","last_became_peered":"2024-09-17T04:24:15.062311+0000","last_unstale":"2024-09-17T04:24:16.224308+0000","last_undegraded":"2024-09-17T04:24:16.224308+0000","last_fullsized":"2024-09-17T04:24:16.224308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:26:42.504856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224742+0000","last_change":"2024-09-17T04:24:15.070288+0000","last_active":"2024-09-17T04:24:16.224742+0000","last_peered":"2024-09-17T04:24:16.224742+0000","last_clean":"2024-09-17T04:24:16.224742+0000","last_became_active":"2024-09-17T04:24:15.068589+0000","last_became_peered":"2024-09-17T04:24:15.068589+0000","last_unstale":"2024-09-17T04:24:16.224742+0000","last_undegraded":"2024-09-17T04:24:16.224742+0000","last_fullsized":"2024-09-17T04:24:16.224742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:33.110035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225192+0000","last_change":"2024-09-17T04:24:15.062446+0000","last_active":"2024-09-17T04:24:16.225192+0000","last_peered":"2024-09-17T04:24:16.225192+0000","last_clean":"2024-09-17T04:24:16.225192+0000","last_became_active":"2024-09-17T04:24:15.060522+0000","last_became_peered":"2024-09-17T04:24:15.060522+0000","last_unstale":"2024-09-17T04:24:16.225192+0000","last_undegraded":"2024-09-17T04:24:16.225192+0000","last_fullsized":"2024-09-17T04:24:16.225192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:09.349585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224815+0000","last_change":"2024-09-17T04:24:15.062048+0000","last_active":"2024-09-17T04:24:16.224815+0000","last_peered":"2024-09-17T04:24:16.224815+0000","last_clean":"2024-09-17T04:24:16.224815+0000","last_became_active":"2024-09-17T04:24:15.060887+0000","last_became_peered":"2024-09-17T04:24:15.060887+0000","last_unstale":"2024-09-17T04:24:16.224815+0000","last_undegraded":"2024-09-17T04:24:16.224815+0000","last_fullsized":"2024-09-17T04:24:16.224815+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:22.452641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054679+0000","last_change":"2024-09-17T04:24:15.068320+0000","last_active":"2024-09-17T04:24:16.054679+0000","last_peered":"2024-09-17T04:24:16.054679+0000","last_clean":"2024-09-17T04:24:16.054679+0000","last_became_active":"2024-09-17T04:24:15.067973+0000","last_became_peered":"2024-09-17T04:24:15.067973+0000","last_unstale":"2024-09-17T04:24:16.054679+0000","last_undegraded":"2024-09-17T04:24:16.054679+0000","last_fullsized":"2024-09-17T04:24:16.054679+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:04.323459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225225+0000","last_change":"2024-09-17T04:24:15.047951+0000","last_active":"2024-09-17T04:24:16.225225+0000","last_peered":"2024-09-17T04:24:16.225225+0000","last_clean":"2024-09-17T04:24:16.225225+0000","last_became_active":"2024-09-17T04:24:15.047536+0000","last_became_peered":"2024-09-17T04:24:15.047536+0000","last_unstale":"2024-09-17T04:24:16.225225+0000","last_undegraded":"2024-09-17T04:24:16.225225+0000","last_fullsized":"2024-09-17T04:24:16.225225+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:38:48.386032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224034+0000","last_change":"2024-09-17T04:24:15.070607+0000","last_active":"2024-09-17T04:24:16.224034+0000","last_peered":"2024-09-17T04:24:16.224034+0000","last_clean":"2024-09-17T04:24:16.224034+0000","last_became_active":"2024-09-17T04:24:15.067087+0000","last_became_peered":"2024-09-17T04:24:15.067087+0000","last_unstale":"2024-09-17T04:24:16.224034+0000","last_undegraded":"2024-09-17T04:24:16.224034+0000","last_fullsized":"2024-09-17T04:24:16.224034+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:09.349585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224091+0000","last_change":"2024-09-17T04:24:15.076095+0000","last_active":"2024-09-17T04:24:16.224091+0000","last_peered":"2024-09-17T04:24:16.224091+0000","last_clean":"2024-09-17T04:24:16.224091+0000","last_became_active":"2024-09-17T04:24:15.075803+0000","last_became_peered":"2024-09-17T04:24:15.075803+0000","last_unstale":"2024-09-17T04:24:16.224091+0000","last_undegraded":"2024-09-17T04:24:16.224091+0000","last_fullsized":"2024-09-17T04:24:16.224091+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:28.104736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055352+0000","last_change":"2024-09-17T04:24:15.053797+0000","last_active":"2024-09-17T04:24:16.055352+0000","last_peered":"2024-09-17T04:24:16.055352+0000","last_clean":"2024-09-17T04:24:16.055352+0000","last_became_active":"2024-09-17T04:24:15.053449+0000","last_became_peered":"2024-09-17T04:24:15.053449+0000","last_unstale":"2024-09-17T04:24:16.055352+0000","last_undegraded":"2024-09-17T04:24:16.055352+0000","last_fullsized":"2024-09-17T04:24:16.055352+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:49:03.743620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056725+0000","last_change":"2024-09-17T04:24:15.055632+0000","last_active":"2024-09-17T04:24:16.056725+0000","last_peered":"2024-09-17T04:24:16.056725+0000","last_clean":"2024-09-17T04:24:16.056725+0000","last_became_active":"2024-09-17T04:24:15.055156+0000","last_became_peered":"2024-09-17T04:24:15.055156+0000","last_unstale":"2024-09-17T04:24:16.056725+0000","last_undegraded":"2024-09-17T04:24:16.056725+0000","last_fullsized":"2024-09-17T04:24:16.056725+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:16.120591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.052124+0000","last_change":"2024-09-17T04:24:15.057788+0000","last_active":"2024-09-17T04:24:16.052124+0000","last_peered":"2024-09-17T04:24:16.052124+0000","last_clean":"2024-09-17T04:24:16.052124+0000","last_became_active":"2024-09-17T04:24:15.057141+0000","last_became_peered":"2024-09-17T04:24:15.057141+0000","last_unstale":"2024-09-17T04:24:16.052124+0000","last_undegraded":"2024-09-17T04:24:16.052124+0000","last_fullsized":"2024-09-17T04:24:16.052124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:10.704647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224426+0000","last_change":"2024-09-17T04:24:15.067312+0000","last_active":"2024-09-17T04:24:16.224426+0000","last_peered":"2024-09-17T04:24:16.224426+0000","last_clean":"2024-09-17T04:24:16.224426+0000","last_became_active":"2024-09-17T04:24:15.059516+0000","last_became_peered":"2024-09-17T04:24:15.059516+0000","last_unstale":"2024-09-17T04:24:16.224426+0000","last_undegraded":"2024-09-17T04:24:16.224426+0000","last_fullsized":"2024-09-17T04:24:16.224426+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:16.120591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224948+0000","last_change":"2024-09-17T04:24:15.076912+0000","last_active":"2024-09-17T04:24:16.224948+0000","last_peered":"2024-09-17T04:24:16.224948+0000","last_clean":"2024-09-17T04:24:16.224948+0000","last_became_active":"2024-09-17T04:24:15.076617+0000","last_became_peered":"2024-09-17T04:24:15.076617+0000","last_unstale":"2024-09-17T04:24:16.224948+0000","last_undegraded":"2024-09-17T04:24:16.224948+0000","last_fullsized":"2024-09-17T04:24:16.224948+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:47.639252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223809+0000","last_change":"2024-09-17T04:24:15.070958+0000","last_active":"2024-09-17T04:24:16.223809+0000","last_peered":"2024-09-17T04:24:16.223809+0000","last_clean":"2024-09-17T04:24:16.223809+0000","last_became_active":"2024-09-17T04:24:15.070624+0000","last_became_peered":"2024-09-17T04:24:15.070624+0000","last_unstale":"2024-09-17T04:24:16.223809+0000","last_undegraded":"2024-09-17T04:24:16.223809+0000","last_fullsized":"2024-09-17T04:24:16.223809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:13.752351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055648+0000","last_change":"2024-09-17T04:24:15.058815+0000","last_active":"2024-09-17T04:24:16.055648+0000","last_peered":"2024-09-17T04:24:16.055648+0000","last_clean":"2024-09-17T04:24:16.055648+0000","last_became_active":"2024-09-17T04:24:15.058482+0000","last_became_peered":"2024-09-17T04:24:15.058482+0000","last_unstale":"2024-09-17T04:24:16.055648+0000","last_undegraded":"2024-09-17T04:24:16.055648+0000","last_fullsized":"2024-09-17T04:24:16.055648+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:18.409788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224382+0000","last_change":"2024-09-17T04:24:15.070200+0000","last_active":"2024-09-17T04:24:16.224382+0000","last_peered":"2024-09-17T04:24:16.224382+0000","last_clean":"2024-09-17T04:24:16.224382+0000","last_became_active":"2024-09-17T04:24:15.069872+0000","last_became_peered":"2024-09-17T04:24:15.069872+0000","last_unstale":"2024-09-17T04:24:16.224382+0000","last_undegraded":"2024-09-17T04:24:16.224382+0000","last_fullsized":"2024-09-17T04:24:16.224382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:22.880022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055404+0000","last_change":"2024-09-17T04:24:15.050935+0000","last_active":"2024-09-17T04:24:16.055404+0000","last_peered":"2024-09-17T04:24:16.055404+0000","last_clean":"2024-09-17T04:24:16.055404+0000","last_became_active":"2024-09-17T04:24:15.048946+0000","last_became_peered":"2024-09-17T04:24:15.048946+0000","last_unstale":"2024-09-17T04:24:16.055404+0000","last_undegraded":"2024-09-17T04:24:16.055404+0000","last_fullsized":"2024-09-17T04:24:16.055404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:11:09.116226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225400+0000","last_change":"2024-09-17T04:24:15.074863+0000","last_active":"2024-09-17T04:24:16.225400+0000","last_peered":"2024-09-17T04:24:16.225400+0000","last_clean":"2024-09-17T04:24:16.225400+0000","last_became_active":"2024-09-17T04:24:15.074467+0000","last_became_peered":"2024-09-17T04:24:15.074467+0000","last_unstale":"2024-09-17T04:24:16.225400+0000","last_undegraded":"2024-09-17T04:24:16.225400+0000","last_fullsized":"2024-09-17T04:24:16.225400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:36:59.215343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225206+0000","last_change":"2024-09-17T04:24:15.069430+0000","last_active":"2024-09-17T04:24:16.225206+0000","last_peered":"2024-09-17T04:24:16.225206+0000","last_clean":"2024-09-17T04:24:16.225206+0000","last_became_active":"2024-09-17T04:24:15.069117+0000","last_became_peered":"2024-09-17T04:24:15.069117+0000","last_unstale":"2024-09-17T04:24:16.225206+0000","last_undegraded":"2024-09-17T04:24:16.225206+0000","last_fullsized":"2024-09-17T04:24:16.225206+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:49:03.743620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224018+0000","last_change":"2024-09-17T04:24:15.057539+0000","last_active":"2024-09-17T04:24:16.224018+0000","last_peered":"2024-09-17T04:24:16.224018+0000","last_clean":"2024-09-17T04:24:16.224018+0000","last_became_active":"2024-09-17T04:24:15.057194+0000","last_became_peered":"2024-09-17T04:24:15.057194+0000","last_unstale":"2024-09-17T04:24:16.224018+0000","last_undegraded":"2024-09-17T04:24:16.224018+0000","last_fullsized":"2024-09-17T04:24:16.224018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:09.349585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225430+0000","last_change":"2024-09-17T04:24:15.075902+0000","last_active":"2024-09-17T04:24:16.225430+0000","last_peered":"2024-09-17T04:24:16.225430+0000","last_clean":"2024-09-17T04:24:16.225430+0000","last_became_active":"2024-09-17T04:24:15.075625+0000","last_became_peered":"2024-09-17T04:24:15.075625+0000","last_unstale":"2024-09-17T04:24:16.225430+0000","last_undegraded":"2024-09-17T04:24:16.225430+0000","last_fullsized":"2024-09-17T04:24:16.225430+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:34.202614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223788+0000","last_change":"2024-09-17T04:24:15.052108+0000","last_active":"2024-09-17T04:24:16.223788+0000","last_peered":"2024-09-17T04:24:16.223788+0000","last_clean":"2024-09-17T04:24:16.223788+0000","last_became_active":"2024-09-17T04:24:15.051794+0000","last_became_peered":"2024-09-17T04:24:15.051794+0000","last_unstale":"2024-09-17T04:24:16.223788+0000","last_undegraded":"2024-09-17T04:24:16.223788+0000","last_fullsized":"2024-09-17T04:24:16.223788+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:13:56.486773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055909+0000","last_change":"2024-09-17T04:24:15.057335+0000","last_active":"2024-09-17T04:24:16.055909+0000","last_peered":"2024-09-17T04:24:16.055909+0000","last_clean":"2024-09-17T04:24:16.055909+0000","last_became_active":"2024-09-17T04:24:15.056924+0000","last_became_peered":"2024-09-17T04:24:15.056924+0000","last_unstale":"2024-09-17T04:24:16.055909+0000","last_undegraded":"2024-09-17T04:24:16.055909+0000","last_fullsized":"2024-09-17T04:24:16.055909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:29:56.515791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.052052+0000","last_change":"2024-09-17T04:24:15.054070+0000","last_active":"2024-09-17T04:24:16.052052+0000","last_peered":"2024-09-17T04:24:16.052052+0000","last_clean":"2024-09-17T04:24:16.052052+0000","last_became_active":"2024-09-17T04:24:15.052619+0000","last_became_peered":"2024-09-17T04:24:15.052619+0000","last_unstale":"2024-09-17T04:24:16.052052+0000","last_undegraded":"2024-09-17T04:24:16.052052+0000","last_fullsized":"2024-09-17T04:24:16.052052+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:49.409374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223562+0000","last_change":"2024-09-17T04:24:15.061084+0000","last_active":"2024-09-17T04:24:16.223562+0000","last_peered":"2024-09-17T04:24:16.223562+0000","last_clean":"2024-09-17T04:24:16.223562+0000","last_became_active":"2024-09-17T04:24:15.060769+0000","last_became_peered":"2024-09-17T04:24:15.060769+0000","last_unstale":"2024-09-17T04:24:16.223562+0000","last_undegraded":"2024-09-17T04:24:16.223562+0000","last_fullsized":"2024-09-17T04:24:16.223562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:13:56.486773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054106+0000","last_change":"2024-09-17T04:24:15.068899+0000","last_active":"2024-09-17T04:24:16.054106+0000","last_peered":"2024-09-17T04:24:16.054106+0000","last_clean":"2024-09-17T04:24:16.054106+0000","last_became_active":"2024-09-17T04:24:15.068595+0000","last_became_peered":"2024-09-17T04:24:15.068595+0000","last_unstale":"2024-09-17T04:24:16.054106+0000","last_undegraded":"2024-09-17T04:24:16.054106+0000","last_fullsized":"2024-09-17T04:24:16.054106+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:13.785027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225374+0000","last_change":"2024-09-17T04:24:14.062806+0000","last_active":"2024-09-17T04:24:16.225374+0000","last_peered":"2024-09-17T04:24:16.225374+0000","last_clean":"2024-09-17T04:24:16.225374+0000","last_became_active":"2024-09-17T04:24:14.062282+0000","last_became_peered":"2024-09-17T04:24:14.062282+0000","last_unstale":"2024-09-17T04:24:16.225374+0000","last_undegraded":"2024-09-17T04:24:16.225374+0000","last_fullsized":"2024-09-17T04:24:16.225374+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:23.698652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224005+0000","last_change":"2024-09-17T04:24:15.075318+0000","last_active":"2024-09-17T04:24:16.224005+0000","last_peered":"2024-09-17T04:24:16.224005+0000","last_clean":"2024-09-17T04:24:16.224005+0000","last_became_active":"2024-09-17T04:24:15.074876+0000","last_became_peered":"2024-09-17T04:24:15.074876+0000","last_unstale":"2024-09-17T04:24:16.224005+0000","last_undegraded":"2024-09-17T04:24:16.224005+0000","last_fullsized":"2024-09-17T04:24:16.224005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:47.639252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224710+0000","last_change":"2024-09-17T04:24:14.056222+0000","last_active":"2024-09-17T04:24:16.224710+0000","last_peered":"2024-09-17T04:24:16.224710+0000","last_clean":"2024-09-17T04:24:16.224710+0000","last_became_active":"2024-09-17T04:24:14.055874+0000","last_became_peered":"2024-09-17T04:24:14.055874+0000","last_unstale":"2024-09-17T04:24:16.224710+0000","last_undegraded":"2024-09-17T04:24:16.224710+0000","last_fullsized":"2024-09-17T04:24:16.224710+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:44.734015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224568+0000","last_change":"2024-09-17T04:24:14.054189+0000","last_active":"2024-09-17T04:24:16.224568+0000","last_peered":"2024-09-17T04:24:16.224568+0000","last_clean":"2024-09-17T04:24:16.224568+0000","last_became_active":"2024-09-17T04:24:14.053870+0000","last_became_peered":"2024-09-17T04:24:14.053870+0000","last_unstale":"2024-09-17T04:24:16.224568+0000","last_undegraded":"2024-09-17T04:24:16.224568+0000","last_fullsized":"2024-09-17T04:24:16.224568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:12.776180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224542+0000","last_change":"2024-09-17T04:24:15.059936+0000","last_active":"2024-09-17T04:24:16.224542+0000","last_peered":"2024-09-17T04:24:16.224542+0000","last_clean":"2024-09-17T04:24:16.224542+0000","last_became_active":"2024-09-17T04:24:15.059645+0000","last_became_peered":"2024-09-17T04:24:15.059645+0000","last_unstale":"2024-09-17T04:24:16.224542+0000","last_undegraded":"2024-09-17T04:24:16.224542+0000","last_fullsized":"2024-09-17T04:24:16.224542+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:08:11.703658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055748+0000","last_change":"2024-09-17T04:24:14.060082+0000","last_active":"2024-09-17T04:24:16.055748+0000","last_peered":"2024-09-17T04:24:16.055748+0000","last_clean":"2024-09-17T04:24:16.055748+0000","last_became_active":"2024-09-17T04:24:14.058795+0000","last_became_peered":"2024-09-17T04:24:14.058795+0000","last_unstale":"2024-09-17T04:24:16.055748+0000","last_undegraded":"2024-09-17T04:24:16.055748+0000","last_fullsized":"2024-09-17T04:24:16.055748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:21.535057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.222826+0000","last_change":"2024-09-17T04:24:15.061788+0000","last_active":"2024-09-17T04:24:16.222826+0000","last_peered":"2024-09-17T04:24:16.222826+0000","last_clean":"2024-09-17T04:24:16.222826+0000","last_became_active":"2024-09-17T04:24:15.055072+0000","last_became_peered":"2024-09-17T04:24:15.055072+0000","last_unstale":"2024-09-17T04:24:16.222826+0000","last_undegraded":"2024-09-17T04:24:16.222826+0000","last_fullsized":"2024-09-17T04:24:16.222826+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:13.752351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225245+0000","last_change":"2024-09-17T04:24:14.053595+0000","last_active":"2024-09-17T04:24:16.225245+0000","last_peered":"2024-09-17T04:24:16.225245+0000","last_clean":"2024-09-17T04:24:16.225245+0000","last_became_active":"2024-09-17T04:24:14.053260+0000","last_became_peered":"2024-09-17T04:24:14.053260+0000","last_unstale":"2024-09-17T04:24:16.225245+0000","last_undegraded":"2024-09-17T04:24:16.225245+0000","last_fullsized":"2024-09-17T04:24:16.225245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:51.429181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056660+0000","last_change":"2024-09-17T04:24:15.053812+0000","last_active":"2024-09-17T04:24:16.056660+0000","last_peered":"2024-09-17T04:24:16.056660+0000","last_clean":"2024-09-17T04:24:16.056660+0000","last_became_active":"2024-09-17T04:24:15.053501+0000","last_became_peered":"2024-09-17T04:24:15.053501+0000","last_unstale":"2024-09-17T04:24:16.056660+0000","last_undegraded":"2024-09-17T04:24:16.056660+0000","last_fullsized":"2024-09-17T04:24:16.056660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:47.959956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056115+0000","last_change":"2024-09-17T04:24:14.059203+0000","last_active":"2024-09-17T04:24:16.056115+0000","last_peered":"2024-09-17T04:24:16.056115+0000","last_clean":"2024-09-17T04:24:16.056115+0000","last_became_active":"2024-09-17T04:24:14.058623+0000","last_became_peered":"2024-09-17T04:24:14.058623+0000","last_unstale":"2024-09-17T04:24:16.056115+0000","last_undegraded":"2024-09-17T04:24:16.056115+0000","last_fullsized":"2024-09-17T04:24:16.056115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:20.372651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223819+0000","last_change":"2024-09-17T04:24:15.068669+0000","last_active":"2024-09-17T04:24:16.223819+0000","last_peered":"2024-09-17T04:24:16.223819+0000","last_clean":"2024-09-17T04:24:16.223819+0000","last_became_active":"2024-09-17T04:24:15.066578+0000","last_became_peered":"2024-09-17T04:24:15.066578+0000","last_unstale":"2024-09-17T04:24:16.223819+0000","last_undegraded":"2024-09-17T04:24:16.223819+0000","last_fullsized":"2024-09-17T04:24:16.223819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:01.706191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223044+0000","last_change":"2024-09-17T04:24:14.061969+0000","last_active":"2024-09-17T04:24:16.223044+0000","last_peered":"2024-09-17T04:24:16.223044+0000","last_clean":"2024-09-17T04:24:16.223044+0000","last_became_active":"2024-09-17T04:24:14.061593+0000","last_became_peered":"2024-09-17T04:24:14.061593+0000","last_unstale":"2024-09-17T04:24:16.223044+0000","last_undegraded":"2024-09-17T04:24:16.223044+0000","last_fullsized":"2024-09-17T04:24:16.223044+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:48.406614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054926+0000","last_change":"2024-09-17T04:24:15.071779+0000","last_active":"2024-09-17T04:24:16.054926+0000","last_peered":"2024-09-17T04:24:16.054926+0000","last_clean":"2024-09-17T04:24:16.054926+0000","last_became_active":"2024-09-17T04:24:15.071431+0000","last_became_peered":"2024-09-17T04:24:15.071431+0000","last_unstale":"2024-09-17T04:24:16.054926+0000","last_undegraded":"2024-09-17T04:24:16.054926+0000","last_fullsized":"2024-09-17T04:24:16.054926+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:10.704647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224939+0000","last_change":"2024-09-17T04:24:14.058729+0000","last_active":"2024-09-17T04:24:16.224939+0000","last_peered":"2024-09-17T04:24:16.224939+0000","last_clean":"2024-09-17T04:24:16.224939+0000","last_became_active":"2024-09-17T04:24:14.057616+0000","last_became_peered":"2024-09-17T04:24:14.057616+0000","last_unstale":"2024-09-17T04:24:16.224939+0000","last_undegraded":"2024-09-17T04:24:16.224939+0000","last_fullsized":"2024-09-17T04:24:16.224939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:42.267414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223701+0000","last_change":"2024-09-17T04:24:15.056613+0000","last_active":"2024-09-17T04:24:16.223701+0000","last_peered":"2024-09-17T04:24:16.223701+0000","last_clean":"2024-09-17T04:24:16.223701+0000","last_became_active":"2024-09-17T04:24:15.056300+0000","last_became_peered":"2024-09-17T04:24:15.056300+0000","last_unstale":"2024-09-17T04:24:16.223701+0000","last_undegraded":"2024-09-17T04:24:16.223701+0000","last_fullsized":"2024-09-17T04:24:16.223701+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:34.142688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223885+0000","last_change":"2024-09-17T04:24:14.054359+0000","last_active":"2024-09-17T04:24:16.223885+0000","last_peered":"2024-09-17T04:24:16.223885+0000","last_clean":"2024-09-17T04:24:16.223885+0000","last_became_active":"2024-09-17T04:24:14.053935+0000","last_became_peered":"2024-09-17T04:24:14.053935+0000","last_unstale":"2024-09-17T04:24:16.223885+0000","last_undegraded":"2024-09-17T04:24:16.223885+0000","last_fullsized":"2024-09-17T04:24:16.223885+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:01.211818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223752+0000","last_change":"2024-09-17T04:24:15.068236+0000","last_active":"2024-09-17T04:24:16.223752+0000","last_peered":"2024-09-17T04:24:16.223752+0000","last_clean":"2024-09-17T04:24:16.223752+0000","last_became_active":"2024-09-17T04:24:15.067460+0000","last_became_peered":"2024-09-17T04:24:15.067460+0000","last_unstale":"2024-09-17T04:24:16.223752+0000","last_undegraded":"2024-09-17T04:24:16.223752+0000","last_fullsized":"2024-09-17T04:24:16.223752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:44:58.329654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056099+0000","last_change":"2024-09-17T04:24:14.058127+0000","last_active":"2024-09-17T04:24:16.056099+0000","last_peered":"2024-09-17T04:24:16.056099+0000","last_clean":"2024-09-17T04:24:16.056099+0000","last_became_active":"2024-09-17T04:24:14.055990+0000","last_became_peered":"2024-09-17T04:24:14.055990+0000","last_unstale":"2024-09-17T04:24:16.056099+0000","last_undegraded":"2024-09-17T04:24:16.056099+0000","last_fullsized":"2024-09-17T04:24:16.056099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:51.429181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223887+0000","last_change":"2024-09-17T04:24:15.076359+0000","last_active":"2024-09-17T04:24:16.223887+0000","last_peered":"2024-09-17T04:24:16.223887+0000","last_clean":"2024-09-17T04:24:16.223887+0000","last_became_active":"2024-09-17T04:24:15.075235+0000","last_became_peered":"2024-09-17T04:24:15.075235+0000","last_unstale":"2024-09-17T04:24:16.223887+0000","last_undegraded":"2024-09-17T04:24:16.223887+0000","last_fullsized":"2024-09-17T04:24:16.223887+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:44.355705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224496+0000","last_change":"2024-09-17T04:24:14.051949+0000","last_active":"2024-09-17T04:24:16.224496+0000","last_peered":"2024-09-17T04:24:16.224496+0000","last_clean":"2024-09-17T04:24:16.224496+0000","last_became_active":"2024-09-17T04:24:14.051548+0000","last_became_peered":"2024-09-17T04:24:14.051548+0000","last_unstale":"2024-09-17T04:24:16.224496+0000","last_undegraded":"2024-09-17T04:24:16.224496+0000","last_fullsized":"2024-09-17T04:24:16.224496+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:21.535057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223899+0000","last_change":"2024-09-17T04:24:15.060676+0000","last_active":"2024-09-17T04:24:16.223899+0000","last_peered":"2024-09-17T04:24:16.223899+0000","last_clean":"2024-09-17T04:24:16.223899+0000","last_became_active":"2024-09-17T04:24:15.060268+0000","last_became_peered":"2024-09-17T04:24:15.060268+0000","last_unstale":"2024-09-17T04:24:16.223899+0000","last_undegraded":"2024-09-17T04:24:16.223899+0000","last_fullsized":"2024-09-17T04:24:16.223899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:13.752351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.051789+0000","last_change":"2024-09-17T04:24:14.043539+0000","last_active":"2024-09-17T04:24:16.051789+0000","last_peered":"2024-09-17T04:24:16.051789+0000","last_clean":"2024-09-17T04:24:16.051789+0000","last_became_active":"2024-09-17T04:24:14.043214+0000","last_became_peered":"2024-09-17T04:24:14.043214+0000","last_unstale":"2024-09-17T04:24:16.051789+0000","last_undegraded":"2024-09-17T04:24:16.051789+0000","last_fullsized":"2024-09-17T04:24:16.051789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:42.267414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223990+0000","last_change":"2024-09-17T04:24:15.068886+0000","last_active":"2024-09-17T04:24:16.223990+0000","last_peered":"2024-09-17T04:24:16.223990+0000","last_clean":"2024-09-17T04:24:16.223990+0000","last_became_active":"2024-09-17T04:24:15.068532+0000","last_became_peered":"2024-09-17T04:24:15.068532+0000","last_unstale":"2024-09-17T04:24:16.223990+0000","last_undegraded":"2024-09-17T04:24:16.223990+0000","last_fullsized":"2024-09-17T04:24:16.223990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:58.719218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054761+0000","last_change":"2024-09-17T04:24:14.059673+0000","last_active":"2024-09-17T04:24:16.054761+0000","last_peered":"2024-09-17T04:24:16.054761+0000","last_clean":"2024-09-17T04:24:16.054761+0000","last_became_active":"2024-09-17T04:24:14.057668+0000","last_became_peered":"2024-09-17T04:24:14.057668+0000","last_unstale":"2024-09-17T04:24:16.054761+0000","last_undegraded":"2024-09-17T04:24:16.054761+0000","last_fullsized":"2024-09-17T04:24:16.054761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:01.211818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224143+0000","last_change":"2024-09-17T04:24:15.069931+0000","last_active":"2024-09-17T04:24:16.224143+0000","last_peered":"2024-09-17T04:24:16.224143+0000","last_clean":"2024-09-17T04:24:16.224143+0000","last_became_active":"2024-09-17T04:24:15.069568+0000","last_became_peered":"2024-09-17T04:24:15.069568+0000","last_unstale":"2024-09-17T04:24:16.224143+0000","last_undegraded":"2024-09-17T04:24:16.224143+0000","last_fullsized":"2024-09-17T04:24:16.224143+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:35.584338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.052093+0000","last_change":"2024-09-17T04:24:14.054912+0000","last_active":"2024-09-17T04:24:16.052093+0000","last_peered":"2024-09-17T04:24:16.052093+0000","last_clean":"2024-09-17T04:24:16.052093+0000","last_became_active":"2024-09-17T04:24:14.054105+0000","last_became_peered":"2024-09-17T04:24:14.054105+0000","last_unstale":"2024-09-17T04:24:16.052093+0000","last_undegraded":"2024-09-17T04:24:16.052093+0000","last_fullsized":"2024-09-17T04:24:16.052093+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:44.734015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224717+0000","last_change":"2024-09-17T04:24:15.075676+0000","last_active":"2024-09-17T04:24:16.224717+0000","last_peered":"2024-09-17T04:24:16.224717+0000","last_clean":"2024-09-17T04:24:16.224717+0000","last_became_active":"2024-09-17T04:24:15.075114+0000","last_became_peered":"2024-09-17T04:24:15.075114+0000","last_unstale":"2024-09-17T04:24:16.224717+0000","last_undegraded":"2024-09-17T04:24:16.224717+0000","last_fullsized":"2024-09-17T04:24:16.224717+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:51:30.987569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223146+0000","last_change":"2024-09-17T04:24:14.039417+0000","last_active":"2024-09-17T04:24:16.223146+0000","last_peered":"2024-09-17T04:24:16.223146+0000","last_clean":"2024-09-17T04:24:16.223146+0000","last_became_active":"2024-09-17T04:24:14.039025+0000","last_became_peered":"2024-09-17T04:24:14.039025+0000","last_unstale":"2024-09-17T04:24:16.223146+0000","last_undegraded":"2024-09-17T04:24:16.223146+0000","last_fullsized":"2024-09-17T04:24:16.223146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:03.320698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056433+0000","last_change":"2024-09-17T04:24:15.061023+0000","last_active":"2024-09-17T04:24:16.056433+0000","last_peered":"2024-09-17T04:24:16.056433+0000","last_clean":"2024-09-17T04:24:16.056433+0000","last_became_active":"2024-09-17T04:24:15.060544+0000","last_became_peered":"2024-09-17T04:24:15.060544+0000","last_unstale":"2024-09-17T04:24:16.056433+0000","last_undegraded":"2024-09-17T04:24:16.056433+0000","last_fullsized":"2024-09-17T04:24:16.056433+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:35:10.792366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.051950+0000","last_change":"2024-09-17T04:24:14.052529+0000","last_active":"2024-09-17T04:24:16.051950+0000","last_peered":"2024-09-17T04:24:16.051950+0000","last_clean":"2024-09-17T04:24:16.051950+0000","last_became_active":"2024-09-17T04:24:14.052135+0000","last_became_peered":"2024-09-17T04:24:14.052135+0000","last_unstale":"2024-09-17T04:24:16.051950+0000","last_undegraded":"2024-09-17T04:24:16.051950+0000","last_fullsized":"2024-09-17T04:24:16.051950+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:28.196011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224039+0000","last_change":"2024-09-17T04:24:15.064435+0000","last_active":"2024-09-17T04:24:16.224039+0000","last_peered":"2024-09-17T04:24:16.224039+0000","last_clean":"2024-09-17T04:24:16.224039+0000","last_became_active":"2024-09-17T04:24:15.064195+0000","last_became_peered":"2024-09-17T04:24:15.064195+0000","last_unstale":"2024-09-17T04:24:16.224039+0000","last_undegraded":"2024-09-17T04:24:16.224039+0000","last_fullsized":"2024-09-17T04:24:16.224039+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:47.959956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223547+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:24:16.223547+0000","last_peered":"2024-09-17T04:24:16.223547+0000","last_clean":"2024-09-17T04:24:16.223547+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:24:16.223547+0000","last_undegraded":"2024-09-17T04:24:16.223547+0000","last_fullsized":"2024-09-17T04:24:16.223547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.055535+0000","last_change":"2024-09-17T04:24:14.049088+0000","last_active":"2024-09-17T04:24:16.055535+0000","last_peered":"2024-09-17T04:24:16.055535+0000","last_clean":"2024-09-17T04:24:16.055535+0000","last_became_active":"2024-09-17T04:24:14.048529+0000","last_became_peered":"2024-09-17T04:24:14.048529+0000","last_unstale":"2024-09-17T04:24:16.055535+0000","last_undegraded":"2024-09-17T04:24:16.055535+0000","last_fullsized":"2024-09-17T04:24:16.055535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:42.267414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225083+0000","last_change":"2024-09-17T04:24:15.065699+0000","last_active":"2024-09-17T04:24:16.225083+0000","last_peered":"2024-09-17T04:24:16.225083+0000","last_clean":"2024-09-17T04:24:16.225083+0000","last_became_active":"2024-09-17T04:24:15.061569+0000","last_became_peered":"2024-09-17T04:24:15.061569+0000","last_unstale":"2024-09-17T04:24:16.225083+0000","last_undegraded":"2024-09-17T04:24:16.225083+0000","last_fullsized":"2024-09-17T04:24:16.225083+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:01:19.496265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223384+0000","last_change":"2024-09-17T04:24:14.061081+0000","last_active":"2024-09-17T04:24:16.223384+0000","last_peered":"2024-09-17T04:24:16.223384+0000","last_clean":"2024-09-17T04:24:16.223384+0000","last_became_active":"2024-09-17T04:24:14.060394+0000","last_became_peered":"2024-09-17T04:24:14.060394+0000","last_unstale":"2024-09-17T04:24:16.223384+0000","last_undegraded":"2024-09-17T04:24:16.223384+0000","last_fullsized":"2024-09-17T04:24:16.223384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:08.023686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223901+0000","last_change":"2024-09-17T04:24:15.070960+0000","last_active":"2024-09-17T04:24:16.223901+0000","last_peered":"2024-09-17T04:24:16.223901+0000","last_clean":"2024-09-17T04:24:16.223901+0000","last_became_active":"2024-09-17T04:24:15.070528+0000","last_became_peered":"2024-09-17T04:24:15.070528+0000","last_unstale":"2024-09-17T04:24:16.223901+0000","last_undegraded":"2024-09-17T04:24:16.223901+0000","last_fullsized":"2024-09-17T04:24:16.223901+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:22.452641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.052102+0000","last_change":"2024-09-17T04:24:14.051042+0000","last_active":"2024-09-17T04:24:16.052102+0000","last_peered":"2024-09-17T04:24:16.052102+0000","last_clean":"2024-09-17T04:24:16.052102+0000","last_became_active":"2024-09-17T04:24:14.050513+0000","last_became_peered":"2024-09-17T04:24:14.050513+0000","last_unstale":"2024-09-17T04:24:16.052102+0000","last_undegraded":"2024-09-17T04:24:16.052102+0000","last_fullsized":"2024-09-17T04:24:16.052102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:21.535057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.222913+0000","last_change":"2024-09-17T04:24:15.061656+0000","last_active":"2024-09-17T04:24:16.222913+0000","last_peered":"2024-09-17T04:24:16.222913+0000","last_clean":"2024-09-17T04:24:16.222913+0000","last_became_active":"2024-09-17T04:24:15.061365+0000","last_became_peered":"2024-09-17T04:24:15.061365+0000","last_unstale":"2024-09-17T04:24:16.222913+0000","last_undegraded":"2024-09-17T04:24:16.222913+0000","last_fullsized":"2024-09-17T04:24:16.222913+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:22.452641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056245+0000","last_change":"2024-09-17T04:24:14.053751+0000","last_active":"2024-09-17T04:24:16.056245+0000","last_peered":"2024-09-17T04:24:16.056245+0000","last_clean":"2024-09-17T04:24:16.056245+0000","last_became_active":"2024-09-17T04:24:14.051943+0000","last_became_peered":"2024-09-17T04:24:14.051943+0000","last_unstale":"2024-09-17T04:24:16.056245+0000","last_undegraded":"2024-09-17T04:24:16.056245+0000","last_fullsized":"2024-09-17T04:24:16.056245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:08.023686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054263+0000","last_change":"2024-09-17T04:24:15.074736+0000","last_active":"2024-09-17T04:24:16.054263+0000","last_peered":"2024-09-17T04:24:16.054263+0000","last_clean":"2024-09-17T04:24:16.054263+0000","last_became_active":"2024-09-17T04:24:15.074374+0000","last_became_peered":"2024-09-17T04:24:15.074374+0000","last_unstale":"2024-09-17T04:24:16.054263+0000","last_undegraded":"2024-09-17T04:24:16.054263+0000","last_fullsized":"2024-09-17T04:24:16.054263+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:49.409374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.222921+0000","last_change":"2024-09-17T04:24:14.057105+0000","last_active":"2024-09-17T04:24:16.222921+0000","last_peered":"2024-09-17T04:24:16.222921+0000","last_clean":"2024-09-17T04:24:16.222921+0000","last_became_active":"2024-09-17T04:24:14.056908+0000","last_became_peered":"2024-09-17T04:24:14.056908+0000","last_unstale":"2024-09-17T04:24:16.222921+0000","last_undegraded":"2024-09-17T04:24:16.222921+0000","last_fullsized":"2024-09-17T04:24:16.222921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:12.782267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056301+0000","last_change":"2024-09-17T04:24:15.058663+0000","last_active":"2024-09-17T04:24:16.056301+0000","last_peered":"2024-09-17T04:24:16.056301+0000","last_clean":"2024-09-17T04:24:16.056301+0000","last_became_active":"2024-09-17T04:24:15.057807+0000","last_became_peered":"2024-09-17T04:24:15.057807+0000","last_unstale":"2024-09-17T04:24:16.056301+0000","last_undegraded":"2024-09-17T04:24:16.056301+0000","last_fullsized":"2024-09-17T04:24:16.056301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:35.584338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224488+0000","last_change":"2024-09-17T04:24:14.063354+0000","last_active":"2024-09-17T04:24:16.224488+0000","last_peered":"2024-09-17T04:24:16.224488+0000","last_clean":"2024-09-17T04:24:16.224488+0000","last_became_active":"2024-09-17T04:24:14.062231+0000","last_became_peered":"2024-09-17T04:24:14.062231+0000","last_unstale":"2024-09-17T04:24:16.224488+0000","last_undegraded":"2024-09-17T04:24:16.224488+0000","last_fullsized":"2024-09-17T04:24:16.224488+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:40.777526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.222850+0000","last_change":"2024-09-17T04:24:15.058346+0000","last_active":"2024-09-17T04:24:16.222850+0000","last_peered":"2024-09-17T04:24:16.222850+0000","last_clean":"2024-09-17T04:24:16.222850+0000","last_became_active":"2024-09-17T04:24:15.057977+0000","last_became_peered":"2024-09-17T04:24:15.057977+0000","last_unstale":"2024-09-17T04:24:16.222850+0000","last_undegraded":"2024-09-17T04:24:16.222850+0000","last_fullsized":"2024-09-17T04:24:16.222850+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:01.706191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223545+0000","last_change":"2024-09-17T04:24:14.049298+0000","last_active":"2024-09-17T04:24:16.223545+0000","last_peered":"2024-09-17T04:24:16.223545+0000","last_clean":"2024-09-17T04:24:16.223545+0000","last_became_active":"2024-09-17T04:24:14.049070+0000","last_became_peered":"2024-09-17T04:24:14.049070+0000","last_unstale":"2024-09-17T04:24:16.223545+0000","last_undegraded":"2024-09-17T04:24:16.223545+0000","last_fullsized":"2024-09-17T04:24:16.223545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:52:59.361386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225582+0000","last_change":"2024-09-17T04:24:15.050480+0000","last_active":"2024-09-17T04:24:16.225582+0000","last_peered":"2024-09-17T04:24:16.225582+0000","last_clean":"2024-09-17T04:24:16.225582+0000","last_became_active":"2024-09-17T04:24:15.050166+0000","last_became_peered":"2024-09-17T04:24:15.050166+0000","last_unstale":"2024-09-17T04:24:16.225582+0000","last_undegraded":"2024-09-17T04:24:16.225582+0000","last_fullsized":"2024-09-17T04:24:16.225582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:02.125884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225264+0000","last_change":"2024-09-17T04:24:14.063222+0000","last_active":"2024-09-17T04:24:16.225264+0000","last_peered":"2024-09-17T04:24:16.225264+0000","last_clean":"2024-09-17T04:24:16.225264+0000","last_became_active":"2024-09-17T04:24:14.062750+0000","last_became_peered":"2024-09-17T04:24:14.062750+0000","last_unstale":"2024-09-17T04:24:16.225264+0000","last_undegraded":"2024-09-17T04:24:16.225264+0000","last_fullsized":"2024-09-17T04:24:16.225264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:08.023686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224491+0000","last_change":"2024-09-17T04:24:15.067950+0000","last_active":"2024-09-17T04:24:16.224491+0000","last_peered":"2024-09-17T04:24:16.224491+0000","last_clean":"2024-09-17T04:24:16.224491+0000","last_became_active":"2024-09-17T04:24:15.066567+0000","last_became_peered":"2024-09-17T04:24:15.066567+0000","last_unstale":"2024-09-17T04:24:16.224491+0000","last_undegraded":"2024-09-17T04:24:16.224491+0000","last_fullsized":"2024-09-17T04:24:16.224491+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:28.104736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223381+0000","last_change":"2024-09-17T04:24:14.059487+0000","last_active":"2024-09-17T04:24:16.223381+0000","last_peered":"2024-09-17T04:24:16.223381+0000","last_clean":"2024-09-17T04:24:16.223381+0000","last_became_active":"2024-09-17T04:24:14.059186+0000","last_became_peered":"2024-09-17T04:24:14.059186+0000","last_unstale":"2024-09-17T04:24:16.223381+0000","last_undegraded":"2024-09-17T04:24:16.223381+0000","last_fullsized":"2024-09-17T04:24:16.223381+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:03.320698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.054147+0000","last_change":"2024-09-17T04:24:15.065941+0000","last_active":"2024-09-17T04:24:16.054147+0000","last_peered":"2024-09-17T04:24:16.054147+0000","last_clean":"2024-09-17T04:24:16.054147+0000","last_became_active":"2024-09-17T04:24:15.065582+0000","last_became_peered":"2024-09-17T04:24:15.065582+0000","last_unstale":"2024-09-17T04:24:16.054147+0000","last_undegraded":"2024-09-17T04:24:16.054147+0000","last_fullsized":"2024-09-17T04:24:16.054147+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:30.122754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.222894+0000","last_change":"2024-09-17T04:24:14.062678+0000","last_active":"2024-09-17T04:24:16.222894+0000","last_peered":"2024-09-17T04:24:16.222894+0000","last_clean":"2024-09-17T04:24:16.222894+0000","last_became_active":"2024-09-17T04:24:14.062281+0000","last_became_peered":"2024-09-17T04:24:14.062281+0000","last_unstale":"2024-09-17T04:24:16.222894+0000","last_undegraded":"2024-09-17T04:24:16.222894+0000","last_fullsized":"2024-09-17T04:24:16.222894+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:40.254206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224778+0000","last_change":"2024-09-17T04:24:15.064498+0000","last_active":"2024-09-17T04:24:16.224778+0000","last_peered":"2024-09-17T04:24:16.224778+0000","last_clean":"2024-09-17T04:24:16.224778+0000","last_became_active":"2024-09-17T04:24:15.062700+0000","last_became_peered":"2024-09-17T04:24:15.062700+0000","last_unstale":"2024-09-17T04:24:16.224778+0000","last_undegraded":"2024-09-17T04:24:16.224778+0000","last_fullsized":"2024-09-17T04:24:16.224778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:31.021313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223265+0000","last_change":"2024-09-17T04:24:14.047923+0000","last_active":"2024-09-17T04:24:16.223265+0000","last_peered":"2024-09-17T04:24:16.223265+0000","last_clean":"2024-09-17T04:24:16.223265+0000","last_became_active":"2024-09-17T04:24:14.047596+0000","last_became_peered":"2024-09-17T04:24:14.047596+0000","last_unstale":"2024-09-17T04:24:16.223265+0000","last_undegraded":"2024-09-17T04:24:16.223265+0000","last_fullsized":"2024-09-17T04:24:16.223265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:29.119993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225575+0000","last_change":"2024-09-17T04:24:15.060939+0000","last_active":"2024-09-17T04:24:16.225575+0000","last_peered":"2024-09-17T04:24:16.225575+0000","last_clean":"2024-09-17T04:24:16.225575+0000","last_became_active":"2024-09-17T04:24:15.060600+0000","last_became_peered":"2024-09-17T04:24:15.060600+0000","last_unstale":"2024-09-17T04:24:16.225575+0000","last_undegraded":"2024-09-17T04:24:16.225575+0000","last_fullsized":"2024-09-17T04:24:16.225575+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:05.895665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224176+0000","last_change":"2024-09-17T04:24:14.062616+0000","last_active":"2024-09-17T04:24:16.224176+0000","last_peered":"2024-09-17T04:24:16.224176+0000","last_clean":"2024-09-17T04:24:16.224176+0000","last_became_active":"2024-09-17T04:24:14.062182+0000","last_became_peered":"2024-09-17T04:24:14.062182+0000","last_unstale":"2024-09-17T04:24:16.224176+0000","last_undegraded":"2024-09-17T04:24:16.224176+0000","last_fullsized":"2024-09-17T04:24:16.224176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:51.246396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224637+0000","last_change":"2024-09-17T04:24:15.079589+0000","last_active":"2024-09-17T04:24:16.224637+0000","last_peered":"2024-09-17T04:24:16.224637+0000","last_clean":"2024-09-17T04:24:16.224637+0000","last_became_active":"2024-09-17T04:24:15.079019+0000","last_became_peered":"2024-09-17T04:24:15.079019+0000","last_unstale":"2024-09-17T04:24:16.224637+0000","last_undegraded":"2024-09-17T04:24:16.224637+0000","last_fullsized":"2024-09-17T04:24:16.224637+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:01.706191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056070+0000","last_change":"2024-09-17T04:24:15.080533+0000","last_active":"2024-09-17T04:24:16.056070+0000","last_peered":"2024-09-17T04:24:16.056070+0000","last_clean":"2024-09-17T04:24:16.056070+0000","last_became_active":"2024-09-17T04:24:15.080179+0000","last_became_peered":"2024-09-17T04:24:15.080179+0000","last_unstale":"2024-09-17T04:24:16.056070+0000","last_undegraded":"2024-09-17T04:24:16.056070+0000","last_fullsized":"2024-09-17T04:24:16.056070+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:58.719218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224154+0000","last_change":"2024-09-17T04:24:14.065850+0000","last_active":"2024-09-17T04:24:16.224154+0000","last_peered":"2024-09-17T04:24:16.224154+0000","last_clean":"2024-09-17T04:24:16.224154+0000","last_became_active":"2024-09-17T04:24:14.065444+0000","last_became_peered":"2024-09-17T04:24:14.065444+0000","last_unstale":"2024-09-17T04:24:16.224154+0000","last_undegraded":"2024-09-17T04:24:16.224154+0000","last_fullsized":"2024-09-17T04:24:16.224154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:51.121630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.056174+0000","last_change":"2024-09-17T04:24:15.076511+0000","last_active":"2024-09-17T04:24:16.056174+0000","last_peered":"2024-09-17T04:24:16.056174+0000","last_clean":"2024-09-17T04:24:16.056174+0000","last_became_active":"2024-09-17T04:24:15.076037+0000","last_became_peered":"2024-09-17T04:24:15.076037+0000","last_unstale":"2024-09-17T04:24:16.056174+0000","last_undegraded":"2024-09-17T04:24:16.056174+0000","last_fullsized":"2024-09-17T04:24:16.056174+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:06:32.518189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.224180+0000","last_change":"2024-09-17T04:24:14.060131+0000","last_active":"2024-09-17T04:24:16.224180+0000","last_peered":"2024-09-17T04:24:16.224180+0000","last_clean":"2024-09-17T04:24:16.224180+0000","last_became_active":"2024-09-17T04:24:14.059707+0000","last_became_peered":"2024-09-17T04:24:14.059707+0000","last_unstale":"2024-09-17T04:24:16.224180+0000","last_undegraded":"2024-09-17T04:24:16.224180+0000","last_fullsized":"2024-09-17T04:24:16.224180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:22.869429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223586+0000","last_change":"2024-09-17T04:24:15.068641+0000","last_active":"2024-09-17T04:24:16.223586+0000","last_peered":"2024-09-17T04:24:16.223586+0000","last_clean":"2024-09-17T04:24:16.223586+0000","last_became_active":"2024-09-17T04:24:15.065609+0000","last_became_peered":"2024-09-17T04:24:15.065609+0000","last_unstale":"2024-09-17T04:24:16.223586+0000","last_undegraded":"2024-09-17T04:24:16.223586+0000","last_fullsized":"2024-09-17T04:24:16.223586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:43:02.004482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225369+0000","last_change":"2024-09-17T04:24:14.056735+0000","last_active":"2024-09-17T04:24:16.225369+0000","last_peered":"2024-09-17T04:24:16.225369+0000","last_clean":"2024-09-17T04:24:16.225369+0000","last_became_active":"2024-09-17T04:24:14.056433+0000","last_became_peered":"2024-09-17T04:24:14.056433+0000","last_unstale":"2024-09-17T04:24:16.225369+0000","last_undegraded":"2024-09-17T04:24:16.225369+0000","last_fullsized":"2024-09-17T04:24:16.225369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:03.618384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.225403+0000","last_change":"2024-09-17T04:24:15.054773+0000","last_active":"2024-09-17T04:24:16.225403+0000","last_peered":"2024-09-17T04:24:16.225403+0000","last_clean":"2024-09-17T04:24:16.225403+0000","last_became_active":"2024-09-17T04:24:15.054453+0000","last_became_peered":"2024-09-17T04:24:15.054453+0000","last_unstale":"2024-09-17T04:24:16.225403+0000","last_undegraded":"2024-09-17T04:24:16.225403+0000","last_fullsized":"2024-09-17T04:24:16.225403+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:14.018386+0000","last_clean_scrub_stamp":"2024-09-17T04:24:14.018386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:58.942093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T04:24:16.223981+0000","last_change":"2024-09-17T04:24:14.062995+0000","last_active":"2024-09-17T04:24:16.223981+0000","last_peered":"2024-09-17T04:24:16.223981+0000","last_clean":"2024-09-17T04:24:16.223981+0000","last_became_active":"2024-09-17T04:24:14.062615+0000","last_became_peered":"2024-09-17T04:24:14.062615+0000","last_unstale":"2024-09-17T04:24:16.223981+0000","last_undegraded":"2024-09-17T04:24:16.223981+0000","last_fullsized":"2024-09-17T04:24:16.223981+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:13.015626+0000","last_clean_scrub_stamp":"2024-09-17T04:24:13.015626+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:20:07.263105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476746,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27844,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343996,"statfs":{"total":96636764160,"available":96608251904,"internally_reserved":0,"allocated":827392,"data_stored":670785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476745,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":81447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476746,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476746,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":79961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":79835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":237568,"data_stored":79995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":835584,"data_stored":670841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542153,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":81447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:24:30.501 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T04:24:30.502 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-17T04:24:30.502 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-17T04:24:30.502 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T04:24:30.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.756+0000 7fdd91400640 1 -- 172.21.15.100:0/3375828701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd8c165480 msgr2=0x7fdd8c1678a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:30.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.756+0000 7fdd91400640 1 --2- 172.21.15.100:0/3375828701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd8c165480 0x7fdd8c1678a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdd84005fd0 tx=0x7fdd8402aaa0 comp rx=0 tx=0).stop 2024-09-17T04:24:30.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.756+0000 7fdd91400640 1 -- 172.21.15.100:0/3375828701 shutdown_connections 2024-09-17T04:24:30.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.756+0000 7fdd91400640 1 --2- 172.21.15.100:0/3375828701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd8c165480 0x7fdd8c1678a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:30.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.756+0000 7fdd91400640 1 -- 172.21.15.100:0/3375828701 >> 172.21.15.100:0/3375828701 conn(0x7fdd8c06d730 msgr2=0x7fdd8c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:30.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.757+0000 7fdd91400640 1 -- 172.21.15.100:0/3375828701 shutdown_connections 2024-09-17T04:24:30.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.757+0000 7fdd91400640 1 -- 172.21.15.100:0/3375828701 wait complete. 2024-09-17T04:24:30.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.759+0000 7fdd91400640 1 Processor -- start 2024-09-17T04:24:30.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.759+0000 7fdd91400640 1 -- start start 2024-09-17T04:24:30.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.760+0000 7fdd91400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd8c0ad0e0 0x7fdd8c1f4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:30.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.760+0000 7fdd91400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd8c15d760 0x7fdd8c148450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:30.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.760+0000 7fdd91400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd8c165480 0x7fdd8c0b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:30.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.760+0000 7fdd91400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8c167de0 con 0x7fdd8c0ad0e0 2024-09-17T04:24:30.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.760+0000 7fdd91400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8c0b4e30 con 0x7fdd8c165480 2024-09-17T04:24:30.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.760+0000 7fdd91400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdd8c167f60 con 0x7fdd8c15d760 2024-09-17T04:24:30.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.760+0000 7fdd8a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd8c0ad0e0 0x7fdd8c1f4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:30.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.761+0000 7fdd8a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd8c0ad0e0 0x7fdd8c1f4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48270/0 (socket says 172.21.15.100:48270) 2024-09-17T04:24:30.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.761+0000 7fdd8a800640 1 -- 172.21.15.100:0/10969856 learned_addr learned my addr 172.21.15.100:0/10969856 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:30.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.761+0000 7fdd83e00640 1 --2- 172.21.15.100:0/10969856 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd8c15d760 0x7fdd8c148450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:30.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.761+0000 7fdd8b200640 1 --2- 172.21.15.100:0/10969856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd8c165480 0x7fdd8c0b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:30.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.761+0000 7fdd8a800640 1 -- 172.21.15.100:0/10969856 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd8c15d760 msgr2=0x7fdd8c148450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:30.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.761+0000 7fdd8a800640 1 --2- 172.21.15.100:0/10969856 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd8c15d760 0x7fdd8c148450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:30.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.761+0000 7fdd8a800640 1 -- 172.21.15.100:0/10969856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd8c165480 msgr2=0x7fdd8c0b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:30.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.761+0000 7fdd8a800640 1 --2- 172.21.15.100:0/10969856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd8c165480 0x7fdd8c0b0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:30.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.761+0000 7fdd8a800640 1 -- 172.21.15.100:0/10969856 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd8c161200 con 0x7fdd8c0ad0e0 2024-09-17T04:24:30.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.761+0000 7fdd8a800640 1 --2- 172.21.15.100:0/10969856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd8c0ad0e0 0x7fdd8c1f4350 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00c370 tx=0x7fdd7c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:30.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.762+0000 7fdd82a00640 1 -- 172.21.15.100:0/10969856 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c017070 con 0x7fdd8c0ad0e0 2024-09-17T04:24:30.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.762+0000 7fdd82a00640 1 -- 172.21.15.100:0/10969856 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd7c0035f0 con 0x7fdd8c0ad0e0 2024-09-17T04:24:30.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.762+0000 7fdd91400640 1 -- 172.21.15.100:0/10969856 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd8c160c60 con 0x7fdd8c0ad0e0 2024-09-17T04:24:30.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.762+0000 7fdd82a00640 1 -- 172.21.15.100:0/10969856 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c003a40 con 0x7fdd8c0ad0e0 2024-09-17T04:24:30.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.762+0000 7fdd91400640 1 -- 172.21.15.100:0/10969856 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd8c1574f0 con 0x7fdd8c0ad0e0 2024-09-17T04:24:30.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.763+0000 7fdd91400640 1 -- 172.21.15.100:0/10969856 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd8c1535c0 con 0x7fdd8c0ad0e0 2024-09-17T04:24:30.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.764+0000 7fdd82a00640 1 -- 172.21.15.100:0/10969856 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdd7c01a070 con 0x7fdd8c0ad0e0 2024-09-17T04:24:30.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.765+0000 7fdd82a00640 1 --2- 172.21.15.100:0/10969856 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdd68078b90 0x7fdd6807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:30.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.765+0000 7fdd82a00640 1 -- 172.21.15.100:0/10969856 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fdd7c09a060 con 0x7fdd8c0ad0e0 2024-09-17T04:24:30.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.766+0000 7fdd83e00640 1 --2- 172.21.15.100:0/10969856 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdd68078b90 0x7fdd6807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:30.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.766+0000 7fdd83e00640 1 --2- 172.21.15.100:0/10969856 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdd68078b90 0x7fdd6807b0a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c1570c0 tx=0x7fdd74009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:30.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.769+0000 7fdd82a00640 1 -- 172.21.15.100:0/10969856 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fdd7c061db0 con 0x7fdd8c0ad0e0 2024-09-17T04:24:30.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.994+0000 7fdd91400640 1 -- 172.21.15.100:0/10969856 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fdd8c06f4c0 con 0x7fdd8c0ad0e0 2024-09-17T04:24:30.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.994+0000 7fdd82a00640 1 -- 172.21.15.100:0/10969856 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fdd7c0667e0 con 0x7fdd8c0ad0e0 2024-09-17T04:24:30.994 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:30.995 INFO:teuthology.orchestra.run.smithi100.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T04:24:30.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.996+0000 7fdd91400640 1 -- 172.21.15.100:0/10969856 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdd68078b90 msgr2=0x7fdd6807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:30.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.996+0000 7fdd91400640 1 --2- 172.21.15.100:0/10969856 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdd68078b90 0x7fdd6807b0a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c1570c0 tx=0x7fdd74009040 comp rx=0 tx=0).stop 2024-09-17T04:24:30.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.996+0000 7fdd91400640 1 -- 172.21.15.100:0/10969856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd8c0ad0e0 msgr2=0x7fdd8c1f4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:30.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.996+0000 7fdd91400640 1 --2- 172.21.15.100:0/10969856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd8c0ad0e0 0x7fdd8c1f4350 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00c370 tx=0x7fdd7c00c830 comp rx=0 tx=0).stop 2024-09-17T04:24:30.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.996+0000 7fdd91400640 1 -- 172.21.15.100:0/10969856 shutdown_connections 2024-09-17T04:24:30.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.996+0000 7fdd91400640 1 --2- 172.21.15.100:0/10969856 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdd68078b90 0x7fdd6807b0a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:30.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.996+0000 7fdd91400640 1 --2- 172.21.15.100:0/10969856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd8c165480 0x7fdd8c0b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:30.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.996+0000 7fdd91400640 1 --2- 172.21.15.100:0/10969856 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd8c15d760 0x7fdd8c148450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:30.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.996+0000 7fdd91400640 1 --2- 172.21.15.100:0/10969856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd8c0ad0e0 0x7fdd8c1f4350 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:30.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.996+0000 7fdd91400640 1 -- 172.21.15.100:0/10969856 >> 172.21.15.100:0/10969856 conn(0x7fdd8c06d730 msgr2=0x7fdd8c06fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:30.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.996+0000 7fdd91400640 1 -- 172.21.15.100:0/10969856 shutdown_connections 2024-09-17T04:24:30.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:30.996+0000 7fdd91400640 1 -- 172.21.15.100:0/10969856 wait complete. 2024-09-17T04:24:31.017 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-17T04:24:31.018 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:24:31.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.239+0000 7f10e4000640 1 -- 172.21.15.100:0/2904761901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10dc0b3550 msgr2=0x7f10dc0b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:31.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.239+0000 7f10e4000640 1 --2- 172.21.15.100:0/2904761901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10dc0b3550 0x7f10dc0b3960 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f10d4005fd0 tx=0x7f10d402a820 comp rx=0 tx=0).stop 2024-09-17T04:24:31.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.239+0000 7f10e4000640 1 -- 172.21.15.100:0/2904761901 shutdown_connections 2024-09-17T04:24:31.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.239+0000 7f10e4000640 1 --2- 172.21.15.100:0/2904761901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10dc0b3550 0x7f10dc0b3960 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:31.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.239+0000 7f10e4000640 1 --2- 172.21.15.100:0/2904761901 >> v2:172.21.15.184:3300/0 conn(0x7f10dc15d760 0x7f10dc1535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:31.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.239+0000 7f10e4000640 1 -- 172.21.15.100:0/2904761901 >> 172.21.15.100:0/2904761901 conn(0x7f10dc06d730 msgr2=0x7f10dc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:31.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.239+0000 7f10e4000640 1 -- 172.21.15.100:0/2904761901 shutdown_connections 2024-09-17T04:24:31.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.239+0000 7f10e4000640 1 -- 172.21.15.100:0/2904761901 wait complete. 2024-09-17T04:24:31.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.241+0000 7f10e4000640 1 Processor -- start 2024-09-17T04:24:31.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.241+0000 7f10e4000640 1 -- start start 2024-09-17T04:24:31.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.243+0000 7f10e4000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10dc0ad0e0 0x7f10dc1f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:31.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.243+0000 7f10e4000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10dc0b3550 0x7f10dc14e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:31.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.243+0000 7f10e4000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10dc15d760 0x7f10dc14f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:31.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.243+0000 7f10e4000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc06f240 con 0x7f10dc0ad0e0 2024-09-17T04:24:31.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.243+0000 7f10e4000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc0b4e30 con 0x7f10dc15d760 2024-09-17T04:24:31.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.243+0000 7f10e4000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f10dc0b3ea0 con 0x7f10dc0b3550 2024-09-17T04:24:31.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.243+0000 7f10e1600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10dc0ad0e0 0x7f10dc1f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:31.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.243+0000 7f10e1600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10dc0ad0e0 0x7f10dc1f5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48274/0 (socket says 172.21.15.100:48274) 2024-09-17T04:24:31.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.243+0000 7f10e1600640 1 -- 172.21.15.100:0/3899239651 learned_addr learned my addr 172.21.15.100:0/3899239651 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:31.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.243+0000 7f10e0c00640 1 --2- 172.21.15.100:0/3899239651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10dc0b3550 0x7f10dc14e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:31.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.243+0000 7f10e2000640 1 --2- 172.21.15.100:0/3899239651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10dc15d760 0x7f10dc14f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:31.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.243+0000 7f10e1600640 1 -- 172.21.15.100:0/3899239651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10dc0b3550 msgr2=0x7f10dc14e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:31.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.243+0000 7f10e1600640 1 --2- 172.21.15.100:0/3899239651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10dc0b3550 0x7f10dc14e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:31.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.243+0000 7f10e1600640 1 -- 172.21.15.100:0/3899239651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10dc15d760 msgr2=0x7f10dc14f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:31.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.243+0000 7f10e1600640 1 --2- 172.21.15.100:0/3899239651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10dc15d760 0x7f10dc14f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:31.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.243+0000 7f10e1600640 1 -- 172.21.15.100:0/3899239651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10dc1f5fa0 con 0x7f10dc0ad0e0 2024-09-17T04:24:31.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.244+0000 7f10e1600640 1 --2- 172.21.15.100:0/3899239651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10dc0ad0e0 0x7f10dc1f5a60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f10d000e9f0 tx=0x7f10d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:31.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.244+0000 7f10d9600640 1 -- 172.21.15.100:0/3899239651 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d000cd80 con 0x7f10dc0ad0e0 2024-09-17T04:24:31.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.244+0000 7f10d9600640 1 -- 172.21.15.100:0/3899239651 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10d0004020 con 0x7f10dc0ad0e0 2024-09-17T04:24:31.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.244+0000 7f10e4000640 1 -- 172.21.15.100:0/3899239651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10dc1f64b0 con 0x7f10dc0ad0e0 2024-09-17T04:24:31.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.244+0000 7f10d9600640 1 -- 172.21.15.100:0/3899239651 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0004470 con 0x7f10dc0ad0e0 2024-09-17T04:24:31.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.244+0000 7f10e4000640 1 -- 172.21.15.100:0/3899239651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10dc14dd10 con 0x7f10dc0ad0e0 2024-09-17T04:24:31.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.245+0000 7f10e4000640 1 -- 172.21.15.100:0/3899239651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10dc0b4360 con 0x7f10dc0ad0e0 2024-09-17T04:24:31.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.246+0000 7f10d9600640 1 -- 172.21.15.100:0/3899239651 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f10d00035f0 con 0x7f10dc0ad0e0 2024-09-17T04:24:31.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.247+0000 7f10d9600640 1 --2- 172.21.15.100:0/3899239651 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f10bc078b90 0x7f10bc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:31.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.247+0000 7f10d9600640 1 -- 172.21.15.100:0/3899239651 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f10d009b7a0 con 0x7f10dc0ad0e0 2024-09-17T04:24:31.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.248+0000 7f10e0c00640 1 --2- 172.21.15.100:0/3899239651 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f10bc078b90 0x7f10bc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:31.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.249+0000 7f10e0c00640 1 --2- 172.21.15.100:0/3899239651 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f10bc078b90 0x7f10bc07b0a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f10c80046b0 tx=0x7f10c8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:31.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.252+0000 7f10d9600640 1 -- 172.21.15.100:0/3899239651 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f10d0063b50 con 0x7f10dc0ad0e0 2024-09-17T04:24:31.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.467+0000 7f10e4000640 1 -- 172.21.15.100:0/3899239651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10dc0b4980 con 0x7f10dc0ad0e0 2024-09-17T04:24:31.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.468+0000 7f10d9600640 1 -- 172.21.15.100:0/3899239651 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7f10d0063450 con 0x7f10dc0ad0e0 2024-09-17T04:24:31.469 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:31.469 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":8,"btime":"2024-09-17T04:24:18:058994+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4328,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2129046042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":2129046042},{"type":"v1","addr":"172.21.15.100:6835","nonce":2129046042}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4334,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/44255118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":44255118},{"type":"v1","addr":"172.21.15.184:6836","nonce":44255118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/1644264480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1644264480},{"type":"v1","addr":"172.21.15.100:6834","nonce":1644264480}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:24:14.527272+0000","modified":"2024-09-17T04:24:17.543043+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4368},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4368":{"gid":4368,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/2211103925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2211103925},{"type":"v1","addr":"172.21.15.184:6837","nonce":2211103925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4368,"qdb_cluster":[4368]},"id":1}]} 2024-09-17T04:24:31.469 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 8 2024-09-17T04:24:31.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.471+0000 7f10e4000640 1 -- 172.21.15.100:0/3899239651 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f10bc078b90 msgr2=0x7f10bc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:31.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.471+0000 7f10e4000640 1 --2- 172.21.15.100:0/3899239651 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f10bc078b90 0x7f10bc07b0a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f10c80046b0 tx=0x7f10c8009040 comp rx=0 tx=0).stop 2024-09-17T04:24:31.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.471+0000 7f10e4000640 1 -- 172.21.15.100:0/3899239651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10dc0ad0e0 msgr2=0x7f10dc1f5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:31.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.471+0000 7f10e4000640 1 --2- 172.21.15.100:0/3899239651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10dc0ad0e0 0x7f10dc1f5a60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f10d000e9f0 tx=0x7f10d000eeb0 comp rx=0 tx=0).stop 2024-09-17T04:24:31.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.471+0000 7f10e4000640 1 -- 172.21.15.100:0/3899239651 shutdown_connections 2024-09-17T04:24:31.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.471+0000 7f10e4000640 1 --2- 172.21.15.100:0/3899239651 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f10bc078b90 0x7f10bc07b0a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:31.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.471+0000 7f10e4000640 1 --2- 172.21.15.100:0/3899239651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10dc15d760 0x7f10dc14f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:31.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.471+0000 7f10e4000640 1 --2- 172.21.15.100:0/3899239651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10dc0b3550 0x7f10dc14e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:31.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.471+0000 7f10e4000640 1 --2- 172.21.15.100:0/3899239651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10dc0ad0e0 0x7f10dc1f5a60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:31.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.471+0000 7f10e4000640 1 -- 172.21.15.100:0/3899239651 >> 172.21.15.100:0/3899239651 conn(0x7f10dc06d730 msgr2=0x7f10dc15eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:31.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.471+0000 7f10e4000640 1 -- 172.21.15.100:0/3899239651 shutdown_connections 2024-09-17T04:24:31.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.471+0000 7f10e4000640 1 -- 172.21.15.100:0/3899239651 wait complete. 2024-09-17T04:24:31.494 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:24:31.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.712+0000 7f0f5c600640 1 -- 172.21.15.100:0/1890154762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f540ad0e0 msgr2=0x7f0f540ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:31.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.712+0000 7f0f5c600640 1 --2- 172.21.15.100:0/1890154762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f540ad0e0 0x7f0f540ad4f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009830 tx=0x7f0f4002a530 comp rx=0 tx=0).stop 2024-09-17T04:24:31.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.712+0000 7f0f5c600640 1 -- 172.21.15.100:0/1890154762 shutdown_connections 2024-09-17T04:24:31.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.712+0000 7f0f5c600640 1 --2- 172.21.15.100:0/1890154762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f540ad0e0 0x7f0f540ad4f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:31.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.712+0000 7f0f5c600640 1 -- 172.21.15.100:0/1890154762 >> 172.21.15.100:0/1890154762 conn(0x7f0f5406d730 msgr2=0x7f0f540b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:31.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.712+0000 7f0f5c600640 1 -- 172.21.15.100:0/1890154762 shutdown_connections 2024-09-17T04:24:31.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.712+0000 7f0f5c600640 1 -- 172.21.15.100:0/1890154762 wait complete. 2024-09-17T04:24:31.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.714+0000 7f0f5c600640 1 Processor -- start 2024-09-17T04:24:31.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.714+0000 7f0f5c600640 1 -- start start 2024-09-17T04:24:31.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.716+0000 7f0f5c600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f540ad0e0 0x7f0f54074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:31.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.716+0000 7f0f5c600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f540b3550 0x7f0f541f58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:31.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.716+0000 7f0f5c600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5415d760 0x7f0f54156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:31.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.716+0000 7f0f5c600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0f540b4da0 con 0x7f0f5415d760 2024-09-17T04:24:31.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.716+0000 7f0f5c600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0f540b4c20 con 0x7f0f540ad0e0 2024-09-17T04:24:31.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.716+0000 7f0f5c600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0f540b4070 con 0x7f0f540b3550 2024-09-17T04:24:31.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.716+0000 7f0f5a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5415d760 0x7f0f54156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:31.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.716+0000 7f0f5a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5415d760 0x7f0f54156170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48286/0 (socket says 172.21.15.100:48286) 2024-09-17T04:24:31.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.716+0000 7f0f5a600640 1 -- 172.21.15.100:0/1364925113 learned_addr learned my addr 172.21.15.100:0/1364925113 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:31.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.716+0000 7f0f59c00640 1 --2- 172.21.15.100:0/1364925113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f540ad0e0 0x7f0f54074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:31.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.716+0000 7f0f59200640 1 --2- 172.21.15.100:0/1364925113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f540b3550 0x7f0f541f58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:31.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.717+0000 7f0f5a600640 1 -- 172.21.15.100:0/1364925113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f540b3550 msgr2=0x7f0f541f58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:31.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.717+0000 7f0f5a600640 1 --2- 172.21.15.100:0/1364925113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f540b3550 0x7f0f541f58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:31.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.717+0000 7f0f5a600640 1 -- 172.21.15.100:0/1364925113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f540ad0e0 msgr2=0x7f0f54074cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:31.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.717+0000 7f0f5a600640 1 --2- 172.21.15.100:0/1364925113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f540ad0e0 0x7f0f54074cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:31.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.717+0000 7f0f5a600640 1 -- 172.21.15.100:0/1364925113 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f541f6160 con 0x7f0f5415d760 2024-09-17T04:24:31.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.717+0000 7f0f59c00640 1 --2- 172.21.15.100:0/1364925113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f540ad0e0 0x7f0f54074cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:24:31.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.717+0000 7f0f5a600640 1 --2- 172.21.15.100:0/1364925113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5415d760 0x7f0f54156170 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00c760 tx=0x7f0f4c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:31.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.717+0000 7f0f51600640 1 -- 172.21.15.100:0/1364925113 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c003da0 con 0x7f0f5415d760 2024-09-17T04:24:31.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.717+0000 7f0f51600640 1 -- 172.21.15.100:0/1364925113 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f4c00ce10 con 0x7f0f5415d760 2024-09-17T04:24:31.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.717+0000 7f0f5c600640 1 -- 172.21.15.100:0/1364925113 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f541f6440 con 0x7f0f5415d760 2024-09-17T04:24:31.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.717+0000 7f0f51600640 1 -- 172.21.15.100:0/1364925113 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c0052d0 con 0x7f0f5415d760 2024-09-17T04:24:31.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.717+0000 7f0f5c600640 1 -- 172.21.15.100:0/1364925113 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f540733c0 con 0x7f0f5415d760 2024-09-17T04:24:31.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.718+0000 7f0f5c600640 1 -- 172.21.15.100:0/1364925113 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f541f5ed0 con 0x7f0f5415d760 2024-09-17T04:24:31.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.720+0000 7f0f51600640 1 -- 172.21.15.100:0/1364925113 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0f4c005470 con 0x7f0f5415d760 2024-09-17T04:24:31.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.720+0000 7f0f51600640 1 --2- 172.21.15.100:0/1364925113 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0f34078b90 0x7f0f3407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:31.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.721+0000 7f0f51600640 1 -- 172.21.15.100:0/1364925113 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f0f4c09aa90 con 0x7f0f5415d760 2024-09-17T04:24:31.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.721+0000 7f0f59c00640 1 --2- 172.21.15.100:0/1364925113 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0f34078b90 0x7f0f3407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:31.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.721+0000 7f0f59c00640 1 --2- 172.21.15.100:0/1364925113 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0f34078b90 0x7f0f3407b0a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0f48007fd0 tx=0x7f0f48009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:31.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.725+0000 7f0f51600640 1 -- 172.21.15.100:0/1364925113 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0f540ada30 con 0x7f0f5415d760 2024-09-17T04:24:31.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.945+0000 7f0f5c600640 1 -- 172.21.15.100:0/1364925113 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f540ad4f0 con 0x7f0f5415d760 2024-09-17T04:24:31.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.946+0000 7f0f51600640 1 -- 172.21.15.100:0/1364925113 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7f0f4c062ec0 con 0x7f0f5415d760 2024-09-17T04:24:31.946 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:31.946 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":8,"btime":"2024-09-17T04:24:18:058994+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4328,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2129046042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":2129046042},{"type":"v1","addr":"172.21.15.100:6835","nonce":2129046042}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4334,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/44255118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":44255118},{"type":"v1","addr":"172.21.15.184:6836","nonce":44255118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/1644264480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1644264480},{"type":"v1","addr":"172.21.15.100:6834","nonce":1644264480}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:24:14.527272+0000","modified":"2024-09-17T04:24:17.543043+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4368},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4368":{"gid":4368,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/2211103925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2211103925},{"type":"v1","addr":"172.21.15.184:6837","nonce":2211103925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4368,"qdb_cluster":[4368]},"id":1}]} 2024-09-17T04:24:31.946 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 8 2024-09-17T04:24:31.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.948+0000 7f0f5c600640 1 -- 172.21.15.100:0/1364925113 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0f34078b90 msgr2=0x7f0f3407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:31.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.948+0000 7f0f5c600640 1 --2- 172.21.15.100:0/1364925113 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0f34078b90 0x7f0f3407b0a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0f48007fd0 tx=0x7f0f48009040 comp rx=0 tx=0).stop 2024-09-17T04:24:31.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.948+0000 7f0f5c600640 1 -- 172.21.15.100:0/1364925113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5415d760 msgr2=0x7f0f54156170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:31.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.948+0000 7f0f5c600640 1 --2- 172.21.15.100:0/1364925113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5415d760 0x7f0f54156170 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00c760 tx=0x7f0f4c00cc20 comp rx=0 tx=0).stop 2024-09-17T04:24:31.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.948+0000 7f0f5c600640 1 -- 172.21.15.100:0/1364925113 shutdown_connections 2024-09-17T04:24:31.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.948+0000 7f0f5c600640 1 --2- 172.21.15.100:0/1364925113 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0f34078b90 0x7f0f3407b0a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:31.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.949+0000 7f0f5c600640 1 --2- 172.21.15.100:0/1364925113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5415d760 0x7f0f54156170 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:31.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.949+0000 7f0f5c600640 1 --2- 172.21.15.100:0/1364925113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f540b3550 0x7f0f541f58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:31.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.949+0000 7f0f5c600640 1 --2- 172.21.15.100:0/1364925113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f540ad0e0 0x7f0f54074cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:31.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.949+0000 7f0f5c600640 1 -- 172.21.15.100:0/1364925113 >> 172.21.15.100:0/1364925113 conn(0x7f0f5406d730 msgr2=0x7f0f5415f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:31.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.949+0000 7f0f5c600640 1 -- 172.21.15.100:0/1364925113 shutdown_connections 2024-09-17T04:24:31.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:31.949+0000 7f0f5c600640 1 -- 172.21.15.100:0/1364925113 wait complete. 2024-09-17T04:24:31.970 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:24:32.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.190+0000 7f6a14a00640 1 -- 172.21.15.100:0/3042882715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a1015d670 msgr2=0x7f6a10070530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:32.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.190+0000 7f6a14a00640 1 --2- 172.21.15.100:0/3042882715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a1015d670 0x7f6a10070530 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6a100d00a0 tx=0x7f6a0802a980 comp rx=0 tx=0).stop 2024-09-17T04:24:32.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.190+0000 7f6a14a00640 1 -- 172.21.15.100:0/3042882715 shutdown_connections 2024-09-17T04:24:32.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.190+0000 7f6a14a00640 1 --2- 172.21.15.100:0/3042882715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a1015d670 0x7f6a10070530 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:32.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.190+0000 7f6a14a00640 1 --2- 172.21.15.100:0/3042882715 >> v2:172.21.15.184:3300/0 conn(0x7f6a100b2f60 0x7f6a1015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:32.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.190+0000 7f6a14a00640 1 --2- 172.21.15.100:0/3042882715 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a100ad9c0 0x7f6a100af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:32.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.190+0000 7f6a14a00640 1 -- 172.21.15.100:0/3042882715 >> 172.21.15.100:0/3042882715 conn(0x7f6a100b1e40 msgr2=0x7f6a100b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:32.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.190+0000 7f6a14a00640 1 -- 172.21.15.100:0/3042882715 shutdown_connections 2024-09-17T04:24:32.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.190+0000 7f6a14a00640 1 -- 172.21.15.100:0/3042882715 wait complete. 2024-09-17T04:24:32.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.192+0000 7f6a14a00640 1 Processor -- start 2024-09-17T04:24:32.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.193+0000 7f6a14a00640 1 -- start start 2024-09-17T04:24:32.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.194+0000 7f6a14a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a100ad9c0 0x7f6a1014cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:32.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.194+0000 7f6a14a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a100b2f60 0x7f6a10146280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:32.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.194+0000 7f6a14a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a1015d670 0x7f6a101467c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:32.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.194+0000 7f6a14a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1006f290 con 0x7f6a1015d670 2024-09-17T04:24:32.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.194+0000 7f6a14a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6a10070c80 con 0x7f6a100ad9c0 2024-09-17T04:24:32.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.194+0000 7f6a14a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6a1006f410 con 0x7f6a100b2f60 2024-09-17T04:24:32.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.194+0000 7f6a0fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a1015d670 0x7f6a101467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:32.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.194+0000 7f6a0fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a1015d670 0x7f6a101467c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48310/0 (socket says 172.21.15.100:48310) 2024-09-17T04:24:32.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.194+0000 7f6a0fe00640 1 -- 172.21.15.100:0/2104609602 learned_addr learned my addr 172.21.15.100:0/2104609602 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:32.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.195+0000 7f6a0ea00640 1 --2- 172.21.15.100:0/2104609602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a100b2f60 0x7f6a10146280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:32.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.195+0000 7f6a0f400640 1 --2- 172.21.15.100:0/2104609602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a100ad9c0 0x7f6a1014cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:32.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.195+0000 7f6a0fe00640 1 -- 172.21.15.100:0/2104609602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a100b2f60 msgr2=0x7f6a10146280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:32.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.195+0000 7f6a0fe00640 1 --2- 172.21.15.100:0/2104609602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a100b2f60 0x7f6a10146280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:32.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.195+0000 7f6a0fe00640 1 -- 172.21.15.100:0/2104609602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a100ad9c0 msgr2=0x7f6a1014cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:32.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.195+0000 7f6a0fe00640 1 --2- 172.21.15.100:0/2104609602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a100ad9c0 0x7f6a1014cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:32.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.195+0000 7f6a0fe00640 1 -- 172.21.15.100:0/2104609602 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a101f4190 con 0x7f6a1015d670 2024-09-17T04:24:32.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.195+0000 7f6a0ea00640 1 --2- 172.21.15.100:0/2104609602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a100b2f60 0x7f6a10146280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:24:32.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.195+0000 7f6a0f400640 1 --2- 172.21.15.100:0/2104609602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a100ad9c0 0x7f6a1014cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:24:32.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.195+0000 7f6a0fe00640 1 --2- 172.21.15.100:0/2104609602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a1015d670 0x7f6a101467c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6a08009d60 tx=0x7f6a08003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:32.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.195+0000 7f6a07400640 1 -- 172.21.15.100:0/2104609602 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a08034040 con 0x7f6a1015d670 2024-09-17T04:24:32.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.195+0000 7f6a07400640 1 -- 172.21.15.100:0/2104609602 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a08004540 con 0x7f6a1015d670 2024-09-17T04:24:32.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.195+0000 7f6a07400640 1 -- 172.21.15.100:0/2104609602 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a08045480 con 0x7f6a1015d670 2024-09-17T04:24:32.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.195+0000 7f6a14a00640 1 -- 172.21.15.100:0/2104609602 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a1009dd50 con 0x7f6a1015d670 2024-09-17T04:24:32.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.195+0000 7f6a14a00640 1 -- 172.21.15.100:0/2104609602 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a101f72a0 con 0x7f6a1015d670 2024-09-17T04:24:32.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.196+0000 7f6a14a00640 1 -- 172.21.15.100:0/2104609602 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a1009e270 con 0x7f6a1015d670 2024-09-17T04:24:32.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.198+0000 7f6a07400640 1 -- 172.21.15.100:0/2104609602 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6a08037020 con 0x7f6a1015d670 2024-09-17T04:24:32.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.199+0000 7f6a07400640 1 --2- 172.21.15.100:0/2104609602 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f69f0078b90 0x7f69f007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:32.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.199+0000 7f6a07400640 1 -- 172.21.15.100:0/2104609602 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6a080bc500 con 0x7f6a1015d670 2024-09-17T04:24:32.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.200+0000 7f6a0f400640 1 --2- 172.21.15.100:0/2104609602 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f69f0078b90 0x7f69f007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:32.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.200+0000 7f6a0f400640 1 --2- 172.21.15.100:0/2104609602 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f69f0078b90 0x7f69f007b0a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6a00009ab0 tx=0x7f6a00005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:32.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.204+0000 7f6a07400640 1 -- 172.21.15.100:0/2104609602 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6a1009e270 con 0x7f6a1015d670 2024-09-17T04:24:32.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.388+0000 7f6a14a00640 1 -- 172.21.15.100:0/2104609602 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6a101f75a0 con 0x7f6a1015d670 2024-09-17T04:24:32.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.389+0000 7f6a07400640 1 -- 172.21.15.100:0/2104609602 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16713 (secure 0 0 0) 0x7f6a101f4190 con 0x7f6a1015d670 2024-09-17T04:24:32.390 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:32.390 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":23,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:24:16.038948+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:24:12.826665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T04:24:13.485446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:24:32.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.392+0000 7f6a14a00640 1 -- 172.21.15.100:0/2104609602 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f69f0078b90 msgr2=0x7f69f007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:32.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.392+0000 7f6a14a00640 1 --2- 172.21.15.100:0/2104609602 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f69f0078b90 0x7f69f007b0a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6a00009ab0 tx=0x7f6a00005f90 comp rx=0 tx=0).stop 2024-09-17T04:24:32.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.392+0000 7f6a14a00640 1 -- 172.21.15.100:0/2104609602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a1015d670 msgr2=0x7f6a101467c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:32.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.392+0000 7f6a14a00640 1 --2- 172.21.15.100:0/2104609602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a1015d670 0x7f6a101467c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6a08009d60 tx=0x7f6a08003f10 comp rx=0 tx=0).stop 2024-09-17T04:24:32.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.392+0000 7f6a14a00640 1 -- 172.21.15.100:0/2104609602 shutdown_connections 2024-09-17T04:24:32.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.392+0000 7f6a14a00640 1 --2- 172.21.15.100:0/2104609602 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f69f0078b90 0x7f69f007b0a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:32.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.392+0000 7f6a14a00640 1 --2- 172.21.15.100:0/2104609602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a1015d670 0x7f6a101467c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:32.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.392+0000 7f6a14a00640 1 --2- 172.21.15.100:0/2104609602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a100b2f60 0x7f6a10146280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:32.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.392+0000 7f6a14a00640 1 --2- 172.21.15.100:0/2104609602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a100ad9c0 0x7f6a1014cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:32.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.392+0000 7f6a14a00640 1 -- 172.21.15.100:0/2104609602 >> 172.21.15.100:0/2104609602 conn(0x7f6a100b1e40 msgr2=0x7f6a100afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:32.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.392+0000 7f6a14a00640 1 -- 172.21.15.100:0/2104609602 shutdown_connections 2024-09-17T04:24:32.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.392+0000 7f6a14a00640 1 -- 172.21.15.100:0/2104609602 wait complete. 2024-09-17T04:24:32.415 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:24:32.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.635+0000 7f02cda00640 1 -- 172.21.15.100:0/3918282924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02c80b9e00 msgr2=0x7f02c8162a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:32.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.635+0000 7f02cda00640 1 --2- 172.21.15.100:0/3918282924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02c80b9e00 0x7f02c8162a50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f02b8009860 tx=0x7f02b802a510 comp rx=0 tx=0).stop 2024-09-17T04:24:32.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.635+0000 7f02cda00640 1 -- 172.21.15.100:0/3918282924 shutdown_connections 2024-09-17T04:24:32.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.635+0000 7f02cda00640 1 --2- 172.21.15.100:0/3918282924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02c80b9e00 0x7f02c8162a50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:32.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.635+0000 7f02cda00640 1 -- 172.21.15.100:0/3918282924 >> 172.21.15.100:0/3918282924 conn(0x7f02c808bfe0 msgr2=0x7f02c809d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:32.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.635+0000 7f02cda00640 1 -- 172.21.15.100:0/3918282924 shutdown_connections 2024-09-17T04:24:32.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.635+0000 7f02cda00640 1 -- 172.21.15.100:0/3918282924 wait complete. 2024-09-17T04:24:32.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.637+0000 7f02cda00640 1 Processor -- start 2024-09-17T04:24:32.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.637+0000 7f02cda00640 1 -- start start 2024-09-17T04:24:32.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.639+0000 7f02cda00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02c80b3160 0x7f02c809fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:32.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.639+0000 7f02cda00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02c80b9e00 0x7f02c80cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:32.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.639+0000 7f02cda00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02c80d1580 0x7f02c80cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:32.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.639+0000 7f02cda00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f02c80d1990 con 0x7f02c80d1580 2024-09-17T04:24:32.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.639+0000 7f02cda00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f02c80d4c50 con 0x7f02c80b9e00 2024-09-17T04:24:32.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.639+0000 7f02cda00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f02c80d1b10 con 0x7f02c80b3160 2024-09-17T04:24:32.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.639+0000 7f02c7e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02c80b3160 0x7f02c809fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:32.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.639+0000 7f02cd000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02c80d1580 0x7f02c80cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:32.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.639+0000 7f02c7e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02c80b3160 0x7f02c809fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.100:53672/0 (socket says 172.21.15.100:53672) 2024-09-17T04:24:32.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.639+0000 7f02c7e00640 1 -- 172.21.15.100:0/3923985300 learned_addr learned my addr 172.21.15.100:0/3923985300 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:32.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.639+0000 7f02c7400640 1 --2- 172.21.15.100:0/3923985300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02c80b9e00 0x7f02c80cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:32.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.640+0000 7f02cd000640 1 -- 172.21.15.100:0/3923985300 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02c80b3160 msgr2=0x7f02c809fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:32.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.640+0000 7f02cd000640 1 --2- 172.21.15.100:0/3923985300 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02c80b3160 0x7f02c809fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:32.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.640+0000 7f02cd000640 1 -- 172.21.15.100:0/3923985300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02c80b9e00 msgr2=0x7f02c80cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:32.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.640+0000 7f02cd000640 1 --2- 172.21.15.100:0/3923985300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02c80b9e00 0x7f02c80cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:32.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.640+0000 7f02cd000640 1 -- 172.21.15.100:0/3923985300 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c80a0960 con 0x7f02c80d1580 2024-09-17T04:24:32.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.640+0000 7f02c7e00640 1 --2- 172.21.15.100:0/3923985300 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02c80b3160 0x7f02c809fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:24:32.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.640+0000 7f02cd000640 1 --2- 172.21.15.100:0/3923985300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02c80d1580 0x7f02c80cdf00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f02c000c700 tx=0x7f02c000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:32.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.640+0000 7f02bfe00640 1 -- 172.21.15.100:0/3923985300 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c0003b70 con 0x7f02c80d1580 2024-09-17T04:24:32.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.640+0000 7f02bfe00640 1 -- 172.21.15.100:0/3923985300 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02c0003d10 con 0x7f02c80d1580 2024-09-17T04:24:32.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.640+0000 7f02cda00640 1 -- 172.21.15.100:0/3923985300 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c80a0c40 con 0x7f02c80d1580 2024-09-17T04:24:32.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.640+0000 7f02bfe00640 1 -- 172.21.15.100:0/3923985300 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c0005330 con 0x7f02c80d1580 2024-09-17T04:24:32.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.640+0000 7f02cda00640 1 -- 172.21.15.100:0/3923985300 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c80a1200 con 0x7f02c80d1580 2024-09-17T04:24:32.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.641+0000 7f02cda00640 1 -- 172.21.15.100:0/3923985300 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c8162a50 con 0x7f02c80d1580 2024-09-17T04:24:32.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.643+0000 7f02bfe00640 1 -- 172.21.15.100:0/3923985300 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f02c00054d0 con 0x7f02c80d1580 2024-09-17T04:24:32.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.644+0000 7f02bfe00640 1 --2- 172.21.15.100:0/3923985300 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f02a8078b90 0x7f02a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:32.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.644+0000 7f02bfe00640 1 -- 172.21.15.100:0/3923985300 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f02c009a9e0 con 0x7f02c80d1580 2024-09-17T04:24:32.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.644+0000 7f02c7e00640 1 --2- 172.21.15.100:0/3923985300 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f02a8078b90 0x7f02a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:32.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.645+0000 7f02c7e00640 1 --2- 172.21.15.100:0/3923985300 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f02a8078b90 0x7f02a807b0a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f02b80095f0 tx=0x7f02b8004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:32.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.649+0000 7f02bfe00640 1 -- 172.21.15.100:0/3923985300 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f02c0062730 con 0x7f02c80d1580 2024-09-17T04:24:32.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.867+0000 7f02cda00640 1 -- 172.21.15.100:0/3923985300 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02c80a06f0 con 0x7f02c80d1580 2024-09-17T04:24:32.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.868+0000 7f02bfe00640 1 -- 172.21.15.100:0/3923985300 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7f02c0067160 con 0x7f02c80d1580 2024-09-17T04:24:32.868 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:32.869 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":8,"btime":"2024-09-17T04:24:18:058994+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4328,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2129046042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":2129046042},{"type":"v1","addr":"172.21.15.100:6835","nonce":2129046042}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4334,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/44255118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":44255118},{"type":"v1","addr":"172.21.15.184:6836","nonce":44255118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/1644264480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1644264480},{"type":"v1","addr":"172.21.15.100:6834","nonce":1644264480}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:24:14.527272+0000","modified":"2024-09-17T04:24:17.543043+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4368},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4368":{"gid":4368,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/2211103925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2211103925},{"type":"v1","addr":"172.21.15.184:6837","nonce":2211103925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4368,"qdb_cluster":[4368]},"id":1}]} 2024-09-17T04:24:32.869 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 8 2024-09-17T04:24:32.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.870+0000 7f02cda00640 1 -- 172.21.15.100:0/3923985300 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f02a8078b90 msgr2=0x7f02a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:32.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.870+0000 7f02cda00640 1 --2- 172.21.15.100:0/3923985300 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f02a8078b90 0x7f02a807b0a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f02b80095f0 tx=0x7f02b8004730 comp rx=0 tx=0).stop 2024-09-17T04:24:32.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.870+0000 7f02cda00640 1 -- 172.21.15.100:0/3923985300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02c80d1580 msgr2=0x7f02c80cdf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:32.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.870+0000 7f02cda00640 1 --2- 172.21.15.100:0/3923985300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02c80d1580 0x7f02c80cdf00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f02c000c700 tx=0x7f02c000cbc0 comp rx=0 tx=0).stop 2024-09-17T04:24:32.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.871+0000 7f02cda00640 1 -- 172.21.15.100:0/3923985300 shutdown_connections 2024-09-17T04:24:32.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.871+0000 7f02cda00640 1 --2- 172.21.15.100:0/3923985300 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f02a8078b90 0x7f02a807b0a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:32.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.871+0000 7f02cda00640 1 --2- 172.21.15.100:0/3923985300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02c80d1580 0x7f02c80cdf00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:32.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.871+0000 7f02cda00640 1 --2- 172.21.15.100:0/3923985300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02c80b9e00 0x7f02c80cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:32.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.871+0000 7f02cda00640 1 --2- 172.21.15.100:0/3923985300 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02c80b3160 0x7f02c809fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:32.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.871+0000 7f02cda00640 1 -- 172.21.15.100:0/3923985300 >> 172.21.15.100:0/3923985300 conn(0x7f02c808bfe0 msgr2=0x7f02c8068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:32.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.871+0000 7f02cda00640 1 -- 172.21.15.100:0/3923985300 shutdown_connections 2024-09-17T04:24:32.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:32.871+0000 7f02cda00640 1 -- 172.21.15.100:0/3923985300 wait complete. 2024-09-17T04:24:33.298 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T04:24:14.527272+0000', 'modified': '2024-09-17T04:24:17.543043+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4368}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4368': {'gid': 4368, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.184:6837/2211103925', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6835', 'nonce': 2211103925}, {'type': 'v1', 'addr': '172.21.15.184:6837', 'nonce': 2211103925}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4368, 'qdb_cluster': [4368]} 2024-09-17T04:24:33.298 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T04:24:33.298 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-17T04:24:33.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.518+0000 7fdc92000640 1 -- 172.21.15.100:0/2776880920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc8c0cfc80 msgr2=0x7fdc8c156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:33.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.518+0000 7fdc92000640 1 --2- 172.21.15.100:0/2776880920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc8c0cfc80 0x7fdc8c156e40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c009830 tx=0x7fdc7c02a4d0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.518+0000 7fdc92000640 1 -- 172.21.15.100:0/2776880920 shutdown_connections 2024-09-17T04:24:33.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.518+0000 7fdc92000640 1 --2- 172.21.15.100:0/2776880920 >> v2:172.21.15.184:3300/0 conn(0x7fdc8c0705b0 0x7fdc8c0709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.518+0000 7fdc92000640 1 --2- 172.21.15.100:0/2776880920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc8c0cfc80 0x7fdc8c156e40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.518+0000 7fdc92000640 1 -- 172.21.15.100:0/2776880920 >> 172.21.15.100:0/2776880920 conn(0x7fdc8c06d3d0 msgr2=0x7fdc8c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:33.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.518+0000 7fdc92000640 1 -- 172.21.15.100:0/2776880920 shutdown_connections 2024-09-17T04:24:33.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.518+0000 7fdc92000640 1 -- 172.21.15.100:0/2776880920 wait complete. 2024-09-17T04:24:33.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.520+0000 7fdc92000640 1 Processor -- start 2024-09-17T04:24:33.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.520+0000 7fdc92000640 1 -- start start 2024-09-17T04:24:33.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.522+0000 7fdc92000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc8c0705b0 0x7fdc8c15a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:33.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.522+0000 7fdc92000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc8c0cfc80 0x7fdc8c1f3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:33.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.522+0000 7fdc92000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc8c15ce30 0x7fdc8c1f4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:33.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.522+0000 7fdc92000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c0cced0 con 0x7fdc8c15ce30 2024-09-17T04:24:33.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.522+0000 7fdc92000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c0d0170 con 0x7fdc8c0cfc80 2024-09-17T04:24:33.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.522+0000 7fdc92000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdc8c0cd050 con 0x7fdc8c0705b0 2024-09-17T04:24:33.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.522+0000 7fdc8be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc8c15ce30 0x7fdc8c1f4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:33.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.522+0000 7fdc8be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc8c15ce30 0x7fdc8c1f4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48326/0 (socket says 172.21.15.100:48326) 2024-09-17T04:24:33.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.522+0000 7fdc8be00640 1 -- 172.21.15.100:0/515168304 learned_addr learned my addr 172.21.15.100:0/515168304 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:33.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.522+0000 7fdc8b400640 1 --2- 172.21.15.100:0/515168304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc8c0705b0 0x7fdc8c15a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:33.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.522+0000 7fdc82a00640 1 --2- 172.21.15.100:0/515168304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc8c0cfc80 0x7fdc8c1f3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:33.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.522+0000 7fdc8be00640 1 -- 172.21.15.100:0/515168304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc8c0705b0 msgr2=0x7fdc8c15a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:33.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.522+0000 7fdc8be00640 1 --2- 172.21.15.100:0/515168304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc8c0705b0 0x7fdc8c15a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.523+0000 7fdc8be00640 1 -- 172.21.15.100:0/515168304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc8c0cfc80 msgr2=0x7fdc8c1f3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:33.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.523+0000 7fdc8be00640 1 --2- 172.21.15.100:0/515168304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc8c0cfc80 0x7fdc8c1f3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.523+0000 7fdc8be00640 1 -- 172.21.15.100:0/515168304 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc8c15a570 con 0x7fdc8c15ce30 2024-09-17T04:24:33.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.523+0000 7fdc8be00640 1 --2- 172.21.15.100:0/515168304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc8c15ce30 0x7fdc8c1f4330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400c730 tx=0x7fdc8400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:33.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.523+0000 7fdc83e00640 1 -- 172.21.15.100:0/515168304 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc84017070 con 0x7fdc8c15ce30 2024-09-17T04:24:33.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.523+0000 7fdc83e00640 1 -- 172.21.15.100:0/515168304 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc84003c60 con 0x7fdc8c15ce30 2024-09-17T04:24:33.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.523+0000 7fdc92000640 1 -- 172.21.15.100:0/515168304 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc8c1f7360 con 0x7fdc8c15ce30 2024-09-17T04:24:33.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.523+0000 7fdc83e00640 1 -- 172.21.15.100:0/515168304 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc840052d0 con 0x7fdc8c15ce30 2024-09-17T04:24:33.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.523+0000 7fdc92000640 1 -- 172.21.15.100:0/515168304 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc8c1f7920 con 0x7fdc8c15ce30 2024-09-17T04:24:33.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.526+0000 7fdc83e00640 1 -- 172.21.15.100:0/515168304 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdc8401a070 con 0x7fdc8c15ce30 2024-09-17T04:24:33.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.527+0000 7fdc9904e740 1 Processor -- start 2024-09-17T04:24:33.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.527+0000 7fdc83e00640 1 --2- 172.21.15.100:0/515168304 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdc68078bc0 0x7fdc6807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:33.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.527+0000 7fdc9904e740 1 -- start start 2024-09-17T04:24:33.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.527+0000 7fdc83e00640 1 -- 172.21.15.100:0/515168304 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fdc8409a340 con 0x7fdc8c15ce30 2024-09-17T04:24:33.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.527+0000 7fdc82a00640 1 --2- 172.21.15.100:0/515168304 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdc68078bc0 0x7fdc6807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:33.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.528+0000 7fdc82a00640 1 --2- 172.21.15.100:0/515168304 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdc68078bc0 0x7fdc6807b0d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c1f4cb0 tx=0x7fdc74008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:33.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.528+0000 7fdc9904e740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de01823270 0x55de01bf3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:33.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.528+0000 7fdc9904e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de017fa3b0 0x55de01bf3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:33.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.528+0000 7fdc9904e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de01802590 0x55de01bc3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:33.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.528+0000 7fdc9904e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55de01b4d430 con 0x55de01802590 2024-09-17T04:24:33.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.528+0000 7fdc9904e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55de01b3d5d0 con 0x55de017fa3b0 2024-09-17T04:24:33.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.528+0000 7fdc9904e740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55de01b2ca00 con 0x55de01823270 2024-09-17T04:24:33.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.529+0000 7fdc8be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de01802590 0x55de01bc3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:33.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.529+0000 7fdc8b400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de01823270 0x55de01bf3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:33.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.529+0000 7fdc82a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de017fa3b0 0x55de01bf3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:33.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.529+0000 7fdc8be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de01802590 0x55de01bc3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48334/0 (socket says 172.21.15.100:48334) 2024-09-17T04:24:33.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.529+0000 7fdc8be00640 1 -- 172.21.15.100:0/4228169078 learned_addr learned my addr 172.21.15.100:0/4228169078 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:33.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.529+0000 7fdc8be00640 1 -- 172.21.15.100:0/4228169078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de01823270 msgr2=0x55de01bf3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:24:33.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.529+0000 7fdc8be00640 1 --2- 172.21.15.100:0/4228169078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de01823270 0x55de01bf3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.529+0000 7fdc8be00640 1 -- 172.21.15.100:0/4228169078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de017fa3b0 msgr2=0x55de01bf3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:24:33.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.529+0000 7fdc8be00640 1 --2- 172.21.15.100:0/4228169078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de017fa3b0 0x55de01bf3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.529+0000 7fdc8be00640 1 -- 172.21.15.100:0/4228169078 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de01af5520 con 0x55de01802590 2024-09-17T04:24:33.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.529+0000 7fdc8be00640 1 --2- 172.21.15.100:0/4228169078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de01802590 0x55de01bc3f60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400c700 tx=0x7fdc84061b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a8fae2a13a92938 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:33.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.529+0000 7fdc7b400640 1 -- 172.21.15.100:0/4228169078 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc84062080 con 0x55de01802590 2024-09-17T04:24:33.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.530+0000 7fdc7b400640 1 -- 172.21.15.100:0/4228169078 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc8406baa0 con 0x55de01802590 2024-09-17T04:24:33.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.530+0000 7fdc7b400640 1 -- 172.21.15.100:0/4228169078 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8406b610 con 0x55de01802590 2024-09-17T04:24:33.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.530+0000 7fdc9904e740 1 -- 172.21.15.100:0/4228169078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de01802590 msgr2=0x55de01bc3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:33.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.530+0000 7fdc9904e740 1 --2- 172.21.15.100:0/4228169078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de01802590 0x55de01bc3f60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400c700 tx=0x7fdc84061b90 comp rx=0 tx=0).stop 2024-09-17T04:24:33.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.530+0000 7fdc9904e740 1 -- 172.21.15.100:0/4228169078 shutdown_connections 2024-09-17T04:24:33.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.530+0000 7fdc9904e740 1 --2- 172.21.15.100:0/4228169078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de01802590 0x55de01bc3f60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.530+0000 7fdc9904e740 1 --2- 172.21.15.100:0/4228169078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de017fa3b0 0x55de01bf3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.530+0000 7fdc9904e740 1 --2- 172.21.15.100:0/4228169078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de01823270 0x55de01bf3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.530+0000 7fdc9904e740 1 -- 172.21.15.100:0/4228169078 >> 172.21.15.100:0/4228169078 conn(0x55de018173c0 msgr2=0x55de01bc9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:33.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.530+0000 7fdc9904e740 1 -- 172.21.15.100:0/4228169078 shutdown_connections 2024-09-17T04:24:33.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.530+0000 7fdc9904e740 1 -- 172.21.15.100:0/4228169078 wait complete. 2024-09-17T04:24:33.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.531+0000 7fdc9904e740 1 -- start start 2024-09-17T04:24:33.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.532+0000 7fdc9904e740 1 Processor -- start 2024-09-17T04:24:33.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.533+0000 7fdc9904e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de017fa3b0 0x55de01c096f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:33.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.533+0000 7fdc9904e740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de01802590 0x55de01bea1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:33.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.533+0000 7fdc9904e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de01823270 0x55de01c211e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:33.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.533+0000 7fdc9904e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55de01ad3330 con 0x55de017fa3b0 2024-09-17T04:24:33.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.533+0000 7fdc9904e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55de01b30810 con 0x55de01823270 2024-09-17T04:24:33.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.533+0000 7fdc9904e740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55de0189d620 con 0x55de01802590 2024-09-17T04:24:33.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.533+0000 7fdc9904e740 10 client.0 _ll_register_callbacks cb 0x55de01ad8420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:24:33.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.533+0000 7fdc8b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de017fa3b0 0x55de01c096f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:33.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.533+0000 7fdc8b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de017fa3b0 0x55de01c096f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48346/0 (socket says 172.21.15.100:48346) 2024-09-17T04:24:33.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.533+0000 7fdc8b400640 1 -- 172.21.15.100:0/1016496016 learned_addr learned my addr 172.21.15.100:0/1016496016 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:33.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.534+0000 7fdc82a00640 1 --2- 172.21.15.100:0/1016496016 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de01802590 0x55de01bea1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:33.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.534+0000 7fdc8be00640 1 --2- 172.21.15.100:0/1016496016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de01823270 0x55de01c211e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:33.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.534+0000 7fdc8b400640 1 -- 172.21.15.100:0/1016496016 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de01802590 msgr2=0x55de01bea1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:24:33.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.534+0000 7fdc8b400640 1 --2- 172.21.15.100:0/1016496016 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de01802590 0x55de01bea1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.534+0000 7fdc8b400640 1 -- 172.21.15.100:0/1016496016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de01823270 msgr2=0x55de01c211e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:24:33.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.534+0000 7fdc8b400640 1 --2- 172.21.15.100:0/1016496016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de01823270 0x55de01c211e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.534+0000 7fdc8b400640 1 -- 172.21.15.100:0/1016496016 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de0188e850 con 0x55de017fa3b0 2024-09-17T04:24:33.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.534+0000 7fdc8b400640 1 --2- 172.21.15.100:0/1016496016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de017fa3b0 0x55de01c096f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c009830 tx=0x7fdc7c003890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a1f6624fe81049 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:33.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.534+0000 7fdc6fe00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-17T04:24:33.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.534+0000 7fdc6fe00640 1 -- 172.21.15.100:0/1016496016 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c003cd0 con 0x55de017fa3b0 2024-09-17T04:24:33.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.534+0000 7fdc6fe00640 1 -- 172.21.15.100:0/1016496016 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fdc7c046ad0 con 0x55de017fa3b0 2024-09-17T04:24:33.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.534+0000 7fdc9904e740 1 -- 172.21.15.100:0/1016496016 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc8406baa0 con 0x55de017fa3b0 2024-09-17T04:24:33.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.535+0000 7fdc6fe00640 1 -- 172.21.15.100:0/1016496016 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fdc7c003e70 con 0x55de017fa3b0 2024-09-17T04:24:33.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.535+0000 7fdc9904e740 10 client.4650 fetch_fsmap learned FSMap version 8 2024-09-17T04:24:33.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.535+0000 7fdc9904e740 1 -- 172.21.15.100:0/1016496016 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55de019056b0 con 0x55de017fa3b0 2024-09-17T04:24:33.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.535+0000 7fdc6fe00640 1 -- 172.21.15.100:0/1016496016 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc8406baa0 con 0x55de017fa3b0 2024-09-17T04:24:33.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.535+0000 7fdc6fe00640 1 -- 172.21.15.100:0/1016496016 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7fdc7c046cd0 con 0x55de017fa3b0 2024-09-17T04:24:33.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.535+0000 7fdc9904e740 10 client.4650 fetch_fsmap finished waiting for FSMap version 8 2024-09-17T04:24:33.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.535+0000 7fdc9904e740 10 client.4650 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T04:24:33.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.535+0000 7fdc9904e740 20 client.4650 populate_metadata read hostname 'smithi100' 2024-09-17T04:24:33.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.535+0000 7fdc9904e740 1 --2- 172.21.15.100:0/1016496016 >> [v2:172.21.15.184:6835/2211103925,v1:172.21.15.184:6837/2211103925] conn(0x55de017e1e40 0x55de01c1ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:33.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.535+0000 7fdc9904e740 4 client.4650 mds_command: new command op to 4368 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T04:24:33.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.535+0000 7fdc9904e740 1 -- 172.21.15.100:0/1016496016 --> [v2:172.21.15.184:6835/2211103925,v1:172.21.15.184:6837/2211103925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de01b3f520 con 0x55de017e1e40 2024-09-17T04:24:33.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.536+0000 7fdc82a00640 1 --2- 172.21.15.100:0/1016496016 >> [v2:172.21.15.184:6835/2211103925,v1:172.21.15.184:6837/2211103925] conn(0x55de017e1e40 0x55de01c1ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:33.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.536+0000 7fdc82a00640 1 --2- 172.21.15.100:0/1016496016 >> [v2:172.21.15.184:6835/2211103925,v1:172.21.15.184:6837/2211103925] conn(0x55de017e1e40 0x55de01c1ddd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bf27a04a88a3ff4 server_cookie=772ffad1013935ea in_seq=0 out_seq=0 2024-09-17T04:24:33.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.536+0000 7fdc6fe00640 10 client.4650 ms_handle_connect on v2:172.21.15.184:6835/2211103925 2024-09-17T04:24:33.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.537+0000 7fdc6fe00640 1 -- 172.21.15.100:0/1016496016 <== mds.0 v2:172.21.15.184:6835/2211103925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55de01b3f520 con 0x55de017e1e40 2024-09-17T04:24:33.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.537+0000 7fdc6fe00640 10 client.4650 handle_command_reply: tid=0 2024-09-17T04:24:33.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.537+0000 7fdc9904e740 1 client.4650 shutdown 2024-09-17T04:24:33.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.537+0000 7fdc9904e740 1 -- 172.21.15.100:0/1016496016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de017fa3b0 msgr2=0x55de01c096f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:33.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.537+0000 7fdc9904e740 1 --2- 172.21.15.100:0/1016496016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de017fa3b0 0x55de01c096f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c009830 tx=0x7fdc7c003890 comp rx=0 tx=0).stop 2024-09-17T04:24:33.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.538+0000 7fdc9904e740 1 -- 172.21.15.100:0/1016496016 shutdown_connections 2024-09-17T04:24:33.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.538+0000 7fdc9904e740 1 --2- 172.21.15.100:0/1016496016 >> [v2:172.21.15.184:6835/2211103925,v1:172.21.15.184:6837/2211103925] conn(0x55de017e1e40 0x55de01c1ddd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.538+0000 7fdc9904e740 1 --2- 172.21.15.100:0/1016496016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de01823270 0x55de01c211e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.538+0000 7fdc9904e740 1 --2- 172.21.15.100:0/1016496016 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de01802590 0x55de01bea1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.538+0000 7fdc9904e740 1 --2- 172.21.15.100:0/1016496016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de017fa3b0 0x55de01c096f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.538+0000 7fdc6fe00640 0 client.4650 ms_handle_reset on v2:172.21.15.184:6835/2211103925 2024-09-17T04:24:33.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.538+0000 7fdc9904e740 1 -- 172.21.15.100:0/1016496016 >> 172.21.15.100:0/1016496016 conn(0x55de018173c0 msgr2=0x55de01c113d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:33.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.538+0000 7fdc9904e740 1 -- 172.21.15.100:0/1016496016 shutdown_connections 2024-09-17T04:24:33.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.538+0000 7fdc9904e740 1 -- 172.21.15.100:0/1016496016 wait complete. 2024-09-17T04:24:33.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.538+0000 7fdc9904e740 20 client.4650 trim_cache size 0 max 16384 2024-09-17T04:24:33.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.557+0000 7fdc9904e740 1 Processor -- start 2024-09-17T04:24:33.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.557+0000 7fdc9904e740 1 -- start start 2024-09-17T04:24:33.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.558+0000 7fdc9904e740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de017fa3b0 0x55de01c0fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:33.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.558+0000 7fdc9904e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de01802590 0x55de01be8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:33.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.558+0000 7fdc9904e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de01823270 0x55de01be92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:33.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.558+0000 7fdc9904e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c0752d0 con 0x55de01802590 2024-09-17T04:24:33.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.558+0000 7fdc9904e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c073b10 con 0x55de01823270 2024-09-17T04:24:33.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.558+0000 7fdc9904e740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdc8c074c80 con 0x55de017fa3b0 2024-09-17T04:24:33.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.558+0000 7fdc82a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de01802590 0x55de01be8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:33.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.558+0000 7fdc82a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de01802590 0x55de01be8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48350/0 (socket says 172.21.15.100:48350) 2024-09-17T04:24:33.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.558+0000 7fdc82a00640 1 -- 172.21.15.100:0/2461050022 learned_addr learned my addr 172.21.15.100:0/2461050022 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:33.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.559+0000 7fdc8b400640 1 --2- 172.21.15.100:0/2461050022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de017fa3b0 0x55de01c0fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:33.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.559+0000 7fdc8be00640 1 --2- 172.21.15.100:0/2461050022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de01823270 0x55de01be92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:33.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.559+0000 7fdc82a00640 1 -- 172.21.15.100:0/2461050022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de017fa3b0 msgr2=0x55de01c0fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:24:33.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.559+0000 7fdc82a00640 1 --2- 172.21.15.100:0/2461050022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de017fa3b0 0x55de01c0fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.559+0000 7fdc82a00640 1 -- 172.21.15.100:0/2461050022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de01823270 msgr2=0x55de01be92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:24:33.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.559+0000 7fdc82a00640 1 --2- 172.21.15.100:0/2461050022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de01823270 0x55de01be92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.559+0000 7fdc82a00640 1 -- 172.21.15.100:0/2461050022 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de01bc9150 con 0x55de01802590 2024-09-17T04:24:33.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.559+0000 7fdc8b400640 1 --2- 172.21.15.100:0/2461050022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de017fa3b0 0x55de01c0fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:24:33.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.559+0000 7fdc8be00640 1 --2- 172.21.15.100:0/2461050022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de01823270 0x55de01be92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:24:33.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.559+0000 7fdc82a00640 1 --2- 172.21.15.100:0/2461050022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de01802590 0x55de01be8d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55de01c0f7f0 tx=0x7fdc74012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef57d6e1225d07ae server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:33.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.559+0000 7fdc7aa00640 1 -- 172.21.15.100:0/2461050022 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc740160d0 con 0x55de01802590 2024-09-17T04:24:33.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.559+0000 7fdc7aa00640 1 -- 172.21.15.100:0/2461050022 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc7402e070 con 0x55de01802590 2024-09-17T04:24:33.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.559+0000 7fdc7aa00640 1 -- 172.21.15.100:0/2461050022 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc74016920 con 0x55de01802590 2024-09-17T04:24:33.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.560+0000 7fdc9904e740 1 -- 172.21.15.100:0/2461050022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de01802590 msgr2=0x55de01be8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:33.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.560+0000 7fdc9904e740 1 --2- 172.21.15.100:0/2461050022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de01802590 0x55de01be8d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55de01c0f7f0 tx=0x7fdc74012000 comp rx=0 tx=0).stop 2024-09-17T04:24:33.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.560+0000 7fdc9904e740 1 -- 172.21.15.100:0/2461050022 shutdown_connections 2024-09-17T04:24:33.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.560+0000 7fdc9904e740 1 --2- 172.21.15.100:0/2461050022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de01823270 0x55de01be92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.560+0000 7fdc9904e740 1 --2- 172.21.15.100:0/2461050022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de01802590 0x55de01be8d90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.560+0000 7fdc9904e740 1 --2- 172.21.15.100:0/2461050022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de017fa3b0 0x55de01c0fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.560+0000 7fdc9904e740 1 -- 172.21.15.100:0/2461050022 >> 172.21.15.100:0/2461050022 conn(0x55de018173c0 msgr2=0x55de01c09300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:33.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.560+0000 7fdc9904e740 1 -- 172.21.15.100:0/2461050022 shutdown_connections 2024-09-17T04:24:33.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.560+0000 7fdc9904e740 1 -- 172.21.15.100:0/2461050022 wait complete. 2024-09-17T04:24:33.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.561+0000 7fdc9904e740 1 -- start start 2024-09-17T04:24:33.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.561+0000 7fdc9904e740 1 Processor -- start 2024-09-17T04:24:33.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.563+0000 7fdc9904e740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de017fa3b0 0x55de01beb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:33.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.563+0000 7fdc9904e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de01802590 0x55de01c928c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:33.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.563+0000 7fdc9904e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de01823270 0x55de01c85200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:33.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.563+0000 7fdc9904e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55de01c21e40 con 0x55de01823270 2024-09-17T04:24:33.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.563+0000 7fdc9904e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55de01bef740 con 0x55de01802590 2024-09-17T04:24:33.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.563+0000 7fdc9904e740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55de01bedc40 con 0x55de017fa3b0 2024-09-17T04:24:33.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.563+0000 7fdc9904e740 10 client.0 _ll_register_callbacks cb 0x7fdc7c004580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:24:33.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.563+0000 7fdc8be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de01823270 0x55de01c85200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:33.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.563+0000 7fdc8be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de01823270 0x55de01c85200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48354/0 (socket says 172.21.15.100:48354) 2024-09-17T04:24:33.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.563+0000 7fdc8be00640 1 -- 172.21.15.100:0/2695844441 learned_addr learned my addr 172.21.15.100:0/2695844441 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:33.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.563+0000 7fdc8b400640 1 --2- 172.21.15.100:0/2695844441 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de017fa3b0 0x55de01beb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:33.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.563+0000 7fdc82a00640 1 --2- 172.21.15.100:0/2695844441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de01802590 0x55de01c928c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:33.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.563+0000 7fdc8be00640 1 -- 172.21.15.100:0/2695844441 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de017fa3b0 msgr2=0x55de01beb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:24:33.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.563+0000 7fdc8be00640 1 --2- 172.21.15.100:0/2695844441 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de017fa3b0 0x55de01beb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.563+0000 7fdc8be00640 1 -- 172.21.15.100:0/2695844441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de01802590 msgr2=0x55de01c928c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:24:33.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.563+0000 7fdc8be00640 1 --2- 172.21.15.100:0/2695844441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de01802590 0x55de01c928c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.563+0000 7fdc8be00640 1 -- 172.21.15.100:0/2695844441 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de01c097b0 con 0x55de01823270 2024-09-17T04:24:33.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.564+0000 7fdc8be00640 1 --2- 172.21.15.100:0/2695844441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de01823270 0x55de01c85200 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400a430 tx=0x7fdc8407c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72fedad327a6f831 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:33.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.564+0000 7fdc6fe00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-17T04:24:33.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.564+0000 7fdc6fe00640 1 -- 172.21.15.100:0/2695844441 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8406baa0 con 0x55de01823270 2024-09-17T04:24:33.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.564+0000 7fdc6fe00640 1 -- 172.21.15.100:0/2695844441 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fdc84084410 con 0x55de01823270 2024-09-17T04:24:33.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.564+0000 7fdc9904e740 1 -- 172.21.15.100:0/2695844441 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc7402e070 con 0x55de01823270 2024-09-17T04:24:33.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.564+0000 7fdc6fe00640 1 -- 172.21.15.100:0/2695844441 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fdc84062080 con 0x55de01823270 2024-09-17T04:24:33.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.564+0000 7fdc9904e740 10 client.4656 fetch_fsmap learned FSMap version 8 2024-09-17T04:24:33.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.564+0000 7fdc9904e740 1 -- 172.21.15.100:0/2695844441 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55de01c8f380 con 0x55de01823270 2024-09-17T04:24:33.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.564+0000 7fdc6fe00640 1 -- 172.21.15.100:0/2695844441 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc7402e070 con 0x55de01823270 2024-09-17T04:24:33.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.564+0000 7fdc6fe00640 1 -- 172.21.15.100:0/2695844441 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7fdc840881c0 con 0x55de01823270 2024-09-17T04:24:33.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.564+0000 7fdc9904e740 10 client.4656 fetch_fsmap finished waiting for FSMap version 8 2024-09-17T04:24:33.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.564+0000 7fdc9904e740 10 client.4656 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T04:24:33.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.564+0000 7fdc9904e740 20 client.4656 populate_metadata read hostname 'smithi100' 2024-09-17T04:24:33.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.564+0000 7fdc9904e740 1 --2- 172.21.15.100:0/2695844441 >> [v2:172.21.15.184:6835/2211103925,v1:172.21.15.184:6837/2211103925] conn(0x55de01bec020 0x55de01bec430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:33.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.564+0000 7fdc9904e740 4 client.4656 mds_command: new command op to 4368 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T04:24:33.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.564+0000 7fdc9904e740 1 -- 172.21.15.100:0/2695844441 --> [v2:172.21.15.184:6835/2211103925,v1:172.21.15.184:6837/2211103925] -- command(tid 0: {"prefix": "status"}) -- 0x55de01b3afd0 con 0x55de01bec020 2024-09-17T04:24:33.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.565+0000 7fdc8b400640 1 --2- 172.21.15.100:0/2695844441 >> [v2:172.21.15.184:6835/2211103925,v1:172.21.15.184:6837/2211103925] conn(0x55de01bec020 0x55de01bec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:33.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.565+0000 7fdc8b400640 1 --2- 172.21.15.100:0/2695844441 >> [v2:172.21.15.184:6835/2211103925,v1:172.21.15.184:6837/2211103925] conn(0x55de01bec020 0x55de01bec430 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbeb63b546148782 server_cookie=3b713af01fe4db4b in_seq=0 out_seq=0 2024-09-17T04:24:33.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.565+0000 7fdc6fe00640 10 client.4656 ms_handle_connect on v2:172.21.15.184:6835/2211103925 2024-09-17T04:24:33.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.566+0000 7fdc6fe00640 1 -- 172.21.15.100:0/2695844441 <== mds.0 v2:172.21.15.184:6835/2211103925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55de01b3afd0 con 0x55de01bec020 2024-09-17T04:24:33.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.566+0000 7fdc6fe00640 10 client.4656 handle_command_reply: tid=0 2024-09-17T04:24:33.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.566+0000 7fdc9904e740 1 client.4656 shutdown 2024-09-17T04:24:33.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.566+0000 7fdc9904e740 1 -- 172.21.15.100:0/2695844441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de01823270 msgr2=0x55de01c85200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:33.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.566+0000 7fdc9904e740 1 --2- 172.21.15.100:0/2695844441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de01823270 0x55de01c85200 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400a430 tx=0x7fdc8407c500 comp rx=0 tx=0).stop 2024-09-17T04:24:33.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.566+0000 7fdc9904e740 1 -- 172.21.15.100:0/2695844441 shutdown_connections 2024-09-17T04:24:33.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.566+0000 7fdc9904e740 1 --2- 172.21.15.100:0/2695844441 >> [v2:172.21.15.184:6835/2211103925,v1:172.21.15.184:6837/2211103925] conn(0x55de01bec020 0x55de01bec430 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.566+0000 7fdc9904e740 1 --2- 172.21.15.100:0/2695844441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de01823270 0x55de01c85200 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.566+0000 7fdc9904e740 1 --2- 172.21.15.100:0/2695844441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55de01802590 0x55de01c928c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.566+0000 7fdc9904e740 1 --2- 172.21.15.100:0/2695844441 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55de017fa3b0 0x55de01beb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.566+0000 7fdc9904e740 1 -- 172.21.15.100:0/2695844441 >> 172.21.15.100:0/2695844441 conn(0x55de018173c0 msgr2=0x55de01be9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:33.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.566+0000 7fdc6fe00640 0 client.4656 ms_handle_reset on v2:172.21.15.184:6835/2211103925 2024-09-17T04:24:33.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.566+0000 7fdc9904e740 1 -- 172.21.15.100:0/2695844441 shutdown_connections 2024-09-17T04:24:33.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.566+0000 7fdc9904e740 1 -- 172.21.15.100:0/2695844441 wait complete. 2024-09-17T04:24:33.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.566+0000 7fdc9904e740 20 client.4656 trim_cache size 0 max 16384 2024-09-17T04:24:33.571 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:24:33.571 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "8a399842-3a5c-4ea2-9afb-b2704332e59f", 2024-09-17T04:24:33.571 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 0, 2024-09-17T04:24:33.571 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 4368, 2024-09-17T04:24:33.571 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-09-17T04:24:33.571 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-09-17T04:24:33.571 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-09-17T04:24:33.572 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 18.537790743999999, 2024-09-17T04:24:33.572 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 8, 2024-09-17T04:24:33.572 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 23, 2024-09-17T04:24:33.572 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 23, 2024-09-17T04:24:33.572 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 24.387560266000001 2024-09-17T04:24:33.572 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:24:33.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.567+0000 7fdc92000640 1 -- 172.21.15.100:0/515168304 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdc68078bc0 msgr2=0x7fdc6807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:33.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.567+0000 7fdc92000640 1 --2- 172.21.15.100:0/515168304 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdc68078bc0 0x7fdc6807b0d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c1f4cb0 tx=0x7fdc74008040 comp rx=0 tx=0).stop 2024-09-17T04:24:33.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.567+0000 7fdc92000640 1 -- 172.21.15.100:0/515168304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc8c15ce30 msgr2=0x7fdc8c1f4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:33.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.567+0000 7fdc92000640 1 --2- 172.21.15.100:0/515168304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc8c15ce30 0x7fdc8c1f4330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400c730 tx=0x7fdc8400cbf0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.567+0000 7fdc92000640 1 -- 172.21.15.100:0/515168304 shutdown_connections 2024-09-17T04:24:33.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.567+0000 7fdc92000640 1 --2- 172.21.15.100:0/515168304 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fdc68078bc0 0x7fdc6807b0d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.567+0000 7fdc92000640 1 --2- 172.21.15.100:0/515168304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc8c15ce30 0x7fdc8c1f4330 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.567+0000 7fdc92000640 1 --2- 172.21.15.100:0/515168304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc8c0cfc80 0x7fdc8c1f3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.567+0000 7fdc92000640 1 --2- 172.21.15.100:0/515168304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc8c0705b0 0x7fdc8c15a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:33.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.567+0000 7fdc92000640 1 -- 172.21.15.100:0/515168304 >> 172.21.15.100:0/515168304 conn(0x7fdc8c06d3d0 msgr2=0x7fdc8c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:33.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.567+0000 7fdc92000640 1 -- 172.21.15.100:0/515168304 shutdown_connections 2024-09-17T04:24:33.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:33.567+0000 7fdc92000640 1 -- 172.21.15.100:0/515168304 wait complete. 2024-09-17T04:24:33.588 INFO:teuthology.run_tasks:Running task kclient... 2024-09-17T04:24:33.625 INFO:tasks.kclient:Mounting kernel clients... 2024-09-17T04:24:33.625 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-17T04:24:33.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:33.625 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-17T04:24:33.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:33.661 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-09-17T04:24:33.760 INFO:teuthology.orchestra.run.smithi100.stderr:Cannot find device "ceph-brx" 2024-09-17T04:24:33.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T04:24:33.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:33.763 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-17T04:24:33.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:33.798 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-09-17T04:24:33.896 INFO:teuthology.orchestra.run.smithi100.stderr:Cannot find device "ceph-brx" 2024-09-17T04:24:33.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T04:24:33.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:33.899 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-09-17T04:24:33.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:33.940 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link delete ceph-brx 2024-09-17T04:24:34.043 INFO:teuthology.orchestra.run.smithi184.stderr:Cannot find device "ceph-brx" 2024-09-17T04:24:34.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T04:24:34.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:34.046 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-09-17T04:24:34.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:34.082 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link delete ceph-brx 2024-09-17T04:24:34.181 INFO:teuthology.orchestra.run.smithi184.stderr:Cannot find device "ceph-brx" 2024-09-17T04:24:34.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T04:24:34.184 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-09-17T04:24:34.184 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T04:24:34.184 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T04:24:34.185 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:24:34.274 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:24:34.278 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:24:34.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.408+0000 7f3753200640 1 -- 172.21.15.100:0/3014484532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f374c0b3550 msgr2=0x7f374c0b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:34.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.408+0000 7f3753200640 1 --2- 172.21.15.100:0/3014484532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f374c0b3550 0x7f374c0b3960 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3744005fd0 tx=0x7f374402aa10 comp rx=0 tx=0).stop 2024-09-17T04:24:34.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.409+0000 7f3753200640 1 -- 172.21.15.100:0/3014484532 shutdown_connections 2024-09-17T04:24:34.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.409+0000 7f3753200640 1 --2- 172.21.15.100:0/3014484532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f374c0b3550 0x7f374c0b3960 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:34.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.409+0000 7f3753200640 1 --2- 172.21.15.100:0/3014484532 >> v2:172.21.15.184:3300/0 conn(0x7f374c15d760 0x7f374c1535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:34.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.409+0000 7f3753200640 1 -- 172.21.15.100:0/3014484532 >> 172.21.15.100:0/3014484532 conn(0x7f374c06d730 msgr2=0x7f374c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:34.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.409+0000 7f3753200640 1 -- 172.21.15.100:0/3014484532 shutdown_connections 2024-09-17T04:24:34.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.409+0000 7f3753200640 1 -- 172.21.15.100:0/3014484532 wait complete. 2024-09-17T04:24:34.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.411+0000 7f3753200640 1 Processor -- start 2024-09-17T04:24:34.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.411+0000 7f3753200640 1 -- start start 2024-09-17T04:24:34.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.413+0000 7f3753200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f374c0ad0e0 0x7f374c14aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:34.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.413+0000 7f3753200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f374c0b3550 0x7f374c148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:34.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.413+0000 7f3753200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f374c15d760 0x7f374c0741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:34.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.413+0000 7f3753200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f374c06f240 con 0x7f374c0b3550 2024-09-17T04:24:34.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.413+0000 7f3753200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f374c0b4e30 con 0x7f374c0ad0e0 2024-09-17T04:24:34.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.413+0000 7f3753200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f374c0b3ea0 con 0x7f374c15d760 2024-09-17T04:24:34.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.413+0000 7f374b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f374c0b3550 0x7f374c148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:34.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.413+0000 7f374b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f374c0b3550 0x7f374c148aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48364/0 (socket says 172.21.15.100:48364) 2024-09-17T04:24:34.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.413+0000 7f374b400640 1 -- 172.21.15.100:0/3837527481 learned_addr learned my addr 172.21.15.100:0/3837527481 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:34.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.413+0000 7f3751200640 1 --2- 172.21.15.100:0/3837527481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f374c15d760 0x7f374c0741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:34.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.413+0000 7f374be00640 1 --2- 172.21.15.100:0/3837527481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f374c0ad0e0 0x7f374c14aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:34.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.413+0000 7f374b400640 1 -- 172.21.15.100:0/3837527481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f374c15d760 msgr2=0x7f374c0741e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:34.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.413+0000 7f374b400640 1 --2- 172.21.15.100:0/3837527481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f374c15d760 0x7f374c0741e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:34.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.413+0000 7f374b400640 1 -- 172.21.15.100:0/3837527481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f374c0ad0e0 msgr2=0x7f374c14aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:34.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.413+0000 7f374b400640 1 --2- 172.21.15.100:0/3837527481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f374c0ad0e0 0x7f374c14aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:34.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.413+0000 7f374b400640 1 -- 172.21.15.100:0/3837527481 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374c073420 con 0x7f374c0b3550 2024-09-17T04:24:34.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.414+0000 7f374b400640 1 --2- 172.21.15.100:0/3837527481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f374c0b3550 0x7f374c148aa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f373800c3f0 tx=0x7f373800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:34.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.414+0000 7f3743e00640 1 -- 172.21.15.100:0/3837527481 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3738017070 con 0x7f374c0b3550 2024-09-17T04:24:34.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.414+0000 7f3743e00640 1 -- 172.21.15.100:0/3837527481 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37380053c0 con 0x7f374c0b3550 2024-09-17T04:24:34.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.414+0000 7f3753200640 1 -- 172.21.15.100:0/3837527481 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f374c073700 con 0x7f374c0b3550 2024-09-17T04:24:34.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.414+0000 7f3743e00640 1 -- 172.21.15.100:0/3837527481 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3738003850 con 0x7f374c0b3550 2024-09-17T04:24:34.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.414+0000 7f3753200640 1 -- 172.21.15.100:0/3837527481 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f374c073cc0 con 0x7f374c0b3550 2024-09-17T04:24:34.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.415+0000 7f3753200640 1 -- 172.21.15.100:0/3837527481 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f374c1f8c60 con 0x7f374c0b3550 2024-09-17T04:24:34.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.416+0000 7f3743e00640 1 -- 172.21.15.100:0/3837527481 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f373801a070 con 0x7f374c0b3550 2024-09-17T04:24:34.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.417+0000 7f3743e00640 1 --2- 172.21.15.100:0/3837527481 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f372c078b90 0x7f372c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:34.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.417+0000 7f3743e00640 1 -- 172.21.15.100:0/3837527481 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f3738099ff0 con 0x7f374c0b3550 2024-09-17T04:24:34.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.417+0000 7f374be00640 1 --2- 172.21.15.100:0/3837527481 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f372c078b90 0x7f372c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:34.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.418+0000 7f374be00640 1 --2- 172.21.15.100:0/3837527481 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f372c078b90 0x7f372c07b0a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f373c0095f0 tx=0x7f373c005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:34.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.421+0000 7f3743e00640 1 -- 172.21.15.100:0/3837527481 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3738061e30 con 0x7f374c0b3550 2024-09-17T04:24:34.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.639+0000 7f3753200640 1 -- 172.21.15.100:0/3837527481 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f374c0b42f0 con 0x7f374c0b3550 2024-09-17T04:24:34.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.639+0000 7f3743e00640 1 -- 172.21.15.100:0/3837527481 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f3738066860 con 0x7f374c0b3550 2024-09-17T04:24:34.640 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:24:34.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.642+0000 7f3753200640 1 -- 172.21.15.100:0/3837527481 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f372c078b90 msgr2=0x7f372c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:34.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.642+0000 7f3753200640 1 --2- 172.21.15.100:0/3837527481 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f372c078b90 0x7f372c07b0a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f373c0095f0 tx=0x7f373c005f90 comp rx=0 tx=0).stop 2024-09-17T04:24:34.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.642+0000 7f3753200640 1 -- 172.21.15.100:0/3837527481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f374c0b3550 msgr2=0x7f374c148aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:34.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.642+0000 7f3753200640 1 --2- 172.21.15.100:0/3837527481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f374c0b3550 0x7f374c148aa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f373800c3f0 tx=0x7f373800c8b0 comp rx=0 tx=0).stop 2024-09-17T04:24:34.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.642+0000 7f3753200640 1 -- 172.21.15.100:0/3837527481 shutdown_connections 2024-09-17T04:24:34.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.642+0000 7f3753200640 1 --2- 172.21.15.100:0/3837527481 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f372c078b90 0x7f372c07b0a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:34.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.642+0000 7f3753200640 1 --2- 172.21.15.100:0/3837527481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f374c15d760 0x7f374c0741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:34.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.642+0000 7f3753200640 1 --2- 172.21.15.100:0/3837527481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f374c0b3550 0x7f374c148aa0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:34.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.642+0000 7f3753200640 1 --2- 172.21.15.100:0/3837527481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f374c0ad0e0 0x7f374c14aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:34.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.642+0000 7f3753200640 1 -- 172.21.15.100:0/3837527481 >> 172.21.15.100:0/3837527481 conn(0x7f374c06d730 msgr2=0x7f374c068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:34.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.642+0000 7f3753200640 1 -- 172.21.15.100:0/3837527481 shutdown_connections 2024-09-17T04:24:34.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34.642+0000 7f3753200640 1 -- 172.21.15.100:0/3837527481 wait complete. 2024-09-17T04:24:34.664 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T04:24:34.664 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:24:34.664 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-09-17T04:24:34.664 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T04:24:34.664 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T04:24:34.664 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T04:24:34.664 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T04:24:34.664 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:24:34.664 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:24:34.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:34.665 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-09-17T04:24:34.700 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:24:34.701 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:24:34.701 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:24:34.701 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:34.701 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-17T04:24:34.701 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:34.701 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:24:34.701 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:24:34.701 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-09-17T04:24:34.701 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:24:34.701 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:24:34.701 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-09-17T04:24:34.701 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:24:34.701 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:24:34.702 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:24:34.702 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:24:34.702 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T04:24:34.702 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42686sec preferred_lft 42686sec 2024-09-17T04:24:34.702 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link noprefixroute 2024-09-17T04:24:34.702 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:34.702 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-17T04:24:34.702 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:24:34.702 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-17T04:24:34.703 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add name ceph-brx type bridge 2024-09-17T04:24:34.703 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr flush dev ceph-brx 2024-09-17T04:24:34.703 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set ceph-brx up 2024-09-17T04:24:34.703 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-17T04:24:34.703 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-17T04:24:34.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:24:35.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:35.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:35.112 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:24:35.168 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-09-17T04:24:35.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:35.172 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-09-17T04:24:35.239 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-09-17T04:24:35.239 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:24:35.239 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T04:24:35.239 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T04:24:35.239 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:24:35.240 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:24:35.240 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-17T04:24:35.240 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T04:24:35.240 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T04:24:35.240 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T04:24:35.240 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-17T04:24:35.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:24:35.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:35.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:35.857 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-17T04:24:35.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:35.893 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list-id 2024-09-17T04:24:35.970 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:24:35.970 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-17T04:24:35.970 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:24:35.970 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-17T04:24:35.971 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-17T04:24:36.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:24:36.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:36.148 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-17T04:24:36.149 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:24:36.149 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-17T04:24:36.149 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-17T04:24:36.149 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-17T04:24:36.149 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-17T04:24:36.149 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-17T04:24:36.149 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-17T04:24:36.149 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-17T04:24:36.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:24:36.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:36.397 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:24:36.397 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-17T04:24:36.397 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-09-17T04:24:36.397 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set dev brx.0 master ceph-brx 2024-09-17T04:24:36.398 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-17T04:24:36.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:24:36.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:36.562 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:24:36.563 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T04:24:36.598 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:24:36.598 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:24:36.598 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:24:36.673 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T04:24:36.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:36.673 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:24:36.881 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:24:36.881 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:24:36.881 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:24:37.499 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:24:37.499 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:24:37.499 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:24:37.499 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:24:37.499 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:24:37.534 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:24:37.534 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:24:37.534 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:24:37.534 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:24:37.534 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:24:37.535 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-09-17T04:24:37.535 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-09-17T04:24:37.535 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8a399842-3a5c-4ea2-9afb-b2704332e59f" will pass to kernel 2024-09-17T04:24:37.536 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-17T04:24:37.536 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-09-17T04:24:37.536 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-17T04:24:37.536 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-09-17T04:24:37.536 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-17T04:24:37.539 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T04:24:37.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:37.539 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:24:37.597 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:24:37.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:24:37.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:37.701 INFO:teuthology.orchestra.run.smithi100.stdout:client4714 2024-09-17T04:24:37.702 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:24:37.702 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:24:37.737 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:24:37.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:24:37.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:37.875 INFO:teuthology.orchestra.run.smithi100.stdout:client4714 2024-09-17T04:24:37.876 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:24:37.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:24:37.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:37.975 INFO:teuthology.orchestra.run.smithi100.stdout:client4714 2024-09-17T04:24:37.976 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/8a399842-3a5c-4ea2-9afb-b2704332e59f.client4714/status') 2024-09-17T04:24:38.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:24:38.084 INFO:teuthology.orchestra.run.smithi100.stdout:instance: client.4714 (3)172.21.15.100:0/3963260989 2024-09-17T04:24:38.084 INFO:teuthology.orchestra.run.smithi100.stdout:blocklisted: false 2024-09-17T04:24:38.085 INFO:teuthology.orchestra.run.smithi100.stderr:0+1 records in 2024-09-17T04:24:38.085 INFO:teuthology.orchestra.run.smithi100.stderr:0+1 records out 2024-09-17T04:24:38.085 INFO:teuthology.orchestra.run.smithi100.stderr:71 bytes copied, 6.9339e-05 s, 1.0 MB/s 2024-09-17T04:24:38.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:38.095 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-09-17T04:24:38.095 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T04:24:38.095 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-17T04:24:38.096 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:24:38.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.318+0000 7efdf2a00640 1 -- 172.21.15.100:0/3334901248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdd0005350 msgr2=0x7efdd000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:38.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.318+0000 7efdf2a00640 1 --2- 172.21.15.100:0/3334901248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdd0005350 0x7efdd000b9c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efde4001d90 tx=0x7efdd0005830 comp rx=0 tx=0).stop 2024-09-17T04:24:38.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.318+0000 7efdf2a00640 1 -- 172.21.15.100:0/3334901248 shutdown_connections 2024-09-17T04:24:38.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.318+0000 7efdf2a00640 1 --2- 172.21.15.100:0/3334901248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdd0005350 0x7efdd000b9c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:38.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.318+0000 7efdf2a00640 1 -- 172.21.15.100:0/3334901248 >> 172.21.15.100:0/3334901248 conn(0x7efdec06d730 msgr2=0x7efdec0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:38.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.318+0000 7efdf2a00640 1 -- 172.21.15.100:0/3334901248 shutdown_connections 2024-09-17T04:24:38.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.318+0000 7efdf2a00640 1 -- 172.21.15.100:0/3334901248 wait complete. 2024-09-17T04:24:38.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.321+0000 7efdf2a00640 1 Processor -- start 2024-09-17T04:24:38.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.321+0000 7efdf2a00640 1 -- start start 2024-09-17T04:24:38.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.322+0000 7efdf2a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdd0005350 0x7efdec159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:38.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.322+0000 7efdf2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdec157730 0x7efdec1f7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:38.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.322+0000 7efdf2a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdec1f30b0 0x7efdec1e5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:38.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.322+0000 7efdf2a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efdec0712b0 con 0x7efdec157730 2024-09-17T04:24:38.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.322+0000 7efdf2a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efdec071130 con 0x7efdd0005350 2024-09-17T04:24:38.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.322+0000 7efdf2a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efdec0ad7d0 con 0x7efdec1f30b0 2024-09-17T04:24:38.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.322+0000 7efdeb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdec157730 0x7efdec1f7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:38.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.322+0000 7efdeb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdec157730 0x7efdec1f7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36998/0 (socket says 172.21.15.100:36998) 2024-09-17T04:24:38.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.322+0000 7efdeb400640 1 -- 172.21.15.100:0/1919792949 learned_addr learned my addr 172.21.15.100:0/1919792949 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:38.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.323+0000 7efdf0a00640 1 --2- 172.21.15.100:0/1919792949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdec1f30b0 0x7efdec1e5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:38.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.323+0000 7efdebe00640 1 --2- 172.21.15.100:0/1919792949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdd0005350 0x7efdec159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:38.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.323+0000 7efdeb400640 1 -- 172.21.15.100:0/1919792949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdec1f30b0 msgr2=0x7efdec1e5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:38.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.323+0000 7efdeb400640 1 --2- 172.21.15.100:0/1919792949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdec1f30b0 0x7efdec1e5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:38.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.323+0000 7efdeb400640 1 -- 172.21.15.100:0/1919792949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdd0005350 msgr2=0x7efdec159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:38.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.323+0000 7efdeb400640 1 --2- 172.21.15.100:0/1919792949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdd0005350 0x7efdec159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:38.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.323+0000 7efdeb400640 1 -- 172.21.15.100:0/1919792949 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdec15a370 con 0x7efdec157730 2024-09-17T04:24:38.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.323+0000 7efdf0a00640 1 --2- 172.21.15.100:0/1919792949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdec1f30b0 0x7efdec1e5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:24:38.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.323+0000 7efdebe00640 1 --2- 172.21.15.100:0/1919792949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdd0005350 0x7efdec159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:24:38.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.323+0000 7efdeb400640 1 --2- 172.21.15.100:0/1919792949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdec157730 0x7efdec1f7700 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efdd800ea40 tx=0x7efdd800ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:38.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.323+0000 7efde3e00640 1 -- 172.21.15.100:0/1919792949 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd8004320 con 0x7efdec157730 2024-09-17T04:24:38.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.323+0000 7efde3e00640 1 -- 172.21.15.100:0/1919792949 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efdd80044c0 con 0x7efdec157730 2024-09-17T04:24:38.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.323+0000 7efde3e00640 1 -- 172.21.15.100:0/1919792949 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd800dc20 con 0x7efdec157730 2024-09-17T04:24:38.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.323+0000 7efdf2a00640 1 -- 172.21.15.100:0/1919792949 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdec1e49a0 con 0x7efdec157730 2024-09-17T04:24:38.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.324+0000 7efdf2a00640 1 -- 172.21.15.100:0/1919792949 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdec1e4f60 con 0x7efdec157730 2024-09-17T04:24:38.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.325+0000 7efdf2a00640 1 -- 172.21.15.100:0/1919792949 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdec0741c0 con 0x7efdec157730 2024-09-17T04:24:38.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.326+0000 7efde3e00640 1 -- 172.21.15.100:0/1919792949 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efdd80035f0 con 0x7efdec157730 2024-09-17T04:24:38.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.327+0000 7efde3e00640 1 --2- 172.21.15.100:0/1919792949 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7efdcc078b90 0x7efdcc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:38.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.327+0000 7efde3e00640 1 -- 172.21.15.100:0/1919792949 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7efdd809b940 con 0x7efdec157730 2024-09-17T04:24:38.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.328+0000 7efdebe00640 1 --2- 172.21.15.100:0/1919792949 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7efdcc078b90 0x7efdcc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:38.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.328+0000 7efdebe00640 1 --2- 172.21.15.100:0/1919792949 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7efdcc078b90 0x7efdcc07b0a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efddc000fb0 tx=0x7efddc0067a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:38.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.332+0000 7efde3e00640 1 -- 172.21.15.100:0/1919792949 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7efdd8063d70 con 0x7efdec157730 2024-09-17T04:24:38.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.548+0000 7efdf2a00640 1 -- 172.21.15.100:0/1919792949 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efdec15a100 con 0x7efdec157730 2024-09-17T04:24:38.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.549+0000 7efde3e00640 1 -- 172.21.15.100:0/1919792949 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7efdd8063670 con 0x7efdec157730 2024-09-17T04:24:38.549 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:24:38.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.551+0000 7efdf2a00640 1 -- 172.21.15.100:0/1919792949 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7efdcc078b90 msgr2=0x7efdcc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:38.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.551+0000 7efdf2a00640 1 --2- 172.21.15.100:0/1919792949 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7efdcc078b90 0x7efdcc07b0a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efddc000fb0 tx=0x7efddc0067a0 comp rx=0 tx=0).stop 2024-09-17T04:24:38.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.551+0000 7efdf2a00640 1 -- 172.21.15.100:0/1919792949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdec157730 msgr2=0x7efdec1f7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:38.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.551+0000 7efdf2a00640 1 --2- 172.21.15.100:0/1919792949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdec157730 0x7efdec1f7700 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efdd800ea40 tx=0x7efdd800ef00 comp rx=0 tx=0).stop 2024-09-17T04:24:38.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.551+0000 7efdf2a00640 1 -- 172.21.15.100:0/1919792949 shutdown_connections 2024-09-17T04:24:38.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.551+0000 7efdf2a00640 1 --2- 172.21.15.100:0/1919792949 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7efdcc078b90 0x7efdcc07b0a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:38.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.551+0000 7efdf2a00640 1 --2- 172.21.15.100:0/1919792949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdec1f30b0 0x7efdec1e5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:38.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.551+0000 7efdf2a00640 1 --2- 172.21.15.100:0/1919792949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdec157730 0x7efdec1f7700 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:38.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.551+0000 7efdf2a00640 1 --2- 172.21.15.100:0/1919792949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdd0005350 0x7efdec159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:38.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.551+0000 7efdf2a00640 1 -- 172.21.15.100:0/1919792949 >> 172.21.15.100:0/1919792949 conn(0x7efdec06d730 msgr2=0x7efdec0b1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:38.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.551+0000 7efdf2a00640 1 -- 172.21.15.100:0/1919792949 shutdown_connections 2024-09-17T04:24:38.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38.551+0000 7efdf2a00640 1 -- 172.21.15.100:0/1919792949 wait complete. 2024-09-17T04:24:38.575 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T04:24:38.575 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:24:38.575 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-09-17T04:24:38.575 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-17T04:24:38.575 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-17T04:24:38.575 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T04:24:38.575 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T04:24:38.575 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:24:38.575 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-17T04:24:38.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:38.576 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-09-17T04:24:38.610 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:24:38.610 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:24:38.611 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:24:38.611 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:38.611 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-17T04:24:38.611 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:38.611 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:24:38.611 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:24:38.611 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-09-17T04:24:38.611 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:24:38.611 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:24:38.611 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-09-17T04:24:38.611 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:24:38.611 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:24:38.611 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:24:38.611 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:24:38.612 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T04:24:38.612 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42682sec preferred_lft 42682sec 2024-09-17T04:24:38.612 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link noprefixroute 2024-09-17T04:24:38.612 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:38.612 INFO:teuthology.orchestra.run.smithi100.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:24:38.612 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether ea:8c:45:9f:34:87 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:24:38.612 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T04:24:38.612 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:38.612 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::cc78:e4ff:fe70:7fbe/64 scope link 2024-09-17T04:24:38.612 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:38.612 INFO:teuthology.orchestra.run.smithi100.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:24:38.612 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether ea:8c:45:9f:34:87 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:24:38.612 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::e88c:45ff:fe9f:3487/64 scope link 2024-09-17T04:24:38.613 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:38.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:38.614 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:24:38.708 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-09-17T04:24:38.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:38.711 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-09-17T04:24:38.747 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-09-17T04:24:38.747 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:24:38.747 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T04:24:38.747 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T04:24:38.747 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:24:38.748 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:24:38.748 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-17T04:24:38.748 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T04:24:38.748 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T04:24:38.748 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T04:24:38.748 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-17T04:24:38.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:24:38.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:38.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:38.942 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-17T04:24:38.977 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T04:24:38.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:38.977 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list-id 2024-09-17T04:24:39.055 INFO:teuthology.orchestra.run.smithi100.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-17T04:24:39.055 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:24:39.055 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-17T04:24:39.055 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:24:39.055 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-17T04:24:39.055 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-17T04:24:39.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:24:39.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:39.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:39.233 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-17T04:24:39.288 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:24:39.288 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:24:39.288 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:24:39.288 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:39.288 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-17T04:24:39.288 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:39.288 INFO:teuthology.orchestra.run.smithi100.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:24:39.288 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 86:dd:6b:db:c5:01 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T04:24:39.288 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T04:24:39.289 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:39.289 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::84dd:6bff:fedb:c501/64 scope link 2024-09-17T04:24:39.289 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:39.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:39.291 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-17T04:24:39.346 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:24:39.347 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:24:39.347 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:24:39.347 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:39.347 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-17T04:24:39.347 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:39.347 INFO:teuthology.orchestra.run.smithi100.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:24:39.347 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 86:dd:6b:db:c5:01 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T04:24:39.347 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T04:24:39.347 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:39.347 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::84dd:6bff:fedb:c501/64 scope link 2024-09-17T04:24:39.347 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:39.350 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-17T04:24:39.350 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:24:39.350 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-17T04:24:39.350 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-17T04:24:39.350 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-17T04:24:39.350 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-17T04:24:39.350 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-17T04:24:39.350 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-17T04:24:39.350 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-17T04:24:39.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:24:39.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:39.564 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:24:39.564 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-17T04:24:39.564 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.1 up 2024-09-17T04:24:39.565 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set dev brx.1 master ceph-brx 2024-09-17T04:24:39.565 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-17T04:24:39.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:24:39.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:39.730 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:24:39.730 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-17T04:24:39.765 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-17T04:24:39.801 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:24:39.802 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-17T04:24:39.840 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T04:24:39.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:39.841 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-17T04:24:40.037 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-17T04:24:40.037 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-17T04:24:40.037 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:24:40.058 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-17T04:24:40.058 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-17T04:24:40.058 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:24:40.058 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:24:40.058 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:24:40.088 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-17T04:24:40.088 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-17T04:24:40.088 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:24:40.088 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:24:40.088 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:24:40.088 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-09-17T04:24:40.088 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-09-17T04:24:40.089 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=8a399842-3a5c-4ea2-9afb-b2704332e59f" will pass to kernel 2024-09-17T04:24:40.090 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-09-17T04:24:40.090 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-09-17T04:24:40.090 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-17T04:24:40.090 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-09-17T04:24:40.090 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-17T04:24:40.093 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T04:24:40.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:40.093 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-17T04:24:40.148 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:24:40.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:24:40.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:40.238 INFO:teuthology.orchestra.run.smithi100.stdout:client4668 2024-09-17T04:24:40.238 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:24:40.238 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:24:40.273 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:24:40.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:24:40.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:40.402 INFO:teuthology.orchestra.run.smithi100.stdout:client4668 2024-09-17T04:24:40.403 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:24:40.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:24:40.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:40.492 INFO:teuthology.orchestra.run.smithi100.stdout:client4668 2024-09-17T04:24:40.492 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/8a399842-3a5c-4ea2-9afb-b2704332e59f.client4668/status') 2024-09-17T04:24:40.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:24:40.590 INFO:teuthology.orchestra.run.smithi100.stdout:instance: client.4668 (3)192.168.0.2:0/429833357 2024-09-17T04:24:40.591 INFO:teuthology.orchestra.run.smithi100.stdout:blocklisted: false 2024-09-17T04:24:40.591 INFO:teuthology.orchestra.run.smithi100.stderr:0+1 records in 2024-09-17T04:24:40.591 INFO:teuthology.orchestra.run.smithi100.stderr:0+1 records out 2024-09-17T04:24:40.591 INFO:teuthology.orchestra.run.smithi100.stderr:68 bytes copied, 6.4158e-05 s, 1.1 MB/s 2024-09-17T04:24:40.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:40.601 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-09-17T04:24:40.601 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T04:24:40.601 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T04:24:40.602 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:24:40.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.821+0000 7fd56d600640 1 -- 172.21.15.100:0/3693336111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5680ad0e0 msgr2=0x7fd5680ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:40.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.821+0000 7fd56d600640 1 --2- 172.21.15.100:0/3693336111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5680ad0e0 0x7fd5680ad4f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd554009830 tx=0x7fd55402a530 comp rx=0 tx=0).stop 2024-09-17T04:24:40.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.821+0000 7fd56d600640 1 -- 172.21.15.100:0/3693336111 shutdown_connections 2024-09-17T04:24:40.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.821+0000 7fd56d600640 1 --2- 172.21.15.100:0/3693336111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5680ad0e0 0x7fd5680ad4f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:40.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.821+0000 7fd56d600640 1 -- 172.21.15.100:0/3693336111 >> 172.21.15.100:0/3693336111 conn(0x7fd56806d730 msgr2=0x7fd5680b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:40.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.822+0000 7fd56d600640 1 -- 172.21.15.100:0/3693336111 shutdown_connections 2024-09-17T04:24:40.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.822+0000 7fd56d600640 1 -- 172.21.15.100:0/3693336111 wait complete. 2024-09-17T04:24:40.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.824+0000 7fd56d600640 1 Processor -- start 2024-09-17T04:24:40.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.824+0000 7fd56d600640 1 -- start start 2024-09-17T04:24:40.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.825+0000 7fd56d600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5680ad0e0 0x7fd5681f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:40.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.825+0000 7fd56d600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5680b3550 0x7fd5681f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:40.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.825+0000 7fd56d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd56815d760 0x7fd568156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:40.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.825+0000 7fd56d600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd5680b4da0 con 0x7fd56815d760 2024-09-17T04:24:40.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.825+0000 7fd56d600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd5680b4c20 con 0x7fd5680ad0e0 2024-09-17T04:24:40.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.825+0000 7fd56d600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd5680b4070 con 0x7fd5680b3550 2024-09-17T04:24:40.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.826+0000 7fd567400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd56815d760 0x7fd568156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:40.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.826+0000 7fd567400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd56815d760 0x7fd568156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37010/0 (socket says 172.21.15.100:37010) 2024-09-17T04:24:40.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.826+0000 7fd567400640 1 -- 172.21.15.100:0/3849635392 learned_addr learned my addr 172.21.15.100:0/3849635392 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:40.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.826+0000 7fd566000640 1 --2- 172.21.15.100:0/3849635392 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5680b3550 0x7fd5681f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:40.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.826+0000 7fd566a00640 1 --2- 172.21.15.100:0/3849635392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5680ad0e0 0x7fd5681f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:40.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.826+0000 7fd567400640 1 -- 172.21.15.100:0/3849635392 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5680b3550 msgr2=0x7fd5681f3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:40.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.826+0000 7fd567400640 1 --2- 172.21.15.100:0/3849635392 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5680b3550 0x7fd5681f3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:40.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.826+0000 7fd567400640 1 -- 172.21.15.100:0/3849635392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5680ad0e0 msgr2=0x7fd5681f62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:40.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.826+0000 7fd567400640 1 --2- 172.21.15.100:0/3849635392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5680ad0e0 0x7fd5681f62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:40.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.826+0000 7fd567400640 1 -- 172.21.15.100:0/3849635392 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd56809a790 con 0x7fd56815d760 2024-09-17T04:24:40.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.826+0000 7fd566a00640 1 --2- 172.21.15.100:0/3849635392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5680ad0e0 0x7fd5681f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:24:40.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.826+0000 7fd567400640 1 --2- 172.21.15.100:0/3849635392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd56815d760 0x7fd568156010 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd56000c700 tx=0x7fd56000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:40.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.827+0000 7fd55ea00640 1 -- 172.21.15.100:0/3849635392 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd560003b70 con 0x7fd56815d760 2024-09-17T04:24:40.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.827+0000 7fd55ea00640 1 -- 172.21.15.100:0/3849635392 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd560003d10 con 0x7fd56815d760 2024-09-17T04:24:40.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.827+0000 7fd55ea00640 1 -- 172.21.15.100:0/3849635392 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56000aab0 con 0x7fd56815d760 2024-09-17T04:24:40.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.827+0000 7fd56d600640 1 -- 172.21.15.100:0/3849635392 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd56809aa70 con 0x7fd56815d760 2024-09-17T04:24:40.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.827+0000 7fd56d600640 1 -- 172.21.15.100:0/3849635392 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd56809b030 con 0x7fd56815d760 2024-09-17T04:24:40.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.828+0000 7fd56d600640 1 -- 172.21.15.100:0/3849635392 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd56809a500 con 0x7fd56815d760 2024-09-17T04:24:40.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.829+0000 7fd55ea00640 1 -- 172.21.15.100:0/3849635392 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd56000ac50 con 0x7fd56815d760 2024-09-17T04:24:40.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.830+0000 7fd55ea00640 1 --2- 172.21.15.100:0/3849635392 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd548078b90 0x7fd54807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:40.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.830+0000 7fd55ea00640 1 -- 172.21.15.100:0/3849635392 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd56009aa40 con 0x7fd56815d760 2024-09-17T04:24:40.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.831+0000 7fd566a00640 1 --2- 172.21.15.100:0/3849635392 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd548078b90 0x7fd54807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:40.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.831+0000 7fd566a00640 1 --2- 172.21.15.100:0/3849635392 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd548078b90 0x7fd54807b0a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd5580013f0 tx=0x7fd558009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:40.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:40.835+0000 7fd55ea00640 1 -- 172.21.15.100:0/3849635392 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd560062f00 con 0x7fd56815d760 2024-09-17T04:24:41.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:41.055+0000 7fd56d600640 1 -- 172.21.15.100:0/3849635392 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd5680ad4f0 con 0x7fd56815d760 2024-09-17T04:24:41.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:41.055+0000 7fd55ea00640 1 -- 172.21.15.100:0/3849635392 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fd560062800 con 0x7fd56815d760 2024-09-17T04:24:41.056 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:24:41.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:41.057+0000 7fd56d600640 1 -- 172.21.15.100:0/3849635392 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd548078b90 msgr2=0x7fd54807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:41.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:41.057+0000 7fd56d600640 1 --2- 172.21.15.100:0/3849635392 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd548078b90 0x7fd54807b0a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd5580013f0 tx=0x7fd558009040 comp rx=0 tx=0).stop 2024-09-17T04:24:41.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:41.057+0000 7fd56d600640 1 -- 172.21.15.100:0/3849635392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd56815d760 msgr2=0x7fd568156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:41.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:41.057+0000 7fd56d600640 1 --2- 172.21.15.100:0/3849635392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd56815d760 0x7fd568156010 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd56000c700 tx=0x7fd56000cbc0 comp rx=0 tx=0).stop 2024-09-17T04:24:41.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:41.057+0000 7fd56d600640 1 -- 172.21.15.100:0/3849635392 shutdown_connections 2024-09-17T04:24:41.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:41.057+0000 7fd56d600640 1 --2- 172.21.15.100:0/3849635392 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd548078b90 0x7fd54807b0a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:41.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:41.057+0000 7fd56d600640 1 --2- 172.21.15.100:0/3849635392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd56815d760 0x7fd568156010 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:41.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:41.057+0000 7fd56d600640 1 --2- 172.21.15.100:0/3849635392 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5680b3550 0x7fd5681f3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:41.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:41.057+0000 7fd56d600640 1 --2- 172.21.15.100:0/3849635392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5680ad0e0 0x7fd5681f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:41.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:41.057+0000 7fd56d600640 1 -- 172.21.15.100:0/3849635392 >> 172.21.15.100:0/3849635392 conn(0x7fd56806d730 msgr2=0x7fd5680b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:41.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:41.058+0000 7fd56d600640 1 -- 172.21.15.100:0/3849635392 shutdown_connections 2024-09-17T04:24:41.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:41.058+0000 7fd56d600640 1 -- 172.21.15.100:0/3849635392 wait complete. 2024-09-17T04:24:41.079 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T04:24:41.079 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:24:41.080 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2024-09-17T04:24:41.080 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T04:24:41.080 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T04:24:41.080 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T04:24:41.080 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T04:24:41.080 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:24:41.080 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:24:41.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:41.081 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2024-09-17T04:24:41.120 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:24:41.121 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:24:41.121 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:24:41.121 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:41.121 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-09-17T04:24:41.121 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:41.121 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:24:41.121 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:24:41.121 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2024-09-17T04:24:41.121 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:24:41.121 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:24:41.121 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2024-09-17T04:24:41.122 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:24:41.122 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2024-09-17T04:24:41.122 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:24:41.122 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2024-09-17T04:24:41.122 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T04:24:41.122 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42663sec preferred_lft 42663sec 2024-09-17T04:24:41.122 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ec4:7aff:fe88:7b8f/64 scope link noprefixroute 2024-09-17T04:24:41.122 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:41.122 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-17T04:24:41.123 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:24:41.123 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-09-17T04:24:41.123 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link add name ceph-brx type bridge 2024-09-17T04:24:41.123 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip addr flush dev ceph-brx 2024-09-17T04:24:41.123 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set ceph-brx up 2024-09-17T04:24:41.123 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-17T04:24:41.123 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-09-17T04:24:41.243 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:24:41.538 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:41.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:41.547 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:24:41.604 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-09-17T04:24:41.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:41.607 DEBUG:teuthology.orchestra.run.smithi184:> route 2024-09-17T04:24:41.654 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2024-09-17T04:24:41.654 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:24:41.654 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T04:24:41.654 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T04:24:41.654 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:24:41.655 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:24:41.655 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-09-17T04:24:41.655 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T04:24:41.655 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T04:24:41.655 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T04:24:41.655 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-09-17T04:24:41.732 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:24:42.220 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:42.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:42.228 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-09-17T04:24:42.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:42.264 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list-id 2024-09-17T04:24:42.339 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:24:42.339 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-09-17T04:24:42.339 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T04:24:42.339 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-17T04:24:42.340 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-09-17T04:24:42.457 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:24:42.507 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:42.516 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-17T04:24:42.516 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:24:42.516 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-09-17T04:24:42.516 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-17T04:24:42.516 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-17T04:24:42.516 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-17T04:24:42.516 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-17T04:24:42.516 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-17T04:24:42.517 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-09-17T04:24:42.593 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:24:42.745 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:42.753 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:24:42.754 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-09-17T04:24:42.754 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.0 up 2024-09-17T04:24:42.754 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set dev brx.0 master ceph-brx 2024-09-17T04:24:42.754 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-09-17T04:24:42.831 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:24:42.910 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:42.918 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:24:42.918 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T04:24:42.953 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:24:42.954 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:24:42.954 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T04:24:43.030 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T04:24:43.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:43.031 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:24:43.261 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:24:43.261 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:24:43.261 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:24:43.880 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:24:43.880 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:24:43.880 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:24:43.880 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:24:43.880 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:24:43.916 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:24:43.916 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:24:43.916 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:24:43.916 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:24:43.916 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:24:43.916 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-09-17T04:24:43.916 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-09-17T04:24:43.916 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8a399842-3a5c-4ea2-9afb-b2704332e59f" will pass to kernel 2024-09-17T04:24:43.918 INFO:teuthology.orchestra.run.smithi184.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-17T04:24:43.918 INFO:teuthology.orchestra.run.smithi184.stdout: You just mounted a file system that supports labels which does not 2024-09-17T04:24:43.918 INFO:teuthology.orchestra.run.smithi184.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-17T04:24:43.918 INFO:teuthology.orchestra.run.smithi184.stdout: applications will generate AVC messages and not be allowed access to 2024-09-17T04:24:43.918 INFO:teuthology.orchestra.run.smithi184.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-17T04:24:43.921 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T04:24:43.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:43.921 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T04:24:43.977 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:24:44.060 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:24:44.064 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:44.072 INFO:teuthology.orchestra.run.smithi184.stdout:client4680 2024-09-17T04:24:44.072 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:24:44.072 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:24:44.108 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:24:44.226 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:24:44.229 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:44.237 INFO:teuthology.orchestra.run.smithi184.stdout:client4680 2024-09-17T04:24:44.238 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:24:44.315 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:24:44.318 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:44.326 INFO:teuthology.orchestra.run.smithi184.stdout:client4680 2024-09-17T04:24:44.326 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/8a399842-3a5c-4ea2-9afb-b2704332e59f.client4680/status') 2024-09-17T04:24:44.403 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:24:44.424 INFO:teuthology.orchestra.run.smithi184.stdout:instance: client.4680 (3)172.21.15.184:0/1844453946 2024-09-17T04:24:44.424 INFO:teuthology.orchestra.run.smithi184.stdout:blocklisted: false 2024-09-17T04:24:44.424 INFO:teuthology.orchestra.run.smithi184.stderr:0+1 records in 2024-09-17T04:24:44.424 INFO:teuthology.orchestra.run.smithi184.stderr:0+1 records out 2024-09-17T04:24:44.424 INFO:teuthology.orchestra.run.smithi184.stderr:71 bytes copied, 5.2218e-05 s, 1.4 MB/s 2024-09-17T04:24:44.426 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:44.434 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-09-17T04:24:44.434 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T04:24:44.434 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-17T04:24:44.435 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:24:44.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.666+0000 7f5a86e00640 1 -- 172.21.15.100:0/3274820614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a800b98d0 msgr2=0x7f5a800b58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:44.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.666+0000 7f5a86e00640 1 --2- 172.21.15.100:0/3274820614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a800b98d0 0x7f5a800b58b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5a8008ccb0 tx=0x7f5a7402a530 comp rx=0 tx=0).stop 2024-09-17T04:24:44.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.666+0000 7f5a86e00640 1 -- 172.21.15.100:0/3274820614 shutdown_connections 2024-09-17T04:24:44.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.666+0000 7f5a86e00640 1 --2- 172.21.15.100:0/3274820614 >> v2:172.21.15.184:3300/0 conn(0x7f5a800b3e70 0x7f5a800b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:44.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.666+0000 7f5a86e00640 1 --2- 172.21.15.100:0/3274820614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a800b98d0 0x7f5a800b58b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:44.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.666+0000 7f5a86e00640 1 -- 172.21.15.100:0/3274820614 >> 172.21.15.100:0/3274820614 conn(0x7f5a8009d820 msgr2=0x7f5a8008bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:44.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.666+0000 7f5a86e00640 1 -- 172.21.15.100:0/3274820614 shutdown_connections 2024-09-17T04:24:44.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.666+0000 7f5a86e00640 1 -- 172.21.15.100:0/3274820614 wait complete. 2024-09-17T04:24:44.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.668+0000 7f5a86e00640 1 Processor -- start 2024-09-17T04:24:44.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.668+0000 7f5a86e00640 1 -- start start 2024-09-17T04:24:44.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.670+0000 7f5a86e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a800b3e70 0x7f5a80151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:44.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.670+0000 7f5a86e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a800b98d0 0x7f5a800ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:44.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.670+0000 7f5a86e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a800d0e70 0x7f5a800ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:44.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.670+0000 7f5a86e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5a800cfcb0 con 0x7f5a800d0e70 2024-09-17T04:24:44.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.670+0000 7f5a86e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5a800cfb30 con 0x7f5a800b3e70 2024-09-17T04:24:44.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.670+0000 7f5a86e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5a800cfe30 con 0x7f5a800b98d0 2024-09-17T04:24:44.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.670+0000 7f5a86400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a800d0e70 0x7f5a800ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:44.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.670+0000 7f5a86400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a800d0e70 0x7f5a800ca6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37030/0 (socket says 172.21.15.100:37030) 2024-09-17T04:24:44.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.670+0000 7f5a86400640 1 -- 172.21.15.100:0/2469574526 learned_addr learned my addr 172.21.15.100:0/2469574526 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:44.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.670+0000 7f5a85a00640 1 --2- 172.21.15.100:0/2469574526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a800b3e70 0x7f5a80151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:44.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.670+0000 7f5a85000640 1 --2- 172.21.15.100:0/2469574526 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a800b98d0 0x7f5a800ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:44.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.671+0000 7f5a86400640 1 -- 172.21.15.100:0/2469574526 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a800b98d0 msgr2=0x7f5a800ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:44.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.671+0000 7f5a86400640 1 --2- 172.21.15.100:0/2469574526 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a800b98d0 0x7f5a800ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:44.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.671+0000 7f5a86400640 1 -- 172.21.15.100:0/2469574526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a800b3e70 msgr2=0x7f5a80151480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:44.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.671+0000 7f5a86400640 1 --2- 172.21.15.100:0/2469574526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a800b3e70 0x7f5a80151480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:44.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.671+0000 7f5a86400640 1 -- 172.21.15.100:0/2469574526 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a800cd1c0 con 0x7f5a800d0e70 2024-09-17T04:24:44.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.671+0000 7f5a86400640 1 --2- 172.21.15.100:0/2469574526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a800d0e70 0x7f5a800ca6f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5a7800c6f0 tx=0x7f5a7800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:44.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.671+0000 7f5a7d600640 1 -- 172.21.15.100:0/2469574526 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a78017070 con 0x7f5a800d0e70 2024-09-17T04:24:44.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.671+0000 7f5a7d600640 1 -- 172.21.15.100:0/2469574526 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a78003c90 con 0x7f5a800d0e70 2024-09-17T04:24:44.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.671+0000 7f5a86e00640 1 -- 172.21.15.100:0/2469574526 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a800cd4a0 con 0x7f5a800d0e70 2024-09-17T04:24:44.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.671+0000 7f5a7d600640 1 -- 172.21.15.100:0/2469574526 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a78003e60 con 0x7f5a800d0e70 2024-09-17T04:24:44.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.671+0000 7f5a86e00640 1 -- 172.21.15.100:0/2469574526 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a800cda60 con 0x7f5a800d0e70 2024-09-17T04:24:44.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.672+0000 7f5a86e00640 1 -- 172.21.15.100:0/2469574526 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a800d0350 con 0x7f5a800d0e70 2024-09-17T04:24:44.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.673+0000 7f5a7d600640 1 -- 172.21.15.100:0/2469574526 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5a7801a070 con 0x7f5a800d0e70 2024-09-17T04:24:44.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.674+0000 7f5a7d600640 1 --2- 172.21.15.100:0/2469574526 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5a60078b90 0x7f5a6007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:44.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.674+0000 7f5a7d600640 1 -- 172.21.15.100:0/2469574526 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f5a7809a300 con 0x7f5a800d0e70 2024-09-17T04:24:44.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.675+0000 7f5a85a00640 1 --2- 172.21.15.100:0/2469574526 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5a60078b90 0x7f5a6007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:44.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.675+0000 7f5a85a00640 1 --2- 172.21.15.100:0/2469574526 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5a60078b90 0x7f5a6007b0a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5a74034000 tx=0x7f5a74004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:44.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.679+0000 7f5a7d600640 1 -- 172.21.15.100:0/2469574526 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f5a78062050 con 0x7f5a800d0e70 2024-09-17T04:24:44.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.897+0000 7f5a86e00640 1 -- 172.21.15.100:0/2469574526 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5a80162000 con 0x7f5a800d0e70 2024-09-17T04:24:44.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.898+0000 7f5a7d600640 1 -- 172.21.15.100:0/2469574526 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f5a78066a80 con 0x7f5a800d0e70 2024-09-17T04:24:44.897 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:24:44.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.900+0000 7f5a86e00640 1 -- 172.21.15.100:0/2469574526 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5a60078b90 msgr2=0x7f5a6007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:44.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.900+0000 7f5a86e00640 1 --2- 172.21.15.100:0/2469574526 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5a60078b90 0x7f5a6007b0a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5a74034000 tx=0x7f5a74004730 comp rx=0 tx=0).stop 2024-09-17T04:24:44.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.900+0000 7f5a86e00640 1 -- 172.21.15.100:0/2469574526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a800d0e70 msgr2=0x7f5a800ca6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:44.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.900+0000 7f5a86e00640 1 --2- 172.21.15.100:0/2469574526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a800d0e70 0x7f5a800ca6f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5a7800c6f0 tx=0x7f5a7800cbb0 comp rx=0 tx=0).stop 2024-09-17T04:24:44.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.900+0000 7f5a86e00640 1 -- 172.21.15.100:0/2469574526 shutdown_connections 2024-09-17T04:24:44.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.900+0000 7f5a86e00640 1 --2- 172.21.15.100:0/2469574526 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5a60078b90 0x7f5a6007b0a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:44.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.900+0000 7f5a86e00640 1 --2- 172.21.15.100:0/2469574526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a800d0e70 0x7f5a800ca6f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:44.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.900+0000 7f5a86e00640 1 --2- 172.21.15.100:0/2469574526 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a800b98d0 0x7f5a800ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:44.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.900+0000 7f5a86e00640 1 --2- 172.21.15.100:0/2469574526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a800b3e70 0x7f5a80151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:44.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.900+0000 7f5a86e00640 1 -- 172.21.15.100:0/2469574526 >> 172.21.15.100:0/2469574526 conn(0x7f5a8009d820 msgr2=0x7f5a80068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:44.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.900+0000 7f5a86e00640 1 -- 172.21.15.100:0/2469574526 shutdown_connections 2024-09-17T04:24:44.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:44.900+0000 7f5a86e00640 1 -- 172.21.15.100:0/2469574526 wait complete. 2024-09-17T04:24:44.921 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T04:24:44.921 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:24:44.921 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2024-09-17T04:24:44.921 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-17T04:24:44.921 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-17T04:24:44.921 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T04:24:44.921 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T04:24:44.921 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:24:44.921 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-17T04:24:44.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:44.922 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2024-09-17T04:24:44.958 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:24:44.958 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:24:44.958 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:24:44.959 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:44.959 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-09-17T04:24:44.959 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:44.959 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:24:44.959 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:24:44.959 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2024-09-17T04:24:44.959 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:24:44.959 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:24:44.959 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2024-09-17T04:24:44.959 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:24:44.959 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2024-09-17T04:24:44.959 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:24:44.959 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2024-09-17T04:24:44.960 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T04:24:44.960 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42659sec preferred_lft 42659sec 2024-09-17T04:24:44.960 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ec4:7aff:fe88:7b8f/64 scope link noprefixroute 2024-09-17T04:24:44.960 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:44.960 INFO:teuthology.orchestra.run.smithi184.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:24:44.960 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether a6:a9:1b:a9:61:ec brd ff:ff:ff:ff:ff:ff 2024-09-17T04:24:44.960 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T04:24:44.960 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:44.960 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::3c9d:aff:fe6e:8fcb/64 scope link 2024-09-17T04:24:44.960 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:44.960 INFO:teuthology.orchestra.run.smithi184.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:24:44.960 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether a6:a9:1b:a9:61:ec brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T04:24:44.960 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::a4a9:1bff:fea9:61ec/64 scope link 2024-09-17T04:24:44.961 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:44.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:44.961 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:24:45.054 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-09-17T04:24:45.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:45.057 DEBUG:teuthology.orchestra.run.smithi184:> route 2024-09-17T04:24:45.093 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2024-09-17T04:24:45.094 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:24:45.094 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T04:24:45.094 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T04:24:45.094 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:24:45.094 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:24:45.094 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-09-17T04:24:45.094 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T04:24:45.094 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T04:24:45.095 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T04:24:45.095 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-09-17T04:24:45.211 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:24:45.282 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:45.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:45.291 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-09-17T04:24:45.326 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T04:24:45.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:45.326 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list-id 2024-09-17T04:24:45.400 INFO:teuthology.orchestra.run.smithi184.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-17T04:24:45.401 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:24:45.401 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-09-17T04:24:45.401 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T04:24:45.402 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-17T04:24:45.402 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-09-17T04:24:45.518 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:24:45.568 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:45.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:45.576 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-17T04:24:45.631 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:24:45.632 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:24:45.632 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:24:45.632 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:45.632 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-09-17T04:24:45.632 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:45.632 INFO:teuthology.orchestra.run.smithi184.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:24:45.632 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 16:3d:c4:53:f3:9b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T04:24:45.632 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T04:24:45.632 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:45.632 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::143d:c4ff:fe53:f39b/64 scope link 2024-09-17T04:24:45.632 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:45.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:45.634 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-17T04:24:45.690 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:24:45.691 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:24:45.691 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:24:45.691 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:45.691 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-09-17T04:24:45.691 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:45.691 INFO:teuthology.orchestra.run.smithi184.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:24:45.691 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 16:3d:c4:53:f3:9b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T04:24:45.691 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T04:24:45.691 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:45.691 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::143d:c4ff:fe53:f39b/64 scope link 2024-09-17T04:24:45.691 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:24:45.693 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-17T04:24:45.694 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:24:45.694 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-09-17T04:24:45.694 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-17T04:24:45.694 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-17T04:24:45.694 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-17T04:24:45.694 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-17T04:24:45.694 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-17T04:24:45.695 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-09-17T04:24:45.771 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:24:45.902 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:45.910 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:24:45.911 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-09-17T04:24:45.911 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.1 up 2024-09-17T04:24:45.911 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set dev brx.1 master ceph-brx 2024-09-17T04:24:45.911 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-09-17T04:24:45.988 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:24:46.069 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:46.077 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:24:46.078 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-17T04:24:46.113 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-17T04:24:46.113 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:24:46.113 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-17T04:24:46.188 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T04:24:46.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:46.188 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-17T04:24:46.382 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-17T04:24:46.383 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-17T04:24:46.383 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:24:46.402 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-17T04:24:46.402 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-17T04:24:46.402 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:24:46.402 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:24:46.403 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:24:46.431 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-17T04:24:46.431 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-17T04:24:46.431 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:24:46.431 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:24:46.431 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:24:46.431 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-09-17T04:24:46.431 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-09-17T04:24:46.431 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=8a399842-3a5c-4ea2-9afb-b2704332e59f" will pass to kernel 2024-09-17T04:24:46.433 INFO:teuthology.orchestra.run.smithi184.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-09-17T04:24:46.433 INFO:teuthology.orchestra.run.smithi184.stdout: You just mounted a file system that supports labels which does not 2024-09-17T04:24:46.433 INFO:teuthology.orchestra.run.smithi184.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-17T04:24:46.433 INFO:teuthology.orchestra.run.smithi184.stdout: applications will generate AVC messages and not be allowed access to 2024-09-17T04:24:46.433 INFO:teuthology.orchestra.run.smithi184.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-17T04:24:46.436 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T04:24:46.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:46.436 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-17T04:24:46.492 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:24:46.569 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:24:46.572 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:46.580 INFO:teuthology.orchestra.run.smithi184.stdout:client4568 2024-09-17T04:24:46.580 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:24:46.581 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:24:46.616 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:24:46.732 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:24:46.735 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:46.743 INFO:teuthology.orchestra.run.smithi184.stdout:client4568 2024-09-17T04:24:46.743 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:24:46.821 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:24:46.824 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:46.832 INFO:teuthology.orchestra.run.smithi184.stdout:client4568 2024-09-17T04:24:46.832 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/8a399842-3a5c-4ea2-9afb-b2704332e59f.client4568/status') 2024-09-17T04:24:46.909 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:24:46.930 INFO:teuthology.orchestra.run.smithi184.stdout:instance: client.4568 (3)192.168.0.2:0/2473342756 2024-09-17T04:24:46.930 INFO:teuthology.orchestra.run.smithi184.stdout:blocklisted: false 2024-09-17T04:24:46.931 INFO:teuthology.orchestra.run.smithi184.stderr:0+1 records in 2024-09-17T04:24:46.931 INFO:teuthology.orchestra.run.smithi184.stderr:0+1 records out 2024-09-17T04:24:46.931 INFO:teuthology.orchestra.run.smithi184.stderr:69 bytes copied, 8.8702e-05 s, 778 kB/s 2024-09-17T04:24:46.933 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:24:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:24:46.941 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-17T04:24:47.061 INFO:tasks.cephfs_test_runner:Starting test: test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate binary .meta file on legacy subvol root doesn't break the system 2024-09-17T04:24:47.061 INFO:tasks.cephfs_test_runner:test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-09-17T04:24:47.061 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:24:47.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.285+0000 7ff712600640 1 -- 172.21.15.100:0/4171737609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff70c0d29e0 msgr2=0x7ff70c0d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:47.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.285+0000 7ff712600640 1 --2- 172.21.15.100:0/4171737609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff70c0d29e0 0x7ff70c0d1580 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff70c0d3820 tx=0x7ff70402aa30 comp rx=0 tx=0).stop 2024-09-17T04:24:47.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.285+0000 7ff712600640 1 -- 172.21.15.100:0/4171737609 shutdown_connections 2024-09-17T04:24:47.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.285+0000 7ff712600640 1 --2- 172.21.15.100:0/4171737609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff70c0d29e0 0x7ff70c0d1580 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:47.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.285+0000 7ff712600640 1 --2- 172.21.15.100:0/4171737609 >> v2:172.21.15.184:3300/0 conn(0x7ff70c0b3160 0x7ff70c0d24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:47.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.285+0000 7ff712600640 1 -- 172.21.15.100:0/4171737609 >> 172.21.15.100:0/4171737609 conn(0x7ff70c08bfe0 msgr2=0x7ff70c09d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:47.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.285+0000 7ff712600640 1 -- 172.21.15.100:0/4171737609 shutdown_connections 2024-09-17T04:24:47.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.285+0000 7ff712600640 1 -- 172.21.15.100:0/4171737609 wait complete. 2024-09-17T04:24:47.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.287+0000 7ff712600640 1 Processor -- start 2024-09-17T04:24:47.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.287+0000 7ff712600640 1 -- start start 2024-09-17T04:24:47.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.288+0000 7ff712600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff70c0b3160 0x7ff70c1f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:47.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.288+0000 7ff712600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff70c0b9e00 0x7ff70c15c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:47.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.288+0000 7ff712600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff70c0d29e0 0x7ff70c15e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:47.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.289+0000 7ff712600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c0d44b0 con 0x7ff70c0d29e0 2024-09-17T04:24:47.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.289+0000 7ff712600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c0d4330 con 0x7ff70c0b9e00 2024-09-17T04:24:47.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.289+0000 7ff712600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff70c0d4630 con 0x7ff70c0b3160 2024-09-17T04:24:47.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.289+0000 7ff711c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff70c0d29e0 0x7ff70c15e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:47.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.289+0000 7ff703e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff70c0b9e00 0x7ff70c15c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:47.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.289+0000 7ff711c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff70c0d29e0 0x7ff70c15e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56776/0 (socket says 172.21.15.100:56776) 2024-09-17T04:24:47.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.289+0000 7ff711c00640 1 -- 172.21.15.100:0/2199687318 learned_addr learned my addr 172.21.15.100:0/2199687318 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:47.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.289+0000 7ff711200640 1 --2- 172.21.15.100:0/2199687318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff70c0b3160 0x7ff70c1f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:47.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.289+0000 7ff711c00640 1 -- 172.21.15.100:0/2199687318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff70c0b3160 msgr2=0x7ff70c1f5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:47.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.289+0000 7ff711c00640 1 --2- 172.21.15.100:0/2199687318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff70c0b3160 0x7ff70c1f5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:47.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.289+0000 7ff711c00640 1 -- 172.21.15.100:0/2199687318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff70c0b9e00 msgr2=0x7ff70c15c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:47.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.289+0000 7ff711c00640 1 --2- 172.21.15.100:0/2199687318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff70c0b9e00 0x7ff70c15c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:47.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.289+0000 7ff711c00640 1 -- 172.21.15.100:0/2199687318 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff70c160720 con 0x7ff70c0d29e0 2024-09-17T04:24:47.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.290+0000 7ff711c00640 1 --2- 172.21.15.100:0/2199687318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff70c0d29e0 0x7ff70c15e320 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff704009d60 tx=0x7ff704003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:47.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.290+0000 7ff709600640 1 -- 172.21.15.100:0/2199687318 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff704033050 con 0x7ff70c0d29e0 2024-09-17T04:24:47.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.290+0000 7ff709600640 1 -- 172.21.15.100:0/2199687318 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff704004450 con 0x7ff70c0d29e0 2024-09-17T04:24:47.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.290+0000 7ff712600640 1 -- 172.21.15.100:0/2199687318 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff70c160a00 con 0x7ff70c0d29e0 2024-09-17T04:24:47.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.290+0000 7ff709600640 1 -- 172.21.15.100:0/2199687318 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff704038450 con 0x7ff70c0d29e0 2024-09-17T04:24:47.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.290+0000 7ff712600640 1 -- 172.21.15.100:0/2199687318 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff70c160fc0 con 0x7ff70c0d29e0 2024-09-17T04:24:47.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.291+0000 7ff712600640 1 -- 172.21.15.100:0/2199687318 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff70c15cf90 con 0x7ff70c0d29e0 2024-09-17T04:24:47.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.292+0000 7ff709600640 1 -- 172.21.15.100:0/2199687318 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff704036070 con 0x7ff70c0d29e0 2024-09-17T04:24:47.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.294+0000 7ff709600640 1 --2- 172.21.15.100:0/2199687318 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff6e8078b90 0x7ff6e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:47.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.294+0000 7ff709600640 1 -- 172.21.15.100:0/2199687318 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff7040bc0c0 con 0x7ff70c0d29e0 2024-09-17T04:24:47.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.294+0000 7ff711200640 1 --2- 172.21.15.100:0/2199687318 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff6e8078b90 0x7ff6e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:47.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.295+0000 7ff711200640 1 --2- 172.21.15.100:0/2199687318 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff6e8078b90 0x7ff6e807b0a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc001720 tx=0x7ff6fc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:47.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.299+0000 7ff709600640 1 -- 172.21.15.100:0/2199687318 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff704083fb0 con 0x7ff70c0d29e0 2024-09-17T04:24:47.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.482+0000 7ff712600640 1 -- 172.21.15.100:0/2199687318 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff70c0d4b50 con 0x7ff70c0d29e0 2024-09-17T04:24:47.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.484+0000 7ff709600640 1 -- 172.21.15.100:0/2199687318 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16713 (secure 0 0 0) 0x7ff7040889e0 con 0x7ff70c0d29e0 2024-09-17T04:24:47.483 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:47.483 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":23,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:24:16.038948+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:24:12.826665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T04:24:13.485446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:24:47.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.486+0000 7ff712600640 1 -- 172.21.15.100:0/2199687318 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff6e8078b90 msgr2=0x7ff6e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:47.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.486+0000 7ff712600640 1 --2- 172.21.15.100:0/2199687318 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff6e8078b90 0x7ff6e807b0a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc001720 tx=0x7ff6fc009040 comp rx=0 tx=0).stop 2024-09-17T04:24:47.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.486+0000 7ff712600640 1 -- 172.21.15.100:0/2199687318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff70c0d29e0 msgr2=0x7ff70c15e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:47.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.486+0000 7ff712600640 1 --2- 172.21.15.100:0/2199687318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff70c0d29e0 0x7ff70c15e320 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff704009d60 tx=0x7ff704003f10 comp rx=0 tx=0).stop 2024-09-17T04:24:47.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.486+0000 7ff712600640 1 -- 172.21.15.100:0/2199687318 shutdown_connections 2024-09-17T04:24:47.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.486+0000 7ff712600640 1 --2- 172.21.15.100:0/2199687318 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff6e8078b90 0x7ff6e807b0a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:47.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.486+0000 7ff712600640 1 --2- 172.21.15.100:0/2199687318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff70c0d29e0 0x7ff70c15e320 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:47.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.486+0000 7ff712600640 1 --2- 172.21.15.100:0/2199687318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff70c0b9e00 0x7ff70c15c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:47.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.486+0000 7ff712600640 1 --2- 172.21.15.100:0/2199687318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff70c0b3160 0x7ff70c1f5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:47.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.486+0000 7ff712600640 1 -- 172.21.15.100:0/2199687318 >> 172.21.15.100:0/2199687318 conn(0x7ff70c08bfe0 msgr2=0x7ff70c0cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:47.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.486+0000 7ff712600640 1 -- 172.21.15.100:0/2199687318 shutdown_connections 2024-09-17T04:24:47.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.486+0000 7ff712600640 1 -- 172.21.15.100:0/2199687318 wait complete. 2024-09-17T04:24:47.508 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T04:24:03.995355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-17T04:24:12.826665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-17T04:24:13.485446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T04:24:47.508 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T04:24:47.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.730+0000 7fd5bb600640 1 -- 172.21.15.100:0/630156151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5b40ad3d0 msgr2=0x7fd5b40ad7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:47.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.730+0000 7fd5bb600640 1 --2- 172.21.15.100:0/630156151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5b40ad3d0 0x7fd5b40ad7e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac005fd0 tx=0x7fd5ac02a940 comp rx=0 tx=0).stop 2024-09-17T04:24:47.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.730+0000 7fd5bb600640 1 -- 172.21.15.100:0/630156151 shutdown_connections 2024-09-17T04:24:47.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.730+0000 7fd5bb600640 1 --2- 172.21.15.100:0/630156151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5b40ad3d0 0x7fd5b40ad7e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:47.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.730+0000 7fd5bb600640 1 -- 172.21.15.100:0/630156151 >> 172.21.15.100:0/630156151 conn(0x7fd5b406d710 msgr2=0x7fd5b40b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:47.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.730+0000 7fd5bb600640 1 -- 172.21.15.100:0/630156151 shutdown_connections 2024-09-17T04:24:47.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.730+0000 7fd5bb600640 1 -- 172.21.15.100:0/630156151 wait complete. 2024-09-17T04:24:47.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.732+0000 7fd5bb600640 1 Processor -- start 2024-09-17T04:24:47.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.732+0000 7fd5bb600640 1 -- start start 2024-09-17T04:24:47.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.733+0000 7fd5bb600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5b40ad3d0 0x7fd5b409ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:47.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.733+0000 7fd5bb600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5b40b4f00 0x7fd5b40b0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:47.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.733+0000 7fd5bb600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5b4160490 0x7fd5b4098e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:47.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.733+0000 7fd5bb600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b406f300 con 0x7fd5b40b4f00 2024-09-17T04:24:47.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.733+0000 7fd5bb600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b406f180 con 0x7fd5b40ad3d0 2024-09-17T04:24:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.733+0000 7fd5bb600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd5b406f480 con 0x7fd5b4160490 2024-09-17T04:24:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.734+0000 7fd5b3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5b40b4f00 0x7fd5b40b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.734+0000 7fd5b3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5b40b4f00 0x7fd5b40b0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56788/0 (socket says 172.21.15.100:56788) 2024-09-17T04:24:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.734+0000 7fd5b3e00640 1 -- 172.21.15.100:0/376791420 learned_addr learned my addr 172.21.15.100:0/376791420 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.734+0000 7fd5b9600640 1 --2- 172.21.15.100:0/376791420 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5b4160490 0x7fd5b4098e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.734+0000 7fd5b8c00640 1 --2- 172.21.15.100:0/376791420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5b40ad3d0 0x7fd5b409ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.734+0000 7fd5b3e00640 1 -- 172.21.15.100:0/376791420 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5b4160490 msgr2=0x7fd5b4098e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.734+0000 7fd5b3e00640 1 --2- 172.21.15.100:0/376791420 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5b4160490 0x7fd5b4098e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.734+0000 7fd5b3e00640 1 -- 172.21.15.100:0/376791420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5b40ad3d0 msgr2=0x7fd5b409ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.734+0000 7fd5b3e00640 1 --2- 172.21.15.100:0/376791420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5b40ad3d0 0x7fd5b409ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.734+0000 7fd5b3e00640 1 -- 172.21.15.100:0/376791420 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b41f4340 con 0x7fd5b40b4f00 2024-09-17T04:24:47.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.734+0000 7fd5b3e00640 1 --2- 172.21.15.100:0/376791420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5b40b4f00 0x7fd5b40b0030 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000c430 tx=0x7fd5a000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:47.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.735+0000 7fd5b0c00640 1 -- 172.21.15.100:0/376791420 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0011790 con 0x7fd5b40b4f00 2024-09-17T04:24:47.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.735+0000 7fd5b0c00640 1 -- 172.21.15.100:0/376791420 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5a0017070 con 0x7fd5b40b4f00 2024-09-17T04:24:47.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.735+0000 7fd5b0c00640 1 -- 172.21.15.100:0/376791420 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0011ce0 con 0x7fd5b40b4f00 2024-09-17T04:24:47.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.735+0000 7fd5bb600640 1 -- 172.21.15.100:0/376791420 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b41f37c0 con 0x7fd5b40b4f00 2024-09-17T04:24:47.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.735+0000 7fd5bb600640 1 -- 172.21.15.100:0/376791420 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b41f3d80 con 0x7fd5b40b4f00 2024-09-17T04:24:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.736+0000 7fd5bb600640 1 -- 172.21.15.100:0/376791420 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5b406f9c0 con 0x7fd5b40b4f00 2024-09-17T04:24:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.737+0000 7fd5b0c00640 1 -- 172.21.15.100:0/376791420 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd5a001c460 con 0x7fd5b40b4f00 2024-09-17T04:24:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.738+0000 7fd5b0c00640 1 --2- 172.21.15.100:0/376791420 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd594078b90 0x7fd59407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.738+0000 7fd5b0c00640 1 -- 172.21.15.100:0/376791420 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd5a009ab90 con 0x7fd5b40b4f00 2024-09-17T04:24:47.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.738+0000 7fd5b8c00640 1 --2- 172.21.15.100:0/376791420 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd594078b90 0x7fd59407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:47.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.739+0000 7fd5b8c00640 1 --2- 172.21.15.100:0/376791420 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd594078b90 0x7fd59407b0a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8007fd0 tx=0x7fd5a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:47.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.742+0000 7fd5b0c00640 1 -- 172.21.15.100:0/376791420 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd5a0062fc0 con 0x7fd5b40b4f00 2024-09-17T04:24:47.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.924+0000 7fd5bb600640 1 -- 172.21.15.100:0/376791420 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd5b40b49c0 con 0x7fd5b40b4f00 2024-09-17T04:24:47.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.925+0000 7fd5b0c00640 1 -- 172.21.15.100:0/376791420 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7fd5a00628c0 con 0x7fd5b40b4f00 2024-09-17T04:24:47.925 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-09-17T04:24:47.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.927+0000 7fd5bb600640 1 -- 172.21.15.100:0/376791420 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd594078b90 msgr2=0x7fd59407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:47.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.927+0000 7fd5bb600640 1 --2- 172.21.15.100:0/376791420 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd594078b90 0x7fd59407b0a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8007fd0 tx=0x7fd5a8009040 comp rx=0 tx=0).stop 2024-09-17T04:24:47.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.927+0000 7fd5bb600640 1 -- 172.21.15.100:0/376791420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5b40b4f00 msgr2=0x7fd5b40b0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:47.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.927+0000 7fd5bb600640 1 --2- 172.21.15.100:0/376791420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5b40b4f00 0x7fd5b40b0030 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000c430 tx=0x7fd5a000c8f0 comp rx=0 tx=0).stop 2024-09-17T04:24:47.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.927+0000 7fd5bb600640 1 -- 172.21.15.100:0/376791420 shutdown_connections 2024-09-17T04:24:47.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.927+0000 7fd5bb600640 1 --2- 172.21.15.100:0/376791420 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd594078b90 0x7fd59407b0a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:47.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.927+0000 7fd5bb600640 1 --2- 172.21.15.100:0/376791420 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5b4160490 0x7fd5b4098e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:47.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.927+0000 7fd5bb600640 1 --2- 172.21.15.100:0/376791420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5b40b4f00 0x7fd5b40b0030 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:47.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.927+0000 7fd5bb600640 1 --2- 172.21.15.100:0/376791420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5b40ad3d0 0x7fd5b409ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:47.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.927+0000 7fd5bb600640 1 -- 172.21.15.100:0/376791420 >> 172.21.15.100:0/376791420 conn(0x7fd5b406d710 msgr2=0x7fd5b415c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:47.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.927+0000 7fd5bb600640 1 -- 172.21.15.100:0/376791420 shutdown_connections 2024-09-17T04:24:47.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:47.928+0000 7fd5bb600640 1 -- 172.21.15.100:0/376791420 wait complete. 2024-09-17T04:24:47.951 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-17T04:24:48.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.176+0000 7f90d4600640 1 -- 172.21.15.100:0/3753633209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90cc146110 msgr2=0x7f90cc0b3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:48.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.176+0000 7f90d4600640 1 --2- 172.21.15.100:0/3753633209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90cc146110 0x7f90cc0b3be0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f90cc0d0520 tx=0x7f90b802a4a0 comp rx=0 tx=0).stop 2024-09-17T04:24:48.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.176+0000 7f90d4600640 1 -- 172.21.15.100:0/3753633209 shutdown_connections 2024-09-17T04:24:48.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.176+0000 7f90d4600640 1 --2- 172.21.15.100:0/3753633209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90cc146110 0x7f90cc0b3be0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:48.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.176+0000 7f90d4600640 1 --2- 172.21.15.100:0/3753633209 >> v2:172.21.15.184:3300/0 conn(0x7f90cc1604a0 0x7f90cc0cca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:48.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.176+0000 7f90d4600640 1 -- 172.21.15.100:0/3753633209 >> 172.21.15.100:0/3753633209 conn(0x7f90cc0b2400 msgr2=0x7f90cc0b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:48.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.176+0000 7f90d4600640 1 -- 172.21.15.100:0/3753633209 shutdown_connections 2024-09-17T04:24:48.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.176+0000 7f90d4600640 1 -- 172.21.15.100:0/3753633209 wait complete. 2024-09-17T04:24:48.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.178+0000 7f90d4600640 1 Processor -- start 2024-09-17T04:24:48.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.178+0000 7f90d4600640 1 -- start start 2024-09-17T04:24:48.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.180+0000 7f90d4600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90cc0b4120 0x7f90cc0b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:48.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.180+0000 7f90d4600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90cc146110 0x7f90cc1f6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:48.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.180+0000 7f90d4600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90cc1604a0 0x7f90cc0cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:48.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.180+0000 7f90d4600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc071280 con 0x7f90cc0b4120 2024-09-17T04:24:48.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.180+0000 7f90d4600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc0d15f0 con 0x7f90cc146110 2024-09-17T04:24:48.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.180+0000 7f90d4600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f90cc071400 con 0x7f90cc1604a0 2024-09-17T04:24:48.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.180+0000 7f90d1c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90cc0b4120 0x7f90cc0b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:48.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.180+0000 7f90d1c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90cc0b4120 0x7f90cc0b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56812/0 (socket says 172.21.15.100:56812) 2024-09-17T04:24:48.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.180+0000 7f90d1c00640 1 -- 172.21.15.100:0/46397523 learned_addr learned my addr 172.21.15.100:0/46397523 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:48.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.180+0000 7f90d1200640 1 --2- 172.21.15.100:0/46397523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90cc146110 0x7f90cc1f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:48.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.180+0000 7f90d2600640 1 --2- 172.21.15.100:0/46397523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90cc1604a0 0x7f90cc0cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:48.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.180+0000 7f90d1c00640 1 -- 172.21.15.100:0/46397523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90cc1604a0 msgr2=0x7f90cc0cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:48.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.180+0000 7f90d1c00640 1 --2- 172.21.15.100:0/46397523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90cc1604a0 0x7f90cc0cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:48.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.180+0000 7f90d1c00640 1 -- 172.21.15.100:0/46397523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90cc146110 msgr2=0x7f90cc1f6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:48.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.180+0000 7f90d1c00640 1 --2- 172.21.15.100:0/46397523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90cc146110 0x7f90cc1f6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:48.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.180+0000 7f90d1c00640 1 -- 172.21.15.100:0/46397523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90cc1f6570 con 0x7f90cc0b4120 2024-09-17T04:24:48.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.180+0000 7f90d1200640 1 --2- 172.21.15.100:0/46397523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90cc146110 0x7f90cc1f6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:24:48.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.181+0000 7f90d1c00640 1 --2- 172.21.15.100:0/46397523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90cc0b4120 0x7f90cc0b1bc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f90c000e790 tx=0x7f90c000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:48.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.181+0000 7f90c9600640 1 -- 172.21.15.100:0/46397523 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c000eea0 con 0x7f90cc0b4120 2024-09-17T04:24:48.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.181+0000 7f90c9600640 1 -- 172.21.15.100:0/46397523 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90c000cdb0 con 0x7f90cc0b4120 2024-09-17T04:24:48.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.181+0000 7f90c9600640 1 -- 172.21.15.100:0/46397523 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0010690 con 0x7f90cc0b4120 2024-09-17T04:24:48.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.181+0000 7f90d4600640 1 -- 172.21.15.100:0/46397523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90cc0cbbd0 con 0x7f90cc0b4120 2024-09-17T04:24:48.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.181+0000 7f90d4600640 1 -- 172.21.15.100:0/46397523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90cc0aaa10 con 0x7f90cc0b4120 2024-09-17T04:24:48.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.182+0000 7f90d4600640 1 -- 172.21.15.100:0/46397523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90cc0b3be0 con 0x7f90cc0b4120 2024-09-17T04:24:48.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.184+0000 7f90c9600640 1 -- 172.21.15.100:0/46397523 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f90c0010830 con 0x7f90cc0b4120 2024-09-17T04:24:48.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.184+0000 7f90c9600640 1 --2- 172.21.15.100:0/46397523 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f90ac078b90 0x7f90ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:48.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.185+0000 7f90c9600640 1 -- 172.21.15.100:0/46397523 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f90c009b880 con 0x7f90cc0b4120 2024-09-17T04:24:48.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.185+0000 7f90d1200640 1 --2- 172.21.15.100:0/46397523 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f90ac078b90 0x7f90ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:48.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.186+0000 7f90d1200640 1 --2- 172.21.15.100:0/46397523 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f90ac078b90 0x7f90ac07b0a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f90b8034000 tx=0x7f90b8004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:48.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.189+0000 7f90c9600640 1 -- 172.21.15.100:0/46397523 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f90cc1f6570 con 0x7f90cc0b4120 2024-09-17T04:24:48.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.371+0000 7f90d4600640 1 -- 172.21.15.100:0/46397523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f90cc1e9310 con 0x7f90cc0b4120 2024-09-17T04:24:48.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.371+0000 7f90c9600640 1 -- 172.21.15.100:0/46397523 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f90c0063c20 con 0x7f90cc0b4120 2024-09-17T04:24:48.371 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-09-17T04:24:48.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.374+0000 7f90d4600640 1 -- 172.21.15.100:0/46397523 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f90ac078b90 msgr2=0x7f90ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:48.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.374+0000 7f90d4600640 1 --2- 172.21.15.100:0/46397523 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f90ac078b90 0x7f90ac07b0a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f90b8034000 tx=0x7f90b8004730 comp rx=0 tx=0).stop 2024-09-17T04:24:48.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.374+0000 7f90d4600640 1 -- 172.21.15.100:0/46397523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90cc0b4120 msgr2=0x7f90cc0b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:48.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.374+0000 7f90d4600640 1 --2- 172.21.15.100:0/46397523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90cc0b4120 0x7f90cc0b1bc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f90c000e790 tx=0x7f90c000ec50 comp rx=0 tx=0).stop 2024-09-17T04:24:48.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.374+0000 7f90d4600640 1 -- 172.21.15.100:0/46397523 shutdown_connections 2024-09-17T04:24:48.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.374+0000 7f90d4600640 1 --2- 172.21.15.100:0/46397523 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f90ac078b90 0x7f90ac07b0a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:48.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.374+0000 7f90d4600640 1 --2- 172.21.15.100:0/46397523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90cc1604a0 0x7f90cc0cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:48.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.374+0000 7f90d4600640 1 --2- 172.21.15.100:0/46397523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90cc146110 0x7f90cc1f6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:48.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.374+0000 7f90d4600640 1 --2- 172.21.15.100:0/46397523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90cc0b4120 0x7f90cc0b1bc0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:48.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.374+0000 7f90d4600640 1 -- 172.21.15.100:0/46397523 >> 172.21.15.100:0/46397523 conn(0x7f90cc0b2400 msgr2=0x7f90cc144350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:48.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.374+0000 7f90d4600640 1 -- 172.21.15.100:0/46397523 shutdown_connections 2024-09-17T04:24:48.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.374+0000 7f90d4600640 1 -- 172.21.15.100:0/46397523 wait complete. 2024-09-17T04:24:48.396 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-17T04:24:48.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.619+0000 7f2317e00640 1 -- 172.21.15.100:0/3632083085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2318156ca0 msgr2=0x7f23180ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:48.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.619+0000 7f2317e00640 1 --2- 172.21.15.100:0/3632083085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2318156ca0 0x7f23180ccac0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2300009830 tx=0x7f230002a5c0 comp rx=0 tx=0).stop 2024-09-17T04:24:48.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.619+0000 7f2317e00640 1 -- 172.21.15.100:0/3632083085 shutdown_connections 2024-09-17T04:24:48.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.619+0000 7f2317e00640 1 --2- 172.21.15.100:0/3632083085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2318156ca0 0x7f23180ccac0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:48.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.619+0000 7f2317e00640 1 -- 172.21.15.100:0/3632083085 >> 172.21.15.100:0/3632083085 conn(0x7f231806d620 msgr2=0x7f23180b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:48.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.619+0000 7f2317e00640 1 -- 172.21.15.100:0/3632083085 shutdown_connections 2024-09-17T04:24:48.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.619+0000 7f2317e00640 1 -- 172.21.15.100:0/3632083085 wait complete. 2024-09-17T04:24:48.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.621+0000 7f2317e00640 1 Processor -- start 2024-09-17T04:24:48.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.622+0000 7f2317e00640 1 -- start start 2024-09-17T04:24:48.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.623+0000 7f2317e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23180b3f30 0x7f231809c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:48.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.623+0000 7f2317e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23180cd000 0x7f2318155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:48.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.623+0000 7f2317e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2318156ca0 0x7f2318157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:48.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.623+0000 7f2317e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f23180af7e0 con 0x7f2318156ca0 2024-09-17T04:24:48.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.623+0000 7f2317e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2318070370 con 0x7f23180b3f30 2024-09-17T04:24:48.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.623+0000 7f2317e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f23180af960 con 0x7f23180cd000 2024-09-17T04:24:48.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.623+0000 7f2317400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2318156ca0 0x7f2318157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:48.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.623+0000 7f2317400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2318156ca0 0x7f2318157300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56818/0 (socket says 172.21.15.100:56818) 2024-09-17T04:24:48.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.623+0000 7f2317400640 1 -- 172.21.15.100:0/2462999089 learned_addr learned my addr 172.21.15.100:0/2462999089 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:48.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.623+0000 7f2316a00640 1 --2- 172.21.15.100:0/2462999089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23180b3f30 0x7f231809c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:48.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.623+0000 7f230e000640 1 --2- 172.21.15.100:0/2462999089 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23180cd000 0x7f2318155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:48.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.624+0000 7f2317400640 1 -- 172.21.15.100:0/2462999089 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23180cd000 msgr2=0x7f2318155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:48.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.624+0000 7f2317400640 1 --2- 172.21.15.100:0/2462999089 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23180cd000 0x7f2318155fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:48.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.624+0000 7f2317400640 1 -- 172.21.15.100:0/2462999089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23180b3f30 msgr2=0x7f231809c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:48.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.624+0000 7f2317400640 1 --2- 172.21.15.100:0/2462999089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23180b3f30 0x7f231809c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:48.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.624+0000 7f2317400640 1 -- 172.21.15.100:0/2462999089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f231809d0d0 con 0x7f2318156ca0 2024-09-17T04:24:48.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.624+0000 7f2316a00640 1 --2- 172.21.15.100:0/2462999089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23180b3f30 0x7f231809c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:24:48.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.624+0000 7f2317400640 1 --2- 172.21.15.100:0/2462999089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2318156ca0 0x7f2318157300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f231000cab0 tx=0x7f231000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:48.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.625+0000 7f230f400640 1 -- 172.21.15.100:0/2462999089 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2310016020 con 0x7f2318156ca0 2024-09-17T04:24:48.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.625+0000 7f230f400640 1 -- 172.21.15.100:0/2462999089 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f231000ad40 con 0x7f2318156ca0 2024-09-17T04:24:48.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.625+0000 7f2317e00640 1 -- 172.21.15.100:0/2462999089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f231809d3b0 con 0x7f2318156ca0 2024-09-17T04:24:48.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.625+0000 7f230f400640 1 -- 172.21.15.100:0/2462999089 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2310012680 con 0x7f2318156ca0 2024-09-17T04:24:48.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.625+0000 7f2317e00640 1 -- 172.21.15.100:0/2462999089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2318157840 con 0x7f2318156ca0 2024-09-17T04:24:48.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.626+0000 7f2317e00640 1 -- 172.21.15.100:0/2462999089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23181567e0 con 0x7f2318156ca0 2024-09-17T04:24:48.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.627+0000 7f230f400640 1 -- 172.21.15.100:0/2462999089 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2310019020 con 0x7f2318156ca0 2024-09-17T04:24:48.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.628+0000 7f230f400640 1 --2- 172.21.15.100:0/2462999089 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f22f4078b90 0x7f22f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:48.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.628+0000 7f230f400640 1 -- 172.21.15.100:0/2462999089 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f231009b260 con 0x7f2318156ca0 2024-09-17T04:24:48.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.628+0000 7f2316a00640 1 --2- 172.21.15.100:0/2462999089 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f22f4078b90 0x7f22f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:48.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.629+0000 7f2316a00640 1 --2- 172.21.15.100:0/2462999089 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f22f4078b90 0x7f22f407b0a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2308000c00 tx=0x7f2308009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:48.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.632+0000 7f230f400640 1 -- 172.21.15.100:0/2462999089 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f2310020020 con 0x7f2318156ca0 2024-09-17T04:24:48.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.814+0000 7f2317e00640 1 -- 172.21.15.100:0/2462999089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f231815e4d0 con 0x7f2318156ca0 2024-09-17T04:24:48.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.814+0000 7f230f400640 1 -- 172.21.15.100:0/2462999089 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f2310063040 con 0x7f2318156ca0 2024-09-17T04:24:48.814 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-09-17T04:24:48.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.817+0000 7f2317e00640 1 -- 172.21.15.100:0/2462999089 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f22f4078b90 msgr2=0x7f22f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:48.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.817+0000 7f2317e00640 1 --2- 172.21.15.100:0/2462999089 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f22f4078b90 0x7f22f407b0a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2308000c00 tx=0x7f2308009040 comp rx=0 tx=0).stop 2024-09-17T04:24:48.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.817+0000 7f2317e00640 1 -- 172.21.15.100:0/2462999089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2318156ca0 msgr2=0x7f2318157300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:48.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.817+0000 7f2317e00640 1 --2- 172.21.15.100:0/2462999089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2318156ca0 0x7f2318157300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f231000cab0 tx=0x7f231000cf70 comp rx=0 tx=0).stop 2024-09-17T04:24:48.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.817+0000 7f2317e00640 1 -- 172.21.15.100:0/2462999089 shutdown_connections 2024-09-17T04:24:48.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.817+0000 7f2317e00640 1 --2- 172.21.15.100:0/2462999089 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f22f4078b90 0x7f22f407b0a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:48.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.817+0000 7f2317e00640 1 --2- 172.21.15.100:0/2462999089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2318156ca0 0x7f2318157300 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:48.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.817+0000 7f2317e00640 1 --2- 172.21.15.100:0/2462999089 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23180cd000 0x7f2318155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:48.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.817+0000 7f2317e00640 1 --2- 172.21.15.100:0/2462999089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23180b3f30 0x7f231809c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:48.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.817+0000 7f2317e00640 1 -- 172.21.15.100:0/2462999089 >> 172.21.15.100:0/2462999089 conn(0x7f231806d620 msgr2=0x7f23180af0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:48.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.817+0000 7f2317e00640 1 -- 172.21.15.100:0/2462999089 shutdown_connections 2024-09-17T04:24:48.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:48.817+0000 7f2317e00640 1 -- 172.21.15.100:0/2462999089 wait complete. 2024-09-17T04:24:48.841 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2024-09-17T04:24:49.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.062+0000 7f8d22600640 1 -- 172.21.15.100:0/3566400817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d1c06f9a0 msgr2=0x7f8d1c0b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:49.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.062+0000 7f8d22600640 1 --2- 172.21.15.100:0/3566400817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d1c06f9a0 0x7f8d1c0b2f60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009be0 tx=0x7f8d1402a890 comp rx=0 tx=0).stop 2024-09-17T04:24:49.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.062+0000 7f8d22600640 1 -- 172.21.15.100:0/3566400817 shutdown_connections 2024-09-17T04:24:49.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.062+0000 7f8d22600640 1 --2- 172.21.15.100:0/3566400817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d1c06f9a0 0x7f8d1c0b2f60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:49.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.062+0000 7f8d22600640 1 -- 172.21.15.100:0/3566400817 >> 172.21.15.100:0/3566400817 conn(0x7f8d1c0b1e40 msgr2=0x7f8d1c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:49.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.063+0000 7f8d22600640 1 -- 172.21.15.100:0/3566400817 shutdown_connections 2024-09-17T04:24:49.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.063+0000 7f8d22600640 1 -- 172.21.15.100:0/3566400817 wait complete. 2024-09-17T04:24:49.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.065+0000 7f8d22600640 1 Processor -- start 2024-09-17T04:24:49.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.065+0000 7f8d22600640 1 -- start start 2024-09-17T04:24:49.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.066+0000 7f8d22600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d1c0ad9c0 0x7f8d1c0c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:49.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.066+0000 7f8d22600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d1c0af170 0x7f8d1c0cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:49.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.066+0000 7f8d22600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d1c0cbd00 0x7f8d1c073d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:49.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.066+0000 7f8d22600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c15a560 con 0x7f8d1c0cbd00 2024-09-17T04:24:49.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.066+0000 7f8d22600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c15a3e0 con 0x7f8d1c0af170 2024-09-17T04:24:49.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.066+0000 7f8d22600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8d1c15d670 con 0x7f8d1c0ad9c0 2024-09-17T04:24:49.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.067+0000 7f8d21c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d1c0cbd00 0x7f8d1c073d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:49.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.067+0000 7f8d21c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d1c0cbd00 0x7f8d1c073d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56820/0 (socket says 172.21.15.100:56820) 2024-09-17T04:24:49.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.067+0000 7f8d21c00640 1 -- 172.21.15.100:0/2349897281 learned_addr learned my addr 172.21.15.100:0/2349897281 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:49.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.067+0000 7f8d1be00640 1 --2- 172.21.15.100:0/2349897281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d1c0af170 0x7f8d1c0cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:49.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.067+0000 7f8d21200640 1 --2- 172.21.15.100:0/2349897281 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d1c0ad9c0 0x7f8d1c0c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:49.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.067+0000 7f8d21c00640 1 -- 172.21.15.100:0/2349897281 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d1c0ad9c0 msgr2=0x7f8d1c0c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:49.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.067+0000 7f8d21c00640 1 --2- 172.21.15.100:0/2349897281 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d1c0ad9c0 0x7f8d1c0c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:49.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.067+0000 7f8d21c00640 1 -- 172.21.15.100:0/2349897281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d1c0af170 msgr2=0x7f8d1c0cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:49.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.067+0000 7f8d21c00640 1 --2- 172.21.15.100:0/2349897281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d1c0af170 0x7f8d1c0cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:49.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.067+0000 7f8d21c00640 1 -- 172.21.15.100:0/2349897281 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d1c072f90 con 0x7f8d1c0cbd00 2024-09-17T04:24:49.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.067+0000 7f8d21c00640 1 --2- 172.21.15.100:0/2349897281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d1c0cbd00 0x7f8d1c073d30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009d00 tx=0x7f8d14033d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:49.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.068+0000 7f8d18c00640 1 -- 172.21.15.100:0/2349897281 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d14031d80 con 0x7f8d1c0cbd00 2024-09-17T04:24:49.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.068+0000 7f8d18c00640 1 -- 172.21.15.100:0/2349897281 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d14003a80 con 0x7f8d1c0cbd00 2024-09-17T04:24:49.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.068+0000 7f8d22600640 1 -- 172.21.15.100:0/2349897281 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d1c073270 con 0x7f8d1c0cbd00 2024-09-17T04:24:49.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.068+0000 7f8d18c00640 1 -- 172.21.15.100:0/2349897281 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d140330d0 con 0x7f8d1c0cbd00 2024-09-17T04:24:49.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.068+0000 7f8d22600640 1 -- 172.21.15.100:0/2349897281 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d1c073830 con 0x7f8d1c0cbd00 2024-09-17T04:24:49.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.069+0000 7f8d22600640 1 -- 172.21.15.100:0/2349897281 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d1c0b2f60 con 0x7f8d1c0cbd00 2024-09-17T04:24:49.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.070+0000 7f8d18c00640 1 -- 172.21.15.100:0/2349897281 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8d14033270 con 0x7f8d1c0cbd00 2024-09-17T04:24:49.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.071+0000 7f8d18c00640 1 --2- 172.21.15.100:0/2349897281 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8cfc078bc0 0x7f8cfc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:49.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.071+0000 7f8d18c00640 1 -- 172.21.15.100:0/2349897281 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f8d140bbdd0 con 0x7f8d1c0cbd00 2024-09-17T04:24:49.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.072+0000 7f8d21200640 1 --2- 172.21.15.100:0/2349897281 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8cfc078bc0 0x7f8cfc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:49.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.072+0000 7f8d21200640 1 --2- 172.21.15.100:0/2349897281 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8cfc078bc0 0x7f8cfc07b0d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8d10001610 tx=0x7f8d10009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:49.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.075+0000 7f8d18c00640 1 -- 172.21.15.100:0/2349897281 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8d1403f030 con 0x7f8d1c0cbd00 2024-09-17T04:24:49.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.327+0000 7f8d22600640 1 -- 172.21.15.100:0/2349897281 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f8d1c070870 con 0x7f8d1c0cbd00 2024-09-17T04:24:49.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.543+0000 7f8d18c00640 1 -- 172.21.15.100:0/2349897281 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v58) ==== 161+0+0 (secure 0 0 0) 0x7f8d14031790 con 0x7f8d1c0cbd00 2024-09-17T04:24:49.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.546+0000 7f8d22600640 1 -- 172.21.15.100:0/2349897281 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8cfc078bc0 msgr2=0x7f8cfc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:49.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.546+0000 7f8d22600640 1 --2- 172.21.15.100:0/2349897281 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8cfc078bc0 0x7f8cfc07b0d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8d10001610 tx=0x7f8d10009040 comp rx=0 tx=0).stop 2024-09-17T04:24:49.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.546+0000 7f8d22600640 1 -- 172.21.15.100:0/2349897281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d1c0cbd00 msgr2=0x7f8d1c073d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:49.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.546+0000 7f8d22600640 1 --2- 172.21.15.100:0/2349897281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d1c0cbd00 0x7f8d1c073d30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009d00 tx=0x7f8d14033d80 comp rx=0 tx=0).stop 2024-09-17T04:24:49.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.546+0000 7f8d22600640 1 -- 172.21.15.100:0/2349897281 shutdown_connections 2024-09-17T04:24:49.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.546+0000 7f8d22600640 1 --2- 172.21.15.100:0/2349897281 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8cfc078bc0 0x7f8cfc07b0d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:49.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.546+0000 7f8d22600640 1 --2- 172.21.15.100:0/2349897281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d1c0cbd00 0x7f8d1c073d30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:49.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.546+0000 7f8d22600640 1 --2- 172.21.15.100:0/2349897281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d1c0af170 0x7f8d1c0cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:49.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.546+0000 7f8d22600640 1 --2- 172.21.15.100:0/2349897281 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d1c0ad9c0 0x7f8d1c0c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:49.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.546+0000 7f8d22600640 1 -- 172.21.15.100:0/2349897281 >> 172.21.15.100:0/2349897281 conn(0x7f8d1c0b1e40 msgr2=0x7f8d1c15bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:49.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.546+0000 7f8d22600640 1 -- 172.21.15.100:0/2349897281 shutdown_connections 2024-09-17T04:24:49.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.546+0000 7f8d22600640 1 -- 172.21.15.100:0/2349897281 wait complete. 2024-09-17T04:24:49.570 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T04:24:49.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.791+0000 7f86d7200640 1 -- 172.21.15.100:0/4056951651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86b4005350 msgr2=0x7f86b400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:49.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.791+0000 7f86d7200640 1 --2- 172.21.15.100:0/4056951651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86b4005350 0x7f86b400b9c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f86d00ae780 tx=0x7f86b4005830 comp rx=0 tx=0).stop 2024-09-17T04:24:49.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.791+0000 7f86d7200640 1 -- 172.21.15.100:0/4056951651 shutdown_connections 2024-09-17T04:24:49.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.791+0000 7f86d7200640 1 --2- 172.21.15.100:0/4056951651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86b4005350 0x7f86b400b9c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:49.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.791+0000 7f86d7200640 1 -- 172.21.15.100:0/4056951651 >> 172.21.15.100:0/4056951651 conn(0x7f86d006d730 msgr2=0x7f86d00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:49.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.791+0000 7f86d7200640 1 -- 172.21.15.100:0/4056951651 shutdown_connections 2024-09-17T04:24:49.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.791+0000 7f86d7200640 1 -- 172.21.15.100:0/4056951651 wait complete. 2024-09-17T04:24:49.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.793+0000 7f86d7200640 1 Processor -- start 2024-09-17T04:24:49.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.793+0000 7f86d7200640 1 -- start start 2024-09-17T04:24:49.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.795+0000 7f86d7200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b4005350 0x7f86d014f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:49.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.795+0000 7f86d7200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86d009dbb0 0x7f86d01f4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:49.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.795+0000 7f86d7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86d01f4e60 0x7f86d01f3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:49.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.795+0000 7f86d7200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f86d00ad4f0 con 0x7f86d01f4e60 2024-09-17T04:24:49.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.795+0000 7f86d7200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f86d00ad670 con 0x7f86d009dbb0 2024-09-17T04:24:49.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.795+0000 7f86d7200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f86d00ad7f0 con 0x7f86b4005350 2024-09-17T04:24:49.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.795+0000 7f86d5200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86d01f4e60 0x7f86d01f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:49.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.795+0000 7f86d5200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86d01f4e60 0x7f86d01f3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56848/0 (socket says 172.21.15.100:56848) 2024-09-17T04:24:49.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.795+0000 7f86d5200640 1 -- 172.21.15.100:0/2851711467 learned_addr learned my addr 172.21.15.100:0/2851711467 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:49.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.795+0000 7f86cfe00640 1 --2- 172.21.15.100:0/2851711467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b4005350 0x7f86d014f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:49.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.795+0000 7f86c7400640 1 --2- 172.21.15.100:0/2851711467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86d009dbb0 0x7f86d01f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:49.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.796+0000 7f86d5200640 1 -- 172.21.15.100:0/2851711467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b4005350 msgr2=0x7f86d014f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:49.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.796+0000 7f86d5200640 1 --2- 172.21.15.100:0/2851711467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b4005350 0x7f86d014f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:49.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.796+0000 7f86d5200640 1 -- 172.21.15.100:0/2851711467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86d009dbb0 msgr2=0x7f86d01f4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:49.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.796+0000 7f86d5200640 1 --2- 172.21.15.100:0/2851711467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86d009dbb0 0x7f86d01f4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:49.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.796+0000 7f86d5200640 1 -- 172.21.15.100:0/2851711467 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d014f6f0 con 0x7f86d01f4e60 2024-09-17T04:24:49.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.796+0000 7f86d5200640 1 --2- 172.21.15.100:0/2851711467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86d01f4e60 0x7f86d01f3760 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f86c802b4b0 tx=0x7f86c8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:49.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.796+0000 7f86ccc00640 1 -- 172.21.15.100:0/2851711467 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c8039070 con 0x7f86d01f4e60 2024-09-17T04:24:49.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.796+0000 7f86ccc00640 1 -- 172.21.15.100:0/2851711467 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86c802bab0 con 0x7f86d01f4e60 2024-09-17T04:24:49.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.796+0000 7f86d7200640 1 -- 172.21.15.100:0/2851711467 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d014dc70 con 0x7f86d01f4e60 2024-09-17T04:24:49.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.796+0000 7f86ccc00640 1 -- 172.21.15.100:0/2851711467 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c8002080 con 0x7f86d01f4e60 2024-09-17T04:24:49.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.796+0000 7f86d7200640 1 -- 172.21.15.100:0/2851711467 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d014e3f0 con 0x7f86d01f4e60 2024-09-17T04:24:49.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.798+0000 7f86d7200640 1 -- 172.21.15.100:0/2851711467 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86d0153730 con 0x7f86d01f4e60 2024-09-17T04:24:49.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.799+0000 7f86ccc00640 1 -- 172.21.15.100:0/2851711467 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f86c8006140 con 0x7f86d01f4e60 2024-09-17T04:24:49.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.799+0000 7f86ccc00640 1 --2- 172.21.15.100:0/2851711467 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f86ac078b90 0x7f86ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:49.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.800+0000 7f86ccc00640 1 -- 172.21.15.100:0/2851711467 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f86c80ba890 con 0x7f86d01f4e60 2024-09-17T04:24:49.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.800+0000 7f86cfe00640 1 --2- 172.21.15.100:0/2851711467 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f86ac078b90 0x7f86ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:49.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.801+0000 7f86cfe00640 1 --2- 172.21.15.100:0/2851711467 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f86ac078b90 0x7f86ac07b0a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f86c0009bd0 tx=0x7f86c0005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:49.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.804+0000 7f86ccc00640 1 -- 172.21.15.100:0/2851711467 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f86c8082e00 con 0x7f86d01f4e60 2024-09-17T04:24:49.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.990+0000 7f86d7200640 1 -- 172.21.15.100:0/2851711467 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f86d014de40 con 0x7f86d01f4e60 2024-09-17T04:24:49.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.990+0000 7f86ccc00640 1 -- 172.21.15.100:0/2851711467 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v31) ==== 59+0+2294 (secure 0 0 0) 0x7f86c8082700 con 0x7f86d01f4e60 2024-09-17T04:24:49.990 INFO:teuthology.orchestra.run.smithi100.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T04:24:49.990 INFO:teuthology.orchestra.run.smithi100.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T04:24:49.990 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced client_mount_timeout 30 2024-09-17T04:24:49.990 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_client 20/20 2024-09-17T04:24:49.990 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_mgr 20/20 2024-09-17T04:24:49.990 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_ms 1/1 2024-09-17T04:24:49.990 INFO:teuthology.orchestra.run.smithi100.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T04:24:49.990 INFO:teuthology.orchestra.run.smithi100.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T04:24:49.990 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3938.598654 2024-09-17T04:24:49.991 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4008.640154 2024-09-17T04:24:49.991 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3706.588546 2024-09-17T04:24:49.991 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 6299.338665 2024-09-17T04:24:49.991 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3633.876699 2024-09-17T04:24:49.991 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3674.821841 2024-09-17T04:24:49.991 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3789.849984 2024-09-17T04:24:49.991 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3467.457298 2024-09-17T04:24:49.991 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_mds 20/20 2024-09-17T04:24:49.991 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T04:24:49.991 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_ms 2/2 2024-09-17T04:24:49.991 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_debug_frag true 2024-09-17T04:24:49.992 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_debug_scatterstat true 2024-09-17T04:24:49.992 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T04:24:49.992 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_verify_scatter true 2024-09-17T04:24:49.992 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T04:24:49.992 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T04:24:49.992 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T04:24:49.992 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced client_mount_timeout 600 2024-09-17T04:24:49.992 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced debug_client 20/20 2024-09-17T04:24:49.992 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced debug_ms 1/1 2024-09-17T04:24:49.992 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T04:24:49.992 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T04:24:49.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.993+0000 7f86d7200640 1 -- 172.21.15.100:0/2851711467 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f86ac078b90 msgr2=0x7f86ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:49.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.993+0000 7f86d7200640 1 --2- 172.21.15.100:0/2851711467 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f86ac078b90 0x7f86ac07b0a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f86c0009bd0 tx=0x7f86c0005f90 comp rx=0 tx=0).stop 2024-09-17T04:24:49.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.993+0000 7f86d7200640 1 -- 172.21.15.100:0/2851711467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86d01f4e60 msgr2=0x7f86d01f3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:49.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.993+0000 7f86d7200640 1 --2- 172.21.15.100:0/2851711467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86d01f4e60 0x7f86d01f3760 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f86c802b4b0 tx=0x7f86c8003ba0 comp rx=0 tx=0).stop 2024-09-17T04:24:49.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.993+0000 7f86d7200640 1 -- 172.21.15.100:0/2851711467 shutdown_connections 2024-09-17T04:24:49.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.993+0000 7f86d7200640 1 --2- 172.21.15.100:0/2851711467 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f86ac078b90 0x7f86ac07b0a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:49.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.993+0000 7f86d7200640 1 --2- 172.21.15.100:0/2851711467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86d01f4e60 0x7f86d01f3760 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:49.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.993+0000 7f86d7200640 1 --2- 172.21.15.100:0/2851711467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86d009dbb0 0x7f86d01f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:49.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.993+0000 7f86d7200640 1 --2- 172.21.15.100:0/2851711467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b4005350 0x7f86d014f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:49.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.993+0000 7f86d7200640 1 -- 172.21.15.100:0/2851711467 >> 172.21.15.100:0/2851711467 conn(0x7f86d006d730 msgr2=0x7f86d00b1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:49.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.993+0000 7f86d7200640 1 -- 172.21.15.100:0/2851711467 shutdown_connections 2024-09-17T04:24:49.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:49.993+0000 7f86d7200640 1 -- 172.21.15.100:0/2851711467 wait complete. 2024-09-17T04:24:50.015 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T04:24:50.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.238+0000 7fec28400640 1 -- 172.21.15.100:0/2990717040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec20145550 msgr2=0x7fec2006fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:50.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.239+0000 7fec28400640 1 --2- 172.21.15.100:0/2990717040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec20145550 0x7fec2006fb40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fec20160280 tx=0x7fec1402a5d0 comp rx=0 tx=0).stop 2024-09-17T04:24:50.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.239+0000 7fec28400640 1 -- 172.21.15.100:0/2990717040 shutdown_connections 2024-09-17T04:24:50.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.239+0000 7fec28400640 1 --2- 172.21.15.100:0/2990717040 >> v2:172.21.15.184:3300/0 conn(0x7fec20145a40 0x7fec20073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:50.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.239+0000 7fec28400640 1 --2- 172.21.15.100:0/2990717040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec20145550 0x7fec2006fb40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:50.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.239+0000 7fec28400640 1 -- 172.21.15.100:0/2990717040 >> 172.21.15.100:0/2990717040 conn(0x7fec2006d1c0 msgr2=0x7fec2006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:50.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.239+0000 7fec28400640 1 -- 172.21.15.100:0/2990717040 shutdown_connections 2024-09-17T04:24:50.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.239+0000 7fec28400640 1 -- 172.21.15.100:0/2990717040 wait complete. 2024-09-17T04:24:50.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.241+0000 7fec28400640 1 Processor -- start 2024-09-17T04:24:50.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.241+0000 7fec28400640 1 -- start start 2024-09-17T04:24:50.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.243+0000 7fec28400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec20073a80 0x7fec2013aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:50.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.243+0000 7fec28400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec20145550 0x7fec200b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:50.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.243+0000 7fec28400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec20145a40 0x7fec200b4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:50.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.243+0000 7fec28400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fec20160a30 con 0x7fec20073a80 2024-09-17T04:24:50.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.243+0000 7fec28400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fec200ce870 con 0x7fec20145550 2024-09-17T04:24:50.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.243+0000 7fec28400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fec20160bb0 con 0x7fec20145a40 2024-09-17T04:24:50.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.243+0000 7fec25a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec20073a80 0x7fec2013aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:50.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.243+0000 7fec25a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec20073a80 0x7fec2013aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56878/0 (socket says 172.21.15.100:56878) 2024-09-17T04:24:50.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.243+0000 7fec25a00640 1 -- 172.21.15.100:0/53475706 learned_addr learned my addr 172.21.15.100:0/53475706 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:50.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.243+0000 7fec25000640 1 --2- 172.21.15.100:0/53475706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec20145550 0x7fec200b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:50.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.243+0000 7fec26400640 1 --2- 172.21.15.100:0/53475706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec20145a40 0x7fec200b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:50.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.243+0000 7fec25a00640 1 -- 172.21.15.100:0/53475706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec20145a40 msgr2=0x7fec200b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:50.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.243+0000 7fec25a00640 1 --2- 172.21.15.100:0/53475706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec20145a40 0x7fec200b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:50.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.243+0000 7fec25a00640 1 -- 172.21.15.100:0/53475706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec20145550 msgr2=0x7fec200b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:50.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.243+0000 7fec25a00640 1 --2- 172.21.15.100:0/53475706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec20145550 0x7fec200b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:50.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.243+0000 7fec25a00640 1 -- 172.21.15.100:0/53475706 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2015eb90 con 0x7fec20073a80 2024-09-17T04:24:50.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.244+0000 7fec25a00640 1 --2- 172.21.15.100:0/53475706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec20073a80 0x7fec2013aae0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fec1402abc0 tx=0x7fec14031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:50.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.244+0000 7fec1d600640 1 -- 172.21.15.100:0/53475706 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec14033e00 con 0x7fec20073a80 2024-09-17T04:24:50.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.244+0000 7fec1d600640 1 -- 172.21.15.100:0/53475706 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec14031a40 con 0x7fec20073a80 2024-09-17T04:24:50.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.244+0000 7fec28400640 1 -- 172.21.15.100:0/53475706 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec2015ee70 con 0x7fec20073a80 2024-09-17T04:24:50.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.244+0000 7fec1d600640 1 -- 172.21.15.100:0/53475706 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec14005020 con 0x7fec20073a80 2024-09-17T04:24:50.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.244+0000 7fec28400640 1 -- 172.21.15.100:0/53475706 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec200af0c0 con 0x7fec20073a80 2024-09-17T04:24:50.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.245+0000 7fec28400640 1 -- 172.21.15.100:0/53475706 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec200af320 con 0x7fec20073a80 2024-09-17T04:24:50.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.246+0000 7fec1d600640 1 -- 172.21.15.100:0/53475706 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fec140051c0 con 0x7fec20073a80 2024-09-17T04:24:50.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.247+0000 7fec1d600640 1 --2- 172.21.15.100:0/53475706 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fec00078bc0 0x7fec0007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:50.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.247+0000 7fec1d600640 1 -- 172.21.15.100:0/53475706 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fec140bbb80 con 0x7fec20073a80 2024-09-17T04:24:50.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.248+0000 7fec25000640 1 --2- 172.21.15.100:0/53475706 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fec00078bc0 0x7fec0007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:50.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.249+0000 7fec25000640 1 --2- 172.21.15.100:0/53475706 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fec00078bc0 0x7fec0007b0d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fec200b5760 tx=0x7fec0c00b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:50.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.252+0000 7fec1d600640 1 -- 172.21.15.100:0/53475706 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fec1403f030 con 0x7fec20073a80 2024-09-17T04:24:50.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.436+0000 7fec28400640 1 -- 172.21.15.100:0/53475706 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fec201610d0 con 0x7fec20073a80 2024-09-17T04:24:50.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.440+0000 7fec1d600640 1 -- 172.21.15.100:0/53475706 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7fec14083960 con 0x7fec20073a80 2024-09-17T04:24:50.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.442+0000 7fec28400640 1 -- 172.21.15.100:0/53475706 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fec00078bc0 msgr2=0x7fec0007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:50.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.442+0000 7fec28400640 1 --2- 172.21.15.100:0/53475706 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fec00078bc0 0x7fec0007b0d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fec200b5760 tx=0x7fec0c00b2e0 comp rx=0 tx=0).stop 2024-09-17T04:24:50.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.442+0000 7fec28400640 1 -- 172.21.15.100:0/53475706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec20073a80 msgr2=0x7fec2013aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:50.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.442+0000 7fec28400640 1 --2- 172.21.15.100:0/53475706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec20073a80 0x7fec2013aae0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fec1402abc0 tx=0x7fec14031630 comp rx=0 tx=0).stop 2024-09-17T04:24:50.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.443+0000 7fec28400640 1 -- 172.21.15.100:0/53475706 shutdown_connections 2024-09-17T04:24:50.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.443+0000 7fec28400640 1 --2- 172.21.15.100:0/53475706 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fec00078bc0 0x7fec0007b0d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:50.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.443+0000 7fec28400640 1 --2- 172.21.15.100:0/53475706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec20145a40 0x7fec200b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:50.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.443+0000 7fec28400640 1 --2- 172.21.15.100:0/53475706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec20145550 0x7fec200b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:50.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.443+0000 7fec28400640 1 --2- 172.21.15.100:0/53475706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec20073a80 0x7fec2013aae0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:50.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.443+0000 7fec28400640 1 -- 172.21.15.100:0/53475706 >> 172.21.15.100:0/53475706 conn(0x7fec2006d1c0 msgr2=0x7fec20068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:50.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.443+0000 7fec28400640 1 -- 172.21.15.100:0/53475706 shutdown_connections 2024-09-17T04:24:50.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:50.443+0000 7fec28400640 1 -- 172.21.15.100:0/53475706 wait complete. 2024-09-17T04:24:50.464 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T04:24:50.464 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:24:50.557 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:24:50.668 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T04:24:50.668 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:24:50.668 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:24:50.705 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:24:50.705 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:24:50.781 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:24:50.781 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:24:50.856 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:24:51.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.117+0000 7fc0d1800640 1 -- 172.21.15.100:0/2740321377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0cc15e770 msgr2=0x7fc0cc0b7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:51.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.117+0000 7fc0d1800640 1 --2- 172.21.15.100:0/2740321377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0cc15e770 0x7fc0cc0b7a00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4009830 tx=0x7fc0b402a480 comp rx=0 tx=0).stop 2024-09-17T04:24:51.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.117+0000 7fc0d1800640 1 -- 172.21.15.100:0/2740321377 shutdown_connections 2024-09-17T04:24:51.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.117+0000 7fc0d1800640 1 --2- 172.21.15.100:0/2740321377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0cc15e770 0x7fc0cc0b7a00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:51.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.117+0000 7fc0d1800640 1 --2- 172.21.15.100:0/2740321377 >> v2:172.21.15.184:3300/0 conn(0x7fc0cc0cf6c0 0x7fc0cc162e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:51.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.117+0000 7fc0d1800640 1 -- 172.21.15.100:0/2740321377 >> 172.21.15.100:0/2740321377 conn(0x7fc0cc0d44f0 msgr2=0x7fc0cc0d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:51.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.117+0000 7fc0d1800640 1 -- 172.21.15.100:0/2740321377 shutdown_connections 2024-09-17T04:24:51.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.117+0000 7fc0d1800640 1 -- 172.21.15.100:0/2740321377 wait complete. 2024-09-17T04:24:51.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.120+0000 7fc0d1800640 1 Processor -- start 2024-09-17T04:24:51.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.120+0000 7fc0d1800640 1 -- start start 2024-09-17T04:24:51.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.121+0000 7fc0d1800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0cc0b7f40 0x7fc0cc1f5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:51.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.121+0000 7fc0d1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0cc15e770 0x7fc0cc1f8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:51.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.121+0000 7fc0d1800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0cc14cce0 0x7fc0cc1f8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:51.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.121+0000 7fc0d1800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc0cc0d39a0 con 0x7fc0cc15e770 2024-09-17T04:24:51.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.121+0000 7fc0d1800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc0cc0d3820 con 0x7fc0cc14cce0 2024-09-17T04:24:51.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.121+0000 7fc0d1800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc0cc0d3b20 con 0x7fc0cc0b7f40 2024-09-17T04:24:51.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.122+0000 7fc0c3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0cc15e770 0x7fc0cc1f8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:51.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.122+0000 7fc0c3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0cc15e770 0x7fc0cc1f8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56886/0 (socket says 172.21.15.100:56886) 2024-09-17T04:24:51.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.122+0000 7fc0c3400640 1 -- 172.21.15.100:0/1496334099 learned_addr learned my addr 172.21.15.100:0/1496334099 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:51.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.122+0000 7fc0cbe00640 1 --2- 172.21.15.100:0/1496334099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0cc0b7f40 0x7fc0cc1f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:51.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.122+0000 7fc0d0e00640 1 --2- 172.21.15.100:0/1496334099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0cc14cce0 0x7fc0cc1f8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:51.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.122+0000 7fc0c3400640 1 -- 172.21.15.100:0/1496334099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0cc0b7f40 msgr2=0x7fc0cc1f5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:51.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.122+0000 7fc0c3400640 1 --2- 172.21.15.100:0/1496334099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0cc0b7f40 0x7fc0cc1f5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:51.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.122+0000 7fc0c3400640 1 -- 172.21.15.100:0/1496334099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0cc14cce0 msgr2=0x7fc0cc1f8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:51.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.122+0000 7fc0c3400640 1 --2- 172.21.15.100:0/1496334099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0cc14cce0 0x7fc0cc1f8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:51.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.122+0000 7fc0c3400640 1 -- 172.21.15.100:0/1496334099 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0cc0aaf80 con 0x7fc0cc15e770 2024-09-17T04:24:51.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.122+0000 7fc0c3400640 1 --2- 172.21.15.100:0/1496334099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0cc15e770 0x7fc0cc1f8970 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4000c00 tx=0x7fc0b4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:51.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.123+0000 7fc0c8c00640 1 -- 172.21.15.100:0/1496334099 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4031d60 con 0x7fc0cc15e770 2024-09-17T04:24:51.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.123+0000 7fc0c8c00640 1 -- 172.21.15.100:0/1496334099 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0b4033e10 con 0x7fc0cc15e770 2024-09-17T04:24:51.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.123+0000 7fc0d1800640 1 -- 172.21.15.100:0/1496334099 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0cc0ab260 con 0x7fc0cc15e770 2024-09-17T04:24:51.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.123+0000 7fc0c8c00640 1 -- 172.21.15.100:0/1496334099 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4033310 con 0x7fc0cc15e770 2024-09-17T04:24:51.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.123+0000 7fc0d1800640 1 -- 172.21.15.100:0/1496334099 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0cc0ab820 con 0x7fc0cc15e770 2024-09-17T04:24:51.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.124+0000 7fc0d1800640 1 -- 172.21.15.100:0/1496334099 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0cc1f6170 con 0x7fc0cc15e770 2024-09-17T04:24:51.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.125+0000 7fc0c8c00640 1 -- 172.21.15.100:0/1496334099 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc0b40334b0 con 0x7fc0cc15e770 2024-09-17T04:24:51.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.126+0000 7fc0c8c00640 1 --2- 172.21.15.100:0/1496334099 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc0a8078b90 0x7fc0a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:51.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.126+0000 7fc0c8c00640 1 -- 172.21.15.100:0/1496334099 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc0b40bc180 con 0x7fc0cc15e770 2024-09-17T04:24:51.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.126+0000 7fc0cbe00640 1 --2- 172.21.15.100:0/1496334099 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc0a8078b90 0x7fc0a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:51.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.127+0000 7fc0cbe00640 1 --2- 172.21.15.100:0/1496334099 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc0a8078b90 0x7fc0a807b0a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009530 tx=0x7fc0bc009260 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:51.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.131+0000 7fc0c8c00640 1 -- 172.21.15.100:0/1496334099 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc0b403f030 con 0x7fc0cc15e770 2024-09-17T04:24:51.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.323+0000 7fc0d1800640 1 -- 172.21.15.100:0/1496334099 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc0cc0cf6c0 con 0x7fc0cc15e770 2024-09-17T04:24:51.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.323+0000 7fc0c8c00640 1 -- 172.21.15.100:0/1496334099 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fc0b4083ed0 con 0x7fc0cc15e770 2024-09-17T04:24:51.323 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-09-17T04:24:51.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.326+0000 7fc0d1800640 1 -- 172.21.15.100:0/1496334099 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc0a8078b90 msgr2=0x7fc0a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:51.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.326+0000 7fc0d1800640 1 --2- 172.21.15.100:0/1496334099 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc0a8078b90 0x7fc0a807b0a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009530 tx=0x7fc0bc009260 comp rx=0 tx=0).stop 2024-09-17T04:24:51.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.326+0000 7fc0d1800640 1 -- 172.21.15.100:0/1496334099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0cc15e770 msgr2=0x7fc0cc1f8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:51.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.326+0000 7fc0d1800640 1 --2- 172.21.15.100:0/1496334099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0cc15e770 0x7fc0cc1f8970 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4000c00 tx=0x7fc0b4033c20 comp rx=0 tx=0).stop 2024-09-17T04:24:51.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.326+0000 7fc0d1800640 1 -- 172.21.15.100:0/1496334099 shutdown_connections 2024-09-17T04:24:51.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.326+0000 7fc0d1800640 1 --2- 172.21.15.100:0/1496334099 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc0a8078b90 0x7fc0a807b0a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:51.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.326+0000 7fc0d1800640 1 --2- 172.21.15.100:0/1496334099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0cc14cce0 0x7fc0cc1f8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:51.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.326+0000 7fc0d1800640 1 --2- 172.21.15.100:0/1496334099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0cc15e770 0x7fc0cc1f8970 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:51.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.326+0000 7fc0d1800640 1 --2- 172.21.15.100:0/1496334099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0cc0b7f40 0x7fc0cc1f5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:51.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.326+0000 7fc0d1800640 1 -- 172.21.15.100:0/1496334099 >> 172.21.15.100:0/1496334099 conn(0x7fc0cc0d44f0 msgr2=0x7fc0cc15da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:51.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.326+0000 7fc0d1800640 1 -- 172.21.15.100:0/1496334099 shutdown_connections 2024-09-17T04:24:51.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.326+0000 7fc0d1800640 1 -- 172.21.15.100:0/1496334099 wait complete. 2024-09-17T04:24:51.349 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T04:24:51.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:51.350 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-17T04:24:51.410 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-09-17T04:24:51.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:51.410 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T04:24:51.453 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:24:51.453 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:24:51.494 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:24:51.494 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:24:51.574 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:24:51.574 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:24:51.652 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:24:51.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.875+0000 7f62d9200640 1 -- 172.21.15.100:0/2953345198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62d40cea50 msgr2=0x7f62d40d0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:51.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.875+0000 7f62d9200640 1 --2- 172.21.15.100:0/2953345198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62d40cea50 0x7f62d40d0270 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f62d40ce530 tx=0x7f62c402a580 comp rx=0 tx=0).stop 2024-09-17T04:24:51.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.875+0000 7f62d9200640 1 -- 172.21.15.100:0/2953345198 shutdown_connections 2024-09-17T04:24:51.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.875+0000 7f62d9200640 1 --2- 172.21.15.100:0/2953345198 >> v2:172.21.15.184:3300/0 conn(0x7f62d4160d90 0x7f62d41611a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:51.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.875+0000 7f62d9200640 1 --2- 172.21.15.100:0/2953345198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d40c5a90 0x7f62d4160850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:51.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.876+0000 7f62d9200640 1 --2- 172.21.15.100:0/2953345198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62d40cea50 0x7f62d40d0270 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:51.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.876+0000 7f62d9200640 1 -- 172.21.15.100:0/2953345198 >> 172.21.15.100:0/2953345198 conn(0x7f62d40cee80 msgr2=0x7f62d40cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:51.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.876+0000 7f62d9200640 1 -- 172.21.15.100:0/2953345198 shutdown_connections 2024-09-17T04:24:51.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.876+0000 7f62d9200640 1 -- 172.21.15.100:0/2953345198 wait complete. 2024-09-17T04:24:51.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.878+0000 7f62d9200640 1 Processor -- start 2024-09-17T04:24:51.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.878+0000 7f62d9200640 1 -- start start 2024-09-17T04:24:51.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.879+0000 7f62d9200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d40c5a90 0x7f62d409fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:51.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.879+0000 7f62d9200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62d40cea50 0x7f62d409e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:51.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.879+0000 7f62d9200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62d4160d90 0x7f62d414e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:51.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.879+0000 7f62d9200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f62d40b5250 con 0x7f62d40cea50 2024-09-17T04:24:51.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.879+0000 7f62d9200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f62d409bc70 con 0x7f62d4160d90 2024-09-17T04:24:51.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.879+0000 7f62d9200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f62d40b53d0 con 0x7f62d40c5a90 2024-09-17T04:24:51.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.880+0000 7f62d1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62d40cea50 0x7f62d409e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:51.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.880+0000 7f62d1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62d40cea50 0x7f62d409e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56902/0 (socket says 172.21.15.100:56902) 2024-09-17T04:24:51.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.880+0000 7f62d1400640 1 -- 172.21.15.100:0/1189414686 learned_addr learned my addr 172.21.15.100:0/1189414686 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:51.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.880+0000 7f62d1e00640 1 --2- 172.21.15.100:0/1189414686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d40c5a90 0x7f62d409fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:51.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.880+0000 7f62d2800640 1 --2- 172.21.15.100:0/1189414686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62d4160d90 0x7f62d414e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:51.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.880+0000 7f62d1400640 1 -- 172.21.15.100:0/1189414686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d40c5a90 msgr2=0x7f62d409fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:51.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.880+0000 7f62d1400640 1 --2- 172.21.15.100:0/1189414686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d40c5a90 0x7f62d409fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:51.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.880+0000 7f62d1400640 1 -- 172.21.15.100:0/1189414686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62d4160d90 msgr2=0x7f62d414e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:51.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.880+0000 7f62d1400640 1 --2- 172.21.15.100:0/1189414686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62d4160d90 0x7f62d414e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:51.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.880+0000 7f62d1400640 1 -- 172.21.15.100:0/1189414686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d409d5a0 con 0x7f62d40cea50 2024-09-17T04:24:51.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.880+0000 7f62d1400640 1 --2- 172.21.15.100:0/1189414686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62d40cea50 0x7f62d409e640 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f62c000e6d0 tx=0x7f62c000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:51.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.880+0000 7f62ca000640 1 -- 172.21.15.100:0/1189414686 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c000cab0 con 0x7f62d40cea50 2024-09-17T04:24:51.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.881+0000 7f62ca000640 1 -- 172.21.15.100:0/1189414686 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62c000cc50 con 0x7f62d40cea50 2024-09-17T04:24:51.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.881+0000 7f62d9200640 1 -- 172.21.15.100:0/1189414686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d409d880 con 0x7f62d40cea50 2024-09-17T04:24:51.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.881+0000 7f62ca000640 1 -- 172.21.15.100:0/1189414686 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0005020 con 0x7f62d40cea50 2024-09-17T04:24:51.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.881+0000 7f62d9200640 1 -- 172.21.15.100:0/1189414686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d409de40 con 0x7f62d40cea50 2024-09-17T04:24:51.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.882+0000 7f62d9200640 1 -- 172.21.15.100:0/1189414686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62d409d310 con 0x7f62d40cea50 2024-09-17T04:24:51.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.883+0000 7f62ca000640 1 -- 172.21.15.100:0/1189414686 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f62c00051c0 con 0x7f62d40cea50 2024-09-17T04:24:51.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.884+0000 7f62ca000640 1 --2- 172.21.15.100:0/1189414686 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f62b4078b90 0x7f62b407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:51.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.884+0000 7f62ca000640 1 -- 172.21.15.100:0/1189414686 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f62c009b6a0 con 0x7f62d40cea50 2024-09-17T04:24:51.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.885+0000 7f62d1e00640 1 --2- 172.21.15.100:0/1189414686 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f62b4078b90 0x7f62b407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:51.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.885+0000 7f62d1e00640 1 --2- 172.21.15.100:0/1189414686 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f62b4078b90 0x7f62b407b0a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f62c4034000 tx=0x7f62c4004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:51.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:51.888+0000 7f62ca000640 1 -- 172.21.15.100:0/1189414686 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f62c0063ad0 con 0x7f62d40cea50 2024-09-17T04:24:52.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.086+0000 7f62d9200640 1 -- 172.21.15.100:0/1189414686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f62d40b6180 con 0x7f62d40cea50 2024-09-17T04:24:52.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.086+0000 7f62ca000640 1 -- 172.21.15.100:0/1189414686 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f62d40b6180 con 0x7f62d40cea50 2024-09-17T04:24:52.085 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-09-17T04:24:52.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.088+0000 7f62d9200640 1 -- 172.21.15.100:0/1189414686 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f62b4078b90 msgr2=0x7f62b407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:52.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.088+0000 7f62d9200640 1 --2- 172.21.15.100:0/1189414686 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f62b4078b90 0x7f62b407b0a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f62c4034000 tx=0x7f62c4004730 comp rx=0 tx=0).stop 2024-09-17T04:24:52.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.088+0000 7f62d9200640 1 -- 172.21.15.100:0/1189414686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62d40cea50 msgr2=0x7f62d409e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:52.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.088+0000 7f62d9200640 1 --2- 172.21.15.100:0/1189414686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62d40cea50 0x7f62d409e640 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f62c000e6d0 tx=0x7f62c000eb90 comp rx=0 tx=0).stop 2024-09-17T04:24:52.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.088+0000 7f62d9200640 1 -- 172.21.15.100:0/1189414686 shutdown_connections 2024-09-17T04:24:52.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.088+0000 7f62d9200640 1 --2- 172.21.15.100:0/1189414686 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f62b4078b90 0x7f62b407b0a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:52.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.088+0000 7f62d9200640 1 --2- 172.21.15.100:0/1189414686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62d4160d90 0x7f62d414e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:52.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.088+0000 7f62d9200640 1 --2- 172.21.15.100:0/1189414686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62d40cea50 0x7f62d409e640 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:52.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.088+0000 7f62d9200640 1 --2- 172.21.15.100:0/1189414686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d40c5a90 0x7f62d409fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:52.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.088+0000 7f62d9200640 1 -- 172.21.15.100:0/1189414686 >> 172.21.15.100:0/1189414686 conn(0x7f62d40cee80 msgr2=0x7f62d415f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:52.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.088+0000 7f62d9200640 1 -- 172.21.15.100:0/1189414686 shutdown_connections 2024-09-17T04:24:52.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.088+0000 7f62d9200640 1 -- 172.21.15.100:0/1189414686 wait complete. 2024-09-17T04:24:52.110 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-17T04:24:52.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:52.110 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-17T04:24:52.175 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2024-09-17T04:24:52.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:52.175 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T04:24:52.223 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:24:52.223 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:24:52.258 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:24:52.258 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:24:52.333 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:24:52.333 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:24:52.410 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:24:52.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.676+0000 7fc99a800640 1 -- 172.21.15.100:0/1140667389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc99415ce30 msgr2=0x7fc99415d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:52.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.676+0000 7fc99a800640 1 --2- 172.21.15.100:0/1140667389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc99415ce30 0x7fc99415d240 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc994071220 tx=0x7fc98c02a980 comp rx=0 tx=0).stop 2024-09-17T04:24:52.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.677+0000 7fc99a800640 1 -- 172.21.15.100:0/1140667389 shutdown_connections 2024-09-17T04:24:52.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.677+0000 7fc99a800640 1 --2- 172.21.15.100:0/1140667389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc99415ce30 0x7fc99415d240 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:52.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.677+0000 7fc99a800640 1 --2- 172.21.15.100:0/1140667389 >> v2:172.21.15.184:3300/0 conn(0x7fc9940705b0 0x7fc9940709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:52.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.677+0000 7fc99a800640 1 --2- 172.21.15.100:0/1140667389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9940cfc80 0x7fc994156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:52.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.677+0000 7fc99a800640 1 -- 172.21.15.100:0/1140667389 >> 172.21.15.100:0/1140667389 conn(0x7fc99406d3d0 msgr2=0x7fc9940b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:52.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.677+0000 7fc99a800640 1 -- 172.21.15.100:0/1140667389 shutdown_connections 2024-09-17T04:24:52.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.677+0000 7fc99a800640 1 -- 172.21.15.100:0/1140667389 wait complete. 2024-09-17T04:24:52.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.679+0000 7fc99a800640 1 Processor -- start 2024-09-17T04:24:52.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.679+0000 7fc99a800640 1 -- start start 2024-09-17T04:24:52.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.680+0000 7fc99a800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9940705b0 0x7fc9940c5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:52.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.680+0000 7fc99a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9940cfc80 0x7fc9940748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:52.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.680+0000 7fc99a800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc99415ce30 0x7fc9941f91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:52.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.680+0000 7fc99a800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc9940cce40 con 0x7fc9940cfc80 2024-09-17T04:24:52.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.680+0000 7fc99a800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc9940cccc0 con 0x7fc99415ce30 2024-09-17T04:24:52.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.680+0000 7fc99a800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc9940ccfc0 con 0x7fc9940705b0 2024-09-17T04:24:52.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.681+0000 7fc992a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9940cfc80 0x7fc9940748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:52.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.681+0000 7fc992a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9940cfc80 0x7fc9940748f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56914/0 (socket says 172.21.15.100:56914) 2024-09-17T04:24:52.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.681+0000 7fc992a00640 1 -- 172.21.15.100:0/3111876397 learned_addr learned my addr 172.21.15.100:0/3111876397 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:52.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.681+0000 7fc993400640 1 --2- 172.21.15.100:0/3111876397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9940705b0 0x7fc9940c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:52.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.681+0000 7fc993e00640 1 --2- 172.21.15.100:0/3111876397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc99415ce30 0x7fc9941f91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:52.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.681+0000 7fc992a00640 1 -- 172.21.15.100:0/3111876397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9940705b0 msgr2=0x7fc9940c5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:52.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.681+0000 7fc992a00640 1 --2- 172.21.15.100:0/3111876397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9940705b0 0x7fc9940c5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:52.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.681+0000 7fc992a00640 1 -- 172.21.15.100:0/3111876397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc99415ce30 msgr2=0x7fc9941f91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:52.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.681+0000 7fc992a00640 1 --2- 172.21.15.100:0/3111876397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc99415ce30 0x7fc9941f91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:52.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.681+0000 7fc992a00640 1 -- 172.21.15.100:0/3111876397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9940c6ba0 con 0x7fc9940cfc80 2024-09-17T04:24:52.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.681+0000 7fc992a00640 1 --2- 172.21.15.100:0/3111876397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9940cfc80 0x7fc9940748f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc98000d720 tx=0x7fc98000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:52.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.682+0000 7fc98b400640 1 -- 172.21.15.100:0/3111876397 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980017070 con 0x7fc9940cfc80 2024-09-17T04:24:52.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.682+0000 7fc98b400640 1 -- 172.21.15.100:0/3111876397 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc980009ab0 con 0x7fc9940cfc80 2024-09-17T04:24:52.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.682+0000 7fc98b400640 1 -- 172.21.15.100:0/3111876397 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980010460 con 0x7fc9940cfc80 2024-09-17T04:24:52.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.682+0000 7fc99a800640 1 -- 172.21.15.100:0/3111876397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9940c5b00 con 0x7fc9940cfc80 2024-09-17T04:24:52.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.682+0000 7fc99a800640 1 -- 172.21.15.100:0/3111876397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9940c60c0 con 0x7fc9940cfc80 2024-09-17T04:24:52.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.683+0000 7fc99a800640 1 -- 172.21.15.100:0/3111876397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9940c5870 con 0x7fc9940cfc80 2024-09-17T04:24:52.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.684+0000 7fc98b400640 1 -- 172.21.15.100:0/3111876397 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc98001a070 con 0x7fc9940cfc80 2024-09-17T04:24:52.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.685+0000 7fc98b400640 1 --2- 172.21.15.100:0/3111876397 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc974078b90 0x7fc97407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:52.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.685+0000 7fc98b400640 1 -- 172.21.15.100:0/3111876397 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7fc98009ae10 con 0x7fc9940cfc80 2024-09-17T04:24:52.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.685+0000 7fc993400640 1 --2- 172.21.15.100:0/3111876397 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc974078b90 0x7fc97407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:52.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.686+0000 7fc993400640 1 --2- 172.21.15.100:0/3111876397 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc974078b90 0x7fc97407b0a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc99409e510 tx=0x7fc98400b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:52.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.690+0000 7fc98b400640 1 -- 172.21.15.100:0/3111876397 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc98009e030 con 0x7fc9940cfc80 2024-09-17T04:24:52.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.884+0000 7fc99a800640 1 -- 172.21.15.100:0/3111876397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc9940cd550 con 0x7fc9940cfc80 2024-09-17T04:24:52.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.885+0000 7fc98b400640 1 -- 172.21.15.100:0/3111876397 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7fc980062b70 con 0x7fc9940cfc80 2024-09-17T04:24:52.885 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-09-17T04:24:52.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.887+0000 7fc99a800640 1 -- 172.21.15.100:0/3111876397 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc974078b90 msgr2=0x7fc97407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:52.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.887+0000 7fc99a800640 1 --2- 172.21.15.100:0/3111876397 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc974078b90 0x7fc97407b0a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc99409e510 tx=0x7fc98400b2e0 comp rx=0 tx=0).stop 2024-09-17T04:24:52.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.887+0000 7fc99a800640 1 -- 172.21.15.100:0/3111876397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9940cfc80 msgr2=0x7fc9940748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:52.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.887+0000 7fc99a800640 1 --2- 172.21.15.100:0/3111876397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9940cfc80 0x7fc9940748f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc98000d720 tx=0x7fc98000dbe0 comp rx=0 tx=0).stop 2024-09-17T04:24:52.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.887+0000 7fc99a800640 1 -- 172.21.15.100:0/3111876397 shutdown_connections 2024-09-17T04:24:52.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.887+0000 7fc99a800640 1 --2- 172.21.15.100:0/3111876397 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc974078b90 0x7fc97407b0a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:52.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.887+0000 7fc99a800640 1 --2- 172.21.15.100:0/3111876397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc99415ce30 0x7fc9941f91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:52.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.887+0000 7fc99a800640 1 --2- 172.21.15.100:0/3111876397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9940cfc80 0x7fc9940748f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:52.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.887+0000 7fc99a800640 1 --2- 172.21.15.100:0/3111876397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9940705b0 0x7fc9940c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:52.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.887+0000 7fc99a800640 1 -- 172.21.15.100:0/3111876397 >> 172.21.15.100:0/3111876397 conn(0x7fc99406d3d0 msgr2=0x7fc9940b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:52.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.887+0000 7fc99a800640 1 -- 172.21.15.100:0/3111876397 shutdown_connections 2024-09-17T04:24:52.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:52.887+0000 7fc99a800640 1 -- 172.21.15.100:0/3111876397 wait complete. 2024-09-17T04:24:52.909 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-17T04:24:52.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:52.909 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-17T04:24:52.968 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-09-17T04:24:52.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:52.969 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-17T04:24:53.004 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:24:53.004 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:24:53.042 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:24:53.043 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:24:53.120 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:24:53.120 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:24:53.197 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:24:53.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.417+0000 7fcdc8600640 1 -- 172.21.15.100:0/4287268949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc0070cc0 msgr2=0x7fcdc0156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:53.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.417+0000 7fcdc8600640 1 --2- 172.21.15.100:0/4287268949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc0070cc0 0x7fcdc0156e40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009900 tx=0x7fcdac02a620 comp rx=0 tx=0).stop 2024-09-17T04:24:53.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.417+0000 7fcdc8600640 1 -- 172.21.15.100:0/4287268949 shutdown_connections 2024-09-17T04:24:53.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.417+0000 7fcdc8600640 1 --2- 172.21.15.100:0/4287268949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc0070cc0 0x7fcdc0156e40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:53.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.417+0000 7fcdc8600640 1 -- 172.21.15.100:0/4287268949 >> 172.21.15.100:0/4287268949 conn(0x7fcdc006d3d0 msgr2=0x7fcdc00b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:53.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.417+0000 7fcdc8600640 1 -- 172.21.15.100:0/4287268949 shutdown_connections 2024-09-17T04:24:53.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.417+0000 7fcdc8600640 1 -- 172.21.15.100:0/4287268949 wait complete. 2024-09-17T04:24:53.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.419+0000 7fcdc8600640 1 Processor -- start 2024-09-17T04:24:53.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.420+0000 7fcdc8600640 1 -- start start 2024-09-17T04:24:53.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.421+0000 7fcdc8600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc00cfc80 0x7fcdc00749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:53.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.421+0000 7fcdc8600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc015ce30 0x7fcdc01f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:53.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.421+0000 7fcdc8600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdc0073370 0x7fcdc00b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:53.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.421+0000 7fcdc8600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc00cd100 con 0x7fcdc00cfc80 2024-09-17T04:24:53.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.421+0000 7fcdc8600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc00ccf80 con 0x7fcdc015ce30 2024-09-17T04:24:53.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.421+0000 7fcdc8600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcdc00cd280 con 0x7fcdc0073370 2024-09-17T04:24:53.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.421+0000 7fcdc5c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc00cfc80 0x7fcdc00749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:53.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.421+0000 7fcdc5c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc00cfc80 0x7fcdc00749a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56940/0 (socket says 172.21.15.100:56940) 2024-09-17T04:24:53.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.421+0000 7fcdc5c00640 1 -- 172.21.15.100:0/2650527262 learned_addr learned my addr 172.21.15.100:0/2650527262 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:53.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.421+0000 7fcdc6600640 1 --2- 172.21.15.100:0/2650527262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdc0073370 0x7fcdc00b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:53.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.421+0000 7fcdc5200640 1 --2- 172.21.15.100:0/2650527262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc015ce30 0x7fcdc01f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:53.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.422+0000 7fcdc5c00640 1 -- 172.21.15.100:0/2650527262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdc0073370 msgr2=0x7fcdc00b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:53.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.422+0000 7fcdc5c00640 1 --2- 172.21.15.100:0/2650527262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdc0073370 0x7fcdc00b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:53.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.422+0000 7fcdc5c00640 1 -- 172.21.15.100:0/2650527262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc015ce30 msgr2=0x7fcdc01f3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:53.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.422+0000 7fcdc5c00640 1 --2- 172.21.15.100:0/2650527262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc015ce30 0x7fcdc01f3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:53.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.422+0000 7fcdc5c00640 1 -- 172.21.15.100:0/2650527262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc01f4370 con 0x7fcdc00cfc80 2024-09-17T04:24:53.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.422+0000 7fcdc5c00640 1 --2- 172.21.15.100:0/2650527262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc00cfc80 0x7fcdc00749a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400c3c0 tx=0x7fcdb400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:53.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.422+0000 7fcdbd600640 1 -- 172.21.15.100:0/2650527262 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4017070 con 0x7fcdc00cfc80 2024-09-17T04:24:53.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.422+0000 7fcdbd600640 1 -- 172.21.15.100:0/2650527262 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcdb40053e0 con 0x7fcdc00cfc80 2024-09-17T04:24:53.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.422+0000 7fcdc8600640 1 -- 172.21.15.100:0/2650527262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdc00b09d0 con 0x7fcdc00cfc80 2024-09-17T04:24:53.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.422+0000 7fcdbd600640 1 -- 172.21.15.100:0/2650527262 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4003870 con 0x7fcdc00cfc80 2024-09-17T04:24:53.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.422+0000 7fcdc8600640 1 -- 172.21.15.100:0/2650527262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdc00b12f0 con 0x7fcdc00cfc80 2024-09-17T04:24:53.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.424+0000 7fcdc8600640 1 -- 172.21.15.100:0/2650527262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdc00cd790 con 0x7fcdc00cfc80 2024-09-17T04:24:53.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.425+0000 7fcdbd600640 1 -- 172.21.15.100:0/2650527262 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcdb401a070 con 0x7fcdc00cfc80 2024-09-17T04:24:53.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.426+0000 7fcdbd600640 1 --2- 172.21.15.100:0/2650527262 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fcda0078b90 0x7fcda007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:53.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.426+0000 7fcdbd600640 1 -- 172.21.15.100:0/2650527262 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7fcdb409a120 con 0x7fcdc00cfc80 2024-09-17T04:24:53.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.426+0000 7fcdc5200640 1 --2- 172.21.15.100:0/2650527262 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fcda0078b90 0x7fcda007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:53.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.427+0000 7fcdc5200640 1 --2- 172.21.15.100:0/2650527262 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fcda0078b90 0x7fcda007b0a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcdac033000 tx=0x7fcdac004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:53.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.430+0000 7fcdbd600640 1 -- 172.21.15.100:0/2650527262 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fcdc01f4370 con 0x7fcdc00cfc80 2024-09-17T04:24:53.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.627+0000 7fcdc8600640 1 -- 172.21.15.100:0/2650527262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcdc00ca720 con 0x7fcdc00cfc80 2024-09-17T04:24:53.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.627+0000 7fcdbd600640 1 -- 172.21.15.100:0/2650527262 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7fcdb4061e80 con 0x7fcdc00cfc80 2024-09-17T04:24:53.628 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-09-17T04:24:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.630+0000 7fcdc8600640 1 -- 172.21.15.100:0/2650527262 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fcda0078b90 msgr2=0x7fcda007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.630+0000 7fcdc8600640 1 --2- 172.21.15.100:0/2650527262 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fcda0078b90 0x7fcda007b0a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcdac033000 tx=0x7fcdac004730 comp rx=0 tx=0).stop 2024-09-17T04:24:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.630+0000 7fcdc8600640 1 -- 172.21.15.100:0/2650527262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc00cfc80 msgr2=0x7fcdc00749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.630+0000 7fcdc8600640 1 --2- 172.21.15.100:0/2650527262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc00cfc80 0x7fcdc00749a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400c3c0 tx=0x7fcdb400c880 comp rx=0 tx=0).stop 2024-09-17T04:24:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.630+0000 7fcdc8600640 1 -- 172.21.15.100:0/2650527262 shutdown_connections 2024-09-17T04:24:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.630+0000 7fcdc8600640 1 --2- 172.21.15.100:0/2650527262 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fcda0078b90 0x7fcda007b0a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.630+0000 7fcdc8600640 1 --2- 172.21.15.100:0/2650527262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdc0073370 0x7fcdc00b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.630+0000 7fcdc8600640 1 --2- 172.21.15.100:0/2650527262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc015ce30 0x7fcdc01f3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.630+0000 7fcdc8600640 1 --2- 172.21.15.100:0/2650527262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc00cfc80 0x7fcdc00749a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.630+0000 7fcdc8600640 1 -- 172.21.15.100:0/2650527262 >> 172.21.15.100:0/2650527262 conn(0x7fcdc006d3d0 msgr2=0x7fcdc015ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.630+0000 7fcdc8600640 1 -- 172.21.15.100:0/2650527262 shutdown_connections 2024-09-17T04:24:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.630+0000 7fcdc8600640 1 -- 172.21.15.100:0/2650527262 wait complete. 2024-09-17T04:24:53.655 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-17T04:24:53.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:53.655 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-17T04:24:53.720 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2024-09-17T04:24:53.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:24:53.721 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-17T04:24:53.758 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T04:24:53.758 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:24:53.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.977+0000 7f35f7e00640 1 -- 172.21.15.100:0/277926541 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35f80ad9c0 msgr2=0x7f35f80af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:53.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.977+0000 7f35f7e00640 1 --2- 172.21.15.100:0/277926541 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35f80ad9c0 0x7f35f80af170 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f35e8005fd0 tx=0x7f35e802a540 comp rx=0 tx=0).stop 2024-09-17T04:24:53.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.977+0000 7f35f7e00640 1 -- 172.21.15.100:0/277926541 shutdown_connections 2024-09-17T04:24:53.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.977+0000 7f35f7e00640 1 --2- 172.21.15.100:0/277926541 >> v2:172.21.15.184:3300/0 conn(0x7f35f80b2f60 0x7f35f815a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:53.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.977+0000 7f35f7e00640 1 --2- 172.21.15.100:0/277926541 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35f80ad9c0 0x7f35f80af170 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:53.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.977+0000 7f35f7e00640 1 -- 172.21.15.100:0/277926541 >> 172.21.15.100:0/277926541 conn(0x7f35f80b1e40 msgr2=0x7f35f80b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:53.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.977+0000 7f35f7e00640 1 -- 172.21.15.100:0/277926541 shutdown_connections 2024-09-17T04:24:53.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.978+0000 7f35f7e00640 1 -- 172.21.15.100:0/277926541 wait complete. 2024-09-17T04:24:53.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.980+0000 7f35f7e00640 1 Processor -- start 2024-09-17T04:24:53.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.980+0000 7f35f7e00640 1 -- start start 2024-09-17T04:24:53.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.981+0000 7f35f7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35f80ad9c0 0x7f35f81f8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:53.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.981+0000 7f35f7e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35f80b2f60 0x7f35f81507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:53.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.981+0000 7f35f7e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35f815d670 0x7f35f8150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:53.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.981+0000 7f35f7e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f35f806f410 con 0x7f35f80ad9c0 2024-09-17T04:24:53.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.981+0000 7f35f7e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f35f806f290 con 0x7f35f80b2f60 2024-09-17T04:24:53.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.981+0000 7f35f7e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f35f806f590 con 0x7f35f815d670 2024-09-17T04:24:53.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.981+0000 7f35f6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35f80ad9c0 0x7f35f81f8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:53.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.982+0000 7f35f6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35f80ad9c0 0x7f35f81f8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56942/0 (socket says 172.21.15.100:56942) 2024-09-17T04:24:53.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.982+0000 7f35f6a00640 1 -- 172.21.15.100:0/2249801513 learned_addr learned my addr 172.21.15.100:0/2249801513 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:53.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.982+0000 7f35ee000640 1 --2- 172.21.15.100:0/2249801513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35f80b2f60 0x7f35f81507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:53.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.982+0000 7f35f7400640 1 --2- 172.21.15.100:0/2249801513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35f815d670 0x7f35f8150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:53.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.982+0000 7f35f6a00640 1 -- 172.21.15.100:0/2249801513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35f815d670 msgr2=0x7f35f8150d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:53.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.982+0000 7f35f6a00640 1 --2- 172.21.15.100:0/2249801513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35f815d670 0x7f35f8150d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:53.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.982+0000 7f35f6a00640 1 -- 172.21.15.100:0/2249801513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35f80b2f60 msgr2=0x7f35f81507e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:53.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.982+0000 7f35f6a00640 1 --2- 172.21.15.100:0/2249801513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35f80b2f60 0x7f35f81507e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:53.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.982+0000 7f35f6a00640 1 -- 172.21.15.100:0/2249801513 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f8074040 con 0x7f35f80ad9c0 2024-09-17T04:24:53.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.982+0000 7f35f6a00640 1 --2- 172.21.15.100:0/2249801513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35f80ad9c0 0x7f35f81f8f10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f35e8034040 tx=0x7f35e80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:53.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.983+0000 7f35efe00640 1 -- 172.21.15.100:0/2249801513 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e803e030 con 0x7f35f80ad9c0 2024-09-17T04:24:53.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.983+0000 7f35efe00640 1 -- 172.21.15.100:0/2249801513 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35e8003db0 con 0x7f35f80ad9c0 2024-09-17T04:24:53.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.983+0000 7f35efe00640 1 -- 172.21.15.100:0/2249801513 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e8035040 con 0x7f35f80ad9c0 2024-09-17T04:24:53.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.983+0000 7f35f7e00640 1 -- 172.21.15.100:0/2249801513 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f8074320 con 0x7f35f80ad9c0 2024-09-17T04:24:53.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.983+0000 7f35f7e00640 1 -- 172.21.15.100:0/2249801513 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f80748e0 con 0x7f35f80ad9c0 2024-09-17T04:24:53.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.984+0000 7f35f7e00640 1 -- 172.21.15.100:0/2249801513 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35f806fa40 con 0x7f35f80ad9c0 2024-09-17T04:24:53.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.985+0000 7f35efe00640 1 -- 172.21.15.100:0/2249801513 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f35e8041030 con 0x7f35f80ad9c0 2024-09-17T04:24:53.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.987+0000 7f35efe00640 1 --2- 172.21.15.100:0/2249801513 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f35d4078b90 0x7f35d407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:53.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.987+0000 7f35efe00640 1 -- 172.21.15.100:0/2249801513 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f35e80bb540 con 0x7f35f80ad9c0 2024-09-17T04:24:53.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.987+0000 7f35ee000640 1 --2- 172.21.15.100:0/2249801513 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f35d4078b90 0x7f35d407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:53.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.988+0000 7f35ee000640 1 --2- 172.21.15.100:0/2249801513 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f35d4078b90 0x7f35d407b0a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f35f80705a0 tx=0x7f35e0005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:53.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:53.991+0000 7f35efe00640 1 -- 172.21.15.100:0/2249801513 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f35e8083210 con 0x7f35f80ad9c0 2024-09-17T04:24:54.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.210+0000 7f35f7e00640 1 -- 172.21.15.100:0/2249801513 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35f8050ff0 con 0x7f35f80ad9c0 2024-09-17T04:24:54.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.211+0000 7f35efe00640 1 -- 172.21.15.100:0/2249801513 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7f35e8087c40 con 0x7f35f80ad9c0 2024-09-17T04:24:54.211 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:54.211 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":8,"btime":"2024-09-17T04:24:18:058994+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4328,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2129046042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":2129046042},{"type":"v1","addr":"172.21.15.100:6835","nonce":2129046042}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4334,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/44255118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":44255118},{"type":"v1","addr":"172.21.15.184:6836","nonce":44255118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/1644264480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1644264480},{"type":"v1","addr":"172.21.15.100:6834","nonce":1644264480}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:24:14.527272+0000","modified":"2024-09-17T04:24:17.543043+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4368},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4368":{"gid":4368,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/2211103925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2211103925},{"type":"v1","addr":"172.21.15.184:6837","nonce":2211103925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4368,"qdb_cluster":[4368]},"id":1}]} 2024-09-17T04:24:54.212 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 8 2024-09-17T04:24:54.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.213+0000 7f35f7e00640 1 -- 172.21.15.100:0/2249801513 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f35d4078b90 msgr2=0x7f35d407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:54.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.213+0000 7f35f7e00640 1 --2- 172.21.15.100:0/2249801513 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f35d4078b90 0x7f35d407b0a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f35f80705a0 tx=0x7f35e0005f90 comp rx=0 tx=0).stop 2024-09-17T04:24:54.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.213+0000 7f35f7e00640 1 -- 172.21.15.100:0/2249801513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35f80ad9c0 msgr2=0x7f35f81f8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:54.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.213+0000 7f35f7e00640 1 --2- 172.21.15.100:0/2249801513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35f80ad9c0 0x7f35f81f8f10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f35e8034040 tx=0x7f35e80038a0 comp rx=0 tx=0).stop 2024-09-17T04:24:54.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.213+0000 7f35f7e00640 1 -- 172.21.15.100:0/2249801513 shutdown_connections 2024-09-17T04:24:54.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.213+0000 7f35f7e00640 1 --2- 172.21.15.100:0/2249801513 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f35d4078b90 0x7f35d407b0a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:54.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.213+0000 7f35f7e00640 1 --2- 172.21.15.100:0/2249801513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35f815d670 0x7f35f8150d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:54.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.213+0000 7f35f7e00640 1 --2- 172.21.15.100:0/2249801513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35f80b2f60 0x7f35f81507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:54.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.213+0000 7f35f7e00640 1 --2- 172.21.15.100:0/2249801513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35f80ad9c0 0x7f35f81f8f10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:54.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.213+0000 7f35f7e00640 1 -- 172.21.15.100:0/2249801513 >> 172.21.15.100:0/2249801513 conn(0x7f35f80b1e40 msgr2=0x7f35f815c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:54.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.213+0000 7f35f7e00640 1 -- 172.21.15.100:0/2249801513 shutdown_connections 2024-09-17T04:24:54.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.213+0000 7f35f7e00640 1 -- 172.21.15.100:0/2249801513 wait complete. 2024-09-17T04:24:54.236 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:24:54.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.456+0000 7fe5c4600640 1 -- 172.21.15.100:0/719654065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5bc165480 msgr2=0x7fe5bc1678a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:54.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.456+0000 7fe5c4600640 1 --2- 172.21.15.100:0/719654065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5bc165480 0x7fe5bc1678a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4009be0 tx=0x7fe5b402a820 comp rx=0 tx=0).stop 2024-09-17T04:24:54.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.456+0000 7fe5c4600640 1 -- 172.21.15.100:0/719654065 shutdown_connections 2024-09-17T04:24:54.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.456+0000 7fe5c4600640 1 --2- 172.21.15.100:0/719654065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5bc165480 0x7fe5bc1678a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:54.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.456+0000 7fe5c4600640 1 -- 172.21.15.100:0/719654065 >> 172.21.15.100:0/719654065 conn(0x7fe5bc06d730 msgr2=0x7fe5bc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:54.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.456+0000 7fe5c4600640 1 -- 172.21.15.100:0/719654065 shutdown_connections 2024-09-17T04:24:54.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.456+0000 7fe5c4600640 1 -- 172.21.15.100:0/719654065 wait complete. 2024-09-17T04:24:54.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.458+0000 7fe5c4600640 1 Processor -- start 2024-09-17T04:24:54.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.458+0000 7fe5c4600640 1 -- start start 2024-09-17T04:24:54.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.460+0000 7fe5c4600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5bc0ad0e0 0x7fe5bc144050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:54.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.460+0000 7fe5c4600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5bc15d760 0x7fe5bc146920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:54.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.460+0000 7fe5c4600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5bc165480 0x7fe5bc0b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:54.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.460+0000 7fe5c4600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc167de0 con 0x7fe5bc15d760 2024-09-17T04:24:54.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.460+0000 7fe5c4600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc0b4e30 con 0x7fe5bc165480 2024-09-17T04:24:54.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.460+0000 7fe5c4600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe5bc167f60 con 0x7fe5bc0ad0e0 2024-09-17T04:24:54.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.460+0000 7fe5c1200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5bc15d760 0x7fe5bc146920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:54.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.460+0000 7fe5c1200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5bc15d760 0x7fe5bc146920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56956/0 (socket says 172.21.15.100:56956) 2024-09-17T04:24:54.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.460+0000 7fe5c1200640 1 -- 172.21.15.100:0/2881132449 learned_addr learned my addr 172.21.15.100:0/2881132449 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:54.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.460+0000 7fe5c2600640 1 --2- 172.21.15.100:0/2881132449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5bc165480 0x7fe5bc0b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:54.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.460+0000 7fe5c1200640 1 -- 172.21.15.100:0/2881132449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5bc0ad0e0 msgr2=0x7fe5bc144050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:54.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.460+0000 7fe5c1200640 1 --2- 172.21.15.100:0/2881132449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5bc0ad0e0 0x7fe5bc144050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:54.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.460+0000 7fe5c1200640 1 -- 172.21.15.100:0/2881132449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5bc165480 msgr2=0x7fe5bc0b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:54.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.461+0000 7fe5c1200640 1 --2- 172.21.15.100:0/2881132449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5bc165480 0x7fe5bc0b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:54.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.461+0000 7fe5c1200640 1 -- 172.21.15.100:0/2881132449 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5bc0b1f90 con 0x7fe5bc15d760 2024-09-17T04:24:54.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.461+0000 7fe5c2600640 1 --2- 172.21.15.100:0/2881132449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5bc165480 0x7fe5bc0b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:24:54.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.461+0000 7fe5c1200640 1 --2- 172.21.15.100:0/2881132449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5bc15d760 0x7fe5bc146920 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800c700 tx=0x7fe5a800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:54.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.461+0000 7fe5b9600640 1 -- 172.21.15.100:0/2881132449 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8016020 con 0x7fe5bc15d760 2024-09-17T04:24:54.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.461+0000 7fe5b9600640 1 -- 172.21.15.100:0/2881132449 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe5a800a8d0 con 0x7fe5bc15d760 2024-09-17T04:24:54.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.461+0000 7fe5c4600640 1 -- 172.21.15.100:0/2881132449 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5bc0b19d0 con 0x7fe5bc15d760 2024-09-17T04:24:54.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.461+0000 7fe5b9600640 1 -- 172.21.15.100:0/2881132449 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a800ace0 con 0x7fe5bc15d760 2024-09-17T04:24:54.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.461+0000 7fe5c4600640 1 -- 172.21.15.100:0/2881132449 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5bc0afdf0 con 0x7fe5bc15d760 2024-09-17T04:24:54.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.462+0000 7fe5c4600640 1 -- 172.21.15.100:0/2881132449 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5bc1535c0 con 0x7fe5bc15d760 2024-09-17T04:24:54.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.463+0000 7fe5b9600640 1 -- 172.21.15.100:0/2881132449 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe5a8019020 con 0x7fe5bc15d760 2024-09-17T04:24:54.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.464+0000 7fe5b9600640 1 --2- 172.21.15.100:0/2881132449 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe59c078bc0 0x7fe59c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:54.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.464+0000 7fe5b9600640 1 -- 172.21.15.100:0/2881132449 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7fe5a809ac40 con 0x7fe5bc15d760 2024-09-17T04:24:54.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.465+0000 7fe5c1c00640 1 --2- 172.21.15.100:0/2881132449 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe59c078bc0 0x7fe59c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:54.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.465+0000 7fe5c1c00640 1 --2- 172.21.15.100:0/2881132449 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe59c078bc0 0x7fe59c07b0d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0001530 tx=0x7fe5b00079d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:54.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.469+0000 7fe5b9600640 1 -- 172.21.15.100:0/2881132449 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe5a8020030 con 0x7fe5bc15d760 2024-09-17T04:24:54.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.688+0000 7fe5c4600640 1 -- 172.21.15.100:0/2881132449 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5bc0b21b0 con 0x7fe5bc15d760 2024-09-17T04:24:54.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.688+0000 7fe5b9600640 1 -- 172.21.15.100:0/2881132449 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7fe5a8062ee0 con 0x7fe5bc15d760 2024-09-17T04:24:54.689 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:54.689 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":8,"btime":"2024-09-17T04:24:18:058994+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4328,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2129046042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":2129046042},{"type":"v1","addr":"172.21.15.100:6835","nonce":2129046042}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4334,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/44255118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":44255118},{"type":"v1","addr":"172.21.15.184:6836","nonce":44255118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/1644264480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1644264480},{"type":"v1","addr":"172.21.15.100:6834","nonce":1644264480}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:24:14.527272+0000","modified":"2024-09-17T04:24:17.543043+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4368},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4368":{"gid":4368,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/2211103925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2211103925},{"type":"v1","addr":"172.21.15.184:6837","nonce":2211103925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4368,"qdb_cluster":[4368]},"id":1}]} 2024-09-17T04:24:54.690 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 8 2024-09-17T04:24:54.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.691+0000 7fe5c4600640 1 -- 172.21.15.100:0/2881132449 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe59c078bc0 msgr2=0x7fe59c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:54.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.691+0000 7fe5c4600640 1 --2- 172.21.15.100:0/2881132449 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe59c078bc0 0x7fe59c07b0d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0001530 tx=0x7fe5b00079d0 comp rx=0 tx=0).stop 2024-09-17T04:24:54.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.691+0000 7fe5c4600640 1 -- 172.21.15.100:0/2881132449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5bc15d760 msgr2=0x7fe5bc146920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:54.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.691+0000 7fe5c4600640 1 --2- 172.21.15.100:0/2881132449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5bc15d760 0x7fe5bc146920 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800c700 tx=0x7fe5a800cbc0 comp rx=0 tx=0).stop 2024-09-17T04:24:54.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.691+0000 7fe5c4600640 1 -- 172.21.15.100:0/2881132449 shutdown_connections 2024-09-17T04:24:54.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.691+0000 7fe5c4600640 1 --2- 172.21.15.100:0/2881132449 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe59c078bc0 0x7fe59c07b0d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:54.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.691+0000 7fe5c4600640 1 --2- 172.21.15.100:0/2881132449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5bc165480 0x7fe5bc0b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:54.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.691+0000 7fe5c4600640 1 --2- 172.21.15.100:0/2881132449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5bc15d760 0x7fe5bc146920 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:54.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.691+0000 7fe5c4600640 1 --2- 172.21.15.100:0/2881132449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5bc0ad0e0 0x7fe5bc144050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:54.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.691+0000 7fe5c4600640 1 -- 172.21.15.100:0/2881132449 >> 172.21.15.100:0/2881132449 conn(0x7fe5bc06d730 msgr2=0x7fe5bc14b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:54.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.691+0000 7fe5c4600640 1 -- 172.21.15.100:0/2881132449 shutdown_connections 2024-09-17T04:24:54.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.691+0000 7fe5c4600640 1 -- 172.21.15.100:0/2881132449 wait complete. 2024-09-17T04:24:54.713 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:24:54.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.933+0000 7f2b4c200640 1 -- 172.21.15.100:0/2325059431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b2c005250 msgr2=0x7f2b2c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:54.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.933+0000 7f2b4c200640 1 --2- 172.21.15.100:0/2325059431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b2c005250 0x7f2b2c00b9c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c00c750 tx=0x7f2b2c005830 comp rx=0 tx=0).stop 2024-09-17T04:24:54.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.933+0000 7f2b4c200640 1 -- 172.21.15.100:0/2325059431 shutdown_connections 2024-09-17T04:24:54.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.933+0000 7f2b4c200640 1 --2- 172.21.15.100:0/2325059431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b2c005250 0x7f2b2c00b9c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:54.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.933+0000 7f2b4c200640 1 -- 172.21.15.100:0/2325059431 >> 172.21.15.100:0/2325059431 conn(0x7f2b4406d730 msgr2=0x7f2b440b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:54.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.933+0000 7f2b4c200640 1 -- 172.21.15.100:0/2325059431 shutdown_connections 2024-09-17T04:24:54.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.933+0000 7f2b4c200640 1 -- 172.21.15.100:0/2325059431 wait complete. 2024-09-17T04:24:54.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.936+0000 7f2b4c200640 1 Processor -- start 2024-09-17T04:24:54.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.936+0000 7f2b4c200640 1 -- start start 2024-09-17T04:24:54.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.937+0000 7f2b4c200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b2c005250 0x7f2b441e9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:54.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.937+0000 7f2b4c200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b4414f800 0x7f2b4414f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:54.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.937+0000 7f2b4c200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b440b3930 0x7f2b440b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:54.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.937+0000 7f2b4c200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4406f0a0 con 0x7f2b2c005250 2024-09-17T04:24:54.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.937+0000 7f2b4c200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2b44073760 con 0x7f2b440b3930 2024-09-17T04:24:54.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.937+0000 7f2b4c200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2b4406f220 con 0x7f2b4414f800 2024-09-17T04:24:54.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.938+0000 7f2b49800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b2c005250 0x7f2b441e9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:54.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.938+0000 7f2b49800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b2c005250 0x7f2b441e9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50990/0 (socket says 172.21.15.100:50990) 2024-09-17T04:24:54.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.938+0000 7f2b49800640 1 -- 172.21.15.100:0/1943217400 learned_addr learned my addr 172.21.15.100:0/1943217400 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:54.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.938+0000 7f2b49800640 1 -- 172.21.15.100:0/1943217400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b4414f800 msgr2=0x7f2b4414f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:54.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.938+0000 7f2b49800640 1 --2- 172.21.15.100:0/1943217400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b4414f800 0x7f2b4414f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:54.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.938+0000 7f2b49800640 1 -- 172.21.15.100:0/1943217400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b440b3930 msgr2=0x7f2b440b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:54.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.938+0000 7f2b49800640 1 --2- 172.21.15.100:0/1943217400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b440b3930 0x7f2b440b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:54.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.938+0000 7f2b49800640 1 -- 172.21.15.100:0/1943217400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b440b4280 con 0x7f2b2c005250 2024-09-17T04:24:54.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.938+0000 7f2b49800640 1 --2- 172.21.15.100:0/1943217400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b2c005250 0x7f2b441e9f00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2b380117d0 tx=0x7f2b38011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:54.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.938+0000 7f2b41600640 1 -- 172.21.15.100:0/1943217400 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b38004460 con 0x7f2b2c005250 2024-09-17T04:24:54.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.939+0000 7f2b41600640 1 -- 172.21.15.100:0/1943217400 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b38016070 con 0x7f2b2c005250 2024-09-17T04:24:54.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.939+0000 7f2b4c200640 1 -- 172.21.15.100:0/1943217400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b44152c80 con 0x7f2b2c005250 2024-09-17T04:24:54.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.939+0000 7f2b41600640 1 -- 172.21.15.100:0/1943217400 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3800fb60 con 0x7f2b2c005250 2024-09-17T04:24:54.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.939+0000 7f2b4c200640 1 -- 172.21.15.100:0/1943217400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b44153240 con 0x7f2b2c005250 2024-09-17T04:24:54.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.940+0000 7f2b4c200640 1 -- 172.21.15.100:0/1943217400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b44162560 con 0x7f2b2c005250 2024-09-17T04:24:54.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.941+0000 7f2b41600640 1 -- 172.21.15.100:0/1943217400 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2b38018030 con 0x7f2b2c005250 2024-09-17T04:24:54.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.942+0000 7f2b41600640 1 --2- 172.21.15.100:0/1943217400 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2b24078bf0 0x7f2b2407b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:54.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.942+0000 7f2b41600640 1 -- 172.21.15.100:0/1943217400 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f2b3809b540 con 0x7f2b2c005250 2024-09-17T04:24:54.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.942+0000 7f2b48e00640 1 --2- 172.21.15.100:0/1943217400 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2b24078bf0 0x7f2b2407b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:54.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.943+0000 7f2b48e00640 1 --2- 172.21.15.100:0/1943217400 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2b24078bf0 0x7f2b2407b100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2b30006850 tx=0x7f2b300067e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:54.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:54.946+0000 7f2b41600640 1 -- 172.21.15.100:0/1943217400 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f2b440b4280 con 0x7f2b2c005250 2024-09-17T04:24:55.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.136+0000 7f2b4c200640 1 -- 172.21.15.100:0/1943217400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b44151470 con 0x7f2b2c005250 2024-09-17T04:24:55.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.137+0000 7f2b41600640 1 -- 172.21.15.100:0/1943217400 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+16904 (secure 0 0 0) 0x7f2b38063980 con 0x7f2b2c005250 2024-09-17T04:24:55.138 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:55.138 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":25,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:24:53.559880+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:24:12.826665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T04:24:13.485446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.b","mappings":[{"from":3,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:24:55.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.139+0000 7f2b4c200640 1 -- 172.21.15.100:0/1943217400 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2b24078bf0 msgr2=0x7f2b2407b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:55.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.139+0000 7f2b4c200640 1 --2- 172.21.15.100:0/1943217400 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2b24078bf0 0x7f2b2407b100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2b30006850 tx=0x7f2b300067e0 comp rx=0 tx=0).stop 2024-09-17T04:24:55.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.139+0000 7f2b4c200640 1 -- 172.21.15.100:0/1943217400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b2c005250 msgr2=0x7f2b441e9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:55.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.139+0000 7f2b4c200640 1 --2- 172.21.15.100:0/1943217400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b2c005250 0x7f2b441e9f00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2b380117d0 tx=0x7f2b38011c90 comp rx=0 tx=0).stop 2024-09-17T04:24:55.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.139+0000 7f2b4c200640 1 -- 172.21.15.100:0/1943217400 shutdown_connections 2024-09-17T04:24:55.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.140+0000 7f2b4c200640 1 --2- 172.21.15.100:0/1943217400 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2b24078bf0 0x7f2b2407b100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:55.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.140+0000 7f2b4c200640 1 --2- 172.21.15.100:0/1943217400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b440b3930 0x7f2b440b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:55.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.140+0000 7f2b4c200640 1 --2- 172.21.15.100:0/1943217400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b4414f800 0x7f2b4414f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:55.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.140+0000 7f2b4c200640 1 --2- 172.21.15.100:0/1943217400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b2c005250 0x7f2b441e9f00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:55.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.140+0000 7f2b4c200640 1 -- 172.21.15.100:0/1943217400 >> 172.21.15.100:0/1943217400 conn(0x7f2b4406d730 msgr2=0x7f2b440b13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:55.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.140+0000 7f2b4c200640 1 -- 172.21.15.100:0/1943217400 shutdown_connections 2024-09-17T04:24:55.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.140+0000 7f2b4c200640 1 -- 172.21.15.100:0/1943217400 wait complete. 2024-09-17T04:24:55.163 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T04:24:55.163 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:24:55.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.383+0000 7f8716000640 1 -- 172.21.15.100:0/2919527957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87101518d0 msgr2=0x7f871006f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:55.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.383+0000 7f8716000640 1 --2- 172.21.15.100:0/2919527957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87101518d0 0x7f871006f9f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f870000a940 tx=0x7f870002b840 comp rx=0 tx=0).stop 2024-09-17T04:24:55.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.383+0000 7f8716000640 1 -- 172.21.15.100:0/2919527957 shutdown_connections 2024-09-17T04:24:55.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.383+0000 7f8716000640 1 --2- 172.21.15.100:0/2919527957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87101518d0 0x7f871006f9f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:55.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.383+0000 7f8716000640 1 -- 172.21.15.100:0/2919527957 >> 172.21.15.100:0/2919527957 conn(0x7f87100b1e40 msgr2=0x7f87100b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:55.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.383+0000 7f8716000640 1 -- 172.21.15.100:0/2919527957 shutdown_connections 2024-09-17T04:24:55.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.383+0000 7f8716000640 1 -- 172.21.15.100:0/2919527957 wait complete. 2024-09-17T04:24:55.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.385+0000 7f8716000640 1 Processor -- start 2024-09-17T04:24:55.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.385+0000 7f8716000640 1 -- start start 2024-09-17T04:24:55.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.387+0000 7f8716000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87100735c0 0x7f8710142650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:55.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.387+0000 7f8716000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87100b3520 0x7f87101596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:55.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.387+0000 7f8716000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87101518d0 0x7f87100b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:55.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.387+0000 7f8716000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8710070130 con 0x7f87101518d0 2024-09-17T04:24:55.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.387+0000 7f8716000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8710071160 con 0x7f87100b3520 2024-09-17T04:24:55.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.387+0000 7f8716000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f87100b4040 con 0x7f87100735c0 2024-09-17T04:24:55.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.387+0000 7f8715600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87101518d0 0x7f87100b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:55.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.387+0000 7f8715600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87101518d0 0x7f87100b08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51004/0 (socket says 172.21.15.100:51004) 2024-09-17T04:24:55.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.387+0000 7f8715600640 1 -- 172.21.15.100:0/2303734946 learned_addr learned my addr 172.21.15.100:0/2303734946 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:55.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.387+0000 7f8714c00640 1 --2- 172.21.15.100:0/2303734946 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87100735c0 0x7f8710142650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:55.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.387+0000 7f8707e00640 1 --2- 172.21.15.100:0/2303734946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87100b3520 0x7f87101596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:55.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.387+0000 7f8715600640 1 -- 172.21.15.100:0/2303734946 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87100735c0 msgr2=0x7f8710142650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:55.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.388+0000 7f8715600640 1 --2- 172.21.15.100:0/2303734946 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87100735c0 0x7f8710142650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:55.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.388+0000 7f8715600640 1 -- 172.21.15.100:0/2303734946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87100b3520 msgr2=0x7f87101596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:55.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.388+0000 7f8715600640 1 --2- 172.21.15.100:0/2303734946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87100b3520 0x7f87101596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:55.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.388+0000 7f8715600640 1 -- 172.21.15.100:0/2303734946 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8710143150 con 0x7f87101518d0 2024-09-17T04:24:55.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.388+0000 7f8715600640 1 --2- 172.21.15.100:0/2303734946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87101518d0 0x7f87100b08a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f870800be60 tx=0x7f870800bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:55.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.388+0000 7f870d600640 1 -- 172.21.15.100:0/2303734946 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708019070 con 0x7f87101518d0 2024-09-17T04:24:55.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.388+0000 7f870d600640 1 -- 172.21.15.100:0/2303734946 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8708014440 con 0x7f87101518d0 2024-09-17T04:24:55.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.388+0000 7f8716000640 1 -- 172.21.15.100:0/2303734946 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8710143430 con 0x7f87101518d0 2024-09-17T04:24:55.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.388+0000 7f870d600640 1 -- 172.21.15.100:0/2303734946 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87080148d0 con 0x7f87101518d0 2024-09-17T04:24:55.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.388+0000 7f8716000640 1 -- 172.21.15.100:0/2303734946 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8710142b90 con 0x7f87101518d0 2024-09-17T04:24:55.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.389+0000 7f8716000640 1 -- 172.21.15.100:0/2303734946 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f871015ddd0 con 0x7f87101518d0 2024-09-17T04:24:55.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.390+0000 7f870d600640 1 -- 172.21.15.100:0/2303734946 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f870801c070 con 0x7f87101518d0 2024-09-17T04:24:55.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.391+0000 7f870d600640 1 --2- 172.21.15.100:0/2303734946 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f86ec078b90 0x7f86ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:55.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.392+0000 7f870d600640 1 -- 172.21.15.100:0/2303734946 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f870809adb0 con 0x7f87101518d0 2024-09-17T04:24:55.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.392+0000 7f8714c00640 1 --2- 172.21.15.100:0/2303734946 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f86ec078b90 0x7f86ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:55.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.393+0000 7f8714c00640 1 --2- 172.21.15.100:0/2303734946 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f86ec078b90 0x7f86ec07b0a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8700036000 tx=0x7f8700004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:55.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.396+0000 7f870d600640 1 -- 172.21.15.100:0/2303734946 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f870809f070 con 0x7f87101518d0 2024-09-17T04:24:55.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.615+0000 7f8716000640 1 -- 172.21.15.100:0/2303734946 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f871015daf0 con 0x7f87101518d0 2024-09-17T04:24:55.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.615+0000 7f870d600640 1 -- 172.21.15.100:0/2303734946 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7f8708062a80 con 0x7f87101518d0 2024-09-17T04:24:55.616 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:55.616 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":8,"btime":"2024-09-17T04:24:18:058994+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4328,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2129046042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":2129046042},{"type":"v1","addr":"172.21.15.100:6835","nonce":2129046042}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4334,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/44255118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":44255118},{"type":"v1","addr":"172.21.15.184:6836","nonce":44255118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/1644264480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1644264480},{"type":"v1","addr":"172.21.15.100:6834","nonce":1644264480}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:24:14.527272+0000","modified":"2024-09-17T04:24:17.543043+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4368},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4368":{"gid":4368,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/2211103925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2211103925},{"type":"v1","addr":"172.21.15.184:6837","nonce":2211103925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4368,"qdb_cluster":[4368]},"id":1}]} 2024-09-17T04:24:55.616 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 8 2024-09-17T04:24:55.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.618+0000 7f8716000640 1 -- 172.21.15.100:0/2303734946 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f86ec078b90 msgr2=0x7f86ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:55.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.618+0000 7f8716000640 1 --2- 172.21.15.100:0/2303734946 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f86ec078b90 0x7f86ec07b0a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8700036000 tx=0x7f8700004730 comp rx=0 tx=0).stop 2024-09-17T04:24:55.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.618+0000 7f8716000640 1 -- 172.21.15.100:0/2303734946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87101518d0 msgr2=0x7f87100b08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:55.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.618+0000 7f8716000640 1 --2- 172.21.15.100:0/2303734946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87101518d0 0x7f87100b08a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f870800be60 tx=0x7f870800bf50 comp rx=0 tx=0).stop 2024-09-17T04:24:55.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.618+0000 7f8716000640 1 -- 172.21.15.100:0/2303734946 shutdown_connections 2024-09-17T04:24:55.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.618+0000 7f8716000640 1 --2- 172.21.15.100:0/2303734946 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f86ec078b90 0x7f86ec07b0a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:55.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.618+0000 7f8716000640 1 --2- 172.21.15.100:0/2303734946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87101518d0 0x7f87100b08a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:55.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.618+0000 7f8716000640 1 --2- 172.21.15.100:0/2303734946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87100b3520 0x7f87101596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:55.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.618+0000 7f8716000640 1 --2- 172.21.15.100:0/2303734946 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87100735c0 0x7f8710142650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:55.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.618+0000 7f8716000640 1 -- 172.21.15.100:0/2303734946 >> 172.21.15.100:0/2303734946 conn(0x7f87100b1e40 msgr2=0x7f87100d0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:55.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.618+0000 7f8716000640 1 -- 172.21.15.100:0/2303734946 shutdown_connections 2024-09-17T04:24:55.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.618+0000 7f8716000640 1 -- 172.21.15.100:0/2303734946 wait complete. 2024-09-17T04:24:55.643 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:24:55.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.862+0000 7f804a800640 1 -- 172.21.15.100:0/2274683728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f804415a0f0 msgr2=0x7f804415d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:55.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.862+0000 7f804a800640 1 --2- 172.21.15.100:0/2274683728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f804415a0f0 0x7f804415d670 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8044051590 tx=0x7f803c02a820 comp rx=0 tx=0).stop 2024-09-17T04:24:55.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.862+0000 7f804a800640 1 -- 172.21.15.100:0/2274683728 shutdown_connections 2024-09-17T04:24:55.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.862+0000 7f804a800640 1 --2- 172.21.15.100:0/2274683728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f804415a0f0 0x7f804415d670 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:55.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.862+0000 7f804a800640 1 --2- 172.21.15.100:0/2274683728 >> v2:172.21.15.184:3300/0 conn(0x7f80440ad9c0 0x7f80440af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:55.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.862+0000 7f804a800640 1 -- 172.21.15.100:0/2274683728 >> 172.21.15.100:0/2274683728 conn(0x7f80440b1e40 msgr2=0x7f80440b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:55.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.862+0000 7f804a800640 1 -- 172.21.15.100:0/2274683728 shutdown_connections 2024-09-17T04:24:55.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.862+0000 7f804a800640 1 -- 172.21.15.100:0/2274683728 wait complete. 2024-09-17T04:24:55.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.864+0000 7f804a800640 1 Processor -- start 2024-09-17T04:24:55.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.865+0000 7f804a800640 1 -- start start 2024-09-17T04:24:55.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.866+0000 7f804a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80440ad9c0 0x7f804414b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:55.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.866+0000 7f804a800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80440b2f60 0x7f80441481d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:55.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.866+0000 7f804a800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f804415a0f0 0x7f80440b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:55.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.866+0000 7f804a800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f804406f290 con 0x7f80440ad9c0 2024-09-17T04:24:55.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.866+0000 7f804a800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f804415a580 con 0x7f80440b2f60 2024-09-17T04:24:55.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.866+0000 7f804a800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f804406f410 con 0x7f804415a0f0 2024-09-17T04:24:55.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.866+0000 7f8049400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80440ad9c0 0x7f804414b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:55.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.866+0000 7f8049400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80440ad9c0 0x7f804414b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51022/0 (socket says 172.21.15.100:51022) 2024-09-17T04:24:55.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.866+0000 7f8049400640 1 -- 172.21.15.100:0/3454796140 learned_addr learned my addr 172.21.15.100:0/3454796140 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:55.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.866+0000 7f8048a00640 1 --2- 172.21.15.100:0/3454796140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80440b2f60 0x7f80441481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:55.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.867+0000 7f8049e00640 1 --2- 172.21.15.100:0/3454796140 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f804415a0f0 0x7f80440b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:55.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.867+0000 7f8049400640 1 -- 172.21.15.100:0/3454796140 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f804415a0f0 msgr2=0x7f80440b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:55.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.867+0000 7f8049400640 1 --2- 172.21.15.100:0/3454796140 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f804415a0f0 0x7f80440b0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:55.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.867+0000 7f8049400640 1 -- 172.21.15.100:0/3454796140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80440b2f60 msgr2=0x7f80441481d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:55.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.867+0000 7f8049400640 1 --2- 172.21.15.100:0/3454796140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80440b2f60 0x7f80441481d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:55.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.867+0000 7f8049400640 1 -- 172.21.15.100:0/3454796140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80441f65e0 con 0x7f80440ad9c0 2024-09-17T04:24:55.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.867+0000 7f8049400640 1 --2- 172.21.15.100:0/3454796140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80440ad9c0 0x7f804414b550 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f803800e790 tx=0x7f803800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:55.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.867+0000 7f8041600640 1 -- 172.21.15.100:0/3454796140 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803800eea0 con 0x7f80440ad9c0 2024-09-17T04:24:55.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.867+0000 7f8041600640 1 -- 172.21.15.100:0/3454796140 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f803800cdb0 con 0x7f80440ad9c0 2024-09-17T04:24:55.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.867+0000 7f8041600640 1 -- 172.21.15.100:0/3454796140 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038010690 con 0x7f80440ad9c0 2024-09-17T04:24:55.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.867+0000 7f804a800640 1 -- 172.21.15.100:0/3454796140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80441f68c0 con 0x7f80440ad9c0 2024-09-17T04:24:55.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.868+0000 7f804a800640 1 -- 172.21.15.100:0/3454796140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80441f6e80 con 0x7f80440ad9c0 2024-09-17T04:24:55.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.869+0000 7f804a800640 1 -- 172.21.15.100:0/3454796140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f804406f900 con 0x7f80440ad9c0 2024-09-17T04:24:55.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.870+0000 7f8041600640 1 -- 172.21.15.100:0/3454796140 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8038010830 con 0x7f80440ad9c0 2024-09-17T04:24:55.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.871+0000 7f8041600640 1 --2- 172.21.15.100:0/3454796140 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8024078bc0 0x7f802407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:55.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.871+0000 7f8041600640 1 -- 172.21.15.100:0/3454796140 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f803809b7b0 con 0x7f80440ad9c0 2024-09-17T04:24:55.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.871+0000 7f8048a00640 1 --2- 172.21.15.100:0/3454796140 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8024078bc0 0x7f802407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:55.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.872+0000 7f8048a00640 1 --2- 172.21.15.100:0/3454796140 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8024078bc0 0x7f802407b0d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8044159a90 tx=0x7f8030009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:55.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:55.875+0000 7f8041600640 1 -- 172.21.15.100:0/3454796140 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8038063480 con 0x7f80440ad9c0 2024-09-17T04:24:56.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.097+0000 7f804a800640 1 -- 172.21.15.100:0/3454796140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f804406de60 con 0x7f80440ad9c0 2024-09-17T04:24:56.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.098+0000 7f8041600640 1 -- 172.21.15.100:0/3454796140 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7f8038067eb0 con 0x7f80440ad9c0 2024-09-17T04:24:56.098 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:56.098 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":8,"btime":"2024-09-17T04:24:18:058994+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4328,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2129046042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":2129046042},{"type":"v1","addr":"172.21.15.100:6835","nonce":2129046042}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4334,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/44255118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":44255118},{"type":"v1","addr":"172.21.15.184:6836","nonce":44255118}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/1644264480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1644264480},{"type":"v1","addr":"172.21.15.100:6834","nonce":1644264480}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:24:14.527272+0000","modified":"2024-09-17T04:24:17.543043+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4368},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4368":{"gid":4368,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/2211103925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2211103925},{"type":"v1","addr":"172.21.15.184:6837","nonce":2211103925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4368,"qdb_cluster":[4368]},"id":1}]} 2024-09-17T04:24:56.099 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 8 2024-09-17T04:24:56.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.100+0000 7f804a800640 1 -- 172.21.15.100:0/3454796140 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8024078bc0 msgr2=0x7f802407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:56.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.100+0000 7f804a800640 1 --2- 172.21.15.100:0/3454796140 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8024078bc0 0x7f802407b0d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8044159a90 tx=0x7f8030009040 comp rx=0 tx=0).stop 2024-09-17T04:24:56.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.100+0000 7f804a800640 1 -- 172.21.15.100:0/3454796140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80440ad9c0 msgr2=0x7f804414b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:56.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.100+0000 7f804a800640 1 --2- 172.21.15.100:0/3454796140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80440ad9c0 0x7f804414b550 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f803800e790 tx=0x7f803800ec50 comp rx=0 tx=0).stop 2024-09-17T04:24:56.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.100+0000 7f804a800640 1 -- 172.21.15.100:0/3454796140 shutdown_connections 2024-09-17T04:24:56.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.100+0000 7f804a800640 1 --2- 172.21.15.100:0/3454796140 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8024078bc0 0x7f802407b0d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:56.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.100+0000 7f804a800640 1 --2- 172.21.15.100:0/3454796140 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f804415a0f0 0x7f80440b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:56.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.100+0000 7f804a800640 1 --2- 172.21.15.100:0/3454796140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80440b2f60 0x7f80441481d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:56.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.100+0000 7f804a800640 1 --2- 172.21.15.100:0/3454796140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80440ad9c0 0x7f804414b550 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:56.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.100+0000 7f804a800640 1 -- 172.21.15.100:0/3454796140 >> 172.21.15.100:0/3454796140 conn(0x7f80440b1e40 msgr2=0x7f80440b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:56.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.100+0000 7f804a800640 1 -- 172.21.15.100:0/3454796140 shutdown_connections 2024-09-17T04:24:56.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.100+0000 7f804a800640 1 -- 172.21.15.100:0/3454796140 wait complete. 2024-09-17T04:24:56.125 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:24:56.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.346+0000 7f61d0c00640 1 -- 172.21.15.100:0/293825826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61cc0b3550 msgr2=0x7f61cc0b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:56.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.346+0000 7f61d0c00640 1 --2- 172.21.15.100:0/293825826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61cc0b3550 0x7f61cc0b3960 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f61c4005fd0 tx=0x7f61c402aa10 comp rx=0 tx=0).stop 2024-09-17T04:24:56.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.346+0000 7f61d0c00640 1 -- 172.21.15.100:0/293825826 shutdown_connections 2024-09-17T04:24:56.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.346+0000 7f61d0c00640 1 --2- 172.21.15.100:0/293825826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61cc0b3550 0x7f61cc0b3960 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:56.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.346+0000 7f61d0c00640 1 --2- 172.21.15.100:0/293825826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61cc0ad0e0 0x7f61cc0ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:56.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.346+0000 7f61d0c00640 1 -- 172.21.15.100:0/293825826 >> 172.21.15.100:0/293825826 conn(0x7f61cc06d730 msgr2=0x7f61cc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:56.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.346+0000 7f61d0c00640 1 -- 172.21.15.100:0/293825826 shutdown_connections 2024-09-17T04:24:56.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.346+0000 7f61d0c00640 1 -- 172.21.15.100:0/293825826 wait complete. 2024-09-17T04:24:56.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.348+0000 7f61d0c00640 1 Processor -- start 2024-09-17T04:24:56.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.348+0000 7f61d0c00640 1 -- start start 2024-09-17T04:24:56.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.350+0000 7f61d0c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61cc0ad0e0 0x7f61cc153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:56.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.350+0000 7f61d0c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61cc0b3550 0x7f61cc09d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:56.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.350+0000 7f61d0c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61cc15d760 0x7f61cc09ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:56.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.350+0000 7f61d0c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f61cc0b4da0 con 0x7f61cc15d760 2024-09-17T04:24:56.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.350+0000 7f61d0c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f61cc0b4c20 con 0x7f61cc0ad0e0 2024-09-17T04:24:56.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.350+0000 7f61d0c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f61cc0b3ea0 con 0x7f61cc0b3550 2024-09-17T04:24:56.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.350+0000 7f61ca800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61cc15d760 0x7f61cc09ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:56.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.350+0000 7f61ca800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61cc15d760 0x7f61cc09ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51042/0 (socket says 172.21.15.100:51042) 2024-09-17T04:24:56.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.350+0000 7f61ca800640 1 -- 172.21.15.100:0/1706546927 learned_addr learned my addr 172.21.15.100:0/1706546927 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:56.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.350+0000 7f61c9e00640 1 --2- 172.21.15.100:0/1706546927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61cc0ad0e0 0x7f61cc153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:56.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.350+0000 7f61c9400640 1 --2- 172.21.15.100:0/1706546927 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61cc0b3550 0x7f61cc09d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:56.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.350+0000 7f61ca800640 1 -- 172.21.15.100:0/1706546927 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61cc0b3550 msgr2=0x7f61cc09d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:56.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.350+0000 7f61ca800640 1 --2- 172.21.15.100:0/1706546927 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61cc0b3550 0x7f61cc09d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:56.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.350+0000 7f61ca800640 1 -- 172.21.15.100:0/1706546927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61cc0ad0e0 msgr2=0x7f61cc153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:56.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.350+0000 7f61ca800640 1 --2- 172.21.15.100:0/1706546927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61cc0ad0e0 0x7f61cc153e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:56.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.350+0000 7f61ca800640 1 -- 172.21.15.100:0/1706546927 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61cc154a90 con 0x7f61cc15d760 2024-09-17T04:24:56.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.351+0000 7f61ca800640 1 --2- 172.21.15.100:0/1706546927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61cc15d760 0x7f61cc09ddb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f61c4006010 tx=0x7f61c4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:56.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.351+0000 7f61c2000640 1 -- 172.21.15.100:0/1706546927 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c4046070 con 0x7f61cc15d760 2024-09-17T04:24:56.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.351+0000 7f61c2000640 1 -- 172.21.15.100:0/1706546927 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61c4004540 con 0x7f61cc15d760 2024-09-17T04:24:56.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.351+0000 7f61d0c00640 1 -- 172.21.15.100:0/1706546927 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61cc154d70 con 0x7f61cc15d760 2024-09-17T04:24:56.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.351+0000 7f61c2000640 1 -- 172.21.15.100:0/1706546927 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c4033070 con 0x7f61cc15d760 2024-09-17T04:24:56.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.351+0000 7f61d0c00640 1 -- 172.21.15.100:0/1706546927 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61cc155330 con 0x7f61cc15d760 2024-09-17T04:24:56.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.352+0000 7f61d0c00640 1 -- 172.21.15.100:0/1706546927 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61cc0b4330 con 0x7f61cc15d760 2024-09-17T04:24:56.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.353+0000 7f61c2000640 1 -- 172.21.15.100:0/1706546927 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f61c4049070 con 0x7f61cc15d760 2024-09-17T04:24:56.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.354+0000 7f61c2000640 1 --2- 172.21.15.100:0/1706546927 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f61ac078bc0 0x7f61ac07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:56.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.355+0000 7f61c2000640 1 -- 172.21.15.100:0/1706546927 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f61c40bbee0 con 0x7f61cc15d760 2024-09-17T04:24:56.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.355+0000 7f61c9e00640 1 --2- 172.21.15.100:0/1706546927 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f61ac078bc0 0x7f61ac07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:56.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.356+0000 7f61c9e00640 1 --2- 172.21.15.100:0/1706546927 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f61ac078bc0 0x7f61ac07b0d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f61bc001720 tx=0x7f61bc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:56.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.359+0000 7f61c2000640 1 -- 172.21.15.100:0/1706546927 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f61c403e070 con 0x7f61cc15d760 2024-09-17T04:24:56.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.541+0000 7f61d0c00640 1 -- 172.21.15.100:0/1706546927 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f61cc1547b0 con 0x7f61cc15d760 2024-09-17T04:24:56.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.542+0000 7f61c2000640 1 -- 172.21.15.100:0/1706546927 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+16904 (secure 0 0 0) 0x7f61c4083c40 con 0x7f61cc15d760 2024-09-17T04:24:56.543 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:24:56.543 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":25,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:24:53.559880+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:24:12.826665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T04:24:13.485446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.b","mappings":[{"from":3,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:24:56.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.545+0000 7f61d0c00640 1 -- 172.21.15.100:0/1706546927 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f61ac078bc0 msgr2=0x7f61ac07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:56.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.545+0000 7f61d0c00640 1 --2- 172.21.15.100:0/1706546927 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f61ac078bc0 0x7f61ac07b0d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f61bc001720 tx=0x7f61bc009040 comp rx=0 tx=0).stop 2024-09-17T04:24:56.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.545+0000 7f61d0c00640 1 -- 172.21.15.100:0/1706546927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61cc15d760 msgr2=0x7f61cc09ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:56.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.545+0000 7f61d0c00640 1 --2- 172.21.15.100:0/1706546927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61cc15d760 0x7f61cc09ddb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f61c4006010 tx=0x7f61c4003f10 comp rx=0 tx=0).stop 2024-09-17T04:24:56.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.545+0000 7f61d0c00640 1 -- 172.21.15.100:0/1706546927 shutdown_connections 2024-09-17T04:24:56.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.545+0000 7f61d0c00640 1 --2- 172.21.15.100:0/1706546927 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f61ac078bc0 0x7f61ac07b0d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:56.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.545+0000 7f61d0c00640 1 --2- 172.21.15.100:0/1706546927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61cc15d760 0x7f61cc09ddb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:56.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.545+0000 7f61d0c00640 1 --2- 172.21.15.100:0/1706546927 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61cc0b3550 0x7f61cc09d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:56.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.545+0000 7f61d0c00640 1 --2- 172.21.15.100:0/1706546927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61cc0ad0e0 0x7f61cc153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:56.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.545+0000 7f61d0c00640 1 -- 172.21.15.100:0/1706546927 >> 172.21.15.100:0/1706546927 conn(0x7f61cc06d730 msgr2=0x7f61cc1535c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:56.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.545+0000 7f61d0c00640 1 -- 172.21.15.100:0/1706546927 shutdown_connections 2024-09-17T04:24:56.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.545+0000 7f61d0c00640 1 -- 172.21.15.100:0/1706546927 wait complete. 2024-09-17T04:24:56.567 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T04:24:56.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.791+0000 7f19a3c00640 1 -- 172.21.15.100:0/4072670842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f199c0ce490 msgr2=0x7f199c0ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:56.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.791+0000 7f19a3c00640 1 --2- 172.21.15.100:0/4072670842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f199c0ce490 0x7f199c0ce8a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f199c06dd40 tx=0x7f198802a440 comp rx=0 tx=0).stop 2024-09-17T04:24:56.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.791+0000 7f19a3c00640 1 -- 172.21.15.100:0/4072670842 shutdown_connections 2024-09-17T04:24:56.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.791+0000 7f19a3c00640 1 --2- 172.21.15.100:0/4072670842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f199c0ad1c0 0x7f199c0ad5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:56.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.791+0000 7f19a3c00640 1 --2- 172.21.15.100:0/4072670842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f199c0ce490 0x7f199c0ce8a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:56.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.791+0000 7f19a3c00640 1 --2- 172.21.15.100:0/4072670842 >> v2:172.21.15.184:3300/0 conn(0x7f199c070660 0x7f199c0af6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:56.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.791+0000 7f19a3c00640 1 -- 172.21.15.100:0/4072670842 >> 172.21.15.100:0/4072670842 conn(0x7f199c06d3d0 msgr2=0x7f199c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:56.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.791+0000 7f19a3c00640 1 -- 172.21.15.100:0/4072670842 shutdown_connections 2024-09-17T04:24:56.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.791+0000 7f19a3c00640 1 -- 172.21.15.100:0/4072670842 wait complete. 2024-09-17T04:24:56.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.794+0000 7f19a3c00640 1 Processor -- start 2024-09-17T04:24:56.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.794+0000 7f19a3c00640 1 -- start start 2024-09-17T04:24:56.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.795+0000 7f19a3c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f199c070660 0x7f199c0c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:56.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.795+0000 7f19a3c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f199c0ad1c0 0x7f199c1f5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:56.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.795+0000 7f19a3c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f199c0ce490 0x7f199c157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:56.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.795+0000 7f19a3c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f199c0adea0 con 0x7f199c0ad1c0 2024-09-17T04:24:56.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.795+0000 7f19a3c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f199c0add20 con 0x7f199c070660 2024-09-17T04:24:56.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.795+0000 7f19a3c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f199c0ccd70 con 0x7f199c0ce490 2024-09-17T04:24:56.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.795+0000 7f199be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f199c0ad1c0 0x7f199c1f5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:56.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.795+0000 7f199be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f199c0ad1c0 0x7f199c1f5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51054/0 (socket says 172.21.15.100:51054) 2024-09-17T04:24:56.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.796+0000 7f199be00640 1 -- 172.21.15.100:0/1495643763 learned_addr learned my addr 172.21.15.100:0/1495643763 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:56.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.796+0000 7f19a1200640 1 --2- 172.21.15.100:0/1495643763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f199c070660 0x7f199c0c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:56.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.796+0000 7f19a1c00640 1 --2- 172.21.15.100:0/1495643763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f199c0ce490 0x7f199c157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:56.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.796+0000 7f199be00640 1 -- 172.21.15.100:0/1495643763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f199c0ce490 msgr2=0x7f199c157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:56.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.796+0000 7f199be00640 1 --2- 172.21.15.100:0/1495643763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f199c0ce490 0x7f199c157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:56.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.796+0000 7f199be00640 1 -- 172.21.15.100:0/1495643763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f199c070660 msgr2=0x7f199c0c90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:56.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.796+0000 7f199be00640 1 --2- 172.21.15.100:0/1495643763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f199c070660 0x7f199c0c90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:56.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.796+0000 7f199be00640 1 -- 172.21.15.100:0/1495643763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f199c0c9ad0 con 0x7f199c0ad1c0 2024-09-17T04:24:56.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.796+0000 7f19a1c00640 1 --2- 172.21.15.100:0/1495643763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f199c0ce490 0x7f199c157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:24:56.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.796+0000 7f19a1200640 1 --2- 172.21.15.100:0/1495643763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f199c070660 0x7f199c0c90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:24:56.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.796+0000 7f199be00640 1 --2- 172.21.15.100:0/1495643763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f199c0ad1c0 0x7f199c1f5290 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1988034040 tx=0x7f1988003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:56.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.796+0000 7f1998c00640 1 -- 172.21.15.100:0/1495643763 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198803e030 con 0x7f199c0ad1c0 2024-09-17T04:24:56.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.797+0000 7f1998c00640 1 -- 172.21.15.100:0/1495643763 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1988003e00 con 0x7f199c0ad1c0 2024-09-17T04:24:56.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.797+0000 7f19a3c00640 1 -- 172.21.15.100:0/1495643763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f199c0c9db0 con 0x7f199c0ad1c0 2024-09-17T04:24:56.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.797+0000 7f1998c00640 1 -- 172.21.15.100:0/1495643763 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1988035040 con 0x7f199c0ad1c0 2024-09-17T04:24:56.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.797+0000 7f19a3c00640 1 -- 172.21.15.100:0/1495643763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f199c0c0cd0 con 0x7f199c0ad1c0 2024-09-17T04:24:56.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.798+0000 7f19a3c00640 1 -- 172.21.15.100:0/1495643763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f199c0c9840 con 0x7f199c0ad1c0 2024-09-17T04:24:56.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.799+0000 7f1998c00640 1 -- 172.21.15.100:0/1495643763 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1988041030 con 0x7f199c0ad1c0 2024-09-17T04:24:56.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.800+0000 7f1998c00640 1 --2- 172.21.15.100:0/1495643763 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f197c078b90 0x7f197c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:56.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.800+0000 7f1998c00640 1 -- 172.21.15.100:0/1495643763 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f19880bb690 con 0x7f199c0ad1c0 2024-09-17T04:24:56.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.800+0000 7f19a1200640 1 --2- 172.21.15.100:0/1495643763 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f197c078b90 0x7f197c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:56.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.801+0000 7f19a1200640 1 --2- 172.21.15.100:0/1495643763 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f197c078b90 0x7f197c07b0a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1990005fd0 tx=0x7f199000b530 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:56.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:56.804+0000 7f1998c00640 1 -- 172.21.15.100:0/1495643763 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f19880839b0 con 0x7f199c0ad1c0 2024-09-17T04:24:57.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.023+0000 7f19a3c00640 1 -- 172.21.15.100:0/1495643763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f199c0ad5d0 con 0x7f199c0ad1c0 2024-09-17T04:24:57.024 INFO:tasks.ceph.mon.a.smithi100.stderr:2024-09-17T04:24:57.024+0000 7fd3d0a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T04:24:57.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.026+0000 7f1998c00640 1 -- 172.21.15.100:0/1495643763 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f19880832b0 con 0x7f199c0ad1c0 2024-09-17T04:24:57.026 INFO:teuthology.orchestra.run.smithi100.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T04:24:57.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.028+0000 7f19a3c00640 1 -- 172.21.15.100:0/1495643763 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f197c078b90 msgr2=0x7f197c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:57.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.028+0000 7f19a3c00640 1 --2- 172.21.15.100:0/1495643763 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f197c078b90 0x7f197c07b0a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1990005fd0 tx=0x7f199000b530 comp rx=0 tx=0).stop 2024-09-17T04:24:57.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.028+0000 7f19a3c00640 1 -- 172.21.15.100:0/1495643763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f199c0ad1c0 msgr2=0x7f199c1f5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:57.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.028+0000 7f19a3c00640 1 --2- 172.21.15.100:0/1495643763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f199c0ad1c0 0x7f199c1f5290 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1988034040 tx=0x7f1988003910 comp rx=0 tx=0).stop 2024-09-17T04:24:57.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.028+0000 7f19a3c00640 1 -- 172.21.15.100:0/1495643763 shutdown_connections 2024-09-17T04:24:57.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.028+0000 7f19a3c00640 1 --2- 172.21.15.100:0/1495643763 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f197c078b90 0x7f197c07b0a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:57.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.028+0000 7f19a3c00640 1 --2- 172.21.15.100:0/1495643763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f199c0ce490 0x7f199c157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:57.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.028+0000 7f19a3c00640 1 --2- 172.21.15.100:0/1495643763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f199c0ad1c0 0x7f199c1f5290 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:57.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.028+0000 7f19a3c00640 1 --2- 172.21.15.100:0/1495643763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f199c070660 0x7f199c0c90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:57.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.028+0000 7f19a3c00640 1 -- 172.21.15.100:0/1495643763 >> 172.21.15.100:0/1495643763 conn(0x7f199c06d3d0 msgr2=0x7f199c0b4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:57.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.029+0000 7f19a3c00640 1 -- 172.21.15.100:0/1495643763 shutdown_connections 2024-09-17T04:24:57.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.029+0000 7f19a3c00640 1 -- 172.21.15.100:0/1495643763 wait complete. 2024-09-17T04:24:57.042 INFO:tasks.ceph.mds.d.smithi184.stderr: -7382> 2024-09-17T04:24:20.173+0000 7f782e000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T04:24:57.054 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T04:24:57.167 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-09-17T04:24:57.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.276+0000 7f6f0c800640 1 -- 172.21.15.100:0/342784262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f04145a40 msgr2=0x7f6f04073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:57.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.276+0000 7f6f0c800640 1 --2- 172.21.15.100:0/342784262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f04145a40 0x7f6f04073540 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4005fd0 tx=0x7f6ef402a670 comp rx=0 tx=0).stop 2024-09-17T04:24:57.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.276+0000 7f6f0c800640 1 -- 172.21.15.100:0/342784262 shutdown_connections 2024-09-17T04:24:57.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.276+0000 7f6f0c800640 1 --2- 172.21.15.100:0/342784262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f04145a40 0x7f6f04073540 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:57.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.276+0000 7f6f0c800640 1 --2- 172.21.15.100:0/342784262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f04145550 0x7f6f0406fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:57.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.276+0000 7f6f0c800640 1 -- 172.21.15.100:0/342784262 >> 172.21.15.100:0/342784262 conn(0x7f6f0406d1c0 msgr2=0x7f6f0406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:57.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.276+0000 7f6f0c800640 1 -- 172.21.15.100:0/342784262 shutdown_connections 2024-09-17T04:24:57.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.276+0000 7f6f0c800640 1 -- 172.21.15.100:0/342784262 wait complete. 2024-09-17T04:24:57.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.278+0000 7f6f0c800640 1 Processor -- start 2024-09-17T04:24:57.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.278+0000 7f6f0c800640 1 -- start start 2024-09-17T04:24:57.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.280+0000 7f6f0c800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f04073a80 0x7f6f041f43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:57.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.280+0000 7f6f0c800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f04145550 0x7f6f041f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:57.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.280+0000 7f6f0c800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f04145a40 0x7f6f0413af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:57.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.280+0000 7f6f0c800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6f04160a30 con 0x7f6f04145550 2024-09-17T04:24:57.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.280+0000 7f6f0c800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6f040ce940 con 0x7f6f04073a80 2024-09-17T04:24:57.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.280+0000 7f6f0c800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6f04160bb0 con 0x7f6f04145a40 2024-09-17T04:24:57.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.280+0000 7f6f09400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f04145550 0x7f6f041f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:57.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.280+0000 7f6f09400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f04145550 0x7f6f041f48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51060/0 (socket says 172.21.15.100:51060) 2024-09-17T04:24:57.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.280+0000 7f6f09400640 1 -- 172.21.15.100:0/2000271041 learned_addr learned my addr 172.21.15.100:0/2000271041 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:57.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.280+0000 7f6f09e00640 1 --2- 172.21.15.100:0/2000271041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f04073a80 0x7f6f041f43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:57.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.280+0000 7f6f0a800640 1 --2- 172.21.15.100:0/2000271041 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f04145a40 0x7f6f0413af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:57.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.280+0000 7f6f09400640 1 -- 172.21.15.100:0/2000271041 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f04145a40 msgr2=0x7f6f0413af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:57.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.280+0000 7f6f09400640 1 --2- 172.21.15.100:0/2000271041 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f04145a40 0x7f6f0413af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:57.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.280+0000 7f6f09400640 1 -- 172.21.15.100:0/2000271041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f04073a80 msgr2=0x7f6f041f43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:57.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.280+0000 7f6f09400640 1 --2- 172.21.15.100:0/2000271041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f04073a80 0x7f6f041f43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:57.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.280+0000 7f6f09400640 1 -- 172.21.15.100:0/2000271041 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f04135df0 con 0x7f6f04145550 2024-09-17T04:24:57.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.280+0000 7f6f0a800640 1 --2- 172.21.15.100:0/2000271041 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f04145a40 0x7f6f0413af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:24:57.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.281+0000 7f6f09400640 1 --2- 172.21.15.100:0/2000271041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f04145550 0x7f6f041f48e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4000c00 tx=0x7f6ef40038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:57.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.281+0000 7f6efe000640 1 -- 172.21.15.100:0/2000271041 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef403e030 con 0x7f6f04145550 2024-09-17T04:24:57.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.281+0000 7f6efe000640 1 -- 172.21.15.100:0/2000271041 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ef4003db0 con 0x7f6f04145550 2024-09-17T04:24:57.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.281+0000 7f6efe000640 1 -- 172.21.15.100:0/2000271041 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4035040 con 0x7f6f04145550 2024-09-17T04:24:57.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.281+0000 7f6f0c800640 1 -- 172.21.15.100:0/2000271041 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f04138960 con 0x7f6f04145550 2024-09-17T04:24:57.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.281+0000 7f6f0c800640 1 -- 172.21.15.100:0/2000271041 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f04138f20 con 0x7f6f04145550 2024-09-17T04:24:57.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.282+0000 7f6f0c800640 1 -- 172.21.15.100:0/2000271041 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f041610d0 con 0x7f6f04145550 2024-09-17T04:24:57.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.283+0000 7f6efe000640 1 -- 172.21.15.100:0/2000271041 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6ef4041030 con 0x7f6f04145550 2024-09-17T04:24:57.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.284+0000 7f6efe000640 1 --2- 172.21.15.100:0/2000271041 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6ee8078b90 0x7f6ee807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:57.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.284+0000 7f6efe000640 1 -- 172.21.15.100:0/2000271041 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5911+0+0 (secure 0 0 0) 0x7f6ef40bb4b0 con 0x7f6f04145550 2024-09-17T04:24:57.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.285+0000 7f6f09e00640 1 --2- 172.21.15.100:0/2000271041 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6ee8078b90 0x7f6ee807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:57.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.285+0000 7f6f09e00640 1 --2- 172.21.15.100:0/2000271041 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6ee8078b90 0x7f6ee807b0a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8009b30 tx=0x7f6ef8005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:57.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.289+0000 7f6efe000640 1 -- 172.21.15.100:0/2000271041 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6ef40831c0 con 0x7f6f04145550 2024-09-17T04:24:57.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:57.505+0000 7f6f0c800640 1 -- 172.21.15.100:0/2000271041 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f6f0406fe60 con 0x7f6f04145550 2024-09-17T04:24:58.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.028+0000 7f6efe000640 1 -- 172.21.15.100:0/2000271041 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f6ef4087bf0 con 0x7f6f04145550 2024-09-17T04:24:58.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.030+0000 7f6f0c800640 1 -- 172.21.15.100:0/2000271041 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6ee8078b90 msgr2=0x7f6ee807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:58.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.030+0000 7f6f0c800640 1 --2- 172.21.15.100:0/2000271041 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6ee8078b90 0x7f6ee807b0a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8009b30 tx=0x7f6ef8005f90 comp rx=0 tx=0).stop 2024-09-17T04:24:58.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.030+0000 7f6f0c800640 1 -- 172.21.15.100:0/2000271041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f04145550 msgr2=0x7f6f041f48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:58.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.030+0000 7f6f0c800640 1 --2- 172.21.15.100:0/2000271041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f04145550 0x7f6f041f48e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4000c00 tx=0x7f6ef40038a0 comp rx=0 tx=0).stop 2024-09-17T04:24:58.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.030+0000 7f6f0c800640 1 -- 172.21.15.100:0/2000271041 shutdown_connections 2024-09-17T04:24:58.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.030+0000 7f6f0c800640 1 --2- 172.21.15.100:0/2000271041 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6ee8078b90 0x7f6ee807b0a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:58.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.030+0000 7f6f0c800640 1 --2- 172.21.15.100:0/2000271041 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f04145a40 0x7f6f0413af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:58.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.030+0000 7f6f0c800640 1 --2- 172.21.15.100:0/2000271041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f04145550 0x7f6f041f48e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:58.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.030+0000 7f6f0c800640 1 --2- 172.21.15.100:0/2000271041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f04073a80 0x7f6f041f43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:58.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.030+0000 7f6f0c800640 1 -- 172.21.15.100:0/2000271041 >> 172.21.15.100:0/2000271041 conn(0x7f6f0406d1c0 msgr2=0x7f6f04141aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:58.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.030+0000 7f6f0c800640 1 -- 172.21.15.100:0/2000271041 shutdown_connections 2024-09-17T04:24:58.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.031+0000 7f6f0c800640 1 -- 172.21.15.100:0/2000271041 wait complete. 2024-09-17T04:24:58.054 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T04:24:58.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.276+0000 7faf84a00640 1 -- 172.21.15.100:0/2558646403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf800b3ec0 msgr2=0x7faf800b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:58.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.276+0000 7faf84a00640 1 --2- 172.21.15.100:0/2558646403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf800b3ec0 0x7faf800b47c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7faf70009830 tx=0x7faf7002a480 comp rx=0 tx=0).stop 2024-09-17T04:24:58.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.276+0000 7faf84a00640 1 -- 172.21.15.100:0/2558646403 shutdown_connections 2024-09-17T04:24:58.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.276+0000 7faf84a00640 1 --2- 172.21.15.100:0/2558646403 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf800705b0 0x7faf800af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:58.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.276+0000 7faf84a00640 1 --2- 172.21.15.100:0/2558646403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf800b3ec0 0x7faf800b47c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:58.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.276+0000 7faf84a00640 1 -- 172.21.15.100:0/2558646403 >> 172.21.15.100:0/2558646403 conn(0x7faf800b1dd0 msgr2=0x7faf800b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:58.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.276+0000 7faf84a00640 1 -- 172.21.15.100:0/2558646403 shutdown_connections 2024-09-17T04:24:58.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.276+0000 7faf84a00640 1 -- 172.21.15.100:0/2558646403 wait complete. 2024-09-17T04:24:58.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.279+0000 7faf84a00640 1 Processor -- start 2024-09-17T04:24:58.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.279+0000 7faf84a00640 1 -- start start 2024-09-17T04:24:58.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.280+0000 7faf84a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf800705b0 0x7faf801f5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:58.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.280+0000 7faf84a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf800b3ec0 0x7faf8014e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:58.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.280+0000 7faf84a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf801532e0 0x7faf80146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:58.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.280+0000 7faf84a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faf80153a80 con 0x7faf801532e0 2024-09-17T04:24:58.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.280+0000 7faf84a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faf80153900 con 0x7faf800b3ec0 2024-09-17T04:24:58.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.280+0000 7faf84a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faf80153c00 con 0x7faf800705b0 2024-09-17T04:24:58.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.280+0000 7faf7fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf801532e0 0x7faf80146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:58.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.281+0000 7faf7fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf801532e0 0x7faf80146ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51082/0 (socket says 172.21.15.100:51082) 2024-09-17T04:24:58.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.281+0000 7faf7fe00640 1 -- 172.21.15.100:0/954106329 learned_addr learned my addr 172.21.15.100:0/954106329 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:58.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.281+0000 7faf7f400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf800705b0 0x7faf801f5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:58.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.281+0000 7faf7ea00640 1 --2- 172.21.15.100:0/954106329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf800b3ec0 0x7faf8014e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:58.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.281+0000 7faf7fe00640 1 -- 172.21.15.100:0/954106329 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf800705b0 msgr2=0x7faf801f5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:58.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.281+0000 7faf7fe00640 1 --2- 172.21.15.100:0/954106329 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf800705b0 0x7faf801f5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:58.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.281+0000 7faf7fe00640 1 -- 172.21.15.100:0/954106329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf800b3ec0 msgr2=0x7faf8014e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:58.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.281+0000 7faf7fe00640 1 --2- 172.21.15.100:0/954106329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf800b3ec0 0x7faf8014e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:58.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.281+0000 7faf7fe00640 1 -- 172.21.15.100:0/954106329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf80072fa0 con 0x7faf801532e0 2024-09-17T04:24:58.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.281+0000 7faf7fe00640 1 --2- 172.21.15.100:0/954106329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf801532e0 0x7faf80146ae0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7faf7800c760 tx=0x7faf7800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:58.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.281+0000 7faf77400640 1 -- 172.21.15.100:0/954106329 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78017070 con 0x7faf801532e0 2024-09-17T04:24:58.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.282+0000 7faf77400640 1 -- 172.21.15.100:0/954106329 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf7800ce10 con 0x7faf801532e0 2024-09-17T04:24:58.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.282+0000 7faf77400640 1 -- 172.21.15.100:0/954106329 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf780052b0 con 0x7faf801532e0 2024-09-17T04:24:58.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.282+0000 7faf84a00640 1 -- 172.21.15.100:0/954106329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf80073280 con 0x7faf801532e0 2024-09-17T04:24:58.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.282+0000 7faf84a00640 1 -- 172.21.15.100:0/954106329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf80073840 con 0x7faf801532e0 2024-09-17T04:24:58.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.283+0000 7faf84a00640 1 -- 172.21.15.100:0/954106329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf800af3c0 con 0x7faf801532e0 2024-09-17T04:24:58.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.284+0000 7faf77400640 1 -- 172.21.15.100:0/954106329 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faf7801a070 con 0x7faf801532e0 2024-09-17T04:24:58.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.285+0000 7faf77400640 1 --2- 172.21.15.100:0/954106329 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faf60078b90 0x7faf6007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:58.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.285+0000 7faf77400640 1 -- 172.21.15.100:0/954106329 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5911+0+0 (secure 0 0 0) 0x7faf7809a410 con 0x7faf801532e0 2024-09-17T04:24:58.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.285+0000 7faf7f400640 1 --2- 172.21.15.100:0/954106329 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faf60078b90 0x7faf6007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:58.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.286+0000 7faf7f400640 1 --2- 172.21.15.100:0/954106329 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faf60078b90 0x7faf6007b0a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7faf70034000 tx=0x7faf70004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:58.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.289+0000 7faf77400640 1 -- 172.21.15.100:0/954106329 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7faf800b4ec0 con 0x7faf801532e0 2024-09-17T04:24:58.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:58.471+0000 7faf84a00640 1 -- 172.21.15.100:0/954106329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7faf800008d0 con 0x7faf801532e0 2024-09-17T04:24:59.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.030+0000 7faf77400640 1 -- 172.21.15.100:0/954106329 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v27) ==== 181+0+0 (secure 0 0 0) 0x7faf78062090 con 0x7faf801532e0 2024-09-17T04:24:59.030 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' removed 2024-09-17T04:24:59.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.033+0000 7faf84a00640 1 -- 172.21.15.100:0/954106329 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faf60078b90 msgr2=0x7faf6007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:59.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.033+0000 7faf84a00640 1 --2- 172.21.15.100:0/954106329 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faf60078b90 0x7faf6007b0a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7faf70034000 tx=0x7faf70004730 comp rx=0 tx=0).stop 2024-09-17T04:24:59.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.033+0000 7faf84a00640 1 -- 172.21.15.100:0/954106329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf801532e0 msgr2=0x7faf80146ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:59.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.033+0000 7faf84a00640 1 --2- 172.21.15.100:0/954106329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf801532e0 0x7faf80146ae0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7faf7800c760 tx=0x7faf7800cc20 comp rx=0 tx=0).stop 2024-09-17T04:24:59.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.033+0000 7faf84a00640 1 -- 172.21.15.100:0/954106329 shutdown_connections 2024-09-17T04:24:59.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.033+0000 7faf84a00640 1 --2- 172.21.15.100:0/954106329 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faf60078b90 0x7faf6007b0a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:59.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.033+0000 7faf84a00640 1 --2- 172.21.15.100:0/954106329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf801532e0 0x7faf80146ae0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:59.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.033+0000 7faf84a00640 1 --2- 172.21.15.100:0/954106329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf800b3ec0 0x7faf8014e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:59.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.033+0000 7faf84a00640 1 --2- 172.21.15.100:0/954106329 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf800705b0 0x7faf801f5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:59.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.033+0000 7faf84a00640 1 -- 172.21.15.100:0/954106329 >> 172.21.15.100:0/954106329 conn(0x7faf800b1dd0 msgr2=0x7faf800afb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:59.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.034+0000 7faf84a00640 1 -- 172.21.15.100:0/954106329 shutdown_connections 2024-09-17T04:24:59.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.034+0000 7faf84a00640 1 -- 172.21.15.100:0/954106329 wait complete. 2024-09-17T04:24:59.059 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T04:24:59.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.282+0000 7ff9eb200640 1 -- 172.21.15.100:0/2058057347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9cc004d40 msgr2=0x7ff9cc005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:59.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.282+0000 7ff9eb200640 1 --2- 172.21.15.100:0/2058057347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9cc004d40 0x7ff9cc005150 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0009ed0 tx=0x7ff9cc005830 comp rx=0 tx=0).stop 2024-09-17T04:24:59.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.282+0000 7ff9ea800640 1 -- 172.21.15.100:0/2058057347 reap_dead start 2024-09-17T04:24:59.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.282+0000 7ff9eb200640 1 -- 172.21.15.100:0/2058057347 shutdown_connections 2024-09-17T04:24:59.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.282+0000 7ff9eb200640 1 -- 172.21.15.100:0/2058057347 >> 172.21.15.100:0/2058057347 conn(0x7ff9e40cd460 msgr2=0x7ff9e40cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:24:59.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.282+0000 7ff9eb200640 1 -- 172.21.15.100:0/2058057347 shutdown_connections 2024-09-17T04:24:59.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.282+0000 7ff9eb200640 1 -- 172.21.15.100:0/2058057347 wait complete. 2024-09-17T04:24:59.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.284+0000 7ff9eb200640 1 Processor -- start 2024-09-17T04:24:59.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.285+0000 7ff9eb200640 1 -- start start 2024-09-17T04:24:59.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.286+0000 7ff9eb200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9cc004d40 0x7ff9e4157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:59.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.286+0000 7ff9eb200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9e414c7f0 0x7ff9e414def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:59.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.286+0000 7ff9eb200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9e414e430 0x7ff9e41f6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:59.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.286+0000 7ff9eb200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e406fe90 con 0x7ff9cc004d40 2024-09-17T04:24:59.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.286+0000 7ff9eb200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e406fd10 con 0x7ff9e414c7f0 2024-09-17T04:24:59.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.286+0000 7ff9eb200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff9e415d060 con 0x7ff9e414e430 2024-09-17T04:24:59.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.286+0000 7ff9e9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9cc004d40 0x7ff9e4157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:59.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.286+0000 7ff9e9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9cc004d40 0x7ff9e4157ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51100/0 (socket says 172.21.15.100:51100) 2024-09-17T04:24:59.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.286+0000 7ff9e9e00640 1 -- 172.21.15.100:0/2715664133 learned_addr learned my addr 172.21.15.100:0/2715664133 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:24:59.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.287+0000 7ff9ea800640 1 --2- 172.21.15.100:0/2715664133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9e414e430 0x7ff9e41f6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:59.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.287+0000 7ff9e9400640 1 --2- 172.21.15.100:0/2715664133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9e414c7f0 0x7ff9e414def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:59.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.287+0000 7ff9e9e00640 1 -- 172.21.15.100:0/2715664133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9e414e430 msgr2=0x7ff9e41f6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:59.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.287+0000 7ff9e9e00640 1 --2- 172.21.15.100:0/2715664133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9e414e430 0x7ff9e41f6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:59.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.287+0000 7ff9e9e00640 1 -- 172.21.15.100:0/2715664133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9e414c7f0 msgr2=0x7ff9e414def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:24:59.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.287+0000 7ff9e9e00640 1 --2- 172.21.15.100:0/2715664133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9e414c7f0 0x7ff9e414def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:24:59.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.287+0000 7ff9e9e00640 1 -- 172.21.15.100:0/2715664133 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e41581e0 con 0x7ff9cc004d40 2024-09-17T04:24:59.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.287+0000 7ff9e9e00640 1 --2- 172.21.15.100:0/2715664133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9cc004d40 0x7ff9e4157ca0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff9d80066f0 tx=0x7ff9d80049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:59.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.287+0000 7ff9de000640 1 -- 172.21.15.100:0/2715664133 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8015020 con 0x7ff9cc004d40 2024-09-17T04:24:59.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.287+0000 7ff9de000640 1 -- 172.21.15.100:0/2715664133 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9d8010980 con 0x7ff9cc004d40 2024-09-17T04:24:59.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.287+0000 7ff9eb200640 1 -- 172.21.15.100:0/2715664133 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9e414e840 con 0x7ff9cc004d40 2024-09-17T04:24:59.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.287+0000 7ff9de000640 1 -- 172.21.15.100:0/2715664133 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8010df0 con 0x7ff9cc004d40 2024-09-17T04:24:59.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.287+0000 7ff9eb200640 1 -- 172.21.15.100:0/2715664133 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9e4157800 con 0x7ff9cc004d40 2024-09-17T04:24:59.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.288+0000 7ff9eb200640 1 -- 172.21.15.100:0/2715664133 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9e4157290 con 0x7ff9cc004d40 2024-09-17T04:24:59.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.290+0000 7ff9de000640 1 -- 172.21.15.100:0/2715664133 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff9d8018020 con 0x7ff9cc004d40 2024-09-17T04:24:59.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.290+0000 7ff9de000640 1 --2- 172.21.15.100:0/2715664133 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff9c8078b90 0x7ff9c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:24:59.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.291+0000 7ff9de000640 1 -- 172.21.15.100:0/2715664133 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5473+0+0 (secure 0 0 0) 0x7ff9d809b870 con 0x7ff9cc004d40 2024-09-17T04:24:59.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.291+0000 7ff9e9400640 1 --2- 172.21.15.100:0/2715664133 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff9c8078b90 0x7ff9c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:24:59.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.292+0000 7ff9e9400640 1 --2- 172.21.15.100:0/2715664133 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff9c8078b90 0x7ff9c807b0a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff9e40ccaa0 tx=0x7ff9d4005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:24:59.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.295+0000 7ff9de000640 1 -- 172.21.15.100:0/2715664133 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff9d801f020 con 0x7ff9cc004d40 2024-09-17T04:24:59.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:24:59.487+0000 7ff9eb200640 1 -- 172.21.15.100:0/2715664133 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff9e40cd210 con 0x7ff9cc004d40 2024-09-17T04:25:00.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.037+0000 7ff9de000640 1 -- 172.21.15.100:0/2715664133 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v28) ==== 169+0+0 (secure 0 0 0) 0x7ff9e414e840 con 0x7ff9cc004d40 2024-09-17T04:25:00.037 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' removed 2024-09-17T04:25:00.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.040+0000 7ff9eb200640 1 -- 172.21.15.100:0/2715664133 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff9c8078b90 msgr2=0x7ff9c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:00.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.040+0000 7ff9eb200640 1 --2- 172.21.15.100:0/2715664133 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff9c8078b90 0x7ff9c807b0a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff9e40ccaa0 tx=0x7ff9d4005f90 comp rx=0 tx=0).stop 2024-09-17T04:25:00.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.040+0000 7ff9eb200640 1 -- 172.21.15.100:0/2715664133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9cc004d40 msgr2=0x7ff9e4157ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:00.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.040+0000 7ff9eb200640 1 --2- 172.21.15.100:0/2715664133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9cc004d40 0x7ff9e4157ca0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff9d80066f0 tx=0x7ff9d80049d0 comp rx=0 tx=0).stop 2024-09-17T04:25:00.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.040+0000 7ff9eb200640 1 -- 172.21.15.100:0/2715664133 shutdown_connections 2024-09-17T04:25:00.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.040+0000 7ff9eb200640 1 --2- 172.21.15.100:0/2715664133 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff9c8078b90 0x7ff9c807b0a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:00.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.040+0000 7ff9eb200640 1 --2- 172.21.15.100:0/2715664133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9e414e430 0x7ff9e41f6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:00.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.040+0000 7ff9eb200640 1 --2- 172.21.15.100:0/2715664133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9e414c7f0 0x7ff9e414def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:00.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.040+0000 7ff9eb200640 1 --2- 172.21.15.100:0/2715664133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9cc004d40 0x7ff9e4157ca0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:00.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.040+0000 7ff9eb200640 1 -- 172.21.15.100:0/2715664133 >> 172.21.15.100:0/2715664133 conn(0x7ff9e40cd460 msgr2=0x7ff9e4150fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:00.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.040+0000 7ff9eb200640 1 -- 172.21.15.100:0/2715664133 shutdown_connections 2024-09-17T04:25:00.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.040+0000 7ff9eb200640 1 -- 172.21.15.100:0/2715664133 wait complete. 2024-09-17T04:25:00.068 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T04:25:00.068 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T04:25:00.069 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T04:25:00.069 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:25:00.069 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T04:25:00.069 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T04:25:00.069 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T04:25:00.069 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:25:00.069 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T04:25:00.070 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T04:25:00.070 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T04:25:00.070 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:25:00.070 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T04:25:00.070 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T04:25:00.070 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T04:25:00.070 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:25:00.089 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T04:25:00.090 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T04:25:00.093 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T04:25:00.093 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T04:25:00.100 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T04:25:00.100 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T04:25:00.103 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T04:25:00.103 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T04:25:00.138 INFO:tasks.ceph.mds.c:Started 2024-09-17T04:25:00.138 DEBUG:teuthology.parallel:result is None 2024-09-17T04:25:00.140 INFO:tasks.ceph.mds.a:Started 2024-09-17T04:25:00.140 DEBUG:teuthology.parallel:result is None 2024-09-17T04:25:00.147 INFO:tasks.ceph.mds.b:Started 2024-09-17T04:25:00.147 DEBUG:teuthology.parallel:result is None 2024-09-17T04:25:00.150 INFO:tasks.ceph.mds.d:Started 2024-09-17T04:25:00.150 DEBUG:teuthology.parallel:result is None 2024-09-17T04:25:00.150 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T04:25:00.334 INFO:tasks.ceph.mds.c.smithi100.stdout:starting mds.c at 2024-09-17T04:25:00.335 INFO:tasks.ceph.mds.a.smithi100.stdout:starting mds.a at 2024-09-17T04:25:00.345 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-09-17T04:25:00.346 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-09-17T04:25:00.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.385+0000 7fbf03e00640 1 -- 172.21.15.100:0/2698167053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf040af170 msgr2=0x7fbf040b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:00.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.385+0000 7fbf03e00640 1 --2- 172.21.15.100:0/2698167053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf040af170 0x7fbf040b2f60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbef0005fd0 tx=0x7fbef002a540 comp rx=0 tx=0).stop 2024-09-17T04:25:00.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.385+0000 7fbf03e00640 1 -- 172.21.15.100:0/2698167053 shutdown_connections 2024-09-17T04:25:00.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.385+0000 7fbf03e00640 1 --2- 172.21.15.100:0/2698167053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf0415a0f0 0x7fbf0415d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:00.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.385+0000 7fbf03e00640 1 --2- 172.21.15.100:0/2698167053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf040af170 0x7fbf040b2f60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:00.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.385+0000 7fbf03e00640 1 -- 172.21.15.100:0/2698167053 >> 172.21.15.100:0/2698167053 conn(0x7fbf040b1e40 msgr2=0x7fbf040b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:00.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.385+0000 7fbf03e00640 1 -- 172.21.15.100:0/2698167053 shutdown_connections 2024-09-17T04:25:00.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.385+0000 7fbf03e00640 1 -- 172.21.15.100:0/2698167053 wait complete. 2024-09-17T04:25:00.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.387+0000 7fbf03e00640 1 Processor -- start 2024-09-17T04:25:00.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.388+0000 7fbf03e00640 1 -- start start 2024-09-17T04:25:00.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.389+0000 7fbf03e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf040ad9c0 0x7fbf0414bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:00.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.389+0000 7fbf03e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf040af170 0x7fbf0409db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:00.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.389+0000 7fbf03e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf0415a0f0 0x7fbf041f6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:00.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.389+0000 7fbf03e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0406f290 con 0x7fbf040af170 2024-09-17T04:25:00.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.389+0000 7fbf03e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0415a5e0 con 0x7fbf0415a0f0 2024-09-17T04:25:00.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.389+0000 7fbf03e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbf0406f410 con 0x7fbf040ad9c0 2024-09-17T04:25:00.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.389+0000 7fbf02000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf040af170 0x7fbf0409db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:00.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.389+0000 7fbf02000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf040af170 0x7fbf0409db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51140/0 (socket says 172.21.15.100:51140) 2024-09-17T04:25:00.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.389+0000 7fbf02000640 1 -- 172.21.15.100:0/3665197428 learned_addr learned my addr 172.21.15.100:0/3665197428 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:00.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.390+0000 7fbf02a00640 1 --2- 172.21.15.100:0/3665197428 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf040ad9c0 0x7fbf0414bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:00.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.390+0000 7fbf03400640 1 --2- 172.21.15.100:0/3665197428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf0415a0f0 0x7fbf041f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:00.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.390+0000 7fbf02000640 1 -- 172.21.15.100:0/3665197428 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf040ad9c0 msgr2=0x7fbf0414bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:00.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.390+0000 7fbf02000640 1 --2- 172.21.15.100:0/3665197428 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf040ad9c0 0x7fbf0414bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:00.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.390+0000 7fbf02000640 1 -- 172.21.15.100:0/3665197428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf0415a0f0 msgr2=0x7fbf041f6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:00.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.390+0000 7fbf02000640 1 --2- 172.21.15.100:0/3665197428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf0415a0f0 0x7fbf041f6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:00.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.390+0000 7fbf02000640 1 -- 172.21.15.100:0/3665197428 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0414c410 con 0x7fbf040af170 2024-09-17T04:25:00.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.390+0000 7fbf02000640 1 --2- 172.21.15.100:0/3665197428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf040af170 0x7fbf0409db70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbef0000c00 tx=0x7fbef00038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:00.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.390+0000 7fbefaa00640 1 -- 172.21.15.100:0/3665197428 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef003e030 con 0x7fbf040af170 2024-09-17T04:25:00.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.390+0000 7fbefaa00640 1 -- 172.21.15.100:0/3665197428 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbef0003db0 con 0x7fbf040af170 2024-09-17T04:25:00.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.390+0000 7fbefaa00640 1 -- 172.21.15.100:0/3665197428 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef0035040 con 0x7fbf040af170 2024-09-17T04:25:00.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.390+0000 7fbf03e00640 1 -- 172.21.15.100:0/3665197428 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf04159310 con 0x7fbf040af170 2024-09-17T04:25:00.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.391+0000 7fbf03e00640 1 -- 172.21.15.100:0/3665197428 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf041569a0 con 0x7fbf040af170 2024-09-17T04:25:00.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.392+0000 7fbf03e00640 1 -- 172.21.15.100:0/3665197428 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf0406f930 con 0x7fbf040af170 2024-09-17T04:25:00.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.393+0000 7fbefaa00640 1 -- 172.21.15.100:0/3665197428 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbef0041030 con 0x7fbf040af170 2024-09-17T04:25:00.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.394+0000 7fbefaa00640 1 --2- 172.21.15.100:0/3665197428 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbee4078bc0 0x7fbee407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:00.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.394+0000 7fbefaa00640 1 -- 172.21.15.100:0/3665197428 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4943+0+0 (secure 0 0 0) 0x7fbef00bb160 con 0x7fbf040af170 2024-09-17T04:25:00.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.394+0000 7fbf02a00640 1 --2- 172.21.15.100:0/3665197428 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbee4078bc0 0x7fbee407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:00.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.395+0000 7fbf02a00640 1 --2- 172.21.15.100:0/3665197428 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbee4078bc0 0x7fbee407b0d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbef4007fd0 tx=0x7fbef4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:00.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.398+0000 7fbefaa00640 1 -- 172.21.15.100:0/3665197428 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fbef0083800 con 0x7fbf040af170 2024-09-17T04:25:00.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:00.592+0000 7fbf03e00640 1 -- 172.21.15.100:0/3665197428 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fbf040b3f80 con 0x7fbf040af170 2024-09-17T04:25:01.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.054+0000 7fbefaa00640 1 -- 172.21.15.100:0/3665197428 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) ==== 97+0+0 (secure 0 0 0) 0x7fbef0083100 con 0x7fbf040af170 2024-09-17T04:25:01.054 INFO:teuthology.orchestra.run.smithi100.stderr: removed all blocklist entries 2024-09-17T04:25:01.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.057+0000 7fbf03e00640 1 -- 172.21.15.100:0/3665197428 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbee4078bc0 msgr2=0x7fbee407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:01.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.057+0000 7fbf03e00640 1 --2- 172.21.15.100:0/3665197428 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbee4078bc0 0x7fbee407b0d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbef4007fd0 tx=0x7fbef4009040 comp rx=0 tx=0).stop 2024-09-17T04:25:01.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.057+0000 7fbf03e00640 1 -- 172.21.15.100:0/3665197428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf040af170 msgr2=0x7fbf0409db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:01.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.057+0000 7fbf03e00640 1 --2- 172.21.15.100:0/3665197428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf040af170 0x7fbf0409db70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbef0000c00 tx=0x7fbef00038a0 comp rx=0 tx=0).stop 2024-09-17T04:25:01.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.057+0000 7fbf03e00640 1 -- 172.21.15.100:0/3665197428 shutdown_connections 2024-09-17T04:25:01.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.057+0000 7fbf03e00640 1 --2- 172.21.15.100:0/3665197428 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbee4078bc0 0x7fbee407b0d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:01.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.057+0000 7fbf03e00640 1 --2- 172.21.15.100:0/3665197428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf0415a0f0 0x7fbf041f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:01.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.057+0000 7fbf03e00640 1 --2- 172.21.15.100:0/3665197428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf040af170 0x7fbf0409db70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:01.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.057+0000 7fbf03e00640 1 --2- 172.21.15.100:0/3665197428 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf040ad9c0 0x7fbf0414bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:01.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.057+0000 7fbf03e00640 1 -- 172.21.15.100:0/3665197428 >> 172.21.15.100:0/3665197428 conn(0x7fbf040b1e40 msgr2=0x7fbf040cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:01.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.057+0000 7fbf03e00640 1 -- 172.21.15.100:0/3665197428 shutdown_connections 2024-09-17T04:25:01.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.057+0000 7fbf03e00640 1 -- 172.21.15.100:0/3665197428 wait complete. 2024-09-17T04:25:01.083 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T04:25:01.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.303+0000 7fa8b7e00640 1 -- 172.21.15.100:0/775822080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8b00d15d0 msgr2=0x7fa8b00cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:01.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.303+0000 7fa8b7e00640 1 --2- 172.21.15.100:0/775822080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8b00d15d0 0x7fa8b00cf470 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8005fd0 tx=0x7fa8a802a820 comp rx=0 tx=0).stop 2024-09-17T04:25:01.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.303+0000 7fa8b7e00640 1 -- 172.21.15.100:0/775822080 shutdown_connections 2024-09-17T04:25:01.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.303+0000 7fa8b7e00640 1 --2- 172.21.15.100:0/775822080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8b00d15d0 0x7fa8b00cf470 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:01.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.303+0000 7fa8b7e00640 1 --2- 172.21.15.100:0/775822080 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8b00b3720 0x7fa8b00d1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:01.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.303+0000 7fa8b7e00640 1 -- 172.21.15.100:0/775822080 >> 172.21.15.100:0/775822080 conn(0x7fa8b00d4680 msgr2=0x7fa8b00d4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:01.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.303+0000 7fa8b7e00640 1 -- 172.21.15.100:0/775822080 shutdown_connections 2024-09-17T04:25:01.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.303+0000 7fa8b7e00640 1 -- 172.21.15.100:0/775822080 wait complete. 2024-09-17T04:25:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.305+0000 7fa8b7e00640 1 Processor -- start 2024-09-17T04:25:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.306+0000 7fa8b7e00640 1 -- start start 2024-09-17T04:25:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.307+0000 7fa8b7e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8b00b3720 0x7fa8b00b5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.307+0000 7fa8b7e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8b00b9780 0x7fa8b014ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.307+0000 7fa8b7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b00d15d0 0x7fa8b014b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:01.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.307+0000 7fa8b7e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b00cfd40 con 0x7fa8b00d15d0 2024-09-17T04:25:01.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.307+0000 7fa8b7e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b00cfbc0 con 0x7fa8b00b3720 2024-09-17T04:25:01.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.307+0000 7fa8b7e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa8b00cfec0 con 0x7fa8b00b9780 2024-09-17T04:25:01.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.307+0000 7fa8b5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b00d15d0 0x7fa8b014b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:01.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.307+0000 7fa8b5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b00d15d0 0x7fa8b014b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51144/0 (socket says 172.21.15.100:51144) 2024-09-17T04:25:01.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.307+0000 7fa8b5e00640 1 -- 172.21.15.100:0/2460901026 learned_addr learned my addr 172.21.15.100:0/2460901026 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:01.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.308+0000 7fa8b4a00640 1 --2- 172.21.15.100:0/2460901026 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8b00b9780 0x7fa8b014ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:01.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.308+0000 7fa8b5400640 1 --2- 172.21.15.100:0/2460901026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8b00b3720 0x7fa8b00b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:01.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.308+0000 7fa8b5e00640 1 -- 172.21.15.100:0/2460901026 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8b00b9780 msgr2=0x7fa8b014ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:01.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.308+0000 7fa8b5e00640 1 --2- 172.21.15.100:0/2460901026 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8b00b9780 0x7fa8b014ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:01.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.308+0000 7fa8b5e00640 1 -- 172.21.15.100:0/2460901026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8b00b3720 msgr2=0x7fa8b00b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:01.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.308+0000 7fa8b5e00640 1 --2- 172.21.15.100:0/2460901026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8b00b3720 0x7fa8b00b5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:01.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.308+0000 7fa8b5e00640 1 -- 172.21.15.100:0/2460901026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b0161de0 con 0x7fa8b00d15d0 2024-09-17T04:25:01.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.308+0000 7fa8b5400640 1 --2- 172.21.15.100:0/2460901026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8b00b3720 0x7fa8b00b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:01.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.308+0000 7fa8b5e00640 1 --2- 172.21.15.100:0/2460901026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b00d15d0 0x7fa8b014b190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8009cd0 tx=0x7fa8a8032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:01.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.308+0000 7fa8ad600640 1 -- 172.21.15.100:0/2460901026 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a8037040 con 0x7fa8b00d15d0 2024-09-17T04:25:01.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.309+0000 7fa8ad600640 1 -- 172.21.15.100:0/2460901026 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8a8042070 con 0x7fa8b00d15d0 2024-09-17T04:25:01.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.309+0000 7fa8b7e00640 1 -- 172.21.15.100:0/2460901026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8b00b5bb0 con 0x7fa8b00d15d0 2024-09-17T04:25:01.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.309+0000 7fa8ad600640 1 -- 172.21.15.100:0/2460901026 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a8003e60 con 0x7fa8b00d15d0 2024-09-17T04:25:01.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.309+0000 7fa8b7e00640 1 -- 172.21.15.100:0/2460901026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8b00b6170 con 0x7fa8b00d15d0 2024-09-17T04:25:01.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.310+0000 7fa8b7e00640 1 -- 172.21.15.100:0/2460901026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8b008d100 con 0x7fa8b00d15d0 2024-09-17T04:25:01.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.311+0000 7fa8ad600640 1 -- 172.21.15.100:0/2460901026 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa8a8004280 con 0x7fa8b00d15d0 2024-09-17T04:25:01.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.312+0000 7fa8ad600640 1 --2- 172.21.15.100:0/2460901026 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa890078b90 0x7fa89007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:01.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.312+0000 7fa8ad600640 1 -- 172.21.15.100:0/2460901026 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7fa8a80bb740 con 0x7fa8b00d15d0 2024-09-17T04:25:01.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.312+0000 7fa8b5400640 1 --2- 172.21.15.100:0/2460901026 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa890078b90 0x7fa89007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:01.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.313+0000 7fa8b5400640 1 --2- 172.21.15.100:0/2460901026 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa890078b90 0x7fa89007b0a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4001710 tx=0x7fa8a4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:01.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.316+0000 7fa8ad600640 1 -- 172.21.15.100:0/2460901026 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa8b008d100 con 0x7fa8b00d15d0 2024-09-17T04:25:01.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.562+0000 7fa8b7e00640 1 -- 172.21.15.100:0/2460901026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fa8b009e530 con 0x7fa8b00d15d0 2024-09-17T04:25:01.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.563+0000 7fa8ad600640 1 -- 172.21.15.100:0/2460901026 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7fa8a8083ed0 con 0x7fa8b00d15d0 2024-09-17T04:25:01.563 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:01.563 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:25:01.563 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_dump": [ 2024-09-17T04:25:01.563 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:01.563 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.a", 2024-09-17T04:25:01.563 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmV8asJRAAkSgIWqGc1OeSRbs9PEOT8w==", 2024-09-17T04:25:01.564 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:01.564 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:25:01.564 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:25:01.564 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:25:01.564 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:01.564 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:01.564 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:01.564 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:01.564 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.b", 2024-09-17T04:25:01.564 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmpM7ZBRAA1Og785tJwa6NA9zBdmNNQA==", 2024-09-17T04:25:01.564 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:01.564 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:25:01.564 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:25:01.565 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:25:01.565 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:01.565 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:01.565 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:01.565 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:01.565 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.c", 2024-09-17T04:25:01.565 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmXaOGMxAAT+96g/5fAzPW8Cc2HtQUyA==", 2024-09-17T04:25:01.565 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:01.565 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:25:01.565 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:25:01.566 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:25:01.566 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:01.566 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:01.566 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:01.566 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:01.566 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.d", 2024-09-17T04:25:01.566 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmRqC9ExAA23kZ0Ks1lGN2k4GXL8ASrg==", 2024-09-17T04:25:01.566 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:01.566 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:25:01.566 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:25:01.566 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:25:01.566 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:01.566 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:01.566 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:01.567 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:01.567 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.0", 2024-09-17T04:25:01.567 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAwBOlmofFmAxAARerYkU9B2a9MM9vHa3o4LQ==", 2024-09-17T04:25:01.567 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:01.567 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:25:01.567 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:25:01.567 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:01.567 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:01.567 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:01.567 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:01.567 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.1", 2024-09-17T04:25:01.567 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAzBOlmbz6VExAAoVqSqXTc1wArKhdA6+NxnA==", 2024-09-17T04:25:01.567 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:01.567 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:25:01.568 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:25:01.568 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:01.568 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:01.568 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:01.568 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:01.568 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.2", 2024-09-17T04:25:01.568 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA2BOlmfeSAJBAAchY6XhyDRusc+zqwQ6v6wQ==", 2024-09-17T04:25:01.568 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:01.568 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:25:01.568 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:25:01.568 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:01.568 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:01.568 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:01.569 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:01.569 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.3", 2024-09-17T04:25:01.569 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA6BOlm/TQiBBAAHhbs2+RCjAenGR1frhbTmQ==", 2024-09-17T04:25:01.569 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:01.569 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:25:01.569 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:25:01.569 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:01.569 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:01.569 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:01.570 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:01.570 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.4", 2024-09-17T04:25:01.570 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBABOlm7OKvBhAALkaAKEUYN5ChwGejoSF8yA==", 2024-09-17T04:25:01.570 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:01.570 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:25:01.570 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:25:01.570 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:01.570 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:01.570 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:01.570 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:01.570 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.5", 2024-09-17T04:25:01.570 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBDBOlmQpi1FhAAbA47dg6ckK1PFoCZL0SBEQ==", 2024-09-17T04:25:01.570 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:01.570 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:25:01.570 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:25:01.571 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:01.571 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:01.571 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:01.571 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:01.571 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.6", 2024-09-17T04:25:01.571 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBGBOlmx/w0JhAAWcPvZkMYtXD/EMKrg/W/4Q==", 2024-09-17T04:25:01.571 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:01.571 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:25:01.571 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:25:01.571 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:01.571 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:01.571 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:01.571 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:01.571 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.7", 2024-09-17T04:25:01.572 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBJBOlmWHjWNBAAgdul2UKY9g5p7EVi0ZdrFw==", 2024-09-17T04:25:01.572 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:01.572 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:25:01.572 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:25:01.572 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:01.572 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:01.572 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:01.572 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:01.572 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.0", 2024-09-17T04:25:01.572 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmfYg9IBAABP1uRKGOugrA47VxwRLkjA==", 2024-09-17T04:25:01.572 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:01.572 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:25:01.572 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:25:01.573 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:25:01.573 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:25:01.573 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:01.573 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:01.573 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:01.573 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.1", 2024-09-17T04:25:01.573 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmByhhNRAAPGaxFCDd8ywFVwwhprrPbQ==", 2024-09-17T04:25:01.573 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:01.573 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:25:01.573 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:25:01.573 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:25:01.573 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:25:01.573 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:01.573 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:01.574 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:01.574 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.2", 2024-09-17T04:25:01.574 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmgcm0KhAAGF06VlHjBM/AGxafN6/3aA==", 2024-09-17T04:25:01.574 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:01.574 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:25:01.574 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:25:01.574 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:25:01.574 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:25:01.574 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:01.574 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:01.574 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:01.574 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.3", 2024-09-17T04:25:01.574 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtBOlm9VQ6BBAAXcahu35/uDm8Z+f+3SF0Gg==", 2024-09-17T04:25:01.574 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:01.574 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:25:01.575 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:25:01.575 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:25:01.575 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:25:01.575 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:01.575 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:01.575 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:01.575 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.admin", 2024-09-17T04:25:01.575 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAqBOlmZ+bIJBAAG3oaMNlYWNakywGDvAGbGw==", 2024-09-17T04:25:01.575 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:01.575 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:25:01.575 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:25:01.575 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:25:01.575 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:01.575 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:01.576 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:01.576 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:01.576 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.bootstrap-mds", 2024-09-17T04:25:01.576 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBSBOlm9yPjMRAARD9CyBWb+a9MF6V4VD1v+g==", 2024-09-17T04:25:01.576 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:01.576 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile bootstrap-mds" 2024-09-17T04:25:01.576 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:01.577 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:01.577 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:01.577 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.bootstrap-mgr", 2024-09-17T04:25:01.577 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBSBOlm4S3jMRAAMEdizo+3NFYA5pqIN/hGbQ==", 2024-09-17T04:25:01.577 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:01.577 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-17T04:25:01.577 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:01.577 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:01.577 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:01.577 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.bootstrap-osd", 2024-09-17T04:25:01.577 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBSBOlmZTbjMRAA1ho8py27UnbLP98ilaiGUQ==", 2024-09-17T04:25:01.577 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:01.578 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile bootstrap-osd" 2024-09-17T04:25:01.578 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:01.578 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:01.578 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:01.578 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.bootstrap-rbd", 2024-09-17T04:25:01.578 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBSBOlmcULjMRAAOGhifKHwrTNN4rhUbnIY0w==", 2024-09-17T04:25:01.578 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:01.578 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-17T04:25:01.578 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:01.578 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:01.578 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:01.578 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-17T04:25:01.578 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBSBOlmUEvjMRAA46eiJrCv86PeLIHXA/+6QA==", 2024-09-17T04:25:01.578 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:01.579 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-17T04:25:01.579 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:01.579 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:01.579 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:01.579 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.bootstrap-rgw", 2024-09-17T04:25:01.579 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBSBOlm2lPjMRAAJcgq2lba0A8XmQkmoTQ4OA==", 2024-09-17T04:25:01.579 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:01.579 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-17T04:25:01.579 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:01.579 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:01.579 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:01.579 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mgr.x", 2024-09-17T04:25:01.579 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmfaL7GhAArqLH66RkbBsz+TNOKtPHOA==", 2024-09-17T04:25:01.580 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:01.580 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:25:01.580 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile mgr", 2024-09-17T04:25:01.580 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:01.580 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:01.580 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:01.580 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:01.580 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mgr.y", 2024-09-17T04:25:01.580 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmM4D1DhAA5uUWgRVOMujWqbYzOjFp9A==", 2024-09-17T04:25:01.580 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:01.580 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:25:01.580 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile mgr", 2024-09-17T04:25:01.580 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:01.580 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:01.581 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:01.581 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-17T04:25:01.581 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:25:01.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.565+0000 7fa8b7e00640 1 -- 172.21.15.100:0/2460901026 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa890078b90 msgr2=0x7fa89007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:01.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.565+0000 7fa8b7e00640 1 --2- 172.21.15.100:0/2460901026 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa890078b90 0x7fa89007b0a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4001710 tx=0x7fa8a4009040 comp rx=0 tx=0).stop 2024-09-17T04:25:01.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.565+0000 7fa8b7e00640 1 -- 172.21.15.100:0/2460901026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b00d15d0 msgr2=0x7fa8b014b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:01.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.565+0000 7fa8b7e00640 1 --2- 172.21.15.100:0/2460901026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b00d15d0 0x7fa8b014b190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8009cd0 tx=0x7fa8a8032c60 comp rx=0 tx=0).stop 2024-09-17T04:25:01.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.565+0000 7fa8b7e00640 1 -- 172.21.15.100:0/2460901026 shutdown_connections 2024-09-17T04:25:01.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.565+0000 7fa8b7e00640 1 --2- 172.21.15.100:0/2460901026 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa890078b90 0x7fa89007b0a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:01.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.565+0000 7fa8b7e00640 1 --2- 172.21.15.100:0/2460901026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b00d15d0 0x7fa8b014b190 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:01.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.565+0000 7fa8b7e00640 1 --2- 172.21.15.100:0/2460901026 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8b00b9780 0x7fa8b014ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:01.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.565+0000 7fa8b7e00640 1 --2- 172.21.15.100:0/2460901026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8b00b3720 0x7fa8b00b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:01.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.565+0000 7fa8b7e00640 1 -- 172.21.15.100:0/2460901026 >> 172.21.15.100:0/2460901026 conn(0x7fa8b00d4680 msgr2=0x7fa8b0163140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:01.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.565+0000 7fa8b7e00640 1 -- 172.21.15.100:0/2460901026 shutdown_connections 2024-09-17T04:25:01.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.565+0000 7fa8b7e00640 1 -- 172.21.15.100:0/2460901026 wait complete. 2024-09-17T04:25:01.589 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-17T04:25:01.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.811+0000 7f8f64800640 1 -- 172.21.15.100:0/247361179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f5c15d540 msgr2=0x7f8f5c156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:01.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.811+0000 7f8f64800640 1 --2- 172.21.15.100:0/247361179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f5c15d540 0x7f8f5c156e40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009ca0 tx=0x7f8f5802aae0 comp rx=0 tx=0).stop 2024-09-17T04:25:01.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.811+0000 7f8f64800640 1 -- 172.21.15.100:0/247361179 shutdown_connections 2024-09-17T04:25:01.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.811+0000 7f8f64800640 1 --2- 172.21.15.100:0/247361179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f5c15d540 0x7f8f5c156e40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:01.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.811+0000 7f8f64800640 1 -- 172.21.15.100:0/247361179 >> 172.21.15.100:0/247361179 conn(0x7f8f5c06d3d0 msgr2=0x7f8f5c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:01.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.811+0000 7f8f64800640 1 -- 172.21.15.100:0/247361179 shutdown_connections 2024-09-17T04:25:01.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.811+0000 7f8f64800640 1 -- 172.21.15.100:0/247361179 wait complete. 2024-09-17T04:25:01.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.813+0000 7f8f64800640 1 Processor -- start 2024-09-17T04:25:01.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.813+0000 7f8f64800640 1 -- start start 2024-09-17T04:25:01.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.814+0000 7f8f64800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f5c0cfc80 0x7f8f5c1f36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:01.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.814+0000 7f8f64800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f5c15d540 0x7f8f5c0c5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:01.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.814+0000 7f8f64800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f5c074e90 0x7f8f5c0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:01.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.814+0000 7f8f64800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5c0cd130 con 0x7f8f5c074e90 2024-09-17T04:25:01.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.814+0000 7f8f64800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5c0ccfb0 con 0x7f8f5c15d540 2024-09-17T04:25:01.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.814+0000 7f8f64800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8f5c0cd2b0 con 0x7f8f5c0cfc80 2024-09-17T04:25:01.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.815+0000 7f8f62800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f5c074e90 0x7f8f5c0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:01.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.815+0000 7f8f62800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f5c074e90 0x7f8f5c0c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51152/0 (socket says 172.21.15.100:51152) 2024-09-17T04:25:01.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.815+0000 7f8f62800640 1 -- 172.21.15.100:0/3915305834 learned_addr learned my addr 172.21.15.100:0/3915305834 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:01.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.815+0000 7f8f61e00640 1 --2- 172.21.15.100:0/3915305834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f5c0cfc80 0x7f8f5c1f36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:01.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.815+0000 7f8f61400640 1 --2- 172.21.15.100:0/3915305834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f5c15d540 0x7f8f5c0c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:01.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.815+0000 7f8f62800640 1 -- 172.21.15.100:0/3915305834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f5c0cfc80 msgr2=0x7f8f5c1f36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:01.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.815+0000 7f8f62800640 1 --2- 172.21.15.100:0/3915305834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f5c0cfc80 0x7f8f5c1f36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:01.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.815+0000 7f8f62800640 1 -- 172.21.15.100:0/3915305834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f5c15d540 msgr2=0x7f8f5c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:01.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.815+0000 7f8f62800640 1 --2- 172.21.15.100:0/3915305834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f5c15d540 0x7f8f5c0c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:01.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.815+0000 7f8f62800640 1 -- 172.21.15.100:0/3915305834 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f5c073e10 con 0x7f8f5c074e90 2024-09-17T04:25:01.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.815+0000 7f8f62800640 1 --2- 172.21.15.100:0/3915305834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f5c074e90 0x7f8f5c0c7cb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009bf0 tx=0x7f8f58003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:01.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.816+0000 7f8f56000640 1 -- 172.21.15.100:0/3915305834 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f58044070 con 0x7f8f5c074e90 2024-09-17T04:25:01.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.816+0000 7f8f56000640 1 -- 172.21.15.100:0/3915305834 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f58004250 con 0x7f8f5c074e90 2024-09-17T04:25:01.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.816+0000 7f8f56000640 1 -- 172.21.15.100:0/3915305834 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f58004680 con 0x7f8f5c074e90 2024-09-17T04:25:01.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.816+0000 7f8f64800640 1 -- 172.21.15.100:0/3915305834 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f5c0740f0 con 0x7f8f5c074e90 2024-09-17T04:25:01.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.816+0000 7f8f64800640 1 -- 172.21.15.100:0/3915305834 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f5c0746b0 con 0x7f8f5c074e90 2024-09-17T04:25:01.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.817+0000 7f8f64800640 1 -- 172.21.15.100:0/3915305834 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f5c0cd7a0 con 0x7f8f5c074e90 2024-09-17T04:25:01.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.818+0000 7f8f56000640 1 -- 172.21.15.100:0/3915305834 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8f58047070 con 0x7f8f5c074e90 2024-09-17T04:25:01.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.819+0000 7f8f56000640 1 --2- 172.21.15.100:0/3915305834 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8f40078b90 0x7f8f4007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:01.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.819+0000 7f8f56000640 1 -- 172.21.15.100:0/3915305834 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f8f580bb9f0 con 0x7f8f5c074e90 2024-09-17T04:25:01.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.819+0000 7f8f61e00640 1 --2- 172.21.15.100:0/3915305834 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8f40078b90 0x7f8f4007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:01.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.820+0000 7f8f61e00640 1 --2- 172.21.15.100:0/3915305834 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8f40078b90 0x7f8f4007b0a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c09e510 tx=0x7f8f50009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:01.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:01.823+0000 7f8f56000640 1 -- 172.21.15.100:0/3915305834 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8f58032030 con 0x7f8f5c074e90 2024-09-17T04:25:02.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.081+0000 7f8f64800640 1 -- 172.21.15.100:0/3915305834 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f8f5c073ba0 con 0x7f8f5c074e90 2024-09-17T04:25:02.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.084+0000 7f8f56000640 1 -- 172.21.15.100:0/3915305834 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f8f58004820 con 0x7f8f5c074e90 2024-09-17T04:25:02.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.086+0000 7f8f64800640 1 -- 172.21.15.100:0/3915305834 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8f40078b90 msgr2=0x7f8f4007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:02.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.086+0000 7f8f64800640 1 --2- 172.21.15.100:0/3915305834 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8f40078b90 0x7f8f4007b0a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c09e510 tx=0x7f8f50009040 comp rx=0 tx=0).stop 2024-09-17T04:25:02.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.086+0000 7f8f64800640 1 -- 172.21.15.100:0/3915305834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f5c074e90 msgr2=0x7f8f5c0c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:02.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.086+0000 7f8f64800640 1 --2- 172.21.15.100:0/3915305834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f5c074e90 0x7f8f5c0c7cb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009bf0 tx=0x7f8f58003f10 comp rx=0 tx=0).stop 2024-09-17T04:25:02.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.086+0000 7f8f64800640 1 -- 172.21.15.100:0/3915305834 shutdown_connections 2024-09-17T04:25:02.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.086+0000 7f8f64800640 1 --2- 172.21.15.100:0/3915305834 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8f40078b90 0x7f8f4007b0a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:02.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.086+0000 7f8f64800640 1 --2- 172.21.15.100:0/3915305834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f5c074e90 0x7f8f5c0c7cb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:02.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.086+0000 7f8f64800640 1 --2- 172.21.15.100:0/3915305834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f5c15d540 0x7f8f5c0c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:02.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.086+0000 7f8f64800640 1 --2- 172.21.15.100:0/3915305834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f5c0cfc80 0x7f8f5c1f36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:02.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.086+0000 7f8f64800640 1 -- 172.21.15.100:0/3915305834 >> 172.21.15.100:0/3915305834 conn(0x7f8f5c06d3d0 msgr2=0x7f8f5c06f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:02.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.086+0000 7f8f64800640 1 -- 172.21.15.100:0/3915305834 shutdown_connections 2024-09-17T04:25:02.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.086+0000 7f8f64800640 1 -- 172.21.15.100:0/3915305834 wait complete. 2024-09-17T04:25:02.108 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-17T04:25:02.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.326+0000 7fa95a600640 1 -- 172.21.15.100:0/2904644953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9540cfc80 msgr2=0x7fa954156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:02.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.326+0000 7fa95a600640 1 --2- 172.21.15.100:0/2904644953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9540cfc80 0x7fa954156e40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa944009900 tx=0x7fa94402a580 comp rx=0 tx=0).stop 2024-09-17T04:25:02.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.326+0000 7fa95a600640 1 -- 172.21.15.100:0/2904644953 shutdown_connections 2024-09-17T04:25:02.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.326+0000 7fa95a600640 1 --2- 172.21.15.100:0/2904644953 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa95415ce30 0x7fa95415d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:02.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.326+0000 7fa95a600640 1 --2- 172.21.15.100:0/2904644953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9540cfc80 0x7fa954156e40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:02.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.326+0000 7fa95a600640 1 -- 172.21.15.100:0/2904644953 >> 172.21.15.100:0/2904644953 conn(0x7fa95406d3d0 msgr2=0x7fa9540b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:02.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.327+0000 7fa95a600640 1 -- 172.21.15.100:0/2904644953 shutdown_connections 2024-09-17T04:25:02.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.327+0000 7fa95a600640 1 -- 172.21.15.100:0/2904644953 wait complete. 2024-09-17T04:25:02.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.329+0000 7fa95a600640 1 Processor -- start 2024-09-17T04:25:02.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.329+0000 7fa95a600640 1 -- start start 2024-09-17T04:25:02.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.330+0000 7fa95a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9540705b0 0x7fa9541f97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:02.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.330+0000 7fa95a600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9540cfc80 0x7fa9540c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:02.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.330+0000 7fa95a600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa95415ce30 0x7fa9540b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:02.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.330+0000 7fa95a600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa9540cd050 con 0x7fa9540705b0 2024-09-17T04:25:02.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.330+0000 7fa95a600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa9540cced0 con 0x7fa9540cfc80 2024-09-17T04:25:02.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.330+0000 7fa95a600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa9540cd1d0 con 0x7fa95415ce30 2024-09-17T04:25:02.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.330+0000 7fa953400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9540705b0 0x7fa9541f97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:02.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.330+0000 7fa953400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9540705b0 0x7fa9541f97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51170/0 (socket says 172.21.15.100:51170) 2024-09-17T04:25:02.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.330+0000 7fa953400640 1 -- 172.21.15.100:0/514670911 learned_addr learned my addr 172.21.15.100:0/514670911 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:02.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.331+0000 7fa953400640 1 -- 172.21.15.100:0/514670911 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa95415ce30 msgr2=0x7fa9540b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:02.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.331+0000 7fa953400640 1 --2- 172.21.15.100:0/514670911 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa95415ce30 0x7fa9540b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:02.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.331+0000 7fa953400640 1 -- 172.21.15.100:0/514670911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9540cfc80 msgr2=0x7fa9540c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:02.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.331+0000 7fa953400640 1 --2- 172.21.15.100:0/514670911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9540cfc80 0x7fa9540c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:02.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.331+0000 7fa953400640 1 -- 172.21.15.100:0/514670911 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9541f5b60 con 0x7fa9540705b0 2024-09-17T04:25:02.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.331+0000 7fa953400640 1 --2- 172.21.15.100:0/514670911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9540705b0 0x7fa9541f97c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa944000c00 tx=0x7fa944003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:02.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.331+0000 7fa94b400640 1 -- 172.21.15.100:0/514670911 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94403e030 con 0x7fa9540705b0 2024-09-17T04:25:02.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.332+0000 7fa94b400640 1 -- 172.21.15.100:0/514670911 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa944003e00 con 0x7fa9540705b0 2024-09-17T04:25:02.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.332+0000 7fa95a600640 1 -- 172.21.15.100:0/514670911 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9541f5e40 con 0x7fa9540705b0 2024-09-17T04:25:02.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.332+0000 7fa94b400640 1 -- 172.21.15.100:0/514670911 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944035040 con 0x7fa9540705b0 2024-09-17T04:25:02.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.332+0000 7fa95a600640 1 -- 172.21.15.100:0/514670911 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9541f6400 con 0x7fa9540705b0 2024-09-17T04:25:02.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.333+0000 7fa95a600640 1 -- 172.21.15.100:0/514670911 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9541f7f60 con 0x7fa9540705b0 2024-09-17T04:25:02.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.334+0000 7fa94b400640 1 -- 172.21.15.100:0/514670911 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa944041030 con 0x7fa9540705b0 2024-09-17T04:25:02.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.335+0000 7fa94b400640 1 --2- 172.21.15.100:0/514670911 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa930078b90 0x7fa93007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:02.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.335+0000 7fa94b400640 1 -- 172.21.15.100:0/514670911 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7fa9440bb5c0 con 0x7fa9540705b0 2024-09-17T04:25:02.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.335+0000 7fa94be00640 1 --2- 172.21.15.100:0/514670911 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa930078b90 0x7fa93007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:02.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.336+0000 7fa94be00640 1 --2- 172.21.15.100:0/514670911 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa930078b90 0x7fa93007b0a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa93c000f60 tx=0x7fa93c0079b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:02.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.339+0000 7fa94b400640 1 -- 172.21.15.100:0/514670911 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa944083cb0 con 0x7fa9540705b0 2024-09-17T04:25:02.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.592+0000 7fa95a600640 1 -- 172.21.15.100:0/514670911 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fa9540cd760 con 0x7fa9540705b0 2024-09-17T04:25:02.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.595+0000 7fa94b400640 1 -- 172.21.15.100:0/514670911 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7fa9440835b0 con 0x7fa9540705b0 2024-09-17T04:25:02.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.598+0000 7fa95a600640 1 -- 172.21.15.100:0/514670911 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa930078b90 msgr2=0x7fa93007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.598+0000 7fa95a600640 1 --2- 172.21.15.100:0/514670911 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa930078b90 0x7fa93007b0a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa93c000f60 tx=0x7fa93c0079b0 comp rx=0 tx=0).stop 2024-09-17T04:25:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.598+0000 7fa95a600640 1 -- 172.21.15.100:0/514670911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9540705b0 msgr2=0x7fa9541f97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.598+0000 7fa95a600640 1 --2- 172.21.15.100:0/514670911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9540705b0 0x7fa9541f97c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa944000c00 tx=0x7fa944003910 comp rx=0 tx=0).stop 2024-09-17T04:25:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.598+0000 7fa95a600640 1 -- 172.21.15.100:0/514670911 shutdown_connections 2024-09-17T04:25:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.598+0000 7fa95a600640 1 --2- 172.21.15.100:0/514670911 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa930078b90 0x7fa93007b0a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.598+0000 7fa95a600640 1 --2- 172.21.15.100:0/514670911 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa95415ce30 0x7fa9540b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.598+0000 7fa95a600640 1 --2- 172.21.15.100:0/514670911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9540cfc80 0x7fa9540c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.598+0000 7fa95a600640 1 --2- 172.21.15.100:0/514670911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9540705b0 0x7fa9541f97c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.598+0000 7fa95a600640 1 -- 172.21.15.100:0/514670911 >> 172.21.15.100:0/514670911 conn(0x7fa95406d3d0 msgr2=0x7fa9540b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.598+0000 7fa95a600640 1 -- 172.21.15.100:0/514670911 shutdown_connections 2024-09-17T04:25:02.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.598+0000 7fa95a600640 1 -- 172.21.15.100:0/514670911 wait complete. 2024-09-17T04:25:02.620 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-17T04:25:02.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.842+0000 7f9c71e00640 1 -- 172.21.15.100:0/4134975732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c50005320 msgr2=0x7f9c5000b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:02.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.842+0000 7f9c71e00640 1 --2- 172.21.15.100:0/4134975732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c50005320 0x7f9c5000b9e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9c64005fd0 tx=0x7f9c50005830 comp rx=0 tx=0).stop 2024-09-17T04:25:02.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.843+0000 7f9c71e00640 1 -- 172.21.15.100:0/4134975732 shutdown_connections 2024-09-17T04:25:02.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.843+0000 7f9c71e00640 1 --2- 172.21.15.100:0/4134975732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c50005320 0x7f9c5000b9e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:02.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.843+0000 7f9c71e00640 1 -- 172.21.15.100:0/4134975732 >> 172.21.15.100:0/4134975732 conn(0x7f9c6c06d3d0 msgr2=0x7f9c6c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:02.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.843+0000 7f9c71e00640 1 -- 172.21.15.100:0/4134975732 shutdown_connections 2024-09-17T04:25:02.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.843+0000 7f9c71e00640 1 -- 172.21.15.100:0/4134975732 wait complete. 2024-09-17T04:25:02.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.845+0000 7f9c71e00640 1 Processor -- start 2024-09-17T04:25:02.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.845+0000 7f9c71e00640 1 -- start start 2024-09-17T04:25:02.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.846+0000 7f9c71e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c50005320 0x7f9c6c1577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:02.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.846+0000 7f9c71e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c6c157f50 0x7f9c6c0b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:02.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.846+0000 7f9c71e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c6c1584a0 0x7f9c6c0b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:02.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.846+0000 7f9c71e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6c070570 con 0x7f9c6c157f50 2024-09-17T04:25:02.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.846+0000 7f9c71e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6c06f4f0 con 0x7f9c6c1584a0 2024-09-17T04:25:02.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.846+0000 7f9c71e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9c6c0706f0 con 0x7f9c50005320 2024-09-17T04:25:02.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.847+0000 7f9c6a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c6c157f50 0x7f9c6c0b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:02.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.847+0000 7f9c6a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c6c157f50 0x7f9c6c0b1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51186/0 (socket says 172.21.15.100:51186) 2024-09-17T04:25:02.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.847+0000 7f9c6a000640 1 -- 172.21.15.100:0/2484248195 learned_addr learned my addr 172.21.15.100:0/2484248195 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:02.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.847+0000 7f9c6aa00640 1 --2- 172.21.15.100:0/2484248195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c50005320 0x7f9c6c1577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:02.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.847+0000 7f9c6b400640 1 --2- 172.21.15.100:0/2484248195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c6c1584a0 0x7f9c6c0b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:02.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.847+0000 7f9c6a000640 1 -- 172.21.15.100:0/2484248195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c50005320 msgr2=0x7f9c6c1577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:02.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.847+0000 7f9c6a000640 1 --2- 172.21.15.100:0/2484248195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c50005320 0x7f9c6c1577e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:02.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.847+0000 7f9c6a000640 1 -- 172.21.15.100:0/2484248195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c6c1584a0 msgr2=0x7f9c6c0b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:02.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.847+0000 7f9c6a000640 1 --2- 172.21.15.100:0/2484248195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c6c1584a0 0x7f9c6c0b18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:02.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.847+0000 7f9c6a000640 1 -- 172.21.15.100:0/2484248195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6c1e71b0 con 0x7f9c6c157f50 2024-09-17T04:25:02.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.847+0000 7f9c6a000640 1 --2- 172.21.15.100:0/2484248195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c6c157f50 0x7f9c6c0b1390 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9c58003d80 tx=0x7f9c58001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:02.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.848+0000 7f9c63400640 1 -- 172.21.15.100:0/2484248195 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c58015070 con 0x7f9c6c157f50 2024-09-17T04:25:02.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.848+0000 7f9c63400640 1 -- 172.21.15.100:0/2484248195 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c58011840 con 0x7f9c6c157f50 2024-09-17T04:25:02.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.848+0000 7f9c63400640 1 -- 172.21.15.100:0/2484248195 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c58011c90 con 0x7f9c6c157f50 2024-09-17T04:25:02.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.848+0000 7f9c71e00640 1 -- 172.21.15.100:0/2484248195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c6c157d20 con 0x7f9c6c157f50 2024-09-17T04:25:02.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.848+0000 7f9c71e00640 1 -- 172.21.15.100:0/2484248195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c6c15a7b0 con 0x7f9c6c157f50 2024-09-17T04:25:02.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.849+0000 7f9c71e00640 1 -- 172.21.15.100:0/2484248195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c6c06f850 con 0x7f9c6c157f50 2024-09-17T04:25:02.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.850+0000 7f9c63400640 1 -- 172.21.15.100:0/2484248195 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9c58018070 con 0x7f9c6c157f50 2024-09-17T04:25:02.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.851+0000 7f9c63400640 1 --2- 172.21.15.100:0/2484248195 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9c4c078bc0 0x7f9c4c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:02.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.851+0000 7f9c63400640 1 -- 172.21.15.100:0/2484248195 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f9c5809b650 con 0x7f9c6c157f50 2024-09-17T04:25:02.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.851+0000 7f9c6aa00640 1 --2- 172.21.15.100:0/2484248195 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9c4c078bc0 0x7f9c4c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:02.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.852+0000 7f9c6aa00640 1 --2- 172.21.15.100:0/2484248195 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9c4c078bc0 0x7f9c4c07b0d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c000c00 tx=0x7f9c5c000f80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:02.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:02.855+0000 7f9c63400640 1 -- 172.21.15.100:0/2484248195 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9c6c06f850 con 0x7f9c6c157f50 2024-09-17T04:25:03.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.111+0000 7f9c71e00640 1 -- 172.21.15.100:0/2484248195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f9c6c070c50 con 0x7f9c6c157f50 2024-09-17T04:25:03.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.114+0000 7f9c63400640 1 -- 172.21.15.100:0/2484248195 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f9c580636f0 con 0x7f9c6c157f50 2024-09-17T04:25:03.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.116+0000 7f9c71e00640 1 -- 172.21.15.100:0/2484248195 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9c4c078bc0 msgr2=0x7f9c4c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:03.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.116+0000 7f9c71e00640 1 --2- 172.21.15.100:0/2484248195 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9c4c078bc0 0x7f9c4c07b0d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c000c00 tx=0x7f9c5c000f80 comp rx=0 tx=0).stop 2024-09-17T04:25:03.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.116+0000 7f9c71e00640 1 -- 172.21.15.100:0/2484248195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c6c157f50 msgr2=0x7f9c6c0b1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:03.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.116+0000 7f9c71e00640 1 --2- 172.21.15.100:0/2484248195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c6c157f50 0x7f9c6c0b1390 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9c58003d80 tx=0x7f9c58001800 comp rx=0 tx=0).stop 2024-09-17T04:25:03.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.116+0000 7f9c71e00640 1 -- 172.21.15.100:0/2484248195 shutdown_connections 2024-09-17T04:25:03.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.116+0000 7f9c71e00640 1 --2- 172.21.15.100:0/2484248195 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9c4c078bc0 0x7f9c4c07b0d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:03.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.116+0000 7f9c71e00640 1 --2- 172.21.15.100:0/2484248195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c6c1584a0 0x7f9c6c0b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:03.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.116+0000 7f9c71e00640 1 --2- 172.21.15.100:0/2484248195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c6c157f50 0x7f9c6c0b1390 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:03.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.116+0000 7f9c71e00640 1 --2- 172.21.15.100:0/2484248195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c50005320 0x7f9c6c1577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:03.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.116+0000 7f9c71e00640 1 -- 172.21.15.100:0/2484248195 >> 172.21.15.100:0/2484248195 conn(0x7f9c6c06d3d0 msgr2=0x7f9c6c0ad1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:03.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.116+0000 7f9c71e00640 1 -- 172.21.15.100:0/2484248195 shutdown_connections 2024-09-17T04:25:03.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.116+0000 7f9c71e00640 1 -- 172.21.15.100:0/2484248195 wait complete. 2024-09-17T04:25:03.137 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-17T04:25:03.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.357+0000 7fd3af400640 1 -- 172.21.15.100:0/2005589610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3a815ce30 msgr2=0x7fd3a815d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:03.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.357+0000 7fd3af400640 1 --2- 172.21.15.100:0/2005589610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3a815ce30 0x7fd3a815d240 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0005fd0 tx=0x7fd3a002aa70 comp rx=0 tx=0).stop 2024-09-17T04:25:03.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.357+0000 7fd3af400640 1 -- 172.21.15.100:0/2005589610 shutdown_connections 2024-09-17T04:25:03.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.357+0000 7fd3af400640 1 --2- 172.21.15.100:0/2005589610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3a815ce30 0x7fd3a815d240 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:03.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.357+0000 7fd3af400640 1 --2- 172.21.15.100:0/2005589610 >> v2:172.21.15.184:3300/0 conn(0x7fd3a80cfc80 0x7fd3a8156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:03.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.357+0000 7fd3af400640 1 -- 172.21.15.100:0/2005589610 >> 172.21.15.100:0/2005589610 conn(0x7fd3a806d3d0 msgr2=0x7fd3a80b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:03.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.357+0000 7fd3af400640 1 -- 172.21.15.100:0/2005589610 shutdown_connections 2024-09-17T04:25:03.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.357+0000 7fd3af400640 1 -- 172.21.15.100:0/2005589610 wait complete. 2024-09-17T04:25:03.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.359+0000 7fd3af400640 1 Processor -- start 2024-09-17T04:25:03.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.360+0000 7fd3af400640 1 -- start start 2024-09-17T04:25:03.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.361+0000 7fd3af400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3a80705b0 0x7fd3a8074d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:03.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.361+0000 7fd3af400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3a80cfc80 0x7fd3a80c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:03.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.361+0000 7fd3af400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3a815ce30 0x7fd3a815bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:03.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.361+0000 7fd3af400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a80cd050 con 0x7fd3a80cfc80 2024-09-17T04:25:03.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.361+0000 7fd3af400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a80cced0 con 0x7fd3a815ce30 2024-09-17T04:25:03.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.361+0000 7fd3af400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd3a80cd1d0 con 0x7fd3a80705b0 2024-09-17T04:25:03.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.361+0000 7fd3a7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3a80cfc80 0x7fd3a80c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:03.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.361+0000 7fd3a7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3a80cfc80 0x7fd3a80c5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51204/0 (socket says 172.21.15.100:51204) 2024-09-17T04:25:03.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.361+0000 7fd3a7e00640 1 -- 172.21.15.100:0/4126237581 learned_addr learned my addr 172.21.15.100:0/4126237581 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:03.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.361+0000 7fd3ad400640 1 --2- 172.21.15.100:0/4126237581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3a815ce30 0x7fd3a815bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:03.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.361+0000 7fd3aca00640 1 --2- 172.21.15.100:0/4126237581 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3a80705b0 0x7fd3a8074d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:03.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.362+0000 7fd3a7e00640 1 -- 172.21.15.100:0/4126237581 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3a80705b0 msgr2=0x7fd3a8074d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:03.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.362+0000 7fd3a7e00640 1 --2- 172.21.15.100:0/4126237581 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3a80705b0 0x7fd3a8074d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:03.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.362+0000 7fd3a7e00640 1 -- 172.21.15.100:0/4126237581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3a815ce30 msgr2=0x7fd3a815bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:03.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.362+0000 7fd3a7e00640 1 --2- 172.21.15.100:0/4126237581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3a815ce30 0x7fd3a815bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:03.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.362+0000 7fd3a7e00640 1 -- 172.21.15.100:0/4126237581 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a81f9160 con 0x7fd3a80cfc80 2024-09-17T04:25:03.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.362+0000 7fd3aca00640 1 --2- 172.21.15.100:0/4126237581 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3a80705b0 0x7fd3a8074d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:03.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.362+0000 7fd3ad400640 1 --2- 172.21.15.100:0/4126237581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3a815ce30 0x7fd3a815bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:03.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.362+0000 7fd3a7e00640 1 --2- 172.21.15.100:0/4126237581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3a80cfc80 0x7fd3a80c5db0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd39400c390 tx=0x7fd39400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:03.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.362+0000 7fd3a4c00640 1 -- 172.21.15.100:0/4126237581 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394017070 con 0x7fd3a80cfc80 2024-09-17T04:25:03.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.362+0000 7fd3a4c00640 1 -- 172.21.15.100:0/4126237581 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3940053c0 con 0x7fd3a80cfc80 2024-09-17T04:25:03.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.362+0000 7fd3a4c00640 1 -- 172.21.15.100:0/4126237581 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394003850 con 0x7fd3a80cfc80 2024-09-17T04:25:03.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.362+0000 7fd3af400640 1 -- 172.21.15.100:0/4126237581 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a81f93d0 con 0x7fd3a80cfc80 2024-09-17T04:25:03.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.362+0000 7fd3af400640 1 -- 172.21.15.100:0/4126237581 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a81f9990 con 0x7fd3a80cfc80 2024-09-17T04:25:03.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.363+0000 7fd3af400640 1 -- 172.21.15.100:0/4126237581 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3a8075360 con 0x7fd3a80cfc80 2024-09-17T04:25:03.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.365+0000 7fd3a4c00640 1 -- 172.21.15.100:0/4126237581 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd39401a070 con 0x7fd3a80cfc80 2024-09-17T04:25:03.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.366+0000 7fd3a4c00640 1 --2- 172.21.15.100:0/4126237581 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd388078b90 0x7fd38807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:03.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.366+0000 7fd3a4c00640 1 -- 172.21.15.100:0/4126237581 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7fd394099e10 con 0x7fd3a80cfc80 2024-09-17T04:25:03.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.366+0000 7fd3aca00640 1 --2- 172.21.15.100:0/4126237581 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd388078b90 0x7fd38807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:03.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.367+0000 7fd3aca00640 1 --2- 172.21.15.100:0/4126237581 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd388078b90 0x7fd38807b0a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd3a809e510 tx=0x7fd398005f10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:03.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.370+0000 7fd3a4c00640 1 -- 172.21.15.100:0/4126237581 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd3940625a0 con 0x7fd3a80cfc80 2024-09-17T04:25:03.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.621+0000 7fd3af400640 1 -- 172.21.15.100:0/4126237581 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fd3a80cd6f0 con 0x7fd3a80cfc80 2024-09-17T04:25:03.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.625+0000 7fd3a4c00640 1 -- 172.21.15.100:0/4126237581 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7fd394061ea0 con 0x7fd3a80cfc80 2024-09-17T04:25:03.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.627+0000 7fd3af400640 1 -- 172.21.15.100:0/4126237581 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd388078b90 msgr2=0x7fd38807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:03.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.627+0000 7fd3af400640 1 --2- 172.21.15.100:0/4126237581 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd388078b90 0x7fd38807b0a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd3a809e510 tx=0x7fd398005f10 comp rx=0 tx=0).stop 2024-09-17T04:25:03.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.627+0000 7fd3af400640 1 -- 172.21.15.100:0/4126237581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3a80cfc80 msgr2=0x7fd3a80c5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:03.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.627+0000 7fd3af400640 1 --2- 172.21.15.100:0/4126237581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3a80cfc80 0x7fd3a80c5db0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd39400c390 tx=0x7fd39400c850 comp rx=0 tx=0).stop 2024-09-17T04:25:03.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.627+0000 7fd3af400640 1 -- 172.21.15.100:0/4126237581 shutdown_connections 2024-09-17T04:25:03.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.627+0000 7fd3af400640 1 --2- 172.21.15.100:0/4126237581 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd388078b90 0x7fd38807b0a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:03.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.627+0000 7fd3af400640 1 --2- 172.21.15.100:0/4126237581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3a815ce30 0x7fd3a815bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:03.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.627+0000 7fd3af400640 1 --2- 172.21.15.100:0/4126237581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3a80cfc80 0x7fd3a80c5db0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:03.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.627+0000 7fd3af400640 1 --2- 172.21.15.100:0/4126237581 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3a80705b0 0x7fd3a8074d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:03.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.627+0000 7fd3af400640 1 -- 172.21.15.100:0/4126237581 >> 172.21.15.100:0/4126237581 conn(0x7fd3a806d3d0 msgr2=0x7fd3a80b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:03.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.628+0000 7fd3af400640 1 -- 172.21.15.100:0/4126237581 shutdown_connections 2024-09-17T04:25:03.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.628+0000 7fd3af400640 1 -- 172.21.15.100:0/4126237581 wait complete. 2024-09-17T04:25:03.652 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-17T04:25:03.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.871+0000 7f336be00640 1 -- 172.21.15.100:0/3449589759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f336c0b2f60 msgr2=0x7f336c15a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:03.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.871+0000 7f336be00640 1 --2- 172.21.15.100:0/3449589759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f336c0b2f60 0x7f336c15a0f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f336c051590 tx=0x7f335402a4d0 comp rx=0 tx=0).stop 2024-09-17T04:25:03.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.871+0000 7f336be00640 1 -- 172.21.15.100:0/3449589759 shutdown_connections 2024-09-17T04:25:03.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.871+0000 7f336be00640 1 --2- 172.21.15.100:0/3449589759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f336c0b2f60 0x7f336c15a0f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:03.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.871+0000 7f336be00640 1 --2- 172.21.15.100:0/3449589759 >> v2:172.21.15.184:3300/0 conn(0x7f336c0ad9c0 0x7f336c0af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:03.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.871+0000 7f336be00640 1 -- 172.21.15.100:0/3449589759 >> 172.21.15.100:0/3449589759 conn(0x7f336c0b1e40 msgr2=0x7f336c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:03.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.871+0000 7f336be00640 1 -- 172.21.15.100:0/3449589759 shutdown_connections 2024-09-17T04:25:03.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.871+0000 7f336be00640 1 -- 172.21.15.100:0/3449589759 wait complete. 2024-09-17T04:25:03.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.873+0000 7f336be00640 1 Processor -- start 2024-09-17T04:25:03.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.874+0000 7f336be00640 1 -- start start 2024-09-17T04:25:03.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.875+0000 7f336be00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f336c0ad9c0 0x7f336c0ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:03.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.875+0000 7f336be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f336c0b2f60 0x7f336c14b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:03.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.875+0000 7f336be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f336c15d670 0x7f336c14b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:03.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.875+0000 7f336be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f336c06f290 con 0x7f336c15d670 2024-09-17T04:25:03.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.875+0000 7f336be00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f336c070d50 con 0x7f336c0b2f60 2024-09-17T04:25:03.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.875+0000 7f336be00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f336c06f410 con 0x7f336c0ad9c0 2024-09-17T04:25:03.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.875+0000 7f336b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f336c15d670 0x7f336c14b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:03.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.875+0000 7f336b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f336c15d670 0x7f336c14b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51224/0 (socket says 172.21.15.100:51224) 2024-09-17T04:25:03.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.875+0000 7f336b400640 1 -- 172.21.15.100:0/2100718923 learned_addr learned my addr 172.21.15.100:0/2100718923 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:03.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.876+0000 7f336aa00640 1 --2- 172.21.15.100:0/2100718923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f336c0ad9c0 0x7f336c0ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:03.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.876+0000 7f3362000640 1 --2- 172.21.15.100:0/2100718923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f336c0b2f60 0x7f336c14b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:03.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.876+0000 7f336b400640 1 -- 172.21.15.100:0/2100718923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f336c0ad9c0 msgr2=0x7f336c0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:03.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.876+0000 7f336b400640 1 --2- 172.21.15.100:0/2100718923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f336c0ad9c0 0x7f336c0ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:03.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.876+0000 7f336b400640 1 -- 172.21.15.100:0/2100718923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f336c0b2f60 msgr2=0x7f336c14b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:03.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.876+0000 7f336b400640 1 --2- 172.21.15.100:0/2100718923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f336c0b2f60 0x7f336c14b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:03.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.876+0000 7f336b400640 1 -- 172.21.15.100:0/2100718923 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336c09bf10 con 0x7f336c15d670 2024-09-17T04:25:03.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.876+0000 7f336b400640 1 --2- 172.21.15.100:0/2100718923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f336c15d670 0x7f336c14b880 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f336400be90 tx=0x7f336400bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:03.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.876+0000 7f3363e00640 1 -- 172.21.15.100:0/2100718923 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364019070 con 0x7f336c15d670 2024-09-17T04:25:03.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.876+0000 7f3363e00640 1 -- 172.21.15.100:0/2100718923 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3364014440 con 0x7f336c15d670 2024-09-17T04:25:03.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.877+0000 7f336be00640 1 -- 172.21.15.100:0/2100718923 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f336c09c1f0 con 0x7f336c15d670 2024-09-17T04:25:03.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.877+0000 7f3363e00640 1 -- 172.21.15.100:0/2100718923 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33640148d0 con 0x7f336c15d670 2024-09-17T04:25:03.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.877+0000 7f336be00640 1 -- 172.21.15.100:0/2100718923 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f336c09c7b0 con 0x7f336c15d670 2024-09-17T04:25:03.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.878+0000 7f336be00640 1 -- 172.21.15.100:0/2100718923 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f336c06f8c0 con 0x7f336c15d670 2024-09-17T04:25:03.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.879+0000 7f3363e00640 1 -- 172.21.15.100:0/2100718923 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f336401c070 con 0x7f336c15d670 2024-09-17T04:25:03.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.880+0000 7f3363e00640 1 --2- 172.21.15.100:0/2100718923 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3348078b90 0x7f334807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:03.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.881+0000 7f3363e00640 1 -- 172.21.15.100:0/2100718923 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f336409a9e0 con 0x7f336c15d670 2024-09-17T04:25:03.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.881+0000 7f336aa00640 1 --2- 172.21.15.100:0/2100718923 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3348078b90 0x7f334807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:03.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.882+0000 7f336aa00640 1 --2- 172.21.15.100:0/2100718923 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3348078b90 0x7f334807b0a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f335c005fd0 tx=0x7f335c00a250 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:03.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:03.885+0000 7f3363e00640 1 -- 172.21.15.100:0/2100718923 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3364062a80 con 0x7f336c15d670 2024-09-17T04:25:04.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.144+0000 7f336be00640 1 -- 172.21.15.100:0/2100718923 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f336c14c170 con 0x7f336c15d670 2024-09-17T04:25:04.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.147+0000 7f3363e00640 1 -- 172.21.15.100:0/2100718923 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7f33640674b0 con 0x7f336c15d670 2024-09-17T04:25:04.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.151+0000 7f336be00640 1 -- 172.21.15.100:0/2100718923 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3348078b90 msgr2=0x7f334807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:04.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.151+0000 7f336be00640 1 --2- 172.21.15.100:0/2100718923 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3348078b90 0x7f334807b0a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f335c005fd0 tx=0x7f335c00a250 comp rx=0 tx=0).stop 2024-09-17T04:25:04.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.151+0000 7f336be00640 1 -- 172.21.15.100:0/2100718923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f336c15d670 msgr2=0x7f336c14b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:04.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.151+0000 7f336be00640 1 --2- 172.21.15.100:0/2100718923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f336c15d670 0x7f336c14b880 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f336400be90 tx=0x7f336400bf80 comp rx=0 tx=0).stop 2024-09-17T04:25:04.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.151+0000 7f336be00640 1 -- 172.21.15.100:0/2100718923 shutdown_connections 2024-09-17T04:25:04.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.151+0000 7f336be00640 1 --2- 172.21.15.100:0/2100718923 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3348078b90 0x7f334807b0a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:04.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.151+0000 7f336be00640 1 --2- 172.21.15.100:0/2100718923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f336c15d670 0x7f336c14b880 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:04.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.151+0000 7f336be00640 1 --2- 172.21.15.100:0/2100718923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f336c0b2f60 0x7f336c14b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:04.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.151+0000 7f336be00640 1 --2- 172.21.15.100:0/2100718923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f336c0ad9c0 0x7f336c0ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:04.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.151+0000 7f336be00640 1 -- 172.21.15.100:0/2100718923 >> 172.21.15.100:0/2100718923 conn(0x7f336c0b1e40 msgr2=0x7f336c070530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:04.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.151+0000 7f336be00640 1 -- 172.21.15.100:0/2100718923 shutdown_connections 2024-09-17T04:25:04.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.151+0000 7f336be00640 1 -- 172.21.15.100:0/2100718923 wait complete. 2024-09-17T04:25:04.178 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-17T04:25:04.362 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:25:04.366 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:25:04.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.406+0000 7f97cd800640 1 -- 172.21.15.100:0/1326538447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97c80749a0 msgr2=0x7f97c80b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:04.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.406+0000 7f97cd800640 1 --2- 172.21.15.100:0/1326538447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97c80749a0 0x7f97c80b4d60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f97b8005fd0 tx=0x7f97b802a5d0 comp rx=0 tx=0).stop 2024-09-17T04:25:04.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.406+0000 7f97cd800640 1 -- 172.21.15.100:0/1326538447 shutdown_connections 2024-09-17T04:25:04.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.406+0000 7f97cd800640 1 --2- 172.21.15.100:0/1326538447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97c8075d20 0x7f97c806ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:04.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.406+0000 7f97cd800640 1 --2- 172.21.15.100:0/1326538447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97c80749a0 0x7f97c80b4d60 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:04.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.406+0000 7f97cd800640 1 -- 172.21.15.100:0/1326538447 >> 172.21.15.100:0/1326538447 conn(0x7f97c8073940 msgr2=0x7f97c8073d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:04.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.406+0000 7f97cd800640 1 -- 172.21.15.100:0/1326538447 shutdown_connections 2024-09-17T04:25:04.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.406+0000 7f97cd800640 1 -- 172.21.15.100:0/1326538447 wait complete. 2024-09-17T04:25:04.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.408+0000 7f97cd800640 1 Processor -- start 2024-09-17T04:25:04.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.409+0000 7f97cd800640 1 -- start start 2024-09-17T04:25:04.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.410+0000 7f97cd800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97c806e2f0 0x7f97c8147150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:04.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.410+0000 7f97cd800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97c80749a0 0x7f97c81e8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:04.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.410+0000 7f97cd800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97c8075d20 0x7f97c81e90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:04.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.410+0000 7f97cd800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f97c8162c80 con 0x7f97c806e2f0 2024-09-17T04:25:04.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.410+0000 7f97cd800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f97c8070bc0 con 0x7f97c8075d20 2024-09-17T04:25:04.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.410+0000 7f97cd800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f97c8162e00 con 0x7f97c80749a0 2024-09-17T04:25:04.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.410+0000 7f97c6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97c806e2f0 0x7f97c8147150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:04.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.410+0000 7f97c6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97c806e2f0 0x7f97c8147150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51246/0 (socket says 172.21.15.100:51246) 2024-09-17T04:25:04.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.410+0000 7f97c6a00640 1 -- 172.21.15.100:0/1425252230 learned_addr learned my addr 172.21.15.100:0/1425252230 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:04.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.411+0000 7f97c7400640 1 --2- 172.21.15.100:0/1425252230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97c8075d20 0x7f97c81e90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:04.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.411+0000 7f97c6000640 1 --2- 172.21.15.100:0/1425252230 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97c80749a0 0x7f97c81e8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:04.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.411+0000 7f97c6a00640 1 -- 172.21.15.100:0/1425252230 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97c80749a0 msgr2=0x7f97c81e8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:04.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.411+0000 7f97c6a00640 1 --2- 172.21.15.100:0/1425252230 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97c80749a0 0x7f97c81e8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:04.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.411+0000 7f97c6a00640 1 -- 172.21.15.100:0/1425252230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97c8075d20 msgr2=0x7f97c81e90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:04.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.411+0000 7f97c6a00640 1 --2- 172.21.15.100:0/1425252230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97c8075d20 0x7f97c81e90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:04.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.411+0000 7f97c6a00640 1 -- 172.21.15.100:0/1425252230 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c8157830 con 0x7f97c806e2f0 2024-09-17T04:25:04.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.411+0000 7f97c6a00640 1 --2- 172.21.15.100:0/1425252230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97c806e2f0 0x7f97c8147150 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f97b8000c00 tx=0x7f97b80035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:04.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.412+0000 7f97bea00640 1 -- 172.21.15.100:0/1425252230 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b803e030 con 0x7f97c806e2f0 2024-09-17T04:25:04.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.412+0000 7f97bea00640 1 -- 172.21.15.100:0/1425252230 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97b8003ae0 con 0x7f97c806e2f0 2024-09-17T04:25:04.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.412+0000 7f97bea00640 1 -- 172.21.15.100:0/1425252230 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b8035040 con 0x7f97c806e2f0 2024-09-17T04:25:04.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.412+0000 7f97cd800640 1 -- 172.21.15.100:0/1425252230 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97c81f5800 con 0x7f97c806e2f0 2024-09-17T04:25:04.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.412+0000 7f97cd800640 1 -- 172.21.15.100:0/1425252230 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97c8156520 con 0x7f97c806e2f0 2024-09-17T04:25:04.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.413+0000 7f97cd800640 1 -- 172.21.15.100:0/1425252230 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97c8071720 con 0x7f97c806e2f0 2024-09-17T04:25:04.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.414+0000 7f97bea00640 1 -- 172.21.15.100:0/1425252230 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f97b8041030 con 0x7f97c806e2f0 2024-09-17T04:25:04.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.415+0000 7f97bea00640 1 --2- 172.21.15.100:0/1425252230 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f97a8078b90 0x7f97a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:04.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.416+0000 7f97bea00640 1 -- 172.21.15.100:0/1425252230 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f97b80bae70 con 0x7f97c806e2f0 2024-09-17T04:25:04.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.416+0000 7f97c6000640 1 --2- 172.21.15.100:0/1425252230 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f97a8078b90 0x7f97a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:04.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.417+0000 7f97c6000640 1 --2- 172.21.15.100:0/1425252230 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f97a8078b90 0x7f97a807b0a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f97c80ccf00 tx=0x7f97b4005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:04.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.422+0000 7f97bea00640 1 -- 172.21.15.100:0/1425252230 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f97c8071720 con 0x7f97c806e2f0 2024-09-17T04:25:04.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.706+0000 7f97cd800640 1 -- 172.21.15.100:0/1425252230 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f97c81632b0 con 0x7f97c806e2f0 2024-09-17T04:25:04.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.709+0000 7f97bea00640 1 -- 172.21.15.100:0/1425252230 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f97b8083030 con 0x7f97c806e2f0 2024-09-17T04:25:04.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.711+0000 7f97cd800640 1 -- 172.21.15.100:0/1425252230 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f97a8078b90 msgr2=0x7f97a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:04.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.711+0000 7f97cd800640 1 --2- 172.21.15.100:0/1425252230 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f97a8078b90 0x7f97a807b0a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f97c80ccf00 tx=0x7f97b4005f90 comp rx=0 tx=0).stop 2024-09-17T04:25:04.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.711+0000 7f97cd800640 1 -- 172.21.15.100:0/1425252230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97c806e2f0 msgr2=0x7f97c8147150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:04.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.711+0000 7f97cd800640 1 --2- 172.21.15.100:0/1425252230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97c806e2f0 0x7f97c8147150 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f97b8000c00 tx=0x7f97b80035f0 comp rx=0 tx=0).stop 2024-09-17T04:25:04.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.711+0000 7f97cd800640 1 -- 172.21.15.100:0/1425252230 shutdown_connections 2024-09-17T04:25:04.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.711+0000 7f97cd800640 1 --2- 172.21.15.100:0/1425252230 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f97a8078b90 0x7f97a807b0a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:04.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.711+0000 7f97cd800640 1 --2- 172.21.15.100:0/1425252230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97c8075d20 0x7f97c81e90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:04.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.712+0000 7f97cd800640 1 --2- 172.21.15.100:0/1425252230 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97c80749a0 0x7f97c81e8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:04.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.712+0000 7f97cd800640 1 --2- 172.21.15.100:0/1425252230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97c806e2f0 0x7f97c8147150 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:04.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.712+0000 7f97cd800640 1 -- 172.21.15.100:0/1425252230 >> 172.21.15.100:0/1425252230 conn(0x7f97c8073940 msgr2=0x7f97c80b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:04.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.712+0000 7f97cd800640 1 -- 172.21.15.100:0/1425252230 shutdown_connections 2024-09-17T04:25:04.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:04.712+0000 7f97cd800640 1 -- 172.21.15.100:0/1425252230 wait complete. 2024-09-17T04:25:04.735 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T04:25:05.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.001+0000 7f114ec00640 1 -- 172.21.15.100:0/1984196326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11480d4fc0 msgr2=0x7f11480d2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:05.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.001+0000 7f114ec00640 1 --2- 172.21.15.100:0/1984196326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11480d4fc0 0x7f11480d2310 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1134005fd0 tx=0x7f113402a540 comp rx=0 tx=0).stop 2024-09-17T04:25:05.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.001+0000 7f114ec00640 1 -- 172.21.15.100:0/1984196326 shutdown_connections 2024-09-17T04:25:05.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.001+0000 7f114ec00640 1 --2- 172.21.15.100:0/1984196326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11480d2850 0x7f11480cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:05.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.001+0000 7f114ec00640 1 --2- 172.21.15.100:0/1984196326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11480d4fc0 0x7f11480d2310 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:05.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.001+0000 7f114ec00640 1 -- 172.21.15.100:0/1984196326 >> 172.21.15.100:0/1984196326 conn(0x7f11480d44a0 msgr2=0x7f11480d48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:05.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.001+0000 7f114ec00640 1 -- 172.21.15.100:0/1984196326 shutdown_connections 2024-09-17T04:25:05.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.001+0000 7f114ec00640 1 -- 172.21.15.100:0/1984196326 wait complete. 2024-09-17T04:25:05.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.003+0000 7f114ec00640 1 Processor -- start 2024-09-17T04:25:05.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.003+0000 7f114ec00640 1 -- start start 2024-09-17T04:25:05.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.005+0000 7f114ec00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11480b9840 0x7f114815a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:05.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.005+0000 7f114ec00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11480d2850 0x7f114815dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:05.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.005+0000 7f114ec00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11480d4fc0 0x7f114815e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:05.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.005+0000 7f114ec00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f11480d0e90 con 0x7f11480d2850 2024-09-17T04:25:05.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.005+0000 7f114ec00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f11480d0d10 con 0x7f11480d4fc0 2024-09-17T04:25:05.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.005+0000 7f114ec00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f11480d1010 con 0x7f11480b9840 2024-09-17T04:25:05.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.005+0000 7f114ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11480d2850 0x7f114815dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:05.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.005+0000 7f114ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11480d2850 0x7f114815dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38758/0 (socket says 172.21.15.100:38758) 2024-09-17T04:25:05.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.005+0000 7f114ce00640 1 -- 172.21.15.100:0/3195657969 learned_addr learned my addr 172.21.15.100:0/3195657969 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:05.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.005+0000 7f114d800640 1 --2- 172.21.15.100:0/3195657969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11480b9840 0x7f114815a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:05.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.005+0000 7f114e200640 1 --2- 172.21.15.100:0/3195657969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11480d4fc0 0x7f114815e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:05.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.005+0000 7f114ce00640 1 -- 172.21.15.100:0/3195657969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11480b9840 msgr2=0x7f114815a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:05.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.005+0000 7f114ce00640 1 --2- 172.21.15.100:0/3195657969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11480b9840 0x7f114815a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:05.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.005+0000 7f114ce00640 1 -- 172.21.15.100:0/3195657969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11480d4fc0 msgr2=0x7f114815e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:05.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.005+0000 7f114ce00640 1 --2- 172.21.15.100:0/3195657969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11480d4fc0 0x7f114815e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:05.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.006+0000 7f114ce00640 1 -- 172.21.15.100:0/3195657969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11480b6d90 con 0x7f11480d2850 2024-09-17T04:25:05.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.006+0000 7f114d800640 1 --2- 172.21.15.100:0/3195657969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11480b9840 0x7f114815a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:05.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.006+0000 7f114e200640 1 --2- 172.21.15.100:0/3195657969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11480d4fc0 0x7f114815e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:05.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.006+0000 7f114ce00640 1 --2- 172.21.15.100:0/3195657969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11480d2850 0x7f114815dae0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1134034040 tx=0x7f1134003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:05.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.006+0000 7f1145600640 1 -- 172.21.15.100:0/3195657969 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113403e030 con 0x7f11480d2850 2024-09-17T04:25:05.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.006+0000 7f1145600640 1 -- 172.21.15.100:0/3195657969 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11340047a0 con 0x7f11480d2850 2024-09-17T04:25:05.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.006+0000 7f114ec00640 1 -- 172.21.15.100:0/3195657969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11480cda00 con 0x7f11480d2850 2024-09-17T04:25:05.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.006+0000 7f1145600640 1 -- 172.21.15.100:0/3195657969 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1134035040 con 0x7f11480d2850 2024-09-17T04:25:05.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.006+0000 7f114ec00640 1 -- 172.21.15.100:0/3195657969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11480b6590 con 0x7f11480d2850 2024-09-17T04:25:05.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.007+0000 7f114ec00640 1 -- 172.21.15.100:0/3195657969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11480cd770 con 0x7f11480d2850 2024-09-17T04:25:05.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.009+0000 7f1145600640 1 -- 172.21.15.100:0/3195657969 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1134041030 con 0x7f11480d2850 2024-09-17T04:25:05.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.010+0000 7f1145600640 1 --2- 172.21.15.100:0/3195657969 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1128078bc0 0x7f112807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:05.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.010+0000 7f1145600640 1 -- 172.21.15.100:0/3195657969 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f11340bb320 con 0x7f11480d2850 2024-09-17T04:25:05.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.010+0000 7f114d800640 1 --2- 172.21.15.100:0/3195657969 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1128078bc0 0x7f112807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:05.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.011+0000 7f114d800640 1 --2- 172.21.15.100:0/3195657969 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1128078bc0 0x7f112807b0d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f113c007bc0 tx=0x7f113c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:05.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.018+0000 7f1145600640 1 -- 172.21.15.100:0/3195657969 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f1134083aa0 con 0x7f11480d2850 2024-09-17T04:25:05.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.211+0000 7f114ec00640 1 -- 172.21.15.100:0/3195657969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f114808cc70 con 0x7f11480d2850 2024-09-17T04:25:05.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.212+0000 7f1145600640 1 -- 172.21.15.100:0/3195657969 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v32) ==== 84+0+65 (secure 0 0 0) 0x7f114808cc70 con 0x7f11480d2850 2024-09-17T04:25:05.212 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:05.212 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-09-17T04:25:05.212 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:05.212 INFO:teuthology.orchestra.run.smithi100.stdout: "poolnum": 1, 2024-09-17T04:25:05.212 INFO:teuthology.orchestra.run.smithi100.stdout: "poolname": ".mgr" 2024-09-17T04:25:05.212 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:05.212 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-09-17T04:25:05.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.214+0000 7f114ec00640 1 -- 172.21.15.100:0/3195657969 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1128078bc0 msgr2=0x7f112807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:05.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.214+0000 7f114ec00640 1 --2- 172.21.15.100:0/3195657969 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1128078bc0 0x7f112807b0d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f113c007bc0 tx=0x7f113c009040 comp rx=0 tx=0).stop 2024-09-17T04:25:05.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.214+0000 7f114ec00640 1 -- 172.21.15.100:0/3195657969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11480d2850 msgr2=0x7f114815dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:05.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.214+0000 7f114ec00640 1 --2- 172.21.15.100:0/3195657969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11480d2850 0x7f114815dae0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1134034040 tx=0x7f1134003970 comp rx=0 tx=0).stop 2024-09-17T04:25:05.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.214+0000 7f114ec00640 1 -- 172.21.15.100:0/3195657969 shutdown_connections 2024-09-17T04:25:05.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.214+0000 7f114ec00640 1 --2- 172.21.15.100:0/3195657969 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1128078bc0 0x7f112807b0d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:05.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.214+0000 7f114ec00640 1 --2- 172.21.15.100:0/3195657969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11480d4fc0 0x7f114815e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:05.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.214+0000 7f114ec00640 1 --2- 172.21.15.100:0/3195657969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11480d2850 0x7f114815dae0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:05.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.214+0000 7f114ec00640 1 --2- 172.21.15.100:0/3195657969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11480b9840 0x7f114815a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:05.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.214+0000 7f114ec00640 1 -- 172.21.15.100:0/3195657969 >> 172.21.15.100:0/3195657969 conn(0x7f11480d44a0 msgr2=0x7f1148068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:05.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.214+0000 7f114ec00640 1 -- 172.21.15.100:0/3195657969 shutdown_connections 2024-09-17T04:25:05.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.214+0000 7f114ec00640 1 -- 172.21.15.100:0/3195657969 wait complete. 2024-09-17T04:25:05.235 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T04:25:05.236 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T04:25:05.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.464+0000 7f38e5400640 1 -- 172.21.15.100:0/1717095194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38e00b9030 msgr2=0x7f38e00d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:05.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.464+0000 7f38e5400640 1 --2- 172.21.15.100:0/1717095194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38e00b9030 0x7f38e00d2b90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f38e008d490 tx=0x7f38d802aac0 comp rx=0 tx=0).stop 2024-09-17T04:25:05.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.464+0000 7f38e5400640 1 -- 172.21.15.100:0/1717095194 shutdown_connections 2024-09-17T04:25:05.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.464+0000 7f38e5400640 1 --2- 172.21.15.100:0/1717095194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38e00b9030 0x7f38e00d2b90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:05.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.464+0000 7f38e5400640 1 --2- 172.21.15.100:0/1717095194 >> v2:172.21.15.184:3300/0 conn(0x7f38e00b9bc0 0x7f38e00d2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:05.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.464+0000 7f38e5400640 1 -- 172.21.15.100:0/1717095194 >> 172.21.15.100:0/1717095194 conn(0x7f38e008bfe0 msgr2=0x7f38e00d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:05.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.464+0000 7f38e5400640 1 -- 172.21.15.100:0/1717095194 shutdown_connections 2024-09-17T04:25:05.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.464+0000 7f38e5400640 1 -- 172.21.15.100:0/1717095194 wait complete. 2024-09-17T04:25:05.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.466+0000 7f38e5400640 1 Processor -- start 2024-09-17T04:25:05.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.466+0000 7f38e5400640 1 -- start start 2024-09-17T04:25:05.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.468+0000 7f38e5400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38e00b6700 0x7f38e015f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:05.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.468+0000 7f38e5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38e00b9030 0x7f38e00cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:05.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.468+0000 7f38e5400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38e00b9bc0 0x7f38e006de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:05.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.468+0000 7f38e5400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f38e00cf280 con 0x7f38e00b9030 2024-09-17T04:25:05.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.468+0000 7f38e5400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f38e00d4e40 con 0x7f38e00b9bc0 2024-09-17T04:25:05.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.468+0000 7f38e5400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f38e00cf400 con 0x7f38e00b6700 2024-09-17T04:25:05.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.468+0000 7f38df400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38e00b9030 0x7f38e00cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:05.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.468+0000 7f38df400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38e00b9030 0x7f38e00cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38776/0 (socket says 172.21.15.100:38776) 2024-09-17T04:25:05.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.468+0000 7f38df400640 1 -- 172.21.15.100:0/4225889733 learned_addr learned my addr 172.21.15.100:0/4225889733 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:05.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.468+0000 7f38e4a00640 1 --2- 172.21.15.100:0/4225889733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38e00b9bc0 0x7f38e006de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:05.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.468+0000 7f38dfe00640 1 --2- 172.21.15.100:0/4225889733 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38e00b6700 0x7f38e015f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:05.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.469+0000 7f38df400640 1 -- 172.21.15.100:0/4225889733 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38e00b6700 msgr2=0x7f38e015f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:05.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.469+0000 7f38df400640 1 --2- 172.21.15.100:0/4225889733 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38e00b6700 0x7f38e015f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:05.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.469+0000 7f38df400640 1 -- 172.21.15.100:0/4225889733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38e00b9bc0 msgr2=0x7f38e006de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:05.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.469+0000 7f38df400640 1 --2- 172.21.15.100:0/4225889733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38e00b9bc0 0x7f38e006de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:05.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.469+0000 7f38df400640 1 -- 172.21.15.100:0/4225889733 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e01fd2f0 con 0x7f38e00b9030 2024-09-17T04:25:05.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.469+0000 7f38df400640 1 --2- 172.21.15.100:0/4225889733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38e00b9030 0x7f38e00cab40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00c3d0 tx=0x7f38cc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:05.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.469+0000 7f38d7e00640 1 -- 172.21.15.100:0/4225889733 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc017070 con 0x7f38e00b9030 2024-09-17T04:25:05.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.469+0000 7f38d7e00640 1 -- 172.21.15.100:0/4225889733 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38cc0053c0 con 0x7f38e00b9030 2024-09-17T04:25:05.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.469+0000 7f38d7e00640 1 -- 172.21.15.100:0/4225889733 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc003850 con 0x7f38e00b9030 2024-09-17T04:25:05.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.469+0000 7f38e5400640 1 -- 172.21.15.100:0/4225889733 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38e01fd5d0 con 0x7f38e00b9030 2024-09-17T04:25:05.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.470+0000 7f38e5400640 1 -- 172.21.15.100:0/4225889733 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38e01fdb90 con 0x7f38e00b9030 2024-09-17T04:25:05.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.471+0000 7f38e5400640 1 -- 172.21.15.100:0/4225889733 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38e00cf8b0 con 0x7f38e00b9030 2024-09-17T04:25:05.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.472+0000 7f38d7e00640 1 -- 172.21.15.100:0/4225889733 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f38cc01a070 con 0x7f38e00b9030 2024-09-17T04:25:05.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.473+0000 7f38d7e00640 1 --2- 172.21.15.100:0/4225889733 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f38c0078b90 0x7f38c007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:05.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.473+0000 7f38d7e00640 1 -- 172.21.15.100:0/4225889733 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f38cc099d10 con 0x7f38e00b9030 2024-09-17T04:25:05.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.473+0000 7f38dfe00640 1 --2- 172.21.15.100:0/4225889733 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f38c0078b90 0x7f38c007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:05.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.474+0000 7f38dfe00640 1 --2- 172.21.15.100:0/4225889733 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f38c0078b90 0x7f38c007b0a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f38d0009b30 tx=0x7f38d0005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:05.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.477+0000 7f38d7e00640 1 -- 172.21.15.100:0/4225889733 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f38cc09f020 con 0x7f38e00b9030 2024-09-17T04:25:05.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:05.682+0000 7f38e5400640 1 -- 172.21.15.100:0/4225889733 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f38e00b6e50 con 0x7f38e00b9030 2024-09-17T04:25:06.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.067+0000 7f38d7e00640 1 -- 172.21.15.100:0/4225889733 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v33) ==== 138+0+0 (secure 0 0 0) 0x7f38cc061e40 con 0x7f38e00b9030 2024-09-17T04:25:06.067 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' created 2024-09-17T04:25:06.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.070+0000 7f38e5400640 1 -- 172.21.15.100:0/4225889733 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f38c0078b90 msgr2=0x7f38c007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:06.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.070+0000 7f38e5400640 1 --2- 172.21.15.100:0/4225889733 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f38c0078b90 0x7f38c007b0a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f38d0009b30 tx=0x7f38d0005f90 comp rx=0 tx=0).stop 2024-09-17T04:25:06.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.070+0000 7f38e5400640 1 -- 172.21.15.100:0/4225889733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38e00b9030 msgr2=0x7f38e00cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:06.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.070+0000 7f38e5400640 1 --2- 172.21.15.100:0/4225889733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38e00b9030 0x7f38e00cab40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00c3d0 tx=0x7f38cc00c890 comp rx=0 tx=0).stop 2024-09-17T04:25:06.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.070+0000 7f38e5400640 1 -- 172.21.15.100:0/4225889733 shutdown_connections 2024-09-17T04:25:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.070+0000 7f38e5400640 1 --2- 172.21.15.100:0/4225889733 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f38c0078b90 0x7f38c007b0a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.070+0000 7f38e5400640 1 --2- 172.21.15.100:0/4225889733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38e00b9bc0 0x7f38e006de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.070+0000 7f38e5400640 1 --2- 172.21.15.100:0/4225889733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38e00b9030 0x7f38e00cab40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.070+0000 7f38e5400640 1 --2- 172.21.15.100:0/4225889733 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38e00b6700 0x7f38e015f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.070+0000 7f38e5400640 1 -- 172.21.15.100:0/4225889733 >> 172.21.15.100:0/4225889733 conn(0x7f38e008bfe0 msgr2=0x7f38e015fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.070+0000 7f38e5400640 1 -- 172.21.15.100:0/4225889733 shutdown_connections 2024-09-17T04:25:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.070+0000 7f38e5400640 1 -- 172.21.15.100:0/4225889733 wait complete. 2024-09-17T04:25:06.095 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T04:25:06.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.322+0000 7f3041600640 1 -- 172.21.15.100:0/1263866714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f303c145a40 msgr2=0x7f303c073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:06.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.322+0000 7f3041600640 1 --2- 172.21.15.100:0/1263866714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f303c145a40 0x7f303c073540 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f303c1464e0 tx=0x7f302802a5f0 comp rx=0 tx=0).stop 2024-09-17T04:25:06.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.322+0000 7f3041600640 1 -- 172.21.15.100:0/1263866714 shutdown_connections 2024-09-17T04:25:06.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.322+0000 7f3041600640 1 --2- 172.21.15.100:0/1263866714 >> v2:172.21.15.184:3300/0 conn(0x7f303c073a80 0x7f303c0cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:06.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.322+0000 7f3041600640 1 --2- 172.21.15.100:0/1263866714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f303c145a40 0x7f303c073540 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:06.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.322+0000 7f3041600640 1 -- 172.21.15.100:0/1263866714 >> 172.21.15.100:0/1263866714 conn(0x7f303c06d1c0 msgr2=0x7f303c06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:06.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.322+0000 7f3041600640 1 -- 172.21.15.100:0/1263866714 shutdown_connections 2024-09-17T04:25:06.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.323+0000 7f3041600640 1 -- 172.21.15.100:0/1263866714 wait complete. 2024-09-17T04:25:06.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.325+0000 7f3041600640 1 Processor -- start 2024-09-17T04:25:06.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.325+0000 7f3041600640 1 -- start start 2024-09-17T04:25:06.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.326+0000 7f3041600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f303c073a80 0x7f303c1e6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:06.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.326+0000 7f3041600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f303c145550 0x7f303c15b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:06.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.326+0000 7f3041600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f303c145a40 0x7f303c0b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:06.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.326+0000 7f3041600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f303c0ce0e0 con 0x7f303c073a80 2024-09-17T04:25:06.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.326+0000 7f3041600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f303c0cdf60 con 0x7f303c145550 2024-09-17T04:25:06.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.326+0000 7f3041600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f303c0ce260 con 0x7f303c145a40 2024-09-17T04:25:06.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.326+0000 7f303aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f303c073a80 0x7f303c1e6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:06.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.327+0000 7f303aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f303c073a80 0x7f303c1e6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38796/0 (socket says 172.21.15.100:38796) 2024-09-17T04:25:06.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.327+0000 7f303aa00640 1 -- 172.21.15.100:0/236042777 learned_addr learned my addr 172.21.15.100:0/236042777 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:06.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.327+0000 7f303a000640 1 --2- 172.21.15.100:0/236042777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f303c145550 0x7f303c15b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:06.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.327+0000 7f303b400640 1 --2- 172.21.15.100:0/236042777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f303c145a40 0x7f303c0b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:06.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.327+0000 7f303aa00640 1 -- 172.21.15.100:0/236042777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f303c145a40 msgr2=0x7f303c0b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:06.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.327+0000 7f303aa00640 1 --2- 172.21.15.100:0/236042777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f303c145a40 0x7f303c0b3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:06.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.327+0000 7f303aa00640 1 -- 172.21.15.100:0/236042777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f303c145550 msgr2=0x7f303c15b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:06.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.327+0000 7f303aa00640 1 --2- 172.21.15.100:0/236042777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f303c145550 0x7f303c15b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:06.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.327+0000 7f303aa00640 1 -- 172.21.15.100:0/236042777 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f303c1e6990 con 0x7f303c073a80 2024-09-17T04:25:06.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.327+0000 7f303a000640 1 --2- 172.21.15.100:0/236042777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f303c145550 0x7f303c15b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:25:06.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.327+0000 7f303b400640 1 --2- 172.21.15.100:0/236042777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f303c145a40 0x7f303c0b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:25:06.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.327+0000 7f303aa00640 1 --2- 172.21.15.100:0/236042777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f303c073a80 0x7f303c1e6160 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f302c00b3f0 tx=0x7f302c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:06.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.328+0000 7f3032a00640 1 -- 172.21.15.100:0/236042777 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c015020 con 0x7f303c073a80 2024-09-17T04:25:06.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.328+0000 7f3032a00640 1 -- 172.21.15.100:0/236042777 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f302c011920 con 0x7f303c073a80 2024-09-17T04:25:06.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.328+0000 7f3032a00640 1 -- 172.21.15.100:0/236042777 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c011d70 con 0x7f303c073a80 2024-09-17T04:25:06.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.328+0000 7f3041600640 1 -- 172.21.15.100:0/236042777 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f303c1e6c70 con 0x7f303c073a80 2024-09-17T04:25:06.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.328+0000 7f3041600640 1 -- 172.21.15.100:0/236042777 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f303c1e8f50 con 0x7f303c073a80 2024-09-17T04:25:06.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.329+0000 7f3041600640 1 -- 172.21.15.100:0/236042777 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f303c1e77d0 con 0x7f303c073a80 2024-09-17T04:25:06.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.330+0000 7f3032a00640 1 -- 172.21.15.100:0/236042777 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f302c018020 con 0x7f303c073a80 2024-09-17T04:25:06.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.331+0000 7f3032a00640 1 --2- 172.21.15.100:0/236042777 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f301c078b90 0x7f301c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:06.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.331+0000 7f3032a00640 1 -- 172.21.15.100:0/236042777 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5227+0+0 (secure 0 0 0) 0x7f302c09a840 con 0x7f303c073a80 2024-09-17T04:25:06.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.332+0000 7f303a000640 1 --2- 172.21.15.100:0/236042777 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f301c078b90 0x7f301c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:06.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.332+0000 7f303a000640 1 --2- 172.21.15.100:0/236042777 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f301c078b90 0x7f301c07b0a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3028033000 tx=0x7f3028004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:06.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.336+0000 7f3032a00640 1 -- 172.21.15.100:0/236042777 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f303c1e77d0 con 0x7f303c073a80 2024-09-17T04:25:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:06.520+0000 7f3041600640 1 -- 172.21.15.100:0/236042777 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f303c1e91b0 con 0x7f303c073a80 2024-09-17T04:25:07.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.070+0000 7f3032a00640 1 -- 172.21.15.100:0/236042777 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v34) ==== 170+0+0 (secure 0 0 0) 0x7f302c01f030 con 0x7f303c073a80 2024-09-17T04:25:07.070 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' created 2024-09-17T04:25:07.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.073+0000 7f3041600640 1 -- 172.21.15.100:0/236042777 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f301c078b90 msgr2=0x7f301c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:07.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.073+0000 7f3041600640 1 --2- 172.21.15.100:0/236042777 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f301c078b90 0x7f301c07b0a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3028033000 tx=0x7f3028004730 comp rx=0 tx=0).stop 2024-09-17T04:25:07.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.073+0000 7f3041600640 1 -- 172.21.15.100:0/236042777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f303c073a80 msgr2=0x7f303c1e6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:07.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.073+0000 7f3041600640 1 --2- 172.21.15.100:0/236042777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f303c073a80 0x7f303c1e6160 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f302c00b3f0 tx=0x7f302c00b8b0 comp rx=0 tx=0).stop 2024-09-17T04:25:07.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.074+0000 7f3041600640 1 -- 172.21.15.100:0/236042777 shutdown_connections 2024-09-17T04:25:07.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.074+0000 7f3041600640 1 --2- 172.21.15.100:0/236042777 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f301c078b90 0x7f301c07b0a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:07.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.074+0000 7f3041600640 1 --2- 172.21.15.100:0/236042777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f303c145a40 0x7f303c0b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:07.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.074+0000 7f3041600640 1 --2- 172.21.15.100:0/236042777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f303c145550 0x7f303c15b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:07.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.074+0000 7f3041600640 1 --2- 172.21.15.100:0/236042777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f303c073a80 0x7f303c1e6160 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:07.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.074+0000 7f3041600640 1 -- 172.21.15.100:0/236042777 >> 172.21.15.100:0/236042777 conn(0x7f303c06d1c0 msgr2=0x7f303c144100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:07.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.074+0000 7f3041600640 1 -- 172.21.15.100:0/236042777 shutdown_connections 2024-09-17T04:25:07.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.075+0000 7f3041600640 1 -- 172.21.15.100:0/236042777 wait complete. 2024-09-17T04:25:07.110 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T04:25:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.334+0000 7fbe7f000640 1 -- 172.21.15.100:0/2328835024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe60004d40 msgr2=0x7fbe600051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.334+0000 7fbe7f000640 1 --2- 172.21.15.100:0/2328835024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe60004d40 0x7fbe600051f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbe7000c610 tx=0x7fbe60005830 comp rx=0 tx=0).stop 2024-09-17T04:25:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.334+0000 7fbe7e600640 1 -- 172.21.15.100:0/2328835024 reap_dead start 2024-09-17T04:25:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.334+0000 7fbe7f000640 1 -- 172.21.15.100:0/2328835024 shutdown_connections 2024-09-17T04:25:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.334+0000 7fbe7f000640 1 -- 172.21.15.100:0/2328835024 >> 172.21.15.100:0/2328835024 conn(0x7fbe7806d620 msgr2=0x7fbe780b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.335+0000 7fbe7f000640 1 -- 172.21.15.100:0/2328835024 shutdown_connections 2024-09-17T04:25:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.335+0000 7fbe7f000640 1 -- 172.21.15.100:0/2328835024 wait complete. 2024-09-17T04:25:07.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.337+0000 7fbe7f000640 1 Processor -- start 2024-09-17T04:25:07.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.337+0000 7fbe7f000640 1 -- start start 2024-09-17T04:25:07.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.338+0000 7fbe7f000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe60004d40 0x7fbe78151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:07.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.338+0000 7fbe7f000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe781f5330 0x7fbe780b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:07.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.338+0000 7fbe7f000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe781f5740 0x7fbe780adc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:07.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.338+0000 7fbe7f000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbe780ccac0 con 0x7fbe781f5740 2024-09-17T04:25:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.338+0000 7fbe7f000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbe78070370 con 0x7fbe781f5330 2024-09-17T04:25:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.338+0000 7fbe7f000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbe780ccc40 con 0x7fbe60004d40 2024-09-17T04:25:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.339+0000 7fbe7e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe781f5740 0x7fbe780adc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.339+0000 7fbe7e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe781f5740 0x7fbe780adc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38828/0 (socket says 172.21.15.100:38828) 2024-09-17T04:25:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.339+0000 7fbe7e600640 1 -- 172.21.15.100:0/771721196 learned_addr learned my addr 172.21.15.100:0/771721196 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.339+0000 7fbe7e600640 1 -- 172.21.15.100:0/771721196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe60004d40 msgr2=0x7fbe78151600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.339+0000 7fbe7e600640 1 --2- 172.21.15.100:0/771721196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe60004d40 0x7fbe78151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.339+0000 7fbe7e600640 1 -- 172.21.15.100:0/771721196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe781f5330 msgr2=0x7fbe780b0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.339+0000 7fbe7e600640 1 --2- 172.21.15.100:0/771721196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe781f5330 0x7fbe780b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:07.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.339+0000 7fbe7e600640 1 -- 172.21.15.100:0/771721196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe780b1790 con 0x7fbe781f5740 2024-09-17T04:25:07.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.339+0000 7fbe7e600640 1 --2- 172.21.15.100:0/771721196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe781f5740 0x7fbe780adc30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbe7002b4d0 tx=0x7fbe70001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:07.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.340+0000 7fbe75600640 1 -- 172.21.15.100:0/771721196 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe70039070 con 0x7fbe781f5740 2024-09-17T04:25:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.340+0000 7fbe75600640 1 -- 172.21.15.100:0/771721196 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe70006590 con 0x7fbe781f5740 2024-09-17T04:25:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.340+0000 7fbe75600640 1 -- 172.21.15.100:0/771721196 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe70002070 con 0x7fbe781f5740 2024-09-17T04:25:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.340+0000 7fbe7f000640 1 -- 172.21.15.100:0/771721196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe780ad150 con 0x7fbe781f5740 2024-09-17T04:25:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.340+0000 7fbe7f000640 1 -- 172.21.15.100:0/771721196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe780ad710 con 0x7fbe781f5740 2024-09-17T04:25:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.341+0000 7fbe7f000640 1 -- 172.21.15.100:0/771721196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe780b1500 con 0x7fbe781f5740 2024-09-17T04:25:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.342+0000 7fbe75600640 1 -- 172.21.15.100:0/771721196 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbe7003c020 con 0x7fbe781f5740 2024-09-17T04:25:07.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.343+0000 7fbe75600640 1 --2- 172.21.15.100:0/771721196 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbe58078b90 0x7fbe5807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:07.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.343+0000 7fbe75600640 1 -- 172.21.15.100:0/771721196 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5609+0+0 (secure 0 0 0) 0x7fbe700ba850 con 0x7fbe781f5740 2024-09-17T04:25:07.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.344+0000 7fbe7dc00640 1 --2- 172.21.15.100:0/771721196 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbe58078b90 0x7fbe5807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:07.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.344+0000 7fbe7dc00640 1 --2- 172.21.15.100:0/771721196 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbe58078b90 0x7fbe5807b0a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c006670 tx=0x7fbe6c006180 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:07.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.348+0000 7fbe75600640 1 -- 172.21.15.100:0/771721196 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fbe70045020 con 0x7fbe781f5740 2024-09-17T04:25:07.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:07.563+0000 7fbe7f000640 1 -- 172.21.15.100:0/771721196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fbe7815d6e0 con 0x7fbe781f5740 2024-09-17T04:25:08.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.080+0000 7fbe75600640 1 -- 172.21.15.100:0/771721196 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7fbe70082d60 con 0x7fbe781f5740 2024-09-17T04:25:08.081 INFO:teuthology.orchestra.run.smithi100.stderr:new fs with metadata pool 4 and data pool 5 2024-09-17T04:25:08.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.083+0000 7fbe7f000640 1 -- 172.21.15.100:0/771721196 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbe58078b90 msgr2=0x7fbe5807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:08.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.083+0000 7fbe7f000640 1 --2- 172.21.15.100:0/771721196 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbe58078b90 0x7fbe5807b0a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c006670 tx=0x7fbe6c006180 comp rx=0 tx=0).stop 2024-09-17T04:25:08.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.083+0000 7fbe7f000640 1 -- 172.21.15.100:0/771721196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe781f5740 msgr2=0x7fbe780adc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:08.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.083+0000 7fbe7f000640 1 --2- 172.21.15.100:0/771721196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe781f5740 0x7fbe780adc30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbe7002b4d0 tx=0x7fbe70001aa0 comp rx=0 tx=0).stop 2024-09-17T04:25:08.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.084+0000 7fbe7f000640 1 -- 172.21.15.100:0/771721196 shutdown_connections 2024-09-17T04:25:08.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.084+0000 7fbe7f000640 1 --2- 172.21.15.100:0/771721196 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbe58078b90 0x7fbe5807b0a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:08.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.084+0000 7fbe7f000640 1 --2- 172.21.15.100:0/771721196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe781f5740 0x7fbe780adc30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:08.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.084+0000 7fbe7f000640 1 --2- 172.21.15.100:0/771721196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe781f5330 0x7fbe780b0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:08.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.084+0000 7fbe7f000640 1 --2- 172.21.15.100:0/771721196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe60004d40 0x7fbe78151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:08.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.084+0000 7fbe7f000640 1 -- 172.21.15.100:0/771721196 >> 172.21.15.100:0/771721196 conn(0x7fbe7806d620 msgr2=0x7fbe78073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:08.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.087+0000 7fbe7f000640 1 -- 172.21.15.100:0/771721196 shutdown_connections 2024-09-17T04:25:08.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.087+0000 7fbe7f000640 1 -- 172.21.15.100:0/771721196 wait complete. 2024-09-17T04:25:08.134 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:25:08.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.355+0000 7f1fc1000640 1 -- 172.21.15.100:0/3811663593 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fbc1626d0 msgr2=0x7f1fbc164af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:08.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.355+0000 7f1fc1000640 1 --2- 172.21.15.100:0/3811663593 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fbc1626d0 0x7f1fbc164af0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4005fd0 tx=0x7f1fb402aa10 comp rx=0 tx=0).stop 2024-09-17T04:25:08.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.356+0000 7f1fc1000640 1 -- 172.21.15.100:0/3811663593 shutdown_connections 2024-09-17T04:25:08.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.356+0000 7f1fc1000640 1 --2- 172.21.15.100:0/3811663593 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fbc1626d0 0x7f1fbc164af0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:08.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.356+0000 7f1fc1000640 1 --2- 172.21.15.100:0/3811663593 >> v2:172.21.15.184:3300/0 conn(0x7f1fbc0704b0 0x7f1fbc073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:08.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.356+0000 7f1fc1000640 1 -- 172.21.15.100:0/3811663593 >> 172.21.15.100:0/3811663593 conn(0x7f1fbc06d730 msgr2=0x7f1fbc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:08.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.356+0000 7f1fc1000640 1 -- 172.21.15.100:0/3811663593 shutdown_connections 2024-09-17T04:25:08.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.356+0000 7f1fc1000640 1 -- 172.21.15.100:0/3811663593 wait complete. 2024-09-17T04:25:08.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.358+0000 7f1fc1000640 1 Processor -- start 2024-09-17T04:25:08.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.358+0000 7f1fc1000640 1 -- start start 2024-09-17T04:25:08.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.359+0000 7f1fc1000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fbc0704b0 0x7f1fbc14a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:08.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.359+0000 7f1fc1000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1fbc0ad3c0 0x7f1fbc1595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:08.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.359+0000 7f1fc1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fbc1626d0 0x7f1fbc1643f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:08.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.359+0000 7f1fc1000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc0ae060 con 0x7f1fbc1626d0 2024-09-17T04:25:08.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.359+0000 7f1fc1000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc0adee0 con 0x7f1fbc0ad3c0 2024-09-17T04:25:08.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.359+0000 7f1fc1000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1fbc071130 con 0x7f1fbc0704b0 2024-09-17T04:25:08.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.360+0000 7f1fba800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fbc1626d0 0x7f1fbc1643f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:08.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.360+0000 7f1fba800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fbc1626d0 0x7f1fbc1643f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38838/0 (socket says 172.21.15.100:38838) 2024-09-17T04:25:08.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.360+0000 7f1fba800640 1 -- 172.21.15.100:0/2192638858 learned_addr learned my addr 172.21.15.100:0/2192638858 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:08.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.360+0000 7f1fb9e00640 1 --2- 172.21.15.100:0/2192638858 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fbc0704b0 0x7f1fbc14a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:08.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.360+0000 7f1fb9400640 1 --2- 172.21.15.100:0/2192638858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1fbc0ad3c0 0x7f1fbc1595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:08.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.360+0000 7f1fba800640 1 -- 172.21.15.100:0/2192638858 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fbc0704b0 msgr2=0x7f1fbc14a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:08.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.360+0000 7f1fba800640 1 --2- 172.21.15.100:0/2192638858 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fbc0704b0 0x7f1fbc14a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:08.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.360+0000 7f1fba800640 1 -- 172.21.15.100:0/2192638858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1fbc0ad3c0 msgr2=0x7f1fbc1595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:08.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.360+0000 7f1fba800640 1 --2- 172.21.15.100:0/2192638858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1fbc0ad3c0 0x7f1fbc1595e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:08.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.360+0000 7f1fba800640 1 -- 172.21.15.100:0/2192638858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fbc15a340 con 0x7f1fbc1626d0 2024-09-17T04:25:08.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.360+0000 7f1fb9400640 1 --2- 172.21.15.100:0/2192638858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1fbc0ad3c0 0x7f1fbc1595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:25:08.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.360+0000 7f1fba800640 1 --2- 172.21.15.100:0/2192638858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fbc1626d0 0x7f1fbc1643f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1fb402b000 tx=0x7f1fb4003f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:08.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.361+0000 7f1fb2000640 1 -- 172.21.15.100:0/2192638858 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4033050 con 0x7f1fbc1626d0 2024-09-17T04:25:08.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.361+0000 7f1fb2000640 1 -- 172.21.15.100:0/2192638858 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1fb4004450 con 0x7f1fbc1626d0 2024-09-17T04:25:08.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.361+0000 7f1fb2000640 1 -- 172.21.15.100:0/2192638858 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4038450 con 0x7f1fbc1626d0 2024-09-17T04:25:08.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.361+0000 7f1fc1000640 1 -- 172.21.15.100:0/2192638858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fbc15a620 con 0x7f1fbc1626d0 2024-09-17T04:25:08.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.361+0000 7f1fc1000640 1 -- 172.21.15.100:0/2192638858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fbc0b08e0 con 0x7f1fbc1626d0 2024-09-17T04:25:08.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.362+0000 7f1fc1000640 1 -- 172.21.15.100:0/2192638858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fbc15a0b0 con 0x7f1fbc1626d0 2024-09-17T04:25:08.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.363+0000 7f1fb2000640 1 -- 172.21.15.100:0/2192638858 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1fb4036070 con 0x7f1fbc1626d0 2024-09-17T04:25:08.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.364+0000 7f1fb2000640 1 --2- 172.21.15.100:0/2192638858 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1f9c078b90 0x7f1f9c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:08.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.364+0000 7f1fb2000640 1 -- 172.21.15.100:0/2192638858 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f1fb40bbfb0 con 0x7f1fbc1626d0 2024-09-17T04:25:08.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.364+0000 7f1fb9e00640 1 --2- 172.21.15.100:0/2192638858 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1f9c078b90 0x7f1f9c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:08.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.365+0000 7f1fb9e00640 1 --2- 172.21.15.100:0/2192638858 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1f9c078b90 0x7f1f9c07b0a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009ac0 tx=0x7f1fac005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:08.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.368+0000 7f1fb2000640 1 -- 172.21.15.100:0/2192638858 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f1fb4084470 con 0x7f1fbc1626d0 2024-09-17T04:25:08.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.551+0000 7f1fc1000640 1 -- 172.21.15.100:0/2192638858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1fbc073040 con 0x7f1fbc1626d0 2024-09-17T04:25:08.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.553+0000 7f1fb2000640 1 -- 172.21.15.100:0/2192638858 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16790 (secure 0 0 0) 0x7f1fb4083d70 con 0x7f1fbc1626d0 2024-09-17T04:25:08.553 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:08.554 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":35,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:25:07.565769+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:25:05.694905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T04:25:06.533348+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:25:08.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.555+0000 7f1fc1000640 1 -- 172.21.15.100:0/2192638858 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1f9c078b90 msgr2=0x7f1f9c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:08.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.555+0000 7f1fc1000640 1 --2- 172.21.15.100:0/2192638858 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1f9c078b90 0x7f1f9c07b0a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009ac0 tx=0x7f1fac005f90 comp rx=0 tx=0).stop 2024-09-17T04:25:08.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.555+0000 7f1fc1000640 1 -- 172.21.15.100:0/2192638858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fbc1626d0 msgr2=0x7f1fbc1643f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:08.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.555+0000 7f1fc1000640 1 --2- 172.21.15.100:0/2192638858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fbc1626d0 0x7f1fbc1643f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1fb402b000 tx=0x7f1fb4003f20 comp rx=0 tx=0).stop 2024-09-17T04:25:08.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.555+0000 7f1fc1000640 1 -- 172.21.15.100:0/2192638858 shutdown_connections 2024-09-17T04:25:08.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.555+0000 7f1fc1000640 1 --2- 172.21.15.100:0/2192638858 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1f9c078b90 0x7f1f9c07b0a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:08.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.555+0000 7f1fc1000640 1 --2- 172.21.15.100:0/2192638858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fbc1626d0 0x7f1fbc1643f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:08.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.555+0000 7f1fc1000640 1 --2- 172.21.15.100:0/2192638858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1fbc0ad3c0 0x7f1fbc1595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:08.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.555+0000 7f1fc1000640 1 --2- 172.21.15.100:0/2192638858 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fbc0704b0 0x7f1fbc14a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:08.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.555+0000 7f1fc1000640 1 -- 172.21.15.100:0/2192638858 >> 172.21.15.100:0/2192638858 conn(0x7f1fbc06d730 msgr2=0x7f1fbc156ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:08.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.555+0000 7f1fc1000640 1 -- 172.21.15.100:0/2192638858 shutdown_connections 2024-09-17T04:25:08.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.555+0000 7f1fc1000640 1 -- 172.21.15.100:0/2192638858 wait complete. 2024-09-17T04:25:08.580 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:25:08.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.801+0000 7fc9c3e00640 1 -- 172.21.15.100:0/3225229029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9c40d2850 msgr2=0x7fc9c40cfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:08.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.801+0000 7fc9c3e00640 1 --2- 172.21.15.100:0/3225229029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9c40d2850 0x7fc9c40cfd40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc9c41632f0 tx=0x7fc9bc02a980 comp rx=0 tx=0).stop 2024-09-17T04:25:08.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.801+0000 7fc9c3e00640 1 -- 172.21.15.100:0/3225229029 shutdown_connections 2024-09-17T04:25:08.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.801+0000 7fc9c3e00640 1 --2- 172.21.15.100:0/3225229029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9c40d2850 0x7fc9c40cfd40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:08.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.801+0000 7fc9c3e00640 1 --2- 172.21.15.100:0/3225229029 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9c40b35c0 0x7fc9c40d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:08.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.801+0000 7fc9c3e00640 1 --2- 172.21.15.100:0/3225229029 >> v2:172.21.15.184:3300/0 conn(0x7fc9c40b9840 0x7fc9c40d4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:08.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.801+0000 7fc9c3e00640 1 -- 172.21.15.100:0/3225229029 >> 172.21.15.100:0/3225229029 conn(0x7fc9c40d44a0 msgr2=0x7fc9c40d48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:08.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.801+0000 7fc9c3e00640 1 -- 172.21.15.100:0/3225229029 shutdown_connections 2024-09-17T04:25:08.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.801+0000 7fc9c3e00640 1 -- 172.21.15.100:0/3225229029 wait complete. 2024-09-17T04:25:08.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.803+0000 7fc9c3e00640 1 Processor -- start 2024-09-17T04:25:08.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.803+0000 7fc9c3e00640 1 -- start start 2024-09-17T04:25:08.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.805+0000 7fc9c3e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9c40b35c0 0x7fc9c40b68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:08.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.805+0000 7fc9c3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9c40b9840 0x7fc9c41f9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:08.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.805+0000 7fc9c3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9c40d2850 0x7fc9c41f9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:08.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.805+0000 7fc9c3e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c40d0d10 con 0x7fc9c40d2850 2024-09-17T04:25:08.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.805+0000 7fc9c3e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c40d0600 con 0x7fc9c40b9840 2024-09-17T04:25:08.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.805+0000 7fc9c3e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc9c40d0e90 con 0x7fc9c40b35c0 2024-09-17T04:25:08.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.805+0000 7fc9c3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9c40d2850 0x7fc9c41f9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:08.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.805+0000 7fc9c3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9c40d2850 0x7fc9c41f9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38850/0 (socket says 172.21.15.100:38850) 2024-09-17T04:25:08.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.805+0000 7fc9c3400640 1 -- 172.21.15.100:0/1262501087 learned_addr learned my addr 172.21.15.100:0/1262501087 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:08.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.805+0000 7fc9ba000640 1 --2- 172.21.15.100:0/1262501087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9c40b9840 0x7fc9c41f9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:08.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.805+0000 7fc9c2a00640 1 --2- 172.21.15.100:0/1262501087 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9c40b35c0 0x7fc9c40b68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:08.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.805+0000 7fc9c3400640 1 -- 172.21.15.100:0/1262501087 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9c40b35c0 msgr2=0x7fc9c40b68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:08.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.805+0000 7fc9c3400640 1 --2- 172.21.15.100:0/1262501087 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9c40b35c0 0x7fc9c40b68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:08.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.805+0000 7fc9c3400640 1 -- 172.21.15.100:0/1262501087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9c40b9840 msgr2=0x7fc9c41f9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:08.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.805+0000 7fc9c3400640 1 --2- 172.21.15.100:0/1262501087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9c40b9840 0x7fc9c41f9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:08.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.805+0000 7fc9c3400640 1 -- 172.21.15.100:0/1262501087 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c40b6e20 con 0x7fc9c40d2850 2024-09-17T04:25:08.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.805+0000 7fc9ba000640 1 --2- 172.21.15.100:0/1262501087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9c40b9840 0x7fc9c41f9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:08.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.806+0000 7fc9c3400640 1 --2- 172.21.15.100:0/1262501087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9c40d2850 0x7fc9c41f9670 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc033040 tx=0x7fc9bc003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:08.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.806+0000 7fc9bbe00640 1 -- 172.21.15.100:0/1262501087 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc034040 con 0x7fc9c40d2850 2024-09-17T04:25:08.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.806+0000 7fc9bbe00640 1 -- 172.21.15.100:0/1262501087 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9bc004540 con 0x7fc9c40d2850 2024-09-17T04:25:08.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.806+0000 7fc9bbe00640 1 -- 172.21.15.100:0/1262501087 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc045480 con 0x7fc9c40d2850 2024-09-17T04:25:08.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.806+0000 7fc9c3e00640 1 -- 172.21.15.100:0/1262501087 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9c4159880 con 0x7fc9c40d2850 2024-09-17T04:25:08.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.806+0000 7fc9c3e00640 1 -- 172.21.15.100:0/1262501087 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9c4159e00 con 0x7fc9c40d2850 2024-09-17T04:25:08.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.807+0000 7fc9c3e00640 1 -- 172.21.15.100:0/1262501087 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9c408cc70 con 0x7fc9c40d2850 2024-09-17T04:25:08.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.808+0000 7fc9bbe00640 1 -- 172.21.15.100:0/1262501087 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc9bc037020 con 0x7fc9c40d2850 2024-09-17T04:25:08.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.810+0000 7fc9bbe00640 1 --2- 172.21.15.100:0/1262501087 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc9a0078b90 0x7fc9a007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:08.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.810+0000 7fc9bbe00640 1 -- 172.21.15.100:0/1262501087 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fc9bc0bc550 con 0x7fc9c40d2850 2024-09-17T04:25:08.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.810+0000 7fc9c2a00640 1 --2- 172.21.15.100:0/1262501087 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc9a0078b90 0x7fc9a007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:08.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.811+0000 7fc9c2a00640 1 --2- 172.21.15.100:0/1262501087 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc9a0078b90 0x7fc9a007b0a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009590 tx=0x7fc9b4005f30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:08.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:08.814+0000 7fc9bbe00640 1 -- 172.21.15.100:0/1262501087 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc9c408cc70 con 0x7fc9c40d2850 2024-09-17T04:25:09.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.000+0000 7fc9c3e00640 1 -- 172.21.15.100:0/1262501087 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc9c40d13b0 con 0x7fc9c40d2850 2024-09-17T04:25:09.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.001+0000 7fc9bbe00640 1 -- 172.21.15.100:0/1262501087 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16790 (secure 0 0 0) 0x7fc9bc084990 con 0x7fc9c40d2850 2024-09-17T04:25:09.001 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:09.002 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":35,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:25:07.565769+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:25:05.694905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T04:25:06.533348+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:25:09.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.003+0000 7fc9c3e00640 1 -- 172.21.15.100:0/1262501087 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc9a0078b90 msgr2=0x7fc9a007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:09.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.003+0000 7fc9c3e00640 1 --2- 172.21.15.100:0/1262501087 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc9a0078b90 0x7fc9a007b0a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009590 tx=0x7fc9b4005f30 comp rx=0 tx=0).stop 2024-09-17T04:25:09.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.003+0000 7fc9c3e00640 1 -- 172.21.15.100:0/1262501087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9c40d2850 msgr2=0x7fc9c41f9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:09.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.003+0000 7fc9c3e00640 1 --2- 172.21.15.100:0/1262501087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9c40d2850 0x7fc9c41f9670 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc033040 tx=0x7fc9bc003f10 comp rx=0 tx=0).stop 2024-09-17T04:25:09.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.003+0000 7fc9c3e00640 1 -- 172.21.15.100:0/1262501087 shutdown_connections 2024-09-17T04:25:09.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.003+0000 7fc9c3e00640 1 --2- 172.21.15.100:0/1262501087 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc9a0078b90 0x7fc9a007b0a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:09.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.003+0000 7fc9c3e00640 1 --2- 172.21.15.100:0/1262501087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9c40d2850 0x7fc9c41f9670 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:09.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.003+0000 7fc9c3e00640 1 --2- 172.21.15.100:0/1262501087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9c40b9840 0x7fc9c41f9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:09.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.003+0000 7fc9c3e00640 1 --2- 172.21.15.100:0/1262501087 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9c40b35c0 0x7fc9c40b68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:09.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.003+0000 7fc9c3e00640 1 -- 172.21.15.100:0/1262501087 >> 172.21.15.100:0/1262501087 conn(0x7fc9c40d44a0 msgr2=0x7fc9c4160a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:09.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.004+0000 7fc9c3e00640 1 -- 172.21.15.100:0/1262501087 shutdown_connections 2024-09-17T04:25:09.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.004+0000 7fc9c3e00640 1 -- 172.21.15.100:0/1262501087 wait complete. 2024-09-17T04:25:09.029 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T04:25:09.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.255+0000 7f1bd1c00640 1 -- 172.21.15.100:0/3173073108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc156ca0 msgr2=0x7f1bcc0ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:09.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.255+0000 7f1bd1c00640 1 --2- 172.21.15.100:0/3173073108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc156ca0 0x7f1bcc0ccac0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4005fd0 tx=0x7f1bb402a5d0 comp rx=0 tx=0).stop 2024-09-17T04:25:09.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.255+0000 7f1bd1c00640 1 -- 172.21.15.100:0/3173073108 shutdown_connections 2024-09-17T04:25:09.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.255+0000 7f1bd1c00640 1 --2- 172.21.15.100:0/3173073108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bcc0cd000 0x7f1bcc0af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:09.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.255+0000 7f1bd1c00640 1 --2- 172.21.15.100:0/3173073108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc156ca0 0x7f1bcc0ccac0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:09.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.255+0000 7f1bd1c00640 1 -- 172.21.15.100:0/3173073108 >> 172.21.15.100:0/3173073108 conn(0x7f1bcc06d620 msgr2=0x7f1bcc0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:09.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.255+0000 7f1bd1c00640 1 -- 172.21.15.100:0/3173073108 shutdown_connections 2024-09-17T04:25:09.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.255+0000 7f1bd1c00640 1 -- 172.21.15.100:0/3173073108 wait complete. 2024-09-17T04:25:09.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.257+0000 7f1bd1c00640 1 Processor -- start 2024-09-17T04:25:09.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.257+0000 7f1bd1c00640 1 -- start start 2024-09-17T04:25:09.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.259+0000 7f1bd1c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bcc0b3f30 0x7f1bcc1f6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:09.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.259+0000 7f1bd1c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bcc0cd000 0x7f1bcc1f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:09.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.259+0000 7f1bd1c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc156ca0 0x7f1bcc1f4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:09.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.259+0000 7f1bd1c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc0af9a0 con 0x7f1bcc156ca0 2024-09-17T04:25:09.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.259+0000 7f1bd1c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc0af820 con 0x7f1bcc0cd000 2024-09-17T04:25:09.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.259+0000 7f1bd1c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1bcc0afb20 con 0x7f1bcc0b3f30 2024-09-17T04:25:09.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.259+0000 7f1bd1200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc156ca0 0x7f1bcc1f4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:09.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.259+0000 7f1bd1200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc156ca0 0x7f1bcc1f4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38856/0 (socket says 172.21.15.100:38856) 2024-09-17T04:25:09.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.259+0000 7f1bd1200640 1 -- 172.21.15.100:0/2354316770 learned_addr learned my addr 172.21.15.100:0/2354316770 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:09.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.259+0000 7f1bc3400640 1 --2- 172.21.15.100:0/2354316770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bcc0cd000 0x7f1bcc1f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:09.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.259+0000 7f1bcbe00640 1 --2- 172.21.15.100:0/2354316770 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bcc0b3f30 0x7f1bcc1f6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:09.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.260+0000 7f1bd1200640 1 -- 172.21.15.100:0/2354316770 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bcc0b3f30 msgr2=0x7f1bcc1f6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:09.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.260+0000 7f1bd1200640 1 --2- 172.21.15.100:0/2354316770 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bcc0b3f30 0x7f1bcc1f6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:09.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.260+0000 7f1bd1200640 1 -- 172.21.15.100:0/2354316770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bcc0cd000 msgr2=0x7f1bcc1f72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:09.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.260+0000 7f1bd1200640 1 --2- 172.21.15.100:0/2354316770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bcc0cd000 0x7f1bcc1f72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:09.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.260+0000 7f1bd1200640 1 -- 172.21.15.100:0/2354316770 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bcc1458e0 con 0x7f1bcc156ca0 2024-09-17T04:25:09.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.260+0000 7f1bd1200640 1 --2- 172.21.15.100:0/2354316770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc156ca0 0x7f1bcc1f4d30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1bc400e960 tx=0x7f1bc400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:09.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.260+0000 7f1bc8c00640 1 -- 172.21.15.100:0/2354316770 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4004d20 con 0x7f1bcc156ca0 2024-09-17T04:25:09.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.260+0000 7f1bc8c00640 1 -- 172.21.15.100:0/2354316770 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1bc400cac0 con 0x7f1bcc156ca0 2024-09-17T04:25:09.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.260+0000 7f1bd1c00640 1 -- 172.21.15.100:0/2354316770 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bcc145bc0 con 0x7f1bcc156ca0 2024-09-17T04:25:09.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.260+0000 7f1bc8c00640 1 -- 172.21.15.100:0/2354316770 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4005020 con 0x7f1bcc156ca0 2024-09-17T04:25:09.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.260+0000 7f1bd1c00640 1 -- 172.21.15.100:0/2354316770 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bcc146180 con 0x7f1bcc156ca0 2024-09-17T04:25:09.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.261+0000 7f1bd1c00640 1 -- 172.21.15.100:0/2354316770 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bcc0af0d0 con 0x7f1bcc156ca0 2024-09-17T04:25:09.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.262+0000 7f1bc8c00640 1 -- 172.21.15.100:0/2354316770 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1bc40051c0 con 0x7f1bcc156ca0 2024-09-17T04:25:09.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.263+0000 7f1bc8c00640 1 --2- 172.21.15.100:0/2354316770 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1ba8078b90 0x7f1ba807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:09.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.264+0000 7f1bc8c00640 1 -- 172.21.15.100:0/2354316770 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f1bc409b610 con 0x7f1bcc156ca0 2024-09-17T04:25:09.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.264+0000 7f1bcbe00640 1 --2- 172.21.15.100:0/2354316770 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1ba8078b90 0x7f1ba807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:09.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.265+0000 7f1bcbe00640 1 --2- 172.21.15.100:0/2354316770 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1ba8078b90 0x7f1ba807b0a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc000f60 tx=0x7f1bbc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:09.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.268+0000 7f1bc8c00640 1 -- 172.21.15.100:0/2354316770 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f1bc4016030 con 0x7f1bcc156ca0 2024-09-17T04:25:09.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:09.489+0000 7f1bd1c00640 1 -- 172.21.15.100:0/2354316770 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1bcc0b1410 con 0x7f1bcc156ca0 2024-09-17T04:25:10.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.110+0000 7f1bc8c00640 1 -- 172.21.15.100:0/2354316770 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f1bc40633f0 con 0x7f1bcc156ca0 2024-09-17T04:25:10.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.113+0000 7f1bd1c00640 1 -- 172.21.15.100:0/2354316770 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1ba8078b90 msgr2=0x7f1ba807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:10.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.113+0000 7f1bd1c00640 1 --2- 172.21.15.100:0/2354316770 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1ba8078b90 0x7f1ba807b0a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc000f60 tx=0x7f1bbc009040 comp rx=0 tx=0).stop 2024-09-17T04:25:10.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.113+0000 7f1bd1c00640 1 -- 172.21.15.100:0/2354316770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc156ca0 msgr2=0x7f1bcc1f4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:10.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.113+0000 7f1bd1c00640 1 --2- 172.21.15.100:0/2354316770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc156ca0 0x7f1bcc1f4d30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1bc400e960 tx=0x7f1bc400ee20 comp rx=0 tx=0).stop 2024-09-17T04:25:10.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.113+0000 7f1bd1c00640 1 -- 172.21.15.100:0/2354316770 shutdown_connections 2024-09-17T04:25:10.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.113+0000 7f1bd1c00640 1 --2- 172.21.15.100:0/2354316770 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1ba8078b90 0x7f1ba807b0a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:10.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.113+0000 7f1bd1c00640 1 --2- 172.21.15.100:0/2354316770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc156ca0 0x7f1bcc1f4d30 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:10.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.113+0000 7f1bd1c00640 1 --2- 172.21.15.100:0/2354316770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bcc0cd000 0x7f1bcc1f72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:10.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.113+0000 7f1bd1c00640 1 --2- 172.21.15.100:0/2354316770 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bcc0b3f30 0x7f1bcc1f6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:10.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.113+0000 7f1bd1c00640 1 -- 172.21.15.100:0/2354316770 >> 172.21.15.100:0/2354316770 conn(0x7f1bcc06d620 msgr2=0x7f1bcc15f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:10.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.113+0000 7f1bd1c00640 1 -- 172.21.15.100:0/2354316770 shutdown_connections 2024-09-17T04:25:10.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.113+0000 7f1bd1c00640 1 -- 172.21.15.100:0/2354316770 wait complete. 2024-09-17T04:25:10.139 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T04:25:10.139 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T04:25:10.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.361+0000 7fb7df800640 1 -- 172.21.15.100:0/3367536640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7d80735e0 msgr2=0x7fb7d80739f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:10.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.361+0000 7fb7df800640 1 --2- 172.21.15.100:0/3367536640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7d80735e0 0x7fb7d80739f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb7d815cd30 tx=0x7fb7c402a560 comp rx=0 tx=0).stop 2024-09-17T04:25:10.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.361+0000 7fb7df800640 1 -- 172.21.15.100:0/3367536640 shutdown_connections 2024-09-17T04:25:10.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.361+0000 7fb7df800640 1 --2- 172.21.15.100:0/3367536640 >> v2:172.21.15.184:3300/0 conn(0x7fb7d80cd610 0x7fb7d80cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:10.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.361+0000 7fb7df800640 1 --2- 172.21.15.100:0/3367536640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7d80735e0 0x7fb7d80739f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:10.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.361+0000 7fb7df800640 1 -- 172.21.15.100:0/3367536640 >> 172.21.15.100:0/3367536640 conn(0x7fb7d806d1c0 msgr2=0x7fb7d806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:10.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.362+0000 7fb7df800640 1 -- 172.21.15.100:0/3367536640 shutdown_connections 2024-09-17T04:25:10.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.362+0000 7fb7df800640 1 -- 172.21.15.100:0/3367536640 wait complete. 2024-09-17T04:25:10.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.364+0000 7fb7df800640 1 Processor -- start 2024-09-17T04:25:10.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.364+0000 7fb7df800640 1 -- start start 2024-09-17T04:25:10.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.365+0000 7fb7df800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7d80735e0 0x7fb7d80cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:10.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.365+0000 7fb7df800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7d80cd610 0x7fb7d80b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:10.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.365+0000 7fb7df800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7d8145450 0x7fb7d80b1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:10.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.365+0000 7fb7df800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d80ce2f0 con 0x7fb7d80cd610 2024-09-17T04:25:10.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.365+0000 7fb7df800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d80ce170 con 0x7fb7d80735e0 2024-09-17T04:25:10.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.365+0000 7fb7df800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb7d80ce470 con 0x7fb7d8145450 2024-09-17T04:25:10.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.366+0000 7fb7d7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7d80cd610 0x7fb7d80b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:10.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.366+0000 7fb7d7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7d80cd610 0x7fb7d80b0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38866/0 (socket says 172.21.15.100:38866) 2024-09-17T04:25:10.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.366+0000 7fb7d7e00640 1 -- 172.21.15.100:0/3686411285 learned_addr learned my addr 172.21.15.100:0/3686411285 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:10.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.366+0000 7fb7dd800640 1 --2- 172.21.15.100:0/3686411285 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7d8145450 0x7fb7d80b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:10.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.366+0000 7fb7dce00640 1 --2- 172.21.15.100:0/3686411285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7d80735e0 0x7fb7d80cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:10.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.366+0000 7fb7d7e00640 1 -- 172.21.15.100:0/3686411285 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7d8145450 msgr2=0x7fb7d80b1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:10.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.366+0000 7fb7d7e00640 1 --2- 172.21.15.100:0/3686411285 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7d8145450 0x7fb7d80b1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:10.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.366+0000 7fb7d7e00640 1 -- 172.21.15.100:0/3686411285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7d80735e0 msgr2=0x7fb7d80cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:10.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.366+0000 7fb7d7e00640 1 --2- 172.21.15.100:0/3686411285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7d80735e0 0x7fb7d80cbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:10.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.366+0000 7fb7d7e00640 1 -- 172.21.15.100:0/3686411285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d8152190 con 0x7fb7d80cd610 2024-09-17T04:25:10.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.366+0000 7fb7dd800640 1 --2- 172.21.15.100:0/3686411285 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7d8145450 0x7fb7d80b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:10.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.366+0000 7fb7dce00640 1 --2- 172.21.15.100:0/3686411285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7d80735e0 0x7fb7d80cbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:10.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.367+0000 7fb7d7e00640 1 --2- 172.21.15.100:0/3686411285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7d80cd610 0x7fb7d80b0b00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4034040 tx=0x7fb7c40039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:10.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.367+0000 7fb7d4c00640 1 -- 172.21.15.100:0/3686411285 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c403e030 con 0x7fb7d80cd610 2024-09-17T04:25:10.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.367+0000 7fb7d4c00640 1 -- 172.21.15.100:0/3686411285 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7c40047a0 con 0x7fb7d80cd610 2024-09-17T04:25:10.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.367+0000 7fb7df800640 1 -- 172.21.15.100:0/3686411285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7d8152470 con 0x7fb7d80cd610 2024-09-17T04:25:10.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.367+0000 7fb7d4c00640 1 -- 172.21.15.100:0/3686411285 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4035040 con 0x7fb7d80cd610 2024-09-17T04:25:10.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.367+0000 7fb7df800640 1 -- 172.21.15.100:0/3686411285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7d8152a30 con 0x7fb7d80cd610 2024-09-17T04:25:10.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.368+0000 7fb7df800640 1 -- 172.21.15.100:0/3686411285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7d8151f00 con 0x7fb7d80cd610 2024-09-17T04:25:10.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.369+0000 7fb7d4c00640 1 -- 172.21.15.100:0/3686411285 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb7c4041030 con 0x7fb7d80cd610 2024-09-17T04:25:10.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.370+0000 7fb7d4c00640 1 --2- 172.21.15.100:0/3686411285 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb7b8078b90 0x7fb7b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:10.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.371+0000 7fb7d4c00640 1 -- 172.21.15.100:0/3686411285 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fb7c40bb5e0 con 0x7fb7d80cd610 2024-09-17T04:25:10.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.371+0000 7fb7dce00640 1 --2- 172.21.15.100:0/3686411285 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb7b8078b90 0x7fb7b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:10.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.372+0000 7fb7dce00640 1 --2- 172.21.15.100:0/3686411285 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb7b8078b90 0x7fb7b807b0a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc006fd0 tx=0x7fb7cc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:10.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.375+0000 7fb7d4c00640 1 -- 172.21.15.100:0/3686411285 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb7d8073f30 con 0x7fb7d80cd610 2024-09-17T04:25:10.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:10.593+0000 7fb7df800640 1 -- 172.21.15.100:0/3686411285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb7d80739f0 con 0x7fb7d80cd610 2024-09-17T04:25:11.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.121+0000 7fb7d4c00640 1 -- 172.21.15.100:0/3686411285 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7fb7c4083aa0 con 0x7fb7d80cd610 2024-09-17T04:25:11.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.124+0000 7fb7df800640 1 -- 172.21.15.100:0/3686411285 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb7b8078b90 msgr2=0x7fb7b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:11.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.124+0000 7fb7df800640 1 --2- 172.21.15.100:0/3686411285 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb7b8078b90 0x7fb7b807b0a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc006fd0 tx=0x7fb7cc008040 comp rx=0 tx=0).stop 2024-09-17T04:25:11.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.124+0000 7fb7df800640 1 -- 172.21.15.100:0/3686411285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7d80cd610 msgr2=0x7fb7d80b0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:11.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.124+0000 7fb7df800640 1 --2- 172.21.15.100:0/3686411285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7d80cd610 0x7fb7d80b0b00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4034040 tx=0x7fb7c40039e0 comp rx=0 tx=0).stop 2024-09-17T04:25:11.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.124+0000 7fb7df800640 1 -- 172.21.15.100:0/3686411285 shutdown_connections 2024-09-17T04:25:11.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.124+0000 7fb7df800640 1 --2- 172.21.15.100:0/3686411285 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb7b8078b90 0x7fb7b807b0a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:11.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.124+0000 7fb7df800640 1 --2- 172.21.15.100:0/3686411285 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7d8145450 0x7fb7d80b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:11.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.124+0000 7fb7df800640 1 --2- 172.21.15.100:0/3686411285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7d80cd610 0x7fb7d80b0b00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:11.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.124+0000 7fb7df800640 1 --2- 172.21.15.100:0/3686411285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7d80735e0 0x7fb7d80cbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:11.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.124+0000 7fb7df800640 1 -- 172.21.15.100:0/3686411285 >> 172.21.15.100:0/3686411285 conn(0x7fb7d806d1c0 msgr2=0x7fb7d80b1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:11.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.124+0000 7fb7df800640 1 -- 172.21.15.100:0/3686411285 shutdown_connections 2024-09-17T04:25:11.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.124+0000 7fb7df800640 1 -- 172.21.15.100:0/3686411285 wait complete. 2024-09-17T04:25:11.147 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T04:25:11.148 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:25:11.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.370+0000 7f4ceca00640 1 -- 172.21.15.100:0/2466231271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ce80ad0e0 msgr2=0x7f4ce80ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:11.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.370+0000 7f4ceca00640 1 --2- 172.21.15.100:0/2466231271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ce80ad0e0 0x7f4ce80ad4f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009830 tx=0x7f4cd002a480 comp rx=0 tx=0).stop 2024-09-17T04:25:11.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.371+0000 7f4ceca00640 1 -- 172.21.15.100:0/2466231271 shutdown_connections 2024-09-17T04:25:11.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.371+0000 7f4ceca00640 1 --2- 172.21.15.100:0/2466231271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ce80ad0e0 0x7f4ce80ad4f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:11.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.371+0000 7f4ceca00640 1 --2- 172.21.15.100:0/2466231271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ce815d760 0x7f4ce81535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:11.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.371+0000 7f4ceca00640 1 -- 172.21.15.100:0/2466231271 >> 172.21.15.100:0/2466231271 conn(0x7f4ce806d730 msgr2=0x7f4ce80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:11.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.371+0000 7f4ceca00640 1 -- 172.21.15.100:0/2466231271 shutdown_connections 2024-09-17T04:25:11.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.371+0000 7f4ceca00640 1 -- 172.21.15.100:0/2466231271 wait complete. 2024-09-17T04:25:11.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.373+0000 7f4ceca00640 1 Processor -- start 2024-09-17T04:25:11.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.373+0000 7f4ceca00640 1 -- start start 2024-09-17T04:25:11.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.374+0000 7f4ceca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ce80ad0e0 0x7f4ce814d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:11.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.375+0000 7f4ceca00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ce815d760 0x7f4ce81f91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:11.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.375+0000 7f4ceca00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ce8160310 0x7f4ce81f96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:11.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.375+0000 7f4ceca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce806f220 con 0x7f4ce80ad0e0 2024-09-17T04:25:11.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.375+0000 7f4ceca00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce806f0a0 con 0x7f4ce815d760 2024-09-17T04:25:11.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.375+0000 7f4ceca00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4ce80b4c20 con 0x7f4ce8160310 2024-09-17T04:25:11.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.375+0000 7f4ce5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ce80ad0e0 0x7f4ce814d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:11.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.375+0000 7f4ce5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ce80ad0e0 0x7f4ce814d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38876/0 (socket says 172.21.15.100:38876) 2024-09-17T04:25:11.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.375+0000 7f4ce5e00640 1 -- 172.21.15.100:0/3383077209 learned_addr learned my addr 172.21.15.100:0/3383077209 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:11.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.375+0000 7f4cdd400640 1 --2- 172.21.15.100:0/3383077209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ce815d760 0x7f4ce81f91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:11.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.375+0000 7f4ce5e00640 1 -- 172.21.15.100:0/3383077209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ce8160310 msgr2=0x7f4ce81f96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:11.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.375+0000 7f4ce6800640 1 --2- 172.21.15.100:0/3383077209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ce8160310 0x7f4ce81f96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:11.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.375+0000 7f4ce5e00640 1 --2- 172.21.15.100:0/3383077209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ce8160310 0x7f4ce81f96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:11.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.375+0000 7f4ce5e00640 1 -- 172.21.15.100:0/3383077209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ce815d760 msgr2=0x7f4ce81f91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:11.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.375+0000 7f4ce5e00640 1 --2- 172.21.15.100:0/3383077209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ce815d760 0x7f4ce81f91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:11.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.375+0000 7f4ce5e00640 1 -- 172.21.15.100:0/3383077209 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce80b1690 con 0x7f4ce80ad0e0 2024-09-17T04:25:11.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.375+0000 7f4cdd400640 1 --2- 172.21.15.100:0/3383077209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ce815d760 0x7f4ce81f91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:25:11.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.375+0000 7f4ce6800640 1 --2- 172.21.15.100:0/3383077209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ce8160310 0x7f4ce81f96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:25:11.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.376+0000 7f4ce5e00640 1 --2- 172.21.15.100:0/3383077209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ce80ad0e0 0x7f4ce814d960 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4cd800e7c0 tx=0x7f4cd800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:11.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.376+0000 7f4cdea00640 1 -- 172.21.15.100:0/3383077209 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd800ccc0 con 0x7f4ce80ad0e0 2024-09-17T04:25:11.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.376+0000 7f4cdea00640 1 -- 172.21.15.100:0/3383077209 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4cd8010430 con 0x7f4ce80ad0e0 2024-09-17T04:25:11.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.376+0000 7f4ceca00640 1 -- 172.21.15.100:0/3383077209 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ce814e4d0 con 0x7f4ce80ad0e0 2024-09-17T04:25:11.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.376+0000 7f4cdea00640 1 -- 172.21.15.100:0/3383077209 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8010860 con 0x7f4ce80ad0e0 2024-09-17T04:25:11.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.376+0000 7f4ceca00640 1 -- 172.21.15.100:0/3383077209 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ce814ea90 con 0x7f4ce80ad0e0 2024-09-17T04:25:11.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.377+0000 7f4ceca00640 1 -- 172.21.15.100:0/3383077209 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ce80d0c30 con 0x7f4ce80ad0e0 2024-09-17T04:25:11.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.378+0000 7f4cdea00640 1 -- 172.21.15.100:0/3383077209 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4cd8010a00 con 0x7f4ce80ad0e0 2024-09-17T04:25:11.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.379+0000 7f4cdea00640 1 --2- 172.21.15.100:0/3383077209 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4cc4078bc0 0x7f4cc407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:11.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.379+0000 7f4cdea00640 1 -- 172.21.15.100:0/3383077209 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f4cd809b760 con 0x7f4ce80ad0e0 2024-09-17T04:25:11.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.380+0000 7f4cdd400640 1 --2- 172.21.15.100:0/3383077209 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4cc4078bc0 0x7f4cc407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:11.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.380+0000 7f4cdd400640 1 --2- 172.21.15.100:0/3383077209 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4cc4078bc0 0x7f4cc407b0d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0034000 tx=0x7f4cd0004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:11.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.384+0000 7f4cdea00640 1 -- 172.21.15.100:0/3383077209 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4ce80d0c30 con 0x7f4ce80ad0e0 2024-09-17T04:25:11.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.602+0000 7f4ceca00640 1 -- 172.21.15.100:0/3383077209 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ce8160fa0 con 0x7f4ce80ad0e0 2024-09-17T04:25:11.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.603+0000 7f4cdea00640 1 -- 172.21.15.100:0/3383077209 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5232 (secure 0 0 0) 0x7f4cd8063b90 con 0x7f4ce80ad0e0 2024-09-17T04:25:11.604 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:11.604 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":17,"btime":"2024-09-17T04:25:11:112457+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4667,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2259943840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":2259943840},{"type":"v1","addr":"172.21.15.100:6835","nonce":2259943840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4670,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/1748167554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1748167554},{"type":"v1","addr":"172.21.15.100:6834","nonce":1748167554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4854,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3159351871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":3159351871},{"type":"v1","addr":"172.21.15.184:6836","nonce":3159351871}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:07.565710+0000","modified":"2024-09-17T04:25:10.595342+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4857},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4857":{"gid":4857,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/294839954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":294839954},{"type":"v1","addr":"172.21.15.184:6837","nonce":294839954}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4857,"qdb_cluster":[4857]},"id":2}]} 2024-09-17T04:25:11.604 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 17 2024-09-17T04:25:11.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.605+0000 7f4ceca00640 1 -- 172.21.15.100:0/3383077209 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4cc4078bc0 msgr2=0x7f4cc407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:11.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.605+0000 7f4ceca00640 1 --2- 172.21.15.100:0/3383077209 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4cc4078bc0 0x7f4cc407b0d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0034000 tx=0x7f4cd0004730 comp rx=0 tx=0).stop 2024-09-17T04:25:11.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.605+0000 7f4ceca00640 1 -- 172.21.15.100:0/3383077209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ce80ad0e0 msgr2=0x7f4ce814d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:11.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.605+0000 7f4ceca00640 1 --2- 172.21.15.100:0/3383077209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ce80ad0e0 0x7f4ce814d960 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4cd800e7c0 tx=0x7f4cd800ec80 comp rx=0 tx=0).stop 2024-09-17T04:25:11.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.606+0000 7f4ceca00640 1 -- 172.21.15.100:0/3383077209 shutdown_connections 2024-09-17T04:25:11.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.606+0000 7f4ceca00640 1 --2- 172.21.15.100:0/3383077209 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4cc4078bc0 0x7f4cc407b0d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:11.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.606+0000 7f4ceca00640 1 --2- 172.21.15.100:0/3383077209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ce8160310 0x7f4ce81f96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:11.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.606+0000 7f4ceca00640 1 --2- 172.21.15.100:0/3383077209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ce815d760 0x7f4ce81f91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:11.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.606+0000 7f4ceca00640 1 --2- 172.21.15.100:0/3383077209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ce80ad0e0 0x7f4ce814d960 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:11.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.606+0000 7f4ceca00640 1 -- 172.21.15.100:0/3383077209 >> 172.21.15.100:0/3383077209 conn(0x7f4ce806d730 msgr2=0x7f4ce80cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:11.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.606+0000 7f4ceca00640 1 -- 172.21.15.100:0/3383077209 shutdown_connections 2024-09-17T04:25:11.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.606+0000 7f4ceca00640 1 -- 172.21.15.100:0/3383077209 wait complete. 2024-09-17T04:25:11.630 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:25:11.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.847+0000 7fbc07200640 1 -- 172.21.15.100:0/597943516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc000b57a0 msgr2=0x7fbc000b5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:11.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.847+0000 7fbc07200640 1 --2- 172.21.15.100:0/597943516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc000b57a0 0x7fbc000b5bb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8005fd0 tx=0x7fbbf802aa00 comp rx=0 tx=0).stop 2024-09-17T04:25:11.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.848+0000 7fbc07200640 1 -- 172.21.15.100:0/597943516 shutdown_connections 2024-09-17T04:25:11.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.848+0000 7fbc07200640 1 --2- 172.21.15.100:0/597943516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc000b57a0 0x7fbc000b5bb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:11.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.848+0000 7fbc07200640 1 --2- 172.21.15.100:0/597943516 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc000b7330 0x7fbc000b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:11.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.848+0000 7fbc07200640 1 -- 172.21.15.100:0/597943516 >> 172.21.15.100:0/597943516 conn(0x7fbc000cee90 msgr2=0x7fbc000cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:11.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.848+0000 7fbc07200640 1 -- 172.21.15.100:0/597943516 shutdown_connections 2024-09-17T04:25:11.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.848+0000 7fbc07200640 1 -- 172.21.15.100:0/597943516 wait complete. 2024-09-17T04:25:11.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.850+0000 7fbc07200640 1 Processor -- start 2024-09-17T04:25:11.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.850+0000 7fbc07200640 1 -- start start 2024-09-17T04:25:11.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.851+0000 7fbc07200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc0009ba60 0x7fbc0015e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:11.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.851+0000 7fbc07200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc000b57a0 0x7fbc001f6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:11.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.851+0000 7fbc07200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc000b7330 0x7fbc001f6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:11.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.851+0000 7fbc07200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbc000d1550 con 0x7fbc000b7330 2024-09-17T04:25:11.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.851+0000 7fbc07200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbc000d13d0 con 0x7fbc000b57a0 2024-09-17T04:25:11.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.851+0000 7fbc07200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbc001670e0 con 0x7fbc0009ba60 2024-09-17T04:25:11.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.852+0000 7fbc05200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc000b7330 0x7fbc001f6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:11.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.852+0000 7fbc05200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc000b7330 0x7fbc001f6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38898/0 (socket says 172.21.15.100:38898) 2024-09-17T04:25:11.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.852+0000 7fbc05200640 1 -- 172.21.15.100:0/3281310075 learned_addr learned my addr 172.21.15.100:0/3281310075 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:11.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.852+0000 7fbbffe00640 1 --2- 172.21.15.100:0/3281310075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc0009ba60 0x7fbc0015e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:11.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.852+0000 7fbbf7400640 1 --2- 172.21.15.100:0/3281310075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc000b57a0 0x7fbc001f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:11.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.852+0000 7fbc05200640 1 -- 172.21.15.100:0/3281310075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc0009ba60 msgr2=0x7fbc0015e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:11.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.852+0000 7fbc05200640 1 --2- 172.21.15.100:0/3281310075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc0009ba60 0x7fbc0015e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:11.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.852+0000 7fbc05200640 1 -- 172.21.15.100:0/3281310075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc000b57a0 msgr2=0x7fbc001f6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:11.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.852+0000 7fbc05200640 1 --2- 172.21.15.100:0/3281310075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc000b57a0 0x7fbc001f6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:11.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.852+0000 7fbc05200640 1 -- 172.21.15.100:0/3281310075 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc001f6ee0 con 0x7fbc000b7330 2024-09-17T04:25:11.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.853+0000 7fbc05200640 1 --2- 172.21.15.100:0/3281310075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc000b7330 0x7fbc001f6640 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009d90 tx=0x7fbbf8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:11.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.853+0000 7fbbfcc00640 1 -- 172.21.15.100:0/3281310075 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8033050 con 0x7fbc000b7330 2024-09-17T04:25:11.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.853+0000 7fbbfcc00640 1 -- 172.21.15.100:0/3281310075 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbf8004450 con 0x7fbc000b7330 2024-09-17T04:25:11.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.853+0000 7fbc07200640 1 -- 172.21.15.100:0/3281310075 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc0006e340 con 0x7fbc000b7330 2024-09-17T04:25:11.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.853+0000 7fbbfcc00640 1 -- 172.21.15.100:0/3281310075 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8038450 con 0x7fbc000b7330 2024-09-17T04:25:11.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.853+0000 7fbc07200640 1 -- 172.21.15.100:0/3281310075 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc0006e900 con 0x7fbc000b7330 2024-09-17T04:25:11.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.854+0000 7fbc07200640 1 -- 172.21.15.100:0/3281310075 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc000b5bb0 con 0x7fbc000b7330 2024-09-17T04:25:11.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.855+0000 7fbbfcc00640 1 -- 172.21.15.100:0/3281310075 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbbf8036070 con 0x7fbc000b7330 2024-09-17T04:25:11.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.856+0000 7fbbfcc00640 1 --2- 172.21.15.100:0/3281310075 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbbdc078b90 0x7fbbdc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:11.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.856+0000 7fbbfcc00640 1 -- 172.21.15.100:0/3281310075 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fbbf80bc180 con 0x7fbc000b7330 2024-09-17T04:25:11.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.856+0000 7fbbffe00640 1 --2- 172.21.15.100:0/3281310075 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbbdc078b90 0x7fbbdc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:11.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.857+0000 7fbbffe00640 1 --2- 172.21.15.100:0/3281310075 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbbdc078b90 0x7fbbdc07b0a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbbf00098c0 tx=0x7fbbf0009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:11.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:11.861+0000 7fbbfcc00640 1 -- 172.21.15.100:0/3281310075 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fbbf8083ee0 con 0x7fbc000b7330 2024-09-17T04:25:12.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.043+0000 7fbc07200640 1 -- 172.21.15.100:0/3281310075 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc001f6c70 con 0x7fbc000b7330 2024-09-17T04:25:12.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.045+0000 7fbbfcc00640 1 -- 172.21.15.100:0/3281310075 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16777 (secure 0 0 0) 0x7fbbf8088910 con 0x7fbc000b7330 2024-09-17T04:25:12.045 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:12.046 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":36,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:25:09.097160+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:25:05.694905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T04:25:06.533348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:25:12.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.047+0000 7fbc07200640 1 -- 172.21.15.100:0/3281310075 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbbdc078b90 msgr2=0x7fbbdc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:12.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.047+0000 7fbc07200640 1 --2- 172.21.15.100:0/3281310075 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbbdc078b90 0x7fbbdc07b0a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbbf00098c0 tx=0x7fbbf0009060 comp rx=0 tx=0).stop 2024-09-17T04:25:12.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.047+0000 7fbc07200640 1 -- 172.21.15.100:0/3281310075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc000b7330 msgr2=0x7fbc001f6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:12.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.047+0000 7fbc07200640 1 --2- 172.21.15.100:0/3281310075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc000b7330 0x7fbc001f6640 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009d90 tx=0x7fbbf8003f10 comp rx=0 tx=0).stop 2024-09-17T04:25:12.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.047+0000 7fbc07200640 1 -- 172.21.15.100:0/3281310075 shutdown_connections 2024-09-17T04:25:12.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.047+0000 7fbc07200640 1 --2- 172.21.15.100:0/3281310075 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbbdc078b90 0x7fbbdc07b0a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:12.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.047+0000 7fbc07200640 1 --2- 172.21.15.100:0/3281310075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc000b7330 0x7fbc001f6640 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:12.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.047+0000 7fbc07200640 1 --2- 172.21.15.100:0/3281310075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc000b57a0 0x7fbc001f6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:12.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.047+0000 7fbc07200640 1 --2- 172.21.15.100:0/3281310075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc0009ba60 0x7fbc0015e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:12.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.047+0000 7fbc07200640 1 -- 172.21.15.100:0/3281310075 >> 172.21.15.100:0/3281310075 conn(0x7fbc000cee90 msgr2=0x7fbc000b37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:12.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.047+0000 7fbc07200640 1 -- 172.21.15.100:0/3281310075 shutdown_connections 2024-09-17T04:25:12.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.047+0000 7fbc07200640 1 -- 172.21.15.100:0/3281310075 wait complete. 2024-09-17T04:25:12.064 INFO:tasks.ceph.mds.b.smithi184.stderr:2024-09-17T04:25:12.063+0000 7f5f4a000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T04:25:12.072 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T04:25:12.072 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:25:12.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.294+0000 7f9722400640 1 -- 172.21.15.100:0/236488531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f971c0b3160 msgr2=0x7f971c0d24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:12.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.294+0000 7f9722400640 1 --2- 172.21.15.100:0/236488531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f971c0b3160 0x7f971c0d24a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f971c08d440 tx=0x7f970402a560 comp rx=0 tx=0).stop 2024-09-17T04:25:12.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.294+0000 7f9722400640 1 -- 172.21.15.100:0/236488531 shutdown_connections 2024-09-17T04:25:12.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.294+0000 7f9722400640 1 --2- 172.21.15.100:0/236488531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f971c0b3160 0x7f971c0d24a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:12.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.294+0000 7f9722400640 1 --2- 172.21.15.100:0/236488531 >> v2:172.21.15.184:3300/0 conn(0x7f971c0b9e00 0x7f971c162a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:12.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.294+0000 7f9722400640 1 -- 172.21.15.100:0/236488531 >> 172.21.15.100:0/236488531 conn(0x7f971c08bfe0 msgr2=0x7f971c09d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:12.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.294+0000 7f9722400640 1 -- 172.21.15.100:0/236488531 shutdown_connections 2024-09-17T04:25:12.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.294+0000 7f9722400640 1 -- 172.21.15.100:0/236488531 wait complete. 2024-09-17T04:25:12.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.296+0000 7f9722400640 1 Processor -- start 2024-09-17T04:25:12.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.296+0000 7f9722400640 1 -- start start 2024-09-17T04:25:12.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.298+0000 7f9722400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f971c0b3160 0x7f971c1f99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:12.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.298+0000 7f9722400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f971c0b9e00 0x7f971c0c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:12.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.298+0000 7f9722400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f971c0d29e0 0x7f971c1577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:12.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.298+0000 7f9722400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f971c0d44b0 con 0x7f971c0b9e00 2024-09-17T04:25:12.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.298+0000 7f9722400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f971c0d4330 con 0x7f971c0b3160 2024-09-17T04:25:12.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.298+0000 7f9722400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f971c0d4630 con 0x7f971c0d29e0 2024-09-17T04:25:12.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.298+0000 7f9713e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f971c0b9e00 0x7f971c0c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:12.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.298+0000 7f9713e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f971c0b9e00 0x7f971c0c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38924/0 (socket says 172.21.15.100:38924) 2024-09-17T04:25:12.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.298+0000 7f9713e00640 1 -- 172.21.15.100:0/2028941537 learned_addr learned my addr 172.21.15.100:0/2028941537 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:12.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.298+0000 7f9721000640 1 --2- 172.21.15.100:0/2028941537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f971c0b3160 0x7f971c1f99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:12.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.298+0000 7f9721a00640 1 --2- 172.21.15.100:0/2028941537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f971c0d29e0 0x7f971c1577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:12.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.298+0000 7f9713e00640 1 -- 172.21.15.100:0/2028941537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f971c0d29e0 msgr2=0x7f971c1577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:12.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.298+0000 7f9713e00640 1 --2- 172.21.15.100:0/2028941537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f971c0d29e0 0x7f971c1577e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:12.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.298+0000 7f9713e00640 1 -- 172.21.15.100:0/2028941537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f971c0b3160 msgr2=0x7f971c1f99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:12.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.298+0000 7f9713e00640 1 --2- 172.21.15.100:0/2028941537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f971c0b3160 0x7f971c1f99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:12.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.298+0000 7f9713e00640 1 -- 172.21.15.100:0/2028941537 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971c160e90 con 0x7f971c0b9e00 2024-09-17T04:25:12.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.298+0000 7f9721a00640 1 --2- 172.21.15.100:0/2028941537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f971c0d29e0 0x7f971c1577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:25:12.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.299+0000 7f9713e00640 1 --2- 172.21.15.100:0/2028941537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f971c0b9e00 0x7f971c0c99b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9704034040 tx=0x7f97040039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:12.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.299+0000 7f9719600640 1 -- 172.21.15.100:0/2028941537 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970403e030 con 0x7f971c0b9e00 2024-09-17T04:25:12.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.299+0000 7f9719600640 1 -- 172.21.15.100:0/2028941537 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97040047a0 con 0x7f971c0b9e00 2024-09-17T04:25:12.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.299+0000 7f9719600640 1 -- 172.21.15.100:0/2028941537 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704035040 con 0x7f971c0b9e00 2024-09-17T04:25:12.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.299+0000 7f9722400640 1 -- 172.21.15.100:0/2028941537 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f971c0c80b0 con 0x7f971c0b9e00 2024-09-17T04:25:12.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.299+0000 7f9722400640 1 -- 172.21.15.100:0/2028941537 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f971c1f6270 con 0x7f971c0b9e00 2024-09-17T04:25:12.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.300+0000 7f9722400640 1 -- 172.21.15.100:0/2028941537 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f971c0d4b50 con 0x7f971c0b9e00 2024-09-17T04:25:12.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.301+0000 7f9719600640 1 -- 172.21.15.100:0/2028941537 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9704041030 con 0x7f971c0b9e00 2024-09-17T04:25:12.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.302+0000 7f9719600640 1 --2- 172.21.15.100:0/2028941537 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f96f8078bc0 0x7f96f807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:12.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.302+0000 7f9719600640 1 -- 172.21.15.100:0/2028941537 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f97040bb590 con 0x7f971c0b9e00 2024-09-17T04:25:12.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.303+0000 7f9721000640 1 --2- 172.21.15.100:0/2028941537 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f96f8078bc0 0x7f96f807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:12.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.304+0000 7f9721000640 1 --2- 172.21.15.100:0/2028941537 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f96f8078bc0 0x7f96f807b0d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f970c00bb50 tx=0x7f970c00b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:12.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.307+0000 7f9719600640 1 -- 172.21.15.100:0/2028941537 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f97040832e0 con 0x7f971c0b9e00 2024-09-17T04:25:12.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.488+0000 7f9722400640 1 -- 172.21.15.100:0/2028941537 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f971c0b4620 con 0x7f96f8078bc0 2024-09-17T04:25:12.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.494+0000 7f9719600640 1 -- 172.21.15.100:0/2028941537 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251103 (secure 0 0 0) 0x7f971c0b4620 con 0x7f96f8078bc0 2024-09-17T04:25:12.497 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:12.498 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:25:12.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.498+0000 7f9722400640 1 -- 172.21.15.100:0/2028941537 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f96f8078bc0 msgr2=0x7f96f807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:12.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.498+0000 7f9722400640 1 --2- 172.21.15.100:0/2028941537 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f96f8078bc0 0x7f96f807b0d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f970c00bb50 tx=0x7f970c00b2e0 comp rx=0 tx=0).stop 2024-09-17T04:25:12.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.498+0000 7f9722400640 1 -- 172.21.15.100:0/2028941537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f971c0b9e00 msgr2=0x7f971c0c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:12.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.498+0000 7f9722400640 1 --2- 172.21.15.100:0/2028941537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f971c0b9e00 0x7f971c0c99b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9704034040 tx=0x7f97040039e0 comp rx=0 tx=0).stop 2024-09-17T04:25:12.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.498+0000 7f9722400640 1 -- 172.21.15.100:0/2028941537 shutdown_connections 2024-09-17T04:25:12.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.498+0000 7f9722400640 1 --2- 172.21.15.100:0/2028941537 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f96f8078bc0 0x7f96f807b0d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:12.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.498+0000 7f9722400640 1 --2- 172.21.15.100:0/2028941537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f971c0d29e0 0x7f971c1577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:12.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.498+0000 7f9722400640 1 --2- 172.21.15.100:0/2028941537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f971c0b9e00 0x7f971c0c99b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:12.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.498+0000 7f9722400640 1 --2- 172.21.15.100:0/2028941537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f971c0b3160 0x7f971c1f99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:12.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.498+0000 7f9722400640 1 -- 172.21.15.100:0/2028941537 >> 172.21.15.100:0/2028941537 conn(0x7f971c08bfe0 msgr2=0x7f971c0b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:12.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.498+0000 7f9722400640 1 -- 172.21.15.100:0/2028941537 shutdown_connections 2024-09-17T04:25:12.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.498+0000 7f9722400640 1 -- 172.21.15.100:0/2028941537 wait complete. 2024-09-17T04:25:12.520 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-09-17T04:25:11.979707+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591166,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":124,"num_write_kb":1639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94,"ondisk_log_size":94,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":96,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220372,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754348,"statfs":{"total":773094113280,"available":772868452352,"internally_reserved":0,"allocated":4235264,"data_stored":2342523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.910164"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.131931+0000","last_change":"2024-09-17T04:25:08.126623+0000","last_active":"2024-09-17T04:25:08.131931+0000","last_peered":"2024-09-17T04:25:08.131931+0000","last_clean":"2024-09-17T04:25:08.131931+0000","last_became_active":"2024-09-17T04:25:08.126204+0000","last_became_peered":"2024-09-17T04:25:08.126204+0000","last_unstale":"2024-09-17T04:25:08.131931+0000","last_undegraded":"2024-09-17T04:25:08.131931+0000","last_fullsized":"2024-09-17T04:25:08.131931+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:40.098175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.088364+0000","last_change":"2024-09-17T04:25:07.088364+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.088364+0000","last_undegraded":"2024-09-17T04:25:07.088364+0000","last_fullsized":"2024-09-17T04:25:07.088364+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.114386+0000","last_change":"2024-09-17T04:25:08.116641+0000","last_active":"2024-09-17T04:25:09.114386+0000","last_peered":"2024-09-17T04:25:09.114386+0000","last_clean":"2024-09-17T04:25:09.114386+0000","last_became_active":"2024-09-17T04:25:08.115852+0000","last_became_peered":"2024-09-17T04:25:08.115852+0000","last_unstale":"2024-09-17T04:25:09.114386+0000","last_undegraded":"2024-09-17T04:25:09.114386+0000","last_fullsized":"2024-09-17T04:25:09.114386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:46.534540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.132543+0000","last_change":"2024-09-17T04:25:08.127729+0000","last_active":"2024-09-17T04:25:08.132543+0000","last_peered":"2024-09-17T04:25:08.132543+0000","last_clean":"2024-09-17T04:25:08.132543+0000","last_became_active":"2024-09-17T04:25:08.125546+0000","last_became_peered":"2024-09-17T04:25:08.125546+0000","last_unstale":"2024-09-17T04:25:08.132543+0000","last_undegraded":"2024-09-17T04:25:08.132543+0000","last_fullsized":"2024-09-17T04:25:08.132543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:01:06.440885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.122163+0000","last_change":"2024-09-17T04:25:08.113523+0000","last_active":"2024-09-17T04:25:08.122163+0000","last_peered":"2024-09-17T04:25:08.122163+0000","last_clean":"2024-09-17T04:25:08.122163+0000","last_became_active":"2024-09-17T04:25:08.113207+0000","last_became_peered":"2024-09-17T04:25:08.113207+0000","last_unstale":"2024-09-17T04:25:08.122163+0000","last_undegraded":"2024-09-17T04:25:08.122163+0000","last_fullsized":"2024-09-17T04:25:08.122163+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:28.634098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.083605+0000","last_change":"2024-09-17T04:25:07.083605+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.083605+0000","last_undegraded":"2024-09-17T04:25:07.083605+0000","last_fullsized":"2024-09-17T04:25:07.083605+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113556+0000","last_change":"2024-09-17T04:25:08.113471+0000","last_active":"2024-09-17T04:25:09.113556+0000","last_peered":"2024-09-17T04:25:09.113556+0000","last_clean":"2024-09-17T04:25:09.113556+0000","last_became_active":"2024-09-17T04:25:08.113145+0000","last_became_peered":"2024-09-17T04:25:08.113145+0000","last_unstale":"2024-09-17T04:25:09.113556+0000","last_undegraded":"2024-09-17T04:25:09.113556+0000","last_fullsized":"2024-09-17T04:25:09.113556+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:52.755680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112954+0000","last_change":"2024-09-17T04:25:08.102460+0000","last_active":"2024-09-17T04:25:09.112954+0000","last_peered":"2024-09-17T04:25:09.112954+0000","last_clean":"2024-09-17T04:25:09.112954+0000","last_became_active":"2024-09-17T04:25:08.101474+0000","last_became_peered":"2024-09-17T04:25:08.101474+0000","last_unstale":"2024-09-17T04:25:09.112954+0000","last_undegraded":"2024-09-17T04:25:09.112954+0000","last_fullsized":"2024-09-17T04:25:09.112954+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:22.538564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.126318+0000","last_change":"2024-09-17T04:25:08.119819+0000","last_active":"2024-09-17T04:25:08.126318+0000","last_peered":"2024-09-17T04:25:08.126318+0000","last_clean":"2024-09-17T04:25:08.126318+0000","last_became_active":"2024-09-17T04:25:08.119511+0000","last_became_peered":"2024-09-17T04:25:08.119511+0000","last_unstale":"2024-09-17T04:25:08.126318+0000","last_undegraded":"2024-09-17T04:25:08.126318+0000","last_fullsized":"2024-09-17T04:25:08.126318+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:49:01.278096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.124020+0000","last_change":"2024-09-17T04:25:08.121720+0000","last_active":"2024-09-17T04:25:08.124020+0000","last_peered":"2024-09-17T04:25:08.124020+0000","last_clean":"2024-09-17T04:25:08.124020+0000","last_became_active":"2024-09-17T04:25:08.121405+0000","last_became_peered":"2024-09-17T04:25:08.121405+0000","last_unstale":"2024-09-17T04:25:08.124020+0000","last_undegraded":"2024-09-17T04:25:08.124020+0000","last_fullsized":"2024-09-17T04:25:08.124020+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:37.261699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112530+0000","last_change":"2024-09-17T04:25:08.116374+0000","last_active":"2024-09-17T04:25:09.112530+0000","last_peered":"2024-09-17T04:25:09.112530+0000","last_clean":"2024-09-17T04:25:09.112530+0000","last_became_active":"2024-09-17T04:25:08.116047+0000","last_became_peered":"2024-09-17T04:25:08.116047+0000","last_unstale":"2024-09-17T04:25:09.112530+0000","last_undegraded":"2024-09-17T04:25:09.112530+0000","last_fullsized":"2024-09-17T04:25:09.112530+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:12:03.863249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113687+0000","last_change":"2024-09-17T04:25:08.117853+0000","last_active":"2024-09-17T04:25:09.113687+0000","last_peered":"2024-09-17T04:25:09.113687+0000","last_clean":"2024-09-17T04:25:09.113687+0000","last_became_active":"2024-09-17T04:25:08.117342+0000","last_became_peered":"2024-09-17T04:25:08.117342+0000","last_unstale":"2024-09-17T04:25:09.113687+0000","last_undegraded":"2024-09-17T04:25:09.113687+0000","last_fullsized":"2024-09-17T04:25:09.113687+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:59.322143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.118578+0000","last_change":"2024-09-17T04:25:08.107583+0000","last_active":"2024-09-17T04:25:08.118578+0000","last_peered":"2024-09-17T04:25:08.118578+0000","last_clean":"2024-09-17T04:25:08.118578+0000","last_became_active":"2024-09-17T04:25:08.107097+0000","last_became_peered":"2024-09-17T04:25:08.107097+0000","last_unstale":"2024-09-17T04:25:08.118578+0000","last_undegraded":"2024-09-17T04:25:08.118578+0000","last_fullsized":"2024-09-17T04:25:08.118578+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:37.405465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.083133+0000","last_change":"2024-09-17T04:25:07.083133+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.083133+0000","last_undegraded":"2024-09-17T04:25:07.083133+0000","last_fullsized":"2024-09-17T04:25:07.083133+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112471+0000","last_change":"2024-09-17T04:25:08.122562+0000","last_active":"2024-09-17T04:25:09.112471+0000","last_peered":"2024-09-17T04:25:09.112471+0000","last_clean":"2024-09-17T04:25:09.112471+0000","last_became_active":"2024-09-17T04:25:08.120406+0000","last_became_peered":"2024-09-17T04:25:08.120406+0000","last_unstale":"2024-09-17T04:25:09.112471+0000","last_undegraded":"2024-09-17T04:25:09.112471+0000","last_fullsized":"2024-09-17T04:25:09.112471+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:14.660711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113172+0000","last_change":"2024-09-17T04:25:08.111402+0000","last_active":"2024-09-17T04:25:09.113172+0000","last_peered":"2024-09-17T04:25:09.113172+0000","last_clean":"2024-09-17T04:25:09.113172+0000","last_became_active":"2024-09-17T04:25:08.111049+0000","last_became_peered":"2024-09-17T04:25:08.111049+0000","last_unstale":"2024-09-17T04:25:09.113172+0000","last_undegraded":"2024-09-17T04:25:09.113172+0000","last_fullsized":"2024-09-17T04:25:09.113172+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:46.534540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112688+0000","last_change":"2024-09-17T04:25:08.113695+0000","last_active":"2024-09-17T04:25:09.112688+0000","last_peered":"2024-09-17T04:25:09.112688+0000","last_clean":"2024-09-17T04:25:09.112688+0000","last_became_active":"2024-09-17T04:25:08.113248+0000","last_became_peered":"2024-09-17T04:25:08.113248+0000","last_unstale":"2024-09-17T04:25:09.112688+0000","last_undegraded":"2024-09-17T04:25:09.112688+0000","last_fullsized":"2024-09-17T04:25:09.112688+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:52.735017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.087894+0000","last_change":"2024-09-17T04:25:07.087894+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.087894+0000","last_undegraded":"2024-09-17T04:25:07.087894+0000","last_fullsized":"2024-09-17T04:25:07.087894+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.084115+0000","last_change":"2024-09-17T04:25:07.084115+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.084115+0000","last_undegraded":"2024-09-17T04:25:07.084115+0000","last_fullsized":"2024-09-17T04:25:07.084115+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.107540+0000","last_change":"2024-09-17T04:25:08.102248+0000","last_active":"2024-09-17T04:25:08.107540+0000","last_peered":"2024-09-17T04:25:08.107540+0000","last_clean":"2024-09-17T04:25:08.107540+0000","last_became_active":"2024-09-17T04:25:08.101915+0000","last_became_peered":"2024-09-17T04:25:08.101915+0000","last_unstale":"2024-09-17T04:25:08.107540+0000","last_undegraded":"2024-09-17T04:25:08.107540+0000","last_fullsized":"2024-09-17T04:25:08.107540+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:11.459548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.125029+0000","last_change":"2024-09-17T04:25:08.116792+0000","last_active":"2024-09-17T04:25:08.125029+0000","last_peered":"2024-09-17T04:25:08.125029+0000","last_clean":"2024-09-17T04:25:08.125029+0000","last_became_active":"2024-09-17T04:25:08.116468+0000","last_became_peered":"2024-09-17T04:25:08.116468+0000","last_unstale":"2024-09-17T04:25:08.125029+0000","last_undegraded":"2024-09-17T04:25:08.125029+0000","last_fullsized":"2024-09-17T04:25:08.125029+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:01.832590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.132325+0000","last_change":"2024-09-17T04:25:08.125202+0000","last_active":"2024-09-17T04:25:08.132325+0000","last_peered":"2024-09-17T04:25:08.132325+0000","last_clean":"2024-09-17T04:25:08.132325+0000","last_became_active":"2024-09-17T04:25:08.124880+0000","last_became_peered":"2024-09-17T04:25:08.124880+0000","last_unstale":"2024-09-17T04:25:08.132325+0000","last_undegraded":"2024-09-17T04:25:08.132325+0000","last_fullsized":"2024-09-17T04:25:08.132325+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:33.145063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.115053+0000","last_change":"2024-09-17T04:25:08.109076+0000","last_active":"2024-09-17T04:25:08.115053+0000","last_peered":"2024-09-17T04:25:08.115053+0000","last_clean":"2024-09-17T04:25:08.115053+0000","last_became_active":"2024-09-17T04:25:08.105699+0000","last_became_peered":"2024-09-17T04:25:08.105699+0000","last_unstale":"2024-09-17T04:25:08.115053+0000","last_undegraded":"2024-09-17T04:25:08.115053+0000","last_fullsized":"2024-09-17T04:25:08.115053+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:33.145063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.111966+0000","last_change":"2024-09-17T04:25:08.099256+0000","last_active":"2024-09-17T04:25:09.111966+0000","last_peered":"2024-09-17T04:25:09.111966+0000","last_clean":"2024-09-17T04:25:09.111966+0000","last_became_active":"2024-09-17T04:25:08.098874+0000","last_became_peered":"2024-09-17T04:25:08.098874+0000","last_unstale":"2024-09-17T04:25:09.111966+0000","last_undegraded":"2024-09-17T04:25:09.111966+0000","last_fullsized":"2024-09-17T04:25:09.111966+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:11.397751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.122652+0000","last_change":"2024-09-17T04:25:08.117901+0000","last_active":"2024-09-17T04:25:08.122652+0000","last_peered":"2024-09-17T04:25:08.122652+0000","last_clean":"2024-09-17T04:25:08.122652+0000","last_became_active":"2024-09-17T04:25:08.115468+0000","last_became_peered":"2024-09-17T04:25:08.115468+0000","last_unstale":"2024-09-17T04:25:08.122652+0000","last_undegraded":"2024-09-17T04:25:08.122652+0000","last_fullsized":"2024-09-17T04:25:08.122652+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:06.724919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.085167+0000","last_change":"2024-09-17T04:25:07.085167+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.085167+0000","last_undegraded":"2024-09-17T04:25:07.085167+0000","last_fullsized":"2024-09-17T04:25:07.085167+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.129679+0000","last_change":"2024-09-17T04:25:08.126392+0000","last_active":"2024-09-17T04:25:08.129679+0000","last_peered":"2024-09-17T04:25:08.129679+0000","last_clean":"2024-09-17T04:25:08.129679+0000","last_became_active":"2024-09-17T04:25:08.126046+0000","last_became_peered":"2024-09-17T04:25:08.126046+0000","last_unstale":"2024-09-17T04:25:08.129679+0000","last_undegraded":"2024-09-17T04:25:08.129679+0000","last_fullsized":"2024-09-17T04:25:08.129679+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:30.659131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113961+0000","last_change":"2024-09-17T04:25:08.104557+0000","last_active":"2024-09-17T04:25:09.113961+0000","last_peered":"2024-09-17T04:25:09.113961+0000","last_clean":"2024-09-17T04:25:09.113961+0000","last_became_active":"2024-09-17T04:25:08.104246+0000","last_became_peered":"2024-09-17T04:25:08.104246+0000","last_unstale":"2024-09-17T04:25:09.113961+0000","last_undegraded":"2024-09-17T04:25:09.113961+0000","last_fullsized":"2024-09-17T04:25:09.113961+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:26.289675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.132086+0000","last_change":"2024-09-17T04:25:08.124218+0000","last_active":"2024-09-17T04:25:08.132086+0000","last_peered":"2024-09-17T04:25:08.132086+0000","last_clean":"2024-09-17T04:25:08.132086+0000","last_became_active":"2024-09-17T04:25:08.123746+0000","last_became_peered":"2024-09-17T04:25:08.123746+0000","last_unstale":"2024-09-17T04:25:08.132086+0000","last_undegraded":"2024-09-17T04:25:08.132086+0000","last_fullsized":"2024-09-17T04:25:08.132086+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:02.874046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112282+0000","last_change":"2024-09-17T04:25:08.113465+0000","last_active":"2024-09-17T04:25:09.112282+0000","last_peered":"2024-09-17T04:25:09.112282+0000","last_clean":"2024-09-17T04:25:09.112282+0000","last_became_active":"2024-09-17T04:25:08.113106+0000","last_became_peered":"2024-09-17T04:25:08.113106+0000","last_unstale":"2024-09-17T04:25:09.112282+0000","last_undegraded":"2024-09-17T04:25:09.112282+0000","last_fullsized":"2024-09-17T04:25:09.112282+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:14.660711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.120556+0000","last_change":"2024-09-17T04:25:08.111799+0000","last_active":"2024-09-17T04:25:08.120556+0000","last_peered":"2024-09-17T04:25:08.120556+0000","last_clean":"2024-09-17T04:25:08.120556+0000","last_became_active":"2024-09-17T04:25:08.111338+0000","last_became_peered":"2024-09-17T04:25:08.111338+0000","last_unstale":"2024-09-17T04:25:08.120556+0000","last_undegraded":"2024-09-17T04:25:08.120556+0000","last_fullsized":"2024-09-17T04:25:08.120556+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:15.918939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112824+0000","last_change":"2024-09-17T04:25:08.109129+0000","last_active":"2024-09-17T04:25:09.112824+0000","last_peered":"2024-09-17T04:25:09.112824+0000","last_clean":"2024-09-17T04:25:09.112824+0000","last_became_active":"2024-09-17T04:25:08.104873+0000","last_became_peered":"2024-09-17T04:25:08.104873+0000","last_unstale":"2024-09-17T04:25:09.112824+0000","last_undegraded":"2024-09-17T04:25:09.112824+0000","last_fullsized":"2024-09-17T04:25:09.112824+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:48.676959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.112046+0000","last_change":"2024-09-17T04:25:08.104171+0000","last_active":"2024-09-17T04:25:08.112046+0000","last_peered":"2024-09-17T04:25:08.112046+0000","last_clean":"2024-09-17T04:25:08.112046+0000","last_became_active":"2024-09-17T04:25:08.103831+0000","last_became_peered":"2024-09-17T04:25:08.103831+0000","last_unstale":"2024-09-17T04:25:08.112046+0000","last_undegraded":"2024-09-17T04:25:08.112046+0000","last_fullsized":"2024-09-17T04:25:08.112046+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:02.874046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.132361+0000","last_change":"2024-09-17T04:25:07.101542+0000","last_active":"2024-09-17T04:25:08.132361+0000","last_peered":"2024-09-17T04:25:08.132361+0000","last_clean":"2024-09-17T04:25:08.132361+0000","last_became_active":"2024-09-17T04:25:07.101168+0000","last_became_peered":"2024-09-17T04:25:07.101168+0000","last_unstale":"2024-09-17T04:25:08.132361+0000","last_undegraded":"2024-09-17T04:25:08.132361+0000","last_fullsized":"2024-09-17T04:25:08.132361+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:05.174227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.127811+0000","last_change":"2024-09-17T04:25:08.122307+0000","last_active":"2024-09-17T04:25:08.127811+0000","last_peered":"2024-09-17T04:25:08.127811+0000","last_clean":"2024-09-17T04:25:08.127811+0000","last_became_active":"2024-09-17T04:25:08.121816+0000","last_became_peered":"2024-09-17T04:25:08.121816+0000","last_unstale":"2024-09-17T04:25:08.127811+0000","last_undegraded":"2024-09-17T04:25:08.127811+0000","last_fullsized":"2024-09-17T04:25:08.127811+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:06:03.732941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.121882+0000","last_change":"2024-09-17T04:25:07.090513+0000","last_active":"2024-09-17T04:25:08.121882+0000","last_peered":"2024-09-17T04:25:08.121882+0000","last_clean":"2024-09-17T04:25:08.121882+0000","last_became_active":"2024-09-17T04:25:07.090181+0000","last_became_peered":"2024-09-17T04:25:07.090181+0000","last_unstale":"2024-09-17T04:25:08.121882+0000","last_undegraded":"2024-09-17T04:25:08.121882+0000","last_fullsized":"2024-09-17T04:25:08.121882+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:00.826540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.126484+0000","last_change":"2024-09-17T04:25:08.118894+0000","last_active":"2024-09-17T04:25:08.126484+0000","last_peered":"2024-09-17T04:25:08.126484+0000","last_clean":"2024-09-17T04:25:08.126484+0000","last_became_active":"2024-09-17T04:25:08.118563+0000","last_became_peered":"2024-09-17T04:25:08.118563+0000","last_unstale":"2024-09-17T04:25:08.126484+0000","last_undegraded":"2024-09-17T04:25:08.126484+0000","last_fullsized":"2024-09-17T04:25:08.126484+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:36.885529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T04:25:07.106875+0000","last_change":"2024-09-17T04:25:07.103013+0000","last_active":"2024-09-17T04:25:07.106875+0000","last_peered":"2024-09-17T04:25:07.106875+0000","last_clean":"2024-09-17T04:25:07.106875+0000","last_became_active":"2024-09-17T04:25:07.100422+0000","last_became_peered":"2024-09-17T04:25:07.100422+0000","last_unstale":"2024-09-17T04:25:07.106875+0000","last_undegraded":"2024-09-17T04:25:07.106875+0000","last_fullsized":"2024-09-17T04:25:07.106875+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:46.886234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.128163+0000","last_change":"2024-09-17T04:25:08.125767+0000","last_active":"2024-09-17T04:25:08.128163+0000","last_peered":"2024-09-17T04:25:08.128163+0000","last_clean":"2024-09-17T04:25:08.128163+0000","last_became_active":"2024-09-17T04:25:08.125475+0000","last_became_peered":"2024-09-17T04:25:08.125475+0000","last_unstale":"2024-09-17T04:25:08.128163+0000","last_undegraded":"2024-09-17T04:25:08.128163+0000","last_fullsized":"2024-09-17T04:25:08.128163+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:21.841901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.124680+0000","last_change":"2024-09-17T04:25:07.097308+0000","last_active":"2024-09-17T04:25:08.124680+0000","last_peered":"2024-09-17T04:25:08.124680+0000","last_clean":"2024-09-17T04:25:08.124680+0000","last_became_active":"2024-09-17T04:25:07.096657+0000","last_became_peered":"2024-09-17T04:25:07.096657+0000","last_unstale":"2024-09-17T04:25:08.124680+0000","last_undegraded":"2024-09-17T04:25:08.124680+0000","last_fullsized":"2024-09-17T04:25:08.124680+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:49:55.787331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112543+0000","last_change":"2024-09-17T04:25:08.123602+0000","last_active":"2024-09-17T04:25:09.112543+0000","last_peered":"2024-09-17T04:25:09.112543+0000","last_clean":"2024-09-17T04:25:09.112543+0000","last_became_active":"2024-09-17T04:25:08.121886+0000","last_became_peered":"2024-09-17T04:25:08.121886+0000","last_unstale":"2024-09-17T04:25:09.112543+0000","last_undegraded":"2024-09-17T04:25:09.112543+0000","last_fullsized":"2024-09-17T04:25:09.112543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:59.322143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.123415+0000","last_change":"2024-09-17T04:25:07.093514+0000","last_active":"2024-09-17T04:25:08.123415+0000","last_peered":"2024-09-17T04:25:08.123415+0000","last_clean":"2024-09-17T04:25:08.123415+0000","last_became_active":"2024-09-17T04:25:07.091028+0000","last_became_peered":"2024-09-17T04:25:07.091028+0000","last_unstale":"2024-09-17T04:25:08.123415+0000","last_undegraded":"2024-09-17T04:25:08.123415+0000","last_fullsized":"2024-09-17T04:25:08.123415+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:47:16.719512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113223+0000","last_change":"2024-09-17T04:25:08.109606+0000","last_active":"2024-09-17T04:25:09.113223+0000","last_peered":"2024-09-17T04:25:09.113223+0000","last_clean":"2024-09-17T04:25:09.113223+0000","last_became_active":"2024-09-17T04:25:08.109288+0000","last_became_peered":"2024-09-17T04:25:08.109288+0000","last_unstale":"2024-09-17T04:25:09.113223+0000","last_undegraded":"2024-09-17T04:25:09.113223+0000","last_fullsized":"2024-09-17T04:25:09.113223+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:12:03.863249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.121410+0000","last_change":"2024-09-17T04:25:07.092548+0000","last_active":"2024-09-17T04:25:08.121410+0000","last_peered":"2024-09-17T04:25:08.121410+0000","last_clean":"2024-09-17T04:25:08.121410+0000","last_became_active":"2024-09-17T04:25:07.092222+0000","last_became_peered":"2024-09-17T04:25:07.092222+0000","last_unstale":"2024-09-17T04:25:08.121410+0000","last_undegraded":"2024-09-17T04:25:08.121410+0000","last_fullsized":"2024-09-17T04:25:08.121410+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:12.533380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.126405+0000","last_change":"2024-09-17T04:25:08.120205+0000","last_active":"2024-09-17T04:25:08.126405+0000","last_peered":"2024-09-17T04:25:08.126405+0000","last_clean":"2024-09-17T04:25:08.126405+0000","last_became_active":"2024-09-17T04:25:08.119921+0000","last_became_peered":"2024-09-17T04:25:08.119921+0000","last_unstale":"2024-09-17T04:25:08.126405+0000","last_undegraded":"2024-09-17T04:25:08.126405+0000","last_fullsized":"2024-09-17T04:25:08.126405+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:20.310340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T04:25:07.102921+0000","last_change":"2024-09-17T04:25:07.096137+0000","last_active":"2024-09-17T04:25:07.102921+0000","last_peered":"2024-09-17T04:25:07.102921+0000","last_clean":"2024-09-17T04:25:07.102921+0000","last_became_active":"2024-09-17T04:25:07.095852+0000","last_became_peered":"2024-09-17T04:25:07.095852+0000","last_unstale":"2024-09-17T04:25:07.102921+0000","last_undegraded":"2024-09-17T04:25:07.102921+0000","last_fullsized":"2024-09-17T04:25:07.102921+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:28.250505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.116518+0000","last_change":"2024-09-17T04:25:08.107235+0000","last_active":"2024-09-17T04:25:08.116518+0000","last_peered":"2024-09-17T04:25:08.116518+0000","last_clean":"2024-09-17T04:25:08.116518+0000","last_became_active":"2024-09-17T04:25:08.106868+0000","last_became_peered":"2024-09-17T04:25:08.106868+0000","last_unstale":"2024-09-17T04:25:08.116518+0000","last_undegraded":"2024-09-17T04:25:08.116518+0000","last_fullsized":"2024-09-17T04:25:08.116518+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:11.868935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.100418+0000","last_change":"2024-09-17T04:25:07.090542+0000","last_active":"2024-09-17T04:25:08.100418+0000","last_peered":"2024-09-17T04:25:08.100418+0000","last_clean":"2024-09-17T04:25:08.100418+0000","last_became_active":"2024-09-17T04:25:07.087859+0000","last_became_peered":"2024-09-17T04:25:07.087859+0000","last_unstale":"2024-09-17T04:25:08.100418+0000","last_undegraded":"2024-09-17T04:25:08.100418+0000","last_fullsized":"2024-09-17T04:25:08.100418+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:35.343496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.114713+0000","last_change":"2024-09-17T04:25:08.120860+0000","last_active":"2024-09-17T04:25:09.114713+0000","last_peered":"2024-09-17T04:25:09.114713+0000","last_clean":"2024-09-17T04:25:09.114713+0000","last_became_active":"2024-09-17T04:25:08.119392+0000","last_became_peered":"2024-09-17T04:25:08.119392+0000","last_unstale":"2024-09-17T04:25:09.114713+0000","last_undegraded":"2024-09-17T04:25:09.114713+0000","last_fullsized":"2024-09-17T04:25:09.114713+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:52.755680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T04:25:07.106897+0000","last_change":"2024-09-17T04:25:07.103331+0000","last_active":"2024-09-17T04:25:07.106897+0000","last_peered":"2024-09-17T04:25:07.106897+0000","last_clean":"2024-09-17T04:25:07.106897+0000","last_became_active":"2024-09-17T04:25:07.102248+0000","last_became_peered":"2024-09-17T04:25:07.102248+0000","last_unstale":"2024-09-17T04:25:07.106897+0000","last_undegraded":"2024-09-17T04:25:07.106897+0000","last_fullsized":"2024-09-17T04:25:07.106897+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:20:28.817834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.082595+0000","last_change":"2024-09-17T04:25:07.082595+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.082595+0000","last_undegraded":"2024-09-17T04:25:07.082595+0000","last_fullsized":"2024-09-17T04:25:07.082595+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.094826+0000","last_change":"2024-09-17T04:25:07.097895+0000","last_active":"2024-09-17T04:25:08.094826+0000","last_peered":"2024-09-17T04:25:08.094826+0000","last_clean":"2024-09-17T04:25:08.094826+0000","last_became_active":"2024-09-17T04:25:07.097541+0000","last_became_peered":"2024-09-17T04:25:07.097541+0000","last_unstale":"2024-09-17T04:25:08.094826+0000","last_undegraded":"2024-09-17T04:25:08.094826+0000","last_fullsized":"2024-09-17T04:25:08.094826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:49:00.272046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.088550+0000","last_change":"2024-09-17T04:25:07.088550+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.088550+0000","last_undegraded":"2024-09-17T04:25:07.088550+0000","last_fullsized":"2024-09-17T04:25:07.088550+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.114464+0000","last_change":"2024-09-17T04:25:07.103494+0000","last_active":"2024-09-17T04:25:09.114464+0000","last_peered":"2024-09-17T04:25:09.114464+0000","last_clean":"2024-09-17T04:25:09.114464+0000","last_became_active":"2024-09-17T04:25:07.103202+0000","last_became_peered":"2024-09-17T04:25:07.103202+0000","last_unstale":"2024-09-17T04:25:09.114464+0000","last_undegraded":"2024-09-17T04:25:09.114464+0000","last_fullsized":"2024-09-17T04:25:09.114464+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:33.029608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.114318+0000","last_change":"2024-09-17T04:25:08.117481+0000","last_active":"2024-09-17T04:25:09.114318+0000","last_peered":"2024-09-17T04:25:09.114318+0000","last_clean":"2024-09-17T04:25:09.114318+0000","last_became_active":"2024-09-17T04:25:08.117068+0000","last_became_peered":"2024-09-17T04:25:08.117068+0000","last_unstale":"2024-09-17T04:25:09.114318+0000","last_undegraded":"2024-09-17T04:25:09.114318+0000","last_fullsized":"2024-09-17T04:25:09.114318+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:40.823059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T04:25:07.104317+0000","last_change":"2024-09-17T04:25:07.099760+0000","last_active":"2024-09-17T04:25:07.104317+0000","last_peered":"2024-09-17T04:25:07.104317+0000","last_clean":"2024-09-17T04:25:07.104317+0000","last_became_active":"2024-09-17T04:25:07.098636+0000","last_became_peered":"2024-09-17T04:25:07.098636+0000","last_unstale":"2024-09-17T04:25:07.104317+0000","last_undegraded":"2024-09-17T04:25:07.104317+0000","last_fullsized":"2024-09-17T04:25:07.104317+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:32:49.032315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.116388+0000","last_change":"2024-09-17T04:25:08.105980+0000","last_active":"2024-09-17T04:25:08.116388+0000","last_peered":"2024-09-17T04:25:08.116388+0000","last_clean":"2024-09-17T04:25:08.116388+0000","last_became_active":"2024-09-17T04:25:08.105227+0000","last_became_peered":"2024-09-17T04:25:08.105227+0000","last_unstale":"2024-09-17T04:25:08.116388+0000","last_undegraded":"2024-09-17T04:25:08.116388+0000","last_fullsized":"2024-09-17T04:25:08.116388+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:40.098175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.124024+0000","last_change":"2024-09-17T04:25:07.090197+0000","last_active":"2024-09-17T04:25:08.124024+0000","last_peered":"2024-09-17T04:25:08.124024+0000","last_clean":"2024-09-17T04:25:08.124024+0000","last_became_active":"2024-09-17T04:25:07.089784+0000","last_became_peered":"2024-09-17T04:25:07.089784+0000","last_unstale":"2024-09-17T04:25:08.124024+0000","last_undegraded":"2024-09-17T04:25:08.124024+0000","last_fullsized":"2024-09-17T04:25:08.124024+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:09.843048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.083774+0000","last_change":"2024-09-17T04:25:07.083774+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.083774+0000","last_undegraded":"2024-09-17T04:25:07.083774+0000","last_fullsized":"2024-09-17T04:25:07.083774+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.100774+0000","last_change":"2024-09-17T04:25:07.100954+0000","last_active":"2024-09-17T04:25:08.100774+0000","last_peered":"2024-09-17T04:25:08.100774+0000","last_clean":"2024-09-17T04:25:08.100774+0000","last_became_active":"2024-09-17T04:25:07.100610+0000","last_became_peered":"2024-09-17T04:25:07.100610+0000","last_unstale":"2024-09-17T04:25:08.100774+0000","last_undegraded":"2024-09-17T04:25:08.100774+0000","last_fullsized":"2024-09-17T04:25:08.100774+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:14.912889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113053+0000","last_change":"2024-09-17T04:25:08.117150+0000","last_active":"2024-09-17T04:25:09.113053+0000","last_peered":"2024-09-17T04:25:09.113053+0000","last_clean":"2024-09-17T04:25:09.113053+0000","last_became_active":"2024-09-17T04:25:08.114530+0000","last_became_peered":"2024-09-17T04:25:08.114530+0000","last_unstale":"2024-09-17T04:25:09.113053+0000","last_undegraded":"2024-09-17T04:25:09.113053+0000","last_fullsized":"2024-09-17T04:25:09.113053+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:40.823059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T04:25:07.104424+0000","last_change":"2024-09-17T04:25:07.101663+0000","last_active":"2024-09-17T04:25:07.104424+0000","last_peered":"2024-09-17T04:25:07.104424+0000","last_clean":"2024-09-17T04:25:07.104424+0000","last_became_active":"2024-09-17T04:25:07.100447+0000","last_became_peered":"2024-09-17T04:25:07.100447+0000","last_unstale":"2024-09-17T04:25:07.104424+0000","last_undegraded":"2024-09-17T04:25:07.104424+0000","last_fullsized":"2024-09-17T04:25:07.104424+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:47.815107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.131919+0000","last_change":"2024-09-17T04:25:08.124593+0000","last_active":"2024-09-17T04:25:08.131919+0000","last_peered":"2024-09-17T04:25:08.131919+0000","last_clean":"2024-09-17T04:25:08.131919+0000","last_became_active":"2024-09-17T04:25:08.121257+0000","last_became_peered":"2024-09-17T04:25:08.121257+0000","last_unstale":"2024-09-17T04:25:08.131919+0000","last_undegraded":"2024-09-17T04:25:08.131919+0000","last_fullsized":"2024-09-17T04:25:08.131919+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:02.807745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T04:25:07.106797+0000","last_change":"2024-09-17T04:25:07.101694+0000","last_active":"2024-09-17T04:25:07.106797+0000","last_peered":"2024-09-17T04:25:07.106797+0000","last_clean":"2024-09-17T04:25:07.106797+0000","last_became_active":"2024-09-17T04:25:07.101006+0000","last_became_peered":"2024-09-17T04:25:07.101006+0000","last_unstale":"2024-09-17T04:25:07.106797+0000","last_undegraded":"2024-09-17T04:25:07.106797+0000","last_fullsized":"2024-09-17T04:25:07.106797+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:14:44.879016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.127019+0000","last_change":"2024-09-17T04:25:08.120978+0000","last_active":"2024-09-17T04:25:08.127019+0000","last_peered":"2024-09-17T04:25:08.127019+0000","last_clean":"2024-09-17T04:25:08.127019+0000","last_became_active":"2024-09-17T04:25:08.118804+0000","last_became_peered":"2024-09-17T04:25:08.118804+0000","last_unstale":"2024-09-17T04:25:08.127019+0000","last_undegraded":"2024-09-17T04:25:08.127019+0000","last_fullsized":"2024-09-17T04:25:08.127019+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:41.743123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.124335+0000","last_change":"2024-09-17T04:25:07.100101+0000","last_active":"2024-09-17T04:25:08.124335+0000","last_peered":"2024-09-17T04:25:08.124335+0000","last_clean":"2024-09-17T04:25:08.124335+0000","last_became_active":"2024-09-17T04:25:07.099657+0000","last_became_peered":"2024-09-17T04:25:07.099657+0000","last_unstale":"2024-09-17T04:25:08.124335+0000","last_undegraded":"2024-09-17T04:25:08.124335+0000","last_fullsized":"2024-09-17T04:25:08.124335+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:06:02.726891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113955+0000","last_change":"2024-09-17T04:25:08.123388+0000","last_active":"2024-09-17T04:25:09.113955+0000","last_peered":"2024-09-17T04:25:09.113955+0000","last_clean":"2024-09-17T04:25:09.113955+0000","last_became_active":"2024-09-17T04:25:08.122769+0000","last_became_peered":"2024-09-17T04:25:08.122769+0000","last_unstale":"2024-09-17T04:25:09.113955+0000","last_undegraded":"2024-09-17T04:25:09.113955+0000","last_fullsized":"2024-09-17T04:25:09.113955+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:52.735017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.118988+0000","last_change":"2024-09-17T04:25:07.089581+0000","last_active":"2024-09-17T04:25:08.118988+0000","last_peered":"2024-09-17T04:25:08.118988+0000","last_clean":"2024-09-17T04:25:08.118988+0000","last_became_active":"2024-09-17T04:25:07.089256+0000","last_became_peered":"2024-09-17T04:25:07.089256+0000","last_unstale":"2024-09-17T04:25:08.118988+0000","last_undegraded":"2024-09-17T04:25:08.118988+0000","last_fullsized":"2024-09-17T04:25:08.118988+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:11:57.535300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.132485+0000","last_change":"2024-09-17T04:25:08.127043+0000","last_active":"2024-09-17T04:25:08.132485+0000","last_peered":"2024-09-17T04:25:08.132485+0000","last_clean":"2024-09-17T04:25:08.132485+0000","last_became_active":"2024-09-17T04:25:08.126806+0000","last_became_peered":"2024-09-17T04:25:08.126806+0000","last_unstale":"2024-09-17T04:25:08.132485+0000","last_undegraded":"2024-09-17T04:25:08.132485+0000","last_fullsized":"2024-09-17T04:25:08.132485+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:03.317036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.123222+0000","last_change":"2024-09-17T04:25:07.098143+0000","last_active":"2024-09-17T04:25:08.123222+0000","last_peered":"2024-09-17T04:25:08.123222+0000","last_clean":"2024-09-17T04:25:08.123222+0000","last_became_active":"2024-09-17T04:25:07.091768+0000","last_became_peered":"2024-09-17T04:25:07.091768+0000","last_unstale":"2024-09-17T04:25:08.123222+0000","last_undegraded":"2024-09-17T04:25:08.123222+0000","last_fullsized":"2024-09-17T04:25:08.123222+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:15:18.524829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.118977+0000","last_change":"2024-09-17T04:25:08.108470+0000","last_active":"2024-09-17T04:25:08.118977+0000","last_peered":"2024-09-17T04:25:08.118977+0000","last_clean":"2024-09-17T04:25:08.118977+0000","last_became_active":"2024-09-17T04:25:08.108144+0000","last_became_peered":"2024-09-17T04:25:08.108144+0000","last_unstale":"2024-09-17T04:25:08.118977+0000","last_undegraded":"2024-09-17T04:25:08.118977+0000","last_fullsized":"2024-09-17T04:25:08.118977+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:51.768978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.100120+0000","last_change":"2024-09-17T04:25:07.103224+0000","last_active":"2024-09-17T04:25:08.100120+0000","last_peered":"2024-09-17T04:25:08.100120+0000","last_clean":"2024-09-17T04:25:08.100120+0000","last_became_active":"2024-09-17T04:25:07.101489+0000","last_became_peered":"2024-09-17T04:25:07.101489+0000","last_unstale":"2024-09-17T04:25:08.100120+0000","last_undegraded":"2024-09-17T04:25:08.100120+0000","last_fullsized":"2024-09-17T04:25:08.100120+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:08.430358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.122588+0000","last_change":"2024-09-17T04:25:08.112749+0000","last_active":"2024-09-17T04:25:08.122588+0000","last_peered":"2024-09-17T04:25:08.122588+0000","last_clean":"2024-09-17T04:25:08.122588+0000","last_became_active":"2024-09-17T04:25:08.112408+0000","last_became_peered":"2024-09-17T04:25:08.112408+0000","last_unstale":"2024-09-17T04:25:08.122588+0000","last_undegraded":"2024-09-17T04:25:08.122588+0000","last_fullsized":"2024-09-17T04:25:08.122588+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:07:25.567949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.123505+0000","last_change":"2024-09-17T04:25:07.092602+0000","last_active":"2024-09-17T04:25:08.123505+0000","last_peered":"2024-09-17T04:25:08.123505+0000","last_clean":"2024-09-17T04:25:08.123505+0000","last_became_active":"2024-09-17T04:25:07.092215+0000","last_became_peered":"2024-09-17T04:25:07.092215+0000","last_unstale":"2024-09-17T04:25:08.123505+0000","last_undegraded":"2024-09-17T04:25:08.123505+0000","last_fullsized":"2024-09-17T04:25:08.123505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:19.304290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.118395+0000","last_change":"2024-09-17T04:25:08.107413+0000","last_active":"2024-09-17T04:25:08.118395+0000","last_peered":"2024-09-17T04:25:08.118395+0000","last_clean":"2024-09-17T04:25:08.118395+0000","last_became_active":"2024-09-17T04:25:08.106516+0000","last_became_peered":"2024-09-17T04:25:08.106516+0000","last_unstale":"2024-09-17T04:25:08.118395+0000","last_undegraded":"2024-09-17T04:25:08.118395+0000","last_fullsized":"2024-09-17T04:25:08.118395+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:06.209252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.103107+0000","last_change":"2024-09-17T04:25:07.101091+0000","last_active":"2024-09-17T04:25:08.103107+0000","last_peered":"2024-09-17T04:25:08.103107+0000","last_clean":"2024-09-17T04:25:08.103107+0000","last_became_active":"2024-09-17T04:25:07.100821+0000","last_became_peered":"2024-09-17T04:25:07.100821+0000","last_unstale":"2024-09-17T04:25:08.103107+0000","last_undegraded":"2024-09-17T04:25:08.103107+0000","last_fullsized":"2024-09-17T04:25:08.103107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:50:25.153746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.108383+0000","last_change":"2024-09-17T04:25:08.104836+0000","last_active":"2024-09-17T04:25:08.108383+0000","last_peered":"2024-09-17T04:25:08.108383+0000","last_clean":"2024-09-17T04:25:08.108383+0000","last_became_active":"2024-09-17T04:25:08.104120+0000","last_became_peered":"2024-09-17T04:25:08.104120+0000","last_unstale":"2024-09-17T04:25:08.108383+0000","last_undegraded":"2024-09-17T04:25:08.108383+0000","last_fullsized":"2024-09-17T04:25:08.108383+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:05:27.192448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.097525+0000","last_change":"2024-09-17T04:25:07.103534+0000","last_active":"2024-09-17T04:25:08.097525+0000","last_peered":"2024-09-17T04:25:08.097525+0000","last_clean":"2024-09-17T04:25:08.097525+0000","last_became_active":"2024-09-17T04:25:07.103070+0000","last_became_peered":"2024-09-17T04:25:07.103070+0000","last_unstale":"2024-09-17T04:25:08.097525+0000","last_undegraded":"2024-09-17T04:25:08.097525+0000","last_fullsized":"2024-09-17T04:25:08.097525+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:57:15.056986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113955+0000","last_change":"2024-09-17T04:25:08.113398+0000","last_active":"2024-09-17T04:25:09.113955+0000","last_peered":"2024-09-17T04:25:09.113955+0000","last_clean":"2024-09-17T04:25:09.113955+0000","last_became_active":"2024-09-17T04:25:08.113049+0000","last_became_peered":"2024-09-17T04:25:08.113049+0000","last_unstale":"2024-09-17T04:25:09.113955+0000","last_undegraded":"2024-09-17T04:25:09.113955+0000","last_fullsized":"2024-09-17T04:25:09.113955+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:12.516316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.111837+0000","last_change":"2024-09-17T04:25:07.103861+0000","last_active":"2024-09-17T04:25:09.111837+0000","last_peered":"2024-09-17T04:25:09.111837+0000","last_clean":"2024-09-17T04:25:09.111837+0000","last_became_active":"2024-09-17T04:25:07.100484+0000","last_became_peered":"2024-09-17T04:25:07.100484+0000","last_unstale":"2024-09-17T04:25:09.111837+0000","last_undegraded":"2024-09-17T04:25:09.111837+0000","last_fullsized":"2024-09-17T04:25:09.111837+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:27.284841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.105143+0000","last_change":"2024-09-17T04:25:08.100426+0000","last_active":"2024-09-17T04:25:08.105143+0000","last_peered":"2024-09-17T04:25:08.105143+0000","last_clean":"2024-09-17T04:25:08.105143+0000","last_became_active":"2024-09-17T04:25:08.100117+0000","last_became_peered":"2024-09-17T04:25:08.100117+0000","last_unstale":"2024-09-17T04:25:08.105143+0000","last_undegraded":"2024-09-17T04:25:08.105143+0000","last_fullsized":"2024-09-17T04:25:08.105143+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:02.165986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.125748+0000","last_change":"2024-09-17T04:25:07.097884+0000","last_active":"2024-09-17T04:25:08.125748+0000","last_peered":"2024-09-17T04:25:08.125748+0000","last_clean":"2024-09-17T04:25:08.125748+0000","last_became_active":"2024-09-17T04:25:07.097545+0000","last_became_peered":"2024-09-17T04:25:07.097545+0000","last_unstale":"2024-09-17T04:25:08.125748+0000","last_undegraded":"2024-09-17T04:25:08.125748+0000","last_fullsized":"2024-09-17T04:25:08.125748+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:01.801695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.114256+0000","last_change":"2024-09-17T04:25:08.123061+0000","last_active":"2024-09-17T04:25:09.114256+0000","last_peered":"2024-09-17T04:25:09.114256+0000","last_clean":"2024-09-17T04:25:09.114256+0000","last_became_active":"2024-09-17T04:25:08.122319+0000","last_became_peered":"2024-09-17T04:25:08.122319+0000","last_unstale":"2024-09-17T04:25:09.114256+0000","last_undegraded":"2024-09-17T04:25:09.114256+0000","last_fullsized":"2024-09-17T04:25:09.114256+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:22.538564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.094731+0000","last_change":"2024-09-17T04:25:07.088222+0000","last_active":"2024-09-17T04:25:08.094731+0000","last_peered":"2024-09-17T04:25:08.094731+0000","last_clean":"2024-09-17T04:25:08.094731+0000","last_became_active":"2024-09-17T04:25:07.087921+0000","last_became_peered":"2024-09-17T04:25:07.087921+0000","last_unstale":"2024-09-17T04:25:08.094731+0000","last_undegraded":"2024-09-17T04:25:08.094731+0000","last_fullsized":"2024-09-17T04:25:08.094731+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:35.879479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.116348+0000","last_change":"2024-09-17T04:25:08.115284+0000","last_active":"2024-09-17T04:25:08.116348+0000","last_peered":"2024-09-17T04:25:08.116348+0000","last_clean":"2024-09-17T04:25:08.116348+0000","last_became_active":"2024-09-17T04:25:08.114998+0000","last_became_peered":"2024-09-17T04:25:08.114998+0000","last_unstale":"2024-09-17T04:25:08.116348+0000","last_undegraded":"2024-09-17T04:25:08.116348+0000","last_fullsized":"2024-09-17T04:25:08.116348+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:40:24.798987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.100440+0000","last_change":"2024-09-17T04:25:07.096188+0000","last_active":"2024-09-17T04:25:08.100440+0000","last_peered":"2024-09-17T04:25:08.100440+0000","last_clean":"2024-09-17T04:25:08.100440+0000","last_became_active":"2024-09-17T04:25:07.094697+0000","last_became_peered":"2024-09-17T04:25:07.094697+0000","last_unstale":"2024-09-17T04:25:08.100440+0000","last_undegraded":"2024-09-17T04:25:08.100440+0000","last_fullsized":"2024-09-17T04:25:08.100440+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:05.174227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.120861+0000","last_change":"2024-09-17T04:25:08.115704+0000","last_active":"2024-09-17T04:25:08.120861+0000","last_peered":"2024-09-17T04:25:08.120861+0000","last_clean":"2024-09-17T04:25:08.120861+0000","last_became_active":"2024-09-17T04:25:08.115380+0000","last_became_peered":"2024-09-17T04:25:08.115380+0000","last_unstale":"2024-09-17T04:25:08.120861+0000","last_undegraded":"2024-09-17T04:25:08.120861+0000","last_fullsized":"2024-09-17T04:25:08.120861+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:21.618822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.097074+0000","last_change":"2024-09-17T04:25:07.095160+0000","last_active":"2024-09-17T04:25:08.097074+0000","last_peered":"2024-09-17T04:25:08.097074+0000","last_clean":"2024-09-17T04:25:08.097074+0000","last_became_active":"2024-09-17T04:25:07.094948+0000","last_became_peered":"2024-09-17T04:25:07.094948+0000","last_unstale":"2024-09-17T04:25:08.097074+0000","last_undegraded":"2024-09-17T04:25:08.097074+0000","last_fullsized":"2024-09-17T04:25:08.097074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:40.737074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.123596+0000","last_change":"2024-09-17T04:25:08.120140+0000","last_active":"2024-09-17T04:25:08.123596+0000","last_peered":"2024-09-17T04:25:08.123596+0000","last_clean":"2024-09-17T04:25:08.123596+0000","last_became_active":"2024-09-17T04:25:08.118074+0000","last_became_peered":"2024-09-17T04:25:08.118074+0000","last_unstale":"2024-09-17T04:25:08.123596+0000","last_undegraded":"2024-09-17T04:25:08.123596+0000","last_fullsized":"2024-09-17T04:25:08.123596+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:15:30.368718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.100996+0000","last_change":"2024-09-17T04:25:07.094913+0000","last_active":"2024-09-17T04:25:08.100996+0000","last_peered":"2024-09-17T04:25:08.100996+0000","last_clean":"2024-09-17T04:25:08.100996+0000","last_became_active":"2024-09-17T04:25:07.094618+0000","last_became_peered":"2024-09-17T04:25:07.094618+0000","last_unstale":"2024-09-17T04:25:08.100996+0000","last_undegraded":"2024-09-17T04:25:08.100996+0000","last_fullsized":"2024-09-17T04:25:08.100996+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:20.612773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.081356+0000","last_change":"2024-09-17T04:25:07.081356+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.081356+0000","last_undegraded":"2024-09-17T04:25:07.081356+0000","last_fullsized":"2024-09-17T04:25:07.081356+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.097817+0000","last_change":"2024-09-17T04:25:07.093510+0000","last_active":"2024-09-17T04:25:08.097817+0000","last_peered":"2024-09-17T04:25:08.097817+0000","last_clean":"2024-09-17T04:25:08.097817+0000","last_became_active":"2024-09-17T04:25:07.093132+0000","last_became_peered":"2024-09-17T04:25:07.093132+0000","last_unstale":"2024-09-17T04:25:08.097817+0000","last_undegraded":"2024-09-17T04:25:08.097817+0000","last_fullsized":"2024-09-17T04:25:08.097817+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:04:39.641973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.132867+0000","last_change":"2024-09-17T04:25:08.127561+0000","last_active":"2024-09-17T04:25:08.132867+0000","last_peered":"2024-09-17T04:25:08.132867+0000","last_clean":"2024-09-17T04:25:08.132867+0000","last_became_active":"2024-09-17T04:25:08.127108+0000","last_became_peered":"2024-09-17T04:25:08.127108+0000","last_unstale":"2024-09-17T04:25:08.132867+0000","last_undegraded":"2024-09-17T04:25:08.132867+0000","last_fullsized":"2024-09-17T04:25:08.132867+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:40:24.798987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T04:25:07.104421+0000","last_change":"2024-09-17T04:25:07.101483+0000","last_active":"2024-09-17T04:25:07.104421+0000","last_peered":"2024-09-17T04:25:07.104421+0000","last_clean":"2024-09-17T04:25:07.104421+0000","last_became_active":"2024-09-17T04:25:07.100021+0000","last_became_peered":"2024-09-17T04:25:07.100021+0000","last_unstale":"2024-09-17T04:25:07.104421+0000","last_undegraded":"2024-09-17T04:25:07.104421+0000","last_fullsized":"2024-09-17T04:25:07.104421+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:30:05.938041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112185+0000","last_change":"2024-09-17T04:25:08.102453+0000","last_active":"2024-09-17T04:25:09.112185+0000","last_peered":"2024-09-17T04:25:09.112185+0000","last_clean":"2024-09-17T04:25:09.112185+0000","last_became_active":"2024-09-17T04:25:08.101181+0000","last_became_peered":"2024-09-17T04:25:08.101181+0000","last_unstale":"2024-09-17T04:25:09.112185+0000","last_undegraded":"2024-09-17T04:25:09.112185+0000","last_fullsized":"2024-09-17T04:25:09.112185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:11.397751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.103937+0000","last_change":"2024-09-17T04:25:07.105074+0000","last_active":"2024-09-17T04:25:08.103937+0000","last_peered":"2024-09-17T04:25:08.103937+0000","last_clean":"2024-09-17T04:25:08.103937+0000","last_became_active":"2024-09-17T04:25:07.104735+0000","last_became_peered":"2024-09-17T04:25:07.104735+0000","last_unstale":"2024-09-17T04:25:08.103937+0000","last_undegraded":"2024-09-17T04:25:08.103937+0000","last_fullsized":"2024-09-17T04:25:08.103937+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:06:46.154246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":136,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113124+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:25:09.113124+0000","last_peered":"2024-09-17T04:25:09.113124+0000","last_clean":"2024-09-17T04:25:09.113124+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:25:09.113124+0000","last_undegraded":"2024-09-17T04:25:09.113124+0000","last_fullsized":"2024-09-17T04:25:09.113124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22,"ondisk_log_size":22,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476754,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":739667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476753,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":149825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476754,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":149881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476754,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":393216,"data_stored":149313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":372736,"data_stored":142434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509456,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":372736,"data_stored":142434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":966656,"data_stored":732802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":360448,"data_stored":136167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:25:12.580 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:25:12.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.802+0000 7f94c9400640 1 -- 172.21.15.100:0/4093435372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94a8004800 msgr2=0x7f94a8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:12.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.802+0000 7f94c9400640 1 --2- 172.21.15.100:0/4093435372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94a8004800 0x7f94a8004c10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f94b0001800 tx=0x7f94a8005830 comp rx=0 tx=0).stop 2024-09-17T04:25:12.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.802+0000 7f94c3200640 1 -- 172.21.15.100:0/4093435372 reap_dead start 2024-09-17T04:25:12.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.802+0000 7f94c9400640 1 -- 172.21.15.100:0/4093435372 shutdown_connections 2024-09-17T04:25:12.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.802+0000 7f94c9400640 1 -- 172.21.15.100:0/4093435372 >> 172.21.15.100:0/4093435372 conn(0x7f94c406d730 msgr2=0x7f94c40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:12.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.802+0000 7f94c9400640 1 -- 172.21.15.100:0/4093435372 shutdown_connections 2024-09-17T04:25:12.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.802+0000 7f94c9400640 1 -- 172.21.15.100:0/4093435372 wait complete. 2024-09-17T04:25:12.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.804+0000 7f94c9400640 1 Processor -- start 2024-09-17T04:25:12.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.805+0000 7f94c9400640 1 -- start start 2024-09-17T04:25:12.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.806+0000 7f94c9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94a8004800 0x7f94c4156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:12.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.806+0000 7f94c9400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94c41f67b0 0x7f94c41494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:12.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.806+0000 7f94c9400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94c41608b0 0x7f94c4160cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:12.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.806+0000 7f94c9400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f94c406f230 con 0x7f94a8004800 2024-09-17T04:25:12.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.806+0000 7f94c9400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f94c40b4e30 con 0x7f94c41f67b0 2024-09-17T04:25:12.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.806+0000 7f94c9400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f94c4165890 con 0x7f94c41608b0 2024-09-17T04:25:12.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.806+0000 7f94c2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94a8004800 0x7f94c4156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:12.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.806+0000 7f94c2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94a8004800 0x7f94c4156520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38938/0 (socket says 172.21.15.100:38938) 2024-09-17T04:25:12.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.806+0000 7f94c2800640 1 -- 172.21.15.100:0/1885039097 learned_addr learned my addr 172.21.15.100:0/1885039097 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:12.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.807+0000 7f94c3200640 1 --2- 172.21.15.100:0/1885039097 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94c41608b0 0x7f94c4160cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:12.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.807+0000 7f94c1e00640 1 --2- 172.21.15.100:0/1885039097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94c41f67b0 0x7f94c41494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:12.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.807+0000 7f94c2800640 1 -- 172.21.15.100:0/1885039097 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94c41608b0 msgr2=0x7f94c4160cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:12.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.807+0000 7f94c2800640 1 --2- 172.21.15.100:0/1885039097 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94c41608b0 0x7f94c4160cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:12.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.807+0000 7f94c2800640 1 -- 172.21.15.100:0/1885039097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94c41f67b0 msgr2=0x7f94c41494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:12.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.807+0000 7f94c2800640 1 --2- 172.21.15.100:0/1885039097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94c41f67b0 0x7f94c41494b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:12.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.807+0000 7f94c2800640 1 -- 172.21.15.100:0/1885039097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c4150b20 con 0x7f94a8004800 2024-09-17T04:25:12.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.807+0000 7f94c3200640 1 --2- 172.21.15.100:0/1885039097 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94c41608b0 0x7f94c4160cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:25:12.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.807+0000 7f94c1e00640 1 --2- 172.21.15.100:0/1885039097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94c41f67b0 0x7f94c41494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:25:12.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.807+0000 7f94c2800640 1 --2- 172.21.15.100:0/1885039097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94a8004800 0x7f94c4156520 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f94b400e590 tx=0x7f94b400ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:12.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.807+0000 7f94baa00640 1 -- 172.21.15.100:0/1885039097 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b4015070 con 0x7f94a8004800 2024-09-17T04:25:12.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.807+0000 7f94baa00640 1 -- 172.21.15.100:0/1885039097 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94b40109a0 con 0x7f94a8004800 2024-09-17T04:25:12.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.807+0000 7f94c9400640 1 -- 172.21.15.100:0/1885039097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c41505a0 con 0x7f94a8004800 2024-09-17T04:25:12.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.807+0000 7f94baa00640 1 -- 172.21.15.100:0/1885039097 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b4010df0 con 0x7f94a8004800 2024-09-17T04:25:12.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.807+0000 7f94c9400640 1 -- 172.21.15.100:0/1885039097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c41f6bc0 con 0x7f94a8004800 2024-09-17T04:25:12.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.808+0000 7f94c9400640 1 -- 172.21.15.100:0/1885039097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94c41535c0 con 0x7f94a8004800 2024-09-17T04:25:12.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.810+0000 7f94baa00640 1 -- 172.21.15.100:0/1885039097 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f94b4018070 con 0x7f94a8004800 2024-09-17T04:25:12.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.811+0000 7f94baa00640 1 --2- 172.21.15.100:0/1885039097 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f94a4078b90 0x7f94a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:12.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.811+0000 7f94baa00640 1 -- 172.21.15.100:0/1885039097 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f94b409b180 con 0x7f94a8004800 2024-09-17T04:25:12.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.811+0000 7f94c1e00640 1 --2- 172.21.15.100:0/1885039097 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f94a4078b90 0x7f94a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:12.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.812+0000 7f94c1e00640 1 --2- 172.21.15.100:0/1885039097 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f94a4078b90 0x7f94a407b0a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f94b0032000 tx=0x7f94b0001360 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:12.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.815+0000 7f94baa00640 1 -- 172.21.15.100:0/1885039097 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f94b4063640 con 0x7f94a8004800 2024-09-17T04:25:12.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:12.995+0000 7f94c9400640 1 -- 172.21.15.100:0/1885039097 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f94c414b510 con 0x7f94a4078b90 2024-09-17T04:25:13.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.001+0000 7f94baa00640 1 -- 172.21.15.100:0/1885039097 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251103 (secure 0 0 0) 0x7f94c414b510 con 0x7f94a4078b90 2024-09-17T04:25:13.004 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:13.006 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:25:13.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.005+0000 7f94c9400640 1 -- 172.21.15.100:0/1885039097 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f94a4078b90 msgr2=0x7f94a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:13.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.005+0000 7f94c9400640 1 --2- 172.21.15.100:0/1885039097 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f94a4078b90 0x7f94a407b0a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f94b0032000 tx=0x7f94b0001360 comp rx=0 tx=0).stop 2024-09-17T04:25:13.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.005+0000 7f94c9400640 1 -- 172.21.15.100:0/1885039097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94a8004800 msgr2=0x7f94c4156520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:13.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.005+0000 7f94c9400640 1 --2- 172.21.15.100:0/1885039097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94a8004800 0x7f94c4156520 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f94b400e590 tx=0x7f94b400ea50 comp rx=0 tx=0).stop 2024-09-17T04:25:13.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.005+0000 7f94c9400640 1 -- 172.21.15.100:0/1885039097 shutdown_connections 2024-09-17T04:25:13.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.005+0000 7f94c9400640 1 --2- 172.21.15.100:0/1885039097 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f94a4078b90 0x7f94a407b0a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:13.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.005+0000 7f94c9400640 1 --2- 172.21.15.100:0/1885039097 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94c41608b0 0x7f94c4160cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:13.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.005+0000 7f94c9400640 1 --2- 172.21.15.100:0/1885039097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94c41f67b0 0x7f94c41494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:13.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.005+0000 7f94c9400640 1 --2- 172.21.15.100:0/1885039097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94a8004800 0x7f94c4156520 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:13.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.005+0000 7f94c9400640 1 -- 172.21.15.100:0/1885039097 >> 172.21.15.100:0/1885039097 conn(0x7f94c406d730 msgr2=0x7f94c415f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:13.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.005+0000 7f94c9400640 1 -- 172.21.15.100:0/1885039097 shutdown_connections 2024-09-17T04:25:13.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.005+0000 7f94c9400640 1 -- 172.21.15.100:0/1885039097 wait complete. 2024-09-17T04:25:13.028 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-09-17T04:25:11.979707+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591166,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":124,"num_write_kb":1639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94,"ondisk_log_size":94,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":96,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220372,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754348,"statfs":{"total":773094113280,"available":772868452352,"internally_reserved":0,"allocated":4235264,"data_stored":2342523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.910164"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.131931+0000","last_change":"2024-09-17T04:25:08.126623+0000","last_active":"2024-09-17T04:25:08.131931+0000","last_peered":"2024-09-17T04:25:08.131931+0000","last_clean":"2024-09-17T04:25:08.131931+0000","last_became_active":"2024-09-17T04:25:08.126204+0000","last_became_peered":"2024-09-17T04:25:08.126204+0000","last_unstale":"2024-09-17T04:25:08.131931+0000","last_undegraded":"2024-09-17T04:25:08.131931+0000","last_fullsized":"2024-09-17T04:25:08.131931+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:40.098175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.088364+0000","last_change":"2024-09-17T04:25:07.088364+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.088364+0000","last_undegraded":"2024-09-17T04:25:07.088364+0000","last_fullsized":"2024-09-17T04:25:07.088364+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.114386+0000","last_change":"2024-09-17T04:25:08.116641+0000","last_active":"2024-09-17T04:25:09.114386+0000","last_peered":"2024-09-17T04:25:09.114386+0000","last_clean":"2024-09-17T04:25:09.114386+0000","last_became_active":"2024-09-17T04:25:08.115852+0000","last_became_peered":"2024-09-17T04:25:08.115852+0000","last_unstale":"2024-09-17T04:25:09.114386+0000","last_undegraded":"2024-09-17T04:25:09.114386+0000","last_fullsized":"2024-09-17T04:25:09.114386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:46.534540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.132543+0000","last_change":"2024-09-17T04:25:08.127729+0000","last_active":"2024-09-17T04:25:08.132543+0000","last_peered":"2024-09-17T04:25:08.132543+0000","last_clean":"2024-09-17T04:25:08.132543+0000","last_became_active":"2024-09-17T04:25:08.125546+0000","last_became_peered":"2024-09-17T04:25:08.125546+0000","last_unstale":"2024-09-17T04:25:08.132543+0000","last_undegraded":"2024-09-17T04:25:08.132543+0000","last_fullsized":"2024-09-17T04:25:08.132543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:01:06.440885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.122163+0000","last_change":"2024-09-17T04:25:08.113523+0000","last_active":"2024-09-17T04:25:08.122163+0000","last_peered":"2024-09-17T04:25:08.122163+0000","last_clean":"2024-09-17T04:25:08.122163+0000","last_became_active":"2024-09-17T04:25:08.113207+0000","last_became_peered":"2024-09-17T04:25:08.113207+0000","last_unstale":"2024-09-17T04:25:08.122163+0000","last_undegraded":"2024-09-17T04:25:08.122163+0000","last_fullsized":"2024-09-17T04:25:08.122163+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:28.634098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.083605+0000","last_change":"2024-09-17T04:25:07.083605+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.083605+0000","last_undegraded":"2024-09-17T04:25:07.083605+0000","last_fullsized":"2024-09-17T04:25:07.083605+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113556+0000","last_change":"2024-09-17T04:25:08.113471+0000","last_active":"2024-09-17T04:25:09.113556+0000","last_peered":"2024-09-17T04:25:09.113556+0000","last_clean":"2024-09-17T04:25:09.113556+0000","last_became_active":"2024-09-17T04:25:08.113145+0000","last_became_peered":"2024-09-17T04:25:08.113145+0000","last_unstale":"2024-09-17T04:25:09.113556+0000","last_undegraded":"2024-09-17T04:25:09.113556+0000","last_fullsized":"2024-09-17T04:25:09.113556+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:52.755680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112954+0000","last_change":"2024-09-17T04:25:08.102460+0000","last_active":"2024-09-17T04:25:09.112954+0000","last_peered":"2024-09-17T04:25:09.112954+0000","last_clean":"2024-09-17T04:25:09.112954+0000","last_became_active":"2024-09-17T04:25:08.101474+0000","last_became_peered":"2024-09-17T04:25:08.101474+0000","last_unstale":"2024-09-17T04:25:09.112954+0000","last_undegraded":"2024-09-17T04:25:09.112954+0000","last_fullsized":"2024-09-17T04:25:09.112954+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:22.538564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.126318+0000","last_change":"2024-09-17T04:25:08.119819+0000","last_active":"2024-09-17T04:25:08.126318+0000","last_peered":"2024-09-17T04:25:08.126318+0000","last_clean":"2024-09-17T04:25:08.126318+0000","last_became_active":"2024-09-17T04:25:08.119511+0000","last_became_peered":"2024-09-17T04:25:08.119511+0000","last_unstale":"2024-09-17T04:25:08.126318+0000","last_undegraded":"2024-09-17T04:25:08.126318+0000","last_fullsized":"2024-09-17T04:25:08.126318+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:49:01.278096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.124020+0000","last_change":"2024-09-17T04:25:08.121720+0000","last_active":"2024-09-17T04:25:08.124020+0000","last_peered":"2024-09-17T04:25:08.124020+0000","last_clean":"2024-09-17T04:25:08.124020+0000","last_became_active":"2024-09-17T04:25:08.121405+0000","last_became_peered":"2024-09-17T04:25:08.121405+0000","last_unstale":"2024-09-17T04:25:08.124020+0000","last_undegraded":"2024-09-17T04:25:08.124020+0000","last_fullsized":"2024-09-17T04:25:08.124020+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:37.261699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112530+0000","last_change":"2024-09-17T04:25:08.116374+0000","last_active":"2024-09-17T04:25:09.112530+0000","last_peered":"2024-09-17T04:25:09.112530+0000","last_clean":"2024-09-17T04:25:09.112530+0000","last_became_active":"2024-09-17T04:25:08.116047+0000","last_became_peered":"2024-09-17T04:25:08.116047+0000","last_unstale":"2024-09-17T04:25:09.112530+0000","last_undegraded":"2024-09-17T04:25:09.112530+0000","last_fullsized":"2024-09-17T04:25:09.112530+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:12:03.863249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113687+0000","last_change":"2024-09-17T04:25:08.117853+0000","last_active":"2024-09-17T04:25:09.113687+0000","last_peered":"2024-09-17T04:25:09.113687+0000","last_clean":"2024-09-17T04:25:09.113687+0000","last_became_active":"2024-09-17T04:25:08.117342+0000","last_became_peered":"2024-09-17T04:25:08.117342+0000","last_unstale":"2024-09-17T04:25:09.113687+0000","last_undegraded":"2024-09-17T04:25:09.113687+0000","last_fullsized":"2024-09-17T04:25:09.113687+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:59.322143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.118578+0000","last_change":"2024-09-17T04:25:08.107583+0000","last_active":"2024-09-17T04:25:08.118578+0000","last_peered":"2024-09-17T04:25:08.118578+0000","last_clean":"2024-09-17T04:25:08.118578+0000","last_became_active":"2024-09-17T04:25:08.107097+0000","last_became_peered":"2024-09-17T04:25:08.107097+0000","last_unstale":"2024-09-17T04:25:08.118578+0000","last_undegraded":"2024-09-17T04:25:08.118578+0000","last_fullsized":"2024-09-17T04:25:08.118578+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:37.405465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.083133+0000","last_change":"2024-09-17T04:25:07.083133+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.083133+0000","last_undegraded":"2024-09-17T04:25:07.083133+0000","last_fullsized":"2024-09-17T04:25:07.083133+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112471+0000","last_change":"2024-09-17T04:25:08.122562+0000","last_active":"2024-09-17T04:25:09.112471+0000","last_peered":"2024-09-17T04:25:09.112471+0000","last_clean":"2024-09-17T04:25:09.112471+0000","last_became_active":"2024-09-17T04:25:08.120406+0000","last_became_peered":"2024-09-17T04:25:08.120406+0000","last_unstale":"2024-09-17T04:25:09.112471+0000","last_undegraded":"2024-09-17T04:25:09.112471+0000","last_fullsized":"2024-09-17T04:25:09.112471+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:14.660711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113172+0000","last_change":"2024-09-17T04:25:08.111402+0000","last_active":"2024-09-17T04:25:09.113172+0000","last_peered":"2024-09-17T04:25:09.113172+0000","last_clean":"2024-09-17T04:25:09.113172+0000","last_became_active":"2024-09-17T04:25:08.111049+0000","last_became_peered":"2024-09-17T04:25:08.111049+0000","last_unstale":"2024-09-17T04:25:09.113172+0000","last_undegraded":"2024-09-17T04:25:09.113172+0000","last_fullsized":"2024-09-17T04:25:09.113172+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:46.534540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112688+0000","last_change":"2024-09-17T04:25:08.113695+0000","last_active":"2024-09-17T04:25:09.112688+0000","last_peered":"2024-09-17T04:25:09.112688+0000","last_clean":"2024-09-17T04:25:09.112688+0000","last_became_active":"2024-09-17T04:25:08.113248+0000","last_became_peered":"2024-09-17T04:25:08.113248+0000","last_unstale":"2024-09-17T04:25:09.112688+0000","last_undegraded":"2024-09-17T04:25:09.112688+0000","last_fullsized":"2024-09-17T04:25:09.112688+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:52.735017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.087894+0000","last_change":"2024-09-17T04:25:07.087894+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.087894+0000","last_undegraded":"2024-09-17T04:25:07.087894+0000","last_fullsized":"2024-09-17T04:25:07.087894+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.084115+0000","last_change":"2024-09-17T04:25:07.084115+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.084115+0000","last_undegraded":"2024-09-17T04:25:07.084115+0000","last_fullsized":"2024-09-17T04:25:07.084115+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.107540+0000","last_change":"2024-09-17T04:25:08.102248+0000","last_active":"2024-09-17T04:25:08.107540+0000","last_peered":"2024-09-17T04:25:08.107540+0000","last_clean":"2024-09-17T04:25:08.107540+0000","last_became_active":"2024-09-17T04:25:08.101915+0000","last_became_peered":"2024-09-17T04:25:08.101915+0000","last_unstale":"2024-09-17T04:25:08.107540+0000","last_undegraded":"2024-09-17T04:25:08.107540+0000","last_fullsized":"2024-09-17T04:25:08.107540+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:11.459548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.125029+0000","last_change":"2024-09-17T04:25:08.116792+0000","last_active":"2024-09-17T04:25:08.125029+0000","last_peered":"2024-09-17T04:25:08.125029+0000","last_clean":"2024-09-17T04:25:08.125029+0000","last_became_active":"2024-09-17T04:25:08.116468+0000","last_became_peered":"2024-09-17T04:25:08.116468+0000","last_unstale":"2024-09-17T04:25:08.125029+0000","last_undegraded":"2024-09-17T04:25:08.125029+0000","last_fullsized":"2024-09-17T04:25:08.125029+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:01.832590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.132325+0000","last_change":"2024-09-17T04:25:08.125202+0000","last_active":"2024-09-17T04:25:08.132325+0000","last_peered":"2024-09-17T04:25:08.132325+0000","last_clean":"2024-09-17T04:25:08.132325+0000","last_became_active":"2024-09-17T04:25:08.124880+0000","last_became_peered":"2024-09-17T04:25:08.124880+0000","last_unstale":"2024-09-17T04:25:08.132325+0000","last_undegraded":"2024-09-17T04:25:08.132325+0000","last_fullsized":"2024-09-17T04:25:08.132325+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:33.145063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.115053+0000","last_change":"2024-09-17T04:25:08.109076+0000","last_active":"2024-09-17T04:25:08.115053+0000","last_peered":"2024-09-17T04:25:08.115053+0000","last_clean":"2024-09-17T04:25:08.115053+0000","last_became_active":"2024-09-17T04:25:08.105699+0000","last_became_peered":"2024-09-17T04:25:08.105699+0000","last_unstale":"2024-09-17T04:25:08.115053+0000","last_undegraded":"2024-09-17T04:25:08.115053+0000","last_fullsized":"2024-09-17T04:25:08.115053+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:33.145063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.111966+0000","last_change":"2024-09-17T04:25:08.099256+0000","last_active":"2024-09-17T04:25:09.111966+0000","last_peered":"2024-09-17T04:25:09.111966+0000","last_clean":"2024-09-17T04:25:09.111966+0000","last_became_active":"2024-09-17T04:25:08.098874+0000","last_became_peered":"2024-09-17T04:25:08.098874+0000","last_unstale":"2024-09-17T04:25:09.111966+0000","last_undegraded":"2024-09-17T04:25:09.111966+0000","last_fullsized":"2024-09-17T04:25:09.111966+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:11.397751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.122652+0000","last_change":"2024-09-17T04:25:08.117901+0000","last_active":"2024-09-17T04:25:08.122652+0000","last_peered":"2024-09-17T04:25:08.122652+0000","last_clean":"2024-09-17T04:25:08.122652+0000","last_became_active":"2024-09-17T04:25:08.115468+0000","last_became_peered":"2024-09-17T04:25:08.115468+0000","last_unstale":"2024-09-17T04:25:08.122652+0000","last_undegraded":"2024-09-17T04:25:08.122652+0000","last_fullsized":"2024-09-17T04:25:08.122652+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:06.724919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.085167+0000","last_change":"2024-09-17T04:25:07.085167+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.085167+0000","last_undegraded":"2024-09-17T04:25:07.085167+0000","last_fullsized":"2024-09-17T04:25:07.085167+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.129679+0000","last_change":"2024-09-17T04:25:08.126392+0000","last_active":"2024-09-17T04:25:08.129679+0000","last_peered":"2024-09-17T04:25:08.129679+0000","last_clean":"2024-09-17T04:25:08.129679+0000","last_became_active":"2024-09-17T04:25:08.126046+0000","last_became_peered":"2024-09-17T04:25:08.126046+0000","last_unstale":"2024-09-17T04:25:08.129679+0000","last_undegraded":"2024-09-17T04:25:08.129679+0000","last_fullsized":"2024-09-17T04:25:08.129679+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:30.659131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113961+0000","last_change":"2024-09-17T04:25:08.104557+0000","last_active":"2024-09-17T04:25:09.113961+0000","last_peered":"2024-09-17T04:25:09.113961+0000","last_clean":"2024-09-17T04:25:09.113961+0000","last_became_active":"2024-09-17T04:25:08.104246+0000","last_became_peered":"2024-09-17T04:25:08.104246+0000","last_unstale":"2024-09-17T04:25:09.113961+0000","last_undegraded":"2024-09-17T04:25:09.113961+0000","last_fullsized":"2024-09-17T04:25:09.113961+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:26.289675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.132086+0000","last_change":"2024-09-17T04:25:08.124218+0000","last_active":"2024-09-17T04:25:08.132086+0000","last_peered":"2024-09-17T04:25:08.132086+0000","last_clean":"2024-09-17T04:25:08.132086+0000","last_became_active":"2024-09-17T04:25:08.123746+0000","last_became_peered":"2024-09-17T04:25:08.123746+0000","last_unstale":"2024-09-17T04:25:08.132086+0000","last_undegraded":"2024-09-17T04:25:08.132086+0000","last_fullsized":"2024-09-17T04:25:08.132086+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:02.874046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112282+0000","last_change":"2024-09-17T04:25:08.113465+0000","last_active":"2024-09-17T04:25:09.112282+0000","last_peered":"2024-09-17T04:25:09.112282+0000","last_clean":"2024-09-17T04:25:09.112282+0000","last_became_active":"2024-09-17T04:25:08.113106+0000","last_became_peered":"2024-09-17T04:25:08.113106+0000","last_unstale":"2024-09-17T04:25:09.112282+0000","last_undegraded":"2024-09-17T04:25:09.112282+0000","last_fullsized":"2024-09-17T04:25:09.112282+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:14.660711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.120556+0000","last_change":"2024-09-17T04:25:08.111799+0000","last_active":"2024-09-17T04:25:08.120556+0000","last_peered":"2024-09-17T04:25:08.120556+0000","last_clean":"2024-09-17T04:25:08.120556+0000","last_became_active":"2024-09-17T04:25:08.111338+0000","last_became_peered":"2024-09-17T04:25:08.111338+0000","last_unstale":"2024-09-17T04:25:08.120556+0000","last_undegraded":"2024-09-17T04:25:08.120556+0000","last_fullsized":"2024-09-17T04:25:08.120556+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:15.918939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112824+0000","last_change":"2024-09-17T04:25:08.109129+0000","last_active":"2024-09-17T04:25:09.112824+0000","last_peered":"2024-09-17T04:25:09.112824+0000","last_clean":"2024-09-17T04:25:09.112824+0000","last_became_active":"2024-09-17T04:25:08.104873+0000","last_became_peered":"2024-09-17T04:25:08.104873+0000","last_unstale":"2024-09-17T04:25:09.112824+0000","last_undegraded":"2024-09-17T04:25:09.112824+0000","last_fullsized":"2024-09-17T04:25:09.112824+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:48.676959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.112046+0000","last_change":"2024-09-17T04:25:08.104171+0000","last_active":"2024-09-17T04:25:08.112046+0000","last_peered":"2024-09-17T04:25:08.112046+0000","last_clean":"2024-09-17T04:25:08.112046+0000","last_became_active":"2024-09-17T04:25:08.103831+0000","last_became_peered":"2024-09-17T04:25:08.103831+0000","last_unstale":"2024-09-17T04:25:08.112046+0000","last_undegraded":"2024-09-17T04:25:08.112046+0000","last_fullsized":"2024-09-17T04:25:08.112046+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:02.874046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.132361+0000","last_change":"2024-09-17T04:25:07.101542+0000","last_active":"2024-09-17T04:25:08.132361+0000","last_peered":"2024-09-17T04:25:08.132361+0000","last_clean":"2024-09-17T04:25:08.132361+0000","last_became_active":"2024-09-17T04:25:07.101168+0000","last_became_peered":"2024-09-17T04:25:07.101168+0000","last_unstale":"2024-09-17T04:25:08.132361+0000","last_undegraded":"2024-09-17T04:25:08.132361+0000","last_fullsized":"2024-09-17T04:25:08.132361+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:05.174227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.127811+0000","last_change":"2024-09-17T04:25:08.122307+0000","last_active":"2024-09-17T04:25:08.127811+0000","last_peered":"2024-09-17T04:25:08.127811+0000","last_clean":"2024-09-17T04:25:08.127811+0000","last_became_active":"2024-09-17T04:25:08.121816+0000","last_became_peered":"2024-09-17T04:25:08.121816+0000","last_unstale":"2024-09-17T04:25:08.127811+0000","last_undegraded":"2024-09-17T04:25:08.127811+0000","last_fullsized":"2024-09-17T04:25:08.127811+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:06:03.732941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.121882+0000","last_change":"2024-09-17T04:25:07.090513+0000","last_active":"2024-09-17T04:25:08.121882+0000","last_peered":"2024-09-17T04:25:08.121882+0000","last_clean":"2024-09-17T04:25:08.121882+0000","last_became_active":"2024-09-17T04:25:07.090181+0000","last_became_peered":"2024-09-17T04:25:07.090181+0000","last_unstale":"2024-09-17T04:25:08.121882+0000","last_undegraded":"2024-09-17T04:25:08.121882+0000","last_fullsized":"2024-09-17T04:25:08.121882+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:00.826540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.126484+0000","last_change":"2024-09-17T04:25:08.118894+0000","last_active":"2024-09-17T04:25:08.126484+0000","last_peered":"2024-09-17T04:25:08.126484+0000","last_clean":"2024-09-17T04:25:08.126484+0000","last_became_active":"2024-09-17T04:25:08.118563+0000","last_became_peered":"2024-09-17T04:25:08.118563+0000","last_unstale":"2024-09-17T04:25:08.126484+0000","last_undegraded":"2024-09-17T04:25:08.126484+0000","last_fullsized":"2024-09-17T04:25:08.126484+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:36.885529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T04:25:07.106875+0000","last_change":"2024-09-17T04:25:07.103013+0000","last_active":"2024-09-17T04:25:07.106875+0000","last_peered":"2024-09-17T04:25:07.106875+0000","last_clean":"2024-09-17T04:25:07.106875+0000","last_became_active":"2024-09-17T04:25:07.100422+0000","last_became_peered":"2024-09-17T04:25:07.100422+0000","last_unstale":"2024-09-17T04:25:07.106875+0000","last_undegraded":"2024-09-17T04:25:07.106875+0000","last_fullsized":"2024-09-17T04:25:07.106875+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:46.886234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.128163+0000","last_change":"2024-09-17T04:25:08.125767+0000","last_active":"2024-09-17T04:25:08.128163+0000","last_peered":"2024-09-17T04:25:08.128163+0000","last_clean":"2024-09-17T04:25:08.128163+0000","last_became_active":"2024-09-17T04:25:08.125475+0000","last_became_peered":"2024-09-17T04:25:08.125475+0000","last_unstale":"2024-09-17T04:25:08.128163+0000","last_undegraded":"2024-09-17T04:25:08.128163+0000","last_fullsized":"2024-09-17T04:25:08.128163+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:21.841901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.124680+0000","last_change":"2024-09-17T04:25:07.097308+0000","last_active":"2024-09-17T04:25:08.124680+0000","last_peered":"2024-09-17T04:25:08.124680+0000","last_clean":"2024-09-17T04:25:08.124680+0000","last_became_active":"2024-09-17T04:25:07.096657+0000","last_became_peered":"2024-09-17T04:25:07.096657+0000","last_unstale":"2024-09-17T04:25:08.124680+0000","last_undegraded":"2024-09-17T04:25:08.124680+0000","last_fullsized":"2024-09-17T04:25:08.124680+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:49:55.787331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112543+0000","last_change":"2024-09-17T04:25:08.123602+0000","last_active":"2024-09-17T04:25:09.112543+0000","last_peered":"2024-09-17T04:25:09.112543+0000","last_clean":"2024-09-17T04:25:09.112543+0000","last_became_active":"2024-09-17T04:25:08.121886+0000","last_became_peered":"2024-09-17T04:25:08.121886+0000","last_unstale":"2024-09-17T04:25:09.112543+0000","last_undegraded":"2024-09-17T04:25:09.112543+0000","last_fullsized":"2024-09-17T04:25:09.112543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:59.322143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.123415+0000","last_change":"2024-09-17T04:25:07.093514+0000","last_active":"2024-09-17T04:25:08.123415+0000","last_peered":"2024-09-17T04:25:08.123415+0000","last_clean":"2024-09-17T04:25:08.123415+0000","last_became_active":"2024-09-17T04:25:07.091028+0000","last_became_peered":"2024-09-17T04:25:07.091028+0000","last_unstale":"2024-09-17T04:25:08.123415+0000","last_undegraded":"2024-09-17T04:25:08.123415+0000","last_fullsized":"2024-09-17T04:25:08.123415+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:47:16.719512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113223+0000","last_change":"2024-09-17T04:25:08.109606+0000","last_active":"2024-09-17T04:25:09.113223+0000","last_peered":"2024-09-17T04:25:09.113223+0000","last_clean":"2024-09-17T04:25:09.113223+0000","last_became_active":"2024-09-17T04:25:08.109288+0000","last_became_peered":"2024-09-17T04:25:08.109288+0000","last_unstale":"2024-09-17T04:25:09.113223+0000","last_undegraded":"2024-09-17T04:25:09.113223+0000","last_fullsized":"2024-09-17T04:25:09.113223+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:12:03.863249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.121410+0000","last_change":"2024-09-17T04:25:07.092548+0000","last_active":"2024-09-17T04:25:08.121410+0000","last_peered":"2024-09-17T04:25:08.121410+0000","last_clean":"2024-09-17T04:25:08.121410+0000","last_became_active":"2024-09-17T04:25:07.092222+0000","last_became_peered":"2024-09-17T04:25:07.092222+0000","last_unstale":"2024-09-17T04:25:08.121410+0000","last_undegraded":"2024-09-17T04:25:08.121410+0000","last_fullsized":"2024-09-17T04:25:08.121410+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:12.533380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.126405+0000","last_change":"2024-09-17T04:25:08.120205+0000","last_active":"2024-09-17T04:25:08.126405+0000","last_peered":"2024-09-17T04:25:08.126405+0000","last_clean":"2024-09-17T04:25:08.126405+0000","last_became_active":"2024-09-17T04:25:08.119921+0000","last_became_peered":"2024-09-17T04:25:08.119921+0000","last_unstale":"2024-09-17T04:25:08.126405+0000","last_undegraded":"2024-09-17T04:25:08.126405+0000","last_fullsized":"2024-09-17T04:25:08.126405+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:20.310340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T04:25:07.102921+0000","last_change":"2024-09-17T04:25:07.096137+0000","last_active":"2024-09-17T04:25:07.102921+0000","last_peered":"2024-09-17T04:25:07.102921+0000","last_clean":"2024-09-17T04:25:07.102921+0000","last_became_active":"2024-09-17T04:25:07.095852+0000","last_became_peered":"2024-09-17T04:25:07.095852+0000","last_unstale":"2024-09-17T04:25:07.102921+0000","last_undegraded":"2024-09-17T04:25:07.102921+0000","last_fullsized":"2024-09-17T04:25:07.102921+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:28.250505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.116518+0000","last_change":"2024-09-17T04:25:08.107235+0000","last_active":"2024-09-17T04:25:08.116518+0000","last_peered":"2024-09-17T04:25:08.116518+0000","last_clean":"2024-09-17T04:25:08.116518+0000","last_became_active":"2024-09-17T04:25:08.106868+0000","last_became_peered":"2024-09-17T04:25:08.106868+0000","last_unstale":"2024-09-17T04:25:08.116518+0000","last_undegraded":"2024-09-17T04:25:08.116518+0000","last_fullsized":"2024-09-17T04:25:08.116518+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:11.868935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.100418+0000","last_change":"2024-09-17T04:25:07.090542+0000","last_active":"2024-09-17T04:25:08.100418+0000","last_peered":"2024-09-17T04:25:08.100418+0000","last_clean":"2024-09-17T04:25:08.100418+0000","last_became_active":"2024-09-17T04:25:07.087859+0000","last_became_peered":"2024-09-17T04:25:07.087859+0000","last_unstale":"2024-09-17T04:25:08.100418+0000","last_undegraded":"2024-09-17T04:25:08.100418+0000","last_fullsized":"2024-09-17T04:25:08.100418+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:35.343496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.114713+0000","last_change":"2024-09-17T04:25:08.120860+0000","last_active":"2024-09-17T04:25:09.114713+0000","last_peered":"2024-09-17T04:25:09.114713+0000","last_clean":"2024-09-17T04:25:09.114713+0000","last_became_active":"2024-09-17T04:25:08.119392+0000","last_became_peered":"2024-09-17T04:25:08.119392+0000","last_unstale":"2024-09-17T04:25:09.114713+0000","last_undegraded":"2024-09-17T04:25:09.114713+0000","last_fullsized":"2024-09-17T04:25:09.114713+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:52.755680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T04:25:07.106897+0000","last_change":"2024-09-17T04:25:07.103331+0000","last_active":"2024-09-17T04:25:07.106897+0000","last_peered":"2024-09-17T04:25:07.106897+0000","last_clean":"2024-09-17T04:25:07.106897+0000","last_became_active":"2024-09-17T04:25:07.102248+0000","last_became_peered":"2024-09-17T04:25:07.102248+0000","last_unstale":"2024-09-17T04:25:07.106897+0000","last_undegraded":"2024-09-17T04:25:07.106897+0000","last_fullsized":"2024-09-17T04:25:07.106897+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:20:28.817834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.082595+0000","last_change":"2024-09-17T04:25:07.082595+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.082595+0000","last_undegraded":"2024-09-17T04:25:07.082595+0000","last_fullsized":"2024-09-17T04:25:07.082595+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.094826+0000","last_change":"2024-09-17T04:25:07.097895+0000","last_active":"2024-09-17T04:25:08.094826+0000","last_peered":"2024-09-17T04:25:08.094826+0000","last_clean":"2024-09-17T04:25:08.094826+0000","last_became_active":"2024-09-17T04:25:07.097541+0000","last_became_peered":"2024-09-17T04:25:07.097541+0000","last_unstale":"2024-09-17T04:25:08.094826+0000","last_undegraded":"2024-09-17T04:25:08.094826+0000","last_fullsized":"2024-09-17T04:25:08.094826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:49:00.272046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.088550+0000","last_change":"2024-09-17T04:25:07.088550+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.088550+0000","last_undegraded":"2024-09-17T04:25:07.088550+0000","last_fullsized":"2024-09-17T04:25:07.088550+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.114464+0000","last_change":"2024-09-17T04:25:07.103494+0000","last_active":"2024-09-17T04:25:09.114464+0000","last_peered":"2024-09-17T04:25:09.114464+0000","last_clean":"2024-09-17T04:25:09.114464+0000","last_became_active":"2024-09-17T04:25:07.103202+0000","last_became_peered":"2024-09-17T04:25:07.103202+0000","last_unstale":"2024-09-17T04:25:09.114464+0000","last_undegraded":"2024-09-17T04:25:09.114464+0000","last_fullsized":"2024-09-17T04:25:09.114464+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:33.029608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.114318+0000","last_change":"2024-09-17T04:25:08.117481+0000","last_active":"2024-09-17T04:25:09.114318+0000","last_peered":"2024-09-17T04:25:09.114318+0000","last_clean":"2024-09-17T04:25:09.114318+0000","last_became_active":"2024-09-17T04:25:08.117068+0000","last_became_peered":"2024-09-17T04:25:08.117068+0000","last_unstale":"2024-09-17T04:25:09.114318+0000","last_undegraded":"2024-09-17T04:25:09.114318+0000","last_fullsized":"2024-09-17T04:25:09.114318+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:40.823059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T04:25:07.104317+0000","last_change":"2024-09-17T04:25:07.099760+0000","last_active":"2024-09-17T04:25:07.104317+0000","last_peered":"2024-09-17T04:25:07.104317+0000","last_clean":"2024-09-17T04:25:07.104317+0000","last_became_active":"2024-09-17T04:25:07.098636+0000","last_became_peered":"2024-09-17T04:25:07.098636+0000","last_unstale":"2024-09-17T04:25:07.104317+0000","last_undegraded":"2024-09-17T04:25:07.104317+0000","last_fullsized":"2024-09-17T04:25:07.104317+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:32:49.032315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.116388+0000","last_change":"2024-09-17T04:25:08.105980+0000","last_active":"2024-09-17T04:25:08.116388+0000","last_peered":"2024-09-17T04:25:08.116388+0000","last_clean":"2024-09-17T04:25:08.116388+0000","last_became_active":"2024-09-17T04:25:08.105227+0000","last_became_peered":"2024-09-17T04:25:08.105227+0000","last_unstale":"2024-09-17T04:25:08.116388+0000","last_undegraded":"2024-09-17T04:25:08.116388+0000","last_fullsized":"2024-09-17T04:25:08.116388+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:40.098175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.124024+0000","last_change":"2024-09-17T04:25:07.090197+0000","last_active":"2024-09-17T04:25:08.124024+0000","last_peered":"2024-09-17T04:25:08.124024+0000","last_clean":"2024-09-17T04:25:08.124024+0000","last_became_active":"2024-09-17T04:25:07.089784+0000","last_became_peered":"2024-09-17T04:25:07.089784+0000","last_unstale":"2024-09-17T04:25:08.124024+0000","last_undegraded":"2024-09-17T04:25:08.124024+0000","last_fullsized":"2024-09-17T04:25:08.124024+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:09.843048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.083774+0000","last_change":"2024-09-17T04:25:07.083774+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.083774+0000","last_undegraded":"2024-09-17T04:25:07.083774+0000","last_fullsized":"2024-09-17T04:25:07.083774+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.100774+0000","last_change":"2024-09-17T04:25:07.100954+0000","last_active":"2024-09-17T04:25:08.100774+0000","last_peered":"2024-09-17T04:25:08.100774+0000","last_clean":"2024-09-17T04:25:08.100774+0000","last_became_active":"2024-09-17T04:25:07.100610+0000","last_became_peered":"2024-09-17T04:25:07.100610+0000","last_unstale":"2024-09-17T04:25:08.100774+0000","last_undegraded":"2024-09-17T04:25:08.100774+0000","last_fullsized":"2024-09-17T04:25:08.100774+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:14.912889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113053+0000","last_change":"2024-09-17T04:25:08.117150+0000","last_active":"2024-09-17T04:25:09.113053+0000","last_peered":"2024-09-17T04:25:09.113053+0000","last_clean":"2024-09-17T04:25:09.113053+0000","last_became_active":"2024-09-17T04:25:08.114530+0000","last_became_peered":"2024-09-17T04:25:08.114530+0000","last_unstale":"2024-09-17T04:25:09.113053+0000","last_undegraded":"2024-09-17T04:25:09.113053+0000","last_fullsized":"2024-09-17T04:25:09.113053+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:40.823059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T04:25:07.104424+0000","last_change":"2024-09-17T04:25:07.101663+0000","last_active":"2024-09-17T04:25:07.104424+0000","last_peered":"2024-09-17T04:25:07.104424+0000","last_clean":"2024-09-17T04:25:07.104424+0000","last_became_active":"2024-09-17T04:25:07.100447+0000","last_became_peered":"2024-09-17T04:25:07.100447+0000","last_unstale":"2024-09-17T04:25:07.104424+0000","last_undegraded":"2024-09-17T04:25:07.104424+0000","last_fullsized":"2024-09-17T04:25:07.104424+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:47.815107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.131919+0000","last_change":"2024-09-17T04:25:08.124593+0000","last_active":"2024-09-17T04:25:08.131919+0000","last_peered":"2024-09-17T04:25:08.131919+0000","last_clean":"2024-09-17T04:25:08.131919+0000","last_became_active":"2024-09-17T04:25:08.121257+0000","last_became_peered":"2024-09-17T04:25:08.121257+0000","last_unstale":"2024-09-17T04:25:08.131919+0000","last_undegraded":"2024-09-17T04:25:08.131919+0000","last_fullsized":"2024-09-17T04:25:08.131919+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:02.807745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T04:25:07.106797+0000","last_change":"2024-09-17T04:25:07.101694+0000","last_active":"2024-09-17T04:25:07.106797+0000","last_peered":"2024-09-17T04:25:07.106797+0000","last_clean":"2024-09-17T04:25:07.106797+0000","last_became_active":"2024-09-17T04:25:07.101006+0000","last_became_peered":"2024-09-17T04:25:07.101006+0000","last_unstale":"2024-09-17T04:25:07.106797+0000","last_undegraded":"2024-09-17T04:25:07.106797+0000","last_fullsized":"2024-09-17T04:25:07.106797+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:14:44.879016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.127019+0000","last_change":"2024-09-17T04:25:08.120978+0000","last_active":"2024-09-17T04:25:08.127019+0000","last_peered":"2024-09-17T04:25:08.127019+0000","last_clean":"2024-09-17T04:25:08.127019+0000","last_became_active":"2024-09-17T04:25:08.118804+0000","last_became_peered":"2024-09-17T04:25:08.118804+0000","last_unstale":"2024-09-17T04:25:08.127019+0000","last_undegraded":"2024-09-17T04:25:08.127019+0000","last_fullsized":"2024-09-17T04:25:08.127019+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:41.743123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.124335+0000","last_change":"2024-09-17T04:25:07.100101+0000","last_active":"2024-09-17T04:25:08.124335+0000","last_peered":"2024-09-17T04:25:08.124335+0000","last_clean":"2024-09-17T04:25:08.124335+0000","last_became_active":"2024-09-17T04:25:07.099657+0000","last_became_peered":"2024-09-17T04:25:07.099657+0000","last_unstale":"2024-09-17T04:25:08.124335+0000","last_undegraded":"2024-09-17T04:25:08.124335+0000","last_fullsized":"2024-09-17T04:25:08.124335+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:06:02.726891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113955+0000","last_change":"2024-09-17T04:25:08.123388+0000","last_active":"2024-09-17T04:25:09.113955+0000","last_peered":"2024-09-17T04:25:09.113955+0000","last_clean":"2024-09-17T04:25:09.113955+0000","last_became_active":"2024-09-17T04:25:08.122769+0000","last_became_peered":"2024-09-17T04:25:08.122769+0000","last_unstale":"2024-09-17T04:25:09.113955+0000","last_undegraded":"2024-09-17T04:25:09.113955+0000","last_fullsized":"2024-09-17T04:25:09.113955+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:52.735017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.118988+0000","last_change":"2024-09-17T04:25:07.089581+0000","last_active":"2024-09-17T04:25:08.118988+0000","last_peered":"2024-09-17T04:25:08.118988+0000","last_clean":"2024-09-17T04:25:08.118988+0000","last_became_active":"2024-09-17T04:25:07.089256+0000","last_became_peered":"2024-09-17T04:25:07.089256+0000","last_unstale":"2024-09-17T04:25:08.118988+0000","last_undegraded":"2024-09-17T04:25:08.118988+0000","last_fullsized":"2024-09-17T04:25:08.118988+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:11:57.535300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.132485+0000","last_change":"2024-09-17T04:25:08.127043+0000","last_active":"2024-09-17T04:25:08.132485+0000","last_peered":"2024-09-17T04:25:08.132485+0000","last_clean":"2024-09-17T04:25:08.132485+0000","last_became_active":"2024-09-17T04:25:08.126806+0000","last_became_peered":"2024-09-17T04:25:08.126806+0000","last_unstale":"2024-09-17T04:25:08.132485+0000","last_undegraded":"2024-09-17T04:25:08.132485+0000","last_fullsized":"2024-09-17T04:25:08.132485+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:03.317036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.123222+0000","last_change":"2024-09-17T04:25:07.098143+0000","last_active":"2024-09-17T04:25:08.123222+0000","last_peered":"2024-09-17T04:25:08.123222+0000","last_clean":"2024-09-17T04:25:08.123222+0000","last_became_active":"2024-09-17T04:25:07.091768+0000","last_became_peered":"2024-09-17T04:25:07.091768+0000","last_unstale":"2024-09-17T04:25:08.123222+0000","last_undegraded":"2024-09-17T04:25:08.123222+0000","last_fullsized":"2024-09-17T04:25:08.123222+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:15:18.524829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.118977+0000","last_change":"2024-09-17T04:25:08.108470+0000","last_active":"2024-09-17T04:25:08.118977+0000","last_peered":"2024-09-17T04:25:08.118977+0000","last_clean":"2024-09-17T04:25:08.118977+0000","last_became_active":"2024-09-17T04:25:08.108144+0000","last_became_peered":"2024-09-17T04:25:08.108144+0000","last_unstale":"2024-09-17T04:25:08.118977+0000","last_undegraded":"2024-09-17T04:25:08.118977+0000","last_fullsized":"2024-09-17T04:25:08.118977+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:51.768978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.100120+0000","last_change":"2024-09-17T04:25:07.103224+0000","last_active":"2024-09-17T04:25:08.100120+0000","last_peered":"2024-09-17T04:25:08.100120+0000","last_clean":"2024-09-17T04:25:08.100120+0000","last_became_active":"2024-09-17T04:25:07.101489+0000","last_became_peered":"2024-09-17T04:25:07.101489+0000","last_unstale":"2024-09-17T04:25:08.100120+0000","last_undegraded":"2024-09-17T04:25:08.100120+0000","last_fullsized":"2024-09-17T04:25:08.100120+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:08.430358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.122588+0000","last_change":"2024-09-17T04:25:08.112749+0000","last_active":"2024-09-17T04:25:08.122588+0000","last_peered":"2024-09-17T04:25:08.122588+0000","last_clean":"2024-09-17T04:25:08.122588+0000","last_became_active":"2024-09-17T04:25:08.112408+0000","last_became_peered":"2024-09-17T04:25:08.112408+0000","last_unstale":"2024-09-17T04:25:08.122588+0000","last_undegraded":"2024-09-17T04:25:08.122588+0000","last_fullsized":"2024-09-17T04:25:08.122588+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:07:25.567949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.123505+0000","last_change":"2024-09-17T04:25:07.092602+0000","last_active":"2024-09-17T04:25:08.123505+0000","last_peered":"2024-09-17T04:25:08.123505+0000","last_clean":"2024-09-17T04:25:08.123505+0000","last_became_active":"2024-09-17T04:25:07.092215+0000","last_became_peered":"2024-09-17T04:25:07.092215+0000","last_unstale":"2024-09-17T04:25:08.123505+0000","last_undegraded":"2024-09-17T04:25:08.123505+0000","last_fullsized":"2024-09-17T04:25:08.123505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:19.304290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.118395+0000","last_change":"2024-09-17T04:25:08.107413+0000","last_active":"2024-09-17T04:25:08.118395+0000","last_peered":"2024-09-17T04:25:08.118395+0000","last_clean":"2024-09-17T04:25:08.118395+0000","last_became_active":"2024-09-17T04:25:08.106516+0000","last_became_peered":"2024-09-17T04:25:08.106516+0000","last_unstale":"2024-09-17T04:25:08.118395+0000","last_undegraded":"2024-09-17T04:25:08.118395+0000","last_fullsized":"2024-09-17T04:25:08.118395+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:06.209252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.103107+0000","last_change":"2024-09-17T04:25:07.101091+0000","last_active":"2024-09-17T04:25:08.103107+0000","last_peered":"2024-09-17T04:25:08.103107+0000","last_clean":"2024-09-17T04:25:08.103107+0000","last_became_active":"2024-09-17T04:25:07.100821+0000","last_became_peered":"2024-09-17T04:25:07.100821+0000","last_unstale":"2024-09-17T04:25:08.103107+0000","last_undegraded":"2024-09-17T04:25:08.103107+0000","last_fullsized":"2024-09-17T04:25:08.103107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:50:25.153746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.108383+0000","last_change":"2024-09-17T04:25:08.104836+0000","last_active":"2024-09-17T04:25:08.108383+0000","last_peered":"2024-09-17T04:25:08.108383+0000","last_clean":"2024-09-17T04:25:08.108383+0000","last_became_active":"2024-09-17T04:25:08.104120+0000","last_became_peered":"2024-09-17T04:25:08.104120+0000","last_unstale":"2024-09-17T04:25:08.108383+0000","last_undegraded":"2024-09-17T04:25:08.108383+0000","last_fullsized":"2024-09-17T04:25:08.108383+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:05:27.192448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.097525+0000","last_change":"2024-09-17T04:25:07.103534+0000","last_active":"2024-09-17T04:25:08.097525+0000","last_peered":"2024-09-17T04:25:08.097525+0000","last_clean":"2024-09-17T04:25:08.097525+0000","last_became_active":"2024-09-17T04:25:07.103070+0000","last_became_peered":"2024-09-17T04:25:07.103070+0000","last_unstale":"2024-09-17T04:25:08.097525+0000","last_undegraded":"2024-09-17T04:25:08.097525+0000","last_fullsized":"2024-09-17T04:25:08.097525+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:57:15.056986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113955+0000","last_change":"2024-09-17T04:25:08.113398+0000","last_active":"2024-09-17T04:25:09.113955+0000","last_peered":"2024-09-17T04:25:09.113955+0000","last_clean":"2024-09-17T04:25:09.113955+0000","last_became_active":"2024-09-17T04:25:08.113049+0000","last_became_peered":"2024-09-17T04:25:08.113049+0000","last_unstale":"2024-09-17T04:25:09.113955+0000","last_undegraded":"2024-09-17T04:25:09.113955+0000","last_fullsized":"2024-09-17T04:25:09.113955+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:12.516316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.111837+0000","last_change":"2024-09-17T04:25:07.103861+0000","last_active":"2024-09-17T04:25:09.111837+0000","last_peered":"2024-09-17T04:25:09.111837+0000","last_clean":"2024-09-17T04:25:09.111837+0000","last_became_active":"2024-09-17T04:25:07.100484+0000","last_became_peered":"2024-09-17T04:25:07.100484+0000","last_unstale":"2024-09-17T04:25:09.111837+0000","last_undegraded":"2024-09-17T04:25:09.111837+0000","last_fullsized":"2024-09-17T04:25:09.111837+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:27.284841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.105143+0000","last_change":"2024-09-17T04:25:08.100426+0000","last_active":"2024-09-17T04:25:08.105143+0000","last_peered":"2024-09-17T04:25:08.105143+0000","last_clean":"2024-09-17T04:25:08.105143+0000","last_became_active":"2024-09-17T04:25:08.100117+0000","last_became_peered":"2024-09-17T04:25:08.100117+0000","last_unstale":"2024-09-17T04:25:08.105143+0000","last_undegraded":"2024-09-17T04:25:08.105143+0000","last_fullsized":"2024-09-17T04:25:08.105143+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:02.165986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.125748+0000","last_change":"2024-09-17T04:25:07.097884+0000","last_active":"2024-09-17T04:25:08.125748+0000","last_peered":"2024-09-17T04:25:08.125748+0000","last_clean":"2024-09-17T04:25:08.125748+0000","last_became_active":"2024-09-17T04:25:07.097545+0000","last_became_peered":"2024-09-17T04:25:07.097545+0000","last_unstale":"2024-09-17T04:25:08.125748+0000","last_undegraded":"2024-09-17T04:25:08.125748+0000","last_fullsized":"2024-09-17T04:25:08.125748+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:01.801695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.114256+0000","last_change":"2024-09-17T04:25:08.123061+0000","last_active":"2024-09-17T04:25:09.114256+0000","last_peered":"2024-09-17T04:25:09.114256+0000","last_clean":"2024-09-17T04:25:09.114256+0000","last_became_active":"2024-09-17T04:25:08.122319+0000","last_became_peered":"2024-09-17T04:25:08.122319+0000","last_unstale":"2024-09-17T04:25:09.114256+0000","last_undegraded":"2024-09-17T04:25:09.114256+0000","last_fullsized":"2024-09-17T04:25:09.114256+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:22.538564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.094731+0000","last_change":"2024-09-17T04:25:07.088222+0000","last_active":"2024-09-17T04:25:08.094731+0000","last_peered":"2024-09-17T04:25:08.094731+0000","last_clean":"2024-09-17T04:25:08.094731+0000","last_became_active":"2024-09-17T04:25:07.087921+0000","last_became_peered":"2024-09-17T04:25:07.087921+0000","last_unstale":"2024-09-17T04:25:08.094731+0000","last_undegraded":"2024-09-17T04:25:08.094731+0000","last_fullsized":"2024-09-17T04:25:08.094731+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:35.879479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.116348+0000","last_change":"2024-09-17T04:25:08.115284+0000","last_active":"2024-09-17T04:25:08.116348+0000","last_peered":"2024-09-17T04:25:08.116348+0000","last_clean":"2024-09-17T04:25:08.116348+0000","last_became_active":"2024-09-17T04:25:08.114998+0000","last_became_peered":"2024-09-17T04:25:08.114998+0000","last_unstale":"2024-09-17T04:25:08.116348+0000","last_undegraded":"2024-09-17T04:25:08.116348+0000","last_fullsized":"2024-09-17T04:25:08.116348+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:40:24.798987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.100440+0000","last_change":"2024-09-17T04:25:07.096188+0000","last_active":"2024-09-17T04:25:08.100440+0000","last_peered":"2024-09-17T04:25:08.100440+0000","last_clean":"2024-09-17T04:25:08.100440+0000","last_became_active":"2024-09-17T04:25:07.094697+0000","last_became_peered":"2024-09-17T04:25:07.094697+0000","last_unstale":"2024-09-17T04:25:08.100440+0000","last_undegraded":"2024-09-17T04:25:08.100440+0000","last_fullsized":"2024-09-17T04:25:08.100440+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:05.174227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.120861+0000","last_change":"2024-09-17T04:25:08.115704+0000","last_active":"2024-09-17T04:25:08.120861+0000","last_peered":"2024-09-17T04:25:08.120861+0000","last_clean":"2024-09-17T04:25:08.120861+0000","last_became_active":"2024-09-17T04:25:08.115380+0000","last_became_peered":"2024-09-17T04:25:08.115380+0000","last_unstale":"2024-09-17T04:25:08.120861+0000","last_undegraded":"2024-09-17T04:25:08.120861+0000","last_fullsized":"2024-09-17T04:25:08.120861+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:21.618822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.097074+0000","last_change":"2024-09-17T04:25:07.095160+0000","last_active":"2024-09-17T04:25:08.097074+0000","last_peered":"2024-09-17T04:25:08.097074+0000","last_clean":"2024-09-17T04:25:08.097074+0000","last_became_active":"2024-09-17T04:25:07.094948+0000","last_became_peered":"2024-09-17T04:25:07.094948+0000","last_unstale":"2024-09-17T04:25:08.097074+0000","last_undegraded":"2024-09-17T04:25:08.097074+0000","last_fullsized":"2024-09-17T04:25:08.097074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:40.737074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.123596+0000","last_change":"2024-09-17T04:25:08.120140+0000","last_active":"2024-09-17T04:25:08.123596+0000","last_peered":"2024-09-17T04:25:08.123596+0000","last_clean":"2024-09-17T04:25:08.123596+0000","last_became_active":"2024-09-17T04:25:08.118074+0000","last_became_peered":"2024-09-17T04:25:08.118074+0000","last_unstale":"2024-09-17T04:25:08.123596+0000","last_undegraded":"2024-09-17T04:25:08.123596+0000","last_fullsized":"2024-09-17T04:25:08.123596+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:15:30.368718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.100996+0000","last_change":"2024-09-17T04:25:07.094913+0000","last_active":"2024-09-17T04:25:08.100996+0000","last_peered":"2024-09-17T04:25:08.100996+0000","last_clean":"2024-09-17T04:25:08.100996+0000","last_became_active":"2024-09-17T04:25:07.094618+0000","last_became_peered":"2024-09-17T04:25:07.094618+0000","last_unstale":"2024-09-17T04:25:08.100996+0000","last_undegraded":"2024-09-17T04:25:08.100996+0000","last_fullsized":"2024-09-17T04:25:08.100996+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:20.612773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.081356+0000","last_change":"2024-09-17T04:25:07.081356+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.081356+0000","last_undegraded":"2024-09-17T04:25:07.081356+0000","last_fullsized":"2024-09-17T04:25:07.081356+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.097817+0000","last_change":"2024-09-17T04:25:07.093510+0000","last_active":"2024-09-17T04:25:08.097817+0000","last_peered":"2024-09-17T04:25:08.097817+0000","last_clean":"2024-09-17T04:25:08.097817+0000","last_became_active":"2024-09-17T04:25:07.093132+0000","last_became_peered":"2024-09-17T04:25:07.093132+0000","last_unstale":"2024-09-17T04:25:08.097817+0000","last_undegraded":"2024-09-17T04:25:08.097817+0000","last_fullsized":"2024-09-17T04:25:08.097817+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:04:39.641973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.132867+0000","last_change":"2024-09-17T04:25:08.127561+0000","last_active":"2024-09-17T04:25:08.132867+0000","last_peered":"2024-09-17T04:25:08.132867+0000","last_clean":"2024-09-17T04:25:08.132867+0000","last_became_active":"2024-09-17T04:25:08.127108+0000","last_became_peered":"2024-09-17T04:25:08.127108+0000","last_unstale":"2024-09-17T04:25:08.132867+0000","last_undegraded":"2024-09-17T04:25:08.132867+0000","last_fullsized":"2024-09-17T04:25:08.132867+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:40:24.798987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T04:25:07.104421+0000","last_change":"2024-09-17T04:25:07.101483+0000","last_active":"2024-09-17T04:25:07.104421+0000","last_peered":"2024-09-17T04:25:07.104421+0000","last_clean":"2024-09-17T04:25:07.104421+0000","last_became_active":"2024-09-17T04:25:07.100021+0000","last_became_peered":"2024-09-17T04:25:07.100021+0000","last_unstale":"2024-09-17T04:25:07.104421+0000","last_undegraded":"2024-09-17T04:25:07.104421+0000","last_fullsized":"2024-09-17T04:25:07.104421+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:30:05.938041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112185+0000","last_change":"2024-09-17T04:25:08.102453+0000","last_active":"2024-09-17T04:25:09.112185+0000","last_peered":"2024-09-17T04:25:09.112185+0000","last_clean":"2024-09-17T04:25:09.112185+0000","last_became_active":"2024-09-17T04:25:08.101181+0000","last_became_peered":"2024-09-17T04:25:08.101181+0000","last_unstale":"2024-09-17T04:25:09.112185+0000","last_undegraded":"2024-09-17T04:25:09.112185+0000","last_fullsized":"2024-09-17T04:25:09.112185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:11.397751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.103937+0000","last_change":"2024-09-17T04:25:07.105074+0000","last_active":"2024-09-17T04:25:08.103937+0000","last_peered":"2024-09-17T04:25:08.103937+0000","last_clean":"2024-09-17T04:25:08.103937+0000","last_became_active":"2024-09-17T04:25:07.104735+0000","last_became_peered":"2024-09-17T04:25:07.104735+0000","last_unstale":"2024-09-17T04:25:08.103937+0000","last_undegraded":"2024-09-17T04:25:08.103937+0000","last_fullsized":"2024-09-17T04:25:08.103937+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:06:46.154246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":136,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113124+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:25:09.113124+0000","last_peered":"2024-09-17T04:25:09.113124+0000","last_clean":"2024-09-17T04:25:09.113124+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:25:09.113124+0000","last_undegraded":"2024-09-17T04:25:09.113124+0000","last_fullsized":"2024-09-17T04:25:09.113124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22,"ondisk_log_size":22,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476754,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":739667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476753,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":149825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476754,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":149881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476754,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":393216,"data_stored":149313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":372736,"data_stored":142434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509456,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":372736,"data_stored":142434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":966656,"data_stored":732802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":360448,"data_stored":136167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:25:13.032 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:25:13.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.290+0000 7f4af4000640 1 -- 172.21.15.100:0/1675711107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ad4003a00 msgr2=0x7f4ad4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:13.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.290+0000 7f4af4000640 1 --2- 172.21.15.100:0/1675711107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ad4003a00 0x7f4ad4003e30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009b50 tx=0x7f4ad4005830 comp rx=0 tx=0).stop 2024-09-17T04:25:13.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.291+0000 7f4af4000640 1 -- 172.21.15.100:0/1675711107 shutdown_connections 2024-09-17T04:25:13.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.291+0000 7f4af4000640 1 --2- 172.21.15.100:0/1675711107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ad4003a00 0x7f4ad4003e30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:13.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.291+0000 7f4af4000640 1 -- 172.21.15.100:0/1675711107 >> 172.21.15.100:0/1675711107 conn(0x7f4aec06d730 msgr2=0x7f4aec0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:13.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.291+0000 7f4af4000640 1 -- 172.21.15.100:0/1675711107 shutdown_connections 2024-09-17T04:25:13.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.291+0000 7f4af4000640 1 -- 172.21.15.100:0/1675711107 wait complete. 2024-09-17T04:25:13.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.293+0000 7f4af4000640 1 Processor -- start 2024-09-17T04:25:13.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.293+0000 7f4af4000640 1 -- start start 2024-09-17T04:25:13.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.294+0000 7f4af4000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ad4003a00 0x7f4aec1ea190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:13.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.294+0000 7f4af4000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4aec156b40 0x7f4aec0b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:13.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.294+0000 7f4af4000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4aec0b45a0 0x7f4aec159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:13.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.295+0000 7f4af4000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec06f0a0 con 0x7f4aec156b40 2024-09-17T04:25:13.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.295+0000 7f4af4000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec073760 con 0x7f4aec0b45a0 2024-09-17T04:25:13.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.295+0000 7f4af4000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4aec06f220 con 0x7f4ad4003a00 2024-09-17T04:25:13.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.295+0000 7f4af0c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4aec156b40 0x7f4aec0b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:13.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.295+0000 7f4af0c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4aec156b40 0x7f4aec0b4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38970/0 (socket says 172.21.15.100:38970) 2024-09-17T04:25:13.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.295+0000 7f4af0c00640 1 -- 172.21.15.100:0/857533512 learned_addr learned my addr 172.21.15.100:0/857533512 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:13.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.295+0000 7f4af1600640 1 --2- 172.21.15.100:0/857533512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ad4003a00 0x7f4aec1ea190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:13.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.295+0000 7f4af2000640 1 --2- 172.21.15.100:0/857533512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4aec0b45a0 0x7f4aec159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:13.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.295+0000 7f4af0c00640 1 -- 172.21.15.100:0/857533512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ad4003a00 msgr2=0x7f4aec1ea190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:13.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.295+0000 7f4af0c00640 1 --2- 172.21.15.100:0/857533512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ad4003a00 0x7f4aec1ea190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:13.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.295+0000 7f4af0c00640 1 -- 172.21.15.100:0/857533512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4aec0b45a0 msgr2=0x7f4aec159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:13.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.295+0000 7f4af0c00640 1 --2- 172.21.15.100:0/857533512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4aec0b45a0 0x7f4aec159a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:13.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.295+0000 7f4af0c00640 1 -- 172.21.15.100:0/857533512 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aec0b49b0 con 0x7f4aec156b40 2024-09-17T04:25:13.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.295+0000 7f4af1600640 1 --2- 172.21.15.100:0/857533512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ad4003a00 0x7f4aec1ea190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:13.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.295+0000 7f4af0c00640 1 --2- 172.21.15.100:0/857533512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4aec156b40 0x7f4aec0b4060 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8006e60 tx=0x7f4ad80049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:13.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.296+0000 7f4ae9600640 1 -- 172.21.15.100:0/857533512 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8015020 con 0x7f4aec156b40 2024-09-17T04:25:13.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.296+0000 7f4ae9600640 1 -- 172.21.15.100:0/857533512 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ad8010980 con 0x7f4aec156b40 2024-09-17T04:25:13.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.296+0000 7f4ae9600640 1 -- 172.21.15.100:0/857533512 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8010d90 con 0x7f4aec156b40 2024-09-17T04:25:13.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.296+0000 7f4af4000640 1 -- 172.21.15.100:0/857533512 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aec155aa0 con 0x7f4aec156b40 2024-09-17T04:25:13.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.296+0000 7f4af4000640 1 -- 172.21.15.100:0/857533512 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aec156060 con 0x7f4aec156b40 2024-09-17T04:25:13.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.297+0000 7f4af4000640 1 -- 172.21.15.100:0/857533512 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aec162560 con 0x7f4aec156b40 2024-09-17T04:25:13.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.298+0000 7f4ae9600640 1 -- 172.21.15.100:0/857533512 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4ad8018020 con 0x7f4aec156b40 2024-09-17T04:25:13.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.299+0000 7f4ae9600640 1 --2- 172.21.15.100:0/857533512 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4acc078b90 0x7f4acc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:13.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.299+0000 7f4ae9600640 1 -- 172.21.15.100:0/857533512 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f4ad809ba60 con 0x7f4aec156b40 2024-09-17T04:25:13.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.300+0000 7f4af1600640 1 --2- 172.21.15.100:0/857533512 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4acc078b90 0x7f4acc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:13.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.300+0000 7f4af1600640 1 --2- 172.21.15.100:0/857533512 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4acc078b90 0x7f4acc07b0a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0035000 tx=0x7f4ae00046c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:13.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.303+0000 7f4ae9600640 1 -- 172.21.15.100:0/857533512 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4ad801f020 con 0x7f4aec156b40 2024-09-17T04:25:13.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.484+0000 7f4af4000640 1 -- 172.21.15.100:0/857533512 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4aec0b22d0 con 0x7f4acc078b90 2024-09-17T04:25:13.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.489+0000 7f4ae9600640 1 -- 172.21.15.100:0/857533512 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251103 (secure 0 0 0) 0x7f4aec0b22d0 con 0x7f4acc078b90 2024-09-17T04:25:13.492 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:13.493 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:25:13.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.493+0000 7f4af4000640 1 -- 172.21.15.100:0/857533512 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4acc078b90 msgr2=0x7f4acc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:13.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.493+0000 7f4af4000640 1 --2- 172.21.15.100:0/857533512 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4acc078b90 0x7f4acc07b0a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0035000 tx=0x7f4ae00046c0 comp rx=0 tx=0).stop 2024-09-17T04:25:13.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.493+0000 7f4af4000640 1 -- 172.21.15.100:0/857533512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4aec156b40 msgr2=0x7f4aec0b4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:13.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.493+0000 7f4af4000640 1 --2- 172.21.15.100:0/857533512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4aec156b40 0x7f4aec0b4060 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8006e60 tx=0x7f4ad80049d0 comp rx=0 tx=0).stop 2024-09-17T04:25:13.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.493+0000 7f4af4000640 1 -- 172.21.15.100:0/857533512 shutdown_connections 2024-09-17T04:25:13.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.493+0000 7f4af4000640 1 --2- 172.21.15.100:0/857533512 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4acc078b90 0x7f4acc07b0a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:13.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.493+0000 7f4af4000640 1 --2- 172.21.15.100:0/857533512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4aec0b45a0 0x7f4aec159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:13.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.493+0000 7f4af4000640 1 --2- 172.21.15.100:0/857533512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4aec156b40 0x7f4aec0b4060 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:13.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.493+0000 7f4af4000640 1 --2- 172.21.15.100:0/857533512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ad4003a00 0x7f4aec1ea190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:13.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.493+0000 7f4af4000640 1 -- 172.21.15.100:0/857533512 >> 172.21.15.100:0/857533512 conn(0x7f4aec06d730 msgr2=0x7f4aec160bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:13.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.493+0000 7f4af4000640 1 -- 172.21.15.100:0/857533512 shutdown_connections 2024-09-17T04:25:13.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.493+0000 7f4af4000640 1 -- 172.21.15.100:0/857533512 wait complete. 2024-09-17T04:25:13.518 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-09-17T04:25:11.979707+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591166,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":124,"num_write_kb":1639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94,"ondisk_log_size":94,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":96,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220372,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754348,"statfs":{"total":773094113280,"available":772868452352,"internally_reserved":0,"allocated":4235264,"data_stored":2342523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.910164"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.131931+0000","last_change":"2024-09-17T04:25:08.126623+0000","last_active":"2024-09-17T04:25:08.131931+0000","last_peered":"2024-09-17T04:25:08.131931+0000","last_clean":"2024-09-17T04:25:08.131931+0000","last_became_active":"2024-09-17T04:25:08.126204+0000","last_became_peered":"2024-09-17T04:25:08.126204+0000","last_unstale":"2024-09-17T04:25:08.131931+0000","last_undegraded":"2024-09-17T04:25:08.131931+0000","last_fullsized":"2024-09-17T04:25:08.131931+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:40.098175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.088364+0000","last_change":"2024-09-17T04:25:07.088364+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.088364+0000","last_undegraded":"2024-09-17T04:25:07.088364+0000","last_fullsized":"2024-09-17T04:25:07.088364+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.114386+0000","last_change":"2024-09-17T04:25:08.116641+0000","last_active":"2024-09-17T04:25:09.114386+0000","last_peered":"2024-09-17T04:25:09.114386+0000","last_clean":"2024-09-17T04:25:09.114386+0000","last_became_active":"2024-09-17T04:25:08.115852+0000","last_became_peered":"2024-09-17T04:25:08.115852+0000","last_unstale":"2024-09-17T04:25:09.114386+0000","last_undegraded":"2024-09-17T04:25:09.114386+0000","last_fullsized":"2024-09-17T04:25:09.114386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:46.534540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.132543+0000","last_change":"2024-09-17T04:25:08.127729+0000","last_active":"2024-09-17T04:25:08.132543+0000","last_peered":"2024-09-17T04:25:08.132543+0000","last_clean":"2024-09-17T04:25:08.132543+0000","last_became_active":"2024-09-17T04:25:08.125546+0000","last_became_peered":"2024-09-17T04:25:08.125546+0000","last_unstale":"2024-09-17T04:25:08.132543+0000","last_undegraded":"2024-09-17T04:25:08.132543+0000","last_fullsized":"2024-09-17T04:25:08.132543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:01:06.440885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.122163+0000","last_change":"2024-09-17T04:25:08.113523+0000","last_active":"2024-09-17T04:25:08.122163+0000","last_peered":"2024-09-17T04:25:08.122163+0000","last_clean":"2024-09-17T04:25:08.122163+0000","last_became_active":"2024-09-17T04:25:08.113207+0000","last_became_peered":"2024-09-17T04:25:08.113207+0000","last_unstale":"2024-09-17T04:25:08.122163+0000","last_undegraded":"2024-09-17T04:25:08.122163+0000","last_fullsized":"2024-09-17T04:25:08.122163+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:28.634098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.083605+0000","last_change":"2024-09-17T04:25:07.083605+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.083605+0000","last_undegraded":"2024-09-17T04:25:07.083605+0000","last_fullsized":"2024-09-17T04:25:07.083605+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113556+0000","last_change":"2024-09-17T04:25:08.113471+0000","last_active":"2024-09-17T04:25:09.113556+0000","last_peered":"2024-09-17T04:25:09.113556+0000","last_clean":"2024-09-17T04:25:09.113556+0000","last_became_active":"2024-09-17T04:25:08.113145+0000","last_became_peered":"2024-09-17T04:25:08.113145+0000","last_unstale":"2024-09-17T04:25:09.113556+0000","last_undegraded":"2024-09-17T04:25:09.113556+0000","last_fullsized":"2024-09-17T04:25:09.113556+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:52.755680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112954+0000","last_change":"2024-09-17T04:25:08.102460+0000","last_active":"2024-09-17T04:25:09.112954+0000","last_peered":"2024-09-17T04:25:09.112954+0000","last_clean":"2024-09-17T04:25:09.112954+0000","last_became_active":"2024-09-17T04:25:08.101474+0000","last_became_peered":"2024-09-17T04:25:08.101474+0000","last_unstale":"2024-09-17T04:25:09.112954+0000","last_undegraded":"2024-09-17T04:25:09.112954+0000","last_fullsized":"2024-09-17T04:25:09.112954+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:22.538564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.126318+0000","last_change":"2024-09-17T04:25:08.119819+0000","last_active":"2024-09-17T04:25:08.126318+0000","last_peered":"2024-09-17T04:25:08.126318+0000","last_clean":"2024-09-17T04:25:08.126318+0000","last_became_active":"2024-09-17T04:25:08.119511+0000","last_became_peered":"2024-09-17T04:25:08.119511+0000","last_unstale":"2024-09-17T04:25:08.126318+0000","last_undegraded":"2024-09-17T04:25:08.126318+0000","last_fullsized":"2024-09-17T04:25:08.126318+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:49:01.278096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.124020+0000","last_change":"2024-09-17T04:25:08.121720+0000","last_active":"2024-09-17T04:25:08.124020+0000","last_peered":"2024-09-17T04:25:08.124020+0000","last_clean":"2024-09-17T04:25:08.124020+0000","last_became_active":"2024-09-17T04:25:08.121405+0000","last_became_peered":"2024-09-17T04:25:08.121405+0000","last_unstale":"2024-09-17T04:25:08.124020+0000","last_undegraded":"2024-09-17T04:25:08.124020+0000","last_fullsized":"2024-09-17T04:25:08.124020+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:37.261699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112530+0000","last_change":"2024-09-17T04:25:08.116374+0000","last_active":"2024-09-17T04:25:09.112530+0000","last_peered":"2024-09-17T04:25:09.112530+0000","last_clean":"2024-09-17T04:25:09.112530+0000","last_became_active":"2024-09-17T04:25:08.116047+0000","last_became_peered":"2024-09-17T04:25:08.116047+0000","last_unstale":"2024-09-17T04:25:09.112530+0000","last_undegraded":"2024-09-17T04:25:09.112530+0000","last_fullsized":"2024-09-17T04:25:09.112530+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:12:03.863249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113687+0000","last_change":"2024-09-17T04:25:08.117853+0000","last_active":"2024-09-17T04:25:09.113687+0000","last_peered":"2024-09-17T04:25:09.113687+0000","last_clean":"2024-09-17T04:25:09.113687+0000","last_became_active":"2024-09-17T04:25:08.117342+0000","last_became_peered":"2024-09-17T04:25:08.117342+0000","last_unstale":"2024-09-17T04:25:09.113687+0000","last_undegraded":"2024-09-17T04:25:09.113687+0000","last_fullsized":"2024-09-17T04:25:09.113687+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:59.322143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.118578+0000","last_change":"2024-09-17T04:25:08.107583+0000","last_active":"2024-09-17T04:25:08.118578+0000","last_peered":"2024-09-17T04:25:08.118578+0000","last_clean":"2024-09-17T04:25:08.118578+0000","last_became_active":"2024-09-17T04:25:08.107097+0000","last_became_peered":"2024-09-17T04:25:08.107097+0000","last_unstale":"2024-09-17T04:25:08.118578+0000","last_undegraded":"2024-09-17T04:25:08.118578+0000","last_fullsized":"2024-09-17T04:25:08.118578+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:37.405465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.083133+0000","last_change":"2024-09-17T04:25:07.083133+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.083133+0000","last_undegraded":"2024-09-17T04:25:07.083133+0000","last_fullsized":"2024-09-17T04:25:07.083133+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112471+0000","last_change":"2024-09-17T04:25:08.122562+0000","last_active":"2024-09-17T04:25:09.112471+0000","last_peered":"2024-09-17T04:25:09.112471+0000","last_clean":"2024-09-17T04:25:09.112471+0000","last_became_active":"2024-09-17T04:25:08.120406+0000","last_became_peered":"2024-09-17T04:25:08.120406+0000","last_unstale":"2024-09-17T04:25:09.112471+0000","last_undegraded":"2024-09-17T04:25:09.112471+0000","last_fullsized":"2024-09-17T04:25:09.112471+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:14.660711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113172+0000","last_change":"2024-09-17T04:25:08.111402+0000","last_active":"2024-09-17T04:25:09.113172+0000","last_peered":"2024-09-17T04:25:09.113172+0000","last_clean":"2024-09-17T04:25:09.113172+0000","last_became_active":"2024-09-17T04:25:08.111049+0000","last_became_peered":"2024-09-17T04:25:08.111049+0000","last_unstale":"2024-09-17T04:25:09.113172+0000","last_undegraded":"2024-09-17T04:25:09.113172+0000","last_fullsized":"2024-09-17T04:25:09.113172+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:46.534540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112688+0000","last_change":"2024-09-17T04:25:08.113695+0000","last_active":"2024-09-17T04:25:09.112688+0000","last_peered":"2024-09-17T04:25:09.112688+0000","last_clean":"2024-09-17T04:25:09.112688+0000","last_became_active":"2024-09-17T04:25:08.113248+0000","last_became_peered":"2024-09-17T04:25:08.113248+0000","last_unstale":"2024-09-17T04:25:09.112688+0000","last_undegraded":"2024-09-17T04:25:09.112688+0000","last_fullsized":"2024-09-17T04:25:09.112688+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:52.735017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.087894+0000","last_change":"2024-09-17T04:25:07.087894+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.087894+0000","last_undegraded":"2024-09-17T04:25:07.087894+0000","last_fullsized":"2024-09-17T04:25:07.087894+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.084115+0000","last_change":"2024-09-17T04:25:07.084115+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.084115+0000","last_undegraded":"2024-09-17T04:25:07.084115+0000","last_fullsized":"2024-09-17T04:25:07.084115+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.107540+0000","last_change":"2024-09-17T04:25:08.102248+0000","last_active":"2024-09-17T04:25:08.107540+0000","last_peered":"2024-09-17T04:25:08.107540+0000","last_clean":"2024-09-17T04:25:08.107540+0000","last_became_active":"2024-09-17T04:25:08.101915+0000","last_became_peered":"2024-09-17T04:25:08.101915+0000","last_unstale":"2024-09-17T04:25:08.107540+0000","last_undegraded":"2024-09-17T04:25:08.107540+0000","last_fullsized":"2024-09-17T04:25:08.107540+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:11.459548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.125029+0000","last_change":"2024-09-17T04:25:08.116792+0000","last_active":"2024-09-17T04:25:08.125029+0000","last_peered":"2024-09-17T04:25:08.125029+0000","last_clean":"2024-09-17T04:25:08.125029+0000","last_became_active":"2024-09-17T04:25:08.116468+0000","last_became_peered":"2024-09-17T04:25:08.116468+0000","last_unstale":"2024-09-17T04:25:08.125029+0000","last_undegraded":"2024-09-17T04:25:08.125029+0000","last_fullsized":"2024-09-17T04:25:08.125029+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:01.832590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.132325+0000","last_change":"2024-09-17T04:25:08.125202+0000","last_active":"2024-09-17T04:25:08.132325+0000","last_peered":"2024-09-17T04:25:08.132325+0000","last_clean":"2024-09-17T04:25:08.132325+0000","last_became_active":"2024-09-17T04:25:08.124880+0000","last_became_peered":"2024-09-17T04:25:08.124880+0000","last_unstale":"2024-09-17T04:25:08.132325+0000","last_undegraded":"2024-09-17T04:25:08.132325+0000","last_fullsized":"2024-09-17T04:25:08.132325+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:33.145063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.115053+0000","last_change":"2024-09-17T04:25:08.109076+0000","last_active":"2024-09-17T04:25:08.115053+0000","last_peered":"2024-09-17T04:25:08.115053+0000","last_clean":"2024-09-17T04:25:08.115053+0000","last_became_active":"2024-09-17T04:25:08.105699+0000","last_became_peered":"2024-09-17T04:25:08.105699+0000","last_unstale":"2024-09-17T04:25:08.115053+0000","last_undegraded":"2024-09-17T04:25:08.115053+0000","last_fullsized":"2024-09-17T04:25:08.115053+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:33.145063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.111966+0000","last_change":"2024-09-17T04:25:08.099256+0000","last_active":"2024-09-17T04:25:09.111966+0000","last_peered":"2024-09-17T04:25:09.111966+0000","last_clean":"2024-09-17T04:25:09.111966+0000","last_became_active":"2024-09-17T04:25:08.098874+0000","last_became_peered":"2024-09-17T04:25:08.098874+0000","last_unstale":"2024-09-17T04:25:09.111966+0000","last_undegraded":"2024-09-17T04:25:09.111966+0000","last_fullsized":"2024-09-17T04:25:09.111966+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:11.397751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.122652+0000","last_change":"2024-09-17T04:25:08.117901+0000","last_active":"2024-09-17T04:25:08.122652+0000","last_peered":"2024-09-17T04:25:08.122652+0000","last_clean":"2024-09-17T04:25:08.122652+0000","last_became_active":"2024-09-17T04:25:08.115468+0000","last_became_peered":"2024-09-17T04:25:08.115468+0000","last_unstale":"2024-09-17T04:25:08.122652+0000","last_undegraded":"2024-09-17T04:25:08.122652+0000","last_fullsized":"2024-09-17T04:25:08.122652+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:06.724919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.085167+0000","last_change":"2024-09-17T04:25:07.085167+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.085167+0000","last_undegraded":"2024-09-17T04:25:07.085167+0000","last_fullsized":"2024-09-17T04:25:07.085167+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.129679+0000","last_change":"2024-09-17T04:25:08.126392+0000","last_active":"2024-09-17T04:25:08.129679+0000","last_peered":"2024-09-17T04:25:08.129679+0000","last_clean":"2024-09-17T04:25:08.129679+0000","last_became_active":"2024-09-17T04:25:08.126046+0000","last_became_peered":"2024-09-17T04:25:08.126046+0000","last_unstale":"2024-09-17T04:25:08.129679+0000","last_undegraded":"2024-09-17T04:25:08.129679+0000","last_fullsized":"2024-09-17T04:25:08.129679+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:30.659131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113961+0000","last_change":"2024-09-17T04:25:08.104557+0000","last_active":"2024-09-17T04:25:09.113961+0000","last_peered":"2024-09-17T04:25:09.113961+0000","last_clean":"2024-09-17T04:25:09.113961+0000","last_became_active":"2024-09-17T04:25:08.104246+0000","last_became_peered":"2024-09-17T04:25:08.104246+0000","last_unstale":"2024-09-17T04:25:09.113961+0000","last_undegraded":"2024-09-17T04:25:09.113961+0000","last_fullsized":"2024-09-17T04:25:09.113961+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:26.289675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.132086+0000","last_change":"2024-09-17T04:25:08.124218+0000","last_active":"2024-09-17T04:25:08.132086+0000","last_peered":"2024-09-17T04:25:08.132086+0000","last_clean":"2024-09-17T04:25:08.132086+0000","last_became_active":"2024-09-17T04:25:08.123746+0000","last_became_peered":"2024-09-17T04:25:08.123746+0000","last_unstale":"2024-09-17T04:25:08.132086+0000","last_undegraded":"2024-09-17T04:25:08.132086+0000","last_fullsized":"2024-09-17T04:25:08.132086+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:02.874046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112282+0000","last_change":"2024-09-17T04:25:08.113465+0000","last_active":"2024-09-17T04:25:09.112282+0000","last_peered":"2024-09-17T04:25:09.112282+0000","last_clean":"2024-09-17T04:25:09.112282+0000","last_became_active":"2024-09-17T04:25:08.113106+0000","last_became_peered":"2024-09-17T04:25:08.113106+0000","last_unstale":"2024-09-17T04:25:09.112282+0000","last_undegraded":"2024-09-17T04:25:09.112282+0000","last_fullsized":"2024-09-17T04:25:09.112282+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:14.660711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.120556+0000","last_change":"2024-09-17T04:25:08.111799+0000","last_active":"2024-09-17T04:25:08.120556+0000","last_peered":"2024-09-17T04:25:08.120556+0000","last_clean":"2024-09-17T04:25:08.120556+0000","last_became_active":"2024-09-17T04:25:08.111338+0000","last_became_peered":"2024-09-17T04:25:08.111338+0000","last_unstale":"2024-09-17T04:25:08.120556+0000","last_undegraded":"2024-09-17T04:25:08.120556+0000","last_fullsized":"2024-09-17T04:25:08.120556+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:15.918939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112824+0000","last_change":"2024-09-17T04:25:08.109129+0000","last_active":"2024-09-17T04:25:09.112824+0000","last_peered":"2024-09-17T04:25:09.112824+0000","last_clean":"2024-09-17T04:25:09.112824+0000","last_became_active":"2024-09-17T04:25:08.104873+0000","last_became_peered":"2024-09-17T04:25:08.104873+0000","last_unstale":"2024-09-17T04:25:09.112824+0000","last_undegraded":"2024-09-17T04:25:09.112824+0000","last_fullsized":"2024-09-17T04:25:09.112824+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:48.676959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.112046+0000","last_change":"2024-09-17T04:25:08.104171+0000","last_active":"2024-09-17T04:25:08.112046+0000","last_peered":"2024-09-17T04:25:08.112046+0000","last_clean":"2024-09-17T04:25:08.112046+0000","last_became_active":"2024-09-17T04:25:08.103831+0000","last_became_peered":"2024-09-17T04:25:08.103831+0000","last_unstale":"2024-09-17T04:25:08.112046+0000","last_undegraded":"2024-09-17T04:25:08.112046+0000","last_fullsized":"2024-09-17T04:25:08.112046+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:02.874046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.132361+0000","last_change":"2024-09-17T04:25:07.101542+0000","last_active":"2024-09-17T04:25:08.132361+0000","last_peered":"2024-09-17T04:25:08.132361+0000","last_clean":"2024-09-17T04:25:08.132361+0000","last_became_active":"2024-09-17T04:25:07.101168+0000","last_became_peered":"2024-09-17T04:25:07.101168+0000","last_unstale":"2024-09-17T04:25:08.132361+0000","last_undegraded":"2024-09-17T04:25:08.132361+0000","last_fullsized":"2024-09-17T04:25:08.132361+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:05.174227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.127811+0000","last_change":"2024-09-17T04:25:08.122307+0000","last_active":"2024-09-17T04:25:08.127811+0000","last_peered":"2024-09-17T04:25:08.127811+0000","last_clean":"2024-09-17T04:25:08.127811+0000","last_became_active":"2024-09-17T04:25:08.121816+0000","last_became_peered":"2024-09-17T04:25:08.121816+0000","last_unstale":"2024-09-17T04:25:08.127811+0000","last_undegraded":"2024-09-17T04:25:08.127811+0000","last_fullsized":"2024-09-17T04:25:08.127811+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:06:03.732941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.121882+0000","last_change":"2024-09-17T04:25:07.090513+0000","last_active":"2024-09-17T04:25:08.121882+0000","last_peered":"2024-09-17T04:25:08.121882+0000","last_clean":"2024-09-17T04:25:08.121882+0000","last_became_active":"2024-09-17T04:25:07.090181+0000","last_became_peered":"2024-09-17T04:25:07.090181+0000","last_unstale":"2024-09-17T04:25:08.121882+0000","last_undegraded":"2024-09-17T04:25:08.121882+0000","last_fullsized":"2024-09-17T04:25:08.121882+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:00.826540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.126484+0000","last_change":"2024-09-17T04:25:08.118894+0000","last_active":"2024-09-17T04:25:08.126484+0000","last_peered":"2024-09-17T04:25:08.126484+0000","last_clean":"2024-09-17T04:25:08.126484+0000","last_became_active":"2024-09-17T04:25:08.118563+0000","last_became_peered":"2024-09-17T04:25:08.118563+0000","last_unstale":"2024-09-17T04:25:08.126484+0000","last_undegraded":"2024-09-17T04:25:08.126484+0000","last_fullsized":"2024-09-17T04:25:08.126484+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:36.885529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T04:25:07.106875+0000","last_change":"2024-09-17T04:25:07.103013+0000","last_active":"2024-09-17T04:25:07.106875+0000","last_peered":"2024-09-17T04:25:07.106875+0000","last_clean":"2024-09-17T04:25:07.106875+0000","last_became_active":"2024-09-17T04:25:07.100422+0000","last_became_peered":"2024-09-17T04:25:07.100422+0000","last_unstale":"2024-09-17T04:25:07.106875+0000","last_undegraded":"2024-09-17T04:25:07.106875+0000","last_fullsized":"2024-09-17T04:25:07.106875+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:46.886234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.128163+0000","last_change":"2024-09-17T04:25:08.125767+0000","last_active":"2024-09-17T04:25:08.128163+0000","last_peered":"2024-09-17T04:25:08.128163+0000","last_clean":"2024-09-17T04:25:08.128163+0000","last_became_active":"2024-09-17T04:25:08.125475+0000","last_became_peered":"2024-09-17T04:25:08.125475+0000","last_unstale":"2024-09-17T04:25:08.128163+0000","last_undegraded":"2024-09-17T04:25:08.128163+0000","last_fullsized":"2024-09-17T04:25:08.128163+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:21.841901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.124680+0000","last_change":"2024-09-17T04:25:07.097308+0000","last_active":"2024-09-17T04:25:08.124680+0000","last_peered":"2024-09-17T04:25:08.124680+0000","last_clean":"2024-09-17T04:25:08.124680+0000","last_became_active":"2024-09-17T04:25:07.096657+0000","last_became_peered":"2024-09-17T04:25:07.096657+0000","last_unstale":"2024-09-17T04:25:08.124680+0000","last_undegraded":"2024-09-17T04:25:08.124680+0000","last_fullsized":"2024-09-17T04:25:08.124680+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:49:55.787331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112543+0000","last_change":"2024-09-17T04:25:08.123602+0000","last_active":"2024-09-17T04:25:09.112543+0000","last_peered":"2024-09-17T04:25:09.112543+0000","last_clean":"2024-09-17T04:25:09.112543+0000","last_became_active":"2024-09-17T04:25:08.121886+0000","last_became_peered":"2024-09-17T04:25:08.121886+0000","last_unstale":"2024-09-17T04:25:09.112543+0000","last_undegraded":"2024-09-17T04:25:09.112543+0000","last_fullsized":"2024-09-17T04:25:09.112543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:59.322143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.123415+0000","last_change":"2024-09-17T04:25:07.093514+0000","last_active":"2024-09-17T04:25:08.123415+0000","last_peered":"2024-09-17T04:25:08.123415+0000","last_clean":"2024-09-17T04:25:08.123415+0000","last_became_active":"2024-09-17T04:25:07.091028+0000","last_became_peered":"2024-09-17T04:25:07.091028+0000","last_unstale":"2024-09-17T04:25:08.123415+0000","last_undegraded":"2024-09-17T04:25:08.123415+0000","last_fullsized":"2024-09-17T04:25:08.123415+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:47:16.719512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113223+0000","last_change":"2024-09-17T04:25:08.109606+0000","last_active":"2024-09-17T04:25:09.113223+0000","last_peered":"2024-09-17T04:25:09.113223+0000","last_clean":"2024-09-17T04:25:09.113223+0000","last_became_active":"2024-09-17T04:25:08.109288+0000","last_became_peered":"2024-09-17T04:25:08.109288+0000","last_unstale":"2024-09-17T04:25:09.113223+0000","last_undegraded":"2024-09-17T04:25:09.113223+0000","last_fullsized":"2024-09-17T04:25:09.113223+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:12:03.863249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.121410+0000","last_change":"2024-09-17T04:25:07.092548+0000","last_active":"2024-09-17T04:25:08.121410+0000","last_peered":"2024-09-17T04:25:08.121410+0000","last_clean":"2024-09-17T04:25:08.121410+0000","last_became_active":"2024-09-17T04:25:07.092222+0000","last_became_peered":"2024-09-17T04:25:07.092222+0000","last_unstale":"2024-09-17T04:25:08.121410+0000","last_undegraded":"2024-09-17T04:25:08.121410+0000","last_fullsized":"2024-09-17T04:25:08.121410+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:12.533380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.126405+0000","last_change":"2024-09-17T04:25:08.120205+0000","last_active":"2024-09-17T04:25:08.126405+0000","last_peered":"2024-09-17T04:25:08.126405+0000","last_clean":"2024-09-17T04:25:08.126405+0000","last_became_active":"2024-09-17T04:25:08.119921+0000","last_became_peered":"2024-09-17T04:25:08.119921+0000","last_unstale":"2024-09-17T04:25:08.126405+0000","last_undegraded":"2024-09-17T04:25:08.126405+0000","last_fullsized":"2024-09-17T04:25:08.126405+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:20.310340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T04:25:07.102921+0000","last_change":"2024-09-17T04:25:07.096137+0000","last_active":"2024-09-17T04:25:07.102921+0000","last_peered":"2024-09-17T04:25:07.102921+0000","last_clean":"2024-09-17T04:25:07.102921+0000","last_became_active":"2024-09-17T04:25:07.095852+0000","last_became_peered":"2024-09-17T04:25:07.095852+0000","last_unstale":"2024-09-17T04:25:07.102921+0000","last_undegraded":"2024-09-17T04:25:07.102921+0000","last_fullsized":"2024-09-17T04:25:07.102921+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:28.250505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.116518+0000","last_change":"2024-09-17T04:25:08.107235+0000","last_active":"2024-09-17T04:25:08.116518+0000","last_peered":"2024-09-17T04:25:08.116518+0000","last_clean":"2024-09-17T04:25:08.116518+0000","last_became_active":"2024-09-17T04:25:08.106868+0000","last_became_peered":"2024-09-17T04:25:08.106868+0000","last_unstale":"2024-09-17T04:25:08.116518+0000","last_undegraded":"2024-09-17T04:25:08.116518+0000","last_fullsized":"2024-09-17T04:25:08.116518+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:11.868935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.100418+0000","last_change":"2024-09-17T04:25:07.090542+0000","last_active":"2024-09-17T04:25:08.100418+0000","last_peered":"2024-09-17T04:25:08.100418+0000","last_clean":"2024-09-17T04:25:08.100418+0000","last_became_active":"2024-09-17T04:25:07.087859+0000","last_became_peered":"2024-09-17T04:25:07.087859+0000","last_unstale":"2024-09-17T04:25:08.100418+0000","last_undegraded":"2024-09-17T04:25:08.100418+0000","last_fullsized":"2024-09-17T04:25:08.100418+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:35.343496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.114713+0000","last_change":"2024-09-17T04:25:08.120860+0000","last_active":"2024-09-17T04:25:09.114713+0000","last_peered":"2024-09-17T04:25:09.114713+0000","last_clean":"2024-09-17T04:25:09.114713+0000","last_became_active":"2024-09-17T04:25:08.119392+0000","last_became_peered":"2024-09-17T04:25:08.119392+0000","last_unstale":"2024-09-17T04:25:09.114713+0000","last_undegraded":"2024-09-17T04:25:09.114713+0000","last_fullsized":"2024-09-17T04:25:09.114713+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:52.755680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T04:25:07.106897+0000","last_change":"2024-09-17T04:25:07.103331+0000","last_active":"2024-09-17T04:25:07.106897+0000","last_peered":"2024-09-17T04:25:07.106897+0000","last_clean":"2024-09-17T04:25:07.106897+0000","last_became_active":"2024-09-17T04:25:07.102248+0000","last_became_peered":"2024-09-17T04:25:07.102248+0000","last_unstale":"2024-09-17T04:25:07.106897+0000","last_undegraded":"2024-09-17T04:25:07.106897+0000","last_fullsized":"2024-09-17T04:25:07.106897+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:20:28.817834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.082595+0000","last_change":"2024-09-17T04:25:07.082595+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.082595+0000","last_undegraded":"2024-09-17T04:25:07.082595+0000","last_fullsized":"2024-09-17T04:25:07.082595+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.094826+0000","last_change":"2024-09-17T04:25:07.097895+0000","last_active":"2024-09-17T04:25:08.094826+0000","last_peered":"2024-09-17T04:25:08.094826+0000","last_clean":"2024-09-17T04:25:08.094826+0000","last_became_active":"2024-09-17T04:25:07.097541+0000","last_became_peered":"2024-09-17T04:25:07.097541+0000","last_unstale":"2024-09-17T04:25:08.094826+0000","last_undegraded":"2024-09-17T04:25:08.094826+0000","last_fullsized":"2024-09-17T04:25:08.094826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:49:00.272046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.088550+0000","last_change":"2024-09-17T04:25:07.088550+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.088550+0000","last_undegraded":"2024-09-17T04:25:07.088550+0000","last_fullsized":"2024-09-17T04:25:07.088550+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.114464+0000","last_change":"2024-09-17T04:25:07.103494+0000","last_active":"2024-09-17T04:25:09.114464+0000","last_peered":"2024-09-17T04:25:09.114464+0000","last_clean":"2024-09-17T04:25:09.114464+0000","last_became_active":"2024-09-17T04:25:07.103202+0000","last_became_peered":"2024-09-17T04:25:07.103202+0000","last_unstale":"2024-09-17T04:25:09.114464+0000","last_undegraded":"2024-09-17T04:25:09.114464+0000","last_fullsized":"2024-09-17T04:25:09.114464+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:33.029608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.114318+0000","last_change":"2024-09-17T04:25:08.117481+0000","last_active":"2024-09-17T04:25:09.114318+0000","last_peered":"2024-09-17T04:25:09.114318+0000","last_clean":"2024-09-17T04:25:09.114318+0000","last_became_active":"2024-09-17T04:25:08.117068+0000","last_became_peered":"2024-09-17T04:25:08.117068+0000","last_unstale":"2024-09-17T04:25:09.114318+0000","last_undegraded":"2024-09-17T04:25:09.114318+0000","last_fullsized":"2024-09-17T04:25:09.114318+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:40.823059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T04:25:07.104317+0000","last_change":"2024-09-17T04:25:07.099760+0000","last_active":"2024-09-17T04:25:07.104317+0000","last_peered":"2024-09-17T04:25:07.104317+0000","last_clean":"2024-09-17T04:25:07.104317+0000","last_became_active":"2024-09-17T04:25:07.098636+0000","last_became_peered":"2024-09-17T04:25:07.098636+0000","last_unstale":"2024-09-17T04:25:07.104317+0000","last_undegraded":"2024-09-17T04:25:07.104317+0000","last_fullsized":"2024-09-17T04:25:07.104317+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:32:49.032315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.116388+0000","last_change":"2024-09-17T04:25:08.105980+0000","last_active":"2024-09-17T04:25:08.116388+0000","last_peered":"2024-09-17T04:25:08.116388+0000","last_clean":"2024-09-17T04:25:08.116388+0000","last_became_active":"2024-09-17T04:25:08.105227+0000","last_became_peered":"2024-09-17T04:25:08.105227+0000","last_unstale":"2024-09-17T04:25:08.116388+0000","last_undegraded":"2024-09-17T04:25:08.116388+0000","last_fullsized":"2024-09-17T04:25:08.116388+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:40.098175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.124024+0000","last_change":"2024-09-17T04:25:07.090197+0000","last_active":"2024-09-17T04:25:08.124024+0000","last_peered":"2024-09-17T04:25:08.124024+0000","last_clean":"2024-09-17T04:25:08.124024+0000","last_became_active":"2024-09-17T04:25:07.089784+0000","last_became_peered":"2024-09-17T04:25:07.089784+0000","last_unstale":"2024-09-17T04:25:08.124024+0000","last_undegraded":"2024-09-17T04:25:08.124024+0000","last_fullsized":"2024-09-17T04:25:08.124024+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:09.843048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.083774+0000","last_change":"2024-09-17T04:25:07.083774+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.083774+0000","last_undegraded":"2024-09-17T04:25:07.083774+0000","last_fullsized":"2024-09-17T04:25:07.083774+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.100774+0000","last_change":"2024-09-17T04:25:07.100954+0000","last_active":"2024-09-17T04:25:08.100774+0000","last_peered":"2024-09-17T04:25:08.100774+0000","last_clean":"2024-09-17T04:25:08.100774+0000","last_became_active":"2024-09-17T04:25:07.100610+0000","last_became_peered":"2024-09-17T04:25:07.100610+0000","last_unstale":"2024-09-17T04:25:08.100774+0000","last_undegraded":"2024-09-17T04:25:08.100774+0000","last_fullsized":"2024-09-17T04:25:08.100774+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:14.912889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113053+0000","last_change":"2024-09-17T04:25:08.117150+0000","last_active":"2024-09-17T04:25:09.113053+0000","last_peered":"2024-09-17T04:25:09.113053+0000","last_clean":"2024-09-17T04:25:09.113053+0000","last_became_active":"2024-09-17T04:25:08.114530+0000","last_became_peered":"2024-09-17T04:25:08.114530+0000","last_unstale":"2024-09-17T04:25:09.113053+0000","last_undegraded":"2024-09-17T04:25:09.113053+0000","last_fullsized":"2024-09-17T04:25:09.113053+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:40.823059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T04:25:07.104424+0000","last_change":"2024-09-17T04:25:07.101663+0000","last_active":"2024-09-17T04:25:07.104424+0000","last_peered":"2024-09-17T04:25:07.104424+0000","last_clean":"2024-09-17T04:25:07.104424+0000","last_became_active":"2024-09-17T04:25:07.100447+0000","last_became_peered":"2024-09-17T04:25:07.100447+0000","last_unstale":"2024-09-17T04:25:07.104424+0000","last_undegraded":"2024-09-17T04:25:07.104424+0000","last_fullsized":"2024-09-17T04:25:07.104424+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:47.815107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.131919+0000","last_change":"2024-09-17T04:25:08.124593+0000","last_active":"2024-09-17T04:25:08.131919+0000","last_peered":"2024-09-17T04:25:08.131919+0000","last_clean":"2024-09-17T04:25:08.131919+0000","last_became_active":"2024-09-17T04:25:08.121257+0000","last_became_peered":"2024-09-17T04:25:08.121257+0000","last_unstale":"2024-09-17T04:25:08.131919+0000","last_undegraded":"2024-09-17T04:25:08.131919+0000","last_fullsized":"2024-09-17T04:25:08.131919+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:02.807745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T04:25:07.106797+0000","last_change":"2024-09-17T04:25:07.101694+0000","last_active":"2024-09-17T04:25:07.106797+0000","last_peered":"2024-09-17T04:25:07.106797+0000","last_clean":"2024-09-17T04:25:07.106797+0000","last_became_active":"2024-09-17T04:25:07.101006+0000","last_became_peered":"2024-09-17T04:25:07.101006+0000","last_unstale":"2024-09-17T04:25:07.106797+0000","last_undegraded":"2024-09-17T04:25:07.106797+0000","last_fullsized":"2024-09-17T04:25:07.106797+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:14:44.879016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.127019+0000","last_change":"2024-09-17T04:25:08.120978+0000","last_active":"2024-09-17T04:25:08.127019+0000","last_peered":"2024-09-17T04:25:08.127019+0000","last_clean":"2024-09-17T04:25:08.127019+0000","last_became_active":"2024-09-17T04:25:08.118804+0000","last_became_peered":"2024-09-17T04:25:08.118804+0000","last_unstale":"2024-09-17T04:25:08.127019+0000","last_undegraded":"2024-09-17T04:25:08.127019+0000","last_fullsized":"2024-09-17T04:25:08.127019+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:41.743123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.124335+0000","last_change":"2024-09-17T04:25:07.100101+0000","last_active":"2024-09-17T04:25:08.124335+0000","last_peered":"2024-09-17T04:25:08.124335+0000","last_clean":"2024-09-17T04:25:08.124335+0000","last_became_active":"2024-09-17T04:25:07.099657+0000","last_became_peered":"2024-09-17T04:25:07.099657+0000","last_unstale":"2024-09-17T04:25:08.124335+0000","last_undegraded":"2024-09-17T04:25:08.124335+0000","last_fullsized":"2024-09-17T04:25:08.124335+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:06:02.726891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113955+0000","last_change":"2024-09-17T04:25:08.123388+0000","last_active":"2024-09-17T04:25:09.113955+0000","last_peered":"2024-09-17T04:25:09.113955+0000","last_clean":"2024-09-17T04:25:09.113955+0000","last_became_active":"2024-09-17T04:25:08.122769+0000","last_became_peered":"2024-09-17T04:25:08.122769+0000","last_unstale":"2024-09-17T04:25:09.113955+0000","last_undegraded":"2024-09-17T04:25:09.113955+0000","last_fullsized":"2024-09-17T04:25:09.113955+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:52.735017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.118988+0000","last_change":"2024-09-17T04:25:07.089581+0000","last_active":"2024-09-17T04:25:08.118988+0000","last_peered":"2024-09-17T04:25:08.118988+0000","last_clean":"2024-09-17T04:25:08.118988+0000","last_became_active":"2024-09-17T04:25:07.089256+0000","last_became_peered":"2024-09-17T04:25:07.089256+0000","last_unstale":"2024-09-17T04:25:08.118988+0000","last_undegraded":"2024-09-17T04:25:08.118988+0000","last_fullsized":"2024-09-17T04:25:08.118988+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:11:57.535300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.132485+0000","last_change":"2024-09-17T04:25:08.127043+0000","last_active":"2024-09-17T04:25:08.132485+0000","last_peered":"2024-09-17T04:25:08.132485+0000","last_clean":"2024-09-17T04:25:08.132485+0000","last_became_active":"2024-09-17T04:25:08.126806+0000","last_became_peered":"2024-09-17T04:25:08.126806+0000","last_unstale":"2024-09-17T04:25:08.132485+0000","last_undegraded":"2024-09-17T04:25:08.132485+0000","last_fullsized":"2024-09-17T04:25:08.132485+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:03.317036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.123222+0000","last_change":"2024-09-17T04:25:07.098143+0000","last_active":"2024-09-17T04:25:08.123222+0000","last_peered":"2024-09-17T04:25:08.123222+0000","last_clean":"2024-09-17T04:25:08.123222+0000","last_became_active":"2024-09-17T04:25:07.091768+0000","last_became_peered":"2024-09-17T04:25:07.091768+0000","last_unstale":"2024-09-17T04:25:08.123222+0000","last_undegraded":"2024-09-17T04:25:08.123222+0000","last_fullsized":"2024-09-17T04:25:08.123222+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:15:18.524829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.118977+0000","last_change":"2024-09-17T04:25:08.108470+0000","last_active":"2024-09-17T04:25:08.118977+0000","last_peered":"2024-09-17T04:25:08.118977+0000","last_clean":"2024-09-17T04:25:08.118977+0000","last_became_active":"2024-09-17T04:25:08.108144+0000","last_became_peered":"2024-09-17T04:25:08.108144+0000","last_unstale":"2024-09-17T04:25:08.118977+0000","last_undegraded":"2024-09-17T04:25:08.118977+0000","last_fullsized":"2024-09-17T04:25:08.118977+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:51.768978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.100120+0000","last_change":"2024-09-17T04:25:07.103224+0000","last_active":"2024-09-17T04:25:08.100120+0000","last_peered":"2024-09-17T04:25:08.100120+0000","last_clean":"2024-09-17T04:25:08.100120+0000","last_became_active":"2024-09-17T04:25:07.101489+0000","last_became_peered":"2024-09-17T04:25:07.101489+0000","last_unstale":"2024-09-17T04:25:08.100120+0000","last_undegraded":"2024-09-17T04:25:08.100120+0000","last_fullsized":"2024-09-17T04:25:08.100120+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:08.430358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.122588+0000","last_change":"2024-09-17T04:25:08.112749+0000","last_active":"2024-09-17T04:25:08.122588+0000","last_peered":"2024-09-17T04:25:08.122588+0000","last_clean":"2024-09-17T04:25:08.122588+0000","last_became_active":"2024-09-17T04:25:08.112408+0000","last_became_peered":"2024-09-17T04:25:08.112408+0000","last_unstale":"2024-09-17T04:25:08.122588+0000","last_undegraded":"2024-09-17T04:25:08.122588+0000","last_fullsized":"2024-09-17T04:25:08.122588+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:07:25.567949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.123505+0000","last_change":"2024-09-17T04:25:07.092602+0000","last_active":"2024-09-17T04:25:08.123505+0000","last_peered":"2024-09-17T04:25:08.123505+0000","last_clean":"2024-09-17T04:25:08.123505+0000","last_became_active":"2024-09-17T04:25:07.092215+0000","last_became_peered":"2024-09-17T04:25:07.092215+0000","last_unstale":"2024-09-17T04:25:08.123505+0000","last_undegraded":"2024-09-17T04:25:08.123505+0000","last_fullsized":"2024-09-17T04:25:08.123505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:19.304290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.118395+0000","last_change":"2024-09-17T04:25:08.107413+0000","last_active":"2024-09-17T04:25:08.118395+0000","last_peered":"2024-09-17T04:25:08.118395+0000","last_clean":"2024-09-17T04:25:08.118395+0000","last_became_active":"2024-09-17T04:25:08.106516+0000","last_became_peered":"2024-09-17T04:25:08.106516+0000","last_unstale":"2024-09-17T04:25:08.118395+0000","last_undegraded":"2024-09-17T04:25:08.118395+0000","last_fullsized":"2024-09-17T04:25:08.118395+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:06.209252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.103107+0000","last_change":"2024-09-17T04:25:07.101091+0000","last_active":"2024-09-17T04:25:08.103107+0000","last_peered":"2024-09-17T04:25:08.103107+0000","last_clean":"2024-09-17T04:25:08.103107+0000","last_became_active":"2024-09-17T04:25:07.100821+0000","last_became_peered":"2024-09-17T04:25:07.100821+0000","last_unstale":"2024-09-17T04:25:08.103107+0000","last_undegraded":"2024-09-17T04:25:08.103107+0000","last_fullsized":"2024-09-17T04:25:08.103107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:50:25.153746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.108383+0000","last_change":"2024-09-17T04:25:08.104836+0000","last_active":"2024-09-17T04:25:08.108383+0000","last_peered":"2024-09-17T04:25:08.108383+0000","last_clean":"2024-09-17T04:25:08.108383+0000","last_became_active":"2024-09-17T04:25:08.104120+0000","last_became_peered":"2024-09-17T04:25:08.104120+0000","last_unstale":"2024-09-17T04:25:08.108383+0000","last_undegraded":"2024-09-17T04:25:08.108383+0000","last_fullsized":"2024-09-17T04:25:08.108383+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:05:27.192448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.097525+0000","last_change":"2024-09-17T04:25:07.103534+0000","last_active":"2024-09-17T04:25:08.097525+0000","last_peered":"2024-09-17T04:25:08.097525+0000","last_clean":"2024-09-17T04:25:08.097525+0000","last_became_active":"2024-09-17T04:25:07.103070+0000","last_became_peered":"2024-09-17T04:25:07.103070+0000","last_unstale":"2024-09-17T04:25:08.097525+0000","last_undegraded":"2024-09-17T04:25:08.097525+0000","last_fullsized":"2024-09-17T04:25:08.097525+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:57:15.056986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113955+0000","last_change":"2024-09-17T04:25:08.113398+0000","last_active":"2024-09-17T04:25:09.113955+0000","last_peered":"2024-09-17T04:25:09.113955+0000","last_clean":"2024-09-17T04:25:09.113955+0000","last_became_active":"2024-09-17T04:25:08.113049+0000","last_became_peered":"2024-09-17T04:25:08.113049+0000","last_unstale":"2024-09-17T04:25:09.113955+0000","last_undegraded":"2024-09-17T04:25:09.113955+0000","last_fullsized":"2024-09-17T04:25:09.113955+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:12.516316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.111837+0000","last_change":"2024-09-17T04:25:07.103861+0000","last_active":"2024-09-17T04:25:09.111837+0000","last_peered":"2024-09-17T04:25:09.111837+0000","last_clean":"2024-09-17T04:25:09.111837+0000","last_became_active":"2024-09-17T04:25:07.100484+0000","last_became_peered":"2024-09-17T04:25:07.100484+0000","last_unstale":"2024-09-17T04:25:09.111837+0000","last_undegraded":"2024-09-17T04:25:09.111837+0000","last_fullsized":"2024-09-17T04:25:09.111837+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:27.284841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.105143+0000","last_change":"2024-09-17T04:25:08.100426+0000","last_active":"2024-09-17T04:25:08.105143+0000","last_peered":"2024-09-17T04:25:08.105143+0000","last_clean":"2024-09-17T04:25:08.105143+0000","last_became_active":"2024-09-17T04:25:08.100117+0000","last_became_peered":"2024-09-17T04:25:08.100117+0000","last_unstale":"2024-09-17T04:25:08.105143+0000","last_undegraded":"2024-09-17T04:25:08.105143+0000","last_fullsized":"2024-09-17T04:25:08.105143+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:02.165986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.125748+0000","last_change":"2024-09-17T04:25:07.097884+0000","last_active":"2024-09-17T04:25:08.125748+0000","last_peered":"2024-09-17T04:25:08.125748+0000","last_clean":"2024-09-17T04:25:08.125748+0000","last_became_active":"2024-09-17T04:25:07.097545+0000","last_became_peered":"2024-09-17T04:25:07.097545+0000","last_unstale":"2024-09-17T04:25:08.125748+0000","last_undegraded":"2024-09-17T04:25:08.125748+0000","last_fullsized":"2024-09-17T04:25:08.125748+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:01.801695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.114256+0000","last_change":"2024-09-17T04:25:08.123061+0000","last_active":"2024-09-17T04:25:09.114256+0000","last_peered":"2024-09-17T04:25:09.114256+0000","last_clean":"2024-09-17T04:25:09.114256+0000","last_became_active":"2024-09-17T04:25:08.122319+0000","last_became_peered":"2024-09-17T04:25:08.122319+0000","last_unstale":"2024-09-17T04:25:09.114256+0000","last_undegraded":"2024-09-17T04:25:09.114256+0000","last_fullsized":"2024-09-17T04:25:09.114256+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:22.538564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.094731+0000","last_change":"2024-09-17T04:25:07.088222+0000","last_active":"2024-09-17T04:25:08.094731+0000","last_peered":"2024-09-17T04:25:08.094731+0000","last_clean":"2024-09-17T04:25:08.094731+0000","last_became_active":"2024-09-17T04:25:07.087921+0000","last_became_peered":"2024-09-17T04:25:07.087921+0000","last_unstale":"2024-09-17T04:25:08.094731+0000","last_undegraded":"2024-09-17T04:25:08.094731+0000","last_fullsized":"2024-09-17T04:25:08.094731+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:35.879479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.116348+0000","last_change":"2024-09-17T04:25:08.115284+0000","last_active":"2024-09-17T04:25:08.116348+0000","last_peered":"2024-09-17T04:25:08.116348+0000","last_clean":"2024-09-17T04:25:08.116348+0000","last_became_active":"2024-09-17T04:25:08.114998+0000","last_became_peered":"2024-09-17T04:25:08.114998+0000","last_unstale":"2024-09-17T04:25:08.116348+0000","last_undegraded":"2024-09-17T04:25:08.116348+0000","last_fullsized":"2024-09-17T04:25:08.116348+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:40:24.798987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.100440+0000","last_change":"2024-09-17T04:25:07.096188+0000","last_active":"2024-09-17T04:25:08.100440+0000","last_peered":"2024-09-17T04:25:08.100440+0000","last_clean":"2024-09-17T04:25:08.100440+0000","last_became_active":"2024-09-17T04:25:07.094697+0000","last_became_peered":"2024-09-17T04:25:07.094697+0000","last_unstale":"2024-09-17T04:25:08.100440+0000","last_undegraded":"2024-09-17T04:25:08.100440+0000","last_fullsized":"2024-09-17T04:25:08.100440+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:05.174227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.120861+0000","last_change":"2024-09-17T04:25:08.115704+0000","last_active":"2024-09-17T04:25:08.120861+0000","last_peered":"2024-09-17T04:25:08.120861+0000","last_clean":"2024-09-17T04:25:08.120861+0000","last_became_active":"2024-09-17T04:25:08.115380+0000","last_became_peered":"2024-09-17T04:25:08.115380+0000","last_unstale":"2024-09-17T04:25:08.120861+0000","last_undegraded":"2024-09-17T04:25:08.120861+0000","last_fullsized":"2024-09-17T04:25:08.120861+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:21.618822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.097074+0000","last_change":"2024-09-17T04:25:07.095160+0000","last_active":"2024-09-17T04:25:08.097074+0000","last_peered":"2024-09-17T04:25:08.097074+0000","last_clean":"2024-09-17T04:25:08.097074+0000","last_became_active":"2024-09-17T04:25:07.094948+0000","last_became_peered":"2024-09-17T04:25:07.094948+0000","last_unstale":"2024-09-17T04:25:08.097074+0000","last_undegraded":"2024-09-17T04:25:08.097074+0000","last_fullsized":"2024-09-17T04:25:08.097074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:40.737074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.123596+0000","last_change":"2024-09-17T04:25:08.120140+0000","last_active":"2024-09-17T04:25:08.123596+0000","last_peered":"2024-09-17T04:25:08.123596+0000","last_clean":"2024-09-17T04:25:08.123596+0000","last_became_active":"2024-09-17T04:25:08.118074+0000","last_became_peered":"2024-09-17T04:25:08.118074+0000","last_unstale":"2024-09-17T04:25:08.123596+0000","last_undegraded":"2024-09-17T04:25:08.123596+0000","last_fullsized":"2024-09-17T04:25:08.123596+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:15:30.368718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.100996+0000","last_change":"2024-09-17T04:25:07.094913+0000","last_active":"2024-09-17T04:25:08.100996+0000","last_peered":"2024-09-17T04:25:08.100996+0000","last_clean":"2024-09-17T04:25:08.100996+0000","last_became_active":"2024-09-17T04:25:07.094618+0000","last_became_peered":"2024-09-17T04:25:07.094618+0000","last_unstale":"2024-09-17T04:25:08.100996+0000","last_undegraded":"2024-09-17T04:25:08.100996+0000","last_fullsized":"2024-09-17T04:25:08.100996+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:20.612773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-17T04:25:07.081356+0000","last_change":"2024-09-17T04:25:07.081356+0000","last_active":"2024-09-17T04:25:07.068146+0000","last_peered":"2024-09-17T04:25:07.068146+0000","last_clean":"2024-09-17T04:25:07.068146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:07.081356+0000","last_undegraded":"2024-09-17T04:25:07.081356+0000","last_fullsized":"2024-09-17T04:25:07.081356+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.097817+0000","last_change":"2024-09-17T04:25:07.093510+0000","last_active":"2024-09-17T04:25:08.097817+0000","last_peered":"2024-09-17T04:25:08.097817+0000","last_clean":"2024-09-17T04:25:08.097817+0000","last_became_active":"2024-09-17T04:25:07.093132+0000","last_became_peered":"2024-09-17T04:25:07.093132+0000","last_unstale":"2024-09-17T04:25:08.097817+0000","last_undegraded":"2024-09-17T04:25:08.097817+0000","last_fullsized":"2024-09-17T04:25:08.097817+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:04:39.641973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.132867+0000","last_change":"2024-09-17T04:25:08.127561+0000","last_active":"2024-09-17T04:25:08.132867+0000","last_peered":"2024-09-17T04:25:08.132867+0000","last_clean":"2024-09-17T04:25:08.132867+0000","last_became_active":"2024-09-17T04:25:08.127108+0000","last_became_peered":"2024-09-17T04:25:08.127108+0000","last_unstale":"2024-09-17T04:25:08.132867+0000","last_undegraded":"2024-09-17T04:25:08.132867+0000","last_fullsized":"2024-09-17T04:25:08.132867+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:40:24.798987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T04:25:07.104421+0000","last_change":"2024-09-17T04:25:07.101483+0000","last_active":"2024-09-17T04:25:07.104421+0000","last_peered":"2024-09-17T04:25:07.104421+0000","last_clean":"2024-09-17T04:25:07.104421+0000","last_became_active":"2024-09-17T04:25:07.100021+0000","last_became_peered":"2024-09-17T04:25:07.100021+0000","last_unstale":"2024-09-17T04:25:07.104421+0000","last_undegraded":"2024-09-17T04:25:07.104421+0000","last_fullsized":"2024-09-17T04:25:07.104421+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:30:05.938041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112185+0000","last_change":"2024-09-17T04:25:08.102453+0000","last_active":"2024-09-17T04:25:09.112185+0000","last_peered":"2024-09-17T04:25:09.112185+0000","last_clean":"2024-09-17T04:25:09.112185+0000","last_became_active":"2024-09-17T04:25:08.101181+0000","last_became_peered":"2024-09-17T04:25:08.101181+0000","last_unstale":"2024-09-17T04:25:09.112185+0000","last_undegraded":"2024-09-17T04:25:09.112185+0000","last_fullsized":"2024-09-17T04:25:09.112185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:11.397751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.103937+0000","last_change":"2024-09-17T04:25:07.105074+0000","last_active":"2024-09-17T04:25:08.103937+0000","last_peered":"2024-09-17T04:25:08.103937+0000","last_clean":"2024-09-17T04:25:08.103937+0000","last_became_active":"2024-09-17T04:25:07.104735+0000","last_became_peered":"2024-09-17T04:25:07.104735+0000","last_unstale":"2024-09-17T04:25:08.103937+0000","last_undegraded":"2024-09-17T04:25:08.103937+0000","last_fullsized":"2024-09-17T04:25:08.103937+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:06:46.154246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":136,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113124+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:25:09.113124+0000","last_peered":"2024-09-17T04:25:09.113124+0000","last_clean":"2024-09-17T04:25:09.113124+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:25:09.113124+0000","last_undegraded":"2024-09-17T04:25:09.113124+0000","last_fullsized":"2024-09-17T04:25:09.113124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22,"ondisk_log_size":22,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476754,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":739667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476753,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":149825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476754,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":149881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476754,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":393216,"data_stored":149313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":372736,"data_stored":142434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509456,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":372736,"data_stored":142434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":966656,"data_stored":732802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":360448,"data_stored":136167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:25:13.522 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3e is not active+clean 2024-09-17T04:25:13.522 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:25:07.088364+0000', 'last_change': '2024-09-17T04:25:07.088364+0000', 'last_active': '2024-09-17T04:25:07.068146+0000', 'last_peered': '2024-09-17T04:25:07.068146+0000', 'last_clean': '2024-09-17T04:25:07.068146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:25:07.088364+0000', 'last_undegraded': '2024-09-17T04:25:07.088364+0000', 'last_fullsized': '2024-09-17T04:25:07.088364+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'last_clean_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T04:25:13.522 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3a is not active+clean 2024-09-17T04:25:13.522 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:25:07.083605+0000', 'last_change': '2024-09-17T04:25:07.083605+0000', 'last_active': '2024-09-17T04:25:07.068146+0000', 'last_peered': '2024-09-17T04:25:07.068146+0000', 'last_clean': '2024-09-17T04:25:07.068146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:25:07.083605+0000', 'last_undegraded': '2024-09-17T04:25:07.083605+0000', 'last_fullsized': '2024-09-17T04:25:07.083605+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'last_clean_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T04:25:13.522 INFO:tasks.ceph.ceph_manager.ceph:PG 5.32 is not active+clean 2024-09-17T04:25:13.522 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:25:07.083133+0000', 'last_change': '2024-09-17T04:25:07.083133+0000', 'last_active': '2024-09-17T04:25:07.068146+0000', 'last_peered': '2024-09-17T04:25:07.068146+0000', 'last_clean': '2024-09-17T04:25:07.068146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:25:07.083133+0000', 'last_undegraded': '2024-09-17T04:25:07.083133+0000', 'last_fullsized': '2024-09-17T04:25:07.083133+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'last_clean_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T04:25:13.522 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2e is not active+clean 2024-09-17T04:25:13.522 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:25:07.087894+0000', 'last_change': '2024-09-17T04:25:07.087894+0000', 'last_active': '2024-09-17T04:25:07.068146+0000', 'last_peered': '2024-09-17T04:25:07.068146+0000', 'last_clean': '2024-09-17T04:25:07.068146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:25:07.087894+0000', 'last_undegraded': '2024-09-17T04:25:07.087894+0000', 'last_fullsized': '2024-09-17T04:25:07.087894+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'last_clean_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T04:25:13.523 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2d is not active+clean 2024-09-17T04:25:13.523 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:25:07.084115+0000', 'last_change': '2024-09-17T04:25:07.084115+0000', 'last_active': '2024-09-17T04:25:07.068146+0000', 'last_peered': '2024-09-17T04:25:07.068146+0000', 'last_clean': '2024-09-17T04:25:07.068146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:25:07.084115+0000', 'last_undegraded': '2024-09-17T04:25:07.084115+0000', 'last_fullsized': '2024-09-17T04:25:07.084115+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'last_clean_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T04:25:13.523 INFO:tasks.ceph.ceph_manager.ceph:PG 5.26 is not active+clean 2024-09-17T04:25:13.523 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:25:07.085167+0000', 'last_change': '2024-09-17T04:25:07.085167+0000', 'last_active': '2024-09-17T04:25:07.068146+0000', 'last_peered': '2024-09-17T04:25:07.068146+0000', 'last_clean': '2024-09-17T04:25:07.068146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:25:07.085167+0000', 'last_undegraded': '2024-09-17T04:25:07.085167+0000', 'last_fullsized': '2024-09-17T04:25:07.085167+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'last_clean_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T04:25:13.523 INFO:tasks.ceph.ceph_manager.ceph:PG 5.17 is not active+clean 2024-09-17T04:25:13.523 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:25:07.082595+0000', 'last_change': '2024-09-17T04:25:07.082595+0000', 'last_active': '2024-09-17T04:25:07.068146+0000', 'last_peered': '2024-09-17T04:25:07.068146+0000', 'last_clean': '2024-09-17T04:25:07.068146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:25:07.082595+0000', 'last_undegraded': '2024-09-17T04:25:07.082595+0000', 'last_fullsized': '2024-09-17T04:25:07.082595+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'last_clean_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T04:25:13.523 INFO:tasks.ceph.ceph_manager.ceph:PG 5.14 is not active+clean 2024-09-17T04:25:13.523 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:25:07.088550+0000', 'last_change': '2024-09-17T04:25:07.088550+0000', 'last_active': '2024-09-17T04:25:07.068146+0000', 'last_peered': '2024-09-17T04:25:07.068146+0000', 'last_clean': '2024-09-17T04:25:07.068146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:25:07.088550+0000', 'last_undegraded': '2024-09-17T04:25:07.088550+0000', 'last_fullsized': '2024-09-17T04:25:07.088550+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'last_clean_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T04:25:13.523 INFO:tasks.ceph.ceph_manager.ceph:PG 5.13 is not active+clean 2024-09-17T04:25:13.523 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:25:07.083774+0000', 'last_change': '2024-09-17T04:25:07.083774+0000', 'last_active': '2024-09-17T04:25:07.068146+0000', 'last_peered': '2024-09-17T04:25:07.068146+0000', 'last_clean': '2024-09-17T04:25:07.068146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:25:07.083774+0000', 'last_undegraded': '2024-09-17T04:25:07.083774+0000', 'last_fullsized': '2024-09-17T04:25:07.083774+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'last_clean_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T04:25:13.523 INFO:tasks.ceph.ceph_manager.ceph:PG 5.0 is not active+clean 2024-09-17T04:25:13.523 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:25:07.081356+0000', 'last_change': '2024-09-17T04:25:07.081356+0000', 'last_active': '2024-09-17T04:25:07.068146+0000', 'last_peered': '2024-09-17T04:25:07.068146+0000', 'last_clean': '2024-09-17T04:25:07.068146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:25:07.081356+0000', 'last_undegraded': '2024-09-17T04:25:07.081356+0000', 'last_fullsized': '2024-09-17T04:25:07.081356+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'last_clean_scrub_stamp': '2024-09-17T04:25:07.068146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T04:25:13.524 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T04:25:13.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.778+0000 7f8e2f400640 1 -- 172.21.15.100:0/996541456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e280b3550 msgr2=0x7f8e280b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:13.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.778+0000 7f8e2f400640 1 --2- 172.21.15.100:0/996541456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e280b3550 0x7f8e280b3960 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8e28075250 tx=0x7f8e2002a980 comp rx=0 tx=0).stop 2024-09-17T04:25:13.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.778+0000 7f8e2f400640 1 -- 172.21.15.100:0/996541456 shutdown_connections 2024-09-17T04:25:13.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.778+0000 7f8e2f400640 1 --2- 172.21.15.100:0/996541456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e280b3550 0x7f8e280b3960 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:13.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.778+0000 7f8e2f400640 1 --2- 172.21.15.100:0/996541456 >> v2:172.21.15.184:3300/0 conn(0x7f8e2815d760 0x7f8e281535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:13.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.778+0000 7f8e2f400640 1 -- 172.21.15.100:0/996541456 >> 172.21.15.100:0/996541456 conn(0x7f8e2806d730 msgr2=0x7f8e280b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:13.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.778+0000 7f8e2f400640 1 -- 172.21.15.100:0/996541456 shutdown_connections 2024-09-17T04:25:13.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.778+0000 7f8e2f400640 1 -- 172.21.15.100:0/996541456 wait complete. 2024-09-17T04:25:13.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.780+0000 7f8e2f400640 1 Processor -- start 2024-09-17T04:25:13.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.781+0000 7f8e2f400640 1 -- start start 2024-09-17T04:25:13.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.782+0000 7f8e2f400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8e280ad0e0 0x7f8e281573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:13.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.782+0000 7f8e2f400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e280b3550 0x7f8e28156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:13.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.782+0000 7f8e2f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e2815d760 0x7f8e280b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:13.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.782+0000 7f8e2f400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8e280b4da0 con 0x7f8e2815d760 2024-09-17T04:25:13.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.782+0000 7f8e2f400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8e280b4c20 con 0x7f8e280b3550 2024-09-17T04:25:13.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.782+0000 7f8e2f400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8e280b3ea0 con 0x7f8e280ad0e0 2024-09-17T04:25:13.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.782+0000 7f8e2d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e2815d760 0x7f8e280b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:13.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.782+0000 7f8e2d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e2815d760 0x7f8e280b0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38986/0 (socket says 172.21.15.100:38986) 2024-09-17T04:25:13.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.782+0000 7f8e2d400640 1 -- 172.21.15.100:0/2975412271 learned_addr learned my addr 172.21.15.100:0/2975412271 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:13.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.783+0000 7f8e2ca00640 1 --2- 172.21.15.100:0/2975412271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8e280ad0e0 0x7f8e281573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:13.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.783+0000 7f8e1fe00640 1 --2- 172.21.15.100:0/2975412271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e280b3550 0x7f8e28156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:13.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.783+0000 7f8e2d400640 1 -- 172.21.15.100:0/2975412271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8e280ad0e0 msgr2=0x7f8e281573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:13.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.783+0000 7f8e2d400640 1 --2- 172.21.15.100:0/2975412271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8e280ad0e0 0x7f8e281573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:13.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.783+0000 7f8e2d400640 1 -- 172.21.15.100:0/2975412271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e280b3550 msgr2=0x7f8e28156630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:13.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.783+0000 7f8e2d400640 1 --2- 172.21.15.100:0/2975412271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e280b3550 0x7f8e28156630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:13.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.783+0000 7f8e2d400640 1 -- 172.21.15.100:0/2975412271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e281f3940 con 0x7f8e2815d760 2024-09-17T04:25:13.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.783+0000 7f8e2d400640 1 --2- 172.21.15.100:0/2975412271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e2815d760 0x7f8e280b0da0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009d60 tx=0x7f8e20003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:13.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.783+0000 7f8e25600640 1 -- 172.21.15.100:0/2975412271 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e20034040 con 0x7f8e2815d760 2024-09-17T04:25:13.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.783+0000 7f8e25600640 1 -- 172.21.15.100:0/2975412271 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e20004540 con 0x7f8e2815d760 2024-09-17T04:25:13.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.783+0000 7f8e2f400640 1 -- 172.21.15.100:0/2975412271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e281f3c20 con 0x7f8e2815d760 2024-09-17T04:25:13.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.783+0000 7f8e25600640 1 -- 172.21.15.100:0/2975412271 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e20045480 con 0x7f8e2815d760 2024-09-17T04:25:13.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.784+0000 7f8e2f400640 1 -- 172.21.15.100:0/2975412271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e2815a220 con 0x7f8e2815d760 2024-09-17T04:25:13.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.785+0000 7f8e2f400640 1 -- 172.21.15.100:0/2975412271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e280b43a0 con 0x7f8e2815d760 2024-09-17T04:25:13.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.786+0000 7f8e25600640 1 -- 172.21.15.100:0/2975412271 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8e20037020 con 0x7f8e2815d760 2024-09-17T04:25:13.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.787+0000 7f8e25600640 1 --2- 172.21.15.100:0/2975412271 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8e04078b90 0x7f8e0407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:13.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.787+0000 7f8e25600640 1 -- 172.21.15.100:0/2975412271 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f8e200bc6d0 con 0x7f8e2815d760 2024-09-17T04:25:13.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.787+0000 7f8e2ca00640 1 --2- 172.21.15.100:0/2975412271 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8e04078b90 0x7f8e0407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:13.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.788+0000 7f8e2ca00640 1 --2- 172.21.15.100:0/2975412271 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8e04078b90 0x7f8e0407b0a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8e18005fd0 tx=0x7f8e18005850 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:13.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:13.791+0000 7f8e25600640 1 -- 172.21.15.100:0/2975412271 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8e20084a80 con 0x7f8e2815d760 2024-09-17T04:25:14.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.046+0000 7f8e2f400640 1 -- 172.21.15.100:0/2975412271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f8e2815b180 con 0x7f8e2815d760 2024-09-17T04:25:14.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.047+0000 7f8e25600640 1 -- 172.21.15.100:0/2975412271 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1688 (secure 0 0 0) 0x7f8e20084380 con 0x7f8e2815d760 2024-09-17T04:25:14.047 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:14.048 INFO:teuthology.orchestra.run.smithi100.stdout:{"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":87,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":8,"num_up_osds":8,"osd_up_since":1726547043,"num_in_osds":8,"osd_in_since":1726547032,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":87},{"state_name":"creating+peering","count":10}],"num_pgs":97,"num_pools":3,"num_objects":14,"data_bytes":591166,"bytes_used":225660928,"bytes_avail":772868452352,"bytes_total":773094113280,"inactive_pgs_ratio":0.10309278219938278,"write_bytes_sec":1905,"read_op_per_sec":0,"write_op_per_sec":3},"fsmap":{"epoch":17,"btime":"2024-09-17T04:25:11:112457+0000","id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"b","status":"up:active","gid":4857}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-17T04:25:05.978140+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T04:25:14.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.049+0000 7f8e2f400640 1 -- 172.21.15.100:0/2975412271 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8e04078b90 msgr2=0x7f8e0407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:14.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.049+0000 7f8e2f400640 1 --2- 172.21.15.100:0/2975412271 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8e04078b90 0x7f8e0407b0a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8e18005fd0 tx=0x7f8e18005850 comp rx=0 tx=0).stop 2024-09-17T04:25:14.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.049+0000 7f8e2f400640 1 -- 172.21.15.100:0/2975412271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e2815d760 msgr2=0x7f8e280b0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:14.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.049+0000 7f8e2f400640 1 --2- 172.21.15.100:0/2975412271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e2815d760 0x7f8e280b0da0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009d60 tx=0x7f8e20003f10 comp rx=0 tx=0).stop 2024-09-17T04:25:14.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.049+0000 7f8e2f400640 1 -- 172.21.15.100:0/2975412271 shutdown_connections 2024-09-17T04:25:14.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.049+0000 7f8e2f400640 1 --2- 172.21.15.100:0/2975412271 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8e04078b90 0x7f8e0407b0a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:14.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.049+0000 7f8e2f400640 1 --2- 172.21.15.100:0/2975412271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e2815d760 0x7f8e280b0da0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:14.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.049+0000 7f8e2f400640 1 --2- 172.21.15.100:0/2975412271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e280b3550 0x7f8e28156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.049+0000 7f8e2f400640 1 --2- 172.21.15.100:0/2975412271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8e280ad0e0 0x7f8e281573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.050+0000 7f8e2f400640 1 -- 172.21.15.100:0/2975412271 >> 172.21.15.100:0/2975412271 conn(0x7f8e2806d730 msgr2=0x7f8e280cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.050+0000 7f8e2f400640 1 -- 172.21.15.100:0/2975412271 shutdown_connections 2024-09-17T04:25:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.050+0000 7f8e2f400640 1 -- 172.21.15.100:0/2975412271 wait complete. 2024-09-17T04:25:14.073 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T04:25:14.073 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:25:14.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.295+0000 7f19b8000640 1 -- 172.21.15.100:0/2140043658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19b01626d0 msgr2=0x7f19b0164af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:14.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.295+0000 7f19b8000640 1 --2- 172.21.15.100:0/2140043658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19b01626d0 0x7f19b0164af0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f19b0072810 tx=0x7f19a802aa30 comp rx=0 tx=0).stop 2024-09-17T04:25:14.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.295+0000 7f19b8000640 1 -- 172.21.15.100:0/2140043658 shutdown_connections 2024-09-17T04:25:14.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.295+0000 7f19b8000640 1 --2- 172.21.15.100:0/2140043658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19b01626d0 0x7f19b0164af0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:14.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.295+0000 7f19b8000640 1 --2- 172.21.15.100:0/2140043658 >> v2:172.21.15.184:3300/0 conn(0x7f19b00704b0 0x7f19b0073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:14.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.295+0000 7f19b8000640 1 -- 172.21.15.100:0/2140043658 >> 172.21.15.100:0/2140043658 conn(0x7f19b006d730 msgr2=0x7f19b00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:14.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.295+0000 7f19b8000640 1 -- 172.21.15.100:0/2140043658 shutdown_connections 2024-09-17T04:25:14.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.295+0000 7f19b8000640 1 -- 172.21.15.100:0/2140043658 wait complete. 2024-09-17T04:25:14.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.297+0000 7f19b8000640 1 Processor -- start 2024-09-17T04:25:14.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.297+0000 7f19b8000640 1 -- start start 2024-09-17T04:25:14.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.299+0000 7f19b8000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19b00704b0 0x7f19b01f3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:14.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.299+0000 7f19b8000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19b00ad3c0 0x7f19b014fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:14.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.299+0000 7f19b8000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19b01626d0 0x7f19b0150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:14.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.299+0000 7f19b8000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f19b00ae060 con 0x7f19b00704b0 2024-09-17T04:25:14.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.299+0000 7f19b8000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f19b00adee0 con 0x7f19b00ad3c0 2024-09-17T04:25:14.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.299+0000 7f19b8000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f19b0071130 con 0x7f19b01626d0 2024-09-17T04:25:14.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.299+0000 7f19b5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19b00704b0 0x7f19b01f3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:14.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.299+0000 7f19b5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19b00704b0 0x7f19b01f3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38998/0 (socket says 172.21.15.100:38998) 2024-09-17T04:25:14.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.299+0000 7f19b5600640 1 -- 172.21.15.100:0/340320924 learned_addr learned my addr 172.21.15.100:0/340320924 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:14.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.299+0000 7f19b6000640 1 --2- 172.21.15.100:0/340320924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19b01626d0 0x7f19b0150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:14.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.299+0000 7f19b4c00640 1 --2- 172.21.15.100:0/340320924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19b00ad3c0 0x7f19b014fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:14.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.299+0000 7f19b5600640 1 -- 172.21.15.100:0/340320924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19b01626d0 msgr2=0x7f19b0150350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:14.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.299+0000 7f19b5600640 1 --2- 172.21.15.100:0/340320924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19b01626d0 0x7f19b0150350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:14.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.299+0000 7f19b5600640 1 -- 172.21.15.100:0/340320924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19b00ad3c0 msgr2=0x7f19b014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:14.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.299+0000 7f19b5600640 1 --2- 172.21.15.100:0/340320924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19b00ad3c0 0x7f19b014fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:14.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.299+0000 7f19b5600640 1 -- 172.21.15.100:0/340320924 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b015c400 con 0x7f19b00704b0 2024-09-17T04:25:14.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.299+0000 7f19b6000640 1 --2- 172.21.15.100:0/340320924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19b01626d0 0x7f19b0150350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:25:14.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.300+0000 7f19b5600640 1 --2- 172.21.15.100:0/340320924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19b00704b0 0x7f19b01f3400 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f19a400db90 tx=0x7f19a400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:14.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.300+0000 7f19ad600640 1 -- 172.21.15.100:0/340320924 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a401b070 con 0x7f19b00704b0 2024-09-17T04:25:14.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.300+0000 7f19ad600640 1 -- 172.21.15.100:0/340320924 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19a400ae50 con 0x7f19b00704b0 2024-09-17T04:25:14.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.300+0000 7f19b8000640 1 -- 172.21.15.100:0/340320924 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19b014e060 con 0x7f19b00704b0 2024-09-17T04:25:14.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.300+0000 7f19ad600640 1 -- 172.21.15.100:0/340320924 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a400e9e0 con 0x7f19b00704b0 2024-09-17T04:25:14.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.300+0000 7f19b8000640 1 -- 172.21.15.100:0/340320924 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19b015b920 con 0x7f19b00704b0 2024-09-17T04:25:14.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.301+0000 7f19b8000640 1 -- 172.21.15.100:0/340320924 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19b0073040 con 0x7f19b00704b0 2024-09-17T04:25:14.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.302+0000 7f19ad600640 1 -- 172.21.15.100:0/340320924 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f19a401e070 con 0x7f19b00704b0 2024-09-17T04:25:14.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.303+0000 7f19ad600640 1 --2- 172.21.15.100:0/340320924 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1990078bc0 0x7f199007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:14.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.303+0000 7f19ad600640 1 -- 172.21.15.100:0/340320924 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f19a409b150 con 0x7f19b00704b0 2024-09-17T04:25:14.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.304+0000 7f19b4c00640 1 --2- 172.21.15.100:0/340320924 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1990078bc0 0x7f199007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:14.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.304+0000 7f19b4c00640 1 --2- 172.21.15.100:0/340320924 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1990078bc0 0x7f199007b0d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f19b0162c70 tx=0x7f199c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:14.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.308+0000 7f19ad600640 1 -- 172.21.15.100:0/340320924 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f19a4062fb0 con 0x7f19b00704b0 2024-09-17T04:25:14.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.488+0000 7f19b8000640 1 -- 172.21.15.100:0/340320924 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f19b015e420 con 0x7f1990078bc0 2024-09-17T04:25:14.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.493+0000 7f19ad600640 1 -- 172.21.15.100:0/340320924 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251916 (secure 0 0 0) 0x7f19b015e420 con 0x7f1990078bc0 2024-09-17T04:25:14.496 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:14.498 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:25:14.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.497+0000 7f19b8000640 1 -- 172.21.15.100:0/340320924 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1990078bc0 msgr2=0x7f199007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:14.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.497+0000 7f19b8000640 1 --2- 172.21.15.100:0/340320924 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1990078bc0 0x7f199007b0d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f19b0162c70 tx=0x7f199c009040 comp rx=0 tx=0).stop 2024-09-17T04:25:14.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.497+0000 7f19b8000640 1 -- 172.21.15.100:0/340320924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19b00704b0 msgr2=0x7f19b01f3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:14.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.497+0000 7f19b8000640 1 --2- 172.21.15.100:0/340320924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19b00704b0 0x7f19b01f3400 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f19a400db90 tx=0x7f19a400a620 comp rx=0 tx=0).stop 2024-09-17T04:25:14.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.497+0000 7f19b8000640 1 -- 172.21.15.100:0/340320924 shutdown_connections 2024-09-17T04:25:14.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.497+0000 7f19b8000640 1 --2- 172.21.15.100:0/340320924 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1990078bc0 0x7f199007b0d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:14.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.497+0000 7f19b8000640 1 --2- 172.21.15.100:0/340320924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19b01626d0 0x7f19b0150350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:14.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.497+0000 7f19b8000640 1 --2- 172.21.15.100:0/340320924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19b00ad3c0 0x7f19b014fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:14.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.497+0000 7f19b8000640 1 --2- 172.21.15.100:0/340320924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19b00704b0 0x7f19b01f3400 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:14.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.497+0000 7f19b8000640 1 -- 172.21.15.100:0/340320924 >> 172.21.15.100:0/340320924 conn(0x7f19b006d730 msgr2=0x7f19b0147030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:14.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.497+0000 7f19b8000640 1 -- 172.21.15.100:0/340320924 shutdown_connections 2024-09-17T04:25:14.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:14.497+0000 7f19b8000640 1 -- 172.21.15.100:0/340320924 wait complete. 2024-09-17T04:25:14.520 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-09-17T04:25:13.980291+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220520,"kb_used_data":4280,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754200,"statfs":{"total":773094113280,"available":772868300800,"internally_reserved":0,"allocated":4382720,"data_stored":2404428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.907540"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.131931+0000","last_change":"2024-09-17T04:25:08.126623+0000","last_active":"2024-09-17T04:25:08.131931+0000","last_peered":"2024-09-17T04:25:08.131931+0000","last_clean":"2024-09-17T04:25:08.131931+0000","last_became_active":"2024-09-17T04:25:08.126204+0000","last_became_peered":"2024-09-17T04:25:08.126204+0000","last_unstale":"2024-09-17T04:25:08.131931+0000","last_undegraded":"2024-09-17T04:25:08.131931+0000","last_fullsized":"2024-09-17T04:25:08.131931+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:40.098175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.456067+0000","last_change":"2024-09-17T04:25:08.116947+0000","last_active":"2024-09-17T04:25:09.456067+0000","last_peered":"2024-09-17T04:25:09.456067+0000","last_clean":"2024-09-17T04:25:09.456067+0000","last_became_active":"2024-09-17T04:25:08.115387+0000","last_became_peered":"2024-09-17T04:25:08.115387+0000","last_unstale":"2024-09-17T04:25:09.456067+0000","last_undegraded":"2024-09-17T04:25:09.456067+0000","last_fullsized":"2024-09-17T04:25:09.456067+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:39:50.486405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.114386+0000","last_change":"2024-09-17T04:25:08.116641+0000","last_active":"2024-09-17T04:25:09.114386+0000","last_peered":"2024-09-17T04:25:09.114386+0000","last_clean":"2024-09-17T04:25:09.114386+0000","last_became_active":"2024-09-17T04:25:08.115852+0000","last_became_peered":"2024-09-17T04:25:08.115852+0000","last_unstale":"2024-09-17T04:25:09.114386+0000","last_undegraded":"2024-09-17T04:25:09.114386+0000","last_fullsized":"2024-09-17T04:25:09.114386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:46.534540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.454354+0000","last_change":"2024-09-17T04:25:08.127729+0000","last_active":"2024-09-17T04:25:09.454354+0000","last_peered":"2024-09-17T04:25:09.454354+0000","last_clean":"2024-09-17T04:25:09.454354+0000","last_became_active":"2024-09-17T04:25:08.125546+0000","last_became_peered":"2024-09-17T04:25:08.125546+0000","last_unstale":"2024-09-17T04:25:09.454354+0000","last_undegraded":"2024-09-17T04:25:09.454354+0000","last_fullsized":"2024-09-17T04:25:09.454354+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:33:23.179319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.105372+0000","last_change":"2024-09-17T04:25:08.113523+0000","last_active":"2024-09-17T04:25:09.105372+0000","last_peered":"2024-09-17T04:25:09.105372+0000","last_clean":"2024-09-17T04:25:09.105372+0000","last_became_active":"2024-09-17T04:25:08.113207+0000","last_became_peered":"2024-09-17T04:25:08.113207+0000","last_unstale":"2024-09-17T04:25:09.105372+0000","last_undegraded":"2024-09-17T04:25:09.105372+0000","last_fullsized":"2024-09-17T04:25:09.105372+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:36.349545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455354+0000","last_change":"2024-09-17T04:25:08.101238+0000","last_active":"2024-09-17T04:25:09.455354+0000","last_peered":"2024-09-17T04:25:09.455354+0000","last_clean":"2024-09-17T04:25:09.455354+0000","last_became_active":"2024-09-17T04:25:08.099079+0000","last_became_peered":"2024-09-17T04:25:08.099079+0000","last_unstale":"2024-09-17T04:25:09.455354+0000","last_undegraded":"2024-09-17T04:25:09.455354+0000","last_fullsized":"2024-09-17T04:25:09.455354+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:01.930915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113556+0000","last_change":"2024-09-17T04:25:08.113471+0000","last_active":"2024-09-17T04:25:09.113556+0000","last_peered":"2024-09-17T04:25:09.113556+0000","last_clean":"2024-09-17T04:25:09.113556+0000","last_became_active":"2024-09-17T04:25:08.113145+0000","last_became_peered":"2024-09-17T04:25:08.113145+0000","last_unstale":"2024-09-17T04:25:09.113556+0000","last_undegraded":"2024-09-17T04:25:09.113556+0000","last_fullsized":"2024-09-17T04:25:09.113556+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:52.755680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112954+0000","last_change":"2024-09-17T04:25:08.102460+0000","last_active":"2024-09-17T04:25:09.112954+0000","last_peered":"2024-09-17T04:25:09.112954+0000","last_clean":"2024-09-17T04:25:09.112954+0000","last_became_active":"2024-09-17T04:25:08.101474+0000","last_became_peered":"2024-09-17T04:25:08.101474+0000","last_unstale":"2024-09-17T04:25:09.112954+0000","last_undegraded":"2024-09-17T04:25:09.112954+0000","last_fullsized":"2024-09-17T04:25:09.112954+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:22.538564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.126318+0000","last_change":"2024-09-17T04:25:08.119819+0000","last_active":"2024-09-17T04:25:08.126318+0000","last_peered":"2024-09-17T04:25:08.126318+0000","last_clean":"2024-09-17T04:25:08.126318+0000","last_became_active":"2024-09-17T04:25:08.119511+0000","last_became_peered":"2024-09-17T04:25:08.119511+0000","last_unstale":"2024-09-17T04:25:08.126318+0000","last_undegraded":"2024-09-17T04:25:08.126318+0000","last_fullsized":"2024-09-17T04:25:08.126318+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:49:01.278096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455421+0000","last_change":"2024-09-17T04:25:08.121720+0000","last_active":"2024-09-17T04:25:09.455421+0000","last_peered":"2024-09-17T04:25:09.455421+0000","last_clean":"2024-09-17T04:25:09.455421+0000","last_became_active":"2024-09-17T04:25:08.121405+0000","last_became_peered":"2024-09-17T04:25:08.121405+0000","last_unstale":"2024-09-17T04:25:09.455421+0000","last_undegraded":"2024-09-17T04:25:09.455421+0000","last_fullsized":"2024-09-17T04:25:09.455421+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:11.397751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112530+0000","last_change":"2024-09-17T04:25:08.116374+0000","last_active":"2024-09-17T04:25:09.112530+0000","last_peered":"2024-09-17T04:25:09.112530+0000","last_clean":"2024-09-17T04:25:09.112530+0000","last_became_active":"2024-09-17T04:25:08.116047+0000","last_became_peered":"2024-09-17T04:25:08.116047+0000","last_unstale":"2024-09-17T04:25:09.112530+0000","last_undegraded":"2024-09-17T04:25:09.112530+0000","last_fullsized":"2024-09-17T04:25:09.112530+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:12:03.863249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113687+0000","last_change":"2024-09-17T04:25:08.117853+0000","last_active":"2024-09-17T04:25:09.113687+0000","last_peered":"2024-09-17T04:25:09.113687+0000","last_clean":"2024-09-17T04:25:09.113687+0000","last_became_active":"2024-09-17T04:25:08.117342+0000","last_became_peered":"2024-09-17T04:25:08.117342+0000","last_unstale":"2024-09-17T04:25:09.113687+0000","last_undegraded":"2024-09-17T04:25:09.113687+0000","last_fullsized":"2024-09-17T04:25:09.113687+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:59.322143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.105491+0000","last_change":"2024-09-17T04:25:08.107583+0000","last_active":"2024-09-17T04:25:09.105491+0000","last_peered":"2024-09-17T04:25:09.105491+0000","last_clean":"2024-09-17T04:25:09.105491+0000","last_became_active":"2024-09-17T04:25:08.107097+0000","last_became_peered":"2024-09-17T04:25:08.107097+0000","last_unstale":"2024-09-17T04:25:09.105491+0000","last_undegraded":"2024-09-17T04:25:09.105491+0000","last_fullsized":"2024-09-17T04:25:09.105491+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:13.539429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455604+0000","last_change":"2024-09-17T04:25:08.092773+0000","last_active":"2024-09-17T04:25:09.455604+0000","last_peered":"2024-09-17T04:25:09.455604+0000","last_clean":"2024-09-17T04:25:09.455604+0000","last_became_active":"2024-09-17T04:25:08.092437+0000","last_became_peered":"2024-09-17T04:25:08.092437+0000","last_unstale":"2024-09-17T04:25:09.455604+0000","last_undegraded":"2024-09-17T04:25:09.455604+0000","last_fullsized":"2024-09-17T04:25:09.455604+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:51.218111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112471+0000","last_change":"2024-09-17T04:25:08.122562+0000","last_active":"2024-09-17T04:25:09.112471+0000","last_peered":"2024-09-17T04:25:09.112471+0000","last_clean":"2024-09-17T04:25:09.112471+0000","last_became_active":"2024-09-17T04:25:08.120406+0000","last_became_peered":"2024-09-17T04:25:08.120406+0000","last_unstale":"2024-09-17T04:25:09.112471+0000","last_undegraded":"2024-09-17T04:25:09.112471+0000","last_fullsized":"2024-09-17T04:25:09.112471+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:14.660711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113172+0000","last_change":"2024-09-17T04:25:08.111402+0000","last_active":"2024-09-17T04:25:09.113172+0000","last_peered":"2024-09-17T04:25:09.113172+0000","last_clean":"2024-09-17T04:25:09.113172+0000","last_became_active":"2024-09-17T04:25:08.111049+0000","last_became_peered":"2024-09-17T04:25:08.111049+0000","last_unstale":"2024-09-17T04:25:09.113172+0000","last_undegraded":"2024-09-17T04:25:09.113172+0000","last_fullsized":"2024-09-17T04:25:09.113172+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:46.534540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112688+0000","last_change":"2024-09-17T04:25:08.113695+0000","last_active":"2024-09-17T04:25:09.112688+0000","last_peered":"2024-09-17T04:25:09.112688+0000","last_clean":"2024-09-17T04:25:09.112688+0000","last_became_active":"2024-09-17T04:25:08.113248+0000","last_became_peered":"2024-09-17T04:25:08.113248+0000","last_unstale":"2024-09-17T04:25:09.112688+0000","last_undegraded":"2024-09-17T04:25:09.112688+0000","last_fullsized":"2024-09-17T04:25:09.112688+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:52.735017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.456121+0000","last_change":"2024-09-17T04:25:08.112012+0000","last_active":"2024-09-17T04:25:09.456121+0000","last_peered":"2024-09-17T04:25:09.456121+0000","last_clean":"2024-09-17T04:25:09.456121+0000","last_became_active":"2024-09-17T04:25:08.111723+0000","last_became_peered":"2024-09-17T04:25:08.111723+0000","last_unstale":"2024-09-17T04:25:09.456121+0000","last_undegraded":"2024-09-17T04:25:09.456121+0000","last_fullsized":"2024-09-17T04:25:09.456121+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:43:52.912008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455267+0000","last_change":"2024-09-17T04:25:08.120093+0000","last_active":"2024-09-17T04:25:09.455267+0000","last_peered":"2024-09-17T04:25:09.455267+0000","last_clean":"2024-09-17T04:25:09.455267+0000","last_became_active":"2024-09-17T04:25:08.119846+0000","last_became_peered":"2024-09-17T04:25:08.119846+0000","last_unstale":"2024-09-17T04:25:09.455267+0000","last_undegraded":"2024-09-17T04:25:09.455267+0000","last_fullsized":"2024-09-17T04:25:09.455267+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:06:21.034599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.105643+0000","last_change":"2024-09-17T04:25:08.102248+0000","last_active":"2024-09-17T04:25:09.105643+0000","last_peered":"2024-09-17T04:25:09.105643+0000","last_clean":"2024-09-17T04:25:09.105643+0000","last_became_active":"2024-09-17T04:25:08.101915+0000","last_became_peered":"2024-09-17T04:25:08.101915+0000","last_unstale":"2024-09-17T04:25:09.105643+0000","last_undegraded":"2024-09-17T04:25:09.105643+0000","last_fullsized":"2024-09-17T04:25:09.105643+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:01.832590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.125029+0000","last_change":"2024-09-17T04:25:08.116792+0000","last_active":"2024-09-17T04:25:08.125029+0000","last_peered":"2024-09-17T04:25:08.125029+0000","last_clean":"2024-09-17T04:25:08.125029+0000","last_became_active":"2024-09-17T04:25:08.116468+0000","last_became_peered":"2024-09-17T04:25:08.116468+0000","last_unstale":"2024-09-17T04:25:08.125029+0000","last_undegraded":"2024-09-17T04:25:08.125029+0000","last_fullsized":"2024-09-17T04:25:08.125029+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:01.832590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.132325+0000","last_change":"2024-09-17T04:25:08.125202+0000","last_active":"2024-09-17T04:25:08.132325+0000","last_peered":"2024-09-17T04:25:08.132325+0000","last_clean":"2024-09-17T04:25:08.132325+0000","last_became_active":"2024-09-17T04:25:08.124880+0000","last_became_peered":"2024-09-17T04:25:08.124880+0000","last_unstale":"2024-09-17T04:25:08.132325+0000","last_undegraded":"2024-09-17T04:25:08.132325+0000","last_fullsized":"2024-09-17T04:25:08.132325+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:33.145063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.401577+0000","last_change":"2024-09-17T04:25:08.109076+0000","last_active":"2024-09-17T04:25:09.401577+0000","last_peered":"2024-09-17T04:25:09.401577+0000","last_clean":"2024-09-17T04:25:09.401577+0000","last_became_active":"2024-09-17T04:25:08.105699+0000","last_became_peered":"2024-09-17T04:25:08.105699+0000","last_unstale":"2024-09-17T04:25:09.401577+0000","last_undegraded":"2024-09-17T04:25:09.401577+0000","last_fullsized":"2024-09-17T04:25:09.401577+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:30.659131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.111966+0000","last_change":"2024-09-17T04:25:08.099256+0000","last_active":"2024-09-17T04:25:09.111966+0000","last_peered":"2024-09-17T04:25:09.111966+0000","last_clean":"2024-09-17T04:25:09.111966+0000","last_became_active":"2024-09-17T04:25:08.098874+0000","last_became_peered":"2024-09-17T04:25:08.098874+0000","last_unstale":"2024-09-17T04:25:09.111966+0000","last_undegraded":"2024-09-17T04:25:09.111966+0000","last_fullsized":"2024-09-17T04:25:09.111966+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:11.397751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.454771+0000","last_change":"2024-09-17T04:25:08.117901+0000","last_active":"2024-09-17T04:25:09.454771+0000","last_peered":"2024-09-17T04:25:09.454771+0000","last_clean":"2024-09-17T04:25:09.454771+0000","last_became_active":"2024-09-17T04:25:08.115468+0000","last_became_peered":"2024-09-17T04:25:08.115468+0000","last_unstale":"2024-09-17T04:25:09.454771+0000","last_undegraded":"2024-09-17T04:25:09.454771+0000","last_fullsized":"2024-09-17T04:25:09.454771+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:03.692785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.456199+0000","last_change":"2024-09-17T04:25:08.110694+0000","last_active":"2024-09-17T04:25:09.456199+0000","last_peered":"2024-09-17T04:25:09.456199+0000","last_clean":"2024-09-17T04:25:09.456199+0000","last_became_active":"2024-09-17T04:25:08.097992+0000","last_became_peered":"2024-09-17T04:25:08.097992+0000","last_unstale":"2024-09-17T04:25:09.456199+0000","last_undegraded":"2024-09-17T04:25:09.456199+0000","last_fullsized":"2024-09-17T04:25:09.456199+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:42.703067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.454792+0000","last_change":"2024-09-17T04:25:08.126392+0000","last_active":"2024-09-17T04:25:09.454792+0000","last_peered":"2024-09-17T04:25:09.454792+0000","last_clean":"2024-09-17T04:25:09.454792+0000","last_became_active":"2024-09-17T04:25:08.126046+0000","last_became_peered":"2024-09-17T04:25:08.126046+0000","last_unstale":"2024-09-17T04:25:09.454792+0000","last_undegraded":"2024-09-17T04:25:09.454792+0000","last_fullsized":"2024-09-17T04:25:09.454792+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:14.054043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113961+0000","last_change":"2024-09-17T04:25:08.104557+0000","last_active":"2024-09-17T04:25:09.113961+0000","last_peered":"2024-09-17T04:25:09.113961+0000","last_clean":"2024-09-17T04:25:09.113961+0000","last_became_active":"2024-09-17T04:25:08.104246+0000","last_became_peered":"2024-09-17T04:25:08.104246+0000","last_unstale":"2024-09-17T04:25:09.113961+0000","last_undegraded":"2024-09-17T04:25:09.113961+0000","last_fullsized":"2024-09-17T04:25:09.113961+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:26.289675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.132086+0000","last_change":"2024-09-17T04:25:08.124218+0000","last_active":"2024-09-17T04:25:08.132086+0000","last_peered":"2024-09-17T04:25:08.132086+0000","last_clean":"2024-09-17T04:25:08.132086+0000","last_became_active":"2024-09-17T04:25:08.123746+0000","last_became_peered":"2024-09-17T04:25:08.123746+0000","last_unstale":"2024-09-17T04:25:08.132086+0000","last_undegraded":"2024-09-17T04:25:08.132086+0000","last_fullsized":"2024-09-17T04:25:08.132086+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:02.874046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112282+0000","last_change":"2024-09-17T04:25:08.113465+0000","last_active":"2024-09-17T04:25:09.112282+0000","last_peered":"2024-09-17T04:25:09.112282+0000","last_clean":"2024-09-17T04:25:09.112282+0000","last_became_active":"2024-09-17T04:25:08.113106+0000","last_became_peered":"2024-09-17T04:25:08.113106+0000","last_unstale":"2024-09-17T04:25:09.112282+0000","last_undegraded":"2024-09-17T04:25:09.112282+0000","last_fullsized":"2024-09-17T04:25:09.112282+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:14.660711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.120556+0000","last_change":"2024-09-17T04:25:08.111799+0000","last_active":"2024-09-17T04:25:08.120556+0000","last_peered":"2024-09-17T04:25:08.120556+0000","last_clean":"2024-09-17T04:25:08.120556+0000","last_became_active":"2024-09-17T04:25:08.111338+0000","last_became_peered":"2024-09-17T04:25:08.111338+0000","last_unstale":"2024-09-17T04:25:08.120556+0000","last_undegraded":"2024-09-17T04:25:08.120556+0000","last_fullsized":"2024-09-17T04:25:08.120556+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:15.918939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112824+0000","last_change":"2024-09-17T04:25:08.109129+0000","last_active":"2024-09-17T04:25:09.112824+0000","last_peered":"2024-09-17T04:25:09.112824+0000","last_clean":"2024-09-17T04:25:09.112824+0000","last_became_active":"2024-09-17T04:25:08.104873+0000","last_became_peered":"2024-09-17T04:25:08.104873+0000","last_unstale":"2024-09-17T04:25:09.112824+0000","last_undegraded":"2024-09-17T04:25:09.112824+0000","last_fullsized":"2024-09-17T04:25:09.112824+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:48.676959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.401857+0000","last_change":"2024-09-17T04:25:08.104171+0000","last_active":"2024-09-17T04:25:09.401857+0000","last_peered":"2024-09-17T04:25:09.401857+0000","last_clean":"2024-09-17T04:25:09.401857+0000","last_became_active":"2024-09-17T04:25:08.103831+0000","last_became_peered":"2024-09-17T04:25:08.103831+0000","last_unstale":"2024-09-17T04:25:09.401857+0000","last_undegraded":"2024-09-17T04:25:09.401857+0000","last_fullsized":"2024-09-17T04:25:09.401857+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:01:06.440885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.132361+0000","last_change":"2024-09-17T04:25:07.101542+0000","last_active":"2024-09-17T04:25:08.132361+0000","last_peered":"2024-09-17T04:25:08.132361+0000","last_clean":"2024-09-17T04:25:08.132361+0000","last_became_active":"2024-09-17T04:25:07.101168+0000","last_became_peered":"2024-09-17T04:25:07.101168+0000","last_unstale":"2024-09-17T04:25:08.132361+0000","last_undegraded":"2024-09-17T04:25:08.132361+0000","last_fullsized":"2024-09-17T04:25:08.132361+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:05.174227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.127811+0000","last_change":"2024-09-17T04:25:08.122307+0000","last_active":"2024-09-17T04:25:08.127811+0000","last_peered":"2024-09-17T04:25:08.127811+0000","last_clean":"2024-09-17T04:25:08.127811+0000","last_became_active":"2024-09-17T04:25:08.121816+0000","last_became_peered":"2024-09-17T04:25:08.121816+0000","last_unstale":"2024-09-17T04:25:08.127811+0000","last_undegraded":"2024-09-17T04:25:08.127811+0000","last_fullsized":"2024-09-17T04:25:08.127811+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:06:03.732941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.401903+0000","last_change":"2024-09-17T04:25:07.090513+0000","last_active":"2024-09-17T04:25:09.401903+0000","last_peered":"2024-09-17T04:25:09.401903+0000","last_clean":"2024-09-17T04:25:09.401903+0000","last_became_active":"2024-09-17T04:25:07.090181+0000","last_became_peered":"2024-09-17T04:25:07.090181+0000","last_unstale":"2024-09-17T04:25:09.401903+0000","last_undegraded":"2024-09-17T04:25:09.401903+0000","last_fullsized":"2024-09-17T04:25:09.401903+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:00.826540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.126484+0000","last_change":"2024-09-17T04:25:08.118894+0000","last_active":"2024-09-17T04:25:08.126484+0000","last_peered":"2024-09-17T04:25:08.126484+0000","last_clean":"2024-09-17T04:25:08.126484+0000","last_became_active":"2024-09-17T04:25:08.118563+0000","last_became_peered":"2024-09-17T04:25:08.118563+0000","last_unstale":"2024-09-17T04:25:08.126484+0000","last_undegraded":"2024-09-17T04:25:08.126484+0000","last_fullsized":"2024-09-17T04:25:08.126484+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:36.885529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455925+0000","last_change":"2024-09-17T04:25:07.103013+0000","last_active":"2024-09-17T04:25:09.455925+0000","last_peered":"2024-09-17T04:25:09.455925+0000","last_clean":"2024-09-17T04:25:09.455925+0000","last_became_active":"2024-09-17T04:25:07.100422+0000","last_became_peered":"2024-09-17T04:25:07.100422+0000","last_unstale":"2024-09-17T04:25:09.455925+0000","last_undegraded":"2024-09-17T04:25:09.455925+0000","last_fullsized":"2024-09-17T04:25:09.455925+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:10.495637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.454835+0000","last_change":"2024-09-17T04:25:08.125767+0000","last_active":"2024-09-17T04:25:09.454835+0000","last_peered":"2024-09-17T04:25:09.454835+0000","last_clean":"2024-09-17T04:25:09.454835+0000","last_became_active":"2024-09-17T04:25:08.125475+0000","last_became_peered":"2024-09-17T04:25:08.125475+0000","last_unstale":"2024-09-17T04:25:09.454835+0000","last_undegraded":"2024-09-17T04:25:09.454835+0000","last_fullsized":"2024-09-17T04:25:09.454835+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:38.129581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.105644+0000","last_change":"2024-09-17T04:25:07.097308+0000","last_active":"2024-09-17T04:25:09.105644+0000","last_peered":"2024-09-17T04:25:09.105644+0000","last_clean":"2024-09-17T04:25:09.105644+0000","last_became_active":"2024-09-17T04:25:07.096657+0000","last_became_peered":"2024-09-17T04:25:07.096657+0000","last_unstale":"2024-09-17T04:25:09.105644+0000","last_undegraded":"2024-09-17T04:25:09.105644+0000","last_fullsized":"2024-09-17T04:25:09.105644+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:49:55.787331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112543+0000","last_change":"2024-09-17T04:25:08.123602+0000","last_active":"2024-09-17T04:25:09.112543+0000","last_peered":"2024-09-17T04:25:09.112543+0000","last_clean":"2024-09-17T04:25:09.112543+0000","last_became_active":"2024-09-17T04:25:08.121886+0000","last_became_peered":"2024-09-17T04:25:08.121886+0000","last_unstale":"2024-09-17T04:25:09.112543+0000","last_undegraded":"2024-09-17T04:25:09.112543+0000","last_fullsized":"2024-09-17T04:25:09.112543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:59.322143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.123415+0000","last_change":"2024-09-17T04:25:07.093514+0000","last_active":"2024-09-17T04:25:08.123415+0000","last_peered":"2024-09-17T04:25:08.123415+0000","last_clean":"2024-09-17T04:25:08.123415+0000","last_became_active":"2024-09-17T04:25:07.091028+0000","last_became_peered":"2024-09-17T04:25:07.091028+0000","last_unstale":"2024-09-17T04:25:08.123415+0000","last_undegraded":"2024-09-17T04:25:08.123415+0000","last_fullsized":"2024-09-17T04:25:08.123415+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:47:16.719512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113223+0000","last_change":"2024-09-17T04:25:08.109606+0000","last_active":"2024-09-17T04:25:09.113223+0000","last_peered":"2024-09-17T04:25:09.113223+0000","last_clean":"2024-09-17T04:25:09.113223+0000","last_became_active":"2024-09-17T04:25:08.109288+0000","last_became_peered":"2024-09-17T04:25:08.109288+0000","last_unstale":"2024-09-17T04:25:09.113223+0000","last_undegraded":"2024-09-17T04:25:09.113223+0000","last_fullsized":"2024-09-17T04:25:09.113223+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:12:03.863249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.121410+0000","last_change":"2024-09-17T04:25:07.092548+0000","last_active":"2024-09-17T04:25:08.121410+0000","last_peered":"2024-09-17T04:25:08.121410+0000","last_clean":"2024-09-17T04:25:08.121410+0000","last_became_active":"2024-09-17T04:25:07.092222+0000","last_became_peered":"2024-09-17T04:25:07.092222+0000","last_unstale":"2024-09-17T04:25:08.121410+0000","last_undegraded":"2024-09-17T04:25:08.121410+0000","last_fullsized":"2024-09-17T04:25:08.121410+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:12.533380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.126405+0000","last_change":"2024-09-17T04:25:08.120205+0000","last_active":"2024-09-17T04:25:08.126405+0000","last_peered":"2024-09-17T04:25:08.126405+0000","last_clean":"2024-09-17T04:25:08.126405+0000","last_became_active":"2024-09-17T04:25:08.119921+0000","last_became_peered":"2024-09-17T04:25:08.119921+0000","last_unstale":"2024-09-17T04:25:08.126405+0000","last_undegraded":"2024-09-17T04:25:08.126405+0000","last_fullsized":"2024-09-17T04:25:08.126405+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:20.310340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455135+0000","last_change":"2024-09-17T04:25:07.096137+0000","last_active":"2024-09-17T04:25:09.455135+0000","last_peered":"2024-09-17T04:25:09.455135+0000","last_clean":"2024-09-17T04:25:09.455135+0000","last_became_active":"2024-09-17T04:25:07.095852+0000","last_became_peered":"2024-09-17T04:25:07.095852+0000","last_unstale":"2024-09-17T04:25:09.455135+0000","last_undegraded":"2024-09-17T04:25:09.455135+0000","last_fullsized":"2024-09-17T04:25:09.455135+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:35:37.851978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.454068+0000","last_change":"2024-09-17T04:25:08.107235+0000","last_active":"2024-09-17T04:25:09.454068+0000","last_peered":"2024-09-17T04:25:09.454068+0000","last_clean":"2024-09-17T04:25:09.454068+0000","last_became_active":"2024-09-17T04:25:08.106868+0000","last_became_peered":"2024-09-17T04:25:08.106868+0000","last_unstale":"2024-09-17T04:25:09.454068+0000","last_undegraded":"2024-09-17T04:25:09.454068+0000","last_fullsized":"2024-09-17T04:25:09.454068+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:26:25.532533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.100418+0000","last_change":"2024-09-17T04:25:07.090542+0000","last_active":"2024-09-17T04:25:08.100418+0000","last_peered":"2024-09-17T04:25:08.100418+0000","last_clean":"2024-09-17T04:25:08.100418+0000","last_became_active":"2024-09-17T04:25:07.087859+0000","last_became_peered":"2024-09-17T04:25:07.087859+0000","last_unstale":"2024-09-17T04:25:08.100418+0000","last_undegraded":"2024-09-17T04:25:08.100418+0000","last_fullsized":"2024-09-17T04:25:08.100418+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:35.343496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.114713+0000","last_change":"2024-09-17T04:25:08.120860+0000","last_active":"2024-09-17T04:25:09.114713+0000","last_peered":"2024-09-17T04:25:09.114713+0000","last_clean":"2024-09-17T04:25:09.114713+0000","last_became_active":"2024-09-17T04:25:08.119392+0000","last_became_peered":"2024-09-17T04:25:08.119392+0000","last_unstale":"2024-09-17T04:25:09.114713+0000","last_undegraded":"2024-09-17T04:25:09.114713+0000","last_fullsized":"2024-09-17T04:25:09.114713+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:52.755680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.456072+0000","last_change":"2024-09-17T04:25:07.103331+0000","last_active":"2024-09-17T04:25:09.456072+0000","last_peered":"2024-09-17T04:25:09.456072+0000","last_clean":"2024-09-17T04:25:09.456072+0000","last_became_active":"2024-09-17T04:25:07.102248+0000","last_became_peered":"2024-09-17T04:25:07.102248+0000","last_unstale":"2024-09-17T04:25:09.456072+0000","last_undegraded":"2024-09-17T04:25:09.456072+0000","last_fullsized":"2024-09-17T04:25:09.456072+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:09:07.109032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.456105+0000","last_change":"2024-09-17T04:25:08.119990+0000","last_active":"2024-09-17T04:25:09.456105+0000","last_peered":"2024-09-17T04:25:09.456105+0000","last_clean":"2024-09-17T04:25:09.456105+0000","last_became_active":"2024-09-17T04:25:08.119656+0000","last_became_peered":"2024-09-17T04:25:08.119656+0000","last_unstale":"2024-09-17T04:25:09.456105+0000","last_undegraded":"2024-09-17T04:25:09.456105+0000","last_fullsized":"2024-09-17T04:25:09.456105+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:15:59.588478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.402035+0000","last_change":"2024-09-17T04:25:07.097895+0000","last_active":"2024-09-17T04:25:09.402035+0000","last_peered":"2024-09-17T04:25:09.402035+0000","last_clean":"2024-09-17T04:25:09.402035+0000","last_became_active":"2024-09-17T04:25:07.097541+0000","last_became_peered":"2024-09-17T04:25:07.097541+0000","last_unstale":"2024-09-17T04:25:09.402035+0000","last_undegraded":"2024-09-17T04:25:09.402035+0000","last_fullsized":"2024-09-17T04:25:09.402035+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:49:00.272046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455727+0000","last_change":"2024-09-17T04:25:08.095787+0000","last_active":"2024-09-17T04:25:09.455727+0000","last_peered":"2024-09-17T04:25:09.455727+0000","last_clean":"2024-09-17T04:25:09.455727+0000","last_became_active":"2024-09-17T04:25:08.094877+0000","last_became_peered":"2024-09-17T04:25:08.094877+0000","last_unstale":"2024-09-17T04:25:09.455727+0000","last_undegraded":"2024-09-17T04:25:09.455727+0000","last_fullsized":"2024-09-17T04:25:09.455727+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:38.052155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.114464+0000","last_change":"2024-09-17T04:25:07.103494+0000","last_active":"2024-09-17T04:25:09.114464+0000","last_peered":"2024-09-17T04:25:09.114464+0000","last_clean":"2024-09-17T04:25:09.114464+0000","last_became_active":"2024-09-17T04:25:07.103202+0000","last_became_peered":"2024-09-17T04:25:07.103202+0000","last_unstale":"2024-09-17T04:25:09.114464+0000","last_undegraded":"2024-09-17T04:25:09.114464+0000","last_fullsized":"2024-09-17T04:25:09.114464+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:33.029608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.114318+0000","last_change":"2024-09-17T04:25:08.117481+0000","last_active":"2024-09-17T04:25:09.114318+0000","last_peered":"2024-09-17T04:25:09.114318+0000","last_clean":"2024-09-17T04:25:09.114318+0000","last_became_active":"2024-09-17T04:25:08.117068+0000","last_became_peered":"2024-09-17T04:25:08.117068+0000","last_unstale":"2024-09-17T04:25:09.114318+0000","last_undegraded":"2024-09-17T04:25:09.114318+0000","last_fullsized":"2024-09-17T04:25:09.114318+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:40.823059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":21,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455784+0000","last_change":"2024-09-17T04:25:07.099760+0000","last_active":"2024-09-17T04:25:09.455784+0000","last_peered":"2024-09-17T04:25:09.455784+0000","last_clean":"2024-09-17T04:25:09.455784+0000","last_became_active":"2024-09-17T04:25:07.098636+0000","last_became_peered":"2024-09-17T04:25:07.098636+0000","last_unstale":"2024-09-17T04:25:09.455784+0000","last_undegraded":"2024-09-17T04:25:09.455784+0000","last_fullsized":"2024-09-17T04:25:09.455784+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:58:57.715872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.454118+0000","last_change":"2024-09-17T04:25:08.105980+0000","last_active":"2024-09-17T04:25:09.454118+0000","last_peered":"2024-09-17T04:25:09.454118+0000","last_clean":"2024-09-17T04:25:09.454118+0000","last_became_active":"2024-09-17T04:25:08.105227+0000","last_became_peered":"2024-09-17T04:25:08.105227+0000","last_unstale":"2024-09-17T04:25:09.454118+0000","last_undegraded":"2024-09-17T04:25:09.454118+0000","last_fullsized":"2024-09-17T04:25:09.454118+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:57:19.324727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.124024+0000","last_change":"2024-09-17T04:25:07.090197+0000","last_active":"2024-09-17T04:25:08.124024+0000","last_peered":"2024-09-17T04:25:08.124024+0000","last_clean":"2024-09-17T04:25:08.124024+0000","last_became_active":"2024-09-17T04:25:07.089784+0000","last_became_peered":"2024-09-17T04:25:07.089784+0000","last_unstale":"2024-09-17T04:25:08.124024+0000","last_undegraded":"2024-09-17T04:25:08.124024+0000","last_fullsized":"2024-09-17T04:25:08.124024+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:09.843048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455543+0000","last_change":"2024-09-17T04:25:08.111349+0000","last_active":"2024-09-17T04:25:09.455543+0000","last_peered":"2024-09-17T04:25:09.455543+0000","last_clean":"2024-09-17T04:25:09.455543+0000","last_became_active":"2024-09-17T04:25:08.110979+0000","last_became_peered":"2024-09-17T04:25:08.110979+0000","last_unstale":"2024-09-17T04:25:09.455543+0000","last_undegraded":"2024-09-17T04:25:09.455543+0000","last_fullsized":"2024-09-17T04:25:09.455543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:19:23.569788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.454308+0000","last_change":"2024-09-17T04:25:07.100954+0000","last_active":"2024-09-17T04:25:09.454308+0000","last_peered":"2024-09-17T04:25:09.454308+0000","last_clean":"2024-09-17T04:25:09.454308+0000","last_became_active":"2024-09-17T04:25:07.100610+0000","last_became_peered":"2024-09-17T04:25:07.100610+0000","last_unstale":"2024-09-17T04:25:09.454308+0000","last_undegraded":"2024-09-17T04:25:09.454308+0000","last_fullsized":"2024-09-17T04:25:09.454308+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:14.912889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113053+0000","last_change":"2024-09-17T04:25:08.117150+0000","last_active":"2024-09-17T04:25:09.113053+0000","last_peered":"2024-09-17T04:25:09.113053+0000","last_clean":"2024-09-17T04:25:09.113053+0000","last_became_active":"2024-09-17T04:25:08.114530+0000","last_became_peered":"2024-09-17T04:25:08.114530+0000","last_unstale":"2024-09-17T04:25:09.113053+0000","last_undegraded":"2024-09-17T04:25:09.113053+0000","last_fullsized":"2024-09-17T04:25:09.113053+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:40.823059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455182+0000","last_change":"2024-09-17T04:25:07.101663+0000","last_active":"2024-09-17T04:25:09.455182+0000","last_peered":"2024-09-17T04:25:09.455182+0000","last_clean":"2024-09-17T04:25:09.455182+0000","last_became_active":"2024-09-17T04:25:07.100447+0000","last_became_peered":"2024-09-17T04:25:07.100447+0000","last_unstale":"2024-09-17T04:25:09.455182+0000","last_undegraded":"2024-09-17T04:25:09.455182+0000","last_fullsized":"2024-09-17T04:25:09.455182+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:43:37.274081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.131919+0000","last_change":"2024-09-17T04:25:08.124593+0000","last_active":"2024-09-17T04:25:08.131919+0000","last_peered":"2024-09-17T04:25:08.131919+0000","last_clean":"2024-09-17T04:25:08.131919+0000","last_became_active":"2024-09-17T04:25:08.121257+0000","last_became_peered":"2024-09-17T04:25:08.121257+0000","last_unstale":"2024-09-17T04:25:08.131919+0000","last_undegraded":"2024-09-17T04:25:08.131919+0000","last_fullsized":"2024-09-17T04:25:08.131919+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:02.807745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":23,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455231+0000","last_change":"2024-09-17T04:25:07.101694+0000","last_active":"2024-09-17T04:25:09.455231+0000","last_peered":"2024-09-17T04:25:09.455231+0000","last_clean":"2024-09-17T04:25:09.455231+0000","last_became_active":"2024-09-17T04:25:07.101006+0000","last_became_peered":"2024-09-17T04:25:07.101006+0000","last_unstale":"2024-09-17T04:25:09.455231+0000","last_undegraded":"2024-09-17T04:25:09.455231+0000","last_fullsized":"2024-09-17T04:25:09.455231+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:56.490357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.127019+0000","last_change":"2024-09-17T04:25:08.120978+0000","last_active":"2024-09-17T04:25:08.127019+0000","last_peered":"2024-09-17T04:25:08.127019+0000","last_clean":"2024-09-17T04:25:08.127019+0000","last_became_active":"2024-09-17T04:25:08.118804+0000","last_became_peered":"2024-09-17T04:25:08.118804+0000","last_unstale":"2024-09-17T04:25:08.127019+0000","last_undegraded":"2024-09-17T04:25:08.127019+0000","last_fullsized":"2024-09-17T04:25:08.127019+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:41.743123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455040+0000","last_change":"2024-09-17T04:25:07.100101+0000","last_active":"2024-09-17T04:25:09.455040+0000","last_peered":"2024-09-17T04:25:09.455040+0000","last_clean":"2024-09-17T04:25:09.455040+0000","last_became_active":"2024-09-17T04:25:07.099657+0000","last_became_peered":"2024-09-17T04:25:07.099657+0000","last_unstale":"2024-09-17T04:25:09.455040+0000","last_undegraded":"2024-09-17T04:25:09.455040+0000","last_fullsized":"2024-09-17T04:25:09.455040+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:06:02.726891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113955+0000","last_change":"2024-09-17T04:25:08.123388+0000","last_active":"2024-09-17T04:25:09.113955+0000","last_peered":"2024-09-17T04:25:09.113955+0000","last_clean":"2024-09-17T04:25:09.113955+0000","last_became_active":"2024-09-17T04:25:08.122769+0000","last_became_peered":"2024-09-17T04:25:08.122769+0000","last_unstale":"2024-09-17T04:25:09.113955+0000","last_undegraded":"2024-09-17T04:25:09.113955+0000","last_fullsized":"2024-09-17T04:25:09.113955+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:52.735017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.118988+0000","last_change":"2024-09-17T04:25:07.089581+0000","last_active":"2024-09-17T04:25:08.118988+0000","last_peered":"2024-09-17T04:25:08.118988+0000","last_clean":"2024-09-17T04:25:08.118988+0000","last_became_active":"2024-09-17T04:25:07.089256+0000","last_became_peered":"2024-09-17T04:25:07.089256+0000","last_unstale":"2024-09-17T04:25:08.118988+0000","last_undegraded":"2024-09-17T04:25:08.118988+0000","last_fullsized":"2024-09-17T04:25:08.118988+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:11:57.535300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.454526+0000","last_change":"2024-09-17T04:25:08.127043+0000","last_active":"2024-09-17T04:25:09.454526+0000","last_peered":"2024-09-17T04:25:09.454526+0000","last_clean":"2024-09-17T04:25:09.454526+0000","last_became_active":"2024-09-17T04:25:08.126806+0000","last_became_peered":"2024-09-17T04:25:08.126806+0000","last_unstale":"2024-09-17T04:25:09.454526+0000","last_undegraded":"2024-09-17T04:25:09.454526+0000","last_fullsized":"2024-09-17T04:25:09.454526+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:09:29.147789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.123222+0000","last_change":"2024-09-17T04:25:07.098143+0000","last_active":"2024-09-17T04:25:08.123222+0000","last_peered":"2024-09-17T04:25:08.123222+0000","last_clean":"2024-09-17T04:25:08.123222+0000","last_became_active":"2024-09-17T04:25:07.091768+0000","last_became_peered":"2024-09-17T04:25:07.091768+0000","last_unstale":"2024-09-17T04:25:08.123222+0000","last_undegraded":"2024-09-17T04:25:08.123222+0000","last_fullsized":"2024-09-17T04:25:08.123222+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:15:18.524829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.105728+0000","last_change":"2024-09-17T04:25:08.108470+0000","last_active":"2024-09-17T04:25:09.105728+0000","last_peered":"2024-09-17T04:25:09.105728+0000","last_clean":"2024-09-17T04:25:09.105728+0000","last_became_active":"2024-09-17T04:25:08.108144+0000","last_became_peered":"2024-09-17T04:25:08.108144+0000","last_unstale":"2024-09-17T04:25:09.105728+0000","last_undegraded":"2024-09-17T04:25:09.105728+0000","last_fullsized":"2024-09-17T04:25:09.105728+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:36.885529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.100120+0000","last_change":"2024-09-17T04:25:07.103224+0000","last_active":"2024-09-17T04:25:08.100120+0000","last_peered":"2024-09-17T04:25:08.100120+0000","last_clean":"2024-09-17T04:25:08.100120+0000","last_became_active":"2024-09-17T04:25:07.101489+0000","last_became_peered":"2024-09-17T04:25:07.101489+0000","last_unstale":"2024-09-17T04:25:08.100120+0000","last_undegraded":"2024-09-17T04:25:08.100120+0000","last_fullsized":"2024-09-17T04:25:08.100120+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:08.430358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.105325+0000","last_change":"2024-09-17T04:25:08.112749+0000","last_active":"2024-09-17T04:25:09.105325+0000","last_peered":"2024-09-17T04:25:09.105325+0000","last_clean":"2024-09-17T04:25:09.105325+0000","last_became_active":"2024-09-17T04:25:08.112408+0000","last_became_peered":"2024-09-17T04:25:08.112408+0000","last_unstale":"2024-09-17T04:25:09.105325+0000","last_undegraded":"2024-09-17T04:25:09.105325+0000","last_fullsized":"2024-09-17T04:25:09.105325+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:09.436407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.401785+0000","last_change":"2024-09-17T04:25:07.092602+0000","last_active":"2024-09-17T04:25:09.401785+0000","last_peered":"2024-09-17T04:25:09.401785+0000","last_clean":"2024-09-17T04:25:09.401785+0000","last_became_active":"2024-09-17T04:25:07.092215+0000","last_became_peered":"2024-09-17T04:25:07.092215+0000","last_unstale":"2024-09-17T04:25:09.401785+0000","last_undegraded":"2024-09-17T04:25:09.401785+0000","last_fullsized":"2024-09-17T04:25:09.401785+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:19.304290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.105634+0000","last_change":"2024-09-17T04:25:08.107413+0000","last_active":"2024-09-17T04:25:09.105634+0000","last_peered":"2024-09-17T04:25:09.105634+0000","last_clean":"2024-09-17T04:25:09.105634+0000","last_became_active":"2024-09-17T04:25:08.106516+0000","last_became_peered":"2024-09-17T04:25:08.106516+0000","last_unstale":"2024-09-17T04:25:09.105634+0000","last_undegraded":"2024-09-17T04:25:09.105634+0000","last_fullsized":"2024-09-17T04:25:09.105634+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:10.849097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.103107+0000","last_change":"2024-09-17T04:25:07.101091+0000","last_active":"2024-09-17T04:25:08.103107+0000","last_peered":"2024-09-17T04:25:08.103107+0000","last_clean":"2024-09-17T04:25:08.103107+0000","last_became_active":"2024-09-17T04:25:07.100821+0000","last_became_peered":"2024-09-17T04:25:07.100821+0000","last_unstale":"2024-09-17T04:25:08.103107+0000","last_undegraded":"2024-09-17T04:25:08.103107+0000","last_fullsized":"2024-09-17T04:25:08.103107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:50:25.153746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.105405+0000","last_change":"2024-09-17T04:25:08.104836+0000","last_active":"2024-09-17T04:25:09.105405+0000","last_peered":"2024-09-17T04:25:09.105405+0000","last_clean":"2024-09-17T04:25:09.105405+0000","last_became_active":"2024-09-17T04:25:08.104120+0000","last_became_peered":"2024-09-17T04:25:08.104120+0000","last_unstale":"2024-09-17T04:25:09.105405+0000","last_undegraded":"2024-09-17T04:25:09.105405+0000","last_fullsized":"2024-09-17T04:25:09.105405+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:47:17.725561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.097525+0000","last_change":"2024-09-17T04:25:07.103534+0000","last_active":"2024-09-17T04:25:08.097525+0000","last_peered":"2024-09-17T04:25:08.097525+0000","last_clean":"2024-09-17T04:25:08.097525+0000","last_became_active":"2024-09-17T04:25:07.103070+0000","last_became_peered":"2024-09-17T04:25:07.103070+0000","last_unstale":"2024-09-17T04:25:08.097525+0000","last_undegraded":"2024-09-17T04:25:08.097525+0000","last_fullsized":"2024-09-17T04:25:08.097525+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:57:15.056986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113955+0000","last_change":"2024-09-17T04:25:08.113398+0000","last_active":"2024-09-17T04:25:09.113955+0000","last_peered":"2024-09-17T04:25:09.113955+0000","last_clean":"2024-09-17T04:25:09.113955+0000","last_became_active":"2024-09-17T04:25:08.113049+0000","last_became_peered":"2024-09-17T04:25:08.113049+0000","last_unstale":"2024-09-17T04:25:09.113955+0000","last_undegraded":"2024-09-17T04:25:09.113955+0000","last_fullsized":"2024-09-17T04:25:09.113955+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:12.516316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.111837+0000","last_change":"2024-09-17T04:25:07.103861+0000","last_active":"2024-09-17T04:25:09.111837+0000","last_peered":"2024-09-17T04:25:09.111837+0000","last_clean":"2024-09-17T04:25:09.111837+0000","last_became_active":"2024-09-17T04:25:07.100484+0000","last_became_peered":"2024-09-17T04:25:07.100484+0000","last_unstale":"2024-09-17T04:25:09.111837+0000","last_undegraded":"2024-09-17T04:25:09.111837+0000","last_fullsized":"2024-09-17T04:25:09.111837+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:27.284841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.105892+0000","last_change":"2024-09-17T04:25:08.100426+0000","last_active":"2024-09-17T04:25:09.105892+0000","last_peered":"2024-09-17T04:25:09.105892+0000","last_clean":"2024-09-17T04:25:09.105892+0000","last_became_active":"2024-09-17T04:25:08.100117+0000","last_became_peered":"2024-09-17T04:25:08.100117+0000","last_unstale":"2024-09-17T04:25:09.105892+0000","last_undegraded":"2024-09-17T04:25:09.105892+0000","last_fullsized":"2024-09-17T04:25:09.105892+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:49:01.278096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455112+0000","last_change":"2024-09-17T04:25:07.097884+0000","last_active":"2024-09-17T04:25:09.455112+0000","last_peered":"2024-09-17T04:25:09.455112+0000","last_clean":"2024-09-17T04:25:09.455112+0000","last_became_active":"2024-09-17T04:25:07.097545+0000","last_became_peered":"2024-09-17T04:25:07.097545+0000","last_unstale":"2024-09-17T04:25:09.455112+0000","last_undegraded":"2024-09-17T04:25:09.455112+0000","last_fullsized":"2024-09-17T04:25:09.455112+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:01.801695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.114256+0000","last_change":"2024-09-17T04:25:08.123061+0000","last_active":"2024-09-17T04:25:09.114256+0000","last_peered":"2024-09-17T04:25:09.114256+0000","last_clean":"2024-09-17T04:25:09.114256+0000","last_became_active":"2024-09-17T04:25:08.122319+0000","last_became_peered":"2024-09-17T04:25:08.122319+0000","last_unstale":"2024-09-17T04:25:09.114256+0000","last_undegraded":"2024-09-17T04:25:09.114256+0000","last_fullsized":"2024-09-17T04:25:09.114256+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:22.538564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"36'1","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.402008+0000","last_change":"2024-09-17T04:25:07.088222+0000","last_active":"2024-09-17T04:25:09.402008+0000","last_peered":"2024-09-17T04:25:09.402008+0000","last_clean":"2024-09-17T04:25:09.402008+0000","last_became_active":"2024-09-17T04:25:07.087921+0000","last_became_peered":"2024-09-17T04:25:07.087921+0000","last_unstale":"2024-09-17T04:25:09.402008+0000","last_undegraded":"2024-09-17T04:25:09.402008+0000","last_fullsized":"2024-09-17T04:25:09.402008+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:35.879479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.401578+0000","last_change":"2024-09-17T04:25:08.115284+0000","last_active":"2024-09-17T04:25:09.401578+0000","last_peered":"2024-09-17T04:25:09.401578+0000","last_clean":"2024-09-17T04:25:09.401578+0000","last_became_active":"2024-09-17T04:25:08.114998+0000","last_became_peered":"2024-09-17T04:25:08.114998+0000","last_unstale":"2024-09-17T04:25:09.401578+0000","last_undegraded":"2024-09-17T04:25:09.401578+0000","last_fullsized":"2024-09-17T04:25:09.401578+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:21.841901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.454596+0000","last_change":"2024-09-17T04:25:07.096188+0000","last_active":"2024-09-17T04:25:09.454596+0000","last_peered":"2024-09-17T04:25:09.454596+0000","last_clean":"2024-09-17T04:25:09.454596+0000","last_became_active":"2024-09-17T04:25:07.094697+0000","last_became_peered":"2024-09-17T04:25:07.094697+0000","last_unstale":"2024-09-17T04:25:09.454596+0000","last_undegraded":"2024-09-17T04:25:09.454596+0000","last_fullsized":"2024-09-17T04:25:09.454596+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:05.174227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.120861+0000","last_change":"2024-09-17T04:25:08.115704+0000","last_active":"2024-09-17T04:25:08.120861+0000","last_peered":"2024-09-17T04:25:08.120861+0000","last_clean":"2024-09-17T04:25:08.120861+0000","last_became_active":"2024-09-17T04:25:08.115380+0000","last_became_peered":"2024-09-17T04:25:08.115380+0000","last_unstale":"2024-09-17T04:25:08.120861+0000","last_undegraded":"2024-09-17T04:25:08.120861+0000","last_fullsized":"2024-09-17T04:25:08.120861+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:21.618822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.401941+0000","last_change":"2024-09-17T04:25:07.095160+0000","last_active":"2024-09-17T04:25:09.401941+0000","last_peered":"2024-09-17T04:25:09.401941+0000","last_clean":"2024-09-17T04:25:09.401941+0000","last_became_active":"2024-09-17T04:25:07.094948+0000","last_became_peered":"2024-09-17T04:25:07.094948+0000","last_unstale":"2024-09-17T04:25:09.401941+0000","last_undegraded":"2024-09-17T04:25:09.401941+0000","last_fullsized":"2024-09-17T04:25:09.401941+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:40.737074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.401823+0000","last_change":"2024-09-17T04:25:08.120140+0000","last_active":"2024-09-17T04:25:09.401823+0000","last_peered":"2024-09-17T04:25:09.401823+0000","last_clean":"2024-09-17T04:25:09.401823+0000","last_became_active":"2024-09-17T04:25:08.118074+0000","last_became_peered":"2024-09-17T04:25:08.118074+0000","last_unstale":"2024-09-17T04:25:09.401823+0000","last_undegraded":"2024-09-17T04:25:09.401823+0000","last_fullsized":"2024-09-17T04:25:09.401823+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:03.317036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.454312+0000","last_change":"2024-09-17T04:25:07.094913+0000","last_active":"2024-09-17T04:25:09.454312+0000","last_peered":"2024-09-17T04:25:09.454312+0000","last_clean":"2024-09-17T04:25:09.454312+0000","last_became_active":"2024-09-17T04:25:07.094618+0000","last_became_peered":"2024-09-17T04:25:07.094618+0000","last_unstale":"2024-09-17T04:25:09.454312+0000","last_undegraded":"2024-09-17T04:25:09.454312+0000","last_fullsized":"2024-09-17T04:25:09.454312+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:20.612773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.454505+0000","last_change":"2024-09-17T04:25:08.118075+0000","last_active":"2024-09-17T04:25:09.454505+0000","last_peered":"2024-09-17T04:25:09.454505+0000","last_clean":"2024-09-17T04:25:09.454505+0000","last_became_active":"2024-09-17T04:25:08.117697+0000","last_became_peered":"2024-09-17T04:25:08.117697+0000","last_unstale":"2024-09-17T04:25:09.454505+0000","last_undegraded":"2024-09-17T04:25:09.454505+0000","last_fullsized":"2024-09-17T04:25:09.454505+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:24.944294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.097817+0000","last_change":"2024-09-17T04:25:07.093510+0000","last_active":"2024-09-17T04:25:08.097817+0000","last_peered":"2024-09-17T04:25:08.097817+0000","last_clean":"2024-09-17T04:25:08.097817+0000","last_became_active":"2024-09-17T04:25:07.093132+0000","last_became_peered":"2024-09-17T04:25:07.093132+0000","last_unstale":"2024-09-17T04:25:08.097817+0000","last_undegraded":"2024-09-17T04:25:08.097817+0000","last_fullsized":"2024-09-17T04:25:08.097817+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:04:39.641973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.132867+0000","last_change":"2024-09-17T04:25:08.127561+0000","last_active":"2024-09-17T04:25:08.132867+0000","last_peered":"2024-09-17T04:25:08.132867+0000","last_clean":"2024-09-17T04:25:08.132867+0000","last_became_active":"2024-09-17T04:25:08.127108+0000","last_became_peered":"2024-09-17T04:25:08.127108+0000","last_unstale":"2024-09-17T04:25:08.132867+0000","last_undegraded":"2024-09-17T04:25:08.132867+0000","last_fullsized":"2024-09-17T04:25:08.132867+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:40:24.798987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455032+0000","last_change":"2024-09-17T04:25:07.101483+0000","last_active":"2024-09-17T04:25:09.455032+0000","last_peered":"2024-09-17T04:25:09.455032+0000","last_clean":"2024-09-17T04:25:09.455032+0000","last_became_active":"2024-09-17T04:25:07.100021+0000","last_became_peered":"2024-09-17T04:25:07.100021+0000","last_unstale":"2024-09-17T04:25:09.455032+0000","last_undegraded":"2024-09-17T04:25:09.455032+0000","last_fullsized":"2024-09-17T04:25:09.455032+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:12:47.750660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112185+0000","last_change":"2024-09-17T04:25:08.102453+0000","last_active":"2024-09-17T04:25:09.112185+0000","last_peered":"2024-09-17T04:25:09.112185+0000","last_clean":"2024-09-17T04:25:09.112185+0000","last_became_active":"2024-09-17T04:25:08.101181+0000","last_became_peered":"2024-09-17T04:25:08.101181+0000","last_unstale":"2024-09-17T04:25:09.112185+0000","last_undegraded":"2024-09-17T04:25:09.112185+0000","last_fullsized":"2024-09-17T04:25:09.112185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:11.397751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-17T04:25:08.103937+0000","last_change":"2024-09-17T04:25:07.105074+0000","last_active":"2024-09-17T04:25:08.103937+0000","last_peered":"2024-09-17T04:25:08.103937+0000","last_clean":"2024-09-17T04:25:08.103937+0000","last_became_active":"2024-09-17T04:25:07.104735+0000","last_became_peered":"2024-09-17T04:25:07.104735+0000","last_unstale":"2024-09-17T04:25:08.103937+0000","last_undegraded":"2024-09-17T04:25:08.103937+0000","last_fullsized":"2024-09-17T04:25:08.103937+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:06:46.154246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":136,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113124+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:25:09.113124+0000","last_peered":"2024-09-17T04:25:09.113124+0000","last_clean":"2024-09-17T04:25:09.113124+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:25:09.113124+0000","last_undegraded":"2024-09-17T04:25:09.113124+0000","last_fullsized":"2024-09-17T04:25:09.113124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476754,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":739667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476753,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":149825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476754,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":149881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476754,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":393216,"data_stored":149313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509458,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":401408,"data_stored":155579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509457,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":413696,"data_stored":157143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":991232,"data_stored":745877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542162,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":413696,"data_stored":157143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:25:17.525 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:25:17.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.752+0000 7f9daac00640 1 -- 172.21.15.100:0/2326518608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9da40ad3c0 msgr2=0x7f9da40ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:17.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.752+0000 7f9daac00640 1 --2- 172.21.15.100:0/2326518608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9da40ad3c0 0x7f9da40ad7d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9d90005fd0 tx=0x7f9d9002a540 comp rx=0 tx=0).stop 2024-09-17T04:25:17.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.753+0000 7f9daac00640 1 -- 172.21.15.100:0/2326518608 shutdown_connections 2024-09-17T04:25:17.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.753+0000 7f9daac00640 1 --2- 172.21.15.100:0/2326518608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9da40ad3c0 0x7f9da40ad7d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:17.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.753+0000 7f9daac00640 1 --2- 172.21.15.100:0/2326518608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9da40704b0 0x7f9da4073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:17.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.753+0000 7f9daac00640 1 -- 172.21.15.100:0/2326518608 >> 172.21.15.100:0/2326518608 conn(0x7f9da406d730 msgr2=0x7f9da40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:17.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.753+0000 7f9daac00640 1 -- 172.21.15.100:0/2326518608 shutdown_connections 2024-09-17T04:25:17.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.753+0000 7f9daac00640 1 -- 172.21.15.100:0/2326518608 wait complete. 2024-09-17T04:25:17.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.755+0000 7f9daac00640 1 Processor -- start 2024-09-17T04:25:17.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.755+0000 7f9daac00640 1 -- start start 2024-09-17T04:25:17.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.756+0000 7f9daac00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9da40704b0 0x7f9da41f7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:17.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.756+0000 7f9daac00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9da40ad3c0 0x7f9da414d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:17.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.756+0000 7f9daac00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9da40add10 0x7f9da409c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:17.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.756+0000 7f9daac00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9da40712b0 con 0x7f9da40704b0 2024-09-17T04:25:17.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.756+0000 7f9daac00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9da4071130 con 0x7f9da40add10 2024-09-17T04:25:17.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.756+0000 7f9daac00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9da40b4520 con 0x7f9da40ad3c0 2024-09-17T04:25:17.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.757+0000 7f9da3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9da40704b0 0x7f9da41f7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:17.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.757+0000 7f9da3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9da40704b0 0x7f9da41f7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57126/0 (socket says 172.21.15.100:57126) 2024-09-17T04:25:17.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.757+0000 7f9da3e00640 1 -- 172.21.15.100:0/3939616714 learned_addr learned my addr 172.21.15.100:0/3939616714 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:17.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.757+0000 7f9da3400640 1 --2- 172.21.15.100:0/3939616714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9da40ad3c0 0x7f9da414d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:17.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.757+0000 7f9da8c00640 1 --2- 172.21.15.100:0/3939616714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9da40add10 0x7f9da409c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:17.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.757+0000 7f9da3e00640 1 -- 172.21.15.100:0/3939616714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9da40ad3c0 msgr2=0x7f9da414d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:17.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.757+0000 7f9da3e00640 1 --2- 172.21.15.100:0/3939616714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9da40ad3c0 0x7f9da414d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:17.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.757+0000 7f9da3e00640 1 -- 172.21.15.100:0/3939616714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9da40add10 msgr2=0x7f9da409c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:17.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.757+0000 7f9da3e00640 1 --2- 172.21.15.100:0/3939616714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9da40add10 0x7f9da409c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:17.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.757+0000 7f9da3e00640 1 -- 172.21.15.100:0/3939616714 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da409b080 con 0x7f9da40704b0 2024-09-17T04:25:17.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.757+0000 7f9da8c00640 1 --2- 172.21.15.100:0/3939616714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9da40add10 0x7f9da409c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:25:17.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.757+0000 7f9da3e00640 1 --2- 172.21.15.100:0/3939616714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9da40704b0 0x7f9da41f7970 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400e790 tx=0x7f9d9400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:17.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.758+0000 7f9d9be00640 1 -- 172.21.15.100:0/3939616714 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9400eea0 con 0x7f9da40704b0 2024-09-17T04:25:17.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.758+0000 7f9d9be00640 1 -- 172.21.15.100:0/3939616714 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d9400cd50 con 0x7f9da40704b0 2024-09-17T04:25:17.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.758+0000 7f9d9be00640 1 -- 172.21.15.100:0/3939616714 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94010690 con 0x7f9da40704b0 2024-09-17T04:25:17.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.758+0000 7f9daac00640 1 -- 172.21.15.100:0/3939616714 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da414cab0 con 0x7f9da40704b0 2024-09-17T04:25:17.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.758+0000 7f9daac00640 1 -- 172.21.15.100:0/3939616714 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da414d070 con 0x7f9da40704b0 2024-09-17T04:25:17.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.759+0000 7f9daac00640 1 -- 172.21.15.100:0/3939616714 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9da40b49d0 con 0x7f9da40704b0 2024-09-17T04:25:17.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.760+0000 7f9d9be00640 1 -- 172.21.15.100:0/3939616714 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9d94010830 con 0x7f9da40704b0 2024-09-17T04:25:17.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.761+0000 7f9d9be00640 1 --2- 172.21.15.100:0/3939616714 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9d84078b90 0x7f9d8407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:17.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.761+0000 7f9d9be00640 1 -- 172.21.15.100:0/3939616714 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f9d9409b710 con 0x7f9da40704b0 2024-09-17T04:25:17.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.761+0000 7f9da3400640 1 --2- 172.21.15.100:0/3939616714 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9d84078b90 0x7f9d8407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:17.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.762+0000 7f9da3400640 1 --2- 172.21.15.100:0/3939616714 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9d84078b90 0x7f9d8407b0a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9d90033000 tx=0x7f9d90004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:17.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.765+0000 7f9d9be00640 1 -- 172.21.15.100:0/3939616714 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9d94063b40 con 0x7f9da40704b0 2024-09-17T04:25:17.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.944+0000 7f9daac00640 1 -- 172.21.15.100:0/3939616714 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9da415e420 con 0x7f9d84078b90 2024-09-17T04:25:17.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.950+0000 7f9d9be00640 1 -- 172.21.15.100:0/3939616714 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251892 (secure 0 0 0) 0x7f9da415e420 con 0x7f9d84078b90 2024-09-17T04:25:17.952 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:17.954 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:25:17.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.953+0000 7f9daac00640 1 -- 172.21.15.100:0/3939616714 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9d84078b90 msgr2=0x7f9d8407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:17.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.953+0000 7f9daac00640 1 --2- 172.21.15.100:0/3939616714 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9d84078b90 0x7f9d8407b0a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9d90033000 tx=0x7f9d90004730 comp rx=0 tx=0).stop 2024-09-17T04:25:17.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.953+0000 7f9daac00640 1 -- 172.21.15.100:0/3939616714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9da40704b0 msgr2=0x7f9da41f7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:17.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.953+0000 7f9daac00640 1 --2- 172.21.15.100:0/3939616714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9da40704b0 0x7f9da41f7970 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400e790 tx=0x7f9d9400ec50 comp rx=0 tx=0).stop 2024-09-17T04:25:17.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.953+0000 7f9daac00640 1 -- 172.21.15.100:0/3939616714 shutdown_connections 2024-09-17T04:25:17.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.953+0000 7f9daac00640 1 --2- 172.21.15.100:0/3939616714 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9d84078b90 0x7f9d8407b0a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:17.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.953+0000 7f9daac00640 1 --2- 172.21.15.100:0/3939616714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9da40add10 0x7f9da409c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:17.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.953+0000 7f9daac00640 1 --2- 172.21.15.100:0/3939616714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9da40ad3c0 0x7f9da414d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:17.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.953+0000 7f9daac00640 1 --2- 172.21.15.100:0/3939616714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9da40704b0 0x7f9da41f7970 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:17.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.953+0000 7f9daac00640 1 -- 172.21.15.100:0/3939616714 >> 172.21.15.100:0/3939616714 conn(0x7f9da406d730 msgr2=0x7f9da40b3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:17.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.953+0000 7f9daac00640 1 -- 172.21.15.100:0/3939616714 shutdown_connections 2024-09-17T04:25:17.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:17.953+0000 7f9daac00640 1 -- 172.21.15.100:0/3939616714 wait complete. 2024-09-17T04:25:17.975 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-09-17T04:25:15.980806+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220584,"kb_used_data":4328,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754136,"statfs":{"total":773094113280,"available":772868235264,"internally_reserved":0,"allocated":4431872,"data_stored":2429628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002267"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455361+0000","last_change":"2024-09-17T04:25:08.126623+0000","last_active":"2024-09-17T04:25:09.455361+0000","last_peered":"2024-09-17T04:25:09.455361+0000","last_clean":"2024-09-17T04:25:09.455361+0000","last_became_active":"2024-09-17T04:25:08.126204+0000","last_became_peered":"2024-09-17T04:25:08.126204+0000","last_unstale":"2024-09-17T04:25:09.455361+0000","last_undegraded":"2024-09-17T04:25:09.455361+0000","last_fullsized":"2024-09-17T04:25:09.455361+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:34.035658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.456067+0000","last_change":"2024-09-17T04:25:08.116947+0000","last_active":"2024-09-17T04:25:09.456067+0000","last_peered":"2024-09-17T04:25:09.456067+0000","last_clean":"2024-09-17T04:25:09.456067+0000","last_became_active":"2024-09-17T04:25:08.115387+0000","last_became_peered":"2024-09-17T04:25:08.115387+0000","last_unstale":"2024-09-17T04:25:09.456067+0000","last_undegraded":"2024-09-17T04:25:09.456067+0000","last_fullsized":"2024-09-17T04:25:09.456067+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:39:50.486405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.114386+0000","last_change":"2024-09-17T04:25:08.116641+0000","last_active":"2024-09-17T04:25:09.114386+0000","last_peered":"2024-09-17T04:25:09.114386+0000","last_clean":"2024-09-17T04:25:09.114386+0000","last_became_active":"2024-09-17T04:25:08.115852+0000","last_became_peered":"2024-09-17T04:25:08.115852+0000","last_unstale":"2024-09-17T04:25:09.114386+0000","last_undegraded":"2024-09-17T04:25:09.114386+0000","last_fullsized":"2024-09-17T04:25:09.114386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:46.534540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.454354+0000","last_change":"2024-09-17T04:25:08.127729+0000","last_active":"2024-09-17T04:25:09.454354+0000","last_peered":"2024-09-17T04:25:09.454354+0000","last_clean":"2024-09-17T04:25:09.454354+0000","last_became_active":"2024-09-17T04:25:08.125546+0000","last_became_peered":"2024-09-17T04:25:08.125546+0000","last_unstale":"2024-09-17T04:25:09.454354+0000","last_undegraded":"2024-09-17T04:25:09.454354+0000","last_fullsized":"2024-09-17T04:25:09.454354+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:33:23.179319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.105372+0000","last_change":"2024-09-17T04:25:08.113523+0000","last_active":"2024-09-17T04:25:09.105372+0000","last_peered":"2024-09-17T04:25:09.105372+0000","last_clean":"2024-09-17T04:25:09.105372+0000","last_became_active":"2024-09-17T04:25:08.113207+0000","last_became_peered":"2024-09-17T04:25:08.113207+0000","last_unstale":"2024-09-17T04:25:09.105372+0000","last_undegraded":"2024-09-17T04:25:09.105372+0000","last_fullsized":"2024-09-17T04:25:09.105372+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:36.349545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455354+0000","last_change":"2024-09-17T04:25:08.101238+0000","last_active":"2024-09-17T04:25:09.455354+0000","last_peered":"2024-09-17T04:25:09.455354+0000","last_clean":"2024-09-17T04:25:09.455354+0000","last_became_active":"2024-09-17T04:25:08.099079+0000","last_became_peered":"2024-09-17T04:25:08.099079+0000","last_unstale":"2024-09-17T04:25:09.455354+0000","last_undegraded":"2024-09-17T04:25:09.455354+0000","last_fullsized":"2024-09-17T04:25:09.455354+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:01.930915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113556+0000","last_change":"2024-09-17T04:25:08.113471+0000","last_active":"2024-09-17T04:25:09.113556+0000","last_peered":"2024-09-17T04:25:09.113556+0000","last_clean":"2024-09-17T04:25:09.113556+0000","last_became_active":"2024-09-17T04:25:08.113145+0000","last_became_peered":"2024-09-17T04:25:08.113145+0000","last_unstale":"2024-09-17T04:25:09.113556+0000","last_undegraded":"2024-09-17T04:25:09.113556+0000","last_fullsized":"2024-09-17T04:25:09.113556+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:52.755680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112954+0000","last_change":"2024-09-17T04:25:08.102460+0000","last_active":"2024-09-17T04:25:09.112954+0000","last_peered":"2024-09-17T04:25:09.112954+0000","last_clean":"2024-09-17T04:25:09.112954+0000","last_became_active":"2024-09-17T04:25:08.101474+0000","last_became_peered":"2024-09-17T04:25:08.101474+0000","last_unstale":"2024-09-17T04:25:09.112954+0000","last_undegraded":"2024-09-17T04:25:09.112954+0000","last_fullsized":"2024-09-17T04:25:09.112954+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:22.538564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.332698+0000","last_change":"2024-09-17T04:25:08.119819+0000","last_active":"2024-09-17T04:25:09.332698+0000","last_peered":"2024-09-17T04:25:09.332698+0000","last_clean":"2024-09-17T04:25:09.332698+0000","last_became_active":"2024-09-17T04:25:08.119511+0000","last_became_peered":"2024-09-17T04:25:08.119511+0000","last_unstale":"2024-09-17T04:25:09.332698+0000","last_undegraded":"2024-09-17T04:25:09.332698+0000","last_fullsized":"2024-09-17T04:25:09.332698+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:33:23.179319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455421+0000","last_change":"2024-09-17T04:25:08.121720+0000","last_active":"2024-09-17T04:25:09.455421+0000","last_peered":"2024-09-17T04:25:09.455421+0000","last_clean":"2024-09-17T04:25:09.455421+0000","last_became_active":"2024-09-17T04:25:08.121405+0000","last_became_peered":"2024-09-17T04:25:08.121405+0000","last_unstale":"2024-09-17T04:25:09.455421+0000","last_undegraded":"2024-09-17T04:25:09.455421+0000","last_fullsized":"2024-09-17T04:25:09.455421+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:11.397751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112530+0000","last_change":"2024-09-17T04:25:08.116374+0000","last_active":"2024-09-17T04:25:09.112530+0000","last_peered":"2024-09-17T04:25:09.112530+0000","last_clean":"2024-09-17T04:25:09.112530+0000","last_became_active":"2024-09-17T04:25:08.116047+0000","last_became_peered":"2024-09-17T04:25:08.116047+0000","last_unstale":"2024-09-17T04:25:09.112530+0000","last_undegraded":"2024-09-17T04:25:09.112530+0000","last_fullsized":"2024-09-17T04:25:09.112530+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:12:03.863249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113687+0000","last_change":"2024-09-17T04:25:08.117853+0000","last_active":"2024-09-17T04:25:09.113687+0000","last_peered":"2024-09-17T04:25:09.113687+0000","last_clean":"2024-09-17T04:25:09.113687+0000","last_became_active":"2024-09-17T04:25:08.117342+0000","last_became_peered":"2024-09-17T04:25:08.117342+0000","last_unstale":"2024-09-17T04:25:09.113687+0000","last_undegraded":"2024-09-17T04:25:09.113687+0000","last_fullsized":"2024-09-17T04:25:09.113687+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:59.322143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.105491+0000","last_change":"2024-09-17T04:25:08.107583+0000","last_active":"2024-09-17T04:25:09.105491+0000","last_peered":"2024-09-17T04:25:09.105491+0000","last_clean":"2024-09-17T04:25:09.105491+0000","last_became_active":"2024-09-17T04:25:08.107097+0000","last_became_peered":"2024-09-17T04:25:08.107097+0000","last_unstale":"2024-09-17T04:25:09.105491+0000","last_undegraded":"2024-09-17T04:25:09.105491+0000","last_fullsized":"2024-09-17T04:25:09.105491+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:13.539429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455604+0000","last_change":"2024-09-17T04:25:08.092773+0000","last_active":"2024-09-17T04:25:09.455604+0000","last_peered":"2024-09-17T04:25:09.455604+0000","last_clean":"2024-09-17T04:25:09.455604+0000","last_became_active":"2024-09-17T04:25:08.092437+0000","last_became_peered":"2024-09-17T04:25:08.092437+0000","last_unstale":"2024-09-17T04:25:09.455604+0000","last_undegraded":"2024-09-17T04:25:09.455604+0000","last_fullsized":"2024-09-17T04:25:09.455604+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:51.218111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112471+0000","last_change":"2024-09-17T04:25:08.122562+0000","last_active":"2024-09-17T04:25:09.112471+0000","last_peered":"2024-09-17T04:25:09.112471+0000","last_clean":"2024-09-17T04:25:09.112471+0000","last_became_active":"2024-09-17T04:25:08.120406+0000","last_became_peered":"2024-09-17T04:25:08.120406+0000","last_unstale":"2024-09-17T04:25:09.112471+0000","last_undegraded":"2024-09-17T04:25:09.112471+0000","last_fullsized":"2024-09-17T04:25:09.112471+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:14.660711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113172+0000","last_change":"2024-09-17T04:25:08.111402+0000","last_active":"2024-09-17T04:25:09.113172+0000","last_peered":"2024-09-17T04:25:09.113172+0000","last_clean":"2024-09-17T04:25:09.113172+0000","last_became_active":"2024-09-17T04:25:08.111049+0000","last_became_peered":"2024-09-17T04:25:08.111049+0000","last_unstale":"2024-09-17T04:25:09.113172+0000","last_undegraded":"2024-09-17T04:25:09.113172+0000","last_fullsized":"2024-09-17T04:25:09.113172+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:46.534540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112688+0000","last_change":"2024-09-17T04:25:08.113695+0000","last_active":"2024-09-17T04:25:09.112688+0000","last_peered":"2024-09-17T04:25:09.112688+0000","last_clean":"2024-09-17T04:25:09.112688+0000","last_became_active":"2024-09-17T04:25:08.113248+0000","last_became_peered":"2024-09-17T04:25:08.113248+0000","last_unstale":"2024-09-17T04:25:09.112688+0000","last_undegraded":"2024-09-17T04:25:09.112688+0000","last_fullsized":"2024-09-17T04:25:09.112688+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:52.735017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.456121+0000","last_change":"2024-09-17T04:25:08.112012+0000","last_active":"2024-09-17T04:25:09.456121+0000","last_peered":"2024-09-17T04:25:09.456121+0000","last_clean":"2024-09-17T04:25:09.456121+0000","last_became_active":"2024-09-17T04:25:08.111723+0000","last_became_peered":"2024-09-17T04:25:08.111723+0000","last_unstale":"2024-09-17T04:25:09.456121+0000","last_undegraded":"2024-09-17T04:25:09.456121+0000","last_fullsized":"2024-09-17T04:25:09.456121+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:43:52.912008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455267+0000","last_change":"2024-09-17T04:25:08.120093+0000","last_active":"2024-09-17T04:25:09.455267+0000","last_peered":"2024-09-17T04:25:09.455267+0000","last_clean":"2024-09-17T04:25:09.455267+0000","last_became_active":"2024-09-17T04:25:08.119846+0000","last_became_peered":"2024-09-17T04:25:08.119846+0000","last_unstale":"2024-09-17T04:25:09.455267+0000","last_undegraded":"2024-09-17T04:25:09.455267+0000","last_fullsized":"2024-09-17T04:25:09.455267+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:06:21.034599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.105643+0000","last_change":"2024-09-17T04:25:08.102248+0000","last_active":"2024-09-17T04:25:09.105643+0000","last_peered":"2024-09-17T04:25:09.105643+0000","last_clean":"2024-09-17T04:25:09.105643+0000","last_became_active":"2024-09-17T04:25:08.101915+0000","last_became_peered":"2024-09-17T04:25:08.101915+0000","last_unstale":"2024-09-17T04:25:09.105643+0000","last_undegraded":"2024-09-17T04:25:09.105643+0000","last_fullsized":"2024-09-17T04:25:09.105643+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:01.832590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.332051+0000","last_change":"2024-09-17T04:25:08.116792+0000","last_active":"2024-09-17T04:25:09.332051+0000","last_peered":"2024-09-17T04:25:09.332051+0000","last_clean":"2024-09-17T04:25:09.332051+0000","last_became_active":"2024-09-17T04:25:08.116468+0000","last_became_peered":"2024-09-17T04:25:08.116468+0000","last_unstale":"2024-09-17T04:25:09.332051+0000","last_undegraded":"2024-09-17T04:25:09.332051+0000","last_fullsized":"2024-09-17T04:25:09.332051+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:47:20.286239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.331817+0000","last_change":"2024-09-17T04:25:08.125202+0000","last_active":"2024-09-17T04:25:09.331817+0000","last_peered":"2024-09-17T04:25:09.331817+0000","last_clean":"2024-09-17T04:25:09.331817+0000","last_became_active":"2024-09-17T04:25:08.124880+0000","last_became_peered":"2024-09-17T04:25:08.124880+0000","last_unstale":"2024-09-17T04:25:09.331817+0000","last_undegraded":"2024-09-17T04:25:09.331817+0000","last_fullsized":"2024-09-17T04:25:09.331817+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:34.035658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.401577+0000","last_change":"2024-09-17T04:25:08.109076+0000","last_active":"2024-09-17T04:25:09.401577+0000","last_peered":"2024-09-17T04:25:09.401577+0000","last_clean":"2024-09-17T04:25:09.401577+0000","last_became_active":"2024-09-17T04:25:08.105699+0000","last_became_peered":"2024-09-17T04:25:08.105699+0000","last_unstale":"2024-09-17T04:25:09.401577+0000","last_undegraded":"2024-09-17T04:25:09.401577+0000","last_fullsized":"2024-09-17T04:25:09.401577+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:30.659131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.111966+0000","last_change":"2024-09-17T04:25:08.099256+0000","last_active":"2024-09-17T04:25:09.111966+0000","last_peered":"2024-09-17T04:25:09.111966+0000","last_clean":"2024-09-17T04:25:09.111966+0000","last_became_active":"2024-09-17T04:25:08.098874+0000","last_became_peered":"2024-09-17T04:25:08.098874+0000","last_unstale":"2024-09-17T04:25:09.111966+0000","last_undegraded":"2024-09-17T04:25:09.111966+0000","last_fullsized":"2024-09-17T04:25:09.111966+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:11.397751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.454771+0000","last_change":"2024-09-17T04:25:08.117901+0000","last_active":"2024-09-17T04:25:09.454771+0000","last_peered":"2024-09-17T04:25:09.454771+0000","last_clean":"2024-09-17T04:25:09.454771+0000","last_became_active":"2024-09-17T04:25:08.115468+0000","last_became_peered":"2024-09-17T04:25:08.115468+0000","last_unstale":"2024-09-17T04:25:09.454771+0000","last_undegraded":"2024-09-17T04:25:09.454771+0000","last_fullsized":"2024-09-17T04:25:09.454771+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:03.692785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.456199+0000","last_change":"2024-09-17T04:25:08.110694+0000","last_active":"2024-09-17T04:25:09.456199+0000","last_peered":"2024-09-17T04:25:09.456199+0000","last_clean":"2024-09-17T04:25:09.456199+0000","last_became_active":"2024-09-17T04:25:08.097992+0000","last_became_peered":"2024-09-17T04:25:08.097992+0000","last_unstale":"2024-09-17T04:25:09.456199+0000","last_undegraded":"2024-09-17T04:25:09.456199+0000","last_fullsized":"2024-09-17T04:25:09.456199+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:42.703067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.454792+0000","last_change":"2024-09-17T04:25:08.126392+0000","last_active":"2024-09-17T04:25:09.454792+0000","last_peered":"2024-09-17T04:25:09.454792+0000","last_clean":"2024-09-17T04:25:09.454792+0000","last_became_active":"2024-09-17T04:25:08.126046+0000","last_became_peered":"2024-09-17T04:25:08.126046+0000","last_unstale":"2024-09-17T04:25:09.454792+0000","last_undegraded":"2024-09-17T04:25:09.454792+0000","last_fullsized":"2024-09-17T04:25:09.454792+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:14.054043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113961+0000","last_change":"2024-09-17T04:25:08.104557+0000","last_active":"2024-09-17T04:25:09.113961+0000","last_peered":"2024-09-17T04:25:09.113961+0000","last_clean":"2024-09-17T04:25:09.113961+0000","last_became_active":"2024-09-17T04:25:08.104246+0000","last_became_peered":"2024-09-17T04:25:08.104246+0000","last_unstale":"2024-09-17T04:25:09.113961+0000","last_undegraded":"2024-09-17T04:25:09.113961+0000","last_fullsized":"2024-09-17T04:25:09.113961+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:26.289675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.331849+0000","last_change":"2024-09-17T04:25:08.124218+0000","last_active":"2024-09-17T04:25:09.331849+0000","last_peered":"2024-09-17T04:25:09.331849+0000","last_clean":"2024-09-17T04:25:09.331849+0000","last_became_active":"2024-09-17T04:25:08.123746+0000","last_became_peered":"2024-09-17T04:25:08.123746+0000","last_unstale":"2024-09-17T04:25:09.331849+0000","last_undegraded":"2024-09-17T04:25:09.331849+0000","last_fullsized":"2024-09-17T04:25:09.331849+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:55.783470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112282+0000","last_change":"2024-09-17T04:25:08.113465+0000","last_active":"2024-09-17T04:25:09.112282+0000","last_peered":"2024-09-17T04:25:09.112282+0000","last_clean":"2024-09-17T04:25:09.112282+0000","last_became_active":"2024-09-17T04:25:08.113106+0000","last_became_peered":"2024-09-17T04:25:08.113106+0000","last_unstale":"2024-09-17T04:25:09.112282+0000","last_undegraded":"2024-09-17T04:25:09.112282+0000","last_fullsized":"2024-09-17T04:25:09.112282+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:14.660711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455081+0000","last_change":"2024-09-17T04:25:08.111799+0000","last_active":"2024-09-17T04:25:09.455081+0000","last_peered":"2024-09-17T04:25:09.455081+0000","last_clean":"2024-09-17T04:25:09.455081+0000","last_became_active":"2024-09-17T04:25:08.111338+0000","last_became_peered":"2024-09-17T04:25:08.111338+0000","last_unstale":"2024-09-17T04:25:09.455081+0000","last_undegraded":"2024-09-17T04:25:09.455081+0000","last_fullsized":"2024-09-17T04:25:09.455081+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:06.724919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112824+0000","last_change":"2024-09-17T04:25:08.109129+0000","last_active":"2024-09-17T04:25:09.112824+0000","last_peered":"2024-09-17T04:25:09.112824+0000","last_clean":"2024-09-17T04:25:09.112824+0000","last_became_active":"2024-09-17T04:25:08.104873+0000","last_became_peered":"2024-09-17T04:25:08.104873+0000","last_unstale":"2024-09-17T04:25:09.112824+0000","last_undegraded":"2024-09-17T04:25:09.112824+0000","last_fullsized":"2024-09-17T04:25:09.112824+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:48.676959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.401857+0000","last_change":"2024-09-17T04:25:08.104171+0000","last_active":"2024-09-17T04:25:09.401857+0000","last_peered":"2024-09-17T04:25:09.401857+0000","last_clean":"2024-09-17T04:25:09.401857+0000","last_became_active":"2024-09-17T04:25:08.103831+0000","last_became_peered":"2024-09-17T04:25:08.103831+0000","last_unstale":"2024-09-17T04:25:09.401857+0000","last_undegraded":"2024-09-17T04:25:09.401857+0000","last_fullsized":"2024-09-17T04:25:09.401857+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:01:06.440885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455502+0000","last_change":"2024-09-17T04:25:07.101542+0000","last_active":"2024-09-17T04:25:09.455502+0000","last_peered":"2024-09-17T04:25:09.455502+0000","last_clean":"2024-09-17T04:25:09.455502+0000","last_became_active":"2024-09-17T04:25:07.101168+0000","last_became_peered":"2024-09-17T04:25:07.101168+0000","last_unstale":"2024-09-17T04:25:09.455502+0000","last_undegraded":"2024-09-17T04:25:09.455502+0000","last_fullsized":"2024-09-17T04:25:09.455502+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:05.174227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455449+0000","last_change":"2024-09-17T04:25:08.122307+0000","last_active":"2024-09-17T04:25:09.455449+0000","last_peered":"2024-09-17T04:25:09.455449+0000","last_clean":"2024-09-17T04:25:09.455449+0000","last_became_active":"2024-09-17T04:25:08.121816+0000","last_became_peered":"2024-09-17T04:25:08.121816+0000","last_unstale":"2024-09-17T04:25:09.455449+0000","last_undegraded":"2024-09-17T04:25:09.455449+0000","last_fullsized":"2024-09-17T04:25:09.455449+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:55.783470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.401903+0000","last_change":"2024-09-17T04:25:07.090513+0000","last_active":"2024-09-17T04:25:09.401903+0000","last_peered":"2024-09-17T04:25:09.401903+0000","last_clean":"2024-09-17T04:25:09.401903+0000","last_became_active":"2024-09-17T04:25:07.090181+0000","last_became_peered":"2024-09-17T04:25:07.090181+0000","last_unstale":"2024-09-17T04:25:09.401903+0000","last_undegraded":"2024-09-17T04:25:09.401903+0000","last_fullsized":"2024-09-17T04:25:09.401903+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:00.826540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.332083+0000","last_change":"2024-09-17T04:25:08.118894+0000","last_active":"2024-09-17T04:25:09.332083+0000","last_peered":"2024-09-17T04:25:09.332083+0000","last_clean":"2024-09-17T04:25:09.332083+0000","last_became_active":"2024-09-17T04:25:08.118563+0000","last_became_peered":"2024-09-17T04:25:08.118563+0000","last_unstale":"2024-09-17T04:25:09.332083+0000","last_undegraded":"2024-09-17T04:25:09.332083+0000","last_fullsized":"2024-09-17T04:25:09.332083+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:26:25.532533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455925+0000","last_change":"2024-09-17T04:25:07.103013+0000","last_active":"2024-09-17T04:25:09.455925+0000","last_peered":"2024-09-17T04:25:09.455925+0000","last_clean":"2024-09-17T04:25:09.455925+0000","last_became_active":"2024-09-17T04:25:07.100422+0000","last_became_peered":"2024-09-17T04:25:07.100422+0000","last_unstale":"2024-09-17T04:25:09.455925+0000","last_undegraded":"2024-09-17T04:25:09.455925+0000","last_fullsized":"2024-09-17T04:25:09.455925+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:10.495637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.454835+0000","last_change":"2024-09-17T04:25:08.125767+0000","last_active":"2024-09-17T04:25:09.454835+0000","last_peered":"2024-09-17T04:25:09.454835+0000","last_clean":"2024-09-17T04:25:09.454835+0000","last_became_active":"2024-09-17T04:25:08.125475+0000","last_became_peered":"2024-09-17T04:25:08.125475+0000","last_unstale":"2024-09-17T04:25:09.454835+0000","last_undegraded":"2024-09-17T04:25:09.454835+0000","last_fullsized":"2024-09-17T04:25:09.454835+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:38.129581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.105644+0000","last_change":"2024-09-17T04:25:07.097308+0000","last_active":"2024-09-17T04:25:09.105644+0000","last_peered":"2024-09-17T04:25:09.105644+0000","last_clean":"2024-09-17T04:25:09.105644+0000","last_became_active":"2024-09-17T04:25:07.096657+0000","last_became_peered":"2024-09-17T04:25:07.096657+0000","last_unstale":"2024-09-17T04:25:09.105644+0000","last_undegraded":"2024-09-17T04:25:09.105644+0000","last_fullsized":"2024-09-17T04:25:09.105644+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:49:55.787331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112543+0000","last_change":"2024-09-17T04:25:08.123602+0000","last_active":"2024-09-17T04:25:09.112543+0000","last_peered":"2024-09-17T04:25:09.112543+0000","last_clean":"2024-09-17T04:25:09.112543+0000","last_became_active":"2024-09-17T04:25:08.121886+0000","last_became_peered":"2024-09-17T04:25:08.121886+0000","last_unstale":"2024-09-17T04:25:09.112543+0000","last_undegraded":"2024-09-17T04:25:09.112543+0000","last_fullsized":"2024-09-17T04:25:09.112543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:59.322143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.332147+0000","last_change":"2024-09-17T04:25:07.093514+0000","last_active":"2024-09-17T04:25:09.332147+0000","last_peered":"2024-09-17T04:25:09.332147+0000","last_clean":"2024-09-17T04:25:09.332147+0000","last_became_active":"2024-09-17T04:25:07.091028+0000","last_became_peered":"2024-09-17T04:25:07.091028+0000","last_unstale":"2024-09-17T04:25:09.332147+0000","last_undegraded":"2024-09-17T04:25:09.332147+0000","last_fullsized":"2024-09-17T04:25:09.332147+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:47:16.719512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113223+0000","last_change":"2024-09-17T04:25:08.109606+0000","last_active":"2024-09-17T04:25:09.113223+0000","last_peered":"2024-09-17T04:25:09.113223+0000","last_clean":"2024-09-17T04:25:09.113223+0000","last_became_active":"2024-09-17T04:25:08.109288+0000","last_became_peered":"2024-09-17T04:25:08.109288+0000","last_unstale":"2024-09-17T04:25:09.113223+0000","last_undegraded":"2024-09-17T04:25:09.113223+0000","last_fullsized":"2024-09-17T04:25:09.113223+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:12:03.863249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.331898+0000","last_change":"2024-09-17T04:25:07.092548+0000","last_active":"2024-09-17T04:25:09.331898+0000","last_peered":"2024-09-17T04:25:09.331898+0000","last_clean":"2024-09-17T04:25:09.331898+0000","last_became_active":"2024-09-17T04:25:07.092222+0000","last_became_peered":"2024-09-17T04:25:07.092222+0000","last_unstale":"2024-09-17T04:25:09.331898+0000","last_undegraded":"2024-09-17T04:25:09.331898+0000","last_fullsized":"2024-09-17T04:25:09.331898+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:12.533380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.331977+0000","last_change":"2024-09-17T04:25:08.120205+0000","last_active":"2024-09-17T04:25:09.331977+0000","last_peered":"2024-09-17T04:25:09.331977+0000","last_clean":"2024-09-17T04:25:09.331977+0000","last_became_active":"2024-09-17T04:25:08.119921+0000","last_became_peered":"2024-09-17T04:25:08.119921+0000","last_unstale":"2024-09-17T04:25:09.331977+0000","last_undegraded":"2024-09-17T04:25:09.331977+0000","last_fullsized":"2024-09-17T04:25:09.331977+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:32.761316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455135+0000","last_change":"2024-09-17T04:25:07.096137+0000","last_active":"2024-09-17T04:25:09.455135+0000","last_peered":"2024-09-17T04:25:09.455135+0000","last_clean":"2024-09-17T04:25:09.455135+0000","last_became_active":"2024-09-17T04:25:07.095852+0000","last_became_peered":"2024-09-17T04:25:07.095852+0000","last_unstale":"2024-09-17T04:25:09.455135+0000","last_undegraded":"2024-09-17T04:25:09.455135+0000","last_fullsized":"2024-09-17T04:25:09.455135+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:35:37.851978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.454068+0000","last_change":"2024-09-17T04:25:08.107235+0000","last_active":"2024-09-17T04:25:09.454068+0000","last_peered":"2024-09-17T04:25:09.454068+0000","last_clean":"2024-09-17T04:25:09.454068+0000","last_became_active":"2024-09-17T04:25:08.106868+0000","last_became_peered":"2024-09-17T04:25:08.106868+0000","last_unstale":"2024-09-17T04:25:09.454068+0000","last_undegraded":"2024-09-17T04:25:09.454068+0000","last_fullsized":"2024-09-17T04:25:09.454068+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:26:25.532533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.331780+0000","last_change":"2024-09-17T04:25:07.090542+0000","last_active":"2024-09-17T04:25:09.331780+0000","last_peered":"2024-09-17T04:25:09.331780+0000","last_clean":"2024-09-17T04:25:09.331780+0000","last_became_active":"2024-09-17T04:25:07.087859+0000","last_became_peered":"2024-09-17T04:25:07.087859+0000","last_unstale":"2024-09-17T04:25:09.331780+0000","last_undegraded":"2024-09-17T04:25:09.331780+0000","last_fullsized":"2024-09-17T04:25:09.331780+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:35.343496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.114713+0000","last_change":"2024-09-17T04:25:08.120860+0000","last_active":"2024-09-17T04:25:09.114713+0000","last_peered":"2024-09-17T04:25:09.114713+0000","last_clean":"2024-09-17T04:25:09.114713+0000","last_became_active":"2024-09-17T04:25:08.119392+0000","last_became_peered":"2024-09-17T04:25:08.119392+0000","last_unstale":"2024-09-17T04:25:09.114713+0000","last_undegraded":"2024-09-17T04:25:09.114713+0000","last_fullsized":"2024-09-17T04:25:09.114713+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:52.755680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.456072+0000","last_change":"2024-09-17T04:25:07.103331+0000","last_active":"2024-09-17T04:25:09.456072+0000","last_peered":"2024-09-17T04:25:09.456072+0000","last_clean":"2024-09-17T04:25:09.456072+0000","last_became_active":"2024-09-17T04:25:07.102248+0000","last_became_peered":"2024-09-17T04:25:07.102248+0000","last_unstale":"2024-09-17T04:25:09.456072+0000","last_undegraded":"2024-09-17T04:25:09.456072+0000","last_fullsized":"2024-09-17T04:25:09.456072+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:09:07.109032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.456105+0000","last_change":"2024-09-17T04:25:08.119990+0000","last_active":"2024-09-17T04:25:09.456105+0000","last_peered":"2024-09-17T04:25:09.456105+0000","last_clean":"2024-09-17T04:25:09.456105+0000","last_became_active":"2024-09-17T04:25:08.119656+0000","last_became_peered":"2024-09-17T04:25:08.119656+0000","last_unstale":"2024-09-17T04:25:09.456105+0000","last_undegraded":"2024-09-17T04:25:09.456105+0000","last_fullsized":"2024-09-17T04:25:09.456105+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:15:59.588478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.402035+0000","last_change":"2024-09-17T04:25:07.097895+0000","last_active":"2024-09-17T04:25:09.402035+0000","last_peered":"2024-09-17T04:25:09.402035+0000","last_clean":"2024-09-17T04:25:09.402035+0000","last_became_active":"2024-09-17T04:25:07.097541+0000","last_became_peered":"2024-09-17T04:25:07.097541+0000","last_unstale":"2024-09-17T04:25:09.402035+0000","last_undegraded":"2024-09-17T04:25:09.402035+0000","last_fullsized":"2024-09-17T04:25:09.402035+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:49:00.272046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455727+0000","last_change":"2024-09-17T04:25:08.095787+0000","last_active":"2024-09-17T04:25:09.455727+0000","last_peered":"2024-09-17T04:25:09.455727+0000","last_clean":"2024-09-17T04:25:09.455727+0000","last_became_active":"2024-09-17T04:25:08.094877+0000","last_became_peered":"2024-09-17T04:25:08.094877+0000","last_unstale":"2024-09-17T04:25:09.455727+0000","last_undegraded":"2024-09-17T04:25:09.455727+0000","last_fullsized":"2024-09-17T04:25:09.455727+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:38.052155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.114464+0000","last_change":"2024-09-17T04:25:07.103494+0000","last_active":"2024-09-17T04:25:09.114464+0000","last_peered":"2024-09-17T04:25:09.114464+0000","last_clean":"2024-09-17T04:25:09.114464+0000","last_became_active":"2024-09-17T04:25:07.103202+0000","last_became_peered":"2024-09-17T04:25:07.103202+0000","last_unstale":"2024-09-17T04:25:09.114464+0000","last_undegraded":"2024-09-17T04:25:09.114464+0000","last_fullsized":"2024-09-17T04:25:09.114464+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:33.029608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.114318+0000","last_change":"2024-09-17T04:25:08.117481+0000","last_active":"2024-09-17T04:25:09.114318+0000","last_peered":"2024-09-17T04:25:09.114318+0000","last_clean":"2024-09-17T04:25:09.114318+0000","last_became_active":"2024-09-17T04:25:08.117068+0000","last_became_peered":"2024-09-17T04:25:08.117068+0000","last_unstale":"2024-09-17T04:25:09.114318+0000","last_undegraded":"2024-09-17T04:25:09.114318+0000","last_fullsized":"2024-09-17T04:25:09.114318+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:40.823059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":21,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455784+0000","last_change":"2024-09-17T04:25:07.099760+0000","last_active":"2024-09-17T04:25:09.455784+0000","last_peered":"2024-09-17T04:25:09.455784+0000","last_clean":"2024-09-17T04:25:09.455784+0000","last_became_active":"2024-09-17T04:25:07.098636+0000","last_became_peered":"2024-09-17T04:25:07.098636+0000","last_unstale":"2024-09-17T04:25:09.455784+0000","last_undegraded":"2024-09-17T04:25:09.455784+0000","last_fullsized":"2024-09-17T04:25:09.455784+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:58:57.715872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.454118+0000","last_change":"2024-09-17T04:25:08.105980+0000","last_active":"2024-09-17T04:25:09.454118+0000","last_peered":"2024-09-17T04:25:09.454118+0000","last_clean":"2024-09-17T04:25:09.454118+0000","last_became_active":"2024-09-17T04:25:08.105227+0000","last_became_peered":"2024-09-17T04:25:08.105227+0000","last_unstale":"2024-09-17T04:25:09.454118+0000","last_undegraded":"2024-09-17T04:25:09.454118+0000","last_fullsized":"2024-09-17T04:25:09.454118+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:57:19.324727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.332326+0000","last_change":"2024-09-17T04:25:07.090197+0000","last_active":"2024-09-17T04:25:09.332326+0000","last_peered":"2024-09-17T04:25:09.332326+0000","last_clean":"2024-09-17T04:25:09.332326+0000","last_became_active":"2024-09-17T04:25:07.089784+0000","last_became_peered":"2024-09-17T04:25:07.089784+0000","last_unstale":"2024-09-17T04:25:09.332326+0000","last_undegraded":"2024-09-17T04:25:09.332326+0000","last_fullsized":"2024-09-17T04:25:09.332326+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:09.843048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455543+0000","last_change":"2024-09-17T04:25:08.111349+0000","last_active":"2024-09-17T04:25:09.455543+0000","last_peered":"2024-09-17T04:25:09.455543+0000","last_clean":"2024-09-17T04:25:09.455543+0000","last_became_active":"2024-09-17T04:25:08.110979+0000","last_became_peered":"2024-09-17T04:25:08.110979+0000","last_unstale":"2024-09-17T04:25:09.455543+0000","last_undegraded":"2024-09-17T04:25:09.455543+0000","last_fullsized":"2024-09-17T04:25:09.455543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:19:23.569788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.454308+0000","last_change":"2024-09-17T04:25:07.100954+0000","last_active":"2024-09-17T04:25:09.454308+0000","last_peered":"2024-09-17T04:25:09.454308+0000","last_clean":"2024-09-17T04:25:09.454308+0000","last_became_active":"2024-09-17T04:25:07.100610+0000","last_became_peered":"2024-09-17T04:25:07.100610+0000","last_unstale":"2024-09-17T04:25:09.454308+0000","last_undegraded":"2024-09-17T04:25:09.454308+0000","last_fullsized":"2024-09-17T04:25:09.454308+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:14.912889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113053+0000","last_change":"2024-09-17T04:25:08.117150+0000","last_active":"2024-09-17T04:25:09.113053+0000","last_peered":"2024-09-17T04:25:09.113053+0000","last_clean":"2024-09-17T04:25:09.113053+0000","last_became_active":"2024-09-17T04:25:08.114530+0000","last_became_peered":"2024-09-17T04:25:08.114530+0000","last_unstale":"2024-09-17T04:25:09.113053+0000","last_undegraded":"2024-09-17T04:25:09.113053+0000","last_fullsized":"2024-09-17T04:25:09.113053+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:40.823059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455182+0000","last_change":"2024-09-17T04:25:07.101663+0000","last_active":"2024-09-17T04:25:09.455182+0000","last_peered":"2024-09-17T04:25:09.455182+0000","last_clean":"2024-09-17T04:25:09.455182+0000","last_became_active":"2024-09-17T04:25:07.100447+0000","last_became_peered":"2024-09-17T04:25:07.100447+0000","last_unstale":"2024-09-17T04:25:09.455182+0000","last_undegraded":"2024-09-17T04:25:09.455182+0000","last_fullsized":"2024-09-17T04:25:09.455182+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:43:37.274081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455215+0000","last_change":"2024-09-17T04:25:08.124593+0000","last_active":"2024-09-17T04:25:09.455215+0000","last_peered":"2024-09-17T04:25:09.455215+0000","last_clean":"2024-09-17T04:25:09.455215+0000","last_became_active":"2024-09-17T04:25:08.121257+0000","last_became_peered":"2024-09-17T04:25:08.121257+0000","last_unstale":"2024-09-17T04:25:09.455215+0000","last_undegraded":"2024-09-17T04:25:09.455215+0000","last_fullsized":"2024-09-17T04:25:09.455215+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:37.261699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":23,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455231+0000","last_change":"2024-09-17T04:25:07.101694+0000","last_active":"2024-09-17T04:25:09.455231+0000","last_peered":"2024-09-17T04:25:09.455231+0000","last_clean":"2024-09-17T04:25:09.455231+0000","last_became_active":"2024-09-17T04:25:07.101006+0000","last_became_peered":"2024-09-17T04:25:07.101006+0000","last_unstale":"2024-09-17T04:25:09.455231+0000","last_undegraded":"2024-09-17T04:25:09.455231+0000","last_fullsized":"2024-09-17T04:25:09.455231+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:56.490357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.332498+0000","last_change":"2024-09-17T04:25:08.120978+0000","last_active":"2024-09-17T04:25:09.332498+0000","last_peered":"2024-09-17T04:25:09.332498+0000","last_clean":"2024-09-17T04:25:09.332498+0000","last_became_active":"2024-09-17T04:25:08.118804+0000","last_became_peered":"2024-09-17T04:25:08.118804+0000","last_unstale":"2024-09-17T04:25:09.332498+0000","last_undegraded":"2024-09-17T04:25:09.332498+0000","last_fullsized":"2024-09-17T04:25:09.332498+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:57:19.324727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455040+0000","last_change":"2024-09-17T04:25:07.100101+0000","last_active":"2024-09-17T04:25:09.455040+0000","last_peered":"2024-09-17T04:25:09.455040+0000","last_clean":"2024-09-17T04:25:09.455040+0000","last_became_active":"2024-09-17T04:25:07.099657+0000","last_became_peered":"2024-09-17T04:25:07.099657+0000","last_unstale":"2024-09-17T04:25:09.455040+0000","last_undegraded":"2024-09-17T04:25:09.455040+0000","last_fullsized":"2024-09-17T04:25:09.455040+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:06:02.726891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113955+0000","last_change":"2024-09-17T04:25:08.123388+0000","last_active":"2024-09-17T04:25:09.113955+0000","last_peered":"2024-09-17T04:25:09.113955+0000","last_clean":"2024-09-17T04:25:09.113955+0000","last_became_active":"2024-09-17T04:25:08.122769+0000","last_became_peered":"2024-09-17T04:25:08.122769+0000","last_unstale":"2024-09-17T04:25:09.113955+0000","last_undegraded":"2024-09-17T04:25:09.113955+0000","last_fullsized":"2024-09-17T04:25:09.113955+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:52.735017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.332511+0000","last_change":"2024-09-17T04:25:07.089581+0000","last_active":"2024-09-17T04:25:09.332511+0000","last_peered":"2024-09-17T04:25:09.332511+0000","last_clean":"2024-09-17T04:25:09.332511+0000","last_became_active":"2024-09-17T04:25:07.089256+0000","last_became_peered":"2024-09-17T04:25:07.089256+0000","last_unstale":"2024-09-17T04:25:09.332511+0000","last_undegraded":"2024-09-17T04:25:09.332511+0000","last_fullsized":"2024-09-17T04:25:09.332511+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:11:57.535300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.454526+0000","last_change":"2024-09-17T04:25:08.127043+0000","last_active":"2024-09-17T04:25:09.454526+0000","last_peered":"2024-09-17T04:25:09.454526+0000","last_clean":"2024-09-17T04:25:09.454526+0000","last_became_active":"2024-09-17T04:25:08.126806+0000","last_became_peered":"2024-09-17T04:25:08.126806+0000","last_unstale":"2024-09-17T04:25:09.454526+0000","last_undegraded":"2024-09-17T04:25:09.454526+0000","last_fullsized":"2024-09-17T04:25:09.454526+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:09:29.147789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.332332+0000","last_change":"2024-09-17T04:25:07.098143+0000","last_active":"2024-09-17T04:25:09.332332+0000","last_peered":"2024-09-17T04:25:09.332332+0000","last_clean":"2024-09-17T04:25:09.332332+0000","last_became_active":"2024-09-17T04:25:07.091768+0000","last_became_peered":"2024-09-17T04:25:07.091768+0000","last_unstale":"2024-09-17T04:25:09.332332+0000","last_undegraded":"2024-09-17T04:25:09.332332+0000","last_fullsized":"2024-09-17T04:25:09.332332+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:15:18.524829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.105728+0000","last_change":"2024-09-17T04:25:08.108470+0000","last_active":"2024-09-17T04:25:09.105728+0000","last_peered":"2024-09-17T04:25:09.105728+0000","last_clean":"2024-09-17T04:25:09.105728+0000","last_became_active":"2024-09-17T04:25:08.108144+0000","last_became_peered":"2024-09-17T04:25:08.108144+0000","last_unstale":"2024-09-17T04:25:09.105728+0000","last_undegraded":"2024-09-17T04:25:09.105728+0000","last_fullsized":"2024-09-17T04:25:09.105728+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:36.885529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.332147+0000","last_change":"2024-09-17T04:25:07.103224+0000","last_active":"2024-09-17T04:25:09.332147+0000","last_peered":"2024-09-17T04:25:09.332147+0000","last_clean":"2024-09-17T04:25:09.332147+0000","last_became_active":"2024-09-17T04:25:07.101489+0000","last_became_peered":"2024-09-17T04:25:07.101489+0000","last_unstale":"2024-09-17T04:25:09.332147+0000","last_undegraded":"2024-09-17T04:25:09.332147+0000","last_fullsized":"2024-09-17T04:25:09.332147+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:08.430358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.105325+0000","last_change":"2024-09-17T04:25:08.112749+0000","last_active":"2024-09-17T04:25:09.105325+0000","last_peered":"2024-09-17T04:25:09.105325+0000","last_clean":"2024-09-17T04:25:09.105325+0000","last_became_active":"2024-09-17T04:25:08.112408+0000","last_became_peered":"2024-09-17T04:25:08.112408+0000","last_unstale":"2024-09-17T04:25:09.105325+0000","last_undegraded":"2024-09-17T04:25:09.105325+0000","last_fullsized":"2024-09-17T04:25:09.105325+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:09.436407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.401785+0000","last_change":"2024-09-17T04:25:07.092602+0000","last_active":"2024-09-17T04:25:09.401785+0000","last_peered":"2024-09-17T04:25:09.401785+0000","last_clean":"2024-09-17T04:25:09.401785+0000","last_became_active":"2024-09-17T04:25:07.092215+0000","last_became_peered":"2024-09-17T04:25:07.092215+0000","last_unstale":"2024-09-17T04:25:09.401785+0000","last_undegraded":"2024-09-17T04:25:09.401785+0000","last_fullsized":"2024-09-17T04:25:09.401785+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:19.304290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.105634+0000","last_change":"2024-09-17T04:25:08.107413+0000","last_active":"2024-09-17T04:25:09.105634+0000","last_peered":"2024-09-17T04:25:09.105634+0000","last_clean":"2024-09-17T04:25:09.105634+0000","last_became_active":"2024-09-17T04:25:08.106516+0000","last_became_peered":"2024-09-17T04:25:08.106516+0000","last_unstale":"2024-09-17T04:25:09.105634+0000","last_undegraded":"2024-09-17T04:25:09.105634+0000","last_fullsized":"2024-09-17T04:25:09.105634+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:10.849097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455110+0000","last_change":"2024-09-17T04:25:07.101091+0000","last_active":"2024-09-17T04:25:09.455110+0000","last_peered":"2024-09-17T04:25:09.455110+0000","last_clean":"2024-09-17T04:25:09.455110+0000","last_became_active":"2024-09-17T04:25:07.100821+0000","last_became_peered":"2024-09-17T04:25:07.100821+0000","last_unstale":"2024-09-17T04:25:09.455110+0000","last_undegraded":"2024-09-17T04:25:09.455110+0000","last_fullsized":"2024-09-17T04:25:09.455110+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:50:25.153746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.105405+0000","last_change":"2024-09-17T04:25:08.104836+0000","last_active":"2024-09-17T04:25:09.105405+0000","last_peered":"2024-09-17T04:25:09.105405+0000","last_clean":"2024-09-17T04:25:09.105405+0000","last_became_active":"2024-09-17T04:25:08.104120+0000","last_became_peered":"2024-09-17T04:25:08.104120+0000","last_unstale":"2024-09-17T04:25:09.105405+0000","last_undegraded":"2024-09-17T04:25:09.105405+0000","last_fullsized":"2024-09-17T04:25:09.105405+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:47:17.725561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.331721+0000","last_change":"2024-09-17T04:25:07.103534+0000","last_active":"2024-09-17T04:25:09.331721+0000","last_peered":"2024-09-17T04:25:09.331721+0000","last_clean":"2024-09-17T04:25:09.331721+0000","last_became_active":"2024-09-17T04:25:07.103070+0000","last_became_peered":"2024-09-17T04:25:07.103070+0000","last_unstale":"2024-09-17T04:25:09.331721+0000","last_undegraded":"2024-09-17T04:25:09.331721+0000","last_fullsized":"2024-09-17T04:25:09.331721+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:57:15.056986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113955+0000","last_change":"2024-09-17T04:25:08.113398+0000","last_active":"2024-09-17T04:25:09.113955+0000","last_peered":"2024-09-17T04:25:09.113955+0000","last_clean":"2024-09-17T04:25:09.113955+0000","last_became_active":"2024-09-17T04:25:08.113049+0000","last_became_peered":"2024-09-17T04:25:08.113049+0000","last_unstale":"2024-09-17T04:25:09.113955+0000","last_undegraded":"2024-09-17T04:25:09.113955+0000","last_fullsized":"2024-09-17T04:25:09.113955+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:12.516316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.111837+0000","last_change":"2024-09-17T04:25:07.103861+0000","last_active":"2024-09-17T04:25:09.111837+0000","last_peered":"2024-09-17T04:25:09.111837+0000","last_clean":"2024-09-17T04:25:09.111837+0000","last_became_active":"2024-09-17T04:25:07.100484+0000","last_became_peered":"2024-09-17T04:25:07.100484+0000","last_unstale":"2024-09-17T04:25:09.111837+0000","last_undegraded":"2024-09-17T04:25:09.111837+0000","last_fullsized":"2024-09-17T04:25:09.111837+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:27.284841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.105892+0000","last_change":"2024-09-17T04:25:08.100426+0000","last_active":"2024-09-17T04:25:09.105892+0000","last_peered":"2024-09-17T04:25:09.105892+0000","last_clean":"2024-09-17T04:25:09.105892+0000","last_became_active":"2024-09-17T04:25:08.100117+0000","last_became_peered":"2024-09-17T04:25:08.100117+0000","last_unstale":"2024-09-17T04:25:09.105892+0000","last_undegraded":"2024-09-17T04:25:09.105892+0000","last_fullsized":"2024-09-17T04:25:09.105892+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:49:01.278096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455112+0000","last_change":"2024-09-17T04:25:07.097884+0000","last_active":"2024-09-17T04:25:09.455112+0000","last_peered":"2024-09-17T04:25:09.455112+0000","last_clean":"2024-09-17T04:25:09.455112+0000","last_became_active":"2024-09-17T04:25:07.097545+0000","last_became_peered":"2024-09-17T04:25:07.097545+0000","last_unstale":"2024-09-17T04:25:09.455112+0000","last_undegraded":"2024-09-17T04:25:09.455112+0000","last_fullsized":"2024-09-17T04:25:09.455112+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:01.801695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.114256+0000","last_change":"2024-09-17T04:25:08.123061+0000","last_active":"2024-09-17T04:25:09.114256+0000","last_peered":"2024-09-17T04:25:09.114256+0000","last_clean":"2024-09-17T04:25:09.114256+0000","last_became_active":"2024-09-17T04:25:08.122319+0000","last_became_peered":"2024-09-17T04:25:08.122319+0000","last_unstale":"2024-09-17T04:25:09.114256+0000","last_undegraded":"2024-09-17T04:25:09.114256+0000","last_fullsized":"2024-09-17T04:25:09.114256+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:22.538564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"36'1","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.402008+0000","last_change":"2024-09-17T04:25:07.088222+0000","last_active":"2024-09-17T04:25:09.402008+0000","last_peered":"2024-09-17T04:25:09.402008+0000","last_clean":"2024-09-17T04:25:09.402008+0000","last_became_active":"2024-09-17T04:25:07.087921+0000","last_became_peered":"2024-09-17T04:25:07.087921+0000","last_unstale":"2024-09-17T04:25:09.402008+0000","last_undegraded":"2024-09-17T04:25:09.402008+0000","last_fullsized":"2024-09-17T04:25:09.402008+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:35.879479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.401578+0000","last_change":"2024-09-17T04:25:08.115284+0000","last_active":"2024-09-17T04:25:09.401578+0000","last_peered":"2024-09-17T04:25:09.401578+0000","last_clean":"2024-09-17T04:25:09.401578+0000","last_became_active":"2024-09-17T04:25:08.114998+0000","last_became_peered":"2024-09-17T04:25:08.114998+0000","last_unstale":"2024-09-17T04:25:09.401578+0000","last_undegraded":"2024-09-17T04:25:09.401578+0000","last_fullsized":"2024-09-17T04:25:09.401578+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:21.841901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.454596+0000","last_change":"2024-09-17T04:25:07.096188+0000","last_active":"2024-09-17T04:25:09.454596+0000","last_peered":"2024-09-17T04:25:09.454596+0000","last_clean":"2024-09-17T04:25:09.454596+0000","last_became_active":"2024-09-17T04:25:07.094697+0000","last_became_peered":"2024-09-17T04:25:07.094697+0000","last_unstale":"2024-09-17T04:25:09.454596+0000","last_undegraded":"2024-09-17T04:25:09.454596+0000","last_fullsized":"2024-09-17T04:25:09.454596+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:05.174227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455189+0000","last_change":"2024-09-17T04:25:08.115704+0000","last_active":"2024-09-17T04:25:09.455189+0000","last_peered":"2024-09-17T04:25:09.455189+0000","last_clean":"2024-09-17T04:25:09.455189+0000","last_became_active":"2024-09-17T04:25:08.115380+0000","last_became_peered":"2024-09-17T04:25:08.115380+0000","last_unstale":"2024-09-17T04:25:09.455189+0000","last_undegraded":"2024-09-17T04:25:09.455189+0000","last_fullsized":"2024-09-17T04:25:09.455189+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:40:24.798987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.401941+0000","last_change":"2024-09-17T04:25:07.095160+0000","last_active":"2024-09-17T04:25:09.401941+0000","last_peered":"2024-09-17T04:25:09.401941+0000","last_clean":"2024-09-17T04:25:09.401941+0000","last_became_active":"2024-09-17T04:25:07.094948+0000","last_became_peered":"2024-09-17T04:25:07.094948+0000","last_unstale":"2024-09-17T04:25:09.401941+0000","last_undegraded":"2024-09-17T04:25:09.401941+0000","last_fullsized":"2024-09-17T04:25:09.401941+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:40.737074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.401823+0000","last_change":"2024-09-17T04:25:08.120140+0000","last_active":"2024-09-17T04:25:09.401823+0000","last_peered":"2024-09-17T04:25:09.401823+0000","last_clean":"2024-09-17T04:25:09.401823+0000","last_became_active":"2024-09-17T04:25:08.118074+0000","last_became_peered":"2024-09-17T04:25:08.118074+0000","last_unstale":"2024-09-17T04:25:09.401823+0000","last_undegraded":"2024-09-17T04:25:09.401823+0000","last_fullsized":"2024-09-17T04:25:09.401823+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:03.317036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.454312+0000","last_change":"2024-09-17T04:25:07.094913+0000","last_active":"2024-09-17T04:25:09.454312+0000","last_peered":"2024-09-17T04:25:09.454312+0000","last_clean":"2024-09-17T04:25:09.454312+0000","last_became_active":"2024-09-17T04:25:07.094618+0000","last_became_peered":"2024-09-17T04:25:07.094618+0000","last_unstale":"2024-09-17T04:25:09.454312+0000","last_undegraded":"2024-09-17T04:25:09.454312+0000","last_fullsized":"2024-09-17T04:25:09.454312+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:20.612773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.454505+0000","last_change":"2024-09-17T04:25:08.118075+0000","last_active":"2024-09-17T04:25:09.454505+0000","last_peered":"2024-09-17T04:25:09.454505+0000","last_clean":"2024-09-17T04:25:09.454505+0000","last_became_active":"2024-09-17T04:25:08.117697+0000","last_became_peered":"2024-09-17T04:25:08.117697+0000","last_unstale":"2024-09-17T04:25:09.454505+0000","last_undegraded":"2024-09-17T04:25:09.454505+0000","last_fullsized":"2024-09-17T04:25:09.454505+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:24.944294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.331912+0000","last_change":"2024-09-17T04:25:07.093510+0000","last_active":"2024-09-17T04:25:09.331912+0000","last_peered":"2024-09-17T04:25:09.331912+0000","last_clean":"2024-09-17T04:25:09.331912+0000","last_became_active":"2024-09-17T04:25:07.093132+0000","last_became_peered":"2024-09-17T04:25:07.093132+0000","last_unstale":"2024-09-17T04:25:09.331912+0000","last_undegraded":"2024-09-17T04:25:09.331912+0000","last_fullsized":"2024-09-17T04:25:09.331912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:04:39.641973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.331670+0000","last_change":"2024-09-17T04:25:08.127561+0000","last_active":"2024-09-17T04:25:09.331670+0000","last_peered":"2024-09-17T04:25:09.331670+0000","last_clean":"2024-09-17T04:25:09.331670+0000","last_became_active":"2024-09-17T04:25:08.127108+0000","last_became_peered":"2024-09-17T04:25:08.127108+0000","last_unstale":"2024-09-17T04:25:09.331670+0000","last_undegraded":"2024-09-17T04:25:09.331670+0000","last_fullsized":"2024-09-17T04:25:09.331670+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:28.290891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455032+0000","last_change":"2024-09-17T04:25:07.101483+0000","last_active":"2024-09-17T04:25:09.455032+0000","last_peered":"2024-09-17T04:25:09.455032+0000","last_clean":"2024-09-17T04:25:09.455032+0000","last_became_active":"2024-09-17T04:25:07.100021+0000","last_became_peered":"2024-09-17T04:25:07.100021+0000","last_unstale":"2024-09-17T04:25:09.455032+0000","last_undegraded":"2024-09-17T04:25:09.455032+0000","last_fullsized":"2024-09-17T04:25:09.455032+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:12:47.750660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.112185+0000","last_change":"2024-09-17T04:25:08.102453+0000","last_active":"2024-09-17T04:25:09.112185+0000","last_peered":"2024-09-17T04:25:09.112185+0000","last_clean":"2024-09-17T04:25:09.112185+0000","last_became_active":"2024-09-17T04:25:08.101181+0000","last_became_peered":"2024-09-17T04:25:08.101181+0000","last_unstale":"2024-09-17T04:25:09.112185+0000","last_undegraded":"2024-09-17T04:25:09.112185+0000","last_fullsized":"2024-09-17T04:25:09.112185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:07.068146+0000","last_clean_scrub_stamp":"2024-09-17T04:25:07.068146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:11.397751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.455084+0000","last_change":"2024-09-17T04:25:07.105074+0000","last_active":"2024-09-17T04:25:09.455084+0000","last_peered":"2024-09-17T04:25:09.455084+0000","last_clean":"2024-09-17T04:25:09.455084+0000","last_became_active":"2024-09-17T04:25:07.104735+0000","last_became_peered":"2024-09-17T04:25:07.104735+0000","last_unstale":"2024-09-17T04:25:09.455084+0000","last_undegraded":"2024-09-17T04:25:09.455084+0000","last_fullsized":"2024-09-17T04:25:09.455084+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:06.062097+0000","last_clean_scrub_stamp":"2024-09-17T04:25:06.062097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:06:46.154246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":136,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-17T04:25:09.113124+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:25:09.113124+0000","last_peered":"2024-09-17T04:25:09.113124+0000","last_clean":"2024-09-17T04:25:09.113124+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:25:09.113124+0000","last_undegraded":"2024-09-17T04:25:09.113124+0000","last_fullsized":"2024-09-17T04:25:09.113124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476755,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":995328,"data_stored":745967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476754,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":405504,"data_stored":156125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476755,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":405504,"data_stored":156181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476755,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":405504,"data_stored":155613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509458,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":401408,"data_stored":155579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509457,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":413696,"data_stored":157143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":991232,"data_stored":745877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542162,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":413696,"data_stored":157143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:25:17.979 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T04:25:17.979 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T04:25:18.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.246+0000 7f4b1e400640 1 -- 172.21.15.100:0/1262214359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b18073a80 msgr2=0x7f4b180cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:18.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.246+0000 7f4b1e400640 1 --2- 172.21.15.100:0/1262214359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b18073a80 0x7f4b180cd610 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4b18160280 tx=0x7f4b1002ab40 comp rx=0 tx=0).stop 2024-09-17T04:25:18.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.246+0000 7f4b1e400640 1 -- 172.21.15.100:0/1262214359 shutdown_connections 2024-09-17T04:25:18.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.246+0000 7f4b1e400640 1 --2- 172.21.15.100:0/1262214359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b18073a80 0x7f4b180cd610 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:18.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.246+0000 7f4b1e400640 1 --2- 172.21.15.100:0/1262214359 >> v2:172.21.15.184:3300/0 conn(0x7f4b18145a40 0x7f4b18073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:18.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.246+0000 7f4b1e400640 1 -- 172.21.15.100:0/1262214359 >> 172.21.15.100:0/1262214359 conn(0x7f4b1806d1c0 msgr2=0x7f4b1806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:18.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.246+0000 7f4b1e400640 1 -- 172.21.15.100:0/1262214359 shutdown_connections 2024-09-17T04:25:18.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.246+0000 7f4b1e400640 1 -- 172.21.15.100:0/1262214359 wait complete. 2024-09-17T04:25:18.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.248+0000 7f4b1e400640 1 Processor -- start 2024-09-17T04:25:18.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.249+0000 7f4b1e400640 1 -- start start 2024-09-17T04:25:18.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.250+0000 7f4b1e400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b18073a80 0x7f4b1815be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:18.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.250+0000 7f4b1e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b18145550 0x7f4b180b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:18.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.250+0000 7f4b1e400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b18145a40 0x7f4b1813ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:18.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.250+0000 7f4b1e400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4b180ce090 con 0x7f4b18145550 2024-09-17T04:25:18.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.250+0000 7f4b1e400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4b180cdf10 con 0x7f4b18073a80 2024-09-17T04:25:18.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.250+0000 7f4b1e400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4b180ce210 con 0x7f4b18145a40 2024-09-17T04:25:18.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.250+0000 7f4b0fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b18145550 0x7f4b180b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:18.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.250+0000 7f4b0fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b18145550 0x7f4b180b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57150/0 (socket says 172.21.15.100:57150) 2024-09-17T04:25:18.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.250+0000 7f4b0fe00640 1 -- 172.21.15.100:0/1311993196 learned_addr learned my addr 172.21.15.100:0/1311993196 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:18.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.250+0000 7f4b17400640 1 --2- 172.21.15.100:0/1311993196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b18073a80 0x7f4b1815be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:18.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.250+0000 7f4b17e00640 1 --2- 172.21.15.100:0/1311993196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b18145a40 0x7f4b1813ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:18.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.251+0000 7f4b0fe00640 1 -- 172.21.15.100:0/1311993196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b18145a40 msgr2=0x7f4b1813ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:18.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.251+0000 7f4b0fe00640 1 --2- 172.21.15.100:0/1311993196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b18145a40 0x7f4b1813ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:18.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.251+0000 7f4b0fe00640 1 -- 172.21.15.100:0/1311993196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b18073a80 msgr2=0x7f4b1815be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:18.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.251+0000 7f4b0fe00640 1 --2- 172.21.15.100:0/1311993196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b18073a80 0x7f4b1815be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:18.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.251+0000 7f4b0fe00640 1 -- 172.21.15.100:0/1311993196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1815c340 con 0x7f4b18145550 2024-09-17T04:25:18.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.251+0000 7f4b0fe00640 1 --2- 172.21.15.100:0/1311993196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b18145550 0x7f4b180b4bb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000d700 tx=0x7f4b0000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:18.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.251+0000 7f4b0f400640 1 -- 172.21.15.100:0/1311993196 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00017070 con 0x7f4b18145550 2024-09-17T04:25:18.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.251+0000 7f4b0f400640 1 -- 172.21.15.100:0/1311993196 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b00004310 con 0x7f4b18145550 2024-09-17T04:25:18.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.251+0000 7f4b0f400640 1 -- 172.21.15.100:0/1311993196 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00009a80 con 0x7f4b18145550 2024-09-17T04:25:18.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.251+0000 7f4b1e400640 1 -- 172.21.15.100:0/1311993196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b181f5f60 con 0x7f4b18145550 2024-09-17T04:25:18.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.252+0000 7f4b1e400640 1 -- 172.21.15.100:0/1311993196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b181f6520 con 0x7f4b18145550 2024-09-17T04:25:18.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.253+0000 7f4b1e400640 1 -- 172.21.15.100:0/1311993196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b180ce730 con 0x7f4b18145550 2024-09-17T04:25:18.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.254+0000 7f4b0f400640 1 -- 172.21.15.100:0/1311993196 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4b00009c20 con 0x7f4b18145550 2024-09-17T04:25:18.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.255+0000 7f4b0f400640 1 --2- 172.21.15.100:0/1311993196 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4af4078b90 0x7f4af407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:18.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.255+0000 7f4b0f400640 1 -- 172.21.15.100:0/1311993196 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f4b0009ada0 con 0x7f4b18145550 2024-09-17T04:25:18.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.255+0000 7f4b17400640 1 --2- 172.21.15.100:0/1311993196 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4af4078b90 0x7f4af407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:18.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.256+0000 7f4b17400640 1 --2- 172.21.15.100:0/1311993196 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4af4078b90 0x7f4af407b0a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4b08001720 tx=0x7f4b08009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:18.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.259+0000 7f4b0f400640 1 -- 172.21.15.100:0/1311993196 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4b00062af0 con 0x7f4b18145550 2024-09-17T04:25:18.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.509+0000 7f4b1e400640 1 -- 172.21.15.100:0/1311993196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f4b1815c550 con 0x7f4b18145550 2024-09-17T04:25:18.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.514+0000 7f4b0f400640 1 -- 172.21.15.100:0/1311993196 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f4b00067520 con 0x7f4b18145550 2024-09-17T04:25:18.514 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-09-17T04:25:18.514 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAsBOlmfYg9IBAABP1uRKGOugrA47VxwRLkjA== 2024-09-17T04:25:18.514 INFO:teuthology.orchestra.run.smithi100.stdout: caps mds = "allow" 2024-09-17T04:25:18.514 INFO:teuthology.orchestra.run.smithi100.stdout: caps mon = "allow r" 2024-09-17T04:25:18.514 INFO:teuthology.orchestra.run.smithi100.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T04:25:18.515 INFO:teuthology.orchestra.run.smithi100.stderr:updated caps for client.0 2024-09-17T04:25:18.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.516+0000 7f4b1e400640 1 -- 172.21.15.100:0/1311993196 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4af4078b90 msgr2=0x7f4af407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:18.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.516+0000 7f4b1e400640 1 --2- 172.21.15.100:0/1311993196 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4af4078b90 0x7f4af407b0a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4b08001720 tx=0x7f4b08009040 comp rx=0 tx=0).stop 2024-09-17T04:25:18.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.516+0000 7f4b1e400640 1 -- 172.21.15.100:0/1311993196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b18145550 msgr2=0x7f4b180b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:18.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.516+0000 7f4b1e400640 1 --2- 172.21.15.100:0/1311993196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b18145550 0x7f4b180b4bb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000d700 tx=0x7f4b0000dbc0 comp rx=0 tx=0).stop 2024-09-17T04:25:18.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.516+0000 7f4b1e400640 1 -- 172.21.15.100:0/1311993196 shutdown_connections 2024-09-17T04:25:18.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.516+0000 7f4b1e400640 1 --2- 172.21.15.100:0/1311993196 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4af4078b90 0x7f4af407b0a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:18.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.516+0000 7f4b1e400640 1 --2- 172.21.15.100:0/1311993196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b18145a40 0x7f4b1813ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:18.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.516+0000 7f4b1e400640 1 --2- 172.21.15.100:0/1311993196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b18145550 0x7f4b180b4bb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:18.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.516+0000 7f4b1e400640 1 --2- 172.21.15.100:0/1311993196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b18073a80 0x7f4b1815be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:18.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.516+0000 7f4b1e400640 1 -- 172.21.15.100:0/1311993196 >> 172.21.15.100:0/1311993196 conn(0x7f4b1806d1c0 msgr2=0x7f4b18068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:18.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.516+0000 7f4b1e400640 1 -- 172.21.15.100:0/1311993196 shutdown_connections 2024-09-17T04:25:18.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.516+0000 7f4b1e400640 1 -- 172.21.15.100:0/1311993196 wait complete. 2024-09-17T04:25:18.539 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:25:18.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.758+0000 7f9d21e00640 1 -- 172.21.15.100:0/3299271987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d1c15d760 msgr2=0x7f9d1c1535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:18.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.758+0000 7f9d21e00640 1 --2- 172.21.15.100:0/3299271987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d1c15d760 0x7f9d1c1535c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c072d60 tx=0x7f9d0c02a560 comp rx=0 tx=0).stop 2024-09-17T04:25:18.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.758+0000 7f9d21e00640 1 -- 172.21.15.100:0/3299271987 shutdown_connections 2024-09-17T04:25:18.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.758+0000 7f9d21e00640 1 --2- 172.21.15.100:0/3299271987 >> v2:172.21.15.184:3300/0 conn(0x7f9d1c0ad0e0 0x7f9d1c0ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:18.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.758+0000 7f9d21e00640 1 --2- 172.21.15.100:0/3299271987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d1c15d760 0x7f9d1c1535c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:18.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.758+0000 7f9d21e00640 1 -- 172.21.15.100:0/3299271987 >> 172.21.15.100:0/3299271987 conn(0x7f9d1c06d730 msgr2=0x7f9d1c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:18.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.758+0000 7f9d21e00640 1 -- 172.21.15.100:0/3299271987 shutdown_connections 2024-09-17T04:25:18.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.758+0000 7f9d21e00640 1 -- 172.21.15.100:0/3299271987 wait complete. 2024-09-17T04:25:18.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.761+0000 7f9d21e00640 1 Processor -- start 2024-09-17T04:25:18.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.761+0000 7f9d21e00640 1 -- start start 2024-09-17T04:25:18.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.762+0000 7f9d21e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d1c0ad0e0 0x7f9d1c147320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:18.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.762+0000 7f9d21e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d1c15d760 0x7f9d1c155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:18.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.762+0000 7f9d21e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d1c160310 0x7f9d1c1556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:18.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.762+0000 7f9d21e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c0b4c20 con 0x7f9d1c0ad0e0 2024-09-17T04:25:18.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.762+0000 7f9d21e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c153c70 con 0x7f9d1c15d760 2024-09-17T04:25:18.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.762+0000 7f9d21e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9d1c0b4da0 con 0x7f9d1c160310 2024-09-17T04:25:18.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.762+0000 7f9d1aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d1c0ad0e0 0x7f9d1c147320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:18.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.762+0000 7f9d1aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d1c0ad0e0 0x7f9d1c147320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57162/0 (socket says 172.21.15.100:57162) 2024-09-17T04:25:18.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.762+0000 7f9d1aa00640 1 -- 172.21.15.100:0/1029431756 learned_addr learned my addr 172.21.15.100:0/1029431756 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:18.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.763+0000 7f9d13e00640 1 --2- 172.21.15.100:0/1029431756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d1c15d760 0x7f9d1c155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:18.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.763+0000 7f9d1b400640 1 --2- 172.21.15.100:0/1029431756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d1c160310 0x7f9d1c1556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:18.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.763+0000 7f9d1aa00640 1 -- 172.21.15.100:0/1029431756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d1c160310 msgr2=0x7f9d1c1556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:18.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.763+0000 7f9d1aa00640 1 --2- 172.21.15.100:0/1029431756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d1c160310 0x7f9d1c1556b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:18.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.763+0000 7f9d1aa00640 1 -- 172.21.15.100:0/1029431756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d1c15d760 msgr2=0x7f9d1c155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:18.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.763+0000 7f9d1aa00640 1 --2- 172.21.15.100:0/1029431756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d1c15d760 0x7f9d1c155170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:18.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.763+0000 7f9d1aa00640 1 -- 172.21.15.100:0/1029431756 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1c150310 con 0x7f9d1c0ad0e0 2024-09-17T04:25:18.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.763+0000 7f9d1b400640 1 --2- 172.21.15.100:0/1029431756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d1c160310 0x7f9d1c1556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:25:18.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.763+0000 7f9d1aa00640 1 --2- 172.21.15.100:0/1029431756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d1c0ad0e0 0x7f9d1c147320 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c034040 tx=0x7f9d0c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:18.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.763+0000 7f9d13400640 1 -- 172.21.15.100:0/1029431756 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c03e030 con 0x7f9d1c0ad0e0 2024-09-17T04:25:18.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.763+0000 7f9d13400640 1 -- 172.21.15.100:0/1029431756 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d0c003e00 con 0x7f9d1c0ad0e0 2024-09-17T04:25:18.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.763+0000 7f9d21e00640 1 -- 172.21.15.100:0/1029431756 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d1c147860 con 0x7f9d1c0ad0e0 2024-09-17T04:25:18.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.763+0000 7f9d13400640 1 -- 172.21.15.100:0/1029431756 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c035040 con 0x7f9d1c0ad0e0 2024-09-17T04:25:18.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.764+0000 7f9d21e00640 1 -- 172.21.15.100:0/1029431756 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d1c15f960 con 0x7f9d1c0ad0e0 2024-09-17T04:25:18.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.765+0000 7f9d21e00640 1 -- 172.21.15.100:0/1029431756 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d1c06a7e0 con 0x7f9d1c0ad0e0 2024-09-17T04:25:18.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.766+0000 7f9d13400640 1 -- 172.21.15.100:0/1029431756 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9d0c041030 con 0x7f9d1c0ad0e0 2024-09-17T04:25:18.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.767+0000 7f9d13400640 1 --2- 172.21.15.100:0/1029431756 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9cf8078bc0 0x7f9cf807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:18.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.767+0000 7f9d13400640 1 -- 172.21.15.100:0/1029431756 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f9d0c0bb610 con 0x7f9d1c0ad0e0 2024-09-17T04:25:18.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.767+0000 7f9d13e00640 1 --2- 172.21.15.100:0/1029431756 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9cf8078bc0 0x7f9cf807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:18.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.768+0000 7f9d13e00640 1 --2- 172.21.15.100:0/1029431756 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9cf8078bc0 0x7f9cf807b0d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9d04005fd0 tx=0x7f9d04005850 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:18.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.771+0000 7f9d13400640 1 -- 172.21.15.100:0/1029431756 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9d1c06a7e0 con 0x7f9d1c0ad0e0 2024-09-17T04:25:18.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.993+0000 7f9d21e00640 1 -- 172.21.15.100:0/1029431756 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d1c15dfa0 con 0x7f9d1c0ad0e0 2024-09-17T04:25:18.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.994+0000 7f9d13400640 1 -- 172.21.15.100:0/1029431756 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5232 (secure 0 0 0) 0x7f9d1c15dfa0 con 0x7f9d1c0ad0e0 2024-09-17T04:25:18.995 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:18.995 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":17,"btime":"2024-09-17T04:25:11:112457+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4667,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2259943840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":2259943840},{"type":"v1","addr":"172.21.15.100:6835","nonce":2259943840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4670,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/1748167554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1748167554},{"type":"v1","addr":"172.21.15.100:6834","nonce":1748167554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4854,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3159351871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":3159351871},{"type":"v1","addr":"172.21.15.184:6836","nonce":3159351871}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:07.565710+0000","modified":"2024-09-17T04:25:10.595342+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4857},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4857":{"gid":4857,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/294839954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":294839954},{"type":"v1","addr":"172.21.15.184:6837","nonce":294839954}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4857,"qdb_cluster":[4857]},"id":2}]} 2024-09-17T04:25:18.995 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 17 2024-09-17T04:25:18.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.997+0000 7f9d21e00640 1 -- 172.21.15.100:0/1029431756 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9cf8078bc0 msgr2=0x7f9cf807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:18.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.997+0000 7f9d21e00640 1 --2- 172.21.15.100:0/1029431756 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9cf8078bc0 0x7f9cf807b0d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9d04005fd0 tx=0x7f9d04005850 comp rx=0 tx=0).stop 2024-09-17T04:25:18.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.997+0000 7f9d21e00640 1 -- 172.21.15.100:0/1029431756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d1c0ad0e0 msgr2=0x7f9d1c147320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:18.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.997+0000 7f9d21e00640 1 --2- 172.21.15.100:0/1029431756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d1c0ad0e0 0x7f9d1c147320 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c034040 tx=0x7f9d0c003910 comp rx=0 tx=0).stop 2024-09-17T04:25:18.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.997+0000 7f9d21e00640 1 -- 172.21.15.100:0/1029431756 shutdown_connections 2024-09-17T04:25:18.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.997+0000 7f9d21e00640 1 --2- 172.21.15.100:0/1029431756 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9cf8078bc0 0x7f9cf807b0d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:18.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.997+0000 7f9d21e00640 1 --2- 172.21.15.100:0/1029431756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d1c160310 0x7f9d1c1556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:18.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.997+0000 7f9d21e00640 1 --2- 172.21.15.100:0/1029431756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d1c15d760 0x7f9d1c155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:18.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.997+0000 7f9d21e00640 1 --2- 172.21.15.100:0/1029431756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d1c0ad0e0 0x7f9d1c147320 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:18.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.997+0000 7f9d21e00640 1 -- 172.21.15.100:0/1029431756 >> 172.21.15.100:0/1029431756 conn(0x7f9d1c06d730 msgr2=0x7f9d1c0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:18.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.997+0000 7f9d21e00640 1 -- 172.21.15.100:0/1029431756 shutdown_connections 2024-09-17T04:25:18.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:18.997+0000 7f9d21e00640 1 -- 172.21.15.100:0/1029431756 wait complete. 2024-09-17T04:25:19.427 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T04:25:07.565710+0000', 'modified': '2024-09-17T04:25:10.595342+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4857}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4857': {'gid': 4857, 'name': 'b', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.184:6837/294839954', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6834', 'nonce': 294839954}, {'type': 'v1', 'addr': '172.21.15.184:6837', 'nonce': 294839954}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4857, 'qdb_cluster': [4857]} 2024-09-17T04:25:19.427 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T04:25:19.428 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-17T04:25:19.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.650+0000 7fd25ca00640 1 -- 172.21.15.100:0/2599824001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd258161fa0 msgr2=0x7fd2580b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:19.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.650+0000 7fd25ca00640 1 --2- 172.21.15.100:0/2599824001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd258161fa0 0x7fd2580b39c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd2580b7050 tx=0x7fd24002a4d0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.651+0000 7fd25ca00640 1 -- 172.21.15.100:0/2599824001 shutdown_connections 2024-09-17T04:25:19.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.651+0000 7fd25ca00640 1 --2- 172.21.15.100:0/2599824001 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2580d0cc0 0x7fd2580d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.651+0000 7fd25ca00640 1 --2- 172.21.15.100:0/2599824001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd258161fa0 0x7fd2580b39c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.651+0000 7fd25ca00640 1 --2- 172.21.15.100:0/2599824001 >> v2:172.21.15.184:3300/0 conn(0x7fd25806f710 0x7fd2580b94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.651+0000 7fd25ca00640 1 -- 172.21.15.100:0/2599824001 >> 172.21.15.100:0/2599824001 conn(0x7fd25808dc30 msgr2=0x7fd25809d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:19.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.651+0000 7fd25ca00640 1 -- 172.21.15.100:0/2599824001 shutdown_connections 2024-09-17T04:25:19.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.651+0000 7fd25ca00640 1 -- 172.21.15.100:0/2599824001 wait complete. 2024-09-17T04:25:19.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.653+0000 7fd25ca00640 1 Processor -- start 2024-09-17T04:25:19.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.653+0000 7fd25ca00640 1 -- start start 2024-09-17T04:25:19.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.654+0000 7fd25ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd25806f710 0x7fd2581f97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:19.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.654+0000 7fd25ca00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2580d0cc0 0x7fd2580cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:19.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.655+0000 7fd25ca00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd258161fa0 0x7fd2580cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:19.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.655+0000 7fd25ca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd2580b65d0 con 0x7fd25806f710 2024-09-17T04:25:19.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.655+0000 7fd25ca00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd2580b6450 con 0x7fd258161fa0 2024-09-17T04:25:19.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.655+0000 7fd25ca00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd2580b6750 con 0x7fd2580d0cc0 2024-09-17T04:25:19.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.655+0000 7fd257400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd25806f710 0x7fd2581f97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:19.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.655+0000 7fd257400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd25806f710 0x7fd2581f97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57168/0 (socket says 172.21.15.100:57168) 2024-09-17T04:25:19.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.655+0000 7fd257400640 1 -- 172.21.15.100:0/794423280 learned_addr learned my addr 172.21.15.100:0/794423280 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:19.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.655+0000 7fd24ea00640 1 --2- 172.21.15.100:0/794423280 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2580d0cc0 0x7fd2580cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:19.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.655+0000 7fd257e00640 1 --2- 172.21.15.100:0/794423280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd258161fa0 0x7fd2580cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:19.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.655+0000 7fd257400640 1 -- 172.21.15.100:0/794423280 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2580d0cc0 msgr2=0x7fd2580cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:19.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.655+0000 7fd257400640 1 --2- 172.21.15.100:0/794423280 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2580d0cc0 0x7fd2580cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.655+0000 7fd257400640 1 -- 172.21.15.100:0/794423280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd258161fa0 msgr2=0x7fd2580cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:19.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.655+0000 7fd257400640 1 --2- 172.21.15.100:0/794423280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd258161fa0 0x7fd2580cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.655+0000 7fd257400640 1 -- 172.21.15.100:0/794423280 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2580c9110 con 0x7fd25806f710 2024-09-17T04:25:19.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.655+0000 7fd24ea00640 1 --2- 172.21.15.100:0/794423280 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2580d0cc0 0x7fd2580cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:19.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.656+0000 7fd257400640 1 --2- 172.21.15.100:0/794423280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd25806f710 0x7fd2581f97e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd24800c940 tx=0x7fd24800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:19.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.656+0000 7fd24fe00640 1 -- 172.21.15.100:0/794423280 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd248007bb0 con 0x7fd25806f710 2024-09-17T04:25:19.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.656+0000 7fd24fe00640 1 -- 172.21.15.100:0/794423280 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd248007e40 con 0x7fd25806f710 2024-09-17T04:25:19.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.656+0000 7fd24fe00640 1 -- 172.21.15.100:0/794423280 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd248005290 con 0x7fd25806f710 2024-09-17T04:25:19.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.656+0000 7fd25ca00640 1 -- 172.21.15.100:0/794423280 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2580ca950 con 0x7fd25806f710 2024-09-17T04:25:19.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.656+0000 7fd25ca00640 1 -- 172.21.15.100:0/794423280 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2580ce610 con 0x7fd25806f710 2024-09-17T04:25:19.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.659+0000 7fd24fe00640 1 -- 172.21.15.100:0/794423280 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd248005430 con 0x7fd25806f710 2024-09-17T04:25:19.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.660+0000 7fd24fe00640 1 --2- 172.21.15.100:0/794423280 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd234078b90 0x7fd23407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:19.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.660+0000 7fd24fe00640 1 -- 172.21.15.100:0/794423280 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fd24809b1d0 con 0x7fd25806f710 2024-09-17T04:25:19.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.660+0000 7fd265250740 1 Processor -- start 2024-09-17T04:25:19.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.661+0000 7fd265250740 1 -- start start 2024-09-17T04:25:19.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.661+0000 7fd24ea00640 1 --2- 172.21.15.100:0/794423280 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd234078b90 0x7fd23407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:19.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.662+0000 7fd24ea00640 1 --2- 172.21.15.100:0/794423280 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd234078b90 0x7fd23407b0a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd240033000 tx=0x7fd240004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:19.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.662+0000 7fd265250740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562eb988a270 0x562eb9c5ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:19.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.662+0000 7fd265250740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562eb98613b0 0x562eb9c5b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:19.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.662+0000 7fd265250740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eb9869590 0x562eb9c2b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:19.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.662+0000 7fd265250740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562eb9bb4430 con 0x562eb98613b0 2024-09-17T04:25:19.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.662+0000 7fd265250740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562eb9ba45d0 con 0x562eb9869590 2024-09-17T04:25:19.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.662+0000 7fd265250740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x562eb9b93a00 con 0x562eb988a270 2024-09-17T04:25:19.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.662+0000 7fd24ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562eb98613b0 0x562eb9c5b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:19.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.662+0000 7fd24ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562eb98613b0 0x562eb9c5b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57178/0 (socket says 172.21.15.100:57178) 2024-09-17T04:25:19.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.662+0000 7fd24ea00640 1 -- 172.21.15.100:0/2366151689 learned_addr learned my addr 172.21.15.100:0/2366151689 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:19.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.662+0000 7fd257e00640 1 --2- 172.21.15.100:0/2366151689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eb9869590 0x562eb9c2b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:19.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.662+0000 7fd257400640 1 --2- 172.21.15.100:0/2366151689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562eb988a270 0x562eb9c5ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:19.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.663+0000 7fd24ea00640 1 -- 172.21.15.100:0/2366151689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562eb988a270 msgr2=0x562eb9c5ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:25:19.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.663+0000 7fd24ea00640 1 --2- 172.21.15.100:0/2366151689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562eb988a270 0x562eb9c5ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.663+0000 7fd24ea00640 1 -- 172.21.15.100:0/2366151689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eb9869590 msgr2=0x562eb9c2b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:25:19.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.663+0000 7fd24ea00640 1 --2- 172.21.15.100:0/2366151689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eb9869590 0x562eb9c2b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.663+0000 7fd24ea00640 1 -- 172.21.15.100:0/2366151689 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eb9b5c520 con 0x562eb98613b0 2024-09-17T04:25:19.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.663+0000 7fd257400640 1 --2- 172.21.15.100:0/2366151689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562eb988a270 0x562eb9c5ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:19.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.663+0000 7fd24ea00640 1 --2- 172.21.15.100:0/2366151689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562eb98613b0 0x562eb9c5b0a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd240034040 tx=0x7fd24003c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22743f217213de2 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:19.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.663+0000 7fd247400640 1 -- 172.21.15.100:0/2366151689 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240035040 con 0x562eb98613b0 2024-09-17T04:25:19.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.663+0000 7fd247400640 1 -- 172.21.15.100:0/2366151689 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd24004d070 con 0x562eb98613b0 2024-09-17T04:25:19.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.663+0000 7fd247400640 1 -- 172.21.15.100:0/2366151689 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2400047a0 con 0x562eb98613b0 2024-09-17T04:25:19.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.664+0000 7fd265250740 1 -- 172.21.15.100:0/2366151689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562eb98613b0 msgr2=0x562eb9c5b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:19.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.664+0000 7fd265250740 1 --2- 172.21.15.100:0/2366151689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562eb98613b0 0x562eb9c5b0a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd240034040 tx=0x7fd24003c000 comp rx=0 tx=0).stop 2024-09-17T04:25:19.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.664+0000 7fd265250740 1 -- 172.21.15.100:0/2366151689 shutdown_connections 2024-09-17T04:25:19.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.664+0000 7fd265250740 1 --2- 172.21.15.100:0/2366151689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eb9869590 0x562eb9c2b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.664+0000 7fd265250740 1 --2- 172.21.15.100:0/2366151689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562eb98613b0 0x562eb9c5b0a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.664+0000 7fd265250740 1 --2- 172.21.15.100:0/2366151689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562eb988a270 0x562eb9c5ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.664+0000 7fd265250740 1 -- 172.21.15.100:0/2366151689 >> 172.21.15.100:0/2366151689 conn(0x562eb987e3c0 msgr2=0x562eb9c30b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:19.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.664+0000 7fd265250740 1 -- 172.21.15.100:0/2366151689 shutdown_connections 2024-09-17T04:25:19.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.664+0000 7fd265250740 1 -- 172.21.15.100:0/2366151689 wait complete. 2024-09-17T04:25:19.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.665+0000 7fd265250740 1 -- start start 2024-09-17T04:25:19.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.665+0000 7fd265250740 1 Processor -- start 2024-09-17T04:25:19.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.667+0000 7fd265250740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562eb98613b0 0x562eb9c3ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:19.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.667+0000 7fd265250740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eb9869590 0x562eb9c2b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:19.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.667+0000 7fd265250740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562eb988a270 0x562eb9c2b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:19.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.667+0000 7fd265250740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562eb9b3a330 con 0x562eb98613b0 2024-09-17T04:25:19.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.667+0000 7fd265250740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562eb9b97810 con 0x562eb9869590 2024-09-17T04:25:19.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.667+0000 7fd265250740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x562eb9904620 con 0x562eb988a270 2024-09-17T04:25:19.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.667+0000 7fd265250740 10 client.0 _ll_register_callbacks cb 0x562eb9b3f420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:25:19.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.667+0000 7fd257400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562eb98613b0 0x562eb9c3ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:19.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.667+0000 7fd257400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562eb98613b0 0x562eb9c3ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57192/0 (socket says 172.21.15.100:57192) 2024-09-17T04:25:19.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.667+0000 7fd257400640 1 -- 172.21.15.100:0/1756548024 learned_addr learned my addr 172.21.15.100:0/1756548024 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:19.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.668+0000 7fd257e00640 1 --2- 172.21.15.100:0/1756548024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562eb988a270 0x562eb9c2b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:19.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.668+0000 7fd24ea00640 1 --2- 172.21.15.100:0/1756548024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eb9869590 0x562eb9c2b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:19.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.668+0000 7fd257400640 1 -- 172.21.15.100:0/1756548024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562eb988a270 msgr2=0x562eb9c2b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:25:19.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.668+0000 7fd257400640 1 --2- 172.21.15.100:0/1756548024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562eb988a270 0x562eb9c2b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.668+0000 7fd257400640 1 -- 172.21.15.100:0/1756548024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eb9869590 msgr2=0x562eb9c2b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:25:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.668+0000 7fd257400640 1 --2- 172.21.15.100:0/1756548024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eb9869590 0x562eb9c2b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.668+0000 7fd257400640 1 -- 172.21.15.100:0/1756548024 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eb98f5850 con 0x562eb98613b0 2024-09-17T04:25:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.668+0000 7fd257e00640 1 --2- 172.21.15.100:0/1756548024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562eb988a270 0x562eb9c2b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:25:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.668+0000 7fd24ea00640 1 --2- 172.21.15.100:0/1756548024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eb9869590 0x562eb9c2b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:25:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.668+0000 7fd257400640 1 --2- 172.21.15.100:0/1756548024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562eb98613b0 0x562eb9c3ece0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd248062ca0 tx=0x7fd248063570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418ec5248227f3e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.668+0000 7fd23be00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-17T04:25:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.668+0000 7fd23be00640 1 -- 172.21.15.100:0/1756548024 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd248089070 con 0x562eb98613b0 2024-09-17T04:25:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.668+0000 7fd23be00640 1 -- 172.21.15.100:0/1756548024 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fd248089210 con 0x562eb98613b0 2024-09-17T04:25:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.668+0000 7fd265250740 1 -- 172.21.15.100:0/1756548024 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd24004d070 con 0x562eb98613b0 2024-09-17T04:25:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.668+0000 7fd23be00640 1 -- 172.21.15.100:0/1756548024 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fd24801d040 con 0x562eb98613b0 2024-09-17T04:25:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.668+0000 7fd265250740 10 client.5046 fetch_fsmap learned FSMap version 17 2024-09-17T04:25:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.668+0000 7fd265250740 1 -- 172.21.15.100:0/1756548024 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x562eb996c6b0 con 0x562eb98613b0 2024-09-17T04:25:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.669+0000 7fd23be00640 1 -- 172.21.15.100:0/1756548024 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd24004d070 con 0x562eb98613b0 2024-09-17T04:25:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.669+0000 7fd23be00640 1 -- 172.21.15.100:0/1756548024 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7fd24807d080 con 0x562eb98613b0 2024-09-17T04:25:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.669+0000 7fd265250740 10 client.5046 fetch_fsmap finished waiting for FSMap version 17 2024-09-17T04:25:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.669+0000 7fd265250740 10 client.5046 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-17T04:25:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.669+0000 7fd265250740 20 client.5046 populate_metadata read hostname 'smithi100' 2024-09-17T04:25:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.669+0000 7fd265250740 1 --2- 172.21.15.100:0/1756548024 >> [v2:172.21.15.184:6834/294839954,v1:172.21.15.184:6837/294839954] conn(0x562eb9848e40 0x562eb9c880b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.669+0000 7fd265250740 4 client.5046 mds_command: new command op to 4857 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T04:25:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.669+0000 7fd265250740 1 -- 172.21.15.100:0/1756548024 --> [v2:172.21.15.184:6834/294839954,v1:172.21.15.184:6837/294839954] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562eb9ba6520 con 0x562eb9848e40 2024-09-17T04:25:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.669+0000 7fd24ea00640 1 --2- 172.21.15.100:0/1756548024 >> [v2:172.21.15.184:6834/294839954,v1:172.21.15.184:6837/294839954] conn(0x562eb9848e40 0x562eb9c880b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:19.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.670+0000 7fd24ea00640 1 --2- 172.21.15.100:0/1756548024 >> [v2:172.21.15.184:6834/294839954,v1:172.21.15.184:6837/294839954] conn(0x562eb9848e40 0x562eb9c880b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8068fb3156e4fe6c server_cookie=c76a43a185568e39 in_seq=0 out_seq=0 2024-09-17T04:25:19.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.670+0000 7fd23be00640 10 client.5046 ms_handle_connect on v2:172.21.15.184:6834/294839954 2024-09-17T04:25:19.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.671+0000 7fd23be00640 1 -- 172.21.15.100:0/1756548024 <== mds.0 v2:172.21.15.184:6834/294839954 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562eb9ba6520 con 0x562eb9848e40 2024-09-17T04:25:19.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.671+0000 7fd23be00640 10 client.5046 handle_command_reply: tid=0 2024-09-17T04:25:19.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.671+0000 7fd265250740 1 client.5046 shutdown 2024-09-17T04:25:19.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.671+0000 7fd265250740 1 -- 172.21.15.100:0/1756548024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562eb98613b0 msgr2=0x562eb9c3ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:19.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.671+0000 7fd265250740 1 --2- 172.21.15.100:0/1756548024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562eb98613b0 0x562eb9c3ece0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd248062ca0 tx=0x7fd248063570 comp rx=0 tx=0).stop 2024-09-17T04:25:19.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.671+0000 7fd265250740 1 -- 172.21.15.100:0/1756548024 shutdown_connections 2024-09-17T04:25:19.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.671+0000 7fd265250740 1 --2- 172.21.15.100:0/1756548024 >> [v2:172.21.15.184:6834/294839954,v1:172.21.15.184:6837/294839954] conn(0x562eb9848e40 0x562eb9c880b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.671+0000 7fd265250740 1 --2- 172.21.15.100:0/1756548024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562eb988a270 0x562eb9c2b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.671+0000 7fd265250740 1 --2- 172.21.15.100:0/1756548024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eb9869590 0x562eb9c2b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.671+0000 7fd265250740 1 --2- 172.21.15.100:0/1756548024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562eb98613b0 0x562eb9c3ece0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.671+0000 7fd23be00640 0 client.5046 ms_handle_reset on v2:172.21.15.184:6834/294839954 2024-09-17T04:25:19.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.671+0000 7fd265250740 1 -- 172.21.15.100:0/1756548024 >> 172.21.15.100:0/1756548024 conn(0x562eb987e3c0 msgr2=0x562eb9c520b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:19.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.672+0000 7fd265250740 1 -- 172.21.15.100:0/1756548024 shutdown_connections 2024-09-17T04:25:19.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.672+0000 7fd265250740 1 -- 172.21.15.100:0/1756548024 wait complete. 2024-09-17T04:25:19.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.672+0000 7fd265250740 20 client.5046 trim_cache size 0 max 16384 2024-09-17T04:25:19.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.690+0000 7fd265250740 1 Processor -- start 2024-09-17T04:25:19.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.691+0000 7fd265250740 1 -- start start 2024-09-17T04:25:19.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.692+0000 7fd265250740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eb98613b0 0x562eb9c5aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:19.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.692+0000 7fd265250740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562eb9869590 0x562eb9c5b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:19.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.692+0000 7fd265250740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562eb988a270 0x562eb9c761a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:19.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.692+0000 7fd265250740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd25814e0a0 con 0x562eb9869590 2024-09-17T04:25:19.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.692+0000 7fd265250740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd2580ca1e0 con 0x562eb98613b0 2024-09-17T04:25:19.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.692+0000 7fd265250740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd25814da50 con 0x562eb988a270 2024-09-17T04:25:19.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.692+0000 7fd24ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562eb9869590 0x562eb9c5b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:19.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.692+0000 7fd24ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562eb9869590 0x562eb9c5b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57208/0 (socket says 172.21.15.100:57208) 2024-09-17T04:25:19.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.692+0000 7fd24ea00640 1 -- 172.21.15.100:0/2393066258 learned_addr learned my addr 172.21.15.100:0/2393066258 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:19.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.692+0000 7fd257e00640 1 --2- 172.21.15.100:0/2393066258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562eb988a270 0x562eb9c761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:19.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.692+0000 7fd257400640 1 --2- 172.21.15.100:0/2393066258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eb98613b0 0x562eb9c5aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:19.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.693+0000 7fd24ea00640 1 -- 172.21.15.100:0/2393066258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562eb988a270 msgr2=0x562eb9c761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:25:19.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.693+0000 7fd24ea00640 1 --2- 172.21.15.100:0/2393066258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562eb988a270 0x562eb9c761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.693+0000 7fd24ea00640 1 -- 172.21.15.100:0/2393066258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eb98613b0 msgr2=0x562eb9c5aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:25:19.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.693+0000 7fd24ea00640 1 --2- 172.21.15.100:0/2393066258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eb98613b0 0x562eb9c5aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.693+0000 7fd24ea00640 1 -- 172.21.15.100:0/2393066258 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eb9c2e760 con 0x562eb9869590 2024-09-17T04:25:19.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.693+0000 7fd257400640 1 --2- 172.21.15.100:0/2393066258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eb98613b0 0x562eb9c5aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:25:19.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.693+0000 7fd24ea00640 1 --2- 172.21.15.100:0/2393066258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562eb9869590 0x562eb9c5b030 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd2400096c0 tx=0x7fd240009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3547cd5da4d8ef7a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:19.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.693+0000 7fd246a00640 1 -- 172.21.15.100:0/2393066258 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240035040 con 0x562eb9869590 2024-09-17T04:25:19.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.693+0000 7fd246a00640 1 -- 172.21.15.100:0/2393066258 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd24004d030 con 0x562eb9869590 2024-09-17T04:25:19.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.693+0000 7fd246a00640 1 -- 172.21.15.100:0/2393066258 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240003e60 con 0x562eb9869590 2024-09-17T04:25:19.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.694+0000 7fd265250740 1 -- 172.21.15.100:0/2393066258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562eb9869590 msgr2=0x562eb9c5b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:19.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.694+0000 7fd265250740 1 --2- 172.21.15.100:0/2393066258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562eb9869590 0x562eb9c5b030 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd2400096c0 tx=0x7fd240009260 comp rx=0 tx=0).stop 2024-09-17T04:25:19.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.694+0000 7fd265250740 1 -- 172.21.15.100:0/2393066258 shutdown_connections 2024-09-17T04:25:19.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.694+0000 7fd265250740 1 --2- 172.21.15.100:0/2393066258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562eb988a270 0x562eb9c761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.694+0000 7fd265250740 1 --2- 172.21.15.100:0/2393066258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562eb9869590 0x562eb9c5b030 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.694+0000 7fd265250740 1 --2- 172.21.15.100:0/2393066258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eb98613b0 0x562eb9c5aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.694+0000 7fd265250740 1 -- 172.21.15.100:0/2393066258 >> 172.21.15.100:0/2393066258 conn(0x562eb987e3c0 msgr2=0x562eb9c77ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.694+0000 7fd265250740 1 -- 172.21.15.100:0/2393066258 shutdown_connections 2024-09-17T04:25:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.694+0000 7fd265250740 1 -- 172.21.15.100:0/2393066258 wait complete. 2024-09-17T04:25:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.695+0000 7fd265250740 1 -- start start 2024-09-17T04:25:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.695+0000 7fd265250740 1 Processor -- start 2024-09-17T04:25:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.696+0000 7fd265250740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562eb98613b0 0x562eb9c700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.696+0000 7fd265250740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562eb9869590 0x562eb9cec1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:19.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.697+0000 7fd265250740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eb988a270 0x562eb9c96650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:19.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.697+0000 7fd265250740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562eb9c3f360 con 0x562eb98613b0 2024-09-17T04:25:19.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.697+0000 7fd265250740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562eb9c40ee0 con 0x562eb988a270 2024-09-17T04:25:19.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.697+0000 7fd265250740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x562eb9c576c0 con 0x562eb9869590 2024-09-17T04:25:19.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.697+0000 7fd265250740 10 client.0 _ll_register_callbacks cb 0x7fd248062de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:25:19.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.697+0000 7fd257400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562eb98613b0 0x562eb9c700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:19.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.697+0000 7fd257400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562eb98613b0 0x562eb9c700e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57216/0 (socket says 172.21.15.100:57216) 2024-09-17T04:25:19.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.697+0000 7fd257400640 1 -- 172.21.15.100:0/3529038366 learned_addr learned my addr 172.21.15.100:0/3529038366 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:19.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.697+0000 7fd24ea00640 1 --2- 172.21.15.100:0/3529038366 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562eb9869590 0x562eb9cec1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:19.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.697+0000 7fd257e00640 1 --2- 172.21.15.100:0/3529038366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eb988a270 0x562eb9c96650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:19.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.697+0000 7fd257400640 1 -- 172.21.15.100:0/3529038366 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562eb9869590 msgr2=0x562eb9cec1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:25:19.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.697+0000 7fd257400640 1 --2- 172.21.15.100:0/3529038366 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562eb9869590 0x562eb9cec1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.697+0000 7fd257400640 1 -- 172.21.15.100:0/3529038366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eb988a270 msgr2=0x562eb9c96650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:25:19.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.697+0000 7fd257400640 1 --2- 172.21.15.100:0/3529038366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eb988a270 0x562eb9c96650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.697+0000 7fd257400640 1 -- 172.21.15.100:0/3529038366 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eb9c31840 con 0x562eb98613b0 2024-09-17T04:25:19.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.697+0000 7fd24ea00640 1 --2- 172.21.15.100:0/3529038366 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562eb9869590 0x562eb9cec1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:25:19.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.697+0000 7fd257e00640 1 --2- 172.21.15.100:0/3529038366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eb988a270 0x562eb9c96650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:25:19.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.697+0000 7fd257400640 1 --2- 172.21.15.100:0/3529038366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562eb98613b0 0x562eb9c700e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd248062c30 tx=0x7fd248062710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeae8cebd095472f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:19.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.697+0000 7fd23be00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-17T04:25:19.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.698+0000 7fd23be00640 1 -- 172.21.15.100:0/3529038366 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd248089040 con 0x562eb98613b0 2024-09-17T04:25:19.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.698+0000 7fd23be00640 1 -- 172.21.15.100:0/3529038366 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fd248062410 con 0x562eb98613b0 2024-09-17T04:25:19.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.698+0000 7fd265250740 1 -- 172.21.15.100:0/3529038366 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd24004d030 con 0x562eb98613b0 2024-09-17T04:25:19.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.698+0000 7fd23be00640 1 -- 172.21.15.100:0/3529038366 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fd248067a80 con 0x562eb98613b0 2024-09-17T04:25:19.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.698+0000 7fd265250740 10 client.5052 fetch_fsmap learned FSMap version 17 2024-09-17T04:25:19.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.698+0000 7fd265250740 1 -- 172.21.15.100:0/3529038366 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x562eb9cf9430 con 0x562eb98613b0 2024-09-17T04:25:19.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.698+0000 7fd23be00640 1 -- 172.21.15.100:0/3529038366 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd24004d030 con 0x562eb98613b0 2024-09-17T04:25:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.698+0000 7fd23be00640 1 -- 172.21.15.100:0/3529038366 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7fd248085040 con 0x562eb98613b0 2024-09-17T04:25:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.698+0000 7fd265250740 10 client.5052 fetch_fsmap finished waiting for FSMap version 17 2024-09-17T04:25:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.698+0000 7fd265250740 10 client.5052 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-17T04:25:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.698+0000 7fd265250740 20 client.5052 populate_metadata read hostname 'smithi100' 2024-09-17T04:25:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.698+0000 7fd265250740 1 --2- 172.21.15.100:0/3529038366 >> [v2:172.21.15.184:6834/294839954,v1:172.21.15.184:6837/294839954] conn(0x562eb9cf61c0 0x562eb9c3fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.698+0000 7fd265250740 4 client.5052 mds_command: new command op to 4857 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T04:25:19.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.698+0000 7fd265250740 1 -- 172.21.15.100:0/3529038366 --> [v2:172.21.15.184:6834/294839954,v1:172.21.15.184:6837/294839954] -- command(tid 0: {"prefix": "status"}) -- 0x562eb9c301a0 con 0x562eb9cf61c0 2024-09-17T04:25:19.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.699+0000 7fd24ea00640 1 --2- 172.21.15.100:0/3529038366 >> [v2:172.21.15.184:6834/294839954,v1:172.21.15.184:6837/294839954] conn(0x562eb9cf61c0 0x562eb9c3fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:19.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.699+0000 7fd24ea00640 1 --2- 172.21.15.100:0/3529038366 >> [v2:172.21.15.184:6834/294839954,v1:172.21.15.184:6837/294839954] conn(0x562eb9cf61c0 0x562eb9c3fcc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea861bb66af8363 server_cookie=913a901172a8fa12 in_seq=0 out_seq=0 2024-09-17T04:25:19.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.699+0000 7fd23be00640 10 client.5052 ms_handle_connect on v2:172.21.15.184:6834/294839954 2024-09-17T04:25:19.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.700+0000 7fd23be00640 1 -- 172.21.15.100:0/3529038366 <== mds.0 v2:172.21.15.184:6834/294839954 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x562eb9c301a0 con 0x562eb9cf61c0 2024-09-17T04:25:19.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.700+0000 7fd23be00640 10 client.5052 handle_command_reply: tid=0 2024-09-17T04:25:19.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.700+0000 7fd265250740 1 client.5052 shutdown 2024-09-17T04:25:19.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.700+0000 7fd265250740 1 -- 172.21.15.100:0/3529038366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562eb98613b0 msgr2=0x562eb9c700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:19.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.700+0000 7fd265250740 1 --2- 172.21.15.100:0/3529038366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562eb98613b0 0x562eb9c700e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd248062c30 tx=0x7fd248062710 comp rx=0 tx=0).stop 2024-09-17T04:25:19.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.700+0000 7fd265250740 1 -- 172.21.15.100:0/3529038366 shutdown_connections 2024-09-17T04:25:19.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.700+0000 7fd265250740 1 --2- 172.21.15.100:0/3529038366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eb988a270 0x562eb9c96650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.700+0000 7fd265250740 1 --2- 172.21.15.100:0/3529038366 >> [v2:172.21.15.184:6834/294839954,v1:172.21.15.184:6837/294839954] conn(0x562eb9cf61c0 0x562eb9c3fcc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.700+0000 7fd265250740 1 --2- 172.21.15.100:0/3529038366 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562eb9869590 0x562eb9cec1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.700+0000 7fd265250740 1 --2- 172.21.15.100:0/3529038366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562eb98613b0 0x562eb9c700e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.700+0000 7fd265250740 1 -- 172.21.15.100:0/3529038366 >> 172.21.15.100:0/3529038366 conn(0x562eb987e3c0 msgr2=0x562eb9c75800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:19.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.700+0000 7fd23be00640 0 client.5052 ms_handle_reset on v2:172.21.15.184:6834/294839954 2024-09-17T04:25:19.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.700+0000 7fd265250740 1 -- 172.21.15.100:0/3529038366 shutdown_connections 2024-09-17T04:25:19.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.700+0000 7fd265250740 1 -- 172.21.15.100:0/3529038366 wait complete. 2024-09-17T04:25:19.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.700+0000 7fd265250740 20 client.5052 trim_cache size 0 max 16384 2024-09-17T04:25:19.710 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:25:19.711 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "8a399842-3a5c-4ea2-9afb-b2704332e59f", 2024-09-17T04:25:19.711 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 0, 2024-09-17T04:25:19.711 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 4857, 2024-09-17T04:25:19.711 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-09-17T04:25:19.711 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-09-17T04:25:19.711 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-09-17T04:25:19.711 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 11.616990354, 2024-09-17T04:25:19.711 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 17, 2024-09-17T04:25:19.711 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 36, 2024-09-17T04:25:19.711 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 36, 2024-09-17T04:25:19.711 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 19.349306689999999 2024-09-17T04:25:19.711 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:25:19.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.701+0000 7fd25ca00640 1 -- 172.21.15.100:0/794423280 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd234078b90 msgr2=0x7fd23407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:19.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.701+0000 7fd25ca00640 1 --2- 172.21.15.100:0/794423280 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd234078b90 0x7fd23407b0a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd240033000 tx=0x7fd240004730 comp rx=0 tx=0).stop 2024-09-17T04:25:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.701+0000 7fd25ca00640 1 -- 172.21.15.100:0/794423280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd25806f710 msgr2=0x7fd2581f97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.701+0000 7fd25ca00640 1 --2- 172.21.15.100:0/794423280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd25806f710 0x7fd2581f97e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd24800c940 tx=0x7fd24800ce00 comp rx=0 tx=0).stop 2024-09-17T04:25:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.701+0000 7fd25ca00640 1 -- 172.21.15.100:0/794423280 shutdown_connections 2024-09-17T04:25:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.701+0000 7fd25ca00640 1 --2- 172.21.15.100:0/794423280 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd234078b90 0x7fd23407b0a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.701+0000 7fd25ca00640 1 --2- 172.21.15.100:0/794423280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd258161fa0 0x7fd2580cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.701+0000 7fd25ca00640 1 --2- 172.21.15.100:0/794423280 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2580d0cc0 0x7fd2580cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.701+0000 7fd25ca00640 1 --2- 172.21.15.100:0/794423280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd25806f710 0x7fd2581f97e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.701+0000 7fd25ca00640 1 -- 172.21.15.100:0/794423280 >> 172.21.15.100:0/794423280 conn(0x7fd25808dc30 msgr2=0x7fd258160630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.701+0000 7fd25ca00640 1 -- 172.21.15.100:0/794423280 shutdown_connections 2024-09-17T04:25:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.701+0000 7fd25ca00640 1 -- 172.21.15.100:0/794423280 wait complete. 2024-09-17T04:25:19.721 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:25:19.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.942+0000 7f6733400640 1 -- 172.21.15.100:0/152198734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f672c0b3550 msgr2=0x7f672c0b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:19.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.942+0000 7f6733400640 1 --2- 172.21.15.100:0/152198734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f672c0b3550 0x7f672c0b3960 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6724005fd0 tx=0x7f672402aa10 comp rx=0 tx=0).stop 2024-09-17T04:25:19.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.942+0000 7f6733400640 1 -- 172.21.15.100:0/152198734 shutdown_connections 2024-09-17T04:25:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.942+0000 7f6733400640 1 --2- 172.21.15.100:0/152198734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f672c0b3550 0x7f672c0b3960 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.942+0000 7f6733400640 1 --2- 172.21.15.100:0/152198734 >> v2:172.21.15.184:3300/0 conn(0x7f672c0ad0e0 0x7f672c0ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.942+0000 7f6733400640 1 -- 172.21.15.100:0/152198734 >> 172.21.15.100:0/152198734 conn(0x7f672c06d730 msgr2=0x7f672c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.943+0000 7f6733400640 1 -- 172.21.15.100:0/152198734 shutdown_connections 2024-09-17T04:25:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.943+0000 7f6733400640 1 -- 172.21.15.100:0/152198734 wait complete. 2024-09-17T04:25:19.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.945+0000 7f6733400640 1 Processor -- start 2024-09-17T04:25:19.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.945+0000 7f6733400640 1 -- start start 2024-09-17T04:25:19.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.946+0000 7f6733400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f672c0ad0e0 0x7f672c1f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:19.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.946+0000 7f6733400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f672c0b3550 0x7f672c0994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:19.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.946+0000 7f6733400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f672c15d760 0x7f672c06f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:19.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.946+0000 7f6733400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f672c0b4da0 con 0x7f672c15d760 2024-09-17T04:25:19.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.946+0000 7f6733400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f672c0b4c20 con 0x7f672c0b3550 2024-09-17T04:25:19.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.946+0000 7f6733400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f672c0b3ea0 con 0x7f672c0ad0e0 2024-09-17T04:25:19.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.947+0000 7f6731400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f672c15d760 0x7f672c06f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:19.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.947+0000 7f6731400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f672c15d760 0x7f672c06f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57226/0 (socket says 172.21.15.100:57226) 2024-09-17T04:25:19.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.947+0000 7f6731400640 1 -- 172.21.15.100:0/1853861760 learned_addr learned my addr 172.21.15.100:0/1853861760 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:19.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.947+0000 7f6723e00640 1 --2- 172.21.15.100:0/1853861760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f672c0b3550 0x7f672c0994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:19.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.947+0000 7f6730a00640 1 --2- 172.21.15.100:0/1853861760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f672c0ad0e0 0x7f672c1f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:19.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.947+0000 7f6731400640 1 -- 172.21.15.100:0/1853861760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f672c0ad0e0 msgr2=0x7f672c1f36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:19.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.947+0000 7f6731400640 1 --2- 172.21.15.100:0/1853861760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f672c0ad0e0 0x7f672c1f36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.947+0000 7f6731400640 1 -- 172.21.15.100:0/1853861760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f672c0b3550 msgr2=0x7f672c0994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:19.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.947+0000 7f6731400640 1 --2- 172.21.15.100:0/1853861760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f672c0b3550 0x7f672c0994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:19.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.947+0000 7f6731400640 1 -- 172.21.15.100:0/1853861760 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f672c1f3c30 con 0x7f672c15d760 2024-09-17T04:25:19.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.947+0000 7f6723e00640 1 --2- 172.21.15.100:0/1853861760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f672c0b3550 0x7f672c0994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:19.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.947+0000 7f6731400640 1 --2- 172.21.15.100:0/1853861760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f672c15d760 0x7f672c06f420 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6724009d90 tx=0x7f6724009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:19.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.948+0000 7f6729600640 1 -- 172.21.15.100:0/1853861760 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724033050 con 0x7f672c15d760 2024-09-17T04:25:19.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.948+0000 7f6729600640 1 -- 172.21.15.100:0/1853861760 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6724004450 con 0x7f672c15d760 2024-09-17T04:25:19.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.948+0000 7f6729600640 1 -- 172.21.15.100:0/1853861760 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724004840 con 0x7f672c15d760 2024-09-17T04:25:19.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.948+0000 7f6733400640 1 -- 172.21.15.100:0/1853861760 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f672c1f4140 con 0x7f672c15d760 2024-09-17T04:25:19.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.948+0000 7f6733400640 1 -- 172.21.15.100:0/1853861760 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f672c1f4a50 con 0x7f672c15d760 2024-09-17T04:25:19.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.949+0000 7f6733400640 1 -- 172.21.15.100:0/1853861760 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f672c1f3eb0 con 0x7f672c15d760 2024-09-17T04:25:19.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.950+0000 7f6729600640 1 -- 172.21.15.100:0/1853861760 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6724036070 con 0x7f672c15d760 2024-09-17T04:25:19.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.951+0000 7f6729600640 1 --2- 172.21.15.100:0/1853861760 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6708078b90 0x7f670807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:19.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.952+0000 7f6729600640 1 -- 172.21.15.100:0/1853861760 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f67240bc130 con 0x7f672c15d760 2024-09-17T04:25:19.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.952+0000 7f6730a00640 1 --2- 172.21.15.100:0/1853861760 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6708078b90 0x7f670807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:19.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.953+0000 7f6730a00640 1 --2- 172.21.15.100:0/1853861760 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6708078b90 0x7f670807b0a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f671c007fd0 tx=0x7f671c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:19.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:19.956+0000 7f6729600640 1 -- 172.21.15.100:0/1853861760 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6724084560 con 0x7f672c15d760 2024-09-17T04:25:20.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:20.177+0000 7f6733400640 1 -- 172.21.15.100:0/1853861760 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f672c1f4660 con 0x7f672c15d760 2024-09-17T04:25:20.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:20.178+0000 7f6729600640 1 -- 172.21.15.100:0/1853861760 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f672c1f4660 con 0x7f672c15d760 2024-09-17T04:25:20.177 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:25:20.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:20.180+0000 7f6733400640 1 -- 172.21.15.100:0/1853861760 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6708078b90 msgr2=0x7f670807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:20.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:20.180+0000 7f6733400640 1 --2- 172.21.15.100:0/1853861760 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6708078b90 0x7f670807b0a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f671c007fd0 tx=0x7f671c009040 comp rx=0 tx=0).stop 2024-09-17T04:25:20.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:20.180+0000 7f6733400640 1 -- 172.21.15.100:0/1853861760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f672c15d760 msgr2=0x7f672c06f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:20.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:20.180+0000 7f6733400640 1 --2- 172.21.15.100:0/1853861760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f672c15d760 0x7f672c06f420 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6724009d90 tx=0x7f6724009360 comp rx=0 tx=0).stop 2024-09-17T04:25:20.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:20.180+0000 7f6733400640 1 -- 172.21.15.100:0/1853861760 shutdown_connections 2024-09-17T04:25:20.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:20.180+0000 7f6733400640 1 --2- 172.21.15.100:0/1853861760 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6708078b90 0x7f670807b0a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:20.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:20.180+0000 7f6733400640 1 --2- 172.21.15.100:0/1853861760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f672c15d760 0x7f672c06f420 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:20.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:20.180+0000 7f6733400640 1 --2- 172.21.15.100:0/1853861760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f672c0b3550 0x7f672c0994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:20.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:20.180+0000 7f6733400640 1 --2- 172.21.15.100:0/1853861760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f672c0ad0e0 0x7f672c1f36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:20.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:20.180+0000 7f6733400640 1 -- 172.21.15.100:0/1853861760 >> 172.21.15.100:0/1853861760 conn(0x7f672c06d730 msgr2=0x7f672c0cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:20.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:20.180+0000 7f6733400640 1 -- 172.21.15.100:0/1853861760 shutdown_connections 2024-09-17T04:25:20.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:20.180+0000 7f6733400640 1 -- 172.21.15.100:0/1853861760 wait complete. 2024-09-17T04:25:20.203 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T04:25:20.203 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:25:20.203 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-09-17T04:25:20.203 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T04:25:20.203 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T04:25:20.204 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T04:25:20.204 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T04:25:20.204 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:25:20.204 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:25:20.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:20.205 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-09-17T04:25:20.242 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:25:20.242 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:25:20.242 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:25:20.242 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:25:20.242 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-17T04:25:20.242 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:25:20.242 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:25:20.242 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:25:20.242 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-09-17T04:25:20.243 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:25:20.243 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:25:20.243 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-09-17T04:25:20.243 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:25:20.243 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:25:20.243 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:25:20.243 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:25:20.243 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T04:25:20.243 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42640sec preferred_lft 42640sec 2024-09-17T04:25:20.243 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link noprefixroute 2024-09-17T04:25:20.243 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:25:20.243 INFO:teuthology.orchestra.run.smithi100.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:25:20.243 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:2e:de:8e:a4:7c brd ff:ff:ff:ff:ff:ff 2024-09-17T04:25:20.244 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T04:25:20.244 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:25:20.244 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::cc78:e4ff:fe70:7fbe/64 scope link 2024-09-17T04:25:20.244 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:25:20.244 INFO:teuthology.orchestra.run.smithi100.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:25:20.244 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether ea:8c:45:9f:34:87 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:25:20.244 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::e88c:45ff:fe9f:3487/64 scope link 2024-09-17T04:25:20.244 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:25:20.244 INFO:teuthology.orchestra.run.smithi100.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:25:20.244 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:2e:de:8e:a4:7c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:25:20.244 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::c2e:deff:fe8e:a47c/64 scope link 2024-09-17T04:25:20.244 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:25:20.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:20.245 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:25:20.335 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-09-17T04:25:20.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:20.339 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-09-17T04:25:20.375 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-09-17T04:25:20.375 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:25:20.375 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T04:25:20.375 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T04:25:20.375 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:25:20.375 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:25:20.376 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-17T04:25:20.376 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T04:25:20.376 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T04:25:20.376 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T04:25:20.376 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-17T04:25:20.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:25:20.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:25:20.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:20.570 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-17T04:25:20.605 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T04:25:20.605 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T04:25:20.606 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:25:20.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:20.606 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-09-17T04:25:20.703 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:25:20.704 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T04:25:20.738 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:25:20.739 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:25:20.739 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:25:20.813 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T04:25:20.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:20.813 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:25:21.009 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:25:21.009 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:25:21.009 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:25:21.029 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:25:21.029 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:25:21.029 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:25:21.030 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:25:21.030 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:25:21.060 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:25:21.060 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:25:21.060 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:25:21.060 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:25:21.060 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:25:21.060 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-09-17T04:25:21.060 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-09-17T04:25:21.060 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8a399842-3a5c-4ea2-9afb-b2704332e59f" will pass to kernel 2024-09-17T04:25:21.061 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-17T04:25:21.061 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-09-17T04:25:21.062 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-17T04:25:21.062 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-09-17T04:25:21.062 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-17T04:25:21.065 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T04:25:21.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:21.065 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:25:21.122 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:25:21.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:25:21.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:25:21.229 INFO:teuthology.orchestra.run.smithi100.stdout:client5058 2024-09-17T04:25:21.229 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:25:21.229 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:25:21.265 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:25:21.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.526+0000 7f86bee00640 1 -- 172.21.15.100:0/2098298698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f869c004af0 msgr2=0x7f869c004f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:21.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.526+0000 7f86bee00640 1 --2- 172.21.15.100:0/2098298698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f869c004af0 0x7f869c004f20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f86a8009b50 tx=0x7f869c006940 comp rx=0 tx=0).stop 2024-09-17T04:25:21.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.526+0000 7f86bce00640 1 -- 172.21.15.100:0/2098298698 reap_dead start 2024-09-17T04:25:21.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.526+0000 7f86bee00640 1 -- 172.21.15.100:0/2098298698 shutdown_connections 2024-09-17T04:25:21.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.526+0000 7f86bee00640 1 -- 172.21.15.100:0/2098298698 >> 172.21.15.100:0/2098298698 conn(0x7f86b806d730 msgr2=0x7f86b80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:21.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.526+0000 7f86bee00640 1 -- 172.21.15.100:0/2098298698 shutdown_connections 2024-09-17T04:25:21.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.527+0000 7f86bee00640 1 -- 172.21.15.100:0/2098298698 wait complete. 2024-09-17T04:25:21.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.529+0000 7f86bee00640 1 Processor -- start 2024-09-17T04:25:21.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.529+0000 7f86bee00640 1 -- start start 2024-09-17T04:25:21.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.530+0000 7f86bee00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f869c004af0 0x7f86b813f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:21.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.530+0000 7f86bee00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b814b480 0x7f86b80b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:21.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.530+0000 7f86bee00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86b80b1710 0x7f86b8159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:21.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.530+0000 7f86bee00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f86b80712b0 con 0x7f86b80b1710 2024-09-17T04:25:21.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.530+0000 7f86bee00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f86b8071130 con 0x7f86b814b480 2024-09-17T04:25:21.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.530+0000 7f86bee00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f86b8073040 con 0x7f869c004af0 2024-09-17T04:25:21.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.531+0000 7f86bce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86b80b1710 0x7f86b8159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:21.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.531+0000 7f86bce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86b80b1710 0x7f86b8159510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57242/0 (socket says 172.21.15.100:57242) 2024-09-17T04:25:21.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.531+0000 7f86bce00640 1 -- 172.21.15.100:0/922621030 learned_addr learned my addr 172.21.15.100:0/922621030 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:21.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.531+0000 7f86b7e00640 1 --2- 172.21.15.100:0/922621030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f869c004af0 0x7f86b813f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:21.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.531+0000 7f86b7400640 1 --2- 172.21.15.100:0/922621030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b814b480 0x7f86b80b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:21.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.531+0000 7f86bce00640 1 -- 172.21.15.100:0/922621030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f869c004af0 msgr2=0x7f86b813f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:21.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.531+0000 7f86bce00640 1 --2- 172.21.15.100:0/922621030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f869c004af0 0x7f86b813f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:21.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.531+0000 7f86bce00640 1 -- 172.21.15.100:0/922621030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b814b480 msgr2=0x7f86b80b11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:21.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.531+0000 7f86bce00640 1 --2- 172.21.15.100:0/922621030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b814b480 0x7f86b80b11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:21.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.531+0000 7f86bce00640 1 -- 172.21.15.100:0/922621030 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b80b1b20 con 0x7f86b80b1710 2024-09-17T04:25:21.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.531+0000 7f86b7400640 1 --2- 172.21.15.100:0/922621030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b814b480 0x7f86b80b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:21.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.531+0000 7f86b7e00640 1 --2- 172.21.15.100:0/922621030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f869c004af0 0x7f86b813f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:21.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.531+0000 7f86bce00640 1 --2- 172.21.15.100:0/922621030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86b80b1710 0x7f86b8159510 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f86b8071090 tx=0x7f86b0006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:21.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.531+0000 7f86afe00640 1 -- 172.21.15.100:0/922621030 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b000fa20 con 0x7f86b80b1710 2024-09-17T04:25:21.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.532+0000 7f86afe00640 1 -- 172.21.15.100:0/922621030 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86b0018070 con 0x7f86b80b1710 2024-09-17T04:25:21.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.532+0000 7f86bee00640 1 -- 172.21.15.100:0/922621030 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b8157b40 con 0x7f86b80b1710 2024-09-17T04:25:21.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.532+0000 7f86afe00640 1 -- 172.21.15.100:0/922621030 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b000fe80 con 0x7f86b80b1710 2024-09-17T04:25:21.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.532+0000 7f86bee00640 1 -- 172.21.15.100:0/922621030 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b8158100 con 0x7f86b80b1710 2024-09-17T04:25:21.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.533+0000 7f86bee00640 1 -- 172.21.15.100:0/922621030 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86b80735d0 con 0x7f86b80b1710 2024-09-17T04:25:21.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.534+0000 7f86afe00640 1 -- 172.21.15.100:0/922621030 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f86b001d460 con 0x7f86b80b1710 2024-09-17T04:25:21.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.535+0000 7f86afe00640 1 --2- 172.21.15.100:0/922621030 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8698078b90 0x7f869807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:21.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.535+0000 7f86afe00640 1 -- 172.21.15.100:0/922621030 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f86b009b4c0 con 0x7f86b80b1710 2024-09-17T04:25:21.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.536+0000 7f86b7e00640 1 --2- 172.21.15.100:0/922621030 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8698078b90 0x7f869807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:21.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.537+0000 7f86b7e00640 1 --2- 172.21.15.100:0/922621030 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8698078b90 0x7f869807b0a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f86a8035000 tx=0x7f86a802d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:21.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.540+0000 7f86afe00640 1 -- 172.21.15.100:0/922621030 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f86b80b1b20 con 0x7f86b80b1710 2024-09-17T04:25:21.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.759+0000 7f86bee00640 1 -- 172.21.15.100:0/922621030 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f86b81578d0 con 0x7f86b80b1710 2024-09-17T04:25:21.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.760+0000 7f86afe00640 1 -- 172.21.15.100:0/922621030 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f86b0063860 con 0x7f86b80b1710 2024-09-17T04:25:21.759 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:25:21.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.762+0000 7f86bee00640 1 -- 172.21.15.100:0/922621030 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8698078b90 msgr2=0x7f869807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:21.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.762+0000 7f86bee00640 1 --2- 172.21.15.100:0/922621030 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8698078b90 0x7f869807b0a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f86a8035000 tx=0x7f86a802d040 comp rx=0 tx=0).stop 2024-09-17T04:25:21.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.762+0000 7f86bee00640 1 -- 172.21.15.100:0/922621030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86b80b1710 msgr2=0x7f86b8159510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:21.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.762+0000 7f86bee00640 1 --2- 172.21.15.100:0/922621030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86b80b1710 0x7f86b8159510 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f86b8071090 tx=0x7f86b0006720 comp rx=0 tx=0).stop 2024-09-17T04:25:21.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.762+0000 7f86bee00640 1 -- 172.21.15.100:0/922621030 shutdown_connections 2024-09-17T04:25:21.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.762+0000 7f86bee00640 1 --2- 172.21.15.100:0/922621030 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8698078b90 0x7f869807b0a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:21.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.762+0000 7f86bee00640 1 --2- 172.21.15.100:0/922621030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86b80b1710 0x7f86b8159510 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:21.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.762+0000 7f86bee00640 1 --2- 172.21.15.100:0/922621030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b814b480 0x7f86b80b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:21.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.762+0000 7f86bee00640 1 --2- 172.21.15.100:0/922621030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f869c004af0 0x7f86b813f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:21.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.762+0000 7f86bee00640 1 -- 172.21.15.100:0/922621030 >> 172.21.15.100:0/922621030 conn(0x7f86b806d730 msgr2=0x7f86b815a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:21.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.762+0000 7f86bee00640 1 -- 172.21.15.100:0/922621030 shutdown_connections 2024-09-17T04:25:21.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:21.762+0000 7f86bee00640 1 -- 172.21.15.100:0/922621030 wait complete. 2024-09-17T04:25:21.782 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T04:25:21.782 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:25:21.782 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2024-09-17T04:25:21.782 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T04:25:21.782 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T04:25:21.783 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T04:25:21.783 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T04:25:21.783 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:25:21.783 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:25:21.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:21.783 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2024-09-17T04:25:21.824 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:25:21.824 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:25:21.824 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:25:21.825 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:25:21.825 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-09-17T04:25:21.825 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:25:21.825 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:25:21.825 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:25:21.825 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2024-09-17T04:25:21.825 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:25:21.825 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:25:21.825 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2024-09-17T04:25:21.825 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:25:21.825 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2024-09-17T04:25:21.825 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:25:21.825 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2024-09-17T04:25:21.826 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T04:25:21.826 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42622sec preferred_lft 42622sec 2024-09-17T04:25:21.826 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ec4:7aff:fe88:7b8f/64 scope link noprefixroute 2024-09-17T04:25:21.826 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:25:21.826 INFO:teuthology.orchestra.run.smithi184.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:25:21.826 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 7e:67:e5:69:96:69 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:25:21.826 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T04:25:21.826 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:25:21.826 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::3c9d:aff:fe6e:8fcb/64 scope link 2024-09-17T04:25:21.826 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:25:21.826 INFO:teuthology.orchestra.run.smithi184.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:25:21.826 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether a6:a9:1b:a9:61:ec brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T04:25:21.826 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::a4a9:1bff:fea9:61ec/64 scope link 2024-09-17T04:25:21.827 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:25:21.827 INFO:teuthology.orchestra.run.smithi184.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:25:21.827 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 7e:67:e5:69:96:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T04:25:21.827 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::7c67:e5ff:fe69:9669/64 scope link 2024-09-17T04:25:21.827 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:25:21.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:21.827 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:25:21.922 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-09-17T04:25:21.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:21.926 DEBUG:teuthology.orchestra.run.smithi184:> route 2024-09-17T04:25:21.962 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2024-09-17T04:25:21.962 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:25:21.962 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T04:25:21.963 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T04:25:21.963 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:25:21.963 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:25:21.963 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-09-17T04:25:21.963 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T04:25:21.963 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T04:25:21.963 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T04:25:21.963 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-09-17T04:25:22.084 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:25:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:25:22.155 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:25:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:25:22.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:22.163 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-09-17T04:25:22.198 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T04:25:22.198 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T04:25:22.198 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:25:22.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:22.199 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.0 up 2024-09-17T04:25:22.294 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:25:22.294 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T04:25:22.329 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:25:22.330 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:25:22.330 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T04:25:22.404 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T04:25:22.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:22.405 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:25:22.599 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:25:22.599 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:25:22.599 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:25:22.619 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:25:22.619 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:25:22.619 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:25:22.619 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:25:22.619 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:25:22.649 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:25:22.649 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:25:22.650 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:25:22.650 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:25:22.650 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:25:22.650 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-09-17T04:25:22.650 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-09-17T04:25:22.650 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8a399842-3a5c-4ea2-9afb-b2704332e59f" will pass to kernel 2024-09-17T04:25:22.651 INFO:teuthology.orchestra.run.smithi184.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-17T04:25:22.651 INFO:teuthology.orchestra.run.smithi184.stdout: You just mounted a file system that supports labels which does not 2024-09-17T04:25:22.651 INFO:teuthology.orchestra.run.smithi184.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-17T04:25:22.651 INFO:teuthology.orchestra.run.smithi184.stdout: applications will generate AVC messages and not be allowed access to 2024-09-17T04:25:22.651 INFO:teuthology.orchestra.run.smithi184.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-17T04:25:22.654 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T04:25:22.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:22.654 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T04:25:22.709 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:25:22.788 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:25:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:25:22.791 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:25:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:25:22.798 INFO:teuthology.orchestra.run.smithi184.stdout:client5047 2024-09-17T04:25:22.799 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:25:22.799 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:25:22.834 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T04:25:23.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.070+0000 7f770e200640 1 -- 172.21.15.100:0/992150757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77080712a0 msgr2=0x7f770806ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:23.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.070+0000 7f770e200640 1 --2- 172.21.15.100:0/992150757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77080712a0 0x7f770806ddb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f76f4009860 tx=0x7f76f402a4b0 comp rx=0 tx=0).stop 2024-09-17T04:25:23.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.071+0000 7f770e200640 1 -- 172.21.15.100:0/992150757 shutdown_connections 2024-09-17T04:25:23.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.071+0000 7f770e200640 1 --2- 172.21.15.100:0/992150757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77080712a0 0x7f770806ddb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:23.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.071+0000 7f770e200640 1 --2- 172.21.15.100:0/992150757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7708152990 0x7f770815c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:23.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.071+0000 7f770e200640 1 -- 172.21.15.100:0/992150757 >> 172.21.15.100:0/992150757 conn(0x7f7708073890 msgr2=0x7f7708073ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:23.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.071+0000 7f770e200640 1 -- 172.21.15.100:0/992150757 shutdown_connections 2024-09-17T04:25:23.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.071+0000 7f770e200640 1 -- 172.21.15.100:0/992150757 wait complete. 2024-09-17T04:25:23.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.073+0000 7f770e200640 1 Processor -- start 2024-09-17T04:25:23.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.073+0000 7f770e200640 1 -- start start 2024-09-17T04:25:23.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.074+0000 7f770e200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77080712a0 0x7f77081e6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:23.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.074+0000 7f770e200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77080716b0 0x7f770814f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:23.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.074+0000 7f770e200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7708152990 0x7f770806f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:23.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.074+0000 7f770e200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f77080b54f0 con 0x7f77080716b0 2024-09-17T04:25:23.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.074+0000 7f770e200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f770806e650 con 0x7f77080712a0 2024-09-17T04:25:23.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.074+0000 7f770e200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f77080b5670 con 0x7f7708152990 2024-09-17T04:25:23.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.075+0000 7f7706a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77080716b0 0x7f770814f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:23.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.075+0000 7f7706a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77080716b0 0x7f770814f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57248/0 (socket says 172.21.15.100:57248) 2024-09-17T04:25:23.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.075+0000 7f7706a00640 1 -- 172.21.15.100:0/3979239745 learned_addr learned my addr 172.21.15.100:0/3979239745 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:23.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.075+0000 7f7707400640 1 --2- 172.21.15.100:0/3979239745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77080712a0 0x7f77081e6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:23.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.075+0000 7f7707e00640 1 --2- 172.21.15.100:0/3979239745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7708152990 0x7f770806f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:23.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.075+0000 7f7706a00640 1 -- 172.21.15.100:0/3979239745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7708152990 msgr2=0x7f770806f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:23.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.075+0000 7f7706a00640 1 --2- 172.21.15.100:0/3979239745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7708152990 0x7f770806f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:23.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.075+0000 7f7706a00640 1 -- 172.21.15.100:0/3979239745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77080712a0 msgr2=0x7f77081e6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:23.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.075+0000 7f7706a00640 1 --2- 172.21.15.100:0/3979239745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77080712a0 0x7f77081e6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:23.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.075+0000 7f7706a00640 1 -- 172.21.15.100:0/3979239745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77081e6a90 con 0x7f77080716b0 2024-09-17T04:25:23.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.075+0000 7f7707400640 1 --2- 172.21.15.100:0/3979239745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77080712a0 0x7f77081e6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:25:23.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.075+0000 7f7706a00640 1 --2- 172.21.15.100:0/3979239745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77080716b0 0x7f770814f210 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f76f402aaa0 tx=0x7f76f4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:23.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.076+0000 7f76ff400640 1 -- 172.21.15.100:0/3979239745 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f4005c90 con 0x7f77080716b0 2024-09-17T04:25:23.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.076+0000 7f76ff400640 1 -- 172.21.15.100:0/3979239745 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76f4031e50 con 0x7f77080716b0 2024-09-17T04:25:23.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.076+0000 7f770e200640 1 -- 172.21.15.100:0/3979239745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77081e6110 con 0x7f77080716b0 2024-09-17T04:25:23.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.076+0000 7f76ff400640 1 -- 172.21.15.100:0/3979239745 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f4005260 con 0x7f77080716b0 2024-09-17T04:25:23.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.076+0000 7f770e200640 1 -- 172.21.15.100:0/3979239745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77081e5b50 con 0x7f77080716b0 2024-09-17T04:25:23.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.077+0000 7f770e200640 1 -- 172.21.15.100:0/3979239745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f770806ddb0 con 0x7f77080716b0 2024-09-17T04:25:23.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.078+0000 7f76ff400640 1 -- 172.21.15.100:0/3979239745 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f76f4005400 con 0x7f77080716b0 2024-09-17T04:25:23.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.079+0000 7f76ff400640 1 --2- 172.21.15.100:0/3979239745 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f76e8078b90 0x7f76e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:23.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.079+0000 7f76ff400640 1 -- 172.21.15.100:0/3979239745 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f76f40bbd30 con 0x7f77080716b0 2024-09-17T04:25:23.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.080+0000 7f7707400640 1 --2- 172.21.15.100:0/3979239745 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f76e8078b90 0x7f76e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:23.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.080+0000 7f7707400640 1 --2- 172.21.15.100:0/3979239745 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f76e8078b90 0x7f76e807b0a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009f20 tx=0x7f76f8005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:23.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.084+0000 7f76ff400640 1 -- 172.21.15.100:0/3979239745 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f76f403f030 con 0x7f77080716b0 2024-09-17T04:25:23.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.305+0000 7f770e200640 1 -- 172.21.15.100:0/3979239745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f77080b50e0 con 0x7f77080716b0 2024-09-17T04:25:23.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.358+0000 7f76ff400640 1 -- 172.21.15.100:0/3979239745 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) ==== 136+0+0 (secure 0 0 0) 0x7f76f4031770 con 0x7f77080716b0 2024-09-17T04:25:23.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.360+0000 7f770e200640 1 -- 172.21.15.100:0/3979239745 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f76e8078b90 msgr2=0x7f76e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:23.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.360+0000 7f770e200640 1 --2- 172.21.15.100:0/3979239745 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f76e8078b90 0x7f76e807b0a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009f20 tx=0x7f76f8005f90 comp rx=0 tx=0).stop 2024-09-17T04:25:23.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.360+0000 7f770e200640 1 -- 172.21.15.100:0/3979239745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77080716b0 msgr2=0x7f770814f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:23.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.360+0000 7f770e200640 1 --2- 172.21.15.100:0/3979239745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77080716b0 0x7f770814f210 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f76f402aaa0 tx=0x7f76f4033c20 comp rx=0 tx=0).stop 2024-09-17T04:25:23.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.360+0000 7f770e200640 1 -- 172.21.15.100:0/3979239745 shutdown_connections 2024-09-17T04:25:23.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.360+0000 7f770e200640 1 --2- 172.21.15.100:0/3979239745 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f76e8078b90 0x7f76e807b0a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:23.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.360+0000 7f770e200640 1 --2- 172.21.15.100:0/3979239745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7708152990 0x7f770806f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:23.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.360+0000 7f770e200640 1 --2- 172.21.15.100:0/3979239745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77080716b0 0x7f770814f210 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:23.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.360+0000 7f770e200640 1 --2- 172.21.15.100:0/3979239745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77080712a0 0x7f77081e6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:23.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.360+0000 7f770e200640 1 -- 172.21.15.100:0/3979239745 >> 172.21.15.100:0/3979239745 conn(0x7f7708073890 msgr2=0x7f770815e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:23.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.360+0000 7f770e200640 1 -- 172.21.15.100:0/3979239745 shutdown_connections 2024-09-17T04:25:23.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.360+0000 7f770e200640 1 -- 172.21.15.100:0/3979239745 wait complete. 2024-09-17T04:25:23.385 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T04:25:23.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.606+0000 7faa5cc00640 1 -- 172.21.15.100:0/2900966317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa3c004800 msgr2=0x7faa3c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:23.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.606+0000 7faa5cc00640 1 --2- 172.21.15.100:0/2900966317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa3c004800 0x7faa3c004c10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7faa44006f90 tx=0x7faa3c005830 comp rx=0 tx=0).stop 2024-09-17T04:25:23.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.606+0000 7faa56800640 1 -- 172.21.15.100:0/2900966317 reap_dead start 2024-09-17T04:25:23.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.606+0000 7faa5cc00640 1 -- 172.21.15.100:0/2900966317 shutdown_connections 2024-09-17T04:25:23.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.606+0000 7faa5cc00640 1 -- 172.21.15.100:0/2900966317 >> 172.21.15.100:0/2900966317 conn(0x7faa5806d3d0 msgr2=0x7faa580b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:23.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.606+0000 7faa5cc00640 1 -- 172.21.15.100:0/2900966317 shutdown_connections 2024-09-17T04:25:23.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.607+0000 7faa5cc00640 1 -- 172.21.15.100:0/2900966317 wait complete. 2024-09-17T04:25:23.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.609+0000 7faa5cc00640 1 Processor -- start 2024-09-17T04:25:23.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.609+0000 7faa5cc00640 1 -- start start 2024-09-17T04:25:23.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.610+0000 7faa5cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa3c004800 0x7faa581566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:23.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.610+0000 7faa5cc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa580b04d0 0x7faa5815b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:23.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.610+0000 7faa5cc00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa580b08e0 0x7faa5815bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:23.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.610+0000 7faa5cc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faa580cccc0 con 0x7faa3c004800 2024-09-17T04:25:23.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.610+0000 7faa5cc00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faa58157290 con 0x7faa580b04d0 2024-09-17T04:25:23.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.610+0000 7faa5cc00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faa580cce40 con 0x7faa580b08e0 2024-09-17T04:25:23.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.611+0000 7faa55e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa3c004800 0x7faa581566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:23.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.611+0000 7faa55e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa3c004800 0x7faa581566a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57256/0 (socket says 172.21.15.100:57256) 2024-09-17T04:25:23.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.611+0000 7faa55e00640 1 -- 172.21.15.100:0/739779439 learned_addr learned my addr 172.21.15.100:0/739779439 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:23.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.611+0000 7faa56800640 1 --2- 172.21.15.100:0/739779439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa580b08e0 0x7faa5815bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:23.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.611+0000 7faa55400640 1 --2- 172.21.15.100:0/739779439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa580b04d0 0x7faa5815b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:23.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.611+0000 7faa55e00640 1 -- 172.21.15.100:0/739779439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa580b08e0 msgr2=0x7faa5815bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:23.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.611+0000 7faa55e00640 1 --2- 172.21.15.100:0/739779439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa580b08e0 0x7faa5815bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:23.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.611+0000 7faa55e00640 1 -- 172.21.15.100:0/739779439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa580b04d0 msgr2=0x7faa5815b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:23.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.611+0000 7faa55e00640 1 --2- 172.21.15.100:0/739779439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa580b04d0 0x7faa5815b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:23.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.611+0000 7faa55e00640 1 -- 172.21.15.100:0/739779439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa581f89e0 con 0x7faa3c004800 2024-09-17T04:25:23.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.611+0000 7faa55e00640 1 --2- 172.21.15.100:0/739779439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa3c004800 0x7faa581566a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7faa48003cd0 tx=0x7faa48001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:23.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.611+0000 7faa4e000640 1 -- 172.21.15.100:0/739779439 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa48015070 con 0x7faa3c004800 2024-09-17T04:25:23.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.612+0000 7faa4e000640 1 -- 172.21.15.100:0/739779439 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa48011840 con 0x7faa3c004800 2024-09-17T04:25:23.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.612+0000 7faa4e000640 1 -- 172.21.15.100:0/739779439 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa48011c90 con 0x7faa3c004800 2024-09-17T04:25:23.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.612+0000 7faa5cc00640 1 -- 172.21.15.100:0/739779439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa581f8cc0 con 0x7faa3c004800 2024-09-17T04:25:23.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.612+0000 7faa5cc00640 1 -- 172.21.15.100:0/739779439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa581f9280 con 0x7faa3c004800 2024-09-17T04:25:23.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.613+0000 7faa5cc00640 1 -- 172.21.15.100:0/739779439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa580cd3d0 con 0x7faa3c004800 2024-09-17T04:25:23.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.614+0000 7faa4e000640 1 -- 172.21.15.100:0/739779439 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faa48018070 con 0x7faa3c004800 2024-09-17T04:25:23.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.615+0000 7faa4e000640 1 --2- 172.21.15.100:0/739779439 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faa38078b90 0x7faa3807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:23.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.615+0000 7faa4e000640 1 -- 172.21.15.100:0/739779439 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7faa4809b950 con 0x7faa3c004800 2024-09-17T04:25:23.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.615+0000 7faa55400640 1 --2- 172.21.15.100:0/739779439 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faa38078b90 0x7faa3807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:23.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.616+0000 7faa55400640 1 --2- 172.21.15.100:0/739779439 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faa38078b90 0x7faa3807b0a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7faa44033000 tx=0x7faa4402d000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:23.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.619+0000 7faa4e000640 1 -- 172.21.15.100:0/739779439 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7faa48063d90 con 0x7faa3c004800 2024-09-17T04:25:23.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.823+0000 7faa5cc00640 1 -- 172.21.15.100:0/739779439 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7faa581612f0 con 0x7faa38078b90 2024-09-17T04:25:23.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.826+0000 7faa4e000640 1 -- 172.21.15.100:0/739779439 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7faa581612f0 con 0x7faa38078b90 2024-09-17T04:25:23.825 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-09-17T04:25:23.825 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:23.825 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephfs" 2024-09-17T04:25:23.825 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:23.825 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-09-17T04:25:23.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.828+0000 7faa5cc00640 1 -- 172.21.15.100:0/739779439 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faa38078b90 msgr2=0x7faa3807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:23.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.828+0000 7faa5cc00640 1 --2- 172.21.15.100:0/739779439 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faa38078b90 0x7faa3807b0a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7faa44033000 tx=0x7faa4402d000 comp rx=0 tx=0).stop 2024-09-17T04:25:23.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.828+0000 7faa5cc00640 1 -- 172.21.15.100:0/739779439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa3c004800 msgr2=0x7faa581566a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:23.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.828+0000 7faa5cc00640 1 --2- 172.21.15.100:0/739779439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa3c004800 0x7faa581566a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7faa48003cd0 tx=0x7faa48001800 comp rx=0 tx=0).stop 2024-09-17T04:25:23.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.829+0000 7faa5cc00640 1 -- 172.21.15.100:0/739779439 shutdown_connections 2024-09-17T04:25:23.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.829+0000 7faa5cc00640 1 --2- 172.21.15.100:0/739779439 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faa38078b90 0x7faa3807b0a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:23.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.829+0000 7faa5cc00640 1 --2- 172.21.15.100:0/739779439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa580b08e0 0x7faa5815bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:23.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.829+0000 7faa5cc00640 1 --2- 172.21.15.100:0/739779439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa580b04d0 0x7faa5815b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:23.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.829+0000 7faa5cc00640 1 --2- 172.21.15.100:0/739779439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa3c004800 0x7faa581566a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:23.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.829+0000 7faa5cc00640 1 -- 172.21.15.100:0/739779439 >> 172.21.15.100:0/739779439 conn(0x7faa5806d3d0 msgr2=0x7faa580b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:23.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.829+0000 7faa5cc00640 1 -- 172.21.15.100:0/739779439 shutdown_connections 2024-09-17T04:25:23.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:23.829+0000 7faa5cc00640 1 -- 172.21.15.100:0/739779439 wait complete. 2024-09-17T04:25:23.854 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T04:25:24.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.079+0000 7fe18a400640 1 -- 172.21.15.100:0/4027438303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1840ce620 msgr2=0x7fe184167480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:24.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.079+0000 7fe18a400640 1 --2- 172.21.15.100:0/4027438303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1840ce620 0x7fe184167480 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe17c005fd0 tx=0x7fe17c02aa90 comp rx=0 tx=0).stop 2024-09-17T04:25:24.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.079+0000 7fe18a400640 1 -- 172.21.15.100:0/4027438303 shutdown_connections 2024-09-17T04:25:24.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.079+0000 7fe18a400640 1 --2- 172.21.15.100:0/4027438303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1840ce620 0x7fe184167480 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:24.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.079+0000 7fe18a400640 1 -- 172.21.15.100:0/4027438303 >> 172.21.15.100:0/4027438303 conn(0x7fe18406d1c0 msgr2=0x7fe18406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:24.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.079+0000 7fe18a400640 1 -- 172.21.15.100:0/4027438303 shutdown_connections 2024-09-17T04:25:24.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.079+0000 7fe18a400640 1 -- 172.21.15.100:0/4027438303 wait complete. 2024-09-17T04:25:24.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.081+0000 7fe18a400640 1 Processor -- start 2024-09-17T04:25:24.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.081+0000 7fe18a400640 1 -- start start 2024-09-17T04:25:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.083+0000 7fe18a400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1840735e0 0x7fe184130e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.083+0000 7fe18a400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1840ce620 0x7fe18415f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.083+0000 7fe18a400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe184145450 0x7fe1841435c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.083+0000 7fe18a400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe184167d50 con 0x7fe1840ce620 2024-09-17T04:25:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.083+0000 7fe18a400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe184167bd0 con 0x7fe184145450 2024-09-17T04:25:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.083+0000 7fe18a400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe184167ed0 con 0x7fe1840735e0 2024-09-17T04:25:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.083+0000 7fe17aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1840ce620 0x7fe18415f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.083+0000 7fe17aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1840ce620 0x7fe18415f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57274/0 (socket says 172.21.15.100:57274) 2024-09-17T04:25:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.083+0000 7fe17aa00640 1 -- 172.21.15.100:0/3882423361 learned_addr learned my addr 172.21.15.100:0/3882423361 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.083+0000 7fe183400640 1 --2- 172.21.15.100:0/3882423361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1840735e0 0x7fe184130e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.083+0000 7fe183e00640 1 --2- 172.21.15.100:0/3882423361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe184145450 0x7fe1841435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.083+0000 7fe17aa00640 1 -- 172.21.15.100:0/3882423361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1840735e0 msgr2=0x7fe184130e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.083+0000 7fe17aa00640 1 --2- 172.21.15.100:0/3882423361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1840735e0 0x7fe184130e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.083+0000 7fe17aa00640 1 -- 172.21.15.100:0/3882423361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe184145450 msgr2=0x7fe1841435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.083+0000 7fe17aa00640 1 --2- 172.21.15.100:0/3882423361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe184145450 0x7fe1841435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.083+0000 7fe17aa00640 1 -- 172.21.15.100:0/3882423361 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18413d690 con 0x7fe1840ce620 2024-09-17T04:25:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.083+0000 7fe183400640 1 --2- 172.21.15.100:0/3882423361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1840735e0 0x7fe184130e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:25:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.083+0000 7fe183e00640 1 --2- 172.21.15.100:0/3882423361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe184145450 0x7fe1841435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:25:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.084+0000 7fe17aa00640 1 --2- 172.21.15.100:0/3882423361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1840ce620 0x7fe18415f940 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00c390 tx=0x7fe16c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.084+0000 7fe17be00640 1 -- 172.21.15.100:0/3882423361 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c017070 con 0x7fe1840ce620 2024-09-17T04:25:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.084+0000 7fe17be00640 1 -- 172.21.15.100:0/3882423361 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe16c0035f0 con 0x7fe1840ce620 2024-09-17T04:25:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.084+0000 7fe18a400640 1 -- 172.21.15.100:0/3882423361 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe184131350 con 0x7fe1840ce620 2024-09-17T04:25:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.084+0000 7fe17be00640 1 -- 172.21.15.100:0/3882423361 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c003a20 con 0x7fe1840ce620 2024-09-17T04:25:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.084+0000 7fe18a400640 1 -- 172.21.15.100:0/3882423361 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe184131860 con 0x7fe1840ce620 2024-09-17T04:25:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.085+0000 7fe18a400640 1 -- 172.21.15.100:0/3882423361 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1841315d0 con 0x7fe1840ce620 2024-09-17T04:25:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.086+0000 7fe17be00640 1 -- 172.21.15.100:0/3882423361 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe16c01a070 con 0x7fe1840ce620 2024-09-17T04:25:24.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.087+0000 7fe17be00640 1 --2- 172.21.15.100:0/3882423361 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe160078b90 0x7fe16007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:24.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.087+0000 7fe17be00640 1 -- 172.21.15.100:0/3882423361 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fe16c09a110 con 0x7fe1840ce620 2024-09-17T04:25:24.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.088+0000 7fe183400640 1 --2- 172.21.15.100:0/3882423361 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe160078b90 0x7fe16007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:24.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.088+0000 7fe183400640 1 --2- 172.21.15.100:0/3882423361 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe160078b90 0x7fe16007b0a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe17400a830 tx=0x7fe174008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:24.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.092+0000 7fe17be00640 1 -- 172.21.15.100:0/3882423361 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe16c062540 con 0x7fe1840ce620 2024-09-17T04:25:24.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.280+0000 7fe18a400640 1 -- 172.21.15.100:0/3882423361 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe1841378f0 con 0x7fe1840ce620 2024-09-17T04:25:24.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.281+0000 7fe17be00640 1 -- 172.21.15.100:0/3882423361 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7fe16c061e40 con 0x7fe1840ce620 2024-09-17T04:25:24.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.283+0000 7fe18a400640 1 -- 172.21.15.100:0/3882423361 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe160078b90 msgr2=0x7fe16007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:24.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.283+0000 7fe18a400640 1 --2- 172.21.15.100:0/3882423361 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe160078b90 0x7fe16007b0a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe17400a830 tx=0x7fe174008040 comp rx=0 tx=0).stop 2024-09-17T04:25:24.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.283+0000 7fe18a400640 1 -- 172.21.15.100:0/3882423361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1840ce620 msgr2=0x7fe18415f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:24.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.283+0000 7fe18a400640 1 --2- 172.21.15.100:0/3882423361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1840ce620 0x7fe18415f940 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00c390 tx=0x7fe16c00c850 comp rx=0 tx=0).stop 2024-09-17T04:25:24.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.283+0000 7fe18a400640 1 -- 172.21.15.100:0/3882423361 shutdown_connections 2024-09-17T04:25:24.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.283+0000 7fe18a400640 1 --2- 172.21.15.100:0/3882423361 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe160078b90 0x7fe16007b0a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:24.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.283+0000 7fe18a400640 1 --2- 172.21.15.100:0/3882423361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe184145450 0x7fe1841435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:24.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.283+0000 7fe18a400640 1 --2- 172.21.15.100:0/3882423361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1840ce620 0x7fe18415f940 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:24.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.283+0000 7fe18a400640 1 --2- 172.21.15.100:0/3882423361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1840735e0 0x7fe184130e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:24.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.283+0000 7fe18a400640 1 -- 172.21.15.100:0/3882423361 >> 172.21.15.100:0/3882423361 conn(0x7fe18406d1c0 msgr2=0x7fe184137e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:24.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.283+0000 7fe18a400640 1 -- 172.21.15.100:0/3882423361 shutdown_connections 2024-09-17T04:25:24.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.283+0000 7fe18a400640 1 -- 172.21.15.100:0/3882423361 wait complete. 2024-09-17T04:25:24.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:24.305 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp99240/subvol57930) 2024-09-17T04:25:24.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:24.371 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-17T04:25:24.409 INFO:teuthology.orchestra.run.smithi100.stdout:cephfs_data 2024-09-17T04:25:24.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:24.409 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp99240/subvol57930) 2024-09-17T04:25:24.521 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:25:24.521 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp99240/subvol57930/.meta 2024-09-17T04:25:24.582 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol57930 subvol_grp99240 2024-09-17T04:25:24.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.848+0000 7f281e400640 1 -- 172.21.15.100:0/2463709373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f281806ee80 msgr2=0x7f281806f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:24.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.848+0000 7f281e400640 1 --2- 172.21.15.100:0/2463709373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f281806ee80 0x7f281806f290 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2810005fd0 tx=0x7f281002a940 comp rx=0 tx=0).stop 2024-09-17T04:25:24.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.849+0000 7f281e400640 1 -- 172.21.15.100:0/2463709373 shutdown_connections 2024-09-17T04:25:24.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.849+0000 7f281e400640 1 --2- 172.21.15.100:0/2463709373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f281806ee80 0x7f281806f290 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:24.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.849+0000 7f281e400640 1 -- 172.21.15.100:0/2463709373 >> 172.21.15.100:0/2463709373 conn(0x7f28180b1dd0 msgr2=0x7f28180b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:24.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.849+0000 7f281e400640 1 -- 172.21.15.100:0/2463709373 shutdown_connections 2024-09-17T04:25:24.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.849+0000 7f281e400640 1 -- 172.21.15.100:0/2463709373 wait complete. 2024-09-17T04:25:24.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.851+0000 7f281e400640 1 Processor -- start 2024-09-17T04:25:24.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.851+0000 7f281e400640 1 -- start start 2024-09-17T04:25:24.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.852+0000 7f281e400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f281806ee80 0x7f281814a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:24.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.852+0000 7f281e400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28181558c0 0x7f28180c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:24.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.852+0000 7f281e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2818162410 0x7f28180c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:24.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.852+0000 7f281e400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f281806fb60 con 0x7f2818162410 2024-09-17T04:25:24.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.852+0000 7f281e400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f281806f9e0 con 0x7f281806ee80 2024-09-17T04:25:24.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.852+0000 7f281e400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f281806fce0 con 0x7f28181558c0 2024-09-17T04:25:24.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.853+0000 7f281da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2818162410 0x7f28180c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:24.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.853+0000 7f281da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2818162410 0x7f28180c97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48478/0 (socket says 172.21.15.100:48478) 2024-09-17T04:25:24.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.853+0000 7f281da00640 1 -- 172.21.15.100:0/2725005906 learned_addr learned my addr 172.21.15.100:0/2725005906 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:24.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.853+0000 7f2817e00640 1 --2- 172.21.15.100:0/2725005906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28181558c0 0x7f28180c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:24.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.853+0000 7f281d000640 1 --2- 172.21.15.100:0/2725005906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f281806ee80 0x7f281814a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:24.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.853+0000 7f281da00640 1 -- 172.21.15.100:0/2725005906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28181558c0 msgr2=0x7f28180c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:24.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.853+0000 7f281da00640 1 --2- 172.21.15.100:0/2725005906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28181558c0 0x7f28180c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:24.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.853+0000 7f281da00640 1 -- 172.21.15.100:0/2725005906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f281806ee80 msgr2=0x7f281814a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:24.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.853+0000 7f281da00640 1 --2- 172.21.15.100:0/2725005906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f281806ee80 0x7f281814a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:24.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.853+0000 7f281da00640 1 -- 172.21.15.100:0/2725005906 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281814e0e0 con 0x7f2818162410 2024-09-17T04:25:24.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.854+0000 7f281da00640 1 --2- 172.21.15.100:0/2725005906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2818162410 0x7f28180c97f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f281002af30 tx=0x7f2810031ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:24.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.854+0000 7f2814c00640 1 -- 172.21.15.100:0/2725005906 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281003e070 con 0x7f2818162410 2024-09-17T04:25:24.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.854+0000 7f2814c00640 1 -- 172.21.15.100:0/2725005906 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2810004560 con 0x7f2818162410 2024-09-17T04:25:24.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.854+0000 7f281e400640 1 -- 172.21.15.100:0/2725005906 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28180cbff0 con 0x7f2818162410 2024-09-17T04:25:24.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.854+0000 7f2814c00640 1 -- 172.21.15.100:0/2725005906 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810043460 con 0x7f2818162410 2024-09-17T04:25:24.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.854+0000 7f281e400640 1 -- 172.21.15.100:0/2725005906 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28181f6ef0 con 0x7f2818162410 2024-09-17T04:25:24.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.855+0000 7f281e400640 1 -- 172.21.15.100:0/2725005906 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f281806f290 con 0x7f2818162410 2024-09-17T04:25:24.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.856+0000 7f2814c00640 1 -- 172.21.15.100:0/2725005906 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f281003a440 con 0x7f2818162410 2024-09-17T04:25:24.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.858+0000 7f2814c00640 1 --2- 172.21.15.100:0/2725005906 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f27f8078bc0 0x7f27f807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:24.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.858+0000 7f2814c00640 1 -- 172.21.15.100:0/2725005906 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f28100bc8f0 con 0x7f2818162410 2024-09-17T04:25:24.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.858+0000 7f281d000640 1 --2- 172.21.15.100:0/2725005906 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f27f8078bc0 0x7f27f807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:24.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.859+0000 7f281d000640 1 --2- 172.21.15.100:0/2725005906 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f27f8078bc0 0x7f27f807b0d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f280c0046b0 tx=0x7f280c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:24.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:24.863+0000 7f2814c00640 1 -- 172.21.15.100:0/2725005906 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f2810084c90 con 0x7f2818162410 2024-09-17T04:25:25.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.072+0000 7f281e400640 1 -- 172.21.15.100:0/2725005906 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol57930", "group_name": "subvol_grp99240", "target": ["mon-mgr", ""]}) -- 0x7f281815e8f0 con 0x7f27f8078bc0 2024-09-17T04:25:25.080 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:25:25.080+0000 7fa165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:25:25.080 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:25:25.080+0000 7fa165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:25:25.080 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:25:25.080+0000 7fa165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:25:25.080 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:25:25.080+0000 7fa165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:25:25.080 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:25:25.080+0000 7fa165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:25:25.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.126+0000 7f2814c00640 1 -- 172.21.15.100:0/2725005906 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f281815e8f0 con 0x7f27f8078bc0 2024-09-17T04:25:25.125 INFO:teuthology.orchestra.run.smithi100.stdout:/volumes/subvol_grp99240/subvol57930 2024-09-17T04:25:25.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.128+0000 7f281e400640 1 -- 172.21.15.100:0/2725005906 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f27f8078bc0 msgr2=0x7f27f807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:25.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.128+0000 7f281e400640 1 --2- 172.21.15.100:0/2725005906 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f27f8078bc0 0x7f27f807b0d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f280c0046b0 tx=0x7f280c009040 comp rx=0 tx=0).stop 2024-09-17T04:25:25.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.128+0000 7f281e400640 1 -- 172.21.15.100:0/2725005906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2818162410 msgr2=0x7f28180c97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:25.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.128+0000 7f281e400640 1 --2- 172.21.15.100:0/2725005906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2818162410 0x7f28180c97f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f281002af30 tx=0x7f2810031ea0 comp rx=0 tx=0).stop 2024-09-17T04:25:25.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.128+0000 7f281e400640 1 -- 172.21.15.100:0/2725005906 shutdown_connections 2024-09-17T04:25:25.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.128+0000 7f281e400640 1 --2- 172.21.15.100:0/2725005906 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f27f8078bc0 0x7f27f807b0d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:25.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.128+0000 7f281e400640 1 --2- 172.21.15.100:0/2725005906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2818162410 0x7f28180c97f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:25.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.128+0000 7f281e400640 1 --2- 172.21.15.100:0/2725005906 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28181558c0 0x7f28180c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:25.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.128+0000 7f281e400640 1 --2- 172.21.15.100:0/2725005906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f281806ee80 0x7f281814a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:25.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.128+0000 7f281e400640 1 -- 172.21.15.100:0/2725005906 >> 172.21.15.100:0/2725005906 conn(0x7f28180b1dd0 msgr2=0x7f2818072e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:25.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.129+0000 7f281e400640 1 -- 172.21.15.100:0/2725005906 shutdown_connections 2024-09-17T04:25:25.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.129+0000 7f281e400640 1 -- 172.21.15.100:0/2725005906 wait complete. 2024-09-17T04:25:25.153 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol57930 subvol_grp99240 2024-09-17T04:25:25.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.379+0000 7fe73b200640 1 -- 172.21.15.100:0/1976483514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe720004800 msgr2=0x7fe720004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:25.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.379+0000 7fe73b200640 1 --2- 172.21.15.100:0/1976483514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe720004800 0x7fe720004c10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe7280016c0 tx=0x7fe720005830 comp rx=0 tx=0).stop 2024-09-17T04:25:25.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.379+0000 7fe73b200640 1 -- 172.21.15.100:0/1976483514 shutdown_connections 2024-09-17T04:25:25.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.379+0000 7fe73b200640 1 --2- 172.21.15.100:0/1976483514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe720004800 0x7fe720004c10 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:25.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.379+0000 7fe73b200640 1 -- 172.21.15.100:0/1976483514 >> 172.21.15.100:0/1976483514 conn(0x7fe73c0b1dd0 msgr2=0x7fe73c0b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:25.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.379+0000 7fe73b200640 1 -- 172.21.15.100:0/1976483514 shutdown_connections 2024-09-17T04:25:25.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.379+0000 7fe73b200640 1 -- 172.21.15.100:0/1976483514 wait complete. 2024-09-17T04:25:25.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.382+0000 7fe73b200640 1 Processor -- start 2024-09-17T04:25:25.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.382+0000 7fe73b200640 1 -- start start 2024-09-17T04:25:25.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.383+0000 7fe73b200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe720004800 0x7fe73c159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:25.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.383+0000 7fe73b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe73c1596c0 0x7fe73c1f5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:25.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.383+0000 7fe73b200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe73c156e30 0x7fe73c1f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:25.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.383+0000 7fe73b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c153460 con 0x7fe73c1596c0 2024-09-17T04:25:25.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.383+0000 7fe73b200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c1532e0 con 0x7fe720004800 2024-09-17T04:25:25.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.383+0000 7fe73b200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe73c1535e0 con 0x7fe73c156e30 2024-09-17T04:25:25.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.384+0000 7fe739400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe73c1596c0 0x7fe73c1f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:25.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.384+0000 7fe739400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe73c1596c0 0x7fe73c1f5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48500/0 (socket says 172.21.15.100:48500) 2024-09-17T04:25:25.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.384+0000 7fe739400640 1 -- 172.21.15.100:0/2404975985 learned_addr learned my addr 172.21.15.100:0/2404975985 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:25.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.384+0000 7fe739e00640 1 --2- 172.21.15.100:0/2404975985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe720004800 0x7fe73c159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:25.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.384+0000 7fe73a800640 1 --2- 172.21.15.100:0/2404975985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe73c156e30 0x7fe73c1f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:25.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.384+0000 7fe739400640 1 -- 172.21.15.100:0/2404975985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe73c156e30 msgr2=0x7fe73c1f5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:25.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.384+0000 7fe739400640 1 --2- 172.21.15.100:0/2404975985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe73c156e30 0x7fe73c1f5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:25.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.384+0000 7fe739400640 1 -- 172.21.15.100:0/2404975985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe720004800 msgr2=0x7fe73c159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:25.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.384+0000 7fe739400640 1 --2- 172.21.15.100:0/2404975985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe720004800 0x7fe73c159180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:25.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.384+0000 7fe739400640 1 -- 172.21.15.100:0/2404975985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73c1f9360 con 0x7fe73c1596c0 2024-09-17T04:25:25.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.385+0000 7fe739400640 1 --2- 172.21.15.100:0/2404975985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe73c1596c0 0x7fe73c1f5590 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe7280017e0 tx=0x7fe7280063a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:25.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.385+0000 7fe732000640 1 -- 172.21.15.100:0/2404975985 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe728042070 con 0x7fe73c1596c0 2024-09-17T04:25:25.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.385+0000 7fe732000640 1 -- 172.21.15.100:0/2404975985 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe728039070 con 0x7fe73c1596c0 2024-09-17T04:25:25.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.385+0000 7fe732000640 1 -- 172.21.15.100:0/2404975985 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe728003d90 con 0x7fe73c1596c0 2024-09-17T04:25:25.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.385+0000 7fe73b200640 1 -- 172.21.15.100:0/2404975985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe73c15a010 con 0x7fe73c1596c0 2024-09-17T04:25:25.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.385+0000 7fe73b200640 1 -- 172.21.15.100:0/2404975985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe73c1f5070 con 0x7fe73c1596c0 2024-09-17T04:25:25.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.386+0000 7fe73b200640 1 -- 172.21.15.100:0/2404975985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe73c0008d0 con 0x7fe73c1596c0 2024-09-17T04:25:25.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.387+0000 7fe732000640 1 -- 172.21.15.100:0/2404975985 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe72803b070 con 0x7fe73c1596c0 2024-09-17T04:25:25.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.388+0000 7fe732000640 1 --2- 172.21.15.100:0/2404975985 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe71c078b90 0x7fe71c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:25.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.388+0000 7fe732000640 1 -- 172.21.15.100:0/2404975985 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fe7280ba2d0 con 0x7fe73c1596c0 2024-09-17T04:25:25.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.389+0000 7fe739e00640 1 --2- 172.21.15.100:0/2404975985 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe71c078b90 0x7fe71c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:25.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.389+0000 7fe739e00640 1 --2- 172.21.15.100:0/2404975985 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe71c078b90 0x7fe71c07b0a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe73c052860 tx=0x7fe72c005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:25.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.393+0000 7fe732000640 1 -- 172.21.15.100:0/2404975985 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe728082020 con 0x7fe73c1596c0 2024-09-17T04:25:25.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.602+0000 7fe73b200640 1 -- 172.21.15.100:0/2404975985 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol57930", "group_name": "subvol_grp99240", "target": ["mon-mgr", ""]}) -- 0x7fe73c06ddf0 con 0x7fe71c078b90 2024-09-17T04:25:25.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.607+0000 7fe732000640 1 -- 172.21.15.100:0/2404975985 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe73c06ddf0 con 0x7fe71c078b90 2024-09-17T04:25:25.606 INFO:teuthology.orchestra.run.smithi100.stdout:/volumes/subvol_grp99240/subvol57930 2024-09-17T04:25:25.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.609+0000 7fe73b200640 1 -- 172.21.15.100:0/2404975985 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe71c078b90 msgr2=0x7fe71c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:25.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.609+0000 7fe73b200640 1 --2- 172.21.15.100:0/2404975985 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe71c078b90 0x7fe71c07b0a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe73c052860 tx=0x7fe72c005f90 comp rx=0 tx=0).stop 2024-09-17T04:25:25.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.609+0000 7fe73b200640 1 -- 172.21.15.100:0/2404975985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe73c1596c0 msgr2=0x7fe73c1f5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:25.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.609+0000 7fe73b200640 1 --2- 172.21.15.100:0/2404975985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe73c1596c0 0x7fe73c1f5590 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe7280017e0 tx=0x7fe7280063a0 comp rx=0 tx=0).stop 2024-09-17T04:25:25.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.609+0000 7fe73b200640 1 -- 172.21.15.100:0/2404975985 shutdown_connections 2024-09-17T04:25:25.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.609+0000 7fe73b200640 1 --2- 172.21.15.100:0/2404975985 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe71c078b90 0x7fe71c07b0a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:25.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.609+0000 7fe73b200640 1 --2- 172.21.15.100:0/2404975985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe73c156e30 0x7fe73c1f5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:25.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.609+0000 7fe73b200640 1 --2- 172.21.15.100:0/2404975985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe73c1596c0 0x7fe73c1f5590 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:25.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.609+0000 7fe73b200640 1 --2- 172.21.15.100:0/2404975985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe720004800 0x7fe73c159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:25.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.609+0000 7fe73b200640 1 -- 172.21.15.100:0/2404975985 >> 172.21.15.100:0/2404975985 conn(0x7fe73c0b1dd0 msgr2=0x7fe73c0b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:25.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.609+0000 7fe73b200640 1 -- 172.21.15.100:0/2404975985 shutdown_connections 2024-09-17T04:25:25.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.609+0000 7fe73b200640 1 -- 172.21.15.100:0/2404975985 wait complete. 2024-09-17T04:25:25.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:25.631 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/214aff6b7935b391a7934b39c65b5fef.meta) 2024-09-17T04:25:25.692 INFO:teuthology.orchestra.run.smithi100.stdout:[GLOBAL] 2024-09-17T04:25:25.692 INFO:teuthology.orchestra.run.smithi100.stdout:version = 1 2024-09-17T04:25:25.692 INFO:teuthology.orchestra.run.smithi100.stdout:type = subvolume 2024-09-17T04:25:25.693 INFO:teuthology.orchestra.run.smithi100.stdout:path = /volumes/subvol_grp99240/subvol57930 2024-09-17T04:25:25.693 INFO:teuthology.orchestra.run.smithi100.stdout:state = complete 2024-09-17T04:25:25.693 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:25.695 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol57930 subvol_grp99240 2024-09-17T04:25:25.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.920+0000 7fbc9b000640 1 -- 172.21.15.100:0/2091312819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc940af100 msgr2=0x7fbc940af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:25.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.920+0000 7fbc9b000640 1 --2- 172.21.15.100:0/2091312819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc940af100 0x7fbc940af510 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c005fd0 tx=0x7fbc8c02aaa0 comp rx=0 tx=0).stop 2024-09-17T04:25:25.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.921+0000 7fbc9b000640 1 -- 172.21.15.100:0/2091312819 shutdown_connections 2024-09-17T04:25:25.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.921+0000 7fbc9b000640 1 --2- 172.21.15.100:0/2091312819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc940af100 0x7fbc940af510 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:25.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.921+0000 7fbc9b000640 1 --2- 172.21.15.100:0/2091312819 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc9406d4b0 0x7fbc940b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:25.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.921+0000 7fbc9b000640 1 -- 172.21.15.100:0/2091312819 >> 172.21.15.100:0/2091312819 conn(0x7fbc940b2540 msgr2=0x7fbc940d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:25.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.921+0000 7fbc9b000640 1 -- 172.21.15.100:0/2091312819 shutdown_connections 2024-09-17T04:25:25.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.921+0000 7fbc9b000640 1 -- 172.21.15.100:0/2091312819 wait complete. 2024-09-17T04:25:25.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.923+0000 7fbc9b000640 1 Processor -- start 2024-09-17T04:25:25.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.923+0000 7fbc9b000640 1 -- start start 2024-09-17T04:25:25.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.924+0000 7fbc9b000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc940af100 0x7fbc94153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:25.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.924+0000 7fbc9b000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc940cc830 0x7fbc941f6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:25.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.924+0000 7fbc9b000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc940ae790 0x7fbc9415cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:25.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.924+0000 7fbc9b000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbc940b35f0 con 0x7fbc940cc830 2024-09-17T04:25:25.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.924+0000 7fbc9b000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbc940b3470 con 0x7fbc940ae790 2024-09-17T04:25:25.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.925+0000 7fbc9b000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbc940b3770 con 0x7fbc940af100 2024-09-17T04:25:25.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.925+0000 7fbc93400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc940cc830 0x7fbc941f6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:25.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.925+0000 7fbc93400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc940cc830 0x7fbc941f6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48504/0 (socket says 172.21.15.100:48504) 2024-09-17T04:25:25.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.925+0000 7fbc93400640 1 -- 172.21.15.100:0/1836195136 learned_addr learned my addr 172.21.15.100:0/1836195136 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:25.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.925+0000 7fbc99000640 1 --2- 172.21.15.100:0/1836195136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc940ae790 0x7fbc9415cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:25.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.925+0000 7fbc93e00640 1 --2- 172.21.15.100:0/1836195136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc940af100 0x7fbc94153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:25.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.925+0000 7fbc93400640 1 -- 172.21.15.100:0/1836195136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc940af100 msgr2=0x7fbc94153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:25.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.925+0000 7fbc93400640 1 --2- 172.21.15.100:0/1836195136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc940af100 0x7fbc94153480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:25.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.925+0000 7fbc93400640 1 -- 172.21.15.100:0/1836195136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc940ae790 msgr2=0x7fbc9415cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:25.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.925+0000 7fbc93400640 1 --2- 172.21.15.100:0/1836195136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc940ae790 0x7fbc9415cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:25.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.925+0000 7fbc93400640 1 -- 172.21.15.100:0/1836195136 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc941f5690 con 0x7fbc940cc830 2024-09-17T04:25:25.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.925+0000 7fbc99000640 1 --2- 172.21.15.100:0/1836195136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc940ae790 0x7fbc9415cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:25:25.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.926+0000 7fbc93400640 1 --2- 172.21.15.100:0/1836195136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc940cc830 0x7fbc941f6150 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbc8000e940 tx=0x7fbc8000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:25.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.926+0000 7fbc8be00640 1 -- 172.21.15.100:0/1836195136 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8000cd80 con 0x7fbc940cc830 2024-09-17T04:25:25.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.926+0000 7fbc8be00640 1 -- 172.21.15.100:0/1836195136 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc80010430 con 0x7fbc940cc830 2024-09-17T04:25:25.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.926+0000 7fbc9b000640 1 -- 172.21.15.100:0/1836195136 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc9415f960 con 0x7fbc940cc830 2024-09-17T04:25:25.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.926+0000 7fbc8be00640 1 -- 172.21.15.100:0/1836195136 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc80010880 con 0x7fbc940cc830 2024-09-17T04:25:25.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.926+0000 7fbc9b000640 1 -- 172.21.15.100:0/1836195136 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc941f5c30 con 0x7fbc940cc830 2024-09-17T04:25:25.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.927+0000 7fbc9b000640 1 -- 172.21.15.100:0/1836195136 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc940b3c90 con 0x7fbc940cc830 2024-09-17T04:25:25.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.928+0000 7fbc8be00640 1 -- 172.21.15.100:0/1836195136 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbc8001f020 con 0x7fbc940cc830 2024-09-17T04:25:25.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.929+0000 7fbc8be00640 1 --2- 172.21.15.100:0/1836195136 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbc74078b90 0x7fbc7407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:25.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.929+0000 7fbc8be00640 1 -- 172.21.15.100:0/1836195136 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fbc8009b540 con 0x7fbc940cc830 2024-09-17T04:25:25.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.930+0000 7fbc93e00640 1 --2- 172.21.15.100:0/1836195136 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbc74078b90 0x7fbc7407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:25.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.930+0000 7fbc93e00640 1 --2- 172.21.15.100:0/1836195136 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbc74078b90 0x7fbc7407b0a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbc84001720 tx=0x7fbc84009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:25.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:25.934+0000 7fbc8be00640 1 -- 172.21.15.100:0/1836195136 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fbc80063210 con 0x7fbc940cc830 2024-09-17T04:25:26.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.043+0000 7fbc8be00640 1 -- 172.21.15.100:0/1836195136 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 6) ==== 101492+0+0 (secure 0 0 0) 0x7fbc8005f480 con 0x7fbc940cc830 2024-09-17T04:25:26.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.148+0000 7fbc9b000640 1 -- 172.21.15.100:0/1836195136 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol57930", "group_name": "subvol_grp99240", "target": ["mon-mgr", ""]}) -- 0x7fbc940cfe20 con 0x7fbc74078b90 2024-09-17T04:25:26.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.178+0000 7fbc8be00640 1 -- 172.21.15.100:0/1836195136 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbc940cfe20 con 0x7fbc74078b90 2024-09-17T04:25:26.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.181+0000 7fbc9b000640 1 -- 172.21.15.100:0/1836195136 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbc74078b90 msgr2=0x7fbc7407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:26.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.181+0000 7fbc9b000640 1 --2- 172.21.15.100:0/1836195136 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbc74078b90 0x7fbc7407b0a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbc84001720 tx=0x7fbc84009040 comp rx=0 tx=0).stop 2024-09-17T04:25:26.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.181+0000 7fbc9b000640 1 -- 172.21.15.100:0/1836195136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc940cc830 msgr2=0x7fbc941f6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:26.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.181+0000 7fbc9b000640 1 --2- 172.21.15.100:0/1836195136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc940cc830 0x7fbc941f6150 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbc8000e940 tx=0x7fbc8000ee00 comp rx=0 tx=0).stop 2024-09-17T04:25:26.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.181+0000 7fbc9b000640 1 -- 172.21.15.100:0/1836195136 shutdown_connections 2024-09-17T04:25:26.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.181+0000 7fbc9b000640 1 --2- 172.21.15.100:0/1836195136 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbc74078b90 0x7fbc7407b0a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:26.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.181+0000 7fbc9b000640 1 --2- 172.21.15.100:0/1836195136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc940ae790 0x7fbc9415cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:26.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.181+0000 7fbc9b000640 1 --2- 172.21.15.100:0/1836195136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc940cc830 0x7fbc941f6150 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:26.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.181+0000 7fbc9b000640 1 --2- 172.21.15.100:0/1836195136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc940af100 0x7fbc94153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:26.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.181+0000 7fbc9b000640 1 -- 172.21.15.100:0/1836195136 >> 172.21.15.100:0/1836195136 conn(0x7fbc940b2540 msgr2=0x7fbc94099820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:26.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.181+0000 7fbc9b000640 1 -- 172.21.15.100:0/1836195136 shutdown_connections 2024-09-17T04:25:26.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.181+0000 7fbc9b000640 1 -- 172.21.15.100:0/1836195136 wait complete. 2024-09-17T04:25:26.182 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:25:26.183+0000 7fa169a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:25:26.182 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:25:26.183+0000 7fa169a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:25:26.182 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:25:26.183+0000 7fa169a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:25:26.182 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:25:26.183+0000 7fa169a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:25:26.182 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:25:26.183+0000 7fa169a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:25:26.201 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:25:26.201+0000 7fa168600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:25:26.201 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:25:26.201+0000 7fa168600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:25:26.201 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:25:26.201+0000 7fa168600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:25:26.201 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:25:26.201+0000 7fa168600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:25:26.201 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:25:26.201+0000 7fa168600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:25:26.204 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-17T04:25:26.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:25:26.290 INFO:teuthology.orchestra.run.smithi100.stdout:2 2024-09-17T04:25:26.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:25:26.298 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp99240 2024-09-17T04:25:26.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.521+0000 7f8f8da00640 1 -- 172.21.15.100:0/3751069255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f880b98b0 msgr2=0x7f8f880b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:26.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.521+0000 7f8f8da00640 1 --2- 172.21.15.100:0/3751069255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f880b98b0 0x7f8f880b4770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8f78009850 tx=0x7f8f7802a5a0 comp rx=0 tx=0).stop 2024-09-17T04:25:26.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.521+0000 7f8f8da00640 1 -- 172.21.15.100:0/3751069255 shutdown_connections 2024-09-17T04:25:26.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.521+0000 7f8f8da00640 1 --2- 172.21.15.100:0/3751069255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f880b98b0 0x7f8f880b4770 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:26.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.521+0000 7f8f8da00640 1 -- 172.21.15.100:0/3751069255 >> 172.21.15.100:0/3751069255 conn(0x7f8f8808bfe0 msgr2=0x7f8f880d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:26.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.521+0000 7f8f8da00640 1 -- 172.21.15.100:0/3751069255 shutdown_connections 2024-09-17T04:25:26.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.521+0000 7f8f8da00640 1 -- 172.21.15.100:0/3751069255 wait complete. 2024-09-17T04:25:26.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.523+0000 7f8f8da00640 1 Processor -- start 2024-09-17T04:25:26.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.524+0000 7f8f8da00640 1 -- start start 2024-09-17T04:25:26.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.525+0000 7f8f8da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f880b4cb0 0x7f8f88155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.525+0000 7f8f8da00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f880b98b0 0x7f8f88160690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.525+0000 7f8f8da00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f880cf870 0x7f8f880b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.525+0000 7f8f8da00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8f880d1690 con 0x7f8f880b4cb0 2024-09-17T04:25:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.525+0000 7f8f8da00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8f880d1510 con 0x7f8f880b98b0 2024-09-17T04:25:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.525+0000 7f8f8da00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8f880d1810 con 0x7f8f880cf870 2024-09-17T04:25:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.525+0000 7f8f87e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f880b4cb0 0x7f8f88155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.525+0000 7f8f87e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f880b4cb0 0x7f8f88155a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48520/0 (socket says 172.21.15.100:48520) 2024-09-17T04:25:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.525+0000 7f8f87e00640 1 -- 172.21.15.100:0/3722324224 learned_addr learned my addr 172.21.15.100:0/3722324224 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.525+0000 7f8f7f400640 1 --2- 172.21.15.100:0/3722324224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f880b98b0 0x7f8f88160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.525+0000 7f8f8d000640 1 --2- 172.21.15.100:0/3722324224 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f880cf870 0x7f8f880b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:26.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.526+0000 7f8f87e00640 1 -- 172.21.15.100:0/3722324224 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f880cf870 msgr2=0x7f8f880b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:26.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.526+0000 7f8f87e00640 1 --2- 172.21.15.100:0/3722324224 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f880cf870 0x7f8f880b1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:26.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.526+0000 7f8f87e00640 1 -- 172.21.15.100:0/3722324224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f880b98b0 msgr2=0x7f8f88160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:26.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.526+0000 7f8f87e00640 1 --2- 172.21.15.100:0/3722324224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f880b98b0 0x7f8f88160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:26.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.526+0000 7f8f87e00640 1 -- 172.21.15.100:0/3722324224 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f88153b40 con 0x7f8f880b4cb0 2024-09-17T04:25:26.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.526+0000 7f8f87e00640 1 --2- 172.21.15.100:0/3722324224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f880b4cb0 0x7f8f88155a70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8f7802ab90 tx=0x7f8f78033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:26.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.526+0000 7f8f84c00640 1 -- 172.21.15.100:0/3722324224 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f78005c30 con 0x7f8f880b4cb0 2024-09-17T04:25:26.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.526+0000 7f8f84c00640 1 -- 172.21.15.100:0/3722324224 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f78031e50 con 0x7f8f880b4cb0 2024-09-17T04:25:26.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.526+0000 7f8f84c00640 1 -- 172.21.15.100:0/3722324224 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f78033310 con 0x7f8f880b4cb0 2024-09-17T04:25:26.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.526+0000 7f8f8da00640 1 -- 172.21.15.100:0/3722324224 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f88161f00 con 0x7f8f880b4cb0 2024-09-17T04:25:26.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.527+0000 7f8f8da00640 1 -- 172.21.15.100:0/3722324224 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f88157910 con 0x7f8f880b4cb0 2024-09-17T04:25:26.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.528+0000 7f8f8da00640 1 -- 172.21.15.100:0/3722324224 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f880b4770 con 0x7f8f880b4cb0 2024-09-17T04:25:26.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.529+0000 7f8f84c00640 1 -- 172.21.15.100:0/3722324224 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 6) ==== 101492+0+0 (secure 0 0 0) 0x7f8f780334b0 con 0x7f8f880b4cb0 2024-09-17T04:25:26.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.530+0000 7f8f84c00640 1 --2- 172.21.15.100:0/3722324224 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8f64078c20 0x7f8f6407b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:26.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.530+0000 7f8f84c00640 1 -- 172.21.15.100:0/3722324224 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f8f780bc080 con 0x7f8f880b4cb0 2024-09-17T04:25:26.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.530+0000 7f8f7f400640 1 --2- 172.21.15.100:0/3722324224 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8f64078c20 0x7f8f6407b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:26.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.531+0000 7f8f7f400640 1 --2- 172.21.15.100:0/3722324224 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8f64078c20 0x7f8f6407b130 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8f70007fd0 tx=0x7f8f70007eb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:26.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.535+0000 7f8f84c00640 1 -- 172.21.15.100:0/3722324224 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8f7803f030 con 0x7f8f880b4cb0 2024-09-17T04:25:26.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.741+0000 7f8f8da00640 1 -- 172.21.15.100:0/3722324224 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp99240", "target": ["mon-mgr", ""]}) -- 0x7f8f880d4e10 con 0x7f8f64078c20 2024-09-17T04:25:26.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.746+0000 7f8f84c00640 1 -- 172.21.15.100:0/3722324224 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8f880d4e10 con 0x7f8f64078c20 2024-09-17T04:25:26.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.748+0000 7f8f8da00640 1 -- 172.21.15.100:0/3722324224 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8f64078c20 msgr2=0x7f8f6407b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:26.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.748+0000 7f8f8da00640 1 --2- 172.21.15.100:0/3722324224 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8f64078c20 0x7f8f6407b130 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8f70007fd0 tx=0x7f8f70007eb0 comp rx=0 tx=0).stop 2024-09-17T04:25:26.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.748+0000 7f8f8da00640 1 -- 172.21.15.100:0/3722324224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f880b4cb0 msgr2=0x7f8f88155a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:26.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.748+0000 7f8f8da00640 1 --2- 172.21.15.100:0/3722324224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f880b4cb0 0x7f8f88155a70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8f7802ab90 tx=0x7f8f78033c20 comp rx=0 tx=0).stop 2024-09-17T04:25:26.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.748+0000 7f8f8da00640 1 -- 172.21.15.100:0/3722324224 shutdown_connections 2024-09-17T04:25:26.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.748+0000 7f8f8da00640 1 --2- 172.21.15.100:0/3722324224 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8f64078c20 0x7f8f6407b130 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:26.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.748+0000 7f8f8da00640 1 --2- 172.21.15.100:0/3722324224 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f880cf870 0x7f8f880b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:26.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.748+0000 7f8f8da00640 1 --2- 172.21.15.100:0/3722324224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f880b98b0 0x7f8f88160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:26.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.748+0000 7f8f8da00640 1 --2- 172.21.15.100:0/3722324224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f880b4cb0 0x7f8f88155a70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:26.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.748+0000 7f8f8da00640 1 -- 172.21.15.100:0/3722324224 >> 172.21.15.100:0/3722324224 conn(0x7f8f8808bfe0 msgr2=0x7f8f88162ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:26.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.749+0000 7f8f8da00640 1 -- 172.21.15.100:0/3722324224 shutdown_connections 2024-09-17T04:25:26.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:26.749+0000 7f8f8da00640 1 -- 172.21.15.100:0/3722324224 wait complete. 2024-09-17T04:25:26.769 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T04:25:26.769 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:25:26.850 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:25:26.944 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T04:25:26.944 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:25:26.944 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:25:26.944 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:25:26.979 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:25:26.980 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:25:27.055 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:25:27.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.316+0000 7f9051200640 1 -- 172.21.15.100:0/4042377226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f904c0705b0 msgr2=0x7f904c0709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:27.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.316+0000 7f9051200640 1 --2- 172.21.15.100:0/4042377226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f904c0705b0 0x7f904c0709c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9034009830 tx=0x7f903402a480 comp rx=0 tx=0).stop 2024-09-17T04:25:27.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.316+0000 7f9051200640 1 -- 172.21.15.100:0/4042377226 shutdown_connections 2024-09-17T04:25:27.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.316+0000 7f9051200640 1 --2- 172.21.15.100:0/4042377226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f904c0705b0 0x7f904c0709c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:27.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.316+0000 7f9051200640 1 --2- 172.21.15.100:0/4042377226 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f904c0cfc80 0x7f904c156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:27.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.316+0000 7f9051200640 1 -- 172.21.15.100:0/4042377226 >> 172.21.15.100:0/4042377226 conn(0x7f904c06d3d0 msgr2=0x7f904c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:27.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.316+0000 7f9051200640 1 -- 172.21.15.100:0/4042377226 shutdown_connections 2024-09-17T04:25:27.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.316+0000 7f9051200640 1 -- 172.21.15.100:0/4042377226 wait complete. 2024-09-17T04:25:27.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.319+0000 7f9051200640 1 Processor -- start 2024-09-17T04:25:27.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.319+0000 7f9051200640 1 -- start start 2024-09-17T04:25:27.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.320+0000 7f9051200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f904c0705b0 0x7f904c099b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:27.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.320+0000 7f9051200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f904c0cfc80 0x7f904c0cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:27.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.320+0000 7f9051200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f904c15ce30 0x7f904c15b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:27.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.320+0000 7f9051200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f904c0cced0 con 0x7f904c15ce30 2024-09-17T04:25:27.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.320+0000 7f9051200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f904c0d0170 con 0x7f904c0705b0 2024-09-17T04:25:27.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.320+0000 7f9051200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f904c0cd050 con 0x7f904c0cfc80 2024-09-17T04:25:27.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.321+0000 7f904a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f904c15ce30 0x7f904c15b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:27.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.321+0000 7f9049e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f904c0705b0 0x7f904c099b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:27.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.321+0000 7f904a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f904c15ce30 0x7f904c15b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48544/0 (socket says 172.21.15.100:48544) 2024-09-17T04:25:27.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.321+0000 7f904a800640 1 -- 172.21.15.100:0/2091511870 learned_addr learned my addr 172.21.15.100:0/2091511870 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:27.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.321+0000 7f9041400640 1 --2- 172.21.15.100:0/2091511870 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f904c0cfc80 0x7f904c0cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:27.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.321+0000 7f904a800640 1 -- 172.21.15.100:0/2091511870 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f904c0cfc80 msgr2=0x7f904c0cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:27.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.321+0000 7f904a800640 1 --2- 172.21.15.100:0/2091511870 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f904c0cfc80 0x7f904c0cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:27.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.321+0000 7f904a800640 1 -- 172.21.15.100:0/2091511870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f904c0705b0 msgr2=0x7f904c099b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:27.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.321+0000 7f904a800640 1 --2- 172.21.15.100:0/2091511870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f904c0705b0 0x7f904c099b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:27.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.321+0000 7f904a800640 1 -- 172.21.15.100:0/2091511870 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904c1f92b0 con 0x7f904c15ce30 2024-09-17T04:25:27.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.321+0000 7f904a800640 1 --2- 172.21.15.100:0/2091511870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f904c15ce30 0x7f904c15b900 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f904400cb10 tx=0x7f904400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:27.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.322+0000 7f9042a00640 1 -- 172.21.15.100:0/2091511870 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044016020 con 0x7f904c15ce30 2024-09-17T04:25:27.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.322+0000 7f9042a00640 1 -- 172.21.15.100:0/2091511870 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f904400ad00 con 0x7f904c15ce30 2024-09-17T04:25:27.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.322+0000 7f9042a00640 1 -- 172.21.15.100:0/2091511870 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044012680 con 0x7f904c15ce30 2024-09-17T04:25:27.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.322+0000 7f9051200640 1 -- 172.21.15.100:0/2091511870 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f904c1f9590 con 0x7f904c15ce30 2024-09-17T04:25:27.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.322+0000 7f9051200640 1 -- 172.21.15.100:0/2091511870 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f904c1f9b50 con 0x7f904c15ce30 2024-09-17T04:25:27.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.323+0000 7f9051200640 1 -- 172.21.15.100:0/2091511870 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f904c0cbd80 con 0x7f904c15ce30 2024-09-17T04:25:27.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.324+0000 7f9042a00640 1 -- 172.21.15.100:0/2091511870 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 6) ==== 101492+0+0 (secure 0 0 0) 0x7f9044019020 con 0x7f904c15ce30 2024-09-17T04:25:27.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.325+0000 7f9042a00640 1 --2- 172.21.15.100:0/2091511870 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9028078c20 0x7f902807b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:27.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.325+0000 7f9042a00640 1 -- 172.21.15.100:0/2091511870 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f904409b430 con 0x7f904c15ce30 2024-09-17T04:25:27.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.325+0000 7f9049e00640 1 --2- 172.21.15.100:0/2091511870 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9028078c20 0x7f902807b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:27.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.326+0000 7f9049e00640 1 --2- 172.21.15.100:0/2091511870 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9028078c20 0x7f902807b130 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f904c09e510 tx=0x7f903c00b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:27.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.330+0000 7f9042a00640 1 -- 172.21.15.100:0/2091511870 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f904c0cbd80 con 0x7f904c15ce30 2024-09-17T04:25:27.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.524+0000 7f9051200640 1 -- 172.21.15.100:0/2091511870 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f904c0cd570 con 0x7f904c15ce30 2024-09-17T04:25:27.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.524+0000 7f9042a00640 1 -- 172.21.15.100:0/2091511870 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) ==== 80+0+0 (secure 0 0 0) 0x7f9044020020 con 0x7f904c15ce30 2024-09-17T04:25:27.524 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-09-17T04:25:27.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.527+0000 7f9051200640 1 -- 172.21.15.100:0/2091511870 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9028078c20 msgr2=0x7f902807b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:27.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.527+0000 7f9051200640 1 --2- 172.21.15.100:0/2091511870 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9028078c20 0x7f902807b130 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f904c09e510 tx=0x7f903c00b2e0 comp rx=0 tx=0).stop 2024-09-17T04:25:27.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.527+0000 7f9051200640 1 -- 172.21.15.100:0/2091511870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f904c15ce30 msgr2=0x7f904c15b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:27.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.527+0000 7f9051200640 1 --2- 172.21.15.100:0/2091511870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f904c15ce30 0x7f904c15b900 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f904400cb10 tx=0x7f904400a060 comp rx=0 tx=0).stop 2024-09-17T04:25:27.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.527+0000 7f9051200640 1 -- 172.21.15.100:0/2091511870 shutdown_connections 2024-09-17T04:25:27.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.527+0000 7f9051200640 1 --2- 172.21.15.100:0/2091511870 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9028078c20 0x7f902807b130 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:27.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.527+0000 7f9051200640 1 --2- 172.21.15.100:0/2091511870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f904c15ce30 0x7f904c15b900 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:27.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.527+0000 7f9051200640 1 --2- 172.21.15.100:0/2091511870 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f904c0cfc80 0x7f904c0cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:27.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.527+0000 7f9051200640 1 --2- 172.21.15.100:0/2091511870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f904c0705b0 0x7f904c099b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:27.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.527+0000 7f9051200640 1 -- 172.21.15.100:0/2091511870 >> 172.21.15.100:0/2091511870 conn(0x7f904c06d3d0 msgr2=0x7f904c0d0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:27.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.527+0000 7f9051200640 1 -- 172.21.15.100:0/2091511870 shutdown_connections 2024-09-17T04:25:27.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:27.527+0000 7f9051200640 1 -- 172.21.15.100:0/2091511870 wait complete. 2024-09-17T04:25:27.551 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T04:25:27.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:27.551 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T04:25:27.648 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-09-17T04:25:27.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:27.648 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T04:25:27.692 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:25:27.692 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:25:27.692 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:25:27.729 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:25:27.729 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:25:27.804 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:25:28.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.027+0000 7f5171a00640 1 -- 172.21.15.100:0/1271876006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f516c0cfc80 msgr2=0x7f516c156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:28.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.027+0000 7f5171a00640 1 --2- 172.21.15.100:0/1271876006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f516c0cfc80 0x7f516c156e40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f515c005fd0 tx=0x7f515c02a540 comp rx=0 tx=0).stop 2024-09-17T04:25:28.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.027+0000 7f5171a00640 1 -- 172.21.15.100:0/1271876006 shutdown_connections 2024-09-17T04:25:28.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.027+0000 7f5171a00640 1 --2- 172.21.15.100:0/1271876006 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f516c0705b0 0x7f516c0709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:28.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.027+0000 7f5171a00640 1 --2- 172.21.15.100:0/1271876006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f516c0cfc80 0x7f516c156e40 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:28.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.027+0000 7f5171a00640 1 -- 172.21.15.100:0/1271876006 >> 172.21.15.100:0/1271876006 conn(0x7f516c06d3d0 msgr2=0x7f516c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:28.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.027+0000 7f5171a00640 1 -- 172.21.15.100:0/1271876006 shutdown_connections 2024-09-17T04:25:28.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.027+0000 7f5171a00640 1 -- 172.21.15.100:0/1271876006 wait complete. 2024-09-17T04:25:28.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.029+0000 7f5171a00640 1 Processor -- start 2024-09-17T04:25:28.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.029+0000 7f5171a00640 1 -- start start 2024-09-17T04:25:28.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.031+0000 7f5171a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f516c0705b0 0x7f516c1565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:28.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.031+0000 7f5171a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f516c0cfc80 0x7f516c14b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:28.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.031+0000 7f5171a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f516c15ce30 0x7f516c0ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:28.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.031+0000 7f5171a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f516c0cd050 con 0x7f516c0705b0 2024-09-17T04:25:28.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.031+0000 7f5171a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f516c0cced0 con 0x7f516c15ce30 2024-09-17T04:25:28.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.031+0000 7f5171a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f516c0cd1d0 con 0x7f516c0cfc80 2024-09-17T04:25:28.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.031+0000 7f516aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f516c0705b0 0x7f516c1565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:28.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.031+0000 7f516aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f516c0705b0 0x7f516c1565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48560/0 (socket says 172.21.15.100:48560) 2024-09-17T04:25:28.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.031+0000 7f516aa00640 1 -- 172.21.15.100:0/120119505 learned_addr learned my addr 172.21.15.100:0/120119505 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:28.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.031+0000 7f516a000640 1 --2- 172.21.15.100:0/120119505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f516c0cfc80 0x7f516c14b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:28.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.031+0000 7f516b400640 1 --2- 172.21.15.100:0/120119505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f516c15ce30 0x7f516c0ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:28.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.031+0000 7f516aa00640 1 -- 172.21.15.100:0/120119505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f516c0cfc80 msgr2=0x7f516c14b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:28.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.031+0000 7f516aa00640 1 --2- 172.21.15.100:0/120119505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f516c0cfc80 0x7f516c14b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:28.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.031+0000 7f516aa00640 1 -- 172.21.15.100:0/120119505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f516c15ce30 msgr2=0x7f516c0ae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:28.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.031+0000 7f516aa00640 1 --2- 172.21.15.100:0/120119505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f516c15ce30 0x7f516c0ae4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:28.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.031+0000 7f516aa00640 1 -- 172.21.15.100:0/120119505 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516c1e7f60 con 0x7f516c0705b0 2024-09-17T04:25:28.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.031+0000 7f516b400640 1 --2- 172.21.15.100:0/120119505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f516c15ce30 0x7f516c0ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:28.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.032+0000 7f516aa00640 1 --2- 172.21.15.100:0/120119505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f516c0705b0 0x7f516c1565d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f515c034040 tx=0x7f515c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:28.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.032+0000 7f5162a00640 1 -- 172.21.15.100:0/120119505 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515c03e030 con 0x7f516c0705b0 2024-09-17T04:25:28.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.032+0000 7f5162a00640 1 -- 172.21.15.100:0/120119505 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f515c003d90 con 0x7f516c0705b0 2024-09-17T04:25:28.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.032+0000 7f5171a00640 1 -- 172.21.15.100:0/120119505 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f516c1e8370 con 0x7f516c0705b0 2024-09-17T04:25:28.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.032+0000 7f5162a00640 1 -- 172.21.15.100:0/120119505 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515c035040 con 0x7f516c0705b0 2024-09-17T04:25:28.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.032+0000 7f5171a00640 1 -- 172.21.15.100:0/120119505 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f516c1e71a0 con 0x7f516c0705b0 2024-09-17T04:25:28.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.033+0000 7f5171a00640 1 -- 172.21.15.100:0/120119505 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f516c0c3f40 con 0x7f516c0705b0 2024-09-17T04:25:28.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.034+0000 7f5162a00640 1 -- 172.21.15.100:0/120119505 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 6) ==== 101492+0+0 (secure 0 0 0) 0x7f515c041030 con 0x7f516c0705b0 2024-09-17T04:25:28.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.035+0000 7f5162a00640 1 --2- 172.21.15.100:0/120119505 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f514c078c20 0x7f514c07b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:28.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.035+0000 7f5162a00640 1 -- 172.21.15.100:0/120119505 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f515c0bb640 con 0x7f516c0705b0 2024-09-17T04:25:28.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.036+0000 7f516a000640 1 --2- 172.21.15.100:0/120119505 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f514c078c20 0x7f514c07b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:28.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.036+0000 7f516a000640 1 --2- 172.21.15.100:0/120119505 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f514c078c20 0x7f514c07b130 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f516c0cc9c0 tx=0x7f5158005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:28.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.040+0000 7f5162a00640 1 -- 172.21.15.100:0/120119505 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f516c1e7f60 con 0x7f516c0705b0 2024-09-17T04:25:28.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.075+0000 7f5162a00640 1 -- 172.21.15.100:0/120119505 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f515c07f4b0 con 0x7f516c0705b0 2024-09-17T04:25:28.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.243+0000 7f5171a00640 1 -- 172.21.15.100:0/120119505 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f516c0cd6f0 con 0x7f516c0705b0 2024-09-17T04:25:28.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.244+0000 7f5162a00640 1 -- 172.21.15.100:0/120119505 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) ==== 80+0+0 (secure 0 0 0) 0x7f515c09c940 con 0x7f516c0705b0 2024-09-17T04:25:28.243 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-09-17T04:25:28.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.246+0000 7f5171a00640 1 -- 172.21.15.100:0/120119505 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f514c078c20 msgr2=0x7f514c07b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.246+0000 7f5171a00640 1 --2- 172.21.15.100:0/120119505 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f514c078c20 0x7f514c07b130 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f516c0cc9c0 tx=0x7f5158005f90 comp rx=0 tx=0).stop 2024-09-17T04:25:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.246+0000 7f5171a00640 1 -- 172.21.15.100:0/120119505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f516c0705b0 msgr2=0x7f516c1565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.246+0000 7f5171a00640 1 --2- 172.21.15.100:0/120119505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f516c0705b0 0x7f516c1565d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f515c034040 tx=0x7f515c0038a0 comp rx=0 tx=0).stop 2024-09-17T04:25:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.246+0000 7f5171a00640 1 -- 172.21.15.100:0/120119505 shutdown_connections 2024-09-17T04:25:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.246+0000 7f5171a00640 1 --2- 172.21.15.100:0/120119505 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f514c078c20 0x7f514c07b130 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.246+0000 7f5171a00640 1 --2- 172.21.15.100:0/120119505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f516c15ce30 0x7f516c0ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.246+0000 7f5171a00640 1 --2- 172.21.15.100:0/120119505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f516c0cfc80 0x7f516c14b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.246+0000 7f5171a00640 1 --2- 172.21.15.100:0/120119505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f516c0705b0 0x7f516c1565d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.246+0000 7f5171a00640 1 -- 172.21.15.100:0/120119505 >> 172.21.15.100:0/120119505 conn(0x7f516c06d3d0 msgr2=0x7f516c15f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.246+0000 7f5171a00640 1 -- 172.21.15.100:0/120119505 shutdown_connections 2024-09-17T04:25:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.246+0000 7f5171a00640 1 -- 172.21.15.100:0/120119505 wait complete. 2024-09-17T04:25:28.271 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-17T04:25:28.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:28.271 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-17T04:25:28.336 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2024-09-17T04:25:28.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:28.336 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T04:25:28.372 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:25:28.373 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:25:28.373 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:25:28.408 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:25:28.408 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:25:28.408 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:25:28.408 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:25:28.447 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:25:28.447 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:25:28.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.666+0000 7f550c000640 1 -- 172.21.15.100:0/2679840626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55040ad0e0 msgr2=0x7f55040ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:28.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.666+0000 7f550c000640 1 --2- 172.21.15.100:0/2679840626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55040ad0e0 0x7f55040ad4f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f54f0005fd0 tx=0x7f54f002a540 comp rx=0 tx=0).stop 2024-09-17T04:25:28.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.666+0000 7f550c000640 1 -- 172.21.15.100:0/2679840626 shutdown_connections 2024-09-17T04:25:28.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.666+0000 7f550c000640 1 --2- 172.21.15.100:0/2679840626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55040b3550 0x7f55040b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:28.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.666+0000 7f550c000640 1 --2- 172.21.15.100:0/2679840626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55040ad0e0 0x7f55040ad4f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:28.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.666+0000 7f550c000640 1 -- 172.21.15.100:0/2679840626 >> 172.21.15.100:0/2679840626 conn(0x7f550406d730 msgr2=0x7f55040b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:28.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.666+0000 7f550c000640 1 -- 172.21.15.100:0/2679840626 shutdown_connections 2024-09-17T04:25:28.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.666+0000 7f550c000640 1 -- 172.21.15.100:0/2679840626 wait complete. 2024-09-17T04:25:28.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.668+0000 7f550c000640 1 Processor -- start 2024-09-17T04:25:28.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.669+0000 7f550c000640 1 -- start start 2024-09-17T04:25:28.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.670+0000 7f550c000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55040ad0e0 0x7f5504156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:28.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.670+0000 7f550c000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55040b3550 0x7f55041f5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:28.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.670+0000 7f550c000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f550415d760 0x7f550415b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:28.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.670+0000 7f550c000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f55040b4da0 con 0x7f55040ad0e0 2024-09-17T04:25:28.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.670+0000 7f550c000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f55040b4c20 con 0x7f55040b3550 2024-09-17T04:25:28.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.670+0000 7f550c000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f55040b3ea0 con 0x7f550415d760 2024-09-17T04:25:28.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.670+0000 7f5509600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55040ad0e0 0x7f5504156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:28.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.670+0000 7f5509600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55040ad0e0 0x7f5504156700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48576/0 (socket says 172.21.15.100:48576) 2024-09-17T04:25:28.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.670+0000 7f5509600640 1 -- 172.21.15.100:0/2900890240 learned_addr learned my addr 172.21.15.100:0/2900890240 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:28.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.670+0000 7f5508c00640 1 --2- 172.21.15.100:0/2900890240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55040b3550 0x7f55041f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:28.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.670+0000 7f550a000640 1 --2- 172.21.15.100:0/2900890240 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f550415d760 0x7f550415b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:28.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.671+0000 7f5509600640 1 -- 172.21.15.100:0/2900890240 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f550415d760 msgr2=0x7f550415b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:28.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.671+0000 7f5509600640 1 --2- 172.21.15.100:0/2900890240 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f550415d760 0x7f550415b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:28.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.671+0000 7f5509600640 1 -- 172.21.15.100:0/2900890240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55040b3550 msgr2=0x7f55041f5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:28.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.671+0000 7f5509600640 1 --2- 172.21.15.100:0/2900890240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55040b3550 0x7f55041f5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:28.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.671+0000 7f5509600640 1 -- 172.21.15.100:0/2900890240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550414f4b0 con 0x7f55040ad0e0 2024-09-17T04:25:28.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.671+0000 7f5509600640 1 --2- 172.21.15.100:0/2900890240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55040ad0e0 0x7f5504156700 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f54f800c3a0 tx=0x7f54f800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:28.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.671+0000 7f5501600640 1 -- 172.21.15.100:0/2900890240 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8017070 con 0x7f55040ad0e0 2024-09-17T04:25:28.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.671+0000 7f5501600640 1 -- 172.21.15.100:0/2900890240 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54f80053c0 con 0x7f55040ad0e0 2024-09-17T04:25:28.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.671+0000 7f5501600640 1 -- 172.21.15.100:0/2900890240 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8003850 con 0x7f55040ad0e0 2024-09-17T04:25:28.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.671+0000 7f550c000640 1 -- 172.21.15.100:0/2900890240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f550414f790 con 0x7f55040ad0e0 2024-09-17T04:25:28.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.671+0000 7f550c000640 1 -- 172.21.15.100:0/2900890240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f550414fd50 con 0x7f55040ad0e0 2024-09-17T04:25:28.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.672+0000 7f550c000640 1 -- 172.21.15.100:0/2900890240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55040b4370 con 0x7f55040ad0e0 2024-09-17T04:25:28.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.674+0000 7f5501600640 1 -- 172.21.15.100:0/2900890240 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f54f801a070 con 0x7f55040ad0e0 2024-09-17T04:25:28.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.675+0000 7f5501600640 1 --2- 172.21.15.100:0/2900890240 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f54e4078d70 0x7f54e407b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:28.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.675+0000 7f5501600640 1 -- 172.21.15.100:0/2900890240 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f54f809a360 con 0x7f55040ad0e0 2024-09-17T04:25:28.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.676+0000 7f5508c00640 1 --2- 172.21.15.100:0/2900890240 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f54e4078d70 0x7f54e407b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:28.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.676+0000 7f5508c00640 1 --2- 172.21.15.100:0/2900890240 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f54e4078d70 0x7f54e407b280 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f54f0033000 tx=0x7f54f0004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:28.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.680+0000 7f5501600640 1 -- 172.21.15.100:0/2900890240 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f55040ada30 con 0x7f55040ad0e0 2024-09-17T04:25:28.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.898+0000 7f550c000640 1 -- 172.21.15.100:0/2900890240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5504069700 con 0x7f55040ad0e0 2024-09-17T04:25:28.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.899+0000 7f5501600640 1 -- 172.21.15.100:0/2900890240 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5232 (secure 0 0 0) 0x7f54f8061e60 con 0x7f55040ad0e0 2024-09-17T04:25:28.899 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:28.899 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":18,"btime":"2024-09-17T04:25:23:356418+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4667,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2259943840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":2259943840},{"type":"v1","addr":"172.21.15.100:6835","nonce":2259943840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4670,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/1748167554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1748167554},{"type":"v1","addr":"172.21.15.100:6834","nonce":1748167554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4854,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3159351871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":3159351871},{"type":"v1","addr":"172.21.15.184:6836","nonce":3159351871}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:07.565710+0000","modified":"2024-09-17T04:25:10.595342+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4857},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4857":{"gid":4857,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/294839954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":294839954},{"type":"v1","addr":"172.21.15.184:6837","nonce":294839954}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4857,"qdb_cluster":[4857]},"id":2}]} 2024-09-17T04:25:28.899 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 18 2024-09-17T04:25:28.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.901+0000 7f550c000640 1 -- 172.21.15.100:0/2900890240 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f54e4078d70 msgr2=0x7f54e407b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:28.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.901+0000 7f550c000640 1 --2- 172.21.15.100:0/2900890240 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f54e4078d70 0x7f54e407b280 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f54f0033000 tx=0x7f54f0004730 comp rx=0 tx=0).stop 2024-09-17T04:25:28.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.901+0000 7f550c000640 1 -- 172.21.15.100:0/2900890240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55040ad0e0 msgr2=0x7f5504156700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:28.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.901+0000 7f550c000640 1 --2- 172.21.15.100:0/2900890240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55040ad0e0 0x7f5504156700 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f54f800c3a0 tx=0x7f54f800c860 comp rx=0 tx=0).stop 2024-09-17T04:25:28.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.901+0000 7f550c000640 1 -- 172.21.15.100:0/2900890240 shutdown_connections 2024-09-17T04:25:28.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.901+0000 7f550c000640 1 --2- 172.21.15.100:0/2900890240 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f54e4078d70 0x7f54e407b280 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:28.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.901+0000 7f550c000640 1 --2- 172.21.15.100:0/2900890240 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f550415d760 0x7f550415b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:28.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.901+0000 7f550c000640 1 --2- 172.21.15.100:0/2900890240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55040b3550 0x7f55041f5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:28.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.901+0000 7f550c000640 1 --2- 172.21.15.100:0/2900890240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55040ad0e0 0x7f5504156700 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:28.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.901+0000 7f550c000640 1 -- 172.21.15.100:0/2900890240 >> 172.21.15.100:0/2900890240 conn(0x7f550406d730 msgr2=0x7f550415dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:28.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.902+0000 7f550c000640 1 -- 172.21.15.100:0/2900890240 shutdown_connections 2024-09-17T04:25:28.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:28.902+0000 7f550c000640 1 -- 172.21.15.100:0/2900890240 wait complete. 2024-09-17T04:25:28.926 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:25:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.147+0000 7fdabfc00640 1 -- 172.21.15.100:0/1419387080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdab8160310 msgr2=0x7fdab8160720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.147+0000 7fdabfc00640 1 --2- 172.21.15.100:0/1419387080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdab8160310 0x7fdab8160720 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdab8075250 tx=0x7fdab002a980 comp rx=0 tx=0).stop 2024-09-17T04:25:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.147+0000 7fdabfc00640 1 -- 172.21.15.100:0/1419387080 shutdown_connections 2024-09-17T04:25:29.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.147+0000 7fdabfc00640 1 --2- 172.21.15.100:0/1419387080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdab8160310 0x7fdab8160720 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.147+0000 7fdabfc00640 1 --2- 172.21.15.100:0/1419387080 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdab80ad0e0 0x7fdab80ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.147+0000 7fdabfc00640 1 --2- 172.21.15.100:0/1419387080 >> v2:172.21.15.184:3300/0 conn(0x7fdab815d760 0x7fdab81535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.147+0000 7fdabfc00640 1 -- 172.21.15.100:0/1419387080 >> 172.21.15.100:0/1419387080 conn(0x7fdab806d730 msgr2=0x7fdab80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.147+0000 7fdabfc00640 1 -- 172.21.15.100:0/1419387080 shutdown_connections 2024-09-17T04:25:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.147+0000 7fdabfc00640 1 -- 172.21.15.100:0/1419387080 wait complete. 2024-09-17T04:25:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.150+0000 7fdabfc00640 1 Processor -- start 2024-09-17T04:25:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.150+0000 7fdabfc00640 1 -- start start 2024-09-17T04:25:29.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.151+0000 7fdabfc00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdab80ad0e0 0x7fdab8155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:29.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.151+0000 7fdabfc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdab815d760 0x7fdab81f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:29.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.151+0000 7fdabfc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdab8160310 0x7fdab8145220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:29.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.151+0000 7fdabfc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdab80b4da0 con 0x7fdab815d760 2024-09-17T04:25:29.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.151+0000 7fdabfc00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdab80b4c20 con 0x7fdab8160310 2024-09-17T04:25:29.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.151+0000 7fdabfc00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdab8160c60 con 0x7fdab80ad0e0 2024-09-17T04:25:29.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.151+0000 7fdaafe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdab815d760 0x7fdab81f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:29.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.151+0000 7fdaafe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdab815d760 0x7fdab81f51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48586/0 (socket says 172.21.15.100:48586) 2024-09-17T04:25:29.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.151+0000 7fdaafe00640 1 -- 172.21.15.100:0/3368945660 learned_addr learned my addr 172.21.15.100:0/3368945660 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:29.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.152+0000 7fdabd200640 1 --2- 172.21.15.100:0/3368945660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdab80ad0e0 0x7fdab8155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:29.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.152+0000 7fdabdc00640 1 --2- 172.21.15.100:0/3368945660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdab8160310 0x7fdab8145220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:29.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.152+0000 7fdaafe00640 1 -- 172.21.15.100:0/3368945660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdab80ad0e0 msgr2=0x7fdab8155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:29.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.152+0000 7fdaafe00640 1 --2- 172.21.15.100:0/3368945660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdab80ad0e0 0x7fdab8155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:29.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.152+0000 7fdaafe00640 1 -- 172.21.15.100:0/3368945660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdab8160310 msgr2=0x7fdab8145220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:29.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.152+0000 7fdaafe00640 1 --2- 172.21.15.100:0/3368945660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdab8160310 0x7fdab8145220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:29.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.152+0000 7fdaafe00640 1 -- 172.21.15.100:0/3368945660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab814f100 con 0x7fdab815d760 2024-09-17T04:25:29.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.152+0000 7fdabd200640 1 --2- 172.21.15.100:0/3368945660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdab80ad0e0 0x7fdab8155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:29.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.152+0000 7fdaafe00640 1 --2- 172.21.15.100:0/3368945660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdab815d760 0x7fdab81f51e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000e770 tx=0x7fdaa000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:29.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.152+0000 7fdab5600640 1 -- 172.21.15.100:0/3368945660 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa000ccc0 con 0x7fdab815d760 2024-09-17T04:25:29.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.152+0000 7fdab5600640 1 -- 172.21.15.100:0/3368945660 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdaa000ee80 con 0x7fdab815d760 2024-09-17T04:25:29.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.152+0000 7fdabfc00640 1 -- 172.21.15.100:0/3368945660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab81548d0 con 0x7fdab815d760 2024-09-17T04:25:29.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.152+0000 7fdab5600640 1 -- 172.21.15.100:0/3368945660 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0004290 con 0x7fdab815d760 2024-09-17T04:25:29.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.153+0000 7fdabfc00640 1 -- 172.21.15.100:0/3368945660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab8154e90 con 0x7fdab815d760 2024-09-17T04:25:29.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.153+0000 7fdabfc00640 1 -- 172.21.15.100:0/3368945660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab8154640 con 0x7fdab815d760 2024-09-17T04:25:29.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.155+0000 7fdab5600640 1 -- 172.21.15.100:0/3368945660 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fdaa00035f0 con 0x7fdab815d760 2024-09-17T04:25:29.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.156+0000 7fdab5600640 1 --2- 172.21.15.100:0/3368945660 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fda94078d40 0x7fda9407b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:29.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.156+0000 7fdab5600640 1 -- 172.21.15.100:0/3368945660 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fdaa009bb10 con 0x7fdab815d760 2024-09-17T04:25:29.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.156+0000 7fdabd200640 1 --2- 172.21.15.100:0/3368945660 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fda94078d40 0x7fda9407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:29.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.157+0000 7fdabd200640 1 --2- 172.21.15.100:0/3368945660 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fda94078d40 0x7fda9407b250 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8009b00 tx=0x7fdaa8005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:29.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.160+0000 7fdab5600640 1 -- 172.21.15.100:0/3368945660 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fdab814f100 con 0x7fdab815d760 2024-09-17T04:25:29.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.378+0000 7fdabfc00640 1 -- 172.21.15.100:0/3368945660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdab81535c0 con 0x7fdab815d760 2024-09-17T04:25:29.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.379+0000 7fdab5600640 1 -- 172.21.15.100:0/3368945660 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5232 (secure 0 0 0) 0x7fdaa0063b20 con 0x7fdab815d760 2024-09-17T04:25:29.379 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:29.379 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":18,"btime":"2024-09-17T04:25:23:356418+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4667,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2259943840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":2259943840},{"type":"v1","addr":"172.21.15.100:6835","nonce":2259943840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4670,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/1748167554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1748167554},{"type":"v1","addr":"172.21.15.100:6834","nonce":1748167554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4854,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3159351871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":3159351871},{"type":"v1","addr":"172.21.15.184:6836","nonce":3159351871}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:07.565710+0000","modified":"2024-09-17T04:25:10.595342+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4857},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4857":{"gid":4857,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/294839954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":294839954},{"type":"v1","addr":"172.21.15.184:6837","nonce":294839954}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4857,"qdb_cluster":[4857]},"id":2}]} 2024-09-17T04:25:29.379 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 18 2024-09-17T04:25:29.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.382+0000 7fdabfc00640 1 -- 172.21.15.100:0/3368945660 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fda94078d40 msgr2=0x7fda9407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:29.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.382+0000 7fdabfc00640 1 --2- 172.21.15.100:0/3368945660 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fda94078d40 0x7fda9407b250 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8009b00 tx=0x7fdaa8005f90 comp rx=0 tx=0).stop 2024-09-17T04:25:29.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.382+0000 7fdabfc00640 1 -- 172.21.15.100:0/3368945660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdab815d760 msgr2=0x7fdab81f51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:29.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.382+0000 7fdabfc00640 1 --2- 172.21.15.100:0/3368945660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdab815d760 0x7fdab81f51e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000e770 tx=0x7fdaa000ec30 comp rx=0 tx=0).stop 2024-09-17T04:25:29.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.382+0000 7fdabfc00640 1 -- 172.21.15.100:0/3368945660 shutdown_connections 2024-09-17T04:25:29.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.382+0000 7fdabfc00640 1 --2- 172.21.15.100:0/3368945660 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fda94078d40 0x7fda9407b250 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:29.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.382+0000 7fdabfc00640 1 --2- 172.21.15.100:0/3368945660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdab8160310 0x7fdab8145220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:29.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.382+0000 7fdabfc00640 1 --2- 172.21.15.100:0/3368945660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdab815d760 0x7fdab81f51e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:29.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.382+0000 7fdabfc00640 1 --2- 172.21.15.100:0/3368945660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdab80ad0e0 0x7fdab8155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:29.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.382+0000 7fdabfc00640 1 -- 172.21.15.100:0/3368945660 >> 172.21.15.100:0/3368945660 conn(0x7fdab806d730 msgr2=0x7fdab80cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:29.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.382+0000 7fdabfc00640 1 -- 172.21.15.100:0/3368945660 shutdown_connections 2024-09-17T04:25:29.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.382+0000 7fdabfc00640 1 -- 172.21.15.100:0/3368945660 wait complete. 2024-09-17T04:25:29.408 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:25:29.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.629+0000 7f2cd8800640 1 -- 172.21.15.100:0/1204693049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cd00ad0e0 msgr2=0x7f2cd00ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:29.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.629+0000 7f2cd8800640 1 --2- 172.21.15.100:0/1204693049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cd00ad0e0 0x7f2cd00ad4f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009830 tx=0x7f2cc002a480 comp rx=0 tx=0).stop 2024-09-17T04:25:29.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.629+0000 7f2cd8800640 1 -- 172.21.15.100:0/1204693049 shutdown_connections 2024-09-17T04:25:29.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.629+0000 7f2cd8800640 1 --2- 172.21.15.100:0/1204693049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cd00ad0e0 0x7f2cd00ad4f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:29.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.629+0000 7f2cd8800640 1 --2- 172.21.15.100:0/1204693049 >> v2:172.21.15.184:3300/0 conn(0x7f2cd015d760 0x7f2cd01535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:29.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.629+0000 7f2cd8800640 1 -- 172.21.15.100:0/1204693049 >> 172.21.15.100:0/1204693049 conn(0x7f2cd006d730 msgr2=0x7f2cd00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:29.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.629+0000 7f2cd8800640 1 -- 172.21.15.100:0/1204693049 shutdown_connections 2024-09-17T04:25:29.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.629+0000 7f2cd8800640 1 -- 172.21.15.100:0/1204693049 wait complete. 2024-09-17T04:25:29.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.631+0000 7f2cd8800640 1 Processor -- start 2024-09-17T04:25:29.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.631+0000 7f2cd8800640 1 -- start start 2024-09-17T04:25:29.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.633+0000 7f2cd8800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd00ad0e0 0x7f2cd00998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:29.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.633+0000 7f2cd8800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cd015d760 0x7f2cd01f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:29.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.633+0000 7f2cd8800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cd0160310 0x7f2cd01f76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:29.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.633+0000 7f2cd8800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd006f220 con 0x7f2cd00ad0e0 2024-09-17T04:25:29.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.633+0000 7f2cd8800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd006f0a0 con 0x7f2cd015d760 2024-09-17T04:25:29.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.633+0000 7f2cd8800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2cd00b4c20 con 0x7f2cd0160310 2024-09-17T04:25:29.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.633+0000 7f2cd5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd00ad0e0 0x7f2cd00998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:29.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.633+0000 7f2cd5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd00ad0e0 0x7f2cd00998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48600/0 (socket says 172.21.15.100:48600) 2024-09-17T04:25:29.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.633+0000 7f2cd5e00640 1 -- 172.21.15.100:0/3988494417 learned_addr learned my addr 172.21.15.100:0/3988494417 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:29.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.633+0000 7f2cd5400640 1 --2- 172.21.15.100:0/3988494417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cd015d760 0x7f2cd01f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:29.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.633+0000 7f2cd6800640 1 --2- 172.21.15.100:0/3988494417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cd0160310 0x7f2cd01f76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:29.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.634+0000 7f2cd5e00640 1 -- 172.21.15.100:0/3988494417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cd0160310 msgr2=0x7f2cd01f76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:29.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.634+0000 7f2cd5e00640 1 --2- 172.21.15.100:0/3988494417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cd0160310 0x7f2cd01f76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:29.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.634+0000 7f2cd5e00640 1 -- 172.21.15.100:0/3988494417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cd015d760 msgr2=0x7f2cd01f71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:29.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.634+0000 7f2cd5e00640 1 --2- 172.21.15.100:0/3988494417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cd015d760 0x7f2cd01f71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:29.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.634+0000 7f2cd5e00640 1 -- 172.21.15.100:0/3988494417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd014f9f0 con 0x7f2cd00ad0e0 2024-09-17T04:25:29.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.634+0000 7f2cd6800640 1 --2- 172.21.15.100:0/3988494417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cd0160310 0x7f2cd01f76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:29.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.634+0000 7f2cd5e00640 1 --2- 172.21.15.100:0/3988494417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd00ad0e0 0x7f2cd00998a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400e7c0 tx=0x7f2cc400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:29.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.634+0000 7f2cca000640 1 -- 172.21.15.100:0/3988494417 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc400ccc0 con 0x7f2cd00ad0e0 2024-09-17T04:25:29.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.634+0000 7f2cca000640 1 -- 172.21.15.100:0/3988494417 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cc400ce60 con 0x7f2cd00ad0e0 2024-09-17T04:25:29.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.634+0000 7f2cca000640 1 -- 172.21.15.100:0/3988494417 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4005280 con 0x7f2cd00ad0e0 2024-09-17T04:25:29.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.634+0000 7f2cd8800640 1 -- 172.21.15.100:0/3988494417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cd014fcd0 con 0x7f2cd00ad0e0 2024-09-17T04:25:29.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.634+0000 7f2cd8800640 1 -- 172.21.15.100:0/3988494417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cd0150290 con 0x7f2cd00ad0e0 2024-09-17T04:25:29.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.636+0000 7f2cd8800640 1 -- 172.21.15.100:0/3988494417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cd0072dc0 con 0x7f2cd00ad0e0 2024-09-17T04:25:29.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.637+0000 7f2cca000640 1 -- 172.21.15.100:0/3988494417 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f2cc4005420 con 0x7f2cd00ad0e0 2024-09-17T04:25:29.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.638+0000 7f2cca000640 1 --2- 172.21.15.100:0/3988494417 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2cb4078d40 0x7f2cb407b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:29.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.638+0000 7f2cca000640 1 -- 172.21.15.100:0/3988494417 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f2cc409bad0 con 0x7f2cd00ad0e0 2024-09-17T04:25:29.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.639+0000 7f2cd5400640 1 --2- 172.21.15.100:0/3988494417 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2cb4078d40 0x7f2cb407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:29.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.639+0000 7f2cd5400640 1 --2- 172.21.15.100:0/3988494417 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2cb4078d40 0x7f2cb407b250 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0034000 tx=0x7f2cc0004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:29.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.643+0000 7f2cca000640 1 -- 172.21.15.100:0/3988494417 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f2cd0072dc0 con 0x7f2cd00ad0e0 2024-09-17T04:25:29.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.827+0000 7f2cd8800640 1 -- 172.21.15.100:0/3988494417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2cd0160fa0 con 0x7f2cd00ad0e0 2024-09-17T04:25:29.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.828+0000 7f2cca000640 1 -- 172.21.15.100:0/3988494417 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16777 (secure 0 0 0) 0x7f2cc4063b70 con 0x7f2cd00ad0e0 2024-09-17T04:25:29.828 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:29.828 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":36,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:25:09.097160+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:25:05.694905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T04:25:06.533348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:25:29.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.831+0000 7f2cd8800640 1 -- 172.21.15.100:0/3988494417 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2cb4078d40 msgr2=0x7f2cb407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:29.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.831+0000 7f2cd8800640 1 --2- 172.21.15.100:0/3988494417 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2cb4078d40 0x7f2cb407b250 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0034000 tx=0x7f2cc0004730 comp rx=0 tx=0).stop 2024-09-17T04:25:29.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.831+0000 7f2cd8800640 1 -- 172.21.15.100:0/3988494417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd00ad0e0 msgr2=0x7f2cd00998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:29.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.831+0000 7f2cd8800640 1 --2- 172.21.15.100:0/3988494417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd00ad0e0 0x7f2cd00998a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400e7c0 tx=0x7f2cc400ec80 comp rx=0 tx=0).stop 2024-09-17T04:25:29.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.831+0000 7f2cd8800640 1 -- 172.21.15.100:0/3988494417 shutdown_connections 2024-09-17T04:25:29.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.831+0000 7f2cd8800640 1 --2- 172.21.15.100:0/3988494417 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2cb4078d40 0x7f2cb407b250 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:29.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.831+0000 7f2cd8800640 1 --2- 172.21.15.100:0/3988494417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cd0160310 0x7f2cd01f76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:29.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.831+0000 7f2cd8800640 1 --2- 172.21.15.100:0/3988494417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cd015d760 0x7f2cd01f71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:29.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.831+0000 7f2cd8800640 1 --2- 172.21.15.100:0/3988494417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd00ad0e0 0x7f2cd00998a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:29.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.831+0000 7f2cd8800640 1 -- 172.21.15.100:0/3988494417 >> 172.21.15.100:0/3988494417 conn(0x7f2cd006d730 msgr2=0x7f2cd0160720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:29.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.831+0000 7f2cd8800640 1 -- 172.21.15.100:0/3988494417 shutdown_connections 2024-09-17T04:25:29.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:29.831+0000 7f2cd8800640 1 -- 172.21.15.100:0/3988494417 wait complete. 2024-09-17T04:25:29.855 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T04:25:29.856 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:25:30.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.076+0000 7fd20a800640 1 -- 172.21.15.100:0/3884708042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2040ad0e0 msgr2=0x7fd2040ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:30.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.076+0000 7fd20a800640 1 --2- 172.21.15.100:0/3884708042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2040ad0e0 0x7fd2040ad4f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec005fd0 tx=0x7fd1ec02a540 comp rx=0 tx=0).stop 2024-09-17T04:25:30.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.076+0000 7fd20a800640 1 -- 172.21.15.100:0/3884708042 shutdown_connections 2024-09-17T04:25:30.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.076+0000 7fd20a800640 1 --2- 172.21.15.100:0/3884708042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2040ad0e0 0x7fd2040ad4f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:30.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.076+0000 7fd20a800640 1 --2- 172.21.15.100:0/3884708042 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd20415d760 0x7fd2041535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:30.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.076+0000 7fd20a800640 1 -- 172.21.15.100:0/3884708042 >> 172.21.15.100:0/3884708042 conn(0x7fd20406d730 msgr2=0x7fd2040b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:30.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.076+0000 7fd20a800640 1 -- 172.21.15.100:0/3884708042 shutdown_connections 2024-09-17T04:25:30.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.076+0000 7fd20a800640 1 -- 172.21.15.100:0/3884708042 wait complete. 2024-09-17T04:25:30.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.080+0000 7fd20a800640 1 Processor -- start 2024-09-17T04:25:30.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.080+0000 7fd20a800640 1 -- start start 2024-09-17T04:25:30.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.081+0000 7fd20a800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2040ad0e0 0x7fd2041422d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:30.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.081+0000 7fd20a800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd20415d760 0x7fd2041f6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:30.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.081+0000 7fd20a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd204160310 0x7fd2041f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:30.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.081+0000 7fd20a800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd20406f220 con 0x7fd204160310 2024-09-17T04:25:30.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.081+0000 7fd20a800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd20406f0a0 con 0x7fd2040ad0e0 2024-09-17T04:25:30.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.081+0000 7fd20a800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd2040b4c20 con 0x7fd20415d760 2024-09-17T04:25:30.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.082+0000 7fd203e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd204160310 0x7fd2041f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:30.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.082+0000 7fd203e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd204160310 0x7fd2041f72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48626/0 (socket says 172.21.15.100:48626) 2024-09-17T04:25:30.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.082+0000 7fd203e00640 1 -- 172.21.15.100:0/1378218089 learned_addr learned my addr 172.21.15.100:0/1378218089 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:30.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.082+0000 7fd203400640 1 --2- 172.21.15.100:0/1378218089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2040ad0e0 0x7fd2041422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:30.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.082+0000 7fd1faa00640 1 --2- 172.21.15.100:0/1378218089 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd20415d760 0x7fd2041f6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:30.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.082+0000 7fd203e00640 1 -- 172.21.15.100:0/1378218089 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd20415d760 msgr2=0x7fd2041f6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:30.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.082+0000 7fd203e00640 1 --2- 172.21.15.100:0/1378218089 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd20415d760 0x7fd2041f6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:30.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.082+0000 7fd203e00640 1 -- 172.21.15.100:0/1378218089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2040ad0e0 msgr2=0x7fd2041422d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:30.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.082+0000 7fd203e00640 1 --2- 172.21.15.100:0/1378218089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2040ad0e0 0x7fd2041422d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:30.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.082+0000 7fd203e00640 1 -- 172.21.15.100:0/1378218089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd204142810 con 0x7fd204160310 2024-09-17T04:25:30.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.082+0000 7fd1faa00640 1 --2- 172.21.15.100:0/1378218089 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd20415d760 0x7fd2041f6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:30.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.082+0000 7fd203400640 1 --2- 172.21.15.100:0/1378218089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2040ad0e0 0x7fd2041422d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:30.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.082+0000 7fd203e00640 1 --2- 172.21.15.100:0/1378218089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd204160310 0x7fd2041f72c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc002230 tx=0x7fd1fc00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:30.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.083+0000 7fd1fbe00640 1 -- 172.21.15.100:0/1378218089 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc003b70 con 0x7fd204160310 2024-09-17T04:25:30.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.083+0000 7fd1fbe00640 1 -- 172.21.15.100:0/1378218089 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1fc003d10 con 0x7fd204160310 2024-09-17T04:25:30.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.083+0000 7fd1fbe00640 1 -- 172.21.15.100:0/1378218089 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc005330 con 0x7fd204160310 2024-09-17T04:25:30.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.083+0000 7fd20a800640 1 -- 172.21.15.100:0/1378218089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd204158760 con 0x7fd204160310 2024-09-17T04:25:30.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.083+0000 7fd20a800640 1 -- 172.21.15.100:0/1378218089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd204158d20 con 0x7fd204160310 2024-09-17T04:25:30.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.084+0000 7fd20a800640 1 -- 172.21.15.100:0/1378218089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd204160fa0 con 0x7fd204160310 2024-09-17T04:25:30.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.085+0000 7fd1fbe00640 1 -- 172.21.15.100:0/1378218089 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fd1fc0056e0 con 0x7fd204160310 2024-09-17T04:25:30.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.086+0000 7fd1fbe00640 1 --2- 172.21.15.100:0/1378218089 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd1e0078d70 0x7fd1e007b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:30.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.086+0000 7fd1fbe00640 1 -- 172.21.15.100:0/1378218089 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fd1fc09b000 con 0x7fd204160310 2024-09-17T04:25:30.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.087+0000 7fd203400640 1 --2- 172.21.15.100:0/1378218089 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd1e0078d70 0x7fd1e007b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:30.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.087+0000 7fd203400640 1 --2- 172.21.15.100:0/1378218089 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd1e0078d70 0x7fd1e007b280 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4009b80 tx=0x7fd1f4005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:30.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.090+0000 7fd1fbe00640 1 -- 172.21.15.100:0/1378218089 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd2040ada30 con 0x7fd204160310 2024-09-17T04:25:30.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.311+0000 7fd20a800640 1 -- 172.21.15.100:0/1378218089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2040cfec0 con 0x7fd204160310 2024-09-17T04:25:30.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.312+0000 7fd1fbe00640 1 -- 172.21.15.100:0/1378218089 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5232 (secure 0 0 0) 0x7fd1fc0631d0 con 0x7fd204160310 2024-09-17T04:25:30.312 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:30.312 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":18,"btime":"2024-09-17T04:25:23:356418+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4667,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2259943840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":2259943840},{"type":"v1","addr":"172.21.15.100:6835","nonce":2259943840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4670,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/1748167554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1748167554},{"type":"v1","addr":"172.21.15.100:6834","nonce":1748167554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4854,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3159351871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":3159351871},{"type":"v1","addr":"172.21.15.184:6836","nonce":3159351871}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:07.565710+0000","modified":"2024-09-17T04:25:10.595342+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4857},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4857":{"gid":4857,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/294839954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":294839954},{"type":"v1","addr":"172.21.15.184:6837","nonce":294839954}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4857,"qdb_cluster":[4857]},"id":2}]} 2024-09-17T04:25:30.312 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 18 2024-09-17T04:25:30.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.314+0000 7fd20a800640 1 -- 172.21.15.100:0/1378218089 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd1e0078d70 msgr2=0x7fd1e007b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:30.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.314+0000 7fd20a800640 1 --2- 172.21.15.100:0/1378218089 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd1e0078d70 0x7fd1e007b280 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4009b80 tx=0x7fd1f4005f90 comp rx=0 tx=0).stop 2024-09-17T04:25:30.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.314+0000 7fd20a800640 1 -- 172.21.15.100:0/1378218089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd204160310 msgr2=0x7fd2041f72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:30.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.314+0000 7fd20a800640 1 --2- 172.21.15.100:0/1378218089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd204160310 0x7fd2041f72c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc002230 tx=0x7fd1fc00cb90 comp rx=0 tx=0).stop 2024-09-17T04:25:30.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.315+0000 7fd20a800640 1 -- 172.21.15.100:0/1378218089 shutdown_connections 2024-09-17T04:25:30.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.315+0000 7fd20a800640 1 --2- 172.21.15.100:0/1378218089 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd1e0078d70 0x7fd1e007b280 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:30.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.315+0000 7fd20a800640 1 --2- 172.21.15.100:0/1378218089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd204160310 0x7fd2041f72c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:30.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.315+0000 7fd20a800640 1 --2- 172.21.15.100:0/1378218089 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd20415d760 0x7fd2041f6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:30.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.315+0000 7fd20a800640 1 --2- 172.21.15.100:0/1378218089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2040ad0e0 0x7fd2041422d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:30.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.315+0000 7fd20a800640 1 -- 172.21.15.100:0/1378218089 >> 172.21.15.100:0/1378218089 conn(0x7fd20406d730 msgr2=0x7fd204160720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:30.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.315+0000 7fd20a800640 1 -- 172.21.15.100:0/1378218089 shutdown_connections 2024-09-17T04:25:30.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.315+0000 7fd20a800640 1 -- 172.21.15.100:0/1378218089 wait complete. 2024-09-17T04:25:30.342 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:25:30.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.561+0000 7f6143e00640 1 -- 172.21.15.100:0/2843609090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f614415a0f0 msgr2=0x7f614415d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:30.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.561+0000 7f6143e00640 1 --2- 172.21.15.100:0/2843609090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f614415a0f0 0x7f614415d670 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f613c009cd0 tx=0x7f613c02aa70 comp rx=0 tx=0).stop 2024-09-17T04:25:30.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.561+0000 7f6143e00640 1 -- 172.21.15.100:0/2843609090 shutdown_connections 2024-09-17T04:25:30.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.561+0000 7f6143e00640 1 --2- 172.21.15.100:0/2843609090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f614415a0f0 0x7f614415d670 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:30.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.561+0000 7f6143e00640 1 --2- 172.21.15.100:0/2843609090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61440af170 0x7f61440b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:30.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.561+0000 7f6143e00640 1 -- 172.21.15.100:0/2843609090 >> 172.21.15.100:0/2843609090 conn(0x7f61440b1e40 msgr2=0x7f61440b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:30.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.561+0000 7f6143e00640 1 -- 172.21.15.100:0/2843609090 shutdown_connections 2024-09-17T04:25:30.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.561+0000 7f6143e00640 1 -- 172.21.15.100:0/2843609090 wait complete. 2024-09-17T04:25:30.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.563+0000 7f6143e00640 1 Processor -- start 2024-09-17T04:25:30.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.564+0000 7f6143e00640 1 -- start start 2024-09-17T04:25:30.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.565+0000 7f6143e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61440ad9c0 0x7f6144154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:30.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.565+0000 7f6143e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61440af170 0x7f6144073fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:30.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.565+0000 7f6143e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f614415a0f0 0x7f6144074520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:30.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.565+0000 7f6143e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f614406f410 con 0x7f61440af170 2024-09-17T04:25:30.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.565+0000 7f6143e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f614406f290 con 0x7f61440ad9c0 2024-09-17T04:25:30.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.565+0000 7f6143e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f614406f590 con 0x7f614415a0f0 2024-09-17T04:25:30.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.565+0000 7f613be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61440af170 0x7f6144073fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:30.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.565+0000 7f613be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61440af170 0x7f6144073fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48638/0 (socket says 172.21.15.100:48638) 2024-09-17T04:25:30.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.565+0000 7f613be00640 1 -- 172.21.15.100:0/3430400891 learned_addr learned my addr 172.21.15.100:0/3430400891 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:30.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.566+0000 7f6143400640 1 --2- 172.21.15.100:0/3430400891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f614415a0f0 0x7f6144074520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:30.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.566+0000 7f6142a00640 1 --2- 172.21.15.100:0/3430400891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61440ad9c0 0x7f6144154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:30.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.566+0000 7f613be00640 1 -- 172.21.15.100:0/3430400891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f614415a0f0 msgr2=0x7f6144074520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:30.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.566+0000 7f613be00640 1 --2- 172.21.15.100:0/3430400891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f614415a0f0 0x7f6144074520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:30.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.566+0000 7f613be00640 1 -- 172.21.15.100:0/3430400891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61440ad9c0 msgr2=0x7f6144154f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:30.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.566+0000 7f613be00640 1 --2- 172.21.15.100:0/3430400891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61440ad9c0 0x7f6144154f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:30.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.566+0000 7f613be00640 1 -- 172.21.15.100:0/3430400891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61440c9320 con 0x7f61440af170 2024-09-17T04:25:30.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.566+0000 7f6143400640 1 --2- 172.21.15.100:0/3430400891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f614415a0f0 0x7f6144074520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:25:30.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.566+0000 7f6142a00640 1 --2- 172.21.15.100:0/3430400891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61440ad9c0 0x7f6144154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:25:30.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.566+0000 7f613be00640 1 --2- 172.21.15.100:0/3430400891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61440af170 0x7f6144073fe0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f612c00d700 tx=0x7f612c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:30.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.566+0000 7f613aa00640 1 -- 172.21.15.100:0/3430400891 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612c017070 con 0x7f61440af170 2024-09-17T04:25:30.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.566+0000 7f613aa00640 1 -- 172.21.15.100:0/3430400891 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f612c009b10 con 0x7f61440af170 2024-09-17T04:25:30.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.566+0000 7f613aa00640 1 -- 172.21.15.100:0/3430400891 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612c010460 con 0x7f61440af170 2024-09-17T04:25:30.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.566+0000 7f6143e00640 1 -- 172.21.15.100:0/3430400891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61441f6330 con 0x7f61440af170 2024-09-17T04:25:30.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.567+0000 7f6143e00640 1 -- 172.21.15.100:0/3430400891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61441579b0 con 0x7f61440af170 2024-09-17T04:25:30.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.567+0000 7f6143e00640 1 -- 172.21.15.100:0/3430400891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f614406fab0 con 0x7f61440af170 2024-09-17T04:25:30.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.569+0000 7f613aa00640 1 -- 172.21.15.100:0/3430400891 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f612c01a070 con 0x7f61440af170 2024-09-17T04:25:30.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.570+0000 7f613aa00640 1 --2- 172.21.15.100:0/3430400891 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6120078d40 0x7f612007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:30.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.570+0000 7f613aa00640 1 -- 172.21.15.100:0/3430400891 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f612c09b3f0 con 0x7f61440af170 2024-09-17T04:25:30.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.570+0000 7f6142a00640 1 --2- 172.21.15.100:0/3430400891 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6120078d40 0x7f612007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:30.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.571+0000 7f6142a00640 1 --2- 172.21.15.100:0/3430400891 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6120078d40 0x7f612007b250 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f61340016a0 tx=0x7f6134009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:30.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.574+0000 7f613aa00640 1 -- 172.21.15.100:0/3430400891 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f612c063400 con 0x7f61440af170 2024-09-17T04:25:30.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.793+0000 7f6143e00640 1 -- 172.21.15.100:0/3430400891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61441577a0 con 0x7f61440af170 2024-09-17T04:25:30.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.794+0000 7f613aa00640 1 -- 172.21.15.100:0/3430400891 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5232 (secure 0 0 0) 0x7f612c062d00 con 0x7f61440af170 2024-09-17T04:25:30.794 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:30.795 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":18,"btime":"2024-09-17T04:25:23:356418+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4667,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2259943840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":2259943840},{"type":"v1","addr":"172.21.15.100:6835","nonce":2259943840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4670,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/1748167554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1748167554},{"type":"v1","addr":"172.21.15.100:6834","nonce":1748167554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4854,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3159351871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":3159351871},{"type":"v1","addr":"172.21.15.184:6836","nonce":3159351871}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:07.565710+0000","modified":"2024-09-17T04:25:10.595342+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4857},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4857":{"gid":4857,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/294839954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":294839954},{"type":"v1","addr":"172.21.15.184:6837","nonce":294839954}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4857,"qdb_cluster":[4857]},"id":2}]} 2024-09-17T04:25:30.795 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 18 2024-09-17T04:25:30.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.796+0000 7f6143e00640 1 -- 172.21.15.100:0/3430400891 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6120078d40 msgr2=0x7f612007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:30.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.796+0000 7f6143e00640 1 --2- 172.21.15.100:0/3430400891 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6120078d40 0x7f612007b250 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f61340016a0 tx=0x7f6134009040 comp rx=0 tx=0).stop 2024-09-17T04:25:30.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.796+0000 7f6143e00640 1 -- 172.21.15.100:0/3430400891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61440af170 msgr2=0x7f6144073fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:30.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.796+0000 7f6143e00640 1 --2- 172.21.15.100:0/3430400891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61440af170 0x7f6144073fe0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f612c00d700 tx=0x7f612c00dbc0 comp rx=0 tx=0).stop 2024-09-17T04:25:30.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.797+0000 7f6143e00640 1 -- 172.21.15.100:0/3430400891 shutdown_connections 2024-09-17T04:25:30.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.797+0000 7f6143e00640 1 --2- 172.21.15.100:0/3430400891 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6120078d40 0x7f612007b250 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:30.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.797+0000 7f6143e00640 1 --2- 172.21.15.100:0/3430400891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f614415a0f0 0x7f6144074520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:30.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.797+0000 7f6143e00640 1 --2- 172.21.15.100:0/3430400891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61440af170 0x7f6144073fe0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:30.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.797+0000 7f6143e00640 1 --2- 172.21.15.100:0/3430400891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61440ad9c0 0x7f6144154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:30.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.797+0000 7f6143e00640 1 -- 172.21.15.100:0/3430400891 >> 172.21.15.100:0/3430400891 conn(0x7f61440b1e40 msgr2=0x7f6144070530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:30.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.797+0000 7f6143e00640 1 -- 172.21.15.100:0/3430400891 shutdown_connections 2024-09-17T04:25:30.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:30.797+0000 7f6143e00640 1 -- 172.21.15.100:0/3430400891 wait complete. 2024-09-17T04:25:30.823 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:25:31.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.042+0000 7f532a000640 1 -- 172.21.15.100:0/3883048919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53240aebc0 msgr2=0x7f5324152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:31.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.042+0000 7f532a000640 1 --2- 172.21.15.100:0/3883048919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53240aebc0 0x7f5324152920 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5314009830 tx=0x7f531402a470 comp rx=0 tx=0).stop 2024-09-17T04:25:31.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.042+0000 7f532a000640 1 -- 172.21.15.100:0/3883048919 shutdown_connections 2024-09-17T04:25:31.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.042+0000 7f532a000640 1 --2- 172.21.15.100:0/3883048919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53240b2750 0x7f532415f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:31.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.042+0000 7f532a000640 1 --2- 172.21.15.100:0/3883048919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53240aebc0 0x7f5324152920 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:31.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.042+0000 7f532a000640 1 -- 172.21.15.100:0/3883048919 >> 172.21.15.100:0/3883048919 conn(0x7f532406d730 msgr2=0x7f53240b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:31.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.042+0000 7f532a000640 1 -- 172.21.15.100:0/3883048919 shutdown_connections 2024-09-17T04:25:31.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.042+0000 7f532a000640 1 -- 172.21.15.100:0/3883048919 wait complete. 2024-09-17T04:25:31.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.045+0000 7f532a000640 1 Processor -- start 2024-09-17T04:25:31.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.045+0000 7f532a000640 1 -- start start 2024-09-17T04:25:31.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.046+0000 7f532a000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53240aebc0 0x7f53241f7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:31.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.046+0000 7f532a000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53240b2750 0x7f53241517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:31.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.046+0000 7f532a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f532415f910 0x7f53240afc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:31.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.046+0000 7f532a000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f532415c4a0 con 0x7f532415f910 2024-09-17T04:25:31.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.046+0000 7f532a000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f53240b4a20 con 0x7f53240aebc0 2024-09-17T04:25:31.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.046+0000 7f532a000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f532415c620 con 0x7f53240b2750 2024-09-17T04:25:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.046+0000 7f5323e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f532415f910 0x7f53240afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.046+0000 7f5323e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f532415f910 0x7f53240afc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48642/0 (socket says 172.21.15.100:48642) 2024-09-17T04:25:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.046+0000 7f5323e00640 1 -- 172.21.15.100:0/538660262 learned_addr learned my addr 172.21.15.100:0/538660262 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.047+0000 7f531aa00640 1 --2- 172.21.15.100:0/538660262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53240b2750 0x7f53241517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.047+0000 7f5323400640 1 --2- 172.21.15.100:0/538660262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53240aebc0 0x7f53241f7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.047+0000 7f5323e00640 1 -- 172.21.15.100:0/538660262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53240b2750 msgr2=0x7f53241517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.047+0000 7f5323e00640 1 --2- 172.21.15.100:0/538660262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53240b2750 0x7f53241517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.047+0000 7f5323e00640 1 -- 172.21.15.100:0/538660262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53240aebc0 msgr2=0x7f53241f7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.047+0000 7f5323e00640 1 --2- 172.21.15.100:0/538660262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53240aebc0 0x7f53241f7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.047+0000 7f5323e00640 1 -- 172.21.15.100:0/538660262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53240cc690 con 0x7f532415f910 2024-09-17T04:25:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.047+0000 7f5323e00640 1 --2- 172.21.15.100:0/538660262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f532415f910 0x7f53240afc70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f531c00c730 tx=0x7f531c00cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:31.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.047+0000 7f531be00640 1 -- 172.21.15.100:0/538660262 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c003b70 con 0x7f532415f910 2024-09-17T04:25:31.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.047+0000 7f531be00640 1 -- 172.21.15.100:0/538660262 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f531c003d10 con 0x7f532415f910 2024-09-17T04:25:31.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.047+0000 7f531be00640 1 -- 172.21.15.100:0/538660262 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c0052d0 con 0x7f532415f910 2024-09-17T04:25:31.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.047+0000 7f532a000640 1 -- 172.21.15.100:0/538660262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53240cc970 con 0x7f532415f910 2024-09-17T04:25:31.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.048+0000 7f532a000640 1 -- 172.21.15.100:0/538660262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53241f4aa0 con 0x7f532415f910 2024-09-17T04:25:31.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.049+0000 7f532a000640 1 -- 172.21.15.100:0/538660262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f532415cb40 con 0x7f532415f910 2024-09-17T04:25:31.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.050+0000 7f531be00640 1 -- 172.21.15.100:0/538660262 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f531c005470 con 0x7f532415f910 2024-09-17T04:25:31.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.051+0000 7f531be00640 1 --2- 172.21.15.100:0/538660262 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5300078d40 0x7f530007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:31.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.051+0000 7f531be00640 1 -- 172.21.15.100:0/538660262 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f531c09adb0 con 0x7f532415f910 2024-09-17T04:25:31.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.051+0000 7f5323400640 1 --2- 172.21.15.100:0/538660262 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5300078d40 0x7f530007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:31.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.052+0000 7f5323400640 1 --2- 172.21.15.100:0/538660262 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5300078d40 0x7f530007b250 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5314034000 tx=0x7f5314004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:31.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.055+0000 7f531be00640 1 -- 172.21.15.100:0/538660262 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f531c09f070 con 0x7f532415f910 2024-09-17T04:25:31.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.242+0000 7f532a000640 1 -- 172.21.15.100:0/538660262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53240cc410 con 0x7f532415f910 2024-09-17T04:25:31.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.243+0000 7f531be00640 1 -- 172.21.15.100:0/538660262 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16777 (secure 0 0 0) 0x7f53240cc410 con 0x7f532415f910 2024-09-17T04:25:31.243 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:31.244 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":36,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:25:09.097160+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:25:05.694905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T04:25:06.533348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:25:31.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.245+0000 7f532a000640 1 -- 172.21.15.100:0/538660262 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5300078d40 msgr2=0x7f530007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:31.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.245+0000 7f532a000640 1 --2- 172.21.15.100:0/538660262 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5300078d40 0x7f530007b250 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5314034000 tx=0x7f5314004730 comp rx=0 tx=0).stop 2024-09-17T04:25:31.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.245+0000 7f532a000640 1 -- 172.21.15.100:0/538660262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f532415f910 msgr2=0x7f53240afc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:31.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.245+0000 7f532a000640 1 --2- 172.21.15.100:0/538660262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f532415f910 0x7f53240afc70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f531c00c730 tx=0x7f531c00cb30 comp rx=0 tx=0).stop 2024-09-17T04:25:31.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.245+0000 7f532a000640 1 -- 172.21.15.100:0/538660262 shutdown_connections 2024-09-17T04:25:31.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.245+0000 7f532a000640 1 --2- 172.21.15.100:0/538660262 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5300078d40 0x7f530007b250 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:31.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.245+0000 7f532a000640 1 --2- 172.21.15.100:0/538660262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f532415f910 0x7f53240afc70 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:31.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.245+0000 7f532a000640 1 --2- 172.21.15.100:0/538660262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53240b2750 0x7f53241517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:31.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.245+0000 7f532a000640 1 --2- 172.21.15.100:0/538660262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53240aebc0 0x7f53241f7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:31.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.245+0000 7f532a000640 1 -- 172.21.15.100:0/538660262 >> 172.21.15.100:0/538660262 conn(0x7f532406d730 msgr2=0x7f53240b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:31.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.245+0000 7f532a000640 1 -- 172.21.15.100:0/538660262 shutdown_connections 2024-09-17T04:25:31.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.245+0000 7f532a000640 1 -- 172.21.15.100:0/538660262 wait complete. 2024-09-17T04:25:31.268 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T04:25:31.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.490+0000 7fd0c3e00640 1 -- 172.21.15.100:0/1541379005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0c40cfc30 msgr2=0x7fd0c415c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:31.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.490+0000 7fd0c3e00640 1 --2- 172.21.15.100:0/1541379005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0c40cfc30 0x7fd0c415c2e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac009900 tx=0x7fd0ac02a610 comp rx=0 tx=0).stop 2024-09-17T04:25:31.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.490+0000 7fd0c3e00640 1 -- 172.21.15.100:0/1541379005 shutdown_connections 2024-09-17T04:25:31.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.490+0000 7fd0c3e00640 1 --2- 172.21.15.100:0/1541379005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0c40cfc30 0x7fd0c415c2e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:31.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.490+0000 7fd0c3e00640 1 -- 172.21.15.100:0/1541379005 >> 172.21.15.100:0/1541379005 conn(0x7fd0c40d1470 msgr2=0x7fd0c4073670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:31.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.490+0000 7fd0c3e00640 1 -- 172.21.15.100:0/1541379005 shutdown_connections 2024-09-17T04:25:31.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.490+0000 7fd0c3e00640 1 -- 172.21.15.100:0/1541379005 wait complete. 2024-09-17T04:25:31.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.492+0000 7fd0c3e00640 1 Processor -- start 2024-09-17T04:25:31.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.492+0000 7fd0c3e00640 1 -- start start 2024-09-17T04:25:31.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.494+0000 7fd0c3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0c40705e0 0x7fd0c40cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:31.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.494+0000 7fd0c3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0c40cfc30 0x7fd0c41f33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:31.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.494+0000 7fd0c3e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0c415c820 0x7fd0c4150920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:31.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.494+0000 7fd0c3e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c4071430 con 0x7fd0c40705e0 2024-09-17T04:25:31.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.494+0000 7fd0c3e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c40712b0 con 0x7fd0c40cfc30 2024-09-17T04:25:31.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.494+0000 7fd0c3e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd0c40715b0 con 0x7fd0c415c820 2024-09-17T04:25:31.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.494+0000 7fd0c2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0c40705e0 0x7fd0c40cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:31.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.494+0000 7fd0c2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0c40705e0 0x7fd0c40cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48664/0 (socket says 172.21.15.100:48664) 2024-09-17T04:25:31.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.494+0000 7fd0c2a00640 1 -- 172.21.15.100:0/2928319231 learned_addr learned my addr 172.21.15.100:0/2928319231 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:31.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.494+0000 7fd0bbe00640 1 --2- 172.21.15.100:0/2928319231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0c40cfc30 0x7fd0c41f33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:31.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.494+0000 7fd0c3400640 1 --2- 172.21.15.100:0/2928319231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0c415c820 0x7fd0c4150920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:31.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.494+0000 7fd0c2a00640 1 -- 172.21.15.100:0/2928319231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0c415c820 msgr2=0x7fd0c4150920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:31.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.494+0000 7fd0c2a00640 1 --2- 172.21.15.100:0/2928319231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0c415c820 0x7fd0c4150920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:31.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.494+0000 7fd0c2a00640 1 -- 172.21.15.100:0/2928319231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0c40cfc30 msgr2=0x7fd0c41f33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:31.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.494+0000 7fd0c2a00640 1 --2- 172.21.15.100:0/2928319231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0c40cfc30 0x7fd0c41f33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:31.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.494+0000 7fd0c2a00640 1 -- 172.21.15.100:0/2928319231 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c40c6b00 con 0x7fd0c40705e0 2024-09-17T04:25:31.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.495+0000 7fd0c2a00640 1 --2- 172.21.15.100:0/2928319231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0c40705e0 0x7fd0c40cc0c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400b420 tx=0x7fd0b400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:31.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.495+0000 7fd0baa00640 1 -- 172.21.15.100:0/2928319231 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4011830 con 0x7fd0c40705e0 2024-09-17T04:25:31.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.495+0000 7fd0baa00640 1 -- 172.21.15.100:0/2928319231 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0b4016070 con 0x7fd0c40705e0 2024-09-17T04:25:31.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.495+0000 7fd0c3e00640 1 -- 172.21.15.100:0/2928319231 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0c40c6de0 con 0x7fd0c40705e0 2024-09-17T04:25:31.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.495+0000 7fd0baa00640 1 -- 172.21.15.100:0/2928319231 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4011d60 con 0x7fd0c40705e0 2024-09-17T04:25:31.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.495+0000 7fd0c3e00640 1 -- 172.21.15.100:0/2928319231 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0c4159ca0 con 0x7fd0c40705e0 2024-09-17T04:25:31.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.496+0000 7fd0c3e00640 1 -- 172.21.15.100:0/2928319231 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0c415a0e0 con 0x7fd0c40705e0 2024-09-17T04:25:31.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.498+0000 7fd0baa00640 1 -- 172.21.15.100:0/2928319231 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fd0b401b460 con 0x7fd0c40705e0 2024-09-17T04:25:31.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.498+0000 7fd0baa00640 1 --2- 172.21.15.100:0/2928319231 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd0a0078d40 0x7fd0a007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:31.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.499+0000 7fd0baa00640 1 -- 172.21.15.100:0/2928319231 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fd0b409ac50 con 0x7fd0c40705e0 2024-09-17T04:25:31.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.499+0000 7fd0bbe00640 1 --2- 172.21.15.100:0/2928319231 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd0a0078d40 0x7fd0a007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:31.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.500+0000 7fd0bbe00640 1 --2- 172.21.15.100:0/2928319231 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd0a0078d40 0x7fd0a007b250 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac033000 tx=0x7fd0ac004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:31.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.503+0000 7fd0baa00640 1 -- 172.21.15.100:0/2928319231 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd0b40a0020 con 0x7fd0c40705e0 2024-09-17T04:25:31.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.721+0000 7fd0c3e00640 1 -- 172.21.15.100:0/2928319231 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd0c415c2e0 con 0x7fd0c40705e0 2024-09-17T04:25:31.721 INFO:tasks.ceph.mon.a.smithi100.stderr:2024-09-17T04:25:31.721+0000 7fd3d0a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T04:25:31.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.723+0000 7fd0baa00640 1 -- 172.21.15.100:0/2928319231 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7fd0b4020020 con 0x7fd0c40705e0 2024-09-17T04:25:31.723 INFO:teuthology.orchestra.run.smithi100.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T04:25:31.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.726+0000 7fd0c3e00640 1 -- 172.21.15.100:0/2928319231 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd0a0078d40 msgr2=0x7fd0a007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:31.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.726+0000 7fd0c3e00640 1 --2- 172.21.15.100:0/2928319231 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd0a0078d40 0x7fd0a007b250 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac033000 tx=0x7fd0ac004730 comp rx=0 tx=0).stop 2024-09-17T04:25:31.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.726+0000 7fd0c3e00640 1 -- 172.21.15.100:0/2928319231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0c40705e0 msgr2=0x7fd0c40cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:31.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.726+0000 7fd0c3e00640 1 --2- 172.21.15.100:0/2928319231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0c40705e0 0x7fd0c40cc0c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400b420 tx=0x7fd0b400b8e0 comp rx=0 tx=0).stop 2024-09-17T04:25:31.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.726+0000 7fd0c3e00640 1 -- 172.21.15.100:0/2928319231 shutdown_connections 2024-09-17T04:25:31.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.726+0000 7fd0c3e00640 1 --2- 172.21.15.100:0/2928319231 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd0a0078d40 0x7fd0a007b250 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:31.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.726+0000 7fd0c3e00640 1 --2- 172.21.15.100:0/2928319231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0c415c820 0x7fd0c4150920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:31.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.726+0000 7fd0c3e00640 1 --2- 172.21.15.100:0/2928319231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0c40cfc30 0x7fd0c41f33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:31.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.726+0000 7fd0c3e00640 1 --2- 172.21.15.100:0/2928319231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0c40705e0 0x7fd0c40cc0c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:31.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.726+0000 7fd0c3e00640 1 -- 172.21.15.100:0/2928319231 >> 172.21.15.100:0/2928319231 conn(0x7fd0c40d1470 msgr2=0x7fd0c406fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:31.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.727+0000 7fd0c3e00640 1 -- 172.21.15.100:0/2928319231 shutdown_connections 2024-09-17T04:25:31.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.727+0000 7fd0c3e00640 1 -- 172.21.15.100:0/2928319231 wait complete. 2024-09-17T04:25:31.751 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T04:25:31.865 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-09-17T04:25:31.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.977+0000 7f60f4800640 1 -- 172.21.15.100:0/506359151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60ec145a40 msgr2=0x7f60ec073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:31.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.977+0000 7f60f4800640 1 --2- 172.21.15.100:0/506359151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60ec145a40 0x7f60ec073540 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f60dc009830 tx=0x7f60dc02a640 comp rx=0 tx=0).stop 2024-09-17T04:25:31.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.978+0000 7f60f4800640 1 -- 172.21.15.100:0/506359151 shutdown_connections 2024-09-17T04:25:31.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.978+0000 7f60f4800640 1 --2- 172.21.15.100:0/506359151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60ec145a40 0x7f60ec073540 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:31.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.978+0000 7f60f4800640 1 -- 172.21.15.100:0/506359151 >> 172.21.15.100:0/506359151 conn(0x7f60ec06d1c0 msgr2=0x7f60ec06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:31.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.978+0000 7f60f4800640 1 -- 172.21.15.100:0/506359151 shutdown_connections 2024-09-17T04:25:31.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.978+0000 7f60f4800640 1 -- 172.21.15.100:0/506359151 wait complete. 2024-09-17T04:25:31.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.980+0000 7f60f4800640 1 Processor -- start 2024-09-17T04:25:31.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.980+0000 7f60f4800640 1 -- start start 2024-09-17T04:25:31.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.981+0000 7f60f4800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60ec073a80 0x7f60ec1607e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:31.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.981+0000 7f60f4800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60ec145550 0x7f60ec15af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:31.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.981+0000 7f60f4800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ec145a40 0x7f60ec15b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:31.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.981+0000 7f60f4800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec0ce350 con 0x7f60ec145a40 2024-09-17T04:25:31.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.981+0000 7f60f4800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec0ce1d0 con 0x7f60ec145550 2024-09-17T04:25:31.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.981+0000 7f60f4800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f60ec0ce4d0 con 0x7f60ec073a80 2024-09-17T04:25:31.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.982+0000 7f60f2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ec145a40 0x7f60ec15b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:31.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.982+0000 7f60f2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ec145a40 0x7f60ec15b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48678/0 (socket says 172.21.15.100:48678) 2024-09-17T04:25:31.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.982+0000 7f60f2800640 1 -- 172.21.15.100:0/1941596991 learned_addr learned my addr 172.21.15.100:0/1941596991 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:31.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.982+0000 7f60f1e00640 1 --2- 172.21.15.100:0/1941596991 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60ec073a80 0x7f60ec1607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:31.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.982+0000 7f60f1400640 1 --2- 172.21.15.100:0/1941596991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60ec145550 0x7f60ec15af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:31.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.982+0000 7f60f2800640 1 -- 172.21.15.100:0/1941596991 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60ec073a80 msgr2=0x7f60ec1607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:31.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.982+0000 7f60f2800640 1 --2- 172.21.15.100:0/1941596991 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60ec073a80 0x7f60ec1607e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:31.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.982+0000 7f60f2800640 1 -- 172.21.15.100:0/1941596991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60ec145550 msgr2=0x7f60ec15af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:31.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.982+0000 7f60f2800640 1 --2- 172.21.15.100:0/1941596991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60ec145550 0x7f60ec15af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:31.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.982+0000 7f60f2800640 1 -- 172.21.15.100:0/1941596991 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ec1e7230 con 0x7f60ec145a40 2024-09-17T04:25:31.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.982+0000 7f60f2800640 1 --2- 172.21.15.100:0/1941596991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ec145a40 0x7f60ec15b4a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f60e800c9f0 tx=0x7f60e800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:31.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.983+0000 7f60e6000640 1 -- 172.21.15.100:0/1941596991 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e8016020 con 0x7f60ec145a40 2024-09-17T04:25:31.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.983+0000 7f60e6000640 1 -- 172.21.15.100:0/1941596991 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60e800ac60 con 0x7f60ec145a40 2024-09-17T04:25:31.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.983+0000 7f60f4800640 1 -- 172.21.15.100:0/1941596991 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60ec1e7510 con 0x7f60ec145a40 2024-09-17T04:25:31.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.983+0000 7f60e6000640 1 -- 172.21.15.100:0/1941596991 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e8012680 con 0x7f60ec145a40 2024-09-17T04:25:31.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.983+0000 7f60f4800640 1 -- 172.21.15.100:0/1941596991 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60ec1e7ad0 con 0x7f60ec145a40 2024-09-17T04:25:31.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.984+0000 7f60f4800640 1 -- 172.21.15.100:0/1941596991 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60ec073540 con 0x7f60ec145a40 2024-09-17T04:25:31.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.985+0000 7f60e6000640 1 -- 172.21.15.100:0/1941596991 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f60e8019020 con 0x7f60ec145a40 2024-09-17T04:25:31.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.986+0000 7f60e6000640 1 --2- 172.21.15.100:0/1941596991 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f60d0078d40 0x7f60d007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:31.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.986+0000 7f60e6000640 1 -- 172.21.15.100:0/1941596991 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5795+0+0 (secure 0 0 0) 0x7f60e809b380 con 0x7f60ec145a40 2024-09-17T04:25:31.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.987+0000 7f60f1e00640 1 --2- 172.21.15.100:0/1941596991 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f60d0078d40 0x7f60d007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:31.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.987+0000 7f60f1e00640 1 --2- 172.21.15.100:0/1941596991 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f60d0078d40 0x7f60d007b250 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f60e0000f60 tx=0x7f60e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:31.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:31.991+0000 7f60e6000640 1 -- 172.21.15.100:0/1941596991 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f60e8020030 con 0x7f60ec145a40 2024-09-17T04:25:32.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.216+0000 7f60f4800640 1 -- 172.21.15.100:0/1941596991 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f60ec1448c0 con 0x7f60ec145a40 2024-09-17T04:25:32.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.729+0000 7f60e6000640 1 -- 172.21.15.100:0/1941596991 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7f60ec073e90 con 0x7f60ec145a40 2024-09-17T04:25:32.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.731+0000 7f60f4800640 1 -- 172.21.15.100:0/1941596991 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f60d0078d40 msgr2=0x7f60d007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:32.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.731+0000 7f60f4800640 1 --2- 172.21.15.100:0/1941596991 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f60d0078d40 0x7f60d007b250 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f60e0000f60 tx=0x7f60e0009040 comp rx=0 tx=0).stop 2024-09-17T04:25:32.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.731+0000 7f60f4800640 1 -- 172.21.15.100:0/1941596991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ec145a40 msgr2=0x7f60ec15b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:32.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.731+0000 7f60f4800640 1 --2- 172.21.15.100:0/1941596991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ec145a40 0x7f60ec15b4a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f60e800c9f0 tx=0x7f60e800ceb0 comp rx=0 tx=0).stop 2024-09-17T04:25:32.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.731+0000 7f60f4800640 1 -- 172.21.15.100:0/1941596991 shutdown_connections 2024-09-17T04:25:32.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.731+0000 7f60f4800640 1 --2- 172.21.15.100:0/1941596991 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f60d0078d40 0x7f60d007b250 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:32.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.731+0000 7f60f4800640 1 --2- 172.21.15.100:0/1941596991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ec145a40 0x7f60ec15b4a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:32.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.731+0000 7f60f4800640 1 --2- 172.21.15.100:0/1941596991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60ec145550 0x7f60ec15af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:32.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.731+0000 7f60f4800640 1 --2- 172.21.15.100:0/1941596991 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60ec073a80 0x7f60ec1607e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:32.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.731+0000 7f60f4800640 1 -- 172.21.15.100:0/1941596991 >> 172.21.15.100:0/1941596991 conn(0x7f60ec06d1c0 msgr2=0x7f60ec140240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:32.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.732+0000 7f60f4800640 1 -- 172.21.15.100:0/1941596991 shutdown_connections 2024-09-17T04:25:32.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.732+0000 7f60f4800640 1 -- 172.21.15.100:0/1941596991 wait complete. 2024-09-17T04:25:32.756 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T04:25:32.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.982+0000 7f9773e00640 1 -- 172.21.15.100:0/4054852447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97740b3ec0 msgr2=0x7f97740b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:32.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.982+0000 7f9773e00640 1 --2- 172.21.15.100:0/4054852447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97740b3ec0 0x7f97740b47c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f977406e750 tx=0x7f976402a4d0 comp rx=0 tx=0).stop 2024-09-17T04:25:32.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.982+0000 7f9773e00640 1 -- 172.21.15.100:0/4054852447 shutdown_connections 2024-09-17T04:25:32.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.982+0000 7f9773e00640 1 --2- 172.21.15.100:0/4054852447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97741532e0 0x7f97741536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:32.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.982+0000 7f9773e00640 1 --2- 172.21.15.100:0/4054852447 >> v2:172.21.15.184:3300/0 conn(0x7f97740705b0 0x7f97740af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:32.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.982+0000 7f9773e00640 1 --2- 172.21.15.100:0/4054852447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97740b3ec0 0x7f97740b47c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:32.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.982+0000 7f9773e00640 1 -- 172.21.15.100:0/4054852447 >> 172.21.15.100:0/4054852447 conn(0x7f97740b1dd0 msgr2=0x7f97740b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:32.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.982+0000 7f9773e00640 1 -- 172.21.15.100:0/4054852447 shutdown_connections 2024-09-17T04:25:32.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.983+0000 7f9773e00640 1 -- 172.21.15.100:0/4054852447 wait complete. 2024-09-17T04:25:32.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.985+0000 7f9773e00640 1 Processor -- start 2024-09-17T04:25:32.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.985+0000 7f9773e00640 1 -- start start 2024-09-17T04:25:32.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.986+0000 7f9773e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97740705b0 0x7f97741587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:32.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.986+0000 7f9773e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97740b3ec0 0x7f97740749e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:32.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.986+0000 7f9773e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97741532e0 0x7f9774147950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:32.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.986+0000 7f9773e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9774160860 con 0x7f97740b3ec0 2024-09-17T04:25:32.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.986+0000 7f9773e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f97741606e0 con 0x7f97741532e0 2024-09-17T04:25:32.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.986+0000 7f9773e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f97741609e0 con 0x7f97740705b0 2024-09-17T04:25:32.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.986+0000 7f976a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97740b3ec0 0x7f97740749e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:32.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.987+0000 7f976a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97740b3ec0 0x7f97740749e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48696/0 (socket says 172.21.15.100:48696) 2024-09-17T04:25:32.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.987+0000 7f976a000640 1 -- 172.21.15.100:0/3377548651 learned_addr learned my addr 172.21.15.100:0/3377548651 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:32.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.987+0000 7f9772a00640 1 --2- 172.21.15.100:0/3377548651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97740705b0 0x7f97741587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:32.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.987+0000 7f9773400640 1 --2- 172.21.15.100:0/3377548651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97741532e0 0x7f9774147950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:32.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.987+0000 7f976a000640 1 -- 172.21.15.100:0/3377548651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97740705b0 msgr2=0x7f97741587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:32.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.987+0000 7f976a000640 1 --2- 172.21.15.100:0/3377548651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97740705b0 0x7f97741587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:32.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.987+0000 7f976a000640 1 -- 172.21.15.100:0/3377548651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97741532e0 msgr2=0x7f9774147950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:32.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.987+0000 7f976a000640 1 --2- 172.21.15.100:0/3377548651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97741532e0 0x7f9774147950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:32.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.987+0000 7f976a000640 1 -- 172.21.15.100:0/3377548651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9774157700 con 0x7f97740b3ec0 2024-09-17T04:25:32.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.987+0000 7f9773400640 1 --2- 172.21.15.100:0/3377548651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97741532e0 0x7f9774147950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:32.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.987+0000 7f976a000640 1 --2- 172.21.15.100:0/3377548651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97740b3ec0 0x7f97740749e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f975c00c940 tx=0x7f975c00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:32.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.988+0000 7f976b400640 1 -- 172.21.15.100:0/3377548651 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c007bb0 con 0x7f97740b3ec0 2024-09-17T04:25:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.988+0000 7f976b400640 1 -- 172.21.15.100:0/3377548651 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f975c007e40 con 0x7f97740b3ec0 2024-09-17T04:25:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.988+0000 7f976b400640 1 -- 172.21.15.100:0/3377548651 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c00f670 con 0x7f97740b3ec0 2024-09-17T04:25:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.988+0000 7f9773e00640 1 -- 172.21.15.100:0/3377548651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97741579e0 con 0x7f97740b3ec0 2024-09-17T04:25:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.988+0000 7f9773e00640 1 -- 172.21.15.100:0/3377548651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9774157fa0 con 0x7f97740b3ec0 2024-09-17T04:25:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.989+0000 7f9773e00640 1 -- 172.21.15.100:0/3377548651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97740b3220 con 0x7f97740b3ec0 2024-09-17T04:25:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.990+0000 7f976b400640 1 -- 172.21.15.100:0/3377548651 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f975c00f810 con 0x7f97740b3ec0 2024-09-17T04:25:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.991+0000 7f976b400640 1 --2- 172.21.15.100:0/3377548651 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9750078d40 0x7f975007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.991+0000 7f976b400640 1 -- 172.21.15.100:0/3377548651 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5795+0+0 (secure 0 0 0) 0x7f975c09b9c0 con 0x7f97740b3ec0 2024-09-17T04:25:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.991+0000 7f9772a00640 1 --2- 172.21.15.100:0/3377548651 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9750078d40 0x7f975007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:32.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.992+0000 7f9772a00640 1 --2- 172.21.15.100:0/3377548651 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9750078d40 0x7f975007b250 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9764033000 tx=0x7f9764004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:32.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:32.995+0000 7f976b400640 1 -- 172.21.15.100:0/3377548651 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f975c014030 con 0x7f97740b3ec0 2024-09-17T04:25:33.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:33.184+0000 7f9773e00640 1 -- 172.21.15.100:0/3377548651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f97740af4c0 con 0x7f97740b3ec0 2024-09-17T04:25:34.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.090+0000 7f976b400640 1 -- 172.21.15.100:0/3377548651 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v38) ==== 181+0+0 (secure 0 0 0) 0x7f975c063970 con 0x7f97740b3ec0 2024-09-17T04:25:34.091 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' removed 2024-09-17T04:25:34.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.094+0000 7f9773e00640 1 -- 172.21.15.100:0/3377548651 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9750078d40 msgr2=0x7f975007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:34.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.094+0000 7f9773e00640 1 --2- 172.21.15.100:0/3377548651 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9750078d40 0x7f975007b250 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9764033000 tx=0x7f9764004730 comp rx=0 tx=0).stop 2024-09-17T04:25:34.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.094+0000 7f9773e00640 1 -- 172.21.15.100:0/3377548651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97740b3ec0 msgr2=0x7f97740749e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:34.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.094+0000 7f9773e00640 1 --2- 172.21.15.100:0/3377548651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97740b3ec0 0x7f97740749e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f975c00c940 tx=0x7f975c00ce00 comp rx=0 tx=0).stop 2024-09-17T04:25:34.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.094+0000 7f9773e00640 1 -- 172.21.15.100:0/3377548651 shutdown_connections 2024-09-17T04:25:34.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.094+0000 7f9773e00640 1 --2- 172.21.15.100:0/3377548651 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9750078d40 0x7f975007b250 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:34.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.094+0000 7f9773e00640 1 --2- 172.21.15.100:0/3377548651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97741532e0 0x7f9774147950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:34.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.094+0000 7f9773e00640 1 --2- 172.21.15.100:0/3377548651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97740b3ec0 0x7f97740749e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:34.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.094+0000 7f9773e00640 1 --2- 172.21.15.100:0/3377548651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97740705b0 0x7f97741587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:34.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.095+0000 7f9773e00640 1 -- 172.21.15.100:0/3377548651 >> 172.21.15.100:0/3377548651 conn(0x7f97740b1dd0 msgr2=0x7f97740b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:34.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.095+0000 7f9773e00640 1 -- 172.21.15.100:0/3377548651 shutdown_connections 2024-09-17T04:25:34.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.095+0000 7f9773e00640 1 -- 172.21.15.100:0/3377548651 wait complete. 2024-09-17T04:25:34.123 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T04:25:34.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.351+0000 7fb43c600640 1 -- 172.21.15.100:0/1881420754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4340cc830 msgr2=0x7fb4340b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:34.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.351+0000 7fb43c600640 1 --2- 172.21.15.100:0/1881420754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4340cc830 0x7fb4340b4620 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb434071040 tx=0x7fb42802a5f0 comp rx=0 tx=0).stop 2024-09-17T04:25:34.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.351+0000 7fb43c600640 1 -- 172.21.15.100:0/1881420754 shutdown_connections 2024-09-17T04:25:34.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.351+0000 7fb43c600640 1 --2- 172.21.15.100:0/1881420754 >> v2:172.21.15.184:3300/0 conn(0x7fb4340af100 0x7fb4340af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:34.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.351+0000 7fb43c600640 1 --2- 172.21.15.100:0/1881420754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4340cc830 0x7fb4340b4620 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:34.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.351+0000 7fb43c600640 1 -- 172.21.15.100:0/1881420754 >> 172.21.15.100:0/1881420754 conn(0x7fb4340b2540 msgr2=0x7fb4340d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:34.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.351+0000 7fb43c600640 1 -- 172.21.15.100:0/1881420754 shutdown_connections 2024-09-17T04:25:34.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.351+0000 7fb43c600640 1 -- 172.21.15.100:0/1881420754 wait complete. 2024-09-17T04:25:34.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.353+0000 7fb43c600640 1 Processor -- start 2024-09-17T04:25:34.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.353+0000 7fb43c600640 1 -- start start 2024-09-17T04:25:34.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.355+0000 7fb43c600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb43406cda0 0x7fb4341f3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:34.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.355+0000 7fb43c600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4340af100 0x7fb43415b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:34.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.355+0000 7fb43c600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4340cc830 0x7fb43415be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:34.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.355+0000 7fb43c600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb4340b3620 con 0x7fb43406cda0 2024-09-17T04:25:34.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.355+0000 7fb43c600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb4340b34a0 con 0x7fb4340cc830 2024-09-17T04:25:34.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.355+0000 7fb43c600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb4340b37a0 con 0x7fb4340af100 2024-09-17T04:25:34.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.355+0000 7fb439c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb43406cda0 0x7fb4341f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:34.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.355+0000 7fb439c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb43406cda0 0x7fb4341f3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48724/0 (socket says 172.21.15.100:48724) 2024-09-17T04:25:34.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.355+0000 7fb439c00640 1 -- 172.21.15.100:0/3744731269 learned_addr learned my addr 172.21.15.100:0/3744731269 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:34.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.355+0000 7fb43a600640 1 --2- 172.21.15.100:0/3744731269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4340cc830 0x7fb43415be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:34.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.355+0000 7fb439200640 1 --2- 172.21.15.100:0/3744731269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4340af100 0x7fb43415b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:34.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.355+0000 7fb439c00640 1 -- 172.21.15.100:0/3744731269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4340af100 msgr2=0x7fb43415b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:34.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.355+0000 7fb439c00640 1 --2- 172.21.15.100:0/3744731269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4340af100 0x7fb43415b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:34.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.355+0000 7fb439c00640 1 -- 172.21.15.100:0/3744731269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4340cc830 msgr2=0x7fb43415be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:34.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.355+0000 7fb439c00640 1 --2- 172.21.15.100:0/3744731269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4340cc830 0x7fb43415be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:34.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.355+0000 7fb439c00640 1 -- 172.21.15.100:0/3744731269 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4341f60b0 con 0x7fb43406cda0 2024-09-17T04:25:34.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.356+0000 7fb439c00640 1 --2- 172.21.15.100:0/3744731269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb43406cda0 0x7fb4341f3580 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb428034040 tx=0x7fb428003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:34.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.356+0000 7fb431600640 1 -- 172.21.15.100:0/3744731269 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42803e030 con 0x7fb43406cda0 2024-09-17T04:25:34.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.356+0000 7fb431600640 1 -- 172.21.15.100:0/3744731269 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb428003e00 con 0x7fb43406cda0 2024-09-17T04:25:34.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.356+0000 7fb43c600640 1 -- 172.21.15.100:0/3744731269 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4341f6390 con 0x7fb43406cda0 2024-09-17T04:25:34.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.356+0000 7fb431600640 1 -- 172.21.15.100:0/3744731269 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428035040 con 0x7fb43406cda0 2024-09-17T04:25:34.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.356+0000 7fb43c600640 1 -- 172.21.15.100:0/3744731269 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4341f6950 con 0x7fb43406cda0 2024-09-17T04:25:34.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.357+0000 7fb43c600640 1 -- 172.21.15.100:0/3744731269 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb43406a700 con 0x7fb43406cda0 2024-09-17T04:25:34.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.358+0000 7fb431600640 1 -- 172.21.15.100:0/3744731269 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fb428041030 con 0x7fb43406cda0 2024-09-17T04:25:34.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.360+0000 7fb431600640 1 --2- 172.21.15.100:0/3744731269 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb414078d40 0x7fb41407b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:34.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.360+0000 7fb431600640 1 -- 172.21.15.100:0/3744731269 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5357+0+0 (secure 0 0 0) 0x7fb4280bb810 con 0x7fb43406cda0 2024-09-17T04:25:34.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.360+0000 7fb439200640 1 --2- 172.21.15.100:0/3744731269 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb414078d40 0x7fb41407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:34.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.361+0000 7fb439200640 1 --2- 172.21.15.100:0/3744731269 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb414078d40 0x7fb41407b250 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb434073000 tx=0x7fb420009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:34.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.364+0000 7fb431600640 1 -- 172.21.15.100:0/3744731269 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb43406a700 con 0x7fb43406cda0 2024-09-17T04:25:34.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:34.549+0000 7fb43c600640 1 -- 172.21.15.100:0/3744731269 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb4340b3cc0 con 0x7fb43406cda0 2024-09-17T04:25:34.691 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:25:34.695 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:25:35.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.092+0000 7fb431600640 1 -- 172.21.15.100:0/3744731269 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v39) ==== 169+0+0 (secure 0 0 0) 0x7fb428083a10 con 0x7fb43406cda0 2024-09-17T04:25:35.093 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' removed 2024-09-17T04:25:35.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.096+0000 7fb43c600640 1 -- 172.21.15.100:0/3744731269 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb414078d40 msgr2=0x7fb41407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:35.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.096+0000 7fb43c600640 1 --2- 172.21.15.100:0/3744731269 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb414078d40 0x7fb41407b250 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb434073000 tx=0x7fb420009040 comp rx=0 tx=0).stop 2024-09-17T04:25:35.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.096+0000 7fb43c600640 1 -- 172.21.15.100:0/3744731269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb43406cda0 msgr2=0x7fb4341f3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:35.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.096+0000 7fb43c600640 1 --2- 172.21.15.100:0/3744731269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb43406cda0 0x7fb4341f3580 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb428034040 tx=0x7fb428003910 comp rx=0 tx=0).stop 2024-09-17T04:25:35.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.096+0000 7fb43c600640 1 -- 172.21.15.100:0/3744731269 shutdown_connections 2024-09-17T04:25:35.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.096+0000 7fb43c600640 1 --2- 172.21.15.100:0/3744731269 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb414078d40 0x7fb41407b250 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:35.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.096+0000 7fb43c600640 1 --2- 172.21.15.100:0/3744731269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4340cc830 0x7fb43415be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:35.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.096+0000 7fb43c600640 1 --2- 172.21.15.100:0/3744731269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4340af100 0x7fb43415b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:35.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.096+0000 7fb43c600640 1 --2- 172.21.15.100:0/3744731269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb43406cda0 0x7fb4341f3580 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:35.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.096+0000 7fb43c600640 1 -- 172.21.15.100:0/3744731269 >> 172.21.15.100:0/3744731269 conn(0x7fb4340b2540 msgr2=0x7fb4340c9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:35.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.096+0000 7fb43c600640 1 -- 172.21.15.100:0/3744731269 shutdown_connections 2024-09-17T04:25:35.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.096+0000 7fb43c600640 1 -- 172.21.15.100:0/3744731269 wait complete. 2024-09-17T04:25:35.120 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T04:25:35.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.352+0000 7f657a000640 1 -- 172.21.15.100:0/488721830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65740cbfc0 msgr2=0x7f65740af8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:35.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.352+0000 7f657a000640 1 --2- 172.21.15.100:0/488721830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65740cbfc0 0x7f65740af8d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6564009830 tx=0x7f656402a530 comp rx=0 tx=0).stop 2024-09-17T04:25:35.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.352+0000 7f657a000640 1 -- 172.21.15.100:0/488721830 shutdown_connections 2024-09-17T04:25:35.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.352+0000 7f657a000640 1 --2- 172.21.15.100:0/488721830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65740cbfc0 0x7f65740af8d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:35.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.352+0000 7f657a000640 1 -- 172.21.15.100:0/488721830 >> 172.21.15.100:0/488721830 conn(0x7f657406d3d0 msgr2=0x7f65740b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:35.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.352+0000 7f657a000640 1 -- 172.21.15.100:0/488721830 shutdown_connections 2024-09-17T04:25:35.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.352+0000 7f657a000640 1 -- 172.21.15.100:0/488721830 wait complete. 2024-09-17T04:25:35.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.355+0000 7f657a000640 1 Processor -- start 2024-09-17T04:25:35.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.355+0000 7f657a000640 1 -- start start 2024-09-17T04:25:35.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.356+0000 7f657a000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f657406f0d0 0x7f65740cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:35.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.356+0000 7f657a000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65740cbfc0 0x7f65740c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:35.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.356+0000 7f657a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6574075020 0x7f65740c6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:35.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.356+0000 7f657a000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f65740b4020 con 0x7f6574075020 2024-09-17T04:25:35.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.356+0000 7f657a000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f65740b3ea0 con 0x7f657406f0d0 2024-09-17T04:25:35.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.356+0000 7f657a000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f657416a170 con 0x7f65740cbfc0 2024-09-17T04:25:35.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.356+0000 7f6573e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6574075020 0x7f65740c6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:35.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.357+0000 7f6573400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f657406f0d0 0x7f65740cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:35.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.357+0000 7f656aa00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65740cbfc0 0x7f65740c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:35.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.357+0000 7f6573e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6574075020 0x7f65740c6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45244/0 (socket says 172.21.15.100:45244) 2024-09-17T04:25:35.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.357+0000 7f6573e00640 1 -- 172.21.15.100:0/2239545519 learned_addr learned my addr 172.21.15.100:0/2239545519 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:35.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.357+0000 7f6573e00640 1 -- 172.21.15.100:0/2239545519 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65740cbfc0 msgr2=0x7f65740c5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:35.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.357+0000 7f6573e00640 1 --2- 172.21.15.100:0/2239545519 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65740cbfc0 0x7f65740c5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:35.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.357+0000 7f6573e00640 1 -- 172.21.15.100:0/2239545519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f657406f0d0 msgr2=0x7f65740cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:35.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.357+0000 7f6573e00640 1 --2- 172.21.15.100:0/2239545519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f657406f0d0 0x7f65740cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:35.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.357+0000 7f6573e00640 1 -- 172.21.15.100:0/2239545519 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65740c7610 con 0x7f6574075020 2024-09-17T04:25:35.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.357+0000 7f6573e00640 1 --2- 172.21.15.100:0/2239545519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6574075020 0x7f65740c6530 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f656c00cad0 tx=0x7f656c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:35.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.357+0000 7f656be00640 1 -- 172.21.15.100:0/2239545519 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c016020 con 0x7f6574075020 2024-09-17T04:25:35.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.358+0000 7f656be00640 1 -- 172.21.15.100:0/2239545519 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f656c00ad20 con 0x7f6574075020 2024-09-17T04:25:35.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.358+0000 7f657a000640 1 -- 172.21.15.100:0/2239545519 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f657415b230 con 0x7f6574075020 2024-09-17T04:25:35.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.358+0000 7f656be00640 1 -- 172.21.15.100:0/2239545519 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c012680 con 0x7f6574075020 2024-09-17T04:25:35.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.358+0000 7f657a000640 1 -- 172.21.15.100:0/2239545519 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65740974f0 con 0x7f6574075020 2024-09-17T04:25:35.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.359+0000 7f657a000640 1 -- 172.21.15.100:0/2239545519 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f657415afa0 con 0x7f6574075020 2024-09-17T04:25:35.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.360+0000 7f656be00640 1 -- 172.21.15.100:0/2239545519 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f656c019020 con 0x7f6574075020 2024-09-17T04:25:35.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.361+0000 7f656be00640 1 --2- 172.21.15.100:0/2239545519 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6550078d70 0x7f655007b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:35.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.361+0000 7f656be00640 1 -- 172.21.15.100:0/2239545519 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7f656c09b2e0 con 0x7f6574075020 2024-09-17T04:25:35.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.361+0000 7f6573400640 1 --2- 172.21.15.100:0/2239545519 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6550078d70 0x7f655007b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:35.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.362+0000 7f6573400640 1 --2- 172.21.15.100:0/2239545519 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6550078d70 0x7f655007b280 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6564034000 tx=0x7f6564004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:35.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.365+0000 7f656be00640 1 -- 172.21.15.100:0/2239545519 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f656c020020 con 0x7f6574075020 2024-09-17T04:25:35.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.553+0000 7f657a000640 1 -- 172.21.15.100:0/2239545519 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f65740af8d0 con 0x7f6574075020 2024-09-17T04:25:35.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.558+0000 7f656be00640 1 -- 172.21.15.100:0/2239545519 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v33) ==== 71+0+0 (secure 0 0 0) 0x7f656c063030 con 0x7f6574075020 2024-09-17T04:25:35.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.560+0000 7f657a000640 1 -- 172.21.15.100:0/2239545519 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6550078d70 msgr2=0x7f655007b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:35.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.560+0000 7f657a000640 1 --2- 172.21.15.100:0/2239545519 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6550078d70 0x7f655007b280 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6564034000 tx=0x7f6564004730 comp rx=0 tx=0).stop 2024-09-17T04:25:35.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.560+0000 7f657a000640 1 -- 172.21.15.100:0/2239545519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6574075020 msgr2=0x7f65740c6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:35.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.560+0000 7f657a000640 1 --2- 172.21.15.100:0/2239545519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6574075020 0x7f65740c6530 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f656c00cad0 tx=0x7f656c00cf90 comp rx=0 tx=0).stop 2024-09-17T04:25:35.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.560+0000 7f657a000640 1 -- 172.21.15.100:0/2239545519 shutdown_connections 2024-09-17T04:25:35.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.560+0000 7f657a000640 1 --2- 172.21.15.100:0/2239545519 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6550078d70 0x7f655007b280 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:35.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.560+0000 7f657a000640 1 --2- 172.21.15.100:0/2239545519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6574075020 0x7f65740c6530 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:35.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.560+0000 7f657a000640 1 --2- 172.21.15.100:0/2239545519 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65740cbfc0 0x7f65740c5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:35.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.560+0000 7f657a000640 1 --2- 172.21.15.100:0/2239545519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f657406f0d0 0x7f65740cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:35.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.560+0000 7f657a000640 1 -- 172.21.15.100:0/2239545519 >> 172.21.15.100:0/2239545519 conn(0x7f657406d3d0 msgr2=0x7f65740d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:35.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.560+0000 7f657a000640 1 -- 172.21.15.100:0/2239545519 shutdown_connections 2024-09-17T04:25:35.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.560+0000 7f657a000640 1 -- 172.21.15.100:0/2239545519 wait complete. 2024-09-17T04:25:35.582 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2024-09-17T04:25:35.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.803+0000 7f90bbe00640 1 -- 172.21.15.100:0/778315505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90a0003a00 msgr2=0x7f90a0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:35.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.803+0000 7f90bbe00640 1 --2- 172.21.15.100:0/778315505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90a0003a00 0x7f90a0003e70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009b50 tx=0x7f90a0005830 comp rx=0 tx=0).stop 2024-09-17T04:25:35.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.803+0000 7f90bb400640 1 -- 172.21.15.100:0/778315505 reap_dead start 2024-09-17T04:25:35.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.803+0000 7f90bbe00640 1 -- 172.21.15.100:0/778315505 shutdown_connections 2024-09-17T04:25:35.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.803+0000 7f90bbe00640 1 -- 172.21.15.100:0/778315505 >> 172.21.15.100:0/778315505 conn(0x7f90bc0b1e40 msgr2=0x7f90bc0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:35.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.803+0000 7f90bbe00640 1 -- 172.21.15.100:0/778315505 shutdown_connections 2024-09-17T04:25:35.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.803+0000 7f90bbe00640 1 -- 172.21.15.100:0/778315505 wait complete. 2024-09-17T04:25:35.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.805+0000 7f90bbe00640 1 Processor -- start 2024-09-17T04:25:35.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.806+0000 7f90bbe00640 1 -- start start 2024-09-17T04:25:35.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.807+0000 7f90bbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90a0003a00 0x7f90bc1f9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:35.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.807+0000 7f90bbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90bc14fe00 0x7f90bc1520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:35.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.807+0000 7f90bbe00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90bc150210 0x7f90bc152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:35.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.807+0000 7f90bbe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc06f810 con 0x7f90a0003a00 2024-09-17T04:25:35.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.807+0000 7f90bbe00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc06f690 con 0x7f90bc14fe00 2024-09-17T04:25:35.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.807+0000 7f90bbe00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f90bc06f990 con 0x7f90bc150210 2024-09-17T04:25:35.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.807+0000 7f90baa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90a0003a00 0x7f90bc1f9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:35.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.807+0000 7f90baa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90a0003a00 0x7f90bc1f9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45260/0 (socket says 172.21.15.100:45260) 2024-09-17T04:25:35.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.807+0000 7f90baa00640 1 -- 172.21.15.100:0/3464441857 learned_addr learned my addr 172.21.15.100:0/3464441857 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:35.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.808+0000 7f90b2000640 1 --2- 172.21.15.100:0/3464441857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90bc14fe00 0x7f90bc1520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:35.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.808+0000 7f90bb400640 1 --2- 172.21.15.100:0/3464441857 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90bc150210 0x7f90bc152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:35.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.808+0000 7f90baa00640 1 -- 172.21.15.100:0/3464441857 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90bc150210 msgr2=0x7f90bc152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:35.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.808+0000 7f90baa00640 1 --2- 172.21.15.100:0/3464441857 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90bc150210 0x7f90bc152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:35.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.808+0000 7f90baa00640 1 -- 172.21.15.100:0/3464441857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90bc14fe00 msgr2=0x7f90bc1520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:35.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.808+0000 7f90baa00640 1 --2- 172.21.15.100:0/3464441857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90bc14fe00 0x7f90bc1520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:35.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.808+0000 7f90baa00640 1 -- 172.21.15.100:0/3464441857 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90bc1f9670 con 0x7f90a0003a00 2024-09-17T04:25:35.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.808+0000 7f90bb400640 1 --2- 172.21.15.100:0/3464441857 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90bc150210 0x7f90bc152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:35.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.808+0000 7f90b2000640 1 --2- 172.21.15.100:0/3464441857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90bc14fe00 0x7f90bc1520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:35.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.808+0000 7f90baa00640 1 --2- 172.21.15.100:0/3464441857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90a0003a00 0x7f90bc1f9130 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f90ac036040 tx=0x7f90ac033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:35.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.808+0000 7f90b3e00640 1 -- 172.21.15.100:0/3464441857 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac038040 con 0x7f90a0003a00 2024-09-17T04:25:35.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.808+0000 7f90b3e00640 1 -- 172.21.15.100:0/3464441857 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90ac004020 con 0x7f90a0003a00 2024-09-17T04:25:35.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.808+0000 7f90bbe00640 1 -- 172.21.15.100:0/3464441857 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90bc0ad040 con 0x7f90a0003a00 2024-09-17T04:25:35.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.808+0000 7f90b3e00640 1 -- 172.21.15.100:0/3464441857 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac004470 con 0x7f90a0003a00 2024-09-17T04:25:35.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.808+0000 7f90bbe00640 1 -- 172.21.15.100:0/3464441857 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90bc09c520 con 0x7f90a0003a00 2024-09-17T04:25:35.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.810+0000 7f90bbe00640 1 -- 172.21.15.100:0/3464441857 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90bc0ad210 con 0x7f90a0003a00 2024-09-17T04:25:35.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.811+0000 7f90b3e00640 1 -- 172.21.15.100:0/3464441857 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f90ac045020 con 0x7f90a0003a00 2024-09-17T04:25:35.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.811+0000 7f90b3e00640 1 --2- 172.21.15.100:0/3464441857 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9098078d40 0x7f909807b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:35.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.812+0000 7f90b3e00640 1 -- 172.21.15.100:0/3464441857 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7f90ac0bba00 con 0x7f90a0003a00 2024-09-17T04:25:35.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.812+0000 7f90b2000640 1 --2- 172.21.15.100:0/3464441857 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9098078d40 0x7f909807b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:35.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.813+0000 7f90b2000640 1 --2- 172.21.15.100:0/3464441857 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9098078d40 0x7f909807b250 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f90bc0b2610 tx=0x7f90a4005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:35.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:35.816+0000 7f90b3e00640 1 -- 172.21.15.100:0/3464441857 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f90bc1f9670 con 0x7f90a0003a00 2024-09-17T04:25:36.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.069+0000 7f90bbe00640 1 -- 172.21.15.100:0/3464441857 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f90bc09c340 con 0x7f90a0003a00 2024-09-17T04:25:36.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.101+0000 7f90b3e00640 1 -- 172.21.15.100:0/3464441857 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v102) ==== 158+0+0 (secure 0 0 0) 0x7f90bc09c340 con 0x7f90a0003a00 2024-09-17T04:25:36.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.104+0000 7f90bbe00640 1 -- 172.21.15.100:0/3464441857 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9098078d40 msgr2=0x7f909807b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:36.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.104+0000 7f90bbe00640 1 --2- 172.21.15.100:0/3464441857 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9098078d40 0x7f909807b250 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f90bc0b2610 tx=0x7f90a4005f90 comp rx=0 tx=0).stop 2024-09-17T04:25:36.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.104+0000 7f90bbe00640 1 -- 172.21.15.100:0/3464441857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90a0003a00 msgr2=0x7f90bc1f9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:36.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.104+0000 7f90bbe00640 1 --2- 172.21.15.100:0/3464441857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90a0003a00 0x7f90bc1f9130 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f90ac036040 tx=0x7f90ac033670 comp rx=0 tx=0).stop 2024-09-17T04:25:36.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.104+0000 7f90bbe00640 1 -- 172.21.15.100:0/3464441857 shutdown_connections 2024-09-17T04:25:36.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.104+0000 7f90bbe00640 1 --2- 172.21.15.100:0/3464441857 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9098078d40 0x7f909807b250 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:36.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.104+0000 7f90bbe00640 1 --2- 172.21.15.100:0/3464441857 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90bc150210 0x7f90bc152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:36.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.104+0000 7f90bbe00640 1 --2- 172.21.15.100:0/3464441857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90bc14fe00 0x7f90bc1520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:36.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.104+0000 7f90bbe00640 1 --2- 172.21.15.100:0/3464441857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90a0003a00 0x7f90bc1f9130 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:36.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.104+0000 7f90bbe00640 1 -- 172.21.15.100:0/3464441857 >> 172.21.15.100:0/3464441857 conn(0x7f90bc0b1e40 msgr2=0x7f90bc1565e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:36.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.104+0000 7f90bbe00640 1 -- 172.21.15.100:0/3464441857 shutdown_connections 2024-09-17T04:25:36.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.104+0000 7f90bbe00640 1 -- 172.21.15.100:0/3464441857 wait complete. 2024-09-17T04:25:36.127 INFO:tasks.cephfs_test_runner:Validate binary .meta file on legacy subvol root doesn't break the system ... ok 2024-09-17T04:25:36.127 INFO:tasks.cephfs_test_runner:Starting test: test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) 2024-09-17T04:25:36.127 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:25:36.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.347+0000 7f388fe00640 1 -- 172.21.15.100:0/811979409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38900d1580 msgr2=0x7f38900d1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:36.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.347+0000 7f388fe00640 1 --2- 172.21.15.100:0/811979409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38900d1580 0x7f38900d1990 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3888009830 tx=0x7f388802a480 comp rx=0 tx=0).stop 2024-09-17T04:25:36.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.347+0000 7f388fe00640 1 -- 172.21.15.100:0/811979409 shutdown_connections 2024-09-17T04:25:36.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.347+0000 7f388fe00640 1 --2- 172.21.15.100:0/811979409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38900d1580 0x7f38900d1990 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:36.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.347+0000 7f388fe00640 1 -- 172.21.15.100:0/811979409 >> 172.21.15.100:0/811979409 conn(0x7f389008bfe0 msgr2=0x7f389009d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:36.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.347+0000 7f388fe00640 1 -- 172.21.15.100:0/811979409 shutdown_connections 2024-09-17T04:25:36.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.347+0000 7f388fe00640 1 -- 172.21.15.100:0/811979409 wait complete. 2024-09-17T04:25:36.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.349+0000 7f388fe00640 1 Processor -- start 2024-09-17T04:25:36.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.349+0000 7f388fe00640 1 -- start start 2024-09-17T04:25:36.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.351+0000 7f388fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38900b3160 0x7f38901f93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:36.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.351+0000 7f388fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38900b9e00 0x7f389014dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:36.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.351+0000 7f388fe00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38900d1580 0x7f38900b87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:36.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.351+0000 7f388fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f38900d4700 con 0x7f38900b9e00 2024-09-17T04:25:36.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.351+0000 7f388fe00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f38900d4580 con 0x7f38900b3160 2024-09-17T04:25:36.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.351+0000 7f388fe00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f38900d4880 con 0x7f38900d1580 2024-09-17T04:25:36.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.351+0000 7f3886000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38900b9e00 0x7f389014dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:36.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.351+0000 7f3886000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38900b9e00 0x7f389014dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45280/0 (socket says 172.21.15.100:45280) 2024-09-17T04:25:36.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.351+0000 7f3886000640 1 -- 172.21.15.100:0/327984674 learned_addr learned my addr 172.21.15.100:0/327984674 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:36.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.351+0000 7f388ea00640 1 --2- 172.21.15.100:0/327984674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38900b3160 0x7f38901f93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:36.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.351+0000 7f388f400640 1 --2- 172.21.15.100:0/327984674 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38900d1580 0x7f38900b87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:36.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.351+0000 7f3886000640 1 -- 172.21.15.100:0/327984674 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38900d1580 msgr2=0x7f38900b87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:36.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.351+0000 7f3886000640 1 --2- 172.21.15.100:0/327984674 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38900d1580 0x7f38900b87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:36.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.351+0000 7f3886000640 1 -- 172.21.15.100:0/327984674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38900b3160 msgr2=0x7f38901f93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:36.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.352+0000 7f3886000640 1 --2- 172.21.15.100:0/327984674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38900b3160 0x7f38901f93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:36.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.352+0000 7f3886000640 1 -- 172.21.15.100:0/327984674 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38901f69b0 con 0x7f38900b9e00 2024-09-17T04:25:36.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.352+0000 7f3886000640 1 --2- 172.21.15.100:0/327984674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38900b9e00 0x7f389014dc50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f387800c7d0 tx=0x7f387800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:36.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.352+0000 7f3887400640 1 -- 172.21.15.100:0/327984674 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3878017070 con 0x7f38900b9e00 2024-09-17T04:25:36.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.352+0000 7f3887400640 1 -- 172.21.15.100:0/327984674 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3878005070 con 0x7f38900b9e00 2024-09-17T04:25:36.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.352+0000 7f3887400640 1 -- 172.21.15.100:0/327984674 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38780054a0 con 0x7f38900b9e00 2024-09-17T04:25:36.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.352+0000 7f388fe00640 1 -- 172.21.15.100:0/327984674 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38901f6c90 con 0x7f38900b9e00 2024-09-17T04:25:36.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.352+0000 7f388fe00640 1 -- 172.21.15.100:0/327984674 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38901f7250 con 0x7f38900b9e00 2024-09-17T04:25:36.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.353+0000 7f388fe00640 1 -- 172.21.15.100:0/327984674 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38901f64e0 con 0x7f38900b9e00 2024-09-17T04:25:36.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.354+0000 7f3887400640 1 -- 172.21.15.100:0/327984674 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f387801a070 con 0x7f38900b9e00 2024-09-17T04:25:36.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.355+0000 7f3887400640 1 --2- 172.21.15.100:0/327984674 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f386c078d40 0x7f386c07b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:36.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.356+0000 7f3887400640 1 -- 172.21.15.100:0/327984674 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7f387809a420 con 0x7f38900b9e00 2024-09-17T04:25:36.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.356+0000 7f388ea00640 1 --2- 172.21.15.100:0/327984674 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f386c078d40 0x7f386c07b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:36.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.357+0000 7f388ea00640 1 --2- 172.21.15.100:0/327984674 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f386c078d40 0x7f386c07b250 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3880006fd0 tx=0x7f3880009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:36.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.360+0000 7f3887400640 1 -- 172.21.15.100:0/327984674 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3878062220 con 0x7f38900b9e00 2024-09-17T04:25:36.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.545+0000 7f388fe00640 1 -- 172.21.15.100:0/327984674 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38900d1990 con 0x7f38900b9e00 2024-09-17T04:25:36.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.546+0000 7f3887400640 1 -- 172.21.15.100:0/327984674 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+13080 (secure 0 0 0) 0x7f3878066c50 con 0x7f38900b9e00 2024-09-17T04:25:36.547 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:36.547 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":39,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:25:35.091821+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.184:6837/294839954":"2024-09-18T04:25:31.722803+0000","172.21.15.184:6834/294839954":"2024-09-18T04:25:31.722803+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:25:36.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.549+0000 7f388fe00640 1 -- 172.21.15.100:0/327984674 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f386c078d40 msgr2=0x7f386c07b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:36.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.549+0000 7f388fe00640 1 --2- 172.21.15.100:0/327984674 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f386c078d40 0x7f386c07b250 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3880006fd0 tx=0x7f3880009060 comp rx=0 tx=0).stop 2024-09-17T04:25:36.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.549+0000 7f388fe00640 1 -- 172.21.15.100:0/327984674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38900b9e00 msgr2=0x7f389014dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:36.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.549+0000 7f388fe00640 1 --2- 172.21.15.100:0/327984674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38900b9e00 0x7f389014dc50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f387800c7d0 tx=0x7f387800cc90 comp rx=0 tx=0).stop 2024-09-17T04:25:36.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.549+0000 7f388fe00640 1 -- 172.21.15.100:0/327984674 shutdown_connections 2024-09-17T04:25:36.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.549+0000 7f388fe00640 1 --2- 172.21.15.100:0/327984674 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f386c078d40 0x7f386c07b250 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:36.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.549+0000 7f388fe00640 1 --2- 172.21.15.100:0/327984674 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38900d1580 0x7f38900b87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:36.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.549+0000 7f388fe00640 1 --2- 172.21.15.100:0/327984674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38900b9e00 0x7f389014dc50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:36.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.549+0000 7f388fe00640 1 --2- 172.21.15.100:0/327984674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38900b3160 0x7f38901f93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:36.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.549+0000 7f388fe00640 1 -- 172.21.15.100:0/327984674 >> 172.21.15.100:0/327984674 conn(0x7f389008bfe0 msgr2=0x7f3890162a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:36.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.549+0000 7f388fe00640 1 -- 172.21.15.100:0/327984674 shutdown_connections 2024-09-17T04:25:36.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.549+0000 7f388fe00640 1 -- 172.21.15.100:0/327984674 wait complete. 2024-09-17T04:25:36.571 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T04:24:03.995355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T04:25:36.571 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T04:25:36.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.795+0000 7f4656200640 1 -- 172.21.15.100:0/3568372494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4650073930 msgr2=0x7f465015ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:36.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.795+0000 7f4656200640 1 --2- 172.21.15.100:0/3568372494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4650073930 0x7f465015ae90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f46500ae0f0 tx=0x7f464002b860 comp rx=0 tx=0).stop 2024-09-17T04:25:36.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.795+0000 7f4656200640 1 -- 172.21.15.100:0/3568372494 shutdown_connections 2024-09-17T04:25:36.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.795+0000 7f4656200640 1 --2- 172.21.15.100:0/3568372494 >> v2:172.21.15.184:3300/0 conn(0x7f46500733e0 0x7f46500b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:36.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.795+0000 7f4656200640 1 --2- 172.21.15.100:0/3568372494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4650073930 0x7f465015ae90 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:36.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.795+0000 7f4656200640 1 -- 172.21.15.100:0/3568372494 >> 172.21.15.100:0/3568372494 conn(0x7f465006d710 msgr2=0x7f46500b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:36.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.795+0000 7f4656200640 1 -- 172.21.15.100:0/3568372494 shutdown_connections 2024-09-17T04:25:36.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.795+0000 7f4656200640 1 -- 172.21.15.100:0/3568372494 wait complete. 2024-09-17T04:25:36.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.798+0000 7f4656200640 1 Processor -- start 2024-09-17T04:25:36.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.798+0000 7f4656200640 1 -- start start 2024-09-17T04:25:36.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.799+0000 7f4656200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f465006f800 0x7f465009a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:36.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.799+0000 7f4656200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46500733e0 0x7f465009d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:36.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.799+0000 7f4656200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4650073930 0x7f46501f6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:36.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.799+0000 7f4656200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f465006f300 con 0x7f465006f800 2024-09-17T04:25:36.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.799+0000 7f4656200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f465006f180 con 0x7f4650073930 2024-09-17T04:25:36.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.799+0000 7f4656200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f465006f480 con 0x7f46500733e0 2024-09-17T04:25:36.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.799+0000 7f464f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f465006f800 0x7f465009a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:36.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.799+0000 7f464f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f465006f800 0x7f465009a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45290/0 (socket says 172.21.15.100:45290) 2024-09-17T04:25:36.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.799+0000 7f464f400640 1 -- 172.21.15.100:0/1089058405 learned_addr learned my addr 172.21.15.100:0/1089058405 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:36.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.800+0000 7f464fe00640 1 --2- 172.21.15.100:0/1089058405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4650073930 0x7f46501f6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:36.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.800+0000 7f4646a00640 1 --2- 172.21.15.100:0/1089058405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46500733e0 0x7f465009d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:36.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.800+0000 7f464f400640 1 -- 172.21.15.100:0/1089058405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46500733e0 msgr2=0x7f465009d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:36.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.800+0000 7f464f400640 1 --2- 172.21.15.100:0/1089058405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46500733e0 0x7f465009d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:36.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.800+0000 7f464f400640 1 -- 172.21.15.100:0/1089058405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4650073930 msgr2=0x7f46501f6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:36.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.800+0000 7f464f400640 1 --2- 172.21.15.100:0/1089058405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4650073930 0x7f46501f6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:36.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.800+0000 7f464f400640 1 -- 172.21.15.100:0/1089058405 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4650158d40 con 0x7f465006f800 2024-09-17T04:25:36.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.800+0000 7f464fe00640 1 --2- 172.21.15.100:0/1089058405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4650073930 0x7f46501f6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:36.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.800+0000 7f464f400640 1 --2- 172.21.15.100:0/1089058405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f465006f800 0x7f465009a2d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4640000c00 tx=0x7f4640031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:36.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.800+0000 7f4647e00640 1 -- 172.21.15.100:0/1089058405 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640005c90 con 0x7f465006f800 2024-09-17T04:25:36.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.801+0000 7f4647e00640 1 -- 172.21.15.100:0/1089058405 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46400037f0 con 0x7f465006f800 2024-09-17T04:25:36.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.801+0000 7f4656200640 1 -- 172.21.15.100:0/1089058405 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4650158fe0 con 0x7f465006f800 2024-09-17T04:25:36.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.801+0000 7f4647e00640 1 -- 172.21.15.100:0/1089058405 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640003c40 con 0x7f465006f800 2024-09-17T04:25:36.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.801+0000 7f4656200640 1 -- 172.21.15.100:0/1089058405 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46501595a0 con 0x7f465006f800 2024-09-17T04:25:36.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.802+0000 7f4656200640 1 -- 172.21.15.100:0/1089058405 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46500b31e0 con 0x7f465006f800 2024-09-17T04:25:36.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.803+0000 7f4647e00640 1 -- 172.21.15.100:0/1089058405 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f464004e440 con 0x7f465006f800 2024-09-17T04:25:36.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.804+0000 7f4647e00640 1 --2- 172.21.15.100:0/1089058405 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f462c078d40 0x7f462c07b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:36.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.804+0000 7f4647e00640 1 -- 172.21.15.100:0/1089058405 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7f46400bc200 con 0x7f465006f800 2024-09-17T04:25:36.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.805+0000 7f4646a00640 1 --2- 172.21.15.100:0/1089058405 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f462c078d40 0x7f462c07b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:36.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.805+0000 7f4646a00640 1 --2- 172.21.15.100:0/1089058405 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f462c078d40 0x7f462c07b250 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f465009e110 tx=0x7f4638005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:36.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.808+0000 7f4647e00640 1 -- 172.21.15.100:0/1089058405 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4650158d40 con 0x7f465006f800 2024-09-17T04:25:36.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.991+0000 7f4656200640 1 -- 172.21.15.100:0/1089058405 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f46501e8c90 con 0x7f465006f800 2024-09-17T04:25:36.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.992+0000 7f4647e00640 1 -- 172.21.15.100:0/1089058405 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) ==== 93+0+10 (secure 0 0 0) 0x7f4640031e90 con 0x7f465006f800 2024-09-17T04:25:36.992 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-09-17T04:25:36.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.994+0000 7f4656200640 1 -- 172.21.15.100:0/1089058405 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f462c078d40 msgr2=0x7f462c07b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:36.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.994+0000 7f4656200640 1 --2- 172.21.15.100:0/1089058405 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f462c078d40 0x7f462c07b250 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f465009e110 tx=0x7f4638005f90 comp rx=0 tx=0).stop 2024-09-17T04:25:36.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.994+0000 7f4656200640 1 -- 172.21.15.100:0/1089058405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f465006f800 msgr2=0x7f465009a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:36.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.994+0000 7f4656200640 1 --2- 172.21.15.100:0/1089058405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f465006f800 0x7f465009a2d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4640000c00 tx=0x7f4640031ca0 comp rx=0 tx=0).stop 2024-09-17T04:25:36.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.994+0000 7f4656200640 1 -- 172.21.15.100:0/1089058405 shutdown_connections 2024-09-17T04:25:36.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.994+0000 7f4656200640 1 --2- 172.21.15.100:0/1089058405 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f462c078d40 0x7f462c07b250 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:36.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.994+0000 7f4656200640 1 --2- 172.21.15.100:0/1089058405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4650073930 0x7f46501f6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:36.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.994+0000 7f4656200640 1 --2- 172.21.15.100:0/1089058405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46500733e0 0x7f465009d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:36.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.994+0000 7f4656200640 1 --2- 172.21.15.100:0/1089058405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f465006f800 0x7f465009a2d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:36.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.994+0000 7f4656200640 1 -- 172.21.15.100:0/1089058405 >> 172.21.15.100:0/1089058405 conn(0x7f465006d710 msgr2=0x7f4650156660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:36.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.994+0000 7f4656200640 1 -- 172.21.15.100:0/1089058405 shutdown_connections 2024-09-17T04:25:36.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:36.994+0000 7f4656200640 1 -- 172.21.15.100:0/1089058405 wait complete. 2024-09-17T04:25:37.021 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2024-09-17T04:25:37.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.241+0000 7f3b24c00640 1 -- 172.21.15.100:0/2391492517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b04003a00 msgr2=0x7f3b04003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:37.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.241+0000 7f3b24c00640 1 --2- 172.21.15.100:0/2391492517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b04003a00 0x7f3b04003e50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000c610 tx=0x7f3b04005830 comp rx=0 tx=0).stop 2024-09-17T04:25:37.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.241+0000 7f3b24c00640 1 -- 172.21.15.100:0/2391492517 shutdown_connections 2024-09-17T04:25:37.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.241+0000 7f3b24c00640 1 --2- 172.21.15.100:0/2391492517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b04003a00 0x7f3b04003e50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:37.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.241+0000 7f3b24c00640 1 -- 172.21.15.100:0/2391492517 >> 172.21.15.100:0/2391492517 conn(0x7f3b200b1e40 msgr2=0x7f3b200b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:37.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.241+0000 7f3b24c00640 1 -- 172.21.15.100:0/2391492517 shutdown_connections 2024-09-17T04:25:37.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.241+0000 7f3b24c00640 1 -- 172.21.15.100:0/2391492517 wait complete. 2024-09-17T04:25:37.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.243+0000 7f3b24c00640 1 Processor -- start 2024-09-17T04:25:37.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.244+0000 7f3b24c00640 1 -- start start 2024-09-17T04:25:37.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.245+0000 7f3b24c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b04003a00 0x7f3b2009da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:37.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.245+0000 7f3b24c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b20155580 0x7f3b20151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:37.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.245+0000 7f3b24c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b20156d40 0x7f3b20157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:37.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.245+0000 7f3b24c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2006fad0 con 0x7f3b20155580 2024-09-17T04:25:37.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.245+0000 7f3b24c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2006f950 con 0x7f3b04003a00 2024-09-17T04:25:37.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.245+0000 7f3b24c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3b201627b0 con 0x7f3b20156d40 2024-09-17T04:25:37.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.245+0000 7f3b1ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b20155580 0x7f3b20151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:37.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.245+0000 7f3b1ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b20155580 0x7f3b20151860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45304/0 (socket says 172.21.15.100:45304) 2024-09-17T04:25:37.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.245+0000 7f3b1ea00640 1 -- 172.21.15.100:0/2794228515 learned_addr learned my addr 172.21.15.100:0/2794228515 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:37.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.245+0000 7f3b1f400640 1 --2- 172.21.15.100:0/2794228515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b04003a00 0x7f3b2009da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:37.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.245+0000 7f3b1fe00640 1 --2- 172.21.15.100:0/2794228515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b20156d40 0x7f3b20157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:37.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.246+0000 7f3b1ea00640 1 -- 172.21.15.100:0/2794228515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b20156d40 msgr2=0x7f3b20157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:37.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.246+0000 7f3b1ea00640 1 --2- 172.21.15.100:0/2794228515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b20156d40 0x7f3b20157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:37.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.246+0000 7f3b1ea00640 1 -- 172.21.15.100:0/2794228515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b04003a00 msgr2=0x7f3b2009da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:37.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.246+0000 7f3b1ea00640 1 --2- 172.21.15.100:0/2794228515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b04003a00 0x7f3b2009da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:37.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.246+0000 7f3b1ea00640 1 -- 172.21.15.100:0/2794228515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b20157690 con 0x7f3b20155580 2024-09-17T04:25:37.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.246+0000 7f3b1f400640 1 --2- 172.21.15.100:0/2794228515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b04003a00 0x7f3b2009da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:37.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.246+0000 7f3b1ea00640 1 --2- 172.21.15.100:0/2794228515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b20155580 0x7f3b20151860 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c00d2d0 tx=0x7f3b0c00d790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:37.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.246+0000 7f3b17400640 1 -- 172.21.15.100:0/2794228515 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c014020 con 0x7f3b20155580 2024-09-17T04:25:37.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.246+0000 7f3b17400640 1 -- 172.21.15.100:0/2794228515 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b0c010970 con 0x7f3b20155580 2024-09-17T04:25:37.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.246+0000 7f3b17400640 1 -- 172.21.15.100:0/2794228515 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c010dc0 con 0x7f3b20155580 2024-09-17T04:25:37.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.246+0000 7f3b24c00640 1 -- 172.21.15.100:0/2794228515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b201568e0 con 0x7f3b20155580 2024-09-17T04:25:37.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.246+0000 7f3b24c00640 1 -- 172.21.15.100:0/2794228515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b20156040 con 0x7f3b20155580 2024-09-17T04:25:37.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.247+0000 7f3b24c00640 1 -- 172.21.15.100:0/2794228515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b20155ad0 con 0x7f3b20155580 2024-09-17T04:25:37.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.249+0000 7f3b17400640 1 -- 172.21.15.100:0/2794228515 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f3b0c017020 con 0x7f3b20155580 2024-09-17T04:25:37.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.250+0000 7f3b17400640 1 --2- 172.21.15.100:0/2794228515 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3b00078d40 0x7f3b0007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:37.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.250+0000 7f3b17400640 1 -- 172.21.15.100:0/2794228515 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7f3b0c09b280 con 0x7f3b20155580 2024-09-17T04:25:37.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.250+0000 7f3b1f400640 1 --2- 172.21.15.100:0/2794228515 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3b00078d40 0x7f3b0007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:37.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.251+0000 7f3b1f400640 1 --2- 172.21.15.100:0/2794228515 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3b00078d40 0x7f3b0007b250 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3b10033000 tx=0x7f3b10038040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:37.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.254+0000 7f3b17400640 1 -- 172.21.15.100:0/2794228515 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3b0c01e020 con 0x7f3b20155580 2024-09-17T04:25:37.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:37.503+0000 7f3b24c00640 1 -- 172.21.15.100:0/2794228515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) -- 0x7f3b20155370 con 0x7f3b20155580 2024-09-17T04:25:38.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.110+0000 7f3b17400640 1 -- 172.21.15.100:0/2794228515 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v104) ==== 143+0+0 (secure 0 0 0) 0x7f3b20157690 con 0x7f3b20155580 2024-09-17T04:25:38.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.114+0000 7f3b24c00640 1 -- 172.21.15.100:0/2794228515 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3b00078d40 msgr2=0x7f3b0007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:38.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.114+0000 7f3b24c00640 1 --2- 172.21.15.100:0/2794228515 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3b00078d40 0x7f3b0007b250 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3b10033000 tx=0x7f3b10038040 comp rx=0 tx=0).stop 2024-09-17T04:25:38.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.114+0000 7f3b24c00640 1 -- 172.21.15.100:0/2794228515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b20155580 msgr2=0x7f3b20151860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:38.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.114+0000 7f3b24c00640 1 --2- 172.21.15.100:0/2794228515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b20155580 0x7f3b20151860 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c00d2d0 tx=0x7f3b0c00d790 comp rx=0 tx=0).stop 2024-09-17T04:25:38.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.114+0000 7f3b24c00640 1 -- 172.21.15.100:0/2794228515 shutdown_connections 2024-09-17T04:25:38.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.114+0000 7f3b24c00640 1 --2- 172.21.15.100:0/2794228515 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3b00078d40 0x7f3b0007b250 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:38.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.114+0000 7f3b24c00640 1 --2- 172.21.15.100:0/2794228515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b20156d40 0x7f3b20157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:38.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.114+0000 7f3b24c00640 1 --2- 172.21.15.100:0/2794228515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b20155580 0x7f3b20151860 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:38.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.114+0000 7f3b24c00640 1 --2- 172.21.15.100:0/2794228515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b04003a00 0x7f3b2009da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:38.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.114+0000 7f3b24c00640 1 -- 172.21.15.100:0/2794228515 >> 172.21.15.100:0/2794228515 conn(0x7f3b200b1e40 msgr2=0x7f3b20161230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:38.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.114+0000 7f3b24c00640 1 -- 172.21.15.100:0/2794228515 shutdown_connections 2024-09-17T04:25:38.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.114+0000 7f3b24c00640 1 -- 172.21.15.100:0/2794228515 wait complete. 2024-09-17T04:25:38.141 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T04:25:38.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.362+0000 7f8261600640 1 -- 172.21.15.100:0/1526781665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8240005150 msgr2=0x7f824000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:38.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.362+0000 7f8261600640 1 --2- 172.21.15.100:0/1526781665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8240005150 0x7f824000b9c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f825400c750 tx=0x7f8240005830 comp rx=0 tx=0).stop 2024-09-17T04:25:38.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.362+0000 7f825b400640 1 -- 172.21.15.100:0/1526781665 reap_dead start 2024-09-17T04:25:38.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.362+0000 7f8261600640 1 -- 172.21.15.100:0/1526781665 shutdown_connections 2024-09-17T04:25:38.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.362+0000 7f8261600640 1 -- 172.21.15.100:0/1526781665 >> 172.21.15.100:0/1526781665 conn(0x7f825c06d730 msgr2=0x7f825c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:38.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.362+0000 7f8261600640 1 -- 172.21.15.100:0/1526781665 shutdown_connections 2024-09-17T04:25:38.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.362+0000 7f8261600640 1 -- 172.21.15.100:0/1526781665 wait complete. 2024-09-17T04:25:38.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.364+0000 7f8261600640 1 Processor -- start 2024-09-17T04:25:38.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.365+0000 7f8261600640 1 -- start start 2024-09-17T04:25:38.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.366+0000 7f8261600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8240005150 0x7f825c1f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:38.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.366+0000 7f8261600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f825c155810 0x7f825c155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:38.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.366+0000 7f8261600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f825c156160 0x7f825c1f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:38.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.366+0000 7f8261600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f825c06f220 con 0x7f825c155810 2024-09-17T04:25:38.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.366+0000 7f8261600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f825c06f0a0 con 0x7f8240005150 2024-09-17T04:25:38.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.366+0000 7f8261600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f825c0b4c20 con 0x7f825c156160 2024-09-17T04:25:38.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.366+0000 7f8252000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f825c155810 0x7f825c155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:38.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.366+0000 7f825aa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8240005150 0x7f825c1f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:38.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.366+0000 7f8252000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f825c155810 0x7f825c155c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45322/0 (socket says 172.21.15.100:45322) 2024-09-17T04:25:38.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.366+0000 7f8252000640 1 -- 172.21.15.100:0/1223452477 learned_addr learned my addr 172.21.15.100:0/1223452477 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:38.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.367+0000 7f825b400640 1 --2- 172.21.15.100:0/1223452477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f825c156160 0x7f825c1f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:38.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.367+0000 7f8252000640 1 -- 172.21.15.100:0/1223452477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f825c156160 msgr2=0x7f825c1f4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:38.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.367+0000 7f8252000640 1 --2- 172.21.15.100:0/1223452477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f825c156160 0x7f825c1f4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:38.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.367+0000 7f8252000640 1 -- 172.21.15.100:0/1223452477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8240005150 msgr2=0x7f825c1f8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:38.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.367+0000 7f8252000640 1 --2- 172.21.15.100:0/1223452477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8240005150 0x7f825c1f8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:38.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.367+0000 7f8252000640 1 -- 172.21.15.100:0/1223452477 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825c1f9610 con 0x7f825c155810 2024-09-17T04:25:38.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.367+0000 7f8252000640 1 --2- 172.21.15.100:0/1223452477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f825c155810 0x7f825c155c20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8244011a60 tx=0x7f8244011f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:38.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.367+0000 7f8253400640 1 -- 172.21.15.100:0/1223452477 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824401f070 con 0x7f825c155810 2024-09-17T04:25:38.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.367+0000 7f8253400640 1 -- 172.21.15.100:0/1223452477 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f824400fe70 con 0x7f825c155810 2024-09-17T04:25:38.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.367+0000 7f8253400640 1 -- 172.21.15.100:0/1223452477 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82440126b0 con 0x7f825c155810 2024-09-17T04:25:38.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.367+0000 7f8261600640 1 -- 172.21.15.100:0/1223452477 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f825c1f98f0 con 0x7f825c155810 2024-09-17T04:25:38.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.368+0000 7f8261600640 1 -- 172.21.15.100:0/1223452477 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f825c156570 con 0x7f825c155810 2024-09-17T04:25:38.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.369+0000 7f8261600640 1 -- 172.21.15.100:0/1223452477 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f825c1f9b30 con 0x7f825c155810 2024-09-17T04:25:38.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.370+0000 7f8253400640 1 -- 172.21.15.100:0/1223452477 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f8244022070 con 0x7f825c155810 2024-09-17T04:25:38.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.371+0000 7f8253400640 1 --2- 172.21.15.100:0/1223452477 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8238078d40 0x7f823807b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:38.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.371+0000 7f8253400640 1 -- 172.21.15.100:0/1223452477 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7f824409ba00 con 0x7f825c155810 2024-09-17T04:25:38.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.371+0000 7f825aa00640 1 --2- 172.21.15.100:0/1223452477 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8238078d40 0x7f823807b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:38.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.372+0000 7f825aa00640 1 --2- 172.21.15.100:0/1223452477 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8238078d40 0x7f823807b250 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f824c009460 tx=0x7f824c005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:38.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.375+0000 7f8253400640 1 -- 172.21.15.100:0/1223452477 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f825c1f9b30 con 0x7f825c155810 2024-09-17T04:25:38.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.559+0000 7f8261600640 1 -- 172.21.15.100:0/1223452477 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f825c160fa0 con 0x7f825c155810 2024-09-17T04:25:38.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.559+0000 7f8253400640 1 -- 172.21.15.100:0/1223452477 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v33) ==== 59+0+2294 (secure 0 0 0) 0x7f8244063770 con 0x7f825c155810 2024-09-17T04:25:38.560 INFO:teuthology.orchestra.run.smithi100.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T04:25:38.560 INFO:teuthology.orchestra.run.smithi100.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T04:25:38.560 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced client_mount_timeout 30 2024-09-17T04:25:38.560 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_client 20/20 2024-09-17T04:25:38.560 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_mgr 20/20 2024-09-17T04:25:38.560 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_ms 1/1 2024-09-17T04:25:38.560 INFO:teuthology.orchestra.run.smithi100.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T04:25:38.560 INFO:teuthology.orchestra.run.smithi100.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T04:25:38.560 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3938.598654 2024-09-17T04:25:38.561 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4008.640154 2024-09-17T04:25:38.561 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3706.588546 2024-09-17T04:25:38.561 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 6299.338665 2024-09-17T04:25:38.561 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3633.876699 2024-09-17T04:25:38.561 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3674.821841 2024-09-17T04:25:38.561 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3789.849984 2024-09-17T04:25:38.561 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3467.457298 2024-09-17T04:25:38.561 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_mds 20/20 2024-09-17T04:25:38.561 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T04:25:38.561 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_ms 2/2 2024-09-17T04:25:38.561 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_debug_frag true 2024-09-17T04:25:38.561 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_debug_scatterstat true 2024-09-17T04:25:38.561 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T04:25:38.562 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_verify_scatter true 2024-09-17T04:25:38.562 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T04:25:38.562 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T04:25:38.562 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T04:25:38.562 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced client_mount_timeout 600 2024-09-17T04:25:38.562 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced debug_client 20/20 2024-09-17T04:25:38.562 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced debug_ms 1/1 2024-09-17T04:25:38.562 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T04:25:38.562 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T04:25:38.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.562+0000 7f8261600640 1 -- 172.21.15.100:0/1223452477 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8238078d40 msgr2=0x7f823807b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:38.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.562+0000 7f8261600640 1 --2- 172.21.15.100:0/1223452477 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8238078d40 0x7f823807b250 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f824c009460 tx=0x7f824c005f90 comp rx=0 tx=0).stop 2024-09-17T04:25:38.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.562+0000 7f8261600640 1 -- 172.21.15.100:0/1223452477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f825c155810 msgr2=0x7f825c155c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:38.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.562+0000 7f8261600640 1 --2- 172.21.15.100:0/1223452477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f825c155810 0x7f825c155c20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8244011a60 tx=0x7f8244011f20 comp rx=0 tx=0).stop 2024-09-17T04:25:38.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.562+0000 7f8261600640 1 -- 172.21.15.100:0/1223452477 shutdown_connections 2024-09-17T04:25:38.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.562+0000 7f8261600640 1 --2- 172.21.15.100:0/1223452477 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8238078d40 0x7f823807b250 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:38.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.562+0000 7f8261600640 1 --2- 172.21.15.100:0/1223452477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f825c156160 0x7f825c1f4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:38.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.562+0000 7f8261600640 1 --2- 172.21.15.100:0/1223452477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f825c155810 0x7f825c155c20 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:38.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.562+0000 7f8261600640 1 --2- 172.21.15.100:0/1223452477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8240005150 0x7f825c1f8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:38.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.562+0000 7f8261600640 1 -- 172.21.15.100:0/1223452477 >> 172.21.15.100:0/1223452477 conn(0x7f825c06d730 msgr2=0x7f825c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:38.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.562+0000 7f8261600640 1 -- 172.21.15.100:0/1223452477 shutdown_connections 2024-09-17T04:25:38.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.562+0000 7f8261600640 1 -- 172.21.15.100:0/1223452477 wait complete. 2024-09-17T04:25:38.586 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T04:25:38.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.809+0000 7fa801e00640 1 -- 172.21.15.100:0/3019538443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7fc0cfab0 msgr2=0x7fa7fc073630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:38.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.809+0000 7fa801e00640 1 --2- 172.21.15.100:0/3019538443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7fc0cfab0 0x7fa7fc073630 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4005fd0 tx=0x7fa7e402a820 comp rx=0 tx=0).stop 2024-09-17T04:25:38.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.809+0000 7fa801e00640 1 -- 172.21.15.100:0/3019538443 shutdown_connections 2024-09-17T04:25:38.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.809+0000 7fa801e00640 1 --2- 172.21.15.100:0/3019538443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7fc0cfab0 0x7fa7fc073630 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:38.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.809+0000 7fa801e00640 1 --2- 172.21.15.100:0/3019538443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7fc0ccc50 0x7fa7fc0cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:38.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.809+0000 7fa801e00640 1 -- 172.21.15.100:0/3019538443 >> 172.21.15.100:0/3019538443 conn(0x7fa7fc06d1c0 msgr2=0x7fa7fc06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:38.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.809+0000 7fa801e00640 1 -- 172.21.15.100:0/3019538443 shutdown_connections 2024-09-17T04:25:38.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.809+0000 7fa801e00640 1 -- 172.21.15.100:0/3019538443 wait complete. 2024-09-17T04:25:38.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.811+0000 7fa801e00640 1 Processor -- start 2024-09-17T04:25:38.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.811+0000 7fa801e00640 1 -- start start 2024-09-17T04:25:38.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.813+0000 7fa801e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7fc073b70 0x7fa7fc14cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:38.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.813+0000 7fa801e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7fc0ccc50 0x7fa7fc13b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:38.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.813+0000 7fa801e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7fc0cfab0 0x7fa7fc1e91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:38.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.813+0000 7fa801e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc15fa00 con 0x7fa7fc073b70 2024-09-17T04:25:38.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.813+0000 7fa801e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc15f880 con 0x7fa7fc0cfab0 2024-09-17T04:25:38.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.813+0000 7fa801e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa7fc15fb80 con 0x7fa7fc0ccc50 2024-09-17T04:25:38.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.813+0000 7fa7faa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7fc073b70 0x7fa7fc14cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:38.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.813+0000 7fa7faa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7fc073b70 0x7fa7fc14cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45336/0 (socket says 172.21.15.100:45336) 2024-09-17T04:25:38.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.813+0000 7fa7faa00640 1 -- 172.21.15.100:0/1748194971 learned_addr learned my addr 172.21.15.100:0/1748194971 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:38.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.813+0000 7fa7f2000640 1 --2- 172.21.15.100:0/1748194971 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7fc0ccc50 0x7fa7fc13b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:38.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.813+0000 7fa7fb400640 1 --2- 172.21.15.100:0/1748194971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7fc0cfab0 0x7fa7fc1e91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:38.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.814+0000 7fa7faa00640 1 -- 172.21.15.100:0/1748194971 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7fc0ccc50 msgr2=0x7fa7fc13b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:38.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.814+0000 7fa7faa00640 1 --2- 172.21.15.100:0/1748194971 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7fc0ccc50 0x7fa7fc13b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:38.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.814+0000 7fa7faa00640 1 -- 172.21.15.100:0/1748194971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7fc0cfab0 msgr2=0x7fa7fc1e91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:38.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.814+0000 7fa7faa00640 1 --2- 172.21.15.100:0/1748194971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7fc0cfab0 0x7fa7fc1e91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:38.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.814+0000 7fa7faa00640 1 -- 172.21.15.100:0/1748194971 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7fc141d80 con 0x7fa7fc073b70 2024-09-17T04:25:38.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.814+0000 7fa7fb400640 1 --2- 172.21.15.100:0/1748194971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7fc0cfab0 0x7fa7fc1e91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:38.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.814+0000 7fa7faa00640 1 --2- 172.21.15.100:0/1748194971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7fc073b70 0x7fa7fc14cfb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00e790 tx=0x7fa7ec00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:38.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.814+0000 7fa7f3e00640 1 -- 172.21.15.100:0/1748194971 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec00eea0 con 0x7fa7fc073b70 2024-09-17T04:25:38.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.814+0000 7fa7f3e00640 1 -- 172.21.15.100:0/1748194971 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7ec00cdb0 con 0x7fa7fc073b70 2024-09-17T04:25:38.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.814+0000 7fa7f3e00640 1 -- 172.21.15.100:0/1748194971 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec010690 con 0x7fa7fc073b70 2024-09-17T04:25:38.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.814+0000 7fa801e00640 1 -- 172.21.15.100:0/1748194971 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7fc151e50 con 0x7fa7fc073b70 2024-09-17T04:25:38.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.814+0000 7fa801e00640 1 -- 172.21.15.100:0/1748194971 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7fc152410 con 0x7fa7fc073b70 2024-09-17T04:25:38.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.815+0000 7fa801e00640 1 -- 172.21.15.100:0/1748194971 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7fc0cf570 con 0x7fa7fc073b70 2024-09-17T04:25:38.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.816+0000 7fa7f3e00640 1 -- 172.21.15.100:0/1748194971 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fa7ec010830 con 0x7fa7fc073b70 2024-09-17T04:25:38.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.818+0000 7fa7f3e00640 1 --2- 172.21.15.100:0/1748194971 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa7d8078d40 0x7fa7d807b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:38.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.818+0000 7fa7f3e00640 1 -- 172.21.15.100:0/1748194971 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7fa7ec09b780 con 0x7fa7fc073b70 2024-09-17T04:25:38.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.819+0000 7fa7f2000640 1 --2- 172.21.15.100:0/1748194971 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa7d8078d40 0x7fa7d807b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:38.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.819+0000 7fa7f2000640 1 --2- 172.21.15.100:0/1748194971 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa7d8078d40 0x7fa7d807b250 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4034000 tx=0x7fa7e4004ab0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:38.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:38.823+0000 7fa7f3e00640 1 -- 172.21.15.100:0/1748194971 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa7ec063560 con 0x7fa7fc073b70 2024-09-17T04:25:39.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.005+0000 7fa801e00640 1 -- 172.21.15.100:0/1748194971 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa7fc0b0620 con 0x7fa7fc073b70 2024-09-17T04:25:39.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.010+0000 7fa7f3e00640 1 -- 172.21.15.100:0/1748194971 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7fa7ec067f90 con 0x7fa7fc073b70 2024-09-17T04:25:39.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.012+0000 7fa801e00640 1 -- 172.21.15.100:0/1748194971 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa7d8078d40 msgr2=0x7fa7d807b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:39.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.012+0000 7fa801e00640 1 --2- 172.21.15.100:0/1748194971 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa7d8078d40 0x7fa7d807b250 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4034000 tx=0x7fa7e4004ab0 comp rx=0 tx=0).stop 2024-09-17T04:25:39.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.012+0000 7fa801e00640 1 -- 172.21.15.100:0/1748194971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7fc073b70 msgr2=0x7fa7fc14cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:39.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.012+0000 7fa801e00640 1 --2- 172.21.15.100:0/1748194971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7fc073b70 0x7fa7fc14cfb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00e790 tx=0x7fa7ec00ec50 comp rx=0 tx=0).stop 2024-09-17T04:25:39.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.012+0000 7fa801e00640 1 -- 172.21.15.100:0/1748194971 shutdown_connections 2024-09-17T04:25:39.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.012+0000 7fa801e00640 1 --2- 172.21.15.100:0/1748194971 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa7d8078d40 0x7fa7d807b250 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:39.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.012+0000 7fa801e00640 1 --2- 172.21.15.100:0/1748194971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7fc0cfab0 0x7fa7fc1e91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:39.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.012+0000 7fa801e00640 1 --2- 172.21.15.100:0/1748194971 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7fc0ccc50 0x7fa7fc13b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:39.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.012+0000 7fa801e00640 1 --2- 172.21.15.100:0/1748194971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7fc073b70 0x7fa7fc14cfb0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:39.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.012+0000 7fa801e00640 1 -- 172.21.15.100:0/1748194971 >> 172.21.15.100:0/1748194971 conn(0x7fa7fc06d1c0 msgr2=0x7fa7fc154c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:39.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.012+0000 7fa801e00640 1 -- 172.21.15.100:0/1748194971 shutdown_connections 2024-09-17T04:25:39.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.012+0000 7fa801e00640 1 -- 172.21.15.100:0/1748194971 wait complete. 2024-09-17T04:25:39.036 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T04:25:39.036 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:25:39.121 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:25:39.213 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T04:25:39.213 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:25:39.213 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:25:39.249 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:25:39.249 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:25:39.249 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:25:39.285 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:25:39.285 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:25:39.285 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:25:39.324 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:25:39.324 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:25:39.324 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:25:39.360 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:25:39.361 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T04:25:39.361 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:25:39.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.584+0000 7fb151200640 1 -- 172.21.15.100:0/23992655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb14c0ad0e0 msgr2=0x7fb14c0ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:39.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.584+0000 7fb151200640 1 --2- 172.21.15.100:0/23992655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb14c0ad0e0 0x7fb14c0ad4f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb134009830 tx=0x7fb13402a480 comp rx=0 tx=0).stop 2024-09-17T04:25:39.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.584+0000 7fb151200640 1 -- 172.21.15.100:0/23992655 shutdown_connections 2024-09-17T04:25:39.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.584+0000 7fb151200640 1 --2- 172.21.15.100:0/23992655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb14c0b3550 0x7fb14c0b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:39.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.584+0000 7fb151200640 1 --2- 172.21.15.100:0/23992655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb14c0ad0e0 0x7fb14c0ad4f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:39.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.584+0000 7fb151200640 1 -- 172.21.15.100:0/23992655 >> 172.21.15.100:0/23992655 conn(0x7fb14c06d730 msgr2=0x7fb14c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:39.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.584+0000 7fb151200640 1 -- 172.21.15.100:0/23992655 shutdown_connections 2024-09-17T04:25:39.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.584+0000 7fb151200640 1 -- 172.21.15.100:0/23992655 wait complete. 2024-09-17T04:25:39.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.587+0000 7fb151200640 1 Processor -- start 2024-09-17T04:25:39.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.587+0000 7fb151200640 1 -- start start 2024-09-17T04:25:39.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.588+0000 7fb151200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb14c0ad0e0 0x7fb14c159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:39.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.588+0000 7fb151200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb14c0b3550 0x7fb14c14a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:39.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.588+0000 7fb151200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb14c15d760 0x7fb14c0b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:39.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.588+0000 7fb151200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c0b4da0 con 0x7fb14c0ad0e0 2024-09-17T04:25:39.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.588+0000 7fb151200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c0b4c20 con 0x7fb14c15d760 2024-09-17T04:25:39.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.588+0000 7fb151200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb14c0b3ea0 con 0x7fb14c0b3550 2024-09-17T04:25:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.588+0000 7fb149e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb14c0ad0e0 0x7fb14c159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.589+0000 7fb149e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb14c0ad0e0 0x7fb14c159e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45340/0 (socket says 172.21.15.100:45340) 2024-09-17T04:25:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.589+0000 7fb149e00640 1 -- 172.21.15.100:0/120224628 learned_addr learned my addr 172.21.15.100:0/120224628 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.589+0000 7fb141400640 1 --2- 172.21.15.100:0/120224628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb14c0b3550 0x7fb14c14a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.589+0000 7fb14a800640 1 --2- 172.21.15.100:0/120224628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb14c15d760 0x7fb14c0b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.589+0000 7fb149e00640 1 -- 172.21.15.100:0/120224628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb14c0b3550 msgr2=0x7fb14c14a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.589+0000 7fb149e00640 1 --2- 172.21.15.100:0/120224628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb14c0b3550 0x7fb14c14a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.589+0000 7fb149e00640 1 -- 172.21.15.100:0/120224628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb14c15d760 msgr2=0x7fb14c0b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.589+0000 7fb149e00640 1 --2- 172.21.15.100:0/120224628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb14c15d760 0x7fb14c0b0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.589+0000 7fb149e00640 1 -- 172.21.15.100:0/120224628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14c15a5a0 con 0x7fb14c0ad0e0 2024-09-17T04:25:39.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.589+0000 7fb149e00640 1 --2- 172.21.15.100:0/120224628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb14c0ad0e0 0x7fb14c159e90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00c730 tx=0x7fb13c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.589+0000 7fb142a00640 1 -- 172.21.15.100:0/120224628 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c016020 con 0x7fb14c0ad0e0 2024-09-17T04:25:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.590+0000 7fb151200640 1 -- 172.21.15.100:0/120224628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb14c1f5990 con 0x7fb14c0ad0e0 2024-09-17T04:25:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.590+0000 7fb142a00640 1 -- 172.21.15.100:0/120224628 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb13c00a950 con 0x7fb14c0ad0e0 2024-09-17T04:25:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.590+0000 7fb142a00640 1 -- 172.21.15.100:0/120224628 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c00ada0 con 0x7fb14c0ad0e0 2024-09-17T04:25:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.590+0000 7fb151200640 1 -- 172.21.15.100:0/120224628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb14c1f5f50 con 0x7fb14c0ad0e0 2024-09-17T04:25:39.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.591+0000 7fb151200640 1 -- 172.21.15.100:0/120224628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb14c0b4370 con 0x7fb14c0ad0e0 2024-09-17T04:25:39.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.592+0000 7fb142a00640 1 -- 172.21.15.100:0/120224628 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fb13c019020 con 0x7fb14c0ad0e0 2024-09-17T04:25:39.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.593+0000 7fb142a00640 1 --2- 172.21.15.100:0/120224628 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb128078d70 0x7fb12807b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:39.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.593+0000 7fb142a00640 1 -- 172.21.15.100:0/120224628 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7fb13c09a8d0 con 0x7fb14c0ad0e0 2024-09-17T04:25:39.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.593+0000 7fb141400640 1 --2- 172.21.15.100:0/120224628 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb128078d70 0x7fb12807b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:39.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.594+0000 7fb141400640 1 --2- 172.21.15.100:0/120224628 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb128078d70 0x7fb12807b280 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb134034000 tx=0x7fb134004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:39.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.597+0000 7fb142a00640 1 -- 172.21.15.100:0/120224628 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb13c020030 con 0x7fb14c0ad0e0 2024-09-17T04:25:39.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.817+0000 7fb151200640 1 -- 172.21.15.100:0/120224628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb14c0cfd50 con 0x7fb14c0ad0e0 2024-09-17T04:25:39.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.817+0000 7fb142a00640 1 -- 172.21.15.100:0/120224628 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+3920 (secure 0 0 0) 0x7fb14c0ada30 con 0x7fb14c0ad0e0 2024-09-17T04:25:39.818 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:39.818 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":20,"btime":"2024-09-17T04:25:32:725275+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4667,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2259943840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":2259943840},{"type":"v1","addr":"172.21.15.100:6835","nonce":2259943840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4670,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/1748167554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1748167554},{"type":"v1","addr":"172.21.15.100:6834","nonce":1748167554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4854,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3159351871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":3159351871},{"type":"v1","addr":"172.21.15.184:6836","nonce":3159351871}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4892,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/545708789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":545708789},{"type":"v1","addr":"172.21.15.184:6837","nonce":545708789}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[]} 2024-09-17T04:25:39.818 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 20 2024-09-17T04:25:39.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.820+0000 7fb151200640 1 -- 172.21.15.100:0/120224628 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb128078d70 msgr2=0x7fb12807b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:39.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.820+0000 7fb151200640 1 --2- 172.21.15.100:0/120224628 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb128078d70 0x7fb12807b280 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb134034000 tx=0x7fb134004730 comp rx=0 tx=0).stop 2024-09-17T04:25:39.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.820+0000 7fb151200640 1 -- 172.21.15.100:0/120224628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb14c0ad0e0 msgr2=0x7fb14c159e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:39.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.820+0000 7fb151200640 1 --2- 172.21.15.100:0/120224628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb14c0ad0e0 0x7fb14c159e90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00c730 tx=0x7fb13c00cbf0 comp rx=0 tx=0).stop 2024-09-17T04:25:39.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.820+0000 7fb151200640 1 -- 172.21.15.100:0/120224628 shutdown_connections 2024-09-17T04:25:39.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.820+0000 7fb151200640 1 --2- 172.21.15.100:0/120224628 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb128078d70 0x7fb12807b280 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:39.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.820+0000 7fb151200640 1 --2- 172.21.15.100:0/120224628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb14c15d760 0x7fb14c0b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:39.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.820+0000 7fb151200640 1 --2- 172.21.15.100:0/120224628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb14c0b3550 0x7fb14c14a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:39.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.820+0000 7fb151200640 1 --2- 172.21.15.100:0/120224628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb14c0ad0e0 0x7fb14c159e90 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:39.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.820+0000 7fb151200640 1 -- 172.21.15.100:0/120224628 >> 172.21.15.100:0/120224628 conn(0x7fb14c06d730 msgr2=0x7fb14c1602a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:39.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.820+0000 7fb151200640 1 -- 172.21.15.100:0/120224628 shutdown_connections 2024-09-17T04:25:39.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:39.820+0000 7fb151200640 1 -- 172.21.15.100:0/120224628 wait complete. 2024-09-17T04:25:39.842 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T04:25:39.842 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T04:25:39.842 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T04:25:39.842 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:25:39.843 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T04:25:39.843 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T04:25:39.843 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T04:25:39.843 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:25:39.843 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T04:25:39.843 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T04:25:39.844 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T04:25:39.844 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:25:39.844 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T04:25:39.844 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T04:25:39.844 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T04:25:39.844 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:25:39.866 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T04:25:39.867 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T04:25:39.868 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T04:25:39.869 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T04:25:39.869 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T04:25:39.869 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T04:25:39.870 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T04:25:39.870 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T04:25:39.912 INFO:tasks.ceph.mds.a:Started 2024-09-17T04:25:39.913 DEBUG:teuthology.parallel:result is None 2024-09-17T04:25:39.913 INFO:tasks.ceph.mds.d:Started 2024-09-17T04:25:39.914 DEBUG:teuthology.parallel:result is None 2024-09-17T04:25:39.914 INFO:tasks.ceph.mds.c:Started 2024-09-17T04:25:39.915 DEBUG:teuthology.parallel:result is None 2024-09-17T04:25:39.916 INFO:tasks.ceph.mds.b:Started 2024-09-17T04:25:39.917 DEBUG:teuthology.parallel:result is None 2024-09-17T04:25:39.917 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T04:25:40.071 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-09-17T04:25:40.072 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-09-17T04:25:40.106 INFO:tasks.ceph.mds.a.smithi100.stdout:starting mds.a at 2024-09-17T04:25:40.109 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-09-17T04:25:40.109 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-09-17T04:25:40.110 INFO:tasks.ceph.mds.c.smithi100.stdout:starting mds.c at 2024-09-17T04:25:40.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.155+0000 7ff326a00640 1 -- 172.21.15.100:0/2918651934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3200cbfc0 msgr2=0x7ff3200af8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:40.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.155+0000 7ff326a00640 1 --2- 172.21.15.100:0/2918651934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3200cbfc0 0x7ff3200af8d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff310005fd0 tx=0x7ff31002a540 comp rx=0 tx=0).stop 2024-09-17T04:25:40.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.155+0000 7ff326a00640 1 -- 172.21.15.100:0/2918651934 shutdown_connections 2024-09-17T04:25:40.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.155+0000 7ff326a00640 1 --2- 172.21.15.100:0/2918651934 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff32006f0d0 0x7ff320070570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:40.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.155+0000 7ff326a00640 1 --2- 172.21.15.100:0/2918651934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3200cbfc0 0x7ff3200af8d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:40.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.155+0000 7ff326a00640 1 -- 172.21.15.100:0/2918651934 >> 172.21.15.100:0/2918651934 conn(0x7ff32006d3d0 msgr2=0x7ff3200b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:40.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.155+0000 7ff326a00640 1 -- 172.21.15.100:0/2918651934 shutdown_connections 2024-09-17T04:25:40.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.155+0000 7ff326a00640 1 -- 172.21.15.100:0/2918651934 wait complete. 2024-09-17T04:25:40.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.157+0000 7ff326a00640 1 Processor -- start 2024-09-17T04:25:40.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.158+0000 7ff326a00640 1 -- start start 2024-09-17T04:25:40.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.159+0000 7ff326a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff32006f0d0 0x7ff3200c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:40.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.159+0000 7ff326a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff320070ab0 0x7ff320154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:40.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.159+0000 7ff326a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3200cbfc0 0x7ff3200d0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:40.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.159+0000 7ff326a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff3200b3ff0 con 0x7ff32006f0d0 2024-09-17T04:25:40.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.159+0000 7ff326a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff3200b3e70 con 0x7ff320070ab0 2024-09-17T04:25:40.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.159+0000 7ff326a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff32006f680 con 0x7ff3200cbfc0 2024-09-17T04:25:40.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.159+0000 7ff31fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff32006f0d0 0x7ff3200c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:40.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.159+0000 7ff31fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff32006f0d0 0x7ff3200c6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45398/0 (socket says 172.21.15.100:45398) 2024-09-17T04:25:40.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.159+0000 7ff317400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff320070ab0 0x7ff320154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:40.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.159+0000 7ff31fe00640 1 -- 172.21.15.100:0/2976677622 learned_addr learned my addr 172.21.15.100:0/2976677622 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:40.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.159+0000 7ff324a00640 1 --2- 172.21.15.100:0/2976677622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3200cbfc0 0x7ff3200d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:40.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.160+0000 7ff317400640 1 -- 172.21.15.100:0/2976677622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3200cbfc0 msgr2=0x7ff3200d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:40.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.160+0000 7ff317400640 1 --2- 172.21.15.100:0/2976677622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3200cbfc0 0x7ff3200d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:40.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.160+0000 7ff317400640 1 -- 172.21.15.100:0/2976677622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff32006f0d0 msgr2=0x7ff3200c6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:40.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.160+0000 7ff317400640 1 --2- 172.21.15.100:0/2976677622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff32006f0d0 0x7ff3200c6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:40.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.160+0000 7ff317400640 1 -- 172.21.15.100:0/2976677622 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3200c6080 con 0x7ff320070ab0 2024-09-17T04:25:40.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.160+0000 7ff31fe00640 1 --2- 172.21.15.100:0/2976677622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff32006f0d0 0x7ff3200c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:25:40.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.160+0000 7ff317400640 1 --2- 172.21.15.100:0/2976677622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff320070ab0 0x7ff320154480 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff30800e9f0 tx=0x7ff30800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:40.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.160+0000 7ff31cc00640 1 -- 172.21.15.100:0/2976677622 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30800ccc0 con 0x7ff320070ab0 2024-09-17T04:25:40.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.161+0000 7ff31cc00640 1 -- 172.21.15.100:0/2976677622 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff308004020 con 0x7ff320070ab0 2024-09-17T04:25:40.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.161+0000 7ff31cc00640 1 -- 172.21.15.100:0/2976677622 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30800ce60 con 0x7ff320070ab0 2024-09-17T04:25:40.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.161+0000 7ff326a00640 1 -- 172.21.15.100:0/2976677622 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3200c7030 con 0x7ff320070ab0 2024-09-17T04:25:40.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.161+0000 7ff326a00640 1 -- 172.21.15.100:0/2976677622 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3200c62f0 con 0x7ff320070ab0 2024-09-17T04:25:40.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.162+0000 7ff326a00640 1 -- 172.21.15.100:0/2976677622 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3200cbd70 con 0x7ff320070ab0 2024-09-17T04:25:40.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.163+0000 7ff31cc00640 1 -- 172.21.15.100:0/2976677622 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7ff3080035f0 con 0x7ff320070ab0 2024-09-17T04:25:40.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.164+0000 7ff31cc00640 1 --2- 172.21.15.100:0/2976677622 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff2fc078d40 0x7ff2fc07b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:40.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.164+0000 7ff31cc00640 1 -- 172.21.15.100:0/2976677622 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4943+0+0 (secure 0 0 0) 0x7ff30809b780 con 0x7ff320070ab0 2024-09-17T04:25:40.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.164+0000 7ff31fe00640 1 --2- 172.21.15.100:0/2976677622 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff2fc078d40 0x7ff2fc07b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:40.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.165+0000 7ff31fe00640 1 --2- 172.21.15.100:0/2976677622 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff2fc078d40 0x7ff2fc07b250 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff310009230 tx=0x7ff310004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:40.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.169+0000 7ff31cc00640 1 -- 172.21.15.100:0/2976677622 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff3200cbd70 con 0x7ff320070ab0 2024-09-17T04:25:40.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:40.365+0000 7ff326a00640 1 -- 172.21.15.100:0/2976677622 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff320158c80 con 0x7ff320070ab0 2024-09-17T04:25:41.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.151+0000 7ff31cc00640 1 -- 172.21.15.100:0/2976677622 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v44) ==== 97+0+0 (secure 0 0 0) 0x7ff308063440 con 0x7ff320070ab0 2024-09-17T04:25:41.151 INFO:teuthology.orchestra.run.smithi100.stderr: removed all blocklist entries 2024-09-17T04:25:41.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.153+0000 7ff326a00640 1 -- 172.21.15.100:0/2976677622 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff2fc078d40 msgr2=0x7ff2fc07b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:41.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.153+0000 7ff326a00640 1 --2- 172.21.15.100:0/2976677622 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff2fc078d40 0x7ff2fc07b250 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff310009230 tx=0x7ff310004730 comp rx=0 tx=0).stop 2024-09-17T04:25:41.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.154+0000 7ff326a00640 1 -- 172.21.15.100:0/2976677622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff320070ab0 msgr2=0x7ff320154480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:41.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.154+0000 7ff326a00640 1 --2- 172.21.15.100:0/2976677622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff320070ab0 0x7ff320154480 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff30800e9f0 tx=0x7ff30800eeb0 comp rx=0 tx=0).stop 2024-09-17T04:25:41.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.154+0000 7ff326a00640 1 -- 172.21.15.100:0/2976677622 shutdown_connections 2024-09-17T04:25:41.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.154+0000 7ff326a00640 1 --2- 172.21.15.100:0/2976677622 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff2fc078d40 0x7ff2fc07b250 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:41.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.154+0000 7ff326a00640 1 --2- 172.21.15.100:0/2976677622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3200cbfc0 0x7ff3200d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:41.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.154+0000 7ff326a00640 1 --2- 172.21.15.100:0/2976677622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff320070ab0 0x7ff320154480 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:41.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.154+0000 7ff326a00640 1 --2- 172.21.15.100:0/2976677622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff32006f0d0 0x7ff3200c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:41.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.154+0000 7ff326a00640 1 -- 172.21.15.100:0/2976677622 >> 172.21.15.100:0/2976677622 conn(0x7ff32006d3d0 msgr2=0x7ff3200cb660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:41.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.154+0000 7ff326a00640 1 -- 172.21.15.100:0/2976677622 shutdown_connections 2024-09-17T04:25:41.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.154+0000 7ff326a00640 1 -- 172.21.15.100:0/2976677622 wait complete. 2024-09-17T04:25:41.177 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T04:25:41.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.400+0000 7f8a44e00640 1 -- 172.21.15.100:0/2353626183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a24004800 msgr2=0x7f8a24004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:41.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.400+0000 7f8a44e00640 1 --2- 172.21.15.100:0/2353626183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a24004800 0x7f8a24004c10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8a400755c0 tx=0x7f8a24005830 comp rx=0 tx=0).stop 2024-09-17T04:25:41.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.400+0000 7f8a3e800640 1 -- 172.21.15.100:0/2353626183 reap_dead start 2024-09-17T04:25:41.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.400+0000 7f8a44e00640 1 -- 172.21.15.100:0/2353626183 shutdown_connections 2024-09-17T04:25:41.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.400+0000 7f8a44e00640 1 -- 172.21.15.100:0/2353626183 >> 172.21.15.100:0/2353626183 conn(0x7f8a4006d730 msgr2=0x7f8a400b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:41.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.400+0000 7f8a44e00640 1 -- 172.21.15.100:0/2353626183 shutdown_connections 2024-09-17T04:25:41.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.400+0000 7f8a44e00640 1 -- 172.21.15.100:0/2353626183 wait complete. 2024-09-17T04:25:41.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.402+0000 7f8a44e00640 1 Processor -- start 2024-09-17T04:25:41.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.403+0000 7f8a44e00640 1 -- start start 2024-09-17T04:25:41.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.404+0000 7f8a44e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a24004800 0x7f8a40149a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:41.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.404+0000 7f8a44e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a4014be50 0x7f8a40148da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:41.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.404+0000 7f8a44e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a401f91d0 0x7f8a401f95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:41.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.404+0000 7f8a44e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8a400b4da0 con 0x7f8a401f91d0 2024-09-17T04:25:41.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.404+0000 7f8a44e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8a400b4c20 con 0x7f8a4014be50 2024-09-17T04:25:41.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.404+0000 7f8a44e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8a400ad4f0 con 0x7f8a24004800 2024-09-17T04:25:41.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.404+0000 7f8a3e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a401f91d0 0x7f8a401f95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:41.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.405+0000 7f8a3e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a401f91d0 0x7f8a401f95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45420/0 (socket says 172.21.15.100:45420) 2024-09-17T04:25:41.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.405+0000 7f8a3e800640 1 -- 172.21.15.100:0/1763450424 learned_addr learned my addr 172.21.15.100:0/1763450424 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:41.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.405+0000 7f8a3de00640 1 --2- 172.21.15.100:0/1763450424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a24004800 0x7f8a40149a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:41.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.405+0000 7f8a35400640 1 --2- 172.21.15.100:0/1763450424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a4014be50 0x7f8a40148da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:41.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.405+0000 7f8a3e800640 1 -- 172.21.15.100:0/1763450424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a24004800 msgr2=0x7f8a40149a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:41.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.405+0000 7f8a3e800640 1 --2- 172.21.15.100:0/1763450424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a24004800 0x7f8a40149a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:41.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.405+0000 7f8a3e800640 1 -- 172.21.15.100:0/1763450424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a4014be50 msgr2=0x7f8a40148da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:41.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.405+0000 7f8a3e800640 1 --2- 172.21.15.100:0/1763450424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a4014be50 0x7f8a40148da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:41.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.405+0000 7f8a3e800640 1 -- 172.21.15.100:0/1763450424 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a401f9b20 con 0x7f8a401f91d0 2024-09-17T04:25:41.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.405+0000 7f8a3e800640 1 --2- 172.21.15.100:0/1763450424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a401f91d0 0x7f8a401f95e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8a400aea60 tx=0x7f8a38006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:41.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.406+0000 7f8a36a00640 1 -- 172.21.15.100:0/1763450424 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a38011900 con 0x7f8a401f91d0 2024-09-17T04:25:41.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.406+0000 7f8a36a00640 1 -- 172.21.15.100:0/1763450424 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a38018070 con 0x7f8a401f91d0 2024-09-17T04:25:41.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.406+0000 7f8a36a00640 1 -- 172.21.15.100:0/1763450424 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a38011d20 con 0x7f8a401f91d0 2024-09-17T04:25:41.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.406+0000 7f8a44e00640 1 -- 172.21.15.100:0/1763450424 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a4014c630 con 0x7f8a401f91d0 2024-09-17T04:25:41.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.406+0000 7f8a44e00640 1 -- 172.21.15.100:0/1763450424 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a400b0560 con 0x7f8a401f91d0 2024-09-17T04:25:41.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.407+0000 7f8a44e00640 1 -- 172.21.15.100:0/1763450424 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a401535c0 con 0x7f8a401f91d0 2024-09-17T04:25:41.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.408+0000 7f8a36a00640 1 -- 172.21.15.100:0/1763450424 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f8a3801d460 con 0x7f8a401f91d0 2024-09-17T04:25:41.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.409+0000 7f8a36a00640 1 --2- 172.21.15.100:0/1763450424 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8a1c078d40 0x7f8a1c07b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:41.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.409+0000 7f8a36a00640 1 -- 172.21.15.100:0/1763450424 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4857+0+0 (secure 0 0 0) 0x7f8a3809b730 con 0x7f8a401f91d0 2024-09-17T04:25:41.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.409+0000 7f8a3de00640 1 --2- 172.21.15.100:0/1763450424 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8a1c078d40 0x7f8a1c07b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:41.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.410+0000 7f8a3de00640 1 --2- 172.21.15.100:0/1763450424 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8a1c078d40 0x7f8a1c07b250 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8a300010e0 tx=0x7f8a30006820 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:41.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.413+0000 7f8a36a00640 1 -- 172.21.15.100:0/1763450424 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8a401f9b20 con 0x7f8a401f91d0 2024-09-17T04:25:41.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.663+0000 7f8a44e00640 1 -- 172.21.15.100:0/1763450424 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f8a4014c3c0 con 0x7f8a401f91d0 2024-09-17T04:25:41.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.663+0000 7f8a36a00640 1 -- 172.21.15.100:0/1763450424 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7f8a38063440 con 0x7f8a401f91d0 2024-09-17T04:25:41.664 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:41.664 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:25:41.664 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_dump": [ 2024-09-17T04:25:41.664 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:41.664 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.a", 2024-09-17T04:25:41.664 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmV8asJRAAkSgIWqGc1OeSRbs9PEOT8w==", 2024-09-17T04:25:41.664 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:41.664 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:25:41.664 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:25:41.665 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:25:41.665 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:41.665 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:41.665 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:41.665 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:41.665 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.b", 2024-09-17T04:25:41.665 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmpM7ZBRAA1Og785tJwa6NA9zBdmNNQA==", 2024-09-17T04:25:41.665 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:41.665 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:25:41.665 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:25:41.665 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:25:41.665 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:41.666 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:41.666 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:41.666 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:41.666 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.c", 2024-09-17T04:25:41.666 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmXaOGMxAAT+96g/5fAzPW8Cc2HtQUyA==", 2024-09-17T04:25:41.666 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:41.666 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:25:41.666 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:25:41.666 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:25:41.666 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:41.666 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:41.666 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:41.666 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:41.666 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.d", 2024-09-17T04:25:41.667 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmRqC9ExAA23kZ0Ks1lGN2k4GXL8ASrg==", 2024-09-17T04:25:41.667 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:41.667 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:25:41.667 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:25:41.667 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:25:41.667 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:41.667 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:41.667 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:41.667 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:41.667 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.0", 2024-09-17T04:25:41.667 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAwBOlmofFmAxAARerYkU9B2a9MM9vHa3o4LQ==", 2024-09-17T04:25:41.667 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:41.667 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:25:41.667 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:25:41.668 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:41.668 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:41.668 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:41.668 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:41.668 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.1", 2024-09-17T04:25:41.668 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAzBOlmbz6VExAAoVqSqXTc1wArKhdA6+NxnA==", 2024-09-17T04:25:41.668 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:41.668 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:25:41.668 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:25:41.668 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:41.668 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:41.668 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:41.668 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:41.669 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.2", 2024-09-17T04:25:41.669 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA2BOlmfeSAJBAAchY6XhyDRusc+zqwQ6v6wQ==", 2024-09-17T04:25:41.669 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:41.669 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:25:41.669 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:25:41.669 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:41.669 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:41.669 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:41.669 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:41.669 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.3", 2024-09-17T04:25:41.669 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA6BOlm/TQiBBAAHhbs2+RCjAenGR1frhbTmQ==", 2024-09-17T04:25:41.669 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:41.669 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:25:41.669 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:25:41.669 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:41.670 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:41.670 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:41.670 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:41.670 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.4", 2024-09-17T04:25:41.670 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBABOlm7OKvBhAALkaAKEUYN5ChwGejoSF8yA==", 2024-09-17T04:25:41.670 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:41.670 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:25:41.670 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:25:41.670 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:41.670 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:41.670 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:41.670 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:41.670 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.5", 2024-09-17T04:25:41.670 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBDBOlmQpi1FhAAbA47dg6ckK1PFoCZL0SBEQ==", 2024-09-17T04:25:41.671 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:41.671 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:25:41.671 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:25:41.671 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:41.671 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:41.671 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:41.671 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:41.671 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.6", 2024-09-17T04:25:41.671 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBGBOlmx/w0JhAAWcPvZkMYtXD/EMKrg/W/4Q==", 2024-09-17T04:25:41.671 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:41.671 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:25:41.671 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:25:41.671 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:41.672 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:41.672 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:41.672 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:41.672 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.7", 2024-09-17T04:25:41.672 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBJBOlmWHjWNBAAgdul2UKY9g5p7EVi0ZdrFw==", 2024-09-17T04:25:41.672 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:41.672 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:25:41.672 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:25:41.672 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:41.672 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:41.672 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:41.672 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:41.672 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.0", 2024-09-17T04:25:41.672 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmfYg9IBAABP1uRKGOugrA47VxwRLkjA==", 2024-09-17T04:25:41.673 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:41.673 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:25:41.673 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow r", 2024-09-17T04:25:41.673 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T04:25:41.673 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:41.673 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:41.673 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:41.673 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.1", 2024-09-17T04:25:41.673 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmByhhNRAAPGaxFCDd8ywFVwwhprrPbQ==", 2024-09-17T04:25:41.673 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:41.673 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:25:41.673 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:25:41.673 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:25:41.673 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:25:41.674 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:41.674 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:41.674 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:41.674 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.2", 2024-09-17T04:25:41.674 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmgcm0KhAAGF06VlHjBM/AGxafN6/3aA==", 2024-09-17T04:25:41.674 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:41.674 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:25:41.674 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:25:41.674 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:25:41.674 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:25:41.674 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:41.674 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:41.674 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:41.675 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.3", 2024-09-17T04:25:41.675 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtBOlm9VQ6BBAAXcahu35/uDm8Z+f+3SF0Gg==", 2024-09-17T04:25:41.675 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:41.675 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:25:41.675 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:25:41.675 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:25:41.675 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:25:41.675 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:41.675 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:41.675 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:41.675 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.admin", 2024-09-17T04:25:41.675 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAqBOlmZ+bIJBAAG3oaMNlYWNakywGDvAGbGw==", 2024-09-17T04:25:41.675 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:41.676 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:25:41.676 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:25:41.676 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:25:41.676 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:41.676 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:41.676 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:41.676 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:41.676 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mgr.x", 2024-09-17T04:25:41.676 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmfaL7GhAArqLH66RkbBsz+TNOKtPHOA==", 2024-09-17T04:25:41.676 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:41.676 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:25:41.676 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile mgr", 2024-09-17T04:25:41.676 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:41.677 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:41.677 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:25:41.677 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:41.677 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mgr.y", 2024-09-17T04:25:41.677 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmM4D1DhAA5uUWgRVOMujWqbYzOjFp9A==", 2024-09-17T04:25:41.677 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:25:41.677 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:25:41.677 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile mgr", 2024-09-17T04:25:41.677 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:25:41.677 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:41.677 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:41.677 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-17T04:25:41.677 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:25:41.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.666+0000 7f8a44e00640 1 -- 172.21.15.100:0/1763450424 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8a1c078d40 msgr2=0x7f8a1c07b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:41.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.666+0000 7f8a44e00640 1 --2- 172.21.15.100:0/1763450424 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8a1c078d40 0x7f8a1c07b250 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8a300010e0 tx=0x7f8a30006820 comp rx=0 tx=0).stop 2024-09-17T04:25:41.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.666+0000 7f8a44e00640 1 -- 172.21.15.100:0/1763450424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a401f91d0 msgr2=0x7f8a401f95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:41.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.666+0000 7f8a44e00640 1 --2- 172.21.15.100:0/1763450424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a401f91d0 0x7f8a401f95e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8a400aea60 tx=0x7f8a38006430 comp rx=0 tx=0).stop 2024-09-17T04:25:41.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.666+0000 7f8a44e00640 1 -- 172.21.15.100:0/1763450424 shutdown_connections 2024-09-17T04:25:41.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.666+0000 7f8a44e00640 1 --2- 172.21.15.100:0/1763450424 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8a1c078d40 0x7f8a1c07b250 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:41.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.666+0000 7f8a44e00640 1 --2- 172.21.15.100:0/1763450424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a401f91d0 0x7f8a401f95e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:41.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.666+0000 7f8a44e00640 1 --2- 172.21.15.100:0/1763450424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a4014be50 0x7f8a40148da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:41.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.666+0000 7f8a44e00640 1 --2- 172.21.15.100:0/1763450424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a24004800 0x7f8a40149a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:41.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.666+0000 7f8a44e00640 1 -- 172.21.15.100:0/1763450424 >> 172.21.15.100:0/1763450424 conn(0x7f8a4006d730 msgr2=0x7f8a400700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:41.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.666+0000 7f8a44e00640 1 -- 172.21.15.100:0/1763450424 shutdown_connections 2024-09-17T04:25:41.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.666+0000 7f8a44e00640 1 -- 172.21.15.100:0/1763450424 wait complete. 2024-09-17T04:25:41.689 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T04:25:41.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.908+0000 7fb627200640 1 -- 172.21.15.100:0/3296576061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb62815a0f0 msgr2=0x7fb62815d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:41.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.908+0000 7fb627200640 1 --2- 172.21.15.100:0/3296576061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb62815a0f0 0x7fb62815d670 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb620005fd0 tx=0x7fb62002a820 comp rx=0 tx=0).stop 2024-09-17T04:25:41.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.908+0000 7fb627200640 1 -- 172.21.15.100:0/3296576061 shutdown_connections 2024-09-17T04:25:41.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.908+0000 7fb627200640 1 --2- 172.21.15.100:0/3296576061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb62815a0f0 0x7fb62815d670 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:41.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.908+0000 7fb627200640 1 --2- 172.21.15.100:0/3296576061 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6280ad9c0 0x7fb6280af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:41.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.908+0000 7fb627200640 1 -- 172.21.15.100:0/3296576061 >> 172.21.15.100:0/3296576061 conn(0x7fb6280b1e40 msgr2=0x7fb6280b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:41.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.908+0000 7fb627200640 1 -- 172.21.15.100:0/3296576061 shutdown_connections 2024-09-17T04:25:41.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.908+0000 7fb627200640 1 -- 172.21.15.100:0/3296576061 wait complete. 2024-09-17T04:25:41.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.910+0000 7fb627200640 1 Processor -- start 2024-09-17T04:25:41.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.911+0000 7fb627200640 1 -- start start 2024-09-17T04:25:41.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.912+0000 7fb627200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6280ad9c0 0x7fb6281486b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:41.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.912+0000 7fb627200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6280b2f60 0x7fb62814a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:41.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.912+0000 7fb627200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb62815a0f0 0x7fb62814a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:41.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.912+0000 7fb627200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb62806f410 con 0x7fb6280ad9c0 2024-09-17T04:25:41.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.912+0000 7fb627200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb62806f290 con 0x7fb6280b2f60 2024-09-17T04:25:41.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.912+0000 7fb627200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb62806f590 con 0x7fb62815a0f0 2024-09-17T04:25:41.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.912+0000 7fb625e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6280ad9c0 0x7fb6281486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:41.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.912+0000 7fb625e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6280ad9c0 0x7fb6281486b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45446/0 (socket says 172.21.15.100:45446) 2024-09-17T04:25:41.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.912+0000 7fb625e00640 1 -- 172.21.15.100:0/202113918 learned_addr learned my addr 172.21.15.100:0/202113918 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:41.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.912+0000 7fb61d400640 1 --2- 172.21.15.100:0/202113918 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6280b2f60 0x7fb62814a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:41.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.912+0000 7fb626800640 1 --2- 172.21.15.100:0/202113918 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb62815a0f0 0x7fb62814a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:41.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.913+0000 7fb625e00640 1 -- 172.21.15.100:0/202113918 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb62815a0f0 msgr2=0x7fb62814a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:41.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.913+0000 7fb625e00640 1 --2- 172.21.15.100:0/202113918 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb62815a0f0 0x7fb62814a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:41.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.913+0000 7fb625e00640 1 -- 172.21.15.100:0/202113918 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6280b2f60 msgr2=0x7fb62814a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:41.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.913+0000 7fb625e00640 1 --2- 172.21.15.100:0/202113918 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6280b2f60 0x7fb62814a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:41.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.913+0000 7fb625e00640 1 -- 172.21.15.100:0/202113918 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb628148f60 con 0x7fb6280ad9c0 2024-09-17T04:25:41.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.913+0000 7fb625e00640 1 --2- 172.21.15.100:0/202113918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6280ad9c0 0x7fb6281486b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb61800e790 tx=0x7fb61800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:41.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.913+0000 7fb61ea00640 1 -- 172.21.15.100:0/202113918 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61800eea0 con 0x7fb6280ad9c0 2024-09-17T04:25:41.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.913+0000 7fb61ea00640 1 -- 172.21.15.100:0/202113918 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb61800cdb0 con 0x7fb6280ad9c0 2024-09-17T04:25:41.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.913+0000 7fb627200640 1 -- 172.21.15.100:0/202113918 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb628149240 con 0x7fb6280ad9c0 2024-09-17T04:25:41.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.913+0000 7fb61ea00640 1 -- 172.21.15.100:0/202113918 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb618010690 con 0x7fb6280ad9c0 2024-09-17T04:25:41.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.913+0000 7fb627200640 1 -- 172.21.15.100:0/202113918 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb628149800 con 0x7fb6280ad9c0 2024-09-17T04:25:41.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.914+0000 7fb627200640 1 -- 172.21.15.100:0/202113918 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb62806fa20 con 0x7fb6280ad9c0 2024-09-17T04:25:41.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.916+0000 7fb61ea00640 1 -- 172.21.15.100:0/202113918 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fb618010830 con 0x7fb6280ad9c0 2024-09-17T04:25:41.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.917+0000 7fb61ea00640 1 --2- 172.21.15.100:0/202113918 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb604078d40 0x7fb60407b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:41.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.917+0000 7fb61ea00640 1 -- 172.21.15.100:0/202113918 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4857+0+0 (secure 0 0 0) 0x7fb61809b810 con 0x7fb6280ad9c0 2024-09-17T04:25:41.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.917+0000 7fb61d400640 1 --2- 172.21.15.100:0/202113918 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb604078d40 0x7fb60407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:41.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.918+0000 7fb61d400640 1 --2- 172.21.15.100:0/202113918 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb604078d40 0x7fb60407b250 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb62815d7a0 tx=0x7fb610009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:41.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:41.921+0000 7fb61ea00640 1 -- 172.21.15.100:0/202113918 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb618063c10 con 0x7fb6280ad9c0 2024-09-17T04:25:42.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.109+0000 7fb627200640 1 -- 172.21.15.100:0/202113918 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb62815d4f0 con 0x7fb6280ad9c0 2024-09-17T04:25:42.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.109+0000 7fb61ea00640 1 -- 172.21.15.100:0/202113918 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v44) ==== 84+0+65 (secure 0 0 0) 0x7fb618063510 con 0x7fb6280ad9c0 2024-09-17T04:25:42.110 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:42.110 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-09-17T04:25:42.110 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:42.110 INFO:teuthology.orchestra.run.smithi100.stdout: "poolnum": 1, 2024-09-17T04:25:42.110 INFO:teuthology.orchestra.run.smithi100.stdout: "poolname": ".mgr" 2024-09-17T04:25:42.110 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:42.110 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-09-17T04:25:42.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.111+0000 7fb627200640 1 -- 172.21.15.100:0/202113918 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb604078d40 msgr2=0x7fb60407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:42.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.111+0000 7fb627200640 1 --2- 172.21.15.100:0/202113918 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb604078d40 0x7fb60407b250 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb62815d7a0 tx=0x7fb610009040 comp rx=0 tx=0).stop 2024-09-17T04:25:42.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.111+0000 7fb627200640 1 -- 172.21.15.100:0/202113918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6280ad9c0 msgr2=0x7fb6281486b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:42.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.111+0000 7fb627200640 1 --2- 172.21.15.100:0/202113918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6280ad9c0 0x7fb6281486b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb61800e790 tx=0x7fb61800ec50 comp rx=0 tx=0).stop 2024-09-17T04:25:42.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.112+0000 7fb627200640 1 -- 172.21.15.100:0/202113918 shutdown_connections 2024-09-17T04:25:42.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.112+0000 7fb627200640 1 --2- 172.21.15.100:0/202113918 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb604078d40 0x7fb60407b250 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:42.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.112+0000 7fb627200640 1 --2- 172.21.15.100:0/202113918 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb62815a0f0 0x7fb62814a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:42.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.112+0000 7fb627200640 1 --2- 172.21.15.100:0/202113918 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6280b2f60 0x7fb62814a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:42.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.112+0000 7fb627200640 1 --2- 172.21.15.100:0/202113918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6280ad9c0 0x7fb6281486b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:42.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.112+0000 7fb627200640 1 -- 172.21.15.100:0/202113918 >> 172.21.15.100:0/202113918 conn(0x7fb6280b1e40 msgr2=0x7fb62815ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:42.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.112+0000 7fb627200640 1 -- 172.21.15.100:0/202113918 shutdown_connections 2024-09-17T04:25:42.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.112+0000 7fb627200640 1 -- 172.21.15.100:0/202113918 wait complete. 2024-09-17T04:25:42.134 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T04:25:42.134 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T04:25:42.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.361+0000 7f6f55600640 1 -- 172.21.15.100:0/673098909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f34004d40 msgr2=0x7f6f34005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:42.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.361+0000 7f6f55600640 1 --2- 172.21.15.100:0/673098909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f34004d40 0x7f6f34005150 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6f48005fd0 tx=0x7f6f34005830 comp rx=0 tx=0).stop 2024-09-17T04:25:42.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.361+0000 7f6f54c00640 1 -- 172.21.15.100:0/673098909 reap_dead start 2024-09-17T04:25:42.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.361+0000 7f6f55600640 1 -- 172.21.15.100:0/673098909 shutdown_connections 2024-09-17T04:25:42.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.361+0000 7f6f55600640 1 -- 172.21.15.100:0/673098909 >> 172.21.15.100:0/673098909 conn(0x7f6f500b1dd0 msgr2=0x7f6f500b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:42.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.361+0000 7f6f55600640 1 -- 172.21.15.100:0/673098909 shutdown_connections 2024-09-17T04:25:42.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.361+0000 7f6f55600640 1 -- 172.21.15.100:0/673098909 wait complete. 2024-09-17T04:25:42.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.364+0000 7f6f55600640 1 Processor -- start 2024-09-17T04:25:42.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.364+0000 7f6f55600640 1 -- start start 2024-09-17T04:25:42.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.365+0000 7f6f55600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f34004d40 0x7f6f5015c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:42.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.365+0000 7f6f55600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f50148960 0x7f6f5009d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:42.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.365+0000 7f6f55600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f5009d700 0x7f6f501f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:42.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.365+0000 7f6f55600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6f50160b80 con 0x7f6f34004d40 2024-09-17T04:25:42.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.365+0000 7f6f55600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6f50160a00 con 0x7f6f5009d700 2024-09-17T04:25:42.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.365+0000 7f6f55600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6f500b47c0 con 0x7f6f50148960 2024-09-17T04:25:42.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.365+0000 7f6f4fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f34004d40 0x7f6f5015c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:42.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.365+0000 7f6f4fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f34004d40 0x7f6f5015c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45464/0 (socket says 172.21.15.100:45464) 2024-09-17T04:25:42.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.365+0000 7f6f4fe00640 1 -- 172.21.15.100:0/533574599 learned_addr learned my addr 172.21.15.100:0/533574599 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:42.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.366+0000 7f6f47400640 1 --2- 172.21.15.100:0/533574599 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f50148960 0x7f6f5009d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:42.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.366+0000 7f6f54c00640 1 --2- 172.21.15.100:0/533574599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f5009d700 0x7f6f501f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:42.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.366+0000 7f6f4fe00640 1 -- 172.21.15.100:0/533574599 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f50148960 msgr2=0x7f6f5009d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:42.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.366+0000 7f6f4fe00640 1 --2- 172.21.15.100:0/533574599 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f50148960 0x7f6f5009d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:42.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.366+0000 7f6f4fe00640 1 -- 172.21.15.100:0/533574599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f5009d700 msgr2=0x7f6f501f89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:42.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.366+0000 7f6f4fe00640 1 --2- 172.21.15.100:0/533574599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f5009d700 0x7f6f501f89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:42.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.366+0000 7f6f4fe00640 1 -- 172.21.15.100:0/533574599 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f5009db10 con 0x7f6f34004d40 2024-09-17T04:25:42.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.366+0000 7f6f54c00640 1 --2- 172.21.15.100:0/533574599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f5009d700 0x7f6f501f89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:25:42.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.366+0000 7f6f4fe00640 1 --2- 172.21.15.100:0/533574599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f34004d40 0x7f6f5015c170 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6f40001eb0 tx=0x7f6f40003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:42.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.366+0000 7f6f4cc00640 1 -- 172.21.15.100:0/533574599 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f40015070 con 0x7f6f34004d40 2024-09-17T04:25:42.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.366+0000 7f6f4cc00640 1 -- 172.21.15.100:0/533574599 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f4000f9a0 con 0x7f6f34004d40 2024-09-17T04:25:42.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.366+0000 7f6f4cc00640 1 -- 172.21.15.100:0/533574599 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4000fdf0 con 0x7f6f34004d40 2024-09-17T04:25:42.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.367+0000 7f6f55600640 1 -- 172.21.15.100:0/533574599 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f5015cb90 con 0x7f6f34004d40 2024-09-17T04:25:42.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.367+0000 7f6f55600640 1 -- 172.21.15.100:0/533574599 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f501e5bb0 con 0x7f6f34004d40 2024-09-17T04:25:42.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.368+0000 7f6f55600640 1 -- 172.21.15.100:0/533574599 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f501602b0 con 0x7f6f34004d40 2024-09-17T04:25:42.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.369+0000 7f6f4cc00640 1 -- 172.21.15.100:0/533574599 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f6f40011430 con 0x7f6f34004d40 2024-09-17T04:25:42.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.370+0000 7f6f4cc00640 1 --2- 172.21.15.100:0/533574599 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6f2c078d40 0x7f6f2c07b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:42.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.370+0000 7f6f4cc00640 1 -- 172.21.15.100:0/533574599 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4857+0+0 (secure 0 0 0) 0x7f6f4009bc50 con 0x7f6f34004d40 2024-09-17T04:25:42.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.370+0000 7f6f47400640 1 --2- 172.21.15.100:0/533574599 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6f2c078d40 0x7f6f2c07b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:42.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.371+0000 7f6f47400640 1 --2- 172.21.15.100:0/533574599 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6f2c078d40 0x7f6f2c07b250 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6f5015cb20 tx=0x7f6f38005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:42.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.374+0000 7f6f4cc00640 1 -- 172.21.15.100:0/533574599 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6f5009db10 con 0x7f6f34004d40 2024-09-17T04:25:42.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:42.558+0000 7f6f55600640 1 -- 172.21.15.100:0/533574599 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f6f5015d610 con 0x7f6f34004d40 2024-09-17T04:25:43.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.160+0000 7f6f4cc00640 1 -- 172.21.15.100:0/533574599 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v45) ==== 138+0+0 (secure 0 0 0) 0x7f6f500b4ec0 con 0x7f6f34004d40 2024-09-17T04:25:43.161 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' created 2024-09-17T04:25:43.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.163+0000 7f6f55600640 1 -- 172.21.15.100:0/533574599 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6f2c078d40 msgr2=0x7f6f2c07b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:43.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.163+0000 7f6f55600640 1 --2- 172.21.15.100:0/533574599 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6f2c078d40 0x7f6f2c07b250 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6f5015cb20 tx=0x7f6f38005f90 comp rx=0 tx=0).stop 2024-09-17T04:25:43.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.163+0000 7f6f55600640 1 -- 172.21.15.100:0/533574599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f34004d40 msgr2=0x7f6f5015c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:43.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.163+0000 7f6f55600640 1 --2- 172.21.15.100:0/533574599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f34004d40 0x7f6f5015c170 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6f40001eb0 tx=0x7f6f40003a80 comp rx=0 tx=0).stop 2024-09-17T04:25:43.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.164+0000 7f6f55600640 1 -- 172.21.15.100:0/533574599 shutdown_connections 2024-09-17T04:25:43.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.164+0000 7f6f55600640 1 --2- 172.21.15.100:0/533574599 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6f2c078d40 0x7f6f2c07b250 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:43.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.164+0000 7f6f55600640 1 --2- 172.21.15.100:0/533574599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f5009d700 0x7f6f501f89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:43.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.164+0000 7f6f55600640 1 --2- 172.21.15.100:0/533574599 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f50148960 0x7f6f5009d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:43.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.164+0000 7f6f55600640 1 --2- 172.21.15.100:0/533574599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f34004d40 0x7f6f5015c170 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:43.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.164+0000 7f6f55600640 1 -- 172.21.15.100:0/533574599 >> 172.21.15.100:0/533574599 conn(0x7f6f500b1dd0 msgr2=0x7f6f50155ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:43.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.164+0000 7f6f55600640 1 -- 172.21.15.100:0/533574599 shutdown_connections 2024-09-17T04:25:43.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.164+0000 7f6f55600640 1 -- 172.21.15.100:0/533574599 wait complete. 2024-09-17T04:25:43.192 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T04:25:43.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.420+0000 7faae4a00640 1 -- 172.21.15.100:0/1028463729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faac4003a30 msgr2=0x7faac4003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:43.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.420+0000 7faae4a00640 1 --2- 172.21.15.100:0/1028463729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faac4003a30 0x7faac4003ea0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7faad00016c0 tx=0x7faac4005830 comp rx=0 tx=0).stop 2024-09-17T04:25:43.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.420+0000 7faade800640 1 -- 172.21.15.100:0/1028463729 reap_dead start 2024-09-17T04:25:43.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.420+0000 7faae4a00640 1 -- 172.21.15.100:0/1028463729 shutdown_connections 2024-09-17T04:25:43.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.420+0000 7faae4a00640 1 -- 172.21.15.100:0/1028463729 >> 172.21.15.100:0/1028463729 conn(0x7faae006d1c0 msgr2=0x7faae006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:43.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.420+0000 7faae4a00640 1 -- 172.21.15.100:0/1028463729 shutdown_connections 2024-09-17T04:25:43.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.420+0000 7faae4a00640 1 -- 172.21.15.100:0/1028463729 wait complete. 2024-09-17T04:25:43.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.422+0000 7faae4a00640 1 Processor -- start 2024-09-17T04:25:43.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.422+0000 7faae4a00640 1 -- start start 2024-09-17T04:25:43.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.424+0000 7faae4a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faac4003a30 0x7faae01642f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:43.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.424+0000 7faae4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faae01f9890 0x7faae013ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:43.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.424+0000 7faae4a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faae01f8cf0 0x7faae01f9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:43.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.424+0000 7faae4a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faae0165490 con 0x7faae01f9890 2024-09-17T04:25:43.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.424+0000 7faae4a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faae00ce8e0 con 0x7faae01f8cf0 2024-09-17T04:25:43.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.424+0000 7faae4a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faae0165610 con 0x7faac4003a30 2024-09-17T04:25:43.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.424+0000 7faad5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faae01f9890 0x7faae013ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:43.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.424+0000 7faad5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faae01f9890 0x7faae013ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45482/0 (socket says 172.21.15.100:45482) 2024-09-17T04:25:43.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.424+0000 7faad5400640 1 -- 172.21.15.100:0/752210907 learned_addr learned my addr 172.21.15.100:0/752210907 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:43.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.424+0000 7faade800640 1 --2- 172.21.15.100:0/752210907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faae01f8cf0 0x7faae01f9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:43.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.424+0000 7faadde00640 1 --2- 172.21.15.100:0/752210907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faac4003a30 0x7faae01642f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:43.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.424+0000 7faad5400640 1 -- 172.21.15.100:0/752210907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faac4003a30 msgr2=0x7faae01642f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:43.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.425+0000 7faad5400640 1 --2- 172.21.15.100:0/752210907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faac4003a30 0x7faae01642f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:43.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.425+0000 7faad5400640 1 -- 172.21.15.100:0/752210907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faae01f8cf0 msgr2=0x7faae01f9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:43.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.425+0000 7faad5400640 1 --2- 172.21.15.100:0/752210907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faae01f8cf0 0x7faae01f9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:43.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.425+0000 7faad5400640 1 -- 172.21.15.100:0/752210907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae01f9640 con 0x7faae01f9890 2024-09-17T04:25:43.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.425+0000 7faade800640 1 --2- 172.21.15.100:0/752210907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faae01f8cf0 0x7faae01f9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:25:43.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.425+0000 7faadde00640 1 --2- 172.21.15.100:0/752210907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faac4003a30 0x7faae01642f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:25:43.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.425+0000 7faad5400640 1 --2- 172.21.15.100:0/752210907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faae01f9890 0x7faae013ed10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7faac80066b0 tx=0x7faac80039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:43.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.425+0000 7faad6a00640 1 -- 172.21.15.100:0/752210907 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac8015070 con 0x7faae01f9890 2024-09-17T04:25:43.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.425+0000 7faad6a00640 1 -- 172.21.15.100:0/752210907 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faac8011900 con 0x7faae01f9890 2024-09-17T04:25:43.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.425+0000 7faae4a00640 1 -- 172.21.15.100:0/752210907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae0164b40 con 0x7faae01f9890 2024-09-17T04:25:43.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.425+0000 7faad6a00640 1 -- 172.21.15.100:0/752210907 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac8011d50 con 0x7faae01f9890 2024-09-17T04:25:43.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.425+0000 7faae4a00640 1 -- 172.21.15.100:0/752210907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae0165100 con 0x7faae01f9890 2024-09-17T04:25:43.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.426+0000 7faae4a00640 1 -- 172.21.15.100:0/752210907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faae01648b0 con 0x7faae01f9890 2024-09-17T04:25:43.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.428+0000 7faad6a00640 1 -- 172.21.15.100:0/752210907 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7faac8018070 con 0x7faae01f9890 2024-09-17T04:25:43.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.429+0000 7faad6a00640 1 --2- 172.21.15.100:0/752210907 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faabc078d70 0x7faabc07b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:43.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.429+0000 7faad6a00640 1 -- 172.21.15.100:0/752210907 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5227+0+0 (secure 0 0 0) 0x7faac809bc40 con 0x7faae01f9890 2024-09-17T04:25:43.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.429+0000 7faadde00640 1 --2- 172.21.15.100:0/752210907 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faabc078d70 0x7faabc07b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:43.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.430+0000 7faadde00640 1 --2- 172.21.15.100:0/752210907 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faabc078d70 0x7faabc07b280 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7faad0032000 tx=0x7faad0001190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:43.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.433+0000 7faad6a00640 1 -- 172.21.15.100:0/752210907 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7faae01f9640 con 0x7faae01f9890 2024-09-17T04:25:43.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:43.618+0000 7faae4a00640 1 -- 172.21.15.100:0/752210907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7faae0051050 con 0x7faae01f9890 2024-09-17T04:25:44.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.170+0000 7faad6a00640 1 -- 172.21.15.100:0/752210907 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v46) ==== 170+0+0 (secure 0 0 0) 0x7faae0051050 con 0x7faae01f9890 2024-09-17T04:25:44.170 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' created 2024-09-17T04:25:44.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.172+0000 7faae4a00640 1 -- 172.21.15.100:0/752210907 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faabc078d70 msgr2=0x7faabc07b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:44.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.172+0000 7faae4a00640 1 --2- 172.21.15.100:0/752210907 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faabc078d70 0x7faabc07b280 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7faad0032000 tx=0x7faad0001190 comp rx=0 tx=0).stop 2024-09-17T04:25:44.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.172+0000 7faae4a00640 1 -- 172.21.15.100:0/752210907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faae01f9890 msgr2=0x7faae013ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:44.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.172+0000 7faae4a00640 1 --2- 172.21.15.100:0/752210907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faae01f9890 0x7faae013ed10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7faac80066b0 tx=0x7faac80039c0 comp rx=0 tx=0).stop 2024-09-17T04:25:44.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.172+0000 7faae4a00640 1 -- 172.21.15.100:0/752210907 shutdown_connections 2024-09-17T04:25:44.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.172+0000 7faae4a00640 1 --2- 172.21.15.100:0/752210907 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faabc078d70 0x7faabc07b280 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:44.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.172+0000 7faae4a00640 1 --2- 172.21.15.100:0/752210907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faae01f8cf0 0x7faae01f9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:44.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.172+0000 7faae4a00640 1 --2- 172.21.15.100:0/752210907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faae01f9890 0x7faae013ed10 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:44.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.173+0000 7faae4a00640 1 --2- 172.21.15.100:0/752210907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faac4003a30 0x7faae01642f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:44.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.173+0000 7faae4a00640 1 -- 172.21.15.100:0/752210907 >> 172.21.15.100:0/752210907 conn(0x7faae006d1c0 msgr2=0x7faae00b00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:44.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.173+0000 7faae4a00640 1 -- 172.21.15.100:0/752210907 shutdown_connections 2024-09-17T04:25:44.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.173+0000 7faae4a00640 1 -- 172.21.15.100:0/752210907 wait complete. 2024-09-17T04:25:44.212 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T04:25:44.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.435+0000 7fc3a3200640 1 -- 172.21.15.100:0/361861661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3a40cd1d0 msgr2=0x7fc3a40af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:44.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.435+0000 7fc3a3200640 1 --2- 172.21.15.100:0/361861661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3a40cd1d0 0x7fc3a40af0d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4075230 tx=0x7fc39c02aac0 comp rx=0 tx=0).stop 2024-09-17T04:25:44.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.435+0000 7fc3a3200640 1 -- 172.21.15.100:0/361861661 shutdown_connections 2024-09-17T04:25:44.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.435+0000 7fc3a3200640 1 --2- 172.21.15.100:0/361861661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3a40cd1d0 0x7fc3a40af0d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:44.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.435+0000 7fc3a3200640 1 --2- 172.21.15.100:0/361861661 >> v2:172.21.15.184:3300/0 conn(0x7fc3a40b3f30 0x7fc3a4156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:44.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.435+0000 7fc3a3200640 1 -- 172.21.15.100:0/361861661 >> 172.21.15.100:0/361861661 conn(0x7fc3a406d620 msgr2=0x7fc3a40b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:44.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.435+0000 7fc3a3200640 1 -- 172.21.15.100:0/361861661 shutdown_connections 2024-09-17T04:25:44.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.435+0000 7fc3a3200640 1 -- 172.21.15.100:0/361861661 wait complete. 2024-09-17T04:25:44.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.437+0000 7fc3a3200640 1 Processor -- start 2024-09-17T04:25:44.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.437+0000 7fc3a3200640 1 -- start start 2024-09-17T04:25:44.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.439+0000 7fc3a3200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3a40b3f30 0x7fc3a415c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:44.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.439+0000 7fc3a3200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3a40cd1d0 0x7fc3a409c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:44.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.439+0000 7fc3a3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3a415d6e0 0x7fc3a41f9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:44.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.439+0000 7fc3a3200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a40af9a0 con 0x7fc3a415d6e0 2024-09-17T04:25:44.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.439+0000 7fc3a3200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a40af820 con 0x7fc3a40cd1d0 2024-09-17T04:25:44.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.439+0000 7fc3a3200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc3a40afb20 con 0x7fc3a40b3f30 2024-09-17T04:25:44.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.439+0000 7fc3a2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3a415d6e0 0x7fc3a41f9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:44.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.439+0000 7fc399400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3a40cd1d0 0x7fc3a409c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:44.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.439+0000 7fc3a1e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3a40b3f30 0x7fc3a415c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:44.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.439+0000 7fc3a2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3a415d6e0 0x7fc3a41f9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45504/0 (socket says 172.21.15.100:45504) 2024-09-17T04:25:44.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.439+0000 7fc3a2800640 1 -- 172.21.15.100:0/321020880 learned_addr learned my addr 172.21.15.100:0/321020880 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:44.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.440+0000 7fc3a2800640 1 -- 172.21.15.100:0/321020880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3a40b3f30 msgr2=0x7fc3a415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:44.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.440+0000 7fc3a2800640 1 --2- 172.21.15.100:0/321020880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3a40b3f30 0x7fc3a415c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:44.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.440+0000 7fc3a2800640 1 -- 172.21.15.100:0/321020880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3a40cd1d0 msgr2=0x7fc3a409c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:44.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.440+0000 7fc3a2800640 1 --2- 172.21.15.100:0/321020880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3a40cd1d0 0x7fc3a409c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:44.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.440+0000 7fc3a2800640 1 -- 172.21.15.100:0/321020880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a415c6d0 con 0x7fc3a415d6e0 2024-09-17T04:25:44.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.440+0000 7fc3a2800640 1 --2- 172.21.15.100:0/321020880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3a415d6e0 0x7fc3a41f9330 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009d60 tx=0x7fc39c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:44.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.440+0000 7fc39aa00640 1 -- 172.21.15.100:0/321020880 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c044070 con 0x7fc3a415d6e0 2024-09-17T04:25:44.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.440+0000 7fc39aa00640 1 -- 172.21.15.100:0/321020880 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc39c004250 con 0x7fc3a415d6e0 2024-09-17T04:25:44.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.440+0000 7fc3a3200640 1 -- 172.21.15.100:0/321020880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3a409a1c0 con 0x7fc3a415d6e0 2024-09-17T04:25:44.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.440+0000 7fc39aa00640 1 -- 172.21.15.100:0/321020880 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c0046a0 con 0x7fc3a415d6e0 2024-09-17T04:25:44.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.441+0000 7fc3a3200640 1 -- 172.21.15.100:0/321020880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3a41f6a80 con 0x7fc3a415d6e0 2024-09-17T04:25:44.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.441+0000 7fc3a3200640 1 -- 172.21.15.100:0/321020880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3a4156ca0 con 0x7fc3a415d6e0 2024-09-17T04:25:44.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.443+0000 7fc39aa00640 1 -- 172.21.15.100:0/321020880 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fc39c036400 con 0x7fc3a415d6e0 2024-09-17T04:25:44.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.444+0000 7fc39aa00640 1 --2- 172.21.15.100:0/321020880 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc380078d40 0x7fc38007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:44.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.444+0000 7fc39aa00640 1 -- 172.21.15.100:0/321020880 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5609+0+0 (secure 0 0 0) 0x7fc39c0bbc50 con 0x7fc3a415d6e0 2024-09-17T04:25:44.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.444+0000 7fc3a1e00640 1 --2- 172.21.15.100:0/321020880 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc380078d40 0x7fc38007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:44.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.445+0000 7fc3a1e00640 1 --2- 172.21.15.100:0/321020880 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc380078d40 0x7fc38007b250 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc394004590 tx=0x7fc39400b390 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:44.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.448+0000 7fc39aa00640 1 -- 172.21.15.100:0/321020880 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc39c032030 con 0x7fc3a415d6e0 2024-09-17T04:25:44.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:44.667+0000 7fc3a3200640 1 -- 172.21.15.100:0/321020880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc3a40ccda0 con 0x7fc3a415d6e0 2024-09-17T04:25:45.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.172+0000 7fc39aa00640 1 -- 172.21.15.100:0/321020880 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v22) ==== 172+0+0 (secure 0 0 0) 0x7fc39c0838c0 con 0x7fc3a415d6e0 2024-09-17T04:25:45.173 INFO:teuthology.orchestra.run.smithi100.stderr:new fs with metadata pool 6 and data pool 7 2024-09-17T04:25:45.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.176+0000 7fc3a3200640 1 -- 172.21.15.100:0/321020880 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc380078d40 msgr2=0x7fc38007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:45.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.176+0000 7fc3a3200640 1 --2- 172.21.15.100:0/321020880 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc380078d40 0x7fc38007b250 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc394004590 tx=0x7fc39400b390 comp rx=0 tx=0).stop 2024-09-17T04:25:45.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.176+0000 7fc3a3200640 1 -- 172.21.15.100:0/321020880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3a415d6e0 msgr2=0x7fc3a41f9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:45.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.176+0000 7fc3a3200640 1 --2- 172.21.15.100:0/321020880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3a415d6e0 0x7fc3a41f9330 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009d60 tx=0x7fc39c003ce0 comp rx=0 tx=0).stop 2024-09-17T04:25:45.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.176+0000 7fc3a3200640 1 -- 172.21.15.100:0/321020880 shutdown_connections 2024-09-17T04:25:45.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.177+0000 7fc3a3200640 1 --2- 172.21.15.100:0/321020880 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc380078d40 0x7fc38007b250 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:45.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.177+0000 7fc3a3200640 1 --2- 172.21.15.100:0/321020880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3a415d6e0 0x7fc3a41f9330 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:45.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.177+0000 7fc3a3200640 1 --2- 172.21.15.100:0/321020880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3a40cd1d0 0x7fc3a409c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:45.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.177+0000 7fc3a3200640 1 --2- 172.21.15.100:0/321020880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3a40b3f30 0x7fc3a415c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:45.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.177+0000 7fc3a3200640 1 -- 172.21.15.100:0/321020880 >> 172.21.15.100:0/321020880 conn(0x7fc3a406d620 msgr2=0x7fc3a415f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:45.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.178+0000 7fc3a3200640 1 -- 172.21.15.100:0/321020880 shutdown_connections 2024-09-17T04:25:45.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.178+0000 7fc3a3200640 1 -- 172.21.15.100:0/321020880 wait complete. 2024-09-17T04:25:45.232 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:25:45.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.454+0000 7f2c52600640 1 -- 172.21.15.100:0/871194114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c30004800 msgr2=0x7f2c30004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:45.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.454+0000 7f2c52600640 1 --2- 172.21.15.100:0/871194114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c30004800 0x7f2c30004c10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009b50 tx=0x7f2c30005830 comp rx=0 tx=0).stop 2024-09-17T04:25:45.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.455+0000 7f2c52600640 1 -- 172.21.15.100:0/871194114 shutdown_connections 2024-09-17T04:25:45.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.455+0000 7f2c52600640 1 --2- 172.21.15.100:0/871194114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c30004800 0x7f2c30004c10 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:45.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.455+0000 7f2c52600640 1 -- 172.21.15.100:0/871194114 >> 172.21.15.100:0/871194114 conn(0x7f2c4c0d0b50 msgr2=0x7f2c4c0d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:45.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.455+0000 7f2c52600640 1 -- 172.21.15.100:0/871194114 shutdown_connections 2024-09-17T04:25:45.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.455+0000 7f2c52600640 1 -- 172.21.15.100:0/871194114 wait complete. 2024-09-17T04:25:45.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.457+0000 7f2c52600640 1 Processor -- start 2024-09-17T04:25:45.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.457+0000 7f2c52600640 1 -- start start 2024-09-17T04:25:45.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.458+0000 7f2c52600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c30004800 0x7f2c4c15cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:45.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.458+0000 7f2c52600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c4c1f5b60 0x7f2c4c151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:45.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.458+0000 7f2c52600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c4c15ab70 0x7f2c4c15af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:45.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.458+0000 7f2c52600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4c15c230 con 0x7f2c4c15ab70 2024-09-17T04:25:45.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.458+0000 7f2c52600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4c15c0b0 con 0x7f2c30004800 2024-09-17T04:25:45.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.458+0000 7f2c52600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2c4c15c3b0 con 0x7f2c4c1f5b60 2024-09-17T04:25:45.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.459+0000 7f2c4be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c4c15ab70 0x7f2c4c15af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:45.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.459+0000 7f2c4be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c4c15ab70 0x7f2c4c15af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55716/0 (socket says 172.21.15.100:55716) 2024-09-17T04:25:45.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.459+0000 7f2c4be00640 1 -- 172.21.15.100:0/3585770651 learned_addr learned my addr 172.21.15.100:0/3585770651 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:45.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.459+0000 7f2c4b400640 1 --2- 172.21.15.100:0/3585770651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c30004800 0x7f2c4c15cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:45.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.459+0000 7f2c42a00640 1 --2- 172.21.15.100:0/3585770651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c4c1f5b60 0x7f2c4c151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:45.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.459+0000 7f2c4be00640 1 -- 172.21.15.100:0/3585770651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c4c1f5b60 msgr2=0x7f2c4c151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:45.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.459+0000 7f2c4be00640 1 --2- 172.21.15.100:0/3585770651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c4c1f5b60 0x7f2c4c151b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:45.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.459+0000 7f2c4be00640 1 -- 172.21.15.100:0/3585770651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c30004800 msgr2=0x7f2c4c15cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:45.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.459+0000 7f2c4be00640 1 --2- 172.21.15.100:0/3585770651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c30004800 0x7f2c4c15cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:45.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.459+0000 7f2c4be00640 1 -- 172.21.15.100:0/3585770651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4c1f6240 con 0x7f2c4c15ab70 2024-09-17T04:25:45.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.459+0000 7f2c4b400640 1 --2- 172.21.15.100:0/3585770651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c30004800 0x7f2c4c15cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:25:45.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.459+0000 7f2c42a00640 1 --2- 172.21.15.100:0/3585770651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c4c1f5b60 0x7f2c4c151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:25:45.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.459+0000 7f2c4be00640 1 --2- 172.21.15.100:0/3585770651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c4c15ab70 0x7f2c4c15af80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c072e10 tx=0x7f2c440019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:45.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.460+0000 7f2c43e00640 1 -- 172.21.15.100:0/3585770651 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c44011870 con 0x7f2c4c15ab70 2024-09-17T04:25:45.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.460+0000 7f2c43e00640 1 -- 172.21.15.100:0/3585770651 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c44017070 con 0x7f2c4c15ab70 2024-09-17T04:25:45.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.460+0000 7f2c43e00640 1 -- 172.21.15.100:0/3585770651 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c44011dc0 con 0x7f2c4c15ab70 2024-09-17T04:25:45.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.460+0000 7f2c52600640 1 -- 172.21.15.100:0/3585770651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4c1f6520 con 0x7f2c4c15ab70 2024-09-17T04:25:45.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.460+0000 7f2c52600640 1 -- 172.21.15.100:0/3585770651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c4c15d550 con 0x7f2c4c15ab70 2024-09-17T04:25:45.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.461+0000 7f2c52600640 1 -- 172.21.15.100:0/3585770651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c4c1f6a40 con 0x7f2c4c15ab70 2024-09-17T04:25:45.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.462+0000 7f2c43e00640 1 -- 172.21.15.100:0/3585770651 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f2c4401c460 con 0x7f2c4c15ab70 2024-09-17T04:25:45.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.463+0000 7f2c43e00640 1 --2- 172.21.15.100:0/3585770651 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2c28078d40 0x7f2c2807b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:45.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.463+0000 7f2c43e00640 1 -- 172.21.15.100:0/3585770651 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f2c4409ba40 con 0x7f2c4c15ab70 2024-09-17T04:25:45.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.464+0000 7f2c4b400640 1 --2- 172.21.15.100:0/3585770651 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2c28078d40 0x7f2c2807b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:45.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.464+0000 7f2c4b400640 1 --2- 172.21.15.100:0/3585770651 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2c28078d40 0x7f2c2807b250 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c001860 tx=0x7f2c3c000f10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:45.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.468+0000 7f2c43e00640 1 -- 172.21.15.100:0/3585770651 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f2c4c1f6a40 con 0x7f2c4c15ab70 2024-09-17T04:25:45.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.651+0000 7f2c52600640 1 -- 172.21.15.100:0/3585770651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c4c1f5fd0 con 0x7f2c4c15ab70 2024-09-17T04:25:45.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.652+0000 7f2c43e00640 1 -- 172.21.15.100:0/3585770651 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16734 (secure 0 0 0) 0x7f2c4406c020 con 0x7f2c4c15ab70 2024-09-17T04:25:45.653 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:45.653 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":47,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:25:44.669260+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:25:42.571024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-17T04:25:43.631985+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:25:45.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.655+0000 7f2c52600640 1 -- 172.21.15.100:0/3585770651 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2c28078d40 msgr2=0x7f2c2807b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:45.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.655+0000 7f2c52600640 1 --2- 172.21.15.100:0/3585770651 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2c28078d40 0x7f2c2807b250 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c001860 tx=0x7f2c3c000f10 comp rx=0 tx=0).stop 2024-09-17T04:25:45.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.655+0000 7f2c52600640 1 -- 172.21.15.100:0/3585770651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c4c15ab70 msgr2=0x7f2c4c15af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:45.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.655+0000 7f2c52600640 1 --2- 172.21.15.100:0/3585770651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c4c15ab70 0x7f2c4c15af80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c072e10 tx=0x7f2c440019b0 comp rx=0 tx=0).stop 2024-09-17T04:25:45.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.655+0000 7f2c52600640 1 -- 172.21.15.100:0/3585770651 shutdown_connections 2024-09-17T04:25:45.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.655+0000 7f2c52600640 1 --2- 172.21.15.100:0/3585770651 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2c28078d40 0x7f2c2807b250 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:45.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.655+0000 7f2c52600640 1 --2- 172.21.15.100:0/3585770651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c4c15ab70 0x7f2c4c15af80 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:45.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.655+0000 7f2c52600640 1 --2- 172.21.15.100:0/3585770651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c4c1f5b60 0x7f2c4c151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:45.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.655+0000 7f2c52600640 1 --2- 172.21.15.100:0/3585770651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c30004800 0x7f2c4c15cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:45.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.655+0000 7f2c52600640 1 -- 172.21.15.100:0/3585770651 >> 172.21.15.100:0/3585770651 conn(0x7f2c4c0d0b50 msgr2=0x7f2c4c14cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:45.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.655+0000 7f2c52600640 1 -- 172.21.15.100:0/3585770651 shutdown_connections 2024-09-17T04:25:45.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.655+0000 7f2c52600640 1 -- 172.21.15.100:0/3585770651 wait complete. 2024-09-17T04:25:45.680 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:25:45.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.899+0000 7f8c8ca00640 1 -- 172.21.15.100:0/1343677444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c880b3550 msgr2=0x7f8c880b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:45.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.899+0000 7f8c8ca00640 1 --2- 172.21.15.100:0/1343677444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c880b3550 0x7f8c880b3960 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8c80005fd0 tx=0x7f8c8002a820 comp rx=0 tx=0).stop 2024-09-17T04:25:45.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.899+0000 7f8c8ca00640 1 -- 172.21.15.100:0/1343677444 shutdown_connections 2024-09-17T04:25:45.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.899+0000 7f8c8ca00640 1 --2- 172.21.15.100:0/1343677444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c880b3550 0x7f8c880b3960 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:45.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.899+0000 7f8c8ca00640 1 --2- 172.21.15.100:0/1343677444 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c8815d760 0x7f8c881535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:45.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.899+0000 7f8c8ca00640 1 -- 172.21.15.100:0/1343677444 >> 172.21.15.100:0/1343677444 conn(0x7f8c8806d730 msgr2=0x7f8c880b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:45.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.899+0000 7f8c8ca00640 1 -- 172.21.15.100:0/1343677444 shutdown_connections 2024-09-17T04:25:45.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.899+0000 7f8c8ca00640 1 -- 172.21.15.100:0/1343677444 wait complete. 2024-09-17T04:25:45.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.901+0000 7f8c8ca00640 1 Processor -- start 2024-09-17T04:25:45.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.902+0000 7f8c8ca00640 1 -- start start 2024-09-17T04:25:45.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.903+0000 7f8c8ca00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c880ad0e0 0x7f8c8814ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:45.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.903+0000 7f8c8ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c880b3550 0x7f8c88143650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:45.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.903+0000 7f8c8ca00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c8815d760 0x7f8c881602a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:45.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.903+0000 7f8c8ca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8c880b4da0 con 0x7f8c880b3550 2024-09-17T04:25:45.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.903+0000 7f8c8ca00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8c880b4c20 con 0x7f8c880ad0e0 2024-09-17T04:25:45.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.903+0000 7f8c8ca00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8c880b3ea0 con 0x7f8c8815d760 2024-09-17T04:25:45.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.903+0000 7f8c85400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c880b3550 0x7f8c88143650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:45.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.903+0000 7f8c85400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c880b3550 0x7f8c88143650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55736/0 (socket says 172.21.15.100:55736) 2024-09-17T04:25:45.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.903+0000 7f8c85400640 1 -- 172.21.15.100:0/2575629226 learned_addr learned my addr 172.21.15.100:0/2575629226 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:45.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.904+0000 7f8c85e00640 1 --2- 172.21.15.100:0/2575629226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c880ad0e0 0x7f8c8814ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:45.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.904+0000 7f8c86800640 1 --2- 172.21.15.100:0/2575629226 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c8815d760 0x7f8c881602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:45.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.904+0000 7f8c85400640 1 -- 172.21.15.100:0/2575629226 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c8815d760 msgr2=0x7f8c881602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:45.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.904+0000 7f8c85400640 1 --2- 172.21.15.100:0/2575629226 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c8815d760 0x7f8c881602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:45.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.904+0000 7f8c85400640 1 -- 172.21.15.100:0/2575629226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c880ad0e0 msgr2=0x7f8c8814ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:45.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.904+0000 7f8c85400640 1 --2- 172.21.15.100:0/2575629226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c880ad0e0 0x7f8c8814ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:45.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.904+0000 7f8c85400640 1 -- 172.21.15.100:0/2575629226 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c881f9ad0 con 0x7f8c880b3550 2024-09-17T04:25:45.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.904+0000 7f8c85400640 1 --2- 172.21.15.100:0/2575629226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c880b3550 0x7f8c88143650 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400c370 tx=0x7f8c7400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:45.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.905+0000 7f8c7e000640 1 -- 172.21.15.100:0/2575629226 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74017070 con 0x7f8c880b3550 2024-09-17T04:25:45.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.905+0000 7f8c7e000640 1 -- 172.21.15.100:0/2575629226 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c740035f0 con 0x7f8c880b3550 2024-09-17T04:25:45.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.905+0000 7f8c7e000640 1 -- 172.21.15.100:0/2575629226 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74003a40 con 0x7f8c880b3550 2024-09-17T04:25:45.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.905+0000 7f8c8ca00640 1 -- 172.21.15.100:0/2575629226 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c881f8d10 con 0x7f8c880b3550 2024-09-17T04:25:45.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.905+0000 7f8c8ca00640 1 -- 172.21.15.100:0/2575629226 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c881f92d0 con 0x7f8c880b3550 2024-09-17T04:25:45.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.906+0000 7f8c8ca00640 1 -- 172.21.15.100:0/2575629226 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c880b4380 con 0x7f8c880b3550 2024-09-17T04:25:45.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.907+0000 7f8c7e000640 1 -- 172.21.15.100:0/2575629226 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f8c7401a070 con 0x7f8c880b3550 2024-09-17T04:25:45.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.908+0000 7f8c7e000640 1 --2- 172.21.15.100:0/2575629226 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8c68078d70 0x7f8c6807b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:45.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.908+0000 7f8c7e000640 1 -- 172.21.15.100:0/2575629226 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f8c7409a320 con 0x7f8c880b3550 2024-09-17T04:25:45.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.908+0000 7f8c85e00640 1 --2- 172.21.15.100:0/2575629226 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8c68078d70 0x7f8c6807b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:45.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.909+0000 7f8c85e00640 1 --2- 172.21.15.100:0/2575629226 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8c68078d70 0x7f8c6807b280 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009ac0 tx=0x7f8c78005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:45.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:45.913+0000 7f8c7e000640 1 -- 172.21.15.100:0/2575629226 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8c74061d70 con 0x7f8c880b3550 2024-09-17T04:25:46.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.096+0000 7f8c8ca00640 1 -- 172.21.15.100:0/2575629226 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8c88152a50 con 0x7f8c880b3550 2024-09-17T04:25:46.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.097+0000 7f8c7e000640 1 -- 172.21.15.100:0/2575629226 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16734 (secure 0 0 0) 0x7f8c740667a0 con 0x7f8c880b3550 2024-09-17T04:25:46.098 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:46.098 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":47,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:25:44.669260+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:25:42.571024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-17T04:25:43.631985+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:25:46.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.100+0000 7f8c8ca00640 1 -- 172.21.15.100:0/2575629226 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8c68078d70 msgr2=0x7f8c6807b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:46.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.100+0000 7f8c8ca00640 1 --2- 172.21.15.100:0/2575629226 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8c68078d70 0x7f8c6807b280 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009ac0 tx=0x7f8c78005f90 comp rx=0 tx=0).stop 2024-09-17T04:25:46.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.100+0000 7f8c8ca00640 1 -- 172.21.15.100:0/2575629226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c880b3550 msgr2=0x7f8c88143650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:46.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.100+0000 7f8c8ca00640 1 --2- 172.21.15.100:0/2575629226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c880b3550 0x7f8c88143650 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400c370 tx=0x7f8c7400c830 comp rx=0 tx=0).stop 2024-09-17T04:25:46.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.100+0000 7f8c8ca00640 1 -- 172.21.15.100:0/2575629226 shutdown_connections 2024-09-17T04:25:46.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.100+0000 7f8c8ca00640 1 --2- 172.21.15.100:0/2575629226 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8c68078d70 0x7f8c6807b280 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:46.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.100+0000 7f8c8ca00640 1 --2- 172.21.15.100:0/2575629226 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c8815d760 0x7f8c881602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:46.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.100+0000 7f8c8ca00640 1 --2- 172.21.15.100:0/2575629226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c880b3550 0x7f8c88143650 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:46.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.100+0000 7f8c8ca00640 1 --2- 172.21.15.100:0/2575629226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c880ad0e0 0x7f8c8814ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:46.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.100+0000 7f8c8ca00640 1 -- 172.21.15.100:0/2575629226 >> 172.21.15.100:0/2575629226 conn(0x7f8c8806d730 msgr2=0x7f8c8815e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:46.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.100+0000 7f8c8ca00640 1 -- 172.21.15.100:0/2575629226 shutdown_connections 2024-09-17T04:25:46.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.100+0000 7f8c8ca00640 1 -- 172.21.15.100:0/2575629226 wait complete. 2024-09-17T04:25:46.122 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T04:25:46.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.350+0000 7f25b9800640 1 -- 172.21.15.100:0/1156794271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2598003a30 msgr2=0x7f2598003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:46.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.350+0000 7f25b9800640 1 --2- 172.21.15.100:0/1156794271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2598003a30 0x7f2598003ea0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f25a4009b50 tx=0x7f2598005830 comp rx=0 tx=0).stop 2024-09-17T04:25:46.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.351+0000 7f25b3400640 1 -- 172.21.15.100:0/1156794271 reap_dead start 2024-09-17T04:25:46.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.351+0000 7f25b9800640 1 -- 172.21.15.100:0/1156794271 shutdown_connections 2024-09-17T04:25:46.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.351+0000 7f25b9800640 1 -- 172.21.15.100:0/1156794271 >> 172.21.15.100:0/1156794271 conn(0x7f25b406d1c0 msgr2=0x7f25b406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:46.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.351+0000 7f25b9800640 1 -- 172.21.15.100:0/1156794271 shutdown_connections 2024-09-17T04:25:46.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.351+0000 7f25b9800640 1 -- 172.21.15.100:0/1156794271 wait complete. 2024-09-17T04:25:46.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.353+0000 7f25b9800640 1 Processor -- start 2024-09-17T04:25:46.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.353+0000 7f25b9800640 1 -- start start 2024-09-17T04:25:46.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.355+0000 7f25b9800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2598003a30 0x7f25b415bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:46.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.355+0000 7f25b9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25b41f8730 0x7f25b41f8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:46.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.355+0000 7f25b9800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25b40b0690 0x7f25b40b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:46.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.355+0000 7f25b9800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f25b40cdea0 con 0x7f25b41f8730 2024-09-17T04:25:46.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.355+0000 7f25b9800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f25b40cdd20 con 0x7f2598003a30 2024-09-17T04:25:46.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.355+0000 7f25b9800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f25b40ce020 con 0x7f25b40b0690 2024-09-17T04:25:46.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.355+0000 7f25b2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25b41f8730 0x7f25b41f8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:46.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.355+0000 7f25b2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25b41f8730 0x7f25b41f8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55744/0 (socket says 172.21.15.100:55744) 2024-09-17T04:25:46.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.355+0000 7f25b2000640 1 -- 172.21.15.100:0/684385110 learned_addr learned my addr 172.21.15.100:0/684385110 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:46.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.355+0000 7f25b2a00640 1 --2- 172.21.15.100:0/684385110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2598003a30 0x7f25b415bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:46.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.355+0000 7f25b3400640 1 --2- 172.21.15.100:0/684385110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25b40b0690 0x7f25b40b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:46.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.355+0000 7f25b2000640 1 -- 172.21.15.100:0/684385110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25b40b0690 msgr2=0x7f25b40b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:46.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.355+0000 7f25b2000640 1 --2- 172.21.15.100:0/684385110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25b40b0690 0x7f25b40b0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:46.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.355+0000 7f25b2000640 1 -- 172.21.15.100:0/684385110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2598003a30 msgr2=0x7f25b415bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:46.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.355+0000 7f25b2000640 1 --2- 172.21.15.100:0/684385110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2598003a30 0x7f25b415bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:46.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.355+0000 7f25b2000640 1 -- 172.21.15.100:0/684385110 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b40b0fe0 con 0x7f25b41f8730 2024-09-17T04:25:46.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.356+0000 7f25b2000640 1 --2- 172.21.15.100:0/684385110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25b41f8730 0x7f25b41f8b40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f25a00064b0 tx=0x7f25a00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:46.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.356+0000 7f25aaa00640 1 -- 172.21.15.100:0/684385110 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a0015070 con 0x7f25b41f8730 2024-09-17T04:25:46.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.356+0000 7f25aaa00640 1 -- 172.21.15.100:0/684385110 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25a0011900 con 0x7f25b41f8730 2024-09-17T04:25:46.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.356+0000 7f25aaa00640 1 -- 172.21.15.100:0/684385110 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a0011d30 con 0x7f25b41f8730 2024-09-17T04:25:46.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.356+0000 7f25b9800640 1 -- 172.21.15.100:0/684385110 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b41f5f80 con 0x7f25b41f8730 2024-09-17T04:25:46.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.356+0000 7f25b9800640 1 -- 172.21.15.100:0/684385110 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b4142de0 con 0x7f25b41f8730 2024-09-17T04:25:46.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.357+0000 7f25b9800640 1 -- 172.21.15.100:0/684385110 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25b40ce580 con 0x7f25b41f8730 2024-09-17T04:25:46.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.358+0000 7f25aaa00640 1 -- 172.21.15.100:0/684385110 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f25a0018070 con 0x7f25b41f8730 2024-09-17T04:25:46.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.359+0000 7f25aaa00640 1 --2- 172.21.15.100:0/684385110 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2594078d70 0x7f259407b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:46.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.360+0000 7f25aaa00640 1 -- 172.21.15.100:0/684385110 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f25a009b9c0 con 0x7f25b41f8730 2024-09-17T04:25:46.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.360+0000 7f25b2a00640 1 --2- 172.21.15.100:0/684385110 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2594078d70 0x7f259407b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:46.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.361+0000 7f25b2a00640 1 --2- 172.21.15.100:0/684385110 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2594078d70 0x7f259407b280 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f25a4035000 tx=0x7f25a402d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:46.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.364+0000 7f25aaa00640 1 -- 172.21.15.100:0/684385110 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f25b40b0fe0 con 0x7f25b41f8730 2024-09-17T04:25:46.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:46.581+0000 7f25b9800640 1 -- 172.21.15.100:0/684385110 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f25b406f220 con 0x7f25b41f8730 2024-09-17T04:25:46.782 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-09-17T04:25:46.782 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-09-17T04:25:47.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.208+0000 7f25aaa00640 1 -- 172.21.15.100:0/684385110 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f25a0063440 con 0x7f25b41f8730 2024-09-17T04:25:47.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.212+0000 7f25b9800640 1 -- 172.21.15.100:0/684385110 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2594078d70 msgr2=0x7f259407b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:47.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.212+0000 7f25b9800640 1 --2- 172.21.15.100:0/684385110 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2594078d70 0x7f259407b280 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f25a4035000 tx=0x7f25a402d040 comp rx=0 tx=0).stop 2024-09-17T04:25:47.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.212+0000 7f25b9800640 1 -- 172.21.15.100:0/684385110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25b41f8730 msgr2=0x7f25b41f8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:47.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.212+0000 7f25b9800640 1 --2- 172.21.15.100:0/684385110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25b41f8730 0x7f25b41f8b40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f25a00064b0 tx=0x7f25a00039c0 comp rx=0 tx=0).stop 2024-09-17T04:25:47.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.212+0000 7f25b9800640 1 -- 172.21.15.100:0/684385110 shutdown_connections 2024-09-17T04:25:47.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.212+0000 7f25b9800640 1 --2- 172.21.15.100:0/684385110 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2594078d70 0x7f259407b280 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:47.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.212+0000 7f25b9800640 1 --2- 172.21.15.100:0/684385110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25b40b0690 0x7f25b40b0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:47.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.212+0000 7f25b9800640 1 --2- 172.21.15.100:0/684385110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25b41f8730 0x7f25b41f8b40 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:47.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.212+0000 7f25b9800640 1 --2- 172.21.15.100:0/684385110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2598003a30 0x7f25b415bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:47.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.212+0000 7f25b9800640 1 -- 172.21.15.100:0/684385110 >> 172.21.15.100:0/684385110 conn(0x7f25b406d1c0 msgr2=0x7f25b40b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:47.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.212+0000 7f25b9800640 1 -- 172.21.15.100:0/684385110 shutdown_connections 2024-09-17T04:25:47.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.212+0000 7f25b9800640 1 -- 172.21.15.100:0/684385110 wait complete. 2024-09-17T04:25:47.234 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T04:25:47.235 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T04:25:47.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.459+0000 7fb227800640 1 -- 172.21.15.100:0/201409703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb204004800 msgr2=0x7fb204004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:47.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.459+0000 7fb227800640 1 --2- 172.21.15.100:0/201409703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb204004800 0x7fb204004c10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb208001690 tx=0x7fb204005830 comp rx=0 tx=0).stop 2024-09-17T04:25:47.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.460+0000 7fb227800640 1 -- 172.21.15.100:0/201409703 shutdown_connections 2024-09-17T04:25:47.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.460+0000 7fb227800640 1 --2- 172.21.15.100:0/201409703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb204004800 0x7fb204004c10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:47.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.460+0000 7fb227800640 1 -- 172.21.15.100:0/201409703 >> 172.21.15.100:0/201409703 conn(0x7fb22006d1c0 msgr2=0x7fb22006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:47.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.460+0000 7fb227800640 1 -- 172.21.15.100:0/201409703 shutdown_connections 2024-09-17T04:25:47.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.460+0000 7fb227800640 1 -- 172.21.15.100:0/201409703 wait complete. 2024-09-17T04:25:47.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.462+0000 7fb227800640 1 Processor -- start 2024-09-17T04:25:47.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.462+0000 7fb227800640 1 -- start start 2024-09-17T04:25:47.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.463+0000 7fb227800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb204004800 0x7fb22013e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:47.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.463+0000 7fb227800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb220140ff0 0x7fb22013f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:47.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.463+0000 7fb227800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb220151f20 0x7fb2201f8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:47.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.463+0000 7fb227800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb220073e90 con 0x7fb220151f20 2024-09-17T04:25:47.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.463+0000 7fb227800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb220073d10 con 0x7fb204004800 2024-09-17T04:25:47.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.463+0000 7fb227800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb22006f180 con 0x7fb220140ff0 2024-09-17T04:25:47.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.464+0000 7fb225800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb220151f20 0x7fb2201f8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:47.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.464+0000 7fb225800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb220151f20 0x7fb2201f8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55766/0 (socket says 172.21.15.100:55766) 2024-09-17T04:25:47.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.464+0000 7fb225800640 1 -- 172.21.15.100:0/4187692855 learned_addr learned my addr 172.21.15.100:0/4187692855 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:47.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.464+0000 7fb224e00640 1 --2- 172.21.15.100:0/4187692855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb204004800 0x7fb22013e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:47.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.464+0000 7fb217e00640 1 --2- 172.21.15.100:0/4187692855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb220140ff0 0x7fb22013f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:47.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.464+0000 7fb225800640 1 -- 172.21.15.100:0/4187692855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb220140ff0 msgr2=0x7fb22013f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:47.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.464+0000 7fb225800640 1 --2- 172.21.15.100:0/4187692855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb220140ff0 0x7fb22013f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:47.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.464+0000 7fb225800640 1 -- 172.21.15.100:0/4187692855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb204004800 msgr2=0x7fb22013e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:47.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.464+0000 7fb225800640 1 --2- 172.21.15.100:0/4187692855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb204004800 0x7fb22013e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:47.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.464+0000 7fb225800640 1 -- 172.21.15.100:0/4187692855 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2200c5520 con 0x7fb220151f20 2024-09-17T04:25:47.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.464+0000 7fb225800640 1 --2- 172.21.15.100:0/4187692855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb220151f20 0x7fb2201f8d20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb220160280 tx=0x7fb21800ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:47.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.465+0000 7fb21d600640 1 -- 172.21.15.100:0/4187692855 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb218010890 con 0x7fb220151f20 2024-09-17T04:25:47.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.465+0000 7fb21d600640 1 -- 172.21.15.100:0/4187692855 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb218018070 con 0x7fb220151f20 2024-09-17T04:25:47.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.465+0000 7fb21d600640 1 -- 172.21.15.100:0/4187692855 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb218010dc0 con 0x7fb220151f20 2024-09-17T04:25:47.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.465+0000 7fb227800640 1 -- 172.21.15.100:0/4187692855 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2200c5800 con 0x7fb220151f20 2024-09-17T04:25:47.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.465+0000 7fb227800640 1 -- 172.21.15.100:0/4187692855 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb22013f240 con 0x7fb220151f20 2024-09-17T04:25:47.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.466+0000 7fb227800640 1 -- 172.21.15.100:0/4187692855 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb22013ed50 con 0x7fb220151f20 2024-09-17T04:25:47.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.467+0000 7fb21d600640 1 -- 172.21.15.100:0/4187692855 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fb21801d460 con 0x7fb220151f20 2024-09-17T04:25:47.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.468+0000 7fb21d600640 1 --2- 172.21.15.100:0/4187692855 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb1fc078d40 0x7fb1fc07b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:47.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.468+0000 7fb21d600640 1 -- 172.21.15.100:0/4187692855 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fb21809b4b0 con 0x7fb220151f20 2024-09-17T04:25:47.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.468+0000 7fb224e00640 1 --2- 172.21.15.100:0/4187692855 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb1fc078d40 0x7fb1fc07b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:47.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.469+0000 7fb224e00640 1 --2- 172.21.15.100:0/4187692855 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb1fc078d40 0x7fb1fc07b250 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb210009cd0 tx=0x7fb210005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:47.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.473+0000 7fb21d600640 1 -- 172.21.15.100:0/4187692855 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb2180634c0 con 0x7fb220151f20 2024-09-17T04:25:47.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:47.691+0000 7fb227800640 1 -- 172.21.15.100:0/4187692855 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb22006f6e0 con 0x7fb220151f20 2024-09-17T04:25:48.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.215+0000 7fb21d600640 1 -- 172.21.15.100:0/4187692855 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7fb218062dc0 con 0x7fb220151f20 2024-09-17T04:25:48.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.218+0000 7fb227800640 1 -- 172.21.15.100:0/4187692855 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb1fc078d40 msgr2=0x7fb1fc07b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:48.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.218+0000 7fb227800640 1 --2- 172.21.15.100:0/4187692855 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb1fc078d40 0x7fb1fc07b250 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb210009cd0 tx=0x7fb210005f90 comp rx=0 tx=0).stop 2024-09-17T04:25:48.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.218+0000 7fb227800640 1 -- 172.21.15.100:0/4187692855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb220151f20 msgr2=0x7fb2201f8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:48.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.218+0000 7fb227800640 1 --2- 172.21.15.100:0/4187692855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb220151f20 0x7fb2201f8d20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb220160280 tx=0x7fb21800ea20 comp rx=0 tx=0).stop 2024-09-17T04:25:48.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.218+0000 7fb227800640 1 -- 172.21.15.100:0/4187692855 shutdown_connections 2024-09-17T04:25:48.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.218+0000 7fb227800640 1 --2- 172.21.15.100:0/4187692855 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb1fc078d40 0x7fb1fc07b250 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:48.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.218+0000 7fb227800640 1 --2- 172.21.15.100:0/4187692855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb220151f20 0x7fb2201f8d20 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:48.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.218+0000 7fb227800640 1 --2- 172.21.15.100:0/4187692855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb220140ff0 0x7fb22013f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:48.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.218+0000 7fb227800640 1 --2- 172.21.15.100:0/4187692855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb204004800 0x7fb22013e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:48.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.218+0000 7fb227800640 1 -- 172.21.15.100:0/4187692855 >> 172.21.15.100:0/4187692855 conn(0x7fb22006d1c0 msgr2=0x7fb2200ad9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:48.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.218+0000 7fb227800640 1 -- 172.21.15.100:0/4187692855 shutdown_connections 2024-09-17T04:25:48.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.218+0000 7fb227800640 1 -- 172.21.15.100:0/4187692855 wait complete. 2024-09-17T04:25:48.240 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T04:25:48.241 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:25:48.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.462+0000 7fa1b0600640 1 -- 172.21.15.100:0/141585946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa190003a00 msgr2=0x7fa190003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:48.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.462+0000 7fa1b0600640 1 --2- 172.21.15.100:0/141585946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa190003a00 0x7fa190003e70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009b50 tx=0x7fa190005830 comp rx=0 tx=0).stop 2024-09-17T04:25:48.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.462+0000 7fa1ae600640 1 -- 172.21.15.100:0/141585946 reap_dead start 2024-09-17T04:25:48.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.462+0000 7fa1b0600640 1 -- 172.21.15.100:0/141585946 shutdown_connections 2024-09-17T04:25:48.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.462+0000 7fa1b0600640 1 -- 172.21.15.100:0/141585946 >> 172.21.15.100:0/141585946 conn(0x7fa1a806d730 msgr2=0x7fa1a80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:48.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.462+0000 7fa1b0600640 1 -- 172.21.15.100:0/141585946 shutdown_connections 2024-09-17T04:25:48.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.462+0000 7fa1b0600640 1 -- 172.21.15.100:0/141585946 wait complete. 2024-09-17T04:25:48.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.465+0000 7fa1b0600640 1 Processor -- start 2024-09-17T04:25:48.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.465+0000 7fa1b0600640 1 -- start start 2024-09-17T04:25:48.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.466+0000 7fa1b0600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa190003a00 0x7fa1a813f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:48.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.466+0000 7fa1b0600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1a81f6b70 0x7fa1a81f6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:48.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.466+0000 7fa1b0600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1a81f74c0 0x7fa1a8155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:48.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.466+0000 7fa1b0600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a80b4da0 con 0x7fa1a81f6b70 2024-09-17T04:25:48.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.466+0000 7fa1b0600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a80b4c20 con 0x7fa190003a00 2024-09-17T04:25:48.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.466+0000 7fa1b0600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a80b3960 con 0x7fa1a81f74c0 2024-09-17T04:25:48.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.467+0000 7fa1ad200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1a81f6b70 0x7fa1a81f6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:48.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.467+0000 7fa1ad200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1a81f6b70 0x7fa1a81f6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55772/0 (socket says 172.21.15.100:55772) 2024-09-17T04:25:48.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.467+0000 7fa1ad200640 1 -- 172.21.15.100:0/1695578697 learned_addr learned my addr 172.21.15.100:0/1695578697 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:48.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.467+0000 7fa1adc00640 1 --2- 172.21.15.100:0/1695578697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa190003a00 0x7fa1a813f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:48.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.467+0000 7fa1ae600640 1 --2- 172.21.15.100:0/1695578697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1a81f74c0 0x7fa1a8155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:48.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.467+0000 7fa1ad200640 1 -- 172.21.15.100:0/1695578697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1a81f74c0 msgr2=0x7fa1a8155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:48.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.467+0000 7fa1ad200640 1 --2- 172.21.15.100:0/1695578697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1a81f74c0 0x7fa1a8155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:48.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.467+0000 7fa1ad200640 1 -- 172.21.15.100:0/1695578697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa190003a00 msgr2=0x7fa1a813f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:48.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.467+0000 7fa1ad200640 1 --2- 172.21.15.100:0/1695578697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa190003a00 0x7fa1a813f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:48.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.467+0000 7fa1ad200640 1 -- 172.21.15.100:0/1695578697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a81490f0 con 0x7fa1a81f6b70 2024-09-17T04:25:48.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.467+0000 7fa1adc00640 1 --2- 172.21.15.100:0/1695578697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa190003a00 0x7fa1a813f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:48.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.467+0000 7fa1ae600640 1 --2- 172.21.15.100:0/1695578697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1a81f74c0 0x7fa1a8155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:48.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.467+0000 7fa1ad200640 1 --2- 172.21.15.100:0/1695578697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1a81f6b70 0x7fa1a81f6f80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa194000fb0 tx=0x7fa19400ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:48.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.468+0000 7fa1a5600640 1 -- 172.21.15.100:0/1695578697 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194015070 con 0x7fa1a81f6b70 2024-09-17T04:25:48.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.468+0000 7fa1a5600640 1 -- 172.21.15.100:0/1695578697 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa19400ca00 con 0x7fa1a81f6b70 2024-09-17T04:25:48.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.468+0000 7fa1a5600640 1 -- 172.21.15.100:0/1695578697 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19400ce30 con 0x7fa1a81f6b70 2024-09-17T04:25:48.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.468+0000 7fa1b0600640 1 -- 172.21.15.100:0/1695578697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a814f010 con 0x7fa1a81f6b70 2024-09-17T04:25:48.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.468+0000 7fa1b0600640 1 -- 172.21.15.100:0/1695578697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a814f5d0 con 0x7fa1a81f6b70 2024-09-17T04:25:48.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.469+0000 7fa1b0600640 1 -- 172.21.15.100:0/1695578697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a81535c0 con 0x7fa1a81f6b70 2024-09-17T04:25:48.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.470+0000 7fa1a5600640 1 -- 172.21.15.100:0/1695578697 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fa194018070 con 0x7fa1a81f6b70 2024-09-17T04:25:48.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.471+0000 7fa1a5600640 1 --2- 172.21.15.100:0/1695578697 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa188078d40 0x7fa18807b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:48.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.471+0000 7fa1a5600640 1 -- 172.21.15.100:0/1695578697 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fa19409b730 con 0x7fa1a81f6b70 2024-09-17T04:25:48.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.471+0000 7fa1adc00640 1 --2- 172.21.15.100:0/1695578697 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa188078d40 0x7fa18807b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:48.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.472+0000 7fa1adc00640 1 --2- 172.21.15.100:0/1695578697 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa188078d40 0x7fa18807b250 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa19c035000 tx=0x7fa19c02d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:48.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.476+0000 7fa1a5600640 1 -- 172.21.15.100:0/1695578697 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa1a81490f0 con 0x7fa1a81f6b70 2024-09-17T04:25:48.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.696+0000 7fa1b0600640 1 -- 172.21.15.100:0/1695578697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1a80d0940 con 0x7fa1a81f6b70 2024-09-17T04:25:48.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.697+0000 7fa1a5600640 1 -- 172.21.15.100:0/1695578697 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5232 (secure 0 0 0) 0x7fa194063740 con 0x7fa1a81f6b70 2024-09-17T04:25:48.697 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:48.698 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"btime":"2024-09-17T04:25:48:210316+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1086873435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1086873435},{"type":"v1","addr":"172.21.15.100:6833","nonce":1086873435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1708229764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1708229764},{"type":"v1","addr":"172.21.15.184:6836","nonce":1708229764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/828832783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":828832783},{"type":"v1","addr":"172.21.15.184:6837","nonce":828832783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:44.669203+0000","modified":"2024-09-17T04:25:47.693203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3801048032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3801048032},{"type":"v1","addr":"172.21.15.100:6835","nonce":3801048032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5295,"qdb_cluster":[5295]},"id":3}]} 2024-09-17T04:25:48.698 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 26 2024-09-17T04:25:48.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.699+0000 7fa1b0600640 1 -- 172.21.15.100:0/1695578697 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa188078d40 msgr2=0x7fa18807b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:48.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.699+0000 7fa1b0600640 1 --2- 172.21.15.100:0/1695578697 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa188078d40 0x7fa18807b250 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa19c035000 tx=0x7fa19c02d040 comp rx=0 tx=0).stop 2024-09-17T04:25:48.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.699+0000 7fa1b0600640 1 -- 172.21.15.100:0/1695578697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1a81f6b70 msgr2=0x7fa1a81f6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:48.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.699+0000 7fa1b0600640 1 --2- 172.21.15.100:0/1695578697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1a81f6b70 0x7fa1a81f6f80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa194000fb0 tx=0x7fa19400ea90 comp rx=0 tx=0).stop 2024-09-17T04:25:48.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.699+0000 7fa1b0600640 1 -- 172.21.15.100:0/1695578697 shutdown_connections 2024-09-17T04:25:48.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.699+0000 7fa1b0600640 1 --2- 172.21.15.100:0/1695578697 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa188078d40 0x7fa18807b250 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:48.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.699+0000 7fa1b0600640 1 --2- 172.21.15.100:0/1695578697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1a81f74c0 0x7fa1a8155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:48.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.699+0000 7fa1b0600640 1 --2- 172.21.15.100:0/1695578697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1a81f6b70 0x7fa1a81f6f80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:48.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.699+0000 7fa1b0600640 1 --2- 172.21.15.100:0/1695578697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa190003a00 0x7fa1a813f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:48.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.699+0000 7fa1b0600640 1 -- 172.21.15.100:0/1695578697 >> 172.21.15.100:0/1695578697 conn(0x7fa1a806d730 msgr2=0x7fa1a815eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:48.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.700+0000 7fa1b0600640 1 -- 172.21.15.100:0/1695578697 shutdown_connections 2024-09-17T04:25:48.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.700+0000 7fa1b0600640 1 -- 172.21.15.100:0/1695578697 wait complete. 2024-09-17T04:25:48.721 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:25:48.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.941+0000 7f1596400640 1 -- 172.21.15.100:0/3955068835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f159015d760 msgr2=0x7f15901535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:48.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.941+0000 7f1596400640 1 --2- 172.21.15.100:0/3955068835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f159015d760 0x7f15901535c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f15900b3100 tx=0x7f158002a4d0 comp rx=0 tx=0).stop 2024-09-17T04:25:48.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.941+0000 7f1596400640 1 -- 172.21.15.100:0/3955068835 shutdown_connections 2024-09-17T04:25:48.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.941+0000 7f1596400640 1 --2- 172.21.15.100:0/3955068835 >> v2:172.21.15.184:3300/0 conn(0x7f15900b3550 0x7f15900b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:48.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.942+0000 7f1596400640 1 --2- 172.21.15.100:0/3955068835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f159015d760 0x7f15901535c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:48.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.942+0000 7f1596400640 1 -- 172.21.15.100:0/3955068835 >> 172.21.15.100:0/3955068835 conn(0x7f159006d730 msgr2=0x7f15900b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:48.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.942+0000 7f1596400640 1 -- 172.21.15.100:0/3955068835 shutdown_connections 2024-09-17T04:25:48.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.942+0000 7f1596400640 1 -- 172.21.15.100:0/3955068835 wait complete. 2024-09-17T04:25:48.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.944+0000 7f1596400640 1 Processor -- start 2024-09-17T04:25:48.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.944+0000 7f1596400640 1 -- start start 2024-09-17T04:25:48.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.945+0000 7f1596400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15900ad0e0 0x7f1590146980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:48.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.945+0000 7f1596400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15900b3550 0x7f159009ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:48.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.945+0000 7f1596400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f159015d760 0x7f159009b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:48.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.945+0000 7f1596400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f15900b4da0 con 0x7f15900ad0e0 2024-09-17T04:25:48.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.945+0000 7f1596400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f15900b4c20 con 0x7f15900b3550 2024-09-17T04:25:48.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.945+0000 7f1596400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f15900b3ea0 con 0x7f159015d760 2024-09-17T04:25:48.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.946+0000 7f158f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15900ad0e0 0x7f1590146980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:48.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.946+0000 7f158f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15900ad0e0 0x7f1590146980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55786/0 (socket says 172.21.15.100:55786) 2024-09-17T04:25:48.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.946+0000 7f158f400640 1 -- 172.21.15.100:0/2853720444 learned_addr learned my addr 172.21.15.100:0/2853720444 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:48.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.946+0000 7f158fe00640 1 --2- 172.21.15.100:0/2853720444 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f159015d760 0x7f159009b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:48.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.946+0000 7f158ea00640 1 --2- 172.21.15.100:0/2853720444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15900b3550 0x7f159009ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:48.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.946+0000 7f158f400640 1 -- 172.21.15.100:0/2853720444 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f159015d760 msgr2=0x7f159009b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:48.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.946+0000 7f158f400640 1 --2- 172.21.15.100:0/2853720444 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f159015d760 0x7f159009b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:48.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.946+0000 7f158f400640 1 -- 172.21.15.100:0/2853720444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15900b3550 msgr2=0x7f159009ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:48.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.946+0000 7f158f400640 1 --2- 172.21.15.100:0/2853720444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15900b3550 0x7f159009ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:48.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.946+0000 7f158f400640 1 -- 172.21.15.100:0/2853720444 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f159009dd40 con 0x7f15900ad0e0 2024-09-17T04:25:48.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.946+0000 7f158f400640 1 --2- 172.21.15.100:0/2853720444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15900ad0e0 0x7f1590146980 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1580034040 tx=0x7f1580003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:48.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.947+0000 7f1587400640 1 -- 172.21.15.100:0/2853720444 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158003e030 con 0x7f15900ad0e0 2024-09-17T04:25:48.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.947+0000 7f1587400640 1 -- 172.21.15.100:0/2853720444 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1580003e00 con 0x7f15900ad0e0 2024-09-17T04:25:48.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.947+0000 7f1587400640 1 -- 172.21.15.100:0/2853720444 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1580035040 con 0x7f15900ad0e0 2024-09-17T04:25:48.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.947+0000 7f1596400640 1 -- 172.21.15.100:0/2853720444 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1590156590 con 0x7f15900ad0e0 2024-09-17T04:25:48.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.947+0000 7f1596400640 1 -- 172.21.15.100:0/2853720444 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f159009a520 con 0x7f15900ad0e0 2024-09-17T04:25:48.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.948+0000 7f1596400640 1 -- 172.21.15.100:0/2853720444 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15900b43c0 con 0x7f15900ad0e0 2024-09-17T04:25:48.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.949+0000 7f1587400640 1 -- 172.21.15.100:0/2853720444 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f1580041030 con 0x7f15900ad0e0 2024-09-17T04:25:48.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.950+0000 7f1587400640 1 --2- 172.21.15.100:0/2853720444 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1570078d40 0x7f157007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:48.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.950+0000 7f1587400640 1 -- 172.21.15.100:0/2853720444 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f15800bb7f0 con 0x7f15900ad0e0 2024-09-17T04:25:48.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.950+0000 7f158ea00640 1 --2- 172.21.15.100:0/2853720444 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1570078d40 0x7f157007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:48.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.951+0000 7f158ea00640 1 --2- 172.21.15.100:0/2853720444 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1570078d40 0x7f157007b250 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1590070130 tx=0x7f157c009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:48.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:48.955+0000 7f1587400640 1 -- 172.21.15.100:0/2853720444 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f1580083380 con 0x7f15900ad0e0 2024-09-17T04:25:49.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.139+0000 7f1596400640 1 -- 172.21.15.100:0/2853720444 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f159015c670 con 0x7f15900ad0e0 2024-09-17T04:25:49.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.140+0000 7f1587400640 1 -- 172.21.15.100:0/2853720444 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16721 (secure 0 0 0) 0x7f1580087db0 con 0x7f15900ad0e0 2024-09-17T04:25:49.141 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:49.141 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":48,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:25:46.188431+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:25:42.571024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-17T04:25:43.631985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:25:49.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.143+0000 7f1596400640 1 -- 172.21.15.100:0/2853720444 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1570078d40 msgr2=0x7f157007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:49.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.143+0000 7f1596400640 1 --2- 172.21.15.100:0/2853720444 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1570078d40 0x7f157007b250 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1590070130 tx=0x7f157c009060 comp rx=0 tx=0).stop 2024-09-17T04:25:49.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.143+0000 7f1596400640 1 -- 172.21.15.100:0/2853720444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15900ad0e0 msgr2=0x7f1590146980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:49.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.143+0000 7f1596400640 1 --2- 172.21.15.100:0/2853720444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15900ad0e0 0x7f1590146980 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1580034040 tx=0x7f1580003910 comp rx=0 tx=0).stop 2024-09-17T04:25:49.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.143+0000 7f1596400640 1 -- 172.21.15.100:0/2853720444 shutdown_connections 2024-09-17T04:25:49.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.143+0000 7f1596400640 1 --2- 172.21.15.100:0/2853720444 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f1570078d40 0x7f157007b250 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:49.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.143+0000 7f1596400640 1 --2- 172.21.15.100:0/2853720444 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f159015d760 0x7f159009b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:49.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.143+0000 7f1596400640 1 --2- 172.21.15.100:0/2853720444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15900b3550 0x7f159009ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:49.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.143+0000 7f1596400640 1 --2- 172.21.15.100:0/2853720444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15900ad0e0 0x7f1590146980 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:49.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.143+0000 7f1596400640 1 -- 172.21.15.100:0/2853720444 >> 172.21.15.100:0/2853720444 conn(0x7f159006d730 msgr2=0x7f159015b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:49.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.143+0000 7f1596400640 1 -- 172.21.15.100:0/2853720444 shutdown_connections 2024-09-17T04:25:49.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.143+0000 7f1596400640 1 -- 172.21.15.100:0/2853720444 wait complete. 2024-09-17T04:25:49.168 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T04:25:49.168 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:25:49.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.387+0000 7f77c4a00640 1 -- 172.21.15.100:0/938726078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77c015d760 msgr2=0x7f77c01535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:49.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.387+0000 7f77c4a00640 1 --2- 172.21.15.100:0/938726078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77c015d760 0x7f77c01535c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f77b0009830 tx=0x7f77b002a480 comp rx=0 tx=0).stop 2024-09-17T04:25:49.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.387+0000 7f77c4a00640 1 -- 172.21.15.100:0/938726078 shutdown_connections 2024-09-17T04:25:49.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.387+0000 7f77c4a00640 1 --2- 172.21.15.100:0/938726078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77c015d760 0x7f77c01535c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:49.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.387+0000 7f77c4a00640 1 -- 172.21.15.100:0/938726078 >> 172.21.15.100:0/938726078 conn(0x7f77c006d730 msgr2=0x7f77c00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:49.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.387+0000 7f77c4a00640 1 -- 172.21.15.100:0/938726078 shutdown_connections 2024-09-17T04:25:49.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.387+0000 7f77c4a00640 1 -- 172.21.15.100:0/938726078 wait complete. 2024-09-17T04:25:49.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.389+0000 7f77c4a00640 1 Processor -- start 2024-09-17T04:25:49.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.389+0000 7f77c4a00640 1 -- start start 2024-09-17T04:25:49.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.391+0000 7f77c4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77c00ad0e0 0x7f77c009de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:49.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.391+0000 7f77c4a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77c00b3550 0x7f77c014a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:49.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.391+0000 7f77c4a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77c015d760 0x7f77c01f6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:49.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.391+0000 7f77c4a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f77c006f240 con 0x7f77c00ad0e0 2024-09-17T04:25:49.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.391+0000 7f77c4a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f77c00b4e30 con 0x7f77c015d760 2024-09-17T04:25:49.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.391+0000 7f77c4a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f77c00b4070 con 0x7f77c00b3550 2024-09-17T04:25:49.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.391+0000 7f77bde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77c00ad0e0 0x7f77c009de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:49.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.391+0000 7f77bde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77c00ad0e0 0x7f77c009de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55816/0 (socket says 172.21.15.100:55816) 2024-09-17T04:25:49.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.391+0000 7f77bde00640 1 -- 172.21.15.100:0/1711975485 learned_addr learned my addr 172.21.15.100:0/1711975485 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:49.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.391+0000 7f77b5400640 1 --2- 172.21.15.100:0/1711975485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77c00b3550 0x7f77c014a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:49.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.391+0000 7f77be800640 1 --2- 172.21.15.100:0/1711975485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77c015d760 0x7f77c01f6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:49.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.392+0000 7f77bde00640 1 -- 172.21.15.100:0/1711975485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77c00b3550 msgr2=0x7f77c014a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:49.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.392+0000 7f77bde00640 1 --2- 172.21.15.100:0/1711975485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77c00b3550 0x7f77c014a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:49.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.392+0000 7f77bde00640 1 -- 172.21.15.100:0/1711975485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77c015d760 msgr2=0x7f77c01f6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:49.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.392+0000 7f77bde00640 1 --2- 172.21.15.100:0/1711975485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77c015d760 0x7f77c01f6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:49.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.392+0000 7f77bde00640 1 -- 172.21.15.100:0/1711975485 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c0151bd0 con 0x7f77c00ad0e0 2024-09-17T04:25:49.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.392+0000 7f77bde00640 1 --2- 172.21.15.100:0/1711975485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77c00ad0e0 0x7f77c009de00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f77b0009950 tx=0x7f77b0033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:49.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.392+0000 7f77b6a00640 1 -- 172.21.15.100:0/1711975485 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b0031d60 con 0x7f77c00ad0e0 2024-09-17T04:25:49.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.392+0000 7f77b6a00640 1 -- 172.21.15.100:0/1711975485 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77b0033070 con 0x7f77c00ad0e0 2024-09-17T04:25:49.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.392+0000 7f77c4a00640 1 -- 172.21.15.100:0/1711975485 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c0156b30 con 0x7f77c00ad0e0 2024-09-17T04:25:49.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.392+0000 7f77b6a00640 1 -- 172.21.15.100:0/1711975485 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b00334c0 con 0x7f77c00ad0e0 2024-09-17T04:25:49.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.392+0000 7f77c4a00640 1 -- 172.21.15.100:0/1711975485 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c01f5920 con 0x7f77c00ad0e0 2024-09-17T04:25:49.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.393+0000 7f77c4a00640 1 -- 172.21.15.100:0/1711975485 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c0070880 con 0x7f77c00ad0e0 2024-09-17T04:25:49.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.395+0000 7f77b6a00640 1 -- 172.21.15.100:0/1711975485 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f77b0033660 con 0x7f77c00ad0e0 2024-09-17T04:25:49.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.396+0000 7f77b6a00640 1 --2- 172.21.15.100:0/1711975485 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f779c078d40 0x7f779c07b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:49.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.396+0000 7f77b6a00640 1 -- 172.21.15.100:0/1711975485 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f77b00bc690 con 0x7f77c00ad0e0 2024-09-17T04:25:49.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.396+0000 7f77b5400640 1 --2- 172.21.15.100:0/1711975485 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f779c078d40 0x7f779c07b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:49.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.397+0000 7f77b5400640 1 --2- 172.21.15.100:0/1711975485 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f779c078d40 0x7f779c07b250 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f77c00b2080 tx=0x7f77a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:49.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.400+0000 7f77b6a00640 1 -- 172.21.15.100:0/1711975485 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f77c0070880 con 0x7f77c00ad0e0 2024-09-17T04:25:49.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.578+0000 7f77c4a00640 1 -- 172.21.15.100:0/1711975485 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f77c01f5710 con 0x7f779c078d40 2024-09-17T04:25:49.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.584+0000 7f77b6a00640 1 -- 172.21.15.100:0/1711975485 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240698 (secure 0 0 0) 0x7f77c01f5710 con 0x7f779c078d40 2024-09-17T04:25:49.587 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:49.588 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:25:49.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.587+0000 7f77c4a00640 1 -- 172.21.15.100:0/1711975485 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f779c078d40 msgr2=0x7f779c07b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:49.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.587+0000 7f77c4a00640 1 --2- 172.21.15.100:0/1711975485 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f779c078d40 0x7f779c07b250 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f77c00b2080 tx=0x7f77a8009040 comp rx=0 tx=0).stop 2024-09-17T04:25:49.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.587+0000 7f77c4a00640 1 -- 172.21.15.100:0/1711975485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77c00ad0e0 msgr2=0x7f77c009de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:49.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.587+0000 7f77c4a00640 1 --2- 172.21.15.100:0/1711975485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77c00ad0e0 0x7f77c009de00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f77b0009950 tx=0x7f77b0033d60 comp rx=0 tx=0).stop 2024-09-17T04:25:49.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.587+0000 7f77c4a00640 1 -- 172.21.15.100:0/1711975485 shutdown_connections 2024-09-17T04:25:49.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.587+0000 7f77c4a00640 1 --2- 172.21.15.100:0/1711975485 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f779c078d40 0x7f779c07b250 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:49.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.587+0000 7f77c4a00640 1 --2- 172.21.15.100:0/1711975485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77c015d760 0x7f77c01f6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:49.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.587+0000 7f77c4a00640 1 --2- 172.21.15.100:0/1711975485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77c00b3550 0x7f77c014a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:49.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.588+0000 7f77c4a00640 1 --2- 172.21.15.100:0/1711975485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77c00ad0e0 0x7f77c009de00 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:49.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.588+0000 7f77c4a00640 1 -- 172.21.15.100:0/1711975485 >> 172.21.15.100:0/1711975485 conn(0x7f77c006d730 msgr2=0x7f77c0068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:49.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.588+0000 7f77c4a00640 1 -- 172.21.15.100:0/1711975485 shutdown_connections 2024-09-17T04:25:49.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.588+0000 7f77c4a00640 1 -- 172.21.15.100:0/1711975485 wait complete. 2024-09-17T04:25:49.610 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2024-09-17T04:25:47.988845+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590984,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":109,"num_write_kb":1636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":64,"acting":64,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":225608,"kb_used_data":5204,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749112,"statfs":{"total":773094113280,"available":772863090688,"internally_reserved":0,"allocated":5328896,"data_stored":2820657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":225496628},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.823584"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203511+0000","last_change":"2024-09-17T04:25:45.212841+0000","last_active":"2024-09-17T04:25:46.203511+0000","last_peered":"2024-09-17T04:25:46.203511+0000","last_clean":"2024-09-17T04:25:46.203511+0000","last_became_active":"2024-09-17T04:25:45.212541+0000","last_became_peered":"2024-09-17T04:25:45.212541+0000","last_unstale":"2024-09-17T04:25:46.203511+0000","last_undegraded":"2024-09-17T04:25:46.203511+0000","last_fullsized":"2024-09-17T04:25:46.203511+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:15:23.230051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203140+0000","last_change":"2024-09-17T04:25:45.206462+0000","last_active":"2024-09-17T04:25:46.203140+0000","last_peered":"2024-09-17T04:25:46.203140+0000","last_clean":"2024-09-17T04:25:46.203140+0000","last_became_active":"2024-09-17T04:25:45.206164+0000","last_became_peered":"2024-09-17T04:25:45.206164+0000","last_unstale":"2024-09-17T04:25:46.203140+0000","last_undegraded":"2024-09-17T04:25:46.203140+0000","last_fullsized":"2024-09-17T04:25:46.203140+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:43.500761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203732+0000","last_change":"2024-09-17T04:25:45.219500+0000","last_active":"2024-09-17T04:25:46.203732+0000","last_peered":"2024-09-17T04:25:46.203732+0000","last_clean":"2024-09-17T04:25:46.203732+0000","last_became_active":"2024-09-17T04:25:45.219178+0000","last_became_peered":"2024-09-17T04:25:45.219178+0000","last_unstale":"2024-09-17T04:25:46.203732+0000","last_undegraded":"2024-09-17T04:25:46.203732+0000","last_fullsized":"2024-09-17T04:25:46.203732+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:02.906761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203175+0000","last_change":"2024-09-17T04:25:45.204503+0000","last_active":"2024-09-17T04:25:46.203175+0000","last_peered":"2024-09-17T04:25:46.203175+0000","last_clean":"2024-09-17T04:25:46.203175+0000","last_became_active":"2024-09-17T04:25:45.204110+0000","last_became_peered":"2024-09-17T04:25:45.204110+0000","last_unstale":"2024-09-17T04:25:46.203175+0000","last_undegraded":"2024-09-17T04:25:46.203175+0000","last_fullsized":"2024-09-17T04:25:46.203175+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:41:05.533679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"47'4","reported_seq":19,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.204048+0000","last_change":"2024-09-17T04:25:44.201907+0000","last_active":"2024-09-17T04:25:46.204048+0000","last_peered":"2024-09-17T04:25:46.204048+0000","last_clean":"2024-09-17T04:25:46.204048+0000","last_became_active":"2024-09-17T04:25:44.201616+0000","last_became_peered":"2024-09-17T04:25:44.201616+0000","last_unstale":"2024-09-17T04:25:46.204048+0000","last_undegraded":"2024-09-17T04:25:46.204048+0000","last_fullsized":"2024-09-17T04:25:46.204048+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:21:18.328528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T04:25:43.170644+0000","last_change":"2024-09-17T04:25:43.170644+0000","last_active":"2024-09-17T04:25:43.150831+0000","last_peered":"2024-09-17T04:25:43.150831+0000","last_clean":"2024-09-17T04:25:43.150831+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:43.170644+0000","last_undegraded":"2024-09-17T04:25:43.170644+0000","last_fullsized":"2024-09-17T04:25:43.170644+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:43.150831+0000","last_change":"2024-09-17T04:25:43.150831+0000","last_active":"2024-09-17T04:25:43.150831+0000","last_peered":"2024-09-17T04:25:43.150831+0000","last_clean":"2024-09-17T04:25:43.150831+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:43.150831+0000","last_undegraded":"2024-09-17T04:25:43.150831+0000","last_fullsized":"2024-09-17T04:25:43.150831+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T04:25:43.168929+0000","last_change":"2024-09-17T04:25:43.168929+0000","last_active":"2024-09-17T04:25:43.150831+0000","last_peered":"2024-09-17T04:25:43.150831+0000","last_clean":"2024-09-17T04:25:43.150831+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:43.168929+0000","last_undegraded":"2024-09-17T04:25:43.168929+0000","last_fullsized":"2024-09-17T04:25:43.168929+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"47'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203706+0000","last_change":"2024-09-17T04:25:44.197016+0000","last_active":"2024-09-17T04:25:46.203706+0000","last_peered":"2024-09-17T04:25:46.203706+0000","last_clean":"2024-09-17T04:25:46.203706+0000","last_became_active":"2024-09-17T04:25:44.196699+0000","last_became_peered":"2024-09-17T04:25:44.196699+0000","last_unstale":"2024-09-17T04:25:46.203706+0000","last_undegraded":"2024-09-17T04:25:46.203706+0000","last_fullsized":"2024-09-17T04:25:46.203706+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:01.401433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T04:25:43.170344+0000","last_change":"2024-09-17T04:25:43.170344+0000","last_active":"2024-09-17T04:25:43.150831+0000","last_peered":"2024-09-17T04:25:43.150831+0000","last_clean":"2024-09-17T04:25:43.150831+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:43.170344+0000","last_undegraded":"2024-09-17T04:25:43.170344+0000","last_fullsized":"2024-09-17T04:25:43.170344+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T04:25:43.168859+0000","last_change":"2024-09-17T04:25:43.168859+0000","last_active":"2024-09-17T04:25:43.150831+0000","last_peered":"2024-09-17T04:25:43.150831+0000","last_clean":"2024-09-17T04:25:43.150831+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:43.168859+0000","last_undegraded":"2024-09-17T04:25:43.168859+0000","last_fullsized":"2024-09-17T04:25:43.168859+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:43.150831+0000","last_change":"2024-09-17T04:25:43.150831+0000","last_active":"2024-09-17T04:25:43.150831+0000","last_peered":"2024-09-17T04:25:43.150831+0000","last_clean":"2024-09-17T04:25:43.150831+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:43.150831+0000","last_undegraded":"2024-09-17T04:25:43.150831+0000","last_fullsized":"2024-09-17T04:25:43.150831+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T04:25:43.171961+0000","last_change":"2024-09-17T04:25:43.171961+0000","last_active":"2024-09-17T04:25:43.150831+0000","last_peered":"2024-09-17T04:25:43.150831+0000","last_clean":"2024-09-17T04:25:43.150831+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:43.171961+0000","last_undegraded":"2024-09-17T04:25:43.171961+0000","last_fullsized":"2024-09-17T04:25:43.171961+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:43.150831+0000","last_change":"2024-09-17T04:25:43.150831+0000","last_active":"2024-09-17T04:25:43.150831+0000","last_peered":"2024-09-17T04:25:43.150831+0000","last_clean":"2024-09-17T04:25:43.150831+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:43.150831+0000","last_undegraded":"2024-09-17T04:25:43.150831+0000","last_fullsized":"2024-09-17T04:25:43.150831+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203780+0000","last_change":"2024-09-17T04:25:45.213566+0000","last_active":"2024-09-17T04:25:46.203780+0000","last_peered":"2024-09-17T04:25:46.203780+0000","last_clean":"2024-09-17T04:25:46.203780+0000","last_became_active":"2024-09-17T04:25:45.213247+0000","last_became_peered":"2024-09-17T04:25:45.213247+0000","last_unstale":"2024-09-17T04:25:46.203780+0000","last_undegraded":"2024-09-17T04:25:46.203780+0000","last_fullsized":"2024-09-17T04:25:46.203780+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:40.700608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T04:25:43.169486+0000","last_change":"2024-09-17T04:25:43.169486+0000","last_active":"2024-09-17T04:25:43.150831+0000","last_peered":"2024-09-17T04:25:43.150831+0000","last_clean":"2024-09-17T04:25:43.150831+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:43.169486+0000","last_undegraded":"2024-09-17T04:25:43.169486+0000","last_fullsized":"2024-09-17T04:25:43.169486+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:43.150831+0000","last_change":"2024-09-17T04:25:43.150831+0000","last_active":"2024-09-17T04:25:43.150831+0000","last_peered":"2024-09-17T04:25:43.150831+0000","last_clean":"2024-09-17T04:25:43.150831+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:43.150831+0000","last_undegraded":"2024-09-17T04:25:43.150831+0000","last_fullsized":"2024-09-17T04:25:43.150831+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203858+0000","last_change":"2024-09-17T04:25:44.195070+0000","last_active":"2024-09-17T04:25:46.203858+0000","last_peered":"2024-09-17T04:25:46.203858+0000","last_clean":"2024-09-17T04:25:46.203858+0000","last_became_active":"2024-09-17T04:25:44.194320+0000","last_became_peered":"2024-09-17T04:25:44.194320+0000","last_unstale":"2024-09-17T04:25:46.203858+0000","last_undegraded":"2024-09-17T04:25:46.203858+0000","last_fullsized":"2024-09-17T04:25:46.203858+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:04.486996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T04:25:43.168364+0000","last_change":"2024-09-17T04:25:43.168364+0000","last_active":"2024-09-17T04:25:43.150831+0000","last_peered":"2024-09-17T04:25:43.150831+0000","last_clean":"2024-09-17T04:25:43.150831+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:43.168364+0000","last_undegraded":"2024-09-17T04:25:43.168364+0000","last_fullsized":"2024-09-17T04:25:43.168364+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203085+0000","last_change":"2024-09-17T04:25:45.211494+0000","last_active":"2024-09-17T04:25:46.203085+0000","last_peered":"2024-09-17T04:25:46.203085+0000","last_clean":"2024-09-17T04:25:46.203085+0000","last_became_active":"2024-09-17T04:25:45.209477+0000","last_became_peered":"2024-09-17T04:25:45.209477+0000","last_unstale":"2024-09-17T04:25:46.203085+0000","last_undegraded":"2024-09-17T04:25:46.203085+0000","last_fullsized":"2024-09-17T04:25:46.203085+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:00:57.227021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203252+0000","last_change":"2024-09-17T04:25:44.200589+0000","last_active":"2024-09-17T04:25:46.203252+0000","last_peered":"2024-09-17T04:25:46.203252+0000","last_clean":"2024-09-17T04:25:46.203252+0000","last_became_active":"2024-09-17T04:25:44.199602+0000","last_became_peered":"2024-09-17T04:25:44.199602+0000","last_unstale":"2024-09-17T04:25:46.203252+0000","last_undegraded":"2024-09-17T04:25:46.203252+0000","last_fullsized":"2024-09-17T04:25:46.203252+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:32:43.072948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:43.150831+0000","last_change":"2024-09-17T04:25:43.150831+0000","last_active":"2024-09-17T04:25:43.150831+0000","last_peered":"2024-09-17T04:25:43.150831+0000","last_clean":"2024-09-17T04:25:43.150831+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:43.150831+0000","last_undegraded":"2024-09-17T04:25:43.150831+0000","last_fullsized":"2024-09-17T04:25:43.150831+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:43.150831+0000","last_change":"2024-09-17T04:25:43.150831+0000","last_active":"2024-09-17T04:25:43.150831+0000","last_peered":"2024-09-17T04:25:43.150831+0000","last_clean":"2024-09-17T04:25:43.150831+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:43.150831+0000","last_undegraded":"2024-09-17T04:25:43.150831+0000","last_fullsized":"2024-09-17T04:25:43.150831+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T04:25:43.170108+0000","last_change":"2024-09-17T04:25:43.170108+0000","last_active":"2024-09-17T04:25:43.150831+0000","last_peered":"2024-09-17T04:25:43.150831+0000","last_clean":"2024-09-17T04:25:43.150831+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:43.170108+0000","last_undegraded":"2024-09-17T04:25:43.170108+0000","last_fullsized":"2024-09-17T04:25:43.170108+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T04:25:43.168564+0000","last_change":"2024-09-17T04:25:43.168564+0000","last_active":"2024-09-17T04:25:43.150831+0000","last_peered":"2024-09-17T04:25:43.150831+0000","last_clean":"2024-09-17T04:25:43.150831+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:43.168564+0000","last_undegraded":"2024-09-17T04:25:43.168564+0000","last_fullsized":"2024-09-17T04:25:43.168564+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203707+0000","last_change":"2024-09-17T04:25:44.200670+0000","last_active":"2024-09-17T04:25:46.203707+0000","last_peered":"2024-09-17T04:25:46.203707+0000","last_clean":"2024-09-17T04:25:46.203707+0000","last_became_active":"2024-09-17T04:25:44.200382+0000","last_became_peered":"2024-09-17T04:25:44.200382+0000","last_unstale":"2024-09-17T04:25:46.203707+0000","last_undegraded":"2024-09-17T04:25:46.203707+0000","last_fullsized":"2024-09-17T04:25:46.203707+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:15:15.268358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203536+0000","last_change":"2024-09-17T04:25:45.210726+0000","last_active":"2024-09-17T04:25:46.203536+0000","last_peered":"2024-09-17T04:25:46.203536+0000","last_clean":"2024-09-17T04:25:46.203536+0000","last_became_active":"2024-09-17T04:25:45.210304+0000","last_became_peered":"2024-09-17T04:25:45.210304+0000","last_unstale":"2024-09-17T04:25:46.203536+0000","last_undegraded":"2024-09-17T04:25:46.203536+0000","last_fullsized":"2024-09-17T04:25:46.203536+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:02.703247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203968+0000","last_change":"2024-09-17T04:25:44.198264+0000","last_active":"2024-09-17T04:25:46.203968+0000","last_peered":"2024-09-17T04:25:46.203968+0000","last_clean":"2024-09-17T04:25:46.203968+0000","last_became_active":"2024-09-17T04:25:44.197966+0000","last_became_peered":"2024-09-17T04:25:44.197966+0000","last_unstale":"2024-09-17T04:25:46.203968+0000","last_undegraded":"2024-09-17T04:25:46.203968+0000","last_fullsized":"2024-09-17T04:25:46.203968+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:02.720620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T04:25:43.168142+0000","last_change":"2024-09-17T04:25:43.168142+0000","last_active":"2024-09-17T04:25:43.150831+0000","last_peered":"2024-09-17T04:25:43.150831+0000","last_clean":"2024-09-17T04:25:43.150831+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:43.168142+0000","last_undegraded":"2024-09-17T04:25:43.168142+0000","last_fullsized":"2024-09-17T04:25:43.168142+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:43.150831+0000","last_change":"2024-09-17T04:25:43.150831+0000","last_active":"2024-09-17T04:25:43.150831+0000","last_peered":"2024-09-17T04:25:43.150831+0000","last_clean":"2024-09-17T04:25:43.150831+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:43.150831+0000","last_undegraded":"2024-09-17T04:25:43.150831+0000","last_fullsized":"2024-09-17T04:25:43.150831+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T04:25:43.168065+0000","last_change":"2024-09-17T04:25:43.168065+0000","last_active":"2024-09-17T04:25:43.150831+0000","last_peered":"2024-09-17T04:25:43.150831+0000","last_clean":"2024-09-17T04:25:43.150831+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:43.168065+0000","last_undegraded":"2024-09-17T04:25:43.168065+0000","last_fullsized":"2024-09-17T04:25:43.168065+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203861+0000","last_change":"2024-09-17T04:25:44.191599+0000","last_active":"2024-09-17T04:25:46.203861+0000","last_peered":"2024-09-17T04:25:46.203861+0000","last_clean":"2024-09-17T04:25:46.203861+0000","last_became_active":"2024-09-17T04:25:44.191038+0000","last_became_peered":"2024-09-17T04:25:44.191038+0000","last_unstale":"2024-09-17T04:25:46.203861+0000","last_undegraded":"2024-09-17T04:25:46.203861+0000","last_fullsized":"2024-09-17T04:25:46.203861+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:23:16.825181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T04:25:43.168930+0000","last_change":"2024-09-17T04:25:43.168930+0000","last_active":"2024-09-17T04:25:43.150831+0000","last_peered":"2024-09-17T04:25:43.150831+0000","last_clean":"2024-09-17T04:25:43.150831+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:43.168930+0000","last_undegraded":"2024-09-17T04:25:43.168930+0000","last_fullsized":"2024-09-17T04:25:43.168930+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:43.150831+0000","last_change":"2024-09-17T04:25:43.150831+0000","last_active":"2024-09-17T04:25:43.150831+0000","last_peered":"2024-09-17T04:25:43.150831+0000","last_clean":"2024-09-17T04:25:43.150831+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:43.150831+0000","last_undegraded":"2024-09-17T04:25:43.150831+0000","last_fullsized":"2024-09-17T04:25:43.150831+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203974+0000","last_change":"2024-09-17T04:25:45.206724+0000","last_active":"2024-09-17T04:25:46.203974+0000","last_peered":"2024-09-17T04:25:46.203974+0000","last_clean":"2024-09-17T04:25:46.203974+0000","last_became_active":"2024-09-17T04:25:45.205733+0000","last_became_peered":"2024-09-17T04:25:45.205733+0000","last_unstale":"2024-09-17T04:25:46.203974+0000","last_undegraded":"2024-09-17T04:25:46.203974+0000","last_fullsized":"2024-09-17T04:25:46.203974+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:15:30.185576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T04:25:43.170978+0000","last_change":"2024-09-17T04:25:43.170978+0000","last_active":"2024-09-17T04:25:43.150831+0000","last_peered":"2024-09-17T04:25:43.150831+0000","last_clean":"2024-09-17T04:25:43.150831+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:43.170978+0000","last_undegraded":"2024-09-17T04:25:43.170978+0000","last_fullsized":"2024-09-17T04:25:43.170978+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203585+0000","last_change":"2024-09-17T04:25:45.210940+0000","last_active":"2024-09-17T04:25:46.203585+0000","last_peered":"2024-09-17T04:25:46.203585+0000","last_clean":"2024-09-17T04:25:46.203585+0000","last_became_active":"2024-09-17T04:25:45.210565+0000","last_became_peered":"2024-09-17T04:25:45.210565+0000","last_unstale":"2024-09-17T04:25:46.203585+0000","last_undegraded":"2024-09-17T04:25:46.203585+0000","last_fullsized":"2024-09-17T04:25:46.203585+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:40.310997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T04:25:43.169655+0000","last_change":"2024-09-17T04:25:43.169655+0000","last_active":"2024-09-17T04:25:43.150831+0000","last_peered":"2024-09-17T04:25:43.150831+0000","last_clean":"2024-09-17T04:25:43.150831+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:43.169655+0000","last_undegraded":"2024-09-17T04:25:43.169655+0000","last_fullsized":"2024-09-17T04:25:43.169655+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:43.150831+0000","last_change":"2024-09-17T04:25:43.150831+0000","last_active":"2024-09-17T04:25:43.150831+0000","last_peered":"2024-09-17T04:25:43.150831+0000","last_clean":"2024-09-17T04:25:43.150831+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:43.150831+0000","last_undegraded":"2024-09-17T04:25:43.150831+0000","last_fullsized":"2024-09-17T04:25:43.150831+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:43.150831+0000","last_change":"2024-09-17T04:25:43.150831+0000","last_active":"2024-09-17T04:25:43.150831+0000","last_peered":"2024-09-17T04:25:43.150831+0000","last_clean":"2024-09-17T04:25:43.150831+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:43.150831+0000","last_undegraded":"2024-09-17T04:25:43.150831+0000","last_fullsized":"2024-09-17T04:25:43.150831+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:25:44.162204+0000","last_change":"2024-09-17T04:25:44.162204+0000","last_active":"2024-09-17T04:25:44.162204+0000","last_peered":"2024-09-17T04:25:44.162204+0000","last_clean":"2024-09-17T04:25:44.162204+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:44.162204+0000","last_undegraded":"2024-09-17T04:25:44.162204+0000","last_fullsized":"2024-09-17T04:25:44.162204+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T04:25:43.168246+0000","last_change":"2024-09-17T04:25:43.168246+0000","last_active":"2024-09-17T04:25:43.150831+0000","last_peered":"2024-09-17T04:25:43.150831+0000","last_clean":"2024-09-17T04:25:43.150831+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:25:43.168246+0000","last_undegraded":"2024-09-17T04:25:43.168246+0000","last_fullsized":"2024-09-17T04:25:43.168246+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":151,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T04:25:43.171745+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:25:43.171745+0000","last_peered":"2024-09-17T04:25:43.171745+0000","last_clean":"2024-09-17T04:25:43.171745+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:25:43.171745+0000","last_undegraded":"2024-09-17T04:25:43.171745+0000","last_fullsized":"2024-09-17T04:25:43.171745+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":1},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":616,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":44,"acting":44,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476761,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1110016,"data_stored":795042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476760,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27536,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":516096,"data_stored":204674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476761,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":516096,"data_stored":204674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476761,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":516096,"data_stored":204674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27524,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":503808,"data_stored":198857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509463,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27520,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":503808,"data_stored":198857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509463,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1097728,"data_stored":789225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542169,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":565248,"data_stored":224654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:25:49.614 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:25:49.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.870+0000 7facf4c00640 1 -- 172.21.15.100:0/492263526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facf015d760 msgr2=0x7facf01535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:49.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.870+0000 7facf4c00640 1 --2- 172.21.15.100:0/492263526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facf015d760 0x7facf01535c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7face0005fd0 tx=0x7face002aa10 comp rx=0 tx=0).stop 2024-09-17T04:25:49.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.870+0000 7facf4c00640 1 -- 172.21.15.100:0/492263526 shutdown_connections 2024-09-17T04:25:49.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.870+0000 7facf4c00640 1 --2- 172.21.15.100:0/492263526 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7facf00ad0e0 0x7facf00ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:49.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.870+0000 7facf4c00640 1 --2- 172.21.15.100:0/492263526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facf015d760 0x7facf01535c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:49.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.870+0000 7facf4c00640 1 -- 172.21.15.100:0/492263526 >> 172.21.15.100:0/492263526 conn(0x7facf006d730 msgr2=0x7facf00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:49.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.870+0000 7facf4c00640 1 -- 172.21.15.100:0/492263526 shutdown_connections 2024-09-17T04:25:49.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.870+0000 7facf4c00640 1 -- 172.21.15.100:0/492263526 wait complete. 2024-09-17T04:25:49.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.872+0000 7facf4c00640 1 Processor -- start 2024-09-17T04:25:49.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.873+0000 7facf4c00640 1 -- start start 2024-09-17T04:25:49.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.874+0000 7facf4c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7facf00ad0e0 0x7facf0148810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:49.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.874+0000 7facf4c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facf015d760 0x7facf009a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:49.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.874+0000 7facf4c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facf0160310 0x7facf009d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:49.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.874+0000 7facf4c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7facf006f220 con 0x7facf015d760 2024-09-17T04:25:49.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.874+0000 7facf4c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7facf006f0a0 con 0x7facf0160310 2024-09-17T04:25:49.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.874+0000 7facf4c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7facf00b4c20 con 0x7facf00ad0e0 2024-09-17T04:25:49.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.874+0000 7face5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facf015d760 0x7facf009a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:49.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.874+0000 7face5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facf015d760 0x7facf009a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55838/0 (socket says 172.21.15.100:55838) 2024-09-17T04:25:49.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.874+0000 7face5400640 1 -- 172.21.15.100:0/3461829326 learned_addr learned my addr 172.21.15.100:0/3461829326 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:49.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.874+0000 7facede00640 1 --2- 172.21.15.100:0/3461829326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7facf00ad0e0 0x7facf0148810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:49.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.875+0000 7facee800640 1 --2- 172.21.15.100:0/3461829326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facf0160310 0x7facf009d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:49.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.875+0000 7face5400640 1 -- 172.21.15.100:0/3461829326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7facf00ad0e0 msgr2=0x7facf0148810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:49.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.875+0000 7face5400640 1 --2- 172.21.15.100:0/3461829326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7facf00ad0e0 0x7facf0148810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:49.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.875+0000 7face5400640 1 -- 172.21.15.100:0/3461829326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facf0160310 msgr2=0x7facf009d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:49.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.875+0000 7face5400640 1 --2- 172.21.15.100:0/3461829326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facf0160310 0x7facf009d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:49.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.875+0000 7face5400640 1 -- 172.21.15.100:0/3461829326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf009c4d0 con 0x7facf015d760 2024-09-17T04:25:49.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.875+0000 7face5400640 1 --2- 172.21.15.100:0/3461829326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facf015d760 0x7facf009a340 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7facd800d6d0 tx=0x7facd800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:49.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.875+0000 7face6a00640 1 -- 172.21.15.100:0/3461829326 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8017070 con 0x7facf015d760 2024-09-17T04:25:49.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.875+0000 7face6a00640 1 -- 172.21.15.100:0/3461829326 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facd8004310 con 0x7facf015d760 2024-09-17T04:25:49.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.875+0000 7facf4c00640 1 -- 172.21.15.100:0/3461829326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf009c7b0 con 0x7facf015d760 2024-09-17T04:25:49.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.876+0000 7face6a00640 1 -- 172.21.15.100:0/3461829326 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8009a20 con 0x7facf015d760 2024-09-17T04:25:49.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.876+0000 7facf4c00640 1 -- 172.21.15.100:0/3461829326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf009cd70 con 0x7facf015d760 2024-09-17T04:25:49.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.877+0000 7facf4c00640 1 -- 172.21.15.100:0/3461829326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facf0160fa0 con 0x7facf015d760 2024-09-17T04:25:49.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.878+0000 7face6a00640 1 -- 172.21.15.100:0/3461829326 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7facd80035f0 con 0x7facf015d760 2024-09-17T04:25:49.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.879+0000 7face6a00640 1 --2- 172.21.15.100:0/3461829326 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faccc078d40 0x7faccc07b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:49.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.879+0000 7face6a00640 1 -- 172.21.15.100:0/3461829326 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7facd809b350 con 0x7facf015d760 2024-09-17T04:25:49.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.879+0000 7facede00640 1 --2- 172.21.15.100:0/3461829326 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faccc078d40 0x7faccc07b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:49.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.880+0000 7facede00640 1 --2- 172.21.15.100:0/3461829326 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faccc078d40 0x7faccc07b250 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7face0035000 tx=0x7face003b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:49.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:49.883+0000 7face6a00640 1 -- 172.21.15.100:0/3461829326 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7facd8062db0 con 0x7facf015d760 2024-09-17T04:25:50.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.063+0000 7facf4c00640 1 -- 172.21.15.100:0/3461829326 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7facf00008d0 con 0x7faccc078d40 2024-09-17T04:25:50.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.069+0000 7face6a00640 1 -- 172.21.15.100:0/3461829326 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7facf00008d0 con 0x7faccc078d40 2024-09-17T04:25:50.071 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:50.073 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:25:50.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.072+0000 7facf4c00640 1 -- 172.21.15.100:0/3461829326 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faccc078d40 msgr2=0x7faccc07b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:50.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.072+0000 7facf4c00640 1 --2- 172.21.15.100:0/3461829326 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faccc078d40 0x7faccc07b250 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7face0035000 tx=0x7face003b040 comp rx=0 tx=0).stop 2024-09-17T04:25:50.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.072+0000 7facf4c00640 1 -- 172.21.15.100:0/3461829326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facf015d760 msgr2=0x7facf009a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:50.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.072+0000 7facf4c00640 1 --2- 172.21.15.100:0/3461829326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facf015d760 0x7facf009a340 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7facd800d6d0 tx=0x7facd800db90 comp rx=0 tx=0).stop 2024-09-17T04:25:50.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.072+0000 7facf4c00640 1 -- 172.21.15.100:0/3461829326 shutdown_connections 2024-09-17T04:25:50.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.072+0000 7facf4c00640 1 --2- 172.21.15.100:0/3461829326 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faccc078d40 0x7faccc07b250 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:50.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.072+0000 7facf4c00640 1 --2- 172.21.15.100:0/3461829326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facf0160310 0x7facf009d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:50.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.072+0000 7facf4c00640 1 --2- 172.21.15.100:0/3461829326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facf015d760 0x7facf009a340 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:50.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.072+0000 7facf4c00640 1 --2- 172.21.15.100:0/3461829326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7facf00ad0e0 0x7facf0148810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:50.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.072+0000 7facf4c00640 1 -- 172.21.15.100:0/3461829326 >> 172.21.15.100:0/3461829326 conn(0x7facf006d730 msgr2=0x7facf00cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:50.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.072+0000 7facf4c00640 1 -- 172.21.15.100:0/3461829326 shutdown_connections 2024-09-17T04:25:50.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.072+0000 7facf4c00640 1 -- 172.21.15.100:0/3461829326 wait complete. 2024-09-17T04:25:50.094 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2024-09-17T04:25:49.989381+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225964,"kb_used_data":5544,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748756,"statfs":{"total":773094113280,"available":772862726144,"internally_reserved":0,"allocated":5677056,"data_stored":2977684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":225496628},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001556"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.202703+0000","last_change":"2024-09-17T04:25:45.193754+0000","last_active":"2024-09-17T04:25:46.202703+0000","last_peered":"2024-09-17T04:25:46.202703+0000","last_clean":"2024-09-17T04:25:46.202703+0000","last_became_active":"2024-09-17T04:25:45.193409+0000","last_became_peered":"2024-09-17T04:25:45.193409+0000","last_unstale":"2024-09-17T04:25:46.202703+0000","last_undegraded":"2024-09-17T04:25:46.202703+0000","last_fullsized":"2024-09-17T04:25:46.202703+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:27.305639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.201239+0000","last_change":"2024-09-17T04:25:45.198348+0000","last_active":"2024-09-17T04:25:46.201239+0000","last_peered":"2024-09-17T04:25:46.201239+0000","last_clean":"2024-09-17T04:25:46.201239+0000","last_became_active":"2024-09-17T04:25:45.198112+0000","last_became_peered":"2024-09-17T04:25:45.198112+0000","last_unstale":"2024-09-17T04:25:46.201239+0000","last_undegraded":"2024-09-17T04:25:46.201239+0000","last_fullsized":"2024-09-17T04:25:46.201239+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:48.516895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.261498+0000","last_change":"2024-09-17T04:25:45.207186+0000","last_active":"2024-09-17T04:25:46.261498+0000","last_peered":"2024-09-17T04:25:46.261498+0000","last_clean":"2024-09-17T04:25:46.261498+0000","last_became_active":"2024-09-17T04:25:45.206799+0000","last_became_peered":"2024-09-17T04:25:45.206799+0000","last_unstale":"2024-09-17T04:25:46.261498+0000","last_undegraded":"2024-09-17T04:25:46.261498+0000","last_fullsized":"2024-09-17T04:25:46.261498+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:04:50.124670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.260450+0000","last_change":"2024-09-17T04:25:45.213750+0000","last_active":"2024-09-17T04:25:46.260450+0000","last_peered":"2024-09-17T04:25:46.260450+0000","last_clean":"2024-09-17T04:25:46.260450+0000","last_became_active":"2024-09-17T04:25:45.213449+0000","last_became_peered":"2024-09-17T04:25:45.213449+0000","last_unstale":"2024-09-17T04:25:46.260450+0000","last_undegraded":"2024-09-17T04:25:46.260450+0000","last_fullsized":"2024-09-17T04:25:46.260450+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:48:54.343124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.201714+0000","last_change":"2024-09-17T04:25:45.199146+0000","last_active":"2024-09-17T04:25:46.201714+0000","last_peered":"2024-09-17T04:25:46.201714+0000","last_clean":"2024-09-17T04:25:46.201714+0000","last_became_active":"2024-09-17T04:25:45.198826+0000","last_became_peered":"2024-09-17T04:25:45.198826+0000","last_unstale":"2024-09-17T04:25:46.201714+0000","last_undegraded":"2024-09-17T04:25:46.201714+0000","last_fullsized":"2024-09-17T04:25:46.201714+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:16:11.354587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203511+0000","last_change":"2024-09-17T04:25:45.212841+0000","last_active":"2024-09-17T04:25:46.203511+0000","last_peered":"2024-09-17T04:25:46.203511+0000","last_clean":"2024-09-17T04:25:46.203511+0000","last_became_active":"2024-09-17T04:25:45.212541+0000","last_became_peered":"2024-09-17T04:25:45.212541+0000","last_unstale":"2024-09-17T04:25:46.203511+0000","last_undegraded":"2024-09-17T04:25:46.203511+0000","last_fullsized":"2024-09-17T04:25:46.203511+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:15:23.230051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.202696+0000","last_change":"2024-09-17T04:25:45.206533+0000","last_active":"2024-09-17T04:25:46.202696+0000","last_peered":"2024-09-17T04:25:46.202696+0000","last_clean":"2024-09-17T04:25:46.202696+0000","last_became_active":"2024-09-17T04:25:45.204877+0000","last_became_peered":"2024-09-17T04:25:45.204877+0000","last_unstale":"2024-09-17T04:25:46.202696+0000","last_undegraded":"2024-09-17T04:25:46.202696+0000","last_fullsized":"2024-09-17T04:25:46.202696+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:25.235038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.260236+0000","last_change":"2024-09-17T04:25:45.197506+0000","last_active":"2024-09-17T04:25:46.260236+0000","last_peered":"2024-09-17T04:25:46.260236+0000","last_clean":"2024-09-17T04:25:46.260236+0000","last_became_active":"2024-09-17T04:25:45.196203+0000","last_became_peered":"2024-09-17T04:25:45.196203+0000","last_unstale":"2024-09-17T04:25:46.260236+0000","last_undegraded":"2024-09-17T04:25:46.260236+0000","last_fullsized":"2024-09-17T04:25:46.260236+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:05:30.568582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.260641+0000","last_change":"2024-09-17T04:25:45.207896+0000","last_active":"2024-09-17T04:25:46.260641+0000","last_peered":"2024-09-17T04:25:46.260641+0000","last_clean":"2024-09-17T04:25:46.260641+0000","last_became_active":"2024-09-17T04:25:45.207388+0000","last_became_peered":"2024-09-17T04:25:45.207388+0000","last_unstale":"2024-09-17T04:25:46.260641+0000","last_undegraded":"2024-09-17T04:25:46.260641+0000","last_fullsized":"2024-09-17T04:25:46.260641+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:40.305214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.261040+0000","last_change":"2024-09-17T04:25:45.211058+0000","last_active":"2024-09-17T04:25:46.261040+0000","last_peered":"2024-09-17T04:25:46.261040+0000","last_clean":"2024-09-17T04:25:46.261040+0000","last_became_active":"2024-09-17T04:25:45.210573+0000","last_became_peered":"2024-09-17T04:25:45.210573+0000","last_unstale":"2024-09-17T04:25:46.261040+0000","last_undegraded":"2024-09-17T04:25:46.261040+0000","last_fullsized":"2024-09-17T04:25:46.261040+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:39.199063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.261802+0000","last_change":"2024-09-17T04:25:45.206576+0000","last_active":"2024-09-17T04:25:46.261802+0000","last_peered":"2024-09-17T04:25:46.261802+0000","last_clean":"2024-09-17T04:25:46.261802+0000","last_became_active":"2024-09-17T04:25:45.206275+0000","last_became_peered":"2024-09-17T04:25:45.206275+0000","last_unstale":"2024-09-17T04:25:46.261802+0000","last_undegraded":"2024-09-17T04:25:46.261802+0000","last_fullsized":"2024-09-17T04:25:46.261802+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:04.440015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.202139+0000","last_change":"2024-09-17T04:25:45.209356+0000","last_active":"2024-09-17T04:25:46.202139+0000","last_peered":"2024-09-17T04:25:46.202139+0000","last_clean":"2024-09-17T04:25:46.202139+0000","last_became_active":"2024-09-17T04:25:45.208961+0000","last_became_peered":"2024-09-17T04:25:45.208961+0000","last_unstale":"2024-09-17T04:25:46.202139+0000","last_undegraded":"2024-09-17T04:25:46.202139+0000","last_fullsized":"2024-09-17T04:25:46.202139+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:46:30.226113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.262371+0000","last_change":"2024-09-17T04:25:45.203893+0000","last_active":"2024-09-17T04:25:46.262371+0000","last_peered":"2024-09-17T04:25:46.262371+0000","last_clean":"2024-09-17T04:25:46.262371+0000","last_became_active":"2024-09-17T04:25:45.203596+0000","last_became_peered":"2024-09-17T04:25:45.203596+0000","last_unstale":"2024-09-17T04:25:46.262371+0000","last_undegraded":"2024-09-17T04:25:46.262371+0000","last_fullsized":"2024-09-17T04:25:46.262371+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:59:36.416201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.260251+0000","last_change":"2024-09-17T04:25:45.203028+0000","last_active":"2024-09-17T04:25:46.260251+0000","last_peered":"2024-09-17T04:25:46.260251+0000","last_clean":"2024-09-17T04:25:46.260251+0000","last_became_active":"2024-09-17T04:25:45.202686+0000","last_became_peered":"2024-09-17T04:25:45.202686+0000","last_unstale":"2024-09-17T04:25:46.260251+0000","last_undegraded":"2024-09-17T04:25:46.260251+0000","last_fullsized":"2024-09-17T04:25:46.260251+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:04:50.124670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203140+0000","last_change":"2024-09-17T04:25:45.206462+0000","last_active":"2024-09-17T04:25:46.203140+0000","last_peered":"2024-09-17T04:25:46.203140+0000","last_clean":"2024-09-17T04:25:46.203140+0000","last_became_active":"2024-09-17T04:25:45.206164+0000","last_became_peered":"2024-09-17T04:25:45.206164+0000","last_unstale":"2024-09-17T04:25:46.203140+0000","last_undegraded":"2024-09-17T04:25:46.203140+0000","last_fullsized":"2024-09-17T04:25:46.203140+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:43.500761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.260736+0000","last_change":"2024-09-17T04:25:45.210874+0000","last_active":"2024-09-17T04:25:46.260736+0000","last_peered":"2024-09-17T04:25:46.260736+0000","last_clean":"2024-09-17T04:25:46.260736+0000","last_became_active":"2024-09-17T04:25:45.210478+0000","last_became_peered":"2024-09-17T04:25:45.210478+0000","last_unstale":"2024-09-17T04:25:46.260736+0000","last_undegraded":"2024-09-17T04:25:46.260736+0000","last_fullsized":"2024-09-17T04:25:46.260736+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:45.960891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.262040+0000","last_change":"2024-09-17T04:25:45.193671+0000","last_active":"2024-09-17T04:25:46.262040+0000","last_peered":"2024-09-17T04:25:46.262040+0000","last_clean":"2024-09-17T04:25:46.262040+0000","last_became_active":"2024-09-17T04:25:45.193357+0000","last_became_peered":"2024-09-17T04:25:45.193357+0000","last_unstale":"2024-09-17T04:25:46.262040+0000","last_undegraded":"2024-09-17T04:25:46.262040+0000","last_fullsized":"2024-09-17T04:25:46.262040+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:48:54.343124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.261778+0000","last_change":"2024-09-17T04:25:45.199220+0000","last_active":"2024-09-17T04:25:46.261778+0000","last_peered":"2024-09-17T04:25:46.261778+0000","last_clean":"2024-09-17T04:25:46.261778+0000","last_became_active":"2024-09-17T04:25:45.198557+0000","last_became_peered":"2024-09-17T04:25:45.198557+0000","last_unstale":"2024-09-17T04:25:46.261778+0000","last_undegraded":"2024-09-17T04:25:46.261778+0000","last_fullsized":"2024-09-17T04:25:46.261778+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:30:55.393999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.261494+0000","last_change":"2024-09-17T04:25:45.200213+0000","last_active":"2024-09-17T04:25:46.261494+0000","last_peered":"2024-09-17T04:25:46.261494+0000","last_clean":"2024-09-17T04:25:46.261494+0000","last_became_active":"2024-09-17T04:25:45.199881+0000","last_became_peered":"2024-09-17T04:25:45.199881+0000","last_unstale":"2024-09-17T04:25:46.261494+0000","last_undegraded":"2024-09-17T04:25:46.261494+0000","last_fullsized":"2024-09-17T04:25:46.261494+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:30.212177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.262679+0000","last_change":"2024-09-17T04:25:45.198554+0000","last_active":"2024-09-17T04:25:46.262679+0000","last_peered":"2024-09-17T04:25:46.262679+0000","last_clean":"2024-09-17T04:25:46.262679+0000","last_became_active":"2024-09-17T04:25:45.198211+0000","last_became_peered":"2024-09-17T04:25:45.198211+0000","last_unstale":"2024-09-17T04:25:46.262679+0000","last_undegraded":"2024-09-17T04:25:46.262679+0000","last_fullsized":"2024-09-17T04:25:46.262679+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:29.829075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203732+0000","last_change":"2024-09-17T04:25:45.219500+0000","last_active":"2024-09-17T04:25:46.203732+0000","last_peered":"2024-09-17T04:25:46.203732+0000","last_clean":"2024-09-17T04:25:46.203732+0000","last_became_active":"2024-09-17T04:25:45.219178+0000","last_became_peered":"2024-09-17T04:25:45.219178+0000","last_unstale":"2024-09-17T04:25:46.203732+0000","last_undegraded":"2024-09-17T04:25:46.203732+0000","last_fullsized":"2024-09-17T04:25:46.203732+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:02.906761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.260523+0000","last_change":"2024-09-17T04:25:45.217515+0000","last_active":"2024-09-17T04:25:46.260523+0000","last_peered":"2024-09-17T04:25:46.260523+0000","last_clean":"2024-09-17T04:25:46.260523+0000","last_became_active":"2024-09-17T04:25:45.213587+0000","last_became_peered":"2024-09-17T04:25:45.213587+0000","last_unstale":"2024-09-17T04:25:46.260523+0000","last_undegraded":"2024-09-17T04:25:46.260523+0000","last_fullsized":"2024-09-17T04:25:46.260523+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:22:26.619473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.261075+0000","last_change":"2024-09-17T04:25:45.198452+0000","last_active":"2024-09-17T04:25:46.261075+0000","last_peered":"2024-09-17T04:25:46.261075+0000","last_clean":"2024-09-17T04:25:46.261075+0000","last_became_active":"2024-09-17T04:25:45.197771+0000","last_became_peered":"2024-09-17T04:25:45.197771+0000","last_unstale":"2024-09-17T04:25:46.261075+0000","last_undegraded":"2024-09-17T04:25:46.261075+0000","last_fullsized":"2024-09-17T04:25:46.261075+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:18.073247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.260795+0000","last_change":"2024-09-17T04:25:45.200594+0000","last_active":"2024-09-17T04:25:46.260795+0000","last_peered":"2024-09-17T04:25:46.260795+0000","last_clean":"2024-09-17T04:25:46.260795+0000","last_became_active":"2024-09-17T04:25:45.196264+0000","last_became_peered":"2024-09-17T04:25:45.196264+0000","last_unstale":"2024-09-17T04:25:46.260795+0000","last_undegraded":"2024-09-17T04:25:46.260795+0000","last_fullsized":"2024-09-17T04:25:46.260795+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:55:11.689996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.260800+0000","last_change":"2024-09-17T04:25:45.209971+0000","last_active":"2024-09-17T04:25:46.260800+0000","last_peered":"2024-09-17T04:25:46.260800+0000","last_clean":"2024-09-17T04:25:46.260800+0000","last_became_active":"2024-09-17T04:25:45.208338+0000","last_became_peered":"2024-09-17T04:25:45.208338+0000","last_unstale":"2024-09-17T04:25:46.260800+0000","last_undegraded":"2024-09-17T04:25:46.260800+0000","last_fullsized":"2024-09-17T04:25:46.260800+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:44:56.281993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.262069+0000","last_change":"2024-09-17T04:25:45.205616+0000","last_active":"2024-09-17T04:25:46.262069+0000","last_peered":"2024-09-17T04:25:46.262069+0000","last_clean":"2024-09-17T04:25:46.262069+0000","last_became_active":"2024-09-17T04:25:45.205281+0000","last_became_peered":"2024-09-17T04:25:45.205281+0000","last_unstale":"2024-09-17T04:25:46.262069+0000","last_undegraded":"2024-09-17T04:25:46.262069+0000","last_fullsized":"2024-09-17T04:25:46.262069+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:51.754769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.261456+0000","last_change":"2024-09-17T04:25:45.192508+0000","last_active":"2024-09-17T04:25:46.261456+0000","last_peered":"2024-09-17T04:25:46.261456+0000","last_clean":"2024-09-17T04:25:46.261456+0000","last_became_active":"2024-09-17T04:25:45.192177+0000","last_became_peered":"2024-09-17T04:25:45.192177+0000","last_unstale":"2024-09-17T04:25:46.261456+0000","last_undegraded":"2024-09-17T04:25:46.261456+0000","last_fullsized":"2024-09-17T04:25:46.261456+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:05:30.568582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.202394+0000","last_change":"2024-09-17T04:25:45.208503+0000","last_active":"2024-09-17T04:25:46.202394+0000","last_peered":"2024-09-17T04:25:46.202394+0000","last_clean":"2024-09-17T04:25:46.202394+0000","last_became_active":"2024-09-17T04:25:45.208145+0000","last_became_peered":"2024-09-17T04:25:45.208145+0000","last_unstale":"2024-09-17T04:25:46.202394+0000","last_undegraded":"2024-09-17T04:25:46.202394+0000","last_fullsized":"2024-09-17T04:25:46.202394+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:18.390023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.261727+0000","last_change":"2024-09-17T04:25:45.212445+0000","last_active":"2024-09-17T04:25:46.261727+0000","last_peered":"2024-09-17T04:25:46.261727+0000","last_clean":"2024-09-17T04:25:46.261727+0000","last_became_active":"2024-09-17T04:25:45.211619+0000","last_became_peered":"2024-09-17T04:25:45.211619+0000","last_unstale":"2024-09-17T04:25:46.261727+0000","last_undegraded":"2024-09-17T04:25:46.261727+0000","last_fullsized":"2024-09-17T04:25:46.261727+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:57:39.219288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.260916+0000","last_change":"2024-09-17T04:25:45.198803+0000","last_active":"2024-09-17T04:25:46.260916+0000","last_peered":"2024-09-17T04:25:46.260916+0000","last_clean":"2024-09-17T04:25:46.260916+0000","last_became_active":"2024-09-17T04:25:45.198449+0000","last_became_peered":"2024-09-17T04:25:45.198449+0000","last_unstale":"2024-09-17T04:25:46.260916+0000","last_undegraded":"2024-09-17T04:25:46.260916+0000","last_fullsized":"2024-09-17T04:25:46.260916+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:07.907798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203175+0000","last_change":"2024-09-17T04:25:45.204503+0000","last_active":"2024-09-17T04:25:46.203175+0000","last_peered":"2024-09-17T04:25:46.203175+0000","last_clean":"2024-09-17T04:25:46.203175+0000","last_became_active":"2024-09-17T04:25:45.204110+0000","last_became_peered":"2024-09-17T04:25:45.204110+0000","last_unstale":"2024-09-17T04:25:46.203175+0000","last_undegraded":"2024-09-17T04:25:46.203175+0000","last_fullsized":"2024-09-17T04:25:46.203175+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:41:05.533679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.261465+0000","last_change":"2024-09-17T04:25:45.205415+0000","last_active":"2024-09-17T04:25:46.261465+0000","last_peered":"2024-09-17T04:25:46.261465+0000","last_clean":"2024-09-17T04:25:46.261465+0000","last_became_active":"2024-09-17T04:25:45.205029+0000","last_became_peered":"2024-09-17T04:25:45.205029+0000","last_unstale":"2024-09-17T04:25:46.261465+0000","last_undegraded":"2024-09-17T04:25:46.261465+0000","last_fullsized":"2024-09-17T04:25:46.261465+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:32:15.223639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.262322+0000","last_change":"2024-09-17T04:25:45.217157+0000","last_active":"2024-09-17T04:25:46.262322+0000","last_peered":"2024-09-17T04:25:46.262322+0000","last_clean":"2024-09-17T04:25:46.262322+0000","last_became_active":"2024-09-17T04:25:45.215009+0000","last_became_peered":"2024-09-17T04:25:45.215009+0000","last_unstale":"2024-09-17T04:25:46.262322+0000","last_undegraded":"2024-09-17T04:25:46.262322+0000","last_fullsized":"2024-09-17T04:25:46.262322+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:44.545327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"47'4","reported_seq":19,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.204048+0000","last_change":"2024-09-17T04:25:44.201907+0000","last_active":"2024-09-17T04:25:46.204048+0000","last_peered":"2024-09-17T04:25:46.204048+0000","last_clean":"2024-09-17T04:25:46.204048+0000","last_became_active":"2024-09-17T04:25:44.201616+0000","last_became_peered":"2024-09-17T04:25:44.201616+0000","last_unstale":"2024-09-17T04:25:46.204048+0000","last_undegraded":"2024-09-17T04:25:46.204048+0000","last_fullsized":"2024-09-17T04:25:46.204048+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:21:18.328528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.261071+0000","last_change":"2024-09-17T04:25:45.202242+0000","last_active":"2024-09-17T04:25:46.261071+0000","last_peered":"2024-09-17T04:25:46.261071+0000","last_clean":"2024-09-17T04:25:46.261071+0000","last_became_active":"2024-09-17T04:25:45.201924+0000","last_became_peered":"2024-09-17T04:25:45.201924+0000","last_unstale":"2024-09-17T04:25:46.261071+0000","last_undegraded":"2024-09-17T04:25:46.261071+0000","last_fullsized":"2024-09-17T04:25:46.261071+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:06.059646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"47'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.261844+0000","last_change":"2024-09-17T04:25:44.201783+0000","last_active":"2024-09-17T04:25:46.261844+0000","last_peered":"2024-09-17T04:25:46.261844+0000","last_clean":"2024-09-17T04:25:46.261844+0000","last_became_active":"2024-09-17T04:25:44.201388+0000","last_became_peered":"2024-09-17T04:25:44.201388+0000","last_unstale":"2024-09-17T04:25:46.261844+0000","last_undegraded":"2024-09-17T04:25:46.261844+0000","last_fullsized":"2024-09-17T04:25:46.261844+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:48.599001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.202670+0000","last_change":"2024-09-17T04:25:45.194470+0000","last_active":"2024-09-17T04:25:46.202670+0000","last_peered":"2024-09-17T04:25:46.202670+0000","last_clean":"2024-09-17T04:25:46.202670+0000","last_became_active":"2024-09-17T04:25:45.194140+0000","last_became_peered":"2024-09-17T04:25:45.194140+0000","last_unstale":"2024-09-17T04:25:46.202670+0000","last_undegraded":"2024-09-17T04:25:46.202670+0000","last_fullsized":"2024-09-17T04:25:46.202670+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:25.061387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.262104+0000","last_change":"2024-09-17T04:25:44.196861+0000","last_active":"2024-09-17T04:25:46.262104+0000","last_peered":"2024-09-17T04:25:46.262104+0000","last_clean":"2024-09-17T04:25:46.262104+0000","last_became_active":"2024-09-17T04:25:44.196498+0000","last_became_peered":"2024-09-17T04:25:44.196498+0000","last_unstale":"2024-09-17T04:25:46.262104+0000","last_undegraded":"2024-09-17T04:25:46.262104+0000","last_fullsized":"2024-09-17T04:25:46.262104+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:44:55.270620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.261964+0000","last_change":"2024-09-17T04:25:45.201472+0000","last_active":"2024-09-17T04:25:46.261964+0000","last_peered":"2024-09-17T04:25:46.261964+0000","last_clean":"2024-09-17T04:25:46.261964+0000","last_became_active":"2024-09-17T04:25:45.201156+0000","last_became_peered":"2024-09-17T04:25:45.201156+0000","last_unstale":"2024-09-17T04:25:46.261964+0000","last_undegraded":"2024-09-17T04:25:46.261964+0000","last_fullsized":"2024-09-17T04:25:46.261964+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:48:12.273365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.202003+0000","last_change":"2024-09-17T04:25:44.197548+0000","last_active":"2024-09-17T04:25:46.202003+0000","last_peered":"2024-09-17T04:25:46.202003+0000","last_clean":"2024-09-17T04:25:46.202003+0000","last_became_active":"2024-09-17T04:25:44.196782+0000","last_became_peered":"2024-09-17T04:25:44.196782+0000","last_unstale":"2024-09-17T04:25:46.202003+0000","last_undegraded":"2024-09-17T04:25:46.202003+0000","last_fullsized":"2024-09-17T04:25:46.202003+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:48:57.390322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.260800+0000","last_change":"2024-09-17T04:25:45.193460+0000","last_active":"2024-09-17T04:25:46.260800+0000","last_peered":"2024-09-17T04:25:46.260800+0000","last_clean":"2024-09-17T04:25:46.260800+0000","last_became_active":"2024-09-17T04:25:45.193161+0000","last_became_peered":"2024-09-17T04:25:45.193161+0000","last_unstale":"2024-09-17T04:25:46.260800+0000","last_undegraded":"2024-09-17T04:25:46.260800+0000","last_fullsized":"2024-09-17T04:25:46.260800+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:05:18.935322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"47'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203706+0000","last_change":"2024-09-17T04:25:44.197016+0000","last_active":"2024-09-17T04:25:46.203706+0000","last_peered":"2024-09-17T04:25:46.203706+0000","last_clean":"2024-09-17T04:25:46.203706+0000","last_became_active":"2024-09-17T04:25:44.196699+0000","last_became_peered":"2024-09-17T04:25:44.196699+0000","last_unstale":"2024-09-17T04:25:46.203706+0000","last_undegraded":"2024-09-17T04:25:46.203706+0000","last_fullsized":"2024-09-17T04:25:46.203706+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:01.401433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.260356+0000","last_change":"2024-09-17T04:25:45.217157+0000","last_active":"2024-09-17T04:25:46.260356+0000","last_peered":"2024-09-17T04:25:46.260356+0000","last_clean":"2024-09-17T04:25:46.260356+0000","last_became_active":"2024-09-17T04:25:45.215435+0000","last_became_peered":"2024-09-17T04:25:45.215435+0000","last_unstale":"2024-09-17T04:25:46.260356+0000","last_undegraded":"2024-09-17T04:25:46.260356+0000","last_fullsized":"2024-09-17T04:25:46.260356+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:30:55.393999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"47'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.260874+0000","last_change":"2024-09-17T04:25:44.187607+0000","last_active":"2024-09-17T04:25:46.260874+0000","last_peered":"2024-09-17T04:25:46.260874+0000","last_clean":"2024-09-17T04:25:46.260874+0000","last_became_active":"2024-09-17T04:25:44.187222+0000","last_became_peered":"2024-09-17T04:25:44.187222+0000","last_unstale":"2024-09-17T04:25:46.260874+0000","last_undegraded":"2024-09-17T04:25:46.260874+0000","last_fullsized":"2024-09-17T04:25:46.260874+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:09:22.617225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.260151+0000","last_change":"2024-09-17T04:25:45.194539+0000","last_active":"2024-09-17T04:25:46.260151+0000","last_peered":"2024-09-17T04:25:46.260151+0000","last_clean":"2024-09-17T04:25:46.260151+0000","last_became_active":"2024-09-17T04:25:45.194205+0000","last_became_peered":"2024-09-17T04:25:45.194205+0000","last_unstale":"2024-09-17T04:25:46.260151+0000","last_undegraded":"2024-09-17T04:25:46.260151+0000","last_fullsized":"2024-09-17T04:25:46.260151+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:07.907798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.201082+0000","last_change":"2024-09-17T04:25:44.194393+0000","last_active":"2024-09-17T04:25:46.201082+0000","last_peered":"2024-09-17T04:25:46.201082+0000","last_clean":"2024-09-17T04:25:46.201082+0000","last_became_active":"2024-09-17T04:25:44.193980+0000","last_became_peered":"2024-09-17T04:25:44.193980+0000","last_unstale":"2024-09-17T04:25:46.201082+0000","last_undegraded":"2024-09-17T04:25:46.201082+0000","last_fullsized":"2024-09-17T04:25:46.201082+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:29.200803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.261113+0000","last_change":"2024-09-17T04:25:45.198818+0000","last_active":"2024-09-17T04:25:46.261113+0000","last_peered":"2024-09-17T04:25:46.261113+0000","last_clean":"2024-09-17T04:25:46.261113+0000","last_became_active":"2024-09-17T04:25:45.198163+0000","last_became_peered":"2024-09-17T04:25:45.198163+0000","last_unstale":"2024-09-17T04:25:46.261113+0000","last_undegraded":"2024-09-17T04:25:46.261113+0000","last_fullsized":"2024-09-17T04:25:46.261113+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:44:13.652211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.260279+0000","last_change":"2024-09-17T04:25:44.195546+0000","last_active":"2024-09-17T04:25:46.260279+0000","last_peered":"2024-09-17T04:25:46.260279+0000","last_clean":"2024-09-17T04:25:46.260279+0000","last_became_active":"2024-09-17T04:25:44.194972+0000","last_became_peered":"2024-09-17T04:25:44.194972+0000","last_unstale":"2024-09-17T04:25:46.260279+0000","last_undegraded":"2024-09-17T04:25:46.260279+0000","last_fullsized":"2024-09-17T04:25:46.260279+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:01.168678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.261844+0000","last_change":"2024-09-17T04:25:45.198886+0000","last_active":"2024-09-17T04:25:46.261844+0000","last_peered":"2024-09-17T04:25:46.261844+0000","last_clean":"2024-09-17T04:25:46.261844+0000","last_became_active":"2024-09-17T04:25:45.197722+0000","last_became_peered":"2024-09-17T04:25:45.197722+0000","last_unstale":"2024-09-17T04:25:46.261844+0000","last_undegraded":"2024-09-17T04:25:46.261844+0000","last_fullsized":"2024-09-17T04:25:46.261844+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:03.544544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.262134+0000","last_change":"2024-09-17T04:25:44.187294+0000","last_active":"2024-09-17T04:25:46.262134+0000","last_peered":"2024-09-17T04:25:46.262134+0000","last_clean":"2024-09-17T04:25:46.262134+0000","last_became_active":"2024-09-17T04:25:44.186930+0000","last_became_peered":"2024-09-17T04:25:44.186930+0000","last_unstale":"2024-09-17T04:25:46.262134+0000","last_undegraded":"2024-09-17T04:25:46.262134+0000","last_fullsized":"2024-09-17T04:25:46.262134+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:28.838365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203024+0000","last_change":"2024-09-17T04:25:45.189003+0000","last_active":"2024-09-17T04:25:46.203024+0000","last_peered":"2024-09-17T04:25:46.203024+0000","last_clean":"2024-09-17T04:25:46.203024+0000","last_became_active":"2024-09-17T04:25:45.188649+0000","last_became_peered":"2024-09-17T04:25:45.188649+0000","last_unstale":"2024-09-17T04:25:46.203024+0000","last_undegraded":"2024-09-17T04:25:46.203024+0000","last_fullsized":"2024-09-17T04:25:46.203024+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:41.545131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.261363+0000","last_change":"2024-09-17T04:25:44.201008+0000","last_active":"2024-09-17T04:25:46.261363+0000","last_peered":"2024-09-17T04:25:46.261363+0000","last_clean":"2024-09-17T04:25:46.261363+0000","last_became_active":"2024-09-17T04:25:44.200714+0000","last_became_peered":"2024-09-17T04:25:44.200714+0000","last_unstale":"2024-09-17T04:25:46.261363+0000","last_undegraded":"2024-09-17T04:25:46.261363+0000","last_fullsized":"2024-09-17T04:25:46.261363+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:33:11.839322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203780+0000","last_change":"2024-09-17T04:25:45.213566+0000","last_active":"2024-09-17T04:25:46.203780+0000","last_peered":"2024-09-17T04:25:46.203780+0000","last_clean":"2024-09-17T04:25:46.203780+0000","last_became_active":"2024-09-17T04:25:45.213247+0000","last_became_peered":"2024-09-17T04:25:45.213247+0000","last_unstale":"2024-09-17T04:25:46.203780+0000","last_undegraded":"2024-09-17T04:25:46.203780+0000","last_fullsized":"2024-09-17T04:25:46.203780+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:40.700608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"47'3","reported_seq":18,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.201744+0000","last_change":"2024-09-17T04:25:44.205387+0000","last_active":"2024-09-17T04:25:46.201744+0000","last_peered":"2024-09-17T04:25:46.201744+0000","last_clean":"2024-09-17T04:25:46.201744+0000","last_became_active":"2024-09-17T04:25:44.205013+0000","last_became_peered":"2024-09-17T04:25:44.205013+0000","last_unstale":"2024-09-17T04:25:46.201744+0000","last_undegraded":"2024-09-17T04:25:46.201744+0000","last_fullsized":"2024-09-17T04:25:46.201744+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:30:54.382626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.201436+0000","last_change":"2024-09-17T04:25:45.204365+0000","last_active":"2024-09-17T04:25:46.201436+0000","last_peered":"2024-09-17T04:25:46.201436+0000","last_clean":"2024-09-17T04:25:46.201436+0000","last_became_active":"2024-09-17T04:25:45.204041+0000","last_became_peered":"2024-09-17T04:25:45.204041+0000","last_unstale":"2024-09-17T04:25:46.201436+0000","last_undegraded":"2024-09-17T04:25:46.201436+0000","last_fullsized":"2024-09-17T04:25:46.201436+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:42:50.591290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"47'6","reported_seq":21,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.262545+0000","last_change":"2024-09-17T04:25:44.184379+0000","last_active":"2024-09-17T04:25:46.262545+0000","last_peered":"2024-09-17T04:25:46.262545+0000","last_clean":"2024-09-17T04:25:46.262545+0000","last_became_active":"2024-09-17T04:25:44.184085+0000","last_became_peered":"2024-09-17T04:25:44.184085+0000","last_unstale":"2024-09-17T04:25:46.262545+0000","last_undegraded":"2024-09-17T04:25:46.262545+0000","last_fullsized":"2024-09-17T04:25:46.262545+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:39.399721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.260902+0000","last_change":"2024-09-17T04:25:45.204605+0000","last_active":"2024-09-17T04:25:46.260902+0000","last_peered":"2024-09-17T04:25:46.260902+0000","last_clean":"2024-09-17T04:25:46.260902+0000","last_became_active":"2024-09-17T04:25:45.204290+0000","last_became_peered":"2024-09-17T04:25:45.204290+0000","last_unstale":"2024-09-17T04:25:46.260902+0000","last_undegraded":"2024-09-17T04:25:46.260902+0000","last_fullsized":"2024-09-17T04:25:46.260902+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:58:03.267870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203858+0000","last_change":"2024-09-17T04:25:44.195070+0000","last_active":"2024-09-17T04:25:46.203858+0000","last_peered":"2024-09-17T04:25:46.203858+0000","last_clean":"2024-09-17T04:25:46.203858+0000","last_became_active":"2024-09-17T04:25:44.194320+0000","last_became_peered":"2024-09-17T04:25:44.194320+0000","last_unstale":"2024-09-17T04:25:46.203858+0000","last_undegraded":"2024-09-17T04:25:46.203858+0000","last_fullsized":"2024-09-17T04:25:46.203858+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:04.486996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.260411+0000","last_change":"2024-09-17T04:25:45.198230+0000","last_active":"2024-09-17T04:25:46.260411+0000","last_peered":"2024-09-17T04:25:46.260411+0000","last_clean":"2024-09-17T04:25:46.260411+0000","last_became_active":"2024-09-17T04:25:45.197643+0000","last_became_peered":"2024-09-17T04:25:45.197643+0000","last_unstale":"2024-09-17T04:25:46.260411+0000","last_undegraded":"2024-09-17T04:25:46.260411+0000","last_fullsized":"2024-09-17T04:25:46.260411+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:03.544544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.201439+0000","last_change":"2024-09-17T04:25:44.195623+0000","last_active":"2024-09-17T04:25:46.201439+0000","last_peered":"2024-09-17T04:25:46.201439+0000","last_clean":"2024-09-17T04:25:46.201439+0000","last_became_active":"2024-09-17T04:25:44.195313+0000","last_became_peered":"2024-09-17T04:25:44.195313+0000","last_unstale":"2024-09-17T04:25:46.201439+0000","last_undegraded":"2024-09-17T04:25:46.201439+0000","last_fullsized":"2024-09-17T04:25:46.201439+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:05:17.923948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203085+0000","last_change":"2024-09-17T04:25:45.211494+0000","last_active":"2024-09-17T04:25:46.203085+0000","last_peered":"2024-09-17T04:25:46.203085+0000","last_clean":"2024-09-17T04:25:46.203085+0000","last_became_active":"2024-09-17T04:25:45.209477+0000","last_became_peered":"2024-09-17T04:25:45.209477+0000","last_unstale":"2024-09-17T04:25:46.203085+0000","last_undegraded":"2024-09-17T04:25:46.203085+0000","last_fullsized":"2024-09-17T04:25:46.203085+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:00:57.227021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203252+0000","last_change":"2024-09-17T04:25:44.200589+0000","last_active":"2024-09-17T04:25:46.203252+0000","last_peered":"2024-09-17T04:25:46.203252+0000","last_clean":"2024-09-17T04:25:46.203252+0000","last_became_active":"2024-09-17T04:25:44.199602+0000","last_became_peered":"2024-09-17T04:25:44.199602+0000","last_unstale":"2024-09-17T04:25:46.203252+0000","last_undegraded":"2024-09-17T04:25:46.203252+0000","last_fullsized":"2024-09-17T04:25:46.203252+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:32:43.072948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.202356+0000","last_change":"2024-09-17T04:25:45.191460+0000","last_active":"2024-09-17T04:25:46.202356+0000","last_peered":"2024-09-17T04:25:46.202356+0000","last_clean":"2024-09-17T04:25:46.202356+0000","last_became_active":"2024-09-17T04:25:45.191107+0000","last_became_peered":"2024-09-17T04:25:45.191107+0000","last_unstale":"2024-09-17T04:25:46.202356+0000","last_undegraded":"2024-09-17T04:25:46.202356+0000","last_fullsized":"2024-09-17T04:25:46.202356+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:42:52.532542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"47'8","reported_seq":23,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.260384+0000","last_change":"2024-09-17T04:25:44.198329+0000","last_active":"2024-09-17T04:25:46.260384+0000","last_peered":"2024-09-17T04:25:46.260384+0000","last_clean":"2024-09-17T04:25:46.260384+0000","last_became_active":"2024-09-17T04:25:44.198077+0000","last_became_peered":"2024-09-17T04:25:44.198077+0000","last_unstale":"2024-09-17T04:25:46.260384+0000","last_undegraded":"2024-09-17T04:25:46.260384+0000","last_fullsized":"2024-09-17T04:25:46.260384+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:16:02.372359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.262309+0000","last_change":"2024-09-17T04:25:45.190611+0000","last_active":"2024-09-17T04:25:46.262309+0000","last_peered":"2024-09-17T04:25:46.262309+0000","last_clean":"2024-09-17T04:25:46.262309+0000","last_became_active":"2024-09-17T04:25:45.190430+0000","last_became_peered":"2024-09-17T04:25:45.190430+0000","last_unstale":"2024-09-17T04:25:46.262309+0000","last_undegraded":"2024-09-17T04:25:46.262309+0000","last_fullsized":"2024-09-17T04:25:46.262309+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:12:40.957307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"47'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.262663+0000","last_change":"2024-09-17T04:25:44.188514+0000","last_active":"2024-09-17T04:25:46.262663+0000","last_peered":"2024-09-17T04:25:46.262663+0000","last_clean":"2024-09-17T04:25:46.262663+0000","last_became_active":"2024-09-17T04:25:44.188202+0000","last_became_peered":"2024-09-17T04:25:44.188202+0000","last_unstale":"2024-09-17T04:25:46.262663+0000","last_undegraded":"2024-09-17T04:25:46.262663+0000","last_fullsized":"2024-09-17T04:25:46.262663+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:01:42.523569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.260817+0000","last_change":"2024-09-17T04:25:45.205569+0000","last_active":"2024-09-17T04:25:46.260817+0000","last_peered":"2024-09-17T04:25:46.260817+0000","last_clean":"2024-09-17T04:25:46.260817+0000","last_became_active":"2024-09-17T04:25:45.205185+0000","last_became_peered":"2024-09-17T04:25:45.205185+0000","last_unstale":"2024-09-17T04:25:46.260817+0000","last_undegraded":"2024-09-17T04:25:46.260817+0000","last_fullsized":"2024-09-17T04:25:46.260817+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:18.006921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.261913+0000","last_change":"2024-09-17T04:25:44.194147+0000","last_active":"2024-09-17T04:25:46.261913+0000","last_peered":"2024-09-17T04:25:46.261913+0000","last_clean":"2024-09-17T04:25:46.261913+0000","last_became_active":"2024-09-17T04:25:44.193814+0000","last_became_peered":"2024-09-17T04:25:44.193814+0000","last_unstale":"2024-09-17T04:25:46.261913+0000","last_undegraded":"2024-09-17T04:25:46.261913+0000","last_fullsized":"2024-09-17T04:25:46.261913+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:16.947749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.261719+0000","last_change":"2024-09-17T04:25:45.208794+0000","last_active":"2024-09-17T04:25:46.261719+0000","last_peered":"2024-09-17T04:25:46.261719+0000","last_clean":"2024-09-17T04:25:46.261719+0000","last_became_active":"2024-09-17T04:25:45.208466+0000","last_became_peered":"2024-09-17T04:25:45.208466+0000","last_unstale":"2024-09-17T04:25:46.261719+0000","last_undegraded":"2024-09-17T04:25:46.261719+0000","last_fullsized":"2024-09-17T04:25:46.261719+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:12:11.633340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"47'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.202184+0000","last_change":"2024-09-17T04:25:44.196200+0000","last_active":"2024-09-17T04:25:46.202184+0000","last_peered":"2024-09-17T04:25:46.202184+0000","last_clean":"2024-09-17T04:25:46.202184+0000","last_became_active":"2024-09-17T04:25:44.192913+0000","last_became_peered":"2024-09-17T04:25:44.192913+0000","last_unstale":"2024-09-17T04:25:46.202184+0000","last_undegraded":"2024-09-17T04:25:46.202184+0000","last_fullsized":"2024-09-17T04:25:46.202184+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:57:38.207914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.261347+0000","last_change":"2024-09-17T04:25:45.211613+0000","last_active":"2024-09-17T04:25:46.261347+0000","last_peered":"2024-09-17T04:25:46.261347+0000","last_clean":"2024-09-17T04:25:46.261347+0000","last_became_active":"2024-09-17T04:25:45.210432+0000","last_became_peered":"2024-09-17T04:25:45.210432+0000","last_unstale":"2024-09-17T04:25:46.261347+0000","last_undegraded":"2024-09-17T04:25:46.261347+0000","last_fullsized":"2024-09-17T04:25:46.261347+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:04.527382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203707+0000","last_change":"2024-09-17T04:25:44.200670+0000","last_active":"2024-09-17T04:25:46.203707+0000","last_peered":"2024-09-17T04:25:46.203707+0000","last_clean":"2024-09-17T04:25:46.203707+0000","last_became_active":"2024-09-17T04:25:44.200382+0000","last_became_peered":"2024-09-17T04:25:44.200382+0000","last_unstale":"2024-09-17T04:25:46.203707+0000","last_undegraded":"2024-09-17T04:25:46.203707+0000","last_fullsized":"2024-09-17T04:25:46.203707+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:15:15.268358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203536+0000","last_change":"2024-09-17T04:25:45.210726+0000","last_active":"2024-09-17T04:25:46.203536+0000","last_peered":"2024-09-17T04:25:46.203536+0000","last_clean":"2024-09-17T04:25:46.203536+0000","last_became_active":"2024-09-17T04:25:45.210304+0000","last_became_peered":"2024-09-17T04:25:45.210304+0000","last_unstale":"2024-09-17T04:25:46.203536+0000","last_undegraded":"2024-09-17T04:25:46.203536+0000","last_fullsized":"2024-09-17T04:25:46.203536+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:02.703247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203968+0000","last_change":"2024-09-17T04:25:44.198264+0000","last_active":"2024-09-17T04:25:46.203968+0000","last_peered":"2024-09-17T04:25:46.203968+0000","last_clean":"2024-09-17T04:25:46.203968+0000","last_became_active":"2024-09-17T04:25:44.197966+0000","last_became_peered":"2024-09-17T04:25:44.197966+0000","last_unstale":"2024-09-17T04:25:46.203968+0000","last_undegraded":"2024-09-17T04:25:46.203968+0000","last_fullsized":"2024-09-17T04:25:46.203968+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:02.720620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.261587+0000","last_change":"2024-09-17T04:25:45.209203+0000","last_active":"2024-09-17T04:25:46.261587+0000","last_peered":"2024-09-17T04:25:46.261587+0000","last_clean":"2024-09-17T04:25:46.261587+0000","last_became_active":"2024-09-17T04:25:45.208270+0000","last_became_peered":"2024-09-17T04:25:45.208270+0000","last_unstale":"2024-09-17T04:25:46.261587+0000","last_undegraded":"2024-09-17T04:25:46.261587+0000","last_fullsized":"2024-09-17T04:25:46.261587+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:31:15.211597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.201311+0000","last_change":"2024-09-17T04:25:44.200687+0000","last_active":"2024-09-17T04:25:46.201311+0000","last_peered":"2024-09-17T04:25:46.201311+0000","last_clean":"2024-09-17T04:25:46.201311+0000","last_became_active":"2024-09-17T04:25:44.200363+0000","last_became_peered":"2024-09-17T04:25:44.200363+0000","last_unstale":"2024-09-17T04:25:46.201311+0000","last_undegraded":"2024-09-17T04:25:46.201311+0000","last_fullsized":"2024-09-17T04:25:46.201311+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:12:10.621967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.261652+0000","last_change":"2024-09-17T04:25:45.197466+0000","last_active":"2024-09-17T04:25:46.261652+0000","last_peered":"2024-09-17T04:25:46.261652+0000","last_clean":"2024-09-17T04:25:46.261652+0000","last_became_active":"2024-09-17T04:25:45.191930+0000","last_became_peered":"2024-09-17T04:25:45.191930+0000","last_unstale":"2024-09-17T04:25:46.261652+0000","last_undegraded":"2024-09-17T04:25:46.261652+0000","last_fullsized":"2024-09-17T04:25:46.261652+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:48.491808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.261860+0000","last_change":"2024-09-17T04:25:44.197375+0000","last_active":"2024-09-17T04:25:46.261860+0000","last_peered":"2024-09-17T04:25:46.261860+0000","last_clean":"2024-09-17T04:25:46.261860+0000","last_became_active":"2024-09-17T04:25:44.197001+0000","last_became_peered":"2024-09-17T04:25:44.197001+0000","last_unstale":"2024-09-17T04:25:46.261860+0000","last_undegraded":"2024-09-17T04:25:46.261860+0000","last_fullsized":"2024-09-17T04:25:46.261860+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:18:50.452793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.262034+0000","last_change":"2024-09-17T04:25:45.204213+0000","last_active":"2024-09-17T04:25:46.262034+0000","last_peered":"2024-09-17T04:25:46.262034+0000","last_clean":"2024-09-17T04:25:46.262034+0000","last_became_active":"2024-09-17T04:25:45.203933+0000","last_became_peered":"2024-09-17T04:25:45.203933+0000","last_unstale":"2024-09-17T04:25:46.262034+0000","last_undegraded":"2024-09-17T04:25:46.262034+0000","last_fullsized":"2024-09-17T04:25:46.262034+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:42:52.532542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.200941+0000","last_change":"2024-09-17T04:25:44.194913+0000","last_active":"2024-09-17T04:25:46.200941+0000","last_peered":"2024-09-17T04:25:46.200941+0000","last_clean":"2024-09-17T04:25:46.200941+0000","last_became_active":"2024-09-17T04:25:44.194498+0000","last_became_peered":"2024-09-17T04:25:44.194498+0000","last_unstale":"2024-09-17T04:25:46.200941+0000","last_undegraded":"2024-09-17T04:25:46.200941+0000","last_fullsized":"2024-09-17T04:25:46.200941+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:02.533170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.261253+0000","last_change":"2024-09-17T04:25:45.208692+0000","last_active":"2024-09-17T04:25:46.261253+0000","last_peered":"2024-09-17T04:25:46.261253+0000","last_clean":"2024-09-17T04:25:46.261253+0000","last_became_active":"2024-09-17T04:25:45.207545+0000","last_became_peered":"2024-09-17T04:25:45.207545+0000","last_unstale":"2024-09-17T04:25:46.261253+0000","last_undegraded":"2024-09-17T04:25:46.261253+0000","last_fullsized":"2024-09-17T04:25:46.261253+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:12.006700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203861+0000","last_change":"2024-09-17T04:25:44.191599+0000","last_active":"2024-09-17T04:25:46.203861+0000","last_peered":"2024-09-17T04:25:46.203861+0000","last_clean":"2024-09-17T04:25:46.203861+0000","last_became_active":"2024-09-17T04:25:44.191038+0000","last_became_peered":"2024-09-17T04:25:44.191038+0000","last_unstale":"2024-09-17T04:25:46.203861+0000","last_undegraded":"2024-09-17T04:25:46.203861+0000","last_fullsized":"2024-09-17T04:25:46.203861+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:23:16.825181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.260697+0000","last_change":"2024-09-17T04:25:45.214782+0000","last_active":"2024-09-17T04:25:46.260697+0000","last_peered":"2024-09-17T04:25:46.260697+0000","last_clean":"2024-09-17T04:25:46.260697+0000","last_became_active":"2024-09-17T04:25:45.214468+0000","last_became_peered":"2024-09-17T04:25:45.214468+0000","last_unstale":"2024-09-17T04:25:46.260697+0000","last_undegraded":"2024-09-17T04:25:46.260697+0000","last_fullsized":"2024-09-17T04:25:46.260697+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:24:02.114972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.201856+0000","last_change":"2024-09-17T04:25:44.204903+0000","last_active":"2024-09-17T04:25:46.201856+0000","last_peered":"2024-09-17T04:25:46.201856+0000","last_clean":"2024-09-17T04:25:46.201856+0000","last_became_active":"2024-09-17T04:25:44.203858+0000","last_became_peered":"2024-09-17T04:25:44.203858+0000","last_unstale":"2024-09-17T04:25:46.201856+0000","last_undegraded":"2024-09-17T04:25:46.201856+0000","last_fullsized":"2024-09-17T04:25:46.201856+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:04:49.113297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.201675+0000","last_change":"2024-09-17T04:25:45.208288+0000","last_active":"2024-09-17T04:25:46.201675+0000","last_peered":"2024-09-17T04:25:46.201675+0000","last_clean":"2024-09-17T04:25:46.201675+0000","last_became_active":"2024-09-17T04:25:45.206421+0000","last_became_peered":"2024-09-17T04:25:45.206421+0000","last_unstale":"2024-09-17T04:25:46.201675+0000","last_undegraded":"2024-09-17T04:25:46.201675+0000","last_fullsized":"2024-09-17T04:25:46.201675+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:04.440015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.261730+0000","last_change":"2024-09-17T04:25:44.192593+0000","last_active":"2024-09-17T04:25:46.261730+0000","last_peered":"2024-09-17T04:25:46.261730+0000","last_clean":"2024-09-17T04:25:46.261730+0000","last_became_active":"2024-09-17T04:25:44.190442+0000","last_became_peered":"2024-09-17T04:25:44.190442+0000","last_unstale":"2024-09-17T04:25:46.261730+0000","last_undegraded":"2024-09-17T04:25:46.261730+0000","last_fullsized":"2024-09-17T04:25:46.261730+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:48.599001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203974+0000","last_change":"2024-09-17T04:25:45.206724+0000","last_active":"2024-09-17T04:25:46.203974+0000","last_peered":"2024-09-17T04:25:46.203974+0000","last_clean":"2024-09-17T04:25:46.203974+0000","last_became_active":"2024-09-17T04:25:45.205733+0000","last_became_peered":"2024-09-17T04:25:45.205733+0000","last_unstale":"2024-09-17T04:25:46.203974+0000","last_undegraded":"2024-09-17T04:25:46.203974+0000","last_fullsized":"2024-09-17T04:25:46.203974+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:15:30.185576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.201708+0000","last_change":"2024-09-17T04:25:44.204187+0000","last_active":"2024-09-17T04:25:46.201708+0000","last_peered":"2024-09-17T04:25:46.201708+0000","last_clean":"2024-09-17T04:25:46.201708+0000","last_became_active":"2024-09-17T04:25:44.203803+0000","last_became_peered":"2024-09-17T04:25:44.203803+0000","last_unstale":"2024-09-17T04:25:46.201708+0000","last_undegraded":"2024-09-17T04:25:46.201708+0000","last_fullsized":"2024-09-17T04:25:46.201708+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:06.896425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.203585+0000","last_change":"2024-09-17T04:25:45.210940+0000","last_active":"2024-09-17T04:25:46.203585+0000","last_peered":"2024-09-17T04:25:46.203585+0000","last_clean":"2024-09-17T04:25:46.203585+0000","last_became_active":"2024-09-17T04:25:45.210565+0000","last_became_peered":"2024-09-17T04:25:45.210565+0000","last_unstale":"2024-09-17T04:25:46.203585+0000","last_undegraded":"2024-09-17T04:25:46.203585+0000","last_fullsized":"2024-09-17T04:25:46.203585+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:40.310997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.260985+0000","last_change":"2024-09-17T04:25:44.188359+0000","last_active":"2024-09-17T04:25:46.260985+0000","last_peered":"2024-09-17T04:25:46.260985+0000","last_clean":"2024-09-17T04:25:46.260985+0000","last_became_active":"2024-09-17T04:25:44.188056+0000","last_became_peered":"2024-09-17T04:25:44.188056+0000","last_unstale":"2024-09-17T04:25:46.260985+0000","last_undegraded":"2024-09-17T04:25:46.260985+0000","last_fullsized":"2024-09-17T04:25:46.260985+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:44:55.270620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.201114+0000","last_change":"2024-09-17T04:25:45.202692+0000","last_active":"2024-09-17T04:25:46.201114+0000","last_peered":"2024-09-17T04:25:46.201114+0000","last_clean":"2024-09-17T04:25:46.201114+0000","last_became_active":"2024-09-17T04:25:45.202369+0000","last_became_peered":"2024-09-17T04:25:45.202369+0000","last_unstale":"2024-09-17T04:25:46.201114+0000","last_undegraded":"2024-09-17T04:25:46.201114+0000","last_fullsized":"2024-09-17T04:25:46.201114+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:56.064236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.261090+0000","last_change":"2024-09-17T04:25:44.203478+0000","last_active":"2024-09-17T04:25:46.261090+0000","last_peered":"2024-09-17T04:25:46.261090+0000","last_clean":"2024-09-17T04:25:46.261090+0000","last_became_active":"2024-09-17T04:25:44.203088+0000","last_became_peered":"2024-09-17T04:25:44.203088+0000","last_unstale":"2024-09-17T04:25:46.261090+0000","last_undegraded":"2024-09-17T04:25:46.261090+0000","last_fullsized":"2024-09-17T04:25:46.261090+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:16.947749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.260517+0000","last_change":"2024-09-17T04:25:45.207529+0000","last_active":"2024-09-17T04:25:46.260517+0000","last_peered":"2024-09-17T04:25:46.260517+0000","last_clean":"2024-09-17T04:25:46.260517+0000","last_became_active":"2024-09-17T04:25:45.206279+0000","last_became_peered":"2024-09-17T04:25:45.206279+0000","last_unstale":"2024-09-17T04:25:46.260517+0000","last_undegraded":"2024-09-17T04:25:46.260517+0000","last_fullsized":"2024-09-17T04:25:46.260517+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:35:04.594371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.260433+0000","last_change":"2024-09-17T04:25:44.195839+0000","last_active":"2024-09-17T04:25:46.260433+0000","last_peered":"2024-09-17T04:25:46.260433+0000","last_clean":"2024-09-17T04:25:46.260433+0000","last_became_active":"2024-09-17T04:25:44.195257+0000","last_became_peered":"2024-09-17T04:25:44.195257+0000","last_unstale":"2024-09-17T04:25:46.260433+0000","last_undegraded":"2024-09-17T04:25:46.260433+0000","last_fullsized":"2024-09-17T04:25:46.260433+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:04:17.533005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.261247+0000","last_change":"2024-09-17T04:25:45.196772+0000","last_active":"2024-09-17T04:25:46.261247+0000","last_peered":"2024-09-17T04:25:46.261247+0000","last_clean":"2024-09-17T04:25:46.261247+0000","last_became_active":"2024-09-17T04:25:45.196213+0000","last_became_peered":"2024-09-17T04:25:45.196213+0000","last_unstale":"2024-09-17T04:25:46.261247+0000","last_undegraded":"2024-09-17T04:25:46.261247+0000","last_fullsized":"2024-09-17T04:25:46.261247+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:44.162204+0000","last_clean_scrub_stamp":"2024-09-17T04:25:44.162204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:42:50.591290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.202977+0000","last_change":"2024-09-17T04:25:44.189355+0000","last_active":"2024-09-17T04:25:46.202977+0000","last_peered":"2024-09-17T04:25:46.202977+0000","last_clean":"2024-09-17T04:25:46.202977+0000","last_became_active":"2024-09-17T04:25:44.188098+0000","last_became_peered":"2024-09-17T04:25:44.188098+0000","last_unstale":"2024-09-17T04:25:46.202977+0000","last_undegraded":"2024-09-17T04:25:46.202977+0000","last_fullsized":"2024-09-17T04:25:46.202977+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:25:43.150831+0000","last_clean_scrub_stamp":"2024-09-17T04:25:43.150831+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:49:37.337909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":157,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T04:25:46.201076+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:25:46.201076+0000","last_peered":"2024-09-17T04:25:46.201076+0000","last_clean":"2024-09-17T04:25:46.201076+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:25:46.201076+0000","last_undegraded":"2024-09-17T04:25:46.201076+0000","last_fullsized":"2024-09-17T04:25:46.201076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476762,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28160,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343680,"statfs":{"total":96636764160,"available":96607928320,"internally_reserved":0,"allocated":1150976,"data_stored":814474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476761,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":561152,"data_stored":224688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476762,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":557056,"data_stored":224086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476762,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":565248,"data_stored":225628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509465,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":569344,"data_stored":225650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509464,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":557056,"data_stored":224086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509464,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1150976,"data_stored":814418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542169,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":565248,"data_stored":224654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:25:50.098 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T04:25:50.098 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T04:25:50.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.363+0000 7fef48600640 1 -- 172.21.15.100:0/4102613986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef40145550 msgr2=0x7fef4006fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:50.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.363+0000 7fef48600640 1 --2- 172.21.15.100:0/4102613986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef40145550 0x7fef4006fb40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fef34005fd0 tx=0x7fef3402a670 comp rx=0 tx=0).stop 2024-09-17T04:25:50.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.363+0000 7fef48600640 1 -- 172.21.15.100:0/4102613986 shutdown_connections 2024-09-17T04:25:50.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.363+0000 7fef48600640 1 --2- 172.21.15.100:0/4102613986 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef40145a40 0x7fef40073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:50.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.363+0000 7fef48600640 1 --2- 172.21.15.100:0/4102613986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef40145550 0x7fef4006fb40 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:50.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.363+0000 7fef48600640 1 -- 172.21.15.100:0/4102613986 >> 172.21.15.100:0/4102613986 conn(0x7fef4006d1c0 msgr2=0x7fef4006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:50.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.363+0000 7fef48600640 1 -- 172.21.15.100:0/4102613986 shutdown_connections 2024-09-17T04:25:50.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.363+0000 7fef48600640 1 -- 172.21.15.100:0/4102613986 wait complete. 2024-09-17T04:25:50.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.366+0000 7fef48600640 1 Processor -- start 2024-09-17T04:25:50.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.366+0000 7fef48600640 1 -- start start 2024-09-17T04:25:50.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.367+0000 7fef48600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef40073a80 0x7fef400b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:50.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.367+0000 7fef48600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef40145550 0x7fef400af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:50.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.367+0000 7fef48600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef40145a40 0x7fef401f9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:50.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.367+0000 7fef48600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fef40160a30 con 0x7fef40145a40 2024-09-17T04:25:50.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.367+0000 7fef48600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fef400ce940 con 0x7fef40073a80 2024-09-17T04:25:50.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.367+0000 7fef48600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fef40160bb0 con 0x7fef40145550 2024-09-17T04:25:50.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.368+0000 7fef46600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef40145a40 0x7fef401f9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:50.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.368+0000 7fef46600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef40145a40 0x7fef401f9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55850/0 (socket says 172.21.15.100:55850) 2024-09-17T04:25:50.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.368+0000 7fef46600640 1 -- 172.21.15.100:0/2624587722 learned_addr learned my addr 172.21.15.100:0/2624587722 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:50.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.368+0000 7fef45c00640 1 --2- 172.21.15.100:0/2624587722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef40073a80 0x7fef400b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:50.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.368+0000 7fef45200640 1 --2- 172.21.15.100:0/2624587722 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef40145550 0x7fef400af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:50.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.368+0000 7fef46600640 1 -- 172.21.15.100:0/2624587722 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef40145550 msgr2=0x7fef400af1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:50.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.368+0000 7fef46600640 1 --2- 172.21.15.100:0/2624587722 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef40145550 0x7fef400af1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:50.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.368+0000 7fef46600640 1 -- 172.21.15.100:0/2624587722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef40073a80 msgr2=0x7fef400b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:50.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.368+0000 7fef46600640 1 --2- 172.21.15.100:0/2624587722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef40073a80 0x7fef400b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:50.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.368+0000 7fef46600640 1 -- 172.21.15.100:0/2624587722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef401fa060 con 0x7fef40145a40 2024-09-17T04:25:50.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.368+0000 7fef45200640 1 --2- 172.21.15.100:0/2624587722 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef40145550 0x7fef400af1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:50.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.368+0000 7fef46600640 1 --2- 172.21.15.100:0/2624587722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef40145a40 0x7fef401f9170 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fef3800c700 tx=0x7fef3800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:50.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.369+0000 7fef3d600640 1 -- 172.21.15.100:0/2624587722 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef38003b70 con 0x7fef40145a40 2024-09-17T04:25:50.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.369+0000 7fef3d600640 1 -- 172.21.15.100:0/2624587722 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef38003d10 con 0x7fef40145a40 2024-09-17T04:25:50.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.369+0000 7fef48600640 1 -- 172.21.15.100:0/2624587722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef400b4430 con 0x7fef40145a40 2024-09-17T04:25:50.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.369+0000 7fef3d600640 1 -- 172.21.15.100:0/2624587722 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3800aab0 con 0x7fef40145a40 2024-09-17T04:25:50.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.369+0000 7fef48600640 1 -- 172.21.15.100:0/2624587722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef4015dc20 con 0x7fef40145a40 2024-09-17T04:25:50.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.370+0000 7fef48600640 1 -- 172.21.15.100:0/2624587722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef4015da40 con 0x7fef40145a40 2024-09-17T04:25:50.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.371+0000 7fef3d600640 1 -- 172.21.15.100:0/2624587722 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fef3800ac50 con 0x7fef40145a40 2024-09-17T04:25:50.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.372+0000 7fef3d600640 1 --2- 172.21.15.100:0/2624587722 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fef20078d40 0x7fef2007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:50.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.372+0000 7fef3d600640 1 -- 172.21.15.100:0/2624587722 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fef3809af10 con 0x7fef40145a40 2024-09-17T04:25:50.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.373+0000 7fef45c00640 1 --2- 172.21.15.100:0/2624587722 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fef20078d40 0x7fef2007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:50.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.374+0000 7fef45c00640 1 --2- 172.21.15.100:0/2624587722 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fef20078d40 0x7fef2007b250 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fef34033000 tx=0x7fef34004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:50.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.377+0000 7fef3d600640 1 -- 172.21.15.100:0/2624587722 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fef4015da40 con 0x7fef40145a40 2024-09-17T04:25:50.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.625+0000 7fef48600640 1 -- 172.21.15.100:0/2624587722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fef401610d0 con 0x7fef40145a40 2024-09-17T04:25:50.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.628+0000 7fef3d600640 1 -- 172.21.15.100:0/2624587722 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7fef40073e90 con 0x7fef40145a40 2024-09-17T04:25:50.629 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-09-17T04:25:50.629 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAsBOlmfYg9IBAABP1uRKGOugrA47VxwRLkjA== 2024-09-17T04:25:50.629 INFO:teuthology.orchestra.run.smithi100.stdout: caps mds = "allow" 2024-09-17T04:25:50.630 INFO:teuthology.orchestra.run.smithi100.stdout: caps mon = "allow r" 2024-09-17T04:25:50.630 INFO:teuthology.orchestra.run.smithi100.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T04:25:50.630 INFO:teuthology.orchestra.run.smithi100.stderr:updated caps for client.0 2024-09-17T04:25:50.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.631+0000 7fef48600640 1 -- 172.21.15.100:0/2624587722 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fef20078d40 msgr2=0x7fef2007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:50.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.631+0000 7fef48600640 1 --2- 172.21.15.100:0/2624587722 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fef20078d40 0x7fef2007b250 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fef34033000 tx=0x7fef34004730 comp rx=0 tx=0).stop 2024-09-17T04:25:50.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.631+0000 7fef48600640 1 -- 172.21.15.100:0/2624587722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef40145a40 msgr2=0x7fef401f9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:50.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.631+0000 7fef48600640 1 --2- 172.21.15.100:0/2624587722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef40145a40 0x7fef401f9170 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fef3800c700 tx=0x7fef3800cbc0 comp rx=0 tx=0).stop 2024-09-17T04:25:50.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.632+0000 7fef48600640 1 -- 172.21.15.100:0/2624587722 shutdown_connections 2024-09-17T04:25:50.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.632+0000 7fef48600640 1 --2- 172.21.15.100:0/2624587722 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fef20078d40 0x7fef2007b250 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:50.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.632+0000 7fef48600640 1 --2- 172.21.15.100:0/2624587722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef40145a40 0x7fef401f9170 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:50.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.632+0000 7fef48600640 1 --2- 172.21.15.100:0/2624587722 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef40145550 0x7fef400af1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:50.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.632+0000 7fef48600640 1 --2- 172.21.15.100:0/2624587722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef40073a80 0x7fef400b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:50.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.632+0000 7fef48600640 1 -- 172.21.15.100:0/2624587722 >> 172.21.15.100:0/2624587722 conn(0x7fef4006d1c0 msgr2=0x7fef4006f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:50.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.632+0000 7fef48600640 1 -- 172.21.15.100:0/2624587722 shutdown_connections 2024-09-17T04:25:50.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.632+0000 7fef48600640 1 -- 172.21.15.100:0/2624587722 wait complete. 2024-09-17T04:25:50.654 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:25:50.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.874+0000 7fa830e00640 1 -- 172.21.15.100:0/3224055411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa82c1518d0 msgr2=0x7fa82c06f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:50.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.874+0000 7fa830e00640 1 --2- 172.21.15.100:0/3224055411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa82c1518d0 0x7fa82c06f9f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0b4ed0 tx=0x7fa81c02a560 comp rx=0 tx=0).stop 2024-09-17T04:25:50.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.874+0000 7fa830e00640 1 -- 172.21.15.100:0/3224055411 shutdown_connections 2024-09-17T04:25:50.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.874+0000 7fa830e00640 1 --2- 172.21.15.100:0/3224055411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa82c15e3d0 0x7fa82c15e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:50.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.874+0000 7fa830e00640 1 --2- 172.21.15.100:0/3224055411 >> v2:172.21.15.184:3300/0 conn(0x7fa82c0735c0 0x7fa82c0739d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:50.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.874+0000 7fa830e00640 1 --2- 172.21.15.100:0/3224055411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa82c1518d0 0x7fa82c06f9f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:50.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.874+0000 7fa830e00640 1 -- 172.21.15.100:0/3224055411 >> 172.21.15.100:0/3224055411 conn(0x7fa82c0b1e40 msgr2=0x7fa82c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:50.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.874+0000 7fa830e00640 1 -- 172.21.15.100:0/3224055411 shutdown_connections 2024-09-17T04:25:50.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.874+0000 7fa830e00640 1 -- 172.21.15.100:0/3224055411 wait complete. 2024-09-17T04:25:50.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.876+0000 7fa830e00640 1 Processor -- start 2024-09-17T04:25:50.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.876+0000 7fa830e00640 1 -- start start 2024-09-17T04:25:50.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.878+0000 7fa830e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa82c0735c0 0x7fa82c13c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:50.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.878+0000 7fa830e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa82c1518d0 0x7fa82c13ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:50.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.878+0000 7fa830e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa82c15e3d0 0x7fa82c13efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:50.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.878+0000 7fa830e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c070f50 con 0x7fa82c1518d0 2024-09-17T04:25:50.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.878+0000 7fa830e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c070140 con 0x7fa82c0735c0 2024-09-17T04:25:50.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.878+0000 7fa830e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa82c0710d0 con 0x7fa82c15e3d0 2024-09-17T04:25:50.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.878+0000 7fa82aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa82c1518d0 0x7fa82c13ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:50.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.878+0000 7fa82aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa82c1518d0 0x7fa82c13ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55860/0 (socket says 172.21.15.100:55860) 2024-09-17T04:25:50.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.878+0000 7fa82aa00640 1 -- 172.21.15.100:0/1102969345 learned_addr learned my addr 172.21.15.100:0/1102969345 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:50.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.878+0000 7fa82b400640 1 --2- 172.21.15.100:0/1102969345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa82c0735c0 0x7fa82c13c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:50.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.878+0000 7fa82be00640 1 --2- 172.21.15.100:0/1102969345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa82c15e3d0 0x7fa82c13efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:50.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.878+0000 7fa82aa00640 1 -- 172.21.15.100:0/1102969345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa82c15e3d0 msgr2=0x7fa82c13efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:50.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.878+0000 7fa82aa00640 1 --2- 172.21.15.100:0/1102969345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa82c15e3d0 0x7fa82c13efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:50.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.878+0000 7fa82aa00640 1 -- 172.21.15.100:0/1102969345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa82c0735c0 msgr2=0x7fa82c13c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:50.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.878+0000 7fa82aa00640 1 --2- 172.21.15.100:0/1102969345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa82c0735c0 0x7fa82c13c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:50.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.878+0000 7fa82aa00640 1 -- 172.21.15.100:0/1102969345 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82c13c720 con 0x7fa82c1518d0 2024-09-17T04:25:50.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.879+0000 7fa82aa00640 1 --2- 172.21.15.100:0/1102969345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa82c1518d0 0x7fa82c13ea80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa81800d750 tx=0x7fa81800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:50.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.879+0000 7fa823400640 1 -- 172.21.15.100:0/1102969345 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818017070 con 0x7fa82c1518d0 2024-09-17T04:25:50.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.879+0000 7fa823400640 1 -- 172.21.15.100:0/1102969345 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa818009cc0 con 0x7fa82c1518d0 2024-09-17T04:25:50.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.879+0000 7fa823400640 1 -- 172.21.15.100:0/1102969345 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8180106c0 con 0x7fa82c1518d0 2024-09-17T04:25:50.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.879+0000 7fa830e00640 1 -- 172.21.15.100:0/1102969345 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82c1e79d0 con 0x7fa82c1518d0 2024-09-17T04:25:50.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.879+0000 7fa830e00640 1 -- 172.21.15.100:0/1102969345 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa82c13cd70 con 0x7fa82c1518d0 2024-09-17T04:25:50.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.880+0000 7fa830e00640 1 -- 172.21.15.100:0/1102969345 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa82c1e7740 con 0x7fa82c1518d0 2024-09-17T04:25:50.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.882+0000 7fa823400640 1 -- 172.21.15.100:0/1102969345 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fa81801a070 con 0x7fa82c1518d0 2024-09-17T04:25:50.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.882+0000 7fa823400640 1 --2- 172.21.15.100:0/1102969345 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa80c078d70 0x7fa80c07b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:50.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.883+0000 7fa823400640 1 -- 172.21.15.100:0/1102969345 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fa81809b170 con 0x7fa82c1518d0 2024-09-17T04:25:50.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.883+0000 7fa82b400640 1 --2- 172.21.15.100:0/1102969345 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa80c078d70 0x7fa80c07b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:50.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.884+0000 7fa82b400640 1 --2- 172.21.15.100:0/1102969345 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa80c078d70 0x7fa80c07b280 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa81c033000 tx=0x7fa81c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:50.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:50.887+0000 7fa823400640 1 -- 172.21.15.100:0/1102969345 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa81800de60 con 0x7fa82c1518d0 2024-09-17T04:25:51.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.106+0000 7fa830e00640 1 -- 172.21.15.100:0/1102969345 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa82c15efa0 con 0x7fa82c1518d0 2024-09-17T04:25:51.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.106+0000 7fa823400640 1 -- 172.21.15.100:0/1102969345 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5232 (secure 0 0 0) 0x7fa818062be0 con 0x7fa82c1518d0 2024-09-17T04:25:51.107 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:51.107 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"btime":"2024-09-17T04:25:48:210316+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1086873435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1086873435},{"type":"v1","addr":"172.21.15.100:6833","nonce":1086873435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1708229764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1708229764},{"type":"v1","addr":"172.21.15.184:6836","nonce":1708229764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/828832783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":828832783},{"type":"v1","addr":"172.21.15.184:6837","nonce":828832783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:44.669203+0000","modified":"2024-09-17T04:25:47.693203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3801048032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3801048032},{"type":"v1","addr":"172.21.15.100:6835","nonce":3801048032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5295,"qdb_cluster":[5295]},"id":3}]} 2024-09-17T04:25:51.107 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 26 2024-09-17T04:25:51.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.109+0000 7fa830e00640 1 -- 172.21.15.100:0/1102969345 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa80c078d70 msgr2=0x7fa80c07b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:51.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.109+0000 7fa830e00640 1 --2- 172.21.15.100:0/1102969345 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa80c078d70 0x7fa80c07b280 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa81c033000 tx=0x7fa81c004730 comp rx=0 tx=0).stop 2024-09-17T04:25:51.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.109+0000 7fa830e00640 1 -- 172.21.15.100:0/1102969345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa82c1518d0 msgr2=0x7fa82c13ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:51.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.109+0000 7fa830e00640 1 --2- 172.21.15.100:0/1102969345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa82c1518d0 0x7fa82c13ea80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa81800d750 tx=0x7fa81800dc10 comp rx=0 tx=0).stop 2024-09-17T04:25:51.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.109+0000 7fa830e00640 1 -- 172.21.15.100:0/1102969345 shutdown_connections 2024-09-17T04:25:51.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.109+0000 7fa830e00640 1 --2- 172.21.15.100:0/1102969345 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa80c078d70 0x7fa80c07b280 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.109+0000 7fa830e00640 1 --2- 172.21.15.100:0/1102969345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa82c15e3d0 0x7fa82c13efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.109+0000 7fa830e00640 1 --2- 172.21.15.100:0/1102969345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa82c1518d0 0x7fa82c13ea80 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.109+0000 7fa830e00640 1 --2- 172.21.15.100:0/1102969345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa82c0735c0 0x7fa82c13c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.109+0000 7fa830e00640 1 -- 172.21.15.100:0/1102969345 >> 172.21.15.100:0/1102969345 conn(0x7fa82c0b1e40 msgr2=0x7fa82c09cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:51.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.109+0000 7fa830e00640 1 -- 172.21.15.100:0/1102969345 shutdown_connections 2024-09-17T04:25:51.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.109+0000 7fa830e00640 1 -- 172.21.15.100:0/1102969345 wait complete. 2024-09-17T04:25:51.125 INFO:tasks.ceph.mds.c.smithi100.stderr:2024-09-17T04:25:51.125+0000 7fbe87e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T04:25:51.537 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T04:25:44.669203+0000', 'modified': '2024-09-17T04:25:47.693203+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5295}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5295': {'gid': 5295, 'name': 'c', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6835/3801048032', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6834', 'nonce': 3801048032}, {'type': 'v1', 'addr': '172.21.15.100:6835', 'nonce': 3801048032}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5295, 'qdb_cluster': [5295]} 2024-09-17T04:25:51.537 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T04:25:51.538 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T04:25:51.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.757+0000 7f091f400640 1 -- 172.21.15.100:0/2165565919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09180af6a0 msgr2=0x7f09180ad1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:51.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.757+0000 7f091f400640 1 --2- 172.21.15.100:0/2165565919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09180af6a0 0x7f09180ad1c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0904009830 tx=0x7f090402a490 comp rx=0 tx=0).stop 2024-09-17T04:25:51.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.757+0000 7f091f400640 1 -- 172.21.15.100:0/2165565919 shutdown_connections 2024-09-17T04:25:51.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.757+0000 7f091f400640 1 --2- 172.21.15.100:0/2165565919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09180af6a0 0x7f09180ad1c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.757+0000 7f091f400640 1 -- 172.21.15.100:0/2165565919 >> 172.21.15.100:0/2165565919 conn(0x7f091806d3d0 msgr2=0x7f09180b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:51.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.758+0000 7f091f400640 1 -- 172.21.15.100:0/2165565919 shutdown_connections 2024-09-17T04:25:51.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.758+0000 7f091f400640 1 -- 172.21.15.100:0/2165565919 wait complete. 2024-09-17T04:25:51.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.760+0000 7f091f400640 1 Processor -- start 2024-09-17T04:25:51.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.760+0000 7f091f400640 1 -- start start 2024-09-17T04:25:51.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.761+0000 7f091f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0918070660 0x7f0918157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:51.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.761+0000 7f091f400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09180ad700 0x7f09181f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:51.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.761+0000 7f091f400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09180af6a0 0x7f09181f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:51.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.761+0000 7f091f400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f09180cb950 con 0x7f0918070660 2024-09-17T04:25:51.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.761+0000 7f091f400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f09180afab0 con 0x7f09180af6a0 2024-09-17T04:25:51.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.761+0000 7f091f400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f09180cbad0 con 0x7f09180ad700 2024-09-17T04:25:51.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.762+0000 7f091ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0918070660 0x7f0918157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:51.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.762+0000 7f091ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0918070660 0x7f0918157390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55862/0 (socket says 172.21.15.100:55862) 2024-09-17T04:25:51.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.762+0000 7f091ca00640 1 -- 172.21.15.100:0/1489843116 learned_addr learned my addr 172.21.15.100:0/1489843116 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:51.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.762+0000 7f0917e00640 1 --2- 172.21.15.100:0/1489843116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09180ad700 0x7f09181f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:51.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.762+0000 7f091d400640 1 --2- 172.21.15.100:0/1489843116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09180af6a0 0x7f09181f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:51.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.762+0000 7f091ca00640 1 -- 172.21.15.100:0/1489843116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09180ad700 msgr2=0x7f09181f48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:51.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.762+0000 7f091ca00640 1 --2- 172.21.15.100:0/1489843116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09180ad700 0x7f09181f48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.762+0000 7f091ca00640 1 -- 172.21.15.100:0/1489843116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09180af6a0 msgr2=0x7f09181f4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:51.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.762+0000 7f091ca00640 1 --2- 172.21.15.100:0/1489843116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09180af6a0 0x7f09181f4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.762+0000 7f091ca00640 1 -- 172.21.15.100:0/1489843116 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09180c6f20 con 0x7f0918070660 2024-09-17T04:25:51.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.762+0000 7f0917e00640 1 --2- 172.21.15.100:0/1489843116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09180ad700 0x7f09181f48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:51.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.762+0000 7f091ca00640 1 --2- 172.21.15.100:0/1489843116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0918070660 0x7f0918157390 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f090c00c390 tx=0x7f090c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:51.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.763+0000 7f0914c00640 1 -- 172.21.15.100:0/1489843116 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c011790 con 0x7f0918070660 2024-09-17T04:25:51.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.763+0000 7f0914c00640 1 -- 172.21.15.100:0/1489843116 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f090c017070 con 0x7f0918070660 2024-09-17T04:25:51.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.763+0000 7f0914c00640 1 -- 172.21.15.100:0/1489843116 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c011cc0 con 0x7f0918070660 2024-09-17T04:25:51.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.763+0000 7f091f400640 1 -- 172.21.15.100:0/1489843116 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09180c9c20 con 0x7f0918070660 2024-09-17T04:25:51.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.763+0000 7f091f400640 1 -- 172.21.15.100:0/1489843116 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09181f5950 con 0x7f0918070660 2024-09-17T04:25:51.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.765+0000 7f0914c00640 1 -- 172.21.15.100:0/1489843116 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f090c01c460 con 0x7f0918070660 2024-09-17T04:25:51.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.766+0000 7f0914c00640 1 --2- 172.21.15.100:0/1489843116 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f08f8078d70 0x7f08f807b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:51.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.766+0000 7f0914c00640 1 -- 172.21.15.100:0/1489843116 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f090c09aed0 con 0x7f0918070660 2024-09-17T04:25:51.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.766+0000 7f09265d3740 1 Processor -- start 2024-09-17T04:25:51.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.767+0000 7f0917e00640 1 --2- 172.21.15.100:0/1489843116 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f08f8078d70 0x7f08f807b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:51.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.767+0000 7f09265d3740 1 -- start start 2024-09-17T04:25:51.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.767+0000 7f0917e00640 1 --2- 172.21.15.100:0/1489843116 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f08f8078d70 0x7f08f807b280 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0904034000 tx=0x7f0904004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:51.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.768+0000 7f09265d3740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55718cead270 0x55718d298a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:51.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.768+0000 7f09265d3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55718ce843b0 0x55718d24df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:51.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.768+0000 7f09265d3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718ce8c590 0x55718d24e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:51.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.768+0000 7f09265d3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55718d1d7430 con 0x55718ce8c590 2024-09-17T04:25:51.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.768+0000 7f09265d3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55718d1c75d0 con 0x55718ce843b0 2024-09-17T04:25:51.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.768+0000 7f09265d3740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55718d1b6a00 con 0x55718cead270 2024-09-17T04:25:51.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.768+0000 7f091d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718ce8c590 0x55718d24e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:51.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.768+0000 7f091d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718ce8c590 0x55718d24e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55866/0 (socket says 172.21.15.100:55866) 2024-09-17T04:25:51.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.768+0000 7f091d400640 1 -- 172.21.15.100:0/2085193354 learned_addr learned my addr 172.21.15.100:0/2085193354 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:51.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.769+0000 7f091ca00640 1 --2- 172.21.15.100:0/2085193354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55718cead270 0x55718d298a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:51.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.769+0000 7f0917e00640 1 --2- 172.21.15.100:0/2085193354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55718ce843b0 0x55718d24df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:51.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.769+0000 7f091d400640 1 -- 172.21.15.100:0/2085193354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55718cead270 msgr2=0x55718d298a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:25:51.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.769+0000 7f091d400640 1 --2- 172.21.15.100:0/2085193354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55718cead270 0x55718d298a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.769+0000 7f091d400640 1 -- 172.21.15.100:0/2085193354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55718ce843b0 msgr2=0x55718d24df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:25:51.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.769+0000 7f091d400640 1 --2- 172.21.15.100:0/2085193354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55718ce843b0 0x55718d24df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.769+0000 7f091d400640 1 -- 172.21.15.100:0/2085193354 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55718d17f520 con 0x55718ce8c590 2024-09-17T04:25:51.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.769+0000 7f091d400640 1 --2- 172.21.15.100:0/2085193354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718ce8c590 0x55718d24e4a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0910001de0 tx=0x7f0910001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0d10994b48cb14 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:51.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.769+0000 7f0909200640 1 -- 172.21.15.100:0/2085193354 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091000fc40 con 0x55718ce8c590 2024-09-17T04:25:51.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.769+0000 7f0909200640 1 -- 172.21.15.100:0/2085193354 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0910018070 con 0x55718ce8c590 2024-09-17T04:25:51.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.769+0000 7f0909200640 1 -- 172.21.15.100:0/2085193354 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0910003bd0 con 0x55718ce8c590 2024-09-17T04:25:51.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.770+0000 7f09265d3740 1 -- 172.21.15.100:0/2085193354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718ce8c590 msgr2=0x55718d24e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:51.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.770+0000 7f09265d3740 1 --2- 172.21.15.100:0/2085193354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718ce8c590 0x55718d24e4a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0910001de0 tx=0x7f0910001630 comp rx=0 tx=0).stop 2024-09-17T04:25:51.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.770+0000 7f09265d3740 1 -- 172.21.15.100:0/2085193354 shutdown_connections 2024-09-17T04:25:51.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.770+0000 7f09265d3740 1 --2- 172.21.15.100:0/2085193354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718ce8c590 0x55718d24e4a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.770+0000 7f09265d3740 1 --2- 172.21.15.100:0/2085193354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55718ce843b0 0x55718d24df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.770+0000 7f09265d3740 1 --2- 172.21.15.100:0/2085193354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55718cead270 0x55718d298a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.770+0000 7f09265d3740 1 -- 172.21.15.100:0/2085193354 >> 172.21.15.100:0/2085193354 conn(0x55718cea13c0 msgr2=0x55718d253970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:51.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.770+0000 7f09265d3740 1 -- 172.21.15.100:0/2085193354 shutdown_connections 2024-09-17T04:25:51.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.770+0000 7f09265d3740 1 -- 172.21.15.100:0/2085193354 wait complete. 2024-09-17T04:25:51.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.771+0000 7f09265d3740 1 -- start start 2024-09-17T04:25:51.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.771+0000 7f09265d3740 1 Processor -- start 2024-09-17T04:25:51.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.773+0000 7f09265d3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718ce843b0 0x55718d292d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:51.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.773+0000 7f09265d3740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55718ce8c590 0x55718d29b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:51.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.773+0000 7f09265d3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55718cead270 0x55718d274d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:51.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.773+0000 7f09265d3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55718d15d330 con 0x55718ce843b0 2024-09-17T04:25:51.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.773+0000 7f09265d3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55718d1ba810 con 0x55718cead270 2024-09-17T04:25:51.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.773+0000 7f09265d3740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55718cf27620 con 0x55718ce8c590 2024-09-17T04:25:51.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.773+0000 7f09265d3740 10 client.0 _ll_register_callbacks cb 0x55718d162420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:25:51.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.773+0000 7f091ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718ce843b0 0x55718d292d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:51.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.773+0000 7f091ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718ce843b0 0x55718d292d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55878/0 (socket says 172.21.15.100:55878) 2024-09-17T04:25:51.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.773+0000 7f091ca00640 1 -- 172.21.15.100:0/1667421057 learned_addr learned my addr 172.21.15.100:0/1667421057 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:51.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.773+0000 7f0917e00640 1 --2- 172.21.15.100:0/1667421057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55718ce8c590 0x55718d29b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:51.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.773+0000 7f091d400640 1 --2- 172.21.15.100:0/1667421057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55718cead270 0x55718d274d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:51.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.774+0000 7f091ca00640 1 -- 172.21.15.100:0/1667421057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55718ce8c590 msgr2=0x55718d29b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:25:51.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.774+0000 7f091ca00640 1 --2- 172.21.15.100:0/1667421057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55718ce8c590 0x55718d29b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.774+0000 7f091ca00640 1 -- 172.21.15.100:0/1667421057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55718cead270 msgr2=0x55718d274d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:25:51.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.774+0000 7f091ca00640 1 --2- 172.21.15.100:0/1667421057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55718cead270 0x55718d274d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.774+0000 7f091ca00640 1 -- 172.21.15.100:0/1667421057 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55718cf18850 con 0x55718ce843b0 2024-09-17T04:25:51.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.774+0000 7f091ca00640 1 --2- 172.21.15.100:0/1667421057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718ce843b0 0x55718d292d40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f090c000f30 tx=0x7f090c0622f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7544c60917399a22 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:51.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.774+0000 7f08fd600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-17T04:25:51.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.774+0000 7f08fd600640 1 -- 172.21.15.100:0/1667421057 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c061c40 con 0x55718ce843b0 2024-09-17T04:25:51.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.774+0000 7f09265d3740 1 -- 172.21.15.100:0/1667421057 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0910018070 con 0x55718ce843b0 2024-09-17T04:25:51.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.774+0000 7f08fd600640 1 -- 172.21.15.100:0/1667421057 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f090c063b10 con 0x55718ce843b0 2024-09-17T04:25:51.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.774+0000 7f08fd600640 1 -- 172.21.15.100:0/1667421057 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f090c0672c0 con 0x55718ce843b0 2024-09-17T04:25:51.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.774+0000 7f09265d3740 10 client.5409 fetch_fsmap learned FSMap version 26 2024-09-17T04:25:51.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.775+0000 7f09265d3740 1 -- 172.21.15.100:0/1667421057 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55718cf8f6b0 con 0x55718ce843b0 2024-09-17T04:25:51.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.775+0000 7f08fd600640 1 -- 172.21.15.100:0/1667421057 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0910018070 con 0x55718ce843b0 2024-09-17T04:25:51.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.775+0000 7f08fd600640 1 -- 172.21.15.100:0/1667421057 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7f090c05e130 con 0x55718ce843b0 2024-09-17T04:25:51.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.775+0000 7f09265d3740 10 client.5409 fetch_fsmap finished waiting for FSMap version 26 2024-09-17T04:25:51.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.775+0000 7f09265d3740 10 client.5409 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T04:25:51.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.775+0000 7f09265d3740 20 client.5409 populate_metadata read hostname 'smithi100' 2024-09-17T04:25:51.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.775+0000 7f09265d3740 1 --2- 172.21.15.100:0/1667421057 >> [v2:172.21.15.100:6834/3801048032,v1:172.21.15.100:6835/3801048032] conn(0x55718ce6be40 0x55718d2ad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:51.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.775+0000 7f09265d3740 4 client.5409 mds_command: new command op to 5295 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T04:25:51.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.775+0000 7f09265d3740 1 -- 172.21.15.100:0/1667421057 --> [v2:172.21.15.100:6834/3801048032,v1:172.21.15.100:6835/3801048032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55718d1c9520 con 0x55718ce6be40 2024-09-17T04:25:51.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.775+0000 7f0917e00640 1 --2- 172.21.15.100:0/1667421057 >> [v2:172.21.15.100:6834/3801048032,v1:172.21.15.100:6835/3801048032] conn(0x55718ce6be40 0x55718d2ad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:51.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.776+0000 7f0917e00640 1 --2- 172.21.15.100:0/1667421057 >> [v2:172.21.15.100:6834/3801048032,v1:172.21.15.100:6835/3801048032] conn(0x55718ce6be40 0x55718d2ad7f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85e2f9c4596dd388 server_cookie=72657cc58030df2d in_seq=0 out_seq=0 2024-09-17T04:25:51.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.776+0000 7f08fd600640 10 client.5409 ms_handle_connect on v2:172.21.15.100:6834/3801048032 2024-09-17T04:25:51.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.777+0000 7f08fd600640 1 -- 172.21.15.100:0/1667421057 <== mds.0 v2:172.21.15.100:6834/3801048032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55718d1c9520 con 0x55718ce6be40 2024-09-17T04:25:51.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.777+0000 7f08fd600640 10 client.5409 handle_command_reply: tid=0 2024-09-17T04:25:51.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.777+0000 7f09265d3740 1 client.5409 shutdown 2024-09-17T04:25:51.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.777+0000 7f09265d3740 1 -- 172.21.15.100:0/1667421057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718ce843b0 msgr2=0x55718d292d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:51.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.777+0000 7f09265d3740 1 --2- 172.21.15.100:0/1667421057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718ce843b0 0x55718d292d40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f090c000f30 tx=0x7f090c0622f0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.777+0000 7f09265d3740 1 -- 172.21.15.100:0/1667421057 shutdown_connections 2024-09-17T04:25:51.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.777+0000 7f09265d3740 1 --2- 172.21.15.100:0/1667421057 >> [v2:172.21.15.100:6834/3801048032,v1:172.21.15.100:6835/3801048032] conn(0x55718ce6be40 0x55718d2ad7f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.777+0000 7f09265d3740 1 --2- 172.21.15.100:0/1667421057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55718cead270 0x55718d274d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.777+0000 7f09265d3740 1 --2- 172.21.15.100:0/1667421057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55718ce8c590 0x55718d29b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.777+0000 7f09265d3740 1 --2- 172.21.15.100:0/1667421057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718ce843b0 0x55718d292d40 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.777+0000 7f08fd600640 0 client.5409 ms_handle_reset on v2:172.21.15.100:6834/3801048032 2024-09-17T04:25:51.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.777+0000 7f09265d3740 1 -- 172.21.15.100:0/1667421057 >> 172.21.15.100:0/1667421057 conn(0x55718cea13c0 msgr2=0x55718d27a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:51.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.777+0000 7f09265d3740 1 -- 172.21.15.100:0/1667421057 shutdown_connections 2024-09-17T04:25:51.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.777+0000 7f09265d3740 1 -- 172.21.15.100:0/1667421057 wait complete. 2024-09-17T04:25:51.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.778+0000 7f09265d3740 20 client.5409 trim_cache size 0 max 16384 2024-09-17T04:25:51.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.796+0000 7f09265d3740 1 Processor -- start 2024-09-17T04:25:51.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.797+0000 7f09265d3740 1 -- start start 2024-09-17T04:25:51.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.798+0000 7f09265d3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55718ce843b0 0x55718d2c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:51.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.798+0000 7f09265d3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718ce8c590 0x55718d298a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:51.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.798+0000 7f09265d3740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55718cead270 0x55718d298f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:51.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.798+0000 7f09265d3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f09180c62b0 con 0x55718ce8c590 2024-09-17T04:25:51.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.798+0000 7f09265d3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f09180c98b0 con 0x55718ce843b0 2024-09-17T04:25:51.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.798+0000 7f09265d3740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f09180c5c60 con 0x55718cead270 2024-09-17T04:25:51.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.798+0000 7f0917e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718ce8c590 0x55718d298a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:51.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.798+0000 7f0917e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718ce8c590 0x55718d298a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55894/0 (socket says 172.21.15.100:55894) 2024-09-17T04:25:51.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.798+0000 7f0917e00640 1 -- 172.21.15.100:0/3693737608 learned_addr learned my addr 172.21.15.100:0/3693737608 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:51.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.798+0000 7f091ca00640 1 --2- 172.21.15.100:0/3693737608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55718ce843b0 0x55718d2c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:51.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.798+0000 7f091d400640 1 --2- 172.21.15.100:0/3693737608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55718cead270 0x55718d298f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:51.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.798+0000 7f0917e00640 1 -- 172.21.15.100:0/3693737608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55718cead270 msgr2=0x55718d298f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:25:51.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.798+0000 7f0917e00640 1 --2- 172.21.15.100:0/3693737608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55718cead270 0x55718d298f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.798+0000 7f0917e00640 1 -- 172.21.15.100:0/3693737608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55718ce843b0 msgr2=0x55718d2c01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:25:51.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.798+0000 7f0917e00640 1 --2- 172.21.15.100:0/3693737608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55718ce843b0 0x55718d2c01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.798+0000 7f0917e00640 1 -- 172.21.15.100:0/3693737608 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55718d254dc0 con 0x55718ce8c590 2024-09-17T04:25:51.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.799+0000 7f0917e00640 1 --2- 172.21.15.100:0/3693737608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718ce8c590 0x55718d298a20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f09040095f0 tx=0x7f090403e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c04d8f8346f56bb server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:51.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.799+0000 7f08ffe00640 1 -- 172.21.15.100:0/3693737608 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0904057070 con 0x55718ce8c590 2024-09-17T04:25:51.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.799+0000 7f08ffe00640 1 -- 172.21.15.100:0/3693737608 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0904003bd0 con 0x55718ce8c590 2024-09-17T04:25:51.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.799+0000 7f08ffe00640 1 -- 172.21.15.100:0/3693737608 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0904049420 con 0x55718ce8c590 2024-09-17T04:25:51.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.800+0000 7f09265d3740 1 -- 172.21.15.100:0/3693737608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718ce8c590 msgr2=0x55718d298a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:51.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.800+0000 7f09265d3740 1 --2- 172.21.15.100:0/3693737608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718ce8c590 0x55718d298a20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f09040095f0 tx=0x7f090403e070 comp rx=0 tx=0).stop 2024-09-17T04:25:51.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.800+0000 7f09265d3740 1 -- 172.21.15.100:0/3693737608 shutdown_connections 2024-09-17T04:25:51.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.800+0000 7f09265d3740 1 --2- 172.21.15.100:0/3693737608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55718cead270 0x55718d298f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.800+0000 7f09265d3740 1 --2- 172.21.15.100:0/3693737608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718ce8c590 0x55718d298a20 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.800+0000 7f09265d3740 1 --2- 172.21.15.100:0/3693737608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55718ce843b0 0x55718d2c01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.800+0000 7f09265d3740 1 -- 172.21.15.100:0/3693737608 >> 172.21.15.100:0/3693737608 conn(0x55718cea13c0 msgr2=0x55718d292f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:51.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.800+0000 7f09265d3740 1 -- 172.21.15.100:0/3693737608 shutdown_connections 2024-09-17T04:25:51.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.800+0000 7f09265d3740 1 -- 172.21.15.100:0/3693737608 wait complete. 2024-09-17T04:25:51.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.801+0000 7f09265d3740 1 -- start start 2024-09-17T04:25:51.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.801+0000 7f09265d3740 1 Processor -- start 2024-09-17T04:25:51.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.803+0000 7f09265d3740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55718ce843b0 0x55718d291ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:51.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.803+0000 7f09265d3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718cead270 0x55718d29aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:51.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.803+0000 7f09265d3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55718d29af70 0x55718d29b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:51.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.803+0000 7f09265d3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55718d2756e0 con 0x55718cead270 2024-09-17T04:25:51.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.803+0000 7f09265d3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55718d275290 con 0x55718d29af70 2024-09-17T04:25:51.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.803+0000 7f09265d3740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55718d2af6e0 con 0x55718ce843b0 2024-09-17T04:25:51.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.803+0000 7f09265d3740 10 client.0 _ll_register_callbacks cb 0x7f090c098ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:25:51.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.803+0000 7f0917e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718cead270 0x55718d29aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:51.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.803+0000 7f0917e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718cead270 0x55718d29aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55898/0 (socket says 172.21.15.100:55898) 2024-09-17T04:25:51.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.803+0000 7f0917e00640 1 -- 172.21.15.100:0/2951351326 learned_addr learned my addr 172.21.15.100:0/2951351326 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:51.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.803+0000 7f091ca00640 1 --2- 172.21.15.100:0/2951351326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55718ce843b0 0x55718d291ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:51.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.803+0000 7f091d400640 1 --2- 172.21.15.100:0/2951351326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55718d29af70 0x55718d29b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:51.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.803+0000 7f0917e00640 1 -- 172.21.15.100:0/2951351326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55718ce843b0 msgr2=0x55718d291ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:25:51.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.803+0000 7f0917e00640 1 --2- 172.21.15.100:0/2951351326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55718ce843b0 0x55718d291ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.803+0000 7f0917e00640 1 -- 172.21.15.100:0/2951351326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55718d29af70 msgr2=0x55718d29b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:25:51.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.803+0000 7f0917e00640 1 --2- 172.21.15.100:0/2951351326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55718d29af70 0x55718d29b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.803+0000 7f0917e00640 1 -- 172.21.15.100:0/2951351326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55718d294390 con 0x55718cead270 2024-09-17T04:25:51.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.803+0000 7f091d400640 1 --2- 172.21.15.100:0/2951351326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55718d29af70 0x55718d29b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:51.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.804+0000 7f0917e00640 1 --2- 172.21.15.100:0/2951351326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718cead270 0x55718d29aa30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f09040095f0 tx=0x7f09040038f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9814ad018a842e79 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:51.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.804+0000 7f08fd600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-17T04:25:51.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.804+0000 7f08fd600640 1 -- 172.21.15.100:0/2951351326 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0904057070 con 0x55718cead270 2024-09-17T04:25:51.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.804+0000 7f08fd600640 1 -- 172.21.15.100:0/2951351326 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f090405cab0 con 0x55718cead270 2024-09-17T04:25:51.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.804+0000 7f09265d3740 1 -- 172.21.15.100:0/2951351326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0904003bd0 con 0x55718cead270 2024-09-17T04:25:51.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.804+0000 7f08fd600640 1 -- 172.21.15.100:0/2951351326 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f090404d070 con 0x55718cead270 2024-09-17T04:25:51.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.804+0000 7f09265d3740 10 client.5415 fetch_fsmap learned FSMap version 26 2024-09-17T04:25:51.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.804+0000 7f09265d3740 1 -- 172.21.15.100:0/2951351326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55718d2af860 con 0x55718cead270 2024-09-17T04:25:51.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.804+0000 7f08fd600640 1 -- 172.21.15.100:0/2951351326 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0904003bd0 con 0x55718cead270 2024-09-17T04:25:51.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.804+0000 7f08fd600640 1 -- 172.21.15.100:0/2951351326 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7f090405ccb0 con 0x55718cead270 2024-09-17T04:25:51.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.804+0000 7f09265d3740 10 client.5415 fetch_fsmap finished waiting for FSMap version 26 2024-09-17T04:25:51.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.804+0000 7f09265d3740 10 client.5415 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T04:25:51.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.804+0000 7f09265d3740 20 client.5415 populate_metadata read hostname 'smithi100' 2024-09-17T04:25:51.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.804+0000 7f09265d3740 1 --2- 172.21.15.100:0/2951351326 >> [v2:172.21.15.100:6834/3801048032,v1:172.21.15.100:6835/3801048032] conn(0x55718d31c6f0 0x55718d31cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:51.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.804+0000 7f09265d3740 4 client.5415 mds_command: new command op to 5295 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T04:25:51.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.804+0000 7f09265d3740 1 -- 172.21.15.100:0/2951351326 --> [v2:172.21.15.100:6834/3801048032,v1:172.21.15.100:6835/3801048032] -- command(tid 0: {"prefix": "status"}) -- 0x55718d1c4fd0 con 0x55718d31c6f0 2024-09-17T04:25:51.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.805+0000 7f091ca00640 1 --2- 172.21.15.100:0/2951351326 >> [v2:172.21.15.100:6834/3801048032,v1:172.21.15.100:6835/3801048032] conn(0x55718d31c6f0 0x55718d31cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:51.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.805+0000 7f091ca00640 1 --2- 172.21.15.100:0/2951351326 >> [v2:172.21.15.100:6834/3801048032,v1:172.21.15.100:6835/3801048032] conn(0x55718d31c6f0 0x55718d31cb00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee52d6f0c7f33b43 server_cookie=71456aadc3a99cbb in_seq=0 out_seq=0 2024-09-17T04:25:51.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.805+0000 7f08fd600640 10 client.5415 ms_handle_connect on v2:172.21.15.100:6834/3801048032 2024-09-17T04:25:51.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.805+0000 7f08fd600640 1 -- 172.21.15.100:0/2951351326 <== mds.0 v2:172.21.15.100:6834/3801048032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55718d1c4fd0 con 0x55718d31c6f0 2024-09-17T04:25:51.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.805+0000 7f08fd600640 10 client.5415 handle_command_reply: tid=0 2024-09-17T04:25:51.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.805+0000 7f09265d3740 1 client.5415 shutdown 2024-09-17T04:25:51.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.806+0000 7f09265d3740 1 -- 172.21.15.100:0/2951351326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718cead270 msgr2=0x55718d29aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:51.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.806+0000 7f09265d3740 1 --2- 172.21.15.100:0/2951351326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718cead270 0x55718d29aa30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f09040095f0 tx=0x7f09040038f0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.806+0000 7f09265d3740 1 -- 172.21.15.100:0/2951351326 shutdown_connections 2024-09-17T04:25:51.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.806+0000 7f09265d3740 1 --2- 172.21.15.100:0/2951351326 >> [v2:172.21.15.100:6834/3801048032,v1:172.21.15.100:6835/3801048032] conn(0x55718d31c6f0 0x55718d31cb00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.806+0000 7f09265d3740 1 --2- 172.21.15.100:0/2951351326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55718d29af70 0x55718d29b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.806+0000 7f09265d3740 1 --2- 172.21.15.100:0/2951351326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55718cead270 0x55718d29aa30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.806+0000 7f09265d3740 1 --2- 172.21.15.100:0/2951351326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55718ce843b0 0x55718d291ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.806+0000 7f09265d3740 1 -- 172.21.15.100:0/2951351326 >> 172.21.15.100:0/2951351326 conn(0x55718cea13c0 msgr2=0x55718d2aeb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:51.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.806+0000 7f08fd600640 0 client.5415 ms_handle_reset on v2:172.21.15.100:6834/3801048032 2024-09-17T04:25:51.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.806+0000 7f09265d3740 1 -- 172.21.15.100:0/2951351326 shutdown_connections 2024-09-17T04:25:51.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.806+0000 7f09265d3740 1 -- 172.21.15.100:0/2951351326 wait complete. 2024-09-17T04:25:51.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.806+0000 7f09265d3740 20 client.5415 trim_cache size 0 max 16384 2024-09-17T04:25:51.810 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:25:51.810 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "8a399842-3a5c-4ea2-9afb-b2704332e59f", 2024-09-17T04:25:51.810 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 0, 2024-09-17T04:25:51.810 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 5295, 2024-09-17T04:25:51.810 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-09-17T04:25:51.810 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-09-17T04:25:51.810 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-09-17T04:25:51.810 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 6.6323567849999998, 2024-09-17T04:25:51.811 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 26, 2024-09-17T04:25:51.811 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 48, 2024-09-17T04:25:51.811 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 48, 2024-09-17T04:25:51.811 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 11.69034978 2024-09-17T04:25:51.811 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:25:51.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.807+0000 7f091f400640 1 -- 172.21.15.100:0/1489843116 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f08f8078d70 msgr2=0x7f08f807b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:51.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.807+0000 7f091f400640 1 --2- 172.21.15.100:0/1489843116 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f08f8078d70 0x7f08f807b280 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0904034000 tx=0x7f0904004730 comp rx=0 tx=0).stop 2024-09-17T04:25:51.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.807+0000 7f091f400640 1 -- 172.21.15.100:0/1489843116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0918070660 msgr2=0x7f0918157390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:51.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.807+0000 7f091f400640 1 --2- 172.21.15.100:0/1489843116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0918070660 0x7f0918157390 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f090c00c390 tx=0x7f090c00c850 comp rx=0 tx=0).stop 2024-09-17T04:25:51.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.807+0000 7f091f400640 1 -- 172.21.15.100:0/1489843116 shutdown_connections 2024-09-17T04:25:51.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.807+0000 7f091f400640 1 --2- 172.21.15.100:0/1489843116 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f08f8078d70 0x7f08f807b280 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.807+0000 7f091f400640 1 --2- 172.21.15.100:0/1489843116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09180af6a0 0x7f09181f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.807+0000 7f091f400640 1 --2- 172.21.15.100:0/1489843116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09180ad700 0x7f09181f48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.807+0000 7f091f400640 1 --2- 172.21.15.100:0/1489843116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0918070660 0x7f0918157390 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:51.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.807+0000 7f091f400640 1 -- 172.21.15.100:0/1489843116 >> 172.21.15.100:0/1489843116 conn(0x7f091806d3d0 msgr2=0x7f09180b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:51.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.807+0000 7f091f400640 1 -- 172.21.15.100:0/1489843116 shutdown_connections 2024-09-17T04:25:51.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:51.807+0000 7f091f400640 1 -- 172.21.15.100:0/1489843116 wait complete. 2024-09-17T04:25:51.832 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:25:52.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.052+0000 7f0542a00640 1 -- 172.21.15.100:0/164714321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f053c15d760 msgr2=0x7f053c1535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:52.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.052+0000 7f0542a00640 1 --2- 172.21.15.100:0/164714321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f053c15d760 0x7f053c1535c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f053c072d60 tx=0x7f052c02a820 comp rx=0 tx=0).stop 2024-09-17T04:25:52.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.052+0000 7f0542a00640 1 -- 172.21.15.100:0/164714321 shutdown_connections 2024-09-17T04:25:52.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.052+0000 7f0542a00640 1 --2- 172.21.15.100:0/164714321 >> v2:172.21.15.184:3300/0 conn(0x7f053c0ad0e0 0x7f053c0ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:52.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.052+0000 7f0542a00640 1 --2- 172.21.15.100:0/164714321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f053c15d760 0x7f053c1535c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:52.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.052+0000 7f0542a00640 1 -- 172.21.15.100:0/164714321 >> 172.21.15.100:0/164714321 conn(0x7f053c06d730 msgr2=0x7f053c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:52.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.052+0000 7f0542a00640 1 -- 172.21.15.100:0/164714321 shutdown_connections 2024-09-17T04:25:52.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.052+0000 7f0542a00640 1 -- 172.21.15.100:0/164714321 wait complete. 2024-09-17T04:25:52.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.055+0000 7f0542a00640 1 Processor -- start 2024-09-17T04:25:52.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.055+0000 7f0542a00640 1 -- start start 2024-09-17T04:25:52.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.056+0000 7f0542a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f053c0ad0e0 0x7f053c09db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:52.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.056+0000 7f0542a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f053c15d760 0x7f053c149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:52.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.056+0000 7f0542a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f053c160310 0x7f053c1561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:52.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.056+0000 7f0542a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f053c06f220 con 0x7f053c0ad0e0 2024-09-17T04:25:52.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.056+0000 7f0542a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f053c06f0a0 con 0x7f053c160310 2024-09-17T04:25:52.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.056+0000 7f0542a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f053c0b4c20 con 0x7f053c15d760 2024-09-17T04:25:52.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.056+0000 7f053be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f053c0ad0e0 0x7f053c09db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:52.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.056+0000 7f053be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f053c0ad0e0 0x7f053c09db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55910/0 (socket says 172.21.15.100:55910) 2024-09-17T04:25:52.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.056+0000 7f053be00640 1 -- 172.21.15.100:0/2218968890 learned_addr learned my addr 172.21.15.100:0/2218968890 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:52.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.057+0000 7f0533400640 1 --2- 172.21.15.100:0/2218968890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f053c15d760 0x7f053c149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:52.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.057+0000 7f0540a00640 1 --2- 172.21.15.100:0/2218968890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f053c160310 0x7f053c1561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:52.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.057+0000 7f053be00640 1 -- 172.21.15.100:0/2218968890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f053c15d760 msgr2=0x7f053c149920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:52.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.057+0000 7f053be00640 1 --2- 172.21.15.100:0/2218968890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f053c15d760 0x7f053c149920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:52.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.057+0000 7f053be00640 1 -- 172.21.15.100:0/2218968890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f053c160310 msgr2=0x7f053c1561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:52.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.057+0000 7f053be00640 1 --2- 172.21.15.100:0/2218968890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f053c160310 0x7f053c1561c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:52.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.057+0000 7f053be00640 1 -- 172.21.15.100:0/2218968890 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053c09a2d0 con 0x7f053c0ad0e0 2024-09-17T04:25:52.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.057+0000 7f0533400640 1 --2- 172.21.15.100:0/2218968890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f053c15d760 0x7f053c149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:25:52.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.057+0000 7f053be00640 1 --2- 172.21.15.100:0/2218968890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f053c0ad0e0 0x7f053c09db30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f052c02ae10 tx=0x7f052c032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:52.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.057+0000 7f0538c00640 1 -- 172.21.15.100:0/2218968890 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c037040 con 0x7f053c0ad0e0 2024-09-17T04:25:52.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.057+0000 7f0538c00640 1 -- 172.21.15.100:0/2218968890 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f052c042070 con 0x7f053c0ad0e0 2024-09-17T04:25:52.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.057+0000 7f0538c00640 1 -- 172.21.15.100:0/2218968890 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c003e60 con 0x7f053c0ad0e0 2024-09-17T04:25:52.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.057+0000 7f0542a00640 1 -- 172.21.15.100:0/2218968890 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f053c149e60 con 0x7f053c0ad0e0 2024-09-17T04:25:52.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.058+0000 7f0542a00640 1 -- 172.21.15.100:0/2218968890 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f053c09a850 con 0x7f053c0ad0e0 2024-09-17T04:25:52.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.059+0000 7f0542a00640 1 -- 172.21.15.100:0/2218968890 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f053c160fa0 con 0x7f053c0ad0e0 2024-09-17T04:25:52.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.060+0000 7f0538c00640 1 -- 172.21.15.100:0/2218968890 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f052c004280 con 0x7f053c0ad0e0 2024-09-17T04:25:52.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.061+0000 7f0538c00640 1 --2- 172.21.15.100:0/2218968890 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0518078d40 0x7f051807b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:52.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.061+0000 7f0538c00640 1 -- 172.21.15.100:0/2218968890 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f052c0bbea0 con 0x7f053c0ad0e0 2024-09-17T04:25:52.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.061+0000 7f0533400640 1 --2- 172.21.15.100:0/2218968890 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0518078d40 0x7f051807b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:52.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.062+0000 7f0533400640 1 --2- 172.21.15.100:0/2218968890 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0518078d40 0x7f051807b250 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f053c156150 tx=0x7f0524005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:52.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.066+0000 7f0538c00640 1 -- 172.21.15.100:0/2218968890 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f052c083f40 con 0x7f053c0ad0e0 2024-09-17T04:25:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.289+0000 7f0542a00640 1 -- 172.21.15.100:0/2218968890 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f053c06e060 con 0x7f053c0ad0e0 2024-09-17T04:25:52.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.290+0000 7f0538c00640 1 -- 172.21.15.100:0/2218968890 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f052c083840 con 0x7f053c0ad0e0 2024-09-17T04:25:52.290 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:25:52.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.292+0000 7f0542a00640 1 -- 172.21.15.100:0/2218968890 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0518078d40 msgr2=0x7f051807b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:52.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.292+0000 7f0542a00640 1 --2- 172.21.15.100:0/2218968890 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0518078d40 0x7f051807b250 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f053c156150 tx=0x7f0524005f90 comp rx=0 tx=0).stop 2024-09-17T04:25:52.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.292+0000 7f0542a00640 1 -- 172.21.15.100:0/2218968890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f053c0ad0e0 msgr2=0x7f053c09db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:52.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.292+0000 7f0542a00640 1 --2- 172.21.15.100:0/2218968890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f053c0ad0e0 0x7f053c09db30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f052c02ae10 tx=0x7f052c032c60 comp rx=0 tx=0).stop 2024-09-17T04:25:52.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.292+0000 7f0542a00640 1 -- 172.21.15.100:0/2218968890 shutdown_connections 2024-09-17T04:25:52.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.292+0000 7f0542a00640 1 --2- 172.21.15.100:0/2218968890 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0518078d40 0x7f051807b250 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:52.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.292+0000 7f0542a00640 1 --2- 172.21.15.100:0/2218968890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f053c160310 0x7f053c1561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:52.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.292+0000 7f0542a00640 1 --2- 172.21.15.100:0/2218968890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f053c15d760 0x7f053c149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:52.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.292+0000 7f0542a00640 1 --2- 172.21.15.100:0/2218968890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f053c0ad0e0 0x7f053c09db30 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:52.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.292+0000 7f0542a00640 1 -- 172.21.15.100:0/2218968890 >> 172.21.15.100:0/2218968890 conn(0x7f053c06d730 msgr2=0x7f053c0cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:52.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.292+0000 7f0542a00640 1 -- 172.21.15.100:0/2218968890 shutdown_connections 2024-09-17T04:25:52.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52.292+0000 7f0542a00640 1 -- 172.21.15.100:0/2218968890 wait complete. 2024-09-17T04:25:52.318 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T04:25:52.318 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:25:52.318 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-09-17T04:25:52.318 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T04:25:52.318 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T04:25:52.318 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T04:25:52.318 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T04:25:52.319 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:25:52.319 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:25:52.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:52.319 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-09-17T04:25:52.354 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:25:52.354 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:25:52.354 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:25:52.354 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:25:52.354 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-17T04:25:52.355 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:25:52.355 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:25:52.355 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:25:52.355 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-09-17T04:25:52.355 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:25:52.355 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:25:52.355 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-09-17T04:25:52.355 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:25:52.355 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:25:52.355 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:25:52.355 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:25:52.355 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T04:25:52.355 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42608sec preferred_lft 42608sec 2024-09-17T04:25:52.356 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link noprefixroute 2024-09-17T04:25:52.356 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:25:52.356 INFO:teuthology.orchestra.run.smithi100.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:25:52.356 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:2e:de:8e:a4:7c brd ff:ff:ff:ff:ff:ff 2024-09-17T04:25:52.356 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T04:25:52.356 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:25:52.356 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::cc78:e4ff:fe70:7fbe/64 scope link 2024-09-17T04:25:52.356 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:25:52.356 INFO:teuthology.orchestra.run.smithi100.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:25:52.356 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether ea:8c:45:9f:34:87 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:25:52.356 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::e88c:45ff:fe9f:3487/64 scope link 2024-09-17T04:25:52.356 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:25:52.356 INFO:teuthology.orchestra.run.smithi100.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:25:52.356 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:2e:de:8e:a4:7c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:25:52.357 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::c2e:deff:fe8e:a47c/64 scope link 2024-09-17T04:25:52.357 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:25:52.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:52.357 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:25:52.449 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-09-17T04:25:52.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:52.452 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-09-17T04:25:52.488 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-09-17T04:25:52.489 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:25:52.489 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T04:25:52.489 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T04:25:52.489 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:25:52.489 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:25:52.489 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-17T04:25:52.489 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T04:25:52.490 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T04:25:52.490 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T04:25:52.490 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-17T04:25:52.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:25:52.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:25:52.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:52.685 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-17T04:25:52.720 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T04:25:52.720 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T04:25:52.720 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:25:52.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:52.721 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-09-17T04:25:52.816 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:25:52.817 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T04:25:52.852 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:25:52.853 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:25:52.853 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:25:52.928 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T04:25:52.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:52.928 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:25:53.122 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:25:53.123 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:25:53.123 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:25:53.143 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:25:53.143 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:25:53.143 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:25:53.143 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:25:53.143 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:25:53.175 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:25:53.175 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:25:53.175 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:25:53.175 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:25:53.175 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:25:53.175 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-09-17T04:25:53.175 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-09-17T04:25:53.175 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8a399842-3a5c-4ea2-9afb-b2704332e59f" will pass to kernel 2024-09-17T04:25:53.176 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-17T04:25:53.177 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-09-17T04:25:53.177 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-17T04:25:53.177 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-09-17T04:25:53.177 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-17T04:25:53.180 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T04:25:53.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:53.180 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:25:53.240 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:25:53.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:25:53.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:25:53.336 INFO:teuthology.orchestra.run.smithi100.stdout:client5048 2024-09-17T04:25:53.336 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:25:53.336 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:25:53.371 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:25:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.631+0000 7fb36c600640 1 -- 172.21.15.100:0/3149223083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb36415d760 msgr2=0x7fb3641535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.631+0000 7fb36c600640 1 --2- 172.21.15.100:0/3149223083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb36415d760 0x7fb3641535c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb364072d60 tx=0x7fb35802a4d0 comp rx=0 tx=0).stop 2024-09-17T04:25:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.631+0000 7fb36c600640 1 -- 172.21.15.100:0/3149223083 shutdown_connections 2024-09-17T04:25:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.631+0000 7fb36c600640 1 --2- 172.21.15.100:0/3149223083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb364160310 0x7fb364160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.631+0000 7fb36c600640 1 --2- 172.21.15.100:0/3149223083 >> v2:172.21.15.184:3300/0 conn(0x7fb3640ad0e0 0x7fb3640ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.631+0000 7fb36c600640 1 --2- 172.21.15.100:0/3149223083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb36415d760 0x7fb3641535c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.631+0000 7fb36c600640 1 -- 172.21.15.100:0/3149223083 >> 172.21.15.100:0/3149223083 conn(0x7fb36406d730 msgr2=0x7fb3640b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.631+0000 7fb36c600640 1 -- 172.21.15.100:0/3149223083 shutdown_connections 2024-09-17T04:25:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.631+0000 7fb36c600640 1 -- 172.21.15.100:0/3149223083 wait complete. 2024-09-17T04:25:53.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.634+0000 7fb36c600640 1 Processor -- start 2024-09-17T04:25:53.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.634+0000 7fb36c600640 1 -- start start 2024-09-17T04:25:53.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.635+0000 7fb36c600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3640ad0e0 0x7fb36409a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:53.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.635+0000 7fb36c600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb36415d760 0x7fb364148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:53.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.635+0000 7fb36c600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb364160310 0x7fb36415b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:53.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.635+0000 7fb36c600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb3640b4da0 con 0x7fb36415d760 2024-09-17T04:25:53.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.635+0000 7fb36c600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb3640b4c20 con 0x7fb364160310 2024-09-17T04:25:53.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.635+0000 7fb36c600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb364160c60 con 0x7fb3640ad0e0 2024-09-17T04:25:53.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.636+0000 7fb369200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb36415d760 0x7fb364148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:53.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.636+0000 7fb369200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb36415d760 0x7fb364148710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55928/0 (socket says 172.21.15.100:55928) 2024-09-17T04:25:53.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.636+0000 7fb369200640 1 -- 172.21.15.100:0/439301524 learned_addr learned my addr 172.21.15.100:0/439301524 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:53.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.636+0000 7fb36a600640 1 --2- 172.21.15.100:0/439301524 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb364160310 0x7fb36415b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:53.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.636+0000 7fb369c00640 1 --2- 172.21.15.100:0/439301524 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3640ad0e0 0x7fb36409a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:53.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.636+0000 7fb369200640 1 -- 172.21.15.100:0/439301524 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3640ad0e0 msgr2=0x7fb36409a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:53.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.636+0000 7fb369200640 1 --2- 172.21.15.100:0/439301524 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3640ad0e0 0x7fb36409a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:53.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.636+0000 7fb369200640 1 -- 172.21.15.100:0/439301524 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb364160310 msgr2=0x7fb36415b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:53.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.636+0000 7fb369200640 1 --2- 172.21.15.100:0/439301524 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb364160310 0x7fb36415b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:53.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.636+0000 7fb369200640 1 -- 172.21.15.100:0/439301524 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb36409a810 con 0x7fb36415d760 2024-09-17T04:25:53.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.636+0000 7fb369200640 1 --2- 172.21.15.100:0/439301524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb36415d760 0x7fb364148710 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb35000d700 tx=0x7fb35000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:53.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.637+0000 7fb361600640 1 -- 172.21.15.100:0/439301524 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb350017070 con 0x7fb36415d760 2024-09-17T04:25:53.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.637+0000 7fb361600640 1 -- 172.21.15.100:0/439301524 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb350004310 con 0x7fb36415d760 2024-09-17T04:25:53.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.637+0000 7fb36c600640 1 -- 172.21.15.100:0/439301524 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3641f9a50 con 0x7fb36415d760 2024-09-17T04:25:53.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.637+0000 7fb361600640 1 -- 172.21.15.100:0/439301524 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb350009a80 con 0x7fb36415d760 2024-09-17T04:25:53.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.637+0000 7fb36c600640 1 -- 172.21.15.100:0/439301524 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb364156b90 con 0x7fb36415d760 2024-09-17T04:25:53.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.638+0000 7fb36c600640 1 -- 172.21.15.100:0/439301524 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb364074b80 con 0x7fb36415d760 2024-09-17T04:25:53.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.639+0000 7fb361600640 1 -- 172.21.15.100:0/439301524 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fb350009c20 con 0x7fb36415d760 2024-09-17T04:25:53.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.640+0000 7fb361600640 1 --2- 172.21.15.100:0/439301524 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb344078d40 0x7fb34407b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:53.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.640+0000 7fb361600640 1 -- 172.21.15.100:0/439301524 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5767+0+0 (secure 0 0 0) 0x7fb35009b180 con 0x7fb36415d760 2024-09-17T04:25:53.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.640+0000 7fb369c00640 1 --2- 172.21.15.100:0/439301524 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb344078d40 0x7fb34407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:53.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.641+0000 7fb369c00640 1 --2- 172.21.15.100:0/439301524 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb344078d40 0x7fb34407b250 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb358033000 tx=0x7fb358004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:53.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.645+0000 7fb361600640 1 -- 172.21.15.100:0/439301524 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb350062c40 con 0x7fb36415d760 2024-09-17T04:25:53.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.861+0000 7fb36c600640 1 -- 172.21.15.100:0/439301524 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb36415e5d0 con 0x7fb36415d760 2024-09-17T04:25:53.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.862+0000 7fb361600640 1 -- 172.21.15.100:0/439301524 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7fb350067670 con 0x7fb36415d760 2024-09-17T04:25:53.862 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:25:53.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.864+0000 7fb36c600640 1 -- 172.21.15.100:0/439301524 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb344078d40 msgr2=0x7fb34407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:53.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.864+0000 7fb36c600640 1 --2- 172.21.15.100:0/439301524 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb344078d40 0x7fb34407b250 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb358033000 tx=0x7fb358004730 comp rx=0 tx=0).stop 2024-09-17T04:25:53.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.864+0000 7fb36c600640 1 -- 172.21.15.100:0/439301524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb36415d760 msgr2=0x7fb364148710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:53.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.864+0000 7fb36c600640 1 --2- 172.21.15.100:0/439301524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb36415d760 0x7fb364148710 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb35000d700 tx=0x7fb35000dbc0 comp rx=0 tx=0).stop 2024-09-17T04:25:53.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.864+0000 7fb36c600640 1 -- 172.21.15.100:0/439301524 shutdown_connections 2024-09-17T04:25:53.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.864+0000 7fb36c600640 1 --2- 172.21.15.100:0/439301524 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb344078d40 0x7fb34407b250 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:53.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.864+0000 7fb36c600640 1 --2- 172.21.15.100:0/439301524 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb364160310 0x7fb36415b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:53.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.864+0000 7fb36c600640 1 --2- 172.21.15.100:0/439301524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb36415d760 0x7fb364148710 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:53.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.864+0000 7fb36c600640 1 --2- 172.21.15.100:0/439301524 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3640ad0e0 0x7fb36409a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:53.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.864+0000 7fb36c600640 1 -- 172.21.15.100:0/439301524 >> 172.21.15.100:0/439301524 conn(0x7fb36406d730 msgr2=0x7fb36415ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:53.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.864+0000 7fb36c600640 1 -- 172.21.15.100:0/439301524 shutdown_connections 2024-09-17T04:25:53.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:53.864+0000 7fb36c600640 1 -- 172.21.15.100:0/439301524 wait complete. 2024-09-17T04:25:53.886 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T04:25:53.886 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:25:53.886 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2024-09-17T04:25:53.887 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T04:25:53.887 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T04:25:53.887 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T04:25:53.887 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T04:25:53.887 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:25:53.887 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:25:53.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:53.887 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2024-09-17T04:25:53.928 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:25:53.928 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:25:53.928 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:25:53.928 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:25:53.928 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-09-17T04:25:53.928 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:25:53.928 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:25:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:25:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2024-09-17T04:25:53.929 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:25:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:25:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2024-09-17T04:25:53.929 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:25:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2024-09-17T04:25:53.929 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:25:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2024-09-17T04:25:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T04:25:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42590sec preferred_lft 42590sec 2024-09-17T04:25:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ec4:7aff:fe88:7b8f/64 scope link noprefixroute 2024-09-17T04:25:53.929 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:25:53.930 INFO:teuthology.orchestra.run.smithi184.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:25:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 7e:67:e5:69:96:69 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:25:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T04:25:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:25:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::3c9d:aff:fe6e:8fcb/64 scope link 2024-09-17T04:25:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:25:53.930 INFO:teuthology.orchestra.run.smithi184.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:25:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether a6:a9:1b:a9:61:ec brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T04:25:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::a4a9:1bff:fea9:61ec/64 scope link 2024-09-17T04:25:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:25:53.930 INFO:teuthology.orchestra.run.smithi184.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:25:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 7e:67:e5:69:96:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T04:25:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::7c67:e5ff:fe69:9669/64 scope link 2024-09-17T04:25:53.930 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:25:53.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:53.931 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:25:54.025 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-09-17T04:25:54.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:54.029 DEBUG:teuthology.orchestra.run.smithi184:> route 2024-09-17T04:25:54.065 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2024-09-17T04:25:54.066 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:25:54.066 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T04:25:54.066 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T04:25:54.066 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:25:54.066 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:25:54.066 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-09-17T04:25:54.066 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T04:25:54.066 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T04:25:54.066 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T04:25:54.066 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-09-17T04:25:54.184 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:25:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:25:54.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:25:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:25:54.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:54.266 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-09-17T04:25:54.302 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T04:25:54.302 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T04:25:54.302 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:25:54.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:54.302 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.0 up 2024-09-17T04:25:54.398 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:25:54.398 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T04:25:54.434 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:25:54.434 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:25:54.434 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T04:25:54.509 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T04:25:54.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:54.510 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:25:54.705 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:25:54.705 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:25:54.705 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:25:54.725 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:25:54.725 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:25:54.725 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:25:54.725 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:25:54.725 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:25:54.756 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:25:54.756 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:25:54.756 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:25:54.756 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:25:54.756 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:25:54.756 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-09-17T04:25:54.756 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-09-17T04:25:54.756 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8a399842-3a5c-4ea2-9afb-b2704332e59f" will pass to kernel 2024-09-17T04:25:54.757 INFO:teuthology.orchestra.run.smithi184.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-17T04:25:54.757 INFO:teuthology.orchestra.run.smithi184.stdout: You just mounted a file system that supports labels which does not 2024-09-17T04:25:54.757 INFO:teuthology.orchestra.run.smithi184.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-17T04:25:54.757 INFO:teuthology.orchestra.run.smithi184.stdout: applications will generate AVC messages and not be allowed access to 2024-09-17T04:25:54.758 INFO:teuthology.orchestra.run.smithi184.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-17T04:25:54.760 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T04:25:54.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:54.761 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T04:25:54.816 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:25:54.894 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:25:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:25:54.898 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:25:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:25:54.906 INFO:teuthology.orchestra.run.smithi184.stdout:client5433 2024-09-17T04:25:54.906 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:25:54.906 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:25:54.941 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T04:25:55.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.167+0000 7f5596400640 1 -- 172.21.15.100:0/2875582349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55900af100 msgr2=0x7f55900af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:55.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.167+0000 7f5596400640 1 --2- 172.21.15.100:0/2875582349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55900af100 0x7f55900af510 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f55900d0d00 tx=0x7f558802aac0 comp rx=0 tx=0).stop 2024-09-17T04:25:55.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.167+0000 7f5596400640 1 -- 172.21.15.100:0/2875582349 shutdown_connections 2024-09-17T04:25:55.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.167+0000 7f5596400640 1 --2- 172.21.15.100:0/2875582349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55900af100 0x7f55900af510 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:55.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.167+0000 7f5596400640 1 --2- 172.21.15.100:0/2875582349 >> v2:172.21.15.184:3300/0 conn(0x7f559006d4b0 0x7f55900b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:55.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.167+0000 7f5596400640 1 -- 172.21.15.100:0/2875582349 >> 172.21.15.100:0/2875582349 conn(0x7f55900b2540 msgr2=0x7f55900d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:55.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.167+0000 7f5596400640 1 -- 172.21.15.100:0/2875582349 shutdown_connections 2024-09-17T04:25:55.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.167+0000 7f5596400640 1 -- 172.21.15.100:0/2875582349 wait complete. 2024-09-17T04:25:55.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.170+0000 7f5596400640 1 Processor -- start 2024-09-17T04:25:55.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.170+0000 7f5596400640 1 -- start start 2024-09-17T04:25:55.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.171+0000 7f5596400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55900af100 0x7f559009c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:55.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.171+0000 7f5596400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55900cc830 0x7f5590098b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:55.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.171+0000 7f5596400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55900ae8d0 0x7f559009d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:55.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.171+0000 7f5596400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f55900b3670 con 0x7f55900af100 2024-09-17T04:25:55.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.171+0000 7f5596400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f55900b34f0 con 0x7f55900ae8d0 2024-09-17T04:25:55.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.171+0000 7f5596400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f55900b37f0 con 0x7f55900cc830 2024-09-17T04:25:55.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.171+0000 7f558f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55900af100 0x7f559009c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:55.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.171+0000 7f558f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55900af100 0x7f559009c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50416/0 (socket says 172.21.15.100:50416) 2024-09-17T04:25:55.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.171+0000 7f558f400640 1 -- 172.21.15.100:0/1676891707 learned_addr learned my addr 172.21.15.100:0/1676891707 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:55.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.172+0000 7f558ea00640 1 --2- 172.21.15.100:0/1676891707 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55900cc830 0x7f5590098b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:55.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.172+0000 7f558fe00640 1 --2- 172.21.15.100:0/1676891707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55900ae8d0 0x7f559009d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:55.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.172+0000 7f558f400640 1 -- 172.21.15.100:0/1676891707 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55900cc830 msgr2=0x7f5590098b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:55.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.172+0000 7f558f400640 1 --2- 172.21.15.100:0/1676891707 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55900cc830 0x7f5590098b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:55.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.172+0000 7f558f400640 1 -- 172.21.15.100:0/1676891707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55900ae8d0 msgr2=0x7f559009d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:55.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.172+0000 7f558f400640 1 --2- 172.21.15.100:0/1676891707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55900ae8d0 0x7f559009d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:55.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.172+0000 7f558f400640 1 -- 172.21.15.100:0/1676891707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559009de40 con 0x7f55900af100 2024-09-17T04:25:55.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.172+0000 7f558f400640 1 --2- 172.21.15.100:0/1676891707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55900af100 0x7f559009c870 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f558000c3d0 tx=0x7f558000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:55.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.172+0000 7f5587400640 1 -- 172.21.15.100:0/1676891707 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5580017070 con 0x7f55900af100 2024-09-17T04:25:55.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.172+0000 7f5587400640 1 -- 172.21.15.100:0/1676891707 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55800053c0 con 0x7f55900af100 2024-09-17T04:25:55.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.172+0000 7f5596400640 1 -- 172.21.15.100:0/1676891707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f559009bd90 con 0x7f55900af100 2024-09-17T04:25:55.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.173+0000 7f5587400640 1 -- 172.21.15.100:0/1676891707 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5580003850 con 0x7f55900af100 2024-09-17T04:25:55.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.173+0000 7f5596400640 1 -- 172.21.15.100:0/1676891707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55901f4830 con 0x7f55900af100 2024-09-17T04:25:55.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.174+0000 7f5596400640 1 -- 172.21.15.100:0/1676891707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55901f4a90 con 0x7f55900af100 2024-09-17T04:25:55.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.175+0000 7f5587400640 1 -- 172.21.15.100:0/1676891707 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f558001a070 con 0x7f55900af100 2024-09-17T04:25:55.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.176+0000 7f5587400640 1 --2- 172.21.15.100:0/1676891707 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5570078d70 0x7f557007b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:55.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.176+0000 7f5587400640 1 -- 172.21.15.100:0/1676891707 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f558009a310 con 0x7f55900af100 2024-09-17T04:25:55.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.176+0000 7f558ea00640 1 --2- 172.21.15.100:0/1676891707 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5570078d70 0x7f557007b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:55.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.177+0000 7f558ea00640 1 --2- 172.21.15.100:0/1676891707 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5570078d70 0x7f557007b280 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f55900730b0 tx=0x7f557c00b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:55.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.180+0000 7f5587400640 1 -- 172.21.15.100:0/1676891707 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f5580061e60 con 0x7f55900af100 2024-09-17T04:25:55.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:55.400+0000 7f5596400640 1 -- 172.21.15.100:0/1676891707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f55900b3d10 con 0x7f55900af100 2024-09-17T04:25:56.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.252+0000 7f5587400640 1 -- 172.21.15.100:0/1676891707 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v27) ==== 136+0+0 (secure 0 0 0) 0x7f5580066890 con 0x7f55900af100 2024-09-17T04:25:56.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.255+0000 7f5596400640 1 -- 172.21.15.100:0/1676891707 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5570078d70 msgr2=0x7f557007b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:56.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.255+0000 7f5596400640 1 --2- 172.21.15.100:0/1676891707 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5570078d70 0x7f557007b280 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f55900730b0 tx=0x7f557c00b2e0 comp rx=0 tx=0).stop 2024-09-17T04:25:56.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.255+0000 7f5596400640 1 -- 172.21.15.100:0/1676891707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55900af100 msgr2=0x7f559009c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:56.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.255+0000 7f5596400640 1 --2- 172.21.15.100:0/1676891707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55900af100 0x7f559009c870 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f558000c3d0 tx=0x7f558000c890 comp rx=0 tx=0).stop 2024-09-17T04:25:56.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.255+0000 7f5596400640 1 -- 172.21.15.100:0/1676891707 shutdown_connections 2024-09-17T04:25:56.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.255+0000 7f5596400640 1 --2- 172.21.15.100:0/1676891707 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5570078d70 0x7f557007b280 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:56.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.255+0000 7f5596400640 1 --2- 172.21.15.100:0/1676891707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55900ae8d0 0x7f559009d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:56.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.255+0000 7f5596400640 1 --2- 172.21.15.100:0/1676891707 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55900cc830 0x7f5590098b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:56.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.255+0000 7f5596400640 1 --2- 172.21.15.100:0/1676891707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55900af100 0x7f559009c870 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:56.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.255+0000 7f5596400640 1 -- 172.21.15.100:0/1676891707 >> 172.21.15.100:0/1676891707 conn(0x7f55900b2540 msgr2=0x7f5590068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:56.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.255+0000 7f5596400640 1 -- 172.21.15.100:0/1676891707 shutdown_connections 2024-09-17T04:25:56.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.255+0000 7f5596400640 1 -- 172.21.15.100:0/1676891707 wait complete. 2024-09-17T04:25:56.285 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T04:25:56.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.507+0000 7f5407200640 1 -- 172.21.15.100:0/1178673097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54080af170 msgr2=0x7f54080b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:56.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.507+0000 7f5407200640 1 --2- 172.21.15.100:0/1178673097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54080af170 0x7f54080b2f60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f53f4009830 tx=0x7f53f402a4a0 comp rx=0 tx=0).stop 2024-09-17T04:25:56.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.508+0000 7f5407200640 1 -- 172.21.15.100:0/1178673097 shutdown_connections 2024-09-17T04:25:56.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.508+0000 7f5407200640 1 --2- 172.21.15.100:0/1178673097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54080af170 0x7f54080b2f60 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:56.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.508+0000 7f5407200640 1 -- 172.21.15.100:0/1178673097 >> 172.21.15.100:0/1178673097 conn(0x7f54080b1e40 msgr2=0x7f54080b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:56.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.508+0000 7f5407200640 1 -- 172.21.15.100:0/1178673097 shutdown_connections 2024-09-17T04:25:56.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.508+0000 7f5407200640 1 -- 172.21.15.100:0/1178673097 wait complete. 2024-09-17T04:25:56.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.510+0000 7f5407200640 1 Processor -- start 2024-09-17T04:25:56.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.510+0000 7f5407200640 1 -- start start 2024-09-17T04:25:56.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.511+0000 7f5407200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54080ad9c0 0x7f54081f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:56.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.512+0000 7f5407200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54080af170 0x7f54080c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:56.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.512+0000 7f5407200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f540815a0f0 0x7f5408154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:56.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.512+0000 7f5407200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f540806fb20 con 0x7f540815a0f0 2024-09-17T04:25:56.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.512+0000 7f5407200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f540806f9a0 con 0x7f54080af170 2024-09-17T04:25:56.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.512+0000 7f5407200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f540815d670 con 0x7f54080ad9c0 2024-09-17T04:25:56.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.512+0000 7f5406800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f540815a0f0 0x7f5408154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:56.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.512+0000 7f5406800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f540815a0f0 0x7f5408154c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50422/0 (socket says 172.21.15.100:50422) 2024-09-17T04:25:56.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.512+0000 7f5405400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54080af170 0x7f54080c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:56.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.512+0000 7f5406800640 1 -- 172.21.15.100:0/3624245281 learned_addr learned my addr 172.21.15.100:0/3624245281 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:56.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.512+0000 7f5405e00640 1 --2- 172.21.15.100:0/3624245281 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54080ad9c0 0x7f54081f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:56.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.512+0000 7f5406800640 1 -- 172.21.15.100:0/3624245281 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54080ad9c0 msgr2=0x7f54081f51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:56.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.512+0000 7f5406800640 1 --2- 172.21.15.100:0/3624245281 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54080ad9c0 0x7f54081f51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:56.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.512+0000 7f5406800640 1 -- 172.21.15.100:0/3624245281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54080af170 msgr2=0x7f54080c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:56.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.512+0000 7f5406800640 1 --2- 172.21.15.100:0/3624245281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54080af170 0x7f54080c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:56.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.512+0000 7f5406800640 1 -- 172.21.15.100:0/3624245281 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54081477a0 con 0x7f540815a0f0 2024-09-17T04:25:56.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.513+0000 7f5406800640 1 --2- 172.21.15.100:0/3624245281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f540815a0f0 0x7f5408154c00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f540000c760 tx=0x7f540000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:56.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.513+0000 7f53fe000640 1 -- 172.21.15.100:0/3624245281 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5400003da0 con 0x7f540815a0f0 2024-09-17T04:25:56.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.513+0000 7f53fe000640 1 -- 172.21.15.100:0/3624245281 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f540000ce10 con 0x7f540815a0f0 2024-09-17T04:25:56.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.513+0000 7f5407200640 1 -- 172.21.15.100:0/3624245281 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5408147a80 con 0x7f540815a0f0 2024-09-17T04:25:56.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.513+0000 7f53fe000640 1 -- 172.21.15.100:0/3624245281 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54000052d0 con 0x7f540815a0f0 2024-09-17T04:25:56.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.513+0000 7f5407200640 1 -- 172.21.15.100:0/3624245281 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5408148040 con 0x7f540815a0f0 2024-09-17T04:25:56.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.514+0000 7f5407200640 1 -- 172.21.15.100:0/3624245281 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54080b2f60 con 0x7f540815a0f0 2024-09-17T04:25:56.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.515+0000 7f53fe000640 1 -- 172.21.15.100:0/3624245281 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f5400005470 con 0x7f540815a0f0 2024-09-17T04:25:56.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.517+0000 7f53fe000640 1 --2- 172.21.15.100:0/3624245281 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f53e8078d70 0x7f53e807b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:56.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.517+0000 7f53fe000640 1 -- 172.21.15.100:0/3624245281 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f540009ad90 con 0x7f540815a0f0 2024-09-17T04:25:56.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.517+0000 7f5405e00640 1 --2- 172.21.15.100:0/3624245281 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f53e8078d70 0x7f53e807b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:56.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.518+0000 7f5405e00640 1 --2- 172.21.15.100:0/3624245281 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f53e8078d70 0x7f53e807b280 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f53f8000f60 tx=0x7f53f8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:56.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.522+0000 7f53fe000640 1 -- 172.21.15.100:0/3624245281 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f5400062830 con 0x7f540815a0f0 2024-09-17T04:25:56.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.724+0000 7f5407200640 1 -- 172.21.15.100:0/3624245281 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f54080b0b10 con 0x7f53e8078d70 2024-09-17T04:25:56.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.726+0000 7f53fe000640 1 -- 172.21.15.100:0/3624245281 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f54080b0b10 con 0x7f53e8078d70 2024-09-17T04:25:56.726 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-09-17T04:25:56.726 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:25:56.726 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephfs" 2024-09-17T04:25:56.726 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:25:56.726 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-09-17T04:25:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.728+0000 7f5407200640 1 -- 172.21.15.100:0/3624245281 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f53e8078d70 msgr2=0x7f53e807b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.728+0000 7f5407200640 1 --2- 172.21.15.100:0/3624245281 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f53e8078d70 0x7f53e807b280 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f53f8000f60 tx=0x7f53f8009040 comp rx=0 tx=0).stop 2024-09-17T04:25:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.728+0000 7f5407200640 1 -- 172.21.15.100:0/3624245281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f540815a0f0 msgr2=0x7f5408154c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.728+0000 7f5407200640 1 --2- 172.21.15.100:0/3624245281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f540815a0f0 0x7f5408154c00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f540000c760 tx=0x7f540000cc20 comp rx=0 tx=0).stop 2024-09-17T04:25:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.729+0000 7f5407200640 1 -- 172.21.15.100:0/3624245281 shutdown_connections 2024-09-17T04:25:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.729+0000 7f5407200640 1 --2- 172.21.15.100:0/3624245281 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f53e8078d70 0x7f53e807b280 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:56.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.729+0000 7f5407200640 1 --2- 172.21.15.100:0/3624245281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f540815a0f0 0x7f5408154c00 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:56.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.729+0000 7f5407200640 1 --2- 172.21.15.100:0/3624245281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54080af170 0x7f54080c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:56.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.729+0000 7f5407200640 1 --2- 172.21.15.100:0/3624245281 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54080ad9c0 0x7f54081f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:56.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.729+0000 7f5407200640 1 -- 172.21.15.100:0/3624245281 >> 172.21.15.100:0/3624245281 conn(0x7f54080b1e40 msgr2=0x7f54080b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:56.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.729+0000 7f5407200640 1 -- 172.21.15.100:0/3624245281 shutdown_connections 2024-09-17T04:25:56.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.729+0000 7f5407200640 1 -- 172.21.15.100:0/3624245281 wait complete. 2024-09-17T04:25:56.750 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T04:25:56.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.974+0000 7fa24e800640 1 -- 172.21.15.100:0/1964523496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2480d24f0 msgr2=0x7fa2480d29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:56.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.974+0000 7fa24e800640 1 --2- 172.21.15.100:0/1964523496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2480d24f0 0x7fa2480d29c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa23c0098e0 tx=0x7fa23c02a5d0 comp rx=0 tx=0).stop 2024-09-17T04:25:56.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.974+0000 7fa24e800640 1 -- 172.21.15.100:0/1964523496 shutdown_connections 2024-09-17T04:25:56.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.974+0000 7fa24e800640 1 --2- 172.21.15.100:0/1964523496 >> v2:172.21.15.184:3300/0 conn(0x7fa2480d2f00 0x7fa2480d4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:56.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.974+0000 7fa24e800640 1 --2- 172.21.15.100:0/1964523496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2480d24f0 0x7fa2480d29c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:56.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.974+0000 7fa24e800640 1 -- 172.21.15.100:0/1964523496 >> 172.21.15.100:0/1964523496 conn(0x7fa24808dd50 msgr2=0x7fa24809d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:56.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.974+0000 7fa24e800640 1 -- 172.21.15.100:0/1964523496 shutdown_connections 2024-09-17T04:25:56.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.974+0000 7fa24e800640 1 -- 172.21.15.100:0/1964523496 wait complete. 2024-09-17T04:25:56.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.977+0000 7fa24e800640 1 Processor -- start 2024-09-17T04:25:56.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.977+0000 7fa24e800640 1 -- start start 2024-09-17T04:25:56.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.978+0000 7fa24e800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2480d24f0 0x7fa2481f9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:56.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.978+0000 7fa24e800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2480d2f00 0x7fa24814f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:56.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.978+0000 7fa24e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa248162100 0x7fa2480b2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:56.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.978+0000 7fa24e800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa2480cff40 con 0x7fa248162100 2024-09-17T04:25:56.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.978+0000 7fa24e800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa2480d4df0 con 0x7fa2480d2f00 2024-09-17T04:25:56.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.978+0000 7fa24e800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa2480d00c0 con 0x7fa2480d24f0 2024-09-17T04:25:56.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.978+0000 7fa24de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa248162100 0x7fa2480b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:56.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.979+0000 7fa24d400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2480d24f0 0x7fa2481f9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:56.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.979+0000 7fa24de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa248162100 0x7fa2480b2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50436/0 (socket says 172.21.15.100:50436) 2024-09-17T04:25:56.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.979+0000 7fa24de00640 1 -- 172.21.15.100:0/760950458 learned_addr learned my addr 172.21.15.100:0/760950458 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:56.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.979+0000 7fa24ca00640 1 --2- 172.21.15.100:0/760950458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2480d2f00 0x7fa24814f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:56.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.979+0000 7fa24de00640 1 -- 172.21.15.100:0/760950458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2480d24f0 msgr2=0x7fa2481f9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:56.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.979+0000 7fa24de00640 1 --2- 172.21.15.100:0/760950458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2480d24f0 0x7fa2481f9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:56.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.979+0000 7fa24de00640 1 -- 172.21.15.100:0/760950458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2480d2f00 msgr2=0x7fa24814f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:56.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.979+0000 7fa24de00640 1 --2- 172.21.15.100:0/760950458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2480d2f00 0x7fa24814f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:56.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.979+0000 7fa24de00640 1 -- 172.21.15.100:0/760950458 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2480a1ab0 con 0x7fa248162100 2024-09-17T04:25:56.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.979+0000 7fa24de00640 1 --2- 172.21.15.100:0/760950458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa248162100 0x7fa2480b2360 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa24000c6b0 tx=0x7fa24000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:56.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.980+0000 7fa245600640 1 -- 172.21.15.100:0/760950458 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240003b70 con 0x7fa248162100 2024-09-17T04:25:56.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.980+0000 7fa245600640 1 -- 172.21.15.100:0/760950458 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa240003d30 con 0x7fa248162100 2024-09-17T04:25:56.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.980+0000 7fa245600640 1 -- 172.21.15.100:0/760950458 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240005330 con 0x7fa248162100 2024-09-17T04:25:56.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.980+0000 7fa24e800640 1 -- 172.21.15.100:0/760950458 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2480a1d90 con 0x7fa248162100 2024-09-17T04:25:56.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.980+0000 7fa24e800640 1 -- 172.21.15.100:0/760950458 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa24814fc70 con 0x7fa248162100 2024-09-17T04:25:56.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.981+0000 7fa24e800640 1 -- 172.21.15.100:0/760950458 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2480d0570 con 0x7fa248162100 2024-09-17T04:25:56.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.982+0000 7fa245600640 1 -- 172.21.15.100:0/760950458 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fa2400054d0 con 0x7fa248162100 2024-09-17T04:25:56.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.983+0000 7fa245600640 1 --2- 172.21.15.100:0/760950458 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa228078d40 0x7fa22807b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:56.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.983+0000 7fa245600640 1 -- 172.21.15.100:0/760950458 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7fa24009aed0 con 0x7fa248162100 2024-09-17T04:25:56.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.983+0000 7fa24d400640 1 --2- 172.21.15.100:0/760950458 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa228078d40 0x7fa22807b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:56.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.984+0000 7fa24d400640 1 --2- 172.21.15.100:0/760950458 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa228078d40 0x7fa22807b250 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa23c034000 tx=0x7fa23c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:56.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:56.988+0000 7fa245600640 1 -- 172.21.15.100:0/760950458 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa240062850 con 0x7fa248162100 2024-09-17T04:25:57.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.172+0000 7fa24e800640 1 -- 172.21.15.100:0/760950458 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa2480b4760 con 0x7fa248162100 2024-09-17T04:25:57.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.172+0000 7fa245600640 1 -- 172.21.15.100:0/760950458 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7fa240067280 con 0x7fa248162100 2024-09-17T04:25:57.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.175+0000 7fa24e800640 1 -- 172.21.15.100:0/760950458 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa228078d40 msgr2=0x7fa22807b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:57.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.175+0000 7fa24e800640 1 --2- 172.21.15.100:0/760950458 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa228078d40 0x7fa22807b250 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa23c034000 tx=0x7fa23c004730 comp rx=0 tx=0).stop 2024-09-17T04:25:57.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.175+0000 7fa24e800640 1 -- 172.21.15.100:0/760950458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa248162100 msgr2=0x7fa2480b2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:57.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.175+0000 7fa24e800640 1 --2- 172.21.15.100:0/760950458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa248162100 0x7fa2480b2360 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa24000c6b0 tx=0x7fa24000cb70 comp rx=0 tx=0).stop 2024-09-17T04:25:57.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.175+0000 7fa24e800640 1 -- 172.21.15.100:0/760950458 shutdown_connections 2024-09-17T04:25:57.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.175+0000 7fa24e800640 1 --2- 172.21.15.100:0/760950458 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa228078d40 0x7fa22807b250 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:57.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.175+0000 7fa24e800640 1 --2- 172.21.15.100:0/760950458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa248162100 0x7fa2480b2360 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:57.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.175+0000 7fa24e800640 1 --2- 172.21.15.100:0/760950458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2480d2f00 0x7fa24814f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:57.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.175+0000 7fa24e800640 1 --2- 172.21.15.100:0/760950458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2480d24f0 0x7fa2481f9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:57.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.175+0000 7fa24e800640 1 -- 172.21.15.100:0/760950458 >> 172.21.15.100:0/760950458 conn(0x7fa24808dd50 msgr2=0x7fa2480b9450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:57.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.175+0000 7fa24e800640 1 -- 172.21.15.100:0/760950458 shutdown_connections 2024-09-17T04:25:57.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.175+0000 7fa24e800640 1 -- 172.21.15.100:0/760950458 wait complete. 2024-09-17T04:25:57.198 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:25:57.198 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:25:57.233 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:25:57.233 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:25:57.309 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:25:57.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.570+0000 7f3a49000640 1 -- 172.21.15.100:0/4243520408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a44070660 msgr2=0x7f3a440af6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:57.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.570+0000 7f3a49000640 1 --2- 172.21.15.100:0/4243520408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a44070660 0x7f3a440af6a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3a34005fd0 tx=0x7f3a3402a7c0 comp rx=0 tx=0).stop 2024-09-17T04:25:57.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.570+0000 7f3a49000640 1 -- 172.21.15.100:0/4243520408 shutdown_connections 2024-09-17T04:25:57.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.570+0000 7f3a49000640 1 --2- 172.21.15.100:0/4243520408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a44070660 0x7f3a440af6a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:57.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.570+0000 7f3a49000640 1 -- 172.21.15.100:0/4243520408 >> 172.21.15.100:0/4243520408 conn(0x7f3a4406d3d0 msgr2=0x7f3a440b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:57.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.570+0000 7f3a49000640 1 -- 172.21.15.100:0/4243520408 shutdown_connections 2024-09-17T04:25:57.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.570+0000 7f3a49000640 1 -- 172.21.15.100:0/4243520408 wait complete. 2024-09-17T04:25:57.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.572+0000 7f3a49000640 1 Processor -- start 2024-09-17T04:25:57.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.572+0000 7f3a49000640 1 -- start start 2024-09-17T04:25:57.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.574+0000 7f3a49000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a44070660 0x7f3a44151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:57.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.574+0000 7f3a49000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a440ce490 0x7f3a440b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:57.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.574+0000 7f3a49000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a440ce8a0 0x7f3a440b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:57.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.574+0000 7f3a49000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3a440cbad0 con 0x7f3a440ce8a0 2024-09-17T04:25:57.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.574+0000 7f3a49000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3a440cb950 con 0x7f3a44070660 2024-09-17T04:25:57.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.574+0000 7f3a49000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3a44166ef0 con 0x7f3a440ce490 2024-09-17T04:25:57.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.574+0000 7f3a42800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a440ce8a0 0x7f3a440b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:57.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.574+0000 7f3a42800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a440ce8a0 0x7f3a440b1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50452/0 (socket says 172.21.15.100:50452) 2024-09-17T04:25:57.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.574+0000 7f3a42800640 1 -- 172.21.15.100:0/1801312722 learned_addr learned my addr 172.21.15.100:0/1801312722 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:57.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.574+0000 7f3a41e00640 1 --2- 172.21.15.100:0/1801312722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a44070660 0x7f3a44151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:57.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.574+0000 7f3a39400640 1 --2- 172.21.15.100:0/1801312722 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a440ce490 0x7f3a440b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:57.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.574+0000 7f3a42800640 1 -- 172.21.15.100:0/1801312722 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a440ce490 msgr2=0x7f3a440b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:57.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.574+0000 7f3a42800640 1 --2- 172.21.15.100:0/1801312722 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a440ce490 0x7f3a440b0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:57.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.574+0000 7f3a42800640 1 -- 172.21.15.100:0/1801312722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a44070660 msgr2=0x7f3a44151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:57.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.574+0000 7f3a42800640 1 --2- 172.21.15.100:0/1801312722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a44070660 0x7f3a44151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:57.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.574+0000 7f3a42800640 1 -- 172.21.15.100:0/1801312722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a44159580 con 0x7f3a440ce8a0 2024-09-17T04:25:57.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.575+0000 7f3a42800640 1 --2- 172.21.15.100:0/1801312722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a440ce8a0 0x7f3a440b1430 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00c3f0 tx=0x7f3a3c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:57.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.575+0000 7f3a3aa00640 1 -- 172.21.15.100:0/1801312722 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c017070 con 0x7f3a440ce8a0 2024-09-17T04:25:57.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.575+0000 7f3a3aa00640 1 -- 172.21.15.100:0/1801312722 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a3c0053c0 con 0x7f3a440ce8a0 2024-09-17T04:25:57.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.575+0000 7f3a3aa00640 1 -- 172.21.15.100:0/1801312722 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c003850 con 0x7f3a440ce8a0 2024-09-17T04:25:57.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.575+0000 7f3a49000640 1 -- 172.21.15.100:0/1801312722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a44159860 con 0x7f3a440ce8a0 2024-09-17T04:25:57.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.575+0000 7f3a49000640 1 -- 172.21.15.100:0/1801312722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a440d0db0 con 0x7f3a440ce8a0 2024-09-17T04:25:57.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.576+0000 7f3a49000640 1 -- 172.21.15.100:0/1801312722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a440af6a0 con 0x7f3a440ce8a0 2024-09-17T04:25:57.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.577+0000 7f3a3aa00640 1 -- 172.21.15.100:0/1801312722 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f3a3c01a070 con 0x7f3a440ce8a0 2024-09-17T04:25:57.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.578+0000 7f3a3aa00640 1 --2- 172.21.15.100:0/1801312722 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3a20078d70 0x7f3a2007b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:57.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.578+0000 7f3a3aa00640 1 -- 172.21.15.100:0/1801312722 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f3a3c09a480 con 0x7f3a440ce8a0 2024-09-17T04:25:57.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.579+0000 7f3a41e00640 1 --2- 172.21.15.100:0/1801312722 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3a20078d70 0x7f3a2007b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:57.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.580+0000 7f3a41e00640 1 --2- 172.21.15.100:0/1801312722 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3a20078d70 0x7f3a2007b280 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3a34033000 tx=0x7f3a34004ab0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:57.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.582+0000 7f3a3aa00640 1 -- 172.21.15.100:0/1801312722 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3a3c061ed0 con 0x7f3a440ce8a0 2024-09-17T04:25:57.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.776+0000 7f3a49000640 1 -- 172.21.15.100:0/1801312722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3a440b2660 con 0x7f3a440ce8a0 2024-09-17T04:25:57.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.776+0000 7f3a3aa00640 1 -- 172.21.15.100:0/1801312722 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v50) ==== 80+0+0 (secure 0 0 0) 0x7f3a3c066900 con 0x7f3a440ce8a0 2024-09-17T04:25:57.776 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-09-17T04:25:57.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.779+0000 7f3a49000640 1 -- 172.21.15.100:0/1801312722 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3a20078d70 msgr2=0x7f3a2007b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:57.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.779+0000 7f3a49000640 1 --2- 172.21.15.100:0/1801312722 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3a20078d70 0x7f3a2007b280 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3a34033000 tx=0x7f3a34004ab0 comp rx=0 tx=0).stop 2024-09-17T04:25:57.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.779+0000 7f3a49000640 1 -- 172.21.15.100:0/1801312722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a440ce8a0 msgr2=0x7f3a440b1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:57.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.779+0000 7f3a49000640 1 --2- 172.21.15.100:0/1801312722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a440ce8a0 0x7f3a440b1430 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00c3f0 tx=0x7f3a3c00c8b0 comp rx=0 tx=0).stop 2024-09-17T04:25:57.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.779+0000 7f3a49000640 1 -- 172.21.15.100:0/1801312722 shutdown_connections 2024-09-17T04:25:57.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.779+0000 7f3a49000640 1 --2- 172.21.15.100:0/1801312722 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3a20078d70 0x7f3a2007b280 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:57.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.779+0000 7f3a49000640 1 --2- 172.21.15.100:0/1801312722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a440ce8a0 0x7f3a440b1430 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:57.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.779+0000 7f3a49000640 1 --2- 172.21.15.100:0/1801312722 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a440ce490 0x7f3a440b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:57.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.779+0000 7f3a49000640 1 --2- 172.21.15.100:0/1801312722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a44070660 0x7f3a44151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:57.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.779+0000 7f3a49000640 1 -- 172.21.15.100:0/1801312722 >> 172.21.15.100:0/1801312722 conn(0x7f3a4406d3d0 msgr2=0x7f3a440ccd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:57.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.779+0000 7f3a49000640 1 -- 172.21.15.100:0/1801312722 shutdown_connections 2024-09-17T04:25:57.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:57.779+0000 7f3a49000640 1 -- 172.21.15.100:0/1801312722 wait complete. 2024-09-17T04:25:57.803 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T04:25:57.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:57.803 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T04:25:57.866 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-09-17T04:25:57.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:57.867 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T04:25:57.902 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:25:57.902 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:25:57.942 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:25:57.942 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:25:58.022 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:25:58.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.242+0000 7f0c26200640 1 -- 172.21.15.100:0/3438660775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c20161fa0 msgr2=0x7f0c200b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.242+0000 7f0c26200640 1 --2- 172.21.15.100:0/3438660775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c20161fa0 0x7f0c200b39c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0c08005fd0 tx=0x7f0c0802a540 comp rx=0 tx=0).stop 2024-09-17T04:25:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.242+0000 7f0c26200640 1 -- 172.21.15.100:0/3438660775 shutdown_connections 2024-09-17T04:25:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.242+0000 7f0c26200640 1 --2- 172.21.15.100:0/3438660775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c20161fa0 0x7f0c200b39c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.242+0000 7f0c26200640 1 --2- 172.21.15.100:0/3438660775 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c2006f710 0x7f0c200b94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.242+0000 7f0c26200640 1 -- 172.21.15.100:0/3438660775 >> 172.21.15.100:0/3438660775 conn(0x7f0c2008dc30 msgr2=0x7f0c2009d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.242+0000 7f0c26200640 1 -- 172.21.15.100:0/3438660775 shutdown_connections 2024-09-17T04:25:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.242+0000 7f0c26200640 1 -- 172.21.15.100:0/3438660775 wait complete. 2024-09-17T04:25:58.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.244+0000 7f0c26200640 1 Processor -- start 2024-09-17T04:25:58.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.244+0000 7f0c26200640 1 -- start start 2024-09-17T04:25:58.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.246+0000 7f0c26200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c2006f710 0x7f0c200ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:58.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.246+0000 7f0c26200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c200d0cc0 0x7f0c200a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:58.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.246+0000 7f0c26200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c20161fa0 0x7f0c200a2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:58.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.246+0000 7f0c26200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0c200d2a20 con 0x7f0c200d0cc0 2024-09-17T04:25:58.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.246+0000 7f0c26200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0c200d28a0 con 0x7f0c20161fa0 2024-09-17T04:25:58.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.246+0000 7f0c26200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0c200d2ba0 con 0x7f0c2006f710 2024-09-17T04:25:58.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.246+0000 7f0c17e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c200d0cc0 0x7f0c200a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:58.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.246+0000 7f0c24e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c2006f710 0x7f0c200ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:58.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.246+0000 7f0c17e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c200d0cc0 0x7f0c200a1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50462/0 (socket says 172.21.15.100:50462) 2024-09-17T04:25:58.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.246+0000 7f0c24e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c2006f710 0x7f0c200ccd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.100:58310/0 (socket says 172.21.15.100:58310) 2024-09-17T04:25:58.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.246+0000 7f0c17e00640 1 -- 172.21.15.100:0/1173560502 learned_addr learned my addr 172.21.15.100:0/1173560502 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:58.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.246+0000 7f0c25800640 1 --2- 172.21.15.100:0/1173560502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c20161fa0 0x7f0c200a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:58.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.247+0000 7f0c17e00640 1 -- 172.21.15.100:0/1173560502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c2006f710 msgr2=0x7f0c200ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:58.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.247+0000 7f0c17e00640 1 --2- 172.21.15.100:0/1173560502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c2006f710 0x7f0c200ccd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:58.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.247+0000 7f0c17e00640 1 -- 172.21.15.100:0/1173560502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c20161fa0 msgr2=0x7f0c200a2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:58.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.247+0000 7f0c17e00640 1 --2- 172.21.15.100:0/1173560502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c20161fa0 0x7f0c200a2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:58.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.247+0000 7f0c17e00640 1 -- 172.21.15.100:0/1173560502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c201f5a50 con 0x7f0c200d0cc0 2024-09-17T04:25:58.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.247+0000 7f0c17e00640 1 --2- 172.21.15.100:0/1173560502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c200d0cc0 0x7f0c200a1ad0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0c08034040 tx=0x7f0c080038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:58.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.247+0000 7f0c1d600640 1 -- 172.21.15.100:0/1173560502 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0803e030 con 0x7f0c200d0cc0 2024-09-17T04:25:58.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.247+0000 7f0c1d600640 1 -- 172.21.15.100:0/1173560502 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c08003d90 con 0x7f0c200d0cc0 2024-09-17T04:25:58.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.247+0000 7f0c1d600640 1 -- 172.21.15.100:0/1173560502 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08035040 con 0x7f0c200d0cc0 2024-09-17T04:25:58.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.247+0000 7f0c26200640 1 -- 172.21.15.100:0/1173560502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c201f5d30 con 0x7f0c200d0cc0 2024-09-17T04:25:58.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.247+0000 7f0c26200640 1 -- 172.21.15.100:0/1173560502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c201f62f0 con 0x7f0c200d0cc0 2024-09-17T04:25:58.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.249+0000 7f0c26200640 1 -- 172.21.15.100:0/1173560502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c200d3050 con 0x7f0c200d0cc0 2024-09-17T04:25:58.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.250+0000 7f0c1d600640 1 -- 172.21.15.100:0/1173560502 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f0c08041030 con 0x7f0c200d0cc0 2024-09-17T04:25:58.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.251+0000 7f0c1d600640 1 --2- 172.21.15.100:0/1173560502 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0bfc078da0 0x7f0bfc07b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:58.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.251+0000 7f0c1d600640 1 -- 172.21.15.100:0/1173560502 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f0c080bbac0 con 0x7f0c200d0cc0 2024-09-17T04:25:58.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.251+0000 7f0c24e00640 1 --2- 172.21.15.100:0/1173560502 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0bfc078da0 0x7f0bfc07b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:58.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.252+0000 7f0c24e00640 1 --2- 172.21.15.100:0/1173560502 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0bfc078da0 0x7f0bfc07b2b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0c10004610 tx=0x7f0c1000b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:58.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.255+0000 7f0c1d600640 1 -- 172.21.15.100:0/1173560502 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0c08083610 con 0x7f0c200d0cc0 2024-09-17T04:25:58.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.450+0000 7f0c26200640 1 -- 172.21.15.100:0/1173560502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0c201f57d0 con 0x7f0c200d0cc0 2024-09-17T04:25:58.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.451+0000 7f0c1d600640 1 -- 172.21.15.100:0/1173560502 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v50) ==== 80+0+0 (secure 0 0 0) 0x7f0c201f57d0 con 0x7f0c200d0cc0 2024-09-17T04:25:58.450 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-09-17T04:25:58.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.453+0000 7f0c26200640 1 -- 172.21.15.100:0/1173560502 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0bfc078da0 msgr2=0x7f0bfc07b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:58.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.453+0000 7f0c26200640 1 --2- 172.21.15.100:0/1173560502 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0bfc078da0 0x7f0bfc07b2b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0c10004610 tx=0x7f0c1000b2e0 comp rx=0 tx=0).stop 2024-09-17T04:25:58.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.453+0000 7f0c26200640 1 -- 172.21.15.100:0/1173560502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c200d0cc0 msgr2=0x7f0c200a1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:58.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.453+0000 7f0c26200640 1 --2- 172.21.15.100:0/1173560502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c200d0cc0 0x7f0c200a1ad0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0c08034040 tx=0x7f0c080038a0 comp rx=0 tx=0).stop 2024-09-17T04:25:58.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.453+0000 7f0c26200640 1 -- 172.21.15.100:0/1173560502 shutdown_connections 2024-09-17T04:25:58.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.453+0000 7f0c26200640 1 --2- 172.21.15.100:0/1173560502 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0bfc078da0 0x7f0bfc07b2b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:58.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.453+0000 7f0c26200640 1 --2- 172.21.15.100:0/1173560502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c20161fa0 0x7f0c200a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:58.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.454+0000 7f0c26200640 1 --2- 172.21.15.100:0/1173560502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c200d0cc0 0x7f0c200a1ad0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:58.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.454+0000 7f0c26200640 1 --2- 172.21.15.100:0/1173560502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c2006f710 0x7f0c200ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:58.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.454+0000 7f0c26200640 1 -- 172.21.15.100:0/1173560502 >> 172.21.15.100:0/1173560502 conn(0x7f0c2008dc30 msgr2=0x7f0c200d10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:58.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.454+0000 7f0c26200640 1 -- 172.21.15.100:0/1173560502 shutdown_connections 2024-09-17T04:25:58.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.454+0000 7f0c26200640 1 -- 172.21.15.100:0/1173560502 wait complete. 2024-09-17T04:25:58.475 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-17T04:25:58.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:58.475 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-17T04:25:58.543 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2024-09-17T04:25:58.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:25:58.543 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T04:25:58.581 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:25:58.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.800+0000 7ff719800640 1 -- 172.21.15.100:0/1696533903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff714160310 msgr2=0x7ff714160720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:58.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.800+0000 7ff719800640 1 --2- 172.21.15.100:0/1696533903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff714160310 0x7ff714160720 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff714075250 tx=0x7ff70c02a980 comp rx=0 tx=0).stop 2024-09-17T04:25:58.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.801+0000 7ff719800640 1 -- 172.21.15.100:0/1696533903 shutdown_connections 2024-09-17T04:25:58.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.801+0000 7ff719800640 1 --2- 172.21.15.100:0/1696533903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff714160310 0x7ff714160720 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:58.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.801+0000 7ff719800640 1 --2- 172.21.15.100:0/1696533903 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7140ad0e0 0x7ff7140ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:58.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.801+0000 7ff719800640 1 --2- 172.21.15.100:0/1696533903 >> v2:172.21.15.184:3300/0 conn(0x7ff71415d760 0x7ff7141535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:58.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.801+0000 7ff719800640 1 -- 172.21.15.100:0/1696533903 >> 172.21.15.100:0/1696533903 conn(0x7ff71406d730 msgr2=0x7ff7140b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:58.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.801+0000 7ff719800640 1 -- 172.21.15.100:0/1696533903 shutdown_connections 2024-09-17T04:25:58.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.801+0000 7ff719800640 1 -- 172.21.15.100:0/1696533903 wait complete. 2024-09-17T04:25:58.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.803+0000 7ff719800640 1 Processor -- start 2024-09-17T04:25:58.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.803+0000 7ff719800640 1 -- start start 2024-09-17T04:25:58.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.804+0000 7ff719800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7140ad0e0 0x7ff71409dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:58.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.804+0000 7ff719800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff71415d760 0x7ff7141f6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:58.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.804+0000 7ff719800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff714160310 0x7ff7141f7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:58.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.804+0000 7ff719800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff7140b4da0 con 0x7ff7140ad0e0 2024-09-17T04:25:58.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.804+0000 7ff719800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff7140b4c20 con 0x7ff714160310 2024-09-17T04:25:58.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.804+0000 7ff719800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff714160c60 con 0x7ff71415d760 2024-09-17T04:25:58.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.805+0000 7ff712a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7140ad0e0 0x7ff71409dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:58.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.805+0000 7ff712a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7140ad0e0 0x7ff71409dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50490/0 (socket says 172.21.15.100:50490) 2024-09-17T04:25:58.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.805+0000 7ff712a00640 1 -- 172.21.15.100:0/3892609907 learned_addr learned my addr 172.21.15.100:0/3892609907 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:58.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.805+0000 7ff713400640 1 --2- 172.21.15.100:0/3892609907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff714160310 0x7ff7141f7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:58.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.805+0000 7ff712000640 1 --2- 172.21.15.100:0/3892609907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff71415d760 0x7ff7141f6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:58.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.805+0000 7ff712a00640 1 -- 172.21.15.100:0/3892609907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff71415d760 msgr2=0x7ff7141f6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:58.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.805+0000 7ff712a00640 1 --2- 172.21.15.100:0/3892609907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff71415d760 0x7ff7141f6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:58.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.805+0000 7ff712a00640 1 -- 172.21.15.100:0/3892609907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff714160310 msgr2=0x7ff7141f7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:58.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.805+0000 7ff712a00640 1 --2- 172.21.15.100:0/3892609907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff714160310 0x7ff7141f7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:58.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.805+0000 7ff712a00640 1 -- 172.21.15.100:0/3892609907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71409e200 con 0x7ff7140ad0e0 2024-09-17T04:25:58.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.805+0000 7ff712a00640 1 --2- 172.21.15.100:0/3892609907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7140ad0e0 0x7ff71409dcc0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff70400e790 tx=0x7ff70400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:58.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.806+0000 7ff70aa00640 1 -- 172.21.15.100:0/3892609907 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70400eea0 con 0x7ff7140ad0e0 2024-09-17T04:25:58.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.806+0000 7ff70aa00640 1 -- 172.21.15.100:0/3892609907 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff70400cdb0 con 0x7ff7140ad0e0 2024-09-17T04:25:58.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.806+0000 7ff70aa00640 1 -- 172.21.15.100:0/3892609907 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff704010690 con 0x7ff7140ad0e0 2024-09-17T04:25:58.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.806+0000 7ff719800640 1 -- 172.21.15.100:0/3892609907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff714155b50 con 0x7ff7140ad0e0 2024-09-17T04:25:58.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.806+0000 7ff719800640 1 -- 172.21.15.100:0/3892609907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff714155590 con 0x7ff7140ad0e0 2024-09-17T04:25:58.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.807+0000 7ff719800640 1 -- 172.21.15.100:0/3892609907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7140cfc70 con 0x7ff7140ad0e0 2024-09-17T04:25:58.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.808+0000 7ff70aa00640 1 -- 172.21.15.100:0/3892609907 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7ff704010830 con 0x7ff7140ad0e0 2024-09-17T04:25:58.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.809+0000 7ff70aa00640 1 --2- 172.21.15.100:0/3892609907 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff6f4078d70 0x7ff6f407b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:58.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.809+0000 7ff70aa00640 1 -- 172.21.15.100:0/3892609907 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7ff70409b970 con 0x7ff7140ad0e0 2024-09-17T04:25:58.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.809+0000 7ff712000640 1 --2- 172.21.15.100:0/3892609907 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff6f4078d70 0x7ff6f407b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:58.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.810+0000 7ff712000640 1 --2- 172.21.15.100:0/3892609907 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff6f4078d70 0x7ff6f407b280 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff7141f6a50 tx=0x7ff700005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:58.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:58.814+0000 7ff70aa00640 1 -- 172.21.15.100:0/3892609907 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff704063560 con 0x7ff7140ad0e0 2024-09-17T04:25:59.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.033+0000 7ff719800640 1 -- 172.21.15.100:0/3892609907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7141535c0 con 0x7ff7140ad0e0 2024-09-17T04:25:59.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.034+0000 7ff70aa00640 1 -- 172.21.15.100:0/3892609907 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7ff704067f90 con 0x7ff7140ad0e0 2024-09-17T04:25:59.033 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:25:59.033 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"btime":"2024-09-17T04:25:56:247389+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1086873435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1086873435},{"type":"v1","addr":"172.21.15.100:6833","nonce":1086873435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1708229764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1708229764},{"type":"v1","addr":"172.21.15.184:6836","nonce":1708229764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/828832783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":828832783},{"type":"v1","addr":"172.21.15.184:6837","nonce":828832783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:44.669203+0000","modified":"2024-09-17T04:25:47.693203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3801048032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3801048032},{"type":"v1","addr":"172.21.15.100:6835","nonce":3801048032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5295,"qdb_cluster":[5295]},"id":3}]} 2024-09-17T04:25:59.033 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-09-17T04:25:59.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.036+0000 7ff719800640 1 -- 172.21.15.100:0/3892609907 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff6f4078d70 msgr2=0x7ff6f407b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:59.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.036+0000 7ff719800640 1 --2- 172.21.15.100:0/3892609907 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff6f4078d70 0x7ff6f407b280 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff7141f6a50 tx=0x7ff700005f90 comp rx=0 tx=0).stop 2024-09-17T04:25:59.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.036+0000 7ff719800640 1 -- 172.21.15.100:0/3892609907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7140ad0e0 msgr2=0x7ff71409dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:59.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.036+0000 7ff719800640 1 --2- 172.21.15.100:0/3892609907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7140ad0e0 0x7ff71409dcc0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff70400e790 tx=0x7ff70400ec50 comp rx=0 tx=0).stop 2024-09-17T04:25:59.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.036+0000 7ff719800640 1 -- 172.21.15.100:0/3892609907 shutdown_connections 2024-09-17T04:25:59.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.036+0000 7ff719800640 1 --2- 172.21.15.100:0/3892609907 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff6f4078d70 0x7ff6f407b280 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:59.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.036+0000 7ff719800640 1 --2- 172.21.15.100:0/3892609907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff714160310 0x7ff7141f7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:59.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.036+0000 7ff719800640 1 --2- 172.21.15.100:0/3892609907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff71415d760 0x7ff7141f6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:59.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.036+0000 7ff719800640 1 --2- 172.21.15.100:0/3892609907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7140ad0e0 0x7ff71409dcc0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:59.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.036+0000 7ff719800640 1 -- 172.21.15.100:0/3892609907 >> 172.21.15.100:0/3892609907 conn(0x7ff71406d730 msgr2=0x7ff714152af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:59.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.036+0000 7ff719800640 1 -- 172.21.15.100:0/3892609907 shutdown_connections 2024-09-17T04:25:59.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.036+0000 7ff719800640 1 -- 172.21.15.100:0/3892609907 wait complete. 2024-09-17T04:25:59.060 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-17T04:25:59.305 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-09-17T04:25:59.305 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-17T04:25:59.305 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol48420 2024-09-17T04:25:59.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.528+0000 7fe941a00640 1 -- 172.21.15.100:0/295137236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe93c0d24f0 msgr2=0x7fe93c0d29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:59.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.528+0000 7fe941a00640 1 --2- 172.21.15.100:0/295137236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe93c0d24f0 0x7fe93c0d29c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe92c005fd0 tx=0x7fe92c02aa10 comp rx=0 tx=0).stop 2024-09-17T04:25:59.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.528+0000 7fe941a00640 1 -- 172.21.15.100:0/295137236 shutdown_connections 2024-09-17T04:25:59.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.528+0000 7fe941a00640 1 --2- 172.21.15.100:0/295137236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe93c0d4c70 0x7fe93c0cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:59.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.528+0000 7fe941a00640 1 --2- 172.21.15.100:0/295137236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe93c0d24f0 0x7fe93c0d29c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:59.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.528+0000 7fe941a00640 1 -- 172.21.15.100:0/295137236 >> 172.21.15.100:0/295137236 conn(0x7fe93c08dd50 msgr2=0x7fe93c09d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:59.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.528+0000 7fe941a00640 1 -- 172.21.15.100:0/295137236 shutdown_connections 2024-09-17T04:25:59.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.528+0000 7fe941a00640 1 -- 172.21.15.100:0/295137236 wait complete. 2024-09-17T04:25:59.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.530+0000 7fe941a00640 1 Processor -- start 2024-09-17T04:25:59.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.531+0000 7fe941a00640 1 -- start start 2024-09-17T04:25:59.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.532+0000 7fe941a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe93c0d24f0 0x7fe93c1601c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:59.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.532+0000 7fe941a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe93c0d2f00 0x7fe93c160700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:59.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.532+0000 7fe941a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe93c15b620 0x7fe93c09e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:59.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.532+0000 7fe941a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c0cfef0 con 0x7fe93c15b620 2024-09-17T04:25:59.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.532+0000 7fe941a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c0cfd70 con 0x7fe93c0d2f00 2024-09-17T04:25:59.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.532+0000 7fe941a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe93c0d0070 con 0x7fe93c0d24f0 2024-09-17T04:25:59.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.532+0000 7fe941000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe93c15b620 0x7fe93c09e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:59.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.532+0000 7fe941000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe93c15b620 0x7fe93c09e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50512/0 (socket says 172.21.15.100:50512) 2024-09-17T04:25:59.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.532+0000 7fe941000640 1 -- 172.21.15.100:0/3219257652 learned_addr learned my addr 172.21.15.100:0/3219257652 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:25:59.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.533+0000 7fe93be00640 1 --2- 172.21.15.100:0/3219257652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe93c0d24f0 0x7fe93c1601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:59.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.533+0000 7fe93b400640 1 --2- 172.21.15.100:0/3219257652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe93c0d2f00 0x7fe93c160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:59.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.533+0000 7fe941000640 1 -- 172.21.15.100:0/3219257652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe93c0d24f0 msgr2=0x7fe93c1601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:59.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.533+0000 7fe941000640 1 --2- 172.21.15.100:0/3219257652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe93c0d24f0 0x7fe93c1601c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:59.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.533+0000 7fe941000640 1 -- 172.21.15.100:0/3219257652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe93c0d2f00 msgr2=0x7fe93c160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:59.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.533+0000 7fe941000640 1 --2- 172.21.15.100:0/3219257652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe93c0d2f00 0x7fe93c160700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:59.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.533+0000 7fe941000640 1 -- 172.21.15.100:0/3219257652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93c0a1a40 con 0x7fe93c15b620 2024-09-17T04:25:59.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.533+0000 7fe941000640 1 --2- 172.21.15.100:0/3219257652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe93c15b620 0x7fe93c09e2f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe934005d00 tx=0x7fe93400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:59.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.533+0000 7fe933e00640 1 -- 172.21.15.100:0/3219257652 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe934017070 con 0x7fe93c15b620 2024-09-17T04:25:59.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.533+0000 7fe933e00640 1 -- 172.21.15.100:0/3219257652 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe934005330 con 0x7fe93c15b620 2024-09-17T04:25:59.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.534+0000 7fe941a00640 1 -- 172.21.15.100:0/3219257652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe93c0a1d20 con 0x7fe93c15b620 2024-09-17T04:25:59.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.534+0000 7fe933e00640 1 -- 172.21.15.100:0/3219257652 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9340047a0 con 0x7fe93c15b620 2024-09-17T04:25:59.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.534+0000 7fe941a00640 1 -- 172.21.15.100:0/3219257652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe93c0b2350 con 0x7fe93c15b620 2024-09-17T04:25:59.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.535+0000 7fe941a00640 1 -- 172.21.15.100:0/3219257652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe93c0d0520 con 0x7fe93c15b620 2024-09-17T04:25:59.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.536+0000 7fe933e00640 1 -- 172.21.15.100:0/3219257652 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fe934004940 con 0x7fe93c15b620 2024-09-17T04:25:59.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.537+0000 7fe933e00640 1 --2- 172.21.15.100:0/3219257652 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe91c078d40 0x7fe91c07b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:25:59.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.537+0000 7fe933e00640 1 -- 172.21.15.100:0/3219257652 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7fe93409b140 con 0x7fe93c15b620 2024-09-17T04:25:59.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.537+0000 7fe93be00640 1 --2- 172.21.15.100:0/3219257652 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe91c078d40 0x7fe91c07b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:25:59.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.538+0000 7fe93be00640 1 --2- 172.21.15.100:0/3219257652 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe91c078d40 0x7fe91c07b250 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe92c035000 tx=0x7fe92c004ab0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:25:59.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.541+0000 7fe933e00640 1 -- 172.21.15.100:0/3219257652 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe934062b70 con 0x7fe93c15b620 2024-09-17T04:25:59.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.746+0000 7fe941a00640 1 -- 172.21.15.100:0/3219257652 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol48420", "target": ["mon-mgr", ""]}) -- 0x7fe93c0d3f00 con 0x7fe91c078d40 2024-09-17T04:25:59.754 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:25:59.754+0000 7fa165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:25:59.754 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:25:59.754+0000 7fa165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:25:59.754 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:25:59.754+0000 7fa165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:25:59.754 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:25:59.754+0000 7fa165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:25:59.754 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:25:59.754+0000 7fa165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:25:59.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.801+0000 7fe933e00640 1 -- 172.21.15.100:0/3219257652 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe93c0d3f00 con 0x7fe91c078d40 2024-09-17T04:25:59.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.804+0000 7fe941a00640 1 -- 172.21.15.100:0/3219257652 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe91c078d40 msgr2=0x7fe91c07b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:59.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.804+0000 7fe941a00640 1 --2- 172.21.15.100:0/3219257652 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe91c078d40 0x7fe91c07b250 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe92c035000 tx=0x7fe92c004ab0 comp rx=0 tx=0).stop 2024-09-17T04:25:59.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.804+0000 7fe941a00640 1 -- 172.21.15.100:0/3219257652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe93c15b620 msgr2=0x7fe93c09e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:25:59.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.804+0000 7fe941a00640 1 --2- 172.21.15.100:0/3219257652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe93c15b620 0x7fe93c09e2f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe934005d00 tx=0x7fe93400dae0 comp rx=0 tx=0).stop 2024-09-17T04:25:59.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.804+0000 7fe941a00640 1 -- 172.21.15.100:0/3219257652 shutdown_connections 2024-09-17T04:25:59.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.804+0000 7fe941a00640 1 --2- 172.21.15.100:0/3219257652 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe91c078d40 0x7fe91c07b250 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:59.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.804+0000 7fe941a00640 1 --2- 172.21.15.100:0/3219257652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe93c15b620 0x7fe93c09e2f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:59.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.804+0000 7fe941a00640 1 --2- 172.21.15.100:0/3219257652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe93c0d2f00 0x7fe93c160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:59.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.804+0000 7fe941a00640 1 --2- 172.21.15.100:0/3219257652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe93c0d24f0 0x7fe93c1601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:25:59.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.804+0000 7fe941a00640 1 -- 172.21.15.100:0/3219257652 >> 172.21.15.100:0/3219257652 conn(0x7fe93c08dd50 msgr2=0x7fe93c0b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:25:59.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.804+0000 7fe941a00640 1 -- 172.21.15.100:0/3219257652 shutdown_connections 2024-09-17T04:25:59.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:25:59.804+0000 7fe941a00640 1 -- 172.21.15.100:0/3219257652 wait complete. 2024-09-17T04:25:59.826 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:26:00.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.047+0000 7f9c1f200640 1 -- 172.21.15.100:0/4269988525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c200ad9c0 msgr2=0x7f9c200af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:00.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.047+0000 7f9c1f200640 1 --2- 172.21.15.100:0/4269988525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c200ad9c0 0x7f9c200af170 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9c10005fd0 tx=0x7f9c1002a540 comp rx=0 tx=0).stop 2024-09-17T04:26:00.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.047+0000 7f9c1f200640 1 -- 172.21.15.100:0/4269988525 shutdown_connections 2024-09-17T04:26:00.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.047+0000 7f9c1f200640 1 --2- 172.21.15.100:0/4269988525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c2015a0f0 0x7f9c2015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:00.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.047+0000 7f9c1f200640 1 --2- 172.21.15.100:0/4269988525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c200ad9c0 0x7f9c200af170 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:00.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.047+0000 7f9c1f200640 1 -- 172.21.15.100:0/4269988525 >> 172.21.15.100:0/4269988525 conn(0x7f9c200b1e40 msgr2=0x7f9c200b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:26:00.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.047+0000 7f9c1f200640 1 -- 172.21.15.100:0/4269988525 shutdown_connections 2024-09-17T04:26:00.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.047+0000 7f9c1f200640 1 -- 172.21.15.100:0/4269988525 wait complete. 2024-09-17T04:26:00.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.050+0000 7f9c1f200640 1 Processor -- start 2024-09-17T04:26:00.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.050+0000 7f9c1f200640 1 -- start start 2024-09-17T04:26:00.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.051+0000 7f9c1f200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c200ad9c0 0x7f9c20152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:00.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.051+0000 7f9c1f200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c200b2f60 0x7f9c2014be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:00.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.051+0000 7f9c1f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c2015a0f0 0x7f9c200ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:00.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.051+0000 7f9c1f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2006f410 con 0x7f9c2015a0f0 2024-09-17T04:26:00.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.051+0000 7f9c1f200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2006f290 con 0x7f9c200b2f60 2024-09-17T04:26:00.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.051+0000 7f9c1f200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9c2006f590 con 0x7f9c200ad9c0 2024-09-17T04:26:00.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.051+0000 7f9c1e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c2015a0f0 0x7f9c200ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:00.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.051+0000 7f9c1e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c2015a0f0 0x7f9c200ced50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50532/0 (socket says 172.21.15.100:50532) 2024-09-17T04:26:00.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.051+0000 7f9c1e800640 1 -- 172.21.15.100:0/1863242415 learned_addr learned my addr 172.21.15.100:0/1863242415 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:26:00.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.052+0000 7f9c1de00640 1 --2- 172.21.15.100:0/1863242415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c200ad9c0 0x7f9c20152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:00.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.052+0000 7f9c15400640 1 --2- 172.21.15.100:0/1863242415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c200b2f60 0x7f9c2014be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:00.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.052+0000 7f9c1e800640 1 -- 172.21.15.100:0/1863242415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c200ad9c0 msgr2=0x7f9c20152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:00.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.052+0000 7f9c1e800640 1 --2- 172.21.15.100:0/1863242415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c200ad9c0 0x7f9c20152050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:00.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.052+0000 7f9c1e800640 1 -- 172.21.15.100:0/1863242415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c200b2f60 msgr2=0x7f9c2014be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:00.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.052+0000 7f9c1e800640 1 --2- 172.21.15.100:0/1863242415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c200b2f60 0x7f9c2014be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:00.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.052+0000 7f9c1e800640 1 -- 172.21.15.100:0/1863242415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c2014c3d0 con 0x7f9c2015a0f0 2024-09-17T04:26:00.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.052+0000 7f9c1e800640 1 --2- 172.21.15.100:0/1863242415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c2015a0f0 0x7f9c200ced50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800e990 tx=0x7f9c1800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:26:00.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.052+0000 7f9c16a00640 1 -- 172.21.15.100:0/1863242415 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c18004d20 con 0x7f9c2015a0f0 2024-09-17T04:26:00.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.052+0000 7f9c16a00640 1 -- 172.21.15.100:0/1863242415 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c1800caa0 con 0x7f9c2015a0f0 2024-09-17T04:26:00.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.052+0000 7f9c16a00640 1 -- 172.21.15.100:0/1863242415 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c18005020 con 0x7f9c2015a0f0 2024-09-17T04:26:00.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.052+0000 7f9c1f200640 1 -- 172.21.15.100:0/1863242415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c200cdcb0 con 0x7f9c2015a0f0 2024-09-17T04:26:00.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.053+0000 7f9c1f200640 1 -- 172.21.15.100:0/1863242415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c200ce270 con 0x7f9c2015a0f0 2024-09-17T04:26:00.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.054+0000 7f9c1f200640 1 -- 172.21.15.100:0/1863242415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c2006fab0 con 0x7f9c2015a0f0 2024-09-17T04:26:00.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.055+0000 7f9c16a00640 1 -- 172.21.15.100:0/1863242415 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f9c180051c0 con 0x7f9c2015a0f0 2024-09-17T04:26:00.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.056+0000 7f9c16a00640 1 --2- 172.21.15.100:0/1863242415 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9bfc078d40 0x7f9bfc07b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:00.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.056+0000 7f9c16a00640 1 -- 172.21.15.100:0/1863242415 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f9c1809bae0 con 0x7f9c2015a0f0 2024-09-17T04:26:00.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.056+0000 7f9c1de00640 1 --2- 172.21.15.100:0/1863242415 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9bfc078d40 0x7f9bfc07b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:00.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.057+0000 7f9c1de00640 1 --2- 172.21.15.100:0/1863242415 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9bfc078d40 0x7f9bfc07b250 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9c10033000 tx=0x7f9c10004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:26:00.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.060+0000 7f9c16a00640 1 -- 172.21.15.100:0/1863242415 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9c18016030 con 0x7f9c2015a0f0 2024-09-17T04:26:00.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.278+0000 7f9c16a00640 1 -- 172.21.15.100:0/1863242415 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7f9c1805f620 con 0x7f9c2015a0f0 2024-09-17T04:26:00.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.289+0000 7f9c1f200640 1 -- 172.21.15.100:0/1863242415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c2009e1b0 con 0x7f9c2015a0f0 2024-09-17T04:26:00.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.289+0000 7f9c16a00640 1 -- 172.21.15.100:0/1863242415 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f9c18063ab0 con 0x7f9c2015a0f0 2024-09-17T04:26:00.289 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:26:00.289 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"btime":"2024-09-17T04:25:56:247389+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1086873435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1086873435},{"type":"v1","addr":"172.21.15.100:6833","nonce":1086873435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1708229764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1708229764},{"type":"v1","addr":"172.21.15.184:6836","nonce":1708229764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/828832783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":828832783},{"type":"v1","addr":"172.21.15.184:6837","nonce":828832783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:44.669203+0000","modified":"2024-09-17T04:25:47.693203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3801048032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3801048032},{"type":"v1","addr":"172.21.15.100:6835","nonce":3801048032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5295,"qdb_cluster":[5295]},"id":3}]} 2024-09-17T04:26:00.289 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-09-17T04:26:00.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.292+0000 7f9c1f200640 1 -- 172.21.15.100:0/1863242415 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9bfc078d40 msgr2=0x7f9bfc07b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:00.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.292+0000 7f9c1f200640 1 --2- 172.21.15.100:0/1863242415 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9bfc078d40 0x7f9bfc07b250 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9c10033000 tx=0x7f9c10004730 comp rx=0 tx=0).stop 2024-09-17T04:26:00.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.292+0000 7f9c1f200640 1 -- 172.21.15.100:0/1863242415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c2015a0f0 msgr2=0x7f9c200ced50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:00.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.292+0000 7f9c1f200640 1 --2- 172.21.15.100:0/1863242415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c2015a0f0 0x7f9c200ced50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800e990 tx=0x7f9c1800ee50 comp rx=0 tx=0).stop 2024-09-17T04:26:00.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.292+0000 7f9c1f200640 1 -- 172.21.15.100:0/1863242415 shutdown_connections 2024-09-17T04:26:00.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.292+0000 7f9c1f200640 1 --2- 172.21.15.100:0/1863242415 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9bfc078d40 0x7f9bfc07b250 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:00.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.292+0000 7f9c1f200640 1 --2- 172.21.15.100:0/1863242415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c2015a0f0 0x7f9c200ced50 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:00.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.292+0000 7f9c1f200640 1 --2- 172.21.15.100:0/1863242415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c200b2f60 0x7f9c2014be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:00.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.292+0000 7f9c1f200640 1 --2- 172.21.15.100:0/1863242415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c200ad9c0 0x7f9c20152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:00.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.292+0000 7f9c1f200640 1 -- 172.21.15.100:0/1863242415 >> 172.21.15.100:0/1863242415 conn(0x7f9c200b1e40 msgr2=0x7f9c20070530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:26:00.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.292+0000 7f9c1f200640 1 -- 172.21.15.100:0/1863242415 shutdown_connections 2024-09-17T04:26:00.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.292+0000 7f9c1f200640 1 -- 172.21.15.100:0/1863242415 wait complete. 2024-09-17T04:26:00.317 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-17T04:26:00.562 INFO:teuthology.orchestra.run.smithi100.stdout:[{"id":5419,"entity":{"name":{"type":"client","num":5419},"addr":{"type":"any","addr":"172.21.15.184:0","nonce":1392734336}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.78097907,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.961170593174419,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5419 172.21.15.184:0/1392734336","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"ce1b342bf8c6349a8c90cd607f83bff2443f21cf","ceph_version":"ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)","entity_id":"x","hostname":"smithi184","pid":"35968","root":"/"}}] 2024-09-17T04:26:00.563 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi184", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35968", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.184:0", "nonce": 1392734336, "type": "any" }, "name": { "num": 5419, "type": "client" } }, "id": 5419, "inst": "client.5419 172.21.15.184:0/1392734336", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.96117059317442 }, "state": "open", "uptime": 0.78097907 } ] 2024-09-17T04:26:00.563 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:26:00.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.781+0000 7f7854400640 1 -- 172.21.15.100:0/3740361581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f784c15d760 msgr2=0x7f784c1535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:00.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.781+0000 7f7854400640 1 --2- 172.21.15.100:0/3740361581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f784c15d760 0x7f784c1535c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f784c072d60 tx=0x7f784002a4b0 comp rx=0 tx=0).stop 2024-09-17T04:26:00.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.781+0000 7f7854400640 1 -- 172.21.15.100:0/3740361581 shutdown_connections 2024-09-17T04:26:00.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.781+0000 7f7854400640 1 --2- 172.21.15.100:0/3740361581 >> v2:172.21.15.184:3300/0 conn(0x7f784c0ad0e0 0x7f784c0ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:00.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.781+0000 7f7854400640 1 --2- 172.21.15.100:0/3740361581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f784c15d760 0x7f784c1535c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:00.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.781+0000 7f7854400640 1 -- 172.21.15.100:0/3740361581 >> 172.21.15.100:0/3740361581 conn(0x7f784c06d730 msgr2=0x7f784c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:26:00.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.781+0000 7f7854400640 1 -- 172.21.15.100:0/3740361581 shutdown_connections 2024-09-17T04:26:00.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.781+0000 7f7854400640 1 -- 172.21.15.100:0/3740361581 wait complete. 2024-09-17T04:26:00.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.784+0000 7f7854400640 1 Processor -- start 2024-09-17T04:26:00.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.784+0000 7f7854400640 1 -- start start 2024-09-17T04:26:00.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.785+0000 7f7854400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f784c0ad0e0 0x7f784c09ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:00.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.785+0000 7f7854400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f784c15d760 0x7f784c06f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:00.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.785+0000 7f7854400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f784c160310 0x7f784c156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:00.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.785+0000 7f7854400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f784c0b4c20 con 0x7f784c15d760 2024-09-17T04:26:00.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.785+0000 7f7854400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f784c153c70 con 0x7f784c0ad0e0 2024-09-17T04:26:00.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.785+0000 7f7854400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f784c0b4da0 con 0x7f784c160310 2024-09-17T04:26:00.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.786+0000 7f7851000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f784c15d760 0x7f784c06f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:00.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.786+0000 7f7851000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f784c15d760 0x7f784c06f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50554/0 (socket says 172.21.15.100:50554) 2024-09-17T04:26:00.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.786+0000 7f7851000640 1 -- 172.21.15.100:0/2230756124 learned_addr learned my addr 172.21.15.100:0/2230756124 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:26:00.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.786+0000 7f7851a00640 1 --2- 172.21.15.100:0/2230756124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f784c0ad0e0 0x7f784c09ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:00.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.786+0000 7f7852400640 1 --2- 172.21.15.100:0/2230756124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f784c160310 0x7f784c156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:00.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.786+0000 7f7851000640 1 -- 172.21.15.100:0/2230756124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f784c160310 msgr2=0x7f784c156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:00.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.786+0000 7f7851000640 1 --2- 172.21.15.100:0/2230756124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f784c160310 0x7f784c156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:00.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.786+0000 7f7851000640 1 -- 172.21.15.100:0/2230756124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f784c0ad0e0 msgr2=0x7f784c09ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:00.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.786+0000 7f7851000640 1 --2- 172.21.15.100:0/2230756124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f784c0ad0e0 0x7f784c09ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:00.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.786+0000 7f7851000640 1 -- 172.21.15.100:0/2230756124 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784c1404d0 con 0x7f784c15d760 2024-09-17T04:26:00.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.786+0000 7f7851a00640 1 --2- 172.21.15.100:0/2230756124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f784c0ad0e0 0x7f784c09ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:26:00.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.786+0000 7f7851000640 1 --2- 172.21.15.100:0/2230756124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f784c15d760 0x7f784c06f520 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f783800d6d0 tx=0x7f783800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:26:00.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.787+0000 7f7849600640 1 -- 172.21.15.100:0/2230756124 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7838017070 con 0x7f784c15d760 2024-09-17T04:26:00.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.787+0000 7f7849600640 1 -- 172.21.15.100:0/2230756124 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7838004310 con 0x7f784c15d760 2024-09-17T04:26:00.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.787+0000 7f7854400640 1 -- 172.21.15.100:0/2230756124 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f784c1f36b0 con 0x7f784c15d760 2024-09-17T04:26:00.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.787+0000 7f7849600640 1 -- 172.21.15.100:0/2230756124 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7838009a80 con 0x7f784c15d760 2024-09-17T04:26:00.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.787+0000 7f7854400640 1 -- 172.21.15.100:0/2230756124 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f784c1f3c70 con 0x7f784c15d760 2024-09-17T04:26:00.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.788+0000 7f7854400640 1 -- 172.21.15.100:0/2230756124 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f784c160ff0 con 0x7f784c15d760 2024-09-17T04:26:00.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.789+0000 7f7849600640 1 -- 172.21.15.100:0/2230756124 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7f78380035f0 con 0x7f784c15d760 2024-09-17T04:26:00.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.790+0000 7f7849600640 1 --2- 172.21.15.100:0/2230756124 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f782c078dd0 0x7f782c07b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:00.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.790+0000 7f7849600640 1 -- 172.21.15.100:0/2230756124 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f783809b610 con 0x7f784c15d760 2024-09-17T04:26:00.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.791+0000 7f7851a00640 1 --2- 172.21.15.100:0/2230756124 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f782c078dd0 0x7f782c07b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:00.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.792+0000 7f7851a00640 1 --2- 172.21.15.100:0/2230756124 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f782c078dd0 0x7f782c07b2e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7840034000 tx=0x7f7840004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:26:00.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:00.795+0000 7f7849600640 1 -- 172.21.15.100:0/2230756124 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f7838063590 con 0x7f784c15d760 2024-09-17T04:26:01.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:01.011+0000 7f7854400640 1 -- 172.21.15.100:0/2230756124 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f784c15e180 con 0x7f784c15d760 2024-09-17T04:26:01.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:01.012+0000 7f7849600640 1 -- 172.21.15.100:0/2230756124 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f7838062e90 con 0x7f784c15d760 2024-09-17T04:26:01.011 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:26:01.011 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"btime":"2024-09-17T04:25:56:247389+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1086873435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1086873435},{"type":"v1","addr":"172.21.15.100:6833","nonce":1086873435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1708229764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1708229764},{"type":"v1","addr":"172.21.15.184:6836","nonce":1708229764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/828832783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":828832783},{"type":"v1","addr":"172.21.15.184:6837","nonce":828832783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:44.669203+0000","modified":"2024-09-17T04:25:47.693203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3801048032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3801048032},{"type":"v1","addr":"172.21.15.100:6835","nonce":3801048032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5295,"qdb_cluster":[5295]},"id":3}]} 2024-09-17T04:26:01.012 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-09-17T04:26:01.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:01.014+0000 7f7854400640 1 -- 172.21.15.100:0/2230756124 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f782c078dd0 msgr2=0x7f782c07b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:01.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:01.014+0000 7f7854400640 1 --2- 172.21.15.100:0/2230756124 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f782c078dd0 0x7f782c07b2e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7840034000 tx=0x7f7840004730 comp rx=0 tx=0).stop 2024-09-17T04:26:01.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:01.014+0000 7f7854400640 1 -- 172.21.15.100:0/2230756124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f784c15d760 msgr2=0x7f784c06f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:01.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:01.014+0000 7f7854400640 1 --2- 172.21.15.100:0/2230756124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f784c15d760 0x7f784c06f520 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f783800d6d0 tx=0x7f783800db90 comp rx=0 tx=0).stop 2024-09-17T04:26:01.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:01.015+0000 7f7854400640 1 -- 172.21.15.100:0/2230756124 shutdown_connections 2024-09-17T04:26:01.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:01.015+0000 7f7854400640 1 --2- 172.21.15.100:0/2230756124 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f782c078dd0 0x7f782c07b2e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:01.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:01.015+0000 7f7854400640 1 --2- 172.21.15.100:0/2230756124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f784c160310 0x7f784c156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:01.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:01.015+0000 7f7854400640 1 --2- 172.21.15.100:0/2230756124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f784c15d760 0x7f784c06f520 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:01.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:01.015+0000 7f7854400640 1 --2- 172.21.15.100:0/2230756124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f784c0ad0e0 0x7f784c09ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:01.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:01.015+0000 7f7854400640 1 -- 172.21.15.100:0/2230756124 >> 172.21.15.100:0/2230756124 conn(0x7f784c06d730 msgr2=0x7f784c0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:26:01.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:01.015+0000 7f7854400640 1 -- 172.21.15.100:0/2230756124 shutdown_connections 2024-09-17T04:26:01.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:01.015+0000 7f7854400640 1 -- 172.21.15.100:0/2230756124 wait complete. 2024-09-17T04:26:01.040 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-17T04:26:01.284 INFO:teuthology.orchestra.run.smithi100.stdout:[{"id":5419,"entity":{"name":{"type":"client","num":5419},"addr":{"type":"any","addr":"172.21.15.184:0","nonce":1392734336}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":1.506959613,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.924364435050666,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5419 172.21.15.184:0/1392734336","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"ce1b342bf8c6349a8c90cd607f83bff2443f21cf","ceph_version":"ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)","entity_id":"x","hostname":"smithi184","pid":"35968","root":"/"}}] 2024-09-17T04:26:01.284 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi184", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35968", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.184:0", "nonce": 1392734336, "type": "any" }, "name": { "num": 5419, "type": "client" } }, "id": 5419, "inst": "client.5419 172.21.15.184:0/1392734336", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.924364435050666 }, "state": "open", "uptime": 1.506959613 } ] 2024-09-17T04:26:01.285 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-17T04:26:04.791 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:26:04.796 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:26:06.286 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:26:06.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.516+0000 7f2834200640 1 -- 172.21.15.100:0/1304909787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2814003a50 msgr2=0x7f2814003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:06.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.516+0000 7f2834200640 1 --2- 172.21.15.100:0/1304909787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2814003a50 0x7f2814003ec0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f28200016c0 tx=0x7f2814005830 comp rx=0 tx=0).stop 2024-09-17T04:26:06.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.516+0000 7f2834200640 1 -- 172.21.15.100:0/1304909787 shutdown_connections 2024-09-17T04:26:06.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.516+0000 7f2834200640 1 --2- 172.21.15.100:0/1304909787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2814003a50 0x7f2814003ec0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:06.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.516+0000 7f2834200640 1 -- 172.21.15.100:0/1304909787 >> 172.21.15.100:0/1304909787 conn(0x7f282c06d730 msgr2=0x7f282c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:26:06.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.516+0000 7f2834200640 1 -- 172.21.15.100:0/1304909787 shutdown_connections 2024-09-17T04:26:06.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.516+0000 7f2834200640 1 -- 172.21.15.100:0/1304909787 wait complete. 2024-09-17T04:26:06.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.519+0000 7f2834200640 1 Processor -- start 2024-09-17T04:26:06.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.519+0000 7f2834200640 1 -- start start 2024-09-17T04:26:06.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.520+0000 7f2834200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2814003a50 0x7f282c1f9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.520+0000 7f2834200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f282c149920 0x7f282c0b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.520+0000 7f2834200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f282c0b0f70 0x7f282c074280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.520+0000 7f2834200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f282c0b2750 con 0x7f282c149920 2024-09-17T04:26:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.520+0000 7f2834200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f282c15f6f0 con 0x7f282c0b0f70 2024-09-17T04:26:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.520+0000 7f2834200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f282c0b28d0 con 0x7f2814003a50 2024-09-17T04:26:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.521+0000 7f2830e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f282c149920 0x7f282c0b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.521+0000 7f2830e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f282c149920 0x7f282c0b0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52944/0 (socket says 172.21.15.100:52944) 2024-09-17T04:26:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.521+0000 7f2830e00640 1 -- 172.21.15.100:0/3666844110 learned_addr learned my addr 172.21.15.100:0/3666844110 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:26:06.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.521+0000 7f2831800640 1 --2- 172.21.15.100:0/3666844110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2814003a50 0x7f282c1f9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:06.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.521+0000 7f2832200640 1 --2- 172.21.15.100:0/3666844110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f282c0b0f70 0x7f282c074280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:06.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.521+0000 7f2830e00640 1 -- 172.21.15.100:0/3666844110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2814003a50 msgr2=0x7f282c1f9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:06.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.521+0000 7f2830e00640 1 --2- 172.21.15.100:0/3666844110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2814003a50 0x7f282c1f9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:06.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.521+0000 7f2830e00640 1 -- 172.21.15.100:0/3666844110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f282c0b0f70 msgr2=0x7f282c074280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:06.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.521+0000 7f2830e00640 1 --2- 172.21.15.100:0/3666844110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f282c0b0f70 0x7f282c074280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:06.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.521+0000 7f2830e00640 1 -- 172.21.15.100:0/3666844110 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f282c1f9f80 con 0x7f282c149920 2024-09-17T04:26:06.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.521+0000 7f2830e00640 1 --2- 172.21.15.100:0/3666844110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f282c149920 0x7f282c0b0a30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2818003ed0 tx=0x7f281800b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:26:06.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.522+0000 7f2829600640 1 -- 172.21.15.100:0/3666844110 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2818011910 con 0x7f282c149920 2024-09-17T04:26:06.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.522+0000 7f2829600640 1 -- 172.21.15.100:0/3666844110 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2818015070 con 0x7f282c149920 2024-09-17T04:26:06.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.522+0000 7f2834200640 1 -- 172.21.15.100:0/3666844110 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f282c0b1380 con 0x7f282c149920 2024-09-17T04:26:06.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.522+0000 7f2829600640 1 -- 172.21.15.100:0/3666844110 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2818011cf0 con 0x7f282c149920 2024-09-17T04:26:06.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.522+0000 7f2834200640 1 -- 172.21.15.100:0/3666844110 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f282c14fc30 con 0x7f282c149920 2024-09-17T04:26:06.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.523+0000 7f2834200640 1 -- 172.21.15.100:0/3666844110 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f282c15bd90 con 0x7f282c149920 2024-09-17T04:26:06.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.524+0000 7f2829600640 1 -- 172.21.15.100:0/3666844110 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7f281801a470 con 0x7f282c149920 2024-09-17T04:26:06.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.526+0000 7f2829600640 1 --2- 172.21.15.100:0/3666844110 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f280c078dd0 0x7f280c07b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:06.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.526+0000 7f2829600640 1 -- 172.21.15.100:0/3666844110 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f281809c130 con 0x7f282c149920 2024-09-17T04:26:06.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.526+0000 7f2831800640 1 --2- 172.21.15.100:0/3666844110 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f280c078dd0 0x7f280c07b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:06.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.527+0000 7f2831800640 1 --2- 172.21.15.100:0/3666844110 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f280c078dd0 0x7f280c07b2e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2820032000 tx=0x7f282002c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:26:06.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.530+0000 7f2829600640 1 -- 172.21.15.100:0/3666844110 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f282c0b1380 con 0x7f282c149920 2024-09-17T04:26:06.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.745+0000 7f2834200640 1 -- 172.21.15.100:0/3666844110 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f282c0cf960 con 0x7f282c149920 2024-09-17T04:26:06.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.746+0000 7f2829600640 1 -- 172.21.15.100:0/3666844110 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f282c0cf960 con 0x7f282c149920 2024-09-17T04:26:06.745 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:26:06.746 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"btime":"2024-09-17T04:25:56:247389+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1086873435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1086873435},{"type":"v1","addr":"172.21.15.100:6833","nonce":1086873435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1708229764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1708229764},{"type":"v1","addr":"172.21.15.184:6836","nonce":1708229764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/828832783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":828832783},{"type":"v1","addr":"172.21.15.184:6837","nonce":828832783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:44.669203+0000","modified":"2024-09-17T04:25:47.693203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3801048032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3801048032},{"type":"v1","addr":"172.21.15.100:6835","nonce":3801048032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5295,"qdb_cluster":[5295]},"id":3}]} 2024-09-17T04:26:06.746 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-09-17T04:26:06.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.748+0000 7f2834200640 1 -- 172.21.15.100:0/3666844110 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f280c078dd0 msgr2=0x7f280c07b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:06.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.748+0000 7f2834200640 1 --2- 172.21.15.100:0/3666844110 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f280c078dd0 0x7f280c07b2e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2820032000 tx=0x7f282002c040 comp rx=0 tx=0).stop 2024-09-17T04:26:06.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.748+0000 7f2834200640 1 -- 172.21.15.100:0/3666844110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f282c149920 msgr2=0x7f282c0b0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:06.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.748+0000 7f2834200640 1 --2- 172.21.15.100:0/3666844110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f282c149920 0x7f282c0b0a30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2818003ed0 tx=0x7f281800b980 comp rx=0 tx=0).stop 2024-09-17T04:26:06.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.748+0000 7f2834200640 1 -- 172.21.15.100:0/3666844110 shutdown_connections 2024-09-17T04:26:06.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.748+0000 7f2834200640 1 --2- 172.21.15.100:0/3666844110 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f280c078dd0 0x7f280c07b2e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:06.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.748+0000 7f2834200640 1 --2- 172.21.15.100:0/3666844110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f282c0b0f70 0x7f282c074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:06.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.748+0000 7f2834200640 1 --2- 172.21.15.100:0/3666844110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f282c149920 0x7f282c0b0a30 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:06.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.748+0000 7f2834200640 1 --2- 172.21.15.100:0/3666844110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2814003a50 0x7f282c1f9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:06.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.748+0000 7f2834200640 1 -- 172.21.15.100:0/3666844110 >> 172.21.15.100:0/3666844110 conn(0x7f282c06d730 msgr2=0x7f282c150030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:26:06.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.749+0000 7f2834200640 1 -- 172.21.15.100:0/3666844110 shutdown_connections 2024-09-17T04:26:06.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:06.749+0000 7f2834200640 1 -- 172.21.15.100:0/3666844110 wait complete. 2024-09-17T04:26:06.774 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-17T04:26:07.015 INFO:teuthology.orchestra.run.smithi100.stdout:[{"id":5419,"entity":{"name":{"type":"client","num":5419},"addr":{"type":"any","addr":"172.21.15.184:0","nonce":1392734336}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":13,"uptime":7.2378060230000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.635975960660005,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5419 172.21.15.184:0/1392734336","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"ce1b342bf8c6349a8c90cd607f83bff2443f21cf","ceph_version":"ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)","entity_id":"x","hostname":"smithi184","pid":"35968","root":"/"}}] 2024-09-17T04:26:07.016 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi184", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35968", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.184:0", "nonce": 1392734336, "type": "any" }, "name": { "num": 5419, "type": "client" } }, "id": 5419, "inst": "client.5419 172.21.15.184:0/1392734336", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 13, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.635975960660005 }, "state": "open", "uptime": 7.237806023 } ] 2024-09-17T04:26:07.016 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-17T04:26:12.017 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:26:12.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.250+0000 7fd30b200640 1 -- 172.21.15.100:0/413627892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2e8004d40 msgr2=0x7fd2e8005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:12.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.250+0000 7fd30b200640 1 --2- 172.21.15.100:0/413627892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2e8004d40 0x7fd2e8005150 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc00c750 tx=0x7fd2e8005830 comp rx=0 tx=0).stop 2024-09-17T04:26:12.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.250+0000 7fd309200640 1 -- 172.21.15.100:0/413627892 reap_dead start 2024-09-17T04:26:12.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.251+0000 7fd30b200640 1 -- 172.21.15.100:0/413627892 shutdown_connections 2024-09-17T04:26:12.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.251+0000 7fd30b200640 1 -- 172.21.15.100:0/413627892 >> 172.21.15.100:0/413627892 conn(0x7fd30406d730 msgr2=0x7fd3040b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:26:12.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.251+0000 7fd30b200640 1 -- 172.21.15.100:0/413627892 shutdown_connections 2024-09-17T04:26:12.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.251+0000 7fd30b200640 1 -- 172.21.15.100:0/413627892 wait complete. 2024-09-17T04:26:12.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.253+0000 7fd30b200640 1 Processor -- start 2024-09-17T04:26:12.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.253+0000 7fd30b200640 1 -- start start 2024-09-17T04:26:12.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.254+0000 7fd30b200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2e8004d40 0x7fd3041f9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:12.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.254+0000 7fd30b200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3041f9cc0 0x7fd3041f9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:12.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.254+0000 7fd30b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3041f5750 0x7fd30414e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:12.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.254+0000 7fd30b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd30406f220 con 0x7fd3041f5750 2024-09-17T04:26:12.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.254+0000 7fd30b200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd30406f0a0 con 0x7fd2e8004d40 2024-09-17T04:26:12.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.254+0000 7fd30b200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd3040b4c20 con 0x7fd3041f9cc0 2024-09-17T04:26:12.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.255+0000 7fd309200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3041f5750 0x7fd30414e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:12.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.255+0000 7fd309200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3041f5750 0x7fd30414e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52970/0 (socket says 172.21.15.100:52970) 2024-09-17T04:26:12.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.255+0000 7fd309200640 1 -- 172.21.15.100:0/2249401481 learned_addr learned my addr 172.21.15.100:0/2249401481 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:26:12.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.255+0000 7fd303e00640 1 --2- 172.21.15.100:0/2249401481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2e8004d40 0x7fd3041f9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:12.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.255+0000 7fd303400640 1 --2- 172.21.15.100:0/2249401481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3041f9cc0 0x7fd3041f9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:12.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.255+0000 7fd309200640 1 -- 172.21.15.100:0/2249401481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3041f9cc0 msgr2=0x7fd3041f9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:12.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.255+0000 7fd309200640 1 --2- 172.21.15.100:0/2249401481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3041f9cc0 0x7fd3041f9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:12.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.255+0000 7fd309200640 1 -- 172.21.15.100:0/2249401481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2e8004d40 msgr2=0x7fd3041f9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:12.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.255+0000 7fd309200640 1 --2- 172.21.15.100:0/2249401481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2e8004d40 0x7fd3041f9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:12.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.255+0000 7fd309200640 1 -- 172.21.15.100:0/2249401481 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3040b1580 con 0x7fd3041f5750 2024-09-17T04:26:12.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.256+0000 7fd309200640 1 --2- 172.21.15.100:0/2249401481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3041f5750 0x7fd30414e2d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc02b660 tx=0x7fd2fc004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:26:12.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.256+0000 7fd2fbe00640 1 -- 172.21.15.100:0/2249401481 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f4009720 con 0x7fd3041f5750 2024-09-17T04:26:12.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.256+0000 7fd2fbe00640 1 -- 172.21.15.100:0/2249401481 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd2fc001a00 con 0x7fd3041f5750 2024-09-17T04:26:12.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.256+0000 7fd2fbe00640 1 -- 172.21.15.100:0/2249401481 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc00c3a0 con 0x7fd3041f5750 2024-09-17T04:26:12.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.256+0000 7fd30b200640 1 -- 172.21.15.100:0/2249401481 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd30414d510 con 0x7fd3041f5750 2024-09-17T04:26:12.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.256+0000 7fd30b200640 1 -- 172.21.15.100:0/2249401481 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd30414dad0 con 0x7fd3041f5750 2024-09-17T04:26:12.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.257+0000 7fd30b200640 1 -- 172.21.15.100:0/2249401481 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd304160fa0 con 0x7fd3041f5750 2024-09-17T04:26:12.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.258+0000 7fd2fbe00640 1 -- 172.21.15.100:0/2249401481 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7fd2fc03f030 con 0x7fd3041f5750 2024-09-17T04:26:12.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.259+0000 7fd2fbe00640 1 --2- 172.21.15.100:0/2249401481 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd2e4078dd0 0x7fd2e407b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:12.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.259+0000 7fd2fbe00640 1 -- 172.21.15.100:0/2249401481 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7fd2fc0ba850 con 0x7fd3041f5750 2024-09-17T04:26:12.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.260+0000 7fd303e00640 1 --2- 172.21.15.100:0/2249401481 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd2e4078dd0 0x7fd2e407b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:12.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.261+0000 7fd303e00640 1 --2- 172.21.15.100:0/2249401481 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd2e4078dd0 0x7fd2e407b2e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4009e70 tx=0x7fd2f4005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:26:12.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.264+0000 7fd2fbe00640 1 -- 172.21.15.100:0/2249401481 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd3040ada30 con 0x7fd3041f5750 2024-09-17T04:26:12.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.482+0000 7fd30b200640 1 -- 172.21.15.100:0/2249401481 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd304152eb0 con 0x7fd3041f5750 2024-09-17T04:26:12.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.483+0000 7fd2fbe00640 1 -- 172.21.15.100:0/2249401481 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7fd2fc082890 con 0x7fd3041f5750 2024-09-17T04:26:12.483 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:26:12.483 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"btime":"2024-09-17T04:25:56:247389+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1086873435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1086873435},{"type":"v1","addr":"172.21.15.100:6833","nonce":1086873435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1708229764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1708229764},{"type":"v1","addr":"172.21.15.184:6836","nonce":1708229764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/828832783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":828832783},{"type":"v1","addr":"172.21.15.184:6837","nonce":828832783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:44.669203+0000","modified":"2024-09-17T04:25:47.693203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3801048032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3801048032},{"type":"v1","addr":"172.21.15.100:6835","nonce":3801048032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5295,"qdb_cluster":[5295]},"id":3}]} 2024-09-17T04:26:12.484 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-09-17T04:26:12.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.485+0000 7fd30b200640 1 -- 172.21.15.100:0/2249401481 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd2e4078dd0 msgr2=0x7fd2e407b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:12.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.485+0000 7fd30b200640 1 --2- 172.21.15.100:0/2249401481 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd2e4078dd0 0x7fd2e407b2e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4009e70 tx=0x7fd2f4005f90 comp rx=0 tx=0).stop 2024-09-17T04:26:12.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.485+0000 7fd30b200640 1 -- 172.21.15.100:0/2249401481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3041f5750 msgr2=0x7fd30414e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:12.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.485+0000 7fd30b200640 1 --2- 172.21.15.100:0/2249401481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3041f5750 0x7fd30414e2d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc02b660 tx=0x7fd2fc004a10 comp rx=0 tx=0).stop 2024-09-17T04:26:12.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.485+0000 7fd30b200640 1 -- 172.21.15.100:0/2249401481 shutdown_connections 2024-09-17T04:26:12.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.485+0000 7fd30b200640 1 --2- 172.21.15.100:0/2249401481 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd2e4078dd0 0x7fd2e407b2e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:12.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.485+0000 7fd30b200640 1 --2- 172.21.15.100:0/2249401481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3041f5750 0x7fd30414e2d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:12.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.485+0000 7fd30b200640 1 --2- 172.21.15.100:0/2249401481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3041f9cc0 0x7fd3041f9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:12.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.485+0000 7fd30b200640 1 --2- 172.21.15.100:0/2249401481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2e8004d40 0x7fd3041f9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:12.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.485+0000 7fd30b200640 1 -- 172.21.15.100:0/2249401481 >> 172.21.15.100:0/2249401481 conn(0x7fd30406d730 msgr2=0x7fd30406fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:26:12.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.486+0000 7fd30b200640 1 -- 172.21.15.100:0/2249401481 shutdown_connections 2024-09-17T04:26:12.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:12.486+0000 7fd30b200640 1 -- 172.21.15.100:0/2249401481 wait complete. 2024-09-17T04:26:12.509 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-17T04:26:12.749 INFO:teuthology.orchestra.run.smithi100.stdout:[{"id":5419,"entity":{"name":{"type":"client","num":5419},"addr":{"type":"any","addr":"172.21.15.184:0","nonce":1392734336}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":12,"uptime":12.972652326,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.351183565082462,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5419 172.21.15.184:0/1392734336","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"ce1b342bf8c6349a8c90cd607f83bff2443f21cf","ceph_version":"ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)","entity_id":"x","hostname":"smithi184","pid":"35968","root":"/"}}] 2024-09-17T04:26:12.750 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi184", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35968", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.184:0", "nonce": 1392734336, "type": "any" }, "name": { "num": 5419, "type": "client" } }, "id": 5419, "inst": "client.5419 172.21.15.184:0/1392734336", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 12, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.35118356508246 }, "state": "open", "uptime": 12.972652326 } ] 2024-09-17T04:26:12.750 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-17T04:26:17.751 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:26:17.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.982+0000 7f40a8800640 1 -- 172.21.15.100:0/982694071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40a01626d0 msgr2=0x7f40a0164af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:17.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.982+0000 7f40a8800640 1 --2- 172.21.15.100:0/982694071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40a01626d0 0x7f40a0164af0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f40a0072810 tx=0x7f409c02aa30 comp rx=0 tx=0).stop 2024-09-17T04:26:17.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.982+0000 7f40a8800640 1 -- 172.21.15.100:0/982694071 shutdown_connections 2024-09-17T04:26:17.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.982+0000 7f40a8800640 1 --2- 172.21.15.100:0/982694071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40a01626d0 0x7f40a0164af0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:17.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.982+0000 7f40a8800640 1 --2- 172.21.15.100:0/982694071 >> v2:172.21.15.184:3300/0 conn(0x7f40a00704b0 0x7f40a0073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:17.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.982+0000 7f40a8800640 1 -- 172.21.15.100:0/982694071 >> 172.21.15.100:0/982694071 conn(0x7f40a006d730 msgr2=0x7f40a00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:26:17.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.982+0000 7f40a8800640 1 -- 172.21.15.100:0/982694071 shutdown_connections 2024-09-17T04:26:17.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.982+0000 7f40a8800640 1 -- 172.21.15.100:0/982694071 wait complete. 2024-09-17T04:26:17.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.984+0000 7f40a8800640 1 Processor -- start 2024-09-17T04:26:17.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.985+0000 7f40a8800640 1 -- start start 2024-09-17T04:26:17.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.986+0000 7f40a8800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40a00704b0 0x7f40a0153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:17.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.986+0000 7f40a8800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40a00ad3c0 0x7f40a0153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:17.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.986+0000 7f40a8800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40a01626d0 0x7f40a0158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:17.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.986+0000 7f40a8800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f40a00ae060 con 0x7f40a01626d0 2024-09-17T04:26:17.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.986+0000 7f40a8800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f40a00adee0 con 0x7f40a00ad3c0 2024-09-17T04:26:17.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.986+0000 7f40a8800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f40a0071130 con 0x7f40a00704b0 2024-09-17T04:26:17.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.986+0000 7f40a6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40a01626d0 0x7f40a0158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:17.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.986+0000 7f40a6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40a01626d0 0x7f40a0158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43572/0 (socket says 172.21.15.100:43572) 2024-09-17T04:26:17.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.986+0000 7f40a6800640 1 -- 172.21.15.100:0/3929221554 learned_addr learned my addr 172.21.15.100:0/3929221554 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:26:17.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.986+0000 7f40a5e00640 1 --2- 172.21.15.100:0/3929221554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40a00704b0 0x7f40a0153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:17.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.987+0000 7f40a5400640 1 --2- 172.21.15.100:0/3929221554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40a00ad3c0 0x7f40a0153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:17.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.987+0000 7f40a6800640 1 -- 172.21.15.100:0/3929221554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40a00704b0 msgr2=0x7f40a0153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:17.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.987+0000 7f40a6800640 1 --2- 172.21.15.100:0/3929221554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40a00704b0 0x7f40a0153620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:17.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.987+0000 7f40a6800640 1 -- 172.21.15.100:0/3929221554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40a00ad3c0 msgr2=0x7f40a0153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:17.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.987+0000 7f40a6800640 1 --2- 172.21.15.100:0/3929221554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40a00ad3c0 0x7f40a0153b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:17.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.987+0000 7f40a6800640 1 -- 172.21.15.100:0/3929221554 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40a0154ed0 con 0x7f40a01626d0 2024-09-17T04:26:17.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.987+0000 7f40a6800640 1 --2- 172.21.15.100:0/3929221554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40a01626d0 0x7f40a0158700 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f409c02b020 tx=0x7f409c003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:26:17.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.987+0000 7f409a000640 1 -- 172.21.15.100:0/3929221554 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c033050 con 0x7f40a01626d0 2024-09-17T04:26:17.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.988+0000 7f409a000640 1 -- 172.21.15.100:0/3929221554 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f409c004450 con 0x7f40a01626d0 2024-09-17T04:26:17.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.988+0000 7f409a000640 1 -- 172.21.15.100:0/3929221554 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c038450 con 0x7f40a01626d0 2024-09-17T04:26:17.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.988+0000 7f40a8800640 1 -- 172.21.15.100:0/3929221554 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40a01541f0 con 0x7f40a01626d0 2024-09-17T04:26:17.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.988+0000 7f40a8800640 1 -- 172.21.15.100:0/3929221554 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40a01547b0 con 0x7f40a01626d0 2024-09-17T04:26:17.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.989+0000 7f40a8800640 1 -- 172.21.15.100:0/3929221554 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40a0073040 con 0x7f40a01626d0 2024-09-17T04:26:17.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.990+0000 7f409a000640 1 -- 172.21.15.100:0/3929221554 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7f409c036070 con 0x7f40a01626d0 2024-09-17T04:26:17.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.991+0000 7f409a000640 1 --2- 172.21.15.100:0/3929221554 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4084078dd0 0x7f408407b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:17.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.991+0000 7f409a000640 1 -- 172.21.15.100:0/3929221554 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f409c0bc820 con 0x7f40a01626d0 2024-09-17T04:26:17.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.991+0000 7f40a5e00640 1 --2- 172.21.15.100:0/3929221554 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4084078dd0 0x7f408407b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:17.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.992+0000 7f40a5e00640 1 --2- 172.21.15.100:0/3929221554 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4084078dd0 0x7f408407b2e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4094009b30 tx=0x7f4094005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:26:17.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:17.996+0000 7f409a000640 1 -- 172.21.15.100:0/3929221554 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f409c084150 con 0x7f40a01626d0 2024-09-17T04:26:18.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:18.217+0000 7f40a8800640 1 -- 172.21.15.100:0/3929221554 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40a0074120 con 0x7f40a01626d0 2024-09-17T04:26:18.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:18.218+0000 7f409a000640 1 -- 172.21.15.100:0/3929221554 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f409c088b80 con 0x7f40a01626d0 2024-09-17T04:26:18.218 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:26:18.218 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"btime":"2024-09-17T04:25:56:247389+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1086873435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1086873435},{"type":"v1","addr":"172.21.15.100:6833","nonce":1086873435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1708229764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1708229764},{"type":"v1","addr":"172.21.15.184:6836","nonce":1708229764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/828832783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":828832783},{"type":"v1","addr":"172.21.15.184:6837","nonce":828832783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:44.669203+0000","modified":"2024-09-17T04:25:47.693203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3801048032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3801048032},{"type":"v1","addr":"172.21.15.100:6835","nonce":3801048032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5295,"qdb_cluster":[5295]},"id":3}]} 2024-09-17T04:26:18.219 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-09-17T04:26:18.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:18.220+0000 7f40a8800640 1 -- 172.21.15.100:0/3929221554 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4084078dd0 msgr2=0x7f408407b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:18.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:18.220+0000 7f40a8800640 1 --2- 172.21.15.100:0/3929221554 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4084078dd0 0x7f408407b2e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4094009b30 tx=0x7f4094005f90 comp rx=0 tx=0).stop 2024-09-17T04:26:18.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:18.220+0000 7f40a8800640 1 -- 172.21.15.100:0/3929221554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40a01626d0 msgr2=0x7f40a0158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:18.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:18.220+0000 7f40a8800640 1 --2- 172.21.15.100:0/3929221554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40a01626d0 0x7f40a0158700 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f409c02b020 tx=0x7f409c003f90 comp rx=0 tx=0).stop 2024-09-17T04:26:18.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:18.220+0000 7f40a8800640 1 -- 172.21.15.100:0/3929221554 shutdown_connections 2024-09-17T04:26:18.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:18.220+0000 7f40a8800640 1 --2- 172.21.15.100:0/3929221554 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4084078dd0 0x7f408407b2e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:18.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:18.220+0000 7f40a8800640 1 --2- 172.21.15.100:0/3929221554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40a01626d0 0x7f40a0158700 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:18.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:18.220+0000 7f40a8800640 1 --2- 172.21.15.100:0/3929221554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40a00ad3c0 0x7f40a0153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:18.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:18.220+0000 7f40a8800640 1 --2- 172.21.15.100:0/3929221554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40a00704b0 0x7f40a0153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:18.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:18.220+0000 7f40a8800640 1 -- 172.21.15.100:0/3929221554 >> 172.21.15.100:0/3929221554 conn(0x7f40a006d730 msgr2=0x7f40a00ae470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:26:18.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:18.220+0000 7f40a8800640 1 -- 172.21.15.100:0/3929221554 shutdown_connections 2024-09-17T04:26:18.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:18.220+0000 7f40a8800640 1 -- 172.21.15.100:0/3929221554 wait complete. 2024-09-17T04:26:18.246 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-17T04:26:18.486 INFO:teuthology.orchestra.run.smithi100.stdout:[{"id":5419,"entity":{"name":{"type":"client","num":5419},"addr":{"type":"any","addr":"172.21.15.184:0","nonce":1392734336}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":12,"uptime":18.709498575000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.070042504194866,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5419 172.21.15.184:0/1392734336","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"ce1b342bf8c6349a8c90cd607f83bff2443f21cf","ceph_version":"ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)","entity_id":"x","hostname":"smithi184","pid":"35968","root":"/"}}] 2024-09-17T04:26:18.487 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi184", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35968", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.184:0", "nonce": 1392734336, "type": "any" }, "name": { "num": 5419, "type": "client" } }, "id": 5419, "inst": "client.5419 172.21.15.184:0/1392734336", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 12, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.070042504194866 }, "state": "open", "uptime": 18.709498575 } ] 2024-09-17T04:26:18.487 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-17T04:26:23.488 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:26:23.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.717+0000 7ff91cc00640 1 -- 172.21.15.100:0/939087185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9180ad0e0 msgr2=0x7ff9180ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:23.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.717+0000 7ff91cc00640 1 --2- 172.21.15.100:0/939087185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9180ad0e0 0x7ff9180ad4f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff904009830 tx=0x7ff90402a480 comp rx=0 tx=0).stop 2024-09-17T04:26:23.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.717+0000 7ff91cc00640 1 -- 172.21.15.100:0/939087185 shutdown_connections 2024-09-17T04:26:23.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.717+0000 7ff91cc00640 1 --2- 172.21.15.100:0/939087185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9180ad0e0 0x7ff9180ad4f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:23.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.717+0000 7ff91cc00640 1 -- 172.21.15.100:0/939087185 >> 172.21.15.100:0/939087185 conn(0x7ff91806d730 msgr2=0x7ff9180b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:26:23.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.718+0000 7ff91cc00640 1 -- 172.21.15.100:0/939087185 shutdown_connections 2024-09-17T04:26:23.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.718+0000 7ff91cc00640 1 -- 172.21.15.100:0/939087185 wait complete. 2024-09-17T04:26:23.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.720+0000 7ff91cc00640 1 Processor -- start 2024-09-17T04:26:23.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.720+0000 7ff91cc00640 1 -- start start 2024-09-17T04:26:23.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.721+0000 7ff91cc00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9180ad0e0 0x7ff91814a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:23.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.721+0000 7ff91cc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9180b3550 0x7ff9180b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:23.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.721+0000 7ff91cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff91815d760 0x7ff9180b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:23.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.721+0000 7ff91cc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff9180b4da0 con 0x7ff91815d760 2024-09-17T04:26:23.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.721+0000 7ff91cc00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff9180b4c20 con 0x7ff9180b3550 2024-09-17T04:26:23.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.721+0000 7ff91cc00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff9180b4070 con 0x7ff9180ad0e0 2024-09-17T04:26:23.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.721+0000 7ff916800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff91815d760 0x7ff9180b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:23.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.721+0000 7ff916800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff91815d760 0x7ff9180b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43592/0 (socket says 172.21.15.100:43592) 2024-09-17T04:26:23.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.721+0000 7ff916800640 1 -- 172.21.15.100:0/886217681 learned_addr learned my addr 172.21.15.100:0/886217681 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:26:23.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.722+0000 7ff915e00640 1 --2- 172.21.15.100:0/886217681 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9180ad0e0 0x7ff91814a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:23.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.722+0000 7ff915400640 1 --2- 172.21.15.100:0/886217681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9180b3550 0x7ff9180b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:23.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.722+0000 7ff916800640 1 -- 172.21.15.100:0/886217681 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9180ad0e0 msgr2=0x7ff91814a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:23.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.722+0000 7ff916800640 1 --2- 172.21.15.100:0/886217681 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9180ad0e0 0x7ff91814a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:23.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.722+0000 7ff916800640 1 -- 172.21.15.100:0/886217681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9180b3550 msgr2=0x7ff9180b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:23.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.722+0000 7ff916800640 1 --2- 172.21.15.100:0/886217681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9180b3550 0x7ff9180b0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:23.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.722+0000 7ff916800640 1 -- 172.21.15.100:0/886217681 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91814ad30 con 0x7ff91815d760 2024-09-17T04:26:23.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.722+0000 7ff916800640 1 --2- 172.21.15.100:0/886217681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff91815d760 0x7ff9180b12e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff91000cb10 tx=0x7ff91000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:26:23.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.722+0000 7ff90e000640 1 -- 172.21.15.100:0/886217681 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910016020 con 0x7ff91815d760 2024-09-17T04:26:23.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.723+0000 7ff90e000640 1 -- 172.21.15.100:0/886217681 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff91000aca0 con 0x7ff91815d760 2024-09-17T04:26:23.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.723+0000 7ff91cc00640 1 -- 172.21.15.100:0/886217681 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9181f5780 con 0x7ff91815d760 2024-09-17T04:26:23.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.723+0000 7ff90e000640 1 -- 172.21.15.100:0/886217681 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910012680 con 0x7ff91815d760 2024-09-17T04:26:23.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.723+0000 7ff91cc00640 1 -- 172.21.15.100:0/886217681 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff918155220 con 0x7ff91815d760 2024-09-17T04:26:23.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.724+0000 7ff91cc00640 1 -- 172.21.15.100:0/886217681 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9181f5950 con 0x7ff91815d760 2024-09-17T04:26:23.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.725+0000 7ff90e000640 1 -- 172.21.15.100:0/886217681 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7ff910019020 con 0x7ff91815d760 2024-09-17T04:26:23.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.726+0000 7ff90e000640 1 --2- 172.21.15.100:0/886217681 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff8f8078dd0 0x7ff8f807b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:23.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.727+0000 7ff90e000640 1 -- 172.21.15.100:0/886217681 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7ff91009b490 con 0x7ff91815d760 2024-09-17T04:26:23.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.727+0000 7ff915e00640 1 --2- 172.21.15.100:0/886217681 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff8f8078dd0 0x7ff8f807b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:23.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.728+0000 7ff915e00640 1 --2- 172.21.15.100:0/886217681 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff8f8078dd0 0x7ff8f807b2e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff908006fd0 tx=0x7ff908009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:26:23.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.731+0000 7ff90e000640 1 -- 172.21.15.100:0/886217681 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff910020030 con 0x7ff91815d760 2024-09-17T04:26:23.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.948+0000 7ff91cc00640 1 -- 172.21.15.100:0/886217681 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff918158340 con 0x7ff91815d760 2024-09-17T04:26:23.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.949+0000 7ff90e000640 1 -- 172.21.15.100:0/886217681 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7ff9180ada30 con 0x7ff91815d760 2024-09-17T04:26:23.949 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:26:23.949 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"btime":"2024-09-17T04:25:56:247389+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1086873435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1086873435},{"type":"v1","addr":"172.21.15.100:6833","nonce":1086873435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1708229764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1708229764},{"type":"v1","addr":"172.21.15.184:6836","nonce":1708229764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/828832783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":828832783},{"type":"v1","addr":"172.21.15.184:6837","nonce":828832783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:44.669203+0000","modified":"2024-09-17T04:25:47.693203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3801048032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3801048032},{"type":"v1","addr":"172.21.15.100:6835","nonce":3801048032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5295,"qdb_cluster":[5295]},"id":3}]} 2024-09-17T04:26:23.950 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-09-17T04:26:23.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.951+0000 7ff91cc00640 1 -- 172.21.15.100:0/886217681 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff8f8078dd0 msgr2=0x7ff8f807b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:23.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.951+0000 7ff91cc00640 1 --2- 172.21.15.100:0/886217681 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff8f8078dd0 0x7ff8f807b2e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff908006fd0 tx=0x7ff908009060 comp rx=0 tx=0).stop 2024-09-17T04:26:23.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.951+0000 7ff91cc00640 1 -- 172.21.15.100:0/886217681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff91815d760 msgr2=0x7ff9180b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:23.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.951+0000 7ff91cc00640 1 --2- 172.21.15.100:0/886217681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff91815d760 0x7ff9180b12e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff91000cb10 tx=0x7ff91000a060 comp rx=0 tx=0).stop 2024-09-17T04:26:23.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.951+0000 7ff91cc00640 1 -- 172.21.15.100:0/886217681 shutdown_connections 2024-09-17T04:26:23.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.951+0000 7ff91cc00640 1 --2- 172.21.15.100:0/886217681 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff8f8078dd0 0x7ff8f807b2e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:23.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.951+0000 7ff91cc00640 1 --2- 172.21.15.100:0/886217681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff91815d760 0x7ff9180b12e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:23.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.951+0000 7ff91cc00640 1 --2- 172.21.15.100:0/886217681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9180b3550 0x7ff9180b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:23.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.951+0000 7ff91cc00640 1 --2- 172.21.15.100:0/886217681 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9180ad0e0 0x7ff91814a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:23.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.951+0000 7ff91cc00640 1 -- 172.21.15.100:0/886217681 >> 172.21.15.100:0/886217681 conn(0x7ff91806d730 msgr2=0x7ff9180b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:26:23.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.951+0000 7ff91cc00640 1 -- 172.21.15.100:0/886217681 shutdown_connections 2024-09-17T04:26:23.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:23.951+0000 7ff91cc00640 1 -- 172.21.15.100:0/886217681 wait complete. 2024-09-17T04:26:23.976 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-17T04:26:24.222 INFO:teuthology.orchestra.run.smithi100.stdout:[{"id":5419,"entity":{"name":{"type":"client","num":5419},"addr":{"type":"any","addr":"172.21.15.184:0","nonce":1392734336}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":11,"uptime":24.442344955999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.792795518689811,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5419 172.21.15.184:0/1392734336","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"ce1b342bf8c6349a8c90cd607f83bff2443f21cf","ceph_version":"ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)","entity_id":"x","hostname":"smithi184","pid":"35968","root":"/"}}] 2024-09-17T04:26:24.222 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi184", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35968", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.184:0", "nonce": 1392734336, "type": "any" }, "name": { "num": 5419, "type": "client" } }, "id": 5419, "inst": "client.5419 172.21.15.184:0/1392734336", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 11, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.79279551868981 }, "state": "open", "uptime": 24.442344956 } ] 2024-09-17T04:26:24.222 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-17T04:26:29.223 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:26:29.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.453+0000 7f3967e00640 1 -- 172.21.15.100:0/1403839443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39680b9e00 msgr2=0x7f3968162a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:29.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.453+0000 7f3967e00640 1 --2- 172.21.15.100:0/1403839443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39680b9e00 0x7f3968162a50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f39680d3820 tx=0x7f395802a4a0 comp rx=0 tx=0).stop 2024-09-17T04:26:29.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.453+0000 7f3967e00640 1 -- 172.21.15.100:0/1403839443 shutdown_connections 2024-09-17T04:26:29.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.453+0000 7f3967e00640 1 --2- 172.21.15.100:0/1403839443 >> v2:172.21.15.184:3300/0 conn(0x7f39680b3160 0x7f39680d24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:29.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.453+0000 7f3967e00640 1 --2- 172.21.15.100:0/1403839443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39680b9e00 0x7f3968162a50 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:29.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.453+0000 7f3967e00640 1 -- 172.21.15.100:0/1403839443 >> 172.21.15.100:0/1403839443 conn(0x7f396808bfe0 msgr2=0x7f396809d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:26:29.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.453+0000 7f3967e00640 1 -- 172.21.15.100:0/1403839443 shutdown_connections 2024-09-17T04:26:29.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.453+0000 7f3967e00640 1 -- 172.21.15.100:0/1403839443 wait complete. 2024-09-17T04:26:29.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.455+0000 7f3967e00640 1 Processor -- start 2024-09-17T04:26:29.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.455+0000 7f3967e00640 1 -- start start 2024-09-17T04:26:29.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.457+0000 7f3967e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39680b3160 0x7f396815fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:29.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.457+0000 7f3967e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39680b9e00 0x7f39680a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:29.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.457+0000 7f3967e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39680d29e0 0x7f39680a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:29.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.457+0000 7f3967e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f39680d44b0 con 0x7f39680d29e0 2024-09-17T04:26:29.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.457+0000 7f3967e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f39680d4330 con 0x7f39680b9e00 2024-09-17T04:26:29.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.457+0000 7f3967e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f39680d4630 con 0x7f39680b3160 2024-09-17T04:26:29.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.457+0000 7f3967400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39680d29e0 0x7f39680a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:29.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.457+0000 7f3967400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39680d29e0 0x7f39680a1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53068/0 (socket says 172.21.15.100:53068) 2024-09-17T04:26:29.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.457+0000 7f3967400640 1 -- 172.21.15.100:0/272516408 learned_addr learned my addr 172.21.15.100:0/272516408 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:26:29.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.457+0000 7f3966a00640 1 --2- 172.21.15.100:0/272516408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39680b3160 0x7f396815fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:29.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.457+0000 7f395e000640 1 --2- 172.21.15.100:0/272516408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39680b9e00 0x7f39680a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:29.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.457+0000 7f3967400640 1 -- 172.21.15.100:0/272516408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39680b3160 msgr2=0x7f396815fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:29.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.457+0000 7f3967400640 1 --2- 172.21.15.100:0/272516408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39680b3160 0x7f396815fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:29.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.457+0000 7f3967400640 1 -- 172.21.15.100:0/272516408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39680b9e00 msgr2=0x7f39680a0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:29.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.457+0000 7f3967400640 1 --2- 172.21.15.100:0/272516408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39680b9e00 0x7f39680a0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:29.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.457+0000 7f3967400640 1 -- 172.21.15.100:0/272516408 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39681f68f0 con 0x7f39680d29e0 2024-09-17T04:26:29.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.458+0000 7f3967400640 1 --2- 172.21.15.100:0/272516408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39680d29e0 0x7f39680a1490 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f396000c760 tx=0x7f396000cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:26:29.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.458+0000 7f395f400640 1 -- 172.21.15.100:0/272516408 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3960003b70 con 0x7f39680d29e0 2024-09-17T04:26:29.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.458+0000 7f395f400640 1 -- 172.21.15.100:0/272516408 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3960003d10 con 0x7f39680d29e0 2024-09-17T04:26:29.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.458+0000 7f3967e00640 1 -- 172.21.15.100:0/272516408 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f396809f250 con 0x7f39680d29e0 2024-09-17T04:26:29.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.458+0000 7f395f400640 1 -- 172.21.15.100:0/272516408 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39600052d0 con 0x7f39680d29e0 2024-09-17T04:26:29.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.458+0000 7f3967e00640 1 -- 172.21.15.100:0/272516408 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39681f71e0 con 0x7f39680d29e0 2024-09-17T04:26:29.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.459+0000 7f3967e00640 1 -- 172.21.15.100:0/272516408 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39680d4b50 con 0x7f39680d29e0 2024-09-17T04:26:29.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.460+0000 7f395f400640 1 -- 172.21.15.100:0/272516408 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7f3960005470 con 0x7f39680d29e0 2024-09-17T04:26:29.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.461+0000 7f395f400640 1 --2- 172.21.15.100:0/272516408 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3944078dd0 0x7f394407b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:29.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.461+0000 7f395f400640 1 -- 172.21.15.100:0/272516408 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f396009ae70 con 0x7f39680d29e0 2024-09-17T04:26:29.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.462+0000 7f3966a00640 1 --2- 172.21.15.100:0/272516408 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3944078dd0 0x7f394407b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:29.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.463+0000 7f3966a00640 1 --2- 172.21.15.100:0/272516408 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3944078dd0 0x7f394407b2e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3958034000 tx=0x7f3958004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:26:29.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.466+0000 7f395f400640 1 -- 172.21.15.100:0/272516408 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3960062830 con 0x7f39680d29e0 2024-09-17T04:26:29.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.686+0000 7f3967e00640 1 -- 172.21.15.100:0/272516408 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39680b4de0 con 0x7f39680d29e0 2024-09-17T04:26:29.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.686+0000 7f395f400640 1 -- 172.21.15.100:0/272516408 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f3960067260 con 0x7f39680d29e0 2024-09-17T04:26:29.687 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:26:29.687 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"btime":"2024-09-17T04:25:56:247389+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1086873435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1086873435},{"type":"v1","addr":"172.21.15.100:6833","nonce":1086873435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1708229764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1708229764},{"type":"v1","addr":"172.21.15.184:6836","nonce":1708229764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/828832783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":828832783},{"type":"v1","addr":"172.21.15.184:6837","nonce":828832783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:44.669203+0000","modified":"2024-09-17T04:25:47.693203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3801048032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3801048032},{"type":"v1","addr":"172.21.15.100:6835","nonce":3801048032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5295,"qdb_cluster":[5295]},"id":3}]} 2024-09-17T04:26:29.687 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-09-17T04:26:29.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.689+0000 7f3967e00640 1 -- 172.21.15.100:0/272516408 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3944078dd0 msgr2=0x7f394407b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:29.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.689+0000 7f3967e00640 1 --2- 172.21.15.100:0/272516408 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3944078dd0 0x7f394407b2e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3958034000 tx=0x7f3958004730 comp rx=0 tx=0).stop 2024-09-17T04:26:29.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.689+0000 7f3967e00640 1 -- 172.21.15.100:0/272516408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39680d29e0 msgr2=0x7f39680a1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:29.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.689+0000 7f3967e00640 1 --2- 172.21.15.100:0/272516408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39680d29e0 0x7f39680a1490 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f396000c760 tx=0x7f396000cb60 comp rx=0 tx=0).stop 2024-09-17T04:26:29.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.689+0000 7f3967e00640 1 -- 172.21.15.100:0/272516408 shutdown_connections 2024-09-17T04:26:29.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.689+0000 7f3967e00640 1 --2- 172.21.15.100:0/272516408 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3944078dd0 0x7f394407b2e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:29.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.689+0000 7f3967e00640 1 --2- 172.21.15.100:0/272516408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39680d29e0 0x7f39680a1490 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:29.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.689+0000 7f3967e00640 1 --2- 172.21.15.100:0/272516408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39680b9e00 0x7f39680a0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:29.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.689+0000 7f3967e00640 1 --2- 172.21.15.100:0/272516408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39680b3160 0x7f396815fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:29.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.689+0000 7f3967e00640 1 -- 172.21.15.100:0/272516408 >> 172.21.15.100:0/272516408 conn(0x7f396808bfe0 msgr2=0x7f3968068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:26:29.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.689+0000 7f3967e00640 1 -- 172.21.15.100:0/272516408 shutdown_connections 2024-09-17T04:26:29.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:29.689+0000 7f3967e00640 1 -- 172.21.15.100:0/272516408 wait complete. 2024-09-17T04:26:29.712 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-17T04:26:29.953 INFO:teuthology.orchestra.run.smithi100.stdout:[{"id":5419,"entity":{"name":{"type":"client","num":5419},"addr":{"type":"any","addr":"172.21.15.184:0","nonce":1392734336}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":10,"uptime":30.176191613,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.519149223506322,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5419 172.21.15.184:0/1392734336","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"ce1b342bf8c6349a8c90cd607f83bff2443f21cf","ceph_version":"ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)","entity_id":"x","hostname":"smithi184","pid":"35968","root":"/"}}] 2024-09-17T04:26:29.954 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi184", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35968", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.184:0", "nonce": 1392734336, "type": "any" }, "name": { "num": 5419, "type": "client" } }, "id": 5419, "inst": "client.5419 172.21.15.184:0/1392734336", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.51914922350632 }, "state": "open", "uptime": 30.176191613 } ] 2024-09-17T04:26:29.954 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-17T04:26:34.889 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:26:34.894 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:26:34.955 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:26:35.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.179+0000 7fddf4000640 1 -- 172.21.15.100:0/1350675807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddd4004800 msgr2=0x7fddd4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:35.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.179+0000 7fddf4000640 1 --2- 172.21.15.100:0/1350675807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddd4004800 0x7fddd4004c10 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fddd8006f50 tx=0x7fddd4005830 comp rx=0 tx=0).stop 2024-09-17T04:26:35.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.179+0000 7fddf2000640 1 -- 172.21.15.100:0/1350675807 reap_dead start 2024-09-17T04:26:35.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.179+0000 7fddf4000640 1 -- 172.21.15.100:0/1350675807 shutdown_connections 2024-09-17T04:26:35.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.179+0000 7fddf4000640 1 -- 172.21.15.100:0/1350675807 >> 172.21.15.100:0/1350675807 conn(0x7fddec06d730 msgr2=0x7fddec0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:26:35.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.179+0000 7fddf4000640 1 -- 172.21.15.100:0/1350675807 shutdown_connections 2024-09-17T04:26:35.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.179+0000 7fddf4000640 1 -- 172.21.15.100:0/1350675807 wait complete. 2024-09-17T04:26:35.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.182+0000 7fddf4000640 1 Processor -- start 2024-09-17T04:26:35.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.182+0000 7fddf4000640 1 -- start start 2024-09-17T04:26:35.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.183+0000 7fddf4000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddd4004800 0x7fddec15b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:35.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.183+0000 7fddf4000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddec074370 0x7fddec1566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:35.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.183+0000 7fddf4000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddec14cf00 0x7fddec14d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:35.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.183+0000 7fddf4000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fddec0b4e30 con 0x7fddd4004800 2024-09-17T04:26:35.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.183+0000 7fddf4000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fddec165890 con 0x7fddec074370 2024-09-17T04:26:35.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.183+0000 7fddf4000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fddec15c200 con 0x7fddec14cf00 2024-09-17T04:26:35.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.183+0000 7fddf1600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddd4004800 0x7fddec15b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:35.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.183+0000 7fddf1600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddd4004800 0x7fddec15b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57506/0 (socket says 172.21.15.100:57506) 2024-09-17T04:26:35.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.183+0000 7fddf1600640 1 -- 172.21.15.100:0/2233456580 learned_addr learned my addr 172.21.15.100:0/2233456580 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:26:35.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.184+0000 7fddf1600640 1 -- 172.21.15.100:0/2233456580 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddec14cf00 msgr2=0x7fddec14d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:35.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.184+0000 7fddf1600640 1 --2- 172.21.15.100:0/2233456580 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddec14cf00 0x7fddec14d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:35.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.184+0000 7fddf1600640 1 -- 172.21.15.100:0/2233456580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddec074370 msgr2=0x7fddec1566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:35.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.184+0000 7fddf1600640 1 --2- 172.21.15.100:0/2233456580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddec074370 0x7fddec1566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:35.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.184+0000 7fddf1600640 1 -- 172.21.15.100:0/2233456580 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddec15bd30 con 0x7fddd4004800 2024-09-17T04:26:35.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.184+0000 7fddf1600640 1 --2- 172.21.15.100:0/2233456580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddd4004800 0x7fddec15b7f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fdde0003f00 tx=0x7fdde0001730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:26:35.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.184+0000 7fdde9600640 1 -- 172.21.15.100:0/2233456580 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde0011640 con 0x7fddd4004800 2024-09-17T04:26:35.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.184+0000 7fdde9600640 1 -- 172.21.15.100:0/2233456580 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdde0016020 con 0x7fddd4004800 2024-09-17T04:26:35.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.184+0000 7fddf4000640 1 -- 172.21.15.100:0/2233456580 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddec074d90 con 0x7fddd4004800 2024-09-17T04:26:35.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.184+0000 7fdde9600640 1 -- 172.21.15.100:0/2233456580 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde0011aa0 con 0x7fddd4004800 2024-09-17T04:26:35.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.184+0000 7fddf4000640 1 -- 172.21.15.100:0/2233456580 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddec144470 con 0x7fddd4004800 2024-09-17T04:26:35.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.185+0000 7fddf4000640 1 -- 172.21.15.100:0/2233456580 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddec1535c0 con 0x7fddd4004800 2024-09-17T04:26:35.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.187+0000 7fdde9600640 1 -- 172.21.15.100:0/2233456580 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7fdde0011c40 con 0x7fddd4004800 2024-09-17T04:26:35.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.188+0000 7fdde9600640 1 --2- 172.21.15.100:0/2233456580 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fddcc078dd0 0x7fddcc07b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:35.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.188+0000 7fdde9600640 1 -- 172.21.15.100:0/2233456580 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7fdde009c7c0 con 0x7fddd4004800 2024-09-17T04:26:35.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.188+0000 7fddf0c00640 1 --2- 172.21.15.100:0/2233456580 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fddcc078dd0 0x7fddcc07b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:35.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.189+0000 7fddf0c00640 1 --2- 172.21.15.100:0/2233456580 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fddcc078dd0 0x7fddcc07b2e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fddd8033000 tx=0x7fddec0af720 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:26:35.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.191+0000 7fdde9600640 1 -- 172.21.15.100:0/2233456580 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fdde0020020 con 0x7fddd4004800 2024-09-17T04:26:35.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.409+0000 7fddf4000640 1 -- 172.21.15.100:0/2233456580 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddec151bf0 con 0x7fddd4004800 2024-09-17T04:26:35.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.410+0000 7fdde9600640 1 -- 172.21.15.100:0/2233456580 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7fdde0064730 con 0x7fddd4004800 2024-09-17T04:26:35.409 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:26:35.410 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"btime":"2024-09-17T04:25:56:247389+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1086873435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1086873435},{"type":"v1","addr":"172.21.15.100:6833","nonce":1086873435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1708229764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1708229764},{"type":"v1","addr":"172.21.15.184:6836","nonce":1708229764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/828832783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":828832783},{"type":"v1","addr":"172.21.15.184:6837","nonce":828832783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:44.669203+0000","modified":"2024-09-17T04:25:47.693203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3801048032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3801048032},{"type":"v1","addr":"172.21.15.100:6835","nonce":3801048032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5295,"qdb_cluster":[5295]},"id":3}]} 2024-09-17T04:26:35.410 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-09-17T04:26:35.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.412+0000 7fddf4000640 1 -- 172.21.15.100:0/2233456580 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fddcc078dd0 msgr2=0x7fddcc07b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:35.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.412+0000 7fddf4000640 1 --2- 172.21.15.100:0/2233456580 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fddcc078dd0 0x7fddcc07b2e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fddd8033000 tx=0x7fddec0af720 comp rx=0 tx=0).stop 2024-09-17T04:26:35.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.412+0000 7fddf4000640 1 -- 172.21.15.100:0/2233456580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddd4004800 msgr2=0x7fddec15b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:35.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.412+0000 7fddf4000640 1 --2- 172.21.15.100:0/2233456580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddd4004800 0x7fddec15b7f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fdde0003f00 tx=0x7fdde0001730 comp rx=0 tx=0).stop 2024-09-17T04:26:35.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.413+0000 7fddf4000640 1 -- 172.21.15.100:0/2233456580 shutdown_connections 2024-09-17T04:26:35.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.413+0000 7fddf4000640 1 --2- 172.21.15.100:0/2233456580 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fddcc078dd0 0x7fddcc07b2e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:35.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.413+0000 7fddf4000640 1 --2- 172.21.15.100:0/2233456580 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddec14cf00 0x7fddec14d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:35.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.413+0000 7fddf4000640 1 --2- 172.21.15.100:0/2233456580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddec074370 0x7fddec1566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:35.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.413+0000 7fddf4000640 1 --2- 172.21.15.100:0/2233456580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddd4004800 0x7fddec15b7f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:35.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.413+0000 7fddf4000640 1 -- 172.21.15.100:0/2233456580 >> 172.21.15.100:0/2233456580 conn(0x7fddec06d730 msgr2=0x7fddec15f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:26:35.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.413+0000 7fddf4000640 1 -- 172.21.15.100:0/2233456580 shutdown_connections 2024-09-17T04:26:35.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:35.413+0000 7fddf4000640 1 -- 172.21.15.100:0/2233456580 wait complete. 2024-09-17T04:26:35.438 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-17T04:26:35.681 INFO:teuthology.orchestra.run.smithi100.stdout:[{"id":5419,"entity":{"name":{"type":"client","num":5419},"addr":{"type":"any","addr":"172.21.15.184:0","nonce":1392734336}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":9,"uptime":35.901038511000003,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.249525347441534,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5419 172.21.15.184:0/1392734336","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"ce1b342bf8c6349a8c90cd607f83bff2443f21cf","ceph_version":"ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)","entity_id":"x","hostname":"smithi184","pid":"35968","root":"/"}}] 2024-09-17T04:26:35.681 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi184", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35968", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.184:0", "nonce": 1392734336, "type": "any" }, "name": { "num": 5419, "type": "client" } }, "id": 5419, "inst": "client.5419 172.21.15.184:0/1392734336", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 9, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.249525347441534 }, "state": "open", "uptime": 35.901038511 } ] 2024-09-17T04:26:35.681 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-17T04:26:40.683 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:26:40.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.914+0000 7f9e33800640 1 -- 172.21.15.100:0/2451273758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e2c15d760 msgr2=0x7f9e2c1535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:40.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.914+0000 7f9e33800640 1 --2- 172.21.15.100:0/2451273758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e2c15d760 0x7f9e2c1535c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c005fd0 tx=0x7f9e1c02a820 comp rx=0 tx=0).stop 2024-09-17T04:26:40.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.914+0000 7f9e33800640 1 -- 172.21.15.100:0/2451273758 shutdown_connections 2024-09-17T04:26:40.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.914+0000 7f9e33800640 1 --2- 172.21.15.100:0/2451273758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e2c15d760 0x7f9e2c1535c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:40.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.914+0000 7f9e33800640 1 -- 172.21.15.100:0/2451273758 >> 172.21.15.100:0/2451273758 conn(0x7f9e2c06d730 msgr2=0x7f9e2c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:26:40.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.914+0000 7f9e33800640 1 -- 172.21.15.100:0/2451273758 shutdown_connections 2024-09-17T04:26:40.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.914+0000 7f9e33800640 1 -- 172.21.15.100:0/2451273758 wait complete. 2024-09-17T04:26:40.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.916+0000 7f9e33800640 1 Processor -- start 2024-09-17T04:26:40.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.917+0000 7f9e33800640 1 -- start start 2024-09-17T04:26:40.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.918+0000 7f9e33800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e2c0ad0e0 0x7f9e2c153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:40.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.918+0000 7f9e33800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e2c0b3550 0x7f9e2c1559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:40.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.918+0000 7f9e33800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e2c15d760 0x7f9e2c14b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:40.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.918+0000 7f9e33800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c0b4da0 con 0x7f9e2c0ad0e0 2024-09-17T04:26:40.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.918+0000 7f9e33800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c0b4c20 con 0x7f9e2c0b3550 2024-09-17T04:26:40.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.918+0000 7f9e33800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9e2c0b4070 con 0x7f9e2c15d760 2024-09-17T04:26:40.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.918+0000 7f9e30e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e2c0ad0e0 0x7f9e2c153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:40.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.918+0000 7f9e30e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e2c0ad0e0 0x7f9e2c153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57520/0 (socket says 172.21.15.100:57520) 2024-09-17T04:26:40.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.918+0000 7f9e30e00640 1 -- 172.21.15.100:0/3419447938 learned_addr learned my addr 172.21.15.100:0/3419447938 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:26:40.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.918+0000 7f9e2be00640 1 --2- 172.21.15.100:0/3419447938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e2c0b3550 0x7f9e2c1559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:40.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.919+0000 7f9e31800640 1 --2- 172.21.15.100:0/3419447938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e2c15d760 0x7f9e2c14b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:40.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.919+0000 7f9e30e00640 1 -- 172.21.15.100:0/3419447938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e2c15d760 msgr2=0x7f9e2c14b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:40.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.919+0000 7f9e30e00640 1 --2- 172.21.15.100:0/3419447938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e2c15d760 0x7f9e2c14b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:40.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.919+0000 7f9e30e00640 1 -- 172.21.15.100:0/3419447938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e2c0b3550 msgr2=0x7f9e2c1559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:40.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.919+0000 7f9e30e00640 1 --2- 172.21.15.100:0/3419447938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e2c0b3550 0x7f9e2c1559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:40.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.919+0000 7f9e30e00640 1 -- 172.21.15.100:0/3419447938 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e2c1f67e0 con 0x7f9e2c0ad0e0 2024-09-17T04:26:40.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.919+0000 7f9e2be00640 1 --2- 172.21.15.100:0/3419447938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e2c0b3550 0x7f9e2c1559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:26:40.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.919+0000 7f9e30e00640 1 --2- 172.21.15.100:0/3419447938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e2c0ad0e0 0x7f9e2c153e40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c009cd0 tx=0x7f9e1c032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:26:40.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.919+0000 7f9e28c00640 1 -- 172.21.15.100:0/3419447938 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c037040 con 0x7f9e2c0ad0e0 2024-09-17T04:26:40.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.919+0000 7f9e28c00640 1 -- 172.21.15.100:0/3419447938 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e1c042070 con 0x7f9e2c0ad0e0 2024-09-17T04:26:40.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.919+0000 7f9e33800640 1 -- 172.21.15.100:0/3419447938 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e2c154690 con 0x7f9e2c0ad0e0 2024-09-17T04:26:40.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.919+0000 7f9e28c00640 1 -- 172.21.15.100:0/3419447938 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c004280 con 0x7f9e2c0ad0e0 2024-09-17T04:26:40.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.920+0000 7f9e33800640 1 -- 172.21.15.100:0/3419447938 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e2c154c50 con 0x7f9e2c0ad0e0 2024-09-17T04:26:40.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.920+0000 7f9e33800640 1 -- 172.21.15.100:0/3419447938 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e2c1535c0 con 0x7f9e2c0ad0e0 2024-09-17T04:26:40.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.922+0000 7f9e28c00640 1 -- 172.21.15.100:0/3419447938 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7f9e1c004420 con 0x7f9e2c0ad0e0 2024-09-17T04:26:40.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.923+0000 7f9e28c00640 1 --2- 172.21.15.100:0/3419447938 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9e0c078e00 0x7f9e0c07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:40.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.923+0000 7f9e28c00640 1 -- 172.21.15.100:0/3419447938 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f9e1c0bc1a0 con 0x7f9e2c0ad0e0 2024-09-17T04:26:40.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.923+0000 7f9e2be00640 1 --2- 172.21.15.100:0/3419447938 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9e0c078e00 0x7f9e0c07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:40.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.924+0000 7f9e2be00640 1 --2- 172.21.15.100:0/3419447938 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9e0c078e00 0x7f9e0c07b310 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9e18006fd0 tx=0x7f9e18008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:26:40.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:40.927+0000 7f9e28c00640 1 -- 172.21.15.100:0/3419447938 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9e1c084090 con 0x7f9e2c0ad0e0 2024-09-17T04:26:41.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:41.143+0000 7f9e33800640 1 -- 172.21.15.100:0/3419447938 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e2c06e060 con 0x7f9e2c0ad0e0 2024-09-17T04:26:41.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:41.144+0000 7f9e28c00640 1 -- 172.21.15.100:0/3419447938 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f9e1c083990 con 0x7f9e2c0ad0e0 2024-09-17T04:26:41.144 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:26:41.144 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"btime":"2024-09-17T04:25:56:247389+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1086873435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1086873435},{"type":"v1","addr":"172.21.15.100:6833","nonce":1086873435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1708229764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1708229764},{"type":"v1","addr":"172.21.15.184:6836","nonce":1708229764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/828832783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":828832783},{"type":"v1","addr":"172.21.15.184:6837","nonce":828832783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:44.669203+0000","modified":"2024-09-17T04:25:47.693203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3801048032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3801048032},{"type":"v1","addr":"172.21.15.100:6835","nonce":3801048032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5295,"qdb_cluster":[5295]},"id":3}]} 2024-09-17T04:26:41.144 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-09-17T04:26:41.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:41.146+0000 7f9e33800640 1 -- 172.21.15.100:0/3419447938 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9e0c078e00 msgr2=0x7f9e0c07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:41.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:41.146+0000 7f9e33800640 1 --2- 172.21.15.100:0/3419447938 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9e0c078e00 0x7f9e0c07b310 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9e18006fd0 tx=0x7f9e18008040 comp rx=0 tx=0).stop 2024-09-17T04:26:41.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:41.146+0000 7f9e33800640 1 -- 172.21.15.100:0/3419447938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e2c0ad0e0 msgr2=0x7f9e2c153e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:41.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:41.146+0000 7f9e33800640 1 --2- 172.21.15.100:0/3419447938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e2c0ad0e0 0x7f9e2c153e40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c009cd0 tx=0x7f9e1c032d10 comp rx=0 tx=0).stop 2024-09-17T04:26:41.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:41.147+0000 7f9e33800640 1 -- 172.21.15.100:0/3419447938 shutdown_connections 2024-09-17T04:26:41.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:41.147+0000 7f9e33800640 1 --2- 172.21.15.100:0/3419447938 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9e0c078e00 0x7f9e0c07b310 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:41.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:41.147+0000 7f9e33800640 1 --2- 172.21.15.100:0/3419447938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e2c15d760 0x7f9e2c14b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:41.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:41.147+0000 7f9e33800640 1 --2- 172.21.15.100:0/3419447938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e2c0b3550 0x7f9e2c1559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:41.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:41.147+0000 7f9e33800640 1 --2- 172.21.15.100:0/3419447938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e2c0ad0e0 0x7f9e2c153e40 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:41.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:41.147+0000 7f9e33800640 1 -- 172.21.15.100:0/3419447938 >> 172.21.15.100:0/3419447938 conn(0x7f9e2c06d730 msgr2=0x7f9e2c1602a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:26:41.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:41.147+0000 7f9e33800640 1 -- 172.21.15.100:0/3419447938 shutdown_connections 2024-09-17T04:26:41.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:41.147+0000 7f9e33800640 1 -- 172.21.15.100:0/3419447938 wait complete. 2024-09-17T04:26:41.171 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-17T04:26:41.415 INFO:teuthology.orchestra.run.smithi100.stdout:[{"id":5419,"entity":{"name":{"type":"client","num":5419},"addr":{"type":"any","addr":"172.21.15.184:0","nonce":1392734336}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":9,"uptime":41.637885089000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.982890322473406,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5419 172.21.15.184:0/1392734336","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"ce1b342bf8c6349a8c90cd607f83bff2443f21cf","ceph_version":"ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)","entity_id":"x","hostname":"smithi184","pid":"35968","root":"/"}}] 2024-09-17T04:26:41.416 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi184", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35968", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.184:0", "nonce": 1392734336, "type": "any" }, "name": { "num": 5419, "type": "client" } }, "id": 5419, "inst": "client.5419 172.21.15.184:0/1392734336", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 9, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.982890322473406 }, "state": "open", "uptime": 41.637885089 } ] 2024-09-17T04:26:41.416 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-17T04:26:46.417 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:26:46.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.650+0000 7fa83ae00640 1 -- 172.21.15.100:0/3651079181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa81c004800 msgr2=0x7fa81c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:46.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.650+0000 7fa83ae00640 1 --2- 172.21.15.100:0/3651079181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa81c004800 0x7fa81c004c10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa82000c310 tx=0x7fa81c005830 comp rx=0 tx=0).stop 2024-09-17T04:26:46.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.650+0000 7fa83ae00640 1 -- 172.21.15.100:0/3651079181 shutdown_connections 2024-09-17T04:26:46.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.650+0000 7fa83ae00640 1 --2- 172.21.15.100:0/3651079181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa81c004800 0x7fa81c004c10 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:46.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.650+0000 7fa83ae00640 1 -- 172.21.15.100:0/3651079181 >> 172.21.15.100:0/3651079181 conn(0x7fa8340b1e40 msgr2=0x7fa8340b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:26:46.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.650+0000 7fa83ae00640 1 -- 172.21.15.100:0/3651079181 shutdown_connections 2024-09-17T04:26:46.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.650+0000 7fa83ae00640 1 -- 172.21.15.100:0/3651079181 wait complete. 2024-09-17T04:26:46.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.652+0000 7fa83ae00640 1 Processor -- start 2024-09-17T04:26:46.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.652+0000 7fa83ae00640 1 -- start start 2024-09-17T04:26:46.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.654+0000 7fa83ae00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa81c004800 0x7fa83414bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:46.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.654+0000 7fa83ae00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8340c4f10 0x7fa8341f6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:46.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.654+0000 7fa83ae00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8341f7190 0x7fa83414eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:46.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.654+0000 7fa83ae00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa8340af500 con 0x7fa8340c4f10 2024-09-17T04:26:46.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.654+0000 7fa83ae00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa8340af380 con 0x7fa81c004800 2024-09-17T04:26:46.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.654+0000 7fa83ae00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa83415d670 con 0x7fa8341f7190 2024-09-17T04:26:46.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.654+0000 7fa839000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8340c4f10 0x7fa8341f6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:46.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.654+0000 7fa839000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8340c4f10 0x7fa8341f6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:32880/0 (socket says 172.21.15.100:32880) 2024-09-17T04:26:46.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.654+0000 7fa839000640 1 -- 172.21.15.100:0/1352583531 learned_addr learned my addr 172.21.15.100:0/1352583531 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:26:46.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.654+0000 7fa83a400640 1 --2- 172.21.15.100:0/1352583531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8341f7190 0x7fa83414eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:46.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.654+0000 7fa839a00640 1 --2- 172.21.15.100:0/1352583531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa81c004800 0x7fa83414bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:46.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.655+0000 7fa839000640 1 -- 172.21.15.100:0/1352583531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8341f7190 msgr2=0x7fa83414eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:46.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.655+0000 7fa839000640 1 --2- 172.21.15.100:0/1352583531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8341f7190 0x7fa83414eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:46.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.655+0000 7fa839000640 1 -- 172.21.15.100:0/1352583531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa81c004800 msgr2=0x7fa83414bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:46.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.655+0000 7fa839000640 1 --2- 172.21.15.100:0/1352583531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa81c004800 0x7fa83414bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:46.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.655+0000 7fa839000640 1 -- 172.21.15.100:0/1352583531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8340c5680 con 0x7fa8340c4f10 2024-09-17T04:26:46.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.655+0000 7fa839000640 1 --2- 172.21.15.100:0/1352583531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8340c4f10 0x7fa8341f6c50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa82000c310 tx=0x7fa820001d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:26:46.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.655+0000 7fa831600640 1 -- 172.21.15.100:0/1352583531 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820031040 con 0x7fa8340c4f10 2024-09-17T04:26:46.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.655+0000 7fa831600640 1 -- 172.21.15.100:0/1352583531 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa820003800 con 0x7fa8340c4f10 2024-09-17T04:26:46.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.655+0000 7fa83ae00640 1 -- 172.21.15.100:0/1352583531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa83409bde0 con 0x7fa8340c4f10 2024-09-17T04:26:46.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.655+0000 7fa831600640 1 -- 172.21.15.100:0/1352583531 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820006600 con 0x7fa8340c4f10 2024-09-17T04:26:46.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.655+0000 7fa83ae00640 1 -- 172.21.15.100:0/1352583531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa83409c3a0 con 0x7fa8340c4f10 2024-09-17T04:26:46.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.656+0000 7fa83ae00640 1 -- 172.21.15.100:0/1352583531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8340c53f0 con 0x7fa8340c4f10 2024-09-17T04:26:46.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.658+0000 7fa831600640 1 -- 172.21.15.100:0/1352583531 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7fa82003e030 con 0x7fa8340c4f10 2024-09-17T04:26:46.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.658+0000 7fa831600640 1 --2- 172.21.15.100:0/1352583531 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa814078dd0 0x7fa81407b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:46.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.659+0000 7fa831600640 1 -- 172.21.15.100:0/1352583531 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7fa8200bb120 con 0x7fa8340c4f10 2024-09-17T04:26:46.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.659+0000 7fa839a00640 1 --2- 172.21.15.100:0/1352583531 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa814078dd0 0x7fa81407b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:46.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.660+0000 7fa839a00640 1 --2- 172.21.15.100:0/1352583531 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa814078dd0 0x7fa81407b2e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa834052860 tx=0x7fa828005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:26:46.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.663+0000 7fa831600640 1 -- 172.21.15.100:0/1352583531 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa8200830a0 con 0x7fa8340c4f10 2024-09-17T04:26:46.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.884+0000 7fa83ae00640 1 -- 172.21.15.100:0/1352583531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa834070a20 con 0x7fa8340c4f10 2024-09-17T04:26:46.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.884+0000 7fa831600640 1 -- 172.21.15.100:0/1352583531 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7fa8200829a0 con 0x7fa8340c4f10 2024-09-17T04:26:46.885 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:26:46.885 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"btime":"2024-09-17T04:25:56:247389+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1086873435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1086873435},{"type":"v1","addr":"172.21.15.100:6833","nonce":1086873435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1708229764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1708229764},{"type":"v1","addr":"172.21.15.184:6836","nonce":1708229764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/828832783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":828832783},{"type":"v1","addr":"172.21.15.184:6837","nonce":828832783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:44.669203+0000","modified":"2024-09-17T04:25:47.693203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3801048032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3801048032},{"type":"v1","addr":"172.21.15.100:6835","nonce":3801048032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5295,"qdb_cluster":[5295]},"id":3}]} 2024-09-17T04:26:46.885 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-09-17T04:26:46.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.887+0000 7fa83ae00640 1 -- 172.21.15.100:0/1352583531 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa814078dd0 msgr2=0x7fa81407b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:46.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.887+0000 7fa83ae00640 1 --2- 172.21.15.100:0/1352583531 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa814078dd0 0x7fa81407b2e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa834052860 tx=0x7fa828005f90 comp rx=0 tx=0).stop 2024-09-17T04:26:46.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.887+0000 7fa83ae00640 1 -- 172.21.15.100:0/1352583531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8340c4f10 msgr2=0x7fa8341f6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:46.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.887+0000 7fa83ae00640 1 --2- 172.21.15.100:0/1352583531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8340c4f10 0x7fa8341f6c50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa82000c310 tx=0x7fa820001d70 comp rx=0 tx=0).stop 2024-09-17T04:26:46.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.887+0000 7fa83ae00640 1 -- 172.21.15.100:0/1352583531 shutdown_connections 2024-09-17T04:26:46.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.887+0000 7fa83ae00640 1 --2- 172.21.15.100:0/1352583531 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa814078dd0 0x7fa81407b2e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:46.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.887+0000 7fa83ae00640 1 --2- 172.21.15.100:0/1352583531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8341f7190 0x7fa83414eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:46.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.887+0000 7fa83ae00640 1 --2- 172.21.15.100:0/1352583531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8340c4f10 0x7fa8341f6c50 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:46.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.887+0000 7fa83ae00640 1 --2- 172.21.15.100:0/1352583531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa81c004800 0x7fa83414bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:46.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.887+0000 7fa83ae00640 1 -- 172.21.15.100:0/1352583531 >> 172.21.15.100:0/1352583531 conn(0x7fa8340b1e40 msgr2=0x7fa8340b14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:26:46.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.887+0000 7fa83ae00640 1 -- 172.21.15.100:0/1352583531 shutdown_connections 2024-09-17T04:26:46.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:46.887+0000 7fa83ae00640 1 -- 172.21.15.100:0/1352583531 wait complete. 2024-09-17T04:26:46.910 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-17T04:26:47.152 INFO:teuthology.orchestra.run.smithi100.stdout:[{"id":5419,"entity":{"name":{"type":"client","num":5419},"addr":{"type":"any","addr":"172.21.15.184:0","nonce":1392734336}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":47.374731664999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.719766206386602,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5419 172.21.15.184:0/1392734336","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"ce1b342bf8c6349a8c90cd607f83bff2443f21cf","ceph_version":"ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)","entity_id":"x","hostname":"smithi184","pid":"35968","root":"/"}}] 2024-09-17T04:26:47.152 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi184", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35968", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.184:0", "nonce": 1392734336, "type": "any" }, "name": { "num": 5419, "type": "client" } }, "id": 5419, "inst": "client.5419 172.21.15.184:0/1392734336", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.719766206386602 }, "state": "open", "uptime": 47.374731665 } ] 2024-09-17T04:26:47.152 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-17T04:26:52.153 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:26:52.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.385+0000 7f91d8800640 1 -- 172.21.15.100:0/346008069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91d015d760 msgr2=0x7f91d01535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:52.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.385+0000 7f91d8800640 1 --2- 172.21.15.100:0/346008069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91d015d760 0x7f91d01535c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f91d0072d60 tx=0x7f91c402a4a0 comp rx=0 tx=0).stop 2024-09-17T04:26:52.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.385+0000 7f91d8800640 1 -- 172.21.15.100:0/346008069 shutdown_connections 2024-09-17T04:26:52.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.385+0000 7f91d8800640 1 --2- 172.21.15.100:0/346008069 >> v2:172.21.15.184:3300/0 conn(0x7f91d00ad0e0 0x7f91d00ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:52.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.385+0000 7f91d8800640 1 --2- 172.21.15.100:0/346008069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91d015d760 0x7f91d01535c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:52.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.385+0000 7f91d8800640 1 -- 172.21.15.100:0/346008069 >> 172.21.15.100:0/346008069 conn(0x7f91d006d730 msgr2=0x7f91d00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:26:52.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.385+0000 7f91d8800640 1 -- 172.21.15.100:0/346008069 shutdown_connections 2024-09-17T04:26:52.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.385+0000 7f91d8800640 1 -- 172.21.15.100:0/346008069 wait complete. 2024-09-17T04:26:52.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.387+0000 7f91d8800640 1 Processor -- start 2024-09-17T04:26:52.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.388+0000 7f91d8800640 1 -- start start 2024-09-17T04:26:52.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.389+0000 7f91d8800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91d00ad0e0 0x7f91d01e8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:52.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.389+0000 7f91d8800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91d015d760 0x7f91d014f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:52.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.389+0000 7f91d8800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91d0160310 0x7f91d0150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:52.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.389+0000 7f91d8800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f91d00b4c20 con 0x7f91d0160310 2024-09-17T04:26:52.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.389+0000 7f91d8800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f91d0153c70 con 0x7f91d00ad0e0 2024-09-17T04:26:52.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.389+0000 7f91d8800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f91d00b4da0 con 0x7f91d015d760 2024-09-17T04:26:52.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.389+0000 7f91d6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91d0160310 0x7f91d0150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:52.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.389+0000 7f91d6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91d0160310 0x7f91d0150d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:32900/0 (socket says 172.21.15.100:32900) 2024-09-17T04:26:52.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.389+0000 7f91d6800640 1 -- 172.21.15.100:0/1140212326 learned_addr learned my addr 172.21.15.100:0/1140212326 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:26:52.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.389+0000 7f91d5e00640 1 --2- 172.21.15.100:0/1140212326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91d00ad0e0 0x7f91d01e8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:52.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.389+0000 7f91d5400640 1 --2- 172.21.15.100:0/1140212326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91d015d760 0x7f91d014f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:52.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.390+0000 7f91d6800640 1 -- 172.21.15.100:0/1140212326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91d015d760 msgr2=0x7f91d014f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:52.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.390+0000 7f91d6800640 1 --2- 172.21.15.100:0/1140212326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91d015d760 0x7f91d014f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:52.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.390+0000 7f91d6800640 1 -- 172.21.15.100:0/1140212326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91d00ad0e0 msgr2=0x7f91d01e8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:52.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.390+0000 7f91d6800640 1 --2- 172.21.15.100:0/1140212326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91d00ad0e0 0x7f91d01e8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:52.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.390+0000 7f91d6800640 1 -- 172.21.15.100:0/1140212326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d01e8f40 con 0x7f91d0160310 2024-09-17T04:26:52.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.390+0000 7f91d5400640 1 --2- 172.21.15.100:0/1140212326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91d015d760 0x7f91d014f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:26:52.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.390+0000 7f91d6800640 1 --2- 172.21.15.100:0/1140212326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91d0160310 0x7f91d0150d70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00caa0 tx=0x7f91cc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:26:52.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.390+0000 7f91ca000640 1 -- 172.21.15.100:0/1140212326 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91cc016020 con 0x7f91d0160310 2024-09-17T04:26:52.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.390+0000 7f91ca000640 1 -- 172.21.15.100:0/1140212326 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91cc00acf0 con 0x7f91d0160310 2024-09-17T04:26:52.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.390+0000 7f91d8800640 1 -- 172.21.15.100:0/1140212326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91d014fa50 con 0x7f91d0160310 2024-09-17T04:26:52.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.390+0000 7f91ca000640 1 -- 172.21.15.100:0/1140212326 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91cc012680 con 0x7f91d0160310 2024-09-17T04:26:52.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.390+0000 7f91d8800640 1 -- 172.21.15.100:0/1140212326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91d0150010 con 0x7f91d0160310 2024-09-17T04:26:52.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.391+0000 7f91d8800640 1 -- 172.21.15.100:0/1140212326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91d0160ff0 con 0x7f91d0160310 2024-09-17T04:26:52.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.392+0000 7f91ca000640 1 -- 172.21.15.100:0/1140212326 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7f91cc019020 con 0x7f91d0160310 2024-09-17T04:26:52.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.393+0000 7f91ca000640 1 --2- 172.21.15.100:0/1140212326 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f91b4078e00 0x7f91b407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:52.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.394+0000 7f91ca000640 1 -- 172.21.15.100:0/1140212326 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f91cc09b7f0 con 0x7f91d0160310 2024-09-17T04:26:52.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.394+0000 7f91d5e00640 1 --2- 172.21.15.100:0/1140212326 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f91b4078e00 0x7f91b407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:52.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.395+0000 7f91d5e00640 1 --2- 172.21.15.100:0/1140212326 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f91b4078e00 0x7f91b407b310 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f91c4034000 tx=0x7f91c4004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:26:52.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.398+0000 7f91ca000640 1 -- 172.21.15.100:0/1140212326 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f91cc020020 con 0x7f91d0160310 2024-09-17T04:26:52.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.616+0000 7f91d8800640 1 -- 172.21.15.100:0/1140212326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91d01607c0 con 0x7f91d0160310 2024-09-17T04:26:52.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.617+0000 7f91ca000640 1 -- 172.21.15.100:0/1140212326 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f91d00ada30 con 0x7f91d0160310 2024-09-17T04:26:52.617 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:26:52.617 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"btime":"2024-09-17T04:25:56:247389+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1086873435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1086873435},{"type":"v1","addr":"172.21.15.100:6833","nonce":1086873435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1708229764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1708229764},{"type":"v1","addr":"172.21.15.184:6836","nonce":1708229764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/828832783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":828832783},{"type":"v1","addr":"172.21.15.184:6837","nonce":828832783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:44.669203+0000","modified":"2024-09-17T04:25:47.693203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3801048032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3801048032},{"type":"v1","addr":"172.21.15.100:6835","nonce":3801048032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5295,"qdb_cluster":[5295]},"id":3}]} 2024-09-17T04:26:52.617 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-09-17T04:26:52.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.619+0000 7f91d8800640 1 -- 172.21.15.100:0/1140212326 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f91b4078e00 msgr2=0x7f91b407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:52.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.619+0000 7f91d8800640 1 --2- 172.21.15.100:0/1140212326 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f91b4078e00 0x7f91b407b310 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f91c4034000 tx=0x7f91c4004730 comp rx=0 tx=0).stop 2024-09-17T04:26:52.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.619+0000 7f91d8800640 1 -- 172.21.15.100:0/1140212326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91d0160310 msgr2=0x7f91d0150d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:52.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.619+0000 7f91d8800640 1 --2- 172.21.15.100:0/1140212326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91d0160310 0x7f91d0150d70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00caa0 tx=0x7f91cc00cf60 comp rx=0 tx=0).stop 2024-09-17T04:26:52.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.619+0000 7f91d8800640 1 -- 172.21.15.100:0/1140212326 shutdown_connections 2024-09-17T04:26:52.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.619+0000 7f91d8800640 1 --2- 172.21.15.100:0/1140212326 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f91b4078e00 0x7f91b407b310 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:52.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.619+0000 7f91d8800640 1 --2- 172.21.15.100:0/1140212326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91d0160310 0x7f91d0150d70 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:52.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.619+0000 7f91d8800640 1 --2- 172.21.15.100:0/1140212326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91d015d760 0x7f91d014f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:52.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.619+0000 7f91d8800640 1 --2- 172.21.15.100:0/1140212326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91d00ad0e0 0x7f91d01e8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:52.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.619+0000 7f91d8800640 1 -- 172.21.15.100:0/1140212326 >> 172.21.15.100:0/1140212326 conn(0x7f91d006d730 msgr2=0x7f91d00b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:26:52.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.619+0000 7f91d8800640 1 -- 172.21.15.100:0/1140212326 shutdown_connections 2024-09-17T04:26:52.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:52.620+0000 7f91d8800640 1 -- 172.21.15.100:0/1140212326 wait complete. 2024-09-17T04:26:52.645 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-17T04:26:52.886 INFO:teuthology.orchestra.run.smithi100.stdout:[{"id":5419,"entity":{"name":{"type":"client","num":5419},"addr":{"type":"any","addr":"172.21.15.184:0","nonce":1392734336}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":53.108578323000003,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.460241653262088,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5419 172.21.15.184:0/1392734336","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"ce1b342bf8c6349a8c90cd607f83bff2443f21cf","ceph_version":"ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)","entity_id":"x","hostname":"smithi184","pid":"35968","root":"/"}}] 2024-09-17T04:26:52.887 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi184", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35968", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.184:0", "nonce": 1392734336, "type": "any" }, "name": { "num": 5419, "type": "client" } }, "id": 5419, "inst": "client.5419 172.21.15.184:0/1392734336", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.460241653262088 }, "state": "open", "uptime": 53.108578323 } ] 2024-09-17T04:26:52.887 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-17T04:26:57.888 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:26:58.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.123+0000 7f2cb7e00640 1 -- 172.21.15.100:0/2274254520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cb00ad0e0 msgr2=0x7f2cb00ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:58.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.123+0000 7f2cb7e00640 1 --2- 172.21.15.100:0/2274254520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cb00ad0e0 0x7f2cb00ad4f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0075250 tx=0x7f2c9c02a4d0 comp rx=0 tx=0).stop 2024-09-17T04:26:58.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.123+0000 7f2cb7e00640 1 -- 172.21.15.100:0/2274254520 shutdown_connections 2024-09-17T04:26:58.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.123+0000 7f2cb7e00640 1 --2- 172.21.15.100:0/2274254520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cb00ad0e0 0x7f2cb00ad4f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:58.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.123+0000 7f2cb7e00640 1 --2- 172.21.15.100:0/2274254520 >> v2:172.21.15.184:3300/0 conn(0x7f2cb015d760 0x7f2cb01535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:58.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.123+0000 7f2cb7e00640 1 -- 172.21.15.100:0/2274254520 >> 172.21.15.100:0/2274254520 conn(0x7f2cb006d730 msgr2=0x7f2cb00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:26:58.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.123+0000 7f2cb7e00640 1 -- 172.21.15.100:0/2274254520 shutdown_connections 2024-09-17T04:26:58.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.123+0000 7f2cb7e00640 1 -- 172.21.15.100:0/2274254520 wait complete. 2024-09-17T04:26:58.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.126+0000 7f2cb7e00640 1 Processor -- start 2024-09-17T04:26:58.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.126+0000 7f2cb7e00640 1 -- start start 2024-09-17T04:26:58.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.127+0000 7f2cb7e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cb00ad0e0 0x7f2cb014a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:58.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.127+0000 7f2cb7e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cb015d760 0x7f2cb0156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:58.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.127+0000 7f2cb7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cb0160310 0x7f2cb00b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:58.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.127+0000 7f2cb7e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb00b4c20 con 0x7f2cb0160310 2024-09-17T04:26:58.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.127+0000 7f2cb7e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb0153c70 con 0x7f2cb015d760 2024-09-17T04:26:58.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.127+0000 7f2cb7e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2cb00b4da0 con 0x7f2cb00ad0e0 2024-09-17T04:26:58.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.128+0000 7f2cb5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cb0160310 0x7f2cb00b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:58.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.128+0000 7f2cb5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cb0160310 0x7f2cb00b0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36788/0 (socket says 172.21.15.100:36788) 2024-09-17T04:26:58.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.128+0000 7f2cb5e00640 1 -- 172.21.15.100:0/2001668104 learned_addr learned my addr 172.21.15.100:0/2001668104 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:26:58.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.128+0000 7f2cb5400640 1 --2- 172.21.15.100:0/2001668104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cb00ad0e0 0x7f2cb014a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:58.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.128+0000 7f2cb4a00640 1 --2- 172.21.15.100:0/2001668104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cb015d760 0x7f2cb0156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:58.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.128+0000 7f2cb5e00640 1 -- 172.21.15.100:0/2001668104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cb00ad0e0 msgr2=0x7f2cb014a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:58.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.128+0000 7f2cb5e00640 1 --2- 172.21.15.100:0/2001668104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cb00ad0e0 0x7f2cb014a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:58.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.128+0000 7f2cb5e00640 1 -- 172.21.15.100:0/2001668104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cb015d760 msgr2=0x7f2cb0156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:58.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.128+0000 7f2cb5e00640 1 --2- 172.21.15.100:0/2001668104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cb015d760 0x7f2cb0156770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:58.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.128+0000 7f2cb5e00640 1 -- 172.21.15.100:0/2001668104 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb01548f0 con 0x7f2cb0160310 2024-09-17T04:26:58.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.129+0000 7f2cb5e00640 1 --2- 172.21.15.100:0/2001668104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cb0160310 0x7f2cb00b0da0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800ba40 tx=0x7f2ca800bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:26:58.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.129+0000 7f2cad600640 1 -- 172.21.15.100:0/2001668104 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca800cb80 con 0x7f2cb0160310 2024-09-17T04:26:58.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.129+0000 7f2cad600640 1 -- 172.21.15.100:0/2001668104 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ca800ce10 con 0x7f2cb0160310 2024-09-17T04:26:58.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.129+0000 7f2cb7e00640 1 -- 172.21.15.100:0/2001668104 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb0154bd0 con 0x7f2cb0160310 2024-09-17T04:26:58.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.129+0000 7f2cad600640 1 -- 172.21.15.100:0/2001668104 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8003bd0 con 0x7f2cb0160310 2024-09-17T04:26:58.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.129+0000 7f2cb7e00640 1 -- 172.21.15.100:0/2001668104 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb0155190 con 0x7f2cb0160310 2024-09-17T04:26:58.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.130+0000 7f2cb7e00640 1 -- 172.21.15.100:0/2001668104 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cb0156ec0 con 0x7f2cb0160310 2024-09-17T04:26:58.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.131+0000 7f2cad600640 1 -- 172.21.15.100:0/2001668104 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7f2ca8014440 con 0x7f2cb0160310 2024-09-17T04:26:58.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.133+0000 7f2cad600640 1 --2- 172.21.15.100:0/2001668104 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2c90078e00 0x7f2c9007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:26:58.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.133+0000 7f2cad600640 1 -- 172.21.15.100:0/2001668104 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f2ca809aff0 con 0x7f2cb0160310 2024-09-17T04:26:58.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.133+0000 7f2cb5400640 1 --2- 172.21.15.100:0/2001668104 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2c90078e00 0x7f2c9007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:26:58.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.134+0000 7f2cb5400640 1 --2- 172.21.15.100:0/2001668104 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2c90078e00 0x7f2c9007b310 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400a490 tx=0x7f2ca400a420 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:26:58.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.138+0000 7f2cad600640 1 -- 172.21.15.100:0/2001668104 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f2ca8062960 con 0x7f2cb0160310 2024-09-17T04:26:58.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.355+0000 7f2cb7e00640 1 -- 172.21.15.100:0/2001668104 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2cb0161010 con 0x7f2cb0160310 2024-09-17T04:26:58.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.356+0000 7f2cad600640 1 -- 172.21.15.100:0/2001668104 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f2ca8067390 con 0x7f2cb0160310 2024-09-17T04:26:58.357 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:26:58.357 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"btime":"2024-09-17T04:25:56:247389+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1086873435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1086873435},{"type":"v1","addr":"172.21.15.100:6833","nonce":1086873435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1708229764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1708229764},{"type":"v1","addr":"172.21.15.184:6836","nonce":1708229764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/828832783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":828832783},{"type":"v1","addr":"172.21.15.184:6837","nonce":828832783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:44.669203+0000","modified":"2024-09-17T04:25:47.693203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3801048032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3801048032},{"type":"v1","addr":"172.21.15.100:6835","nonce":3801048032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5295,"qdb_cluster":[5295]},"id":3}]} 2024-09-17T04:26:58.357 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-09-17T04:26:58.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.359+0000 7f2cb7e00640 1 -- 172.21.15.100:0/2001668104 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2c90078e00 msgr2=0x7f2c9007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:58.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.359+0000 7f2cb7e00640 1 --2- 172.21.15.100:0/2001668104 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2c90078e00 0x7f2c9007b310 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400a490 tx=0x7f2ca400a420 comp rx=0 tx=0).stop 2024-09-17T04:26:58.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.359+0000 7f2cb7e00640 1 -- 172.21.15.100:0/2001668104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cb0160310 msgr2=0x7f2cb00b0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:26:58.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.359+0000 7f2cb7e00640 1 --2- 172.21.15.100:0/2001668104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cb0160310 0x7f2cb00b0da0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800ba40 tx=0x7f2ca800bf00 comp rx=0 tx=0).stop 2024-09-17T04:26:58.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.359+0000 7f2cb7e00640 1 -- 172.21.15.100:0/2001668104 shutdown_connections 2024-09-17T04:26:58.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.359+0000 7f2cb7e00640 1 --2- 172.21.15.100:0/2001668104 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2c90078e00 0x7f2c9007b310 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:58.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.359+0000 7f2cb7e00640 1 --2- 172.21.15.100:0/2001668104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cb0160310 0x7f2cb00b0da0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:58.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.359+0000 7f2cb7e00640 1 --2- 172.21.15.100:0/2001668104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2cb015d760 0x7f2cb0156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:58.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.359+0000 7f2cb7e00640 1 --2- 172.21.15.100:0/2001668104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2cb00ad0e0 0x7f2cb014a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:26:58.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.359+0000 7f2cb7e00640 1 -- 172.21.15.100:0/2001668104 >> 172.21.15.100:0/2001668104 conn(0x7f2cb006d730 msgr2=0x7f2cb00cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:26:58.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.359+0000 7f2cb7e00640 1 -- 172.21.15.100:0/2001668104 shutdown_connections 2024-09-17T04:26:58.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:26:58.359+0000 7f2cb7e00640 1 -- 172.21.15.100:0/2001668104 wait complete. 2024-09-17T04:26:58.382 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-17T04:26:58.622 INFO:teuthology.orchestra.run.smithi100.stdout:[{"id":5419,"entity":{"name":{"type":"client","num":5419},"addr":{"type":"any","addr":"172.21.15.184:0","nonce":1392734336}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":58.844424930000002,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.204043867373475,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5419 172.21.15.184:0/1392734336","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"ce1b342bf8c6349a8c90cd607f83bff2443f21cf","ceph_version":"ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)","entity_id":"x","hostname":"smithi184","pid":"35968","root":"/"}}] 2024-09-17T04:26:58.622 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi184", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35968", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.184:0", "nonce": 1392734336, "type": "any" }, "name": { "num": 5419, "type": "client" } }, "id": 5419, "inst": "client.5419 172.21.15.184:0/1392734336", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.204043867373475 }, "state": "open", "uptime": 58.84442493 } ] 2024-09-17T04:26:58.622 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-17T04:27:03.624 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:27:03.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.854+0000 7fee5cc00640 1 -- 172.21.15.100:0/1177556074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee58071130 msgr2=0x7fee580704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:03.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.854+0000 7fee5cc00640 1 --2- 172.21.15.100:0/1177556074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee58071130 0x7fee580704b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fee48009830 tx=0x7fee4802a520 comp rx=0 tx=0).stop 2024-09-17T04:27:03.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.854+0000 7fee5cc00640 1 -- 172.21.15.100:0/1177556074 shutdown_connections 2024-09-17T04:27:03.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.854+0000 7fee5cc00640 1 --2- 172.21.15.100:0/1177556074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee580ad3c0 0x7fee580ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:03.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.854+0000 7fee5cc00640 1 --2- 172.21.15.100:0/1177556074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee58071130 0x7fee580704b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:03.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.854+0000 7fee5cc00640 1 -- 172.21.15.100:0/1177556074 >> 172.21.15.100:0/1177556074 conn(0x7fee5806d730 msgr2=0x7fee580b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:03.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.854+0000 7fee5cc00640 1 -- 172.21.15.100:0/1177556074 shutdown_connections 2024-09-17T04:27:03.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.854+0000 7fee5cc00640 1 -- 172.21.15.100:0/1177556074 wait complete. 2024-09-17T04:27:03.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.856+0000 7fee5cc00640 1 Processor -- start 2024-09-17T04:27:03.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.856+0000 7fee5cc00640 1 -- start start 2024-09-17T04:27:03.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.857+0000 7fee5cc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee58071130 0x7fee58145b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:03.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.858+0000 7fee5cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee580ad3c0 0x7fee58155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:03.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.858+0000 7fee5cc00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee580add10 0x7fee581555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:03.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.858+0000 7fee5cc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fee58073480 con 0x7fee580ad3c0 2024-09-17T04:27:03.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.858+0000 7fee5cc00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fee58073300 con 0x7fee58071130 2024-09-17T04:27:03.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.858+0000 7fee5cc00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fee58073600 con 0x7fee580add10 2024-09-17T04:27:03.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.858+0000 7fee55400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee580ad3c0 0x7fee58155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:03.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.858+0000 7fee55400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee580ad3c0 0x7fee58155080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36800/0 (socket says 172.21.15.100:36800) 2024-09-17T04:27:03.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.858+0000 7fee55400640 1 -- 172.21.15.100:0/828563297 learned_addr learned my addr 172.21.15.100:0/828563297 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:03.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.858+0000 7fee55e00640 1 --2- 172.21.15.100:0/828563297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee58071130 0x7fee58145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:03.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.858+0000 7fee56800640 1 --2- 172.21.15.100:0/828563297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee580add10 0x7fee581555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:03.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.858+0000 7fee55400640 1 -- 172.21.15.100:0/828563297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee580add10 msgr2=0x7fee581555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:03.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.858+0000 7fee55400640 1 --2- 172.21.15.100:0/828563297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee580add10 0x7fee581555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:03.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.858+0000 7fee55400640 1 -- 172.21.15.100:0/828563297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee58071130 msgr2=0x7fee58145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:03.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.858+0000 7fee55400640 1 --2- 172.21.15.100:0/828563297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee58071130 0x7fee58145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:03.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.858+0000 7fee55400640 1 -- 172.21.15.100:0/828563297 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5815b1d0 con 0x7fee580ad3c0 2024-09-17T04:27:03.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.858+0000 7fee56800640 1 --2- 172.21.15.100:0/828563297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee580add10 0x7fee581555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:27:03.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.859+0000 7fee55400640 1 --2- 172.21.15.100:0/828563297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee580ad3c0 0x7fee58155080 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fee4400ea70 tx=0x7fee4400ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:03.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.859+0000 7fee4e000640 1 -- 172.21.15.100:0/828563297 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4400cd40 con 0x7fee580ad3c0 2024-09-17T04:27:03.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.859+0000 7fee4e000640 1 -- 172.21.15.100:0/828563297 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee44004020 con 0x7fee580ad3c0 2024-09-17T04:27:03.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.859+0000 7fee5cc00640 1 -- 172.21.15.100:0/828563297 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee58155b00 con 0x7fee580ad3c0 2024-09-17T04:27:03.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.859+0000 7fee4e000640 1 -- 172.21.15.100:0/828563297 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee44004470 con 0x7fee580ad3c0 2024-09-17T04:27:03.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.859+0000 7fee5cc00640 1 -- 172.21.15.100:0/828563297 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee5815d850 con 0x7fee580ad3c0 2024-09-17T04:27:03.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.860+0000 7fee5cc00640 1 -- 172.21.15.100:0/828563297 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee580ad7d0 con 0x7fee580ad3c0 2024-09-17T04:27:03.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.861+0000 7fee4e000640 1 -- 172.21.15.100:0/828563297 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7fee440035f0 con 0x7fee580ad3c0 2024-09-17T04:27:03.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.862+0000 7fee4e000640 1 --2- 172.21.15.100:0/828563297 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fee38078e00 0x7fee3807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:03.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.862+0000 7fee4e000640 1 -- 172.21.15.100:0/828563297 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7fee4409b840 con 0x7fee580ad3c0 2024-09-17T04:27:03.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.863+0000 7fee55e00640 1 --2- 172.21.15.100:0/828563297 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fee38078e00 0x7fee3807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:03.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.863+0000 7fee55e00640 1 --2- 172.21.15.100:0/828563297 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fee38078e00 0x7fee3807b310 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fee48034000 tx=0x7fee48004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:03.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:03.867+0000 7fee4e000640 1 -- 172.21.15.100:0/828563297 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fee440637c0 con 0x7fee580ad3c0 2024-09-17T04:27:04.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:04.086+0000 7fee5cc00640 1 -- 172.21.15.100:0/828563297 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee58069a10 con 0x7fee580ad3c0 2024-09-17T04:27:04.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:04.087+0000 7fee4e000640 1 -- 172.21.15.100:0/828563297 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7fee440630c0 con 0x7fee580ad3c0 2024-09-17T04:27:04.087 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:27:04.088 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"btime":"2024-09-17T04:25:56:247389+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1086873435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1086873435},{"type":"v1","addr":"172.21.15.100:6833","nonce":1086873435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1708229764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1708229764},{"type":"v1","addr":"172.21.15.184:6836","nonce":1708229764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/828832783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":828832783},{"type":"v1","addr":"172.21.15.184:6837","nonce":828832783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:44.669203+0000","modified":"2024-09-17T04:25:47.693203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3801048032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3801048032},{"type":"v1","addr":"172.21.15.100:6835","nonce":3801048032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5295,"qdb_cluster":[5295]},"id":3}]} 2024-09-17T04:27:04.088 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-09-17T04:27:04.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:04.089+0000 7fee5cc00640 1 -- 172.21.15.100:0/828563297 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fee38078e00 msgr2=0x7fee3807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:04.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:04.089+0000 7fee5cc00640 1 --2- 172.21.15.100:0/828563297 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fee38078e00 0x7fee3807b310 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fee48034000 tx=0x7fee48004730 comp rx=0 tx=0).stop 2024-09-17T04:27:04.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:04.089+0000 7fee5cc00640 1 -- 172.21.15.100:0/828563297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee580ad3c0 msgr2=0x7fee58155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:04.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:04.089+0000 7fee5cc00640 1 --2- 172.21.15.100:0/828563297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee580ad3c0 0x7fee58155080 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fee4400ea70 tx=0x7fee4400ef30 comp rx=0 tx=0).stop 2024-09-17T04:27:04.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:04.089+0000 7fee5cc00640 1 -- 172.21.15.100:0/828563297 shutdown_connections 2024-09-17T04:27:04.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:04.089+0000 7fee5cc00640 1 --2- 172.21.15.100:0/828563297 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fee38078e00 0x7fee3807b310 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:04.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:04.089+0000 7fee5cc00640 1 --2- 172.21.15.100:0/828563297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee580add10 0x7fee581555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:04.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:04.089+0000 7fee5cc00640 1 --2- 172.21.15.100:0/828563297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee580ad3c0 0x7fee58155080 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:04.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:04.089+0000 7fee5cc00640 1 --2- 172.21.15.100:0/828563297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee58071130 0x7fee58145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:04.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:04.089+0000 7fee5cc00640 1 -- 172.21.15.100:0/828563297 >> 172.21.15.100:0/828563297 conn(0x7fee5806d730 msgr2=0x7fee581575d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:04.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:04.090+0000 7fee5cc00640 1 -- 172.21.15.100:0/828563297 shutdown_connections 2024-09-17T04:27:04.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:04.090+0000 7fee5cc00640 1 -- 172.21.15.100:0/828563297 wait complete. 2024-09-17T04:27:04.115 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-17T04:27:04.357 INFO:teuthology.orchestra.run.smithi100.stdout:[{"id":5419,"entity":{"name":{"type":"client","num":5419},"addr":{"type":"any","addr":"172.21.15.184:0","nonce":1392734336}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":64.580271533000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.951218974281982,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5419 172.21.15.184:0/1392734336","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"ce1b342bf8c6349a8c90cd607f83bff2443f21cf","ceph_version":"ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)","entity_id":"x","hostname":"smithi184","pid":"35968","root":"/"}}] 2024-09-17T04:27:04.358 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi184", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35968", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.184:0", "nonce": 1392734336, "type": "any" }, "name": { "num": 5419, "type": "client" } }, "id": 5419, "inst": "client.5419 172.21.15.184:0/1392734336", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.951218974281982 }, "state": "open", "uptime": 64.580271533 } ] 2024-09-17T04:27:04.358 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-17T04:27:04.991 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:27:04.996 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:27:09.359 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:27:09.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.588+0000 7f0d47e00640 1 -- 172.21.15.100:0/922028475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d4015d760 msgr2=0x7f0d401535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:09.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.588+0000 7f0d47e00640 1 --2- 172.21.15.100:0/922028475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d4015d760 0x7f0d401535c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0d34009830 tx=0x7f0d3402a480 comp rx=0 tx=0).stop 2024-09-17T04:27:09.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.588+0000 7f0d47e00640 1 -- 172.21.15.100:0/922028475 shutdown_connections 2024-09-17T04:27:09.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.588+0000 7f0d47e00640 1 --2- 172.21.15.100:0/922028475 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d400b3550 0x7f0d400b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:09.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.588+0000 7f0d47e00640 1 --2- 172.21.15.100:0/922028475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d4015d760 0x7f0d401535c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:09.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.588+0000 7f0d47e00640 1 -- 172.21.15.100:0/922028475 >> 172.21.15.100:0/922028475 conn(0x7f0d4006d730 msgr2=0x7f0d400b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:09.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.588+0000 7f0d47e00640 1 -- 172.21.15.100:0/922028475 shutdown_connections 2024-09-17T04:27:09.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.588+0000 7f0d47e00640 1 -- 172.21.15.100:0/922028475 wait complete. 2024-09-17T04:27:09.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.590+0000 7f0d47e00640 1 Processor -- start 2024-09-17T04:27:09.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.591+0000 7f0d47e00640 1 -- start start 2024-09-17T04:27:09.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.592+0000 7f0d47e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d400ad0e0 0x7f0d40148f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:09.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.592+0000 7f0d47e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d400b3550 0x7f0d401511f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:09.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.592+0000 7f0d47e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d4015d760 0x7f0d40151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:09.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.592+0000 7f0d47e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4006f240 con 0x7f0d400b3550 2024-09-17T04:27:09.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.592+0000 7f0d47e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0d400b4e30 con 0x7f0d4015d760 2024-09-17T04:27:09.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.592+0000 7f0d47e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0d400b3ea0 con 0x7f0d400ad0e0 2024-09-17T04:27:09.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.592+0000 7f0d44a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d400b3550 0x7f0d401511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:09.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.592+0000 7f0d44a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d400b3550 0x7f0d401511f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59000/0 (socket says 172.21.15.100:59000) 2024-09-17T04:27:09.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.592+0000 7f0d44a00640 1 -- 172.21.15.100:0/243595425 learned_addr learned my addr 172.21.15.100:0/243595425 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:09.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.593+0000 7f0d45400640 1 --2- 172.21.15.100:0/243595425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d400ad0e0 0x7f0d40148f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:09.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.593+0000 7f0d45e00640 1 --2- 172.21.15.100:0/243595425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d4015d760 0x7f0d40151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:09.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.593+0000 7f0d44a00640 1 -- 172.21.15.100:0/243595425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d400ad0e0 msgr2=0x7f0d40148f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:09.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.593+0000 7f0d44a00640 1 --2- 172.21.15.100:0/243595425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d400ad0e0 0x7f0d40148f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:09.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.593+0000 7f0d44a00640 1 -- 172.21.15.100:0/243595425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d4015d760 msgr2=0x7f0d40151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:09.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.593+0000 7f0d44a00640 1 --2- 172.21.15.100:0/243595425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d4015d760 0x7f0d40151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:09.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.593+0000 7f0d44a00640 1 -- 172.21.15.100:0/243595425 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d401f5c50 con 0x7f0d400b3550 2024-09-17T04:27:09.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.593+0000 7f0d45400640 1 --2- 172.21.15.100:0/243595425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d400ad0e0 0x7f0d40148f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:27:09.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.593+0000 7f0d44a00640 1 --2- 172.21.15.100:0/243595425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d400b3550 0x7f0d401511f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00c3c0 tx=0x7f0d2c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:09.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.593+0000 7f0d3d600640 1 -- 172.21.15.100:0/243595425 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c017070 con 0x7f0d400b3550 2024-09-17T04:27:09.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.593+0000 7f0d3d600640 1 -- 172.21.15.100:0/243595425 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d2c0035f0 con 0x7f0d400b3550 2024-09-17T04:27:09.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.593+0000 7f0d3d600640 1 -- 172.21.15.100:0/243595425 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c003a20 con 0x7f0d400b3550 2024-09-17T04:27:09.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.593+0000 7f0d47e00640 1 -- 172.21.15.100:0/243595425 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d401f5f30 con 0x7f0d400b3550 2024-09-17T04:27:09.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.594+0000 7f0d47e00640 1 -- 172.21.15.100:0/243595425 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d401f64f0 con 0x7f0d400b3550 2024-09-17T04:27:09.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.595+0000 7f0d47e00640 1 -- 172.21.15.100:0/243595425 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d400b4350 con 0x7f0d400b3550 2024-09-17T04:27:09.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.596+0000 7f0d3d600640 1 -- 172.21.15.100:0/243595425 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7f0d2c01a070 con 0x7f0d400b3550 2024-09-17T04:27:09.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.597+0000 7f0d3d600640 1 --2- 172.21.15.100:0/243595425 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0d20078e00 0x7f0d2007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:09.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.597+0000 7f0d3d600640 1 -- 172.21.15.100:0/243595425 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f0d2c09a620 con 0x7f0d400b3550 2024-09-17T04:27:09.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.597+0000 7f0d45400640 1 --2- 172.21.15.100:0/243595425 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0d20078e00 0x7f0d2007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:09.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.598+0000 7f0d45400640 1 --2- 172.21.15.100:0/243595425 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0d20078e00 0x7f0d2007b310 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0d34034000 tx=0x7f0d34004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:09.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.601+0000 7f0d3d600640 1 -- 172.21.15.100:0/243595425 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0d2c062510 con 0x7f0d400b3550 2024-09-17T04:27:09.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.818+0000 7f0d47e00640 1 -- 172.21.15.100:0/243595425 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d40152f50 con 0x7f0d400b3550 2024-09-17T04:27:09.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.819+0000 7f0d3d600640 1 -- 172.21.15.100:0/243595425 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f0d2c061e10 con 0x7f0d400b3550 2024-09-17T04:27:09.820 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:27:09.820 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"btime":"2024-09-17T04:25:56:247389+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1086873435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1086873435},{"type":"v1","addr":"172.21.15.100:6833","nonce":1086873435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1708229764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1708229764},{"type":"v1","addr":"172.21.15.184:6836","nonce":1708229764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/828832783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":828832783},{"type":"v1","addr":"172.21.15.184:6837","nonce":828832783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:44.669203+0000","modified":"2024-09-17T04:25:47.693203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3801048032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3801048032},{"type":"v1","addr":"172.21.15.100:6835","nonce":3801048032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5295,"qdb_cluster":[5295]},"id":3}]} 2024-09-17T04:27:09.820 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-09-17T04:27:09.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.821+0000 7f0d47e00640 1 -- 172.21.15.100:0/243595425 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0d20078e00 msgr2=0x7f0d2007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:09.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.821+0000 7f0d47e00640 1 --2- 172.21.15.100:0/243595425 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0d20078e00 0x7f0d2007b310 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0d34034000 tx=0x7f0d34004730 comp rx=0 tx=0).stop 2024-09-17T04:27:09.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.821+0000 7f0d47e00640 1 -- 172.21.15.100:0/243595425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d400b3550 msgr2=0x7f0d401511f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:09.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.821+0000 7f0d47e00640 1 --2- 172.21.15.100:0/243595425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d400b3550 0x7f0d401511f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00c3c0 tx=0x7f0d2c00c880 comp rx=0 tx=0).stop 2024-09-17T04:27:09.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.821+0000 7f0d47e00640 1 -- 172.21.15.100:0/243595425 shutdown_connections 2024-09-17T04:27:09.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.822+0000 7f0d47e00640 1 --2- 172.21.15.100:0/243595425 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0d20078e00 0x7f0d2007b310 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:09.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.822+0000 7f0d47e00640 1 --2- 172.21.15.100:0/243595425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d4015d760 0x7f0d40151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:09.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.822+0000 7f0d47e00640 1 --2- 172.21.15.100:0/243595425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d400b3550 0x7f0d401511f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:09.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.822+0000 7f0d47e00640 1 --2- 172.21.15.100:0/243595425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d400ad0e0 0x7f0d40148f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:09.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.822+0000 7f0d47e00640 1 -- 172.21.15.100:0/243595425 >> 172.21.15.100:0/243595425 conn(0x7f0d4006d730 msgr2=0x7f0d4015b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:09.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.822+0000 7f0d47e00640 1 -- 172.21.15.100:0/243595425 shutdown_connections 2024-09-17T04:27:09.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:09.822+0000 7f0d47e00640 1 -- 172.21.15.100:0/243595425 wait complete. 2024-09-17T04:27:09.845 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-17T04:27:10.088 INFO:teuthology.orchestra.run.smithi100.stdout:[{"id":5419,"entity":{"name":{"type":"client","num":5419},"addr":{"type":"any","addr":"172.21.15.184:0","nonce":1392734336}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":6,"uptime":70.310118298000006,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.701981824756754,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5419 172.21.15.184:0/1392734336","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"ce1b342bf8c6349a8c90cd607f83bff2443f21cf","ceph_version":"ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)","entity_id":"x","hostname":"smithi184","pid":"35968","root":"/"}}] 2024-09-17T04:27:10.089 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi184", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35968", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.184:0", "nonce": 1392734336, "type": "any" }, "name": { "num": 5419, "type": "client" } }, "id": 5419, "inst": "client.5419 172.21.15.184:0/1392734336", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.701981824756754 }, "state": "open", "uptime": 70.310118298 } ] 2024-09-17T04:27:10.089 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-17T04:27:15.090 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:27:15.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.322+0000 7f5c7f000640 1 -- 172.21.15.100:0/3801512613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c60003c30 msgr2=0x7f5c60004080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:15.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.322+0000 7f5c7f000640 1 --2- 172.21.15.100:0/3801512613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c60003c30 0x7f5c60004080 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c006ef0 tx=0x7f5c60005830 comp rx=0 tx=0).stop 2024-09-17T04:27:15.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.322+0000 7f5c7f000640 1 -- 172.21.15.100:0/3801512613 shutdown_connections 2024-09-17T04:27:15.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.322+0000 7f5c7f000640 1 --2- 172.21.15.100:0/3801512613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c60003c30 0x7f5c60004080 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:15.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.322+0000 7f5c7f000640 1 -- 172.21.15.100:0/3801512613 >> 172.21.15.100:0/3801512613 conn(0x7f5c780b1e40 msgr2=0x7f5c780b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:15.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.322+0000 7f5c7f000640 1 -- 172.21.15.100:0/3801512613 shutdown_connections 2024-09-17T04:27:15.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.322+0000 7f5c7f000640 1 -- 172.21.15.100:0/3801512613 wait complete. 2024-09-17T04:27:15.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.324+0000 7f5c7f000640 1 Processor -- start 2024-09-17T04:27:15.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.324+0000 7f5c7f000640 1 -- start start 2024-09-17T04:27:15.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.326+0000 7f5c7f000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c60003c30 0x7f5c781f9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:15.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.326+0000 7f5c7f000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c781f5700 0x7f5c7814c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:15.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.326+0000 7f5c7f000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c7815bb60 0x7f5c7815bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:15.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.326+0000 7f5c7f000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5c78072e60 con 0x7f5c7815bb60 2024-09-17T04:27:15.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.326+0000 7f5c7f000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5c78070260 con 0x7f5c60003c30 2024-09-17T04:27:15.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.326+0000 7f5c7f000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5c78072fe0 con 0x7f5c781f5700 2024-09-17T04:27:15.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.326+0000 7f5c7e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c7815bb60 0x7f5c7815bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:15.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.326+0000 7f5c7e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c7815bb60 0x7f5c7815bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51410/0 (socket says 172.21.15.100:51410) 2024-09-17T04:27:15.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.326+0000 7f5c7e600640 1 -- 172.21.15.100:0/1614749591 learned_addr learned my addr 172.21.15.100:0/1614749591 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:15.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.326+0000 7f5c7dc00640 1 --2- 172.21.15.100:0/1614749591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c60003c30 0x7f5c781f9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:15.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.326+0000 7f5c7d200640 1 --2- 172.21.15.100:0/1614749591 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c781f5700 0x7f5c7814c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:15.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.326+0000 7f5c7e600640 1 -- 172.21.15.100:0/1614749591 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c781f5700 msgr2=0x7f5c7814c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:15.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.326+0000 7f5c7e600640 1 --2- 172.21.15.100:0/1614749591 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c781f5700 0x7f5c7814c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:15.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.326+0000 7f5c7e600640 1 -- 172.21.15.100:0/1614749591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c60003c30 msgr2=0x7f5c781f9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:15.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.326+0000 7f5c7e600640 1 --2- 172.21.15.100:0/1614749591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c60003c30 0x7f5c781f9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:15.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.326+0000 7f5c7e600640 1 -- 172.21.15.100:0/1614749591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c781f9c90 con 0x7f5c7815bb60 2024-09-17T04:27:15.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.327+0000 7f5c7e600640 1 --2- 172.21.15.100:0/1614749591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c7815bb60 0x7f5c7815bf70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5c780d1160 tx=0x7f5c70006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:15.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.327+0000 7f5c75600640 1 -- 172.21.15.100:0/1614749591 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c70001b50 con 0x7f5c7815bb60 2024-09-17T04:27:15.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.327+0000 7f5c75600640 1 -- 172.21.15.100:0/1614749591 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c70006850 con 0x7f5c7815bb60 2024-09-17T04:27:15.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.327+0000 7f5c7f000640 1 -- 172.21.15.100:0/1614749591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c7815c4b0 con 0x7f5c7815bb60 2024-09-17T04:27:15.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.327+0000 7f5c75600640 1 -- 172.21.15.100:0/1614749591 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c70011430 con 0x7f5c7815bb60 2024-09-17T04:27:15.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.327+0000 7f5c7f000640 1 -- 172.21.15.100:0/1614749591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c781f6060 con 0x7f5c7815bb60 2024-09-17T04:27:15.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.328+0000 7f5c7f000640 1 -- 172.21.15.100:0/1614749591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c78073540 con 0x7f5c7815bb60 2024-09-17T04:27:15.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.330+0000 7f5c75600640 1 -- 172.21.15.100:0/1614749591 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7f5c700115d0 con 0x7f5c7815bb60 2024-09-17T04:27:15.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.330+0000 7f5c75600640 1 --2- 172.21.15.100:0/1614749591 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5c58078dd0 0x7f5c5807b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:15.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.331+0000 7f5c75600640 1 -- 172.21.15.100:0/1614749591 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f5c7009b9e0 con 0x7f5c7815bb60 2024-09-17T04:27:15.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.331+0000 7f5c7dc00640 1 --2- 172.21.15.100:0/1614749591 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5c58078dd0 0x7f5c5807b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:15.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.332+0000 7f5c7dc00640 1 --2- 172.21.15.100:0/1614749591 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5c58078dd0 0x7f5c5807b2e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c033000 tx=0x7f5c6c001940 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:15.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.335+0000 7f5c75600640 1 -- 172.21.15.100:0/1614749591 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f5c781f9c90 con 0x7f5c7815bb60 2024-09-17T04:27:15.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.553+0000 7f5c7f000640 1 -- 172.21.15.100:0/1614749591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c7815ce90 con 0x7f5c7815bb60 2024-09-17T04:27:15.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.554+0000 7f5c75600640 1 -- 172.21.15.100:0/1614749591 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f5c70063960 con 0x7f5c7815bb60 2024-09-17T04:27:15.554 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:27:15.554 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"btime":"2024-09-17T04:25:56:247389+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1086873435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1086873435},{"type":"v1","addr":"172.21.15.100:6833","nonce":1086873435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1708229764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1708229764},{"type":"v1","addr":"172.21.15.184:6836","nonce":1708229764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/828832783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":828832783},{"type":"v1","addr":"172.21.15.184:6837","nonce":828832783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:44.669203+0000","modified":"2024-09-17T04:25:47.693203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3801048032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3801048032},{"type":"v1","addr":"172.21.15.100:6835","nonce":3801048032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5295,"qdb_cluster":[5295]},"id":3}]} 2024-09-17T04:27:15.554 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-09-17T04:27:15.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.556+0000 7f5c7f000640 1 -- 172.21.15.100:0/1614749591 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5c58078dd0 msgr2=0x7f5c5807b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:15.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.556+0000 7f5c7f000640 1 --2- 172.21.15.100:0/1614749591 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5c58078dd0 0x7f5c5807b2e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c033000 tx=0x7f5c6c001940 comp rx=0 tx=0).stop 2024-09-17T04:27:15.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.556+0000 7f5c7f000640 1 -- 172.21.15.100:0/1614749591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c7815bb60 msgr2=0x7f5c7815bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:15.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.556+0000 7f5c7f000640 1 --2- 172.21.15.100:0/1614749591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c7815bb60 0x7f5c7815bf70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5c780d1160 tx=0x7f5c70006660 comp rx=0 tx=0).stop 2024-09-17T04:27:15.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.557+0000 7f5c7f000640 1 -- 172.21.15.100:0/1614749591 shutdown_connections 2024-09-17T04:27:15.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.557+0000 7f5c7f000640 1 --2- 172.21.15.100:0/1614749591 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5c58078dd0 0x7f5c5807b2e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:15.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.557+0000 7f5c7f000640 1 --2- 172.21.15.100:0/1614749591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c7815bb60 0x7f5c7815bf70 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:15.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.557+0000 7f5c7f000640 1 --2- 172.21.15.100:0/1614749591 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c781f5700 0x7f5c7814c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:15.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.557+0000 7f5c7f000640 1 --2- 172.21.15.100:0/1614749591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c60003c30 0x7f5c781f9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:15.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.557+0000 7f5c7f000640 1 -- 172.21.15.100:0/1614749591 >> 172.21.15.100:0/1614749591 conn(0x7f5c780b1e40 msgr2=0x7f5c781e7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:15.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.557+0000 7f5c7f000640 1 -- 172.21.15.100:0/1614749591 shutdown_connections 2024-09-17T04:27:15.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:15.557+0000 7f5c7f000640 1 -- 172.21.15.100:0/1614749591 wait complete. 2024-09-17T04:27:15.580 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-17T04:27:15.821 INFO:teuthology.orchestra.run.smithi100.stdout:[{"id":5419,"entity":{"name":{"type":"client","num":5419},"addr":{"type":"any","addr":"172.21.15.184:0","nonce":1392734336}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":6,"uptime":76.043964958999993,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.45585195550083,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5419 172.21.15.184:0/1392734336","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"ce1b342bf8c6349a8c90cd607f83bff2443f21cf","ceph_version":"ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)","entity_id":"x","hostname":"smithi184","pid":"35968","root":"/"}}] 2024-09-17T04:27:15.822 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi184", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35968", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.184:0", "nonce": 1392734336, "type": "any" }, "name": { "num": 5419, "type": "client" } }, "id": 5419, "inst": "client.5419 172.21.15.184:0/1392734336", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.45585195550083 }, "state": "open", "uptime": 76.043964959 } ] 2024-09-17T04:27:15.822 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-17T04:27:20.823 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:27:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.053+0000 7f6ea4a00640 1 -- 172.21.15.100:0/4229233024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ea00ad9c0 msgr2=0x7f6ea00af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.053+0000 7f6ea4a00640 1 --2- 172.21.15.100:0/4229233024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ea00ad9c0 0x7f6ea00af170 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6e90005fd0 tx=0x7f6e9002a7a0 comp rx=0 tx=0).stop 2024-09-17T04:27:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.053+0000 7f6ea4a00640 1 -- 172.21.15.100:0/4229233024 shutdown_connections 2024-09-17T04:27:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.053+0000 7f6ea4a00640 1 --2- 172.21.15.100:0/4229233024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ea00b2f60 0x7f6ea015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.053+0000 7f6ea4a00640 1 --2- 172.21.15.100:0/4229233024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ea00ad9c0 0x7f6ea00af170 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.053+0000 7f6ea4a00640 1 -- 172.21.15.100:0/4229233024 >> 172.21.15.100:0/4229233024 conn(0x7f6ea00b1e40 msgr2=0x7f6ea00b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.053+0000 7f6ea4a00640 1 -- 172.21.15.100:0/4229233024 shutdown_connections 2024-09-17T04:27:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.053+0000 7f6ea4a00640 1 -- 172.21.15.100:0/4229233024 wait complete. 2024-09-17T04:27:21.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.056+0000 7f6ea4a00640 1 Processor -- start 2024-09-17T04:27:21.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.056+0000 7f6ea4a00640 1 -- start start 2024-09-17T04:27:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.057+0000 7f6ea4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea00ad9c0 0x7f6ea0074a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.057+0000 7f6ea4a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ea00b2f60 0x7f6ea014abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.057+0000 7f6ea4a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ea015d670 0x7f6ea00c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.057+0000 7f6ea4a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea006f410 con 0x7f6ea00ad9c0 2024-09-17T04:27:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.057+0000 7f6ea4a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea006f290 con 0x7f6ea00b2f60 2024-09-17T04:27:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.057+0000 7f6ea4a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6ea006f590 con 0x7f6ea015d670 2024-09-17T04:27:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.057+0000 7f6e9e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea00ad9c0 0x7f6ea0074a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.058+0000 7f6e9e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea00ad9c0 0x7f6ea0074a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51414/0 (socket says 172.21.15.100:51414) 2024-09-17T04:27:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.058+0000 7f6e9e800640 1 -- 172.21.15.100:0/1487244358 learned_addr learned my addr 172.21.15.100:0/1487244358 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.058+0000 7f6e95e00640 1 --2- 172.21.15.100:0/1487244358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ea00b2f60 0x7f6ea014abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.058+0000 7f6e9f200640 1 --2- 172.21.15.100:0/1487244358 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ea015d670 0x7f6ea00c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.058+0000 7f6e9e800640 1 -- 172.21.15.100:0/1487244358 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ea015d670 msgr2=0x7f6ea00c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.058+0000 7f6e9e800640 1 --2- 172.21.15.100:0/1487244358 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ea015d670 0x7f6ea00c9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.058+0000 7f6e9e800640 1 -- 172.21.15.100:0/1487244358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ea00b2f60 msgr2=0x7f6ea014abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.058+0000 7f6e9e800640 1 --2- 172.21.15.100:0/1487244358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ea00b2f60 0x7f6ea014abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.058+0000 7f6e9e800640 1 -- 172.21.15.100:0/1487244358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea014b0f0 con 0x7f6ea00ad9c0 2024-09-17T04:27:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.058+0000 7f6e95e00640 1 --2- 172.21.15.100:0/1487244358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ea00b2f60 0x7f6ea014abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:27:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.058+0000 7f6e9e800640 1 --2- 172.21.15.100:0/1487244358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea00ad9c0 0x7f6ea0074a90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6e90034040 tx=0x7f6e90032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.059+0000 7f6e97400640 1 -- 172.21.15.100:0/1487244358 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e90041020 con 0x7f6ea00ad9c0 2024-09-17T04:27:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.059+0000 7f6e97400640 1 -- 172.21.15.100:0/1487244358 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e90032e50 con 0x7f6ea00ad9c0 2024-09-17T04:27:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.059+0000 7f6e97400640 1 -- 172.21.15.100:0/1487244358 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e90036040 con 0x7f6ea00ad9c0 2024-09-17T04:27:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.059+0000 7f6ea4a00640 1 -- 172.21.15.100:0/1487244358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea01f6590 con 0x7f6ea00ad9c0 2024-09-17T04:27:21.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.059+0000 7f6ea4a00640 1 -- 172.21.15.100:0/1487244358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea0073fb0 con 0x7f6ea00ad9c0 2024-09-17T04:27:21.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.060+0000 7f6ea4a00640 1 -- 172.21.15.100:0/1487244358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ea006fa60 con 0x7f6ea00ad9c0 2024-09-17T04:27:21.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.061+0000 7f6e97400640 1 -- 172.21.15.100:0/1487244358 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7f6e90044020 con 0x7f6ea00ad9c0 2024-09-17T04:27:21.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.062+0000 7f6e97400640 1 --2- 172.21.15.100:0/1487244358 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6e7c078e00 0x7f6e7c07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:21.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.063+0000 7f6e97400640 1 -- 172.21.15.100:0/1487244358 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f6e900bc060 con 0x7f6ea00ad9c0 2024-09-17T04:27:21.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.063+0000 7f6e95e00640 1 --2- 172.21.15.100:0/1487244358 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6e7c078e00 0x7f6e7c07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:21.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.064+0000 7f6e95e00640 1 --2- 172.21.15.100:0/1487244358 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6e7c078e00 0x7f6e7c07b310 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6ea015bae0 tx=0x7f6e88005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:21.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.067+0000 7f6e97400640 1 -- 172.21.15.100:0/1487244358 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6e9004b070 con 0x7f6ea00ad9c0 2024-09-17T04:27:21.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.289+0000 7f6ea4a00640 1 -- 172.21.15.100:0/1487244358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ea00b43e0 con 0x7f6ea00ad9c0 2024-09-17T04:27:21.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.290+0000 7f6e97400640 1 -- 172.21.15.100:0/1487244358 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f6ea014b0f0 con 0x7f6ea00ad9c0 2024-09-17T04:27:21.289 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:27:21.290 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"btime":"2024-09-17T04:25:56:247389+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1086873435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1086873435},{"type":"v1","addr":"172.21.15.100:6833","nonce":1086873435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1708229764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1708229764},{"type":"v1","addr":"172.21.15.184:6836","nonce":1708229764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/828832783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":828832783},{"type":"v1","addr":"172.21.15.184:6837","nonce":828832783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:44.669203+0000","modified":"2024-09-17T04:25:47.693203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3801048032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3801048032},{"type":"v1","addr":"172.21.15.100:6835","nonce":3801048032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5295,"qdb_cluster":[5295]},"id":3}]} 2024-09-17T04:27:21.290 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-09-17T04:27:21.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.292+0000 7f6ea4a00640 1 -- 172.21.15.100:0/1487244358 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6e7c078e00 msgr2=0x7f6e7c07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:21.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.292+0000 7f6ea4a00640 1 --2- 172.21.15.100:0/1487244358 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6e7c078e00 0x7f6e7c07b310 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6ea015bae0 tx=0x7f6e88005f90 comp rx=0 tx=0).stop 2024-09-17T04:27:21.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.292+0000 7f6ea4a00640 1 -- 172.21.15.100:0/1487244358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea00ad9c0 msgr2=0x7f6ea0074a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:21.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.292+0000 7f6ea4a00640 1 --2- 172.21.15.100:0/1487244358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea00ad9c0 0x7f6ea0074a90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6e90034040 tx=0x7f6e90032c60 comp rx=0 tx=0).stop 2024-09-17T04:27:21.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.292+0000 7f6ea4a00640 1 -- 172.21.15.100:0/1487244358 shutdown_connections 2024-09-17T04:27:21.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.292+0000 7f6ea4a00640 1 --2- 172.21.15.100:0/1487244358 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6e7c078e00 0x7f6e7c07b310 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:21.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.292+0000 7f6ea4a00640 1 --2- 172.21.15.100:0/1487244358 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ea015d670 0x7f6ea00c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:21.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.292+0000 7f6ea4a00640 1 --2- 172.21.15.100:0/1487244358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ea00b2f60 0x7f6ea014abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:21.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.292+0000 7f6ea4a00640 1 --2- 172.21.15.100:0/1487244358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea00ad9c0 0x7f6ea0074a90 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:21.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.292+0000 7f6ea4a00640 1 -- 172.21.15.100:0/1487244358 >> 172.21.15.100:0/1487244358 conn(0x7f6ea00b1e40 msgr2=0x7f6ea00b4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:21.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.292+0000 7f6ea4a00640 1 -- 172.21.15.100:0/1487244358 shutdown_connections 2024-09-17T04:27:21.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:21.293+0000 7f6ea4a00640 1 -- 172.21.15.100:0/1487244358 wait complete. 2024-09-17T04:27:21.314 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-17T04:27:21.555 INFO:teuthology.orchestra.run.smithi100.stdout:[{"id":5419,"entity":{"name":{"type":"client","num":5419},"addr":{"type":"any","addr":"172.21.15.184:0","nonce":1392734336}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":5,"uptime":81.778811591999997,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.212919230818574,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5419 172.21.15.184:0/1392734336","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"ce1b342bf8c6349a8c90cd607f83bff2443f21cf","ceph_version":"ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)","entity_id":"x","hostname":"smithi184","pid":"35968","root":"/"}}] 2024-09-17T04:27:21.556 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi184", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35968", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.184:0", "nonce": 1392734336, "type": "any" }, "name": { "num": 5419, "type": "client" } }, "id": 5419, "inst": "client.5419 172.21.15.184:0/1392734336", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 5, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.212919230818574 }, "state": "open", "uptime": 81.778811592 } ] 2024-09-17T04:27:21.556 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-17T04:27:26.558 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:27:26.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.788+0000 7fc41d000640 1 -- 172.21.15.100:0/493186890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4180ad0e0 msgr2=0x7fc4180ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:26.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.788+0000 7fc41d000640 1 --2- 172.21.15.100:0/493186890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4180ad0e0 0x7fc4180ad4f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc418072d90 tx=0x7fc40002a4d0 comp rx=0 tx=0).stop 2024-09-17T04:27:26.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.788+0000 7fc41d000640 1 -- 172.21.15.100:0/493186890 shutdown_connections 2024-09-17T04:27:26.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.788+0000 7fc41d000640 1 --2- 172.21.15.100:0/493186890 >> v2:172.21.15.184:3300/0 conn(0x7fc418160310 0x7fc418160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:26.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.788+0000 7fc41d000640 1 --2- 172.21.15.100:0/493186890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4180ad0e0 0x7fc4180ad4f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:26.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.788+0000 7fc41d000640 1 --2- 172.21.15.100:0/493186890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc41815d760 0x7fc4181535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:26.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.788+0000 7fc41d000640 1 -- 172.21.15.100:0/493186890 >> 172.21.15.100:0/493186890 conn(0x7fc41806d730 msgr2=0x7fc4180b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:26.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.788+0000 7fc41d000640 1 -- 172.21.15.100:0/493186890 shutdown_connections 2024-09-17T04:27:26.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.788+0000 7fc41d000640 1 -- 172.21.15.100:0/493186890 wait complete. 2024-09-17T04:27:26.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.790+0000 7fc41d000640 1 Processor -- start 2024-09-17T04:27:26.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.790+0000 7fc41d000640 1 -- start start 2024-09-17T04:27:26.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.792+0000 7fc41d000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4180ad0e0 0x7fc41809a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:26.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.792+0000 7fc41d000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc41815d760 0x7fc4180ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:26.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.792+0000 7fc41d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc418160310 0x7fc418155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:26.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.792+0000 7fc41d000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc4180b4da0 con 0x7fc418160310 2024-09-17T04:27:26.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.792+0000 7fc41d000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc4180b4c20 con 0x7fc4180ad0e0 2024-09-17T04:27:26.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.792+0000 7fc41d000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc418160c60 con 0x7fc41815d760 2024-09-17T04:27:26.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.792+0000 7fc416800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc418160310 0x7fc418155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:26.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.792+0000 7fc416800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc418160310 0x7fc418155e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57760/0 (socket says 172.21.15.100:57760) 2024-09-17T04:27:26.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.792+0000 7fc415e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4180ad0e0 0x7fc41809a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:26.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.792+0000 7fc416800640 1 -- 172.21.15.100:0/879621700 learned_addr learned my addr 172.21.15.100:0/879621700 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:26.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.792+0000 7fc40d400640 1 --2- 172.21.15.100:0/879621700 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc41815d760 0x7fc4180ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:26.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.793+0000 7fc416800640 1 -- 172.21.15.100:0/879621700 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc41815d760 msgr2=0x7fc4180ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:26.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.793+0000 7fc416800640 1 --2- 172.21.15.100:0/879621700 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc41815d760 0x7fc4180ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:26.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.793+0000 7fc416800640 1 -- 172.21.15.100:0/879621700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4180ad0e0 msgr2=0x7fc41809a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:26.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.793+0000 7fc416800640 1 --2- 172.21.15.100:0/879621700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4180ad0e0 0x7fc41809a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:26.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.793+0000 7fc416800640 1 -- 172.21.15.100:0/879621700 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4181f8c20 con 0x7fc418160310 2024-09-17T04:27:26.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.793+0000 7fc416800640 1 --2- 172.21.15.100:0/879621700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc418160310 0x7fc418155e30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc41000caa0 tx=0x7fc41000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:26.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.793+0000 7fc40ea00640 1 -- 172.21.15.100:0/879621700 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41000adf0 con 0x7fc418160310 2024-09-17T04:27:26.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.793+0000 7fc40ea00640 1 -- 172.21.15.100:0/879621700 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc41000f540 con 0x7fc418160310 2024-09-17T04:27:26.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.793+0000 7fc41d000640 1 -- 172.21.15.100:0/879621700 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4181f8f00 con 0x7fc418160310 2024-09-17T04:27:26.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.793+0000 7fc40ea00640 1 -- 172.21.15.100:0/879621700 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41000f990 con 0x7fc418160310 2024-09-17T04:27:26.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.793+0000 7fc41d000640 1 -- 172.21.15.100:0/879621700 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4181f94c0 con 0x7fc418160310 2024-09-17T04:27:26.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.795+0000 7fc41d000640 1 -- 172.21.15.100:0/879621700 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4181535c0 con 0x7fc418160310 2024-09-17T04:27:26.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.796+0000 7fc40ea00640 1 -- 172.21.15.100:0/879621700 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7fc41000fb30 con 0x7fc418160310 2024-09-17T04:27:26.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.797+0000 7fc40ea00640 1 --2- 172.21.15.100:0/879621700 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc3f4078d40 0x7fc3f407b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:26.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.797+0000 7fc40ea00640 1 -- 172.21.15.100:0/879621700 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7fc41009bbe0 con 0x7fc418160310 2024-09-17T04:27:26.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.797+0000 7fc415e00640 1 --2- 172.21.15.100:0/879621700 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc3f4078d40 0x7fc3f407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:26.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.798+0000 7fc415e00640 1 --2- 172.21.15.100:0/879621700 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc3f4078d40 0x7fc3f407b250 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc408005fd0 tx=0x7fc40800a250 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:26.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:26.801+0000 7fc40ea00640 1 -- 172.21.15.100:0/879621700 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc410014030 con 0x7fc418160310 2024-09-17T04:27:27.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.020+0000 7fc41d000640 1 -- 172.21.15.100:0/879621700 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4181f89b0 con 0x7fc418160310 2024-09-17T04:27:27.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.021+0000 7fc40ea00640 1 -- 172.21.15.100:0/879621700 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7fc41000fe20 con 0x7fc418160310 2024-09-17T04:27:27.022 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:27:27.022 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"btime":"2024-09-17T04:25:56:247389+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1086873435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1086873435},{"type":"v1","addr":"172.21.15.100:6833","nonce":1086873435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1708229764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1708229764},{"type":"v1","addr":"172.21.15.184:6836","nonce":1708229764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/828832783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":828832783},{"type":"v1","addr":"172.21.15.184:6837","nonce":828832783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:44.669203+0000","modified":"2024-09-17T04:25:47.693203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3801048032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3801048032},{"type":"v1","addr":"172.21.15.100:6835","nonce":3801048032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5295,"qdb_cluster":[5295]},"id":3}]} 2024-09-17T04:27:27.022 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-09-17T04:27:27.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.023+0000 7fc41d000640 1 -- 172.21.15.100:0/879621700 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc3f4078d40 msgr2=0x7fc3f407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:27.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.023+0000 7fc41d000640 1 --2- 172.21.15.100:0/879621700 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc3f4078d40 0x7fc3f407b250 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc408005fd0 tx=0x7fc40800a250 comp rx=0 tx=0).stop 2024-09-17T04:27:27.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.023+0000 7fc41d000640 1 -- 172.21.15.100:0/879621700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc418160310 msgr2=0x7fc418155e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:27.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.023+0000 7fc41d000640 1 --2- 172.21.15.100:0/879621700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc418160310 0x7fc418155e30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc41000caa0 tx=0x7fc41000cf60 comp rx=0 tx=0).stop 2024-09-17T04:27:27.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.023+0000 7fc41d000640 1 -- 172.21.15.100:0/879621700 shutdown_connections 2024-09-17T04:27:27.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.023+0000 7fc41d000640 1 --2- 172.21.15.100:0/879621700 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc3f4078d40 0x7fc3f407b250 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:27.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.023+0000 7fc41d000640 1 --2- 172.21.15.100:0/879621700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc418160310 0x7fc418155e30 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:27.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.023+0000 7fc41d000640 1 --2- 172.21.15.100:0/879621700 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc41815d760 0x7fc4180ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:27.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.024+0000 7fc41d000640 1 --2- 172.21.15.100:0/879621700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4180ad0e0 0x7fc41809a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:27.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.024+0000 7fc41d000640 1 -- 172.21.15.100:0/879621700 >> 172.21.15.100:0/879621700 conn(0x7fc41806d730 msgr2=0x7fc4180cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:27.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.024+0000 7fc41d000640 1 -- 172.21.15.100:0/879621700 shutdown_connections 2024-09-17T04:27:27.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.024+0000 7fc41d000640 1 -- 172.21.15.100:0/879621700 wait complete. 2024-09-17T04:27:27.049 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-17T04:27:27.292 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-09-17T04:27:27.292 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-17T04:27:27.292 DEBUG:tasks.ceph_test_case:wait_until_true: success in 75s and 0 retries 2024-09-17T04:27:27.293 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T04:27:27.293 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:27:27.386 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:27:27.476 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T04:27:27.476 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:27:27.476 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:27:27.476 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:27:27.511 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:27:27.511 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:27:27.511 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:27:27.511 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:27:27.548 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:27:27.549 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:27:27.549 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:27:27.549 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:27:27.586 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:27:27.586 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:27:27.586 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:27:27.586 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:27:27.624 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:27:27.624 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:27:27.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.846+0000 7f3a27600640 1 -- 172.21.15.100:0/1541916782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a200b3550 msgr2=0x7f3a200b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:27.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.846+0000 7f3a27600640 1 --2- 172.21.15.100:0/1541916782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a200b3550 0x7f3a200b3960 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3a20075250 tx=0x7f3a1802a980 comp rx=0 tx=0).stop 2024-09-17T04:27:27.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.846+0000 7f3a27600640 1 -- 172.21.15.100:0/1541916782 shutdown_connections 2024-09-17T04:27:27.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.846+0000 7f3a27600640 1 --2- 172.21.15.100:0/1541916782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a200b3550 0x7f3a200b3960 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:27.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.846+0000 7f3a27600640 1 --2- 172.21.15.100:0/1541916782 >> v2:172.21.15.184:3300/0 conn(0x7f3a2015d760 0x7f3a201535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:27.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.846+0000 7f3a27600640 1 -- 172.21.15.100:0/1541916782 >> 172.21.15.100:0/1541916782 conn(0x7f3a2006d730 msgr2=0x7f3a200b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:27.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.846+0000 7f3a27600640 1 -- 172.21.15.100:0/1541916782 shutdown_connections 2024-09-17T04:27:27.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.846+0000 7f3a27600640 1 -- 172.21.15.100:0/1541916782 wait complete. 2024-09-17T04:27:27.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.849+0000 7f3a27600640 1 Processor -- start 2024-09-17T04:27:27.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.849+0000 7f3a27600640 1 -- start start 2024-09-17T04:27:27.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.850+0000 7f3a27600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a200ad0e0 0x7f3a20151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:27.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.850+0000 7f3a27600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a200b3550 0x7f3a20151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:27.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.850+0000 7f3a27600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a2015d760 0x7f3a201f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:27.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.850+0000 7f3a27600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3a200b4da0 con 0x7f3a2015d760 2024-09-17T04:27:27.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.850+0000 7f3a27600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3a200b4c20 con 0x7f3a200ad0e0 2024-09-17T04:27:27.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.850+0000 7f3a27600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3a200b3ea0 con 0x7f3a200b3550 2024-09-17T04:27:27.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.851+0000 7f3a25600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a2015d760 0x7f3a201f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:27.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.851+0000 7f3a25600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a2015d760 0x7f3a201f73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57770/0 (socket says 172.21.15.100:57770) 2024-09-17T04:27:27.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.851+0000 7f3a25600640 1 -- 172.21.15.100:0/2802435213 learned_addr learned my addr 172.21.15.100:0/2802435213 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:27.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.851+0000 7f3a24c00640 1 --2- 172.21.15.100:0/2802435213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a200ad0e0 0x7f3a20151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:27.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.851+0000 7f3a1fe00640 1 --2- 172.21.15.100:0/2802435213 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a200b3550 0x7f3a20151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:27.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.851+0000 7f3a25600640 1 -- 172.21.15.100:0/2802435213 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a200b3550 msgr2=0x7f3a20151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:27.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.851+0000 7f3a25600640 1 --2- 172.21.15.100:0/2802435213 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a200b3550 0x7f3a20151b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:27.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.851+0000 7f3a25600640 1 -- 172.21.15.100:0/2802435213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a200ad0e0 msgr2=0x7f3a20151600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:27.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.851+0000 7f3a25600640 1 --2- 172.21.15.100:0/2802435213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a200ad0e0 0x7f3a20151600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:27.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.851+0000 7f3a25600640 1 -- 172.21.15.100:0/2802435213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a201f3ea0 con 0x7f3a2015d760 2024-09-17T04:27:27.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.851+0000 7f3a1fe00640 1 --2- 172.21.15.100:0/2802435213 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a200b3550 0x7f3a20151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:27:27.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.851+0000 7f3a25600640 1 --2- 172.21.15.100:0/2802435213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a2015d760 0x7f3a201f73a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3a1802af70 tx=0x7f3a18003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:27.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.852+0000 7f3a1cc00640 1 -- 172.21.15.100:0/2802435213 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a18034040 con 0x7f3a2015d760 2024-09-17T04:27:27.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.852+0000 7f3a1cc00640 1 -- 172.21.15.100:0/2802435213 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a18004500 con 0x7f3a2015d760 2024-09-17T04:27:27.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.852+0000 7f3a1cc00640 1 -- 172.21.15.100:0/2802435213 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a18045480 con 0x7f3a2015d760 2024-09-17T04:27:27.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.852+0000 7f3a27600640 1 -- 172.21.15.100:0/2802435213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a201f4180 con 0x7f3a2015d760 2024-09-17T04:27:27.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.852+0000 7f3a27600640 1 -- 172.21.15.100:0/2802435213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a201f4740 con 0x7f3a2015d760 2024-09-17T04:27:27.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.853+0000 7f3a27600640 1 -- 172.21.15.100:0/2802435213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a201f3c00 con 0x7f3a2015d760 2024-09-17T04:27:27.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.854+0000 7f3a1cc00640 1 -- 172.21.15.100:0/2802435213 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f3a18037020 con 0x7f3a2015d760 2024-09-17T04:27:27.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.856+0000 7f3a1cc00640 1 --2- 172.21.15.100:0/2802435213 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3a00078d70 0x7f3a0007b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:27.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.856+0000 7f3a1cc00640 1 -- 172.21.15.100:0/2802435213 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f3a180bc9c0 con 0x7f3a2015d760 2024-09-17T04:27:27.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.856+0000 7f3a24c00640 1 --2- 172.21.15.100:0/2802435213 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3a00078d70 0x7f3a0007b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:27.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.857+0000 7f3a24c00640 1 --2- 172.21.15.100:0/2802435213 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3a00078d70 0x7f3a0007b280 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000bb50 tx=0x7f3a1000b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:27.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:27.861+0000 7f3a1cc00640 1 -- 172.21.15.100:0/2802435213 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3a201f3c00 con 0x7f3a2015d760 2024-09-17T04:27:28.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.082+0000 7f3a27600640 1 -- 172.21.15.100:0/2802435213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a200b4380 con 0x7f3a2015d760 2024-09-17T04:27:28.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.082+0000 7f3a1cc00640 1 -- 172.21.15.100:0/2802435213 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f3a18084a30 con 0x7f3a2015d760 2024-09-17T04:27:28.082 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:27:28.083 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"btime":"2024-09-17T04:25:56:247389+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1086873435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1086873435},{"type":"v1","addr":"172.21.15.100:6833","nonce":1086873435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1708229764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1708229764},{"type":"v1","addr":"172.21.15.184:6836","nonce":1708229764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/828832783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":828832783},{"type":"v1","addr":"172.21.15.184:6837","nonce":828832783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:44.669203+0000","modified":"2024-09-17T04:25:47.693203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3801048032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3801048032},{"type":"v1","addr":"172.21.15.100:6835","nonce":3801048032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5295,"qdb_cluster":[5295]},"id":3}]} 2024-09-17T04:27:28.083 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-09-17T04:27:28.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.085+0000 7f3a27600640 1 -- 172.21.15.100:0/2802435213 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3a00078d70 msgr2=0x7f3a0007b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:28.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.085+0000 7f3a27600640 1 --2- 172.21.15.100:0/2802435213 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3a00078d70 0x7f3a0007b280 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000bb50 tx=0x7f3a1000b2e0 comp rx=0 tx=0).stop 2024-09-17T04:27:28.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.085+0000 7f3a27600640 1 -- 172.21.15.100:0/2802435213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a2015d760 msgr2=0x7f3a201f73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:28.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.085+0000 7f3a27600640 1 --2- 172.21.15.100:0/2802435213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a2015d760 0x7f3a201f73a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3a1802af70 tx=0x7f3a18003f10 comp rx=0 tx=0).stop 2024-09-17T04:27:28.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.085+0000 7f3a27600640 1 -- 172.21.15.100:0/2802435213 shutdown_connections 2024-09-17T04:27:28.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.085+0000 7f3a27600640 1 --2- 172.21.15.100:0/2802435213 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3a00078d70 0x7f3a0007b280 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:28.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.085+0000 7f3a27600640 1 --2- 172.21.15.100:0/2802435213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a2015d760 0x7f3a201f73a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:28.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.085+0000 7f3a27600640 1 --2- 172.21.15.100:0/2802435213 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a200b3550 0x7f3a20151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:28.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.085+0000 7f3a27600640 1 --2- 172.21.15.100:0/2802435213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a200ad0e0 0x7f3a20151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:28.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.085+0000 7f3a27600640 1 -- 172.21.15.100:0/2802435213 >> 172.21.15.100:0/2802435213 conn(0x7f3a2006d730 msgr2=0x7f3a200cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:28.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.085+0000 7f3a27600640 1 -- 172.21.15.100:0/2802435213 shutdown_connections 2024-09-17T04:27:28.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.085+0000 7f3a27600640 1 -- 172.21.15.100:0/2802435213 wait complete. 2024-09-17T04:27:28.110 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:27:28.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.332+0000 7f925dc00640 1 -- 172.21.15.100:0/2492100724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f923c004d40 msgr2=0x7f923c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:28.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.333+0000 7f925dc00640 1 --2- 172.21.15.100:0/2492100724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f923c004d40 0x7f923c005150 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9250005fd0 tx=0x7f923c005830 comp rx=0 tx=0).stop 2024-09-17T04:27:28.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.333+0000 7f9257400640 1 -- 172.21.15.100:0/2492100724 reap_dead start 2024-09-17T04:27:28.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.333+0000 7f925dc00640 1 -- 172.21.15.100:0/2492100724 shutdown_connections 2024-09-17T04:27:28.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.333+0000 7f925dc00640 1 -- 172.21.15.100:0/2492100724 >> 172.21.15.100:0/2492100724 conn(0x7f925806d730 msgr2=0x7f92580b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:28.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.333+0000 7f925dc00640 1 -- 172.21.15.100:0/2492100724 shutdown_connections 2024-09-17T04:27:28.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.333+0000 7f925dc00640 1 -- 172.21.15.100:0/2492100724 wait complete. 2024-09-17T04:27:28.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.335+0000 7f925dc00640 1 Processor -- start 2024-09-17T04:27:28.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.335+0000 7f925dc00640 1 -- start start 2024-09-17T04:27:28.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.336+0000 7f925dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f923c004d40 0x7f92581f95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:28.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.336+0000 7f925dc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9258074b30 0x7f9258154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:28.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.337+0000 7f925dc00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9258154fa0 0x7f92580cf030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:28.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.337+0000 7f925dc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f92580b3b30 con 0x7f923c004d40 2024-09-17T04:27:28.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.337+0000 7f925dc00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f92580b39b0 con 0x7f9258074b30 2024-09-17T04:27:28.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.337+0000 7f925dc00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f92580b3cb0 con 0x7f9258154fa0 2024-09-17T04:27:28.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.337+0000 7f9256a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f923c004d40 0x7f92581f95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:28.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.337+0000 7f9256a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f923c004d40 0x7f92581f95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57802/0 (socket says 172.21.15.100:57802) 2024-09-17T04:27:28.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.337+0000 7f9256a00640 1 -- 172.21.15.100:0/3178372636 learned_addr learned my addr 172.21.15.100:0/3178372636 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:28.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.337+0000 7f924e000640 1 --2- 172.21.15.100:0/3178372636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9258074b30 0x7f9258154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:28.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.337+0000 7f9257400640 1 --2- 172.21.15.100:0/3178372636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9258154fa0 0x7f92580cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:28.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.337+0000 7f9256a00640 1 -- 172.21.15.100:0/3178372636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9258154fa0 msgr2=0x7f92580cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:28.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.337+0000 7f9256a00640 1 --2- 172.21.15.100:0/3178372636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9258154fa0 0x7f92580cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:28.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.337+0000 7f9256a00640 1 -- 172.21.15.100:0/3178372636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9258074b30 msgr2=0x7f9258154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:28.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.337+0000 7f9256a00640 1 --2- 172.21.15.100:0/3178372636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9258074b30 0x7f9258154a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:28.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.337+0000 7f9256a00640 1 -- 172.21.15.100:0/3178372636 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92581f57c0 con 0x7f923c004d40 2024-09-17T04:27:28.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.337+0000 7f9257400640 1 --2- 172.21.15.100:0/3178372636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9258154fa0 0x7f92580cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:27:28.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.337+0000 7f9256a00640 1 --2- 172.21.15.100:0/3178372636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f923c004d40 0x7f92581f95b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f92480065e0 tx=0x7f92480039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:28.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.338+0000 7f924f400640 1 -- 172.21.15.100:0/3178372636 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9248015070 con 0x7f923c004d40 2024-09-17T04:27:28.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.338+0000 7f924f400640 1 -- 172.21.15.100:0/3178372636 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9248011920 con 0x7f923c004d40 2024-09-17T04:27:28.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.338+0000 7f924f400640 1 -- 172.21.15.100:0/3178372636 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9248011d70 con 0x7f923c004d40 2024-09-17T04:27:28.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.338+0000 7f925dc00640 1 -- 172.21.15.100:0/3178372636 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92581f5aa0 con 0x7f923c004d40 2024-09-17T04:27:28.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.338+0000 7f925dc00640 1 -- 172.21.15.100:0/3178372636 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92581f6060 con 0x7f923c004d40 2024-09-17T04:27:28.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.339+0000 7f925dc00640 1 -- 172.21.15.100:0/3178372636 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f925815f3d0 con 0x7f923c004d40 2024-09-17T04:27:28.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.340+0000 7f924f400640 1 -- 172.21.15.100:0/3178372636 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f924800f430 con 0x7f923c004d40 2024-09-17T04:27:28.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.342+0000 7f924f400640 1 --2- 172.21.15.100:0/3178372636 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9234078d40 0x7f923407b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:28.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.342+0000 7f924f400640 1 -- 172.21.15.100:0/3178372636 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f924809c070 con 0x7f923c004d40 2024-09-17T04:27:28.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.342+0000 7f924e000640 1 --2- 172.21.15.100:0/3178372636 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9234078d40 0x7f923407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:28.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.343+0000 7f924e000640 1 --2- 172.21.15.100:0/3178372636 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9234078d40 0x7f923407b250 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9258074a90 tx=0x7f9240005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:28.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.346+0000 7f924f400640 1 -- 172.21.15.100:0/3178372636 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f925815fd20 con 0x7f923c004d40 2024-09-17T04:27:28.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.562+0000 7f925dc00640 1 -- 172.21.15.100:0/3178372636 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f925815d2c0 con 0x7f923c004d40 2024-09-17T04:27:28.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.563+0000 7f924f400640 1 -- 172.21.15.100:0/3178372636 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f9248064050 con 0x7f923c004d40 2024-09-17T04:27:28.563 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:27:28.563 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"btime":"2024-09-17T04:25:56:247389+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1086873435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1086873435},{"type":"v1","addr":"172.21.15.100:6833","nonce":1086873435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1708229764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1708229764},{"type":"v1","addr":"172.21.15.184:6836","nonce":1708229764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/828832783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":828832783},{"type":"v1","addr":"172.21.15.184:6837","nonce":828832783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:44.669203+0000","modified":"2024-09-17T04:25:47.693203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3801048032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3801048032},{"type":"v1","addr":"172.21.15.100:6835","nonce":3801048032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5295,"qdb_cluster":[5295]},"id":3}]} 2024-09-17T04:27:28.564 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-09-17T04:27:28.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.565+0000 7f925dc00640 1 -- 172.21.15.100:0/3178372636 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9234078d40 msgr2=0x7f923407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:28.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.565+0000 7f925dc00640 1 --2- 172.21.15.100:0/3178372636 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9234078d40 0x7f923407b250 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9258074a90 tx=0x7f9240005f90 comp rx=0 tx=0).stop 2024-09-17T04:27:28.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.565+0000 7f925dc00640 1 -- 172.21.15.100:0/3178372636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f923c004d40 msgr2=0x7f92581f95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:28.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.565+0000 7f925dc00640 1 --2- 172.21.15.100:0/3178372636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f923c004d40 0x7f92581f95b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f92480065e0 tx=0x7f92480039c0 comp rx=0 tx=0).stop 2024-09-17T04:27:28.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.565+0000 7f925dc00640 1 -- 172.21.15.100:0/3178372636 shutdown_connections 2024-09-17T04:27:28.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.565+0000 7f925dc00640 1 --2- 172.21.15.100:0/3178372636 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9234078d40 0x7f923407b250 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:28.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.565+0000 7f925dc00640 1 --2- 172.21.15.100:0/3178372636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9258154fa0 0x7f92580cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:28.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.565+0000 7f925dc00640 1 --2- 172.21.15.100:0/3178372636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9258074b30 0x7f9258154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:28.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.565+0000 7f925dc00640 1 --2- 172.21.15.100:0/3178372636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f923c004d40 0x7f92581f95b0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:28.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.565+0000 7f925dc00640 1 -- 172.21.15.100:0/3178372636 >> 172.21.15.100:0/3178372636 conn(0x7f925806d730 msgr2=0x7f925815dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:28.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.565+0000 7f925dc00640 1 -- 172.21.15.100:0/3178372636 shutdown_connections 2024-09-17T04:27:28.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.565+0000 7f925dc00640 1 -- 172.21.15.100:0/3178372636 wait complete. 2024-09-17T04:27:28.590 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:27:28.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.811+0000 7f5e2ca00640 1 -- 172.21.15.100:0/1419188768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e0c0041c0 msgr2=0x7f5e0c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:28.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.811+0000 7f5e2ca00640 1 --2- 172.21.15.100:0/1419188768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e0c0041c0 0x7f5e0c004800 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5e14006f80 tx=0x7f5e0c005830 comp rx=0 tx=0).stop 2024-09-17T04:27:28.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.811+0000 7f5e26800640 1 -- 172.21.15.100:0/1419188768 reap_dead start 2024-09-17T04:27:28.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.811+0000 7f5e2ca00640 1 -- 172.21.15.100:0/1419188768 shutdown_connections 2024-09-17T04:27:28.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.811+0000 7f5e2ca00640 1 -- 172.21.15.100:0/1419188768 >> 172.21.15.100:0/1419188768 conn(0x7f5e2806d730 msgr2=0x7f5e280b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:28.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.811+0000 7f5e2ca00640 1 -- 172.21.15.100:0/1419188768 shutdown_connections 2024-09-17T04:27:28.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.811+0000 7f5e2ca00640 1 -- 172.21.15.100:0/1419188768 wait complete. 2024-09-17T04:27:28.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.813+0000 7f5e2ca00640 1 Processor -- start 2024-09-17T04:27:28.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.813+0000 7f5e2ca00640 1 -- start start 2024-09-17T04:27:28.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.815+0000 7f5e2ca00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e0c0041c0 0x7f5e28074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:28.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.815+0000 7f5e2ca00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e2814ed10 0x7f5e281453d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:28.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.815+0000 7f5e2ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e28145910 0x7f5e28073650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:28.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.815+0000 7f5e2ca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2806f260 con 0x7f5e28145910 2024-09-17T04:27:28.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.815+0000 7f5e2ca00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5e280b4e30 con 0x7f5e0c0041c0 2024-09-17T04:27:28.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.815+0000 7f5e2ca00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5e28165890 con 0x7f5e2814ed10 2024-09-17T04:27:28.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.815+0000 7f5e26800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e28145910 0x7f5e28073650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:28.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.815+0000 7f5e26800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e28145910 0x7f5e28073650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57826/0 (socket says 172.21.15.100:57826) 2024-09-17T04:27:28.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.815+0000 7f5e26800640 1 -- 172.21.15.100:0/4222237246 learned_addr learned my addr 172.21.15.100:0/4222237246 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:28.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.815+0000 7f5e25e00640 1 --2- 172.21.15.100:0/4222237246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e0c0041c0 0x7f5e28074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:28.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.816+0000 7f5e25400640 1 --2- 172.21.15.100:0/4222237246 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e2814ed10 0x7f5e281453d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:28.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.816+0000 7f5e26800640 1 -- 172.21.15.100:0/4222237246 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e2814ed10 msgr2=0x7f5e281453d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:28.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.816+0000 7f5e26800640 1 --2- 172.21.15.100:0/4222237246 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e2814ed10 0x7f5e281453d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:28.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.816+0000 7f5e26800640 1 -- 172.21.15.100:0/4222237246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e0c0041c0 msgr2=0x7f5e28074bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:28.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.816+0000 7f5e26800640 1 --2- 172.21.15.100:0/4222237246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e0c0041c0 0x7f5e28074bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:28.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.816+0000 7f5e26800640 1 -- 172.21.15.100:0/4222237246 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e28145d20 con 0x7f5e28145910 2024-09-17T04:27:28.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.816+0000 7f5e25400640 1 --2- 172.21.15.100:0/4222237246 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e2814ed10 0x7f5e281453d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:27:28.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.816+0000 7f5e26800640 1 --2- 172.21.15.100:0/4222237246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e28145910 0x7f5e28073650 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5e20005fd0 tx=0x7f5e20010e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:28.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.816+0000 7f5e1e000640 1 -- 172.21.15.100:0/4222237246 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2000ede0 con 0x7f5e28145910 2024-09-17T04:27:28.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.816+0000 7f5e1e000640 1 -- 172.21.15.100:0/4222237246 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e20015070 con 0x7f5e28145910 2024-09-17T04:27:28.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.816+0000 7f5e2ca00640 1 -- 172.21.15.100:0/4222237246 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e281f9c10 con 0x7f5e28145910 2024-09-17T04:27:28.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.816+0000 7f5e1e000640 1 -- 172.21.15.100:0/4222237246 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e20003d00 con 0x7f5e28145910 2024-09-17T04:27:28.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.816+0000 7f5e2ca00640 1 -- 172.21.15.100:0/4222237246 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e281f5e20 con 0x7f5e28145910 2024-09-17T04:27:28.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.817+0000 7f5e2ca00640 1 -- 172.21.15.100:0/4222237246 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e281f9e30 con 0x7f5e28145910 2024-09-17T04:27:28.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.819+0000 7f5e1e000640 1 -- 172.21.15.100:0/4222237246 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f5e20004030 con 0x7f5e28145910 2024-09-17T04:27:28.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.819+0000 7f5e1e000640 1 --2- 172.21.15.100:0/4222237246 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5e08078d70 0x7f5e0807b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:28.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.820+0000 7f5e1e000640 1 -- 172.21.15.100:0/4222237246 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f5e2009bc10 con 0x7f5e28145910 2024-09-17T04:27:28.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.820+0000 7f5e25e00640 1 --2- 172.21.15.100:0/4222237246 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5e08078d70 0x7f5e0807b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:28.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.821+0000 7f5e25e00640 1 --2- 172.21.15.100:0/4222237246 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5e08078d70 0x7f5e0807b280 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5e180063f0 tx=0x7f5e18006dd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:28.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:28.824+0000 7f5e1e000640 1 -- 172.21.15.100:0/4222237246 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f5e28145d20 con 0x7f5e28145910 2024-09-17T04:27:29.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.009+0000 7f5e2ca00640 1 -- 172.21.15.100:0/4222237246 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e281eaf20 con 0x7f5e28145910 2024-09-17T04:27:29.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.010+0000 7f5e1e000640 1 -- 172.21.15.100:0/4222237246 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+16813 (secure 0 0 0) 0x7f5e20063c70 con 0x7f5e28145910 2024-09-17T04:27:29.012 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:27:29.012 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":50,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:25:54.239620+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:25:42.571024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-17T04:25:43.631985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.32","mappings":[{"from":0,"to":5}]},{"pgid":"7.37","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:27:29.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.013+0000 7f5e2ca00640 1 -- 172.21.15.100:0/4222237246 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5e08078d70 msgr2=0x7f5e0807b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:29.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.013+0000 7f5e2ca00640 1 --2- 172.21.15.100:0/4222237246 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5e08078d70 0x7f5e0807b280 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5e180063f0 tx=0x7f5e18006dd0 comp rx=0 tx=0).stop 2024-09-17T04:27:29.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.013+0000 7f5e2ca00640 1 -- 172.21.15.100:0/4222237246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e28145910 msgr2=0x7f5e28073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:29.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.013+0000 7f5e2ca00640 1 --2- 172.21.15.100:0/4222237246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e28145910 0x7f5e28073650 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5e20005fd0 tx=0x7f5e20010e70 comp rx=0 tx=0).stop 2024-09-17T04:27:29.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.013+0000 7f5e2ca00640 1 -- 172.21.15.100:0/4222237246 shutdown_connections 2024-09-17T04:27:29.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.013+0000 7f5e2ca00640 1 --2- 172.21.15.100:0/4222237246 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5e08078d70 0x7f5e0807b280 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:29.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.013+0000 7f5e2ca00640 1 --2- 172.21.15.100:0/4222237246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e28145910 0x7f5e28073650 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:29.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.013+0000 7f5e2ca00640 1 --2- 172.21.15.100:0/4222237246 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e2814ed10 0x7f5e281453d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:29.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.013+0000 7f5e2ca00640 1 --2- 172.21.15.100:0/4222237246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e0c0041c0 0x7f5e28074bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:29.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.013+0000 7f5e2ca00640 1 -- 172.21.15.100:0/4222237246 >> 172.21.15.100:0/4222237246 conn(0x7f5e2806d730 msgr2=0x7f5e28149af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:29.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.013+0000 7f5e2ca00640 1 -- 172.21.15.100:0/4222237246 shutdown_connections 2024-09-17T04:27:29.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.013+0000 7f5e2ca00640 1 -- 172.21.15.100:0/4222237246 wait complete. 2024-09-17T04:27:29.036 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T04:27:29.036 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:27:29.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.262+0000 7f71fe200640 1 -- 172.21.15.100:0/1963677627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f80b2750 msgr2=0x7f71f815f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:29.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.262+0000 7f71fe200640 1 --2- 172.21.15.100:0/1963677627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f80b2750 0x7f71f815f3d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f71e4009830 tx=0x7f71e402a470 comp rx=0 tx=0).stop 2024-09-17T04:27:29.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.262+0000 7f71fe200640 1 -- 172.21.15.100:0/1963677627 shutdown_connections 2024-09-17T04:27:29.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.262+0000 7f71fe200640 1 --2- 172.21.15.100:0/1963677627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f80b2750 0x7f71f815f3d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:29.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.262+0000 7f71fe200640 1 --2- 172.21.15.100:0/1963677627 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71f80aebc0 0x7f71f8152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:29.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.262+0000 7f71fe200640 1 -- 172.21.15.100:0/1963677627 >> 172.21.15.100:0/1963677627 conn(0x7f71f806d730 msgr2=0x7f71f80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:29.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.263+0000 7f71fe200640 1 -- 172.21.15.100:0/1963677627 shutdown_connections 2024-09-17T04:27:29.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.263+0000 7f71fe200640 1 -- 172.21.15.100:0/1963677627 wait complete. 2024-09-17T04:27:29.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.265+0000 7f71fe200640 1 Processor -- start 2024-09-17T04:27:29.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.265+0000 7f71fe200640 1 -- start start 2024-09-17T04:27:29.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.266+0000 7f71fe200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71f80aebc0 0x7f71f80cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:29.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.266+0000 7f71fe200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71f80b2750 0x7f71f814a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:29.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.266+0000 7f71fe200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f815f910 0x7f71f8073f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:29.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.266+0000 7f71fe200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f71f815c4a0 con 0x7f71f815f910 2024-09-17T04:27:29.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.266+0000 7f71fe200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f71f80b4a20 con 0x7f71f80b2750 2024-09-17T04:27:29.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.266+0000 7f71fe200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f71f815c620 con 0x7f71f80aebc0 2024-09-17T04:27:29.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.267+0000 7f71f7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f815f910 0x7f71f8073f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:29.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.267+0000 7f71f7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f815f910 0x7f71f8073f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57842/0 (socket says 172.21.15.100:57842) 2024-09-17T04:27:29.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.267+0000 7f71f7e00640 1 -- 172.21.15.100:0/3591802154 learned_addr learned my addr 172.21.15.100:0/3591802154 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:29.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.267+0000 7f71f7400640 1 --2- 172.21.15.100:0/3591802154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71f80aebc0 0x7f71f80cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:29.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.267+0000 7f71f6a00640 1 --2- 172.21.15.100:0/3591802154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71f80b2750 0x7f71f814a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:29.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.267+0000 7f71f7e00640 1 -- 172.21.15.100:0/3591802154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71f80aebc0 msgr2=0x7f71f80cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:29.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.267+0000 7f71f7e00640 1 --2- 172.21.15.100:0/3591802154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71f80aebc0 0x7f71f80cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:29.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.267+0000 7f71f7e00640 1 -- 172.21.15.100:0/3591802154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71f80b2750 msgr2=0x7f71f814a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:29.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.267+0000 7f71f7e00640 1 --2- 172.21.15.100:0/3591802154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71f80b2750 0x7f71f814a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:29.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.267+0000 7f71f7e00640 1 -- 172.21.15.100:0/3591802154 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f814ae70 con 0x7f71f815f910 2024-09-17T04:27:29.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.267+0000 7f71f6a00640 1 --2- 172.21.15.100:0/3591802154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71f80b2750 0x7f71f814a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:27:29.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.267+0000 7f71f7e00640 1 --2- 172.21.15.100:0/3591802154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f815f910 0x7f71f8073f50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f71f000c6f0 tx=0x7f71f000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:29.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.268+0000 7f71ef400640 1 -- 172.21.15.100:0/3591802154 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f0017070 con 0x7f71f815f910 2024-09-17T04:27:29.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.268+0000 7f71ef400640 1 -- 172.21.15.100:0/3591802154 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71f0003c90 con 0x7f71f815f910 2024-09-17T04:27:29.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.268+0000 7f71ef400640 1 -- 172.21.15.100:0/3591802154 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f0003e60 con 0x7f71f815f910 2024-09-17T04:27:29.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.268+0000 7f71fe200640 1 -- 172.21.15.100:0/3591802154 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f8156d30 con 0x7f71f815f910 2024-09-17T04:27:29.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.268+0000 7f71fe200640 1 -- 172.21.15.100:0/3591802154 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f8073470 con 0x7f71f815f910 2024-09-17T04:27:29.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.269+0000 7f71fe200640 1 -- 172.21.15.100:0/3591802154 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71f815cb40 con 0x7f71f815f910 2024-09-17T04:27:29.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.270+0000 7f71ef400640 1 -- 172.21.15.100:0/3591802154 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f71f001a070 con 0x7f71f815f910 2024-09-17T04:27:29.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.271+0000 7f71ef400640 1 --2- 172.21.15.100:0/3591802154 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f71d8078d40 0x7f71d807b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:29.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.271+0000 7f71ef400640 1 -- 172.21.15.100:0/3591802154 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f71f009a7f0 con 0x7f71f815f910 2024-09-17T04:27:29.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.271+0000 7f71f7400640 1 --2- 172.21.15.100:0/3591802154 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f71d8078d40 0x7f71d807b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:29.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.272+0000 7f71f7400640 1 --2- 172.21.15.100:0/3591802154 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f71d8078d40 0x7f71d807b250 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009ac0 tx=0x7f71e8005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:29.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.275+0000 7f71ef400640 1 -- 172.21.15.100:0/3591802154 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f71f814ae70 con 0x7f71f815f910 2024-09-17T04:27:29.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.491+0000 7f71fe200640 1 -- 172.21.15.100:0/3591802154 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71f8070820 con 0x7f71f815f910 2024-09-17T04:27:29.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.491+0000 7f71ef400640 1 -- 172.21.15.100:0/3591802154 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f71f815fd20 con 0x7f71f815f910 2024-09-17T04:27:29.492 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:27:29.492 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"btime":"2024-09-17T04:25:56:247389+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1086873435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1086873435},{"type":"v1","addr":"172.21.15.100:6833","nonce":1086873435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1708229764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1708229764},{"type":"v1","addr":"172.21.15.184:6836","nonce":1708229764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/828832783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":828832783},{"type":"v1","addr":"172.21.15.184:6837","nonce":828832783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:44.669203+0000","modified":"2024-09-17T04:25:47.693203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3801048032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3801048032},{"type":"v1","addr":"172.21.15.100:6835","nonce":3801048032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5295,"qdb_cluster":[5295]},"id":3}]} 2024-09-17T04:27:29.492 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-09-17T04:27:29.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.494+0000 7f71fe200640 1 -- 172.21.15.100:0/3591802154 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f71d8078d40 msgr2=0x7f71d807b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:29.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.494+0000 7f71fe200640 1 --2- 172.21.15.100:0/3591802154 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f71d8078d40 0x7f71d807b250 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009ac0 tx=0x7f71e8005f90 comp rx=0 tx=0).stop 2024-09-17T04:27:29.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.494+0000 7f71fe200640 1 -- 172.21.15.100:0/3591802154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f815f910 msgr2=0x7f71f8073f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:29.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.494+0000 7f71fe200640 1 --2- 172.21.15.100:0/3591802154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f815f910 0x7f71f8073f50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f71f000c6f0 tx=0x7f71f000cbb0 comp rx=0 tx=0).stop 2024-09-17T04:27:29.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.494+0000 7f71fe200640 1 -- 172.21.15.100:0/3591802154 shutdown_connections 2024-09-17T04:27:29.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.494+0000 7f71fe200640 1 --2- 172.21.15.100:0/3591802154 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f71d8078d40 0x7f71d807b250 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:29.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.494+0000 7f71fe200640 1 --2- 172.21.15.100:0/3591802154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f815f910 0x7f71f8073f50 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:29.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.494+0000 7f71fe200640 1 --2- 172.21.15.100:0/3591802154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71f80b2750 0x7f71f814a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:29.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.494+0000 7f71fe200640 1 --2- 172.21.15.100:0/3591802154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71f80aebc0 0x7f71f80cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:29.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.494+0000 7f71fe200640 1 -- 172.21.15.100:0/3591802154 >> 172.21.15.100:0/3591802154 conn(0x7f71f806d730 msgr2=0x7f71f815e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:29.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.494+0000 7f71fe200640 1 -- 172.21.15.100:0/3591802154 shutdown_connections 2024-09-17T04:27:29.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.494+0000 7f71fe200640 1 -- 172.21.15.100:0/3591802154 wait complete. 2024-09-17T04:27:29.520 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:27:29.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.739+0000 7f6562400640 1 -- 172.21.15.100:0/393583314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f655c15d760 msgr2=0x7f655c1535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:29.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.739+0000 7f6562400640 1 --2- 172.21.15.100:0/393583314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f655c15d760 0x7f655c1535c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f655c0b3100 tx=0x7f654c02a4d0 comp rx=0 tx=0).stop 2024-09-17T04:27:29.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.739+0000 7f6562400640 1 -- 172.21.15.100:0/393583314 shutdown_connections 2024-09-17T04:27:29.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.739+0000 7f6562400640 1 --2- 172.21.15.100:0/393583314 >> v2:172.21.15.184:3300/0 conn(0x7f655c0b3550 0x7f655c0b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:29.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.739+0000 7f6562400640 1 --2- 172.21.15.100:0/393583314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f655c15d760 0x7f655c1535c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:29.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.739+0000 7f6562400640 1 -- 172.21.15.100:0/393583314 >> 172.21.15.100:0/393583314 conn(0x7f655c06d730 msgr2=0x7f655c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:29.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.739+0000 7f6562400640 1 -- 172.21.15.100:0/393583314 shutdown_connections 2024-09-17T04:27:29.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.740+0000 7f6562400640 1 -- 172.21.15.100:0/393583314 wait complete. 2024-09-17T04:27:29.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.742+0000 7f6562400640 1 Processor -- start 2024-09-17T04:27:29.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.742+0000 7f6562400640 1 -- start start 2024-09-17T04:27:29.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.743+0000 7f6562400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f655c0ad0e0 0x7f655c1f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:29.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.743+0000 7f6562400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f655c0b3550 0x7f655c0998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:29.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.743+0000 7f6562400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f655c15d760 0x7f655c155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:29.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.743+0000 7f6562400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f655c0b4da0 con 0x7f655c15d760 2024-09-17T04:27:29.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.743+0000 7f6562400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f655c0b4c20 con 0x7f655c0b3550 2024-09-17T04:27:29.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.743+0000 7f6562400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f655c0b3ea0 con 0x7f655c0ad0e0 2024-09-17T04:27:29.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.743+0000 7f655be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f655c15d760 0x7f655c155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:29.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.744+0000 7f655be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f655c15d760 0x7f655c155d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57870/0 (socket says 172.21.15.100:57870) 2024-09-17T04:27:29.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.744+0000 7f655be00640 1 -- 172.21.15.100:0/3920815251 learned_addr learned my addr 172.21.15.100:0/3920815251 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:29.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.744+0000 7f655b400640 1 --2- 172.21.15.100:0/3920815251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f655c0ad0e0 0x7f655c1f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:29.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.744+0000 7f655aa00640 1 --2- 172.21.15.100:0/3920815251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f655c0b3550 0x7f655c0998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:29.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.744+0000 7f655be00640 1 -- 172.21.15.100:0/3920815251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f655c0ad0e0 msgr2=0x7f655c1f64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:29.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.744+0000 7f655be00640 1 --2- 172.21.15.100:0/3920815251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f655c0ad0e0 0x7f655c1f64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:29.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.744+0000 7f655be00640 1 -- 172.21.15.100:0/3920815251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f655c0b3550 msgr2=0x7f655c0998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:29.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.744+0000 7f655be00640 1 --2- 172.21.15.100:0/3920815251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f655c0b3550 0x7f655c0998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:29.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.744+0000 7f655be00640 1 -- 172.21.15.100:0/3920815251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655c151a20 con 0x7f655c15d760 2024-09-17T04:27:29.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.744+0000 7f655aa00640 1 --2- 172.21.15.100:0/3920815251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f655c0b3550 0x7f655c0998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:27:29.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.744+0000 7f655b400640 1 --2- 172.21.15.100:0/3920815251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f655c0ad0e0 0x7f655c1f64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:27:29.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.744+0000 7f655be00640 1 --2- 172.21.15.100:0/3920815251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f655c15d760 0x7f655c155d90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f655400e990 tx=0x7f655400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:29.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.745+0000 7f6553400640 1 -- 172.21.15.100:0/3920815251 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6554004d20 con 0x7f655c15d760 2024-09-17T04:27:29.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.745+0000 7f6553400640 1 -- 172.21.15.100:0/3920815251 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f655400caa0 con 0x7f655c15d760 2024-09-17T04:27:29.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.745+0000 7f6553400640 1 -- 172.21.15.100:0/3920815251 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6554005020 con 0x7f655c15d760 2024-09-17T04:27:29.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.745+0000 7f6562400640 1 -- 172.21.15.100:0/3920815251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f655c1e8f50 con 0x7f655c15d760 2024-09-17T04:27:29.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.745+0000 7f6562400640 1 -- 172.21.15.100:0/3920815251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f655c154250 con 0x7f655c15d760 2024-09-17T04:27:29.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.746+0000 7f6562400640 1 -- 172.21.15.100:0/3920815251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f655c0b43a0 con 0x7f655c15d760 2024-09-17T04:27:29.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.747+0000 7f6553400640 1 -- 172.21.15.100:0/3920815251 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f65540051c0 con 0x7f655c15d760 2024-09-17T04:27:29.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.748+0000 7f6553400640 1 --2- 172.21.15.100:0/3920815251 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f653c078d40 0x7f653c07b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:29.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.748+0000 7f6553400640 1 -- 172.21.15.100:0/3920815251 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f655409b9f0 con 0x7f655c15d760 2024-09-17T04:27:29.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.749+0000 7f655b400640 1 --2- 172.21.15.100:0/3920815251 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f653c078d40 0x7f653c07b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:29.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.749+0000 7f655b400640 1 --2- 172.21.15.100:0/3920815251 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f653c078d40 0x7f653c07b250 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f654c033000 tx=0x7f654c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:29.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.753+0000 7f6553400640 1 -- 172.21.15.100:0/3920815251 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6554016030 con 0x7f655c15d760 2024-09-17T04:27:29.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.967+0000 7f6562400640 1 -- 172.21.15.100:0/3920815251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f655c0b2020 con 0x7f655c15d760 2024-09-17T04:27:29.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.968+0000 7f6553400640 1 -- 172.21.15.100:0/3920815251 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f6554063b80 con 0x7f655c15d760 2024-09-17T04:27:29.968 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:27:29.968 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"btime":"2024-09-17T04:25:56:247389+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1086873435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1086873435},{"type":"v1","addr":"172.21.15.100:6833","nonce":1086873435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1708229764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1708229764},{"type":"v1","addr":"172.21.15.184:6836","nonce":1708229764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/828832783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":828832783},{"type":"v1","addr":"172.21.15.184:6837","nonce":828832783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:25:44.669203+0000","modified":"2024-09-17T04:25:47.693203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3801048032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3801048032},{"type":"v1","addr":"172.21.15.100:6835","nonce":3801048032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5295,"qdb_cluster":[5295]},"id":3}]} 2024-09-17T04:27:29.969 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-09-17T04:27:29.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.970+0000 7f6562400640 1 -- 172.21.15.100:0/3920815251 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f653c078d40 msgr2=0x7f653c07b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:29.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.970+0000 7f6562400640 1 --2- 172.21.15.100:0/3920815251 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f653c078d40 0x7f653c07b250 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f654c033000 tx=0x7f654c004730 comp rx=0 tx=0).stop 2024-09-17T04:27:29.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.970+0000 7f6562400640 1 -- 172.21.15.100:0/3920815251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f655c15d760 msgr2=0x7f655c155d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:29.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.970+0000 7f6562400640 1 --2- 172.21.15.100:0/3920815251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f655c15d760 0x7f655c155d90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f655400e990 tx=0x7f655400ee50 comp rx=0 tx=0).stop 2024-09-17T04:27:29.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.970+0000 7f6562400640 1 -- 172.21.15.100:0/3920815251 shutdown_connections 2024-09-17T04:27:29.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.970+0000 7f6562400640 1 --2- 172.21.15.100:0/3920815251 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f653c078d40 0x7f653c07b250 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:29.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.970+0000 7f6562400640 1 --2- 172.21.15.100:0/3920815251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f655c15d760 0x7f655c155d90 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:29.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.970+0000 7f6562400640 1 --2- 172.21.15.100:0/3920815251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f655c0b3550 0x7f655c0998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:29.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.970+0000 7f6562400640 1 --2- 172.21.15.100:0/3920815251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f655c0ad0e0 0x7f655c1f64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:29.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.970+0000 7f6562400640 1 -- 172.21.15.100:0/3920815251 >> 172.21.15.100:0/3920815251 conn(0x7f655c06d730 msgr2=0x7f655c1602a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:29.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.970+0000 7f6562400640 1 -- 172.21.15.100:0/3920815251 shutdown_connections 2024-09-17T04:27:29.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:29.970+0000 7f6562400640 1 -- 172.21.15.100:0/3920815251 wait complete. 2024-09-17T04:27:29.992 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:27:30.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.214+0000 7fbbe4e00640 1 -- 172.21.15.100:0/2635367198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbe00ad9c0 msgr2=0x7fbbe00af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:30.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.214+0000 7fbbe4e00640 1 --2- 172.21.15.100:0/2635367198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbe00ad9c0 0x7fbbe00af170 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0005fd0 tx=0x7fbbd002a540 comp rx=0 tx=0).stop 2024-09-17T04:27:30.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.214+0000 7fbbe4e00640 1 -- 172.21.15.100:0/2635367198 shutdown_connections 2024-09-17T04:27:30.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.214+0000 7fbbe4e00640 1 --2- 172.21.15.100:0/2635367198 >> v2:172.21.15.184:3300/0 conn(0x7fbbe015a0f0 0x7fbbe015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:30.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.214+0000 7fbbe4e00640 1 --2- 172.21.15.100:0/2635367198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbe00ad9c0 0x7fbbe00af170 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:30.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.214+0000 7fbbe4e00640 1 -- 172.21.15.100:0/2635367198 >> 172.21.15.100:0/2635367198 conn(0x7fbbe00b1e40 msgr2=0x7fbbe00b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:30.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.215+0000 7fbbe4e00640 1 -- 172.21.15.100:0/2635367198 shutdown_connections 2024-09-17T04:27:30.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.215+0000 7fbbe4e00640 1 -- 172.21.15.100:0/2635367198 wait complete. 2024-09-17T04:27:30.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.217+0000 7fbbe4e00640 1 Processor -- start 2024-09-17T04:27:30.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.217+0000 7fbbe4e00640 1 -- start start 2024-09-17T04:27:30.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.218+0000 7fbbe4e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbe00ad9c0 0x7fbbe00ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:30.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.218+0000 7fbbe4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbe00b2f60 0x7fbbe01437c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:30.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.218+0000 7fbbe4e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbe015a0f0 0x7fbbe0158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:30.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.218+0000 7fbbe4e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe006f410 con 0x7fbbe00b2f60 2024-09-17T04:27:30.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.218+0000 7fbbe4e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe006f290 con 0x7fbbe00ad9c0 2024-09-17T04:27:30.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.218+0000 7fbbe4e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbbe006f590 con 0x7fbbe015a0f0 2024-09-17T04:27:30.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.219+0000 7fbbd6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbe00b2f60 0x7fbbe01437c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:30.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.219+0000 7fbbd6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbe00b2f60 0x7fbbe01437c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57872/0 (socket says 172.21.15.100:57872) 2024-09-17T04:27:30.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.219+0000 7fbbd6a00640 1 -- 172.21.15.100:0/2605713698 learned_addr learned my addr 172.21.15.100:0/2605713698 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:30.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.219+0000 7fbbdf400640 1 --2- 172.21.15.100:0/2605713698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbe00ad9c0 0x7fbbe00ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:30.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.219+0000 7fbbdfe00640 1 --2- 172.21.15.100:0/2605713698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbe015a0f0 0x7fbbe0158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:30.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.219+0000 7fbbd6a00640 1 -- 172.21.15.100:0/2605713698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbe015a0f0 msgr2=0x7fbbe0158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:30.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.219+0000 7fbbd6a00640 1 --2- 172.21.15.100:0/2605713698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbe015a0f0 0x7fbbe0158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:30.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.219+0000 7fbbd6a00640 1 -- 172.21.15.100:0/2605713698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbe00ad9c0 msgr2=0x7fbbe00ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:30.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.219+0000 7fbbd6a00640 1 --2- 172.21.15.100:0/2605713698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbe00ad9c0 0x7fbbe00ce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:30.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.219+0000 7fbbd6a00640 1 -- 172.21.15.100:0/2605713698 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe0156130 con 0x7fbbe00b2f60 2024-09-17T04:27:30.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.219+0000 7fbbd6a00640 1 --2- 172.21.15.100:0/2605713698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbe00b2f60 0x7fbbe01437c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800c3c0 tx=0x7fbbc800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:30.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.220+0000 7fbbd7e00640 1 -- 172.21.15.100:0/2605713698 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8017070 con 0x7fbbe00b2f60 2024-09-17T04:27:30.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.220+0000 7fbbd7e00640 1 -- 172.21.15.100:0/2605713698 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbc80053c0 con 0x7fbbe00b2f60 2024-09-17T04:27:30.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.220+0000 7fbbe4e00640 1 -- 172.21.15.100:0/2605713698 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe00ccff0 con 0x7fbbe00b2f60 2024-09-17T04:27:30.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.220+0000 7fbbd7e00640 1 -- 172.21.15.100:0/2605713698 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8003850 con 0x7fbbe00b2f60 2024-09-17T04:27:30.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.220+0000 7fbbe4e00640 1 -- 172.21.15.100:0/2605713698 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe00cd5b0 con 0x7fbbe00b2f60 2024-09-17T04:27:30.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.221+0000 7fbbe4e00640 1 -- 172.21.15.100:0/2605713698 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbe00ccd60 con 0x7fbbe00b2f60 2024-09-17T04:27:30.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.222+0000 7fbbd7e00640 1 -- 172.21.15.100:0/2605713698 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7fbbc801a070 con 0x7fbbe00b2f60 2024-09-17T04:27:30.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.223+0000 7fbbd7e00640 1 --2- 172.21.15.100:0/2605713698 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbbbc078d70 0x7fbbbc07b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:30.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.223+0000 7fbbd7e00640 1 -- 172.21.15.100:0/2605713698 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7fbbc809a540 con 0x7fbbe00b2f60 2024-09-17T04:27:30.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.223+0000 7fbbdf400640 1 --2- 172.21.15.100:0/2605713698 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbbbc078d70 0x7fbbbc07b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:30.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.224+0000 7fbbdf400640 1 --2- 172.21.15.100:0/2605713698 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbbbc078d70 0x7fbbbc07b280 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0033000 tx=0x7fbbd0004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:30.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.227+0000 7fbbd7e00640 1 -- 172.21.15.100:0/2605713698 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fbbe0156130 con 0x7fbbe00b2f60 2024-09-17T04:27:30.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.411+0000 7fbbe4e00640 1 -- 172.21.15.100:0/2605713698 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbbe006fab0 con 0x7fbbe00b2f60 2024-09-17T04:27:30.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.413+0000 7fbbd7e00640 1 -- 172.21.15.100:0/2605713698 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+16813 (secure 0 0 0) 0x7fbbc8062520 con 0x7fbbe00b2f60 2024-09-17T04:27:30.414 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:27:30.414 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":50,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:25:54.239620+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:25:42.571024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-17T04:25:43.631985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.32","mappings":[{"from":0,"to":5}]},{"pgid":"7.37","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:27:30.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.415+0000 7fbbe4e00640 1 -- 172.21.15.100:0/2605713698 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbbbc078d70 msgr2=0x7fbbbc07b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:30.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.415+0000 7fbbe4e00640 1 --2- 172.21.15.100:0/2605713698 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbbbc078d70 0x7fbbbc07b280 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0033000 tx=0x7fbbd0004730 comp rx=0 tx=0).stop 2024-09-17T04:27:30.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.415+0000 7fbbe4e00640 1 -- 172.21.15.100:0/2605713698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbe00b2f60 msgr2=0x7fbbe01437c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:30.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.415+0000 7fbbe4e00640 1 --2- 172.21.15.100:0/2605713698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbe00b2f60 0x7fbbe01437c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800c3c0 tx=0x7fbbc800c880 comp rx=0 tx=0).stop 2024-09-17T04:27:30.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.415+0000 7fbbe4e00640 1 -- 172.21.15.100:0/2605713698 shutdown_connections 2024-09-17T04:27:30.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.415+0000 7fbbe4e00640 1 --2- 172.21.15.100:0/2605713698 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbbbc078d70 0x7fbbbc07b280 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:30.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.415+0000 7fbbe4e00640 1 --2- 172.21.15.100:0/2605713698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbe015a0f0 0x7fbbe0158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:30.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.415+0000 7fbbe4e00640 1 --2- 172.21.15.100:0/2605713698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbe00b2f60 0x7fbbe01437c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:30.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.415+0000 7fbbe4e00640 1 --2- 172.21.15.100:0/2605713698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbe00ad9c0 0x7fbbe00ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:30.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.415+0000 7fbbe4e00640 1 -- 172.21.15.100:0/2605713698 >> 172.21.15.100:0/2605713698 conn(0x7fbbe00b1e40 msgr2=0x7fbbe00b4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:30.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.415+0000 7fbbe4e00640 1 -- 172.21.15.100:0/2605713698 shutdown_connections 2024-09-17T04:27:30.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.416+0000 7fbbe4e00640 1 -- 172.21.15.100:0/2605713698 wait complete. 2024-09-17T04:27:30.438 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T04:27:30.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.661+0000 7f0b24a00640 1 -- 172.21.15.100:0/2852429284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b04004800 msgr2=0x7f0b04004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:30.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.661+0000 7f0b24a00640 1 --2- 172.21.15.100:0/2852429284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b04004800 0x7f0b04004c10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0b20071300 tx=0x7f0b04005830 comp rx=0 tx=0).stop 2024-09-17T04:27:30.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.661+0000 7f0b24a00640 1 -- 172.21.15.100:0/2852429284 shutdown_connections 2024-09-17T04:27:30.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.661+0000 7f0b24a00640 1 --2- 172.21.15.100:0/2852429284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b04004800 0x7f0b04004c10 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:30.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.661+0000 7f0b24a00640 1 -- 172.21.15.100:0/2852429284 >> 172.21.15.100:0/2852429284 conn(0x7f0b2006d3d0 msgr2=0x7f0b200b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:30.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.661+0000 7f0b24a00640 1 -- 172.21.15.100:0/2852429284 shutdown_connections 2024-09-17T04:27:30.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.661+0000 7f0b24a00640 1 -- 172.21.15.100:0/2852429284 wait complete. 2024-09-17T04:27:30.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.663+0000 7f0b24a00640 1 Processor -- start 2024-09-17T04:27:30.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.663+0000 7f0b24a00640 1 -- start start 2024-09-17T04:27:30.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.665+0000 7f0b24a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b04004800 0x7f0b200994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:30.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.665+0000 7f0b24a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b20154270 0x7f0b201f9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:30.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.665+0000 7f0b24a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b201547c0 0x7f0b201f9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:30.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.665+0000 7f0b24a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2006f0d0 con 0x7f0b04004800 2024-09-17T04:27:30.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.665+0000 7f0b24a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0b20070830 con 0x7f0b20154270 2024-09-17T04:27:30.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.665+0000 7f0b24a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0b2006f250 con 0x7f0b201547c0 2024-09-17T04:27:30.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.665+0000 7f0b1de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b04004800 0x7f0b200994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:30.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.665+0000 7f0b1de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b04004800 0x7f0b200994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57886/0 (socket says 172.21.15.100:57886) 2024-09-17T04:27:30.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.665+0000 7f0b1de00640 1 -- 172.21.15.100:0/2044297396 learned_addr learned my addr 172.21.15.100:0/2044297396 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:30.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.665+0000 7f0b15400640 1 --2- 172.21.15.100:0/2044297396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b20154270 0x7f0b201f9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:30.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.665+0000 7f0b1e800640 1 --2- 172.21.15.100:0/2044297396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b201547c0 0x7f0b201f9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:30.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.665+0000 7f0b1de00640 1 -- 172.21.15.100:0/2044297396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b201547c0 msgr2=0x7f0b201f9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:30.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.665+0000 7f0b1de00640 1 --2- 172.21.15.100:0/2044297396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b201547c0 0x7f0b201f9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:30.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.665+0000 7f0b1de00640 1 -- 172.21.15.100:0/2044297396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b20154270 msgr2=0x7f0b201f9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:30.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.665+0000 7f0b1de00640 1 --2- 172.21.15.100:0/2044297396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b20154270 0x7f0b201f9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:30.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.665+0000 7f0b1de00640 1 -- 172.21.15.100:0/2044297396 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2009a370 con 0x7f0b04004800 2024-09-17T04:27:30.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.665+0000 7f0b1e800640 1 --2- 172.21.15.100:0/2044297396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b201547c0 0x7f0b201f9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:27:30.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.666+0000 7f0b1de00640 1 --2- 172.21.15.100:0/2044297396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b04004800 0x7f0b200994c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0b10006730 tx=0x7f0b100049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:30.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.666+0000 7f0b16a00640 1 -- 172.21.15.100:0/2044297396 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10015020 con 0x7f0b04004800 2024-09-17T04:27:30.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.666+0000 7f0b16a00640 1 -- 172.21.15.100:0/2044297396 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b10011960 con 0x7f0b04004800 2024-09-17T04:27:30.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.666+0000 7f0b24a00640 1 -- 172.21.15.100:0/2044297396 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b201f9df0 con 0x7f0b04004800 2024-09-17T04:27:30.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.666+0000 7f0b16a00640 1 -- 172.21.15.100:0/2044297396 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10011db0 con 0x7f0b04004800 2024-09-17T04:27:30.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.666+0000 7f0b24a00640 1 -- 172.21.15.100:0/2044297396 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b2015a260 con 0x7f0b04004800 2024-09-17T04:27:30.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.667+0000 7f0b24a00640 1 -- 172.21.15.100:0/2044297396 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b20076850 con 0x7f0b04004800 2024-09-17T04:27:30.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.668+0000 7f0b16a00640 1 -- 172.21.15.100:0/2044297396 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f0b10018020 con 0x7f0b04004800 2024-09-17T04:27:30.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.669+0000 7f0b16a00640 1 --2- 172.21.15.100:0/2044297396 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0afc078d40 0x7f0afc07b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:30.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.669+0000 7f0b16a00640 1 -- 172.21.15.100:0/2044297396 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f0b1009bba0 con 0x7f0b04004800 2024-09-17T04:27:30.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.670+0000 7f0b15400640 1 --2- 172.21.15.100:0/2044297396 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0afc078d40 0x7f0afc07b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:30.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.670+0000 7f0b15400640 1 --2- 172.21.15.100:0/2044297396 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0afc078d40 0x7f0afc07b250 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0b08035000 tx=0x7f0b080046c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:30.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.674+0000 7f0b16a00640 1 -- 172.21.15.100:0/2044297396 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0b20076850 con 0x7f0b04004800 2024-09-17T04:27:30.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.892+0000 7f0b24a00640 1 -- 172.21.15.100:0/2044297396 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0b200b3550 con 0x7f0b04004800 2024-09-17T04:27:30.892 INFO:tasks.ceph.mon.a.smithi100.stderr:2024-09-17T04:27:30.892+0000 7fd3d0a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T04:27:30.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.894+0000 7f0b16a00640 1 -- 172.21.15.100:0/2044297396 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7f0b1001f020 con 0x7f0b04004800 2024-09-17T04:27:30.895 INFO:teuthology.orchestra.run.smithi100.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T04:27:30.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.898+0000 7f0b24a00640 1 -- 172.21.15.100:0/2044297396 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0afc078d40 msgr2=0x7f0afc07b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:30.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.898+0000 7f0b24a00640 1 --2- 172.21.15.100:0/2044297396 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0afc078d40 0x7f0afc07b250 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0b08035000 tx=0x7f0b080046c0 comp rx=0 tx=0).stop 2024-09-17T04:27:30.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.898+0000 7f0b24a00640 1 -- 172.21.15.100:0/2044297396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b04004800 msgr2=0x7f0b200994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:30.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.898+0000 7f0b24a00640 1 --2- 172.21.15.100:0/2044297396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b04004800 0x7f0b200994c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0b10006730 tx=0x7f0b100049d0 comp rx=0 tx=0).stop 2024-09-17T04:27:30.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.898+0000 7f0b24a00640 1 -- 172.21.15.100:0/2044297396 shutdown_connections 2024-09-17T04:27:30.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.898+0000 7f0b24a00640 1 --2- 172.21.15.100:0/2044297396 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0afc078d40 0x7f0afc07b250 secure :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0b08035000 tx=0x7f0b080046c0 comp rx=0 tx=0).stop 2024-09-17T04:27:30.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.898+0000 7f0b24a00640 1 --2- 172.21.15.100:0/2044297396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b201547c0 0x7f0b201f9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:30.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.898+0000 7f0b24a00640 1 --2- 172.21.15.100:0/2044297396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b20154270 0x7f0b201f9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:30.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.898+0000 7f0b24a00640 1 --2- 172.21.15.100:0/2044297396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b04004800 0x7f0b200994c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:30.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.898+0000 7f0b24a00640 1 -- 172.21.15.100:0/2044297396 >> 172.21.15.100:0/2044297396 conn(0x7f0b2006d3d0 msgr2=0x7f0b200ad310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:30.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.898+0000 7f0b24a00640 1 -- 172.21.15.100:0/2044297396 shutdown_connections 2024-09-17T04:27:30.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:30.898+0000 7f0b24a00640 1 -- 172.21.15.100:0/2044297396 wait complete. 2024-09-17T04:27:30.924 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T04:27:31.036 INFO:tasks.ceph.mds.c.smithi100.stdout:starting mds.c at 2024-09-17T04:27:31.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.153+0000 7fc97c400640 1 -- 172.21.15.100:0/2856251358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc974145a40 msgr2=0x7fc974073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:31.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.153+0000 7fc97c400640 1 --2- 172.21.15.100:0/2856251358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc974145a40 0x7fc974073540 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc960009830 tx=0x7fc96002a640 comp rx=0 tx=0).stop 2024-09-17T04:27:31.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.153+0000 7fc97c400640 1 -- 172.21.15.100:0/2856251358 shutdown_connections 2024-09-17T04:27:31.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.153+0000 7fc97c400640 1 --2- 172.21.15.100:0/2856251358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc974145a40 0x7fc974073540 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:31.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.153+0000 7fc97c400640 1 -- 172.21.15.100:0/2856251358 >> 172.21.15.100:0/2856251358 conn(0x7fc97406d1c0 msgr2=0x7fc97406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:31.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.153+0000 7fc97c400640 1 -- 172.21.15.100:0/2856251358 shutdown_connections 2024-09-17T04:27:31.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.153+0000 7fc97c400640 1 -- 172.21.15.100:0/2856251358 wait complete. 2024-09-17T04:27:31.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.155+0000 7fc97c400640 1 Processor -- start 2024-09-17T04:27:31.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.155+0000 7fc97c400640 1 -- start start 2024-09-17T04:27:31.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.157+0000 7fc97c400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc974073a80 0x7fc9741f6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:31.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.157+0000 7fc97c400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc974145550 0x7fc974158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:31.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.157+0000 7fc97c400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc974145a40 0x7fc97415aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:31.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.157+0000 7fc97c400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc9740ce350 con 0x7fc974145a40 2024-09-17T04:27:31.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.157+0000 7fc97c400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc9740ce1d0 con 0x7fc974073a80 2024-09-17T04:27:31.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.157+0000 7fc97c400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc9740ce4d0 con 0x7fc974145550 2024-09-17T04:27:31.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.157+0000 7fc97a400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc974145a40 0x7fc97415aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:31.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.157+0000 7fc979a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc974073a80 0x7fc9741f6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:31.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.157+0000 7fc97a400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc974145a40 0x7fc97415aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57922/0 (socket says 172.21.15.100:57922) 2024-09-17T04:27:31.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.157+0000 7fc97a400640 1 -- 172.21.15.100:0/2930832498 learned_addr learned my addr 172.21.15.100:0/2930832498 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:31.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.157+0000 7fc979000640 1 --2- 172.21.15.100:0/2930832498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc974145550 0x7fc974158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:31.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.158+0000 7fc97a400640 1 -- 172.21.15.100:0/2930832498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc974145550 msgr2=0x7fc974158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:31.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.158+0000 7fc97a400640 1 --2- 172.21.15.100:0/2930832498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc974145550 0x7fc974158cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:31.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.158+0000 7fc97a400640 1 -- 172.21.15.100:0/2930832498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc974073a80 msgr2=0x7fc9741f6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:31.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.158+0000 7fc97a400640 1 --2- 172.21.15.100:0/2930832498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc974073a80 0x7fc9741f6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:31.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.158+0000 7fc97a400640 1 -- 172.21.15.100:0/2930832498 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97413ab80 con 0x7fc974145a40 2024-09-17T04:27:31.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.158+0000 7fc97a400640 1 --2- 172.21.15.100:0/2930832498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc974145a40 0x7fc97415aff0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc96c00ca70 tx=0x7fc96c00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:31.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.158+0000 7fc971600640 1 -- 172.21.15.100:0/2930832498 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c016020 con 0x7fc974145a40 2024-09-17T04:27:31.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.158+0000 7fc971600640 1 -- 172.21.15.100:0/2930832498 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc96c00acc0 con 0x7fc974145a40 2024-09-17T04:27:31.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.158+0000 7fc97c400640 1 -- 172.21.15.100:0/2930832498 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9740b2fb0 con 0x7fc974145a40 2024-09-17T04:27:31.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.158+0000 7fc971600640 1 -- 172.21.15.100:0/2930832498 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c012680 con 0x7fc974145a40 2024-09-17T04:27:31.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.158+0000 7fc97c400640 1 -- 172.21.15.100:0/2930832498 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9740b1970 con 0x7fc974145a40 2024-09-17T04:27:31.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.159+0000 7fc97c400640 1 -- 172.21.15.100:0/2930832498 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc974073540 con 0x7fc974145a40 2024-09-17T04:27:31.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.161+0000 7fc971600640 1 -- 172.21.15.100:0/2930832498 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7fc96c019020 con 0x7fc974145a40 2024-09-17T04:27:31.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.162+0000 7fc971600640 1 --2- 172.21.15.100:0/2930832498 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc954078d70 0x7fc95407b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:31.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.162+0000 7fc971600640 1 -- 172.21.15.100:0/2930832498 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5853+0+0 (secure 0 0 0) 0x7fc96c09b390 con 0x7fc974145a40 2024-09-17T04:27:31.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.162+0000 7fc979a00640 1 --2- 172.21.15.100:0/2930832498 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc954078d70 0x7fc95407b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:31.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.163+0000 7fc979a00640 1 --2- 172.21.15.100:0/2930832498 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc954078d70 0x7fc95407b280 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc968000c00 tx=0x7fc968009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:31.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.166+0000 7fc971600640 1 -- 172.21.15.100:0/2930832498 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc96c020030 con 0x7fc974145a40 2024-09-17T04:27:31.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.387+0000 7fc97c400640 1 -- 172.21.15.100:0/2930832498 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc974141280 con 0x7fc974145a40 2024-09-17T04:27:31.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.896+0000 7fc971600640 1 -- 172.21.15.100:0/2930832498 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7fc974073e90 con 0x7fc974145a40 2024-09-17T04:27:31.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.899+0000 7fc97c400640 1 -- 172.21.15.100:0/2930832498 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc954078d70 msgr2=0x7fc95407b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:31.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.899+0000 7fc97c400640 1 --2- 172.21.15.100:0/2930832498 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc954078d70 0x7fc95407b280 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc968000c00 tx=0x7fc968009040 comp rx=0 tx=0).stop 2024-09-17T04:27:31.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.899+0000 7fc97c400640 1 -- 172.21.15.100:0/2930832498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc974145a40 msgr2=0x7fc97415aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:31.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.899+0000 7fc97c400640 1 --2- 172.21.15.100:0/2930832498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc974145a40 0x7fc97415aff0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc96c00ca70 tx=0x7fc96c00cf30 comp rx=0 tx=0).stop 2024-09-17T04:27:31.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.899+0000 7fc97c400640 1 -- 172.21.15.100:0/2930832498 shutdown_connections 2024-09-17T04:27:31.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.899+0000 7fc97c400640 1 --2- 172.21.15.100:0/2930832498 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc954078d70 0x7fc95407b280 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:31.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.899+0000 7fc97c400640 1 --2- 172.21.15.100:0/2930832498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc974145a40 0x7fc97415aff0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:31.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.899+0000 7fc97c400640 1 --2- 172.21.15.100:0/2930832498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc974145550 0x7fc974158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:31.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.899+0000 7fc97c400640 1 --2- 172.21.15.100:0/2930832498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc974073a80 0x7fc9741f6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:31.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.899+0000 7fc97c400640 1 -- 172.21.15.100:0/2930832498 >> 172.21.15.100:0/2930832498 conn(0x7fc97406d1c0 msgr2=0x7fc974143620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:31.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.900+0000 7fc97c400640 1 -- 172.21.15.100:0/2930832498 shutdown_connections 2024-09-17T04:27:31.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:31.900+0000 7fc97c400640 1 -- 172.21.15.100:0/2930832498 wait complete. 2024-09-17T04:27:31.926 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T04:27:32.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.153+0000 7fb29e800640 1 -- 172.21.15.100:0/2343976805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb298162ff0 msgr2=0x7fb298158e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:32.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.153+0000 7fb29e800640 1 --2- 172.21.15.100:0/2343976805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb298162ff0 0x7fb298158e70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb2980710a0 tx=0x7fb28002a560 comp rx=0 tx=0).stop 2024-09-17T04:27:32.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.153+0000 7fb29e800640 1 -- 172.21.15.100:0/2343976805 shutdown_connections 2024-09-17T04:27:32.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.153+0000 7fb29e800640 1 --2- 172.21.15.100:0/2343976805 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb29806cda0 0x7fb29806d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:32.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.153+0000 7fb29e800640 1 --2- 172.21.15.100:0/2343976805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb298162ff0 0x7fb298158e70 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:32.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.153+0000 7fb29e800640 1 --2- 172.21.15.100:0/2343976805 >> v2:172.21.15.184:3300/0 conn(0x7fb29815fd30 0x7fb2980b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:32.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.153+0000 7fb29e800640 1 -- 172.21.15.100:0/2343976805 >> 172.21.15.100:0/2343976805 conn(0x7fb2980b2540 msgr2=0x7fb2980d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:32.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.153+0000 7fb29e800640 1 -- 172.21.15.100:0/2343976805 shutdown_connections 2024-09-17T04:27:32.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.153+0000 7fb29e800640 1 -- 172.21.15.100:0/2343976805 wait complete. 2024-09-17T04:27:32.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.155+0000 7fb29e800640 1 Processor -- start 2024-09-17T04:27:32.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.156+0000 7fb29e800640 1 -- start start 2024-09-17T04:27:32.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.157+0000 7fb29e800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb29806cda0 0x7fb2981f84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:32.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.157+0000 7fb29e800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb29815fd30 0x7fb2981f8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:32.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.157+0000 7fb29e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb298162ff0 0x7fb2981f9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:32.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.157+0000 7fb29e800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb2980af280 con 0x7fb298162ff0 2024-09-17T04:27:32.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.157+0000 7fb29e800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb2980af100 con 0x7fb29806cda0 2024-09-17T04:27:32.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.157+0000 7fb29e800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb2980af400 con 0x7fb29815fd30 2024-09-17T04:27:32.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.157+0000 7fb297e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb298162ff0 0x7fb2981f9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:32.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.157+0000 7fb297e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb298162ff0 0x7fb2981f9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57942/0 (socket says 172.21.15.100:57942) 2024-09-17T04:27:32.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.157+0000 7fb297e00640 1 -- 172.21.15.100:0/2638574972 learned_addr learned my addr 172.21.15.100:0/2638574972 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:32.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.157+0000 7fb297400640 1 --2- 172.21.15.100:0/2638574972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb29806cda0 0x7fb2981f84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:32.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.157+0000 7fb28ea00640 1 --2- 172.21.15.100:0/2638574972 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb29815fd30 0x7fb2981f8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:32.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.158+0000 7fb297e00640 1 -- 172.21.15.100:0/2638574972 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb29815fd30 msgr2=0x7fb2981f8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:32.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.158+0000 7fb297e00640 1 --2- 172.21.15.100:0/2638574972 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb29815fd30 0x7fb2981f8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:32.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.158+0000 7fb297e00640 1 -- 172.21.15.100:0/2638574972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb29806cda0 msgr2=0x7fb2981f84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:32.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.158+0000 7fb297e00640 1 --2- 172.21.15.100:0/2638574972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb29806cda0 0x7fb2981f84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:32.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.158+0000 7fb297e00640 1 -- 172.21.15.100:0/2638574972 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2981f9640 con 0x7fb298162ff0 2024-09-17T04:27:32.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.158+0000 7fb297e00640 1 --2- 172.21.15.100:0/2638574972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb298162ff0 0x7fb2981f9100 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb29000d980 tx=0x7fb29000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:32.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.158+0000 7fb28fe00640 1 -- 172.21.15.100:0/2638574972 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290016070 con 0x7fb298162ff0 2024-09-17T04:27:32.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.158+0000 7fb28fe00640 1 -- 172.21.15.100:0/2638574972 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb29000bd00 con 0x7fb298162ff0 2024-09-17T04:27:32.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.158+0000 7fb29e800640 1 -- 172.21.15.100:0/2638574972 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2981f9ab0 con 0x7fb298162ff0 2024-09-17T04:27:32.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.158+0000 7fb28fe00640 1 -- 172.21.15.100:0/2638574972 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2900106c0 con 0x7fb298162ff0 2024-09-17T04:27:32.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.158+0000 7fb29e800640 1 -- 172.21.15.100:0/2638574972 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb298073180 con 0x7fb298162ff0 2024-09-17T04:27:32.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.159+0000 7fb29e800640 1 -- 172.21.15.100:0/2638574972 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2981f9810 con 0x7fb298162ff0 2024-09-17T04:27:32.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.161+0000 7fb28fe00640 1 -- 172.21.15.100:0/2638574972 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7fb290019070 con 0x7fb298162ff0 2024-09-17T04:27:32.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.162+0000 7fb28fe00640 1 --2- 172.21.15.100:0/2638574972 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb274078d40 0x7fb27407b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:32.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.162+0000 7fb28fe00640 1 -- 172.21.15.100:0/2638574972 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5853+0+0 (secure 0 0 0) 0x7fb29009b7e0 con 0x7fb298162ff0 2024-09-17T04:27:32.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.162+0000 7fb297400640 1 --2- 172.21.15.100:0/2638574972 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb274078d40 0x7fb27407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:32.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.163+0000 7fb297400640 1 --2- 172.21.15.100:0/2638574972 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb274078d40 0x7fb27407b250 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb288009b30 tx=0x7fb288005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:32.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.166+0000 7fb28fe00640 1 -- 172.21.15.100:0/2638574972 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb2981f9810 con 0x7fb298162ff0 2024-09-17T04:27:32.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:32.348+0000 7fb29e800640 1 -- 172.21.15.100:0/2638574972 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb2980af960 con 0x7fb298162ff0 2024-09-17T04:27:33.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.130+0000 7fb28fe00640 1 -- 172.21.15.100:0/2638574972 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v52) ==== 181+0+0 (secure 0 0 0) 0x7fb2900631c0 con 0x7fb298162ff0 2024-09-17T04:27:33.131 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' removed 2024-09-17T04:27:33.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.134+0000 7fb29e800640 1 -- 172.21.15.100:0/2638574972 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb274078d40 msgr2=0x7fb27407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:33.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.134+0000 7fb29e800640 1 --2- 172.21.15.100:0/2638574972 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb274078d40 0x7fb27407b250 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb288009b30 tx=0x7fb288005f90 comp rx=0 tx=0).stop 2024-09-17T04:27:33.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.134+0000 7fb29e800640 1 -- 172.21.15.100:0/2638574972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb298162ff0 msgr2=0x7fb2981f9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:33.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.134+0000 7fb29e800640 1 --2- 172.21.15.100:0/2638574972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb298162ff0 0x7fb2981f9100 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb29000d980 tx=0x7fb29000de40 comp rx=0 tx=0).stop 2024-09-17T04:27:33.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.134+0000 7fb29e800640 1 -- 172.21.15.100:0/2638574972 shutdown_connections 2024-09-17T04:27:33.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.134+0000 7fb29e800640 1 --2- 172.21.15.100:0/2638574972 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb274078d40 0x7fb27407b250 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:33.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.134+0000 7fb29e800640 1 --2- 172.21.15.100:0/2638574972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb298162ff0 0x7fb2981f9100 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:33.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.134+0000 7fb29e800640 1 --2- 172.21.15.100:0/2638574972 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb29815fd30 0x7fb2981f8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:33.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.134+0000 7fb29e800640 1 --2- 172.21.15.100:0/2638574972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb29806cda0 0x7fb2981f84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:33.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.134+0000 7fb29e800640 1 -- 172.21.15.100:0/2638574972 >> 172.21.15.100:0/2638574972 conn(0x7fb2980b2540 msgr2=0x7fb29806f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:33.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.134+0000 7fb29e800640 1 -- 172.21.15.100:0/2638574972 shutdown_connections 2024-09-17T04:27:33.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.135+0000 7fb29e800640 1 -- 172.21.15.100:0/2638574972 wait complete. 2024-09-17T04:27:33.159 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T04:27:33.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.384+0000 7f54e1800640 1 -- 172.21.15.100:0/1450792413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54c0004d40 msgr2=0x7f54c00051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:33.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.384+0000 7f54e1800640 1 --2- 172.21.15.100:0/1450792413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54c0004d40 0x7f54c00051f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f54d4001d90 tx=0x7f54c0005830 comp rx=0 tx=0).stop 2024-09-17T04:27:33.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.384+0000 7f54db400640 1 -- 172.21.15.100:0/1450792413 reap_dead start 2024-09-17T04:27:33.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.384+0000 7f54e1800640 1 -- 172.21.15.100:0/1450792413 shutdown_connections 2024-09-17T04:27:33.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.384+0000 7f54e1800640 1 -- 172.21.15.100:0/1450792413 >> 172.21.15.100:0/1450792413 conn(0x7f54dc0b2540 msgr2=0x7f54dc0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:33.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.384+0000 7f54e1800640 1 -- 172.21.15.100:0/1450792413 shutdown_connections 2024-09-17T04:27:33.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.384+0000 7f54e1800640 1 -- 172.21.15.100:0/1450792413 wait complete. 2024-09-17T04:27:33.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.386+0000 7f54e1800640 1 Processor -- start 2024-09-17T04:27:33.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.386+0000 7f54e1800640 1 -- start start 2024-09-17T04:27:33.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.388+0000 7f54e1800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54c0004d40 0x7f54dc1f3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:33.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.388+0000 7f54e1800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54dc1f3890 0x7f54dc1f70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:33.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.388+0000 7f54e1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54dc1f75e0 0x7f54dc1f79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:33.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.388+0000 7f54e1800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f54dc0b32a0 con 0x7f54dc1f75e0 2024-09-17T04:27:33.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.388+0000 7f54e1800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f54dc0b3120 con 0x7f54dc1f3890 2024-09-17T04:27:33.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.388+0000 7f54e1800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f54dc0b3420 con 0x7f54c0004d40 2024-09-17T04:27:33.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.388+0000 7f54db400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54dc1f75e0 0x7f54dc1f79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:33.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.388+0000 7f54db400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54dc1f75e0 0x7f54dc1f79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57966/0 (socket says 172.21.15.100:57966) 2024-09-17T04:27:33.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.388+0000 7f54db400640 1 -- 172.21.15.100:0/290639169 learned_addr learned my addr 172.21.15.100:0/290639169 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:33.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.388+0000 7f54daa00640 1 --2- 172.21.15.100:0/290639169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54c0004d40 0x7f54dc1f3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:33.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.388+0000 7f54d2000640 1 --2- 172.21.15.100:0/290639169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54dc1f3890 0x7f54dc1f70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:33.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.389+0000 7f54db400640 1 -- 172.21.15.100:0/290639169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54c0004d40 msgr2=0x7f54dc1f3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:33.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.389+0000 7f54db400640 1 --2- 172.21.15.100:0/290639169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54c0004d40 0x7f54dc1f3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:33.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.389+0000 7f54db400640 1 -- 172.21.15.100:0/290639169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54dc1f3890 msgr2=0x7f54dc1f70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:33.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.389+0000 7f54db400640 1 --2- 172.21.15.100:0/290639169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54dc1f3890 0x7f54dc1f70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:33.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.389+0000 7f54db400640 1 -- 172.21.15.100:0/290639169 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54dc09c4d0 con 0x7f54dc1f75e0 2024-09-17T04:27:33.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.389+0000 7f54db400640 1 --2- 172.21.15.100:0/290639169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54dc1f75e0 0x7f54dc1f79f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f54d402ac80 tx=0x7f54d4001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:33.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.389+0000 7f54d3400640 1 -- 172.21.15.100:0/290639169 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d40060b0 con 0x7f54dc1f75e0 2024-09-17T04:27:33.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.389+0000 7f54d3400640 1 -- 172.21.15.100:0/290639169 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54d4006250 con 0x7f54dc1f75e0 2024-09-17T04:27:33.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.389+0000 7f54d3400640 1 -- 172.21.15.100:0/290639169 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d4006660 con 0x7f54dc1f75e0 2024-09-17T04:27:33.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.389+0000 7f54e1800640 1 -- 172.21.15.100:0/290639169 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54dc09c7b0 con 0x7f54dc1f75e0 2024-09-17T04:27:33.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.389+0000 7f54e1800640 1 -- 172.21.15.100:0/290639169 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54dc09cd70 con 0x7f54dc1f75e0 2024-09-17T04:27:33.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.390+0000 7f54e1800640 1 -- 172.21.15.100:0/290639169 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54dc0b3980 con 0x7f54dc1f75e0 2024-09-17T04:27:33.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.391+0000 7f54d3400640 1 -- 172.21.15.100:0/290639169 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f54d4033480 con 0x7f54dc1f75e0 2024-09-17T04:27:33.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.392+0000 7f54d3400640 1 --2- 172.21.15.100:0/290639169 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f54b8078d40 0x7f54b807b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:33.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.392+0000 7f54d3400640 1 -- 172.21.15.100:0/290639169 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5415+0+0 (secure 0 0 0) 0x7f54d40bab80 con 0x7f54dc1f75e0 2024-09-17T04:27:33.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.393+0000 7f54daa00640 1 --2- 172.21.15.100:0/290639169 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f54b8078d40 0x7f54b807b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:33.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.394+0000 7f54daa00640 1 --2- 172.21.15.100:0/290639169 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f54b8078d40 0x7f54b807b250 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f54cc009c50 tx=0x7f54cc005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:33.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.397+0000 7f54d3400640 1 -- 172.21.15.100:0/290639169 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f54d40826f0 con 0x7f54dc1f75e0 2024-09-17T04:27:33.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:33.580+0000 7f54e1800640 1 -- 172.21.15.100:0/290639169 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f54dc0d1030 con 0x7f54dc1f75e0 2024-09-17T04:27:34.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.138+0000 7f54d3400640 1 -- 172.21.15.100:0/290639169 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v53) ==== 169+0+0 (secure 0 0 0) 0x7f54d4087120 con 0x7f54dc1f75e0 2024-09-17T04:27:34.139 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' removed 2024-09-17T04:27:34.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.142+0000 7f54e1800640 1 -- 172.21.15.100:0/290639169 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f54b8078d40 msgr2=0x7f54b807b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:34.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.142+0000 7f54e1800640 1 --2- 172.21.15.100:0/290639169 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f54b8078d40 0x7f54b807b250 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f54cc009c50 tx=0x7f54cc005f90 comp rx=0 tx=0).stop 2024-09-17T04:27:34.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.142+0000 7f54e1800640 1 -- 172.21.15.100:0/290639169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54dc1f75e0 msgr2=0x7f54dc1f79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:34.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.142+0000 7f54e1800640 1 --2- 172.21.15.100:0/290639169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54dc1f75e0 0x7f54dc1f79f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f54d402ac80 tx=0x7f54d4001b10 comp rx=0 tx=0).stop 2024-09-17T04:27:34.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.142+0000 7f54e1800640 1 -- 172.21.15.100:0/290639169 shutdown_connections 2024-09-17T04:27:34.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.142+0000 7f54e1800640 1 --2- 172.21.15.100:0/290639169 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f54b8078d40 0x7f54b807b250 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:34.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.142+0000 7f54e1800640 1 --2- 172.21.15.100:0/290639169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54dc1f75e0 0x7f54dc1f79f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:34.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.142+0000 7f54e1800640 1 --2- 172.21.15.100:0/290639169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54dc1f3890 0x7f54dc1f70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:34.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.142+0000 7f54e1800640 1 --2- 172.21.15.100:0/290639169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54c0004d40 0x7f54dc1f3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:34.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.142+0000 7f54e1800640 1 -- 172.21.15.100:0/290639169 >> 172.21.15.100:0/290639169 conn(0x7f54dc0b2540 msgr2=0x7f54dc06f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:34.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.143+0000 7f54e1800640 1 -- 172.21.15.100:0/290639169 shutdown_connections 2024-09-17T04:27:34.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.143+0000 7f54e1800640 1 -- 172.21.15.100:0/290639169 wait complete. 2024-09-17T04:27:34.171 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T04:27:34.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.391+0000 7fb9d1600640 1 -- 172.21.15.100:0/2246695208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9cc15ce30 msgr2=0x7fb9cc15d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:34.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.391+0000 7fb9d1600640 1 --2- 172.21.15.100:0/2246695208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9cc15ce30 0x7fb9cc15d240 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc06dd90 tx=0x7fb9c402a980 comp rx=0 tx=0).stop 2024-09-17T04:27:34.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.391+0000 7fb9d1600640 1 -- 172.21.15.100:0/2246695208 shutdown_connections 2024-09-17T04:27:34.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.391+0000 7fb9d1600640 1 --2- 172.21.15.100:0/2246695208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9cc15ce30 0x7fb9cc15d240 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:34.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.391+0000 7fb9d1600640 1 --2- 172.21.15.100:0/2246695208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9cc0705b0 0x7fb9cc0709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:34.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.391+0000 7fb9d1600640 1 --2- 172.21.15.100:0/2246695208 >> v2:172.21.15.184:3300/0 conn(0x7fb9cc0cfc80 0x7fb9cc156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:34.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.391+0000 7fb9d1600640 1 -- 172.21.15.100:0/2246695208 >> 172.21.15.100:0/2246695208 conn(0x7fb9cc06d3d0 msgr2=0x7fb9cc0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:34.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.391+0000 7fb9d1600640 1 -- 172.21.15.100:0/2246695208 shutdown_connections 2024-09-17T04:27:34.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.391+0000 7fb9d1600640 1 -- 172.21.15.100:0/2246695208 wait complete. 2024-09-17T04:27:34.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.394+0000 7fb9d1600640 1 Processor -- start 2024-09-17T04:27:34.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.394+0000 7fb9d1600640 1 -- start start 2024-09-17T04:27:34.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.395+0000 7fb9d1600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9cc0705b0 0x7fb9cc155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:34.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.395+0000 7fb9d1600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9cc0cfc80 0x7fb9cc0ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:34.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.395+0000 7fb9d1600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9cc15ce30 0x7fb9cc0ae750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:34.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.395+0000 7fb9d1600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc0cce40 con 0x7fb9cc0705b0 2024-09-17T04:27:34.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.395+0000 7fb9d1600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc0cccc0 con 0x7fb9cc15ce30 2024-09-17T04:27:34.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.395+0000 7fb9d1600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb9cc0ccfc0 con 0x7fb9cc0cfc80 2024-09-17T04:27:34.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.395+0000 7fb9caa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9cc0705b0 0x7fb9cc155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:34.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.395+0000 7fb9caa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9cc0705b0 0x7fb9cc155e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57984/0 (socket says 172.21.15.100:57984) 2024-09-17T04:27:34.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.395+0000 7fb9caa00640 1 -- 172.21.15.100:0/4206542448 learned_addr learned my addr 172.21.15.100:0/4206542448 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:34.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.396+0000 7fb9c2000640 1 --2- 172.21.15.100:0/4206542448 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9cc0cfc80 0x7fb9cc0ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:34.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.396+0000 7fb9cb400640 1 --2- 172.21.15.100:0/4206542448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9cc15ce30 0x7fb9cc0ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:34.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.396+0000 7fb9caa00640 1 -- 172.21.15.100:0/4206542448 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9cc0cfc80 msgr2=0x7fb9cc0ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:34.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.396+0000 7fb9caa00640 1 --2- 172.21.15.100:0/4206542448 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9cc0cfc80 0x7fb9cc0ae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:34.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.396+0000 7fb9caa00640 1 -- 172.21.15.100:0/4206542448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9cc15ce30 msgr2=0x7fb9cc0ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:34.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.396+0000 7fb9caa00640 1 --2- 172.21.15.100:0/4206542448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9cc15ce30 0x7fb9cc0ae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:34.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.396+0000 7fb9caa00640 1 -- 172.21.15.100:0/4206542448 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9cc0b1470 con 0x7fb9cc0705b0 2024-09-17T04:27:34.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.396+0000 7fb9caa00640 1 --2- 172.21.15.100:0/4206542448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9cc0705b0 0x7fb9cc155e30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc00d700 tx=0x7fb9bc00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:34.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.396+0000 7fb9c3400640 1 -- 172.21.15.100:0/4206542448 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc017070 con 0x7fb9cc0705b0 2024-09-17T04:27:34.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.397+0000 7fb9c3400640 1 -- 172.21.15.100:0/4206542448 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9bc004310 con 0x7fb9cc0705b0 2024-09-17T04:27:34.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.397+0000 7fb9d1600640 1 -- 172.21.15.100:0/4206542448 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9cc0c6aa0 con 0x7fb9cc0705b0 2024-09-17T04:27:34.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.397+0000 7fb9c3400640 1 -- 172.21.15.100:0/4206542448 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc009a80 con 0x7fb9cc0705b0 2024-09-17T04:27:34.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.397+0000 7fb9d1600640 1 -- 172.21.15.100:0/4206542448 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9cc0b1730 con 0x7fb9cc0705b0 2024-09-17T04:27:34.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.398+0000 7fb9d1600640 1 -- 172.21.15.100:0/4206542448 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9cc0cd550 con 0x7fb9cc0705b0 2024-09-17T04:27:34.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.399+0000 7fb9c3400640 1 -- 172.21.15.100:0/4206542448 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7fb9bc009c20 con 0x7fb9cc0705b0 2024-09-17T04:27:34.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.400+0000 7fb9c3400640 1 --2- 172.21.15.100:0/4206542448 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb9a8078d40 0x7fb9a807b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:34.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.400+0000 7fb9c3400640 1 -- 172.21.15.100:0/4206542448 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4943+0+0 (secure 0 0 0) 0x7fb9bc09afd0 con 0x7fb9cc0705b0 2024-09-17T04:27:34.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.400+0000 7fb9c2000640 1 --2- 172.21.15.100:0/4206542448 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb9a8078d40 0x7fb9a807b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:34.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.401+0000 7fb9c2000640 1 --2- 172.21.15.100:0/4206542448 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb9a8078d40 0x7fb9a807b250 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400daa0 tx=0x7fb9b4005d70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:34.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.404+0000 7fb9c3400640 1 -- 172.21.15.100:0/4206542448 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb9bc0632e0 con 0x7fb9cc0705b0 2024-09-17T04:27:34.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.586+0000 7fb9d1600640 1 -- 172.21.15.100:0/4206542448 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fb9cc075390 con 0x7fb9cc0705b0 2024-09-17T04:27:34.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.591+0000 7fb9c3400640 1 -- 172.21.15.100:0/4206542448 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v35) ==== 71+0+0 (secure 0 0 0) 0x7fb9bc062be0 con 0x7fb9cc0705b0 2024-09-17T04:27:34.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.594+0000 7fb9d1600640 1 -- 172.21.15.100:0/4206542448 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb9a8078d40 msgr2=0x7fb9a807b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:34.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.594+0000 7fb9d1600640 1 --2- 172.21.15.100:0/4206542448 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb9a8078d40 0x7fb9a807b250 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400daa0 tx=0x7fb9b4005d70 comp rx=0 tx=0).stop 2024-09-17T04:27:34.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.594+0000 7fb9d1600640 1 -- 172.21.15.100:0/4206542448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9cc0705b0 msgr2=0x7fb9cc155e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:34.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.594+0000 7fb9d1600640 1 --2- 172.21.15.100:0/4206542448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9cc0705b0 0x7fb9cc155e30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc00d700 tx=0x7fb9bc00dbc0 comp rx=0 tx=0).stop 2024-09-17T04:27:34.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.594+0000 7fb9d1600640 1 -- 172.21.15.100:0/4206542448 shutdown_connections 2024-09-17T04:27:34.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.594+0000 7fb9d1600640 1 --2- 172.21.15.100:0/4206542448 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb9a8078d40 0x7fb9a807b250 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:34.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.594+0000 7fb9d1600640 1 --2- 172.21.15.100:0/4206542448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9cc15ce30 0x7fb9cc0ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:34.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.594+0000 7fb9d1600640 1 --2- 172.21.15.100:0/4206542448 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9cc0cfc80 0x7fb9cc0ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:34.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.594+0000 7fb9d1600640 1 --2- 172.21.15.100:0/4206542448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9cc0705b0 0x7fb9cc155e30 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:34.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.594+0000 7fb9d1600640 1 -- 172.21.15.100:0/4206542448 >> 172.21.15.100:0/4206542448 conn(0x7fb9cc06d3d0 msgr2=0x7fb9cc15dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:34.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.594+0000 7fb9d1600640 1 -- 172.21.15.100:0/4206542448 shutdown_connections 2024-09-17T04:27:34.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.594+0000 7fb9d1600640 1 -- 172.21.15.100:0/4206542448 wait complete. 2024-09-17T04:27:34.617 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2024-09-17T04:27:34.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.842+0000 7f88cc000640 1 -- 172.21.15.100:0/3421248022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88c415d760 msgr2=0x7f88c41535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:34.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.842+0000 7f88cc000640 1 --2- 172.21.15.100:0/3421248022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88c415d760 0x7f88c41535c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f88b8005fd0 tx=0x7f88b802a540 comp rx=0 tx=0).stop 2024-09-17T04:27:34.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.842+0000 7f88cc000640 1 -- 172.21.15.100:0/3421248022 shutdown_connections 2024-09-17T04:27:34.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.842+0000 7f88cc000640 1 --2- 172.21.15.100:0/3421248022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88c40b3550 0x7f88c40b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:34.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.842+0000 7f88cc000640 1 --2- 172.21.15.100:0/3421248022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88c415d760 0x7f88c41535c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:34.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.842+0000 7f88cc000640 1 -- 172.21.15.100:0/3421248022 >> 172.21.15.100:0/3421248022 conn(0x7f88c406d730 msgr2=0x7f88c40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:34.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.842+0000 7f88cc000640 1 -- 172.21.15.100:0/3421248022 shutdown_connections 2024-09-17T04:27:34.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.842+0000 7f88cc000640 1 -- 172.21.15.100:0/3421248022 wait complete. 2024-09-17T04:27:34.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.844+0000 7f88cc000640 1 Processor -- start 2024-09-17T04:27:34.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.844+0000 7f88cc000640 1 -- start start 2024-09-17T04:27:34.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.846+0000 7f88cc000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88c40ad0e0 0x7f88c413f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:34.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.846+0000 7f88cc000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88c40b3550 0x7f88c4155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:34.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.846+0000 7f88cc000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88c415d760 0x7f88c41f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:34.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.846+0000 7f88cc000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f88c406f240 con 0x7f88c415d760 2024-09-17T04:27:34.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.846+0000 7f88cc000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f88c40b4e30 con 0x7f88c40ad0e0 2024-09-17T04:27:34.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.846+0000 7f88cc000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f88c40b3ea0 con 0x7f88c40b3550 2024-09-17T04:27:34.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.846+0000 7f88ca000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88c415d760 0x7f88c41f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:34.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.846+0000 7f88ca000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88c415d760 0x7f88c41f6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48770/0 (socket says 172.21.15.100:48770) 2024-09-17T04:27:34.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.846+0000 7f88ca000640 1 -- 172.21.15.100:0/1515294885 learned_addr learned my addr 172.21.15.100:0/1515294885 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:34.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.846+0000 7f88c9600640 1 --2- 172.21.15.100:0/1515294885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88c40ad0e0 0x7f88c413f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:34.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.846+0000 7f88c8c00640 1 --2- 172.21.15.100:0/1515294885 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88c40b3550 0x7f88c4155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:34.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.847+0000 7f88ca000640 1 -- 172.21.15.100:0/1515294885 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88c40b3550 msgr2=0x7f88c4155960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:34.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.847+0000 7f88ca000640 1 --2- 172.21.15.100:0/1515294885 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88c40b3550 0x7f88c4155960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:34.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.847+0000 7f88ca000640 1 -- 172.21.15.100:0/1515294885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88c40ad0e0 msgr2=0x7f88c413f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:34.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.847+0000 7f88ca000640 1 --2- 172.21.15.100:0/1515294885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88c40ad0e0 0x7f88c413f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:34.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.847+0000 7f88ca000640 1 -- 172.21.15.100:0/1515294885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c4156500 con 0x7f88c415d760 2024-09-17T04:27:34.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.847+0000 7f88ca000640 1 --2- 172.21.15.100:0/1515294885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88c415d760 0x7f88c41f6f20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00e990 tx=0x7f88bc00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:34.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.847+0000 7f88c1600640 1 -- 172.21.15.100:0/1515294885 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc004d20 con 0x7f88c415d760 2024-09-17T04:27:34.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.847+0000 7f88c1600640 1 -- 172.21.15.100:0/1515294885 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88bc00cb00 con 0x7f88c415d760 2024-09-17T04:27:34.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.847+0000 7f88cc000640 1 -- 172.21.15.100:0/1515294885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c41567e0 con 0x7f88c415d760 2024-09-17T04:27:34.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.847+0000 7f88c1600640 1 -- 172.21.15.100:0/1515294885 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc005020 con 0x7f88c415d760 2024-09-17T04:27:34.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.847+0000 7f88cc000640 1 -- 172.21.15.100:0/1515294885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c414aea0 con 0x7f88c415d760 2024-09-17T04:27:34.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.848+0000 7f88cc000640 1 -- 172.21.15.100:0/1515294885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88c4156270 con 0x7f88c415d760 2024-09-17T04:27:34.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.850+0000 7f88c1600640 1 -- 172.21.15.100:0/1515294885 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f88bc0051c0 con 0x7f88c415d760 2024-09-17T04:27:34.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.851+0000 7f88c1600640 1 --2- 172.21.15.100:0/1515294885 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f88a4078d70 0x7f88a407b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:34.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.851+0000 7f88c1600640 1 -- 172.21.15.100:0/1515294885 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4943+0+0 (secure 0 0 0) 0x7f88bc09b720 con 0x7f88c415d760 2024-09-17T04:27:34.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.851+0000 7f88c9600640 1 --2- 172.21.15.100:0/1515294885 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f88a4078d70 0x7f88a407b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:34.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.852+0000 7f88c9600640 1 --2- 172.21.15.100:0/1515294885 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f88a4078d70 0x7f88a407b280 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f88b8033000 tx=0x7f88b8004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:34.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:34.855+0000 7f88c1600640 1 -- 172.21.15.100:0/1515294885 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f88bc016030 con 0x7f88c415d760 2024-09-17T04:27:35.098 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:27:35.102 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:27:35.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.106+0000 7f88cc000640 1 -- 172.21.15.100:0/1515294885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) -- 0x7f88c40b42f0 con 0x7f88c415d760 2024-09-17T04:27:35.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.140+0000 7f88c1600640 1 -- 172.21.15.100:0/1515294885 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v194) ==== 140+0+0 (secure 0 0 0) 0x7f88bc063410 con 0x7f88c415d760 2024-09-17T04:27:35.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.143+0000 7f88cc000640 1 -- 172.21.15.100:0/1515294885 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f88a4078d70 msgr2=0x7f88a407b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:35.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.143+0000 7f88cc000640 1 --2- 172.21.15.100:0/1515294885 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f88a4078d70 0x7f88a407b280 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f88b8033000 tx=0x7f88b8004730 comp rx=0 tx=0).stop 2024-09-17T04:27:35.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.143+0000 7f88cc000640 1 -- 172.21.15.100:0/1515294885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88c415d760 msgr2=0x7f88c41f6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:35.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.143+0000 7f88cc000640 1 --2- 172.21.15.100:0/1515294885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88c415d760 0x7f88c41f6f20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00e990 tx=0x7f88bc00ee50 comp rx=0 tx=0).stop 2024-09-17T04:27:35.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.143+0000 7f88cc000640 1 -- 172.21.15.100:0/1515294885 shutdown_connections 2024-09-17T04:27:35.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.143+0000 7f88cc000640 1 --2- 172.21.15.100:0/1515294885 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f88a4078d70 0x7f88a407b280 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:35.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.143+0000 7f88cc000640 1 --2- 172.21.15.100:0/1515294885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88c415d760 0x7f88c41f6f20 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:35.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.143+0000 7f88cc000640 1 --2- 172.21.15.100:0/1515294885 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88c40b3550 0x7f88c4155960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:35.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.143+0000 7f88cc000640 1 --2- 172.21.15.100:0/1515294885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88c40ad0e0 0x7f88c413f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:35.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.143+0000 7f88cc000640 1 -- 172.21.15.100:0/1515294885 >> 172.21.15.100:0/1515294885 conn(0x7f88c406d730 msgr2=0x7f88c40b18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:35.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.143+0000 7f88cc000640 1 -- 172.21.15.100:0/1515294885 shutdown_connections 2024-09-17T04:27:35.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.143+0000 7f88cc000640 1 -- 172.21.15.100:0/1515294885 wait complete. 2024-09-17T04:27:35.166 INFO:tasks.cephfs_test_runner:test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-09-17T04:27:35.166 INFO:tasks.cephfs_test_runner:Starting test: test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate handcrafted .meta file on legacy subvol root doesn't break the system 2024-09-17T04:27:35.166 INFO:tasks.cephfs_test_runner:test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-09-17T04:27:35.166 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:27:35.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.438+0000 7fb042600640 1 -- 172.21.15.100:0/2052080302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb020003a50 msgr2=0x7fb020003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:35.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.438+0000 7fb042600640 1 --2- 172.21.15.100:0/2052080302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb020003a50 0x7fb020003ec0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00c310 tx=0x7fb020005830 comp rx=0 tx=0).stop 2024-09-17T04:27:35.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.438+0000 7fb042600640 1 -- 172.21.15.100:0/2052080302 shutdown_connections 2024-09-17T04:27:35.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.438+0000 7fb042600640 1 --2- 172.21.15.100:0/2052080302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb020003a50 0x7fb020003ec0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:35.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.438+0000 7fb042600640 1 -- 172.21.15.100:0/2052080302 >> 172.21.15.100:0/2052080302 conn(0x7fb03c06d730 msgr2=0x7fb03c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:35.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.439+0000 7fb042600640 1 -- 172.21.15.100:0/2052080302 shutdown_connections 2024-09-17T04:27:35.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.439+0000 7fb042600640 1 -- 172.21.15.100:0/2052080302 wait complete. 2024-09-17T04:27:35.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.441+0000 7fb042600640 1 Processor -- start 2024-09-17T04:27:35.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.441+0000 7fb042600640 1 -- start start 2024-09-17T04:27:35.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.442+0000 7fb042600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb020003a50 0x7fb03c098b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:35.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.442+0000 7fb042600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb03c099790 0x7fb03c14d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:35.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.442+0000 7fb042600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb03c1457a0 0x7fb03c14d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:35.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.442+0000 7fb042600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c0ad4f0 con 0x7fb03c099790 2024-09-17T04:27:35.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.442+0000 7fb042600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c0b4e30 con 0x7fb03c1457a0 2024-09-17T04:27:35.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.442+0000 7fb042600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb03c0ad670 con 0x7fb020003a50 2024-09-17T04:27:35.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.443+0000 7fb03aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb03c099790 0x7fb03c14d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:35.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.443+0000 7fb03aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb03c099790 0x7fb03c14d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48784/0 (socket says 172.21.15.100:48784) 2024-09-17T04:27:35.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.443+0000 7fb03aa00640 1 -- 172.21.15.100:0/443966187 learned_addr learned my addr 172.21.15.100:0/443966187 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:35.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.443+0000 7fb03b400640 1 --2- 172.21.15.100:0/443966187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb020003a50 0x7fb03c098b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:35.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.443+0000 7fb03be00640 1 --2- 172.21.15.100:0/443966187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb03c1457a0 0x7fb03c14d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:35.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.443+0000 7fb03aa00640 1 -- 172.21.15.100:0/443966187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb020003a50 msgr2=0x7fb03c098b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:35.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.443+0000 7fb03aa00640 1 --2- 172.21.15.100:0/443966187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb020003a50 0x7fb03c098b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:35.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.443+0000 7fb03aa00640 1 -- 172.21.15.100:0/443966187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb03c1457a0 msgr2=0x7fb03c14d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:35.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.443+0000 7fb03aa00640 1 --2- 172.21.15.100:0/443966187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb03c1457a0 0x7fb03c14d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:35.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.443+0000 7fb03aa00640 1 -- 172.21.15.100:0/443966187 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03c099490 con 0x7fb03c099790 2024-09-17T04:27:35.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.443+0000 7fb03be00640 1 --2- 172.21.15.100:0/443966187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb03c1457a0 0x7fb03c14d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:27:35.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.444+0000 7fb03aa00640 1 --2- 172.21.15.100:0/443966187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb03c099790 0x7fb03c14d2d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb028004dc0 tx=0x7fb028001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:35.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.444+0000 7fb033400640 1 -- 172.21.15.100:0/443966187 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb028001d40 con 0x7fb03c099790 2024-09-17T04:27:35.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.444+0000 7fb033400640 1 -- 172.21.15.100:0/443966187 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb028016070 con 0x7fb03c099790 2024-09-17T04:27:35.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.444+0000 7fb033400640 1 -- 172.21.15.100:0/443966187 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb028010b60 con 0x7fb03c099790 2024-09-17T04:27:35.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.444+0000 7fb042600640 1 -- 172.21.15.100:0/443966187 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03c14c510 con 0x7fb03c099790 2024-09-17T04:27:35.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.444+0000 7fb042600640 1 -- 172.21.15.100:0/443966187 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb03c14cad0 con 0x7fb03c099790 2024-09-17T04:27:35.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.445+0000 7fb042600640 1 -- 172.21.15.100:0/443966187 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb03c1535c0 con 0x7fb03c099790 2024-09-17T04:27:35.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.446+0000 7fb033400640 1 -- 172.21.15.100:0/443966187 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7fb028010d00 con 0x7fb03c099790 2024-09-17T04:27:35.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.447+0000 7fb033400640 1 --2- 172.21.15.100:0/443966187 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb01c078d70 0x7fb01c07b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:35.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.447+0000 7fb033400640 1 -- 172.21.15.100:0/443966187 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4943+0+0 (secure 0 0 0) 0x7fb02809b5b0 con 0x7fb03c099790 2024-09-17T04:27:35.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.448+0000 7fb03b400640 1 --2- 172.21.15.100:0/443966187 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb01c078d70 0x7fb01c07b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:35.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.448+0000 7fb03b400640 1 --2- 172.21.15.100:0/443966187 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb01c078d70 0x7fb01c07b280 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb02c033000 tx=0x7fb02c038040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:35.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.452+0000 7fb033400640 1 -- 172.21.15.100:0/443966187 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb0280638c0 con 0x7fb03c099790 2024-09-17T04:27:35.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.632+0000 7fb042600640 1 -- 172.21.15.100:0/443966187 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb03c15e340 con 0x7fb03c099790 2024-09-17T04:27:35.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.633+0000 7fb033400640 1 -- 172.21.15.100:0/443966187 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+13082 (secure 0 0 0) 0x7fb0280631c0 con 0x7fb03c099790 2024-09-17T04:27:35.634 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:27:35.634 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":53,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:27:34.132050+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6835/3801048032":"2024-09-18T04:27:30.893764+0000","172.21.15.100:6834/3801048032":"2024-09-18T04:27:30.893764+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:27:35.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.636+0000 7fb042600640 1 -- 172.21.15.100:0/443966187 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb01c078d70 msgr2=0x7fb01c07b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:35.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.636+0000 7fb042600640 1 --2- 172.21.15.100:0/443966187 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb01c078d70 0x7fb01c07b280 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb02c033000 tx=0x7fb02c038040 comp rx=0 tx=0).stop 2024-09-17T04:27:35.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.636+0000 7fb042600640 1 -- 172.21.15.100:0/443966187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb03c099790 msgr2=0x7fb03c14d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:35.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.636+0000 7fb042600640 1 --2- 172.21.15.100:0/443966187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb03c099790 0x7fb03c14d2d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb028004dc0 tx=0x7fb028001630 comp rx=0 tx=0).stop 2024-09-17T04:27:35.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.636+0000 7fb042600640 1 -- 172.21.15.100:0/443966187 shutdown_connections 2024-09-17T04:27:35.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.636+0000 7fb042600640 1 --2- 172.21.15.100:0/443966187 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb01c078d70 0x7fb01c07b280 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:35.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.636+0000 7fb042600640 1 --2- 172.21.15.100:0/443966187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb03c1457a0 0x7fb03c14d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:35.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.636+0000 7fb042600640 1 --2- 172.21.15.100:0/443966187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb03c099790 0x7fb03c14d2d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:35.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.636+0000 7fb042600640 1 --2- 172.21.15.100:0/443966187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb020003a50 0x7fb03c098b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:35.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.636+0000 7fb042600640 1 -- 172.21.15.100:0/443966187 >> 172.21.15.100:0/443966187 conn(0x7fb03c06d730 msgr2=0x7fb03c0d0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:35.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.636+0000 7fb042600640 1 -- 172.21.15.100:0/443966187 shutdown_connections 2024-09-17T04:27:35.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.636+0000 7fb042600640 1 -- 172.21.15.100:0/443966187 wait complete. 2024-09-17T04:27:35.659 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T04:24:03.995355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T04:27:35.660 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T04:27:35.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.883+0000 7f18d2c00640 1 -- 172.21.15.100:0/2899778630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b4003c10 msgr2=0x7f18b4004080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:35.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.883+0000 7f18d2c00640 1 --2- 172.21.15.100:0/2899778630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b4003c10 0x7f18b4004080 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f18c0001860 tx=0x7f18b4005830 comp rx=0 tx=0).stop 2024-09-17T04:27:35.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.883+0000 7f18d2200640 1 -- 172.21.15.100:0/2899778630 reap_dead start 2024-09-17T04:27:35.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.883+0000 7f18d2c00640 1 -- 172.21.15.100:0/2899778630 shutdown_connections 2024-09-17T04:27:35.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.883+0000 7f18d2c00640 1 -- 172.21.15.100:0/2899778630 >> 172.21.15.100:0/2899778630 conn(0x7f18cc06cf80 msgr2=0x7f18cc06d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:35.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.883+0000 7f18d2c00640 1 -- 172.21.15.100:0/2899778630 shutdown_connections 2024-09-17T04:27:35.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.883+0000 7f18d2c00640 1 -- 172.21.15.100:0/2899778630 wait complete. 2024-09-17T04:27:35.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.885+0000 7f18d2c00640 1 Processor -- start 2024-09-17T04:27:35.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.886+0000 7f18d2c00640 1 -- start start 2024-09-17T04:27:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.887+0000 7f18d2c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b4003c10 0x7f18cc1f5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.887+0000 7f18d2c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18cc137d40 0x7f18cc137160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.887+0000 7f18d2c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18cc1f5b50 0x7f18cc1376a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.887+0000 7f18d2c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc15cc80 con 0x7f18b4003c10 2024-09-17T04:27:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.887+0000 7f18d2c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc0d0a20 con 0x7f18cc1f5b50 2024-09-17T04:27:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.887+0000 7f18d2c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f18cc15ce00 con 0x7f18cc137d40 2024-09-17T04:27:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.887+0000 7f18d1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b4003c10 0x7f18cc1f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.887+0000 7f18d1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b4003c10 0x7f18cc1f5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48812/0 (socket says 172.21.15.100:48812) 2024-09-17T04:27:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.887+0000 7f18d1800640 1 -- 172.21.15.100:0/1511320619 learned_addr learned my addr 172.21.15.100:0/1511320619 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.888+0000 7f18d0e00640 1 --2- 172.21.15.100:0/1511320619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18cc137d40 0x7f18cc137160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.888+0000 7f18d2200640 1 --2- 172.21.15.100:0/1511320619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18cc1f5b50 0x7f18cc1376a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.888+0000 7f18d1800640 1 -- 172.21.15.100:0/1511320619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18cc137d40 msgr2=0x7f18cc137160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:35.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.888+0000 7f18d1800640 1 --2- 172.21.15.100:0/1511320619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18cc137d40 0x7f18cc137160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:35.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.888+0000 7f18d1800640 1 -- 172.21.15.100:0/1511320619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18cc1f5b50 msgr2=0x7f18cc1376a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:35.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.888+0000 7f18d1800640 1 --2- 172.21.15.100:0/1511320619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18cc1f5b50 0x7f18cc1376a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:35.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.888+0000 7f18d1800640 1 -- 172.21.15.100:0/1511320619 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18cc0a9630 con 0x7f18b4003c10 2024-09-17T04:27:35.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.888+0000 7f18d1800640 1 --2- 172.21.15.100:0/1511320619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b4003c10 0x7f18cc1f5610 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f18c0001860 tx=0x7f18c0006060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:35.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.888+0000 7f18c9600640 1 -- 172.21.15.100:0/1511320619 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c0043070 con 0x7f18b4003c10 2024-09-17T04:27:35.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.888+0000 7f18c9600640 1 -- 172.21.15.100:0/1511320619 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18c00038e0 con 0x7f18b4003c10 2024-09-17T04:27:35.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.888+0000 7f18c9600640 1 -- 172.21.15.100:0/1511320619 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c0003cf0 con 0x7f18b4003c10 2024-09-17T04:27:35.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.888+0000 7f18d2c00640 1 -- 172.21.15.100:0/1511320619 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18cc0a9910 con 0x7f18b4003c10 2024-09-17T04:27:35.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.889+0000 7f18d2c00640 1 -- 172.21.15.100:0/1511320619 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18cc1381f0 con 0x7f18b4003c10 2024-09-17T04:27:35.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.890+0000 7f18d2c00640 1 -- 172.21.15.100:0/1511320619 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18cc0cf9a0 con 0x7f18b4003c10 2024-09-17T04:27:35.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.891+0000 7f18c9600640 1 -- 172.21.15.100:0/1511320619 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f18c0033480 con 0x7f18b4003c10 2024-09-17T04:27:35.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.892+0000 7f18c9600640 1 --2- 172.21.15.100:0/1511320619 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f18ac078d40 0x7f18ac07b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:35.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.892+0000 7f18c9600640 1 -- 172.21.15.100:0/1511320619 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4943+0+0 (secure 0 0 0) 0x7f18c00bae20 con 0x7f18b4003c10 2024-09-17T04:27:35.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.892+0000 7f18d0e00640 1 --2- 172.21.15.100:0/1511320619 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f18ac078d40 0x7f18ac07b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:35.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.893+0000 7f18d0e00640 1 --2- 172.21.15.100:0/1511320619 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f18ac078d40 0x7f18ac07b250 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f18b8001410 tx=0x7f18b8001320 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:35.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:35.896+0000 7f18c9600640 1 -- 172.21.15.100:0/1511320619 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f18c0083130 con 0x7f18b4003c10 2024-09-17T04:27:36.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.083+0000 7f18d2c00640 1 -- 172.21.15.100:0/1511320619 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f18cc15ec00 con 0x7f18b4003c10 2024-09-17T04:27:36.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.083+0000 7f18c9600640 1 -- 172.21.15.100:0/1511320619 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) ==== 93+0+10 (secure 0 0 0) 0x7f18c0082a30 con 0x7f18b4003c10 2024-09-17T04:27:36.084 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-09-17T04:27:36.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.086+0000 7f18d2c00640 1 -- 172.21.15.100:0/1511320619 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f18ac078d40 msgr2=0x7f18ac07b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:36.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.086+0000 7f18d2c00640 1 --2- 172.21.15.100:0/1511320619 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f18ac078d40 0x7f18ac07b250 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f18b8001410 tx=0x7f18b8001320 comp rx=0 tx=0).stop 2024-09-17T04:27:36.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.086+0000 7f18d2c00640 1 -- 172.21.15.100:0/1511320619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b4003c10 msgr2=0x7f18cc1f5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:36.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.086+0000 7f18d2c00640 1 --2- 172.21.15.100:0/1511320619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b4003c10 0x7f18cc1f5610 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f18c0001860 tx=0x7f18c0006060 comp rx=0 tx=0).stop 2024-09-17T04:27:36.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.086+0000 7f18d2c00640 1 -- 172.21.15.100:0/1511320619 shutdown_connections 2024-09-17T04:27:36.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.086+0000 7f18d2c00640 1 --2- 172.21.15.100:0/1511320619 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f18ac078d40 0x7f18ac07b250 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:36.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.086+0000 7f18d2c00640 1 --2- 172.21.15.100:0/1511320619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18cc1f5b50 0x7f18cc1376a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:36.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.086+0000 7f18d2c00640 1 --2- 172.21.15.100:0/1511320619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18cc137d40 0x7f18cc137160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:36.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.086+0000 7f18d2c00640 1 --2- 172.21.15.100:0/1511320619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b4003c10 0x7f18cc1f5610 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:36.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.086+0000 7f18d2c00640 1 -- 172.21.15.100:0/1511320619 >> 172.21.15.100:0/1511320619 conn(0x7f18cc06cf80 msgr2=0x7f18cc1568f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:36.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.086+0000 7f18d2c00640 1 -- 172.21.15.100:0/1511320619 shutdown_connections 2024-09-17T04:27:36.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.086+0000 7f18d2c00640 1 -- 172.21.15.100:0/1511320619 wait complete. 2024-09-17T04:27:36.108 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2024-09-17T04:27:36.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.330+0000 7f4e5a800640 1 -- 172.21.15.100:0/1120204197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e540b3550 msgr2=0x7f4e540b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:36.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.330+0000 7f4e5a800640 1 --2- 172.21.15.100:0/1120204197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e540b3550 0x7f4e540b3960 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c005fd0 tx=0x7f4e4c02a820 comp rx=0 tx=0).stop 2024-09-17T04:27:36.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.330+0000 7f4e5a800640 1 -- 172.21.15.100:0/1120204197 shutdown_connections 2024-09-17T04:27:36.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.330+0000 7f4e5a800640 1 --2- 172.21.15.100:0/1120204197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e540b3550 0x7f4e540b3960 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:36.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.330+0000 7f4e5a800640 1 --2- 172.21.15.100:0/1120204197 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e5415d760 0x7f4e541535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:36.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.330+0000 7f4e5a800640 1 -- 172.21.15.100:0/1120204197 >> 172.21.15.100:0/1120204197 conn(0x7f4e5406d730 msgr2=0x7f4e540b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:36.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.330+0000 7f4e5a800640 1 -- 172.21.15.100:0/1120204197 shutdown_connections 2024-09-17T04:27:36.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.330+0000 7f4e5a800640 1 -- 172.21.15.100:0/1120204197 wait complete. 2024-09-17T04:27:36.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.332+0000 7f4e5a800640 1 Processor -- start 2024-09-17T04:27:36.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.333+0000 7f4e5a800640 1 -- start start 2024-09-17T04:27:36.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.334+0000 7f4e5a800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e540ad0e0 0x7f4e540b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:36.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.334+0000 7f4e5a800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e540b3550 0x7f4e54155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:36.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.334+0000 7f4e5a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e5415d760 0x7f4e541563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:36.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.334+0000 7f4e5a800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4e540b4da0 con 0x7f4e5415d760 2024-09-17T04:27:36.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.334+0000 7f4e5a800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4e540b4c20 con 0x7f4e540b3550 2024-09-17T04:27:36.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.334+0000 7f4e5a800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4e540b3ea0 con 0x7f4e540ad0e0 2024-09-17T04:27:36.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.334+0000 7f4e53e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e5415d760 0x7f4e541563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:36.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.334+0000 7f4e53e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e5415d760 0x7f4e541563e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48828/0 (socket says 172.21.15.100:48828) 2024-09-17T04:27:36.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.334+0000 7f4e53e00640 1 -- 172.21.15.100:0/3050257566 learned_addr learned my addr 172.21.15.100:0/3050257566 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:36.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.335+0000 7f4e52a00640 1 --2- 172.21.15.100:0/3050257566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e540b3550 0x7f4e54155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:36.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.335+0000 7f4e53400640 1 --2- 172.21.15.100:0/3050257566 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e540ad0e0 0x7f4e540b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:36.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.335+0000 7f4e53e00640 1 -- 172.21.15.100:0/3050257566 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e540ad0e0 msgr2=0x7f4e540b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:36.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.335+0000 7f4e53e00640 1 --2- 172.21.15.100:0/3050257566 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e540ad0e0 0x7f4e540b13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:36.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.335+0000 7f4e53e00640 1 -- 172.21.15.100:0/3050257566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e540b3550 msgr2=0x7f4e54155ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:36.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.335+0000 7f4e53e00640 1 --2- 172.21.15.100:0/3050257566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e540b3550 0x7f4e54155ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:36.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.335+0000 7f4e53e00640 1 -- 172.21.15.100:0/3050257566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e54151320 con 0x7f4e5415d760 2024-09-17T04:27:36.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.335+0000 7f4e52a00640 1 --2- 172.21.15.100:0/3050257566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e540b3550 0x7f4e54155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:27:36.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.335+0000 7f4e53e00640 1 --2- 172.21.15.100:0/3050257566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e5415d760 0x7f4e541563e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009b30 tx=0x7f4e4c032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:36.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.335+0000 7f4e4b400640 1 -- 172.21.15.100:0/3050257566 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c037040 con 0x7f4e5415d760 2024-09-17T04:27:36.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.335+0000 7f4e4b400640 1 -- 172.21.15.100:0/3050257566 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e4c042070 con 0x7f4e5415d760 2024-09-17T04:27:36.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.335+0000 7f4e4b400640 1 -- 172.21.15.100:0/3050257566 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c003e60 con 0x7f4e5415d760 2024-09-17T04:27:36.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.336+0000 7f4e5a800640 1 -- 172.21.15.100:0/3050257566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e541f6240 con 0x7f4e5415d760 2024-09-17T04:27:36.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.336+0000 7f4e5a800640 1 -- 172.21.15.100:0/3050257566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e541f6800 con 0x7f4e5415d760 2024-09-17T04:27:36.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.337+0000 7f4e5a800640 1 -- 172.21.15.100:0/3050257566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e540b43c0 con 0x7f4e5415d760 2024-09-17T04:27:36.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.338+0000 7f4e4b400640 1 -- 172.21.15.100:0/3050257566 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f4e4c004280 con 0x7f4e5415d760 2024-09-17T04:27:36.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.339+0000 7f4e4b400640 1 --2- 172.21.15.100:0/3050257566 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4e34078d40 0x7f4e3407b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:36.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.339+0000 7f4e4b400640 1 -- 172.21.15.100:0/3050257566 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4943+0+0 (secure 0 0 0) 0x7f4e4c0bbc50 con 0x7f4e5415d760 2024-09-17T04:27:36.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.339+0000 7f4e53400640 1 --2- 172.21.15.100:0/3050257566 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4e34078d40 0x7f4e3407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:36.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.340+0000 7f4e53400640 1 --2- 172.21.15.100:0/3050257566 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4e34078d40 0x7f4e3407b250 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009ac0 tx=0x7f4e44005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:36.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.343+0000 7f4e4b400640 1 -- 172.21.15.100:0/3050257566 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4e4c0c1020 con 0x7f4e5415d760 2024-09-17T04:27:36.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:36.594+0000 7f4e5a800640 1 -- 172.21.15.100:0/3050257566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f4e541f5fd0 con 0x7f4e5415d760 2024-09-17T04:27:37.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.147+0000 7f4e4b400640 1 -- 172.21.15.100:0/3050257566 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v196) ==== 164+0+0 (secure 0 0 0) 0x7f4e4c084090 con 0x7f4e5415d760 2024-09-17T04:27:37.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.150+0000 7f4e5a800640 1 -- 172.21.15.100:0/3050257566 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4e34078d40 msgr2=0x7f4e3407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:37.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.150+0000 7f4e5a800640 1 --2- 172.21.15.100:0/3050257566 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4e34078d40 0x7f4e3407b250 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009ac0 tx=0x7f4e44005f90 comp rx=0 tx=0).stop 2024-09-17T04:27:37.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.150+0000 7f4e5a800640 1 -- 172.21.15.100:0/3050257566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e5415d760 msgr2=0x7f4e541563e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:37.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.150+0000 7f4e5a800640 1 --2- 172.21.15.100:0/3050257566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e5415d760 0x7f4e541563e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009b30 tx=0x7f4e4c032d10 comp rx=0 tx=0).stop 2024-09-17T04:27:37.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.150+0000 7f4e5a800640 1 -- 172.21.15.100:0/3050257566 shutdown_connections 2024-09-17T04:27:37.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.150+0000 7f4e5a800640 1 --2- 172.21.15.100:0/3050257566 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4e34078d40 0x7f4e3407b250 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:37.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.150+0000 7f4e5a800640 1 --2- 172.21.15.100:0/3050257566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e5415d760 0x7f4e541563e0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:37.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.150+0000 7f4e5a800640 1 --2- 172.21.15.100:0/3050257566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e540b3550 0x7f4e54155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:37.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.150+0000 7f4e5a800640 1 --2- 172.21.15.100:0/3050257566 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e540ad0e0 0x7f4e540b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:37.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.150+0000 7f4e5a800640 1 -- 172.21.15.100:0/3050257566 >> 172.21.15.100:0/3050257566 conn(0x7f4e5406d730 msgr2=0x7f4e5415c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:37.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.151+0000 7f4e5a800640 1 -- 172.21.15.100:0/3050257566 shutdown_connections 2024-09-17T04:27:37.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.151+0000 7f4e5a800640 1 -- 172.21.15.100:0/3050257566 wait complete. 2024-09-17T04:27:37.174 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T04:27:37.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.395+0000 7fd1e3e00640 1 -- 172.21.15.100:0/2400527956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1e40b9e00 msgr2=0x7fd1e4162a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:37.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.395+0000 7fd1e3e00640 1 --2- 172.21.15.100:0/2400527956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1e40b9e00 0x7fd1e4162a50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009830 tx=0x7fd1d402a480 comp rx=0 tx=0).stop 2024-09-17T04:27:37.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.395+0000 7fd1e3e00640 1 -- 172.21.15.100:0/2400527956 shutdown_connections 2024-09-17T04:27:37.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.395+0000 7fd1e3e00640 1 --2- 172.21.15.100:0/2400527956 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e40b3160 0x7fd1e40d24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:37.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.395+0000 7fd1e3e00640 1 --2- 172.21.15.100:0/2400527956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1e40b9e00 0x7fd1e4162a50 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:37.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.395+0000 7fd1e3e00640 1 -- 172.21.15.100:0/2400527956 >> 172.21.15.100:0/2400527956 conn(0x7fd1e408bfe0 msgr2=0x7fd1e409d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:37.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.395+0000 7fd1e3e00640 1 -- 172.21.15.100:0/2400527956 shutdown_connections 2024-09-17T04:27:37.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.395+0000 7fd1e3e00640 1 -- 172.21.15.100:0/2400527956 wait complete. 2024-09-17T04:27:37.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.397+0000 7fd1e3e00640 1 Processor -- start 2024-09-17T04:27:37.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.397+0000 7fd1e3e00640 1 -- start start 2024-09-17T04:27:37.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.398+0000 7fd1e3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1e40b3160 0x7fd1e4158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:37.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.398+0000 7fd1e3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1e40b9e00 0x7fd1e40a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:37.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.398+0000 7fd1e3e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e40d29e0 0x7fd1e40a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:37.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.398+0000 7fd1e3e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e40d44b0 con 0x7fd1e40b3160 2024-09-17T04:27:37.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.398+0000 7fd1e3e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e40d4330 con 0x7fd1e40b9e00 2024-09-17T04:27:37.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.398+0000 7fd1e3e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd1e40d4630 con 0x7fd1e40d29e0 2024-09-17T04:27:37.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.399+0000 7fd1e2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1e40b3160 0x7fd1e4158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:37.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.399+0000 7fd1e2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1e40b3160 0x7fd1e4158d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48860/0 (socket says 172.21.15.100:48860) 2024-09-17T04:27:37.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.399+0000 7fd1e2a00640 1 -- 172.21.15.100:0/2930775272 learned_addr learned my addr 172.21.15.100:0/2930775272 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:37.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.399+0000 7fd1e3400640 1 --2- 172.21.15.100:0/2930775272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e40d29e0 0x7fd1e40a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:37.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.399+0000 7fd1da000640 1 --2- 172.21.15.100:0/2930775272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1e40b9e00 0x7fd1e40a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:37.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.399+0000 7fd1e2a00640 1 -- 172.21.15.100:0/2930775272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e40d29e0 msgr2=0x7fd1e40a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:37.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.399+0000 7fd1e2a00640 1 --2- 172.21.15.100:0/2930775272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e40d29e0 0x7fd1e40a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:37.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.399+0000 7fd1e2a00640 1 -- 172.21.15.100:0/2930775272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1e40b9e00 msgr2=0x7fd1e40a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:37.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.399+0000 7fd1e2a00640 1 --2- 172.21.15.100:0/2930775272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1e40b9e00 0x7fd1e40a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:37.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.399+0000 7fd1e2a00640 1 -- 172.21.15.100:0/2930775272 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e40caf00 con 0x7fd1e40b3160 2024-09-17T04:27:37.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.399+0000 7fd1da000640 1 --2- 172.21.15.100:0/2930775272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1e40b9e00 0x7fd1e40a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:27:37.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.399+0000 7fd1e2a00640 1 --2- 172.21.15.100:0/2930775272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1e40b3160 0x7fd1e4158d00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009950 tx=0x7fd1d4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:37.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.400+0000 7fd1db400640 1 -- 172.21.15.100:0/2930775272 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4031d60 con 0x7fd1e40b3160 2024-09-17T04:27:37.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.400+0000 7fd1db400640 1 -- 172.21.15.100:0/2930775272 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1d4033e10 con 0x7fd1e40b3160 2024-09-17T04:27:37.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.400+0000 7fd1e3e00640 1 -- 172.21.15.100:0/2930775272 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1e40ce520 con 0x7fd1e40b3160 2024-09-17T04:27:37.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.400+0000 7fd1db400640 1 -- 172.21.15.100:0/2930775272 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4005260 con 0x7fd1e40b3160 2024-09-17T04:27:37.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.400+0000 7fd1e3e00640 1 -- 172.21.15.100:0/2930775272 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1e40cb4a0 con 0x7fd1e40b3160 2024-09-17T04:27:37.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.401+0000 7fd1e3e00640 1 -- 172.21.15.100:0/2930775272 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1e40d4b00 con 0x7fd1e40b3160 2024-09-17T04:27:37.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.402+0000 7fd1db400640 1 -- 172.21.15.100:0/2930775272 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7fd1d4005400 con 0x7fd1e40b3160 2024-09-17T04:27:37.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.403+0000 7fd1db400640 1 --2- 172.21.15.100:0/2930775272 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd1c0078d40 0x7fd1c007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:37.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.403+0000 7fd1db400640 1 -- 172.21.15.100:0/2930775272 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4943+0+0 (secure 0 0 0) 0x7fd1d40bc0d0 con 0x7fd1e40b3160 2024-09-17T04:27:37.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.403+0000 7fd1da000640 1 --2- 172.21.15.100:0/2930775272 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd1c0078d40 0x7fd1c007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:37.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.404+0000 7fd1da000640 1 --2- 172.21.15.100:0/2930775272 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd1c0078d40 0x7fd1c007b250 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40b3110 tx=0x7fd1cc005f10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:37.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.408+0000 7fd1db400640 1 -- 172.21.15.100:0/2930775272 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd1d403f030 con 0x7fd1e40b3160 2024-09-17T04:27:37.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.590+0000 7fd1e3e00640 1 -- 172.21.15.100:0/2930775272 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fd1e40b4620 con 0x7fd1e40b3160 2024-09-17T04:27:37.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.590+0000 7fd1db400640 1 -- 172.21.15.100:0/2930775272 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v35) ==== 59+0+2294 (secure 0 0 0) 0x7fd1d4084470 con 0x7fd1e40b3160 2024-09-17T04:27:37.591 INFO:teuthology.orchestra.run.smithi100.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T04:27:37.591 INFO:teuthology.orchestra.run.smithi100.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T04:27:37.592 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced client_mount_timeout 30 2024-09-17T04:27:37.592 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_client 20/20 2024-09-17T04:27:37.592 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_mgr 20/20 2024-09-17T04:27:37.592 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_ms 1/1 2024-09-17T04:27:37.592 INFO:teuthology.orchestra.run.smithi100.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T04:27:37.593 INFO:teuthology.orchestra.run.smithi100.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T04:27:37.593 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3938.598654 2024-09-17T04:27:37.593 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4008.640154 2024-09-17T04:27:37.593 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3706.588546 2024-09-17T04:27:37.593 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 6299.338665 2024-09-17T04:27:37.593 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3633.876699 2024-09-17T04:27:37.593 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3674.821841 2024-09-17T04:27:37.593 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3789.849984 2024-09-17T04:27:37.593 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3467.457298 2024-09-17T04:27:37.593 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_mds 20/20 2024-09-17T04:27:37.593 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T04:27:37.594 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_ms 2/2 2024-09-17T04:27:37.594 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_debug_frag true 2024-09-17T04:27:37.594 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_debug_scatterstat true 2024-09-17T04:27:37.594 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T04:27:37.594 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_verify_scatter true 2024-09-17T04:27:37.594 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T04:27:37.594 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T04:27:37.594 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T04:27:37.594 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced client_mount_timeout 600 2024-09-17T04:27:37.594 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced debug_client 20/20 2024-09-17T04:27:37.594 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced debug_ms 1/1 2024-09-17T04:27:37.594 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T04:27:37.594 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T04:27:37.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.592+0000 7fd1e3e00640 1 -- 172.21.15.100:0/2930775272 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd1c0078d40 msgr2=0x7fd1c007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:37.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.592+0000 7fd1e3e00640 1 --2- 172.21.15.100:0/2930775272 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd1c0078d40 0x7fd1c007b250 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40b3110 tx=0x7fd1cc005f10 comp rx=0 tx=0).stop 2024-09-17T04:27:37.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.592+0000 7fd1e3e00640 1 -- 172.21.15.100:0/2930775272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1e40b3160 msgr2=0x7fd1e4158d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:37.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.592+0000 7fd1e3e00640 1 --2- 172.21.15.100:0/2930775272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1e40b3160 0x7fd1e4158d00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009950 tx=0x7fd1d4033c20 comp rx=0 tx=0).stop 2024-09-17T04:27:37.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.593+0000 7fd1e3e00640 1 -- 172.21.15.100:0/2930775272 shutdown_connections 2024-09-17T04:27:37.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.593+0000 7fd1e3e00640 1 --2- 172.21.15.100:0/2930775272 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd1c0078d40 0x7fd1c007b250 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:37.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.593+0000 7fd1e3e00640 1 --2- 172.21.15.100:0/2930775272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e40d29e0 0x7fd1e40a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:37.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.593+0000 7fd1e3e00640 1 --2- 172.21.15.100:0/2930775272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1e40b9e00 0x7fd1e40a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:37.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.593+0000 7fd1e3e00640 1 --2- 172.21.15.100:0/2930775272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1e40b3160 0x7fd1e4158d00 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:37.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.593+0000 7fd1e3e00640 1 -- 172.21.15.100:0/2930775272 >> 172.21.15.100:0/2930775272 conn(0x7fd1e408bfe0 msgr2=0x7fd1e40b4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:37.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.593+0000 7fd1e3e00640 1 -- 172.21.15.100:0/2930775272 shutdown_connections 2024-09-17T04:27:37.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.593+0000 7fd1e3e00640 1 -- 172.21.15.100:0/2930775272 wait complete. 2024-09-17T04:27:37.617 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T04:27:37.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.840+0000 7f406e000640 1 -- 172.21.15.100:0/4017554777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4068145550 msgr2=0x7f406806fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:37.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.840+0000 7f406e000640 1 --2- 172.21.15.100:0/4017554777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4068145550 0x7f406806fb40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f40681464e0 tx=0x7f405802a5f0 comp rx=0 tx=0).stop 2024-09-17T04:27:37.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.840+0000 7f406e000640 1 -- 172.21.15.100:0/4017554777 shutdown_connections 2024-09-17T04:27:37.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.840+0000 7f406e000640 1 --2- 172.21.15.100:0/4017554777 >> v2:172.21.15.184:3300/0 conn(0x7f4068073c50 0x7f40680cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:37.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.840+0000 7f406e000640 1 --2- 172.21.15.100:0/4017554777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4068145550 0x7f406806fb40 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:37.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.840+0000 7f406e000640 1 -- 172.21.15.100:0/4017554777 >> 172.21.15.100:0/4017554777 conn(0x7f406806d1c0 msgr2=0x7f406806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:37.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.840+0000 7f406e000640 1 -- 172.21.15.100:0/4017554777 shutdown_connections 2024-09-17T04:27:37.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.840+0000 7f406e000640 1 -- 172.21.15.100:0/4017554777 wait complete. 2024-09-17T04:27:37.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.843+0000 7f406e000640 1 Processor -- start 2024-09-17T04:27:37.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.843+0000 7f406e000640 1 -- start start 2024-09-17T04:27:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.844+0000 7f406e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4068145550 0x7f406813dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.844+0000 7f406e000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4068145a40 0x7f40681f8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.844+0000 7f406e000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40681e7530 0x7f40681f91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.844+0000 7f406e000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f40680ce150 con 0x7f4068145550 2024-09-17T04:27:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.844+0000 7f406e000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f40680cdfd0 con 0x7f4068145a40 2024-09-17T04:27:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.844+0000 7f406e000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f40680ce2d0 con 0x7f40681e7530 2024-09-17T04:27:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.844+0000 7f4067400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4068145550 0x7f406813dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.844+0000 7f4067400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4068145550 0x7f406813dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48866/0 (socket says 172.21.15.100:48866) 2024-09-17T04:27:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.844+0000 7f4067400640 1 -- 172.21.15.100:0/429645398 learned_addr learned my addr 172.21.15.100:0/429645398 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.845+0000 7f4067e00640 1 --2- 172.21.15.100:0/429645398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40681e7530 0x7f40681f91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:37.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.845+0000 7f4066a00640 1 --2- 172.21.15.100:0/429645398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4068145a40 0x7f40681f8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:37.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.845+0000 7f4067400640 1 -- 172.21.15.100:0/429645398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40681e7530 msgr2=0x7f40681f91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:37.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.845+0000 7f4067400640 1 --2- 172.21.15.100:0/429645398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40681e7530 0x7f40681f91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:37.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.845+0000 7f4067400640 1 -- 172.21.15.100:0/429645398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4068145a40 msgr2=0x7f40681f8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:37.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.845+0000 7f4067400640 1 --2- 172.21.15.100:0/429645398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4068145a40 0x7f40681f8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:37.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.845+0000 7f4067400640 1 -- 172.21.15.100:0/429645398 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40681f9a00 con 0x7f4068145550 2024-09-17T04:27:37.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.845+0000 7f4066a00640 1 --2- 172.21.15.100:0/429645398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4068145a40 0x7f40681f8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:27:37.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.845+0000 7f4067400640 1 --2- 172.21.15.100:0/429645398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4068145550 0x7f406813dd00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4058009920 tx=0x7f4058033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:37.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.845+0000 7f405f400640 1 -- 172.21.15.100:0/429645398 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4058031d60 con 0x7f4068145550 2024-09-17T04:27:37.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.845+0000 7f405f400640 1 -- 172.21.15.100:0/429645398 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4058043070 con 0x7f4068145550 2024-09-17T04:27:37.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.845+0000 7f406e000640 1 -- 172.21.15.100:0/429645398 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f406813e970 con 0x7f4068145550 2024-09-17T04:27:37.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.845+0000 7f405f400640 1 -- 172.21.15.100:0/429645398 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40580332f0 con 0x7f4068145550 2024-09-17T04:27:37.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.846+0000 7f406e000640 1 -- 172.21.15.100:0/429645398 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40681f9700 con 0x7f4068145550 2024-09-17T04:27:37.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.847+0000 7f406e000640 1 -- 172.21.15.100:0/429645398 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f406813e760 con 0x7f4068145550 2024-09-17T04:27:37.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.848+0000 7f405f400640 1 -- 172.21.15.100:0/429645398 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f4058033490 con 0x7f4068145550 2024-09-17T04:27:37.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.849+0000 7f405f400640 1 --2- 172.21.15.100:0/429645398 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4048078d40 0x7f404807b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:37.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.849+0000 7f405f400640 1 -- 172.21.15.100:0/429645398 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4943+0+0 (secure 0 0 0) 0x7f40580bbeb0 con 0x7f4068145550 2024-09-17T04:27:37.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.849+0000 7f4066a00640 1 --2- 172.21.15.100:0/429645398 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4048078d40 0x7f404807b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:37.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.850+0000 7f4066a00640 1 --2- 172.21.15.100:0/429645398 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4048078d40 0x7f404807b250 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f40681e6820 tx=0x7f4054009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:37.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:37.853+0000 7f405f400640 1 -- 172.21.15.100:0/429645398 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4058084260 con 0x7f4068145550 2024-09-17T04:27:38.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.036+0000 7f406e000640 1 -- 172.21.15.100:0/429645398 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f40680ce870 con 0x7f4068145550 2024-09-17T04:27:38.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.040+0000 7f405f400640 1 -- 172.21.15.100:0/429645398 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v36)=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7f4058083b60 con 0x7f4068145550 2024-09-17T04:27:38.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.042+0000 7f406e000640 1 -- 172.21.15.100:0/429645398 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4048078d40 msgr2=0x7f404807b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:38.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.042+0000 7f406e000640 1 --2- 172.21.15.100:0/429645398 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4048078d40 0x7f404807b250 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f40681e6820 tx=0x7f4054009040 comp rx=0 tx=0).stop 2024-09-17T04:27:38.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.042+0000 7f406e000640 1 -- 172.21.15.100:0/429645398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4068145550 msgr2=0x7f406813dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:38.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.042+0000 7f406e000640 1 --2- 172.21.15.100:0/429645398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4068145550 0x7f406813dd00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4058009920 tx=0x7f4058033d60 comp rx=0 tx=0).stop 2024-09-17T04:27:38.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.043+0000 7f406e000640 1 -- 172.21.15.100:0/429645398 shutdown_connections 2024-09-17T04:27:38.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.043+0000 7f406e000640 1 --2- 172.21.15.100:0/429645398 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4048078d40 0x7f404807b250 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:38.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.043+0000 7f406e000640 1 --2- 172.21.15.100:0/429645398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40681e7530 0x7f40681f91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:38.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.043+0000 7f406e000640 1 --2- 172.21.15.100:0/429645398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4068145a40 0x7f40681f8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:38.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.043+0000 7f406e000640 1 --2- 172.21.15.100:0/429645398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4068145550 0x7f406813dd00 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:38.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.043+0000 7f406e000640 1 -- 172.21.15.100:0/429645398 >> 172.21.15.100:0/429645398 conn(0x7f406806d1c0 msgr2=0x7f40680af2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:38.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.043+0000 7f406e000640 1 -- 172.21.15.100:0/429645398 shutdown_connections 2024-09-17T04:27:38.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.043+0000 7f406e000640 1 -- 172.21.15.100:0/429645398 wait complete. 2024-09-17T04:27:38.065 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T04:27:38.065 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:27:38.153 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:27:38.263 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T04:27:38.263 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:27:38.263 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:27:38.298 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:27:38.299 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:27:38.299 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:27:38.336 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:27:38.336 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:27:38.336 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:27:38.374 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:27:38.374 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:27:38.374 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:27:38.414 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:27:38.414 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T04:27:38.414 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:27:38.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.634+0000 7fbc76e00640 1 -- 172.21.15.100:0/1870114203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc58004800 msgr2=0x7fbc58004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:38.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.634+0000 7fbc76e00640 1 --2- 172.21.15.100:0/1870114203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc58004800 0x7fbc58004c10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fbc700747d0 tx=0x7fbc58005830 comp rx=0 tx=0).stop 2024-09-17T04:27:38.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.635+0000 7fbc76400640 1 -- 172.21.15.100:0/1870114203 reap_dead start 2024-09-17T04:27:38.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.635+0000 7fbc76e00640 1 -- 172.21.15.100:0/1870114203 shutdown_connections 2024-09-17T04:27:38.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.635+0000 7fbc76e00640 1 -- 172.21.15.100:0/1870114203 >> 172.21.15.100:0/1870114203 conn(0x7fbc7006d400 msgr2=0x7fbc700710f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:38.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.635+0000 7fbc76e00640 1 -- 172.21.15.100:0/1870114203 shutdown_connections 2024-09-17T04:27:38.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.635+0000 7fbc76e00640 1 -- 172.21.15.100:0/1870114203 wait complete. 2024-09-17T04:27:38.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.637+0000 7fbc76e00640 1 Processor -- start 2024-09-17T04:27:38.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.637+0000 7fbc76e00640 1 -- start start 2024-09-17T04:27:38.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.638+0000 7fbc76e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc58004800 0x7fbc7014d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:38.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.638+0000 7fbc76e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc70153a30 0x7fbc70153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:38.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.638+0000 7fbc76e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc7014f0b0 0x7fbc7014f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:38.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.638+0000 7fbc76e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbc70157010 con 0x7fbc7014f0b0 2024-09-17T04:27:38.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.638+0000 7fbc76e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbc70156e90 con 0x7fbc58004800 2024-09-17T04:27:38.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.638+0000 7fbc76e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbc70157190 con 0x7fbc70153a30 2024-09-17T04:27:38.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.639+0000 7fbc76400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc7014f0b0 0x7fbc7014f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:38.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.639+0000 7fbc76400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc7014f0b0 0x7fbc7014f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48898/0 (socket says 172.21.15.100:48898) 2024-09-17T04:27:38.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.639+0000 7fbc76400640 1 -- 172.21.15.100:0/1310148686 learned_addr learned my addr 172.21.15.100:0/1310148686 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:38.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.639+0000 7fbc75a00640 1 --2- 172.21.15.100:0/1310148686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc58004800 0x7fbc7014d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:38.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.639+0000 7fbc75000640 1 --2- 172.21.15.100:0/1310148686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc70153a30 0x7fbc70153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:38.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.639+0000 7fbc76400640 1 -- 172.21.15.100:0/1310148686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc70153a30 msgr2=0x7fbc70153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:38.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.639+0000 7fbc76400640 1 --2- 172.21.15.100:0/1310148686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc70153a30 0x7fbc70153e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:38.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.639+0000 7fbc76400640 1 -- 172.21.15.100:0/1310148686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc58004800 msgr2=0x7fbc7014d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:38.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.639+0000 7fbc76400640 1 --2- 172.21.15.100:0/1310148686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc58004800 0x7fbc7014d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:38.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.639+0000 7fbc76400640 1 -- 172.21.15.100:0/1310148686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc7014fa00 con 0x7fbc7014f0b0 2024-09-17T04:27:38.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.640+0000 7fbc76400640 1 --2- 172.21.15.100:0/1310148686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc7014f0b0 0x7fbc7014f4c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fbc70071ff0 tx=0x7fbc6800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:38.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.640+0000 7fbc6d600640 1 -- 172.21.15.100:0/1310148686 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6801d070 con 0x7fbc7014f0b0 2024-09-17T04:27:38.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.640+0000 7fbc6d600640 1 -- 172.21.15.100:0/1310148686 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc68018440 con 0x7fbc7014f0b0 2024-09-17T04:27:38.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.640+0000 7fbc76e00640 1 -- 172.21.15.100:0/1310148686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc700c9300 con 0x7fbc7014f0b0 2024-09-17T04:27:38.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.640+0000 7fbc6d600640 1 -- 172.21.15.100:0/1310148686 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68018890 con 0x7fbc7014f0b0 2024-09-17T04:27:38.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.640+0000 7fbc76e00640 1 -- 172.21.15.100:0/1310148686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc701f3c30 con 0x7fbc7014f0b0 2024-09-17T04:27:38.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.641+0000 7fbc76e00640 1 -- 172.21.15.100:0/1310148686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc7015e2b0 con 0x7fbc7014f0b0 2024-09-17T04:27:38.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.642+0000 7fbc6d600640 1 -- 172.21.15.100:0/1310148686 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7fbc68020070 con 0x7fbc7014f0b0 2024-09-17T04:27:38.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.643+0000 7fbc6d600640 1 --2- 172.21.15.100:0/1310148686 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbc50078d40 0x7fbc5007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:38.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.643+0000 7fbc6d600640 1 -- 172.21.15.100:0/1310148686 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4943+0+0 (secure 0 0 0) 0x7fbc6809b690 con 0x7fbc7014f0b0 2024-09-17T04:27:38.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.644+0000 7fbc75a00640 1 --2- 172.21.15.100:0/1310148686 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbc50078d40 0x7fbc5007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:38.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.644+0000 7fbc75a00640 1 --2- 172.21.15.100:0/1310148686 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbc50078d40 0x7fbc5007b250 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbc640018e0 tx=0x7fbc64009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:38.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.648+0000 7fbc6d600640 1 -- 172.21.15.100:0/1310148686 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fbc7014fa00 con 0x7fbc7014f0b0 2024-09-17T04:27:38.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.867+0000 7fbc76e00640 1 -- 172.21.15.100:0/1310148686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc701f36b0 con 0x7fbc7014f0b0 2024-09-17T04:27:38.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.867+0000 7fbc6d600640 1 -- 172.21.15.100:0/1310148686 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+3920 (secure 0 0 0) 0x7fbc68063510 con 0x7fbc7014f0b0 2024-09-17T04:27:38.868 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:27:38.868 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":29,"btime":"2024-09-17T04:27:31:896016+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1086873435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1086873435},{"type":"v1","addr":"172.21.15.100:6833","nonce":1086873435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1708229764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1708229764},{"type":"v1","addr":"172.21.15.184:6836","nonce":1708229764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/828832783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":828832783},{"type":"v1","addr":"172.21.15.184:6837","nonce":828832783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5652,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/4271194387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4271194387},{"type":"v1","addr":"172.21.15.100:6835","nonce":4271194387}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[]} 2024-09-17T04:27:38.868 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 29 2024-09-17T04:27:38.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.870+0000 7fbc76e00640 1 -- 172.21.15.100:0/1310148686 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbc50078d40 msgr2=0x7fbc5007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:38.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.870+0000 7fbc76e00640 1 --2- 172.21.15.100:0/1310148686 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbc50078d40 0x7fbc5007b250 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbc640018e0 tx=0x7fbc64009040 comp rx=0 tx=0).stop 2024-09-17T04:27:38.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.870+0000 7fbc76e00640 1 -- 172.21.15.100:0/1310148686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc7014f0b0 msgr2=0x7fbc7014f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:38.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.870+0000 7fbc76e00640 1 --2- 172.21.15.100:0/1310148686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc7014f0b0 0x7fbc7014f4c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fbc70071ff0 tx=0x7fbc6800d710 comp rx=0 tx=0).stop 2024-09-17T04:27:38.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.870+0000 7fbc76e00640 1 -- 172.21.15.100:0/1310148686 shutdown_connections 2024-09-17T04:27:38.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.870+0000 7fbc76e00640 1 --2- 172.21.15.100:0/1310148686 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbc50078d40 0x7fbc5007b250 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:38.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.870+0000 7fbc76e00640 1 --2- 172.21.15.100:0/1310148686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc7014f0b0 0x7fbc7014f4c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:38.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.870+0000 7fbc76e00640 1 --2- 172.21.15.100:0/1310148686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc70153a30 0x7fbc70153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:38.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.870+0000 7fbc76e00640 1 --2- 172.21.15.100:0/1310148686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc58004800 0x7fbc7014d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:38.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.870+0000 7fbc76e00640 1 -- 172.21.15.100:0/1310148686 >> 172.21.15.100:0/1310148686 conn(0x7fbc7006d400 msgr2=0x7fbc700b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:38.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.870+0000 7fbc76e00640 1 -- 172.21.15.100:0/1310148686 shutdown_connections 2024-09-17T04:27:38.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:38.870+0000 7fbc76e00640 1 -- 172.21.15.100:0/1310148686 wait complete. 2024-09-17T04:27:38.891 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T04:27:38.891 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T04:27:38.892 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T04:27:38.892 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:27:38.892 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T04:27:38.892 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T04:27:38.893 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T04:27:38.893 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:27:38.893 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T04:27:38.893 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T04:27:38.893 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T04:27:38.893 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:27:38.893 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T04:27:38.893 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T04:27:38.894 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T04:27:38.894 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:27:38.915 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T04:27:38.915 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T04:27:38.916 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T04:27:38.917 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T04:27:38.918 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T04:27:38.918 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T04:27:38.919 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T04:27:38.919 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T04:27:38.960 INFO:tasks.ceph.mds.c:Started 2024-09-17T04:27:38.961 DEBUG:teuthology.parallel:result is None 2024-09-17T04:27:38.962 INFO:tasks.ceph.mds.d:Started 2024-09-17T04:27:38.963 DEBUG:teuthology.parallel:result is None 2024-09-17T04:27:38.964 INFO:tasks.ceph.mds.a:Started 2024-09-17T04:27:38.964 DEBUG:teuthology.parallel:result is None 2024-09-17T04:27:38.965 INFO:tasks.ceph.mds.b:Started 2024-09-17T04:27:38.965 DEBUG:teuthology.parallel:result is None 2024-09-17T04:27:38.966 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T04:27:39.155 INFO:tasks.ceph.mds.c.smithi100.stdout:starting mds.c at 2024-09-17T04:27:39.156 INFO:tasks.ceph.mds.a.smithi100.stdout:starting mds.a at 2024-09-17T04:27:39.161 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-09-17T04:27:39.161 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-09-17T04:27:39.163 INFO:tasks.ceph.mds.d.smithi184.stderr:2024-09-17T04:27:39.162+0000 7f81e3c00640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-17T04:27:39.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.197+0000 7ff946000640 1 -- 172.21.15.100:0/3104279048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff924004af0 msgr2=0x7ff924004f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:39.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.197+0000 7ff946000640 1 --2- 172.21.15.100:0/3104279048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff924004af0 0x7ff924004f20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff930006f90 tx=0x7ff924006940 comp rx=0 tx=0).stop 2024-09-17T04:27:39.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.197+0000 7ff93fe00640 1 -- 172.21.15.100:0/3104279048 reap_dead start 2024-09-17T04:27:39.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.197+0000 7ff946000640 1 -- 172.21.15.100:0/3104279048 shutdown_connections 2024-09-17T04:27:39.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.197+0000 7ff946000640 1 -- 172.21.15.100:0/3104279048 >> 172.21.15.100:0/3104279048 conn(0x7ff94006d3d0 msgr2=0x7ff9400b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:39.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.197+0000 7ff946000640 1 -- 172.21.15.100:0/3104279048 shutdown_connections 2024-09-17T04:27:39.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.197+0000 7ff946000640 1 -- 172.21.15.100:0/3104279048 wait complete. 2024-09-17T04:27:39.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.199+0000 7ff946000640 1 Processor -- start 2024-09-17T04:27:39.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.199+0000 7ff946000640 1 -- start start 2024-09-17T04:27:39.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.201+0000 7ff946000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff924004af0 0x7ff9401f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:39.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.201+0000 7ff946000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9400c9880 0x7ff9400c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:39.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.201+0000 7ff946000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9401f8c20 0x7ff9401f9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:39.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.201+0000 7ff946000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff9400cd390 con 0x7ff924004af0 2024-09-17T04:27:39.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.201+0000 7ff946000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff9400cd210 con 0x7ff9401f8c20 2024-09-17T04:27:39.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.201+0000 7ff946000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff9400cd510 con 0x7ff9400c9880 2024-09-17T04:27:39.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.201+0000 7ff93f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff924004af0 0x7ff9401f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:39.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.201+0000 7ff93ea00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9400c9880 0x7ff9400c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:39.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.201+0000 7ff93f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff924004af0 0x7ff9401f4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48954/0 (socket says 172.21.15.100:48954) 2024-09-17T04:27:39.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.201+0000 7ff93fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9401f8c20 0x7ff9401f9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:39.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.201+0000 7ff93f400640 1 -- 172.21.15.100:0/4108675637 learned_addr learned my addr 172.21.15.100:0/4108675637 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:39.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.202+0000 7ff93ea00640 1 -- 172.21.15.100:0/4108675637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9401f8c20 msgr2=0x7ff9401f9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:39.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.202+0000 7ff93ea00640 1 --2- 172.21.15.100:0/4108675637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9401f8c20 0x7ff9401f9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:39.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.202+0000 7ff93ea00640 1 -- 172.21.15.100:0/4108675637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff924004af0 msgr2=0x7ff9401f4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:39.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.202+0000 7ff93ea00640 1 --2- 172.21.15.100:0/4108675637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff924004af0 0x7ff9401f4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:39.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.202+0000 7ff93ea00640 1 -- 172.21.15.100:0/4108675637 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9400ca1d0 con 0x7ff9400c9880 2024-09-17T04:27:39.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.202+0000 7ff93f400640 1 --2- 172.21.15.100:0/4108675637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff924004af0 0x7ff9401f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:27:39.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.202+0000 7ff93ea00640 1 --2- 172.21.15.100:0/4108675637 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9400c9880 0x7ff9400c9c90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff92c011890 tx=0x7ff92c011d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:39.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.202+0000 7ff937400640 1 -- 172.21.15.100:0/4108675637 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c00f9a0 con 0x7ff9400c9880 2024-09-17T04:27:39.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.202+0000 7ff937400640 1 -- 172.21.15.100:0/4108675637 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff92c00fb40 con 0x7ff9400c9880 2024-09-17T04:27:39.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.202+0000 7ff946000640 1 -- 172.21.15.100:0/4108675637 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9400b0fd0 con 0x7ff9400c9880 2024-09-17T04:27:39.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.202+0000 7ff937400640 1 -- 172.21.15.100:0/4108675637 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c0035f0 con 0x7ff9400c9880 2024-09-17T04:27:39.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.202+0000 7ff946000640 1 -- 172.21.15.100:0/4108675637 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9400b2240 con 0x7ff9400c9880 2024-09-17T04:27:39.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.203+0000 7ff946000640 1 -- 172.21.15.100:0/4108675637 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9400b3d60 con 0x7ff9400c9880 2024-09-17T04:27:39.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.205+0000 7ff937400640 1 -- 172.21.15.100:0/4108675637 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7ff92c003790 con 0x7ff9400c9880 2024-09-17T04:27:39.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.206+0000 7ff937400640 1 --2- 172.21.15.100:0/4108675637 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff920078d40 0x7ff92007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:39.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.206+0000 7ff937400640 1 -- 172.21.15.100:0/4108675637 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(53..53 src has 1..53) ==== 4943+0+0 (secure 0 0 0) 0x7ff92c09c110 con 0x7ff9400c9880 2024-09-17T04:27:39.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.206+0000 7ff93f400640 1 --2- 172.21.15.100:0/4108675637 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff920078d40 0x7ff92007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:39.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.207+0000 7ff93f400640 1 --2- 172.21.15.100:0/4108675637 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff920078d40 0x7ff92007b250 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff930006f90 tx=0x7ff9300019a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:39.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.210+0000 7ff937400640 1 -- 172.21.15.100:0/4108675637 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff9400ca1d0 con 0x7ff9400c9880 2024-09-17T04:27:39.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:39.406+0000 7ff946000640 1 -- 172.21.15.100:0/4108675637 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff9400b48b0 con 0x7ff9400c9880 2024-09-17T04:27:40.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.178+0000 7ff937400640 1 -- 172.21.15.100:0/4108675637 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v55) ==== 97+0+0 (secure 0 0 0) 0x7ff92c063ee0 con 0x7ff9400c9880 2024-09-17T04:27:40.179 INFO:teuthology.orchestra.run.smithi100.stderr: removed all blocklist entries 2024-09-17T04:27:40.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.181+0000 7ff946000640 1 -- 172.21.15.100:0/4108675637 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff920078d40 msgr2=0x7ff92007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:40.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.181+0000 7ff946000640 1 --2- 172.21.15.100:0/4108675637 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff920078d40 0x7ff92007b250 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff930006f90 tx=0x7ff9300019a0 comp rx=0 tx=0).stop 2024-09-17T04:27:40.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.181+0000 7ff946000640 1 -- 172.21.15.100:0/4108675637 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9400c9880 msgr2=0x7ff9400c9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:40.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.181+0000 7ff946000640 1 --2- 172.21.15.100:0/4108675637 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9400c9880 0x7ff9400c9c90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff92c011890 tx=0x7ff92c011d50 comp rx=0 tx=0).stop 2024-09-17T04:27:40.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.181+0000 7ff946000640 1 -- 172.21.15.100:0/4108675637 shutdown_connections 2024-09-17T04:27:40.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.181+0000 7ff946000640 1 --2- 172.21.15.100:0/4108675637 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff920078d40 0x7ff92007b250 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:40.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.181+0000 7ff946000640 1 --2- 172.21.15.100:0/4108675637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9401f8c20 0x7ff9401f9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:40.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.181+0000 7ff946000640 1 --2- 172.21.15.100:0/4108675637 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9400c9880 0x7ff9400c9c90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:40.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.181+0000 7ff946000640 1 --2- 172.21.15.100:0/4108675637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff924004af0 0x7ff9401f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:40.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.181+0000 7ff946000640 1 -- 172.21.15.100:0/4108675637 >> 172.21.15.100:0/4108675637 conn(0x7ff94006d3d0 msgr2=0x7ff94006ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:40.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.181+0000 7ff946000640 1 -- 172.21.15.100:0/4108675637 shutdown_connections 2024-09-17T04:27:40.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.181+0000 7ff946000640 1 -- 172.21.15.100:0/4108675637 wait complete. 2024-09-17T04:27:40.206 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T04:27:40.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.427+0000 7ff1e6400640 1 -- 172.21.15.100:0/3156020867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1c4005350 msgr2=0x7ff1c400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:40.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.427+0000 7ff1e6400640 1 --2- 172.21.15.100:0/3156020867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1c4005350 0x7ff1c400b9c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8001d90 tx=0x7ff1c4005830 comp rx=0 tx=0).stop 2024-09-17T04:27:40.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.427+0000 7ff1e6400640 1 -- 172.21.15.100:0/3156020867 shutdown_connections 2024-09-17T04:27:40.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.427+0000 7ff1e6400640 1 --2- 172.21.15.100:0/3156020867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1c4005350 0x7ff1c400b9c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:40.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.427+0000 7ff1e6400640 1 -- 172.21.15.100:0/3156020867 >> 172.21.15.100:0/3156020867 conn(0x7ff1e006d730 msgr2=0x7ff1e00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:40.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.427+0000 7ff1e6400640 1 -- 172.21.15.100:0/3156020867 shutdown_connections 2024-09-17T04:27:40.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.427+0000 7ff1e6400640 1 -- 172.21.15.100:0/3156020867 wait complete. 2024-09-17T04:27:40.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.430+0000 7ff1e6400640 1 Processor -- start 2024-09-17T04:27:40.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.430+0000 7ff1e6400640 1 -- start start 2024-09-17T04:27:40.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.431+0000 7ff1e6400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1c4005350 0x7ff1e01f6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:40.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.431+0000 7ff1e6400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e00b0b00 0x7ff1e014cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:40.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.431+0000 7ff1e6400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1e015b7f0 0x7ff1e015bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:40.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.431+0000 7ff1e6400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e006f260 con 0x7ff1e00b0b00 2024-09-17T04:27:40.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.431+0000 7ff1e6400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e00b4e30 con 0x7ff1c4005350 2024-09-17T04:27:40.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.431+0000 7ff1e6400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff1e00ad4f0 con 0x7ff1e015b7f0 2024-09-17T04:27:40.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.431+0000 7ff1d6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e00b0b00 0x7ff1e014cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:40.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.431+0000 7ff1d6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e00b0b00 0x7ff1e014cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48968/0 (socket says 172.21.15.100:48968) 2024-09-17T04:27:40.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.431+0000 7ff1d6a00640 1 -- 172.21.15.100:0/3578372465 learned_addr learned my addr 172.21.15.100:0/3578372465 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:40.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.432+0000 7ff1df400640 1 --2- 172.21.15.100:0/3578372465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1c4005350 0x7ff1e01f6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:40.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.432+0000 7ff1dfe00640 1 --2- 172.21.15.100:0/3578372465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1e015b7f0 0x7ff1e015bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:40.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.432+0000 7ff1d6a00640 1 -- 172.21.15.100:0/3578372465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1e015b7f0 msgr2=0x7ff1e015bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:40.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.432+0000 7ff1d6a00640 1 --2- 172.21.15.100:0/3578372465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1e015b7f0 0x7ff1e015bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:40.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.432+0000 7ff1d6a00640 1 -- 172.21.15.100:0/3578372465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1c4005350 msgr2=0x7ff1e01f6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:40.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.432+0000 7ff1d6a00640 1 --2- 172.21.15.100:0/3578372465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1c4005350 0x7ff1e01f6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:40.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.432+0000 7ff1d6a00640 1 -- 172.21.15.100:0/3578372465 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e015c140 con 0x7ff1e00b0b00 2024-09-17T04:27:40.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.432+0000 7ff1d6a00640 1 --2- 172.21.15.100:0/3578372465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e00b0b00 0x7ff1e014cc90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff1c80118a0 tx=0x7ff1c8011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:40.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.432+0000 7ff1d7e00640 1 -- 172.21.15.100:0/3578372465 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c800fb50 con 0x7ff1e00b0b00 2024-09-17T04:27:40.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.432+0000 7ff1d7e00640 1 -- 172.21.15.100:0/3578372465 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1c800fcf0 con 0x7ff1e00b0b00 2024-09-17T04:27:40.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.432+0000 7ff1d7e00640 1 -- 172.21.15.100:0/3578372465 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8019670 con 0x7ff1e00b0b00 2024-09-17T04:27:40.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.432+0000 7ff1e6400640 1 -- 172.21.15.100:0/3578372465 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1e014c0a0 con 0x7ff1e00b0b00 2024-09-17T04:27:40.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.433+0000 7ff1e6400640 1 -- 172.21.15.100:0/3578372465 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1e01f5b20 con 0x7ff1e00b0b00 2024-09-17T04:27:40.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.434+0000 7ff1e6400640 1 -- 172.21.15.100:0/3578372465 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1e01535c0 con 0x7ff1e00b0b00 2024-09-17T04:27:40.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.435+0000 7ff1d7e00640 1 -- 172.21.15.100:0/3578372465 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7ff1c8019810 con 0x7ff1e00b0b00 2024-09-17T04:27:40.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.436+0000 7ff1d7e00640 1 --2- 172.21.15.100:0/3578372465 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff1bc078d70 0x7ff1bc07b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:40.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.436+0000 7ff1d7e00640 1 -- 172.21.15.100:0/3578372465 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 4857+0+0 (secure 0 0 0) 0x7ff1c809b670 con 0x7ff1e00b0b00 2024-09-17T04:27:40.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.436+0000 7ff1df400640 1 --2- 172.21.15.100:0/3578372465 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff1bc078d70 0x7ff1bc07b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:40.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.437+0000 7ff1df400640 1 --2- 172.21.15.100:0/3578372465 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff1bc078d70 0x7ff1bc07b280 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0001010 tx=0x7ff1d0006970 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:40.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.440+0000 7ff1d7e00640 1 -- 172.21.15.100:0/3578372465 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff1e015c140 con 0x7ff1e00b0b00 2024-09-17T04:27:40.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.691+0000 7ff1e6400640 1 -- 172.21.15.100:0/3578372465 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7ff1e014bdf0 con 0x7ff1e00b0b00 2024-09-17T04:27:40.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.692+0000 7ff1d7e00640 1 -- 172.21.15.100:0/3578372465 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5304 (secure 0 0 0) 0x7ff1c8063580 con 0x7ff1e00b0b00 2024-09-17T04:27:40.693 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:27:40.693 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:27:40.693 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_dump": [ 2024-09-17T04:27:40.694 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:27:40.694 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.a", 2024-09-17T04:27:40.694 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmV8asJRAAkSgIWqGc1OeSRbs9PEOT8w==", 2024-09-17T04:27:40.694 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:27:40.694 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:27:40.694 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:27:40.695 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:27:40.695 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:27:40.695 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:27:40.695 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:27:40.695 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:27:40.695 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.b", 2024-09-17T04:27:40.696 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmpM7ZBRAA1Og785tJwa6NA9zBdmNNQA==", 2024-09-17T04:27:40.696 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:27:40.696 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:27:40.696 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:27:40.696 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:27:40.696 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:27:40.697 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:27:40.697 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:27:40.697 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:27:40.697 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.c", 2024-09-17T04:27:40.697 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmXaOGMxAAT+96g/5fAzPW8Cc2HtQUyA==", 2024-09-17T04:27:40.697 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:27:40.698 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:27:40.698 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:27:40.698 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:27:40.698 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:27:40.698 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:27:40.698 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:27:40.699 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:27:40.699 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.d", 2024-09-17T04:27:40.699 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmRqC9ExAA23kZ0Ks1lGN2k4GXL8ASrg==", 2024-09-17T04:27:40.699 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:27:40.699 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:27:40.699 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:27:40.700 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:27:40.700 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:27:40.700 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:27:40.700 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:27:40.700 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:27:40.700 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.0", 2024-09-17T04:27:40.701 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAwBOlmofFmAxAARerYkU9B2a9MM9vHa3o4LQ==", 2024-09-17T04:27:40.701 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:27:40.701 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:27:40.701 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:27:40.701 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:27:40.701 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:27:40.702 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:27:40.702 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:27:40.702 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.1", 2024-09-17T04:27:40.702 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAzBOlmbz6VExAAoVqSqXTc1wArKhdA6+NxnA==", 2024-09-17T04:27:40.702 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:27:40.702 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:27:40.703 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:27:40.703 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:27:40.703 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:27:40.703 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:27:40.703 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:27:40.703 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.2", 2024-09-17T04:27:40.704 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA2BOlmfeSAJBAAchY6XhyDRusc+zqwQ6v6wQ==", 2024-09-17T04:27:40.704 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:27:40.704 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:27:40.704 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:27:40.704 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:27:40.704 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:27:40.705 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:27:40.705 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:27:40.705 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.3", 2024-09-17T04:27:40.705 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA6BOlm/TQiBBAAHhbs2+RCjAenGR1frhbTmQ==", 2024-09-17T04:27:40.705 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:27:40.705 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:27:40.706 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:27:40.706 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:27:40.706 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:27:40.706 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:27:40.706 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:27:40.706 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.4", 2024-09-17T04:27:40.707 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBABOlm7OKvBhAALkaAKEUYN5ChwGejoSF8yA==", 2024-09-17T04:27:40.707 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:27:40.707 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:27:40.707 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:27:40.707 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:27:40.707 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:27:40.708 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:27:40.708 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:27:40.708 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.5", 2024-09-17T04:27:40.708 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBDBOlmQpi1FhAAbA47dg6ckK1PFoCZL0SBEQ==", 2024-09-17T04:27:40.708 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:27:40.708 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:27:40.709 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:27:40.709 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:27:40.709 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:27:40.709 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:27:40.709 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:27:40.709 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.6", 2024-09-17T04:27:40.710 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBGBOlmx/w0JhAAWcPvZkMYtXD/EMKrg/W/4Q==", 2024-09-17T04:27:40.710 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:27:40.710 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:27:40.710 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:27:40.710 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:27:40.710 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:27:40.710 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:27:40.711 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:27:40.711 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.7", 2024-09-17T04:27:40.711 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBJBOlmWHjWNBAAgdul2UKY9g5p7EVi0ZdrFw==", 2024-09-17T04:27:40.711 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:27:40.711 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:27:40.711 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:27:40.712 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:27:40.712 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:27:40.712 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:27:40.712 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:27:40.712 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.0", 2024-09-17T04:27:40.712 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmfYg9IBAABP1uRKGOugrA47VxwRLkjA==", 2024-09-17T04:27:40.713 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:27:40.713 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:27:40.713 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow r", 2024-09-17T04:27:40.713 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T04:27:40.713 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:27:40.713 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:27:40.714 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:27:40.714 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.1", 2024-09-17T04:27:40.714 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmByhhNRAAPGaxFCDd8ywFVwwhprrPbQ==", 2024-09-17T04:27:40.714 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:27:40.714 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:27:40.714 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:27:40.715 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:27:40.715 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:27:40.715 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:27:40.715 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:27:40.715 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:27:40.715 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.2", 2024-09-17T04:27:40.716 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmgcm0KhAAGF06VlHjBM/AGxafN6/3aA==", 2024-09-17T04:27:40.716 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:27:40.716 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:27:40.716 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:27:40.716 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:27:40.716 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:27:40.716 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:27:40.717 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:27:40.717 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:27:40.717 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.3", 2024-09-17T04:27:40.717 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtBOlm9VQ6BBAAXcahu35/uDm8Z+f+3SF0Gg==", 2024-09-17T04:27:40.717 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:27:40.717 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:27:40.718 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:27:40.718 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:27:40.718 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:27:40.718 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:27:40.718 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:27:40.718 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:27:40.719 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.admin", 2024-09-17T04:27:40.719 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAqBOlmZ+bIJBAAG3oaMNlYWNakywGDvAGbGw==", 2024-09-17T04:27:40.719 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:27:40.719 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:27:40.719 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:27:40.719 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:27:40.719 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:27:40.720 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:27:40.720 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:27:40.720 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:27:40.720 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mgr.x", 2024-09-17T04:27:40.720 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmfaL7GhAArqLH66RkbBsz+TNOKtPHOA==", 2024-09-17T04:27:40.720 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:27:40.721 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:27:40.721 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile mgr", 2024-09-17T04:27:40.721 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:27:40.721 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:27:40.721 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:27:40.721 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:27:40.721 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mgr.y", 2024-09-17T04:27:40.722 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmM4D1DhAA5uUWgRVOMujWqbYzOjFp9A==", 2024-09-17T04:27:40.722 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:27:40.722 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:27:40.722 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile mgr", 2024-09-17T04:27:40.722 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:27:40.722 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:27:40.723 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:27:40.723 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-17T04:27:40.723 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:27:40.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.694+0000 7ff1e6400640 1 -- 172.21.15.100:0/3578372465 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff1bc078d70 msgr2=0x7ff1bc07b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:40.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.694+0000 7ff1e6400640 1 --2- 172.21.15.100:0/3578372465 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff1bc078d70 0x7ff1bc07b280 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0001010 tx=0x7ff1d0006970 comp rx=0 tx=0).stop 2024-09-17T04:27:40.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.694+0000 7ff1e6400640 1 -- 172.21.15.100:0/3578372465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e00b0b00 msgr2=0x7ff1e014cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:40.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.694+0000 7ff1e6400640 1 --2- 172.21.15.100:0/3578372465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e00b0b00 0x7ff1e014cc90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff1c80118a0 tx=0x7ff1c8011d60 comp rx=0 tx=0).stop 2024-09-17T04:27:40.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.694+0000 7ff1e6400640 1 -- 172.21.15.100:0/3578372465 shutdown_connections 2024-09-17T04:27:40.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.694+0000 7ff1e6400640 1 --2- 172.21.15.100:0/3578372465 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff1bc078d70 0x7ff1bc07b280 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:40.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.695+0000 7ff1e6400640 1 --2- 172.21.15.100:0/3578372465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1e015b7f0 0x7ff1e015bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:40.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.695+0000 7ff1e6400640 1 --2- 172.21.15.100:0/3578372465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1e00b0b00 0x7ff1e014cc90 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:40.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.695+0000 7ff1e6400640 1 --2- 172.21.15.100:0/3578372465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1c4005350 0x7ff1e01f6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:40.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.695+0000 7ff1e6400640 1 -- 172.21.15.100:0/3578372465 >> 172.21.15.100:0/3578372465 conn(0x7ff1e006d730 msgr2=0x7ff1e006f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:40.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.695+0000 7ff1e6400640 1 -- 172.21.15.100:0/3578372465 shutdown_connections 2024-09-17T04:27:40.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.695+0000 7ff1e6400640 1 -- 172.21.15.100:0/3578372465 wait complete. 2024-09-17T04:27:40.727 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T04:27:40.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.948+0000 7f8636e00640 1 -- 172.21.15.100:0/3405758461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f863015fae0 msgr2=0x7f8630166550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:40.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.948+0000 7f8636e00640 1 --2- 172.21.15.100:0/3405758461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f863015fae0 0x7f8630166550 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8628005fd0 tx=0x7f862802a8b0 comp rx=0 tx=0).stop 2024-09-17T04:27:40.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.948+0000 7f8636e00640 1 -- 172.21.15.100:0/3405758461 shutdown_connections 2024-09-17T04:27:40.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.948+0000 7f8636e00640 1 --2- 172.21.15.100:0/3405758461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f863015fae0 0x7f8630166550 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:40.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.948+0000 7f8636e00640 1 -- 172.21.15.100:0/3405758461 >> 172.21.15.100:0/3405758461 conn(0x7f863006d730 msgr2=0x7f86300b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:40.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.948+0000 7f8636e00640 1 -- 172.21.15.100:0/3405758461 shutdown_connections 2024-09-17T04:27:40.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.948+0000 7f8636e00640 1 -- 172.21.15.100:0/3405758461 wait complete. 2024-09-17T04:27:40.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.950+0000 7f8636e00640 1 Processor -- start 2024-09-17T04:27:40.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.950+0000 7f8636e00640 1 -- start start 2024-09-17T04:27:40.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.952+0000 7f8636e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86300aebc0 0x7f86300cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:40.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.952+0000 7f8636e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8630152920 0x7f863014a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:40.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.952+0000 7f8636e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8630149c70 0x7f863014a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:40.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.952+0000 7f8636e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8630166e20 con 0x7f8630152920 2024-09-17T04:27:40.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.952+0000 7f8636e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8630166ca0 con 0x7f8630149c70 2024-09-17T04:27:40.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.952+0000 7f8636e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8630166fa0 con 0x7f86300aebc0 2024-09-17T04:27:40.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.952+0000 7f862f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8630152920 0x7f863014a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:40.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.952+0000 7f862f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8630152920 0x7f863014a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48976/0 (socket says 172.21.15.100:48976) 2024-09-17T04:27:40.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.952+0000 7f862f400640 1 -- 172.21.15.100:0/4242460337 learned_addr learned my addr 172.21.15.100:0/4242460337 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:40.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.952+0000 7f862f400640 1 -- 172.21.15.100:0/4242460337 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86300aebc0 msgr2=0x7f86300cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:40.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.952+0000 7f862fe00640 1 --2- 172.21.15.100:0/4242460337 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86300aebc0 0x7f86300cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:40.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.952+0000 7f8634e00640 1 --2- 172.21.15.100:0/4242460337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8630149c70 0x7f863014a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:40.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.952+0000 7f862f400640 1 --2- 172.21.15.100:0/4242460337 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86300aebc0 0x7f86300cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:40.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.953+0000 7f862f400640 1 -- 172.21.15.100:0/4242460337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8630149c70 msgr2=0x7f863014a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:40.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.953+0000 7f862f400640 1 --2- 172.21.15.100:0/4242460337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8630149c70 0x7f863014a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:40.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.953+0000 7f862f400640 1 -- 172.21.15.100:0/4242460337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86300b3dd0 con 0x7f8630152920 2024-09-17T04:27:40.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.953+0000 7f8634e00640 1 --2- 172.21.15.100:0/4242460337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8630149c70 0x7f863014a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:27:40.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.953+0000 7f862fe00640 1 --2- 172.21.15.100:0/4242460337 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86300aebc0 0x7f86300cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:27:40.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.953+0000 7f862f400640 1 --2- 172.21.15.100:0/4242460337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8630152920 0x7f863014a490 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f861c00c440 tx=0x7f861c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:40.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.953+0000 7f8627e00640 1 -- 172.21.15.100:0/4242460337 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c016020 con 0x7f8630152920 2024-09-17T04:27:40.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.953+0000 7f8627e00640 1 -- 172.21.15.100:0/4242460337 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f861c012920 con 0x7f8630152920 2024-09-17T04:27:40.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.953+0000 7f8636e00640 1 -- 172.21.15.100:0/4242460337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86300b4070 con 0x7f8630152920 2024-09-17T04:27:40.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.953+0000 7f8627e00640 1 -- 172.21.15.100:0/4242460337 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c012d70 con 0x7f8630152920 2024-09-17T04:27:40.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.953+0000 7f8636e00640 1 -- 172.21.15.100:0/4242460337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f863014bdc0 con 0x7f8630152920 2024-09-17T04:27:40.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.954+0000 7f8636e00640 1 -- 172.21.15.100:0/4242460337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f863014c000 con 0x7f8630152920 2024-09-17T04:27:40.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.956+0000 7f8627e00640 1 -- 172.21.15.100:0/4242460337 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f861c019020 con 0x7f8630152920 2024-09-17T04:27:40.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.956+0000 7f8627e00640 1 --2- 172.21.15.100:0/4242460337 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8610078d70 0x7f861007b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:40.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.957+0000 7f8627e00640 1 -- 172.21.15.100:0/4242460337 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 4857+0+0 (secure 0 0 0) 0x7f861c09abb0 con 0x7f8630152920 2024-09-17T04:27:40.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.957+0000 7f862fe00640 1 --2- 172.21.15.100:0/4242460337 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8610078d70 0x7f861007b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:40.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.958+0000 7f862fe00640 1 --2- 172.21.15.100:0/4242460337 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8610078d70 0x7f861007b280 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8620001120 tx=0x7f8620009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:40.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:40.961+0000 7f8627e00640 1 -- 172.21.15.100:0/4242460337 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f863014c000 con 0x7f8630152920 2024-09-17T04:27:41.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.147+0000 7f8636e00640 1 -- 172.21.15.100:0/4242460337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f86300b2750 con 0x7f8630152920 2024-09-17T04:27:41.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.147+0000 7f8627e00640 1 -- 172.21.15.100:0/4242460337 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v57) ==== 84+0+65 (secure 0 0 0) 0x7f861c020030 con 0x7f8630152920 2024-09-17T04:27:41.148 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:27:41.148 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-09-17T04:27:41.148 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:27:41.148 INFO:teuthology.orchestra.run.smithi100.stdout: "poolnum": 1, 2024-09-17T04:27:41.148 INFO:teuthology.orchestra.run.smithi100.stdout: "poolname": ".mgr" 2024-09-17T04:27:41.148 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:27:41.149 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-09-17T04:27:41.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.150+0000 7f8636e00640 1 -- 172.21.15.100:0/4242460337 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8610078d70 msgr2=0x7f861007b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:41.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.150+0000 7f8636e00640 1 --2- 172.21.15.100:0/4242460337 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8610078d70 0x7f861007b280 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8620001120 tx=0x7f8620009040 comp rx=0 tx=0).stop 2024-09-17T04:27:41.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.150+0000 7f8636e00640 1 -- 172.21.15.100:0/4242460337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8630152920 msgr2=0x7f863014a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:41.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.150+0000 7f8636e00640 1 --2- 172.21.15.100:0/4242460337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8630152920 0x7f863014a490 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f861c00c440 tx=0x7f861c00c900 comp rx=0 tx=0).stop 2024-09-17T04:27:41.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.150+0000 7f8636e00640 1 -- 172.21.15.100:0/4242460337 shutdown_connections 2024-09-17T04:27:41.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.150+0000 7f8636e00640 1 --2- 172.21.15.100:0/4242460337 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8610078d70 0x7f861007b280 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:41.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.150+0000 7f8636e00640 1 --2- 172.21.15.100:0/4242460337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8630149c70 0x7f863014a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:41.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.150+0000 7f8636e00640 1 --2- 172.21.15.100:0/4242460337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8630152920 0x7f863014a490 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:41.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.150+0000 7f8636e00640 1 --2- 172.21.15.100:0/4242460337 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86300aebc0 0x7f86300cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:41.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.150+0000 7f8636e00640 1 -- 172.21.15.100:0/4242460337 >> 172.21.15.100:0/4242460337 conn(0x7f863006d730 msgr2=0x7f86300b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:41.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.150+0000 7f8636e00640 1 -- 172.21.15.100:0/4242460337 shutdown_connections 2024-09-17T04:27:41.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.150+0000 7f8636e00640 1 -- 172.21.15.100:0/4242460337 wait complete. 2024-09-17T04:27:41.175 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T04:27:41.175 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T04:27:41.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.398+0000 7f545d600640 1 -- 172.21.15.100:0/2595512070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5458162410 msgr2=0x7f5458162820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:41.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.398+0000 7f545d600640 1 --2- 172.21.15.100:0/2595512070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5458162410 0x7f5458162820 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5440009860 tx=0x7f544002a540 comp rx=0 tx=0).stop 2024-09-17T04:27:41.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.398+0000 7f545d600640 1 -- 172.21.15.100:0/2595512070 shutdown_connections 2024-09-17T04:27:41.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.398+0000 7f545d600640 1 --2- 172.21.15.100:0/2595512070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5458162410 0x7f5458162820 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:41.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.398+0000 7f545d600640 1 --2- 172.21.15.100:0/2595512070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54581558c0 0x7f54580b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:41.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.398+0000 7f545d600640 1 -- 172.21.15.100:0/2595512070 >> 172.21.15.100:0/2595512070 conn(0x7f54580b1dd0 msgr2=0x7f54580b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:41.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.398+0000 7f545d600640 1 -- 172.21.15.100:0/2595512070 shutdown_connections 2024-09-17T04:27:41.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.398+0000 7f545d600640 1 -- 172.21.15.100:0/2595512070 wait complete. 2024-09-17T04:27:41.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.401+0000 7f545d600640 1 Processor -- start 2024-09-17T04:27:41.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.401+0000 7f545d600640 1 -- start start 2024-09-17T04:27:41.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.402+0000 7f545d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54581558c0 0x7f54581f57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:41.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.402+0000 7f545d600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f545815edd0 0x7f5458147330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:41.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.402+0000 7f545d600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5458162410 0x7f54580ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:41.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.402+0000 7f545d600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f545815fa70 con 0x7f54581558c0 2024-09-17T04:27:41.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.402+0000 7f545d600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f545815f8f0 con 0x7f5458162410 2024-09-17T04:27:41.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.402+0000 7f545d600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f545815fbf0 con 0x7f545815edd0 2024-09-17T04:27:41.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.402+0000 7f5457e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54581558c0 0x7f54581f57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:41.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.402+0000 7f5457e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54581558c0 0x7f54581f57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48994/0 (socket says 172.21.15.100:48994) 2024-09-17T04:27:41.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.402+0000 7f5457e00640 1 -- 172.21.15.100:0/1292332234 learned_addr learned my addr 172.21.15.100:0/1292332234 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:41.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.403+0000 7f545cc00640 1 --2- 172.21.15.100:0/1292332234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5458162410 0x7f54580ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:41.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.403+0000 7f544f400640 1 --2- 172.21.15.100:0/1292332234 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f545815edd0 0x7f5458147330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:41.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.403+0000 7f5457e00640 1 -- 172.21.15.100:0/1292332234 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f545815edd0 msgr2=0x7f5458147330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:41.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.403+0000 7f5457e00640 1 --2- 172.21.15.100:0/1292332234 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f545815edd0 0x7f5458147330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:41.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.403+0000 7f5457e00640 1 -- 172.21.15.100:0/1292332234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5458162410 msgr2=0x7f54580ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:41.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.403+0000 7f5457e00640 1 --2- 172.21.15.100:0/1292332234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5458162410 0x7f54580ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:41.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.403+0000 7f5457e00640 1 -- 172.21.15.100:0/1292332234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54581f60f0 con 0x7f54581558c0 2024-09-17T04:27:41.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.403+0000 7f545cc00640 1 --2- 172.21.15.100:0/1292332234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5458162410 0x7f54580ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:27:41.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.403+0000 7f5457e00640 1 --2- 172.21.15.100:0/1292332234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54581558c0 0x7f54581f57e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f544800e790 tx=0x7f544800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:41.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.403+0000 7f5454c00640 1 -- 172.21.15.100:0/1292332234 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544800eea0 con 0x7f54581558c0 2024-09-17T04:27:41.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.403+0000 7f5454c00640 1 -- 172.21.15.100:0/1292332234 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f544800cd50 con 0x7f54581558c0 2024-09-17T04:27:41.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.403+0000 7f545d600640 1 -- 172.21.15.100:0/1292332234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54581f63d0 con 0x7f54581558c0 2024-09-17T04:27:41.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.404+0000 7f5454c00640 1 -- 172.21.15.100:0/1292332234 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448010690 con 0x7f54581558c0 2024-09-17T04:27:41.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.404+0000 7f545d600640 1 -- 172.21.15.100:0/1292332234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5458146b30 con 0x7f54581558c0 2024-09-17T04:27:41.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.405+0000 7f545d600640 1 -- 172.21.15.100:0/1292332234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54580b2ef0 con 0x7f54581558c0 2024-09-17T04:27:41.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.406+0000 7f5454c00640 1 -- 172.21.15.100:0/1292332234 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f5448010830 con 0x7f54581558c0 2024-09-17T04:27:41.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.407+0000 7f5454c00640 1 --2- 172.21.15.100:0/1292332234 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5434078d40 0x7f543407b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:41.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.407+0000 7f5454c00640 1 -- 172.21.15.100:0/1292332234 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 4857+0+0 (secure 0 0 0) 0x7f544809b730 con 0x7f54581558c0 2024-09-17T04:27:41.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.407+0000 7f544f400640 1 --2- 172.21.15.100:0/1292332234 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5434078d40 0x7f543407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:41.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.408+0000 7f544f400640 1 --2- 172.21.15.100:0/1292332234 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5434078d40 0x7f543407b250 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5440034000 tx=0x7f5440004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:41.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.411+0000 7f5454c00640 1 -- 172.21.15.100:0/1292332234 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f54480634d0 con 0x7f54581558c0 2024-09-17T04:27:41.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:41.595+0000 7f545d600640 1 -- 172.21.15.100:0/1292332234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f54580b4ed0 con 0x7f54581558c0 2024-09-17T04:27:42.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.174+0000 7f5454c00640 1 -- 172.21.15.100:0/1292332234 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v58) ==== 138+0+0 (secure 0 0 0) 0x7f5448067f00 con 0x7f54581558c0 2024-09-17T04:27:42.175 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' created 2024-09-17T04:27:42.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.177+0000 7f545d600640 1 -- 172.21.15.100:0/1292332234 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5434078d40 msgr2=0x7f543407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:42.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.177+0000 7f545d600640 1 --2- 172.21.15.100:0/1292332234 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5434078d40 0x7f543407b250 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5440034000 tx=0x7f5440004730 comp rx=0 tx=0).stop 2024-09-17T04:27:42.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.177+0000 7f545d600640 1 -- 172.21.15.100:0/1292332234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54581558c0 msgr2=0x7f54581f57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:42.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.177+0000 7f545d600640 1 --2- 172.21.15.100:0/1292332234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54581558c0 0x7f54581f57e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f544800e790 tx=0x7f544800ec50 comp rx=0 tx=0).stop 2024-09-17T04:27:42.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.177+0000 7f545d600640 1 -- 172.21.15.100:0/1292332234 shutdown_connections 2024-09-17T04:27:42.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.177+0000 7f545d600640 1 --2- 172.21.15.100:0/1292332234 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5434078d40 0x7f543407b250 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:42.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.177+0000 7f545d600640 1 --2- 172.21.15.100:0/1292332234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5458162410 0x7f54580ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:42.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.178+0000 7f545d600640 1 --2- 172.21.15.100:0/1292332234 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f545815edd0 0x7f5458147330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:42.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.178+0000 7f545d600640 1 --2- 172.21.15.100:0/1292332234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54581558c0 0x7f54581f57e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:42.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.178+0000 7f545d600640 1 -- 172.21.15.100:0/1292332234 >> 172.21.15.100:0/1292332234 conn(0x7f54580b1dd0 msgr2=0x7f545815f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:42.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.178+0000 7f545d600640 1 -- 172.21.15.100:0/1292332234 shutdown_connections 2024-09-17T04:27:42.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.178+0000 7f545d600640 1 -- 172.21.15.100:0/1292332234 wait complete. 2024-09-17T04:27:42.208 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T04:27:42.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.435+0000 7f017ee00640 1 -- 172.21.15.100:0/2425541080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01780709a0 msgr2=0x7f01780b4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:42.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.435+0000 7f017ee00640 1 --2- 172.21.15.100:0/2425541080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01780709a0 0x7f01780b4390 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f017815f380 tx=0x7f017002a890 comp rx=0 tx=0).stop 2024-09-17T04:27:42.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.435+0000 7f017ee00640 1 -- 172.21.15.100:0/2425541080 shutdown_connections 2024-09-17T04:27:42.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.435+0000 7f017ee00640 1 --2- 172.21.15.100:0/2425541080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01780709a0 0x7f01780b4390 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:42.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.435+0000 7f017ee00640 1 -- 172.21.15.100:0/2425541080 >> 172.21.15.100:0/2425541080 conn(0x7f017806d1c0 msgr2=0x7f017806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:42.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.435+0000 7f017ee00640 1 -- 172.21.15.100:0/2425541080 shutdown_connections 2024-09-17T04:27:42.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.435+0000 7f017ee00640 1 -- 172.21.15.100:0/2425541080 wait complete. 2024-09-17T04:27:42.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.437+0000 7f017ee00640 1 Processor -- start 2024-09-17T04:27:42.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.437+0000 7f017ee00640 1 -- start start 2024-09-17T04:27:42.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.439+0000 7f017ee00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01780ccc30 0x7f017815ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:42.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.439+0000 7f017ee00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01780cfed0 0x7f0178158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:42.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.439+0000 7f017ee00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f017815dc70 0x7f017815e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:42.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.439+0000 7f017ee00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f017815c600 con 0x7f017815dc70 2024-09-17T04:27:42.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.439+0000 7f017ee00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f017815c480 con 0x7f01780cfed0 2024-09-17T04:27:42.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.439+0000 7f017ee00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f017815c780 con 0x7f01780ccc30 2024-09-17T04:27:42.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.439+0000 7f017ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f017815dc70 0x7f017815e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:42.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.439+0000 7f017ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f017815dc70 0x7f017815e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49008/0 (socket says 172.21.15.100:49008) 2024-09-17T04:27:42.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.439+0000 7f017ce00640 1 -- 172.21.15.100:0/4260334622 learned_addr learned my addr 172.21.15.100:0/4260334622 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:42.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.439+0000 7f016f400640 1 --2- 172.21.15.100:0/4260334622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01780cfed0 0x7f0178158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:42.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.439+0000 7f0177e00640 1 --2- 172.21.15.100:0/4260334622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01780ccc30 0x7f017815ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:42.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.439+0000 7f017ce00640 1 -- 172.21.15.100:0/4260334622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01780ccc30 msgr2=0x7f017815ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:42.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.439+0000 7f017ce00640 1 --2- 172.21.15.100:0/4260334622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01780ccc30 0x7f017815ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:42.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.439+0000 7f017ce00640 1 -- 172.21.15.100:0/4260334622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01780cfed0 msgr2=0x7f0178158980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:42.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.439+0000 7f017ce00640 1 --2- 172.21.15.100:0/4260334622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01780cfed0 0x7f0178158980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:42.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.439+0000 7f017ce00640 1 -- 172.21.15.100:0/4260334622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017815e5c0 con 0x7f017815dc70 2024-09-17T04:27:42.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.440+0000 7f017ce00640 1 --2- 172.21.15.100:0/4260334622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f017815dc70 0x7f017815e080 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0170009ca0 tx=0x7f0170033f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:42.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.440+0000 7f0174c00640 1 -- 172.21.15.100:0/4260334622 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0170031970 con 0x7f017815dc70 2024-09-17T04:27:42.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.440+0000 7f0174c00640 1 -- 172.21.15.100:0/4260334622 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0170031c00 con 0x7f017815dc70 2024-09-17T04:27:42.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.440+0000 7f0174c00640 1 -- 172.21.15.100:0/4260334622 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01700330d0 con 0x7f017815dc70 2024-09-17T04:27:42.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.440+0000 7f017ee00640 1 -- 172.21.15.100:0/4260334622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01781f9a60 con 0x7f017815dc70 2024-09-17T04:27:42.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.440+0000 7f017ee00640 1 -- 172.21.15.100:0/4260334622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01781563b0 con 0x7f017815dc70 2024-09-17T04:27:42.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.441+0000 7f017ee00640 1 -- 172.21.15.100:0/4260334622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01781f9780 con 0x7f017815dc70 2024-09-17T04:27:42.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.442+0000 7f0174c00640 1 -- 172.21.15.100:0/4260334622 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f0170033270 con 0x7f017815dc70 2024-09-17T04:27:42.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.443+0000 7f0174c00640 1 --2- 172.21.15.100:0/4260334622 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0154078d40 0x7f015407b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:42.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.443+0000 7f0174c00640 1 -- 172.21.15.100:0/4260334622 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5227+0+0 (secure 0 0 0) 0x7f01700bc240 con 0x7f017815dc70 2024-09-17T04:27:42.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.444+0000 7f0177e00640 1 --2- 172.21.15.100:0/4260334622 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0154078d40 0x7f015407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:42.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.445+0000 7f0177e00640 1 --2- 172.21.15.100:0/4260334622 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0154078d40 0x7f015407b250 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0168006fd0 tx=0x7f0168008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:42.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.449+0000 7f0174c00640 1 -- 172.21.15.100:0/4260334622 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f017815e5c0 con 0x7f017815dc70 2024-09-17T04:27:42.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:42.632+0000 7f017ee00640 1 -- 172.21.15.100:0/4260334622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f01781eafc0 con 0x7f017815dc70 2024-09-17T04:27:43.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.177+0000 7f0174c00640 1 -- 172.21.15.100:0/4260334622 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v59) ==== 170+0+0 (secure 0 0 0) 0x7f017003f030 con 0x7f017815dc70 2024-09-17T04:27:43.178 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' created 2024-09-17T04:27:43.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.181+0000 7f017ee00640 1 -- 172.21.15.100:0/4260334622 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0154078d40 msgr2=0x7f015407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:43.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.181+0000 7f017ee00640 1 --2- 172.21.15.100:0/4260334622 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0154078d40 0x7f015407b250 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0168006fd0 tx=0x7f0168008040 comp rx=0 tx=0).stop 2024-09-17T04:27:43.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.181+0000 7f017ee00640 1 -- 172.21.15.100:0/4260334622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f017815dc70 msgr2=0x7f017815e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:43.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.181+0000 7f017ee00640 1 --2- 172.21.15.100:0/4260334622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f017815dc70 0x7f017815e080 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0170009ca0 tx=0x7f0170033f30 comp rx=0 tx=0).stop 2024-09-17T04:27:43.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.182+0000 7f017ee00640 1 -- 172.21.15.100:0/4260334622 shutdown_connections 2024-09-17T04:27:43.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.182+0000 7f017ee00640 1 --2- 172.21.15.100:0/4260334622 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0154078d40 0x7f015407b250 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:43.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.182+0000 7f017ee00640 1 --2- 172.21.15.100:0/4260334622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f017815dc70 0x7f017815e080 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:43.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.182+0000 7f017ee00640 1 --2- 172.21.15.100:0/4260334622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01780cfed0 0x7f0178158980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:43.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.182+0000 7f017ee00640 1 --2- 172.21.15.100:0/4260334622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01780ccc30 0x7f017815ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:43.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.182+0000 7f017ee00640 1 -- 172.21.15.100:0/4260334622 >> 172.21.15.100:0/4260334622 conn(0x7f017806d1c0 msgr2=0x7f017814e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:43.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.182+0000 7f017ee00640 1 -- 172.21.15.100:0/4260334622 shutdown_connections 2024-09-17T04:27:43.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.183+0000 7f017ee00640 1 -- 172.21.15.100:0/4260334622 wait complete. 2024-09-17T04:27:43.223 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T04:27:43.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.447+0000 7f897ac00640 1 -- 172.21.15.100:0/2374514455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8974073a80 msgr2=0x7f8974160320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:43.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.447+0000 7f897ac00640 1 --2- 172.21.15.100:0/2374514455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8974073a80 0x7f8974160320 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8974072bd0 tx=0x7f896c02aac0 comp rx=0 tx=0).stop 2024-09-17T04:27:43.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.447+0000 7f897ac00640 1 -- 172.21.15.100:0/2374514455 shutdown_connections 2024-09-17T04:27:43.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.447+0000 7f897ac00640 1 --2- 172.21.15.100:0/2374514455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8974073a80 0x7f8974160320 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:43.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.447+0000 7f897ac00640 1 --2- 172.21.15.100:0/2374514455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8974145a40 0x7f8974073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:43.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.447+0000 7f897ac00640 1 --2- 172.21.15.100:0/2374514455 >> v2:172.21.15.184:3300/0 conn(0x7f8974145550 0x7f897406fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:43.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.447+0000 7f897ac00640 1 -- 172.21.15.100:0/2374514455 >> 172.21.15.100:0/2374514455 conn(0x7f897406d1c0 msgr2=0x7f897406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:43.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.447+0000 7f897ac00640 1 -- 172.21.15.100:0/2374514455 shutdown_connections 2024-09-17T04:27:43.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.447+0000 7f897ac00640 1 -- 172.21.15.100:0/2374514455 wait complete. 2024-09-17T04:27:43.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.449+0000 7f897ac00640 1 Processor -- start 2024-09-17T04:27:43.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.449+0000 7f897ac00640 1 -- start start 2024-09-17T04:27:43.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.451+0000 7f897ac00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8974073a80 0x7f897413d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:43.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.451+0000 7f897ac00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8974145550 0x7f8974142af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:43.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.451+0000 7f897ac00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8974145a40 0x7f8974143030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:43.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.451+0000 7f897ac00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8974160bf0 con 0x7f8974145550 2024-09-17T04:27:43.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.451+0000 7f897ac00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8974160a70 con 0x7f8974073a80 2024-09-17T04:27:43.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.451+0000 7f897ac00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8974160d70 con 0x7f8974145a40 2024-09-17T04:27:43.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.451+0000 7f8973400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8974145550 0x7f8974142af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:43.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.451+0000 7f8973400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8974145550 0x7f8974142af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49024/0 (socket says 172.21.15.100:49024) 2024-09-17T04:27:43.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.451+0000 7f8973400640 1 -- 172.21.15.100:0/533787595 learned_addr learned my addr 172.21.15.100:0/533787595 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:43.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.451+0000 7f8973e00640 1 --2- 172.21.15.100:0/533787595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8974073a80 0x7f897413d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:43.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.451+0000 7f8978c00640 1 --2- 172.21.15.100:0/533787595 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8974145a40 0x7f8974143030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:43.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.452+0000 7f8973400640 1 -- 172.21.15.100:0/533787595 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8974145a40 msgr2=0x7f8974143030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:43.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.452+0000 7f8973400640 1 --2- 172.21.15.100:0/533787595 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8974145a40 0x7f8974143030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:43.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.452+0000 7f8973400640 1 -- 172.21.15.100:0/533787595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8974073a80 msgr2=0x7f897413d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:43.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.452+0000 7f8973400640 1 --2- 172.21.15.100:0/533787595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8974073a80 0x7f897413d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:43.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.452+0000 7f8973400640 1 -- 172.21.15.100:0/533787595 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897413d610 con 0x7f8974145550 2024-09-17T04:27:43.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.452+0000 7f8973400640 1 --2- 172.21.15.100:0/533787595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8974145550 0x7f8974142af0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f896000e6f0 tx=0x7f896000ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:43.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.452+0000 7f896be00640 1 -- 172.21.15.100:0/533787595 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8960004220 con 0x7f8974145550 2024-09-17T04:27:43.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.452+0000 7f896be00640 1 -- 172.21.15.100:0/533787595 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f896000c8b0 con 0x7f8974145550 2024-09-17T04:27:43.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.452+0000 7f896be00640 1 -- 172.21.15.100:0/533787595 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896000cd00 con 0x7f8974145550 2024-09-17T04:27:43.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.452+0000 7f897ac00640 1 -- 172.21.15.100:0/533787595 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89740aeef0 con 0x7f8974145550 2024-09-17T04:27:43.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.453+0000 7f897ac00640 1 -- 172.21.15.100:0/533787595 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89740af4b0 con 0x7f8974145550 2024-09-17T04:27:43.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.454+0000 7f897ac00640 1 -- 172.21.15.100:0/533787595 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89740aec60 con 0x7f8974145550 2024-09-17T04:27:43.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.455+0000 7f896be00640 1 -- 172.21.15.100:0/533787595 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f89600035f0 con 0x7f8974145550 2024-09-17T04:27:43.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.456+0000 7f896be00640 1 --2- 172.21.15.100:0/533787595 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8954078d40 0x7f895407b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:43.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.457+0000 7f896be00640 1 -- 172.21.15.100:0/533787595 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5609+0+0 (secure 0 0 0) 0x7f896009b710 con 0x7f8974145550 2024-09-17T04:27:43.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.457+0000 7f8973e00640 1 --2- 172.21.15.100:0/533787595 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8954078d40 0x7f895407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:43.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.458+0000 7f8973e00640 1 --2- 172.21.15.100:0/533787595 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8954078d40 0x7f895407b250 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8964009ab0 tx=0x7f8964005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:43.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.462+0000 7f896be00640 1 -- 172.21.15.100:0/533787595 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f89600630b0 con 0x7f8974145550 2024-09-17T04:27:43.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:43.680+0000 7f897ac00640 1 -- 172.21.15.100:0/533787595 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f897406fb40 con 0x7f8974145550 2024-09-17T04:27:44.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.206+0000 7f896be00640 1 -- 172.21.15.100:0/533787595 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v32) ==== 172+0+0 (secure 0 0 0) 0x7f8960067ae0 con 0x7f8974145550 2024-09-17T04:27:44.207 INFO:teuthology.orchestra.run.smithi100.stderr:new fs with metadata pool 8 and data pool 9 2024-09-17T04:27:44.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.211+0000 7f897ac00640 1 -- 172.21.15.100:0/533787595 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8954078d40 msgr2=0x7f895407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:44.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.211+0000 7f897ac00640 1 --2- 172.21.15.100:0/533787595 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8954078d40 0x7f895407b250 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8964009ab0 tx=0x7f8964005f90 comp rx=0 tx=0).stop 2024-09-17T04:27:44.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.211+0000 7f897ac00640 1 -- 172.21.15.100:0/533787595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8974145550 msgr2=0x7f8974142af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:44.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.211+0000 7f897ac00640 1 --2- 172.21.15.100:0/533787595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8974145550 0x7f8974142af0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f896000e6f0 tx=0x7f896000ebb0 comp rx=0 tx=0).stop 2024-09-17T04:27:44.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.211+0000 7f897ac00640 1 -- 172.21.15.100:0/533787595 shutdown_connections 2024-09-17T04:27:44.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.211+0000 7f897ac00640 1 --2- 172.21.15.100:0/533787595 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f8954078d40 0x7f895407b250 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:44.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.211+0000 7f897ac00640 1 --2- 172.21.15.100:0/533787595 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8974145a40 0x7f8974143030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:44.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.211+0000 7f897ac00640 1 --2- 172.21.15.100:0/533787595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8974145550 0x7f8974142af0 secure :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f896000e6f0 tx=0x7f896000ebb0 comp rx=0 tx=0).stop 2024-09-17T04:27:44.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.211+0000 7f897ac00640 1 --2- 172.21.15.100:0/533787595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8974073a80 0x7f897413d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:44.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.211+0000 7f897ac00640 1 -- 172.21.15.100:0/533787595 >> 172.21.15.100:0/533787595 conn(0x7f897406d1c0 msgr2=0x7f897413d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:44.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.213+0000 7f897ac00640 1 -- 172.21.15.100:0/533787595 shutdown_connections 2024-09-17T04:27:44.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.213+0000 7f897ac00640 1 -- 172.21.15.100:0/533787595 wait complete. 2024-09-17T04:27:44.249 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:27:44.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.469+0000 7f562ba00640 1 -- 172.21.15.100:0/918740799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f560c004800 msgr2=0x7f560c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:44.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.469+0000 7f562ba00640 1 --2- 172.21.15.100:0/918740799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f560c004800 0x7f560c004c10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5610009b50 tx=0x7f560c005830 comp rx=0 tx=0).stop 2024-09-17T04:27:44.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.469+0000 7f562ba00640 1 -- 172.21.15.100:0/918740799 shutdown_connections 2024-09-17T04:27:44.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.469+0000 7f562ba00640 1 --2- 172.21.15.100:0/918740799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f560c004800 0x7f560c004c10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:44.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.469+0000 7f562ba00640 1 -- 172.21.15.100:0/918740799 >> 172.21.15.100:0/918740799 conn(0x7f562406d730 msgr2=0x7f56240b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:44.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.469+0000 7f562ba00640 1 -- 172.21.15.100:0/918740799 shutdown_connections 2024-09-17T04:27:44.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.470+0000 7f562ba00640 1 -- 172.21.15.100:0/918740799 wait complete. 2024-09-17T04:27:44.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.472+0000 7f562ba00640 1 Processor -- start 2024-09-17T04:27:44.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.472+0000 7f562ba00640 1 -- start start 2024-09-17T04:27:44.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.473+0000 7f562ba00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f560c004800 0x7f56240cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:44.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.473+0000 7f562ba00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f562414f980 0x7f56240ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:44.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.473+0000 7f562ba00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56240cef20 0x7f56241f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:44.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.473+0000 7f562ba00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f56240b2750 con 0x7f562414f980 2024-09-17T04:27:44.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.473+0000 7f562ba00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f562415f6f0 con 0x7f56240cef20 2024-09-17T04:27:44.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.473+0000 7f562ba00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f56240b28d0 con 0x7f560c004800 2024-09-17T04:27:44.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.474+0000 7f5623e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f562414f980 0x7f56240ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:44.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.474+0000 7f5623e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f562414f980 0x7f56240ce9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49042/0 (socket says 172.21.15.100:49042) 2024-09-17T04:27:44.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.474+0000 7f5623e00640 1 -- 172.21.15.100:0/4002142345 learned_addr learned my addr 172.21.15.100:0/4002142345 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:44.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.474+0000 7f5629000640 1 --2- 172.21.15.100:0/4002142345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f560c004800 0x7f56240cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:44.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.474+0000 7f5629a00640 1 --2- 172.21.15.100:0/4002142345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56240cef20 0x7f56241f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:44.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.474+0000 7f5623e00640 1 -- 172.21.15.100:0/4002142345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f560c004800 msgr2=0x7f56240cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:44.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.474+0000 7f5623e00640 1 --2- 172.21.15.100:0/4002142345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f560c004800 0x7f56240cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:44.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.474+0000 7f5623e00640 1 -- 172.21.15.100:0/4002142345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56240cef20 msgr2=0x7f56241f89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:44.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.474+0000 7f5623e00640 1 --2- 172.21.15.100:0/4002142345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56240cef20 0x7f56241f89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:44.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.474+0000 7f5623e00640 1 -- 172.21.15.100:0/4002142345 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56240cf330 con 0x7f562414f980 2024-09-17T04:27:44.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.474+0000 7f5623e00640 1 --2- 172.21.15.100:0/4002142345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f562414f980 0x7f56240ce9e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5610009a70 tx=0x7f56100336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:44.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.475+0000 7f5620c00640 1 -- 172.21.15.100:0/4002142345 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610038040 con 0x7f562414f980 2024-09-17T04:27:44.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.475+0000 7f5620c00640 1 -- 172.21.15.100:0/4002142345 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56100339d0 con 0x7f562414f980 2024-09-17T04:27:44.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.475+0000 7f5620c00640 1 -- 172.21.15.100:0/4002142345 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610033e20 con 0x7f562414f980 2024-09-17T04:27:44.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.475+0000 7f562ba00640 1 -- 172.21.15.100:0/4002142345 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5624074c20 con 0x7f562414f980 2024-09-17T04:27:44.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.475+0000 7f562ba00640 1 -- 172.21.15.100:0/4002142345 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5624073e60 con 0x7f562414f980 2024-09-17T04:27:44.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.476+0000 7f562ba00640 1 -- 172.21.15.100:0/4002142345 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f562415bd90 con 0x7f562414f980 2024-09-17T04:27:44.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.477+0000 7f5620c00640 1 -- 172.21.15.100:0/4002142345 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f5610043020 con 0x7f562414f980 2024-09-17T04:27:44.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.478+0000 7f5620c00640 1 --2- 172.21.15.100:0/4002142345 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5604078d40 0x7f560407b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:44.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.478+0000 7f5620c00640 1 -- 172.21.15.100:0/4002142345 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f56100bb300 con 0x7f562414f980 2024-09-17T04:27:44.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.479+0000 7f5629000640 1 --2- 172.21.15.100:0/4002142345 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5604078d40 0x7f560407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:44.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.479+0000 7f5629000640 1 --2- 172.21.15.100:0/4002142345 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5604078d40 0x7f560407b250 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5618001eb0 tx=0x7f56180067a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:44.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.483+0000 7f5620c00640 1 -- 172.21.15.100:0/4002142345 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f561004a020 con 0x7f562414f980 2024-09-17T04:27:44.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.665+0000 7f562ba00640 1 -- 172.21.15.100:0/4002142345 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5624159410 con 0x7f562414f980 2024-09-17T04:27:44.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.666+0000 7f5620c00640 1 -- 172.21.15.100:0/4002142345 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16786 (secure 0 0 0) 0x7f56240cf330 con 0x7f562414f980 2024-09-17T04:27:44.667 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:27:44.667 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":60,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:27:43.682638+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:27:41.608179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-17T04:27:42.645978+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:27:44.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.668+0000 7f562ba00640 1 -- 172.21.15.100:0/4002142345 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5604078d40 msgr2=0x7f560407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:44.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.668+0000 7f562ba00640 1 --2- 172.21.15.100:0/4002142345 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5604078d40 0x7f560407b250 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5618001eb0 tx=0x7f56180067a0 comp rx=0 tx=0).stop 2024-09-17T04:27:44.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.668+0000 7f562ba00640 1 -- 172.21.15.100:0/4002142345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f562414f980 msgr2=0x7f56240ce9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:44.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.668+0000 7f562ba00640 1 --2- 172.21.15.100:0/4002142345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f562414f980 0x7f56240ce9e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5610009a70 tx=0x7f56100336f0 comp rx=0 tx=0).stop 2024-09-17T04:27:44.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.668+0000 7f562ba00640 1 -- 172.21.15.100:0/4002142345 shutdown_connections 2024-09-17T04:27:44.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.668+0000 7f562ba00640 1 --2- 172.21.15.100:0/4002142345 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5604078d40 0x7f560407b250 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:44.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.668+0000 7f562ba00640 1 --2- 172.21.15.100:0/4002142345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56240cef20 0x7f56241f89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:44.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.668+0000 7f562ba00640 1 --2- 172.21.15.100:0/4002142345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f562414f980 0x7f56240ce9e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:44.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.668+0000 7f562ba00640 1 --2- 172.21.15.100:0/4002142345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f560c004800 0x7f56240cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:44.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.668+0000 7f562ba00640 1 -- 172.21.15.100:0/4002142345 >> 172.21.15.100:0/4002142345 conn(0x7f562406d730 msgr2=0x7f562415c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:44.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.668+0000 7f562ba00640 1 -- 172.21.15.100:0/4002142345 shutdown_connections 2024-09-17T04:27:44.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.669+0000 7f562ba00640 1 -- 172.21.15.100:0/4002142345 wait complete. 2024-09-17T04:27:44.693 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:27:44.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.914+0000 7f4725c00640 1 -- 172.21.15.100:0/402171110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47200ad9c0 msgr2=0x7f47200af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:44.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.914+0000 7f4725c00640 1 --2- 172.21.15.100:0/402171110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47200ad9c0 0x7f47200af170 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4710005fd0 tx=0x7f471002a5f0 comp rx=0 tx=0).stop 2024-09-17T04:27:44.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.914+0000 7f4725c00640 1 -- 172.21.15.100:0/402171110 shutdown_connections 2024-09-17T04:27:44.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.914+0000 7f4725c00640 1 --2- 172.21.15.100:0/402171110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47200ad9c0 0x7f47200af170 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:44.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.914+0000 7f4725c00640 1 -- 172.21.15.100:0/402171110 >> 172.21.15.100:0/402171110 conn(0x7f47200b1e40 msgr2=0x7f47200b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:44.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.914+0000 7f4725c00640 1 -- 172.21.15.100:0/402171110 shutdown_connections 2024-09-17T04:27:44.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.914+0000 7f4725c00640 1 -- 172.21.15.100:0/402171110 wait complete. 2024-09-17T04:27:44.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.916+0000 7f4725c00640 1 Processor -- start 2024-09-17T04:27:44.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.916+0000 7f4725c00640 1 -- start start 2024-09-17T04:27:44.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.918+0000 7f4725c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47200ad9c0 0x7f47201f9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:44.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.918+0000 7f4725c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47200b2f60 0x7f47201f50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:44.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.918+0000 7f4725c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f472015a0f0 0x7f4720158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:44.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.918+0000 7f4725c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f472006fb20 con 0x7f47200ad9c0 2024-09-17T04:27:44.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.918+0000 7f4725c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f472006f9a0 con 0x7f472015a0f0 2024-09-17T04:27:44.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.918+0000 7f4725c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f472015d670 con 0x7f47200b2f60 2024-09-17T04:27:44.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.918+0000 7f471fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47200ad9c0 0x7f47201f9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:44.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.918+0000 7f471fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47200ad9c0 0x7f47201f9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56744/0 (socket says 172.21.15.100:56744) 2024-09-17T04:27:44.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.918+0000 7f471fe00640 1 -- 172.21.15.100:0/2896174294 learned_addr learned my addr 172.21.15.100:0/2896174294 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:44.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.918+0000 7f4725200640 1 --2- 172.21.15.100:0/2896174294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f472015a0f0 0x7f4720158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:44.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.918+0000 7f4717400640 1 --2- 172.21.15.100:0/2896174294 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47200b2f60 0x7f47201f50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:44.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.919+0000 7f471fe00640 1 -- 172.21.15.100:0/2896174294 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47200b2f60 msgr2=0x7f47201f50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:44.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.919+0000 7f471fe00640 1 --2- 172.21.15.100:0/2896174294 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47200b2f60 0x7f47201f50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:44.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.919+0000 7f471fe00640 1 -- 172.21.15.100:0/2896174294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f472015a0f0 msgr2=0x7f4720158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:44.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.919+0000 7f471fe00640 1 --2- 172.21.15.100:0/2896174294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f472015a0f0 0x7f4720158ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:44.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.919+0000 7f471fe00640 1 -- 172.21.15.100:0/2896174294 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4720157e50 con 0x7f47200ad9c0 2024-09-17T04:27:44.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.919+0000 7f4717400640 1 --2- 172.21.15.100:0/2896174294 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47200b2f60 0x7f47201f50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:27:44.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.919+0000 7f471fe00640 1 --2- 172.21.15.100:0/2896174294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47200ad9c0 0x7f47201f9750 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4710009230 tx=0x7f4710003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:44.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.919+0000 7f471cc00640 1 -- 172.21.15.100:0/2896174294 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471003e030 con 0x7f47200ad9c0 2024-09-17T04:27:44.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.919+0000 7f471cc00640 1 -- 172.21.15.100:0/2896174294 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4710003e00 con 0x7f47200ad9c0 2024-09-17T04:27:44.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.919+0000 7f471cc00640 1 -- 172.21.15.100:0/2896174294 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710035040 con 0x7f47200ad9c0 2024-09-17T04:27:44.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.919+0000 7f4725c00640 1 -- 172.21.15.100:0/2896174294 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4720158130 con 0x7f47200ad9c0 2024-09-17T04:27:44.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.919+0000 7f4725c00640 1 -- 172.21.15.100:0/2896174294 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47201586f0 con 0x7f47200ad9c0 2024-09-17T04:27:44.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.920+0000 7f4725c00640 1 -- 172.21.15.100:0/2896174294 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47200ad1b0 con 0x7f47200ad9c0 2024-09-17T04:27:44.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.922+0000 7f471cc00640 1 -- 172.21.15.100:0/2896174294 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f4710041030 con 0x7f47200ad9c0 2024-09-17T04:27:44.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.923+0000 7f471cc00640 1 --2- 172.21.15.100:0/2896174294 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f46fc078d40 0x7f46fc07b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:44.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.923+0000 7f471cc00640 1 -- 172.21.15.100:0/2896174294 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f47100bba40 con 0x7f47200ad9c0 2024-09-17T04:27:44.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.923+0000 7f4717400640 1 --2- 172.21.15.100:0/2896174294 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f46fc078d40 0x7f46fc07b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:44.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.924+0000 7f4717400640 1 --2- 172.21.15.100:0/2896174294 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f46fc078d40 0x7f46fc07b250 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4708007fd0 tx=0x7f4708009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:44.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:44.927+0000 7f471cc00640 1 -- 172.21.15.100:0/2896174294 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f47200ad1b0 con 0x7f47200ad9c0 2024-09-17T04:27:45.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.112+0000 7f4725c00640 1 -- 172.21.15.100:0/2896174294 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f47200af4a0 con 0x7f47200ad9c0 2024-09-17T04:27:45.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.113+0000 7f471cc00640 1 -- 172.21.15.100:0/2896174294 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16786 (secure 0 0 0) 0x7f47200af4a0 con 0x7f47200ad9c0 2024-09-17T04:27:45.114 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:27:45.114 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":60,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:27:43.682638+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:27:41.608179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-17T04:27:42.645978+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:27:45.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.116+0000 7f4725c00640 1 -- 172.21.15.100:0/2896174294 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f46fc078d40 msgr2=0x7f46fc07b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:45.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.116+0000 7f4725c00640 1 --2- 172.21.15.100:0/2896174294 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f46fc078d40 0x7f46fc07b250 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4708007fd0 tx=0x7f4708009040 comp rx=0 tx=0).stop 2024-09-17T04:27:45.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.116+0000 7f4725c00640 1 -- 172.21.15.100:0/2896174294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47200ad9c0 msgr2=0x7f47201f9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:45.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.116+0000 7f4725c00640 1 --2- 172.21.15.100:0/2896174294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47200ad9c0 0x7f47201f9750 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4710009230 tx=0x7f4710003910 comp rx=0 tx=0).stop 2024-09-17T04:27:45.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.116+0000 7f4725c00640 1 -- 172.21.15.100:0/2896174294 shutdown_connections 2024-09-17T04:27:45.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.116+0000 7f4725c00640 1 --2- 172.21.15.100:0/2896174294 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f46fc078d40 0x7f46fc07b250 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:45.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.116+0000 7f4725c00640 1 --2- 172.21.15.100:0/2896174294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f472015a0f0 0x7f4720158ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:45.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.116+0000 7f4725c00640 1 --2- 172.21.15.100:0/2896174294 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47200b2f60 0x7f47201f50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:45.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.116+0000 7f4725c00640 1 --2- 172.21.15.100:0/2896174294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47200ad9c0 0x7f47201f9750 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:45.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.116+0000 7f4725c00640 1 -- 172.21.15.100:0/2896174294 >> 172.21.15.100:0/2896174294 conn(0x7f47200b1e40 msgr2=0x7f47200b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:45.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.116+0000 7f4725c00640 1 -- 172.21.15.100:0/2896174294 shutdown_connections 2024-09-17T04:27:45.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.116+0000 7f4725c00640 1 -- 172.21.15.100:0/2896174294 wait complete. 2024-09-17T04:27:45.142 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T04:27:45.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.368+0000 7f3dbec00640 1 -- 172.21.15.100:0/849953114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3db80b3f30 msgr2=0x7f3db8156ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:45.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.368+0000 7f3dbec00640 1 --2- 172.21.15.100:0/849953114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3db80b3f30 0x7f3db8156ca0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009860 tx=0x7f3dac02a530 comp rx=0 tx=0).stop 2024-09-17T04:27:45.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.368+0000 7f3dbec00640 1 -- 172.21.15.100:0/849953114 shutdown_connections 2024-09-17T04:27:45.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.368+0000 7f3dbec00640 1 --2- 172.21.15.100:0/849953114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3db80b3f30 0x7f3db8156ca0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:45.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.368+0000 7f3dbec00640 1 -- 172.21.15.100:0/849953114 >> 172.21.15.100:0/849953114 conn(0x7f3db806d620 msgr2=0x7f3db80b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:45.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.368+0000 7f3dbec00640 1 -- 172.21.15.100:0/849953114 shutdown_connections 2024-09-17T04:27:45.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.368+0000 7f3dbec00640 1 -- 172.21.15.100:0/849953114 wait complete. 2024-09-17T04:27:45.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.370+0000 7f3dbec00640 1 Processor -- start 2024-09-17T04:27:45.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.370+0000 7f3dbec00640 1 -- start start 2024-09-17T04:27:45.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.371+0000 7f3dbec00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3db80b3f30 0x7f3db80d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:45.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.371+0000 7f3dbec00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3db80cd1d0 0x7f3db80990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:45.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.371+0000 7f3dbec00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3db815d6e0 0x7f3db80995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:45.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.371+0000 7f3dbec00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3db80af7e0 con 0x7f3db815d6e0 2024-09-17T04:27:45.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.371+0000 7f3dbec00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3db8070370 con 0x7f3db80cd1d0 2024-09-17T04:27:45.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.371+0000 7f3dbec00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3db80af960 con 0x7f3db80b3f30 2024-09-17T04:27:45.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.372+0000 7f3dbe200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3db815d6e0 0x7f3db80995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:45.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.372+0000 7f3dbe200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3db815d6e0 0x7f3db80995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56758/0 (socket says 172.21.15.100:56758) 2024-09-17T04:27:45.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.372+0000 7f3dbe200640 1 -- 172.21.15.100:0/618101299 learned_addr learned my addr 172.21.15.100:0/618101299 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:45.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.372+0000 7f3dbd800640 1 --2- 172.21.15.100:0/618101299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3db80b3f30 0x7f3db80d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:45.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.372+0000 7f3dbce00640 1 --2- 172.21.15.100:0/618101299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3db80cd1d0 0x7f3db80990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:45.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.372+0000 7f3dbe200640 1 -- 172.21.15.100:0/618101299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3db80b3f30 msgr2=0x7f3db80d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:45.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.372+0000 7f3dbe200640 1 --2- 172.21.15.100:0/618101299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3db80b3f30 0x7f3db80d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:45.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.372+0000 7f3dbe200640 1 -- 172.21.15.100:0/618101299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3db80cd1d0 msgr2=0x7f3db80990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:45.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.372+0000 7f3dbe200640 1 --2- 172.21.15.100:0/618101299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3db80cd1d0 0x7f3db80990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:45.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.372+0000 7f3dbe200640 1 -- 172.21.15.100:0/618101299 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db81f9a70 con 0x7f3db815d6e0 2024-09-17T04:27:45.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.372+0000 7f3dbce00640 1 --2- 172.21.15.100:0/618101299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3db80cd1d0 0x7f3db80990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:27:45.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.372+0000 7f3dbe200640 1 --2- 172.21.15.100:0/618101299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3db815d6e0 0x7f3db80995f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3db000c730 tx=0x7f3db000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:45.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.373+0000 7f3db5600640 1 -- 172.21.15.100:0/618101299 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db0003b70 con 0x7f3db815d6e0 2024-09-17T04:27:45.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.373+0000 7f3db5600640 1 -- 172.21.15.100:0/618101299 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3db0003d10 con 0x7f3db815d6e0 2024-09-17T04:27:45.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.373+0000 7f3db5600640 1 -- 172.21.15.100:0/618101299 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db000aab0 con 0x7f3db815d6e0 2024-09-17T04:27:45.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.373+0000 7f3dbec00640 1 -- 172.21.15.100:0/618101299 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3db81f9d50 con 0x7f3db815d6e0 2024-09-17T04:27:45.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.373+0000 7f3dbec00640 1 -- 172.21.15.100:0/618101299 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3db81f8680 con 0x7f3db815d6e0 2024-09-17T04:27:45.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.374+0000 7f3dbec00640 1 -- 172.21.15.100:0/618101299 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3db81f97e0 con 0x7f3db815d6e0 2024-09-17T04:27:45.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.375+0000 7f3db5600640 1 -- 172.21.15.100:0/618101299 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f3db000ac50 con 0x7f3db815d6e0 2024-09-17T04:27:45.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.376+0000 7f3db5600640 1 --2- 172.21.15.100:0/618101299 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3d98078d40 0x7f3d9807b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:45.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.376+0000 7f3db5600640 1 -- 172.21.15.100:0/618101299 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7f3db009aed0 con 0x7f3db815d6e0 2024-09-17T04:27:45.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.377+0000 7f3dbd800640 1 --2- 172.21.15.100:0/618101299 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3d98078d40 0x7f3d9807b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:45.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.377+0000 7f3dbd800640 1 --2- 172.21.15.100:0/618101299 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3d98078d40 0x7f3d9807b250 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3dac034000 tx=0x7f3dac004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:45.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.380+0000 7f3db5600640 1 -- 172.21.15.100:0/618101299 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3db0062f70 con 0x7f3db815d6e0 2024-09-17T04:27:45.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:45.599+0000 7f3dbec00640 1 -- 172.21.15.100:0/618101299 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3db81fa270 con 0x7f3db815d6e0 2024-09-17T04:27:46.175 INFO:tasks.ceph.mds.a.smithi100.stderr:2024-09-17T04:27:46.175+0000 7f9935400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T04:27:46.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.218+0000 7f3db5600640 1 -- 172.21.15.100:0/618101299 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v35) ==== 118+0+0 (secure 0 0 0) 0x7f3db81fa270 con 0x7f3db815d6e0 2024-09-17T04:27:46.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.221+0000 7f3dbec00640 1 -- 172.21.15.100:0/618101299 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3d98078d40 msgr2=0x7f3d9807b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:46.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.221+0000 7f3dbec00640 1 --2- 172.21.15.100:0/618101299 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3d98078d40 0x7f3d9807b250 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3dac034000 tx=0x7f3dac004730 comp rx=0 tx=0).stop 2024-09-17T04:27:46.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.221+0000 7f3dbec00640 1 -- 172.21.15.100:0/618101299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3db815d6e0 msgr2=0x7f3db80995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:46.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.221+0000 7f3dbec00640 1 --2- 172.21.15.100:0/618101299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3db815d6e0 0x7f3db80995f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3db000c730 tx=0x7f3db000cbf0 comp rx=0 tx=0).stop 2024-09-17T04:27:46.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.221+0000 7f3dbec00640 1 -- 172.21.15.100:0/618101299 shutdown_connections 2024-09-17T04:27:46.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.221+0000 7f3dbec00640 1 --2- 172.21.15.100:0/618101299 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3d98078d40 0x7f3d9807b250 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:46.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.221+0000 7f3dbec00640 1 --2- 172.21.15.100:0/618101299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3db815d6e0 0x7f3db80995f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:46.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.221+0000 7f3dbec00640 1 --2- 172.21.15.100:0/618101299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3db80cd1d0 0x7f3db80990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:46.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.221+0000 7f3dbec00640 1 --2- 172.21.15.100:0/618101299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3db80b3f30 0x7f3db80d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:46.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.221+0000 7f3dbec00640 1 -- 172.21.15.100:0/618101299 >> 172.21.15.100:0/618101299 conn(0x7f3db806d620 msgr2=0x7f3db815b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:46.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.221+0000 7f3dbec00640 1 -- 172.21.15.100:0/618101299 shutdown_connections 2024-09-17T04:27:46.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.221+0000 7f3dbec00640 1 -- 172.21.15.100:0/618101299 wait complete. 2024-09-17T04:27:46.243 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T04:27:46.243 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T04:27:46.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.466+0000 7f38c9e00640 1 -- 172.21.15.100:0/677768992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38c40cd1d0 msgr2=0x7f38c40af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:46.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.466+0000 7f38c9e00640 1 --2- 172.21.15.100:0/677768992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38c40cd1d0 0x7f38c40af0d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f38c4075230 tx=0x7f38bc02aac0 comp rx=0 tx=0).stop 2024-09-17T04:27:46.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.466+0000 7f38c9e00640 1 -- 172.21.15.100:0/677768992 shutdown_connections 2024-09-17T04:27:46.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.466+0000 7f38c9e00640 1 --2- 172.21.15.100:0/677768992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38c40cd1d0 0x7f38c40af0d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:46.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.466+0000 7f38c9e00640 1 --2- 172.21.15.100:0/677768992 >> v2:172.21.15.184:3300/0 conn(0x7f38c40b3f30 0x7f38c4156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:46.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.466+0000 7f38c9e00640 1 -- 172.21.15.100:0/677768992 >> 172.21.15.100:0/677768992 conn(0x7f38c406d620 msgr2=0x7f38c40b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:46.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.466+0000 7f38c9e00640 1 -- 172.21.15.100:0/677768992 shutdown_connections 2024-09-17T04:27:46.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.466+0000 7f38c9e00640 1 -- 172.21.15.100:0/677768992 wait complete. 2024-09-17T04:27:46.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.469+0000 7f38c9e00640 1 Processor -- start 2024-09-17T04:27:46.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.469+0000 7f38c9e00640 1 -- start start 2024-09-17T04:27:46.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.470+0000 7f38c9e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38c40b3f30 0x7f38c41f5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:46.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.470+0000 7f38c9e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38c40cd1d0 0x7f38c41443d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:46.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.470+0000 7f38c9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38c415d6e0 0x7f38c4144910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:46.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.470+0000 7f38c9e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f38c40af9a0 con 0x7f38c415d6e0 2024-09-17T04:27:46.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.470+0000 7f38c9e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f38c40af820 con 0x7f38c40b3f30 2024-09-17T04:27:46.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.470+0000 7f38c9e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f38c40afb20 con 0x7f38c40cd1d0 2024-09-17T04:27:46.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.470+0000 7f38c9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38c415d6e0 0x7f38c4144910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:46.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.471+0000 7f38c9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38c415d6e0 0x7f38c4144910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56766/0 (socket says 172.21.15.100:56766) 2024-09-17T04:27:46.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.471+0000 7f38c9400640 1 -- 172.21.15.100:0/2185944397 learned_addr learned my addr 172.21.15.100:0/2185944397 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:46.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.471+0000 7f38c8a00640 1 --2- 172.21.15.100:0/2185944397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38c40b3f30 0x7f38c41f5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:46.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.471+0000 7f38bbe00640 1 --2- 172.21.15.100:0/2185944397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38c40cd1d0 0x7f38c41443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:46.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.471+0000 7f38c9400640 1 -- 172.21.15.100:0/2185944397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38c40cd1d0 msgr2=0x7f38c41443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:46.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.471+0000 7f38c9400640 1 --2- 172.21.15.100:0/2185944397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38c40cd1d0 0x7f38c41443d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:46.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.471+0000 7f38c9400640 1 -- 172.21.15.100:0/2185944397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38c40b3f30 msgr2=0x7f38c41f5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:46.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.471+0000 7f38c9400640 1 --2- 172.21.15.100:0/2185944397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38c40b3f30 0x7f38c41f5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:46.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.471+0000 7f38c9400640 1 -- 172.21.15.100:0/2185944397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c4074110 con 0x7f38c415d6e0 2024-09-17T04:27:46.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.471+0000 7f38c8a00640 1 --2- 172.21.15.100:0/2185944397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38c40b3f30 0x7f38c41f5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:27:46.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.471+0000 7f38c9400640 1 --2- 172.21.15.100:0/2185944397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38c415d6e0 0x7f38c4144910 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f38bc02b0b0 tx=0x7f38bc003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:46.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.471+0000 7f38c1600640 1 -- 172.21.15.100:0/2185944397 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc003e50 con 0x7f38c415d6e0 2024-09-17T04:27:46.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.472+0000 7f38c1600640 1 -- 172.21.15.100:0/2185944397 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38bc004250 con 0x7f38c415d6e0 2024-09-17T04:27:46.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.472+0000 7f38c1600640 1 -- 172.21.15.100:0/2185944397 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc004680 con 0x7f38c415d6e0 2024-09-17T04:27:46.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.472+0000 7f38c9e00640 1 -- 172.21.15.100:0/2185944397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c40743f0 con 0x7f38c415d6e0 2024-09-17T04:27:46.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.472+0000 7f38c9e00640 1 -- 172.21.15.100:0/2185944397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c4074b50 con 0x7f38c415d6e0 2024-09-17T04:27:46.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.473+0000 7f38c9e00640 1 -- 172.21.15.100:0/2185944397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38c4156ca0 con 0x7f38c415d6e0 2024-09-17T04:27:46.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.474+0000 7f38c1600640 1 -- 172.21.15.100:0/2185944397 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f38bc049430 con 0x7f38c415d6e0 2024-09-17T04:27:46.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.475+0000 7f38c1600640 1 --2- 172.21.15.100:0/2185944397 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f38a0078f10 0x7f38a007b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:46.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.475+0000 7f38c1600640 1 -- 172.21.15.100:0/2185944397 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7f38bc0bbe90 con 0x7f38c415d6e0 2024-09-17T04:27:46.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.475+0000 7f38c8a00640 1 --2- 172.21.15.100:0/2185944397 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f38a0078f10 0x7f38a007b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:46.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.476+0000 7f38c8a00640 1 --2- 172.21.15.100:0/2185944397 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f38a0078f10 0x7f38a007b420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f38b4009b80 tx=0x7f38b4005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:46.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.479+0000 7f38c1600640 1 -- 172.21.15.100:0/2185944397 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f38bc032030 con 0x7f38c415d6e0 2024-09-17T04:27:46.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:46.700+0000 7f38c9e00640 1 -- 172.21.15.100:0/2185944397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f38c4074dc0 con 0x7f38c415d6e0 2024-09-17T04:27:47.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.220+0000 7f38c1600640 1 -- 172.21.15.100:0/2185944397 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7f38bc004820 con 0x7f38c415d6e0 2024-09-17T04:27:47.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.223+0000 7f38c9e00640 1 -- 172.21.15.100:0/2185944397 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f38a0078f10 msgr2=0x7f38a007b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:47.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.223+0000 7f38c9e00640 1 --2- 172.21.15.100:0/2185944397 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f38a0078f10 0x7f38a007b420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f38b4009b80 tx=0x7f38b4005f90 comp rx=0 tx=0).stop 2024-09-17T04:27:47.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.223+0000 7f38c9e00640 1 -- 172.21.15.100:0/2185944397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38c415d6e0 msgr2=0x7f38c4144910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:47.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.223+0000 7f38c9e00640 1 --2- 172.21.15.100:0/2185944397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38c415d6e0 0x7f38c4144910 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f38bc02b0b0 tx=0x7f38bc003ce0 comp rx=0 tx=0).stop 2024-09-17T04:27:47.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.223+0000 7f38c9e00640 1 -- 172.21.15.100:0/2185944397 shutdown_connections 2024-09-17T04:27:47.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.223+0000 7f38c9e00640 1 --2- 172.21.15.100:0/2185944397 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f38a0078f10 0x7f38a007b420 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:47.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.223+0000 7f38c9e00640 1 --2- 172.21.15.100:0/2185944397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38c415d6e0 0x7f38c4144910 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:47.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.223+0000 7f38c9e00640 1 --2- 172.21.15.100:0/2185944397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38c40cd1d0 0x7f38c41443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:47.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.223+0000 7f38c9e00640 1 --2- 172.21.15.100:0/2185944397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38c40b3f30 0x7f38c41f5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:47.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.223+0000 7f38c9e00640 1 -- 172.21.15.100:0/2185944397 >> 172.21.15.100:0/2185944397 conn(0x7f38c406d620 msgr2=0x7f38c40b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:47.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.223+0000 7f38c9e00640 1 -- 172.21.15.100:0/2185944397 shutdown_connections 2024-09-17T04:27:47.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.223+0000 7f38c9e00640 1 -- 172.21.15.100:0/2185944397 wait complete. 2024-09-17T04:27:47.247 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T04:27:47.247 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:27:47.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.467+0000 7ff3f4e00640 1 -- 172.21.15.100:0/3120453237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3f00ad0e0 msgr2=0x7ff3f00ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:47.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.467+0000 7ff3f4e00640 1 --2- 172.21.15.100:0/3120453237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3f00ad0e0 0x7ff3f00ad4f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff3f00b3100 tx=0x7ff3d802a4a0 comp rx=0 tx=0).stop 2024-09-17T04:27:47.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.467+0000 7ff3f4e00640 1 -- 172.21.15.100:0/3120453237 shutdown_connections 2024-09-17T04:27:47.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.467+0000 7ff3f4e00640 1 --2- 172.21.15.100:0/3120453237 >> v2:172.21.15.184:3300/0 conn(0x7ff3f00b3550 0x7ff3f00b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:47.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.467+0000 7ff3f4e00640 1 --2- 172.21.15.100:0/3120453237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3f00ad0e0 0x7ff3f00ad4f0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:47.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.467+0000 7ff3f4e00640 1 -- 172.21.15.100:0/3120453237 >> 172.21.15.100:0/3120453237 conn(0x7ff3f006d730 msgr2=0x7ff3f00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:47.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.467+0000 7ff3f4e00640 1 -- 172.21.15.100:0/3120453237 shutdown_connections 2024-09-17T04:27:47.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.467+0000 7ff3f4e00640 1 -- 172.21.15.100:0/3120453237 wait complete. 2024-09-17T04:27:47.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.469+0000 7ff3f4e00640 1 Processor -- start 2024-09-17T04:27:47.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.469+0000 7ff3f4e00640 1 -- start start 2024-09-17T04:27:47.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.471+0000 7ff3f4e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3f00ad0e0 0x7ff3f0099670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:47.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.471+0000 7ff3f4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3f00b3550 0x7ff3f0156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:47.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.471+0000 7ff3f4e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3f015d760 0x7ff3f0156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:47.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.471+0000 7ff3f4e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f00b4da0 con 0x7ff3f00b3550 2024-09-17T04:27:47.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.471+0000 7ff3f4e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f00b4c20 con 0x7ff3f00ad0e0 2024-09-17T04:27:47.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.471+0000 7ff3f4e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff3f00b3ea0 con 0x7ff3f015d760 2024-09-17T04:27:47.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.471+0000 7ff3e5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3f00b3550 0x7ff3f0156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:47.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.471+0000 7ff3e5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3f00b3550 0x7ff3f0156410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56784/0 (socket says 172.21.15.100:56784) 2024-09-17T04:27:47.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.471+0000 7ff3e5400640 1 -- 172.21.15.100:0/2585225753 learned_addr learned my addr 172.21.15.100:0/2585225753 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:47.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.471+0000 7ff3ede00640 1 --2- 172.21.15.100:0/2585225753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3f00ad0e0 0x7ff3f0099670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:47.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.471+0000 7ff3ee800640 1 --2- 172.21.15.100:0/2585225753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3f015d760 0x7ff3f0156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:47.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.472+0000 7ff3e5400640 1 -- 172.21.15.100:0/2585225753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3f015d760 msgr2=0x7ff3f0156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:47.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.472+0000 7ff3e5400640 1 --2- 172.21.15.100:0/2585225753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3f015d760 0x7ff3f0156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:47.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.472+0000 7ff3e5400640 1 -- 172.21.15.100:0/2585225753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3f00ad0e0 msgr2=0x7ff3f0099670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:47.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.472+0000 7ff3e5400640 1 --2- 172.21.15.100:0/2585225753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3f00ad0e0 0x7ff3f0099670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:47.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.472+0000 7ff3e5400640 1 -- 172.21.15.100:0/2585225753 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f0157630 con 0x7ff3f00b3550 2024-09-17T04:27:47.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.472+0000 7ff3e5400640 1 --2- 172.21.15.100:0/2585225753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3f00b3550 0x7ff3f0156410 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff3d802aa90 tx=0x7ff3d8033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:47.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.472+0000 7ff3e6a00640 1 -- 172.21.15.100:0/2585225753 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8031d60 con 0x7ff3f00b3550 2024-09-17T04:27:47.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.472+0000 7ff3e6a00640 1 -- 172.21.15.100:0/2585225753 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3d8033e10 con 0x7ff3f00b3550 2024-09-17T04:27:47.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.472+0000 7ff3f4e00640 1 -- 172.21.15.100:0/2585225753 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f0157910 con 0x7ff3f00b3550 2024-09-17T04:27:47.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.472+0000 7ff3e6a00640 1 -- 172.21.15.100:0/2585225753 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8005260 con 0x7ff3f00b3550 2024-09-17T04:27:47.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.472+0000 7ff3f4e00640 1 -- 172.21.15.100:0/2585225753 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f0157ed0 con 0x7ff3f00b3550 2024-09-17T04:27:47.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.473+0000 7ff3f4e00640 1 -- 172.21.15.100:0/2585225753 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3f00b4360 con 0x7ff3f00b3550 2024-09-17T04:27:47.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.475+0000 7ff3e6a00640 1 -- 172.21.15.100:0/2585225753 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7ff3d8005400 con 0x7ff3f00b3550 2024-09-17T04:27:47.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.476+0000 7ff3e6a00640 1 --2- 172.21.15.100:0/2585225753 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff3cc078d70 0x7ff3cc07b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:47.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.476+0000 7ff3e6a00640 1 -- 172.21.15.100:0/2585225753 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7ff3d80bc290 con 0x7ff3f00b3550 2024-09-17T04:27:47.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.477+0000 7ff3ede00640 1 --2- 172.21.15.100:0/2585225753 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff3cc078d70 0x7ff3cc07b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:47.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.477+0000 7ff3ede00640 1 --2- 172.21.15.100:0/2585225753 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff3cc078d70 0x7ff3cc07b280 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0007fd0 tx=0x7ff3e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:47.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.481+0000 7ff3e6a00640 1 -- 172.21.15.100:0/2585225753 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff3d803f030 con 0x7ff3f00b3550 2024-09-17T04:27:47.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.697+0000 7ff3f4e00640 1 -- 172.21.15.100:0/2585225753 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3f01573c0 con 0x7ff3f00b3550 2024-09-17T04:27:47.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.698+0000 7ff3e6a00640 1 -- 172.21.15.100:0/2585225753 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5229 (secure 0 0 0) 0x7ff3d8005d00 con 0x7ff3f00b3550 2024-09-17T04:27:47.698 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:27:47.698 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":36,"btime":"2024-09-17T04:27:47:219814+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1735051989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1735051989},{"type":"v1","addr":"172.21.15.184:6836","nonce":1735051989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5638,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3935107098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":3935107098},{"type":"v1","addr":"172.21.15.184:6837","nonce":3935107098}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/904123596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904123596},{"type":"v1","addr":"172.21.15.100:6834","nonce":904123596}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:27:43.682593+0000","modified":"2024-09-17T04:27:46.702665+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/270906112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":270906112},{"type":"v1","addr":"172.21.15.100:6835","nonce":270906112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5739,"qdb_cluster":[5739]},"id":4}]} 2024-09-17T04:27:47.699 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 36 2024-09-17T04:27:47.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.700+0000 7ff3f4e00640 1 -- 172.21.15.100:0/2585225753 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff3cc078d70 msgr2=0x7ff3cc07b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:47.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.700+0000 7ff3f4e00640 1 --2- 172.21.15.100:0/2585225753 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff3cc078d70 0x7ff3cc07b280 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0007fd0 tx=0x7ff3e0009040 comp rx=0 tx=0).stop 2024-09-17T04:27:47.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.700+0000 7ff3f4e00640 1 -- 172.21.15.100:0/2585225753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3f00b3550 msgr2=0x7ff3f0156410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:47.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.700+0000 7ff3f4e00640 1 --2- 172.21.15.100:0/2585225753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3f00b3550 0x7ff3f0156410 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff3d802aa90 tx=0x7ff3d8033c20 comp rx=0 tx=0).stop 2024-09-17T04:27:47.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.700+0000 7ff3f4e00640 1 -- 172.21.15.100:0/2585225753 shutdown_connections 2024-09-17T04:27:47.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.700+0000 7ff3f4e00640 1 --2- 172.21.15.100:0/2585225753 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff3cc078d70 0x7ff3cc07b280 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:47.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.700+0000 7ff3f4e00640 1 --2- 172.21.15.100:0/2585225753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3f015d760 0x7ff3f0156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:47.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.700+0000 7ff3f4e00640 1 --2- 172.21.15.100:0/2585225753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3f00b3550 0x7ff3f0156410 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:47.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.700+0000 7ff3f4e00640 1 --2- 172.21.15.100:0/2585225753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3f00ad0e0 0x7ff3f0099670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:47.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.700+0000 7ff3f4e00640 1 -- 172.21.15.100:0/2585225753 >> 172.21.15.100:0/2585225753 conn(0x7ff3f006d730 msgr2=0x7ff3f00cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:47.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.700+0000 7ff3f4e00640 1 -- 172.21.15.100:0/2585225753 shutdown_connections 2024-09-17T04:27:47.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.700+0000 7ff3f4e00640 1 -- 172.21.15.100:0/2585225753 wait complete. 2024-09-17T04:27:47.722 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:27:47.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.942+0000 7feb58e00640 1 -- 172.21.15.100:0/2388932710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb38003a00 msgr2=0x7feb38003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:47.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.942+0000 7feb58e00640 1 --2- 172.21.15.100:0/2388932710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb38003a00 0x7feb38003e70 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7feb44009b50 tx=0x7feb38005830 comp rx=0 tx=0).stop 2024-09-17T04:27:47.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.942+0000 7feb52800640 1 -- 172.21.15.100:0/2388932710 reap_dead start 2024-09-17T04:27:47.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.942+0000 7feb58e00640 1 -- 172.21.15.100:0/2388932710 shutdown_connections 2024-09-17T04:27:47.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.942+0000 7feb58e00640 1 -- 172.21.15.100:0/2388932710 >> 172.21.15.100:0/2388932710 conn(0x7feb5406d730 msgr2=0x7feb540b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:47.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.942+0000 7feb58e00640 1 -- 172.21.15.100:0/2388932710 shutdown_connections 2024-09-17T04:27:47.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.942+0000 7feb58e00640 1 -- 172.21.15.100:0/2388932710 wait complete. 2024-09-17T04:27:47.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.945+0000 7feb58e00640 1 Processor -- start 2024-09-17T04:27:47.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.945+0000 7feb58e00640 1 -- start start 2024-09-17T04:27:47.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.946+0000 7feb58e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb38003a00 0x7feb54146780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:47.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.946+0000 7feb58e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb5409a4a0 0x7feb5409c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:47.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.946+0000 7feb58e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb54145340 0x7feb5415ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:47.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.946+0000 7feb58e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7feb5406f220 con 0x7feb38003a00 2024-09-17T04:27:47.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.946+0000 7feb58e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feb5406f0a0 con 0x7feb54145340 2024-09-17T04:27:47.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.946+0000 7feb58e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7feb540b4c20 con 0x7feb5409a4a0 2024-09-17T04:27:47.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.946+0000 7feb51e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb38003a00 0x7feb54146780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:47.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.946+0000 7feb51e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb38003a00 0x7feb54146780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56802/0 (socket says 172.21.15.100:56802) 2024-09-17T04:27:47.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.946+0000 7feb51e00640 1 -- 172.21.15.100:0/3190625236 learned_addr learned my addr 172.21.15.100:0/3190625236 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:47.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.947+0000 7feb52800640 1 --2- 172.21.15.100:0/3190625236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb54145340 0x7feb5415ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:47.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.947+0000 7feb4be00640 1 --2- 172.21.15.100:0/3190625236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb5409a4a0 0x7feb5409c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:47.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.947+0000 7feb51e00640 1 -- 172.21.15.100:0/3190625236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb5409a4a0 msgr2=0x7feb5409c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:47.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.947+0000 7feb51e00640 1 --2- 172.21.15.100:0/3190625236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb5409a4a0 0x7feb5409c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:47.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.947+0000 7feb51e00640 1 -- 172.21.15.100:0/3190625236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb54145340 msgr2=0x7feb5415ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:47.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.947+0000 7feb51e00640 1 --2- 172.21.15.100:0/3190625236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb54145340 0x7feb5415ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:47.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.947+0000 7feb51e00640 1 -- 172.21.15.100:0/3190625236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb54146cc0 con 0x7feb38003a00 2024-09-17T04:27:47.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.947+0000 7feb51e00640 1 --2- 172.21.15.100:0/3190625236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb38003a00 0x7feb54146780 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7feb44009a70 tx=0x7feb44033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:47.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.947+0000 7feb4a000640 1 -- 172.21.15.100:0/3190625236 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb44038040 con 0x7feb38003a00 2024-09-17T04:27:47.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.947+0000 7feb4a000640 1 -- 172.21.15.100:0/3190625236 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb44004020 con 0x7feb38003a00 2024-09-17T04:27:47.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.947+0000 7feb58e00640 1 -- 172.21.15.100:0/3190625236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb541442a0 con 0x7feb38003a00 2024-09-17T04:27:47.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.948+0000 7feb4a000640 1 -- 172.21.15.100:0/3190625236 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb44004470 con 0x7feb38003a00 2024-09-17T04:27:47.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.948+0000 7feb58e00640 1 -- 172.21.15.100:0/3190625236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb54144860 con 0x7feb38003a00 2024-09-17T04:27:47.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.949+0000 7feb58e00640 1 -- 172.21.15.100:0/3190625236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb54160fa0 con 0x7feb38003a00 2024-09-17T04:27:47.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.950+0000 7feb4a000640 1 -- 172.21.15.100:0/3190625236 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7feb44045020 con 0x7feb38003a00 2024-09-17T04:27:47.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.951+0000 7feb4a000640 1 --2- 172.21.15.100:0/3190625236 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7feb30078d70 0x7feb3007b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:47.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.951+0000 7feb4a000640 1 -- 172.21.15.100:0/3190625236 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7feb440bbb00 con 0x7feb38003a00 2024-09-17T04:27:47.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.951+0000 7feb4be00640 1 --2- 172.21.15.100:0/3190625236 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7feb30078d70 0x7feb3007b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:47.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.952+0000 7feb4be00640 1 --2- 172.21.15.100:0/3190625236 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7feb30078d70 0x7feb3007b280 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7feb5415cd20 tx=0x7feb3c005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:47.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:47.955+0000 7feb4a000640 1 -- 172.21.15.100:0/3190625236 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7feb54146cc0 con 0x7feb38003a00 2024-09-17T04:27:48.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.142+0000 7feb58e00640 1 -- 172.21.15.100:0/3190625236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb540b22a0 con 0x7feb38003a00 2024-09-17T04:27:48.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.143+0000 7feb4a000640 1 -- 172.21.15.100:0/3190625236 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+16773 (secure 0 0 0) 0x7feb440bbe20 con 0x7feb38003a00 2024-09-17T04:27:48.144 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:27:48.144 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":61,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:27:45.203997+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:27:41.608179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-17T04:27:42.645978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:27:48.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.146+0000 7feb58e00640 1 -- 172.21.15.100:0/3190625236 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7feb30078d70 msgr2=0x7feb3007b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:48.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.146+0000 7feb58e00640 1 --2- 172.21.15.100:0/3190625236 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7feb30078d70 0x7feb3007b280 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7feb5415cd20 tx=0x7feb3c005f90 comp rx=0 tx=0).stop 2024-09-17T04:27:48.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.146+0000 7feb58e00640 1 -- 172.21.15.100:0/3190625236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb38003a00 msgr2=0x7feb54146780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:48.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.146+0000 7feb58e00640 1 --2- 172.21.15.100:0/3190625236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb38003a00 0x7feb54146780 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7feb44009a70 tx=0x7feb44033670 comp rx=0 tx=0).stop 2024-09-17T04:27:48.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.146+0000 7feb58e00640 1 -- 172.21.15.100:0/3190625236 shutdown_connections 2024-09-17T04:27:48.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.146+0000 7feb58e00640 1 --2- 172.21.15.100:0/3190625236 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7feb30078d70 0x7feb3007b280 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:48.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.146+0000 7feb58e00640 1 --2- 172.21.15.100:0/3190625236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb54145340 0x7feb5415ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:48.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.146+0000 7feb58e00640 1 --2- 172.21.15.100:0/3190625236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb5409a4a0 0x7feb5409c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:48.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.146+0000 7feb58e00640 1 --2- 172.21.15.100:0/3190625236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb38003a00 0x7feb54146780 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:48.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.146+0000 7feb58e00640 1 -- 172.21.15.100:0/3190625236 >> 172.21.15.100:0/3190625236 conn(0x7feb5406d730 msgr2=0x7feb540cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:48.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.146+0000 7feb58e00640 1 -- 172.21.15.100:0/3190625236 shutdown_connections 2024-09-17T04:27:48.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.146+0000 7feb58e00640 1 -- 172.21.15.100:0/3190625236 wait complete. 2024-09-17T04:27:48.172 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T04:27:48.173 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:27:48.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.393+0000 7fc434600640 1 -- 172.21.15.100:0/2285100891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc42c0b3550 msgr2=0x7fc42c0b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:48.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.393+0000 7fc434600640 1 --2- 172.21.15.100:0/2285100891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc42c0b3550 0x7fc42c0b3960 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc42c072d60 tx=0x7fc42402aa30 comp rx=0 tx=0).stop 2024-09-17T04:27:48.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.393+0000 7fc434600640 1 -- 172.21.15.100:0/2285100891 shutdown_connections 2024-09-17T04:27:48.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.393+0000 7fc434600640 1 --2- 172.21.15.100:0/2285100891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc42c0b3550 0x7fc42c0b3960 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:48.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.393+0000 7fc434600640 1 --2- 172.21.15.100:0/2285100891 >> v2:172.21.15.184:3300/0 conn(0x7fc42c0ad0e0 0x7fc42c0ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:48.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.393+0000 7fc434600640 1 -- 172.21.15.100:0/2285100891 >> 172.21.15.100:0/2285100891 conn(0x7fc42c06d730 msgr2=0x7fc42c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:48.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.394+0000 7fc434600640 1 -- 172.21.15.100:0/2285100891 shutdown_connections 2024-09-17T04:27:48.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.394+0000 7fc434600640 1 -- 172.21.15.100:0/2285100891 wait complete. 2024-09-17T04:27:48.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.396+0000 7fc434600640 1 Processor -- start 2024-09-17T04:27:48.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.396+0000 7fc434600640 1 -- start start 2024-09-17T04:27:48.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.397+0000 7fc434600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc42c0ad0e0 0x7fc42c150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:48.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.397+0000 7fc434600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc42c0b3550 0x7fc42c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:48.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.397+0000 7fc434600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc42c15d760 0x7fc42c15b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:48.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.397+0000 7fc434600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c0b4da0 con 0x7fc42c0b3550 2024-09-17T04:27:48.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.397+0000 7fc434600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c0b4c20 con 0x7fc42c15d760 2024-09-17T04:27:48.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.397+0000 7fc434600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc42c0b3ea0 con 0x7fc42c0ad0e0 2024-09-17T04:27:48.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.398+0000 7fc431200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc42c0b3550 0x7fc42c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:48.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.398+0000 7fc431200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc42c0b3550 0x7fc42c074aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56824/0 (socket says 172.21.15.100:56824) 2024-09-17T04:27:48.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.398+0000 7fc431200640 1 -- 172.21.15.100:0/3657111508 learned_addr learned my addr 172.21.15.100:0/3657111508 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:48.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.398+0000 7fc432600640 1 --2- 172.21.15.100:0/3657111508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc42c15d760 0x7fc42c15b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:48.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.398+0000 7fc431c00640 1 --2- 172.21.15.100:0/3657111508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc42c0ad0e0 0x7fc42c150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:48.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.398+0000 7fc431200640 1 -- 172.21.15.100:0/3657111508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc42c0ad0e0 msgr2=0x7fc42c150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:48.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.398+0000 7fc431200640 1 --2- 172.21.15.100:0/3657111508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc42c0ad0e0 0x7fc42c150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:48.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.398+0000 7fc431200640 1 -- 172.21.15.100:0/3657111508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc42c15d760 msgr2=0x7fc42c15b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:48.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.398+0000 7fc431200640 1 --2- 172.21.15.100:0/3657111508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc42c15d760 0x7fc42c15b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:48.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.398+0000 7fc431200640 1 -- 172.21.15.100:0/3657111508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42c150ea0 con 0x7fc42c0b3550 2024-09-17T04:27:48.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.399+0000 7fc431200640 1 --2- 172.21.15.100:0/3657111508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc42c0b3550 0x7fc42c074aa0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc418009fd0 tx=0x7fc41800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:48.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.399+0000 7fc429600640 1 -- 172.21.15.100:0/3657111508 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41800ccc0 con 0x7fc42c0b3550 2024-09-17T04:27:48.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.399+0000 7fc429600640 1 -- 172.21.15.100:0/3657111508 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc41800ee70 con 0x7fc42c0b3550 2024-09-17T04:27:48.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.399+0000 7fc429600640 1 -- 172.21.15.100:0/3657111508 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41800ce60 con 0x7fc42c0b3550 2024-09-17T04:27:48.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.399+0000 7fc434600640 1 -- 172.21.15.100:0/3657111508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc42c1f59b0 con 0x7fc42c0b3550 2024-09-17T04:27:48.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.399+0000 7fc434600640 1 -- 172.21.15.100:0/3657111508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc42c1f5f50 con 0x7fc42c0b3550 2024-09-17T04:27:48.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.400+0000 7fc434600640 1 -- 172.21.15.100:0/3657111508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc42c0b4360 con 0x7fc42c0b3550 2024-09-17T04:27:48.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.401+0000 7fc429600640 1 -- 172.21.15.100:0/3657111508 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7fc418004280 con 0x7fc42c0b3550 2024-09-17T04:27:48.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.404+0000 7fc429600640 1 --2- 172.21.15.100:0/3657111508 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc40c078d40 0x7fc40c07b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:48.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.404+0000 7fc429600640 1 -- 172.21.15.100:0/3657111508 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7fc41809b9f0 con 0x7fc42c0b3550 2024-09-17T04:27:48.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.404+0000 7fc431c00640 1 --2- 172.21.15.100:0/3657111508 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc40c078d40 0x7fc40c07b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:48.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.404+0000 7fc431c00640 1 --2- 172.21.15.100:0/3657111508 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc40c078d40 0x7fc40c07b250 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc420001720 tx=0x7fc420009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:48.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.407+0000 7fc429600640 1 -- 172.21.15.100:0/3657111508 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc418063440 con 0x7fc42c0b3550 2024-09-17T04:27:48.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.586+0000 7fc434600640 1 -- 172.21.15.100:0/3657111508 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc42c074fe0 con 0x7fc40c078d40 2024-09-17T04:27:48.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.592+0000 7fc429600640 1 -- 172.21.15.100:0/3657111508 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245270 (secure 0 0 0) 0x7fc42c074fe0 con 0x7fc40c078d40 2024-09-17T04:27:48.595 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:27:48.596 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:27:48.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.597+0000 7fc434600640 1 -- 172.21.15.100:0/3657111508 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc40c078d40 msgr2=0x7fc40c07b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:48.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.597+0000 7fc434600640 1 --2- 172.21.15.100:0/3657111508 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc40c078d40 0x7fc40c07b250 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc420001720 tx=0x7fc420009040 comp rx=0 tx=0).stop 2024-09-17T04:27:48.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.597+0000 7fc434600640 1 -- 172.21.15.100:0/3657111508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc42c0b3550 msgr2=0x7fc42c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:48.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.597+0000 7fc434600640 1 --2- 172.21.15.100:0/3657111508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc42c0b3550 0x7fc42c074aa0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc418009fd0 tx=0x7fc41800ec20 comp rx=0 tx=0).stop 2024-09-17T04:27:48.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.597+0000 7fc434600640 1 -- 172.21.15.100:0/3657111508 shutdown_connections 2024-09-17T04:27:48.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.597+0000 7fc434600640 1 --2- 172.21.15.100:0/3657111508 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc40c078d40 0x7fc40c07b250 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:48.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.597+0000 7fc434600640 1 --2- 172.21.15.100:0/3657111508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc42c15d760 0x7fc42c15b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:48.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.597+0000 7fc434600640 1 --2- 172.21.15.100:0/3657111508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc42c0b3550 0x7fc42c074aa0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:48.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.597+0000 7fc434600640 1 --2- 172.21.15.100:0/3657111508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc42c0ad0e0 0x7fc42c150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:48.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.597+0000 7fc434600640 1 -- 172.21.15.100:0/3657111508 >> 172.21.15.100:0/3657111508 conn(0x7fc42c06d730 msgr2=0x7fc42c160c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:48.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.597+0000 7fc434600640 1 -- 172.21.15.100:0/3657111508 shutdown_connections 2024-09-17T04:27:48.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.597+0000 7fc434600640 1 -- 172.21.15.100:0/3657111508 wait complete. 2024-09-17T04:27:48.620 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-09-17T04:27:48.024133+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590424,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":106,"num_write_kb":1636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":150,"acting":150,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":235328,"kb_used_data":6596,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754739392,"statfs":{"total":773094113280,"available":772853137408,"internally_reserved":0,"allocated":6754304,"data_stored":3465394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":234016308},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":56,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.846898"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212739+0000","last_change":"2024-09-17T04:27:44.238721+0000","last_active":"2024-09-17T04:27:45.212739+0000","last_peered":"2024-09-17T04:27:45.212739+0000","last_clean":"2024-09-17T04:27:45.212739+0000","last_became_active":"2024-09-17T04:27:44.237900+0000","last_became_peered":"2024-09-17T04:27:44.237900+0000","last_unstale":"2024-09-17T04:27:45.212739+0000","last_undegraded":"2024-09-17T04:27:45.212739+0000","last_fullsized":"2024-09-17T04:27:45.212739+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:09:57.201730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.188353+0000","last_change":"2024-09-17T04:27:43.188353+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.188353+0000","last_undegraded":"2024-09-17T04:27:43.188353+0000","last_fullsized":"2024-09-17T04:27:43.188353+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.194226+0000","last_change":"2024-09-17T04:27:43.194226+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.194226+0000","last_undegraded":"2024-09-17T04:27:43.194226+0000","last_fullsized":"2024-09-17T04:27:43.194226+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.190634+0000","last_change":"2024-09-17T04:27:43.190634+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.190634+0000","last_undegraded":"2024-09-17T04:27:43.190634+0000","last_fullsized":"2024-09-17T04:27:43.190634+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212387+0000","last_change":"2024-09-17T04:27:44.222903+0000","last_active":"2024-09-17T04:27:45.212387+0000","last_peered":"2024-09-17T04:27:45.212387+0000","last_clean":"2024-09-17T04:27:45.212387+0000","last_became_active":"2024-09-17T04:27:44.222587+0000","last_became_peered":"2024-09-17T04:27:44.222587+0000","last_unstale":"2024-09-17T04:27:45.212387+0000","last_undegraded":"2024-09-17T04:27:45.212387+0000","last_fullsized":"2024-09-17T04:27:45.212387+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:37:39.457226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.189985+0000","last_change":"2024-09-17T04:27:43.189985+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.189985+0000","last_undegraded":"2024-09-17T04:27:43.189985+0000","last_fullsized":"2024-09-17T04:27:43.189985+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.193736+0000","last_change":"2024-09-17T04:27:43.193736+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.193736+0000","last_undegraded":"2024-09-17T04:27:43.193736+0000","last_fullsized":"2024-09-17T04:27:43.193736+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.184126+0000","last_change":"2024-09-17T04:27:43.184126+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.184126+0000","last_undegraded":"2024-09-17T04:27:43.184126+0000","last_fullsized":"2024-09-17T04:27:43.184126+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.189690+0000","last_change":"2024-09-17T04:27:43.189690+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.189690+0000","last_undegraded":"2024-09-17T04:27:43.189690+0000","last_fullsized":"2024-09-17T04:27:43.189690+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.188426+0000","last_change":"2024-09-17T04:27:43.188426+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.188426+0000","last_undegraded":"2024-09-17T04:27:43.188426+0000","last_fullsized":"2024-09-17T04:27:43.188426+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212938+0000","last_change":"2024-09-17T04:27:44.221917+0000","last_active":"2024-09-17T04:27:45.212938+0000","last_peered":"2024-09-17T04:27:45.212938+0000","last_clean":"2024-09-17T04:27:45.212938+0000","last_became_active":"2024-09-17T04:27:44.221632+0000","last_became_peered":"2024-09-17T04:27:44.221632+0000","last_unstale":"2024-09-17T04:27:45.212938+0000","last_undegraded":"2024-09-17T04:27:45.212938+0000","last_fullsized":"2024-09-17T04:27:45.212938+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:27.954596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.185462+0000","last_change":"2024-09-17T04:27:43.185462+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.185462+0000","last_undegraded":"2024-09-17T04:27:43.185462+0000","last_fullsized":"2024-09-17T04:27:43.185462+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212533+0000","last_change":"2024-09-17T04:27:44.220909+0000","last_active":"2024-09-17T04:27:45.212533+0000","last_peered":"2024-09-17T04:27:45.212533+0000","last_clean":"2024-09-17T04:27:45.212533+0000","last_became_active":"2024-09-17T04:27:44.220605+0000","last_became_peered":"2024-09-17T04:27:44.220605+0000","last_unstale":"2024-09-17T04:27:45.212533+0000","last_undegraded":"2024-09-17T04:27:45.212533+0000","last_fullsized":"2024-09-17T04:27:45.212533+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:48:13.238235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.188790+0000","last_change":"2024-09-17T04:27:43.188790+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.188790+0000","last_undegraded":"2024-09-17T04:27:43.188790+0000","last_fullsized":"2024-09-17T04:27:43.188790+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212746+0000","last_change":"2024-09-17T04:27:44.216830+0000","last_active":"2024-09-17T04:27:45.212746+0000","last_peered":"2024-09-17T04:27:45.212746+0000","last_clean":"2024-09-17T04:27:45.212746+0000","last_became_active":"2024-09-17T04:27:44.216520+0000","last_became_peered":"2024-09-17T04:27:44.216520+0000","last_unstale":"2024-09-17T04:27:45.212746+0000","last_undegraded":"2024-09-17T04:27:45.212746+0000","last_fullsized":"2024-09-17T04:27:45.212746+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:15:47.203299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.185548+0000","last_change":"2024-09-17T04:27:43.185548+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.185548+0000","last_undegraded":"2024-09-17T04:27:43.185548+0000","last_fullsized":"2024-09-17T04:27:43.185548+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.185510+0000","last_change":"2024-09-17T04:27:43.185510+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.185510+0000","last_undegraded":"2024-09-17T04:27:43.185510+0000","last_fullsized":"2024-09-17T04:27:43.185510+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.192118+0000","last_change":"2024-09-17T04:27:43.192118+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.192118+0000","last_undegraded":"2024-09-17T04:27:43.192118+0000","last_fullsized":"2024-09-17T04:27:43.192118+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.213050+0000","last_change":"2024-09-17T04:27:44.218277+0000","last_active":"2024-09-17T04:27:45.213050+0000","last_peered":"2024-09-17T04:27:45.213050+0000","last_clean":"2024-09-17T04:27:45.213050+0000","last_became_active":"2024-09-17T04:27:44.217942+0000","last_became_peered":"2024-09-17T04:27:44.217942+0000","last_unstale":"2024-09-17T04:27:45.213050+0000","last_undegraded":"2024-09-17T04:27:45.213050+0000","last_fullsized":"2024-09-17T04:27:45.213050+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:18.353865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.183706+0000","last_change":"2024-09-17T04:27:43.183706+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.183706+0000","last_undegraded":"2024-09-17T04:27:43.183706+0000","last_fullsized":"2024-09-17T04:27:43.183706+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.186596+0000","last_change":"2024-09-17T04:27:42.186596+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.186596+0000","last_undegraded":"2024-09-17T04:27:42.186596+0000","last_fullsized":"2024-09-17T04:27:42.186596+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"60'1","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212757+0000","last_change":"2024-09-17T04:27:43.206766+0000","last_active":"2024-09-17T04:27:45.212757+0000","last_peered":"2024-09-17T04:27:45.212757+0000","last_clean":"2024-09-17T04:27:45.212757+0000","last_became_active":"2024-09-17T04:27:43.206455+0000","last_became_peered":"2024-09-17T04:27:43.206455+0000","last_unstale":"2024-09-17T04:27:45.212757+0000","last_undegraded":"2024-09-17T04:27:45.212757+0000","last_fullsized":"2024-09-17T04:27:45.212757+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:06:22.067347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.186307+0000","last_change":"2024-09-17T04:27:42.186307+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.186307+0000","last_undegraded":"2024-09-17T04:27:42.186307+0000","last_fullsized":"2024-09-17T04:27:42.186307+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.209139+0000","last_change":"2024-09-17T04:27:43.207037+0000","last_active":"2024-09-17T04:27:43.209139+0000","last_peered":"2024-09-17T04:27:43.209139+0000","last_clean":"2024-09-17T04:27:43.209139+0000","last_became_active":"2024-09-17T04:27:43.206672+0000","last_became_peered":"2024-09-17T04:27:43.206672+0000","last_unstale":"2024-09-17T04:27:43.209139+0000","last_undegraded":"2024-09-17T04:27:43.209139+0000","last_fullsized":"2024-09-17T04:27:43.209139+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:10.033325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.213106+0000","last_change":"2024-09-17T04:27:44.217751+0000","last_active":"2024-09-17T04:27:45.213106+0000","last_peered":"2024-09-17T04:27:45.213106+0000","last_clean":"2024-09-17T04:27:45.213106+0000","last_became_active":"2024-09-17T04:27:44.214072+0000","last_became_peered":"2024-09-17T04:27:44.214072+0000","last_unstale":"2024-09-17T04:27:45.213106+0000","last_undegraded":"2024-09-17T04:27:45.213106+0000","last_fullsized":"2024-09-17T04:27:45.213106+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:27.160546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.208764+0000","last_change":"2024-09-17T04:27:43.198922+0000","last_active":"2024-09-17T04:27:43.208764+0000","last_peered":"2024-09-17T04:27:43.208764+0000","last_clean":"2024-09-17T04:27:43.208764+0000","last_became_active":"2024-09-17T04:27:43.198453+0000","last_became_peered":"2024-09-17T04:27:43.198453+0000","last_unstale":"2024-09-17T04:27:43.208764+0000","last_undegraded":"2024-09-17T04:27:43.208764+0000","last_fullsized":"2024-09-17T04:27:43.208764+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:46.166554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212477+0000","last_change":"2024-09-17T04:27:44.222601+0000","last_active":"2024-09-17T04:27:45.212477+0000","last_peered":"2024-09-17T04:27:45.212477+0000","last_clean":"2024-09-17T04:27:45.212477+0000","last_became_active":"2024-09-17T04:27:44.221587+0000","last_became_peered":"2024-09-17T04:27:44.221587+0000","last_unstale":"2024-09-17T04:27:45.212477+0000","last_undegraded":"2024-09-17T04:27:45.212477+0000","last_fullsized":"2024-09-17T04:27:45.212477+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:22.587679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.185403+0000","last_change":"2024-09-17T04:27:42.185403+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.185403+0000","last_undegraded":"2024-09-17T04:27:42.185403+0000","last_fullsized":"2024-09-17T04:27:42.185403+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.183685+0000","last_change":"2024-09-17T04:27:43.183685+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.183685+0000","last_undegraded":"2024-09-17T04:27:43.183685+0000","last_fullsized":"2024-09-17T04:27:43.183685+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.207657+0000","last_change":"2024-09-17T04:27:43.200857+0000","last_active":"2024-09-17T04:27:43.207657+0000","last_peered":"2024-09-17T04:27:43.207657+0000","last_clean":"2024-09-17T04:27:43.207657+0000","last_became_active":"2024-09-17T04:27:43.200518+0000","last_became_peered":"2024-09-17T04:27:43.200518+0000","last_unstale":"2024-09-17T04:27:43.207657+0000","last_undegraded":"2024-09-17T04:27:43.207657+0000","last_fullsized":"2024-09-17T04:27:43.207657+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:11.559902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212733+0000","last_change":"2024-09-17T04:27:44.225414+0000","last_active":"2024-09-17T04:27:45.212733+0000","last_peered":"2024-09-17T04:27:45.212733+0000","last_clean":"2024-09-17T04:27:45.212733+0000","last_became_active":"2024-09-17T04:27:44.225112+0000","last_became_peered":"2024-09-17T04:27:44.225112+0000","last_unstale":"2024-09-17T04:27:45.212733+0000","last_undegraded":"2024-09-17T04:27:45.212733+0000","last_fullsized":"2024-09-17T04:27:45.212733+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:05:18.890521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.213768+0000","last_change":"2024-09-17T04:27:43.209713+0000","last_active":"2024-09-17T04:27:43.213768+0000","last_peered":"2024-09-17T04:27:43.213768+0000","last_clean":"2024-09-17T04:27:43.213768+0000","last_became_active":"2024-09-17T04:27:43.209316+0000","last_became_peered":"2024-09-17T04:27:43.209316+0000","last_unstale":"2024-09-17T04:27:43.213768+0000","last_undegraded":"2024-09-17T04:27:43.213768+0000","last_fullsized":"2024-09-17T04:27:43.213768+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:01.773526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.189782+0000","last_change":"2024-09-17T04:27:43.189782+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.189782+0000","last_undegraded":"2024-09-17T04:27:43.189782+0000","last_fullsized":"2024-09-17T04:27:43.189782+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.206591+0000","last_change":"2024-09-17T04:27:43.204007+0000","last_active":"2024-09-17T04:27:43.206591+0000","last_peered":"2024-09-17T04:27:43.206591+0000","last_clean":"2024-09-17T04:27:43.206591+0000","last_became_active":"2024-09-17T04:27:43.203747+0000","last_became_peered":"2024-09-17T04:27:43.203747+0000","last_unstale":"2024-09-17T04:27:43.206591+0000","last_undegraded":"2024-09-17T04:27:43.206591+0000","last_fullsized":"2024-09-17T04:27:43.206591+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:56:12.570282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.185434+0000","last_change":"2024-09-17T04:27:43.185434+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.185434+0000","last_undegraded":"2024-09-17T04:27:43.185434+0000","last_fullsized":"2024-09-17T04:27:43.185434+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212854+0000","last_change":"2024-09-17T04:27:43.201502+0000","last_active":"2024-09-17T04:27:45.212854+0000","last_peered":"2024-09-17T04:27:45.212854+0000","last_clean":"2024-09-17T04:27:45.212854+0000","last_became_active":"2024-09-17T04:27:43.201071+0000","last_became_peered":"2024-09-17T04:27:43.201071+0000","last_unstale":"2024-09-17T04:27:45.212854+0000","last_undegraded":"2024-09-17T04:27:45.212854+0000","last_fullsized":"2024-09-17T04:27:45.212854+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:48:58.744170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.206263+0000","last_change":"2024-09-17T04:27:43.203357+0000","last_active":"2024-09-17T04:27:43.206263+0000","last_peered":"2024-09-17T04:27:43.206263+0000","last_clean":"2024-09-17T04:27:43.206263+0000","last_became_active":"2024-09-17T04:27:43.202999+0000","last_became_peered":"2024-09-17T04:27:43.202999+0000","last_unstale":"2024-09-17T04:27:43.206263+0000","last_undegraded":"2024-09-17T04:27:43.206263+0000","last_fullsized":"2024-09-17T04:27:43.206263+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:10.033325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.187796+0000","last_change":"2024-09-17T04:27:43.187796+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.187796+0000","last_undegraded":"2024-09-17T04:27:43.187796+0000","last_fullsized":"2024-09-17T04:27:43.187796+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.208866+0000","last_change":"2024-09-17T04:27:43.200549+0000","last_active":"2024-09-17T04:27:43.208866+0000","last_peered":"2024-09-17T04:27:43.208866+0000","last_clean":"2024-09-17T04:27:43.208866+0000","last_became_active":"2024-09-17T04:27:43.200245+0000","last_became_peered":"2024-09-17T04:27:43.200245+0000","last_unstale":"2024-09-17T04:27:43.208866+0000","last_undegraded":"2024-09-17T04:27:43.208866+0000","last_fullsized":"2024-09-17T04:27:43.208866+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:44.583373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.190626+0000","last_change":"2024-09-17T04:27:43.190626+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.190626+0000","last_undegraded":"2024-09-17T04:27:43.190626+0000","last_fullsized":"2024-09-17T04:27:43.190626+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"60'2","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212414+0000","last_change":"2024-09-17T04:27:43.204124+0000","last_active":"2024-09-17T04:27:45.212414+0000","last_peered":"2024-09-17T04:27:45.212414+0000","last_clean":"2024-09-17T04:27:45.212414+0000","last_became_active":"2024-09-17T04:27:43.203730+0000","last_became_peered":"2024-09-17T04:27:43.203730+0000","last_unstale":"2024-09-17T04:27:45.212414+0000","last_undegraded":"2024-09-17T04:27:45.212414+0000","last_fullsized":"2024-09-17T04:27:45.212414+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:15:50.701517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.187412+0000","last_change":"2024-09-17T04:27:43.187412+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.187412+0000","last_undegraded":"2024-09-17T04:27:43.187412+0000","last_fullsized":"2024-09-17T04:27:43.187412+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.200751+0000","last_change":"2024-09-17T04:27:43.191819+0000","last_active":"2024-09-17T04:27:43.200751+0000","last_peered":"2024-09-17T04:27:43.200751+0000","last_clean":"2024-09-17T04:27:43.200751+0000","last_became_active":"2024-09-17T04:27:43.191432+0000","last_became_peered":"2024-09-17T04:27:43.191432+0000","last_unstale":"2024-09-17T04:27:43.200751+0000","last_undegraded":"2024-09-17T04:27:43.200751+0000","last_fullsized":"2024-09-17T04:27:43.200751+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:31.004288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.188547+0000","last_change":"2024-09-17T04:27:43.188547+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.188547+0000","last_undegraded":"2024-09-17T04:27:43.188547+0000","last_fullsized":"2024-09-17T04:27:43.188547+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.209284+0000","last_change":"2024-09-17T04:27:43.206779+0000","last_active":"2024-09-17T04:27:43.209284+0000","last_peered":"2024-09-17T04:27:43.209284+0000","last_clean":"2024-09-17T04:27:43.209284+0000","last_became_active":"2024-09-17T04:27:43.205945+0000","last_became_peered":"2024-09-17T04:27:43.205945+0000","last_unstale":"2024-09-17T04:27:43.209284+0000","last_undegraded":"2024-09-17T04:27:43.209284+0000","last_fullsized":"2024-09-17T04:27:43.209284+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:03.547426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.185001+0000","last_change":"2024-09-17T04:27:43.185001+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.185001+0000","last_undegraded":"2024-09-17T04:27:43.185001+0000","last_fullsized":"2024-09-17T04:27:43.185001+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.206344+0000","last_change":"2024-09-17T04:27:43.203507+0000","last_active":"2024-09-17T04:27:43.206344+0000","last_peered":"2024-09-17T04:27:43.206344+0000","last_clean":"2024-09-17T04:27:43.206344+0000","last_became_active":"2024-09-17T04:27:43.202223+0000","last_became_peered":"2024-09-17T04:27:43.202223+0000","last_unstale":"2024-09-17T04:27:43.206344+0000","last_undegraded":"2024-09-17T04:27:43.206344+0000","last_fullsized":"2024-09-17T04:27:43.206344+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:05.218532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.184912+0000","last_change":"2024-09-17T04:27:43.184912+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.184912+0000","last_undegraded":"2024-09-17T04:27:43.184912+0000","last_fullsized":"2024-09-17T04:27:43.184912+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"60'1","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.213160+0000","last_change":"2024-09-17T04:27:43.206771+0000","last_active":"2024-09-17T04:27:45.213160+0000","last_peered":"2024-09-17T04:27:45.213160+0000","last_clean":"2024-09-17T04:27:45.213160+0000","last_became_active":"2024-09-17T04:27:43.206481+0000","last_became_peered":"2024-09-17T04:27:43.206481+0000","last_unstale":"2024-09-17T04:27:45.213160+0000","last_undegraded":"2024-09-17T04:27:45.213160+0000","last_fullsized":"2024-09-17T04:27:45.213160+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:48:49.832510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.193299+0000","last_change":"2024-09-17T04:27:43.193299+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.193299+0000","last_undegraded":"2024-09-17T04:27:43.193299+0000","last_fullsized":"2024-09-17T04:27:43.193299+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.186798+0000","last_change":"2024-09-17T04:27:42.186798+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.186798+0000","last_undegraded":"2024-09-17T04:27:42.186798+0000","last_fullsized":"2024-09-17T04:27:42.186798+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212603+0000","last_change":"2024-09-17T04:27:44.222455+0000","last_active":"2024-09-17T04:27:45.212603+0000","last_peered":"2024-09-17T04:27:45.212603+0000","last_clean":"2024-09-17T04:27:45.212603+0000","last_became_active":"2024-09-17T04:27:44.222015+0000","last_became_peered":"2024-09-17T04:27:44.222015+0000","last_unstale":"2024-09-17T04:27:45.212603+0000","last_undegraded":"2024-09-17T04:27:45.212603+0000","last_fullsized":"2024-09-17T04:27:45.212603+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:01:05.264040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.209162+0000","last_change":"2024-09-17T04:27:43.207246+0000","last_active":"2024-09-17T04:27:43.209162+0000","last_peered":"2024-09-17T04:27:43.209162+0000","last_clean":"2024-09-17T04:27:43.209162+0000","last_became_active":"2024-09-17T04:27:43.206835+0000","last_became_peered":"2024-09-17T04:27:43.206835+0000","last_unstale":"2024-09-17T04:27:43.209162+0000","last_undegraded":"2024-09-17T04:27:43.209162+0000","last_fullsized":"2024-09-17T04:27:43.209162+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:05.218532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.186711+0000","last_change":"2024-09-17T04:27:43.186711+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.186711+0000","last_undegraded":"2024-09-17T04:27:43.186711+0000","last_fullsized":"2024-09-17T04:27:43.186711+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.185903+0000","last_change":"2024-09-17T04:27:42.185903+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.185903+0000","last_undegraded":"2024-09-17T04:27:42.185903+0000","last_fullsized":"2024-09-17T04:27:42.185903+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.189283+0000","last_change":"2024-09-17T04:27:43.189283+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.189283+0000","last_undegraded":"2024-09-17T04:27:43.189283+0000","last_fullsized":"2024-09-17T04:27:43.189283+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"60'2","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212605+0000","last_change":"2024-09-17T04:27:43.203669+0000","last_active":"2024-09-17T04:27:45.212605+0000","last_peered":"2024-09-17T04:27:45.212605+0000","last_clean":"2024-09-17T04:27:45.212605+0000","last_became_active":"2024-09-17T04:27:43.203307+0000","last_became_peered":"2024-09-17T04:27:43.203307+0000","last_unstale":"2024-09-17T04:27:45.212605+0000","last_undegraded":"2024-09-17T04:27:45.212605+0000","last_fullsized":"2024-09-17T04:27:45.212605+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:53.295497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.188161+0000","last_change":"2024-09-17T04:27:42.188161+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.188161+0000","last_undegraded":"2024-09-17T04:27:42.188161+0000","last_fullsized":"2024-09-17T04:27:42.188161+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.210443+0000","last_change":"2024-09-17T04:27:43.209348+0000","last_active":"2024-09-17T04:27:43.210443+0000","last_peered":"2024-09-17T04:27:43.210443+0000","last_clean":"2024-09-17T04:27:43.210443+0000","last_became_active":"2024-09-17T04:27:43.209055+0000","last_became_peered":"2024-09-17T04:27:43.209055+0000","last_unstale":"2024-09-17T04:27:43.210443+0000","last_undegraded":"2024-09-17T04:27:43.210443+0000","last_fullsized":"2024-09-17T04:27:43.210443+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:56:12.570282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.191601+0000","last_change":"2024-09-17T04:27:43.191601+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.191601+0000","last_undegraded":"2024-09-17T04:27:43.191601+0000","last_fullsized":"2024-09-17T04:27:43.191601+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.188855+0000","last_change":"2024-09-17T04:27:42.188855+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.188855+0000","last_undegraded":"2024-09-17T04:27:42.188855+0000","last_fullsized":"2024-09-17T04:27:42.188855+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.192785+0000","last_change":"2024-09-17T04:27:43.192785+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.192785+0000","last_undegraded":"2024-09-17T04:27:43.192785+0000","last_fullsized":"2024-09-17T04:27:43.192785+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.186212+0000","last_change":"2024-09-17T04:27:42.186212+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.186212+0000","last_undegraded":"2024-09-17T04:27:42.186212+0000","last_fullsized":"2024-09-17T04:27:42.186212+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.206084+0000","last_change":"2024-09-17T04:27:43.200897+0000","last_active":"2024-09-17T04:27:43.206084+0000","last_peered":"2024-09-17T04:27:43.206084+0000","last_clean":"2024-09-17T04:27:43.206084+0000","last_became_active":"2024-09-17T04:27:43.200425+0000","last_became_peered":"2024-09-17T04:27:43.200425+0000","last_unstale":"2024-09-17T04:27:43.206084+0000","last_undegraded":"2024-09-17T04:27:43.206084+0000","last_fullsized":"2024-09-17T04:27:43.206084+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:29:36.941642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.208984+0000","last_change":"2024-09-17T04:27:43.203031+0000","last_active":"2024-09-17T04:27:43.208984+0000","last_peered":"2024-09-17T04:27:43.208984+0000","last_clean":"2024-09-17T04:27:43.208984+0000","last_became_active":"2024-09-17T04:27:43.202637+0000","last_became_peered":"2024-09-17T04:27:43.202637+0000","last_unstale":"2024-09-17T04:27:43.208984+0000","last_undegraded":"2024-09-17T04:27:43.208984+0000","last_fullsized":"2024-09-17T04:27:43.208984+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:12:58.279556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.213008+0000","last_change":"2024-09-17T04:27:44.217560+0000","last_active":"2024-09-17T04:27:45.213008+0000","last_peered":"2024-09-17T04:27:45.213008+0000","last_clean":"2024-09-17T04:27:45.213008+0000","last_became_active":"2024-09-17T04:27:44.217250+0000","last_became_peered":"2024-09-17T04:27:44.217250+0000","last_unstale":"2024-09-17T04:27:45.213008+0000","last_undegraded":"2024-09-17T04:27:45.213008+0000","last_fullsized":"2024-09-17T04:27:45.213008+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:47:24.767897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.188211+0000","last_change":"2024-09-17T04:27:42.188211+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.188211+0000","last_undegraded":"2024-09-17T04:27:42.188211+0000","last_fullsized":"2024-09-17T04:27:42.188211+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.183762+0000","last_change":"2024-09-17T04:27:43.183762+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.183762+0000","last_undegraded":"2024-09-17T04:27:43.183762+0000","last_fullsized":"2024-09-17T04:27:43.183762+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.209037+0000","last_change":"2024-09-17T04:27:43.201986+0000","last_active":"2024-09-17T04:27:43.209037+0000","last_peered":"2024-09-17T04:27:43.209037+0000","last_clean":"2024-09-17T04:27:43.209037+0000","last_became_active":"2024-09-17T04:27:43.201654+0000","last_became_peered":"2024-09-17T04:27:43.201654+0000","last_unstale":"2024-09-17T04:27:43.209037+0000","last_undegraded":"2024-09-17T04:27:43.209037+0000","last_fullsized":"2024-09-17T04:27:43.209037+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:43:24.891449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.188178+0000","last_change":"2024-09-17T04:27:42.188178+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.188178+0000","last_undegraded":"2024-09-17T04:27:42.188178+0000","last_fullsized":"2024-09-17T04:27:42.188178+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212826+0000","last_change":"2024-09-17T04:27:44.223877+0000","last_active":"2024-09-17T04:27:45.212826+0000","last_peered":"2024-09-17T04:27:45.212826+0000","last_clean":"2024-09-17T04:27:45.212826+0000","last_became_active":"2024-09-17T04:27:44.223582+0000","last_became_peered":"2024-09-17T04:27:44.223582+0000","last_unstale":"2024-09-17T04:27:45.212826+0000","last_undegraded":"2024-09-17T04:27:45.212826+0000","last_fullsized":"2024-09-17T04:27:45.212826+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:09.084549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212546+0000","last_change":"2024-09-17T04:27:43.201747+0000","last_active":"2024-09-17T04:27:45.212546+0000","last_peered":"2024-09-17T04:27:45.212546+0000","last_clean":"2024-09-17T04:27:45.212546+0000","last_became_active":"2024-09-17T04:27:43.200792+0000","last_became_peered":"2024-09-17T04:27:43.200792+0000","last_unstale":"2024-09-17T04:27:45.212546+0000","last_undegraded":"2024-09-17T04:27:45.212546+0000","last_fullsized":"2024-09-17T04:27:45.212546+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:30.181955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212302+0000","last_change":"2024-09-17T04:27:44.223584+0000","last_active":"2024-09-17T04:27:45.212302+0000","last_peered":"2024-09-17T04:27:45.212302+0000","last_clean":"2024-09-17T04:27:45.212302+0000","last_became_active":"2024-09-17T04:27:44.220615+0000","last_became_peered":"2024-09-17T04:27:44.220615+0000","last_unstale":"2024-09-17T04:27:45.212302+0000","last_undegraded":"2024-09-17T04:27:45.212302+0000","last_fullsized":"2024-09-17T04:27:45.212302+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:46.132448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.207926+0000","last_change":"2024-09-17T04:27:43.203968+0000","last_active":"2024-09-17T04:27:43.207926+0000","last_peered":"2024-09-17T04:27:43.207926+0000","last_clean":"2024-09-17T04:27:43.207926+0000","last_became_active":"2024-09-17T04:27:43.203677+0000","last_became_peered":"2024-09-17T04:27:43.203677+0000","last_unstale":"2024-09-17T04:27:43.207926+0000","last_undegraded":"2024-09-17T04:27:43.207926+0000","last_fullsized":"2024-09-17T04:27:43.207926+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:47.357122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":180,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.191448+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:27:43.191448+0000","last_peered":"2024-09-17T04:27:43.191448+0000","last_clean":"2024-09-17T04:27:43.191448+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:27:43.191448+0000","last_undegraded":"2024-09-17T04:27:43.191448+0000","last_fullsized":"2024-09-17T04:27:43.191448+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":84,"acting":84,"num_store_stats":5},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":56,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476785,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1290240,"data_stored":876659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476784,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":696320,"data_stored":286291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476785,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":696320,"data_stored":286291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476785,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":696320,"data_stored":286291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509488,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27696,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":684032,"data_stored":280024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509487,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":684032,"data_stored":280024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509487,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1277952,"data_stored":870392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542193,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":729088,"data_stored":299422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:27:48.624 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:27:48.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.881+0000 7f4536000640 1 -- 172.21.15.100:0/1293608377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4518003fd0 msgr2=0x7f4518004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:48.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.881+0000 7f4536000640 1 --2- 172.21.15.100:0/1293608377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4518003fd0 0x7f4518004800 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f451c009b50 tx=0x7f4518005830 comp rx=0 tx=0).stop 2024-09-17T04:27:48.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.881+0000 7f4535600640 1 -- 172.21.15.100:0/1293608377 reap_dead start 2024-09-17T04:27:48.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.881+0000 7f4536000640 1 -- 172.21.15.100:0/1293608377 shutdown_connections 2024-09-17T04:27:48.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.881+0000 7f4536000640 1 -- 172.21.15.100:0/1293608377 >> 172.21.15.100:0/1293608377 conn(0x7f4530071070 msgr2=0x7f4530071480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:48.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.881+0000 7f4536000640 1 -- 172.21.15.100:0/1293608377 shutdown_connections 2024-09-17T04:27:48.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.881+0000 7f4536000640 1 -- 172.21.15.100:0/1293608377 wait complete. 2024-09-17T04:27:48.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.883+0000 7f4536000640 1 Processor -- start 2024-09-17T04:27:48.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.883+0000 7f4536000640 1 -- start start 2024-09-17T04:27:48.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.885+0000 7f4536000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4518003fd0 0x7f45301516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:48.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.885+0000 7f4536000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45300b1470 0x7f45301f8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:48.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.885+0000 7f4536000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45301f9080 0x7f45301f6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:48.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.885+0000 7f4536000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f45300ceb90 con 0x7f45301f9080 2024-09-17T04:27:48.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.885+0000 7f4536000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f45300cea10 con 0x7f45300b1470 2024-09-17T04:27:48.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.885+0000 7f4536000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f45300ced10 con 0x7f4518003fd0 2024-09-17T04:27:48.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.885+0000 7f4535600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45301f9080 0x7f45301f6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:48.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.885+0000 7f4535600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45301f9080 0x7f45301f6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56856/0 (socket says 172.21.15.100:56856) 2024-09-17T04:27:48.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.885+0000 7f4535600640 1 -- 172.21.15.100:0/398616917 learned_addr learned my addr 172.21.15.100:0/398616917 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:48.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.885+0000 7f452fe00640 1 --2- 172.21.15.100:0/398616917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45300b1470 0x7f45301f8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:48.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.885+0000 7f4534c00640 1 --2- 172.21.15.100:0/398616917 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4518003fd0 0x7f45301516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:48.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.886+0000 7f4535600640 1 -- 172.21.15.100:0/398616917 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4518003fd0 msgr2=0x7f45301516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:48.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.886+0000 7f4535600640 1 --2- 172.21.15.100:0/398616917 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4518003fd0 0x7f45301516e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:48.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.886+0000 7f4535600640 1 -- 172.21.15.100:0/398616917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45300b1470 msgr2=0x7f45301f8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:48.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.886+0000 7f4535600640 1 --2- 172.21.15.100:0/398616917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45300b1470 0x7f45301f8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:48.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.886+0000 7f4535600640 1 -- 172.21.15.100:0/398616917 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4530151c20 con 0x7f45301f9080 2024-09-17T04:27:48.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.886+0000 7f4535600640 1 --2- 172.21.15.100:0/398616917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45301f9080 0x7f45301f6710 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f45300d1420 tx=0x7f4528004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:48.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.886+0000 7f452cc00640 1 -- 172.21.15.100:0/398616917 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4528015020 con 0x7f45301f9080 2024-09-17T04:27:48.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.886+0000 7f452cc00640 1 -- 172.21.15.100:0/398616917 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45280109c0 con 0x7f45301f9080 2024-09-17T04:27:48.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.886+0000 7f4536000640 1 -- 172.21.15.100:0/398616917 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45300adb20 con 0x7f45301f9080 2024-09-17T04:27:48.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.886+0000 7f452cc00640 1 -- 172.21.15.100:0/398616917 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4528010e10 con 0x7f45301f9080 2024-09-17T04:27:48.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.886+0000 7f4536000640 1 -- 172.21.15.100:0/398616917 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45300ae0e0 con 0x7f45301f9080 2024-09-17T04:27:48.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.888+0000 7f4536000640 1 -- 172.21.15.100:0/398616917 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f453015e9a0 con 0x7f45301f9080 2024-09-17T04:27:48.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.889+0000 7f452cc00640 1 -- 172.21.15.100:0/398616917 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f4528018020 con 0x7f45301f9080 2024-09-17T04:27:48.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.889+0000 7f452cc00640 1 --2- 172.21.15.100:0/398616917 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4510078d40 0x7f451007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:48.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.890+0000 7f452cc00640 1 -- 172.21.15.100:0/398616917 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7f452809b7f0 con 0x7f45301f9080 2024-09-17T04:27:48.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.890+0000 7f4534c00640 1 --2- 172.21.15.100:0/398616917 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4510078d40 0x7f451007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:48.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.891+0000 7f4534c00640 1 --2- 172.21.15.100:0/398616917 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4510078d40 0x7f451007b250 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4524009cd0 tx=0x7f4524005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:48.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:48.894+0000 7f452cc00640 1 -- 172.21.15.100:0/398616917 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f452801f030 con 0x7f45301f9080 2024-09-17T04:27:49.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.073+0000 7f4536000640 1 -- 172.21.15.100:0/398616917 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f45301f9970 con 0x7f4510078d40 2024-09-17T04:27:49.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.079+0000 7f452cc00640 1 -- 172.21.15.100:0/398616917 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245270 (secure 0 0 0) 0x7f452405ce30 con 0x7f4510078d40 2024-09-17T04:27:49.081 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:27:49.082 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:27:49.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.082+0000 7f4536000640 1 -- 172.21.15.100:0/398616917 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4510078d40 msgr2=0x7f451007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:49.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.082+0000 7f4536000640 1 --2- 172.21.15.100:0/398616917 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4510078d40 0x7f451007b250 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4524009cd0 tx=0x7f4524005f90 comp rx=0 tx=0).stop 2024-09-17T04:27:49.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.082+0000 7f4536000640 1 -- 172.21.15.100:0/398616917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45301f9080 msgr2=0x7f45301f6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:49.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.082+0000 7f4536000640 1 --2- 172.21.15.100:0/398616917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45301f9080 0x7f45301f6710 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f45300d1420 tx=0x7f4528004f90 comp rx=0 tx=0).stop 2024-09-17T04:27:49.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.083+0000 7f4536000640 1 -- 172.21.15.100:0/398616917 shutdown_connections 2024-09-17T04:27:49.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.083+0000 7f4536000640 1 --2- 172.21.15.100:0/398616917 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4510078d40 0x7f451007b250 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:49.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.083+0000 7f4536000640 1 --2- 172.21.15.100:0/398616917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45301f9080 0x7f45301f6710 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:49.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.083+0000 7f4536000640 1 --2- 172.21.15.100:0/398616917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45300b1470 0x7f45301f8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:49.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.083+0000 7f4536000640 1 --2- 172.21.15.100:0/398616917 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4518003fd0 0x7f45301516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:49.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.083+0000 7f4536000640 1 -- 172.21.15.100:0/398616917 >> 172.21.15.100:0/398616917 conn(0x7f4530071070 msgr2=0x7f45300d22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:49.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.083+0000 7f4536000640 1 -- 172.21.15.100:0/398616917 shutdown_connections 2024-09-17T04:27:49.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.083+0000 7f4536000640 1 -- 172.21.15.100:0/398616917 wait complete. 2024-09-17T04:27:49.106 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-09-17T04:27:48.024133+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590424,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":106,"num_write_kb":1636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":150,"acting":150,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":235328,"kb_used_data":6596,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754739392,"statfs":{"total":773094113280,"available":772853137408,"internally_reserved":0,"allocated":6754304,"data_stored":3465394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":234016308},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":56,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.846898"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212739+0000","last_change":"2024-09-17T04:27:44.238721+0000","last_active":"2024-09-17T04:27:45.212739+0000","last_peered":"2024-09-17T04:27:45.212739+0000","last_clean":"2024-09-17T04:27:45.212739+0000","last_became_active":"2024-09-17T04:27:44.237900+0000","last_became_peered":"2024-09-17T04:27:44.237900+0000","last_unstale":"2024-09-17T04:27:45.212739+0000","last_undegraded":"2024-09-17T04:27:45.212739+0000","last_fullsized":"2024-09-17T04:27:45.212739+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:09:57.201730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.188353+0000","last_change":"2024-09-17T04:27:43.188353+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.188353+0000","last_undegraded":"2024-09-17T04:27:43.188353+0000","last_fullsized":"2024-09-17T04:27:43.188353+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.194226+0000","last_change":"2024-09-17T04:27:43.194226+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.194226+0000","last_undegraded":"2024-09-17T04:27:43.194226+0000","last_fullsized":"2024-09-17T04:27:43.194226+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.190634+0000","last_change":"2024-09-17T04:27:43.190634+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.190634+0000","last_undegraded":"2024-09-17T04:27:43.190634+0000","last_fullsized":"2024-09-17T04:27:43.190634+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212387+0000","last_change":"2024-09-17T04:27:44.222903+0000","last_active":"2024-09-17T04:27:45.212387+0000","last_peered":"2024-09-17T04:27:45.212387+0000","last_clean":"2024-09-17T04:27:45.212387+0000","last_became_active":"2024-09-17T04:27:44.222587+0000","last_became_peered":"2024-09-17T04:27:44.222587+0000","last_unstale":"2024-09-17T04:27:45.212387+0000","last_undegraded":"2024-09-17T04:27:45.212387+0000","last_fullsized":"2024-09-17T04:27:45.212387+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:37:39.457226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.189985+0000","last_change":"2024-09-17T04:27:43.189985+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.189985+0000","last_undegraded":"2024-09-17T04:27:43.189985+0000","last_fullsized":"2024-09-17T04:27:43.189985+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.193736+0000","last_change":"2024-09-17T04:27:43.193736+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.193736+0000","last_undegraded":"2024-09-17T04:27:43.193736+0000","last_fullsized":"2024-09-17T04:27:43.193736+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.184126+0000","last_change":"2024-09-17T04:27:43.184126+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.184126+0000","last_undegraded":"2024-09-17T04:27:43.184126+0000","last_fullsized":"2024-09-17T04:27:43.184126+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.189690+0000","last_change":"2024-09-17T04:27:43.189690+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.189690+0000","last_undegraded":"2024-09-17T04:27:43.189690+0000","last_fullsized":"2024-09-17T04:27:43.189690+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.188426+0000","last_change":"2024-09-17T04:27:43.188426+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.188426+0000","last_undegraded":"2024-09-17T04:27:43.188426+0000","last_fullsized":"2024-09-17T04:27:43.188426+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212938+0000","last_change":"2024-09-17T04:27:44.221917+0000","last_active":"2024-09-17T04:27:45.212938+0000","last_peered":"2024-09-17T04:27:45.212938+0000","last_clean":"2024-09-17T04:27:45.212938+0000","last_became_active":"2024-09-17T04:27:44.221632+0000","last_became_peered":"2024-09-17T04:27:44.221632+0000","last_unstale":"2024-09-17T04:27:45.212938+0000","last_undegraded":"2024-09-17T04:27:45.212938+0000","last_fullsized":"2024-09-17T04:27:45.212938+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:27.954596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.185462+0000","last_change":"2024-09-17T04:27:43.185462+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.185462+0000","last_undegraded":"2024-09-17T04:27:43.185462+0000","last_fullsized":"2024-09-17T04:27:43.185462+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212533+0000","last_change":"2024-09-17T04:27:44.220909+0000","last_active":"2024-09-17T04:27:45.212533+0000","last_peered":"2024-09-17T04:27:45.212533+0000","last_clean":"2024-09-17T04:27:45.212533+0000","last_became_active":"2024-09-17T04:27:44.220605+0000","last_became_peered":"2024-09-17T04:27:44.220605+0000","last_unstale":"2024-09-17T04:27:45.212533+0000","last_undegraded":"2024-09-17T04:27:45.212533+0000","last_fullsized":"2024-09-17T04:27:45.212533+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:48:13.238235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.188790+0000","last_change":"2024-09-17T04:27:43.188790+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.188790+0000","last_undegraded":"2024-09-17T04:27:43.188790+0000","last_fullsized":"2024-09-17T04:27:43.188790+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212746+0000","last_change":"2024-09-17T04:27:44.216830+0000","last_active":"2024-09-17T04:27:45.212746+0000","last_peered":"2024-09-17T04:27:45.212746+0000","last_clean":"2024-09-17T04:27:45.212746+0000","last_became_active":"2024-09-17T04:27:44.216520+0000","last_became_peered":"2024-09-17T04:27:44.216520+0000","last_unstale":"2024-09-17T04:27:45.212746+0000","last_undegraded":"2024-09-17T04:27:45.212746+0000","last_fullsized":"2024-09-17T04:27:45.212746+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:15:47.203299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.185548+0000","last_change":"2024-09-17T04:27:43.185548+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.185548+0000","last_undegraded":"2024-09-17T04:27:43.185548+0000","last_fullsized":"2024-09-17T04:27:43.185548+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.185510+0000","last_change":"2024-09-17T04:27:43.185510+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.185510+0000","last_undegraded":"2024-09-17T04:27:43.185510+0000","last_fullsized":"2024-09-17T04:27:43.185510+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.192118+0000","last_change":"2024-09-17T04:27:43.192118+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.192118+0000","last_undegraded":"2024-09-17T04:27:43.192118+0000","last_fullsized":"2024-09-17T04:27:43.192118+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.213050+0000","last_change":"2024-09-17T04:27:44.218277+0000","last_active":"2024-09-17T04:27:45.213050+0000","last_peered":"2024-09-17T04:27:45.213050+0000","last_clean":"2024-09-17T04:27:45.213050+0000","last_became_active":"2024-09-17T04:27:44.217942+0000","last_became_peered":"2024-09-17T04:27:44.217942+0000","last_unstale":"2024-09-17T04:27:45.213050+0000","last_undegraded":"2024-09-17T04:27:45.213050+0000","last_fullsized":"2024-09-17T04:27:45.213050+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:18.353865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.183706+0000","last_change":"2024-09-17T04:27:43.183706+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.183706+0000","last_undegraded":"2024-09-17T04:27:43.183706+0000","last_fullsized":"2024-09-17T04:27:43.183706+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.186596+0000","last_change":"2024-09-17T04:27:42.186596+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.186596+0000","last_undegraded":"2024-09-17T04:27:42.186596+0000","last_fullsized":"2024-09-17T04:27:42.186596+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"60'1","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212757+0000","last_change":"2024-09-17T04:27:43.206766+0000","last_active":"2024-09-17T04:27:45.212757+0000","last_peered":"2024-09-17T04:27:45.212757+0000","last_clean":"2024-09-17T04:27:45.212757+0000","last_became_active":"2024-09-17T04:27:43.206455+0000","last_became_peered":"2024-09-17T04:27:43.206455+0000","last_unstale":"2024-09-17T04:27:45.212757+0000","last_undegraded":"2024-09-17T04:27:45.212757+0000","last_fullsized":"2024-09-17T04:27:45.212757+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:06:22.067347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.186307+0000","last_change":"2024-09-17T04:27:42.186307+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.186307+0000","last_undegraded":"2024-09-17T04:27:42.186307+0000","last_fullsized":"2024-09-17T04:27:42.186307+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.209139+0000","last_change":"2024-09-17T04:27:43.207037+0000","last_active":"2024-09-17T04:27:43.209139+0000","last_peered":"2024-09-17T04:27:43.209139+0000","last_clean":"2024-09-17T04:27:43.209139+0000","last_became_active":"2024-09-17T04:27:43.206672+0000","last_became_peered":"2024-09-17T04:27:43.206672+0000","last_unstale":"2024-09-17T04:27:43.209139+0000","last_undegraded":"2024-09-17T04:27:43.209139+0000","last_fullsized":"2024-09-17T04:27:43.209139+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:10.033325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.213106+0000","last_change":"2024-09-17T04:27:44.217751+0000","last_active":"2024-09-17T04:27:45.213106+0000","last_peered":"2024-09-17T04:27:45.213106+0000","last_clean":"2024-09-17T04:27:45.213106+0000","last_became_active":"2024-09-17T04:27:44.214072+0000","last_became_peered":"2024-09-17T04:27:44.214072+0000","last_unstale":"2024-09-17T04:27:45.213106+0000","last_undegraded":"2024-09-17T04:27:45.213106+0000","last_fullsized":"2024-09-17T04:27:45.213106+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:27.160546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.208764+0000","last_change":"2024-09-17T04:27:43.198922+0000","last_active":"2024-09-17T04:27:43.208764+0000","last_peered":"2024-09-17T04:27:43.208764+0000","last_clean":"2024-09-17T04:27:43.208764+0000","last_became_active":"2024-09-17T04:27:43.198453+0000","last_became_peered":"2024-09-17T04:27:43.198453+0000","last_unstale":"2024-09-17T04:27:43.208764+0000","last_undegraded":"2024-09-17T04:27:43.208764+0000","last_fullsized":"2024-09-17T04:27:43.208764+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:46.166554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212477+0000","last_change":"2024-09-17T04:27:44.222601+0000","last_active":"2024-09-17T04:27:45.212477+0000","last_peered":"2024-09-17T04:27:45.212477+0000","last_clean":"2024-09-17T04:27:45.212477+0000","last_became_active":"2024-09-17T04:27:44.221587+0000","last_became_peered":"2024-09-17T04:27:44.221587+0000","last_unstale":"2024-09-17T04:27:45.212477+0000","last_undegraded":"2024-09-17T04:27:45.212477+0000","last_fullsized":"2024-09-17T04:27:45.212477+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:22.587679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.185403+0000","last_change":"2024-09-17T04:27:42.185403+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.185403+0000","last_undegraded":"2024-09-17T04:27:42.185403+0000","last_fullsized":"2024-09-17T04:27:42.185403+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.183685+0000","last_change":"2024-09-17T04:27:43.183685+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.183685+0000","last_undegraded":"2024-09-17T04:27:43.183685+0000","last_fullsized":"2024-09-17T04:27:43.183685+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.207657+0000","last_change":"2024-09-17T04:27:43.200857+0000","last_active":"2024-09-17T04:27:43.207657+0000","last_peered":"2024-09-17T04:27:43.207657+0000","last_clean":"2024-09-17T04:27:43.207657+0000","last_became_active":"2024-09-17T04:27:43.200518+0000","last_became_peered":"2024-09-17T04:27:43.200518+0000","last_unstale":"2024-09-17T04:27:43.207657+0000","last_undegraded":"2024-09-17T04:27:43.207657+0000","last_fullsized":"2024-09-17T04:27:43.207657+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:11.559902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212733+0000","last_change":"2024-09-17T04:27:44.225414+0000","last_active":"2024-09-17T04:27:45.212733+0000","last_peered":"2024-09-17T04:27:45.212733+0000","last_clean":"2024-09-17T04:27:45.212733+0000","last_became_active":"2024-09-17T04:27:44.225112+0000","last_became_peered":"2024-09-17T04:27:44.225112+0000","last_unstale":"2024-09-17T04:27:45.212733+0000","last_undegraded":"2024-09-17T04:27:45.212733+0000","last_fullsized":"2024-09-17T04:27:45.212733+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:05:18.890521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.213768+0000","last_change":"2024-09-17T04:27:43.209713+0000","last_active":"2024-09-17T04:27:43.213768+0000","last_peered":"2024-09-17T04:27:43.213768+0000","last_clean":"2024-09-17T04:27:43.213768+0000","last_became_active":"2024-09-17T04:27:43.209316+0000","last_became_peered":"2024-09-17T04:27:43.209316+0000","last_unstale":"2024-09-17T04:27:43.213768+0000","last_undegraded":"2024-09-17T04:27:43.213768+0000","last_fullsized":"2024-09-17T04:27:43.213768+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:01.773526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.189782+0000","last_change":"2024-09-17T04:27:43.189782+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.189782+0000","last_undegraded":"2024-09-17T04:27:43.189782+0000","last_fullsized":"2024-09-17T04:27:43.189782+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.206591+0000","last_change":"2024-09-17T04:27:43.204007+0000","last_active":"2024-09-17T04:27:43.206591+0000","last_peered":"2024-09-17T04:27:43.206591+0000","last_clean":"2024-09-17T04:27:43.206591+0000","last_became_active":"2024-09-17T04:27:43.203747+0000","last_became_peered":"2024-09-17T04:27:43.203747+0000","last_unstale":"2024-09-17T04:27:43.206591+0000","last_undegraded":"2024-09-17T04:27:43.206591+0000","last_fullsized":"2024-09-17T04:27:43.206591+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:56:12.570282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.185434+0000","last_change":"2024-09-17T04:27:43.185434+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.185434+0000","last_undegraded":"2024-09-17T04:27:43.185434+0000","last_fullsized":"2024-09-17T04:27:43.185434+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212854+0000","last_change":"2024-09-17T04:27:43.201502+0000","last_active":"2024-09-17T04:27:45.212854+0000","last_peered":"2024-09-17T04:27:45.212854+0000","last_clean":"2024-09-17T04:27:45.212854+0000","last_became_active":"2024-09-17T04:27:43.201071+0000","last_became_peered":"2024-09-17T04:27:43.201071+0000","last_unstale":"2024-09-17T04:27:45.212854+0000","last_undegraded":"2024-09-17T04:27:45.212854+0000","last_fullsized":"2024-09-17T04:27:45.212854+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:48:58.744170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.206263+0000","last_change":"2024-09-17T04:27:43.203357+0000","last_active":"2024-09-17T04:27:43.206263+0000","last_peered":"2024-09-17T04:27:43.206263+0000","last_clean":"2024-09-17T04:27:43.206263+0000","last_became_active":"2024-09-17T04:27:43.202999+0000","last_became_peered":"2024-09-17T04:27:43.202999+0000","last_unstale":"2024-09-17T04:27:43.206263+0000","last_undegraded":"2024-09-17T04:27:43.206263+0000","last_fullsized":"2024-09-17T04:27:43.206263+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:10.033325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.187796+0000","last_change":"2024-09-17T04:27:43.187796+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.187796+0000","last_undegraded":"2024-09-17T04:27:43.187796+0000","last_fullsized":"2024-09-17T04:27:43.187796+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.208866+0000","last_change":"2024-09-17T04:27:43.200549+0000","last_active":"2024-09-17T04:27:43.208866+0000","last_peered":"2024-09-17T04:27:43.208866+0000","last_clean":"2024-09-17T04:27:43.208866+0000","last_became_active":"2024-09-17T04:27:43.200245+0000","last_became_peered":"2024-09-17T04:27:43.200245+0000","last_unstale":"2024-09-17T04:27:43.208866+0000","last_undegraded":"2024-09-17T04:27:43.208866+0000","last_fullsized":"2024-09-17T04:27:43.208866+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:44.583373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.190626+0000","last_change":"2024-09-17T04:27:43.190626+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.190626+0000","last_undegraded":"2024-09-17T04:27:43.190626+0000","last_fullsized":"2024-09-17T04:27:43.190626+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"60'2","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212414+0000","last_change":"2024-09-17T04:27:43.204124+0000","last_active":"2024-09-17T04:27:45.212414+0000","last_peered":"2024-09-17T04:27:45.212414+0000","last_clean":"2024-09-17T04:27:45.212414+0000","last_became_active":"2024-09-17T04:27:43.203730+0000","last_became_peered":"2024-09-17T04:27:43.203730+0000","last_unstale":"2024-09-17T04:27:45.212414+0000","last_undegraded":"2024-09-17T04:27:45.212414+0000","last_fullsized":"2024-09-17T04:27:45.212414+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:15:50.701517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.187412+0000","last_change":"2024-09-17T04:27:43.187412+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.187412+0000","last_undegraded":"2024-09-17T04:27:43.187412+0000","last_fullsized":"2024-09-17T04:27:43.187412+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.200751+0000","last_change":"2024-09-17T04:27:43.191819+0000","last_active":"2024-09-17T04:27:43.200751+0000","last_peered":"2024-09-17T04:27:43.200751+0000","last_clean":"2024-09-17T04:27:43.200751+0000","last_became_active":"2024-09-17T04:27:43.191432+0000","last_became_peered":"2024-09-17T04:27:43.191432+0000","last_unstale":"2024-09-17T04:27:43.200751+0000","last_undegraded":"2024-09-17T04:27:43.200751+0000","last_fullsized":"2024-09-17T04:27:43.200751+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:31.004288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.188547+0000","last_change":"2024-09-17T04:27:43.188547+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.188547+0000","last_undegraded":"2024-09-17T04:27:43.188547+0000","last_fullsized":"2024-09-17T04:27:43.188547+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.209284+0000","last_change":"2024-09-17T04:27:43.206779+0000","last_active":"2024-09-17T04:27:43.209284+0000","last_peered":"2024-09-17T04:27:43.209284+0000","last_clean":"2024-09-17T04:27:43.209284+0000","last_became_active":"2024-09-17T04:27:43.205945+0000","last_became_peered":"2024-09-17T04:27:43.205945+0000","last_unstale":"2024-09-17T04:27:43.209284+0000","last_undegraded":"2024-09-17T04:27:43.209284+0000","last_fullsized":"2024-09-17T04:27:43.209284+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:03.547426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.185001+0000","last_change":"2024-09-17T04:27:43.185001+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.185001+0000","last_undegraded":"2024-09-17T04:27:43.185001+0000","last_fullsized":"2024-09-17T04:27:43.185001+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.206344+0000","last_change":"2024-09-17T04:27:43.203507+0000","last_active":"2024-09-17T04:27:43.206344+0000","last_peered":"2024-09-17T04:27:43.206344+0000","last_clean":"2024-09-17T04:27:43.206344+0000","last_became_active":"2024-09-17T04:27:43.202223+0000","last_became_peered":"2024-09-17T04:27:43.202223+0000","last_unstale":"2024-09-17T04:27:43.206344+0000","last_undegraded":"2024-09-17T04:27:43.206344+0000","last_fullsized":"2024-09-17T04:27:43.206344+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:05.218532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.184912+0000","last_change":"2024-09-17T04:27:43.184912+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.184912+0000","last_undegraded":"2024-09-17T04:27:43.184912+0000","last_fullsized":"2024-09-17T04:27:43.184912+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"60'1","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.213160+0000","last_change":"2024-09-17T04:27:43.206771+0000","last_active":"2024-09-17T04:27:45.213160+0000","last_peered":"2024-09-17T04:27:45.213160+0000","last_clean":"2024-09-17T04:27:45.213160+0000","last_became_active":"2024-09-17T04:27:43.206481+0000","last_became_peered":"2024-09-17T04:27:43.206481+0000","last_unstale":"2024-09-17T04:27:45.213160+0000","last_undegraded":"2024-09-17T04:27:45.213160+0000","last_fullsized":"2024-09-17T04:27:45.213160+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:48:49.832510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.193299+0000","last_change":"2024-09-17T04:27:43.193299+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.193299+0000","last_undegraded":"2024-09-17T04:27:43.193299+0000","last_fullsized":"2024-09-17T04:27:43.193299+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.186798+0000","last_change":"2024-09-17T04:27:42.186798+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.186798+0000","last_undegraded":"2024-09-17T04:27:42.186798+0000","last_fullsized":"2024-09-17T04:27:42.186798+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212603+0000","last_change":"2024-09-17T04:27:44.222455+0000","last_active":"2024-09-17T04:27:45.212603+0000","last_peered":"2024-09-17T04:27:45.212603+0000","last_clean":"2024-09-17T04:27:45.212603+0000","last_became_active":"2024-09-17T04:27:44.222015+0000","last_became_peered":"2024-09-17T04:27:44.222015+0000","last_unstale":"2024-09-17T04:27:45.212603+0000","last_undegraded":"2024-09-17T04:27:45.212603+0000","last_fullsized":"2024-09-17T04:27:45.212603+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:01:05.264040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.209162+0000","last_change":"2024-09-17T04:27:43.207246+0000","last_active":"2024-09-17T04:27:43.209162+0000","last_peered":"2024-09-17T04:27:43.209162+0000","last_clean":"2024-09-17T04:27:43.209162+0000","last_became_active":"2024-09-17T04:27:43.206835+0000","last_became_peered":"2024-09-17T04:27:43.206835+0000","last_unstale":"2024-09-17T04:27:43.209162+0000","last_undegraded":"2024-09-17T04:27:43.209162+0000","last_fullsized":"2024-09-17T04:27:43.209162+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:05.218532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.186711+0000","last_change":"2024-09-17T04:27:43.186711+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.186711+0000","last_undegraded":"2024-09-17T04:27:43.186711+0000","last_fullsized":"2024-09-17T04:27:43.186711+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.185903+0000","last_change":"2024-09-17T04:27:42.185903+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.185903+0000","last_undegraded":"2024-09-17T04:27:42.185903+0000","last_fullsized":"2024-09-17T04:27:42.185903+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.189283+0000","last_change":"2024-09-17T04:27:43.189283+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.189283+0000","last_undegraded":"2024-09-17T04:27:43.189283+0000","last_fullsized":"2024-09-17T04:27:43.189283+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"60'2","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212605+0000","last_change":"2024-09-17T04:27:43.203669+0000","last_active":"2024-09-17T04:27:45.212605+0000","last_peered":"2024-09-17T04:27:45.212605+0000","last_clean":"2024-09-17T04:27:45.212605+0000","last_became_active":"2024-09-17T04:27:43.203307+0000","last_became_peered":"2024-09-17T04:27:43.203307+0000","last_unstale":"2024-09-17T04:27:45.212605+0000","last_undegraded":"2024-09-17T04:27:45.212605+0000","last_fullsized":"2024-09-17T04:27:45.212605+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:53.295497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.188161+0000","last_change":"2024-09-17T04:27:42.188161+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.188161+0000","last_undegraded":"2024-09-17T04:27:42.188161+0000","last_fullsized":"2024-09-17T04:27:42.188161+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.210443+0000","last_change":"2024-09-17T04:27:43.209348+0000","last_active":"2024-09-17T04:27:43.210443+0000","last_peered":"2024-09-17T04:27:43.210443+0000","last_clean":"2024-09-17T04:27:43.210443+0000","last_became_active":"2024-09-17T04:27:43.209055+0000","last_became_peered":"2024-09-17T04:27:43.209055+0000","last_unstale":"2024-09-17T04:27:43.210443+0000","last_undegraded":"2024-09-17T04:27:43.210443+0000","last_fullsized":"2024-09-17T04:27:43.210443+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:56:12.570282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.191601+0000","last_change":"2024-09-17T04:27:43.191601+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.191601+0000","last_undegraded":"2024-09-17T04:27:43.191601+0000","last_fullsized":"2024-09-17T04:27:43.191601+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.188855+0000","last_change":"2024-09-17T04:27:42.188855+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.188855+0000","last_undegraded":"2024-09-17T04:27:42.188855+0000","last_fullsized":"2024-09-17T04:27:42.188855+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.192785+0000","last_change":"2024-09-17T04:27:43.192785+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.192785+0000","last_undegraded":"2024-09-17T04:27:43.192785+0000","last_fullsized":"2024-09-17T04:27:43.192785+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.186212+0000","last_change":"2024-09-17T04:27:42.186212+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.186212+0000","last_undegraded":"2024-09-17T04:27:42.186212+0000","last_fullsized":"2024-09-17T04:27:42.186212+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.206084+0000","last_change":"2024-09-17T04:27:43.200897+0000","last_active":"2024-09-17T04:27:43.206084+0000","last_peered":"2024-09-17T04:27:43.206084+0000","last_clean":"2024-09-17T04:27:43.206084+0000","last_became_active":"2024-09-17T04:27:43.200425+0000","last_became_peered":"2024-09-17T04:27:43.200425+0000","last_unstale":"2024-09-17T04:27:43.206084+0000","last_undegraded":"2024-09-17T04:27:43.206084+0000","last_fullsized":"2024-09-17T04:27:43.206084+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:29:36.941642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.208984+0000","last_change":"2024-09-17T04:27:43.203031+0000","last_active":"2024-09-17T04:27:43.208984+0000","last_peered":"2024-09-17T04:27:43.208984+0000","last_clean":"2024-09-17T04:27:43.208984+0000","last_became_active":"2024-09-17T04:27:43.202637+0000","last_became_peered":"2024-09-17T04:27:43.202637+0000","last_unstale":"2024-09-17T04:27:43.208984+0000","last_undegraded":"2024-09-17T04:27:43.208984+0000","last_fullsized":"2024-09-17T04:27:43.208984+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:12:58.279556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.213008+0000","last_change":"2024-09-17T04:27:44.217560+0000","last_active":"2024-09-17T04:27:45.213008+0000","last_peered":"2024-09-17T04:27:45.213008+0000","last_clean":"2024-09-17T04:27:45.213008+0000","last_became_active":"2024-09-17T04:27:44.217250+0000","last_became_peered":"2024-09-17T04:27:44.217250+0000","last_unstale":"2024-09-17T04:27:45.213008+0000","last_undegraded":"2024-09-17T04:27:45.213008+0000","last_fullsized":"2024-09-17T04:27:45.213008+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:47:24.767897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.188211+0000","last_change":"2024-09-17T04:27:42.188211+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.188211+0000","last_undegraded":"2024-09-17T04:27:42.188211+0000","last_fullsized":"2024-09-17T04:27:42.188211+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.183762+0000","last_change":"2024-09-17T04:27:43.183762+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.183762+0000","last_undegraded":"2024-09-17T04:27:43.183762+0000","last_fullsized":"2024-09-17T04:27:43.183762+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.209037+0000","last_change":"2024-09-17T04:27:43.201986+0000","last_active":"2024-09-17T04:27:43.209037+0000","last_peered":"2024-09-17T04:27:43.209037+0000","last_clean":"2024-09-17T04:27:43.209037+0000","last_became_active":"2024-09-17T04:27:43.201654+0000","last_became_peered":"2024-09-17T04:27:43.201654+0000","last_unstale":"2024-09-17T04:27:43.209037+0000","last_undegraded":"2024-09-17T04:27:43.209037+0000","last_fullsized":"2024-09-17T04:27:43.209037+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:43:24.891449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.188178+0000","last_change":"2024-09-17T04:27:42.188178+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.188178+0000","last_undegraded":"2024-09-17T04:27:42.188178+0000","last_fullsized":"2024-09-17T04:27:42.188178+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212826+0000","last_change":"2024-09-17T04:27:44.223877+0000","last_active":"2024-09-17T04:27:45.212826+0000","last_peered":"2024-09-17T04:27:45.212826+0000","last_clean":"2024-09-17T04:27:45.212826+0000","last_became_active":"2024-09-17T04:27:44.223582+0000","last_became_peered":"2024-09-17T04:27:44.223582+0000","last_unstale":"2024-09-17T04:27:45.212826+0000","last_undegraded":"2024-09-17T04:27:45.212826+0000","last_fullsized":"2024-09-17T04:27:45.212826+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:09.084549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212546+0000","last_change":"2024-09-17T04:27:43.201747+0000","last_active":"2024-09-17T04:27:45.212546+0000","last_peered":"2024-09-17T04:27:45.212546+0000","last_clean":"2024-09-17T04:27:45.212546+0000","last_became_active":"2024-09-17T04:27:43.200792+0000","last_became_peered":"2024-09-17T04:27:43.200792+0000","last_unstale":"2024-09-17T04:27:45.212546+0000","last_undegraded":"2024-09-17T04:27:45.212546+0000","last_fullsized":"2024-09-17T04:27:45.212546+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:30.181955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212302+0000","last_change":"2024-09-17T04:27:44.223584+0000","last_active":"2024-09-17T04:27:45.212302+0000","last_peered":"2024-09-17T04:27:45.212302+0000","last_clean":"2024-09-17T04:27:45.212302+0000","last_became_active":"2024-09-17T04:27:44.220615+0000","last_became_peered":"2024-09-17T04:27:44.220615+0000","last_unstale":"2024-09-17T04:27:45.212302+0000","last_undegraded":"2024-09-17T04:27:45.212302+0000","last_fullsized":"2024-09-17T04:27:45.212302+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:46.132448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.207926+0000","last_change":"2024-09-17T04:27:43.203968+0000","last_active":"2024-09-17T04:27:43.207926+0000","last_peered":"2024-09-17T04:27:43.207926+0000","last_clean":"2024-09-17T04:27:43.207926+0000","last_became_active":"2024-09-17T04:27:43.203677+0000","last_became_peered":"2024-09-17T04:27:43.203677+0000","last_unstale":"2024-09-17T04:27:43.207926+0000","last_undegraded":"2024-09-17T04:27:43.207926+0000","last_fullsized":"2024-09-17T04:27:43.207926+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:47.357122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":180,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.191448+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:27:43.191448+0000","last_peered":"2024-09-17T04:27:43.191448+0000","last_clean":"2024-09-17T04:27:43.191448+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:27:43.191448+0000","last_undegraded":"2024-09-17T04:27:43.191448+0000","last_fullsized":"2024-09-17T04:27:43.191448+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":84,"acting":84,"num_store_stats":5},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":56,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476785,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1290240,"data_stored":876659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476784,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":696320,"data_stored":286291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476785,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":696320,"data_stored":286291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476785,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":696320,"data_stored":286291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509488,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27696,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":684032,"data_stored":280024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509487,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":684032,"data_stored":280024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509487,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1277952,"data_stored":870392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542193,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":729088,"data_stored":299422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:27:49.110 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:27:49.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.371+0000 7f1622e00640 1 -- 172.21.15.100:0/1752460151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1604005280 msgr2=0x7f160400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:49.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.371+0000 7f1622e00640 1 --2- 172.21.15.100:0/1752460151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1604005280 0x7f160400b9c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1614001f30 tx=0x7f1604005830 comp rx=0 tx=0).stop 2024-09-17T04:27:49.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.371+0000 7f1622e00640 1 -- 172.21.15.100:0/1752460151 shutdown_connections 2024-09-17T04:27:49.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.371+0000 7f1622e00640 1 --2- 172.21.15.100:0/1752460151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1604005280 0x7f160400b9c0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:49.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.371+0000 7f1622e00640 1 -- 172.21.15.100:0/1752460151 >> 172.21.15.100:0/1752460151 conn(0x7f161c09d820 msgr2=0x7f161c08bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:49.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.371+0000 7f1622e00640 1 -- 172.21.15.100:0/1752460151 shutdown_connections 2024-09-17T04:27:49.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.371+0000 7f1622e00640 1 -- 172.21.15.100:0/1752460151 wait complete. 2024-09-17T04:27:49.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.373+0000 7f1622e00640 1 Processor -- start 2024-09-17T04:27:49.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.374+0000 7f1622e00640 1 -- start start 2024-09-17T04:27:49.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.375+0000 7f1622e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1604005280 0x7f161c15e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:49.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.375+0000 7f1622e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f161c1e9cf0 0x7f161c0cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:49.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.375+0000 7f1622e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f161c1ea100 0x7f161c0cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:49.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.375+0000 7f1622e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f161c0b5c40 con 0x7f161c1e9cf0 2024-09-17T04:27:49.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.375+0000 7f1622e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f161c0b5ac0 con 0x7f1604005280 2024-09-17T04:27:49.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.375+0000 7f1622e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f161c0b4490 con 0x7f161c1ea100 2024-09-17T04:27:49.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.375+0000 7f1621000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f161c1e9cf0 0x7f161c0cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:49.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.375+0000 7f1621000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f161c1e9cf0 0x7f161c0cc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56870/0 (socket says 172.21.15.100:56870) 2024-09-17T04:27:49.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.375+0000 7f1621000640 1 -- 172.21.15.100:0/2137842628 learned_addr learned my addr 172.21.15.100:0/2137842628 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:49.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.376+0000 7f1621a00640 1 --2- 172.21.15.100:0/2137842628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1604005280 0x7f161c15e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:49.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.376+0000 7f1622400640 1 --2- 172.21.15.100:0/2137842628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f161c1ea100 0x7f161c0cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:49.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.376+0000 7f1621000640 1 -- 172.21.15.100:0/2137842628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f161c1ea100 msgr2=0x7f161c0cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:49.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.376+0000 7f1621000640 1 --2- 172.21.15.100:0/2137842628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f161c1ea100 0x7f161c0cca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:49.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.376+0000 7f1621000640 1 -- 172.21.15.100:0/2137842628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1604005280 msgr2=0x7f161c15e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:49.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.376+0000 7f1621000640 1 --2- 172.21.15.100:0/2137842628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1604005280 0x7f161c15e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:49.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.376+0000 7f1621000640 1 -- 172.21.15.100:0/2137842628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161c0ca490 con 0x7f161c1e9cf0 2024-09-17T04:27:49.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.376+0000 7f1621000640 1 --2- 172.21.15.100:0/2137842628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f161c1e9cf0 0x7f161c0cc4d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1608006fd0 tx=0x7f16080039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:49.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.376+0000 7f1619600640 1 -- 172.21.15.100:0/2137842628 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1608015070 con 0x7f161c1e9cf0 2024-09-17T04:27:49.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.376+0000 7f1619600640 1 -- 172.21.15.100:0/2137842628 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1608011840 con 0x7f161c1e9cf0 2024-09-17T04:27:49.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.376+0000 7f1622e00640 1 -- 172.21.15.100:0/2137842628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f161c0ca770 con 0x7f161c1e9cf0 2024-09-17T04:27:49.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.376+0000 7f1619600640 1 -- 172.21.15.100:0/2137842628 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1608011c90 con 0x7f161c1e9cf0 2024-09-17T04:27:49.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.377+0000 7f1622e00640 1 -- 172.21.15.100:0/2137842628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f161c1e89b0 con 0x7f161c1e9cf0 2024-09-17T04:27:49.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.378+0000 7f1622e00640 1 -- 172.21.15.100:0/2137842628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f161c0ca200 con 0x7f161c1e9cf0 2024-09-17T04:27:49.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.379+0000 7f1619600640 1 -- 172.21.15.100:0/2137842628 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f1608018070 con 0x7f161c1e9cf0 2024-09-17T04:27:49.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.380+0000 7f1619600640 1 --2- 172.21.15.100:0/2137842628 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f15fc078d70 0x7f15fc07b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:49.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.380+0000 7f1619600640 1 -- 172.21.15.100:0/2137842628 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7f160809bec0 con 0x7f161c1e9cf0 2024-09-17T04:27:49.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.381+0000 7f1621a00640 1 --2- 172.21.15.100:0/2137842628 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f15fc078d70 0x7f15fc07b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:49.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.382+0000 7f1621a00640 1 --2- 172.21.15.100:0/2137842628 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f15fc078d70 0x7f15fc07b280 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1610009ca0 tx=0x7f1610005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:49.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.385+0000 7f1619600640 1 -- 172.21.15.100:0/2137842628 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f1608063f70 con 0x7f161c1e9cf0 2024-09-17T04:27:49.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.565+0000 7f1622e00640 1 -- 172.21.15.100:0/2137842628 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f161c0b49a0 con 0x7f15fc078d70 2024-09-17T04:27:49.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.570+0000 7f1619600640 1 -- 172.21.15.100:0/2137842628 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245270 (secure 0 0 0) 0x7f161c0b49a0 con 0x7f15fc078d70 2024-09-17T04:27:49.571 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:27:49.573 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:27:49.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.573+0000 7f1622e00640 1 -- 172.21.15.100:0/2137842628 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f15fc078d70 msgr2=0x7f15fc07b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:49.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.573+0000 7f1622e00640 1 --2- 172.21.15.100:0/2137842628 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f15fc078d70 0x7f15fc07b280 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1610009ca0 tx=0x7f1610005f90 comp rx=0 tx=0).stop 2024-09-17T04:27:49.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.573+0000 7f1622e00640 1 -- 172.21.15.100:0/2137842628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f161c1e9cf0 msgr2=0x7f161c0cc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:49.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.573+0000 7f1622e00640 1 --2- 172.21.15.100:0/2137842628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f161c1e9cf0 0x7f161c0cc4d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1608006fd0 tx=0x7f16080039c0 comp rx=0 tx=0).stop 2024-09-17T04:27:49.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.574+0000 7f1622e00640 1 -- 172.21.15.100:0/2137842628 shutdown_connections 2024-09-17T04:27:49.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.574+0000 7f1622e00640 1 --2- 172.21.15.100:0/2137842628 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f15fc078d70 0x7f15fc07b280 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:49.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.574+0000 7f1622e00640 1 --2- 172.21.15.100:0/2137842628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f161c1ea100 0x7f161c0cca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:49.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.574+0000 7f1622e00640 1 --2- 172.21.15.100:0/2137842628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f161c1e9cf0 0x7f161c0cc4d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:49.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.574+0000 7f1622e00640 1 --2- 172.21.15.100:0/2137842628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1604005280 0x7f161c15e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:49.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.574+0000 7f1622e00640 1 -- 172.21.15.100:0/2137842628 >> 172.21.15.100:0/2137842628 conn(0x7f161c09d820 msgr2=0x7f161c0b3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:49.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.574+0000 7f1622e00640 1 -- 172.21.15.100:0/2137842628 shutdown_connections 2024-09-17T04:27:49.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.574+0000 7f1622e00640 1 -- 172.21.15.100:0/2137842628 wait complete. 2024-09-17T04:27:49.595 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-09-17T04:27:48.024133+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590424,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":106,"num_write_kb":1636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":150,"acting":150,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":235328,"kb_used_data":6596,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754739392,"statfs":{"total":773094113280,"available":772853137408,"internally_reserved":0,"allocated":6754304,"data_stored":3465394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":234016308},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":56,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.846898"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212739+0000","last_change":"2024-09-17T04:27:44.238721+0000","last_active":"2024-09-17T04:27:45.212739+0000","last_peered":"2024-09-17T04:27:45.212739+0000","last_clean":"2024-09-17T04:27:45.212739+0000","last_became_active":"2024-09-17T04:27:44.237900+0000","last_became_peered":"2024-09-17T04:27:44.237900+0000","last_unstale":"2024-09-17T04:27:45.212739+0000","last_undegraded":"2024-09-17T04:27:45.212739+0000","last_fullsized":"2024-09-17T04:27:45.212739+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:09:57.201730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.188353+0000","last_change":"2024-09-17T04:27:43.188353+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.188353+0000","last_undegraded":"2024-09-17T04:27:43.188353+0000","last_fullsized":"2024-09-17T04:27:43.188353+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.194226+0000","last_change":"2024-09-17T04:27:43.194226+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.194226+0000","last_undegraded":"2024-09-17T04:27:43.194226+0000","last_fullsized":"2024-09-17T04:27:43.194226+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.190634+0000","last_change":"2024-09-17T04:27:43.190634+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.190634+0000","last_undegraded":"2024-09-17T04:27:43.190634+0000","last_fullsized":"2024-09-17T04:27:43.190634+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212387+0000","last_change":"2024-09-17T04:27:44.222903+0000","last_active":"2024-09-17T04:27:45.212387+0000","last_peered":"2024-09-17T04:27:45.212387+0000","last_clean":"2024-09-17T04:27:45.212387+0000","last_became_active":"2024-09-17T04:27:44.222587+0000","last_became_peered":"2024-09-17T04:27:44.222587+0000","last_unstale":"2024-09-17T04:27:45.212387+0000","last_undegraded":"2024-09-17T04:27:45.212387+0000","last_fullsized":"2024-09-17T04:27:45.212387+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:37:39.457226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.189985+0000","last_change":"2024-09-17T04:27:43.189985+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.189985+0000","last_undegraded":"2024-09-17T04:27:43.189985+0000","last_fullsized":"2024-09-17T04:27:43.189985+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.193736+0000","last_change":"2024-09-17T04:27:43.193736+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.193736+0000","last_undegraded":"2024-09-17T04:27:43.193736+0000","last_fullsized":"2024-09-17T04:27:43.193736+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.184126+0000","last_change":"2024-09-17T04:27:43.184126+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.184126+0000","last_undegraded":"2024-09-17T04:27:43.184126+0000","last_fullsized":"2024-09-17T04:27:43.184126+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.189690+0000","last_change":"2024-09-17T04:27:43.189690+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.189690+0000","last_undegraded":"2024-09-17T04:27:43.189690+0000","last_fullsized":"2024-09-17T04:27:43.189690+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.188426+0000","last_change":"2024-09-17T04:27:43.188426+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.188426+0000","last_undegraded":"2024-09-17T04:27:43.188426+0000","last_fullsized":"2024-09-17T04:27:43.188426+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212938+0000","last_change":"2024-09-17T04:27:44.221917+0000","last_active":"2024-09-17T04:27:45.212938+0000","last_peered":"2024-09-17T04:27:45.212938+0000","last_clean":"2024-09-17T04:27:45.212938+0000","last_became_active":"2024-09-17T04:27:44.221632+0000","last_became_peered":"2024-09-17T04:27:44.221632+0000","last_unstale":"2024-09-17T04:27:45.212938+0000","last_undegraded":"2024-09-17T04:27:45.212938+0000","last_fullsized":"2024-09-17T04:27:45.212938+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:27.954596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.185462+0000","last_change":"2024-09-17T04:27:43.185462+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.185462+0000","last_undegraded":"2024-09-17T04:27:43.185462+0000","last_fullsized":"2024-09-17T04:27:43.185462+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212533+0000","last_change":"2024-09-17T04:27:44.220909+0000","last_active":"2024-09-17T04:27:45.212533+0000","last_peered":"2024-09-17T04:27:45.212533+0000","last_clean":"2024-09-17T04:27:45.212533+0000","last_became_active":"2024-09-17T04:27:44.220605+0000","last_became_peered":"2024-09-17T04:27:44.220605+0000","last_unstale":"2024-09-17T04:27:45.212533+0000","last_undegraded":"2024-09-17T04:27:45.212533+0000","last_fullsized":"2024-09-17T04:27:45.212533+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:48:13.238235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.188790+0000","last_change":"2024-09-17T04:27:43.188790+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.188790+0000","last_undegraded":"2024-09-17T04:27:43.188790+0000","last_fullsized":"2024-09-17T04:27:43.188790+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212746+0000","last_change":"2024-09-17T04:27:44.216830+0000","last_active":"2024-09-17T04:27:45.212746+0000","last_peered":"2024-09-17T04:27:45.212746+0000","last_clean":"2024-09-17T04:27:45.212746+0000","last_became_active":"2024-09-17T04:27:44.216520+0000","last_became_peered":"2024-09-17T04:27:44.216520+0000","last_unstale":"2024-09-17T04:27:45.212746+0000","last_undegraded":"2024-09-17T04:27:45.212746+0000","last_fullsized":"2024-09-17T04:27:45.212746+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:15:47.203299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.185548+0000","last_change":"2024-09-17T04:27:43.185548+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.185548+0000","last_undegraded":"2024-09-17T04:27:43.185548+0000","last_fullsized":"2024-09-17T04:27:43.185548+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.185510+0000","last_change":"2024-09-17T04:27:43.185510+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.185510+0000","last_undegraded":"2024-09-17T04:27:43.185510+0000","last_fullsized":"2024-09-17T04:27:43.185510+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.192118+0000","last_change":"2024-09-17T04:27:43.192118+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.192118+0000","last_undegraded":"2024-09-17T04:27:43.192118+0000","last_fullsized":"2024-09-17T04:27:43.192118+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.213050+0000","last_change":"2024-09-17T04:27:44.218277+0000","last_active":"2024-09-17T04:27:45.213050+0000","last_peered":"2024-09-17T04:27:45.213050+0000","last_clean":"2024-09-17T04:27:45.213050+0000","last_became_active":"2024-09-17T04:27:44.217942+0000","last_became_peered":"2024-09-17T04:27:44.217942+0000","last_unstale":"2024-09-17T04:27:45.213050+0000","last_undegraded":"2024-09-17T04:27:45.213050+0000","last_fullsized":"2024-09-17T04:27:45.213050+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:18.353865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.183706+0000","last_change":"2024-09-17T04:27:43.183706+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.183706+0000","last_undegraded":"2024-09-17T04:27:43.183706+0000","last_fullsized":"2024-09-17T04:27:43.183706+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.186596+0000","last_change":"2024-09-17T04:27:42.186596+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.186596+0000","last_undegraded":"2024-09-17T04:27:42.186596+0000","last_fullsized":"2024-09-17T04:27:42.186596+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"60'1","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212757+0000","last_change":"2024-09-17T04:27:43.206766+0000","last_active":"2024-09-17T04:27:45.212757+0000","last_peered":"2024-09-17T04:27:45.212757+0000","last_clean":"2024-09-17T04:27:45.212757+0000","last_became_active":"2024-09-17T04:27:43.206455+0000","last_became_peered":"2024-09-17T04:27:43.206455+0000","last_unstale":"2024-09-17T04:27:45.212757+0000","last_undegraded":"2024-09-17T04:27:45.212757+0000","last_fullsized":"2024-09-17T04:27:45.212757+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:06:22.067347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.186307+0000","last_change":"2024-09-17T04:27:42.186307+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.186307+0000","last_undegraded":"2024-09-17T04:27:42.186307+0000","last_fullsized":"2024-09-17T04:27:42.186307+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.209139+0000","last_change":"2024-09-17T04:27:43.207037+0000","last_active":"2024-09-17T04:27:43.209139+0000","last_peered":"2024-09-17T04:27:43.209139+0000","last_clean":"2024-09-17T04:27:43.209139+0000","last_became_active":"2024-09-17T04:27:43.206672+0000","last_became_peered":"2024-09-17T04:27:43.206672+0000","last_unstale":"2024-09-17T04:27:43.209139+0000","last_undegraded":"2024-09-17T04:27:43.209139+0000","last_fullsized":"2024-09-17T04:27:43.209139+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:10.033325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.213106+0000","last_change":"2024-09-17T04:27:44.217751+0000","last_active":"2024-09-17T04:27:45.213106+0000","last_peered":"2024-09-17T04:27:45.213106+0000","last_clean":"2024-09-17T04:27:45.213106+0000","last_became_active":"2024-09-17T04:27:44.214072+0000","last_became_peered":"2024-09-17T04:27:44.214072+0000","last_unstale":"2024-09-17T04:27:45.213106+0000","last_undegraded":"2024-09-17T04:27:45.213106+0000","last_fullsized":"2024-09-17T04:27:45.213106+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:27.160546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.208764+0000","last_change":"2024-09-17T04:27:43.198922+0000","last_active":"2024-09-17T04:27:43.208764+0000","last_peered":"2024-09-17T04:27:43.208764+0000","last_clean":"2024-09-17T04:27:43.208764+0000","last_became_active":"2024-09-17T04:27:43.198453+0000","last_became_peered":"2024-09-17T04:27:43.198453+0000","last_unstale":"2024-09-17T04:27:43.208764+0000","last_undegraded":"2024-09-17T04:27:43.208764+0000","last_fullsized":"2024-09-17T04:27:43.208764+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:46.166554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212477+0000","last_change":"2024-09-17T04:27:44.222601+0000","last_active":"2024-09-17T04:27:45.212477+0000","last_peered":"2024-09-17T04:27:45.212477+0000","last_clean":"2024-09-17T04:27:45.212477+0000","last_became_active":"2024-09-17T04:27:44.221587+0000","last_became_peered":"2024-09-17T04:27:44.221587+0000","last_unstale":"2024-09-17T04:27:45.212477+0000","last_undegraded":"2024-09-17T04:27:45.212477+0000","last_fullsized":"2024-09-17T04:27:45.212477+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:22.587679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.185403+0000","last_change":"2024-09-17T04:27:42.185403+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.185403+0000","last_undegraded":"2024-09-17T04:27:42.185403+0000","last_fullsized":"2024-09-17T04:27:42.185403+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.183685+0000","last_change":"2024-09-17T04:27:43.183685+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.183685+0000","last_undegraded":"2024-09-17T04:27:43.183685+0000","last_fullsized":"2024-09-17T04:27:43.183685+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.207657+0000","last_change":"2024-09-17T04:27:43.200857+0000","last_active":"2024-09-17T04:27:43.207657+0000","last_peered":"2024-09-17T04:27:43.207657+0000","last_clean":"2024-09-17T04:27:43.207657+0000","last_became_active":"2024-09-17T04:27:43.200518+0000","last_became_peered":"2024-09-17T04:27:43.200518+0000","last_unstale":"2024-09-17T04:27:43.207657+0000","last_undegraded":"2024-09-17T04:27:43.207657+0000","last_fullsized":"2024-09-17T04:27:43.207657+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:11.559902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212733+0000","last_change":"2024-09-17T04:27:44.225414+0000","last_active":"2024-09-17T04:27:45.212733+0000","last_peered":"2024-09-17T04:27:45.212733+0000","last_clean":"2024-09-17T04:27:45.212733+0000","last_became_active":"2024-09-17T04:27:44.225112+0000","last_became_peered":"2024-09-17T04:27:44.225112+0000","last_unstale":"2024-09-17T04:27:45.212733+0000","last_undegraded":"2024-09-17T04:27:45.212733+0000","last_fullsized":"2024-09-17T04:27:45.212733+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:05:18.890521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.213768+0000","last_change":"2024-09-17T04:27:43.209713+0000","last_active":"2024-09-17T04:27:43.213768+0000","last_peered":"2024-09-17T04:27:43.213768+0000","last_clean":"2024-09-17T04:27:43.213768+0000","last_became_active":"2024-09-17T04:27:43.209316+0000","last_became_peered":"2024-09-17T04:27:43.209316+0000","last_unstale":"2024-09-17T04:27:43.213768+0000","last_undegraded":"2024-09-17T04:27:43.213768+0000","last_fullsized":"2024-09-17T04:27:43.213768+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:01.773526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.189782+0000","last_change":"2024-09-17T04:27:43.189782+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.189782+0000","last_undegraded":"2024-09-17T04:27:43.189782+0000","last_fullsized":"2024-09-17T04:27:43.189782+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.206591+0000","last_change":"2024-09-17T04:27:43.204007+0000","last_active":"2024-09-17T04:27:43.206591+0000","last_peered":"2024-09-17T04:27:43.206591+0000","last_clean":"2024-09-17T04:27:43.206591+0000","last_became_active":"2024-09-17T04:27:43.203747+0000","last_became_peered":"2024-09-17T04:27:43.203747+0000","last_unstale":"2024-09-17T04:27:43.206591+0000","last_undegraded":"2024-09-17T04:27:43.206591+0000","last_fullsized":"2024-09-17T04:27:43.206591+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:56:12.570282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.185434+0000","last_change":"2024-09-17T04:27:43.185434+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.185434+0000","last_undegraded":"2024-09-17T04:27:43.185434+0000","last_fullsized":"2024-09-17T04:27:43.185434+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212854+0000","last_change":"2024-09-17T04:27:43.201502+0000","last_active":"2024-09-17T04:27:45.212854+0000","last_peered":"2024-09-17T04:27:45.212854+0000","last_clean":"2024-09-17T04:27:45.212854+0000","last_became_active":"2024-09-17T04:27:43.201071+0000","last_became_peered":"2024-09-17T04:27:43.201071+0000","last_unstale":"2024-09-17T04:27:45.212854+0000","last_undegraded":"2024-09-17T04:27:45.212854+0000","last_fullsized":"2024-09-17T04:27:45.212854+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:48:58.744170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.206263+0000","last_change":"2024-09-17T04:27:43.203357+0000","last_active":"2024-09-17T04:27:43.206263+0000","last_peered":"2024-09-17T04:27:43.206263+0000","last_clean":"2024-09-17T04:27:43.206263+0000","last_became_active":"2024-09-17T04:27:43.202999+0000","last_became_peered":"2024-09-17T04:27:43.202999+0000","last_unstale":"2024-09-17T04:27:43.206263+0000","last_undegraded":"2024-09-17T04:27:43.206263+0000","last_fullsized":"2024-09-17T04:27:43.206263+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:10.033325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.187796+0000","last_change":"2024-09-17T04:27:43.187796+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.187796+0000","last_undegraded":"2024-09-17T04:27:43.187796+0000","last_fullsized":"2024-09-17T04:27:43.187796+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.208866+0000","last_change":"2024-09-17T04:27:43.200549+0000","last_active":"2024-09-17T04:27:43.208866+0000","last_peered":"2024-09-17T04:27:43.208866+0000","last_clean":"2024-09-17T04:27:43.208866+0000","last_became_active":"2024-09-17T04:27:43.200245+0000","last_became_peered":"2024-09-17T04:27:43.200245+0000","last_unstale":"2024-09-17T04:27:43.208866+0000","last_undegraded":"2024-09-17T04:27:43.208866+0000","last_fullsized":"2024-09-17T04:27:43.208866+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:44.583373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.190626+0000","last_change":"2024-09-17T04:27:43.190626+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.190626+0000","last_undegraded":"2024-09-17T04:27:43.190626+0000","last_fullsized":"2024-09-17T04:27:43.190626+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"60'2","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212414+0000","last_change":"2024-09-17T04:27:43.204124+0000","last_active":"2024-09-17T04:27:45.212414+0000","last_peered":"2024-09-17T04:27:45.212414+0000","last_clean":"2024-09-17T04:27:45.212414+0000","last_became_active":"2024-09-17T04:27:43.203730+0000","last_became_peered":"2024-09-17T04:27:43.203730+0000","last_unstale":"2024-09-17T04:27:45.212414+0000","last_undegraded":"2024-09-17T04:27:45.212414+0000","last_fullsized":"2024-09-17T04:27:45.212414+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:15:50.701517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.187412+0000","last_change":"2024-09-17T04:27:43.187412+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.187412+0000","last_undegraded":"2024-09-17T04:27:43.187412+0000","last_fullsized":"2024-09-17T04:27:43.187412+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.200751+0000","last_change":"2024-09-17T04:27:43.191819+0000","last_active":"2024-09-17T04:27:43.200751+0000","last_peered":"2024-09-17T04:27:43.200751+0000","last_clean":"2024-09-17T04:27:43.200751+0000","last_became_active":"2024-09-17T04:27:43.191432+0000","last_became_peered":"2024-09-17T04:27:43.191432+0000","last_unstale":"2024-09-17T04:27:43.200751+0000","last_undegraded":"2024-09-17T04:27:43.200751+0000","last_fullsized":"2024-09-17T04:27:43.200751+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:31.004288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.188547+0000","last_change":"2024-09-17T04:27:43.188547+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.188547+0000","last_undegraded":"2024-09-17T04:27:43.188547+0000","last_fullsized":"2024-09-17T04:27:43.188547+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.209284+0000","last_change":"2024-09-17T04:27:43.206779+0000","last_active":"2024-09-17T04:27:43.209284+0000","last_peered":"2024-09-17T04:27:43.209284+0000","last_clean":"2024-09-17T04:27:43.209284+0000","last_became_active":"2024-09-17T04:27:43.205945+0000","last_became_peered":"2024-09-17T04:27:43.205945+0000","last_unstale":"2024-09-17T04:27:43.209284+0000","last_undegraded":"2024-09-17T04:27:43.209284+0000","last_fullsized":"2024-09-17T04:27:43.209284+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:03.547426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.185001+0000","last_change":"2024-09-17T04:27:43.185001+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.185001+0000","last_undegraded":"2024-09-17T04:27:43.185001+0000","last_fullsized":"2024-09-17T04:27:43.185001+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.206344+0000","last_change":"2024-09-17T04:27:43.203507+0000","last_active":"2024-09-17T04:27:43.206344+0000","last_peered":"2024-09-17T04:27:43.206344+0000","last_clean":"2024-09-17T04:27:43.206344+0000","last_became_active":"2024-09-17T04:27:43.202223+0000","last_became_peered":"2024-09-17T04:27:43.202223+0000","last_unstale":"2024-09-17T04:27:43.206344+0000","last_undegraded":"2024-09-17T04:27:43.206344+0000","last_fullsized":"2024-09-17T04:27:43.206344+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:05.218532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.184912+0000","last_change":"2024-09-17T04:27:43.184912+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.184912+0000","last_undegraded":"2024-09-17T04:27:43.184912+0000","last_fullsized":"2024-09-17T04:27:43.184912+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"60'1","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.213160+0000","last_change":"2024-09-17T04:27:43.206771+0000","last_active":"2024-09-17T04:27:45.213160+0000","last_peered":"2024-09-17T04:27:45.213160+0000","last_clean":"2024-09-17T04:27:45.213160+0000","last_became_active":"2024-09-17T04:27:43.206481+0000","last_became_peered":"2024-09-17T04:27:43.206481+0000","last_unstale":"2024-09-17T04:27:45.213160+0000","last_undegraded":"2024-09-17T04:27:45.213160+0000","last_fullsized":"2024-09-17T04:27:45.213160+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:48:49.832510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.193299+0000","last_change":"2024-09-17T04:27:43.193299+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.193299+0000","last_undegraded":"2024-09-17T04:27:43.193299+0000","last_fullsized":"2024-09-17T04:27:43.193299+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.186798+0000","last_change":"2024-09-17T04:27:42.186798+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.186798+0000","last_undegraded":"2024-09-17T04:27:42.186798+0000","last_fullsized":"2024-09-17T04:27:42.186798+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212603+0000","last_change":"2024-09-17T04:27:44.222455+0000","last_active":"2024-09-17T04:27:45.212603+0000","last_peered":"2024-09-17T04:27:45.212603+0000","last_clean":"2024-09-17T04:27:45.212603+0000","last_became_active":"2024-09-17T04:27:44.222015+0000","last_became_peered":"2024-09-17T04:27:44.222015+0000","last_unstale":"2024-09-17T04:27:45.212603+0000","last_undegraded":"2024-09-17T04:27:45.212603+0000","last_fullsized":"2024-09-17T04:27:45.212603+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:01:05.264040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.209162+0000","last_change":"2024-09-17T04:27:43.207246+0000","last_active":"2024-09-17T04:27:43.209162+0000","last_peered":"2024-09-17T04:27:43.209162+0000","last_clean":"2024-09-17T04:27:43.209162+0000","last_became_active":"2024-09-17T04:27:43.206835+0000","last_became_peered":"2024-09-17T04:27:43.206835+0000","last_unstale":"2024-09-17T04:27:43.209162+0000","last_undegraded":"2024-09-17T04:27:43.209162+0000","last_fullsized":"2024-09-17T04:27:43.209162+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:05.218532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.186711+0000","last_change":"2024-09-17T04:27:43.186711+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.186711+0000","last_undegraded":"2024-09-17T04:27:43.186711+0000","last_fullsized":"2024-09-17T04:27:43.186711+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.185903+0000","last_change":"2024-09-17T04:27:42.185903+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.185903+0000","last_undegraded":"2024-09-17T04:27:42.185903+0000","last_fullsized":"2024-09-17T04:27:42.185903+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.189283+0000","last_change":"2024-09-17T04:27:43.189283+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.189283+0000","last_undegraded":"2024-09-17T04:27:43.189283+0000","last_fullsized":"2024-09-17T04:27:43.189283+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"60'2","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212605+0000","last_change":"2024-09-17T04:27:43.203669+0000","last_active":"2024-09-17T04:27:45.212605+0000","last_peered":"2024-09-17T04:27:45.212605+0000","last_clean":"2024-09-17T04:27:45.212605+0000","last_became_active":"2024-09-17T04:27:43.203307+0000","last_became_peered":"2024-09-17T04:27:43.203307+0000","last_unstale":"2024-09-17T04:27:45.212605+0000","last_undegraded":"2024-09-17T04:27:45.212605+0000","last_fullsized":"2024-09-17T04:27:45.212605+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:53.295497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.188161+0000","last_change":"2024-09-17T04:27:42.188161+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.188161+0000","last_undegraded":"2024-09-17T04:27:42.188161+0000","last_fullsized":"2024-09-17T04:27:42.188161+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.210443+0000","last_change":"2024-09-17T04:27:43.209348+0000","last_active":"2024-09-17T04:27:43.210443+0000","last_peered":"2024-09-17T04:27:43.210443+0000","last_clean":"2024-09-17T04:27:43.210443+0000","last_became_active":"2024-09-17T04:27:43.209055+0000","last_became_peered":"2024-09-17T04:27:43.209055+0000","last_unstale":"2024-09-17T04:27:43.210443+0000","last_undegraded":"2024-09-17T04:27:43.210443+0000","last_fullsized":"2024-09-17T04:27:43.210443+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:56:12.570282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.191601+0000","last_change":"2024-09-17T04:27:43.191601+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.191601+0000","last_undegraded":"2024-09-17T04:27:43.191601+0000","last_fullsized":"2024-09-17T04:27:43.191601+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.188855+0000","last_change":"2024-09-17T04:27:42.188855+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.188855+0000","last_undegraded":"2024-09-17T04:27:42.188855+0000","last_fullsized":"2024-09-17T04:27:42.188855+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.192785+0000","last_change":"2024-09-17T04:27:43.192785+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.192785+0000","last_undegraded":"2024-09-17T04:27:43.192785+0000","last_fullsized":"2024-09-17T04:27:43.192785+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.186212+0000","last_change":"2024-09-17T04:27:42.186212+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.186212+0000","last_undegraded":"2024-09-17T04:27:42.186212+0000","last_fullsized":"2024-09-17T04:27:42.186212+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.206084+0000","last_change":"2024-09-17T04:27:43.200897+0000","last_active":"2024-09-17T04:27:43.206084+0000","last_peered":"2024-09-17T04:27:43.206084+0000","last_clean":"2024-09-17T04:27:43.206084+0000","last_became_active":"2024-09-17T04:27:43.200425+0000","last_became_peered":"2024-09-17T04:27:43.200425+0000","last_unstale":"2024-09-17T04:27:43.206084+0000","last_undegraded":"2024-09-17T04:27:43.206084+0000","last_fullsized":"2024-09-17T04:27:43.206084+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:29:36.941642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.208984+0000","last_change":"2024-09-17T04:27:43.203031+0000","last_active":"2024-09-17T04:27:43.208984+0000","last_peered":"2024-09-17T04:27:43.208984+0000","last_clean":"2024-09-17T04:27:43.208984+0000","last_became_active":"2024-09-17T04:27:43.202637+0000","last_became_peered":"2024-09-17T04:27:43.202637+0000","last_unstale":"2024-09-17T04:27:43.208984+0000","last_undegraded":"2024-09-17T04:27:43.208984+0000","last_fullsized":"2024-09-17T04:27:43.208984+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:12:58.279556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.213008+0000","last_change":"2024-09-17T04:27:44.217560+0000","last_active":"2024-09-17T04:27:45.213008+0000","last_peered":"2024-09-17T04:27:45.213008+0000","last_clean":"2024-09-17T04:27:45.213008+0000","last_became_active":"2024-09-17T04:27:44.217250+0000","last_became_peered":"2024-09-17T04:27:44.217250+0000","last_unstale":"2024-09-17T04:27:45.213008+0000","last_undegraded":"2024-09-17T04:27:45.213008+0000","last_fullsized":"2024-09-17T04:27:45.213008+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:47:24.767897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.188211+0000","last_change":"2024-09-17T04:27:42.188211+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.188211+0000","last_undegraded":"2024-09-17T04:27:42.188211+0000","last_fullsized":"2024-09-17T04:27:42.188211+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-09-17T04:27:43.183762+0000","last_change":"2024-09-17T04:27:43.183762+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.183762+0000","last_undegraded":"2024-09-17T04:27:43.183762+0000","last_fullsized":"2024-09-17T04:27:43.183762+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.209037+0000","last_change":"2024-09-17T04:27:43.201986+0000","last_active":"2024-09-17T04:27:43.209037+0000","last_peered":"2024-09-17T04:27:43.209037+0000","last_clean":"2024-09-17T04:27:43.209037+0000","last_became_active":"2024-09-17T04:27:43.201654+0000","last_became_peered":"2024-09-17T04:27:43.201654+0000","last_unstale":"2024-09-17T04:27:43.209037+0000","last_undegraded":"2024-09-17T04:27:43.209037+0000","last_fullsized":"2024-09-17T04:27:43.209037+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:43:24.891449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:27:43.175780+0000","last_change":"2024-09-17T04:27:43.175780+0000","last_active":"2024-09-17T04:27:43.175780+0000","last_peered":"2024-09-17T04:27:43.175780+0000","last_clean":"2024-09-17T04:27:43.175780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:43.175780+0000","last_undegraded":"2024-09-17T04:27:43.175780+0000","last_fullsized":"2024-09-17T04:27:43.175780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T04:27:42.188178+0000","last_change":"2024-09-17T04:27:42.188178+0000","last_active":"2024-09-17T04:27:42.169395+0000","last_peered":"2024-09-17T04:27:42.169395+0000","last_clean":"2024-09-17T04:27:42.169395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:27:42.188178+0000","last_undegraded":"2024-09-17T04:27:42.188178+0000","last_fullsized":"2024-09-17T04:27:42.188178+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212826+0000","last_change":"2024-09-17T04:27:44.223877+0000","last_active":"2024-09-17T04:27:45.212826+0000","last_peered":"2024-09-17T04:27:45.212826+0000","last_clean":"2024-09-17T04:27:45.212826+0000","last_became_active":"2024-09-17T04:27:44.223582+0000","last_became_peered":"2024-09-17T04:27:44.223582+0000","last_unstale":"2024-09-17T04:27:45.212826+0000","last_undegraded":"2024-09-17T04:27:45.212826+0000","last_fullsized":"2024-09-17T04:27:45.212826+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:09.084549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212546+0000","last_change":"2024-09-17T04:27:43.201747+0000","last_active":"2024-09-17T04:27:45.212546+0000","last_peered":"2024-09-17T04:27:45.212546+0000","last_clean":"2024-09-17T04:27:45.212546+0000","last_became_active":"2024-09-17T04:27:43.200792+0000","last_became_peered":"2024-09-17T04:27:43.200792+0000","last_unstale":"2024-09-17T04:27:45.212546+0000","last_undegraded":"2024-09-17T04:27:45.212546+0000","last_fullsized":"2024-09-17T04:27:45.212546+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:30.181955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212302+0000","last_change":"2024-09-17T04:27:44.223584+0000","last_active":"2024-09-17T04:27:45.212302+0000","last_peered":"2024-09-17T04:27:45.212302+0000","last_clean":"2024-09-17T04:27:45.212302+0000","last_became_active":"2024-09-17T04:27:44.220615+0000","last_became_peered":"2024-09-17T04:27:44.220615+0000","last_unstale":"2024-09-17T04:27:45.212302+0000","last_undegraded":"2024-09-17T04:27:45.212302+0000","last_fullsized":"2024-09-17T04:27:45.212302+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:46.132448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.207926+0000","last_change":"2024-09-17T04:27:43.203968+0000","last_active":"2024-09-17T04:27:43.207926+0000","last_peered":"2024-09-17T04:27:43.207926+0000","last_clean":"2024-09-17T04:27:43.207926+0000","last_became_active":"2024-09-17T04:27:43.203677+0000","last_became_peered":"2024-09-17T04:27:43.203677+0000","last_unstale":"2024-09-17T04:27:43.207926+0000","last_undegraded":"2024-09-17T04:27:43.207926+0000","last_fullsized":"2024-09-17T04:27:43.207926+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:47.357122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":180,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T04:27:43.191448+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:27:43.191448+0000","last_peered":"2024-09-17T04:27:43.191448+0000","last_clean":"2024-09-17T04:27:43.191448+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:27:43.191448+0000","last_undegraded":"2024-09-17T04:27:43.191448+0000","last_fullsized":"2024-09-17T04:27:43.191448+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":84,"acting":84,"num_store_stats":5},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":56,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476785,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1290240,"data_stored":876659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476784,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":696320,"data_stored":286291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476785,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":696320,"data_stored":286291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476785,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":696320,"data_stored":286291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509488,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27696,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":684032,"data_stored":280024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509487,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":684032,"data_stored":280024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509487,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1277952,"data_stored":870392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542193,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":729088,"data_stored":299422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:27:49.599 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3f is not active+clean 2024-09-17T04:27:49.599 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T04:27:43.175780+0000', 'last_change': '2024-09-17T04:27:43.175780+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.175780+0000', 'last_undegraded': '2024-09-17T04:27:43.175780+0000', 'last_fullsized': '2024-09-17T04:27:43.175780+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T04:27:49.599 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3d is not active+clean 2024-09-17T04:27:49.599 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.188353+0000', 'last_change': '2024-09-17T04:27:43.188353+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.188353+0000', 'last_undegraded': '2024-09-17T04:27:43.188353+0000', 'last_fullsized': '2024-09-17T04:27:43.188353+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:27:49.599 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3c is not active+clean 2024-09-17T04:27:49.599 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.194226+0000', 'last_change': '2024-09-17T04:27:43.194226+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.194226+0000', 'last_undegraded': '2024-09-17T04:27:43.194226+0000', 'last_fullsized': '2024-09-17T04:27:43.194226+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T04:27:49.600 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3b is not active+clean 2024-09-17T04:27:49.600 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T04:27:43.175780+0000', 'last_change': '2024-09-17T04:27:43.175780+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.175780+0000', 'last_undegraded': '2024-09-17T04:27:43.175780+0000', 'last_fullsized': '2024-09-17T04:27:43.175780+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T04:27:49.600 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3a is not active+clean 2024-09-17T04:27:49.600 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T04:27:43.175780+0000', 'last_change': '2024-09-17T04:27:43.175780+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.175780+0000', 'last_undegraded': '2024-09-17T04:27:43.175780+0000', 'last_fullsized': '2024-09-17T04:27:43.175780+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T04:27:49.600 INFO:tasks.ceph.ceph_manager.ceph:PG 9.39 is not active+clean 2024-09-17T04:27:49.600 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.190634+0000', 'last_change': '2024-09-17T04:27:43.190634+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.190634+0000', 'last_undegraded': '2024-09-17T04:27:43.190634+0000', 'last_fullsized': '2024-09-17T04:27:43.190634+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T04:27:49.600 INFO:tasks.ceph.ceph_manager.ceph:PG 9.37 is not active+clean 2024-09-17T04:27:49.600 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.189985+0000', 'last_change': '2024-09-17T04:27:43.189985+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.189985+0000', 'last_undegraded': '2024-09-17T04:27:43.189985+0000', 'last_fullsized': '2024-09-17T04:27:43.189985+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T04:27:49.600 INFO:tasks.ceph.ceph_manager.ceph:PG 9.36 is not active+clean 2024-09-17T04:27:49.600 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T04:27:43.175780+0000', 'last_change': '2024-09-17T04:27:43.175780+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.175780+0000', 'last_undegraded': '2024-09-17T04:27:43.175780+0000', 'last_fullsized': '2024-09-17T04:27:43.175780+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T04:27:49.601 INFO:tasks.ceph.ceph_manager.ceph:PG 9.35 is not active+clean 2024-09-17T04:27:49.601 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T04:27:43.175780+0000', 'last_change': '2024-09-17T04:27:43.175780+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.175780+0000', 'last_undegraded': '2024-09-17T04:27:43.175780+0000', 'last_fullsized': '2024-09-17T04:27:43.175780+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T04:27:49.601 INFO:tasks.ceph.ceph_manager.ceph:PG 9.34 is not active+clean 2024-09-17T04:27:49.601 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.193736+0000', 'last_change': '2024-09-17T04:27:43.193736+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.193736+0000', 'last_undegraded': '2024-09-17T04:27:43.193736+0000', 'last_fullsized': '2024-09-17T04:27:43.193736+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T04:27:49.601 INFO:tasks.ceph.ceph_manager.ceph:PG 9.33 is not active+clean 2024-09-17T04:27:49.601 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.184126+0000', 'last_change': '2024-09-17T04:27:43.184126+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.184126+0000', 'last_undegraded': '2024-09-17T04:27:43.184126+0000', 'last_fullsized': '2024-09-17T04:27:43.184126+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T04:27:49.601 INFO:tasks.ceph.ceph_manager.ceph:PG 9.32 is not active+clean 2024-09-17T04:27:49.601 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.189690+0000', 'last_change': '2024-09-17T04:27:43.189690+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.189690+0000', 'last_undegraded': '2024-09-17T04:27:43.189690+0000', 'last_fullsized': '2024-09-17T04:27:43.189690+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T04:27:49.601 INFO:tasks.ceph.ceph_manager.ceph:PG 9.31 is not active+clean 2024-09-17T04:27:49.601 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.188426+0000', 'last_change': '2024-09-17T04:27:43.188426+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.188426+0000', 'last_undegraded': '2024-09-17T04:27:43.188426+0000', 'last_fullsized': '2024-09-17T04:27:43.188426+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T04:27:49.602 INFO:tasks.ceph.ceph_manager.ceph:PG 9.30 is not active+clean 2024-09-17T04:27:49.602 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T04:27:43.175780+0000', 'last_change': '2024-09-17T04:27:43.175780+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.175780+0000', 'last_undegraded': '2024-09-17T04:27:43.175780+0000', 'last_fullsized': '2024-09-17T04:27:43.175780+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T04:27:49.602 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2e is not active+clean 2024-09-17T04:27:49.602 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.185462+0000', 'last_change': '2024-09-17T04:27:43.185462+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.185462+0000', 'last_undegraded': '2024-09-17T04:27:43.185462+0000', 'last_fullsized': '2024-09-17T04:27:43.185462+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:27:49.602 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2d is not active+clean 2024-09-17T04:27:49.602 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T04:27:43.175780+0000', 'last_change': '2024-09-17T04:27:43.175780+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.175780+0000', 'last_undegraded': '2024-09-17T04:27:43.175780+0000', 'last_fullsized': '2024-09-17T04:27:43.175780+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T04:27:49.602 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2b is not active+clean 2024-09-17T04:27:49.602 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.188790+0000', 'last_change': '2024-09-17T04:27:43.188790+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.188790+0000', 'last_undegraded': '2024-09-17T04:27:43.188790+0000', 'last_fullsized': '2024-09-17T04:27:43.188790+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:27:49.602 INFO:tasks.ceph.ceph_manager.ceph:PG 9.29 is not active+clean 2024-09-17T04:27:49.602 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.185548+0000', 'last_change': '2024-09-17T04:27:43.185548+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.185548+0000', 'last_undegraded': '2024-09-17T04:27:43.185548+0000', 'last_fullsized': '2024-09-17T04:27:43.185548+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:27:49.602 INFO:tasks.ceph.ceph_manager.ceph:PG 9.28 is not active+clean 2024-09-17T04:27:49.603 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.185510+0000', 'last_change': '2024-09-17T04:27:43.185510+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.185510+0000', 'last_undegraded': '2024-09-17T04:27:43.185510+0000', 'last_fullsized': '2024-09-17T04:27:43.185510+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T04:27:49.603 INFO:tasks.ceph.ceph_manager.ceph:PG 9.27 is not active+clean 2024-09-17T04:27:49.603 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T04:27:43.175780+0000', 'last_change': '2024-09-17T04:27:43.175780+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.175780+0000', 'last_undegraded': '2024-09-17T04:27:43.175780+0000', 'last_fullsized': '2024-09-17T04:27:43.175780+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T04:27:49.603 INFO:tasks.ceph.ceph_manager.ceph:PG 9.26 is not active+clean 2024-09-17T04:27:49.603 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.192118+0000', 'last_change': '2024-09-17T04:27:43.192118+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.192118+0000', 'last_undegraded': '2024-09-17T04:27:43.192118+0000', 'last_fullsized': '2024-09-17T04:27:43.192118+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T04:27:49.603 INFO:tasks.ceph.ceph_manager.ceph:PG 9.25 is not active+clean 2024-09-17T04:27:49.603 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T04:27:43.175780+0000', 'last_change': '2024-09-17T04:27:43.175780+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.175780+0000', 'last_undegraded': '2024-09-17T04:27:43.175780+0000', 'last_fullsized': '2024-09-17T04:27:43.175780+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T04:27:49.603 INFO:tasks.ceph.ceph_manager.ceph:PG 9.24 is not active+clean 2024-09-17T04:27:49.604 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T04:27:43.175780+0000', 'last_change': '2024-09-17T04:27:43.175780+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.175780+0000', 'last_undegraded': '2024-09-17T04:27:43.175780+0000', 'last_fullsized': '2024-09-17T04:27:43.175780+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T04:27:49.604 INFO:tasks.ceph.ceph_manager.ceph:PG 9.22 is not active+clean 2024-09-17T04:27:49.604 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.183706+0000', 'last_change': '2024-09-17T04:27:43.183706+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.183706+0000', 'last_undegraded': '2024-09-17T04:27:43.183706+0000', 'last_fullsized': '2024-09-17T04:27:43.183706+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:27:49.604 INFO:tasks.ceph.ceph_manager.ceph:PG 9.21 is not active+clean 2024-09-17T04:27:49.605 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T04:27:43.175780+0000', 'last_change': '2024-09-17T04:27:43.175780+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.175780+0000', 'last_undegraded': '2024-09-17T04:27:43.175780+0000', 'last_fullsized': '2024-09-17T04:27:43.175780+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T04:27:49.605 INFO:tasks.ceph.ceph_manager.ceph:PG 9.20 is not active+clean 2024-09-17T04:27:49.605 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T04:27:43.175780+0000', 'last_change': '2024-09-17T04:27:43.175780+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.175780+0000', 'last_undegraded': '2024-09-17T04:27:43.175780+0000', 'last_fullsized': '2024-09-17T04:27:43.175780+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T04:27:49.605 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1e is not active+clean 2024-09-17T04:27:49.605 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T04:27:43.175780+0000', 'last_change': '2024-09-17T04:27:43.175780+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.175780+0000', 'last_undegraded': '2024-09-17T04:27:43.175780+0000', 'last_fullsized': '2024-09-17T04:27:43.175780+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T04:27:49.606 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1f is not active+clean 2024-09-17T04:27:49.606 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:42.186596+0000', 'last_change': '2024-09-17T04:27:42.186596+0000', 'last_active': '2024-09-17T04:27:42.169395+0000', 'last_peered': '2024-09-17T04:27:42.169395+0000', 'last_clean': '2024-09-17T04:27:42.169395+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:42.186596+0000', 'last_undegraded': '2024-09-17T04:27:42.186596+0000', 'last_fullsized': '2024-09-17T04:27:42.186596+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T04:27:49.606 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1f is not active+clean 2024-09-17T04:27:49.606 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T04:27:43.175780+0000', 'last_change': '2024-09-17T04:27:43.175780+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.175780+0000', 'last_undegraded': '2024-09-17T04:27:43.175780+0000', 'last_fullsized': '2024-09-17T04:27:43.175780+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T04:27:49.606 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1c is not active+clean 2024-09-17T04:27:49.607 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T04:27:43.175780+0000', 'last_change': '2024-09-17T04:27:43.175780+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.175780+0000', 'last_undegraded': '2024-09-17T04:27:43.175780+0000', 'last_fullsized': '2024-09-17T04:27:43.175780+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T04:27:49.607 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1d is not active+clean 2024-09-17T04:27:49.607 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:42.186307+0000', 'last_change': '2024-09-17T04:27:42.186307+0000', 'last_active': '2024-09-17T04:27:42.169395+0000', 'last_peered': '2024-09-17T04:27:42.169395+0000', 'last_clean': '2024-09-17T04:27:42.169395+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:42.186307+0000', 'last_undegraded': '2024-09-17T04:27:42.186307+0000', 'last_fullsized': '2024-09-17T04:27:42.186307+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T04:27:49.607 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1d is not active+clean 2024-09-17T04:27:49.607 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T04:27:43.175780+0000', 'last_change': '2024-09-17T04:27:43.175780+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.175780+0000', 'last_undegraded': '2024-09-17T04:27:43.175780+0000', 'last_fullsized': '2024-09-17T04:27:43.175780+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T04:27:49.608 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1a is not active+clean 2024-09-17T04:27:49.608 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:42.185403+0000', 'last_change': '2024-09-17T04:27:42.185403+0000', 'last_active': '2024-09-17T04:27:42.169395+0000', 'last_peered': '2024-09-17T04:27:42.169395+0000', 'last_clean': '2024-09-17T04:27:42.169395+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:42.185403+0000', 'last_undegraded': '2024-09-17T04:27:42.185403+0000', 'last_fullsized': '2024-09-17T04:27:42.185403+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T04:27:49.608 INFO:tasks.ceph.ceph_manager.ceph:PG 9.18 is not active+clean 2024-09-17T04:27:49.608 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.183685+0000', 'last_change': '2024-09-17T04:27:43.183685+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.183685+0000', 'last_undegraded': '2024-09-17T04:27:43.183685+0000', 'last_fullsized': '2024-09-17T04:27:43.183685+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T04:27:49.608 INFO:tasks.ceph.ceph_manager.ceph:PG 9.16 is not active+clean 2024-09-17T04:27:49.609 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.189782+0000', 'last_change': '2024-09-17T04:27:43.189782+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.189782+0000', 'last_undegraded': '2024-09-17T04:27:43.189782+0000', 'last_fullsized': '2024-09-17T04:27:43.189782+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T04:27:49.609 INFO:tasks.ceph.ceph_manager.ceph:PG 9.17 is not active+clean 2024-09-17T04:27:49.609 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.185434+0000', 'last_change': '2024-09-17T04:27:43.185434+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.185434+0000', 'last_undegraded': '2024-09-17T04:27:43.185434+0000', 'last_fullsized': '2024-09-17T04:27:43.185434+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:27:49.609 INFO:tasks.ceph.ceph_manager.ceph:PG 9.14 is not active+clean 2024-09-17T04:27:49.609 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T04:27:43.175780+0000', 'last_change': '2024-09-17T04:27:43.175780+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.175780+0000', 'last_undegraded': '2024-09-17T04:27:43.175780+0000', 'last_fullsized': '2024-09-17T04:27:43.175780+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T04:27:49.610 INFO:tasks.ceph.ceph_manager.ceph:PG 9.15 is not active+clean 2024-09-17T04:27:49.610 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.187796+0000', 'last_change': '2024-09-17T04:27:43.187796+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.187796+0000', 'last_undegraded': '2024-09-17T04:27:43.187796+0000', 'last_fullsized': '2024-09-17T04:27:43.187796+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:27:49.610 INFO:tasks.ceph.ceph_manager.ceph:PG 9.12 is not active+clean 2024-09-17T04:27:49.610 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.190626+0000', 'last_change': '2024-09-17T04:27:43.190626+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.190626+0000', 'last_undegraded': '2024-09-17T04:27:43.190626+0000', 'last_fullsized': '2024-09-17T04:27:43.190626+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T04:27:49.610 INFO:tasks.ceph.ceph_manager.ceph:PG 9.13 is not active+clean 2024-09-17T04:27:49.610 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.187412+0000', 'last_change': '2024-09-17T04:27:43.187412+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.187412+0000', 'last_undegraded': '2024-09-17T04:27:43.187412+0000', 'last_fullsized': '2024-09-17T04:27:43.187412+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:27:49.611 INFO:tasks.ceph.ceph_manager.ceph:PG 9.10 is not active+clean 2024-09-17T04:27:49.611 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.188547+0000', 'last_change': '2024-09-17T04:27:43.188547+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.188547+0000', 'last_undegraded': '2024-09-17T04:27:43.188547+0000', 'last_fullsized': '2024-09-17T04:27:43.188547+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T04:27:49.611 INFO:tasks.ceph.ceph_manager.ceph:PG 9.11 is not active+clean 2024-09-17T04:27:49.611 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.185001+0000', 'last_change': '2024-09-17T04:27:43.185001+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.185001+0000', 'last_undegraded': '2024-09-17T04:27:43.185001+0000', 'last_fullsized': '2024-09-17T04:27:43.185001+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:27:49.611 INFO:tasks.ceph.ceph_manager.ceph:PG 9.e is not active+clean 2024-09-17T04:27:49.611 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.184912+0000', 'last_change': '2024-09-17T04:27:43.184912+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.184912+0000', 'last_undegraded': '2024-09-17T04:27:43.184912+0000', 'last_fullsized': '2024-09-17T04:27:43.184912+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:27:49.612 INFO:tasks.ceph.ceph_manager.ceph:PG 9.f is not active+clean 2024-09-17T04:27:49.612 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.193299+0000', 'last_change': '2024-09-17T04:27:43.193299+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.193299+0000', 'last_undegraded': '2024-09-17T04:27:43.193299+0000', 'last_fullsized': '2024-09-17T04:27:43.193299+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T04:27:49.612 INFO:tasks.ceph.ceph_manager.ceph:PG 8.e is not active+clean 2024-09-17T04:27:49.612 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:42.186798+0000', 'last_change': '2024-09-17T04:27:42.186798+0000', 'last_active': '2024-09-17T04:27:42.169395+0000', 'last_peered': '2024-09-17T04:27:42.169395+0000', 'last_clean': '2024-09-17T04:27:42.169395+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:42.186798+0000', 'last_undegraded': '2024-09-17T04:27:42.186798+0000', 'last_fullsized': '2024-09-17T04:27:42.186798+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T04:27:49.612 INFO:tasks.ceph.ceph_manager.ceph:PG 9.d is not active+clean 2024-09-17T04:27:49.612 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.186711+0000', 'last_change': '2024-09-17T04:27:43.186711+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.186711+0000', 'last_undegraded': '2024-09-17T04:27:43.186711+0000', 'last_fullsized': '2024-09-17T04:27:43.186711+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:27:49.613 INFO:tasks.ceph.ceph_manager.ceph:PG 8.c is not active+clean 2024-09-17T04:27:49.613 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:42.185903+0000', 'last_change': '2024-09-17T04:27:42.185903+0000', 'last_active': '2024-09-17T04:27:42.169395+0000', 'last_peered': '2024-09-17T04:27:42.169395+0000', 'last_clean': '2024-09-17T04:27:42.169395+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:42.185903+0000', 'last_undegraded': '2024-09-17T04:27:42.185903+0000', 'last_fullsized': '2024-09-17T04:27:42.185903+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T04:27:49.613 INFO:tasks.ceph.ceph_manager.ceph:PG 9.a is not active+clean 2024-09-17T04:27:49.613 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.189283+0000', 'last_change': '2024-09-17T04:27:43.189283+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.189283+0000', 'last_undegraded': '2024-09-17T04:27:43.189283+0000', 'last_fullsized': '2024-09-17T04:27:43.189283+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T04:27:49.613 INFO:tasks.ceph.ceph_manager.ceph:PG 9.b is not active+clean 2024-09-17T04:27:49.613 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T04:27:43.175780+0000', 'last_change': '2024-09-17T04:27:43.175780+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.175780+0000', 'last_undegraded': '2024-09-17T04:27:43.175780+0000', 'last_fullsized': '2024-09-17T04:27:43.175780+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T04:27:49.614 INFO:tasks.ceph.ceph_manager.ceph:PG 8.a is not active+clean 2024-09-17T04:27:49.614 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:42.188161+0000', 'last_change': '2024-09-17T04:27:42.188161+0000', 'last_active': '2024-09-17T04:27:42.169395+0000', 'last_peered': '2024-09-17T04:27:42.169395+0000', 'last_clean': '2024-09-17T04:27:42.169395+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:42.188161+0000', 'last_undegraded': '2024-09-17T04:27:42.188161+0000', 'last_fullsized': '2024-09-17T04:27:42.188161+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T04:27:49.614 INFO:tasks.ceph.ceph_manager.ceph:PG 9.9 is not active+clean 2024-09-17T04:27:49.614 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T04:27:43.175780+0000', 'last_change': '2024-09-17T04:27:43.175780+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.175780+0000', 'last_undegraded': '2024-09-17T04:27:43.175780+0000', 'last_fullsized': '2024-09-17T04:27:43.175780+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T04:27:49.614 INFO:tasks.ceph.ceph_manager.ceph:PG 9.6 is not active+clean 2024-09-17T04:27:49.614 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.191601+0000', 'last_change': '2024-09-17T04:27:43.191601+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.191601+0000', 'last_undegraded': '2024-09-17T04:27:43.191601+0000', 'last_fullsized': '2024-09-17T04:27:43.191601+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T04:27:49.614 INFO:tasks.ceph.ceph_manager.ceph:PG 8.7 is not active+clean 2024-09-17T04:27:49.615 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:42.188855+0000', 'last_change': '2024-09-17T04:27:42.188855+0000', 'last_active': '2024-09-17T04:27:42.169395+0000', 'last_peered': '2024-09-17T04:27:42.169395+0000', 'last_clean': '2024-09-17T04:27:42.169395+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:42.188855+0000', 'last_undegraded': '2024-09-17T04:27:42.188855+0000', 'last_fullsized': '2024-09-17T04:27:42.188855+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T04:27:49.615 INFO:tasks.ceph.ceph_manager.ceph:PG 9.7 is not active+clean 2024-09-17T04:27:49.615 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.192785+0000', 'last_change': '2024-09-17T04:27:43.192785+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.192785+0000', 'last_undegraded': '2024-09-17T04:27:43.192785+0000', 'last_fullsized': '2024-09-17T04:27:43.192785+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T04:27:49.615 INFO:tasks.ceph.ceph_manager.ceph:PG 8.6 is not active+clean 2024-09-17T04:27:49.615 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:42.186212+0000', 'last_change': '2024-09-17T04:27:42.186212+0000', 'last_active': '2024-09-17T04:27:42.169395+0000', 'last_peered': '2024-09-17T04:27:42.169395+0000', 'last_clean': '2024-09-17T04:27:42.169395+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:42.186212+0000', 'last_undegraded': '2024-09-17T04:27:42.186212+0000', 'last_fullsized': '2024-09-17T04:27:42.186212+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T04:27:49.615 INFO:tasks.ceph.ceph_manager.ceph:PG 9.4 is not active+clean 2024-09-17T04:27:49.616 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T04:27:43.175780+0000', 'last_change': '2024-09-17T04:27:43.175780+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.175780+0000', 'last_undegraded': '2024-09-17T04:27:43.175780+0000', 'last_fullsized': '2024-09-17T04:27:43.175780+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T04:27:49.616 INFO:tasks.ceph.ceph_manager.ceph:PG 9.5 is not active+clean 2024-09-17T04:27:49.616 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T04:27:43.175780+0000', 'last_change': '2024-09-17T04:27:43.175780+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.175780+0000', 'last_undegraded': '2024-09-17T04:27:43.175780+0000', 'last_fullsized': '2024-09-17T04:27:43.175780+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T04:27:49.616 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3 is not active+clean 2024-09-17T04:27:49.616 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:42.188211+0000', 'last_change': '2024-09-17T04:27:42.188211+0000', 'last_active': '2024-09-17T04:27:42.169395+0000', 'last_peered': '2024-09-17T04:27:42.169395+0000', 'last_clean': '2024-09-17T04:27:42.169395+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:42.188211+0000', 'last_undegraded': '2024-09-17T04:27:42.188211+0000', 'last_fullsized': '2024-09-17T04:27:42.188211+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T04:27:49.617 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3 is not active+clean 2024-09-17T04:27:49.617 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:43.183762+0000', 'last_change': '2024-09-17T04:27:43.183762+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.183762+0000', 'last_undegraded': '2024-09-17T04:27:43.183762+0000', 'last_fullsized': '2024-09-17T04:27:43.183762+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:27:49.617 INFO:tasks.ceph.ceph_manager.ceph:PG 9.0 is not active+clean 2024-09-17T04:27:49.617 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T04:27:43.175780+0000', 'last_change': '2024-09-17T04:27:43.175780+0000', 'last_active': '2024-09-17T04:27:43.175780+0000', 'last_peered': '2024-09-17T04:27:43.175780+0000', 'last_clean': '2024-09-17T04:27:43.175780+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:43.175780+0000', 'last_undegraded': '2024-09-17T04:27:43.175780+0000', 'last_fullsized': '2024-09-17T04:27:43.175780+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:43.175780+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T04:27:49.617 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1 is not active+clean 2024-09-17T04:27:49.617 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:27:42.188178+0000', 'last_change': '2024-09-17T04:27:42.188178+0000', 'last_active': '2024-09-17T04:27:42.169395+0000', 'last_peered': '2024-09-17T04:27:42.169395+0000', 'last_clean': '2024-09-17T04:27:42.169395+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:27:42.188178+0000', 'last_undegraded': '2024-09-17T04:27:42.188178+0000', 'last_fullsized': '2024-09-17T04:27:42.188178+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'last_clean_scrub_stamp': '2024-09-17T04:27:42.169395+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T04:27:49.618 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T04:27:49.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.854+0000 7f3c47e00640 1 -- 172.21.15.100:0/2077186704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c480d1580 msgr2=0x7f3c480d1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:49.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.854+0000 7f3c47e00640 1 --2- 172.21.15.100:0/2077186704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c480d1580 0x7f3c480d1990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009be0 tx=0x7f3c4002a890 comp rx=0 tx=0).stop 2024-09-17T04:27:49.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.854+0000 7f3c47e00640 1 -- 172.21.15.100:0/2077186704 shutdown_connections 2024-09-17T04:27:49.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.854+0000 7f3c47e00640 1 --2- 172.21.15.100:0/2077186704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c480d1580 0x7f3c480d1990 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:49.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.854+0000 7f3c47e00640 1 -- 172.21.15.100:0/2077186704 >> 172.21.15.100:0/2077186704 conn(0x7f3c4808bfe0 msgr2=0x7f3c4809d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:49.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.854+0000 7f3c47e00640 1 -- 172.21.15.100:0/2077186704 shutdown_connections 2024-09-17T04:27:49.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.854+0000 7f3c47e00640 1 -- 172.21.15.100:0/2077186704 wait complete. 2024-09-17T04:27:49.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.857+0000 7f3c47e00640 1 Processor -- start 2024-09-17T04:27:49.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.857+0000 7f3c47e00640 1 -- start start 2024-09-17T04:27:49.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.858+0000 7f3c47e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c480b3160 0x7f3c480cd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:49.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.858+0000 7f3c47e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c480b9e00 0x7f3c48160650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:49.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.858+0000 7f3c47e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c480d1580 0x7f3c480b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:49.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.858+0000 7f3c47e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3c480d46c0 con 0x7f3c480b3160 2024-09-17T04:27:49.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.858+0000 7f3c47e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3c480d4540 con 0x7f3c480b9e00 2024-09-17T04:27:49.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.858+0000 7f3c47e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3c480d4840 con 0x7f3c480d1580 2024-09-17T04:27:49.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.858+0000 7f3c46a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c480b3160 0x7f3c480cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:49.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.859+0000 7f3c46a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c480b3160 0x7f3c480cd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56884/0 (socket says 172.21.15.100:56884) 2024-09-17T04:27:49.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.859+0000 7f3c46a00640 1 -- 172.21.15.100:0/432776332 learned_addr learned my addr 172.21.15.100:0/432776332 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:49.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.859+0000 7f3c46000640 1 --2- 172.21.15.100:0/432776332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c480b9e00 0x7f3c48160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:49.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.859+0000 7f3c47400640 1 --2- 172.21.15.100:0/432776332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c480d1580 0x7f3c480b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:49.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.859+0000 7f3c46a00640 1 -- 172.21.15.100:0/432776332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c480d1580 msgr2=0x7f3c480b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:49.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.859+0000 7f3c46a00640 1 --2- 172.21.15.100:0/432776332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c480d1580 0x7f3c480b5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:49.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.859+0000 7f3c46a00640 1 -- 172.21.15.100:0/432776332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c480b9e00 msgr2=0x7f3c48160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:49.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.859+0000 7f3c46a00640 1 --2- 172.21.15.100:0/432776332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c480b9e00 0x7f3c48160650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:49.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.859+0000 7f3c46a00640 1 -- 172.21.15.100:0/432776332 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c48160b90 con 0x7f3c480b3160 2024-09-17T04:27:49.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.859+0000 7f3c46a00640 1 --2- 172.21.15.100:0/432776332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c480b3160 0x7f3c480cd790 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800c3f0 tx=0x7f3c3800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:49.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.860+0000 7f3c3ea00640 1 -- 172.21.15.100:0/432776332 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c38017070 con 0x7f3c480b3160 2024-09-17T04:27:49.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.860+0000 7f3c3ea00640 1 -- 172.21.15.100:0/432776332 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c380035f0 con 0x7f3c480b3160 2024-09-17T04:27:49.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.860+0000 7f3c3ea00640 1 -- 172.21.15.100:0/432776332 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c38003a40 con 0x7f3c480b3160 2024-09-17T04:27:49.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.860+0000 7f3c47e00640 1 -- 172.21.15.100:0/432776332 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c481f5640 con 0x7f3c480b3160 2024-09-17T04:27:49.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.860+0000 7f3c47e00640 1 -- 172.21.15.100:0/432776332 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c481f5c00 con 0x7f3c480b3160 2024-09-17T04:27:49.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.861+0000 7f3c47e00640 1 -- 172.21.15.100:0/432776332 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c480d1990 con 0x7f3c480b3160 2024-09-17T04:27:49.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.862+0000 7f3c3ea00640 1 -- 172.21.15.100:0/432776332 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f3c3801a070 con 0x7f3c480b3160 2024-09-17T04:27:49.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.863+0000 7f3c3ea00640 1 --2- 172.21.15.100:0/432776332 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3c28078d40 0x7f3c2807b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:49.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.863+0000 7f3c3ea00640 1 -- 172.21.15.100:0/432776332 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7f3c3809a3c0 con 0x7f3c480b3160 2024-09-17T04:27:49.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.864+0000 7f3c46000640 1 --2- 172.21.15.100:0/432776332 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3c28078d40 0x7f3c2807b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:49.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.864+0000 7f3c46000640 1 --2- 172.21.15.100:0/432776332 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3c28078d40 0x7f3c2807b250 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3c34007fd0 tx=0x7f3c34001390 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:49.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:49.868+0000 7f3c3ea00640 1 -- 172.21.15.100:0/432776332 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3c38061e40 con 0x7f3c480b3160 2024-09-17T04:27:50.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.122+0000 7f3c47e00640 1 -- 172.21.15.100:0/432776332 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f3c480b4620 con 0x7f3c480b3160 2024-09-17T04:27:50.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.122+0000 7f3c3ea00640 1 -- 172.21.15.100:0/432776332 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1216 (secure 0 0 0) 0x7f3c38066870 con 0x7f3c480b3160 2024-09-17T04:27:50.122 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:27:50.122 INFO:teuthology.orchestra.run.smithi100.stdout:{"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":243,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":8,"num_up_osds":8,"osd_up_since":1726547043,"num_in_osds":8,"osd_in_since":1726547032,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":39},{"state_name":"active+clean","count":36},{"state_name":"unknown","count":22}],"num_pgs":97,"num_pools":3,"num_objects":6,"data_bytes":590424,"bytes_used":240975872,"bytes_avail":772853137408,"bytes_total":773094113280,"unknown_pgs_ratio":0.22680412232875824,"inactive_pgs_ratio":0.40206184983253479,"write_bytes_sec":1401,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":36,"btime":"2024-09-17T04:27:47:219814+0000","id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"a","status":"up:active","gid":5739}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T04:26:52.007752+0000","services":{}},"progress_events":{}} 2024-09-17T04:27:50.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.125+0000 7f3c47e00640 1 -- 172.21.15.100:0/432776332 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3c28078d40 msgr2=0x7f3c2807b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:50.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.125+0000 7f3c47e00640 1 --2- 172.21.15.100:0/432776332 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3c28078d40 0x7f3c2807b250 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3c34007fd0 tx=0x7f3c34001390 comp rx=0 tx=0).stop 2024-09-17T04:27:50.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.125+0000 7f3c47e00640 1 -- 172.21.15.100:0/432776332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c480b3160 msgr2=0x7f3c480cd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:50.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.125+0000 7f3c47e00640 1 --2- 172.21.15.100:0/432776332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c480b3160 0x7f3c480cd790 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800c3f0 tx=0x7f3c3800c8b0 comp rx=0 tx=0).stop 2024-09-17T04:27:50.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.125+0000 7f3c47e00640 1 -- 172.21.15.100:0/432776332 shutdown_connections 2024-09-17T04:27:50.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.125+0000 7f3c47e00640 1 --2- 172.21.15.100:0/432776332 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3c28078d40 0x7f3c2807b250 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:50.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.125+0000 7f3c47e00640 1 --2- 172.21.15.100:0/432776332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c480d1580 0x7f3c480b5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:50.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.125+0000 7f3c47e00640 1 --2- 172.21.15.100:0/432776332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c480b9e00 0x7f3c48160650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:50.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.125+0000 7f3c47e00640 1 --2- 172.21.15.100:0/432776332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c480b3160 0x7f3c480cd790 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:50.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.125+0000 7f3c47e00640 1 -- 172.21.15.100:0/432776332 >> 172.21.15.100:0/432776332 conn(0x7f3c4808bfe0 msgr2=0x7f3c4806f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:50.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.125+0000 7f3c47e00640 1 -- 172.21.15.100:0/432776332 shutdown_connections 2024-09-17T04:27:50.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.125+0000 7f3c47e00640 1 -- 172.21.15.100:0/432776332 wait complete. 2024-09-17T04:27:50.150 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T04:27:50.150 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:27:50.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.377+0000 7fa535a00640 1 -- 172.21.15.100:0/1249027520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5140052b0 msgr2=0x7fa51400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:50.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.377+0000 7fa535a00640 1 --2- 172.21.15.100:0/1249027520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5140052b0 0x7fa51400b9c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa52800c710 tx=0x7fa514005830 comp rx=0 tx=0).stop 2024-09-17T04:27:50.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.377+0000 7fa52f400640 1 -- 172.21.15.100:0/1249027520 reap_dead start 2024-09-17T04:27:50.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.377+0000 7fa535a00640 1 -- 172.21.15.100:0/1249027520 shutdown_connections 2024-09-17T04:27:50.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.377+0000 7fa535a00640 1 -- 172.21.15.100:0/1249027520 >> 172.21.15.100:0/1249027520 conn(0x7fa53006d730 msgr2=0x7fa5300b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:50.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.378+0000 7fa535a00640 1 -- 172.21.15.100:0/1249027520 shutdown_connections 2024-09-17T04:27:50.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.378+0000 7fa535a00640 1 -- 172.21.15.100:0/1249027520 wait complete. 2024-09-17T04:27:50.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.380+0000 7fa535a00640 1 Processor -- start 2024-09-17T04:27:50.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.380+0000 7fa535a00640 1 -- start start 2024-09-17T04:27:50.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.381+0000 7fa535a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5140052b0 0x7fa530149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:50.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.381+0000 7fa535a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa530151a60 0x7fa5301f8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:50.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.381+0000 7fa535a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5301f91d0 0x7fa5301f95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:50.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.381+0000 7fa535a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa5300b4da0 con 0x7fa530151a60 2024-09-17T04:27:50.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.381+0000 7fa535a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa5300b4c20 con 0x7fa5140052b0 2024-09-17T04:27:50.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.381+0000 7fa535a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa5300ad4f0 con 0x7fa5301f91d0 2024-09-17T04:27:50.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.382+0000 7fa526000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa530151a60 0x7fa5301f8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:50.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.382+0000 7fa526000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa530151a60 0x7fa5301f8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56912/0 (socket says 172.21.15.100:56912) 2024-09-17T04:27:50.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.382+0000 7fa526000640 1 -- 172.21.15.100:0/2365643931 learned_addr learned my addr 172.21.15.100:0/2365643931 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:50.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.382+0000 7fa52f400640 1 --2- 172.21.15.100:0/2365643931 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5301f91d0 0x7fa5301f95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:50.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.382+0000 7fa52ea00640 1 --2- 172.21.15.100:0/2365643931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5140052b0 0x7fa530149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:50.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.382+0000 7fa526000640 1 -- 172.21.15.100:0/2365643931 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5301f91d0 msgr2=0x7fa5301f95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:50.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.382+0000 7fa526000640 1 --2- 172.21.15.100:0/2365643931 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5301f91d0 0x7fa5301f95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:50.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.382+0000 7fa526000640 1 -- 172.21.15.100:0/2365643931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5140052b0 msgr2=0x7fa530149920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:50.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.382+0000 7fa526000640 1 --2- 172.21.15.100:0/2365643931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5140052b0 0x7fa530149920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:50.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.382+0000 7fa526000640 1 -- 172.21.15.100:0/2365643931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53009b5f0 con 0x7fa530151a60 2024-09-17T04:27:50.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.382+0000 7fa52ea00640 1 --2- 172.21.15.100:0/2365643931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5140052b0 0x7fa530149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:27:50.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.382+0000 7fa526000640 1 --2- 172.21.15.100:0/2365643931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa530151a60 0x7fa5301f8c90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa5180119f0 tx=0x7fa518011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:50.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.383+0000 7fa527400640 1 -- 172.21.15.100:0/2365643931 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51800fcd0 con 0x7fa530151a60 2024-09-17T04:27:50.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.383+0000 7fa527400640 1 -- 172.21.15.100:0/2365643931 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa518004030 con 0x7fa530151a60 2024-09-17T04:27:50.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.383+0000 7fa527400640 1 -- 172.21.15.100:0/2365643931 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518004460 con 0x7fa530151a60 2024-09-17T04:27:50.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.383+0000 7fa535a00640 1 -- 172.21.15.100:0/2365643931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa53009ddd0 con 0x7fa530151a60 2024-09-17T04:27:50.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.383+0000 7fa535a00640 1 -- 172.21.15.100:0/2365643931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa53009c060 con 0x7fa530151a60 2024-09-17T04:27:50.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.384+0000 7fa535a00640 1 -- 172.21.15.100:0/2365643931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5301535c0 con 0x7fa530151a60 2024-09-17T04:27:50.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.385+0000 7fa527400640 1 -- 172.21.15.100:0/2365643931 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7fa518005020 con 0x7fa530151a60 2024-09-17T04:27:50.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.386+0000 7fa527400640 1 --2- 172.21.15.100:0/2365643931 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa50c078d40 0x7fa50c07b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:50.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.386+0000 7fa527400640 1 -- 172.21.15.100:0/2365643931 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7fa51809bf40 con 0x7fa530151a60 2024-09-17T04:27:50.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.386+0000 7fa52ea00640 1 --2- 172.21.15.100:0/2365643931 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa50c078d40 0x7fa50c07b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:50.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.387+0000 7fa52ea00640 1 --2- 172.21.15.100:0/2365643931 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa50c078d40 0x7fa50c07b250 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa520000c00 tx=0x7fa520006b10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:50.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.390+0000 7fa527400640 1 -- 172.21.15.100:0/2365643931 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa518063f50 con 0x7fa530151a60 2024-09-17T04:27:50.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.570+0000 7fa535a00640 1 -- 172.21.15.100:0/2365643931 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa53006f5c0 con 0x7fa50c078d40 2024-09-17T04:27:50.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.575+0000 7fa527400640 1 -- 172.21.15.100:0/2365643931 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251944 (secure 0 0 0) 0x7fa53006f5c0 con 0x7fa50c078d40 2024-09-17T04:27:50.578 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:27:50.579 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:27:50.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.579+0000 7fa535a00640 1 -- 172.21.15.100:0/2365643931 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa50c078d40 msgr2=0x7fa50c07b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:50.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.579+0000 7fa535a00640 1 --2- 172.21.15.100:0/2365643931 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa50c078d40 0x7fa50c07b250 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa520000c00 tx=0x7fa520006b10 comp rx=0 tx=0).stop 2024-09-17T04:27:50.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.579+0000 7fa535a00640 1 -- 172.21.15.100:0/2365643931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa530151a60 msgr2=0x7fa5301f8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:50.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.579+0000 7fa535a00640 1 --2- 172.21.15.100:0/2365643931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa530151a60 0x7fa5301f8c90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa5180119f0 tx=0x7fa518011eb0 comp rx=0 tx=0).stop 2024-09-17T04:27:50.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.580+0000 7fa535a00640 1 -- 172.21.15.100:0/2365643931 shutdown_connections 2024-09-17T04:27:50.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.580+0000 7fa535a00640 1 --2- 172.21.15.100:0/2365643931 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa50c078d40 0x7fa50c07b250 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:50.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.580+0000 7fa535a00640 1 --2- 172.21.15.100:0/2365643931 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5301f91d0 0x7fa5301f95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:50.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.580+0000 7fa535a00640 1 --2- 172.21.15.100:0/2365643931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa530151a60 0x7fa5301f8c90 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:50.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.580+0000 7fa535a00640 1 --2- 172.21.15.100:0/2365643931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5140052b0 0x7fa530149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:50.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.580+0000 7fa535a00640 1 -- 172.21.15.100:0/2365643931 >> 172.21.15.100:0/2365643931 conn(0x7fa53006d730 msgr2=0x7fa53015dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:50.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.580+0000 7fa535a00640 1 -- 172.21.15.100:0/2365643931 shutdown_connections 2024-09-17T04:27:50.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:50.580+0000 7fa535a00640 1 -- 172.21.15.100:0/2365643931 wait complete. 2024-09-17T04:27:50.601 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-09-17T04:27:50.024762+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239748,"kb_used_data":6856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754734972,"statfs":{"total":773094113280,"available":772848611328,"internally_reserved":0,"allocated":7020544,"data_stored":3580484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":238276148},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.843725"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483699+0000","last_change":"2024-09-17T04:27:44.224649+0000","last_active":"2024-09-17T04:27:45.483699+0000","last_peered":"2024-09-17T04:27:45.483699+0000","last_clean":"2024-09-17T04:27:45.483699+0000","last_became_active":"2024-09-17T04:27:44.224334+0000","last_became_peered":"2024-09-17T04:27:44.224334+0000","last_unstale":"2024-09-17T04:27:45.483699+0000","last_undegraded":"2024-09-17T04:27:45.483699+0000","last_fullsized":"2024-09-17T04:27:45.483699+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:17.020497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212739+0000","last_change":"2024-09-17T04:27:44.238721+0000","last_active":"2024-09-17T04:27:45.212739+0000","last_peered":"2024-09-17T04:27:45.212739+0000","last_clean":"2024-09-17T04:27:45.212739+0000","last_became_active":"2024-09-17T04:27:44.237900+0000","last_became_peered":"2024-09-17T04:27:44.237900+0000","last_unstale":"2024-09-17T04:27:45.212739+0000","last_undegraded":"2024-09-17T04:27:45.212739+0000","last_fullsized":"2024-09-17T04:27:45.212739+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:09:57.201730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483850+0000","last_change":"2024-09-17T04:27:44.220473+0000","last_active":"2024-09-17T04:27:45.483850+0000","last_peered":"2024-09-17T04:27:45.483850+0000","last_clean":"2024-09-17T04:27:45.483850+0000","last_became_active":"2024-09-17T04:27:44.220013+0000","last_became_peered":"2024-09-17T04:27:44.220013+0000","last_unstale":"2024-09-17T04:27:45.483850+0000","last_undegraded":"2024-09-17T04:27:45.483850+0000","last_fullsized":"2024-09-17T04:27:45.483850+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:22.439743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.216564+0000","last_change":"2024-09-17T04:27:44.212039+0000","last_active":"2024-09-17T04:27:45.216564+0000","last_peered":"2024-09-17T04:27:45.216564+0000","last_clean":"2024-09-17T04:27:45.216564+0000","last_became_active":"2024-09-17T04:27:44.211689+0000","last_became_peered":"2024-09-17T04:27:44.211689+0000","last_unstale":"2024-09-17T04:27:45.216564+0000","last_undegraded":"2024-09-17T04:27:45.216564+0000","last_fullsized":"2024-09-17T04:27:45.216564+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:12:13.395203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483076+0000","last_change":"2024-09-17T04:27:44.203901+0000","last_active":"2024-09-17T04:27:45.483076+0000","last_peered":"2024-09-17T04:27:45.483076+0000","last_clean":"2024-09-17T04:27:45.483076+0000","last_became_active":"2024-09-17T04:27:44.203509+0000","last_became_peered":"2024-09-17T04:27:44.203509+0000","last_unstale":"2024-09-17T04:27:45.483076+0000","last_undegraded":"2024-09-17T04:27:45.483076+0000","last_fullsized":"2024-09-17T04:27:45.483076+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:55.077812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222259+0000","last_change":"2024-09-17T04:27:44.215787+0000","last_active":"2024-09-17T04:27:45.222259+0000","last_peered":"2024-09-17T04:27:45.222259+0000","last_clean":"2024-09-17T04:27:45.222259+0000","last_became_active":"2024-09-17T04:27:44.215468+0000","last_became_peered":"2024-09-17T04:27:44.215468+0000","last_unstale":"2024-09-17T04:27:45.222259+0000","last_undegraded":"2024-09-17T04:27:45.222259+0000","last_fullsized":"2024-09-17T04:27:45.222259+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:35.616439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.215959+0000","last_change":"2024-09-17T04:27:44.218294+0000","last_active":"2024-09-17T04:27:45.215959+0000","last_peered":"2024-09-17T04:27:45.215959+0000","last_clean":"2024-09-17T04:27:45.215959+0000","last_became_active":"2024-09-17T04:27:44.214481+0000","last_became_peered":"2024-09-17T04:27:44.214481+0000","last_unstale":"2024-09-17T04:27:45.215959+0000","last_undegraded":"2024-09-17T04:27:45.215959+0000","last_fullsized":"2024-09-17T04:27:45.215959+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:20.764786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212387+0000","last_change":"2024-09-17T04:27:44.222903+0000","last_active":"2024-09-17T04:27:45.212387+0000","last_peered":"2024-09-17T04:27:45.212387+0000","last_clean":"2024-09-17T04:27:45.212387+0000","last_became_active":"2024-09-17T04:27:44.222587+0000","last_became_peered":"2024-09-17T04:27:44.222587+0000","last_unstale":"2024-09-17T04:27:45.212387+0000","last_undegraded":"2024-09-17T04:27:45.212387+0000","last_fullsized":"2024-09-17T04:27:45.212387+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:37:39.457226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.216488+0000","last_change":"2024-09-17T04:27:44.218087+0000","last_active":"2024-09-17T04:27:45.216488+0000","last_peered":"2024-09-17T04:27:45.216488+0000","last_clean":"2024-09-17T04:27:45.216488+0000","last_became_active":"2024-09-17T04:27:44.216344+0000","last_became_peered":"2024-09-17T04:27:44.216344+0000","last_unstale":"2024-09-17T04:27:45.216488+0000","last_undegraded":"2024-09-17T04:27:45.216488+0000","last_fullsized":"2024-09-17T04:27:45.216488+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:48.140070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.484260+0000","last_change":"2024-09-17T04:27:44.222987+0000","last_active":"2024-09-17T04:27:45.484260+0000","last_peered":"2024-09-17T04:27:45.484260+0000","last_clean":"2024-09-17T04:27:45.484260+0000","last_became_active":"2024-09-17T04:27:44.222642+0000","last_became_peered":"2024-09-17T04:27:44.222642+0000","last_unstale":"2024-09-17T04:27:45.484260+0000","last_undegraded":"2024-09-17T04:27:45.484260+0000","last_fullsized":"2024-09-17T04:27:45.484260+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:26.116120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.484318+0000","last_change":"2024-09-17T04:27:44.234721+0000","last_active":"2024-09-17T04:27:45.484318+0000","last_peered":"2024-09-17T04:27:45.484318+0000","last_clean":"2024-09-17T04:27:45.484318+0000","last_became_active":"2024-09-17T04:27:44.233924+0000","last_became_peered":"2024-09-17T04:27:44.233924+0000","last_unstale":"2024-09-17T04:27:45.484318+0000","last_undegraded":"2024-09-17T04:27:45.484318+0000","last_fullsized":"2024-09-17T04:27:45.484318+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:44.967738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.216866+0000","last_change":"2024-09-17T04:27:44.215001+0000","last_active":"2024-09-17T04:27:45.216866+0000","last_peered":"2024-09-17T04:27:45.216866+0000","last_clean":"2024-09-17T04:27:45.216866+0000","last_became_active":"2024-09-17T04:27:44.214615+0000","last_became_peered":"2024-09-17T04:27:44.214615+0000","last_unstale":"2024-09-17T04:27:45.216866+0000","last_undegraded":"2024-09-17T04:27:45.216866+0000","last_fullsized":"2024-09-17T04:27:45.216866+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:15.359266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.225894+0000","last_change":"2024-09-17T04:27:44.227474+0000","last_active":"2024-09-17T04:27:45.225894+0000","last_peered":"2024-09-17T04:27:45.225894+0000","last_clean":"2024-09-17T04:27:45.225894+0000","last_became_active":"2024-09-17T04:27:44.227111+0000","last_became_peered":"2024-09-17T04:27:44.227111+0000","last_unstale":"2024-09-17T04:27:45.225894+0000","last_undegraded":"2024-09-17T04:27:45.225894+0000","last_fullsized":"2024-09-17T04:27:45.225894+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:58:30.716243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.216878+0000","last_change":"2024-09-17T04:27:44.212906+0000","last_active":"2024-09-17T04:27:45.216878+0000","last_peered":"2024-09-17T04:27:45.216878+0000","last_clean":"2024-09-17T04:27:45.216878+0000","last_became_active":"2024-09-17T04:27:44.212538+0000","last_became_peered":"2024-09-17T04:27:44.212538+0000","last_unstale":"2024-09-17T04:27:45.216878+0000","last_undegraded":"2024-09-17T04:27:45.216878+0000","last_fullsized":"2024-09-17T04:27:45.216878+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:08.736158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.216321+0000","last_change":"2024-09-17T04:27:44.234972+0000","last_active":"2024-09-17T04:27:45.216321+0000","last_peered":"2024-09-17T04:27:45.216321+0000","last_clean":"2024-09-17T04:27:45.216321+0000","last_became_active":"2024-09-17T04:27:44.234201+0000","last_became_peered":"2024-09-17T04:27:44.234201+0000","last_unstale":"2024-09-17T04:27:45.216321+0000","last_undegraded":"2024-09-17T04:27:45.216321+0000","last_fullsized":"2024-09-17T04:27:45.216321+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:54:00.667260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222741+0000","last_change":"2024-09-17T04:27:44.224641+0000","last_active":"2024-09-17T04:27:45.222741+0000","last_peered":"2024-09-17T04:27:45.222741+0000","last_clean":"2024-09-17T04:27:45.222741+0000","last_became_active":"2024-09-17T04:27:44.224324+0000","last_became_peered":"2024-09-17T04:27:44.224324+0000","last_unstale":"2024-09-17T04:27:45.222741+0000","last_undegraded":"2024-09-17T04:27:45.222741+0000","last_fullsized":"2024-09-17T04:27:45.222741+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:49.705694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212938+0000","last_change":"2024-09-17T04:27:44.221917+0000","last_active":"2024-09-17T04:27:45.212938+0000","last_peered":"2024-09-17T04:27:45.212938+0000","last_clean":"2024-09-17T04:27:45.212938+0000","last_became_active":"2024-09-17T04:27:44.221632+0000","last_became_peered":"2024-09-17T04:27:44.221632+0000","last_unstale":"2024-09-17T04:27:45.212938+0000","last_undegraded":"2024-09-17T04:27:45.212938+0000","last_fullsized":"2024-09-17T04:27:45.212938+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:27.954596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483971+0000","last_change":"2024-09-17T04:27:44.222680+0000","last_active":"2024-09-17T04:27:45.483971+0000","last_peered":"2024-09-17T04:27:45.483971+0000","last_clean":"2024-09-17T04:27:45.483971+0000","last_became_active":"2024-09-17T04:27:44.222082+0000","last_became_peered":"2024-09-17T04:27:44.222082+0000","last_unstale":"2024-09-17T04:27:45.483971+0000","last_undegraded":"2024-09-17T04:27:45.483971+0000","last_fullsized":"2024-09-17T04:27:45.483971+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:20:40.092936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483521+0000","last_change":"2024-09-17T04:27:44.223173+0000","last_active":"2024-09-17T04:27:45.483521+0000","last_peered":"2024-09-17T04:27:45.483521+0000","last_clean":"2024-09-17T04:27:45.483521+0000","last_became_active":"2024-09-17T04:27:44.222349+0000","last_became_peered":"2024-09-17T04:27:44.222349+0000","last_unstale":"2024-09-17T04:27:45.483521+0000","last_undegraded":"2024-09-17T04:27:45.483521+0000","last_fullsized":"2024-09-17T04:27:45.483521+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:45:47.530471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212533+0000","last_change":"2024-09-17T04:27:44.220909+0000","last_active":"2024-09-17T04:27:45.212533+0000","last_peered":"2024-09-17T04:27:45.212533+0000","last_clean":"2024-09-17T04:27:45.212533+0000","last_became_active":"2024-09-17T04:27:44.220605+0000","last_became_peered":"2024-09-17T04:27:44.220605+0000","last_unstale":"2024-09-17T04:27:45.212533+0000","last_undegraded":"2024-09-17T04:27:45.212533+0000","last_fullsized":"2024-09-17T04:27:45.212533+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:48:13.238235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483544+0000","last_change":"2024-09-17T04:27:44.217774+0000","last_active":"2024-09-17T04:27:45.483544+0000","last_peered":"2024-09-17T04:27:45.483544+0000","last_clean":"2024-09-17T04:27:45.483544+0000","last_became_active":"2024-09-17T04:27:44.217219+0000","last_became_peered":"2024-09-17T04:27:44.217219+0000","last_unstale":"2024-09-17T04:27:45.483544+0000","last_undegraded":"2024-09-17T04:27:45.483544+0000","last_fullsized":"2024-09-17T04:27:45.483544+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:15.359266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212746+0000","last_change":"2024-09-17T04:27:44.216830+0000","last_active":"2024-09-17T04:27:45.212746+0000","last_peered":"2024-09-17T04:27:45.212746+0000","last_clean":"2024-09-17T04:27:45.212746+0000","last_became_active":"2024-09-17T04:27:44.216520+0000","last_became_peered":"2024-09-17T04:27:44.216520+0000","last_unstale":"2024-09-17T04:27:45.212746+0000","last_undegraded":"2024-09-17T04:27:45.212746+0000","last_fullsized":"2024-09-17T04:27:45.212746+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:15:47.203299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.484100+0000","last_change":"2024-09-17T04:27:44.228366+0000","last_active":"2024-09-17T04:27:45.484100+0000","last_peered":"2024-09-17T04:27:45.484100+0000","last_clean":"2024-09-17T04:27:45.484100+0000","last_became_active":"2024-09-17T04:27:44.227967+0000","last_became_peered":"2024-09-17T04:27:44.227967+0000","last_unstale":"2024-09-17T04:27:45.484100+0000","last_undegraded":"2024-09-17T04:27:45.484100+0000","last_fullsized":"2024-09-17T04:27:45.484100+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:48.593633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.225723+0000","last_change":"2024-09-17T04:27:44.223208+0000","last_active":"2024-09-17T04:27:45.225723+0000","last_peered":"2024-09-17T04:27:45.225723+0000","last_clean":"2024-09-17T04:27:45.225723+0000","last_became_active":"2024-09-17T04:27:44.222867+0000","last_became_peered":"2024-09-17T04:27:44.222867+0000","last_unstale":"2024-09-17T04:27:45.225723+0000","last_undegraded":"2024-09-17T04:27:45.225723+0000","last_fullsized":"2024-09-17T04:27:45.225723+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:27.882897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483901+0000","last_change":"2024-09-17T04:27:44.218162+0000","last_active":"2024-09-17T04:27:45.483901+0000","last_peered":"2024-09-17T04:27:45.483901+0000","last_clean":"2024-09-17T04:27:45.483901+0000","last_became_active":"2024-09-17T04:27:44.217831+0000","last_became_peered":"2024-09-17T04:27:44.217831+0000","last_unstale":"2024-09-17T04:27:45.483901+0000","last_undegraded":"2024-09-17T04:27:45.483901+0000","last_fullsized":"2024-09-17T04:27:45.483901+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:17.403599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.217861+0000","last_change":"2024-09-17T04:27:44.206558+0000","last_active":"2024-09-17T04:27:45.217861+0000","last_peered":"2024-09-17T04:27:45.217861+0000","last_clean":"2024-09-17T04:27:45.217861+0000","last_became_active":"2024-09-17T04:27:44.206253+0000","last_became_peered":"2024-09-17T04:27:44.206253+0000","last_unstale":"2024-09-17T04:27:45.217861+0000","last_undegraded":"2024-09-17T04:27:45.217861+0000","last_fullsized":"2024-09-17T04:27:45.217861+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:14.532539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222481+0000","last_change":"2024-09-17T04:27:44.221446+0000","last_active":"2024-09-17T04:27:45.222481+0000","last_peered":"2024-09-17T04:27:45.222481+0000","last_clean":"2024-09-17T04:27:45.222481+0000","last_became_active":"2024-09-17T04:27:44.221111+0000","last_became_peered":"2024-09-17T04:27:44.221111+0000","last_unstale":"2024-09-17T04:27:45.222481+0000","last_undegraded":"2024-09-17T04:27:45.222481+0000","last_fullsized":"2024-09-17T04:27:45.222481+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:10.625074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483709+0000","last_change":"2024-09-17T04:27:44.211926+0000","last_active":"2024-09-17T04:27:45.483709+0000","last_peered":"2024-09-17T04:27:45.483709+0000","last_clean":"2024-09-17T04:27:45.483709+0000","last_became_active":"2024-09-17T04:27:44.211608+0000","last_became_peered":"2024-09-17T04:27:44.211608+0000","last_unstale":"2024-09-17T04:27:45.483709+0000","last_undegraded":"2024-09-17T04:27:45.483709+0000","last_fullsized":"2024-09-17T04:27:45.483709+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:18:10.368163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.213050+0000","last_change":"2024-09-17T04:27:44.218277+0000","last_active":"2024-09-17T04:27:45.213050+0000","last_peered":"2024-09-17T04:27:45.213050+0000","last_clean":"2024-09-17T04:27:45.213050+0000","last_became_active":"2024-09-17T04:27:44.217942+0000","last_became_peered":"2024-09-17T04:27:44.217942+0000","last_unstale":"2024-09-17T04:27:45.213050+0000","last_undegraded":"2024-09-17T04:27:45.213050+0000","last_fullsized":"2024-09-17T04:27:45.213050+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:18.353865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483703+0000","last_change":"2024-09-17T04:27:44.220878+0000","last_active":"2024-09-17T04:27:45.483703+0000","last_peered":"2024-09-17T04:27:45.483703+0000","last_clean":"2024-09-17T04:27:45.483703+0000","last_became_active":"2024-09-17T04:27:44.220534+0000","last_became_peered":"2024-09-17T04:27:44.220534+0000","last_unstale":"2024-09-17T04:27:45.483703+0000","last_undegraded":"2024-09-17T04:27:45.483703+0000","last_fullsized":"2024-09-17T04:27:45.483703+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:49.106024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.482885+0000","last_change":"2024-09-17T04:27:44.223625+0000","last_active":"2024-09-17T04:27:45.482885+0000","last_peered":"2024-09-17T04:27:45.482885+0000","last_clean":"2024-09-17T04:27:45.482885+0000","last_became_active":"2024-09-17T04:27:44.220587+0000","last_became_peered":"2024-09-17T04:27:44.220587+0000","last_unstale":"2024-09-17T04:27:45.482885+0000","last_undegraded":"2024-09-17T04:27:45.482885+0000","last_fullsized":"2024-09-17T04:27:45.482885+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:44:05.407748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222393+0000","last_change":"2024-09-17T04:27:44.228534+0000","last_active":"2024-09-17T04:27:45.222393+0000","last_peered":"2024-09-17T04:27:45.222393+0000","last_clean":"2024-09-17T04:27:45.222393+0000","last_became_active":"2024-09-17T04:27:44.227779+0000","last_became_peered":"2024-09-17T04:27:44.227779+0000","last_unstale":"2024-09-17T04:27:45.222393+0000","last_undegraded":"2024-09-17T04:27:45.222393+0000","last_fullsized":"2024-09-17T04:27:45.222393+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:45:22.653237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222475+0000","last_change":"2024-09-17T04:27:44.215232+0000","last_active":"2024-09-17T04:27:45.222475+0000","last_peered":"2024-09-17T04:27:45.222475+0000","last_clean":"2024-09-17T04:27:45.222475+0000","last_became_active":"2024-09-17T04:27:44.214803+0000","last_became_peered":"2024-09-17T04:27:44.214803+0000","last_unstale":"2024-09-17T04:27:45.222475+0000","last_undegraded":"2024-09-17T04:27:45.222475+0000","last_fullsized":"2024-09-17T04:27:45.222475+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:48.593633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"60'4","reported_seq":19,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222849+0000","last_change":"2024-09-17T04:27:43.200387+0000","last_active":"2024-09-17T04:27:45.222849+0000","last_peered":"2024-09-17T04:27:45.222849+0000","last_clean":"2024-09-17T04:27:45.222849+0000","last_became_active":"2024-09-17T04:27:43.199763+0000","last_became_peered":"2024-09-17T04:27:43.199763+0000","last_unstale":"2024-09-17T04:27:45.222849+0000","last_undegraded":"2024-09-17T04:27:45.222849+0000","last_fullsized":"2024-09-17T04:27:45.222849+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:19.758401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.484326+0000","last_change":"2024-09-17T04:27:44.216650+0000","last_active":"2024-09-17T04:27:45.484326+0000","last_peered":"2024-09-17T04:27:45.484326+0000","last_clean":"2024-09-17T04:27:45.484326+0000","last_became_active":"2024-09-17T04:27:44.216237+0000","last_became_peered":"2024-09-17T04:27:44.216237+0000","last_unstale":"2024-09-17T04:27:45.484326+0000","last_undegraded":"2024-09-17T04:27:45.484326+0000","last_fullsized":"2024-09-17T04:27:45.484326+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:37.241286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"60'1","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212757+0000","last_change":"2024-09-17T04:27:43.206766+0000","last_active":"2024-09-17T04:27:45.212757+0000","last_peered":"2024-09-17T04:27:45.212757+0000","last_clean":"2024-09-17T04:27:45.212757+0000","last_became_active":"2024-09-17T04:27:43.206455+0000","last_became_peered":"2024-09-17T04:27:43.206455+0000","last_unstale":"2024-09-17T04:27:45.212757+0000","last_undegraded":"2024-09-17T04:27:45.212757+0000","last_fullsized":"2024-09-17T04:27:45.212757+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:06:22.067347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222393+0000","last_change":"2024-09-17T04:27:44.230666+0000","last_active":"2024-09-17T04:27:45.222393+0000","last_peered":"2024-09-17T04:27:45.222393+0000","last_clean":"2024-09-17T04:27:45.222393+0000","last_became_active":"2024-09-17T04:27:44.230350+0000","last_became_peered":"2024-09-17T04:27:44.230350+0000","last_unstale":"2024-09-17T04:27:45.222393+0000","last_undegraded":"2024-09-17T04:27:45.222393+0000","last_fullsized":"2024-09-17T04:27:45.222393+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:14.532539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"60'2","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222622+0000","last_change":"2024-09-17T04:27:43.201555+0000","last_active":"2024-09-17T04:27:45.222622+0000","last_peered":"2024-09-17T04:27:45.222622+0000","last_clean":"2024-09-17T04:27:45.222622+0000","last_became_active":"2024-09-17T04:27:43.200837+0000","last_became_peered":"2024-09-17T04:27:43.200837+0000","last_unstale":"2024-09-17T04:27:45.222622+0000","last_undegraded":"2024-09-17T04:27:45.222622+0000","last_fullsized":"2024-09-17T04:27:45.222622+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:59.660874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222582+0000","last_change":"2024-09-17T04:27:44.219425+0000","last_active":"2024-09-17T04:27:45.222582+0000","last_peered":"2024-09-17T04:27:45.222582+0000","last_clean":"2024-09-17T04:27:45.222582+0000","last_became_active":"2024-09-17T04:27:44.217358+0000","last_became_peered":"2024-09-17T04:27:44.217358+0000","last_unstale":"2024-09-17T04:27:45.222582+0000","last_undegraded":"2024-09-17T04:27:45.222582+0000","last_fullsized":"2024-09-17T04:27:45.222582+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:23:10.166526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"60'2","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.217052+0000","last_change":"2024-09-17T04:27:43.207037+0000","last_active":"2024-09-17T04:27:45.217052+0000","last_peered":"2024-09-17T04:27:45.217052+0000","last_clean":"2024-09-17T04:27:45.217052+0000","last_became_active":"2024-09-17T04:27:43.206672+0000","last_became_peered":"2024-09-17T04:27:43.206672+0000","last_unstale":"2024-09-17T04:27:45.217052+0000","last_undegraded":"2024-09-17T04:27:45.217052+0000","last_fullsized":"2024-09-17T04:27:45.217052+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:47.133684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.213106+0000","last_change":"2024-09-17T04:27:44.217751+0000","last_active":"2024-09-17T04:27:45.213106+0000","last_peered":"2024-09-17T04:27:45.213106+0000","last_clean":"2024-09-17T04:27:45.213106+0000","last_became_active":"2024-09-17T04:27:44.214072+0000","last_became_peered":"2024-09-17T04:27:44.214072+0000","last_unstale":"2024-09-17T04:27:45.213106+0000","last_undegraded":"2024-09-17T04:27:45.213106+0000","last_fullsized":"2024-09-17T04:27:45.213106+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:27.160546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"60'1","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.215965+0000","last_change":"2024-09-17T04:27:43.198922+0000","last_active":"2024-09-17T04:27:45.215965+0000","last_peered":"2024-09-17T04:27:45.215965+0000","last_clean":"2024-09-17T04:27:45.215965+0000","last_became_active":"2024-09-17T04:27:43.198453+0000","last_became_peered":"2024-09-17T04:27:43.198453+0000","last_unstale":"2024-09-17T04:27:45.215965+0000","last_undegraded":"2024-09-17T04:27:45.215965+0000","last_fullsized":"2024-09-17T04:27:45.215965+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:43.961353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212477+0000","last_change":"2024-09-17T04:27:44.222601+0000","last_active":"2024-09-17T04:27:45.212477+0000","last_peered":"2024-09-17T04:27:45.212477+0000","last_clean":"2024-09-17T04:27:45.212477+0000","last_became_active":"2024-09-17T04:27:44.221587+0000","last_became_peered":"2024-09-17T04:27:44.221587+0000","last_unstale":"2024-09-17T04:27:45.212477+0000","last_undegraded":"2024-09-17T04:27:45.212477+0000","last_fullsized":"2024-09-17T04:27:45.212477+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:22.587679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"60'1","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222375+0000","last_change":"2024-09-17T04:27:43.207536+0000","last_active":"2024-09-17T04:27:45.222375+0000","last_peered":"2024-09-17T04:27:45.222375+0000","last_clean":"2024-09-17T04:27:45.222375+0000","last_became_active":"2024-09-17T04:27:43.206356+0000","last_became_peered":"2024-09-17T04:27:43.206356+0000","last_unstale":"2024-09-17T04:27:45.222375+0000","last_undegraded":"2024-09-17T04:27:45.222375+0000","last_fullsized":"2024-09-17T04:27:45.222375+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:47.133684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.225791+0000","last_change":"2024-09-17T04:27:44.220110+0000","last_active":"2024-09-17T04:27:45.225791+0000","last_peered":"2024-09-17T04:27:45.225791+0000","last_clean":"2024-09-17T04:27:45.225791+0000","last_became_active":"2024-09-17T04:27:44.219708+0000","last_became_peered":"2024-09-17T04:27:44.219708+0000","last_unstale":"2024-09-17T04:27:45.225791+0000","last_undegraded":"2024-09-17T04:27:45.225791+0000","last_fullsized":"2024-09-17T04:27:45.225791+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:32:28.733174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.216093+0000","last_change":"2024-09-17T04:27:43.200857+0000","last_active":"2024-09-17T04:27:45.216093+0000","last_peered":"2024-09-17T04:27:45.216093+0000","last_clean":"2024-09-17T04:27:45.216093+0000","last_became_active":"2024-09-17T04:27:43.200518+0000","last_became_peered":"2024-09-17T04:27:43.200518+0000","last_unstale":"2024-09-17T04:27:45.216093+0000","last_undegraded":"2024-09-17T04:27:45.216093+0000","last_fullsized":"2024-09-17T04:27:45.216093+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:00:58.429429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212733+0000","last_change":"2024-09-17T04:27:44.225414+0000","last_active":"2024-09-17T04:27:45.212733+0000","last_peered":"2024-09-17T04:27:45.212733+0000","last_clean":"2024-09-17T04:27:45.212733+0000","last_became_active":"2024-09-17T04:27:44.225112+0000","last_became_peered":"2024-09-17T04:27:44.225112+0000","last_unstale":"2024-09-17T04:27:45.212733+0000","last_undegraded":"2024-09-17T04:27:45.212733+0000","last_fullsized":"2024-09-17T04:27:45.212733+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:05:18.890521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.217327+0000","last_change":"2024-09-17T04:27:43.209713+0000","last_active":"2024-09-17T04:27:45.217327+0000","last_peered":"2024-09-17T04:27:45.217327+0000","last_clean":"2024-09-17T04:27:45.217327+0000","last_became_active":"2024-09-17T04:27:43.209316+0000","last_became_peered":"2024-09-17T04:27:43.209316+0000","last_unstale":"2024-09-17T04:27:45.217327+0000","last_undegraded":"2024-09-17T04:27:45.217327+0000","last_fullsized":"2024-09-17T04:27:45.217327+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:11.559902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.217008+0000","last_change":"2024-09-17T04:27:44.237993+0000","last_active":"2024-09-17T04:27:45.217008+0000","last_peered":"2024-09-17T04:27:45.217008+0000","last_clean":"2024-09-17T04:27:45.217008+0000","last_became_active":"2024-09-17T04:27:44.237721+0000","last_became_peered":"2024-09-17T04:27:44.237721+0000","last_unstale":"2024-09-17T04:27:45.217008+0000","last_undegraded":"2024-09-17T04:27:45.217008+0000","last_fullsized":"2024-09-17T04:27:45.217008+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:30.929757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.484271+0000","last_change":"2024-09-17T04:27:43.204007+0000","last_active":"2024-09-17T04:27:45.484271+0000","last_peered":"2024-09-17T04:27:45.484271+0000","last_clean":"2024-09-17T04:27:45.484271+0000","last_became_active":"2024-09-17T04:27:43.203747+0000","last_became_peered":"2024-09-17T04:27:43.203747+0000","last_unstale":"2024-09-17T04:27:45.484271+0000","last_undegraded":"2024-09-17T04:27:45.484271+0000","last_fullsized":"2024-09-17T04:27:45.484271+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:02.426588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.484295+0000","last_change":"2024-09-17T04:27:44.238312+0000","last_active":"2024-09-17T04:27:45.484295+0000","last_peered":"2024-09-17T04:27:45.484295+0000","last_clean":"2024-09-17T04:27:45.484295+0000","last_became_active":"2024-09-17T04:27:44.237930+0000","last_became_peered":"2024-09-17T04:27:44.237930+0000","last_unstale":"2024-09-17T04:27:45.484295+0000","last_undegraded":"2024-09-17T04:27:45.484295+0000","last_fullsized":"2024-09-17T04:27:45.484295+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:16.774330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212854+0000","last_change":"2024-09-17T04:27:43.201502+0000","last_active":"2024-09-17T04:27:45.212854+0000","last_peered":"2024-09-17T04:27:45.212854+0000","last_clean":"2024-09-17T04:27:45.212854+0000","last_became_active":"2024-09-17T04:27:43.201071+0000","last_became_peered":"2024-09-17T04:27:43.201071+0000","last_unstale":"2024-09-17T04:27:45.212854+0000","last_undegraded":"2024-09-17T04:27:45.212854+0000","last_fullsized":"2024-09-17T04:27:45.212854+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:48:58.744170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483780+0000","last_change":"2024-09-17T04:27:44.226655+0000","last_active":"2024-09-17T04:27:45.483780+0000","last_peered":"2024-09-17T04:27:45.483780+0000","last_clean":"2024-09-17T04:27:45.483780+0000","last_became_active":"2024-09-17T04:27:44.226363+0000","last_became_peered":"2024-09-17T04:27:44.226363+0000","last_unstale":"2024-09-17T04:27:45.483780+0000","last_undegraded":"2024-09-17T04:27:45.483780+0000","last_fullsized":"2024-09-17T04:27:45.483780+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:43:25.897835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"60'3","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.484600+0000","last_change":"2024-09-17T04:27:43.203357+0000","last_active":"2024-09-17T04:27:45.484600+0000","last_peered":"2024-09-17T04:27:45.484600+0000","last_clean":"2024-09-17T04:27:45.484600+0000","last_became_active":"2024-09-17T04:27:43.202999+0000","last_became_peered":"2024-09-17T04:27:43.202999+0000","last_unstale":"2024-09-17T04:27:45.484600+0000","last_undegraded":"2024-09-17T04:27:45.484600+0000","last_fullsized":"2024-09-17T04:27:45.484600+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:58.011654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.484428+0000","last_change":"2024-09-17T04:27:44.240301+0000","last_active":"2024-09-17T04:27:45.484428+0000","last_peered":"2024-09-17T04:27:45.484428+0000","last_clean":"2024-09-17T04:27:45.484428+0000","last_became_active":"2024-09-17T04:27:44.237792+0000","last_became_peered":"2024-09-17T04:27:44.237792+0000","last_unstale":"2024-09-17T04:27:45.484428+0000","last_undegraded":"2024-09-17T04:27:45.484428+0000","last_fullsized":"2024-09-17T04:27:45.484428+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:36.313761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"60'6","reported_seq":21,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.216322+0000","last_change":"2024-09-17T04:27:43.200549+0000","last_active":"2024-09-17T04:27:45.216322+0000","last_peered":"2024-09-17T04:27:45.216322+0000","last_clean":"2024-09-17T04:27:45.216322+0000","last_became_active":"2024-09-17T04:27:43.200245+0000","last_became_peered":"2024-09-17T04:27:43.200245+0000","last_unstale":"2024-09-17T04:27:45.216322+0000","last_undegraded":"2024-09-17T04:27:45.216322+0000","last_fullsized":"2024-09-17T04:27:45.216322+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:36.234900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.216656+0000","last_change":"2024-09-17T04:27:44.225782+0000","last_active":"2024-09-17T04:27:45.216656+0000","last_peered":"2024-09-17T04:27:45.216656+0000","last_clean":"2024-09-17T04:27:45.216656+0000","last_became_active":"2024-09-17T04:27:44.225436+0000","last_became_peered":"2024-09-17T04:27:44.225436+0000","last_unstale":"2024-09-17T04:27:45.216656+0000","last_undegraded":"2024-09-17T04:27:45.216656+0000","last_fullsized":"2024-09-17T04:27:45.216656+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:49.106024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"60'2","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212414+0000","last_change":"2024-09-17T04:27:43.204124+0000","last_active":"2024-09-17T04:27:45.212414+0000","last_peered":"2024-09-17T04:27:45.212414+0000","last_clean":"2024-09-17T04:27:45.212414+0000","last_became_active":"2024-09-17T04:27:43.203730+0000","last_became_peered":"2024-09-17T04:27:43.203730+0000","last_unstale":"2024-09-17T04:27:45.212414+0000","last_undegraded":"2024-09-17T04:27:45.212414+0000","last_fullsized":"2024-09-17T04:27:45.212414+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:15:50.701517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483825+0000","last_change":"2024-09-17T04:27:44.219797+0000","last_active":"2024-09-17T04:27:45.483825+0000","last_peered":"2024-09-17T04:27:45.483825+0000","last_clean":"2024-09-17T04:27:45.483825+0000","last_became_active":"2024-09-17T04:27:44.219229+0000","last_became_peered":"2024-09-17T04:27:44.219229+0000","last_unstale":"2024-09-17T04:27:45.483825+0000","last_undegraded":"2024-09-17T04:27:45.483825+0000","last_fullsized":"2024-09-17T04:27:45.483825+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:34.529555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.225532+0000","last_change":"2024-09-17T04:27:43.191819+0000","last_active":"2024-09-17T04:27:45.225532+0000","last_peered":"2024-09-17T04:27:45.225532+0000","last_clean":"2024-09-17T04:27:45.225532+0000","last_became_active":"2024-09-17T04:27:43.191432+0000","last_became_peered":"2024-09-17T04:27:43.191432+0000","last_unstale":"2024-09-17T04:27:45.225532+0000","last_undegraded":"2024-09-17T04:27:45.225532+0000","last_fullsized":"2024-09-17T04:27:45.225532+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:46:51.373799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.215756+0000","last_change":"2024-09-17T04:27:44.216494+0000","last_active":"2024-09-17T04:27:45.215756+0000","last_peered":"2024-09-17T04:27:45.215756+0000","last_clean":"2024-09-17T04:27:45.215756+0000","last_became_active":"2024-09-17T04:27:44.216164+0000","last_became_peered":"2024-09-17T04:27:44.216164+0000","last_unstale":"2024-09-17T04:27:45.215756+0000","last_undegraded":"2024-09-17T04:27:45.215756+0000","last_fullsized":"2024-09-17T04:27:45.215756+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:00:59.435815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.216251+0000","last_change":"2024-09-17T04:27:43.206779+0000","last_active":"2024-09-17T04:27:45.216251+0000","last_peered":"2024-09-17T04:27:45.216251+0000","last_clean":"2024-09-17T04:27:45.216251+0000","last_became_active":"2024-09-17T04:27:43.205945+0000","last_became_peered":"2024-09-17T04:27:43.205945+0000","last_unstale":"2024-09-17T04:27:45.216251+0000","last_undegraded":"2024-09-17T04:27:45.216251+0000","last_fullsized":"2024-09-17T04:27:45.216251+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:59.660874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483883+0000","last_change":"2024-09-17T04:27:44.228358+0000","last_active":"2024-09-17T04:27:45.483883+0000","last_peered":"2024-09-17T04:27:45.483883+0000","last_clean":"2024-09-17T04:27:45.483883+0000","last_became_active":"2024-09-17T04:27:44.227977+0000","last_became_peered":"2024-09-17T04:27:44.227977+0000","last_unstale":"2024-09-17T04:27:45.483883+0000","last_undegraded":"2024-09-17T04:27:45.483883+0000","last_fullsized":"2024-09-17T04:27:45.483883+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:40.580708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"60'8","reported_seq":23,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483542+0000","last_change":"2024-09-17T04:27:43.203507+0000","last_active":"2024-09-17T04:27:45.483542+0000","last_peered":"2024-09-17T04:27:45.483542+0000","last_clean":"2024-09-17T04:27:45.483542+0000","last_became_active":"2024-09-17T04:27:43.202223+0000","last_became_peered":"2024-09-17T04:27:43.202223+0000","last_unstale":"2024-09-17T04:27:45.483542+0000","last_undegraded":"2024-09-17T04:27:45.483542+0000","last_fullsized":"2024-09-17T04:27:45.483542+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:25.714760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.484101+0000","last_change":"2024-09-17T04:27:44.221929+0000","last_active":"2024-09-17T04:27:45.484101+0000","last_peered":"2024-09-17T04:27:45.484101+0000","last_clean":"2024-09-17T04:27:45.484101+0000","last_became_active":"2024-09-17T04:27:44.221064+0000","last_became_peered":"2024-09-17T04:27:44.221064+0000","last_unstale":"2024-09-17T04:27:45.484101+0000","last_undegraded":"2024-09-17T04:27:45.484101+0000","last_fullsized":"2024-09-17T04:27:45.484101+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:18:43.380081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"60'1","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.213160+0000","last_change":"2024-09-17T04:27:43.206771+0000","last_active":"2024-09-17T04:27:45.213160+0000","last_peered":"2024-09-17T04:27:45.213160+0000","last_clean":"2024-09-17T04:27:45.213160+0000","last_became_active":"2024-09-17T04:27:43.206481+0000","last_became_peered":"2024-09-17T04:27:43.206481+0000","last_unstale":"2024-09-17T04:27:45.213160+0000","last_undegraded":"2024-09-17T04:27:45.213160+0000","last_fullsized":"2024-09-17T04:27:45.213160+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:48:49.832510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.217874+0000","last_change":"2024-09-17T04:27:44.218916+0000","last_active":"2024-09-17T04:27:45.217874+0000","last_peered":"2024-09-17T04:27:45.217874+0000","last_clean":"2024-09-17T04:27:45.217874+0000","last_became_active":"2024-09-17T04:27:44.218602+0000","last_became_peered":"2024-09-17T04:27:44.218602+0000","last_unstale":"2024-09-17T04:27:45.217874+0000","last_undegraded":"2024-09-17T04:27:45.217874+0000","last_fullsized":"2024-09-17T04:27:45.217874+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:20:40.092936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"60'2","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222554+0000","last_change":"2024-09-17T04:27:43.205128+0000","last_active":"2024-09-17T04:27:45.222554+0000","last_peered":"2024-09-17T04:27:45.222554+0000","last_clean":"2024-09-17T04:27:45.222554+0000","last_became_active":"2024-09-17T04:27:43.204836+0000","last_became_peered":"2024-09-17T04:27:43.204836+0000","last_unstale":"2024-09-17T04:27:45.222554+0000","last_undegraded":"2024-09-17T04:27:45.222554+0000","last_fullsized":"2024-09-17T04:27:45.222554+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:07.729772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212603+0000","last_change":"2024-09-17T04:27:44.222455+0000","last_active":"2024-09-17T04:27:45.212603+0000","last_peered":"2024-09-17T04:27:45.212603+0000","last_clean":"2024-09-17T04:27:45.212603+0000","last_became_active":"2024-09-17T04:27:44.222015+0000","last_became_peered":"2024-09-17T04:27:44.222015+0000","last_unstale":"2024-09-17T04:27:45.212603+0000","last_undegraded":"2024-09-17T04:27:45.212603+0000","last_fullsized":"2024-09-17T04:27:45.212603+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:01:05.264040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"60'1","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.217724+0000","last_change":"2024-09-17T04:27:43.207246+0000","last_active":"2024-09-17T04:27:45.217724+0000","last_peered":"2024-09-17T04:27:45.217724+0000","last_clean":"2024-09-17T04:27:45.217724+0000","last_became_active":"2024-09-17T04:27:43.206835+0000","last_became_peered":"2024-09-17T04:27:43.206835+0000","last_unstale":"2024-09-17T04:27:45.217724+0000","last_undegraded":"2024-09-17T04:27:45.217724+0000","last_fullsized":"2024-09-17T04:27:45.217724+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:19.758401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.484447+0000","last_change":"2024-09-17T04:27:44.238294+0000","last_active":"2024-09-17T04:27:45.484447+0000","last_peered":"2024-09-17T04:27:45.484447+0000","last_clean":"2024-09-17T04:27:45.484447+0000","last_became_active":"2024-09-17T04:27:44.237968+0000","last_became_peered":"2024-09-17T04:27:44.237968+0000","last_unstale":"2024-09-17T04:27:45.484447+0000","last_undegraded":"2024-09-17T04:27:45.484447+0000","last_fullsized":"2024-09-17T04:27:45.484447+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:49.705694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222471+0000","last_change":"2024-09-17T04:27:43.209312+0000","last_active":"2024-09-17T04:27:45.222471+0000","last_peered":"2024-09-17T04:27:45.222471+0000","last_clean":"2024-09-17T04:27:45.222471+0000","last_became_active":"2024-09-17T04:27:43.208984+0000","last_became_peered":"2024-09-17T04:27:43.208984+0000","last_unstale":"2024-09-17T04:27:45.222471+0000","last_undegraded":"2024-09-17T04:27:45.222471+0000","last_fullsized":"2024-09-17T04:27:45.222471+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:00:58.429429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.216952+0000","last_change":"2024-09-17T04:27:44.218883+0000","last_active":"2024-09-17T04:27:45.216952+0000","last_peered":"2024-09-17T04:27:45.216952+0000","last_clean":"2024-09-17T04:27:45.216952+0000","last_became_active":"2024-09-17T04:27:44.218549+0000","last_became_peered":"2024-09-17T04:27:44.218549+0000","last_unstale":"2024-09-17T04:27:45.216952+0000","last_undegraded":"2024-09-17T04:27:45.216952+0000","last_fullsized":"2024-09-17T04:27:45.216952+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:34.529555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"60'2","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212605+0000","last_change":"2024-09-17T04:27:43.203669+0000","last_active":"2024-09-17T04:27:45.212605+0000","last_peered":"2024-09-17T04:27:45.212605+0000","last_clean":"2024-09-17T04:27:45.212605+0000","last_became_active":"2024-09-17T04:27:43.203307+0000","last_became_peered":"2024-09-17T04:27:43.203307+0000","last_unstale":"2024-09-17T04:27:45.212605+0000","last_undegraded":"2024-09-17T04:27:45.212605+0000","last_fullsized":"2024-09-17T04:27:45.212605+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:53.295497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222398+0000","last_change":"2024-09-17T04:27:44.222562+0000","last_active":"2024-09-17T04:27:45.222398+0000","last_peered":"2024-09-17T04:27:45.222398+0000","last_clean":"2024-09-17T04:27:45.222398+0000","last_became_active":"2024-09-17T04:27:44.221122+0000","last_became_peered":"2024-09-17T04:27:44.221122+0000","last_unstale":"2024-09-17T04:27:45.222398+0000","last_undegraded":"2024-09-17T04:27:45.222398+0000","last_fullsized":"2024-09-17T04:27:45.222398+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:40.580708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483419+0000","last_change":"2024-09-17T04:27:43.189344+0000","last_active":"2024-09-17T04:27:45.483419+0000","last_peered":"2024-09-17T04:27:45.483419+0000","last_clean":"2024-09-17T04:27:45.483419+0000","last_became_active":"2024-09-17T04:27:43.189023+0000","last_became_peered":"2024-09-17T04:27:43.189023+0000","last_unstale":"2024-09-17T04:27:45.483419+0000","last_undegraded":"2024-09-17T04:27:45.483419+0000","last_fullsized":"2024-09-17T04:27:45.483419+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:07.652705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222197+0000","last_change":"2024-09-17T04:27:44.216597+0000","last_active":"2024-09-17T04:27:45.222197+0000","last_peered":"2024-09-17T04:27:45.222197+0000","last_clean":"2024-09-17T04:27:45.222197+0000","last_became_active":"2024-09-17T04:27:44.216253+0000","last_became_peered":"2024-09-17T04:27:44.216253+0000","last_unstale":"2024-09-17T04:27:45.222197+0000","last_undegraded":"2024-09-17T04:27:45.222197+0000","last_fullsized":"2024-09-17T04:27:45.222197+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:04.948283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.216031+0000","last_change":"2024-09-17T04:27:43.209348+0000","last_active":"2024-09-17T04:27:45.216031+0000","last_peered":"2024-09-17T04:27:45.216031+0000","last_clean":"2024-09-17T04:27:45.216031+0000","last_became_active":"2024-09-17T04:27:43.209055+0000","last_became_peered":"2024-09-17T04:27:43.209055+0000","last_unstale":"2024-09-17T04:27:45.216031+0000","last_undegraded":"2024-09-17T04:27:45.216031+0000","last_fullsized":"2024-09-17T04:27:45.216031+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:37:42.378184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.217864+0000","last_change":"2024-09-17T04:27:44.213695+0000","last_active":"2024-09-17T04:27:45.217864+0000","last_peered":"2024-09-17T04:27:45.217864+0000","last_clean":"2024-09-17T04:27:45.217864+0000","last_became_active":"2024-09-17T04:27:44.213346+0000","last_became_peered":"2024-09-17T04:27:44.213346+0000","last_unstale":"2024-09-17T04:27:45.217864+0000","last_undegraded":"2024-09-17T04:27:45.217864+0000","last_fullsized":"2024-09-17T04:27:45.217864+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:22.439743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"60'2","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.484268+0000","last_change":"2024-09-17T04:27:43.204645+0000","last_active":"2024-09-17T04:27:45.484268+0000","last_peered":"2024-09-17T04:27:45.484268+0000","last_clean":"2024-09-17T04:27:45.484268+0000","last_became_active":"2024-09-17T04:27:43.204334+0000","last_became_peered":"2024-09-17T04:27:43.204334+0000","last_unstale":"2024-09-17T04:27:45.484268+0000","last_undegraded":"2024-09-17T04:27:45.484268+0000","last_fullsized":"2024-09-17T04:27:45.484268+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:13.461537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.218051+0000","last_change":"2024-09-17T04:27:44.214273+0000","last_active":"2024-09-17T04:27:45.218051+0000","last_peered":"2024-09-17T04:27:45.218051+0000","last_clean":"2024-09-17T04:27:45.218051+0000","last_became_active":"2024-09-17T04:27:44.213931+0000","last_became_peered":"2024-09-17T04:27:44.213931+0000","last_unstale":"2024-09-17T04:27:45.218051+0000","last_undegraded":"2024-09-17T04:27:45.218051+0000","last_fullsized":"2024-09-17T04:27:45.218051+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:58:10.407213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"61'1","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222644+0000","last_change":"2024-09-17T04:27:43.206910+0000","last_active":"2024-09-17T04:27:45.222644+0000","last_peered":"2024-09-17T04:27:45.222644+0000","last_clean":"2024-09-17T04:27:45.222644+0000","last_became_active":"2024-09-17T04:27:43.206607+0000","last_became_peered":"2024-09-17T04:27:43.206607+0000","last_unstale":"2024-09-17T04:27:45.222644+0000","last_undegraded":"2024-09-17T04:27:45.222644+0000","last_fullsized":"2024-09-17T04:27:45.222644+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:29.923371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.484054+0000","last_change":"2024-09-17T04:27:44.229969+0000","last_active":"2024-09-17T04:27:45.484054+0000","last_peered":"2024-09-17T04:27:45.484054+0000","last_clean":"2024-09-17T04:27:45.484054+0000","last_became_active":"2024-09-17T04:27:44.229665+0000","last_became_peered":"2024-09-17T04:27:44.229665+0000","last_unstale":"2024-09-17T04:27:45.484054+0000","last_undegraded":"2024-09-17T04:27:45.484054+0000","last_fullsized":"2024-09-17T04:27:45.484054+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:02.779912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.225738+0000","last_change":"2024-09-17T04:27:43.200897+0000","last_active":"2024-09-17T04:27:45.225738+0000","last_peered":"2024-09-17T04:27:45.225738+0000","last_clean":"2024-09-17T04:27:45.225738+0000","last_became_active":"2024-09-17T04:27:43.200425+0000","last_became_peered":"2024-09-17T04:27:43.200425+0000","last_unstale":"2024-09-17T04:27:45.225738+0000","last_undegraded":"2024-09-17T04:27:45.225738+0000","last_fullsized":"2024-09-17T04:27:45.225738+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:05:00.277014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222723+0000","last_change":"2024-09-17T04:27:44.218886+0000","last_active":"2024-09-17T04:27:45.222723+0000","last_peered":"2024-09-17T04:27:45.222723+0000","last_clean":"2024-09-17T04:27:45.222723+0000","last_became_active":"2024-09-17T04:27:44.218277+0000","last_became_peered":"2024-09-17T04:27:44.218277+0000","last_unstale":"2024-09-17T04:27:45.222723+0000","last_undegraded":"2024-09-17T04:27:45.222723+0000","last_fullsized":"2024-09-17T04:27:45.222723+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:56:07.557036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.217175+0000","last_change":"2024-09-17T04:27:43.203031+0000","last_active":"2024-09-17T04:27:45.217175+0000","last_peered":"2024-09-17T04:27:45.217175+0000","last_clean":"2024-09-17T04:27:45.217175+0000","last_became_active":"2024-09-17T04:27:43.202637+0000","last_became_peered":"2024-09-17T04:27:43.202637+0000","last_unstale":"2024-09-17T04:27:45.217175+0000","last_undegraded":"2024-09-17T04:27:45.217175+0000","last_fullsized":"2024-09-17T04:27:45.217175+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:07.729772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.213008+0000","last_change":"2024-09-17T04:27:44.217560+0000","last_active":"2024-09-17T04:27:45.213008+0000","last_peered":"2024-09-17T04:27:45.213008+0000","last_clean":"2024-09-17T04:27:45.213008+0000","last_became_active":"2024-09-17T04:27:44.217250+0000","last_became_peered":"2024-09-17T04:27:44.217250+0000","last_unstale":"2024-09-17T04:27:45.213008+0000","last_undegraded":"2024-09-17T04:27:45.213008+0000","last_fullsized":"2024-09-17T04:27:45.213008+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:47:24.767897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483466+0000","last_change":"2024-09-17T04:27:43.201109+0000","last_active":"2024-09-17T04:27:45.483466+0000","last_peered":"2024-09-17T04:27:45.483466+0000","last_clean":"2024-09-17T04:27:45.483466+0000","last_became_active":"2024-09-17T04:27:43.200806+0000","last_became_peered":"2024-09-17T04:27:43.200806+0000","last_unstale":"2024-09-17T04:27:45.483466+0000","last_undegraded":"2024-09-17T04:27:45.483466+0000","last_fullsized":"2024-09-17T04:27:45.483466+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:35.455090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.484131+0000","last_change":"2024-09-17T04:27:44.229991+0000","last_active":"2024-09-17T04:27:45.484131+0000","last_peered":"2024-09-17T04:27:45.484131+0000","last_clean":"2024-09-17T04:27:45.484131+0000","last_became_active":"2024-09-17T04:27:44.229682+0000","last_became_peered":"2024-09-17T04:27:44.229682+0000","last_unstale":"2024-09-17T04:27:45.484131+0000","last_undegraded":"2024-09-17T04:27:45.484131+0000","last_fullsized":"2024-09-17T04:27:45.484131+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:58:10.407213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.216903+0000","last_change":"2024-09-17T04:27:43.201986+0000","last_active":"2024-09-17T04:27:45.216903+0000","last_peered":"2024-09-17T04:27:45.216903+0000","last_clean":"2024-09-17T04:27:45.216903+0000","last_became_active":"2024-09-17T04:27:43.201654+0000","last_became_peered":"2024-09-17T04:27:43.201654+0000","last_unstale":"2024-09-17T04:27:45.216903+0000","last_undegraded":"2024-09-17T04:27:45.216903+0000","last_fullsized":"2024-09-17T04:27:45.216903+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:25.109734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483266+0000","last_change":"2024-09-17T04:27:44.241055+0000","last_active":"2024-09-17T04:27:45.483266+0000","last_peered":"2024-09-17T04:27:45.483266+0000","last_clean":"2024-09-17T04:27:45.483266+0000","last_became_active":"2024-09-17T04:27:44.238151+0000","last_became_peered":"2024-09-17T04:27:44.238151+0000","last_unstale":"2024-09-17T04:27:45.483266+0000","last_undegraded":"2024-09-17T04:27:45.483266+0000","last_fullsized":"2024-09-17T04:27:45.483266+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:45.589759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483212+0000","last_change":"2024-09-17T04:27:43.200262+0000","last_active":"2024-09-17T04:27:45.483212+0000","last_peered":"2024-09-17T04:27:45.483212+0000","last_clean":"2024-09-17T04:27:45.483212+0000","last_became_active":"2024-09-17T04:27:43.199434+0000","last_became_peered":"2024-09-17T04:27:43.199434+0000","last_unstale":"2024-09-17T04:27:45.483212+0000","last_undegraded":"2024-09-17T04:27:45.483212+0000","last_fullsized":"2024-09-17T04:27:45.483212+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:14.709674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212826+0000","last_change":"2024-09-17T04:27:44.223877+0000","last_active":"2024-09-17T04:27:45.212826+0000","last_peered":"2024-09-17T04:27:45.212826+0000","last_clean":"2024-09-17T04:27:45.212826+0000","last_became_active":"2024-09-17T04:27:44.223582+0000","last_became_peered":"2024-09-17T04:27:44.223582+0000","last_unstale":"2024-09-17T04:27:45.212826+0000","last_undegraded":"2024-09-17T04:27:45.212826+0000","last_fullsized":"2024-09-17T04:27:45.212826+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:09.084549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212546+0000","last_change":"2024-09-17T04:27:43.201747+0000","last_active":"2024-09-17T04:27:45.212546+0000","last_peered":"2024-09-17T04:27:45.212546+0000","last_clean":"2024-09-17T04:27:45.212546+0000","last_became_active":"2024-09-17T04:27:43.200792+0000","last_became_peered":"2024-09-17T04:27:43.200792+0000","last_unstale":"2024-09-17T04:27:45.212546+0000","last_undegraded":"2024-09-17T04:27:45.212546+0000","last_fullsized":"2024-09-17T04:27:45.212546+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:30.181955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212302+0000","last_change":"2024-09-17T04:27:44.223584+0000","last_active":"2024-09-17T04:27:45.212302+0000","last_peered":"2024-09-17T04:27:45.212302+0000","last_clean":"2024-09-17T04:27:45.212302+0000","last_became_active":"2024-09-17T04:27:44.220615+0000","last_became_peered":"2024-09-17T04:27:44.220615+0000","last_unstale":"2024-09-17T04:27:45.212302+0000","last_undegraded":"2024-09-17T04:27:45.212302+0000","last_fullsized":"2024-09-17T04:27:45.212302+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:46.132448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.225624+0000","last_change":"2024-09-17T04:27:43.203968+0000","last_active":"2024-09-17T04:27:45.225624+0000","last_peered":"2024-09-17T04:27:45.225624+0000","last_clean":"2024-09-17T04:27:45.225624+0000","last_became_active":"2024-09-17T04:27:43.203677+0000","last_became_peered":"2024-09-17T04:27:43.203677+0000","last_unstale":"2024-09-17T04:27:45.225624+0000","last_undegraded":"2024-09-17T04:27:45.225624+0000","last_fullsized":"2024-09-17T04:27:45.225624+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:45:17.816227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":184,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483827+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:27:45.483827+0000","last_peered":"2024-09-17T04:27:45.483827+0000","last_clean":"2024-09-17T04:27:45.483827+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:27:45.483827+0000","last_undegraded":"2024-09-17T04:27:45.483827+0000","last_fullsized":"2024-09-17T04:27:45.483827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476786,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1323008,"data_stored":889894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476785,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27732,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":720896,"data_stored":299366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476786,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":729088,"data_stored":300018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476786,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":733184,"data_stored":300978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":729088,"data_stored":299422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509488,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31900,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339940,"statfs":{"total":96636764160,"available":96604098560,"internally_reserved":0,"allocated":724992,"data_stored":299456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509488,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1331200,"data_stored":891928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542193,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":729088,"data_stored":299422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:27:53.606 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:27:53.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.838+0000 7f965c000640 1 -- 172.21.15.100:0/3368894231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f963c0050c0 msgr2=0x7f963c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:53.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.838+0000 7f965c000640 1 --2- 172.21.15.100:0/3368894231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f963c0050c0 0x7f963c00b9c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f964c009ed0 tx=0x7f963c005830 comp rx=0 tx=0).stop 2024-09-17T04:27:53.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.838+0000 7f965a000640 1 -- 172.21.15.100:0/3368894231 reap_dead start 2024-09-17T04:27:53.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.838+0000 7f965c000640 1 -- 172.21.15.100:0/3368894231 shutdown_connections 2024-09-17T04:27:53.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.838+0000 7f965c000640 1 -- 172.21.15.100:0/3368894231 >> 172.21.15.100:0/3368894231 conn(0x7f965406d730 msgr2=0x7f96540b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:53.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.838+0000 7f965c000640 1 -- 172.21.15.100:0/3368894231 shutdown_connections 2024-09-17T04:27:53.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.838+0000 7f965c000640 1 -- 172.21.15.100:0/3368894231 wait complete. 2024-09-17T04:27:53.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.840+0000 7f965c000640 1 Processor -- start 2024-09-17T04:27:53.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.840+0000 7f965c000640 1 -- start start 2024-09-17T04:27:53.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.842+0000 7f965c000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f963c0050c0 0x7f965415af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:53.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.842+0000 7f965c000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9654147870 0x7f9654073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:53.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.842+0000 7f965c000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96540cbd40 0x7f9654073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:53.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.842+0000 7f965c000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9654164540 con 0x7f9654147870 2024-09-17T04:27:53.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.842+0000 7f965c000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f96540b4970 con 0x7f96540cbd40 2024-09-17T04:27:53.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.842+0000 7f965c000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f96541646c0 con 0x7f963c0050c0 2024-09-17T04:27:53.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.842+0000 7f9658c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9654147870 0x7f9654073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:53.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.842+0000 7f9658c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9654147870 0x7f9654073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56932/0 (socket says 172.21.15.100:56932) 2024-09-17T04:27:53.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.842+0000 7f9658c00640 1 -- 172.21.15.100:0/1168413972 learned_addr learned my addr 172.21.15.100:0/1168413972 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:53.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.842+0000 7f9659600640 1 --2- 172.21.15.100:0/1168413972 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f963c0050c0 0x7f965415af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:53.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.842+0000 7f965a000640 1 --2- 172.21.15.100:0/1168413972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96540cbd40 0x7f9654073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:53.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.843+0000 7f9658c00640 1 -- 172.21.15.100:0/1168413972 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f963c0050c0 msgr2=0x7f965415af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:53.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.843+0000 7f9658c00640 1 --2- 172.21.15.100:0/1168413972 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f963c0050c0 0x7f965415af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:53.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.843+0000 7f9658c00640 1 -- 172.21.15.100:0/1168413972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96540cbd40 msgr2=0x7f9654073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:53.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.843+0000 7f9658c00640 1 --2- 172.21.15.100:0/1168413972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96540cbd40 0x7f9654073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:53.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.843+0000 7f9658c00640 1 -- 172.21.15.100:0/1168413972 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9654159e70 con 0x7f9654147870 2024-09-17T04:27:53.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.843+0000 7f965a000640 1 --2- 172.21.15.100:0/1168413972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96540cbd40 0x7f9654073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:27:53.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.843+0000 7f9658c00640 1 --2- 172.21.15.100:0/1168413972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9654147870 0x7f9654073040 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f96400066b0 tx=0x7f96400039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:53.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.843+0000 7f9651600640 1 -- 172.21.15.100:0/1168413972 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9640015070 con 0x7f9654147870 2024-09-17T04:27:53.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.843+0000 7f9651600640 1 -- 172.21.15.100:0/1168413972 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9640011900 con 0x7f9654147870 2024-09-17T04:27:53.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.843+0000 7f9651600640 1 -- 172.21.15.100:0/1168413972 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9640011d50 con 0x7f9654147870 2024-09-17T04:27:53.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.843+0000 7f965c000640 1 -- 172.21.15.100:0/1168413972 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f965415a150 con 0x7f9654147870 2024-09-17T04:27:53.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.843+0000 7f965c000640 1 -- 172.21.15.100:0/1168413972 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f965415a710 con 0x7f9654147870 2024-09-17T04:27:53.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.844+0000 7f965c000640 1 -- 172.21.15.100:0/1168413972 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f965415f3d0 con 0x7f9654147870 2024-09-17T04:27:53.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.846+0000 7f9651600640 1 -- 172.21.15.100:0/1168413972 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f9640018070 con 0x7f9654147870 2024-09-17T04:27:53.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.847+0000 7f9651600640 1 --2- 172.21.15.100:0/1168413972 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9634078d70 0x7f963407b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:53.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.847+0000 7f9651600640 1 -- 172.21.15.100:0/1168413972 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7f964009bd90 con 0x7f9654147870 2024-09-17T04:27:53.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.847+0000 7f9659600640 1 --2- 172.21.15.100:0/1168413972 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9634078d70 0x7f963407b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:53.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.848+0000 7f9659600640 1 --2- 172.21.15.100:0/1168413972 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9634078d70 0x7f963407b280 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9648003790 tx=0x7f9648006350 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:53.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:53.851+0000 7f9651600640 1 -- 172.21.15.100:0/1168413972 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f96400637f0 con 0x7f9654147870 2024-09-17T04:27:54.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.031+0000 7f965c000640 1 -- 172.21.15.100:0/1168413972 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9654073ac0 con 0x7f9634078d70 2024-09-17T04:27:54.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.037+0000 7f9651600640 1 -- 172.21.15.100:0/1168413972 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251896 (secure 0 0 0) 0x7f9648003d10 con 0x7f9634078d70 2024-09-17T04:27:54.039 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:27:54.041 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:27:54.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.040+0000 7f965c000640 1 -- 172.21.15.100:0/1168413972 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9634078d70 msgr2=0x7f963407b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:54.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.040+0000 7f965c000640 1 --2- 172.21.15.100:0/1168413972 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9634078d70 0x7f963407b280 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9648003790 tx=0x7f9648006350 comp rx=0 tx=0).stop 2024-09-17T04:27:54.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.040+0000 7f965c000640 1 -- 172.21.15.100:0/1168413972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9654147870 msgr2=0x7f9654073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:54.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.040+0000 7f965c000640 1 --2- 172.21.15.100:0/1168413972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9654147870 0x7f9654073040 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f96400066b0 tx=0x7f96400039c0 comp rx=0 tx=0).stop 2024-09-17T04:27:54.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.040+0000 7f965c000640 1 -- 172.21.15.100:0/1168413972 shutdown_connections 2024-09-17T04:27:54.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.040+0000 7f965c000640 1 --2- 172.21.15.100:0/1168413972 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f9634078d70 0x7f963407b280 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:54.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.040+0000 7f965c000640 1 --2- 172.21.15.100:0/1168413972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96540cbd40 0x7f9654073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:54.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.040+0000 7f965c000640 1 --2- 172.21.15.100:0/1168413972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9654147870 0x7f9654073040 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:54.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.040+0000 7f965c000640 1 --2- 172.21.15.100:0/1168413972 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f963c0050c0 0x7f965415af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:54.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.040+0000 7f965c000640 1 -- 172.21.15.100:0/1168413972 >> 172.21.15.100:0/1168413972 conn(0x7f965406d730 msgr2=0x7f965414e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:54.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.040+0000 7f965c000640 1 -- 172.21.15.100:0/1168413972 shutdown_connections 2024-09-17T04:27:54.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.041+0000 7f965c000640 1 -- 172.21.15.100:0/1168413972 wait complete. 2024-09-17T04:27:54.062 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-09-17T04:27:54.025979+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239748,"kb_used_data":6856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754734972,"statfs":{"total":773094113280,"available":772848611328,"internally_reserved":0,"allocated":7020544,"data_stored":3580484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":238276148},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.830275"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483699+0000","last_change":"2024-09-17T04:27:44.224649+0000","last_active":"2024-09-17T04:27:45.483699+0000","last_peered":"2024-09-17T04:27:45.483699+0000","last_clean":"2024-09-17T04:27:45.483699+0000","last_became_active":"2024-09-17T04:27:44.224334+0000","last_became_peered":"2024-09-17T04:27:44.224334+0000","last_unstale":"2024-09-17T04:27:45.483699+0000","last_undegraded":"2024-09-17T04:27:45.483699+0000","last_fullsized":"2024-09-17T04:27:45.483699+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:17.020497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212739+0000","last_change":"2024-09-17T04:27:44.238721+0000","last_active":"2024-09-17T04:27:45.212739+0000","last_peered":"2024-09-17T04:27:45.212739+0000","last_clean":"2024-09-17T04:27:45.212739+0000","last_became_active":"2024-09-17T04:27:44.237900+0000","last_became_peered":"2024-09-17T04:27:44.237900+0000","last_unstale":"2024-09-17T04:27:45.212739+0000","last_undegraded":"2024-09-17T04:27:45.212739+0000","last_fullsized":"2024-09-17T04:27:45.212739+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:09:57.201730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483850+0000","last_change":"2024-09-17T04:27:44.220473+0000","last_active":"2024-09-17T04:27:45.483850+0000","last_peered":"2024-09-17T04:27:45.483850+0000","last_clean":"2024-09-17T04:27:45.483850+0000","last_became_active":"2024-09-17T04:27:44.220013+0000","last_became_peered":"2024-09-17T04:27:44.220013+0000","last_unstale":"2024-09-17T04:27:45.483850+0000","last_undegraded":"2024-09-17T04:27:45.483850+0000","last_fullsized":"2024-09-17T04:27:45.483850+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:22.439743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.216564+0000","last_change":"2024-09-17T04:27:44.212039+0000","last_active":"2024-09-17T04:27:45.216564+0000","last_peered":"2024-09-17T04:27:45.216564+0000","last_clean":"2024-09-17T04:27:45.216564+0000","last_became_active":"2024-09-17T04:27:44.211689+0000","last_became_peered":"2024-09-17T04:27:44.211689+0000","last_unstale":"2024-09-17T04:27:45.216564+0000","last_undegraded":"2024-09-17T04:27:45.216564+0000","last_fullsized":"2024-09-17T04:27:45.216564+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:12:13.395203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483076+0000","last_change":"2024-09-17T04:27:44.203901+0000","last_active":"2024-09-17T04:27:45.483076+0000","last_peered":"2024-09-17T04:27:45.483076+0000","last_clean":"2024-09-17T04:27:45.483076+0000","last_became_active":"2024-09-17T04:27:44.203509+0000","last_became_peered":"2024-09-17T04:27:44.203509+0000","last_unstale":"2024-09-17T04:27:45.483076+0000","last_undegraded":"2024-09-17T04:27:45.483076+0000","last_fullsized":"2024-09-17T04:27:45.483076+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:55.077812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222259+0000","last_change":"2024-09-17T04:27:44.215787+0000","last_active":"2024-09-17T04:27:45.222259+0000","last_peered":"2024-09-17T04:27:45.222259+0000","last_clean":"2024-09-17T04:27:45.222259+0000","last_became_active":"2024-09-17T04:27:44.215468+0000","last_became_peered":"2024-09-17T04:27:44.215468+0000","last_unstale":"2024-09-17T04:27:45.222259+0000","last_undegraded":"2024-09-17T04:27:45.222259+0000","last_fullsized":"2024-09-17T04:27:45.222259+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:35.616439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.215959+0000","last_change":"2024-09-17T04:27:44.218294+0000","last_active":"2024-09-17T04:27:45.215959+0000","last_peered":"2024-09-17T04:27:45.215959+0000","last_clean":"2024-09-17T04:27:45.215959+0000","last_became_active":"2024-09-17T04:27:44.214481+0000","last_became_peered":"2024-09-17T04:27:44.214481+0000","last_unstale":"2024-09-17T04:27:45.215959+0000","last_undegraded":"2024-09-17T04:27:45.215959+0000","last_fullsized":"2024-09-17T04:27:45.215959+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:20.764786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212387+0000","last_change":"2024-09-17T04:27:44.222903+0000","last_active":"2024-09-17T04:27:45.212387+0000","last_peered":"2024-09-17T04:27:45.212387+0000","last_clean":"2024-09-17T04:27:45.212387+0000","last_became_active":"2024-09-17T04:27:44.222587+0000","last_became_peered":"2024-09-17T04:27:44.222587+0000","last_unstale":"2024-09-17T04:27:45.212387+0000","last_undegraded":"2024-09-17T04:27:45.212387+0000","last_fullsized":"2024-09-17T04:27:45.212387+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:37:39.457226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.216488+0000","last_change":"2024-09-17T04:27:44.218087+0000","last_active":"2024-09-17T04:27:45.216488+0000","last_peered":"2024-09-17T04:27:45.216488+0000","last_clean":"2024-09-17T04:27:45.216488+0000","last_became_active":"2024-09-17T04:27:44.216344+0000","last_became_peered":"2024-09-17T04:27:44.216344+0000","last_unstale":"2024-09-17T04:27:45.216488+0000","last_undegraded":"2024-09-17T04:27:45.216488+0000","last_fullsized":"2024-09-17T04:27:45.216488+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:48.140070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.484260+0000","last_change":"2024-09-17T04:27:44.222987+0000","last_active":"2024-09-17T04:27:45.484260+0000","last_peered":"2024-09-17T04:27:45.484260+0000","last_clean":"2024-09-17T04:27:45.484260+0000","last_became_active":"2024-09-17T04:27:44.222642+0000","last_became_peered":"2024-09-17T04:27:44.222642+0000","last_unstale":"2024-09-17T04:27:45.484260+0000","last_undegraded":"2024-09-17T04:27:45.484260+0000","last_fullsized":"2024-09-17T04:27:45.484260+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:26.116120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.484318+0000","last_change":"2024-09-17T04:27:44.234721+0000","last_active":"2024-09-17T04:27:45.484318+0000","last_peered":"2024-09-17T04:27:45.484318+0000","last_clean":"2024-09-17T04:27:45.484318+0000","last_became_active":"2024-09-17T04:27:44.233924+0000","last_became_peered":"2024-09-17T04:27:44.233924+0000","last_unstale":"2024-09-17T04:27:45.484318+0000","last_undegraded":"2024-09-17T04:27:45.484318+0000","last_fullsized":"2024-09-17T04:27:45.484318+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:44.967738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.216866+0000","last_change":"2024-09-17T04:27:44.215001+0000","last_active":"2024-09-17T04:27:45.216866+0000","last_peered":"2024-09-17T04:27:45.216866+0000","last_clean":"2024-09-17T04:27:45.216866+0000","last_became_active":"2024-09-17T04:27:44.214615+0000","last_became_peered":"2024-09-17T04:27:44.214615+0000","last_unstale":"2024-09-17T04:27:45.216866+0000","last_undegraded":"2024-09-17T04:27:45.216866+0000","last_fullsized":"2024-09-17T04:27:45.216866+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:15.359266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.225894+0000","last_change":"2024-09-17T04:27:44.227474+0000","last_active":"2024-09-17T04:27:45.225894+0000","last_peered":"2024-09-17T04:27:45.225894+0000","last_clean":"2024-09-17T04:27:45.225894+0000","last_became_active":"2024-09-17T04:27:44.227111+0000","last_became_peered":"2024-09-17T04:27:44.227111+0000","last_unstale":"2024-09-17T04:27:45.225894+0000","last_undegraded":"2024-09-17T04:27:45.225894+0000","last_fullsized":"2024-09-17T04:27:45.225894+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:58:30.716243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.216878+0000","last_change":"2024-09-17T04:27:44.212906+0000","last_active":"2024-09-17T04:27:45.216878+0000","last_peered":"2024-09-17T04:27:45.216878+0000","last_clean":"2024-09-17T04:27:45.216878+0000","last_became_active":"2024-09-17T04:27:44.212538+0000","last_became_peered":"2024-09-17T04:27:44.212538+0000","last_unstale":"2024-09-17T04:27:45.216878+0000","last_undegraded":"2024-09-17T04:27:45.216878+0000","last_fullsized":"2024-09-17T04:27:45.216878+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:08.736158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.216321+0000","last_change":"2024-09-17T04:27:44.234972+0000","last_active":"2024-09-17T04:27:45.216321+0000","last_peered":"2024-09-17T04:27:45.216321+0000","last_clean":"2024-09-17T04:27:45.216321+0000","last_became_active":"2024-09-17T04:27:44.234201+0000","last_became_peered":"2024-09-17T04:27:44.234201+0000","last_unstale":"2024-09-17T04:27:45.216321+0000","last_undegraded":"2024-09-17T04:27:45.216321+0000","last_fullsized":"2024-09-17T04:27:45.216321+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:54:00.667260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222741+0000","last_change":"2024-09-17T04:27:44.224641+0000","last_active":"2024-09-17T04:27:45.222741+0000","last_peered":"2024-09-17T04:27:45.222741+0000","last_clean":"2024-09-17T04:27:45.222741+0000","last_became_active":"2024-09-17T04:27:44.224324+0000","last_became_peered":"2024-09-17T04:27:44.224324+0000","last_unstale":"2024-09-17T04:27:45.222741+0000","last_undegraded":"2024-09-17T04:27:45.222741+0000","last_fullsized":"2024-09-17T04:27:45.222741+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:49.705694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212938+0000","last_change":"2024-09-17T04:27:44.221917+0000","last_active":"2024-09-17T04:27:45.212938+0000","last_peered":"2024-09-17T04:27:45.212938+0000","last_clean":"2024-09-17T04:27:45.212938+0000","last_became_active":"2024-09-17T04:27:44.221632+0000","last_became_peered":"2024-09-17T04:27:44.221632+0000","last_unstale":"2024-09-17T04:27:45.212938+0000","last_undegraded":"2024-09-17T04:27:45.212938+0000","last_fullsized":"2024-09-17T04:27:45.212938+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:27.954596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483971+0000","last_change":"2024-09-17T04:27:44.222680+0000","last_active":"2024-09-17T04:27:45.483971+0000","last_peered":"2024-09-17T04:27:45.483971+0000","last_clean":"2024-09-17T04:27:45.483971+0000","last_became_active":"2024-09-17T04:27:44.222082+0000","last_became_peered":"2024-09-17T04:27:44.222082+0000","last_unstale":"2024-09-17T04:27:45.483971+0000","last_undegraded":"2024-09-17T04:27:45.483971+0000","last_fullsized":"2024-09-17T04:27:45.483971+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:20:40.092936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483521+0000","last_change":"2024-09-17T04:27:44.223173+0000","last_active":"2024-09-17T04:27:45.483521+0000","last_peered":"2024-09-17T04:27:45.483521+0000","last_clean":"2024-09-17T04:27:45.483521+0000","last_became_active":"2024-09-17T04:27:44.222349+0000","last_became_peered":"2024-09-17T04:27:44.222349+0000","last_unstale":"2024-09-17T04:27:45.483521+0000","last_undegraded":"2024-09-17T04:27:45.483521+0000","last_fullsized":"2024-09-17T04:27:45.483521+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:45:47.530471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212533+0000","last_change":"2024-09-17T04:27:44.220909+0000","last_active":"2024-09-17T04:27:45.212533+0000","last_peered":"2024-09-17T04:27:45.212533+0000","last_clean":"2024-09-17T04:27:45.212533+0000","last_became_active":"2024-09-17T04:27:44.220605+0000","last_became_peered":"2024-09-17T04:27:44.220605+0000","last_unstale":"2024-09-17T04:27:45.212533+0000","last_undegraded":"2024-09-17T04:27:45.212533+0000","last_fullsized":"2024-09-17T04:27:45.212533+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:48:13.238235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483544+0000","last_change":"2024-09-17T04:27:44.217774+0000","last_active":"2024-09-17T04:27:45.483544+0000","last_peered":"2024-09-17T04:27:45.483544+0000","last_clean":"2024-09-17T04:27:45.483544+0000","last_became_active":"2024-09-17T04:27:44.217219+0000","last_became_peered":"2024-09-17T04:27:44.217219+0000","last_unstale":"2024-09-17T04:27:45.483544+0000","last_undegraded":"2024-09-17T04:27:45.483544+0000","last_fullsized":"2024-09-17T04:27:45.483544+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:15.359266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212746+0000","last_change":"2024-09-17T04:27:44.216830+0000","last_active":"2024-09-17T04:27:45.212746+0000","last_peered":"2024-09-17T04:27:45.212746+0000","last_clean":"2024-09-17T04:27:45.212746+0000","last_became_active":"2024-09-17T04:27:44.216520+0000","last_became_peered":"2024-09-17T04:27:44.216520+0000","last_unstale":"2024-09-17T04:27:45.212746+0000","last_undegraded":"2024-09-17T04:27:45.212746+0000","last_fullsized":"2024-09-17T04:27:45.212746+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:15:47.203299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.484100+0000","last_change":"2024-09-17T04:27:44.228366+0000","last_active":"2024-09-17T04:27:45.484100+0000","last_peered":"2024-09-17T04:27:45.484100+0000","last_clean":"2024-09-17T04:27:45.484100+0000","last_became_active":"2024-09-17T04:27:44.227967+0000","last_became_peered":"2024-09-17T04:27:44.227967+0000","last_unstale":"2024-09-17T04:27:45.484100+0000","last_undegraded":"2024-09-17T04:27:45.484100+0000","last_fullsized":"2024-09-17T04:27:45.484100+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:48.593633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.225723+0000","last_change":"2024-09-17T04:27:44.223208+0000","last_active":"2024-09-17T04:27:45.225723+0000","last_peered":"2024-09-17T04:27:45.225723+0000","last_clean":"2024-09-17T04:27:45.225723+0000","last_became_active":"2024-09-17T04:27:44.222867+0000","last_became_peered":"2024-09-17T04:27:44.222867+0000","last_unstale":"2024-09-17T04:27:45.225723+0000","last_undegraded":"2024-09-17T04:27:45.225723+0000","last_fullsized":"2024-09-17T04:27:45.225723+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:27.882897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483901+0000","last_change":"2024-09-17T04:27:44.218162+0000","last_active":"2024-09-17T04:27:45.483901+0000","last_peered":"2024-09-17T04:27:45.483901+0000","last_clean":"2024-09-17T04:27:45.483901+0000","last_became_active":"2024-09-17T04:27:44.217831+0000","last_became_peered":"2024-09-17T04:27:44.217831+0000","last_unstale":"2024-09-17T04:27:45.483901+0000","last_undegraded":"2024-09-17T04:27:45.483901+0000","last_fullsized":"2024-09-17T04:27:45.483901+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:17.403599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.217861+0000","last_change":"2024-09-17T04:27:44.206558+0000","last_active":"2024-09-17T04:27:45.217861+0000","last_peered":"2024-09-17T04:27:45.217861+0000","last_clean":"2024-09-17T04:27:45.217861+0000","last_became_active":"2024-09-17T04:27:44.206253+0000","last_became_peered":"2024-09-17T04:27:44.206253+0000","last_unstale":"2024-09-17T04:27:45.217861+0000","last_undegraded":"2024-09-17T04:27:45.217861+0000","last_fullsized":"2024-09-17T04:27:45.217861+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:14.532539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222481+0000","last_change":"2024-09-17T04:27:44.221446+0000","last_active":"2024-09-17T04:27:45.222481+0000","last_peered":"2024-09-17T04:27:45.222481+0000","last_clean":"2024-09-17T04:27:45.222481+0000","last_became_active":"2024-09-17T04:27:44.221111+0000","last_became_peered":"2024-09-17T04:27:44.221111+0000","last_unstale":"2024-09-17T04:27:45.222481+0000","last_undegraded":"2024-09-17T04:27:45.222481+0000","last_fullsized":"2024-09-17T04:27:45.222481+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:10.625074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483709+0000","last_change":"2024-09-17T04:27:44.211926+0000","last_active":"2024-09-17T04:27:45.483709+0000","last_peered":"2024-09-17T04:27:45.483709+0000","last_clean":"2024-09-17T04:27:45.483709+0000","last_became_active":"2024-09-17T04:27:44.211608+0000","last_became_peered":"2024-09-17T04:27:44.211608+0000","last_unstale":"2024-09-17T04:27:45.483709+0000","last_undegraded":"2024-09-17T04:27:45.483709+0000","last_fullsized":"2024-09-17T04:27:45.483709+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:18:10.368163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.213050+0000","last_change":"2024-09-17T04:27:44.218277+0000","last_active":"2024-09-17T04:27:45.213050+0000","last_peered":"2024-09-17T04:27:45.213050+0000","last_clean":"2024-09-17T04:27:45.213050+0000","last_became_active":"2024-09-17T04:27:44.217942+0000","last_became_peered":"2024-09-17T04:27:44.217942+0000","last_unstale":"2024-09-17T04:27:45.213050+0000","last_undegraded":"2024-09-17T04:27:45.213050+0000","last_fullsized":"2024-09-17T04:27:45.213050+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:18.353865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483703+0000","last_change":"2024-09-17T04:27:44.220878+0000","last_active":"2024-09-17T04:27:45.483703+0000","last_peered":"2024-09-17T04:27:45.483703+0000","last_clean":"2024-09-17T04:27:45.483703+0000","last_became_active":"2024-09-17T04:27:44.220534+0000","last_became_peered":"2024-09-17T04:27:44.220534+0000","last_unstale":"2024-09-17T04:27:45.483703+0000","last_undegraded":"2024-09-17T04:27:45.483703+0000","last_fullsized":"2024-09-17T04:27:45.483703+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:49.106024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.482885+0000","last_change":"2024-09-17T04:27:44.223625+0000","last_active":"2024-09-17T04:27:45.482885+0000","last_peered":"2024-09-17T04:27:45.482885+0000","last_clean":"2024-09-17T04:27:45.482885+0000","last_became_active":"2024-09-17T04:27:44.220587+0000","last_became_peered":"2024-09-17T04:27:44.220587+0000","last_unstale":"2024-09-17T04:27:45.482885+0000","last_undegraded":"2024-09-17T04:27:45.482885+0000","last_fullsized":"2024-09-17T04:27:45.482885+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:44:05.407748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222393+0000","last_change":"2024-09-17T04:27:44.228534+0000","last_active":"2024-09-17T04:27:45.222393+0000","last_peered":"2024-09-17T04:27:45.222393+0000","last_clean":"2024-09-17T04:27:45.222393+0000","last_became_active":"2024-09-17T04:27:44.227779+0000","last_became_peered":"2024-09-17T04:27:44.227779+0000","last_unstale":"2024-09-17T04:27:45.222393+0000","last_undegraded":"2024-09-17T04:27:45.222393+0000","last_fullsized":"2024-09-17T04:27:45.222393+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:45:22.653237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222475+0000","last_change":"2024-09-17T04:27:44.215232+0000","last_active":"2024-09-17T04:27:45.222475+0000","last_peered":"2024-09-17T04:27:45.222475+0000","last_clean":"2024-09-17T04:27:45.222475+0000","last_became_active":"2024-09-17T04:27:44.214803+0000","last_became_peered":"2024-09-17T04:27:44.214803+0000","last_unstale":"2024-09-17T04:27:45.222475+0000","last_undegraded":"2024-09-17T04:27:45.222475+0000","last_fullsized":"2024-09-17T04:27:45.222475+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:48.593633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"60'4","reported_seq":19,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222849+0000","last_change":"2024-09-17T04:27:43.200387+0000","last_active":"2024-09-17T04:27:45.222849+0000","last_peered":"2024-09-17T04:27:45.222849+0000","last_clean":"2024-09-17T04:27:45.222849+0000","last_became_active":"2024-09-17T04:27:43.199763+0000","last_became_peered":"2024-09-17T04:27:43.199763+0000","last_unstale":"2024-09-17T04:27:45.222849+0000","last_undegraded":"2024-09-17T04:27:45.222849+0000","last_fullsized":"2024-09-17T04:27:45.222849+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:19.758401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.484326+0000","last_change":"2024-09-17T04:27:44.216650+0000","last_active":"2024-09-17T04:27:45.484326+0000","last_peered":"2024-09-17T04:27:45.484326+0000","last_clean":"2024-09-17T04:27:45.484326+0000","last_became_active":"2024-09-17T04:27:44.216237+0000","last_became_peered":"2024-09-17T04:27:44.216237+0000","last_unstale":"2024-09-17T04:27:45.484326+0000","last_undegraded":"2024-09-17T04:27:45.484326+0000","last_fullsized":"2024-09-17T04:27:45.484326+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:37.241286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"60'1","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212757+0000","last_change":"2024-09-17T04:27:43.206766+0000","last_active":"2024-09-17T04:27:45.212757+0000","last_peered":"2024-09-17T04:27:45.212757+0000","last_clean":"2024-09-17T04:27:45.212757+0000","last_became_active":"2024-09-17T04:27:43.206455+0000","last_became_peered":"2024-09-17T04:27:43.206455+0000","last_unstale":"2024-09-17T04:27:45.212757+0000","last_undegraded":"2024-09-17T04:27:45.212757+0000","last_fullsized":"2024-09-17T04:27:45.212757+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:06:22.067347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222393+0000","last_change":"2024-09-17T04:27:44.230666+0000","last_active":"2024-09-17T04:27:45.222393+0000","last_peered":"2024-09-17T04:27:45.222393+0000","last_clean":"2024-09-17T04:27:45.222393+0000","last_became_active":"2024-09-17T04:27:44.230350+0000","last_became_peered":"2024-09-17T04:27:44.230350+0000","last_unstale":"2024-09-17T04:27:45.222393+0000","last_undegraded":"2024-09-17T04:27:45.222393+0000","last_fullsized":"2024-09-17T04:27:45.222393+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:14.532539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"60'2","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222622+0000","last_change":"2024-09-17T04:27:43.201555+0000","last_active":"2024-09-17T04:27:45.222622+0000","last_peered":"2024-09-17T04:27:45.222622+0000","last_clean":"2024-09-17T04:27:45.222622+0000","last_became_active":"2024-09-17T04:27:43.200837+0000","last_became_peered":"2024-09-17T04:27:43.200837+0000","last_unstale":"2024-09-17T04:27:45.222622+0000","last_undegraded":"2024-09-17T04:27:45.222622+0000","last_fullsized":"2024-09-17T04:27:45.222622+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:59.660874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222582+0000","last_change":"2024-09-17T04:27:44.219425+0000","last_active":"2024-09-17T04:27:45.222582+0000","last_peered":"2024-09-17T04:27:45.222582+0000","last_clean":"2024-09-17T04:27:45.222582+0000","last_became_active":"2024-09-17T04:27:44.217358+0000","last_became_peered":"2024-09-17T04:27:44.217358+0000","last_unstale":"2024-09-17T04:27:45.222582+0000","last_undegraded":"2024-09-17T04:27:45.222582+0000","last_fullsized":"2024-09-17T04:27:45.222582+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:23:10.166526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"60'2","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.217052+0000","last_change":"2024-09-17T04:27:43.207037+0000","last_active":"2024-09-17T04:27:45.217052+0000","last_peered":"2024-09-17T04:27:45.217052+0000","last_clean":"2024-09-17T04:27:45.217052+0000","last_became_active":"2024-09-17T04:27:43.206672+0000","last_became_peered":"2024-09-17T04:27:43.206672+0000","last_unstale":"2024-09-17T04:27:45.217052+0000","last_undegraded":"2024-09-17T04:27:45.217052+0000","last_fullsized":"2024-09-17T04:27:45.217052+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:47.133684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.213106+0000","last_change":"2024-09-17T04:27:44.217751+0000","last_active":"2024-09-17T04:27:45.213106+0000","last_peered":"2024-09-17T04:27:45.213106+0000","last_clean":"2024-09-17T04:27:45.213106+0000","last_became_active":"2024-09-17T04:27:44.214072+0000","last_became_peered":"2024-09-17T04:27:44.214072+0000","last_unstale":"2024-09-17T04:27:45.213106+0000","last_undegraded":"2024-09-17T04:27:45.213106+0000","last_fullsized":"2024-09-17T04:27:45.213106+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:27.160546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"60'1","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.215965+0000","last_change":"2024-09-17T04:27:43.198922+0000","last_active":"2024-09-17T04:27:45.215965+0000","last_peered":"2024-09-17T04:27:45.215965+0000","last_clean":"2024-09-17T04:27:45.215965+0000","last_became_active":"2024-09-17T04:27:43.198453+0000","last_became_peered":"2024-09-17T04:27:43.198453+0000","last_unstale":"2024-09-17T04:27:45.215965+0000","last_undegraded":"2024-09-17T04:27:45.215965+0000","last_fullsized":"2024-09-17T04:27:45.215965+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:43.961353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212477+0000","last_change":"2024-09-17T04:27:44.222601+0000","last_active":"2024-09-17T04:27:45.212477+0000","last_peered":"2024-09-17T04:27:45.212477+0000","last_clean":"2024-09-17T04:27:45.212477+0000","last_became_active":"2024-09-17T04:27:44.221587+0000","last_became_peered":"2024-09-17T04:27:44.221587+0000","last_unstale":"2024-09-17T04:27:45.212477+0000","last_undegraded":"2024-09-17T04:27:45.212477+0000","last_fullsized":"2024-09-17T04:27:45.212477+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:22.587679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"60'1","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222375+0000","last_change":"2024-09-17T04:27:43.207536+0000","last_active":"2024-09-17T04:27:45.222375+0000","last_peered":"2024-09-17T04:27:45.222375+0000","last_clean":"2024-09-17T04:27:45.222375+0000","last_became_active":"2024-09-17T04:27:43.206356+0000","last_became_peered":"2024-09-17T04:27:43.206356+0000","last_unstale":"2024-09-17T04:27:45.222375+0000","last_undegraded":"2024-09-17T04:27:45.222375+0000","last_fullsized":"2024-09-17T04:27:45.222375+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:47.133684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.225791+0000","last_change":"2024-09-17T04:27:44.220110+0000","last_active":"2024-09-17T04:27:45.225791+0000","last_peered":"2024-09-17T04:27:45.225791+0000","last_clean":"2024-09-17T04:27:45.225791+0000","last_became_active":"2024-09-17T04:27:44.219708+0000","last_became_peered":"2024-09-17T04:27:44.219708+0000","last_unstale":"2024-09-17T04:27:45.225791+0000","last_undegraded":"2024-09-17T04:27:45.225791+0000","last_fullsized":"2024-09-17T04:27:45.225791+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:32:28.733174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.216093+0000","last_change":"2024-09-17T04:27:43.200857+0000","last_active":"2024-09-17T04:27:45.216093+0000","last_peered":"2024-09-17T04:27:45.216093+0000","last_clean":"2024-09-17T04:27:45.216093+0000","last_became_active":"2024-09-17T04:27:43.200518+0000","last_became_peered":"2024-09-17T04:27:43.200518+0000","last_unstale":"2024-09-17T04:27:45.216093+0000","last_undegraded":"2024-09-17T04:27:45.216093+0000","last_fullsized":"2024-09-17T04:27:45.216093+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:00:58.429429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212733+0000","last_change":"2024-09-17T04:27:44.225414+0000","last_active":"2024-09-17T04:27:45.212733+0000","last_peered":"2024-09-17T04:27:45.212733+0000","last_clean":"2024-09-17T04:27:45.212733+0000","last_became_active":"2024-09-17T04:27:44.225112+0000","last_became_peered":"2024-09-17T04:27:44.225112+0000","last_unstale":"2024-09-17T04:27:45.212733+0000","last_undegraded":"2024-09-17T04:27:45.212733+0000","last_fullsized":"2024-09-17T04:27:45.212733+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:05:18.890521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.217327+0000","last_change":"2024-09-17T04:27:43.209713+0000","last_active":"2024-09-17T04:27:45.217327+0000","last_peered":"2024-09-17T04:27:45.217327+0000","last_clean":"2024-09-17T04:27:45.217327+0000","last_became_active":"2024-09-17T04:27:43.209316+0000","last_became_peered":"2024-09-17T04:27:43.209316+0000","last_unstale":"2024-09-17T04:27:45.217327+0000","last_undegraded":"2024-09-17T04:27:45.217327+0000","last_fullsized":"2024-09-17T04:27:45.217327+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:11.559902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.217008+0000","last_change":"2024-09-17T04:27:44.237993+0000","last_active":"2024-09-17T04:27:45.217008+0000","last_peered":"2024-09-17T04:27:45.217008+0000","last_clean":"2024-09-17T04:27:45.217008+0000","last_became_active":"2024-09-17T04:27:44.237721+0000","last_became_peered":"2024-09-17T04:27:44.237721+0000","last_unstale":"2024-09-17T04:27:45.217008+0000","last_undegraded":"2024-09-17T04:27:45.217008+0000","last_fullsized":"2024-09-17T04:27:45.217008+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:30.929757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.484271+0000","last_change":"2024-09-17T04:27:43.204007+0000","last_active":"2024-09-17T04:27:45.484271+0000","last_peered":"2024-09-17T04:27:45.484271+0000","last_clean":"2024-09-17T04:27:45.484271+0000","last_became_active":"2024-09-17T04:27:43.203747+0000","last_became_peered":"2024-09-17T04:27:43.203747+0000","last_unstale":"2024-09-17T04:27:45.484271+0000","last_undegraded":"2024-09-17T04:27:45.484271+0000","last_fullsized":"2024-09-17T04:27:45.484271+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:02.426588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.484295+0000","last_change":"2024-09-17T04:27:44.238312+0000","last_active":"2024-09-17T04:27:45.484295+0000","last_peered":"2024-09-17T04:27:45.484295+0000","last_clean":"2024-09-17T04:27:45.484295+0000","last_became_active":"2024-09-17T04:27:44.237930+0000","last_became_peered":"2024-09-17T04:27:44.237930+0000","last_unstale":"2024-09-17T04:27:45.484295+0000","last_undegraded":"2024-09-17T04:27:45.484295+0000","last_fullsized":"2024-09-17T04:27:45.484295+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:16.774330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212854+0000","last_change":"2024-09-17T04:27:43.201502+0000","last_active":"2024-09-17T04:27:45.212854+0000","last_peered":"2024-09-17T04:27:45.212854+0000","last_clean":"2024-09-17T04:27:45.212854+0000","last_became_active":"2024-09-17T04:27:43.201071+0000","last_became_peered":"2024-09-17T04:27:43.201071+0000","last_unstale":"2024-09-17T04:27:45.212854+0000","last_undegraded":"2024-09-17T04:27:45.212854+0000","last_fullsized":"2024-09-17T04:27:45.212854+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:48:58.744170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483780+0000","last_change":"2024-09-17T04:27:44.226655+0000","last_active":"2024-09-17T04:27:45.483780+0000","last_peered":"2024-09-17T04:27:45.483780+0000","last_clean":"2024-09-17T04:27:45.483780+0000","last_became_active":"2024-09-17T04:27:44.226363+0000","last_became_peered":"2024-09-17T04:27:44.226363+0000","last_unstale":"2024-09-17T04:27:45.483780+0000","last_undegraded":"2024-09-17T04:27:45.483780+0000","last_fullsized":"2024-09-17T04:27:45.483780+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:43:25.897835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"60'3","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.484600+0000","last_change":"2024-09-17T04:27:43.203357+0000","last_active":"2024-09-17T04:27:45.484600+0000","last_peered":"2024-09-17T04:27:45.484600+0000","last_clean":"2024-09-17T04:27:45.484600+0000","last_became_active":"2024-09-17T04:27:43.202999+0000","last_became_peered":"2024-09-17T04:27:43.202999+0000","last_unstale":"2024-09-17T04:27:45.484600+0000","last_undegraded":"2024-09-17T04:27:45.484600+0000","last_fullsized":"2024-09-17T04:27:45.484600+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:58.011654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.484428+0000","last_change":"2024-09-17T04:27:44.240301+0000","last_active":"2024-09-17T04:27:45.484428+0000","last_peered":"2024-09-17T04:27:45.484428+0000","last_clean":"2024-09-17T04:27:45.484428+0000","last_became_active":"2024-09-17T04:27:44.237792+0000","last_became_peered":"2024-09-17T04:27:44.237792+0000","last_unstale":"2024-09-17T04:27:45.484428+0000","last_undegraded":"2024-09-17T04:27:45.484428+0000","last_fullsized":"2024-09-17T04:27:45.484428+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:36.313761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"60'6","reported_seq":21,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.216322+0000","last_change":"2024-09-17T04:27:43.200549+0000","last_active":"2024-09-17T04:27:45.216322+0000","last_peered":"2024-09-17T04:27:45.216322+0000","last_clean":"2024-09-17T04:27:45.216322+0000","last_became_active":"2024-09-17T04:27:43.200245+0000","last_became_peered":"2024-09-17T04:27:43.200245+0000","last_unstale":"2024-09-17T04:27:45.216322+0000","last_undegraded":"2024-09-17T04:27:45.216322+0000","last_fullsized":"2024-09-17T04:27:45.216322+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:36.234900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.216656+0000","last_change":"2024-09-17T04:27:44.225782+0000","last_active":"2024-09-17T04:27:45.216656+0000","last_peered":"2024-09-17T04:27:45.216656+0000","last_clean":"2024-09-17T04:27:45.216656+0000","last_became_active":"2024-09-17T04:27:44.225436+0000","last_became_peered":"2024-09-17T04:27:44.225436+0000","last_unstale":"2024-09-17T04:27:45.216656+0000","last_undegraded":"2024-09-17T04:27:45.216656+0000","last_fullsized":"2024-09-17T04:27:45.216656+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:49.106024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"60'2","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212414+0000","last_change":"2024-09-17T04:27:43.204124+0000","last_active":"2024-09-17T04:27:45.212414+0000","last_peered":"2024-09-17T04:27:45.212414+0000","last_clean":"2024-09-17T04:27:45.212414+0000","last_became_active":"2024-09-17T04:27:43.203730+0000","last_became_peered":"2024-09-17T04:27:43.203730+0000","last_unstale":"2024-09-17T04:27:45.212414+0000","last_undegraded":"2024-09-17T04:27:45.212414+0000","last_fullsized":"2024-09-17T04:27:45.212414+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:15:50.701517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483825+0000","last_change":"2024-09-17T04:27:44.219797+0000","last_active":"2024-09-17T04:27:45.483825+0000","last_peered":"2024-09-17T04:27:45.483825+0000","last_clean":"2024-09-17T04:27:45.483825+0000","last_became_active":"2024-09-17T04:27:44.219229+0000","last_became_peered":"2024-09-17T04:27:44.219229+0000","last_unstale":"2024-09-17T04:27:45.483825+0000","last_undegraded":"2024-09-17T04:27:45.483825+0000","last_fullsized":"2024-09-17T04:27:45.483825+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:34.529555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.225532+0000","last_change":"2024-09-17T04:27:43.191819+0000","last_active":"2024-09-17T04:27:45.225532+0000","last_peered":"2024-09-17T04:27:45.225532+0000","last_clean":"2024-09-17T04:27:45.225532+0000","last_became_active":"2024-09-17T04:27:43.191432+0000","last_became_peered":"2024-09-17T04:27:43.191432+0000","last_unstale":"2024-09-17T04:27:45.225532+0000","last_undegraded":"2024-09-17T04:27:45.225532+0000","last_fullsized":"2024-09-17T04:27:45.225532+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:46:51.373799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.215756+0000","last_change":"2024-09-17T04:27:44.216494+0000","last_active":"2024-09-17T04:27:45.215756+0000","last_peered":"2024-09-17T04:27:45.215756+0000","last_clean":"2024-09-17T04:27:45.215756+0000","last_became_active":"2024-09-17T04:27:44.216164+0000","last_became_peered":"2024-09-17T04:27:44.216164+0000","last_unstale":"2024-09-17T04:27:45.215756+0000","last_undegraded":"2024-09-17T04:27:45.215756+0000","last_fullsized":"2024-09-17T04:27:45.215756+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:00:59.435815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.216251+0000","last_change":"2024-09-17T04:27:43.206779+0000","last_active":"2024-09-17T04:27:45.216251+0000","last_peered":"2024-09-17T04:27:45.216251+0000","last_clean":"2024-09-17T04:27:45.216251+0000","last_became_active":"2024-09-17T04:27:43.205945+0000","last_became_peered":"2024-09-17T04:27:43.205945+0000","last_unstale":"2024-09-17T04:27:45.216251+0000","last_undegraded":"2024-09-17T04:27:45.216251+0000","last_fullsized":"2024-09-17T04:27:45.216251+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:59.660874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483883+0000","last_change":"2024-09-17T04:27:44.228358+0000","last_active":"2024-09-17T04:27:45.483883+0000","last_peered":"2024-09-17T04:27:45.483883+0000","last_clean":"2024-09-17T04:27:45.483883+0000","last_became_active":"2024-09-17T04:27:44.227977+0000","last_became_peered":"2024-09-17T04:27:44.227977+0000","last_unstale":"2024-09-17T04:27:45.483883+0000","last_undegraded":"2024-09-17T04:27:45.483883+0000","last_fullsized":"2024-09-17T04:27:45.483883+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:40.580708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"60'8","reported_seq":23,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483542+0000","last_change":"2024-09-17T04:27:43.203507+0000","last_active":"2024-09-17T04:27:45.483542+0000","last_peered":"2024-09-17T04:27:45.483542+0000","last_clean":"2024-09-17T04:27:45.483542+0000","last_became_active":"2024-09-17T04:27:43.202223+0000","last_became_peered":"2024-09-17T04:27:43.202223+0000","last_unstale":"2024-09-17T04:27:45.483542+0000","last_undegraded":"2024-09-17T04:27:45.483542+0000","last_fullsized":"2024-09-17T04:27:45.483542+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:25.714760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.484101+0000","last_change":"2024-09-17T04:27:44.221929+0000","last_active":"2024-09-17T04:27:45.484101+0000","last_peered":"2024-09-17T04:27:45.484101+0000","last_clean":"2024-09-17T04:27:45.484101+0000","last_became_active":"2024-09-17T04:27:44.221064+0000","last_became_peered":"2024-09-17T04:27:44.221064+0000","last_unstale":"2024-09-17T04:27:45.484101+0000","last_undegraded":"2024-09-17T04:27:45.484101+0000","last_fullsized":"2024-09-17T04:27:45.484101+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:18:43.380081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"60'1","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.213160+0000","last_change":"2024-09-17T04:27:43.206771+0000","last_active":"2024-09-17T04:27:45.213160+0000","last_peered":"2024-09-17T04:27:45.213160+0000","last_clean":"2024-09-17T04:27:45.213160+0000","last_became_active":"2024-09-17T04:27:43.206481+0000","last_became_peered":"2024-09-17T04:27:43.206481+0000","last_unstale":"2024-09-17T04:27:45.213160+0000","last_undegraded":"2024-09-17T04:27:45.213160+0000","last_fullsized":"2024-09-17T04:27:45.213160+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:48:49.832510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.217874+0000","last_change":"2024-09-17T04:27:44.218916+0000","last_active":"2024-09-17T04:27:45.217874+0000","last_peered":"2024-09-17T04:27:45.217874+0000","last_clean":"2024-09-17T04:27:45.217874+0000","last_became_active":"2024-09-17T04:27:44.218602+0000","last_became_peered":"2024-09-17T04:27:44.218602+0000","last_unstale":"2024-09-17T04:27:45.217874+0000","last_undegraded":"2024-09-17T04:27:45.217874+0000","last_fullsized":"2024-09-17T04:27:45.217874+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:20:40.092936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"60'2","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222554+0000","last_change":"2024-09-17T04:27:43.205128+0000","last_active":"2024-09-17T04:27:45.222554+0000","last_peered":"2024-09-17T04:27:45.222554+0000","last_clean":"2024-09-17T04:27:45.222554+0000","last_became_active":"2024-09-17T04:27:43.204836+0000","last_became_peered":"2024-09-17T04:27:43.204836+0000","last_unstale":"2024-09-17T04:27:45.222554+0000","last_undegraded":"2024-09-17T04:27:45.222554+0000","last_fullsized":"2024-09-17T04:27:45.222554+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:07.729772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212603+0000","last_change":"2024-09-17T04:27:44.222455+0000","last_active":"2024-09-17T04:27:45.212603+0000","last_peered":"2024-09-17T04:27:45.212603+0000","last_clean":"2024-09-17T04:27:45.212603+0000","last_became_active":"2024-09-17T04:27:44.222015+0000","last_became_peered":"2024-09-17T04:27:44.222015+0000","last_unstale":"2024-09-17T04:27:45.212603+0000","last_undegraded":"2024-09-17T04:27:45.212603+0000","last_fullsized":"2024-09-17T04:27:45.212603+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:01:05.264040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"60'1","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.217724+0000","last_change":"2024-09-17T04:27:43.207246+0000","last_active":"2024-09-17T04:27:45.217724+0000","last_peered":"2024-09-17T04:27:45.217724+0000","last_clean":"2024-09-17T04:27:45.217724+0000","last_became_active":"2024-09-17T04:27:43.206835+0000","last_became_peered":"2024-09-17T04:27:43.206835+0000","last_unstale":"2024-09-17T04:27:45.217724+0000","last_undegraded":"2024-09-17T04:27:45.217724+0000","last_fullsized":"2024-09-17T04:27:45.217724+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:19.758401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.484447+0000","last_change":"2024-09-17T04:27:44.238294+0000","last_active":"2024-09-17T04:27:45.484447+0000","last_peered":"2024-09-17T04:27:45.484447+0000","last_clean":"2024-09-17T04:27:45.484447+0000","last_became_active":"2024-09-17T04:27:44.237968+0000","last_became_peered":"2024-09-17T04:27:44.237968+0000","last_unstale":"2024-09-17T04:27:45.484447+0000","last_undegraded":"2024-09-17T04:27:45.484447+0000","last_fullsized":"2024-09-17T04:27:45.484447+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:49.705694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222471+0000","last_change":"2024-09-17T04:27:43.209312+0000","last_active":"2024-09-17T04:27:45.222471+0000","last_peered":"2024-09-17T04:27:45.222471+0000","last_clean":"2024-09-17T04:27:45.222471+0000","last_became_active":"2024-09-17T04:27:43.208984+0000","last_became_peered":"2024-09-17T04:27:43.208984+0000","last_unstale":"2024-09-17T04:27:45.222471+0000","last_undegraded":"2024-09-17T04:27:45.222471+0000","last_fullsized":"2024-09-17T04:27:45.222471+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:00:58.429429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.216952+0000","last_change":"2024-09-17T04:27:44.218883+0000","last_active":"2024-09-17T04:27:45.216952+0000","last_peered":"2024-09-17T04:27:45.216952+0000","last_clean":"2024-09-17T04:27:45.216952+0000","last_became_active":"2024-09-17T04:27:44.218549+0000","last_became_peered":"2024-09-17T04:27:44.218549+0000","last_unstale":"2024-09-17T04:27:45.216952+0000","last_undegraded":"2024-09-17T04:27:45.216952+0000","last_fullsized":"2024-09-17T04:27:45.216952+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:34.529555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"60'2","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212605+0000","last_change":"2024-09-17T04:27:43.203669+0000","last_active":"2024-09-17T04:27:45.212605+0000","last_peered":"2024-09-17T04:27:45.212605+0000","last_clean":"2024-09-17T04:27:45.212605+0000","last_became_active":"2024-09-17T04:27:43.203307+0000","last_became_peered":"2024-09-17T04:27:43.203307+0000","last_unstale":"2024-09-17T04:27:45.212605+0000","last_undegraded":"2024-09-17T04:27:45.212605+0000","last_fullsized":"2024-09-17T04:27:45.212605+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:53.295497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222398+0000","last_change":"2024-09-17T04:27:44.222562+0000","last_active":"2024-09-17T04:27:45.222398+0000","last_peered":"2024-09-17T04:27:45.222398+0000","last_clean":"2024-09-17T04:27:45.222398+0000","last_became_active":"2024-09-17T04:27:44.221122+0000","last_became_peered":"2024-09-17T04:27:44.221122+0000","last_unstale":"2024-09-17T04:27:45.222398+0000","last_undegraded":"2024-09-17T04:27:45.222398+0000","last_fullsized":"2024-09-17T04:27:45.222398+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:40.580708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483419+0000","last_change":"2024-09-17T04:27:43.189344+0000","last_active":"2024-09-17T04:27:45.483419+0000","last_peered":"2024-09-17T04:27:45.483419+0000","last_clean":"2024-09-17T04:27:45.483419+0000","last_became_active":"2024-09-17T04:27:43.189023+0000","last_became_peered":"2024-09-17T04:27:43.189023+0000","last_unstale":"2024-09-17T04:27:45.483419+0000","last_undegraded":"2024-09-17T04:27:45.483419+0000","last_fullsized":"2024-09-17T04:27:45.483419+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:07.652705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222197+0000","last_change":"2024-09-17T04:27:44.216597+0000","last_active":"2024-09-17T04:27:45.222197+0000","last_peered":"2024-09-17T04:27:45.222197+0000","last_clean":"2024-09-17T04:27:45.222197+0000","last_became_active":"2024-09-17T04:27:44.216253+0000","last_became_peered":"2024-09-17T04:27:44.216253+0000","last_unstale":"2024-09-17T04:27:45.222197+0000","last_undegraded":"2024-09-17T04:27:45.222197+0000","last_fullsized":"2024-09-17T04:27:45.222197+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:04.948283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.216031+0000","last_change":"2024-09-17T04:27:43.209348+0000","last_active":"2024-09-17T04:27:45.216031+0000","last_peered":"2024-09-17T04:27:45.216031+0000","last_clean":"2024-09-17T04:27:45.216031+0000","last_became_active":"2024-09-17T04:27:43.209055+0000","last_became_peered":"2024-09-17T04:27:43.209055+0000","last_unstale":"2024-09-17T04:27:45.216031+0000","last_undegraded":"2024-09-17T04:27:45.216031+0000","last_fullsized":"2024-09-17T04:27:45.216031+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:37:42.378184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.217864+0000","last_change":"2024-09-17T04:27:44.213695+0000","last_active":"2024-09-17T04:27:45.217864+0000","last_peered":"2024-09-17T04:27:45.217864+0000","last_clean":"2024-09-17T04:27:45.217864+0000","last_became_active":"2024-09-17T04:27:44.213346+0000","last_became_peered":"2024-09-17T04:27:44.213346+0000","last_unstale":"2024-09-17T04:27:45.217864+0000","last_undegraded":"2024-09-17T04:27:45.217864+0000","last_fullsized":"2024-09-17T04:27:45.217864+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:22.439743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"60'2","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.484268+0000","last_change":"2024-09-17T04:27:43.204645+0000","last_active":"2024-09-17T04:27:45.484268+0000","last_peered":"2024-09-17T04:27:45.484268+0000","last_clean":"2024-09-17T04:27:45.484268+0000","last_became_active":"2024-09-17T04:27:43.204334+0000","last_became_peered":"2024-09-17T04:27:43.204334+0000","last_unstale":"2024-09-17T04:27:45.484268+0000","last_undegraded":"2024-09-17T04:27:45.484268+0000","last_fullsized":"2024-09-17T04:27:45.484268+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:13.461537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.218051+0000","last_change":"2024-09-17T04:27:44.214273+0000","last_active":"2024-09-17T04:27:45.218051+0000","last_peered":"2024-09-17T04:27:45.218051+0000","last_clean":"2024-09-17T04:27:45.218051+0000","last_became_active":"2024-09-17T04:27:44.213931+0000","last_became_peered":"2024-09-17T04:27:44.213931+0000","last_unstale":"2024-09-17T04:27:45.218051+0000","last_undegraded":"2024-09-17T04:27:45.218051+0000","last_fullsized":"2024-09-17T04:27:45.218051+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:58:10.407213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"61'1","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222644+0000","last_change":"2024-09-17T04:27:43.206910+0000","last_active":"2024-09-17T04:27:45.222644+0000","last_peered":"2024-09-17T04:27:45.222644+0000","last_clean":"2024-09-17T04:27:45.222644+0000","last_became_active":"2024-09-17T04:27:43.206607+0000","last_became_peered":"2024-09-17T04:27:43.206607+0000","last_unstale":"2024-09-17T04:27:45.222644+0000","last_undegraded":"2024-09-17T04:27:45.222644+0000","last_fullsized":"2024-09-17T04:27:45.222644+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:29.923371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.484054+0000","last_change":"2024-09-17T04:27:44.229969+0000","last_active":"2024-09-17T04:27:45.484054+0000","last_peered":"2024-09-17T04:27:45.484054+0000","last_clean":"2024-09-17T04:27:45.484054+0000","last_became_active":"2024-09-17T04:27:44.229665+0000","last_became_peered":"2024-09-17T04:27:44.229665+0000","last_unstale":"2024-09-17T04:27:45.484054+0000","last_undegraded":"2024-09-17T04:27:45.484054+0000","last_fullsized":"2024-09-17T04:27:45.484054+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:02.779912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.225738+0000","last_change":"2024-09-17T04:27:43.200897+0000","last_active":"2024-09-17T04:27:45.225738+0000","last_peered":"2024-09-17T04:27:45.225738+0000","last_clean":"2024-09-17T04:27:45.225738+0000","last_became_active":"2024-09-17T04:27:43.200425+0000","last_became_peered":"2024-09-17T04:27:43.200425+0000","last_unstale":"2024-09-17T04:27:45.225738+0000","last_undegraded":"2024-09-17T04:27:45.225738+0000","last_fullsized":"2024-09-17T04:27:45.225738+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:05:00.277014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.222723+0000","last_change":"2024-09-17T04:27:44.218886+0000","last_active":"2024-09-17T04:27:45.222723+0000","last_peered":"2024-09-17T04:27:45.222723+0000","last_clean":"2024-09-17T04:27:45.222723+0000","last_became_active":"2024-09-17T04:27:44.218277+0000","last_became_peered":"2024-09-17T04:27:44.218277+0000","last_unstale":"2024-09-17T04:27:45.222723+0000","last_undegraded":"2024-09-17T04:27:45.222723+0000","last_fullsized":"2024-09-17T04:27:45.222723+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:56:07.557036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.217175+0000","last_change":"2024-09-17T04:27:43.203031+0000","last_active":"2024-09-17T04:27:45.217175+0000","last_peered":"2024-09-17T04:27:45.217175+0000","last_clean":"2024-09-17T04:27:45.217175+0000","last_became_active":"2024-09-17T04:27:43.202637+0000","last_became_peered":"2024-09-17T04:27:43.202637+0000","last_unstale":"2024-09-17T04:27:45.217175+0000","last_undegraded":"2024-09-17T04:27:45.217175+0000","last_fullsized":"2024-09-17T04:27:45.217175+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:07.729772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.213008+0000","last_change":"2024-09-17T04:27:44.217560+0000","last_active":"2024-09-17T04:27:45.213008+0000","last_peered":"2024-09-17T04:27:45.213008+0000","last_clean":"2024-09-17T04:27:45.213008+0000","last_became_active":"2024-09-17T04:27:44.217250+0000","last_became_peered":"2024-09-17T04:27:44.217250+0000","last_unstale":"2024-09-17T04:27:45.213008+0000","last_undegraded":"2024-09-17T04:27:45.213008+0000","last_fullsized":"2024-09-17T04:27:45.213008+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:47:24.767897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483466+0000","last_change":"2024-09-17T04:27:43.201109+0000","last_active":"2024-09-17T04:27:45.483466+0000","last_peered":"2024-09-17T04:27:45.483466+0000","last_clean":"2024-09-17T04:27:45.483466+0000","last_became_active":"2024-09-17T04:27:43.200806+0000","last_became_peered":"2024-09-17T04:27:43.200806+0000","last_unstale":"2024-09-17T04:27:45.483466+0000","last_undegraded":"2024-09-17T04:27:45.483466+0000","last_fullsized":"2024-09-17T04:27:45.483466+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:35.455090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.484131+0000","last_change":"2024-09-17T04:27:44.229991+0000","last_active":"2024-09-17T04:27:45.484131+0000","last_peered":"2024-09-17T04:27:45.484131+0000","last_clean":"2024-09-17T04:27:45.484131+0000","last_became_active":"2024-09-17T04:27:44.229682+0000","last_became_peered":"2024-09-17T04:27:44.229682+0000","last_unstale":"2024-09-17T04:27:45.484131+0000","last_undegraded":"2024-09-17T04:27:45.484131+0000","last_fullsized":"2024-09-17T04:27:45.484131+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:58:10.407213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.216903+0000","last_change":"2024-09-17T04:27:43.201986+0000","last_active":"2024-09-17T04:27:45.216903+0000","last_peered":"2024-09-17T04:27:45.216903+0000","last_clean":"2024-09-17T04:27:45.216903+0000","last_became_active":"2024-09-17T04:27:43.201654+0000","last_became_peered":"2024-09-17T04:27:43.201654+0000","last_unstale":"2024-09-17T04:27:45.216903+0000","last_undegraded":"2024-09-17T04:27:45.216903+0000","last_fullsized":"2024-09-17T04:27:45.216903+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:25.109734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483266+0000","last_change":"2024-09-17T04:27:44.241055+0000","last_active":"2024-09-17T04:27:45.483266+0000","last_peered":"2024-09-17T04:27:45.483266+0000","last_clean":"2024-09-17T04:27:45.483266+0000","last_became_active":"2024-09-17T04:27:44.238151+0000","last_became_peered":"2024-09-17T04:27:44.238151+0000","last_unstale":"2024-09-17T04:27:45.483266+0000","last_undegraded":"2024-09-17T04:27:45.483266+0000","last_fullsized":"2024-09-17T04:27:45.483266+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:45.589759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483212+0000","last_change":"2024-09-17T04:27:43.200262+0000","last_active":"2024-09-17T04:27:45.483212+0000","last_peered":"2024-09-17T04:27:45.483212+0000","last_clean":"2024-09-17T04:27:45.483212+0000","last_became_active":"2024-09-17T04:27:43.199434+0000","last_became_peered":"2024-09-17T04:27:43.199434+0000","last_unstale":"2024-09-17T04:27:45.483212+0000","last_undegraded":"2024-09-17T04:27:45.483212+0000","last_fullsized":"2024-09-17T04:27:45.483212+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:14.709674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212826+0000","last_change":"2024-09-17T04:27:44.223877+0000","last_active":"2024-09-17T04:27:45.212826+0000","last_peered":"2024-09-17T04:27:45.212826+0000","last_clean":"2024-09-17T04:27:45.212826+0000","last_became_active":"2024-09-17T04:27:44.223582+0000","last_became_peered":"2024-09-17T04:27:44.223582+0000","last_unstale":"2024-09-17T04:27:45.212826+0000","last_undegraded":"2024-09-17T04:27:45.212826+0000","last_fullsized":"2024-09-17T04:27:45.212826+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:09.084549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212546+0000","last_change":"2024-09-17T04:27:43.201747+0000","last_active":"2024-09-17T04:27:45.212546+0000","last_peered":"2024-09-17T04:27:45.212546+0000","last_clean":"2024-09-17T04:27:45.212546+0000","last_became_active":"2024-09-17T04:27:43.200792+0000","last_became_peered":"2024-09-17T04:27:43.200792+0000","last_unstale":"2024-09-17T04:27:45.212546+0000","last_undegraded":"2024-09-17T04:27:45.212546+0000","last_fullsized":"2024-09-17T04:27:45.212546+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:30.181955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.212302+0000","last_change":"2024-09-17T04:27:44.223584+0000","last_active":"2024-09-17T04:27:45.212302+0000","last_peered":"2024-09-17T04:27:45.212302+0000","last_clean":"2024-09-17T04:27:45.212302+0000","last_became_active":"2024-09-17T04:27:44.220615+0000","last_became_peered":"2024-09-17T04:27:44.220615+0000","last_unstale":"2024-09-17T04:27:45.212302+0000","last_undegraded":"2024-09-17T04:27:45.212302+0000","last_fullsized":"2024-09-17T04:27:45.212302+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:43.175780+0000","last_clean_scrub_stamp":"2024-09-17T04:27:43.175780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:46.132448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.225624+0000","last_change":"2024-09-17T04:27:43.203968+0000","last_active":"2024-09-17T04:27:45.225624+0000","last_peered":"2024-09-17T04:27:45.225624+0000","last_clean":"2024-09-17T04:27:45.225624+0000","last_became_active":"2024-09-17T04:27:43.203677+0000","last_became_peered":"2024-09-17T04:27:43.203677+0000","last_unstale":"2024-09-17T04:27:45.225624+0000","last_undegraded":"2024-09-17T04:27:45.225624+0000","last_fullsized":"2024-09-17T04:27:45.225624+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:27:42.169395+0000","last_clean_scrub_stamp":"2024-09-17T04:27:42.169395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:45:17.816227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":184,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T04:27:45.483827+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:27:45.483827+0000","last_peered":"2024-09-17T04:27:45.483827+0000","last_clean":"2024-09-17T04:27:45.483827+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:27:45.483827+0000","last_undegraded":"2024-09-17T04:27:45.483827+0000","last_fullsized":"2024-09-17T04:27:45.483827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476786,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1323008,"data_stored":889894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476785,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27732,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":720896,"data_stored":299366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476786,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":729088,"data_stored":300018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476786,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":733184,"data_stored":300978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509490,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":729088,"data_stored":299422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509489,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31900,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339940,"statfs":{"total":96636764160,"available":96604098560,"internally_reserved":0,"allocated":724992,"data_stored":299456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509489,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1331200,"data_stored":891928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542194,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":729088,"data_stored":299422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:27:54.067 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T04:27:54.067 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T04:27:54.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.331+0000 7f4e1ce00640 1 -- 172.21.15.100:0/273669059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e18160780 msgr2=0x7f4e180cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:54.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.331+0000 7f4e1ce00640 1 --2- 172.21.15.100:0/273669059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e18160780 0x7f4e180cd070 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4e18072980 tx=0x7f4e1002a980 comp rx=0 tx=0).stop 2024-09-17T04:27:54.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.331+0000 7f4e1ce00640 1 -- 172.21.15.100:0/273669059 shutdown_connections 2024-09-17T04:27:54.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.331+0000 7f4e1ce00640 1 --2- 172.21.15.100:0/273669059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e18160780 0x7f4e180cd070 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:54.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.331+0000 7f4e1ce00640 1 --2- 172.21.15.100:0/273669059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e18160370 0x7f4e180ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:54.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.331+0000 7f4e1ce00640 1 --2- 172.21.15.100:0/273669059 >> v2:172.21.15.184:3300/0 conn(0x7f4e180af4e0 0x7f4e1815d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:54.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.331+0000 7f4e1ce00640 1 -- 172.21.15.100:0/273669059 >> 172.21.15.100:0/273669059 conn(0x7f4e180b1dd0 msgr2=0x7f4e180b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:54.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.332+0000 7f4e1ce00640 1 -- 172.21.15.100:0/273669059 shutdown_connections 2024-09-17T04:27:54.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.332+0000 7f4e1ce00640 1 -- 172.21.15.100:0/273669059 wait complete. 2024-09-17T04:27:54.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.334+0000 7f4e1ce00640 1 Processor -- start 2024-09-17T04:27:54.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.334+0000 7f4e1ce00640 1 -- start start 2024-09-17T04:27:54.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.335+0000 7f4e1ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e180af4e0 0x7f4e1809ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:54.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.335+0000 7f4e1ce00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e18160370 0x7f4e18146230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:54.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.335+0000 7f4e1ce00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e18160780 0x7f4e181f3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:54.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.335+0000 7f4e1ce00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4e180b4140 con 0x7f4e180af4e0 2024-09-17T04:27:54.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.335+0000 7f4e1ce00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4e180b3fc0 con 0x7f4e18160780 2024-09-17T04:27:54.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.335+0000 7f4e1ce00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4e180b42c0 con 0x7f4e18160370 2024-09-17T04:27:54.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.335+0000 7f4e17400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e180af4e0 0x7f4e1809ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:54.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.335+0000 7f4e17400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e180af4e0 0x7f4e1809ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56954/0 (socket says 172.21.15.100:56954) 2024-09-17T04:27:54.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.336+0000 7f4e17400640 1 -- 172.21.15.100:0/1599663233 learned_addr learned my addr 172.21.15.100:0/1599663233 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:54.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.336+0000 7f4e16a00640 1 --2- 172.21.15.100:0/1599663233 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e18160370 0x7f4e18146230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:54.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.336+0000 7f4e17400640 1 -- 172.21.15.100:0/1599663233 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e18160370 msgr2=0x7f4e18146230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:54.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.336+0000 7f4e17e00640 1 --2- 172.21.15.100:0/1599663233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e18160780 0x7f4e181f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:54.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.336+0000 7f4e17400640 1 --2- 172.21.15.100:0/1599663233 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e18160370 0x7f4e18146230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:54.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.336+0000 7f4e17400640 1 -- 172.21.15.100:0/1599663233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e18160780 msgr2=0x7f4e181f3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:54.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.336+0000 7f4e17400640 1 --2- 172.21.15.100:0/1599663233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e18160780 0x7f4e181f3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:54.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.336+0000 7f4e17400640 1 -- 172.21.15.100:0/1599663233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e1815b7b0 con 0x7f4e180af4e0 2024-09-17T04:27:54.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.336+0000 7f4e16a00640 1 --2- 172.21.15.100:0/1599663233 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e18160370 0x7f4e18146230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:27:54.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.336+0000 7f4e17e00640 1 --2- 172.21.15.100:0/1599663233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e18160780 0x7f4e181f3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:27:54.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.336+0000 7f4e17400640 1 --2- 172.21.15.100:0/1599663233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e180af4e0 0x7f4e1809ddf0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800d660 tx=0x7f4e0800db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:54.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.336+0000 7f4e0f400640 1 -- 172.21.15.100:0/1599663233 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e08017070 con 0x7f4e180af4e0 2024-09-17T04:27:54.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.336+0000 7f4e0f400640 1 -- 172.21.15.100:0/1599663233 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e08005310 con 0x7f4e180af4e0 2024-09-17T04:27:54.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.336+0000 7f4e0f400640 1 -- 172.21.15.100:0/1599663233 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e08009820 con 0x7f4e180af4e0 2024-09-17T04:27:54.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.337+0000 7f4e1ce00640 1 -- 172.21.15.100:0/1599663233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e1815ba90 con 0x7f4e180af4e0 2024-09-17T04:27:54.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.337+0000 7f4e1ce00640 1 -- 172.21.15.100:0/1599663233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e1815b4c0 con 0x7f4e180af4e0 2024-09-17T04:27:54.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.338+0000 7f4e1ce00640 1 -- 172.21.15.100:0/1599663233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e1815d790 con 0x7f4e180af4e0 2024-09-17T04:27:54.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.339+0000 7f4e0f400640 1 -- 172.21.15.100:0/1599663233 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f4e080099c0 con 0x7f4e180af4e0 2024-09-17T04:27:54.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.340+0000 7f4e0f400640 1 --2- 172.21.15.100:0/1599663233 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4df8078d40 0x7f4df807b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:54.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.340+0000 7f4e0f400640 1 -- 172.21.15.100:0/1599663233 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7f4e0809b2e0 con 0x7f4e180af4e0 2024-09-17T04:27:54.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.341+0000 7f4e16a00640 1 --2- 172.21.15.100:0/1599663233 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4df8078d40 0x7f4df807b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:54.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.342+0000 7f4e16a00640 1 --2- 172.21.15.100:0/1599663233 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4df8078d40 0x7f4df807b250 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4e1806f350 tx=0x7f4e04005dd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:54.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.345+0000 7f4e0f400640 1 -- 172.21.15.100:0/1599663233 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4e080632f0 con 0x7f4e180af4e0 2024-09-17T04:27:54.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.599+0000 7f4e1ce00640 1 -- 172.21.15.100:0/1599663233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f4e180b02a0 con 0x7f4e180af4e0 2024-09-17T04:27:54.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.602+0000 7f4e0f400640 1 -- 172.21.15.100:0/1599663233 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f4e08062bf0 con 0x7f4e180af4e0 2024-09-17T04:27:54.602 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-09-17T04:27:54.602 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAsBOlmfYg9IBAABP1uRKGOugrA47VxwRLkjA== 2024-09-17T04:27:54.602 INFO:teuthology.orchestra.run.smithi100.stdout: caps mds = "allow" 2024-09-17T04:27:54.602 INFO:teuthology.orchestra.run.smithi100.stdout: caps mon = "allow r" 2024-09-17T04:27:54.602 INFO:teuthology.orchestra.run.smithi100.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T04:27:54.602 INFO:teuthology.orchestra.run.smithi100.stderr:updated caps for client.0 2024-09-17T04:27:54.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.605+0000 7f4e1ce00640 1 -- 172.21.15.100:0/1599663233 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4df8078d40 msgr2=0x7f4df807b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:54.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.605+0000 7f4e1ce00640 1 --2- 172.21.15.100:0/1599663233 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4df8078d40 0x7f4df807b250 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4e1806f350 tx=0x7f4e04005dd0 comp rx=0 tx=0).stop 2024-09-17T04:27:54.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.605+0000 7f4e1ce00640 1 -- 172.21.15.100:0/1599663233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e180af4e0 msgr2=0x7f4e1809ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:54.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.605+0000 7f4e1ce00640 1 --2- 172.21.15.100:0/1599663233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e180af4e0 0x7f4e1809ddf0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800d660 tx=0x7f4e0800db20 comp rx=0 tx=0).stop 2024-09-17T04:27:54.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.605+0000 7f4e1ce00640 1 -- 172.21.15.100:0/1599663233 shutdown_connections 2024-09-17T04:27:54.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.605+0000 7f4e1ce00640 1 --2- 172.21.15.100:0/1599663233 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4df8078d40 0x7f4df807b250 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:54.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.605+0000 7f4e1ce00640 1 --2- 172.21.15.100:0/1599663233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e18160780 0x7f4e181f3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:54.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.605+0000 7f4e1ce00640 1 --2- 172.21.15.100:0/1599663233 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e18160370 0x7f4e18146230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:54.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.605+0000 7f4e1ce00640 1 --2- 172.21.15.100:0/1599663233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e180af4e0 0x7f4e1809ddf0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:54.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.605+0000 7f4e1ce00640 1 -- 172.21.15.100:0/1599663233 >> 172.21.15.100:0/1599663233 conn(0x7f4e180b1dd0 msgr2=0x7f4e180b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:54.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.605+0000 7f4e1ce00640 1 -- 172.21.15.100:0/1599663233 shutdown_connections 2024-09-17T04:27:54.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.605+0000 7f4e1ce00640 1 -- 172.21.15.100:0/1599663233 wait complete. 2024-09-17T04:27:54.628 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:27:54.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.850+0000 7fa53e600640 1 -- 172.21.15.100:0/1122405227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa538152920 msgr2=0x7fa53815f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:54.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.850+0000 7fa53e600640 1 --2- 172.21.15.100:0/1122405227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa538152920 0x7fa53815f3d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa524009830 tx=0x7fa52402a480 comp rx=0 tx=0).stop 2024-09-17T04:27:54.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.850+0000 7fa53e600640 1 -- 172.21.15.100:0/1122405227 shutdown_connections 2024-09-17T04:27:54.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.850+0000 7fa53e600640 1 --2- 172.21.15.100:0/1122405227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa538152920 0x7fa53815f3d0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:54.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.850+0000 7fa53e600640 1 -- 172.21.15.100:0/1122405227 >> 172.21.15.100:0/1122405227 conn(0x7fa53806d730 msgr2=0x7fa5380b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:54.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.850+0000 7fa53e600640 1 -- 172.21.15.100:0/1122405227 shutdown_connections 2024-09-17T04:27:54.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.850+0000 7fa53e600640 1 -- 172.21.15.100:0/1122405227 wait complete. 2024-09-17T04:27:54.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.853+0000 7fa53e600640 1 Processor -- start 2024-09-17T04:27:54.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.853+0000 7fa53e600640 1 -- start start 2024-09-17T04:27:54.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.854+0000 7fa53e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5380aebc0 0x7fa5380afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:54.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.854+0000 7fa53e600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa538152920 0x7fa5380b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:54.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.854+0000 7fa53e600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa53815f910 0x7fa5380b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:54.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.854+0000 7fa53e600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa5380b4390 con 0x7fa5380aebc0 2024-09-17T04:27:54.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.854+0000 7fa53e600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa5380b4210 con 0x7fa53815f910 2024-09-17T04:27:54.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.854+0000 7fa53e600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa5380b4510 con 0x7fa538152920 2024-09-17T04:27:54.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.854+0000 7fa537400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5380aebc0 0x7fa5380afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:54.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.854+0000 7fa537400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5380aebc0 0x7fa5380afe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41588/0 (socket says 172.21.15.100:41588) 2024-09-17T04:27:54.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.854+0000 7fa537400640 1 -- 172.21.15.100:0/257256955 learned_addr learned my addr 172.21.15.100:0/257256955 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:54.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.855+0000 7fa536a00640 1 --2- 172.21.15.100:0/257256955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa538152920 0x7fa5380b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:54.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.855+0000 7fa537e00640 1 --2- 172.21.15.100:0/257256955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa53815f910 0x7fa5380b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:54.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.855+0000 7fa537400640 1 -- 172.21.15.100:0/257256955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa538152920 msgr2=0x7fa5380b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:54.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.855+0000 7fa537400640 1 --2- 172.21.15.100:0/257256955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa538152920 0x7fa5380b0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:54.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.855+0000 7fa537400640 1 -- 172.21.15.100:0/257256955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa53815f910 msgr2=0x7fa5380b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:54.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.855+0000 7fa537400640 1 --2- 172.21.15.100:0/257256955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa53815f910 0x7fa5380b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:54.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.855+0000 7fa537400640 1 -- 172.21.15.100:0/257256955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53813f390 con 0x7fa5380aebc0 2024-09-17T04:27:54.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.855+0000 7fa537400640 1 --2- 172.21.15.100:0/257256955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5380aebc0 0x7fa5380afe50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa52800c3f0 tx=0x7fa52800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:54.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.855+0000 7fa52f400640 1 -- 172.21.15.100:0/257256955 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa528017070 con 0x7fa5380aebc0 2024-09-17T04:27:54.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.856+0000 7fa52f400640 1 -- 172.21.15.100:0/257256955 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5280035f0 con 0x7fa5380aebc0 2024-09-17T04:27:54.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.856+0000 7fa53e600640 1 -- 172.21.15.100:0/257256955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5381592c0 con 0x7fa5380aebc0 2024-09-17T04:27:54.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.856+0000 7fa52f400640 1 -- 172.21.15.100:0/257256955 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa528003a40 con 0x7fa5380aebc0 2024-09-17T04:27:54.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.856+0000 7fa53e600640 1 -- 172.21.15.100:0/257256955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5380cf0f0 con 0x7fa5380aebc0 2024-09-17T04:27:54.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.857+0000 7fa53e600640 1 -- 172.21.15.100:0/257256955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5380cf330 con 0x7fa5380aebc0 2024-09-17T04:27:54.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.858+0000 7fa52f400640 1 -- 172.21.15.100:0/257256955 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7fa52801a070 con 0x7fa5380aebc0 2024-09-17T04:27:54.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.859+0000 7fa52f400640 1 --2- 172.21.15.100:0/257256955 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa518078d70 0x7fa51807b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:54.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.859+0000 7fa52f400640 1 -- 172.21.15.100:0/257256955 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7fa52809a440 con 0x7fa5380aebc0 2024-09-17T04:27:54.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.859+0000 7fa536a00640 1 --2- 172.21.15.100:0/257256955 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa518078d70 0x7fa51807b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:54.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.860+0000 7fa536a00640 1 --2- 172.21.15.100:0/257256955 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa518078d70 0x7fa51807b280 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa524034000 tx=0x7fa524004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:54.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:54.863+0000 7fa52f400640 1 -- 172.21.15.100:0/257256955 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa5380cf330 con 0x7fa5380aebc0 2024-09-17T04:27:55.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.083+0000 7fa53e600640 1 -- 172.21.15.100:0/257256955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5380b49e0 con 0x7fa5380aebc0 2024-09-17T04:27:55.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.084+0000 7fa52f400640 1 -- 172.21.15.100:0/257256955 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5229 (secure 0 0 0) 0x7fa528061e00 con 0x7fa5380aebc0 2024-09-17T04:27:55.084 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:27:55.084 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":36,"btime":"2024-09-17T04:27:47:219814+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1735051989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1735051989},{"type":"v1","addr":"172.21.15.184:6836","nonce":1735051989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5638,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3935107098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":3935107098},{"type":"v1","addr":"172.21.15.184:6837","nonce":3935107098}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/904123596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904123596},{"type":"v1","addr":"172.21.15.100:6834","nonce":904123596}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:27:43.682593+0000","modified":"2024-09-17T04:27:46.702665+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/270906112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":270906112},{"type":"v1","addr":"172.21.15.100:6835","nonce":270906112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5739,"qdb_cluster":[5739]},"id":4}]} 2024-09-17T04:27:55.084 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 36 2024-09-17T04:27:55.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.087+0000 7fa53e600640 1 -- 172.21.15.100:0/257256955 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa518078d70 msgr2=0x7fa51807b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.087+0000 7fa53e600640 1 --2- 172.21.15.100:0/257256955 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa518078d70 0x7fa51807b280 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa524034000 tx=0x7fa524004730 comp rx=0 tx=0).stop 2024-09-17T04:27:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.087+0000 7fa53e600640 1 -- 172.21.15.100:0/257256955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5380aebc0 msgr2=0x7fa5380afe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.087+0000 7fa53e600640 1 --2- 172.21.15.100:0/257256955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5380aebc0 0x7fa5380afe50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa52800c3f0 tx=0x7fa52800c8b0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.087+0000 7fa53e600640 1 -- 172.21.15.100:0/257256955 shutdown_connections 2024-09-17T04:27:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.087+0000 7fa53e600640 1 --2- 172.21.15.100:0/257256955 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa518078d70 0x7fa51807b280 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.087+0000 7fa53e600640 1 --2- 172.21.15.100:0/257256955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa53815f910 0x7fa5380b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.087+0000 7fa53e600640 1 --2- 172.21.15.100:0/257256955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa538152920 0x7fa5380b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.087+0000 7fa53e600640 1 --2- 172.21.15.100:0/257256955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5380aebc0 0x7fa5380afe50 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.087+0000 7fa53e600640 1 -- 172.21.15.100:0/257256955 >> 172.21.15.100:0/257256955 conn(0x7fa53806d730 msgr2=0x7fa53815cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.087+0000 7fa53e600640 1 -- 172.21.15.100:0/257256955 shutdown_connections 2024-09-17T04:27:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.087+0000 7fa53e600640 1 -- 172.21.15.100:0/257256955 wait complete. 2024-09-17T04:27:55.513 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T04:27:43.682593+0000', 'modified': '2024-09-17T04:27:46.702665+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5739}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5739': {'gid': 5739, 'name': 'a', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6835/270906112', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6833', 'nonce': 270906112}, {'type': 'v1', 'addr': '172.21.15.100:6835', 'nonce': 270906112}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5739, 'qdb_cluster': [5739]} 2024-09-17T04:27:55.513 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T04:27:55.513 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-17T04:27:55.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.733+0000 7f0f5d000640 1 -- 172.21.15.100:0/2656068119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5815a0f0 msgr2=0x7f0f5815d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:55.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.733+0000 7f0f5d000640 1 --2- 172.21.15.100:0/2656068119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5815a0f0 0x7f0f5815d670 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0f58051590 tx=0x7f0f5002aa30 comp rx=0 tx=0).stop 2024-09-17T04:27:55.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.733+0000 7f0f5d000640 1 -- 172.21.15.100:0/2656068119 shutdown_connections 2024-09-17T04:27:55.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.733+0000 7f0f5d000640 1 --2- 172.21.15.100:0/2656068119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5815a0f0 0x7f0f5815d670 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.733+0000 7f0f5d000640 1 --2- 172.21.15.100:0/2656068119 >> v2:172.21.15.184:3300/0 conn(0x7f0f580ad9c0 0x7f0f580af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.733+0000 7f0f5d000640 1 -- 172.21.15.100:0/2656068119 >> 172.21.15.100:0/2656068119 conn(0x7f0f580b1e40 msgr2=0x7f0f580b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:55.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.733+0000 7f0f5d000640 1 -- 172.21.15.100:0/2656068119 shutdown_connections 2024-09-17T04:27:55.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.733+0000 7f0f5d000640 1 -- 172.21.15.100:0/2656068119 wait complete. 2024-09-17T04:27:55.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.735+0000 7f0f5d000640 1 Processor -- start 2024-09-17T04:27:55.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.735+0000 7f0f5d000640 1 -- start start 2024-09-17T04:27:55.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.737+0000 7f0f5d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f580ad9c0 0x7f0f580cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:55.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.737+0000 7f0f5d000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f580b2f60 0x7f0f581f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:55.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.737+0000 7f0f5d000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f5815a0f0 0x7f0f581f9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:55.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.737+0000 7f0f5d000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5806f290 con 0x7f0f580ad9c0 2024-09-17T04:27:55.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.737+0000 7f0f5d000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5815a5e0 con 0x7f0f580b2f60 2024-09-17T04:27:55.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.737+0000 7f0f5d000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0f5806f410 con 0x7f0f5815a0f0 2024-09-17T04:27:55.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.737+0000 7f0f57400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f580ad9c0 0x7f0f580cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:55.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.737+0000 7f0f57400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f580ad9c0 0x7f0f580cdb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41602/0 (socket says 172.21.15.100:41602) 2024-09-17T04:27:55.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.737+0000 7f0f57400640 1 -- 172.21.15.100:0/3879492713 learned_addr learned my addr 172.21.15.100:0/3879492713 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:55.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.737+0000 7f0f4ea00640 1 --2- 172.21.15.100:0/3879492713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f580b2f60 0x7f0f581f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:55.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.737+0000 7f0f57e00640 1 --2- 172.21.15.100:0/3879492713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f5815a0f0 0x7f0f581f9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:55.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.737+0000 7f0f57400640 1 -- 172.21.15.100:0/3879492713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f5815a0f0 msgr2=0x7f0f581f9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:55.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.737+0000 7f0f57400640 1 --2- 172.21.15.100:0/3879492713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f5815a0f0 0x7f0f581f9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.737+0000 7f0f57400640 1 -- 172.21.15.100:0/3879492713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f580b2f60 msgr2=0x7f0f581f8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:55.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.737+0000 7f0f57400640 1 --2- 172.21.15.100:0/3879492713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f580b2f60 0x7f0f581f8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.737+0000 7f0f57400640 1 -- 172.21.15.100:0/3879492713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f580ce5d0 con 0x7f0f580ad9c0 2024-09-17T04:27:55.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.738+0000 7f0f57400640 1 --2- 172.21.15.100:0/3879492713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f580ad9c0 0x7f0f580cdb10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800d770 tx=0x7f0f4800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:55.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.738+0000 7f0f4fe00640 1 -- 172.21.15.100:0/3879492713 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48017070 con 0x7f0f580ad9c0 2024-09-17T04:27:55.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.738+0000 7f0f4fe00640 1 -- 172.21.15.100:0/3879492713 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f48009d20 con 0x7f0f580ad9c0 2024-09-17T04:27:55.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.738+0000 7f0f5d000640 1 -- 172.21.15.100:0/3879492713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f580ce8b0 con 0x7f0f580ad9c0 2024-09-17T04:27:55.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.738+0000 7f0f4fe00640 1 -- 172.21.15.100:0/3879492713 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f480106c0 con 0x7f0f580ad9c0 2024-09-17T04:27:55.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.738+0000 7f0f5d000640 1 -- 172.21.15.100:0/3879492713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f580cee70 con 0x7f0f580ad9c0 2024-09-17T04:27:55.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.741+0000 7f0f4fe00640 1 -- 172.21.15.100:0/3879492713 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f0f4801a070 con 0x7f0f580ad9c0 2024-09-17T04:27:55.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.742+0000 7f0f4fe00640 1 --2- 172.21.15.100:0/3879492713 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0f34078d70 0x7f0f3407b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:55.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.742+0000 7f0f4fe00640 1 -- 172.21.15.100:0/3879492713 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7f0f4809b3a0 con 0x7f0f580ad9c0 2024-09-17T04:27:55.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.743+0000 7f0f65833740 1 Processor -- start 2024-09-17T04:27:55.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.743+0000 7f0f4ea00640 1 --2- 172.21.15.100:0/3879492713 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0f34078d70 0x7f0f3407b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:55.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.743+0000 7f0f65833740 1 -- start start 2024-09-17T04:27:55.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.744+0000 7f0f4ea00640 1 --2- 172.21.15.100:0/3879492713 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0f34078d70 0x7f0f3407b280 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0f580b4d30 tx=0x7f0f40009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:55.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.744+0000 7f0f65833740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556959e8270 0x555695db8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:55.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.744+0000 7f0f65833740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556959bf3b0 0x555695db90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:55.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.744+0000 7f0f65833740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556959c7590 0x555695d89150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:55.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.744+0000 7f0f65833740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555695d12430 con 0x5556959e8270 2024-09-17T04:27:55.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.744+0000 7f0f65833740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555695d025d0 con 0x5556959bf3b0 2024-09-17T04:27:55.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.744+0000 7f0f65833740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x555695cf1a00 con 0x5556959c7590 2024-09-17T04:27:55.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.744+0000 7f0f57400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556959e8270 0x555695db8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:55.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.744+0000 7f0f57400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556959e8270 0x555695db8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41604/0 (socket says 172.21.15.100:41604) 2024-09-17T04:27:55.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.744+0000 7f0f57400640 1 -- 172.21.15.100:0/1271253688 learned_addr learned my addr 172.21.15.100:0/1271253688 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:55.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.745+0000 7f0f57e00640 1 --2- 172.21.15.100:0/1271253688 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556959c7590 0x555695d89150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:55.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.745+0000 7f0f4ea00640 1 --2- 172.21.15.100:0/1271253688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556959bf3b0 0x555695db90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:55.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.745+0000 7f0f57400640 1 -- 172.21.15.100:0/1271253688 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556959c7590 msgr2=0x555695d89150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:27:55.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.745+0000 7f0f57400640 1 --2- 172.21.15.100:0/1271253688 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556959c7590 0x555695d89150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.745+0000 7f0f57400640 1 -- 172.21.15.100:0/1271253688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556959bf3b0 msgr2=0x555695db90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:27:55.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.745+0000 7f0f57400640 1 --2- 172.21.15.100:0/1271253688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556959bf3b0 0x555695db90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.745+0000 7f0f57400640 1 -- 172.21.15.100:0/1271253688 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555695cba520 con 0x5556959e8270 2024-09-17T04:27:55.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.745+0000 7f0f57400640 1 --2- 172.21.15.100:0/1271253688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556959e8270 0x555695db8b60 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0f48062bc0 tx=0x7f0f4805ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3531c40dd7b68fe9 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:55.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.745+0000 7f0f47e00640 1 -- 172.21.15.100:0/1271253688 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48021070 con 0x5556959e8270 2024-09-17T04:27:55.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.745+0000 7f0f47e00640 1 -- 172.21.15.100:0/1271253688 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f48063330 con 0x5556959e8270 2024-09-17T04:27:55.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.745+0000 7f0f47e00640 1 -- 172.21.15.100:0/1271253688 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48062c30 con 0x5556959e8270 2024-09-17T04:27:55.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.746+0000 7f0f65833740 1 -- 172.21.15.100:0/1271253688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556959e8270 msgr2=0x555695db8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:55.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.746+0000 7f0f65833740 1 --2- 172.21.15.100:0/1271253688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556959e8270 0x555695db8b60 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0f48062bc0 tx=0x7f0f4805ec00 comp rx=0 tx=0).stop 2024-09-17T04:27:55.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.746+0000 7f0f65833740 1 -- 172.21.15.100:0/1271253688 shutdown_connections 2024-09-17T04:27:55.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.746+0000 7f0f65833740 1 --2- 172.21.15.100:0/1271253688 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556959c7590 0x555695d89150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.746+0000 7f0f65833740 1 --2- 172.21.15.100:0/1271253688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556959bf3b0 0x555695db90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.746+0000 7f0f65833740 1 --2- 172.21.15.100:0/1271253688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556959e8270 0x555695db8b60 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.746+0000 7f0f65833740 1 -- 172.21.15.100:0/1271253688 >> 172.21.15.100:0/1271253688 conn(0x5556959dc3c0 msgr2=0x555695d8eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:55.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.746+0000 7f0f65833740 1 -- 172.21.15.100:0/1271253688 shutdown_connections 2024-09-17T04:27:55.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.746+0000 7f0f65833740 1 -- 172.21.15.100:0/1271253688 wait complete. 2024-09-17T04:27:55.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.747+0000 7f0f65833740 1 -- start start 2024-09-17T04:27:55.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.747+0000 7f0f65833740 1 Processor -- start 2024-09-17T04:27:55.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.749+0000 7f0f65833740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556959bf3b0 0x555695db40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:55.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.749+0000 7f0f65833740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556959c7590 0x555695dd4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:55.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.749+0000 7f0f65833740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556959e8270 0x555695daf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:55.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.749+0000 7f0f65833740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555695c98330 con 0x5556959bf3b0 2024-09-17T04:27:55.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.749+0000 7f0f65833740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555695cf5810 con 0x5556959c7590 2024-09-17T04:27:55.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.749+0000 7f0f65833740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x555695a62620 con 0x5556959e8270 2024-09-17T04:27:55.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.749+0000 7f0f65833740 10 client.0 _ll_register_callbacks cb 0x555695c9d420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:27:55.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.749+0000 7f0f57400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556959bf3b0 0x555695db40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:55.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.749+0000 7f0f57400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556959bf3b0 0x555695db40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41606/0 (socket says 172.21.15.100:41606) 2024-09-17T04:27:55.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.749+0000 7f0f57400640 1 -- 172.21.15.100:0/2531420252 learned_addr learned my addr 172.21.15.100:0/2531420252 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:55.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.749+0000 7f0f4ea00640 1 --2- 172.21.15.100:0/2531420252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556959c7590 0x555695dd4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:55.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.749+0000 7f0f57e00640 1 --2- 172.21.15.100:0/2531420252 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556959e8270 0x555695daf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:55.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.750+0000 7f0f57400640 1 -- 172.21.15.100:0/2531420252 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556959e8270 msgr2=0x555695daf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:27:55.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.750+0000 7f0f57400640 1 --2- 172.21.15.100:0/2531420252 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556959e8270 0x555695daf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.750+0000 7f0f57400640 1 -- 172.21.15.100:0/2531420252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556959c7590 msgr2=0x555695dd4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:27:55.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.750+0000 7f0f57400640 1 --2- 172.21.15.100:0/2531420252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556959c7590 0x555695dd4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.750+0000 7f0f57400640 1 -- 172.21.15.100:0/2531420252 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555695a53850 con 0x5556959bf3b0 2024-09-17T04:27:55.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.750+0000 7f0f4ea00640 1 --2- 172.21.15.100:0/2531420252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556959c7590 0x555695dd4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:27:55.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.750+0000 7f0f57400640 1 --2- 172.21.15.100:0/2531420252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556959bf3b0 0x555695db40f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0f48062bc0 tx=0x7f0f4805f0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ccacc553966e69 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:55.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.750+0000 7f0f44c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-17T04:27:55.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.750+0000 7f0f44c00640 1 -- 172.21.15.100:0/2531420252 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48021070 con 0x5556959bf3b0 2024-09-17T04:27:55.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.750+0000 7f0f44c00640 1 -- 172.21.15.100:0/2531420252 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7f0f48083d40 con 0x5556959bf3b0 2024-09-17T04:27:55.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.750+0000 7f0f65833740 1 -- 172.21.15.100:0/2531420252 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f48063330 con 0x5556959bf3b0 2024-09-17T04:27:55.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.750+0000 7f0f44c00640 1 -- 172.21.15.100:0/2531420252 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0f48067710 con 0x5556959bf3b0 2024-09-17T04:27:55.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.750+0000 7f0f65833740 10 client.5892 fetch_fsmap learned FSMap version 36 2024-09-17T04:27:55.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.750+0000 7f0f65833740 1 -- 172.21.15.100:0/2531420252 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x555695aca6b0 con 0x5556959bf3b0 2024-09-17T04:27:55.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.751+0000 7f0f44c00640 1 -- 172.21.15.100:0/2531420252 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f4807e080 con 0x5556959bf3b0 2024-09-17T04:27:55.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.751+0000 7f0f44c00640 1 -- 172.21.15.100:0/2531420252 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7f0f48061e90 con 0x5556959bf3b0 2024-09-17T04:27:55.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.751+0000 7f0f65833740 10 client.5892 fetch_fsmap finished waiting for FSMap version 36 2024-09-17T04:27:55.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.751+0000 7f0f65833740 10 client.5892 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T04:27:55.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.751+0000 7f0f65833740 20 client.5892 populate_metadata read hostname 'smithi100' 2024-09-17T04:27:55.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.751+0000 7f0f65833740 1 --2- 172.21.15.100:0/2531420252 >> [v2:172.21.15.100:6833/270906112,v1:172.21.15.100:6835/270906112] conn(0x5556959a6e40 0x555695db5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:55.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.751+0000 7f0f65833740 4 client.5892 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T04:27:55.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.751+0000 7f0f65833740 1 -- 172.21.15.100:0/2531420252 --> [v2:172.21.15.100:6833/270906112,v1:172.21.15.100:6835/270906112] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555695d04520 con 0x5556959a6e40 2024-09-17T04:27:55.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.751+0000 7f0f4ea00640 1 --2- 172.21.15.100:0/2531420252 >> [v2:172.21.15.100:6833/270906112,v1:172.21.15.100:6835/270906112] conn(0x5556959a6e40 0x555695db5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:55.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.752+0000 7f0f44c00640 10 client.5892 ms_handle_connect on v2:172.21.15.100:6833/270906112 2024-09-17T04:27:55.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.752+0000 7f0f4ea00640 1 --2- 172.21.15.100:0/2531420252 >> [v2:172.21.15.100:6833/270906112,v1:172.21.15.100:6835/270906112] conn(0x5556959a6e40 0x555695db5230 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ce85055fc45033a server_cookie=b552d0bc4ac3932c in_seq=0 out_seq=0 2024-09-17T04:27:55.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.752+0000 7f0f44c00640 1 -- 172.21.15.100:0/2531420252 <== mds.0 v2:172.21.15.100:6833/270906112 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555695d04520 con 0x5556959a6e40 2024-09-17T04:27:55.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.752+0000 7f0f44c00640 10 client.5892 handle_command_reply: tid=0 2024-09-17T04:27:55.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.753+0000 7f0f65833740 1 client.5892 shutdown 2024-09-17T04:27:55.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.753+0000 7f0f65833740 1 -- 172.21.15.100:0/2531420252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556959bf3b0 msgr2=0x555695db40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:55.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.753+0000 7f0f65833740 1 --2- 172.21.15.100:0/2531420252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556959bf3b0 0x555695db40f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0f48062bc0 tx=0x7f0f4805f0d0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.753+0000 7f0f65833740 1 -- 172.21.15.100:0/2531420252 shutdown_connections 2024-09-17T04:27:55.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.753+0000 7f0f65833740 1 --2- 172.21.15.100:0/2531420252 >> [v2:172.21.15.100:6833/270906112,v1:172.21.15.100:6835/270906112] conn(0x5556959a6e40 0x555695db5230 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.753+0000 7f0f65833740 1 --2- 172.21.15.100:0/2531420252 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556959e8270 0x555695daf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.753+0000 7f0f65833740 1 --2- 172.21.15.100:0/2531420252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556959c7590 0x555695dd4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.753+0000 7f0f65833740 1 --2- 172.21.15.100:0/2531420252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556959bf3b0 0x555695db40f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.753+0000 7f0f65833740 1 -- 172.21.15.100:0/2531420252 >> 172.21.15.100:0/2531420252 conn(0x5556959dc3c0 msgr2=0x555695dd6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:55.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.753+0000 7f0f44c00640 0 client.5892 ms_handle_reset on v2:172.21.15.100:6833/270906112 2024-09-17T04:27:55.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.753+0000 7f0f65833740 1 -- 172.21.15.100:0/2531420252 shutdown_connections 2024-09-17T04:27:55.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.753+0000 7f0f65833740 1 -- 172.21.15.100:0/2531420252 wait complete. 2024-09-17T04:27:55.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.753+0000 7f0f65833740 20 client.5892 trim_cache size 0 max 16384 2024-09-17T04:27:55.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.772+0000 7f0f65833740 1 Processor -- start 2024-09-17T04:27:55.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.772+0000 7f0f65833740 1 -- start start 2024-09-17T04:27:55.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.773+0000 7f0f65833740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556959c7590 0x555695e4c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:55.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.773+0000 7f0f65833740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556959e8270 0x555695dafcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:55.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.773+0000 7f0f65833740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55569597c280 0x555695db01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:55.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.773+0000 7f0f65833740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5814e630 con 0x55569597c280 2024-09-17T04:27:55.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.773+0000 7f0f65833740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5814bdf0 con 0x5556959e8270 2024-09-17T04:27:55.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.773+0000 7f0f65833740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0f5814dfe0 con 0x5556959c7590 2024-09-17T04:27:55.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.773+0000 7f0f57e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55569597c280 0x555695db01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:55.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.773+0000 7f0f57e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55569597c280 0x555695db01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41618/0 (socket says 172.21.15.100:41618) 2024-09-17T04:27:55.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.773+0000 7f0f57e00640 1 -- 172.21.15.100:0/1839698219 learned_addr learned my addr 172.21.15.100:0/1839698219 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:55.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.773+0000 7f0f4ea00640 1 --2- 172.21.15.100:0/1839698219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556959e8270 0x555695dafcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:55.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.773+0000 7f0f57400640 1 --2- 172.21.15.100:0/1839698219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556959c7590 0x555695e4c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:55.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.774+0000 7f0f57e00640 1 -- 172.21.15.100:0/1839698219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556959c7590 msgr2=0x555695e4c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:27:55.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.774+0000 7f0f57e00640 1 --2- 172.21.15.100:0/1839698219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556959c7590 0x555695e4c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.774+0000 7f0f57e00640 1 -- 172.21.15.100:0/1839698219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556959e8270 msgr2=0x555695dafcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:27:55.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.774+0000 7f0f57e00640 1 --2- 172.21.15.100:0/1839698219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556959e8270 0x555695dafcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.774+0000 7f0f57e00640 1 -- 172.21.15.100:0/1839698219 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555695de6210 con 0x55569597c280 2024-09-17T04:27:55.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.774+0000 7f0f57400640 1 --2- 172.21.15.100:0/1839698219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556959c7590 0x555695e4c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:27:55.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.774+0000 7f0f4ea00640 1 --2- 172.21.15.100:0/1839698219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556959e8270 0x555695dafcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:27:55.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.774+0000 7f0f57e00640 1 --2- 172.21.15.100:0/1839698219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55569597c280 0x555695db01f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009d60 tx=0x7f0f50004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b30e760e28ec1f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:55.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.774+0000 7f0f47400640 1 -- 172.21.15.100:0/1839698219 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f50033040 con 0x55569597c280 2024-09-17T04:27:55.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.774+0000 7f0f47400640 1 -- 172.21.15.100:0/1839698219 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f5002fb80 con 0x55569597c280 2024-09-17T04:27:55.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.774+0000 7f0f47400640 1 -- 172.21.15.100:0/1839698219 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f50003970 con 0x55569597c280 2024-09-17T04:27:55.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.775+0000 7f0f65833740 1 -- 172.21.15.100:0/1839698219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55569597c280 msgr2=0x555695db01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:55.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.775+0000 7f0f65833740 1 --2- 172.21.15.100:0/1839698219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55569597c280 0x555695db01f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009d60 tx=0x7f0f50004ab0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.775+0000 7f0f65833740 1 -- 172.21.15.100:0/1839698219 shutdown_connections 2024-09-17T04:27:55.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.775+0000 7f0f65833740 1 --2- 172.21.15.100:0/1839698219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55569597c280 0x555695db01f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.775+0000 7f0f65833740 1 --2- 172.21.15.100:0/1839698219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556959e8270 0x555695dafcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.775+0000 7f0f65833740 1 --2- 172.21.15.100:0/1839698219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556959c7590 0x555695e4c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.775+0000 7f0f65833740 1 -- 172.21.15.100:0/1839698219 >> 172.21.15.100:0/1839698219 conn(0x5556959dc3c0 msgr2=0x555695dcf100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:55.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.775+0000 7f0f65833740 1 -- 172.21.15.100:0/1839698219 shutdown_connections 2024-09-17T04:27:55.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.775+0000 7f0f65833740 1 -- 172.21.15.100:0/1839698219 wait complete. 2024-09-17T04:27:55.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.776+0000 7f0f65833740 1 -- start start 2024-09-17T04:27:55.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.776+0000 7f0f65833740 1 Processor -- start 2024-09-17T04:27:55.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.778+0000 7f0f65833740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55569597c280 0x555695e49ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:55.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.778+0000 7f0f65833740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556959c7590 0x555695daf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:55.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.778+0000 7f0f65833740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556959e8270 0x555695e4a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:55.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.778+0000 7f0f65833740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555695dce3b0 con 0x5556959c7590 2024-09-17T04:27:55.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.778+0000 7f0f65833740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555695c98330 con 0x55569597c280 2024-09-17T04:27:55.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.778+0000 7f0f65833740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x555695db1da0 con 0x5556959e8270 2024-09-17T04:27:55.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.778+0000 7f0f65833740 10 client.0 _ll_register_callbacks cb 0x555695c9d420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:27:55.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.778+0000 7f0f4ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556959c7590 0x555695daf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:55.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.778+0000 7f0f4ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556959c7590 0x555695daf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41626/0 (socket says 172.21.15.100:41626) 2024-09-17T04:27:55.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.778+0000 7f0f4ea00640 1 -- 172.21.15.100:0/2094852840 learned_addr learned my addr 172.21.15.100:0/2094852840 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:55.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.778+0000 7f0f57400640 1 --2- 172.21.15.100:0/2094852840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55569597c280 0x555695e49ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:55.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.778+0000 7f0f57e00640 1 --2- 172.21.15.100:0/2094852840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556959e8270 0x555695e4a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:55.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.778+0000 7f0f4ea00640 1 -- 172.21.15.100:0/2094852840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556959e8270 msgr2=0x555695e4a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:27:55.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.778+0000 7f0f4ea00640 1 --2- 172.21.15.100:0/2094852840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556959e8270 0x555695e4a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.778+0000 7f0f4ea00640 1 -- 172.21.15.100:0/2094852840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55569597c280 msgr2=0x555695e49ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:27:55.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.778+0000 7f0f4ea00640 1 --2- 172.21.15.100:0/2094852840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55569597c280 0x555695e49ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.778+0000 7f0f4ea00640 1 -- 172.21.15.100:0/2094852840 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555695df5e40 con 0x5556959c7590 2024-09-17T04:27:55.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.779+0000 7f0f4ea00640 1 --2- 172.21.15.100:0/2094852840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556959c7590 0x555695daf620 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0f400054e0 tx=0x7f0f40000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4139524b2fe23aa server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:55.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.779+0000 7f0f44c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-17T04:27:55.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.779+0000 7f0f44c00640 1 -- 172.21.15.100:0/2094852840 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f40026660 con 0x5556959c7590 2024-09-17T04:27:55.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.779+0000 7f0f44c00640 1 -- 172.21.15.100:0/2094852840 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7f0f40010040 con 0x5556959c7590 2024-09-17T04:27:55.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.779+0000 7f0f65833740 1 -- 172.21.15.100:0/2094852840 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f5002fb80 con 0x5556959c7590 2024-09-17T04:27:55.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.779+0000 7f0f44c00640 1 -- 172.21.15.100:0/2094852840 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0f40037050 con 0x5556959c7590 2024-09-17T04:27:55.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.779+0000 7f0f65833740 10 client.5898 fetch_fsmap learned FSMap version 36 2024-09-17T04:27:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.779+0000 7f0f65833740 1 -- 172.21.15.100:0/2094852840 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x555695e4ad00 con 0x5556959c7590 2024-09-17T04:27:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.780+0000 7f0f44c00640 1 -- 172.21.15.100:0/2094852840 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f5002fb80 con 0x5556959c7590 2024-09-17T04:27:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.780+0000 7f0f44c00640 1 -- 172.21.15.100:0/2094852840 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7f0f4003a020 con 0x5556959c7590 2024-09-17T04:27:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.780+0000 7f0f65833740 10 client.5898 fetch_fsmap finished waiting for FSMap version 36 2024-09-17T04:27:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.780+0000 7f0f65833740 10 client.5898 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T04:27:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.780+0000 7f0f65833740 20 client.5898 populate_metadata read hostname 'smithi100' 2024-09-17T04:27:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.780+0000 7f0f65833740 1 --2- 172.21.15.100:0/2094852840 >> [v2:172.21.15.100:6833/270906112,v1:172.21.15.100:6835/270906112] conn(0x555695db0320 0x555695db1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.780+0000 7f0f65833740 4 client.5898 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T04:27:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.780+0000 7f0f65833740 1 -- 172.21.15.100:0/2094852840 --> [v2:172.21.15.100:6833/270906112,v1:172.21.15.100:6835/270906112] -- command(tid 0: {"prefix": "status"}) -- 0x555695cfffd0 con 0x555695db0320 2024-09-17T04:27:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.780+0000 7f0f57400640 1 --2- 172.21.15.100:0/2094852840 >> [v2:172.21.15.100:6833/270906112,v1:172.21.15.100:6835/270906112] conn(0x555695db0320 0x555695db1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.781+0000 7f0f57400640 1 --2- 172.21.15.100:0/2094852840 >> [v2:172.21.15.100:6833/270906112,v1:172.21.15.100:6835/270906112] conn(0x555695db0320 0x555695db1f20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcd2b5b567e83c6f server_cookie=4a1216fcaaff2d10 in_seq=0 out_seq=0 2024-09-17T04:27:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.781+0000 7f0f44c00640 10 client.5898 ms_handle_connect on v2:172.21.15.100:6833/270906112 2024-09-17T04:27:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.781+0000 7f0f44c00640 1 -- 172.21.15.100:0/2094852840 <== mds.0 v2:172.21.15.100:6833/270906112 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x555695cfffd0 con 0x555695db0320 2024-09-17T04:27:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.781+0000 7f0f44c00640 10 client.5898 handle_command_reply: tid=0 2024-09-17T04:27:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.781+0000 7f0f65833740 1 client.5898 shutdown 2024-09-17T04:27:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.781+0000 7f0f65833740 1 -- 172.21.15.100:0/2094852840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556959c7590 msgr2=0x555695daf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.781+0000 7f0f65833740 1 --2- 172.21.15.100:0/2094852840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556959c7590 0x555695daf620 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0f400054e0 tx=0x7f0f40000bc0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.781+0000 7f0f65833740 1 -- 172.21.15.100:0/2094852840 shutdown_connections 2024-09-17T04:27:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.781+0000 7f0f65833740 1 --2- 172.21.15.100:0/2094852840 >> [v2:172.21.15.100:6833/270906112,v1:172.21.15.100:6835/270906112] conn(0x555695db0320 0x555695db1f20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.781+0000 7f0f65833740 1 --2- 172.21.15.100:0/2094852840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5556959e8270 0x555695e4a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.781+0000 7f0f65833740 1 --2- 172.21.15.100:0/2094852840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556959c7590 0x555695daf620 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.781+0000 7f0f44c00640 0 client.5898 ms_handle_reset on v2:172.21.15.100:6833/270906112 2024-09-17T04:27:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.781+0000 7f0f65833740 1 --2- 172.21.15.100:0/2094852840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55569597c280 0x555695e49ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.781+0000 7f0f65833740 1 -- 172.21.15.100:0/2094852840 >> 172.21.15.100:0/2094852840 conn(0x5556959dc3c0 msgr2=0x555695db4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.781+0000 7f0f65833740 1 -- 172.21.15.100:0/2094852840 shutdown_connections 2024-09-17T04:27:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.781+0000 7f0f65833740 1 -- 172.21.15.100:0/2094852840 wait complete. 2024-09-17T04:27:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.781+0000 7f0f65833740 20 client.5898 trim_cache size 0 max 16384 2024-09-17T04:27:55.785 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:27:55.785 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "8a399842-3a5c-4ea2-9afb-b2704332e59f", 2024-09-17T04:27:55.785 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 0, 2024-09-17T04:27:55.785 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 5739, 2024-09-17T04:27:55.785 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-09-17T04:27:55.785 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-09-17T04:27:55.786 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-09-17T04:27:55.786 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 11.57341446, 2024-09-17T04:27:55.786 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 36, 2024-09-17T04:27:55.786 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 61, 2024-09-17T04:27:55.786 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 61, 2024-09-17T04:27:55.786 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 16.619375613999999 2024-09-17T04:27:55.786 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:27:55.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.782+0000 7f0f5d000640 1 -- 172.21.15.100:0/3879492713 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0f34078d70 msgr2=0x7f0f3407b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:55.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.782+0000 7f0f5d000640 1 --2- 172.21.15.100:0/3879492713 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0f34078d70 0x7f0f3407b280 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0f580b4d30 tx=0x7f0f40009040 comp rx=0 tx=0).stop 2024-09-17T04:27:55.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.782+0000 7f0f5d000640 1 -- 172.21.15.100:0/3879492713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f580ad9c0 msgr2=0x7f0f580cdb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:55.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.782+0000 7f0f5d000640 1 --2- 172.21.15.100:0/3879492713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f580ad9c0 0x7f0f580cdb10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800d770 tx=0x7f0f4800dc30 comp rx=0 tx=0).stop 2024-09-17T04:27:55.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.782+0000 7f0f5d000640 1 -- 172.21.15.100:0/3879492713 shutdown_connections 2024-09-17T04:27:55.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.782+0000 7f0f5d000640 1 --2- 172.21.15.100:0/3879492713 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0f34078d70 0x7f0f3407b280 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.782+0000 7f0f5d000640 1 --2- 172.21.15.100:0/3879492713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f5815a0f0 0x7f0f581f9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.782+0000 7f0f5d000640 1 --2- 172.21.15.100:0/3879492713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f580b2f60 0x7f0f581f8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.782+0000 7f0f5d000640 1 --2- 172.21.15.100:0/3879492713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f580ad9c0 0x7f0f580cdb10 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:55.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.782+0000 7f0f5d000640 1 -- 172.21.15.100:0/3879492713 >> 172.21.15.100:0/3879492713 conn(0x7f0f580b1e40 msgr2=0x7f0f580b0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:55.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.782+0000 7f0f5d000640 1 -- 172.21.15.100:0/3879492713 shutdown_connections 2024-09-17T04:27:55.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:55.782+0000 7f0f5d000640 1 -- 172.21.15.100:0/3879492713 wait complete. 2024-09-17T04:27:55.806 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:27:56.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.027+0000 7f617f200640 1 -- 172.21.15.100:0/1826331961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61800705f0 msgr2=0x7f61800cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:56.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.027+0000 7f617f200640 1 --2- 172.21.15.100:0/1826331961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61800705f0 0x7f61800cf3b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f616c009830 tx=0x7f616c02a510 comp rx=0 tx=0).stop 2024-09-17T04:27:56.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.027+0000 7f617f200640 1 -- 172.21.15.100:0/1826331961 shutdown_connections 2024-09-17T04:27:56.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.027+0000 7f617f200640 1 --2- 172.21.15.100:0/1826331961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61800705f0 0x7f61800cf3b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:56.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.027+0000 7f617f200640 1 --2- 172.21.15.100:0/1826331961 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f618015d7b0 0x7f61800c7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:56.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.027+0000 7f617f200640 1 -- 172.21.15.100:0/1826331961 >> 172.21.15.100:0/1826331961 conn(0x7f6180076350 msgr2=0x7f618006cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:56.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.027+0000 7f617f200640 1 -- 172.21.15.100:0/1826331961 shutdown_connections 2024-09-17T04:27:56.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.027+0000 7f617f200640 1 -- 172.21.15.100:0/1826331961 wait complete. 2024-09-17T04:27:56.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.029+0000 7f617f200640 1 Processor -- start 2024-09-17T04:27:56.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.029+0000 7f617f200640 1 -- start start 2024-09-17T04:27:56.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.031+0000 7f617f200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f618006d4b0 0x7f61800cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:56.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.031+0000 7f617f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61800705f0 0x7f61800cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:56.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.031+0000 7f617f200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f618015d7b0 0x7f61800731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:56.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.031+0000 7f617f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f61800c5dd0 con 0x7f61800705f0 2024-09-17T04:27:56.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.031+0000 7f617f200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f61800cfc70 con 0x7f618006d4b0 2024-09-17T04:27:56.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.031+0000 7f617f200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f61800c5f50 con 0x7f618015d7b0 2024-09-17T04:27:56.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.031+0000 7f617d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61800705f0 0x7f61800cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:56.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.031+0000 7f617de00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f618006d4b0 0x7f61800cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:56.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.031+0000 7f617d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61800705f0 0x7f61800cbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41634/0 (socket says 172.21.15.100:41634) 2024-09-17T04:27:56.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.031+0000 7f617d400640 1 -- 172.21.15.100:0/2728246211 learned_addr learned my addr 172.21.15.100:0/2728246211 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:56.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.031+0000 7f617e800640 1 --2- 172.21.15.100:0/2728246211 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f618015d7b0 0x7f61800731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:56.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.032+0000 7f617d400640 1 -- 172.21.15.100:0/2728246211 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f618015d7b0 msgr2=0x7f61800731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:56.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.032+0000 7f617d400640 1 --2- 172.21.15.100:0/2728246211 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f618015d7b0 0x7f61800731e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:56.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.032+0000 7f617d400640 1 -- 172.21.15.100:0/2728246211 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f618006d4b0 msgr2=0x7f61800cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:56.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.032+0000 7f617d400640 1 --2- 172.21.15.100:0/2728246211 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f618006d4b0 0x7f61800cacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:56.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.032+0000 7f617d400640 1 -- 172.21.15.100:0/2728246211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61801f8700 con 0x7f61800705f0 2024-09-17T04:27:56.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.032+0000 7f617d400640 1 --2- 172.21.15.100:0/2728246211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61800705f0 0x7f61800cbbe0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f616c000c00 tx=0x7f616c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:56.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.032+0000 7f6176000640 1 -- 172.21.15.100:0/2728246211 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c005c90 con 0x7f61800705f0 2024-09-17T04:27:56.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.032+0000 7f6176000640 1 -- 172.21.15.100:0/2728246211 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f616c031e50 con 0x7f61800705f0 2024-09-17T04:27:56.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.032+0000 7f617f200640 1 -- 172.21.15.100:0/2728246211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61801f9310 con 0x7f61800705f0 2024-09-17T04:27:56.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.032+0000 7f6176000640 1 -- 172.21.15.100:0/2728246211 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c0332f0 con 0x7f61800705f0 2024-09-17T04:27:56.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.032+0000 7f617f200640 1 -- 172.21.15.100:0/2728246211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61801f8cc0 con 0x7f61800705f0 2024-09-17T04:27:56.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.033+0000 7f617f200640 1 -- 172.21.15.100:0/2728246211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61800c6470 con 0x7f61800705f0 2024-09-17T04:27:56.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.035+0000 7f6176000640 1 -- 172.21.15.100:0/2728246211 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f616c033490 con 0x7f61800705f0 2024-09-17T04:27:56.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.036+0000 7f6176000640 1 --2- 172.21.15.100:0/2728246211 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6160078d70 0x7f616007b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:56.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.036+0000 7f6176000640 1 -- 172.21.15.100:0/2728246211 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7f616c0bc4b0 con 0x7f61800705f0 2024-09-17T04:27:56.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.036+0000 7f617de00640 1 --2- 172.21.15.100:0/2728246211 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6160078d70 0x7f616007b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:56.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.037+0000 7f617de00640 1 --2- 172.21.15.100:0/2728246211 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6160078d70 0x7f616007b280 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f61700095f0 tx=0x7f6170005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:56.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.040+0000 7f6176000640 1 -- 172.21.15.100:0/2728246211 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f616c03f030 con 0x7f61800705f0 2024-09-17T04:27:56.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.261+0000 7f617f200640 1 -- 172.21.15.100:0/2728246211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f61800cc120 con 0x7f61800705f0 2024-09-17T04:27:56.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.262+0000 7f6176000640 1 -- 172.21.15.100:0/2728246211 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) ==== 53+0+73 (secure 0 0 0) 0x7f616c083e70 con 0x7f61800705f0 2024-09-17T04:27:56.264 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:27:56.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.264+0000 7f617f200640 1 -- 172.21.15.100:0/2728246211 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6160078d70 msgr2=0x7f616007b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:56.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.264+0000 7f617f200640 1 --2- 172.21.15.100:0/2728246211 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6160078d70 0x7f616007b280 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f61700095f0 tx=0x7f6170005f90 comp rx=0 tx=0).stop 2024-09-17T04:27:56.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.264+0000 7f617f200640 1 -- 172.21.15.100:0/2728246211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61800705f0 msgr2=0x7f61800cbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:56.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.264+0000 7f617f200640 1 --2- 172.21.15.100:0/2728246211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61800705f0 0x7f61800cbbe0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f616c000c00 tx=0x7f616c033c20 comp rx=0 tx=0).stop 2024-09-17T04:27:56.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.264+0000 7f617f200640 1 -- 172.21.15.100:0/2728246211 shutdown_connections 2024-09-17T04:27:56.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.264+0000 7f617f200640 1 --2- 172.21.15.100:0/2728246211 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6160078d70 0x7f616007b280 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:56.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.264+0000 7f617f200640 1 --2- 172.21.15.100:0/2728246211 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f618015d7b0 0x7f61800731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:56.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.264+0000 7f617f200640 1 --2- 172.21.15.100:0/2728246211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61800705f0 0x7f61800cbbe0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:56.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.264+0000 7f617f200640 1 --2- 172.21.15.100:0/2728246211 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f618006d4b0 0x7f61800cacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:56.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.264+0000 7f617f200640 1 -- 172.21.15.100:0/2728246211 >> 172.21.15.100:0/2728246211 conn(0x7f6180076350 msgr2=0x7f61800b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:56.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.264+0000 7f617f200640 1 -- 172.21.15.100:0/2728246211 shutdown_connections 2024-09-17T04:27:56.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56.264+0000 7f617f200640 1 -- 172.21.15.100:0/2728246211 wait complete. 2024-09-17T04:27:56.287 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T04:27:56.287 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:27:56.287 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-09-17T04:27:56.287 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T04:27:56.287 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T04:27:56.287 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T04:27:56.287 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T04:27:56.287 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:27:56.288 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:27:56.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:27:56.288 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-09-17T04:27:56.323 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:27:56.324 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:27:56.324 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:27:56.324 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:27:56.324 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-17T04:27:56.324 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:27:56.324 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:27:56.324 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:27:56.324 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-09-17T04:27:56.324 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:27:56.324 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:27:56.324 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-09-17T04:27:56.324 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:27:56.325 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:27:56.325 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:27:56.325 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:27:56.325 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T04:27:56.325 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42484sec preferred_lft 42484sec 2024-09-17T04:27:56.325 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link noprefixroute 2024-09-17T04:27:56.325 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:27:56.325 INFO:teuthology.orchestra.run.smithi100.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:27:56.325 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:2e:de:8e:a4:7c brd ff:ff:ff:ff:ff:ff 2024-09-17T04:27:56.325 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T04:27:56.325 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:27:56.325 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::cc78:e4ff:fe70:7fbe/64 scope link 2024-09-17T04:27:56.325 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:27:56.326 INFO:teuthology.orchestra.run.smithi100.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:27:56.326 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether ea:8c:45:9f:34:87 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:27:56.326 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::e88c:45ff:fe9f:3487/64 scope link 2024-09-17T04:27:56.326 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:27:56.326 INFO:teuthology.orchestra.run.smithi100.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:27:56.326 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:2e:de:8e:a4:7c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:27:56.326 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::c2e:deff:fe8e:a47c/64 scope link 2024-09-17T04:27:56.326 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:27:56.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:27:56.326 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:27:56.417 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-09-17T04:27:56.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:27:56.420 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-09-17T04:27:56.456 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-09-17T04:27:56.456 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:27:56.456 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T04:27:56.456 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T04:27:56.456 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:27:56.457 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:27:56.457 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-17T04:27:56.457 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T04:27:56.457 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T04:27:56.457 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T04:27:56.457 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-17T04:27:56.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:27:56.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:27:56.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:27:56.651 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-17T04:27:56.686 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T04:27:56.686 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T04:27:56.686 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:27:56.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:27:56.687 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-09-17T04:27:56.782 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:27:56.782 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T04:27:56.817 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:27:56.817 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:27:56.817 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:27:56.892 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T04:27:56.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:27:56.893 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:27:57.087 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:27:57.088 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:27:57.088 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:27:57.109 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:27:57.109 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:27:57.109 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:27:57.109 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:27:57.110 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:27:57.144 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:27:57.144 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:27:57.144 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:27:57.144 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:27:57.144 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:27:57.144 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-09-17T04:27:57.144 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-09-17T04:27:57.145 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8a399842-3a5c-4ea2-9afb-b2704332e59f" will pass to kernel 2024-09-17T04:27:57.146 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-17T04:27:57.146 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-09-17T04:27:57.146 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-17T04:27:57.146 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-09-17T04:27:57.146 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-17T04:27:57.149 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T04:27:57.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:27:57.149 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:27:57.206 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:27:57.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:27:57.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:27:57.300 INFO:teuthology.orchestra.run.smithi100.stdout:client5794 2024-09-17T04:27:57.301 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:27:57.301 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:27:57.336 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:27:57.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.598+0000 7f7d1b400640 1 -- 172.21.15.100:0/2961149046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d1415d760 msgr2=0x7f7d141535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:57.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.598+0000 7f7d1b400640 1 --2- 172.21.15.100:0/2961149046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d1415d760 0x7f7d141535c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009830 tx=0x7f7d0402a480 comp rx=0 tx=0).stop 2024-09-17T04:27:57.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.598+0000 7f7d1b400640 1 -- 172.21.15.100:0/2961149046 shutdown_connections 2024-09-17T04:27:57.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.598+0000 7f7d1b400640 1 --2- 172.21.15.100:0/2961149046 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d140ad0e0 0x7f7d140ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:57.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.598+0000 7f7d1b400640 1 --2- 172.21.15.100:0/2961149046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d1415d760 0x7f7d141535c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:57.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.598+0000 7f7d1b400640 1 -- 172.21.15.100:0/2961149046 >> 172.21.15.100:0/2961149046 conn(0x7f7d1406d730 msgr2=0x7f7d140b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:57.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.598+0000 7f7d1b400640 1 -- 172.21.15.100:0/2961149046 shutdown_connections 2024-09-17T04:27:57.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.598+0000 7f7d1b400640 1 -- 172.21.15.100:0/2961149046 wait complete. 2024-09-17T04:27:57.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.600+0000 7f7d1b400640 1 Processor -- start 2024-09-17T04:27:57.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.600+0000 7f7d1b400640 1 -- start start 2024-09-17T04:27:57.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.602+0000 7f7d1b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d140ad0e0 0x7f7d14146860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:57.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.602+0000 7f7d1b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d1415d760 0x7f7d14155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:57.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.602+0000 7f7d1b400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d14160310 0x7f7d1409d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:57.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.602+0000 7f7d1b400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1406f220 con 0x7f7d1415d760 2024-09-17T04:27:57.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.602+0000 7f7d1b400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1406f0a0 con 0x7f7d140ad0e0 2024-09-17T04:27:57.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.602+0000 7f7d1b400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7d140b4c20 con 0x7f7d14160310 2024-09-17T04:27:57.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.602+0000 7f7d0be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d1415d760 0x7f7d14155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:57.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.602+0000 7f7d18a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d140ad0e0 0x7f7d14146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:57.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.602+0000 7f7d0be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d1415d760 0x7f7d14155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41650/0 (socket says 172.21.15.100:41650) 2024-09-17T04:27:57.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.602+0000 7f7d0be00640 1 -- 172.21.15.100:0/3562002628 learned_addr learned my addr 172.21.15.100:0/3562002628 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:57.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.602+0000 7f7d19400640 1 --2- 172.21.15.100:0/3562002628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d14160310 0x7f7d1409d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:57.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.603+0000 7f7d0be00640 1 -- 172.21.15.100:0/3562002628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d14160310 msgr2=0x7f7d1409d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:57.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.603+0000 7f7d0be00640 1 --2- 172.21.15.100:0/3562002628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d14160310 0x7f7d1409d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:57.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.603+0000 7f7d0be00640 1 -- 172.21.15.100:0/3562002628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d140ad0e0 msgr2=0x7f7d14146860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:57.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.603+0000 7f7d0be00640 1 --2- 172.21.15.100:0/3562002628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d140ad0e0 0x7f7d14146860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:57.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.603+0000 7f7d0be00640 1 -- 172.21.15.100:0/3562002628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d14146da0 con 0x7f7d1415d760 2024-09-17T04:27:57.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.603+0000 7f7d18a00640 1 --2- 172.21.15.100:0/3562002628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d140ad0e0 0x7f7d14146860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:27:57.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.603+0000 7f7d0be00640 1 --2- 172.21.15.100:0/3562002628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d1415d760 0x7f7d14155fe0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00e790 tx=0x7f7cfc00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:57.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.603+0000 7f7d11600640 1 -- 172.21.15.100:0/3562002628 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc00eea0 con 0x7f7d1415d760 2024-09-17T04:27:57.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.603+0000 7f7d11600640 1 -- 172.21.15.100:0/3562002628 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cfc00ccc0 con 0x7f7d1415d760 2024-09-17T04:27:57.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.603+0000 7f7d11600640 1 -- 172.21.15.100:0/3562002628 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc004280 con 0x7f7d1415d760 2024-09-17T04:27:57.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.603+0000 7f7d1b400640 1 -- 172.21.15.100:0/3562002628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d14156d50 con 0x7f7d1415d760 2024-09-17T04:27:57.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.603+0000 7f7d1b400640 1 -- 172.21.15.100:0/3562002628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d141542c0 con 0x7f7d1415d760 2024-09-17T04:27:57.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.604+0000 7f7d1b400640 1 -- 172.21.15.100:0/3562002628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d14156ac0 con 0x7f7d1415d760 2024-09-17T04:27:57.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.606+0000 7f7d11600640 1 -- 172.21.15.100:0/3562002628 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f7cfc0035f0 con 0x7f7d1415d760 2024-09-17T04:27:57.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.606+0000 7f7d11600640 1 --2- 172.21.15.100:0/3562002628 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7cf0078d70 0x7f7cf007b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:57.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.607+0000 7f7d11600640 1 -- 172.21.15.100:0/3562002628 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7f7cfc09b9b0 con 0x7f7d1415d760 2024-09-17T04:27:57.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.607+0000 7f7d18a00640 1 --2- 172.21.15.100:0/3562002628 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7cf0078d70 0x7f7cf007b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:57.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.608+0000 7f7d18a00640 1 --2- 172.21.15.100:0/3562002628 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7cf0078d70 0x7f7cf007b280 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7d04034000 tx=0x7f7d04004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:57.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.611+0000 7f7d11600640 1 -- 172.21.15.100:0/3562002628 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f7d14146da0 con 0x7f7d1415d760 2024-09-17T04:27:57.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.830+0000 7f7d1b400640 1 -- 172.21.15.100:0/3562002628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7d14160fa0 con 0x7f7d1415d760 2024-09-17T04:27:57.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.831+0000 7f7d11600640 1 -- 172.21.15.100:0/3562002628 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) ==== 53+0+73 (secure 0 0 0) 0x7f7cfc063380 con 0x7f7d1415d760 2024-09-17T04:27:57.830 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:27:57.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.833+0000 7f7d1b400640 1 -- 172.21.15.100:0/3562002628 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7cf0078d70 msgr2=0x7f7cf007b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:57.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.833+0000 7f7d1b400640 1 --2- 172.21.15.100:0/3562002628 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7cf0078d70 0x7f7cf007b280 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7d04034000 tx=0x7f7d04004730 comp rx=0 tx=0).stop 2024-09-17T04:27:57.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.833+0000 7f7d1b400640 1 -- 172.21.15.100:0/3562002628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d1415d760 msgr2=0x7f7d14155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:57.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.833+0000 7f7d1b400640 1 --2- 172.21.15.100:0/3562002628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d1415d760 0x7f7d14155fe0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00e790 tx=0x7f7cfc00ec50 comp rx=0 tx=0).stop 2024-09-17T04:27:57.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.833+0000 7f7d1b400640 1 -- 172.21.15.100:0/3562002628 shutdown_connections 2024-09-17T04:27:57.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.833+0000 7f7d1b400640 1 --2- 172.21.15.100:0/3562002628 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7cf0078d70 0x7f7cf007b280 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:57.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.833+0000 7f7d1b400640 1 --2- 172.21.15.100:0/3562002628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d14160310 0x7f7d1409d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:57.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.833+0000 7f7d1b400640 1 --2- 172.21.15.100:0/3562002628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d1415d760 0x7f7d14155fe0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:57.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.833+0000 7f7d1b400640 1 --2- 172.21.15.100:0/3562002628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d140ad0e0 0x7f7d14146860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:57.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.833+0000 7f7d1b400640 1 -- 172.21.15.100:0/3562002628 >> 172.21.15.100:0/3562002628 conn(0x7f7d1406d730 msgr2=0x7f7d14160720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:57.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.833+0000 7f7d1b400640 1 -- 172.21.15.100:0/3562002628 shutdown_connections 2024-09-17T04:27:57.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:57.833+0000 7f7d1b400640 1 -- 172.21.15.100:0/3562002628 wait complete. 2024-09-17T04:27:57.859 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T04:27:57.859 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:27:57.859 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2024-09-17T04:27:57.859 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T04:27:57.859 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T04:27:57.859 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T04:27:57.859 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T04:27:57.859 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:27:57.859 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:27:57.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:27:57.860 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2024-09-17T04:27:57.901 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:27:57.901 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:27:57.901 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:27:57.901 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:27:57.902 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-09-17T04:27:57.902 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:27:57.902 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:27:57.902 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:27:57.902 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2024-09-17T04:27:57.902 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:27:57.902 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:27:57.902 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2024-09-17T04:27:57.902 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:27:57.902 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2024-09-17T04:27:57.902 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:27:57.902 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2024-09-17T04:27:57.902 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T04:27:57.903 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42466sec preferred_lft 42466sec 2024-09-17T04:27:57.903 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ec4:7aff:fe88:7b8f/64 scope link noprefixroute 2024-09-17T04:27:57.903 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:27:57.903 INFO:teuthology.orchestra.run.smithi184.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:27:57.903 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 7e:67:e5:69:96:69 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:27:57.903 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T04:27:57.903 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:27:57.903 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::3c9d:aff:fe6e:8fcb/64 scope link 2024-09-17T04:27:57.903 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:27:57.903 INFO:teuthology.orchestra.run.smithi184.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:27:57.903 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether a6:a9:1b:a9:61:ec brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T04:27:57.903 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::a4a9:1bff:fea9:61ec/64 scope link 2024-09-17T04:27:57.903 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:27:57.904 INFO:teuthology.orchestra.run.smithi184.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:27:57.904 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 7e:67:e5:69:96:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T04:27:57.904 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::7c67:e5ff:fe69:9669/64 scope link 2024-09-17T04:27:57.904 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:27:57.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:27:57.904 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:27:58.000 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-09-17T04:27:58.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:27:58.004 DEBUG:teuthology.orchestra.run.smithi184:> route 2024-09-17T04:27:58.042 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2024-09-17T04:27:58.042 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:27:58.042 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T04:27:58.042 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T04:27:58.042 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:27:58.043 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:27:58.043 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-09-17T04:27:58.043 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T04:27:58.043 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T04:27:58.043 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T04:27:58.043 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-09-17T04:27:58.161 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:27:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:27:58.231 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:27:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:27:58.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:27:58.240 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-09-17T04:27:58.275 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T04:27:58.275 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T04:27:58.276 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:27:58.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:27:58.276 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.0 up 2024-09-17T04:27:58.372 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:27:58.372 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T04:27:58.407 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:27:58.407 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:27:58.408 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T04:27:58.482 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T04:27:58.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:27:58.482 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:27:58.676 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:27:58.676 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:27:58.676 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:27:58.696 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:27:58.696 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:27:58.696 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:27:58.696 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:27:58.696 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:27:58.727 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:27:58.727 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:27:58.727 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:27:58.727 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:27:58.727 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:27:58.727 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-09-17T04:27:58.727 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-09-17T04:27:58.727 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8a399842-3a5c-4ea2-9afb-b2704332e59f" will pass to kernel 2024-09-17T04:27:58.728 INFO:teuthology.orchestra.run.smithi184.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-17T04:27:58.728 INFO:teuthology.orchestra.run.smithi184.stdout: You just mounted a file system that supports labels which does not 2024-09-17T04:27:58.729 INFO:teuthology.orchestra.run.smithi184.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-17T04:27:58.729 INFO:teuthology.orchestra.run.smithi184.stdout: applications will generate AVC messages and not be allowed access to 2024-09-17T04:27:58.729 INFO:teuthology.orchestra.run.smithi184.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-17T04:27:58.731 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T04:27:58.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:27:58.732 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T04:27:58.787 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:27:58.865 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:27:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:27:58.869 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:27:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:27:58.876 INFO:teuthology.orchestra.run.smithi184.stdout:client5910 2024-09-17T04:27:58.877 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:27:58.877 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:27:58.912 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T04:27:59.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.146+0000 7f4fe3600640 1 -- 172.21.15.100:0/816850998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fdc0cc830 msgr2=0x7f4fdc0b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:59.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.146+0000 7f4fe3600640 1 --2- 172.21.15.100:0/816850998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fdc0cc830 0x7f4fdc0b4620 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0d0d00 tx=0x7f4fd002a5f0 comp rx=0 tx=0).stop 2024-09-17T04:27:59.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.146+0000 7f4fe3600640 1 -- 172.21.15.100:0/816850998 shutdown_connections 2024-09-17T04:27:59.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.146+0000 7f4fe3600640 1 --2- 172.21.15.100:0/816850998 >> v2:172.21.15.184:3300/0 conn(0x7f4fdc06cda0 0x7f4fdc06d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:59.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.146+0000 7f4fe3600640 1 --2- 172.21.15.100:0/816850998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fdc0cc830 0x7f4fdc0b4620 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:59.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.146+0000 7f4fe3600640 1 -- 172.21.15.100:0/816850998 >> 172.21.15.100:0/816850998 conn(0x7f4fdc0b2540 msgr2=0x7f4fdc0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:59.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.146+0000 7f4fe3600640 1 -- 172.21.15.100:0/816850998 shutdown_connections 2024-09-17T04:27:59.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.147+0000 7f4fe3600640 1 -- 172.21.15.100:0/816850998 wait complete. 2024-09-17T04:27:59.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.149+0000 7f4fe3600640 1 Processor -- start 2024-09-17T04:27:59.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.149+0000 7f4fe3600640 1 -- start start 2024-09-17T04:27:59.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.150+0000 7f4fe3600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fdc06cda0 0x7f4fdc1f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:59.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.150+0000 7f4fe3600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fdc0af100 0x7f4fdc1530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:59.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.150+0000 7f4fe3600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fdc0cc830 0x7f4fdc1535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:59.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.150+0000 7f4fe3600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4fdc0b35f0 con 0x7f4fdc06cda0 2024-09-17T04:27:59.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.150+0000 7f4fe3600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4fdc0b3470 con 0x7f4fdc0cc830 2024-09-17T04:27:59.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.150+0000 7f4fe3600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4fdc0b3770 con 0x7f4fdc0af100 2024-09-17T04:27:59.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.150+0000 7f4fe0c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fdc06cda0 0x7f4fdc1f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:59.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.151+0000 7f4fe0c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fdc06cda0 0x7f4fdc1f5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41670/0 (socket says 172.21.15.100:41670) 2024-09-17T04:27:59.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.151+0000 7f4fe0c00640 1 -- 172.21.15.100:0/418855111 learned_addr learned my addr 172.21.15.100:0/418855111 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:59.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.151+0000 7f4fdbe00640 1 --2- 172.21.15.100:0/418855111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fdc0af100 0x7f4fdc1530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:59.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.151+0000 7f4fe1600640 1 --2- 172.21.15.100:0/418855111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fdc0cc830 0x7f4fdc1535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:59.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.151+0000 7f4fe0c00640 1 -- 172.21.15.100:0/418855111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fdc0af100 msgr2=0x7f4fdc1530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:59.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.151+0000 7f4fe0c00640 1 --2- 172.21.15.100:0/418855111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fdc0af100 0x7f4fdc1530a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:59.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.151+0000 7f4fe0c00640 1 -- 172.21.15.100:0/418855111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fdc0cc830 msgr2=0x7f4fdc1535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:59.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.151+0000 7f4fe0c00640 1 --2- 172.21.15.100:0/418855111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fdc0cc830 0x7f4fdc1535e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:59.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.151+0000 7f4fe0c00640 1 -- 172.21.15.100:0/418855111 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fdc1f5fa0 con 0x7f4fdc06cda0 2024-09-17T04:27:59.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.151+0000 7f4fdbe00640 1 --2- 172.21.15.100:0/418855111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fdc0af100 0x7f4fdc1530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:27:59.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.151+0000 7f4fe1600640 1 --2- 172.21.15.100:0/418855111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fdc0cc830 0x7f4fdc1535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:27:59.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.151+0000 7f4fe0c00640 1 --2- 172.21.15.100:0/418855111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fdc06cda0 0x7f4fdc1f5a60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0034040 tx=0x7f4fd0003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:59.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.152+0000 7f4fd8c00640 1 -- 172.21.15.100:0/418855111 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd003e030 con 0x7f4fdc06cda0 2024-09-17T04:27:59.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.152+0000 7f4fd8c00640 1 -- 172.21.15.100:0/418855111 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4fd0003d10 con 0x7f4fdc06cda0 2024-09-17T04:27:59.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.152+0000 7f4fe3600640 1 -- 172.21.15.100:0/418855111 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fdc09b4f0 con 0x7f4fdc06cda0 2024-09-17T04:27:59.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.152+0000 7f4fd8c00640 1 -- 172.21.15.100:0/418855111 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0035040 con 0x7f4fdc06cda0 2024-09-17T04:27:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.152+0000 7f4fe3600640 1 -- 172.21.15.100:0/418855111 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fdc09dee0 con 0x7f4fdc06cda0 2024-09-17T04:27:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.153+0000 7f4fe3600640 1 -- 172.21.15.100:0/418855111 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fdc0b3c90 con 0x7f4fdc06cda0 2024-09-17T04:27:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.154+0000 7f4fd8c00640 1 -- 172.21.15.100:0/418855111 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f4fd0041030 con 0x7f4fdc06cda0 2024-09-17T04:27:59.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.155+0000 7f4fd8c00640 1 --2- 172.21.15.100:0/418855111 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4fbc078d70 0x7f4fbc07b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:59.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.155+0000 7f4fd8c00640 1 -- 172.21.15.100:0/418855111 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7f4fd00bb940 con 0x7f4fdc06cda0 2024-09-17T04:27:59.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.155+0000 7f4fdbe00640 1 --2- 172.21.15.100:0/418855111 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4fbc078d70 0x7f4fbc07b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:59.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.156+0000 7f4fdbe00640 1 --2- 172.21.15.100:0/418855111 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4fbc078d70 0x7f4fbc07b280 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc09d850 tx=0x7f4fc8005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:59.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.159+0000 7f4fd8c00640 1 -- 172.21.15.100:0/418855111 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4fdc09b4f0 con 0x7f4fdc06cda0 2024-09-17T04:27:59.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.381+0000 7f4fe3600640 1 -- 172.21.15.100:0/418855111 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f4fdc0d1030 con 0x7f4fdc06cda0 2024-09-17T04:27:59.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.434+0000 7f4fd8c00640 1 -- 172.21.15.100:0/418855111 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v37) ==== 136+0+0 (secure 0 0 0) 0x7f4fd00839e0 con 0x7f4fdc06cda0 2024-09-17T04:27:59.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.436+0000 7f4fe3600640 1 -- 172.21.15.100:0/418855111 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4fbc078d70 msgr2=0x7f4fbc07b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:59.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.436+0000 7f4fe3600640 1 --2- 172.21.15.100:0/418855111 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4fbc078d70 0x7f4fbc07b280 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc09d850 tx=0x7f4fc8005f90 comp rx=0 tx=0).stop 2024-09-17T04:27:59.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.436+0000 7f4fe3600640 1 -- 172.21.15.100:0/418855111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fdc06cda0 msgr2=0x7f4fdc1f5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:59.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.436+0000 7f4fe3600640 1 --2- 172.21.15.100:0/418855111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fdc06cda0 0x7f4fdc1f5a60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0034040 tx=0x7f4fd0003820 comp rx=0 tx=0).stop 2024-09-17T04:27:59.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.437+0000 7f4fe3600640 1 -- 172.21.15.100:0/418855111 shutdown_connections 2024-09-17T04:27:59.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.437+0000 7f4fe3600640 1 --2- 172.21.15.100:0/418855111 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4fbc078d70 0x7f4fbc07b280 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:59.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.437+0000 7f4fe3600640 1 --2- 172.21.15.100:0/418855111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fdc0cc830 0x7f4fdc1535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:59.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.437+0000 7f4fe3600640 1 --2- 172.21.15.100:0/418855111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fdc0af100 0x7f4fdc1530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:59.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.437+0000 7f4fe3600640 1 --2- 172.21.15.100:0/418855111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fdc06cda0 0x7f4fdc1f5a60 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:59.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.437+0000 7f4fe3600640 1 -- 172.21.15.100:0/418855111 >> 172.21.15.100:0/418855111 conn(0x7f4fdc0b2540 msgr2=0x7f4fdc0ca1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:59.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.437+0000 7f4fe3600640 1 -- 172.21.15.100:0/418855111 shutdown_connections 2024-09-17T04:27:59.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.437+0000 7f4fe3600640 1 -- 172.21.15.100:0/418855111 wait complete. 2024-09-17T04:27:59.458 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T04:27:59.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.677+0000 7fc477200640 1 -- 172.21.15.100:0/2088118123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc47806f8b0 msgr2=0x7fc4780b88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:59.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.677+0000 7fc477200640 1 --2- 172.21.15.100:0/2088118123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc47806f8b0 0x7fc4780b88b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc4780b70c0 tx=0x7fc47002aac0 comp rx=0 tx=0).stop 2024-09-17T04:27:59.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.677+0000 7fc477200640 1 -- 172.21.15.100:0/2088118123 shutdown_connections 2024-09-17T04:27:59.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.677+0000 7fc477200640 1 --2- 172.21.15.100:0/2088118123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc47806f8b0 0x7fc4780b88b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:59.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.677+0000 7fc477200640 1 --2- 172.21.15.100:0/2088118123 >> v2:172.21.15.184:3300/0 conn(0x7fc4780b9570 0x7fc4780d28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:59.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.677+0000 7fc477200640 1 -- 172.21.15.100:0/2088118123 >> 172.21.15.100:0/2088118123 conn(0x7fc47809d880 msgr2=0x7fc47808bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:59.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.677+0000 7fc477200640 1 -- 172.21.15.100:0/2088118123 shutdown_connections 2024-09-17T04:27:59.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.677+0000 7fc477200640 1 -- 172.21.15.100:0/2088118123 wait complete. 2024-09-17T04:27:59.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.679+0000 7fc477200640 1 Processor -- start 2024-09-17T04:27:59.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.679+0000 7fc477200640 1 -- start start 2024-09-17T04:27:59.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.681+0000 7fc477200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc47806f8b0 0x7fc4781f6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:59.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.681+0000 7fc477200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4780b9570 0x7fc4781f87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:59.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.681+0000 7fc477200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4780d2de0 0x7fc4780b5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:59.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.681+0000 7fc477200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc478162350 con 0x7fc4780b9570 2024-09-17T04:27:59.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.681+0000 7fc477200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc4781621d0 con 0x7fc4780d2de0 2024-09-17T04:27:59.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.681+0000 7fc477200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc4780b8df0 con 0x7fc47806f8b0 2024-09-17T04:27:59.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.681+0000 7fc475400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4780b9570 0x7fc4781f87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:59.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.681+0000 7fc475400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4780b9570 0x7fc4781f87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41686/0 (socket says 172.21.15.100:41686) 2024-09-17T04:27:59.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.681+0000 7fc475400640 1 -- 172.21.15.100:0/3815269262 learned_addr learned my addr 172.21.15.100:0/3815269262 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:27:59.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.681+0000 7fc476800640 1 --2- 172.21.15.100:0/3815269262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4780d2de0 0x7fc4780b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:59.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.681+0000 7fc475e00640 1 --2- 172.21.15.100:0/3815269262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc47806f8b0 0x7fc4781f6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:59.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.681+0000 7fc475400640 1 -- 172.21.15.100:0/3815269262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc47806f8b0 msgr2=0x7fc4781f6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:59.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.681+0000 7fc475400640 1 --2- 172.21.15.100:0/3815269262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc47806f8b0 0x7fc4781f6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:59.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.681+0000 7fc475400640 1 -- 172.21.15.100:0/3815269262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4780d2de0 msgr2=0x7fc4780b5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:59.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.681+0000 7fc475400640 1 --2- 172.21.15.100:0/3815269262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4780d2de0 0x7fc4780b5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:59.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.681+0000 7fc475400640 1 -- 172.21.15.100:0/3815269262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc478155f20 con 0x7fc4780b9570 2024-09-17T04:27:59.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.682+0000 7fc475400640 1 --2- 172.21.15.100:0/3815269262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4780b9570 0x7fc4781f87c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc46400c3d0 tx=0x7fc46400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:59.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.682+0000 7fc46e000640 1 -- 172.21.15.100:0/3815269262 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc464017070 con 0x7fc4780b9570 2024-09-17T04:27:59.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.682+0000 7fc46e000640 1 -- 172.21.15.100:0/3815269262 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4640053c0 con 0x7fc4780b9570 2024-09-17T04:27:59.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.682+0000 7fc46e000640 1 -- 172.21.15.100:0/3815269262 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc464003850 con 0x7fc4780b9570 2024-09-17T04:27:59.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.682+0000 7fc477200640 1 -- 172.21.15.100:0/3815269262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4780cac20 con 0x7fc4780b9570 2024-09-17T04:27:59.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.682+0000 7fc477200640 1 -- 172.21.15.100:0/3815269262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4780d1ad0 con 0x7fc4780b9570 2024-09-17T04:27:59.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.683+0000 7fc477200640 1 -- 172.21.15.100:0/3815269262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4780b92a0 con 0x7fc4780b9570 2024-09-17T04:27:59.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.684+0000 7fc46e000640 1 -- 172.21.15.100:0/3815269262 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7fc46401a070 con 0x7fc4780b9570 2024-09-17T04:27:59.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.685+0000 7fc46e000640 1 --2- 172.21.15.100:0/3815269262 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc458078d70 0x7fc45807b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:27:59.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.686+0000 7fc46e000640 1 -- 172.21.15.100:0/3815269262 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7fc46409a450 con 0x7fc4780b9570 2024-09-17T04:27:59.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.686+0000 7fc475e00640 1 --2- 172.21.15.100:0/3815269262 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc458078d70 0x7fc45807b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:27:59.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.687+0000 7fc475e00640 1 --2- 172.21.15.100:0/3815269262 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc458078d70 0x7fc45807b280 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc4780ba230 tx=0x7fc468005f30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:27:59.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.690+0000 7fc46e000640 1 -- 172.21.15.100:0/3815269262 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc4780d31f0 con 0x7fc4780b9570 2024-09-17T04:27:59.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.894+0000 7fc477200640 1 -- 172.21.15.100:0/3815269262 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fc4780b18e0 con 0x7fc458078d70 2024-09-17T04:27:59.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.896+0000 7fc46e000640 1 -- 172.21.15.100:0/3815269262 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fc4780b18e0 con 0x7fc458078d70 2024-09-17T04:27:59.897 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-09-17T04:27:59.897 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:27:59.897 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephfs" 2024-09-17T04:27:59.897 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:27:59.897 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-09-17T04:27:59.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.899+0000 7fc477200640 1 -- 172.21.15.100:0/3815269262 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc458078d70 msgr2=0x7fc45807b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:59.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.899+0000 7fc477200640 1 --2- 172.21.15.100:0/3815269262 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc458078d70 0x7fc45807b280 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc4780ba230 tx=0x7fc468005f30 comp rx=0 tx=0).stop 2024-09-17T04:27:59.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.899+0000 7fc477200640 1 -- 172.21.15.100:0/3815269262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4780b9570 msgr2=0x7fc4781f87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:27:59.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.899+0000 7fc477200640 1 --2- 172.21.15.100:0/3815269262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4780b9570 0x7fc4781f87c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc46400c3d0 tx=0x7fc46400c890 comp rx=0 tx=0).stop 2024-09-17T04:27:59.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.899+0000 7fc477200640 1 -- 172.21.15.100:0/3815269262 shutdown_connections 2024-09-17T04:27:59.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.899+0000 7fc477200640 1 --2- 172.21.15.100:0/3815269262 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc458078d70 0x7fc45807b280 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:59.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.899+0000 7fc477200640 1 --2- 172.21.15.100:0/3815269262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4780d2de0 0x7fc4780b5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:59.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.899+0000 7fc477200640 1 --2- 172.21.15.100:0/3815269262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4780b9570 0x7fc4781f87c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:59.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.899+0000 7fc477200640 1 --2- 172.21.15.100:0/3815269262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc47806f8b0 0x7fc4781f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:27:59.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.899+0000 7fc477200640 1 -- 172.21.15.100:0/3815269262 >> 172.21.15.100:0/3815269262 conn(0x7fc47809d880 msgr2=0x7fc4780cd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:27:59.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.899+0000 7fc477200640 1 -- 172.21.15.100:0/3815269262 shutdown_connections 2024-09-17T04:27:59.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:27:59.899+0000 7fc477200640 1 -- 172.21.15.100:0/3815269262 wait complete. 2024-09-17T04:27:59.922 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T04:28:00.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.145+0000 7f6993200640 1 -- 172.21.15.100:0/3658413117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f698c0cd610 msgr2=0x7f698c0cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:00.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.145+0000 7f6993200640 1 --2- 172.21.15.100:0/3658413117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f698c0cd610 0x7f698c0cda20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f698c071590 tx=0x7f698402aac0 comp rx=0 tx=0).stop 2024-09-17T04:28:00.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.146+0000 7f6993200640 1 -- 172.21.15.100:0/3658413117 shutdown_connections 2024-09-17T04:28:00.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.146+0000 7f6993200640 1 --2- 172.21.15.100:0/3658413117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f698c0cd610 0x7f698c0cda20 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:00.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.146+0000 7f6993200640 1 --2- 172.21.15.100:0/3658413117 >> v2:172.21.15.184:3300/0 conn(0x7f698c0735e0 0x7f698c0739f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:00.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.146+0000 7f6993200640 1 -- 172.21.15.100:0/3658413117 >> 172.21.15.100:0/3658413117 conn(0x7f698c06d1c0 msgr2=0x7f698c06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:00.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.146+0000 7f6993200640 1 -- 172.21.15.100:0/3658413117 shutdown_connections 2024-09-17T04:28:00.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.146+0000 7f6993200640 1 -- 172.21.15.100:0/3658413117 wait complete. 2024-09-17T04:28:00.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.148+0000 7f6993200640 1 Processor -- start 2024-09-17T04:28:00.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.148+0000 7f6993200640 1 -- start start 2024-09-17T04:28:00.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.149+0000 7f6993200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f698c0735e0 0x7f698c1f61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:00.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.149+0000 7f6993200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f698c0cd610 0x7f698c15c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:00.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.150+0000 7f6993200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f698c145450 0x7f698c0aabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:00.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.150+0000 7f6993200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f698c0ce2f0 con 0x7f698c0cd610 2024-09-17T04:28:00.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.150+0000 7f6993200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f698c0ce170 con 0x7f698c145450 2024-09-17T04:28:00.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.150+0000 7f6993200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f698c0ce470 con 0x7f698c0735e0 2024-09-17T04:28:00.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.150+0000 7f698b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f698c0cd610 0x7f698c15c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:00.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.150+0000 7f698b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f698c0cd610 0x7f698c15c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41708/0 (socket says 172.21.15.100:41708) 2024-09-17T04:28:00.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.150+0000 7f698b400640 1 -- 172.21.15.100:0/4136175755 learned_addr learned my addr 172.21.15.100:0/4136175755 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:00.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.150+0000 7f6991200640 1 --2- 172.21.15.100:0/4136175755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f698c145450 0x7f698c0aabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:00.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.150+0000 7f698be00640 1 --2- 172.21.15.100:0/4136175755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f698c0735e0 0x7f698c1f61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:00.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.150+0000 7f698b400640 1 -- 172.21.15.100:0/4136175755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f698c0735e0 msgr2=0x7f698c1f61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:00.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.150+0000 7f698b400640 1 --2- 172.21.15.100:0/4136175755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f698c0735e0 0x7f698c1f61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:00.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.150+0000 7f698b400640 1 -- 172.21.15.100:0/4136175755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f698c145450 msgr2=0x7f698c0aabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:00.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.150+0000 7f698b400640 1 --2- 172.21.15.100:0/4136175755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f698c145450 0x7f698c0aabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:00.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.150+0000 7f698b400640 1 -- 172.21.15.100:0/4136175755 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698c0ab110 con 0x7f698c0cd610 2024-09-17T04:28:00.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.151+0000 7f698b400640 1 --2- 172.21.15.100:0/4136175755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f698c0cd610 0x7f698c15c0c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6978009fd0 tx=0x7f697800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:00.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.151+0000 7f6983e00640 1 -- 172.21.15.100:0/4136175755 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697800ccc0 con 0x7f698c0cd610 2024-09-17T04:28:00.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.151+0000 7f6983e00640 1 -- 172.21.15.100:0/4136175755 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f697800ee70 con 0x7f698c0cd610 2024-09-17T04:28:00.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.151+0000 7f6993200640 1 -- 172.21.15.100:0/4136175755 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698c0a9b30 con 0x7f698c0cd610 2024-09-17T04:28:00.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.151+0000 7f6983e00640 1 -- 172.21.15.100:0/4136175755 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697800ce60 con 0x7f698c0cd610 2024-09-17T04:28:00.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.151+0000 7f6993200640 1 -- 172.21.15.100:0/4136175755 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f698c0aa0f0 con 0x7f698c0cd610 2024-09-17T04:28:00.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.152+0000 7f6993200640 1 -- 172.21.15.100:0/4136175755 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f698c0739f0 con 0x7f698c0cd610 2024-09-17T04:28:00.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.153+0000 7f6983e00640 1 -- 172.21.15.100:0/4136175755 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f6978004280 con 0x7f698c0cd610 2024-09-17T04:28:00.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.154+0000 7f6983e00640 1 --2- 172.21.15.100:0/4136175755 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f696c078d40 0x7f696c07b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:00.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.154+0000 7f6983e00640 1 -- 172.21.15.100:0/4136175755 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7f697809ba40 con 0x7f698c0cd610 2024-09-17T04:28:00.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.155+0000 7f698be00640 1 --2- 172.21.15.100:0/4136175755 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f696c078d40 0x7f696c07b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:00.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.155+0000 7f698be00640 1 --2- 172.21.15.100:0/4136175755 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f696c078d40 0x7f696c07b250 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f697c001720 tx=0x7f697c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:00.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.159+0000 7f6983e00640 1 -- 172.21.15.100:0/4136175755 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6978063400 con 0x7f698c0cd610 2024-09-17T04:28:00.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.349+0000 7f6993200640 1 -- 172.21.15.100:0/4136175755 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f698c070420 con 0x7f698c0cd610 2024-09-17T04:28:00.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.349+0000 7f6983e00640 1 -- 172.21.15.100:0/4136175755 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v36)=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7f697806c030 con 0x7f698c0cd610 2024-09-17T04:28:00.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.352+0000 7f6993200640 1 -- 172.21.15.100:0/4136175755 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f696c078d40 msgr2=0x7f696c07b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:00.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.352+0000 7f6993200640 1 --2- 172.21.15.100:0/4136175755 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f696c078d40 0x7f696c07b250 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f697c001720 tx=0x7f697c009040 comp rx=0 tx=0).stop 2024-09-17T04:28:00.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.352+0000 7f6993200640 1 -- 172.21.15.100:0/4136175755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f698c0cd610 msgr2=0x7f698c15c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:00.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.352+0000 7f6993200640 1 --2- 172.21.15.100:0/4136175755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f698c0cd610 0x7f698c15c0c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6978009fd0 tx=0x7f697800ec20 comp rx=0 tx=0).stop 2024-09-17T04:28:00.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.352+0000 7f6993200640 1 -- 172.21.15.100:0/4136175755 shutdown_connections 2024-09-17T04:28:00.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.352+0000 7f6993200640 1 --2- 172.21.15.100:0/4136175755 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f696c078d40 0x7f696c07b250 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:00.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.352+0000 7f6993200640 1 --2- 172.21.15.100:0/4136175755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f698c145450 0x7f698c0aabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:00.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.352+0000 7f6993200640 1 --2- 172.21.15.100:0/4136175755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f698c0cd610 0x7f698c15c0c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:00.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.352+0000 7f6993200640 1 --2- 172.21.15.100:0/4136175755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f698c0735e0 0x7f698c1f61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:00.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.352+0000 7f6993200640 1 -- 172.21.15.100:0/4136175755 >> 172.21.15.100:0/4136175755 conn(0x7f698c06d1c0 msgr2=0x7f698c0aff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:00.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.352+0000 7f6993200640 1 -- 172.21.15.100:0/4136175755 shutdown_connections 2024-09-17T04:28:00.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.352+0000 7f6993200640 1 -- 172.21.15.100:0/4136175755 wait complete. 2024-09-17T04:28:00.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:28:00.374 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvol75040) 2024-09-17T04:28:00.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:28:00.437 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-17T04:28:00.474 INFO:teuthology.orchestra.run.smithi100.stdout:cephfs_data 2024-09-17T04:28:00.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:28:00.474 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/_nogroup/subvol75040) 2024-09-17T04:28:00.575 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol90781 2024-09-17T04:28:00.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.799+0000 7efd1b200640 1 -- 172.21.15.100:0/1301744012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd14153950 msgr2=0x7efd140cdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:00.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.799+0000 7efd1b200640 1 --2- 172.21.15.100:0/1301744012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd14153950 0x7efd140cdf20 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7efd140d0900 tx=0x7efd0002a530 comp rx=0 tx=0).stop 2024-09-17T04:28:00.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.799+0000 7efd1b200640 1 -- 172.21.15.100:0/1301744012 shutdown_connections 2024-09-17T04:28:00.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.799+0000 7efd1b200640 1 --2- 172.21.15.100:0/1301744012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd14153950 0x7efd140cdf20 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:00.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.799+0000 7efd1b200640 1 --2- 172.21.15.100:0/1301744012 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd1406f960 0x7efd140747e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:00.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.799+0000 7efd1b200640 1 -- 172.21.15.100:0/1301744012 >> 172.21.15.100:0/1301744012 conn(0x7efd14073920 msgr2=0x7efd14073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:00.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.799+0000 7efd1b200640 1 -- 172.21.15.100:0/1301744012 shutdown_connections 2024-09-17T04:28:00.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.799+0000 7efd1b200640 1 -- 172.21.15.100:0/1301744012 wait complete. 2024-09-17T04:28:00.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.801+0000 7efd1b200640 1 Processor -- start 2024-09-17T04:28:00.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.801+0000 7efd1b200640 1 -- start start 2024-09-17T04:28:00.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.803+0000 7efd1b200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd1406ddb0 0x7efd141f7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:00.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.803+0000 7efd1b200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd1406f960 0x7efd14152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:00.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.803+0000 7efd1b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd14153950 0x7efd1413fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:00.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.803+0000 7efd1b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efd1415d4b0 con 0x7efd14153950 2024-09-17T04:28:00.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.803+0000 7efd1b200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efd1415d330 con 0x7efd1406f960 2024-09-17T04:28:00.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.803+0000 7efd1b200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efd1415d630 con 0x7efd1406ddb0 2024-09-17T04:28:00.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.803+0000 7efd19200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd14153950 0x7efd1413fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:00.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.803+0000 7efd19200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd14153950 0x7efd1413fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41722/0 (socket says 172.21.15.100:41722) 2024-09-17T04:28:00.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.803+0000 7efd19200640 1 -- 172.21.15.100:0/576890058 learned_addr learned my addr 172.21.15.100:0/576890058 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:00.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.804+0000 7efd13e00640 1 --2- 172.21.15.100:0/576890058 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd1406ddb0 0x7efd141f7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:00.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.804+0000 7efd13400640 1 --2- 172.21.15.100:0/576890058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd1406f960 0x7efd14152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:00.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.804+0000 7efd19200640 1 -- 172.21.15.100:0/576890058 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd1406ddb0 msgr2=0x7efd141f7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:00.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.804+0000 7efd19200640 1 --2- 172.21.15.100:0/576890058 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd1406ddb0 0x7efd141f7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:00.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.804+0000 7efd19200640 1 -- 172.21.15.100:0/576890058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd1406f960 msgr2=0x7efd14152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:00.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.804+0000 7efd19200640 1 --2- 172.21.15.100:0/576890058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd1406f960 0x7efd14152250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:00.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.804+0000 7efd19200640 1 -- 172.21.15.100:0/576890058 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd141f4100 con 0x7efd14153950 2024-09-17T04:28:00.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.804+0000 7efd13400640 1 --2- 172.21.15.100:0/576890058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd1406f960 0x7efd14152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:28:00.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.804+0000 7efd19200640 1 --2- 172.21.15.100:0/576890058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd14153950 0x7efd1413fca0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7efd0c00c730 tx=0x7efd0c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:00.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.804+0000 7efd0be00640 1 -- 172.21.15.100:0/576890058 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c003b70 con 0x7efd14153950 2024-09-17T04:28:00.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.804+0000 7efd0be00640 1 -- 172.21.15.100:0/576890058 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd0c003d30 con 0x7efd14153950 2024-09-17T04:28:00.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.804+0000 7efd0be00640 1 -- 172.21.15.100:0/576890058 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c00aab0 con 0x7efd14153950 2024-09-17T04:28:00.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.804+0000 7efd1b200640 1 -- 172.21.15.100:0/576890058 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd141f43e0 con 0x7efd14153950 2024-09-17T04:28:00.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.804+0000 7efd1b200640 1 -- 172.21.15.100:0/576890058 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd141f49a0 con 0x7efd14153950 2024-09-17T04:28:00.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.806+0000 7efd1b200640 1 -- 172.21.15.100:0/576890058 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd140b1370 con 0x7efd14153950 2024-09-17T04:28:00.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.807+0000 7efd0be00640 1 -- 172.21.15.100:0/576890058 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7efd0c00ac50 con 0x7efd14153950 2024-09-17T04:28:00.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.808+0000 7efd0be00640 1 --2- 172.21.15.100:0/576890058 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7efcf4078d40 0x7efcf407b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:00.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.808+0000 7efd0be00640 1 -- 172.21.15.100:0/576890058 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7efd0c09afa0 con 0x7efd14153950 2024-09-17T04:28:00.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.808+0000 7efd13e00640 1 --2- 172.21.15.100:0/576890058 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7efcf4078d40 0x7efcf407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:00.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.809+0000 7efd13e00640 1 --2- 172.21.15.100:0/576890058 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7efcf4078d40 0x7efcf407b250 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efd04009ac0 tx=0x7efd04005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:00.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:00.812+0000 7efd0be00640 1 -- 172.21.15.100:0/576890058 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7efd140b1370 con 0x7efd14153950 2024-09-17T04:28:01.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:01.019+0000 7efd1b200640 1 -- 172.21.15.100:0/576890058 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol90781", "target": ["mon-mgr", ""]}) -- 0x7efd141f3c40 con 0x7efcf4078d40 2024-09-17T04:28:01.025 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:28:01.026+0000 7fa165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:28:01.025 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:28:01.026+0000 7fa165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:28:01.025 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:28:01.026+0000 7fa165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:28:01.026 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:28:01.026+0000 7fa165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:28:01.026 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:28:01.026+0000 7fa165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:28:01.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:01.075+0000 7efd0be00640 1 -- 172.21.15.100:0/576890058 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efd04009710 con 0x7efcf4078d40 2024-09-17T04:28:01.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:01.077+0000 7efd1b200640 1 -- 172.21.15.100:0/576890058 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7efcf4078d40 msgr2=0x7efcf407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:01.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:01.077+0000 7efd1b200640 1 --2- 172.21.15.100:0/576890058 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7efcf4078d40 0x7efcf407b250 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efd04009ac0 tx=0x7efd04005f90 comp rx=0 tx=0).stop 2024-09-17T04:28:01.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:01.077+0000 7efd1b200640 1 -- 172.21.15.100:0/576890058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd14153950 msgr2=0x7efd1413fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:01.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:01.077+0000 7efd1b200640 1 --2- 172.21.15.100:0/576890058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd14153950 0x7efd1413fca0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7efd0c00c730 tx=0x7efd0c00cbf0 comp rx=0 tx=0).stop 2024-09-17T04:28:01.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:01.078+0000 7efd1b200640 1 -- 172.21.15.100:0/576890058 shutdown_connections 2024-09-17T04:28:01.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:01.078+0000 7efd1b200640 1 --2- 172.21.15.100:0/576890058 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7efcf4078d40 0x7efcf407b250 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:01.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:01.078+0000 7efd1b200640 1 --2- 172.21.15.100:0/576890058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd14153950 0x7efd1413fca0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:01.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:01.078+0000 7efd1b200640 1 --2- 172.21.15.100:0/576890058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd1406f960 0x7efd14152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:01.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:01.078+0000 7efd1b200640 1 --2- 172.21.15.100:0/576890058 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd1406ddb0 0x7efd141f7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:01.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:01.078+0000 7efd1b200640 1 -- 172.21.15.100:0/576890058 >> 172.21.15.100:0/576890058 conn(0x7efd14073920 msgr2=0x7efd140b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:01.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:01.078+0000 7efd1b200640 1 -- 172.21.15.100:0/576890058 shutdown_connections 2024-09-17T04:28:01.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:01.078+0000 7efd1b200640 1 -- 172.21.15.100:0/576890058 wait complete. 2024-09-17T04:28:01.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:28:01.100 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp ./volumes/_nogroup/subvol90781/.meta ./volumes/_nogroup/subvol75040) 2024-09-17T04:28:04.180 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol75040 2024-09-17T04:28:04.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.411+0000 7fd3afc00640 1 -- 172.21.15.100:0/2383325045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd38c005270 msgr2=0x7fd38c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:04.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.411+0000 7fd3afc00640 1 --2- 172.21.15.100:0/2383325045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd38c005270 0x7fd38c00b9c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd3a80730d0 tx=0x7fd38c005830 comp rx=0 tx=0).stop 2024-09-17T04:28:04.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.411+0000 7fd3adc00640 1 -- 172.21.15.100:0/2383325045 reap_dead start 2024-09-17T04:28:04.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.412+0000 7fd3afc00640 1 -- 172.21.15.100:0/2383325045 shutdown_connections 2024-09-17T04:28:04.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.412+0000 7fd3afc00640 1 -- 172.21.15.100:0/2383325045 >> 172.21.15.100:0/2383325045 conn(0x7fd3a806d1c0 msgr2=0x7fd3a806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:04.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.412+0000 7fd3afc00640 1 -- 172.21.15.100:0/2383325045 shutdown_connections 2024-09-17T04:28:04.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.412+0000 7fd3afc00640 1 -- 172.21.15.100:0/2383325045 wait complete. 2024-09-17T04:28:04.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.414+0000 7fd3afc00640 1 Processor -- start 2024-09-17T04:28:04.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.414+0000 7fd3afc00640 1 -- start start 2024-09-17T04:28:04.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.416+0000 7fd3afc00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd38c005270 0x7fd3a80b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:04.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.416+0000 7fd3afc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3a8134260 0x7fd3a80b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:04.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.416+0000 7fd3afc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3a80b0630 0x7fd3a815b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:04.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.416+0000 7fd3afc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a8165490 con 0x7fd3a80b0630 2024-09-17T04:28:04.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.416+0000 7fd3afc00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a80ce830 con 0x7fd3a8134260 2024-09-17T04:28:04.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.416+0000 7fd3afc00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd3a8165610 con 0x7fd38c005270 2024-09-17T04:28:04.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.416+0000 7fd3adc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3a80b0630 0x7fd3a815b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:04.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.416+0000 7fd3adc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3a80b0630 0x7fd3a815b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41744/0 (socket says 172.21.15.100:41744) 2024-09-17T04:28:04.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.416+0000 7fd3adc00640 1 -- 172.21.15.100:0/4223378449 learned_addr learned my addr 172.21.15.100:0/4223378449 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:04.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.416+0000 7fd3ad200640 1 --2- 172.21.15.100:0/4223378449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd38c005270 0x7fd3a80b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:04.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.416+0000 7fd39fe00640 1 --2- 172.21.15.100:0/4223378449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3a8134260 0x7fd3a80b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:04.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.416+0000 7fd3adc00640 1 -- 172.21.15.100:0/4223378449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd38c005270 msgr2=0x7fd3a80b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:04.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.416+0000 7fd3adc00640 1 --2- 172.21.15.100:0/4223378449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd38c005270 0x7fd3a80b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:04.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.416+0000 7fd3adc00640 1 -- 172.21.15.100:0/4223378449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3a8134260 msgr2=0x7fd3a80b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:04.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.416+0000 7fd3adc00640 1 --2- 172.21.15.100:0/4223378449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3a8134260 0x7fd3a80b00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:04.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.416+0000 7fd3adc00640 1 -- 172.21.15.100:0/4223378449 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a81334a0 con 0x7fd3a80b0630 2024-09-17T04:28:04.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.416+0000 7fd39fe00640 1 --2- 172.21.15.100:0/4223378449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3a8134260 0x7fd3a80b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:28:04.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.417+0000 7fd3adc00640 1 --2- 172.21.15.100:0/4223378449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3a80b0630 0x7fd3a815b6d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0009ee0 tx=0x7fd3a0033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:04.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.417+0000 7fd3a5600640 1 -- 172.21.15.100:0/4223378449 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0038040 con 0x7fd3a80b0630 2024-09-17T04:28:04.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.417+0000 7fd3a5600640 1 -- 172.21.15.100:0/4223378449 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3a0004b20 con 0x7fd3a80b0630 2024-09-17T04:28:04.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.417+0000 7fd3a5600640 1 -- 172.21.15.100:0/4223378449 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0033760 con 0x7fd3a80b0630 2024-09-17T04:28:04.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.417+0000 7fd3afc00640 1 -- 172.21.15.100:0/4223378449 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a8133780 con 0x7fd3a80b0630 2024-09-17T04:28:04.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.417+0000 7fd3afc00640 1 -- 172.21.15.100:0/4223378449 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a80b0ab0 con 0x7fd3a80b0630 2024-09-17T04:28:04.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.418+0000 7fd3afc00640 1 -- 172.21.15.100:0/4223378449 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3a8145ee0 con 0x7fd3a80b0630 2024-09-17T04:28:04.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.419+0000 7fd3a5600640 1 -- 172.21.15.100:0/4223378449 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 10) ==== 101645+0+0 (secure 0 0 0) 0x7fd3a0044020 con 0x7fd3a80b0630 2024-09-17T04:28:04.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.420+0000 7fd3a5600640 1 --2- 172.21.15.100:0/4223378449 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd384078dd0 0x7fd38407b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:04.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.420+0000 7fd3a5600640 1 -- 172.21.15.100:0/4223378449 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7fd3a00bb100 con 0x7fd3a80b0630 2024-09-17T04:28:04.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.421+0000 7fd3ad200640 1 --2- 172.21.15.100:0/4223378449 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd384078dd0 0x7fd38407b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:04.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.421+0000 7fd3ad200640 1 --2- 172.21.15.100:0/4223378449 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd384078dd0 0x7fd38407b2e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd398001860 tx=0x7fd398001510 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:04.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.425+0000 7fd3a5600640 1 -- 172.21.15.100:0/4223378449 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd3a8145ee0 con 0x7fd3a80b0630 2024-09-17T04:28:04.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.637+0000 7fd3afc00640 1 -- 172.21.15.100:0/4223378449 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol75040", "target": ["mon-mgr", ""]}) -- 0x7fd3a80b0d10 con 0x7fd384078dd0 2024-09-17T04:28:04.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.677+0000 7fd3a5600640 1 -- 172.21.15.100:0/4223378449 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7fd3a80b0d10 con 0x7fd384078dd0 2024-09-17T04:28:04.677 INFO:teuthology.orchestra.run.smithi100.stdout:/volumes/_nogroup/subvol75040 2024-09-17T04:28:04.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.679+0000 7fd3afc00640 1 -- 172.21.15.100:0/4223378449 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd384078dd0 msgr2=0x7fd38407b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:04.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.679+0000 7fd3afc00640 1 --2- 172.21.15.100:0/4223378449 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd384078dd0 0x7fd38407b2e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd398001860 tx=0x7fd398001510 comp rx=0 tx=0).stop 2024-09-17T04:28:04.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.679+0000 7fd3afc00640 1 -- 172.21.15.100:0/4223378449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3a80b0630 msgr2=0x7fd3a815b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:04.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.679+0000 7fd3afc00640 1 --2- 172.21.15.100:0/4223378449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3a80b0630 0x7fd3a815b6d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0009ee0 tx=0x7fd3a0033670 comp rx=0 tx=0).stop 2024-09-17T04:28:04.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.679+0000 7fd3afc00640 1 -- 172.21.15.100:0/4223378449 shutdown_connections 2024-09-17T04:28:04.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.679+0000 7fd3afc00640 1 --2- 172.21.15.100:0/4223378449 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd384078dd0 0x7fd38407b2e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:04.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.679+0000 7fd3afc00640 1 --2- 172.21.15.100:0/4223378449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3a80b0630 0x7fd3a815b6d0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:04.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.679+0000 7fd3afc00640 1 --2- 172.21.15.100:0/4223378449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3a8134260 0x7fd3a80b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:04.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.679+0000 7fd3afc00640 1 --2- 172.21.15.100:0/4223378449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd38c005270 0x7fd3a80b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:04.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.679+0000 7fd3afc00640 1 -- 172.21.15.100:0/4223378449 >> 172.21.15.100:0/4223378449 conn(0x7fd3a806d1c0 msgr2=0x7fd3a8068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:04.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.679+0000 7fd3afc00640 1 -- 172.21.15.100:0/4223378449 shutdown_connections 2024-09-17T04:28:04.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.679+0000 7fd3afc00640 1 -- 172.21.15.100:0/4223378449 wait complete. 2024-09-17T04:28:04.701 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol75040 2024-09-17T04:28:04.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.930+0000 7f3f56400640 1 -- 172.21.15.100:0/77517905 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f500cd610 msgr2=0x7f3f500cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:04.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.930+0000 7f3f56400640 1 --2- 172.21.15.100:0/77517905 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f500cd610 0x7f3f500cda20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3f48005fd0 tx=0x7f3f4802a8b0 comp rx=0 tx=0).stop 2024-09-17T04:28:04.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.930+0000 7f3f56400640 1 -- 172.21.15.100:0/77517905 shutdown_connections 2024-09-17T04:28:04.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.930+0000 7f3f56400640 1 --2- 172.21.15.100:0/77517905 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f500cd610 0x7f3f500cda20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:04.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.930+0000 7f3f56400640 1 --2- 172.21.15.100:0/77517905 >> v2:172.21.15.184:3300/0 conn(0x7f3f500735e0 0x7f3f500739f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:04.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.930+0000 7f3f56400640 1 -- 172.21.15.100:0/77517905 >> 172.21.15.100:0/77517905 conn(0x7f3f5006d1c0 msgr2=0x7f3f5006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:04.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.930+0000 7f3f56400640 1 -- 172.21.15.100:0/77517905 shutdown_connections 2024-09-17T04:28:04.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.930+0000 7f3f56400640 1 -- 172.21.15.100:0/77517905 wait complete. 2024-09-17T04:28:04.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.932+0000 7f3f56400640 1 Processor -- start 2024-09-17T04:28:04.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.933+0000 7f3f56400640 1 -- start start 2024-09-17T04:28:04.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.934+0000 7f3f56400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f500735e0 0x7f3f5015a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:04.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.934+0000 7f3f56400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f500cd610 0x7f3f50156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:04.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.934+0000 7f3f56400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f50145450 0x7f3f501f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:04.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.934+0000 7f3f56400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3f500ce2f0 con 0x7f3f500735e0 2024-09-17T04:28:04.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.934+0000 7f3f56400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3f500ce170 con 0x7f3f50145450 2024-09-17T04:28:04.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.934+0000 7f3f56400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3f500ce470 con 0x7f3f500cd610 2024-09-17T04:28:04.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.934+0000 7f3f4f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f500735e0 0x7f3f5015a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:04.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.935+0000 7f3f4f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f500735e0 0x7f3f5015a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60306/0 (socket says 172.21.15.100:60306) 2024-09-17T04:28:04.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.935+0000 7f3f4f400640 1 -- 172.21.15.100:0/155044741 learned_addr learned my addr 172.21.15.100:0/155044741 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:04.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.935+0000 7f3f4ea00640 1 --2- 172.21.15.100:0/155044741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f500cd610 0x7f3f50156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:04.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.935+0000 7f3f4fe00640 1 --2- 172.21.15.100:0/155044741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f50145450 0x7f3f501f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:04.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.935+0000 7f3f4f400640 1 -- 172.21.15.100:0/155044741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f500cd610 msgr2=0x7f3f50156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:04.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.935+0000 7f3f4f400640 1 --2- 172.21.15.100:0/155044741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f500cd610 0x7f3f50156a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:04.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.935+0000 7f3f4f400640 1 -- 172.21.15.100:0/155044741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f50145450 msgr2=0x7f3f501f4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:04.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.935+0000 7f3f4f400640 1 --2- 172.21.15.100:0/155044741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f50145450 0x7f3f501f4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:04.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.935+0000 7f3f4f400640 1 -- 172.21.15.100:0/155044741 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f50159510 con 0x7f3f500735e0 2024-09-17T04:28:04.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.935+0000 7f3f4f400640 1 --2- 172.21.15.100:0/155044741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f500735e0 0x7f3f5015a5b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000c730 tx=0x7f3f4000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:04.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.936+0000 7f3f47400640 1 -- 172.21.15.100:0/155044741 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f400052d0 con 0x7f3f500735e0 2024-09-17T04:28:04.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.936+0000 7f3f56400640 1 -- 172.21.15.100:0/155044741 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f501597f0 con 0x7f3f500735e0 2024-09-17T04:28:04.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.936+0000 7f3f47400640 1 -- 172.21.15.100:0/155044741 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f40005490 con 0x7f3f500735e0 2024-09-17T04:28:04.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.936+0000 7f3f47400640 1 -- 172.21.15.100:0/155044741 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f40003850 con 0x7f3f500735e0 2024-09-17T04:28:04.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.936+0000 7f3f56400640 1 -- 172.21.15.100:0/155044741 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f50159db0 con 0x7f3f500735e0 2024-09-17T04:28:04.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.937+0000 7f3f56400640 1 -- 172.21.15.100:0/155044741 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f500739f0 con 0x7f3f500735e0 2024-09-17T04:28:04.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.938+0000 7f3f47400640 1 -- 172.21.15.100:0/155044741 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 10) ==== 101645+0+0 (secure 0 0 0) 0x7f3f400039f0 con 0x7f3f500735e0 2024-09-17T04:28:04.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.939+0000 7f3f47400640 1 --2- 172.21.15.100:0/155044741 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3f30078e00 0x7f3f3007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:04.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.939+0000 7f3f47400640 1 -- 172.21.15.100:0/155044741 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7f3f4009ab60 con 0x7f3f500735e0 2024-09-17T04:28:04.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.939+0000 7f3f4ea00640 1 --2- 172.21.15.100:0/155044741 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3f30078e00 0x7f3f3007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:04.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.940+0000 7f3f4ea00640 1 --2- 172.21.15.100:0/155044741 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3f30078e00 0x7f3f3007b310 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3f50070d50 tx=0x7f3f3c009260 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:04.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:04.943+0000 7f3f47400640 1 -- 172.21.15.100:0/155044741 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3f50073f30 con 0x7f3f500735e0 2024-09-17T04:28:05.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:05.154+0000 7f3f56400640 1 -- 172.21.15.100:0/155044741 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol75040", "target": ["mon-mgr", ""]}) -- 0x7f3f5013f010 con 0x7f3f30078e00 2024-09-17T04:28:05.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:05.162+0000 7f3f47400640 1 -- 172.21.15.100:0/155044741 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7f3f5013f010 con 0x7f3f30078e00 2024-09-17T04:28:05.163 INFO:teuthology.orchestra.run.smithi100.stdout:/volumes/_nogroup/subvol75040 2024-09-17T04:28:05.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:05.164+0000 7f3f56400640 1 -- 172.21.15.100:0/155044741 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3f30078e00 msgr2=0x7f3f3007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:05.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:05.164+0000 7f3f56400640 1 --2- 172.21.15.100:0/155044741 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3f30078e00 0x7f3f3007b310 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3f50070d50 tx=0x7f3f3c009260 comp rx=0 tx=0).stop 2024-09-17T04:28:05.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:05.164+0000 7f3f56400640 1 -- 172.21.15.100:0/155044741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f500735e0 msgr2=0x7f3f5015a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:05.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:05.164+0000 7f3f56400640 1 --2- 172.21.15.100:0/155044741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f500735e0 0x7f3f5015a5b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000c730 tx=0x7f3f4000cbf0 comp rx=0 tx=0).stop 2024-09-17T04:28:05.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:05.164+0000 7f3f56400640 1 -- 172.21.15.100:0/155044741 shutdown_connections 2024-09-17T04:28:05.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:05.164+0000 7f3f56400640 1 --2- 172.21.15.100:0/155044741 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3f30078e00 0x7f3f3007b310 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:05.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:05.164+0000 7f3f56400640 1 --2- 172.21.15.100:0/155044741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f50145450 0x7f3f501f4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:05.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:05.164+0000 7f3f56400640 1 --2- 172.21.15.100:0/155044741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f500cd610 0x7f3f50156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:05.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:05.164+0000 7f3f56400640 1 --2- 172.21.15.100:0/155044741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f500735e0 0x7f3f5015a5b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:05.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:05.164+0000 7f3f56400640 1 -- 172.21.15.100:0/155044741 >> 172.21.15.100:0/155044741 conn(0x7f3f5006d1c0 msgr2=0x7f3f5015c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:05.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:05.164+0000 7f3f56400640 1 -- 172.21.15.100:0/155044741 shutdown_connections 2024-09-17T04:28:05.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:05.164+0000 7f3f56400640 1 -- 172.21.15.100:0/155044741 wait complete. 2024-09-17T04:28:05.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:28:05.186 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/db0dadc3fdff3b8fea4913c34f4018dc.meta) 2024-09-17T04:28:05.204 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:28:05.234 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:28:09.172 INFO:teuthology.orchestra.run.smithi100.stdout:[GLOBAL] 2024-09-17T04:28:09.172 INFO:teuthology.orchestra.run.smithi100.stdout:version = 1 2024-09-17T04:28:09.173 INFO:teuthology.orchestra.run.smithi100.stdout:type = subvolume 2024-09-17T04:28:09.173 INFO:teuthology.orchestra.run.smithi100.stdout:path = /volumes/_nogroup/subvol75040 2024-09-17T04:28:09.173 INFO:teuthology.orchestra.run.smithi100.stdout:state = complete 2024-09-17T04:28:09.173 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:28:09.176 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume authorize cephfs subvol75040 alice 2024-09-17T04:28:09.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.407+0000 7fa544a00640 1 -- 172.21.15.100:0/2825972381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa54006d4b0 msgr2=0x7fa5400b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:09.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.407+0000 7fa544a00640 1 --2- 172.21.15.100:0/2825972381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa54006d4b0 0x7fa5400b4620 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa540071040 tx=0x7fa52c02a540 comp rx=0 tx=0).stop 2024-09-17T04:28:09.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.407+0000 7fa544a00640 1 -- 172.21.15.100:0/2825972381 shutdown_connections 2024-09-17T04:28:09.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.407+0000 7fa544a00640 1 --2- 172.21.15.100:0/2825972381 >> v2:172.21.15.184:3300/0 conn(0x7fa5400af100 0x7fa5400af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:09.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.407+0000 7fa544a00640 1 --2- 172.21.15.100:0/2825972381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa54006d4b0 0x7fa5400b4620 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:09.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.407+0000 7fa544a00640 1 -- 172.21.15.100:0/2825972381 >> 172.21.15.100:0/2825972381 conn(0x7fa5400b2540 msgr2=0x7fa5400d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:09.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.407+0000 7fa544a00640 1 -- 172.21.15.100:0/2825972381 shutdown_connections 2024-09-17T04:28:09.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.407+0000 7fa544a00640 1 -- 172.21.15.100:0/2825972381 wait complete. 2024-09-17T04:28:09.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.409+0000 7fa544a00640 1 Processor -- start 2024-09-17T04:28:09.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.409+0000 7fa544a00640 1 -- start start 2024-09-17T04:28:09.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.411+0000 7fa544a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5400af100 0x7fa5401f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:09.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.411+0000 7fa544a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5400cc830 0x7fa540073690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:09.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.411+0000 7fa544a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa540074ae0 0x7fa540073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:09.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.411+0000 7fa544a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa5400b3620 con 0x7fa5400cc830 2024-09-17T04:28:09.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.411+0000 7fa544a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa5400b34a0 con 0x7fa540074ae0 2024-09-17T04:28:09.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.411+0000 7fa544a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa5400b37a0 con 0x7fa5400af100 2024-09-17T04:28:09.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.411+0000 7fa53d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5400cc830 0x7fa540073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:09.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.411+0000 7fa53d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5400cc830 0x7fa540073690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60324/0 (socket says 172.21.15.100:60324) 2024-09-17T04:28:09.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.411+0000 7fa53d400640 1 -- 172.21.15.100:0/366216710 learned_addr learned my addr 172.21.15.100:0/366216710 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:09.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.411+0000 7fa53e800640 1 --2- 172.21.15.100:0/366216710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa540074ae0 0x7fa540073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:09.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.411+0000 7fa53de00640 1 --2- 172.21.15.100:0/366216710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5400af100 0x7fa5401f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:09.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.411+0000 7fa53d400640 1 -- 172.21.15.100:0/366216710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5400af100 msgr2=0x7fa5401f7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:09.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.411+0000 7fa53d400640 1 --2- 172.21.15.100:0/366216710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5400af100 0x7fa5401f7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:09.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.411+0000 7fa53d400640 1 -- 172.21.15.100:0/366216710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa540074ae0 msgr2=0x7fa540073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:09.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.411+0000 7fa53d400640 1 --2- 172.21.15.100:0/366216710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa540074ae0 0x7fa540073bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:09.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.411+0000 7fa53d400640 1 -- 172.21.15.100:0/366216710 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa540156de0 con 0x7fa5400cc830 2024-09-17T04:28:09.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.411+0000 7fa53de00640 1 --2- 172.21.15.100:0/366216710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5400af100 0x7fa5401f7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:28:09.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.412+0000 7fa53d400640 1 --2- 172.21.15.100:0/366216710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5400cc830 0x7fa540073690 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009920 tx=0x7fa52c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:09.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.412+0000 7fa536000640 1 -- 172.21.15.100:0/366216710 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c031d60 con 0x7fa5400cc830 2024-09-17T04:28:09.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.412+0000 7fa536000640 1 -- 172.21.15.100:0/366216710 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa52c033e10 con 0x7fa5400cc830 2024-09-17T04:28:09.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.412+0000 7fa536000640 1 -- 172.21.15.100:0/366216710 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c005260 con 0x7fa5400cc830 2024-09-17T04:28:09.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.412+0000 7fa544a00640 1 -- 172.21.15.100:0/366216710 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5401570c0 con 0x7fa5400cc830 2024-09-17T04:28:09.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.412+0000 7fa544a00640 1 -- 172.21.15.100:0/366216710 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa54009a800 con 0x7fa5400cc830 2024-09-17T04:28:09.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.413+0000 7fa544a00640 1 -- 172.21.15.100:0/366216710 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5400b3cc0 con 0x7fa5400cc830 2024-09-17T04:28:09.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.414+0000 7fa536000640 1 -- 172.21.15.100:0/366216710 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 10) ==== 101645+0+0 (secure 0 0 0) 0x7fa52c005400 con 0x7fa5400cc830 2024-09-17T04:28:09.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.415+0000 7fa536000640 1 --2- 172.21.15.100:0/366216710 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa520078dd0 0x7fa52007b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:09.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.415+0000 7fa536000640 1 -- 172.21.15.100:0/366216710 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7fa52c0bc310 con 0x7fa5400cc830 2024-09-17T04:28:09.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.416+0000 7fa53de00640 1 --2- 172.21.15.100:0/366216710 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa520078dd0 0x7fa52007b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:09.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.416+0000 7fa53de00640 1 --2- 172.21.15.100:0/366216710 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa520078dd0 0x7fa52007b2e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa5300016a0 tx=0x7fa530009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:09.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.420+0000 7fa536000640 1 -- 172.21.15.100:0/366216710 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa52c03f030 con 0x7fa5400cc830 2024-09-17T04:28:09.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.628+0000 7fa544a00640 1 -- 172.21.15.100:0/366216710 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvol75040", "auth_id": "alice", "target": ["mon-mgr", ""]}) -- 0x7fa540156b90 con 0x7fa520078dd0 2024-09-17T04:28:09.640 INFO:tasks.ceph.mgr.x.smithi184.stderr:Creating meta for ID alice with tenant None 2024-09-17T04:28:09.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.680+0000 7fa536000640 1 -- 172.21.15.100:0/366216710 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa540156b90 con 0x7fa520078dd0 2024-09-17T04:28:09.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.682+0000 7fa544a00640 1 -- 172.21.15.100:0/366216710 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa520078dd0 msgr2=0x7fa52007b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:09.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.682+0000 7fa544a00640 1 --2- 172.21.15.100:0/366216710 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa520078dd0 0x7fa52007b2e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa5300016a0 tx=0x7fa530009040 comp rx=0 tx=0).stop 2024-09-17T04:28:09.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.682+0000 7fa544a00640 1 -- 172.21.15.100:0/366216710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5400cc830 msgr2=0x7fa540073690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:09.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.682+0000 7fa544a00640 1 --2- 172.21.15.100:0/366216710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5400cc830 0x7fa540073690 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009920 tx=0x7fa52c033c20 comp rx=0 tx=0).stop 2024-09-17T04:28:09.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.682+0000 7fa544a00640 1 -- 172.21.15.100:0/366216710 shutdown_connections 2024-09-17T04:28:09.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.682+0000 7fa544a00640 1 --2- 172.21.15.100:0/366216710 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa520078dd0 0x7fa52007b2e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:09.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.682+0000 7fa544a00640 1 --2- 172.21.15.100:0/366216710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa540074ae0 0x7fa540073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:09.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.682+0000 7fa544a00640 1 --2- 172.21.15.100:0/366216710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5400cc830 0x7fa540073690 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:09.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.682+0000 7fa544a00640 1 --2- 172.21.15.100:0/366216710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5400af100 0x7fa5401f7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:09.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.682+0000 7fa544a00640 1 -- 172.21.15.100:0/366216710 >> 172.21.15.100:0/366216710 conn(0x7fa5400b2540 msgr2=0x7fa540068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:09.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.682+0000 7fa544a00640 1 -- 172.21.15.100:0/366216710 shutdown_connections 2024-09-17T04:28:09.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.682+0000 7fa544a00640 1 -- 172.21.15.100:0/366216710 wait complete. 2024-09-17T04:28:09.708 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.alice --format=json-pretty 2024-09-17T04:28:09.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.927+0000 7f4751800640 1 -- 172.21.15.100:0/4032167558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f474c0ce640 msgr2=0x7f474c06ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:09.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.927+0000 7f4751800640 1 --2- 172.21.15.100:0/4032167558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f474c0ce640 0x7f474c06ddb0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4738005fd0 tx=0x7f473802a670 comp rx=0 tx=0).stop 2024-09-17T04:28:09.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.928+0000 7f4751800640 1 -- 172.21.15.100:0/4032167558 shutdown_connections 2024-09-17T04:28:09.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.928+0000 7f4751800640 1 --2- 172.21.15.100:0/4032167558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f474c06e2f0 0x7f474c06f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:09.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.928+0000 7f4751800640 1 --2- 172.21.15.100:0/4032167558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f474c0ce640 0x7f474c06ddb0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:09.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.928+0000 7f4751800640 1 -- 172.21.15.100:0/4032167558 >> 172.21.15.100:0/4032167558 conn(0x7f474c073940 msgr2=0x7f474c073d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:09.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.928+0000 7f4751800640 1 -- 172.21.15.100:0/4032167558 shutdown_connections 2024-09-17T04:28:09.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.928+0000 7f4751800640 1 -- 172.21.15.100:0/4032167558 wait complete. 2024-09-17T04:28:09.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.930+0000 7f4751800640 1 Processor -- start 2024-09-17T04:28:09.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.930+0000 7f4751800640 1 -- start start 2024-09-17T04:28:09.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.931+0000 7f4751800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f474c06e2f0 0x7f474c145580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:09.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.931+0000 7f4751800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f474c0ce640 0x7f474c151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:09.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.931+0000 7f4751800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f474c153880 0x7f474c1521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:09.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.931+0000 7f4751800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f474c075ea0 con 0x7f474c06e2f0 2024-09-17T04:28:09.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.931+0000 7f4751800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f474c075d20 con 0x7f474c0ce640 2024-09-17T04:28:09.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.931+0000 7f4751800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f474c06fee0 con 0x7f474c153880 2024-09-17T04:28:09.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.932+0000 7f474aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f474c06e2f0 0x7f474c145580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:09.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.932+0000 7f474aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f474c06e2f0 0x7f474c145580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60346/0 (socket says 172.21.15.100:60346) 2024-09-17T04:28:09.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.932+0000 7f474aa00640 1 -- 172.21.15.100:0/1111780358 learned_addr learned my addr 172.21.15.100:0/1111780358 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:09.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.932+0000 7f474b400640 1 --2- 172.21.15.100:0/1111780358 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f474c153880 0x7f474c1521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:09.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.932+0000 7f474a000640 1 --2- 172.21.15.100:0/1111780358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f474c0ce640 0x7f474c151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:09.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.932+0000 7f474aa00640 1 -- 172.21.15.100:0/1111780358 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f474c153880 msgr2=0x7f474c1521a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:09.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.932+0000 7f474aa00640 1 --2- 172.21.15.100:0/1111780358 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f474c153880 0x7f474c1521a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:09.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.932+0000 7f474aa00640 1 -- 172.21.15.100:0/1111780358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f474c0ce640 msgr2=0x7f474c151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:09.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.932+0000 7f474aa00640 1 --2- 172.21.15.100:0/1111780358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f474c0ce640 0x7f474c151c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:09.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.932+0000 7f474aa00640 1 -- 172.21.15.100:0/1111780358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f474c1f9c40 con 0x7f474c06e2f0 2024-09-17T04:28:09.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.933+0000 7f474aa00640 1 --2- 172.21.15.100:0/1111780358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f474c06e2f0 0x7f474c145580 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f473c00c3c0 tx=0x7f473c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:09.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.933+0000 7f4742a00640 1 -- 172.21.15.100:0/1111780358 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c017070 con 0x7f474c06e2f0 2024-09-17T04:28:09.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.933+0000 7f4742a00640 1 -- 172.21.15.100:0/1111780358 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f473c0035f0 con 0x7f474c06e2f0 2024-09-17T04:28:09.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.933+0000 7f4742a00640 1 -- 172.21.15.100:0/1111780358 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c003a80 con 0x7f474c06e2f0 2024-09-17T04:28:09.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.933+0000 7f4751800640 1 -- 172.21.15.100:0/1111780358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f474c0ad8c0 con 0x7f474c06e2f0 2024-09-17T04:28:09.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.933+0000 7f4751800640 1 -- 172.21.15.100:0/1111780358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f474c155a20 con 0x7f474c06e2f0 2024-09-17T04:28:09.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.934+0000 7f4751800640 1 -- 172.21.15.100:0/1111780358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f474c074a70 con 0x7f474c06e2f0 2024-09-17T04:28:09.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.935+0000 7f4742a00640 1 -- 172.21.15.100:0/1111780358 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 10) ==== 101645+0+0 (secure 0 0 0) 0x7f473c01a070 con 0x7f474c06e2f0 2024-09-17T04:28:09.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.936+0000 7f4742a00640 1 --2- 172.21.15.100:0/1111780358 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f472c078dd0 0x7f472c07b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:09.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.936+0000 7f4742a00640 1 -- 172.21.15.100:0/1111780358 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7f473c09a4b0 con 0x7f474c06e2f0 2024-09-17T04:28:09.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.937+0000 7f474a000640 1 --2- 172.21.15.100:0/1111780358 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f472c078dd0 0x7f472c07b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:09.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.937+0000 7f474a000640 1 --2- 172.21.15.100:0/1111780358 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f472c078dd0 0x7f472c07b2e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4738033000 tx=0x7f4738004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:09.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:09.941+0000 7f4742a00640 1 -- 172.21.15.100:0/1111780358 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f474c06e700 con 0x7f474c06e2f0 2024-09-17T04:28:10.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.193+0000 7f4751800640 1 -- 172.21.15.100:0/1111780358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"} v 0) -- 0x7f474c154210 con 0x7f474c06e2f0 2024-09-17T04:28:10.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.194+0000 7f4742a00640 1 -- 172.21.15.100:0/1111780358 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"}]=0 v14) ==== 107+0+280 (secure 0 0 0) 0x7f473c061eb0 con 0x7f474c06e2f0 2024-09-17T04:28:10.194 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:28:10.195 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-09-17T04:28:10.195 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:28:10.195 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.alice", 2024-09-17T04:28:10.195 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBZBelmbWY/JxAAwE6vqDQNgC9cyEdUFZwtBA==", 2024-09-17T04:28:10.195 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:28:10.195 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol75040", 2024-09-17T04:28:10.195 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow r", 2024-09-17T04:28:10.195 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rw pool=cephfs_data" 2024-09-17T04:28:10.195 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:28:10.195 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:28:10.195 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-09-17T04:28:10.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.196+0000 7f4751800640 1 -- 172.21.15.100:0/1111780358 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f472c078dd0 msgr2=0x7f472c07b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:10.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.196+0000 7f4751800640 1 --2- 172.21.15.100:0/1111780358 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f472c078dd0 0x7f472c07b2e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4738033000 tx=0x7f4738004730 comp rx=0 tx=0).stop 2024-09-17T04:28:10.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.196+0000 7f4751800640 1 -- 172.21.15.100:0/1111780358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f474c06e2f0 msgr2=0x7f474c145580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:10.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.196+0000 7f4751800640 1 --2- 172.21.15.100:0/1111780358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f474c06e2f0 0x7f474c145580 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f473c00c3c0 tx=0x7f473c00c880 comp rx=0 tx=0).stop 2024-09-17T04:28:10.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.196+0000 7f4751800640 1 -- 172.21.15.100:0/1111780358 shutdown_connections 2024-09-17T04:28:10.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.196+0000 7f4751800640 1 --2- 172.21.15.100:0/1111780358 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f472c078dd0 0x7f472c07b2e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:10.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.196+0000 7f4751800640 1 --2- 172.21.15.100:0/1111780358 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f474c153880 0x7f474c1521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:10.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.196+0000 7f4751800640 1 --2- 172.21.15.100:0/1111780358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f474c0ce640 0x7f474c151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:10.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.196+0000 7f4751800640 1 --2- 172.21.15.100:0/1111780358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f474c06e2f0 0x7f474c145580 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:10.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.196+0000 7f4751800640 1 -- 172.21.15.100:0/1111780358 >> 172.21.15.100:0/1111780358 conn(0x7f474c073940 msgr2=0x7f474c15a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:10.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.196+0000 7f4751800640 1 -- 172.21.15.100:0/1111780358 shutdown_connections 2024-09-17T04:28:10.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.196+0000 7f4751800640 1 -- 172.21.15.100:0/1111780358 wait complete. 2024-09-17T04:28:10.222 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol75040 2024-09-17T04:28:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.445+0000 7f7485600640 1 -- 172.21.15.100:0/261364969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7480156ca0 msgr2=0x7f74800ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.445+0000 7f7485600640 1 --2- 172.21.15.100:0/261364969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7480156ca0 0x7f74800ccac0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f746c009860 tx=0x7f746c02a5a0 comp rx=0 tx=0).stop 2024-09-17T04:28:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.445+0000 7f7485600640 1 -- 172.21.15.100:0/261364969 shutdown_connections 2024-09-17T04:28:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.445+0000 7f7485600640 1 --2- 172.21.15.100:0/261364969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7480156ca0 0x7f74800ccac0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.445+0000 7f7485600640 1 -- 172.21.15.100:0/261364969 >> 172.21.15.100:0/261364969 conn(0x7f748006d620 msgr2=0x7f74800b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.445+0000 7f7485600640 1 -- 172.21.15.100:0/261364969 shutdown_connections 2024-09-17T04:28:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.445+0000 7f7485600640 1 -- 172.21.15.100:0/261364969 wait complete. 2024-09-17T04:28:10.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.448+0000 7f7485600640 1 Processor -- start 2024-09-17T04:28:10.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.448+0000 7f7485600640 1 -- start start 2024-09-17T04:28:10.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.449+0000 7f7485600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74800b3f30 0x7f7480099aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:10.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.449+0000 7f7485600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74800cd000 0x7f748014fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:10.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.449+0000 7f7485600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7480156ca0 0x7f7480150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:10.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.449+0000 7f7485600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f74800af7e0 con 0x7f7480156ca0 2024-09-17T04:28:10.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.449+0000 7f7485600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f74800703b0 con 0x7f74800b3f30 2024-09-17T04:28:10.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.449+0000 7f7485600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f74800af960 con 0x7f74800cd000 2024-09-17T04:28:10.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.449+0000 7f7484c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7480156ca0 0x7f7480150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:10.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.449+0000 7f7484c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7480156ca0 0x7f7480150390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60362/0 (socket says 172.21.15.100:60362) 2024-09-17T04:28:10.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.449+0000 7f7484c00640 1 -- 172.21.15.100:0/631024808 learned_addr learned my addr 172.21.15.100:0/631024808 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:10.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.450+0000 7f747f400640 1 --2- 172.21.15.100:0/631024808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74800cd000 0x7f748014fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:10.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.450+0000 7f747fe00640 1 --2- 172.21.15.100:0/631024808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74800b3f30 0x7f7480099aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:10.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.450+0000 7f7484c00640 1 -- 172.21.15.100:0/631024808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74800cd000 msgr2=0x7f748014fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:10.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.450+0000 7f7484c00640 1 --2- 172.21.15.100:0/631024808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74800cd000 0x7f748014fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:10.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.450+0000 7f7484c00640 1 -- 172.21.15.100:0/631024808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74800b3f30 msgr2=0x7f7480099aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:10.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.450+0000 7f7484c00640 1 --2- 172.21.15.100:0/631024808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74800b3f30 0x7f7480099aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:10.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.450+0000 7f7484c00640 1 -- 172.21.15.100:0/631024808 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7480074780 con 0x7f7480156ca0 2024-09-17T04:28:10.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.450+0000 7f747fe00640 1 --2- 172.21.15.100:0/631024808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74800b3f30 0x7f7480099aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:28:10.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.450+0000 7f7484c00640 1 --2- 172.21.15.100:0/631024808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7480156ca0 0x7f7480150390 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f747800cb40 tx=0x7f747800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:10.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.450+0000 7f7477e00640 1 -- 172.21.15.100:0/631024808 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7478016020 con 0x7f7480156ca0 2024-09-17T04:28:10.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.451+0000 7f7477e00640 1 -- 172.21.15.100:0/631024808 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f747800adb0 con 0x7f7480156ca0 2024-09-17T04:28:10.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.451+0000 7f7485600640 1 -- 172.21.15.100:0/631024808 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7480074a60 con 0x7f7480156ca0 2024-09-17T04:28:10.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.451+0000 7f7477e00640 1 -- 172.21.15.100:0/631024808 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7478012680 con 0x7f7480156ca0 2024-09-17T04:28:10.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.451+0000 7f7485600640 1 -- 172.21.15.100:0/631024808 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f748014e750 con 0x7f7480156ca0 2024-09-17T04:28:10.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.452+0000 7f7485600640 1 -- 172.21.15.100:0/631024808 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74800744f0 con 0x7f7480156ca0 2024-09-17T04:28:10.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.453+0000 7f7477e00640 1 -- 172.21.15.100:0/631024808 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 10) ==== 101645+0+0 (secure 0 0 0) 0x7f7478019020 con 0x7f7480156ca0 2024-09-17T04:28:10.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.454+0000 7f7477e00640 1 --2- 172.21.15.100:0/631024808 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7460078e00 0x7f746007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:10.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.454+0000 7f7477e00640 1 -- 172.21.15.100:0/631024808 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7f747809b880 con 0x7f7480156ca0 2024-09-17T04:28:10.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.454+0000 7f747fe00640 1 --2- 172.21.15.100:0/631024808 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7460078e00 0x7f746007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:10.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.455+0000 7f747fe00640 1 --2- 172.21.15.100:0/631024808 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7460078e00 0x7f746007b310 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7470007fd0 tx=0x7f7470009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:10.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.458+0000 7f7477e00640 1 -- 172.21.15.100:0/631024808 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f7478020020 con 0x7f7480156ca0 2024-09-17T04:28:10.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.665+0000 7f7485600640 1 -- 172.21.15.100:0/631024808 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol75040", "target": ["mon-mgr", ""]}) -- 0x7f748015dbf0 con 0x7f7460078e00 2024-09-17T04:28:10.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.695+0000 7f7477e00640 1 -- 172.21.15.100:0/631024808 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f748015dbf0 con 0x7f7460078e00 2024-09-17T04:28:10.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.697+0000 7f7485600640 1 -- 172.21.15.100:0/631024808 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7460078e00 msgr2=0x7f746007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:10.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.697+0000 7f7485600640 1 --2- 172.21.15.100:0/631024808 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7460078e00 0x7f746007b310 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7470007fd0 tx=0x7f7470009040 comp rx=0 tx=0).stop 2024-09-17T04:28:10.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.697+0000 7f7485600640 1 -- 172.21.15.100:0/631024808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7480156ca0 msgr2=0x7f7480150390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:10.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.697+0000 7f7485600640 1 --2- 172.21.15.100:0/631024808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7480156ca0 0x7f7480150390 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f747800cb40 tx=0x7f747800a060 comp rx=0 tx=0).stop 2024-09-17T04:28:10.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.698+0000 7f7485600640 1 -- 172.21.15.100:0/631024808 shutdown_connections 2024-09-17T04:28:10.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.698+0000 7f7485600640 1 --2- 172.21.15.100:0/631024808 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7460078e00 0x7f746007b310 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:10.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.698+0000 7f7485600640 1 --2- 172.21.15.100:0/631024808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7480156ca0 0x7f7480150390 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:10.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.698+0000 7f7485600640 1 --2- 172.21.15.100:0/631024808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74800cd000 0x7f748014fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:10.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.698+0000 7f7485600640 1 --2- 172.21.15.100:0/631024808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74800b3f30 0x7f7480099aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:10.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.698+0000 7f7485600640 1 -- 172.21.15.100:0/631024808 >> 172.21.15.100:0/631024808 conn(0x7f748006d620 msgr2=0x7f748015b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:10.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.698+0000 7f7485600640 1 -- 172.21.15.100:0/631024808 shutdown_connections 2024-09-17T04:28:10.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.698+0000 7f7485600640 1 -- 172.21.15.100:0/631024808 wait complete. 2024-09-17T04:28:10.704 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:28:10.705+0000 7fa168600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:28:10.704 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:28:10.705+0000 7fa168600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:28:10.705 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:28:10.705+0000 7fa168600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:28:10.705 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:28:10.705+0000 7fa168600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:28:10.705 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:28:10.705+0000 7fa168600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:28:10.725 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:28:10.725+0000 7fa169a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:28:10.725 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:28:10.725+0000 7fa169a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:28:10.725 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:28:10.725+0000 7fa169a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:28:10.725 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:28:10.725+0000 7fa169a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:28:10.725 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:28:10.725+0000 7fa169a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:28:10.726 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol90781 2024-09-17T04:28:10.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.951+0000 7fd49c000640 1 -- 172.21.15.100:0/320922780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd494145450 msgr2=0x7fd49406f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:10.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.951+0000 7fd49c000640 1 --2- 172.21.15.100:0/320922780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd494145450 0x7fd49406f180 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd49415cd30 tx=0x7fd48802a560 comp rx=0 tx=0).stop 2024-09-17T04:28:10.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.951+0000 7fd49c000640 1 -- 172.21.15.100:0/320922780 shutdown_connections 2024-09-17T04:28:10.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.951+0000 7fd49c000640 1 --2- 172.21.15.100:0/320922780 >> v2:172.21.15.184:3300/0 conn(0x7fd4940cd610 0x7fd4940cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:10.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.951+0000 7fd49c000640 1 --2- 172.21.15.100:0/320922780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd494145450 0x7fd49406f180 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:10.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.951+0000 7fd49c000640 1 -- 172.21.15.100:0/320922780 >> 172.21.15.100:0/320922780 conn(0x7fd49406d1c0 msgr2=0x7fd49406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:10.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.951+0000 7fd49c000640 1 -- 172.21.15.100:0/320922780 shutdown_connections 2024-09-17T04:28:10.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.951+0000 7fd49c000640 1 -- 172.21.15.100:0/320922780 wait complete. 2024-09-17T04:28:10.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.953+0000 7fd49c000640 1 Processor -- start 2024-09-17T04:28:10.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.954+0000 7fd49c000640 1 -- start start 2024-09-17T04:28:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.955+0000 7fd49c000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4940735e0 0x7fd4940ac010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.955+0000 7fd49c000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4940cd610 0x7fd4940ac550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.955+0000 7fd49c000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd494145450 0x7fd4941f6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.955+0000 7fd49c000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd4940ce2f0 con 0x7fd494145450 2024-09-17T04:28:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.955+0000 7fd49c000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd4940ce170 con 0x7fd4940cd610 2024-09-17T04:28:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.955+0000 7fd49c000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd4940ce470 con 0x7fd4940735e0 2024-09-17T04:28:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.955+0000 7fd49a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd494145450 0x7fd4941f6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.955+0000 7fd49a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd494145450 0x7fd4941f6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60380/0 (socket says 172.21.15.100:60380) 2024-09-17T04:28:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.955+0000 7fd49a000640 1 -- 172.21.15.100:0/151215326 learned_addr learned my addr 172.21.15.100:0/151215326 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.955+0000 7fd499600640 1 --2- 172.21.15.100:0/151215326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4940735e0 0x7fd4940ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:10.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.955+0000 7fd498c00640 1 --2- 172.21.15.100:0/151215326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4940cd610 0x7fd4940ac550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:10.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.956+0000 7fd49a000640 1 -- 172.21.15.100:0/151215326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4940735e0 msgr2=0x7fd4940ac010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:10.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.956+0000 7fd49a000640 1 --2- 172.21.15.100:0/151215326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4940735e0 0x7fd4940ac010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:10.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.956+0000 7fd49a000640 1 -- 172.21.15.100:0/151215326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4940cd610 msgr2=0x7fd4940ac550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:10.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.956+0000 7fd49a000640 1 --2- 172.21.15.100:0/151215326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4940cd610 0x7fd4940ac550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:10.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.956+0000 7fd49a000640 1 -- 172.21.15.100:0/151215326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4940aea60 con 0x7fd494145450 2024-09-17T04:28:10.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.956+0000 7fd49a000640 1 --2- 172.21.15.100:0/151215326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd494145450 0x7fd4941f6df0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd48c00e960 tx=0x7fd48c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:10.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.956+0000 7fd491600640 1 -- 172.21.15.100:0/151215326 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c004d20 con 0x7fd494145450 2024-09-17T04:28:10.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.956+0000 7fd491600640 1 -- 172.21.15.100:0/151215326 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd48c00cb20 con 0x7fd494145450 2024-09-17T04:28:10.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.956+0000 7fd49c000640 1 -- 172.21.15.100:0/151215326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4940aed40 con 0x7fd494145450 2024-09-17T04:28:10.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.956+0000 7fd491600640 1 -- 172.21.15.100:0/151215326 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c005020 con 0x7fd494145450 2024-09-17T04:28:10.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.956+0000 7fd49c000640 1 -- 172.21.15.100:0/151215326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4940af300 con 0x7fd494145450 2024-09-17T04:28:10.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.957+0000 7fd49c000640 1 -- 172.21.15.100:0/151215326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4940cda20 con 0x7fd494145450 2024-09-17T04:28:10.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.959+0000 7fd491600640 1 -- 172.21.15.100:0/151215326 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 10) ==== 101645+0+0 (secure 0 0 0) 0x7fd48c0051c0 con 0x7fd494145450 2024-09-17T04:28:10.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.959+0000 7fd491600640 1 --2- 172.21.15.100:0/151215326 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd474078dd0 0x7fd47407b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:10.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.960+0000 7fd491600640 1 -- 172.21.15.100:0/151215326 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7fd48c09bb40 con 0x7fd494145450 2024-09-17T04:28:10.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.960+0000 7fd499600640 1 --2- 172.21.15.100:0/151215326 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd474078dd0 0x7fd47407b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:10.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.961+0000 7fd499600640 1 --2- 172.21.15.100:0/151215326 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd474078dd0 0x7fd47407b2e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd488033000 tx=0x7fd488004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:10.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:10.964+0000 7fd491600640 1 -- 172.21.15.100:0/151215326 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd48c016030 con 0x7fd494145450 2024-09-17T04:28:11.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:11.173+0000 7fd49c000640 1 -- 172.21.15.100:0/151215326 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol90781", "target": ["mon-mgr", ""]}) -- 0x7fd494142800 con 0x7fd474078dd0 2024-09-17T04:28:12.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:12.715+0000 7fd491600640 1 -- 172.21.15.100:0/151215326 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fd48c05f720 con 0x7fd494145450 2024-09-17T04:28:19.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.185+0000 7fd491600640 1 -- 172.21.15.100:0/151215326 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd494142800 con 0x7fd474078dd0 2024-09-17T04:28:19.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.189+0000 7fd49c000640 1 -- 172.21.15.100:0/151215326 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd474078dd0 msgr2=0x7fd47407b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:19.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.189+0000 7fd49c000640 1 --2- 172.21.15.100:0/151215326 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd474078dd0 0x7fd47407b2e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd488033000 tx=0x7fd488004730 comp rx=0 tx=0).stop 2024-09-17T04:28:19.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.189+0000 7fd49c000640 1 -- 172.21.15.100:0/151215326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd494145450 msgr2=0x7fd4941f6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:19.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.189+0000 7fd49c000640 1 --2- 172.21.15.100:0/151215326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd494145450 0x7fd4941f6df0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd48c00e960 tx=0x7fd48c00ee20 comp rx=0 tx=0).stop 2024-09-17T04:28:19.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.189+0000 7fd49c000640 1 -- 172.21.15.100:0/151215326 shutdown_connections 2024-09-17T04:28:19.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.189+0000 7fd49c000640 1 --2- 172.21.15.100:0/151215326 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd474078dd0 0x7fd47407b2e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:19.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.189+0000 7fd49c000640 1 --2- 172.21.15.100:0/151215326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd494145450 0x7fd4941f6df0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:19.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.189+0000 7fd49c000640 1 --2- 172.21.15.100:0/151215326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4940cd610 0x7fd4940ac550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:19.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.189+0000 7fd49c000640 1 --2- 172.21.15.100:0/151215326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4940735e0 0x7fd4940ac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:19.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.189+0000 7fd49c000640 1 -- 172.21.15.100:0/151215326 >> 172.21.15.100:0/151215326 conn(0x7fd49406d1c0 msgr2=0x7fd4941432d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:19.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.189+0000 7fd49c000640 1 -- 172.21.15.100:0/151215326 shutdown_connections 2024-09-17T04:28:19.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.189+0000 7fd49c000640 1 -- 172.21.15.100:0/151215326 wait complete. 2024-09-17T04:28:19.220 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-17T04:28:19.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:28:19.308 INFO:teuthology.orchestra.run.smithi100.stdout:2 2024-09-17T04:28:19.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:28:19.317 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T04:28:19.318 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:28:19.408 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:28:19.510 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T04:28:19.510 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:28:19.510 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:28:19.510 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:28:19.545 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:28:19.545 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:28:19.622 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:28:19.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.883+0000 7f4597600640 1 -- 172.21.15.100:0/3610528300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4590070cc0 msgr2=0x7f4590156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:19.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.883+0000 7f4597600640 1 --2- 172.21.15.100:0/3610528300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4590070cc0 0x7f4590156e40 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f457c009900 tx=0x7f457c02a620 comp rx=0 tx=0).stop 2024-09-17T04:28:19.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.884+0000 7f4597600640 1 -- 172.21.15.100:0/3610528300 shutdown_connections 2024-09-17T04:28:19.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.884+0000 7f4597600640 1 --2- 172.21.15.100:0/3610528300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4590070cc0 0x7f4590156e40 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:19.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.884+0000 7f4597600640 1 -- 172.21.15.100:0/3610528300 >> 172.21.15.100:0/3610528300 conn(0x7f459006d3d0 msgr2=0x7f45900b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:19.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.884+0000 7f4597600640 1 -- 172.21.15.100:0/3610528300 shutdown_connections 2024-09-17T04:28:19.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.884+0000 7f4597600640 1 -- 172.21.15.100:0/3610528300 wait complete. 2024-09-17T04:28:19.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.886+0000 7f4597600640 1 Processor -- start 2024-09-17T04:28:19.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.886+0000 7f4597600640 1 -- start start 2024-09-17T04:28:19.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.887+0000 7f4597600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45900cfc80 0x7f45901f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:19.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.887+0000 7f4597600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f459015ce30 0x7f459015b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:19.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.887+0000 7f4597600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45901f5470 0x7f459015be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:19.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.887+0000 7f4597600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f45900cd130 con 0x7f45900cfc80 2024-09-17T04:28:19.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.887+0000 7f4597600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f45900ccfb0 con 0x7f45901f5470 2024-09-17T04:28:19.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.887+0000 7f4597600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f45900cd2b0 con 0x7f459015ce30 2024-09-17T04:28:19.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.888+0000 7f4594c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45900cfc80 0x7f45901f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:19.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.888+0000 7f4594c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45900cfc80 0x7f45901f4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51974/0 (socket says 172.21.15.100:51974) 2024-09-17T04:28:19.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.888+0000 7f4594c00640 1 -- 172.21.15.100:0/2992227058 learned_addr learned my addr 172.21.15.100:0/2992227058 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:19.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.888+0000 7f4595600640 1 --2- 172.21.15.100:0/2992227058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45901f5470 0x7f459015be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:19.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.888+0000 7f458fe00640 1 --2- 172.21.15.100:0/2992227058 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f459015ce30 0x7f459015b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:19.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.888+0000 7f4594c00640 1 -- 172.21.15.100:0/2992227058 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f459015ce30 msgr2=0x7f459015b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:19.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.888+0000 7f4594c00640 1 --2- 172.21.15.100:0/2992227058 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f459015ce30 0x7f459015b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:19.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.888+0000 7f4594c00640 1 -- 172.21.15.100:0/2992227058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45901f5470 msgr2=0x7f459015be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:19.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.888+0000 7f4594c00640 1 --2- 172.21.15.100:0/2992227058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45901f5470 0x7f459015be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:19.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.888+0000 7f4594c00640 1 -- 172.21.15.100:0/2992227058 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45900ae460 con 0x7f45900cfc80 2024-09-17T04:28:19.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.888+0000 7f4595600640 1 --2- 172.21.15.100:0/2992227058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45901f5470 0x7f459015be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:28:19.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.888+0000 7f4594c00640 1 --2- 172.21.15.100:0/2992227058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45900cfc80 0x7f45901f4f30 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f458400c390 tx=0x7f458400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:19.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.889+0000 7f458cc00640 1 -- 172.21.15.100:0/2992227058 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4584017070 con 0x7f45900cfc80 2024-09-17T04:28:19.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.889+0000 7f458cc00640 1 -- 172.21.15.100:0/2992227058 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45840053e0 con 0x7f45900cfc80 2024-09-17T04:28:19.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.889+0000 7f4597600640 1 -- 172.21.15.100:0/2992227058 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45900ae740 con 0x7f45900cfc80 2024-09-17T04:28:19.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.889+0000 7f458cc00640 1 -- 172.21.15.100:0/2992227058 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4584003850 con 0x7f45900cfc80 2024-09-17T04:28:19.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.889+0000 7f4597600640 1 -- 172.21.15.100:0/2992227058 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45900aed00 con 0x7f45900cfc80 2024-09-17T04:28:19.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.890+0000 7f4597600640 1 -- 172.21.15.100:0/2992227058 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45900ca640 con 0x7f45900cfc80 2024-09-17T04:28:19.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.891+0000 7f458cc00640 1 -- 172.21.15.100:0/2992227058 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f458401a070 con 0x7f45900cfc80 2024-09-17T04:28:19.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.893+0000 7f458cc00640 1 --2- 172.21.15.100:0/2992227058 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4570078ef0 0x7f457007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:19.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.893+0000 7f458cc00640 1 -- 172.21.15.100:0/2992227058 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7f458409a610 con 0x7f45900cfc80 2024-09-17T04:28:19.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.893+0000 7f458fe00640 1 --2- 172.21.15.100:0/2992227058 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4570078ef0 0x7f457007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:19.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.894+0000 7f458fe00640 1 --2- 172.21.15.100:0/2992227058 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4570078ef0 0x7f457007b400 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f457c033000 tx=0x7f457c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:19.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:19.898+0000 7f458cc00640 1 -- 172.21.15.100:0/2992227058 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f45900ca640 con 0x7f45900cfc80 2024-09-17T04:28:20.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.094+0000 7f4597600640 1 -- 172.21.15.100:0/2992227058 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f45900cd7a0 con 0x7f45900cfc80 2024-09-17T04:28:20.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.096+0000 7f458cc00640 1 -- 172.21.15.100:0/2992227058 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v61) ==== 80+0+0 (secure 0 0 0) 0x7f4584062650 con 0x7f45900cfc80 2024-09-17T04:28:20.096 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-09-17T04:28:20.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.098+0000 7f4597600640 1 -- 172.21.15.100:0/2992227058 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4570078ef0 msgr2=0x7f457007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:20.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.098+0000 7f4597600640 1 --2- 172.21.15.100:0/2992227058 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4570078ef0 0x7f457007b400 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f457c033000 tx=0x7f457c004730 comp rx=0 tx=0).stop 2024-09-17T04:28:20.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.098+0000 7f4597600640 1 -- 172.21.15.100:0/2992227058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45900cfc80 msgr2=0x7f45901f4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:20.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.098+0000 7f4597600640 1 --2- 172.21.15.100:0/2992227058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45900cfc80 0x7f45901f4f30 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f458400c390 tx=0x7f458400c850 comp rx=0 tx=0).stop 2024-09-17T04:28:20.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.098+0000 7f4597600640 1 -- 172.21.15.100:0/2992227058 shutdown_connections 2024-09-17T04:28:20.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.098+0000 7f4597600640 1 --2- 172.21.15.100:0/2992227058 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4570078ef0 0x7f457007b400 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:20.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.098+0000 7f4597600640 1 --2- 172.21.15.100:0/2992227058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45901f5470 0x7f459015be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:20.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.098+0000 7f4597600640 1 --2- 172.21.15.100:0/2992227058 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f459015ce30 0x7f459015b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:20.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.098+0000 7f4597600640 1 --2- 172.21.15.100:0/2992227058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45900cfc80 0x7f45901f4f30 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:20.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.098+0000 7f4597600640 1 -- 172.21.15.100:0/2992227058 >> 172.21.15.100:0/2992227058 conn(0x7f459006d3d0 msgr2=0x7f45900b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:20.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.098+0000 7f4597600640 1 -- 172.21.15.100:0/2992227058 shutdown_connections 2024-09-17T04:28:20.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.098+0000 7f4597600640 1 -- 172.21.15.100:0/2992227058 wait complete. 2024-09-17T04:28:20.122 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T04:28:20.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:28:20.122 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T04:28:20.211 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-09-17T04:28:20.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:28:20.211 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T04:28:20.264 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:28:20.265 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:28:20.265 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:28:20.302 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:28:20.302 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:28:20.379 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:28:20.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.601+0000 7fb583400640 1 -- 172.21.15.100:0/1698296268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb560003a50 msgr2=0x7fb560003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:20.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.601+0000 7fb583400640 1 --2- 172.21.15.100:0/1698296268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb560003a50 0x7fb560003ec0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb56c001690 tx=0x7fb560005830 comp rx=0 tx=0).stop 2024-09-17T04:28:20.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.601+0000 7fb583400640 1 -- 172.21.15.100:0/1698296268 shutdown_connections 2024-09-17T04:28:20.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.601+0000 7fb583400640 1 --2- 172.21.15.100:0/1698296268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb560003a50 0x7fb560003ec0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:20.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.601+0000 7fb583400640 1 -- 172.21.15.100:0/1698296268 >> 172.21.15.100:0/1698296268 conn(0x7fb57c06d3d0 msgr2=0x7fb57c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:20.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.601+0000 7fb583400640 1 -- 172.21.15.100:0/1698296268 shutdown_connections 2024-09-17T04:28:20.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.601+0000 7fb583400640 1 -- 172.21.15.100:0/1698296268 wait complete. 2024-09-17T04:28:20.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.603+0000 7fb583400640 1 Processor -- start 2024-09-17T04:28:20.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.604+0000 7fb583400640 1 -- start start 2024-09-17T04:28:20.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.605+0000 7fb583400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb560003a50 0x7fb57c0c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:20.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.605+0000 7fb583400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb57c0c4b80 0x7fb57c0c3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:20.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.605+0000 7fb583400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb57c1f5700 0x7fb57c1f5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:20.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.605+0000 7fb583400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c15ce30 con 0x7fb560003a50 2024-09-17T04:28:20.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.605+0000 7fb583400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c157260 con 0x7fb57c1f5700 2024-09-17T04:28:20.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.605+0000 7fb583400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb57c15cfb0 con 0x7fb57c0c4b80 2024-09-17T04:28:20.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.605+0000 7fb580a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb560003a50 0x7fb57c0c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:20.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.605+0000 7fb580a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb560003a50 0x7fb57c0c65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51990/0 (socket says 172.21.15.100:51990) 2024-09-17T04:28:20.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.605+0000 7fb580a00640 1 -- 172.21.15.100:0/496349114 learned_addr learned my addr 172.21.15.100:0/496349114 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:20.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.606+0000 7fb57be00640 1 --2- 172.21.15.100:0/496349114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb57c0c4b80 0x7fb57c0c3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:20.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.606+0000 7fb581400640 1 --2- 172.21.15.100:0/496349114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb57c1f5700 0x7fb57c1f5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:20.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.606+0000 7fb580a00640 1 -- 172.21.15.100:0/496349114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb57c0c4b80 msgr2=0x7fb57c0c3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:20.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.606+0000 7fb580a00640 1 --2- 172.21.15.100:0/496349114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb57c0c4b80 0x7fb57c0c3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:20.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.606+0000 7fb580a00640 1 -- 172.21.15.100:0/496349114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb57c1f5700 msgr2=0x7fb57c1f5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:20.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.606+0000 7fb580a00640 1 --2- 172.21.15.100:0/496349114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb57c1f5700 0x7fb57c1f5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:20.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.606+0000 7fb580a00640 1 -- 172.21.15.100:0/496349114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57c1f8bf0 con 0x7fb560003a50 2024-09-17T04:28:20.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.606+0000 7fb57be00640 1 --2- 172.21.15.100:0/496349114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb57c0c4b80 0x7fb57c0c3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:28:20.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.606+0000 7fb581400640 1 --2- 172.21.15.100:0/496349114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb57c1f5700 0x7fb57c1f5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:28:20.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.606+0000 7fb580a00640 1 --2- 172.21.15.100:0/496349114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb560003a50 0x7fb57c0c65e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb56c02a8b0 tx=0x7fb56c0063a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:20.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.606+0000 7fb578c00640 1 -- 172.21.15.100:0/496349114 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c003a00 con 0x7fb560003a50 2024-09-17T04:28:20.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.606+0000 7fb578c00640 1 -- 172.21.15.100:0/496349114 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb56c039070 con 0x7fb560003a50 2024-09-17T04:28:20.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.606+0000 7fb583400640 1 -- 172.21.15.100:0/496349114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb57c1f8ed0 con 0x7fb560003a50 2024-09-17T04:28:20.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.606+0000 7fb578c00640 1 -- 172.21.15.100:0/496349114 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c033420 con 0x7fb560003a50 2024-09-17T04:28:20.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.607+0000 7fb583400640 1 -- 172.21.15.100:0/496349114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb57c0af0e0 con 0x7fb560003a50 2024-09-17T04:28:20.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.608+0000 7fb583400640 1 -- 172.21.15.100:0/496349114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb57c0afa70 con 0x7fb560003a50 2024-09-17T04:28:20.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.609+0000 7fb578c00640 1 -- 172.21.15.100:0/496349114 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fb56c044020 con 0x7fb560003a50 2024-09-17T04:28:20.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.610+0000 7fb578c00640 1 --2- 172.21.15.100:0/496349114 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb55c078f20 0x7fb55c07b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:20.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.610+0000 7fb578c00640 1 -- 172.21.15.100:0/496349114 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7fb56c0badf0 con 0x7fb560003a50 2024-09-17T04:28:20.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.610+0000 7fb57be00640 1 --2- 172.21.15.100:0/496349114 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb55c078f20 0x7fb55c07b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:20.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.611+0000 7fb57be00640 1 --2- 172.21.15.100:0/496349114 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb55c078f20 0x7fb55c07b430 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb568001260 tx=0x7fb5680067d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:20.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.614+0000 7fb578c00640 1 -- 172.21.15.100:0/496349114 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb56c082cd0 con 0x7fb560003a50 2024-09-17T04:28:20.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.810+0000 7fb583400640 1 -- 172.21.15.100:0/496349114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb57c15adb0 con 0x7fb560003a50 2024-09-17T04:28:20.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.810+0000 7fb578c00640 1 -- 172.21.15.100:0/496349114 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v61) ==== 80+0+0 (secure 0 0 0) 0x7fb56c0825d0 con 0x7fb560003a50 2024-09-17T04:28:20.811 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-09-17T04:28:20.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.813+0000 7fb583400640 1 -- 172.21.15.100:0/496349114 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb55c078f20 msgr2=0x7fb55c07b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:20.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.813+0000 7fb583400640 1 --2- 172.21.15.100:0/496349114 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb55c078f20 0x7fb55c07b430 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb568001260 tx=0x7fb5680067d0 comp rx=0 tx=0).stop 2024-09-17T04:28:20.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.813+0000 7fb583400640 1 -- 172.21.15.100:0/496349114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb560003a50 msgr2=0x7fb57c0c65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:20.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.813+0000 7fb583400640 1 --2- 172.21.15.100:0/496349114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb560003a50 0x7fb57c0c65e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb56c02a8b0 tx=0x7fb56c0063a0 comp rx=0 tx=0).stop 2024-09-17T04:28:20.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.813+0000 7fb583400640 1 -- 172.21.15.100:0/496349114 shutdown_connections 2024-09-17T04:28:20.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.813+0000 7fb583400640 1 --2- 172.21.15.100:0/496349114 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb55c078f20 0x7fb55c07b430 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:20.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.813+0000 7fb583400640 1 --2- 172.21.15.100:0/496349114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb57c1f5700 0x7fb57c1f5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:20.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.813+0000 7fb583400640 1 --2- 172.21.15.100:0/496349114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb57c0c4b80 0x7fb57c0c3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:20.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.813+0000 7fb583400640 1 --2- 172.21.15.100:0/496349114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb560003a50 0x7fb57c0c65e0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:20.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.813+0000 7fb583400640 1 -- 172.21.15.100:0/496349114 >> 172.21.15.100:0/496349114 conn(0x7fb57c06d3d0 msgr2=0x7fb57c15b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:20.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.813+0000 7fb583400640 1 -- 172.21.15.100:0/496349114 shutdown_connections 2024-09-17T04:28:20.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:20.813+0000 7fb583400640 1 -- 172.21.15.100:0/496349114 wait complete. 2024-09-17T04:28:20.835 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-17T04:28:20.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:28:20.835 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-17T04:28:20.901 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2024-09-17T04:28:20.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:28:20.901 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T04:28:20.938 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:28:20.938 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:28:20.938 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:28:20.973 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:28:20.973 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:28:20.973 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:28:20.973 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:28:21.013 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:28:21.013 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:28:21.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.235+0000 7f3f79e00640 1 -- 172.21.15.100:0/397034417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f7406f9a0 msgr2=0x7f3f740b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:21.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.235+0000 7f3f79e00640 1 --2- 172.21.15.100:0/397034417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f7406f9a0 0x7f3f740b2f60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c005fd0 tx=0x7f3f6c02a820 comp rx=0 tx=0).stop 2024-09-17T04:28:21.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.235+0000 7f3f79e00640 1 -- 172.21.15.100:0/397034417 shutdown_connections 2024-09-17T04:28:21.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.235+0000 7f3f79e00640 1 --2- 172.21.15.100:0/397034417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f7406f9a0 0x7f3f740b2f60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:21.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.235+0000 7f3f79e00640 1 -- 172.21.15.100:0/397034417 >> 172.21.15.100:0/397034417 conn(0x7f3f740b1e40 msgr2=0x7f3f740b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:21.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.235+0000 7f3f79e00640 1 -- 172.21.15.100:0/397034417 shutdown_connections 2024-09-17T04:28:21.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.235+0000 7f3f79e00640 1 -- 172.21.15.100:0/397034417 wait complete. 2024-09-17T04:28:21.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.237+0000 7f3f79e00640 1 Processor -- start 2024-09-17T04:28:21.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.238+0000 7f3f79e00640 1 -- start start 2024-09-17T04:28:21.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.239+0000 7f3f79e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f740ad9c0 0x7f3f741f4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:21.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.239+0000 7f3f79e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f740af170 0x7f3f740ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:21.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.239+0000 7f3f79e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f74154cf0 0x7f3f741f8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:21.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.239+0000 7f3f79e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7415a4c0 con 0x7f3f74154cf0 2024-09-17T04:28:21.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.239+0000 7f3f79e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7415a340 con 0x7f3f740ad9c0 2024-09-17T04:28:21.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.239+0000 7f3f79e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3f7415d670 con 0x7f3f740af170 2024-09-17T04:28:21.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.239+0000 7f3f79400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f74154cf0 0x7f3f741f8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:21.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.239+0000 7f3f79400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f74154cf0 0x7f3f741f8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52002/0 (socket says 172.21.15.100:52002) 2024-09-17T04:28:21.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.239+0000 7f3f79400640 1 -- 172.21.15.100:0/51000495 learned_addr learned my addr 172.21.15.100:0/51000495 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:21.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.239+0000 7f3f6be00640 1 --2- 172.21.15.100:0/51000495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f740af170 0x7f3f740ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:21.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.240+0000 7f3f78a00640 1 --2- 172.21.15.100:0/51000495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f740ad9c0 0x7f3f741f4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:21.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.240+0000 7f3f79400640 1 -- 172.21.15.100:0/51000495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f740af170 msgr2=0x7f3f740ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:21.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.240+0000 7f3f79400640 1 --2- 172.21.15.100:0/51000495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f740af170 0x7f3f740ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:21.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.240+0000 7f3f79400640 1 -- 172.21.15.100:0/51000495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f740ad9c0 msgr2=0x7f3f741f4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:21.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.240+0000 7f3f79400640 1 --2- 172.21.15.100:0/51000495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f740ad9c0 0x7f3f741f4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:21.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.240+0000 7f3f79400640 1 -- 172.21.15.100:0/51000495 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f740ce6c0 con 0x7f3f74154cf0 2024-09-17T04:28:21.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.240+0000 7f3f78a00640 1 --2- 172.21.15.100:0/51000495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f740ad9c0 0x7f3f741f4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:28:21.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.240+0000 7f3f79400640 1 --2- 172.21.15.100:0/51000495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f74154cf0 0x7f3f741f8af0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c006010 tx=0x7f3f6c032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:21.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.240+0000 7f3f71600640 1 -- 172.21.15.100:0/51000495 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c037040 con 0x7f3f74154cf0 2024-09-17T04:28:21.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.240+0000 7f3f71600640 1 -- 172.21.15.100:0/51000495 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f6c042070 con 0x7f3f74154cf0 2024-09-17T04:28:21.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.240+0000 7f3f79e00640 1 -- 172.21.15.100:0/51000495 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f741f5620 con 0x7f3f74154cf0 2024-09-17T04:28:21.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.240+0000 7f3f71600640 1 -- 172.21.15.100:0/51000495 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c003e60 con 0x7f3f74154cf0 2024-09-17T04:28:21.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.241+0000 7f3f79e00640 1 -- 172.21.15.100:0/51000495 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f741f5be0 con 0x7f3f74154cf0 2024-09-17T04:28:21.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.242+0000 7f3f79e00640 1 -- 172.21.15.100:0/51000495 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f741f5390 con 0x7f3f74154cf0 2024-09-17T04:28:21.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.243+0000 7f3f71600640 1 -- 172.21.15.100:0/51000495 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f3f6c004280 con 0x7f3f74154cf0 2024-09-17T04:28:21.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.243+0000 7f3f71600640 1 --2- 172.21.15.100:0/51000495 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3f50078ef0 0x7f3f5007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:21.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.244+0000 7f3f71600640 1 -- 172.21.15.100:0/51000495 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7f3f6c0bc150 con 0x7f3f74154cf0 2024-09-17T04:28:21.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.244+0000 7f3f78a00640 1 --2- 172.21.15.100:0/51000495 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3f50078ef0 0x7f3f5007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:21.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.245+0000 7f3f78a00640 1 --2- 172.21.15.100:0/51000495 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3f50078ef0 0x7f3f5007b400 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3f64007a50 tx=0x7f3f64009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:21.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.249+0000 7f3f71600640 1 -- 172.21.15.100:0/51000495 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3f6c084040 con 0x7f3f74154cf0 2024-09-17T04:28:21.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.467+0000 7f3f79e00640 1 -- 172.21.15.100:0/51000495 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f740b1a20 con 0x7f3f74154cf0 2024-09-17T04:28:21.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.468+0000 7f3f71600640 1 -- 172.21.15.100:0/51000495 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5229 (secure 0 0 0) 0x7f3f6c083940 con 0x7f3f74154cf0 2024-09-17T04:28:21.469 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:28:21.469 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":37,"btime":"2024-09-17T04:27:59:432735+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1735051989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1735051989},{"type":"v1","addr":"172.21.15.184:6836","nonce":1735051989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5638,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3935107098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":3935107098},{"type":"v1","addr":"172.21.15.184:6837","nonce":3935107098}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/904123596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904123596},{"type":"v1","addr":"172.21.15.100:6834","nonce":904123596}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:27:43.682593+0000","modified":"2024-09-17T04:27:46.702665+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/270906112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":270906112},{"type":"v1","addr":"172.21.15.100:6835","nonce":270906112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5739,"qdb_cluster":[5739]},"id":4}]} 2024-09-17T04:28:21.469 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 37 2024-09-17T04:28:21.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.470+0000 7f3f79e00640 1 -- 172.21.15.100:0/51000495 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3f50078ef0 msgr2=0x7f3f5007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:21.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.470+0000 7f3f79e00640 1 --2- 172.21.15.100:0/51000495 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3f50078ef0 0x7f3f5007b400 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3f64007a50 tx=0x7f3f64009040 comp rx=0 tx=0).stop 2024-09-17T04:28:21.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.470+0000 7f3f79e00640 1 -- 172.21.15.100:0/51000495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f74154cf0 msgr2=0x7f3f741f8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:21.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.470+0000 7f3f79e00640 1 --2- 172.21.15.100:0/51000495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f74154cf0 0x7f3f741f8af0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c006010 tx=0x7f3f6c032d10 comp rx=0 tx=0).stop 2024-09-17T04:28:21.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.470+0000 7f3f79e00640 1 -- 172.21.15.100:0/51000495 shutdown_connections 2024-09-17T04:28:21.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.470+0000 7f3f79e00640 1 --2- 172.21.15.100:0/51000495 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3f50078ef0 0x7f3f5007b400 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:21.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.470+0000 7f3f79e00640 1 --2- 172.21.15.100:0/51000495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f74154cf0 0x7f3f741f8af0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:21.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.470+0000 7f3f79e00640 1 --2- 172.21.15.100:0/51000495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f740af170 0x7f3f740ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:21.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.470+0000 7f3f79e00640 1 --2- 172.21.15.100:0/51000495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f740ad9c0 0x7f3f741f4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:21.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.470+0000 7f3f79e00640 1 -- 172.21.15.100:0/51000495 >> 172.21.15.100:0/51000495 conn(0x7f3f740b1e40 msgr2=0x7f3f740b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:21.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.471+0000 7f3f79e00640 1 -- 172.21.15.100:0/51000495 shutdown_connections 2024-09-17T04:28:21.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.471+0000 7f3f79e00640 1 -- 172.21.15.100:0/51000495 wait complete. 2024-09-17T04:28:21.495 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:28:21.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.713+0000 7f11d4a00640 1 -- 172.21.15.100:0/190796579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d01626d0 msgr2=0x7f11d0164af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:21.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.713+0000 7f11d4a00640 1 --2- 172.21.15.100:0/190796579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d01626d0 0x7f11d0164af0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f11d0072810 tx=0x7f11c802aa30 comp rx=0 tx=0).stop 2024-09-17T04:28:21.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.713+0000 7f11d4a00640 1 -- 172.21.15.100:0/190796579 shutdown_connections 2024-09-17T04:28:21.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.713+0000 7f11d4a00640 1 --2- 172.21.15.100:0/190796579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d01626d0 0x7f11d0164af0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:21.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.713+0000 7f11d4a00640 1 --2- 172.21.15.100:0/190796579 >> v2:172.21.15.184:3300/0 conn(0x7f11d00704b0 0x7f11d0073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:21.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.713+0000 7f11d4a00640 1 -- 172.21.15.100:0/190796579 >> 172.21.15.100:0/190796579 conn(0x7f11d006d730 msgr2=0x7f11d00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:21.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.713+0000 7f11d4a00640 1 -- 172.21.15.100:0/190796579 shutdown_connections 2024-09-17T04:28:21.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.713+0000 7f11d4a00640 1 -- 172.21.15.100:0/190796579 wait complete. 2024-09-17T04:28:21.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.716+0000 7f11d4a00640 1 Processor -- start 2024-09-17T04:28:21.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.716+0000 7f11d4a00640 1 -- start start 2024-09-17T04:28:21.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.717+0000 7f11d4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d00704b0 0x7f11d014d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:21.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.717+0000 7f11d4a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11d00ad3c0 0x7f11d01650c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:21.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.717+0000 7f11d4a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11d01626d0 0x7f11d0165600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:21.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.717+0000 7f11d4a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f11d00ae060 con 0x7f11d00704b0 2024-09-17T04:28:21.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.717+0000 7f11d4a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f11d00adee0 con 0x7f11d00ad3c0 2024-09-17T04:28:21.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.717+0000 7f11d4a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f11d0071130 con 0x7f11d01626d0 2024-09-17T04:28:21.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.717+0000 7f11cde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d00704b0 0x7f11d014d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:21.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.717+0000 7f11cde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d00704b0 0x7f11d014d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52016/0 (socket says 172.21.15.100:52016) 2024-09-17T04:28:21.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.717+0000 7f11cde00640 1 -- 172.21.15.100:0/2228646656 learned_addr learned my addr 172.21.15.100:0/2228646656 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:21.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.718+0000 7f11ce800640 1 --2- 172.21.15.100:0/2228646656 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11d01626d0 0x7f11d0165600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:21.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.718+0000 7f11cd400640 1 --2- 172.21.15.100:0/2228646656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11d00ad3c0 0x7f11d01650c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:21.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.718+0000 7f11cde00640 1 -- 172.21.15.100:0/2228646656 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11d01626d0 msgr2=0x7f11d0165600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:21.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.718+0000 7f11cde00640 1 --2- 172.21.15.100:0/2228646656 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11d01626d0 0x7f11d0165600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:21.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.718+0000 7f11cde00640 1 -- 172.21.15.100:0/2228646656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11d00ad3c0 msgr2=0x7f11d01650c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:21.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.718+0000 7f11cde00640 1 --2- 172.21.15.100:0/2228646656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11d00ad3c0 0x7f11d01650c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:21.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.718+0000 7f11cde00640 1 -- 172.21.15.100:0/2228646656 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d0166ec0 con 0x7f11d00704b0 2024-09-17T04:28:21.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.718+0000 7f11cde00640 1 --2- 172.21.15.100:0/2228646656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d00704b0 0x7f11d014d660 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f11c000d780 tx=0x7f11c000dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:21.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.718+0000 7f11c6000640 1 -- 172.21.15.100:0/2228646656 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c0017070 con 0x7f11d00704b0 2024-09-17T04:28:21.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.718+0000 7f11c6000640 1 -- 172.21.15.100:0/2228646656 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11c000de90 con 0x7f11d00704b0 2024-09-17T04:28:21.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.718+0000 7f11d4a00640 1 -- 172.21.15.100:0/2228646656 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d00aff10 con 0x7f11d00704b0 2024-09-17T04:28:21.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.718+0000 7f11c6000640 1 -- 172.21.15.100:0/2228646656 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c00099d0 con 0x7f11d00704b0 2024-09-17T04:28:21.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.719+0000 7f11d4a00640 1 -- 172.21.15.100:0/2228646656 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d00b1a70 con 0x7f11d00704b0 2024-09-17T04:28:21.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.720+0000 7f11d4a00640 1 -- 172.21.15.100:0/2228646656 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11d0073040 con 0x7f11d00704b0 2024-09-17T04:28:21.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.721+0000 7f11c6000640 1 -- 172.21.15.100:0/2228646656 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f11c0009b70 con 0x7f11d00704b0 2024-09-17T04:28:21.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.722+0000 7f11c6000640 1 --2- 172.21.15.100:0/2228646656 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f11b0078f20 0x7f11b007b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:21.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.722+0000 7f11c6000640 1 -- 172.21.15.100:0/2228646656 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7f11c009b640 con 0x7f11d00704b0 2024-09-17T04:28:21.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.722+0000 7f11cd400640 1 --2- 172.21.15.100:0/2228646656 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f11b0078f20 0x7f11b007b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:21.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.723+0000 7f11cd400640 1 --2- 172.21.15.100:0/2228646656 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f11b0078f20 0x7f11b007b430 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f11d00b2100 tx=0x7f11bc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:21.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.726+0000 7f11c6000640 1 -- 172.21.15.100:0/2228646656 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f11c0020020 con 0x7f11d00704b0 2024-09-17T04:28:21.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.946+0000 7f11d4a00640 1 -- 172.21.15.100:0/2228646656 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11d015e420 con 0x7f11d00704b0 2024-09-17T04:28:21.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.946+0000 7f11c6000640 1 -- 172.21.15.100:0/2228646656 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5229 (secure 0 0 0) 0x7f11c0062f60 con 0x7f11d00704b0 2024-09-17T04:28:21.947 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:28:21.947 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":37,"btime":"2024-09-17T04:27:59:432735+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1735051989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1735051989},{"type":"v1","addr":"172.21.15.184:6836","nonce":1735051989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5638,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3935107098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":3935107098},{"type":"v1","addr":"172.21.15.184:6837","nonce":3935107098}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/904123596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904123596},{"type":"v1","addr":"172.21.15.100:6834","nonce":904123596}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:27:43.682593+0000","modified":"2024-09-17T04:27:46.702665+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/270906112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":270906112},{"type":"v1","addr":"172.21.15.100:6835","nonce":270906112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5739,"qdb_cluster":[5739]},"id":4}]} 2024-09-17T04:28:21.947 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 37 2024-09-17T04:28:21.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.949+0000 7f11d4a00640 1 -- 172.21.15.100:0/2228646656 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f11b0078f20 msgr2=0x7f11b007b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:21.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.949+0000 7f11d4a00640 1 --2- 172.21.15.100:0/2228646656 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f11b0078f20 0x7f11b007b430 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f11d00b2100 tx=0x7f11bc009040 comp rx=0 tx=0).stop 2024-09-17T04:28:21.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.949+0000 7f11d4a00640 1 -- 172.21.15.100:0/2228646656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d00704b0 msgr2=0x7f11d014d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:21.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.949+0000 7f11d4a00640 1 --2- 172.21.15.100:0/2228646656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d00704b0 0x7f11d014d660 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f11c000d780 tx=0x7f11c000dc40 comp rx=0 tx=0).stop 2024-09-17T04:28:21.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.949+0000 7f11d4a00640 1 -- 172.21.15.100:0/2228646656 shutdown_connections 2024-09-17T04:28:21.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.949+0000 7f11d4a00640 1 --2- 172.21.15.100:0/2228646656 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f11b0078f20 0x7f11b007b430 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:21.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.949+0000 7f11d4a00640 1 --2- 172.21.15.100:0/2228646656 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11d01626d0 0x7f11d0165600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:21.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.949+0000 7f11d4a00640 1 --2- 172.21.15.100:0/2228646656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11d00ad3c0 0x7f11d01650c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:21.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.949+0000 7f11d4a00640 1 --2- 172.21.15.100:0/2228646656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d00704b0 0x7f11d014d660 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:21.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.949+0000 7f11d4a00640 1 -- 172.21.15.100:0/2228646656 >> 172.21.15.100:0/2228646656 conn(0x7f11d006d730 msgr2=0x7f11d00ae270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:21.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.949+0000 7f11d4a00640 1 -- 172.21.15.100:0/2228646656 shutdown_connections 2024-09-17T04:28:21.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:21.949+0000 7f11d4a00640 1 -- 172.21.15.100:0/2228646656 wait complete. 2024-09-17T04:28:21.974 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:28:22.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.198+0000 7f8114000640 1 -- 172.21.15.100:0/3177268473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f810c0b3550 msgr2=0x7f810c0b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:22.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.198+0000 7f8114000640 1 --2- 172.21.15.100:0/3177268473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f810c0b3550 0x7f810c0b3960 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8104005fd0 tx=0x7f810402aa10 comp rx=0 tx=0).stop 2024-09-17T04:28:22.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.199+0000 7f8114000640 1 -- 172.21.15.100:0/3177268473 shutdown_connections 2024-09-17T04:28:22.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.199+0000 7f8114000640 1 --2- 172.21.15.100:0/3177268473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f810c0b3550 0x7f810c0b3960 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:22.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.199+0000 7f8114000640 1 --2- 172.21.15.100:0/3177268473 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f810c15d760 0x7f810c1535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:22.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.199+0000 7f8114000640 1 -- 172.21.15.100:0/3177268473 >> 172.21.15.100:0/3177268473 conn(0x7f810c06d730 msgr2=0x7f810c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:22.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.199+0000 7f8114000640 1 -- 172.21.15.100:0/3177268473 shutdown_connections 2024-09-17T04:28:22.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.199+0000 7f8114000640 1 -- 172.21.15.100:0/3177268473 wait complete. 2024-09-17T04:28:22.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.201+0000 7f8114000640 1 Processor -- start 2024-09-17T04:28:22.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.201+0000 7f8114000640 1 -- start start 2024-09-17T04:28:22.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.202+0000 7f8114000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f810c0ad0e0 0x7f810c0b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:22.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.202+0000 7f8114000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f810c0b3550 0x7f810c1f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:22.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.202+0000 7f8114000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f810c15d760 0x7f810c1587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:22.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.202+0000 7f8114000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f810c0b4da0 con 0x7f810c0ad0e0 2024-09-17T04:28:22.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.202+0000 7f8114000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f810c0b4c20 con 0x7f810c15d760 2024-09-17T04:28:22.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.202+0000 7f8114000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f810c0b3ea0 con 0x7f810c0b3550 2024-09-17T04:28:22.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.203+0000 7f8111600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f810c0ad0e0 0x7f810c0b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:22.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.203+0000 7f8111600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f810c0ad0e0 0x7f810c0b1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52030/0 (socket says 172.21.15.100:52030) 2024-09-17T04:28:22.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.203+0000 7f8111600640 1 -- 172.21.15.100:0/4143843908 learned_addr learned my addr 172.21.15.100:0/4143843908 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:22.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.203+0000 7f8112000640 1 --2- 172.21.15.100:0/4143843908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f810c15d760 0x7f810c1587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:22.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.203+0000 7f8110c00640 1 --2- 172.21.15.100:0/4143843908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f810c0b3550 0x7f810c1f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:22.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.203+0000 7f8111600640 1 -- 172.21.15.100:0/4143843908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f810c0b3550 msgr2=0x7f810c1f50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:22.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.203+0000 7f8111600640 1 --2- 172.21.15.100:0/4143843908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f810c0b3550 0x7f810c1f50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:22.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.203+0000 7f8111600640 1 -- 172.21.15.100:0/4143843908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f810c15d760 msgr2=0x7f810c1587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:22.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.203+0000 7f8111600640 1 --2- 172.21.15.100:0/4143843908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f810c15d760 0x7f810c1587b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:22.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.203+0000 7f8111600640 1 -- 172.21.15.100:0/4143843908 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810c0b0dc0 con 0x7f810c0ad0e0 2024-09-17T04:28:22.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.203+0000 7f8111600640 1 --2- 172.21.15.100:0/4143843908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f810c0ad0e0 0x7f810c0b1890 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f810000c9a0 tx=0x7f810000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:22.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.204+0000 7f8109600640 1 -- 172.21.15.100:0/4143843908 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100007bb0 con 0x7f810c0ad0e0 2024-09-17T04:28:22.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.204+0000 7f8109600640 1 -- 172.21.15.100:0/4143843908 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8100007e40 con 0x7f810c0ad0e0 2024-09-17T04:28:22.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.204+0000 7f8109600640 1 -- 172.21.15.100:0/4143843908 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810000f690 con 0x7f810c0ad0e0 2024-09-17T04:28:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.204+0000 7f8114000640 1 -- 172.21.15.100:0/4143843908 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f810c15aa30 con 0x7f810c0ad0e0 2024-09-17T04:28:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.204+0000 7f8114000640 1 -- 172.21.15.100:0/4143843908 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f810c1f5600 con 0x7f810c0ad0e0 2024-09-17T04:28:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.205+0000 7f8114000640 1 -- 172.21.15.100:0/4143843908 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f810c0b4330 con 0x7f810c0ad0e0 2024-09-17T04:28:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.206+0000 7f8109600640 1 -- 172.21.15.100:0/4143843908 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f810000f830 con 0x7f810c0ad0e0 2024-09-17T04:28:22.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.207+0000 7f8109600640 1 --2- 172.21.15.100:0/4143843908 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f80ec078f20 0x7f80ec07b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:22.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.207+0000 7f8109600640 1 -- 172.21.15.100:0/4143843908 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7f810009bbf0 con 0x7f810c0ad0e0 2024-09-17T04:28:22.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.208+0000 7f8110c00640 1 --2- 172.21.15.100:0/4143843908 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f80ec078f20 0x7f80ec07b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:22.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.208+0000 7f8110c00640 1 --2- 172.21.15.100:0/4143843908 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f80ec078f20 0x7f80ec07b430 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f810c1f8d50 tx=0x7f80f8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:22.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.212+0000 7f8109600640 1 -- 172.21.15.100:0/4143843908 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8100014030 con 0x7f810c0ad0e0 2024-09-17T04:28:22.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.396+0000 7f8114000640 1 -- 172.21.15.100:0/4143843908 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f810c15dec0 con 0x7f810c0ad0e0 2024-09-17T04:28:22.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.398+0000 7f8109600640 1 -- 172.21.15.100:0/4143843908 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+16773 (secure 0 0 0) 0x7f81000633c0 con 0x7f810c0ad0e0 2024-09-17T04:28:22.398 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:28:22.398 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":61,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:27:45.203997+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:27:41.608179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-17T04:27:42.645978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:28:22.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.400+0000 7f8114000640 1 -- 172.21.15.100:0/4143843908 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f80ec078f20 msgr2=0x7f80ec07b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:22.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.400+0000 7f8114000640 1 --2- 172.21.15.100:0/4143843908 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f80ec078f20 0x7f80ec07b430 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f810c1f8d50 tx=0x7f80f8009040 comp rx=0 tx=0).stop 2024-09-17T04:28:22.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.400+0000 7f8114000640 1 -- 172.21.15.100:0/4143843908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f810c0ad0e0 msgr2=0x7f810c0b1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:22.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.400+0000 7f8114000640 1 --2- 172.21.15.100:0/4143843908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f810c0ad0e0 0x7f810c0b1890 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f810000c9a0 tx=0x7f810000ce60 comp rx=0 tx=0).stop 2024-09-17T04:28:22.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.400+0000 7f8114000640 1 -- 172.21.15.100:0/4143843908 shutdown_connections 2024-09-17T04:28:22.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.400+0000 7f8114000640 1 --2- 172.21.15.100:0/4143843908 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f80ec078f20 0x7f80ec07b430 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:22.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.400+0000 7f8114000640 1 --2- 172.21.15.100:0/4143843908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f810c15d760 0x7f810c1587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:22.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.400+0000 7f8114000640 1 --2- 172.21.15.100:0/4143843908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f810c0b3550 0x7f810c1f50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:22.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.400+0000 7f8114000640 1 --2- 172.21.15.100:0/4143843908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f810c0ad0e0 0x7f810c0b1890 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:22.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.400+0000 7f8114000640 1 -- 172.21.15.100:0/4143843908 >> 172.21.15.100:0/4143843908 conn(0x7f810c06d730 msgr2=0x7f810c15e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:22.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.400+0000 7f8114000640 1 -- 172.21.15.100:0/4143843908 shutdown_connections 2024-09-17T04:28:22.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.400+0000 7f8114000640 1 -- 172.21.15.100:0/4143843908 wait complete. 2024-09-17T04:28:22.427 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T04:28:22.428 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:28:22.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.648+0000 7fa19fe00640 1 -- 172.21.15.100:0/2189596463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa19815d760 msgr2=0x7fa1981535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:22.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.648+0000 7fa19fe00640 1 --2- 172.21.15.100:0/2189596463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa19815d760 0x7fa1981535c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa198072d90 tx=0x7fa18c02a4d0 comp rx=0 tx=0).stop 2024-09-17T04:28:22.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.648+0000 7fa19fe00640 1 -- 172.21.15.100:0/2189596463 shutdown_connections 2024-09-17T04:28:22.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.648+0000 7fa19fe00640 1 --2- 172.21.15.100:0/2189596463 >> v2:172.21.15.184:3300/0 conn(0x7fa198160310 0x7fa198160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:22.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.648+0000 7fa19fe00640 1 --2- 172.21.15.100:0/2189596463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1980ad0e0 0x7fa1980ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:22.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.648+0000 7fa19fe00640 1 --2- 172.21.15.100:0/2189596463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa19815d760 0x7fa1981535c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:22.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.648+0000 7fa19fe00640 1 -- 172.21.15.100:0/2189596463 >> 172.21.15.100:0/2189596463 conn(0x7fa19806d730 msgr2=0x7fa1980b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:22.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.648+0000 7fa19fe00640 1 -- 172.21.15.100:0/2189596463 shutdown_connections 2024-09-17T04:28:22.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.648+0000 7fa19fe00640 1 -- 172.21.15.100:0/2189596463 wait complete. 2024-09-17T04:28:22.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.651+0000 7fa19fe00640 1 Processor -- start 2024-09-17T04:28:22.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.651+0000 7fa19fe00640 1 -- start start 2024-09-17T04:28:22.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.652+0000 7fa19fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1980ad0e0 0x7fa198099a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:22.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.652+0000 7fa19fe00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa19815d760 0x7fa1981f3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:22.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.652+0000 7fa19fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa198160310 0x7fa1981545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:22.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.652+0000 7fa19fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa1980b4da0 con 0x7fa1980ad0e0 2024-09-17T04:28:22.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.652+0000 7fa19fe00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa1980b4c20 con 0x7fa198160310 2024-09-17T04:28:22.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.652+0000 7fa19fe00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa198160c60 con 0x7fa19815d760 2024-09-17T04:28:22.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.652+0000 7fa19d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1980ad0e0 0x7fa198099a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:22.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.652+0000 7fa19d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1980ad0e0 0x7fa198099a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52060/0 (socket says 172.21.15.100:52060) 2024-09-17T04:28:22.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.652+0000 7fa19d400640 1 -- 172.21.15.100:0/1316370754 learned_addr learned my addr 172.21.15.100:0/1316370754 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:22.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.653+0000 7fa19de00640 1 --2- 172.21.15.100:0/1316370754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa198160310 0x7fa1981545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:22.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.653+0000 7fa19ca00640 1 --2- 172.21.15.100:0/1316370754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa19815d760 0x7fa1981f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:22.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.653+0000 7fa19d400640 1 -- 172.21.15.100:0/1316370754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa19815d760 msgr2=0x7fa1981f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:22.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.653+0000 7fa19d400640 1 --2- 172.21.15.100:0/1316370754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa19815d760 0x7fa1981f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:22.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.653+0000 7fa19d400640 1 -- 172.21.15.100:0/1316370754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa198160310 msgr2=0x7fa1981545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:22.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.653+0000 7fa19d400640 1 --2- 172.21.15.100:0/1316370754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa198160310 0x7fa1981545e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:22.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.653+0000 7fa19d400640 1 -- 172.21.15.100:0/1316370754 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1980b1e00 con 0x7fa1980ad0e0 2024-09-17T04:28:22.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.653+0000 7fa19d400640 1 --2- 172.21.15.100:0/1316370754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1980ad0e0 0x7fa198099a10 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa18c034040 tx=0x7fa18c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:22.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.653+0000 7fa195600640 1 -- 172.21.15.100:0/1316370754 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c03e030 con 0x7fa1980ad0e0 2024-09-17T04:28:22.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.654+0000 7fa195600640 1 -- 172.21.15.100:0/1316370754 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa18c003e00 con 0x7fa1980ad0e0 2024-09-17T04:28:22.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.654+0000 7fa19fe00640 1 -- 172.21.15.100:0/1316370754 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1980b13d0 con 0x7fa1980ad0e0 2024-09-17T04:28:22.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.654+0000 7fa195600640 1 -- 172.21.15.100:0/1316370754 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c035040 con 0x7fa1980ad0e0 2024-09-17T04:28:22.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.654+0000 7fa19fe00640 1 -- 172.21.15.100:0/1316370754 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1980b1840 con 0x7fa1980ad0e0 2024-09-17T04:28:22.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.655+0000 7fa19fe00640 1 -- 172.21.15.100:0/1316370754 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1981535c0 con 0x7fa1980ad0e0 2024-09-17T04:28:22.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.656+0000 7fa195600640 1 -- 172.21.15.100:0/1316370754 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fa18c041030 con 0x7fa1980ad0e0 2024-09-17T04:28:22.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.657+0000 7fa195600640 1 --2- 172.21.15.100:0/1316370754 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa178078ef0 0x7fa17807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:22.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.657+0000 7fa195600640 1 -- 172.21.15.100:0/1316370754 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7fa18c0bbb10 con 0x7fa1980ad0e0 2024-09-17T04:28:22.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.657+0000 7fa19ca00640 1 --2- 172.21.15.100:0/1316370754 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa178078ef0 0x7fa17807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:22.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.658+0000 7fa19ca00640 1 --2- 172.21.15.100:0/1316370754 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa178078ef0 0x7fa17807b400 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa19815cc40 tx=0x7fa18400b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:22.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.662+0000 7fa195600640 1 -- 172.21.15.100:0/1316370754 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa18c083430 con 0x7fa1980ad0e0 2024-09-17T04:28:22.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.880+0000 7fa19fe00640 1 -- 172.21.15.100:0/1316370754 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa19815f6f0 con 0x7fa1980ad0e0 2024-09-17T04:28:22.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.881+0000 7fa195600640 1 -- 172.21.15.100:0/1316370754 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5229 (secure 0 0 0) 0x7fa18c087e60 con 0x7fa1980ad0e0 2024-09-17T04:28:22.881 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:28:22.881 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":37,"btime":"2024-09-17T04:27:59:432735+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1735051989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1735051989},{"type":"v1","addr":"172.21.15.184:6836","nonce":1735051989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5638,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3935107098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":3935107098},{"type":"v1","addr":"172.21.15.184:6837","nonce":3935107098}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/904123596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904123596},{"type":"v1","addr":"172.21.15.100:6834","nonce":904123596}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:27:43.682593+0000","modified":"2024-09-17T04:27:46.702665+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/270906112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":270906112},{"type":"v1","addr":"172.21.15.100:6835","nonce":270906112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5739,"qdb_cluster":[5739]},"id":4}]} 2024-09-17T04:28:22.882 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 37 2024-09-17T04:28:22.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.883+0000 7fa19fe00640 1 -- 172.21.15.100:0/1316370754 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa178078ef0 msgr2=0x7fa17807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:22.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.883+0000 7fa19fe00640 1 --2- 172.21.15.100:0/1316370754 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa178078ef0 0x7fa17807b400 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa19815cc40 tx=0x7fa18400b2e0 comp rx=0 tx=0).stop 2024-09-17T04:28:22.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.883+0000 7fa19fe00640 1 -- 172.21.15.100:0/1316370754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1980ad0e0 msgr2=0x7fa198099a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:22.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.883+0000 7fa19fe00640 1 --2- 172.21.15.100:0/1316370754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1980ad0e0 0x7fa198099a10 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa18c034040 tx=0x7fa18c003910 comp rx=0 tx=0).stop 2024-09-17T04:28:22.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.883+0000 7fa19fe00640 1 -- 172.21.15.100:0/1316370754 shutdown_connections 2024-09-17T04:28:22.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.883+0000 7fa19fe00640 1 --2- 172.21.15.100:0/1316370754 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa178078ef0 0x7fa17807b400 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:22.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.883+0000 7fa19fe00640 1 --2- 172.21.15.100:0/1316370754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa198160310 0x7fa1981545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:22.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.883+0000 7fa19fe00640 1 --2- 172.21.15.100:0/1316370754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa19815d760 0x7fa1981f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:22.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.883+0000 7fa19fe00640 1 --2- 172.21.15.100:0/1316370754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1980ad0e0 0x7fa198099a10 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:22.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.883+0000 7fa19fe00640 1 -- 172.21.15.100:0/1316370754 >> 172.21.15.100:0/1316370754 conn(0x7fa19806d730 msgr2=0x7fa198152a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:22.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.883+0000 7fa19fe00640 1 -- 172.21.15.100:0/1316370754 shutdown_connections 2024-09-17T04:28:22.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:22.884+0000 7fa19fe00640 1 -- 172.21.15.100:0/1316370754 wait complete. 2024-09-17T04:28:22.909 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:28:23.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.131+0000 7f741f000640 1 -- 172.21.15.100:0/3405458003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7418071970 msgr2=0x7f741815cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:23.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.131+0000 7f741f000640 1 --2- 172.21.15.100:0/3405458003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7418071970 0x7f741815cba0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7410005fd0 tx=0x7f741002a940 comp rx=0 tx=0).stop 2024-09-17T04:28:23.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.131+0000 7f741f000640 1 -- 172.21.15.100:0/3405458003 shutdown_connections 2024-09-17T04:28:23.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.131+0000 7f741f000640 1 --2- 172.21.15.100:0/3405458003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7418071970 0x7f741815cba0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:23.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.131+0000 7f741f000640 1 --2- 172.21.15.100:0/3405458003 >> v2:172.21.15.184:3300/0 conn(0x7f74180b6510 0x7f74180cf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:23.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.131+0000 7f741f000640 1 -- 172.21.15.100:0/3405458003 >> 172.21.15.100:0/3405458003 conn(0x7f741806cda0 msgr2=0x7f741806d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:23.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.131+0000 7f741f000640 1 -- 172.21.15.100:0/3405458003 shutdown_connections 2024-09-17T04:28:23.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.131+0000 7f741f000640 1 -- 172.21.15.100:0/3405458003 wait complete. 2024-09-17T04:28:23.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.134+0000 7f741f000640 1 Processor -- start 2024-09-17T04:28:23.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.134+0000 7f741f000640 1 -- start start 2024-09-17T04:28:23.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.135+0000 7f741f000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7418071970 0x7f741814e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:23.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.135+0000 7f741f000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7418153770 0x7f74180aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:23.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.135+0000 7f741f000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f741814f2a0 0x7f74180af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:23.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.135+0000 7f741f000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f741815d470 con 0x7f741814f2a0 2024-09-17T04:28:23.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.135+0000 7f741f000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f741815d2f0 con 0x7f7418071970 2024-09-17T04:28:23.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.135+0000 7f741f000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f741815d5f0 con 0x7f7418153770 2024-09-17T04:28:23.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.135+0000 7f741e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f741814f2a0 0x7f74180af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:23.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.135+0000 7f741e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f741814f2a0 0x7f74180af4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52074/0 (socket says 172.21.15.100:52074) 2024-09-17T04:28:23.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.136+0000 7f741e600640 1 -- 172.21.15.100:0/621121375 learned_addr learned my addr 172.21.15.100:0/621121375 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:23.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.136+0000 7f741dc00640 1 --2- 172.21.15.100:0/621121375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7418071970 0x7f741814e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:23.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.136+0000 7f741d200640 1 --2- 172.21.15.100:0/621121375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7418153770 0x7f74180aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:23.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.136+0000 7f741e600640 1 -- 172.21.15.100:0/621121375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7418153770 msgr2=0x7f74180aefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:23.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.136+0000 7f741e600640 1 --2- 172.21.15.100:0/621121375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7418153770 0x7f74180aefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:23.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.136+0000 7f741e600640 1 -- 172.21.15.100:0/621121375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7418071970 msgr2=0x7f741814e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:23.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.136+0000 7f741e600640 1 --2- 172.21.15.100:0/621121375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7418071970 0x7f741814e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:23.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.136+0000 7f741e600640 1 -- 172.21.15.100:0/621121375 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f741814eb40 con 0x7f741814f2a0 2024-09-17T04:28:23.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.136+0000 7f741e600640 1 --2- 172.21.15.100:0/621121375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f741814f2a0 0x7f74180af4f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7410009cd0 tx=0x7f7410031e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:23.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.136+0000 7f7415600640 1 -- 172.21.15.100:0/621121375 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741003f030 con 0x7f741814f2a0 2024-09-17T04:28:23.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.137+0000 7f741f000640 1 -- 172.21.15.100:0/621121375 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74181f9750 con 0x7f741814f2a0 2024-09-17T04:28:23.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.137+0000 7f7415600640 1 -- 172.21.15.100:0/621121375 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7410004250 con 0x7f741814f2a0 2024-09-17T04:28:23.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.137+0000 7f7415600640 1 -- 172.21.15.100:0/621121375 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7410030430 con 0x7f741814f2a0 2024-09-17T04:28:23.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.137+0000 7f741f000640 1 -- 172.21.15.100:0/621121375 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74181519a0 con 0x7f741814f2a0 2024-09-17T04:28:23.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.138+0000 7f741f000640 1 -- 172.21.15.100:0/621121375 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7418076530 con 0x7f741814f2a0 2024-09-17T04:28:23.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.139+0000 7f7415600640 1 -- 172.21.15.100:0/621121375 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f7410042030 con 0x7f741814f2a0 2024-09-17T04:28:23.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.140+0000 7f7415600640 1 --2- 172.21.15.100:0/621121375 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f73f8078ef0 0x7f73f807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:23.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.140+0000 7f7415600640 1 -- 172.21.15.100:0/621121375 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7f74100bc4d0 con 0x7f741814f2a0 2024-09-17T04:28:23.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.140+0000 7f741dc00640 1 --2- 172.21.15.100:0/621121375 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f73f8078ef0 0x7f73f807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:23.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.141+0000 7f741dc00640 1 --2- 172.21.15.100:0/621121375 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f73f8078ef0 0x7f73f807b400 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f74181600f0 tx=0x7f740c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:23.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.144+0000 7f7415600640 1 -- 172.21.15.100:0/621121375 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f7410083ce0 con 0x7f741814f2a0 2024-09-17T04:28:23.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.365+0000 7f741f000640 1 -- 172.21.15.100:0/621121375 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f741815db10 con 0x7f741814f2a0 2024-09-17T04:28:23.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.366+0000 7f7415600640 1 -- 172.21.15.100:0/621121375 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5229 (secure 0 0 0) 0x7f7410088710 con 0x7f741814f2a0 2024-09-17T04:28:23.367 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:28:23.368 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":37,"btime":"2024-09-17T04:27:59:432735+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1735051989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1735051989},{"type":"v1","addr":"172.21.15.184:6836","nonce":1735051989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5638,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3935107098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":3935107098},{"type":"v1","addr":"172.21.15.184:6837","nonce":3935107098}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/904123596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904123596},{"type":"v1","addr":"172.21.15.100:6834","nonce":904123596}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:27:43.682593+0000","modified":"2024-09-17T04:27:46.702665+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/270906112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":270906112},{"type":"v1","addr":"172.21.15.100:6835","nonce":270906112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5739,"qdb_cluster":[5739]},"id":4}]} 2024-09-17T04:28:23.368 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 37 2024-09-17T04:28:23.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.369+0000 7f741f000640 1 -- 172.21.15.100:0/621121375 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f73f8078ef0 msgr2=0x7f73f807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:23.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.369+0000 7f741f000640 1 --2- 172.21.15.100:0/621121375 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f73f8078ef0 0x7f73f807b400 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f74181600f0 tx=0x7f740c009040 comp rx=0 tx=0).stop 2024-09-17T04:28:23.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.369+0000 7f741f000640 1 -- 172.21.15.100:0/621121375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f741814f2a0 msgr2=0x7f74180af4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:23.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.369+0000 7f741f000640 1 --2- 172.21.15.100:0/621121375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f741814f2a0 0x7f74180af4f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7410009cd0 tx=0x7f7410031e20 comp rx=0 tx=0).stop 2024-09-17T04:28:23.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.369+0000 7f741f000640 1 -- 172.21.15.100:0/621121375 shutdown_connections 2024-09-17T04:28:23.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.369+0000 7f741f000640 1 --2- 172.21.15.100:0/621121375 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f73f8078ef0 0x7f73f807b400 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:23.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.369+0000 7f741f000640 1 --2- 172.21.15.100:0/621121375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f741814f2a0 0x7f74180af4f0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:23.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.369+0000 7f741f000640 1 --2- 172.21.15.100:0/621121375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7418153770 0x7f74180aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:23.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.369+0000 7f741f000640 1 --2- 172.21.15.100:0/621121375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7418071970 0x7f741814e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:23.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.369+0000 7f741f000640 1 -- 172.21.15.100:0/621121375 >> 172.21.15.100:0/621121375 conn(0x7f741806cda0 msgr2=0x7f74180cd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:23.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.369+0000 7f741f000640 1 -- 172.21.15.100:0/621121375 shutdown_connections 2024-09-17T04:28:23.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.369+0000 7f741f000640 1 -- 172.21.15.100:0/621121375 wait complete. 2024-09-17T04:28:23.395 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:28:23.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.614+0000 7fbfe2000640 1 -- 172.21.15.100:0/3239090525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfdc0ad0e0 msgr2=0x7fbfdc0ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:23.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.614+0000 7fbfe2000640 1 --2- 172.21.15.100:0/3239090525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfdc0ad0e0 0x7fbfdc0ad4f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc075250 tx=0x7fbfc802a560 comp rx=0 tx=0).stop 2024-09-17T04:28:23.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.614+0000 7fbfe2000640 1 -- 172.21.15.100:0/3239090525 shutdown_connections 2024-09-17T04:28:23.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.614+0000 7fbfe2000640 1 --2- 172.21.15.100:0/3239090525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfdc0ad0e0 0x7fbfdc0ad4f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:23.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.614+0000 7fbfe2000640 1 --2- 172.21.15.100:0/3239090525 >> v2:172.21.15.184:3300/0 conn(0x7fbfdc15d760 0x7fbfdc1535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:23.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.614+0000 7fbfe2000640 1 -- 172.21.15.100:0/3239090525 >> 172.21.15.100:0/3239090525 conn(0x7fbfdc06d730 msgr2=0x7fbfdc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:23.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.614+0000 7fbfe2000640 1 -- 172.21.15.100:0/3239090525 shutdown_connections 2024-09-17T04:28:23.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.614+0000 7fbfe2000640 1 -- 172.21.15.100:0/3239090525 wait complete. 2024-09-17T04:28:23.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.617+0000 7fbfe2000640 1 Processor -- start 2024-09-17T04:28:23.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.617+0000 7fbfe2000640 1 -- start start 2024-09-17T04:28:23.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.618+0000 7fbfe2000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbfdc0ad0e0 0x7fbfdc156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:23.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.618+0000 7fbfe2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfdc15d760 0x7fbfdc156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:23.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.618+0000 7fbfe2000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbfdc160310 0x7fbfdc1f7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:23.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.618+0000 7fbfe2000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbfdc0b4c20 con 0x7fbfdc15d760 2024-09-17T04:28:23.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.618+0000 7fbfe2000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbfdc153c70 con 0x7fbfdc160310 2024-09-17T04:28:23.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.618+0000 7fbfe2000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbfdc0b4da0 con 0x7fbfdc0ad0e0 2024-09-17T04:28:23.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.618+0000 7fbfdaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfdc15d760 0x7fbfdc156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:23.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.618+0000 7fbfdaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfdc15d760 0x7fbfdc156660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52096/0 (socket says 172.21.15.100:52096) 2024-09-17T04:28:23.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.618+0000 7fbfdaa00640 1 -- 172.21.15.100:0/3686503040 learned_addr learned my addr 172.21.15.100:0/3686503040 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:23.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.619+0000 7fbfdb400640 1 --2- 172.21.15.100:0/3686503040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbfdc0ad0e0 0x7fbfdc156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:23.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.619+0000 7fbfdbe00640 1 --2- 172.21.15.100:0/3686503040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbfdc160310 0x7fbfdc1f7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:23.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.619+0000 7fbfdaa00640 1 -- 172.21.15.100:0/3686503040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbfdc0ad0e0 msgr2=0x7fbfdc156120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:23.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.619+0000 7fbfdaa00640 1 --2- 172.21.15.100:0/3686503040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbfdc0ad0e0 0x7fbfdc156120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:23.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.619+0000 7fbfdaa00640 1 -- 172.21.15.100:0/3686503040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbfdc160310 msgr2=0x7fbfdc1f7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:23.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.619+0000 7fbfdaa00640 1 --2- 172.21.15.100:0/3686503040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbfdc160310 0x7fbfdc1f7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:23.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.619+0000 7fbfdaa00640 1 -- 172.21.15.100:0/3686503040 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfdc157f90 con 0x7fbfdc15d760 2024-09-17T04:28:23.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.619+0000 7fbfdaa00640 1 --2- 172.21.15.100:0/3686503040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfdc15d760 0x7fbfdc156660 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8034040 tx=0x7fbfc80039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:23.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.620+0000 7fbfd3400640 1 -- 172.21.15.100:0/3686503040 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc803e030 con 0x7fbfdc15d760 2024-09-17T04:28:23.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.620+0000 7fbfd3400640 1 -- 172.21.15.100:0/3686503040 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbfc80047a0 con 0x7fbfdc15d760 2024-09-17T04:28:23.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.620+0000 7fbfe2000640 1 -- 172.21.15.100:0/3686503040 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfdc158270 con 0x7fbfdc15d760 2024-09-17T04:28:23.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.620+0000 7fbfd3400640 1 -- 172.21.15.100:0/3686503040 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8035040 con 0x7fbfdc15d760 2024-09-17T04:28:23.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.620+0000 7fbfe2000640 1 -- 172.21.15.100:0/3686503040 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfdc158830 con 0x7fbfdc15d760 2024-09-17T04:28:23.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.621+0000 7fbfe2000640 1 -- 172.21.15.100:0/3686503040 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfdc15e2a0 con 0x7fbfdc15d760 2024-09-17T04:28:23.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.622+0000 7fbfd3400640 1 -- 172.21.15.100:0/3686503040 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fbfc8041030 con 0x7fbfdc15d760 2024-09-17T04:28:23.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.623+0000 7fbfd3400640 1 --2- 172.21.15.100:0/3686503040 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbfbc078ef0 0x7fbfbc07b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:23.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.623+0000 7fbfd3400640 1 -- 172.21.15.100:0/3686503040 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7fbfc80bbaa0 con 0x7fbfdc15d760 2024-09-17T04:28:23.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.623+0000 7fbfdb400640 1 --2- 172.21.15.100:0/3686503040 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbfbc078ef0 0x7fbfbc07b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:23.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.624+0000 7fbfdb400640 1 --2- 172.21.15.100:0/3686503040 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbfbc078ef0 0x7fbfbc07b400 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc004590 tx=0x7fbfcc00b390 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:23.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.627+0000 7fbfd3400640 1 -- 172.21.15.100:0/3686503040 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fbfdc15e2a0 con 0x7fbfdc15d760 2024-09-17T04:28:23.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.812+0000 7fbfe2000640 1 -- 172.21.15.100:0/3686503040 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbfdc050f60 con 0x7fbfdc15d760 2024-09-17T04:28:23.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.813+0000 7fbfd3400640 1 -- 172.21.15.100:0/3686503040 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+16773 (secure 0 0 0) 0x7fbfdc050f60 con 0x7fbfdc15d760 2024-09-17T04:28:23.814 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:28:23.814 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":61,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:27:45.203997+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:27:41.608179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-17T04:27:42.645978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:28:23.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.816+0000 7fbfe2000640 1 -- 172.21.15.100:0/3686503040 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbfbc078ef0 msgr2=0x7fbfbc07b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:23.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.816+0000 7fbfe2000640 1 --2- 172.21.15.100:0/3686503040 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbfbc078ef0 0x7fbfbc07b400 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc004590 tx=0x7fbfcc00b390 comp rx=0 tx=0).stop 2024-09-17T04:28:23.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.816+0000 7fbfe2000640 1 -- 172.21.15.100:0/3686503040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfdc15d760 msgr2=0x7fbfdc156660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:23.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.816+0000 7fbfe2000640 1 --2- 172.21.15.100:0/3686503040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfdc15d760 0x7fbfdc156660 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8034040 tx=0x7fbfc80039e0 comp rx=0 tx=0).stop 2024-09-17T04:28:23.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.816+0000 7fbfe2000640 1 -- 172.21.15.100:0/3686503040 shutdown_connections 2024-09-17T04:28:23.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.816+0000 7fbfe2000640 1 --2- 172.21.15.100:0/3686503040 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fbfbc078ef0 0x7fbfbc07b400 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:23.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.816+0000 7fbfe2000640 1 --2- 172.21.15.100:0/3686503040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbfdc160310 0x7fbfdc1f7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:23.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.816+0000 7fbfe2000640 1 --2- 172.21.15.100:0/3686503040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfdc15d760 0x7fbfdc156660 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:23.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.816+0000 7fbfe2000640 1 --2- 172.21.15.100:0/3686503040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbfdc0ad0e0 0x7fbfdc156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:23.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.816+0000 7fbfe2000640 1 -- 172.21.15.100:0/3686503040 >> 172.21.15.100:0/3686503040 conn(0x7fbfdc06d730 msgr2=0x7fbfdc15e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:23.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.816+0000 7fbfe2000640 1 -- 172.21.15.100:0/3686503040 shutdown_connections 2024-09-17T04:28:23.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:23.816+0000 7fbfe2000640 1 -- 172.21.15.100:0/3686503040 wait complete. 2024-09-17T04:28:23.839 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T04:28:24.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.060+0000 7fa798a00640 1 -- 172.21.15.100:0/1290779611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa79415d540 msgr2=0x7fa794156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:24.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.060+0000 7fa798a00640 1 --2- 172.21.15.100:0/1290779611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa79415d540 0x7fa794156e40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009c10 tx=0x7fa78c02a8b0 comp rx=0 tx=0).stop 2024-09-17T04:28:24.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.060+0000 7fa798a00640 1 -- 172.21.15.100:0/1290779611 shutdown_connections 2024-09-17T04:28:24.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.060+0000 7fa798a00640 1 --2- 172.21.15.100:0/1290779611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa79415d540 0x7fa794156e40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:24.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.060+0000 7fa798a00640 1 -- 172.21.15.100:0/1290779611 >> 172.21.15.100:0/1290779611 conn(0x7fa79406d3d0 msgr2=0x7fa7940b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:24.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.060+0000 7fa798a00640 1 -- 172.21.15.100:0/1290779611 shutdown_connections 2024-09-17T04:28:24.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.061+0000 7fa798a00640 1 -- 172.21.15.100:0/1290779611 wait complete. 2024-09-17T04:28:24.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.063+0000 7fa798a00640 1 Processor -- start 2024-09-17T04:28:24.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.063+0000 7fa798a00640 1 -- start start 2024-09-17T04:28:24.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.064+0000 7fa798a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7940cfc80 0x7fa7940b0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:24.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.064+0000 7fa798a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa79415d540 0x7fa7940c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:24.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.064+0000 7fa798a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7941f96b0 0x7fa7941f8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:24.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.064+0000 7fa798a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa7940cd1e0 con 0x7fa79415d540 2024-09-17T04:28:24.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.064+0000 7fa798a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa7940cd060 con 0x7fa7940cfc80 2024-09-17T04:28:24.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.064+0000 7fa798a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa7940cd360 con 0x7fa7941f96b0 2024-09-17T04:28:24.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.065+0000 7fa791400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa79415d540 0x7fa7940c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:24.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.065+0000 7fa791400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa79415d540 0x7fa7940c53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52108/0 (socket says 172.21.15.100:52108) 2024-09-17T04:28:24.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.065+0000 7fa791400640 1 -- 172.21.15.100:0/3260642158 learned_addr learned my addr 172.21.15.100:0/3260642158 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:24.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.065+0000 7fa792800640 1 --2- 172.21.15.100:0/3260642158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7941f96b0 0x7fa7941f8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:24.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.065+0000 7fa791e00640 1 --2- 172.21.15.100:0/3260642158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7940cfc80 0x7fa7940b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:24.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.065+0000 7fa791400640 1 -- 172.21.15.100:0/3260642158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7941f96b0 msgr2=0x7fa7941f8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:24.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.065+0000 7fa791400640 1 --2- 172.21.15.100:0/3260642158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7941f96b0 0x7fa7941f8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:24.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.065+0000 7fa791400640 1 -- 172.21.15.100:0/3260642158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7940cfc80 msgr2=0x7fa7940b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:24.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.065+0000 7fa791400640 1 --2- 172.21.15.100:0/3260642158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7940cfc80 0x7fa7940b0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:24.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.065+0000 7fa791400640 1 -- 172.21.15.100:0/3260642158 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7941f6790 con 0x7fa79415d540 2024-09-17T04:28:24.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.065+0000 7fa791400640 1 --2- 172.21.15.100:0/3260642158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa79415d540 0x7fa7940c53c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa78000c430 tx=0x7fa78000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:24.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.066+0000 7fa78a000640 1 -- 172.21.15.100:0/3260642158 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa780011790 con 0x7fa79415d540 2024-09-17T04:28:24.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.066+0000 7fa78a000640 1 -- 172.21.15.100:0/3260642158 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa780017070 con 0x7fa79415d540 2024-09-17T04:28:24.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.066+0000 7fa798a00640 1 -- 172.21.15.100:0/3260642158 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7940b1020 con 0x7fa79415d540 2024-09-17T04:28:24.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.066+0000 7fa78a000640 1 -- 172.21.15.100:0/3260642158 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa780011cc0 con 0x7fa79415d540 2024-09-17T04:28:24.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.066+0000 7fa798a00640 1 -- 172.21.15.100:0/3260642158 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7940b15e0 con 0x7fa79415d540 2024-09-17T04:28:24.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.067+0000 7fa798a00640 1 -- 172.21.15.100:0/3260642158 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7940b0d90 con 0x7fa79415d540 2024-09-17T04:28:24.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.068+0000 7fa78a000640 1 -- 172.21.15.100:0/3260642158 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fa78001c460 con 0x7fa79415d540 2024-09-17T04:28:24.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.069+0000 7fa78a000640 1 --2- 172.21.15.100:0/3260642158 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa774078ef0 0x7fa77407b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:24.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.069+0000 7fa78a000640 1 -- 172.21.15.100:0/3260642158 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7fa78009b140 con 0x7fa79415d540 2024-09-17T04:28:24.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.070+0000 7fa791e00640 1 --2- 172.21.15.100:0/3260642158 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa774078ef0 0x7fa77407b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:24.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.070+0000 7fa791e00640 1 --2- 172.21.15.100:0/3260642158 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa774078ef0 0x7fa77407b400 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa784007fd0 tx=0x7fa784009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:24.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.073+0000 7fa78a000640 1 -- 172.21.15.100:0/3260642158 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa7800630b0 con 0x7fa79415d540 2024-09-17T04:28:24.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.298+0000 7fa798a00640 1 -- 172.21.15.100:0/3260642158 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa7941f6960 con 0x7fa79415d540 2024-09-17T04:28:24.299 INFO:tasks.ceph.mon.a.smithi100.stderr:2024-09-17T04:28:24.299+0000 7fd3d0a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T04:28:24.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.301+0000 7fa78a000640 1 -- 172.21.15.100:0/3260642158 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) ==== 161+0+0 (secure 0 0 0) 0x7fa7800629b0 con 0x7fa79415d540 2024-09-17T04:28:24.301 INFO:teuthology.orchestra.run.smithi100.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T04:28:24.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.304+0000 7fa798a00640 1 -- 172.21.15.100:0/3260642158 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa774078ef0 msgr2=0x7fa77407b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:24.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.304+0000 7fa798a00640 1 --2- 172.21.15.100:0/3260642158 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa774078ef0 0x7fa77407b400 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa784007fd0 tx=0x7fa784009040 comp rx=0 tx=0).stop 2024-09-17T04:28:24.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.304+0000 7fa798a00640 1 -- 172.21.15.100:0/3260642158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa79415d540 msgr2=0x7fa7940c53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:24.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.304+0000 7fa798a00640 1 --2- 172.21.15.100:0/3260642158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa79415d540 0x7fa7940c53c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa78000c430 tx=0x7fa78000c8f0 comp rx=0 tx=0).stop 2024-09-17T04:28:24.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.304+0000 7fa798a00640 1 -- 172.21.15.100:0/3260642158 shutdown_connections 2024-09-17T04:28:24.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.304+0000 7fa798a00640 1 --2- 172.21.15.100:0/3260642158 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa774078ef0 0x7fa77407b400 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:24.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.304+0000 7fa798a00640 1 --2- 172.21.15.100:0/3260642158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7941f96b0 0x7fa7941f8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:24.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.304+0000 7fa798a00640 1 --2- 172.21.15.100:0/3260642158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa79415d540 0x7fa7940c53c0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:24.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.304+0000 7fa798a00640 1 --2- 172.21.15.100:0/3260642158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7940cfc80 0x7fa7940b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:24.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.304+0000 7fa798a00640 1 -- 172.21.15.100:0/3260642158 >> 172.21.15.100:0/3260642158 conn(0x7fa79406d3d0 msgr2=0x7fa79415ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:24.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.304+0000 7fa798a00640 1 -- 172.21.15.100:0/3260642158 shutdown_connections 2024-09-17T04:28:24.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.304+0000 7fa798a00640 1 -- 172.21.15.100:0/3260642158 wait complete. 2024-09-17T04:28:24.328 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T04:28:24.447 INFO:tasks.ceph.mds.a.smithi100.stdout:starting mds.a at 2024-09-17T04:28:24.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.555+0000 7fc2f2600640 1 -- 172.21.15.100:0/1502668060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2ec0cf310 msgr2=0x7fc2ec0d45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:24.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.555+0000 7fc2f2600640 1 --2- 172.21.15.100:0/1502668060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2ec0cf310 0x7fc2ec0d45a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec08ca50 tx=0x7fc2e402aa30 comp rx=0 tx=0).stop 2024-09-17T04:28:24.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.555+0000 7fc2f2600640 1 -- 172.21.15.100:0/1502668060 shutdown_connections 2024-09-17T04:28:24.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.555+0000 7fc2f2600640 1 --2- 172.21.15.100:0/1502668060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2ec0cf310 0x7fc2ec0d45a0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:24.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.555+0000 7fc2f2600640 1 --2- 172.21.15.100:0/1502668060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2ec161e90 0x7fc2ec0cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:24.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.555+0000 7fc2f2600640 1 --2- 172.21.15.100:0/1502668060 >> v2:172.21.15.184:3300/0 conn(0x7fc2ec0b9680 0x7fc2ec0b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:24.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.555+0000 7fc2f2600640 1 -- 172.21.15.100:0/1502668060 >> 172.21.15.100:0/1502668060 conn(0x7fc2ec09d960 msgr2=0x7fc2ec08bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:24.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.555+0000 7fc2f2600640 1 -- 172.21.15.100:0/1502668060 shutdown_connections 2024-09-17T04:28:24.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.555+0000 7fc2f2600640 1 -- 172.21.15.100:0/1502668060 wait complete. 2024-09-17T04:28:24.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.557+0000 7fc2f2600640 1 Processor -- start 2024-09-17T04:28:24.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.557+0000 7fc2f2600640 1 -- start start 2024-09-17T04:28:24.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.559+0000 7fc2f2600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2ec0b9680 0x7fc2ec1529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:24.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.559+0000 7fc2f2600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2ec0cf310 0x7fc2ec0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:24.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.559+0000 7fc2f2600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2ec161e90 0x7fc2ec0cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:24.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.559+0000 7fc2f2600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec06f220 con 0x7fc2ec161e90 2024-09-17T04:28:24.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.559+0000 7fc2f2600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec0d4e30 con 0x7fc2ec0b9680 2024-09-17T04:28:24.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.559+0000 7fc2f2600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc2ec06f3a0 con 0x7fc2ec0cf310 2024-09-17T04:28:24.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.559+0000 7fc2f1c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2ec161e90 0x7fc2ec0cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:24.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.559+0000 7fc2f1c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2ec161e90 0x7fc2ec0cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52138/0 (socket says 172.21.15.100:52138) 2024-09-17T04:28:24.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.559+0000 7fc2f1c00640 1 -- 172.21.15.100:0/2029894607 learned_addr learned my addr 172.21.15.100:0/2029894607 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:24.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.559+0000 7fc2f1200640 1 --2- 172.21.15.100:0/2029894607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2ec0b9680 0x7fc2ec1529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:24.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.559+0000 7fc2e3e00640 1 --2- 172.21.15.100:0/2029894607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2ec0cf310 0x7fc2ec0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:24.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.560+0000 7fc2f1c00640 1 -- 172.21.15.100:0/2029894607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2ec0cf310 msgr2=0x7fc2ec0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:24.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.560+0000 7fc2f1c00640 1 --2- 172.21.15.100:0/2029894607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2ec0cf310 0x7fc2ec0cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:24.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.560+0000 7fc2f1c00640 1 -- 172.21.15.100:0/2029894607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2ec0b9680 msgr2=0x7fc2ec1529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:24.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.560+0000 7fc2f1c00640 1 --2- 172.21.15.100:0/2029894607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2ec0b9680 0x7fc2ec1529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:24.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.560+0000 7fc2f1c00640 1 -- 172.21.15.100:0/2029894607 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ec0ce660 con 0x7fc2ec161e90 2024-09-17T04:28:24.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.560+0000 7fc2e3e00640 1 --2- 172.21.15.100:0/2029894607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2ec0cf310 0x7fc2ec0cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:28:24.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.560+0000 7fc2f1200640 1 --2- 172.21.15.100:0/2029894607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2ec0b9680 0x7fc2ec1529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:28:24.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.560+0000 7fc2f1c00640 1 --2- 172.21.15.100:0/2029894607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2ec161e90 0x7fc2ec0cca40 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4009d60 tx=0x7fc2e4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:24.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.560+0000 7fc2e9600640 1 -- 172.21.15.100:0/2029894607 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4033050 con 0x7fc2ec161e90 2024-09-17T04:28:24.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.560+0000 7fc2e9600640 1 -- 172.21.15.100:0/2029894607 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2e4004450 con 0x7fc2ec161e90 2024-09-17T04:28:24.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.560+0000 7fc2f2600640 1 -- 172.21.15.100:0/2029894607 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2ec1f8e00 con 0x7fc2ec161e90 2024-09-17T04:28:24.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.560+0000 7fc2e9600640 1 -- 172.21.15.100:0/2029894607 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4038450 con 0x7fc2ec161e90 2024-09-17T04:28:24.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.560+0000 7fc2f2600640 1 -- 172.21.15.100:0/2029894607 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2ec1f93c0 con 0x7fc2ec161e90 2024-09-17T04:28:24.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.562+0000 7fc2f2600640 1 -- 172.21.15.100:0/2029894607 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2ec0ce3b0 con 0x7fc2ec161e90 2024-09-17T04:28:24.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.563+0000 7fc2e9600640 1 -- 172.21.15.100:0/2029894607 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fc2e4036070 con 0x7fc2ec161e90 2024-09-17T04:28:24.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.564+0000 7fc2e9600640 1 --2- 172.21.15.100:0/2029894607 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc2c8078ef0 0x7fc2c807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:24.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.564+0000 7fc2e9600640 1 -- 172.21.15.100:0/2029894607 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7fc2e40bc8d0 con 0x7fc2ec161e90 2024-09-17T04:28:24.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.564+0000 7fc2f1200640 1 --2- 172.21.15.100:0/2029894607 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc2c8078ef0 0x7fc2c807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:24.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.565+0000 7fc2f1200640 1 --2- 172.21.15.100:0/2029894607 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc2c8078ef0 0x7fc2c807b400 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc009bd0 tx=0x7fc2dc005f30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:24.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.568+0000 7fc2e9600640 1 -- 172.21.15.100:0/2029894607 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc2e40846f0 con 0x7fc2ec161e90 2024-09-17T04:28:24.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:24.790+0000 7fc2f2600640 1 -- 172.21.15.100:0/2029894607 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc2ec0d45a0 con 0x7fc2ec161e90 2024-09-17T04:28:25.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.303+0000 7fc2e9600640 1 -- 172.21.15.100:0/2029894607 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) ==== 104+0+0 (secure 0 0 0) 0x7fc2e4083ff0 con 0x7fc2ec161e90 2024-09-17T04:28:25.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.305+0000 7fc2f2600640 1 -- 172.21.15.100:0/2029894607 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc2c8078ef0 msgr2=0x7fc2c807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:25.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.305+0000 7fc2f2600640 1 --2- 172.21.15.100:0/2029894607 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc2c8078ef0 0x7fc2c807b400 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc009bd0 tx=0x7fc2dc005f30 comp rx=0 tx=0).stop 2024-09-17T04:28:25.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.306+0000 7fc2f2600640 1 -- 172.21.15.100:0/2029894607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2ec161e90 msgr2=0x7fc2ec0cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:25.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.306+0000 7fc2f2600640 1 --2- 172.21.15.100:0/2029894607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2ec161e90 0x7fc2ec0cca40 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4009d60 tx=0x7fc2e4003f10 comp rx=0 tx=0).stop 2024-09-17T04:28:25.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.306+0000 7fc2f2600640 1 -- 172.21.15.100:0/2029894607 shutdown_connections 2024-09-17T04:28:25.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.306+0000 7fc2f2600640 1 --2- 172.21.15.100:0/2029894607 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc2c8078ef0 0x7fc2c807b400 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:25.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.306+0000 7fc2f2600640 1 --2- 172.21.15.100:0/2029894607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2ec161e90 0x7fc2ec0cca40 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:25.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.306+0000 7fc2f2600640 1 --2- 172.21.15.100:0/2029894607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2ec0cf310 0x7fc2ec0cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:25.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.306+0000 7fc2f2600640 1 --2- 172.21.15.100:0/2029894607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2ec0b9680 0x7fc2ec1529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:25.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.306+0000 7fc2f2600640 1 -- 172.21.15.100:0/2029894607 >> 172.21.15.100:0/2029894607 conn(0x7fc2ec09d960 msgr2=0x7fc2ec0b8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:25.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.306+0000 7fc2f2600640 1 -- 172.21.15.100:0/2029894607 shutdown_connections 2024-09-17T04:28:25.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.306+0000 7fc2f2600640 1 -- 172.21.15.100:0/2029894607 wait complete. 2024-09-17T04:28:25.334 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T04:28:25.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.559+0000 7f4428a00640 1 -- 172.21.15.100:0/2894077732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44240b3ec0 msgr2=0x7f44240b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:25.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.559+0000 7f4428a00640 1 --2- 172.21.15.100:0/2894077732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44240b3ec0 0x7f44240b47c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f44240d0050 tx=0x7f441402a4a0 comp rx=0 tx=0).stop 2024-09-17T04:28:25.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.560+0000 7f4428a00640 1 -- 172.21.15.100:0/2894077732 shutdown_connections 2024-09-17T04:28:25.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.560+0000 7f4428a00640 1 --2- 172.21.15.100:0/2894077732 >> v2:172.21.15.184:3300/0 conn(0x7f44240af3c0 0x7f44241602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:25.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.560+0000 7f4428a00640 1 --2- 172.21.15.100:0/2894077732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44240b3ec0 0x7f44240b47c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:25.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.560+0000 7f4428a00640 1 -- 172.21.15.100:0/2894077732 >> 172.21.15.100:0/2894077732 conn(0x7f44240b1dd0 msgr2=0x7f44240b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:25.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.560+0000 7f4428a00640 1 -- 172.21.15.100:0/2894077732 shutdown_connections 2024-09-17T04:28:25.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.560+0000 7f4428a00640 1 -- 172.21.15.100:0/2894077732 wait complete. 2024-09-17T04:28:25.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.562+0000 7f4428a00640 1 Processor -- start 2024-09-17T04:28:25.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.562+0000 7f4428a00640 1 -- start start 2024-09-17T04:28:25.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.563+0000 7f4428a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44240705b0 0x7f4424156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:25.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.563+0000 7f4428a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44240af3c0 0x7f4424146660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:25.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.564+0000 7f4428a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44240b3ec0 0x7f44241f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:25.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.564+0000 7f4428a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4424160b80 con 0x7f44240af3c0 2024-09-17T04:28:25.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.564+0000 7f4428a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4424160a00 con 0x7f44240705b0 2024-09-17T04:28:25.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.564+0000 7f4428a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f442415cc60 con 0x7f44240b3ec0 2024-09-17T04:28:25.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.564+0000 7f4422a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44240af3c0 0x7f4424146660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:25.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.564+0000 7f4422a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44240af3c0 0x7f4424146660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57792/0 (socket says 172.21.15.100:57792) 2024-09-17T04:28:25.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.564+0000 7f4422a00640 1 -- 172.21.15.100:0/146419243 learned_addr learned my addr 172.21.15.100:0/146419243 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:25.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.564+0000 7f4423400640 1 --2- 172.21.15.100:0/146419243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44240705b0 0x7f4424156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:25.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.564+0000 7f4423e00640 1 --2- 172.21.15.100:0/146419243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44240b3ec0 0x7f44241f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:25.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.564+0000 7f4422a00640 1 -- 172.21.15.100:0/146419243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44240b3ec0 msgr2=0x7f44241f4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:25.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.564+0000 7f4422a00640 1 --2- 172.21.15.100:0/146419243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44240b3ec0 0x7f44241f4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:25.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.564+0000 7f4422a00640 1 -- 172.21.15.100:0/146419243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44240705b0 msgr2=0x7f4424156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:25.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.564+0000 7f4422a00640 1 --2- 172.21.15.100:0/146419243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44240705b0 0x7f4424156600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:25.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.564+0000 7f4422a00640 1 -- 172.21.15.100:0/146419243 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4424149540 con 0x7f44240af3c0 2024-09-17T04:28:25.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.565+0000 7f4422a00640 1 --2- 172.21.15.100:0/146419243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44240af3c0 0x7f4424146660 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f441000c3c0 tx=0x7f441000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:25.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.565+0000 7f441b400640 1 -- 172.21.15.100:0/146419243 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4410017070 con 0x7f44240af3c0 2024-09-17T04:28:25.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.565+0000 7f441b400640 1 -- 172.21.15.100:0/146419243 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44100035f0 con 0x7f44240af3c0 2024-09-17T04:28:25.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.565+0000 7f441b400640 1 -- 172.21.15.100:0/146419243 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4410003a80 con 0x7f44240af3c0 2024-09-17T04:28:25.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.565+0000 7f4428a00640 1 -- 172.21.15.100:0/146419243 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4424155560 con 0x7f44240af3c0 2024-09-17T04:28:25.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.565+0000 7f4428a00640 1 -- 172.21.15.100:0/146419243 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4424155b20 con 0x7f44240af3c0 2024-09-17T04:28:25.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.566+0000 7f4428a00640 1 -- 172.21.15.100:0/146419243 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44241602b0 con 0x7f44240af3c0 2024-09-17T04:28:25.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.567+0000 7f441b400640 1 -- 172.21.15.100:0/146419243 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f441001a070 con 0x7f44240af3c0 2024-09-17T04:28:25.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.569+0000 7f441b400640 1 --2- 172.21.15.100:0/146419243 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4404078ef0 0x7f440407b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:25.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.569+0000 7f441b400640 1 -- 172.21.15.100:0/146419243 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f441009a5c0 con 0x7f44240af3c0 2024-09-17T04:28:25.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.569+0000 7f4423400640 1 --2- 172.21.15.100:0/146419243 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4404078ef0 0x7f440407b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:25.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.570+0000 7f4423400640 1 --2- 172.21.15.100:0/146419243 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4404078ef0 0x7f440407b400 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4414034000 tx=0x7f4414004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:25.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.574+0000 7f441b400640 1 -- 172.21.15.100:0/146419243 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4424149540 con 0x7f44240af3c0 2024-09-17T04:28:25.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:25.755+0000 7f4428a00640 1 -- 172.21.15.100:0/146419243 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f44240b25f0 con 0x7f44240af3c0 2024-09-17T04:28:26.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.313+0000 7f441b400640 1 -- 172.21.15.100:0/146419243 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v63) ==== 181+0+0 (secure 0 0 0) 0x7f44240b4ec0 con 0x7f44240af3c0 2024-09-17T04:28:26.312 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' removed 2024-09-17T04:28:26.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.315+0000 7f4428a00640 1 -- 172.21.15.100:0/146419243 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4404078ef0 msgr2=0x7f440407b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:26.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.315+0000 7f4428a00640 1 --2- 172.21.15.100:0/146419243 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4404078ef0 0x7f440407b400 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4414034000 tx=0x7f4414004730 comp rx=0 tx=0).stop 2024-09-17T04:28:26.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.315+0000 7f4428a00640 1 -- 172.21.15.100:0/146419243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44240af3c0 msgr2=0x7f4424146660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:26.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.315+0000 7f4428a00640 1 --2- 172.21.15.100:0/146419243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44240af3c0 0x7f4424146660 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f441000c3c0 tx=0x7f441000c880 comp rx=0 tx=0).stop 2024-09-17T04:28:26.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.315+0000 7f4428a00640 1 -- 172.21.15.100:0/146419243 shutdown_connections 2024-09-17T04:28:26.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.315+0000 7f4428a00640 1 --2- 172.21.15.100:0/146419243 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4404078ef0 0x7f440407b400 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:26.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.315+0000 7f4428a00640 1 --2- 172.21.15.100:0/146419243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44240b3ec0 0x7f44241f4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:26.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.315+0000 7f4428a00640 1 --2- 172.21.15.100:0/146419243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44240af3c0 0x7f4424146660 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:26.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.315+0000 7f4428a00640 1 --2- 172.21.15.100:0/146419243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44240705b0 0x7f4424156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:26.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.315+0000 7f4428a00640 1 -- 172.21.15.100:0/146419243 >> 172.21.15.100:0/146419243 conn(0x7f44240b1dd0 msgr2=0x7f44240cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:26.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.316+0000 7f4428a00640 1 -- 172.21.15.100:0/146419243 shutdown_connections 2024-09-17T04:28:26.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.316+0000 7f4428a00640 1 -- 172.21.15.100:0/146419243 wait complete. 2024-09-17T04:28:26.340 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T04:28:26.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.567+0000 7faf4be00640 1 -- 172.21.15.100:0/1976426349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf2c003a00 msgr2=0x7faf2c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:26.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.567+0000 7faf4be00640 1 --2- 172.21.15.100:0/1976426349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf2c003a00 0x7faf2c003e70 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7faf3800e3d0 tx=0x7faf2c005830 comp rx=0 tx=0).stop 2024-09-17T04:28:26.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.567+0000 7faf49e00640 1 -- 172.21.15.100:0/1976426349 reap_dead start 2024-09-17T04:28:26.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.567+0000 7faf4be00640 1 -- 172.21.15.100:0/1976426349 shutdown_connections 2024-09-17T04:28:26.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.567+0000 7faf4be00640 1 -- 172.21.15.100:0/1976426349 >> 172.21.15.100:0/1976426349 conn(0x7faf440b2540 msgr2=0x7faf440d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:26.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.567+0000 7faf4be00640 1 -- 172.21.15.100:0/1976426349 shutdown_connections 2024-09-17T04:28:26.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.567+0000 7faf4be00640 1 -- 172.21.15.100:0/1976426349 wait complete. 2024-09-17T04:28:26.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.569+0000 7faf4be00640 1 Processor -- start 2024-09-17T04:28:26.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.570+0000 7faf4be00640 1 -- start start 2024-09-17T04:28:26.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.571+0000 7faf4be00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf2c003a00 0x7faf4409a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:26.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.571+0000 7faf4be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf440989c0 0x7faf441531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:26.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.571+0000 7faf4be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf44155730 0x7faf44155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:26.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.571+0000 7faf4be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faf440b3120 con 0x7faf440989c0 2024-09-17T04:28:26.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.571+0000 7faf4be00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faf440afb60 con 0x7faf44155730 2024-09-17T04:28:26.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.571+0000 7faf4be00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faf440b32a0 con 0x7faf2c003a00 2024-09-17T04:28:26.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.571+0000 7faf48a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf440989c0 0x7faf441531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:26.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.571+0000 7faf48a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf440989c0 0x7faf441531d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57808/0 (socket says 172.21.15.100:57808) 2024-09-17T04:28:26.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.571+0000 7faf48a00640 1 -- 172.21.15.100:0/1974052115 learned_addr learned my addr 172.21.15.100:0/1974052115 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:26.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.571+0000 7faf49400640 1 --2- 172.21.15.100:0/1974052115 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf2c003a00 0x7faf4409a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:26.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.572+0000 7faf49e00640 1 --2- 172.21.15.100:0/1974052115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf44155730 0x7faf44155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:26.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.572+0000 7faf48a00640 1 -- 172.21.15.100:0/1974052115 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf2c003a00 msgr2=0x7faf4409a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:26.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.572+0000 7faf48a00640 1 --2- 172.21.15.100:0/1974052115 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf2c003a00 0x7faf4409a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:26.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.572+0000 7faf48a00640 1 -- 172.21.15.100:0/1974052115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf44155730 msgr2=0x7faf44155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:26.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.572+0000 7faf48a00640 1 --2- 172.21.15.100:0/1974052115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf44155730 0x7faf44155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:26.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.572+0000 7faf48a00640 1 -- 172.21.15.100:0/1974052115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4409daf0 con 0x7faf440989c0 2024-09-17T04:28:26.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.572+0000 7faf48a00640 1 --2- 172.21.15.100:0/1974052115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf440989c0 0x7faf441531d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7faf30006630 tx=0x7faf300039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:26.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.572+0000 7faf41600640 1 -- 172.21.15.100:0/1974052115 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf30015070 con 0x7faf440989c0 2024-09-17T04:28:26.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.572+0000 7faf41600640 1 -- 172.21.15.100:0/1974052115 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf300119c0 con 0x7faf440989c0 2024-09-17T04:28:26.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.572+0000 7faf4be00640 1 -- 172.21.15.100:0/1974052115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf4409ddd0 con 0x7faf440989c0 2024-09-17T04:28:26.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.572+0000 7faf41600640 1 -- 172.21.15.100:0/1974052115 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf30011e10 con 0x7faf440989c0 2024-09-17T04:28:26.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.573+0000 7faf4be00640 1 -- 172.21.15.100:0/1974052115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf44155210 con 0x7faf440989c0 2024-09-17T04:28:26.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.574+0000 7faf4be00640 1 -- 172.21.15.100:0/1974052115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf4409d860 con 0x7faf440989c0 2024-09-17T04:28:26.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.575+0000 7faf41600640 1 -- 172.21.15.100:0/1974052115 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7faf30018070 con 0x7faf440989c0 2024-09-17T04:28:26.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.576+0000 7faf41600640 1 --2- 172.21.15.100:0/1974052115 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faf24078ef0 0x7faf2407b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:26.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.576+0000 7faf41600640 1 -- 172.21.15.100:0/1974052115 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5357+0+0 (secure 0 0 0) 0x7faf3009bfe0 con 0x7faf440989c0 2024-09-17T04:28:26.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.576+0000 7faf49400640 1 --2- 172.21.15.100:0/1974052115 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faf24078ef0 0x7faf2407b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:26.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.577+0000 7faf49400640 1 --2- 172.21.15.100:0/1974052115 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faf24078ef0 0x7faf2407b400 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faf38035000 tx=0x7faf38004a40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:26.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.580+0000 7faf41600640 1 -- 172.21.15.100:0/1974052115 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7faf300639a0 con 0x7faf440989c0 2024-09-17T04:28:26.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:26.763+0000 7faf4be00640 1 -- 172.21.15.100:0/1974052115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7faf44074eb0 con 0x7faf440989c0 2024-09-17T04:28:27.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.319+0000 7faf41600640 1 -- 172.21.15.100:0/1974052115 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v64) ==== 169+0+0 (secure 0 0 0) 0x7faf300683d0 con 0x7faf440989c0 2024-09-17T04:28:27.319 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' removed 2024-09-17T04:28:27.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.323+0000 7faf4be00640 1 -- 172.21.15.100:0/1974052115 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faf24078ef0 msgr2=0x7faf2407b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:27.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.323+0000 7faf4be00640 1 --2- 172.21.15.100:0/1974052115 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faf24078ef0 0x7faf2407b400 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faf38035000 tx=0x7faf38004a40 comp rx=0 tx=0).stop 2024-09-17T04:28:27.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.323+0000 7faf4be00640 1 -- 172.21.15.100:0/1974052115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf440989c0 msgr2=0x7faf441531d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:27.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.323+0000 7faf4be00640 1 --2- 172.21.15.100:0/1974052115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf440989c0 0x7faf441531d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7faf30006630 tx=0x7faf300039c0 comp rx=0 tx=0).stop 2024-09-17T04:28:27.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.324+0000 7faf4be00640 1 -- 172.21.15.100:0/1974052115 shutdown_connections 2024-09-17T04:28:27.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.324+0000 7faf4be00640 1 --2- 172.21.15.100:0/1974052115 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7faf24078ef0 0x7faf2407b400 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:27.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.324+0000 7faf4be00640 1 --2- 172.21.15.100:0/1974052115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf44155730 0x7faf44155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:27.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.324+0000 7faf4be00640 1 --2- 172.21.15.100:0/1974052115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf440989c0 0x7faf441531d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:27.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.324+0000 7faf4be00640 1 --2- 172.21.15.100:0/1974052115 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf2c003a00 0x7faf4409a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:27.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.324+0000 7faf4be00640 1 -- 172.21.15.100:0/1974052115 >> 172.21.15.100:0/1974052115 conn(0x7faf440b2540 msgr2=0x7faf440b38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:27.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.324+0000 7faf4be00640 1 -- 172.21.15.100:0/1974052115 shutdown_connections 2024-09-17T04:28:27.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.324+0000 7faf4be00640 1 -- 172.21.15.100:0/1974052115 wait complete. 2024-09-17T04:28:27.350 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T04:28:27.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.571+0000 7fc70ca00640 1 -- 172.21.15.100:0/212902640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc70815ce30 msgr2=0x7fc70815d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:27.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.571+0000 7fc70ca00640 1 --2- 172.21.15.100:0/212902640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc70815ce30 0x7fc70815d240 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fc700005fd0 tx=0x7fc70002a870 comp rx=0 tx=0).stop 2024-09-17T04:28:27.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.571+0000 7fc70ca00640 1 -- 172.21.15.100:0/212902640 shutdown_connections 2024-09-17T04:28:27.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.571+0000 7fc70ca00640 1 --2- 172.21.15.100:0/212902640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc70815ce30 0x7fc70815d240 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:27.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.571+0000 7fc70ca00640 1 --2- 172.21.15.100:0/212902640 >> v2:172.21.15.184:3300/0 conn(0x7fc708070cc0 0x7fc708156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:27.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.571+0000 7fc70ca00640 1 -- 172.21.15.100:0/212902640 >> 172.21.15.100:0/212902640 conn(0x7fc70806d3d0 msgr2=0x7fc7080b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:27.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.571+0000 7fc70ca00640 1 -- 172.21.15.100:0/212902640 shutdown_connections 2024-09-17T04:28:27.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.571+0000 7fc70ca00640 1 -- 172.21.15.100:0/212902640 wait complete. 2024-09-17T04:28:27.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.573+0000 7fc70ca00640 1 Processor -- start 2024-09-17T04:28:27.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.574+0000 7fc70ca00640 1 -- start start 2024-09-17T04:28:27.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.575+0000 7fc70ca00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7080cfc80 0x7fc708155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:27.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.575+0000 7fc70ca00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc70815ce30 0x7fc7080b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:27.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.575+0000 7fc70ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7081f4060 0x7fc7080b1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:27.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.575+0000 7fc70ca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc7080cd050 con 0x7fc7081f4060 2024-09-17T04:28:27.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.575+0000 7fc70ca00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc7080cced0 con 0x7fc7080cfc80 2024-09-17T04:28:27.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.575+0000 7fc70ca00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc7080cd1d0 con 0x7fc70815ce30 2024-09-17T04:28:27.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.575+0000 7fc706800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7081f4060 0x7fc7080b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:27.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.575+0000 7fc706800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7081f4060 0x7fc7080b1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57822/0 (socket says 172.21.15.100:57822) 2024-09-17T04:28:27.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.575+0000 7fc706800640 1 -- 172.21.15.100:0/199555505 learned_addr learned my addr 172.21.15.100:0/199555505 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:27.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.576+0000 7fc706800640 1 -- 172.21.15.100:0/199555505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc70815ce30 msgr2=0x7fc7080b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:27.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.576+0000 7fc706800640 1 --2- 172.21.15.100:0/199555505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc70815ce30 0x7fc7080b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:27.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.576+0000 7fc706800640 1 -- 172.21.15.100:0/199555505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7080cfc80 msgr2=0x7fc708155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:27.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.576+0000 7fc705e00640 1 --2- 172.21.15.100:0/199555505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7080cfc80 0x7fc708155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:27.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.576+0000 7fc706800640 1 --2- 172.21.15.100:0/199555505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7080cfc80 0x7fc708155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:27.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.576+0000 7fc706800640 1 -- 172.21.15.100:0/199555505 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7080af040 con 0x7fc7081f4060 2024-09-17T04:28:27.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.576+0000 7fc705e00640 1 --2- 172.21.15.100:0/199555505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7080cfc80 0x7fc708155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:28:27.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.576+0000 7fc706800640 1 --2- 172.21.15.100:0/199555505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7081f4060 0x7fc7080b1280 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fc700009cd0 tx=0x7fc700033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:27.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.576+0000 7fc6fe000640 1 -- 172.21.15.100:0/199555505 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700042020 con 0x7fc7081f4060 2024-09-17T04:28:27.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.576+0000 7fc6fe000640 1 -- 172.21.15.100:0/199555505 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc700033db0 con 0x7fc7081f4060 2024-09-17T04:28:27.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.576+0000 7fc70ca00640 1 -- 172.21.15.100:0/199555505 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7081f3580 con 0x7fc7081f4060 2024-09-17T04:28:27.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.576+0000 7fc6fe000640 1 -- 172.21.15.100:0/199555505 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700037040 con 0x7fc7081f4060 2024-09-17T04:28:27.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.577+0000 7fc70ca00640 1 -- 172.21.15.100:0/199555505 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7081f3b40 con 0x7fc7081f4060 2024-09-17T04:28:27.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.577+0000 7fc70ca00640 1 -- 172.21.15.100:0/199555505 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7080aedb0 con 0x7fc7081f4060 2024-09-17T04:28:27.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.579+0000 7fc6fe000640 1 -- 172.21.15.100:0/199555505 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fc700045020 con 0x7fc7081f4060 2024-09-17T04:28:27.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.580+0000 7fc6fe000640 1 --2- 172.21.15.100:0/199555505 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc6e8078ef0 0x7fc6e807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:27.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.580+0000 7fc6fe000640 1 -- 172.21.15.100:0/199555505 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4943+0+0 (secure 0 0 0) 0x7fc7000bc0f0 con 0x7fc7081f4060 2024-09-17T04:28:27.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.580+0000 7fc705e00640 1 --2- 172.21.15.100:0/199555505 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc6e8078ef0 0x7fc6e807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:27.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.581+0000 7fc705e00640 1 --2- 172.21.15.100:0/199555505 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc6e8078ef0 0x7fc6e807b400 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc6f80046b0 tx=0x7fc6f8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:27.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.584+0000 7fc6fe000640 1 -- 172.21.15.100:0/199555505 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc70004c030 con 0x7fc7081f4060 2024-09-17T04:28:27.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.784+0000 7fc70ca00640 1 -- 172.21.15.100:0/199555505 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fc7081f3390 con 0x7fc7081f4060 2024-09-17T04:28:27.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.789+0000 7fc6fe000640 1 -- 172.21.15.100:0/199555505 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v37) ==== 71+0+0 (secure 0 0 0) 0x7fc700084360 con 0x7fc7081f4060 2024-09-17T04:28:27.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.791+0000 7fc70ca00640 1 -- 172.21.15.100:0/199555505 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc6e8078ef0 msgr2=0x7fc6e807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:27.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.791+0000 7fc70ca00640 1 --2- 172.21.15.100:0/199555505 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc6e8078ef0 0x7fc6e807b400 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc6f80046b0 tx=0x7fc6f8009040 comp rx=0 tx=0).stop 2024-09-17T04:28:27.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.791+0000 7fc70ca00640 1 -- 172.21.15.100:0/199555505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7081f4060 msgr2=0x7fc7080b1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:27.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.791+0000 7fc70ca00640 1 --2- 172.21.15.100:0/199555505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7081f4060 0x7fc7080b1280 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fc700009cd0 tx=0x7fc700033ad0 comp rx=0 tx=0).stop 2024-09-17T04:28:27.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.791+0000 7fc70ca00640 1 -- 172.21.15.100:0/199555505 shutdown_connections 2024-09-17T04:28:27.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.791+0000 7fc70ca00640 1 --2- 172.21.15.100:0/199555505 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc6e8078ef0 0x7fc6e807b400 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:27.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.791+0000 7fc70ca00640 1 --2- 172.21.15.100:0/199555505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7081f4060 0x7fc7080b1280 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:27.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.791+0000 7fc70ca00640 1 --2- 172.21.15.100:0/199555505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc70815ce30 0x7fc7080b0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:27.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.791+0000 7fc70ca00640 1 --2- 172.21.15.100:0/199555505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7080cfc80 0x7fc708155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:27.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.791+0000 7fc70ca00640 1 -- 172.21.15.100:0/199555505 >> 172.21.15.100:0/199555505 conn(0x7fc70806d3d0 msgr2=0x7fc7081603b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:27.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.791+0000 7fc70ca00640 1 -- 172.21.15.100:0/199555505 shutdown_connections 2024-09-17T04:28:27.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:27.791+0000 7fc70ca00640 1 -- 172.21.15.100:0/199555505 wait complete. 2024-09-17T04:28:27.817 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2024-09-17T04:28:28.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.038+0000 7fcb3ba00640 1 -- 172.21.15.100:0/413439587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb340ad0e0 msgr2=0x7fcb340ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:28.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.038+0000 7fcb3ba00640 1 --2- 172.21.15.100:0/413439587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb340ad0e0 0x7fcb340ad4f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcb20009830 tx=0x7fcb2002a410 comp rx=0 tx=0).stop 2024-09-17T04:28:28.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.038+0000 7fcb3ba00640 1 -- 172.21.15.100:0/413439587 shutdown_connections 2024-09-17T04:28:28.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.038+0000 7fcb3ba00640 1 --2- 172.21.15.100:0/413439587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb340ad0e0 0x7fcb340ad4f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:28.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.038+0000 7fcb3ba00640 1 --2- 172.21.15.100:0/413439587 >> v2:172.21.15.184:3300/0 conn(0x7fcb3415d760 0x7fcb341535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:28.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.038+0000 7fcb3ba00640 1 -- 172.21.15.100:0/413439587 >> 172.21.15.100:0/413439587 conn(0x7fcb3406d730 msgr2=0x7fcb340b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:28.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.038+0000 7fcb3ba00640 1 -- 172.21.15.100:0/413439587 shutdown_connections 2024-09-17T04:28:28.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.038+0000 7fcb3ba00640 1 -- 172.21.15.100:0/413439587 wait complete. 2024-09-17T04:28:28.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.040+0000 7fcb3ba00640 1 Processor -- start 2024-09-17T04:28:28.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.040+0000 7fcb3ba00640 1 -- start start 2024-09-17T04:28:28.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.042+0000 7fcb3ba00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb340ad0e0 0x7fcb34155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:28.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.042+0000 7fcb3ba00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb3415d760 0x7fcb3414b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:28.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.042+0000 7fcb3ba00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb34160310 0x7fcb341f69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:28.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.042+0000 7fcb3ba00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3406f220 con 0x7fcb3415d760 2024-09-17T04:28:28.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.042+0000 7fcb3ba00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3406f0a0 con 0x7fcb34160310 2024-09-17T04:28:28.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.042+0000 7fcb3ba00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcb340b4c20 con 0x7fcb340ad0e0 2024-09-17T04:28:28.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.042+0000 7fcb33e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb3415d760 0x7fcb3414b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:28.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.042+0000 7fcb33e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb3415d760 0x7fcb3414b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57824/0 (socket says 172.21.15.100:57824) 2024-09-17T04:28:28.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.042+0000 7fcb33e00640 1 -- 172.21.15.100:0/3934417314 learned_addr learned my addr 172.21.15.100:0/3934417314 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:28.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.042+0000 7fcb39000640 1 --2- 172.21.15.100:0/3934417314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb340ad0e0 0x7fcb34155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:28.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.042+0000 7fcb39a00640 1 --2- 172.21.15.100:0/3934417314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb34160310 0x7fcb341f69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:28.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.042+0000 7fcb33e00640 1 -- 172.21.15.100:0/3934417314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb340ad0e0 msgr2=0x7fcb34155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:28.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.042+0000 7fcb33e00640 1 --2- 172.21.15.100:0/3934417314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb340ad0e0 0x7fcb34155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:28.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.042+0000 7fcb33e00640 1 -- 172.21.15.100:0/3934417314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb34160310 msgr2=0x7fcb341f69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:28.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.042+0000 7fcb33e00640 1 --2- 172.21.15.100:0/3934417314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb34160310 0x7fcb341f69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:28.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.043+0000 7fcb33e00640 1 -- 172.21.15.100:0/3934417314 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb34152b30 con 0x7fcb3415d760 2024-09-17T04:28:28.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.043+0000 7fcb39a00640 1 --2- 172.21.15.100:0/3934417314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb34160310 0x7fcb341f69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:28:28.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.043+0000 7fcb33e00640 1 --2- 172.21.15.100:0/3934417314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb3415d760 0x7fcb3414b720 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fcb20009950 tx=0x7fcb20003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:28.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.043+0000 7fcb30c00640 1 -- 172.21.15.100:0/3934417314 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2003f030 con 0x7fcb3415d760 2024-09-17T04:28:28.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.043+0000 7fcb30c00640 1 -- 172.21.15.100:0/3934417314 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb20004510 con 0x7fcb3415d760 2024-09-17T04:28:28.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.043+0000 7fcb30c00640 1 -- 172.21.15.100:0/3934417314 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb20037040 con 0x7fcb3415d760 2024-09-17T04:28:28.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.043+0000 7fcb3ba00640 1 -- 172.21.15.100:0/3934417314 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb34154960 con 0x7fcb3415d760 2024-09-17T04:28:28.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.043+0000 7fcb3ba00640 1 -- 172.21.15.100:0/3934417314 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb34154f20 con 0x7fcb3415d760 2024-09-17T04:28:28.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.044+0000 7fcb3ba00640 1 -- 172.21.15.100:0/3934417314 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb34160fa0 con 0x7fcb3415d760 2024-09-17T04:28:28.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.045+0000 7fcb30c00640 1 -- 172.21.15.100:0/3934417314 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fcb20042070 con 0x7fcb3415d760 2024-09-17T04:28:28.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.046+0000 7fcb30c00640 1 --2- 172.21.15.100:0/3934417314 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fcb14078f20 0x7fcb1407b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:28.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.047+0000 7fcb30c00640 1 -- 172.21.15.100:0/3934417314 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4943+0+0 (secure 0 0 0) 0x7fcb200bac40 con 0x7fcb3415d760 2024-09-17T04:28:28.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.047+0000 7fcb39000640 1 --2- 172.21.15.100:0/3934417314 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fcb14078f20 0x7fcb1407b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:28.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.048+0000 7fcb39000640 1 --2- 172.21.15.100:0/3934417314 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fcb14078f20 0x7fcb1407b430 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009b30 tx=0x7fcb28005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:28.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.051+0000 7fcb30c00640 1 -- 172.21.15.100:0/3934417314 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fcb20049070 con 0x7fcb3415d760 2024-09-17T04:28:28.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.304+0000 7fcb3ba00640 1 -- 172.21.15.100:0/3934417314 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7fcb341538e0 con 0x7fcb3415d760 2024-09-17T04:28:28.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.321+0000 7fcb30c00640 1 -- 172.21.15.100:0/3934417314 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v242) ==== 161+0+0 (secure 0 0 0) 0x7fcb20082eb0 con 0x7fcb3415d760 2024-09-17T04:28:28.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.324+0000 7fcb3ba00640 1 -- 172.21.15.100:0/3934417314 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fcb14078f20 msgr2=0x7fcb1407b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:28.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.324+0000 7fcb3ba00640 1 --2- 172.21.15.100:0/3934417314 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fcb14078f20 0x7fcb1407b430 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009b30 tx=0x7fcb28005f90 comp rx=0 tx=0).stop 2024-09-17T04:28:28.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.324+0000 7fcb3ba00640 1 -- 172.21.15.100:0/3934417314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb3415d760 msgr2=0x7fcb3414b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:28.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.324+0000 7fcb3ba00640 1 --2- 172.21.15.100:0/3934417314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb3415d760 0x7fcb3414b720 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fcb20009950 tx=0x7fcb20003b60 comp rx=0 tx=0).stop 2024-09-17T04:28:28.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.324+0000 7fcb3ba00640 1 -- 172.21.15.100:0/3934417314 shutdown_connections 2024-09-17T04:28:28.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.324+0000 7fcb3ba00640 1 --2- 172.21.15.100:0/3934417314 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fcb14078f20 0x7fcb1407b430 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:28.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.324+0000 7fcb3ba00640 1 --2- 172.21.15.100:0/3934417314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb34160310 0x7fcb341f69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:28.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.324+0000 7fcb3ba00640 1 --2- 172.21.15.100:0/3934417314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb3415d760 0x7fcb3414b720 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:28.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.324+0000 7fcb3ba00640 1 --2- 172.21.15.100:0/3934417314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb340ad0e0 0x7fcb34155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:28.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.324+0000 7fcb3ba00640 1 -- 172.21.15.100:0/3934417314 >> 172.21.15.100:0/3934417314 conn(0x7fcb3406d730 msgr2=0x7fcb340b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:28.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.324+0000 7fcb3ba00640 1 -- 172.21.15.100:0/3934417314 shutdown_connections 2024-09-17T04:28:28.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.324+0000 7fcb3ba00640 1 -- 172.21.15.100:0/3934417314 wait complete. 2024-09-17T04:28:28.346 INFO:tasks.cephfs_test_runner:Validate handcrafted .meta file on legacy subvol root doesn't break the system ... ok 2024-09-17T04:28:28.346 INFO:tasks.cephfs_test_runner:Starting test: test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) 2024-09-17T04:28:28.346 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:28:28.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.567+0000 7ffb6b600640 1 -- 172.21.15.100:0/470333328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb6415d760 msgr2=0x7ffb641535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:28.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.567+0000 7ffb6b600640 1 --2- 172.21.15.100:0/470333328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb6415d760 0x7ffb641535c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ffb54005fd0 tx=0x7ffb5402a540 comp rx=0 tx=0).stop 2024-09-17T04:28:28.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.567+0000 7ffb6b600640 1 -- 172.21.15.100:0/470333328 shutdown_connections 2024-09-17T04:28:28.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.567+0000 7ffb6b600640 1 --2- 172.21.15.100:0/470333328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb6415d760 0x7ffb641535c0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:28.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.567+0000 7ffb6b600640 1 -- 172.21.15.100:0/470333328 >> 172.21.15.100:0/470333328 conn(0x7ffb6406d730 msgr2=0x7ffb640b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:28.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.567+0000 7ffb6b600640 1 -- 172.21.15.100:0/470333328 shutdown_connections 2024-09-17T04:28:28.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.567+0000 7ffb6b600640 1 -- 172.21.15.100:0/470333328 wait complete. 2024-09-17T04:28:28.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.569+0000 7ffb6b600640 1 Processor -- start 2024-09-17T04:28:28.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.569+0000 7ffb6b600640 1 -- start start 2024-09-17T04:28:28.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.571+0000 7ffb6b600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb640ad0e0 0x7ffb6413ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:28.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.571+0000 7ffb6b600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb640b3550 0x7ffb6415e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:28.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.571+0000 7ffb6b600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb6415d760 0x7ffb6413f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:28.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.571+0000 7ffb6b600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6406f240 con 0x7ffb640ad0e0 2024-09-17T04:28:28.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.571+0000 7ffb6b600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffb640b4e30 con 0x7ffb640b3550 2024-09-17T04:28:28.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.571+0000 7ffb6b600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ffb640b4070 con 0x7ffb6415d760 2024-09-17T04:28:28.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.571+0000 7ffb68c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb640ad0e0 0x7ffb6413ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:28.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.571+0000 7ffb68c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb640ad0e0 0x7ffb6413ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57848/0 (socket says 172.21.15.100:57848) 2024-09-17T04:28:28.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.571+0000 7ffb68c00640 1 -- 172.21.15.100:0/615144603 learned_addr learned my addr 172.21.15.100:0/615144603 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:28.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.571+0000 7ffb5be00640 1 --2- 172.21.15.100:0/615144603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb640b3550 0x7ffb6415e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:28.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.572+0000 7ffb68c00640 1 -- 172.21.15.100:0/615144603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb6415d760 msgr2=0x7ffb6413f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:28.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.572+0000 7ffb68c00640 1 --2- 172.21.15.100:0/615144603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb6415d760 0x7ffb6413f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:28.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.572+0000 7ffb68c00640 1 -- 172.21.15.100:0/615144603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb640b3550 msgr2=0x7ffb6415e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:28.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.572+0000 7ffb68c00640 1 --2- 172.21.15.100:0/615144603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb640b3550 0x7ffb6415e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:28.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.572+0000 7ffb68c00640 1 -- 172.21.15.100:0/615144603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb641f3b50 con 0x7ffb640ad0e0 2024-09-17T04:28:28.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.572+0000 7ffb5be00640 1 --2- 172.21.15.100:0/615144603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb640b3550 0x7ffb6415e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:28:28.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.572+0000 7ffb68c00640 1 --2- 172.21.15.100:0/615144603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb640ad0e0 0x7ffb6413ffe0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ffb54034040 tx=0x7ffb540039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:28.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.572+0000 7ffb61600640 1 -- 172.21.15.100:0/615144603 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5403e030 con 0x7ffb640ad0e0 2024-09-17T04:28:28.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.572+0000 7ffb61600640 1 -- 172.21.15.100:0/615144603 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb540047a0 con 0x7ffb640ad0e0 2024-09-17T04:28:28.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.572+0000 7ffb6b600640 1 -- 172.21.15.100:0/615144603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb641f4a20 con 0x7ffb640ad0e0 2024-09-17T04:28:28.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.572+0000 7ffb61600640 1 -- 172.21.15.100:0/615144603 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb54035040 con 0x7ffb640ad0e0 2024-09-17T04:28:28.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.572+0000 7ffb6b600640 1 -- 172.21.15.100:0/615144603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb6413c450 con 0x7ffb640ad0e0 2024-09-17T04:28:28.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.573+0000 7ffb6b600640 1 -- 172.21.15.100:0/615144603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb641535c0 con 0x7ffb640ad0e0 2024-09-17T04:28:28.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.575+0000 7ffb61600640 1 -- 172.21.15.100:0/615144603 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7ffb54041030 con 0x7ffb640ad0e0 2024-09-17T04:28:28.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.576+0000 7ffb61600640 1 --2- 172.21.15.100:0/615144603 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ffb40078ef0 0x7ffb4007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:28.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.576+0000 7ffb61600640 1 -- 172.21.15.100:0/615144603 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4943+0+0 (secure 0 0 0) 0x7ffb540bb990 con 0x7ffb640ad0e0 2024-09-17T04:28:28.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.577+0000 7ffb5be00640 1 --2- 172.21.15.100:0/615144603 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ffb40078ef0 0x7ffb4007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:28.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.577+0000 7ffb5be00640 1 --2- 172.21.15.100:0/615144603 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ffb40078ef0 0x7ffb4007b400 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ffb640cfdf0 tx=0x7ffb4c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:28.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.581+0000 7ffb61600640 1 -- 172.21.15.100:0/615144603 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ffb641f3b50 con 0x7ffb640ad0e0 2024-09-17T04:28:28.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.763+0000 7ffb6b600640 1 -- 172.21.15.100:0/615144603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffb640aff60 con 0x7ffb640ad0e0 2024-09-17T04:28:28.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.764+0000 7ffb61600640 1 -- 172.21.15.100:0/615144603 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) ==== 74+0+13080 (secure 0 0 0) 0x7ffb54083c00 con 0x7ffb640ad0e0 2024-09-17T04:28:28.764 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:28:28.764 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":64,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:28:27.313622+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6835/270906112":"2024-09-18T04:28:24.299989+0000","172.21.15.100:6833/270906112":"2024-09-18T04:28:24.299989+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:28:28.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.766+0000 7ffb6b600640 1 -- 172.21.15.100:0/615144603 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ffb40078ef0 msgr2=0x7ffb4007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:28.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.766+0000 7ffb6b600640 1 --2- 172.21.15.100:0/615144603 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ffb40078ef0 0x7ffb4007b400 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ffb640cfdf0 tx=0x7ffb4c009040 comp rx=0 tx=0).stop 2024-09-17T04:28:28.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.766+0000 7ffb6b600640 1 -- 172.21.15.100:0/615144603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb640ad0e0 msgr2=0x7ffb6413ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:28.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.766+0000 7ffb6b600640 1 --2- 172.21.15.100:0/615144603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb640ad0e0 0x7ffb6413ffe0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ffb54034040 tx=0x7ffb540039e0 comp rx=0 tx=0).stop 2024-09-17T04:28:28.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.767+0000 7ffb6b600640 1 -- 172.21.15.100:0/615144603 shutdown_connections 2024-09-17T04:28:28.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.767+0000 7ffb6b600640 1 --2- 172.21.15.100:0/615144603 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ffb40078ef0 0x7ffb4007b400 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:28.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.767+0000 7ffb6b600640 1 --2- 172.21.15.100:0/615144603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb6415d760 0x7ffb6413f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:28.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.767+0000 7ffb6b600640 1 --2- 172.21.15.100:0/615144603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb640b3550 0x7ffb6415e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:28.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.767+0000 7ffb6b600640 1 --2- 172.21.15.100:0/615144603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb640ad0e0 0x7ffb6413ffe0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:28.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.767+0000 7ffb6b600640 1 -- 172.21.15.100:0/615144603 >> 172.21.15.100:0/615144603 conn(0x7ffb6406d730 msgr2=0x7ffb640b1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:28.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.767+0000 7ffb6b600640 1 -- 172.21.15.100:0/615144603 shutdown_connections 2024-09-17T04:28:28.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:28.767+0000 7ffb6b600640 1 -- 172.21.15.100:0/615144603 wait complete. 2024-09-17T04:28:28.789 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T04:24:03.995355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T04:28:28.789 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T04:28:29.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.013+0000 7f724e400640 1 -- 172.21.15.100:0/2700519747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72480d0ac0 msgr2=0x7f72480b4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:29.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.013+0000 7f724e400640 1 --2- 172.21.15.100:0/2700519747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72480d0ac0 0x7f72480b4190 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7234009830 tx=0x7f723402a5c0 comp rx=0 tx=0).stop 2024-09-17T04:28:29.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.013+0000 7f724e400640 1 -- 172.21.15.100:0/2700519747 shutdown_connections 2024-09-17T04:28:29.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.013+0000 7f724e400640 1 --2- 172.21.15.100:0/2700519747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72480d0ac0 0x7f72480b4190 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:29.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.013+0000 7f724e400640 1 -- 172.21.15.100:0/2700519747 >> 172.21.15.100:0/2700519747 conn(0x7f724806cf80 msgr2=0x7f724806d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:29.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.013+0000 7f724e400640 1 -- 172.21.15.100:0/2700519747 shutdown_connections 2024-09-17T04:28:29.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.013+0000 7f724e400640 1 -- 172.21.15.100:0/2700519747 wait complete. 2024-09-17T04:28:29.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.015+0000 7f724e400640 1 Processor -- start 2024-09-17T04:28:29.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.016+0000 7f724e400640 1 -- start start 2024-09-17T04:28:29.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.017+0000 7f724e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7248155970 0x7f72480ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:29.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.017+0000 7f724e400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f724815f090 0x7f724815cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:29.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.017+0000 7f724e400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f724815d2c0 0x7f7248158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:29.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.017+0000 7f724e400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7248074960 con 0x7f7248155970 2024-09-17T04:28:29.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.017+0000 7f724e400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f72480747e0 con 0x7f724815d2c0 2024-09-17T04:28:29.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.017+0000 7f724e400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7248074ae0 con 0x7f724815f090 2024-09-17T04:28:29.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.017+0000 7f724d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7248155970 0x7f72480ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:29.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.017+0000 7f724d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7248155970 0x7f72480ce7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57860/0 (socket says 172.21.15.100:57860) 2024-09-17T04:28:29.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.017+0000 7f724d000640 1 -- 172.21.15.100:0/3709689613 learned_addr learned my addr 172.21.15.100:0/3709689613 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:29.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.018+0000 7f7247e00640 1 --2- 172.21.15.100:0/3709689613 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f724815f090 0x7f724815cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:29.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.017+0000 7f724da00640 1 --2- 172.21.15.100:0/3709689613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f724815d2c0 0x7f7248158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:29.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.018+0000 7f724d000640 1 -- 172.21.15.100:0/3709689613 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f724815f090 msgr2=0x7f724815cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:29.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.018+0000 7f724d000640 1 --2- 172.21.15.100:0/3709689613 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f724815f090 0x7f724815cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:29.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.018+0000 7f724d000640 1 -- 172.21.15.100:0/3709689613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f724815d2c0 msgr2=0x7f7248158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:29.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.018+0000 7f724d000640 1 --2- 172.21.15.100:0/3709689613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f724815d2c0 0x7f7248158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:29.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.018+0000 7f724d000640 1 -- 172.21.15.100:0/3709689613 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72481f5120 con 0x7f7248155970 2024-09-17T04:28:29.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.018+0000 7f724da00640 1 --2- 172.21.15.100:0/3709689613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f724815d2c0 0x7f7248158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:28:29.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.018+0000 7f724d000640 1 --2- 172.21.15.100:0/3709689613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7248155970 0x7f72480ce7f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f723c00c400 tx=0x7f723c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:29.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.018+0000 7f7244c00640 1 -- 172.21.15.100:0/3709689613 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c017070 con 0x7f7248155970 2024-09-17T04:28:29.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.018+0000 7f7244c00640 1 -- 172.21.15.100:0/3709689613 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f723c0053c0 con 0x7f7248155970 2024-09-17T04:28:29.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.018+0000 7f7244c00640 1 -- 172.21.15.100:0/3709689613 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c003850 con 0x7f7248155970 2024-09-17T04:28:29.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.018+0000 7f724e400640 1 -- 172.21.15.100:0/3709689613 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72481f5400 con 0x7f7248155970 2024-09-17T04:28:29.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.018+0000 7f724e400640 1 -- 172.21.15.100:0/3709689613 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72481f59c0 con 0x7f7248155970 2024-09-17T04:28:29.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.019+0000 7f724e400640 1 -- 172.21.15.100:0/3709689613 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7248074f90 con 0x7f7248155970 2024-09-17T04:28:29.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.021+0000 7f7244c00640 1 -- 172.21.15.100:0/3709689613 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f723c01a070 con 0x7f7248155970 2024-09-17T04:28:29.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.022+0000 7f7244c00640 1 --2- 172.21.15.100:0/3709689613 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7228078ef0 0x7f722807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:29.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.022+0000 7f7244c00640 1 -- 172.21.15.100:0/3709689613 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4943+0+0 (secure 0 0 0) 0x7f723c09a4b0 con 0x7f7248155970 2024-09-17T04:28:29.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.022+0000 7f7247e00640 1 --2- 172.21.15.100:0/3709689613 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7228078ef0 0x7f722807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:29.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.023+0000 7f7247e00640 1 --2- 172.21.15.100:0/3709689613 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7228078ef0 0x7f722807b400 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7234034000 tx=0x7f7234004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:29.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.027+0000 7f7244c00640 1 -- 172.21.15.100:0/3709689613 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f723c0626b0 con 0x7f7248155970 2024-09-17T04:28:29.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.209+0000 7f724e400640 1 -- 172.21.15.100:0/3709689613 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f724815f540 con 0x7f7248155970 2024-09-17T04:28:29.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.210+0000 7f7244c00640 1 -- 172.21.15.100:0/3709689613 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) ==== 93+0+10 (secure 0 0 0) 0x7f723c061fb0 con 0x7f7248155970 2024-09-17T04:28:29.209 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-09-17T04:28:29.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.212+0000 7f724e400640 1 -- 172.21.15.100:0/3709689613 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7228078ef0 msgr2=0x7f722807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:29.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.212+0000 7f724e400640 1 --2- 172.21.15.100:0/3709689613 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7228078ef0 0x7f722807b400 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7234034000 tx=0x7f7234004730 comp rx=0 tx=0).stop 2024-09-17T04:28:29.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.212+0000 7f724e400640 1 -- 172.21.15.100:0/3709689613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7248155970 msgr2=0x7f72480ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:29.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.212+0000 7f724e400640 1 --2- 172.21.15.100:0/3709689613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7248155970 0x7f72480ce7f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f723c00c400 tx=0x7f723c00c8c0 comp rx=0 tx=0).stop 2024-09-17T04:28:29.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.212+0000 7f724e400640 1 -- 172.21.15.100:0/3709689613 shutdown_connections 2024-09-17T04:28:29.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.212+0000 7f724e400640 1 --2- 172.21.15.100:0/3709689613 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7228078ef0 0x7f722807b400 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:29.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.212+0000 7f724e400640 1 --2- 172.21.15.100:0/3709689613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f724815d2c0 0x7f7248158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:29.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.212+0000 7f724e400640 1 --2- 172.21.15.100:0/3709689613 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f724815f090 0x7f724815cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:29.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.212+0000 7f724e400640 1 --2- 172.21.15.100:0/3709689613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7248155970 0x7f72480ce7f0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:29.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.212+0000 7f724e400640 1 -- 172.21.15.100:0/3709689613 >> 172.21.15.100:0/3709689613 conn(0x7f724806cf80 msgr2=0x7f7248154ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:29.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.212+0000 7f724e400640 1 -- 172.21.15.100:0/3709689613 shutdown_connections 2024-09-17T04:28:29.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.212+0000 7f724e400640 1 -- 172.21.15.100:0/3709689613 wait complete. 2024-09-17T04:28:29.234 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2024-09-17T04:28:29.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.455+0000 7fa9ef200640 1 -- 172.21.15.100:0/3058821798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9e80aebc0 msgr2=0x7fa9e8152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:29.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.455+0000 7fa9ef200640 1 --2- 172.21.15.100:0/3058821798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9e80aebc0 0x7fa9e8152920 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8005fd0 tx=0x7fa9d802a560 comp rx=0 tx=0).stop 2024-09-17T04:28:29.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.455+0000 7fa9ef200640 1 -- 172.21.15.100:0/3058821798 shutdown_connections 2024-09-17T04:28:29.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.455+0000 7fa9ef200640 1 --2- 172.21.15.100:0/3058821798 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9e80b2750 0x7fa9e815f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:29.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.455+0000 7fa9ef200640 1 --2- 172.21.15.100:0/3058821798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9e80aebc0 0x7fa9e8152920 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:29.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.455+0000 7fa9ef200640 1 -- 172.21.15.100:0/3058821798 >> 172.21.15.100:0/3058821798 conn(0x7fa9e806d730 msgr2=0x7fa9e80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:29.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.455+0000 7fa9ef200640 1 -- 172.21.15.100:0/3058821798 shutdown_connections 2024-09-17T04:28:29.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.455+0000 7fa9ef200640 1 -- 172.21.15.100:0/3058821798 wait complete. 2024-09-17T04:28:29.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.457+0000 7fa9ef200640 1 Processor -- start 2024-09-17T04:28:29.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.458+0000 7fa9ef200640 1 -- start start 2024-09-17T04:28:29.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.459+0000 7fa9ef200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9e80aebc0 0x7fa9e81f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:29.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.459+0000 7fa9ef200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9e80b2750 0x7fa9e81546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:29.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.459+0000 7fa9ef200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9e815f910 0x7fa9e8151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:29.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.459+0000 7fa9ef200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e815c4a0 con 0x7fa9e80b2750 2024-09-17T04:28:29.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.459+0000 7fa9ef200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e80b4a20 con 0x7fa9e815f910 2024-09-17T04:28:29.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.459+0000 7fa9ef200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e815c620 con 0x7fa9e80aebc0 2024-09-17T04:28:29.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.459+0000 7fa9dfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9e80b2750 0x7fa9e81546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:29.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.459+0000 7fa9dfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9e80b2750 0x7fa9e81546f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57876/0 (socket says 172.21.15.100:57876) 2024-09-17T04:28:29.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.459+0000 7fa9dfe00640 1 -- 172.21.15.100:0/1463185655 learned_addr learned my addr 172.21.15.100:0/1463185655 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:29.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.459+0000 7fa9e7e00640 1 --2- 172.21.15.100:0/1463185655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9e80aebc0 0x7fa9e81f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:29.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.460+0000 7fa9ed200640 1 --2- 172.21.15.100:0/1463185655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9e815f910 0x7fa9e8151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:29.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.460+0000 7fa9dfe00640 1 -- 172.21.15.100:0/1463185655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9e80aebc0 msgr2=0x7fa9e81f7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:29.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.460+0000 7fa9dfe00640 1 --2- 172.21.15.100:0/1463185655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9e80aebc0 0x7fa9e81f7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:29.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.460+0000 7fa9dfe00640 1 -- 172.21.15.100:0/1463185655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9e815f910 msgr2=0x7fa9e8151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:29.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.460+0000 7fa9dfe00640 1 --2- 172.21.15.100:0/1463185655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9e815f910 0x7fa9e8151810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:29.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.460+0000 7fa9dfe00640 1 -- 172.21.15.100:0/1463185655 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e8154c30 con 0x7fa9e80b2750 2024-09-17T04:28:29.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.460+0000 7fa9dfe00640 1 --2- 172.21.15.100:0/1463185655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9e80b2750 0x7fa9e81546f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000b3e0 tx=0x7fa9d000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:29.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.460+0000 7fa9e4c00640 1 -- 172.21.15.100:0/1463185655 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0015020 con 0x7fa9e80b2750 2024-09-17T04:28:29.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.460+0000 7fa9e4c00640 1 -- 172.21.15.100:0/1463185655 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9d0011940 con 0x7fa9e80b2750 2024-09-17T04:28:29.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.460+0000 7fa9e4c00640 1 -- 172.21.15.100:0/1463185655 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0011d90 con 0x7fa9e80b2750 2024-09-17T04:28:29.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.460+0000 7fa9ef200640 1 -- 172.21.15.100:0/1463185655 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e80cf890 con 0x7fa9e80b2750 2024-09-17T04:28:29.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.460+0000 7fa9ef200640 1 -- 172.21.15.100:0/1463185655 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e81f6d50 con 0x7fa9e80b2750 2024-09-17T04:28:29.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.461+0000 7fa9ef200640 1 -- 172.21.15.100:0/1463185655 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9e80cf600 con 0x7fa9e80b2750 2024-09-17T04:28:29.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.463+0000 7fa9e4c00640 1 -- 172.21.15.100:0/1463185655 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fa9d0018020 con 0x7fa9e80b2750 2024-09-17T04:28:29.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.464+0000 7fa9e4c00640 1 --2- 172.21.15.100:0/1463185655 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa9c4078f20 0x7fa9c407b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:29.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.464+0000 7fa9e4c00640 1 -- 172.21.15.100:0/1463185655 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4943+0+0 (secure 0 0 0) 0x7fa9d009afb0 con 0x7fa9e80b2750 2024-09-17T04:28:29.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.464+0000 7fa9e7e00640 1 --2- 172.21.15.100:0/1463185655 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa9c4078f20 0x7fa9c407b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:29.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.465+0000 7fa9e7e00640 1 --2- 172.21.15.100:0/1463185655 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa9c4078f20 0x7fa9c407b430 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8033000 tx=0x7fa9d8004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:29.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.468+0000 7fa9e4c00640 1 -- 172.21.15.100:0/1463185655 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa9d001f020 con 0x7fa9e80b2750 2024-09-17T04:28:29.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:29.717+0000 7fa9ef200640 1 -- 172.21.15.100:0/1463185655 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) -- 0x7fa9e815cbc0 con 0x7fa9e80b2750 2024-09-17T04:28:30.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.331+0000 7fa9e4c00640 1 -- 172.21.15.100:0/1463185655 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v244) ==== 134+0+0 (secure 0 0 0) 0x7fa9d0063190 con 0x7fa9e80b2750 2024-09-17T04:28:30.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.334+0000 7fa9ef200640 1 -- 172.21.15.100:0/1463185655 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa9c4078f20 msgr2=0x7fa9c407b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:30.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.334+0000 7fa9ef200640 1 --2- 172.21.15.100:0/1463185655 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa9c4078f20 0x7fa9c407b430 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8033000 tx=0x7fa9d8004730 comp rx=0 tx=0).stop 2024-09-17T04:28:30.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.334+0000 7fa9ef200640 1 -- 172.21.15.100:0/1463185655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9e80b2750 msgr2=0x7fa9e81546f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:30.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.334+0000 7fa9ef200640 1 --2- 172.21.15.100:0/1463185655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9e80b2750 0x7fa9e81546f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000b3e0 tx=0x7fa9d000b8a0 comp rx=0 tx=0).stop 2024-09-17T04:28:30.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.334+0000 7fa9ef200640 1 -- 172.21.15.100:0/1463185655 shutdown_connections 2024-09-17T04:28:30.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.334+0000 7fa9ef200640 1 --2- 172.21.15.100:0/1463185655 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa9c4078f20 0x7fa9c407b430 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:30.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.334+0000 7fa9ef200640 1 --2- 172.21.15.100:0/1463185655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9e815f910 0x7fa9e8151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:30.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.334+0000 7fa9ef200640 1 --2- 172.21.15.100:0/1463185655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9e80b2750 0x7fa9e81546f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:30.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.334+0000 7fa9ef200640 1 --2- 172.21.15.100:0/1463185655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9e80aebc0 0x7fa9e81f7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:30.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.334+0000 7fa9ef200640 1 -- 172.21.15.100:0/1463185655 >> 172.21.15.100:0/1463185655 conn(0x7fa9e806d730 msgr2=0x7fa9e815cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:30.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.334+0000 7fa9ef200640 1 -- 172.21.15.100:0/1463185655 shutdown_connections 2024-09-17T04:28:30.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.334+0000 7fa9ef200640 1 -- 172.21.15.100:0/1463185655 wait complete. 2024-09-17T04:28:30.356 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T04:28:30.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.577+0000 7f46c2600640 1 -- 172.21.15.100:0/846663531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46bc0b2750 msgr2=0x7f46bc15f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:30.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.577+0000 7f46c2600640 1 --2- 172.21.15.100:0/846663531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46bc0b2750 0x7f46bc15f3d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f46bc072810 tx=0x7f46a802a560 comp rx=0 tx=0).stop 2024-09-17T04:28:30.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.578+0000 7f46c2600640 1 -- 172.21.15.100:0/846663531 shutdown_connections 2024-09-17T04:28:30.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.578+0000 7f46c2600640 1 --2- 172.21.15.100:0/846663531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46bc0b2750 0x7f46bc15f3d0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:30.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.578+0000 7f46c2600640 1 --2- 172.21.15.100:0/846663531 >> v2:172.21.15.184:3300/0 conn(0x7f46bc0aebc0 0x7f46bc152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:30.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.578+0000 7f46c2600640 1 -- 172.21.15.100:0/846663531 >> 172.21.15.100:0/846663531 conn(0x7f46bc06d730 msgr2=0x7f46bc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:30.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.578+0000 7f46c2600640 1 -- 172.21.15.100:0/846663531 shutdown_connections 2024-09-17T04:28:30.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.578+0000 7f46c2600640 1 -- 172.21.15.100:0/846663531 wait complete. 2024-09-17T04:28:30.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.580+0000 7f46c2600640 1 Processor -- start 2024-09-17T04:28:30.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.580+0000 7f46c2600640 1 -- start start 2024-09-17T04:28:30.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.582+0000 7f46c2600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46bc0aebc0 0x7f46bc14dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:30.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.582+0000 7f46c2600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46bc0b2750 0x7f46bc14c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:30.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.582+0000 7f46c2600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46bc15f910 0x7f46bc1e9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:30.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.582+0000 7f46c2600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc15c4a0 con 0x7f46bc0b2750 2024-09-17T04:28:30.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.582+0000 7f46c2600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc0b4970 con 0x7f46bc15f910 2024-09-17T04:28:30.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.582+0000 7f46c2600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f46bc15c620 con 0x7f46bc0aebc0 2024-09-17T04:28:30.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.582+0000 7f46baa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46bc0b2750 0x7f46bc14c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:30.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.582+0000 7f46baa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46bc0b2750 0x7f46bc14c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57890/0 (socket says 172.21.15.100:57890) 2024-09-17T04:28:30.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.582+0000 7f46baa00640 1 -- 172.21.15.100:0/3909238347 learned_addr learned my addr 172.21.15.100:0/3909238347 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:30.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.582+0000 7f46bb400640 1 --2- 172.21.15.100:0/3909238347 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46bc0aebc0 0x7f46bc14dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:30.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.582+0000 7f46bbe00640 1 --2- 172.21.15.100:0/3909238347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46bc15f910 0x7f46bc1e9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:30.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.582+0000 7f46baa00640 1 -- 172.21.15.100:0/3909238347 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46bc0aebc0 msgr2=0x7f46bc14dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:30.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.582+0000 7f46baa00640 1 --2- 172.21.15.100:0/3909238347 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46bc0aebc0 0x7f46bc14dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:30.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.582+0000 7f46baa00640 1 -- 172.21.15.100:0/3909238347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46bc15f910 msgr2=0x7f46bc1e9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:30.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.582+0000 7f46baa00640 1 --2- 172.21.15.100:0/3909238347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46bc15f910 0x7f46bc1e9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:30.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.582+0000 7f46baa00640 1 -- 172.21.15.100:0/3909238347 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46bc0737d0 con 0x7f46bc0b2750 2024-09-17T04:28:30.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.582+0000 7f46bbe00640 1 --2- 172.21.15.100:0/3909238347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46bc15f910 0x7f46bc1e9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:28:30.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.583+0000 7f46baa00640 1 --2- 172.21.15.100:0/3909238347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46bc0b2750 0x7f46bc14c140 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f46a8034040 tx=0x7f46a8003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:30.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.583+0000 7f46b3400640 1 -- 172.21.15.100:0/3909238347 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a803e030 con 0x7f46bc0b2750 2024-09-17T04:28:30.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.583+0000 7f46b3400640 1 -- 172.21.15.100:0/3909238347 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46a8003d10 con 0x7f46bc0b2750 2024-09-17T04:28:30.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.583+0000 7f46c2600640 1 -- 172.21.15.100:0/3909238347 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46bc150040 con 0x7f46bc0b2750 2024-09-17T04:28:30.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.583+0000 7f46b3400640 1 -- 172.21.15.100:0/3909238347 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a8035040 con 0x7f46bc0b2750 2024-09-17T04:28:30.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.583+0000 7f46c2600640 1 -- 172.21.15.100:0/3909238347 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46bc14fd60 con 0x7f46bc0b2750 2024-09-17T04:28:30.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.584+0000 7f46c2600640 1 -- 172.21.15.100:0/3909238347 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46bc14e380 con 0x7f46bc0b2750 2024-09-17T04:28:30.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.586+0000 7f46b3400640 1 -- 172.21.15.100:0/3909238347 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f46a8041030 con 0x7f46bc0b2750 2024-09-17T04:28:30.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.587+0000 7f46b3400640 1 --2- 172.21.15.100:0/3909238347 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f469c078ef0 0x7f469c07b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:30.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.587+0000 7f46b3400640 1 -- 172.21.15.100:0/3909238347 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4943+0+0 (secure 0 0 0) 0x7f46a80bb6d0 con 0x7f46bc0b2750 2024-09-17T04:28:30.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.587+0000 7f46bb400640 1 --2- 172.21.15.100:0/3909238347 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f469c078ef0 0x7f469c07b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:30.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.588+0000 7f46bb400640 1 --2- 172.21.15.100:0/3909238347 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f469c078ef0 0x7f469c07b400 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009b30 tx=0x7f46ac005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:30.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.591+0000 7f46b3400640 1 -- 172.21.15.100:0/3909238347 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f46a8083330 con 0x7f46bc0b2750 2024-09-17T04:28:30.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.772+0000 7f46c2600640 1 -- 172.21.15.100:0/3909238347 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f46bc15cad0 con 0x7f46bc0b2750 2024-09-17T04:28:30.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.772+0000 7f46b3400640 1 -- 172.21.15.100:0/3909238347 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v37) ==== 59+0+2294 (secure 0 0 0) 0x7f46a8087d60 con 0x7f46bc0b2750 2024-09-17T04:28:30.772 INFO:teuthology.orchestra.run.smithi100.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T04:28:30.772 INFO:teuthology.orchestra.run.smithi100.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T04:28:30.772 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced client_mount_timeout 30 2024-09-17T04:28:30.772 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_client 20/20 2024-09-17T04:28:30.772 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_mgr 20/20 2024-09-17T04:28:30.772 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_ms 1/1 2024-09-17T04:28:30.773 INFO:teuthology.orchestra.run.smithi100.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T04:28:30.773 INFO:teuthology.orchestra.run.smithi100.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T04:28:30.774 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3938.598654 2024-09-17T04:28:30.774 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4008.640154 2024-09-17T04:28:30.774 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3706.588546 2024-09-17T04:28:30.774 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 6299.338665 2024-09-17T04:28:30.774 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3633.876699 2024-09-17T04:28:30.774 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3674.821841 2024-09-17T04:28:30.774 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3789.849984 2024-09-17T04:28:30.774 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3467.457298 2024-09-17T04:28:30.774 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_mds 20/20 2024-09-17T04:28:30.774 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T04:28:30.774 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_ms 2/2 2024-09-17T04:28:30.774 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_debug_frag true 2024-09-17T04:28:30.774 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_debug_scatterstat true 2024-09-17T04:28:30.775 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T04:28:30.775 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_verify_scatter true 2024-09-17T04:28:30.775 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T04:28:30.775 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T04:28:30.775 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T04:28:30.775 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced client_mount_timeout 600 2024-09-17T04:28:30.775 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced debug_client 20/20 2024-09-17T04:28:30.775 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced debug_ms 1/1 2024-09-17T04:28:30.775 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T04:28:30.775 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T04:28:30.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.775+0000 7f46c2600640 1 -- 172.21.15.100:0/3909238347 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f469c078ef0 msgr2=0x7f469c07b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:30.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.775+0000 7f46c2600640 1 --2- 172.21.15.100:0/3909238347 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f469c078ef0 0x7f469c07b400 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009b30 tx=0x7f46ac005f90 comp rx=0 tx=0).stop 2024-09-17T04:28:30.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.775+0000 7f46c2600640 1 -- 172.21.15.100:0/3909238347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46bc0b2750 msgr2=0x7f46bc14c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:30.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.775+0000 7f46c2600640 1 --2- 172.21.15.100:0/3909238347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46bc0b2750 0x7f46bc14c140 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f46a8034040 tx=0x7f46a8003820 comp rx=0 tx=0).stop 2024-09-17T04:28:30.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.775+0000 7f46c2600640 1 -- 172.21.15.100:0/3909238347 shutdown_connections 2024-09-17T04:28:30.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.775+0000 7f46c2600640 1 --2- 172.21.15.100:0/3909238347 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f469c078ef0 0x7f469c07b400 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:30.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.775+0000 7f46c2600640 1 --2- 172.21.15.100:0/3909238347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46bc15f910 0x7f46bc1e9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:30.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.775+0000 7f46c2600640 1 --2- 172.21.15.100:0/3909238347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46bc0b2750 0x7f46bc14c140 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:30.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.775+0000 7f46c2600640 1 --2- 172.21.15.100:0/3909238347 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46bc0aebc0 0x7f46bc14dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:30.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.775+0000 7f46c2600640 1 -- 172.21.15.100:0/3909238347 >> 172.21.15.100:0/3909238347 conn(0x7f46bc06d730 msgr2=0x7f46bc0b0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:30.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.775+0000 7f46c2600640 1 -- 172.21.15.100:0/3909238347 shutdown_connections 2024-09-17T04:28:30.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:30.775+0000 7f46c2600640 1 -- 172.21.15.100:0/3909238347 wait complete. 2024-09-17T04:28:30.798 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T04:28:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.022+0000 7ff65fe00640 1 -- 172.21.15.100:0/3822720232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff660070160 msgr2=0x7ff6600af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.022+0000 7ff65fe00640 1 --2- 172.21.15.100:0/3822720232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff660070160 0x7ff6600af0d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff658005fd0 tx=0x7ff65802a940 comp rx=0 tx=0).stop 2024-09-17T04:28:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.022+0000 7ff65fe00640 1 -- 172.21.15.100:0/3822720232 shutdown_connections 2024-09-17T04:28:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.022+0000 7ff65fe00640 1 --2- 172.21.15.100:0/3822720232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff660070160 0x7ff6600af0d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.022+0000 7ff65fe00640 1 -- 172.21.15.100:0/3822720232 >> 172.21.15.100:0/3822720232 conn(0x7ff66006d620 msgr2=0x7ff6600b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.023+0000 7ff65fe00640 1 -- 172.21.15.100:0/3822720232 shutdown_connections 2024-09-17T04:28:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.023+0000 7ff65fe00640 1 -- 172.21.15.100:0/3822720232 wait complete. 2024-09-17T04:28:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.025+0000 7ff65fe00640 1 Processor -- start 2024-09-17T04:28:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.025+0000 7ff65fe00640 1 -- start start 2024-09-17T04:28:31.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.026+0000 7ff65fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff660070160 0x7ff6601f56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:31.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.026+0000 7ff65fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6600b3f30 0x7ff66015b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:31.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.026+0000 7ff65fe00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff660156ca0 0x7ff66015bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:31.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.026+0000 7ff65fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff6600af790 con 0x7ff6600b3f30 2024-09-17T04:28:31.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.026+0000 7ff65fe00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff6600af610 con 0x7ff660070160 2024-09-17T04:28:31.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.026+0000 7ff65fe00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff6600af910 con 0x7ff660156ca0 2024-09-17T04:28:31.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.027+0000 7ff656000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6600b3f30 0x7ff66015b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:31.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.027+0000 7ff65ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff660070160 0x7ff6601f56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:31.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.027+0000 7ff656000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6600b3f30 0x7ff66015b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57902/0 (socket says 172.21.15.100:57902) 2024-09-17T04:28:31.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.027+0000 7ff656000640 1 -- 172.21.15.100:0/2636578075 learned_addr learned my addr 172.21.15.100:0/2636578075 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:31.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.027+0000 7ff65ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff660070160 0x7ff6601f56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.100:51060/0 (socket says 172.21.15.100:51060) 2024-09-17T04:28:31.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.027+0000 7ff65f400640 1 --2- 172.21.15.100:0/2636578075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff660156ca0 0x7ff66015bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:31.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.027+0000 7ff656000640 1 -- 172.21.15.100:0/2636578075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff660156ca0 msgr2=0x7ff66015bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:31.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.027+0000 7ff656000640 1 --2- 172.21.15.100:0/2636578075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff660156ca0 0x7ff66015bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:31.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.027+0000 7ff656000640 1 -- 172.21.15.100:0/2636578075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff660070160 msgr2=0x7ff6601f56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:31.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.027+0000 7ff656000640 1 --2- 172.21.15.100:0/2636578075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff660070160 0x7ff6601f56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:31.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.027+0000 7ff656000640 1 -- 172.21.15.100:0/2636578075 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6601537c0 con 0x7ff6600b3f30 2024-09-17T04:28:31.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.028+0000 7ff656000640 1 --2- 172.21.15.100:0/2636578075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6600b3f30 0x7ff66015b710 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff64800c310 tx=0x7ff64800c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:31.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.028+0000 7ff657400640 1 -- 172.21.15.100:0/2636578075 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648011790 con 0x7ff6600b3f30 2024-09-17T04:28:31.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.028+0000 7ff657400640 1 -- 172.21.15.100:0/2636578075 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff648017070 con 0x7ff6600b3f30 2024-09-17T04:28:31.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.028+0000 7ff65fe00640 1 -- 172.21.15.100:0/2636578075 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff660145230 con 0x7ff6600b3f30 2024-09-17T04:28:31.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.028+0000 7ff657400640 1 -- 172.21.15.100:0/2636578075 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648011cc0 con 0x7ff6600b3f30 2024-09-17T04:28:31.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.028+0000 7ff65fe00640 1 -- 172.21.15.100:0/2636578075 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff660074c30 con 0x7ff6600b3f30 2024-09-17T04:28:31.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.029+0000 7ff65fe00640 1 -- 172.21.15.100:0/2636578075 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff660153530 con 0x7ff6600b3f30 2024-09-17T04:28:31.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.030+0000 7ff657400640 1 -- 172.21.15.100:0/2636578075 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7ff64801c460 con 0x7ff6600b3f30 2024-09-17T04:28:31.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.031+0000 7ff657400640 1 --2- 172.21.15.100:0/2636578075 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff63c078f20 0x7ff63c07b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:31.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.032+0000 7ff657400640 1 -- 172.21.15.100:0/2636578075 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4943+0+0 (secure 0 0 0) 0x7ff64809ac60 con 0x7ff6600b3f30 2024-09-17T04:28:31.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.032+0000 7ff65ea00640 1 --2- 172.21.15.100:0/2636578075 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff63c078f20 0x7ff63c07b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:31.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.032+0000 7ff65ea00640 1 --2- 172.21.15.100:0/2636578075 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff63c078f20 0x7ff63c07b430 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff650000f60 tx=0x7ff650009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:31.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.036+0000 7ff657400640 1 -- 172.21.15.100:0/2636578075 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff6480627f0 con 0x7ff6600b3f30 2024-09-17T04:28:31.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.220+0000 7ff65fe00640 1 -- 172.21.15.100:0/2636578075 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff6600af0d0 con 0x7ff6600b3f30 2024-09-17T04:28:31.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.224+0000 7ff657400640 1 -- 172.21.15.100:0/2636578075 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v38)=0 v38) ==== 122+0+0 (secure 0 0 0) 0x7ff648067220 con 0x7ff6600b3f30 2024-09-17T04:28:31.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.227+0000 7ff65fe00640 1 -- 172.21.15.100:0/2636578075 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff63c078f20 msgr2=0x7ff63c07b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:31.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.227+0000 7ff65fe00640 1 --2- 172.21.15.100:0/2636578075 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff63c078f20 0x7ff63c07b430 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff650000f60 tx=0x7ff650009040 comp rx=0 tx=0).stop 2024-09-17T04:28:31.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.227+0000 7ff65fe00640 1 -- 172.21.15.100:0/2636578075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6600b3f30 msgr2=0x7ff66015b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:31.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.227+0000 7ff65fe00640 1 --2- 172.21.15.100:0/2636578075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6600b3f30 0x7ff66015b710 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff64800c310 tx=0x7ff64800c7d0 comp rx=0 tx=0).stop 2024-09-17T04:28:31.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.227+0000 7ff65fe00640 1 -- 172.21.15.100:0/2636578075 shutdown_connections 2024-09-17T04:28:31.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.227+0000 7ff65fe00640 1 --2- 172.21.15.100:0/2636578075 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff63c078f20 0x7ff63c07b430 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:31.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.227+0000 7ff65fe00640 1 --2- 172.21.15.100:0/2636578075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff660156ca0 0x7ff66015bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:31.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.227+0000 7ff65fe00640 1 --2- 172.21.15.100:0/2636578075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6600b3f30 0x7ff66015b710 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:31.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.227+0000 7ff65fe00640 1 --2- 172.21.15.100:0/2636578075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff660070160 0x7ff6601f56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:31.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.227+0000 7ff65fe00640 1 -- 172.21.15.100:0/2636578075 >> 172.21.15.100:0/2636578075 conn(0x7ff66006d620 msgr2=0x7ff6600adec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:31.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.227+0000 7ff65fe00640 1 -- 172.21.15.100:0/2636578075 shutdown_connections 2024-09-17T04:28:31.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.227+0000 7ff65fe00640 1 -- 172.21.15.100:0/2636578075 wait complete. 2024-09-17T04:28:31.249 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T04:28:31.249 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:28:31.335 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:28:31.441 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T04:28:31.441 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:28:31.441 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:28:31.477 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:28:31.477 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:28:31.478 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:28:31.519 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:28:31.519 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:28:31.519 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:28:31.555 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:28:31.556 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:28:31.556 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:28:31.596 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:28:31.597 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T04:28:31.597 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:28:31.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.816+0000 7f6e47e00640 1 -- 172.21.15.100:0/1199647516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e48160490 msgr2=0x7f6e480ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:31.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.816+0000 7f6e47e00640 1 --2- 172.21.15.100:0/1199647516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e48160490 0x7f6e480ccaf0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6e34009830 tx=0x7f6e3402a510 comp rx=0 tx=0).stop 2024-09-17T04:28:31.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.816+0000 7f6e47e00640 1 -- 172.21.15.100:0/1199647516 shutdown_connections 2024-09-17T04:28:31.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.816+0000 7f6e47e00640 1 --2- 172.21.15.100:0/1199647516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e48160490 0x7f6e480ccaf0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:31.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.816+0000 7f6e47e00640 1 --2- 172.21.15.100:0/1199647516 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e480d01b0 0x7f6e48156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:31.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.816+0000 7f6e47e00640 1 -- 172.21.15.100:0/1199647516 >> 172.21.15.100:0/1199647516 conn(0x7f6e4806d400 msgr2=0x7f6e480710f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:31.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.816+0000 7f6e47e00640 1 -- 172.21.15.100:0/1199647516 shutdown_connections 2024-09-17T04:28:31.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.816+0000 7f6e47e00640 1 -- 172.21.15.100:0/1199647516 wait complete. 2024-09-17T04:28:31.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.818+0000 7f6e47e00640 1 Processor -- start 2024-09-17T04:28:31.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.818+0000 7f6e47e00640 1 -- start start 2024-09-17T04:28:31.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.820+0000 7f6e47e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e480cd030 0x7f6e480c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:31.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.820+0000 7f6e47e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e480d01b0 0x7f6e48147680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:31.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.820+0000 7f6e47e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e48160490 0x7f6e480c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:31.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.820+0000 7f6e47e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6e480cf340 con 0x7f6e480cd030 2024-09-17T04:28:31.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.820+0000 7f6e47e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4815f510 con 0x7f6e480d01b0 2024-09-17T04:28:31.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.820+0000 7f6e47e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6e480cf4c0 con 0x7f6e48160490 2024-09-17T04:28:31.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.820+0000 7f6e46a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e480cd030 0x7f6e480c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:31.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.820+0000 7f6e46a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e480cd030 0x7f6e480c9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57906/0 (socket says 172.21.15.100:57906) 2024-09-17T04:28:31.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.820+0000 7f6e46a00640 1 -- 172.21.15.100:0/2869984418 learned_addr learned my addr 172.21.15.100:0/2869984418 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:31.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.820+0000 7f6e46000640 1 --2- 172.21.15.100:0/2869984418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e480d01b0 0x7f6e48147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:31.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.820+0000 7f6e47400640 1 --2- 172.21.15.100:0/2869984418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e48160490 0x7f6e480c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:31.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.820+0000 7f6e46a00640 1 -- 172.21.15.100:0/2869984418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e48160490 msgr2=0x7f6e480c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:31.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.820+0000 7f6e46a00640 1 --2- 172.21.15.100:0/2869984418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e48160490 0x7f6e480c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:31.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.820+0000 7f6e46a00640 1 -- 172.21.15.100:0/2869984418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e480d01b0 msgr2=0x7f6e48147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:31.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.820+0000 7f6e46a00640 1 --2- 172.21.15.100:0/2869984418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e480d01b0 0x7f6e48147680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:31.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.820+0000 7f6e46a00640 1 -- 172.21.15.100:0/2869984418 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e48147bc0 con 0x7f6e480cd030 2024-09-17T04:28:31.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.820+0000 7f6e46000640 1 --2- 172.21.15.100:0/2869984418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e480d01b0 0x7f6e48147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:28:31.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.821+0000 7f6e46a00640 1 --2- 172.21.15.100:0/2869984418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e480cd030 0x7f6e480c9300 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6e3800c9a0 tx=0x7f6e3800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:31.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.821+0000 7f6e3ea00640 1 -- 172.21.15.100:0/2869984418 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e38007bb0 con 0x7f6e480cd030 2024-09-17T04:28:31.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.821+0000 7f6e3ea00640 1 -- 172.21.15.100:0/2869984418 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e38007e40 con 0x7f6e480cd030 2024-09-17T04:28:31.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.821+0000 7f6e47e00640 1 -- 172.21.15.100:0/2869984418 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e481f70e0 con 0x7f6e480cd030 2024-09-17T04:28:31.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.821+0000 7f6e3ea00640 1 -- 172.21.15.100:0/2869984418 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3800f690 con 0x7f6e480cd030 2024-09-17T04:28:31.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.821+0000 7f6e47e00640 1 -- 172.21.15.100:0/2869984418 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e481f76a0 con 0x7f6e480cd030 2024-09-17T04:28:31.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.822+0000 7f6e47e00640 1 -- 172.21.15.100:0/2869984418 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e480cf9e0 con 0x7f6e480cd030 2024-09-17T04:28:31.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.823+0000 7f6e3ea00640 1 -- 172.21.15.100:0/2869984418 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f6e3800f830 con 0x7f6e480cd030 2024-09-17T04:28:31.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.825+0000 7f6e3ea00640 1 --2- 172.21.15.100:0/2869984418 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6e28078ef0 0x7f6e2807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:31.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.825+0000 7f6e3ea00640 1 -- 172.21.15.100:0/2869984418 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4943+0+0 (secure 0 0 0) 0x7f6e3809b910 con 0x7f6e480cd030 2024-09-17T04:28:31.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.825+0000 7f6e46000640 1 --2- 172.21.15.100:0/2869984418 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6e28078ef0 0x7f6e2807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:31.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.826+0000 7f6e46000640 1 --2- 172.21.15.100:0/2869984418 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6e28078ef0 0x7f6e2807b400 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6e34034000 tx=0x7f6e34004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:31.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:31.829+0000 7f6e3ea00640 1 -- 172.21.15.100:0/2869984418 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6e48147bc0 con 0x7f6e480cd030 2024-09-17T04:28:32.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:32.046+0000 7f6e47e00640 1 -- 172.21.15.100:0/2869984418 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e480714e0 con 0x7f6e480cd030 2024-09-17T04:28:32.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:32.047+0000 7f6e3ea00640 1 -- 172.21.15.100:0/2869984418 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+3920 (secure 0 0 0) 0x7f6e38014030 con 0x7f6e480cd030 2024-09-17T04:28:32.046 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:28:32.046 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":39,"btime":"2024-09-17T04:28:25:302369+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1735051989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1735051989},{"type":"v1","addr":"172.21.15.184:6836","nonce":1735051989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5638,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3935107098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":3935107098},{"type":"v1","addr":"172.21.15.184:6837","nonce":3935107098}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/904123596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904123596},{"type":"v1","addr":"172.21.15.100:6834","nonce":904123596}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6063,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/4026973065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":4026973065},{"type":"v1","addr":"172.21.15.100:6835","nonce":4026973065}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[]} 2024-09-17T04:28:32.046 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 39 2024-09-17T04:28:32.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:32.049+0000 7f6e47e00640 1 -- 172.21.15.100:0/2869984418 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6e28078ef0 msgr2=0x7f6e2807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:32.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:32.049+0000 7f6e47e00640 1 --2- 172.21.15.100:0/2869984418 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6e28078ef0 0x7f6e2807b400 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6e34034000 tx=0x7f6e34004730 comp rx=0 tx=0).stop 2024-09-17T04:28:32.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:32.049+0000 7f6e47e00640 1 -- 172.21.15.100:0/2869984418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e480cd030 msgr2=0x7f6e480c9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:32.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:32.049+0000 7f6e47e00640 1 --2- 172.21.15.100:0/2869984418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e480cd030 0x7f6e480c9300 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6e3800c9a0 tx=0x7f6e3800ce60 comp rx=0 tx=0).stop 2024-09-17T04:28:32.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:32.049+0000 7f6e47e00640 1 -- 172.21.15.100:0/2869984418 shutdown_connections 2024-09-17T04:28:32.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:32.049+0000 7f6e47e00640 1 --2- 172.21.15.100:0/2869984418 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f6e28078ef0 0x7f6e2807b400 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:32.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:32.049+0000 7f6e47e00640 1 --2- 172.21.15.100:0/2869984418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e48160490 0x7f6e480c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:32.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:32.049+0000 7f6e47e00640 1 --2- 172.21.15.100:0/2869984418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e480d01b0 0x7f6e48147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:32.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:32.049+0000 7f6e47e00640 1 --2- 172.21.15.100:0/2869984418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e480cd030 0x7f6e480c9300 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:32.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:32.049+0000 7f6e47e00640 1 -- 172.21.15.100:0/2869984418 >> 172.21.15.100:0/2869984418 conn(0x7f6e4806d400 msgr2=0x7f6e480d0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:32.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:32.049+0000 7f6e47e00640 1 -- 172.21.15.100:0/2869984418 shutdown_connections 2024-09-17T04:28:32.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:32.049+0000 7f6e47e00640 1 -- 172.21.15.100:0/2869984418 wait complete. 2024-09-17T04:28:32.074 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T04:28:32.074 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T04:28:32.074 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T04:28:32.074 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:28:32.074 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T04:28:32.074 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T04:28:32.075 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T04:28:32.075 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:28:32.075 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T04:28:32.075 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T04:28:32.075 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T04:28:32.075 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:28:32.075 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T04:28:32.076 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T04:28:32.076 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T04:28:32.076 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:28:32.101 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T04:28:32.101 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T04:28:32.104 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T04:28:32.104 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T04:28:32.106 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T04:28:32.106 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T04:28:32.150 INFO:tasks.ceph.mds.a:Started 2024-09-17T04:28:32.150 DEBUG:teuthology.parallel:result is None 2024-09-17T04:28:32.151 INFO:tasks.ceph.mds.d:Started 2024-09-17T04:28:32.151 DEBUG:teuthology.parallel:result is None 2024-09-17T04:28:32.153 INFO:tasks.ceph.mds.c:Started 2024-09-17T04:28:32.153 DEBUG:teuthology.parallel:result is None 2024-09-17T04:28:32.344 INFO:tasks.ceph.mds.a.smithi100.stdout:starting mds.a at 2024-09-17T04:28:32.345 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-09-17T04:28:32.346 INFO:tasks.ceph.mds.c.smithi100.stdout:starting mds.c at 2024-09-17T04:28:34.523 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-09-17T04:28:35.326 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:28:35.332 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:28:38.104 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T04:28:38.104 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T04:28:38.109 INFO:tasks.ceph.mds.b:Started 2024-09-17T04:28:38.110 DEBUG:teuthology.parallel:result is None 2024-09-17T04:28:38.110 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T04:28:38.309 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-09-17T04:28:38.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.341+0000 7fefd7000640 1 -- 172.21.15.100:0/3935529731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefd00705b0 msgr2=0x7fefd00709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:38.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.341+0000 7fefd7000640 1 --2- 172.21.15.100:0/3935529731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefd00705b0 0x7fefd00709c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fefb8005fd0 tx=0x7fefb802a5a0 comp rx=0 tx=0).stop 2024-09-17T04:28:38.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.341+0000 7fefd7000640 1 -- 172.21.15.100:0/3935529731 shutdown_connections 2024-09-17T04:28:38.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.341+0000 7fefd7000640 1 --2- 172.21.15.100:0/3935529731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefd00705b0 0x7fefd00709c0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:38.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.341+0000 7fefd7000640 1 --2- 172.21.15.100:0/3935529731 >> v2:172.21.15.184:3300/0 conn(0x7fefd00cfc80 0x7fefd0156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:38.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.341+0000 7fefd7000640 1 -- 172.21.15.100:0/3935529731 >> 172.21.15.100:0/3935529731 conn(0x7fefd006d3d0 msgr2=0x7fefd00b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:38.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.341+0000 7fefd7000640 1 -- 172.21.15.100:0/3935529731 shutdown_connections 2024-09-17T04:28:38.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.341+0000 7fefd7000640 1 -- 172.21.15.100:0/3935529731 wait complete. 2024-09-17T04:28:38.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.343+0000 7fefd7000640 1 Processor -- start 2024-09-17T04:28:38.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.343+0000 7fefd7000640 1 -- start start 2024-09-17T04:28:38.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.345+0000 7fefd7000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefd00705b0 0x7fefd0156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:38.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.345+0000 7fefd7000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefd00cfc80 0x7fefd00c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:38.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.345+0000 7fefd7000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefd015ce30 0x7fefd00c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:38.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.345+0000 7fefd7000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fefd00cced0 con 0x7fefd015ce30 2024-09-17T04:28:38.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.345+0000 7fefd7000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fefd00d0170 con 0x7fefd00cfc80 2024-09-17T04:28:38.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.345+0000 7fefd7000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fefd00cd050 con 0x7fefd00705b0 2024-09-17T04:28:38.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.345+0000 7fefd5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefd015ce30 0x7fefd00c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:38.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.345+0000 7fefd5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefd015ce30 0x7fefd00c6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49572/0 (socket says 172.21.15.100:49572) 2024-09-17T04:28:38.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.345+0000 7fefd5000640 1 -- 172.21.15.100:0/311644159 learned_addr learned my addr 172.21.15.100:0/311644159 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:38.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.345+0000 7fefc7400640 1 --2- 172.21.15.100:0/311644159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefd00cfc80 0x7fefd00c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:38.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.345+0000 7fefcfe00640 1 --2- 172.21.15.100:0/311644159 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefd00705b0 0x7fefd0156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:38.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.345+0000 7fefd5000640 1 -- 172.21.15.100:0/311644159 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefd00705b0 msgr2=0x7fefd0156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:38.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.345+0000 7fefd5000640 1 --2- 172.21.15.100:0/311644159 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefd00705b0 0x7fefd0156050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:38.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.345+0000 7fefd5000640 1 -- 172.21.15.100:0/311644159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefd00cfc80 msgr2=0x7fefd00c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:38.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.345+0000 7fefd5000640 1 --2- 172.21.15.100:0/311644159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefd00cfc80 0x7fefd00c5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:38.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.345+0000 7fefd5000640 1 -- 172.21.15.100:0/311644159 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd01f4dd0 con 0x7fefd015ce30 2024-09-17T04:28:38.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.346+0000 7fefd5000640 1 --2- 172.21.15.100:0/311644159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefd015ce30 0x7fefd00c6480 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fefc800c760 tx=0x7fefc800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:38.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.346+0000 7fefccc00640 1 -- 172.21.15.100:0/311644159 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8003da0 con 0x7fefd015ce30 2024-09-17T04:28:38.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.346+0000 7fefccc00640 1 -- 172.21.15.100:0/311644159 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefc800ce10 con 0x7fefd015ce30 2024-09-17T04:28:38.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.346+0000 7fefd7000640 1 -- 172.21.15.100:0/311644159 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd01f48a0 con 0x7fefd015ce30 2024-09-17T04:28:38.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.346+0000 7fefccc00640 1 -- 172.21.15.100:0/311644159 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc80052d0 con 0x7fefd015ce30 2024-09-17T04:28:38.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.346+0000 7fefd7000640 1 -- 172.21.15.100:0/311644159 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd0159ea0 con 0x7fefd015ce30 2024-09-17T04:28:38.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.347+0000 7fefd7000640 1 -- 172.21.15.100:0/311644159 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefd01f4b40 con 0x7fefd015ce30 2024-09-17T04:28:38.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.348+0000 7fefccc00640 1 -- 172.21.15.100:0/311644159 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fefc8005470 con 0x7fefd015ce30 2024-09-17T04:28:38.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.349+0000 7fefccc00640 1 --2- 172.21.15.100:0/311644159 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fefac078f20 0x7fefac07b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:38.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.349+0000 7fefccc00640 1 -- 172.21.15.100:0/311644159 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4943+0+0 (secure 0 0 0) 0x7fefc809b1c0 con 0x7fefd015ce30 2024-09-17T04:28:38.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.350+0000 7fefcfe00640 1 --2- 172.21.15.100:0/311644159 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fefac078f20 0x7fefac07b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:38.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.350+0000 7fefcfe00640 1 --2- 172.21.15.100:0/311644159 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fefac078f20 0x7fefac07b430 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fefd009e510 tx=0x7fefc0005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:38.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.353+0000 7fefccc00640 1 -- 172.21.15.100:0/311644159 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fefd0070f00 con 0x7fefd015ce30 2024-09-17T04:28:38.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:38.548+0000 7fefd7000640 1 -- 172.21.15.100:0/311644159 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fefd00cd5b0 con 0x7fefd015ce30 2024-09-17T04:28:40.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.099+0000 7fefccc00640 1 -- 172.21.15.100:0/311644159 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v69) ==== 97+0+0 (secure 0 0 0) 0x7fefc8062cd0 con 0x7fefd015ce30 2024-09-17T04:28:40.100 INFO:teuthology.orchestra.run.smithi100.stderr: removed all blocklist entries 2024-09-17T04:28:40.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.103+0000 7fefd7000640 1 -- 172.21.15.100:0/311644159 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fefac078f20 msgr2=0x7fefac07b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:40.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.103+0000 7fefd7000640 1 --2- 172.21.15.100:0/311644159 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fefac078f20 0x7fefac07b430 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fefd009e510 tx=0x7fefc0005f90 comp rx=0 tx=0).stop 2024-09-17T04:28:40.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.103+0000 7fefd7000640 1 -- 172.21.15.100:0/311644159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefd015ce30 msgr2=0x7fefd00c6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:40.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.103+0000 7fefd7000640 1 --2- 172.21.15.100:0/311644159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefd015ce30 0x7fefd00c6480 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fefc800c760 tx=0x7fefc800cc20 comp rx=0 tx=0).stop 2024-09-17T04:28:40.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.103+0000 7fefd7000640 1 -- 172.21.15.100:0/311644159 shutdown_connections 2024-09-17T04:28:40.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.103+0000 7fefd7000640 1 --2- 172.21.15.100:0/311644159 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fefac078f20 0x7fefac07b430 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:40.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.103+0000 7fefd7000640 1 --2- 172.21.15.100:0/311644159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefd015ce30 0x7fefd00c6480 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:40.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.103+0000 7fefd7000640 1 --2- 172.21.15.100:0/311644159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefd00cfc80 0x7fefd00c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:40.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.103+0000 7fefd7000640 1 --2- 172.21.15.100:0/311644159 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefd00705b0 0x7fefd0156050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:40.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.103+0000 7fefd7000640 1 -- 172.21.15.100:0/311644159 >> 172.21.15.100:0/311644159 conn(0x7fefd006d3d0 msgr2=0x7fefd00d0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:40.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.103+0000 7fefd7000640 1 -- 172.21.15.100:0/311644159 shutdown_connections 2024-09-17T04:28:40.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.103+0000 7fefd7000640 1 -- 172.21.15.100:0/311644159 wait complete. 2024-09-17T04:28:40.132 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T04:28:40.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.357+0000 7f80dfe00640 1 -- 172.21.15.100:0/883380814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80c4004800 msgr2=0x7f80c4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:40.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.357+0000 7f80dfe00640 1 --2- 172.21.15.100:0/883380814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80c4004800 0x7f80c4004c10 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f80c8001740 tx=0x7f80c4005830 comp rx=0 tx=0).stop 2024-09-17T04:28:40.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.357+0000 7f80dfe00640 1 -- 172.21.15.100:0/883380814 shutdown_connections 2024-09-17T04:28:40.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.357+0000 7f80dfe00640 1 --2- 172.21.15.100:0/883380814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80c4004800 0x7f80c4004c10 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:40.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.357+0000 7f80dfe00640 1 -- 172.21.15.100:0/883380814 >> 172.21.15.100:0/883380814 conn(0x7f80e00b1e40 msgr2=0x7f80e00b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:40.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.358+0000 7f80dfe00640 1 -- 172.21.15.100:0/883380814 shutdown_connections 2024-09-17T04:28:40.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.358+0000 7f80dfe00640 1 -- 172.21.15.100:0/883380814 wait complete. 2024-09-17T04:28:40.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.360+0000 7f80dfe00640 1 Processor -- start 2024-09-17T04:28:40.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.360+0000 7f80dfe00640 1 -- start start 2024-09-17T04:28:40.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.361+0000 7f80dfe00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80c4004800 0x7f80e0142240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:40.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.361+0000 7f80dfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80e01f2d90 0x7f80e0157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:40.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.361+0000 7f80dfe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80e01f6400 0x7f80e01f6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:40.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.361+0000 7f80dfe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f80e0073600 con 0x7f80e01f2d90 2024-09-17T04:28:40.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.361+0000 7f80dfe00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f80e0073480 con 0x7f80e01f6400 2024-09-17T04:28:40.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.361+0000 7f80dfe00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f80e0070000 con 0x7f80c4004800 2024-09-17T04:28:40.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.362+0000 7f80d6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80e01f2d90 0x7f80e0157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:40.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.362+0000 7f80d6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80e01f2d90 0x7f80e0157050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49580/0 (socket says 172.21.15.100:49580) 2024-09-17T04:28:40.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.362+0000 7f80d6000640 1 -- 172.21.15.100:0/154984064 learned_addr learned my addr 172.21.15.100:0/154984064 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:40.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.362+0000 7f80dea00640 1 --2- 172.21.15.100:0/154984064 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80c4004800 0x7f80e0142240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:40.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.362+0000 7f80df400640 1 --2- 172.21.15.100:0/154984064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80e01f6400 0x7f80e01f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:40.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.362+0000 7f80d6000640 1 -- 172.21.15.100:0/154984064 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80c4004800 msgr2=0x7f80e0142240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:40.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.362+0000 7f80d6000640 1 --2- 172.21.15.100:0/154984064 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80c4004800 0x7f80e0142240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:40.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.362+0000 7f80d6000640 1 -- 172.21.15.100:0/154984064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80e01f6400 msgr2=0x7f80e01f6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:40.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.362+0000 7f80d6000640 1 --2- 172.21.15.100:0/154984064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80e01f6400 0x7f80e01f6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:40.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.362+0000 7f80d6000640 1 -- 172.21.15.100:0/154984064 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e0142780 con 0x7f80e01f2d90 2024-09-17T04:28:40.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.362+0000 7f80df400640 1 --2- 172.21.15.100:0/154984064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80e01f6400 0x7f80e01f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:28:40.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.362+0000 7f80d6000640 1 --2- 172.21.15.100:0/154984064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80e01f2d90 0x7f80e0157050 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f80c8000c00 tx=0x7f80c80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:40.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.363+0000 7f80d7400640 1 -- 172.21.15.100:0/154984064 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c80068c0 con 0x7f80e01f2d90 2024-09-17T04:28:40.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.363+0000 7f80d7400640 1 -- 172.21.15.100:0/154984064 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80c8039070 con 0x7f80e01f2d90 2024-09-17T04:28:40.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.363+0000 7f80d7400640 1 -- 172.21.15.100:0/154984064 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c80336d0 con 0x7f80e01f2d90 2024-09-17T04:28:40.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.363+0000 7f80dfe00640 1 -- 172.21.15.100:0/154984064 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e01f6d50 con 0x7f80e01f2d90 2024-09-17T04:28:40.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.363+0000 7f80dfe00640 1 -- 172.21.15.100:0/154984064 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e01f3720 con 0x7f80e01f2d90 2024-09-17T04:28:40.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.364+0000 7f80dfe00640 1 -- 172.21.15.100:0/154984064 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80e0074db0 con 0x7f80e01f2d90 2024-09-17T04:28:40.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.365+0000 7f80d7400640 1 -- 172.21.15.100:0/154984064 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f80c8044020 con 0x7f80e01f2d90 2024-09-17T04:28:40.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.366+0000 7f80d7400640 1 --2- 172.21.15.100:0/154984064 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f80bc078f20 0x7f80bc07b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:40.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.366+0000 7f80d7400640 1 -- 172.21.15.100:0/154984064 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 4857+0+0 (secure 0 0 0) 0x7f80c80bad20 con 0x7f80e01f2d90 2024-09-17T04:28:40.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.367+0000 7f80dea00640 1 --2- 172.21.15.100:0/154984064 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f80bc078f20 0x7f80bc07b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:40.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.367+0000 7f80dea00640 1 --2- 172.21.15.100:0/154984064 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f80bc078f20 0x7f80bc07b430 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009cd0 tx=0x7f80d0005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:40.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.371+0000 7f80d7400640 1 -- 172.21.15.100:0/154984064 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f80e0074db0 con 0x7f80e01f2d90 2024-09-17T04:28:40.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.620+0000 7f80dfe00640 1 -- 172.21.15.100:0/154984064 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f80e01f31d0 con 0x7f80e01f2d90 2024-09-17T04:28:40.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.620+0000 7f80d7400640 1 -- 172.21.15.100:0/154984064 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5617 (secure 0 0 0) 0x7f80e01f6d50 con 0x7f80e01f2d90 2024-09-17T04:28:40.622 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:28:40.622 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:28:40.622 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_dump": [ 2024-09-17T04:28:40.622 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:28:40.622 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.a", 2024-09-17T04:28:40.623 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmV8asJRAAkSgIWqGc1OeSRbs9PEOT8w==", 2024-09-17T04:28:40.623 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:28:40.623 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:28:40.623 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:28:40.623 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:28:40.623 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:28:40.624 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:28:40.624 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:28:40.624 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:28:40.624 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.b", 2024-09-17T04:28:40.624 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmpM7ZBRAA1Og785tJwa6NA9zBdmNNQA==", 2024-09-17T04:28:40.624 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:28:40.624 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:28:40.625 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:28:40.625 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:28:40.625 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:28:40.625 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:28:40.625 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:28:40.626 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:28:40.626 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.c", 2024-09-17T04:28:40.626 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmXaOGMxAAT+96g/5fAzPW8Cc2HtQUyA==", 2024-09-17T04:28:40.626 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:28:40.626 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:28:40.626 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:28:40.626 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:28:40.627 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:28:40.627 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:28:40.627 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:28:40.627 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:28:40.627 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.d", 2024-09-17T04:28:40.627 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmRqC9ExAA23kZ0Ks1lGN2k4GXL8ASrg==", 2024-09-17T04:28:40.628 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:28:40.628 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:28:40.628 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:28:40.628 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:28:40.628 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:28:40.628 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:28:40.629 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:28:40.629 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:28:40.629 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.0", 2024-09-17T04:28:40.629 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAwBOlmofFmAxAARerYkU9B2a9MM9vHa3o4LQ==", 2024-09-17T04:28:40.629 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:28:40.630 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:28:40.630 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:28:40.630 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:28:40.630 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:28:40.630 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:28:40.630 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:28:40.631 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.1", 2024-09-17T04:28:40.631 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAzBOlmbz6VExAAoVqSqXTc1wArKhdA6+NxnA==", 2024-09-17T04:28:40.631 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:28:40.631 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:28:40.631 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:28:40.631 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:28:40.632 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:28:40.632 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:28:40.632 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:28:40.632 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.2", 2024-09-17T04:28:40.632 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA2BOlmfeSAJBAAchY6XhyDRusc+zqwQ6v6wQ==", 2024-09-17T04:28:40.632 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:28:40.633 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:28:40.633 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:28:40.633 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:28:40.633 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:28:40.633 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:28:40.633 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:28:40.633 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.3", 2024-09-17T04:28:40.633 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA6BOlm/TQiBBAAHhbs2+RCjAenGR1frhbTmQ==", 2024-09-17T04:28:40.634 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:28:40.634 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:28:40.634 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:28:40.634 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:28:40.634 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:28:40.634 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:28:40.634 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:28:40.634 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.4", 2024-09-17T04:28:40.634 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBABOlm7OKvBhAALkaAKEUYN5ChwGejoSF8yA==", 2024-09-17T04:28:40.634 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:28:40.635 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:28:40.635 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:28:40.635 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:28:40.635 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:28:40.635 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:28:40.635 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:28:40.635 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.5", 2024-09-17T04:28:40.635 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBDBOlmQpi1FhAAbA47dg6ckK1PFoCZL0SBEQ==", 2024-09-17T04:28:40.636 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:28:40.636 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:28:40.636 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:28:40.636 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:28:40.636 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:28:40.636 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:28:40.636 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:28:40.636 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.6", 2024-09-17T04:28:40.636 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBGBOlmx/w0JhAAWcPvZkMYtXD/EMKrg/W/4Q==", 2024-09-17T04:28:40.636 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:28:40.637 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:28:40.637 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:28:40.637 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:28:40.637 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:28:40.637 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:28:40.637 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:28:40.637 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.7", 2024-09-17T04:28:40.637 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBJBOlmWHjWNBAAgdul2UKY9g5p7EVi0ZdrFw==", 2024-09-17T04:28:40.637 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:28:40.638 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:28:40.638 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:28:40.638 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:28:40.638 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:28:40.638 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:28:40.638 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:28:40.638 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.0", 2024-09-17T04:28:40.638 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmfYg9IBAABP1uRKGOugrA47VxwRLkjA==", 2024-09-17T04:28:40.638 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:28:40.639 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:28:40.639 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow r", 2024-09-17T04:28:40.639 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T04:28:40.639 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:28:40.639 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:28:40.639 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:28:40.639 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.1", 2024-09-17T04:28:40.639 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmByhhNRAAPGaxFCDd8ywFVwwhprrPbQ==", 2024-09-17T04:28:40.640 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:28:40.640 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:28:40.640 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:28:40.640 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:28:40.640 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:28:40.640 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:28:40.640 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:28:40.640 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:28:40.641 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.2", 2024-09-17T04:28:40.641 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmgcm0KhAAGF06VlHjBM/AGxafN6/3aA==", 2024-09-17T04:28:40.641 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:28:40.641 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:28:40.641 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:28:40.641 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:28:40.641 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:28:40.641 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:28:40.641 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:28:40.642 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:28:40.642 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.3", 2024-09-17T04:28:40.642 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtBOlm9VQ6BBAAXcahu35/uDm8Z+f+3SF0Gg==", 2024-09-17T04:28:40.642 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:28:40.642 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:28:40.642 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:28:40.642 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:28:40.642 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:28:40.642 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:28:40.643 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:28:40.643 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:28:40.643 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.admin", 2024-09-17T04:28:40.643 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAqBOlmZ+bIJBAAG3oaMNlYWNakywGDvAGbGw==", 2024-09-17T04:28:40.643 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:28:40.643 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:28:40.643 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:28:40.643 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:28:40.643 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:28:40.643 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:28:40.644 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:28:40.644 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:28:40.644 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.alice", 2024-09-17T04:28:40.644 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBZBelmbWY/JxAAwE6vqDQNgC9cyEdUFZwtBA==", 2024-09-17T04:28:40.644 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:28:40.644 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol75040", 2024-09-17T04:28:40.644 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow r", 2024-09-17T04:28:40.644 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rw pool=cephfs_data" 2024-09-17T04:28:40.645 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:28:40.645 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:28:40.645 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:28:40.645 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mgr.x", 2024-09-17T04:28:40.645 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmfaL7GhAArqLH66RkbBsz+TNOKtPHOA==", 2024-09-17T04:28:40.645 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:28:40.645 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:28:40.646 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile mgr", 2024-09-17T04:28:40.646 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:28:40.646 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:28:40.646 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:28:40.646 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:28:40.646 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mgr.y", 2024-09-17T04:28:40.647 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmM4D1DhAA5uUWgRVOMujWqbYzOjFp9A==", 2024-09-17T04:28:40.647 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:28:40.647 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:28:40.647 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile mgr", 2024-09-17T04:28:40.647 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:28:40.647 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:28:40.648 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:28:40.648 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-17T04:28:40.648 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:28:40.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.623+0000 7f80dfe00640 1 -- 172.21.15.100:0/154984064 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f80bc078f20 msgr2=0x7f80bc07b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:40.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.623+0000 7f80dfe00640 1 --2- 172.21.15.100:0/154984064 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f80bc078f20 0x7f80bc07b430 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009cd0 tx=0x7f80d0005f90 comp rx=0 tx=0).stop 2024-09-17T04:28:40.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.623+0000 7f80dfe00640 1 -- 172.21.15.100:0/154984064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80e01f2d90 msgr2=0x7f80e0157050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:40.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.623+0000 7f80dfe00640 1 --2- 172.21.15.100:0/154984064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80e01f2d90 0x7f80e0157050 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f80c8000c00 tx=0x7f80c80048f0 comp rx=0 tx=0).stop 2024-09-17T04:28:40.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.623+0000 7f80dfe00640 1 -- 172.21.15.100:0/154984064 shutdown_connections 2024-09-17T04:28:40.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.623+0000 7f80dfe00640 1 --2- 172.21.15.100:0/154984064 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f80bc078f20 0x7f80bc07b430 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:40.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.623+0000 7f80dfe00640 1 --2- 172.21.15.100:0/154984064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80e01f6400 0x7f80e01f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:40.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.623+0000 7f80dfe00640 1 --2- 172.21.15.100:0/154984064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80e01f2d90 0x7f80e0157050 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:40.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.623+0000 7f80dfe00640 1 --2- 172.21.15.100:0/154984064 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80c4004800 0x7f80e0142240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:40.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.623+0000 7f80dfe00640 1 -- 172.21.15.100:0/154984064 >> 172.21.15.100:0/154984064 conn(0x7f80e00b1e40 msgr2=0x7f80e00746a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:40.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.623+0000 7f80dfe00640 1 -- 172.21.15.100:0/154984064 shutdown_connections 2024-09-17T04:28:40.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.623+0000 7f80dfe00640 1 -- 172.21.15.100:0/154984064 wait complete. 2024-09-17T04:28:40.653 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.alice 2024-09-17T04:28:40.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.875+0000 7f3059800640 1 -- 172.21.15.100:0/3770683120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3054070cc0 msgr2=0x7f3054156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:40.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.875+0000 7f3059800640 1 --2- 172.21.15.100:0/3770683120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3054070cc0 0x7f3054156e40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f303c009860 tx=0x7f303c02a4f0 comp rx=0 tx=0).stop 2024-09-17T04:28:40.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.875+0000 7f3059800640 1 -- 172.21.15.100:0/3770683120 shutdown_connections 2024-09-17T04:28:40.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.875+0000 7f3059800640 1 --2- 172.21.15.100:0/3770683120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3054070cc0 0x7f3054156e40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:40.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.875+0000 7f3059800640 1 -- 172.21.15.100:0/3770683120 >> 172.21.15.100:0/3770683120 conn(0x7f305406d3d0 msgr2=0x7f30540b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:40.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.875+0000 7f3059800640 1 -- 172.21.15.100:0/3770683120 shutdown_connections 2024-09-17T04:28:40.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.875+0000 7f3059800640 1 -- 172.21.15.100:0/3770683120 wait complete. 2024-09-17T04:28:40.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.877+0000 7f3059800640 1 Processor -- start 2024-09-17T04:28:40.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.877+0000 7f3059800640 1 -- start start 2024-09-17T04:28:40.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.879+0000 7f3059800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30540cfc80 0x7f30541599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:40.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.879+0000 7f3059800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f305415ce30 0x7f30540b0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:40.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.879+0000 7f3059800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30541f54c0 0x7f30541574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:40.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.879+0000 7f3059800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f30540cd100 con 0x7f305415ce30 2024-09-17T04:28:40.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.879+0000 7f3059800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f30540ccf80 con 0x7f30541f54c0 2024-09-17T04:28:40.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.879+0000 7f3059800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f30540cd280 con 0x7f30540cfc80 2024-09-17T04:28:40.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.879+0000 7f304be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f305415ce30 0x7f30540b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:40.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.879+0000 7f304be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f305415ce30 0x7f30540b0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49586/0 (socket says 172.21.15.100:49586) 2024-09-17T04:28:40.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.879+0000 7f304be00640 1 -- 172.21.15.100:0/3255922743 learned_addr learned my addr 172.21.15.100:0/3255922743 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:40.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.879+0000 7f3052a00640 1 --2- 172.21.15.100:0/3255922743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30540cfc80 0x7f30541599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:40.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.879+0000 7f3053400640 1 --2- 172.21.15.100:0/3255922743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30541f54c0 0x7f30541574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:40.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.879+0000 7f304be00640 1 -- 172.21.15.100:0/3255922743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30540cfc80 msgr2=0x7f30541599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:40.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.879+0000 7f304be00640 1 --2- 172.21.15.100:0/3255922743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30540cfc80 0x7f30541599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:40.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.879+0000 7f304be00640 1 -- 172.21.15.100:0/3255922743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30541f54c0 msgr2=0x7f30541574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:40.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.879+0000 7f304be00640 1 --2- 172.21.15.100:0/3255922743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30541f54c0 0x7f30541574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:40.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.879+0000 7f304be00640 1 -- 172.21.15.100:0/3255922743 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30540b0c80 con 0x7f305415ce30 2024-09-17T04:28:40.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.879+0000 7f3052a00640 1 --2- 172.21.15.100:0/3255922743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30540cfc80 0x7f30541599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:28:40.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.880+0000 7f304be00640 1 --2- 172.21.15.100:0/3255922743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f305415ce30 0x7f30540b0740 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f303c02aae0 tx=0x7f303c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:40.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.880+0000 7f304aa00640 1 -- 172.21.15.100:0/3255922743 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c003e20 con 0x7f305415ce30 2024-09-17T04:28:40.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.880+0000 7f304aa00640 1 -- 172.21.15.100:0/3255922743 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f303c0315b0 con 0x7f305415ce30 2024-09-17T04:28:40.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.880+0000 7f3059800640 1 -- 172.21.15.100:0/3255922743 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30541f5f80 con 0x7f305415ce30 2024-09-17T04:28:40.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.880+0000 7f304aa00640 1 -- 172.21.15.100:0/3255922743 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c0319c0 con 0x7f305415ce30 2024-09-17T04:28:40.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.880+0000 7f3059800640 1 -- 172.21.15.100:0/3255922743 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30540aff40 con 0x7f305415ce30 2024-09-17T04:28:40.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.881+0000 7f3059800640 1 -- 172.21.15.100:0/3255922743 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30541f5cf0 con 0x7f305415ce30 2024-09-17T04:28:40.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.882+0000 7f304aa00640 1 -- 172.21.15.100:0/3255922743 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f303c031b60 con 0x7f305415ce30 2024-09-17T04:28:40.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.883+0000 7f304aa00640 1 --2- 172.21.15.100:0/3255922743 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3030078ef0 0x7f303007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:40.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.883+0000 7f304aa00640 1 -- 172.21.15.100:0/3255922743 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 4857+0+0 (secure 0 0 0) 0x7f303c0bc040 con 0x7f305415ce30 2024-09-17T04:28:40.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.884+0000 7f3052a00640 1 --2- 172.21.15.100:0/3255922743 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3030078ef0 0x7f303007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:40.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.884+0000 7f3052a00640 1 --2- 172.21.15.100:0/3255922743 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3030078ef0 0x7f303007b400 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3044001120 tx=0x7f3044009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:40.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:40.888+0000 7f304aa00640 1 -- 172.21.15.100:0/3255922743 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f303c03f030 con 0x7f305415ce30 2024-09-17T04:28:41.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.140+0000 7f3059800640 1 -- 172.21.15.100:0/3255922743 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.alice"} v 0) -- 0x7f30540cd7b0 con 0x7f305415ce30 2024-09-17T04:28:41.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.143+0000 7f304aa00640 1 -- 172.21.15.100:0/3255922743 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.alice"}]=0 v15) ==== 82+0+0 (secure 0 0 0) 0x7f303c084200 con 0x7f305415ce30 2024-09-17T04:28:41.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.145+0000 7f3059800640 1 -- 172.21.15.100:0/3255922743 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3030078ef0 msgr2=0x7f303007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:41.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.145+0000 7f3059800640 1 --2- 172.21.15.100:0/3255922743 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3030078ef0 0x7f303007b400 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3044001120 tx=0x7f3044009040 comp rx=0 tx=0).stop 2024-09-17T04:28:41.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.145+0000 7f3059800640 1 -- 172.21.15.100:0/3255922743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f305415ce30 msgr2=0x7f30540b0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:41.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.145+0000 7f3059800640 1 --2- 172.21.15.100:0/3255922743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f305415ce30 0x7f30540b0740 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f303c02aae0 tx=0x7f303c003820 comp rx=0 tx=0).stop 2024-09-17T04:28:41.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.145+0000 7f3059800640 1 -- 172.21.15.100:0/3255922743 shutdown_connections 2024-09-17T04:28:41.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.145+0000 7f3059800640 1 --2- 172.21.15.100:0/3255922743 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3030078ef0 0x7f303007b400 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:41.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.145+0000 7f3059800640 1 --2- 172.21.15.100:0/3255922743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30541f54c0 0x7f30541574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:41.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.145+0000 7f3059800640 1 --2- 172.21.15.100:0/3255922743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f305415ce30 0x7f30540b0740 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:41.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.145+0000 7f3059800640 1 --2- 172.21.15.100:0/3255922743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30540cfc80 0x7f30541599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:41.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.145+0000 7f3059800640 1 -- 172.21.15.100:0/3255922743 >> 172.21.15.100:0/3255922743 conn(0x7f305406d3d0 msgr2=0x7f305406f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:41.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.145+0000 7f3059800640 1 -- 172.21.15.100:0/3255922743 shutdown_connections 2024-09-17T04:28:41.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.146+0000 7f3059800640 1 -- 172.21.15.100:0/3255922743 wait complete. 2024-09-17T04:28:41.171 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T04:28:41.254 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-09-17T04:28:41.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.454+0000 7f02f5400640 1 -- 172.21.15.100:0/3481251182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02f00af630 msgr2=0x7f02f0073500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:41.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.454+0000 7f02f5400640 1 --2- 172.21.15.100:0/3481251182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02f00af630 0x7f02f0073500 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f02f00b2cb0 tx=0x7f02e002a5f0 comp rx=0 tx=0).stop 2024-09-17T04:28:41.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.454+0000 7f02f5400640 1 -- 172.21.15.100:0/3481251182 shutdown_connections 2024-09-17T04:28:41.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.454+0000 7f02f5400640 1 --2- 172.21.15.100:0/3481251182 >> v2:172.21.15.184:3300/0 conn(0x7f02f0073a40 0x7f02f006f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:41.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.454+0000 7f02f5400640 1 --2- 172.21.15.100:0/3481251182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02f00af630 0x7f02f0073500 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:41.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.454+0000 7f02f5400640 1 -- 172.21.15.100:0/3481251182 >> 172.21.15.100:0/3481251182 conn(0x7f02f006d730 msgr2=0x7f02f00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:41.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.454+0000 7f02f5400640 1 -- 172.21.15.100:0/3481251182 shutdown_connections 2024-09-17T04:28:41.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.454+0000 7f02f5400640 1 -- 172.21.15.100:0/3481251182 wait complete. 2024-09-17T04:28:41.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.457+0000 7f02f5400640 1 Processor -- start 2024-09-17T04:28:41.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.457+0000 7f02f5400640 1 -- start start 2024-09-17T04:28:41.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.458+0000 7f02f5400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02f006f5e0 0x7f02f0147770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:41.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.458+0000 7f02f5400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02f0073a40 0x7f02f01e92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:41.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.458+0000 7f02f5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02f00af630 0x7f02f01e9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:41.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.458+0000 7f02f5400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f02f0162c70 con 0x7f02f00af630 2024-09-17T04:28:41.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.458+0000 7f02f5400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f02f00b4400 con 0x7f02f006f5e0 2024-09-17T04:28:41.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.458+0000 7f02f5400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f02f0162df0 con 0x7f02f0073a40 2024-09-17T04:28:41.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.458+0000 7f02ef200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02f00af630 0x7f02f01e9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:41.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.458+0000 7f02ef200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02f00af630 0x7f02f01e9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49598/0 (socket says 172.21.15.100:49598) 2024-09-17T04:28:41.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.458+0000 7f02ef200640 1 -- 172.21.15.100:0/2357393631 learned_addr learned my addr 172.21.15.100:0/2357393631 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:41.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.459+0000 7f02ede00640 1 --2- 172.21.15.100:0/2357393631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02f0073a40 0x7f02f01e92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:41.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.459+0000 7f02ee800640 1 --2- 172.21.15.100:0/2357393631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02f006f5e0 0x7f02f0147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:41.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.459+0000 7f02ef200640 1 -- 172.21.15.100:0/2357393631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02f0073a40 msgr2=0x7f02f01e92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:41.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.459+0000 7f02ef200640 1 --2- 172.21.15.100:0/2357393631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02f0073a40 0x7f02f01e92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:41.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.459+0000 7f02ef200640 1 -- 172.21.15.100:0/2357393631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02f006f5e0 msgr2=0x7f02f0147770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:41.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.459+0000 7f02ef200640 1 --2- 172.21.15.100:0/2357393631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02f006f5e0 0x7f02f0147770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:41.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.459+0000 7f02ef200640 1 -- 172.21.15.100:0/2357393631 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f0158120 con 0x7f02f00af630 2024-09-17T04:28:41.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.459+0000 7f02ede00640 1 --2- 172.21.15.100:0/2357393631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02f0073a40 0x7f02f01e92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:28:41.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.459+0000 7f02ef200640 1 --2- 172.21.15.100:0/2357393631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02f00af630 0x7f02f01e9820 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f02e800c730 tx=0x7f02e800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:41.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.459+0000 7f02e6a00640 1 -- 172.21.15.100:0/2357393631 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e8003b70 con 0x7f02f00af630 2024-09-17T04:28:41.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.460+0000 7f02e6a00640 1 -- 172.21.15.100:0/2357393631 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02e8003d30 con 0x7f02f00af630 2024-09-17T04:28:41.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.460+0000 7f02f5400640 1 -- 172.21.15.100:0/2357393631 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f009d400 con 0x7f02f00af630 2024-09-17T04:28:41.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.460+0000 7f02e6a00640 1 -- 172.21.15.100:0/2357393631 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e80052d0 con 0x7f02f00af630 2024-09-17T04:28:41.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.460+0000 7f02f5400640 1 -- 172.21.15.100:0/2357393631 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f01e8ae0 con 0x7f02f00af630 2024-09-17T04:28:41.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.461+0000 7f02f5400640 1 -- 172.21.15.100:0/2357393631 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02f009d170 con 0x7f02f00af630 2024-09-17T04:28:41.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.462+0000 7f02e6a00640 1 -- 172.21.15.100:0/2357393631 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f02e8005680 con 0x7f02f00af630 2024-09-17T04:28:41.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.463+0000 7f02e6a00640 1 --2- 172.21.15.100:0/2357393631 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f02d0078ef0 0x7f02d007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:41.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.463+0000 7f02e6a00640 1 -- 172.21.15.100:0/2357393631 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 4857+0+0 (secure 0 0 0) 0x7f02e809afd0 con 0x7f02f00af630 2024-09-17T04:28:41.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.463+0000 7f02ee800640 1 --2- 172.21.15.100:0/2357393631 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f02d0078ef0 0x7f02d007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:41.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.464+0000 7f02ee800640 1 --2- 172.21.15.100:0/2357393631 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f02d0078ef0 0x7f02d007b400 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f02e0033000 tx=0x7f02e0004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:41.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.467+0000 7f02e6a00640 1 -- 172.21.15.100:0/2357393631 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f02f0158120 con 0x7f02f00af630 2024-09-17T04:28:41.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.652+0000 7f02f5400640 1 -- 172.21.15.100:0/2357393631 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f02f0163310 con 0x7f02f00af630 2024-09-17T04:28:41.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.652+0000 7f02e6a00640 1 -- 172.21.15.100:0/2357393631 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v69) ==== 84+0+65 (secure 0 0 0) 0x7f02f0073e50 con 0x7f02f00af630 2024-09-17T04:28:41.653 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:28:41.653 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-09-17T04:28:41.653 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:28:41.653 INFO:teuthology.orchestra.run.smithi100.stdout: "poolnum": 1, 2024-09-17T04:28:41.653 INFO:teuthology.orchestra.run.smithi100.stdout: "poolname": ".mgr" 2024-09-17T04:28:41.653 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:28:41.653 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-09-17T04:28:41.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.655+0000 7f02f5400640 1 -- 172.21.15.100:0/2357393631 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f02d0078ef0 msgr2=0x7f02d007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:41.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.655+0000 7f02f5400640 1 --2- 172.21.15.100:0/2357393631 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f02d0078ef0 0x7f02d007b400 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f02e0033000 tx=0x7f02e0004730 comp rx=0 tx=0).stop 2024-09-17T04:28:41.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.655+0000 7f02f5400640 1 -- 172.21.15.100:0/2357393631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02f00af630 msgr2=0x7f02f01e9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:41.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.655+0000 7f02f5400640 1 --2- 172.21.15.100:0/2357393631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02f00af630 0x7f02f01e9820 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f02e800c730 tx=0x7f02e800cbf0 comp rx=0 tx=0).stop 2024-09-17T04:28:41.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.655+0000 7f02f5400640 1 -- 172.21.15.100:0/2357393631 shutdown_connections 2024-09-17T04:28:41.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.655+0000 7f02f5400640 1 --2- 172.21.15.100:0/2357393631 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f02d0078ef0 0x7f02d007b400 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:41.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.655+0000 7f02f5400640 1 --2- 172.21.15.100:0/2357393631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02f00af630 0x7f02f01e9820 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:41.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.655+0000 7f02f5400640 1 --2- 172.21.15.100:0/2357393631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02f0073a40 0x7f02f01e92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:41.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.655+0000 7f02f5400640 1 --2- 172.21.15.100:0/2357393631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02f006f5e0 0x7f02f0147770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:41.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.655+0000 7f02f5400640 1 -- 172.21.15.100:0/2357393631 >> 172.21.15.100:0/2357393631 conn(0x7f02f006d730 msgr2=0x7f02f0162070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:41.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.655+0000 7f02f5400640 1 -- 172.21.15.100:0/2357393631 shutdown_connections 2024-09-17T04:28:41.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.655+0000 7f02f5400640 1 -- 172.21.15.100:0/2357393631 wait complete. 2024-09-17T04:28:41.677 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T04:28:41.678 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T04:28:41.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.903+0000 7fecdb800640 1 -- 172.21.15.100:0/3087480896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecb8004800 msgr2=0x7fecb8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:41.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.903+0000 7fecdb800640 1 --2- 172.21.15.100:0/3087480896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecb8004800 0x7fecb8004c10 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fecd4051f40 tx=0x7fecb8005830 comp rx=0 tx=0).stop 2024-09-17T04:28:41.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.903+0000 7fecd9800640 1 -- 172.21.15.100:0/3087480896 reap_dead start 2024-09-17T04:28:41.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.903+0000 7fecdb800640 1 -- 172.21.15.100:0/3087480896 shutdown_connections 2024-09-17T04:28:41.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.903+0000 7fecdb800640 1 -- 172.21.15.100:0/3087480896 >> 172.21.15.100:0/3087480896 conn(0x7fecd40cef30 msgr2=0x7fecd40cf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:41.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.903+0000 7fecdb800640 1 -- 172.21.15.100:0/3087480896 shutdown_connections 2024-09-17T04:28:41.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.904+0000 7fecdb800640 1 -- 172.21.15.100:0/3087480896 wait complete. 2024-09-17T04:28:41.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.906+0000 7fecdb800640 1 Processor -- start 2024-09-17T04:28:41.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.906+0000 7fecdb800640 1 -- start start 2024-09-17T04:28:41.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.907+0000 7fecdb800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecb8004800 0x7fecd406fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:41.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.907+0000 7fecdb800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecd406e710 0x7fecd40b6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:41.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.907+0000 7fecdb800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecd415a3a0 0x7fecd41f8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:41.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.907+0000 7fecdb800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fecd4070f20 con 0x7fecd415a3a0 2024-09-17T04:28:41.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.907+0000 7fecdb800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fecd40d3310 con 0x7fecd406e710 2024-09-17T04:28:41.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.907+0000 7fecdb800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fecd40b7e60 con 0x7fecb8004800 2024-09-17T04:28:41.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.908+0000 7fecd9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecd415a3a0 0x7fecd41f8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:41.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.908+0000 7fecd9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecd415a3a0 0x7fecd41f8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49604/0 (socket says 172.21.15.100:49604) 2024-09-17T04:28:41.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.908+0000 7fecd9800640 1 -- 172.21.15.100:0/4040343032 learned_addr learned my addr 172.21.15.100:0/4040343032 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:41.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.908+0000 7fecd8e00640 1 --2- 172.21.15.100:0/4040343032 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecb8004800 0x7fecd406fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:41.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.908+0000 7fecd3e00640 1 --2- 172.21.15.100:0/4040343032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecd406e710 0x7fecd40b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:41.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.908+0000 7fecd9800640 1 -- 172.21.15.100:0/4040343032 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecb8004800 msgr2=0x7fecd406fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:41.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.908+0000 7fecd9800640 1 --2- 172.21.15.100:0/4040343032 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecb8004800 0x7fecd406fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:41.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.908+0000 7fecd9800640 1 -- 172.21.15.100:0/4040343032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecd406e710 msgr2=0x7fecd40b6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:41.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.908+0000 7fecd9800640 1 --2- 172.21.15.100:0/4040343032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecd406e710 0x7fecd40b6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:41.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.908+0000 7fecd9800640 1 -- 172.21.15.100:0/4040343032 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd406eec0 con 0x7fecd415a3a0 2024-09-17T04:28:41.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.908+0000 7fecd3e00640 1 --2- 172.21.15.100:0/4040343032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecd406e710 0x7fecd40b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:28:41.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.908+0000 7fecd9800640 1 --2- 172.21.15.100:0/4040343032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecd415a3a0 0x7fecd41f8cf0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fecd40c5f80 tx=0x7feccc011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:41.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.908+0000 7fecd0c00640 1 -- 172.21.15.100:0/4040343032 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feccc004440 con 0x7fecd415a3a0 2024-09-17T04:28:41.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.908+0000 7fecd0c00640 1 -- 172.21.15.100:0/4040343032 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feccc00f8f0 con 0x7fecd415a3a0 2024-09-17T04:28:41.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.909+0000 7fecdb800640 1 -- 172.21.15.100:0/4040343032 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecd406ebe0 con 0x7fecd415a3a0 2024-09-17T04:28:41.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.909+0000 7fecd0c00640 1 -- 172.21.15.100:0/4040343032 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feccc00fd40 con 0x7fecd415a3a0 2024-09-17T04:28:41.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.909+0000 7fecdb800640 1 -- 172.21.15.100:0/4040343032 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecd4159ba0 con 0x7fecd415a3a0 2024-09-17T04:28:41.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.910+0000 7fecdb800640 1 -- 172.21.15.100:0/4040343032 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecd40d0d70 con 0x7fecd415a3a0 2024-09-17T04:28:41.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.911+0000 7fecd0c00640 1 -- 172.21.15.100:0/4040343032 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7feccc0035f0 con 0x7fecd415a3a0 2024-09-17T04:28:41.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.912+0000 7fecd0c00640 1 --2- 172.21.15.100:0/4040343032 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fecb4078ef0 0x7fecb407b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:41.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.912+0000 7fecd0c00640 1 -- 172.21.15.100:0/4040343032 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 4857+0+0 (secure 0 0 0) 0x7feccc09ba90 con 0x7fecd415a3a0 2024-09-17T04:28:41.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.912+0000 7fecd8e00640 1 --2- 172.21.15.100:0/4040343032 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fecb4078ef0 0x7fecb407b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:41.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.913+0000 7fecd8e00640 1 --2- 172.21.15.100:0/4040343032 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fecb4078ef0 0x7fecb407b400 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fecc4001be0 tx=0x7fecc40020e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:41.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:41.917+0000 7fecd0c00640 1 -- 172.21.15.100:0/4040343032 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7feccc063c90 con 0x7fecd415a3a0 2024-09-17T04:28:42.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.099+0000 7fecdb800640 1 -- 172.21.15.100:0/4040343032 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fecd4071370 con 0x7fecd415a3a0 2024-09-17T04:28:42.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.148+0000 7fecd0c00640 1 -- 172.21.15.100:0/4040343032 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v70) ==== 138+0+0 (secure 0 0 0) 0x7feccc063590 con 0x7fecd415a3a0 2024-09-17T04:28:42.149 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' created 2024-09-17T04:28:42.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.152+0000 7fecdb800640 1 -- 172.21.15.100:0/4040343032 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fecb4078ef0 msgr2=0x7fecb407b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:42.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.152+0000 7fecdb800640 1 --2- 172.21.15.100:0/4040343032 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fecb4078ef0 0x7fecb407b400 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fecc4001be0 tx=0x7fecc40020e0 comp rx=0 tx=0).stop 2024-09-17T04:28:42.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.152+0000 7fecdb800640 1 -- 172.21.15.100:0/4040343032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecd415a3a0 msgr2=0x7fecd41f8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:42.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.152+0000 7fecdb800640 1 --2- 172.21.15.100:0/4040343032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecd415a3a0 0x7fecd41f8cf0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fecd40c5f80 tx=0x7feccc011c60 comp rx=0 tx=0).stop 2024-09-17T04:28:42.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.152+0000 7fecdb800640 1 -- 172.21.15.100:0/4040343032 shutdown_connections 2024-09-17T04:28:42.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.152+0000 7fecdb800640 1 --2- 172.21.15.100:0/4040343032 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fecb4078ef0 0x7fecb407b400 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:42.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.152+0000 7fecdb800640 1 --2- 172.21.15.100:0/4040343032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecd415a3a0 0x7fecd41f8cf0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:42.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.152+0000 7fecdb800640 1 --2- 172.21.15.100:0/4040343032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecd406e710 0x7fecd40b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:42.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.152+0000 7fecdb800640 1 --2- 172.21.15.100:0/4040343032 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecb8004800 0x7fecd406fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:42.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.152+0000 7fecdb800640 1 -- 172.21.15.100:0/4040343032 >> 172.21.15.100:0/4040343032 conn(0x7fecd40cef30 msgr2=0x7fecd415c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:42.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.152+0000 7fecdb800640 1 -- 172.21.15.100:0/4040343032 shutdown_connections 2024-09-17T04:28:42.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.152+0000 7fecdb800640 1 -- 172.21.15.100:0/4040343032 wait complete. 2024-09-17T04:28:42.180 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T04:28:42.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.409+0000 7f5f6e800640 1 -- 172.21.15.100:0/1451206536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f68145550 msgr2=0x7f5f6806fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:42.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.409+0000 7f5f6e800640 1 --2- 172.21.15.100:0/1451206536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f68145550 0x7f5f6806fb40 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5f681464e0 tx=0x7f5f5802a680 comp rx=0 tx=0).stop 2024-09-17T04:28:42.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.409+0000 7f5f6e800640 1 -- 172.21.15.100:0/1451206536 shutdown_connections 2024-09-17T04:28:42.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.409+0000 7f5f6e800640 1 --2- 172.21.15.100:0/1451206536 >> v2:172.21.15.184:3300/0 conn(0x7f5f68073c50 0x7f5f680cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:42.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.409+0000 7f5f6e800640 1 --2- 172.21.15.100:0/1451206536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f68145550 0x7f5f6806fb40 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:42.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.409+0000 7f5f6e800640 1 -- 172.21.15.100:0/1451206536 >> 172.21.15.100:0/1451206536 conn(0x7f5f6806d1c0 msgr2=0x7f5f6806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:42.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.409+0000 7f5f6e800640 1 -- 172.21.15.100:0/1451206536 shutdown_connections 2024-09-17T04:28:42.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.409+0000 7f5f6e800640 1 -- 172.21.15.100:0/1451206536 wait complete. 2024-09-17T04:28:42.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.412+0000 7f5f6e800640 1 Processor -- start 2024-09-17T04:28:42.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.412+0000 7f5f6e800640 1 -- start start 2024-09-17T04:28:42.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.413+0000 7f5f6e800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f68145550 0x7f5f6815b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:42.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.413+0000 7f5f6e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f68145a40 0x7f5f680b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:42.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.413+0000 7f5f6e800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f6815fc80 0x7f5f680b4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:42.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.413+0000 7f5f6e800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5f680ce130 con 0x7f5f68145a40 2024-09-17T04:28:42.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.413+0000 7f5f6e800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5f680cdfb0 con 0x7f5f6815fc80 2024-09-17T04:28:42.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.413+0000 7f5f6e800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5f680ce2b0 con 0x7f5f68145550 2024-09-17T04:28:42.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.413+0000 7f5f66a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f68145a40 0x7f5f680b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:42.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.414+0000 7f5f66a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f68145a40 0x7f5f680b3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49618/0 (socket says 172.21.15.100:49618) 2024-09-17T04:28:42.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.414+0000 7f5f66a00640 1 -- 172.21.15.100:0/1443508785 learned_addr learned my addr 172.21.15.100:0/1443508785 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:42.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.414+0000 7f5f67400640 1 --2- 172.21.15.100:0/1443508785 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f68145550 0x7f5f6815b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:42.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.414+0000 7f5f67e00640 1 --2- 172.21.15.100:0/1443508785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f6815fc80 0x7f5f680b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:42.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.414+0000 7f5f66a00640 1 -- 172.21.15.100:0/1443508785 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f68145550 msgr2=0x7f5f6815b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:42.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.414+0000 7f5f66a00640 1 --2- 172.21.15.100:0/1443508785 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f68145550 0x7f5f6815b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:42.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.414+0000 7f5f66a00640 1 -- 172.21.15.100:0/1443508785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f6815fc80 msgr2=0x7f5f680b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:42.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.414+0000 7f5f66a00640 1 --2- 172.21.15.100:0/1443508785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f6815fc80 0x7f5f680b4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:42.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.414+0000 7f5f66a00640 1 -- 172.21.15.100:0/1443508785 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f680b24b0 con 0x7f5f68145a40 2024-09-17T04:28:42.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.414+0000 7f5f67400640 1 --2- 172.21.15.100:0/1443508785 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f68145550 0x7f5f6815b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:28:42.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.414+0000 7f5f66a00640 1 --2- 172.21.15.100:0/1443508785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f68145a40 0x7f5f680b3ec0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400c390 tx=0x7f5f5400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:42.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.415+0000 7f5f5f400640 1 -- 172.21.15.100:0/1443508785 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54017070 con 0x7f5f68145a40 2024-09-17T04:28:42.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.415+0000 7f5f5f400640 1 -- 172.21.15.100:0/1443508785 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f540035f0 con 0x7f5f68145a40 2024-09-17T04:28:42.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.415+0000 7f5f5f400640 1 -- 172.21.15.100:0/1443508785 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54003a20 con 0x7f5f68145a40 2024-09-17T04:28:42.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.415+0000 7f5f6e800640 1 -- 172.21.15.100:0/1443508785 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f6815bdf0 con 0x7f5f68145a40 2024-09-17T04:28:42.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.415+0000 7f5f6e800640 1 -- 172.21.15.100:0/1443508785 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f681f9390 con 0x7f5f68145a40 2024-09-17T04:28:42.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.416+0000 7f5f6e800640 1 -- 172.21.15.100:0/1443508785 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f68160090 con 0x7f5f68145a40 2024-09-17T04:28:42.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.418+0000 7f5f5f400640 1 -- 172.21.15.100:0/1443508785 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f5f5401a070 con 0x7f5f68145a40 2024-09-17T04:28:42.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.418+0000 7f5f5f400640 1 --2- 172.21.15.100:0/1443508785 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5f48078ef0 0x7f5f4807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:42.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.419+0000 7f5f5f400640 1 -- 172.21.15.100:0/1443508785 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5227+0+0 (secure 0 0 0) 0x7f5f5409a820 con 0x7f5f68145a40 2024-09-17T04:28:42.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.419+0000 7f5f67400640 1 --2- 172.21.15.100:0/1443508785 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5f48078ef0 0x7f5f4807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:42.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.420+0000 7f5f67400640 1 --2- 172.21.15.100:0/1443508785 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5f48078ef0 0x7f5f4807b400 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5f58033000 tx=0x7f5f58004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:42.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.422+0000 7f5f5f400640 1 -- 172.21.15.100:0/1443508785 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f5f54021020 con 0x7f5f68145a40 2024-09-17T04:28:42.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:42.607+0000 7f5f6e800640 1 -- 172.21.15.100:0/1443508785 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5f680ce7f0 con 0x7f5f68145a40 2024-09-17T04:28:43.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.151+0000 7f5f5f400640 1 -- 172.21.15.100:0/1443508785 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v71) ==== 170+0+0 (secure 0 0 0) 0x7f5f54062ab0 con 0x7f5f68145a40 2024-09-17T04:28:43.152 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' created 2024-09-17T04:28:43.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.155+0000 7f5f6e800640 1 -- 172.21.15.100:0/1443508785 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5f48078ef0 msgr2=0x7f5f4807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:43.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.155+0000 7f5f6e800640 1 --2- 172.21.15.100:0/1443508785 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5f48078ef0 0x7f5f4807b400 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5f58033000 tx=0x7f5f58004730 comp rx=0 tx=0).stop 2024-09-17T04:28:43.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.155+0000 7f5f6e800640 1 -- 172.21.15.100:0/1443508785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f68145a40 msgr2=0x7f5f680b3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:43.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.155+0000 7f5f6e800640 1 --2- 172.21.15.100:0/1443508785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f68145a40 0x7f5f680b3ec0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400c390 tx=0x7f5f5400c850 comp rx=0 tx=0).stop 2024-09-17T04:28:43.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.155+0000 7f5f6e800640 1 -- 172.21.15.100:0/1443508785 shutdown_connections 2024-09-17T04:28:43.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.155+0000 7f5f6e800640 1 --2- 172.21.15.100:0/1443508785 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5f48078ef0 0x7f5f4807b400 secure :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5f58033000 tx=0x7f5f58004730 comp rx=0 tx=0).stop 2024-09-17T04:28:43.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.155+0000 7f5f6e800640 1 --2- 172.21.15.100:0/1443508785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f6815fc80 0x7f5f680b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:43.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.155+0000 7f5f6e800640 1 --2- 172.21.15.100:0/1443508785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f68145a40 0x7f5f680b3ec0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:43.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.155+0000 7f5f6e800640 1 --2- 172.21.15.100:0/1443508785 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f68145550 0x7f5f6815b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:43.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.156+0000 7f5f6e800640 1 -- 172.21.15.100:0/1443508785 >> 172.21.15.100:0/1443508785 conn(0x7f5f6806d1c0 msgr2=0x7f5f680cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:43.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.156+0000 7f5f6e800640 1 -- 172.21.15.100:0/1443508785 shutdown_connections 2024-09-17T04:28:43.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.156+0000 7f5f6e800640 1 -- 172.21.15.100:0/1443508785 wait complete. 2024-09-17T04:28:43.191 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T04:28:43.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.415+0000 7fa1d9800640 1 -- 172.21.15.100:0/2896698617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1b8003c30 msgr2=0x7fa1b8004080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:43.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.415+0000 7fa1d9800640 1 --2- 172.21.15.100:0/2896698617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1b8003c30 0x7fa1b8004080 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4001830 tx=0x7fa1b8005830 comp rx=0 tx=0).stop 2024-09-17T04:28:43.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.415+0000 7fa1d9800640 1 -- 172.21.15.100:0/2896698617 shutdown_connections 2024-09-17T04:28:43.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.415+0000 7fa1d9800640 1 --2- 172.21.15.100:0/2896698617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1b8003c30 0x7fa1b8004080 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:43.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.415+0000 7fa1d9800640 1 -- 172.21.15.100:0/2896698617 >> 172.21.15.100:0/2896698617 conn(0x7fa1d406d1c0 msgr2=0x7fa1d406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:43.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.415+0000 7fa1d9800640 1 -- 172.21.15.100:0/2896698617 shutdown_connections 2024-09-17T04:28:43.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.415+0000 7fa1d9800640 1 -- 172.21.15.100:0/2896698617 wait complete. 2024-09-17T04:28:43.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.418+0000 7fa1d9800640 1 Processor -- start 2024-09-17T04:28:43.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.418+0000 7fa1d9800640 1 -- start start 2024-09-17T04:28:43.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.419+0000 7fa1d9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1b8003c30 0x7fa1d41f71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:43.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.419+0000 7fa1d9800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1d40adb70 0x7fa1d41f0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:43.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.419+0000 7fa1d9800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1d41433a0 0x7fa1d41437b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:43.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.419+0000 7fa1d9800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d4073540 con 0x7fa1b8003c30 2024-09-17T04:28:43.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.419+0000 7fa1d9800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d4070130 con 0x7fa1d41433a0 2024-09-17T04:28:43.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.419+0000 7fa1d9800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa1d40736c0 con 0x7fa1d40adb70 2024-09-17T04:28:43.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.419+0000 7fa1d2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1b8003c30 0x7fa1d41f71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:43.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.419+0000 7fa1d2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1b8003c30 0x7fa1d41f71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49642/0 (socket says 172.21.15.100:49642) 2024-09-17T04:28:43.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.419+0000 7fa1d2a00640 1 -- 172.21.15.100:0/447335799 learned_addr learned my addr 172.21.15.100:0/447335799 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:43.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.420+0000 7fa1ca000640 1 --2- 172.21.15.100:0/447335799 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1d40adb70 0x7fa1d41f0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:43.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.420+0000 7fa1d3400640 1 --2- 172.21.15.100:0/447335799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1d41433a0 0x7fa1d41437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:43.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.420+0000 7fa1d2a00640 1 -- 172.21.15.100:0/447335799 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1d40adb70 msgr2=0x7fa1d41f0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:43.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.420+0000 7fa1d2a00640 1 --2- 172.21.15.100:0/447335799 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1d40adb70 0x7fa1d41f0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:43.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.420+0000 7fa1d2a00640 1 -- 172.21.15.100:0/447335799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1d41433a0 msgr2=0x7fa1d41437b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:43.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.420+0000 7fa1d2a00640 1 --2- 172.21.15.100:0/447335799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1d41433a0 0x7fa1d41437b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:43.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.420+0000 7fa1d2a00640 1 -- 172.21.15.100:0/447335799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d41f7730 con 0x7fa1b8003c30 2024-09-17T04:28:43.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.420+0000 7fa1d3400640 1 --2- 172.21.15.100:0/447335799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1d41433a0 0x7fa1d41437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:28:43.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.420+0000 7fa1d2a00640 1 --2- 172.21.15.100:0/447335799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1b8003c30 0x7fa1d41f71f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4001920 tx=0x7fa1c40060e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:43.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.420+0000 7fa1cb400640 1 -- 172.21.15.100:0/447335799 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4043070 con 0x7fa1b8003c30 2024-09-17T04:28:43.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.420+0000 7fa1cb400640 1 -- 172.21.15.100:0/447335799 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1c4003af0 con 0x7fa1b8003c30 2024-09-17T04:28:43.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.420+0000 7fa1cb400640 1 -- 172.21.15.100:0/447335799 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c40064c0 con 0x7fa1b8003c30 2024-09-17T04:28:43.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.420+0000 7fa1d9800640 1 -- 172.21.15.100:0/447335799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d4143cf0 con 0x7fa1b8003c30 2024-09-17T04:28:43.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.421+0000 7fa1d9800640 1 -- 172.21.15.100:0/447335799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d4130ee0 con 0x7fa1b8003c30 2024-09-17T04:28:43.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.422+0000 7fa1d9800640 1 -- 172.21.15.100:0/447335799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1d415c800 con 0x7fa1b8003c30 2024-09-17T04:28:43.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.423+0000 7fa1cb400640 1 -- 172.21.15.100:0/447335799 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fa1c4033480 con 0x7fa1b8003c30 2024-09-17T04:28:43.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.424+0000 7fa1cb400640 1 --2- 172.21.15.100:0/447335799 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa1b0078ef0 0x7fa1b007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:43.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.424+0000 7fa1cb400640 1 -- 172.21.15.100:0/447335799 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5609+0+0 (secure 0 0 0) 0x7fa1c40bb3d0 con 0x7fa1b8003c30 2024-09-17T04:28:43.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.424+0000 7fa1ca000640 1 --2- 172.21.15.100:0/447335799 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa1b0078ef0 0x7fa1b007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:43.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.425+0000 7fa1ca000640 1 --2- 172.21.15.100:0/447335799 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa1b0078ef0 0x7fa1b007b400 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc001240 tx=0x7fa1bc001110 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:43.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.429+0000 7fa1cb400640 1 -- 172.21.15.100:0/447335799 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa1d4143cf0 con 0x7fa1b8003c30 2024-09-17T04:28:43.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:43.647+0000 7fa1d9800640 1 -- 172.21.15.100:0/447335799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa1d4073bb0 con 0x7fa1b8003c30 2024-09-17T04:28:44.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.167+0000 7fa1cb400640 1 -- 172.21.15.100:0/447335799 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v43) ==== 174+0+0 (secure 0 0 0) 0x7fa1d41f7730 con 0x7fa1b8003c30 2024-09-17T04:28:44.168 INFO:teuthology.orchestra.run.smithi100.stderr:new fs with metadata pool 10 and data pool 11 2024-09-17T04:28:44.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.174+0000 7fa1d9800640 1 -- 172.21.15.100:0/447335799 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa1b0078ef0 msgr2=0x7fa1b007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:44.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.174+0000 7fa1d9800640 1 --2- 172.21.15.100:0/447335799 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa1b0078ef0 0x7fa1b007b400 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc001240 tx=0x7fa1bc001110 comp rx=0 tx=0).stop 2024-09-17T04:28:44.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.174+0000 7fa1d9800640 1 -- 172.21.15.100:0/447335799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1b8003c30 msgr2=0x7fa1d41f71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:44.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.174+0000 7fa1d9800640 1 --2- 172.21.15.100:0/447335799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1b8003c30 0x7fa1d41f71f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4001920 tx=0x7fa1c40060e0 comp rx=0 tx=0).stop 2024-09-17T04:28:44.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.176+0000 7fa1d9800640 1 -- 172.21.15.100:0/447335799 shutdown_connections 2024-09-17T04:28:44.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.176+0000 7fa1d9800640 1 --2- 172.21.15.100:0/447335799 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fa1b0078ef0 0x7fa1b007b400 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:44.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.176+0000 7fa1d9800640 1 --2- 172.21.15.100:0/447335799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1d41433a0 0x7fa1d41437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:44.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.176+0000 7fa1d9800640 1 --2- 172.21.15.100:0/447335799 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1d40adb70 0x7fa1d41f0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:44.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.176+0000 7fa1d9800640 1 --2- 172.21.15.100:0/447335799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1b8003c30 0x7fa1d41f71f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:44.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.176+0000 7fa1d9800640 1 -- 172.21.15.100:0/447335799 >> 172.21.15.100:0/447335799 conn(0x7fa1d406d1c0 msgr2=0x7fa1d4144a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:44.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.178+0000 7fa1d9800640 1 -- 172.21.15.100:0/447335799 shutdown_connections 2024-09-17T04:28:44.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.178+0000 7fa1d9800640 1 -- 172.21.15.100:0/447335799 wait complete. 2024-09-17T04:28:44.219 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:28:44.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.439+0000 7fd660400640 1 -- 172.21.15.100:0/2313627931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd65815d760 msgr2=0x7fd6581535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:44.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.439+0000 7fd660400640 1 --2- 172.21.15.100:0/2313627931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd65815d760 0x7fd6581535c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd658072d60 tx=0x7fd64c02a560 comp rx=0 tx=0).stop 2024-09-17T04:28:44.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.440+0000 7fd660400640 1 -- 172.21.15.100:0/2313627931 shutdown_connections 2024-09-17T04:28:44.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.440+0000 7fd660400640 1 --2- 172.21.15.100:0/2313627931 >> v2:172.21.15.184:3300/0 conn(0x7fd6580ad0e0 0x7fd6580ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:44.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.440+0000 7fd660400640 1 --2- 172.21.15.100:0/2313627931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd65815d760 0x7fd6581535c0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:44.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.440+0000 7fd660400640 1 -- 172.21.15.100:0/2313627931 >> 172.21.15.100:0/2313627931 conn(0x7fd65806d730 msgr2=0x7fd6580b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:44.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.440+0000 7fd660400640 1 -- 172.21.15.100:0/2313627931 shutdown_connections 2024-09-17T04:28:44.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.440+0000 7fd660400640 1 -- 172.21.15.100:0/2313627931 wait complete. 2024-09-17T04:28:44.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.442+0000 7fd660400640 1 Processor -- start 2024-09-17T04:28:44.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.442+0000 7fd660400640 1 -- start start 2024-09-17T04:28:44.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.443+0000 7fd660400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6580ad0e0 0x7fd6581f98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:44.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.443+0000 7fd660400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd65815d760 0x7fd6581f6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:44.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.443+0000 7fd660400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd658160310 0x7fd65814cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:44.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.443+0000 7fd660400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd6580b4c20 con 0x7fd6580ad0e0 2024-09-17T04:28:44.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.443+0000 7fd660400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd658153c70 con 0x7fd658160310 2024-09-17T04:28:44.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.443+0000 7fd660400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd6580b4da0 con 0x7fd65815d760 2024-09-17T04:28:44.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.444+0000 7fd65da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6580ad0e0 0x7fd6581f98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:44.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.444+0000 7fd65da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6580ad0e0 0x7fd6581f98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49668/0 (socket says 172.21.15.100:49668) 2024-09-17T04:28:44.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.444+0000 7fd65da00640 1 -- 172.21.15.100:0/1951210677 learned_addr learned my addr 172.21.15.100:0/1951210677 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:44.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.444+0000 7fd65e400640 1 --2- 172.21.15.100:0/1951210677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd658160310 0x7fd65814cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:44.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.444+0000 7fd65d000640 1 --2- 172.21.15.100:0/1951210677 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd65815d760 0x7fd6581f6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:44.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.444+0000 7fd65da00640 1 -- 172.21.15.100:0/1951210677 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd65815d760 msgr2=0x7fd6581f6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:44.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.444+0000 7fd65da00640 1 --2- 172.21.15.100:0/1951210677 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd65815d760 0x7fd6581f6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:44.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.444+0000 7fd65da00640 1 -- 172.21.15.100:0/1951210677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd658160310 msgr2=0x7fd65814cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:44.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.444+0000 7fd65da00640 1 --2- 172.21.15.100:0/1951210677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd658160310 0x7fd65814cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:44.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.444+0000 7fd65da00640 1 -- 172.21.15.100:0/1951210677 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6581f68b0 con 0x7fd6580ad0e0 2024-09-17T04:28:44.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.444+0000 7fd65e400640 1 --2- 172.21.15.100:0/1951210677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd658160310 0x7fd65814cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:28:44.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.444+0000 7fd65d000640 1 --2- 172.21.15.100:0/1951210677 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd65815d760 0x7fd6581f6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:28:44.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.444+0000 7fd65da00640 1 --2- 172.21.15.100:0/1951210677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6580ad0e0 0x7fd6581f98f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd64c034040 tx=0x7fd64c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:44.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.445+0000 7fd655600640 1 -- 172.21.15.100:0/1951210677 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c03e030 con 0x7fd6580ad0e0 2024-09-17T04:28:44.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.445+0000 7fd655600640 1 -- 172.21.15.100:0/1951210677 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd64c003e00 con 0x7fd6580ad0e0 2024-09-17T04:28:44.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.445+0000 7fd655600640 1 -- 172.21.15.100:0/1951210677 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c035040 con 0x7fd6580ad0e0 2024-09-17T04:28:44.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.445+0000 7fd660400640 1 -- 172.21.15.100:0/1951210677 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd65809dc90 con 0x7fd6580ad0e0 2024-09-17T04:28:44.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.445+0000 7fd660400640 1 -- 172.21.15.100:0/1951210677 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6581f5b70 con 0x7fd6580ad0e0 2024-09-17T04:28:44.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.446+0000 7fd660400640 1 -- 172.21.15.100:0/1951210677 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd658160ff0 con 0x7fd6580ad0e0 2024-09-17T04:28:44.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.447+0000 7fd655600640 1 -- 172.21.15.100:0/1951210677 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fd64c041030 con 0x7fd6580ad0e0 2024-09-17T04:28:44.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.448+0000 7fd655600640 1 --2- 172.21.15.100:0/1951210677 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd638078ef0 0x7fd63807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:44.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.448+0000 7fd655600640 1 -- 172.21.15.100:0/1951210677 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7fd64c0bbb70 con 0x7fd6580ad0e0 2024-09-17T04:28:44.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.449+0000 7fd65d000640 1 --2- 172.21.15.100:0/1951210677 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd638078ef0 0x7fd63807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:44.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.449+0000 7fd65d000640 1 --2- 172.21.15.100:0/1951210677 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd638078ef0 0x7fd63807b400 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd644005d00 tx=0x7fd644005df0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:44.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.453+0000 7fd655600640 1 -- 172.21.15.100:0/1951210677 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd64c083ae0 con 0x7fd6580ad0e0 2024-09-17T04:28:44.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.636+0000 7fd660400640 1 -- 172.21.15.100:0/1951210677 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd65814d120 con 0x7fd6580ad0e0 2024-09-17T04:28:44.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.638+0000 7fd655600640 1 -- 172.21.15.100:0/1951210677 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+16789 (secure 0 0 0) 0x7fd64c08c020 con 0x7fd6580ad0e0 2024-09-17T04:28:44.638 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:28:44.639 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":72,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:28:43.649003+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:28:42.112814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-17T04:28:42.620048+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:28:44.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.640+0000 7fd660400640 1 -- 172.21.15.100:0/1951210677 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd638078ef0 msgr2=0x7fd63807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:44.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.640+0000 7fd660400640 1 --2- 172.21.15.100:0/1951210677 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd638078ef0 0x7fd63807b400 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd644005d00 tx=0x7fd644005df0 comp rx=0 tx=0).stop 2024-09-17T04:28:44.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.640+0000 7fd660400640 1 -- 172.21.15.100:0/1951210677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6580ad0e0 msgr2=0x7fd6581f98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:44.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.640+0000 7fd660400640 1 --2- 172.21.15.100:0/1951210677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6580ad0e0 0x7fd6581f98f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd64c034040 tx=0x7fd64c003910 comp rx=0 tx=0).stop 2024-09-17T04:28:44.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.640+0000 7fd660400640 1 -- 172.21.15.100:0/1951210677 shutdown_connections 2024-09-17T04:28:44.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.640+0000 7fd660400640 1 --2- 172.21.15.100:0/1951210677 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fd638078ef0 0x7fd63807b400 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:44.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.640+0000 7fd660400640 1 --2- 172.21.15.100:0/1951210677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd658160310 0x7fd65814cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:44.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.640+0000 7fd660400640 1 --2- 172.21.15.100:0/1951210677 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd65815d760 0x7fd6581f6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:44.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.640+0000 7fd660400640 1 --2- 172.21.15.100:0/1951210677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6580ad0e0 0x7fd6581f98f0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:44.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.640+0000 7fd660400640 1 -- 172.21.15.100:0/1951210677 >> 172.21.15.100:0/1951210677 conn(0x7fd65806d730 msgr2=0x7fd6580b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:44.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.640+0000 7fd660400640 1 -- 172.21.15.100:0/1951210677 shutdown_connections 2024-09-17T04:28:44.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.640+0000 7fd660400640 1 -- 172.21.15.100:0/1951210677 wait complete. 2024-09-17T04:28:44.666 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:28:44.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.886+0000 7fc75e200640 1 -- 172.21.15.100:0/777220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc73c005320 msgr2=0x7fc73c00b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:44.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.886+0000 7fc75e200640 1 --2- 172.21.15.100:0/777220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc73c005320 0x7fc73c00b9e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc75000c610 tx=0x7fc73c005830 comp rx=0 tx=0).stop 2024-09-17T04:28:44.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.886+0000 7fc75e200640 1 -- 172.21.15.100:0/777220 shutdown_connections 2024-09-17T04:28:44.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.886+0000 7fc75e200640 1 --2- 172.21.15.100:0/777220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc73c005320 0x7fc73c00b9e0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:44.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.886+0000 7fc75e200640 1 -- 172.21.15.100:0/777220 >> 172.21.15.100:0/777220 conn(0x7fc75806d730 msgr2=0x7fc7580b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:44.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.886+0000 7fc75e200640 1 -- 172.21.15.100:0/777220 shutdown_connections 2024-09-17T04:28:44.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.887+0000 7fc75e200640 1 -- 172.21.15.100:0/777220 wait complete. 2024-09-17T04:28:44.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.889+0000 7fc75e200640 1 Processor -- start 2024-09-17T04:28:44.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.889+0000 7fc75e200640 1 -- start start 2024-09-17T04:28:44.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.890+0000 7fc75e200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc73c005320 0x7fc75809d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:44.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.890+0000 7fc75e200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc75814f990 0x7fc75809de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:44.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.890+0000 7fc75e200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7581f91a0 0x7fc7581f95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:44.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.890+0000 7fc75e200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc7580ad4f0 con 0x7fc75814f990 2024-09-17T04:28:44.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.890+0000 7fc75e200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc7580ad670 con 0x7fc73c005320 2024-09-17T04:28:44.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.890+0000 7fc75e200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc7580ad7f0 con 0x7fc7581f91a0 2024-09-17T04:28:44.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.891+0000 7fc74ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc75814f990 0x7fc75809de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:44.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.891+0000 7fc74ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc75814f990 0x7fc75809de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38680/0 (socket says 172.21.15.100:38680) 2024-09-17T04:28:44.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.891+0000 7fc74ea00640 1 -- 172.21.15.100:0/26764531 learned_addr learned my addr 172.21.15.100:0/26764531 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:44.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.891+0000 7fc757400640 1 --2- 172.21.15.100:0/26764531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc73c005320 0x7fc75809d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:44.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.891+0000 7fc757e00640 1 --2- 172.21.15.100:0/26764531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7581f91a0 0x7fc7581f95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:44.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.891+0000 7fc74ea00640 1 -- 172.21.15.100:0/26764531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7581f91a0 msgr2=0x7fc7581f95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:44.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.891+0000 7fc74ea00640 1 --2- 172.21.15.100:0/26764531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7581f91a0 0x7fc7581f95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:44.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.891+0000 7fc74ea00640 1 -- 172.21.15.100:0/26764531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc73c005320 msgr2=0x7fc75809d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:44.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.891+0000 7fc74ea00640 1 --2- 172.21.15.100:0/26764531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc73c005320 0x7fc75809d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:44.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.891+0000 7fc74ea00640 1 -- 172.21.15.100:0/26764531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7581f6730 con 0x7fc75814f990 2024-09-17T04:28:44.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.891+0000 7fc757400640 1 --2- 172.21.15.100:0/26764531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc73c005320 0x7fc75809d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:28:44.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.891+0000 7fc757e00640 1 --2- 172.21.15.100:0/26764531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7581f91a0 0x7fc7581f95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:28:44.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.891+0000 7fc74ea00640 1 --2- 172.21.15.100:0/26764531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc75814f990 0x7fc75809de20 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc740003d80 tx=0x7fc740001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:44.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.892+0000 7fc74fe00640 1 -- 172.21.15.100:0/26764531 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc740011700 con 0x7fc75814f990 2024-09-17T04:28:44.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.892+0000 7fc74fe00640 1 -- 172.21.15.100:0/26764531 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc740016020 con 0x7fc75814f990 2024-09-17T04:28:44.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.892+0000 7fc74fe00640 1 -- 172.21.15.100:0/26764531 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc740011b40 con 0x7fc75814f990 2024-09-17T04:28:44.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.892+0000 7fc75e200640 1 -- 172.21.15.100:0/26764531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7580731e0 con 0x7fc75814f990 2024-09-17T04:28:44.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.892+0000 7fc75e200640 1 -- 172.21.15.100:0/26764531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7580737a0 con 0x7fc75814f990 2024-09-17T04:28:44.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.893+0000 7fc75e200640 1 -- 172.21.15.100:0/26764531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc758153730 con 0x7fc75814f990 2024-09-17T04:28:44.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.894+0000 7fc74fe00640 1 -- 172.21.15.100:0/26764531 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fc740011ce0 con 0x7fc75814f990 2024-09-17T04:28:44.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.895+0000 7fc74fe00640 1 --2- 172.21.15.100:0/26764531 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc734078ef0 0x7fc73407b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:44.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.895+0000 7fc74fe00640 1 -- 172.21.15.100:0/26764531 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7fc74009c1a0 con 0x7fc75814f990 2024-09-17T04:28:44.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.896+0000 7fc757400640 1 --2- 172.21.15.100:0/26764531 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc734078ef0 0x7fc73407b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:44.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.896+0000 7fc757400640 1 --2- 172.21.15.100:0/26764531 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc734078ef0 0x7fc73407b400 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc748009ae0 tx=0x7fc748005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:44.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:44.899+0000 7fc74fe00640 1 -- 172.21.15.100:0/26764531 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc740020020 con 0x7fc75814f990 2024-09-17T04:28:45.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.083+0000 7fc75e200640 1 -- 172.21.15.100:0/26764531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc75815fc10 con 0x7fc75814f990 2024-09-17T04:28:45.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.084+0000 7fc74fe00640 1 -- 172.21.15.100:0/26764531 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+16789 (secure 0 0 0) 0x7fc740064110 con 0x7fc75814f990 2024-09-17T04:28:45.085 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:28:45.085 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":72,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:28:43.649003+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:28:42.112814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-17T04:28:42.620048+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:28:45.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.087+0000 7fc75e200640 1 -- 172.21.15.100:0/26764531 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc734078ef0 msgr2=0x7fc73407b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:45.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.087+0000 7fc75e200640 1 --2- 172.21.15.100:0/26764531 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc734078ef0 0x7fc73407b400 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc748009ae0 tx=0x7fc748005f90 comp rx=0 tx=0).stop 2024-09-17T04:28:45.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.087+0000 7fc75e200640 1 -- 172.21.15.100:0/26764531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc75814f990 msgr2=0x7fc75809de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:45.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.087+0000 7fc75e200640 1 --2- 172.21.15.100:0/26764531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc75814f990 0x7fc75809de20 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc740003d80 tx=0x7fc740001800 comp rx=0 tx=0).stop 2024-09-17T04:28:45.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.087+0000 7fc75e200640 1 -- 172.21.15.100:0/26764531 shutdown_connections 2024-09-17T04:28:45.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.087+0000 7fc75e200640 1 --2- 172.21.15.100:0/26764531 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc734078ef0 0x7fc73407b400 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:45.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.087+0000 7fc75e200640 1 --2- 172.21.15.100:0/26764531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7581f91a0 0x7fc7581f95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:45.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.087+0000 7fc75e200640 1 --2- 172.21.15.100:0/26764531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc75814f990 0x7fc75809de20 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:45.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.087+0000 7fc75e200640 1 --2- 172.21.15.100:0/26764531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc73c005320 0x7fc75809d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:45.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.087+0000 7fc75e200640 1 -- 172.21.15.100:0/26764531 >> 172.21.15.100:0/26764531 conn(0x7fc75806d730 msgr2=0x7fc758152b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:45.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.087+0000 7fc75e200640 1 -- 172.21.15.100:0/26764531 shutdown_connections 2024-09-17T04:28:45.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.087+0000 7fc75e200640 1 -- 172.21.15.100:0/26764531 wait complete. 2024-09-17T04:28:45.114 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T04:28:45.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.338+0000 7f1534c00640 1 -- 172.21.15.100:0/3262721415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1514003c10 msgr2=0x7f1514004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:45.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.338+0000 7f1534c00640 1 --2- 172.21.15.100:0/3262721415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1514003c10 0x7f1514004060 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1520006f90 tx=0x7f1514005830 comp rx=0 tx=0).stop 2024-09-17T04:28:45.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.338+0000 7f152fe00640 1 -- 172.21.15.100:0/3262721415 reap_dead start 2024-09-17T04:28:45.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.338+0000 7f1534c00640 1 -- 172.21.15.100:0/3262721415 shutdown_connections 2024-09-17T04:28:45.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.338+0000 7f1534c00640 1 -- 172.21.15.100:0/3262721415 >> 172.21.15.100:0/3262721415 conn(0x7f153006d620 msgr2=0x7f15300b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:45.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.338+0000 7f1534c00640 1 -- 172.21.15.100:0/3262721415 shutdown_connections 2024-09-17T04:28:45.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.338+0000 7f1534c00640 1 -- 172.21.15.100:0/3262721415 wait complete. 2024-09-17T04:28:45.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.341+0000 7f1534c00640 1 Processor -- start 2024-09-17T04:28:45.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.341+0000 7f1534c00640 1 -- start start 2024-09-17T04:28:45.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.342+0000 7f1534c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1514003c10 0x7f1530074390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:45.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.342+0000 7f1534c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15300748d0 0x7f153009d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:45.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.342+0000 7f1534c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f153009d820 0x7f1530072eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:45.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.342+0000 7f1534c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f15300703b0 con 0x7f153009d820 2024-09-17T04:28:45.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.342+0000 7f1534c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f15301571f0 con 0x7f15300748d0 2024-09-17T04:28:45.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.342+0000 7f1534c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f15300ccac0 con 0x7f1514003c10 2024-09-17T04:28:45.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.342+0000 7f152fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f153009d820 0x7f1530072eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:45.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.343+0000 7f152fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f153009d820 0x7f1530072eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38688/0 (socket says 172.21.15.100:38688) 2024-09-17T04:28:45.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.343+0000 7f152fe00640 1 -- 172.21.15.100:0/1984327643 learned_addr learned my addr 172.21.15.100:0/1984327643 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:45.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.343+0000 7f152f400640 1 --2- 172.21.15.100:0/1984327643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1514003c10 0x7f1530074390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:45.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.343+0000 7f1527e00640 1 --2- 172.21.15.100:0/1984327643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15300748d0 0x7f153009d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:45.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.343+0000 7f152fe00640 1 -- 172.21.15.100:0/1984327643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1514003c10 msgr2=0x7f1530074390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:45.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.343+0000 7f152fe00640 1 --2- 172.21.15.100:0/1984327643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1514003c10 0x7f1530074390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:45.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.343+0000 7f152fe00640 1 -- 172.21.15.100:0/1984327643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15300748d0 msgr2=0x7f153009d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:45.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.343+0000 7f152fe00640 1 --2- 172.21.15.100:0/1984327643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15300748d0 0x7f153009d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:45.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.343+0000 7f152fe00640 1 -- 172.21.15.100:0/1984327643 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153009dc30 con 0x7f153009d820 2024-09-17T04:28:45.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.343+0000 7f1527e00640 1 --2- 172.21.15.100:0/1984327643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15300748d0 0x7f153009d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:28:45.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.343+0000 7f152fe00640 1 --2- 172.21.15.100:0/1984327643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f153009d820 0x7f1530072eb0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1530071350 tx=0x7f1528001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:45.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.343+0000 7f1527400640 1 -- 172.21.15.100:0/1984327643 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1528017070 con 0x7f153009d820 2024-09-17T04:28:45.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.343+0000 7f1527400640 1 -- 172.21.15.100:0/1984327643 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15280109a0 con 0x7f153009d820 2024-09-17T04:28:45.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.343+0000 7f1527400640 1 -- 172.21.15.100:0/1984327643 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1528010df0 con 0x7f153009d820 2024-09-17T04:28:45.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.343+0000 7f1534c00640 1 -- 172.21.15.100:0/1984327643 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1530157370 con 0x7f153009d820 2024-09-17T04:28:45.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.344+0000 7f1534c00640 1 -- 172.21.15.100:0/1984327643 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15301f9100 con 0x7f153009d820 2024-09-17T04:28:45.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.345+0000 7f1534c00640 1 -- 172.21.15.100:0/1984327643 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1530157540 con 0x7f153009d820 2024-09-17T04:28:45.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.346+0000 7f1527400640 1 -- 172.21.15.100:0/1984327643 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f152801a070 con 0x7f153009d820 2024-09-17T04:28:45.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.347+0000 7f1527400640 1 --2- 172.21.15.100:0/1984327643 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f150c078ef0 0x7f150c07b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:45.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.347+0000 7f1527400640 1 -- 172.21.15.100:0/1984327643 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7f152809b210 con 0x7f153009d820 2024-09-17T04:28:45.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.348+0000 7f152f400640 1 --2- 172.21.15.100:0/1984327643 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f150c078ef0 0x7f150c07b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:45.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.349+0000 7f152f400640 1 --2- 172.21.15.100:0/1984327643 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f150c078ef0 0x7f150c07b400 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1520033000 tx=0x7f15300b4930 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:45.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.352+0000 7f1527400640 1 -- 172.21.15.100:0/1984327643 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f153009dc30 con 0x7f153009d820 2024-09-17T04:28:45.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:45.574+0000 7f1534c00640 1 -- 172.21.15.100:0/1984327643 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1530074e30 con 0x7f153009d820 2024-09-17T04:28:46.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.181+0000 7f1527400640 1 -- 172.21.15.100:0/1984327643 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v46) ==== 118+0+0 (secure 0 0 0) 0x7f1528062ae0 con 0x7f153009d820 2024-09-17T04:28:46.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.184+0000 7f1534c00640 1 -- 172.21.15.100:0/1984327643 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f150c078ef0 msgr2=0x7f150c07b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:46.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.184+0000 7f1534c00640 1 --2- 172.21.15.100:0/1984327643 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f150c078ef0 0x7f150c07b400 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1520033000 tx=0x7f15300b4930 comp rx=0 tx=0).stop 2024-09-17T04:28:46.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.184+0000 7f1534c00640 1 -- 172.21.15.100:0/1984327643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f153009d820 msgr2=0x7f1530072eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:46.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.184+0000 7f1534c00640 1 --2- 172.21.15.100:0/1984327643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f153009d820 0x7f1530072eb0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1530071350 tx=0x7f1528001b80 comp rx=0 tx=0).stop 2024-09-17T04:28:46.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.184+0000 7f1534c00640 1 -- 172.21.15.100:0/1984327643 shutdown_connections 2024-09-17T04:28:46.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.184+0000 7f1534c00640 1 --2- 172.21.15.100:0/1984327643 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f150c078ef0 0x7f150c07b400 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:46.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.184+0000 7f1534c00640 1 --2- 172.21.15.100:0/1984327643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f153009d820 0x7f1530072eb0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:46.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.184+0000 7f1534c00640 1 --2- 172.21.15.100:0/1984327643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15300748d0 0x7f153009d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:46.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.184+0000 7f1534c00640 1 --2- 172.21.15.100:0/1984327643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1514003c10 0x7f1530074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:46.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.184+0000 7f1534c00640 1 -- 172.21.15.100:0/1984327643 >> 172.21.15.100:0/1984327643 conn(0x7f153006d620 msgr2=0x7f15300d01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:46.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.184+0000 7f1534c00640 1 -- 172.21.15.100:0/1984327643 shutdown_connections 2024-09-17T04:28:46.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.184+0000 7f1534c00640 1 -- 172.21.15.100:0/1984327643 wait complete. 2024-09-17T04:28:46.208 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T04:28:46.208 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T04:28:46.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.433+0000 7feb46200640 1 -- 172.21.15.100:0/2202454900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb24004800 msgr2=0x7feb24004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:46.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.433+0000 7feb46200640 1 --2- 172.21.15.100:0/2202454900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb24004800 0x7feb24004c10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7feb401613e0 tx=0x7feb24005830 comp rx=0 tx=0).stop 2024-09-17T04:28:46.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.433+0000 7feb3fe00640 1 -- 172.21.15.100:0/2202454900 reap_dead start 2024-09-17T04:28:46.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.433+0000 7feb46200640 1 -- 172.21.15.100:0/2202454900 shutdown_connections 2024-09-17T04:28:46.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.433+0000 7feb46200640 1 -- 172.21.15.100:0/2202454900 >> 172.21.15.100:0/2202454900 conn(0x7feb4006d1c0 msgr2=0x7feb4006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:46.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.433+0000 7feb46200640 1 -- 172.21.15.100:0/2202454900 shutdown_connections 2024-09-17T04:28:46.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.433+0000 7feb46200640 1 -- 172.21.15.100:0/2202454900 wait complete. 2024-09-17T04:28:46.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.436+0000 7feb46200640 1 Processor -- start 2024-09-17T04:28:46.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.436+0000 7feb46200640 1 -- start start 2024-09-17T04:28:46.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.437+0000 7feb46200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb24004800 0x7feb40143310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:46.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.437+0000 7feb46200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb400b2ae0 0x7feb40143850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:46.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.437+0000 7feb46200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb4013eb80 0x7feb4013ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:46.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.437+0000 7feb46200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7feb400cdea0 con 0x7feb400b2ae0 2024-09-17T04:28:46.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.437+0000 7feb46200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feb400cdd20 con 0x7feb24004800 2024-09-17T04:28:46.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.437+0000 7feb46200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7feb400ce020 con 0x7feb4013eb80 2024-09-17T04:28:46.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.437+0000 7feb3ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb400b2ae0 0x7feb40143850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:46.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.437+0000 7feb3ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb400b2ae0 0x7feb40143850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38710/0 (socket says 172.21.15.100:38710) 2024-09-17T04:28:46.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.437+0000 7feb3ea00640 1 -- 172.21.15.100:0/3771099785 learned_addr learned my addr 172.21.15.100:0/3771099785 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:46.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.438+0000 7feb3fe00640 1 --2- 172.21.15.100:0/3771099785 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb4013eb80 0x7feb4013ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:46.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.438+0000 7feb3f400640 1 --2- 172.21.15.100:0/3771099785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb24004800 0x7feb40143310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:46.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.438+0000 7feb3ea00640 1 -- 172.21.15.100:0/3771099785 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb4013eb80 msgr2=0x7feb4013ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:46.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.438+0000 7feb3ea00640 1 --2- 172.21.15.100:0/3771099785 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb4013eb80 0x7feb4013ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:46.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.438+0000 7feb3ea00640 1 -- 172.21.15.100:0/3771099785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb24004800 msgr2=0x7feb40143310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:46.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.438+0000 7feb3ea00640 1 --2- 172.21.15.100:0/3771099785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb24004800 0x7feb40143310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:46.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.438+0000 7feb3ea00640 1 -- 172.21.15.100:0/3771099785 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb4013f4d0 con 0x7feb400b2ae0 2024-09-17T04:28:46.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.438+0000 7feb3fe00640 1 --2- 172.21.15.100:0/3771099785 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb4013eb80 0x7feb4013ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:28:46.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.438+0000 7feb3f400640 1 --2- 172.21.15.100:0/3771099785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb24004800 0x7feb40143310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:28:46.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.438+0000 7feb3ea00640 1 --2- 172.21.15.100:0/3771099785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb400b2ae0 0x7feb40143850 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7feb40072330 tx=0x7feb2c006310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:46.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.438+0000 7feb37400640 1 -- 172.21.15.100:0/3771099785 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c041070 con 0x7feb400b2ae0 2024-09-17T04:28:46.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.439+0000 7feb37400640 1 -- 172.21.15.100:0/3771099785 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb2c003af0 con 0x7feb400b2ae0 2024-09-17T04:28:46.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.439+0000 7feb46200640 1 -- 172.21.15.100:0/3771099785 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb40141cd0 con 0x7feb400b2ae0 2024-09-17T04:28:46.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.439+0000 7feb37400640 1 -- 172.21.15.100:0/3771099785 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c038070 con 0x7feb400b2ae0 2024-09-17T04:28:46.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.439+0000 7feb46200640 1 -- 172.21.15.100:0/3771099785 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb40142290 con 0x7feb400b2ae0 2024-09-17T04:28:46.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.440+0000 7feb46200640 1 -- 172.21.15.100:0/3771099785 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb400b3580 con 0x7feb400b2ae0 2024-09-17T04:28:46.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.441+0000 7feb37400640 1 -- 172.21.15.100:0/3771099785 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7feb2c044070 con 0x7feb400b2ae0 2024-09-17T04:28:46.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.442+0000 7feb37400640 1 --2- 172.21.15.100:0/3771099785 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7feb20078ef0 0x7feb2007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:46.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.442+0000 7feb37400640 1 -- 172.21.15.100:0/3771099785 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7feb2c0bad60 con 0x7feb400b2ae0 2024-09-17T04:28:46.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.442+0000 7feb3f400640 1 --2- 172.21.15.100:0/3771099785 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7feb20078ef0 0x7feb2007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:46.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.443+0000 7feb3f400640 1 --2- 172.21.15.100:0/3771099785 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7feb20078ef0 0x7feb2007b400 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7feb30009ac0 tx=0x7feb30005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:46.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.446+0000 7feb37400640 1 -- 172.21.15.100:0/3771099785 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7feb400b3580 con 0x7feb400b2ae0 2024-09-17T04:28:46.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:46.663+0000 7feb46200640 1 -- 172.21.15.100:0/3771099785 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7feb400ce580 con 0x7feb400b2ae0 2024-09-17T04:28:47.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.188+0000 7feb37400640 1 -- 172.21.15.100:0/3771099785 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v47) ==== 122+0+0 (secure 0 0 0) 0x7feb4013f4d0 con 0x7feb400b2ae0 2024-09-17T04:28:47.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.190+0000 7feb46200640 1 -- 172.21.15.100:0/3771099785 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7feb20078ef0 msgr2=0x7feb2007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:47.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.190+0000 7feb46200640 1 --2- 172.21.15.100:0/3771099785 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7feb20078ef0 0x7feb2007b400 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7feb30009ac0 tx=0x7feb30005f90 comp rx=0 tx=0).stop 2024-09-17T04:28:47.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.190+0000 7feb46200640 1 -- 172.21.15.100:0/3771099785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb400b2ae0 msgr2=0x7feb40143850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:47.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.190+0000 7feb46200640 1 --2- 172.21.15.100:0/3771099785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb400b2ae0 0x7feb40143850 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7feb40072330 tx=0x7feb2c006310 comp rx=0 tx=0).stop 2024-09-17T04:28:47.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.191+0000 7feb46200640 1 -- 172.21.15.100:0/3771099785 shutdown_connections 2024-09-17T04:28:47.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.191+0000 7feb46200640 1 --2- 172.21.15.100:0/3771099785 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7feb20078ef0 0x7feb2007b400 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:47.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.191+0000 7feb46200640 1 --2- 172.21.15.100:0/3771099785 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb4013eb80 0x7feb4013ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:47.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.191+0000 7feb46200640 1 --2- 172.21.15.100:0/3771099785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb400b2ae0 0x7feb40143850 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:47.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.191+0000 7feb46200640 1 --2- 172.21.15.100:0/3771099785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb24004800 0x7feb40143310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:47.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.191+0000 7feb46200640 1 -- 172.21.15.100:0/3771099785 >> 172.21.15.100:0/3771099785 conn(0x7feb4006d1c0 msgr2=0x7feb400b4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:47.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.191+0000 7feb46200640 1 -- 172.21.15.100:0/3771099785 shutdown_connections 2024-09-17T04:28:47.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.191+0000 7feb46200640 1 -- 172.21.15.100:0/3771099785 wait complete. 2024-09-17T04:28:47.215 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T04:28:47.215 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:28:47.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.434+0000 7f80c3200640 1 -- 172.21.15.100:0/2077411570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80bc15f910 msgr2=0x7f80bc0b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:47.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.434+0000 7f80c3200640 1 --2- 172.21.15.100:0/2077411570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80bc15f910 0x7f80bc0b3750 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f80b4005fd0 tx=0x7f80b402a960 comp rx=0 tx=0).stop 2024-09-17T04:28:47.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.435+0000 7f80c3200640 1 -- 172.21.15.100:0/2077411570 shutdown_connections 2024-09-17T04:28:47.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.435+0000 7f80c3200640 1 --2- 172.21.15.100:0/2077411570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80bc15f910 0x7f80bc0b3750 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:47.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.435+0000 7f80c3200640 1 --2- 172.21.15.100:0/2077411570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80bc152920 0x7f80bc15f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:47.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.435+0000 7f80c3200640 1 -- 172.21.15.100:0/2077411570 >> 172.21.15.100:0/2077411570 conn(0x7f80bc06d730 msgr2=0x7f80bc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:47.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.435+0000 7f80c3200640 1 -- 172.21.15.100:0/2077411570 shutdown_connections 2024-09-17T04:28:47.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.435+0000 7f80c3200640 1 -- 172.21.15.100:0/2077411570 wait complete. 2024-09-17T04:28:47.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.437+0000 7f80c3200640 1 Processor -- start 2024-09-17T04:28:47.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.437+0000 7f80c3200640 1 -- start start 2024-09-17T04:28:47.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.438+0000 7f80c3200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80bc0aebc0 0x7f80bc1f6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:47.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.438+0000 7f80c3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80bc152920 0x7f80bc158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:47.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.438+0000 7f80c3200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80bc15f910 0x7f80bc158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:47.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.438+0000 7f80c3200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc0b41e0 con 0x7f80bc152920 2024-09-17T04:28:47.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.438+0000 7f80c3200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc0b4060 con 0x7f80bc15f910 2024-09-17T04:28:47.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.438+0000 7f80c3200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f80bc0b4360 con 0x7f80bc0aebc0 2024-09-17T04:28:47.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.439+0000 7f80bb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80bc152920 0x7f80bc158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:47.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.439+0000 7f80bb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80bc152920 0x7f80bc158650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38734/0 (socket says 172.21.15.100:38734) 2024-09-17T04:28:47.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.439+0000 7f80bb400640 1 -- 172.21.15.100:0/110472416 learned_addr learned my addr 172.21.15.100:0/110472416 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:47.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.439+0000 7f80bbe00640 1 --2- 172.21.15.100:0/110472416 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80bc0aebc0 0x7f80bc1f6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:47.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.439+0000 7f80c1200640 1 --2- 172.21.15.100:0/110472416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80bc15f910 0x7f80bc158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:47.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.439+0000 7f80bb400640 1 -- 172.21.15.100:0/110472416 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80bc0aebc0 msgr2=0x7f80bc1f6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:47.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.439+0000 7f80bb400640 1 --2- 172.21.15.100:0/110472416 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80bc0aebc0 0x7f80bc1f6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:47.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.439+0000 7f80bb400640 1 -- 172.21.15.100:0/110472416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80bc15f910 msgr2=0x7f80bc158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:47.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.439+0000 7f80bb400640 1 --2- 172.21.15.100:0/110472416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80bc15f910 0x7f80bc158b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:47.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.439+0000 7f80bb400640 1 -- 172.21.15.100:0/110472416 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80bc159420 con 0x7f80bc152920 2024-09-17T04:28:47.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.440+0000 7f80bb400640 1 --2- 172.21.15.100:0/110472416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80bc152920 0x7f80bc158650 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f80a800b410 tx=0x7f80a800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:47.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.440+0000 7f80b3e00640 1 -- 172.21.15.100:0/110472416 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8015020 con 0x7f80bc152920 2024-09-17T04:28:47.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.440+0000 7f80b3e00640 1 -- 172.21.15.100:0/110472416 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80a8011940 con 0x7f80bc152920 2024-09-17T04:28:47.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.440+0000 7f80b3e00640 1 -- 172.21.15.100:0/110472416 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8011d90 con 0x7f80bc152920 2024-09-17T04:28:47.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.440+0000 7f80c3200640 1 -- 172.21.15.100:0/110472416 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80bc0cd490 con 0x7f80bc152920 2024-09-17T04:28:47.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.440+0000 7f80c3200640 1 -- 172.21.15.100:0/110472416 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80bc0cda50 con 0x7f80bc152920 2024-09-17T04:28:47.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.441+0000 7f80c3200640 1 -- 172.21.15.100:0/110472416 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80bc159190 con 0x7f80bc152920 2024-09-17T04:28:47.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.442+0000 7f80b3e00640 1 -- 172.21.15.100:0/110472416 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f80a8018020 con 0x7f80bc152920 2024-09-17T04:28:47.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.443+0000 7f80b3e00640 1 --2- 172.21.15.100:0/110472416 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f809c078ef0 0x7f809c07b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:47.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.443+0000 7f80b3e00640 1 -- 172.21.15.100:0/110472416 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7f80a809b260 con 0x7f80bc152920 2024-09-17T04:28:47.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.443+0000 7f80bbe00640 1 --2- 172.21.15.100:0/110472416 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f809c078ef0 0x7f809c07b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:47.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.444+0000 7f80bbe00640 1 --2- 172.21.15.100:0/110472416 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f809c078ef0 0x7f809c07b400 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f80ac006fd0 tx=0x7f80ac009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:47.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.448+0000 7f80b3e00640 1 -- 172.21.15.100:0/110472416 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f80a801f020 con 0x7f80bc152920 2024-09-17T04:28:47.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.665+0000 7f80c3200640 1 -- 172.21.15.100:0/110472416 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80bc0b4950 con 0x7f80bc152920 2024-09-17T04:28:47.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.665+0000 7f80b3e00640 1 -- 172.21.15.100:0/110472416 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5237 (secure 0 0 0) 0x7f80a8062af0 con 0x7f80bc152920 2024-09-17T04:28:47.666 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:28:47.666 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":47,"btime":"2024-09-17T04:28:47:182892+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5513,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2569013403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2569013403},{"type":"v1","addr":"172.21.15.184:6835","nonce":2569013403}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5519,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3475403612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3475403612},{"type":"v1","addr":"172.21.15.184:6837","nonce":3475403612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6156,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2177833369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2177833369},{"type":"v1","addr":"172.21.15.100:6833","nonce":2177833369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:28:43.648961+0000","modified":"2024-09-17T04:28:46.665573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6162":{"gid":6162,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6835/2913442826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2913442826},{"type":"v1","addr":"172.21.15.100:6835","nonce":2913442826}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6162,"qdb_cluster":[6162]},"id":5}]} 2024-09-17T04:28:47.666 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 47 2024-09-17T04:28:47.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.668+0000 7f80c3200640 1 -- 172.21.15.100:0/110472416 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f809c078ef0 msgr2=0x7f809c07b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:47.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.668+0000 7f80c3200640 1 --2- 172.21.15.100:0/110472416 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f809c078ef0 0x7f809c07b400 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f80ac006fd0 tx=0x7f80ac009060 comp rx=0 tx=0).stop 2024-09-17T04:28:47.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.668+0000 7f80c3200640 1 -- 172.21.15.100:0/110472416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80bc152920 msgr2=0x7f80bc158650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:47.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.668+0000 7f80c3200640 1 --2- 172.21.15.100:0/110472416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80bc152920 0x7f80bc158650 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f80a800b410 tx=0x7f80a800b8d0 comp rx=0 tx=0).stop 2024-09-17T04:28:47.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.668+0000 7f80c3200640 1 -- 172.21.15.100:0/110472416 shutdown_connections 2024-09-17T04:28:47.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.668+0000 7f80c3200640 1 --2- 172.21.15.100:0/110472416 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f809c078ef0 0x7f809c07b400 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:47.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.668+0000 7f80c3200640 1 --2- 172.21.15.100:0/110472416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80bc15f910 0x7f80bc158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:47.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.668+0000 7f80c3200640 1 --2- 172.21.15.100:0/110472416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80bc152920 0x7f80bc158650 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:47.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.668+0000 7f80c3200640 1 --2- 172.21.15.100:0/110472416 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80bc0aebc0 0x7f80bc1f6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:47.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.668+0000 7f80c3200640 1 -- 172.21.15.100:0/110472416 >> 172.21.15.100:0/110472416 conn(0x7f80bc06d730 msgr2=0x7f80bc15f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:47.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.668+0000 7f80c3200640 1 -- 172.21.15.100:0/110472416 shutdown_connections 2024-09-17T04:28:47.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.668+0000 7f80c3200640 1 -- 172.21.15.100:0/110472416 wait complete. 2024-09-17T04:28:47.690 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:28:47.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.908+0000 7f2535e00640 1 -- 172.21.15.100:0/3382573893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25300ad0e0 msgr2=0x7f25300ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:47.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.908+0000 7f2535e00640 1 --2- 172.21.15.100:0/3382573893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25300ad0e0 0x7f25300ad4f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f2530075250 tx=0x7f251c02a4d0 comp rx=0 tx=0).stop 2024-09-17T04:28:47.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.908+0000 7f2535e00640 1 -- 172.21.15.100:0/3382573893 shutdown_connections 2024-09-17T04:28:47.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.908+0000 7f2535e00640 1 --2- 172.21.15.100:0/3382573893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25300ad0e0 0x7f25300ad4f0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:47.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.908+0000 7f2535e00640 1 --2- 172.21.15.100:0/3382573893 >> v2:172.21.15.184:3300/0 conn(0x7f253015d760 0x7f25301535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:47.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.908+0000 7f2535e00640 1 -- 172.21.15.100:0/3382573893 >> 172.21.15.100:0/3382573893 conn(0x7f253006d730 msgr2=0x7f25300b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:47.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.908+0000 7f2535e00640 1 -- 172.21.15.100:0/3382573893 shutdown_connections 2024-09-17T04:28:47.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.908+0000 7f2535e00640 1 -- 172.21.15.100:0/3382573893 wait complete. 2024-09-17T04:28:47.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.911+0000 7f2535e00640 1 Processor -- start 2024-09-17T04:28:47.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.911+0000 7f2535e00640 1 -- start start 2024-09-17T04:28:47.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.912+0000 7f2535e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25300ad0e0 0x7f25301515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:47.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.912+0000 7f2535e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f253015d760 0x7f25300b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:47.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.912+0000 7f2535e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2530160310 0x7f25300b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:47.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.912+0000 7f2535e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f25300b4c20 con 0x7f25300ad0e0 2024-09-17T04:28:47.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.912+0000 7f2535e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2530153c70 con 0x7f253015d760 2024-09-17T04:28:47.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.912+0000 7f2535e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f25300b4da0 con 0x7f2530160310 2024-09-17T04:28:47.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.912+0000 7f252ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25300ad0e0 0x7f25301515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:47.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.912+0000 7f252ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25300ad0e0 0x7f25301515e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38742/0 (socket says 172.21.15.100:38742) 2024-09-17T04:28:47.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.912+0000 7f252ea00640 1 -- 172.21.15.100:0/2757396804 learned_addr learned my addr 172.21.15.100:0/2757396804 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:47.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.913+0000 7f252f400640 1 --2- 172.21.15.100:0/2757396804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2530160310 0x7f25300b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:47.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.913+0000 7f252e000640 1 --2- 172.21.15.100:0/2757396804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f253015d760 0x7f25300b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:47.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.913+0000 7f252ea00640 1 -- 172.21.15.100:0/2757396804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2530160310 msgr2=0x7f25300b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:47.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.913+0000 7f252ea00640 1 --2- 172.21.15.100:0/2757396804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2530160310 0x7f25300b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:47.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.913+0000 7f252ea00640 1 -- 172.21.15.100:0/2757396804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f253015d760 msgr2=0x7f25300b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:47.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.913+0000 7f252ea00640 1 --2- 172.21.15.100:0/2757396804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f253015d760 0x7f25300b11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:47.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.913+0000 7f252ea00640 1 -- 172.21.15.100:0/2757396804 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2530154af0 con 0x7f25300ad0e0 2024-09-17T04:28:47.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.913+0000 7f252e000640 1 --2- 172.21.15.100:0/2757396804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f253015d760 0x7f25300b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:28:47.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.913+0000 7f252f400640 1 --2- 172.21.15.100:0/2757396804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2530160310 0x7f25300b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:28:47.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.913+0000 7f252ea00640 1 --2- 172.21.15.100:0/2757396804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25300ad0e0 0x7f25301515e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f252000c970 tx=0x7f252000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:47.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.913+0000 7f2527400640 1 -- 172.21.15.100:0/2757396804 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2520007bb0 con 0x7f25300ad0e0 2024-09-17T04:28:47.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.913+0000 7f2527400640 1 -- 172.21.15.100:0/2757396804 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2520007e40 con 0x7f25300ad0e0 2024-09-17T04:28:47.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.913+0000 7f2527400640 1 -- 172.21.15.100:0/2757396804 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252000f690 con 0x7f25300ad0e0 2024-09-17T04:28:47.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.913+0000 7f2535e00640 1 -- 172.21.15.100:0/2757396804 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2530154dd0 con 0x7f25300ad0e0 2024-09-17T04:28:47.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.913+0000 7f2535e00640 1 -- 172.21.15.100:0/2757396804 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2530155390 con 0x7f25300ad0e0 2024-09-17T04:28:47.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.915+0000 7f2535e00640 1 -- 172.21.15.100:0/2757396804 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2530161010 con 0x7f25300ad0e0 2024-09-17T04:28:47.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.916+0000 7f2527400640 1 -- 172.21.15.100:0/2757396804 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f252000f830 con 0x7f25300ad0e0 2024-09-17T04:28:47.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.917+0000 7f2527400640 1 --2- 172.21.15.100:0/2757396804 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2510078f20 0x7f251007b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:47.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.917+0000 7f2527400640 1 -- 172.21.15.100:0/2757396804 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7f252009bd10 con 0x7f25300ad0e0 2024-09-17T04:28:47.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.917+0000 7f252e000640 1 --2- 172.21.15.100:0/2757396804 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2510078f20 0x7f251007b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:47.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.918+0000 7f252e000640 1 --2- 172.21.15.100:0/2757396804 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2510078f20 0x7f251007b430 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f251c033000 tx=0x7f251c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:47.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:47.921+0000 7f2527400640 1 -- 172.21.15.100:0/2757396804 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f25200a1020 con 0x7f25300ad0e0 2024-09-17T04:28:48.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.107+0000 7f2535e00640 1 -- 172.21.15.100:0/2757396804 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f253015df20 con 0x7f25300ad0e0 2024-09-17T04:28:48.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.108+0000 7f2527400640 1 -- 172.21.15.100:0/2757396804 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+16776 (secure 0 0 0) 0x7f2520014030 con 0x7f25300ad0e0 2024-09-17T04:28:48.109 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:28:48.109 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":73,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:28:45.165779+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:28:42.112814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-17T04:28:42.620048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:28:48.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.110+0000 7f2535e00640 1 -- 172.21.15.100:0/2757396804 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2510078f20 msgr2=0x7f251007b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:48.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.110+0000 7f2535e00640 1 --2- 172.21.15.100:0/2757396804 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2510078f20 0x7f251007b430 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f251c033000 tx=0x7f251c004730 comp rx=0 tx=0).stop 2024-09-17T04:28:48.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.110+0000 7f2535e00640 1 -- 172.21.15.100:0/2757396804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25300ad0e0 msgr2=0x7f25301515e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:48.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.110+0000 7f2535e00640 1 --2- 172.21.15.100:0/2757396804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25300ad0e0 0x7f25301515e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f252000c970 tx=0x7f252000ce30 comp rx=0 tx=0).stop 2024-09-17T04:28:48.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.110+0000 7f2535e00640 1 -- 172.21.15.100:0/2757396804 shutdown_connections 2024-09-17T04:28:48.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.110+0000 7f2535e00640 1 --2- 172.21.15.100:0/2757396804 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2510078f20 0x7f251007b430 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:48.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.110+0000 7f2535e00640 1 --2- 172.21.15.100:0/2757396804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2530160310 0x7f25300b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:48.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.110+0000 7f2535e00640 1 --2- 172.21.15.100:0/2757396804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f253015d760 0x7f25300b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:48.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.111+0000 7f2535e00640 1 --2- 172.21.15.100:0/2757396804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25300ad0e0 0x7f25301515e0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:48.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.111+0000 7f2535e00640 1 -- 172.21.15.100:0/2757396804 >> 172.21.15.100:0/2757396804 conn(0x7f253006d730 msgr2=0x7f25300cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:48.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.111+0000 7f2535e00640 1 -- 172.21.15.100:0/2757396804 shutdown_connections 2024-09-17T04:28:48.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.111+0000 7f2535e00640 1 -- 172.21.15.100:0/2757396804 wait complete. 2024-09-17T04:28:48.132 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T04:28:48.132 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:28:48.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.351+0000 7f0b79800640 1 -- 172.21.15.100:0/2654505949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b740ad9c0 msgr2=0x7f0b740af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:48.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.351+0000 7f0b79800640 1 --2- 172.21.15.100:0/2654505949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b740ad9c0 0x7f0b740af170 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f0b740af6b0 tx=0x7f0b6402a560 comp rx=0 tx=0).stop 2024-09-17T04:28:48.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.351+0000 7f0b79800640 1 -- 172.21.15.100:0/2654505949 shutdown_connections 2024-09-17T04:28:48.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.351+0000 7f0b79800640 1 --2- 172.21.15.100:0/2654505949 >> v2:172.21.15.184:3300/0 conn(0x7f0b7415a0f0 0x7f0b7415d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:48.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.351+0000 7f0b79800640 1 --2- 172.21.15.100:0/2654505949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b740ad9c0 0x7f0b740af170 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:48.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.351+0000 7f0b79800640 1 -- 172.21.15.100:0/2654505949 >> 172.21.15.100:0/2654505949 conn(0x7f0b740b1e40 msgr2=0x7f0b740b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:48.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.351+0000 7f0b79800640 1 -- 172.21.15.100:0/2654505949 shutdown_connections 2024-09-17T04:28:48.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.351+0000 7f0b79800640 1 -- 172.21.15.100:0/2654505949 wait complete. 2024-09-17T04:28:48.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.353+0000 7f0b79800640 1 Processor -- start 2024-09-17T04:28:48.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.354+0000 7f0b79800640 1 -- start start 2024-09-17T04:28:48.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.355+0000 7f0b79800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b740ad9c0 0x7f0b7414a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:48.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.355+0000 7f0b79800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b740b2f60 0x7f0b74073d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:48.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.355+0000 7f0b79800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b7415a0f0 0x7f0b741f8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:48.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.355+0000 7f0b79800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7406f290 con 0x7f0b7415a0f0 2024-09-17T04:28:48.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.355+0000 7f0b79800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7415a5e0 con 0x7f0b740ad9c0 2024-09-17T04:28:48.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.355+0000 7f0b79800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0b7406f410 con 0x7f0b740b2f60 2024-09-17T04:28:48.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.355+0000 7f0b78e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b7415a0f0 0x7f0b741f8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:48.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.355+0000 7f0b78e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b7415a0f0 0x7f0b741f8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38758/0 (socket says 172.21.15.100:38758) 2024-09-17T04:28:48.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.356+0000 7f0b78e00640 1 -- 172.21.15.100:0/198179467 learned_addr learned my addr 172.21.15.100:0/198179467 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:48.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.356+0000 7f0b73e00640 1 --2- 172.21.15.100:0/198179467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b740ad9c0 0x7f0b7414a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:48.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.356+0000 7f0b73400640 1 --2- 172.21.15.100:0/198179467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b740b2f60 0x7f0b74073d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:48.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.356+0000 7f0b78e00640 1 -- 172.21.15.100:0/198179467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b740b2f60 msgr2=0x7f0b74073d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:48.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.356+0000 7f0b78e00640 1 --2- 172.21.15.100:0/198179467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b740b2f60 0x7f0b74073d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:48.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.356+0000 7f0b78e00640 1 -- 172.21.15.100:0/198179467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b740ad9c0 msgr2=0x7f0b7414a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:48.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.356+0000 7f0b78e00640 1 --2- 172.21.15.100:0/198179467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b740ad9c0 0x7f0b7414a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:48.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.356+0000 7f0b78e00640 1 -- 172.21.15.100:0/198179467 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b74149ef0 con 0x7f0b7415a0f0 2024-09-17T04:28:48.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.356+0000 7f0b73400640 1 --2- 172.21.15.100:0/198179467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b740b2f60 0x7f0b74073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:28:48.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.356+0000 7f0b78e00640 1 --2- 172.21.15.100:0/198179467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b7415a0f0 0x7f0b741f8ac0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00ea10 tx=0x7f0b6c00eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:48.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.356+0000 7f0b6be00640 1 -- 172.21.15.100:0/198179467 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c00cde0 con 0x7f0b7415a0f0 2024-09-17T04:28:48.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.357+0000 7f0b6be00640 1 -- 172.21.15.100:0/198179467 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b6c004c10 con 0x7f0b7415a0f0 2024-09-17T04:28:48.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.357+0000 7f0b79800640 1 -- 172.21.15.100:0/198179467 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b74074720 con 0x7f0b7415a0f0 2024-09-17T04:28:48.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.357+0000 7f0b6be00640 1 -- 172.21.15.100:0/198179467 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c005020 con 0x7f0b7415a0f0 2024-09-17T04:28:48.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.357+0000 7f0b79800640 1 -- 172.21.15.100:0/198179467 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b741579f0 con 0x7f0b7415a0f0 2024-09-17T04:28:48.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.358+0000 7f0b79800640 1 -- 172.21.15.100:0/198179467 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b74157800 con 0x7f0b7415a0f0 2024-09-17T04:28:48.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.359+0000 7f0b6be00640 1 -- 172.21.15.100:0/198179467 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f0b6c0051c0 con 0x7f0b7415a0f0 2024-09-17T04:28:48.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.360+0000 7f0b6be00640 1 --2- 172.21.15.100:0/198179467 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0b54078f20 0x7f0b5407b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:48.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.360+0000 7f0b6be00640 1 -- 172.21.15.100:0/198179467 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7f0b6c09c270 con 0x7f0b7415a0f0 2024-09-17T04:28:48.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.360+0000 7f0b73e00640 1 --2- 172.21.15.100:0/198179467 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0b54078f20 0x7f0b5407b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:48.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.361+0000 7f0b73e00640 1 --2- 172.21.15.100:0/198179467 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0b54078f20 0x7f0b5407b430 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0b64033000 tx=0x7f0b64004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:48.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.364+0000 7f0b6be00640 1 -- 172.21.15.100:0/198179467 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0b6c064050 con 0x7f0b7415a0f0 2024-09-17T04:28:48.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.544+0000 7f0b79800640 1 -- 172.21.15.100:0/198179467 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0b7406f930 con 0x7f0b54078f20 2024-09-17T04:28:48.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.549+0000 7f0b6be00640 1 -- 172.21.15.100:0/198179467 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248568 (secure 0 0 0) 0x7f0b7406f930 con 0x7f0b54078f20 2024-09-17T04:28:48.553 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:28:48.554 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:28:48.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.553+0000 7f0b79800640 1 -- 172.21.15.100:0/198179467 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0b54078f20 msgr2=0x7f0b5407b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:48.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.553+0000 7f0b79800640 1 --2- 172.21.15.100:0/198179467 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0b54078f20 0x7f0b5407b430 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0b64033000 tx=0x7f0b64004730 comp rx=0 tx=0).stop 2024-09-17T04:28:48.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.553+0000 7f0b79800640 1 -- 172.21.15.100:0/198179467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b7415a0f0 msgr2=0x7f0b741f8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:48.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.553+0000 7f0b79800640 1 --2- 172.21.15.100:0/198179467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b7415a0f0 0x7f0b741f8ac0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00ea10 tx=0x7f0b6c00eed0 comp rx=0 tx=0).stop 2024-09-17T04:28:48.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.553+0000 7f0b79800640 1 -- 172.21.15.100:0/198179467 shutdown_connections 2024-09-17T04:28:48.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.553+0000 7f0b79800640 1 --2- 172.21.15.100:0/198179467 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0b54078f20 0x7f0b5407b430 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:48.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.553+0000 7f0b79800640 1 --2- 172.21.15.100:0/198179467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b7415a0f0 0x7f0b741f8ac0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:48.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.553+0000 7f0b79800640 1 --2- 172.21.15.100:0/198179467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b740b2f60 0x7f0b74073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:48.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.553+0000 7f0b79800640 1 --2- 172.21.15.100:0/198179467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b740ad9c0 0x7f0b7414a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:48.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.553+0000 7f0b79800640 1 -- 172.21.15.100:0/198179467 >> 172.21.15.100:0/198179467 conn(0x7f0b740b1e40 msgr2=0x7f0b7415c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:48.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.553+0000 7f0b79800640 1 -- 172.21.15.100:0/198179467 shutdown_connections 2024-09-17T04:28:48.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.554+0000 7f0b79800640 1 -- 172.21.15.100:0/198179467 wait complete. 2024-09-17T04:28:48.576 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-09-17T04:28:48.041281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590458,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":105,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":43,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253180,"kb_used_data":7812,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721540,"statfs":{"total":773094113280,"available":772834856960,"internally_reserved":0,"allocated":7999488,"data_stored":4013546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":251055668},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.890116"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.178663+0000","last_change":"2024-09-17T04:28:44.178546+0000","last_active":"2024-09-17T04:28:44.178663+0000","last_peered":"2024-09-17T04:28:44.178663+0000","last_clean":"2024-09-17T04:28:44.178663+0000","last_became_active":"2024-09-17T04:28:44.178226+0000","last_became_peered":"2024-09-17T04:28:44.178226+0000","last_unstale":"2024-09-17T04:28:44.178663+0000","last_undegraded":"2024-09-17T04:28:44.178663+0000","last_fullsized":"2024-09-17T04:28:44.178663+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:32.865614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.176591+0000","last_change":"2024-09-17T04:28:44.176591+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.176591+0000","last_undegraded":"2024-09-17T04:28:44.176591+0000","last_fullsized":"2024-09-17T04:28:44.176591+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836882+0000","last_change":"2024-09-17T04:28:44.187901+0000","last_active":"2024-09-17T04:28:45.836882+0000","last_peered":"2024-09-17T04:28:45.836882+0000","last_clean":"2024-09-17T04:28:45.836882+0000","last_became_active":"2024-09-17T04:28:44.187552+0000","last_became_peered":"2024-09-17T04:28:44.187552+0000","last_unstale":"2024-09-17T04:28:45.836882+0000","last_undegraded":"2024-09-17T04:28:45.836882+0000","last_fullsized":"2024-09-17T04:28:45.836882+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:33.519054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.158403+0000","last_change":"2024-09-17T04:28:43.158403+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.158403+0000","last_undegraded":"2024-09-17T04:28:43.158403+0000","last_fullsized":"2024-09-17T04:28:43.158403+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.160752+0000","last_change":"2024-09-17T04:28:43.160752+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.160752+0000","last_undegraded":"2024-09-17T04:28:43.160752+0000","last_fullsized":"2024-09-17T04:28:43.160752+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.172729+0000","last_change":"2024-09-17T04:28:44.172729+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.172729+0000","last_undegraded":"2024-09-17T04:28:44.172729+0000","last_fullsized":"2024-09-17T04:28:44.172729+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.180383+0000","last_change":"2024-09-17T04:28:44.180281+0000","last_active":"2024-09-17T04:28:44.180383+0000","last_peered":"2024-09-17T04:28:44.180383+0000","last_clean":"2024-09-17T04:28:44.180383+0000","last_became_active":"2024-09-17T04:28:44.179971+0000","last_became_peered":"2024-09-17T04:28:44.179971+0000","last_unstale":"2024-09-17T04:28:44.180383+0000","last_undegraded":"2024-09-17T04:28:44.180383+0000","last_fullsized":"2024-09-17T04:28:44.180383+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:40.899996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.174265+0000","last_change":"2024-09-17T04:28:44.174265+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.174265+0000","last_undegraded":"2024-09-17T04:28:44.174265+0000","last_fullsized":"2024-09-17T04:28:44.174265+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":["6","1"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.161096+0000","last_change":"2024-09-17T04:28:43.161096+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.161096+0000","last_undegraded":"2024-09-17T04:28:43.161096+0000","last_fullsized":"2024-09-17T04:28:43.161096+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.174630+0000","last_change":"2024-09-17T04:28:44.166800+0000","last_active":"2024-09-17T04:28:44.174630+0000","last_peered":"2024-09-17T04:28:44.174630+0000","last_clean":"2024-09-17T04:28:44.174630+0000","last_became_active":"2024-09-17T04:28:44.166256+0000","last_became_peered":"2024-09-17T04:28:44.166256+0000","last_unstale":"2024-09-17T04:28:44.174630+0000","last_undegraded":"2024-09-17T04:28:44.174630+0000","last_fullsized":"2024-09-17T04:28:44.174630+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:49:09.101261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.168359+0000","last_change":"2024-09-17T04:28:44.168359+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.168359+0000","last_undegraded":"2024-09-17T04:28:44.168359+0000","last_fullsized":"2024-09-17T04:28:44.168359+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":["7","3"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.157976+0000","last_change":"2024-09-17T04:28:43.157976+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.157976+0000","last_undegraded":"2024-09-17T04:28:43.157976+0000","last_fullsized":"2024-09-17T04:28:43.157976+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.164794+0000","last_change":"2024-09-17T04:28:43.164794+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.164794+0000","last_undegraded":"2024-09-17T04:28:43.164794+0000","last_fullsized":"2024-09-17T04:28:43.164794+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.159533+0000","last_change":"2024-09-17T04:28:43.159533+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.159533+0000","last_undegraded":"2024-09-17T04:28:43.159533+0000","last_fullsized":"2024-09-17T04:28:43.159533+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.170926+0000","last_change":"2024-09-17T04:28:43.170926+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.170926+0000","last_undegraded":"2024-09-17T04:28:43.170926+0000","last_fullsized":"2024-09-17T04:28:43.170926+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.191866+0000","last_change":"2024-09-17T04:28:44.186823+0000","last_active":"2024-09-17T04:28:44.191866+0000","last_peered":"2024-09-17T04:28:44.191866+0000","last_clean":"2024-09-17T04:28:44.191866+0000","last_became_active":"2024-09-17T04:28:44.186451+0000","last_became_peered":"2024-09-17T04:28:44.186451+0000","last_unstale":"2024-09-17T04:28:44.191866+0000","last_undegraded":"2024-09-17T04:28:44.191866+0000","last_fullsized":"2024-09-17T04:28:44.191866+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:56:31.348648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.159764+0000","last_change":"2024-09-17T04:28:43.159764+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.159764+0000","last_undegraded":"2024-09-17T04:28:43.159764+0000","last_fullsized":"2024-09-17T04:28:43.159764+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.157003+0000","last_change":"2024-09-17T04:28:43.157003+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.157003+0000","last_undegraded":"2024-09-17T04:28:43.157003+0000","last_fullsized":"2024-09-17T04:28:43.157003+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.182609+0000","last_change":"2024-09-17T04:28:44.182489+0000","last_active":"2024-09-17T04:28:44.182609+0000","last_peered":"2024-09-17T04:28:44.182609+0000","last_clean":"2024-09-17T04:28:44.182609+0000","last_became_active":"2024-09-17T04:28:44.182178+0000","last_became_peered":"2024-09-17T04:28:44.182178+0000","last_unstale":"2024-09-17T04:28:44.182609+0000","last_undegraded":"2024-09-17T04:28:44.182609+0000","last_fullsized":"2024-09-17T04:28:44.182609+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:14.506557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.166386+0000","last_change":"2024-09-17T04:28:43.166386+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.166386+0000","last_undegraded":"2024-09-17T04:28:43.166386+0000","last_fullsized":"2024-09-17T04:28:43.166386+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.175852+0000","last_change":"2024-09-17T04:28:44.172195+0000","last_active":"2024-09-17T04:28:44.175852+0000","last_peered":"2024-09-17T04:28:44.175852+0000","last_clean":"2024-09-17T04:28:44.175852+0000","last_became_active":"2024-09-17T04:28:44.171667+0000","last_became_peered":"2024-09-17T04:28:44.171667+0000","last_unstale":"2024-09-17T04:28:44.175852+0000","last_undegraded":"2024-09-17T04:28:44.175852+0000","last_fullsized":"2024-09-17T04:28:44.175852+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:25:20.752959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.185891+0000","last_change":"2024-09-17T04:28:44.176739+0000","last_active":"2024-09-17T04:28:44.185891+0000","last_peered":"2024-09-17T04:28:44.185891+0000","last_clean":"2024-09-17T04:28:44.185891+0000","last_became_active":"2024-09-17T04:28:44.169094+0000","last_became_peered":"2024-09-17T04:28:44.169094+0000","last_unstale":"2024-09-17T04:28:44.185891+0000","last_undegraded":"2024-09-17T04:28:44.185891+0000","last_fullsized":"2024-09-17T04:28:44.185891+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:30:41.503062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.168918+0000","last_change":"2024-09-17T04:28:44.168918+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.168918+0000","last_undegraded":"2024-09-17T04:28:44.168918+0000","last_fullsized":"2024-09-17T04:28:44.168918+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.168388+0000","last_change":"2024-09-17T04:28:44.168388+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.168388+0000","last_undegraded":"2024-09-17T04:28:44.168388+0000","last_fullsized":"2024-09-17T04:28:44.168388+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.175800+0000","last_change":"2024-09-17T04:28:44.170440+0000","last_active":"2024-09-17T04:28:44.175800+0000","last_peered":"2024-09-17T04:28:44.175800+0000","last_clean":"2024-09-17T04:28:44.175800+0000","last_became_active":"2024-09-17T04:28:44.169980+0000","last_became_peered":"2024-09-17T04:28:44.169980+0000","last_unstale":"2024-09-17T04:28:44.175800+0000","last_undegraded":"2024-09-17T04:28:44.175800+0000","last_fullsized":"2024-09-17T04:28:44.175800+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:53.396145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.160892+0000","last_change":"2024-09-17T04:28:44.160892+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.160892+0000","last_undegraded":"2024-09-17T04:28:44.160892+0000","last_fullsized":"2024-09-17T04:28:44.160892+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836922+0000","last_change":"2024-09-17T04:28:44.174317+0000","last_active":"2024-09-17T04:28:45.836922+0000","last_peered":"2024-09-17T04:28:45.836922+0000","last_clean":"2024-09-17T04:28:45.836922+0000","last_became_active":"2024-09-17T04:28:44.173878+0000","last_became_peered":"2024-09-17T04:28:44.173878+0000","last_unstale":"2024-09-17T04:28:45.836922+0000","last_undegraded":"2024-09-17T04:28:45.836922+0000","last_fullsized":"2024-09-17T04:28:45.836922+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:07.283249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.177201+0000","last_change":"2024-09-17T04:28:44.170671+0000","last_active":"2024-09-17T04:28:44.177201+0000","last_peered":"2024-09-17T04:28:44.177201+0000","last_clean":"2024-09-17T04:28:44.177201+0000","last_became_active":"2024-09-17T04:28:44.169266+0000","last_became_peered":"2024-09-17T04:28:44.169266+0000","last_unstale":"2024-09-17T04:28:44.177201+0000","last_undegraded":"2024-09-17T04:28:44.177201+0000","last_fullsized":"2024-09-17T04:28:44.177201+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:53.851004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836607+0000","last_change":"2024-09-17T04:28:44.188197+0000","last_active":"2024-09-17T04:28:45.836607+0000","last_peered":"2024-09-17T04:28:45.836607+0000","last_clean":"2024-09-17T04:28:45.836607+0000","last_became_active":"2024-09-17T04:28:44.187879+0000","last_became_peered":"2024-09-17T04:28:44.187879+0000","last_unstale":"2024-09-17T04:28:45.836607+0000","last_undegraded":"2024-09-17T04:28:45.836607+0000","last_fullsized":"2024-09-17T04:28:45.836607+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:28:18.827139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.182803+0000","last_change":"2024-09-17T04:28:44.182642+0000","last_active":"2024-09-17T04:28:44.182803+0000","last_peered":"2024-09-17T04:28:44.182803+0000","last_clean":"2024-09-17T04:28:44.182803+0000","last_became_active":"2024-09-17T04:28:44.182271+0000","last_became_peered":"2024-09-17T04:28:44.182271+0000","last_unstale":"2024-09-17T04:28:44.182803+0000","last_undegraded":"2024-09-17T04:28:44.182803+0000","last_fullsized":"2024-09-17T04:28:44.182803+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:08.481224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.168668+0000","last_change":"2024-09-17T04:28:44.166100+0000","last_active":"2024-09-17T04:28:44.168668+0000","last_peered":"2024-09-17T04:28:44.168668+0000","last_clean":"2024-09-17T04:28:44.168668+0000","last_became_active":"2024-09-17T04:28:44.165456+0000","last_became_peered":"2024-09-17T04:28:44.165456+0000","last_unstale":"2024-09-17T04:28:44.168668+0000","last_undegraded":"2024-09-17T04:28:44.168668+0000","last_fullsized":"2024-09-17T04:28:44.168668+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:20.771270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.173852+0000","last_change":"2024-09-17T04:28:44.173852+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.173852+0000","last_undegraded":"2024-09-17T04:28:44.173852+0000","last_fullsized":"2024-09-17T04:28:44.173852+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":["6","3"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.168046+0000","last_change":"2024-09-17T04:28:44.168046+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.168046+0000","last_undegraded":"2024-09-17T04:28:44.168046+0000","last_fullsized":"2024-09-17T04:28:44.168046+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.169200+0000","last_change":"2024-09-17T04:28:43.181281+0000","last_active":"2024-09-17T04:28:44.169200+0000","last_peered":"2024-09-17T04:28:44.169200+0000","last_clean":"2024-09-17T04:28:44.169200+0000","last_became_active":"2024-09-17T04:28:43.180960+0000","last_became_peered":"2024-09-17T04:28:43.180960+0000","last_unstale":"2024-09-17T04:28:44.169200+0000","last_undegraded":"2024-09-17T04:28:44.169200+0000","last_fullsized":"2024-09-17T04:28:44.169200+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:19.766145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.159258+0000","last_change":"2024-09-17T04:28:43.159258+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.159258+0000","last_undegraded":"2024-09-17T04:28:43.159258+0000","last_fullsized":"2024-09-17T04:28:43.159258+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":4,"reported_epoch":71,"state":"creating+activating","last_fresh":"2024-09-17T04:28:43.170907+0000","last_change":"2024-09-17T04:28:43.165532+0000","last_active":"2024-09-17T04:28:42.144673+0000","last_peered":"2024-09-17T04:28:42.144673+0000","last_clean":"2024-09-17T04:28:42.144673+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.170907+0000","last_undegraded":"2024-09-17T04:28:43.170907+0000","last_fullsized":"2024-09-17T04:28:43.170907+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.173734+0000","last_change":"2024-09-17T04:28:44.173734+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.173734+0000","last_undegraded":"2024-09-17T04:28:44.173734+0000","last_fullsized":"2024-09-17T04:28:44.173734+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.163300+0000","last_change":"2024-09-17T04:28:43.187020+0000","last_active":"2024-09-17T04:28:44.163300+0000","last_peered":"2024-09-17T04:28:44.163300+0000","last_clean":"2024-09-17T04:28:44.163300+0000","last_became_active":"2024-09-17T04:28:43.186227+0000","last_became_peered":"2024-09-17T04:28:43.186227+0000","last_unstale":"2024-09-17T04:28:44.163300+0000","last_undegraded":"2024-09-17T04:28:44.163300+0000","last_fullsized":"2024-09-17T04:28:44.163300+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:06.525929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.167817+0000","last_change":"2024-09-17T04:28:44.167817+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.167817+0000","last_undegraded":"2024-09-17T04:28:44.167817+0000","last_fullsized":"2024-09-17T04:28:44.167817+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":["7","0"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.178386+0000","last_change":"2024-09-17T04:28:43.183969+0000","last_active":"2024-09-17T04:28:44.178386+0000","last_peered":"2024-09-17T04:28:44.178386+0000","last_clean":"2024-09-17T04:28:44.178386+0000","last_became_active":"2024-09-17T04:28:43.183603+0000","last_became_peered":"2024-09-17T04:28:43.183603+0000","last_unstale":"2024-09-17T04:28:44.178386+0000","last_undegraded":"2024-09-17T04:28:44.178386+0000","last_fullsized":"2024-09-17T04:28:44.178386+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:47.866148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.183433+0000","last_change":"2024-09-17T04:28:44.183332+0000","last_active":"2024-09-17T04:28:44.183433+0000","last_peered":"2024-09-17T04:28:44.183433+0000","last_clean":"2024-09-17T04:28:44.183433+0000","last_became_active":"2024-09-17T04:28:44.175938+0000","last_became_peered":"2024-09-17T04:28:44.175938+0000","last_unstale":"2024-09-17T04:28:44.183433+0000","last_undegraded":"2024-09-17T04:28:44.183433+0000","last_fullsized":"2024-09-17T04:28:44.183433+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:02.045028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"72'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836677+0000","last_change":"2024-09-17T04:28:43.171042+0000","last_active":"2024-09-17T04:28:45.836677+0000","last_peered":"2024-09-17T04:28:45.836677+0000","last_clean":"2024-09-17T04:28:45.836677+0000","last_became_active":"2024-09-17T04:28:43.170733+0000","last_became_peered":"2024-09-17T04:28:43.170733+0000","last_unstale":"2024-09-17T04:28:45.836677+0000","last_undegraded":"2024-09-17T04:28:45.836677+0000","last_fullsized":"2024-09-17T04:28:45.836677+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:33:35.104722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.185431+0000","last_change":"2024-09-17T04:28:44.181651+0000","last_active":"2024-09-17T04:28:44.185431+0000","last_peered":"2024-09-17T04:28:44.185431+0000","last_clean":"2024-09-17T04:28:44.185431+0000","last_became_active":"2024-09-17T04:28:44.181249+0000","last_became_peered":"2024-09-17T04:28:44.181249+0000","last_unstale":"2024-09-17T04:28:44.185431+0000","last_undegraded":"2024-09-17T04:28:44.185431+0000","last_fullsized":"2024-09-17T04:28:44.185431+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:55:58.969567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.180193+0000","last_change":"2024-09-17T04:28:43.181324+0000","last_active":"2024-09-17T04:28:44.180193+0000","last_peered":"2024-09-17T04:28:44.180193+0000","last_clean":"2024-09-17T04:28:44.180193+0000","last_became_active":"2024-09-17T04:28:43.180647+0000","last_became_peered":"2024-09-17T04:28:43.180647+0000","last_unstale":"2024-09-17T04:28:44.180193+0000","last_undegraded":"2024-09-17T04:28:44.180193+0000","last_fullsized":"2024-09-17T04:28:44.180193+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:01:55.909462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.159075+0000","last_change":"2024-09-17T04:28:43.159075+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.159075+0000","last_undegraded":"2024-09-17T04:28:43.159075+0000","last_fullsized":"2024-09-17T04:28:43.159075+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T04:28:43.171114+0000","last_change":"2024-09-17T04:28:43.163736+0000","last_active":"2024-09-17T04:28:43.171114+0000","last_peered":"2024-09-17T04:28:43.171114+0000","last_clean":"2024-09-17T04:28:43.171114+0000","last_became_active":"2024-09-17T04:28:43.163431+0000","last_became_peered":"2024-09-17T04:28:43.163431+0000","last_unstale":"2024-09-17T04:28:43.171114+0000","last_undegraded":"2024-09-17T04:28:43.171114+0000","last_fullsized":"2024-09-17T04:28:43.171114+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:57.986932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.168710+0000","last_change":"2024-09-17T04:28:43.168710+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.168710+0000","last_undegraded":"2024-09-17T04:28:43.168710+0000","last_fullsized":"2024-09-17T04:28:43.168710+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T04:28:43.163264+0000","last_change":"2024-09-17T04:28:43.163133+0000","last_active":"2024-09-17T04:28:43.163264+0000","last_peered":"2024-09-17T04:28:43.163264+0000","last_clean":"2024-09-17T04:28:43.163264+0000","last_became_active":"2024-09-17T04:28:43.162404+0000","last_became_peered":"2024-09-17T04:28:43.162404+0000","last_unstale":"2024-09-17T04:28:43.163264+0000","last_undegraded":"2024-09-17T04:28:43.163264+0000","last_fullsized":"2024-09-17T04:28:43.163264+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:47.332401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.183082+0000","last_change":"2024-09-17T04:28:44.175084+0000","last_active":"2024-09-17T04:28:44.183082+0000","last_peered":"2024-09-17T04:28:44.183082+0000","last_clean":"2024-09-17T04:28:44.183082+0000","last_became_active":"2024-09-17T04:28:44.174333+0000","last_became_peered":"2024-09-17T04:28:44.174333+0000","last_unstale":"2024-09-17T04:28:44.183082+0000","last_undegraded":"2024-09-17T04:28:44.183082+0000","last_fullsized":"2024-09-17T04:28:44.183082+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:52.267667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.164778+0000","last_change":"2024-09-17T04:28:43.174543+0000","last_active":"2024-09-17T04:28:44.164778+0000","last_peered":"2024-09-17T04:28:44.164778+0000","last_clean":"2024-09-17T04:28:44.164778+0000","last_became_active":"2024-09-17T04:28:43.174199+0000","last_became_peered":"2024-09-17T04:28:43.174199+0000","last_unstale":"2024-09-17T04:28:44.164778+0000","last_undegraded":"2024-09-17T04:28:44.164778+0000","last_fullsized":"2024-09-17T04:28:44.164778+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:22.807921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.168793+0000","last_change":"2024-09-17T04:28:44.168793+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.168793+0000","last_undegraded":"2024-09-17T04:28:44.168793+0000","last_fullsized":"2024-09-17T04:28:44.168793+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.179651+0000","last_change":"2024-09-17T04:28:43.185414+0000","last_active":"2024-09-17T04:28:44.179651+0000","last_peered":"2024-09-17T04:28:44.179651+0000","last_clean":"2024-09-17T04:28:44.179651+0000","last_became_active":"2024-09-17T04:28:43.184969+0000","last_became_peered":"2024-09-17T04:28:43.184969+0000","last_unstale":"2024-09-17T04:28:44.179651+0000","last_undegraded":"2024-09-17T04:28:44.179651+0000","last_fullsized":"2024-09-17T04:28:44.179651+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:58.068125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.173127+0000","last_change":"2024-09-17T04:28:44.173127+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.173127+0000","last_undegraded":"2024-09-17T04:28:44.173127+0000","last_fullsized":"2024-09-17T04:28:44.173127+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"72'3","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836958+0000","last_change":"2024-09-17T04:28:43.172982+0000","last_active":"2024-09-17T04:28:45.836958+0000","last_peered":"2024-09-17T04:28:45.836958+0000","last_clean":"2024-09-17T04:28:45.836958+0000","last_became_active":"2024-09-17T04:28:43.172659+0000","last_became_peered":"2024-09-17T04:28:43.172659+0000","last_unstale":"2024-09-17T04:28:45.836958+0000","last_undegraded":"2024-09-17T04:28:45.836958+0000","last_fullsized":"2024-09-17T04:28:45.836958+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:15.908868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.160386+0000","last_change":"2024-09-17T04:28:43.160386+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.160386+0000","last_undegraded":"2024-09-17T04:28:43.160386+0000","last_fullsized":"2024-09-17T04:28:43.160386+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T04:28:43.175199+0000","last_change":"2024-09-17T04:28:43.168326+0000","last_active":"2024-09-17T04:28:43.175199+0000","last_peered":"2024-09-17T04:28:43.175199+0000","last_clean":"2024-09-17T04:28:43.175199+0000","last_became_active":"2024-09-17T04:28:43.167133+0000","last_became_peered":"2024-09-17T04:28:43.167133+0000","last_unstale":"2024-09-17T04:28:43.175199+0000","last_undegraded":"2024-09-17T04:28:43.175199+0000","last_fullsized":"2024-09-17T04:28:43.175199+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:02.401866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.180070+0000","last_change":"2024-09-17T04:28:44.179971+0000","last_active":"2024-09-17T04:28:44.180070+0000","last_peered":"2024-09-17T04:28:44.180070+0000","last_clean":"2024-09-17T04:28:44.180070+0000","last_became_active":"2024-09-17T04:28:44.179660+0000","last_became_peered":"2024-09-17T04:28:44.179660+0000","last_unstale":"2024-09-17T04:28:44.180070+0000","last_undegraded":"2024-09-17T04:28:44.180070+0000","last_fullsized":"2024-09-17T04:28:44.180070+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:18.874211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":3,"reported_epoch":71,"state":"creating+activating","last_fresh":"2024-09-17T04:28:43.156870+0000","last_change":"2024-09-17T04:28:43.156870+0000","last_active":"2024-09-17T04:28:42.144673+0000","last_peered":"2024-09-17T04:28:42.144673+0000","last_clean":"2024-09-17T04:28:42.144673+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.156870+0000","last_undegraded":"2024-09-17T04:28:43.156870+0000","last_fullsized":"2024-09-17T04:28:43.156870+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":["1","7"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.179314+0000","last_change":"2024-09-17T04:28:44.171317+0000","last_active":"2024-09-17T04:28:44.179314+0000","last_peered":"2024-09-17T04:28:44.179314+0000","last_clean":"2024-09-17T04:28:44.179314+0000","last_became_active":"2024-09-17T04:28:44.170995+0000","last_became_peered":"2024-09-17T04:28:44.170995+0000","last_unstale":"2024-09-17T04:28:44.179314+0000","last_undegraded":"2024-09-17T04:28:44.179314+0000","last_fullsized":"2024-09-17T04:28:44.179314+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:32.865614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.180446+0000","last_change":"2024-09-17T04:28:43.180632+0000","last_active":"2024-09-17T04:28:44.180446+0000","last_peered":"2024-09-17T04:28:44.180446+0000","last_clean":"2024-09-17T04:28:44.180446+0000","last_became_active":"2024-09-17T04:28:43.180146+0000","last_became_peered":"2024-09-17T04:28:43.180146+0000","last_unstale":"2024-09-17T04:28:44.180446+0000","last_undegraded":"2024-09-17T04:28:44.180446+0000","last_fullsized":"2024-09-17T04:28:44.180446+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:03.917176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.168817+0000","last_change":"2024-09-17T04:28:43.168817+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.168817+0000","last_undegraded":"2024-09-17T04:28:43.168817+0000","last_fullsized":"2024-09-17T04:28:43.168817+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.161973+0000","last_change":"2024-09-17T04:28:43.175788+0000","last_active":"2024-09-17T04:28:44.161973+0000","last_peered":"2024-09-17T04:28:44.161973+0000","last_clean":"2024-09-17T04:28:44.161973+0000","last_became_active":"2024-09-17T04:28:43.175486+0000","last_became_peered":"2024-09-17T04:28:43.175486+0000","last_unstale":"2024-09-17T04:28:44.161973+0000","last_undegraded":"2024-09-17T04:28:44.161973+0000","last_fullsized":"2024-09-17T04:28:44.161973+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:56:30.343523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.158958+0000","last_change":"2024-09-17T04:28:43.158958+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.158958+0000","last_undegraded":"2024-09-17T04:28:43.158958+0000","last_fullsized":"2024-09-17T04:28:43.158958+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.162942+0000","last_change":"2024-09-17T04:28:43.175547+0000","last_active":"2024-09-17T04:28:44.162942+0000","last_peered":"2024-09-17T04:28:44.162942+0000","last_clean":"2024-09-17T04:28:44.162942+0000","last_became_active":"2024-09-17T04:28:43.175224+0000","last_became_peered":"2024-09-17T04:28:43.175224+0000","last_unstale":"2024-09-17T04:28:44.162942+0000","last_undegraded":"2024-09-17T04:28:44.162942+0000","last_fullsized":"2024-09-17T04:28:44.162942+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:09.135418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.185472+0000","last_change":"2024-09-17T04:28:44.183124+0000","last_active":"2024-09-17T04:28:44.185472+0000","last_peered":"2024-09-17T04:28:44.185472+0000","last_clean":"2024-09-17T04:28:44.185472+0000","last_became_active":"2024-09-17T04:28:44.181974+0000","last_became_peered":"2024-09-17T04:28:44.181974+0000","last_unstale":"2024-09-17T04:28:44.185472+0000","last_undegraded":"2024-09-17T04:28:44.185472+0000","last_fullsized":"2024-09-17T04:28:44.185472+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:21.789405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.173105+0000","last_change":"2024-09-17T04:28:43.183827+0000","last_active":"2024-09-17T04:28:44.173105+0000","last_peered":"2024-09-17T04:28:44.173105+0000","last_clean":"2024-09-17T04:28:44.173105+0000","last_became_active":"2024-09-17T04:28:43.183492+0000","last_became_peered":"2024-09-17T04:28:43.183492+0000","last_unstale":"2024-09-17T04:28:44.173105+0000","last_undegraded":"2024-09-17T04:28:44.173105+0000","last_fullsized":"2024-09-17T04:28:44.173105+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:02.478321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.169202+0000","last_change":"2024-09-17T04:28:44.169202+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.169202+0000","last_undegraded":"2024-09-17T04:28:44.169202+0000","last_fullsized":"2024-09-17T04:28:44.169202+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":["7","1"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.163486+0000","last_change":"2024-09-17T04:28:43.185602+0000","last_active":"2024-09-17T04:28:44.163486+0000","last_peered":"2024-09-17T04:28:44.163486+0000","last_clean":"2024-09-17T04:28:44.163486+0000","last_became_active":"2024-09-17T04:28:43.185261+0000","last_became_peered":"2024-09-17T04:28:43.185261+0000","last_unstale":"2024-09-17T04:28:44.163486+0000","last_undegraded":"2024-09-17T04:28:44.163486+0000","last_fullsized":"2024-09-17T04:28:44.163486+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:48.674587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.157002+0000","last_change":"2024-09-17T04:28:43.157002+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.157002+0000","last_undegraded":"2024-09-17T04:28:43.157002+0000","last_fullsized":"2024-09-17T04:28:43.157002+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.171911+0000","last_change":"2024-09-17T04:28:43.182926+0000","last_active":"2024-09-17T04:28:44.171911+0000","last_peered":"2024-09-17T04:28:44.171911+0000","last_clean":"2024-09-17T04:28:44.171911+0000","last_became_active":"2024-09-17T04:28:43.179719+0000","last_became_peered":"2024-09-17T04:28:43.179719+0000","last_unstale":"2024-09-17T04:28:44.171911+0000","last_undegraded":"2024-09-17T04:28:44.171911+0000","last_fullsized":"2024-09-17T04:28:44.171911+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:28.997811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.159593+0000","last_change":"2024-09-17T04:28:43.159593+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.159593+0000","last_undegraded":"2024-09-17T04:28:43.159593+0000","last_fullsized":"2024-09-17T04:28:43.159593+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.168165+0000","last_change":"2024-09-17T04:28:43.184237+0000","last_active":"2024-09-17T04:28:44.168165+0000","last_peered":"2024-09-17T04:28:44.168165+0000","last_clean":"2024-09-17T04:28:44.168165+0000","last_became_active":"2024-09-17T04:28:43.183879+0000","last_became_peered":"2024-09-17T04:28:43.183879+0000","last_unstale":"2024-09-17T04:28:44.168165+0000","last_undegraded":"2024-09-17T04:28:44.168165+0000","last_fullsized":"2024-09-17T04:28:44.168165+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:27.493169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836817+0000","last_change":"2024-09-17T04:28:44.179419+0000","last_active":"2024-09-17T04:28:45.836817+0000","last_peered":"2024-09-17T04:28:45.836817+0000","last_clean":"2024-09-17T04:28:45.836817+0000","last_became_active":"2024-09-17T04:28:44.179132+0000","last_became_peered":"2024-09-17T04:28:44.179132+0000","last_unstale":"2024-09-17T04:28:45.836817+0000","last_undegraded":"2024-09-17T04:28:45.836817+0000","last_fullsized":"2024-09-17T04:28:45.836817+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:47:46.546993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.180713+0000","last_change":"2024-09-17T04:28:43.181357+0000","last_active":"2024-09-17T04:28:44.180713+0000","last_peered":"2024-09-17T04:28:44.180713+0000","last_clean":"2024-09-17T04:28:44.180713+0000","last_became_active":"2024-09-17T04:28:43.181053+0000","last_became_peered":"2024-09-17T04:28:43.181053+0000","last_unstale":"2024-09-17T04:28:44.180713+0000","last_undegraded":"2024-09-17T04:28:44.180713+0000","last_fullsized":"2024-09-17T04:28:44.180713+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:53:34.585332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836565+0000","last_change":"2024-09-17T04:28:44.188889+0000","last_active":"2024-09-17T04:28:45.836565+0000","last_peered":"2024-09-17T04:28:45.836565+0000","last_clean":"2024-09-17T04:28:45.836565+0000","last_became_active":"2024-09-17T04:28:44.188524+0000","last_became_peered":"2024-09-17T04:28:44.188524+0000","last_unstale":"2024-09-17T04:28:45.836565+0000","last_undegraded":"2024-09-17T04:28:45.836565+0000","last_fullsized":"2024-09-17T04:28:45.836565+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:17.290337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.162271+0000","last_change":"2024-09-17T04:28:43.184098+0000","last_active":"2024-09-17T04:28:44.162271+0000","last_peered":"2024-09-17T04:28:44.162271+0000","last_clean":"2024-09-17T04:28:44.162271+0000","last_became_active":"2024-09-17T04:28:43.178392+0000","last_became_peered":"2024-09-17T04:28:43.178392+0000","last_unstale":"2024-09-17T04:28:44.162271+0000","last_undegraded":"2024-09-17T04:28:44.162271+0000","last_fullsized":"2024-09-17T04:28:44.162271+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:26.238316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836874+0000","last_change":"2024-09-17T04:28:44.186208+0000","last_active":"2024-09-17T04:28:45.836874+0000","last_peered":"2024-09-17T04:28:45.836874+0000","last_clean":"2024-09-17T04:28:45.836874+0000","last_became_active":"2024-09-17T04:28:44.185834+0000","last_became_peered":"2024-09-17T04:28:44.185834+0000","last_unstale":"2024-09-17T04:28:45.836874+0000","last_undegraded":"2024-09-17T04:28:45.836874+0000","last_fullsized":"2024-09-17T04:28:45.836874+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:24:29.707796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":3,"reported_epoch":71,"state":"creating+activating","last_fresh":"2024-09-17T04:28:43.169387+0000","last_change":"2024-09-17T04:28:43.169387+0000","last_active":"2024-09-17T04:28:42.144673+0000","last_peered":"2024-09-17T04:28:42.144673+0000","last_clean":"2024-09-17T04:28:42.144673+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.169387+0000","last_undegraded":"2024-09-17T04:28:43.169387+0000","last_fullsized":"2024-09-17T04:28:43.169387+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.168884+0000","last_change":"2024-09-17T04:28:43.168884+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.168884+0000","last_undegraded":"2024-09-17T04:28:43.168884+0000","last_fullsized":"2024-09-17T04:28:43.168884+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.171119+0000","last_change":"2024-09-17T04:28:43.183089+0000","last_active":"2024-09-17T04:28:44.171119+0000","last_peered":"2024-09-17T04:28:44.171119+0000","last_clean":"2024-09-17T04:28:44.171119+0000","last_became_active":"2024-09-17T04:28:43.179747+0000","last_became_peered":"2024-09-17T04:28:43.179747+0000","last_unstale":"2024-09-17T04:28:44.171119+0000","last_undegraded":"2024-09-17T04:28:44.171119+0000","last_fullsized":"2024-09-17T04:28:44.171119+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:08.291909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.183882+0000","last_change":"2024-09-17T04:28:44.179171+0000","last_active":"2024-09-17T04:28:44.183882+0000","last_peered":"2024-09-17T04:28:44.183882+0000","last_clean":"2024-09-17T04:28:44.183882+0000","last_became_active":"2024-09-17T04:28:44.178313+0000","last_became_peered":"2024-09-17T04:28:44.178313+0000","last_unstale":"2024-09-17T04:28:44.183882+0000","last_undegraded":"2024-09-17T04:28:44.183882+0000","last_fullsized":"2024-09-17T04:28:44.183882+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:18.874211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.164248+0000","last_change":"2024-09-17T04:28:43.178629+0000","last_active":"2024-09-17T04:28:44.164248+0000","last_peered":"2024-09-17T04:28:44.164248+0000","last_clean":"2024-09-17T04:28:44.164248+0000","last_became_active":"2024-09-17T04:28:43.176654+0000","last_became_peered":"2024-09-17T04:28:43.176654+0000","last_unstale":"2024-09-17T04:28:44.164248+0000","last_undegraded":"2024-09-17T04:28:44.164248+0000","last_fullsized":"2024-09-17T04:28:44.164248+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:27.493169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.167756+0000","last_change":"2024-09-17T04:28:44.167756+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.167756+0000","last_undegraded":"2024-09-17T04:28:44.167756+0000","last_fullsized":"2024-09-17T04:28:44.167756+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":3,"reported_epoch":71,"state":"creating+activating","last_fresh":"2024-09-17T04:28:43.155487+0000","last_change":"2024-09-17T04:28:43.155487+0000","last_active":"2024-09-17T04:28:42.144673+0000","last_peered":"2024-09-17T04:28:42.144673+0000","last_clean":"2024-09-17T04:28:42.144673+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.155487+0000","last_undegraded":"2024-09-17T04:28:43.155487+0000","last_fullsized":"2024-09-17T04:28:43.155487+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.168219+0000","last_change":"2024-09-17T04:28:44.168219+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.168219+0000","last_undegraded":"2024-09-17T04:28:44.168219+0000","last_fullsized":"2024-09-17T04:28:44.168219+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":3,"reported_epoch":71,"state":"creating+activating","last_fresh":"2024-09-17T04:28:43.165983+0000","last_change":"2024-09-17T04:28:43.165983+0000","last_active":"2024-09-17T04:28:42.144673+0000","last_peered":"2024-09-17T04:28:42.144673+0000","last_clean":"2024-09-17T04:28:42.144673+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.165983+0000","last_undegraded":"2024-09-17T04:28:43.165983+0000","last_fullsized":"2024-09-17T04:28:43.165983+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.158679+0000","last_change":"2024-09-17T04:28:43.158679+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.158679+0000","last_undegraded":"2024-09-17T04:28:43.158679+0000","last_fullsized":"2024-09-17T04:28:43.158679+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":3,"reported_epoch":71,"state":"creating+activating","last_fresh":"2024-09-17T04:28:43.155388+0000","last_change":"2024-09-17T04:28:43.155388+0000","last_active":"2024-09-17T04:28:42.144673+0000","last_peered":"2024-09-17T04:28:42.144673+0000","last_clean":"2024-09-17T04:28:42.144673+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.155388+0000","last_undegraded":"2024-09-17T04:28:43.155388+0000","last_fullsized":"2024-09-17T04:28:43.155388+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.171118+0000","last_change":"2024-09-17T04:28:44.171118+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.171118+0000","last_undegraded":"2024-09-17T04:28:44.171118+0000","last_fullsized":"2024-09-17T04:28:44.171118+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T04:28:43.170474+0000","last_change":"2024-09-17T04:28:43.170357+0000","last_active":"2024-09-17T04:28:43.170474+0000","last_peered":"2024-09-17T04:28:43.170474+0000","last_clean":"2024-09-17T04:28:43.170474+0000","last_became_active":"2024-09-17T04:28:43.170017+0000","last_became_peered":"2024-09-17T04:28:43.170017+0000","last_unstale":"2024-09-17T04:28:43.170474+0000","last_undegraded":"2024-09-17T04:28:43.170474+0000","last_fullsized":"2024-09-17T04:28:43.170474+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:16:22.147657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.157808+0000","last_change":"2024-09-17T04:28:43.157808+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.157808+0000","last_undegraded":"2024-09-17T04:28:43.157808+0000","last_fullsized":"2024-09-17T04:28:43.157808+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.162044+0000","last_change":"2024-09-17T04:28:43.177322+0000","last_active":"2024-09-17T04:28:44.162044+0000","last_peered":"2024-09-17T04:28:44.162044+0000","last_clean":"2024-09-17T04:28:44.162044+0000","last_became_active":"2024-09-17T04:28:43.177004+0000","last_became_peered":"2024-09-17T04:28:43.177004+0000","last_unstale":"2024-09-17T04:28:44.162044+0000","last_undegraded":"2024-09-17T04:28:44.162044+0000","last_fullsized":"2024-09-17T04:28:44.162044+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:58.922041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.172672+0000","last_change":"2024-09-17T04:28:44.172672+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.172672+0000","last_undegraded":"2024-09-17T04:28:44.172672+0000","last_fullsized":"2024-09-17T04:28:44.172672+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":["6","3"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.183738+0000","last_change":"2024-09-17T04:28:43.183805+0000","last_active":"2024-09-17T04:28:44.183738+0000","last_peered":"2024-09-17T04:28:44.183738+0000","last_clean":"2024-09-17T04:28:44.183738+0000","last_became_active":"2024-09-17T04:28:43.183414+0000","last_became_peered":"2024-09-17T04:28:43.183414+0000","last_unstale":"2024-09-17T04:28:44.183738+0000","last_undegraded":"2024-09-17T04:28:44.183738+0000","last_fullsized":"2024-09-17T04:28:44.183738+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:46:21.622130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836616+0000","last_change":"2024-09-17T04:28:44.189995+0000","last_active":"2024-09-17T04:28:45.836616+0000","last_peered":"2024-09-17T04:28:45.836616+0000","last_clean":"2024-09-17T04:28:45.836616+0000","last_became_active":"2024-09-17T04:28:44.189715+0000","last_became_peered":"2024-09-17T04:28:44.189715+0000","last_unstale":"2024-09-17T04:28:45.836616+0000","last_undegraded":"2024-09-17T04:28:45.836616+0000","last_fullsized":"2024-09-17T04:28:45.836616+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:25.483700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.178186+0000","last_change":"2024-09-17T04:28:43.184519+0000","last_active":"2024-09-17T04:28:44.178186+0000","last_peered":"2024-09-17T04:28:44.178186+0000","last_clean":"2024-09-17T04:28:44.178186+0000","last_became_active":"2024-09-17T04:28:43.183311+0000","last_became_peered":"2024-09-17T04:28:43.183311+0000","last_unstale":"2024-09-17T04:28:44.178186+0000","last_undegraded":"2024-09-17T04:28:44.178186+0000","last_fullsized":"2024-09-17T04:28:44.178186+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:41.963229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":206,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.167592+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:28:44.167592+0000","last_peered":"2024-09-17T04:28:44.167592+0000","last_clean":"2024-09-17T04:28:44.167592+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:28:44.167592+0000","last_undegraded":"2024-09-17T04:28:44.167592+0000","last_fullsized":"2024-09-17T04:28:44.167592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476797,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1445888,"data_stored":945174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476796,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":851968,"data_stored":354806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476797,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32024,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339816,"statfs":{"total":96636764160,"available":96603971584,"internally_reserved":0,"allocated":851968,"data_stored":354806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476797,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32024,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339816,"statfs":{"total":96636764160,"available":96603971584,"internally_reserved":0,"allocated":851968,"data_stored":354806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509500,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":839680,"data_stored":348531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509499,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":839680,"data_stored":348531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509499,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1433600,"data_stored":938899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542205,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32060,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339780,"statfs":{"total":96636764160,"available":96603934720,"internally_reserved":0,"allocated":884736,"data_stored":367993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:28:48.580 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:28:48.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.839+0000 7f8700200640 1 -- 172.21.15.100:0/3057448955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86f815d760 msgr2=0x7f86f81535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:48.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.839+0000 7f8700200640 1 --2- 172.21.15.100:0/3057448955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86f815d760 0x7f86f81535c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f86ec005fd0 tx=0x7f86ec02a540 comp rx=0 tx=0).stop 2024-09-17T04:28:48.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.839+0000 7f8700200640 1 -- 172.21.15.100:0/3057448955 shutdown_connections 2024-09-17T04:28:48.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.839+0000 7f8700200640 1 --2- 172.21.15.100:0/3057448955 >> v2:172.21.15.184:3300/0 conn(0x7f86f80ad0e0 0x7f86f80ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:48.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.839+0000 7f8700200640 1 --2- 172.21.15.100:0/3057448955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86f815d760 0x7f86f81535c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:48.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.839+0000 7f8700200640 1 -- 172.21.15.100:0/3057448955 >> 172.21.15.100:0/3057448955 conn(0x7f86f806d730 msgr2=0x7f86f80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:48.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.839+0000 7f8700200640 1 -- 172.21.15.100:0/3057448955 shutdown_connections 2024-09-17T04:28:48.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.839+0000 7f8700200640 1 -- 172.21.15.100:0/3057448955 wait complete. 2024-09-17T04:28:48.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.841+0000 7f8700200640 1 Processor -- start 2024-09-17T04:28:48.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.841+0000 7f8700200640 1 -- start start 2024-09-17T04:28:48.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.843+0000 7f8700200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86f80ad0e0 0x7f86f8156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:48.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.843+0000 7f8700200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86f815d760 0x7f86f81f8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:48.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.843+0000 7f8700200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86f8160310 0x7f86f8149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:48.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.843+0000 7f8700200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f86f806f220 con 0x7f86f815d760 2024-09-17T04:28:48.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.843+0000 7f8700200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f86f806f0a0 con 0x7f86f8160310 2024-09-17T04:28:48.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.843+0000 7f8700200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f86f80b4c20 con 0x7f86f80ad0e0 2024-09-17T04:28:48.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.843+0000 7f86fce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86f815d760 0x7f86f81f8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:48.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.843+0000 7f86fce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86f815d760 0x7f86f81f8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38770/0 (socket says 172.21.15.100:38770) 2024-09-17T04:28:48.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.843+0000 7f86fce00640 1 -- 172.21.15.100:0/866644557 learned_addr learned my addr 172.21.15.100:0/866644557 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:48.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.843+0000 7f86fd800640 1 --2- 172.21.15.100:0/866644557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86f80ad0e0 0x7f86f8156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:48.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.843+0000 7f86fe200640 1 --2- 172.21.15.100:0/866644557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86f8160310 0x7f86f8149dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:48.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.844+0000 7f86fce00640 1 -- 172.21.15.100:0/866644557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86f80ad0e0 msgr2=0x7f86f8156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:48.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.844+0000 7f86fce00640 1 --2- 172.21.15.100:0/866644557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86f80ad0e0 0x7f86f8156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:48.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.844+0000 7f86fce00640 1 -- 172.21.15.100:0/866644557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86f8160310 msgr2=0x7f86f8149dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:48.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.844+0000 7f86fce00640 1 --2- 172.21.15.100:0/866644557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86f8160310 0x7f86f8149dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.844+0000 7f86fce00640 1 -- 172.21.15.100:0/866644557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f809def0 con 0x7f86f815d760 2024-09-17T04:28:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.844+0000 7f86fce00640 1 --2- 172.21.15.100:0/866644557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86f815d760 0x7f86f81f8d60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f86e400e910 tx=0x7f86e400edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.844+0000 7f86f5600640 1 -- 172.21.15.100:0/866644557 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e400cd80 con 0x7f86f815d760 2024-09-17T04:28:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.844+0000 7f86f5600640 1 -- 172.21.15.100:0/866644557 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86e4010430 con 0x7f86f815d760 2024-09-17T04:28:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.844+0000 7f8700200640 1 -- 172.21.15.100:0/866644557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f809e1d0 con 0x7f86f815d760 2024-09-17T04:28:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.844+0000 7f86f5600640 1 -- 172.21.15.100:0/866644557 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e4010860 con 0x7f86f815d760 2024-09-17T04:28:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.844+0000 7f8700200640 1 -- 172.21.15.100:0/866644557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f8156580 con 0x7f86f815d760 2024-09-17T04:28:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.846+0000 7f8700200640 1 -- 172.21.15.100:0/866644557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86f809dc60 con 0x7f86f815d760 2024-09-17T04:28:48.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.847+0000 7f86f5600640 1 -- 172.21.15.100:0/866644557 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f86e4010a00 con 0x7f86f815d760 2024-09-17T04:28:48.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.848+0000 7f86f5600640 1 --2- 172.21.15.100:0/866644557 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f86d8078ef0 0x7f86d807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:48.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.848+0000 7f86f5600640 1 -- 172.21.15.100:0/866644557 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7f86e409bd70 con 0x7f86f815d760 2024-09-17T04:28:48.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.848+0000 7f86fd800640 1 --2- 172.21.15.100:0/866644557 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f86d8078ef0 0x7f86d807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:48.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.849+0000 7f86fd800640 1 --2- 172.21.15.100:0/866644557 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f86d8078ef0 0x7f86d807b400 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f86ec033000 tx=0x7f86ec004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:48.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:48.852+0000 7f86f5600640 1 -- 172.21.15.100:0/866644557 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f86e4063690 con 0x7f86f815d760 2024-09-17T04:28:49.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.032+0000 7f8700200640 1 -- 172.21.15.100:0/866644557 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f86f8160fa0 con 0x7f86d8078ef0 2024-09-17T04:28:49.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.037+0000 7f86f5600640 1 -- 172.21.15.100:0/866644557 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248568 (secure 0 0 0) 0x7f86f8160fa0 con 0x7f86d8078ef0 2024-09-17T04:28:49.041 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:28:49.042 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:28:49.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.041+0000 7f8700200640 1 -- 172.21.15.100:0/866644557 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f86d8078ef0 msgr2=0x7f86d807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:49.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.041+0000 7f8700200640 1 --2- 172.21.15.100:0/866644557 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f86d8078ef0 0x7f86d807b400 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f86ec033000 tx=0x7f86ec004730 comp rx=0 tx=0).stop 2024-09-17T04:28:49.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.041+0000 7f8700200640 1 -- 172.21.15.100:0/866644557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86f815d760 msgr2=0x7f86f81f8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:49.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.041+0000 7f8700200640 1 --2- 172.21.15.100:0/866644557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86f815d760 0x7f86f81f8d60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f86e400e910 tx=0x7f86e400edd0 comp rx=0 tx=0).stop 2024-09-17T04:28:49.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.041+0000 7f8700200640 1 -- 172.21.15.100:0/866644557 shutdown_connections 2024-09-17T04:28:49.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.041+0000 7f8700200640 1 --2- 172.21.15.100:0/866644557 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f86d8078ef0 0x7f86d807b400 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:49.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.041+0000 7f8700200640 1 --2- 172.21.15.100:0/866644557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86f8160310 0x7f86f8149dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:49.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.041+0000 7f8700200640 1 --2- 172.21.15.100:0/866644557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86f815d760 0x7f86f81f8d60 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:49.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.041+0000 7f8700200640 1 --2- 172.21.15.100:0/866644557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86f80ad0e0 0x7f86f8156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:49.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.041+0000 7f8700200640 1 -- 172.21.15.100:0/866644557 >> 172.21.15.100:0/866644557 conn(0x7f86f806d730 msgr2=0x7f86f80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:49.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.042+0000 7f8700200640 1 -- 172.21.15.100:0/866644557 shutdown_connections 2024-09-17T04:28:49.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.042+0000 7f8700200640 1 -- 172.21.15.100:0/866644557 wait complete. 2024-09-17T04:28:49.062 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-09-17T04:28:48.041281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590458,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":105,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":43,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253180,"kb_used_data":7812,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721540,"statfs":{"total":773094113280,"available":772834856960,"internally_reserved":0,"allocated":7999488,"data_stored":4013546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":251055668},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.890116"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.178663+0000","last_change":"2024-09-17T04:28:44.178546+0000","last_active":"2024-09-17T04:28:44.178663+0000","last_peered":"2024-09-17T04:28:44.178663+0000","last_clean":"2024-09-17T04:28:44.178663+0000","last_became_active":"2024-09-17T04:28:44.178226+0000","last_became_peered":"2024-09-17T04:28:44.178226+0000","last_unstale":"2024-09-17T04:28:44.178663+0000","last_undegraded":"2024-09-17T04:28:44.178663+0000","last_fullsized":"2024-09-17T04:28:44.178663+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:32.865614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.176591+0000","last_change":"2024-09-17T04:28:44.176591+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.176591+0000","last_undegraded":"2024-09-17T04:28:44.176591+0000","last_fullsized":"2024-09-17T04:28:44.176591+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836882+0000","last_change":"2024-09-17T04:28:44.187901+0000","last_active":"2024-09-17T04:28:45.836882+0000","last_peered":"2024-09-17T04:28:45.836882+0000","last_clean":"2024-09-17T04:28:45.836882+0000","last_became_active":"2024-09-17T04:28:44.187552+0000","last_became_peered":"2024-09-17T04:28:44.187552+0000","last_unstale":"2024-09-17T04:28:45.836882+0000","last_undegraded":"2024-09-17T04:28:45.836882+0000","last_fullsized":"2024-09-17T04:28:45.836882+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:33.519054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.158403+0000","last_change":"2024-09-17T04:28:43.158403+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.158403+0000","last_undegraded":"2024-09-17T04:28:43.158403+0000","last_fullsized":"2024-09-17T04:28:43.158403+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.160752+0000","last_change":"2024-09-17T04:28:43.160752+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.160752+0000","last_undegraded":"2024-09-17T04:28:43.160752+0000","last_fullsized":"2024-09-17T04:28:43.160752+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.172729+0000","last_change":"2024-09-17T04:28:44.172729+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.172729+0000","last_undegraded":"2024-09-17T04:28:44.172729+0000","last_fullsized":"2024-09-17T04:28:44.172729+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.180383+0000","last_change":"2024-09-17T04:28:44.180281+0000","last_active":"2024-09-17T04:28:44.180383+0000","last_peered":"2024-09-17T04:28:44.180383+0000","last_clean":"2024-09-17T04:28:44.180383+0000","last_became_active":"2024-09-17T04:28:44.179971+0000","last_became_peered":"2024-09-17T04:28:44.179971+0000","last_unstale":"2024-09-17T04:28:44.180383+0000","last_undegraded":"2024-09-17T04:28:44.180383+0000","last_fullsized":"2024-09-17T04:28:44.180383+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:40.899996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.174265+0000","last_change":"2024-09-17T04:28:44.174265+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.174265+0000","last_undegraded":"2024-09-17T04:28:44.174265+0000","last_fullsized":"2024-09-17T04:28:44.174265+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":["6","1"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.161096+0000","last_change":"2024-09-17T04:28:43.161096+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.161096+0000","last_undegraded":"2024-09-17T04:28:43.161096+0000","last_fullsized":"2024-09-17T04:28:43.161096+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.174630+0000","last_change":"2024-09-17T04:28:44.166800+0000","last_active":"2024-09-17T04:28:44.174630+0000","last_peered":"2024-09-17T04:28:44.174630+0000","last_clean":"2024-09-17T04:28:44.174630+0000","last_became_active":"2024-09-17T04:28:44.166256+0000","last_became_peered":"2024-09-17T04:28:44.166256+0000","last_unstale":"2024-09-17T04:28:44.174630+0000","last_undegraded":"2024-09-17T04:28:44.174630+0000","last_fullsized":"2024-09-17T04:28:44.174630+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:49:09.101261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.168359+0000","last_change":"2024-09-17T04:28:44.168359+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.168359+0000","last_undegraded":"2024-09-17T04:28:44.168359+0000","last_fullsized":"2024-09-17T04:28:44.168359+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":["7","3"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.157976+0000","last_change":"2024-09-17T04:28:43.157976+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.157976+0000","last_undegraded":"2024-09-17T04:28:43.157976+0000","last_fullsized":"2024-09-17T04:28:43.157976+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.164794+0000","last_change":"2024-09-17T04:28:43.164794+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.164794+0000","last_undegraded":"2024-09-17T04:28:43.164794+0000","last_fullsized":"2024-09-17T04:28:43.164794+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.159533+0000","last_change":"2024-09-17T04:28:43.159533+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.159533+0000","last_undegraded":"2024-09-17T04:28:43.159533+0000","last_fullsized":"2024-09-17T04:28:43.159533+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.170926+0000","last_change":"2024-09-17T04:28:43.170926+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.170926+0000","last_undegraded":"2024-09-17T04:28:43.170926+0000","last_fullsized":"2024-09-17T04:28:43.170926+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.191866+0000","last_change":"2024-09-17T04:28:44.186823+0000","last_active":"2024-09-17T04:28:44.191866+0000","last_peered":"2024-09-17T04:28:44.191866+0000","last_clean":"2024-09-17T04:28:44.191866+0000","last_became_active":"2024-09-17T04:28:44.186451+0000","last_became_peered":"2024-09-17T04:28:44.186451+0000","last_unstale":"2024-09-17T04:28:44.191866+0000","last_undegraded":"2024-09-17T04:28:44.191866+0000","last_fullsized":"2024-09-17T04:28:44.191866+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:56:31.348648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.159764+0000","last_change":"2024-09-17T04:28:43.159764+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.159764+0000","last_undegraded":"2024-09-17T04:28:43.159764+0000","last_fullsized":"2024-09-17T04:28:43.159764+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.157003+0000","last_change":"2024-09-17T04:28:43.157003+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.157003+0000","last_undegraded":"2024-09-17T04:28:43.157003+0000","last_fullsized":"2024-09-17T04:28:43.157003+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.182609+0000","last_change":"2024-09-17T04:28:44.182489+0000","last_active":"2024-09-17T04:28:44.182609+0000","last_peered":"2024-09-17T04:28:44.182609+0000","last_clean":"2024-09-17T04:28:44.182609+0000","last_became_active":"2024-09-17T04:28:44.182178+0000","last_became_peered":"2024-09-17T04:28:44.182178+0000","last_unstale":"2024-09-17T04:28:44.182609+0000","last_undegraded":"2024-09-17T04:28:44.182609+0000","last_fullsized":"2024-09-17T04:28:44.182609+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:14.506557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.166386+0000","last_change":"2024-09-17T04:28:43.166386+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.166386+0000","last_undegraded":"2024-09-17T04:28:43.166386+0000","last_fullsized":"2024-09-17T04:28:43.166386+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.175852+0000","last_change":"2024-09-17T04:28:44.172195+0000","last_active":"2024-09-17T04:28:44.175852+0000","last_peered":"2024-09-17T04:28:44.175852+0000","last_clean":"2024-09-17T04:28:44.175852+0000","last_became_active":"2024-09-17T04:28:44.171667+0000","last_became_peered":"2024-09-17T04:28:44.171667+0000","last_unstale":"2024-09-17T04:28:44.175852+0000","last_undegraded":"2024-09-17T04:28:44.175852+0000","last_fullsized":"2024-09-17T04:28:44.175852+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:25:20.752959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.185891+0000","last_change":"2024-09-17T04:28:44.176739+0000","last_active":"2024-09-17T04:28:44.185891+0000","last_peered":"2024-09-17T04:28:44.185891+0000","last_clean":"2024-09-17T04:28:44.185891+0000","last_became_active":"2024-09-17T04:28:44.169094+0000","last_became_peered":"2024-09-17T04:28:44.169094+0000","last_unstale":"2024-09-17T04:28:44.185891+0000","last_undegraded":"2024-09-17T04:28:44.185891+0000","last_fullsized":"2024-09-17T04:28:44.185891+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:30:41.503062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.168918+0000","last_change":"2024-09-17T04:28:44.168918+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.168918+0000","last_undegraded":"2024-09-17T04:28:44.168918+0000","last_fullsized":"2024-09-17T04:28:44.168918+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.168388+0000","last_change":"2024-09-17T04:28:44.168388+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.168388+0000","last_undegraded":"2024-09-17T04:28:44.168388+0000","last_fullsized":"2024-09-17T04:28:44.168388+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.175800+0000","last_change":"2024-09-17T04:28:44.170440+0000","last_active":"2024-09-17T04:28:44.175800+0000","last_peered":"2024-09-17T04:28:44.175800+0000","last_clean":"2024-09-17T04:28:44.175800+0000","last_became_active":"2024-09-17T04:28:44.169980+0000","last_became_peered":"2024-09-17T04:28:44.169980+0000","last_unstale":"2024-09-17T04:28:44.175800+0000","last_undegraded":"2024-09-17T04:28:44.175800+0000","last_fullsized":"2024-09-17T04:28:44.175800+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:53.396145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.160892+0000","last_change":"2024-09-17T04:28:44.160892+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.160892+0000","last_undegraded":"2024-09-17T04:28:44.160892+0000","last_fullsized":"2024-09-17T04:28:44.160892+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836922+0000","last_change":"2024-09-17T04:28:44.174317+0000","last_active":"2024-09-17T04:28:45.836922+0000","last_peered":"2024-09-17T04:28:45.836922+0000","last_clean":"2024-09-17T04:28:45.836922+0000","last_became_active":"2024-09-17T04:28:44.173878+0000","last_became_peered":"2024-09-17T04:28:44.173878+0000","last_unstale":"2024-09-17T04:28:45.836922+0000","last_undegraded":"2024-09-17T04:28:45.836922+0000","last_fullsized":"2024-09-17T04:28:45.836922+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:07.283249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.177201+0000","last_change":"2024-09-17T04:28:44.170671+0000","last_active":"2024-09-17T04:28:44.177201+0000","last_peered":"2024-09-17T04:28:44.177201+0000","last_clean":"2024-09-17T04:28:44.177201+0000","last_became_active":"2024-09-17T04:28:44.169266+0000","last_became_peered":"2024-09-17T04:28:44.169266+0000","last_unstale":"2024-09-17T04:28:44.177201+0000","last_undegraded":"2024-09-17T04:28:44.177201+0000","last_fullsized":"2024-09-17T04:28:44.177201+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:53.851004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836607+0000","last_change":"2024-09-17T04:28:44.188197+0000","last_active":"2024-09-17T04:28:45.836607+0000","last_peered":"2024-09-17T04:28:45.836607+0000","last_clean":"2024-09-17T04:28:45.836607+0000","last_became_active":"2024-09-17T04:28:44.187879+0000","last_became_peered":"2024-09-17T04:28:44.187879+0000","last_unstale":"2024-09-17T04:28:45.836607+0000","last_undegraded":"2024-09-17T04:28:45.836607+0000","last_fullsized":"2024-09-17T04:28:45.836607+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:28:18.827139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.182803+0000","last_change":"2024-09-17T04:28:44.182642+0000","last_active":"2024-09-17T04:28:44.182803+0000","last_peered":"2024-09-17T04:28:44.182803+0000","last_clean":"2024-09-17T04:28:44.182803+0000","last_became_active":"2024-09-17T04:28:44.182271+0000","last_became_peered":"2024-09-17T04:28:44.182271+0000","last_unstale":"2024-09-17T04:28:44.182803+0000","last_undegraded":"2024-09-17T04:28:44.182803+0000","last_fullsized":"2024-09-17T04:28:44.182803+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:08.481224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.168668+0000","last_change":"2024-09-17T04:28:44.166100+0000","last_active":"2024-09-17T04:28:44.168668+0000","last_peered":"2024-09-17T04:28:44.168668+0000","last_clean":"2024-09-17T04:28:44.168668+0000","last_became_active":"2024-09-17T04:28:44.165456+0000","last_became_peered":"2024-09-17T04:28:44.165456+0000","last_unstale":"2024-09-17T04:28:44.168668+0000","last_undegraded":"2024-09-17T04:28:44.168668+0000","last_fullsized":"2024-09-17T04:28:44.168668+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:20.771270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.173852+0000","last_change":"2024-09-17T04:28:44.173852+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.173852+0000","last_undegraded":"2024-09-17T04:28:44.173852+0000","last_fullsized":"2024-09-17T04:28:44.173852+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":["6","3"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.168046+0000","last_change":"2024-09-17T04:28:44.168046+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.168046+0000","last_undegraded":"2024-09-17T04:28:44.168046+0000","last_fullsized":"2024-09-17T04:28:44.168046+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.169200+0000","last_change":"2024-09-17T04:28:43.181281+0000","last_active":"2024-09-17T04:28:44.169200+0000","last_peered":"2024-09-17T04:28:44.169200+0000","last_clean":"2024-09-17T04:28:44.169200+0000","last_became_active":"2024-09-17T04:28:43.180960+0000","last_became_peered":"2024-09-17T04:28:43.180960+0000","last_unstale":"2024-09-17T04:28:44.169200+0000","last_undegraded":"2024-09-17T04:28:44.169200+0000","last_fullsized":"2024-09-17T04:28:44.169200+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:19.766145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.159258+0000","last_change":"2024-09-17T04:28:43.159258+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.159258+0000","last_undegraded":"2024-09-17T04:28:43.159258+0000","last_fullsized":"2024-09-17T04:28:43.159258+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":4,"reported_epoch":71,"state":"creating+activating","last_fresh":"2024-09-17T04:28:43.170907+0000","last_change":"2024-09-17T04:28:43.165532+0000","last_active":"2024-09-17T04:28:42.144673+0000","last_peered":"2024-09-17T04:28:42.144673+0000","last_clean":"2024-09-17T04:28:42.144673+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.170907+0000","last_undegraded":"2024-09-17T04:28:43.170907+0000","last_fullsized":"2024-09-17T04:28:43.170907+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.173734+0000","last_change":"2024-09-17T04:28:44.173734+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.173734+0000","last_undegraded":"2024-09-17T04:28:44.173734+0000","last_fullsized":"2024-09-17T04:28:44.173734+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.163300+0000","last_change":"2024-09-17T04:28:43.187020+0000","last_active":"2024-09-17T04:28:44.163300+0000","last_peered":"2024-09-17T04:28:44.163300+0000","last_clean":"2024-09-17T04:28:44.163300+0000","last_became_active":"2024-09-17T04:28:43.186227+0000","last_became_peered":"2024-09-17T04:28:43.186227+0000","last_unstale":"2024-09-17T04:28:44.163300+0000","last_undegraded":"2024-09-17T04:28:44.163300+0000","last_fullsized":"2024-09-17T04:28:44.163300+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:06.525929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.167817+0000","last_change":"2024-09-17T04:28:44.167817+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.167817+0000","last_undegraded":"2024-09-17T04:28:44.167817+0000","last_fullsized":"2024-09-17T04:28:44.167817+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":["7","0"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.178386+0000","last_change":"2024-09-17T04:28:43.183969+0000","last_active":"2024-09-17T04:28:44.178386+0000","last_peered":"2024-09-17T04:28:44.178386+0000","last_clean":"2024-09-17T04:28:44.178386+0000","last_became_active":"2024-09-17T04:28:43.183603+0000","last_became_peered":"2024-09-17T04:28:43.183603+0000","last_unstale":"2024-09-17T04:28:44.178386+0000","last_undegraded":"2024-09-17T04:28:44.178386+0000","last_fullsized":"2024-09-17T04:28:44.178386+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:47.866148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.183433+0000","last_change":"2024-09-17T04:28:44.183332+0000","last_active":"2024-09-17T04:28:44.183433+0000","last_peered":"2024-09-17T04:28:44.183433+0000","last_clean":"2024-09-17T04:28:44.183433+0000","last_became_active":"2024-09-17T04:28:44.175938+0000","last_became_peered":"2024-09-17T04:28:44.175938+0000","last_unstale":"2024-09-17T04:28:44.183433+0000","last_undegraded":"2024-09-17T04:28:44.183433+0000","last_fullsized":"2024-09-17T04:28:44.183433+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:02.045028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"72'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836677+0000","last_change":"2024-09-17T04:28:43.171042+0000","last_active":"2024-09-17T04:28:45.836677+0000","last_peered":"2024-09-17T04:28:45.836677+0000","last_clean":"2024-09-17T04:28:45.836677+0000","last_became_active":"2024-09-17T04:28:43.170733+0000","last_became_peered":"2024-09-17T04:28:43.170733+0000","last_unstale":"2024-09-17T04:28:45.836677+0000","last_undegraded":"2024-09-17T04:28:45.836677+0000","last_fullsized":"2024-09-17T04:28:45.836677+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:33:35.104722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.185431+0000","last_change":"2024-09-17T04:28:44.181651+0000","last_active":"2024-09-17T04:28:44.185431+0000","last_peered":"2024-09-17T04:28:44.185431+0000","last_clean":"2024-09-17T04:28:44.185431+0000","last_became_active":"2024-09-17T04:28:44.181249+0000","last_became_peered":"2024-09-17T04:28:44.181249+0000","last_unstale":"2024-09-17T04:28:44.185431+0000","last_undegraded":"2024-09-17T04:28:44.185431+0000","last_fullsized":"2024-09-17T04:28:44.185431+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:55:58.969567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.180193+0000","last_change":"2024-09-17T04:28:43.181324+0000","last_active":"2024-09-17T04:28:44.180193+0000","last_peered":"2024-09-17T04:28:44.180193+0000","last_clean":"2024-09-17T04:28:44.180193+0000","last_became_active":"2024-09-17T04:28:43.180647+0000","last_became_peered":"2024-09-17T04:28:43.180647+0000","last_unstale":"2024-09-17T04:28:44.180193+0000","last_undegraded":"2024-09-17T04:28:44.180193+0000","last_fullsized":"2024-09-17T04:28:44.180193+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:01:55.909462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.159075+0000","last_change":"2024-09-17T04:28:43.159075+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.159075+0000","last_undegraded":"2024-09-17T04:28:43.159075+0000","last_fullsized":"2024-09-17T04:28:43.159075+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T04:28:43.171114+0000","last_change":"2024-09-17T04:28:43.163736+0000","last_active":"2024-09-17T04:28:43.171114+0000","last_peered":"2024-09-17T04:28:43.171114+0000","last_clean":"2024-09-17T04:28:43.171114+0000","last_became_active":"2024-09-17T04:28:43.163431+0000","last_became_peered":"2024-09-17T04:28:43.163431+0000","last_unstale":"2024-09-17T04:28:43.171114+0000","last_undegraded":"2024-09-17T04:28:43.171114+0000","last_fullsized":"2024-09-17T04:28:43.171114+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:57.986932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.168710+0000","last_change":"2024-09-17T04:28:43.168710+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.168710+0000","last_undegraded":"2024-09-17T04:28:43.168710+0000","last_fullsized":"2024-09-17T04:28:43.168710+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T04:28:43.163264+0000","last_change":"2024-09-17T04:28:43.163133+0000","last_active":"2024-09-17T04:28:43.163264+0000","last_peered":"2024-09-17T04:28:43.163264+0000","last_clean":"2024-09-17T04:28:43.163264+0000","last_became_active":"2024-09-17T04:28:43.162404+0000","last_became_peered":"2024-09-17T04:28:43.162404+0000","last_unstale":"2024-09-17T04:28:43.163264+0000","last_undegraded":"2024-09-17T04:28:43.163264+0000","last_fullsized":"2024-09-17T04:28:43.163264+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:47.332401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.183082+0000","last_change":"2024-09-17T04:28:44.175084+0000","last_active":"2024-09-17T04:28:44.183082+0000","last_peered":"2024-09-17T04:28:44.183082+0000","last_clean":"2024-09-17T04:28:44.183082+0000","last_became_active":"2024-09-17T04:28:44.174333+0000","last_became_peered":"2024-09-17T04:28:44.174333+0000","last_unstale":"2024-09-17T04:28:44.183082+0000","last_undegraded":"2024-09-17T04:28:44.183082+0000","last_fullsized":"2024-09-17T04:28:44.183082+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:52.267667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.164778+0000","last_change":"2024-09-17T04:28:43.174543+0000","last_active":"2024-09-17T04:28:44.164778+0000","last_peered":"2024-09-17T04:28:44.164778+0000","last_clean":"2024-09-17T04:28:44.164778+0000","last_became_active":"2024-09-17T04:28:43.174199+0000","last_became_peered":"2024-09-17T04:28:43.174199+0000","last_unstale":"2024-09-17T04:28:44.164778+0000","last_undegraded":"2024-09-17T04:28:44.164778+0000","last_fullsized":"2024-09-17T04:28:44.164778+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:22.807921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.168793+0000","last_change":"2024-09-17T04:28:44.168793+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.168793+0000","last_undegraded":"2024-09-17T04:28:44.168793+0000","last_fullsized":"2024-09-17T04:28:44.168793+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.179651+0000","last_change":"2024-09-17T04:28:43.185414+0000","last_active":"2024-09-17T04:28:44.179651+0000","last_peered":"2024-09-17T04:28:44.179651+0000","last_clean":"2024-09-17T04:28:44.179651+0000","last_became_active":"2024-09-17T04:28:43.184969+0000","last_became_peered":"2024-09-17T04:28:43.184969+0000","last_unstale":"2024-09-17T04:28:44.179651+0000","last_undegraded":"2024-09-17T04:28:44.179651+0000","last_fullsized":"2024-09-17T04:28:44.179651+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:58.068125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.173127+0000","last_change":"2024-09-17T04:28:44.173127+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.173127+0000","last_undegraded":"2024-09-17T04:28:44.173127+0000","last_fullsized":"2024-09-17T04:28:44.173127+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"72'3","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836958+0000","last_change":"2024-09-17T04:28:43.172982+0000","last_active":"2024-09-17T04:28:45.836958+0000","last_peered":"2024-09-17T04:28:45.836958+0000","last_clean":"2024-09-17T04:28:45.836958+0000","last_became_active":"2024-09-17T04:28:43.172659+0000","last_became_peered":"2024-09-17T04:28:43.172659+0000","last_unstale":"2024-09-17T04:28:45.836958+0000","last_undegraded":"2024-09-17T04:28:45.836958+0000","last_fullsized":"2024-09-17T04:28:45.836958+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:15.908868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.160386+0000","last_change":"2024-09-17T04:28:43.160386+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.160386+0000","last_undegraded":"2024-09-17T04:28:43.160386+0000","last_fullsized":"2024-09-17T04:28:43.160386+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T04:28:43.175199+0000","last_change":"2024-09-17T04:28:43.168326+0000","last_active":"2024-09-17T04:28:43.175199+0000","last_peered":"2024-09-17T04:28:43.175199+0000","last_clean":"2024-09-17T04:28:43.175199+0000","last_became_active":"2024-09-17T04:28:43.167133+0000","last_became_peered":"2024-09-17T04:28:43.167133+0000","last_unstale":"2024-09-17T04:28:43.175199+0000","last_undegraded":"2024-09-17T04:28:43.175199+0000","last_fullsized":"2024-09-17T04:28:43.175199+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:02.401866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.180070+0000","last_change":"2024-09-17T04:28:44.179971+0000","last_active":"2024-09-17T04:28:44.180070+0000","last_peered":"2024-09-17T04:28:44.180070+0000","last_clean":"2024-09-17T04:28:44.180070+0000","last_became_active":"2024-09-17T04:28:44.179660+0000","last_became_peered":"2024-09-17T04:28:44.179660+0000","last_unstale":"2024-09-17T04:28:44.180070+0000","last_undegraded":"2024-09-17T04:28:44.180070+0000","last_fullsized":"2024-09-17T04:28:44.180070+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:18.874211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":3,"reported_epoch":71,"state":"creating+activating","last_fresh":"2024-09-17T04:28:43.156870+0000","last_change":"2024-09-17T04:28:43.156870+0000","last_active":"2024-09-17T04:28:42.144673+0000","last_peered":"2024-09-17T04:28:42.144673+0000","last_clean":"2024-09-17T04:28:42.144673+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.156870+0000","last_undegraded":"2024-09-17T04:28:43.156870+0000","last_fullsized":"2024-09-17T04:28:43.156870+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":["1","7"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.179314+0000","last_change":"2024-09-17T04:28:44.171317+0000","last_active":"2024-09-17T04:28:44.179314+0000","last_peered":"2024-09-17T04:28:44.179314+0000","last_clean":"2024-09-17T04:28:44.179314+0000","last_became_active":"2024-09-17T04:28:44.170995+0000","last_became_peered":"2024-09-17T04:28:44.170995+0000","last_unstale":"2024-09-17T04:28:44.179314+0000","last_undegraded":"2024-09-17T04:28:44.179314+0000","last_fullsized":"2024-09-17T04:28:44.179314+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:32.865614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.180446+0000","last_change":"2024-09-17T04:28:43.180632+0000","last_active":"2024-09-17T04:28:44.180446+0000","last_peered":"2024-09-17T04:28:44.180446+0000","last_clean":"2024-09-17T04:28:44.180446+0000","last_became_active":"2024-09-17T04:28:43.180146+0000","last_became_peered":"2024-09-17T04:28:43.180146+0000","last_unstale":"2024-09-17T04:28:44.180446+0000","last_undegraded":"2024-09-17T04:28:44.180446+0000","last_fullsized":"2024-09-17T04:28:44.180446+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:03.917176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.168817+0000","last_change":"2024-09-17T04:28:43.168817+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.168817+0000","last_undegraded":"2024-09-17T04:28:43.168817+0000","last_fullsized":"2024-09-17T04:28:43.168817+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.161973+0000","last_change":"2024-09-17T04:28:43.175788+0000","last_active":"2024-09-17T04:28:44.161973+0000","last_peered":"2024-09-17T04:28:44.161973+0000","last_clean":"2024-09-17T04:28:44.161973+0000","last_became_active":"2024-09-17T04:28:43.175486+0000","last_became_peered":"2024-09-17T04:28:43.175486+0000","last_unstale":"2024-09-17T04:28:44.161973+0000","last_undegraded":"2024-09-17T04:28:44.161973+0000","last_fullsized":"2024-09-17T04:28:44.161973+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:56:30.343523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.158958+0000","last_change":"2024-09-17T04:28:43.158958+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.158958+0000","last_undegraded":"2024-09-17T04:28:43.158958+0000","last_fullsized":"2024-09-17T04:28:43.158958+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.162942+0000","last_change":"2024-09-17T04:28:43.175547+0000","last_active":"2024-09-17T04:28:44.162942+0000","last_peered":"2024-09-17T04:28:44.162942+0000","last_clean":"2024-09-17T04:28:44.162942+0000","last_became_active":"2024-09-17T04:28:43.175224+0000","last_became_peered":"2024-09-17T04:28:43.175224+0000","last_unstale":"2024-09-17T04:28:44.162942+0000","last_undegraded":"2024-09-17T04:28:44.162942+0000","last_fullsized":"2024-09-17T04:28:44.162942+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:09.135418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.185472+0000","last_change":"2024-09-17T04:28:44.183124+0000","last_active":"2024-09-17T04:28:44.185472+0000","last_peered":"2024-09-17T04:28:44.185472+0000","last_clean":"2024-09-17T04:28:44.185472+0000","last_became_active":"2024-09-17T04:28:44.181974+0000","last_became_peered":"2024-09-17T04:28:44.181974+0000","last_unstale":"2024-09-17T04:28:44.185472+0000","last_undegraded":"2024-09-17T04:28:44.185472+0000","last_fullsized":"2024-09-17T04:28:44.185472+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:21.789405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.173105+0000","last_change":"2024-09-17T04:28:43.183827+0000","last_active":"2024-09-17T04:28:44.173105+0000","last_peered":"2024-09-17T04:28:44.173105+0000","last_clean":"2024-09-17T04:28:44.173105+0000","last_became_active":"2024-09-17T04:28:43.183492+0000","last_became_peered":"2024-09-17T04:28:43.183492+0000","last_unstale":"2024-09-17T04:28:44.173105+0000","last_undegraded":"2024-09-17T04:28:44.173105+0000","last_fullsized":"2024-09-17T04:28:44.173105+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:02.478321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.169202+0000","last_change":"2024-09-17T04:28:44.169202+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.169202+0000","last_undegraded":"2024-09-17T04:28:44.169202+0000","last_fullsized":"2024-09-17T04:28:44.169202+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":["7","1"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.163486+0000","last_change":"2024-09-17T04:28:43.185602+0000","last_active":"2024-09-17T04:28:44.163486+0000","last_peered":"2024-09-17T04:28:44.163486+0000","last_clean":"2024-09-17T04:28:44.163486+0000","last_became_active":"2024-09-17T04:28:43.185261+0000","last_became_peered":"2024-09-17T04:28:43.185261+0000","last_unstale":"2024-09-17T04:28:44.163486+0000","last_undegraded":"2024-09-17T04:28:44.163486+0000","last_fullsized":"2024-09-17T04:28:44.163486+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:48.674587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.157002+0000","last_change":"2024-09-17T04:28:43.157002+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.157002+0000","last_undegraded":"2024-09-17T04:28:43.157002+0000","last_fullsized":"2024-09-17T04:28:43.157002+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.171911+0000","last_change":"2024-09-17T04:28:43.182926+0000","last_active":"2024-09-17T04:28:44.171911+0000","last_peered":"2024-09-17T04:28:44.171911+0000","last_clean":"2024-09-17T04:28:44.171911+0000","last_became_active":"2024-09-17T04:28:43.179719+0000","last_became_peered":"2024-09-17T04:28:43.179719+0000","last_unstale":"2024-09-17T04:28:44.171911+0000","last_undegraded":"2024-09-17T04:28:44.171911+0000","last_fullsized":"2024-09-17T04:28:44.171911+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:28.997811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.159593+0000","last_change":"2024-09-17T04:28:43.159593+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.159593+0000","last_undegraded":"2024-09-17T04:28:43.159593+0000","last_fullsized":"2024-09-17T04:28:43.159593+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.168165+0000","last_change":"2024-09-17T04:28:43.184237+0000","last_active":"2024-09-17T04:28:44.168165+0000","last_peered":"2024-09-17T04:28:44.168165+0000","last_clean":"2024-09-17T04:28:44.168165+0000","last_became_active":"2024-09-17T04:28:43.183879+0000","last_became_peered":"2024-09-17T04:28:43.183879+0000","last_unstale":"2024-09-17T04:28:44.168165+0000","last_undegraded":"2024-09-17T04:28:44.168165+0000","last_fullsized":"2024-09-17T04:28:44.168165+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:27.493169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836817+0000","last_change":"2024-09-17T04:28:44.179419+0000","last_active":"2024-09-17T04:28:45.836817+0000","last_peered":"2024-09-17T04:28:45.836817+0000","last_clean":"2024-09-17T04:28:45.836817+0000","last_became_active":"2024-09-17T04:28:44.179132+0000","last_became_peered":"2024-09-17T04:28:44.179132+0000","last_unstale":"2024-09-17T04:28:45.836817+0000","last_undegraded":"2024-09-17T04:28:45.836817+0000","last_fullsized":"2024-09-17T04:28:45.836817+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:47:46.546993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.180713+0000","last_change":"2024-09-17T04:28:43.181357+0000","last_active":"2024-09-17T04:28:44.180713+0000","last_peered":"2024-09-17T04:28:44.180713+0000","last_clean":"2024-09-17T04:28:44.180713+0000","last_became_active":"2024-09-17T04:28:43.181053+0000","last_became_peered":"2024-09-17T04:28:43.181053+0000","last_unstale":"2024-09-17T04:28:44.180713+0000","last_undegraded":"2024-09-17T04:28:44.180713+0000","last_fullsized":"2024-09-17T04:28:44.180713+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:53:34.585332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836565+0000","last_change":"2024-09-17T04:28:44.188889+0000","last_active":"2024-09-17T04:28:45.836565+0000","last_peered":"2024-09-17T04:28:45.836565+0000","last_clean":"2024-09-17T04:28:45.836565+0000","last_became_active":"2024-09-17T04:28:44.188524+0000","last_became_peered":"2024-09-17T04:28:44.188524+0000","last_unstale":"2024-09-17T04:28:45.836565+0000","last_undegraded":"2024-09-17T04:28:45.836565+0000","last_fullsized":"2024-09-17T04:28:45.836565+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:17.290337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.162271+0000","last_change":"2024-09-17T04:28:43.184098+0000","last_active":"2024-09-17T04:28:44.162271+0000","last_peered":"2024-09-17T04:28:44.162271+0000","last_clean":"2024-09-17T04:28:44.162271+0000","last_became_active":"2024-09-17T04:28:43.178392+0000","last_became_peered":"2024-09-17T04:28:43.178392+0000","last_unstale":"2024-09-17T04:28:44.162271+0000","last_undegraded":"2024-09-17T04:28:44.162271+0000","last_fullsized":"2024-09-17T04:28:44.162271+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:26.238316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836874+0000","last_change":"2024-09-17T04:28:44.186208+0000","last_active":"2024-09-17T04:28:45.836874+0000","last_peered":"2024-09-17T04:28:45.836874+0000","last_clean":"2024-09-17T04:28:45.836874+0000","last_became_active":"2024-09-17T04:28:44.185834+0000","last_became_peered":"2024-09-17T04:28:44.185834+0000","last_unstale":"2024-09-17T04:28:45.836874+0000","last_undegraded":"2024-09-17T04:28:45.836874+0000","last_fullsized":"2024-09-17T04:28:45.836874+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:24:29.707796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":3,"reported_epoch":71,"state":"creating+activating","last_fresh":"2024-09-17T04:28:43.169387+0000","last_change":"2024-09-17T04:28:43.169387+0000","last_active":"2024-09-17T04:28:42.144673+0000","last_peered":"2024-09-17T04:28:42.144673+0000","last_clean":"2024-09-17T04:28:42.144673+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.169387+0000","last_undegraded":"2024-09-17T04:28:43.169387+0000","last_fullsized":"2024-09-17T04:28:43.169387+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.168884+0000","last_change":"2024-09-17T04:28:43.168884+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.168884+0000","last_undegraded":"2024-09-17T04:28:43.168884+0000","last_fullsized":"2024-09-17T04:28:43.168884+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.171119+0000","last_change":"2024-09-17T04:28:43.183089+0000","last_active":"2024-09-17T04:28:44.171119+0000","last_peered":"2024-09-17T04:28:44.171119+0000","last_clean":"2024-09-17T04:28:44.171119+0000","last_became_active":"2024-09-17T04:28:43.179747+0000","last_became_peered":"2024-09-17T04:28:43.179747+0000","last_unstale":"2024-09-17T04:28:44.171119+0000","last_undegraded":"2024-09-17T04:28:44.171119+0000","last_fullsized":"2024-09-17T04:28:44.171119+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:08.291909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.183882+0000","last_change":"2024-09-17T04:28:44.179171+0000","last_active":"2024-09-17T04:28:44.183882+0000","last_peered":"2024-09-17T04:28:44.183882+0000","last_clean":"2024-09-17T04:28:44.183882+0000","last_became_active":"2024-09-17T04:28:44.178313+0000","last_became_peered":"2024-09-17T04:28:44.178313+0000","last_unstale":"2024-09-17T04:28:44.183882+0000","last_undegraded":"2024-09-17T04:28:44.183882+0000","last_fullsized":"2024-09-17T04:28:44.183882+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:18.874211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.164248+0000","last_change":"2024-09-17T04:28:43.178629+0000","last_active":"2024-09-17T04:28:44.164248+0000","last_peered":"2024-09-17T04:28:44.164248+0000","last_clean":"2024-09-17T04:28:44.164248+0000","last_became_active":"2024-09-17T04:28:43.176654+0000","last_became_peered":"2024-09-17T04:28:43.176654+0000","last_unstale":"2024-09-17T04:28:44.164248+0000","last_undegraded":"2024-09-17T04:28:44.164248+0000","last_fullsized":"2024-09-17T04:28:44.164248+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:27.493169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.167756+0000","last_change":"2024-09-17T04:28:44.167756+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.167756+0000","last_undegraded":"2024-09-17T04:28:44.167756+0000","last_fullsized":"2024-09-17T04:28:44.167756+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":3,"reported_epoch":71,"state":"creating+activating","last_fresh":"2024-09-17T04:28:43.155487+0000","last_change":"2024-09-17T04:28:43.155487+0000","last_active":"2024-09-17T04:28:42.144673+0000","last_peered":"2024-09-17T04:28:42.144673+0000","last_clean":"2024-09-17T04:28:42.144673+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.155487+0000","last_undegraded":"2024-09-17T04:28:43.155487+0000","last_fullsized":"2024-09-17T04:28:43.155487+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.168219+0000","last_change":"2024-09-17T04:28:44.168219+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.168219+0000","last_undegraded":"2024-09-17T04:28:44.168219+0000","last_fullsized":"2024-09-17T04:28:44.168219+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":3,"reported_epoch":71,"state":"creating+activating","last_fresh":"2024-09-17T04:28:43.165983+0000","last_change":"2024-09-17T04:28:43.165983+0000","last_active":"2024-09-17T04:28:42.144673+0000","last_peered":"2024-09-17T04:28:42.144673+0000","last_clean":"2024-09-17T04:28:42.144673+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.165983+0000","last_undegraded":"2024-09-17T04:28:43.165983+0000","last_fullsized":"2024-09-17T04:28:43.165983+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.158679+0000","last_change":"2024-09-17T04:28:43.158679+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.158679+0000","last_undegraded":"2024-09-17T04:28:43.158679+0000","last_fullsized":"2024-09-17T04:28:43.158679+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":3,"reported_epoch":71,"state":"creating+activating","last_fresh":"2024-09-17T04:28:43.155388+0000","last_change":"2024-09-17T04:28:43.155388+0000","last_active":"2024-09-17T04:28:42.144673+0000","last_peered":"2024-09-17T04:28:42.144673+0000","last_clean":"2024-09-17T04:28:42.144673+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.155388+0000","last_undegraded":"2024-09-17T04:28:43.155388+0000","last_fullsized":"2024-09-17T04:28:43.155388+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.171118+0000","last_change":"2024-09-17T04:28:44.171118+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.171118+0000","last_undegraded":"2024-09-17T04:28:44.171118+0000","last_fullsized":"2024-09-17T04:28:44.171118+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T04:28:43.170474+0000","last_change":"2024-09-17T04:28:43.170357+0000","last_active":"2024-09-17T04:28:43.170474+0000","last_peered":"2024-09-17T04:28:43.170474+0000","last_clean":"2024-09-17T04:28:43.170474+0000","last_became_active":"2024-09-17T04:28:43.170017+0000","last_became_peered":"2024-09-17T04:28:43.170017+0000","last_unstale":"2024-09-17T04:28:43.170474+0000","last_undegraded":"2024-09-17T04:28:43.170474+0000","last_fullsized":"2024-09-17T04:28:43.170474+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:16:22.147657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.157808+0000","last_change":"2024-09-17T04:28:43.157808+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.157808+0000","last_undegraded":"2024-09-17T04:28:43.157808+0000","last_fullsized":"2024-09-17T04:28:43.157808+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.162044+0000","last_change":"2024-09-17T04:28:43.177322+0000","last_active":"2024-09-17T04:28:44.162044+0000","last_peered":"2024-09-17T04:28:44.162044+0000","last_clean":"2024-09-17T04:28:44.162044+0000","last_became_active":"2024-09-17T04:28:43.177004+0000","last_became_peered":"2024-09-17T04:28:43.177004+0000","last_unstale":"2024-09-17T04:28:44.162044+0000","last_undegraded":"2024-09-17T04:28:44.162044+0000","last_fullsized":"2024-09-17T04:28:44.162044+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:58.922041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.172672+0000","last_change":"2024-09-17T04:28:44.172672+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.172672+0000","last_undegraded":"2024-09-17T04:28:44.172672+0000","last_fullsized":"2024-09-17T04:28:44.172672+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":["6","3"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.183738+0000","last_change":"2024-09-17T04:28:43.183805+0000","last_active":"2024-09-17T04:28:44.183738+0000","last_peered":"2024-09-17T04:28:44.183738+0000","last_clean":"2024-09-17T04:28:44.183738+0000","last_became_active":"2024-09-17T04:28:43.183414+0000","last_became_peered":"2024-09-17T04:28:43.183414+0000","last_unstale":"2024-09-17T04:28:44.183738+0000","last_undegraded":"2024-09-17T04:28:44.183738+0000","last_fullsized":"2024-09-17T04:28:44.183738+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:46:21.622130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836616+0000","last_change":"2024-09-17T04:28:44.189995+0000","last_active":"2024-09-17T04:28:45.836616+0000","last_peered":"2024-09-17T04:28:45.836616+0000","last_clean":"2024-09-17T04:28:45.836616+0000","last_became_active":"2024-09-17T04:28:44.189715+0000","last_became_peered":"2024-09-17T04:28:44.189715+0000","last_unstale":"2024-09-17T04:28:45.836616+0000","last_undegraded":"2024-09-17T04:28:45.836616+0000","last_fullsized":"2024-09-17T04:28:45.836616+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:25.483700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.178186+0000","last_change":"2024-09-17T04:28:43.184519+0000","last_active":"2024-09-17T04:28:44.178186+0000","last_peered":"2024-09-17T04:28:44.178186+0000","last_clean":"2024-09-17T04:28:44.178186+0000","last_became_active":"2024-09-17T04:28:43.183311+0000","last_became_peered":"2024-09-17T04:28:43.183311+0000","last_unstale":"2024-09-17T04:28:44.178186+0000","last_undegraded":"2024-09-17T04:28:44.178186+0000","last_fullsized":"2024-09-17T04:28:44.178186+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:41.963229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":206,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.167592+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:28:44.167592+0000","last_peered":"2024-09-17T04:28:44.167592+0000","last_clean":"2024-09-17T04:28:44.167592+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:28:44.167592+0000","last_undegraded":"2024-09-17T04:28:44.167592+0000","last_fullsized":"2024-09-17T04:28:44.167592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476797,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1445888,"data_stored":945174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476796,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":851968,"data_stored":354806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476797,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32024,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339816,"statfs":{"total":96636764160,"available":96603971584,"internally_reserved":0,"allocated":851968,"data_stored":354806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476797,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32024,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339816,"statfs":{"total":96636764160,"available":96603971584,"internally_reserved":0,"allocated":851968,"data_stored":354806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509500,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":839680,"data_stored":348531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509499,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":839680,"data_stored":348531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509499,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1433600,"data_stored":938899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542205,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32060,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339780,"statfs":{"total":96636764160,"available":96603934720,"internally_reserved":0,"allocated":884736,"data_stored":367993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:28:49.066 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:28:49.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.322+0000 7f5a12800640 1 -- 172.21.15.100:0/1894229772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a0c0ad9c0 msgr2=0x7f5a0c0af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:49.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.322+0000 7f5a12800640 1 --2- 172.21.15.100:0/1894229772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a0c0ad9c0 0x7f5a0c0af170 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c06e790 tx=0x7f5a0002a4d0 comp rx=0 tx=0).stop 2024-09-17T04:28:49.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.322+0000 7f5a12800640 1 -- 172.21.15.100:0/1894229772 shutdown_connections 2024-09-17T04:28:49.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.322+0000 7f5a12800640 1 --2- 172.21.15.100:0/1894229772 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a0c15d670 0x7f5a0c070530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:49.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.322+0000 7f5a12800640 1 --2- 172.21.15.100:0/1894229772 >> v2:172.21.15.184:3300/0 conn(0x7f5a0c0b2f60 0x7f5a0c15a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:49.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.322+0000 7f5a12800640 1 --2- 172.21.15.100:0/1894229772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a0c0ad9c0 0x7f5a0c0af170 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:49.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.322+0000 7f5a12800640 1 -- 172.21.15.100:0/1894229772 >> 172.21.15.100:0/1894229772 conn(0x7f5a0c0b1e40 msgr2=0x7f5a0c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:49.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.322+0000 7f5a12800640 1 -- 172.21.15.100:0/1894229772 shutdown_connections 2024-09-17T04:28:49.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.322+0000 7f5a12800640 1 -- 172.21.15.100:0/1894229772 wait complete. 2024-09-17T04:28:49.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.324+0000 7f5a12800640 1 Processor -- start 2024-09-17T04:28:49.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.324+0000 7f5a12800640 1 -- start start 2024-09-17T04:28:49.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.326+0000 7f5a12800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a0c0ad9c0 0x7f5a0c0ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:49.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.326+0000 7f5a12800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a0c0b2f60 0x7f5a0c14dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:49.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.326+0000 7f5a12800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a0c15d670 0x7f5a0c15b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:49.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.326+0000 7f5a12800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c070c80 con 0x7f5a0c0ad9c0 2024-09-17T04:28:49.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.326+0000 7f5a12800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c06f290 con 0x7f5a0c0b2f60 2024-09-17T04:28:49.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.326+0000 7f5a12800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5a0c156ab0 con 0x7f5a0c15d670 2024-09-17T04:28:49.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.326+0000 7f5a11400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a0c0ad9c0 0x7f5a0c0ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:49.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.326+0000 7f5a11400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a0c0ad9c0 0x7f5a0c0ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38788/0 (socket says 172.21.15.100:38788) 2024-09-17T04:28:49.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.326+0000 7f5a11400640 1 -- 172.21.15.100:0/1920380973 learned_addr learned my addr 172.21.15.100:0/1920380973 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:49.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.326+0000 7f5a11e00640 1 --2- 172.21.15.100:0/1920380973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a0c15d670 0x7f5a0c15b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:49.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.326+0000 7f5a10a00640 1 --2- 172.21.15.100:0/1920380973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a0c0b2f60 0x7f5a0c14dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:49.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.326+0000 7f5a11400640 1 -- 172.21.15.100:0/1920380973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a0c15d670 msgr2=0x7f5a0c15b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:49.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.326+0000 7f5a11400640 1 --2- 172.21.15.100:0/1920380973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a0c15d670 0x7f5a0c15b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:49.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.326+0000 7f5a11400640 1 -- 172.21.15.100:0/1920380973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a0c0b2f60 msgr2=0x7f5a0c14dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:49.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.326+0000 7f5a11400640 1 --2- 172.21.15.100:0/1920380973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a0c0b2f60 0x7f5a0c14dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:49.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.326+0000 7f5a11400640 1 -- 172.21.15.100:0/1920380973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a0c156620 con 0x7f5a0c0ad9c0 2024-09-17T04:28:49.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.327+0000 7f5a11400640 1 --2- 172.21.15.100:0/1920380973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a0c0ad9c0 0x7f5a0c0ccac0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5a000099c0 tx=0x7f5a000038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:49.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.327+0000 7f5a09600640 1 -- 172.21.15.100:0/1920380973 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0003e030 con 0x7f5a0c0ad9c0 2024-09-17T04:28:49.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.327+0000 7f5a09600640 1 -- 172.21.15.100:0/1920380973 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a00003ca0 con 0x7f5a0c0ad9c0 2024-09-17T04:28:49.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.327+0000 7f5a12800640 1 -- 172.21.15.100:0/1920380973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a0c0cd4a0 con 0x7f5a0c0ad9c0 2024-09-17T04:28:49.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.327+0000 7f5a09600640 1 -- 172.21.15.100:0/1920380973 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a00035040 con 0x7f5a0c0ad9c0 2024-09-17T04:28:49.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.327+0000 7f5a12800640 1 -- 172.21.15.100:0/1920380973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a0c0cda60 con 0x7f5a0c0ad9c0 2024-09-17T04:28:49.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.328+0000 7f5a12800640 1 -- 172.21.15.100:0/1920380973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a0c15a0f0 con 0x7f5a0c0ad9c0 2024-09-17T04:28:49.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.329+0000 7f5a09600640 1 -- 172.21.15.100:0/1920380973 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f5a00041030 con 0x7f5a0c0ad9c0 2024-09-17T04:28:49.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.330+0000 7f5a09600640 1 --2- 172.21.15.100:0/1920380973 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f59ec078ef0 0x7f59ec07b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:49.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.330+0000 7f5a09600640 1 -- 172.21.15.100:0/1920380973 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7f5a000bb9e0 con 0x7f5a0c0ad9c0 2024-09-17T04:28:49.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.331+0000 7f5a10a00640 1 --2- 172.21.15.100:0/1920380973 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f59ec078ef0 0x7f59ec07b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:49.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.331+0000 7f5a10a00640 1 --2- 172.21.15.100:0/1920380973 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f59ec078ef0 0x7f59ec07b400 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009ac0 tx=0x7f59f8005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:49.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.334+0000 7f5a09600640 1 -- 172.21.15.100:0/1920380973 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f5a00083950 con 0x7f5a0c0ad9c0 2024-09-17T04:28:49.366 INFO:tasks.ceph.mds.c.smithi100.stderr:2024-09-17T04:28:49.365+0000 7fb410000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T04:28:49.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.514+0000 7f5a12800640 1 -- 172.21.15.100:0/1920380973 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5a0c15db60 con 0x7f59ec078ef0 2024-09-17T04:28:49.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.519+0000 7f5a09600640 1 -- 172.21.15.100:0/1920380973 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248568 (secure 0 0 0) 0x7f59f8011b00 con 0x7f59ec078ef0 2024-09-17T04:28:49.522 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:28:49.524 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:28:49.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.523+0000 7f5a12800640 1 -- 172.21.15.100:0/1920380973 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f59ec078ef0 msgr2=0x7f59ec07b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:49.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.523+0000 7f5a12800640 1 --2- 172.21.15.100:0/1920380973 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f59ec078ef0 0x7f59ec07b400 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009ac0 tx=0x7f59f8005f90 comp rx=0 tx=0).stop 2024-09-17T04:28:49.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.523+0000 7f5a12800640 1 -- 172.21.15.100:0/1920380973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a0c0ad9c0 msgr2=0x7f5a0c0ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:49.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.523+0000 7f5a12800640 1 --2- 172.21.15.100:0/1920380973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a0c0ad9c0 0x7f5a0c0ccac0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5a000099c0 tx=0x7f5a000038a0 comp rx=0 tx=0).stop 2024-09-17T04:28:49.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.523+0000 7f5a12800640 1 -- 172.21.15.100:0/1920380973 shutdown_connections 2024-09-17T04:28:49.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.523+0000 7f5a12800640 1 --2- 172.21.15.100:0/1920380973 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f59ec078ef0 0x7f59ec07b400 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:49.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.523+0000 7f5a12800640 1 --2- 172.21.15.100:0/1920380973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a0c15d670 0x7f5a0c15b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:49.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.523+0000 7f5a12800640 1 --2- 172.21.15.100:0/1920380973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a0c0b2f60 0x7f5a0c14dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:49.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.523+0000 7f5a12800640 1 --2- 172.21.15.100:0/1920380973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a0c0ad9c0 0x7f5a0c0ccac0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:49.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.524+0000 7f5a12800640 1 -- 172.21.15.100:0/1920380973 >> 172.21.15.100:0/1920380973 conn(0x7f5a0c0b1e40 msgr2=0x7f5a0c06f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:49.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.524+0000 7f5a12800640 1 -- 172.21.15.100:0/1920380973 shutdown_connections 2024-09-17T04:28:49.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.524+0000 7f5a12800640 1 -- 172.21.15.100:0/1920380973 wait complete. 2024-09-17T04:28:49.548 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-09-17T04:28:48.041281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590458,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":105,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":43,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253180,"kb_used_data":7812,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721540,"statfs":{"total":773094113280,"available":772834856960,"internally_reserved":0,"allocated":7999488,"data_stored":4013546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":251055668},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.890116"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.178663+0000","last_change":"2024-09-17T04:28:44.178546+0000","last_active":"2024-09-17T04:28:44.178663+0000","last_peered":"2024-09-17T04:28:44.178663+0000","last_clean":"2024-09-17T04:28:44.178663+0000","last_became_active":"2024-09-17T04:28:44.178226+0000","last_became_peered":"2024-09-17T04:28:44.178226+0000","last_unstale":"2024-09-17T04:28:44.178663+0000","last_undegraded":"2024-09-17T04:28:44.178663+0000","last_fullsized":"2024-09-17T04:28:44.178663+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:32.865614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.176591+0000","last_change":"2024-09-17T04:28:44.176591+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.176591+0000","last_undegraded":"2024-09-17T04:28:44.176591+0000","last_fullsized":"2024-09-17T04:28:44.176591+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836882+0000","last_change":"2024-09-17T04:28:44.187901+0000","last_active":"2024-09-17T04:28:45.836882+0000","last_peered":"2024-09-17T04:28:45.836882+0000","last_clean":"2024-09-17T04:28:45.836882+0000","last_became_active":"2024-09-17T04:28:44.187552+0000","last_became_peered":"2024-09-17T04:28:44.187552+0000","last_unstale":"2024-09-17T04:28:45.836882+0000","last_undegraded":"2024-09-17T04:28:45.836882+0000","last_fullsized":"2024-09-17T04:28:45.836882+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:33.519054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.158403+0000","last_change":"2024-09-17T04:28:43.158403+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.158403+0000","last_undegraded":"2024-09-17T04:28:43.158403+0000","last_fullsized":"2024-09-17T04:28:43.158403+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.160752+0000","last_change":"2024-09-17T04:28:43.160752+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.160752+0000","last_undegraded":"2024-09-17T04:28:43.160752+0000","last_fullsized":"2024-09-17T04:28:43.160752+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.172729+0000","last_change":"2024-09-17T04:28:44.172729+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.172729+0000","last_undegraded":"2024-09-17T04:28:44.172729+0000","last_fullsized":"2024-09-17T04:28:44.172729+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.180383+0000","last_change":"2024-09-17T04:28:44.180281+0000","last_active":"2024-09-17T04:28:44.180383+0000","last_peered":"2024-09-17T04:28:44.180383+0000","last_clean":"2024-09-17T04:28:44.180383+0000","last_became_active":"2024-09-17T04:28:44.179971+0000","last_became_peered":"2024-09-17T04:28:44.179971+0000","last_unstale":"2024-09-17T04:28:44.180383+0000","last_undegraded":"2024-09-17T04:28:44.180383+0000","last_fullsized":"2024-09-17T04:28:44.180383+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:40.899996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.174265+0000","last_change":"2024-09-17T04:28:44.174265+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.174265+0000","last_undegraded":"2024-09-17T04:28:44.174265+0000","last_fullsized":"2024-09-17T04:28:44.174265+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":["6","1"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.161096+0000","last_change":"2024-09-17T04:28:43.161096+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.161096+0000","last_undegraded":"2024-09-17T04:28:43.161096+0000","last_fullsized":"2024-09-17T04:28:43.161096+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.174630+0000","last_change":"2024-09-17T04:28:44.166800+0000","last_active":"2024-09-17T04:28:44.174630+0000","last_peered":"2024-09-17T04:28:44.174630+0000","last_clean":"2024-09-17T04:28:44.174630+0000","last_became_active":"2024-09-17T04:28:44.166256+0000","last_became_peered":"2024-09-17T04:28:44.166256+0000","last_unstale":"2024-09-17T04:28:44.174630+0000","last_undegraded":"2024-09-17T04:28:44.174630+0000","last_fullsized":"2024-09-17T04:28:44.174630+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:49:09.101261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.168359+0000","last_change":"2024-09-17T04:28:44.168359+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.168359+0000","last_undegraded":"2024-09-17T04:28:44.168359+0000","last_fullsized":"2024-09-17T04:28:44.168359+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":["7","3"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.157976+0000","last_change":"2024-09-17T04:28:43.157976+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.157976+0000","last_undegraded":"2024-09-17T04:28:43.157976+0000","last_fullsized":"2024-09-17T04:28:43.157976+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.164794+0000","last_change":"2024-09-17T04:28:43.164794+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.164794+0000","last_undegraded":"2024-09-17T04:28:43.164794+0000","last_fullsized":"2024-09-17T04:28:43.164794+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.159533+0000","last_change":"2024-09-17T04:28:43.159533+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.159533+0000","last_undegraded":"2024-09-17T04:28:43.159533+0000","last_fullsized":"2024-09-17T04:28:43.159533+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.170926+0000","last_change":"2024-09-17T04:28:43.170926+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.170926+0000","last_undegraded":"2024-09-17T04:28:43.170926+0000","last_fullsized":"2024-09-17T04:28:43.170926+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.191866+0000","last_change":"2024-09-17T04:28:44.186823+0000","last_active":"2024-09-17T04:28:44.191866+0000","last_peered":"2024-09-17T04:28:44.191866+0000","last_clean":"2024-09-17T04:28:44.191866+0000","last_became_active":"2024-09-17T04:28:44.186451+0000","last_became_peered":"2024-09-17T04:28:44.186451+0000","last_unstale":"2024-09-17T04:28:44.191866+0000","last_undegraded":"2024-09-17T04:28:44.191866+0000","last_fullsized":"2024-09-17T04:28:44.191866+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:56:31.348648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.159764+0000","last_change":"2024-09-17T04:28:43.159764+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.159764+0000","last_undegraded":"2024-09-17T04:28:43.159764+0000","last_fullsized":"2024-09-17T04:28:43.159764+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.157003+0000","last_change":"2024-09-17T04:28:43.157003+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.157003+0000","last_undegraded":"2024-09-17T04:28:43.157003+0000","last_fullsized":"2024-09-17T04:28:43.157003+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.182609+0000","last_change":"2024-09-17T04:28:44.182489+0000","last_active":"2024-09-17T04:28:44.182609+0000","last_peered":"2024-09-17T04:28:44.182609+0000","last_clean":"2024-09-17T04:28:44.182609+0000","last_became_active":"2024-09-17T04:28:44.182178+0000","last_became_peered":"2024-09-17T04:28:44.182178+0000","last_unstale":"2024-09-17T04:28:44.182609+0000","last_undegraded":"2024-09-17T04:28:44.182609+0000","last_fullsized":"2024-09-17T04:28:44.182609+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:14.506557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.166386+0000","last_change":"2024-09-17T04:28:43.166386+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.166386+0000","last_undegraded":"2024-09-17T04:28:43.166386+0000","last_fullsized":"2024-09-17T04:28:43.166386+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.175852+0000","last_change":"2024-09-17T04:28:44.172195+0000","last_active":"2024-09-17T04:28:44.175852+0000","last_peered":"2024-09-17T04:28:44.175852+0000","last_clean":"2024-09-17T04:28:44.175852+0000","last_became_active":"2024-09-17T04:28:44.171667+0000","last_became_peered":"2024-09-17T04:28:44.171667+0000","last_unstale":"2024-09-17T04:28:44.175852+0000","last_undegraded":"2024-09-17T04:28:44.175852+0000","last_fullsized":"2024-09-17T04:28:44.175852+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:25:20.752959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.185891+0000","last_change":"2024-09-17T04:28:44.176739+0000","last_active":"2024-09-17T04:28:44.185891+0000","last_peered":"2024-09-17T04:28:44.185891+0000","last_clean":"2024-09-17T04:28:44.185891+0000","last_became_active":"2024-09-17T04:28:44.169094+0000","last_became_peered":"2024-09-17T04:28:44.169094+0000","last_unstale":"2024-09-17T04:28:44.185891+0000","last_undegraded":"2024-09-17T04:28:44.185891+0000","last_fullsized":"2024-09-17T04:28:44.185891+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:30:41.503062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.168918+0000","last_change":"2024-09-17T04:28:44.168918+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.168918+0000","last_undegraded":"2024-09-17T04:28:44.168918+0000","last_fullsized":"2024-09-17T04:28:44.168918+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.168388+0000","last_change":"2024-09-17T04:28:44.168388+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.168388+0000","last_undegraded":"2024-09-17T04:28:44.168388+0000","last_fullsized":"2024-09-17T04:28:44.168388+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.175800+0000","last_change":"2024-09-17T04:28:44.170440+0000","last_active":"2024-09-17T04:28:44.175800+0000","last_peered":"2024-09-17T04:28:44.175800+0000","last_clean":"2024-09-17T04:28:44.175800+0000","last_became_active":"2024-09-17T04:28:44.169980+0000","last_became_peered":"2024-09-17T04:28:44.169980+0000","last_unstale":"2024-09-17T04:28:44.175800+0000","last_undegraded":"2024-09-17T04:28:44.175800+0000","last_fullsized":"2024-09-17T04:28:44.175800+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:53.396145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.160892+0000","last_change":"2024-09-17T04:28:44.160892+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.160892+0000","last_undegraded":"2024-09-17T04:28:44.160892+0000","last_fullsized":"2024-09-17T04:28:44.160892+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836922+0000","last_change":"2024-09-17T04:28:44.174317+0000","last_active":"2024-09-17T04:28:45.836922+0000","last_peered":"2024-09-17T04:28:45.836922+0000","last_clean":"2024-09-17T04:28:45.836922+0000","last_became_active":"2024-09-17T04:28:44.173878+0000","last_became_peered":"2024-09-17T04:28:44.173878+0000","last_unstale":"2024-09-17T04:28:45.836922+0000","last_undegraded":"2024-09-17T04:28:45.836922+0000","last_fullsized":"2024-09-17T04:28:45.836922+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:07.283249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.177201+0000","last_change":"2024-09-17T04:28:44.170671+0000","last_active":"2024-09-17T04:28:44.177201+0000","last_peered":"2024-09-17T04:28:44.177201+0000","last_clean":"2024-09-17T04:28:44.177201+0000","last_became_active":"2024-09-17T04:28:44.169266+0000","last_became_peered":"2024-09-17T04:28:44.169266+0000","last_unstale":"2024-09-17T04:28:44.177201+0000","last_undegraded":"2024-09-17T04:28:44.177201+0000","last_fullsized":"2024-09-17T04:28:44.177201+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:53.851004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836607+0000","last_change":"2024-09-17T04:28:44.188197+0000","last_active":"2024-09-17T04:28:45.836607+0000","last_peered":"2024-09-17T04:28:45.836607+0000","last_clean":"2024-09-17T04:28:45.836607+0000","last_became_active":"2024-09-17T04:28:44.187879+0000","last_became_peered":"2024-09-17T04:28:44.187879+0000","last_unstale":"2024-09-17T04:28:45.836607+0000","last_undegraded":"2024-09-17T04:28:45.836607+0000","last_fullsized":"2024-09-17T04:28:45.836607+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:28:18.827139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.182803+0000","last_change":"2024-09-17T04:28:44.182642+0000","last_active":"2024-09-17T04:28:44.182803+0000","last_peered":"2024-09-17T04:28:44.182803+0000","last_clean":"2024-09-17T04:28:44.182803+0000","last_became_active":"2024-09-17T04:28:44.182271+0000","last_became_peered":"2024-09-17T04:28:44.182271+0000","last_unstale":"2024-09-17T04:28:44.182803+0000","last_undegraded":"2024-09-17T04:28:44.182803+0000","last_fullsized":"2024-09-17T04:28:44.182803+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:08.481224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.168668+0000","last_change":"2024-09-17T04:28:44.166100+0000","last_active":"2024-09-17T04:28:44.168668+0000","last_peered":"2024-09-17T04:28:44.168668+0000","last_clean":"2024-09-17T04:28:44.168668+0000","last_became_active":"2024-09-17T04:28:44.165456+0000","last_became_peered":"2024-09-17T04:28:44.165456+0000","last_unstale":"2024-09-17T04:28:44.168668+0000","last_undegraded":"2024-09-17T04:28:44.168668+0000","last_fullsized":"2024-09-17T04:28:44.168668+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:20.771270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.173852+0000","last_change":"2024-09-17T04:28:44.173852+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.173852+0000","last_undegraded":"2024-09-17T04:28:44.173852+0000","last_fullsized":"2024-09-17T04:28:44.173852+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":["6","3"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.168046+0000","last_change":"2024-09-17T04:28:44.168046+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.168046+0000","last_undegraded":"2024-09-17T04:28:44.168046+0000","last_fullsized":"2024-09-17T04:28:44.168046+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.169200+0000","last_change":"2024-09-17T04:28:43.181281+0000","last_active":"2024-09-17T04:28:44.169200+0000","last_peered":"2024-09-17T04:28:44.169200+0000","last_clean":"2024-09-17T04:28:44.169200+0000","last_became_active":"2024-09-17T04:28:43.180960+0000","last_became_peered":"2024-09-17T04:28:43.180960+0000","last_unstale":"2024-09-17T04:28:44.169200+0000","last_undegraded":"2024-09-17T04:28:44.169200+0000","last_fullsized":"2024-09-17T04:28:44.169200+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:19.766145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.159258+0000","last_change":"2024-09-17T04:28:43.159258+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.159258+0000","last_undegraded":"2024-09-17T04:28:43.159258+0000","last_fullsized":"2024-09-17T04:28:43.159258+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":4,"reported_epoch":71,"state":"creating+activating","last_fresh":"2024-09-17T04:28:43.170907+0000","last_change":"2024-09-17T04:28:43.165532+0000","last_active":"2024-09-17T04:28:42.144673+0000","last_peered":"2024-09-17T04:28:42.144673+0000","last_clean":"2024-09-17T04:28:42.144673+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.170907+0000","last_undegraded":"2024-09-17T04:28:43.170907+0000","last_fullsized":"2024-09-17T04:28:43.170907+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.173734+0000","last_change":"2024-09-17T04:28:44.173734+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.173734+0000","last_undegraded":"2024-09-17T04:28:44.173734+0000","last_fullsized":"2024-09-17T04:28:44.173734+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.163300+0000","last_change":"2024-09-17T04:28:43.187020+0000","last_active":"2024-09-17T04:28:44.163300+0000","last_peered":"2024-09-17T04:28:44.163300+0000","last_clean":"2024-09-17T04:28:44.163300+0000","last_became_active":"2024-09-17T04:28:43.186227+0000","last_became_peered":"2024-09-17T04:28:43.186227+0000","last_unstale":"2024-09-17T04:28:44.163300+0000","last_undegraded":"2024-09-17T04:28:44.163300+0000","last_fullsized":"2024-09-17T04:28:44.163300+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:06.525929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.167817+0000","last_change":"2024-09-17T04:28:44.167817+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.167817+0000","last_undegraded":"2024-09-17T04:28:44.167817+0000","last_fullsized":"2024-09-17T04:28:44.167817+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":["7","0"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.178386+0000","last_change":"2024-09-17T04:28:43.183969+0000","last_active":"2024-09-17T04:28:44.178386+0000","last_peered":"2024-09-17T04:28:44.178386+0000","last_clean":"2024-09-17T04:28:44.178386+0000","last_became_active":"2024-09-17T04:28:43.183603+0000","last_became_peered":"2024-09-17T04:28:43.183603+0000","last_unstale":"2024-09-17T04:28:44.178386+0000","last_undegraded":"2024-09-17T04:28:44.178386+0000","last_fullsized":"2024-09-17T04:28:44.178386+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:47.866148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.183433+0000","last_change":"2024-09-17T04:28:44.183332+0000","last_active":"2024-09-17T04:28:44.183433+0000","last_peered":"2024-09-17T04:28:44.183433+0000","last_clean":"2024-09-17T04:28:44.183433+0000","last_became_active":"2024-09-17T04:28:44.175938+0000","last_became_peered":"2024-09-17T04:28:44.175938+0000","last_unstale":"2024-09-17T04:28:44.183433+0000","last_undegraded":"2024-09-17T04:28:44.183433+0000","last_fullsized":"2024-09-17T04:28:44.183433+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:02.045028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"72'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836677+0000","last_change":"2024-09-17T04:28:43.171042+0000","last_active":"2024-09-17T04:28:45.836677+0000","last_peered":"2024-09-17T04:28:45.836677+0000","last_clean":"2024-09-17T04:28:45.836677+0000","last_became_active":"2024-09-17T04:28:43.170733+0000","last_became_peered":"2024-09-17T04:28:43.170733+0000","last_unstale":"2024-09-17T04:28:45.836677+0000","last_undegraded":"2024-09-17T04:28:45.836677+0000","last_fullsized":"2024-09-17T04:28:45.836677+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:33:35.104722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.185431+0000","last_change":"2024-09-17T04:28:44.181651+0000","last_active":"2024-09-17T04:28:44.185431+0000","last_peered":"2024-09-17T04:28:44.185431+0000","last_clean":"2024-09-17T04:28:44.185431+0000","last_became_active":"2024-09-17T04:28:44.181249+0000","last_became_peered":"2024-09-17T04:28:44.181249+0000","last_unstale":"2024-09-17T04:28:44.185431+0000","last_undegraded":"2024-09-17T04:28:44.185431+0000","last_fullsized":"2024-09-17T04:28:44.185431+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:55:58.969567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.180193+0000","last_change":"2024-09-17T04:28:43.181324+0000","last_active":"2024-09-17T04:28:44.180193+0000","last_peered":"2024-09-17T04:28:44.180193+0000","last_clean":"2024-09-17T04:28:44.180193+0000","last_became_active":"2024-09-17T04:28:43.180647+0000","last_became_peered":"2024-09-17T04:28:43.180647+0000","last_unstale":"2024-09-17T04:28:44.180193+0000","last_undegraded":"2024-09-17T04:28:44.180193+0000","last_fullsized":"2024-09-17T04:28:44.180193+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:01:55.909462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.159075+0000","last_change":"2024-09-17T04:28:43.159075+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.159075+0000","last_undegraded":"2024-09-17T04:28:43.159075+0000","last_fullsized":"2024-09-17T04:28:43.159075+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T04:28:43.171114+0000","last_change":"2024-09-17T04:28:43.163736+0000","last_active":"2024-09-17T04:28:43.171114+0000","last_peered":"2024-09-17T04:28:43.171114+0000","last_clean":"2024-09-17T04:28:43.171114+0000","last_became_active":"2024-09-17T04:28:43.163431+0000","last_became_peered":"2024-09-17T04:28:43.163431+0000","last_unstale":"2024-09-17T04:28:43.171114+0000","last_undegraded":"2024-09-17T04:28:43.171114+0000","last_fullsized":"2024-09-17T04:28:43.171114+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:57.986932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.168710+0000","last_change":"2024-09-17T04:28:43.168710+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.168710+0000","last_undegraded":"2024-09-17T04:28:43.168710+0000","last_fullsized":"2024-09-17T04:28:43.168710+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T04:28:43.163264+0000","last_change":"2024-09-17T04:28:43.163133+0000","last_active":"2024-09-17T04:28:43.163264+0000","last_peered":"2024-09-17T04:28:43.163264+0000","last_clean":"2024-09-17T04:28:43.163264+0000","last_became_active":"2024-09-17T04:28:43.162404+0000","last_became_peered":"2024-09-17T04:28:43.162404+0000","last_unstale":"2024-09-17T04:28:43.163264+0000","last_undegraded":"2024-09-17T04:28:43.163264+0000","last_fullsized":"2024-09-17T04:28:43.163264+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:47.332401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.183082+0000","last_change":"2024-09-17T04:28:44.175084+0000","last_active":"2024-09-17T04:28:44.183082+0000","last_peered":"2024-09-17T04:28:44.183082+0000","last_clean":"2024-09-17T04:28:44.183082+0000","last_became_active":"2024-09-17T04:28:44.174333+0000","last_became_peered":"2024-09-17T04:28:44.174333+0000","last_unstale":"2024-09-17T04:28:44.183082+0000","last_undegraded":"2024-09-17T04:28:44.183082+0000","last_fullsized":"2024-09-17T04:28:44.183082+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:52.267667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.164778+0000","last_change":"2024-09-17T04:28:43.174543+0000","last_active":"2024-09-17T04:28:44.164778+0000","last_peered":"2024-09-17T04:28:44.164778+0000","last_clean":"2024-09-17T04:28:44.164778+0000","last_became_active":"2024-09-17T04:28:43.174199+0000","last_became_peered":"2024-09-17T04:28:43.174199+0000","last_unstale":"2024-09-17T04:28:44.164778+0000","last_undegraded":"2024-09-17T04:28:44.164778+0000","last_fullsized":"2024-09-17T04:28:44.164778+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:22.807921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.168793+0000","last_change":"2024-09-17T04:28:44.168793+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.168793+0000","last_undegraded":"2024-09-17T04:28:44.168793+0000","last_fullsized":"2024-09-17T04:28:44.168793+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.179651+0000","last_change":"2024-09-17T04:28:43.185414+0000","last_active":"2024-09-17T04:28:44.179651+0000","last_peered":"2024-09-17T04:28:44.179651+0000","last_clean":"2024-09-17T04:28:44.179651+0000","last_became_active":"2024-09-17T04:28:43.184969+0000","last_became_peered":"2024-09-17T04:28:43.184969+0000","last_unstale":"2024-09-17T04:28:44.179651+0000","last_undegraded":"2024-09-17T04:28:44.179651+0000","last_fullsized":"2024-09-17T04:28:44.179651+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:58.068125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.173127+0000","last_change":"2024-09-17T04:28:44.173127+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.173127+0000","last_undegraded":"2024-09-17T04:28:44.173127+0000","last_fullsized":"2024-09-17T04:28:44.173127+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"72'3","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836958+0000","last_change":"2024-09-17T04:28:43.172982+0000","last_active":"2024-09-17T04:28:45.836958+0000","last_peered":"2024-09-17T04:28:45.836958+0000","last_clean":"2024-09-17T04:28:45.836958+0000","last_became_active":"2024-09-17T04:28:43.172659+0000","last_became_peered":"2024-09-17T04:28:43.172659+0000","last_unstale":"2024-09-17T04:28:45.836958+0000","last_undegraded":"2024-09-17T04:28:45.836958+0000","last_fullsized":"2024-09-17T04:28:45.836958+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:15.908868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.160386+0000","last_change":"2024-09-17T04:28:43.160386+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.160386+0000","last_undegraded":"2024-09-17T04:28:43.160386+0000","last_fullsized":"2024-09-17T04:28:43.160386+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T04:28:43.175199+0000","last_change":"2024-09-17T04:28:43.168326+0000","last_active":"2024-09-17T04:28:43.175199+0000","last_peered":"2024-09-17T04:28:43.175199+0000","last_clean":"2024-09-17T04:28:43.175199+0000","last_became_active":"2024-09-17T04:28:43.167133+0000","last_became_peered":"2024-09-17T04:28:43.167133+0000","last_unstale":"2024-09-17T04:28:43.175199+0000","last_undegraded":"2024-09-17T04:28:43.175199+0000","last_fullsized":"2024-09-17T04:28:43.175199+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:02.401866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.180070+0000","last_change":"2024-09-17T04:28:44.179971+0000","last_active":"2024-09-17T04:28:44.180070+0000","last_peered":"2024-09-17T04:28:44.180070+0000","last_clean":"2024-09-17T04:28:44.180070+0000","last_became_active":"2024-09-17T04:28:44.179660+0000","last_became_peered":"2024-09-17T04:28:44.179660+0000","last_unstale":"2024-09-17T04:28:44.180070+0000","last_undegraded":"2024-09-17T04:28:44.180070+0000","last_fullsized":"2024-09-17T04:28:44.180070+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:18.874211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":3,"reported_epoch":71,"state":"creating+activating","last_fresh":"2024-09-17T04:28:43.156870+0000","last_change":"2024-09-17T04:28:43.156870+0000","last_active":"2024-09-17T04:28:42.144673+0000","last_peered":"2024-09-17T04:28:42.144673+0000","last_clean":"2024-09-17T04:28:42.144673+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.156870+0000","last_undegraded":"2024-09-17T04:28:43.156870+0000","last_fullsized":"2024-09-17T04:28:43.156870+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":["1","7"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.179314+0000","last_change":"2024-09-17T04:28:44.171317+0000","last_active":"2024-09-17T04:28:44.179314+0000","last_peered":"2024-09-17T04:28:44.179314+0000","last_clean":"2024-09-17T04:28:44.179314+0000","last_became_active":"2024-09-17T04:28:44.170995+0000","last_became_peered":"2024-09-17T04:28:44.170995+0000","last_unstale":"2024-09-17T04:28:44.179314+0000","last_undegraded":"2024-09-17T04:28:44.179314+0000","last_fullsized":"2024-09-17T04:28:44.179314+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:32.865614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.180446+0000","last_change":"2024-09-17T04:28:43.180632+0000","last_active":"2024-09-17T04:28:44.180446+0000","last_peered":"2024-09-17T04:28:44.180446+0000","last_clean":"2024-09-17T04:28:44.180446+0000","last_became_active":"2024-09-17T04:28:43.180146+0000","last_became_peered":"2024-09-17T04:28:43.180146+0000","last_unstale":"2024-09-17T04:28:44.180446+0000","last_undegraded":"2024-09-17T04:28:44.180446+0000","last_fullsized":"2024-09-17T04:28:44.180446+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:03.917176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.168817+0000","last_change":"2024-09-17T04:28:43.168817+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.168817+0000","last_undegraded":"2024-09-17T04:28:43.168817+0000","last_fullsized":"2024-09-17T04:28:43.168817+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.161973+0000","last_change":"2024-09-17T04:28:43.175788+0000","last_active":"2024-09-17T04:28:44.161973+0000","last_peered":"2024-09-17T04:28:44.161973+0000","last_clean":"2024-09-17T04:28:44.161973+0000","last_became_active":"2024-09-17T04:28:43.175486+0000","last_became_peered":"2024-09-17T04:28:43.175486+0000","last_unstale":"2024-09-17T04:28:44.161973+0000","last_undegraded":"2024-09-17T04:28:44.161973+0000","last_fullsized":"2024-09-17T04:28:44.161973+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:56:30.343523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.158958+0000","last_change":"2024-09-17T04:28:43.158958+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.158958+0000","last_undegraded":"2024-09-17T04:28:43.158958+0000","last_fullsized":"2024-09-17T04:28:43.158958+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.162942+0000","last_change":"2024-09-17T04:28:43.175547+0000","last_active":"2024-09-17T04:28:44.162942+0000","last_peered":"2024-09-17T04:28:44.162942+0000","last_clean":"2024-09-17T04:28:44.162942+0000","last_became_active":"2024-09-17T04:28:43.175224+0000","last_became_peered":"2024-09-17T04:28:43.175224+0000","last_unstale":"2024-09-17T04:28:44.162942+0000","last_undegraded":"2024-09-17T04:28:44.162942+0000","last_fullsized":"2024-09-17T04:28:44.162942+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:09.135418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.185472+0000","last_change":"2024-09-17T04:28:44.183124+0000","last_active":"2024-09-17T04:28:44.185472+0000","last_peered":"2024-09-17T04:28:44.185472+0000","last_clean":"2024-09-17T04:28:44.185472+0000","last_became_active":"2024-09-17T04:28:44.181974+0000","last_became_peered":"2024-09-17T04:28:44.181974+0000","last_unstale":"2024-09-17T04:28:44.185472+0000","last_undegraded":"2024-09-17T04:28:44.185472+0000","last_fullsized":"2024-09-17T04:28:44.185472+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:21.789405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.173105+0000","last_change":"2024-09-17T04:28:43.183827+0000","last_active":"2024-09-17T04:28:44.173105+0000","last_peered":"2024-09-17T04:28:44.173105+0000","last_clean":"2024-09-17T04:28:44.173105+0000","last_became_active":"2024-09-17T04:28:43.183492+0000","last_became_peered":"2024-09-17T04:28:43.183492+0000","last_unstale":"2024-09-17T04:28:44.173105+0000","last_undegraded":"2024-09-17T04:28:44.173105+0000","last_fullsized":"2024-09-17T04:28:44.173105+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:02.478321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.169202+0000","last_change":"2024-09-17T04:28:44.169202+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.169202+0000","last_undegraded":"2024-09-17T04:28:44.169202+0000","last_fullsized":"2024-09-17T04:28:44.169202+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":["7","1"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.163486+0000","last_change":"2024-09-17T04:28:43.185602+0000","last_active":"2024-09-17T04:28:44.163486+0000","last_peered":"2024-09-17T04:28:44.163486+0000","last_clean":"2024-09-17T04:28:44.163486+0000","last_became_active":"2024-09-17T04:28:43.185261+0000","last_became_peered":"2024-09-17T04:28:43.185261+0000","last_unstale":"2024-09-17T04:28:44.163486+0000","last_undegraded":"2024-09-17T04:28:44.163486+0000","last_fullsized":"2024-09-17T04:28:44.163486+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:48.674587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.157002+0000","last_change":"2024-09-17T04:28:43.157002+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.157002+0000","last_undegraded":"2024-09-17T04:28:43.157002+0000","last_fullsized":"2024-09-17T04:28:43.157002+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.171911+0000","last_change":"2024-09-17T04:28:43.182926+0000","last_active":"2024-09-17T04:28:44.171911+0000","last_peered":"2024-09-17T04:28:44.171911+0000","last_clean":"2024-09-17T04:28:44.171911+0000","last_became_active":"2024-09-17T04:28:43.179719+0000","last_became_peered":"2024-09-17T04:28:43.179719+0000","last_unstale":"2024-09-17T04:28:44.171911+0000","last_undegraded":"2024-09-17T04:28:44.171911+0000","last_fullsized":"2024-09-17T04:28:44.171911+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:28.997811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.159593+0000","last_change":"2024-09-17T04:28:43.159593+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.159593+0000","last_undegraded":"2024-09-17T04:28:43.159593+0000","last_fullsized":"2024-09-17T04:28:43.159593+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.168165+0000","last_change":"2024-09-17T04:28:43.184237+0000","last_active":"2024-09-17T04:28:44.168165+0000","last_peered":"2024-09-17T04:28:44.168165+0000","last_clean":"2024-09-17T04:28:44.168165+0000","last_became_active":"2024-09-17T04:28:43.183879+0000","last_became_peered":"2024-09-17T04:28:43.183879+0000","last_unstale":"2024-09-17T04:28:44.168165+0000","last_undegraded":"2024-09-17T04:28:44.168165+0000","last_fullsized":"2024-09-17T04:28:44.168165+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:27.493169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836817+0000","last_change":"2024-09-17T04:28:44.179419+0000","last_active":"2024-09-17T04:28:45.836817+0000","last_peered":"2024-09-17T04:28:45.836817+0000","last_clean":"2024-09-17T04:28:45.836817+0000","last_became_active":"2024-09-17T04:28:44.179132+0000","last_became_peered":"2024-09-17T04:28:44.179132+0000","last_unstale":"2024-09-17T04:28:45.836817+0000","last_undegraded":"2024-09-17T04:28:45.836817+0000","last_fullsized":"2024-09-17T04:28:45.836817+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:47:46.546993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.180713+0000","last_change":"2024-09-17T04:28:43.181357+0000","last_active":"2024-09-17T04:28:44.180713+0000","last_peered":"2024-09-17T04:28:44.180713+0000","last_clean":"2024-09-17T04:28:44.180713+0000","last_became_active":"2024-09-17T04:28:43.181053+0000","last_became_peered":"2024-09-17T04:28:43.181053+0000","last_unstale":"2024-09-17T04:28:44.180713+0000","last_undegraded":"2024-09-17T04:28:44.180713+0000","last_fullsized":"2024-09-17T04:28:44.180713+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:53:34.585332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836565+0000","last_change":"2024-09-17T04:28:44.188889+0000","last_active":"2024-09-17T04:28:45.836565+0000","last_peered":"2024-09-17T04:28:45.836565+0000","last_clean":"2024-09-17T04:28:45.836565+0000","last_became_active":"2024-09-17T04:28:44.188524+0000","last_became_peered":"2024-09-17T04:28:44.188524+0000","last_unstale":"2024-09-17T04:28:45.836565+0000","last_undegraded":"2024-09-17T04:28:45.836565+0000","last_fullsized":"2024-09-17T04:28:45.836565+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:17.290337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.162271+0000","last_change":"2024-09-17T04:28:43.184098+0000","last_active":"2024-09-17T04:28:44.162271+0000","last_peered":"2024-09-17T04:28:44.162271+0000","last_clean":"2024-09-17T04:28:44.162271+0000","last_became_active":"2024-09-17T04:28:43.178392+0000","last_became_peered":"2024-09-17T04:28:43.178392+0000","last_unstale":"2024-09-17T04:28:44.162271+0000","last_undegraded":"2024-09-17T04:28:44.162271+0000","last_fullsized":"2024-09-17T04:28:44.162271+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:26.238316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836874+0000","last_change":"2024-09-17T04:28:44.186208+0000","last_active":"2024-09-17T04:28:45.836874+0000","last_peered":"2024-09-17T04:28:45.836874+0000","last_clean":"2024-09-17T04:28:45.836874+0000","last_became_active":"2024-09-17T04:28:44.185834+0000","last_became_peered":"2024-09-17T04:28:44.185834+0000","last_unstale":"2024-09-17T04:28:45.836874+0000","last_undegraded":"2024-09-17T04:28:45.836874+0000","last_fullsized":"2024-09-17T04:28:45.836874+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:24:29.707796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":3,"reported_epoch":71,"state":"creating+activating","last_fresh":"2024-09-17T04:28:43.169387+0000","last_change":"2024-09-17T04:28:43.169387+0000","last_active":"2024-09-17T04:28:42.144673+0000","last_peered":"2024-09-17T04:28:42.144673+0000","last_clean":"2024-09-17T04:28:42.144673+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.169387+0000","last_undegraded":"2024-09-17T04:28:43.169387+0000","last_fullsized":"2024-09-17T04:28:43.169387+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.168884+0000","last_change":"2024-09-17T04:28:43.168884+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.168884+0000","last_undegraded":"2024-09-17T04:28:43.168884+0000","last_fullsized":"2024-09-17T04:28:43.168884+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.171119+0000","last_change":"2024-09-17T04:28:43.183089+0000","last_active":"2024-09-17T04:28:44.171119+0000","last_peered":"2024-09-17T04:28:44.171119+0000","last_clean":"2024-09-17T04:28:44.171119+0000","last_became_active":"2024-09-17T04:28:43.179747+0000","last_became_peered":"2024-09-17T04:28:43.179747+0000","last_unstale":"2024-09-17T04:28:44.171119+0000","last_undegraded":"2024-09-17T04:28:44.171119+0000","last_fullsized":"2024-09-17T04:28:44.171119+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:08.291909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.183882+0000","last_change":"2024-09-17T04:28:44.179171+0000","last_active":"2024-09-17T04:28:44.183882+0000","last_peered":"2024-09-17T04:28:44.183882+0000","last_clean":"2024-09-17T04:28:44.183882+0000","last_became_active":"2024-09-17T04:28:44.178313+0000","last_became_peered":"2024-09-17T04:28:44.178313+0000","last_unstale":"2024-09-17T04:28:44.183882+0000","last_undegraded":"2024-09-17T04:28:44.183882+0000","last_fullsized":"2024-09-17T04:28:44.183882+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:18.874211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.164248+0000","last_change":"2024-09-17T04:28:43.178629+0000","last_active":"2024-09-17T04:28:44.164248+0000","last_peered":"2024-09-17T04:28:44.164248+0000","last_clean":"2024-09-17T04:28:44.164248+0000","last_became_active":"2024-09-17T04:28:43.176654+0000","last_became_peered":"2024-09-17T04:28:43.176654+0000","last_unstale":"2024-09-17T04:28:44.164248+0000","last_undegraded":"2024-09-17T04:28:44.164248+0000","last_fullsized":"2024-09-17T04:28:44.164248+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:27.493169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.167756+0000","last_change":"2024-09-17T04:28:44.167756+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.167756+0000","last_undegraded":"2024-09-17T04:28:44.167756+0000","last_fullsized":"2024-09-17T04:28:44.167756+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":3,"reported_epoch":71,"state":"creating+activating","last_fresh":"2024-09-17T04:28:43.155487+0000","last_change":"2024-09-17T04:28:43.155487+0000","last_active":"2024-09-17T04:28:42.144673+0000","last_peered":"2024-09-17T04:28:42.144673+0000","last_clean":"2024-09-17T04:28:42.144673+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.155487+0000","last_undegraded":"2024-09-17T04:28:43.155487+0000","last_fullsized":"2024-09-17T04:28:43.155487+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.168219+0000","last_change":"2024-09-17T04:28:44.168219+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.168219+0000","last_undegraded":"2024-09-17T04:28:44.168219+0000","last_fullsized":"2024-09-17T04:28:44.168219+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":3,"reported_epoch":71,"state":"creating+activating","last_fresh":"2024-09-17T04:28:43.165983+0000","last_change":"2024-09-17T04:28:43.165983+0000","last_active":"2024-09-17T04:28:42.144673+0000","last_peered":"2024-09-17T04:28:42.144673+0000","last_clean":"2024-09-17T04:28:42.144673+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.165983+0000","last_undegraded":"2024-09-17T04:28:43.165983+0000","last_fullsized":"2024-09-17T04:28:43.165983+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.158679+0000","last_change":"2024-09-17T04:28:43.158679+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.158679+0000","last_undegraded":"2024-09-17T04:28:43.158679+0000","last_fullsized":"2024-09-17T04:28:43.158679+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":3,"reported_epoch":71,"state":"creating+activating","last_fresh":"2024-09-17T04:28:43.155388+0000","last_change":"2024-09-17T04:28:43.155388+0000","last_active":"2024-09-17T04:28:42.144673+0000","last_peered":"2024-09-17T04:28:42.144673+0000","last_clean":"2024-09-17T04:28:42.144673+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.155388+0000","last_undegraded":"2024-09-17T04:28:43.155388+0000","last_fullsized":"2024-09-17T04:28:43.155388+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.171118+0000","last_change":"2024-09-17T04:28:44.171118+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.171118+0000","last_undegraded":"2024-09-17T04:28:44.171118+0000","last_fullsized":"2024-09-17T04:28:44.171118+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T04:28:43.170474+0000","last_change":"2024-09-17T04:28:43.170357+0000","last_active":"2024-09-17T04:28:43.170474+0000","last_peered":"2024-09-17T04:28:43.170474+0000","last_clean":"2024-09-17T04:28:43.170474+0000","last_became_active":"2024-09-17T04:28:43.170017+0000","last_became_peered":"2024-09-17T04:28:43.170017+0000","last_unstale":"2024-09-17T04:28:43.170474+0000","last_undegraded":"2024-09-17T04:28:43.170474+0000","last_fullsized":"2024-09-17T04:28:43.170474+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:16:22.147657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-17T04:28:43.157808+0000","last_change":"2024-09-17T04:28:43.157808+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:43.157808+0000","last_undegraded":"2024-09-17T04:28:43.157808+0000","last_fullsized":"2024-09-17T04:28:43.157808+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.162044+0000","last_change":"2024-09-17T04:28:43.177322+0000","last_active":"2024-09-17T04:28:44.162044+0000","last_peered":"2024-09-17T04:28:44.162044+0000","last_clean":"2024-09-17T04:28:44.162044+0000","last_became_active":"2024-09-17T04:28:43.177004+0000","last_became_peered":"2024-09-17T04:28:43.177004+0000","last_unstale":"2024-09-17T04:28:44.162044+0000","last_undegraded":"2024-09-17T04:28:44.162044+0000","last_fullsized":"2024-09-17T04:28:44.162044+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:58.922041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":3,"reported_epoch":72,"state":"creating+activating","last_fresh":"2024-09-17T04:28:44.172672+0000","last_change":"2024-09-17T04:28:44.172672+0000","last_active":"2024-09-17T04:28:43.149798+0000","last_peered":"2024-09-17T04:28:43.149798+0000","last_clean":"2024-09-17T04:28:43.149798+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:28:44.172672+0000","last_undegraded":"2024-09-17T04:28:44.172672+0000","last_fullsized":"2024-09-17T04:28:44.172672+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":["6","3"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.183738+0000","last_change":"2024-09-17T04:28:43.183805+0000","last_active":"2024-09-17T04:28:44.183738+0000","last_peered":"2024-09-17T04:28:44.183738+0000","last_clean":"2024-09-17T04:28:44.183738+0000","last_became_active":"2024-09-17T04:28:43.183414+0000","last_became_peered":"2024-09-17T04:28:43.183414+0000","last_unstale":"2024-09-17T04:28:44.183738+0000","last_undegraded":"2024-09-17T04:28:44.183738+0000","last_fullsized":"2024-09-17T04:28:44.183738+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:46:21.622130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836616+0000","last_change":"2024-09-17T04:28:44.189995+0000","last_active":"2024-09-17T04:28:45.836616+0000","last_peered":"2024-09-17T04:28:45.836616+0000","last_clean":"2024-09-17T04:28:45.836616+0000","last_became_active":"2024-09-17T04:28:44.189715+0000","last_became_peered":"2024-09-17T04:28:44.189715+0000","last_unstale":"2024-09-17T04:28:45.836616+0000","last_undegraded":"2024-09-17T04:28:45.836616+0000","last_fullsized":"2024-09-17T04:28:45.836616+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:25.483700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.178186+0000","last_change":"2024-09-17T04:28:43.184519+0000","last_active":"2024-09-17T04:28:44.178186+0000","last_peered":"2024-09-17T04:28:44.178186+0000","last_clean":"2024-09-17T04:28:44.178186+0000","last_became_active":"2024-09-17T04:28:43.183311+0000","last_became_peered":"2024-09-17T04:28:43.183311+0000","last_unstale":"2024-09-17T04:28:44.178186+0000","last_undegraded":"2024-09-17T04:28:44.178186+0000","last_fullsized":"2024-09-17T04:28:44.178186+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:41.963229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":206,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T04:28:44.167592+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:28:44.167592+0000","last_peered":"2024-09-17T04:28:44.167592+0000","last_clean":"2024-09-17T04:28:44.167592+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:28:44.167592+0000","last_undegraded":"2024-09-17T04:28:44.167592+0000","last_fullsized":"2024-09-17T04:28:44.167592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476797,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1445888,"data_stored":945174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476796,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":851968,"data_stored":354806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476797,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32024,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339816,"statfs":{"total":96636764160,"available":96603971584,"internally_reserved":0,"allocated":851968,"data_stored":354806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476797,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32024,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339816,"statfs":{"total":96636764160,"available":96603971584,"internally_reserved":0,"allocated":851968,"data_stored":354806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509500,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":839680,"data_stored":348531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509499,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":839680,"data_stored":348531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509499,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1433600,"data_stored":938899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542205,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32060,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339780,"statfs":{"total":96636764160,"available":96603934720,"internally_reserved":0,"allocated":884736,"data_stored":367993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:28:49.553 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3e is not active+clean 2024-09-17T04:28:49.553 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3e', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 72, 'state': 'creating+activating', 'last_fresh': '2024-09-17T04:28:44.176591+0000', 'last_change': '2024-09-17T04:28:44.176591+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:44.176591+0000', 'last_undegraded': '2024-09-17T04:28:44.176591+0000', 'last_fullsized': '2024-09-17T04:28:44.176591+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': ['6', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T04:28:49.553 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3c is not active+clean 2024-09-17T04:28:49.553 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:28:43.158403+0000', 'last_change': '2024-09-17T04:28:43.158403+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:43.158403+0000', 'last_undegraded': '2024-09-17T04:28:43.158403+0000', 'last_fullsized': '2024-09-17T04:28:43.158403+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T04:28:49.553 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3b is not active+clean 2024-09-17T04:28:49.553 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:28:43.160752+0000', 'last_change': '2024-09-17T04:28:43.160752+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:43.160752+0000', 'last_undegraded': '2024-09-17T04:28:43.160752+0000', 'last_fullsized': '2024-09-17T04:28:43.160752+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T04:28:49.553 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3a is not active+clean 2024-09-17T04:28:49.553 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3a', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 72, 'state': 'creating+activating', 'last_fresh': '2024-09-17T04:28:44.172729+0000', 'last_change': '2024-09-17T04:28:44.172729+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:44.172729+0000', 'last_undegraded': '2024-09-17T04:28:44.172729+0000', 'last_fullsized': '2024-09-17T04:28:44.172729+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': ['6', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T04:28:49.553 INFO:tasks.ceph.ceph_manager.ceph:PG 11.38 is not active+clean 2024-09-17T04:28:49.554 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.38', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 72, 'state': 'creating+activating', 'last_fresh': '2024-09-17T04:28:44.174265+0000', 'last_change': '2024-09-17T04:28:44.174265+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:44.174265+0000', 'last_undegraded': '2024-09-17T04:28:44.174265+0000', 'last_fullsized': '2024-09-17T04:28:44.174265+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': ['6', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T04:28:49.554 INFO:tasks.ceph.ceph_manager.ceph:PG 11.37 is not active+clean 2024-09-17T04:28:49.554 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:28:43.161096+0000', 'last_change': '2024-09-17T04:28:43.161096+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:43.161096+0000', 'last_undegraded': '2024-09-17T04:28:43.161096+0000', 'last_fullsized': '2024-09-17T04:28:43.161096+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T04:28:49.554 INFO:tasks.ceph.ceph_manager.ceph:PG 11.35 is not active+clean 2024-09-17T04:28:49.554 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 72, 'state': 'creating+activating', 'last_fresh': '2024-09-17T04:28:44.168359+0000', 'last_change': '2024-09-17T04:28:44.168359+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:44.168359+0000', 'last_undegraded': '2024-09-17T04:28:44.168359+0000', 'last_fullsized': '2024-09-17T04:28:44.168359+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': ['7', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:28:49.554 INFO:tasks.ceph.ceph_manager.ceph:PG 11.34 is not active+clean 2024-09-17T04:28:49.554 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:28:43.157976+0000', 'last_change': '2024-09-17T04:28:43.157976+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:43.157976+0000', 'last_undegraded': '2024-09-17T04:28:43.157976+0000', 'last_fullsized': '2024-09-17T04:28:43.157976+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T04:28:49.554 INFO:tasks.ceph.ceph_manager.ceph:PG 11.33 is not active+clean 2024-09-17T04:28:49.554 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:28:43.164794+0000', 'last_change': '2024-09-17T04:28:43.164794+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:43.164794+0000', 'last_undegraded': '2024-09-17T04:28:43.164794+0000', 'last_fullsized': '2024-09-17T04:28:43.164794+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T04:28:49.554 INFO:tasks.ceph.ceph_manager.ceph:PG 11.32 is not active+clean 2024-09-17T04:28:49.555 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:28:43.159533+0000', 'last_change': '2024-09-17T04:28:43.159533+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:43.159533+0000', 'last_undegraded': '2024-09-17T04:28:43.159533+0000', 'last_fullsized': '2024-09-17T04:28:43.159533+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T04:28:49.555 INFO:tasks.ceph.ceph_manager.ceph:PG 11.31 is not active+clean 2024-09-17T04:28:49.555 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:28:43.170926+0000', 'last_change': '2024-09-17T04:28:43.170926+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:43.170926+0000', 'last_undegraded': '2024-09-17T04:28:43.170926+0000', 'last_fullsized': '2024-09-17T04:28:43.170926+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T04:28:49.555 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2f is not active+clean 2024-09-17T04:28:49.555 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:28:43.159764+0000', 'last_change': '2024-09-17T04:28:43.159764+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:43.159764+0000', 'last_undegraded': '2024-09-17T04:28:43.159764+0000', 'last_fullsized': '2024-09-17T04:28:43.159764+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T04:28:49.555 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2e is not active+clean 2024-09-17T04:28:49.555 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:28:43.157003+0000', 'last_change': '2024-09-17T04:28:43.157003+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:43.157003+0000', 'last_undegraded': '2024-09-17T04:28:43.157003+0000', 'last_fullsized': '2024-09-17T04:28:43.157003+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T04:28:49.555 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2c is not active+clean 2024-09-17T04:28:49.555 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:28:43.166386+0000', 'last_change': '2024-09-17T04:28:43.166386+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:43.166386+0000', 'last_undegraded': '2024-09-17T04:28:43.166386+0000', 'last_fullsized': '2024-09-17T04:28:43.166386+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T04:28:49.555 INFO:tasks.ceph.ceph_manager.ceph:PG 11.29 is not active+clean 2024-09-17T04:28:49.555 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 72, 'state': 'creating+activating', 'last_fresh': '2024-09-17T04:28:44.168918+0000', 'last_change': '2024-09-17T04:28:44.168918+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:44.168918+0000', 'last_undegraded': '2024-09-17T04:28:44.168918+0000', 'last_fullsized': '2024-09-17T04:28:44.168918+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': ['7', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:28:49.556 INFO:tasks.ceph.ceph_manager.ceph:PG 11.28 is not active+clean 2024-09-17T04:28:49.556 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 72, 'state': 'creating+activating', 'last_fresh': '2024-09-17T04:28:44.168388+0000', 'last_change': '2024-09-17T04:28:44.168388+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:44.168388+0000', 'last_undegraded': '2024-09-17T04:28:44.168388+0000', 'last_fullsized': '2024-09-17T04:28:44.168388+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': ['7', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:28:49.556 INFO:tasks.ceph.ceph_manager.ceph:PG 11.26 is not active+clean 2024-09-17T04:28:49.556 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 72, 'state': 'creating+activating', 'last_fresh': '2024-09-17T04:28:44.160892+0000', 'last_change': '2024-09-17T04:28:44.160892+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:44.160892+0000', 'last_undegraded': '2024-09-17T04:28:44.160892+0000', 'last_fullsized': '2024-09-17T04:28:44.160892+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': ['5', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T04:28:49.556 INFO:tasks.ceph.ceph_manager.ceph:PG 11.20 is not active+clean 2024-09-17T04:28:49.556 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.20', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 72, 'state': 'creating+activating', 'last_fresh': '2024-09-17T04:28:44.173852+0000', 'last_change': '2024-09-17T04:28:44.173852+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:44.173852+0000', 'last_undegraded': '2024-09-17T04:28:44.173852+0000', 'last_fullsized': '2024-09-17T04:28:44.173852+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': ['6', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T04:28:49.556 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1e is not active+clean 2024-09-17T04:28:49.556 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 72, 'state': 'creating+activating', 'last_fresh': '2024-09-17T04:28:44.168046+0000', 'last_change': '2024-09-17T04:28:44.168046+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:44.168046+0000', 'last_undegraded': '2024-09-17T04:28:44.168046+0000', 'last_fullsized': '2024-09-17T04:28:44.168046+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': ['7', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:28:49.556 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1f is not active+clean 2024-09-17T04:28:49.556 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:28:43.159258+0000', 'last_change': '2024-09-17T04:28:43.159258+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:43.159258+0000', 'last_undegraded': '2024-09-17T04:28:43.159258+0000', 'last_fullsized': '2024-09-17T04:28:43.159258+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T04:28:49.556 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1e is not active+clean 2024-09-17T04:28:49.557 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 4, 'reported_epoch': 71, 'state': 'creating+activating', 'last_fresh': '2024-09-17T04:28:43.170907+0000', 'last_change': '2024-09-17T04:28:43.165532+0000', 'last_active': '2024-09-17T04:28:42.144673+0000', 'last_peered': '2024-09-17T04:28:42.144673+0000', 'last_clean': '2024-09-17T04:28:42.144673+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:43.170907+0000', 'last_undegraded': '2024-09-17T04:28:43.170907+0000', 'last_fullsized': '2024-09-17T04:28:43.170907+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:42.144673+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:42.144673+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:42.144673+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': ['0', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T04:28:49.557 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1c is not active+clean 2024-09-17T04:28:49.557 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1c', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 72, 'state': 'creating+activating', 'last_fresh': '2024-09-17T04:28:44.173734+0000', 'last_change': '2024-09-17T04:28:44.173734+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:44.173734+0000', 'last_undegraded': '2024-09-17T04:28:44.173734+0000', 'last_fullsized': '2024-09-17T04:28:44.173734+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': ['6', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T04:28:49.557 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1d is not active+clean 2024-09-17T04:28:49.557 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 72, 'state': 'creating+activating', 'last_fresh': '2024-09-17T04:28:44.167817+0000', 'last_change': '2024-09-17T04:28:44.167817+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:44.167817+0000', 'last_undegraded': '2024-09-17T04:28:44.167817+0000', 'last_fullsized': '2024-09-17T04:28:44.167817+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': ['7', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:28:49.557 INFO:tasks.ceph.ceph_manager.ceph:PG 11.18 is not active+clean 2024-09-17T04:28:49.557 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:28:43.159075+0000', 'last_change': '2024-09-17T04:28:43.159075+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:43.159075+0000', 'last_undegraded': '2024-09-17T04:28:43.159075+0000', 'last_fullsized': '2024-09-17T04:28:43.159075+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T04:28:49.557 INFO:tasks.ceph.ceph_manager.ceph:PG 11.19 is not active+clean 2024-09-17T04:28:49.557 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:28:43.168710+0000', 'last_change': '2024-09-17T04:28:43.168710+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:43.168710+0000', 'last_undegraded': '2024-09-17T04:28:43.168710+0000', 'last_fullsized': '2024-09-17T04:28:43.168710+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T04:28:49.557 INFO:tasks.ceph.ceph_manager.ceph:PG 11.17 is not active+clean 2024-09-17T04:28:49.558 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 72, 'state': 'creating+activating', 'last_fresh': '2024-09-17T04:28:44.168793+0000', 'last_change': '2024-09-17T04:28:44.168793+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:44.168793+0000', 'last_undegraded': '2024-09-17T04:28:44.168793+0000', 'last_fullsized': '2024-09-17T04:28:44.168793+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': ['7', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:28:49.558 INFO:tasks.ceph.ceph_manager.ceph:PG 11.14 is not active+clean 2024-09-17T04:28:49.558 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.14', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 72, 'state': 'creating+activating', 'last_fresh': '2024-09-17T04:28:44.173127+0000', 'last_change': '2024-09-17T04:28:44.173127+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:44.173127+0000', 'last_undegraded': '2024-09-17T04:28:44.173127+0000', 'last_fullsized': '2024-09-17T04:28:44.173127+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': ['6', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T04:28:49.558 INFO:tasks.ceph.ceph_manager.ceph:PG 11.15 is not active+clean 2024-09-17T04:28:49.558 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:28:43.160386+0000', 'last_change': '2024-09-17T04:28:43.160386+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:43.160386+0000', 'last_undegraded': '2024-09-17T04:28:43.160386+0000', 'last_fullsized': '2024-09-17T04:28:43.160386+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T04:28:49.558 INFO:tasks.ceph.ceph_manager.ceph:PG 10.13 is not active+clean 2024-09-17T04:28:49.558 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 71, 'state': 'creating+activating', 'last_fresh': '2024-09-17T04:28:43.156870+0000', 'last_change': '2024-09-17T04:28:43.156870+0000', 'last_active': '2024-09-17T04:28:42.144673+0000', 'last_peered': '2024-09-17T04:28:42.144673+0000', 'last_clean': '2024-09-17T04:28:42.144673+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:43.156870+0000', 'last_undegraded': '2024-09-17T04:28:43.156870+0000', 'last_fullsized': '2024-09-17T04:28:43.156870+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:42.144673+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:42.144673+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:42.144673+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': ['1', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T04:28:49.558 INFO:tasks.ceph.ceph_manager.ceph:PG 11.10 is not active+clean 2024-09-17T04:28:49.558 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:28:43.168817+0000', 'last_change': '2024-09-17T04:28:43.168817+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:43.168817+0000', 'last_undegraded': '2024-09-17T04:28:43.168817+0000', 'last_fullsized': '2024-09-17T04:28:43.168817+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T04:28:49.558 INFO:tasks.ceph.ceph_manager.ceph:PG 11.11 is not active+clean 2024-09-17T04:28:49.558 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:28:43.158958+0000', 'last_change': '2024-09-17T04:28:43.158958+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:43.158958+0000', 'last_undegraded': '2024-09-17T04:28:43.158958+0000', 'last_fullsized': '2024-09-17T04:28:43.158958+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T04:28:49.559 INFO:tasks.ceph.ceph_manager.ceph:PG 11.f is not active+clean 2024-09-17T04:28:49.559 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 72, 'state': 'creating+activating', 'last_fresh': '2024-09-17T04:28:44.169202+0000', 'last_change': '2024-09-17T04:28:44.169202+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:44.169202+0000', 'last_undegraded': '2024-09-17T04:28:44.169202+0000', 'last_fullsized': '2024-09-17T04:28:44.169202+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': ['7', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:28:49.559 INFO:tasks.ceph.ceph_manager.ceph:PG 11.c is not active+clean 2024-09-17T04:28:49.559 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:28:43.157002+0000', 'last_change': '2024-09-17T04:28:43.157002+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:43.157002+0000', 'last_undegraded': '2024-09-17T04:28:43.157002+0000', 'last_fullsized': '2024-09-17T04:28:43.157002+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T04:28:49.559 INFO:tasks.ceph.ceph_manager.ceph:PG 11.d is not active+clean 2024-09-17T04:28:49.559 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:28:43.159593+0000', 'last_change': '2024-09-17T04:28:43.159593+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:43.159593+0000', 'last_undegraded': '2024-09-17T04:28:43.159593+0000', 'last_fullsized': '2024-09-17T04:28:43.159593+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T04:28:49.559 INFO:tasks.ceph.ceph_manager.ceph:PG 10.8 is not active+clean 2024-09-17T04:28:49.559 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 71, 'state': 'creating+activating', 'last_fresh': '2024-09-17T04:28:43.169387+0000', 'last_change': '2024-09-17T04:28:43.169387+0000', 'last_active': '2024-09-17T04:28:42.144673+0000', 'last_peered': '2024-09-17T04:28:42.144673+0000', 'last_clean': '2024-09-17T04:28:42.144673+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:43.169387+0000', 'last_undegraded': '2024-09-17T04:28:43.169387+0000', 'last_fullsized': '2024-09-17T04:28:43.169387+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:42.144673+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:42.144673+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:42.144673+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': ['0', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T04:28:49.559 INFO:tasks.ceph.ceph_manager.ceph:PG 11.6 is not active+clean 2024-09-17T04:28:49.559 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:28:43.168884+0000', 'last_change': '2024-09-17T04:28:43.168884+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:43.168884+0000', 'last_undegraded': '2024-09-17T04:28:43.168884+0000', 'last_fullsized': '2024-09-17T04:28:43.168884+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T04:28:49.559 INFO:tasks.ceph.ceph_manager.ceph:PG 11.4 is not active+clean 2024-09-17T04:28:49.560 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 72, 'state': 'creating+activating', 'last_fresh': '2024-09-17T04:28:44.167756+0000', 'last_change': '2024-09-17T04:28:44.167756+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:44.167756+0000', 'last_undegraded': '2024-09-17T04:28:44.167756+0000', 'last_fullsized': '2024-09-17T04:28:44.167756+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': ['7', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:28:49.560 INFO:tasks.ceph.ceph_manager.ceph:PG 10.5 is not active+clean 2024-09-17T04:28:49.560 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 71, 'state': 'creating+activating', 'last_fresh': '2024-09-17T04:28:43.155487+0000', 'last_change': '2024-09-17T04:28:43.155487+0000', 'last_active': '2024-09-17T04:28:42.144673+0000', 'last_peered': '2024-09-17T04:28:42.144673+0000', 'last_clean': '2024-09-17T04:28:42.144673+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:43.155487+0000', 'last_undegraded': '2024-09-17T04:28:43.155487+0000', 'last_fullsized': '2024-09-17T04:28:43.155487+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:42.144673+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:42.144673+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:42.144673+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': ['2', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T04:28:49.560 INFO:tasks.ceph.ceph_manager.ceph:PG 11.5 is not active+clean 2024-09-17T04:28:49.560 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 72, 'state': 'creating+activating', 'last_fresh': '2024-09-17T04:28:44.168219+0000', 'last_change': '2024-09-17T04:28:44.168219+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:44.168219+0000', 'last_undegraded': '2024-09-17T04:28:44.168219+0000', 'last_fullsized': '2024-09-17T04:28:44.168219+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': ['7', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:28:49.560 INFO:tasks.ceph.ceph_manager.ceph:PG 10.4 is not active+clean 2024-09-17T04:28:49.560 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 71, 'state': 'creating+activating', 'last_fresh': '2024-09-17T04:28:43.165983+0000', 'last_change': '2024-09-17T04:28:43.165983+0000', 'last_active': '2024-09-17T04:28:42.144673+0000', 'last_peered': '2024-09-17T04:28:42.144673+0000', 'last_clean': '2024-09-17T04:28:42.144673+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:43.165983+0000', 'last_undegraded': '2024-09-17T04:28:43.165983+0000', 'last_fullsized': '2024-09-17T04:28:43.165983+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:42.144673+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:42.144673+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:42.144673+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': ['0', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T04:28:49.561 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2 is not active+clean 2024-09-17T04:28:49.561 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:28:43.158679+0000', 'last_change': '2024-09-17T04:28:43.158679+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:43.158679+0000', 'last_undegraded': '2024-09-17T04:28:43.158679+0000', 'last_fullsized': '2024-09-17T04:28:43.158679+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T04:28:49.561 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3 is not active+clean 2024-09-17T04:28:49.561 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 71, 'state': 'creating+activating', 'last_fresh': '2024-09-17T04:28:43.155388+0000', 'last_change': '2024-09-17T04:28:43.155388+0000', 'last_active': '2024-09-17T04:28:42.144673+0000', 'last_peered': '2024-09-17T04:28:42.144673+0000', 'last_clean': '2024-09-17T04:28:42.144673+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:43.155388+0000', 'last_undegraded': '2024-09-17T04:28:43.155388+0000', 'last_fullsized': '2024-09-17T04:28:43.155388+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:42.144673+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:42.144673+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:42.144673+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': ['2', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T04:28:49.561 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3 is not active+clean 2024-09-17T04:28:49.561 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 72, 'state': 'creating+activating', 'last_fresh': '2024-09-17T04:28:44.171118+0000', 'last_change': '2024-09-17T04:28:44.171118+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:44.171118+0000', 'last_undegraded': '2024-09-17T04:28:44.171118+0000', 'last_fullsized': '2024-09-17T04:28:44.171118+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': ['6', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T04:28:49.561 INFO:tasks.ceph.ceph_manager.ceph:PG 11.0 is not active+clean 2024-09-17T04:28:49.561 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:28:43.157808+0000', 'last_change': '2024-09-17T04:28:43.157808+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:43.157808+0000', 'last_undegraded': '2024-09-17T04:28:43.157808+0000', 'last_fullsized': '2024-09-17T04:28:43.157808+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T04:28:49.561 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1 is not active+clean 2024-09-17T04:28:49.561 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 72, 'state': 'creating+activating', 'last_fresh': '2024-09-17T04:28:44.172672+0000', 'last_change': '2024-09-17T04:28:44.172672+0000', 'last_active': '2024-09-17T04:28:43.149798+0000', 'last_peered': '2024-09-17T04:28:43.149798+0000', 'last_clean': '2024-09-17T04:28:43.149798+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:28:44.172672+0000', 'last_undegraded': '2024-09-17T04:28:44.172672+0000', 'last_fullsized': '2024-09-17T04:28:44.172672+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'last_clean_scrub_stamp': '2024-09-17T04:28:43.149798+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': ['6', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T04:28:49.562 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T04:28:49.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.804+0000 7fabffa00640 1 -- 172.21.15.100:0/3453002000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabf815fae0 msgr2=0x7fabf8166550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:49.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.804+0000 7fabffa00640 1 --2- 172.21.15.100:0/3453002000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabf815fae0 0x7fabf8166550 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fabf0005fd0 tx=0x7fabf002a820 comp rx=0 tx=0).stop 2024-09-17T04:28:49.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.804+0000 7fabffa00640 1 -- 172.21.15.100:0/3453002000 shutdown_connections 2024-09-17T04:28:49.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.804+0000 7fabffa00640 1 --2- 172.21.15.100:0/3453002000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabf815fae0 0x7fabf8166550 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:49.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.804+0000 7fabffa00640 1 -- 172.21.15.100:0/3453002000 >> 172.21.15.100:0/3453002000 conn(0x7fabf806d730 msgr2=0x7fabf80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:49.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.804+0000 7fabffa00640 1 -- 172.21.15.100:0/3453002000 shutdown_connections 2024-09-17T04:28:49.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.804+0000 7fabffa00640 1 -- 172.21.15.100:0/3453002000 wait complete. 2024-09-17T04:28:49.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.806+0000 7fabffa00640 1 Processor -- start 2024-09-17T04:28:49.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.807+0000 7fabffa00640 1 -- start start 2024-09-17T04:28:49.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.808+0000 7fabffa00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fabf80aebc0 0x7fabf8099880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:49.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.808+0000 7fabffa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabf8152920 0x7fabf80b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:49.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.808+0000 7fabffa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabf814d4e0 0x7fabf80b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:49.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.808+0000 7fabffa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fabf8166ca0 con 0x7fabf814d4e0 2024-09-17T04:28:49.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.808+0000 7fabffa00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fabf80b4a30 con 0x7fabf8152920 2024-09-17T04:28:49.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.808+0000 7fabffa00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fabf8166e20 con 0x7fabf80aebc0 2024-09-17T04:28:49.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.808+0000 7fabfda00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabf814d4e0 0x7fabf80b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:49.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.808+0000 7fabfda00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabf814d4e0 0x7fabf80b18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38806/0 (socket says 172.21.15.100:38806) 2024-09-17T04:28:49.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.808+0000 7fabfda00640 1 -- 172.21.15.100:0/893383891 learned_addr learned my addr 172.21.15.100:0/893383891 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:49.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.809+0000 7fabf7e00640 1 --2- 172.21.15.100:0/893383891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabf8152920 0x7fabf80b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:49.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.809+0000 7fabfda00640 1 -- 172.21.15.100:0/893383891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fabf80aebc0 msgr2=0x7fabf8099880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:49.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.809+0000 7fabfd000640 1 --2- 172.21.15.100:0/893383891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fabf80aebc0 0x7fabf8099880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:49.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.809+0000 7fabfda00640 1 --2- 172.21.15.100:0/893383891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fabf80aebc0 0x7fabf8099880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:49.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.809+0000 7fabfda00640 1 -- 172.21.15.100:0/893383891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabf8152920 msgr2=0x7fabf80b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:49.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.809+0000 7fabfda00640 1 --2- 172.21.15.100:0/893383891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabf8152920 0x7fabf80b1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:49.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.809+0000 7fabfda00640 1 -- 172.21.15.100:0/893383891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf814aa80 con 0x7fabf814d4e0 2024-09-17T04:28:49.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.809+0000 7fabf7e00640 1 --2- 172.21.15.100:0/893383891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabf8152920 0x7fabf80b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:28:49.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.809+0000 7fabfd000640 1 --2- 172.21.15.100:0/893383891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fabf80aebc0 0x7fabf8099880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:28:49.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.809+0000 7fabfda00640 1 --2- 172.21.15.100:0/893383891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabf814d4e0 0x7fabf80b18d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fabf0006010 tx=0x7fabf0032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:49.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.810+0000 7fabf4c00640 1 -- 172.21.15.100:0/893383891 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf0037040 con 0x7fabf814d4e0 2024-09-17T04:28:49.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.810+0000 7fabf4c00640 1 -- 172.21.15.100:0/893383891 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabf0042070 con 0x7fabf814d4e0 2024-09-17T04:28:49.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.810+0000 7fabffa00640 1 -- 172.21.15.100:0/893383891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf814ad60 con 0x7fabf814d4e0 2024-09-17T04:28:49.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.810+0000 7fabf4c00640 1 -- 172.21.15.100:0/893383891 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf00042a0 con 0x7fabf814d4e0 2024-09-17T04:28:49.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.810+0000 7fabffa00640 1 -- 172.21.15.100:0/893383891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf814b320 con 0x7fabf814d4e0 2024-09-17T04:28:49.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.811+0000 7fabffa00640 1 -- 172.21.15.100:0/893383891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabf814a7f0 con 0x7fabf814d4e0 2024-09-17T04:28:49.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.812+0000 7fabf4c00640 1 -- 172.21.15.100:0/893383891 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fabf0004440 con 0x7fabf814d4e0 2024-09-17T04:28:49.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.813+0000 7fabf4c00640 1 --2- 172.21.15.100:0/893383891 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fabd8078ef0 0x7fabd807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:49.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.813+0000 7fabf4c00640 1 -- 172.21.15.100:0/893383891 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7fabf00bc180 con 0x7fabf814d4e0 2024-09-17T04:28:49.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.813+0000 7fabfd000640 1 --2- 172.21.15.100:0/893383891 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fabd8078ef0 0x7fabd807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:49.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.814+0000 7fabfd000640 1 --2- 172.21.15.100:0/893383891 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fabd8078ef0 0x7fabd807b400 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fabec007fd0 tx=0x7fabec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:49.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:49.817+0000 7fabf4c00640 1 -- 172.21.15.100:0/893383891 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fabf0084180 con 0x7fabf814d4e0 2024-09-17T04:28:50.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.068+0000 7fabffa00640 1 -- 172.21.15.100:0/893383891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fabf80b2750 con 0x7fabf814d4e0 2024-09-17T04:28:50.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.069+0000 7fabf4c00640 1 -- 172.21.15.100:0/893383891 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1187 (secure 0 0 0) 0x7fabf0083a80 con 0x7fabf814d4e0 2024-09-17T04:28:50.069 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:28:50.069 INFO:teuthology.orchestra.run.smithi100.stdout:{"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":303,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":73,"num_osds":8,"num_up_osds":8,"osd_up_since":1726547043,"num_in_osds":8,"osd_in_since":1726547032,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":52},{"state_name":"creating+activating","count":24},{"state_name":"creating+peering","count":21}],"num_pgs":97,"num_pools":3,"num_objects":4,"data_bytes":590458,"bytes_used":259256320,"bytes_avail":772834856960,"bytes_total":773094113280,"inactive_pgs_ratio":0.46391752362251282,"write_bytes_sec":173,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":47,"btime":"2024-09-17T04:28:47:182892+0000","id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"c","status":"up:active","gid":6162}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T04:26:52.007752+0000","services":{}},"progress_events":{}} 2024-09-17T04:28:50.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.071+0000 7fabffa00640 1 -- 172.21.15.100:0/893383891 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fabd8078ef0 msgr2=0x7fabd807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:50.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.071+0000 7fabffa00640 1 --2- 172.21.15.100:0/893383891 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fabd8078ef0 0x7fabd807b400 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fabec007fd0 tx=0x7fabec009040 comp rx=0 tx=0).stop 2024-09-17T04:28:50.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.071+0000 7fabffa00640 1 -- 172.21.15.100:0/893383891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabf814d4e0 msgr2=0x7fabf80b18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:50.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.071+0000 7fabffa00640 1 --2- 172.21.15.100:0/893383891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabf814d4e0 0x7fabf80b18d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fabf0006010 tx=0x7fabf0032d10 comp rx=0 tx=0).stop 2024-09-17T04:28:50.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.071+0000 7fabffa00640 1 -- 172.21.15.100:0/893383891 shutdown_connections 2024-09-17T04:28:50.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.071+0000 7fabffa00640 1 --2- 172.21.15.100:0/893383891 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fabd8078ef0 0x7fabd807b400 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:50.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.071+0000 7fabffa00640 1 --2- 172.21.15.100:0/893383891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabf814d4e0 0x7fabf80b18d0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:50.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.071+0000 7fabffa00640 1 --2- 172.21.15.100:0/893383891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabf8152920 0x7fabf80b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:50.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.071+0000 7fabffa00640 1 --2- 172.21.15.100:0/893383891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fabf80aebc0 0x7fabf8099880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:50.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.071+0000 7fabffa00640 1 -- 172.21.15.100:0/893383891 >> 172.21.15.100:0/893383891 conn(0x7fabf806d730 msgr2=0x7fabf8070260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:50.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.071+0000 7fabffa00640 1 -- 172.21.15.100:0/893383891 shutdown_connections 2024-09-17T04:28:50.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.071+0000 7fabffa00640 1 -- 172.21.15.100:0/893383891 wait complete. 2024-09-17T04:28:50.096 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T04:28:50.097 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:28:50.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.320+0000 7f2e82a00640 1 -- 172.21.15.100:0/3526616516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e64004d40 msgr2=0x7f2e64005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:50.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.320+0000 7f2e82a00640 1 --2- 172.21.15.100:0/3526616516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e64004d40 0x7f2e64005210 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2e74009e20 tx=0x7f2e64005830 comp rx=0 tx=0).stop 2024-09-17T04:28:50.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.320+0000 7f2e82000640 1 -- 172.21.15.100:0/3526616516 reap_dead start 2024-09-17T04:28:50.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.320+0000 7f2e82a00640 1 -- 172.21.15.100:0/3526616516 shutdown_connections 2024-09-17T04:28:50.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.320+0000 7f2e82a00640 1 -- 172.21.15.100:0/3526616516 >> 172.21.15.100:0/3526616516 conn(0x7f2e7c0b1e40 msgr2=0x7f2e7c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:50.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.320+0000 7f2e82a00640 1 -- 172.21.15.100:0/3526616516 shutdown_connections 2024-09-17T04:28:50.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.320+0000 7f2e82a00640 1 -- 172.21.15.100:0/3526616516 wait complete. 2024-09-17T04:28:50.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.322+0000 7f2e82a00640 1 Processor -- start 2024-09-17T04:28:50.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.322+0000 7f2e82a00640 1 -- start start 2024-09-17T04:28:50.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.324+0000 7f2e82a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e64004d40 0x7f2e7c156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:50.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.324+0000 7f2e82a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e7c155c40 0x7f2e7c0cf140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:50.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.324+0000 7f2e82a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e7c0cdce0 0x7f2e7c0ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:50.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.324+0000 7f2e82a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7c0b30e0 con 0x7f2e64004d40 2024-09-17T04:28:50.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.324+0000 7f2e82a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7c0b2f60 con 0x7f2e7c155c40 2024-09-17T04:28:50.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.324+0000 7f2e82a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2e7c0b3260 con 0x7f2e7c0cdce0 2024-09-17T04:28:50.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.324+0000 7f2e81600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e64004d40 0x7f2e7c156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:50.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.324+0000 7f2e81600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e64004d40 0x7f2e7c156620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38832/0 (socket says 172.21.15.100:38832) 2024-09-17T04:28:50.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.324+0000 7f2e81600640 1 -- 172.21.15.100:0/1210514178 learned_addr learned my addr 172.21.15.100:0/1210514178 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:50.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.324+0000 7f2e80c00640 1 --2- 172.21.15.100:0/1210514178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e7c155c40 0x7f2e7c0cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:50.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.324+0000 7f2e82000640 1 --2- 172.21.15.100:0/1210514178 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e7c0cdce0 0x7f2e7c0ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:50.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.324+0000 7f2e81600640 1 -- 172.21.15.100:0/1210514178 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e7c0cdce0 msgr2=0x7f2e7c0ce0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:50.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.324+0000 7f2e81600640 1 --2- 172.21.15.100:0/1210514178 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e7c0cdce0 0x7f2e7c0ce0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:50.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.324+0000 7f2e81600640 1 -- 172.21.15.100:0/1210514178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e7c155c40 msgr2=0x7f2e7c0cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:50.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.324+0000 7f2e81600640 1 --2- 172.21.15.100:0/1210514178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e7c155c40 0x7f2e7c0cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:50.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.324+0000 7f2e81600640 1 -- 172.21.15.100:0/1210514178 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e7c1f9130 con 0x7f2e64004d40 2024-09-17T04:28:50.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.325+0000 7f2e81600640 1 --2- 172.21.15.100:0/1210514178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e64004d40 0x7f2e7c156620 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2e70006610 tx=0x7f2e700039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:50.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.325+0000 7f2e79600640 1 -- 172.21.15.100:0/1210514178 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e70015070 con 0x7f2e64004d40 2024-09-17T04:28:50.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.325+0000 7f2e79600640 1 -- 172.21.15.100:0/1210514178 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e7000f960 con 0x7f2e64004d40 2024-09-17T04:28:50.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.325+0000 7f2e79600640 1 -- 172.21.15.100:0/1210514178 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7000fd90 con 0x7f2e64004d40 2024-09-17T04:28:50.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.325+0000 7f2e82a00640 1 -- 172.21.15.100:0/1210514178 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e7c0ccc40 con 0x7f2e64004d40 2024-09-17T04:28:50.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.325+0000 7f2e82a00640 1 -- 172.21.15.100:0/1210514178 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e7c0cd200 con 0x7f2e64004d40 2024-09-17T04:28:50.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.326+0000 7f2e82a00640 1 -- 172.21.15.100:0/1210514178 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e7c051080 con 0x7f2e64004d40 2024-09-17T04:28:50.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.327+0000 7f2e79600640 1 -- 172.21.15.100:0/1210514178 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f2e70011430 con 0x7f2e64004d40 2024-09-17T04:28:50.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.328+0000 7f2e79600640 1 --2- 172.21.15.100:0/1210514178 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2e5c078f20 0x7f2e5c07b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:50.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.328+0000 7f2e79600640 1 -- 172.21.15.100:0/1210514178 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7f2e7009c0f0 con 0x7f2e64004d40 2024-09-17T04:28:50.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.329+0000 7f2e80c00640 1 --2- 172.21.15.100:0/1210514178 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2e5c078f20 0x7f2e5c07b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:50.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.329+0000 7f2e80c00640 1 --2- 172.21.15.100:0/1210514178 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2e5c078f20 0x7f2e5c07b430 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0cbfe0 tx=0x7f2e6800b260 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:50.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.333+0000 7f2e79600640 1 -- 172.21.15.100:0/1210514178 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f2e7c051080 con 0x7f2e64004d40 2024-09-17T04:28:50.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.512+0000 7f2e82a00640 1 -- 172.21.15.100:0/1210514178 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2e7c15d670 con 0x7f2e5c078f20 2024-09-17T04:28:50.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.517+0000 7f2e79600640 1 -- 172.21.15.100:0/1210514178 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252058 (secure 0 0 0) 0x7f2e7c15d670 con 0x7f2e5c078f20 2024-09-17T04:28:50.521 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:28:50.522 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:28:50.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.522+0000 7f2e82a00640 1 -- 172.21.15.100:0/1210514178 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2e5c078f20 msgr2=0x7f2e5c07b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:50.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.522+0000 7f2e82a00640 1 --2- 172.21.15.100:0/1210514178 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2e5c078f20 0x7f2e5c07b430 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0cbfe0 tx=0x7f2e6800b260 comp rx=0 tx=0).stop 2024-09-17T04:28:50.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.522+0000 7f2e82a00640 1 -- 172.21.15.100:0/1210514178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e64004d40 msgr2=0x7f2e7c156620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:50.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.522+0000 7f2e82a00640 1 --2- 172.21.15.100:0/1210514178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e64004d40 0x7f2e7c156620 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2e70006610 tx=0x7f2e700039c0 comp rx=0 tx=0).stop 2024-09-17T04:28:50.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.522+0000 7f2e82a00640 1 -- 172.21.15.100:0/1210514178 shutdown_connections 2024-09-17T04:28:50.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.522+0000 7f2e82a00640 1 --2- 172.21.15.100:0/1210514178 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f2e5c078f20 0x7f2e5c07b430 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:50.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.522+0000 7f2e82a00640 1 --2- 172.21.15.100:0/1210514178 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e7c0cdce0 0x7f2e7c0ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:50.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.522+0000 7f2e82a00640 1 --2- 172.21.15.100:0/1210514178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e7c155c40 0x7f2e7c0cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:50.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.522+0000 7f2e82a00640 1 --2- 172.21.15.100:0/1210514178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e64004d40 0x7f2e7c156620 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:50.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.522+0000 7f2e82a00640 1 -- 172.21.15.100:0/1210514178 >> 172.21.15.100:0/1210514178 conn(0x7f2e7c0b1e40 msgr2=0x7f2e7c0b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:50.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.522+0000 7f2e82a00640 1 -- 172.21.15.100:0/1210514178 shutdown_connections 2024-09-17T04:28:50.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:50.522+0000 7f2e82a00640 1 -- 172.21.15.100:0/1210514178 wait complete. 2024-09-17T04:28:50.544 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-09-17T04:28:50.041738+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257604,"kb_used_data":8072,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717116,"statfs":{"total":773094113280,"available":772830326784,"internally_reserved":0,"allocated":8265728,"data_stored":4128604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.887207"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.391360+0000","last_change":"2024-09-17T04:28:44.178546+0000","last_active":"2024-09-17T04:28:45.391360+0000","last_peered":"2024-09-17T04:28:45.391360+0000","last_clean":"2024-09-17T04:28:45.391360+0000","last_became_active":"2024-09-17T04:28:44.178226+0000","last_became_peered":"2024-09-17T04:28:44.178226+0000","last_unstale":"2024-09-17T04:28:45.391360+0000","last_undegraded":"2024-09-17T04:28:45.391360+0000","last_fullsized":"2024-09-17T04:28:45.391360+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:42.957899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837706+0000","last_change":"2024-09-17T04:28:44.187195+0000","last_active":"2024-09-17T04:28:45.837706+0000","last_peered":"2024-09-17T04:28:45.837706+0000","last_clean":"2024-09-17T04:28:45.837706+0000","last_became_active":"2024-09-17T04:28:44.186765+0000","last_became_peered":"2024-09-17T04:28:44.186765+0000","last_unstale":"2024-09-17T04:28:45.837706+0000","last_undegraded":"2024-09-17T04:28:45.837706+0000","last_fullsized":"2024-09-17T04:28:45.837706+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:59:32.661314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836882+0000","last_change":"2024-09-17T04:28:44.187901+0000","last_active":"2024-09-17T04:28:45.836882+0000","last_peered":"2024-09-17T04:28:45.836882+0000","last_clean":"2024-09-17T04:28:45.836882+0000","last_became_active":"2024-09-17T04:28:44.187552+0000","last_became_peered":"2024-09-17T04:28:44.187552+0000","last_unstale":"2024-09-17T04:28:45.836882+0000","last_undegraded":"2024-09-17T04:28:45.836882+0000","last_fullsized":"2024-09-17T04:28:45.836882+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:33.519054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836695+0000","last_change":"2024-09-17T04:28:44.184158+0000","last_active":"2024-09-17T04:28:45.836695+0000","last_peered":"2024-09-17T04:28:45.836695+0000","last_clean":"2024-09-17T04:28:45.836695+0000","last_became_active":"2024-09-17T04:28:44.183807+0000","last_became_peered":"2024-09-17T04:28:44.183807+0000","last_unstale":"2024-09-17T04:28:45.836695+0000","last_undegraded":"2024-09-17T04:28:45.836695+0000","last_fullsized":"2024-09-17T04:28:45.836695+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:49.080042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.177322+0000","last_change":"2024-09-17T04:28:44.172010+0000","last_active":"2024-09-17T04:28:45.177322+0000","last_peered":"2024-09-17T04:28:45.177322+0000","last_clean":"2024-09-17T04:28:45.177322+0000","last_became_active":"2024-09-17T04:28:44.171633+0000","last_became_peered":"2024-09-17T04:28:44.171633+0000","last_unstale":"2024-09-17T04:28:45.177322+0000","last_undegraded":"2024-09-17T04:28:45.177322+0000","last_fullsized":"2024-09-17T04:28:45.177322+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:38:43.358588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837029+0000","last_change":"2024-09-17T04:28:44.190298+0000","last_active":"2024-09-17T04:28:45.837029+0000","last_peered":"2024-09-17T04:28:45.837029+0000","last_clean":"2024-09-17T04:28:45.837029+0000","last_became_active":"2024-09-17T04:28:44.187991+0000","last_became_peered":"2024-09-17T04:28:44.187991+0000","last_unstale":"2024-09-17T04:28:45.837029+0000","last_undegraded":"2024-09-17T04:28:45.837029+0000","last_fullsized":"2024-09-17T04:28:45.837029+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:09.319597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.180600+0000","last_change":"2024-09-17T04:28:44.180281+0000","last_active":"2024-09-17T04:28:45.180600+0000","last_peered":"2024-09-17T04:28:45.180600+0000","last_clean":"2024-09-17T04:28:45.180600+0000","last_became_active":"2024-09-17T04:28:44.179971+0000","last_became_peered":"2024-09-17T04:28:44.179971+0000","last_unstale":"2024-09-17T04:28:45.180600+0000","last_undegraded":"2024-09-17T04:28:45.180600+0000","last_fullsized":"2024-09-17T04:28:45.180600+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:28.683793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837098+0000","last_change":"2024-09-17T04:28:44.190967+0000","last_active":"2024-09-17T04:28:45.837098+0000","last_peered":"2024-09-17T04:28:45.837098+0000","last_clean":"2024-09-17T04:28:45.837098+0000","last_became_active":"2024-09-17T04:28:44.190523+0000","last_became_peered":"2024-09-17T04:28:44.190523+0000","last_unstale":"2024-09-17T04:28:45.837098+0000","last_undegraded":"2024-09-17T04:28:45.837098+0000","last_fullsized":"2024-09-17T04:28:45.837098+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:42.525462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.178028+0000","last_change":"2024-09-17T04:28:44.180005+0000","last_active":"2024-09-17T04:28:45.178028+0000","last_peered":"2024-09-17T04:28:45.178028+0000","last_clean":"2024-09-17T04:28:45.178028+0000","last_became_active":"2024-09-17T04:28:44.179352+0000","last_became_peered":"2024-09-17T04:28:44.179352+0000","last_unstale":"2024-09-17T04:28:45.178028+0000","last_undegraded":"2024-09-17T04:28:45.178028+0000","last_fullsized":"2024-09-17T04:28:45.178028+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:21:08.710175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.183033+0000","last_change":"2024-09-17T04:28:44.166800+0000","last_active":"2024-09-17T04:28:45.183033+0000","last_peered":"2024-09-17T04:28:45.183033+0000","last_clean":"2024-09-17T04:28:45.183033+0000","last_became_active":"2024-09-17T04:28:44.166256+0000","last_became_peered":"2024-09-17T04:28:44.166256+0000","last_unstale":"2024-09-17T04:28:45.183033+0000","last_undegraded":"2024-09-17T04:28:45.183033+0000","last_fullsized":"2024-09-17T04:28:45.183033+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:18.874211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.391108+0000","last_change":"2024-09-17T04:28:44.183172+0000","last_active":"2024-09-17T04:28:45.391108+0000","last_peered":"2024-09-17T04:28:45.391108+0000","last_clean":"2024-09-17T04:28:45.391108+0000","last_became_active":"2024-09-17T04:28:44.182857+0000","last_became_peered":"2024-09-17T04:28:44.182857+0000","last_unstale":"2024-09-17T04:28:45.391108+0000","last_undegraded":"2024-09-17T04:28:45.391108+0000","last_fullsized":"2024-09-17T04:28:45.391108+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:03.529440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836899+0000","last_change":"2024-09-17T04:28:44.177609+0000","last_active":"2024-09-17T04:28:45.836899+0000","last_peered":"2024-09-17T04:28:45.836899+0000","last_clean":"2024-09-17T04:28:45.836899+0000","last_became_active":"2024-09-17T04:28:44.177243+0000","last_became_peered":"2024-09-17T04:28:44.177243+0000","last_unstale":"2024-09-17T04:28:45.836899+0000","last_undegraded":"2024-09-17T04:28:45.836899+0000","last_fullsized":"2024-09-17T04:28:45.836899+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:30:15.333284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.186892+0000","last_change":"2024-09-17T04:28:44.184657+0000","last_active":"2024-09-17T04:28:45.186892+0000","last_peered":"2024-09-17T04:28:45.186892+0000","last_clean":"2024-09-17T04:28:45.186892+0000","last_became_active":"2024-09-17T04:28:44.184350+0000","last_became_peered":"2024-09-17T04:28:44.184350+0000","last_unstale":"2024-09-17T04:28:45.186892+0000","last_undegraded":"2024-09-17T04:28:45.186892+0000","last_fullsized":"2024-09-17T04:28:45.186892+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:21.723977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.177357+0000","last_change":"2024-09-17T04:28:44.167744+0000","last_active":"2024-09-17T04:28:45.177357+0000","last_peered":"2024-09-17T04:28:45.177357+0000","last_clean":"2024-09-17T04:28:45.177357+0000","last_became_active":"2024-09-17T04:28:44.166731+0000","last_became_peered":"2024-09-17T04:28:44.166731+0000","last_unstale":"2024-09-17T04:28:45.177357+0000","last_undegraded":"2024-09-17T04:28:45.177357+0000","last_fullsized":"2024-09-17T04:28:45.177357+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:59.409832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.187053+0000","last_change":"2024-09-17T04:28:44.187858+0000","last_active":"2024-09-17T04:28:45.187053+0000","last_peered":"2024-09-17T04:28:45.187053+0000","last_clean":"2024-09-17T04:28:45.187053+0000","last_became_active":"2024-09-17T04:28:44.187287+0000","last_became_peered":"2024-09-17T04:28:44.187287+0000","last_unstale":"2024-09-17T04:28:45.187053+0000","last_undegraded":"2024-09-17T04:28:45.187053+0000","last_fullsized":"2024-09-17T04:28:45.187053+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:10:24.302128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.181520+0000","last_change":"2024-09-17T04:28:44.186823+0000","last_active":"2024-09-17T04:28:45.181520+0000","last_peered":"2024-09-17T04:28:45.181520+0000","last_clean":"2024-09-17T04:28:45.181520+0000","last_became_active":"2024-09-17T04:28:44.186451+0000","last_became_peered":"2024-09-17T04:28:44.186451+0000","last_unstale":"2024-09-17T04:28:45.181520+0000","last_undegraded":"2024-09-17T04:28:45.181520+0000","last_fullsized":"2024-09-17T04:28:45.181520+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:27.243441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.178040+0000","last_change":"2024-09-17T04:28:44.172893+0000","last_active":"2024-09-17T04:28:45.178040+0000","last_peered":"2024-09-17T04:28:45.178040+0000","last_clean":"2024-09-17T04:28:45.178040+0000","last_became_active":"2024-09-17T04:28:44.172600+0000","last_became_peered":"2024-09-17T04:28:44.172600+0000","last_unstale":"2024-09-17T04:28:45.178040+0000","last_undegraded":"2024-09-17T04:28:45.178040+0000","last_fullsized":"2024-09-17T04:28:45.178040+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:04:07.718199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837121+0000","last_change":"2024-09-17T04:28:44.186476+0000","last_active":"2024-09-17T04:28:45.837121+0000","last_peered":"2024-09-17T04:28:45.837121+0000","last_clean":"2024-09-17T04:28:45.837121+0000","last_became_active":"2024-09-17T04:28:44.185747+0000","last_became_peered":"2024-09-17T04:28:44.185747+0000","last_unstale":"2024-09-17T04:28:45.837121+0000","last_undegraded":"2024-09-17T04:28:45.837121+0000","last_fullsized":"2024-09-17T04:28:45.837121+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:22.413761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837195+0000","last_change":"2024-09-17T04:28:44.182489+0000","last_active":"2024-09-17T04:28:45.837195+0000","last_peered":"2024-09-17T04:28:45.837195+0000","last_clean":"2024-09-17T04:28:45.837195+0000","last_became_active":"2024-09-17T04:28:44.182178+0000","last_became_peered":"2024-09-17T04:28:44.182178+0000","last_unstale":"2024-09-17T04:28:45.837195+0000","last_undegraded":"2024-09-17T04:28:45.837195+0000","last_fullsized":"2024-09-17T04:28:45.837195+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:55:33.943721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.186906+0000","last_change":"2024-09-17T04:28:44.185354+0000","last_active":"2024-09-17T04:28:45.186906+0000","last_peered":"2024-09-17T04:28:45.186906+0000","last_clean":"2024-09-17T04:28:45.186906+0000","last_became_active":"2024-09-17T04:28:44.184977+0000","last_became_peered":"2024-09-17T04:28:44.184977+0000","last_unstale":"2024-09-17T04:28:45.186906+0000","last_undegraded":"2024-09-17T04:28:45.186906+0000","last_fullsized":"2024-09-17T04:28:45.186906+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:06.029830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.182707+0000","last_change":"2024-09-17T04:28:44.172195+0000","last_active":"2024-09-17T04:28:45.182707+0000","last_peered":"2024-09-17T04:28:45.182707+0000","last_clean":"2024-09-17T04:28:45.182707+0000","last_became_active":"2024-09-17T04:28:44.171667+0000","last_became_peered":"2024-09-17T04:28:44.171667+0000","last_unstale":"2024-09-17T04:28:45.182707+0000","last_undegraded":"2024-09-17T04:28:45.182707+0000","last_fullsized":"2024-09-17T04:28:45.182707+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:20.771270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.182389+0000","last_change":"2024-09-17T04:28:44.176739+0000","last_active":"2024-09-17T04:28:45.182389+0000","last_peered":"2024-09-17T04:28:45.182389+0000","last_clean":"2024-09-17T04:28:45.182389+0000","last_became_active":"2024-09-17T04:28:44.169094+0000","last_became_peered":"2024-09-17T04:28:44.169094+0000","last_unstale":"2024-09-17T04:28:45.182389+0000","last_undegraded":"2024-09-17T04:28:45.182389+0000","last_fullsized":"2024-09-17T04:28:45.182389+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:28.498294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.390724+0000","last_change":"2024-09-17T04:28:44.191151+0000","last_active":"2024-09-17T04:28:45.390724+0000","last_peered":"2024-09-17T04:28:45.390724+0000","last_clean":"2024-09-17T04:28:45.390724+0000","last_became_active":"2024-09-17T04:28:44.190798+0000","last_became_peered":"2024-09-17T04:28:44.190798+0000","last_unstale":"2024-09-17T04:28:45.390724+0000","last_undegraded":"2024-09-17T04:28:45.390724+0000","last_fullsized":"2024-09-17T04:28:45.390724+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:10:24.302128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.390688+0000","last_change":"2024-09-17T04:28:44.194503+0000","last_active":"2024-09-17T04:28:45.390688+0000","last_peered":"2024-09-17T04:28:45.390688+0000","last_clean":"2024-09-17T04:28:45.390688+0000","last_became_active":"2024-09-17T04:28:44.194172+0000","last_became_peered":"2024-09-17T04:28:44.194172+0000","last_unstale":"2024-09-17T04:28:45.390688+0000","last_undegraded":"2024-09-17T04:28:45.390688+0000","last_fullsized":"2024-09-17T04:28:45.390688+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:06.029830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.183118+0000","last_change":"2024-09-17T04:28:44.170440+0000","last_active":"2024-09-17T04:28:45.183118+0000","last_peered":"2024-09-17T04:28:45.183118+0000","last_clean":"2024-09-17T04:28:45.183118+0000","last_became_active":"2024-09-17T04:28:44.169980+0000","last_became_peered":"2024-09-17T04:28:44.169980+0000","last_unstale":"2024-09-17T04:28:45.183118+0000","last_undegraded":"2024-09-17T04:28:45.183118+0000","last_fullsized":"2024-09-17T04:28:45.183118+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:08.481224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.181983+0000","last_change":"2024-09-17T04:28:44.188325+0000","last_active":"2024-09-17T04:28:45.181983+0000","last_peered":"2024-09-17T04:28:45.181983+0000","last_clean":"2024-09-17T04:28:45.181983+0000","last_became_active":"2024-09-17T04:28:44.188006+0000","last_became_peered":"2024-09-17T04:28:44.188006+0000","last_unstale":"2024-09-17T04:28:45.181983+0000","last_undegraded":"2024-09-17T04:28:45.181983+0000","last_fullsized":"2024-09-17T04:28:45.181983+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:16.202008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836922+0000","last_change":"2024-09-17T04:28:44.174317+0000","last_active":"2024-09-17T04:28:45.836922+0000","last_peered":"2024-09-17T04:28:45.836922+0000","last_clean":"2024-09-17T04:28:45.836922+0000","last_became_active":"2024-09-17T04:28:44.173878+0000","last_became_peered":"2024-09-17T04:28:44.173878+0000","last_unstale":"2024-09-17T04:28:45.836922+0000","last_undegraded":"2024-09-17T04:28:45.836922+0000","last_fullsized":"2024-09-17T04:28:45.836922+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:07.283249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.183080+0000","last_change":"2024-09-17T04:28:44.170671+0000","last_active":"2024-09-17T04:28:45.183080+0000","last_peered":"2024-09-17T04:28:45.183080+0000","last_clean":"2024-09-17T04:28:45.183080+0000","last_became_active":"2024-09-17T04:28:44.169266+0000","last_became_peered":"2024-09-17T04:28:44.169266+0000","last_unstale":"2024-09-17T04:28:45.183080+0000","last_undegraded":"2024-09-17T04:28:45.183080+0000","last_fullsized":"2024-09-17T04:28:45.183080+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:40.899996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836607+0000","last_change":"2024-09-17T04:28:44.188197+0000","last_active":"2024-09-17T04:28:45.836607+0000","last_peered":"2024-09-17T04:28:45.836607+0000","last_clean":"2024-09-17T04:28:45.836607+0000","last_became_active":"2024-09-17T04:28:44.187879+0000","last_became_peered":"2024-09-17T04:28:44.187879+0000","last_unstale":"2024-09-17T04:28:45.836607+0000","last_undegraded":"2024-09-17T04:28:45.836607+0000","last_fullsized":"2024-09-17T04:28:45.836607+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:28:18.827139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.180859+0000","last_change":"2024-09-17T04:28:44.182642+0000","last_active":"2024-09-17T04:28:45.180859+0000","last_peered":"2024-09-17T04:28:45.180859+0000","last_clean":"2024-09-17T04:28:45.180859+0000","last_became_active":"2024-09-17T04:28:44.182271+0000","last_became_peered":"2024-09-17T04:28:44.182271+0000","last_unstale":"2024-09-17T04:28:45.180859+0000","last_undegraded":"2024-09-17T04:28:45.180859+0000","last_fullsized":"2024-09-17T04:28:45.180859+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:54:37.212209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.181078+0000","last_change":"2024-09-17T04:28:44.166100+0000","last_active":"2024-09-17T04:28:45.181078+0000","last_peered":"2024-09-17T04:28:45.181078+0000","last_clean":"2024-09-17T04:28:45.181078+0000","last_became_active":"2024-09-17T04:28:44.165456+0000","last_became_peered":"2024-09-17T04:28:44.165456+0000","last_unstale":"2024-09-17T04:28:45.181078+0000","last_undegraded":"2024-09-17T04:28:45.181078+0000","last_fullsized":"2024-09-17T04:28:45.181078+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:17.830980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837079+0000","last_change":"2024-09-17T04:28:44.192060+0000","last_active":"2024-09-17T04:28:45.837079+0000","last_peered":"2024-09-17T04:28:45.837079+0000","last_clean":"2024-09-17T04:28:45.837079+0000","last_became_active":"2024-09-17T04:28:44.190066+0000","last_became_peered":"2024-09-17T04:28:44.190066+0000","last_unstale":"2024-09-17T04:28:45.837079+0000","last_undegraded":"2024-09-17T04:28:45.837079+0000","last_fullsized":"2024-09-17T04:28:45.837079+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:43:07.615773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.391225+0000","last_change":"2024-09-17T04:28:44.187180+0000","last_active":"2024-09-17T04:28:45.391225+0000","last_peered":"2024-09-17T04:28:45.391225+0000","last_clean":"2024-09-17T04:28:45.391225+0000","last_became_active":"2024-09-17T04:28:44.186726+0000","last_became_peered":"2024-09-17T04:28:44.186726+0000","last_unstale":"2024-09-17T04:28:45.391225+0000","last_undegraded":"2024-09-17T04:28:45.391225+0000","last_fullsized":"2024-09-17T04:28:45.391225+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:45:03.039793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"72'4","reported_seq":19,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.391642+0000","last_change":"2024-09-17T04:28:43.181281+0000","last_active":"2024-09-17T04:28:45.391642+0000","last_peered":"2024-09-17T04:28:45.391642+0000","last_clean":"2024-09-17T04:28:45.391642+0000","last_became_active":"2024-09-17T04:28:43.180960+0000","last_became_peered":"2024-09-17T04:28:43.180960+0000","last_unstale":"2024-09-17T04:28:45.391642+0000","last_undegraded":"2024-09-17T04:28:45.391642+0000","last_fullsized":"2024-09-17T04:28:45.391642+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:19.766145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.178220+0000","last_change":"2024-09-17T04:28:44.179667+0000","last_active":"2024-09-17T04:28:45.178220+0000","last_peered":"2024-09-17T04:28:45.178220+0000","last_clean":"2024-09-17T04:28:45.178220+0000","last_became_active":"2024-09-17T04:28:44.177165+0000","last_became_peered":"2024-09-17T04:28:44.177165+0000","last_unstale":"2024-09-17T04:28:45.178220+0000","last_undegraded":"2024-09-17T04:28:45.178220+0000","last_fullsized":"2024-09-17T04:28:45.178220+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:48.625764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"72'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.187236+0000","last_change":"2024-09-17T04:28:43.175705+0000","last_active":"2024-09-17T04:28:45.187236+0000","last_peered":"2024-09-17T04:28:45.187236+0000","last_clean":"2024-09-17T04:28:45.187236+0000","last_became_active":"2024-09-17T04:28:43.174696+0000","last_became_peered":"2024-09-17T04:28:43.174696+0000","last_unstale":"2024-09-17T04:28:45.187236+0000","last_undegraded":"2024-09-17T04:28:45.187236+0000","last_fullsized":"2024-09-17T04:28:45.187236+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:56.241725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837279+0000","last_change":"2024-09-17T04:28:44.194350+0000","last_active":"2024-09-17T04:28:45.837279+0000","last_peered":"2024-09-17T04:28:45.837279+0000","last_clean":"2024-09-17T04:28:45.837279+0000","last_became_active":"2024-09-17T04:28:44.193812+0000","last_became_peered":"2024-09-17T04:28:44.193812+0000","last_unstale":"2024-09-17T04:28:45.837279+0000","last_undegraded":"2024-09-17T04:28:45.837279+0000","last_fullsized":"2024-09-17T04:28:45.837279+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:21.789405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"72'2","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.183193+0000","last_change":"2024-09-17T04:28:43.187020+0000","last_active":"2024-09-17T04:28:45.183193+0000","last_peered":"2024-09-17T04:28:45.183193+0000","last_clean":"2024-09-17T04:28:45.183193+0000","last_became_active":"2024-09-17T04:28:43.186227+0000","last_became_peered":"2024-09-17T04:28:43.186227+0000","last_unstale":"2024-09-17T04:28:45.183193+0000","last_undegraded":"2024-09-17T04:28:45.183193+0000","last_fullsized":"2024-09-17T04:28:45.183193+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:06.525929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.391314+0000","last_change":"2024-09-17T04:28:44.185641+0000","last_active":"2024-09-17T04:28:45.391314+0000","last_peered":"2024-09-17T04:28:45.391314+0000","last_clean":"2024-09-17T04:28:45.391314+0000","last_became_active":"2024-09-17T04:28:44.184654+0000","last_became_peered":"2024-09-17T04:28:44.184654+0000","last_unstale":"2024-09-17T04:28:45.391314+0000","last_undegraded":"2024-09-17T04:28:45.391314+0000","last_fullsized":"2024-09-17T04:28:45.391314+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:56.185855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"72'2","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837731+0000","last_change":"2024-09-17T04:28:43.183969+0000","last_active":"2024-09-17T04:28:45.837731+0000","last_peered":"2024-09-17T04:28:45.837731+0000","last_clean":"2024-09-17T04:28:45.837731+0000","last_became_active":"2024-09-17T04:28:43.183603+0000","last_became_peered":"2024-09-17T04:28:43.183603+0000","last_unstale":"2024-09-17T04:28:45.837731+0000","last_undegraded":"2024-09-17T04:28:45.837731+0000","last_fullsized":"2024-09-17T04:28:45.837731+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:47.866148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.181636+0000","last_change":"2024-09-17T04:28:44.183332+0000","last_active":"2024-09-17T04:28:45.181636+0000","last_peered":"2024-09-17T04:28:45.181636+0000","last_clean":"2024-09-17T04:28:45.181636+0000","last_became_active":"2024-09-17T04:28:44.175938+0000","last_became_peered":"2024-09-17T04:28:44.175938+0000","last_unstale":"2024-09-17T04:28:45.181636+0000","last_undegraded":"2024-09-17T04:28:45.181636+0000","last_fullsized":"2024-09-17T04:28:45.181636+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:54:39.841060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"72'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836677+0000","last_change":"2024-09-17T04:28:43.171042+0000","last_active":"2024-09-17T04:28:45.836677+0000","last_peered":"2024-09-17T04:28:45.836677+0000","last_clean":"2024-09-17T04:28:45.836677+0000","last_became_active":"2024-09-17T04:28:43.170733+0000","last_became_peered":"2024-09-17T04:28:43.170733+0000","last_unstale":"2024-09-17T04:28:45.836677+0000","last_undegraded":"2024-09-17T04:28:45.836677+0000","last_fullsized":"2024-09-17T04:28:45.836677+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:33:35.104722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.182786+0000","last_change":"2024-09-17T04:28:44.181651+0000","last_active":"2024-09-17T04:28:45.182786+0000","last_peered":"2024-09-17T04:28:45.182786+0000","last_clean":"2024-09-17T04:28:45.182786+0000","last_became_active":"2024-09-17T04:28:44.181249+0000","last_became_peered":"2024-09-17T04:28:44.181249+0000","last_unstale":"2024-09-17T04:28:45.182786+0000","last_undegraded":"2024-09-17T04:28:45.182786+0000","last_fullsized":"2024-09-17T04:28:45.182786+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:32.865614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"72'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837297+0000","last_change":"2024-09-17T04:28:43.181324+0000","last_active":"2024-09-17T04:28:45.837297+0000","last_peered":"2024-09-17T04:28:45.837297+0000","last_clean":"2024-09-17T04:28:45.837297+0000","last_became_active":"2024-09-17T04:28:43.180647+0000","last_became_peered":"2024-09-17T04:28:43.180647+0000","last_unstale":"2024-09-17T04:28:45.837297+0000","last_undegraded":"2024-09-17T04:28:45.837297+0000","last_fullsized":"2024-09-17T04:28:45.837297+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:01:55.909462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.177079+0000","last_change":"2024-09-17T04:28:44.168250+0000","last_active":"2024-09-17T04:28:45.177079+0000","last_peered":"2024-09-17T04:28:45.177079+0000","last_clean":"2024-09-17T04:28:45.177079+0000","last_became_active":"2024-09-17T04:28:44.166618+0000","last_became_peered":"2024-09-17T04:28:44.166618+0000","last_unstale":"2024-09-17T04:28:45.177079+0000","last_undegraded":"2024-09-17T04:28:45.177079+0000","last_fullsized":"2024-09-17T04:28:45.177079+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:06.198935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836183+0000","last_change":"2024-09-17T04:28:43.163736+0000","last_active":"2024-09-17T04:28:45.836183+0000","last_peered":"2024-09-17T04:28:45.836183+0000","last_clean":"2024-09-17T04:28:45.836183+0000","last_became_active":"2024-09-17T04:28:43.163431+0000","last_became_peered":"2024-09-17T04:28:43.163431+0000","last_unstale":"2024-09-17T04:28:45.836183+0000","last_undegraded":"2024-09-17T04:28:45.836183+0000","last_fullsized":"2024-09-17T04:28:45.836183+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:41.686144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.186727+0000","last_change":"2024-09-17T04:28:44.182372+0000","last_active":"2024-09-17T04:28:45.186727+0000","last_peered":"2024-09-17T04:28:45.186727+0000","last_clean":"2024-09-17T04:28:45.186727+0000","last_became_active":"2024-09-17T04:28:44.181995+0000","last_became_peered":"2024-09-17T04:28:44.181995+0000","last_unstale":"2024-09-17T04:28:45.186727+0000","last_undegraded":"2024-09-17T04:28:45.186727+0000","last_fullsized":"2024-09-17T04:28:45.186727+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:21:53.031978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.177046+0000","last_change":"2024-09-17T04:28:43.163133+0000","last_active":"2024-09-17T04:28:45.177046+0000","last_peered":"2024-09-17T04:28:45.177046+0000","last_clean":"2024-09-17T04:28:45.177046+0000","last_became_active":"2024-09-17T04:28:43.162404+0000","last_became_peered":"2024-09-17T04:28:43.162404+0000","last_unstale":"2024-09-17T04:28:45.177046+0000","last_undegraded":"2024-09-17T04:28:45.177046+0000","last_fullsized":"2024-09-17T04:28:45.177046+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:47:51.349077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.182070+0000","last_change":"2024-09-17T04:28:44.175084+0000","last_active":"2024-09-17T04:28:45.182070+0000","last_peered":"2024-09-17T04:28:45.182070+0000","last_clean":"2024-09-17T04:28:45.182070+0000","last_became_active":"2024-09-17T04:28:44.174333+0000","last_became_peered":"2024-09-17T04:28:44.174333+0000","last_unstale":"2024-09-17T04:28:45.182070+0000","last_undegraded":"2024-09-17T04:28:45.182070+0000","last_fullsized":"2024-09-17T04:28:45.182070+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:13.813644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.183244+0000","last_change":"2024-09-17T04:28:43.174543+0000","last_active":"2024-09-17T04:28:45.183244+0000","last_peered":"2024-09-17T04:28:45.183244+0000","last_clean":"2024-09-17T04:28:45.183244+0000","last_became_active":"2024-09-17T04:28:43.174199+0000","last_became_peered":"2024-09-17T04:28:43.174199+0000","last_unstale":"2024-09-17T04:28:45.183244+0000","last_undegraded":"2024-09-17T04:28:45.183244+0000","last_fullsized":"2024-09-17T04:28:45.183244+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:22.807921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.391422+0000","last_change":"2024-09-17T04:28:44.187251+0000","last_active":"2024-09-17T04:28:45.391422+0000","last_peered":"2024-09-17T04:28:45.391422+0000","last_clean":"2024-09-17T04:28:45.391422+0000","last_became_active":"2024-09-17T04:28:44.186817+0000","last_became_peered":"2024-09-17T04:28:44.186817+0000","last_unstale":"2024-09-17T04:28:45.391422+0000","last_undegraded":"2024-09-17T04:28:45.391422+0000","last_fullsized":"2024-09-17T04:28:45.391422+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:54.411990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837861+0000","last_change":"2024-09-17T04:28:43.185414+0000","last_active":"2024-09-17T04:28:45.837861+0000","last_peered":"2024-09-17T04:28:45.837861+0000","last_clean":"2024-09-17T04:28:45.837861+0000","last_became_active":"2024-09-17T04:28:43.184969+0000","last_became_peered":"2024-09-17T04:28:43.184969+0000","last_unstale":"2024-09-17T04:28:45.837861+0000","last_undegraded":"2024-09-17T04:28:45.837861+0000","last_fullsized":"2024-09-17T04:28:45.837861+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:58.068125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837483+0000","last_change":"2024-09-17T04:28:44.187210+0000","last_active":"2024-09-17T04:28:45.837483+0000","last_peered":"2024-09-17T04:28:45.837483+0000","last_clean":"2024-09-17T04:28:45.837483+0000","last_became_active":"2024-09-17T04:28:44.186603+0000","last_became_peered":"2024-09-17T04:28:44.186603+0000","last_unstale":"2024-09-17T04:28:45.837483+0000","last_undegraded":"2024-09-17T04:28:45.837483+0000","last_fullsized":"2024-09-17T04:28:45.837483+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:01.266881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"72'3","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836958+0000","last_change":"2024-09-17T04:28:43.172982+0000","last_active":"2024-09-17T04:28:45.836958+0000","last_peered":"2024-09-17T04:28:45.836958+0000","last_clean":"2024-09-17T04:28:45.836958+0000","last_became_active":"2024-09-17T04:28:43.172659+0000","last_became_peered":"2024-09-17T04:28:43.172659+0000","last_unstale":"2024-09-17T04:28:45.836958+0000","last_undegraded":"2024-09-17T04:28:45.836958+0000","last_fullsized":"2024-09-17T04:28:45.836958+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:15.908868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.177700+0000","last_change":"2024-09-17T04:28:44.181186+0000","last_active":"2024-09-17T04:28:45.177700+0000","last_peered":"2024-09-17T04:28:45.177700+0000","last_clean":"2024-09-17T04:28:45.177700+0000","last_became_active":"2024-09-17T04:28:44.180853+0000","last_became_peered":"2024-09-17T04:28:44.180853+0000","last_unstale":"2024-09-17T04:28:45.177700+0000","last_undegraded":"2024-09-17T04:28:45.177700+0000","last_fullsized":"2024-09-17T04:28:45.177700+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:55:00.641278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"72'6","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837075+0000","last_change":"2024-09-17T04:28:43.168326+0000","last_active":"2024-09-17T04:28:45.837075+0000","last_peered":"2024-09-17T04:28:45.837075+0000","last_clean":"2024-09-17T04:28:45.837075+0000","last_became_active":"2024-09-17T04:28:43.167133+0000","last_became_peered":"2024-09-17T04:28:43.167133+0000","last_unstale":"2024-09-17T04:28:45.837075+0000","last_undegraded":"2024-09-17T04:28:45.837075+0000","last_fullsized":"2024-09-17T04:28:45.837075+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:45:50.645390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.390806+0000","last_change":"2024-09-17T04:28:44.179971+0000","last_active":"2024-09-17T04:28:45.390806+0000","last_peered":"2024-09-17T04:28:45.390806+0000","last_clean":"2024-09-17T04:28:45.390806+0000","last_became_active":"2024-09-17T04:28:44.179660+0000","last_became_peered":"2024-09-17T04:28:44.179660+0000","last_unstale":"2024-09-17T04:28:45.390806+0000","last_undegraded":"2024-09-17T04:28:45.390806+0000","last_fullsized":"2024-09-17T04:28:45.390806+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:24:25.189556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"72'2","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836286+0000","last_change":"2024-09-17T04:28:43.179010+0000","last_active":"2024-09-17T04:28:45.836286+0000","last_peered":"2024-09-17T04:28:45.836286+0000","last_clean":"2024-09-17T04:28:45.836286+0000","last_became_active":"2024-09-17T04:28:43.178520+0000","last_became_peered":"2024-09-17T04:28:43.178520+0000","last_unstale":"2024-09-17T04:28:45.836286+0000","last_undegraded":"2024-09-17T04:28:45.836286+0000","last_fullsized":"2024-09-17T04:28:45.836286+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:04:06.713074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.181013+0000","last_change":"2024-09-17T04:28:44.171317+0000","last_active":"2024-09-17T04:28:45.181013+0000","last_peered":"2024-09-17T04:28:45.181013+0000","last_clean":"2024-09-17T04:28:45.181013+0000","last_became_active":"2024-09-17T04:28:44.170995+0000","last_became_peered":"2024-09-17T04:28:44.170995+0000","last_unstale":"2024-09-17T04:28:45.181013+0000","last_undegraded":"2024-09-17T04:28:45.181013+0000","last_fullsized":"2024-09-17T04:28:45.181013+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:43.084390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837527+0000","last_change":"2024-09-17T04:28:43.180632+0000","last_active":"2024-09-17T04:28:45.837527+0000","last_peered":"2024-09-17T04:28:45.837527+0000","last_clean":"2024-09-17T04:28:45.837527+0000","last_became_active":"2024-09-17T04:28:43.180146+0000","last_became_peered":"2024-09-17T04:28:43.180146+0000","last_unstale":"2024-09-17T04:28:45.837527+0000","last_undegraded":"2024-09-17T04:28:45.837527+0000","last_fullsized":"2024-09-17T04:28:45.837527+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:03.917176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.186730+0000","last_change":"2024-09-17T04:28:44.195022+0000","last_active":"2024-09-17T04:28:45.186730+0000","last_peered":"2024-09-17T04:28:45.186730+0000","last_clean":"2024-09-17T04:28:45.186730+0000","last_became_active":"2024-09-17T04:28:44.194654+0000","last_became_peered":"2024-09-17T04:28:44.194654+0000","last_unstale":"2024-09-17T04:28:45.186730+0000","last_undegraded":"2024-09-17T04:28:45.186730+0000","last_fullsized":"2024-09-17T04:28:45.186730+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:27.037078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.180561+0000","last_change":"2024-09-17T04:28:43.175788+0000","last_active":"2024-09-17T04:28:45.180561+0000","last_peered":"2024-09-17T04:28:45.180561+0000","last_clean":"2024-09-17T04:28:45.180561+0000","last_became_active":"2024-09-17T04:28:43.175486+0000","last_became_peered":"2024-09-17T04:28:43.175486+0000","last_unstale":"2024-09-17T04:28:45.180561+0000","last_undegraded":"2024-09-17T04:28:45.180561+0000","last_fullsized":"2024-09-17T04:28:45.180561+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:56:30.343523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.177218+0000","last_change":"2024-09-17T04:28:44.174721+0000","last_active":"2024-09-17T04:28:45.177218+0000","last_peered":"2024-09-17T04:28:45.177218+0000","last_clean":"2024-09-17T04:28:45.177218+0000","last_became_active":"2024-09-17T04:28:44.174437+0000","last_became_peered":"2024-09-17T04:28:44.174437+0000","last_unstale":"2024-09-17T04:28:45.177218+0000","last_undegraded":"2024-09-17T04:28:45.177218+0000","last_fullsized":"2024-09-17T04:28:45.177218+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:57:13.550686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"72'8","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.181726+0000","last_change":"2024-09-17T04:28:43.175547+0000","last_active":"2024-09-17T04:28:45.181726+0000","last_peered":"2024-09-17T04:28:45.181726+0000","last_clean":"2024-09-17T04:28:45.181726+0000","last_became_active":"2024-09-17T04:28:43.175224+0000","last_became_peered":"2024-09-17T04:28:43.175224+0000","last_unstale":"2024-09-17T04:28:45.181726+0000","last_undegraded":"2024-09-17T04:28:45.181726+0000","last_fullsized":"2024-09-17T04:28:45.181726+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:09.135418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.183404+0000","last_change":"2024-09-17T04:28:44.183124+0000","last_active":"2024-09-17T04:28:45.183404+0000","last_peered":"2024-09-17T04:28:45.183404+0000","last_clean":"2024-09-17T04:28:45.183404+0000","last_became_active":"2024-09-17T04:28:44.181974+0000","last_became_peered":"2024-09-17T04:28:44.181974+0000","last_unstale":"2024-09-17T04:28:45.183404+0000","last_undegraded":"2024-09-17T04:28:45.183404+0000","last_fullsized":"2024-09-17T04:28:45.183404+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:52.267667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"72'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837665+0000","last_change":"2024-09-17T04:28:43.183827+0000","last_active":"2024-09-17T04:28:45.837665+0000","last_peered":"2024-09-17T04:28:45.837665+0000","last_clean":"2024-09-17T04:28:45.837665+0000","last_became_active":"2024-09-17T04:28:43.183492+0000","last_became_peered":"2024-09-17T04:28:43.183492+0000","last_unstale":"2024-09-17T04:28:45.837665+0000","last_undegraded":"2024-09-17T04:28:45.837665+0000","last_fullsized":"2024-09-17T04:28:45.837665+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:02.478321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.391552+0000","last_change":"2024-09-17T04:28:44.192168+0000","last_active":"2024-09-17T04:28:45.391552+0000","last_peered":"2024-09-17T04:28:45.391552+0000","last_clean":"2024-09-17T04:28:45.391552+0000","last_became_active":"2024-09-17T04:28:44.189424+0000","last_became_peered":"2024-09-17T04:28:44.189424+0000","last_unstale":"2024-09-17T04:28:45.391552+0000","last_undegraded":"2024-09-17T04:28:45.391552+0000","last_fullsized":"2024-09-17T04:28:45.391552+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:57.196554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"72'2","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.183288+0000","last_change":"2024-09-17T04:28:43.185602+0000","last_active":"2024-09-17T04:28:45.183288+0000","last_peered":"2024-09-17T04:28:45.183288+0000","last_clean":"2024-09-17T04:28:45.183288+0000","last_became_active":"2024-09-17T04:28:43.185261+0000","last_became_peered":"2024-09-17T04:28:43.185261+0000","last_unstale":"2024-09-17T04:28:45.183288+0000","last_undegraded":"2024-09-17T04:28:45.183288+0000","last_fullsized":"2024-09-17T04:28:45.183288+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:48.674587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836950+0000","last_change":"2024-09-17T04:28:44.176837+0000","last_active":"2024-09-17T04:28:45.836950+0000","last_peered":"2024-09-17T04:28:45.836950+0000","last_clean":"2024-09-17T04:28:45.836950+0000","last_became_active":"2024-09-17T04:28:44.176537+0000","last_became_peered":"2024-09-17T04:28:44.176537+0000","last_unstale":"2024-09-17T04:28:45.836950+0000","last_undegraded":"2024-09-17T04:28:45.836950+0000","last_fullsized":"2024-09-17T04:28:45.836950+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:34.503572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"72'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837383+0000","last_change":"2024-09-17T04:28:43.182926+0000","last_active":"2024-09-17T04:28:45.837383+0000","last_peered":"2024-09-17T04:28:45.837383+0000","last_clean":"2024-09-17T04:28:45.837383+0000","last_became_active":"2024-09-17T04:28:43.179719+0000","last_became_peered":"2024-09-17T04:28:43.179719+0000","last_unstale":"2024-09-17T04:28:45.837383+0000","last_undegraded":"2024-09-17T04:28:45.837383+0000","last_fullsized":"2024-09-17T04:28:45.837383+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:28.997811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.177708+0000","last_change":"2024-09-17T04:28:44.172649+0000","last_active":"2024-09-17T04:28:45.177708+0000","last_peered":"2024-09-17T04:28:45.177708+0000","last_clean":"2024-09-17T04:28:45.177708+0000","last_became_active":"2024-09-17T04:28:44.172326+0000","last_became_peered":"2024-09-17T04:28:44.172326+0000","last_unstale":"2024-09-17T04:28:45.177708+0000","last_undegraded":"2024-09-17T04:28:45.177708+0000","last_fullsized":"2024-09-17T04:28:45.177708+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:16:23.152782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.391053+0000","last_change":"2024-09-17T04:28:43.184237+0000","last_active":"2024-09-17T04:28:45.391053+0000","last_peered":"2024-09-17T04:28:45.391053+0000","last_clean":"2024-09-17T04:28:45.391053+0000","last_became_active":"2024-09-17T04:28:43.183879+0000","last_became_peered":"2024-09-17T04:28:43.183879+0000","last_unstale":"2024-09-17T04:28:45.391053+0000","last_undegraded":"2024-09-17T04:28:45.391053+0000","last_fullsized":"2024-09-17T04:28:45.391053+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:27.493169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836817+0000","last_change":"2024-09-17T04:28:44.179419+0000","last_active":"2024-09-17T04:28:45.836817+0000","last_peered":"2024-09-17T04:28:45.836817+0000","last_clean":"2024-09-17T04:28:45.836817+0000","last_became_active":"2024-09-17T04:28:44.179132+0000","last_became_peered":"2024-09-17T04:28:44.179132+0000","last_unstale":"2024-09-17T04:28:45.836817+0000","last_undegraded":"2024-09-17T04:28:45.836817+0000","last_fullsized":"2024-09-17T04:28:45.836817+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:47:46.546993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837616+0000","last_change":"2024-09-17T04:28:43.181357+0000","last_active":"2024-09-17T04:28:45.837616+0000","last_peered":"2024-09-17T04:28:45.837616+0000","last_clean":"2024-09-17T04:28:45.837616+0000","last_became_active":"2024-09-17T04:28:43.181053+0000","last_became_peered":"2024-09-17T04:28:43.181053+0000","last_unstale":"2024-09-17T04:28:45.837616+0000","last_undegraded":"2024-09-17T04:28:45.837616+0000","last_fullsized":"2024-09-17T04:28:45.837616+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:53:34.585332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836565+0000","last_change":"2024-09-17T04:28:44.188889+0000","last_active":"2024-09-17T04:28:45.836565+0000","last_peered":"2024-09-17T04:28:45.836565+0000","last_clean":"2024-09-17T04:28:45.836565+0000","last_became_active":"2024-09-17T04:28:44.188524+0000","last_became_peered":"2024-09-17T04:28:44.188524+0000","last_unstale":"2024-09-17T04:28:45.836565+0000","last_undegraded":"2024-09-17T04:28:45.836565+0000","last_fullsized":"2024-09-17T04:28:45.836565+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:17.290337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.180828+0000","last_change":"2024-09-17T04:28:43.184098+0000","last_active":"2024-09-17T04:28:45.180828+0000","last_peered":"2024-09-17T04:28:45.180828+0000","last_clean":"2024-09-17T04:28:45.180828+0000","last_became_active":"2024-09-17T04:28:43.178392+0000","last_became_peered":"2024-09-17T04:28:43.178392+0000","last_unstale":"2024-09-17T04:28:45.180828+0000","last_undegraded":"2024-09-17T04:28:45.180828+0000","last_fullsized":"2024-09-17T04:28:45.180828+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:26.238316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836874+0000","last_change":"2024-09-17T04:28:44.186208+0000","last_active":"2024-09-17T04:28:45.836874+0000","last_peered":"2024-09-17T04:28:45.836874+0000","last_clean":"2024-09-17T04:28:45.836874+0000","last_became_active":"2024-09-17T04:28:44.185834+0000","last_became_peered":"2024-09-17T04:28:44.185834+0000","last_unstale":"2024-09-17T04:28:45.836874+0000","last_undegraded":"2024-09-17T04:28:45.836874+0000","last_fullsized":"2024-09-17T04:28:45.836874+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:24:29.707796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.186626+0000","last_change":"2024-09-17T04:28:43.177447+0000","last_active":"2024-09-17T04:28:45.186626+0000","last_peered":"2024-09-17T04:28:45.186626+0000","last_clean":"2024-09-17T04:28:45.186626+0000","last_became_active":"2024-09-17T04:28:43.177133+0000","last_became_peered":"2024-09-17T04:28:43.177133+0000","last_unstale":"2024-09-17T04:28:45.186626+0000","last_undegraded":"2024-09-17T04:28:45.186626+0000","last_fullsized":"2024-09-17T04:28:45.186626+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:44.270517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.187229+0000","last_change":"2024-09-17T04:28:44.181280+0000","last_active":"2024-09-17T04:28:45.187229+0000","last_peered":"2024-09-17T04:28:45.187229+0000","last_clean":"2024-09-17T04:28:45.187229+0000","last_became_active":"2024-09-17T04:28:44.180969+0000","last_became_peered":"2024-09-17T04:28:44.180969+0000","last_unstale":"2024-09-17T04:28:45.187229+0000","last_undegraded":"2024-09-17T04:28:45.187229+0000","last_fullsized":"2024-09-17T04:28:45.187229+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:24:25.189556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"72'2","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837733+0000","last_change":"2024-09-17T04:28:43.183089+0000","last_active":"2024-09-17T04:28:45.837733+0000","last_peered":"2024-09-17T04:28:45.837733+0000","last_clean":"2024-09-17T04:28:45.837733+0000","last_became_active":"2024-09-17T04:28:43.179747+0000","last_became_peered":"2024-09-17T04:28:43.179747+0000","last_unstale":"2024-09-17T04:28:45.837733+0000","last_undegraded":"2024-09-17T04:28:45.837733+0000","last_fullsized":"2024-09-17T04:28:45.837733+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:08.291909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.181075+0000","last_change":"2024-09-17T04:28:44.179171+0000","last_active":"2024-09-17T04:28:45.181075+0000","last_peered":"2024-09-17T04:28:45.181075+0000","last_clean":"2024-09-17T04:28:45.181075+0000","last_became_active":"2024-09-17T04:28:44.178313+0000","last_became_peered":"2024-09-17T04:28:44.178313+0000","last_unstale":"2024-09-17T04:28:45.181075+0000","last_undegraded":"2024-09-17T04:28:45.181075+0000","last_fullsized":"2024-09-17T04:28:45.181075+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:21.723977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"73'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.182251+0000","last_change":"2024-09-17T04:28:43.178629+0000","last_active":"2024-09-17T04:28:45.182251+0000","last_peered":"2024-09-17T04:28:45.182251+0000","last_clean":"2024-09-17T04:28:45.182251+0000","last_became_active":"2024-09-17T04:28:43.176654+0000","last_became_peered":"2024-09-17T04:28:43.176654+0000","last_unstale":"2024-09-17T04:28:45.182251+0000","last_undegraded":"2024-09-17T04:28:45.182251+0000","last_fullsized":"2024-09-17T04:28:45.182251+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:27.493169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.391051+0000","last_change":"2024-09-17T04:28:44.187809+0000","last_active":"2024-09-17T04:28:45.391051+0000","last_peered":"2024-09-17T04:28:45.391051+0000","last_clean":"2024-09-17T04:28:45.391051+0000","last_became_active":"2024-09-17T04:28:44.187476+0000","last_became_peered":"2024-09-17T04:28:44.187476+0000","last_unstale":"2024-09-17T04:28:45.391051+0000","last_undegraded":"2024-09-17T04:28:45.391051+0000","last_fullsized":"2024-09-17T04:28:45.391051+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:53.823460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.177875+0000","last_change":"2024-09-17T04:28:43.176154+0000","last_active":"2024-09-17T04:28:45.177875+0000","last_peered":"2024-09-17T04:28:45.177875+0000","last_clean":"2024-09-17T04:28:45.177875+0000","last_became_active":"2024-09-17T04:28:43.175853+0000","last_became_peered":"2024-09-17T04:28:43.175853+0000","last_unstale":"2024-09-17T04:28:45.177875+0000","last_undegraded":"2024-09-17T04:28:45.177875+0000","last_fullsized":"2024-09-17T04:28:45.177875+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:19:36.188334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.391450+0000","last_change":"2024-09-17T04:28:44.192356+0000","last_active":"2024-09-17T04:28:45.391450+0000","last_peered":"2024-09-17T04:28:45.391450+0000","last_clean":"2024-09-17T04:28:45.391450+0000","last_became_active":"2024-09-17T04:28:44.190656+0000","last_became_peered":"2024-09-17T04:28:44.190656+0000","last_unstale":"2024-09-17T04:28:45.391450+0000","last_undegraded":"2024-09-17T04:28:45.391450+0000","last_fullsized":"2024-09-17T04:28:45.391450+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:50:34.240788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.187058+0000","last_change":"2024-09-17T04:28:43.178576+0000","last_active":"2024-09-17T04:28:45.187058+0000","last_peered":"2024-09-17T04:28:45.187058+0000","last_clean":"2024-09-17T04:28:45.187058+0000","last_became_active":"2024-09-17T04:28:43.178301+0000","last_became_peered":"2024-09-17T04:28:43.178301+0000","last_unstale":"2024-09-17T04:28:45.187058+0000","last_undegraded":"2024-09-17T04:28:45.187058+0000","last_fullsized":"2024-09-17T04:28:45.187058+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:05:46.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.177384+0000","last_change":"2024-09-17T04:28:44.178513+0000","last_active":"2024-09-17T04:28:45.177384+0000","last_peered":"2024-09-17T04:28:45.177384+0000","last_clean":"2024-09-17T04:28:45.177384+0000","last_became_active":"2024-09-17T04:28:44.173870+0000","last_became_peered":"2024-09-17T04:28:44.173870+0000","last_unstale":"2024-09-17T04:28:45.177384+0000","last_undegraded":"2024-09-17T04:28:45.177384+0000","last_fullsized":"2024-09-17T04:28:45.177384+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:20.738804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.177575+0000","last_change":"2024-09-17T04:28:43.168256+0000","last_active":"2024-09-17T04:28:45.177575+0000","last_peered":"2024-09-17T04:28:45.177575+0000","last_clean":"2024-09-17T04:28:45.177575+0000","last_became_active":"2024-09-17T04:28:43.167900+0000","last_became_peered":"2024-09-17T04:28:43.167900+0000","last_unstale":"2024-09-17T04:28:45.177575+0000","last_undegraded":"2024-09-17T04:28:45.177575+0000","last_fullsized":"2024-09-17T04:28:45.177575+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:05:25.864041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837477+0000","last_change":"2024-09-17T04:28:44.189003+0000","last_active":"2024-09-17T04:28:45.837477+0000","last_peered":"2024-09-17T04:28:45.837477+0000","last_clean":"2024-09-17T04:28:45.837477+0000","last_became_active":"2024-09-17T04:28:44.188693+0000","last_became_peered":"2024-09-17T04:28:44.188693+0000","last_unstale":"2024-09-17T04:28:45.837477+0000","last_undegraded":"2024-09-17T04:28:45.837477+0000","last_fullsized":"2024-09-17T04:28:45.837477+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:53.321976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836083+0000","last_change":"2024-09-17T04:28:43.170357+0000","last_active":"2024-09-17T04:28:45.836083+0000","last_peered":"2024-09-17T04:28:45.836083+0000","last_clean":"2024-09-17T04:28:45.836083+0000","last_became_active":"2024-09-17T04:28:43.170017+0000","last_became_peered":"2024-09-17T04:28:43.170017+0000","last_unstale":"2024-09-17T04:28:45.836083+0000","last_undegraded":"2024-09-17T04:28:45.836083+0000","last_fullsized":"2024-09-17T04:28:45.836083+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:45:31.690608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836171+0000","last_change":"2024-09-17T04:28:44.177117+0000","last_active":"2024-09-17T04:28:45.836171+0000","last_peered":"2024-09-17T04:28:45.836171+0000","last_clean":"2024-09-17T04:28:45.836171+0000","last_became_active":"2024-09-17T04:28:44.176720+0000","last_became_peered":"2024-09-17T04:28:44.176720+0000","last_unstale":"2024-09-17T04:28:45.836171+0000","last_undegraded":"2024-09-17T04:28:45.836171+0000","last_fullsized":"2024-09-17T04:28:45.836171+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:13.369221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.182125+0000","last_change":"2024-09-17T04:28:43.177322+0000","last_active":"2024-09-17T04:28:45.182125+0000","last_peered":"2024-09-17T04:28:45.182125+0000","last_clean":"2024-09-17T04:28:45.182125+0000","last_became_active":"2024-09-17T04:28:43.177004+0000","last_became_peered":"2024-09-17T04:28:43.177004+0000","last_unstale":"2024-09-17T04:28:45.182125+0000","last_undegraded":"2024-09-17T04:28:45.182125+0000","last_fullsized":"2024-09-17T04:28:45.182125+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:58.922041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836859+0000","last_change":"2024-09-17T04:28:44.192783+0000","last_active":"2024-09-17T04:28:45.836859+0000","last_peered":"2024-09-17T04:28:45.836859+0000","last_clean":"2024-09-17T04:28:45.836859+0000","last_became_active":"2024-09-17T04:28:44.191353+0000","last_became_peered":"2024-09-17T04:28:44.191353+0000","last_unstale":"2024-09-17T04:28:45.836859+0000","last_undegraded":"2024-09-17T04:28:45.836859+0000","last_fullsized":"2024-09-17T04:28:45.836859+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:02:14.822715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837279+0000","last_change":"2024-09-17T04:28:43.183805+0000","last_active":"2024-09-17T04:28:45.837279+0000","last_peered":"2024-09-17T04:28:45.837279+0000","last_clean":"2024-09-17T04:28:45.837279+0000","last_became_active":"2024-09-17T04:28:43.183414+0000","last_became_peered":"2024-09-17T04:28:43.183414+0000","last_unstale":"2024-09-17T04:28:45.837279+0000","last_undegraded":"2024-09-17T04:28:45.837279+0000","last_fullsized":"2024-09-17T04:28:45.837279+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:46:21.622130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836616+0000","last_change":"2024-09-17T04:28:44.189995+0000","last_active":"2024-09-17T04:28:45.836616+0000","last_peered":"2024-09-17T04:28:45.836616+0000","last_clean":"2024-09-17T04:28:45.836616+0000","last_became_active":"2024-09-17T04:28:44.189715+0000","last_became_peered":"2024-09-17T04:28:44.189715+0000","last_unstale":"2024-09-17T04:28:45.836616+0000","last_undegraded":"2024-09-17T04:28:45.836616+0000","last_fullsized":"2024-09-17T04:28:45.836616+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:25.483700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"72'2","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837473+0000","last_change":"2024-09-17T04:28:43.184519+0000","last_active":"2024-09-17T04:28:45.837473+0000","last_peered":"2024-09-17T04:28:45.837473+0000","last_clean":"2024-09-17T04:28:45.837473+0000","last_became_active":"2024-09-17T04:28:43.183311+0000","last_became_peered":"2024-09-17T04:28:43.183311+0000","last_unstale":"2024-09-17T04:28:45.837473+0000","last_undegraded":"2024-09-17T04:28:45.837473+0000","last_fullsized":"2024-09-17T04:28:45.837473+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:41.963229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":208,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.390756+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:28:45.390756+0000","last_peered":"2024-09-17T04:28:45.390756+0000","last_clean":"2024-09-17T04:28:45.390756+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:28:45.390756+0000","last_undegraded":"2024-09-17T04:28:45.390756+0000","last_fullsized":"2024-09-17T04:28:45.390756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476798,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1478656,"data_stored":958865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476797,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":884736,"data_stored":367973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476798,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":880640,"data_stored":367971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476798,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":880640,"data_stored":367951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509501,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32060,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339780,"statfs":{"total":96636764160,"available":96603934720,"internally_reserved":0,"allocated":888832,"data_stored":369493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509500,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":897024,"data_stored":370109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509500,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1470464,"data_stored":958249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542205,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32060,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339780,"statfs":{"total":96636764160,"available":96603934720,"internally_reserved":0,"allocated":884736,"data_stored":367993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:28:53.549 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:28:53.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.781+0000 7f52fe600640 1 -- 172.21.15.100:0/3469501525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52f806f9a0 msgr2=0x7f52f80b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:53.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.781+0000 7f52fe600640 1 --2- 172.21.15.100:0/3469501525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52f806f9a0 0x7f52f80b2f60 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f52f0005fd0 tx=0x7f52f002a820 comp rx=0 tx=0).stop 2024-09-17T04:28:53.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.781+0000 7f52fe600640 1 -- 172.21.15.100:0/3469501525 shutdown_connections 2024-09-17T04:28:53.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.781+0000 7f52fe600640 1 --2- 172.21.15.100:0/3469501525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52f806f9a0 0x7f52f80b2f60 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:53.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.781+0000 7f52fe600640 1 -- 172.21.15.100:0/3469501525 >> 172.21.15.100:0/3469501525 conn(0x7f52f80b1e40 msgr2=0x7f52f80b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:53.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.781+0000 7f52fe600640 1 -- 172.21.15.100:0/3469501525 shutdown_connections 2024-09-17T04:28:53.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.781+0000 7f52fe600640 1 -- 172.21.15.100:0/3469501525 wait complete. 2024-09-17T04:28:53.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.783+0000 7f52fe600640 1 Processor -- start 2024-09-17T04:28:53.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.783+0000 7f52fe600640 1 -- start start 2024-09-17T04:28:53.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.785+0000 7f52fe600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52f80ad9c0 0x7f52f81f4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:53.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.785+0000 7f52fe600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52f80af170 0x7f52f809d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:53.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.785+0000 7f52fe600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52f8158400 0x7f52f8158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:53.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.785+0000 7f52fe600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f52f815a4c0 con 0x7f52f80ad9c0 2024-09-17T04:28:53.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.785+0000 7f52fe600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f52f815a340 con 0x7f52f8158400 2024-09-17T04:28:53.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.785+0000 7f52fe600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f52f815d670 con 0x7f52f80af170 2024-09-17T04:28:53.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.785+0000 7f52fd200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52f80ad9c0 0x7f52f81f4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:53.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.785+0000 7f52fd200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52f80ad9c0 0x7f52f81f4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38846/0 (socket says 172.21.15.100:38846) 2024-09-17T04:28:53.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.785+0000 7f52fd200640 1 -- 172.21.15.100:0/831007810 learned_addr learned my addr 172.21.15.100:0/831007810 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:53.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.785+0000 7f52fdc00640 1 --2- 172.21.15.100:0/831007810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52f8158400 0x7f52f8158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:53.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.785+0000 7f52efe00640 1 --2- 172.21.15.100:0/831007810 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52f80af170 0x7f52f809d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:53.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.786+0000 7f52fd200640 1 -- 172.21.15.100:0/831007810 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52f80af170 msgr2=0x7f52f809d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:53.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.786+0000 7f52fd200640 1 --2- 172.21.15.100:0/831007810 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52f80af170 0x7f52f809d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:53.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.786+0000 7f52fd200640 1 -- 172.21.15.100:0/831007810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52f8158400 msgr2=0x7f52f8158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:53.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.786+0000 7f52fd200640 1 --2- 172.21.15.100:0/831007810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52f8158400 0x7f52f8158810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:53.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.786+0000 7f52fd200640 1 -- 172.21.15.100:0/831007810 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f8158d50 con 0x7f52f80ad9c0 2024-09-17T04:28:53.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.786+0000 7f52fdc00640 1 --2- 172.21.15.100:0/831007810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52f8158400 0x7f52f8158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:28:53.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.786+0000 7f52fd200640 1 --2- 172.21.15.100:0/831007810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52f80ad9c0 0x7f52f81f4ea0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f52e800c3a0 tx=0x7f52e800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:53.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.786+0000 7f52f5600640 1 -- 172.21.15.100:0/831007810 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e8017070 con 0x7f52f80ad9c0 2024-09-17T04:28:53.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.786+0000 7f52f5600640 1 -- 172.21.15.100:0/831007810 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52e80035f0 con 0x7f52f80ad9c0 2024-09-17T04:28:53.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.786+0000 7f52f5600640 1 -- 172.21.15.100:0/831007810 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e8003a40 con 0x7f52f80ad9c0 2024-09-17T04:28:53.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.786+0000 7f52fe600640 1 -- 172.21.15.100:0/831007810 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52f80cf860 con 0x7f52f80ad9c0 2024-09-17T04:28:53.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.786+0000 7f52fe600640 1 -- 172.21.15.100:0/831007810 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52f8156f90 con 0x7f52f80ad9c0 2024-09-17T04:28:53.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.787+0000 7f52fe600640 1 -- 172.21.15.100:0/831007810 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52f80b2f60 con 0x7f52f80ad9c0 2024-09-17T04:28:53.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.788+0000 7f52f5600640 1 -- 172.21.15.100:0/831007810 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f52e801a070 con 0x7f52f80ad9c0 2024-09-17T04:28:53.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.789+0000 7f52f5600640 1 --2- 172.21.15.100:0/831007810 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f52d4078f20 0x7f52d407b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:53.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.790+0000 7f52f5600640 1 -- 172.21.15.100:0/831007810 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5767+0+0 (secure 0 0 0) 0x7f52e809a6c0 con 0x7f52f80ad9c0 2024-09-17T04:28:53.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.790+0000 7f52efe00640 1 --2- 172.21.15.100:0/831007810 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f52d4078f20 0x7f52d407b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:53.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.791+0000 7f52efe00640 1 --2- 172.21.15.100:0/831007810 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f52d4078f20 0x7f52d407b430 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f52f80cea90 tx=0x7f52e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:53.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.794+0000 7f52f5600640 1 -- 172.21.15.100:0/831007810 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f52f8158d50 con 0x7f52f80ad9c0 2024-09-17T04:28:53.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.972+0000 7f52fe600640 1 -- 172.21.15.100:0/831007810 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f52f80b0c30 con 0x7f52d4078f20 2024-09-17T04:28:53.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.977+0000 7f52f5600640 1 -- 172.21.15.100:0/831007810 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252010 (secure 0 0 0) 0x7f52f80b0c30 con 0x7f52d4078f20 2024-09-17T04:28:53.980 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:28:53.981 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:28:53.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.981+0000 7f52fe600640 1 -- 172.21.15.100:0/831007810 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f52d4078f20 msgr2=0x7f52d407b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:53.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.981+0000 7f52fe600640 1 --2- 172.21.15.100:0/831007810 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f52d4078f20 0x7f52d407b430 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f52f80cea90 tx=0x7f52e0009040 comp rx=0 tx=0).stop 2024-09-17T04:28:53.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.981+0000 7f52fe600640 1 -- 172.21.15.100:0/831007810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52f80ad9c0 msgr2=0x7f52f81f4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:53.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.981+0000 7f52fe600640 1 --2- 172.21.15.100:0/831007810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52f80ad9c0 0x7f52f81f4ea0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f52e800c3a0 tx=0x7f52e800c860 comp rx=0 tx=0).stop 2024-09-17T04:28:53.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.981+0000 7f52fe600640 1 -- 172.21.15.100:0/831007810 shutdown_connections 2024-09-17T04:28:53.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.981+0000 7f52fe600640 1 --2- 172.21.15.100:0/831007810 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f52d4078f20 0x7f52d407b430 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:53.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.981+0000 7f52fe600640 1 --2- 172.21.15.100:0/831007810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52f8158400 0x7f52f8158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:53.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.981+0000 7f52fe600640 1 --2- 172.21.15.100:0/831007810 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52f80af170 0x7f52f809d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:53.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.981+0000 7f52fe600640 1 --2- 172.21.15.100:0/831007810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52f80ad9c0 0x7f52f81f4ea0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:53.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.981+0000 7f52fe600640 1 -- 172.21.15.100:0/831007810 >> 172.21.15.100:0/831007810 conn(0x7f52f80b1e40 msgr2=0x7f52f80747e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:53.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.981+0000 7f52fe600640 1 -- 172.21.15.100:0/831007810 shutdown_connections 2024-09-17T04:28:53.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:53.981+0000 7f52fe600640 1 -- 172.21.15.100:0/831007810 wait complete. 2024-09-17T04:28:54.005 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2024-09-17T04:28:53.325748+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257604,"kb_used_data":8072,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717116,"statfs":{"total":773094113280,"available":772830326784,"internally_reserved":0,"allocated":8265728,"data_stored":4128604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.168755"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.391360+0000","last_change":"2024-09-17T04:28:44.178546+0000","last_active":"2024-09-17T04:28:45.391360+0000","last_peered":"2024-09-17T04:28:45.391360+0000","last_clean":"2024-09-17T04:28:45.391360+0000","last_became_active":"2024-09-17T04:28:44.178226+0000","last_became_peered":"2024-09-17T04:28:44.178226+0000","last_unstale":"2024-09-17T04:28:45.391360+0000","last_undegraded":"2024-09-17T04:28:45.391360+0000","last_fullsized":"2024-09-17T04:28:45.391360+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:42.957899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837706+0000","last_change":"2024-09-17T04:28:44.187195+0000","last_active":"2024-09-17T04:28:45.837706+0000","last_peered":"2024-09-17T04:28:45.837706+0000","last_clean":"2024-09-17T04:28:45.837706+0000","last_became_active":"2024-09-17T04:28:44.186765+0000","last_became_peered":"2024-09-17T04:28:44.186765+0000","last_unstale":"2024-09-17T04:28:45.837706+0000","last_undegraded":"2024-09-17T04:28:45.837706+0000","last_fullsized":"2024-09-17T04:28:45.837706+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:59:32.661314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836882+0000","last_change":"2024-09-17T04:28:44.187901+0000","last_active":"2024-09-17T04:28:45.836882+0000","last_peered":"2024-09-17T04:28:45.836882+0000","last_clean":"2024-09-17T04:28:45.836882+0000","last_became_active":"2024-09-17T04:28:44.187552+0000","last_became_peered":"2024-09-17T04:28:44.187552+0000","last_unstale":"2024-09-17T04:28:45.836882+0000","last_undegraded":"2024-09-17T04:28:45.836882+0000","last_fullsized":"2024-09-17T04:28:45.836882+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:33.519054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836695+0000","last_change":"2024-09-17T04:28:44.184158+0000","last_active":"2024-09-17T04:28:45.836695+0000","last_peered":"2024-09-17T04:28:45.836695+0000","last_clean":"2024-09-17T04:28:45.836695+0000","last_became_active":"2024-09-17T04:28:44.183807+0000","last_became_peered":"2024-09-17T04:28:44.183807+0000","last_unstale":"2024-09-17T04:28:45.836695+0000","last_undegraded":"2024-09-17T04:28:45.836695+0000","last_fullsized":"2024-09-17T04:28:45.836695+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:49.080042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.177322+0000","last_change":"2024-09-17T04:28:44.172010+0000","last_active":"2024-09-17T04:28:45.177322+0000","last_peered":"2024-09-17T04:28:45.177322+0000","last_clean":"2024-09-17T04:28:45.177322+0000","last_became_active":"2024-09-17T04:28:44.171633+0000","last_became_peered":"2024-09-17T04:28:44.171633+0000","last_unstale":"2024-09-17T04:28:45.177322+0000","last_undegraded":"2024-09-17T04:28:45.177322+0000","last_fullsized":"2024-09-17T04:28:45.177322+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:38:43.358588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837029+0000","last_change":"2024-09-17T04:28:44.190298+0000","last_active":"2024-09-17T04:28:45.837029+0000","last_peered":"2024-09-17T04:28:45.837029+0000","last_clean":"2024-09-17T04:28:45.837029+0000","last_became_active":"2024-09-17T04:28:44.187991+0000","last_became_peered":"2024-09-17T04:28:44.187991+0000","last_unstale":"2024-09-17T04:28:45.837029+0000","last_undegraded":"2024-09-17T04:28:45.837029+0000","last_fullsized":"2024-09-17T04:28:45.837029+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:09.319597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.180600+0000","last_change":"2024-09-17T04:28:44.180281+0000","last_active":"2024-09-17T04:28:45.180600+0000","last_peered":"2024-09-17T04:28:45.180600+0000","last_clean":"2024-09-17T04:28:45.180600+0000","last_became_active":"2024-09-17T04:28:44.179971+0000","last_became_peered":"2024-09-17T04:28:44.179971+0000","last_unstale":"2024-09-17T04:28:45.180600+0000","last_undegraded":"2024-09-17T04:28:45.180600+0000","last_fullsized":"2024-09-17T04:28:45.180600+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:28.683793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837098+0000","last_change":"2024-09-17T04:28:44.190967+0000","last_active":"2024-09-17T04:28:45.837098+0000","last_peered":"2024-09-17T04:28:45.837098+0000","last_clean":"2024-09-17T04:28:45.837098+0000","last_became_active":"2024-09-17T04:28:44.190523+0000","last_became_peered":"2024-09-17T04:28:44.190523+0000","last_unstale":"2024-09-17T04:28:45.837098+0000","last_undegraded":"2024-09-17T04:28:45.837098+0000","last_fullsized":"2024-09-17T04:28:45.837098+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:42.525462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.178028+0000","last_change":"2024-09-17T04:28:44.180005+0000","last_active":"2024-09-17T04:28:45.178028+0000","last_peered":"2024-09-17T04:28:45.178028+0000","last_clean":"2024-09-17T04:28:45.178028+0000","last_became_active":"2024-09-17T04:28:44.179352+0000","last_became_peered":"2024-09-17T04:28:44.179352+0000","last_unstale":"2024-09-17T04:28:45.178028+0000","last_undegraded":"2024-09-17T04:28:45.178028+0000","last_fullsized":"2024-09-17T04:28:45.178028+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:21:08.710175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.183033+0000","last_change":"2024-09-17T04:28:44.166800+0000","last_active":"2024-09-17T04:28:45.183033+0000","last_peered":"2024-09-17T04:28:45.183033+0000","last_clean":"2024-09-17T04:28:45.183033+0000","last_became_active":"2024-09-17T04:28:44.166256+0000","last_became_peered":"2024-09-17T04:28:44.166256+0000","last_unstale":"2024-09-17T04:28:45.183033+0000","last_undegraded":"2024-09-17T04:28:45.183033+0000","last_fullsized":"2024-09-17T04:28:45.183033+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:18.874211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.391108+0000","last_change":"2024-09-17T04:28:44.183172+0000","last_active":"2024-09-17T04:28:45.391108+0000","last_peered":"2024-09-17T04:28:45.391108+0000","last_clean":"2024-09-17T04:28:45.391108+0000","last_became_active":"2024-09-17T04:28:44.182857+0000","last_became_peered":"2024-09-17T04:28:44.182857+0000","last_unstale":"2024-09-17T04:28:45.391108+0000","last_undegraded":"2024-09-17T04:28:45.391108+0000","last_fullsized":"2024-09-17T04:28:45.391108+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:03.529440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836899+0000","last_change":"2024-09-17T04:28:44.177609+0000","last_active":"2024-09-17T04:28:45.836899+0000","last_peered":"2024-09-17T04:28:45.836899+0000","last_clean":"2024-09-17T04:28:45.836899+0000","last_became_active":"2024-09-17T04:28:44.177243+0000","last_became_peered":"2024-09-17T04:28:44.177243+0000","last_unstale":"2024-09-17T04:28:45.836899+0000","last_undegraded":"2024-09-17T04:28:45.836899+0000","last_fullsized":"2024-09-17T04:28:45.836899+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:30:15.333284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.186892+0000","last_change":"2024-09-17T04:28:44.184657+0000","last_active":"2024-09-17T04:28:45.186892+0000","last_peered":"2024-09-17T04:28:45.186892+0000","last_clean":"2024-09-17T04:28:45.186892+0000","last_became_active":"2024-09-17T04:28:44.184350+0000","last_became_peered":"2024-09-17T04:28:44.184350+0000","last_unstale":"2024-09-17T04:28:45.186892+0000","last_undegraded":"2024-09-17T04:28:45.186892+0000","last_fullsized":"2024-09-17T04:28:45.186892+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:21.723977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.177357+0000","last_change":"2024-09-17T04:28:44.167744+0000","last_active":"2024-09-17T04:28:45.177357+0000","last_peered":"2024-09-17T04:28:45.177357+0000","last_clean":"2024-09-17T04:28:45.177357+0000","last_became_active":"2024-09-17T04:28:44.166731+0000","last_became_peered":"2024-09-17T04:28:44.166731+0000","last_unstale":"2024-09-17T04:28:45.177357+0000","last_undegraded":"2024-09-17T04:28:45.177357+0000","last_fullsized":"2024-09-17T04:28:45.177357+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:59.409832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.187053+0000","last_change":"2024-09-17T04:28:44.187858+0000","last_active":"2024-09-17T04:28:45.187053+0000","last_peered":"2024-09-17T04:28:45.187053+0000","last_clean":"2024-09-17T04:28:45.187053+0000","last_became_active":"2024-09-17T04:28:44.187287+0000","last_became_peered":"2024-09-17T04:28:44.187287+0000","last_unstale":"2024-09-17T04:28:45.187053+0000","last_undegraded":"2024-09-17T04:28:45.187053+0000","last_fullsized":"2024-09-17T04:28:45.187053+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:10:24.302128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.181520+0000","last_change":"2024-09-17T04:28:44.186823+0000","last_active":"2024-09-17T04:28:45.181520+0000","last_peered":"2024-09-17T04:28:45.181520+0000","last_clean":"2024-09-17T04:28:45.181520+0000","last_became_active":"2024-09-17T04:28:44.186451+0000","last_became_peered":"2024-09-17T04:28:44.186451+0000","last_unstale":"2024-09-17T04:28:45.181520+0000","last_undegraded":"2024-09-17T04:28:45.181520+0000","last_fullsized":"2024-09-17T04:28:45.181520+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:27.243441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.178040+0000","last_change":"2024-09-17T04:28:44.172893+0000","last_active":"2024-09-17T04:28:45.178040+0000","last_peered":"2024-09-17T04:28:45.178040+0000","last_clean":"2024-09-17T04:28:45.178040+0000","last_became_active":"2024-09-17T04:28:44.172600+0000","last_became_peered":"2024-09-17T04:28:44.172600+0000","last_unstale":"2024-09-17T04:28:45.178040+0000","last_undegraded":"2024-09-17T04:28:45.178040+0000","last_fullsized":"2024-09-17T04:28:45.178040+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:04:07.718199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837121+0000","last_change":"2024-09-17T04:28:44.186476+0000","last_active":"2024-09-17T04:28:45.837121+0000","last_peered":"2024-09-17T04:28:45.837121+0000","last_clean":"2024-09-17T04:28:45.837121+0000","last_became_active":"2024-09-17T04:28:44.185747+0000","last_became_peered":"2024-09-17T04:28:44.185747+0000","last_unstale":"2024-09-17T04:28:45.837121+0000","last_undegraded":"2024-09-17T04:28:45.837121+0000","last_fullsized":"2024-09-17T04:28:45.837121+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:22.413761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837195+0000","last_change":"2024-09-17T04:28:44.182489+0000","last_active":"2024-09-17T04:28:45.837195+0000","last_peered":"2024-09-17T04:28:45.837195+0000","last_clean":"2024-09-17T04:28:45.837195+0000","last_became_active":"2024-09-17T04:28:44.182178+0000","last_became_peered":"2024-09-17T04:28:44.182178+0000","last_unstale":"2024-09-17T04:28:45.837195+0000","last_undegraded":"2024-09-17T04:28:45.837195+0000","last_fullsized":"2024-09-17T04:28:45.837195+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:55:33.943721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.186906+0000","last_change":"2024-09-17T04:28:44.185354+0000","last_active":"2024-09-17T04:28:45.186906+0000","last_peered":"2024-09-17T04:28:45.186906+0000","last_clean":"2024-09-17T04:28:45.186906+0000","last_became_active":"2024-09-17T04:28:44.184977+0000","last_became_peered":"2024-09-17T04:28:44.184977+0000","last_unstale":"2024-09-17T04:28:45.186906+0000","last_undegraded":"2024-09-17T04:28:45.186906+0000","last_fullsized":"2024-09-17T04:28:45.186906+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:06.029830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.182707+0000","last_change":"2024-09-17T04:28:44.172195+0000","last_active":"2024-09-17T04:28:45.182707+0000","last_peered":"2024-09-17T04:28:45.182707+0000","last_clean":"2024-09-17T04:28:45.182707+0000","last_became_active":"2024-09-17T04:28:44.171667+0000","last_became_peered":"2024-09-17T04:28:44.171667+0000","last_unstale":"2024-09-17T04:28:45.182707+0000","last_undegraded":"2024-09-17T04:28:45.182707+0000","last_fullsized":"2024-09-17T04:28:45.182707+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:20.771270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.182389+0000","last_change":"2024-09-17T04:28:44.176739+0000","last_active":"2024-09-17T04:28:45.182389+0000","last_peered":"2024-09-17T04:28:45.182389+0000","last_clean":"2024-09-17T04:28:45.182389+0000","last_became_active":"2024-09-17T04:28:44.169094+0000","last_became_peered":"2024-09-17T04:28:44.169094+0000","last_unstale":"2024-09-17T04:28:45.182389+0000","last_undegraded":"2024-09-17T04:28:45.182389+0000","last_fullsized":"2024-09-17T04:28:45.182389+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:28.498294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.390724+0000","last_change":"2024-09-17T04:28:44.191151+0000","last_active":"2024-09-17T04:28:45.390724+0000","last_peered":"2024-09-17T04:28:45.390724+0000","last_clean":"2024-09-17T04:28:45.390724+0000","last_became_active":"2024-09-17T04:28:44.190798+0000","last_became_peered":"2024-09-17T04:28:44.190798+0000","last_unstale":"2024-09-17T04:28:45.390724+0000","last_undegraded":"2024-09-17T04:28:45.390724+0000","last_fullsized":"2024-09-17T04:28:45.390724+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:10:24.302128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.390688+0000","last_change":"2024-09-17T04:28:44.194503+0000","last_active":"2024-09-17T04:28:45.390688+0000","last_peered":"2024-09-17T04:28:45.390688+0000","last_clean":"2024-09-17T04:28:45.390688+0000","last_became_active":"2024-09-17T04:28:44.194172+0000","last_became_peered":"2024-09-17T04:28:44.194172+0000","last_unstale":"2024-09-17T04:28:45.390688+0000","last_undegraded":"2024-09-17T04:28:45.390688+0000","last_fullsized":"2024-09-17T04:28:45.390688+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:06.029830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.183118+0000","last_change":"2024-09-17T04:28:44.170440+0000","last_active":"2024-09-17T04:28:45.183118+0000","last_peered":"2024-09-17T04:28:45.183118+0000","last_clean":"2024-09-17T04:28:45.183118+0000","last_became_active":"2024-09-17T04:28:44.169980+0000","last_became_peered":"2024-09-17T04:28:44.169980+0000","last_unstale":"2024-09-17T04:28:45.183118+0000","last_undegraded":"2024-09-17T04:28:45.183118+0000","last_fullsized":"2024-09-17T04:28:45.183118+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:08.481224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.181983+0000","last_change":"2024-09-17T04:28:44.188325+0000","last_active":"2024-09-17T04:28:45.181983+0000","last_peered":"2024-09-17T04:28:45.181983+0000","last_clean":"2024-09-17T04:28:45.181983+0000","last_became_active":"2024-09-17T04:28:44.188006+0000","last_became_peered":"2024-09-17T04:28:44.188006+0000","last_unstale":"2024-09-17T04:28:45.181983+0000","last_undegraded":"2024-09-17T04:28:45.181983+0000","last_fullsized":"2024-09-17T04:28:45.181983+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:16.202008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836922+0000","last_change":"2024-09-17T04:28:44.174317+0000","last_active":"2024-09-17T04:28:45.836922+0000","last_peered":"2024-09-17T04:28:45.836922+0000","last_clean":"2024-09-17T04:28:45.836922+0000","last_became_active":"2024-09-17T04:28:44.173878+0000","last_became_peered":"2024-09-17T04:28:44.173878+0000","last_unstale":"2024-09-17T04:28:45.836922+0000","last_undegraded":"2024-09-17T04:28:45.836922+0000","last_fullsized":"2024-09-17T04:28:45.836922+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:07.283249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.183080+0000","last_change":"2024-09-17T04:28:44.170671+0000","last_active":"2024-09-17T04:28:45.183080+0000","last_peered":"2024-09-17T04:28:45.183080+0000","last_clean":"2024-09-17T04:28:45.183080+0000","last_became_active":"2024-09-17T04:28:44.169266+0000","last_became_peered":"2024-09-17T04:28:44.169266+0000","last_unstale":"2024-09-17T04:28:45.183080+0000","last_undegraded":"2024-09-17T04:28:45.183080+0000","last_fullsized":"2024-09-17T04:28:45.183080+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:40.899996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836607+0000","last_change":"2024-09-17T04:28:44.188197+0000","last_active":"2024-09-17T04:28:45.836607+0000","last_peered":"2024-09-17T04:28:45.836607+0000","last_clean":"2024-09-17T04:28:45.836607+0000","last_became_active":"2024-09-17T04:28:44.187879+0000","last_became_peered":"2024-09-17T04:28:44.187879+0000","last_unstale":"2024-09-17T04:28:45.836607+0000","last_undegraded":"2024-09-17T04:28:45.836607+0000","last_fullsized":"2024-09-17T04:28:45.836607+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:28:18.827139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.180859+0000","last_change":"2024-09-17T04:28:44.182642+0000","last_active":"2024-09-17T04:28:45.180859+0000","last_peered":"2024-09-17T04:28:45.180859+0000","last_clean":"2024-09-17T04:28:45.180859+0000","last_became_active":"2024-09-17T04:28:44.182271+0000","last_became_peered":"2024-09-17T04:28:44.182271+0000","last_unstale":"2024-09-17T04:28:45.180859+0000","last_undegraded":"2024-09-17T04:28:45.180859+0000","last_fullsized":"2024-09-17T04:28:45.180859+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:54:37.212209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.181078+0000","last_change":"2024-09-17T04:28:44.166100+0000","last_active":"2024-09-17T04:28:45.181078+0000","last_peered":"2024-09-17T04:28:45.181078+0000","last_clean":"2024-09-17T04:28:45.181078+0000","last_became_active":"2024-09-17T04:28:44.165456+0000","last_became_peered":"2024-09-17T04:28:44.165456+0000","last_unstale":"2024-09-17T04:28:45.181078+0000","last_undegraded":"2024-09-17T04:28:45.181078+0000","last_fullsized":"2024-09-17T04:28:45.181078+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:17.830980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837079+0000","last_change":"2024-09-17T04:28:44.192060+0000","last_active":"2024-09-17T04:28:45.837079+0000","last_peered":"2024-09-17T04:28:45.837079+0000","last_clean":"2024-09-17T04:28:45.837079+0000","last_became_active":"2024-09-17T04:28:44.190066+0000","last_became_peered":"2024-09-17T04:28:44.190066+0000","last_unstale":"2024-09-17T04:28:45.837079+0000","last_undegraded":"2024-09-17T04:28:45.837079+0000","last_fullsized":"2024-09-17T04:28:45.837079+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:43:07.615773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.391225+0000","last_change":"2024-09-17T04:28:44.187180+0000","last_active":"2024-09-17T04:28:45.391225+0000","last_peered":"2024-09-17T04:28:45.391225+0000","last_clean":"2024-09-17T04:28:45.391225+0000","last_became_active":"2024-09-17T04:28:44.186726+0000","last_became_peered":"2024-09-17T04:28:44.186726+0000","last_unstale":"2024-09-17T04:28:45.391225+0000","last_undegraded":"2024-09-17T04:28:45.391225+0000","last_fullsized":"2024-09-17T04:28:45.391225+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:45:03.039793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"72'4","reported_seq":19,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.391642+0000","last_change":"2024-09-17T04:28:43.181281+0000","last_active":"2024-09-17T04:28:45.391642+0000","last_peered":"2024-09-17T04:28:45.391642+0000","last_clean":"2024-09-17T04:28:45.391642+0000","last_became_active":"2024-09-17T04:28:43.180960+0000","last_became_peered":"2024-09-17T04:28:43.180960+0000","last_unstale":"2024-09-17T04:28:45.391642+0000","last_undegraded":"2024-09-17T04:28:45.391642+0000","last_fullsized":"2024-09-17T04:28:45.391642+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:19.766145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.178220+0000","last_change":"2024-09-17T04:28:44.179667+0000","last_active":"2024-09-17T04:28:45.178220+0000","last_peered":"2024-09-17T04:28:45.178220+0000","last_clean":"2024-09-17T04:28:45.178220+0000","last_became_active":"2024-09-17T04:28:44.177165+0000","last_became_peered":"2024-09-17T04:28:44.177165+0000","last_unstale":"2024-09-17T04:28:45.178220+0000","last_undegraded":"2024-09-17T04:28:45.178220+0000","last_fullsized":"2024-09-17T04:28:45.178220+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:48.625764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"72'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.187236+0000","last_change":"2024-09-17T04:28:43.175705+0000","last_active":"2024-09-17T04:28:45.187236+0000","last_peered":"2024-09-17T04:28:45.187236+0000","last_clean":"2024-09-17T04:28:45.187236+0000","last_became_active":"2024-09-17T04:28:43.174696+0000","last_became_peered":"2024-09-17T04:28:43.174696+0000","last_unstale":"2024-09-17T04:28:45.187236+0000","last_undegraded":"2024-09-17T04:28:45.187236+0000","last_fullsized":"2024-09-17T04:28:45.187236+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:56.241725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837279+0000","last_change":"2024-09-17T04:28:44.194350+0000","last_active":"2024-09-17T04:28:45.837279+0000","last_peered":"2024-09-17T04:28:45.837279+0000","last_clean":"2024-09-17T04:28:45.837279+0000","last_became_active":"2024-09-17T04:28:44.193812+0000","last_became_peered":"2024-09-17T04:28:44.193812+0000","last_unstale":"2024-09-17T04:28:45.837279+0000","last_undegraded":"2024-09-17T04:28:45.837279+0000","last_fullsized":"2024-09-17T04:28:45.837279+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:21.789405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"72'2","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.183193+0000","last_change":"2024-09-17T04:28:43.187020+0000","last_active":"2024-09-17T04:28:45.183193+0000","last_peered":"2024-09-17T04:28:45.183193+0000","last_clean":"2024-09-17T04:28:45.183193+0000","last_became_active":"2024-09-17T04:28:43.186227+0000","last_became_peered":"2024-09-17T04:28:43.186227+0000","last_unstale":"2024-09-17T04:28:45.183193+0000","last_undegraded":"2024-09-17T04:28:45.183193+0000","last_fullsized":"2024-09-17T04:28:45.183193+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:06.525929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.391314+0000","last_change":"2024-09-17T04:28:44.185641+0000","last_active":"2024-09-17T04:28:45.391314+0000","last_peered":"2024-09-17T04:28:45.391314+0000","last_clean":"2024-09-17T04:28:45.391314+0000","last_became_active":"2024-09-17T04:28:44.184654+0000","last_became_peered":"2024-09-17T04:28:44.184654+0000","last_unstale":"2024-09-17T04:28:45.391314+0000","last_undegraded":"2024-09-17T04:28:45.391314+0000","last_fullsized":"2024-09-17T04:28:45.391314+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:56.185855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"72'2","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837731+0000","last_change":"2024-09-17T04:28:43.183969+0000","last_active":"2024-09-17T04:28:45.837731+0000","last_peered":"2024-09-17T04:28:45.837731+0000","last_clean":"2024-09-17T04:28:45.837731+0000","last_became_active":"2024-09-17T04:28:43.183603+0000","last_became_peered":"2024-09-17T04:28:43.183603+0000","last_unstale":"2024-09-17T04:28:45.837731+0000","last_undegraded":"2024-09-17T04:28:45.837731+0000","last_fullsized":"2024-09-17T04:28:45.837731+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:47.866148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.181636+0000","last_change":"2024-09-17T04:28:44.183332+0000","last_active":"2024-09-17T04:28:45.181636+0000","last_peered":"2024-09-17T04:28:45.181636+0000","last_clean":"2024-09-17T04:28:45.181636+0000","last_became_active":"2024-09-17T04:28:44.175938+0000","last_became_peered":"2024-09-17T04:28:44.175938+0000","last_unstale":"2024-09-17T04:28:45.181636+0000","last_undegraded":"2024-09-17T04:28:45.181636+0000","last_fullsized":"2024-09-17T04:28:45.181636+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:54:39.841060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"72'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836677+0000","last_change":"2024-09-17T04:28:43.171042+0000","last_active":"2024-09-17T04:28:45.836677+0000","last_peered":"2024-09-17T04:28:45.836677+0000","last_clean":"2024-09-17T04:28:45.836677+0000","last_became_active":"2024-09-17T04:28:43.170733+0000","last_became_peered":"2024-09-17T04:28:43.170733+0000","last_unstale":"2024-09-17T04:28:45.836677+0000","last_undegraded":"2024-09-17T04:28:45.836677+0000","last_fullsized":"2024-09-17T04:28:45.836677+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:33:35.104722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.182786+0000","last_change":"2024-09-17T04:28:44.181651+0000","last_active":"2024-09-17T04:28:45.182786+0000","last_peered":"2024-09-17T04:28:45.182786+0000","last_clean":"2024-09-17T04:28:45.182786+0000","last_became_active":"2024-09-17T04:28:44.181249+0000","last_became_peered":"2024-09-17T04:28:44.181249+0000","last_unstale":"2024-09-17T04:28:45.182786+0000","last_undegraded":"2024-09-17T04:28:45.182786+0000","last_fullsized":"2024-09-17T04:28:45.182786+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:32.865614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"72'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837297+0000","last_change":"2024-09-17T04:28:43.181324+0000","last_active":"2024-09-17T04:28:45.837297+0000","last_peered":"2024-09-17T04:28:45.837297+0000","last_clean":"2024-09-17T04:28:45.837297+0000","last_became_active":"2024-09-17T04:28:43.180647+0000","last_became_peered":"2024-09-17T04:28:43.180647+0000","last_unstale":"2024-09-17T04:28:45.837297+0000","last_undegraded":"2024-09-17T04:28:45.837297+0000","last_fullsized":"2024-09-17T04:28:45.837297+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:01:55.909462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.177079+0000","last_change":"2024-09-17T04:28:44.168250+0000","last_active":"2024-09-17T04:28:45.177079+0000","last_peered":"2024-09-17T04:28:45.177079+0000","last_clean":"2024-09-17T04:28:45.177079+0000","last_became_active":"2024-09-17T04:28:44.166618+0000","last_became_peered":"2024-09-17T04:28:44.166618+0000","last_unstale":"2024-09-17T04:28:45.177079+0000","last_undegraded":"2024-09-17T04:28:45.177079+0000","last_fullsized":"2024-09-17T04:28:45.177079+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:06.198935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836183+0000","last_change":"2024-09-17T04:28:43.163736+0000","last_active":"2024-09-17T04:28:45.836183+0000","last_peered":"2024-09-17T04:28:45.836183+0000","last_clean":"2024-09-17T04:28:45.836183+0000","last_became_active":"2024-09-17T04:28:43.163431+0000","last_became_peered":"2024-09-17T04:28:43.163431+0000","last_unstale":"2024-09-17T04:28:45.836183+0000","last_undegraded":"2024-09-17T04:28:45.836183+0000","last_fullsized":"2024-09-17T04:28:45.836183+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:41.686144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.186727+0000","last_change":"2024-09-17T04:28:44.182372+0000","last_active":"2024-09-17T04:28:45.186727+0000","last_peered":"2024-09-17T04:28:45.186727+0000","last_clean":"2024-09-17T04:28:45.186727+0000","last_became_active":"2024-09-17T04:28:44.181995+0000","last_became_peered":"2024-09-17T04:28:44.181995+0000","last_unstale":"2024-09-17T04:28:45.186727+0000","last_undegraded":"2024-09-17T04:28:45.186727+0000","last_fullsized":"2024-09-17T04:28:45.186727+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:21:53.031978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.177046+0000","last_change":"2024-09-17T04:28:43.163133+0000","last_active":"2024-09-17T04:28:45.177046+0000","last_peered":"2024-09-17T04:28:45.177046+0000","last_clean":"2024-09-17T04:28:45.177046+0000","last_became_active":"2024-09-17T04:28:43.162404+0000","last_became_peered":"2024-09-17T04:28:43.162404+0000","last_unstale":"2024-09-17T04:28:45.177046+0000","last_undegraded":"2024-09-17T04:28:45.177046+0000","last_fullsized":"2024-09-17T04:28:45.177046+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:47:51.349077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.182070+0000","last_change":"2024-09-17T04:28:44.175084+0000","last_active":"2024-09-17T04:28:45.182070+0000","last_peered":"2024-09-17T04:28:45.182070+0000","last_clean":"2024-09-17T04:28:45.182070+0000","last_became_active":"2024-09-17T04:28:44.174333+0000","last_became_peered":"2024-09-17T04:28:44.174333+0000","last_unstale":"2024-09-17T04:28:45.182070+0000","last_undegraded":"2024-09-17T04:28:45.182070+0000","last_fullsized":"2024-09-17T04:28:45.182070+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:13.813644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.183244+0000","last_change":"2024-09-17T04:28:43.174543+0000","last_active":"2024-09-17T04:28:45.183244+0000","last_peered":"2024-09-17T04:28:45.183244+0000","last_clean":"2024-09-17T04:28:45.183244+0000","last_became_active":"2024-09-17T04:28:43.174199+0000","last_became_peered":"2024-09-17T04:28:43.174199+0000","last_unstale":"2024-09-17T04:28:45.183244+0000","last_undegraded":"2024-09-17T04:28:45.183244+0000","last_fullsized":"2024-09-17T04:28:45.183244+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:22.807921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.391422+0000","last_change":"2024-09-17T04:28:44.187251+0000","last_active":"2024-09-17T04:28:45.391422+0000","last_peered":"2024-09-17T04:28:45.391422+0000","last_clean":"2024-09-17T04:28:45.391422+0000","last_became_active":"2024-09-17T04:28:44.186817+0000","last_became_peered":"2024-09-17T04:28:44.186817+0000","last_unstale":"2024-09-17T04:28:45.391422+0000","last_undegraded":"2024-09-17T04:28:45.391422+0000","last_fullsized":"2024-09-17T04:28:45.391422+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:54.411990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837861+0000","last_change":"2024-09-17T04:28:43.185414+0000","last_active":"2024-09-17T04:28:45.837861+0000","last_peered":"2024-09-17T04:28:45.837861+0000","last_clean":"2024-09-17T04:28:45.837861+0000","last_became_active":"2024-09-17T04:28:43.184969+0000","last_became_peered":"2024-09-17T04:28:43.184969+0000","last_unstale":"2024-09-17T04:28:45.837861+0000","last_undegraded":"2024-09-17T04:28:45.837861+0000","last_fullsized":"2024-09-17T04:28:45.837861+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:58.068125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837483+0000","last_change":"2024-09-17T04:28:44.187210+0000","last_active":"2024-09-17T04:28:45.837483+0000","last_peered":"2024-09-17T04:28:45.837483+0000","last_clean":"2024-09-17T04:28:45.837483+0000","last_became_active":"2024-09-17T04:28:44.186603+0000","last_became_peered":"2024-09-17T04:28:44.186603+0000","last_unstale":"2024-09-17T04:28:45.837483+0000","last_undegraded":"2024-09-17T04:28:45.837483+0000","last_fullsized":"2024-09-17T04:28:45.837483+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:01.266881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"72'3","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836958+0000","last_change":"2024-09-17T04:28:43.172982+0000","last_active":"2024-09-17T04:28:45.836958+0000","last_peered":"2024-09-17T04:28:45.836958+0000","last_clean":"2024-09-17T04:28:45.836958+0000","last_became_active":"2024-09-17T04:28:43.172659+0000","last_became_peered":"2024-09-17T04:28:43.172659+0000","last_unstale":"2024-09-17T04:28:45.836958+0000","last_undegraded":"2024-09-17T04:28:45.836958+0000","last_fullsized":"2024-09-17T04:28:45.836958+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:15.908868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.177700+0000","last_change":"2024-09-17T04:28:44.181186+0000","last_active":"2024-09-17T04:28:45.177700+0000","last_peered":"2024-09-17T04:28:45.177700+0000","last_clean":"2024-09-17T04:28:45.177700+0000","last_became_active":"2024-09-17T04:28:44.180853+0000","last_became_peered":"2024-09-17T04:28:44.180853+0000","last_unstale":"2024-09-17T04:28:45.177700+0000","last_undegraded":"2024-09-17T04:28:45.177700+0000","last_fullsized":"2024-09-17T04:28:45.177700+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:55:00.641278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"72'6","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837075+0000","last_change":"2024-09-17T04:28:43.168326+0000","last_active":"2024-09-17T04:28:45.837075+0000","last_peered":"2024-09-17T04:28:45.837075+0000","last_clean":"2024-09-17T04:28:45.837075+0000","last_became_active":"2024-09-17T04:28:43.167133+0000","last_became_peered":"2024-09-17T04:28:43.167133+0000","last_unstale":"2024-09-17T04:28:45.837075+0000","last_undegraded":"2024-09-17T04:28:45.837075+0000","last_fullsized":"2024-09-17T04:28:45.837075+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:45:50.645390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.390806+0000","last_change":"2024-09-17T04:28:44.179971+0000","last_active":"2024-09-17T04:28:45.390806+0000","last_peered":"2024-09-17T04:28:45.390806+0000","last_clean":"2024-09-17T04:28:45.390806+0000","last_became_active":"2024-09-17T04:28:44.179660+0000","last_became_peered":"2024-09-17T04:28:44.179660+0000","last_unstale":"2024-09-17T04:28:45.390806+0000","last_undegraded":"2024-09-17T04:28:45.390806+0000","last_fullsized":"2024-09-17T04:28:45.390806+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:24:25.189556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"72'2","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836286+0000","last_change":"2024-09-17T04:28:43.179010+0000","last_active":"2024-09-17T04:28:45.836286+0000","last_peered":"2024-09-17T04:28:45.836286+0000","last_clean":"2024-09-17T04:28:45.836286+0000","last_became_active":"2024-09-17T04:28:43.178520+0000","last_became_peered":"2024-09-17T04:28:43.178520+0000","last_unstale":"2024-09-17T04:28:45.836286+0000","last_undegraded":"2024-09-17T04:28:45.836286+0000","last_fullsized":"2024-09-17T04:28:45.836286+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:04:06.713074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.181013+0000","last_change":"2024-09-17T04:28:44.171317+0000","last_active":"2024-09-17T04:28:45.181013+0000","last_peered":"2024-09-17T04:28:45.181013+0000","last_clean":"2024-09-17T04:28:45.181013+0000","last_became_active":"2024-09-17T04:28:44.170995+0000","last_became_peered":"2024-09-17T04:28:44.170995+0000","last_unstale":"2024-09-17T04:28:45.181013+0000","last_undegraded":"2024-09-17T04:28:45.181013+0000","last_fullsized":"2024-09-17T04:28:45.181013+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:43.084390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837527+0000","last_change":"2024-09-17T04:28:43.180632+0000","last_active":"2024-09-17T04:28:45.837527+0000","last_peered":"2024-09-17T04:28:45.837527+0000","last_clean":"2024-09-17T04:28:45.837527+0000","last_became_active":"2024-09-17T04:28:43.180146+0000","last_became_peered":"2024-09-17T04:28:43.180146+0000","last_unstale":"2024-09-17T04:28:45.837527+0000","last_undegraded":"2024-09-17T04:28:45.837527+0000","last_fullsized":"2024-09-17T04:28:45.837527+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:03.917176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.186730+0000","last_change":"2024-09-17T04:28:44.195022+0000","last_active":"2024-09-17T04:28:45.186730+0000","last_peered":"2024-09-17T04:28:45.186730+0000","last_clean":"2024-09-17T04:28:45.186730+0000","last_became_active":"2024-09-17T04:28:44.194654+0000","last_became_peered":"2024-09-17T04:28:44.194654+0000","last_unstale":"2024-09-17T04:28:45.186730+0000","last_undegraded":"2024-09-17T04:28:45.186730+0000","last_fullsized":"2024-09-17T04:28:45.186730+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:27.037078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.180561+0000","last_change":"2024-09-17T04:28:43.175788+0000","last_active":"2024-09-17T04:28:45.180561+0000","last_peered":"2024-09-17T04:28:45.180561+0000","last_clean":"2024-09-17T04:28:45.180561+0000","last_became_active":"2024-09-17T04:28:43.175486+0000","last_became_peered":"2024-09-17T04:28:43.175486+0000","last_unstale":"2024-09-17T04:28:45.180561+0000","last_undegraded":"2024-09-17T04:28:45.180561+0000","last_fullsized":"2024-09-17T04:28:45.180561+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:56:30.343523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.177218+0000","last_change":"2024-09-17T04:28:44.174721+0000","last_active":"2024-09-17T04:28:45.177218+0000","last_peered":"2024-09-17T04:28:45.177218+0000","last_clean":"2024-09-17T04:28:45.177218+0000","last_became_active":"2024-09-17T04:28:44.174437+0000","last_became_peered":"2024-09-17T04:28:44.174437+0000","last_unstale":"2024-09-17T04:28:45.177218+0000","last_undegraded":"2024-09-17T04:28:45.177218+0000","last_fullsized":"2024-09-17T04:28:45.177218+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:57:13.550686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"72'8","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.181726+0000","last_change":"2024-09-17T04:28:43.175547+0000","last_active":"2024-09-17T04:28:45.181726+0000","last_peered":"2024-09-17T04:28:45.181726+0000","last_clean":"2024-09-17T04:28:45.181726+0000","last_became_active":"2024-09-17T04:28:43.175224+0000","last_became_peered":"2024-09-17T04:28:43.175224+0000","last_unstale":"2024-09-17T04:28:45.181726+0000","last_undegraded":"2024-09-17T04:28:45.181726+0000","last_fullsized":"2024-09-17T04:28:45.181726+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:09.135418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.183404+0000","last_change":"2024-09-17T04:28:44.183124+0000","last_active":"2024-09-17T04:28:45.183404+0000","last_peered":"2024-09-17T04:28:45.183404+0000","last_clean":"2024-09-17T04:28:45.183404+0000","last_became_active":"2024-09-17T04:28:44.181974+0000","last_became_peered":"2024-09-17T04:28:44.181974+0000","last_unstale":"2024-09-17T04:28:45.183404+0000","last_undegraded":"2024-09-17T04:28:45.183404+0000","last_fullsized":"2024-09-17T04:28:45.183404+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:52.267667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"72'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837665+0000","last_change":"2024-09-17T04:28:43.183827+0000","last_active":"2024-09-17T04:28:45.837665+0000","last_peered":"2024-09-17T04:28:45.837665+0000","last_clean":"2024-09-17T04:28:45.837665+0000","last_became_active":"2024-09-17T04:28:43.183492+0000","last_became_peered":"2024-09-17T04:28:43.183492+0000","last_unstale":"2024-09-17T04:28:45.837665+0000","last_undegraded":"2024-09-17T04:28:45.837665+0000","last_fullsized":"2024-09-17T04:28:45.837665+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:02.478321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.391552+0000","last_change":"2024-09-17T04:28:44.192168+0000","last_active":"2024-09-17T04:28:45.391552+0000","last_peered":"2024-09-17T04:28:45.391552+0000","last_clean":"2024-09-17T04:28:45.391552+0000","last_became_active":"2024-09-17T04:28:44.189424+0000","last_became_peered":"2024-09-17T04:28:44.189424+0000","last_unstale":"2024-09-17T04:28:45.391552+0000","last_undegraded":"2024-09-17T04:28:45.391552+0000","last_fullsized":"2024-09-17T04:28:45.391552+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:57.196554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"72'2","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.183288+0000","last_change":"2024-09-17T04:28:43.185602+0000","last_active":"2024-09-17T04:28:45.183288+0000","last_peered":"2024-09-17T04:28:45.183288+0000","last_clean":"2024-09-17T04:28:45.183288+0000","last_became_active":"2024-09-17T04:28:43.185261+0000","last_became_peered":"2024-09-17T04:28:43.185261+0000","last_unstale":"2024-09-17T04:28:45.183288+0000","last_undegraded":"2024-09-17T04:28:45.183288+0000","last_fullsized":"2024-09-17T04:28:45.183288+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:48.674587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836950+0000","last_change":"2024-09-17T04:28:44.176837+0000","last_active":"2024-09-17T04:28:45.836950+0000","last_peered":"2024-09-17T04:28:45.836950+0000","last_clean":"2024-09-17T04:28:45.836950+0000","last_became_active":"2024-09-17T04:28:44.176537+0000","last_became_peered":"2024-09-17T04:28:44.176537+0000","last_unstale":"2024-09-17T04:28:45.836950+0000","last_undegraded":"2024-09-17T04:28:45.836950+0000","last_fullsized":"2024-09-17T04:28:45.836950+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:34.503572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"72'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837383+0000","last_change":"2024-09-17T04:28:43.182926+0000","last_active":"2024-09-17T04:28:45.837383+0000","last_peered":"2024-09-17T04:28:45.837383+0000","last_clean":"2024-09-17T04:28:45.837383+0000","last_became_active":"2024-09-17T04:28:43.179719+0000","last_became_peered":"2024-09-17T04:28:43.179719+0000","last_unstale":"2024-09-17T04:28:45.837383+0000","last_undegraded":"2024-09-17T04:28:45.837383+0000","last_fullsized":"2024-09-17T04:28:45.837383+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:28.997811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.177708+0000","last_change":"2024-09-17T04:28:44.172649+0000","last_active":"2024-09-17T04:28:45.177708+0000","last_peered":"2024-09-17T04:28:45.177708+0000","last_clean":"2024-09-17T04:28:45.177708+0000","last_became_active":"2024-09-17T04:28:44.172326+0000","last_became_peered":"2024-09-17T04:28:44.172326+0000","last_unstale":"2024-09-17T04:28:45.177708+0000","last_undegraded":"2024-09-17T04:28:45.177708+0000","last_fullsized":"2024-09-17T04:28:45.177708+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:16:23.152782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.391053+0000","last_change":"2024-09-17T04:28:43.184237+0000","last_active":"2024-09-17T04:28:45.391053+0000","last_peered":"2024-09-17T04:28:45.391053+0000","last_clean":"2024-09-17T04:28:45.391053+0000","last_became_active":"2024-09-17T04:28:43.183879+0000","last_became_peered":"2024-09-17T04:28:43.183879+0000","last_unstale":"2024-09-17T04:28:45.391053+0000","last_undegraded":"2024-09-17T04:28:45.391053+0000","last_fullsized":"2024-09-17T04:28:45.391053+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:27.493169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836817+0000","last_change":"2024-09-17T04:28:44.179419+0000","last_active":"2024-09-17T04:28:45.836817+0000","last_peered":"2024-09-17T04:28:45.836817+0000","last_clean":"2024-09-17T04:28:45.836817+0000","last_became_active":"2024-09-17T04:28:44.179132+0000","last_became_peered":"2024-09-17T04:28:44.179132+0000","last_unstale":"2024-09-17T04:28:45.836817+0000","last_undegraded":"2024-09-17T04:28:45.836817+0000","last_fullsized":"2024-09-17T04:28:45.836817+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:47:46.546993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837616+0000","last_change":"2024-09-17T04:28:43.181357+0000","last_active":"2024-09-17T04:28:45.837616+0000","last_peered":"2024-09-17T04:28:45.837616+0000","last_clean":"2024-09-17T04:28:45.837616+0000","last_became_active":"2024-09-17T04:28:43.181053+0000","last_became_peered":"2024-09-17T04:28:43.181053+0000","last_unstale":"2024-09-17T04:28:45.837616+0000","last_undegraded":"2024-09-17T04:28:45.837616+0000","last_fullsized":"2024-09-17T04:28:45.837616+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:53:34.585332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836565+0000","last_change":"2024-09-17T04:28:44.188889+0000","last_active":"2024-09-17T04:28:45.836565+0000","last_peered":"2024-09-17T04:28:45.836565+0000","last_clean":"2024-09-17T04:28:45.836565+0000","last_became_active":"2024-09-17T04:28:44.188524+0000","last_became_peered":"2024-09-17T04:28:44.188524+0000","last_unstale":"2024-09-17T04:28:45.836565+0000","last_undegraded":"2024-09-17T04:28:45.836565+0000","last_fullsized":"2024-09-17T04:28:45.836565+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:17.290337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.180828+0000","last_change":"2024-09-17T04:28:43.184098+0000","last_active":"2024-09-17T04:28:45.180828+0000","last_peered":"2024-09-17T04:28:45.180828+0000","last_clean":"2024-09-17T04:28:45.180828+0000","last_became_active":"2024-09-17T04:28:43.178392+0000","last_became_peered":"2024-09-17T04:28:43.178392+0000","last_unstale":"2024-09-17T04:28:45.180828+0000","last_undegraded":"2024-09-17T04:28:45.180828+0000","last_fullsized":"2024-09-17T04:28:45.180828+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:26.238316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836874+0000","last_change":"2024-09-17T04:28:44.186208+0000","last_active":"2024-09-17T04:28:45.836874+0000","last_peered":"2024-09-17T04:28:45.836874+0000","last_clean":"2024-09-17T04:28:45.836874+0000","last_became_active":"2024-09-17T04:28:44.185834+0000","last_became_peered":"2024-09-17T04:28:44.185834+0000","last_unstale":"2024-09-17T04:28:45.836874+0000","last_undegraded":"2024-09-17T04:28:45.836874+0000","last_fullsized":"2024-09-17T04:28:45.836874+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:24:29.707796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.186626+0000","last_change":"2024-09-17T04:28:43.177447+0000","last_active":"2024-09-17T04:28:45.186626+0000","last_peered":"2024-09-17T04:28:45.186626+0000","last_clean":"2024-09-17T04:28:45.186626+0000","last_became_active":"2024-09-17T04:28:43.177133+0000","last_became_peered":"2024-09-17T04:28:43.177133+0000","last_unstale":"2024-09-17T04:28:45.186626+0000","last_undegraded":"2024-09-17T04:28:45.186626+0000","last_fullsized":"2024-09-17T04:28:45.186626+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:44.270517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.187229+0000","last_change":"2024-09-17T04:28:44.181280+0000","last_active":"2024-09-17T04:28:45.187229+0000","last_peered":"2024-09-17T04:28:45.187229+0000","last_clean":"2024-09-17T04:28:45.187229+0000","last_became_active":"2024-09-17T04:28:44.180969+0000","last_became_peered":"2024-09-17T04:28:44.180969+0000","last_unstale":"2024-09-17T04:28:45.187229+0000","last_undegraded":"2024-09-17T04:28:45.187229+0000","last_fullsized":"2024-09-17T04:28:45.187229+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:24:25.189556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"72'2","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837733+0000","last_change":"2024-09-17T04:28:43.183089+0000","last_active":"2024-09-17T04:28:45.837733+0000","last_peered":"2024-09-17T04:28:45.837733+0000","last_clean":"2024-09-17T04:28:45.837733+0000","last_became_active":"2024-09-17T04:28:43.179747+0000","last_became_peered":"2024-09-17T04:28:43.179747+0000","last_unstale":"2024-09-17T04:28:45.837733+0000","last_undegraded":"2024-09-17T04:28:45.837733+0000","last_fullsized":"2024-09-17T04:28:45.837733+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:08.291909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.181075+0000","last_change":"2024-09-17T04:28:44.179171+0000","last_active":"2024-09-17T04:28:45.181075+0000","last_peered":"2024-09-17T04:28:45.181075+0000","last_clean":"2024-09-17T04:28:45.181075+0000","last_became_active":"2024-09-17T04:28:44.178313+0000","last_became_peered":"2024-09-17T04:28:44.178313+0000","last_unstale":"2024-09-17T04:28:45.181075+0000","last_undegraded":"2024-09-17T04:28:45.181075+0000","last_fullsized":"2024-09-17T04:28:45.181075+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:21.723977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"73'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.182251+0000","last_change":"2024-09-17T04:28:43.178629+0000","last_active":"2024-09-17T04:28:45.182251+0000","last_peered":"2024-09-17T04:28:45.182251+0000","last_clean":"2024-09-17T04:28:45.182251+0000","last_became_active":"2024-09-17T04:28:43.176654+0000","last_became_peered":"2024-09-17T04:28:43.176654+0000","last_unstale":"2024-09-17T04:28:45.182251+0000","last_undegraded":"2024-09-17T04:28:45.182251+0000","last_fullsized":"2024-09-17T04:28:45.182251+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:27.493169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.391051+0000","last_change":"2024-09-17T04:28:44.187809+0000","last_active":"2024-09-17T04:28:45.391051+0000","last_peered":"2024-09-17T04:28:45.391051+0000","last_clean":"2024-09-17T04:28:45.391051+0000","last_became_active":"2024-09-17T04:28:44.187476+0000","last_became_peered":"2024-09-17T04:28:44.187476+0000","last_unstale":"2024-09-17T04:28:45.391051+0000","last_undegraded":"2024-09-17T04:28:45.391051+0000","last_fullsized":"2024-09-17T04:28:45.391051+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:53.823460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.177875+0000","last_change":"2024-09-17T04:28:43.176154+0000","last_active":"2024-09-17T04:28:45.177875+0000","last_peered":"2024-09-17T04:28:45.177875+0000","last_clean":"2024-09-17T04:28:45.177875+0000","last_became_active":"2024-09-17T04:28:43.175853+0000","last_became_peered":"2024-09-17T04:28:43.175853+0000","last_unstale":"2024-09-17T04:28:45.177875+0000","last_undegraded":"2024-09-17T04:28:45.177875+0000","last_fullsized":"2024-09-17T04:28:45.177875+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:19:36.188334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.391450+0000","last_change":"2024-09-17T04:28:44.192356+0000","last_active":"2024-09-17T04:28:45.391450+0000","last_peered":"2024-09-17T04:28:45.391450+0000","last_clean":"2024-09-17T04:28:45.391450+0000","last_became_active":"2024-09-17T04:28:44.190656+0000","last_became_peered":"2024-09-17T04:28:44.190656+0000","last_unstale":"2024-09-17T04:28:45.391450+0000","last_undegraded":"2024-09-17T04:28:45.391450+0000","last_fullsized":"2024-09-17T04:28:45.391450+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:50:34.240788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.187058+0000","last_change":"2024-09-17T04:28:43.178576+0000","last_active":"2024-09-17T04:28:45.187058+0000","last_peered":"2024-09-17T04:28:45.187058+0000","last_clean":"2024-09-17T04:28:45.187058+0000","last_became_active":"2024-09-17T04:28:43.178301+0000","last_became_peered":"2024-09-17T04:28:43.178301+0000","last_unstale":"2024-09-17T04:28:45.187058+0000","last_undegraded":"2024-09-17T04:28:45.187058+0000","last_fullsized":"2024-09-17T04:28:45.187058+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:05:46.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.177384+0000","last_change":"2024-09-17T04:28:44.178513+0000","last_active":"2024-09-17T04:28:45.177384+0000","last_peered":"2024-09-17T04:28:45.177384+0000","last_clean":"2024-09-17T04:28:45.177384+0000","last_became_active":"2024-09-17T04:28:44.173870+0000","last_became_peered":"2024-09-17T04:28:44.173870+0000","last_unstale":"2024-09-17T04:28:45.177384+0000","last_undegraded":"2024-09-17T04:28:45.177384+0000","last_fullsized":"2024-09-17T04:28:45.177384+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:20.738804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.177575+0000","last_change":"2024-09-17T04:28:43.168256+0000","last_active":"2024-09-17T04:28:45.177575+0000","last_peered":"2024-09-17T04:28:45.177575+0000","last_clean":"2024-09-17T04:28:45.177575+0000","last_became_active":"2024-09-17T04:28:43.167900+0000","last_became_peered":"2024-09-17T04:28:43.167900+0000","last_unstale":"2024-09-17T04:28:45.177575+0000","last_undegraded":"2024-09-17T04:28:45.177575+0000","last_fullsized":"2024-09-17T04:28:45.177575+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:05:25.864041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837477+0000","last_change":"2024-09-17T04:28:44.189003+0000","last_active":"2024-09-17T04:28:45.837477+0000","last_peered":"2024-09-17T04:28:45.837477+0000","last_clean":"2024-09-17T04:28:45.837477+0000","last_became_active":"2024-09-17T04:28:44.188693+0000","last_became_peered":"2024-09-17T04:28:44.188693+0000","last_unstale":"2024-09-17T04:28:45.837477+0000","last_undegraded":"2024-09-17T04:28:45.837477+0000","last_fullsized":"2024-09-17T04:28:45.837477+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:53.321976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836083+0000","last_change":"2024-09-17T04:28:43.170357+0000","last_active":"2024-09-17T04:28:45.836083+0000","last_peered":"2024-09-17T04:28:45.836083+0000","last_clean":"2024-09-17T04:28:45.836083+0000","last_became_active":"2024-09-17T04:28:43.170017+0000","last_became_peered":"2024-09-17T04:28:43.170017+0000","last_unstale":"2024-09-17T04:28:45.836083+0000","last_undegraded":"2024-09-17T04:28:45.836083+0000","last_fullsized":"2024-09-17T04:28:45.836083+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:45:31.690608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836171+0000","last_change":"2024-09-17T04:28:44.177117+0000","last_active":"2024-09-17T04:28:45.836171+0000","last_peered":"2024-09-17T04:28:45.836171+0000","last_clean":"2024-09-17T04:28:45.836171+0000","last_became_active":"2024-09-17T04:28:44.176720+0000","last_became_peered":"2024-09-17T04:28:44.176720+0000","last_unstale":"2024-09-17T04:28:45.836171+0000","last_undegraded":"2024-09-17T04:28:45.836171+0000","last_fullsized":"2024-09-17T04:28:45.836171+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:13.369221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.182125+0000","last_change":"2024-09-17T04:28:43.177322+0000","last_active":"2024-09-17T04:28:45.182125+0000","last_peered":"2024-09-17T04:28:45.182125+0000","last_clean":"2024-09-17T04:28:45.182125+0000","last_became_active":"2024-09-17T04:28:43.177004+0000","last_became_peered":"2024-09-17T04:28:43.177004+0000","last_unstale":"2024-09-17T04:28:45.182125+0000","last_undegraded":"2024-09-17T04:28:45.182125+0000","last_fullsized":"2024-09-17T04:28:45.182125+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:58.922041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836859+0000","last_change":"2024-09-17T04:28:44.192783+0000","last_active":"2024-09-17T04:28:45.836859+0000","last_peered":"2024-09-17T04:28:45.836859+0000","last_clean":"2024-09-17T04:28:45.836859+0000","last_became_active":"2024-09-17T04:28:44.191353+0000","last_became_peered":"2024-09-17T04:28:44.191353+0000","last_unstale":"2024-09-17T04:28:45.836859+0000","last_undegraded":"2024-09-17T04:28:45.836859+0000","last_fullsized":"2024-09-17T04:28:45.836859+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:02:14.822715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837279+0000","last_change":"2024-09-17T04:28:43.183805+0000","last_active":"2024-09-17T04:28:45.837279+0000","last_peered":"2024-09-17T04:28:45.837279+0000","last_clean":"2024-09-17T04:28:45.837279+0000","last_became_active":"2024-09-17T04:28:43.183414+0000","last_became_peered":"2024-09-17T04:28:43.183414+0000","last_unstale":"2024-09-17T04:28:45.837279+0000","last_undegraded":"2024-09-17T04:28:45.837279+0000","last_fullsized":"2024-09-17T04:28:45.837279+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:46:21.622130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.836616+0000","last_change":"2024-09-17T04:28:44.189995+0000","last_active":"2024-09-17T04:28:45.836616+0000","last_peered":"2024-09-17T04:28:45.836616+0000","last_clean":"2024-09-17T04:28:45.836616+0000","last_became_active":"2024-09-17T04:28:44.189715+0000","last_became_peered":"2024-09-17T04:28:44.189715+0000","last_unstale":"2024-09-17T04:28:45.836616+0000","last_undegraded":"2024-09-17T04:28:45.836616+0000","last_fullsized":"2024-09-17T04:28:45.836616+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:43.149798+0000","last_clean_scrub_stamp":"2024-09-17T04:28:43.149798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:25.483700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"72'2","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.837473+0000","last_change":"2024-09-17T04:28:43.184519+0000","last_active":"2024-09-17T04:28:45.837473+0000","last_peered":"2024-09-17T04:28:45.837473+0000","last_clean":"2024-09-17T04:28:45.837473+0000","last_became_active":"2024-09-17T04:28:43.183311+0000","last_became_peered":"2024-09-17T04:28:43.183311+0000","last_unstale":"2024-09-17T04:28:45.837473+0000","last_undegraded":"2024-09-17T04:28:45.837473+0000","last_fullsized":"2024-09-17T04:28:45.837473+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:28:42.144673+0000","last_clean_scrub_stamp":"2024-09-17T04:28:42.144673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:41.963229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":208,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T04:28:45.390756+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:28:45.390756+0000","last_peered":"2024-09-17T04:28:45.390756+0000","last_clean":"2024-09-17T04:28:45.390756+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:28:45.390756+0000","last_undegraded":"2024-09-17T04:28:45.390756+0000","last_fullsized":"2024-09-17T04:28:45.390756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476798,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1478656,"data_stored":958865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476797,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":884736,"data_stored":367973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476798,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":880640,"data_stored":367971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476798,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":880640,"data_stored":367951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509502,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32060,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339780,"statfs":{"total":96636764160,"available":96603934720,"internally_reserved":0,"allocated":888832,"data_stored":369493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509501,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":897024,"data_stored":370109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509501,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1470464,"data_stored":958249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542206,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32060,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339780,"statfs":{"total":96636764160,"available":96603934720,"internally_reserved":0,"allocated":884736,"data_stored":367993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:28:54.009 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T04:28:54.009 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T04:28:54.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.275+0000 7f28df000640 1 -- 172.21.15.100:0/2549089308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28d8155a90 msgr2=0x7f28d815f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:54.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.275+0000 7f28df000640 1 --2- 172.21.15.100:0/2549089308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28d8155a90 0x7f28d815f680 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f28c4009830 tx=0x7f28c402a500 comp rx=0 tx=0).stop 2024-09-17T04:28:54.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.275+0000 7f28df000640 1 -- 172.21.15.100:0/2549089308 shutdown_connections 2024-09-17T04:28:54.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.275+0000 7f28df000640 1 --2- 172.21.15.100:0/2549089308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28d8155a90 0x7f28d815f680 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:54.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.275+0000 7f28df000640 1 --2- 172.21.15.100:0/2549089308 >> v2:172.21.15.184:3300/0 conn(0x7f28d80cea10 0x7f28d8162560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:54.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.275+0000 7f28df000640 1 -- 172.21.15.100:0/2549089308 >> 172.21.15.100:0/2549089308 conn(0x7f28d80b1dd0 msgr2=0x7f28d80b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:54.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.275+0000 7f28df000640 1 -- 172.21.15.100:0/2549089308 shutdown_connections 2024-09-17T04:28:54.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.275+0000 7f28df000640 1 -- 172.21.15.100:0/2549089308 wait complete. 2024-09-17T04:28:54.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.277+0000 7f28df000640 1 Processor -- start 2024-09-17T04:28:54.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.277+0000 7f28df000640 1 -- start start 2024-09-17T04:28:54.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.279+0000 7f28df000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28d80cea10 0x7f28d814eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:54.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.279+0000 7f28df000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d8155a90 0x7f28d81f8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:54.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.279+0000 7f28df000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28d815fbc0 0x7f28d81f9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:54.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.279+0000 7f28df000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f28d80cf3a0 con 0x7f28d8155a90 2024-09-17T04:28:54.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.279+0000 7f28df000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f28d80cf220 con 0x7f28d815fbc0 2024-09-17T04:28:54.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.279+0000 7f28df000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f28d80cf520 con 0x7f28d80cea10 2024-09-17T04:28:54.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.279+0000 7f28dd200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d8155a90 0x7f28d81f8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:54.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.279+0000 7f28dd200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d8155a90 0x7f28d81f8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38850/0 (socket says 172.21.15.100:38850) 2024-09-17T04:28:54.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.279+0000 7f28dd200640 1 -- 172.21.15.100:0/2881112076 learned_addr learned my addr 172.21.15.100:0/2881112076 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:54.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.279+0000 7f28ddc00640 1 --2- 172.21.15.100:0/2881112076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28d80cea10 0x7f28d814eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:54.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.279+0000 7f28de600640 1 --2- 172.21.15.100:0/2881112076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28d815fbc0 0x7f28d81f9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:54.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.280+0000 7f28dd200640 1 -- 172.21.15.100:0/2881112076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28d80cea10 msgr2=0x7f28d814eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:54.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.280+0000 7f28dd200640 1 --2- 172.21.15.100:0/2881112076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28d80cea10 0x7f28d814eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:54.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.280+0000 7f28dd200640 1 -- 172.21.15.100:0/2881112076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28d815fbc0 msgr2=0x7f28d81f9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:54.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.280+0000 7f28dd200640 1 --2- 172.21.15.100:0/2881112076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28d815fbc0 0x7f28d81f9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:54.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.280+0000 7f28dd200640 1 -- 172.21.15.100:0/2881112076 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d80ae290 con 0x7f28d8155a90 2024-09-17T04:28:54.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.280+0000 7f28ddc00640 1 --2- 172.21.15.100:0/2881112076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28d80cea10 0x7f28d814eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:28:54.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.280+0000 7f28dd200640 1 --2- 172.21.15.100:0/2881112076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d8155a90 0x7f28d81f8d40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f28c4000c00 tx=0x7f28c4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:54.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.280+0000 7f28d5600640 1 -- 172.21.15.100:0/2881112076 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c4031d60 con 0x7f28d8155a90 2024-09-17T04:28:54.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.280+0000 7f28d5600640 1 -- 172.21.15.100:0/2881112076 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28c4033e10 con 0x7f28d8155a90 2024-09-17T04:28:54.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.280+0000 7f28d5600640 1 -- 172.21.15.100:0/2881112076 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c4033310 con 0x7f28d8155a90 2024-09-17T04:28:54.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.280+0000 7f28df000640 1 -- 172.21.15.100:0/2881112076 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d80ae570 con 0x7f28d8155a90 2024-09-17T04:28:54.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.280+0000 7f28df000640 1 -- 172.21.15.100:0/2881112076 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d80aeb30 con 0x7f28d8155a90 2024-09-17T04:28:54.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.281+0000 7f28df000640 1 -- 172.21.15.100:0/2881112076 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28d8162210 con 0x7f28d8155a90 2024-09-17T04:28:54.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.283+0000 7f28d5600640 1 -- 172.21.15.100:0/2881112076 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f28c40334b0 con 0x7f28d8155a90 2024-09-17T04:28:54.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.284+0000 7f28d5600640 1 --2- 172.21.15.100:0/2881112076 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f28b8078ef0 0x7f28b807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:54.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.284+0000 7f28d5600640 1 -- 172.21.15.100:0/2881112076 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5767+0+0 (secure 0 0 0) 0x7f28c40bc3a0 con 0x7f28d8155a90 2024-09-17T04:28:54.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.284+0000 7f28ddc00640 1 --2- 172.21.15.100:0/2881112076 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f28b8078ef0 0x7f28b807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:54.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.285+0000 7f28ddc00640 1 --2- 172.21.15.100:0/2881112076 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f28b8078ef0 0x7f28b807b400 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f28cc0095f0 tx=0x7f28cc005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:54.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.288+0000 7f28d5600640 1 -- 172.21.15.100:0/2881112076 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f28d8162210 con 0x7f28d8155a90 2024-09-17T04:28:54.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.545+0000 7f28df000640 1 -- 172.21.15.100:0/2881112076 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f28d80ae020 con 0x7f28d8155a90 2024-09-17T04:28:54.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.548+0000 7f28d5600640 1 -- 172.21.15.100:0/2881112076 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7f28c404a030 con 0x7f28d8155a90 2024-09-17T04:28:54.548 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-09-17T04:28:54.549 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAsBOlmfYg9IBAABP1uRKGOugrA47VxwRLkjA== 2024-09-17T04:28:54.549 INFO:teuthology.orchestra.run.smithi100.stdout: caps mds = "allow" 2024-09-17T04:28:54.549 INFO:teuthology.orchestra.run.smithi100.stdout: caps mon = "allow r" 2024-09-17T04:28:54.549 INFO:teuthology.orchestra.run.smithi100.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T04:28:54.549 INFO:teuthology.orchestra.run.smithi100.stderr:updated caps for client.0 2024-09-17T04:28:54.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.550+0000 7f28df000640 1 -- 172.21.15.100:0/2881112076 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f28b8078ef0 msgr2=0x7f28b807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:54.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.550+0000 7f28df000640 1 --2- 172.21.15.100:0/2881112076 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f28b8078ef0 0x7f28b807b400 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f28cc0095f0 tx=0x7f28cc005f90 comp rx=0 tx=0).stop 2024-09-17T04:28:54.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.550+0000 7f28df000640 1 -- 172.21.15.100:0/2881112076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d8155a90 msgr2=0x7f28d81f8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:54.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.551+0000 7f28df000640 1 --2- 172.21.15.100:0/2881112076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d8155a90 0x7f28d81f8d40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f28c4000c00 tx=0x7f28c4033c20 comp rx=0 tx=0).stop 2024-09-17T04:28:54.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.551+0000 7f28df000640 1 -- 172.21.15.100:0/2881112076 shutdown_connections 2024-09-17T04:28:54.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.551+0000 7f28df000640 1 --2- 172.21.15.100:0/2881112076 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f28b8078ef0 0x7f28b807b400 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:54.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.551+0000 7f28df000640 1 --2- 172.21.15.100:0/2881112076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28d815fbc0 0x7f28d81f9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:54.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.551+0000 7f28df000640 1 --2- 172.21.15.100:0/2881112076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d8155a90 0x7f28d81f8d40 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:54.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.551+0000 7f28df000640 1 --2- 172.21.15.100:0/2881112076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28d80cea10 0x7f28d814eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:54.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.551+0000 7f28df000640 1 -- 172.21.15.100:0/2881112076 >> 172.21.15.100:0/2881112076 conn(0x7f28d80b1dd0 msgr2=0x7f28d806ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:54.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.551+0000 7f28df000640 1 -- 172.21.15.100:0/2881112076 shutdown_connections 2024-09-17T04:28:54.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.551+0000 7f28df000640 1 -- 172.21.15.100:0/2881112076 wait complete. 2024-09-17T04:28:54.574 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:28:54.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.794+0000 7f7a80000640 1 -- 172.21.15.100:0/412828033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a60003a30 msgr2=0x7f7a60003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:54.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.794+0000 7f7a80000640 1 --2- 172.21.15.100:0/412828033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a60003a30 0x7f7a60003ea0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00c310 tx=0x7f7a60005830 comp rx=0 tx=0).stop 2024-09-17T04:28:54.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.794+0000 7f7a7e000640 1 -- 172.21.15.100:0/412828033 reap_dead start 2024-09-17T04:28:54.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.794+0000 7f7a80000640 1 -- 172.21.15.100:0/412828033 shutdown_connections 2024-09-17T04:28:54.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.794+0000 7f7a80000640 1 -- 172.21.15.100:0/412828033 >> 172.21.15.100:0/412828033 conn(0x7f7a7806d730 msgr2=0x7f7a780b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:54.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.794+0000 7f7a80000640 1 -- 172.21.15.100:0/412828033 shutdown_connections 2024-09-17T04:28:54.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.794+0000 7f7a80000640 1 -- 172.21.15.100:0/412828033 wait complete. 2024-09-17T04:28:54.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.797+0000 7f7a80000640 1 Processor -- start 2024-09-17T04:28:54.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.797+0000 7f7a80000640 1 -- start start 2024-09-17T04:28:54.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.798+0000 7f7a80000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a60003a30 0x7f7a7814e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:54.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.798+0000 7f7a80000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a7814fdc0 0x7f7a7814b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:54.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.798+0000 7f7a80000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a781eac30 0x7f7a781eb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:54.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.798+0000 7f7a80000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7a780712b0 con 0x7f7a7814fdc0 2024-09-17T04:28:54.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.798+0000 7f7a80000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7a78071130 con 0x7f7a781eac30 2024-09-17T04:28:54.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.798+0000 7f7a80000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7a78163af0 con 0x7f7a60003a30 2024-09-17T04:28:54.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.799+0000 7f7a7cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a7814fdc0 0x7f7a7814b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:54.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.799+0000 7f7a7cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a7814fdc0 0x7f7a7814b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34660/0 (socket says 172.21.15.100:34660) 2024-09-17T04:28:54.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.799+0000 7f7a7cc00640 1 -- 172.21.15.100:0/756934048 learned_addr learned my addr 172.21.15.100:0/756934048 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:54.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.799+0000 7f7a7d600640 1 --2- 172.21.15.100:0/756934048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a60003a30 0x7f7a7814e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:54.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.799+0000 7f7a7e000640 1 --2- 172.21.15.100:0/756934048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a781eac30 0x7f7a781eb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:54.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.799+0000 7f7a7cc00640 1 -- 172.21.15.100:0/756934048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a60003a30 msgr2=0x7f7a7814e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:54.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.799+0000 7f7a7cc00640 1 --2- 172.21.15.100:0/756934048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a60003a30 0x7f7a7814e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:54.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.799+0000 7f7a7cc00640 1 -- 172.21.15.100:0/756934048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a781eac30 msgr2=0x7f7a781eb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:54.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.799+0000 7f7a7cc00640 1 --2- 172.21.15.100:0/756934048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a781eac30 0x7f7a781eb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:54.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.799+0000 7f7a7cc00640 1 -- 172.21.15.100:0/756934048 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a781eb580 con 0x7f7a7814fdc0 2024-09-17T04:28:54.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.799+0000 7f7a7cc00640 1 --2- 172.21.15.100:0/756934048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a7814fdc0 0x7f7a7814b740 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f7a64004e50 tx=0x7f7a64001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:54.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.800+0000 7f7a75600640 1 -- 172.21.15.100:0/756934048 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a64011660 con 0x7f7a7814fdc0 2024-09-17T04:28:54.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.800+0000 7f7a75600640 1 -- 172.21.15.100:0/756934048 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a64016070 con 0x7f7a7814fdc0 2024-09-17T04:28:54.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.800+0000 7f7a75600640 1 -- 172.21.15.100:0/756934048 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a64011af0 con 0x7f7a7814fdc0 2024-09-17T04:28:54.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.800+0000 7f7a80000640 1 -- 172.21.15.100:0/756934048 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a7815b2d0 con 0x7f7a7814fdc0 2024-09-17T04:28:54.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.800+0000 7f7a80000640 1 -- 172.21.15.100:0/756934048 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a7815b890 con 0x7f7a7814fdc0 2024-09-17T04:28:54.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.801+0000 7f7a80000640 1 -- 172.21.15.100:0/756934048 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a780ad7d0 con 0x7f7a7814fdc0 2024-09-17T04:28:54.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.802+0000 7f7a75600640 1 -- 172.21.15.100:0/756934048 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f7a64011c90 con 0x7f7a7814fdc0 2024-09-17T04:28:54.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.803+0000 7f7a75600640 1 --2- 172.21.15.100:0/756934048 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7a58078ef0 0x7f7a5807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:54.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.803+0000 7f7a75600640 1 -- 172.21.15.100:0/756934048 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5767+0+0 (secure 0 0 0) 0x7f7a6409ba20 con 0x7f7a7814fdc0 2024-09-17T04:28:54.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.804+0000 7f7a7d600640 1 --2- 172.21.15.100:0/756934048 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7a58078ef0 0x7f7a5807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:54.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.804+0000 7f7a7d600640 1 --2- 172.21.15.100:0/756934048 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7a58078ef0 0x7f7a5807b400 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c033000 tx=0x7f7a6c038040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:54.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:54.808+0000 7f7a75600640 1 -- 172.21.15.100:0/756934048 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f7a781eb580 con 0x7f7a7814fdc0 2024-09-17T04:28:55.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.025+0000 7f7a80000640 1 -- 172.21.15.100:0/756934048 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a7815e420 con 0x7f7a7814fdc0 2024-09-17T04:28:55.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.026+0000 7f7a75600640 1 -- 172.21.15.100:0/756934048 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5237 (secure 0 0 0) 0x7f7a64063900 con 0x7f7a7814fdc0 2024-09-17T04:28:55.027 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:28:55.027 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":47,"btime":"2024-09-17T04:28:47:182892+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5513,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2569013403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2569013403},{"type":"v1","addr":"172.21.15.184:6835","nonce":2569013403}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5519,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3475403612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3475403612},{"type":"v1","addr":"172.21.15.184:6837","nonce":3475403612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6156,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2177833369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2177833369},{"type":"v1","addr":"172.21.15.100:6833","nonce":2177833369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:28:43.648961+0000","modified":"2024-09-17T04:28:46.665573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6162":{"gid":6162,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6835/2913442826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2913442826},{"type":"v1","addr":"172.21.15.100:6835","nonce":2913442826}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6162,"qdb_cluster":[6162]},"id":5}]} 2024-09-17T04:28:55.027 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 47 2024-09-17T04:28:55.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.028+0000 7f7a80000640 1 -- 172.21.15.100:0/756934048 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7a58078ef0 msgr2=0x7f7a5807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:55.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.028+0000 7f7a80000640 1 --2- 172.21.15.100:0/756934048 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7a58078ef0 0x7f7a5807b400 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c033000 tx=0x7f7a6c038040 comp rx=0 tx=0).stop 2024-09-17T04:28:55.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.028+0000 7f7a80000640 1 -- 172.21.15.100:0/756934048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a7814fdc0 msgr2=0x7f7a7814b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:55.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.028+0000 7f7a80000640 1 --2- 172.21.15.100:0/756934048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a7814fdc0 0x7f7a7814b740 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f7a64004e50 tx=0x7f7a64001630 comp rx=0 tx=0).stop 2024-09-17T04:28:55.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.028+0000 7f7a80000640 1 -- 172.21.15.100:0/756934048 shutdown_connections 2024-09-17T04:28:55.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.028+0000 7f7a80000640 1 --2- 172.21.15.100:0/756934048 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f7a58078ef0 0x7f7a5807b400 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.028+0000 7f7a80000640 1 --2- 172.21.15.100:0/756934048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a781eac30 0x7f7a781eb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.028+0000 7f7a80000640 1 --2- 172.21.15.100:0/756934048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a7814fdc0 0x7f7a7814b740 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.028+0000 7f7a80000640 1 --2- 172.21.15.100:0/756934048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a60003a30 0x7f7a7814e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.028+0000 7f7a80000640 1 -- 172.21.15.100:0/756934048 >> 172.21.15.100:0/756934048 conn(0x7f7a7806d730 msgr2=0x7f7a780af5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:55.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.028+0000 7f7a80000640 1 -- 172.21.15.100:0/756934048 shutdown_connections 2024-09-17T04:28:55.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.028+0000 7f7a80000640 1 -- 172.21.15.100:0/756934048 wait complete. 2024-09-17T04:28:55.456 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T04:28:43.648961+0000', 'modified': '2024-09-17T04:28:46.665573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6162': {'gid': 6162, 'name': 'c', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.100:6835/2913442826', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6834', 'nonce': 2913442826}, {'type': 'v1', 'addr': '172.21.15.100:6835', 'nonce': 2913442826}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6162, 'qdb_cluster': [6162]} 2024-09-17T04:28:55.456 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T04:28:55.456 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T04:28:55.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.677+0000 7ff745000640 1 -- 172.21.15.100:0/521172789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7400af050 msgr2=0x7ff7400ae080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:55.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.677+0000 7ff745000640 1 --2- 172.21.15.100:0/521172789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7400af050 0x7ff7400ae080 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff730005fd0 tx=0x7ff73002a5d0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.677+0000 7ff745000640 1 -- 172.21.15.100:0/521172789 shutdown_connections 2024-09-17T04:28:55.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.677+0000 7ff745000640 1 --2- 172.21.15.100:0/521172789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff740072e60 0x7ff740073270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.677+0000 7ff745000640 1 --2- 172.21.15.100:0/521172789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7400af050 0x7ff7400ae080 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.677+0000 7ff745000640 1 -- 172.21.15.100:0/521172789 >> 172.21.15.100:0/521172789 conn(0x7ff7400b1e40 msgr2=0x7ff7400b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:55.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.677+0000 7ff745000640 1 -- 172.21.15.100:0/521172789 shutdown_connections 2024-09-17T04:28:55.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.677+0000 7ff745000640 1 -- 172.21.15.100:0/521172789 wait complete. 2024-09-17T04:28:55.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.679+0000 7ff745000640 1 Processor -- start 2024-09-17T04:28:55.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.680+0000 7ff745000640 1 -- start start 2024-09-17T04:28:55.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.681+0000 7ff745000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff740070000 0x7ff740143660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:55.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.681+0000 7ff745000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff740072e60 0x7ff7401f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:55.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.681+0000 7ff745000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7400af050 0x7ff74014b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:55.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.681+0000 7ff745000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff740165870 con 0x7ff740070000 2024-09-17T04:28:55.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.681+0000 7ff745000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff7401656f0 con 0x7ff7400af050 2024-09-17T04:28:55.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.681+0000 7ff745000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff7401659f0 con 0x7ff740072e60 2024-09-17T04:28:55.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.681+0000 7ff73f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff740070000 0x7ff740143660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:55.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.681+0000 7ff73f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff740070000 0x7ff740143660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34680/0 (socket says 172.21.15.100:34680) 2024-09-17T04:28:55.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.681+0000 7ff73f400640 1 -- 172.21.15.100:0/3522771745 learned_addr learned my addr 172.21.15.100:0/3522771745 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:55.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.682+0000 7ff736a00640 1 --2- 172.21.15.100:0/3522771745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff740072e60 0x7ff7401f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:55.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.682+0000 7ff73fe00640 1 --2- 172.21.15.100:0/3522771745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7400af050 0x7ff74014b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:55.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.682+0000 7ff73f400640 1 -- 172.21.15.100:0/3522771745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff740072e60 msgr2=0x7ff7401f77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:55.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.682+0000 7ff73f400640 1 --2- 172.21.15.100:0/3522771745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff740072e60 0x7ff7401f77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.682+0000 7ff73f400640 1 -- 172.21.15.100:0/3522771745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7400af050 msgr2=0x7ff74014b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:55.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.682+0000 7ff73f400640 1 --2- 172.21.15.100:0/3522771745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7400af050 0x7ff74014b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.682+0000 7ff73f400640 1 -- 172.21.15.100:0/3522771745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff740153340 con 0x7ff740070000 2024-09-17T04:28:55.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.682+0000 7ff736a00640 1 --2- 172.21.15.100:0/3522771745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff740072e60 0x7ff7401f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:28:55.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.682+0000 7ff73f400640 1 --2- 172.21.15.100:0/3522771745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff740070000 0x7ff740143660 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff730034040 tx=0x7ff7300038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:55.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.682+0000 7ff737e00640 1 -- 172.21.15.100:0/3522771745 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73003e030 con 0x7ff740070000 2024-09-17T04:28:55.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.682+0000 7ff737e00640 1 -- 172.21.15.100:0/3522771745 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff730003ca0 con 0x7ff740070000 2024-09-17T04:28:55.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.682+0000 7ff745000640 1 -- 172.21.15.100:0/3522771745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff740153620 con 0x7ff740070000 2024-09-17T04:28:55.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.682+0000 7ff737e00640 1 -- 172.21.15.100:0/3522771745 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff730035040 con 0x7ff740070000 2024-09-17T04:28:55.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.683+0000 7ff745000640 1 -- 172.21.15.100:0/3522771745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff740153be0 con 0x7ff740070000 2024-09-17T04:28:55.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.686+0000 7ff737e00640 1 -- 172.21.15.100:0/3522771745 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7ff730041030 con 0x7ff740070000 2024-09-17T04:28:55.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.687+0000 7ff737e00640 1 --2- 172.21.15.100:0/3522771745 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff71c078ef0 0x7ff71c07b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:55.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.687+0000 7ff737e00640 1 -- 172.21.15.100:0/3522771745 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5767+0+0 (secure 0 0 0) 0x7ff7300bba50 con 0x7ff740070000 2024-09-17T04:28:55.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.687+0000 7ff74d7eb740 1 Processor -- start 2024-09-17T04:28:55.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.687+0000 7ff74d7eb740 1 -- start start 2024-09-17T04:28:55.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.688+0000 7ff736a00640 1 --2- 172.21.15.100:0/3522771745 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff71c078ef0 0x7ff71c07b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:55.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.688+0000 7ff736a00640 1 --2- 172.21.15.100:0/3522771745 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff71c078ef0 0x7ff71c07b400 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff7401561b0 tx=0x7ff728005f10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:55.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.689+0000 7ff74d7eb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56468cdec270 0x56468d1bcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:55.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.689+0000 7ff74d7eb740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56468cdc33b0 0x56468d1bd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:55.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.689+0000 7ff74d7eb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56468cdcb590 0x56468d18d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:55.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.689+0000 7ff74d7eb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56468d116430 con 0x56468cdcb590 2024-09-17T04:28:55.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.689+0000 7ff74d7eb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56468d1065d0 con 0x56468cdec270 2024-09-17T04:28:55.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.689+0000 7ff74d7eb740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56468d0f5a00 con 0x56468cdc33b0 2024-09-17T04:28:55.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.689+0000 7ff73fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56468cdcb590 0x56468d18d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:55.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.689+0000 7ff73fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56468cdcb590 0x56468d18d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34688/0 (socket says 172.21.15.100:34688) 2024-09-17T04:28:55.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.689+0000 7ff73fe00640 1 -- 172.21.15.100:0/2329745229 learned_addr learned my addr 172.21.15.100:0/2329745229 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:55.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.689+0000 7ff736a00640 1 --2- 172.21.15.100:0/2329745229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56468cdc33b0 0x56468d1bd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:55.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.689+0000 7ff73f400640 1 --2- 172.21.15.100:0/2329745229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56468cdec270 0x56468d1bcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:55.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.690+0000 7ff73fe00640 1 -- 172.21.15.100:0/2329745229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56468cdc33b0 msgr2=0x56468d1bd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:28:55.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.690+0000 7ff73fe00640 1 --2- 172.21.15.100:0/2329745229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56468cdc33b0 0x56468d1bd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.690+0000 7ff73fe00640 1 -- 172.21.15.100:0/2329745229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56468cdec270 msgr2=0x56468d1bcb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:28:55.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.690+0000 7ff73fe00640 1 --2- 172.21.15.100:0/2329745229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56468cdec270 0x56468d1bcb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.690+0000 7ff73fe00640 1 -- 172.21.15.100:0/2329745229 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56468d0be520 con 0x56468cdcb590 2024-09-17T04:28:55.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.690+0000 7ff736a00640 1 --2- 172.21.15.100:0/2329745229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56468cdc33b0 0x56468d1bd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:28:55.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.690+0000 7ff73f400640 1 --2- 172.21.15.100:0/2329745229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56468cdec270 0x56468d1bcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:28:55.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.690+0000 7ff73fe00640 1 --2- 172.21.15.100:0/2329745229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56468cdcb590 0x56468d18d150 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff738001b90 tx=0x7ff738002050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e100a9ce613d60f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:55.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.690+0000 7ff72fe00640 1 -- 172.21.15.100:0/2329745229 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff738019070 con 0x56468cdcb590 2024-09-17T04:28:55.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.690+0000 7ff72fe00640 1 -- 172.21.15.100:0/2329745229 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff738003c70 con 0x56468cdcb590 2024-09-17T04:28:55.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.690+0000 7ff72fe00640 1 -- 172.21.15.100:0/2329745229 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73800ea60 con 0x56468cdcb590 2024-09-17T04:28:55.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.691+0000 7ff74d7eb740 1 -- 172.21.15.100:0/2329745229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56468cdcb590 msgr2=0x56468d18d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:55.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.691+0000 7ff74d7eb740 1 --2- 172.21.15.100:0/2329745229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56468cdcb590 0x56468d18d150 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff738001b90 tx=0x7ff738002050 comp rx=0 tx=0).stop 2024-09-17T04:28:55.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.691+0000 7ff74d7eb740 1 -- 172.21.15.100:0/2329745229 shutdown_connections 2024-09-17T04:28:55.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.691+0000 7ff74d7eb740 1 --2- 172.21.15.100:0/2329745229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56468cdcb590 0x56468d18d150 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.691+0000 7ff74d7eb740 1 --2- 172.21.15.100:0/2329745229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56468cdc33b0 0x56468d1bd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.691+0000 7ff74d7eb740 1 --2- 172.21.15.100:0/2329745229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56468cdec270 0x56468d1bcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.691+0000 7ff74d7eb740 1 -- 172.21.15.100:0/2329745229 >> 172.21.15.100:0/2329745229 conn(0x56468cde03c0 msgr2=0x56468d192b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:55.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.691+0000 7ff74d7eb740 1 -- 172.21.15.100:0/2329745229 shutdown_connections 2024-09-17T04:28:55.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.691+0000 7ff74d7eb740 1 -- 172.21.15.100:0/2329745229 wait complete. 2024-09-17T04:28:55.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.692+0000 7ff74d7eb740 1 -- start start 2024-09-17T04:28:55.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.692+0000 7ff74d7eb740 1 Processor -- start 2024-09-17T04:28:55.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.694+0000 7ff74d7eb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56468cdc33b0 0x56468d1e7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:55.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.694+0000 7ff74d7eb740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56468cdcb590 0x56468d1a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:55.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.694+0000 7ff74d7eb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56468cdec270 0x56468d1d9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:55.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.694+0000 7ff74d7eb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56468d09c330 con 0x56468cdc33b0 2024-09-17T04:28:55.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.694+0000 7ff74d7eb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56468d0f9810 con 0x56468cdec270 2024-09-17T04:28:55.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.694+0000 7ff74d7eb740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56468ce66620 con 0x56468cdcb590 2024-09-17T04:28:55.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.694+0000 7ff74d7eb740 10 client.0 _ll_register_callbacks cb 0x56468d0a1420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:28:55.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.694+0000 7ff73f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56468cdc33b0 0x56468d1e7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:55.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.694+0000 7ff73f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56468cdc33b0 0x56468d1e7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34690/0 (socket says 172.21.15.100:34690) 2024-09-17T04:28:55.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.694+0000 7ff73f400640 1 -- 172.21.15.100:0/3892125057 learned_addr learned my addr 172.21.15.100:0/3892125057 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:55.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.694+0000 7ff736a00640 1 --2- 172.21.15.100:0/3892125057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56468cdcb590 0x56468d1a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:55.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.694+0000 7ff73fe00640 1 --2- 172.21.15.100:0/3892125057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56468cdec270 0x56468d1d9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:55.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.694+0000 7ff73f400640 1 -- 172.21.15.100:0/3892125057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56468cdcb590 msgr2=0x56468d1a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:28:55.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.694+0000 7ff73f400640 1 --2- 172.21.15.100:0/3892125057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56468cdcb590 0x56468d1a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.695+0000 7ff73f400640 1 -- 172.21.15.100:0/3892125057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56468cdec270 msgr2=0x56468d1d9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:28:55.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.695+0000 7ff73f400640 1 --2- 172.21.15.100:0/3892125057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56468cdec270 0x56468d1d9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.695+0000 7ff73f400640 1 -- 172.21.15.100:0/3892125057 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56468ce57850 con 0x56468cdc33b0 2024-09-17T04:28:55.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.695+0000 7ff73f400640 1 --2- 172.21.15.100:0/3892125057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56468cdc33b0 0x56468d1e7220 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff730083100 tx=0x7ff730083790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b0af7c3ae2e988 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:55.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.695+0000 7ff72cc00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-17T04:28:55.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.695+0000 7ff72cc00640 1 -- 172.21.15.100:0/3892125057 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7300a9070 con 0x56468cdc33b0 2024-09-17T04:28:55.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.695+0000 7ff72cc00640 1 -- 172.21.15.100:0/3892125057 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 5767+0+0 (secure 0 0 0) 0x7ff73007eac0 con 0x56468cdc33b0 2024-09-17T04:28:55.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.695+0000 7ff74d7eb740 1 -- 172.21.15.100:0/3892125057 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff738003c70 con 0x56468cdc33b0 2024-09-17T04:28:55.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.695+0000 7ff72cc00640 1 -- 172.21.15.100:0/3892125057 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff730048040 con 0x56468cdc33b0 2024-09-17T04:28:55.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.695+0000 7ff74d7eb740 10 client.6318 fetch_fsmap learned FSMap version 47 2024-09-17T04:28:55.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.695+0000 7ff74d7eb740 1 -- 172.21.15.100:0/3892125057 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) -- 0x56468cece6b0 con 0x56468cdc33b0 2024-09-17T04:28:55.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.695+0000 7ff72cc00640 1 -- 172.21.15.100:0/3892125057 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff738003c70 con 0x56468cdc33b0 2024-09-17T04:28:55.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.695+0000 7ff72cc00640 1 -- 172.21.15.100:0/3892125057 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 47) ==== 3502+0+0 (secure 0 0 0) 0x7ff73007ecc0 con 0x56468cdc33b0 2024-09-17T04:28:55.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.696+0000 7ff74d7eb740 10 client.6318 fetch_fsmap finished waiting for FSMap version 47 2024-09-17T04:28:55.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.696+0000 7ff74d7eb740 10 client.6318 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T04:28:55.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.696+0000 7ff74d7eb740 20 client.6318 populate_metadata read hostname 'smithi100' 2024-09-17T04:28:55.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.696+0000 7ff74d7eb740 1 --2- 172.21.15.100:0/3892125057 >> [v2:172.21.15.100:6834/2913442826,v1:172.21.15.100:6835/2913442826] conn(0x56468cdaae40 0x56468d1b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:55.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.696+0000 7ff74d7eb740 4 client.6318 mds_command: new command op to 6162 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T04:28:55.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.696+0000 7ff74d7eb740 1 -- 172.21.15.100:0/3892125057 --> [v2:172.21.15.100:6834/2913442826,v1:172.21.15.100:6835/2913442826] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56468d108520 con 0x56468cdaae40 2024-09-17T04:28:55.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.696+0000 7ff736a00640 1 --2- 172.21.15.100:0/3892125057 >> [v2:172.21.15.100:6834/2913442826,v1:172.21.15.100:6835/2913442826] conn(0x56468cdaae40 0x56468d1b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:55.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.696+0000 7ff72cc00640 10 client.6318 ms_handle_connect on v2:172.21.15.100:6834/2913442826 2024-09-17T04:28:55.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.696+0000 7ff736a00640 1 --2- 172.21.15.100:0/3892125057 >> [v2:172.21.15.100:6834/2913442826,v1:172.21.15.100:6835/2913442826] conn(0x56468cdaae40 0x56468d1b5d50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2855015ba844e39e server_cookie=56a6308296de2cb3 in_seq=0 out_seq=0 2024-09-17T04:28:55.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.697+0000 7ff72cc00640 1 -- 172.21.15.100:0/3892125057 <== mds.0 v2:172.21.15.100:6834/2913442826 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56468d108520 con 0x56468cdaae40 2024-09-17T04:28:55.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.697+0000 7ff72cc00640 10 client.6318 handle_command_reply: tid=0 2024-09-17T04:28:55.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.697+0000 7ff74d7eb740 1 client.6318 shutdown 2024-09-17T04:28:55.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.698+0000 7ff74d7eb740 1 -- 172.21.15.100:0/3892125057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56468cdc33b0 msgr2=0x56468d1e7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:55.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.698+0000 7ff74d7eb740 1 --2- 172.21.15.100:0/3892125057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56468cdc33b0 0x56468d1e7220 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff730083100 tx=0x7ff730083790 comp rx=0 tx=0).stop 2024-09-17T04:28:55.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.698+0000 7ff74d7eb740 1 -- 172.21.15.100:0/3892125057 shutdown_connections 2024-09-17T04:28:55.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.698+0000 7ff74d7eb740 1 --2- 172.21.15.100:0/3892125057 >> [v2:172.21.15.100:6834/2913442826,v1:172.21.15.100:6835/2913442826] conn(0x56468cdaae40 0x56468d1b5d50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.698+0000 7ff74d7eb740 1 --2- 172.21.15.100:0/3892125057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56468cdec270 0x56468d1d9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.698+0000 7ff74d7eb740 1 --2- 172.21.15.100:0/3892125057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56468cdcb590 0x56468d1a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.698+0000 7ff74d7eb740 1 --2- 172.21.15.100:0/3892125057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56468cdc33b0 0x56468d1e7220 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.698+0000 7ff72cc00640 0 client.6318 ms_handle_reset on v2:172.21.15.100:6834/2913442826 2024-09-17T04:28:55.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.698+0000 7ff74d7eb740 1 -- 172.21.15.100:0/3892125057 >> 172.21.15.100:0/3892125057 conn(0x56468cde03c0 msgr2=0x56468d1d81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:55.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.698+0000 7ff74d7eb740 1 -- 172.21.15.100:0/3892125057 shutdown_connections 2024-09-17T04:28:55.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.698+0000 7ff74d7eb740 1 -- 172.21.15.100:0/3892125057 wait complete. 2024-09-17T04:28:55.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.698+0000 7ff74d7eb740 20 client.6318 trim_cache size 0 max 16384 2024-09-17T04:28:55.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.717+0000 7ff74d7eb740 1 Processor -- start 2024-09-17T04:28:55.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.717+0000 7ff74d7eb740 1 -- start start 2024-09-17T04:28:55.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.718+0000 7ff74d7eb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56468cdc33b0 0x56468d1dad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:55.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.718+0000 7ff74d7eb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56468cdcb590 0x56468d1fa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:55.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.718+0000 7ff74d7eb740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56468cdec270 0x56468d1fab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:55.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.718+0000 7ff74d7eb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff7401ff1f0 con 0x56468cdc33b0 2024-09-17T04:28:55.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.718+0000 7ff74d7eb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff740144cd0 con 0x56468cdcb590 2024-09-17T04:28:55.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.718+0000 7ff74d7eb740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff740145320 con 0x56468cdec270 2024-09-17T04:28:55.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.719+0000 7ff73f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56468cdc33b0 0x56468d1dad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:55.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.719+0000 7ff73f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56468cdc33b0 0x56468d1dad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34704/0 (socket says 172.21.15.100:34704) 2024-09-17T04:28:55.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.719+0000 7ff73f400640 1 -- 172.21.15.100:0/1348967434 learned_addr learned my addr 172.21.15.100:0/1348967434 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:55.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.719+0000 7ff736a00640 1 --2- 172.21.15.100:0/1348967434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56468cdcb590 0x56468d1fa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:55.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.719+0000 7ff73fe00640 1 --2- 172.21.15.100:0/1348967434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56468cdec270 0x56468d1fab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:55.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.719+0000 7ff73f400640 1 -- 172.21.15.100:0/1348967434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56468cdec270 msgr2=0x56468d1fab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:28:55.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.719+0000 7ff73f400640 1 --2- 172.21.15.100:0/1348967434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56468cdec270 0x56468d1fab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.719+0000 7ff73f400640 1 -- 172.21.15.100:0/1348967434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56468cdcb590 msgr2=0x56468d1fa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:28:55.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.719+0000 7ff73f400640 1 --2- 172.21.15.100:0/1348967434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56468cdcb590 0x56468d1fa640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.719+0000 7ff73f400640 1 -- 172.21.15.100:0/1348967434 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56468d193840 con 0x56468cdc33b0 2024-09-17T04:28:55.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.719+0000 7ff73f400640 1 --2- 172.21.15.100:0/1348967434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56468cdc33b0 0x56468d1dad10 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff7300099c0 tx=0x7ff730082830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90613c00fdd23aee server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:55.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.719+0000 7ff72f400640 1 -- 172.21.15.100:0/1348967434 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7300a9050 con 0x56468cdc33b0 2024-09-17T04:28:55.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.720+0000 7ff72f400640 1 -- 172.21.15.100:0/1348967434 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff730083870 con 0x56468cdc33b0 2024-09-17T04:28:55.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.720+0000 7ff72f400640 1 -- 172.21.15.100:0/1348967434 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73007ed00 con 0x56468cdc33b0 2024-09-17T04:28:55.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.720+0000 7ff74d7eb740 1 -- 172.21.15.100:0/1348967434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56468cdc33b0 msgr2=0x56468d1dad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:55.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.720+0000 7ff74d7eb740 1 --2- 172.21.15.100:0/1348967434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56468cdc33b0 0x56468d1dad10 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff7300099c0 tx=0x7ff730082830 comp rx=0 tx=0).stop 2024-09-17T04:28:55.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.720+0000 7ff74d7eb740 1 -- 172.21.15.100:0/1348967434 shutdown_connections 2024-09-17T04:28:55.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.720+0000 7ff74d7eb740 1 --2- 172.21.15.100:0/1348967434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56468cdec270 0x56468d1fab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.720+0000 7ff74d7eb740 1 --2- 172.21.15.100:0/1348967434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56468cdcb590 0x56468d1fa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.720+0000 7ff74d7eb740 1 --2- 172.21.15.100:0/1348967434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56468cdc33b0 0x56468d1dad10 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.720+0000 7ff74d7eb740 1 -- 172.21.15.100:0/1348967434 >> 172.21.15.100:0/1348967434 conn(0x56468cde03c0 msgr2=0x56468d1886c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:55.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.720+0000 7ff74d7eb740 1 -- 172.21.15.100:0/1348967434 shutdown_connections 2024-09-17T04:28:55.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.720+0000 7ff74d7eb740 1 -- 172.21.15.100:0/1348967434 wait complete. 2024-09-17T04:28:55.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.721+0000 7ff74d7eb740 1 -- start start 2024-09-17T04:28:55.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.722+0000 7ff74d7eb740 1 Processor -- start 2024-09-17T04:28:55.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.723+0000 7ff74d7eb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56468cdc33b0 0x56468d25afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:55.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.723+0000 7ff74d7eb740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56468cdcb590 0x56468d2584e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:55.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.723+0000 7ff74d7eb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56468cdec270 0x56468d1fa740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:55.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.723+0000 7ff74d7eb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56468d1b7270 con 0x56468cdec270 2024-09-17T04:28:55.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.723+0000 7ff74d7eb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56468d1a3bc0 con 0x56468cdc33b0 2024-09-17T04:28:55.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.723+0000 7ff74d7eb740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56468d1e9ef0 con 0x56468cdcb590 2024-09-17T04:28:55.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.723+0000 7ff74d7eb740 10 client.0 _ll_register_callbacks cb 0x7ff73007f610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:28:55.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.723+0000 7ff73fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56468cdec270 0x56468d1fa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:55.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.723+0000 7ff73fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56468cdec270 0x56468d1fa740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34712/0 (socket says 172.21.15.100:34712) 2024-09-17T04:28:55.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.723+0000 7ff73fe00640 1 -- 172.21.15.100:0/138609958 learned_addr learned my addr 172.21.15.100:0/138609958 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:55.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.723+0000 7ff73f400640 1 --2- 172.21.15.100:0/138609958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56468cdc33b0 0x56468d25afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:55.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.723+0000 7ff736a00640 1 --2- 172.21.15.100:0/138609958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56468cdcb590 0x56468d2584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:55.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.724+0000 7ff73fe00640 1 -- 172.21.15.100:0/138609958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56468cdcb590 msgr2=0x56468d2584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:28:55.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.724+0000 7ff73fe00640 1 --2- 172.21.15.100:0/138609958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56468cdcb590 0x56468d2584e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.724+0000 7ff73fe00640 1 -- 172.21.15.100:0/138609958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56468cdc33b0 msgr2=0x56468d25afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:28:55.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.724+0000 7ff73fe00640 1 --2- 172.21.15.100:0/138609958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56468cdc33b0 0x56468d25afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.724+0000 7ff73fe00640 1 -- 172.21.15.100:0/138609958 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56468d1b7000 con 0x56468cdec270 2024-09-17T04:28:55.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.724+0000 7ff73fe00640 1 --2- 172.21.15.100:0/138609958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56468cdec270 0x56468d1fa740 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff738013000 tx=0x7ff7380017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ebea68cb2570f6b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:55.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.724+0000 7ff72cc00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-17T04:28:55.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.724+0000 7ff72cc00640 1 -- 172.21.15.100:0/138609958 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff738023030 con 0x56468cdec270 2024-09-17T04:28:55.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.724+0000 7ff72cc00640 1 -- 172.21.15.100:0/138609958 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 5767+0+0 (secure 0 0 0) 0x7ff73800ed40 con 0x56468cdec270 2024-09-17T04:28:55.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.724+0000 7ff74d7eb740 1 -- 172.21.15.100:0/138609958 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff730083870 con 0x56468cdec270 2024-09-17T04:28:55.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.725+0000 7ff72cc00640 1 -- 172.21.15.100:0/138609958 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff738019040 con 0x56468cdec270 2024-09-17T04:28:55.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.725+0000 7ff74d7eb740 10 client.6324 fetch_fsmap learned FSMap version 47 2024-09-17T04:28:55.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.725+0000 7ff74d7eb740 1 -- 172.21.15.100:0/138609958 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) -- 0x56468d2174e0 con 0x56468cdec270 2024-09-17T04:28:55.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.725+0000 7ff72cc00640 1 -- 172.21.15.100:0/138609958 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff730083870 con 0x56468cdec270 2024-09-17T04:28:55.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.725+0000 7ff72cc00640 1 -- 172.21.15.100:0/138609958 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 47) ==== 3502+0+0 (secure 0 0 0) 0x7ff738014480 con 0x56468cdec270 2024-09-17T04:28:55.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.725+0000 7ff74d7eb740 10 client.6324 fetch_fsmap finished waiting for FSMap version 47 2024-09-17T04:28:55.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.725+0000 7ff74d7eb740 10 client.6324 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T04:28:55.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.725+0000 7ff74d7eb740 20 client.6324 populate_metadata read hostname 'smithi100' 2024-09-17T04:28:55.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.725+0000 7ff74d7eb740 1 --2- 172.21.15.100:0/138609958 >> [v2:172.21.15.100:6834/2913442826,v1:172.21.15.100:6835/2913442826] conn(0x56468d1d8f90 0x56468d1f9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:55.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.725+0000 7ff74d7eb740 4 client.6324 mds_command: new command op to 6162 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T04:28:55.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.725+0000 7ff74d7eb740 1 -- 172.21.15.100:0/138609958 --> [v2:172.21.15.100:6834/2913442826,v1:172.21.15.100:6835/2913442826] -- command(tid 0: {"prefix": "status"}) -- 0x56468d1921a0 con 0x56468d1d8f90 2024-09-17T04:28:55.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.725+0000 7ff73f400640 1 --2- 172.21.15.100:0/138609958 >> [v2:172.21.15.100:6834/2913442826,v1:172.21.15.100:6835/2913442826] conn(0x56468d1d8f90 0x56468d1f9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:55.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.726+0000 7ff73f400640 1 --2- 172.21.15.100:0/138609958 >> [v2:172.21.15.100:6834/2913442826,v1:172.21.15.100:6835/2913442826] conn(0x56468d1d8f90 0x56468d1f9200 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3848696552dee9e server_cookie=bba8a14a894a8cb7 in_seq=0 out_seq=0 2024-09-17T04:28:55.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.726+0000 7ff72cc00640 10 client.6324 ms_handle_connect on v2:172.21.15.100:6834/2913442826 2024-09-17T04:28:55.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.726+0000 7ff72cc00640 1 -- 172.21.15.100:0/138609958 <== mds.0 v2:172.21.15.100:6834/2913442826 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x56468d1921a0 con 0x56468d1d8f90 2024-09-17T04:28:55.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.726+0000 7ff72cc00640 10 client.6324 handle_command_reply: tid=0 2024-09-17T04:28:55.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.726+0000 7ff74d7eb740 1 client.6324 shutdown 2024-09-17T04:28:55.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.726+0000 7ff74d7eb740 1 -- 172.21.15.100:0/138609958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56468cdec270 msgr2=0x56468d1fa740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:55.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.726+0000 7ff74d7eb740 1 --2- 172.21.15.100:0/138609958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56468cdec270 0x56468d1fa740 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff738013000 tx=0x7ff7380017b0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.726+0000 7ff74d7eb740 1 -- 172.21.15.100:0/138609958 shutdown_connections 2024-09-17T04:28:55.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.726+0000 7ff74d7eb740 1 --2- 172.21.15.100:0/138609958 >> [v2:172.21.15.100:6834/2913442826,v1:172.21.15.100:6835/2913442826] conn(0x56468d1d8f90 0x56468d1f9200 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.726+0000 7ff74d7eb740 1 --2- 172.21.15.100:0/138609958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56468cdec270 0x56468d1fa740 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.726+0000 7ff74d7eb740 1 --2- 172.21.15.100:0/138609958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56468cdcb590 0x56468d2584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.726+0000 7ff74d7eb740 1 --2- 172.21.15.100:0/138609958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56468cdc33b0 0x56468d25afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.726+0000 7ff74d7eb740 1 -- 172.21.15.100:0/138609958 >> 172.21.15.100:0/138609958 conn(0x56468cde03c0 msgr2=0x56468d1884b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:55.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.726+0000 7ff72cc00640 0 client.6324 ms_handle_reset on v2:172.21.15.100:6834/2913442826 2024-09-17T04:28:55.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.726+0000 7ff74d7eb740 1 -- 172.21.15.100:0/138609958 shutdown_connections 2024-09-17T04:28:55.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.726+0000 7ff74d7eb740 1 -- 172.21.15.100:0/138609958 wait complete. 2024-09-17T04:28:55.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.726+0000 7ff74d7eb740 20 client.6324 trim_cache size 0 max 16384 2024-09-17T04:28:55.734 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:28:55.734 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "8a399842-3a5c-4ea2-9afb-b2704332e59f", 2024-09-17T04:28:55.735 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 0, 2024-09-17T04:28:55.735 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 6162, 2024-09-17T04:28:55.735 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-09-17T04:28:55.735 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-09-17T04:28:55.735 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-09-17T04:28:55.735 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 11.559276722, 2024-09-17T04:28:55.735 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 47, 2024-09-17T04:28:55.735 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 75, 2024-09-17T04:28:55.735 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 73, 2024-09-17T04:28:55.735 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 23.375350060999999 2024-09-17T04:28:55.735 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:28:55.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.727+0000 7ff745000640 1 -- 172.21.15.100:0/3522771745 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff71c078ef0 msgr2=0x7ff71c07b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:55.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.727+0000 7ff745000640 1 --2- 172.21.15.100:0/3522771745 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff71c078ef0 0x7ff71c07b400 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff7401561b0 tx=0x7ff728005f10 comp rx=0 tx=0).stop 2024-09-17T04:28:55.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.727+0000 7ff745000640 1 -- 172.21.15.100:0/3522771745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff740070000 msgr2=0x7ff740143660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:55.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.727+0000 7ff745000640 1 --2- 172.21.15.100:0/3522771745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff740070000 0x7ff740143660 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff730034040 tx=0x7ff7300038a0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.727+0000 7ff745000640 1 -- 172.21.15.100:0/3522771745 shutdown_connections 2024-09-17T04:28:55.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.727+0000 7ff745000640 1 --2- 172.21.15.100:0/3522771745 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff71c078ef0 0x7ff71c07b400 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.727+0000 7ff745000640 1 --2- 172.21.15.100:0/3522771745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7400af050 0x7ff74014b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.727+0000 7ff745000640 1 --2- 172.21.15.100:0/3522771745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff740072e60 0x7ff7401f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.727+0000 7ff745000640 1 --2- 172.21.15.100:0/3522771745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff740070000 0x7ff740143660 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.727+0000 7ff745000640 1 -- 172.21.15.100:0/3522771745 >> 172.21.15.100:0/3522771745 conn(0x7ff7400b1e40 msgr2=0x7ff74009cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:55.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.727+0000 7ff745000640 1 -- 172.21.15.100:0/3522771745 shutdown_connections 2024-09-17T04:28:55.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.727+0000 7ff745000640 1 -- 172.21.15.100:0/3522771745 wait complete. 2024-09-17T04:28:55.751 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:28:55.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.970+0000 7fc6d5000640 1 -- 172.21.15.100:0/4243713900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6d00ad0e0 msgr2=0x7fc6d00ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:55.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.970+0000 7fc6d5000640 1 --2- 172.21.15.100:0/4243713900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6d00ad0e0 0x7fc6d00ad4f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009830 tx=0x7fc6bc02a480 comp rx=0 tx=0).stop 2024-09-17T04:28:55.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.970+0000 7fc6d5000640 1 -- 172.21.15.100:0/4243713900 shutdown_connections 2024-09-17T04:28:55.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.970+0000 7fc6d5000640 1 --2- 172.21.15.100:0/4243713900 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6d00b3550 0x7fc6d00b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.970+0000 7fc6d5000640 1 --2- 172.21.15.100:0/4243713900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6d00ad0e0 0x7fc6d00ad4f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.970+0000 7fc6d5000640 1 -- 172.21.15.100:0/4243713900 >> 172.21.15.100:0/4243713900 conn(0x7fc6d006d730 msgr2=0x7fc6d00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:55.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.970+0000 7fc6d5000640 1 -- 172.21.15.100:0/4243713900 shutdown_connections 2024-09-17T04:28:55.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.970+0000 7fc6d5000640 1 -- 172.21.15.100:0/4243713900 wait complete. 2024-09-17T04:28:55.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.972+0000 7fc6d5000640 1 Processor -- start 2024-09-17T04:28:55.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.973+0000 7fc6d5000640 1 -- start start 2024-09-17T04:28:55.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.974+0000 7fc6d5000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6d00ad0e0 0x7fc6d015a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:55.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.974+0000 7fc6d5000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6d00b3550 0x7fc6d0154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:55.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.974+0000 7fc6d5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6d015d760 0x7fc6d0147770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:55.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.974+0000 7fc6d5000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d00b4da0 con 0x7fc6d015d760 2024-09-17T04:28:55.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.974+0000 7fc6d5000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d00b4c20 con 0x7fc6d00ad0e0 2024-09-17T04:28:55.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.974+0000 7fc6d5000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc6d00b3ea0 con 0x7fc6d00b3550 2024-09-17T04:28:55.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.974+0000 7fc6ce800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6d015d760 0x7fc6d0147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:55.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.974+0000 7fc6ce800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6d015d760 0x7fc6d0147770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34728/0 (socket says 172.21.15.100:34728) 2024-09-17T04:28:55.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.974+0000 7fc6ce800640 1 -- 172.21.15.100:0/1434945233 learned_addr learned my addr 172.21.15.100:0/1434945233 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:55.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.975+0000 7fc6cd400640 1 --2- 172.21.15.100:0/1434945233 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6d00b3550 0x7fc6d0154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:55.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.975+0000 7fc6cde00640 1 --2- 172.21.15.100:0/1434945233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6d00ad0e0 0x7fc6d015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:55.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.975+0000 7fc6ce800640 1 -- 172.21.15.100:0/1434945233 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6d00b3550 msgr2=0x7fc6d0154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:55.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.975+0000 7fc6ce800640 1 --2- 172.21.15.100:0/1434945233 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6d00b3550 0x7fc6d0154770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.975+0000 7fc6ce800640 1 -- 172.21.15.100:0/1434945233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6d00ad0e0 msgr2=0x7fc6d015a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:55.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.975+0000 7fc6ce800640 1 --2- 172.21.15.100:0/1434945233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6d00ad0e0 0x7fc6d015a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:55.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.975+0000 7fc6ce800640 1 -- 172.21.15.100:0/1434945233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d0157510 con 0x7fc6d015d760 2024-09-17T04:28:55.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.975+0000 7fc6ce800640 1 --2- 172.21.15.100:0/1434945233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6d015d760 0x7fc6d0147770 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fc6c800ec50 tx=0x7fc6c800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:55.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.976+0000 7fc6c6000640 1 -- 172.21.15.100:0/1434945233 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8003b70 con 0x7fc6d015d760 2024-09-17T04:28:55.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.976+0000 7fc6c6000640 1 -- 172.21.15.100:0/1434945233 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6c8003d10 con 0x7fc6d015d760 2024-09-17T04:28:55.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.976+0000 7fc6d5000640 1 -- 172.21.15.100:0/1434945233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d01577f0 con 0x7fc6d015d760 2024-09-17T04:28:55.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.976+0000 7fc6c6000640 1 -- 172.21.15.100:0/1434945233 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8004280 con 0x7fc6d015d760 2024-09-17T04:28:55.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.976+0000 7fc6d5000640 1 -- 172.21.15.100:0/1434945233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d0154250 con 0x7fc6d015d760 2024-09-17T04:28:55.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.977+0000 7fc6d5000640 1 -- 172.21.15.100:0/1434945233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6d00b4370 con 0x7fc6d015d760 2024-09-17T04:28:55.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.978+0000 7fc6c6000640 1 -- 172.21.15.100:0/1434945233 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fc6c8004420 con 0x7fc6d015d760 2024-09-17T04:28:55.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.979+0000 7fc6c6000640 1 --2- 172.21.15.100:0/1434945233 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc6b0078f20 0x7fc6b007b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:55.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.979+0000 7fc6c6000640 1 -- 172.21.15.100:0/1434945233 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5767+0+0 (secure 0 0 0) 0x7fc6c809c160 con 0x7fc6d015d760 2024-09-17T04:28:55.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.979+0000 7fc6cde00640 1 --2- 172.21.15.100:0/1434945233 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc6b0078f20 0x7fc6b007b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:55.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.980+0000 7fc6cde00640 1 --2- 172.21.15.100:0/1434945233 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc6b0078f20 0x7fc6b007b430 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0007d60 tx=0x7fc6c0007cf0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:55.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:55.983+0000 7fc6c6000640 1 -- 172.21.15.100:0/1434945233 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc6c80639d0 con 0x7fc6d015d760 2024-09-17T04:28:56.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:56.201+0000 7fc6d5000640 1 -- 172.21.15.100:0/1434945233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc6d00cfcc0 con 0x7fc6d015d760 2024-09-17T04:28:56.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:56.202+0000 7fc6c6000640 1 -- 172.21.15.100:0/1434945233 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) ==== 53+0+73 (secure 0 0 0) 0x7fc6c8068400 con 0x7fc6d015d760 2024-09-17T04:28:56.202 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:28:56.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:56.204+0000 7fc6d5000640 1 -- 172.21.15.100:0/1434945233 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc6b0078f20 msgr2=0x7fc6b007b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:56.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:56.204+0000 7fc6d5000640 1 --2- 172.21.15.100:0/1434945233 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc6b0078f20 0x7fc6b007b430 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0007d60 tx=0x7fc6c0007cf0 comp rx=0 tx=0).stop 2024-09-17T04:28:56.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:56.204+0000 7fc6d5000640 1 -- 172.21.15.100:0/1434945233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6d015d760 msgr2=0x7fc6d0147770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:56.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:56.204+0000 7fc6d5000640 1 --2- 172.21.15.100:0/1434945233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6d015d760 0x7fc6d0147770 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fc6c800ec50 tx=0x7fc6c800c6a0 comp rx=0 tx=0).stop 2024-09-17T04:28:56.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:56.204+0000 7fc6d5000640 1 -- 172.21.15.100:0/1434945233 shutdown_connections 2024-09-17T04:28:56.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:56.204+0000 7fc6d5000640 1 --2- 172.21.15.100:0/1434945233 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc6b0078f20 0x7fc6b007b430 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:56.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:56.204+0000 7fc6d5000640 1 --2- 172.21.15.100:0/1434945233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6d015d760 0x7fc6d0147770 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:56.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:56.204+0000 7fc6d5000640 1 --2- 172.21.15.100:0/1434945233 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6d00b3550 0x7fc6d0154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:56.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:56.204+0000 7fc6d5000640 1 --2- 172.21.15.100:0/1434945233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6d00ad0e0 0x7fc6d015a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:56.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:56.204+0000 7fc6d5000640 1 -- 172.21.15.100:0/1434945233 >> 172.21.15.100:0/1434945233 conn(0x7fc6d006d730 msgr2=0x7fc6d015f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:56.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:56.204+0000 7fc6d5000640 1 -- 172.21.15.100:0/1434945233 shutdown_connections 2024-09-17T04:28:56.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:56.204+0000 7fc6d5000640 1 -- 172.21.15.100:0/1434945233 wait complete. 2024-09-17T04:28:56.227 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T04:28:56.227 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:28:56.227 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-09-17T04:28:56.227 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T04:28:56.227 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T04:28:56.227 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T04:28:56.227 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T04:28:56.227 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:28:56.228 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:28:56.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:28:56.228 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-09-17T04:28:56.264 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:28:56.265 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:28:56.265 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:28:56.265 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:28:56.265 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-17T04:28:56.265 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:28:56.265 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:28:56.265 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:28:56.265 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-09-17T04:28:56.266 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:28:56.266 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:28:56.266 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-09-17T04:28:56.266 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:28:56.266 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:28:56.266 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:28:56.266 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:28:56.266 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T04:28:56.267 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42424sec preferred_lft 42424sec 2024-09-17T04:28:56.267 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link noprefixroute 2024-09-17T04:28:56.267 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:28:56.267 INFO:teuthology.orchestra.run.smithi100.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:28:56.267 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:2e:de:8e:a4:7c brd ff:ff:ff:ff:ff:ff 2024-09-17T04:28:56.267 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T04:28:56.267 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:28:56.267 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::cc78:e4ff:fe70:7fbe/64 scope link 2024-09-17T04:28:56.267 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:28:56.267 INFO:teuthology.orchestra.run.smithi100.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:28:56.267 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether ea:8c:45:9f:34:87 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:28:56.267 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::e88c:45ff:fe9f:3487/64 scope link 2024-09-17T04:28:56.267 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:28:56.267 INFO:teuthology.orchestra.run.smithi100.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:28:56.268 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:2e:de:8e:a4:7c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:28:56.268 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::c2e:deff:fe8e:a47c/64 scope link 2024-09-17T04:28:56.268 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:28:56.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:28:56.268 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:28:56.360 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-09-17T04:28:56.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:28:56.362 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-09-17T04:28:56.399 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-09-17T04:28:56.399 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:28:56.399 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T04:28:56.399 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T04:28:56.399 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:28:56.400 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:28:56.400 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-17T04:28:56.400 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T04:28:56.400 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T04:28:56.400 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T04:28:56.400 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-17T04:28:56.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:28:56.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:28:56.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:28:56.596 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-17T04:28:56.632 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T04:28:56.632 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T04:28:56.632 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:28:56.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:28:56.633 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-09-17T04:28:56.729 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:28:56.729 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T04:28:56.765 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:28:56.765 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:28:56.765 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:28:56.841 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T04:28:56.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:28:56.841 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:28:57.036 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:28:57.037 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:28:57.037 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:28:57.056 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:28:57.057 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:28:57.057 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:28:57.057 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:28:57.057 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:28:57.090 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:28:57.090 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:28:57.090 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:28:57.090 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:28:57.090 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:28:57.090 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-09-17T04:28:57.090 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-09-17T04:28:57.090 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8a399842-3a5c-4ea2-9afb-b2704332e59f" will pass to kernel 2024-09-17T04:28:57.091 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-17T04:28:57.091 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-09-17T04:28:57.092 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-17T04:28:57.092 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-09-17T04:28:57.092 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-17T04:28:57.094 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T04:28:57.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:28:57.095 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:28:57.152 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:28:57.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:28:57.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:28:57.247 INFO:teuthology.orchestra.run.smithi100.stdout:client5624 2024-09-17T04:28:57.247 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:28:57.247 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:28:57.284 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:28:57.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.546+0000 7f3f77400640 1 -- 172.21.15.100:0/1527005197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f58003c10 msgr2=0x7f3f58004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:57.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.546+0000 7f3f77400640 1 --2- 172.21.15.100:0/1527005197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f58003c10 0x7f3f58004060 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f3f64001830 tx=0x7f3f58005830 comp rx=0 tx=0).stop 2024-09-17T04:28:57.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.546+0000 7f3f75400640 1 -- 172.21.15.100:0/1527005197 reap_dead start 2024-09-17T04:28:57.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.546+0000 7f3f77400640 1 -- 172.21.15.100:0/1527005197 shutdown_connections 2024-09-17T04:28:57.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.546+0000 7f3f77400640 1 -- 172.21.15.100:0/1527005197 >> 172.21.15.100:0/1527005197 conn(0x7f3f7006d730 msgr2=0x7f3f700b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:57.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.546+0000 7f3f77400640 1 -- 172.21.15.100:0/1527005197 shutdown_connections 2024-09-17T04:28:57.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.546+0000 7f3f77400640 1 -- 172.21.15.100:0/1527005197 wait complete. 2024-09-17T04:28:57.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.549+0000 7f3f77400640 1 Processor -- start 2024-09-17T04:28:57.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.549+0000 7f3f77400640 1 -- start start 2024-09-17T04:28:57.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.550+0000 7f3f77400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f58003c10 0x7f3f701f95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:57.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.550+0000 7f3f77400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f700b1ab0 0x7f3f701e4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:57.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.550+0000 7f3f77400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f701e4cb0 0x7f3f701f2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:57.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.550+0000 7f3f77400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3f700712b0 con 0x7f3f701e4cb0 2024-09-17T04:28:57.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.550+0000 7f3f77400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3f70071130 con 0x7f3f700b1ab0 2024-09-17T04:28:57.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.550+0000 7f3f77400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3f700ad7d0 con 0x7f3f58003c10 2024-09-17T04:28:57.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.550+0000 7f3f75400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f701e4cb0 0x7f3f701f2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:57.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.550+0000 7f3f75400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f701e4cb0 0x7f3f701f2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34746/0 (socket says 172.21.15.100:34746) 2024-09-17T04:28:57.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.550+0000 7f3f75400640 1 -- 172.21.15.100:0/2411864253 learned_addr learned my addr 172.21.15.100:0/2411864253 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:57.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.551+0000 7f3f74a00640 1 --2- 172.21.15.100:0/2411864253 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f58003c10 0x7f3f701f95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:57.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.551+0000 7f3f6fe00640 1 --2- 172.21.15.100:0/2411864253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f700b1ab0 0x7f3f701e4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:57.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.551+0000 7f3f75400640 1 -- 172.21.15.100:0/2411864253 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f58003c10 msgr2=0x7f3f701f95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:57.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.551+0000 7f3f75400640 1 --2- 172.21.15.100:0/2411864253 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f58003c10 0x7f3f701f95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:57.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.551+0000 7f3f75400640 1 -- 172.21.15.100:0/2411864253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f700b1ab0 msgr2=0x7f3f701e4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:57.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.551+0000 7f3f75400640 1 --2- 172.21.15.100:0/2411864253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f700b1ab0 0x7f3f701e4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:57.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.551+0000 7f3f75400640 1 -- 172.21.15.100:0/2411864253 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f700b1370 con 0x7f3f701e4cb0 2024-09-17T04:28:57.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.551+0000 7f3f75400640 1 --2- 172.21.15.100:0/2411864253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f701e4cb0 0x7f3f701f2620 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f3f700745c0 tx=0x7f3f68006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:57.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.551+0000 7f3f6cc00640 1 -- 172.21.15.100:0/2411864253 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6800fa00 con 0x7f3f701e4cb0 2024-09-17T04:28:57.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.551+0000 7f3f6cc00640 1 -- 172.21.15.100:0/2411864253 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f6800fba0 con 0x7f3f701e4cb0 2024-09-17T04:28:57.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.551+0000 7f3f77400640 1 -- 172.21.15.100:0/2411864253 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f7015a1e0 con 0x7f3f701e4cb0 2024-09-17T04:28:57.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.552+0000 7f3f6cc00640 1 -- 172.21.15.100:0/2411864253 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f68011430 con 0x7f3f701e4cb0 2024-09-17T04:28:57.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.552+0000 7f3f77400640 1 -- 172.21.15.100:0/2411864253 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f700afee0 con 0x7f3f701e4cb0 2024-09-17T04:28:57.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.553+0000 7f3f77400640 1 -- 172.21.15.100:0/2411864253 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f700b10e0 con 0x7f3f701e4cb0 2024-09-17T04:28:57.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.554+0000 7f3f6cc00640 1 -- 172.21.15.100:0/2411864253 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f3f680115d0 con 0x7f3f701e4cb0 2024-09-17T04:28:57.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.555+0000 7f3f6cc00640 1 --2- 172.21.15.100:0/2411864253 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3f50078f20 0x7f3f5007b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:57.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.555+0000 7f3f6cc00640 1 -- 172.21.15.100:0/2411864253 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5767+0+0 (secure 0 0 0) 0x7f3f6809bca0 con 0x7f3f701e4cb0 2024-09-17T04:28:57.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.555+0000 7f3f74a00640 1 --2- 172.21.15.100:0/2411864253 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3f50078f20 0x7f3f5007b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:57.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.556+0000 7f3f74a00640 1 --2- 172.21.15.100:0/2411864253 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3f50078f20 0x7f3f5007b430 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3f64032000 tx=0x7f3f64001360 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:57.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.559+0000 7f3f6cc00640 1 -- 172.21.15.100:0/2411864253 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3f68063b60 con 0x7f3f701e4cb0 2024-09-17T04:28:57.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.780+0000 7f3f77400640 1 -- 172.21.15.100:0/2411864253 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3f700afd10 con 0x7f3f701e4cb0 2024-09-17T04:28:57.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.781+0000 7f3f6cc00640 1 -- 172.21.15.100:0/2411864253 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) ==== 53+0+73 (secure 0 0 0) 0x7f3f68006d70 con 0x7f3f701e4cb0 2024-09-17T04:28:57.781 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:28:57.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.783+0000 7f3f77400640 1 -- 172.21.15.100:0/2411864253 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3f50078f20 msgr2=0x7f3f5007b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:57.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.783+0000 7f3f77400640 1 --2- 172.21.15.100:0/2411864253 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3f50078f20 0x7f3f5007b430 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3f64032000 tx=0x7f3f64001360 comp rx=0 tx=0).stop 2024-09-17T04:28:57.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.783+0000 7f3f77400640 1 -- 172.21.15.100:0/2411864253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f701e4cb0 msgr2=0x7f3f701f2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:57.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.783+0000 7f3f77400640 1 --2- 172.21.15.100:0/2411864253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f701e4cb0 0x7f3f701f2620 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f3f700745c0 tx=0x7f3f68006660 comp rx=0 tx=0).stop 2024-09-17T04:28:57.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.783+0000 7f3f77400640 1 -- 172.21.15.100:0/2411864253 shutdown_connections 2024-09-17T04:28:57.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.783+0000 7f3f77400640 1 --2- 172.21.15.100:0/2411864253 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f3f50078f20 0x7f3f5007b430 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:57.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.783+0000 7f3f77400640 1 --2- 172.21.15.100:0/2411864253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f701e4cb0 0x7f3f701f2620 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:57.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.783+0000 7f3f77400640 1 --2- 172.21.15.100:0/2411864253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f700b1ab0 0x7f3f701e4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:57.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.783+0000 7f3f77400640 1 --2- 172.21.15.100:0/2411864253 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f58003c10 0x7f3f701f95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:57.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.783+0000 7f3f77400640 1 -- 172.21.15.100:0/2411864253 >> 172.21.15.100:0/2411864253 conn(0x7f3f7006d730 msgr2=0x7f3f7014fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:57.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.783+0000 7f3f77400640 1 -- 172.21.15.100:0/2411864253 shutdown_connections 2024-09-17T04:28:57.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:57.783+0000 7f3f77400640 1 -- 172.21.15.100:0/2411864253 wait complete. 2024-09-17T04:28:57.807 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T04:28:57.807 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:28:57.807 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2024-09-17T04:28:57.807 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T04:28:57.807 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T04:28:57.807 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T04:28:57.807 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T04:28:57.807 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:28:57.807 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:28:57.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:28:57.808 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2024-09-17T04:28:57.848 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:28:57.849 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:28:57.849 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:28:57.849 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:28:57.849 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-09-17T04:28:57.849 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:28:57.849 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:28:57.849 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:28:57.849 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2024-09-17T04:28:57.849 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:28:57.850 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:28:57.850 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2024-09-17T04:28:57.850 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:28:57.850 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2024-09-17T04:28:57.850 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:28:57.850 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2024-09-17T04:28:57.850 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T04:28:57.850 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42406sec preferred_lft 42406sec 2024-09-17T04:28:57.850 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ec4:7aff:fe88:7b8f/64 scope link noprefixroute 2024-09-17T04:28:57.850 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:28:57.850 INFO:teuthology.orchestra.run.smithi184.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:28:57.850 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 7e:67:e5:69:96:69 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:28:57.851 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T04:28:57.851 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:28:57.851 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::3c9d:aff:fe6e:8fcb/64 scope link 2024-09-17T04:28:57.851 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:28:57.851 INFO:teuthology.orchestra.run.smithi184.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:28:57.851 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether a6:a9:1b:a9:61:ec brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T04:28:57.851 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::a4a9:1bff:fea9:61ec/64 scope link 2024-09-17T04:28:57.851 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:28:57.851 INFO:teuthology.orchestra.run.smithi184.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:28:57.851 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 7e:67:e5:69:96:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T04:28:57.851 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::7c67:e5ff:fe69:9669/64 scope link 2024-09-17T04:28:57.851 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:28:57.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:28:57.852 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:28:57.946 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-09-17T04:28:57.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:28:57.950 DEBUG:teuthology.orchestra.run.smithi184:> route 2024-09-17T04:28:57.987 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2024-09-17T04:28:57.988 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:28:57.988 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T04:28:57.988 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T04:28:57.988 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:28:57.989 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:28:57.989 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-09-17T04:28:57.989 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T04:28:57.989 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T04:28:57.989 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T04:28:57.990 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-09-17T04:28:58.106 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:28:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:28:58.176 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:28:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:28:58.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:28:58.185 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-09-17T04:28:58.222 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T04:28:58.222 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T04:28:58.222 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:28:58.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:28:58.222 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.0 up 2024-09-17T04:28:58.318 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:28:58.318 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T04:28:58.355 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:28:58.355 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:28:58.355 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T04:28:58.430 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T04:28:58.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:28:58.431 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:28:58.624 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:28:58.625 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:28:58.625 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:28:58.644 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:28:58.645 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:28:58.645 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:28:58.645 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:28:58.645 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:28:58.675 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:28:58.675 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:28:58.675 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:28:58.676 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:28:58.676 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:28:58.676 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-09-17T04:28:58.676 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-09-17T04:28:58.676 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8a399842-3a5c-4ea2-9afb-b2704332e59f" will pass to kernel 2024-09-17T04:28:58.677 INFO:teuthology.orchestra.run.smithi184.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-17T04:28:58.677 INFO:teuthology.orchestra.run.smithi184.stdout: You just mounted a file system that supports labels which does not 2024-09-17T04:28:58.677 INFO:teuthology.orchestra.run.smithi184.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-17T04:28:58.677 INFO:teuthology.orchestra.run.smithi184.stdout: applications will generate AVC messages and not be allowed access to 2024-09-17T04:28:58.677 INFO:teuthology.orchestra.run.smithi184.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-17T04:28:58.680 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T04:28:58.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:28:58.680 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T04:28:58.736 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:28:58.814 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:28:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:28:58.817 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:28:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:28:58.825 INFO:teuthology.orchestra.run.smithi184.stdout:client6336 2024-09-17T04:28:58.825 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:28:58.826 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:28:58.861 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T04:28:59.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.089+0000 7f5e7f200640 1 -- 172.21.15.100:0/3084603745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e78070470 msgr2=0x7f5e780ae080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:59.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.089+0000 7f5e7f200640 1 --2- 172.21.15.100:0/3084603745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e78070470 0x7f5e780ae080 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f5e74005fd0 tx=0x7f5e7402a8b0 comp rx=0 tx=0).stop 2024-09-17T04:28:59.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.089+0000 7f5e7f200640 1 -- 172.21.15.100:0/3084603745 shutdown_connections 2024-09-17T04:28:59.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.089+0000 7f5e7f200640 1 --2- 172.21.15.100:0/3084603745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e78070470 0x7f5e780ae080 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:59.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.089+0000 7f5e7f200640 1 -- 172.21.15.100:0/3084603745 >> 172.21.15.100:0/3084603745 conn(0x7f5e780b2220 msgr2=0x7f5e780d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:59.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.089+0000 7f5e7f200640 1 -- 172.21.15.100:0/3084603745 shutdown_connections 2024-09-17T04:28:59.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.089+0000 7f5e7f200640 1 -- 172.21.15.100:0/3084603745 wait complete. 2024-09-17T04:28:59.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.091+0000 7f5e7f200640 1 Processor -- start 2024-09-17T04:28:59.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.092+0000 7f5e7f200640 1 -- start start 2024-09-17T04:28:59.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.093+0000 7f5e7f200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e780aec80 0x7f5e780c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:59.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.093+0000 7f5e7f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e780b56e0 0x7f5e78156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:59.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.093+0000 7f5e7f200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e781570c0 0x7f5e7815a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:59.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.093+0000 7f5e7f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5e780ae740 con 0x7f5e780b56e0 2024-09-17T04:28:59.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.093+0000 7f5e7f200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5e780ae5c0 con 0x7f5e780aec80 2024-09-17T04:28:59.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.093+0000 7f5e7f200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5e780ae8c0 con 0x7f5e781570c0 2024-09-17T04:28:59.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.093+0000 7f5e7d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e780b56e0 0x7f5e78156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:59.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.093+0000 7f5e7d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e780b56e0 0x7f5e78156770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34768/0 (socket says 172.21.15.100:34768) 2024-09-17T04:28:59.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.093+0000 7f5e7d400640 1 -- 172.21.15.100:0/3303426565 learned_addr learned my addr 172.21.15.100:0/3303426565 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:59.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.093+0000 7f5e7e800640 1 --2- 172.21.15.100:0/3303426565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e781570c0 0x7f5e7815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:59.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.093+0000 7f5e7de00640 1 --2- 172.21.15.100:0/3303426565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e780aec80 0x7f5e780c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:59.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.094+0000 7f5e7d400640 1 -- 172.21.15.100:0/3303426565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e781570c0 msgr2=0x7f5e7815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:59.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.094+0000 7f5e7d400640 1 --2- 172.21.15.100:0/3303426565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e781570c0 0x7f5e7815a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:59.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.094+0000 7f5e7d400640 1 -- 172.21.15.100:0/3303426565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e780aec80 msgr2=0x7f5e780c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:59.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.094+0000 7f5e7d400640 1 --2- 172.21.15.100:0/3303426565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e780aec80 0x7f5e780c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:59.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.094+0000 7f5e7d400640 1 -- 172.21.15.100:0/3303426565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e780c7cb0 con 0x7f5e780b56e0 2024-09-17T04:28:59.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.094+0000 7f5e7d400640 1 --2- 172.21.15.100:0/3303426565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e780b56e0 0x7f5e78156770 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f5e6800c3a0 tx=0x7f5e6800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:59.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.094+0000 7f5e72000640 1 -- 172.21.15.100:0/3303426565 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e68017070 con 0x7f5e780b56e0 2024-09-17T04:28:59.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.094+0000 7f5e72000640 1 -- 172.21.15.100:0/3303426565 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e680053c0 con 0x7f5e780b56e0 2024-09-17T04:28:59.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.094+0000 7f5e72000640 1 -- 172.21.15.100:0/3303426565 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e68003850 con 0x7f5e780b56e0 2024-09-17T04:28:59.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.094+0000 7f5e7f200640 1 -- 172.21.15.100:0/3303426565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e780c7f90 con 0x7f5e780b56e0 2024-09-17T04:28:59.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.095+0000 7f5e7f200640 1 -- 172.21.15.100:0/3303426565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e780c8550 con 0x7f5e780b56e0 2024-09-17T04:28:59.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.096+0000 7f5e7f200640 1 -- 172.21.15.100:0/3303426565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e780c9680 con 0x7f5e780b56e0 2024-09-17T04:28:59.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.097+0000 7f5e72000640 1 -- 172.21.15.100:0/3303426565 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f5e6801a070 con 0x7f5e780b56e0 2024-09-17T04:28:59.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.098+0000 7f5e72000640 1 --2- 172.21.15.100:0/3303426565 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5e5c078f20 0x7f5e5c07b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:59.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.098+0000 7f5e72000640 1 -- 172.21.15.100:0/3303426565 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5767+0+0 (secure 0 0 0) 0x7f5e6809a650 con 0x7f5e780b56e0 2024-09-17T04:28:59.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.098+0000 7f5e7de00640 1 --2- 172.21.15.100:0/3303426565 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5e5c078f20 0x7f5e5c07b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:59.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.099+0000 7f5e7de00640 1 --2- 172.21.15.100:0/3303426565 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5e5c078f20 0x7f5e5c07b430 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c000f60 tx=0x7f5e6c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:59.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.103+0000 7f5e72000640 1 -- 172.21.15.100:0/3303426565 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f5e780b2ef0 con 0x7f5e780b56e0 2024-09-17T04:28:59.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.324+0000 7f5e7f200640 1 -- 172.21.15.100:0/3303426565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f5e780ad970 con 0x7f5e780b56e0 2024-09-17T04:28:59.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.346+0000 7f5e72000640 1 -- 172.21.15.100:0/3303426565 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v48) ==== 136+0+0 (secure 0 0 0) 0x7f5e68061f30 con 0x7f5e780b56e0 2024-09-17T04:28:59.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.349+0000 7f5e7f200640 1 -- 172.21.15.100:0/3303426565 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5e5c078f20 msgr2=0x7f5e5c07b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:59.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.349+0000 7f5e7f200640 1 --2- 172.21.15.100:0/3303426565 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5e5c078f20 0x7f5e5c07b430 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c000f60 tx=0x7f5e6c009040 comp rx=0 tx=0).stop 2024-09-17T04:28:59.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.349+0000 7f5e7f200640 1 -- 172.21.15.100:0/3303426565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e780b56e0 msgr2=0x7f5e78156770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:59.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.349+0000 7f5e7f200640 1 --2- 172.21.15.100:0/3303426565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e780b56e0 0x7f5e78156770 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f5e6800c3a0 tx=0x7f5e6800c860 comp rx=0 tx=0).stop 2024-09-17T04:28:59.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.349+0000 7f5e7f200640 1 -- 172.21.15.100:0/3303426565 shutdown_connections 2024-09-17T04:28:59.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.349+0000 7f5e7f200640 1 --2- 172.21.15.100:0/3303426565 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f5e5c078f20 0x7f5e5c07b430 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:59.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.349+0000 7f5e7f200640 1 --2- 172.21.15.100:0/3303426565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e781570c0 0x7f5e7815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:59.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.349+0000 7f5e7f200640 1 --2- 172.21.15.100:0/3303426565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e780b56e0 0x7f5e78156770 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:59.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.349+0000 7f5e7f200640 1 --2- 172.21.15.100:0/3303426565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e780aec80 0x7f5e780c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:59.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.349+0000 7f5e7f200640 1 -- 172.21.15.100:0/3303426565 >> 172.21.15.100:0/3303426565 conn(0x7f5e780b2220 msgr2=0x7f5e780afd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:59.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.349+0000 7f5e7f200640 1 -- 172.21.15.100:0/3303426565 shutdown_connections 2024-09-17T04:28:59.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.349+0000 7f5e7f200640 1 -- 172.21.15.100:0/3303426565 wait complete. 2024-09-17T04:28:59.375 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T04:28:59.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.596+0000 7fe55fa00640 1 -- 172.21.15.100:0/2414511224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe540003a00 msgr2=0x7fe540003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:59.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.596+0000 7fe55fa00640 1 --2- 172.21.15.100:0/2414511224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe540003a00 0x7fe540003e70 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009cf0 tx=0x7fe540005830 comp rx=0 tx=0).stop 2024-09-17T04:28:59.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.596+0000 7fe55da00640 1 -- 172.21.15.100:0/2414511224 reap_dead start 2024-09-17T04:28:59.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.596+0000 7fe55fa00640 1 -- 172.21.15.100:0/2414511224 shutdown_connections 2024-09-17T04:28:59.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.596+0000 7fe55fa00640 1 -- 172.21.15.100:0/2414511224 >> 172.21.15.100:0/2414511224 conn(0x7fe55806d3d0 msgr2=0x7fe5580b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:59.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.596+0000 7fe55fa00640 1 -- 172.21.15.100:0/2414511224 shutdown_connections 2024-09-17T04:28:59.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.596+0000 7fe55fa00640 1 -- 172.21.15.100:0/2414511224 wait complete. 2024-09-17T04:28:59.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.598+0000 7fe55fa00640 1 Processor -- start 2024-09-17T04:28:59.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.598+0000 7fe55fa00640 1 -- start start 2024-09-17T04:28:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.600+0000 7fe55fa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe540003a00 0x7fe5580c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.600+0000 7fe55fa00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5581f3cf0 0x7fe5580b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.600+0000 7fe55fa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5580b21a0 0x7fe5581f67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.600+0000 7fe55fa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe5580adce0 con 0x7fe5580b21a0 2024-09-17T04:28:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.600+0000 7fe55fa00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe5580cca80 con 0x7fe540003a00 2024-09-17T04:28:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.600+0000 7fe55fa00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe5580ade60 con 0x7fe5581f3cf0 2024-09-17T04:28:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.600+0000 7fe55da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5580b21a0 0x7fe5581f67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:59.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.600+0000 7fe55d000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe540003a00 0x7fe5580c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:59.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.600+0000 7fe55da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5580b21a0 0x7fe5581f67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34792/0 (socket says 172.21.15.100:34792) 2024-09-17T04:28:59.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.600+0000 7fe55da00640 1 -- 172.21.15.100:0/4107821808 learned_addr learned my addr 172.21.15.100:0/4107821808 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:28:59.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.600+0000 7fe557e00640 1 --2- 172.21.15.100:0/4107821808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5581f3cf0 0x7fe5580b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:59.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.600+0000 7fe55da00640 1 -- 172.21.15.100:0/4107821808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5581f3cf0 msgr2=0x7fe5580b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:59.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.600+0000 7fe55da00640 1 --2- 172.21.15.100:0/4107821808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5581f3cf0 0x7fe5580b1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:59.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.600+0000 7fe55da00640 1 -- 172.21.15.100:0/4107821808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe540003a00 msgr2=0x7fe5580c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:59.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.600+0000 7fe55da00640 1 --2- 172.21.15.100:0/4107821808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe540003a00 0x7fe5580c83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:59.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.600+0000 7fe55da00640 1 -- 172.21.15.100:0/4107821808 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5580b25b0 con 0x7fe5580b21a0 2024-09-17T04:28:59.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.601+0000 7fe55da00640 1 --2- 172.21.15.100:0/4107821808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5580b21a0 0x7fe5581f67f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe550002230 tx=0x7fe55000ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:59.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.601+0000 7fe554c00640 1 -- 172.21.15.100:0/4107821808 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe550010890 con 0x7fe5580b21a0 2024-09-17T04:28:59.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.601+0000 7fe554c00640 1 -- 172.21.15.100:0/4107821808 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe550017070 con 0x7fe5580b21a0 2024-09-17T04:28:59.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.601+0000 7fe554c00640 1 -- 172.21.15.100:0/4107821808 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe550010dc0 con 0x7fe5580b21a0 2024-09-17T04:28:59.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.601+0000 7fe55fa00640 1 -- 172.21.15.100:0/4107821808 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5580c8d60 con 0x7fe5580b21a0 2024-09-17T04:28:59.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.601+0000 7fe55fa00640 1 -- 172.21.15.100:0/4107821808 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5580c9320 con 0x7fe5580b21a0 2024-09-17T04:28:59.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.602+0000 7fe55fa00640 1 -- 172.21.15.100:0/4107821808 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5580ceb20 con 0x7fe5580b21a0 2024-09-17T04:28:59.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.603+0000 7fe554c00640 1 -- 172.21.15.100:0/4107821808 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fe55001c460 con 0x7fe5580b21a0 2024-09-17T04:28:59.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.604+0000 7fe554c00640 1 --2- 172.21.15.100:0/4107821808 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe538078ef0 0x7fe53807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:28:59.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.604+0000 7fe554c00640 1 -- 172.21.15.100:0/4107821808 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5767+0+0 (secure 0 0 0) 0x7fe55009b600 con 0x7fe5580b21a0 2024-09-17T04:28:59.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.605+0000 7fe55d000640 1 --2- 172.21.15.100:0/4107821808 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe538078ef0 0x7fe53807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:28:59.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.605+0000 7fe55d000640 1 --2- 172.21.15.100:0/4107821808 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe538078ef0 0x7fe53807b400 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe54c034000 tx=0x7fe54c0046c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:28:59.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.609+0000 7fe554c00640 1 -- 172.21.15.100:0/4107821808 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe5580b25b0 con 0x7fe5580b21a0 2024-09-17T04:28:59.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.815+0000 7fe55fa00640 1 -- 172.21.15.100:0/4107821808 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fe5580b2b60 con 0x7fe538078ef0 2024-09-17T04:28:59.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.817+0000 7fe554c00640 1 -- 172.21.15.100:0/4107821808 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fe5580b2b60 con 0x7fe538078ef0 2024-09-17T04:28:59.818 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-09-17T04:28:59.818 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:28:59.818 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephfs" 2024-09-17T04:28:59.818 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:28:59.818 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-09-17T04:28:59.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.820+0000 7fe55fa00640 1 -- 172.21.15.100:0/4107821808 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe538078ef0 msgr2=0x7fe53807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:59.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.820+0000 7fe55fa00640 1 --2- 172.21.15.100:0/4107821808 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe538078ef0 0x7fe53807b400 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe54c034000 tx=0x7fe54c0046c0 comp rx=0 tx=0).stop 2024-09-17T04:28:59.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.820+0000 7fe55fa00640 1 -- 172.21.15.100:0/4107821808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5580b21a0 msgr2=0x7fe5581f67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:28:59.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.820+0000 7fe55fa00640 1 --2- 172.21.15.100:0/4107821808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5580b21a0 0x7fe5581f67f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe550002230 tx=0x7fe55000ea40 comp rx=0 tx=0).stop 2024-09-17T04:28:59.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.820+0000 7fe55fa00640 1 -- 172.21.15.100:0/4107821808 shutdown_connections 2024-09-17T04:28:59.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.820+0000 7fe55fa00640 1 --2- 172.21.15.100:0/4107821808 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fe538078ef0 0x7fe53807b400 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:59.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.820+0000 7fe55fa00640 1 --2- 172.21.15.100:0/4107821808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5580b21a0 0x7fe5581f67f0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:59.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.820+0000 7fe55fa00640 1 --2- 172.21.15.100:0/4107821808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5581f3cf0 0x7fe5580b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:59.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.820+0000 7fe55fa00640 1 --2- 172.21.15.100:0/4107821808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe540003a00 0x7fe5580c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:28:59.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.820+0000 7fe55fa00640 1 -- 172.21.15.100:0/4107821808 >> 172.21.15.100:0/4107821808 conn(0x7fe55806d3d0 msgr2=0x7fe5580747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:28:59.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.820+0000 7fe55fa00640 1 -- 172.21.15.100:0/4107821808 shutdown_connections 2024-09-17T04:28:59.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:28:59.820+0000 7fe55fa00640 1 -- 172.21.15.100:0/4107821808 wait complete. 2024-09-17T04:28:59.846 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T04:29:00.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.068+0000 7f981e400640 1 -- 172.21.15.100:0/1544334510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98180cd610 msgr2=0x7f98180cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:00.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.068+0000 7f981e400640 1 --2- 172.21.15.100:0/1544334510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98180cd610 0x7f98180cda20 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9818072bd0 tx=0x7f981002a5f0 comp rx=0 tx=0).stop 2024-09-17T04:29:00.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.068+0000 7f981e400640 1 -- 172.21.15.100:0/1544334510 shutdown_connections 2024-09-17T04:29:00.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.068+0000 7f981e400640 1 --2- 172.21.15.100:0/1544334510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98180cd610 0x7f98180cda20 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:00.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.068+0000 7f981e400640 1 --2- 172.21.15.100:0/1544334510 >> v2:172.21.15.184:3300/0 conn(0x7f9818145450 0x7f981806f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:00.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.068+0000 7f981e400640 1 -- 172.21.15.100:0/1544334510 >> 172.21.15.100:0/1544334510 conn(0x7f981806d1c0 msgr2=0x7f981806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:00.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.069+0000 7f981e400640 1 -- 172.21.15.100:0/1544334510 shutdown_connections 2024-09-17T04:29:00.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.069+0000 7f981e400640 1 -- 172.21.15.100:0/1544334510 wait complete. 2024-09-17T04:29:00.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.071+0000 7f981e400640 1 Processor -- start 2024-09-17T04:29:00.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.071+0000 7f981e400640 1 -- start start 2024-09-17T04:29:00.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.072+0000 7f981e400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98180735e0 0x7f98181f68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:00.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.072+0000 7f981e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98180cd610 0x7f9818157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:00.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.072+0000 7f981e400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9818145450 0x7f9818143af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:00.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.072+0000 7f981e400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f98180ce2f0 con 0x7f98180cd610 2024-09-17T04:29:00.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.072+0000 7f981e400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f98180ce170 con 0x7f98180735e0 2024-09-17T04:29:00.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.072+0000 7f981e400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f98180ce470 con 0x7f9818145450 2024-09-17T04:29:00.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.073+0000 7f980ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98180cd610 0x7f9818157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:00.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.073+0000 7f980ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98180cd610 0x7f9818157070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34800/0 (socket says 172.21.15.100:34800) 2024-09-17T04:29:00.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.073+0000 7f980ea00640 1 -- 172.21.15.100:0/965091060 learned_addr learned my addr 172.21.15.100:0/965091060 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:00.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.073+0000 7f9817e00640 1 --2- 172.21.15.100:0/965091060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9818145450 0x7f9818143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:00.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.073+0000 7f9817400640 1 --2- 172.21.15.100:0/965091060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98180735e0 0x7f98181f68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:00.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.073+0000 7f980ea00640 1 -- 172.21.15.100:0/965091060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9818145450 msgr2=0x7f9818143af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:00.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.073+0000 7f980ea00640 1 --2- 172.21.15.100:0/965091060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9818145450 0x7f9818143af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:00.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.073+0000 7f980ea00640 1 -- 172.21.15.100:0/965091060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98180735e0 msgr2=0x7f98181f68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:00.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.073+0000 7f980ea00640 1 --2- 172.21.15.100:0/965091060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98180735e0 0x7f98181f68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:00.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.073+0000 7f980ea00640 1 -- 172.21.15.100:0/965091060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98181575b0 con 0x7f98180cd610 2024-09-17T04:29:00.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.073+0000 7f9817e00640 1 --2- 172.21.15.100:0/965091060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9818145450 0x7f9818143af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:29:00.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.073+0000 7f9817400640 1 --2- 172.21.15.100:0/965091060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98180735e0 0x7f98181f68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:29:00.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.073+0000 7f980ea00640 1 --2- 172.21.15.100:0/965091060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98180cd610 0x7f9818157070 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f980000c740 tx=0x7f980000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:00.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.073+0000 7f980fe00640 1 -- 172.21.15.100:0/965091060 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9800017070 con 0x7f98180cd610 2024-09-17T04:29:00.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.074+0000 7f980fe00640 1 -- 172.21.15.100:0/965091060 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f980000ce50 con 0x7f98180cd610 2024-09-17T04:29:00.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.074+0000 7f980fe00640 1 -- 172.21.15.100:0/965091060 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98000052d0 con 0x7f98180cd610 2024-09-17T04:29:00.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.074+0000 7f981e400640 1 -- 172.21.15.100:0/965091060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98181f71b0 con 0x7f98180cd610 2024-09-17T04:29:00.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.074+0000 7f981e400640 1 -- 172.21.15.100:0/965091060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98181f7770 con 0x7f98180cd610 2024-09-17T04:29:00.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.075+0000 7f981e400640 1 -- 172.21.15.100:0/965091060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98180cda20 con 0x7f98180cd610 2024-09-17T04:29:00.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.076+0000 7f980fe00640 1 -- 172.21.15.100:0/965091060 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f980001a070 con 0x7f98180cd610 2024-09-17T04:29:00.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.077+0000 7f980fe00640 1 --2- 172.21.15.100:0/965091060 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f97f4078f20 0x7f97f407b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:00.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.077+0000 7f980fe00640 1 -- 172.21.15.100:0/965091060 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5767+0+0 (secure 0 0 0) 0x7f980009aa30 con 0x7f98180cd610 2024-09-17T04:29:00.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.078+0000 7f9817400640 1 --2- 172.21.15.100:0/965091060 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f97f4078f20 0x7f97f407b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:00.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.078+0000 7f9817400640 1 --2- 172.21.15.100:0/965091060 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f97f4078f20 0x7f97f407b430 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f980800ba60 tx=0x7f9808005cf0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:00.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.082+0000 7f980fe00640 1 -- 172.21.15.100:0/965091060 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f98181575b0 con 0x7f98180cd610 2024-09-17T04:29:00.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.269+0000 7f981e400640 1 -- 172.21.15.100:0/965091060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f981813ec30 con 0x7f98180cd610 2024-09-17T04:29:00.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.270+0000 7f980fe00640 1 -- 172.21.15.100:0/965091060 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v38)=0 v38) ==== 122+0+0 (secure 0 0 0) 0x7f98000628e0 con 0x7f98180cd610 2024-09-17T04:29:00.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.272+0000 7f981e400640 1 -- 172.21.15.100:0/965091060 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f97f4078f20 msgr2=0x7f97f407b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:00.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.272+0000 7f981e400640 1 --2- 172.21.15.100:0/965091060 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f97f4078f20 0x7f97f407b430 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f980800ba60 tx=0x7f9808005cf0 comp rx=0 tx=0).stop 2024-09-17T04:29:00.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.272+0000 7f981e400640 1 -- 172.21.15.100:0/965091060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98180cd610 msgr2=0x7f9818157070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:00.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.272+0000 7f981e400640 1 --2- 172.21.15.100:0/965091060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98180cd610 0x7f9818157070 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f980000c740 tx=0x7f980000cc00 comp rx=0 tx=0).stop 2024-09-17T04:29:00.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.272+0000 7f981e400640 1 -- 172.21.15.100:0/965091060 shutdown_connections 2024-09-17T04:29:00.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.272+0000 7f981e400640 1 --2- 172.21.15.100:0/965091060 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f97f4078f20 0x7f97f407b430 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:00.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.272+0000 7f981e400640 1 --2- 172.21.15.100:0/965091060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9818145450 0x7f9818143af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:00.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.272+0000 7f981e400640 1 --2- 172.21.15.100:0/965091060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98180cd610 0x7f9818157070 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:00.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.272+0000 7f981e400640 1 --2- 172.21.15.100:0/965091060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98180735e0 0x7f98181f68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:00.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.272+0000 7f981e400640 1 -- 172.21.15.100:0/965091060 >> 172.21.15.100:0/965091060 conn(0x7f981806d1c0 msgr2=0x7f981815f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:00.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.272+0000 7f981e400640 1 -- 172.21.15.100:0/965091060 shutdown_connections 2024-09-17T04:29:00.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.273+0000 7f981e400640 1 -- 172.21.15.100:0/965091060 wait complete. 2024-09-17T04:29:00.296 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:29:00.296 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:29:00.331 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:29:00.331 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:29:00.408 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:29:00.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.668+0000 7fc8a2a00640 1 -- 172.21.15.100:0/2381873416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc880003a30 msgr2=0x7fc880003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:00.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.668+0000 7fc8a2a00640 1 --2- 172.21.15.100:0/2381873416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc880003a30 0x7fc880003ea0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009b20 tx=0x7fc880005830 comp rx=0 tx=0).stop 2024-09-17T04:29:00.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.668+0000 7fc8a0a00640 1 -- 172.21.15.100:0/2381873416 reap_dead start 2024-09-17T04:29:00.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.668+0000 7fc8a2a00640 1 -- 172.21.15.100:0/2381873416 shutdown_connections 2024-09-17T04:29:00.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.668+0000 7fc8a2a00640 1 -- 172.21.15.100:0/2381873416 >> 172.21.15.100:0/2381873416 conn(0x7fc89c06d3d0 msgr2=0x7fc89c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:00.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.668+0000 7fc8a2a00640 1 -- 172.21.15.100:0/2381873416 shutdown_connections 2024-09-17T04:29:00.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.668+0000 7fc8a2a00640 1 -- 172.21.15.100:0/2381873416 wait complete. 2024-09-17T04:29:00.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.671+0000 7fc8a2a00640 1 Processor -- start 2024-09-17T04:29:00.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.671+0000 7fc8a2a00640 1 -- start start 2024-09-17T04:29:00.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.672+0000 7fc8a2a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc880003a30 0x7fc89c0b3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:00.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.672+0000 7fc8a2a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc89c15c4d0 0x7fc89c155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:00.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.672+0000 7fc8a2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc89c15f4b0 0x7fc89c15f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:00.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.672+0000 7fc8a2a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c0af820 con 0x7fc89c15f4b0 2024-09-17T04:29:00.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.672+0000 7fc8a2a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c0af6a0 con 0x7fc89c15c4d0 2024-09-17T04:29:00.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.672+0000 7fc8a2a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc89c0af9a0 con 0x7fc880003a30 2024-09-17T04:29:00.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.672+0000 7fc8a0a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc89c15f4b0 0x7fc89c15f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:00.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.673+0000 7fc8a0a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc89c15f4b0 0x7fc89c15f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34814/0 (socket says 172.21.15.100:34814) 2024-09-17T04:29:00.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.673+0000 7fc8a0a00640 1 -- 172.21.15.100:0/1315990583 learned_addr learned my addr 172.21.15.100:0/1315990583 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:00.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.673+0000 7fc89be00640 1 --2- 172.21.15.100:0/1315990583 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc880003a30 0x7fc89c0b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:00.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.673+0000 7fc893e00640 1 --2- 172.21.15.100:0/1315990583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc89c15c4d0 0x7fc89c155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:00.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.673+0000 7fc8a0a00640 1 -- 172.21.15.100:0/1315990583 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc880003a30 msgr2=0x7fc89c0b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:00.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.673+0000 7fc8a0a00640 1 --2- 172.21.15.100:0/1315990583 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc880003a30 0x7fc89c0b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:00.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.673+0000 7fc8a0a00640 1 -- 172.21.15.100:0/1315990583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc89c15c4d0 msgr2=0x7fc89c155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:00.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.673+0000 7fc8a0a00640 1 --2- 172.21.15.100:0/1315990583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc89c15c4d0 0x7fc89c155400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:00.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.673+0000 7fc8a0a00640 1 -- 172.21.15.100:0/1315990583 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc89c15e410 con 0x7fc89c15f4b0 2024-09-17T04:29:00.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.673+0000 7fc8a0a00640 1 --2- 172.21.15.100:0/1315990583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc89c15f4b0 0x7fc89c15f8c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc894001330 tx=0x7fc89400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:00.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.674+0000 7fc898c00640 1 -- 172.21.15.100:0/1315990583 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc894010890 con 0x7fc89c15f4b0 2024-09-17T04:29:00.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.674+0000 7fc898c00640 1 -- 172.21.15.100:0/1315990583 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc894017070 con 0x7fc89c15f4b0 2024-09-17T04:29:00.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.674+0000 7fc898c00640 1 -- 172.21.15.100:0/1315990583 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc894010e00 con 0x7fc89c15f4b0 2024-09-17T04:29:00.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.674+0000 7fc8a2a00640 1 -- 172.21.15.100:0/1315990583 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc89c15e6f0 con 0x7fc89c15f4b0 2024-09-17T04:29:00.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.674+0000 7fc8a2a00640 1 -- 172.21.15.100:0/1315990583 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc89c15ecb0 con 0x7fc89c15f4b0 2024-09-17T04:29:00.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.675+0000 7fc8a2a00640 1 -- 172.21.15.100:0/1315990583 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc89c0ad1c0 con 0x7fc89c15f4b0 2024-09-17T04:29:00.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.676+0000 7fc898c00640 1 -- 172.21.15.100:0/1315990583 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fc89401c460 con 0x7fc89c15f4b0 2024-09-17T04:29:00.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.677+0000 7fc898c00640 1 --2- 172.21.15.100:0/1315990583 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc878078ef0 0x7fc87807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:00.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.677+0000 7fc898c00640 1 -- 172.21.15.100:0/1315990583 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5767+0+0 (secure 0 0 0) 0x7fc89409b550 con 0x7fc89c15f4b0 2024-09-17T04:29:00.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.677+0000 7fc89be00640 1 --2- 172.21.15.100:0/1315990583 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc878078ef0 0x7fc87807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:00.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.678+0000 7fc89be00640 1 --2- 172.21.15.100:0/1315990583 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc878078ef0 0x7fc87807b400 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc88c035000 tx=0x7fc88c02d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:00.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.681+0000 7fc898c00640 1 -- 172.21.15.100:0/1315990583 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc894062de0 con 0x7fc89c15f4b0 2024-09-17T04:29:00.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.880+0000 7fc8a2a00640 1 -- 172.21.15.100:0/1315990583 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc89c15cd20 con 0x7fc89c15f4b0 2024-09-17T04:29:00.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.881+0000 7fc898c00640 1 -- 172.21.15.100:0/1315990583 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v75) ==== 80+0+0 (secure 0 0 0) 0x7fc894067810 con 0x7fc89c15f4b0 2024-09-17T04:29:00.881 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-09-17T04:29:00.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.883+0000 7fc8a2a00640 1 -- 172.21.15.100:0/1315990583 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc878078ef0 msgr2=0x7fc87807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:00.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.883+0000 7fc8a2a00640 1 --2- 172.21.15.100:0/1315990583 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc878078ef0 0x7fc87807b400 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc88c035000 tx=0x7fc88c02d040 comp rx=0 tx=0).stop 2024-09-17T04:29:00.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.883+0000 7fc8a2a00640 1 -- 172.21.15.100:0/1315990583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc89c15f4b0 msgr2=0x7fc89c15f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:00.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.883+0000 7fc8a2a00640 1 --2- 172.21.15.100:0/1315990583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc89c15f4b0 0x7fc89c15f8c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc894001330 tx=0x7fc89400ea10 comp rx=0 tx=0).stop 2024-09-17T04:29:00.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.883+0000 7fc8a2a00640 1 -- 172.21.15.100:0/1315990583 shutdown_connections 2024-09-17T04:29:00.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.883+0000 7fc8a2a00640 1 --2- 172.21.15.100:0/1315990583 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fc878078ef0 0x7fc87807b400 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:00.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.883+0000 7fc8a2a00640 1 --2- 172.21.15.100:0/1315990583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc89c15f4b0 0x7fc89c15f8c0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:00.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.883+0000 7fc8a2a00640 1 --2- 172.21.15.100:0/1315990583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc89c15c4d0 0x7fc89c155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:00.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.883+0000 7fc8a2a00640 1 --2- 172.21.15.100:0/1315990583 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc880003a30 0x7fc89c0b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:00.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.883+0000 7fc8a2a00640 1 -- 172.21.15.100:0/1315990583 >> 172.21.15.100:0/1315990583 conn(0x7fc89c06d3d0 msgr2=0x7fc89c0b1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:00.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.883+0000 7fc8a2a00640 1 -- 172.21.15.100:0/1315990583 shutdown_connections 2024-09-17T04:29:00.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:00.883+0000 7fc8a2a00640 1 -- 172.21.15.100:0/1315990583 wait complete. 2024-09-17T04:29:00.906 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T04:29:00.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:29:00.906 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T04:29:00.969 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-09-17T04:29:00.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:29:00.969 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T04:29:01.006 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:29:01.006 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:29:01.046 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:29:01.046 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:29:01.124 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:29:01.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.345+0000 7f4e09c00640 1 -- 172.21.15.100:0/3636324084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e04070660 msgr2=0x7f4e040af6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:01.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.345+0000 7f4e09c00640 1 --2- 172.21.15.100:0/3636324084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e04070660 0x7f4e040af6a0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f4e04072b80 tx=0x7f4df402a440 comp rx=0 tx=0).stop 2024-09-17T04:29:01.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.345+0000 7f4e09c00640 1 -- 172.21.15.100:0/3636324084 shutdown_connections 2024-09-17T04:29:01.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.345+0000 7f4e09c00640 1 --2- 172.21.15.100:0/3636324084 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e040ad1c0 0x7f4e040ad5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:01.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.345+0000 7f4e09c00640 1 --2- 172.21.15.100:0/3636324084 >> v2:172.21.15.184:3300/0 conn(0x7f4e040ce490 0x7f4e040ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:01.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.345+0000 7f4e09c00640 1 --2- 172.21.15.100:0/3636324084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e04070660 0x7f4e040af6a0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:01.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.345+0000 7f4e09c00640 1 -- 172.21.15.100:0/3636324084 >> 172.21.15.100:0/3636324084 conn(0x7f4e0406d3d0 msgr2=0x7f4e040b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:01.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.345+0000 7f4e09c00640 1 -- 172.21.15.100:0/3636324084 shutdown_connections 2024-09-17T04:29:01.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.345+0000 7f4e09c00640 1 -- 172.21.15.100:0/3636324084 wait complete. 2024-09-17T04:29:01.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.347+0000 7f4e09c00640 1 Processor -- start 2024-09-17T04:29:01.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.347+0000 7f4e09c00640 1 -- start start 2024-09-17T04:29:01.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.349+0000 7f4e09c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e04070660 0x7f4e040c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:01.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.349+0000 7f4e09c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e040ad1c0 0x7f4e041f66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:01.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.349+0000 7f4e09c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e040ce490 0x7f4e041f5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:01.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.349+0000 7f4e09c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4e040adea0 con 0x7f4e040ce490 2024-09-17T04:29:01.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.349+0000 7f4e09c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4e040add20 con 0x7f4e04070660 2024-09-17T04:29:01.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.349+0000 7f4e09c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4e040ca940 con 0x7f4e040ad1c0 2024-09-17T04:29:01.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.349+0000 7f4e03400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e040ce490 0x7f4e041f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:01.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.349+0000 7f4e03400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e040ce490 0x7f4e041f5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34842/0 (socket says 172.21.15.100:34842) 2024-09-17T04:29:01.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.349+0000 7f4e03400640 1 -- 172.21.15.100:0/1256639638 learned_addr learned my addr 172.21.15.100:0/1256639638 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:01.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.349+0000 7f4e02a00640 1 --2- 172.21.15.100:0/1256639638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e04070660 0x7f4e040c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:01.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.349+0000 7f4dfa000640 1 --2- 172.21.15.100:0/1256639638 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e040ad1c0 0x7f4e041f66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:01.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.349+0000 7f4e03400640 1 -- 172.21.15.100:0/1256639638 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e040ad1c0 msgr2=0x7f4e041f66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:01.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.349+0000 7f4e03400640 1 --2- 172.21.15.100:0/1256639638 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e040ad1c0 0x7f4e041f66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:01.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.349+0000 7f4e03400640 1 -- 172.21.15.100:0/1256639638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e04070660 msgr2=0x7f4e040c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:01.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.349+0000 7f4e03400640 1 --2- 172.21.15.100:0/1256639638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e04070660 0x7f4e040c9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:01.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.349+0000 7f4e03400640 1 -- 172.21.15.100:0/1256639638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0414cdd0 con 0x7f4e040ce490 2024-09-17T04:29:01.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.349+0000 7f4e02a00640 1 --2- 172.21.15.100:0/1256639638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e04070660 0x7f4e040c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:01.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.350+0000 7f4e03400640 1 --2- 172.21.15.100:0/1256639638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e040ce490 0x7f4e041f5090 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc005f40 tx=0x7f4dfc00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:01.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.350+0000 7f4dfb400640 1 -- 172.21.15.100:0/1256639638 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc00cd80 con 0x7f4e040ce490 2024-09-17T04:29:01.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.350+0000 7f4dfb400640 1 -- 172.21.15.100:0/1256639638 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4dfc004b20 con 0x7f4e040ce490 2024-09-17T04:29:01.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.350+0000 7f4dfb400640 1 -- 172.21.15.100:0/1256639638 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc005020 con 0x7f4e040ce490 2024-09-17T04:29:01.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.350+0000 7f4e09c00640 1 -- 172.21.15.100:0/1256639638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e0414d070 con 0x7f4e040ce490 2024-09-17T04:29:01.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.350+0000 7f4e09c00640 1 -- 172.21.15.100:0/1256639638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e040ca190 con 0x7f4e040ce490 2024-09-17T04:29:01.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.351+0000 7f4e09c00640 1 -- 172.21.15.100:0/1256639638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e040ad5d0 con 0x7f4e040ce490 2024-09-17T04:29:01.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.352+0000 7f4dfb400640 1 -- 172.21.15.100:0/1256639638 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f4dfc0051c0 con 0x7f4e040ce490 2024-09-17T04:29:01.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.354+0000 7f4dfb400640 1 --2- 172.21.15.100:0/1256639638 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4de0078ef0 0x7f4de007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:01.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.354+0000 7f4dfb400640 1 -- 172.21.15.100:0/1256639638 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5767+0+0 (secure 0 0 0) 0x7f4dfc09c040 con 0x7f4e040ce490 2024-09-17T04:29:01.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.354+0000 7f4e02a00640 1 --2- 172.21.15.100:0/1256639638 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4de0078ef0 0x7f4de007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:01.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.355+0000 7f4e02a00640 1 --2- 172.21.15.100:0/1256639638 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4de0078ef0 0x7f4de007b400 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4df4033000 tx=0x7f4df4004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:01.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.358+0000 7f4dfb400640 1 -- 172.21.15.100:0/1256639638 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4dfc063e70 con 0x7f4e040ce490 2024-09-17T04:29:01.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.552+0000 7f4e09c00640 1 -- 172.21.15.100:0/1256639638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4e040c9f20 con 0x7f4e040ce490 2024-09-17T04:29:01.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.552+0000 7f4dfb400640 1 -- 172.21.15.100:0/1256639638 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v75) ==== 80+0+0 (secure 0 0 0) 0x7f4dfc063770 con 0x7f4e040ce490 2024-09-17T04:29:01.553 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-09-17T04:29:01.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.554+0000 7f4e09c00640 1 -- 172.21.15.100:0/1256639638 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4de0078ef0 msgr2=0x7f4de007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:01.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.554+0000 7f4e09c00640 1 --2- 172.21.15.100:0/1256639638 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4de0078ef0 0x7f4de007b400 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4df4033000 tx=0x7f4df4004730 comp rx=0 tx=0).stop 2024-09-17T04:29:01.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.554+0000 7f4e09c00640 1 -- 172.21.15.100:0/1256639638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e040ce490 msgr2=0x7f4e041f5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:01.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.554+0000 7f4e09c00640 1 --2- 172.21.15.100:0/1256639638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e040ce490 0x7f4e041f5090 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc005f40 tx=0x7f4dfc00ee60 comp rx=0 tx=0).stop 2024-09-17T04:29:01.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.555+0000 7f4e09c00640 1 -- 172.21.15.100:0/1256639638 shutdown_connections 2024-09-17T04:29:01.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.555+0000 7f4e09c00640 1 --2- 172.21.15.100:0/1256639638 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f4de0078ef0 0x7f4de007b400 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:01.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.555+0000 7f4e09c00640 1 --2- 172.21.15.100:0/1256639638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e040ce490 0x7f4e041f5090 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:01.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.555+0000 7f4e09c00640 1 --2- 172.21.15.100:0/1256639638 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e040ad1c0 0x7f4e041f66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:01.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.555+0000 7f4e09c00640 1 --2- 172.21.15.100:0/1256639638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e04070660 0x7f4e040c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:01.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.555+0000 7f4e09c00640 1 -- 172.21.15.100:0/1256639638 >> 172.21.15.100:0/1256639638 conn(0x7f4e0406d3d0 msgr2=0x7f4e04073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:01.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.555+0000 7f4e09c00640 1 -- 172.21.15.100:0/1256639638 shutdown_connections 2024-09-17T04:29:01.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.555+0000 7f4e09c00640 1 -- 172.21.15.100:0/1256639638 wait complete. 2024-09-17T04:29:01.580 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-17T04:29:01.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:29:01.581 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-17T04:29:01.650 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2024-09-17T04:29:01.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:29:01.650 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T04:29:01.687 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:29:01.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.920+0000 7f880ec00640 1 -- 172.21.15.100:0/3953163922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f880815d760 msgr2=0x7f88081535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:01.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.920+0000 7f880ec00640 1 --2- 172.21.15.100:0/3953163922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f880815d760 0x7f88081535c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009830 tx=0x7f87f802a4b0 comp rx=0 tx=0).stop 2024-09-17T04:29:01.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.920+0000 7f880ec00640 1 -- 172.21.15.100:0/3953163922 shutdown_connections 2024-09-17T04:29:01.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.920+0000 7f880ec00640 1 --2- 172.21.15.100:0/3953163922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f880815d760 0x7f88081535c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:01.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.920+0000 7f880ec00640 1 -- 172.21.15.100:0/3953163922 >> 172.21.15.100:0/3953163922 conn(0x7f880806d730 msgr2=0x7f88080b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:01.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.920+0000 7f880ec00640 1 -- 172.21.15.100:0/3953163922 shutdown_connections 2024-09-17T04:29:01.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.920+0000 7f880ec00640 1 -- 172.21.15.100:0/3953163922 wait complete. 2024-09-17T04:29:01.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.922+0000 7f880ec00640 1 Processor -- start 2024-09-17T04:29:01.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.922+0000 7f880ec00640 1 -- start start 2024-09-17T04:29:01.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.923+0000 7f880ec00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88080ad0e0 0x7f8808099a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:01.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.923+0000 7f880ec00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88080b3550 0x7f88080b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:01.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.923+0000 7f880ec00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f880815d760 0x7f8808150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:01.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.923+0000 7f880ec00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f880806f240 con 0x7f880815d760 2024-09-17T04:29:01.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.923+0000 7f880ec00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f88080b4e30 con 0x7f88080b3550 2024-09-17T04:29:01.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.923+0000 7f880ec00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f88080b4070 con 0x7f88080ad0e0 2024-09-17T04:29:01.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.924+0000 7f880cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f880815d760 0x7f8808150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:01.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.924+0000 7f880cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f880815d760 0x7f8808150ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34858/0 (socket says 172.21.15.100:34858) 2024-09-17T04:29:01.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.924+0000 7f880cc00640 1 -- 172.21.15.100:0/4091659301 learned_addr learned my addr 172.21.15.100:0/4091659301 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:01.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.924+0000 7f8807e00640 1 --2- 172.21.15.100:0/4091659301 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88080ad0e0 0x7f8808099a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:01.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.924+0000 7f8807400640 1 --2- 172.21.15.100:0/4091659301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88080b3550 0x7f88080b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:01.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.924+0000 7f880cc00640 1 -- 172.21.15.100:0/4091659301 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88080ad0e0 msgr2=0x7f8808099a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:01.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.924+0000 7f880cc00640 1 --2- 172.21.15.100:0/4091659301 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88080ad0e0 0x7f8808099a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:01.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.924+0000 7f880cc00640 1 -- 172.21.15.100:0/4091659301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88080b3550 msgr2=0x7f88080b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:01.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.924+0000 7f880cc00640 1 --2- 172.21.15.100:0/4091659301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88080b3550 0x7f88080b1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:01.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.924+0000 7f880cc00640 1 -- 172.21.15.100:0/4091659301 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88081f93a0 con 0x7f880815d760 2024-09-17T04:29:01.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.924+0000 7f8807400640 1 --2- 172.21.15.100:0/4091659301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88080b3550 0x7f88080b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:01.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.925+0000 7f880cc00640 1 --2- 172.21.15.100:0/4091659301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f880815d760 0x7f8808150ed0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f880000c700 tx=0x7f880000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:01.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.925+0000 7f87ffe00640 1 -- 172.21.15.100:0/4091659301 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800003b70 con 0x7f880815d760 2024-09-17T04:29:01.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.925+0000 7f87ffe00640 1 -- 172.21.15.100:0/4091659301 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8800003d10 con 0x7f880815d760 2024-09-17T04:29:01.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.925+0000 7f880ec00640 1 -- 172.21.15.100:0/4091659301 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88081f9680 con 0x7f880815d760 2024-09-17T04:29:01.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.925+0000 7f87ffe00640 1 -- 172.21.15.100:0/4091659301 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880000aab0 con 0x7f880815d760 2024-09-17T04:29:01.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.925+0000 7f880ec00640 1 -- 172.21.15.100:0/4091659301 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88081f9c40 con 0x7f880815d760 2024-09-17T04:29:01.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.926+0000 7f880ec00640 1 -- 172.21.15.100:0/4091659301 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88081535c0 con 0x7f880815d760 2024-09-17T04:29:01.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.927+0000 7f87ffe00640 1 -- 172.21.15.100:0/4091659301 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f880000ac50 con 0x7f880815d760 2024-09-17T04:29:01.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.928+0000 7f87ffe00640 1 --2- 172.21.15.100:0/4091659301 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f87e8078ef0 0x7f87e807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:01.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.928+0000 7f87ffe00640 1 -- 172.21.15.100:0/4091659301 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5767+0+0 (secure 0 0 0) 0x7f880009b280 con 0x7f880815d760 2024-09-17T04:29:01.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.929+0000 7f8807e00640 1 --2- 172.21.15.100:0/4091659301 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f87e8078ef0 0x7f87e807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:01.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.929+0000 7f8807e00640 1 --2- 172.21.15.100:0/4091659301 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f87e8078ef0 0x7f87e807b400 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f87f8034000 tx=0x7f87f8004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:01.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:01.933+0000 7f87ffe00640 1 -- 172.21.15.100:0/4091659301 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8800062fa0 con 0x7f880815d760 2024-09-17T04:29:02.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.153+0000 7f880ec00640 1 -- 172.21.15.100:0/4091659301 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88081f9130 con 0x7f880815d760 2024-09-17T04:29:02.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.154+0000 7f87ffe00640 1 -- 172.21.15.100:0/4091659301 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5237 (secure 0 0 0) 0x7f88000628a0 con 0x7f880815d760 2024-09-17T04:29:02.154 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:02.154 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":48,"btime":"2024-09-17T04:28:59:346395+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5513,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2569013403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2569013403},{"type":"v1","addr":"172.21.15.184:6835","nonce":2569013403}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5519,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3475403612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3475403612},{"type":"v1","addr":"172.21.15.184:6837","nonce":3475403612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6156,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2177833369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2177833369},{"type":"v1","addr":"172.21.15.100:6833","nonce":2177833369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:28:43.648961+0000","modified":"2024-09-17T04:28:46.665573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6162":{"gid":6162,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6835/2913442826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2913442826},{"type":"v1","addr":"172.21.15.100:6835","nonce":2913442826}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6162,"qdb_cluster":[6162]},"id":5}]} 2024-09-17T04:29:02.155 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 48 2024-09-17T04:29:02.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.156+0000 7f880ec00640 1 -- 172.21.15.100:0/4091659301 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f87e8078ef0 msgr2=0x7f87e807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:02.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.156+0000 7f880ec00640 1 --2- 172.21.15.100:0/4091659301 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f87e8078ef0 0x7f87e807b400 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f87f8034000 tx=0x7f87f8004730 comp rx=0 tx=0).stop 2024-09-17T04:29:02.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.156+0000 7f880ec00640 1 -- 172.21.15.100:0/4091659301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f880815d760 msgr2=0x7f8808150ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:02.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.156+0000 7f880ec00640 1 --2- 172.21.15.100:0/4091659301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f880815d760 0x7f8808150ed0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f880000c700 tx=0x7f880000cbc0 comp rx=0 tx=0).stop 2024-09-17T04:29:02.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.156+0000 7f880ec00640 1 -- 172.21.15.100:0/4091659301 shutdown_connections 2024-09-17T04:29:02.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.156+0000 7f880ec00640 1 --2- 172.21.15.100:0/4091659301 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f87e8078ef0 0x7f87e807b400 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:02.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.156+0000 7f880ec00640 1 --2- 172.21.15.100:0/4091659301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f880815d760 0x7f8808150ed0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:02.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.156+0000 7f880ec00640 1 --2- 172.21.15.100:0/4091659301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88080b3550 0x7f88080b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:02.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.156+0000 7f880ec00640 1 --2- 172.21.15.100:0/4091659301 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88080ad0e0 0x7f8808099a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:02.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.156+0000 7f880ec00640 1 -- 172.21.15.100:0/4091659301 >> 172.21.15.100:0/4091659301 conn(0x7f880806d730 msgr2=0x7f88081602a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:02.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.156+0000 7f880ec00640 1 -- 172.21.15.100:0/4091659301 shutdown_connections 2024-09-17T04:29:02.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.156+0000 7f880ec00640 1 -- 172.21.15.100:0/4091659301 wait complete. 2024-09-17T04:29:02.179 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-17T04:29:02.422 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-09-17T04:29:02.422 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-17T04:29:02.422 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol68590 2024-09-17T04:29:02.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.644+0000 7f093ce00640 1 -- 172.21.15.100:0/3965204474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f093806ddb0 msgr2=0x7f093806e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:02.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.644+0000 7f093ce00640 1 --2- 172.21.15.100:0/3965204474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f093806ddb0 0x7f093806e1c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f09380733d0 tx=0x7f093002a980 comp rx=0 tx=0).stop 2024-09-17T04:29:02.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.644+0000 7f093ce00640 1 -- 172.21.15.100:0/3965204474 shutdown_connections 2024-09-17T04:29:02.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.644+0000 7f093ce00640 1 --2- 172.21.15.100:0/3965204474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f093806ddb0 0x7f093806e1c0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:02.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.644+0000 7f093ce00640 1 --2- 172.21.15.100:0/3965204474 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0938153950 0x7f09380cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:02.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.644+0000 7f093ce00640 1 --2- 172.21.15.100:0/3965204474 >> v2:172.21.15.184:3300/0 conn(0x7f093806f960 0x7f09380747e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:02.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.644+0000 7f093ce00640 1 -- 172.21.15.100:0/3965204474 >> 172.21.15.100:0/3965204474 conn(0x7f0938073920 msgr2=0x7f0938073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:02.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.644+0000 7f093ce00640 1 -- 172.21.15.100:0/3965204474 shutdown_connections 2024-09-17T04:29:02.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.644+0000 7f093ce00640 1 -- 172.21.15.100:0/3965204474 wait complete. 2024-09-17T04:29:02.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.647+0000 7f093ce00640 1 Processor -- start 2024-09-17T04:29:02.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.647+0000 7f093ce00640 1 -- start start 2024-09-17T04:29:02.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.648+0000 7f093ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f093806ddb0 0x7f093815b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:02.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.648+0000 7f093ce00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f093806f960 0x7f093814cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:02.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.648+0000 7f093ce00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0938153950 0x7f0938158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:02.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.648+0000 7f093ce00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0938160770 con 0x7f093806ddb0 2024-09-17T04:29:02.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.648+0000 7f093ce00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f09381605f0 con 0x7f093806f960 2024-09-17T04:29:02.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.648+0000 7f093ce00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f09381608f0 con 0x7f0938153950 2024-09-17T04:29:02.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.648+0000 7f0935e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f093806ddb0 0x7f093815b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:02.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.648+0000 7f0935e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f093806ddb0 0x7f093815b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34878/0 (socket says 172.21.15.100:34878) 2024-09-17T04:29:02.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.648+0000 7f0935e00640 1 -- 172.21.15.100:0/1743493493 learned_addr learned my addr 172.21.15.100:0/1743493493 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:02.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.648+0000 7f0935e00640 1 -- 172.21.15.100:0/1743493493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0938153950 msgr2=0x7f0938158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:02.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.648+0000 7f0935e00640 1 --2- 172.21.15.100:0/1743493493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0938153950 0x7f0938158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:02.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.648+0000 7f0935e00640 1 -- 172.21.15.100:0/1743493493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f093806f960 msgr2=0x7f093814cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:02.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.648+0000 7f0935e00640 1 --2- 172.21.15.100:0/1743493493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f093806f960 0x7f093814cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:02.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.648+0000 7f0935e00640 1 -- 172.21.15.100:0/1743493493 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093815c1c0 con 0x7f093806ddb0 2024-09-17T04:29:02.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.648+0000 7f0935e00640 1 --2- 172.21.15.100:0/1743493493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f093806ddb0 0x7f093815b890 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f092800e7c0 tx=0x7f092800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:02.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.649+0000 7f092e000640 1 -- 172.21.15.100:0/1743493493 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092800ccc0 con 0x7f093806ddb0 2024-09-17T04:29:02.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.649+0000 7f092e000640 1 -- 172.21.15.100:0/1743493493 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0928010430 con 0x7f093806ddb0 2024-09-17T04:29:02.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.649+0000 7f093ce00640 1 -- 172.21.15.100:0/1743493493 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f093814ff40 con 0x7f093806ddb0 2024-09-17T04:29:02.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.649+0000 7f092e000640 1 -- 172.21.15.100:0/1743493493 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928010860 con 0x7f093806ddb0 2024-09-17T04:29:02.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.649+0000 7f093ce00640 1 -- 172.21.15.100:0/1743493493 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0938150500 con 0x7f093806ddb0 2024-09-17T04:29:02.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.650+0000 7f093ce00640 1 -- 172.21.15.100:0/1743493493 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09380747e0 con 0x7f093806ddb0 2024-09-17T04:29:02.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.651+0000 7f092e000640 1 -- 172.21.15.100:0/1743493493 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f0928010a00 con 0x7f093806ddb0 2024-09-17T04:29:02.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.652+0000 7f092e000640 1 --2- 172.21.15.100:0/1743493493 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0918078ef0 0x7f091807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:02.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.652+0000 7f092e000640 1 -- 172.21.15.100:0/1743493493 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5767+0+0 (secure 0 0 0) 0x7f092809be20 con 0x7f093806ddb0 2024-09-17T04:29:02.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.653+0000 7f0935400640 1 --2- 172.21.15.100:0/1743493493 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0918078ef0 0x7f091807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:02.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.654+0000 7f0935400640 1 --2- 172.21.15.100:0/1743493493 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0918078ef0 0x7f091807b400 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0924005fd0 tx=0x7f0924005ce0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:02.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.656+0000 7f092e000640 1 -- 172.21.15.100:0/1743493493 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0928063cc0 con 0x7f093806ddb0 2024-09-17T04:29:02.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.866+0000 7f093ce00640 1 -- 172.21.15.100:0/1743493493 --> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol68590", "target": ["mon-mgr", ""]}) -- 0x7f093806f640 con 0x7f0918078ef0 2024-09-17T04:29:02.875 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:29:02.875+0000 7fa165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:29:02.875 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:29:02.875+0000 7fa165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:29:02.875 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:29:02.875+0000 7fa165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:29:02.876 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:29:02.875+0000 7fa165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:29:02.876 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:29:02.875+0000 7fa165e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:29:02.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.913+0000 7f092e000640 1 -- 172.21.15.100:0/1743493493 <== mgr.4103 v2:172.21.15.184:6800/4096933233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f093806f640 con 0x7f0918078ef0 2024-09-17T04:29:02.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.916+0000 7f093ce00640 1 -- 172.21.15.100:0/1743493493 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0918078ef0 msgr2=0x7f091807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:02.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.916+0000 7f093ce00640 1 --2- 172.21.15.100:0/1743493493 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0918078ef0 0x7f091807b400 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0924005fd0 tx=0x7f0924005ce0 comp rx=0 tx=0).stop 2024-09-17T04:29:02.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.916+0000 7f093ce00640 1 -- 172.21.15.100:0/1743493493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f093806ddb0 msgr2=0x7f093815b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:02.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.916+0000 7f093ce00640 1 --2- 172.21.15.100:0/1743493493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f093806ddb0 0x7f093815b890 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f092800e7c0 tx=0x7f092800ec80 comp rx=0 tx=0).stop 2024-09-17T04:29:02.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.916+0000 7f093ce00640 1 -- 172.21.15.100:0/1743493493 shutdown_connections 2024-09-17T04:29:02.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.916+0000 7f093ce00640 1 --2- 172.21.15.100:0/1743493493 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f0918078ef0 0x7f091807b400 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:02.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.916+0000 7f093ce00640 1 --2- 172.21.15.100:0/1743493493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0938153950 0x7f0938158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:02.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.916+0000 7f093ce00640 1 --2- 172.21.15.100:0/1743493493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f093806f960 0x7f093814cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:02.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.916+0000 7f093ce00640 1 --2- 172.21.15.100:0/1743493493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f093806ddb0 0x7f093815b890 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:02.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.916+0000 7f093ce00640 1 -- 172.21.15.100:0/1743493493 >> 172.21.15.100:0/1743493493 conn(0x7f0938073920 msgr2=0x7f093815f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:02.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.916+0000 7f093ce00640 1 -- 172.21.15.100:0/1743493493 shutdown_connections 2024-09-17T04:29:02.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:02.916+0000 7f093ce00640 1 -- 172.21.15.100:0/1743493493 wait complete. 2024-09-17T04:29:02.942 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:29:03.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.160+0000 7f87b7600640 1 -- 172.21.15.100:0/2117590002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87b00ad0e0 msgr2=0x7f87b00ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:03.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.160+0000 7f87b7600640 1 --2- 172.21.15.100:0/2117590002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87b00ad0e0 0x7f87b00ad4f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8798009830 tx=0x7f879802a530 comp rx=0 tx=0).stop 2024-09-17T04:29:03.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.160+0000 7f87b7600640 1 -- 172.21.15.100:0/2117590002 shutdown_connections 2024-09-17T04:29:03.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.160+0000 7f87b7600640 1 --2- 172.21.15.100:0/2117590002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87b00ad0e0 0x7f87b00ad4f0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:03.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.160+0000 7f87b7600640 1 -- 172.21.15.100:0/2117590002 >> 172.21.15.100:0/2117590002 conn(0x7f87b006d730 msgr2=0x7f87b00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:03.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.160+0000 7f87b7600640 1 -- 172.21.15.100:0/2117590002 shutdown_connections 2024-09-17T04:29:03.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.160+0000 7f87b7600640 1 -- 172.21.15.100:0/2117590002 wait complete. 2024-09-17T04:29:03.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.162+0000 7f87b7600640 1 Processor -- start 2024-09-17T04:29:03.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.163+0000 7f87b7600640 1 -- start start 2024-09-17T04:29:03.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.164+0000 7f87b7600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87b00ad0e0 0x7f87b0156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:03.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.164+0000 7f87b7600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87b00b3550 0x7f87b009d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:03.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.164+0000 7f87b7600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87b015d760 0x7f87b015b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:03.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.164+0000 7f87b7600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f87b00b4da0 con 0x7f87b015d760 2024-09-17T04:29:03.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.164+0000 7f87b7600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f87b00b4c20 con 0x7f87b00ad0e0 2024-09-17T04:29:03.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.164+0000 7f87b7600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f87b00b4070 con 0x7f87b00b3550 2024-09-17T04:29:03.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.164+0000 7f87b5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87b015d760 0x7f87b015b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:03.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.164+0000 7f87b5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87b015d760 0x7f87b015b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34910/0 (socket says 172.21.15.100:34910) 2024-09-17T04:29:03.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.164+0000 7f87b5600640 1 -- 172.21.15.100:0/2730816283 learned_addr learned my addr 172.21.15.100:0/2730816283 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:03.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.165+0000 7f87a7e00640 1 --2- 172.21.15.100:0/2730816283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87b00b3550 0x7f87b009d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:03.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.165+0000 7f87b4c00640 1 --2- 172.21.15.100:0/2730816283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87b00ad0e0 0x7f87b0156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:03.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.165+0000 7f87b5600640 1 -- 172.21.15.100:0/2730816283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87b00b3550 msgr2=0x7f87b009d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:03.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.165+0000 7f87b5600640 1 --2- 172.21.15.100:0/2730816283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87b00b3550 0x7f87b009d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:03.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.165+0000 7f87b5600640 1 -- 172.21.15.100:0/2730816283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87b00ad0e0 msgr2=0x7f87b0156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:03.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.165+0000 7f87b5600640 1 --2- 172.21.15.100:0/2730816283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87b00ad0e0 0x7f87b0156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:03.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.165+0000 7f87b5600640 1 -- 172.21.15.100:0/2730816283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b01566a0 con 0x7f87b015d760 2024-09-17T04:29:03.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.165+0000 7f87b4c00640 1 --2- 172.21.15.100:0/2730816283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87b00ad0e0 0x7f87b0156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:03.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.165+0000 7f87a7e00640 1 --2- 172.21.15.100:0/2730816283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87b00b3550 0x7f87b009d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:03.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.165+0000 7f87b5600640 1 --2- 172.21.15.100:0/2730816283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87b015d760 0x7f87b015b7f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f87a800c700 tx=0x7f87a800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:03.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.165+0000 7f87ad600640 1 -- 172.21.15.100:0/2730816283 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8003b70 con 0x7f87b015d760 2024-09-17T04:29:03.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.165+0000 7f87ad600640 1 -- 172.21.15.100:0/2730816283 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87a8003d10 con 0x7f87b015d760 2024-09-17T04:29:03.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.165+0000 7f87ad600640 1 -- 172.21.15.100:0/2730816283 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a800aab0 con 0x7f87b015d760 2024-09-17T04:29:03.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.165+0000 7f87b7600640 1 -- 172.21.15.100:0/2730816283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b01f56f0 con 0x7f87b015d760 2024-09-17T04:29:03.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.166+0000 7f87b7600640 1 -- 172.21.15.100:0/2730816283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b01f5cb0 con 0x7f87b015d760 2024-09-17T04:29:03.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.167+0000 7f87b7600640 1 -- 172.21.15.100:0/2730816283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87b009e170 con 0x7f87b015d760 2024-09-17T04:29:03.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.168+0000 7f87ad600640 1 -- 172.21.15.100:0/2730816283 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f87a800ac50 con 0x7f87b015d760 2024-09-17T04:29:03.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.169+0000 7f87ad600640 1 --2- 172.21.15.100:0/2730816283 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f878c078ef0 0x7f878c07b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:03.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.169+0000 7f87ad600640 1 -- 172.21.15.100:0/2730816283 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5767+0+0 (secure 0 0 0) 0x7f87a809b230 con 0x7f87b015d760 2024-09-17T04:29:03.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.169+0000 7f87b4c00640 1 --2- 172.21.15.100:0/2730816283 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f878c078ef0 0x7f878c07b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:03.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.170+0000 7f87b4c00640 1 --2- 172.21.15.100:0/2730816283 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f878c078ef0 0x7f878c07b400 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f87a00013f0 tx=0x7f87a0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:03.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.173+0000 7f87ad600640 1 -- 172.21.15.100:0/2730816283 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f87a80630a0 con 0x7f87b015d760 2024-09-17T04:29:03.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.394+0000 7f87b7600640 1 -- 172.21.15.100:0/2730816283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87b00ad4f0 con 0x7f87b015d760 2024-09-17T04:29:03.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.394+0000 7f87ad600640 1 -- 172.21.15.100:0/2730816283 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5237 (secure 0 0 0) 0x7f87a80629a0 con 0x7f87b015d760 2024-09-17T04:29:03.394 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:03.394 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":48,"btime":"2024-09-17T04:28:59:346395+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5513,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2569013403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2569013403},{"type":"v1","addr":"172.21.15.184:6835","nonce":2569013403}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5519,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3475403612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3475403612},{"type":"v1","addr":"172.21.15.184:6837","nonce":3475403612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6156,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2177833369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2177833369},{"type":"v1","addr":"172.21.15.100:6833","nonce":2177833369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:28:43.648961+0000","modified":"2024-09-17T04:28:46.665573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6162":{"gid":6162,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6835/2913442826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2913442826},{"type":"v1","addr":"172.21.15.100:6835","nonce":2913442826}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6162,"qdb_cluster":[6162]},"id":5}]} 2024-09-17T04:29:03.394 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 48 2024-09-17T04:29:03.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.397+0000 7f87b7600640 1 -- 172.21.15.100:0/2730816283 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f878c078ef0 msgr2=0x7f878c07b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:03.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.397+0000 7f87b7600640 1 --2- 172.21.15.100:0/2730816283 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f878c078ef0 0x7f878c07b400 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f87a00013f0 tx=0x7f87a0009040 comp rx=0 tx=0).stop 2024-09-17T04:29:03.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.397+0000 7f87b7600640 1 -- 172.21.15.100:0/2730816283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87b015d760 msgr2=0x7f87b015b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:03.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.397+0000 7f87b7600640 1 --2- 172.21.15.100:0/2730816283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87b015d760 0x7f87b015b7f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f87a800c700 tx=0x7f87a800cbc0 comp rx=0 tx=0).stop 2024-09-17T04:29:03.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.397+0000 7f87b7600640 1 -- 172.21.15.100:0/2730816283 shutdown_connections 2024-09-17T04:29:03.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.397+0000 7f87b7600640 1 --2- 172.21.15.100:0/2730816283 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7f878c078ef0 0x7f878c07b400 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:03.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.397+0000 7f87b7600640 1 --2- 172.21.15.100:0/2730816283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87b015d760 0x7f87b015b7f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:03.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.397+0000 7f87b7600640 1 --2- 172.21.15.100:0/2730816283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87b00b3550 0x7f87b009d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:03.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.397+0000 7f87b7600640 1 --2- 172.21.15.100:0/2730816283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87b00ad0e0 0x7f87b0156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:03.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.397+0000 7f87b7600640 1 -- 172.21.15.100:0/2730816283 >> 172.21.15.100:0/2730816283 conn(0x7f87b006d730 msgr2=0x7f87b00cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:03.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.397+0000 7f87b7600640 1 -- 172.21.15.100:0/2730816283 shutdown_connections 2024-09-17T04:29:03.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.397+0000 7f87b7600640 1 -- 172.21.15.100:0/2730816283 wait complete. 2024-09-17T04:29:03.421 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-17T04:29:03.662 INFO:teuthology.orchestra.run.smithi100.stdout:[{"id":6390,"entity":{"name":{"type":"client","num":6390},"addr":{"type":"any","addr":"172.21.15.184:0","nonce":659661481}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.76497948500000001,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.961834830018976,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6390 172.21.15.184:0/659661481","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"ce1b342bf8c6349a8c90cd607f83bff2443f21cf","ceph_version":"ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)","entity_id":"x","hostname":"smithi184","pid":"35968","root":"/"}}] 2024-09-17T04:29:03.662 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi184", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35968", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.184:0", "nonce": 659661481, "type": "any" }, "name": { "num": 6390, "type": "client" } }, "id": 6390, "inst": "client.6390 172.21.15.184:0/659661481", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.961834830018976 }, "state": "open", "uptime": 0.764979485 } ] 2024-09-17T04:29:03.663 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-17T04:29:03.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.884+0000 7ff4cd000640 1 -- 172.21.15.100:0/2977959558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c815fae0 msgr2=0x7ff4c80b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:03.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.884+0000 7ff4cd000640 1 --2- 172.21.15.100:0/2977959558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c815fae0 0x7ff4c80b3750 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8072810 tx=0x7ff4c002aa30 comp rx=0 tx=0).stop 2024-09-17T04:29:03.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.884+0000 7ff4cd000640 1 -- 172.21.15.100:0/2977959558 shutdown_connections 2024-09-17T04:29:03.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.884+0000 7ff4cd000640 1 --2- 172.21.15.100:0/2977959558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c815fae0 0x7ff4c80b3750 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:03.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.884+0000 7ff4cd000640 1 --2- 172.21.15.100:0/2977959558 >> v2:172.21.15.184:3300/0 conn(0x7ff4c80aebc0 0x7ff4c8152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:03.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.884+0000 7ff4cd000640 1 -- 172.21.15.100:0/2977959558 >> 172.21.15.100:0/2977959558 conn(0x7ff4c806d730 msgr2=0x7ff4c80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:03.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.884+0000 7ff4cd000640 1 -- 172.21.15.100:0/2977959558 shutdown_connections 2024-09-17T04:29:03.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.885+0000 7ff4cd000640 1 -- 172.21.15.100:0/2977959558 wait complete. 2024-09-17T04:29:03.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.887+0000 7ff4cd000640 1 Processor -- start 2024-09-17T04:29:03.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.887+0000 7ff4cd000640 1 -- start start 2024-09-17T04:29:03.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.888+0000 7ff4cd000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c80aebc0 0x7ff4c80b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:03.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.888+0000 7ff4cd000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4c80b2750 0x7ff4c80ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:03.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.888+0000 7ff4cd000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4c81f9000 0x7ff4c81f9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:03.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.888+0000 7ff4cd000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c80b40d0 con 0x7ff4c80aebc0 2024-09-17T04:29:03.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.888+0000 7ff4cd000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c80b3f50 con 0x7ff4c81f9000 2024-09-17T04:29:03.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.888+0000 7ff4cd000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff4c80b4250 con 0x7ff4c80b2750 2024-09-17T04:29:03.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.888+0000 7ff4c5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c80aebc0 0x7ff4c80b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:03.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.889+0000 7ff4c5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c80aebc0 0x7ff4c80b1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34930/0 (socket says 172.21.15.100:34930) 2024-09-17T04:29:03.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.889+0000 7ff4c5e00640 1 -- 172.21.15.100:0/3710624614 learned_addr learned my addr 172.21.15.100:0/3710624614 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:03.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.889+0000 7ff4c6800640 1 --2- 172.21.15.100:0/3710624614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4c81f9000 0x7ff4c81f9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:03.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.889+0000 7ff4c5400640 1 --2- 172.21.15.100:0/3710624614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4c80b2750 0x7ff4c80ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:03.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.889+0000 7ff4c5e00640 1 -- 172.21.15.100:0/3710624614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4c80b2750 msgr2=0x7ff4c80ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:03.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.889+0000 7ff4c5e00640 1 --2- 172.21.15.100:0/3710624614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4c80b2750 0x7ff4c80ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:03.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.889+0000 7ff4c5e00640 1 -- 172.21.15.100:0/3710624614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4c81f9000 msgr2=0x7ff4c81f9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:03.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.889+0000 7ff4c5e00640 1 --2- 172.21.15.100:0/3710624614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4c81f9000 0x7ff4c81f9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:03.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.889+0000 7ff4c5e00640 1 -- 172.21.15.100:0/3710624614 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c80b0010 con 0x7ff4c80aebc0 2024-09-17T04:29:03.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.889+0000 7ff4c6800640 1 --2- 172.21.15.100:0/3710624614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4c81f9000 0x7ff4c81f9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:03.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.889+0000 7ff4c5e00640 1 --2- 172.21.15.100:0/3710624614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c80aebc0 0x7ff4c80b1b10 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800e790 tx=0x7ff4b800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:03.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.890+0000 7ff4be000640 1 -- 172.21.15.100:0/3710624614 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b800eea0 con 0x7ff4c80aebc0 2024-09-17T04:29:03.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.890+0000 7ff4cd000640 1 -- 172.21.15.100:0/3710624614 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c80b02f0 con 0x7ff4c80aebc0 2024-09-17T04:29:03.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.890+0000 7ff4be000640 1 -- 172.21.15.100:0/3710624614 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4b800cdb0 con 0x7ff4c80aebc0 2024-09-17T04:29:03.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.890+0000 7ff4be000640 1 -- 172.21.15.100:0/3710624614 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8010690 con 0x7ff4c80aebc0 2024-09-17T04:29:03.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.890+0000 7ff4cd000640 1 -- 172.21.15.100:0/3710624614 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c80b08b0 con 0x7ff4c80aebc0 2024-09-17T04:29:03.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.891+0000 7ff4cd000640 1 -- 172.21.15.100:0/3710624614 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4c80b4770 con 0x7ff4c80aebc0 2024-09-17T04:29:03.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.892+0000 7ff4be000640 1 -- 172.21.15.100:0/3710624614 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7ff4b8010830 con 0x7ff4c80aebc0 2024-09-17T04:29:03.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.893+0000 7ff4be000640 1 --2- 172.21.15.100:0/3710624614 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff4a8078ef0 0x7ff4a807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:03.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.893+0000 7ff4be000640 1 -- 172.21.15.100:0/3710624614 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5767+0+0 (secure 0 0 0) 0x7ff4b809bee0 con 0x7ff4c80aebc0 2024-09-17T04:29:03.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.893+0000 7ff4c5400640 1 --2- 172.21.15.100:0/3710624614 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff4a8078ef0 0x7ff4a807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:03.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.894+0000 7ff4c5400640 1 --2- 172.21.15.100:0/3710624614 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff4a8078ef0 0x7ff4a807b400 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4007fd0 tx=0x7ff4b4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:03.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:03.897+0000 7ff4be000640 1 -- 172.21.15.100:0/3710624614 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff4b8063d00 con 0x7ff4c80aebc0 2024-09-17T04:29:04.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:04.130+0000 7ff4cd000640 1 -- 172.21.15.100:0/3710624614 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7ff4c80698b0 con 0x7ff4c80aebc0 2024-09-17T04:29:04.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:04.138+0000 7ff4be000640 1 -- 172.21.15.100:0/3710624614 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v11) ==== 81+0+530962 (secure 0 0 0) 0x7ff4b8063600 con 0x7ff4c80aebc0 2024-09-17T04:29:04.138 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:04.139 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:29:04.139 INFO:teuthology.orchestra.run.smithi100.stdout: "epoch": 11, 2024-09-17T04:29:04.139 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.139 INFO:teuthology.orchestra.run.smithi100.stdout: "active_gid": 4103, 2024-09-17T04:29:04.139 INFO:teuthology.orchestra.run.smithi100.stdout: "active_name": "x", 2024-09-17T04:29:04.139 INFO:teuthology.orchestra.run.smithi100.stdout: "active_addrs": { 2024-09-17T04:29:04.139 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-17T04:29:04.139 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.139 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-17T04:29:04.139 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.184:6800", 2024-09-17T04:29:04.139 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4096933233 2024-09-17T04:29:04.139 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.140 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.140 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-17T04:29:04.140 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.184:6801", 2024-09-17T04:29:04.140 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4096933233 2024-09-17T04:29:04.140 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.140 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-17T04:29:04.140 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.140 INFO:teuthology.orchestra.run.smithi100.stdout: "active_addr": "172.21.15.184:6801/4096933233", 2024-09-17T04:29:04.140 INFO:teuthology.orchestra.run.smithi100.stdout: "active_change": "2024-09-17T04:23:51.951168+0000", 2024-09-17T04:29:04.140 INFO:teuthology.orchestra.run.smithi100.stdout: "active_mgr_features": 4540719139924082687, 2024-09-17T04:29:04.140 INFO:teuthology.orchestra.run.smithi100.stdout: "available": true, 2024-09-17T04:29:04.140 INFO:teuthology.orchestra.run.smithi100.stdout: "standbys": [ 2024-09-17T04:29:04.140 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.141 INFO:teuthology.orchestra.run.smithi100.stdout: "gid": 4108, 2024-09-17T04:29:04.141 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "y", 2024-09-17T04:29:04.141 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_features": 4540719139924082687, 2024-09-17T04:29:04.141 INFO:teuthology.orchestra.run.smithi100.stdout: "available_modules": [ 2024-09-17T04:29:04.141 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.141 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "alerts", 2024-09-17T04:29:04.141 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.141 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.141 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.141 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-09-17T04:29:04.141 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-09-17T04:29:04.141 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.141 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.142 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.142 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-09-17T04:29:04.142 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.142 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.142 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.142 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How frequently to reexamine health status", 2024-09-17T04:29:04.142 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.142 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.142 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.142 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.142 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.142 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.142 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.142 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.142 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.143 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.143 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.143 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.143 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.143 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.143 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.143 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.143 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.143 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.143 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.143 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.143 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.143 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.144 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.144 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.144 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.144 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.144 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.144 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.144 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.144 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.144 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.144 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.144 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.144 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.144 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.145 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.145 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.145 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.145 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.145 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.145 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.145 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.145 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.145 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.145 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.145 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.145 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.145 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.146 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.146 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.146 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.146 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.146 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.146 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.146 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.146 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.146 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.146 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.147 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.147 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.147 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.147 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.147 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.147 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.147 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.147 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.147 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.147 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.147 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.147 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.148 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.148 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.148 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.148 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.148 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_destination": { 2024-09-17T04:29:04.148 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_destination", 2024-09-17T04:29:04.148 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.148 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.148 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.148 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.148 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.148 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.148 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.149 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-17T04:29:04.149 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.149 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.149 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.149 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.149 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_from_name": { 2024-09-17T04:29:04.149 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_from_name", 2024-09-17T04:29:04.149 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.149 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.149 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.149 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "Ceph", 2024-09-17T04:29:04.149 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.149 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.150 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.150 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Email From: name", 2024-09-17T04:29:04.150 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.150 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.150 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.150 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.150 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_host": { 2024-09-17T04:29:04.150 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_host", 2024-09-17T04:29:04.150 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.150 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.150 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.150 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.150 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.150 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.151 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.151 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SMTP server", 2024-09-17T04:29:04.151 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.151 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.151 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.151 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.151 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_password": { 2024-09-17T04:29:04.151 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_password", 2024-09-17T04:29:04.151 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.151 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.151 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.151 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.151 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.151 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.152 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.152 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Password to authenticate with", 2024-09-17T04:29:04.152 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.152 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.152 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.152 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.152 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_port": { 2024-09-17T04:29:04.152 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_port", 2024-09-17T04:29:04.152 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.152 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.152 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.152 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "465", 2024-09-17T04:29:04.152 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.152 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.153 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.153 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SMTP port", 2024-09-17T04:29:04.153 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.153 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.153 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.153 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.153 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_sender": { 2024-09-17T04:29:04.153 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_sender", 2024-09-17T04:29:04.153 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.153 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.153 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.153 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.153 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.153 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.154 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.154 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SMTP envelope sender", 2024-09-17T04:29:04.154 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.154 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.154 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.154 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.154 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_ssl": { 2024-09-17T04:29:04.154 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_ssl", 2024-09-17T04:29:04.154 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.154 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.154 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.154 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.154 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.155 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.155 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.155 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-17T04:29:04.155 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.155 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.155 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.155 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.155 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_user": { 2024-09-17T04:29:04.155 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_user", 2024-09-17T04:29:04.155 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.155 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.155 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.155 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.155 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.156 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.156 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.156 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "User to authenticate as", 2024-09-17T04:29:04.156 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.156 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.156 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.156 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.156 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.156 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.156 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.156 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.156 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.156 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.156 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.157 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.157 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.157 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.157 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.157 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.157 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.157 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.157 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.157 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.157 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.157 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "balancer", 2024-09-17T04:29:04.157 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.157 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.158 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.158 INFO:teuthology.orchestra.run.smithi100.stdout: "active": { 2024-09-17T04:29:04.158 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "active", 2024-09-17T04:29:04.158 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.158 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.158 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.158 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.158 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.158 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.158 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.158 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "automatically balance PGs across cluster", 2024-09-17T04:29:04.158 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.158 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.159 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.159 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.159 INFO:teuthology.orchestra.run.smithi100.stdout: "begin_time": { 2024-09-17T04:29:04.159 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "begin_time", 2024-09-17T04:29:04.159 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.159 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.159 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.159 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0000", 2024-09-17T04:29:04.159 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.159 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.159 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.159 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "beginning time of day to automatically balance", 2024-09-17T04:29:04.159 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-17T04:29:04.159 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.159 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.160 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.160 INFO:teuthology.orchestra.run.smithi100.stdout: "begin_weekday": { 2024-09-17T04:29:04.160 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "begin_weekday", 2024-09-17T04:29:04.160 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-09-17T04:29:04.160 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.160 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.160 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.160 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "0", 2024-09-17T04:29:04.160 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "6", 2024-09-17T04:29:04.160 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.160 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-17T04:29:04.160 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-17T04:29:04.160 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.161 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.161 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.161 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_compat_max_iterations": { 2024-09-17T04:29:04.161 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crush_compat_max_iterations", 2024-09-17T04:29:04.161 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-09-17T04:29:04.161 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.161 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.161 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "25", 2024-09-17T04:29:04.161 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-09-17T04:29:04.161 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "250", 2024-09-17T04:29:04.161 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.161 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-17T04:29:04.161 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.161 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.162 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.162 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.162 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_compat_metrics": { 2024-09-17T04:29:04.162 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crush_compat_metrics", 2024-09-17T04:29:04.162 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.162 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.162 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.162 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "pgs,objects,bytes", 2024-09-17T04:29:04.162 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.162 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.162 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.162 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-17T04:29:04.162 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-17T04:29:04.162 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.162 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.163 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.163 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_compat_step": { 2024-09-17T04:29:04.163 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crush_compat_step", 2024-09-17T04:29:04.163 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-09-17T04:29:04.163 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.163 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.163 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0.5", 2024-09-17T04:29:04.163 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "0.001", 2024-09-17T04:29:04.163 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "0.999", 2024-09-17T04:29:04.163 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.163 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "aggressiveness of optimization", 2024-09-17T04:29:04.163 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-17T04:29:04.163 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.163 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.163 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.164 INFO:teuthology.orchestra.run.smithi100.stdout: "end_time": { 2024-09-17T04:29:04.164 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "end_time", 2024-09-17T04:29:04.164 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.164 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.164 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.164 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "2359", 2024-09-17T04:29:04.164 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.164 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.164 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.164 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "ending time of day to automatically balance", 2024-09-17T04:29:04.164 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-17T04:29:04.164 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.164 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.165 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.165 INFO:teuthology.orchestra.run.smithi100.stdout: "end_weekday": { 2024-09-17T04:29:04.165 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "end_weekday", 2024-09-17T04:29:04.165 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-09-17T04:29:04.165 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.165 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.165 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.165 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "0", 2024-09-17T04:29:04.165 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "6", 2024-09-17T04:29:04.165 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.165 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-17T04:29:04.165 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-17T04:29:04.165 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.165 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.166 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.166 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.166 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.166 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.166 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.166 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.166 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.166 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.166 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.166 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.166 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.166 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.166 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.166 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.167 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.167 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.167 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.167 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.167 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.167 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.167 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.167 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.167 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.167 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.167 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.167 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.167 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.167 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.167 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.168 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.168 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.168 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.168 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.168 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.168 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.168 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.169 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.169 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.169 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.169 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.169 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.169 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.169 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.169 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.169 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.169 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.169 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.169 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.169 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.170 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.170 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.170 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.170 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.170 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.170 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.170 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.170 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.170 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.170 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.170 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.170 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.170 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.170 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.171 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.171 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.171 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.171 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.171 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.171 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.171 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.171 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.171 INFO:teuthology.orchestra.run.smithi100.stdout: "min_score": { 2024-09-17T04:29:04.171 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "min_score", 2024-09-17T04:29:04.171 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-09-17T04:29:04.171 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.171 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.172 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.172 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.172 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.172 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.172 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-17T04:29:04.172 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.172 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.172 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.172 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.172 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": { 2024-09-17T04:29:04.172 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mode", 2024-09-17T04:29:04.172 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.172 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.173 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.173 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "upmap", 2024-09-17T04:29:04.173 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.173 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.173 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.173 INFO:teuthology.orchestra.run.smithi100.stdout: "crush-compat", 2024-09-17T04:29:04.173 INFO:teuthology.orchestra.run.smithi100.stdout: "none", 2024-09-17T04:29:04.173 INFO:teuthology.orchestra.run.smithi100.stdout: "read", 2024-09-17T04:29:04.173 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap", 2024-09-17T04:29:04.173 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap-read" 2024-09-17T04:29:04.173 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.173 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Balancer mode", 2024-09-17T04:29:04.173 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.174 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.174 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.174 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.174 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_ids": { 2024-09-17T04:29:04.174 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pool_ids", 2024-09-17T04:29:04.174 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.174 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.174 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.174 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.174 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.174 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.174 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.174 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-17T04:29:04.174 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.175 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.175 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.175 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.175 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-09-17T04:29:04.175 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-09-17T04:29:04.175 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.175 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.175 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.175 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-09-17T04:29:04.175 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.175 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.175 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.175 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-17T04:29:04.176 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.176 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.176 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.176 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.176 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.176 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.176 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.176 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.176 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.176 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.176 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.176 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.176 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.176 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.177 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.177 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.177 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.177 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.177 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap_max_deviation": { 2024-09-17T04:29:04.177 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "upmap_max_deviation", 2024-09-17T04:29:04.177 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.177 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.177 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.177 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-09-17T04:29:04.177 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-09-17T04:29:04.177 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.177 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.178 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-17T04:29:04.178 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-17T04:29:04.178 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.178 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.178 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.178 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap_max_optimizations": { 2024-09-17T04:29:04.178 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "upmap_max_optimizations", 2024-09-17T04:29:04.178 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-09-17T04:29:04.178 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.178 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.178 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-09-17T04:29:04.178 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.178 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.178 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.179 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-17T04:29:04.179 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.179 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.179 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.179 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.179 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.179 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.179 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.179 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephadm", 2024-09-17T04:29:04.179 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.179 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.179 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.179 INFO:teuthology.orchestra.run.smithi100.stdout: "agent_down_multiplier": { 2024-09-17T04:29:04.179 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "agent_down_multiplier", 2024-09-17T04:29:04.180 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-09-17T04:29:04.180 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.180 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.180 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "3.0", 2024-09-17T04:29:04.180 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.180 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.180 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.180 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-17T04:29:04.180 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.180 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.180 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.180 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.180 INFO:teuthology.orchestra.run.smithi100.stdout: "agent_refresh_rate": { 2024-09-17T04:29:04.180 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "agent_refresh_rate", 2024-09-17T04:29:04.181 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.181 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.181 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.181 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "20", 2024-09-17T04:29:04.181 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.181 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.181 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.181 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-17T04:29:04.181 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.181 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.181 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.181 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.181 INFO:teuthology.orchestra.run.smithi100.stdout: "agent_starting_port": { 2024-09-17T04:29:04.182 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "agent_starting_port", 2024-09-17T04:29:04.182 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.182 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.182 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.182 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "4721", 2024-09-17T04:29:04.182 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.182 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.182 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.182 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-17T04:29:04.182 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.182 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.182 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.182 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.183 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_ptrace": { 2024-09-17T04:29:04.183 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "allow_ptrace", 2024-09-17T04:29:04.183 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.183 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.183 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.183 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.183 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.183 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.183 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.183 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-17T04:29:04.183 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-17T04:29:04.183 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.183 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.184 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.184 INFO:teuthology.orchestra.run.smithi100.stdout: "autotune_interval": { 2024-09-17T04:29:04.184 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "autotune_interval", 2024-09-17T04:29:04.184 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.184 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.184 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.184 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-09-17T04:29:04.184 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.184 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.184 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.184 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-17T04:29:04.184 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.184 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.184 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.185 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.185 INFO:teuthology.orchestra.run.smithi100.stdout: "autotune_memory_target_ratio": { 2024-09-17T04:29:04.185 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "autotune_memory_target_ratio", 2024-09-17T04:29:04.185 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-09-17T04:29:04.185 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.185 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.185 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0.7", 2024-09-17T04:29:04.185 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.185 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.185 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.185 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-17T04:29:04.185 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.185 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.185 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.186 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.186 INFO:teuthology.orchestra.run.smithi100.stdout: "cephadm_log_destination": { 2024-09-17T04:29:04.186 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephadm_log_destination", 2024-09-17T04:29:04.186 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.186 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.186 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.186 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.186 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.186 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.186 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.186 INFO:teuthology.orchestra.run.smithi100.stdout: "file", 2024-09-17T04:29:04.186 INFO:teuthology.orchestra.run.smithi100.stdout: "file,syslog", 2024-09-17T04:29:04.186 INFO:teuthology.orchestra.run.smithi100.stdout: "syslog" 2024-09-17T04:29:04.186 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.187 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-17T04:29:04.187 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.187 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.187 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.187 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.187 INFO:teuthology.orchestra.run.smithi100.stdout: "cgroups_split": { 2024-09-17T04:29:04.187 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cgroups_split", 2024-09-17T04:29:04.187 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.187 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.187 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.188 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.188 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.188 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.188 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.188 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-17T04:29:04.188 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.188 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.188 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.188 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.188 INFO:teuthology.orchestra.run.smithi100.stdout: "config_checks_enabled": { 2024-09-17T04:29:04.188 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "config_checks_enabled", 2024-09-17T04:29:04.188 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.188 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.189 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.189 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.189 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.189 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.189 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.189 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-17T04:29:04.189 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.189 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.189 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.189 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.189 INFO:teuthology.orchestra.run.smithi100.stdout: "config_dashboard": { 2024-09-17T04:29:04.189 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "config_dashboard", 2024-09-17T04:29:04.189 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.189 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.190 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.190 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.190 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.190 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.190 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.190 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-17T04:29:04.190 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.190 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.190 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.190 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.190 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_alertmanager": { 2024-09-17T04:29:04.190 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_alertmanager", 2024-09-17T04:29:04.190 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.190 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.191 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.191 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-17T04:29:04.191 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.191 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.191 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.191 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-09-17T04:29:04.191 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.191 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.191 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.191 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.191 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_base": { 2024-09-17T04:29:04.191 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_base", 2024-09-17T04:29:04.191 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.192 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.192 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.192 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-17T04:29:04.192 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.192 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.192 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.192 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Container image name, without the tag", 2024-09-17T04:29:04.192 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.192 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.192 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.192 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.192 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_elasticsearch": { 2024-09-17T04:29:04.192 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_elasticsearch", 2024-09-17T04:29:04.192 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.192 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.193 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.193 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-17T04:29:04.193 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.193 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.193 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.193 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "elasticsearch container image", 2024-09-17T04:29:04.193 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.193 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.193 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.193 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.193 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_grafana": { 2024-09-17T04:29:04.193 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_grafana", 2024-09-17T04:29:04.194 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.194 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.194 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.194 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-17T04:29:04.194 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.194 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.194 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.194 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-09-17T04:29:04.194 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.194 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.194 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.194 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.194 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_haproxy": { 2024-09-17T04:29:04.194 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_haproxy", 2024-09-17T04:29:04.195 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.195 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.195 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.195 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-17T04:29:04.195 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.195 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.195 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.195 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "HAproxy container image", 2024-09-17T04:29:04.195 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.195 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.195 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.195 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.195 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_jaeger_agent": { 2024-09-17T04:29:04.195 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_jaeger_agent", 2024-09-17T04:29:04.196 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.196 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.196 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.196 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-17T04:29:04.196 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.196 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.196 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.196 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Jaeger agent container image", 2024-09-17T04:29:04.196 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.196 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.196 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.196 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.196 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_jaeger_collector": { 2024-09-17T04:29:04.197 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_jaeger_collector", 2024-09-17T04:29:04.197 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.197 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.197 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.197 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-17T04:29:04.197 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.197 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.197 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.197 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Jaeger collector container image", 2024-09-17T04:29:04.197 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.197 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.197 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.197 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.197 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_jaeger_query": { 2024-09-17T04:29:04.198 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_jaeger_query", 2024-09-17T04:29:04.198 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.198 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.198 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.198 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-17T04:29:04.198 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.198 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.198 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.198 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Jaeger query container image", 2024-09-17T04:29:04.198 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.198 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.198 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.198 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.199 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_keepalived": { 2024-09-17T04:29:04.199 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_keepalived", 2024-09-17T04:29:04.199 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.199 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.199 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.199 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-17T04:29:04.199 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.199 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.199 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.199 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Keepalived container image", 2024-09-17T04:29:04.199 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.199 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.199 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.199 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.200 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_loki": { 2024-09-17T04:29:04.200 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_loki", 2024-09-17T04:29:04.200 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.200 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.200 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.200 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-17T04:29:04.200 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.200 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.200 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.200 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Loki container image", 2024-09-17T04:29:04.200 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.200 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.200 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.201 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.201 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_nginx": { 2024-09-17T04:29:04.201 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_nginx", 2024-09-17T04:29:04.201 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.201 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.201 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.201 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-17T04:29:04.201 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.201 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.201 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.201 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Nginx container image", 2024-09-17T04:29:04.201 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.201 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.201 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.202 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.202 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_node_exporter": { 2024-09-17T04:29:04.202 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_node_exporter", 2024-09-17T04:29:04.202 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.202 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.202 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.202 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-17T04:29:04.202 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.202 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.202 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.202 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-09-17T04:29:04.202 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.202 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.202 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.203 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.203 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_nvmeof": { 2024-09-17T04:29:04.203 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_nvmeof", 2024-09-17T04:29:04.203 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.203 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.203 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.203 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-17T04:29:04.203 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.203 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.203 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.203 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Nvme-of container image", 2024-09-17T04:29:04.203 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.203 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.204 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.204 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.204 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_oauth2_proxy": { 2024-09-17T04:29:04.204 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_oauth2_proxy", 2024-09-17T04:29:04.204 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.204 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.204 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.204 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-17T04:29:04.204 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.204 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.204 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.204 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "oauth2-proxy container image", 2024-09-17T04:29:04.204 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.204 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.205 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.205 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.205 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_prometheus": { 2024-09-17T04:29:04.205 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_prometheus", 2024-09-17T04:29:04.205 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.205 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.205 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.205 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-17T04:29:04.205 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.205 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.205 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.205 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-09-17T04:29:04.205 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.206 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.206 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.206 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.206 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_promtail": { 2024-09-17T04:29:04.206 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_promtail", 2024-09-17T04:29:04.206 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.206 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.206 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.206 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-17T04:29:04.206 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.206 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.206 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.206 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Promtail container image", 2024-09-17T04:29:04.206 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.207 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.207 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.207 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.207 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_samba": { 2024-09-17T04:29:04.207 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_samba", 2024-09-17T04:29:04.207 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.207 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.207 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.207 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-17T04:29:04.207 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.207 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.207 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.207 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Samba/SMB container image", 2024-09-17T04:29:04.207 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.208 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.208 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.208 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.208 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_samba_metrics": { 2024-09-17T04:29:04.208 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_samba_metrics", 2024-09-17T04:29:04.208 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.208 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.208 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.208 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-17T04:29:04.208 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.208 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.208 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.208 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-17T04:29:04.208 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.209 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.209 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.209 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.209 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_snmp_gateway": { 2024-09-17T04:29:04.209 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_snmp_gateway", 2024-09-17T04:29:04.209 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.209 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.209 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.209 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-17T04:29:04.209 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.209 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.209 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.209 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SNMP Gateway container image", 2024-09-17T04:29:04.210 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.210 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.210 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.210 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.210 INFO:teuthology.orchestra.run.smithi100.stdout: "container_init": { 2024-09-17T04:29:04.210 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_init", 2024-09-17T04:29:04.210 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.210 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.210 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.210 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.210 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.210 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.210 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.210 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Run podman/docker with `--init`", 2024-09-17T04:29:04.211 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.211 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.211 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.211 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.211 INFO:teuthology.orchestra.run.smithi100.stdout: "daemon_cache_timeout": { 2024-09-17T04:29:04.211 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "daemon_cache_timeout", 2024-09-17T04:29:04.211 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.211 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.211 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.211 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-09-17T04:29:04.211 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.211 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.211 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.211 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-17T04:29:04.211 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.212 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.212 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.212 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.212 INFO:teuthology.orchestra.run.smithi100.stdout: "default_cephadm_command_timeout": { 2024-09-17T04:29:04.212 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "default_cephadm_command_timeout", 2024-09-17T04:29:04.212 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.212 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.212 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.212 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "900", 2024-09-17T04:29:04.212 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.212 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.212 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.212 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-17T04:29:04.213 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.213 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.213 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.213 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.213 INFO:teuthology.orchestra.run.smithi100.stdout: "default_registry": { 2024-09-17T04:29:04.213 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "default_registry", 2024-09-17T04:29:04.213 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.213 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.213 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.213 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io", 2024-09-17T04:29:04.213 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.213 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.213 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.213 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-17T04:29:04.214 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.214 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.214 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.214 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.214 INFO:teuthology.orchestra.run.smithi100.stdout: "device_cache_timeout": { 2024-09-17T04:29:04.214 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "device_cache_timeout", 2024-09-17T04:29:04.214 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.214 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.214 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.214 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "1800", 2024-09-17T04:29:04.214 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.214 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.214 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.215 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "seconds to cache device inventory", 2024-09-17T04:29:04.215 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.215 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.215 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.215 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.215 INFO:teuthology.orchestra.run.smithi100.stdout: "device_enhanced_scan": { 2024-09-17T04:29:04.215 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "device_enhanced_scan", 2024-09-17T04:29:04.215 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.215 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.215 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.215 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.215 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.215 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.216 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.216 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-17T04:29:04.216 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.216 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.216 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.216 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.216 INFO:teuthology.orchestra.run.smithi100.stdout: "facts_cache_timeout": { 2024-09-17T04:29:04.216 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "facts_cache_timeout", 2024-09-17T04:29:04.216 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.216 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.216 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.216 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-09-17T04:29:04.216 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.217 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.217 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.217 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "seconds to cache host facts data", 2024-09-17T04:29:04.217 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.217 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.217 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.217 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.217 INFO:teuthology.orchestra.run.smithi100.stdout: "grafana_dashboards_path": { 2024-09-17T04:29:04.217 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "grafana_dashboards_path", 2024-09-17T04:29:04.217 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.217 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.217 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.217 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-17T04:29:04.217 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.218 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.218 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.223 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-17T04:29:04.224 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.224 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.224 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.224 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.224 INFO:teuthology.orchestra.run.smithi100.stdout: "host_check_interval": { 2024-09-17T04:29:04.224 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "host_check_interval", 2024-09-17T04:29:04.224 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.224 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.224 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.224 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-09-17T04:29:04.224 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.224 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.224 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.225 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to perform a host check", 2024-09-17T04:29:04.225 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.225 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.225 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.225 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.225 INFO:teuthology.orchestra.run.smithi100.stdout: "hw_monitoring": { 2024-09-17T04:29:04.225 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "hw_monitoring", 2024-09-17T04:29:04.225 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.225 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.225 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.225 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.225 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.225 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.226 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.226 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-17T04:29:04.226 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.226 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.226 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.226 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.226 INFO:teuthology.orchestra.run.smithi100.stdout: "inventory_list_all": { 2024-09-17T04:29:04.226 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "inventory_list_all", 2024-09-17T04:29:04.226 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.226 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.226 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.226 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.226 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.227 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.227 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.227 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-17T04:29:04.227 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.227 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.227 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.227 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.227 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.227 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.227 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.227 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.227 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.227 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.227 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.228 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.228 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.228 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.228 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.228 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.228 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.228 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.228 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.228 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.228 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.228 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.228 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.228 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.228 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.229 INFO:teuthology.orchestra.run.smithi100.stdout: "log_refresh_metadata": { 2024-09-17T04:29:04.229 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_refresh_metadata", 2024-09-17T04:29:04.229 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.229 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.229 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.229 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.229 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.229 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.229 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.229 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-17T04:29:04.229 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.229 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.229 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.229 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.230 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.230 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.230 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.230 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.230 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.230 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.230 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.230 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.230 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.230 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-17T04:29:04.230 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.230 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.230 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.230 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.230 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.231 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.231 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.231 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.231 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.231 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.231 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.231 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.231 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.231 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.231 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.231 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.231 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.231 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.232 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.232 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.232 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.232 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.232 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.232 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.232 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.232 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.232 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.232 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.232 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.232 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.232 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.233 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.233 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.233 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.233 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.233 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.233 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.233 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.233 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.233 INFO:teuthology.orchestra.run.smithi100.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-17T04:29:04.233 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-17T04:29:04.233 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.233 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.233 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.234 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.234 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.234 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.234 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.234 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-17T04:29:04.234 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.234 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.234 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.234 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.234 INFO:teuthology.orchestra.run.smithi100.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-17T04:29:04.234 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-17T04:29:04.234 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.234 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.234 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.235 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "*", 2024-09-17T04:29:04.235 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.235 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.235 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.235 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-17T04:29:04.235 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.235 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.235 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.235 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.235 INFO:teuthology.orchestra.run.smithi100.stdout: "max_count_per_host": { 2024-09-17T04:29:04.235 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_count_per_host", 2024-09-17T04:29:04.235 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.235 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.235 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.236 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-09-17T04:29:04.236 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.236 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.236 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.236 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "max number of daemons per service per host", 2024-09-17T04:29:04.236 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.236 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.236 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.236 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.236 INFO:teuthology.orchestra.run.smithi100.stdout: "max_osd_draining_count": { 2024-09-17T04:29:04.236 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_osd_draining_count", 2024-09-17T04:29:04.236 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.236 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.237 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.237 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-09-17T04:29:04.237 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.237 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.237 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.237 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-17T04:29:04.237 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.237 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.237 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.237 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.238 INFO:teuthology.orchestra.run.smithi100.stdout: "migration_current": { 2024-09-17T04:29:04.238 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "migration_current", 2024-09-17T04:29:04.238 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.238 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.238 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.238 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.238 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.238 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.238 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.238 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "internal - do not modify", 2024-09-17T04:29:04.238 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.238 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.239 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.239 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.239 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": { 2024-09-17T04:29:04.239 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mode", 2024-09-17T04:29:04.239 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.239 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.239 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.239 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "root", 2024-09-17T04:29:04.239 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.239 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.239 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.239 INFO:teuthology.orchestra.run.smithi100.stdout: "cephadm-package", 2024-09-17T04:29:04.239 INFO:teuthology.orchestra.run.smithi100.stdout: "root" 2024-09-17T04:29:04.239 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.240 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "mode for remote execution of cephadm", 2024-09-17T04:29:04.240 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.240 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.240 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.240 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.240 INFO:teuthology.orchestra.run.smithi100.stdout: "oob_default_addr": { 2024-09-17T04:29:04.240 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "oob_default_addr", 2024-09-17T04:29:04.240 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.240 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.240 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.240 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "169.254.1.1", 2024-09-17T04:29:04.240 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.240 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.240 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.241 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-17T04:29:04.241 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.241 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.241 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.241 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.241 INFO:teuthology.orchestra.run.smithi100.stdout: "prometheus_alerts_path": { 2024-09-17T04:29:04.241 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "prometheus_alerts_path", 2024-09-17T04:29:04.241 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.241 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.241 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.241 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-17T04:29:04.241 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.241 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.241 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.242 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-17T04:29:04.242 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.242 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.242 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.242 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.242 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_insecure": { 2024-09-17T04:29:04.242 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_insecure", 2024-09-17T04:29:04.242 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.242 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.242 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.242 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.242 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.242 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.242 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.242 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-17T04:29:04.243 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.243 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.243 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.243 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.243 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_password": { 2024-09-17T04:29:04.243 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_password", 2024-09-17T04:29:04.243 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.243 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.243 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.243 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.243 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.243 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.243 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.243 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-17T04:29:04.244 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.244 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.244 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.244 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.244 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_url": { 2024-09-17T04:29:04.244 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_url", 2024-09-17T04:29:04.244 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.244 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.244 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.244 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.244 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.244 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.244 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.244 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-17T04:29:04.245 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.245 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.245 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.245 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.245 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_username": { 2024-09-17T04:29:04.245 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_username", 2024-09-17T04:29:04.245 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.245 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.245 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.245 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.245 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.245 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.245 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.245 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-17T04:29:04.246 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.246 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.246 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.246 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.246 INFO:teuthology.orchestra.run.smithi100.stdout: "secure_monitoring_stack": { 2024-09-17T04:29:04.246 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "secure_monitoring_stack", 2024-09-17T04:29:04.246 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.246 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.246 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.246 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.246 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.246 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.246 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.246 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-17T04:29:04.246 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.247 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.247 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.247 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.247 INFO:teuthology.orchestra.run.smithi100.stdout: "service_discovery_port": { 2024-09-17T04:29:04.247 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "service_discovery_port", 2024-09-17T04:29:04.247 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.247 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.247 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.247 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8765", 2024-09-17T04:29:04.247 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.247 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.247 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.247 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "cephadm service discovery port", 2024-09-17T04:29:04.247 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.248 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.248 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.248 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.248 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.248 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.248 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.248 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.248 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.248 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.248 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.248 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.248 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.248 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.248 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.249 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.249 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.249 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.249 INFO:teuthology.orchestra.run.smithi100.stdout: "ssh_config_file": { 2024-09-17T04:29:04.249 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssh_config_file", 2024-09-17T04:29:04.249 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.249 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.249 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.249 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.249 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.249 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.249 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.249 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-17T04:29:04.250 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.250 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.250 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.250 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.250 INFO:teuthology.orchestra.run.smithi100.stdout: "ssh_keepalive_count_max": { 2024-09-17T04:29:04.250 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssh_keepalive_count_max", 2024-09-17T04:29:04.250 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.250 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.250 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.250 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "3", 2024-09-17T04:29:04.250 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.250 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.250 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.250 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-17T04:29:04.251 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.251 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.251 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.251 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.251 INFO:teuthology.orchestra.run.smithi100.stdout: "ssh_keepalive_interval": { 2024-09-17T04:29:04.251 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssh_keepalive_interval", 2024-09-17T04:29:04.251 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.251 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.251 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.251 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "7", 2024-09-17T04:29:04.251 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.251 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.251 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.252 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-17T04:29:04.252 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.252 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.252 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.252 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.252 INFO:teuthology.orchestra.run.smithi100.stdout: "use_agent": { 2024-09-17T04:29:04.252 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "use_agent", 2024-09-17T04:29:04.252 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.252 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.252 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.252 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.252 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.252 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.252 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.253 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-17T04:29:04.253 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.253 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.253 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.253 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.253 INFO:teuthology.orchestra.run.smithi100.stdout: "use_repo_digest": { 2024-09-17T04:29:04.253 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "use_repo_digest", 2024-09-17T04:29:04.253 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.253 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.253 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.253 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.253 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.253 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.253 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.254 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-17T04:29:04.254 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.254 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.254 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.254 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.254 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_on_failed_host_check": { 2024-09-17T04:29:04.254 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_on_failed_host_check", 2024-09-17T04:29:04.254 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.254 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.254 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.254 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.254 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.255 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.255 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.255 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise a health warning if the host check fails", 2024-09-17T04:29:04.255 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.255 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.255 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.255 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.255 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_on_stray_daemons": { 2024-09-17T04:29:04.255 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_on_stray_daemons", 2024-09-17T04:29:04.255 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.255 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.255 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.255 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.255 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.256 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.256 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.256 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-17T04:29:04.256 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.256 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.256 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.256 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.256 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_on_stray_hosts": { 2024-09-17T04:29:04.256 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_on_stray_hosts", 2024-09-17T04:29:04.256 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.256 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.256 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.256 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.256 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.257 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.257 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.257 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-17T04:29:04.257 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.257 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.257 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.257 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.257 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.257 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.257 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.257 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crash", 2024-09-17T04:29:04.257 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.257 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.257 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.258 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.258 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.258 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.258 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.258 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.258 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.258 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.258 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.258 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.258 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.258 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.258 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.258 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.258 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.259 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.259 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.259 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.259 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.259 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.259 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.259 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.259 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.259 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.259 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.259 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.259 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.259 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.259 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.260 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.260 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.260 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.260 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.260 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.260 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.260 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.260 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.260 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.260 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.260 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.260 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.261 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.261 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.261 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.261 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.261 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.261 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.261 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.261 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.261 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.261 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.261 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.261 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.261 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.261 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.262 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.262 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.262 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.262 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.262 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.262 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.262 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.262 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.262 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.262 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.262 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.262 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.262 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.263 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.263 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.263 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.263 INFO:teuthology.orchestra.run.smithi100.stdout: "retain_interval": { 2024-09-17T04:29:04.263 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "retain_interval", 2024-09-17T04:29:04.263 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.263 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.263 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.263 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "31536000", 2024-09-17T04:29:04.263 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.263 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.263 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.263 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-17T04:29:04.263 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.264 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.264 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.264 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.264 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.264 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.264 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.264 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.264 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.264 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.264 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.264 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.264 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.264 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.265 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.265 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.265 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.265 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.265 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_recent_interval": { 2024-09-17T04:29:04.265 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_recent_interval", 2024-09-17T04:29:04.265 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.265 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.265 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.265 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "1209600", 2024-09-17T04:29:04.265 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.265 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.265 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.265 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-17T04:29:04.266 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.266 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.266 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.266 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.266 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.266 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.266 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.266 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "dashboard", 2024-09-17T04:29:04.266 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.266 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.266 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.266 INFO:teuthology.orchestra.run.smithi100.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-17T04:29:04.266 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-17T04:29:04.266 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.267 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.267 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.267 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-09-17T04:29:04.267 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.267 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.267 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.267 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.267 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.267 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.267 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.267 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.267 INFO:teuthology.orchestra.run.smithi100.stdout: "ALERTMANAGER_API_HOST": { 2024-09-17T04:29:04.267 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-17T04:29:04.267 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.268 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.268 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.268 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.268 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.268 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.268 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.268 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.268 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.269 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.269 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.269 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.269 INFO:teuthology.orchestra.run.smithi100.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-17T04:29:04.269 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-17T04:29:04.269 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.269 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.269 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.269 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.269 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.269 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.269 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.269 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.269 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.270 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.270 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.270 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.270 INFO:teuthology.orchestra.run.smithi100.stdout: "AUDIT_API_ENABLED": { 2024-09-17T04:29:04.270 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "AUDIT_API_ENABLED", 2024-09-17T04:29:04.270 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.270 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.270 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.270 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.270 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.270 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.270 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.270 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.270 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.271 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.271 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.271 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.271 INFO:teuthology.orchestra.run.smithi100.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-17T04:29:04.271 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-17T04:29:04.271 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.271 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.271 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.271 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.271 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.271 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.271 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.271 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.272 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.272 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.272 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.272 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.272 INFO:teuthology.orchestra.run.smithi100.stdout: "ENABLE_BROWSABLE_API": { 2024-09-17T04:29:04.272 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-17T04:29:04.272 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.272 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.272 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.272 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.272 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.272 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.272 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.272 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.273 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.273 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.273 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.273 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.273 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-17T04:29:04.273 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-17T04:29:04.273 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.273 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.273 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.273 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.273 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.273 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.273 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.274 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.274 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.274 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.274 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.274 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.274 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-17T04:29:04.274 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-17T04:29:04.274 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.274 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.274 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.274 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.274 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.274 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.274 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.274 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.275 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.275 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.275 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.275 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.275 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-17T04:29:04.275 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-17T04:29:04.275 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.275 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.275 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.275 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.275 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.275 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.275 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.275 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.276 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.276 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.276 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.276 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.276 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-17T04:29:04.276 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-17T04:29:04.276 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.276 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.276 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.276 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.276 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.276 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.276 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.277 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.277 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.277 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.277 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.277 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.277 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-17T04:29:04.277 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-17T04:29:04.277 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.277 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.277 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.277 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.277 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.277 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.278 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.278 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.278 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.278 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.278 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.278 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.278 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-17T04:29:04.278 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-17T04:29:04.278 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.278 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.278 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.278 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.278 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.278 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.279 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.279 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.279 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.279 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.279 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.279 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.279 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-17T04:29:04.279 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-17T04:29:04.279 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.279 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.279 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.279 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.279 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.279 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.280 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.280 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.280 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.280 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.280 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.280 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.280 INFO:teuthology.orchestra.run.smithi100.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-17T04:29:04.280 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-17T04:29:04.280 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.280 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.280 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.280 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.280 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.280 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.281 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.281 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.281 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.281 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.281 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.281 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.281 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_PASSWORD": { 2024-09-17T04:29:04.281 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-17T04:29:04.281 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.281 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.281 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.281 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "admin", 2024-09-17T04:29:04.281 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.282 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.282 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.282 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.282 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.282 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.282 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.282 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.282 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-17T04:29:04.282 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-17T04:29:04.282 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.282 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.282 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.282 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.282 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.283 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.283 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.283 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.283 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.283 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.283 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.283 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.283 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_URL": { 2024-09-17T04:29:04.283 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_URL", 2024-09-17T04:29:04.283 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.283 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.283 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.283 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.284 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.284 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.284 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.284 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.284 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.284 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.284 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.284 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.284 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_USERNAME": { 2024-09-17T04:29:04.284 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-17T04:29:04.284 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.284 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.284 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.285 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "admin", 2024-09-17T04:29:04.285 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.285 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.285 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.285 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.285 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.285 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.285 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.285 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.285 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-17T04:29:04.285 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-17T04:29:04.285 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.285 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.285 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.286 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.286 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.286 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.286 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.286 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.286 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.286 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.286 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.286 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.286 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-17T04:29:04.286 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-17T04:29:04.286 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.286 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.287 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.287 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.287 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.287 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.287 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.287 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.287 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.287 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.287 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.287 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.287 INFO:teuthology.orchestra.run.smithi100.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-17T04:29:04.287 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-17T04:29:04.287 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.287 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.288 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.288 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.288 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.288 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.288 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.288 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.288 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.288 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.288 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.288 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.288 INFO:teuthology.orchestra.run.smithi100.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-17T04:29:04.288 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-17T04:29:04.288 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.288 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.289 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.289 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.289 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.289 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.289 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.289 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.289 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.289 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.289 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.289 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.289 INFO:teuthology.orchestra.run.smithi100.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-17T04:29:04.289 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-17T04:29:04.289 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.289 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.290 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.290 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "[]", 2024-09-17T04:29:04.290 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.290 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.290 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.290 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.290 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.290 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.290 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.290 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.290 INFO:teuthology.orchestra.run.smithi100.stdout: "MULTICLUSTER_CONFIG": { 2024-09-17T04:29:04.290 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-17T04:29:04.290 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.291 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.291 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.291 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "{}", 2024-09-17T04:29:04.291 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.291 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.291 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.291 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.291 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.291 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.291 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.291 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.291 INFO:teuthology.orchestra.run.smithi100.stdout: "PROMETHEUS_API_HOST": { 2024-09-17T04:29:04.291 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-17T04:29:04.291 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.292 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.292 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.292 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.292 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.292 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.292 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.292 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.292 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.292 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.292 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.292 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.292 INFO:teuthology.orchestra.run.smithi100.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-17T04:29:04.292 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-17T04:29:04.292 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.293 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.293 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.293 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.293 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.293 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.293 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.293 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.293 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.293 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.293 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.293 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.293 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-17T04:29:04.293 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-17T04:29:04.293 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.294 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.294 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.294 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.294 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.294 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.294 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.294 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.294 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.294 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.294 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.294 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.294 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-17T04:29:04.294 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-17T04:29:04.295 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.295 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.295 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.295 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.295 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.295 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.295 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.295 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.295 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.295 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.295 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.295 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.295 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-17T04:29:04.295 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-17T04:29:04.296 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.296 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.296 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.296 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.296 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.296 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.296 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.296 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.296 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.296 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.296 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.296 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.296 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-17T04:29:04.296 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-17T04:29:04.297 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.297 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.297 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.297 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.297 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.297 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.297 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.297 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.297 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.297 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.297 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.297 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.297 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-17T04:29:04.298 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-17T04:29:04.298 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.298 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.298 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.298 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.298 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.298 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.298 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.298 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.298 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.298 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.298 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.298 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.298 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-17T04:29:04.299 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-17T04:29:04.299 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.299 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.299 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.299 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.299 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.299 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.299 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.299 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.299 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.299 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.299 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.299 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.300 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-17T04:29:04.300 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-17T04:29:04.300 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.300 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.300 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.300 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.300 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.300 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.300 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.300 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.300 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.300 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.300 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.300 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.301 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_ENABLED": { 2024-09-17T04:29:04.301 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-17T04:29:04.301 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.301 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.301 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.301 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.301 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.301 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.301 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.301 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.301 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.301 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.301 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.301 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.301 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-17T04:29:04.302 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-17T04:29:04.302 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.302 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.302 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.302 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-17T04:29:04.302 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.302 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.302 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.302 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.302 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.302 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.302 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.302 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.302 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-17T04:29:04.302 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-17T04:29:04.303 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.303 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.303 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.303 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-09-17T04:29:04.303 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.303 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.303 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.303 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.303 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.303 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.303 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.304 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-17T04:29:04.304 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-17T04:29:04.304 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.304 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.304 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.304 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8", 2024-09-17T04:29:04.304 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.304 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.304 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.304 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.304 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.304 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.304 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.305 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.305 INFO:teuthology.orchestra.run.smithi100.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-17T04:29:04.305 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-17T04:29:04.305 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.305 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.305 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.305 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "45", 2024-09-17T04:29:04.305 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.305 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.305 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.305 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.305 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.305 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.305 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.306 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_ACCESS_KEY": { 2024-09-17T04:29:04.306 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-17T04:29:04.306 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.306 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.306 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.306 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.306 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.306 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.306 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.306 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.306 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.306 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.306 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.307 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.307 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-17T04:29:04.307 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-17T04:29:04.307 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.307 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.307 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.307 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "admin", 2024-09-17T04:29:04.307 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.307 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.307 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.307 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.307 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.307 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.307 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.307 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.307 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_SECRET_KEY": { 2024-09-17T04:29:04.308 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-17T04:29:04.308 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.308 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.308 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.308 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.308 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.308 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.308 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.308 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.308 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.308 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.308 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.308 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.308 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_SSL_VERIFY": { 2024-09-17T04:29:04.309 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-17T04:29:04.309 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.309 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.309 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.309 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.309 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.309 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.309 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.309 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.309 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.309 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.309 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.309 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.310 INFO:teuthology.orchestra.run.smithi100.stdout: "UNSAFE_TLS_v1_2": { 2024-09-17T04:29:04.310 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-17T04:29:04.310 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.310 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.310 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.310 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.310 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.310 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.310 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.310 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.310 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.310 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.310 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.310 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.310 INFO:teuthology.orchestra.run.smithi100.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-17T04:29:04.310 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-17T04:29:04.311 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.311 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.311 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.311 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.311 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.311 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.311 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.311 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.311 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.311 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.311 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.311 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.311 INFO:teuthology.orchestra.run.smithi100.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-17T04:29:04.311 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-17T04:29:04.312 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.312 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.312 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.312 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-09-17T04:29:04.312 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.312 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.312 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.312 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.312 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.312 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.312 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.312 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.312 INFO:teuthology.orchestra.run.smithi100.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-17T04:29:04.313 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-17T04:29:04.313 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.313 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.313 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.313 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-09-17T04:29:04.313 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.313 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.313 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.313 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.313 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.313 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.313 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.313 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.313 INFO:teuthology.orchestra.run.smithi100.stdout: "cross_origin_url": { 2024-09-17T04:29:04.314 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cross_origin_url", 2024-09-17T04:29:04.314 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.314 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.314 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.314 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.314 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.314 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.314 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.314 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.314 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.314 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.314 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.315 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.315 INFO:teuthology.orchestra.run.smithi100.stdout: "crt_file": { 2024-09-17T04:29:04.315 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crt_file", 2024-09-17T04:29:04.315 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.315 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.315 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.315 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.315 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.315 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.315 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.315 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.315 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.315 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.316 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.316 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.316 INFO:teuthology.orchestra.run.smithi100.stdout: "debug": { 2024-09-17T04:29:04.316 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "debug", 2024-09-17T04:29:04.316 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.316 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.316 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.316 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.316 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.316 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.316 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.316 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Enable/disable debug options", 2024-09-17T04:29:04.316 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.316 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.317 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.317 INFO:teuthology.orchestra.run.smithi100.stdout: "jwt_token_ttl": { 2024-09-17T04:29:04.317 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "jwt_token_ttl", 2024-09-17T04:29:04.317 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.317 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.317 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.317 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "28800", 2024-09-17T04:29:04.317 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.317 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.317 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.318 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.318 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.318 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.318 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.318 INFO:teuthology.orchestra.run.smithi100.stdout: "key_file": { 2024-09-17T04:29:04.318 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "key_file", 2024-09-17T04:29:04.318 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.318 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.318 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.318 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.319 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.319 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.319 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.319 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.319 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.319 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.319 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.319 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.319 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.319 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.319 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.319 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.319 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.319 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.320 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.320 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.320 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.320 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.320 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.320 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.320 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.320 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.320 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.320 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.320 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.320 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.320 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.320 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.321 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.321 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.321 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.321 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.321 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.321 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.321 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.321 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.321 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.321 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.321 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.321 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.321 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.321 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.322 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.322 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.322 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.322 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.322 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.322 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.322 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.322 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.322 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.322 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.322 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.322 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.322 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.322 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.322 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.323 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.323 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.323 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.323 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.323 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.323 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.323 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.323 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.323 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.323 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.323 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.323 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.323 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.324 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.324 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.324 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.324 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.324 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.324 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.324 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.324 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.324 INFO:teuthology.orchestra.run.smithi100.stdout: "motd": { 2024-09-17T04:29:04.324 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "motd", 2024-09-17T04:29:04.324 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.324 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.324 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.325 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.325 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.325 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.325 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.325 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "The message of the day", 2024-09-17T04:29:04.325 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.325 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.325 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.325 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.325 INFO:teuthology.orchestra.run.smithi100.stdout: "redirect_resolve_ip_addr": { 2024-09-17T04:29:04.325 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "redirect_resolve_ip_addr", 2024-09-17T04:29:04.325 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.325 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.325 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.326 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.326 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.326 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.326 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.326 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.326 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.326 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.326 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.326 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.326 INFO:teuthology.orchestra.run.smithi100.stdout: "server_addr": { 2024-09-17T04:29:04.326 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_addr", 2024-09-17T04:29:04.326 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.326 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.326 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.327 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "::", 2024-09-17T04:29:04.327 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.327 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.327 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.327 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.327 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.327 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.327 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.327 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.327 INFO:teuthology.orchestra.run.smithi100.stdout: "server_port": { 2024-09-17T04:29:04.327 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_port", 2024-09-17T04:29:04.327 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.327 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.327 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.328 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8080", 2024-09-17T04:29:04.328 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.328 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.328 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.328 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.328 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.328 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.328 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.328 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.328 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.328 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.328 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.328 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.328 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.329 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.329 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.329 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.329 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.329 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.329 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.329 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.329 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.329 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.329 INFO:teuthology.orchestra.run.smithi100.stdout: "ssl": { 2024-09-17T04:29:04.329 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssl", 2024-09-17T04:29:04.329 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.329 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.329 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.329 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.330 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.330 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.330 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.330 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.330 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.330 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.330 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.330 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.330 INFO:teuthology.orchestra.run.smithi100.stdout: "ssl_server_port": { 2024-09-17T04:29:04.330 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssl_server_port", 2024-09-17T04:29:04.330 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.330 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.330 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.330 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8443", 2024-09-17T04:29:04.331 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.331 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.331 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.331 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.331 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.331 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.331 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.331 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.331 INFO:teuthology.orchestra.run.smithi100.stdout: "sso_oauth2": { 2024-09-17T04:29:04.331 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sso_oauth2", 2024-09-17T04:29:04.331 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.331 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.331 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.332 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.332 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.332 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.332 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.332 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.332 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.332 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.332 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.332 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.332 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_behaviour": { 2024-09-17T04:29:04.332 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_behaviour", 2024-09-17T04:29:04.332 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.332 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.333 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.333 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "redirect", 2024-09-17T04:29:04.333 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.333 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.333 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.333 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.333 INFO:teuthology.orchestra.run.smithi100.stdout: "redirect" 2024-09-17T04:29:04.333 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.333 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.333 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.333 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.333 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.333 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.334 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_error_status_code": { 2024-09-17T04:29:04.334 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_error_status_code", 2024-09-17T04:29:04.334 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.334 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.334 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.334 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "500", 2024-09-17T04:29:04.334 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "400", 2024-09-17T04:29:04.334 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "599", 2024-09-17T04:29:04.334 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.334 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.334 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.334 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.334 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.334 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.335 INFO:teuthology.orchestra.run.smithi100.stdout: "url_prefix": { 2024-09-17T04:29:04.335 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "url_prefix", 2024-09-17T04:29:04.335 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.335 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.335 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.335 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.335 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.335 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.335 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.335 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.335 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.335 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.335 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.335 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.336 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.336 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.336 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.336 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "devicehealth", 2024-09-17T04:29:04.336 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.336 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.336 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.336 INFO:teuthology.orchestra.run.smithi100.stdout: "enable_monitoring": { 2024-09-17T04:29:04.336 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enable_monitoring", 2024-09-17T04:29:04.336 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.336 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.336 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.336 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.337 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.337 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.337 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.337 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "monitor device health metrics", 2024-09-17T04:29:04.337 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.337 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.337 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.337 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.337 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.337 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.337 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.337 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.337 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.337 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.338 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.338 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.338 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.338 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.338 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.338 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.338 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.338 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.338 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.338 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.339 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.339 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.339 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.339 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.339 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.339 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.339 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.339 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.339 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.339 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.339 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.339 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.339 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.339 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.340 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.340 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.340 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.340 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.340 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.340 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.340 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.340 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.340 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.340 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.340 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.340 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.340 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.340 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.341 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.341 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.341 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.341 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.341 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.341 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.341 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.341 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.341 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.341 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.341 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.341 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.341 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.341 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.342 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.342 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.342 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.342 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.342 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.342 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.342 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.342 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.342 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.342 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.342 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.342 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.342 INFO:teuthology.orchestra.run.smithi100.stdout: "mark_out_threshold": { 2024-09-17T04:29:04.342 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mark_out_threshold", 2024-09-17T04:29:04.343 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.343 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.343 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.343 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "2419200", 2024-09-17T04:29:04.343 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.343 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.343 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.343 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-17T04:29:04.343 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.343 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.343 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.343 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.343 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_name": { 2024-09-17T04:29:04.343 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pool_name", 2024-09-17T04:29:04.344 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.344 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.344 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.344 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "device_health_metrics", 2024-09-17T04:29:04.344 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.344 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.344 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.344 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-17T04:29:04.344 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.344 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.344 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.344 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.344 INFO:teuthology.orchestra.run.smithi100.stdout: "retention_period": { 2024-09-17T04:29:04.345 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "retention_period", 2024-09-17T04:29:04.345 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.345 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.345 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.345 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "15552000", 2024-09-17T04:29:04.345 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.345 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.345 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.345 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how long to retain device health metrics", 2024-09-17T04:29:04.345 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.345 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.345 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.345 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.346 INFO:teuthology.orchestra.run.smithi100.stdout: "scrape_frequency": { 2024-09-17T04:29:04.346 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "scrape_frequency", 2024-09-17T04:29:04.346 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.346 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.346 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.346 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "86400", 2024-09-17T04:29:04.346 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.346 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.346 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.346 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-17T04:29:04.346 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.346 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.346 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.347 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.347 INFO:teuthology.orchestra.run.smithi100.stdout: "self_heal": { 2024-09-17T04:29:04.347 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "self_heal", 2024-09-17T04:29:04.347 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.347 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.347 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.347 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.347 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.347 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.347 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.347 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-17T04:29:04.347 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.347 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.347 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.348 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.348 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-09-17T04:29:04.348 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-09-17T04:29:04.348 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.348 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.348 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.348 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-09-17T04:29:04.348 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.348 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.348 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.348 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to wake up and check device health", 2024-09-17T04:29:04.348 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.348 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.348 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.349 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.349 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.349 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.349 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.349 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.349 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.349 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.349 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.349 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.349 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.349 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.349 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.349 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.349 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.350 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.350 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_threshold": { 2024-09-17T04:29:04.350 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_threshold", 2024-09-17T04:29:04.350 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.350 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.350 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.350 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "7257600", 2024-09-17T04:29:04.350 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.350 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.350 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.350 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-17T04:29:04.350 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.350 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.350 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.351 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.351 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.351 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.351 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.351 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "diskprediction_local", 2024-09-17T04:29:04.351 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.351 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.351 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.351 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.351 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.351 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.351 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.351 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.352 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.352 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.352 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.352 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.352 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.352 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.352 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.352 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.352 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.352 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.352 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.352 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.352 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.352 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.353 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.353 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.353 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.353 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.353 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.353 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.353 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.353 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.353 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.353 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.353 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.353 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.353 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.353 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.354 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.354 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.354 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.354 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.354 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.354 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.354 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.354 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.354 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.354 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.354 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.354 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.356 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.356 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.356 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.356 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.356 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.356 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.356 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.356 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.357 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.357 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.357 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.357 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.357 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.357 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.357 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.357 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.357 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.357 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.357 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.357 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.357 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.358 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.358 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.358 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.358 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.358 INFO:teuthology.orchestra.run.smithi100.stdout: "predict_interval": { 2024-09-17T04:29:04.358 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "predict_interval", 2024-09-17T04:29:04.358 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.358 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.358 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.358 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "86400", 2024-09-17T04:29:04.358 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.358 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.358 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.358 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.359 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.359 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.359 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.359 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.359 INFO:teuthology.orchestra.run.smithi100.stdout: "predictor_model": { 2024-09-17T04:29:04.359 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "predictor_model", 2024-09-17T04:29:04.359 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.359 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.359 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.359 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "prophetstor", 2024-09-17T04:29:04.359 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.359 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.359 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.359 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.360 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.360 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.360 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.360 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.360 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-09-17T04:29:04.360 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-09-17T04:29:04.360 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.360 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.360 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.360 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-09-17T04:29:04.360 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.360 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.361 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.361 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.361 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.361 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.361 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.361 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.361 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.361 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.361 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.361 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.361 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.361 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.362 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.362 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.362 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.362 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.362 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.362 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.362 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.362 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.362 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.362 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.362 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.362 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "influx", 2024-09-17T04:29:04.362 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": false, 2024-09-17T04:29:04.363 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "influxdb python module not found", 2024-09-17T04:29:04.363 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.363 INFO:teuthology.orchestra.run.smithi100.stdout: "batch_size": { 2024-09-17T04:29:04.363 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "batch_size", 2024-09-17T04:29:04.363 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.363 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.363 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.363 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5000", 2024-09-17T04:29:04.363 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.363 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.363 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.363 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-17T04:29:04.363 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.363 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.364 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.364 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.364 INFO:teuthology.orchestra.run.smithi100.stdout: "database": { 2024-09-17T04:29:04.364 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "database", 2024-09-17T04:29:04.364 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.364 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.364 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.364 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "ceph", 2024-09-17T04:29:04.364 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.364 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.364 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.364 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-17T04:29:04.365 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.365 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.365 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.365 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.365 INFO:teuthology.orchestra.run.smithi100.stdout: "hostname": { 2024-09-17T04:29:04.365 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "hostname", 2024-09-17T04:29:04.365 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.365 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.365 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.365 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.365 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.365 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.365 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.365 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "InfluxDB server hostname", 2024-09-17T04:29:04.366 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.366 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.366 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.366 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.366 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-09-17T04:29:04.366 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-09-17T04:29:04.366 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.366 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.366 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.366 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "30", 2024-09-17T04:29:04.366 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "5", 2024-09-17T04:29:04.366 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.366 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.367 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-17T04:29:04.367 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.367 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.367 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.367 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.367 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.367 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.367 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.367 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.367 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.367 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.367 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.367 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.367 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.368 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.368 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.368 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.368 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.368 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.368 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.368 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.368 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.368 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.368 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.368 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.368 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.368 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.369 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.369 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.369 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.369 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.369 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.369 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.369 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.369 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.369 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.369 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.369 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.369 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.369 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.370 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.370 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.370 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.370 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.370 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.370 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.370 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.370 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.370 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.370 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.370 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.370 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.370 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.371 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.371 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.371 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.371 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.371 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.371 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.371 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.371 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.371 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.371 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.371 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.371 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.371 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.371 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.372 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.372 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.372 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.372 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.372 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.372 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.372 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.372 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.372 INFO:teuthology.orchestra.run.smithi100.stdout: "password": { 2024-09-17T04:29:04.372 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "password", 2024-09-17T04:29:04.372 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.372 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.372 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.373 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.373 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.373 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.373 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.373 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "password of InfluxDB server user", 2024-09-17T04:29:04.373 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.373 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.373 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.373 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.373 INFO:teuthology.orchestra.run.smithi100.stdout: "port": { 2024-09-17T04:29:04.374 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "port", 2024-09-17T04:29:04.374 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.374 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.374 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.374 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8086", 2024-09-17T04:29:04.374 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.374 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.374 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.374 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "InfluxDB server port", 2024-09-17T04:29:04.374 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.374 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.374 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.374 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.374 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.375 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.375 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.375 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.375 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.375 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.375 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.375 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.375 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.375 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.375 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.375 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.376 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.376 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.376 INFO:teuthology.orchestra.run.smithi100.stdout: "ssl": { 2024-09-17T04:29:04.376 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssl", 2024-09-17T04:29:04.376 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.376 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.376 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.376 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "false", 2024-09-17T04:29:04.376 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.376 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.376 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.376 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-17T04:29:04.376 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.377 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.377 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.377 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.377 INFO:teuthology.orchestra.run.smithi100.stdout: "threads": { 2024-09-17T04:29:04.377 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "threads", 2024-09-17T04:29:04.377 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.377 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.377 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.377 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-09-17T04:29:04.377 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-09-17T04:29:04.377 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "32", 2024-09-17T04:29:04.377 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.377 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-17T04:29:04.377 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.378 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.378 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.378 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.378 INFO:teuthology.orchestra.run.smithi100.stdout: "username": { 2024-09-17T04:29:04.378 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "username", 2024-09-17T04:29:04.378 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.378 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.378 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.378 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.378 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.378 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.378 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.378 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "username of InfluxDB server user", 2024-09-17T04:29:04.378 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.379 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.379 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.379 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.379 INFO:teuthology.orchestra.run.smithi100.stdout: "verify_ssl": { 2024-09-17T04:29:04.379 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "verify_ssl", 2024-09-17T04:29:04.379 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.379 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.379 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.379 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "true", 2024-09-17T04:29:04.379 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.379 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.379 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.379 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-17T04:29:04.379 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.380 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.380 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.380 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.380 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.380 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.380 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.380 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "insights", 2024-09-17T04:29:04.380 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.380 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.380 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.380 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.380 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.380 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.381 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.381 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.381 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.381 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.381 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.381 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.381 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.381 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.381 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.381 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.381 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.381 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.382 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.382 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.382 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.382 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.382 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.382 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.382 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.382 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.382 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.382 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.382 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.382 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.382 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.382 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.383 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.383 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.383 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.383 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.383 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.383 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.383 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.383 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.383 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.383 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.383 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.383 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.384 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.384 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.384 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.384 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.384 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.384 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.384 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.384 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.384 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.384 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.384 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.384 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.384 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.384 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.385 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.385 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.385 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.385 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.385 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.385 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.385 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.385 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.385 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.385 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.385 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.385 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.385 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.386 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.386 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.386 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.386 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.386 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.386 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.386 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.386 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.386 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.386 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.386 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.386 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.386 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.386 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.387 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.387 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.387 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.387 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.387 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.387 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "iostat", 2024-09-17T04:29:04.387 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.387 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.387 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.387 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.387 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.387 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.388 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.388 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.388 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.388 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.388 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.388 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.388 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.388 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.388 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.388 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.388 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.388 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.389 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.389 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.389 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.389 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.389 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.389 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.389 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.389 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.389 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.389 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.389 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.389 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.389 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.390 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.390 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.390 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.390 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.390 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.390 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.390 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.390 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.390 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.390 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.390 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.390 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.390 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.391 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.391 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.391 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.391 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.391 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.391 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.391 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.391 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.391 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.391 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.391 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.391 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.391 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.391 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.392 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.392 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.392 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.392 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.392 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.392 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.392 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.392 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.392 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.392 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.392 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.392 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.392 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.393 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.393 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.393 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.393 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.393 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.393 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.393 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.393 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.393 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.393 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.393 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.393 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.393 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.394 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.394 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.394 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.394 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.394 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.394 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.394 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "localpool", 2024-09-17T04:29:04.394 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.394 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.394 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.394 INFO:teuthology.orchestra.run.smithi100.stdout: "failure_domain": { 2024-09-17T04:29:04.394 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "failure_domain", 2024-09-17T04:29:04.394 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.395 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.395 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.395 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "host", 2024-09-17T04:29:04.395 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.395 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.395 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.395 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "failure domain for any created local pool", 2024-09-17T04:29:04.395 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-17T04:29:04.395 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.395 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.395 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.395 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.395 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.396 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.396 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.396 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.396 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.396 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.396 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.396 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.396 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.396 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.396 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.396 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.396 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.396 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.396 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.397 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.397 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.397 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.397 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.397 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.397 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.397 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.397 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.397 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.397 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.397 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.397 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.397 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.397 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.398 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.398 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.398 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.398 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.398 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.398 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.398 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.398 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.398 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.398 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.398 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.398 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.399 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.399 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.399 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.399 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.399 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.399 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.399 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.399 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.399 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.399 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.399 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.399 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.399 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.399 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.399 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.400 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.400 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.400 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.400 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.400 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.400 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.400 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.400 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.400 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.400 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.400 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.400 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.400 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.401 INFO:teuthology.orchestra.run.smithi100.stdout: "min_size": { 2024-09-17T04:29:04.401 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "min_size", 2024-09-17T04:29:04.401 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.401 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.401 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.401 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.401 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.401 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.401 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.401 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "default min_size for any created local pool", 2024-09-17T04:29:04.401 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-17T04:29:04.401 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.401 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.402 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.402 INFO:teuthology.orchestra.run.smithi100.stdout: "num_rep": { 2024-09-17T04:29:04.402 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "num_rep", 2024-09-17T04:29:04.402 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.402 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.402 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.402 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "3", 2024-09-17T04:29:04.402 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.402 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.402 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.402 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "default replica count for any created local pool", 2024-09-17T04:29:04.402 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.402 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.402 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.403 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.403 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num": { 2024-09-17T04:29:04.403 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pg_num", 2024-09-17T04:29:04.403 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.403 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.403 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.403 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "128", 2024-09-17T04:29:04.403 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.403 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.403 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.403 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "default pg_num for any created local pool", 2024-09-17T04:29:04.403 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.403 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.404 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.404 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.404 INFO:teuthology.orchestra.run.smithi100.stdout: "prefix": { 2024-09-17T04:29:04.404 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "prefix", 2024-09-17T04:29:04.404 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.404 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.404 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.404 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.404 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.404 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.404 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.404 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "name prefix for any created local pool", 2024-09-17T04:29:04.404 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.405 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.405 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.405 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.405 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.405 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.405 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.405 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.405 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.405 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.405 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.405 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.405 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.405 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.406 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.406 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.406 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.406 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.406 INFO:teuthology.orchestra.run.smithi100.stdout: "subtree": { 2024-09-17T04:29:04.406 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "subtree", 2024-09-17T04:29:04.406 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.406 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.406 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.406 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "rack", 2024-09-17T04:29:04.406 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.406 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.406 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.406 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-17T04:29:04.407 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-17T04:29:04.407 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.407 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.407 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.407 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.407 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.407 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.407 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mds_autoscaler", 2024-09-17T04:29:04.407 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.407 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.407 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.407 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.407 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.407 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.408 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.408 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.408 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.408 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.408 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.408 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.408 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.408 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.408 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.408 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.408 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.408 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.408 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.408 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.409 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.409 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.409 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.409 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.409 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.409 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.409 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.409 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.409 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.409 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.409 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.409 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.410 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.410 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.410 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.410 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.410 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.410 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.410 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.410 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.410 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.410 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.410 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.410 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.410 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.410 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.411 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.411 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.411 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.411 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.411 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.411 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.411 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.411 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.411 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.411 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.411 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.411 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.411 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.412 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.412 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.412 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.412 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.412 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.412 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.412 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.412 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.412 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.412 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.412 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.412 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.412 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.413 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.413 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.413 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.413 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.413 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.413 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.413 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.413 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.413 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.413 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.413 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.413 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.413 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.413 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.414 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.414 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.414 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.414 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.414 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mirroring", 2024-09-17T04:29:04.414 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.414 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.414 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.414 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.414 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.414 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.414 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.414 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.415 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.415 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.415 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.415 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.415 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.415 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.415 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.415 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.415 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.415 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.415 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.415 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.415 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.416 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.416 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.416 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.416 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.416 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.416 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.416 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.416 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.416 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.416 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.416 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.416 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.417 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.417 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.417 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.417 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.417 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.417 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.417 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.417 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.417 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.417 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.417 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.417 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.417 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.418 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.418 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.418 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.418 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.418 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.418 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.418 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.418 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.418 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.418 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.418 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.418 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.419 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.419 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.419 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.419 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.419 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.419 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.419 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.419 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.419 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.419 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.419 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.419 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.420 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.420 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.420 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.420 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.420 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.420 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.420 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.420 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.420 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.420 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.420 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.420 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.421 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.421 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.421 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.421 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.421 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.421 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.421 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.421 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.421 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "nfs", 2024-09-17T04:29:04.421 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.421 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.421 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.422 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.422 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.422 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.422 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.422 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.422 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.422 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.422 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.422 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.422 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.422 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.422 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.422 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.423 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.423 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.423 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.423 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.423 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.423 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.423 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.423 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.423 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.423 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.423 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.423 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.423 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.424 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.424 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.424 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.424 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.424 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.424 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.424 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.424 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.424 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.424 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.424 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.424 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.425 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.425 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.425 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.425 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.425 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.425 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.425 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.425 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.425 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.425 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.425 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.425 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.425 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.426 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.426 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.426 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.426 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.426 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.426 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.426 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.426 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.426 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.426 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.426 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.426 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.426 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.426 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.427 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.427 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.427 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.427 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.427 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.427 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.427 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.427 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.427 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.427 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.427 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.427 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.427 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.427 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.428 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.428 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.428 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.428 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.428 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.428 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.428 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.428 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.428 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "orchestrator", 2024-09-17T04:29:04.428 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.428 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.428 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.428 INFO:teuthology.orchestra.run.smithi100.stdout: "fail_fs": { 2024-09-17T04:29:04.428 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "fail_fs", 2024-09-17T04:29:04.429 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.429 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.429 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.429 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.429 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.429 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.429 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.429 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-17T04:29:04.429 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.429 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.429 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.429 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.429 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.430 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.430 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.430 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.430 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.430 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.430 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.430 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.430 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.430 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.430 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.430 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.430 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.430 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.430 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.430 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.431 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.431 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.431 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.431 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.431 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.431 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.431 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.431 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.431 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.431 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.431 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.431 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.431 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.432 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.432 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.432 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.432 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.432 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.432 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.432 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.432 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.432 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.432 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.432 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.432 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.432 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.432 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.433 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.433 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.433 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.433 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.433 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.433 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.433 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.433 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.433 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.433 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.433 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.433 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.434 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.434 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.434 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.434 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.434 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.434 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.434 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.434 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.434 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.434 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.434 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.434 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.434 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.435 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.435 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.435 INFO:teuthology.orchestra.run.smithi100.stdout: "orchestrator": { 2024-09-17T04:29:04.435 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "orchestrator", 2024-09-17T04:29:04.435 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.435 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.435 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.435 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.435 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.435 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.435 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.435 INFO:teuthology.orchestra.run.smithi100.stdout: "cephadm", 2024-09-17T04:29:04.435 INFO:teuthology.orchestra.run.smithi100.stdout: "rook", 2024-09-17T04:29:04.435 INFO:teuthology.orchestra.run.smithi100.stdout: "test_orchestrator" 2024-09-17T04:29:04.435 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.436 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Orchestrator backend", 2024-09-17T04:29:04.436 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.436 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.436 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.436 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.436 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.436 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.436 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.436 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.436 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.436 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.436 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.436 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.436 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.437 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.437 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.437 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.437 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.437 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.437 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.437 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.437 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.437 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "osd_perf_query", 2024-09-17T04:29:04.437 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.437 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.437 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.437 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.437 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.438 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.438 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.438 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.438 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.438 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.438 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.438 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.438 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.438 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.439 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.439 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.439 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.439 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.439 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.439 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.439 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.439 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.439 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.439 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.439 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.439 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.439 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.439 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.439 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.440 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.440 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.440 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.440 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.440 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.440 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.440 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.440 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.440 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.440 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.440 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.440 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.440 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.441 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.441 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.441 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.441 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.441 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.441 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.441 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.441 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.441 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.441 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.441 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.441 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.441 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.441 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.441 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.442 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.442 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.442 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.442 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.442 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.442 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.442 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.442 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.442 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.442 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.442 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.442 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.442 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.442 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.442 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.443 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.443 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.443 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.443 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.443 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.443 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.443 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.443 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.443 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.443 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.443 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.443 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.443 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.443 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.444 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.444 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.444 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.444 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.444 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "osd_support", 2024-09-17T04:29:04.444 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.444 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.444 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.444 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.444 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.444 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.444 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.444 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.444 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.445 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.445 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.445 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.445 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.445 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.445 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.445 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.445 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.445 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.445 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.445 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.445 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.445 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.445 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.446 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.446 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.446 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.446 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.446 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.446 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.446 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.446 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.446 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.446 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.446 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.446 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.446 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.446 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.446 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.447 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.447 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.447 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.447 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.447 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.447 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.447 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.447 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.447 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.447 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.447 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.447 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.447 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.448 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.448 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.448 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.448 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.448 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.448 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.448 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.448 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.448 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.448 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.448 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.448 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.448 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.448 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.449 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.449 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.449 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.449 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.449 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.449 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.449 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.449 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.449 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.449 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.449 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.449 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.449 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.449 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.450 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.450 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.450 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.450 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.450 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.450 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.450 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.450 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.450 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.450 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.450 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.450 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pg_autoscaler", 2024-09-17T04:29:04.450 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.450 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.451 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.451 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.451 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.451 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.451 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.451 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.451 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.451 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.451 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.451 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.451 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.451 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.451 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.452 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.452 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.452 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.452 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.452 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.452 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.452 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.452 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.452 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.452 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.452 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.452 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.452 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.453 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.453 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.453 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.453 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.453 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.453 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.453 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.453 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.453 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.453 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.453 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.453 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.453 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.454 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.454 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.454 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.454 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.454 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.454 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.454 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.454 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.454 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.454 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.454 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.454 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.454 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.455 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.455 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.455 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.455 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.455 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.455 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.455 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.455 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.455 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.455 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.455 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.455 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.455 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.456 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.456 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.456 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.456 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.456 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.456 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.456 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-09-17T04:29:04.456 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-09-17T04:29:04.456 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.456 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.456 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.456 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-09-17T04:29:04.456 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.456 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.457 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.457 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.457 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.457 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.457 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.457 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.457 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.457 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.457 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.457 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.457 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.457 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.457 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.458 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.458 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.458 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.458 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.458 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.458 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.458 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.458 INFO:teuthology.orchestra.run.smithi100.stdout: "threshold": { 2024-09-17T04:29:04.458 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "threshold", 2024-09-17T04:29:04.458 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-09-17T04:29:04.458 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.458 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.458 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "3.0", 2024-09-17T04:29:04.459 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1.0", 2024-09-17T04:29:04.459 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.459 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.459 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "scaling threshold", 2024-09-17T04:29:04.459 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-17T04:29:04.459 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.459 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.459 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.459 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.459 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.459 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.459 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "progress", 2024-09-17T04:29:04.459 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.459 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.460 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.460 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_pg_recovery_event": { 2024-09-17T04:29:04.460 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "allow_pg_recovery_event", 2024-09-17T04:29:04.460 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.460 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.460 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.460 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.460 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.460 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.460 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.460 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-17T04:29:04.460 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.460 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.461 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.461 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.461 INFO:teuthology.orchestra.run.smithi100.stdout: "enabled": { 2024-09-17T04:29:04.461 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enabled", 2024-09-17T04:29:04.461 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.461 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.461 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.461 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.461 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.461 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.461 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.461 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.461 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.461 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.462 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.462 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.462 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.462 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.462 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.462 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.462 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.462 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.462 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.462 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.462 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.462 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.462 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.462 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.463 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.463 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.463 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.463 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.463 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.463 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.463 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.463 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.463 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.463 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.463 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.463 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.463 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.463 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.464 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.464 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.464 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.464 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.464 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.464 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.464 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.464 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.464 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.464 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.464 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.464 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.464 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.465 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.465 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.465 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.465 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.465 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.465 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.465 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.465 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.465 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.465 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.465 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.465 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.465 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.465 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.466 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.466 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.466 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.466 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.466 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.466 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.466 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.466 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.466 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.466 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.466 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.466 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.466 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.466 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.466 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.467 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.467 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.467 INFO:teuthology.orchestra.run.smithi100.stdout: "max_completed_events": { 2024-09-17T04:29:04.467 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_completed_events", 2024-09-17T04:29:04.467 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.467 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.467 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.467 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "50", 2024-09-17T04:29:04.467 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.467 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.467 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.467 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "number of past completed events to remember", 2024-09-17T04:29:04.467 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.467 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.468 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.468 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-09-17T04:29:04.468 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-09-17T04:29:04.468 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.468 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.468 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.468 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-09-17T04:29:04.468 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.468 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.468 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.468 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how long the module is going to sleep", 2024-09-17T04:29:04.468 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.468 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.469 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.469 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.469 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.469 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.469 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.469 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.469 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.469 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.469 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.469 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.469 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.469 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.469 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.470 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.470 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.470 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.470 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.470 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.470 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.470 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "prometheus", 2024-09-17T04:29:04.470 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.470 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.470 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.470 INFO:teuthology.orchestra.run.smithi100.stdout: "cache": { 2024-09-17T04:29:04.470 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cache", 2024-09-17T04:29:04.470 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.470 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.471 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.471 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.471 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.471 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.471 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.471 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.471 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.471 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.471 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.471 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.471 INFO:teuthology.orchestra.run.smithi100.stdout: "exclude_perf_counters": { 2024-09-17T04:29:04.471 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "exclude_perf_counters", 2024-09-17T04:29:04.471 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.471 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.472 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.472 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.472 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.472 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.472 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.472 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-17T04:29:04.472 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-17T04:29:04.472 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.472 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.472 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.472 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.472 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.472 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.473 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.473 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.473 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.473 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.473 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.473 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.473 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.473 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.473 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.473 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.473 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.473 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.473 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.473 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.474 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.474 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.474 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.474 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.474 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.474 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.474 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.474 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.474 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.474 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.474 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.474 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.474 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.474 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.475 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.475 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.475 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.475 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.475 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.475 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.475 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.475 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.475 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.475 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.476 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.476 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.476 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.476 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.476 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.476 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.476 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.476 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.476 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.476 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.476 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.476 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.476 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.477 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.477 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.477 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.477 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.477 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.477 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.477 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.477 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.477 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.477 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.477 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.477 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.477 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.478 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.478 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.478 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.478 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_stats_pools": { 2024-09-17T04:29:04.478 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rbd_stats_pools", 2024-09-17T04:29:04.478 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.478 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.478 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.478 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.478 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.478 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.478 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.478 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.478 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.479 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.479 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.479 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.479 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-17T04:29:04.479 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-17T04:29:04.479 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.479 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.479 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.479 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "300", 2024-09-17T04:29:04.479 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.479 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.479 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.479 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.479 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.480 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.480 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.480 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.480 INFO:teuthology.orchestra.run.smithi100.stdout: "scrape_interval": { 2024-09-17T04:29:04.480 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "scrape_interval", 2024-09-17T04:29:04.480 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-09-17T04:29:04.480 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.480 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.480 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "15.0", 2024-09-17T04:29:04.480 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.480 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.480 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.480 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.481 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.481 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.481 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.481 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.481 INFO:teuthology.orchestra.run.smithi100.stdout: "server_addr": { 2024-09-17T04:29:04.481 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_addr", 2024-09-17T04:29:04.481 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.481 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.481 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.481 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "::", 2024-09-17T04:29:04.481 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.481 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.481 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.481 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-17T04:29:04.482 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.482 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.482 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.482 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.482 INFO:teuthology.orchestra.run.smithi100.stdout: "server_port": { 2024-09-17T04:29:04.482 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_port", 2024-09-17T04:29:04.482 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.482 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.482 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.482 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "9283", 2024-09-17T04:29:04.482 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.482 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.482 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.483 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-17T04:29:04.483 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.483 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.483 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.483 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.483 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.483 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.483 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.483 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.483 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.483 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.483 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.484 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.484 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.484 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.484 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.484 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.484 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.484 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.484 INFO:teuthology.orchestra.run.smithi100.stdout: "stale_cache_strategy": { 2024-09-17T04:29:04.484 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "stale_cache_strategy", 2024-09-17T04:29:04.484 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.484 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.484 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.484 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "log", 2024-09-17T04:29:04.484 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.485 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.485 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.485 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.485 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.485 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.485 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.485 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.485 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_behaviour": { 2024-09-17T04:29:04.485 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_behaviour", 2024-09-17T04:29:04.485 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.485 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.485 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.485 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "default", 2024-09-17T04:29:04.486 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.486 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.486 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.486 INFO:teuthology.orchestra.run.smithi100.stdout: "default", 2024-09-17T04:29:04.486 INFO:teuthology.orchestra.run.smithi100.stdout: "error" 2024-09-17T04:29:04.486 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.486 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.486 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.486 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.486 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.486 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.486 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_error_status_code": { 2024-09-17T04:29:04.486 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_error_status_code", 2024-09-17T04:29:04.486 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.487 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.487 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.487 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "500", 2024-09-17T04:29:04.487 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "400", 2024-09-17T04:29:04.487 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "599", 2024-09-17T04:29:04.487 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.487 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.487 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.487 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.487 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.487 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.487 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.487 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.487 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.488 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rbd_support", 2024-09-17T04:29:04.488 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.488 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.488 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.488 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.488 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.488 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.488 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.488 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.488 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.488 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.488 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.488 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.489 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.489 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.489 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.489 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.489 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.489 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.489 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.489 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.489 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.489 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.489 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.489 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.489 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.490 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.490 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.490 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.490 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.490 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.490 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.490 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.490 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.490 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.490 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.490 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.490 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.490 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.491 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.491 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.491 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.491 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.491 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.491 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.491 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.491 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.491 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.491 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.491 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.491 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.491 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.492 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.492 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.492 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.492 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.492 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.492 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.492 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.492 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.492 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.492 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.492 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.492 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.493 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.493 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.493 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.493 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.493 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.493 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.493 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.493 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.493 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.493 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.493 INFO:teuthology.orchestra.run.smithi100.stdout: "max_concurrent_snap_create": { 2024-09-17T04:29:04.493 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_concurrent_snap_create", 2024-09-17T04:29:04.493 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.494 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.494 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.494 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-09-17T04:29:04.494 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.494 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.494 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.494 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.494 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.494 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.494 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.494 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.494 INFO:teuthology.orchestra.run.smithi100.stdout: "mirror_snapshot_schedule": { 2024-09-17T04:29:04.494 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mirror_snapshot_schedule", 2024-09-17T04:29:04.494 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.495 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.495 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.495 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.495 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.495 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.495 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.495 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.495 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.495 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.495 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.495 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.495 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.495 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.496 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.496 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.496 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.496 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.496 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.496 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.496 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.496 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.496 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.496 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.496 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.496 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.496 INFO:teuthology.orchestra.run.smithi100.stdout: "trash_purge_schedule": { 2024-09-17T04:29:04.496 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "trash_purge_schedule", 2024-09-17T04:29:04.497 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.497 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.497 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.497 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.497 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.497 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.497 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.497 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.497 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.497 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.497 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.497 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.497 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.498 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.498 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.498 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "restful", 2024-09-17T04:29:04.498 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.498 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.498 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.498 INFO:teuthology.orchestra.run.smithi100.stdout: "enable_auth": { 2024-09-17T04:29:04.498 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enable_auth", 2024-09-17T04:29:04.498 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.498 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.498 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.498 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.498 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.499 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.499 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.499 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.499 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.499 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.499 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.499 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.499 INFO:teuthology.orchestra.run.smithi100.stdout: "key_file": { 2024-09-17T04:29:04.499 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "key_file", 2024-09-17T04:29:04.499 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.499 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.499 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.499 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.500 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.500 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.500 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.500 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.500 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.500 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.500 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.500 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.500 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.500 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.500 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.500 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.500 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.500 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.501 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.501 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.501 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.501 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.501 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.501 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.501 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.501 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.501 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.501 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.501 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.501 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.501 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.501 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.502 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.502 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.502 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.502 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.502 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.502 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.502 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.502 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.502 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.502 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.502 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.502 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.502 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.502 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.503 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.503 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.503 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.503 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.503 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.503 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.503 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.503 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.503 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.503 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.503 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.503 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.503 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.503 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.504 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.504 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.504 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.504 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.504 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.504 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.504 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.504 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.504 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.504 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.504 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.504 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.505 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.505 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.505 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.505 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.505 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.505 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.505 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.505 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.505 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.505 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.505 INFO:teuthology.orchestra.run.smithi100.stdout: "max_requests": { 2024-09-17T04:29:04.505 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_requests", 2024-09-17T04:29:04.505 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.506 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.506 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.506 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "500", 2024-09-17T04:29:04.506 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.506 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.506 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.506 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-17T04:29:04.506 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.506 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.506 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.506 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.506 INFO:teuthology.orchestra.run.smithi100.stdout: "server_addr": { 2024-09-17T04:29:04.506 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_addr", 2024-09-17T04:29:04.507 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.507 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.507 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.507 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.507 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.507 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.507 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.507 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.507 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.507 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.507 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.507 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.507 INFO:teuthology.orchestra.run.smithi100.stdout: "server_port": { 2024-09-17T04:29:04.507 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_port", 2024-09-17T04:29:04.508 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.508 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.508 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.508 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.508 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.508 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.508 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.508 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.508 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.508 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.508 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.508 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.508 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.508 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.508 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.509 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.509 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.509 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.509 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.509 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.509 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.509 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.509 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.509 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.509 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.509 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.509 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.509 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.509 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.510 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rgw", 2024-09-17T04:29:04.510 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.510 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.510 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.510 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.510 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.510 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.510 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.510 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.510 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.510 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.510 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.510 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.511 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.511 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.511 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.511 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.511 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.511 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.511 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.511 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.511 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.511 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.511 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.511 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.511 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.511 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.512 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.512 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.512 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.512 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.512 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.512 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.512 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.512 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.512 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.512 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.512 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.512 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.512 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.512 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.513 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.513 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.513 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.513 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.513 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.513 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.513 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.513 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.513 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.513 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.513 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.513 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.513 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.513 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.514 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.514 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.514 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.514 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.514 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.514 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.514 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.514 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.514 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.514 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.514 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.514 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.514 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.514 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.514 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.515 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.515 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.515 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.515 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.515 INFO:teuthology.orchestra.run.smithi100.stdout: "secondary_zone_period_retry_limit": { 2024-09-17T04:29:04.515 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-17T04:29:04.515 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.515 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.515 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.515 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-09-17T04:29:04.515 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.515 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.515 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.515 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-17T04:29:04.515 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.515 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.516 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.516 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.516 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.516 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.516 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.516 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.516 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.516 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.516 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.516 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.516 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.516 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.516 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.516 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.517 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.517 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.517 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.517 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.517 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.517 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rook", 2024-09-17T04:29:04.517 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.517 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.517 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.517 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.517 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.517 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.517 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.517 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.518 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.518 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.518 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.518 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.518 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.518 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.518 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.518 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.518 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.518 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.518 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.518 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.518 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.518 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.519 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.519 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.519 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.519 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.519 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.519 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.519 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.519 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.519 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.519 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.519 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.519 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.519 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.519 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.519 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.520 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.520 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.520 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.520 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.520 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.520 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.520 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.520 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.520 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.520 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.520 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.520 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.520 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.520 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.521 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.521 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.521 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.521 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.521 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.521 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.521 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.521 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.521 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.521 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.521 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.521 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.521 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.521 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.522 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.522 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.522 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.522 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.522 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.522 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.522 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.522 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.522 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.522 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.522 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.522 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.522 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.522 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.522 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.523 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.523 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.523 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.523 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.523 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.523 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.523 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.523 INFO:teuthology.orchestra.run.smithi100.stdout: "storage_class": { 2024-09-17T04:29:04.523 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "storage_class", 2024-09-17T04:29:04.523 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.523 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.523 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.523 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "local", 2024-09-17T04:29:04.523 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.524 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.524 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.524 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-17T04:29:04.524 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.524 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.524 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.524 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.524 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.524 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.524 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.524 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "selftest", 2024-09-17T04:29:04.524 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.524 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.524 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.525 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.525 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.525 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.525 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.525 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.525 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.525 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.525 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.525 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.525 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.525 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.525 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.525 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.525 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.526 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.526 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.526 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.526 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.526 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.526 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.526 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.526 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.526 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.526 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.526 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.526 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.526 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.526 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.527 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.527 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.527 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.527 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.527 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.527 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.527 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.527 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.527 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.527 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.527 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.527 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.527 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.528 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.528 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.528 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.528 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.528 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.528 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.528 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.528 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.528 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.528 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.528 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.528 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.528 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.528 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.529 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.529 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.529 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.529 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.529 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.529 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.529 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.529 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.529 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.529 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.529 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.529 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.529 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.529 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.530 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.530 INFO:teuthology.orchestra.run.smithi100.stdout: "roption1": { 2024-09-17T04:29:04.530 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "roption1", 2024-09-17T04:29:04.530 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.530 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.530 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.530 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.530 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.530 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.530 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.530 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.530 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.530 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.530 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.531 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.531 INFO:teuthology.orchestra.run.smithi100.stdout: "roption2": { 2024-09-17T04:29:04.531 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "roption2", 2024-09-17T04:29:04.531 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.531 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.531 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.531 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "xyz", 2024-09-17T04:29:04.531 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.531 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.531 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.531 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.531 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.531 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.532 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.532 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.532 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption1": { 2024-09-17T04:29:04.532 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption1", 2024-09-17T04:29:04.532 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.532 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.532 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.532 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.532 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.532 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.532 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.532 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.533 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.533 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.533 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.533 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.533 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption2": { 2024-09-17T04:29:04.533 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption2", 2024-09-17T04:29:04.533 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.533 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.533 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.533 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.533 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.533 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.533 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.534 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.534 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.534 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.534 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.534 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.534 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption3": { 2024-09-17T04:29:04.534 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption3", 2024-09-17T04:29:04.534 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-09-17T04:29:04.534 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.534 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.534 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.534 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.534 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.535 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.535 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.535 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.535 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.535 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.535 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.535 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption4": { 2024-09-17T04:29:04.535 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption4", 2024-09-17T04:29:04.535 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.535 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.535 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.535 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.535 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.535 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.536 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.536 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.536 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.536 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.536 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.536 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.536 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption5": { 2024-09-17T04:29:04.536 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption5", 2024-09-17T04:29:04.536 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.536 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.536 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.536 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.536 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.537 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.537 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.537 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.537 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.537 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.537 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.537 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.537 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption6": { 2024-09-17T04:29:04.537 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption6", 2024-09-17T04:29:04.537 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.537 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.537 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.537 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.538 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.538 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.538 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.538 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.538 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.538 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.538 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.538 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.538 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption7": { 2024-09-17T04:29:04.538 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption7", 2024-09-17T04:29:04.538 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.538 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.538 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.539 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.539 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-09-17T04:29:04.539 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "42", 2024-09-17T04:29:04.539 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.539 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.539 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.539 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.539 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.539 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.540 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.540 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.540 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.540 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.540 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.540 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.540 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.540 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.540 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.540 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.540 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.540 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.540 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.540 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.541 INFO:teuthology.orchestra.run.smithi100.stdout: "testkey": { 2024-09-17T04:29:04.541 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "testkey", 2024-09-17T04:29:04.541 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.541 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.541 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.541 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.541 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.541 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.541 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.541 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.541 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.541 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.541 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.542 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.542 INFO:teuthology.orchestra.run.smithi100.stdout: "testlkey": { 2024-09-17T04:29:04.542 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "testlkey", 2024-09-17T04:29:04.542 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.542 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.542 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.542 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.542 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.542 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.542 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.542 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.542 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.542 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.542 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.543 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.543 INFO:teuthology.orchestra.run.smithi100.stdout: "testnewline": { 2024-09-17T04:29:04.543 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "testnewline", 2024-09-17T04:29:04.543 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.543 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.543 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.543 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.543 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.543 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.543 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.543 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.543 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.543 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.544 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.544 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.544 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.544 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.544 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.544 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smb", 2024-09-17T04:29:04.544 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.544 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.544 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.544 INFO:teuthology.orchestra.run.smithi100.stdout: "internal_store_backend": { 2024-09-17T04:29:04.544 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "internal_store_backend", 2024-09-17T04:29:04.544 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.544 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.545 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.545 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.545 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.545 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.545 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.545 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-17T04:29:04.545 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.545 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.545 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.545 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.545 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.545 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.545 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.546 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.546 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.546 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.546 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.546 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.546 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.546 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.546 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.546 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.546 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.546 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.546 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.546 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.547 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.547 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.547 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.547 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.547 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.547 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.547 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.547 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.547 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.547 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.547 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.547 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.547 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.548 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.548 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.548 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.548 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.548 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.548 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.548 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.548 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.548 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.548 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.548 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.548 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.548 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.549 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.549 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.549 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.549 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.549 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.549 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.549 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.549 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.549 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.549 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.549 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.549 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.549 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.549 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.550 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.550 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.550 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.550 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.550 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.550 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.550 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.550 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.550 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.550 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.550 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.550 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.550 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.550 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.550 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.551 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.551 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.551 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.551 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.551 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.551 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.551 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.551 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.551 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.551 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.551 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.551 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.551 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.551 INFO:teuthology.orchestra.run.smithi100.stdout: "update_orchestration": { 2024-09-17T04:29:04.552 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "update_orchestration", 2024-09-17T04:29:04.552 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.552 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.552 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.552 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.552 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.552 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.552 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.552 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-17T04:29:04.552 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.552 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.552 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.552 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.552 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.553 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.553 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.553 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "snap_schedule", 2024-09-17T04:29:04.553 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.553 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.553 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.553 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_m_granularity": { 2024-09-17T04:29:04.553 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "allow_m_granularity", 2024-09-17T04:29:04.553 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.553 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.553 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.553 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.553 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.554 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.554 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.554 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "allow minute scheduled snapshots", 2024-09-17T04:29:04.554 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.554 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.554 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.554 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.554 INFO:teuthology.orchestra.run.smithi100.stdout: "dump_on_update": { 2024-09-17T04:29:04.554 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "dump_on_update", 2024-09-17T04:29:04.554 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.554 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.554 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.554 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.555 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.555 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.555 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.555 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "dump database to debug log on update", 2024-09-17T04:29:04.555 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.555 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.555 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.555 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.555 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.555 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.555 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.555 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.555 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.555 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.556 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.556 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.556 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.556 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.556 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.556 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.556 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.556 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.556 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.556 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.556 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.556 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.556 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.557 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.557 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.557 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.557 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.557 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.557 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.557 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.557 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.557 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.557 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.557 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.557 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.557 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.558 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.558 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.558 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.558 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.558 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.558 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.558 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.558 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.558 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.558 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.558 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.558 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.558 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.558 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.559 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.559 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.559 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.559 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.559 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.559 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.559 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.559 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.559 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.559 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.559 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.559 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.559 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.560 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.560 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.560 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.560 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.560 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.560 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.560 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.560 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.560 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.560 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.560 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.560 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.560 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.561 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.561 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.561 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.561 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.561 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.561 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.561 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.561 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.561 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.561 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.561 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.561 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.561 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.561 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.562 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.562 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "stats", 2024-09-17T04:29:04.562 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.562 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.562 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.562 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.562 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.562 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.562 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.562 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.562 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.562 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.562 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.562 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.563 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.563 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.563 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.563 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.563 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.563 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.563 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.563 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.563 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.563 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.563 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.563 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.563 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.564 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.564 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.564 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.564 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.564 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.564 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.564 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.564 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.564 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.564 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.564 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.564 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.564 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.564 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.565 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.565 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.565 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.565 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.565 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.565 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.565 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.565 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.565 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.565 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.565 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.565 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.565 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.565 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.566 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.566 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.566 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.566 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.566 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.566 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.566 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.566 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.566 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.566 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.566 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.566 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.566 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.566 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.566 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.567 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.567 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.567 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.567 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.567 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.567 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.567 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.567 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.567 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.567 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.567 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.567 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.567 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.567 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.568 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.568 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.568 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.568 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.568 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.568 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.568 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.568 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.568 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "status", 2024-09-17T04:29:04.568 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.568 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.568 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.568 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.568 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.568 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.569 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.569 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.569 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.569 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.569 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.569 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.569 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.569 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.569 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.569 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.569 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.569 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.569 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.569 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.570 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.570 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.570 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.570 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.570 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.570 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.570 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.570 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.570 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.570 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.570 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.570 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.570 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.570 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.571 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.571 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.571 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.571 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.571 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.571 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.571 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.571 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.571 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.571 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.571 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.571 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.571 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.571 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.571 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.572 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.572 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.572 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.572 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.572 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.572 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.572 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.572 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.572 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.572 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.572 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.572 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.572 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.573 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.573 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.573 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.573 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.573 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.573 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.573 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.573 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.573 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.573 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.573 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.573 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.573 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.573 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.574 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.574 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.574 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.574 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.574 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.574 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.574 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.574 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.574 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.574 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.574 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.574 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.574 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.574 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.575 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "telegraf", 2024-09-17T04:29:04.575 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.575 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.575 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.575 INFO:teuthology.orchestra.run.smithi100.stdout: "address": { 2024-09-17T04:29:04.575 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "address", 2024-09-17T04:29:04.575 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.575 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.575 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.575 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-17T04:29:04.575 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.575 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.575 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.576 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.576 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.576 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.576 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.576 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.576 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-09-17T04:29:04.576 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-09-17T04:29:04.576 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.576 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.576 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.576 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "15", 2024-09-17T04:29:04.576 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.576 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.576 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.577 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.577 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.577 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.577 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.577 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.577 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.577 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.577 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.577 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.577 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.577 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.577 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.577 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.578 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.578 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.578 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.578 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.578 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.578 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.578 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.578 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.578 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.578 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.578 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.578 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.578 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.578 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.579 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.579 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.579 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.579 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.579 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.579 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.579 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.579 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.579 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.579 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.579 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.579 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.580 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.580 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.580 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.580 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.580 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.580 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.580 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.580 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.580 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.580 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.580 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.580 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.580 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.581 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.581 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.581 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.581 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.581 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.581 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.581 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.581 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.581 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.581 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.581 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.581 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.581 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.582 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.582 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.582 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.582 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.582 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.582 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.582 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.582 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.582 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.582 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.582 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.582 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.582 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.582 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.583 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.583 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.583 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.583 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.583 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.583 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.583 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.583 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.583 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.583 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.583 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.583 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.583 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.584 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "telemetry", 2024-09-17T04:29:04.584 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.584 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.584 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.584 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_basic": { 2024-09-17T04:29:04.584 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_basic", 2024-09-17T04:29:04.584 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.584 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.584 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.584 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.584 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.584 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.584 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.585 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-17T04:29:04.585 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.585 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.585 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.585 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.585 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_crash": { 2024-09-17T04:29:04.585 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_crash", 2024-09-17T04:29:04.585 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.585 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.585 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.585 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.585 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.585 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.585 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.586 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-17T04:29:04.586 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.586 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.586 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.586 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.586 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_device": { 2024-09-17T04:29:04.586 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_device", 2024-09-17T04:29:04.586 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.586 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.586 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.586 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.586 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.586 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.587 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.587 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-17T04:29:04.587 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.587 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.587 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.587 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.587 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_ident": { 2024-09-17T04:29:04.587 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_ident", 2024-09-17T04:29:04.587 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.587 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.587 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.587 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.588 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.588 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.588 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.588 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-17T04:29:04.588 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.588 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.588 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.588 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.588 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_perf": { 2024-09-17T04:29:04.588 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_perf", 2024-09-17T04:29:04.588 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.588 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.588 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.588 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.589 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.589 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.589 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.589 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-17T04:29:04.589 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.589 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.589 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.589 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.589 INFO:teuthology.orchestra.run.smithi100.stdout: "contact": { 2024-09-17T04:29:04.589 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "contact", 2024-09-17T04:29:04.589 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.589 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.590 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.590 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.590 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.590 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.590 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.590 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.590 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.590 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.590 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.590 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.590 INFO:teuthology.orchestra.run.smithi100.stdout: "description": { 2024-09-17T04:29:04.590 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "description", 2024-09-17T04:29:04.590 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.590 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.591 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.591 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.591 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.591 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.591 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.591 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.591 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.591 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.591 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.591 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.591 INFO:teuthology.orchestra.run.smithi100.stdout: "device_url": { 2024-09-17T04:29:04.591 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "device_url", 2024-09-17T04:29:04.591 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.592 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.592 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.592 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-17T04:29:04.592 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.592 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.592 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.592 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.592 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.592 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.592 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.592 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.592 INFO:teuthology.orchestra.run.smithi100.stdout: "enabled": { 2024-09-17T04:29:04.592 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enabled", 2024-09-17T04:29:04.592 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.593 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.593 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.593 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.593 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.593 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.593 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.593 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.593 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.593 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.593 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.593 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.593 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-09-17T04:29:04.593 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-09-17T04:29:04.594 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.594 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.594 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.594 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "24", 2024-09-17T04:29:04.594 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "8", 2024-09-17T04:29:04.594 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.594 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.594 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.594 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.594 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.594 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.594 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.594 INFO:teuthology.orchestra.run.smithi100.stdout: "last_opt_revision": { 2024-09-17T04:29:04.594 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "last_opt_revision", 2024-09-17T04:29:04.595 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.595 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.595 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.595 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "1", 2024-09-17T04:29:04.595 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.595 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.595 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.595 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.595 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.595 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.595 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.595 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.595 INFO:teuthology.orchestra.run.smithi100.stdout: "leaderboard": { 2024-09-17T04:29:04.595 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "leaderboard", 2024-09-17T04:29:04.596 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.596 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.596 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.596 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.596 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.596 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.596 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.596 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.596 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.596 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.596 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.596 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.597 INFO:teuthology.orchestra.run.smithi100.stdout: "leaderboard_description": { 2024-09-17T04:29:04.597 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "leaderboard_description", 2024-09-17T04:29:04.597 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.597 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.597 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.597 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.597 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.597 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.597 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.597 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.597 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.597 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.597 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.597 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.598 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.598 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.598 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.598 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.598 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.598 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.598 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.598 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.598 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.598 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.598 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.598 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.599 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.599 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.599 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.599 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.599 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.599 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.599 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.599 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.599 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.599 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.599 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.599 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.599 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.600 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.600 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.600 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.600 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.600 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.600 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.600 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.600 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.600 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.600 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.600 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.600 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.600 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.601 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.601 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.601 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.601 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.601 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.601 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.601 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.601 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.601 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.601 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.601 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.601 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.601 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.601 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.602 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.602 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.602 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.602 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.602 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.602 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.602 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.602 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.602 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.602 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.602 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.602 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.602 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.603 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.603 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.603 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.603 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.603 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.603 INFO:teuthology.orchestra.run.smithi100.stdout: "organization": { 2024-09-17T04:29:04.603 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "organization", 2024-09-17T04:29:04.603 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.603 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.603 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.603 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.603 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.603 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.603 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.604 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.604 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.604 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.604 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.604 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.604 INFO:teuthology.orchestra.run.smithi100.stdout: "proxy": { 2024-09-17T04:29:04.604 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "proxy", 2024-09-17T04:29:04.604 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.604 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.604 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.604 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.604 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.605 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.605 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.605 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.605 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.605 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.605 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.605 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.605 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.605 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.605 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.605 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.605 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.605 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.605 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.606 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.606 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.606 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.606 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.606 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.606 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.606 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.606 INFO:teuthology.orchestra.run.smithi100.stdout: "url": { 2024-09-17T04:29:04.606 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "url", 2024-09-17T04:29:04.606 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.606 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.606 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.606 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-17T04:29:04.607 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.607 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.607 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.607 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.607 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.607 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.607 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.607 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.607 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.607 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.607 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.607 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "test_orchestrator", 2024-09-17T04:29:04.607 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.608 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.608 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.608 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.608 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.608 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.608 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.608 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.608 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.608 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.608 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.608 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.608 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.608 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.609 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.609 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.609 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.609 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.609 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.609 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.609 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.609 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.609 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.609 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.609 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.609 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.610 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.610 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.610 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.610 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.610 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.610 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.610 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.610 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.610 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.610 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.610 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.610 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.610 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.610 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.611 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.611 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.611 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.611 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.611 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.611 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.611 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.611 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.611 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.611 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.611 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.611 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.611 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.612 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.612 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.612 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.612 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.612 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.612 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.612 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.612 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.612 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.612 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.612 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.612 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.612 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.612 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.613 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.613 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.613 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.613 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.613 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.613 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.613 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.613 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.613 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.613 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.613 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.613 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.613 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.614 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.614 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.614 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.614 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.614 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.614 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.614 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.614 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.614 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.614 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.614 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "volumes", 2024-09-17T04:29:04.614 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.614 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.614 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.614 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.615 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.615 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.615 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.615 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.615 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.615 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.615 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.615 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.615 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.615 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.615 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.615 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.615 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.615 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.616 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.616 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.616 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.616 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.616 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.616 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.616 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.616 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.616 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.616 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.616 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.616 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.616 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.617 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.617 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.617 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.617 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.617 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.617 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.617 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.617 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.617 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.617 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.617 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.617 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.617 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.617 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.618 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.618 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.618 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.618 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.618 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.618 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.618 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.618 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.618 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.618 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.618 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.618 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.618 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.619 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.619 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.619 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.619 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.619 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.619 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.619 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.619 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.619 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.619 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.619 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.619 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.619 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.620 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.620 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.620 INFO:teuthology.orchestra.run.smithi100.stdout: "max_concurrent_clones": { 2024-09-17T04:29:04.620 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_concurrent_clones", 2024-09-17T04:29:04.620 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.620 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.620 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.620 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "4", 2024-09-17T04:29:04.620 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.620 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.620 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.620 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-17T04:29:04.620 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.621 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.621 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.621 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.621 INFO:teuthology.orchestra.run.smithi100.stdout: "periodic_async_work": { 2024-09-17T04:29:04.621 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "periodic_async_work", 2024-09-17T04:29:04.621 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.621 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.621 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.621 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.621 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.621 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.621 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.621 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Periodically check for async work", 2024-09-17T04:29:04.622 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.622 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.622 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.622 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.622 INFO:teuthology.orchestra.run.smithi100.stdout: "snapshot_clone_delay": { 2024-09-17T04:29:04.622 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "snapshot_clone_delay", 2024-09-17T04:29:04.622 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.622 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.622 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.622 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.622 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.622 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.622 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.623 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-17T04:29:04.623 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.623 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.623 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.623 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.623 INFO:teuthology.orchestra.run.smithi100.stdout: "snapshot_clone_no_wait": { 2024-09-17T04:29:04.623 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "snapshot_clone_no_wait", 2024-09-17T04:29:04.623 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.623 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.623 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.623 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.623 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.623 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.624 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.624 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-17T04:29:04.624 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.624 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.624 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.624 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.624 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.624 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.624 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.624 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.624 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.624 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.625 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.625 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.625 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.625 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.625 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.625 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.625 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.625 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.625 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.625 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.625 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.625 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix", 2024-09-17T04:29:04.625 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.625 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.626 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.626 INFO:teuthology.orchestra.run.smithi100.stdout: "discovery_interval": { 2024-09-17T04:29:04.626 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "discovery_interval", 2024-09-17T04:29:04.626 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-09-17T04:29:04.626 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.626 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.626 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "100", 2024-09-17T04:29:04.626 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.626 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.626 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.626 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.626 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.627 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.627 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.627 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.627 INFO:teuthology.orchestra.run.smithi100.stdout: "identifier": { 2024-09-17T04:29:04.627 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "identifier", 2024-09-17T04:29:04.627 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.627 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.627 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.627 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.627 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.627 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.627 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.627 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.628 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.628 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.628 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.628 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.628 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-09-17T04:29:04.628 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-09-17T04:29:04.628 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.628 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.628 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.628 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-09-17T04:29:04.628 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.628 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.628 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.629 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.629 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.629 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.629 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.629 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.629 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.629 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.629 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.629 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.629 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.629 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.629 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.629 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.629 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.630 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.630 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.630 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.630 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.630 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.630 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.630 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.630 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.630 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.630 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.630 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.630 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.630 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.630 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.631 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.631 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.631 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.631 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.631 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.631 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.631 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.631 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.631 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.631 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.631 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.631 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.631 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.631 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.632 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.632 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.632 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.632 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.632 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.632 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.632 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.632 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.632 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.632 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.632 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.632 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.633 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.633 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.633 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.633 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.633 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.633 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.633 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.633 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.633 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.633 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.633 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.633 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.634 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.634 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.634 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.634 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.634 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.634 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.634 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.634 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.634 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.634 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.634 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.634 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.634 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.635 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.635 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.635 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.635 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.635 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.635 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.635 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.635 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.635 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.635 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.635 INFO:teuthology.orchestra.run.smithi100.stdout: "zabbix_host": { 2024-09-17T04:29:04.635 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix_host", 2024-09-17T04:29:04.635 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.636 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.636 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.636 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.636 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.636 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.636 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.636 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.636 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.636 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.636 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.636 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.636 INFO:teuthology.orchestra.run.smithi100.stdout: "zabbix_port": { 2024-09-17T04:29:04.636 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix_port", 2024-09-17T04:29:04.637 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.637 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.637 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.637 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10051", 2024-09-17T04:29:04.637 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.637 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.637 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.637 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.637 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.637 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.637 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.637 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.637 INFO:teuthology.orchestra.run.smithi100.stdout: "zabbix_sender": { 2024-09-17T04:29:04.637 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix_sender", 2024-09-17T04:29:04.638 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.638 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.638 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.638 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-17T04:29:04.638 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.638 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.638 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.638 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.638 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.638 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.638 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.638 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.638 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.638 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.639 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-17T04:29:04.639 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.639 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.639 INFO:teuthology.orchestra.run.smithi100.stdout: "modules": [ 2024-09-17T04:29:04.639 INFO:teuthology.orchestra.run.smithi100.stdout: "iostat", 2024-09-17T04:29:04.639 INFO:teuthology.orchestra.run.smithi100.stdout: "nfs", 2024-09-17T04:29:04.639 INFO:teuthology.orchestra.run.smithi100.stdout: "restful" 2024-09-17T04:29:04.639 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.639 INFO:teuthology.orchestra.run.smithi100.stdout: "available_modules": [ 2024-09-17T04:29:04.640 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.640 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "alerts", 2024-09-17T04:29:04.640 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.640 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.640 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.640 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-09-17T04:29:04.640 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-09-17T04:29:04.640 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.640 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.640 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.640 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-09-17T04:29:04.640 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.640 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.641 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.641 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How frequently to reexamine health status", 2024-09-17T04:29:04.641 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.641 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.641 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.641 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.641 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.641 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.641 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.641 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.641 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.641 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.641 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.641 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.642 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.642 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.642 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.642 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.642 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.642 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.642 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.642 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.642 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.642 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.642 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.642 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.642 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.643 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.643 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.643 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.643 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.643 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.643 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.643 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.643 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.643 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.643 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.643 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.643 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.643 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.643 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.644 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.644 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.644 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.644 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.644 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.644 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.644 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.644 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.644 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.644 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.644 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.644 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.644 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.645 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.645 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.645 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.645 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.645 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.645 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.645 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.645 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.645 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.645 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.645 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.645 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.645 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.646 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.646 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.646 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.646 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.646 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.646 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.646 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.646 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.646 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.646 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_destination": { 2024-09-17T04:29:04.646 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_destination", 2024-09-17T04:29:04.646 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.646 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.646 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.647 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.647 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.647 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.647 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.647 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-17T04:29:04.647 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.647 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.647 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.647 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.647 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_from_name": { 2024-09-17T04:29:04.647 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_from_name", 2024-09-17T04:29:04.647 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.647 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.647 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.648 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "Ceph", 2024-09-17T04:29:04.648 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.648 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.648 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.648 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Email From: name", 2024-09-17T04:29:04.648 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.648 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.648 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.648 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.648 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_host": { 2024-09-17T04:29:04.648 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_host", 2024-09-17T04:29:04.648 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.648 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.648 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.649 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.649 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.649 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.649 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.649 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SMTP server", 2024-09-17T04:29:04.649 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.649 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.649 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.649 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.649 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_password": { 2024-09-17T04:29:04.649 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_password", 2024-09-17T04:29:04.649 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.649 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.650 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.650 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.650 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.650 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.650 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.650 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Password to authenticate with", 2024-09-17T04:29:04.650 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.650 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.650 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.650 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.650 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_port": { 2024-09-17T04:29:04.650 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_port", 2024-09-17T04:29:04.650 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.650 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.650 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.651 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "465", 2024-09-17T04:29:04.651 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.651 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.651 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.651 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SMTP port", 2024-09-17T04:29:04.651 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.651 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.651 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.651 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.651 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_sender": { 2024-09-17T04:29:04.651 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_sender", 2024-09-17T04:29:04.651 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.651 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.651 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.652 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.652 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.652 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.652 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.652 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SMTP envelope sender", 2024-09-17T04:29:04.652 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.652 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.652 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.652 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.652 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_ssl": { 2024-09-17T04:29:04.652 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_ssl", 2024-09-17T04:29:04.652 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.652 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.652 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.653 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.653 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.653 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.653 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.653 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-17T04:29:04.653 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.653 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.653 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.653 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.653 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_user": { 2024-09-17T04:29:04.653 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_user", 2024-09-17T04:29:04.653 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.653 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.654 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.654 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.654 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.654 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.654 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.654 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "User to authenticate as", 2024-09-17T04:29:04.654 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.654 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.654 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.654 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.654 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.654 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.654 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.655 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.655 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.655 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.655 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.655 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.655 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.655 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.655 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.655 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.655 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.655 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.655 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.655 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.655 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.656 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "balancer", 2024-09-17T04:29:04.656 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.656 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.656 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.656 INFO:teuthology.orchestra.run.smithi100.stdout: "active": { 2024-09-17T04:29:04.656 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "active", 2024-09-17T04:29:04.656 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.656 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.656 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.656 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.656 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.656 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.656 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.657 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "automatically balance PGs across cluster", 2024-09-17T04:29:04.657 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.657 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.657 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.657 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.657 INFO:teuthology.orchestra.run.smithi100.stdout: "begin_time": { 2024-09-17T04:29:04.657 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "begin_time", 2024-09-17T04:29:04.657 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.657 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.657 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.657 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0000", 2024-09-17T04:29:04.657 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.657 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.657 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.658 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "beginning time of day to automatically balance", 2024-09-17T04:29:04.658 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-17T04:29:04.658 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.658 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.658 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.658 INFO:teuthology.orchestra.run.smithi100.stdout: "begin_weekday": { 2024-09-17T04:29:04.658 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "begin_weekday", 2024-09-17T04:29:04.658 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-09-17T04:29:04.658 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.658 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.658 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.658 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "0", 2024-09-17T04:29:04.659 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "6", 2024-09-17T04:29:04.659 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.659 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-17T04:29:04.659 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-17T04:29:04.659 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.659 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.659 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.659 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_compat_max_iterations": { 2024-09-17T04:29:04.659 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crush_compat_max_iterations", 2024-09-17T04:29:04.659 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-09-17T04:29:04.659 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.659 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.659 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "25", 2024-09-17T04:29:04.659 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-09-17T04:29:04.660 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "250", 2024-09-17T04:29:04.660 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.660 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-17T04:29:04.660 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.660 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.660 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.660 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.660 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_compat_metrics": { 2024-09-17T04:29:04.660 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crush_compat_metrics", 2024-09-17T04:29:04.660 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.660 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.660 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.660 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "pgs,objects,bytes", 2024-09-17T04:29:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-17T04:29:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-17T04:29:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_compat_step": { 2024-09-17T04:29:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crush_compat_step", 2024-09-17T04:29:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-09-17T04:29:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0.5", 2024-09-17T04:29:04.662 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "0.001", 2024-09-17T04:29:04.662 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "0.999", 2024-09-17T04:29:04.662 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.662 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "aggressiveness of optimization", 2024-09-17T04:29:04.662 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-17T04:29:04.662 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.662 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.662 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.662 INFO:teuthology.orchestra.run.smithi100.stdout: "end_time": { 2024-09-17T04:29:04.662 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "end_time", 2024-09-17T04:29:04.662 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.662 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.662 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.663 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "2359", 2024-09-17T04:29:04.663 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.663 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.663 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.663 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "ending time of day to automatically balance", 2024-09-17T04:29:04.663 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-17T04:29:04.663 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.663 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.663 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.663 INFO:teuthology.orchestra.run.smithi100.stdout: "end_weekday": { 2024-09-17T04:29:04.663 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "end_weekday", 2024-09-17T04:29:04.663 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-09-17T04:29:04.663 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "0", 2024-09-17T04:29:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "6", 2024-09-17T04:29:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-17T04:29:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-17T04:29:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.665 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.665 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.665 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.665 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.665 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.665 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.665 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.665 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.665 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.665 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.665 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.665 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.665 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.666 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.666 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.666 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.666 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.666 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.666 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.666 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.666 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.666 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.666 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.666 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.666 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.666 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.668 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.668 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.668 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.668 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.668 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.668 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.668 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.668 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.668 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.668 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.668 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.668 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.668 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.668 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.668 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.669 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.669 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.669 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.669 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.669 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.669 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.669 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.669 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.669 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.669 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.669 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.669 INFO:teuthology.orchestra.run.smithi100.stdout: "min_score": { 2024-09-17T04:29:04.669 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "min_score", 2024-09-17T04:29:04.669 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-09-17T04:29:04.669 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.670 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.670 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.670 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.670 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.670 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.670 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-17T04:29:04.670 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.670 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.670 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.670 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.670 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": { 2024-09-17T04:29:04.670 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mode", 2024-09-17T04:29:04.670 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.670 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.671 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.671 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "upmap", 2024-09-17T04:29:04.671 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.671 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.671 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.671 INFO:teuthology.orchestra.run.smithi100.stdout: "crush-compat", 2024-09-17T04:29:04.671 INFO:teuthology.orchestra.run.smithi100.stdout: "none", 2024-09-17T04:29:04.671 INFO:teuthology.orchestra.run.smithi100.stdout: "read", 2024-09-17T04:29:04.671 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap", 2024-09-17T04:29:04.671 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap-read" 2024-09-17T04:29:04.671 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.671 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Balancer mode", 2024-09-17T04:29:04.671 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.671 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_ids": { 2024-09-17T04:29:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pool_ids", 2024-09-17T04:29:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-17T04:29:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.673 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.673 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.673 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-09-17T04:29:04.673 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-09-17T04:29:04.673 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.673 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.673 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.673 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-09-17T04:29:04.673 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.673 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.673 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.673 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-17T04:29:04.673 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.673 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap_max_deviation": { 2024-09-17T04:29:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "upmap_max_deviation", 2024-09-17T04:29:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-09-17T04:29:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-09-17T04:29:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-17T04:29:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-17T04:29:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap_max_optimizations": { 2024-09-17T04:29:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "upmap_max_optimizations", 2024-09-17T04:29:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-09-17T04:29:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-09-17T04:29:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-17T04:29:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.677 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.677 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.677 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.677 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephadm", 2024-09-17T04:29:04.677 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.677 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.677 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.677 INFO:teuthology.orchestra.run.smithi100.stdout: "agent_down_multiplier": { 2024-09-17T04:29:04.677 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "agent_down_multiplier", 2024-09-17T04:29:04.677 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-09-17T04:29:04.677 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.677 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.677 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "3.0", 2024-09-17T04:29:04.678 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.678 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.678 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.678 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-17T04:29:04.678 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.678 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.678 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.678 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.678 INFO:teuthology.orchestra.run.smithi100.stdout: "agent_refresh_rate": { 2024-09-17T04:29:04.678 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "agent_refresh_rate", 2024-09-17T04:29:04.678 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.678 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "20", 2024-09-17T04:29:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-17T04:29:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: "agent_starting_port": { 2024-09-17T04:29:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "agent_starting_port", 2024-09-17T04:29:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "4721", 2024-09-17T04:29:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-17T04:29:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_ptrace": { 2024-09-17T04:29:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "allow_ptrace", 2024-09-17T04:29:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.681 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.681 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.681 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.681 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-17T04:29:04.681 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-17T04:29:04.681 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.681 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.681 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.681 INFO:teuthology.orchestra.run.smithi100.stdout: "autotune_interval": { 2024-09-17T04:29:04.681 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "autotune_interval", 2024-09-17T04:29:04.681 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.681 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.682 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.682 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-09-17T04:29:04.682 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.682 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.682 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.682 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-17T04:29:04.682 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.682 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.682 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.682 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.682 INFO:teuthology.orchestra.run.smithi100.stdout: "autotune_memory_target_ratio": { 2024-09-17T04:29:04.682 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "autotune_memory_target_ratio", 2024-09-17T04:29:04.682 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-09-17T04:29:04.682 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.683 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.683 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0.7", 2024-09-17T04:29:04.683 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.683 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.683 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.683 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-17T04:29:04.683 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.683 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.683 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.683 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.683 INFO:teuthology.orchestra.run.smithi100.stdout: "cephadm_log_destination": { 2024-09-17T04:29:04.683 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephadm_log_destination", 2024-09-17T04:29:04.683 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.684 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.684 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.684 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.684 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.684 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.684 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.684 INFO:teuthology.orchestra.run.smithi100.stdout: "file", 2024-09-17T04:29:04.684 INFO:teuthology.orchestra.run.smithi100.stdout: "file,syslog", 2024-09-17T04:29:04.684 INFO:teuthology.orchestra.run.smithi100.stdout: "syslog" 2024-09-17T04:29:04.684 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.684 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-17T04:29:04.684 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.684 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.685 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.685 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.685 INFO:teuthology.orchestra.run.smithi100.stdout: "cgroups_split": { 2024-09-17T04:29:04.685 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cgroups_split", 2024-09-17T04:29:04.685 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.685 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.685 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.685 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.685 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.685 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.685 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.685 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-17T04:29:04.685 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.686 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.686 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.686 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.686 INFO:teuthology.orchestra.run.smithi100.stdout: "config_checks_enabled": { 2024-09-17T04:29:04.686 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "config_checks_enabled", 2024-09-17T04:29:04.686 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.686 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.686 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.686 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.686 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.686 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.686 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.686 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-17T04:29:04.686 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.687 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.687 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.687 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.687 INFO:teuthology.orchestra.run.smithi100.stdout: "config_dashboard": { 2024-09-17T04:29:04.687 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "config_dashboard", 2024-09-17T04:29:04.687 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.687 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.687 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.687 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.687 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.687 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.687 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.687 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-17T04:29:04.687 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.688 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.688 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.688 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.688 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_alertmanager": { 2024-09-17T04:29:04.688 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_alertmanager", 2024-09-17T04:29:04.688 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.688 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.688 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.688 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-17T04:29:04.688 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.688 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.688 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.688 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-09-17T04:29:04.688 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.689 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.689 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.689 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.689 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_base": { 2024-09-17T04:29:04.689 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_base", 2024-09-17T04:29:04.689 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.689 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.689 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.689 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-17T04:29:04.689 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.689 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.689 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.689 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Container image name, without the tag", 2024-09-17T04:29:04.689 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.689 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.690 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.690 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.690 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_elasticsearch": { 2024-09-17T04:29:04.690 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_elasticsearch", 2024-09-17T04:29:04.690 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.690 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.690 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.690 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-17T04:29:04.690 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.690 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.690 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.690 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "elasticsearch container image", 2024-09-17T04:29:04.690 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.690 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.691 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.691 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.691 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_grafana": { 2024-09-17T04:29:04.691 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_grafana", 2024-09-17T04:29:04.691 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.691 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.691 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.691 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-17T04:29:04.691 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.691 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.691 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.691 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-09-17T04:29:04.691 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.692 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.692 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.692 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.692 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_haproxy": { 2024-09-17T04:29:04.692 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_haproxy", 2024-09-17T04:29:04.692 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.692 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.692 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.692 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-17T04:29:04.692 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.692 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.692 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.692 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "HAproxy container image", 2024-09-17T04:29:04.692 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.693 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.693 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.693 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.693 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_jaeger_agent": { 2024-09-17T04:29:04.693 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_jaeger_agent", 2024-09-17T04:29:04.693 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.693 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.693 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.693 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-17T04:29:04.693 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.693 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.693 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.693 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Jaeger agent container image", 2024-09-17T04:29:04.694 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.694 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.694 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.694 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.694 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_jaeger_collector": { 2024-09-17T04:29:04.694 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_jaeger_collector", 2024-09-17T04:29:04.694 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.694 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.694 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.694 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-17T04:29:04.694 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.694 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.694 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.695 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Jaeger collector container image", 2024-09-17T04:29:04.695 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.695 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.695 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.695 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.695 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_jaeger_query": { 2024-09-17T04:29:04.695 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_jaeger_query", 2024-09-17T04:29:04.695 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.695 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.695 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.695 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-17T04:29:04.695 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.695 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.696 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.696 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Jaeger query container image", 2024-09-17T04:29:04.696 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.696 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.696 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.696 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.696 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_keepalived": { 2024-09-17T04:29:04.696 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_keepalived", 2024-09-17T04:29:04.696 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.696 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.696 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.696 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-17T04:29:04.696 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.697 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.697 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.697 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Keepalived container image", 2024-09-17T04:29:04.697 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.697 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.697 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.697 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.697 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_loki": { 2024-09-17T04:29:04.697 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_loki", 2024-09-17T04:29:04.697 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.697 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.697 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.697 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-17T04:29:04.697 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.698 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.698 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.698 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Loki container image", 2024-09-17T04:29:04.698 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.698 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.698 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.698 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.698 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_nginx": { 2024-09-17T04:29:04.698 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_nginx", 2024-09-17T04:29:04.698 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.698 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.698 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.698 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-17T04:29:04.699 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.699 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.699 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.699 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Nginx container image", 2024-09-17T04:29:04.699 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.699 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.699 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.699 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.699 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_node_exporter": { 2024-09-17T04:29:04.699 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_node_exporter", 2024-09-17T04:29:04.699 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.699 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.699 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.700 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-17T04:29:04.700 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.700 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.700 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.700 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-09-17T04:29:04.700 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.700 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.700 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.700 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.700 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_nvmeof": { 2024-09-17T04:29:04.700 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_nvmeof", 2024-09-17T04:29:04.700 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.700 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.700 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.701 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-17T04:29:04.701 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.701 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.701 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.701 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Nvme-of container image", 2024-09-17T04:29:04.701 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.701 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.701 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.701 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.701 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_oauth2_proxy": { 2024-09-17T04:29:04.701 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_oauth2_proxy", 2024-09-17T04:29:04.701 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.701 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.702 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.702 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-17T04:29:04.702 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.702 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.702 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.702 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "oauth2-proxy container image", 2024-09-17T04:29:04.702 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.702 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.702 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.702 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.702 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_prometheus": { 2024-09-17T04:29:04.702 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_prometheus", 2024-09-17T04:29:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-17T04:29:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-09-17T04:29:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_promtail": { 2024-09-17T04:29:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_promtail", 2024-09-17T04:29:04.704 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.704 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.704 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.704 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-17T04:29:04.704 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.704 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.704 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.704 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Promtail container image", 2024-09-17T04:29:04.704 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.704 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.704 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.704 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.704 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_samba": { 2024-09-17T04:29:04.705 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_samba", 2024-09-17T04:29:04.705 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.705 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.705 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.705 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-17T04:29:04.705 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.705 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.705 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.705 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Samba/SMB container image", 2024-09-17T04:29:04.705 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.705 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_samba_metrics": { 2024-09-17T04:29:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_samba_metrics", 2024-09-17T04:29:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-17T04:29:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-17T04:29:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.707 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.707 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.707 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_snmp_gateway": { 2024-09-17T04:29:04.707 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_snmp_gateway", 2024-09-17T04:29:04.707 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.707 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.707 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.707 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-17T04:29:04.707 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.707 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.707 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.707 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SNMP Gateway container image", 2024-09-17T04:29:04.708 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.708 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.708 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.708 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.708 INFO:teuthology.orchestra.run.smithi100.stdout: "container_init": { 2024-09-17T04:29:04.708 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_init", 2024-09-17T04:29:04.708 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.708 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.708 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.708 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.708 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.708 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.708 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.709 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Run podman/docker with `--init`", 2024-09-17T04:29:04.709 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.709 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.709 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.709 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.709 INFO:teuthology.orchestra.run.smithi100.stdout: "daemon_cache_timeout": { 2024-09-17T04:29:04.709 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "daemon_cache_timeout", 2024-09-17T04:29:04.709 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.709 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.709 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.709 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-09-17T04:29:04.709 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.709 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.710 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.710 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-17T04:29:04.710 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.710 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.710 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.710 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.710 INFO:teuthology.orchestra.run.smithi100.stdout: "default_cephadm_command_timeout": { 2024-09-17T04:29:04.710 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "default_cephadm_command_timeout", 2024-09-17T04:29:04.710 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.710 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.710 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.710 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "900", 2024-09-17T04:29:04.710 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.711 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.711 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.711 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-17T04:29:04.711 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.711 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.711 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.711 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.711 INFO:teuthology.orchestra.run.smithi100.stdout: "default_registry": { 2024-09-17T04:29:04.711 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "default_registry", 2024-09-17T04:29:04.711 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.711 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.711 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.712 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io", 2024-09-17T04:29:04.712 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.712 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.712 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.712 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-17T04:29:04.712 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.712 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.712 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.712 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.712 INFO:teuthology.orchestra.run.smithi100.stdout: "device_cache_timeout": { 2024-09-17T04:29:04.712 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "device_cache_timeout", 2024-09-17T04:29:04.712 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.712 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.713 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.713 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "1800", 2024-09-17T04:29:04.713 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.713 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.713 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.713 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "seconds to cache device inventory", 2024-09-17T04:29:04.713 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.713 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.713 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.713 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.713 INFO:teuthology.orchestra.run.smithi100.stdout: "device_enhanced_scan": { 2024-09-17T04:29:04.713 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "device_enhanced_scan", 2024-09-17T04:29:04.714 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.714 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.714 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.714 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.714 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.714 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.714 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.714 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-17T04:29:04.714 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.714 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.714 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.714 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.714 INFO:teuthology.orchestra.run.smithi100.stdout: "facts_cache_timeout": { 2024-09-17T04:29:04.714 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "facts_cache_timeout", 2024-09-17T04:29:04.715 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.715 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.715 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.715 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-09-17T04:29:04.715 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.715 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.715 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.715 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "seconds to cache host facts data", 2024-09-17T04:29:04.715 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.715 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.715 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.715 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.716 INFO:teuthology.orchestra.run.smithi100.stdout: "grafana_dashboards_path": { 2024-09-17T04:29:04.716 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "grafana_dashboards_path", 2024-09-17T04:29:04.716 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.716 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.716 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.716 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-17T04:29:04.716 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.716 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.716 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.716 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-17T04:29:04.716 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: "host_check_interval": { 2024-09-17T04:29:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "host_check_interval", 2024-09-17T04:29:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-09-17T04:29:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to perform a host check", 2024-09-17T04:29:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: "hw_monitoring": { 2024-09-17T04:29:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "hw_monitoring", 2024-09-17T04:29:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-17T04:29:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: "inventory_list_all": { 2024-09-17T04:29:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "inventory_list_all", 2024-09-17T04:29:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-17T04:29:04.720 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.720 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.720 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.720 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.720 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.720 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.720 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.720 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.720 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.720 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.720 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.720 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.720 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.720 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.721 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.721 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.721 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.721 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.721 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.721 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.721 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.721 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.721 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.721 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.721 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.721 INFO:teuthology.orchestra.run.smithi100.stdout: "log_refresh_metadata": { 2024-09-17T04:29:04.721 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_refresh_metadata", 2024-09-17T04:29:04.722 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.722 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.722 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.722 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.722 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.722 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.722 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.722 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-17T04:29:04.722 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.722 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.722 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.722 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.722 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.723 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.723 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.723 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.723 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.723 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.723 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.723 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.723 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.723 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-17T04:29:04.723 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.723 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.723 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.723 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.724 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.724 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.724 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.724 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.724 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.724 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.724 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.724 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.724 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.724 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.724 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.724 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.724 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.724 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.725 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.725 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.725 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.725 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.725 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.725 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.725 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.725 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.725 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.725 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.725 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.725 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.725 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.726 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.726 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.726 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.726 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.726 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.726 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.726 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.726 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.726 INFO:teuthology.orchestra.run.smithi100.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-17T04:29:04.726 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-17T04:29:04.726 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.726 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.726 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.727 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.727 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.727 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.727 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.727 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-17T04:29:04.727 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.727 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.727 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.727 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.727 INFO:teuthology.orchestra.run.smithi100.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-17T04:29:04.727 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-17T04:29:04.727 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.727 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.728 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.728 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "*", 2024-09-17T04:29:04.728 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.728 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.728 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.728 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-17T04:29:04.728 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.728 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.728 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.728 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.728 INFO:teuthology.orchestra.run.smithi100.stdout: "max_count_per_host": { 2024-09-17T04:29:04.728 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_count_per_host", 2024-09-17T04:29:04.728 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.729 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.729 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.729 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-09-17T04:29:04.729 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.729 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.729 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.729 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "max number of daemons per service per host", 2024-09-17T04:29:04.729 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.729 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.729 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.729 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.729 INFO:teuthology.orchestra.run.smithi100.stdout: "max_osd_draining_count": { 2024-09-17T04:29:04.729 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_osd_draining_count", 2024-09-17T04:29:04.730 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.730 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.730 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.730 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-09-17T04:29:04.730 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.730 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.730 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.730 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-17T04:29:04.730 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.730 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.730 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.730 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.730 INFO:teuthology.orchestra.run.smithi100.stdout: "migration_current": { 2024-09-17T04:29:04.731 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "migration_current", 2024-09-17T04:29:04.731 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.731 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.731 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.731 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.731 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.731 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.731 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.731 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "internal - do not modify", 2024-09-17T04:29:04.731 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.731 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.731 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.731 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.732 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": { 2024-09-17T04:29:04.732 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mode", 2024-09-17T04:29:04.732 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.732 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.732 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.732 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "root", 2024-09-17T04:29:04.732 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.732 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.732 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.732 INFO:teuthology.orchestra.run.smithi100.stdout: "cephadm-package", 2024-09-17T04:29:04.732 INFO:teuthology.orchestra.run.smithi100.stdout: "root" 2024-09-17T04:29:04.732 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.732 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "mode for remote execution of cephadm", 2024-09-17T04:29:04.732 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.733 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.733 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.733 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.733 INFO:teuthology.orchestra.run.smithi100.stdout: "oob_default_addr": { 2024-09-17T04:29:04.733 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "oob_default_addr", 2024-09-17T04:29:04.733 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.733 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.733 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.733 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "169.254.1.1", 2024-09-17T04:29:04.733 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.733 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.733 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-17T04:29:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: "prometheus_alerts_path": { 2024-09-17T04:29:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "prometheus_alerts_path", 2024-09-17T04:29:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-17T04:29:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-17T04:29:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_insecure": { 2024-09-17T04:29:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_insecure", 2024-09-17T04:29:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-17T04:29:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.736 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.736 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.736 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.736 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_password": { 2024-09-17T04:29:04.736 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_password", 2024-09-17T04:29:04.736 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.736 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.736 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.736 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.736 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.736 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.736 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.736 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-17T04:29:04.736 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_url": { 2024-09-17T04:29:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_url", 2024-09-17T04:29:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-17T04:29:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.738 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.738 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.738 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_username": { 2024-09-17T04:29:04.738 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_username", 2024-09-17T04:29:04.738 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.738 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.738 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.738 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.738 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.738 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.738 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.738 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-17T04:29:04.738 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.738 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.739 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.739 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.739 INFO:teuthology.orchestra.run.smithi100.stdout: "secure_monitoring_stack": { 2024-09-17T04:29:04.739 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "secure_monitoring_stack", 2024-09-17T04:29:04.739 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.739 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.739 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.739 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.739 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.739 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.739 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.740 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-17T04:29:04.740 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.740 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.740 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.740 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.740 INFO:teuthology.orchestra.run.smithi100.stdout: "service_discovery_port": { 2024-09-17T04:29:04.740 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "service_discovery_port", 2024-09-17T04:29:04.740 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.740 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.740 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.740 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8765", 2024-09-17T04:29:04.740 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.741 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.741 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.741 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "cephadm service discovery port", 2024-09-17T04:29:04.741 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.741 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.741 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.741 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.741 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.741 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.741 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.741 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.741 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.741 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.742 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.742 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.742 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.742 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.742 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.742 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.742 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.742 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.742 INFO:teuthology.orchestra.run.smithi100.stdout: "ssh_config_file": { 2024-09-17T04:29:04.742 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssh_config_file", 2024-09-17T04:29:04.742 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.742 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.742 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.743 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.743 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.743 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.743 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.743 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-17T04:29:04.743 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.743 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.743 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.743 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.743 INFO:teuthology.orchestra.run.smithi100.stdout: "ssh_keepalive_count_max": { 2024-09-17T04:29:04.743 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssh_keepalive_count_max", 2024-09-17T04:29:04.743 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.744 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.744 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.744 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "3", 2024-09-17T04:29:04.744 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.744 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.744 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.744 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-17T04:29:04.744 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.744 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.744 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.744 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.744 INFO:teuthology.orchestra.run.smithi100.stdout: "ssh_keepalive_interval": { 2024-09-17T04:29:04.745 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssh_keepalive_interval", 2024-09-17T04:29:04.745 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.745 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.745 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.745 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "7", 2024-09-17T04:29:04.745 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.745 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.745 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.745 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-17T04:29:04.745 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.745 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.745 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.745 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.746 INFO:teuthology.orchestra.run.smithi100.stdout: "use_agent": { 2024-09-17T04:29:04.746 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "use_agent", 2024-09-17T04:29:04.746 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.746 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.746 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.746 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.746 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.747 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.747 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.747 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-17T04:29:04.747 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.748 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.748 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.748 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.748 INFO:teuthology.orchestra.run.smithi100.stdout: "use_repo_digest": { 2024-09-17T04:29:04.748 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "use_repo_digest", 2024-09-17T04:29:04.748 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.748 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.748 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.748 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.748 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.748 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.748 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.749 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-17T04:29:04.749 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.749 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.749 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.749 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.749 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_on_failed_host_check": { 2024-09-17T04:29:04.749 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_on_failed_host_check", 2024-09-17T04:29:04.749 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.749 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.749 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.749 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.749 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.750 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.750 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.750 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise a health warning if the host check fails", 2024-09-17T04:29:04.750 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.750 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.750 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.750 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.750 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_on_stray_daemons": { 2024-09-17T04:29:04.750 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_on_stray_daemons", 2024-09-17T04:29:04.750 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.750 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.750 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.751 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.751 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.751 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.751 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.751 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-17T04:29:04.751 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.751 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.751 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.751 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.751 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_on_stray_hosts": { 2024-09-17T04:29:04.751 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_on_stray_hosts", 2024-09-17T04:29:04.751 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.751 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.752 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.752 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.752 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.752 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.752 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.752 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-17T04:29:04.752 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.752 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.752 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.752 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.752 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.753 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.753 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.753 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crash", 2024-09-17T04:29:04.753 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.753 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.753 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.753 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.753 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.753 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.753 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.753 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.753 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.753 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.754 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.754 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.754 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.754 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.754 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.754 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.754 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.754 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.754 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.754 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.754 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.754 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.754 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.754 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.755 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.755 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.755 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.755 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.755 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.755 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.755 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.755 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.755 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.755 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.755 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.755 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.755 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.755 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.756 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.756 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.756 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.756 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.756 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.756 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.756 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.756 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.756 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.756 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.756 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.756 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.756 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.756 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.756 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.757 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.757 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.757 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.757 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.757 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.757 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.757 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.757 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.757 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.757 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.757 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.757 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.758 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.758 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.758 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.758 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.758 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.758 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.758 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.758 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.758 INFO:teuthology.orchestra.run.smithi100.stdout: "retain_interval": { 2024-09-17T04:29:04.758 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "retain_interval", 2024-09-17T04:29:04.758 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.758 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.759 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.759 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "31536000", 2024-09-17T04:29:04.759 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.759 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.759 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.759 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-17T04:29:04.759 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.759 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.759 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.759 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.759 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.759 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.759 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.760 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.760 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.760 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.760 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.760 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.760 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.760 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.760 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.760 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.760 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.760 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.760 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_recent_interval": { 2024-09-17T04:29:04.760 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_recent_interval", 2024-09-17T04:29:04.761 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.761 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.761 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.761 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "1209600", 2024-09-17T04:29:04.761 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.761 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.761 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.761 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-17T04:29:04.761 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.761 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.761 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.761 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.761 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.762 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.762 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.762 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "dashboard", 2024-09-17T04:29:04.762 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.762 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.762 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.762 INFO:teuthology.orchestra.run.smithi100.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-17T04:29:04.762 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-17T04:29:04.762 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.762 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.762 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.762 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-09-17T04:29:04.762 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.762 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.762 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.763 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.763 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.763 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.763 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.763 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.763 INFO:teuthology.orchestra.run.smithi100.stdout: "ALERTMANAGER_API_HOST": { 2024-09-17T04:29:04.763 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-17T04:29:04.763 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.763 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.763 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.763 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.763 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.763 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.763 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.764 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.764 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.764 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.764 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.764 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.764 INFO:teuthology.orchestra.run.smithi100.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-17T04:29:04.764 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-17T04:29:04.764 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.764 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.764 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.764 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.764 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.764 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.764 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.765 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.765 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.765 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.765 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.765 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.765 INFO:teuthology.orchestra.run.smithi100.stdout: "AUDIT_API_ENABLED": { 2024-09-17T04:29:04.765 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "AUDIT_API_ENABLED", 2024-09-17T04:29:04.765 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.765 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.765 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.765 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.765 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.765 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.765 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.765 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.766 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.766 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.766 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.766 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.766 INFO:teuthology.orchestra.run.smithi100.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-17T04:29:04.766 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-17T04:29:04.766 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.766 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.766 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.766 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.766 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.766 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.766 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.766 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.766 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.767 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.767 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.767 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.767 INFO:teuthology.orchestra.run.smithi100.stdout: "ENABLE_BROWSABLE_API": { 2024-09-17T04:29:04.767 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-17T04:29:04.767 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.767 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.767 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.767 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.767 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.767 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.767 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.767 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.767 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.767 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.767 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.768 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.768 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-17T04:29:04.768 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-17T04:29:04.768 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.768 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.768 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.768 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.768 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.768 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.768 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.768 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.768 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.768 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.768 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.768 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.769 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-17T04:29:04.769 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-17T04:29:04.769 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.769 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.769 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.769 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.769 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.769 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.769 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.769 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.769 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.769 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.769 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.770 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.770 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-17T04:29:04.770 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-17T04:29:04.770 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.770 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.770 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.770 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.770 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.770 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.770 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.770 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.770 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.770 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.770 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.771 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.771 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-17T04:29:04.771 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-17T04:29:04.771 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.771 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.771 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.771 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.771 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.771 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.771 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.771 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.771 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.771 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.771 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.771 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.771 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-17T04:29:04.772 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-17T04:29:04.772 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.772 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.772 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.772 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.772 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.772 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.772 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.772 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.772 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.772 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.772 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.772 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.773 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-17T04:29:04.773 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-17T04:29:04.773 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.773 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.773 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.773 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.773 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.773 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.773 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.773 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.773 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.773 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.773 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.774 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.774 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-17T04:29:04.774 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-17T04:29:04.774 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.774 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.774 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.774 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.774 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.774 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.774 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.774 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.774 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.774 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.774 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.774 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.774 INFO:teuthology.orchestra.run.smithi100.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-17T04:29:04.775 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-17T04:29:04.775 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.775 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.775 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.775 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.775 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.775 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.775 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.775 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.775 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.775 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.775 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.775 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.775 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_PASSWORD": { 2024-09-17T04:29:04.776 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-17T04:29:04.776 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.776 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.776 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.776 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "admin", 2024-09-17T04:29:04.776 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.776 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.776 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.776 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.776 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.776 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.776 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.776 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.776 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-17T04:29:04.777 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-17T04:29:04.777 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.777 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.777 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.777 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.777 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.777 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.777 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.777 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.777 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.777 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.777 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.777 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.778 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_URL": { 2024-09-17T04:29:04.778 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_URL", 2024-09-17T04:29:04.778 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.778 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.778 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.778 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.778 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.778 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.778 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.778 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.778 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.778 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.778 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.778 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.778 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_USERNAME": { 2024-09-17T04:29:04.779 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-17T04:29:04.779 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.779 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.779 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.779 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "admin", 2024-09-17T04:29:04.779 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.779 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.779 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.779 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.779 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.779 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.779 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.779 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.780 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-17T04:29:04.780 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-17T04:29:04.780 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.780 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.780 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.780 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.780 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.780 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.780 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.780 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.780 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.780 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.780 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.780 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.781 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-17T04:29:04.781 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-17T04:29:04.781 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.781 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.781 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.781 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.781 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.781 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.781 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.781 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.781 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.781 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.781 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.781 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.782 INFO:teuthology.orchestra.run.smithi100.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-17T04:29:04.782 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-17T04:29:04.782 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.782 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.782 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.782 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.782 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.782 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.782 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.782 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.782 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.782 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.782 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.782 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.783 INFO:teuthology.orchestra.run.smithi100.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-17T04:29:04.783 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-17T04:29:04.783 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.783 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.783 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.783 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.783 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.783 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.783 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.783 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.783 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.783 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.783 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.783 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.784 INFO:teuthology.orchestra.run.smithi100.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-17T04:29:04.784 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-17T04:29:04.784 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.784 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.784 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.784 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "[]", 2024-09-17T04:29:04.784 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.784 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.784 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.784 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.784 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.784 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.784 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.785 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.785 INFO:teuthology.orchestra.run.smithi100.stdout: "MULTICLUSTER_CONFIG": { 2024-09-17T04:29:04.785 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-17T04:29:04.785 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.785 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.785 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.785 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "{}", 2024-09-17T04:29:04.785 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.785 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.785 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.785 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.785 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.785 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.786 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.786 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.786 INFO:teuthology.orchestra.run.smithi100.stdout: "PROMETHEUS_API_HOST": { 2024-09-17T04:29:04.786 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-17T04:29:04.786 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.786 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.786 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.786 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.786 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.786 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.786 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.786 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.786 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.786 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.787 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.787 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.787 INFO:teuthology.orchestra.run.smithi100.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-17T04:29:04.787 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-17T04:29:04.787 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.787 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.787 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.787 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.787 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.787 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.787 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.787 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.787 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.788 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.788 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.788 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.788 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-17T04:29:04.788 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-17T04:29:04.788 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.788 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.788 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.788 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.788 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.788 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.788 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.788 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.788 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.788 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.789 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.789 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.789 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-17T04:29:04.789 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-17T04:29:04.789 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.789 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.789 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.789 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.789 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.789 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.789 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.789 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.789 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.789 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.789 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.790 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.790 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-17T04:29:04.790 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-17T04:29:04.790 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.790 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.790 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.790 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.790 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.790 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.790 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.790 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.790 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.790 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.790 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.790 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.790 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-17T04:29:04.791 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-17T04:29:04.791 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.791 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.791 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.791 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.791 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.791 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.791 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.791 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.791 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.791 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.791 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.791 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.791 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-17T04:29:04.792 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-17T04:29:04.792 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.792 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.792 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.792 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.792 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.792 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.792 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.792 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.792 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.792 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.792 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.792 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.792 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-17T04:29:04.792 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-17T04:29:04.793 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.793 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.793 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.793 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.793 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.793 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.793 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.793 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.793 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.793 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.793 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.793 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.793 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-17T04:29:04.794 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-17T04:29:04.794 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.794 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.794 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.794 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.794 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.794 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.794 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.794 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.794 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.794 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.794 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.794 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.794 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_ENABLED": { 2024-09-17T04:29:04.795 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-17T04:29:04.795 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.795 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.795 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.795 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.795 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.795 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.795 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.795 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.795 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.795 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.795 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.795 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.796 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-17T04:29:04.796 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-17T04:29:04.796 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.796 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.796 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.796 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-17T04:29:04.796 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.796 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.796 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.796 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.796 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.796 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.796 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.796 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.797 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-17T04:29:04.797 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-17T04:29:04.797 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.797 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.797 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.797 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-09-17T04:29:04.797 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.797 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.797 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.797 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.797 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.797 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.797 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.798 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.798 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-17T04:29:04.798 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-17T04:29:04.798 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.798 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.798 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.798 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8", 2024-09-17T04:29:04.798 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.798 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.798 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.798 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.798 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.798 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.799 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.799 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.799 INFO:teuthology.orchestra.run.smithi100.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-17T04:29:04.799 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-17T04:29:04.799 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.799 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.799 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.799 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "45", 2024-09-17T04:29:04.799 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.799 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.799 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.799 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.799 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.800 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.800 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.800 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.800 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_ACCESS_KEY": { 2024-09-17T04:29:04.800 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-17T04:29:04.800 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.800 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.800 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.800 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.800 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.800 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.800 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.800 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.801 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.801 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.801 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.801 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.801 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-17T04:29:04.801 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-17T04:29:04.801 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.801 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.801 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.801 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "admin", 2024-09-17T04:29:04.801 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.801 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.801 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.802 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.802 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.802 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.802 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.802 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.802 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_SECRET_KEY": { 2024-09-17T04:29:04.802 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-17T04:29:04.802 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.802 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.802 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.802 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.802 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.802 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.803 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.803 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.803 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.803 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.803 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.803 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.803 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_SSL_VERIFY": { 2024-09-17T04:29:04.803 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-17T04:29:04.803 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.803 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.803 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.803 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.803 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.804 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.804 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.804 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.804 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.804 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.804 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.804 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.804 INFO:teuthology.orchestra.run.smithi100.stdout: "UNSAFE_TLS_v1_2": { 2024-09-17T04:29:04.804 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-17T04:29:04.804 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.804 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.804 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.804 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.805 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.805 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.805 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.805 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.805 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.805 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.805 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.805 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.805 INFO:teuthology.orchestra.run.smithi100.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-17T04:29:04.805 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-17T04:29:04.805 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.805 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.805 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.806 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.806 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.806 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.806 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.806 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.806 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.806 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.806 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.806 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.806 INFO:teuthology.orchestra.run.smithi100.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-17T04:29:04.806 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-17T04:29:04.806 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.806 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.807 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.807 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-09-17T04:29:04.807 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.807 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.807 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.807 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.807 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.807 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.807 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.807 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.807 INFO:teuthology.orchestra.run.smithi100.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-17T04:29:04.807 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-17T04:29:04.807 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.807 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.808 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.808 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-09-17T04:29:04.808 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.808 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.808 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.808 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.808 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.808 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.808 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.808 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.808 INFO:teuthology.orchestra.run.smithi100.stdout: "cross_origin_url": { 2024-09-17T04:29:04.808 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cross_origin_url", 2024-09-17T04:29:04.808 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.809 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.809 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.809 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.809 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.809 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.809 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.809 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.809 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.809 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.809 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.809 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.809 INFO:teuthology.orchestra.run.smithi100.stdout: "crt_file": { 2024-09-17T04:29:04.809 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crt_file", 2024-09-17T04:29:04.810 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.810 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.810 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.810 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.810 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.810 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.810 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.810 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.810 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.810 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.810 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.810 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.811 INFO:teuthology.orchestra.run.smithi100.stdout: "debug": { 2024-09-17T04:29:04.811 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "debug", 2024-09-17T04:29:04.811 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.811 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.811 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.811 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.811 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.811 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.811 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.811 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Enable/disable debug options", 2024-09-17T04:29:04.811 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.811 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.811 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.812 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.812 INFO:teuthology.orchestra.run.smithi100.stdout: "jwt_token_ttl": { 2024-09-17T04:29:04.812 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "jwt_token_ttl", 2024-09-17T04:29:04.812 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.812 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.812 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.812 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "28800", 2024-09-17T04:29:04.812 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.812 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.812 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.812 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.812 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.812 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.813 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.813 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.813 INFO:teuthology.orchestra.run.smithi100.stdout: "key_file": { 2024-09-17T04:29:04.813 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "key_file", 2024-09-17T04:29:04.813 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.813 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.813 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.813 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.813 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.813 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.813 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.813 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.813 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.813 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.814 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.814 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.814 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.814 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.814 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.814 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.814 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.814 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.814 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.814 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.814 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.814 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.814 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.815 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.815 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.815 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.815 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.815 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.815 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.815 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.815 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.815 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.815 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.815 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.816 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.816 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.816 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.816 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.816 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.816 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.816 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.816 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.816 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.816 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.816 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.816 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.816 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.817 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.817 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.817 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.817 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.817 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.817 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.817 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.817 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.817 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.817 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.817 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.817 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.817 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.818 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.818 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.818 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.818 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.818 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.818 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.818 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.818 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.818 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.818 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.818 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.818 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.818 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.819 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.819 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.819 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.819 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.819 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.819 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.819 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.819 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.819 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.819 INFO:teuthology.orchestra.run.smithi100.stdout: "motd": { 2024-09-17T04:29:04.819 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "motd", 2024-09-17T04:29:04.819 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.819 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.820 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.820 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.820 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.820 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.820 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.820 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "The message of the day", 2024-09-17T04:29:04.820 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.820 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.820 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.820 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.820 INFO:teuthology.orchestra.run.smithi100.stdout: "redirect_resolve_ip_addr": { 2024-09-17T04:29:04.820 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "redirect_resolve_ip_addr", 2024-09-17T04:29:04.820 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.821 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.821 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.821 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.821 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.821 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.821 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.821 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.821 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.821 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.821 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.821 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.821 INFO:teuthology.orchestra.run.smithi100.stdout: "server_addr": { 2024-09-17T04:29:04.821 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_addr", 2024-09-17T04:29:04.822 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.822 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.822 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.822 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "::", 2024-09-17T04:29:04.822 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.822 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.822 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.822 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.822 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.822 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.822 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.822 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.822 INFO:teuthology.orchestra.run.smithi100.stdout: "server_port": { 2024-09-17T04:29:04.823 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_port", 2024-09-17T04:29:04.823 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.823 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.823 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.823 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8080", 2024-09-17T04:29:04.823 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.823 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.823 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.823 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.823 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.823 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.823 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.823 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.824 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.824 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.824 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.824 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.824 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.824 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.824 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.824 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.824 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.824 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.824 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.824 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.824 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.824 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.825 INFO:teuthology.orchestra.run.smithi100.stdout: "ssl": { 2024-09-17T04:29:04.825 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssl", 2024-09-17T04:29:04.825 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.825 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.825 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.825 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.825 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.825 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.825 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.825 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.825 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.825 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.825 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.825 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.826 INFO:teuthology.orchestra.run.smithi100.stdout: "ssl_server_port": { 2024-09-17T04:29:04.826 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssl_server_port", 2024-09-17T04:29:04.826 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.826 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.826 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.826 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8443", 2024-09-17T04:29:04.826 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.826 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.826 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.826 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.826 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.826 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.826 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.827 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.827 INFO:teuthology.orchestra.run.smithi100.stdout: "sso_oauth2": { 2024-09-17T04:29:04.827 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sso_oauth2", 2024-09-17T04:29:04.827 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.827 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.827 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.827 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.827 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.827 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.827 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.827 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.827 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.827 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.827 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.828 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.828 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_behaviour": { 2024-09-17T04:29:04.828 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_behaviour", 2024-09-17T04:29:04.828 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.828 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.828 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.828 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "redirect", 2024-09-17T04:29:04.828 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.828 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.828 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.828 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.828 INFO:teuthology.orchestra.run.smithi100.stdout: "redirect" 2024-09-17T04:29:04.829 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.829 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.829 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.829 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.829 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.829 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.829 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_error_status_code": { 2024-09-17T04:29:04.829 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_error_status_code", 2024-09-17T04:29:04.829 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.829 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.829 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.829 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "500", 2024-09-17T04:29:04.829 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "400", 2024-09-17T04:29:04.830 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "599", 2024-09-17T04:29:04.830 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.830 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.830 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.830 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.830 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.830 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.830 INFO:teuthology.orchestra.run.smithi100.stdout: "url_prefix": { 2024-09-17T04:29:04.830 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "url_prefix", 2024-09-17T04:29:04.830 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.830 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.830 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.830 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.830 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.831 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.831 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.831 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.831 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.831 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.831 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.831 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.831 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.831 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.831 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.831 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "devicehealth", 2024-09-17T04:29:04.831 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.831 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.831 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.832 INFO:teuthology.orchestra.run.smithi100.stdout: "enable_monitoring": { 2024-09-17T04:29:04.832 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enable_monitoring", 2024-09-17T04:29:04.832 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.832 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.832 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.832 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.832 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.832 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.832 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.832 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "monitor device health metrics", 2024-09-17T04:29:04.832 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.832 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.832 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.832 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.833 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.833 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.833 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.833 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.833 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.833 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.833 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.833 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.833 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.833 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.833 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.833 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.833 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.834 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.834 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.834 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.834 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.834 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.834 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.834 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.834 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.834 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.834 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.834 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.834 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.834 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.834 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.835 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.835 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.835 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.835 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.835 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.835 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.835 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.835 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.835 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.835 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.835 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.835 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.835 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.835 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.835 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.836 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.836 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.836 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.836 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.836 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.836 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.836 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.836 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.836 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.836 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.836 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.836 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.836 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.836 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.837 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.837 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.837 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.837 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.837 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.837 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.837 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.837 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.837 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.837 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.837 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.837 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.837 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.838 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.838 INFO:teuthology.orchestra.run.smithi100.stdout: "mark_out_threshold": { 2024-09-17T04:29:04.838 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mark_out_threshold", 2024-09-17T04:29:04.838 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.838 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.838 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.838 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "2419200", 2024-09-17T04:29:04.838 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.838 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.838 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.838 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-17T04:29:04.838 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.839 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.839 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.839 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.839 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_name": { 2024-09-17T04:29:04.839 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pool_name", 2024-09-17T04:29:04.839 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.839 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.839 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.839 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "device_health_metrics", 2024-09-17T04:29:04.839 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.839 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.839 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.839 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-17T04:29:04.839 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.840 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.840 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.840 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.840 INFO:teuthology.orchestra.run.smithi100.stdout: "retention_period": { 2024-09-17T04:29:04.840 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "retention_period", 2024-09-17T04:29:04.840 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.840 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.840 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.840 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "15552000", 2024-09-17T04:29:04.841 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.841 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.841 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.841 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how long to retain device health metrics", 2024-09-17T04:29:04.841 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.841 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.841 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.841 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.841 INFO:teuthology.orchestra.run.smithi100.stdout: "scrape_frequency": { 2024-09-17T04:29:04.841 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "scrape_frequency", 2024-09-17T04:29:04.841 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.841 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.841 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.842 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "86400", 2024-09-17T04:29:04.842 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.842 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.842 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.842 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-17T04:29:04.842 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.842 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.842 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.842 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.842 INFO:teuthology.orchestra.run.smithi100.stdout: "self_heal": { 2024-09-17T04:29:04.842 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "self_heal", 2024-09-17T04:29:04.842 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.842 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.842 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.843 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.843 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.843 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.843 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.843 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-17T04:29:04.843 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.843 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.843 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.843 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.843 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-09-17T04:29:04.843 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-09-17T04:29:04.843 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.843 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.843 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.843 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-09-17T04:29:04.844 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.844 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.844 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.844 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to wake up and check device health", 2024-09-17T04:29:04.844 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.844 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.844 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.844 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.844 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.844 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.844 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.844 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.844 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.844 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.844 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.845 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.845 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.845 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.845 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.845 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.845 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.845 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.845 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_threshold": { 2024-09-17T04:29:04.845 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_threshold", 2024-09-17T04:29:04.845 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.845 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.845 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.845 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "7257600", 2024-09-17T04:29:04.846 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.846 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.846 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.846 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-17T04:29:04.846 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.846 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.846 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.846 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.846 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.846 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.846 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.846 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "diskprediction_local", 2024-09-17T04:29:04.846 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.846 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.846 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.847 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.847 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.847 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.847 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.847 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.847 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.847 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.847 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.847 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.847 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.847 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.847 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.847 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.847 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.847 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.848 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.848 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.848 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.848 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.848 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.848 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.848 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.848 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.848 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.848 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.848 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.848 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.848 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.848 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.849 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.849 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.849 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.849 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.849 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.849 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.849 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.849 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.849 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.849 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.849 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.849 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.849 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.850 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.850 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.850 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.850 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.850 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.850 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.850 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.850 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.850 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.850 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.850 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.850 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.850 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.850 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.851 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.851 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.851 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.851 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.851 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.851 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.851 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.851 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.851 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.851 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.851 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.851 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.851 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.851 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.852 INFO:teuthology.orchestra.run.smithi100.stdout: "predict_interval": { 2024-09-17T04:29:04.852 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "predict_interval", 2024-09-17T04:29:04.852 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.852 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.852 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.852 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "86400", 2024-09-17T04:29:04.852 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.852 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.852 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.852 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.852 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.852 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.852 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.853 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.853 INFO:teuthology.orchestra.run.smithi100.stdout: "predictor_model": { 2024-09-17T04:29:04.853 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "predictor_model", 2024-09-17T04:29:04.853 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.853 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.853 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.853 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "prophetstor", 2024-09-17T04:29:04.853 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.853 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.853 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.853 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.853 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.853 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.853 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.854 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.854 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-09-17T04:29:04.854 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-09-17T04:29:04.854 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.854 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.854 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.854 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-09-17T04:29:04.854 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.854 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.854 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.854 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.854 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.854 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.854 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.855 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.855 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.855 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.855 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.855 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.855 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.855 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.855 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.855 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.855 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.855 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.855 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.855 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.855 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.855 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.856 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.856 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.856 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.856 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "influx", 2024-09-17T04:29:04.856 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": false, 2024-09-17T04:29:04.856 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "influxdb python module not found", 2024-09-17T04:29:04.856 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.856 INFO:teuthology.orchestra.run.smithi100.stdout: "batch_size": { 2024-09-17T04:29:04.856 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "batch_size", 2024-09-17T04:29:04.856 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.856 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.856 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.856 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5000", 2024-09-17T04:29:04.856 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.856 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.857 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.857 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-17T04:29:04.857 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.857 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.857 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.857 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.857 INFO:teuthology.orchestra.run.smithi100.stdout: "database": { 2024-09-17T04:29:04.857 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "database", 2024-09-17T04:29:04.857 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.857 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.857 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.857 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "ceph", 2024-09-17T04:29:04.857 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.857 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.857 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.858 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-17T04:29:04.858 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.858 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.858 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.858 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.858 INFO:teuthology.orchestra.run.smithi100.stdout: "hostname": { 2024-09-17T04:29:04.858 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "hostname", 2024-09-17T04:29:04.858 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.858 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.858 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.858 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.858 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.858 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.859 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.859 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "InfluxDB server hostname", 2024-09-17T04:29:04.859 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.859 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.859 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.859 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.859 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-09-17T04:29:04.859 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-09-17T04:29:04.859 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.859 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.859 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.859 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "30", 2024-09-17T04:29:04.859 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "5", 2024-09-17T04:29:04.859 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.860 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.860 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-17T04:29:04.860 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.860 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.860 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.860 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.860 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.860 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.860 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.860 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.860 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.860 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.860 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.860 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.861 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.861 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.861 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.861 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.861 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.861 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.861 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.861 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.861 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.861 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.861 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.861 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.861 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.862 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.862 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.862 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.862 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.862 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.862 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.862 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.862 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.862 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.862 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.862 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.862 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.863 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.863 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.863 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.863 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.863 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.863 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.863 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.863 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.863 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.863 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.863 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.863 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.864 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.864 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.864 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.864 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.864 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.864 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.864 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.864 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.864 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.864 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.864 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.864 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.864 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.865 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.865 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.865 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.865 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.865 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.865 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.865 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.865 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.865 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.865 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.865 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.865 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.865 INFO:teuthology.orchestra.run.smithi100.stdout: "password": { 2024-09-17T04:29:04.866 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "password", 2024-09-17T04:29:04.866 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.866 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.866 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.866 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.866 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.866 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.866 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.866 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "password of InfluxDB server user", 2024-09-17T04:29:04.866 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.866 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.866 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.866 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.866 INFO:teuthology.orchestra.run.smithi100.stdout: "port": { 2024-09-17T04:29:04.867 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "port", 2024-09-17T04:29:04.867 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.867 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.867 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.867 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8086", 2024-09-17T04:29:04.867 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.867 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.867 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.867 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "InfluxDB server port", 2024-09-17T04:29:04.867 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.867 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.867 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.867 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.868 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.868 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.868 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.868 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.868 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.868 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.868 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.868 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.868 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.868 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.868 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.868 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.868 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.868 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.869 INFO:teuthology.orchestra.run.smithi100.stdout: "ssl": { 2024-09-17T04:29:04.869 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssl", 2024-09-17T04:29:04.869 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.869 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.869 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.869 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "false", 2024-09-17T04:29:04.869 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.869 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.869 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.869 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-17T04:29:04.869 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.869 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.869 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.870 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.870 INFO:teuthology.orchestra.run.smithi100.stdout: "threads": { 2024-09-17T04:29:04.870 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "threads", 2024-09-17T04:29:04.870 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.870 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.870 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.870 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-09-17T04:29:04.870 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-09-17T04:29:04.870 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "32", 2024-09-17T04:29:04.870 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.870 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-17T04:29:04.870 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.870 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.871 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.871 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.871 INFO:teuthology.orchestra.run.smithi100.stdout: "username": { 2024-09-17T04:29:04.871 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "username", 2024-09-17T04:29:04.871 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.871 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.871 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.871 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.871 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.871 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.871 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.871 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "username of InfluxDB server user", 2024-09-17T04:29:04.871 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.871 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.872 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.872 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.872 INFO:teuthology.orchestra.run.smithi100.stdout: "verify_ssl": { 2024-09-17T04:29:04.872 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "verify_ssl", 2024-09-17T04:29:04.872 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.872 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.872 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.872 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "true", 2024-09-17T04:29:04.872 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.872 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.872 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.872 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-17T04:29:04.872 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.873 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.873 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.873 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.873 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.873 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.873 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.873 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "insights", 2024-09-17T04:29:04.873 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.873 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.873 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.873 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.873 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.873 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.874 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.874 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.874 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.874 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.874 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.874 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.874 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.874 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.874 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.874 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.874 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.874 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.874 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.874 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.874 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.875 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.875 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.875 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.875 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.875 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.875 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.875 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.875 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.875 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.875 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.875 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.875 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.875 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.875 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.875 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.876 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.876 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.876 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.876 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.876 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.876 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.876 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.876 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.876 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.876 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.876 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.876 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.876 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.877 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.877 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.877 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.877 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.877 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.877 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.877 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.877 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.877 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.877 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.877 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.877 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.877 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.878 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.878 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.878 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.878 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.878 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.878 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.878 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.878 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.878 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.878 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.878 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.878 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.878 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.879 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.879 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.879 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.879 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.879 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.879 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.879 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.879 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.879 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.879 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.880 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.880 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.880 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.880 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.880 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.880 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "iostat", 2024-09-17T04:29:04.880 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.880 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.880 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.880 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.880 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.880 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.880 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.880 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.881 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.881 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.881 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.881 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.881 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.881 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.881 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.881 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.881 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.881 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.881 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.881 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.881 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.881 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.881 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.882 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.882 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.882 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.882 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.882 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.882 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.882 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.882 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.882 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.882 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.882 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.882 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.882 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.882 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.883 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.883 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.883 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.883 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.883 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.883 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.883 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.883 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.883 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.883 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.883 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.883 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.883 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.883 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.884 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.884 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.884 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.884 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.884 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.884 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.884 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.884 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.884 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.884 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.884 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.884 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.884 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.884 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.884 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.885 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.885 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.885 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.885 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.885 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.885 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.885 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.885 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.885 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.885 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.885 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.885 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.885 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.885 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.886 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.886 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.886 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.886 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.886 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.886 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.886 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.886 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.886 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.886 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.886 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "localpool", 2024-09-17T04:29:04.886 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.886 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.886 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.886 INFO:teuthology.orchestra.run.smithi100.stdout: "failure_domain": { 2024-09-17T04:29:04.886 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "failure_domain", 2024-09-17T04:29:04.887 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.887 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.887 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.887 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "host", 2024-09-17T04:29:04.887 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.887 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.887 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.887 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "failure domain for any created local pool", 2024-09-17T04:29:04.887 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-17T04:29:04.887 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.887 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.887 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.888 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.888 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.888 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.888 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.888 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.888 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.888 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.888 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.888 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.888 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.888 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.888 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.888 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.888 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.889 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.889 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.889 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.889 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.889 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.889 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.889 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.889 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.889 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.889 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.889 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.889 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.889 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.889 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.889 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.890 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.890 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.890 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.890 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.890 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.890 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.890 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.890 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.890 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.890 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.890 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.890 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.890 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.890 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.891 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.891 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.891 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.891 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.891 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.891 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.891 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.891 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.891 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.891 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.891 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.891 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.891 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.892 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.892 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.892 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.892 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.892 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.892 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.892 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.892 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.892 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.892 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.892 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.892 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.892 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.892 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.892 INFO:teuthology.orchestra.run.smithi100.stdout: "min_size": { 2024-09-17T04:29:04.893 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "min_size", 2024-09-17T04:29:04.893 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.893 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.893 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.893 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.893 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.893 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.893 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.893 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "default min_size for any created local pool", 2024-09-17T04:29:04.893 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-17T04:29:04.893 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.893 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.893 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.894 INFO:teuthology.orchestra.run.smithi100.stdout: "num_rep": { 2024-09-17T04:29:04.894 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "num_rep", 2024-09-17T04:29:04.894 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.894 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.894 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.894 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "3", 2024-09-17T04:29:04.894 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.894 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.894 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.894 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "default replica count for any created local pool", 2024-09-17T04:29:04.894 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.894 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.894 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.894 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.895 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num": { 2024-09-17T04:29:04.895 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pg_num", 2024-09-17T04:29:04.895 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.895 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.895 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.895 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "128", 2024-09-17T04:29:04.895 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.895 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.895 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.895 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "default pg_num for any created local pool", 2024-09-17T04:29:04.895 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.895 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.895 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.895 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.896 INFO:teuthology.orchestra.run.smithi100.stdout: "prefix": { 2024-09-17T04:29:04.896 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "prefix", 2024-09-17T04:29:04.896 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.896 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.896 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.896 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.896 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.896 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.896 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.896 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "name prefix for any created local pool", 2024-09-17T04:29:04.896 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.896 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.896 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.897 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.897 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.897 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.897 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.897 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.897 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.897 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.897 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.897 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.897 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.897 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.897 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.897 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.898 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.898 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.898 INFO:teuthology.orchestra.run.smithi100.stdout: "subtree": { 2024-09-17T04:29:04.898 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "subtree", 2024-09-17T04:29:04.898 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.898 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.898 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.898 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "rack", 2024-09-17T04:29:04.898 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.898 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.898 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.898 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-17T04:29:04.898 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-17T04:29:04.898 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.899 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.899 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.899 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.899 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.899 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.899 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mds_autoscaler", 2024-09-17T04:29:04.899 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.899 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.899 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.899 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.899 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.899 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.899 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.900 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.900 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.900 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.900 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.900 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.900 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.900 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.900 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.900 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.900 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.900 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.900 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.900 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.901 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.901 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.901 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.901 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.901 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.901 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.901 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.901 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.901 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.901 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.901 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.901 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.901 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.901 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.902 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.902 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.902 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.902 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.902 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.902 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.902 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.902 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.902 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.902 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.902 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.902 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.902 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.902 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.902 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.903 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.903 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.903 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.903 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.903 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.903 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.903 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.903 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.903 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.903 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.903 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.903 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.903 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.904 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.904 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.904 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.904 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.904 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.904 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.904 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.904 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.904 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.904 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.904 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.904 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.905 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.905 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.905 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.905 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.905 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.905 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.905 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.905 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.905 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.905 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.905 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.905 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.905 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.906 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.906 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.906 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.906 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mirroring", 2024-09-17T04:29:04.906 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.906 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.906 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.906 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.906 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.906 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.906 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.906 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.906 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.907 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.907 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.907 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.907 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.907 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.907 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.907 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.907 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.907 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.907 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.907 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.907 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.907 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.908 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.908 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.908 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.908 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.908 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.908 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.908 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.908 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.908 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.908 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.908 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.908 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.908 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.908 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.909 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.909 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.909 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.909 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.909 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.909 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.909 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.909 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.909 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.909 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.909 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.909 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.909 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.910 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.910 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.910 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.910 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.910 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.910 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.910 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.910 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.910 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.910 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.910 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.910 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.910 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.910 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.911 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.911 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.911 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.911 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.911 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.911 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.911 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.911 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.911 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.911 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.911 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.911 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.911 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.911 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.912 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.912 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.912 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.912 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.912 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.912 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.912 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.912 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.912 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.912 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.912 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.912 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.913 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.913 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "nfs", 2024-09-17T04:29:04.913 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.913 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.913 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.913 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.913 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.913 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.913 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.913 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.913 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.913 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.914 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.914 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.914 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.914 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.914 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.914 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.914 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.914 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.914 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.914 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.914 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.914 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.914 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.914 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.915 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.915 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.915 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.915 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.915 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.915 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.915 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.915 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.915 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.915 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.915 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.915 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.915 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.915 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.915 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.916 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.916 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.916 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.916 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.916 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.916 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.916 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.916 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.916 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.916 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.916 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.916 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.916 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.917 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.917 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.917 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.917 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.917 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.917 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.917 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.917 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.917 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.917 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.917 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.917 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.917 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.917 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.918 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.918 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.918 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.918 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.918 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.918 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.918 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.918 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.918 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.918 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.918 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.918 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.918 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.918 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.918 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.919 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.919 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.919 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.919 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.919 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.919 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.919 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.919 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.919 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.919 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "orchestrator", 2024-09-17T04:29:04.919 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.919 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.919 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.919 INFO:teuthology.orchestra.run.smithi100.stdout: "fail_fs": { 2024-09-17T04:29:04.919 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "fail_fs", 2024-09-17T04:29:04.920 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.920 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.920 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.920 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.920 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.920 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.920 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.920 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-17T04:29:04.920 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.920 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.920 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.920 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.920 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.920 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.921 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.921 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.921 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.921 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.921 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.921 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.921 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.921 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.921 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.921 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.921 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.921 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.922 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.922 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.922 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.922 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.922 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.922 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.922 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.922 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.922 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.922 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.922 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.922 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.923 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.923 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.923 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.923 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.923 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.923 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.923 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.923 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.923 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.923 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.923 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.923 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.924 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.924 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.924 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.924 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.924 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.924 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.924 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.924 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.924 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.924 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.924 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.925 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.925 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.925 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.925 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.925 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.925 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.925 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.925 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.926 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.926 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.926 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.926 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.926 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.926 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.926 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.926 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.926 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.926 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.926 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.927 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.927 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.927 INFO:teuthology.orchestra.run.smithi100.stdout: "orchestrator": { 2024-09-17T04:29:04.927 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "orchestrator", 2024-09-17T04:29:04.927 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.927 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.927 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.927 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.927 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.927 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.927 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.927 INFO:teuthology.orchestra.run.smithi100.stdout: "cephadm", 2024-09-17T04:29:04.927 INFO:teuthology.orchestra.run.smithi100.stdout: "rook", 2024-09-17T04:29:04.927 INFO:teuthology.orchestra.run.smithi100.stdout: "test_orchestrator" 2024-09-17T04:29:04.928 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.928 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Orchestrator backend", 2024-09-17T04:29:04.928 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.928 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.928 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.928 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.928 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.928 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.928 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.928 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.928 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.928 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.929 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.929 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.929 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.929 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.929 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.929 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.929 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.929 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.929 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.929 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.929 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.929 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "osd_perf_query", 2024-09-17T04:29:04.929 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.930 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.930 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.930 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.930 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.930 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.930 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.930 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.930 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.930 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.930 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.930 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.930 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.930 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.930 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.930 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.930 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.931 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.931 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.931 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.931 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.931 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.931 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.931 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.931 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.931 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.931 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.931 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.931 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.931 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.931 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.931 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.932 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.932 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.932 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.932 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.932 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.932 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.932 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.932 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.932 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.932 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.932 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.932 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.932 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.932 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.932 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.933 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.933 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.933 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.933 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.933 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.933 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.933 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.933 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.933 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.933 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.933 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.933 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.934 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.934 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.934 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.934 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.934 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.934 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.934 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.934 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.934 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.934 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.934 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.934 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.934 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.934 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.935 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.935 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.935 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.935 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.935 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.935 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.935 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.935 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.935 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.935 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.935 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.935 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.935 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.935 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.935 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.936 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.936 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.936 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "osd_support", 2024-09-17T04:29:04.936 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.936 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.936 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.936 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.936 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.936 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.936 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.936 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.936 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.936 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.936 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.937 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.937 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.937 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.937 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.937 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.937 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.937 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.937 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.937 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.937 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.937 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.937 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.937 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.938 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.938 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.938 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.938 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.938 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.938 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.938 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.938 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.938 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.938 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.938 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.938 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.938 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.938 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.938 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.938 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.939 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.939 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.939 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.939 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.939 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.939 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.939 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.939 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.939 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.939 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.939 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.939 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.939 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.939 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.940 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.940 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.940 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.940 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.940 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.940 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.940 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.940 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.940 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.940 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.940 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.940 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.940 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.941 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.941 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.941 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.941 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.941 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.941 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.941 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.941 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.941 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.941 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.941 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.941 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.941 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.941 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.942 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.942 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.942 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.942 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.942 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.942 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.942 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.942 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.942 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.942 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pg_autoscaler", 2024-09-17T04:29:04.942 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.942 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.942 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.942 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.943 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.943 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.943 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.943 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.943 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.943 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.943 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.943 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.943 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.943 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.943 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.943 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.944 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.944 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.944 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.944 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.944 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.944 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.944 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.944 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.944 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.944 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.944 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.944 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.944 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.944 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.945 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.945 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.945 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.945 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.945 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.945 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.945 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.945 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.945 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.945 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.945 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.945 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.945 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.945 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.946 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.946 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.946 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.946 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.946 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.946 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.946 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.946 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.946 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.946 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.946 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.946 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.946 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.946 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.946 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.946 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.947 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.947 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.947 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.947 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.947 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.947 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.947 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.947 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.947 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.947 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.947 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.947 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.947 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.947 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-09-17T04:29:04.947 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-09-17T04:29:04.947 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.948 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.948 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.948 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-09-17T04:29:04.948 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.948 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.948 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.948 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.948 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.948 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.948 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.948 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.948 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.948 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.948 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.949 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.949 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.949 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.949 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.949 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.949 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.949 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.949 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.949 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.949 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.949 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.949 INFO:teuthology.orchestra.run.smithi100.stdout: "threshold": { 2024-09-17T04:29:04.949 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "threshold", 2024-09-17T04:29:04.949 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-09-17T04:29:04.950 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.950 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.950 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "3.0", 2024-09-17T04:29:04.950 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1.0", 2024-09-17T04:29:04.950 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.950 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.950 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "scaling threshold", 2024-09-17T04:29:04.950 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-17T04:29:04.950 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.950 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.950 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.950 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.950 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.951 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.951 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "progress", 2024-09-17T04:29:04.951 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.951 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.951 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.951 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_pg_recovery_event": { 2024-09-17T04:29:04.951 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "allow_pg_recovery_event", 2024-09-17T04:29:04.951 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.951 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.951 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.951 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.951 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.951 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.951 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.951 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-17T04:29:04.952 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.952 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.952 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.952 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.952 INFO:teuthology.orchestra.run.smithi100.stdout: "enabled": { 2024-09-17T04:29:04.952 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enabled", 2024-09-17T04:29:04.952 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.952 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.952 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.952 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.952 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.952 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.953 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.953 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.953 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.953 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.953 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.953 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.953 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.953 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.953 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.953 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.953 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.953 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.954 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.954 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.954 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.954 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.954 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.954 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.954 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.954 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.954 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.954 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.954 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.954 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.955 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.955 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.955 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.955 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.955 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.955 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.955 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.955 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.955 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.955 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.955 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.955 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.955 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.956 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.956 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.956 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.956 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.956 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.956 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.956 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.956 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.956 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.956 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.956 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.956 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.956 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.957 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.957 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.957 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.957 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.957 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.957 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.957 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.957 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.957 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.957 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.957 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.957 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.957 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.957 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.958 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.958 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.958 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.958 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.958 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.958 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.958 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.958 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.958 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.958 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.958 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.958 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.959 INFO:teuthology.orchestra.run.smithi100.stdout: "max_completed_events": { 2024-09-17T04:29:04.959 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_completed_events", 2024-09-17T04:29:04.959 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.959 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.959 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.959 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "50", 2024-09-17T04:29:04.959 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.959 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.959 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.959 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "number of past completed events to remember", 2024-09-17T04:29:04.959 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.959 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.959 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.960 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.960 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-09-17T04:29:04.960 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-09-17T04:29:04.960 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:04.960 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.960 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.960 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-09-17T04:29:04.960 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.960 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.960 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.960 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how long the module is going to sleep", 2024-09-17T04:29:04.960 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.960 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.961 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.961 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.961 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.961 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.961 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.961 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.961 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.961 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.961 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.961 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.961 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.961 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.961 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.961 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.962 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.962 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.962 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.962 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.962 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.962 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "prometheus", 2024-09-17T04:29:04.962 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.962 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.962 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.962 INFO:teuthology.orchestra.run.smithi100.stdout: "cache": { 2024-09-17T04:29:04.962 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cache", 2024-09-17T04:29:04.962 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.962 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.963 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.963 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.963 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.963 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.963 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.963 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.963 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.963 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.963 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.963 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.963 INFO:teuthology.orchestra.run.smithi100.stdout: "exclude_perf_counters": { 2024-09-17T04:29:04.963 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "exclude_perf_counters", 2024-09-17T04:29:04.963 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.963 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.964 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.964 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.964 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.964 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.964 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.964 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-17T04:29:04.964 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-17T04:29:04.964 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.964 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.964 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.964 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.964 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.964 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.964 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.964 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.965 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.965 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.965 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.965 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.965 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.965 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.965 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.965 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.965 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.965 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.965 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.965 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.965 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.966 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.966 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.966 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.966 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.966 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.966 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.966 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.966 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.966 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.966 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.966 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.966 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.966 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.966 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.966 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.967 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.967 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.967 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.967 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.967 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.967 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.967 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.967 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.967 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.967 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.967 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.967 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.967 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.967 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.967 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.968 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.968 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.968 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.968 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.968 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.968 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.968 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.968 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.968 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.968 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.968 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.968 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.968 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.968 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.968 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.969 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.969 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.969 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.969 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.969 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.969 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.969 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.969 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_stats_pools": { 2024-09-17T04:29:04.969 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rbd_stats_pools", 2024-09-17T04:29:04.969 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.969 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.969 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.969 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.969 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.969 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.970 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.970 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.970 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.970 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.970 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.970 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.970 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-17T04:29:04.970 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-17T04:29:04.970 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.970 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.970 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.970 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "300", 2024-09-17T04:29:04.970 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.970 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.971 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.971 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.971 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.971 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.971 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.971 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.971 INFO:teuthology.orchestra.run.smithi100.stdout: "scrape_interval": { 2024-09-17T04:29:04.971 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "scrape_interval", 2024-09-17T04:29:04.971 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-09-17T04:29:04.971 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.971 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.971 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "15.0", 2024-09-17T04:29:04.971 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.971 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.971 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.972 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.972 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.972 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.972 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.972 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.972 INFO:teuthology.orchestra.run.smithi100.stdout: "server_addr": { 2024-09-17T04:29:04.972 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_addr", 2024-09-17T04:29:04.972 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.972 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.972 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.972 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "::", 2024-09-17T04:29:04.972 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.972 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.972 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.972 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-17T04:29:04.973 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.973 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.973 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.973 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.973 INFO:teuthology.orchestra.run.smithi100.stdout: "server_port": { 2024-09-17T04:29:04.973 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_port", 2024-09-17T04:29:04.973 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.973 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.973 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.973 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "9283", 2024-09-17T04:29:04.973 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.973 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.973 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.974 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-17T04:29:04.974 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.974 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.974 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.974 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.974 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.974 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.974 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.974 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.974 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.974 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.974 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.974 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.974 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.974 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.975 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.975 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.975 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.975 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.975 INFO:teuthology.orchestra.run.smithi100.stdout: "stale_cache_strategy": { 2024-09-17T04:29:04.975 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "stale_cache_strategy", 2024-09-17T04:29:04.975 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.975 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.975 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.975 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "log", 2024-09-17T04:29:04.975 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.975 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.975 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.975 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.976 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.976 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.976 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.976 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.976 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_behaviour": { 2024-09-17T04:29:04.976 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_behaviour", 2024-09-17T04:29:04.976 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.976 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.976 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.976 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "default", 2024-09-17T04:29:04.976 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.976 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.976 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.976 INFO:teuthology.orchestra.run.smithi100.stdout: "default", 2024-09-17T04:29:04.977 INFO:teuthology.orchestra.run.smithi100.stdout: "error" 2024-09-17T04:29:04.977 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.977 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.977 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.977 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.977 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.977 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.977 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_error_status_code": { 2024-09-17T04:29:04.977 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_error_status_code", 2024-09-17T04:29:04.977 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.977 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.977 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.977 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "500", 2024-09-17T04:29:04.977 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "400", 2024-09-17T04:29:04.978 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "599", 2024-09-17T04:29:04.978 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.978 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.978 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.978 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.978 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.978 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.978 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.978 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.978 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.978 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rbd_support", 2024-09-17T04:29:04.978 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.978 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.979 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.979 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.979 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.979 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.979 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.979 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.979 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.979 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.979 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.979 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.979 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.979 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.979 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.979 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.980 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.980 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.980 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.980 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.980 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.980 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.980 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.980 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.980 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.980 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.980 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.980 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.981 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.981 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.981 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.981 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.981 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.981 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.981 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.981 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.981 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.981 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.981 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.981 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.981 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.982 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.982 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.982 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.982 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.982 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.982 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.982 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.982 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.982 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.982 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.982 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.982 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.982 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.983 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.983 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.983 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.983 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.983 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.983 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.983 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.983 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.983 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.983 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.983 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.983 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.983 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.984 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.984 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.984 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.984 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.984 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.984 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.984 INFO:teuthology.orchestra.run.smithi100.stdout: "max_concurrent_snap_create": { 2024-09-17T04:29:04.984 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_concurrent_snap_create", 2024-09-17T04:29:04.984 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.984 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.984 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.984 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-09-17T04:29:04.984 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.985 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.985 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.985 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.985 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.985 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.985 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.985 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.985 INFO:teuthology.orchestra.run.smithi100.stdout: "mirror_snapshot_schedule": { 2024-09-17T04:29:04.985 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mirror_snapshot_schedule", 2024-09-17T04:29:04.985 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.985 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.985 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.986 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.986 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.986 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.986 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.986 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.986 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.986 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.986 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.986 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.986 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.986 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.986 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.986 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.986 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.986 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.987 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.987 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.987 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.987 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.987 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.987 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.987 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.987 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.987 INFO:teuthology.orchestra.run.smithi100.stdout: "trash_purge_schedule": { 2024-09-17T04:29:04.987 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "trash_purge_schedule", 2024-09-17T04:29:04.987 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.987 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.988 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.988 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.988 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.988 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.988 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.988 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.988 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.988 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.988 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.988 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.988 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:04.988 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.988 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:04.989 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "restful", 2024-09-17T04:29:04.989 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:04.989 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:04.989 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:04.989 INFO:teuthology.orchestra.run.smithi100.stdout: "enable_auth": { 2024-09-17T04:29:04.989 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enable_auth", 2024-09-17T04:29:04.989 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.989 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.989 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.989 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:04.989 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.989 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.989 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.989 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.989 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.990 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.990 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.990 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.990 INFO:teuthology.orchestra.run.smithi100.stdout: "key_file": { 2024-09-17T04:29:04.990 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "key_file", 2024-09-17T04:29:04.990 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.990 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.990 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.990 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.990 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.990 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.990 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.990 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.990 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.991 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.991 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.991 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.991 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:04.991 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:04.991 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.991 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.991 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.991 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.991 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.991 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.991 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.991 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.992 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.992 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.992 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.992 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.992 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.992 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.992 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.992 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.992 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.992 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.992 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.992 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:04.992 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:04.992 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.993 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.993 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.993 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.993 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.993 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.993 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.993 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.993 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.993 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.993 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.993 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.993 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:04.993 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:04.994 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.994 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.994 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.994 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:04.994 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.994 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.994 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:04.994 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:04.994 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:04.994 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:04.994 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:04.994 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:04.994 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:04.994 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:04.994 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.995 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.995 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.995 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.995 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.995 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:04.995 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:04.995 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:04.995 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.995 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.995 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:04.995 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.995 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.995 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.995 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.996 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.996 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.996 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.996 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.996 INFO:teuthology.orchestra.run.smithi100.stdout: "max_requests": { 2024-09-17T04:29:04.996 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_requests", 2024-09-17T04:29:04.996 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.996 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.996 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.996 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "500", 2024-09-17T04:29:04.996 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.996 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.996 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.996 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-17T04:29:04.996 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.997 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.997 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.997 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.997 INFO:teuthology.orchestra.run.smithi100.stdout: "server_addr": { 2024-09-17T04:29:04.997 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_addr", 2024-09-17T04:29:04.997 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.997 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.997 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.997 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.997 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.997 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.997 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.997 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.997 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.998 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.998 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.998 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.998 INFO:teuthology.orchestra.run.smithi100.stdout: "server_port": { 2024-09-17T04:29:04.998 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_port", 2024-09-17T04:29:04.998 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:04.998 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:04.998 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:04.998 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:04.998 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.998 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.998 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.998 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.998 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:04.999 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:04.999 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:04.999 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:04.999 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:04.999 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:04.999 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:04.999 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:04.999 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:04.999 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:04.999 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:04.999 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:04.999 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:04.999 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:04.999 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.000 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.000 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.000 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.000 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.000 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.000 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.000 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rgw", 2024-09-17T04:29:05.000 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:05.000 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:05.000 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:05.000 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:05.000 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:05.000 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.000 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.001 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.001 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.001 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.001 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.001 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:05.001 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:05.001 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:05.001 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:05.001 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:05.001 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:05.001 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:05.001 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.001 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.001 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.002 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.002 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.002 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.002 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:05.002 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:05.002 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.002 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.002 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.002 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.002 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.002 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.002 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.002 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.002 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.002 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.003 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.003 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.003 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:05.003 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:05.003 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.003 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.003 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.003 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:05.003 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.003 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.003 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:05.003 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:05.003 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:05.004 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:05.004 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:05.004 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:05.004 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:05.004 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.004 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.004 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.004 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.004 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.004 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.004 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:05.004 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:05.004 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.004 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.005 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.005 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.005 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.005 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.005 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.005 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.005 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.005 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.005 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.005 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.005 INFO:teuthology.orchestra.run.smithi100.stdout: "secondary_zone_period_retry_limit": { 2024-09-17T04:29:05.005 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-17T04:29:05.005 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:05.005 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.006 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.006 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-09-17T04:29:05.006 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.006 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.006 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.006 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-17T04:29:05.006 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.006 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.006 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.006 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.006 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:05.006 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:05.006 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:05.006 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:05.006 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.007 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:05.007 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.007 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.007 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.007 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.007 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.007 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.007 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.007 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.007 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.007 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.007 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.007 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rook", 2024-09-17T04:29:05.007 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:05.007 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:05.008 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:05.008 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:05.008 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:05.008 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.008 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.008 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.008 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.008 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.008 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.008 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:05.008 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:05.008 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:05.008 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:05.008 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:05.009 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:05.009 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:05.009 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.009 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.009 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.009 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.009 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.009 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.009 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:05.009 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:05.009 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.009 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.010 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.010 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.010 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.010 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.010 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.010 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.010 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.010 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.010 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.010 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.010 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:05.010 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:05.010 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.010 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.011 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.011 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:05.011 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.011 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.011 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:05.011 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:05.011 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:05.011 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:05.011 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:05.011 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:05.011 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:05.011 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.011 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.012 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.012 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.012 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.012 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.012 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:05.012 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:05.012 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.012 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.012 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.012 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.012 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.012 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.012 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.012 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.013 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.013 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.013 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.013 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.013 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:05.013 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:05.013 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:05.013 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:05.013 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.013 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:05.013 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.013 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.014 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.014 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.014 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.014 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.014 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.014 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.014 INFO:teuthology.orchestra.run.smithi100.stdout: "storage_class": { 2024-09-17T04:29:05.014 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "storage_class", 2024-09-17T04:29:05.014 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.014 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.014 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.014 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "local", 2024-09-17T04:29:05.014 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.014 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.015 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.015 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-17T04:29:05.015 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.015 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.015 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.015 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.015 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.015 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.015 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.015 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "selftest", 2024-09-17T04:29:05.015 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:05.015 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:05.015 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:05.016 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:05.016 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:05.016 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.016 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.016 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.016 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.016 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.016 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.016 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:05.016 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:05.016 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:05.016 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:05.016 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:05.017 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:05.017 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:05.017 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.017 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.017 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.017 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.017 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.017 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.017 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:05.017 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:05.017 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.017 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.018 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.018 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.018 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.018 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.018 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.018 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.018 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.018 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.018 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.018 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.018 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:05.019 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:05.019 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.019 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.019 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.019 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:05.019 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.019 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.019 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:05.019 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:05.019 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:05.019 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:05.019 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:05.019 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:05.020 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:05.020 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.020 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.020 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.020 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.020 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.020 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.020 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:05.020 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:05.020 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.020 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.020 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.020 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.020 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.021 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.021 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.021 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.021 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.021 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.021 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.021 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.021 INFO:teuthology.orchestra.run.smithi100.stdout: "roption1": { 2024-09-17T04:29:05.021 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "roption1", 2024-09-17T04:29:05.021 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.021 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.021 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.022 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.022 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.022 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.022 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.022 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.022 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.022 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.022 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.022 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.022 INFO:teuthology.orchestra.run.smithi100.stdout: "roption2": { 2024-09-17T04:29:05.022 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "roption2", 2024-09-17T04:29:05.022 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.022 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.023 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.023 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "xyz", 2024-09-17T04:29:05.023 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.023 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.023 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.023 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.023 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.023 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.023 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.023 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.023 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption1": { 2024-09-17T04:29:05.023 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption1", 2024-09-17T04:29:05.023 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.024 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.024 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.024 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.024 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.024 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.024 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.024 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.024 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.024 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.024 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.024 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.024 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption2": { 2024-09-17T04:29:05.024 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption2", 2024-09-17T04:29:05.025 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:05.025 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.025 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.025 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.025 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.025 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.025 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.025 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.025 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.025 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.025 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.025 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.025 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption3": { 2024-09-17T04:29:05.025 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption3", 2024-09-17T04:29:05.026 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-09-17T04:29:05.026 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.026 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.026 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.026 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.026 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.026 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.026 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.026 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.026 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.026 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.026 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.026 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption4": { 2024-09-17T04:29:05.026 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption4", 2024-09-17T04:29:05.027 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.027 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.027 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.027 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.027 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.027 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.027 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.027 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.027 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.027 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.027 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.027 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.027 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption5": { 2024-09-17T04:29:05.028 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption5", 2024-09-17T04:29:05.028 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.028 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.028 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.028 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.028 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.028 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.028 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.028 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.028 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.028 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.028 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.028 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.029 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption6": { 2024-09-17T04:29:05.029 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption6", 2024-09-17T04:29:05.029 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.029 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.029 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.029 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:05.029 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.029 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.029 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.029 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.029 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.029 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.030 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.030 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.030 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption7": { 2024-09-17T04:29:05.030 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption7", 2024-09-17T04:29:05.030 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:05.030 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.030 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.030 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.030 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-09-17T04:29:05.030 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "42", 2024-09-17T04:29:05.030 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.030 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.030 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.031 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.031 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.031 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.031 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:05.031 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:05.031 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:05.031 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:05.031 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.031 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:05.031 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.031 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.031 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.031 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.031 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.032 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.032 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.032 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.032 INFO:teuthology.orchestra.run.smithi100.stdout: "testkey": { 2024-09-17T04:29:05.032 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "testkey", 2024-09-17T04:29:05.032 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.032 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.032 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.032 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.032 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.032 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.032 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.032 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.032 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.033 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.033 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.033 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.033 INFO:teuthology.orchestra.run.smithi100.stdout: "testlkey": { 2024-09-17T04:29:05.033 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "testlkey", 2024-09-17T04:29:05.033 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.033 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.033 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.033 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.033 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.033 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.033 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.033 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.034 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.034 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.034 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.034 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.034 INFO:teuthology.orchestra.run.smithi100.stdout: "testnewline": { 2024-09-17T04:29:05.034 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "testnewline", 2024-09-17T04:29:05.034 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.034 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.034 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.034 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.034 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.034 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.034 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.034 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.034 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.035 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.035 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.035 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.035 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.035 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.035 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.035 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smb", 2024-09-17T04:29:05.035 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:05.035 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:05.035 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:05.035 INFO:teuthology.orchestra.run.smithi100.stdout: "internal_store_backend": { 2024-09-17T04:29:05.035 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "internal_store_backend", 2024-09-17T04:29:05.035 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.035 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:05.036 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.036 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.036 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.036 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.036 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.036 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-17T04:29:05.036 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.036 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.036 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.036 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.036 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:05.036 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:05.036 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.037 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.037 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.037 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.037 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.037 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.037 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:05.037 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:05.037 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:05.037 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:05.037 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:05.037 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:05.037 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:05.037 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.038 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.038 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.038 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.038 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.038 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.038 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:05.038 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:05.038 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.038 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.038 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.038 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.038 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.038 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.039 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.039 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.039 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.039 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.039 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.039 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.039 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:05.039 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:05.039 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.039 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.039 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.039 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:05.039 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.040 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.040 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:05.040 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:05.040 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:05.040 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:05.040 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:05.040 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:05.040 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:05.040 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.040 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.040 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.040 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.041 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.041 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.041 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:05.041 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:05.041 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.041 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.041 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.041 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.041 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.041 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.041 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.041 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.041 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.042 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.042 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.042 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.042 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:05.042 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:05.042 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:05.042 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:05.042 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.042 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:05.042 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.042 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.042 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.042 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.043 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.043 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.043 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.043 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.043 INFO:teuthology.orchestra.run.smithi100.stdout: "update_orchestration": { 2024-09-17T04:29:05.043 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "update_orchestration", 2024-09-17T04:29:05.043 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.043 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.043 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.043 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:05.043 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.043 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.043 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.044 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-17T04:29:05.044 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.044 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.044 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.044 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.044 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.044 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.044 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.044 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "snap_schedule", 2024-09-17T04:29:05.044 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:05.044 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:05.044 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:05.044 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_m_granularity": { 2024-09-17T04:29:05.044 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "allow_m_granularity", 2024-09-17T04:29:05.045 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.045 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.045 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.045 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.045 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.045 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.045 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.045 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "allow minute scheduled snapshots", 2024-09-17T04:29:05.045 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.045 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.045 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.045 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.045 INFO:teuthology.orchestra.run.smithi100.stdout: "dump_on_update": { 2024-09-17T04:29:05.046 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "dump_on_update", 2024-09-17T04:29:05.046 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.046 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.046 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.046 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.046 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.046 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.046 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.046 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "dump database to debug log on update", 2024-09-17T04:29:05.046 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.046 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.046 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.046 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.046 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:05.047 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:05.047 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.047 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.047 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.047 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.047 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.047 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.047 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:05.047 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:05.047 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:05.047 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:05.047 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:05.047 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:05.047 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:05.048 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.048 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.048 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.048 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.048 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.048 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.048 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:05.048 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:05.048 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.048 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.048 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.048 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.048 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.049 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.049 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.049 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.049 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.049 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.049 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.049 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.049 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:05.049 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:05.049 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.049 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.049 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.049 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:05.050 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.050 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.050 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:05.050 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:05.050 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:05.050 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:05.050 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:05.050 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:05.050 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:05.050 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.050 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.050 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.050 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.050 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.051 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.051 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:05.051 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:05.051 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.051 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.051 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.051 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.051 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.051 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.051 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.051 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.051 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.051 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.052 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.052 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.052 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:05.052 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:05.052 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:05.052 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:05.052 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.052 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:05.052 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.052 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.052 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.052 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.052 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.053 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.053 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.053 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.053 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.053 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.053 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.053 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "stats", 2024-09-17T04:29:05.053 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:05.053 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:05.053 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:05.053 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:05.053 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:05.054 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.054 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.054 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.054 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.054 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.054 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.054 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:05.054 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:05.054 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:05.054 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:05.054 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:05.054 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:05.054 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:05.055 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.055 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.055 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.055 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.055 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.055 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.055 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:05.055 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:05.055 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.055 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.055 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.055 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.055 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.056 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.056 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.056 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.056 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.056 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.056 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.056 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.056 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:05.056 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:05.056 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.056 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.056 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.056 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:05.056 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.057 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.057 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:05.057 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:05.057 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:05.057 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:05.057 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:05.057 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:05.057 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:05.057 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.057 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.057 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.057 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.057 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.058 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.058 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:05.058 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:05.058 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.058 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.058 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.058 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.058 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.058 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.058 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.058 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.058 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.058 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.058 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.059 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.059 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:05.059 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:05.059 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:05.059 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:05.059 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.059 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:05.059 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.059 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.059 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.059 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.059 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.059 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.059 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.059 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.060 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.060 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.060 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.060 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "status", 2024-09-17T04:29:05.060 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:05.060 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:05.060 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:05.060 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:05.060 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:05.060 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.060 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.060 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.060 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.060 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.061 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.061 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:05.061 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:05.061 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:05.061 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:05.061 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:05.061 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:05.061 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:05.061 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.061 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.061 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.061 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.061 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.062 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.062 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:05.062 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:05.062 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.062 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.062 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.062 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.062 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.062 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.062 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.062 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.062 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.062 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.062 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.063 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.063 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:05.063 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:05.063 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.063 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.063 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.063 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:05.063 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.063 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.063 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:05.063 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:05.063 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:05.064 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:05.064 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:05.064 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:05.064 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:05.064 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.064 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.064 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.064 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.064 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.064 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.064 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:05.064 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:05.064 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.065 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.065 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.065 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.065 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.065 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.065 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.065 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.065 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.065 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.065 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.065 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.065 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:05.065 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:05.065 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:05.066 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:05.066 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.066 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:05.066 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.066 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.066 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.066 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.066 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.066 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.066 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.066 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.066 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.067 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.067 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.067 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "telegraf", 2024-09-17T04:29:05.067 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:05.067 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:05.067 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:05.067 INFO:teuthology.orchestra.run.smithi100.stdout: "address": { 2024-09-17T04:29:05.067 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "address", 2024-09-17T04:29:05.067 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.067 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.067 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.067 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-17T04:29:05.067 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.067 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.068 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.068 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.068 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.068 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.068 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.068 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.068 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-09-17T04:29:05.068 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-09-17T04:29:05.068 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:05.068 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.068 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.068 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "15", 2024-09-17T04:29:05.068 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.069 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.069 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.069 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.069 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.069 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.069 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.069 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.069 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:05.069 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:05.069 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.069 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.069 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.069 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.070 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.070 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.070 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:05.070 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:05.070 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:05.070 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:05.070 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:05.070 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:05.070 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:05.070 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.070 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.070 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.070 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.071 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.071 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.071 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:05.071 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:05.071 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.071 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.071 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.071 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.071 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.071 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.071 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.071 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.072 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.072 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.072 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.072 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.072 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:05.072 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:05.072 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.072 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.072 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.072 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:05.072 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.072 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.072 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:05.073 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:05.073 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:05.073 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:05.073 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:05.073 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:05.073 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:05.073 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.073 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.073 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.073 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.073 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.073 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.074 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:05.074 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:05.074 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.074 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.074 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.074 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.074 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.074 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.074 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.074 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.074 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.074 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.074 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.075 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.075 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:05.075 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:05.075 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:05.075 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:05.075 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.075 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:05.075 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.075 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.075 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.075 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.075 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.075 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.075 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.076 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.076 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.076 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.076 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.076 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "telemetry", 2024-09-17T04:29:05.076 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:05.076 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:05.076 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:05.076 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_basic": { 2024-09-17T04:29:05.076 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_basic", 2024-09-17T04:29:05.076 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.076 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.076 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.077 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:05.077 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.077 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.077 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.077 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-17T04:29:05.077 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.077 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.077 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.077 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.077 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_crash": { 2024-09-17T04:29:05.077 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_crash", 2024-09-17T04:29:05.077 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.077 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.078 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.078 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:05.078 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.078 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.078 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.078 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-17T04:29:05.078 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.078 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.078 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.078 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.078 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_device": { 2024-09-17T04:29:05.078 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_device", 2024-09-17T04:29:05.078 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.078 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.079 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.079 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:05.079 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.079 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.079 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.079 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-17T04:29:05.079 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.079 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.079 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.079 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.079 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_ident": { 2024-09-17T04:29:05.079 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_ident", 2024-09-17T04:29:05.079 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.080 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.080 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.080 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.080 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.080 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.080 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.080 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-17T04:29:05.080 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.080 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.080 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.080 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.080 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_perf": { 2024-09-17T04:29:05.081 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_perf", 2024-09-17T04:29:05.081 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.081 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.081 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.081 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.081 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.081 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.081 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.081 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-17T04:29:05.081 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.081 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.081 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.082 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.082 INFO:teuthology.orchestra.run.smithi100.stdout: "contact": { 2024-09-17T04:29:05.082 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "contact", 2024-09-17T04:29:05.082 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.082 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.082 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.082 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.082 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.082 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.082 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.082 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.082 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.082 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.082 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.082 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.083 INFO:teuthology.orchestra.run.smithi100.stdout: "description": { 2024-09-17T04:29:05.083 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "description", 2024-09-17T04:29:05.083 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.083 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.083 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.083 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.083 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.083 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.083 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.083 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.083 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.083 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.083 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.084 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.084 INFO:teuthology.orchestra.run.smithi100.stdout: "device_url": { 2024-09-17T04:29:05.084 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "device_url", 2024-09-17T04:29:05.084 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.084 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.084 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.084 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-17T04:29:05.084 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.084 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.084 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.084 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.084 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.084 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.084 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.084 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.085 INFO:teuthology.orchestra.run.smithi100.stdout: "enabled": { 2024-09-17T04:29:05.085 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enabled", 2024-09-17T04:29:05.085 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.085 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.085 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.085 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.085 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.085 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.085 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.085 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.085 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.085 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.085 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.085 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.086 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-09-17T04:29:05.086 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-09-17T04:29:05.086 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:05.086 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.086 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.086 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "24", 2024-09-17T04:29:05.086 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "8", 2024-09-17T04:29:05.086 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.086 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.086 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.086 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.086 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.086 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.087 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.087 INFO:teuthology.orchestra.run.smithi100.stdout: "last_opt_revision": { 2024-09-17T04:29:05.087 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "last_opt_revision", 2024-09-17T04:29:05.087 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:05.087 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.087 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.087 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "1", 2024-09-17T04:29:05.087 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.087 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.087 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.087 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.087 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.087 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.087 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.088 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.088 INFO:teuthology.orchestra.run.smithi100.stdout: "leaderboard": { 2024-09-17T04:29:05.088 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "leaderboard", 2024-09-17T04:29:05.088 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.088 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.088 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.088 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.088 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.088 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.088 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.088 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.088 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.088 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.088 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.089 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.089 INFO:teuthology.orchestra.run.smithi100.stdout: "leaderboard_description": { 2024-09-17T04:29:05.089 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "leaderboard_description", 2024-09-17T04:29:05.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:04.144+0000 7ff4cd000640 1 -- 172.21.15.100:0/3710624614 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff4a8078ef0 msgr2=0x7ff4a807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:05.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:04.144+0000 7ff4cd000640 1 --2- 172.21.15.100:0/3710624614 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff4a8078ef0 0x7ff4a807b400 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4007fd0 tx=0x7ff4b4009040 comp rx=0 tx=0).stop 2024-09-17T04:29:05.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:04.144+0000 7ff4cd000640 1 -- 172.21.15.100:0/3710624614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c80aebc0 msgr2=0x7ff4c80b1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:05.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:04.144+0000 7ff4cd000640 1 --2- 172.21.15.100:0/3710624614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c80aebc0 0x7ff4c80b1b10 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800e790 tx=0x7ff4b800ec50 comp rx=0 tx=0).stop 2024-09-17T04:29:05.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:04.144+0000 7ff4cd000640 1 -- 172.21.15.100:0/3710624614 shutdown_connections 2024-09-17T04:29:05.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:04.144+0000 7ff4cd000640 1 --2- 172.21.15.100:0/3710624614 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7ff4a8078ef0 0x7ff4a807b400 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:05.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:04.144+0000 7ff4cd000640 1 --2- 172.21.15.100:0/3710624614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4c81f9000 0x7ff4c81f9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:05.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:04.144+0000 7ff4cd000640 1 --2- 172.21.15.100:0/3710624614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4c80b2750 0x7ff4c80ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:05.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:04.144+0000 7ff4cd000640 1 --2- 172.21.15.100:0/3710624614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c80aebc0 0x7ff4c80b1b10 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:05.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:04.144+0000 7ff4cd000640 1 -- 172.21.15.100:0/3710624614 >> 172.21.15.100:0/3710624614 conn(0x7ff4c806d730 msgr2=0x7ff4c80d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:05.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:04.144+0000 7ff4cd000640 1 -- 172.21.15.100:0/3710624614 shutdown_connections 2024-09-17T04:29:05.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:04.144+0000 7ff4cd000640 1 -- 172.21.15.100:0/3710624614 wait complete. 2024-09-17T04:29:05.091 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.091 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.091 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.091 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.091 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.091 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.091 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.091 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.091 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.092 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.092 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.092 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.092 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:05.092 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:05.092 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.092 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.092 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.092 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.092 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.092 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.092 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:05.092 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:05.092 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:05.093 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:05.093 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:05.093 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:05.093 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:05.093 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.093 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.093 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.093 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.093 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.093 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.093 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:05.093 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:05.094 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.094 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.094 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.094 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.094 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.094 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.094 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.094 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.094 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.094 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.094 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.094 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.094 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:05.094 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:05.094 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.095 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.095 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.095 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:05.095 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.095 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.095 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:05.095 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:05.095 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:05.095 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:05.095 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:05.095 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:05.095 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:05.095 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.095 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.095 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.095 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.096 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.096 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.096 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:05.096 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:05.096 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.096 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.096 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.096 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.096 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.096 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.096 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.096 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.096 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.096 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.097 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.097 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.097 INFO:teuthology.orchestra.run.smithi100.stdout: "organization": { 2024-09-17T04:29:05.097 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "organization", 2024-09-17T04:29:05.097 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.097 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.097 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.097 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.097 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.097 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.097 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.097 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.097 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.097 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.098 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.098 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.098 INFO:teuthology.orchestra.run.smithi100.stdout: "proxy": { 2024-09-17T04:29:05.098 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "proxy", 2024-09-17T04:29:05.098 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.098 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.098 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.098 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.098 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.098 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.098 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.098 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.098 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.098 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.099 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.099 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.099 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:05.099 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:05.099 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:05.099 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:05.099 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.099 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:05.099 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.099 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.099 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.099 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.099 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.100 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.100 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.100 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.100 INFO:teuthology.orchestra.run.smithi100.stdout: "url": { 2024-09-17T04:29:05.100 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "url", 2024-09-17T04:29:05.100 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.100 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.100 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.100 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-17T04:29:05.100 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.100 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.100 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.100 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.100 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.101 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.101 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.101 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.101 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.101 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.101 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.101 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "test_orchestrator", 2024-09-17T04:29:05.101 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:05.101 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:05.101 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:05.101 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:05.102 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:05.102 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.102 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.102 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.102 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.102 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.102 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.102 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:05.102 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:05.102 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:05.102 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:05.102 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:05.102 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:05.102 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:05.103 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.103 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.103 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.103 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.103 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.103 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.103 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:05.103 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:05.103 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.103 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.103 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.103 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.104 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.104 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.104 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.104 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.104 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.104 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.104 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.104 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.104 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:05.104 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:05.104 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.104 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.104 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.104 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:05.105 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.105 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.105 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:05.105 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:05.105 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:05.105 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:05.105 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:05.105 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:05.105 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:05.105 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.105 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.105 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.105 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.105 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.106 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.106 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:05.106 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:05.106 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.106 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.106 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.106 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.106 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.106 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.106 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.106 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.106 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.106 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.106 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.107 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.107 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:05.107 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:05.107 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:05.107 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:05.107 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.107 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:05.107 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.107 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.107 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.107 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.107 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.107 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.108 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.108 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.108 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.108 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.108 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.108 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "volumes", 2024-09-17T04:29:05.108 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:05.108 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:05.108 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:05.108 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:05.108 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:05.108 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.108 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.109 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.109 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.109 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.109 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.109 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:05.109 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:05.109 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:05.109 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:05.109 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:05.109 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:05.109 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:05.109 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.109 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.110 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.110 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.110 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.110 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.110 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:05.110 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:05.110 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.110 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.110 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.110 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.110 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.110 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.110 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.110 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.111 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.111 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.111 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.111 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.111 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:05.111 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:05.111 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.111 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.111 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.111 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:05.111 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.111 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.111 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:05.112 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:05.112 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:05.112 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:05.112 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:05.112 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:05.112 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:05.112 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.112 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.112 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.112 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.112 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.112 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.113 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:05.113 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:05.113 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.113 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.113 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.113 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.113 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.113 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.113 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.113 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.113 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.113 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.114 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.114 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.114 INFO:teuthology.orchestra.run.smithi100.stdout: "max_concurrent_clones": { 2024-09-17T04:29:05.114 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_concurrent_clones", 2024-09-17T04:29:05.114 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:05.114 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.114 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.114 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "4", 2024-09-17T04:29:05.114 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.114 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.114 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.114 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-17T04:29:05.114 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.114 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.115 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.115 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.115 INFO:teuthology.orchestra.run.smithi100.stdout: "periodic_async_work": { 2024-09-17T04:29:05.115 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "periodic_async_work", 2024-09-17T04:29:05.115 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.115 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.115 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.115 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.115 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.115 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.115 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.115 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Periodically check for async work", 2024-09-17T04:29:05.115 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.115 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.115 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.116 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.116 INFO:teuthology.orchestra.run.smithi100.stdout: "snapshot_clone_delay": { 2024-09-17T04:29:05.116 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "snapshot_clone_delay", 2024-09-17T04:29:05.116 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:05.116 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.116 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.116 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:05.116 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.116 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.116 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.116 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-17T04:29:05.116 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.116 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.117 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.117 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.117 INFO:teuthology.orchestra.run.smithi100.stdout: "snapshot_clone_no_wait": { 2024-09-17T04:29:05.117 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "snapshot_clone_no_wait", 2024-09-17T04:29:05.117 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.117 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.117 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.117 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-09-17T04:29:05.117 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.117 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.117 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.117 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-17T04:29:05.117 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.117 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.118 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.118 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.118 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:05.118 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:05.118 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:05.118 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:05.118 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.118 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:05.118 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.118 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.118 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.118 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.118 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.118 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.118 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.119 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.119 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.119 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.119 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.119 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix", 2024-09-17T04:29:05.119 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-09-17T04:29:05.119 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-09-17T04:29:05.119 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-09-17T04:29:05.119 INFO:teuthology.orchestra.run.smithi100.stdout: "discovery_interval": { 2024-09-17T04:29:05.119 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "discovery_interval", 2024-09-17T04:29:05.119 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-09-17T04:29:05.119 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.119 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.119 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "100", 2024-09-17T04:29:05.120 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.120 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.120 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.120 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.120 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.120 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.120 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.120 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.120 INFO:teuthology.orchestra.run.smithi100.stdout: "identifier": { 2024-09-17T04:29:05.120 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "identifier", 2024-09-17T04:29:05.120 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.120 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.120 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.121 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.121 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.121 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.121 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.121 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.121 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.121 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.121 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.121 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.121 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-09-17T04:29:05.121 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-09-17T04:29:05.121 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-09-17T04:29:05.121 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.122 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.122 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-09-17T04:29:05.122 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.122 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.122 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.122 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.122 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.122 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.122 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.122 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.122 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-09-17T04:29:05.122 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-09-17T04:29:05.122 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.122 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.123 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.123 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.123 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.123 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.123 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:05.123 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:05.123 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:05.123 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:05.123 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:05.123 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:05.123 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:05.123 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.123 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.124 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.124 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.124 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.124 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.124 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-09-17T04:29:05.124 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-09-17T04:29:05.124 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.124 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.124 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.124 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.124 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.124 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.124 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.125 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.125 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.125 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.125 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.125 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.125 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-09-17T04:29:05.125 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-09-17T04:29:05.125 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.125 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.125 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.125 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-09-17T04:29:05.125 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.125 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.126 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-09-17T04:29:05.126 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-09-17T04:29:05.126 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-09-17T04:29:05.126 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-09-17T04:29:05.126 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-09-17T04:29:05.126 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-09-17T04:29:05.126 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-09-17T04:29:05.126 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.126 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.126 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.126 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.126 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.127 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.127 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-09-17T04:29:05.127 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-09-17T04:29:05.127 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-09-17T04:29:05.127 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.127 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.127 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-09-17T04:29:05.127 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.127 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.127 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.127 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.127 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.127 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.128 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.128 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.128 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-09-17T04:29:05.128 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-09-17T04:29:05.128 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:05.128 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-09-17T04:29:05.128 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-17T04:29:05.128 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-09-17T04:29:05.128 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.128 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.128 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.128 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.128 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.129 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.129 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.129 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.129 INFO:teuthology.orchestra.run.smithi100.stdout: "zabbix_host": { 2024-09-17T04:29:05.129 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix_host", 2024-09-17T04:29:05.129 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.129 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.129 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.129 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-09-17T04:29:05.129 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.129 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.129 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.129 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.129 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.130 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.130 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.130 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.130 INFO:teuthology.orchestra.run.smithi100.stdout: "zabbix_port": { 2024-09-17T04:29:05.130 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix_port", 2024-09-17T04:29:05.130 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-09-17T04:29:05.130 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.130 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.130 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10051", 2024-09-17T04:29:05.130 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.130 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.130 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.130 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.130 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.131 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.131 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.131 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.131 INFO:teuthology.orchestra.run.smithi100.stdout: "zabbix_sender": { 2024-09-17T04:29:05.131 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix_sender", 2024-09-17T04:29:05.131 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-09-17T04:29:05.131 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-09-17T04:29:05.131 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-09-17T04:29:05.131 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-17T04:29:05.131 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-09-17T04:29:05.131 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-09-17T04:29:05.131 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-09-17T04:29:05.131 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-09-17T04:29:05.131 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-09-17T04:29:05.132 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-09-17T04:29:05.132 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-09-17T04:29:05.132 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.132 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.132 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.132 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.132 INFO:teuthology.orchestra.run.smithi100.stdout: "services": {}, 2024-09-17T04:29:05.132 INFO:teuthology.orchestra.run.smithi100.stdout: "always_on_modules": { 2024-09-17T04:29:05.132 INFO:teuthology.orchestra.run.smithi100.stdout: "octopus": [ 2024-09-17T04:29:05.132 INFO:teuthology.orchestra.run.smithi100.stdout: "balancer", 2024-09-17T04:29:05.132 INFO:teuthology.orchestra.run.smithi100.stdout: "crash", 2024-09-17T04:29:05.132 INFO:teuthology.orchestra.run.smithi100.stdout: "devicehealth", 2024-09-17T04:29:05.132 INFO:teuthology.orchestra.run.smithi100.stdout: "orchestrator", 2024-09-17T04:29:05.132 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscaler", 2024-09-17T04:29:05.133 INFO:teuthology.orchestra.run.smithi100.stdout: "progress", 2024-09-17T04:29:05.133 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_support", 2024-09-17T04:29:05.133 INFO:teuthology.orchestra.run.smithi100.stdout: "status", 2024-09-17T04:29:05.133 INFO:teuthology.orchestra.run.smithi100.stdout: "telemetry", 2024-09-17T04:29:05.133 INFO:teuthology.orchestra.run.smithi100.stdout: "volumes" 2024-09-17T04:29:05.133 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.133 INFO:teuthology.orchestra.run.smithi100.stdout: "pacific": [ 2024-09-17T04:29:05.133 INFO:teuthology.orchestra.run.smithi100.stdout: "balancer", 2024-09-17T04:29:05.133 INFO:teuthology.orchestra.run.smithi100.stdout: "crash", 2024-09-17T04:29:05.133 INFO:teuthology.orchestra.run.smithi100.stdout: "devicehealth", 2024-09-17T04:29:05.133 INFO:teuthology.orchestra.run.smithi100.stdout: "orchestrator", 2024-09-17T04:29:05.133 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscaler", 2024-09-17T04:29:05.133 INFO:teuthology.orchestra.run.smithi100.stdout: "progress", 2024-09-17T04:29:05.134 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_support", 2024-09-17T04:29:05.134 INFO:teuthology.orchestra.run.smithi100.stdout: "status", 2024-09-17T04:29:05.134 INFO:teuthology.orchestra.run.smithi100.stdout: "telemetry", 2024-09-17T04:29:05.134 INFO:teuthology.orchestra.run.smithi100.stdout: "volumes" 2024-09-17T04:29:05.134 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.134 INFO:teuthology.orchestra.run.smithi100.stdout: "quincy": [ 2024-09-17T04:29:05.134 INFO:teuthology.orchestra.run.smithi100.stdout: "balancer", 2024-09-17T04:29:05.134 INFO:teuthology.orchestra.run.smithi100.stdout: "crash", 2024-09-17T04:29:05.134 INFO:teuthology.orchestra.run.smithi100.stdout: "devicehealth", 2024-09-17T04:29:05.134 INFO:teuthology.orchestra.run.smithi100.stdout: "orchestrator", 2024-09-17T04:29:05.134 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscaler", 2024-09-17T04:29:05.134 INFO:teuthology.orchestra.run.smithi100.stdout: "progress", 2024-09-17T04:29:05.134 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_support", 2024-09-17T04:29:05.135 INFO:teuthology.orchestra.run.smithi100.stdout: "status", 2024-09-17T04:29:05.135 INFO:teuthology.orchestra.run.smithi100.stdout: "telemetry", 2024-09-17T04:29:05.135 INFO:teuthology.orchestra.run.smithi100.stdout: "volumes" 2024-09-17T04:29:05.135 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.135 INFO:teuthology.orchestra.run.smithi100.stdout: "reef": [ 2024-09-17T04:29:05.135 INFO:teuthology.orchestra.run.smithi100.stdout: "balancer", 2024-09-17T04:29:05.135 INFO:teuthology.orchestra.run.smithi100.stdout: "crash", 2024-09-17T04:29:05.135 INFO:teuthology.orchestra.run.smithi100.stdout: "devicehealth", 2024-09-17T04:29:05.135 INFO:teuthology.orchestra.run.smithi100.stdout: "orchestrator", 2024-09-17T04:29:05.135 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscaler", 2024-09-17T04:29:05.135 INFO:teuthology.orchestra.run.smithi100.stdout: "progress", 2024-09-17T04:29:05.135 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_support", 2024-09-17T04:29:05.135 INFO:teuthology.orchestra.run.smithi100.stdout: "status", 2024-09-17T04:29:05.135 INFO:teuthology.orchestra.run.smithi100.stdout: "telemetry", 2024-09-17T04:29:05.136 INFO:teuthology.orchestra.run.smithi100.stdout: "volumes" 2024-09-17T04:29:05.136 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:29:05.136 INFO:teuthology.orchestra.run.smithi100.stdout: "squid": [ 2024-09-17T04:29:05.136 INFO:teuthology.orchestra.run.smithi100.stdout: "balancer", 2024-09-17T04:29:05.136 INFO:teuthology.orchestra.run.smithi100.stdout: "crash", 2024-09-17T04:29:05.136 INFO:teuthology.orchestra.run.smithi100.stdout: "devicehealth", 2024-09-17T04:29:05.136 INFO:teuthology.orchestra.run.smithi100.stdout: "orchestrator", 2024-09-17T04:29:05.136 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscaler", 2024-09-17T04:29:05.136 INFO:teuthology.orchestra.run.smithi100.stdout: "progress", 2024-09-17T04:29:05.136 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_support", 2024-09-17T04:29:05.136 INFO:teuthology.orchestra.run.smithi100.stdout: "status", 2024-09-17T04:29:05.136 INFO:teuthology.orchestra.run.smithi100.stdout: "telemetry", 2024-09-17T04:29:05.136 INFO:teuthology.orchestra.run.smithi100.stdout: "volumes" 2024-09-17T04:29:05.136 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-17T04:29:05.136 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.137 INFO:teuthology.orchestra.run.smithi100.stdout: "last_failure_osd_epoch": 0, 2024-09-17T04:29:05.137 INFO:teuthology.orchestra.run.smithi100.stdout: "active_clients": [ 2024-09-17T04:29:05.137 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.137 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "devicehealth", 2024-09-17T04:29:05.137 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-17T04:29:05.137 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.137 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-17T04:29:05.137 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.184:0", 2024-09-17T04:29:05.137 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2967819727 2024-09-17T04:29:05.137 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.137 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-17T04:29:05.137 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.137 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.138 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "libcephsqlite", 2024-09-17T04:29:05.138 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-17T04:29:05.138 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.138 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-17T04:29:05.138 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.184:0", 2024-09-17T04:29:05.138 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3031882921 2024-09-17T04:29:05.138 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.138 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-17T04:29:05.138 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.138 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.138 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rbd_support", 2024-09-17T04:29:05.138 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-17T04:29:05.138 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.138 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-17T04:29:05.139 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.184:0", 2024-09-17T04:29:05.139 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4274574405 2024-09-17T04:29:05.139 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.139 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-17T04:29:05.139 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.139 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.139 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "volumes", 2024-09-17T04:29:05.139 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-17T04:29:05.139 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.139 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-17T04:29:05.139 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.184:0", 2024-09-17T04:29:05.139 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1536258390 2024-09-17T04:29:05.139 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.140 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-17T04:29:05.140 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.140 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.140 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "volumes", 2024-09-17T04:29:05.140 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-17T04:29:05.140 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.140 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-17T04:29:05.140 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.184:0", 2024-09-17T04:29:05.140 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2757444911 2024-09-17T04:29:05.140 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.140 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-17T04:29:05.140 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.140 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.140 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "volumes", 2024-09-17T04:29:05.141 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-17T04:29:05.141 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.141 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-17T04:29:05.141 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.184:0", 2024-09-17T04:29:05.141 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 749630281 2024-09-17T04:29:05.141 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.141 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-17T04:29:05.141 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.141 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.141 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "volumes", 2024-09-17T04:29:05.141 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-17T04:29:05.141 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.141 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-17T04:29:05.142 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.184:0", 2024-09-17T04:29:05.142 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2777608743 2024-09-17T04:29:05.142 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.142 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-17T04:29:05.142 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.142 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.142 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "volumes", 2024-09-17T04:29:05.142 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-17T04:29:05.142 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.142 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-17T04:29:05.142 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.184:0", 2024-09-17T04:29:05.142 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2438003331 2024-09-17T04:29:05.142 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.142 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-17T04:29:05.143 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.143 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.143 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "volumes", 2024-09-17T04:29:05.143 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-17T04:29:05.143 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.143 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-17T04:29:05.143 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.184:0", 2024-09-17T04:29:05.143 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2128832114 2024-09-17T04:29:05.143 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.143 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-17T04:29:05.143 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:05.143 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.143 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "volumes", 2024-09-17T04:29:05.144 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-17T04:29:05.144 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:05.144 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-17T04:29:05.144 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.184:0", 2024-09-17T04:29:05.144 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4222181944 2024-09-17T04:29:05.144 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.144 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-17T04:29:05.144 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:05.144 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-17T04:29:05.144 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:29:05.149 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr fail x 2024-09-17T04:29:05.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.381+0000 7fb931200640 1 -- 172.21.15.100:0/204465068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb92c15ce30 msgr2=0x7fb92c15d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:05.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.381+0000 7fb931200640 1 --2- 172.21.15.100:0/204465068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb92c15ce30 0x7fb92c15d240 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0b4fa0 tx=0x7fb92402a800 comp rx=0 tx=0).stop 2024-09-17T04:29:05.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.381+0000 7fb931200640 1 -- 172.21.15.100:0/204465068 shutdown_connections 2024-09-17T04:29:05.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.381+0000 7fb931200640 1 --2- 172.21.15.100:0/204465068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb92c15ce30 0x7fb92c15d240 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:05.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.381+0000 7fb931200640 1 --2- 172.21.15.100:0/204465068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb92c0cfc80 0x7fb92c156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:05.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.381+0000 7fb931200640 1 -- 172.21.15.100:0/204465068 >> 172.21.15.100:0/204465068 conn(0x7fb92c06d3d0 msgr2=0x7fb92c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:05.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.381+0000 7fb931200640 1 -- 172.21.15.100:0/204465068 shutdown_connections 2024-09-17T04:29:05.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.381+0000 7fb931200640 1 -- 172.21.15.100:0/204465068 wait complete. 2024-09-17T04:29:05.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.383+0000 7fb931200640 1 Processor -- start 2024-09-17T04:29:05.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.383+0000 7fb931200640 1 -- start start 2024-09-17T04:29:05.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.385+0000 7fb931200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb92c0705b0 0x7fb92c0747a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:05.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.385+0000 7fb931200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb92c0cfc80 0x7fb92c0c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:05.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.385+0000 7fb931200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb92c15ce30 0x7fb92c0ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:05.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.385+0000 7fb931200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c0cd050 con 0x7fb92c0cfc80 2024-09-17T04:29:05.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.385+0000 7fb931200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c0cced0 con 0x7fb92c0705b0 2024-09-17T04:29:05.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.385+0000 7fb931200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb92c0cd1d0 con 0x7fb92c15ce30 2024-09-17T04:29:05.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.385+0000 7fb921400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb92c0cfc80 0x7fb92c0c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:05.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.385+0000 7fb921400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb92c0cfc80 0x7fb92c0c8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45062/0 (socket says 172.21.15.100:45062) 2024-09-17T04:29:05.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.385+0000 7fb921400640 1 -- 172.21.15.100:0/3502000780 learned_addr learned my addr 172.21.15.100:0/3502000780 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:05.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.385+0000 7fb92a800640 1 --2- 172.21.15.100:0/3502000780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb92c15ce30 0x7fb92c0ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:05.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.385+0000 7fb929e00640 1 --2- 172.21.15.100:0/3502000780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb92c0705b0 0x7fb92c0747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:05.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.386+0000 7fb921400640 1 -- 172.21.15.100:0/3502000780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb92c15ce30 msgr2=0x7fb92c0ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:05.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.386+0000 7fb921400640 1 --2- 172.21.15.100:0/3502000780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb92c15ce30 0x7fb92c0ae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:05.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.386+0000 7fb921400640 1 -- 172.21.15.100:0/3502000780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb92c0705b0 msgr2=0x7fb92c0747a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:05.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.386+0000 7fb921400640 1 --2- 172.21.15.100:0/3502000780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb92c0705b0 0x7fb92c0747a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:05.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.386+0000 7fb921400640 1 -- 172.21.15.100:0/3502000780 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92c072fc0 con 0x7fb92c0cfc80 2024-09-17T04:29:05.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.386+0000 7fb921400640 1 --2- 172.21.15.100:0/3502000780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb92c0cfc80 0x7fb92c0c8c70 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb91400c460 tx=0x7fb91400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:05.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.386+0000 7fb922a00640 1 -- 172.21.15.100:0/3502000780 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914016020 con 0x7fb92c0cfc80 2024-09-17T04:29:05.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.386+0000 7fb922a00640 1 -- 172.21.15.100:0/3502000780 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb914012940 con 0x7fb92c0cfc80 2024-09-17T04:29:05.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.386+0000 7fb922a00640 1 -- 172.21.15.100:0/3502000780 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914012d90 con 0x7fb92c0cfc80 2024-09-17T04:29:05.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.386+0000 7fb931200640 1 -- 172.21.15.100:0/3502000780 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92c0732a0 con 0x7fb92c0cfc80 2024-09-17T04:29:05.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.386+0000 7fb931200640 1 -- 172.21.15.100:0/3502000780 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb92c073860 con 0x7fb92c0cfc80 2024-09-17T04:29:05.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.388+0000 7fb931200640 1 -- 172.21.15.100:0/3502000780 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb92c0cd6f0 con 0x7fb92c0cfc80 2024-09-17T04:29:05.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.389+0000 7fb922a00640 1 -- 172.21.15.100:0/3502000780 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 12) ==== 101798+0+0 (secure 0 0 0) 0x7fb914019020 con 0x7fb92c0cfc80 2024-09-17T04:29:05.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.390+0000 7fb922a00640 1 --2- 172.21.15.100:0/3502000780 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb908078f80 0x7fb90807b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:05.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.391+0000 7fb922a00640 1 -- 172.21.15.100:0/3502000780 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5767+0+0 (secure 0 0 0) 0x7fb91409b460 con 0x7fb92c0cfc80 2024-09-17T04:29:05.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.391+0000 7fb929e00640 1 --2- 172.21.15.100:0/3502000780 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb908078f80 0x7fb90807b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:05.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.392+0000 7fb929e00640 1 --2- 172.21.15.100:0/3502000780 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb908078f80 0x7fb90807b490 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb92c09e510 tx=0x7fb91c00b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:05.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.395+0000 7fb922a00640 1 -- 172.21.15.100:0/3502000780 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb914020020 con 0x7fb92c0cfc80 2024-09-17T04:29:05.457 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:29:05.462 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:29:05.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.634+0000 7fb931200640 1 -- 172.21.15.100:0/3502000780 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr fail", "who": "x"} v 0) -- 0x7fb92c074580 con 0x7fb92c0cfc80 2024-09-17T04:29:05.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.638+0000 7fb922a00640 1 -- 172.21.15.100:0/3502000780 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mgr fail", "who": "x"}]=0 v13) ==== 68+0+0 (secure 0 0 0) 0x7fb914062ae0 con 0x7fb92c0cfc80 2024-09-17T04:29:05.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.641+0000 7fb922a00640 1 -- 172.21.15.100:0/3502000780 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mgrmap(e 13) ==== 50742+0+0 (secure 0 0 0) 0x7fb91405ed50 con 0x7fb92c0cfc80 2024-09-17T04:29:05.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.641+0000 7fb922a00640 1 -- 172.21.15.100:0/3502000780 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb908078f80 msgr2=0x7fb90807b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:05.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.641+0000 7fb922a00640 1 --2- 172.21.15.100:0/3502000780 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb908078f80 0x7fb90807b490 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb92c09e510 tx=0x7fb91c00b2e0 comp rx=0 tx=0).stop 2024-09-17T04:29:05.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.641+0000 7fb931200640 1 -- 172.21.15.100:0/3502000780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb92c0cfc80 msgr2=0x7fb92c0c8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:05.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.641+0000 7fb931200640 1 --2- 172.21.15.100:0/3502000780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb92c0cfc80 0x7fb92c0c8c70 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb91400c460 tx=0x7fb91400c920 comp rx=0 tx=0).stop 2024-09-17T04:29:05.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.641+0000 7fb931200640 1 -- 172.21.15.100:0/3502000780 shutdown_connections 2024-09-17T04:29:05.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.641+0000 7fb931200640 1 --2- 172.21.15.100:0/3502000780 >> [v2:172.21.15.184:6800/4096933233,v1:172.21.15.184:6801/4096933233] conn(0x7fb908078f80 0x7fb90807b490 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:05.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.641+0000 7fb931200640 1 --2- 172.21.15.100:0/3502000780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb92c15ce30 0x7fb92c0ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:05.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.641+0000 7fb931200640 1 --2- 172.21.15.100:0/3502000780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb92c0cfc80 0x7fb92c0c8c70 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:05.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.641+0000 7fb931200640 1 --2- 172.21.15.100:0/3502000780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb92c0705b0 0x7fb92c0747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:05.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.641+0000 7fb931200640 1 -- 172.21.15.100:0/3502000780 >> 172.21.15.100:0/3502000780 conn(0x7fb92c06d3d0 msgr2=0x7fb92c15dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:05.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.642+0000 7fb931200640 1 -- 172.21.15.100:0/3502000780 shutdown_connections 2024-09-17T04:29:05.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.642+0000 7fb931200640 1 -- 172.21.15.100:0/3502000780 wait complete. 2024-09-17T04:29:05.646 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-17T04:29:05.645+0000 7fa1eb000640 -1 mgr handle_mgr_map I was active but no longer am 2024-09-17T04:29:05.676 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:29:05.770 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:29:05.770+0000 7f30b4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:29:05.770 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:29:05.770+0000 7f30b4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:29:05.770 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:29:05.770+0000 7f30b4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:29:05.770 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:29:05.770+0000 7f30b4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:29:05.770 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:29:05.770+0000 7f30b4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:29:05.771 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:29:05.771+0000 7f30ba800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:29:05.771 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:29:05.771+0000 7f30ba800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:29:05.771 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:29:05.771+0000 7f30ba800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:29:05.771 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:29:05.771+0000 7f30ba800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:29:05.771 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:29:05.771+0000 7f30ba800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:29:05.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.928+0000 7f3227a00640 1 -- 172.21.15.100:0/375802934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32200aebc0 msgr2=0x7f3220152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:05.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.928+0000 7f3227a00640 1 --2- 172.21.15.100:0/375802934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32200aebc0 0x7f3220152920 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f3210005fd0 tx=0x7f321002a540 comp rx=0 tx=0).stop 2024-09-17T04:29:05.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.928+0000 7f3227a00640 1 -- 172.21.15.100:0/375802934 shutdown_connections 2024-09-17T04:29:05.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.928+0000 7f3227a00640 1 --2- 172.21.15.100:0/375802934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32200aebc0 0x7f3220152920 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:05.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.928+0000 7f3227a00640 1 -- 172.21.15.100:0/375802934 >> 172.21.15.100:0/375802934 conn(0x7f322006d730 msgr2=0x7f32200b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:05.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.928+0000 7f3227a00640 1 -- 172.21.15.100:0/375802934 shutdown_connections 2024-09-17T04:29:05.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.928+0000 7f3227a00640 1 -- 172.21.15.100:0/375802934 wait complete. 2024-09-17T04:29:05.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.930+0000 7f3227a00640 1 Processor -- start 2024-09-17T04:29:05.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.931+0000 7f3227a00640 1 -- start start 2024-09-17T04:29:05.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.932+0000 7f3227a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32200aebc0 0x7f32200cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:05.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.932+0000 7f3227a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32200b2750 0x7f32201f2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:05.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.932+0000 7f3227a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3220158c20 0x7f32201f6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:05.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.932+0000 7f3227a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f32200b4360 con 0x7f32200b2750 2024-09-17T04:29:05.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.932+0000 7f3227a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f32200b41e0 con 0x7f3220158c20 2024-09-17T04:29:05.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.932+0000 7f3227a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f32200b44e0 con 0x7f32200aebc0 2024-09-17T04:29:05.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.932+0000 7f3217e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32200b2750 0x7f32201f2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:05.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.932+0000 7f3225000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32200aebc0 0x7f32200cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:05.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.932+0000 7f3217e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32200b2750 0x7f32201f2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45194/0 (socket says 172.21.15.100:45194) 2024-09-17T04:29:05.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.932+0000 7f3217e00640 1 -- 172.21.15.100:0/2217478039 learned_addr learned my addr 172.21.15.100:0/2217478039 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:05.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.932+0000 7f3225a00640 1 --2- 172.21.15.100:0/2217478039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3220158c20 0x7f32201f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:05.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.933+0000 7f3217e00640 1 -- 172.21.15.100:0/2217478039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32200aebc0 msgr2=0x7f32200cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:05.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.933+0000 7f3217e00640 1 --2- 172.21.15.100:0/2217478039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32200aebc0 0x7f32200cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:05.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.933+0000 7f3217e00640 1 -- 172.21.15.100:0/2217478039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3220158c20 msgr2=0x7f32201f6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:05.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.933+0000 7f3217e00640 1 --2- 172.21.15.100:0/2217478039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3220158c20 0x7f32201f6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:05.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.933+0000 7f3217e00640 1 -- 172.21.15.100:0/2217478039 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3220157b80 con 0x7f32200b2750 2024-09-17T04:29:05.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.933+0000 7f3225000640 1 --2- 172.21.15.100:0/2217478039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32200aebc0 0x7f32200cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:29:05.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.933+0000 7f3217e00640 1 --2- 172.21.15.100:0/2217478039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32200b2750 0x7f32201f2b70 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f320800c3a0 tx=0x7f320800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:05.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.933+0000 7f321d600640 1 -- 172.21.15.100:0/2217478039 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208017070 con 0x7f32200b2750 2024-09-17T04:29:05.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.933+0000 7f321d600640 1 -- 172.21.15.100:0/2217478039 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32080053c0 con 0x7f32200b2750 2024-09-17T04:29:05.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.933+0000 7f321d600640 1 -- 172.21.15.100:0/2217478039 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208003850 con 0x7f32200b2750 2024-09-17T04:29:05.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.933+0000 7f3227a00640 1 -- 172.21.15.100:0/2217478039 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3220157e60 con 0x7f32200b2750 2024-09-17T04:29:05.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.934+0000 7f3227a00640 1 -- 172.21.15.100:0/2217478039 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3220158420 con 0x7f32200b2750 2024-09-17T04:29:05.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.934+0000 7f3227a00640 1 -- 172.21.15.100:0/2217478039 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32201578f0 con 0x7f32200b2750 2024-09-17T04:29:05.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.935+0000 7f321d600640 1 -- 172.21.15.100:0/2217478039 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) ==== 50742+0+0 (secure 0 0 0) 0x7f320801a070 con 0x7f32200b2750 2024-09-17T04:29:05.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.935+0000 7f321d600640 1 -- 172.21.15.100:0/2217478039 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6326+0+0 (secure 0 0 0) 0x7f3208052980 con 0x7f32200b2750 2024-09-17T04:29:05.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:05.940+0000 7f321d600640 1 -- 172.21.15.100:0/2217478039 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f32080102e0 con 0x7f32200b2750 2024-09-17T04:29:06.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.157+0000 7f3227a00640 1 -- 172.21.15.100:0/2217478039 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32200b4990 con 0x7f32200b2750 2024-09-17T04:29:06.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.158+0000 7f321d600640 1 -- 172.21.15.100:0/2217478039 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5237 (secure 0 0 0) 0x7f32080090a0 con 0x7f32200b2750 2024-09-17T04:29:06.157 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:06.158 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":48,"btime":"2024-09-17T04:28:59:346395+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5513,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2569013403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2569013403},{"type":"v1","addr":"172.21.15.184:6835","nonce":2569013403}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5519,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3475403612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3475403612},{"type":"v1","addr":"172.21.15.184:6837","nonce":3475403612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6156,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2177833369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2177833369},{"type":"v1","addr":"172.21.15.100:6833","nonce":2177833369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:28:43.648961+0000","modified":"2024-09-17T04:28:46.665573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6162":{"gid":6162,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6835/2913442826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2913442826},{"type":"v1","addr":"172.21.15.100:6835","nonce":2913442826}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6162,"qdb_cluster":[6162]},"id":5}]} 2024-09-17T04:29:06.158 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 48 2024-09-17T04:29:06.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.160+0000 7f3227a00640 1 -- 172.21.15.100:0/2217478039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32200b2750 msgr2=0x7f32201f2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:06.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.160+0000 7f3227a00640 1 --2- 172.21.15.100:0/2217478039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32200b2750 0x7f32201f2b70 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f320800c3a0 tx=0x7f320800c860 comp rx=0 tx=0).stop 2024-09-17T04:29:06.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.160+0000 7f3227a00640 1 -- 172.21.15.100:0/2217478039 shutdown_connections 2024-09-17T04:29:06.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.160+0000 7f3227a00640 1 --2- 172.21.15.100:0/2217478039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3220158c20 0x7f32201f6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:06.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.160+0000 7f3227a00640 1 --2- 172.21.15.100:0/2217478039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32200b2750 0x7f32201f2b70 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:06.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.160+0000 7f3227a00640 1 --2- 172.21.15.100:0/2217478039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32200aebc0 0x7f32200cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:06.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.160+0000 7f3227a00640 1 -- 172.21.15.100:0/2217478039 >> 172.21.15.100:0/2217478039 conn(0x7f322006d730 msgr2=0x7f322015f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:06.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.161+0000 7f3227a00640 1 -- 172.21.15.100:0/2217478039 shutdown_connections 2024-09-17T04:29:06.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.161+0000 7f3227a00640 1 -- 172.21.15.100:0/2217478039 wait complete. 2024-09-17T04:29:06.182 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-17T04:29:06.425 INFO:teuthology.orchestra.run.smithi100.stdout:[{"id":5702,"entity":{"name":{"type":"client","num":5702},"addr":{"type":"any","addr":"172.21.15.100:0","nonce":3456036993}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.63298302399999995,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9971025129473223,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5702 172.21.15.100:0/3456036993","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"ce1b342bf8c6349a8c90cd607f83bff2443f21cf","ceph_version":"ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)","entity_id":"y","hostname":"smithi100","pid":"36030","root":"/"}},{"id":6241,"entity":{"name":{"type":"client","num":6241},"addr":{"type":"any","addr":"172.21.15.100:0","nonce":2521528644}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.63398299700000005,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9971071271146543,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6241 172.21.15.100:0/2521528644","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"ce1b342bf8c6349a8c90cd607f83bff2443f21cf","ceph_version":"ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)","entity_id":"y","hostname":"smithi100","pid":"36030","root":"/"}}] 2024-09-17T04:29:06.426 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi100", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "36030", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.100:0", "nonce": 3456036993, "type": "any" }, "name": { "num": 5702, "type": "client" } }, "id": 5702, "inst": "client.5702 172.21.15.100:0/3456036993", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9971025129473223 }, "state": "open", "uptime": 0.632983024 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi100", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "36030", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.100:0", "nonce": 2521528644, "type": "any" }, "name": { "num": 6241, "type": "client" } }, "id": 6241, "inst": "client.6241 172.21.15.100:0/2521528644", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9971071271146543 }, "state": "open", "uptime": 0.633982997 } ] 2024-09-17T04:29:06.426 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T04:29:06.426 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T04:29:06.426 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:29:06.514 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:29:06.609 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T04:29:06.609 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:29:06.609 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:29:06.609 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:29:06.647 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:29:06.647 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:29:06.647 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:29:06.647 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:29:06.684 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:29:06.684 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:29:06.684 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:29:06.684 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:29:06.723 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:29:06.723 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:29:06.723 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:29:06.723 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:29:06.760 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:29:06.760 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:29:06.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.984+0000 7f3dcec00640 1 -- 172.21.15.100:0/815644040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dc80cd200 msgr2=0x7f3dc815e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:06.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.984+0000 7f3dcec00640 1 --2- 172.21.15.100:0/815644040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dc80cd200 0x7f3dc815e2b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8073f30 tx=0x7f3dc002aac0 comp rx=0 tx=0).stop 2024-09-17T04:29:06.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.985+0000 7f3dcec00640 1 -- 172.21.15.100:0/815644040 shutdown_connections 2024-09-17T04:29:06.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.985+0000 7f3dcec00640 1 --2- 172.21.15.100:0/815644040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dc80cd200 0x7f3dc815e2b0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:06.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.985+0000 7f3dcec00640 1 --2- 172.21.15.100:0/815644040 >> v2:172.21.15.184:3300/0 conn(0x7f3dc80d01b0 0x7f3dc8156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:06.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.985+0000 7f3dcec00640 1 -- 172.21.15.100:0/815644040 >> 172.21.15.100:0/815644040 conn(0x7f3dc806d400 msgr2=0x7f3dc80710f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:06.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.985+0000 7f3dcec00640 1 -- 172.21.15.100:0/815644040 shutdown_connections 2024-09-17T04:29:06.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.985+0000 7f3dcec00640 1 -- 172.21.15.100:0/815644040 wait complete. 2024-09-17T04:29:06.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.987+0000 7f3dcec00640 1 Processor -- start 2024-09-17T04:29:06.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.987+0000 7f3dcec00640 1 -- start start 2024-09-17T04:29:06.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.988+0000 7f3dcec00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3dc80d01b0 0x7f3dc814d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:06.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.988+0000 7f3dcec00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dc8160490 0x7f3dc814bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:06.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.989+0000 7f3dcec00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3dc814d8d0 0x7f3dc814c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:06.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.989+0000 7f3dcec00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc815ecf0 con 0x7f3dc8160490 2024-09-17T04:29:06.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.989+0000 7f3dcec00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc815eb70 con 0x7f3dc80d01b0 2024-09-17T04:29:06.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.989+0000 7f3dcec00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3dc815ee70 con 0x7f3dc814d8d0 2024-09-17T04:29:06.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.989+0000 7f3dcce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dc8160490 0x7f3dc814bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:06.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.989+0000 7f3dcce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dc8160490 0x7f3dc814bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45210/0 (socket says 172.21.15.100:45210) 2024-09-17T04:29:06.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.989+0000 7f3dcce00640 1 -- 172.21.15.100:0/2805856485 learned_addr learned my addr 172.21.15.100:0/2805856485 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:06.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.989+0000 7f3dcd800640 1 --2- 172.21.15.100:0/2805856485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3dc80d01b0 0x7f3dc814d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:06.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.989+0000 7f3dce200640 1 --2- 172.21.15.100:0/2805856485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3dc814d8d0 0x7f3dc814c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:06.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.989+0000 7f3dcce00640 1 -- 172.21.15.100:0/2805856485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3dc814d8d0 msgr2=0x7f3dc814c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:06.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.989+0000 7f3dcce00640 1 --2- 172.21.15.100:0/2805856485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3dc814d8d0 0x7f3dc814c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:06.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.989+0000 7f3dcce00640 1 -- 172.21.15.100:0/2805856485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3dc80d01b0 msgr2=0x7f3dc814d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:06.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.989+0000 7f3dcce00640 1 --2- 172.21.15.100:0/2805856485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3dc80d01b0 0x7f3dc814d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:06.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.989+0000 7f3dcce00640 1 -- 172.21.15.100:0/2805856485 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc806e8c0 con 0x7f3dc8160490 2024-09-17T04:29:06.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.990+0000 7f3dcce00640 1 --2- 172.21.15.100:0/2805856485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dc8160490 0x7f3dc814bbc0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3db400c3a0 tx=0x7f3db400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:06.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.990+0000 7f3dc5600640 1 -- 172.21.15.100:0/2805856485 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db4017070 con 0x7f3dc8160490 2024-09-17T04:29:06.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.990+0000 7f3dc5600640 1 -- 172.21.15.100:0/2805856485 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3db40053c0 con 0x7f3dc8160490 2024-09-17T04:29:06.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.990+0000 7f3dcec00640 1 -- 172.21.15.100:0/2805856485 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc806f800 con 0x7f3dc8160490 2024-09-17T04:29:06.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.990+0000 7f3dc5600640 1 -- 172.21.15.100:0/2805856485 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db4003850 con 0x7f3dc8160490 2024-09-17T04:29:06.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.990+0000 7f3dcec00640 1 -- 172.21.15.100:0/2805856485 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc806f320 con 0x7f3dc8160490 2024-09-17T04:29:06.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.991+0000 7f3dcec00640 1 -- 172.21.15.100:0/2805856485 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dc815f320 con 0x7f3dc8160490 2024-09-17T04:29:06.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.991+0000 7f3dc5600640 1 -- 172.21.15.100:0/2805856485 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 14) ==== 50869+0+0 (secure 0 0 0) 0x7f3db401a070 con 0x7f3dc8160490 2024-09-17T04:29:06.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.992+0000 7f3dc5600640 1 --2- 172.21.15.100:0/2805856485 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3da803e670 0x7f3da8040b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:06.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.992+0000 7f3dc5600640 1 -- 172.21.15.100:0/2805856485 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6326+0+0 (secure 0 0 0) 0x7f3db4052a00 con 0x7f3dc8160490 2024-09-17T04:29:06.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.992+0000 7f3dcd800640 1 --2- 172.21.15.100:0/2805856485 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3da803e670 0x7f3da8040b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:06.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.993+0000 7f3dcd800640 1 --2- 172.21.15.100:0/2805856485 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3da803e670 0x7f3da8040b80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009b30 tx=0x7f3dbc005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:06.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:06.997+0000 7f3dc5600640 1 -- 172.21.15.100:0/2805856485 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3db40102e0 con 0x7f3dc8160490 2024-09-17T04:29:07.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.217+0000 7f3dcec00640 1 -- 172.21.15.100:0/2805856485 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3dc806f590 con 0x7f3dc8160490 2024-09-17T04:29:07.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.218+0000 7f3dc5600640 1 -- 172.21.15.100:0/2805856485 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5237 (secure 0 0 0) 0x7f3db40090a0 con 0x7f3dc8160490 2024-09-17T04:29:07.218 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:07.218 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":48,"btime":"2024-09-17T04:28:59:346395+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5513,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2569013403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2569013403},{"type":"v1","addr":"172.21.15.184:6835","nonce":2569013403}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5519,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3475403612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3475403612},{"type":"v1","addr":"172.21.15.184:6837","nonce":3475403612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6156,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2177833369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2177833369},{"type":"v1","addr":"172.21.15.100:6833","nonce":2177833369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:28:43.648961+0000","modified":"2024-09-17T04:28:46.665573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6162":{"gid":6162,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6835/2913442826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2913442826},{"type":"v1","addr":"172.21.15.100:6835","nonce":2913442826}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6162,"qdb_cluster":[6162]},"id":5}]} 2024-09-17T04:29:07.218 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 48 2024-09-17T04:29:07.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.220+0000 7f3dcec00640 1 -- 172.21.15.100:0/2805856485 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3da803e670 msgr2=0x7f3da8040b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:07.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.220+0000 7f3dcec00640 1 --2- 172.21.15.100:0/2805856485 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3da803e670 0x7f3da8040b80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009b30 tx=0x7f3dbc005f90 comp rx=0 tx=0).stop 2024-09-17T04:29:07.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.220+0000 7f3dcec00640 1 -- 172.21.15.100:0/2805856485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dc8160490 msgr2=0x7f3dc814bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:07.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.220+0000 7f3dcec00640 1 --2- 172.21.15.100:0/2805856485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dc8160490 0x7f3dc814bbc0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3db400c3a0 tx=0x7f3db400c860 comp rx=0 tx=0).stop 2024-09-17T04:29:07.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.220+0000 7f3dcec00640 1 -- 172.21.15.100:0/2805856485 shutdown_connections 2024-09-17T04:29:07.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.220+0000 7f3dcec00640 1 --2- 172.21.15.100:0/2805856485 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3da803e670 0x7f3da8040b80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:07.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.220+0000 7f3dcec00640 1 --2- 172.21.15.100:0/2805856485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3dc814d8d0 0x7f3dc814c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:07.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.220+0000 7f3dcec00640 1 --2- 172.21.15.100:0/2805856485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dc8160490 0x7f3dc814bbc0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:07.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.220+0000 7f3dcec00640 1 --2- 172.21.15.100:0/2805856485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3dc80d01b0 0x7f3dc814d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:07.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.220+0000 7f3dcec00640 1 -- 172.21.15.100:0/2805856485 >> 172.21.15.100:0/2805856485 conn(0x7f3dc806d400 msgr2=0x7f3dc80d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:07.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.221+0000 7f3dcec00640 1 -- 172.21.15.100:0/2805856485 shutdown_connections 2024-09-17T04:29:07.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.221+0000 7f3dcec00640 1 -- 172.21.15.100:0/2805856485 wait complete. 2024-09-17T04:29:07.242 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:29:07.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.462+0000 7f3919000640 1 -- 172.21.15.100:0/1807749905 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f391415d760 msgr2=0x7f39141535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:07.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.462+0000 7f3919000640 1 --2- 172.21.15.100:0/1807749905 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f391415d760 0x7f39141535c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3904009860 tx=0x7f390402a510 comp rx=0 tx=0).stop 2024-09-17T04:29:07.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.462+0000 7f3919000640 1 -- 172.21.15.100:0/1807749905 shutdown_connections 2024-09-17T04:29:07.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.462+0000 7f3919000640 1 --2- 172.21.15.100:0/1807749905 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f391415d760 0x7f39141535c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:07.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.462+0000 7f3919000640 1 -- 172.21.15.100:0/1807749905 >> 172.21.15.100:0/1807749905 conn(0x7f391406d730 msgr2=0x7f39140b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:07.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.462+0000 7f3919000640 1 -- 172.21.15.100:0/1807749905 shutdown_connections 2024-09-17T04:29:07.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.462+0000 7f3919000640 1 -- 172.21.15.100:0/1807749905 wait complete. 2024-09-17T04:29:07.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.464+0000 7f3919000640 1 Processor -- start 2024-09-17T04:29:07.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.464+0000 7f3919000640 1 -- start start 2024-09-17T04:29:07.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.466+0000 7f3919000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39140ad0e0 0x7f391414aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:07.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.466+0000 7f3919000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39140b3550 0x7f39141f5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:07.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.466+0000 7f3919000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f391415d760 0x7f391415b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:07.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.466+0000 7f3919000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f39140b4da0 con 0x7f39140ad0e0 2024-09-17T04:29:07.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.466+0000 7f3919000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f39140b4c20 con 0x7f39140b3550 2024-09-17T04:29:07.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.466+0000 7f3919000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f39140b4070 con 0x7f391415d760 2024-09-17T04:29:07.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.466+0000 7f3911e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39140ad0e0 0x7f391414aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:07.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.466+0000 7f3911e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39140ad0e0 0x7f391414aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45212/0 (socket says 172.21.15.100:45212) 2024-09-17T04:29:07.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.466+0000 7f3911e00640 1 -- 172.21.15.100:0/2418771940 learned_addr learned my addr 172.21.15.100:0/2418771940 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:07.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.466+0000 7f3911400640 1 --2- 172.21.15.100:0/2418771940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39140b3550 0x7f39141f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:07.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.466+0000 7f3912800640 1 --2- 172.21.15.100:0/2418771940 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f391415d760 0x7f391415b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:07.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.466+0000 7f3911e00640 1 -- 172.21.15.100:0/2418771940 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f391415d760 msgr2=0x7f391415b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:07.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.466+0000 7f3911e00640 1 --2- 172.21.15.100:0/2418771940 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f391415d760 0x7f391415b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:07.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.466+0000 7f3911e00640 1 -- 172.21.15.100:0/2418771940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39140b3550 msgr2=0x7f39141f5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:07.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.466+0000 7f3911e00640 1 --2- 172.21.15.100:0/2418771940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39140b3550 0x7f39141f5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:07.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.466+0000 7f3911e00640 1 -- 172.21.15.100:0/2418771940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3914149100 con 0x7f39140ad0e0 2024-09-17T04:29:07.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.466+0000 7f3912800640 1 --2- 172.21.15.100:0/2418771940 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f391415d760 0x7f391415b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:07.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.467+0000 7f3911e00640 1 --2- 172.21.15.100:0/2418771940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39140ad0e0 0x7f391414aff0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3904009980 tx=0x7f3904033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:07.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.467+0000 7f390a000640 1 -- 172.21.15.100:0/2418771940 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3904031b50 con 0x7f39140ad0e0 2024-09-17T04:29:07.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.467+0000 7f390a000640 1 -- 172.21.15.100:0/2418771940 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3904031de0 con 0x7f39140ad0e0 2024-09-17T04:29:07.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.467+0000 7f3919000640 1 -- 172.21.15.100:0/2418771940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f391413ebd0 con 0x7f39140ad0e0 2024-09-17T04:29:07.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.467+0000 7f390a000640 1 -- 172.21.15.100:0/2418771940 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3904033330 con 0x7f39140ad0e0 2024-09-17T04:29:07.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.467+0000 7f3919000640 1 -- 172.21.15.100:0/2418771940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39141496b0 con 0x7f39140ad0e0 2024-09-17T04:29:07.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.468+0000 7f3919000640 1 -- 172.21.15.100:0/2418771940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39141535c0 con 0x7f39140ad0e0 2024-09-17T04:29:07.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.468+0000 7f390a000640 1 -- 172.21.15.100:0/2418771940 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 14) ==== 50869+0+0 (secure 0 0 0) 0x7f39040334d0 con 0x7f39140ad0e0 2024-09-17T04:29:07.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.469+0000 7f390a000640 1 --2- 172.21.15.100:0/2418771940 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f38f403e670 0x7f38f4040b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:07.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.469+0000 7f390a000640 1 -- 172.21.15.100:0/2418771940 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6326+0+0 (secure 0 0 0) 0x7f3904074ac0 con 0x7f39140ad0e0 2024-09-17T04:29:07.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.469+0000 7f3911400640 1 --2- 172.21.15.100:0/2418771940 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f38f403e670 0x7f38f4040b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:07.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.470+0000 7f3911400640 1 --2- 172.21.15.100:0/2418771940 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f38f403e670 0x7f38f4040b80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3900007f30 tx=0x7f3900007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.473+0000 7f390a000640 1 -- 172.21.15.100:0/2418771940 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f390403f030 con 0x7f39140ad0e0 2024-09-17T04:29:07.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.651+0000 7f390a000640 1 -- 172.21.15.100:0/2418771940 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 15) ==== 51077+0+0 (secure 0 0 0) 0x7f3904003970 con 0x7f39140ad0e0 2024-09-17T04:29:07.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.694+0000 7f3919000640 1 -- 172.21.15.100:0/2418771940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39141530e0 con 0x7f39140ad0e0 2024-09-17T04:29:07.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.694+0000 7f390a000640 1 -- 172.21.15.100:0/2418771940 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5237 (secure 0 0 0) 0x7f39040393a0 con 0x7f39140ad0e0 2024-09-17T04:29:07.694 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:07.694 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":48,"btime":"2024-09-17T04:28:59:346395+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5513,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2569013403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2569013403},{"type":"v1","addr":"172.21.15.184:6835","nonce":2569013403}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5519,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3475403612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3475403612},{"type":"v1","addr":"172.21.15.184:6837","nonce":3475403612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6156,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2177833369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2177833369},{"type":"v1","addr":"172.21.15.100:6833","nonce":2177833369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:28:43.648961+0000","modified":"2024-09-17T04:28:46.665573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6162":{"gid":6162,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6835/2913442826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2913442826},{"type":"v1","addr":"172.21.15.100:6835","nonce":2913442826}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6162,"qdb_cluster":[6162]},"id":5}]} 2024-09-17T04:29:07.694 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 48 2024-09-17T04:29:07.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.697+0000 7f3919000640 1 -- 172.21.15.100:0/2418771940 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f38f403e670 msgr2=0x7f38f4040b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:07.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.697+0000 7f3919000640 1 --2- 172.21.15.100:0/2418771940 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f38f403e670 0x7f38f4040b80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3900007f30 tx=0x7f3900007ec0 comp rx=0 tx=0).stop 2024-09-17T04:29:07.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.697+0000 7f3919000640 1 -- 172.21.15.100:0/2418771940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39140ad0e0 msgr2=0x7f391414aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:07.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.697+0000 7f3919000640 1 --2- 172.21.15.100:0/2418771940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39140ad0e0 0x7f391414aff0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3904009980 tx=0x7f3904033ba0 comp rx=0 tx=0).stop 2024-09-17T04:29:07.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.697+0000 7f3919000640 1 -- 172.21.15.100:0/2418771940 shutdown_connections 2024-09-17T04:29:07.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.697+0000 7f3919000640 1 --2- 172.21.15.100:0/2418771940 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f38f403e670 0x7f38f4040b80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:07.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.697+0000 7f3919000640 1 --2- 172.21.15.100:0/2418771940 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f391415d760 0x7f391415b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:07.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.697+0000 7f3919000640 1 --2- 172.21.15.100:0/2418771940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39140b3550 0x7f39141f5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:07.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.697+0000 7f3919000640 1 --2- 172.21.15.100:0/2418771940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39140ad0e0 0x7f391414aff0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:07.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.697+0000 7f3919000640 1 -- 172.21.15.100:0/2418771940 >> 172.21.15.100:0/2418771940 conn(0x7f391406d730 msgr2=0x7f39140b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:07.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.697+0000 7f3919000640 1 -- 172.21.15.100:0/2418771940 shutdown_connections 2024-09-17T04:29:07.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.697+0000 7f3919000640 1 -- 172.21.15.100:0/2418771940 wait complete. 2024-09-17T04:29:07.718 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:29:07.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.944+0000 7f87f5800640 1 -- 172.21.15.100:0/4260478081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f00b3550 msgr2=0x7f87f00b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:07.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.944+0000 7f87f5800640 1 --2- 172.21.15.100:0/4260478081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f00b3550 0x7f87f00b3960 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f87f0075250 tx=0x7f87e802aa30 comp rx=0 tx=0).stop 2024-09-17T04:29:07.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.944+0000 7f87f5800640 1 -- 172.21.15.100:0/4260478081 shutdown_connections 2024-09-17T04:29:07.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.944+0000 7f87f5800640 1 --2- 172.21.15.100:0/4260478081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f00b3550 0x7f87f00b3960 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:07.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.944+0000 7f87f5800640 1 --2- 172.21.15.100:0/4260478081 >> v2:172.21.15.184:3300/0 conn(0x7f87f015d760 0x7f87f01535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:07.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.944+0000 7f87f5800640 1 -- 172.21.15.100:0/4260478081 >> 172.21.15.100:0/4260478081 conn(0x7f87f006d730 msgr2=0x7f87f00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:07.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.944+0000 7f87f5800640 1 -- 172.21.15.100:0/4260478081 shutdown_connections 2024-09-17T04:29:07.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.944+0000 7f87f5800640 1 -- 172.21.15.100:0/4260478081 wait complete. 2024-09-17T04:29:07.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.946+0000 7f87f5800640 1 Processor -- start 2024-09-17T04:29:07.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.946+0000 7f87f5800640 1 -- start start 2024-09-17T04:29:07.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.948+0000 7f87f5800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87f00ad0e0 0x7f87f013f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:07.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.948+0000 7f87f5800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f00b3550 0x7f87f0154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:07.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.948+0000 7f87f5800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87f015d760 0x7f87f01553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:07.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.948+0000 7f87f5800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f87f00b4da0 con 0x7f87f00b3550 2024-09-17T04:29:07.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.948+0000 7f87f5800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f87f00b4c20 con 0x7f87f015d760 2024-09-17T04:29:07.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.948+0000 7f87f5800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f87f00b3ea0 con 0x7f87f00ad0e0 2024-09-17T04:29:07.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.948+0000 7f87e6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f00b3550 0x7f87f0154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:07.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.948+0000 7f87e6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f00b3550 0x7f87f0154e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45250/0 (socket says 172.21.15.100:45250) 2024-09-17T04:29:07.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.948+0000 7f87e6000640 1 -- 172.21.15.100:0/1441323200 learned_addr learned my addr 172.21.15.100:0/1441323200 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:07.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.948+0000 7f87eea00640 1 --2- 172.21.15.100:0/1441323200 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87f00ad0e0 0x7f87f013f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:07.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.948+0000 7f87ef400640 1 --2- 172.21.15.100:0/1441323200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87f015d760 0x7f87f01553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:07.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.948+0000 7f87e6000640 1 -- 172.21.15.100:0/1441323200 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87f00ad0e0 msgr2=0x7f87f013f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:07.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.948+0000 7f87e6000640 1 --2- 172.21.15.100:0/1441323200 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87f00ad0e0 0x7f87f013f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:07.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.948+0000 7f87e6000640 1 -- 172.21.15.100:0/1441323200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87f015d760 msgr2=0x7f87f01553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:07.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.948+0000 7f87e6000640 1 --2- 172.21.15.100:0/1441323200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87f015d760 0x7f87f01553d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:07.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.948+0000 7f87e6000640 1 -- 172.21.15.100:0/1441323200 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f014bc80 con 0x7f87f00b3550 2024-09-17T04:29:07.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.949+0000 7f87e6000640 1 --2- 172.21.15.100:0/1441323200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f00b3550 0x7f87f0154e90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f87d800c370 tx=0x7f87d800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:07.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.949+0000 7f87e7400640 1 -- 172.21.15.100:0/1441323200 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d8017070 con 0x7f87f00b3550 2024-09-17T04:29:07.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.949+0000 7f87e7400640 1 -- 172.21.15.100:0/1441323200 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87d80053c0 con 0x7f87f00b3550 2024-09-17T04:29:07.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.949+0000 7f87f5800640 1 -- 172.21.15.100:0/1441323200 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f014bf60 con 0x7f87f00b3550 2024-09-17T04:29:07.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.949+0000 7f87e7400640 1 -- 172.21.15.100:0/1441323200 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d8003850 con 0x7f87f00b3550 2024-09-17T04:29:07.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.949+0000 7f87f5800640 1 -- 172.21.15.100:0/1441323200 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f013fb30 con 0x7f87f00b3550 2024-09-17T04:29:07.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.950+0000 7f87f5800640 1 -- 172.21.15.100:0/1441323200 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87f014c1a0 con 0x7f87f00b3550 2024-09-17T04:29:07.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.950+0000 7f87e7400640 1 -- 172.21.15.100:0/1441323200 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 15) ==== 51077+0+0 (secure 0 0 0) 0x7f87d801a070 con 0x7f87f00b3550 2024-09-17T04:29:07.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.951+0000 7f87e7400640 1 --2- 172.21.15.100:0/1441323200 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f87cc03e8b0 0x7f87cc040dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:07.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.951+0000 7f87e7400640 1 -- 172.21.15.100:0/1441323200 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6326+0+0 (secure 0 0 0) 0x7f87d8052f80 con 0x7f87f00b3550 2024-09-17T04:29:07.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.951+0000 7f87eea00640 1 --2- 172.21.15.100:0/1441323200 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f87cc03e8b0 0x7f87cc040dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:07.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.952+0000 7f87eea00640 1 --2- 172.21.15.100:0/1441323200 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f87cc03e8b0 0x7f87cc040dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f87e0009b30 tx=0x7f87e0005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:07.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:07.956+0000 7f87e7400640 1 -- 172.21.15.100:0/1441323200 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f87f014c1a0 con 0x7f87f00b3550 2024-09-17T04:29:08.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.139+0000 7f87f5800640 1 -- 172.21.15.100:0/1441323200 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f87f014ba10 con 0x7f87f00b3550 2024-09-17T04:29:08.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.140+0000 7f87e7400640 1 -- 172.21.15.100:0/1441323200 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+17696 (secure 0 0 0) 0x7f87d8011590 con 0x7f87f00b3550 2024-09-17T04:29:08.140 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:08.140 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":76,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:29:05.634972+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:28:42.112814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-17T04:28:42.620048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.2d","mappings":[{"from":6,"to":0}]},{"pgid":"11.30","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.184:0/659661481":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/2967819727":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/2438003331":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/2128832114":"2024-09-18T04:29:05.634928+0000","172.21.15.184:6800/4096933233":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/749630281":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/3031882921":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/1536258390":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/4274574405":"2024-09-18T04:29:05.634928+0000","172.21.15.184:6801/4096933233":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/4222181944":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/2757444911":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/2777608743":"2024-09-18T04:29:05.634928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:29:08.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.142+0000 7f87f5800640 1 -- 172.21.15.100:0/1441323200 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f87cc03e8b0 msgr2=0x7f87cc040dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:08.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.142+0000 7f87f5800640 1 --2- 172.21.15.100:0/1441323200 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f87cc03e8b0 0x7f87cc040dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f87e0009b30 tx=0x7f87e0005f90 comp rx=0 tx=0).stop 2024-09-17T04:29:08.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.142+0000 7f87f5800640 1 -- 172.21.15.100:0/1441323200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f00b3550 msgr2=0x7f87f0154e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:08.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.142+0000 7f87f5800640 1 --2- 172.21.15.100:0/1441323200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f00b3550 0x7f87f0154e90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f87d800c370 tx=0x7f87d800c830 comp rx=0 tx=0).stop 2024-09-17T04:29:08.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.142+0000 7f87f5800640 1 -- 172.21.15.100:0/1441323200 shutdown_connections 2024-09-17T04:29:08.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.142+0000 7f87f5800640 1 --2- 172.21.15.100:0/1441323200 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f87cc03e8b0 0x7f87cc040dc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:08.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.142+0000 7f87f5800640 1 --2- 172.21.15.100:0/1441323200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87f015d760 0x7f87f01553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:08.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.142+0000 7f87f5800640 1 --2- 172.21.15.100:0/1441323200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f00b3550 0x7f87f0154e90 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:08.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.142+0000 7f87f5800640 1 --2- 172.21.15.100:0/1441323200 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87f00ad0e0 0x7f87f013f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:08.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.142+0000 7f87f5800640 1 -- 172.21.15.100:0/1441323200 >> 172.21.15.100:0/1441323200 conn(0x7f87f006d730 msgr2=0x7f87f0068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:08.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.142+0000 7f87f5800640 1 -- 172.21.15.100:0/1441323200 shutdown_connections 2024-09-17T04:29:08.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.143+0000 7f87f5800640 1 -- 172.21.15.100:0/1441323200 wait complete. 2024-09-17T04:29:08.163 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T04:29:08.163 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:29:08.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.386+0000 7fd855000640 1 -- 172.21.15.100:0/974149659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8500b9e00 msgr2=0x7fd850162a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:08.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.386+0000 7fd855000640 1 --2- 172.21.15.100:0/974149659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8500b9e00 0x7fd850162a50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd8500d2050 tx=0x7fd84002a780 comp rx=0 tx=0).stop 2024-09-17T04:29:08.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.386+0000 7fd855000640 1 -- 172.21.15.100:0/974149659 shutdown_connections 2024-09-17T04:29:08.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.386+0000 7fd855000640 1 --2- 172.21.15.100:0/974149659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8500d1580 0x7fd8500d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:08.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.386+0000 7fd855000640 1 --2- 172.21.15.100:0/974149659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8500b9e00 0x7fd850162a50 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:08.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.386+0000 7fd855000640 1 -- 172.21.15.100:0/974149659 >> 172.21.15.100:0/974149659 conn(0x7fd85008bfe0 msgr2=0x7fd85009d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:08.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.386+0000 7fd855000640 1 -- 172.21.15.100:0/974149659 shutdown_connections 2024-09-17T04:29:08.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.386+0000 7fd855000640 1 -- 172.21.15.100:0/974149659 wait complete. 2024-09-17T04:29:08.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.388+0000 7fd855000640 1 Processor -- start 2024-09-17T04:29:08.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.389+0000 7fd855000640 1 -- start start 2024-09-17T04:29:08.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.390+0000 7fd855000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8500b3160 0x7fd8500a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:08.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.390+0000 7fd855000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8500b9e00 0x7fd85009efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:08.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.390+0000 7fd855000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8500d1580 0x7fd8501f28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:08.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.390+0000 7fd855000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd8500d44b0 con 0x7fd8500b3160 2024-09-17T04:29:08.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.390+0000 7fd855000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd8500d4330 con 0x7fd8500b9e00 2024-09-17T04:29:08.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.390+0000 7fd855000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd8500d4630 con 0x7fd8500d1580 2024-09-17T04:29:08.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.390+0000 7fd84f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8500b3160 0x7fd8500a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:08.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.390+0000 7fd84f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8500b3160 0x7fd8500a00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45260/0 (socket says 172.21.15.100:45260) 2024-09-17T04:29:08.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.390+0000 7fd84f400640 1 -- 172.21.15.100:0/1698653293 learned_addr learned my addr 172.21.15.100:0/1698653293 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:08.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.390+0000 7fd84ea00640 1 --2- 172.21.15.100:0/1698653293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8500b9e00 0x7fd85009efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:08.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.390+0000 7fd84fe00640 1 --2- 172.21.15.100:0/1698653293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8500d1580 0x7fd8501f28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:08.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.391+0000 7fd84f400640 1 -- 172.21.15.100:0/1698653293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8500d1580 msgr2=0x7fd8501f28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:08.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.391+0000 7fd84f400640 1 --2- 172.21.15.100:0/1698653293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8500d1580 0x7fd8501f28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:08.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.391+0000 7fd84f400640 1 -- 172.21.15.100:0/1698653293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8500b9e00 msgr2=0x7fd85009efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:08.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.391+0000 7fd84f400640 1 --2- 172.21.15.100:0/1698653293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8500b9e00 0x7fd85009efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:08.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.391+0000 7fd84f400640 1 -- 172.21.15.100:0/1698653293 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd85009e200 con 0x7fd8500b3160 2024-09-17T04:29:08.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.391+0000 7fd84f400640 1 --2- 172.21.15.100:0/1698653293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8500b3160 0x7fd8500a00a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fd840009ca0 tx=0x7fd840032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:08.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.391+0000 7fd847400640 1 -- 172.21.15.100:0/1698653293 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840041020 con 0x7fd8500b3160 2024-09-17T04:29:08.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.391+0000 7fd847400640 1 -- 172.21.15.100:0/1698653293 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd840032e50 con 0x7fd8500b3160 2024-09-17T04:29:08.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.391+0000 7fd847400640 1 -- 172.21.15.100:0/1698653293 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840036040 con 0x7fd8500b3160 2024-09-17T04:29:08.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.391+0000 7fd855000640 1 -- 172.21.15.100:0/1698653293 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd85009e4e0 con 0x7fd8500b3160 2024-09-17T04:29:08.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.391+0000 7fd855000640 1 -- 172.21.15.100:0/1698653293 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd85009eaa0 con 0x7fd8500b3160 2024-09-17T04:29:08.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.392+0000 7fd855000640 1 -- 172.21.15.100:0/1698653293 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8500d4b50 con 0x7fd8500b3160 2024-09-17T04:29:08.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.393+0000 7fd847400640 1 -- 172.21.15.100:0/1698653293 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 15) ==== 51077+0+0 (secure 0 0 0) 0x7fd840044020 con 0x7fd8500b3160 2024-09-17T04:29:08.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.393+0000 7fd847400640 1 --2- 172.21.15.100:0/1698653293 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd83003e8b0 0x7fd830040dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:08.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.393+0000 7fd847400640 1 -- 172.21.15.100:0/1698653293 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6326+0+0 (secure 0 0 0) 0x7fd8400748f0 con 0x7fd8500b3160 2024-09-17T04:29:08.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.393+0000 7fd84ea00640 1 --2- 172.21.15.100:0/1698653293 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd83003e8b0 0x7fd830040dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:08.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.394+0000 7fd84ea00640 1 --2- 172.21.15.100:0/1698653293 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd83003e8b0 0x7fd830040dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd85015d510 tx=0x7fd83c001d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:08.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.398+0000 7fd847400640 1 -- 172.21.15.100:0/1698653293 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd84004b070 con 0x7fd8500b3160 2024-09-17T04:29:08.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.616+0000 7fd855000640 1 -- 172.21.15.100:0/1698653293 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8500b57a0 con 0x7fd8500b3160 2024-09-17T04:29:08.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.616+0000 7fd847400640 1 -- 172.21.15.100:0/1698653293 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5237 (secure 0 0 0) 0x7fd84003cd20 con 0x7fd8500b3160 2024-09-17T04:29:08.616 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:08.616 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":48,"btime":"2024-09-17T04:28:59:346395+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5513,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2569013403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2569013403},{"type":"v1","addr":"172.21.15.184:6835","nonce":2569013403}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5519,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3475403612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3475403612},{"type":"v1","addr":"172.21.15.184:6837","nonce":3475403612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6156,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2177833369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2177833369},{"type":"v1","addr":"172.21.15.100:6833","nonce":2177833369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:28:43.648961+0000","modified":"2024-09-17T04:28:46.665573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6162":{"gid":6162,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6835/2913442826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2913442826},{"type":"v1","addr":"172.21.15.100:6835","nonce":2913442826}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6162,"qdb_cluster":[6162]},"id":5}]} 2024-09-17T04:29:08.616 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 48 2024-09-17T04:29:08.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.619+0000 7fd855000640 1 -- 172.21.15.100:0/1698653293 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd83003e8b0 msgr2=0x7fd830040dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:08.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.619+0000 7fd855000640 1 --2- 172.21.15.100:0/1698653293 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd83003e8b0 0x7fd830040dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd85015d510 tx=0x7fd83c001d50 comp rx=0 tx=0).stop 2024-09-17T04:29:08.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.619+0000 7fd855000640 1 -- 172.21.15.100:0/1698653293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8500b3160 msgr2=0x7fd8500a00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:08.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.619+0000 7fd855000640 1 --2- 172.21.15.100:0/1698653293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8500b3160 0x7fd8500a00a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fd840009ca0 tx=0x7fd840032c60 comp rx=0 tx=0).stop 2024-09-17T04:29:08.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.619+0000 7fd855000640 1 -- 172.21.15.100:0/1698653293 shutdown_connections 2024-09-17T04:29:08.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.619+0000 7fd855000640 1 --2- 172.21.15.100:0/1698653293 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd83003e8b0 0x7fd830040dc0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:08.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.619+0000 7fd855000640 1 --2- 172.21.15.100:0/1698653293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8500d1580 0x7fd8501f28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:08.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.619+0000 7fd855000640 1 --2- 172.21.15.100:0/1698653293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8500b9e00 0x7fd85009efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:08.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.619+0000 7fd855000640 1 --2- 172.21.15.100:0/1698653293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8500b3160 0x7fd8500a00a0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:08.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.619+0000 7fd855000640 1 -- 172.21.15.100:0/1698653293 >> 172.21.15.100:0/1698653293 conn(0x7fd85008bfe0 msgr2=0x7fd85006e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:08.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.619+0000 7fd855000640 1 -- 172.21.15.100:0/1698653293 shutdown_connections 2024-09-17T04:29:08.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.619+0000 7fd855000640 1 -- 172.21.15.100:0/1698653293 wait complete. 2024-09-17T04:29:08.643 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:29:08.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.864+0000 7feeb6200640 1 -- 172.21.15.100:0/145172679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feeb00af170 msgr2=0x7feeb00b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:08.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.864+0000 7feeb6200640 1 --2- 172.21.15.100:0/145172679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feeb00af170 0x7feeb00b2f60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fee9c009830 tx=0x7fee9c02a4a0 comp rx=0 tx=0).stop 2024-09-17T04:29:08.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.864+0000 7feeb6200640 1 -- 172.21.15.100:0/145172679 shutdown_connections 2024-09-17T04:29:08.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.864+0000 7feeb6200640 1 --2- 172.21.15.100:0/145172679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feeb00af170 0x7feeb00b2f60 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:08.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.864+0000 7feeb6200640 1 -- 172.21.15.100:0/145172679 >> 172.21.15.100:0/145172679 conn(0x7feeb00b1e40 msgr2=0x7feeb00b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:08.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.864+0000 7feeb6200640 1 -- 172.21.15.100:0/145172679 shutdown_connections 2024-09-17T04:29:08.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.864+0000 7feeb6200640 1 -- 172.21.15.100:0/145172679 wait complete. 2024-09-17T04:29:08.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.866+0000 7feeb6200640 1 Processor -- start 2024-09-17T04:29:08.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.866+0000 7feeb6200640 1 -- start start 2024-09-17T04:29:08.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.868+0000 7feeb6200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feeb00ad9c0 0x7feeb00ce690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:08.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.868+0000 7feeb6200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feeb00af170 0x7feeb0151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:08.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.868+0000 7feeb6200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feeb015a0f0 0x7feeb01f9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:08.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.868+0000 7feeb6200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7feeb006fb20 con 0x7feeb00ad9c0 2024-09-17T04:29:08.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.868+0000 7feeb6200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feeb006f9a0 con 0x7feeb00af170 2024-09-17T04:29:08.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.868+0000 7feeb6200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7feeb015d670 con 0x7feeb015a0f0 2024-09-17T04:29:08.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.868+0000 7feeb4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feeb00ad9c0 0x7feeb00ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:08.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.868+0000 7feeb4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feeb00ad9c0 0x7feeb00ce690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45272/0 (socket says 172.21.15.100:45272) 2024-09-17T04:29:08.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.868+0000 7feeb4e00640 1 -- 172.21.15.100:0/2743213164 learned_addr learned my addr 172.21.15.100:0/2743213164 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:08.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.868+0000 7feeafe00640 1 --2- 172.21.15.100:0/2743213164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feeb00af170 0x7feeb0151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:08.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.868+0000 7feeb5800640 1 --2- 172.21.15.100:0/2743213164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feeb015a0f0 0x7feeb01f9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:08.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.868+0000 7feeb4e00640 1 -- 172.21.15.100:0/2743213164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feeb015a0f0 msgr2=0x7feeb01f9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:08.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.868+0000 7feeb4e00640 1 --2- 172.21.15.100:0/2743213164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feeb015a0f0 0x7feeb01f9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:08.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.868+0000 7feeb4e00640 1 -- 172.21.15.100:0/2743213164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feeb00af170 msgr2=0x7feeb0151d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:08.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.868+0000 7feeb4e00640 1 --2- 172.21.15.100:0/2743213164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feeb00af170 0x7feeb0151d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:08.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.868+0000 7feeb4e00640 1 -- 172.21.15.100:0/2743213164 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb01f9e80 con 0x7feeb00ad9c0 2024-09-17T04:29:08.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.868+0000 7feeb5800640 1 --2- 172.21.15.100:0/2743213164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feeb015a0f0 0x7feeb01f9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:08.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.869+0000 7feeb4e00640 1 --2- 172.21.15.100:0/2743213164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feeb00ad9c0 0x7feeb00ce690 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7feea000c7d0 tx=0x7feea000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:08.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.869+0000 7feeacc00640 1 -- 172.21.15.100:0/2743213164 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea0017070 con 0x7feeb00ad9c0 2024-09-17T04:29:08.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.869+0000 7feeacc00640 1 -- 172.21.15.100:0/2743213164 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feea0005070 con 0x7feeb00ad9c0 2024-09-17T04:29:08.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.869+0000 7feeb6200640 1 -- 172.21.15.100:0/2743213164 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeb01f9ba0 con 0x7feeb00ad9c0 2024-09-17T04:29:08.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.869+0000 7feeacc00640 1 -- 172.21.15.100:0/2743213164 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea00054a0 con 0x7feeb00ad9c0 2024-09-17T04:29:08.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.869+0000 7feeb6200640 1 -- 172.21.15.100:0/2743213164 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeb00cde90 con 0x7feeb00ad9c0 2024-09-17T04:29:08.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.870+0000 7feeacc00640 1 -- 172.21.15.100:0/2743213164 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 15) ==== 51077+0+0 (secure 0 0 0) 0x7feea001a070 con 0x7feeb00ad9c0 2024-09-17T04:29:08.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.870+0000 7feeb6200640 1 -- 172.21.15.100:0/2743213164 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feeb015a740 con 0x7feeb00ad9c0 2024-09-17T04:29:08.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.871+0000 7feeacc00640 1 --2- 172.21.15.100:0/2743213164 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fee9003e8b0 0x7fee90040dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:08.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.871+0000 7feeacc00640 1 -- 172.21.15.100:0/2743213164 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6326+0+0 (secure 0 0 0) 0x7feea0053360 con 0x7feeb00ad9c0 2024-09-17T04:29:08.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.871+0000 7feeafe00640 1 --2- 172.21.15.100:0/2743213164 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fee9003e8b0 0x7fee90040dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:08.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.872+0000 7feeafe00640 1 --2- 172.21.15.100:0/2743213164 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fee9003e8b0 0x7fee90040dc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fee9c034000 tx=0x7fee9c004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:08.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:08.876+0000 7feeacc00640 1 -- 172.21.15.100:0/2743213164 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7feeb015a740 con 0x7feeb00ad9c0 2024-09-17T04:29:09.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.092+0000 7feeb6200640 1 -- 172.21.15.100:0/2743213164 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feeb01563d0 con 0x7feeb00ad9c0 2024-09-17T04:29:09.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.093+0000 7feeacc00640 1 -- 172.21.15.100:0/2743213164 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5237 (secure 0 0 0) 0x7feeb01563d0 con 0x7feeb00ad9c0 2024-09-17T04:29:09.092 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:09.092 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":48,"btime":"2024-09-17T04:28:59:346395+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5513,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2569013403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2569013403},{"type":"v1","addr":"172.21.15.184:6835","nonce":2569013403}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5519,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3475403612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3475403612},{"type":"v1","addr":"172.21.15.184:6837","nonce":3475403612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6156,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2177833369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2177833369},{"type":"v1","addr":"172.21.15.100:6833","nonce":2177833369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:28:43.648961+0000","modified":"2024-09-17T04:28:46.665573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6162":{"gid":6162,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6835/2913442826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2913442826},{"type":"v1","addr":"172.21.15.100:6835","nonce":2913442826}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6162,"qdb_cluster":[6162]},"id":5}]} 2024-09-17T04:29:09.092 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 48 2024-09-17T04:29:09.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.095+0000 7feeb6200640 1 -- 172.21.15.100:0/2743213164 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fee9003e8b0 msgr2=0x7fee90040dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:09.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.095+0000 7feeb6200640 1 --2- 172.21.15.100:0/2743213164 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fee9003e8b0 0x7fee90040dc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fee9c034000 tx=0x7fee9c004730 comp rx=0 tx=0).stop 2024-09-17T04:29:09.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.095+0000 7feeb6200640 1 -- 172.21.15.100:0/2743213164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feeb00ad9c0 msgr2=0x7feeb00ce690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:09.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.095+0000 7feeb6200640 1 --2- 172.21.15.100:0/2743213164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feeb00ad9c0 0x7feeb00ce690 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7feea000c7d0 tx=0x7feea000cc90 comp rx=0 tx=0).stop 2024-09-17T04:29:09.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.095+0000 7feeb6200640 1 -- 172.21.15.100:0/2743213164 shutdown_connections 2024-09-17T04:29:09.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.095+0000 7feeb6200640 1 --2- 172.21.15.100:0/2743213164 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fee9003e8b0 0x7fee90040dc0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:09.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.095+0000 7feeb6200640 1 --2- 172.21.15.100:0/2743213164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feeb015a0f0 0x7feeb01f9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:09.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.095+0000 7feeb6200640 1 --2- 172.21.15.100:0/2743213164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feeb00af170 0x7feeb0151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:09.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.095+0000 7feeb6200640 1 --2- 172.21.15.100:0/2743213164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feeb00ad9c0 0x7feeb00ce690 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:09.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.095+0000 7feeb6200640 1 -- 172.21.15.100:0/2743213164 >> 172.21.15.100:0/2743213164 conn(0x7feeb00b1e40 msgr2=0x7feeb015b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:09.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.095+0000 7feeb6200640 1 -- 172.21.15.100:0/2743213164 shutdown_connections 2024-09-17T04:29:09.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.095+0000 7feeb6200640 1 -- 172.21.15.100:0/2743213164 wait complete. 2024-09-17T04:29:09.120 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:29:09.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.340+0000 7f4922e00640 1 -- 172.21.15.100:0/880415996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f491c15f910 msgr2=0x7f491c0b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:09.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.340+0000 7f4922e00640 1 --2- 172.21.15.100:0/880415996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f491c15f910 0x7f491c0b3750 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f4914005fd0 tx=0x7f491402a810 comp rx=0 tx=0).stop 2024-09-17T04:29:09.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.340+0000 7f4922e00640 1 -- 172.21.15.100:0/880415996 shutdown_connections 2024-09-17T04:29:09.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.340+0000 7f4922e00640 1 --2- 172.21.15.100:0/880415996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f491c15f910 0x7f491c0b3750 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:09.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.340+0000 7f4922e00640 1 --2- 172.21.15.100:0/880415996 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f491c152920 0x7f491c15f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:09.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.340+0000 7f4922e00640 1 -- 172.21.15.100:0/880415996 >> 172.21.15.100:0/880415996 conn(0x7f491c06d730 msgr2=0x7f491c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:09.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.340+0000 7f4922e00640 1 -- 172.21.15.100:0/880415996 shutdown_connections 2024-09-17T04:29:09.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.340+0000 7f4922e00640 1 -- 172.21.15.100:0/880415996 wait complete. 2024-09-17T04:29:09.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.342+0000 7f4922e00640 1 Processor -- start 2024-09-17T04:29:09.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.343+0000 7f4922e00640 1 -- start start 2024-09-17T04:29:09.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.344+0000 7f4922e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f491c0aebc0 0x7f491c14cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:09.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.344+0000 7f4922e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f491c152920 0x7f491c1f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:09.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.344+0000 7f4922e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f491c15f910 0x7f491c1f70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:09.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.344+0000 7f4922e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f491c0b4190 con 0x7f491c152920 2024-09-17T04:29:09.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.344+0000 7f4922e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f491c0b4010 con 0x7f491c0aebc0 2024-09-17T04:29:09.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.344+0000 7f4922e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f491c0b4310 con 0x7f491c15f910 2024-09-17T04:29:09.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.344+0000 7f491b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f491c152920 0x7f491c1f6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:09.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.344+0000 7f491b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f491c152920 0x7f491c1f6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45282/0 (socket says 172.21.15.100:45282) 2024-09-17T04:29:09.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.344+0000 7f491b400640 1 -- 172.21.15.100:0/3070120353 learned_addr learned my addr 172.21.15.100:0/3070120353 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:09.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.345+0000 7f491be00640 1 --2- 172.21.15.100:0/3070120353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f491c0aebc0 0x7f491c14cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:09.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.345+0000 7f4920e00640 1 --2- 172.21.15.100:0/3070120353 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f491c15f910 0x7f491c1f70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:09.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.345+0000 7f491b400640 1 -- 172.21.15.100:0/3070120353 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f491c15f910 msgr2=0x7f491c1f70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:09.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.345+0000 7f491b400640 1 --2- 172.21.15.100:0/3070120353 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f491c15f910 0x7f491c1f70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:09.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.345+0000 7f491b400640 1 -- 172.21.15.100:0/3070120353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f491c0aebc0 msgr2=0x7f491c14cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:09.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.345+0000 7f491b400640 1 --2- 172.21.15.100:0/3070120353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f491c0aebc0 0x7f491c14cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:09.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.345+0000 7f491b400640 1 -- 172.21.15.100:0/3070120353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f491c14d3e0 con 0x7f491c152920 2024-09-17T04:29:09.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.345+0000 7f491b400640 1 --2- 172.21.15.100:0/3070120353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f491c152920 0x7f491c1f6b90 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f490800d780 tx=0x7f490800dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:09.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.345+0000 7f4913e00640 1 -- 172.21.15.100:0/3070120353 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4908017070 con 0x7f491c152920 2024-09-17T04:29:09.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.345+0000 7f4913e00640 1 -- 172.21.15.100:0/3070120353 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f490800de90 con 0x7f491c152920 2024-09-17T04:29:09.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.345+0000 7f4922e00640 1 -- 172.21.15.100:0/3070120353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f491c1f7a30 con 0x7f491c152920 2024-09-17T04:29:09.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.346+0000 7f4913e00640 1 -- 172.21.15.100:0/3070120353 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49080099d0 con 0x7f491c152920 2024-09-17T04:29:09.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.346+0000 7f4922e00640 1 -- 172.21.15.100:0/3070120353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f491c0cf340 con 0x7f491c152920 2024-09-17T04:29:09.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.347+0000 7f4913e00640 1 -- 172.21.15.100:0/3070120353 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 15) ==== 51077+0+0 (secure 0 0 0) 0x7f4908009b70 con 0x7f491c152920 2024-09-17T04:29:09.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.347+0000 7f4922e00640 1 -- 172.21.15.100:0/3070120353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f491c1f7780 con 0x7f491c152920 2024-09-17T04:29:09.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.347+0000 7f4913e00640 1 --2- 172.21.15.100:0/3070120353 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f48fc03e8b0 0x7f48fc040dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:09.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.347+0000 7f4913e00640 1 -- 172.21.15.100:0/3070120353 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6326+0+0 (secure 0 0 0) 0x7f4908054080 con 0x7f491c152920 2024-09-17T04:29:09.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.348+0000 7f491be00640 1 --2- 172.21.15.100:0/3070120353 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f48fc03e8b0 0x7f48fc040dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:09.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.348+0000 7f491be00640 1 --2- 172.21.15.100:0/3070120353 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f48fc03e8b0 0x7f48fc040dc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f490c000f60 tx=0x7f490c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:09.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.353+0000 7f4913e00640 1 -- 172.21.15.100:0/3070120353 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4908020020 con 0x7f491c152920 2024-09-17T04:29:09.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.537+0000 7f4922e00640 1 -- 172.21.15.100:0/3070120353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f491c0b48a0 con 0x7f491c152920 2024-09-17T04:29:09.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.539+0000 7f4913e00640 1 -- 172.21.15.100:0/3070120353 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+17696 (secure 0 0 0) 0x7f4908016170 con 0x7f491c152920 2024-09-17T04:29:09.539 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:09.539 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":76,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:29:05.634972+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:28:42.112814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-17T04:28:42.620048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.2d","mappings":[{"from":6,"to":0}]},{"pgid":"11.30","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.184:0/659661481":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/2967819727":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/2438003331":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/2128832114":"2024-09-18T04:29:05.634928+0000","172.21.15.184:6800/4096933233":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/749630281":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/3031882921":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/1536258390":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/4274574405":"2024-09-18T04:29:05.634928+0000","172.21.15.184:6801/4096933233":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/4222181944":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/2757444911":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/2777608743":"2024-09-18T04:29:05.634928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:29:09.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.541+0000 7f4922e00640 1 -- 172.21.15.100:0/3070120353 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f48fc03e8b0 msgr2=0x7f48fc040dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:09.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.541+0000 7f4922e00640 1 --2- 172.21.15.100:0/3070120353 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f48fc03e8b0 0x7f48fc040dc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f490c000f60 tx=0x7f490c009040 comp rx=0 tx=0).stop 2024-09-17T04:29:09.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.541+0000 7f4922e00640 1 -- 172.21.15.100:0/3070120353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f491c152920 msgr2=0x7f491c1f6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:09.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.541+0000 7f4922e00640 1 --2- 172.21.15.100:0/3070120353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f491c152920 0x7f491c1f6b90 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f490800d780 tx=0x7f490800dc40 comp rx=0 tx=0).stop 2024-09-17T04:29:09.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.541+0000 7f4922e00640 1 -- 172.21.15.100:0/3070120353 shutdown_connections 2024-09-17T04:29:09.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.541+0000 7f4922e00640 1 --2- 172.21.15.100:0/3070120353 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f48fc03e8b0 0x7f48fc040dc0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:09.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.541+0000 7f4922e00640 1 --2- 172.21.15.100:0/3070120353 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f491c15f910 0x7f491c1f70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:09.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.541+0000 7f4922e00640 1 --2- 172.21.15.100:0/3070120353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f491c152920 0x7f491c1f6b90 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:09.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.541+0000 7f4922e00640 1 --2- 172.21.15.100:0/3070120353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f491c0aebc0 0x7f491c14cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:09.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.541+0000 7f4922e00640 1 -- 172.21.15.100:0/3070120353 >> 172.21.15.100:0/3070120353 conn(0x7f491c06d730 msgr2=0x7f491c15ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:09.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.541+0000 7f4922e00640 1 -- 172.21.15.100:0/3070120353 shutdown_connections 2024-09-17T04:29:09.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.541+0000 7f4922e00640 1 -- 172.21.15.100:0/3070120353 wait complete. 2024-09-17T04:29:09.563 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T04:29:09.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.787+0000 7f1452000640 1 -- 172.21.15.100:0/909965558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f144c070cc0 msgr2=0x7f144c156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:09.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.787+0000 7f1452000640 1 --2- 172.21.15.100:0/909965558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f144c070cc0 0x7f144c156e40 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f1434009860 tx=0x7f143402a560 comp rx=0 tx=0).stop 2024-09-17T04:29:09.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.788+0000 7f1452000640 1 -- 172.21.15.100:0/909965558 shutdown_connections 2024-09-17T04:29:09.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.788+0000 7f1452000640 1 --2- 172.21.15.100:0/909965558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f144c070cc0 0x7f144c156e40 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:09.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.788+0000 7f1452000640 1 -- 172.21.15.100:0/909965558 >> 172.21.15.100:0/909965558 conn(0x7f144c06d3d0 msgr2=0x7f144c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:09.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.788+0000 7f1452000640 1 -- 172.21.15.100:0/909965558 shutdown_connections 2024-09-17T04:29:09.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.788+0000 7f1452000640 1 -- 172.21.15.100:0/909965558 wait complete. 2024-09-17T04:29:09.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.790+0000 7f1452000640 1 Processor -- start 2024-09-17T04:29:09.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.790+0000 7f1452000640 1 -- start start 2024-09-17T04:29:09.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.791+0000 7f1452000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f144c0cfc80 0x7f144c0b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:09.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.792+0000 7f1452000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f144c15ce30 0x7f144c09c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:09.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.792+0000 7f1452000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f144c09c730 0x7f144c1f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:09.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.792+0000 7f1452000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f144c0cd100 con 0x7f144c09c730 2024-09-17T04:29:09.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.792+0000 7f1452000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f144c0ccf80 con 0x7f144c0cfc80 2024-09-17T04:29:09.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.792+0000 7f1452000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f144c0cd280 con 0x7f144c15ce30 2024-09-17T04:29:09.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.792+0000 7f144be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f144c09c730 0x7f144c1f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:09.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.792+0000 7f144be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f144c09c730 0x7f144c1f53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45298/0 (socket says 172.21.15.100:45298) 2024-09-17T04:29:09.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.792+0000 7f144be00640 1 -- 172.21.15.100:0/3948203011 learned_addr learned my addr 172.21.15.100:0/3948203011 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:09.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.792+0000 7f144b400640 1 --2- 172.21.15.100:0/3948203011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f144c0cfc80 0x7f144c0b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:09.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.792+0000 7f1442a00640 1 --2- 172.21.15.100:0/3948203011 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f144c15ce30 0x7f144c09c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:09.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.792+0000 7f144be00640 1 -- 172.21.15.100:0/3948203011 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f144c15ce30 msgr2=0x7f144c09c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:09.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.792+0000 7f144be00640 1 --2- 172.21.15.100:0/3948203011 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f144c15ce30 0x7f144c09c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:09.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.792+0000 7f144be00640 1 -- 172.21.15.100:0/3948203011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f144c0cfc80 msgr2=0x7f144c0b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:09.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.792+0000 7f144be00640 1 --2- 172.21.15.100:0/3948203011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f144c0cfc80 0x7f144c0b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:09.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.792+0000 7f144be00640 1 -- 172.21.15.100:0/3948203011 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144c09cb40 con 0x7f144c09c730 2024-09-17T04:29:09.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.793+0000 7f144be00640 1 --2- 172.21.15.100:0/3948203011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f144c09c730 0x7f144c1f53a0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1444002230 tx=0x7f144400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:09.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.793+0000 7f1443e00640 1 -- 172.21.15.100:0/3948203011 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1444003b70 con 0x7f144c09c730 2024-09-17T04:29:09.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.793+0000 7f1443e00640 1 -- 172.21.15.100:0/3948203011 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1444003d10 con 0x7f144c09c730 2024-09-17T04:29:09.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.793+0000 7f1452000640 1 -- 172.21.15.100:0/3948203011 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f144c0b2010 con 0x7f144c09c730 2024-09-17T04:29:09.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.793+0000 7f1443e00640 1 -- 172.21.15.100:0/3948203011 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14440052f0 con 0x7f144c09c730 2024-09-17T04:29:09.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.793+0000 7f1452000640 1 -- 172.21.15.100:0/3948203011 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f144c0b0d10 con 0x7f144c09c730 2024-09-17T04:29:09.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.794+0000 7f1452000640 1 -- 172.21.15.100:0/3948203011 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f144c0b0790 con 0x7f144c09c730 2024-09-17T04:29:09.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.795+0000 7f1443e00640 1 -- 172.21.15.100:0/3948203011 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 51133+0+0 (secure 0 0 0) 0x7f14440056a0 con 0x7f144c09c730 2024-09-17T04:29:09.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.796+0000 7f1443e00640 1 --2- 172.21.15.100:0/3948203011 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f142803e940 0x7f1428040e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:09.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.796+0000 7f1443e00640 1 -- 172.21.15.100:0/3948203011 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6326+0+0 (secure 0 0 0) 0x7f1444053d10 con 0x7f144c09c730 2024-09-17T04:29:09.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.796+0000 7f144b400640 1 --2- 172.21.15.100:0/3948203011 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f142803e940 0x7f1428040e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:09.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.796+0000 7f144b400640 1 --2- 172.21.15.100:0/3948203011 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f142803e940 0x7f1428040e50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f143c001150 tx=0x7f143c0010e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:09.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:09.801+0000 7f1443e00640 1 -- 172.21.15.100:0/3948203011 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f144c0b0790 con 0x7f144c09c730 2024-09-17T04:29:10.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.022+0000 7f1452000640 1 -- 172.21.15.100:0/3948203011 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f144c0cd790 con 0x7f144c09c730 2024-09-17T04:29:10.022 INFO:tasks.ceph.mon.a.smithi100.stderr:2024-09-17T04:29:10.023+0000 7fd3d0a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T04:29:10.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.025+0000 7f1443e00640 1 -- 172.21.15.100:0/3948203011 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v49) ==== 161+0+0 (secure 0 0 0) 0x7f144c09cb40 con 0x7f144c09c730 2024-09-17T04:29:10.024 INFO:teuthology.orchestra.run.smithi100.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T04:29:10.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.029+0000 7f1452000640 1 -- 172.21.15.100:0/3948203011 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f142803e940 msgr2=0x7f1428040e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:10.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.029+0000 7f1452000640 1 --2- 172.21.15.100:0/3948203011 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f142803e940 0x7f1428040e50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f143c001150 tx=0x7f143c0010e0 comp rx=0 tx=0).stop 2024-09-17T04:29:10.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.029+0000 7f1452000640 1 -- 172.21.15.100:0/3948203011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f144c09c730 msgr2=0x7f144c1f53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:10.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.029+0000 7f1452000640 1 --2- 172.21.15.100:0/3948203011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f144c09c730 0x7f144c1f53a0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1444002230 tx=0x7f144400cb90 comp rx=0 tx=0).stop 2024-09-17T04:29:10.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.029+0000 7f1452000640 1 -- 172.21.15.100:0/3948203011 shutdown_connections 2024-09-17T04:29:10.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.029+0000 7f1452000640 1 --2- 172.21.15.100:0/3948203011 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f142803e940 0x7f1428040e50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:10.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.029+0000 7f1452000640 1 --2- 172.21.15.100:0/3948203011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f144c09c730 0x7f144c1f53a0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:10.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.029+0000 7f1452000640 1 --2- 172.21.15.100:0/3948203011 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f144c15ce30 0x7f144c09c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:10.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.029+0000 7f1452000640 1 --2- 172.21.15.100:0/3948203011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f144c0cfc80 0x7f144c0b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:10.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.029+0000 7f1452000640 1 -- 172.21.15.100:0/3948203011 >> 172.21.15.100:0/3948203011 conn(0x7f144c06d3d0 msgr2=0x7f144c0b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:10.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.029+0000 7f1452000640 1 -- 172.21.15.100:0/3948203011 shutdown_connections 2024-09-17T04:29:10.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.029+0000 7f1452000640 1 -- 172.21.15.100:0/3948203011 wait complete. 2024-09-17T04:29:10.040 INFO:tasks.ceph.mds.c.smithi100.stderr: -8310> 2024-09-17T04:28:49.365+0000 7fb410000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T04:29:10.057 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T04:29:10.166 INFO:tasks.ceph.mds.c.smithi100.stdout:starting mds.c at 2024-09-17T04:29:10.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.287+0000 7f4605a00640 1 -- 172.21.15.100:0/459541962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4600145550 msgr2=0x7f460006fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:10.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.287+0000 7f4605a00640 1 --2- 172.21.15.100:0/459541962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4600145550 0x7f460006fb40 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f4600160280 tx=0x7f45f002a680 comp rx=0 tx=0).stop 2024-09-17T04:29:10.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.288+0000 7f4605a00640 1 -- 172.21.15.100:0/459541962 shutdown_connections 2024-09-17T04:29:10.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.288+0000 7f4605a00640 1 --2- 172.21.15.100:0/459541962 >> v2:172.21.15.184:3300/0 conn(0x7f4600145a40 0x7f4600073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:10.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.288+0000 7f4605a00640 1 --2- 172.21.15.100:0/459541962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4600145550 0x7f460006fb40 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:10.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.288+0000 7f4605a00640 1 -- 172.21.15.100:0/459541962 >> 172.21.15.100:0/459541962 conn(0x7f460006d1c0 msgr2=0x7f460006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:10.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.288+0000 7f4605a00640 1 -- 172.21.15.100:0/459541962 shutdown_connections 2024-09-17T04:29:10.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.288+0000 7f4605a00640 1 -- 172.21.15.100:0/459541962 wait complete. 2024-09-17T04:29:10.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.290+0000 7f4605a00640 1 Processor -- start 2024-09-17T04:29:10.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.290+0000 7f4605a00640 1 -- start start 2024-09-17T04:29:10.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.292+0000 7f4605a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4600073a80 0x7f46001f6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:10.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.292+0000 7f4605a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4600145550 0x7f46001f9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:10.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.292+0000 7f4605a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4600145a40 0x7f46001ec3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:10.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.292+0000 7f4605a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4600160a30 con 0x7f4600073a80 2024-09-17T04:29:10.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.292+0000 7f4605a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f46000ce870 con 0x7f4600145a40 2024-09-17T04:29:10.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.292+0000 7f4605a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4600160bb0 con 0x7f4600145550 2024-09-17T04:29:10.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.292+0000 7f45fea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4600073a80 0x7f46001f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:10.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.292+0000 7f45fea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4600073a80 0x7f46001f6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45328/0 (socket says 172.21.15.100:45328) 2024-09-17T04:29:10.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.292+0000 7f45fea00640 1 -- 172.21.15.100:0/3609218227 learned_addr learned my addr 172.21.15.100:0/3609218227 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:10.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.292+0000 7f45f6000640 1 --2- 172.21.15.100:0/3609218227 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4600145550 0x7f46001f9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:10.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.292+0000 7f45ff400640 1 --2- 172.21.15.100:0/3609218227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4600145a40 0x7f46001ec3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:10.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.292+0000 7f45fea00640 1 -- 172.21.15.100:0/3609218227 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4600145550 msgr2=0x7f46001f9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:10.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.292+0000 7f45fea00640 1 --2- 172.21.15.100:0/3609218227 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4600145550 0x7f46001f9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:10.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.292+0000 7f45fea00640 1 -- 172.21.15.100:0/3609218227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4600145a40 msgr2=0x7f46001ec3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:10.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.292+0000 7f45fea00640 1 --2- 172.21.15.100:0/3609218227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4600145a40 0x7f46001ec3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:10.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.292+0000 7f45fea00640 1 -- 172.21.15.100:0/3609218227 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46001f5ac0 con 0x7f4600073a80 2024-09-17T04:29:10.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.293+0000 7f45fea00640 1 --2- 172.21.15.100:0/3609218227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4600073a80 0x7f46001f6590 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f45f0034040 tx=0x7f45f0003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:10.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.293+0000 7f45f7400640 1 -- 172.21.15.100:0/3609218227 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f003e030 con 0x7f4600073a80 2024-09-17T04:29:10.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.293+0000 7f45f7400640 1 -- 172.21.15.100:0/3609218227 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45f0003d10 con 0x7f4600073a80 2024-09-17T04:29:10.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.293+0000 7f4605a00640 1 -- 172.21.15.100:0/3609218227 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46001f9bc0 con 0x7f4600073a80 2024-09-17T04:29:10.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.293+0000 7f45f7400640 1 -- 172.21.15.100:0/3609218227 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0035040 con 0x7f4600073a80 2024-09-17T04:29:10.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.293+0000 7f4605a00640 1 -- 172.21.15.100:0/3609218227 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46001f5d90 con 0x7f4600073a80 2024-09-17T04:29:10.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.294+0000 7f45f7400640 1 -- 172.21.15.100:0/3609218227 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 51133+0+0 (secure 0 0 0) 0x7f45f0041030 con 0x7f4600073a80 2024-09-17T04:29:10.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.294+0000 7f4605a00640 1 -- 172.21.15.100:0/3609218227 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46001610d0 con 0x7f4600073a80 2024-09-17T04:29:10.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.295+0000 7f45f7400640 1 --2- 172.21.15.100:0/3609218227 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f45dc03e940 0x7f45dc040e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:10.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.295+0000 7f45f7400640 1 -- 172.21.15.100:0/3609218227 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6412+0+0 (secure 0 0 0) 0x7f45f0074660 con 0x7f4600073a80 2024-09-17T04:29:10.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.295+0000 7f45f6000640 1 --2- 172.21.15.100:0/3609218227 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f45dc03e940 0x7f45dc040e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:10.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.296+0000 7f45f6000640 1 --2- 172.21.15.100:0/3609218227 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f45dc03e940 0x7f45dc040e50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f46000b1e50 tx=0x7f45e8005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:10.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.300+0000 7f45f7400640 1 -- 172.21.15.100:0/3609218227 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f45f0004ee0 con 0x7f4600073a80 2024-09-17T04:29:10.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.522+0000 7f4605a00640 1 -- 172.21.15.100:0/3609218227 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f46000b4c80 con 0x7f4600073a80 2024-09-17T04:29:10.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:10.668+0000 7f45f7400640 1 -- 172.21.15.100:0/3609218227 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f45f00049c0 con 0x7f4600073a80 2024-09-17T04:29:11.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.027+0000 7f45f7400640 1 -- 172.21.15.100:0/3609218227 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v50) ==== 104+0+0 (secure 0 0 0) 0x7f45f00532d0 con 0x7f4600073a80 2024-09-17T04:29:11.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.029+0000 7f4605a00640 1 -- 172.21.15.100:0/3609218227 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f45dc03e940 msgr2=0x7f45dc040e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:11.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.029+0000 7f4605a00640 1 --2- 172.21.15.100:0/3609218227 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f45dc03e940 0x7f45dc040e50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f46000b1e50 tx=0x7f45e8005f90 comp rx=0 tx=0).stop 2024-09-17T04:29:11.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.029+0000 7f4605a00640 1 -- 172.21.15.100:0/3609218227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4600073a80 msgr2=0x7f46001f6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:11.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.029+0000 7f4605a00640 1 --2- 172.21.15.100:0/3609218227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4600073a80 0x7f46001f6590 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f45f0034040 tx=0x7f45f0003820 comp rx=0 tx=0).stop 2024-09-17T04:29:11.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.029+0000 7f4605a00640 1 -- 172.21.15.100:0/3609218227 shutdown_connections 2024-09-17T04:29:11.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.029+0000 7f4605a00640 1 --2- 172.21.15.100:0/3609218227 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f45dc03e940 0x7f45dc040e50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:11.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.029+0000 7f4605a00640 1 --2- 172.21.15.100:0/3609218227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4600145a40 0x7f46001ec3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:11.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.029+0000 7f4605a00640 1 --2- 172.21.15.100:0/3609218227 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4600145550 0x7f46001f9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:11.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.029+0000 7f4605a00640 1 --2- 172.21.15.100:0/3609218227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4600073a80 0x7f46001f6590 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:11.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.029+0000 7f4605a00640 1 -- 172.21.15.100:0/3609218227 >> 172.21.15.100:0/3609218227 conn(0x7f460006d1c0 msgr2=0x7f4600068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:11.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.030+0000 7f4605a00640 1 -- 172.21.15.100:0/3609218227 shutdown_connections 2024-09-17T04:29:11.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.030+0000 7f4605a00640 1 -- 172.21.15.100:0/3609218227 wait complete. 2024-09-17T04:29:11.057 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T04:29:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.278+0000 7f7be2400640 1 -- 172.21.15.100:0/2964153384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bdc0705b0 msgr2=0x7f7bdc0af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.278+0000 7f7be2400640 1 --2- 172.21.15.100:0/2964153384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bdc0705b0 0x7f7bdc0af3c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc0b38b0 tx=0x7f7bc802a4d0 comp rx=0 tx=0).stop 2024-09-17T04:29:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.279+0000 7f7be2400640 1 -- 172.21.15.100:0/2964153384 shutdown_connections 2024-09-17T04:29:11.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.279+0000 7f7be2400640 1 --2- 172.21.15.100:0/2964153384 >> v2:172.21.15.184:3300/0 conn(0x7f7bdc1532e0 0x7f7bdc1536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:11.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.279+0000 7f7be2400640 1 --2- 172.21.15.100:0/2964153384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bdc0705b0 0x7f7bdc0af3c0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:11.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.279+0000 7f7be2400640 1 --2- 172.21.15.100:0/2964153384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bdc0b3ec0 0x7f7bdc0b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:11.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.279+0000 7f7be2400640 1 -- 172.21.15.100:0/2964153384 >> 172.21.15.100:0/2964153384 conn(0x7f7bdc0b1dd0 msgr2=0x7f7bdc0b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:11.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.279+0000 7f7be2400640 1 -- 172.21.15.100:0/2964153384 shutdown_connections 2024-09-17T04:29:11.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.279+0000 7f7be2400640 1 -- 172.21.15.100:0/2964153384 wait complete. 2024-09-17T04:29:11.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.281+0000 7f7be2400640 1 Processor -- start 2024-09-17T04:29:11.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.281+0000 7f7be2400640 1 -- start start 2024-09-17T04:29:11.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.283+0000 7f7be2400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bdc0705b0 0x7f7bdc09aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:11.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.283+0000 7f7be2400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bdc0b3ec0 0x7f7bdc1459d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:11.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.283+0000 7f7be2400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bdc1532e0 0x7f7bdc155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:11.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.283+0000 7f7be2400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc160860 con 0x7f7bdc0b3ec0 2024-09-17T04:29:11.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.283+0000 7f7be2400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc1606e0 con 0x7f7bdc0705b0 2024-09-17T04:29:11.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.283+0000 7f7be2400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7bdc1609e0 con 0x7f7bdc1532e0 2024-09-17T04:29:11.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.283+0000 7f7bdbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bdc0b3ec0 0x7f7bdc1459d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:11.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.283+0000 7f7bdbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bdc0b3ec0 0x7f7bdc1459d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45346/0 (socket says 172.21.15.100:45346) 2024-09-17T04:29:11.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.283+0000 7f7bdbe00640 1 -- 172.21.15.100:0/853570859 learned_addr learned my addr 172.21.15.100:0/853570859 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:11.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.283+0000 7f7be1000640 1 --2- 172.21.15.100:0/853570859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bdc0705b0 0x7f7bdc09aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:11.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.283+0000 7f7be1a00640 1 --2- 172.21.15.100:0/853570859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bdc1532e0 0x7f7bdc155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:11.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.283+0000 7f7bdbe00640 1 -- 172.21.15.100:0/853570859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bdc1532e0 msgr2=0x7f7bdc155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:11.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.283+0000 7f7bdbe00640 1 --2- 172.21.15.100:0/853570859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bdc1532e0 0x7f7bdc155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:11.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.283+0000 7f7bdbe00640 1 -- 172.21.15.100:0/853570859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bdc0705b0 msgr2=0x7f7bdc09aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:11.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.283+0000 7f7bdbe00640 1 --2- 172.21.15.100:0/853570859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bdc0705b0 0x7f7bdc09aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:11.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.283+0000 7f7bdbe00640 1 -- 172.21.15.100:0/853570859 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bdc09df10 con 0x7f7bdc0b3ec0 2024-09-17T04:29:11.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.284+0000 7f7bdbe00640 1 --2- 172.21.15.100:0/853570859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bdc0b3ec0 0x7f7bdc1459d0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8034040 tx=0x7f7bc80039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:11.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.284+0000 7f7bd8c00640 1 -- 172.21.15.100:0/853570859 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc803e030 con 0x7f7bdc0b3ec0 2024-09-17T04:29:11.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.284+0000 7f7bd8c00640 1 -- 172.21.15.100:0/853570859 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7bc80047a0 con 0x7f7bdc0b3ec0 2024-09-17T04:29:11.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.284+0000 7f7bd8c00640 1 -- 172.21.15.100:0/853570859 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8035040 con 0x7f7bdc0b3ec0 2024-09-17T04:29:11.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.284+0000 7f7be2400640 1 -- 172.21.15.100:0/853570859 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bdc09d670 con 0x7f7bdc0b3ec0 2024-09-17T04:29:11.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.284+0000 7f7be2400640 1 -- 172.21.15.100:0/853570859 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bdc09dc30 con 0x7f7bdc0b3ec0 2024-09-17T04:29:11.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.285+0000 7f7be2400640 1 -- 172.21.15.100:0/853570859 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bdc0af410 con 0x7f7bdc0b3ec0 2024-09-17T04:29:11.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.286+0000 7f7bd8c00640 1 -- 172.21.15.100:0/853570859 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f7bc8041030 con 0x7f7bdc0b3ec0 2024-09-17T04:29:11.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.287+0000 7f7bd8c00640 1 --2- 172.21.15.100:0/853570859 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7bbc078cb0 0x7f7bbc07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:11.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.287+0000 7f7bd8c00640 1 -- 172.21.15.100:0/853570859 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6412+0+0 (secure 0 0 0) 0x7f7bc80bba80 con 0x7f7bdc0b3ec0 2024-09-17T04:29:11.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.288+0000 7f7be1000640 1 --2- 172.21.15.100:0/853570859 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7bbc078cb0 0x7f7bbc07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:11.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.288+0000 7f7be1000640 1 --2- 172.21.15.100:0/853570859 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7bbc078cb0 0x7f7bbc07b1c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0005fd0 tx=0x7f7bd000b260 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:11.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.292+0000 7f7bd8c00640 1 -- 172.21.15.100:0/853570859 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f7bc80bde20 con 0x7f7bdc0b3ec0 2024-09-17T04:29:11.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:11.475+0000 7f7be2400640 1 -- 172.21.15.100:0/853570859 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7bdc15fa70 con 0x7f7bdc0b3ec0 2024-09-17T04:29:12.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.032+0000 7f7bd8c00640 1 -- 172.21.15.100:0/853570859 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v78) ==== 181+0+0 (secure 0 0 0) 0x7f7bc80832f0 con 0x7f7bdc0b3ec0 2024-09-17T04:29:12.032 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' removed 2024-09-17T04:29:12.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.035+0000 7f7be2400640 1 -- 172.21.15.100:0/853570859 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7bbc078cb0 msgr2=0x7f7bbc07b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:12.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.035+0000 7f7be2400640 1 --2- 172.21.15.100:0/853570859 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7bbc078cb0 0x7f7bbc07b1c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0005fd0 tx=0x7f7bd000b260 comp rx=0 tx=0).stop 2024-09-17T04:29:12.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.035+0000 7f7be2400640 1 -- 172.21.15.100:0/853570859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bdc0b3ec0 msgr2=0x7f7bdc1459d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:12.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.035+0000 7f7be2400640 1 --2- 172.21.15.100:0/853570859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bdc0b3ec0 0x7f7bdc1459d0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8034040 tx=0x7f7bc80039e0 comp rx=0 tx=0).stop 2024-09-17T04:29:12.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.036+0000 7f7be2400640 1 -- 172.21.15.100:0/853570859 shutdown_connections 2024-09-17T04:29:12.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.036+0000 7f7be2400640 1 --2- 172.21.15.100:0/853570859 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7bbc078cb0 0x7f7bbc07b1c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:12.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.036+0000 7f7be2400640 1 --2- 172.21.15.100:0/853570859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bdc1532e0 0x7f7bdc155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:12.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.036+0000 7f7be2400640 1 --2- 172.21.15.100:0/853570859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bdc0b3ec0 0x7f7bdc1459d0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:12.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.036+0000 7f7be2400640 1 --2- 172.21.15.100:0/853570859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bdc0705b0 0x7f7bdc09aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:12.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.036+0000 7f7be2400640 1 -- 172.21.15.100:0/853570859 >> 172.21.15.100:0/853570859 conn(0x7f7bdc0b1dd0 msgr2=0x7f7bdc152b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:12.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.036+0000 7f7be2400640 1 -- 172.21.15.100:0/853570859 shutdown_connections 2024-09-17T04:29:12.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.037+0000 7f7be2400640 1 -- 172.21.15.100:0/853570859 wait complete. 2024-09-17T04:29:12.060 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T04:29:12.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.284+0000 7f55a9400640 1 -- 172.21.15.100:0/1198049978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55a406d4b0 msgr2=0x7f55a40b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:12.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.284+0000 7f55a9400640 1 --2- 172.21.15.100:0/1198049978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55a406d4b0 0x7f55a40b4620 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f5590009830 tx=0x7f559002a510 comp rx=0 tx=0).stop 2024-09-17T04:29:12.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.284+0000 7f55a9400640 1 -- 172.21.15.100:0/1198049978 shutdown_connections 2024-09-17T04:29:12.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.284+0000 7f55a9400640 1 --2- 172.21.15.100:0/1198049978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55a406d4b0 0x7f55a40b4620 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:12.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.284+0000 7f55a9400640 1 -- 172.21.15.100:0/1198049978 >> 172.21.15.100:0/1198049978 conn(0x7f55a40b2540 msgr2=0x7f55a40d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:12.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.284+0000 7f55a9400640 1 -- 172.21.15.100:0/1198049978 shutdown_connections 2024-09-17T04:29:12.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.284+0000 7f55a9400640 1 -- 172.21.15.100:0/1198049978 wait complete. 2024-09-17T04:29:12.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.287+0000 7f55a9400640 1 Processor -- start 2024-09-17T04:29:12.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.287+0000 7f55a9400640 1 -- start start 2024-09-17T04:29:12.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.288+0000 7f55a9400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55a40af100 0x7f55a41f8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:12.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.288+0000 7f55a9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55a40cc830 0x7f55a41e5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:12.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.288+0000 7f55a9400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55a4099540 0x7f55a41e9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:12.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.288+0000 7f55a9400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f55a40b3780 con 0x7f55a40cc830 2024-09-17T04:29:12.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.288+0000 7f55a9400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f55a40b3600 con 0x7f55a4099540 2024-09-17T04:29:12.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.288+0000 7f55a9400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f55a40b3900 con 0x7f55a40af100 2024-09-17T04:29:12.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.288+0000 7f55a1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55a40cc830 0x7f55a41e5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:12.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.289+0000 7f55a1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55a40cc830 0x7f55a41e5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45370/0 (socket says 172.21.15.100:45370) 2024-09-17T04:29:12.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.289+0000 7f55a1e00640 1 -- 172.21.15.100:0/3440330626 learned_addr learned my addr 172.21.15.100:0/3440330626 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:12.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.289+0000 7f55a3200640 1 --2- 172.21.15.100:0/3440330626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55a4099540 0x7f55a41e9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:12.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.289+0000 7f55a2800640 1 --2- 172.21.15.100:0/3440330626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55a40af100 0x7f55a41f8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:12.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.289+0000 7f55a1e00640 1 -- 172.21.15.100:0/3440330626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55a40af100 msgr2=0x7f55a41f8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:12.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.289+0000 7f55a1e00640 1 --2- 172.21.15.100:0/3440330626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55a40af100 0x7f55a41f8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:12.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.289+0000 7f55a1e00640 1 -- 172.21.15.100:0/3440330626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55a4099540 msgr2=0x7f55a41e9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:12.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.289+0000 7f55a1e00640 1 --2- 172.21.15.100:0/3440330626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55a4099540 0x7f55a41e9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:12.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.289+0000 7f55a1e00640 1 -- 172.21.15.100:0/3440330626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a41f9030 con 0x7f55a40cc830 2024-09-17T04:29:12.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.289+0000 7f55a3200640 1 --2- 172.21.15.100:0/3440330626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55a4099540 0x7f55a41e9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:29:12.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.289+0000 7f55a2800640 1 --2- 172.21.15.100:0/3440330626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55a40af100 0x7f55a41f8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:29:12.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.289+0000 7f55a1e00640 1 --2- 172.21.15.100:0/3440330626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55a40cc830 0x7f55a41e5620 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f5590009950 tx=0x7f5590033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:12.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.290+0000 7f559aa00640 1 -- 172.21.15.100:0/3440330626 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590031d60 con 0x7f55a40cc830 2024-09-17T04:29:12.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.290+0000 7f559aa00640 1 -- 172.21.15.100:0/3440330626 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55900330d0 con 0x7f55a40cc830 2024-09-17T04:29:12.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.290+0000 7f559aa00640 1 -- 172.21.15.100:0/3440330626 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590033520 con 0x7f55a40cc830 2024-09-17T04:29:12.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.290+0000 7f55a9400640 1 -- 172.21.15.100:0/3440330626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a40cfbd0 con 0x7f55a40cc830 2024-09-17T04:29:12.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.290+0000 7f55a9400640 1 -- 172.21.15.100:0/3440330626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a41e97a0 con 0x7f55a40cc830 2024-09-17T04:29:12.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.291+0000 7f55a9400640 1 -- 172.21.15.100:0/3440330626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55a40b1760 con 0x7f55a40cc830 2024-09-17T04:29:12.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.292+0000 7f559aa00640 1 -- 172.21.15.100:0/3440330626 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f55900035f0 con 0x7f55a40cc830 2024-09-17T04:29:12.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.293+0000 7f559aa00640 1 --2- 172.21.15.100:0/3440330626 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5584078cb0 0x7f558407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:12.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.293+0000 7f559aa00640 1 -- 172.21.15.100:0/3440330626 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5974+0+0 (secure 0 0 0) 0x7f55900bc5f0 con 0x7f55a40cc830 2024-09-17T04:29:12.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.293+0000 7f55a2800640 1 --2- 172.21.15.100:0/3440330626 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5584078cb0 0x7f558407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:12.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.294+0000 7f55a2800640 1 --2- 172.21.15.100:0/3440330626 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5584078cb0 0x7f558407b1c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5594000f60 tx=0x7f5594009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:12.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.298+0000 7f559aa00640 1 -- 172.21.15.100:0/3440330626 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f55a40b1760 con 0x7f55a40cc830 2024-09-17T04:29:12.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:12.485+0000 7f55a9400640 1 -- 172.21.15.100:0/3440330626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f55a40b3e30 con 0x7f55a40cc830 2024-09-17T04:29:13.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.035+0000 7f559aa00640 1 -- 172.21.15.100:0/3440330626 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v79) ==== 169+0+0 (secure 0 0 0) 0x7f55a41f9030 con 0x7f55a40cc830 2024-09-17T04:29:13.034 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' removed 2024-09-17T04:29:13.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.038+0000 7f55a9400640 1 -- 172.21.15.100:0/3440330626 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5584078cb0 msgr2=0x7f558407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:13.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.038+0000 7f55a9400640 1 --2- 172.21.15.100:0/3440330626 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5584078cb0 0x7f558407b1c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5594000f60 tx=0x7f5594009040 comp rx=0 tx=0).stop 2024-09-17T04:29:13.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.038+0000 7f55a9400640 1 -- 172.21.15.100:0/3440330626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55a40cc830 msgr2=0x7f55a41e5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:13.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.038+0000 7f55a9400640 1 --2- 172.21.15.100:0/3440330626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55a40cc830 0x7f55a41e5620 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f5590009950 tx=0x7f5590033d60 comp rx=0 tx=0).stop 2024-09-17T04:29:13.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.039+0000 7f55a9400640 1 -- 172.21.15.100:0/3440330626 shutdown_connections 2024-09-17T04:29:13.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.039+0000 7f55a9400640 1 --2- 172.21.15.100:0/3440330626 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5584078cb0 0x7f558407b1c0 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5594000f60 tx=0x7f5594009040 comp rx=0 tx=0).stop 2024-09-17T04:29:13.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.039+0000 7f55a9400640 1 --2- 172.21.15.100:0/3440330626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55a4099540 0x7f55a41e9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:13.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.039+0000 7f55a9400640 1 --2- 172.21.15.100:0/3440330626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55a40cc830 0x7f55a41e5620 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:13.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.039+0000 7f55a9400640 1 --2- 172.21.15.100:0/3440330626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55a40af100 0x7f55a41f8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:13.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.039+0000 7f55a9400640 1 -- 172.21.15.100:0/3440330626 >> 172.21.15.100:0/3440330626 conn(0x7f55a40b2540 msgr2=0x7f55a40c9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:13.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.040+0000 7f55a9400640 1 -- 172.21.15.100:0/3440330626 shutdown_connections 2024-09-17T04:29:13.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.040+0000 7f55a9400640 1 -- 172.21.15.100:0/3440330626 wait complete. 2024-09-17T04:29:13.069 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T04:29:13.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.298+0000 7f55c7200640 1 -- 172.21.15.100:0/3528506089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55c015d540 msgr2=0x7f55c0156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:13.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.298+0000 7f55c7200640 1 --2- 172.21.15.100:0/3528506089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55c015d540 0x7f55c0156e40 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f55b8009ca0 tx=0x7f55b802aae0 comp rx=0 tx=0).stop 2024-09-17T04:29:13.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.298+0000 7f55c7200640 1 -- 172.21.15.100:0/3528506089 shutdown_connections 2024-09-17T04:29:13.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.298+0000 7f55c7200640 1 --2- 172.21.15.100:0/3528506089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55c015d540 0x7f55c0156e40 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:13.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.298+0000 7f55c7200640 1 -- 172.21.15.100:0/3528506089 >> 172.21.15.100:0/3528506089 conn(0x7f55c006d3d0 msgr2=0x7f55c00b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:13.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.298+0000 7f55c7200640 1 -- 172.21.15.100:0/3528506089 shutdown_connections 2024-09-17T04:29:13.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.298+0000 7f55c7200640 1 -- 172.21.15.100:0/3528506089 wait complete. 2024-09-17T04:29:13.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.300+0000 7f55c7200640 1 Processor -- start 2024-09-17T04:29:13.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.300+0000 7f55c7200640 1 -- start start 2024-09-17T04:29:13.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.302+0000 7f55c7200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55c00cfc80 0x7f55c01f5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:13.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.302+0000 7f55c7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55c015d540 0x7f55c00afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:13.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.302+0000 7f55c7200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55c00b0230 0x7f55c00b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:13.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.302+0000 7f55c7200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f55c00cd190 con 0x7f55c015d540 2024-09-17T04:29:13.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.302+0000 7f55c7200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f55c00cd010 con 0x7f55c00b0230 2024-09-17T04:29:13.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.302+0000 7f55c7200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f55c00cd310 con 0x7f55c00cfc80 2024-09-17T04:29:13.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.302+0000 7f55bf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55c015d540 0x7f55c00afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:13.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.302+0000 7f55bf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55c015d540 0x7f55c00afcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45392/0 (socket says 172.21.15.100:45392) 2024-09-17T04:29:13.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.302+0000 7f55bf400640 1 -- 172.21.15.100:0/4202076458 learned_addr learned my addr 172.21.15.100:0/4202076458 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:13.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.302+0000 7f55c5200640 1 --2- 172.21.15.100:0/4202076458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55c00b0230 0x7f55c00b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:13.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.302+0000 7f55bfe00640 1 --2- 172.21.15.100:0/4202076458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55c00cfc80 0x7f55c01f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:13.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.302+0000 7f55bf400640 1 -- 172.21.15.100:0/4202076458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55c00cfc80 msgr2=0x7f55c01f5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:13.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.302+0000 7f55bf400640 1 --2- 172.21.15.100:0/4202076458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55c00cfc80 0x7f55c01f5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:13.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.302+0000 7f55bf400640 1 -- 172.21.15.100:0/4202076458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55c00b0230 msgr2=0x7f55c00b0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:13.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.302+0000 7f55bf400640 1 --2- 172.21.15.100:0/4202076458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55c00b0230 0x7f55c00b0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:13.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.302+0000 7f55bf400640 1 -- 172.21.15.100:0/4202076458 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c009b610 con 0x7f55c015d540 2024-09-17T04:29:13.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.302+0000 7f55bfe00640 1 --2- 172.21.15.100:0/4202076458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55c00cfc80 0x7f55c01f5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:13.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.302+0000 7f55c5200640 1 --2- 172.21.15.100:0/4202076458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55c00b0230 0x7f55c00b0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:13.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.303+0000 7f55bf400640 1 --2- 172.21.15.100:0/4202076458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55c015d540 0x7f55c00afcf0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00c340 tx=0x7f55ac00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:13.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.303+0000 7f55b7e00640 1 -- 172.21.15.100:0/4202076458 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac017070 con 0x7f55c015d540 2024-09-17T04:29:13.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.303+0000 7f55b7e00640 1 -- 172.21.15.100:0/4202076458 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55ac005400 con 0x7f55c015d540 2024-09-17T04:29:13.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.303+0000 7f55b7e00640 1 -- 172.21.15.100:0/4202076458 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac003870 con 0x7f55c015d540 2024-09-17T04:29:13.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.303+0000 7f55c7200640 1 -- 172.21.15.100:0/4202076458 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c009b8f0 con 0x7f55c015d540 2024-09-17T04:29:13.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.303+0000 7f55c7200640 1 -- 172.21.15.100:0/4202076458 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c009beb0 con 0x7f55c015d540 2024-09-17T04:29:13.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.304+0000 7f55c7200640 1 -- 172.21.15.100:0/4202076458 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55c00cd800 con 0x7f55c015d540 2024-09-17T04:29:13.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.305+0000 7f55b7e00640 1 -- 172.21.15.100:0/4202076458 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f55ac01a070 con 0x7f55c015d540 2024-09-17T04:29:13.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.306+0000 7f55b7e00640 1 --2- 172.21.15.100:0/4202076458 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f55a0078cb0 0x7f55a007b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:13.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.307+0000 7f55b7e00640 1 -- 172.21.15.100:0/4202076458 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5502+0+0 (secure 0 0 0) 0x7f55ac09a0d0 con 0x7f55c015d540 2024-09-17T04:29:13.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.307+0000 7f55bfe00640 1 --2- 172.21.15.100:0/4202076458 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f55a0078cb0 0x7f55a007b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:13.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.307+0000 7f55bfe00640 1 --2- 172.21.15.100:0/4202076458 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f55a0078cb0 0x7f55a007b1c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f55b0007fd0 tx=0x7f55b0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:13.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.311+0000 7f55b7e00640 1 -- 172.21.15.100:0/4202076458 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f55ac061d70 con 0x7f55c015d540 2024-09-17T04:29:13.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.494+0000 7f55c7200640 1 -- 172.21.15.100:0/4202076458 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f55c009b360 con 0x7f55c015d540 2024-09-17T04:29:13.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.498+0000 7f55b7e00640 1 -- 172.21.15.100:0/4202076458 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v39) ==== 71+0+0 (secure 0 0 0) 0x7f55ac0667a0 con 0x7f55c015d540 2024-09-17T04:29:13.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.501+0000 7f55c7200640 1 -- 172.21.15.100:0/4202076458 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f55a0078cb0 msgr2=0x7f55a007b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:13.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.501+0000 7f55c7200640 1 --2- 172.21.15.100:0/4202076458 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f55a0078cb0 0x7f55a007b1c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f55b0007fd0 tx=0x7f55b0009040 comp rx=0 tx=0).stop 2024-09-17T04:29:13.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.501+0000 7f55c7200640 1 -- 172.21.15.100:0/4202076458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55c015d540 msgr2=0x7f55c00afcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:13.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.501+0000 7f55c7200640 1 --2- 172.21.15.100:0/4202076458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55c015d540 0x7f55c00afcf0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00c340 tx=0x7f55ac00c800 comp rx=0 tx=0).stop 2024-09-17T04:29:13.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.501+0000 7f55c7200640 1 -- 172.21.15.100:0/4202076458 shutdown_connections 2024-09-17T04:29:13.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.501+0000 7f55c7200640 1 --2- 172.21.15.100:0/4202076458 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f55a0078cb0 0x7f55a007b1c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:13.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.501+0000 7f55c7200640 1 --2- 172.21.15.100:0/4202076458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55c00b0230 0x7f55c00b0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:13.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.501+0000 7f55c7200640 1 --2- 172.21.15.100:0/4202076458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55c015d540 0x7f55c00afcf0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:13.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.501+0000 7f55c7200640 1 --2- 172.21.15.100:0/4202076458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55c00cfc80 0x7f55c01f5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:13.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.501+0000 7f55c7200640 1 -- 172.21.15.100:0/4202076458 >> 172.21.15.100:0/4202076458 conn(0x7f55c006d3d0 msgr2=0x7f55c01603b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:13.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.501+0000 7f55c7200640 1 -- 172.21.15.100:0/4202076458 shutdown_connections 2024-09-17T04:29:13.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.501+0000 7f55c7200640 1 -- 172.21.15.100:0/4202076458 wait complete. 2024-09-17T04:29:13.527 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2024-09-17T04:29:13.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.749+0000 7f7defe00640 1 -- 172.21.15.100:0/954959376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7dd4004020 msgr2=0x7f7dd4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:13.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.749+0000 7f7defe00640 1 --2- 172.21.15.100:0/954959376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7dd4004020 0x7f7dd4004800 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc009b20 tx=0x7f7dd4005830 comp rx=0 tx=0).stop 2024-09-17T04:29:13.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.749+0000 7f7def400640 1 -- 172.21.15.100:0/954959376 reap_dead start 2024-09-17T04:29:13.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.749+0000 7f7defe00640 1 -- 172.21.15.100:0/954959376 shutdown_connections 2024-09-17T04:29:13.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.749+0000 7f7defe00640 1 -- 172.21.15.100:0/954959376 >> 172.21.15.100:0/954959376 conn(0x7f7df00b1e40 msgr2=0x7f7df00b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:13.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.749+0000 7f7defe00640 1 -- 172.21.15.100:0/954959376 shutdown_connections 2024-09-17T04:29:13.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.749+0000 7f7defe00640 1 -- 172.21.15.100:0/954959376 wait complete. 2024-09-17T04:29:13.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.752+0000 7f7defe00640 1 Processor -- start 2024-09-17T04:29:13.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.752+0000 7f7defe00640 1 -- start start 2024-09-17T04:29:13.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.753+0000 7f7defe00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7dd4004020 0x7f7df01f4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:13.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.753+0000 7f7defe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7df0147f50 0x7f7df0158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:13.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.753+0000 7f7defe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7df01585d0 0x7f7df01589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:13.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.753+0000 7f7defe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7df00b30e0 con 0x7f7df0147f50 2024-09-17T04:29:13.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.753+0000 7f7defe00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7df00b2f60 con 0x7f7df01585d0 2024-09-17T04:29:13.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.753+0000 7f7defe00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7df00b3260 con 0x7f7dd4004020 2024-09-17T04:29:13.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.754+0000 7f7dee000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7df0147f50 0x7f7df0158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:13.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.754+0000 7f7dee000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7df0147f50 0x7f7df0158090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45408/0 (socket says 172.21.15.100:45408) 2024-09-17T04:29:13.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.754+0000 7f7dee000640 1 -- 172.21.15.100:0/3070031159 learned_addr learned my addr 172.21.15.100:0/3070031159 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:13.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.754+0000 7f7deea00640 1 --2- 172.21.15.100:0/3070031159 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7dd4004020 0x7f7df01f4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:13.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.754+0000 7f7def400640 1 --2- 172.21.15.100:0/3070031159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7df01585d0 0x7f7df01589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:13.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.754+0000 7f7dee000640 1 -- 172.21.15.100:0/3070031159 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7dd4004020 msgr2=0x7f7df01f4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:13.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.754+0000 7f7dee000640 1 --2- 172.21.15.100:0/3070031159 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7dd4004020 0x7f7df01f4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:13.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.754+0000 7f7dee000640 1 -- 172.21.15.100:0/3070031159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7df01585d0 msgr2=0x7f7df01589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:13.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.754+0000 7f7dee000640 1 --2- 172.21.15.100:0/3070031159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7df01585d0 0x7f7df01589e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:13.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.754+0000 7f7dee000640 1 -- 172.21.15.100:0/3070031159 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df0156870 con 0x7f7df0147f50 2024-09-17T04:29:13.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.754+0000 7f7dee000640 1 --2- 172.21.15.100:0/3070031159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7df0147f50 0x7f7df0158090 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc036040 tx=0x7f7ddc033650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:13.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.755+0000 7f7de6a00640 1 -- 172.21.15.100:0/3070031159 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc038040 con 0x7f7df0147f50 2024-09-17T04:29:13.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.755+0000 7f7de6a00640 1 -- 172.21.15.100:0/3070031159 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ddc041070 con 0x7f7df0147f50 2024-09-17T04:29:13.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.755+0000 7f7defe00640 1 -- 172.21.15.100:0/3070031159 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df0072fe0 con 0x7f7df0147f50 2024-09-17T04:29:13.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.755+0000 7f7de6a00640 1 -- 172.21.15.100:0/3070031159 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc033c80 con 0x7f7df0147f50 2024-09-17T04:29:13.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.755+0000 7f7defe00640 1 -- 172.21.15.100:0/3070031159 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df00735a0 con 0x7f7df0147f50 2024-09-17T04:29:13.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.756+0000 7f7defe00640 1 -- 172.21.15.100:0/3070031159 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df01453e0 con 0x7f7df0147f50 2024-09-17T04:29:13.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.757+0000 7f7de6a00640 1 -- 172.21.15.100:0/3070031159 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f7ddc02b870 con 0x7f7df0147f50 2024-09-17T04:29:13.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.758+0000 7f7de6a00640 1 --2- 172.21.15.100:0/3070031159 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7dd0078cb0 0x7f7dd007b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:13.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.758+0000 7f7de6a00640 1 -- 172.21.15.100:0/3070031159 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5502+0+0 (secure 0 0 0) 0x7f7ddc0bb700 con 0x7f7df0147f50 2024-09-17T04:29:13.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.758+0000 7f7deea00640 1 --2- 172.21.15.100:0/3070031159 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7dd0078cb0 0x7f7dd007b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:13.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.759+0000 7f7deea00640 1 --2- 172.21.15.100:0/3070031159 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7dd0078cb0 0x7f7dd007b1c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7de0000fb0 tx=0x7f7de0006040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:13.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:13.762+0000 7f7de6a00640 1 -- 172.21.15.100:0/3070031159 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f7df01453e0 con 0x7f7df0147f50 2024-09-17T04:29:14.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.013+0000 7f7defe00640 1 -- 172.21.15.100:0/3070031159 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) -- 0x7f7df015d670 con 0x7f7df0147f50 2024-09-17T04:29:14.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.037+0000 7f7de6a00640 1 -- 172.21.15.100:0/3070031159 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v286) ==== 131+0+0 (secure 0 0 0) 0x7f7ddc033e20 con 0x7f7df0147f50 2024-09-17T04:29:14.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.039+0000 7f7defe00640 1 -- 172.21.15.100:0/3070031159 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7dd0078cb0 msgr2=0x7f7dd007b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:14.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.039+0000 7f7defe00640 1 --2- 172.21.15.100:0/3070031159 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7dd0078cb0 0x7f7dd007b1c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7de0000fb0 tx=0x7f7de0006040 comp rx=0 tx=0).stop 2024-09-17T04:29:14.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.039+0000 7f7defe00640 1 -- 172.21.15.100:0/3070031159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7df0147f50 msgr2=0x7f7df0158090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:14.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.039+0000 7f7defe00640 1 --2- 172.21.15.100:0/3070031159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7df0147f50 0x7f7df0158090 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc036040 tx=0x7f7ddc033650 comp rx=0 tx=0).stop 2024-09-17T04:29:14.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.039+0000 7f7defe00640 1 -- 172.21.15.100:0/3070031159 shutdown_connections 2024-09-17T04:29:14.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.039+0000 7f7defe00640 1 --2- 172.21.15.100:0/3070031159 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7dd0078cb0 0x7f7dd007b1c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:14.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.039+0000 7f7defe00640 1 --2- 172.21.15.100:0/3070031159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7df01585d0 0x7f7df01589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:14.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.039+0000 7f7defe00640 1 --2- 172.21.15.100:0/3070031159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7df0147f50 0x7f7df0158090 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:14.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.039+0000 7f7defe00640 1 --2- 172.21.15.100:0/3070031159 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7dd4004020 0x7f7df01f4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:14.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.039+0000 7f7defe00640 1 -- 172.21.15.100:0/3070031159 >> 172.21.15.100:0/3070031159 conn(0x7f7df00b1e40 msgr2=0x7f7df0155b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:14.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.039+0000 7f7defe00640 1 -- 172.21.15.100:0/3070031159 shutdown_connections 2024-09-17T04:29:14.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.039+0000 7f7defe00640 1 -- 172.21.15.100:0/3070031159 wait complete. 2024-09-17T04:29:14.061 INFO:tasks.cephfs_test_runner:test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-09-17T04:29:14.061 INFO:tasks.cephfs_test_runner:Starting test: test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) Test the creating vols, subvols subvolgroups fails when their names uses 2024-09-17T04:29:14.062 INFO:tasks.cephfs_test_runner:test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) 2024-09-17T04:29:14.062 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:29:14.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.285+0000 7f5e8d000640 1 -- 172.21.15.100:0/4014076655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e6c005350 msgr2=0x7f5e6c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:14.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.285+0000 7f5e8d000640 1 --2- 172.21.15.100:0/4014076655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e6c005350 0x7f5e6c00b9c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f5e80001dc0 tx=0x7f5e6c005830 comp rx=0 tx=0).stop 2024-09-17T04:29:14.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.285+0000 7f5e8d000640 1 -- 172.21.15.100:0/4014076655 shutdown_connections 2024-09-17T04:29:14.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.285+0000 7f5e8d000640 1 --2- 172.21.15.100:0/4014076655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e6c005350 0x7f5e6c00b9c0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:14.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.285+0000 7f5e8d000640 1 -- 172.21.15.100:0/4014076655 >> 172.21.15.100:0/4014076655 conn(0x7f5e8808bfe0 msgr2=0x7f5e8809d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:14.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.285+0000 7f5e8d000640 1 -- 172.21.15.100:0/4014076655 shutdown_connections 2024-09-17T04:29:14.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.285+0000 7f5e8d000640 1 -- 172.21.15.100:0/4014076655 wait complete. 2024-09-17T04:29:14.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.287+0000 7f5e8d000640 1 Processor -- start 2024-09-17T04:29:14.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.287+0000 7f5e8d000640 1 -- start start 2024-09-17T04:29:14.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.289+0000 7f5e8d000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e6c005350 0x7f5e8815b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:14.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.289+0000 7f5e8d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e88158040 0x7f5e8814e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:14.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.289+0000 7f5e8d000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e881609b0 0x7f5e88160dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:14.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.289+0000 7f5e8d000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5e88162bd0 con 0x7f5e88158040 2024-09-17T04:29:14.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.289+0000 7f5e8d000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5e88162a50 con 0x7f5e6c005350 2024-09-17T04:29:14.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.289+0000 7f5e8d000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5e88162d50 con 0x7f5e881609b0 2024-09-17T04:29:14.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.289+0000 7f5e7fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e88158040 0x7f5e8814e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:14.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.289+0000 7f5e7fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e88158040 0x7f5e8814e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45424/0 (socket says 172.21.15.100:45424) 2024-09-17T04:29:14.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.289+0000 7f5e7fe00640 1 -- 172.21.15.100:0/2420538513 learned_addr learned my addr 172.21.15.100:0/2420538513 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:14.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.289+0000 7f5e87400640 1 --2- 172.21.15.100:0/2420538513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e6c005350 0x7f5e8815b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:14.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.289+0000 7f5e87e00640 1 --2- 172.21.15.100:0/2420538513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e881609b0 0x7f5e88160dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:14.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.289+0000 7f5e7fe00640 1 -- 172.21.15.100:0/2420538513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e881609b0 msgr2=0x7f5e88160dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:14.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.289+0000 7f5e7fe00640 1 --2- 172.21.15.100:0/2420538513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e881609b0 0x7f5e88160dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:14.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.289+0000 7f5e7fe00640 1 -- 172.21.15.100:0/2420538513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e6c005350 msgr2=0x7f5e8815b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:14.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.289+0000 7f5e7fe00640 1 --2- 172.21.15.100:0/2420538513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e6c005350 0x7f5e8815b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:14.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.289+0000 7f5e7fe00640 1 -- 172.21.15.100:0/2420538513 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e88161300 con 0x7f5e88158040 2024-09-17T04:29:14.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.289+0000 7f5e87e00640 1 --2- 172.21.15.100:0/2420538513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e881609b0 0x7f5e88160dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:14.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.290+0000 7f5e7fe00640 1 --2- 172.21.15.100:0/2420538513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e88158040 0x7f5e8814e710 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f5e70011820 tx=0x7f5e70011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:14.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.290+0000 7f5e7f400640 1 -- 172.21.15.100:0/2420538513 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7000fd50 con 0x7f5e88158040 2024-09-17T04:29:14.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.290+0000 7f5e7f400640 1 -- 172.21.15.100:0/2420538513 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e70016070 con 0x7f5e88158040 2024-09-17T04:29:14.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.290+0000 7f5e7f400640 1 -- 172.21.15.100:0/2420538513 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e70012690 con 0x7f5e88158040 2024-09-17T04:29:14.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.290+0000 7f5e8d000640 1 -- 172.21.15.100:0/2420538513 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e8814ec50 con 0x7f5e88158040 2024-09-17T04:29:14.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.290+0000 7f5e8d000640 1 -- 172.21.15.100:0/2420538513 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e881f98f0 con 0x7f5e88158040 2024-09-17T04:29:14.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.291+0000 7f5e8d000640 1 -- 172.21.15.100:0/2420538513 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e88051020 con 0x7f5e88158040 2024-09-17T04:29:14.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.293+0000 7f5e7f400640 1 -- 172.21.15.100:0/2420538513 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f5e70018030 con 0x7f5e88158040 2024-09-17T04:29:14.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.293+0000 7f5e7f400640 1 --2- 172.21.15.100:0/2420538513 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5e64078cb0 0x7f5e6407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:14.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.294+0000 7f5e7f400640 1 -- 172.21.15.100:0/2420538513 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5502+0+0 (secure 0 0 0) 0x7f5e7009b770 con 0x7f5e88158040 2024-09-17T04:29:14.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.294+0000 7f5e87400640 1 --2- 172.21.15.100:0/2420538513 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5e64078cb0 0x7f5e6407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:14.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.294+0000 7f5e87400640 1 --2- 172.21.15.100:0/2420538513 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5e64078cb0 0x7f5e6407b1c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5e78006c10 tx=0x7f5e780067a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:14.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.298+0000 7f5e7f400640 1 -- 172.21.15.100:0/2420538513 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f5e88051020 con 0x7f5e88158040 2024-09-17T04:29:14.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.482+0000 7f5e8d000640 1 -- 172.21.15.100:0/2420538513 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e880d16f0 con 0x7f5e88158040 2024-09-17T04:29:14.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.483+0000 7f5e7f400640 1 -- 172.21.15.100:0/2420538513 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+13906 (secure 0 0 0) 0x7f5e88161300 con 0x7f5e88158040 2024-09-17T04:29:14.484 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:14.484 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":79,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:29:13.033147+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6835/2913442826":"2024-09-18T04:29:10.023455+0000","172.21.15.184:0/2757444911":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/4222181944":"2024-09-18T04:29:05.634928+0000","172.21.15.184:6801/4096933233":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/1536258390":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/3031882921":"2024-09-18T04:29:05.634928+0000","172.21.15.184:6800/4096933233":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/4274574405":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/2128832114":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/2777608743":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/749630281":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/2438003331":"2024-09-18T04:29:05.634928+0000","172.21.15.184:0/2967819727":"2024-09-18T04:29:05.634928+0000","172.21.15.100:6834/2913442826":"2024-09-18T04:29:10.023455+0000","172.21.15.184:0/659661481":"2024-09-18T04:29:05.634928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:29:14.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.486+0000 7f5e8d000640 1 -- 172.21.15.100:0/2420538513 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5e64078cb0 msgr2=0x7f5e6407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:14.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.486+0000 7f5e8d000640 1 --2- 172.21.15.100:0/2420538513 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5e64078cb0 0x7f5e6407b1c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5e78006c10 tx=0x7f5e780067a0 comp rx=0 tx=0).stop 2024-09-17T04:29:14.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.486+0000 7f5e8d000640 1 -- 172.21.15.100:0/2420538513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e88158040 msgr2=0x7f5e8814e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:14.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.486+0000 7f5e8d000640 1 --2- 172.21.15.100:0/2420538513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e88158040 0x7f5e8814e710 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f5e70011820 tx=0x7f5e70011ce0 comp rx=0 tx=0).stop 2024-09-17T04:29:14.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.486+0000 7f5e8d000640 1 -- 172.21.15.100:0/2420538513 shutdown_connections 2024-09-17T04:29:14.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.486+0000 7f5e8d000640 1 --2- 172.21.15.100:0/2420538513 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5e64078cb0 0x7f5e6407b1c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:14.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.486+0000 7f5e8d000640 1 --2- 172.21.15.100:0/2420538513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e881609b0 0x7f5e88160dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:14.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.486+0000 7f5e8d000640 1 --2- 172.21.15.100:0/2420538513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e88158040 0x7f5e8814e710 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:14.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.486+0000 7f5e8d000640 1 --2- 172.21.15.100:0/2420538513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e6c005350 0x7f5e8815b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:14.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.486+0000 7f5e8d000640 1 -- 172.21.15.100:0/2420538513 >> 172.21.15.100:0/2420538513 conn(0x7f5e8808bfe0 msgr2=0x7f5e880d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:14.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.486+0000 7f5e8d000640 1 -- 172.21.15.100:0/2420538513 shutdown_connections 2024-09-17T04:29:14.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.486+0000 7f5e8d000640 1 -- 172.21.15.100:0/2420538513 wait complete. 2024-09-17T04:29:14.508 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T04:24:03.995355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T04:29:14.508 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T04:29:14.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.733+0000 7fcce2200640 1 -- 172.21.15.100:0/265995427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc0004000 msgr2=0x7fccc0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:14.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.733+0000 7fcce2200640 1 --2- 172.21.15.100:0/265995427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc0004000 0x7fccc0004800 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fccdc071450 tx=0x7fccc0005830 comp rx=0 tx=0).stop 2024-09-17T04:29:14.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.733+0000 7fccdbe00640 1 -- 172.21.15.100:0/265995427 reap_dead start 2024-09-17T04:29:14.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.733+0000 7fcce2200640 1 -- 172.21.15.100:0/265995427 shutdown_connections 2024-09-17T04:29:14.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.733+0000 7fcce2200640 1 -- 172.21.15.100:0/265995427 >> 172.21.15.100:0/265995427 conn(0x7fccdc06d710 msgr2=0x7fccdc0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:14.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.733+0000 7fcce2200640 1 -- 172.21.15.100:0/265995427 shutdown_connections 2024-09-17T04:29:14.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.733+0000 7fcce2200640 1 -- 172.21.15.100:0/265995427 wait complete. 2024-09-17T04:29:14.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.735+0000 7fcce2200640 1 Processor -- start 2024-09-17T04:29:14.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.736+0000 7fcce2200640 1 -- start start 2024-09-17T04:29:14.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.737+0000 7fcce2200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc0004000 0x7fccdc1f7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:14.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.737+0000 7fcce2200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccdc146f20 0x7fccdc0d0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:14.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.737+0000 7fcce2200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccdc0d0c70 0x7fccdc144480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:14.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.737+0000 7fcce2200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fccdc15dc60 con 0x7fccc0004000 2024-09-17T04:29:14.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.737+0000 7fcce2200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fccdc15dae0 con 0x7fccdc146f20 2024-09-17T04:29:14.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.737+0000 7fcce2200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fccdc06ef70 con 0x7fccdc0d0c70 2024-09-17T04:29:14.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.737+0000 7fccdb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc0004000 0x7fccdc1f7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:14.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.737+0000 7fccdb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc0004000 0x7fccdc1f7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45438/0 (socket says 172.21.15.100:45438) 2024-09-17T04:29:14.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.737+0000 7fccdb400640 1 -- 172.21.15.100:0/1594003949 learned_addr learned my addr 172.21.15.100:0/1594003949 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:14.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.737+0000 7fccdbe00640 1 --2- 172.21.15.100:0/1594003949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccdc0d0c70 0x7fccdc144480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:14.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.738+0000 7fccd2a00640 1 --2- 172.21.15.100:0/1594003949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccdc146f20 0x7fccdc0d0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:14.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.738+0000 7fccdb400640 1 -- 172.21.15.100:0/1594003949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccdc0d0c70 msgr2=0x7fccdc144480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:14.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.738+0000 7fccdb400640 1 --2- 172.21.15.100:0/1594003949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccdc0d0c70 0x7fccdc144480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:14.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.738+0000 7fccdb400640 1 -- 172.21.15.100:0/1594003949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccdc146f20 msgr2=0x7fccdc0d0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:14.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.738+0000 7fccdb400640 1 --2- 172.21.15.100:0/1594003949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccdc146f20 0x7fccdc0d0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:14.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.738+0000 7fccdb400640 1 -- 172.21.15.100:0/1594003949 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccdc157570 con 0x7fccc0004000 2024-09-17T04:29:14.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.738+0000 7fccdb400640 1 --2- 172.21.15.100:0/1594003949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc0004000 0x7fccdc1f7380 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fcccc0066b0 tx=0x7fcccc0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:14.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.738+0000 7fccd3e00640 1 -- 172.21.15.100:0/1594003949 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc011700 con 0x7fccc0004000 2024-09-17T04:29:14.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.738+0000 7fccd3e00640 1 -- 172.21.15.100:0/1594003949 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcccc016020 con 0x7fccc0004000 2024-09-17T04:29:14.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.738+0000 7fcce2200640 1 -- 172.21.15.100:0/1594003949 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccdc1f68a0 con 0x7fccc0004000 2024-09-17T04:29:14.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.738+0000 7fccd3e00640 1 -- 172.21.15.100:0/1594003949 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc011b60 con 0x7fccc0004000 2024-09-17T04:29:14.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.738+0000 7fcce2200640 1 -- 172.21.15.100:0/1594003949 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccdc1f6e60 con 0x7fccc0004000 2024-09-17T04:29:14.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.739+0000 7fcce2200640 1 -- 172.21.15.100:0/1594003949 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccdc06f4d0 con 0x7fccc0004000 2024-09-17T04:29:14.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.741+0000 7fccd3e00640 1 -- 172.21.15.100:0/1594003949 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fcccc011d00 con 0x7fccc0004000 2024-09-17T04:29:14.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.741+0000 7fccd3e00640 1 --2- 172.21.15.100:0/1594003949 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fccb8078cb0 0x7fccb807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:14.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.742+0000 7fccd3e00640 1 -- 172.21.15.100:0/1594003949 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5502+0+0 (secure 0 0 0) 0x7fcccc09b9e0 con 0x7fccc0004000 2024-09-17T04:29:14.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.742+0000 7fccd2a00640 1 --2- 172.21.15.100:0/1594003949 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fccb8078cb0 0x7fccb807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:14.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.742+0000 7fccd2a00640 1 --2- 172.21.15.100:0/1594003949 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fccb8078cb0 0x7fccb807b1c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fccc4035000 tx=0x7fccc40046c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:14.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.746+0000 7fccd3e00640 1 -- 172.21.15.100:0/1594003949 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fcccc020020 con 0x7fccc0004000 2024-09-17T04:29:14.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.928+0000 7fcce2200640 1 -- 172.21.15.100:0/1594003949 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fccdc14a790 con 0x7fccc0004000 2024-09-17T04:29:14.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.929+0000 7fccd3e00640 1 -- 172.21.15.100:0/1594003949 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7fcccc063ce0 con 0x7fccc0004000 2024-09-17T04:29:14.929 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-09-17T04:29:14.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.931+0000 7fcce2200640 1 -- 172.21.15.100:0/1594003949 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fccb8078cb0 msgr2=0x7fccb807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:14.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.931+0000 7fcce2200640 1 --2- 172.21.15.100:0/1594003949 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fccb8078cb0 0x7fccb807b1c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fccc4035000 tx=0x7fccc40046c0 comp rx=0 tx=0).stop 2024-09-17T04:29:14.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.931+0000 7fcce2200640 1 -- 172.21.15.100:0/1594003949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc0004000 msgr2=0x7fccdc1f7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:14.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.931+0000 7fcce2200640 1 --2- 172.21.15.100:0/1594003949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc0004000 0x7fccdc1f7380 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fcccc0066b0 tx=0x7fcccc0039c0 comp rx=0 tx=0).stop 2024-09-17T04:29:14.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.931+0000 7fcce2200640 1 -- 172.21.15.100:0/1594003949 shutdown_connections 2024-09-17T04:29:14.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.931+0000 7fcce2200640 1 --2- 172.21.15.100:0/1594003949 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fccb8078cb0 0x7fccb807b1c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:14.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.931+0000 7fcce2200640 1 --2- 172.21.15.100:0/1594003949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccdc0d0c70 0x7fccdc144480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:14.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.931+0000 7fcce2200640 1 --2- 172.21.15.100:0/1594003949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccdc146f20 0x7fccdc0d0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:14.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.931+0000 7fcce2200640 1 --2- 172.21.15.100:0/1594003949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc0004000 0x7fccdc1f7380 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:14.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.931+0000 7fcce2200640 1 -- 172.21.15.100:0/1594003949 >> 172.21.15.100:0/1594003949 conn(0x7fccdc06d710 msgr2=0x7fccdc156910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:14.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.931+0000 7fcce2200640 1 -- 172.21.15.100:0/1594003949 shutdown_connections 2024-09-17T04:29:14.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:14.931+0000 7fcce2200640 1 -- 172.21.15.100:0/1594003949 wait complete. 2024-09-17T04:29:14.953 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2024-09-17T04:29:15.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.175+0000 7f7895e00640 1 -- 172.21.15.100:0/1752969032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7874004800 msgr2=0x7f7874004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:15.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.175+0000 7f7895e00640 1 --2- 172.21.15.100:0/1752969032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7874004800 0x7f7874004c10 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f787c00c310 tx=0x7f7874005830 comp rx=0 tx=0).stop 2024-09-17T04:29:15.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.175+0000 7f788f400640 1 -- 172.21.15.100:0/1752969032 reap_dead start 2024-09-17T04:29:15.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.175+0000 7f7895e00640 1 -- 172.21.15.100:0/1752969032 shutdown_connections 2024-09-17T04:29:15.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.176+0000 7f7895e00640 1 -- 172.21.15.100:0/1752969032 >> 172.21.15.100:0/1752969032 conn(0x7f789006d730 msgr2=0x7f78900b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:15.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.176+0000 7f7895e00640 1 -- 172.21.15.100:0/1752969032 shutdown_connections 2024-09-17T04:29:15.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.176+0000 7f7895e00640 1 -- 172.21.15.100:0/1752969032 wait complete. 2024-09-17T04:29:15.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.178+0000 7f7895e00640 1 Processor -- start 2024-09-17T04:29:15.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.178+0000 7f7895e00640 1 -- start start 2024-09-17T04:29:15.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.179+0000 7f7895e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7874004800 0x7f78900cde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:15.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.179+0000 7f7895e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7890152ec0 0x7f78900998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:15.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.179+0000 7f7895e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78901f4280 0x7f78901f4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:15.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.179+0000 7f7895e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f78900b3fe0 con 0x7f7890152ec0 2024-09-17T04:29:15.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.179+0000 7f7895e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78900b3e60 con 0x7f78901f4280 2024-09-17T04:29:15.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.179+0000 7f7895e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f78900b4160 con 0x7f7874004800 2024-09-17T04:29:15.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.180+0000 7f788e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7890152ec0 0x7f78900998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:15.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.180+0000 7f788e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7890152ec0 0x7f78900998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51708/0 (socket says 172.21.15.100:51708) 2024-09-17T04:29:15.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.180+0000 7f788e000640 1 -- 172.21.15.100:0/3578185131 learned_addr learned my addr 172.21.15.100:0/3578185131 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:15.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.180+0000 7f788ea00640 1 --2- 172.21.15.100:0/3578185131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7874004800 0x7f78900cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:15.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.180+0000 7f788f400640 1 --2- 172.21.15.100:0/3578185131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78901f4280 0x7f78901f4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:15.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.180+0000 7f788e000640 1 -- 172.21.15.100:0/3578185131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7874004800 msgr2=0x7f78900cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:15.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.180+0000 7f788e000640 1 --2- 172.21.15.100:0/3578185131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7874004800 0x7f78900cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:15.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.180+0000 7f788e000640 1 -- 172.21.15.100:0/3578185131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78901f4280 msgr2=0x7f78901f4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:15.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.180+0000 7f788e000640 1 --2- 172.21.15.100:0/3578185131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78901f4280 0x7f78901f4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:15.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.180+0000 7f788e000640 1 -- 172.21.15.100:0/3578185131 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78901f4bd0 con 0x7f7890152ec0 2024-09-17T04:29:15.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.180+0000 7f788e000640 1 --2- 172.21.15.100:0/3578185131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7890152ec0 0x7f78900998a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f787c02b170 tx=0x7f787c001d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:15.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.181+0000 7f7887400640 1 -- 172.21.15.100:0/3578185131 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c031040 con 0x7f7890152ec0 2024-09-17T04:29:15.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.181+0000 7f7887400640 1 -- 172.21.15.100:0/3578185131 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f787c0035f0 con 0x7f7890152ec0 2024-09-17T04:29:15.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.181+0000 7f7895e00640 1 -- 172.21.15.100:0/3578185131 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78900ce370 con 0x7f7890152ec0 2024-09-17T04:29:15.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.181+0000 7f7887400640 1 -- 172.21.15.100:0/3578185131 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c003a20 con 0x7f7890152ec0 2024-09-17T04:29:15.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.181+0000 7f7895e00640 1 -- 172.21.15.100:0/3578185131 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7890153840 con 0x7f7890152ec0 2024-09-17T04:29:15.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.182+0000 7f7895e00640 1 -- 172.21.15.100:0/3578185131 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78900b46f0 con 0x7f7890152ec0 2024-09-17T04:29:15.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.183+0000 7f7887400640 1 -- 172.21.15.100:0/3578185131 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f787c03e030 con 0x7f7890152ec0 2024-09-17T04:29:15.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.184+0000 7f7887400640 1 --2- 172.21.15.100:0/3578185131 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7870078ce0 0x7f787007b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:15.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.184+0000 7f7887400640 1 -- 172.21.15.100:0/3578185131 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5502+0+0 (secure 0 0 0) 0x7f787c0bac80 con 0x7f7890152ec0 2024-09-17T04:29:15.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.184+0000 7f788ea00640 1 --2- 172.21.15.100:0/3578185131 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7870078ce0 0x7f787007b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:15.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.185+0000 7f788ea00640 1 --2- 172.21.15.100:0/3578185131 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7870078ce0 0x7f787007b1f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7880009c50 tx=0x7f7880005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:15.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.188+0000 7f7887400640 1 -- 172.21.15.100:0/3578185131 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f78901f4bd0 con 0x7f7890152ec0 2024-09-17T04:29:15.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:15.442+0000 7f7895e00640 1 -- 172.21.15.100:0/3578185131 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) -- 0x7f78901532f0 con 0x7f7890152ec0 2024-09-17T04:29:16.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.044+0000 7f7887400640 1 -- 172.21.15.100:0/3578185131 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v288) ==== 149+0+0 (secure 0 0 0) 0x7f787c082880 con 0x7f7890152ec0 2024-09-17T04:29:16.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.046+0000 7f7895e00640 1 -- 172.21.15.100:0/3578185131 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7870078ce0 msgr2=0x7f787007b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:16.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.046+0000 7f7895e00640 1 --2- 172.21.15.100:0/3578185131 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7870078ce0 0x7f787007b1f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7880009c50 tx=0x7f7880005f90 comp rx=0 tx=0).stop 2024-09-17T04:29:16.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.046+0000 7f7895e00640 1 -- 172.21.15.100:0/3578185131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7890152ec0 msgr2=0x7f78900998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:16.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.046+0000 7f7895e00640 1 --2- 172.21.15.100:0/3578185131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7890152ec0 0x7f78900998a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f787c02b170 tx=0x7f787c001d90 comp rx=0 tx=0).stop 2024-09-17T04:29:16.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.047+0000 7f7895e00640 1 -- 172.21.15.100:0/3578185131 shutdown_connections 2024-09-17T04:29:16.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.047+0000 7f7895e00640 1 --2- 172.21.15.100:0/3578185131 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7870078ce0 0x7f787007b1f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:16.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.047+0000 7f7895e00640 1 --2- 172.21.15.100:0/3578185131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78901f4280 0x7f78901f4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:16.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.047+0000 7f7895e00640 1 --2- 172.21.15.100:0/3578185131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7890152ec0 0x7f78900998a0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:16.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.047+0000 7f7895e00640 1 --2- 172.21.15.100:0/3578185131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7874004800 0x7f78900cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:16.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.047+0000 7f7895e00640 1 -- 172.21.15.100:0/3578185131 >> 172.21.15.100:0/3578185131 conn(0x7f789006d730 msgr2=0x7f789015df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:16.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.047+0000 7f7895e00640 1 -- 172.21.15.100:0/3578185131 shutdown_connections 2024-09-17T04:29:16.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.047+0000 7f7895e00640 1 -- 172.21.15.100:0/3578185131 wait complete. 2024-09-17T04:29:16.074 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T04:29:16.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.297+0000 7f5922a00640 1 -- 172.21.15.100:0/2062323672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5900003c10 msgr2=0x7f5900004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:16.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.297+0000 7f5922a00640 1 --2- 172.21.15.100:0/2062323672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5900003c10 0x7f5900004060 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f590c009ed0 tx=0x7f5900005830 comp rx=0 tx=0).stop 2024-09-17T04:29:16.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.297+0000 7f5920a00640 1 -- 172.21.15.100:0/2062323672 reap_dead start 2024-09-17T04:29:16.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.297+0000 7f5922a00640 1 -- 172.21.15.100:0/2062323672 shutdown_connections 2024-09-17T04:29:16.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.297+0000 7f5922a00640 1 -- 172.21.15.100:0/2062323672 >> 172.21.15.100:0/2062323672 conn(0x7f591c06d730 msgr2=0x7f591c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:16.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.297+0000 7f5922a00640 1 -- 172.21.15.100:0/2062323672 shutdown_connections 2024-09-17T04:29:16.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.297+0000 7f5922a00640 1 -- 172.21.15.100:0/2062323672 wait complete. 2024-09-17T04:29:16.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.300+0000 7f5922a00640 1 Processor -- start 2024-09-17T04:29:16.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.300+0000 7f5922a00640 1 -- start start 2024-09-17T04:29:16.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.301+0000 7f5922a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5900003c10 0x7f591c1f99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:16.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.301+0000 7f5922a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f591c074520 0x7f591c14a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:16.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.301+0000 7f5922a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f591c1519d0 0x7f591c09a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:16.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.301+0000 7f5922a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f591c0b41f0 con 0x7f591c1519d0 2024-09-17T04:29:16.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.301+0000 7f5922a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f591c0b4070 con 0x7f5900003c10 2024-09-17T04:29:16.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.301+0000 7f5922a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f591c0b4370 con 0x7f591c074520 2024-09-17T04:29:16.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.302+0000 7f5920a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f591c1519d0 0x7f591c09a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:16.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.302+0000 7f5920a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f591c1519d0 0x7f591c09a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51714/0 (socket says 172.21.15.100:51714) 2024-09-17T04:29:16.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.302+0000 7f5920a00640 1 -- 172.21.15.100:0/2926519361 learned_addr learned my addr 172.21.15.100:0/2926519361 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:16.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.302+0000 7f591b400640 1 --2- 172.21.15.100:0/2926519361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f591c074520 0x7f591c14a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:16.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.302+0000 7f591be00640 1 --2- 172.21.15.100:0/2926519361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5900003c10 0x7f591c1f99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:16.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.302+0000 7f5920a00640 1 -- 172.21.15.100:0/2926519361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f591c074520 msgr2=0x7f591c14a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:16.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.302+0000 7f5920a00640 1 --2- 172.21.15.100:0/2926519361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f591c074520 0x7f591c14a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:16.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.302+0000 7f5920a00640 1 -- 172.21.15.100:0/2926519361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5900003c10 msgr2=0x7f591c1f99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:16.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.302+0000 7f5920a00640 1 --2- 172.21.15.100:0/2926519361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5900003c10 0x7f591c1f99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:16.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.302+0000 7f5920a00640 1 -- 172.21.15.100:0/2926519361 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f591c1f8f20 con 0x7f591c1519d0 2024-09-17T04:29:16.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.302+0000 7f591be00640 1 --2- 172.21.15.100:0/2926519361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5900003c10 0x7f591c1f99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:16.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.302+0000 7f591b400640 1 --2- 172.21.15.100:0/2926519361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f591c074520 0x7f591c14a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:16.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.302+0000 7f5920a00640 1 --2- 172.21.15.100:0/2926519361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f591c1519d0 0x7f591c09a2d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f591c075250 tx=0x7f591400e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:16.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.302+0000 7f5913e00640 1 -- 172.21.15.100:0/2926519361 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5914011660 con 0x7f591c1519d0 2024-09-17T04:29:16.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.303+0000 7f5913e00640 1 -- 172.21.15.100:0/2926519361 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5914016070 con 0x7f591c1519d0 2024-09-17T04:29:16.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.303+0000 7f5913e00640 1 -- 172.21.15.100:0/2926519361 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5914011bb0 con 0x7f591c1519d0 2024-09-17T04:29:16.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.303+0000 7f5922a00640 1 -- 172.21.15.100:0/2926519361 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f591c074c70 con 0x7f591c1519d0 2024-09-17T04:29:16.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.303+0000 7f5922a00640 1 -- 172.21.15.100:0/2926519361 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f591c1f91e0 con 0x7f591c1519d0 2024-09-17T04:29:16.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.304+0000 7f5922a00640 1 -- 172.21.15.100:0/2926519361 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f591c070430 con 0x7f591c1519d0 2024-09-17T04:29:16.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.305+0000 7f5913e00640 1 -- 172.21.15.100:0/2926519361 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f591401b460 con 0x7f591c1519d0 2024-09-17T04:29:16.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.307+0000 7f5913e00640 1 --2- 172.21.15.100:0/2926519361 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f58fc078cb0 0x7f58fc07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:16.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.307+0000 7f5913e00640 1 -- 172.21.15.100:0/2926519361 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5502+0+0 (secure 0 0 0) 0x7f591409b180 con 0x7f591c1519d0 2024-09-17T04:29:16.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.307+0000 7f591be00640 1 --2- 172.21.15.100:0/2926519361 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f58fc078cb0 0x7f58fc07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:16.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.307+0000 7f591be00640 1 --2- 172.21.15.100:0/2926519361 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f58fc078cb0 0x7f58fc07b1c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f590c035000 tx=0x7f590c004a40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:16.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.311+0000 7f5913e00640 1 -- 172.21.15.100:0/2926519361 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f591c070430 con 0x7f591c1519d0 2024-09-17T04:29:16.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.493+0000 7f5922a00640 1 -- 172.21.15.100:0/2926519361 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f591c0b3960 con 0x7f591c1519d0 2024-09-17T04:29:16.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.494+0000 7f5913e00640 1 -- 172.21.15.100:0/2926519361 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v39) ==== 59+0+2294 (secure 0 0 0) 0x7f5914063500 con 0x7f591c1519d0 2024-09-17T04:29:16.494 INFO:teuthology.orchestra.run.smithi100.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T04:29:16.494 INFO:teuthology.orchestra.run.smithi100.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T04:29:16.494 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced client_mount_timeout 30 2024-09-17T04:29:16.495 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_client 20/20 2024-09-17T04:29:16.495 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_mgr 20/20 2024-09-17T04:29:16.495 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_ms 1/1 2024-09-17T04:29:16.495 INFO:teuthology.orchestra.run.smithi100.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T04:29:16.495 INFO:teuthology.orchestra.run.smithi100.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T04:29:16.495 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3938.598654 2024-09-17T04:29:16.495 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4008.640154 2024-09-17T04:29:16.495 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3706.588546 2024-09-17T04:29:16.495 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 6299.338665 2024-09-17T04:29:16.495 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3633.876699 2024-09-17T04:29:16.495 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3674.821841 2024-09-17T04:29:16.495 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3789.849984 2024-09-17T04:29:16.495 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3467.457298 2024-09-17T04:29:16.495 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_mds 20/20 2024-09-17T04:29:16.496 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T04:29:16.496 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_ms 2/2 2024-09-17T04:29:16.496 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_debug_frag true 2024-09-17T04:29:16.496 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_debug_scatterstat true 2024-09-17T04:29:16.496 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T04:29:16.496 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_verify_scatter true 2024-09-17T04:29:16.496 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T04:29:16.496 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T04:29:16.496 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T04:29:16.496 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced client_mount_timeout 600 2024-09-17T04:29:16.496 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced debug_client 20/20 2024-09-17T04:29:16.496 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced debug_ms 1/1 2024-09-17T04:29:16.496 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T04:29:16.497 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T04:29:16.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.496+0000 7f5922a00640 1 -- 172.21.15.100:0/2926519361 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f58fc078cb0 msgr2=0x7f58fc07b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:16.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.496+0000 7f5922a00640 1 --2- 172.21.15.100:0/2926519361 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f58fc078cb0 0x7f58fc07b1c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f590c035000 tx=0x7f590c004a40 comp rx=0 tx=0).stop 2024-09-17T04:29:16.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.496+0000 7f5922a00640 1 -- 172.21.15.100:0/2926519361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f591c1519d0 msgr2=0x7f591c09a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:16.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.496+0000 7f5922a00640 1 --2- 172.21.15.100:0/2926519361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f591c1519d0 0x7f591c09a2d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f591c075250 tx=0x7f591400e6b0 comp rx=0 tx=0).stop 2024-09-17T04:29:16.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.496+0000 7f5922a00640 1 -- 172.21.15.100:0/2926519361 shutdown_connections 2024-09-17T04:29:16.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.496+0000 7f5922a00640 1 --2- 172.21.15.100:0/2926519361 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f58fc078cb0 0x7f58fc07b1c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:16.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.496+0000 7f5922a00640 1 --2- 172.21.15.100:0/2926519361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f591c1519d0 0x7f591c09a2d0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:16.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.496+0000 7f5922a00640 1 --2- 172.21.15.100:0/2926519361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f591c074520 0x7f591c14a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:16.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.496+0000 7f5922a00640 1 --2- 172.21.15.100:0/2926519361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5900003c10 0x7f591c1f99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:16.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.496+0000 7f5922a00640 1 -- 172.21.15.100:0/2926519361 >> 172.21.15.100:0/2926519361 conn(0x7f591c06d730 msgr2=0x7f591c15efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:16.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.496+0000 7f5922a00640 1 -- 172.21.15.100:0/2926519361 shutdown_connections 2024-09-17T04:29:16.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.496+0000 7f5922a00640 1 -- 172.21.15.100:0/2926519361 wait complete. 2024-09-17T04:29:16.522 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T04:29:16.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.746+0000 7f10a5600640 1 -- 172.21.15.100:0/3670606606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1084004800 msgr2=0x7f1084004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:16.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.746+0000 7f10a5600640 1 --2- 172.21.15.100:0/3670606606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1084004800 0x7f1084004c10 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1088006f60 tx=0x7f1084005830 comp rx=0 tx=0).stop 2024-09-17T04:29:16.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.746+0000 7f109f400640 1 -- 172.21.15.100:0/3670606606 reap_dead start 2024-09-17T04:29:16.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.747+0000 7f10a5600640 1 -- 172.21.15.100:0/3670606606 shutdown_connections 2024-09-17T04:29:16.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.747+0000 7f10a5600640 1 -- 172.21.15.100:0/3670606606 >> 172.21.15.100:0/3670606606 conn(0x7f10a006d1c0 msgr2=0x7f10a006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:16.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.747+0000 7f10a5600640 1 -- 172.21.15.100:0/3670606606 shutdown_connections 2024-09-17T04:29:16.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.747+0000 7f10a5600640 1 -- 172.21.15.100:0/3670606606 wait complete. 2024-09-17T04:29:16.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.749+0000 7f10a5600640 1 Processor -- start 2024-09-17T04:29:16.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.749+0000 7f10a5600640 1 -- start start 2024-09-17T04:29:16.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.750+0000 7f10a5600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1084004800 0x7f10a0132890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:16.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.750+0000 7f10a5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a00b0950 0x7f10a00b2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:16.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.750+0000 7f10a5600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10a0144380 0x7f10a0144790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:16.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.750+0000 7f10a5600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f10a0165490 con 0x7f10a00b0950 2024-09-17T04:29:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.750+0000 7f10a5600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f10a00ce880 con 0x7f10a0144380 2024-09-17T04:29:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.750+0000 7f10a5600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f10a0165610 con 0x7f1084004800 2024-09-17T04:29:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.751+0000 7f1097e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a00b0950 0x7f10a00b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.751+0000 7f1097e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a00b0950 0x7f10a00b2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51744/0 (socket says 172.21.15.100:51744) 2024-09-17T04:29:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.751+0000 7f1097e00640 1 -- 172.21.15.100:0/2336810594 learned_addr learned my addr 172.21.15.100:0/2336810594 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.751+0000 7f109f400640 1 --2- 172.21.15.100:0/2336810594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10a0144380 0x7f10a0144790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.751+0000 7f109ea00640 1 --2- 172.21.15.100:0/2336810594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1084004800 0x7f10a0132890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.751+0000 7f1097e00640 1 -- 172.21.15.100:0/2336810594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1084004800 msgr2=0x7f10a0132890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.751+0000 7f1097e00640 1 --2- 172.21.15.100:0/2336810594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1084004800 0x7f10a0132890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.751+0000 7f1097e00640 1 -- 172.21.15.100:0/2336810594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10a0144380 msgr2=0x7f10a0144790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.751+0000 7f1097e00640 1 --2- 172.21.15.100:0/2336810594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10a0144380 0x7f10a0144790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.751+0000 7f1097e00640 1 -- 172.21.15.100:0/2336810594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a0144cd0 con 0x7f10a00b0950 2024-09-17T04:29:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.751+0000 7f109f400640 1 --2- 172.21.15.100:0/2336810594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10a0144380 0x7f10a0144790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:16.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.752+0000 7f1097e00640 1 --2- 172.21.15.100:0/2336810594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a00b0950 0x7f10a00b2490 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1088031040 tx=0x7f1088006ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:16.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.752+0000 7f1096a00640 1 -- 172.21.15.100:0/2336810594 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1088032030 con 0x7f10a00b0950 2024-09-17T04:29:16.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.752+0000 7f1096a00640 1 -- 172.21.15.100:0/2336810594 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1088003af0 con 0x7f10a00b0950 2024-09-17T04:29:16.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.752+0000 7f10a5600640 1 -- 172.21.15.100:0/2336810594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a00b2d70 con 0x7f10a00b0950 2024-09-17T04:29:16.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.752+0000 7f1096a00640 1 -- 172.21.15.100:0/2336810594 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1088042480 con 0x7f10a00b0950 2024-09-17T04:29:16.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.752+0000 7f10a5600640 1 -- 172.21.15.100:0/2336810594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a01f8e20 con 0x7f10a00b0950 2024-09-17T04:29:16.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.753+0000 7f10a5600640 1 -- 172.21.15.100:0/2336810594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10a006fb40 con 0x7f10a00b0950 2024-09-17T04:29:16.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.754+0000 7f1096a00640 1 -- 172.21.15.100:0/2336810594 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f1088035020 con 0x7f10a00b0950 2024-09-17T04:29:16.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.755+0000 7f1096a00640 1 --2- 172.21.15.100:0/2336810594 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f107c078cb0 0x7f107c07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:16.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.755+0000 7f1096a00640 1 -- 172.21.15.100:0/2336810594 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5502+0+0 (secure 0 0 0) 0x7f10880bb930 con 0x7f10a00b0950 2024-09-17T04:29:16.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.755+0000 7f109ea00640 1 --2- 172.21.15.100:0/2336810594 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f107c078cb0 0x7f107c07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:16.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.756+0000 7f109ea00640 1 --2- 172.21.15.100:0/2336810594 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f107c078cb0 0x7f107c07b1c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1090006560 tx=0x7f1090006b10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:16.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.760+0000 7f1096a00640 1 -- 172.21.15.100:0/2336810594 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f10a0144cd0 con 0x7f10a00b0950 2024-09-17T04:29:16.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.945+0000 7f10a5600640 1 -- 172.21.15.100:0/2336810594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f10a013d610 con 0x7f10a00b0950 2024-09-17T04:29:16.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.949+0000 7f1096a00640 1 -- 172.21.15.100:0/2336810594 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) ==== 122+0+0 (secure 0 0 0) 0x7f1088083b80 con 0x7f10a00b0950 2024-09-17T04:29:16.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.951+0000 7f10a5600640 1 -- 172.21.15.100:0/2336810594 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f107c078cb0 msgr2=0x7f107c07b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:16.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.951+0000 7f10a5600640 1 --2- 172.21.15.100:0/2336810594 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f107c078cb0 0x7f107c07b1c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1090006560 tx=0x7f1090006b10 comp rx=0 tx=0).stop 2024-09-17T04:29:16.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.951+0000 7f10a5600640 1 -- 172.21.15.100:0/2336810594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a00b0950 msgr2=0x7f10a00b2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:16.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.951+0000 7f10a5600640 1 --2- 172.21.15.100:0/2336810594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a00b0950 0x7f10a00b2490 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1088031040 tx=0x7f1088006ca0 comp rx=0 tx=0).stop 2024-09-17T04:29:16.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.951+0000 7f10a5600640 1 -- 172.21.15.100:0/2336810594 shutdown_connections 2024-09-17T04:29:16.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.951+0000 7f10a5600640 1 --2- 172.21.15.100:0/2336810594 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f107c078cb0 0x7f107c07b1c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:16.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.951+0000 7f10a5600640 1 --2- 172.21.15.100:0/2336810594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10a0144380 0x7f10a0144790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:16.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.951+0000 7f10a5600640 1 --2- 172.21.15.100:0/2336810594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a00b0950 0x7f10a00b2490 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:16.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.951+0000 7f10a5600640 1 --2- 172.21.15.100:0/2336810594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1084004800 0x7f10a0132890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:16.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.951+0000 7f10a5600640 1 -- 172.21.15.100:0/2336810594 >> 172.21.15.100:0/2336810594 conn(0x7f10a006d1c0 msgr2=0x7f10a0140780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:16.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.951+0000 7f10a5600640 1 -- 172.21.15.100:0/2336810594 shutdown_connections 2024-09-17T04:29:16.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:16.951+0000 7f10a5600640 1 -- 172.21.15.100:0/2336810594 wait complete. 2024-09-17T04:29:16.974 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T04:29:16.974 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:29:17.068 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:29:17.174 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T04:29:17.174 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:29:17.174 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:29:17.210 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:29:17.210 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:29:17.210 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:29:17.247 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:29:17.248 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:29:17.248 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:29:17.286 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:29:17.286 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:29:17.287 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:29:17.323 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:29:17.323 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T04:29:17.324 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:29:17.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.545+0000 7fca6fc00640 1 -- 172.21.15.100:0/2761795219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca6815d760 msgr2=0x7fca681535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:17.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.545+0000 7fca6fc00640 1 --2- 172.21.15.100:0/2761795219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca6815d760 0x7fca681535c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fca58005fd0 tx=0x7fca5802a540 comp rx=0 tx=0).stop 2024-09-17T04:29:17.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.545+0000 7fca6fc00640 1 -- 172.21.15.100:0/2761795219 shutdown_connections 2024-09-17T04:29:17.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.545+0000 7fca6fc00640 1 --2- 172.21.15.100:0/2761795219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca680ad0e0 0x7fca680ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:17.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.545+0000 7fca6fc00640 1 --2- 172.21.15.100:0/2761795219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca6815d760 0x7fca681535c0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:17.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.545+0000 7fca6fc00640 1 -- 172.21.15.100:0/2761795219 >> 172.21.15.100:0/2761795219 conn(0x7fca6806d730 msgr2=0x7fca680b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:17.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.545+0000 7fca6fc00640 1 -- 172.21.15.100:0/2761795219 shutdown_connections 2024-09-17T04:29:17.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.545+0000 7fca6fc00640 1 -- 172.21.15.100:0/2761795219 wait complete. 2024-09-17T04:29:17.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.547+0000 7fca6fc00640 1 Processor -- start 2024-09-17T04:29:17.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.547+0000 7fca6fc00640 1 -- start start 2024-09-17T04:29:17.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.549+0000 7fca6fc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca680ad0e0 0x7fca68156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:17.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.549+0000 7fca6fc00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca6815d760 0x7fca6809d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:17.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.549+0000 7fca6fc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca68160310 0x7fca6815b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:17.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.549+0000 7fca6fc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fca6806f220 con 0x7fca68160310 2024-09-17T04:29:17.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.549+0000 7fca6fc00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fca6806f0a0 con 0x7fca680ad0e0 2024-09-17T04:29:17.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.549+0000 7fca6fc00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fca680b4c20 con 0x7fca6815d760 2024-09-17T04:29:17.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.549+0000 7fca6dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca68160310 0x7fca6815b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:17.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.549+0000 7fca6d200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca680ad0e0 0x7fca68156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:17.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.549+0000 7fca6dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca68160310 0x7fca6815b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51750/0 (socket says 172.21.15.100:51750) 2024-09-17T04:29:17.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.549+0000 7fca6dc00640 1 -- 172.21.15.100:0/1526265693 learned_addr learned my addr 172.21.15.100:0/1526265693 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:17.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.549+0000 7fca6d200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca680ad0e0 0x7fca68156320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.100:43718/0 (socket says 172.21.15.100:43718) 2024-09-17T04:29:17.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.549+0000 7fca67e00640 1 --2- 172.21.15.100:0/1526265693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca6815d760 0x7fca6809d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:17.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.550+0000 7fca6dc00640 1 -- 172.21.15.100:0/1526265693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca6815d760 msgr2=0x7fca6809d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:17.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.550+0000 7fca6dc00640 1 --2- 172.21.15.100:0/1526265693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca6815d760 0x7fca6809d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:17.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.550+0000 7fca6dc00640 1 -- 172.21.15.100:0/1526265693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca680ad0e0 msgr2=0x7fca68156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:17.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.550+0000 7fca6dc00640 1 --2- 172.21.15.100:0/1526265693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca680ad0e0 0x7fca68156320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:17.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.550+0000 7fca6dc00640 1 -- 172.21.15.100:0/1526265693 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca681f5a70 con 0x7fca68160310 2024-09-17T04:29:17.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.550+0000 7fca6dc00640 1 --2- 172.21.15.100:0/1526265693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca68160310 0x7fca6815b7f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fca6000c730 tx=0x7fca6000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:17.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.550+0000 7fca64c00640 1 -- 172.21.15.100:0/1526265693 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca60003b70 con 0x7fca68160310 2024-09-17T04:29:17.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.550+0000 7fca64c00640 1 -- 172.21.15.100:0/1526265693 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca60003d10 con 0x7fca68160310 2024-09-17T04:29:17.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.550+0000 7fca64c00640 1 -- 172.21.15.100:0/1526265693 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca60005330 con 0x7fca68160310 2024-09-17T04:29:17.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.550+0000 7fca6fc00640 1 -- 172.21.15.100:0/1526265693 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca68156b70 con 0x7fca68160310 2024-09-17T04:29:17.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.550+0000 7fca6fc00640 1 -- 172.21.15.100:0/1526265693 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca681f5fd0 con 0x7fca68160310 2024-09-17T04:29:17.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.551+0000 7fca6fc00640 1 -- 172.21.15.100:0/1526265693 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca68160fa0 con 0x7fca68160310 2024-09-17T04:29:17.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.552+0000 7fca64c00640 1 -- 172.21.15.100:0/1526265693 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fca600054d0 con 0x7fca68160310 2024-09-17T04:29:17.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.553+0000 7fca64c00640 1 --2- 172.21.15.100:0/1526265693 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fca48078cb0 0x7fca4807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:17.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.553+0000 7fca64c00640 1 -- 172.21.15.100:0/1526265693 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5502+0+0 (secure 0 0 0) 0x7fca6009ab60 con 0x7fca68160310 2024-09-17T04:29:17.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.554+0000 7fca6d200640 1 --2- 172.21.15.100:0/1526265693 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fca48078cb0 0x7fca4807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:17.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.554+0000 7fca6d200640 1 --2- 172.21.15.100:0/1526265693 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fca48078cb0 0x7fca4807b1c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fca58033000 tx=0x7fca58004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:17.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.558+0000 7fca64c00640 1 -- 172.21.15.100:0/1526265693 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fca681f5a70 con 0x7fca68160310 2024-09-17T04:29:17.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.776+0000 7fca6fc00640 1 -- 172.21.15.100:0/1526265693 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca68156900 con 0x7fca68160310 2024-09-17T04:29:17.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.777+0000 7fca64c00640 1 -- 172.21.15.100:0/1526265693 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+3920 (secure 0 0 0) 0x7fca680ada30 con 0x7fca68160310 2024-09-17T04:29:17.777 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:17.778 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":50,"btime":"2024-09-17T04:29:11:025641+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5513,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2569013403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2569013403},{"type":"v1","addr":"172.21.15.184:6835","nonce":2569013403}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5519,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3475403612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3475403612},{"type":"v1","addr":"172.21.15.184:6837","nonce":3475403612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6156,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2177833369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2177833369},{"type":"v1","addr":"172.21.15.100:6833","nonce":2177833369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6516,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/407009320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":407009320},{"type":"v1","addr":"172.21.15.100:6835","nonce":407009320}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[]} 2024-09-17T04:29:17.778 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 50 2024-09-17T04:29:17.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.779+0000 7fca6fc00640 1 -- 172.21.15.100:0/1526265693 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fca48078cb0 msgr2=0x7fca4807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:17.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.779+0000 7fca6fc00640 1 --2- 172.21.15.100:0/1526265693 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fca48078cb0 0x7fca4807b1c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fca58033000 tx=0x7fca58004730 comp rx=0 tx=0).stop 2024-09-17T04:29:17.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.779+0000 7fca6fc00640 1 -- 172.21.15.100:0/1526265693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca68160310 msgr2=0x7fca6815b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:17.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.779+0000 7fca6fc00640 1 --2- 172.21.15.100:0/1526265693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca68160310 0x7fca6815b7f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fca6000c730 tx=0x7fca6000cb30 comp rx=0 tx=0).stop 2024-09-17T04:29:17.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.779+0000 7fca6fc00640 1 -- 172.21.15.100:0/1526265693 shutdown_connections 2024-09-17T04:29:17.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.779+0000 7fca6fc00640 1 --2- 172.21.15.100:0/1526265693 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fca48078cb0 0x7fca4807b1c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:17.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.779+0000 7fca6fc00640 1 --2- 172.21.15.100:0/1526265693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca68160310 0x7fca6815b7f0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:17.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.779+0000 7fca6fc00640 1 --2- 172.21.15.100:0/1526265693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca6815d760 0x7fca6809d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:17.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.779+0000 7fca6fc00640 1 --2- 172.21.15.100:0/1526265693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca680ad0e0 0x7fca68156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:17.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.780+0000 7fca6fc00640 1 -- 172.21.15.100:0/1526265693 >> 172.21.15.100:0/1526265693 conn(0x7fca6806d730 msgr2=0x7fca68160720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:17.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.780+0000 7fca6fc00640 1 -- 172.21.15.100:0/1526265693 shutdown_connections 2024-09-17T04:29:17.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:17.780+0000 7fca6fc00640 1 -- 172.21.15.100:0/1526265693 wait complete. 2024-09-17T04:29:17.806 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T04:29:17.806 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T04:29:17.806 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T04:29:17.806 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:29:17.806 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T04:29:17.806 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T04:29:17.807 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T04:29:17.807 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:29:17.807 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T04:29:17.807 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T04:29:17.807 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T04:29:17.807 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:29:17.807 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T04:29:17.807 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T04:29:17.807 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T04:29:17.808 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:29:17.829 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T04:29:17.829 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T04:29:17.830 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T04:29:17.830 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T04:29:17.832 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T04:29:17.832 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T04:29:17.837 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T04:29:17.837 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T04:29:17.877 INFO:tasks.ceph.mds.a:Started 2024-09-17T04:29:17.878 DEBUG:teuthology.parallel:result is None 2024-09-17T04:29:17.880 INFO:tasks.ceph.mds.b:Started 2024-09-17T04:29:17.880 DEBUG:teuthology.parallel:result is None 2024-09-17T04:29:17.881 INFO:tasks.ceph.mds.c:Started 2024-09-17T04:29:17.881 DEBUG:teuthology.parallel:result is None 2024-09-17T04:29:17.883 INFO:tasks.ceph.mds.d:Started 2024-09-17T04:29:17.883 DEBUG:teuthology.parallel:result is None 2024-09-17T04:29:17.884 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T04:29:18.073 INFO:tasks.ceph.mds.a.smithi100.stdout:starting mds.a at 2024-09-17T04:29:18.075 INFO:tasks.ceph.mds.c.smithi100.stdout:starting mds.c at 2024-09-17T04:29:18.083 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-09-17T04:29:18.083 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-09-17T04:29:18.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.116+0000 7f6026600640 1 -- 172.21.15.100:0/377583219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6004003c10 msgr2=0x7f6004004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:18.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.116+0000 7f6026600640 1 --2- 172.21.15.100:0/377583219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6004003c10 0x7f6004004060 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f6010006fa0 tx=0x7f6004005830 comp rx=0 tx=0).stop 2024-09-17T04:29:18.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.116+0000 7f601fe00640 1 -- 172.21.15.100:0/377583219 reap_dead start 2024-09-17T04:29:18.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.116+0000 7f6026600640 1 -- 172.21.15.100:0/377583219 shutdown_connections 2024-09-17T04:29:18.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.116+0000 7f6026600640 1 -- 172.21.15.100:0/377583219 >> 172.21.15.100:0/377583219 conn(0x7f602006d3d0 msgr2=0x7f60200b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:18.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.116+0000 7f6026600640 1 -- 172.21.15.100:0/377583219 shutdown_connections 2024-09-17T04:29:18.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.116+0000 7f6026600640 1 -- 172.21.15.100:0/377583219 wait complete. 2024-09-17T04:29:18.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.118+0000 7f6026600640 1 Processor -- start 2024-09-17T04:29:18.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.118+0000 7f6026600640 1 -- start start 2024-09-17T04:29:18.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.120+0000 7f6026600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6004003c10 0x7f602009a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:18.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.120+0000 7f6026600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f602015c230 0x7f602015c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:18.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.120+0000 7f6026600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f602015cb80 0x7f602009d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:18.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.120+0000 7f6026600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f60200711c0 con 0x7f6004003c10 2024-09-17T04:29:18.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.120+0000 7f6026600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f60200b4380 con 0x7f602015cb80 2024-09-17T04:29:18.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.120+0000 7f6026600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6020071340 con 0x7f602015c230 2024-09-17T04:29:18.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.120+0000 7f601f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6004003c10 0x7f602009a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:18.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.120+0000 7f601f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6004003c10 0x7f602009a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51794/0 (socket says 172.21.15.100:51794) 2024-09-17T04:29:18.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.120+0000 7f601f400640 1 -- 172.21.15.100:0/2726140913 learned_addr learned my addr 172.21.15.100:0/2726140913 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:18.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.120+0000 7f601ea00640 1 --2- 172.21.15.100:0/2726140913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f602015c230 0x7f602015c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:18.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.120+0000 7f601fe00640 1 --2- 172.21.15.100:0/2726140913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f602015cb80 0x7f602009d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:18.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.120+0000 7f601f400640 1 -- 172.21.15.100:0/2726140913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f602015c230 msgr2=0x7f602015c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:18.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.120+0000 7f601f400640 1 --2- 172.21.15.100:0/2726140913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f602015c230 0x7f602015c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:18.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.120+0000 7f601f400640 1 -- 172.21.15.100:0/2726140913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f602015cb80 msgr2=0x7f602009d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:18.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.120+0000 7f601f400640 1 --2- 172.21.15.100:0/2726140913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f602015cb80 0x7f602009d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:18.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.120+0000 7f601f400640 1 -- 172.21.15.100:0/2726140913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60201f9af0 con 0x7f6004003c10 2024-09-17T04:29:18.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.120+0000 7f601fe00640 1 --2- 172.21.15.100:0/2726140913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f602015cb80 0x7f602009d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:18.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.121+0000 7f601f400640 1 --2- 172.21.15.100:0/2726140913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6004003c10 0x7f602009a2d0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f601000c320 tx=0x7f6010006ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:18.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.121+0000 7f6017400640 1 -- 172.21.15.100:0/2726140913 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6010003a00 con 0x7f6004003c10 2024-09-17T04:29:18.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.121+0000 7f6017400640 1 -- 172.21.15.100:0/2726140913 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6010003c90 con 0x7f6004003c10 2024-09-17T04:29:18.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.121+0000 7f6026600640 1 -- 172.21.15.100:0/2726140913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f602009be60 con 0x7f6004003c10 2024-09-17T04:29:18.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.121+0000 7f6017400640 1 -- 172.21.15.100:0/2726140913 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601002e6e0 con 0x7f6004003c10 2024-09-17T04:29:18.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.121+0000 7f6026600640 1 -- 172.21.15.100:0/2726140913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f602009c2d0 con 0x7f6004003c10 2024-09-17T04:29:18.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.122+0000 7f6026600640 1 -- 172.21.15.100:0/2726140913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60200af510 con 0x7f6004003c10 2024-09-17T04:29:18.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.123+0000 7f6017400640 1 -- 172.21.15.100:0/2726140913 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f6010035020 con 0x7f6004003c10 2024-09-17T04:29:18.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.124+0000 7f6017400640 1 --2- 172.21.15.100:0/2726140913 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6000078cb0 0x7f600007b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:18.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.124+0000 7f6017400640 1 -- 172.21.15.100:0/2726140913 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5502+0+0 (secure 0 0 0) 0x7f60100bb4f0 con 0x7f6004003c10 2024-09-17T04:29:18.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.125+0000 7f601ea00640 1 --2- 172.21.15.100:0/2726140913 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6000078cb0 0x7f600007b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:18.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.125+0000 7f601ea00640 1 --2- 172.21.15.100:0/2726140913 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6000078cb0 0x7f600007b1c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f60200ca890 tx=0x7f600c005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:18.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.129+0000 7f6017400640 1 -- 172.21.15.100:0/2726140913 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6010047030 con 0x7f6004003c10 2024-09-17T04:29:18.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:18.325+0000 7f6026600640 1 -- 172.21.15.100:0/2726140913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f602015a510 con 0x7f6004003c10 2024-09-17T04:29:19.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.063+0000 7f6017400640 1 -- 172.21.15.100:0/2726140913 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v81) ==== 97+0+0 (secure 0 0 0) 0x7f601003e070 con 0x7f6004003c10 2024-09-17T04:29:19.064 INFO:teuthology.orchestra.run.smithi100.stderr: removed all blocklist entries 2024-09-17T04:29:19.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.066+0000 7f6026600640 1 -- 172.21.15.100:0/2726140913 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6000078cb0 msgr2=0x7f600007b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:19.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.066+0000 7f6026600640 1 --2- 172.21.15.100:0/2726140913 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6000078cb0 0x7f600007b1c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f60200ca890 tx=0x7f600c005f90 comp rx=0 tx=0).stop 2024-09-17T04:29:19.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.066+0000 7f6026600640 1 -- 172.21.15.100:0/2726140913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6004003c10 msgr2=0x7f602009a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:19.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.066+0000 7f6026600640 1 --2- 172.21.15.100:0/2726140913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6004003c10 0x7f602009a2d0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f601000c320 tx=0x7f6010006ce0 comp rx=0 tx=0).stop 2024-09-17T04:29:19.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.066+0000 7f6026600640 1 -- 172.21.15.100:0/2726140913 shutdown_connections 2024-09-17T04:29:19.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.066+0000 7f6026600640 1 --2- 172.21.15.100:0/2726140913 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6000078cb0 0x7f600007b1c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:19.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.066+0000 7f6026600640 1 --2- 172.21.15.100:0/2726140913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f602015cb80 0x7f602009d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:19.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.066+0000 7f6026600640 1 --2- 172.21.15.100:0/2726140913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f602015c230 0x7f602015c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:19.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.066+0000 7f6026600640 1 --2- 172.21.15.100:0/2726140913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6004003c10 0x7f602009a2d0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:19.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.066+0000 7f6026600640 1 -- 172.21.15.100:0/2726140913 >> 172.21.15.100:0/2726140913 conn(0x7f602006d3d0 msgr2=0x7f60200b1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:19.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.066+0000 7f6026600640 1 -- 172.21.15.100:0/2726140913 shutdown_connections 2024-09-17T04:29:19.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.066+0000 7f6026600640 1 -- 172.21.15.100:0/2726140913 wait complete. 2024-09-17T04:29:19.091 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T04:29:19.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.312+0000 7fd061000640 1 -- 172.21.15.100:0/2424110501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd05c15a0f0 msgr2=0x7fd05c15d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:19.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.312+0000 7fd061000640 1 --2- 172.21.15.100:0/2424110501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd05c15a0f0 0x7fd05c15d670 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fd054009c10 tx=0x7fd05402a870 comp rx=0 tx=0).stop 2024-09-17T04:29:19.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.312+0000 7fd061000640 1 -- 172.21.15.100:0/2424110501 shutdown_connections 2024-09-17T04:29:19.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.312+0000 7fd061000640 1 --2- 172.21.15.100:0/2424110501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd05c15a0f0 0x7fd05c15d670 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:19.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.312+0000 7fd061000640 1 --2- 172.21.15.100:0/2424110501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd05c0af170 0x7fd05c0b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:19.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.312+0000 7fd061000640 1 -- 172.21.15.100:0/2424110501 >> 172.21.15.100:0/2424110501 conn(0x7fd05c0b1e40 msgr2=0x7fd05c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:19.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.312+0000 7fd061000640 1 -- 172.21.15.100:0/2424110501 shutdown_connections 2024-09-17T04:29:19.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.313+0000 7fd061000640 1 -- 172.21.15.100:0/2424110501 wait complete. 2024-09-17T04:29:19.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.315+0000 7fd061000640 1 Processor -- start 2024-09-17T04:29:19.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.315+0000 7fd061000640 1 -- start start 2024-09-17T04:29:19.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.316+0000 7fd061000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd05c0ad9c0 0x7fd05c1f55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:19.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.316+0000 7fd061000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd05c0af170 0x7fd05c072e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:19.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.316+0000 7fd061000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd05c15a0f0 0x7fd05c155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:19.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.316+0000 7fd061000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c06f410 con 0x7fd05c15a0f0 2024-09-17T04:29:19.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.316+0000 7fd061000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c06f290 con 0x7fd05c0af170 2024-09-17T04:29:19.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.316+0000 7fd061000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd05c06f590 con 0x7fd05c0ad9c0 2024-09-17T04:29:19.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.317+0000 7fd05be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd05c15a0f0 0x7fd05c155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:19.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.317+0000 7fd05be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd05c15a0f0 0x7fd05c155180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51822/0 (socket says 172.21.15.100:51822) 2024-09-17T04:29:19.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.317+0000 7fd05be00640 1 -- 172.21.15.100:0/1215462671 learned_addr learned my addr 172.21.15.100:0/1215462671 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:19.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.317+0000 7fd052a00640 1 --2- 172.21.15.100:0/1215462671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd05c0af170 0x7fd05c072e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:19.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.317+0000 7fd05b400640 1 --2- 172.21.15.100:0/1215462671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd05c0ad9c0 0x7fd05c1f55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:19.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.317+0000 7fd05be00640 1 -- 172.21.15.100:0/1215462671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd05c0ad9c0 msgr2=0x7fd05c1f55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:19.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.317+0000 7fd05be00640 1 --2- 172.21.15.100:0/1215462671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd05c0ad9c0 0x7fd05c1f55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:19.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.317+0000 7fd05be00640 1 -- 172.21.15.100:0/1215462671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd05c0af170 msgr2=0x7fd05c072e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:19.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.317+0000 7fd05be00640 1 --2- 172.21.15.100:0/1215462671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd05c0af170 0x7fd05c072e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:19.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.317+0000 7fd05be00640 1 -- 172.21.15.100:0/1215462671 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05c0c97b0 con 0x7fd05c15a0f0 2024-09-17T04:29:19.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.317+0000 7fd05b400640 1 --2- 172.21.15.100:0/1215462671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd05c0ad9c0 0x7fd05c1f55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:19.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.317+0000 7fd05be00640 1 --2- 172.21.15.100:0/1215462671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd05c15a0f0 0x7fd05c155180 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fd054009d30 tx=0x7fd054033a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:19.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.318+0000 7fd053e00640 1 -- 172.21.15.100:0/1215462671 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054042020 con 0x7fd05c15a0f0 2024-09-17T04:29:19.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.318+0000 7fd053e00640 1 -- 172.21.15.100:0/1215462671 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd054033d30 con 0x7fd05c15a0f0 2024-09-17T04:29:19.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.318+0000 7fd061000640 1 -- 172.21.15.100:0/1215462671 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd05c0c9a90 con 0x7fd05c15a0f0 2024-09-17T04:29:19.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.318+0000 7fd053e00640 1 -- 172.21.15.100:0/1215462671 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054037040 con 0x7fd05c15a0f0 2024-09-17T04:29:19.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.318+0000 7fd061000640 1 -- 172.21.15.100:0/1215462671 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd05c0ca050 con 0x7fd05c15a0f0 2024-09-17T04:29:19.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.319+0000 7fd061000640 1 -- 172.21.15.100:0/1215462671 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd05c06fab0 con 0x7fd05c15a0f0 2024-09-17T04:29:19.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.320+0000 7fd053e00640 1 -- 172.21.15.100:0/1215462671 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fd054045020 con 0x7fd05c15a0f0 2024-09-17T04:29:19.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.321+0000 7fd053e00640 1 --2- 172.21.15.100:0/1215462671 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd038078ce0 0x7fd03807b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:19.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.321+0000 7fd053e00640 1 -- 172.21.15.100:0/1215462671 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4857+0+0 (secure 0 0 0) 0x7fd0540bbab0 con 0x7fd05c15a0f0 2024-09-17T04:29:19.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.321+0000 7fd05b400640 1 --2- 172.21.15.100:0/1215462671 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd038078ce0 0x7fd03807b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:19.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.322+0000 7fd05b400640 1 --2- 172.21.15.100:0/1215462671 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd038078ce0 0x7fd03807b1f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd04c007a50 tx=0x7fd04c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:19.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.325+0000 7fd053e00640 1 -- 172.21.15.100:0/1215462671 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd05404c030 con 0x7fd05c15a0f0 2024-09-17T04:29:19.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.575+0000 7fd061000640 1 -- 172.21.15.100:0/1215462671 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fd05c0ad150 con 0x7fd05c15a0f0 2024-09-17T04:29:19.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.576+0000 7fd053e00640 1 -- 172.21.15.100:0/1215462671 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7fd054083f90 con 0x7fd05c15a0f0 2024-09-17T04:29:19.576 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:19.576 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:29:19.577 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_dump": [ 2024-09-17T04:29:19.577 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:19.577 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.a", 2024-09-17T04:29:19.577 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmV8asJRAAkSgIWqGc1OeSRbs9PEOT8w==", 2024-09-17T04:29:19.577 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:19.577 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:29:19.577 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:29:19.577 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:29:19.577 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:19.577 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:19.577 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:19.578 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:19.578 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.b", 2024-09-17T04:29:19.578 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmpM7ZBRAA1Og785tJwa6NA9zBdmNNQA==", 2024-09-17T04:29:19.578 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:19.578 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:29:19.578 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:29:19.578 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:29:19.578 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:19.578 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:19.578 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:19.578 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:19.578 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.c", 2024-09-17T04:29:19.579 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmXaOGMxAAT+96g/5fAzPW8Cc2HtQUyA==", 2024-09-17T04:29:19.579 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:19.579 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:29:19.579 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:29:19.579 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:29:19.579 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:19.579 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:19.579 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:19.579 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:19.579 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.d", 2024-09-17T04:29:19.579 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmRqC9ExAA23kZ0Ks1lGN2k4GXL8ASrg==", 2024-09-17T04:29:19.579 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:19.583 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:29:19.583 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:29:19.583 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:29:19.583 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:19.583 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:19.583 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:19.583 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:19.583 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.0", 2024-09-17T04:29:19.583 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAwBOlmofFmAxAARerYkU9B2a9MM9vHa3o4LQ==", 2024-09-17T04:29:19.583 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:19.583 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:29:19.583 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:29:19.584 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:19.584 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:19.584 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:19.584 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:19.584 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.1", 2024-09-17T04:29:19.584 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAzBOlmbz6VExAAoVqSqXTc1wArKhdA6+NxnA==", 2024-09-17T04:29:19.584 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:19.584 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:29:19.584 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:29:19.584 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:19.584 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:19.584 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:19.584 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:19.585 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.2", 2024-09-17T04:29:19.585 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA2BOlmfeSAJBAAchY6XhyDRusc+zqwQ6v6wQ==", 2024-09-17T04:29:19.585 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:19.585 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:29:19.585 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:29:19.585 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:19.585 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:19.585 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:19.585 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:19.585 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.3", 2024-09-17T04:29:19.585 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA6BOlm/TQiBBAAHhbs2+RCjAenGR1frhbTmQ==", 2024-09-17T04:29:19.585 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:19.585 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:29:19.585 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:29:19.586 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:19.586 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:19.586 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:19.586 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:19.586 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.4", 2024-09-17T04:29:19.586 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBABOlm7OKvBhAALkaAKEUYN5ChwGejoSF8yA==", 2024-09-17T04:29:19.586 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:19.586 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:29:19.586 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:29:19.586 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:19.586 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:19.586 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:19.586 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:19.587 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.5", 2024-09-17T04:29:19.587 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBDBOlmQpi1FhAAbA47dg6ckK1PFoCZL0SBEQ==", 2024-09-17T04:29:19.587 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:19.587 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:29:19.587 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:29:19.587 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:19.587 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:19.587 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:19.587 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:19.587 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.6", 2024-09-17T04:29:19.588 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBGBOlmx/w0JhAAWcPvZkMYtXD/EMKrg/W/4Q==", 2024-09-17T04:29:19.588 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:19.588 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:29:19.588 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:29:19.588 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:19.588 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:19.588 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:19.588 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:19.588 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.7", 2024-09-17T04:29:19.588 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBJBOlmWHjWNBAAgdul2UKY9g5p7EVi0ZdrFw==", 2024-09-17T04:29:19.588 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:19.588 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:29:19.588 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:29:19.588 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:19.589 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:19.589 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:19.589 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:19.589 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.0", 2024-09-17T04:29:19.589 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmfYg9IBAABP1uRKGOugrA47VxwRLkjA==", 2024-09-17T04:29:19.589 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:19.589 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:29:19.589 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow r", 2024-09-17T04:29:19.589 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T04:29:19.589 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:19.590 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:19.590 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:19.590 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.1", 2024-09-17T04:29:19.590 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmByhhNRAAPGaxFCDd8ywFVwwhprrPbQ==", 2024-09-17T04:29:19.590 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:19.590 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:29:19.591 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:29:19.591 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:29:19.591 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:29:19.591 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:19.591 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:19.591 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:19.591 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.2", 2024-09-17T04:29:19.592 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmgcm0KhAAGF06VlHjBM/AGxafN6/3aA==", 2024-09-17T04:29:19.592 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:19.592 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:29:19.592 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:29:19.592 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:29:19.592 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:29:19.593 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:19.593 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:19.593 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:19.593 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.3", 2024-09-17T04:29:19.594 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtBOlm9VQ6BBAAXcahu35/uDm8Z+f+3SF0Gg==", 2024-09-17T04:29:19.594 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:19.594 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:29:19.594 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:29:19.594 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:29:19.594 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:29:19.594 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:19.595 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:19.595 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:19.595 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.admin", 2024-09-17T04:29:19.595 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAqBOlmZ+bIJBAAG3oaMNlYWNakywGDvAGbGw==", 2024-09-17T04:29:19.595 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:19.595 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:29:19.596 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:29:19.596 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:29:19.596 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:19.596 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:19.596 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:19.596 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:19.596 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mgr.x", 2024-09-17T04:29:19.597 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmfaL7GhAArqLH66RkbBsz+TNOKtPHOA==", 2024-09-17T04:29:19.597 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:19.597 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:29:19.597 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile mgr", 2024-09-17T04:29:19.597 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:19.597 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:19.598 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:19.598 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:19.598 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mgr.y", 2024-09-17T04:29:19.598 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmM4D1DhAA5uUWgRVOMujWqbYzOjFp9A==", 2024-09-17T04:29:19.598 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:19.598 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:29:19.598 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile mgr", 2024-09-17T04:29:19.598 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:19.598 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:19.598 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:19.599 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-17T04:29:19.599 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:29:19.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.578+0000 7fd061000640 1 -- 172.21.15.100:0/1215462671 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd038078ce0 msgr2=0x7fd03807b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:19.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.578+0000 7fd061000640 1 --2- 172.21.15.100:0/1215462671 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd038078ce0 0x7fd03807b1f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd04c007a50 tx=0x7fd04c009040 comp rx=0 tx=0).stop 2024-09-17T04:29:19.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.578+0000 7fd061000640 1 -- 172.21.15.100:0/1215462671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd05c15a0f0 msgr2=0x7fd05c155180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:19.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.578+0000 7fd061000640 1 --2- 172.21.15.100:0/1215462671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd05c15a0f0 0x7fd05c155180 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fd054009d30 tx=0x7fd054033a50 comp rx=0 tx=0).stop 2024-09-17T04:29:19.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.578+0000 7fd061000640 1 -- 172.21.15.100:0/1215462671 shutdown_connections 2024-09-17T04:29:19.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.578+0000 7fd061000640 1 --2- 172.21.15.100:0/1215462671 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd038078ce0 0x7fd03807b1f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:19.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.578+0000 7fd061000640 1 --2- 172.21.15.100:0/1215462671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd05c15a0f0 0x7fd05c155180 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:19.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.578+0000 7fd061000640 1 --2- 172.21.15.100:0/1215462671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd05c0af170 0x7fd05c072e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:19.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.578+0000 7fd061000640 1 --2- 172.21.15.100:0/1215462671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd05c0ad9c0 0x7fd05c1f55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:19.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.578+0000 7fd061000640 1 -- 172.21.15.100:0/1215462671 >> 172.21.15.100:0/1215462671 conn(0x7fd05c0b1e40 msgr2=0x7fd05c0cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:19.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.578+0000 7fd061000640 1 -- 172.21.15.100:0/1215462671 shutdown_connections 2024-09-17T04:29:19.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.578+0000 7fd061000640 1 -- 172.21.15.100:0/1215462671 wait complete. 2024-09-17T04:29:19.602 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T04:29:19.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.823+0000 7f3878200640 1 -- 172.21.15.100:0/3116098397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38700b3550 msgr2=0x7f38700b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:19.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.823+0000 7f3878200640 1 --2- 172.21.15.100:0/3116098397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38700b3550 0x7f38700b3960 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f3870072d60 tx=0x7f386802a980 comp rx=0 tx=0).stop 2024-09-17T04:29:19.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.823+0000 7f3878200640 1 -- 172.21.15.100:0/3116098397 shutdown_connections 2024-09-17T04:29:19.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.823+0000 7f3878200640 1 --2- 172.21.15.100:0/3116098397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38700b3550 0x7f38700b3960 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:19.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.823+0000 7f3878200640 1 --2- 172.21.15.100:0/3116098397 >> v2:172.21.15.184:3300/0 conn(0x7f38700ad0e0 0x7f38700ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:19.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.823+0000 7f3878200640 1 -- 172.21.15.100:0/3116098397 >> 172.21.15.100:0/3116098397 conn(0x7f387006d730 msgr2=0x7f38700b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:19.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.823+0000 7f3878200640 1 -- 172.21.15.100:0/3116098397 shutdown_connections 2024-09-17T04:29:19.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.823+0000 7f3878200640 1 -- 172.21.15.100:0/3116098397 wait complete. 2024-09-17T04:29:19.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.826+0000 7f3878200640 1 Processor -- start 2024-09-17T04:29:19.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.826+0000 7f3878200640 1 -- start start 2024-09-17T04:29:19.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.828+0000 7f3878200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38700ad0e0 0x7f38701440f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:19.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.828+0000 7f3878200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38700b3550 0x7f387009d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:19.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.828+0000 7f3878200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f387015d760 0x7f3870156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:19.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.828+0000 7f3878200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f38700b4da0 con 0x7f38700ad0e0 2024-09-17T04:29:19.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.828+0000 7f3878200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f38700b4c20 con 0x7f38700b3550 2024-09-17T04:29:19.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.828+0000 7f3878200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f38700b3ea0 con 0x7f387015d760 2024-09-17T04:29:19.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.828+0000 7f3875800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38700ad0e0 0x7f38701440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:19.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.828+0000 7f3875800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38700ad0e0 0x7f38701440f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51840/0 (socket says 172.21.15.100:51840) 2024-09-17T04:29:19.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.828+0000 7f3875800640 1 -- 172.21.15.100:0/3957958782 learned_addr learned my addr 172.21.15.100:0/3957958782 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:19.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.828+0000 7f3874e00640 1 --2- 172.21.15.100:0/3957958782 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38700b3550 0x7f387009d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:19.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.828+0000 7f3876200640 1 --2- 172.21.15.100:0/3957958782 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f387015d760 0x7f3870156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:19.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.828+0000 7f3875800640 1 -- 172.21.15.100:0/3957958782 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f387015d760 msgr2=0x7f3870156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:19.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.828+0000 7f3875800640 1 --2- 172.21.15.100:0/3957958782 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f387015d760 0x7f3870156480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:19.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.828+0000 7f3875800640 1 -- 172.21.15.100:0/3957958782 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38700b3550 msgr2=0x7f387009d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:19.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.828+0000 7f3875800640 1 --2- 172.21.15.100:0/3957958782 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38700b3550 0x7f387009d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:19.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.828+0000 7f3875800640 1 -- 172.21.15.100:0/3957958782 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f387009dde0 con 0x7f38700ad0e0 2024-09-17T04:29:19.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.828+0000 7f3874e00640 1 --2- 172.21.15.100:0/3957958782 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38700b3550 0x7f387009d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:19.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.829+0000 7f3875800640 1 --2- 172.21.15.100:0/3957958782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38700ad0e0 0x7f38701440f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f386400b3e0 tx=0x7f386400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:19.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.829+0000 7f386d600640 1 -- 172.21.15.100:0/3957958782 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3864015020 con 0x7f38700ad0e0 2024-09-17T04:29:19.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.829+0000 7f386d600640 1 -- 172.21.15.100:0/3957958782 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3864011940 con 0x7f38700ad0e0 2024-09-17T04:29:19.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.829+0000 7f386d600640 1 -- 172.21.15.100:0/3957958782 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3864011d90 con 0x7f38700ad0e0 2024-09-17T04:29:19.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.829+0000 7f3878200640 1 -- 172.21.15.100:0/3957958782 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38701556c0 con 0x7f38700ad0e0 2024-09-17T04:29:19.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.829+0000 7f3878200640 1 -- 172.21.15.100:0/3957958782 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3870155c80 con 0x7f38700ad0e0 2024-09-17T04:29:19.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.830+0000 7f3878200640 1 -- 172.21.15.100:0/3957958782 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38700b43e0 con 0x7f38700ad0e0 2024-09-17T04:29:19.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.832+0000 7f386d600640 1 -- 172.21.15.100:0/3957958782 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f3864018020 con 0x7f38700ad0e0 2024-09-17T04:29:19.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.832+0000 7f386d600640 1 --2- 172.21.15.100:0/3957958782 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3850078ce0 0x7f385007b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:19.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.833+0000 7f386d600640 1 -- 172.21.15.100:0/3957958782 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4857+0+0 (secure 0 0 0) 0x7f386409b0f0 con 0x7f38700ad0e0 2024-09-17T04:29:19.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.833+0000 7f3874e00640 1 --2- 172.21.15.100:0/3957958782 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3850078ce0 0x7f385007b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:19.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.833+0000 7f3874e00640 1 --2- 172.21.15.100:0/3957958782 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3850078ce0 0x7f385007b1f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f387015d350 tx=0x7f385c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:19.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:19.836+0000 7f386d600640 1 -- 172.21.15.100:0/3957958782 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f386401f020 con 0x7f38700ad0e0 2024-09-17T04:29:20.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.022+0000 7f3878200640 1 -- 172.21.15.100:0/3957958782 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f387015f4f0 con 0x7f38700ad0e0 2024-09-17T04:29:20.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.023+0000 7f386d600640 1 -- 172.21.15.100:0/3957958782 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) ==== 84+0+65 (secure 0 0 0) 0x7f38640636d0 con 0x7f38700ad0e0 2024-09-17T04:29:20.023 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:20.024 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-09-17T04:29:20.024 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:20.024 INFO:teuthology.orchestra.run.smithi100.stdout: "poolnum": 1, 2024-09-17T04:29:20.024 INFO:teuthology.orchestra.run.smithi100.stdout: "poolname": ".mgr" 2024-09-17T04:29:20.024 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:20.024 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-09-17T04:29:20.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.025+0000 7f3878200640 1 -- 172.21.15.100:0/3957958782 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3850078ce0 msgr2=0x7f385007b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:20.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.025+0000 7f3878200640 1 --2- 172.21.15.100:0/3957958782 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3850078ce0 0x7f385007b1f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f387015d350 tx=0x7f385c008040 comp rx=0 tx=0).stop 2024-09-17T04:29:20.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.025+0000 7f3878200640 1 -- 172.21.15.100:0/3957958782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38700ad0e0 msgr2=0x7f38701440f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:20.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.025+0000 7f3878200640 1 --2- 172.21.15.100:0/3957958782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38700ad0e0 0x7f38701440f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f386400b3e0 tx=0x7f386400b8a0 comp rx=0 tx=0).stop 2024-09-17T04:29:20.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.025+0000 7f3878200640 1 -- 172.21.15.100:0/3957958782 shutdown_connections 2024-09-17T04:29:20.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.025+0000 7f3878200640 1 --2- 172.21.15.100:0/3957958782 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3850078ce0 0x7f385007b1f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:20.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.025+0000 7f3878200640 1 --2- 172.21.15.100:0/3957958782 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f387015d760 0x7f3870156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:20.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.025+0000 7f3878200640 1 --2- 172.21.15.100:0/3957958782 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38700b3550 0x7f387009d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:20.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.025+0000 7f3878200640 1 --2- 172.21.15.100:0/3957958782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38700ad0e0 0x7f38701440f0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:20.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.025+0000 7f3878200640 1 -- 172.21.15.100:0/3957958782 >> 172.21.15.100:0/3957958782 conn(0x7f387006d730 msgr2=0x7f387015ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:20.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.025+0000 7f3878200640 1 -- 172.21.15.100:0/3957958782 shutdown_connections 2024-09-17T04:29:20.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.025+0000 7f3878200640 1 -- 172.21.15.100:0/3957958782 wait complete. 2024-09-17T04:29:20.049 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T04:29:20.049 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T04:29:20.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.274+0000 7fe9b6a00640 1 -- 172.21.15.100:0/841637165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9b00b36f0 msgr2=0x7fe9b00cb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:20.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.275+0000 7fe9b6a00640 1 --2- 172.21.15.100:0/841637165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9b00b36f0 0x7fe9b00cb930 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0072f40 tx=0x7fe9a802a420 comp rx=0 tx=0).stop 2024-09-17T04:29:20.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.275+0000 7fe9b6a00640 1 -- 172.21.15.100:0/841637165 shutdown_connections 2024-09-17T04:29:20.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.275+0000 7fe9b6a00640 1 --2- 172.21.15.100:0/841637165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9b00b36f0 0x7fe9b00cb930 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:20.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.275+0000 7fe9b6a00640 1 --2- 172.21.15.100:0/841637165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9b00cc7d0 0x7fe9b00b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:20.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.275+0000 7fe9b6a00640 1 -- 172.21.15.100:0/841637165 >> 172.21.15.100:0/841637165 conn(0x7fe9b00b2540 msgr2=0x7fe9b00d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:20.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.275+0000 7fe9b6a00640 1 -- 172.21.15.100:0/841637165 shutdown_connections 2024-09-17T04:29:20.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.275+0000 7fe9b6a00640 1 -- 172.21.15.100:0/841637165 wait complete. 2024-09-17T04:29:20.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.277+0000 7fe9b6a00640 1 Processor -- start 2024-09-17T04:29:20.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.277+0000 7fe9b6a00640 1 -- start start 2024-09-17T04:29:20.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.278+0000 7fe9b6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9b00b36f0 0x7fe9b0099aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:20.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.278+0000 7fe9b6a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9b00cc7d0 0x7fe9b0154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:20.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.278+0000 7fe9b6a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9b015ba50 0x7fe9b01551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:20.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.278+0000 7fe9b6a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b006f8a0 con 0x7fe9b00b36f0 2024-09-17T04:29:20.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.278+0000 7fe9b6a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b006f720 con 0x7fe9b00cc7d0 2024-09-17T04:29:20.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.278+0000 7fe9b6a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe9b006fa20 con 0x7fe9b015ba50 2024-09-17T04:29:20.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.279+0000 7fe9afe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9b00b36f0 0x7fe9b0099aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:20.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.279+0000 7fe9afe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9b00b36f0 0x7fe9b0099aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51856/0 (socket says 172.21.15.100:51856) 2024-09-17T04:29:20.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.279+0000 7fe9afe00640 1 -- 172.21.15.100:0/2070868691 learned_addr learned my addr 172.21.15.100:0/2070868691 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:20.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.279+0000 7fe9b4a00640 1 --2- 172.21.15.100:0/2070868691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9b015ba50 0x7fe9b01551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:20.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.279+0000 7fe9af400640 1 --2- 172.21.15.100:0/2070868691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9b00cc7d0 0x7fe9b0154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:20.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.279+0000 7fe9afe00640 1 -- 172.21.15.100:0/2070868691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9b015ba50 msgr2=0x7fe9b01551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:20.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.279+0000 7fe9afe00640 1 --2- 172.21.15.100:0/2070868691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9b015ba50 0x7fe9b01551c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:20.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.279+0000 7fe9afe00640 1 -- 172.21.15.100:0/2070868691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9b00cc7d0 msgr2=0x7fe9b0154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:20.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.279+0000 7fe9afe00640 1 --2- 172.21.15.100:0/2070868691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9b00cc7d0 0x7fe9b0154c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:20.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.279+0000 7fe9afe00640 1 -- 172.21.15.100:0/2070868691 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b00b3ea0 con 0x7fe9b00b36f0 2024-09-17T04:29:20.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.279+0000 7fe9af400640 1 --2- 172.21.15.100:0/2070868691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9b00cc7d0 0x7fe9b0154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:29:20.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.279+0000 7fe9afe00640 1 --2- 172.21.15.100:0/2070868691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9b00b36f0 0x7fe9b0099aa0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000cd10 tx=0x7fe9a00075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:20.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.279+0000 7fe9a7e00640 1 -- 172.21.15.100:0/2070868691 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0004b20 con 0x7fe9b00b36f0 2024-09-17T04:29:20.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.280+0000 7fe9a7e00640 1 -- 172.21.15.100:0/2070868691 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9a0004ce0 con 0x7fe9b00b36f0 2024-09-17T04:29:20.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.280+0000 7fe9a7e00640 1 -- 172.21.15.100:0/2070868691 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0005280 con 0x7fe9b00b36f0 2024-09-17T04:29:20.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.280+0000 7fe9b6a00640 1 -- 172.21.15.100:0/2070868691 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b00b41f0 con 0x7fe9b00b36f0 2024-09-17T04:29:20.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.280+0000 7fe9b6a00640 1 -- 172.21.15.100:0/2070868691 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b015d550 con 0x7fe9b00b36f0 2024-09-17T04:29:20.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.281+0000 7fe9b6a00640 1 -- 172.21.15.100:0/2070868691 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9b015d7b0 con 0x7fe9b00b36f0 2024-09-17T04:29:20.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.282+0000 7fe9a7e00640 1 -- 172.21.15.100:0/2070868691 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fe9a0005420 con 0x7fe9b00b36f0 2024-09-17T04:29:20.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.283+0000 7fe9a7e00640 1 --2- 172.21.15.100:0/2070868691 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe990078ce0 0x7fe99007b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:20.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.283+0000 7fe9a7e00640 1 -- 172.21.15.100:0/2070868691 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4857+0+0 (secure 0 0 0) 0x7fe9a009b3d0 con 0x7fe9b00b36f0 2024-09-17T04:29:20.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.283+0000 7fe9af400640 1 --2- 172.21.15.100:0/2070868691 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe990078ce0 0x7fe99007b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:20.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.284+0000 7fe9af400640 1 --2- 172.21.15.100:0/2070868691 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe990078ce0 0x7fe99007b1f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe9b009e5e0 tx=0x7fe99c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:20.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.288+0000 7fe9a7e00640 1 -- 172.21.15.100:0/2070868691 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe9a0014030 con 0x7fe9b00b36f0 2024-09-17T04:29:20.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:20.472+0000 7fe9b6a00640 1 -- 172.21.15.100:0/2070868691 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe9b006ff30 con 0x7fe9b00b36f0 2024-09-17T04:29:21.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.078+0000 7fe9a7e00640 1 -- 172.21.15.100:0/2070868691 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) ==== 138+0+0 (secure 0 0 0) 0x7fe9a00639c0 con 0x7fe9b00b36f0 2024-09-17T04:29:21.079 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' created 2024-09-17T04:29:21.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.081+0000 7fe9b6a00640 1 -- 172.21.15.100:0/2070868691 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe990078ce0 msgr2=0x7fe99007b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:21.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.081+0000 7fe9b6a00640 1 --2- 172.21.15.100:0/2070868691 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe990078ce0 0x7fe99007b1f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe9b009e5e0 tx=0x7fe99c008040 comp rx=0 tx=0).stop 2024-09-17T04:29:21.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.081+0000 7fe9b6a00640 1 -- 172.21.15.100:0/2070868691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9b00b36f0 msgr2=0x7fe9b0099aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:21.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.081+0000 7fe9b6a00640 1 --2- 172.21.15.100:0/2070868691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9b00b36f0 0x7fe9b0099aa0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000cd10 tx=0x7fe9a00075f0 comp rx=0 tx=0).stop 2024-09-17T04:29:21.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.081+0000 7fe9b6a00640 1 -- 172.21.15.100:0/2070868691 shutdown_connections 2024-09-17T04:29:21.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.081+0000 7fe9b6a00640 1 --2- 172.21.15.100:0/2070868691 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe990078ce0 0x7fe99007b1f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:21.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.081+0000 7fe9b6a00640 1 --2- 172.21.15.100:0/2070868691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9b015ba50 0x7fe9b01551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:21.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.081+0000 7fe9b6a00640 1 --2- 172.21.15.100:0/2070868691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9b00cc7d0 0x7fe9b0154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:21.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.081+0000 7fe9b6a00640 1 --2- 172.21.15.100:0/2070868691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9b00b36f0 0x7fe9b0099aa0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:21.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.081+0000 7fe9b6a00640 1 -- 172.21.15.100:0/2070868691 >> 172.21.15.100:0/2070868691 conn(0x7fe9b00b2540 msgr2=0x7fe9b015a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:21.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.081+0000 7fe9b6a00640 1 -- 172.21.15.100:0/2070868691 shutdown_connections 2024-09-17T04:29:21.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.082+0000 7fe9b6a00640 1 -- 172.21.15.100:0/2070868691 wait complete. 2024-09-17T04:29:21.114 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T04:29:21.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.341+0000 7f0abe400640 1 -- 172.21.15.100:0/226173444 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ab80b95e0 msgr2=0x7f0ab80b39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:21.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.341+0000 7f0abe400640 1 --2- 172.21.15.100:0/226173444 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ab80b95e0 0x7f0ab80b39d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0098e0 tx=0x7f0aac02a5d0 comp rx=0 tx=0).stop 2024-09-17T04:29:21.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.341+0000 7f0abe400640 1 -- 172.21.15.100:0/226173444 shutdown_connections 2024-09-17T04:29:21.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.341+0000 7f0abe400640 1 --2- 172.21.15.100:0/226173444 >> v2:172.21.15.184:3300/0 conn(0x7f0ab80d1390 0x7f0ab80cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:21.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.341+0000 7f0abe400640 1 --2- 172.21.15.100:0/226173444 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ab80b95e0 0x7f0ab80b39d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:21.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.341+0000 7f0abe400640 1 -- 172.21.15.100:0/226173444 >> 172.21.15.100:0/226173444 conn(0x7f0ab809d860 msgr2=0x7f0ab808bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:21.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.341+0000 7f0abe400640 1 -- 172.21.15.100:0/226173444 shutdown_connections 2024-09-17T04:29:21.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.341+0000 7f0abe400640 1 -- 172.21.15.100:0/226173444 wait complete. 2024-09-17T04:29:21.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.343+0000 7f0abe400640 1 Processor -- start 2024-09-17T04:29:21.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.343+0000 7f0abe400640 1 -- start start 2024-09-17T04:29:21.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.345+0000 7f0abe400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ab806e890 0x7f0ab814d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:21.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.345+0000 7f0abe400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ab80b95e0 0x7f0ab814a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:21.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.345+0000 7f0abe400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ab80b16a0 0x7f0ab80b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:21.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.345+0000 7f0abe400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab80d3020 con 0x7f0ab80b95e0 2024-09-17T04:29:21.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.345+0000 7f0abe400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab80d2ea0 con 0x7f0ab806e890 2024-09-17T04:29:21.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.345+0000 7f0abe400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0ab80d31a0 con 0x7f0ab80b16a0 2024-09-17T04:29:21.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.345+0000 7f0ab7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ab80b95e0 0x7f0ab814a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:21.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.345+0000 7f0ab7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ab80b95e0 0x7f0ab814a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51864/0 (socket says 172.21.15.100:51864) 2024-09-17T04:29:21.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.345+0000 7f0ab7e00640 1 -- 172.21.15.100:0/2727250513 learned_addr learned my addr 172.21.15.100:0/2727250513 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:21.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.345+0000 7f0abd000640 1 --2- 172.21.15.100:0/2727250513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ab806e890 0x7f0ab814d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:21.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.345+0000 7f0abda00640 1 --2- 172.21.15.100:0/2727250513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ab80b16a0 0x7f0ab80b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:21.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.346+0000 7f0ab7e00640 1 -- 172.21.15.100:0/2727250513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ab80b16a0 msgr2=0x7f0ab80b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:21.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.346+0000 7f0ab7e00640 1 --2- 172.21.15.100:0/2727250513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ab80b16a0 0x7f0ab80b1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:21.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.346+0000 7f0ab7e00640 1 -- 172.21.15.100:0/2727250513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ab806e890 msgr2=0x7f0ab814d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:21.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.346+0000 7f0ab7e00640 1 --2- 172.21.15.100:0/2727250513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ab806e890 0x7f0ab814d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:21.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.346+0000 7f0ab7e00640 1 -- 172.21.15.100:0/2727250513 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab80b1ff0 con 0x7f0ab80b95e0 2024-09-17T04:29:21.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.346+0000 7f0ab7e00640 1 --2- 172.21.15.100:0/2727250513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ab80b95e0 0x7f0ab814a270 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f0aa400c730 tx=0x7f0aa400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:21.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.346+0000 7f0ab4c00640 1 -- 172.21.15.100:0/2727250513 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa40052d0 con 0x7f0ab80b95e0 2024-09-17T04:29:21.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.346+0000 7f0ab4c00640 1 -- 172.21.15.100:0/2727250513 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0aa4005490 con 0x7f0ab80b95e0 2024-09-17T04:29:21.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.346+0000 7f0abe400640 1 -- 172.21.15.100:0/2727250513 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ab814dba0 con 0x7f0ab80b95e0 2024-09-17T04:29:21.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.346+0000 7f0ab4c00640 1 -- 172.21.15.100:0/2727250513 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa400a8a0 con 0x7f0ab80b95e0 2024-09-17T04:29:21.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.346+0000 7f0abe400640 1 -- 172.21.15.100:0/2727250513 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ab81f9580 con 0x7f0ab80b95e0 2024-09-17T04:29:21.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.347+0000 7f0abe400640 1 -- 172.21.15.100:0/2727250513 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ab80d56a0 con 0x7f0ab80b95e0 2024-09-17T04:29:21.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.349+0000 7f0ab4c00640 1 -- 172.21.15.100:0/2727250513 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f0aa400aa40 con 0x7f0ab80b95e0 2024-09-17T04:29:21.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.349+0000 7f0ab4c00640 1 --2- 172.21.15.100:0/2727250513 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0a98078cb0 0x7f0a9807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:21.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.350+0000 7f0ab4c00640 1 -- 172.21.15.100:0/2727250513 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5227+0+0 (secure 0 0 0) 0x7f0aa409a900 con 0x7f0ab80b95e0 2024-09-17T04:29:21.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.350+0000 7f0abd000640 1 --2- 172.21.15.100:0/2727250513 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0a98078cb0 0x7f0a9807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:21.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.350+0000 7f0abd000640 1 --2- 172.21.15.100:0/2727250513 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0a98078cb0 0x7f0a9807b1c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0aac034000 tx=0x7f0aac004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:21.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.354+0000 7f0ab4c00640 1 -- 172.21.15.100:0/2727250513 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0ab80b1ff0 con 0x7f0ab80b95e0 2024-09-17T04:29:21.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:21.539+0000 7f0abe400640 1 -- 172.21.15.100:0/2727250513 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f0ab81f9030 con 0x7f0ab80b95e0 2024-09-17T04:29:22.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.085+0000 7f0ab4c00640 1 -- 172.21.15.100:0/2727250513 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) ==== 170+0+0 (secure 0 0 0) 0x7f0ab806eca0 con 0x7f0ab80b95e0 2024-09-17T04:29:22.086 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' created 2024-09-17T04:29:22.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.088+0000 7f0abe400640 1 -- 172.21.15.100:0/2727250513 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0a98078cb0 msgr2=0x7f0a9807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:22.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.088+0000 7f0abe400640 1 --2- 172.21.15.100:0/2727250513 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0a98078cb0 0x7f0a9807b1c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0aac034000 tx=0x7f0aac004730 comp rx=0 tx=0).stop 2024-09-17T04:29:22.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.088+0000 7f0abe400640 1 -- 172.21.15.100:0/2727250513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ab80b95e0 msgr2=0x7f0ab814a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:22.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.088+0000 7f0abe400640 1 --2- 172.21.15.100:0/2727250513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ab80b95e0 0x7f0ab814a270 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f0aa400c730 tx=0x7f0aa400cb30 comp rx=0 tx=0).stop 2024-09-17T04:29:22.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.089+0000 7f0abe400640 1 -- 172.21.15.100:0/2727250513 shutdown_connections 2024-09-17T04:29:22.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.089+0000 7f0abe400640 1 --2- 172.21.15.100:0/2727250513 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0a98078cb0 0x7f0a9807b1c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:22.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.089+0000 7f0abe400640 1 --2- 172.21.15.100:0/2727250513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ab80b16a0 0x7f0ab80b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:22.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.089+0000 7f0abe400640 1 --2- 172.21.15.100:0/2727250513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ab80b95e0 0x7f0ab814a270 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:22.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.089+0000 7f0abe400640 1 --2- 172.21.15.100:0/2727250513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ab806e890 0x7f0ab814d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:22.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.089+0000 7f0abe400640 1 -- 172.21.15.100:0/2727250513 >> 172.21.15.100:0/2727250513 conn(0x7f0ab809d860 msgr2=0x7f0ab806dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:22.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.089+0000 7f0abe400640 1 -- 172.21.15.100:0/2727250513 shutdown_connections 2024-09-17T04:29:22.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.089+0000 7f0abe400640 1 -- 172.21.15.100:0/2727250513 wait complete. 2024-09-17T04:29:22.129 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T04:29:22.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.350+0000 7f1e6f000640 1 -- 172.21.15.100:0/1708799873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e680742b0 msgr2=0x7f1e680b42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:22.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.350+0000 7f1e6f000640 1 --2- 172.21.15.100:0/1708799873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e680742b0 0x7f1e680b42a0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f1e680b52e0 tx=0x7f1e5c02a4d0 comp rx=0 tx=0).stop 2024-09-17T04:29:22.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.350+0000 7f1e6f000640 1 -- 172.21.15.100:0/1708799873 shutdown_connections 2024-09-17T04:29:22.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.350+0000 7f1e6f000640 1 --2- 172.21.15.100:0/1708799873 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e6815fbe0 0x7f1e680cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:22.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.351+0000 7f1e6f000640 1 --2- 172.21.15.100:0/1708799873 >> v2:172.21.15.184:3300/0 conn(0x7f1e681624b0 0x7f1e6815f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:22.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.351+0000 7f1e6f000640 1 --2- 172.21.15.100:0/1708799873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e680742b0 0x7f1e680b42a0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:22.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.351+0000 7f1e6f000640 1 -- 172.21.15.100:0/1708799873 >> 172.21.15.100:0/1708799873 conn(0x7f1e6806d620 msgr2=0x7f1e680b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:22.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.351+0000 7f1e6f000640 1 -- 172.21.15.100:0/1708799873 shutdown_connections 2024-09-17T04:29:22.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.351+0000 7f1e6f000640 1 -- 172.21.15.100:0/1708799873 wait complete. 2024-09-17T04:29:22.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.353+0000 7f1e6f000640 1 Processor -- start 2024-09-17T04:29:22.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.353+0000 7f1e6f000640 1 -- start start 2024-09-17T04:29:22.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.354+0000 7f1e6f000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e680742b0 0x7f1e681f9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:22.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.354+0000 7f1e6f000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e6815fbe0 0x7f1e68145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:22.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.354+0000 7f1e6f000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e681624b0 0x7f1e680ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:22.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.354+0000 7f1e6f000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1e68072eb0 con 0x7f1e680742b0 2024-09-17T04:29:22.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.354+0000 7f1e6f000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1e680cfb60 con 0x7f1e6815fbe0 2024-09-17T04:29:22.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.354+0000 7f1e6f000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1e68073030 con 0x7f1e681624b0 2024-09-17T04:29:22.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.355+0000 7f1e6dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e680742b0 0x7f1e681f9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:22.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.355+0000 7f1e6dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e680742b0 0x7f1e681f9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51888/0 (socket says 172.21.15.100:51888) 2024-09-17T04:29:22.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.355+0000 7f1e6dc00640 1 -- 172.21.15.100:0/629939242 learned_addr learned my addr 172.21.15.100:0/629939242 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:22.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.355+0000 7f1e6d200640 1 --2- 172.21.15.100:0/629939242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e6815fbe0 0x7f1e68145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:22.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.355+0000 7f1e6e600640 1 --2- 172.21.15.100:0/629939242 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e681624b0 0x7f1e680ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:22.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.355+0000 7f1e6dc00640 1 -- 172.21.15.100:0/629939242 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e681624b0 msgr2=0x7f1e680ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:22.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.355+0000 7f1e6dc00640 1 --2- 172.21.15.100:0/629939242 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e681624b0 0x7f1e680ae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:22.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.355+0000 7f1e6dc00640 1 -- 172.21.15.100:0/629939242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e6815fbe0 msgr2=0x7f1e68145ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:22.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.355+0000 7f1e6dc00640 1 --2- 172.21.15.100:0/629939242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e6815fbe0 0x7f1e68145ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:22.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.355+0000 7f1e6dc00640 1 -- 172.21.15.100:0/629939242 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e68149020 con 0x7f1e680742b0 2024-09-17T04:29:22.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.355+0000 7f1e6dc00640 1 --2- 172.21.15.100:0/629939242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e680742b0 0x7f1e681f9a20 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c034040 tx=0x7f1e5c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:22.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.356+0000 7f1e65600640 1 -- 172.21.15.100:0/629939242 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c03e030 con 0x7f1e680742b0 2024-09-17T04:29:22.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.356+0000 7f1e65600640 1 -- 172.21.15.100:0/629939242 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e5c003e00 con 0x7f1e680742b0 2024-09-17T04:29:22.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.356+0000 7f1e6f000640 1 -- 172.21.15.100:0/629939242 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e68149300 con 0x7f1e680742b0 2024-09-17T04:29:22.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.356+0000 7f1e65600640 1 -- 172.21.15.100:0/629939242 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c035040 con 0x7f1e680742b0 2024-09-17T04:29:22.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.356+0000 7f1e6f000640 1 -- 172.21.15.100:0/629939242 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e68147f60 con 0x7f1e680742b0 2024-09-17T04:29:22.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.357+0000 7f1e6f000640 1 -- 172.21.15.100:0/629939242 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e681481c0 con 0x7f1e680742b0 2024-09-17T04:29:22.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.358+0000 7f1e65600640 1 -- 172.21.15.100:0/629939242 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f1e5c041030 con 0x7f1e680742b0 2024-09-17T04:29:22.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.359+0000 7f1e65600640 1 --2- 172.21.15.100:0/629939242 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1e48078ce0 0x7f1e4807b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:22.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.359+0000 7f1e65600640 1 -- 172.21.15.100:0/629939242 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5609+0+0 (secure 0 0 0) 0x7f1e5c0bb6c0 con 0x7f1e680742b0 2024-09-17T04:29:22.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.360+0000 7f1e6d200640 1 --2- 172.21.15.100:0/629939242 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1e48078ce0 0x7f1e4807b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:22.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.360+0000 7f1e6d200640 1 --2- 172.21.15.100:0/629939242 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1e48078ce0 0x7f1e4807b1f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1e54005fd0 tx=0x7f1e54005850 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:22.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.364+0000 7f1e65600640 1 -- 172.21.15.100:0/629939242 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f1e5c083940 con 0x7f1e680742b0 2024-09-17T04:29:22.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:22.581+0000 7f1e6f000640 1 -- 172.21.15.100:0/629939242 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1e68073550 con 0x7f1e680742b0 2024-09-17T04:29:23.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.087+0000 7f1e65600640 1 -- 172.21.15.100:0/629939242 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v53) ==== 174+0+0 (secure 0 0 0) 0x7f1e5c083240 con 0x7f1e680742b0 2024-09-17T04:29:23.088 INFO:teuthology.orchestra.run.smithi100.stderr:new fs with metadata pool 12 and data pool 13 2024-09-17T04:29:23.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.091+0000 7f1e6f000640 1 -- 172.21.15.100:0/629939242 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1e48078ce0 msgr2=0x7f1e4807b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:23.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.091+0000 7f1e6f000640 1 --2- 172.21.15.100:0/629939242 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1e48078ce0 0x7f1e4807b1f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1e54005fd0 tx=0x7f1e54005850 comp rx=0 tx=0).stop 2024-09-17T04:29:23.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.091+0000 7f1e6f000640 1 -- 172.21.15.100:0/629939242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e680742b0 msgr2=0x7f1e681f9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:23.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.091+0000 7f1e6f000640 1 --2- 172.21.15.100:0/629939242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e680742b0 0x7f1e681f9a20 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c034040 tx=0x7f1e5c003910 comp rx=0 tx=0).stop 2024-09-17T04:29:23.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.091+0000 7f1e6f000640 1 -- 172.21.15.100:0/629939242 shutdown_connections 2024-09-17T04:29:23.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.091+0000 7f1e6f000640 1 --2- 172.21.15.100:0/629939242 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1e48078ce0 0x7f1e4807b1f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:23.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.091+0000 7f1e6f000640 1 --2- 172.21.15.100:0/629939242 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e681624b0 0x7f1e680ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:23.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.091+0000 7f1e6f000640 1 --2- 172.21.15.100:0/629939242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e6815fbe0 0x7f1e68145ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:23.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.091+0000 7f1e6f000640 1 --2- 172.21.15.100:0/629939242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e680742b0 0x7f1e681f9a20 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:23.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.091+0000 7f1e6f000640 1 -- 172.21.15.100:0/629939242 >> 172.21.15.100:0/629939242 conn(0x7f1e6806d620 msgr2=0x7f1e6815fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:23.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.091+0000 7f1e6f000640 1 -- 172.21.15.100:0/629939242 shutdown_connections 2024-09-17T04:29:23.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.091+0000 7f1e6f000640 1 -- 172.21.15.100:0/629939242 wait complete. 2024-09-17T04:29:23.138 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:29:23.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.361+0000 7f450f800640 1 -- 172.21.15.100:0/2505644966 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f450815d760 msgr2=0x7f45081535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:23.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.361+0000 7f450f800640 1 --2- 172.21.15.100:0/2505644966 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f450815d760 0x7f45081535c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f44f8009830 tx=0x7f44f802a480 comp rx=0 tx=0).stop 2024-09-17T04:29:23.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.361+0000 7f450f800640 1 -- 172.21.15.100:0/2505644966 shutdown_connections 2024-09-17T04:29:23.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.361+0000 7f450f800640 1 --2- 172.21.15.100:0/2505644966 >> v2:172.21.15.184:3300/0 conn(0x7f45080ad0e0 0x7f45080ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:23.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.361+0000 7f450f800640 1 --2- 172.21.15.100:0/2505644966 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f450815d760 0x7f45081535c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:23.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.361+0000 7f450f800640 1 -- 172.21.15.100:0/2505644966 >> 172.21.15.100:0/2505644966 conn(0x7f450806d730 msgr2=0x7f45080b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:23.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.361+0000 7f450f800640 1 -- 172.21.15.100:0/2505644966 shutdown_connections 2024-09-17T04:29:23.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.361+0000 7f450f800640 1 -- 172.21.15.100:0/2505644966 wait complete. 2024-09-17T04:29:23.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.363+0000 7f450f800640 1 Processor -- start 2024-09-17T04:29:23.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.363+0000 7f450f800640 1 -- start start 2024-09-17T04:29:23.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.364+0000 7f450f800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45080ad0e0 0x7f45081f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:23.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.364+0000 7f450f800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f450815d760 0x7f4508153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:23.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.365+0000 7f450f800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4508160310 0x7f450809b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:23.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.365+0000 7f450f800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f450806f220 con 0x7f4508160310 2024-09-17T04:29:23.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.365+0000 7f450f800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f450806f0a0 con 0x7f450815d760 2024-09-17T04:29:23.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.365+0000 7f450f800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f45080b4c20 con 0x7f45080ad0e0 2024-09-17T04:29:23.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.365+0000 7f450d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4508160310 0x7f450809b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:23.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.365+0000 7f450d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4508160310 0x7f450809b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51904/0 (socket says 172.21.15.100:51904) 2024-09-17T04:29:23.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.365+0000 7f44ffe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f450815d760 0x7f4508153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:23.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.365+0000 7f450d800640 1 -- 172.21.15.100:0/308929067 learned_addr learned my addr 172.21.15.100:0/308929067 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:23.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.365+0000 7f450ce00640 1 --2- 172.21.15.100:0/308929067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45080ad0e0 0x7f45081f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:23.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.365+0000 7f450d800640 1 -- 172.21.15.100:0/308929067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45080ad0e0 msgr2=0x7f45081f3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:23.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.365+0000 7f450d800640 1 --2- 172.21.15.100:0/308929067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45080ad0e0 0x7f45081f3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:23.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.365+0000 7f450d800640 1 -- 172.21.15.100:0/308929067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f450815d760 msgr2=0x7f4508153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:23.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.365+0000 7f450d800640 1 --2- 172.21.15.100:0/308929067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f450815d760 0x7f4508153e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:23.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.365+0000 7f450d800640 1 -- 172.21.15.100:0/308929067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f450809a600 con 0x7f4508160310 2024-09-17T04:29:23.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.365+0000 7f44ffe00640 1 --2- 172.21.15.100:0/308929067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f450815d760 0x7f4508153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:29:23.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.366+0000 7f450d800640 1 --2- 172.21.15.100:0/308929067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4508160310 0x7f450809b6a0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f450000caa0 tx=0x7f450000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:23.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.366+0000 7f4505600640 1 -- 172.21.15.100:0/308929067 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4500016020 con 0x7f4508160310 2024-09-17T04:29:23.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.366+0000 7f4505600640 1 -- 172.21.15.100:0/308929067 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f450000acf0 con 0x7f4508160310 2024-09-17T04:29:23.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.366+0000 7f450f800640 1 -- 172.21.15.100:0/308929067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f450809a8e0 con 0x7f4508160310 2024-09-17T04:29:23.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.366+0000 7f4505600640 1 -- 172.21.15.100:0/308929067 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4500012680 con 0x7f4508160310 2024-09-17T04:29:23.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.366+0000 7f450f800640 1 -- 172.21.15.100:0/308929067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f450809aea0 con 0x7f4508160310 2024-09-17T04:29:23.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.367+0000 7f450f800640 1 -- 172.21.15.100:0/308929067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4508160fa0 con 0x7f4508160310 2024-09-17T04:29:23.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.368+0000 7f4505600640 1 -- 172.21.15.100:0/308929067 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f4500019020 con 0x7f4508160310 2024-09-17T04:29:23.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.369+0000 7f4505600640 1 --2- 172.21.15.100:0/308929067 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f44e4078ce0 0x7f44e407b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:23.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.369+0000 7f4505600640 1 -- 172.21.15.100:0/308929067 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7f450009b450 con 0x7f4508160310 2024-09-17T04:29:23.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.370+0000 7f450ce00640 1 --2- 172.21.15.100:0/308929067 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f44e4078ce0 0x7f44e407b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:23.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.370+0000 7f450ce00640 1 --2- 172.21.15.100:0/308929067 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f44e4078ce0 0x7f44e407b1f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f44f8034000 tx=0x7f44f8004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:23.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.374+0000 7f4505600640 1 -- 172.21.15.100:0/308929067 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4500020020 con 0x7f4508160310 2024-09-17T04:29:23.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.558+0000 7f450f800640 1 -- 172.21.15.100:0/308929067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f450809a2d0 con 0x7f4508160310 2024-09-17T04:29:23.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.560+0000 7f4505600640 1 -- 172.21.15.100:0/308929067 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+16781 (secure 0 0 0) 0x7f45080ada30 con 0x7f4508160310 2024-09-17T04:29:23.560 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:23.561 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":86,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:29:22.583187+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:29:20.485434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-17T04:29:21.553185+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:29:23.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.562+0000 7f450f800640 1 -- 172.21.15.100:0/308929067 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f44e4078ce0 msgr2=0x7f44e407b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:23.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.562+0000 7f450f800640 1 --2- 172.21.15.100:0/308929067 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f44e4078ce0 0x7f44e407b1f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f44f8034000 tx=0x7f44f8004730 comp rx=0 tx=0).stop 2024-09-17T04:29:23.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.562+0000 7f450f800640 1 -- 172.21.15.100:0/308929067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4508160310 msgr2=0x7f450809b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:23.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.562+0000 7f450f800640 1 --2- 172.21.15.100:0/308929067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4508160310 0x7f450809b6a0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f450000caa0 tx=0x7f450000cf60 comp rx=0 tx=0).stop 2024-09-17T04:29:23.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.562+0000 7f450f800640 1 -- 172.21.15.100:0/308929067 shutdown_connections 2024-09-17T04:29:23.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.562+0000 7f450f800640 1 --2- 172.21.15.100:0/308929067 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f44e4078ce0 0x7f44e407b1f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:23.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.562+0000 7f450f800640 1 --2- 172.21.15.100:0/308929067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4508160310 0x7f450809b6a0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:23.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.562+0000 7f450f800640 1 --2- 172.21.15.100:0/308929067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f450815d760 0x7f4508153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:23.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.562+0000 7f450f800640 1 --2- 172.21.15.100:0/308929067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45080ad0e0 0x7f45081f3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:23.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.562+0000 7f450f800640 1 -- 172.21.15.100:0/308929067 >> 172.21.15.100:0/308929067 conn(0x7f450806d730 msgr2=0x7f4508160720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:23.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.562+0000 7f450f800640 1 -- 172.21.15.100:0/308929067 shutdown_connections 2024-09-17T04:29:23.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.563+0000 7f450f800640 1 -- 172.21.15.100:0/308929067 wait complete. 2024-09-17T04:29:23.585 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:29:23.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.804+0000 7ff903600640 1 -- 172.21.15.100:0/451527233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8fc0aebc0 msgr2=0x7ff8fc152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:23.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.804+0000 7ff903600640 1 --2- 172.21.15.100:0/451527233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8fc0aebc0 0x7ff8fc152920 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009830 tx=0x7ff8ec02a480 comp rx=0 tx=0).stop 2024-09-17T04:29:23.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.804+0000 7ff903600640 1 -- 172.21.15.100:0/451527233 shutdown_connections 2024-09-17T04:29:23.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.804+0000 7ff903600640 1 --2- 172.21.15.100:0/451527233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8fc0aebc0 0x7ff8fc152920 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:23.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.804+0000 7ff903600640 1 -- 172.21.15.100:0/451527233 >> 172.21.15.100:0/451527233 conn(0x7ff8fc06d730 msgr2=0x7ff8fc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:23.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.804+0000 7ff903600640 1 -- 172.21.15.100:0/451527233 shutdown_connections 2024-09-17T04:29:23.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.804+0000 7ff903600640 1 -- 172.21.15.100:0/451527233 wait complete. 2024-09-17T04:29:23.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.807+0000 7ff903600640 1 Processor -- start 2024-09-17T04:29:23.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.807+0000 7ff903600640 1 -- start start 2024-09-17T04:29:23.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.808+0000 7ff903600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8fc0aebc0 0x7ff8fc1f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:23.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.808+0000 7ff903600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8fc0b2750 0x7ff8fc151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:23.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.808+0000 7ff903600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8fc0746c0 0x7ff8fc0cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:23.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.808+0000 7ff903600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc0b4360 con 0x7ff8fc0746c0 2024-09-17T04:29:23.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.808+0000 7ff903600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc0b41e0 con 0x7ff8fc0b2750 2024-09-17T04:29:23.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.808+0000 7ff903600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff8fc0b44e0 con 0x7ff8fc0aebc0 2024-09-17T04:29:23.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.808+0000 7ff901600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8fc0746c0 0x7ff8fc0cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:23.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.808+0000 7ff901600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8fc0746c0 0x7ff8fc0cda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51922/0 (socket says 172.21.15.100:51922) 2024-09-17T04:29:23.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.808+0000 7ff901600640 1 -- 172.21.15.100:0/1603881914 learned_addr learned my addr 172.21.15.100:0/1603881914 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:23.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.809+0000 7ff900c00640 1 --2- 172.21.15.100:0/1603881914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8fc0aebc0 0x7ff8fc1f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:23.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.809+0000 7ff8f3e00640 1 --2- 172.21.15.100:0/1603881914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8fc0b2750 0x7ff8fc151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:23.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.809+0000 7ff901600640 1 -- 172.21.15.100:0/1603881914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8fc0aebc0 msgr2=0x7ff8fc1f4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:23.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.809+0000 7ff901600640 1 --2- 172.21.15.100:0/1603881914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8fc0aebc0 0x7ff8fc1f4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:23.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.809+0000 7ff901600640 1 -- 172.21.15.100:0/1603881914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8fc0b2750 msgr2=0x7ff8fc151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:23.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.809+0000 7ff901600640 1 --2- 172.21.15.100:0/1603881914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8fc0b2750 0x7ff8fc151c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:23.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.809+0000 7ff901600640 1 -- 172.21.15.100:0/1603881914 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8fc073620 con 0x7ff8fc0746c0 2024-09-17T04:29:23.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.809+0000 7ff901600640 1 --2- 172.21.15.100:0/1603881914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8fc0746c0 0x7ff8fc0cda30 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff8f400c730 tx=0x7ff8f400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:23.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.809+0000 7ff8f9600640 1 -- 172.21.15.100:0/1603881914 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4003b70 con 0x7ff8fc0746c0 2024-09-17T04:29:23.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.810+0000 7ff8f9600640 1 -- 172.21.15.100:0/1603881914 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8f4003d10 con 0x7ff8fc0746c0 2024-09-17T04:29:23.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.810+0000 7ff8f9600640 1 -- 172.21.15.100:0/1603881914 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f40052d0 con 0x7ff8fc0746c0 2024-09-17T04:29:23.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.810+0000 7ff903600640 1 -- 172.21.15.100:0/1603881914 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8fc073900 con 0x7ff8fc0746c0 2024-09-17T04:29:23.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.810+0000 7ff903600640 1 -- 172.21.15.100:0/1603881914 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8fc073ec0 con 0x7ff8fc0746c0 2024-09-17T04:29:23.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.811+0000 7ff903600640 1 -- 172.21.15.100:0/1603881914 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8fc0b49b0 con 0x7ff8fc0746c0 2024-09-17T04:29:23.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.812+0000 7ff8f9600640 1 -- 172.21.15.100:0/1603881914 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7ff8f4005470 con 0x7ff8fc0746c0 2024-09-17T04:29:23.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.813+0000 7ff8f9600640 1 --2- 172.21.15.100:0/1603881914 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff8d8078d10 0x7ff8d807b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:23.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.813+0000 7ff8f9600640 1 -- 172.21.15.100:0/1603881914 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7ff8f409ac50 con 0x7ff8fc0746c0 2024-09-17T04:29:23.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.813+0000 7ff900c00640 1 --2- 172.21.15.100:0/1603881914 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff8d8078d10 0x7ff8d807b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:23.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.814+0000 7ff900c00640 1 --2- 172.21.15.100:0/1603881914 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff8d8078d10 0x7ff8d807b220 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec034000 tx=0x7ff8ec004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:23.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:23.817+0000 7ff8f9600640 1 -- 172.21.15.100:0/1603881914 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff8f40a0020 con 0x7ff8fc0746c0 2024-09-17T04:29:24.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.001+0000 7ff903600640 1 -- 172.21.15.100:0/1603881914 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8fc15f670 con 0x7ff8fc0746c0 2024-09-17T04:29:24.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.002+0000 7ff8f9600640 1 -- 172.21.15.100:0/1603881914 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+16781 (secure 0 0 0) 0x7ff8f4062820 con 0x7ff8fc0746c0 2024-09-17T04:29:24.003 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:24.003 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":86,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:29:22.583187+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:29:20.485434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-17T04:29:21.553185+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:29:24.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.005+0000 7ff903600640 1 -- 172.21.15.100:0/1603881914 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff8d8078d10 msgr2=0x7ff8d807b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:24.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.005+0000 7ff903600640 1 --2- 172.21.15.100:0/1603881914 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff8d8078d10 0x7ff8d807b220 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec034000 tx=0x7ff8ec004730 comp rx=0 tx=0).stop 2024-09-17T04:29:24.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.005+0000 7ff903600640 1 -- 172.21.15.100:0/1603881914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8fc0746c0 msgr2=0x7ff8fc0cda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:24.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.005+0000 7ff903600640 1 --2- 172.21.15.100:0/1603881914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8fc0746c0 0x7ff8fc0cda30 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff8f400c730 tx=0x7ff8f400cb30 comp rx=0 tx=0).stop 2024-09-17T04:29:24.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.005+0000 7ff903600640 1 -- 172.21.15.100:0/1603881914 shutdown_connections 2024-09-17T04:29:24.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.005+0000 7ff903600640 1 --2- 172.21.15.100:0/1603881914 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff8d8078d10 0x7ff8d807b220 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:24.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.005+0000 7ff903600640 1 --2- 172.21.15.100:0/1603881914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8fc0746c0 0x7ff8fc0cda30 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:24.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.005+0000 7ff903600640 1 --2- 172.21.15.100:0/1603881914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8fc0b2750 0x7ff8fc151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:24.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.005+0000 7ff903600640 1 --2- 172.21.15.100:0/1603881914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8fc0aebc0 0x7ff8fc1f4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:24.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.005+0000 7ff903600640 1 -- 172.21.15.100:0/1603881914 >> 172.21.15.100:0/1603881914 conn(0x7ff8fc06d730 msgr2=0x7ff8fc06fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:24.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.005+0000 7ff903600640 1 -- 172.21.15.100:0/1603881914 shutdown_connections 2024-09-17T04:29:24.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.005+0000 7ff903600640 1 -- 172.21.15.100:0/1603881914 wait complete. 2024-09-17T04:29:24.030 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T04:29:24.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.255+0000 7fd25dc00640 1 -- 172.21.15.100:0/2023175184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2580b3f30 msgr2=0x7fd258156ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:24.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.255+0000 7fd25dc00640 1 --2- 172.21.15.100:0/2023175184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2580b3f30 0x7fd258156ca0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd248005fd0 tx=0x7fd24802a5d0 comp rx=0 tx=0).stop 2024-09-17T04:29:24.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.255+0000 7fd25dc00640 1 -- 172.21.15.100:0/2023175184 shutdown_connections 2024-09-17T04:29:24.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.255+0000 7fd25dc00640 1 --2- 172.21.15.100:0/2023175184 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd25815d6e0 0x7fd2580ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:24.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.255+0000 7fd25dc00640 1 --2- 172.21.15.100:0/2023175184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2580b3f30 0x7fd258156ca0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:24.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.255+0000 7fd25dc00640 1 -- 172.21.15.100:0/2023175184 >> 172.21.15.100:0/2023175184 conn(0x7fd25806d620 msgr2=0x7fd2580b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:24.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.255+0000 7fd25dc00640 1 -- 172.21.15.100:0/2023175184 shutdown_connections 2024-09-17T04:29:24.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.255+0000 7fd25dc00640 1 -- 172.21.15.100:0/2023175184 wait complete. 2024-09-17T04:29:24.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.258+0000 7fd25dc00640 1 Processor -- start 2024-09-17T04:29:24.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.258+0000 7fd25dc00640 1 -- start start 2024-09-17T04:29:24.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.259+0000 7fd25dc00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2580b3f30 0x7fd25809b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:24.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.259+0000 7fd25dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2580cd000 0x7fd2581e9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:24.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.259+0000 7fd25dc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd25815d6e0 0x7fd2581e9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:24.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.259+0000 7fd25dc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd2580af960 con 0x7fd2580cd000 2024-09-17T04:29:24.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.259+0000 7fd25dc00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd2580af7e0 con 0x7fd25815d6e0 2024-09-17T04:29:24.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.259+0000 7fd25dc00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd2580afae0 con 0x7fd2580b3f30 2024-09-17T04:29:24.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.260+0000 7fd257400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2580cd000 0x7fd2581e9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:24.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.260+0000 7fd257400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2580cd000 0x7fd2581e9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51946/0 (socket says 172.21.15.100:51946) 2024-09-17T04:29:24.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.260+0000 7fd257400640 1 -- 172.21.15.100:0/493241999 learned_addr learned my addr 172.21.15.100:0/493241999 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:24.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.260+0000 7fd257e00640 1 --2- 172.21.15.100:0/493241999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2580b3f30 0x7fd25809b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:24.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.260+0000 7fd25d200640 1 --2- 172.21.15.100:0/493241999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd25815d6e0 0x7fd2581e9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:24.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.260+0000 7fd257400640 1 -- 172.21.15.100:0/493241999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2580b3f30 msgr2=0x7fd25809b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:24.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.260+0000 7fd257400640 1 --2- 172.21.15.100:0/493241999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2580b3f30 0x7fd25809b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:24.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.260+0000 7fd257400640 1 -- 172.21.15.100:0/493241999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd25815d6e0 msgr2=0x7fd2581e9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:24.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.260+0000 7fd257400640 1 --2- 172.21.15.100:0/493241999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd25815d6e0 0x7fd2581e9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:24.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.260+0000 7fd257400640 1 -- 172.21.15.100:0/493241999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2581f8c90 con 0x7fd2580cd000 2024-09-17T04:29:24.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.260+0000 7fd25d200640 1 --2- 172.21.15.100:0/493241999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd25815d6e0 0x7fd2581e9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:24.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.260+0000 7fd257400640 1 --2- 172.21.15.100:0/493241999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2580cd000 0x7fd2581e9350 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fd24400d6a0 tx=0x7fd24400db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:24.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.261+0000 7fd24fe00640 1 -- 172.21.15.100:0/493241999 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd244017070 con 0x7fd2580cd000 2024-09-17T04:29:24.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.261+0000 7fd24fe00640 1 -- 172.21.15.100:0/493241999 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd244005310 con 0x7fd2580cd000 2024-09-17T04:29:24.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.261+0000 7fd24fe00640 1 -- 172.21.15.100:0/493241999 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd244009820 con 0x7fd2580cd000 2024-09-17T04:29:24.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.261+0000 7fd25dc00640 1 -- 172.21.15.100:0/493241999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2581f8f70 con 0x7fd2580cd000 2024-09-17T04:29:24.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.261+0000 7fd25dc00640 1 -- 172.21.15.100:0/493241999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2581f9530 con 0x7fd2580cd000 2024-09-17T04:29:24.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.262+0000 7fd25dc00640 1 -- 172.21.15.100:0/493241999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd258156ca0 con 0x7fd2580cd000 2024-09-17T04:29:24.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.263+0000 7fd24fe00640 1 -- 172.21.15.100:0/493241999 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fd2440099c0 con 0x7fd2580cd000 2024-09-17T04:29:24.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.264+0000 7fd24fe00640 1 --2- 172.21.15.100:0/493241999 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd238078cb0 0x7fd23807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:24.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.264+0000 7fd24fe00640 1 -- 172.21.15.100:0/493241999 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fd24409b180 con 0x7fd2580cd000 2024-09-17T04:29:24.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.264+0000 7fd257e00640 1 --2- 172.21.15.100:0/493241999 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd238078cb0 0x7fd23807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:24.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.265+0000 7fd257e00640 1 --2- 172.21.15.100:0/493241999 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd238078cb0 0x7fd23807b1c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd248033000 tx=0x7fd248004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:24.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.268+0000 7fd24fe00640 1 -- 172.21.15.100:0/493241999 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd244063300 con 0x7fd2580cd000 2024-09-17T04:29:24.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:24.491+0000 7fd25dc00640 1 -- 172.21.15.100:0/493241999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd2581567e0 con 0x7fd2580cd000 2024-09-17T04:29:25.070 INFO:tasks.ceph.mds.c.smithi100.stderr:2024-09-17T04:29:25.070+0000 7fad70400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T04:29:25.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.127+0000 7fd24fe00640 1 -- 172.21.15.100:0/493241999 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v56) ==== 118+0+0 (secure 0 0 0) 0x7fd244062c00 con 0x7fd2580cd000 2024-09-17T04:29:25.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.130+0000 7fd25dc00640 1 -- 172.21.15.100:0/493241999 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd238078cb0 msgr2=0x7fd23807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:25.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.130+0000 7fd25dc00640 1 --2- 172.21.15.100:0/493241999 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd238078cb0 0x7fd23807b1c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd248033000 tx=0x7fd248004730 comp rx=0 tx=0).stop 2024-09-17T04:29:25.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.130+0000 7fd25dc00640 1 -- 172.21.15.100:0/493241999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2580cd000 msgr2=0x7fd2581e9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:25.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.130+0000 7fd25dc00640 1 --2- 172.21.15.100:0/493241999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2580cd000 0x7fd2581e9350 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fd24400d6a0 tx=0x7fd24400db60 comp rx=0 tx=0).stop 2024-09-17T04:29:25.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.130+0000 7fd25dc00640 1 -- 172.21.15.100:0/493241999 shutdown_connections 2024-09-17T04:29:25.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.130+0000 7fd25dc00640 1 --2- 172.21.15.100:0/493241999 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd238078cb0 0x7fd23807b1c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:25.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.130+0000 7fd25dc00640 1 --2- 172.21.15.100:0/493241999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd25815d6e0 0x7fd2581e9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:25.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.130+0000 7fd25dc00640 1 --2- 172.21.15.100:0/493241999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2580cd000 0x7fd2581e9350 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:25.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.130+0000 7fd25dc00640 1 --2- 172.21.15.100:0/493241999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2580b3f30 0x7fd25809b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:25.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.130+0000 7fd25dc00640 1 -- 172.21.15.100:0/493241999 >> 172.21.15.100:0/493241999 conn(0x7fd25806d620 msgr2=0x7fd2580ae3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:25.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.130+0000 7fd25dc00640 1 -- 172.21.15.100:0/493241999 shutdown_connections 2024-09-17T04:29:25.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.130+0000 7fd25dc00640 1 -- 172.21.15.100:0/493241999 wait complete. 2024-09-17T04:29:25.155 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T04:29:25.155 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T04:29:25.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.380+0000 7f4bdf800640 1 -- 172.21.15.100:0/34724763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bbc004d40 msgr2=0x7f4bbc005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:25.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.380+0000 7f4bdf800640 1 --2- 172.21.15.100:0/34724763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bbc004d40 0x7f4bbc005150 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0005fd0 tx=0x7f4bbc005830 comp rx=0 tx=0).stop 2024-09-17T04:29:25.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.380+0000 7f4bdd800640 1 -- 172.21.15.100:0/34724763 reap_dead start 2024-09-17T04:29:25.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.381+0000 7f4bdf800640 1 -- 172.21.15.100:0/34724763 shutdown_connections 2024-09-17T04:29:25.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.381+0000 7f4bdf800640 1 -- 172.21.15.100:0/34724763 >> 172.21.15.100:0/34724763 conn(0x7f4bd806d1c0 msgr2=0x7f4bd806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:25.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.381+0000 7f4bdf800640 1 -- 172.21.15.100:0/34724763 shutdown_connections 2024-09-17T04:29:25.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.381+0000 7f4bdf800640 1 -- 172.21.15.100:0/34724763 wait complete. 2024-09-17T04:29:25.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.383+0000 7f4bdf800640 1 Processor -- start 2024-09-17T04:29:25.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.383+0000 7f4bdf800640 1 -- start start 2024-09-17T04:29:25.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.384+0000 7f4bdf800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4bbc004d40 0x7f4bd81585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:25.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.384+0000 7f4bdf800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bd8155a10 0x7f4bd8142c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:25.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.384+0000 7f4bdf800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bd80b21e0 0x7f4bd80b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:25.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.384+0000 7f4bdf800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd8073c70 con 0x7f4bd80b21e0 2024-09-17T04:29:25.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.384+0000 7f4bdf800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd8073af0 con 0x7f4bd8155a10 2024-09-17T04:29:25.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.384+0000 7f4bdf800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4bd80b3d70 con 0x7f4bbc004d40 2024-09-17T04:29:25.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.385+0000 7f4bdd800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bd80b21e0 0x7f4bd80b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:25.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.385+0000 7f4bdd800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bd80b21e0 0x7f4bd80b25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33004/0 (socket says 172.21.15.100:33004) 2024-09-17T04:29:25.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.385+0000 7f4bdd800640 1 -- 172.21.15.100:0/4268698930 learned_addr learned my addr 172.21.15.100:0/4268698930 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:25.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.385+0000 7f4bcfe00640 1 --2- 172.21.15.100:0/4268698930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bd8155a10 0x7f4bd8142c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:25.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.385+0000 7f4bdce00640 1 --2- 172.21.15.100:0/4268698930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4bbc004d40 0x7f4bd81585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:25.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.385+0000 7f4bdd800640 1 -- 172.21.15.100:0/4268698930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4bbc004d40 msgr2=0x7f4bd81585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:25.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.385+0000 7f4bdd800640 1 --2- 172.21.15.100:0/4268698930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4bbc004d40 0x7f4bd81585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:25.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.385+0000 7f4bdd800640 1 -- 172.21.15.100:0/4268698930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bd8155a10 msgr2=0x7f4bd8142c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:25.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.385+0000 7f4bdd800640 1 --2- 172.21.15.100:0/4268698930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bd8155a10 0x7f4bd8142c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:25.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.385+0000 7f4bdd800640 1 -- 172.21.15.100:0/4268698930 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd80b2b30 con 0x7f4bd80b21e0 2024-09-17T04:29:25.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.385+0000 7f4bdce00640 1 --2- 172.21.15.100:0/4268698930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4bbc004d40 0x7f4bd81585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:25.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.385+0000 7f4bdd800640 1 --2- 172.21.15.100:0/4268698930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bd80b21e0 0x7f4bd80b25f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000f550 tx=0x7f4bd0034f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:25.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.386+0000 7f4bd5600640 1 -- 172.21.15.100:0/4268698930 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0039040 con 0x7f4bd80b21e0 2024-09-17T04:29:25.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.386+0000 7f4bd5600640 1 -- 172.21.15.100:0/4268698930 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4bd0003c60 con 0x7f4bd80b21e0 2024-09-17T04:29:25.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.386+0000 7f4bd5600640 1 -- 172.21.15.100:0/4268698930 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd000abe0 con 0x7f4bd80b21e0 2024-09-17T04:29:25.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.386+0000 7f4bdf800640 1 -- 172.21.15.100:0/4268698930 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd81f9c10 con 0x7f4bd80b21e0 2024-09-17T04:29:25.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.386+0000 7f4bdf800640 1 -- 172.21.15.100:0/4268698930 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd80b0470 con 0x7f4bd80b21e0 2024-09-17T04:29:25.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.387+0000 7f4bdf800640 1 -- 172.21.15.100:0/4268698930 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bd81f9980 con 0x7f4bd80b21e0 2024-09-17T04:29:25.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.388+0000 7f4bd5600640 1 -- 172.21.15.100:0/4268698930 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f4bd0047020 con 0x7f4bd80b21e0 2024-09-17T04:29:25.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.389+0000 7f4bd5600640 1 --2- 172.21.15.100:0/4268698930 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4bb4078cb0 0x7f4bb407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:25.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.389+0000 7f4bd5600640 1 -- 172.21.15.100:0/4268698930 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f4bd00bbc80 con 0x7f4bd80b21e0 2024-09-17T04:29:25.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.389+0000 7f4bdce00640 1 --2- 172.21.15.100:0/4268698930 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4bb4078cb0 0x7f4bb407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:25.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.390+0000 7f4bdce00640 1 --2- 172.21.15.100:0/4268698930 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4bb4078cb0 0x7f4bb407b1c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8001330 tx=0x7f4bc8001180 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:25.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.394+0000 7f4bd5600640 1 -- 172.21.15.100:0/4268698930 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4bd80b2b30 con 0x7f4bd80b21e0 2024-09-17T04:29:25.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:25.612+0000 7f4bdf800640 1 -- 172.21.15.100:0/4268698930 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f4bd80b4300 con 0x7f4bd80b21e0 2024-09-17T04:29:26.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.133+0000 7f4bd5600640 1 -- 172.21.15.100:0/4268698930 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v57) ==== 122+0+0 (secure 0 0 0) 0x7f4bd0083e90 con 0x7f4bd80b21e0 2024-09-17T04:29:26.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.136+0000 7f4bdf800640 1 -- 172.21.15.100:0/4268698930 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4bb4078cb0 msgr2=0x7f4bb407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:26.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.136+0000 7f4bdf800640 1 --2- 172.21.15.100:0/4268698930 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4bb4078cb0 0x7f4bb407b1c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8001330 tx=0x7f4bc8001180 comp rx=0 tx=0).stop 2024-09-17T04:29:26.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.136+0000 7f4bdf800640 1 -- 172.21.15.100:0/4268698930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bd80b21e0 msgr2=0x7f4bd80b25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:26.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.136+0000 7f4bdf800640 1 --2- 172.21.15.100:0/4268698930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bd80b21e0 0x7f4bd80b25f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000f550 tx=0x7f4bd0034f60 comp rx=0 tx=0).stop 2024-09-17T04:29:26.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.136+0000 7f4bdf800640 1 -- 172.21.15.100:0/4268698930 shutdown_connections 2024-09-17T04:29:26.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.136+0000 7f4bdf800640 1 --2- 172.21.15.100:0/4268698930 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4bb4078cb0 0x7f4bb407b1c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:26.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.136+0000 7f4bdf800640 1 --2- 172.21.15.100:0/4268698930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bd80b21e0 0x7f4bd80b25f0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:26.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.136+0000 7f4bdf800640 1 --2- 172.21.15.100:0/4268698930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bd8155a10 0x7f4bd8142c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:26.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.136+0000 7f4bdf800640 1 --2- 172.21.15.100:0/4268698930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4bbc004d40 0x7f4bd81585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:26.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.136+0000 7f4bdf800640 1 -- 172.21.15.100:0/4268698930 >> 172.21.15.100:0/4268698930 conn(0x7f4bd806d1c0 msgr2=0x7f4bd815dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:26.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.136+0000 7f4bdf800640 1 -- 172.21.15.100:0/4268698930 shutdown_connections 2024-09-17T04:29:26.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.136+0000 7f4bdf800640 1 -- 172.21.15.100:0/4268698930 wait complete. 2024-09-17T04:29:26.163 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T04:29:26.163 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:29:26.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.386+0000 7fc291e00640 1 -- 172.21.15.100:0/2351139023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28c06d4b0 msgr2=0x7fc28c15c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:26.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.386+0000 7fc291e00640 1 --2- 172.21.15.100:0/2351139023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28c06d4b0 0x7fc28c15c060 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fc28c0cc670 tx=0x7fc27c02a530 comp rx=0 tx=0).stop 2024-09-17T04:29:26.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.387+0000 7fc291e00640 1 -- 172.21.15.100:0/2351139023 shutdown_connections 2024-09-17T04:29:26.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.387+0000 7fc291e00640 1 --2- 172.21.15.100:0/2351139023 >> v2:172.21.15.184:3300/0 conn(0x7fc28c0cd8f0 0x7fc28c15c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:26.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.387+0000 7fc291e00640 1 --2- 172.21.15.100:0/2351139023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28c06d4b0 0x7fc28c15c060 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:26.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.387+0000 7fc291e00640 1 -- 172.21.15.100:0/2351139023 >> 172.21.15.100:0/2351139023 conn(0x7fc28c0b2fd0 msgr2=0x7fc28c0b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:26.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.387+0000 7fc291e00640 1 -- 172.21.15.100:0/2351139023 shutdown_connections 2024-09-17T04:29:26.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.387+0000 7fc291e00640 1 -- 172.21.15.100:0/2351139023 wait complete. 2024-09-17T04:29:26.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.389+0000 7fc291e00640 1 Processor -- start 2024-09-17T04:29:26.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.389+0000 7fc291e00640 1 -- start start 2024-09-17T04:29:26.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.390+0000 7fc291e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28c0b2360 0x7fc28c1f32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:26.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.390+0000 7fc291e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc28c0cd8f0 0x7fc28c158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:26.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.391+0000 7fc291e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc28c1e48f0 0x7fc28c09e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:26.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.391+0000 7fc291e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c0b5000 con 0x7fc28c0b2360 2024-09-17T04:29:26.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.391+0000 7fc291e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c0b4e80 con 0x7fc28c0cd8f0 2024-09-17T04:29:26.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.391+0000 7fc291e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc28c0b5180 con 0x7fc28c1e48f0 2024-09-17T04:29:26.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.391+0000 7fc28aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28c0b2360 0x7fc28c1f32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:26.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.391+0000 7fc28aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28c0b2360 0x7fc28c1f32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33028/0 (socket says 172.21.15.100:33028) 2024-09-17T04:29:26.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.391+0000 7fc28aa00640 1 -- 172.21.15.100:0/3624033489 learned_addr learned my addr 172.21.15.100:0/3624033489 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:26.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.391+0000 7fc28aa00640 1 -- 172.21.15.100:0/3624033489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc28c1e48f0 msgr2=0x7fc28c09e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:26.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.391+0000 7fc28aa00640 1 --2- 172.21.15.100:0/3624033489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc28c1e48f0 0x7fc28c09e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:26.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.391+0000 7fc28aa00640 1 -- 172.21.15.100:0/3624033489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc28c0cd8f0 msgr2=0x7fc28c158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:26.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.391+0000 7fc28aa00640 1 --2- 172.21.15.100:0/3624033489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc28c0cd8f0 0x7fc28c158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:26.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.391+0000 7fc28aa00640 1 -- 172.21.15.100:0/3624033489 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc28c1591f0 con 0x7fc28c0b2360 2024-09-17T04:29:26.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.392+0000 7fc28aa00640 1 --2- 172.21.15.100:0/3624033489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28c0b2360 0x7fc28c1f32d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fc28400c7d0 tx=0x7fc28400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:26.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.392+0000 7fc283400640 1 -- 172.21.15.100:0/3624033489 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc284017070 con 0x7fc28c0b2360 2024-09-17T04:29:26.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.392+0000 7fc283400640 1 -- 172.21.15.100:0/3624033489 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc284005070 con 0x7fc28c0b2360 2024-09-17T04:29:26.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.392+0000 7fc283400640 1 -- 172.21.15.100:0/3624033489 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2840054a0 con 0x7fc28c0b2360 2024-09-17T04:29:26.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.392+0000 7fc291e00640 1 -- 172.21.15.100:0/3624033489 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc28c1e1c50 con 0x7fc28c0b2360 2024-09-17T04:29:26.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.392+0000 7fc291e00640 1 -- 172.21.15.100:0/3624033489 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc28c1f3f20 con 0x7fc28c0b2360 2024-09-17T04:29:26.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.393+0000 7fc291e00640 1 -- 172.21.15.100:0/3624033489 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc28c15c060 con 0x7fc28c0b2360 2024-09-17T04:29:26.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.394+0000 7fc283400640 1 -- 172.21.15.100:0/3624033489 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fc28401a070 con 0x7fc28c0b2360 2024-09-17T04:29:26.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.395+0000 7fc283400640 1 --2- 172.21.15.100:0/3624033489 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc26c078cb0 0x7fc26c07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:26.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.395+0000 7fc283400640 1 -- 172.21.15.100:0/3624033489 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fc28409a7a0 con 0x7fc28c0b2360 2024-09-17T04:29:26.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.396+0000 7fc28a000640 1 --2- 172.21.15.100:0/3624033489 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc26c078cb0 0x7fc26c07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:26.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.396+0000 7fc28a000640 1 --2- 172.21.15.100:0/3624033489 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc26c078cb0 0x7fc26c07b1c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc27c034000 tx=0x7fc27c0046c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:26.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.400+0000 7fc283400640 1 -- 172.21.15.100:0/3624033489 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc2840628a0 con 0x7fc28c0b2360 2024-09-17T04:29:26.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.616+0000 7fc291e00640 1 -- 172.21.15.100:0/3624033489 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc28c1e1e20 con 0x7fc28c0b2360 2024-09-17T04:29:26.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.617+0000 7fc283400640 1 -- 172.21.15.100:0/3624033489 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5237 (secure 0 0 0) 0x7fc2840621a0 con 0x7fc28c0b2360 2024-09-17T04:29:26.617 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:26.617 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":57,"btime":"2024-09-17T04:29:26:129057+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5825,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3480776805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3480776805},{"type":"v1","addr":"172.21.15.184:6835","nonce":3480776805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":6391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/3591753470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3591753470},{"type":"v1","addr":"172.21.15.100:6833","nonce":3591753470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6400,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/2063784289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2063784289},{"type":"v1","addr":"172.21.15.184:6801","nonce":2063784289}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:29:22.583149+0000","modified":"2024-09-17T04:29:25.614111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6606},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6606":{"gid":6606,"name":"c","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/2660333125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2660333125},{"type":"v1","addr":"172.21.15.100:6835","nonce":2660333125}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6606,"qdb_cluster":[6606]},"id":6}]} 2024-09-17T04:29:26.618 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 57 2024-09-17T04:29:26.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.619+0000 7fc291e00640 1 -- 172.21.15.100:0/3624033489 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc26c078cb0 msgr2=0x7fc26c07b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:26.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.619+0000 7fc291e00640 1 --2- 172.21.15.100:0/3624033489 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc26c078cb0 0x7fc26c07b1c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc27c034000 tx=0x7fc27c0046c0 comp rx=0 tx=0).stop 2024-09-17T04:29:26.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.619+0000 7fc291e00640 1 -- 172.21.15.100:0/3624033489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28c0b2360 msgr2=0x7fc28c1f32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:26.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.619+0000 7fc291e00640 1 --2- 172.21.15.100:0/3624033489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28c0b2360 0x7fc28c1f32d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fc28400c7d0 tx=0x7fc28400cc90 comp rx=0 tx=0).stop 2024-09-17T04:29:26.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.619+0000 7fc291e00640 1 -- 172.21.15.100:0/3624033489 shutdown_connections 2024-09-17T04:29:26.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.619+0000 7fc291e00640 1 --2- 172.21.15.100:0/3624033489 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc26c078cb0 0x7fc26c07b1c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:26.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.619+0000 7fc291e00640 1 --2- 172.21.15.100:0/3624033489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc28c1e48f0 0x7fc28c09e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:26.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.619+0000 7fc291e00640 1 --2- 172.21.15.100:0/3624033489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc28c0cd8f0 0x7fc28c158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:26.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.619+0000 7fc291e00640 1 --2- 172.21.15.100:0/3624033489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28c0b2360 0x7fc28c1f32d0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:26.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.619+0000 7fc291e00640 1 -- 172.21.15.100:0/3624033489 >> 172.21.15.100:0/3624033489 conn(0x7fc28c0b2fd0 msgr2=0x7fc28c0b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:26.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.619+0000 7fc291e00640 1 -- 172.21.15.100:0/3624033489 shutdown_connections 2024-09-17T04:29:26.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.619+0000 7fc291e00640 1 -- 172.21.15.100:0/3624033489 wait complete. 2024-09-17T04:29:26.641 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:29:26.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.862+0000 7fa5e3200640 1 -- 172.21.15.100:0/2070684529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5c4003a50 msgr2=0x7fa5c4003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:26.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.862+0000 7fa5e3200640 1 --2- 172.21.15.100:0/2070684529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5c4003a50 0x7fa5c4003ec0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fa5d00016c0 tx=0x7fa5c4005830 comp rx=0 tx=0).stop 2024-09-17T04:29:26.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.862+0000 7fa5e3200640 1 -- 172.21.15.100:0/2070684529 shutdown_connections 2024-09-17T04:29:26.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.862+0000 7fa5e3200640 1 --2- 172.21.15.100:0/2070684529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5c4003a50 0x7fa5c4003ec0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:26.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.862+0000 7fa5e3200640 1 -- 172.21.15.100:0/2070684529 >> 172.21.15.100:0/2070684529 conn(0x7fa5dc0b1e40 msgr2=0x7fa5dc0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:26.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.862+0000 7fa5e3200640 1 -- 172.21.15.100:0/2070684529 shutdown_connections 2024-09-17T04:29:26.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.862+0000 7fa5e3200640 1 -- 172.21.15.100:0/2070684529 wait complete. 2024-09-17T04:29:26.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.864+0000 7fa5e3200640 1 Processor -- start 2024-09-17T04:29:26.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.864+0000 7fa5e3200640 1 -- start start 2024-09-17T04:29:26.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.866+0000 7fa5e3200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5c4003a50 0x7fa5dc151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:26.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.866+0000 7fa5e3200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5dc150c00 0x7fa5dc14db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:26.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.866+0000 7fa5e3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5dc0b09a0 0x7fa5dc0b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:26.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.866+0000 7fa5e3200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc15a270 con 0x7fa5dc0b09a0 2024-09-17T04:29:26.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.866+0000 7fa5e3200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc15a0f0 con 0x7fa5c4003a50 2024-09-17T04:29:26.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.866+0000 7fa5e3200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa5dc15a3f0 con 0x7fa5dc150c00 2024-09-17T04:29:26.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.866+0000 7fa5e2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5dc0b09a0 0x7fa5dc0b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:26.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.866+0000 7fa5e2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5dc0b09a0 0x7fa5dc0b0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33042/0 (socket says 172.21.15.100:33042) 2024-09-17T04:29:26.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.866+0000 7fa5e2800640 1 -- 172.21.15.100:0/307653668 learned_addr learned my addr 172.21.15.100:0/307653668 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:26.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.866+0000 7fa5e1e00640 1 --2- 172.21.15.100:0/307653668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5c4003a50 0x7fa5dc151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:26.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.866+0000 7fa5e1400640 1 --2- 172.21.15.100:0/307653668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5dc150c00 0x7fa5dc14db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:26.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.866+0000 7fa5e2800640 1 -- 172.21.15.100:0/307653668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5dc150c00 msgr2=0x7fa5dc14db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:26.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.866+0000 7fa5e2800640 1 --2- 172.21.15.100:0/307653668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5dc150c00 0x7fa5dc14db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:26.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.866+0000 7fa5e2800640 1 -- 172.21.15.100:0/307653668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5c4003a50 msgr2=0x7fa5dc151150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:26.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.866+0000 7fa5e2800640 1 --2- 172.21.15.100:0/307653668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5c4003a50 0x7fa5dc151150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:26.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.866+0000 7fa5e2800640 1 -- 172.21.15.100:0/307653668 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5dc0b12f0 con 0x7fa5dc0b09a0 2024-09-17T04:29:26.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.867+0000 7fa5e2800640 1 --2- 172.21.15.100:0/307653668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5dc0b09a0 0x7fa5dc0b0db0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc15cc30 tx=0x7fa5d8010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:26.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.867+0000 7fa5d6000640 1 -- 172.21.15.100:0/307653668 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d8017070 con 0x7fa5dc0b09a0 2024-09-17T04:29:26.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.867+0000 7fa5d6000640 1 -- 172.21.15.100:0/307653668 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5d8015070 con 0x7fa5dc0b09a0 2024-09-17T04:29:26.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.867+0000 7fa5e3200640 1 -- 172.21.15.100:0/307653668 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5dc09ad30 con 0x7fa5dc0b09a0 2024-09-17T04:29:26.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.867+0000 7fa5d6000640 1 -- 172.21.15.100:0/307653668 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d801c6a0 con 0x7fa5dc0b09a0 2024-09-17T04:29:26.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.867+0000 7fa5e3200640 1 -- 172.21.15.100:0/307653668 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5dc09b2f0 con 0x7fa5dc0b09a0 2024-09-17T04:29:26.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.868+0000 7fa5e3200640 1 -- 172.21.15.100:0/307653668 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5dc15d750 con 0x7fa5dc0b09a0 2024-09-17T04:29:26.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.869+0000 7fa5d6000640 1 -- 172.21.15.100:0/307653668 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fa5d801c840 con 0x7fa5dc0b09a0 2024-09-17T04:29:26.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.870+0000 7fa5d6000640 1 --2- 172.21.15.100:0/307653668 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa5c0078cb0 0x7fa5c007b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:26.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.870+0000 7fa5d6000640 1 -- 172.21.15.100:0/307653668 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fa5d809bb10 con 0x7fa5dc0b09a0 2024-09-17T04:29:26.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.870+0000 7fa5e1e00640 1 --2- 172.21.15.100:0/307653668 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa5c0078cb0 0x7fa5c007b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:26.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.871+0000 7fa5e1e00640 1 --2- 172.21.15.100:0/307653668 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa5c0078cb0 0x7fa5c007b1c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0032000 tx=0x7fa5d002c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:26.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:26.875+0000 7fa5d6000640 1 -- 172.21.15.100:0/307653668 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa5dc0b12f0 con 0x7fa5dc0b09a0 2024-09-17T04:29:27.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.057+0000 7fa5e3200640 1 -- 172.21.15.100:0/307653668 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5dc06f670 con 0x7fa5dc0b09a0 2024-09-17T04:29:27.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.058+0000 7fa5d6000640 1 -- 172.21.15.100:0/307653668 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+16768 (secure 0 0 0) 0x7fa5d80636d0 con 0x7fa5dc0b09a0 2024-09-17T04:29:27.059 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:27.059 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":87,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:29:24.113176+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:29:20.485434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-17T04:29:21.553185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:29:27.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.061+0000 7fa5e3200640 1 -- 172.21.15.100:0/307653668 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa5c0078cb0 msgr2=0x7fa5c007b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:27.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.061+0000 7fa5e3200640 1 --2- 172.21.15.100:0/307653668 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa5c0078cb0 0x7fa5c007b1c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0032000 tx=0x7fa5d002c040 comp rx=0 tx=0).stop 2024-09-17T04:29:27.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.061+0000 7fa5e3200640 1 -- 172.21.15.100:0/307653668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5dc0b09a0 msgr2=0x7fa5dc0b0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:27.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.061+0000 7fa5e3200640 1 --2- 172.21.15.100:0/307653668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5dc0b09a0 0x7fa5dc0b0db0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc15cc30 tx=0x7fa5d8010f40 comp rx=0 tx=0).stop 2024-09-17T04:29:27.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.061+0000 7fa5e3200640 1 -- 172.21.15.100:0/307653668 shutdown_connections 2024-09-17T04:29:27.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.061+0000 7fa5e3200640 1 --2- 172.21.15.100:0/307653668 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa5c0078cb0 0x7fa5c007b1c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:27.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.061+0000 7fa5e3200640 1 --2- 172.21.15.100:0/307653668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5dc0b09a0 0x7fa5dc0b0db0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:27.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.061+0000 7fa5e3200640 1 --2- 172.21.15.100:0/307653668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5dc150c00 0x7fa5dc14db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:27.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.061+0000 7fa5e3200640 1 --2- 172.21.15.100:0/307653668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5c4003a50 0x7fa5dc151150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:27.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.061+0000 7fa5e3200640 1 -- 172.21.15.100:0/307653668 >> 172.21.15.100:0/307653668 conn(0x7fa5dc0b1e40 msgr2=0x7fa5dc070530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:27.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.061+0000 7fa5e3200640 1 -- 172.21.15.100:0/307653668 shutdown_connections 2024-09-17T04:29:27.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.061+0000 7fa5e3200640 1 -- 172.21.15.100:0/307653668 wait complete. 2024-09-17T04:29:27.086 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T04:29:27.086 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:29:27.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.307+0000 7f93d7000640 1 -- 172.21.15.100:0/691112359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93d00b3550 msgr2=0x7f93d00b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:27.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.307+0000 7f93d7000640 1 --2- 172.21.15.100:0/691112359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93d00b3550 0x7f93d00b3960 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f93c8005fd0 tx=0x7f93c802a7a0 comp rx=0 tx=0).stop 2024-09-17T04:29:27.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.307+0000 7f93d7000640 1 -- 172.21.15.100:0/691112359 shutdown_connections 2024-09-17T04:29:27.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.307+0000 7f93d7000640 1 --2- 172.21.15.100:0/691112359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93d00b3550 0x7f93d00b3960 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:27.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.307+0000 7f93d7000640 1 --2- 172.21.15.100:0/691112359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93d015d760 0x7f93d01535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:27.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.307+0000 7f93d7000640 1 -- 172.21.15.100:0/691112359 >> 172.21.15.100:0/691112359 conn(0x7f93d006d730 msgr2=0x7f93d00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:27.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.307+0000 7f93d7000640 1 -- 172.21.15.100:0/691112359 shutdown_connections 2024-09-17T04:29:27.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.307+0000 7f93d7000640 1 -- 172.21.15.100:0/691112359 wait complete. 2024-09-17T04:29:27.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.309+0000 7f93d7000640 1 Processor -- start 2024-09-17T04:29:27.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.309+0000 7f93d7000640 1 -- start start 2024-09-17T04:29:27.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.310+0000 7f93d7000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93d00ad0e0 0x7f93d00743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:27.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.310+0000 7f93d7000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93d00b3550 0x7f93d01f5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:27.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.310+0000 7f93d7000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93d015d760 0x7f93d00b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:27.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.310+0000 7f93d7000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f93d00b4da0 con 0x7f93d00ad0e0 2024-09-17T04:29:27.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.311+0000 7f93d7000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f93d00b4c20 con 0x7f93d00b3550 2024-09-17T04:29:27.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.311+0000 7f93d7000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f93d00b3ea0 con 0x7f93d015d760 2024-09-17T04:29:27.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.311+0000 7f93cfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93d00ad0e0 0x7f93d00743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:27.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.311+0000 7f93cfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93d00ad0e0 0x7f93d00743b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33052/0 (socket says 172.21.15.100:33052) 2024-09-17T04:29:27.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.311+0000 7f93cfe00640 1 -- 172.21.15.100:0/1873239866 learned_addr learned my addr 172.21.15.100:0/1873239866 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:27.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.311+0000 7f93c7400640 1 --2- 172.21.15.100:0/1873239866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93d00b3550 0x7f93d01f5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:27.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.311+0000 7f93d5000640 1 --2- 172.21.15.100:0/1873239866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93d015d760 0x7f93d00b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:27.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.311+0000 7f93cfe00640 1 -- 172.21.15.100:0/1873239866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93d015d760 msgr2=0x7f93d00b0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:27.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.311+0000 7f93cfe00640 1 --2- 172.21.15.100:0/1873239866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93d015d760 0x7f93d00b0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:27.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.311+0000 7f93cfe00640 1 -- 172.21.15.100:0/1873239866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93d00b3550 msgr2=0x7f93d01f5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:27.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.311+0000 7f93cfe00640 1 --2- 172.21.15.100:0/1873239866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93d00b3550 0x7f93d01f5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:27.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.311+0000 7f93cfe00640 1 -- 172.21.15.100:0/1873239866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d0074e50 con 0x7f93d00ad0e0 2024-09-17T04:29:27.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.311+0000 7f93cfe00640 1 --2- 172.21.15.100:0/1873239866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93d00ad0e0 0x7f93d00743b0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f93c000e7f0 tx=0x7f93c000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:27.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.312+0000 7f93ccc00640 1 -- 172.21.15.100:0/1873239866 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c000ccc0 con 0x7f93d00ad0e0 2024-09-17T04:29:27.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.312+0000 7f93ccc00640 1 -- 172.21.15.100:0/1873239866 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93c000ce80 con 0x7f93d00ad0e0 2024-09-17T04:29:27.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.312+0000 7f93d7000640 1 -- 172.21.15.100:0/1873239866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d0073250 con 0x7f93d00ad0e0 2024-09-17T04:29:27.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.312+0000 7f93ccc00640 1 -- 172.21.15.100:0/1873239866 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c0005280 con 0x7f93d00ad0e0 2024-09-17T04:29:27.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.312+0000 7f93d7000640 1 -- 172.21.15.100:0/1873239866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d0073810 con 0x7f93d00ad0e0 2024-09-17T04:29:27.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.313+0000 7f93d7000640 1 -- 172.21.15.100:0/1873239866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93d0073040 con 0x7f93d00ad0e0 2024-09-17T04:29:27.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.314+0000 7f93ccc00640 1 -- 172.21.15.100:0/1873239866 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f93c0005420 con 0x7f93d00ad0e0 2024-09-17T04:29:27.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.315+0000 7f93ccc00640 1 --2- 172.21.15.100:0/1873239866 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f93ac078cb0 0x7f93ac07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:27.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.315+0000 7f93ccc00640 1 -- 172.21.15.100:0/1873239866 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f93c009ba60 con 0x7f93d00ad0e0 2024-09-17T04:29:27.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.315+0000 7f93c7400640 1 --2- 172.21.15.100:0/1873239866 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f93ac078cb0 0x7f93ac07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:27.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.316+0000 7f93c7400640 1 --2- 172.21.15.100:0/1873239866 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f93ac078cb0 0x7f93ac07b1c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f93b8001eb0 tx=0x7f93b8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:27.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.319+0000 7f93ccc00640 1 -- 172.21.15.100:0/1873239866 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f93c0063be0 con 0x7f93d00ad0e0 2024-09-17T04:29:27.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.500+0000 7f93d7000640 1 -- 172.21.15.100:0/1873239866 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f93d00b43a0 con 0x7f93ac078cb0 2024-09-17T04:29:27.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.505+0000 7f93ccc00640 1 -- 172.21.15.100:0/1873239866 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240185 (secure 0 0 0) 0x7f93d00b43a0 con 0x7f93ac078cb0 2024-09-17T04:29:27.509 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:27.510 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:29:27.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.509+0000 7f93d7000640 1 -- 172.21.15.100:0/1873239866 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f93ac078cb0 msgr2=0x7f93ac07b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:27.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.509+0000 7f93d7000640 1 --2- 172.21.15.100:0/1873239866 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f93ac078cb0 0x7f93ac07b1c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f93b8001eb0 tx=0x7f93b8009040 comp rx=0 tx=0).stop 2024-09-17T04:29:27.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.509+0000 7f93d7000640 1 -- 172.21.15.100:0/1873239866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93d00ad0e0 msgr2=0x7f93d00743b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:27.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.509+0000 7f93d7000640 1 --2- 172.21.15.100:0/1873239866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93d00ad0e0 0x7f93d00743b0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f93c000e7f0 tx=0x7f93c000ecb0 comp rx=0 tx=0).stop 2024-09-17T04:29:27.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.510+0000 7f93d7000640 1 -- 172.21.15.100:0/1873239866 shutdown_connections 2024-09-17T04:29:27.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.510+0000 7f93d7000640 1 --2- 172.21.15.100:0/1873239866 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f93ac078cb0 0x7f93ac07b1c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:27.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.510+0000 7f93d7000640 1 --2- 172.21.15.100:0/1873239866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93d015d760 0x7f93d00b0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:27.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.510+0000 7f93d7000640 1 --2- 172.21.15.100:0/1873239866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93d00b3550 0x7f93d01f5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:27.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.510+0000 7f93d7000640 1 --2- 172.21.15.100:0/1873239866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93d00ad0e0 0x7f93d00743b0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:27.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.510+0000 7f93d7000640 1 -- 172.21.15.100:0/1873239866 >> 172.21.15.100:0/1873239866 conn(0x7f93d006d730 msgr2=0x7f93d00cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:27.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.510+0000 7f93d7000640 1 -- 172.21.15.100:0/1873239866 shutdown_connections 2024-09-17T04:29:27.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.510+0000 7f93d7000640 1 -- 172.21.15.100:0/1873239866 wait complete. 2024-09-17T04:29:27.537 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-17T04:29:25.662839+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258604,"kb_used_data":9080,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716116,"statfs":{"total":773094113280,"available":772829302784,"internally_reserved":0,"allocated":9297920,"data_stored":4626788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.582297"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.091881+0000","last_change":"2024-09-17T04:29:21.091881+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.091881+0000","last_undegraded":"2024-09-17T04:29:21.091881+0000","last_fullsized":"2024-09-17T04:29:21.091881+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.089976+0000","last_change":"2024-09-17T04:29:21.089976+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.089976+0000","last_undegraded":"2024-09-17T04:29:21.089976+0000","last_fullsized":"2024-09-17T04:29:21.089976+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.089845+0000","last_change":"2024-09-17T04:29:21.089845+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.089845+0000","last_undegraded":"2024-09-17T04:29:21.089845+0000","last_fullsized":"2024-09-17T04:29:21.089845+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.091857+0000","last_change":"2024-09-17T04:29:21.091857+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.091857+0000","last_undegraded":"2024-09-17T04:29:21.091857+0000","last_fullsized":"2024-09-17T04:29:21.091857+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.089951+0000","last_change":"2024-09-17T04:29:21.089951+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.089951+0000","last_undegraded":"2024-09-17T04:29:21.089951+0000","last_fullsized":"2024-09-17T04:29:21.089951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.090638+0000","last_change":"2024-09-17T04:29:21.090638+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.090638+0000","last_undegraded":"2024-09-17T04:29:21.090638+0000","last_fullsized":"2024-09-17T04:29:21.090638+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.089641+0000","last_change":"2024-09-17T04:29:21.089641+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.089641+0000","last_undegraded":"2024-09-17T04:29:21.089641+0000","last_fullsized":"2024-09-17T04:29:21.089641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.086410+0000","last_change":"2024-09-17T04:29:21.086410+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.086410+0000","last_undegraded":"2024-09-17T04:29:21.086410+0000","last_fullsized":"2024-09-17T04:29:21.086410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.091390+0000","last_change":"2024-09-17T04:29:21.091390+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.091390+0000","last_undegraded":"2024-09-17T04:29:21.091390+0000","last_fullsized":"2024-09-17T04:29:21.091390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.093092+0000","last_change":"2024-09-17T04:29:21.093092+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.093092+0000","last_undegraded":"2024-09-17T04:29:21.093092+0000","last_fullsized":"2024-09-17T04:29:21.093092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.092926+0000","last_change":"2024-09-17T04:29:21.092926+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.092926+0000","last_undegraded":"2024-09-17T04:29:21.092926+0000","last_fullsized":"2024-09-17T04:29:21.092926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.087965+0000","last_change":"2024-09-17T04:29:21.087965+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.087965+0000","last_undegraded":"2024-09-17T04:29:21.087965+0000","last_fullsized":"2024-09-17T04:29:21.087965+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.086550+0000","last_change":"2024-09-17T04:29:21.086550+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.086550+0000","last_undegraded":"2024-09-17T04:29:21.086550+0000","last_fullsized":"2024-09-17T04:29:21.086550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.084023+0000","last_change":"2024-09-17T04:29:21.084023+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.084023+0000","last_undegraded":"2024-09-17T04:29:21.084023+0000","last_fullsized":"2024-09-17T04:29:21.084023+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.088051+0000","last_change":"2024-09-17T04:29:21.088051+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.088051+0000","last_undegraded":"2024-09-17T04:29:21.088051+0000","last_fullsized":"2024-09-17T04:29:21.088051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.089636+0000","last_change":"2024-09-17T04:29:21.089636+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.089636+0000","last_undegraded":"2024-09-17T04:29:21.089636+0000","last_fullsized":"2024-09-17T04:29:21.089636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.090896+0000","last_change":"2024-09-17T04:29:21.090896+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.090896+0000","last_undegraded":"2024-09-17T04:29:21.090896+0000","last_fullsized":"2024-09-17T04:29:21.090896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.090461+0000","last_change":"2024-09-17T04:29:21.090461+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.090461+0000","last_undegraded":"2024-09-17T04:29:21.090461+0000","last_fullsized":"2024-09-17T04:29:21.090461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.090540+0000","last_change":"2024-09-17T04:29:21.090540+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.090540+0000","last_undegraded":"2024-09-17T04:29:21.090540+0000","last_fullsized":"2024-09-17T04:29:21.090540+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.084034+0000","last_change":"2024-09-17T04:29:21.084034+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.084034+0000","last_undegraded":"2024-09-17T04:29:21.084034+0000","last_fullsized":"2024-09-17T04:29:21.084034+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.087248+0000","last_change":"2024-09-17T04:29:21.087248+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.087248+0000","last_undegraded":"2024-09-17T04:29:21.087248+0000","last_fullsized":"2024-09-17T04:29:21.087248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.086626+0000","last_change":"2024-09-17T04:29:21.086626+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.086626+0000","last_undegraded":"2024-09-17T04:29:21.086626+0000","last_fullsized":"2024-09-17T04:29:21.086626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.083978+0000","last_change":"2024-09-17T04:29:21.083978+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.083978+0000","last_undegraded":"2024-09-17T04:29:21.083978+0000","last_fullsized":"2024-09-17T04:29:21.083978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.086955+0000","last_change":"2024-09-17T04:29:21.086955+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.086955+0000","last_undegraded":"2024-09-17T04:29:21.086955+0000","last_fullsized":"2024-09-17T04:29:21.086955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.086412+0000","last_change":"2024-09-17T04:29:21.086412+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.086412+0000","last_undegraded":"2024-09-17T04:29:21.086412+0000","last_fullsized":"2024-09-17T04:29:21.086412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.092146+0000","last_change":"2024-09-17T04:29:21.092146+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.092146+0000","last_undegraded":"2024-09-17T04:29:21.092146+0000","last_fullsized":"2024-09-17T04:29:21.092146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.091602+0000","last_change":"2024-09-17T04:29:21.091602+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.091602+0000","last_undegraded":"2024-09-17T04:29:21.091602+0000","last_fullsized":"2024-09-17T04:29:21.091602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.091249+0000","last_change":"2024-09-17T04:29:21.091249+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.091249+0000","last_undegraded":"2024-09-17T04:29:21.091249+0000","last_fullsized":"2024-09-17T04:29:21.091249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.089962+0000","last_change":"2024-09-17T04:29:21.089962+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.089962+0000","last_undegraded":"2024-09-17T04:29:21.089962+0000","last_fullsized":"2024-09-17T04:29:21.089962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.090767+0000","last_change":"2024-09-17T04:29:21.090767+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.090767+0000","last_undegraded":"2024-09-17T04:29:21.090767+0000","last_fullsized":"2024-09-17T04:29:21.090767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.090027+0000","last_change":"2024-09-17T04:29:21.090027+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.090027+0000","last_undegraded":"2024-09-17T04:29:21.090027+0000","last_fullsized":"2024-09-17T04:29:21.090027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:29:22.080485+0000","last_change":"2024-09-17T04:29:22.080485+0000","last_active":"2024-09-17T04:29:22.080485+0000","last_peered":"2024-09-17T04:29:22.080485+0000","last_clean":"2024-09-17T04:29:22.080485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:22.080485+0000","last_undegraded":"2024-09-17T04:29:22.080485+0000","last_fullsized":"2024-09-17T04:29:22.080485+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-17T04:29:21.088426+0000","last_change":"2024-09-17T04:29:21.088426+0000","last_active":"2024-09-17T04:29:21.076807+0000","last_peered":"2024-09-17T04:29:21.076807+0000","last_clean":"2024-09-17T04:29:21.076807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:29:21.088426+0000","last_undegraded":"2024-09-17T04:29:21.088426+0000","last_fullsized":"2024-09-17T04:29:21.088426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"76'79","reported_seq":252,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T04:29:21.087492+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:29:21.087492+0000","last_peered":"2024-09-17T04:29:21.087492+0000","last_clean":"2024-09-17T04:29:21.087492+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:29:21.087492+0000","last_undegraded":"2024-09-17T04:29:21.087492+0000","last_fullsized":"2024-09-17T04:29:21.087492+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476805,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32752,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339088,"statfs":{"total":96636764160,"available":96603226112,"internally_reserved":0,"allocated":1601536,"data_stored":1018216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476804,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1007616,"data_stored":427848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476805,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1007616,"data_stored":427848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476805,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1019904,"data_stored":433665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542212,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1019904,"data_stored":433665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509508,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1019904,"data_stored":433665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509507,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1019904,"data_stored":433665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509507,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32756,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339084,"statfs":{"total":96636764160,"available":96603222016,"internally_reserved":0,"allocated":1601536,"data_stored":1018216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:29:27.542 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:29:27.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.799+0000 7fd530e00640 1 -- 172.21.15.100:0/1606881827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd52c152920 msgr2=0x7fd52c15f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:27.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.799+0000 7fd530e00640 1 --2- 172.21.15.100:0/1606881827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd52c152920 0x7fd52c15f3d0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fd518009830 tx=0x7fd51802a480 comp rx=0 tx=0).stop 2024-09-17T04:29:27.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.799+0000 7fd530e00640 1 -- 172.21.15.100:0/1606881827 shutdown_connections 2024-09-17T04:29:27.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.799+0000 7fd530e00640 1 --2- 172.21.15.100:0/1606881827 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd52c15f910 0x7fd52c0b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:27.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.799+0000 7fd530e00640 1 --2- 172.21.15.100:0/1606881827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd52c152920 0x7fd52c15f3d0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:27.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.799+0000 7fd530e00640 1 -- 172.21.15.100:0/1606881827 >> 172.21.15.100:0/1606881827 conn(0x7fd52c06d730 msgr2=0x7fd52c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:27.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.799+0000 7fd530e00640 1 -- 172.21.15.100:0/1606881827 shutdown_connections 2024-09-17T04:29:27.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.799+0000 7fd530e00640 1 -- 172.21.15.100:0/1606881827 wait complete. 2024-09-17T04:29:27.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.801+0000 7fd530e00640 1 Processor -- start 2024-09-17T04:29:27.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.801+0000 7fd530e00640 1 -- start start 2024-09-17T04:29:27.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.803+0000 7fd530e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd52c0aebc0 0x7fd52c159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:27.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.803+0000 7fd530e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd52c152920 0x7fd52c1f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:27.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.803+0000 7fd530e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd52c15f910 0x7fd52c1f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:27.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.803+0000 7fd530e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c0b4190 con 0x7fd52c152920 2024-09-17T04:29:27.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.803+0000 7fd530e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c0b4010 con 0x7fd52c15f910 2024-09-17T04:29:27.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.803+0000 7fd530e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd52c0b4310 con 0x7fd52c0aebc0 2024-09-17T04:29:27.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.803+0000 7fd529400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd52c152920 0x7fd52c1f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:27.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.803+0000 7fd529400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd52c152920 0x7fd52c1f5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33064/0 (socket says 172.21.15.100:33064) 2024-09-17T04:29:27.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.803+0000 7fd529400640 1 -- 172.21.15.100:0/232933258 learned_addr learned my addr 172.21.15.100:0/232933258 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:27.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.803+0000 7fd52a800640 1 --2- 172.21.15.100:0/232933258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd52c15f910 0x7fd52c1f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:27.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.803+0000 7fd529e00640 1 --2- 172.21.15.100:0/232933258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd52c0aebc0 0x7fd52c159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:27.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.803+0000 7fd529400640 1 -- 172.21.15.100:0/232933258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd52c0aebc0 msgr2=0x7fd52c159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:27.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.803+0000 7fd529400640 1 --2- 172.21.15.100:0/232933258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd52c0aebc0 0x7fd52c159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:27.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.803+0000 7fd529400640 1 -- 172.21.15.100:0/232933258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd52c15f910 msgr2=0x7fd52c1f5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:27.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.803+0000 7fd529400640 1 --2- 172.21.15.100:0/232933258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd52c15f910 0x7fd52c1f5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:27.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.803+0000 7fd529400640 1 -- 172.21.15.100:0/232933258 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52c1f6770 con 0x7fd52c152920 2024-09-17T04:29:27.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.803+0000 7fd529e00640 1 --2- 172.21.15.100:0/232933258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd52c0aebc0 0x7fd52c159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:27.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.804+0000 7fd529400640 1 --2- 172.21.15.100:0/232933258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd52c152920 0x7fd52c1f5a10 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fd518009950 tx=0x7fd518033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:27.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.804+0000 7fd522000640 1 -- 172.21.15.100:0/232933258 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518005c90 con 0x7fd52c152920 2024-09-17T04:29:27.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.804+0000 7fd522000640 1 -- 172.21.15.100:0/232933258 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd518031e50 con 0x7fd52c152920 2024-09-17T04:29:27.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.804+0000 7fd530e00640 1 -- 172.21.15.100:0/232933258 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd52c1f91d0 con 0x7fd52c152920 2024-09-17T04:29:27.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.804+0000 7fd522000640 1 -- 172.21.15.100:0/232933258 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518005260 con 0x7fd52c152920 2024-09-17T04:29:27.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.804+0000 7fd530e00640 1 -- 172.21.15.100:0/232933258 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd52c1f9cb0 con 0x7fd52c152920 2024-09-17T04:29:27.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.805+0000 7fd530e00640 1 -- 172.21.15.100:0/232933258 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd52c0b48a0 con 0x7fd52c152920 2024-09-17T04:29:27.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.806+0000 7fd522000640 1 -- 172.21.15.100:0/232933258 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fd518005400 con 0x7fd52c152920 2024-09-17T04:29:27.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.807+0000 7fd522000640 1 --2- 172.21.15.100:0/232933258 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd50c078cb0 0x7fd50c07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:27.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.807+0000 7fd522000640 1 -- 172.21.15.100:0/232933258 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fd5180bbfe0 con 0x7fd52c152920 2024-09-17T04:29:27.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.807+0000 7fd529e00640 1 --2- 172.21.15.100:0/232933258 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd50c078cb0 0x7fd50c07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:27.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.808+0000 7fd529e00640 1 --2- 172.21.15.100:0/232933258 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd50c078cb0 0x7fd50c07b1c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd51c007a50 tx=0x7fd51c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:27.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.812+0000 7fd522000640 1 -- 172.21.15.100:0/232933258 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd51803f030 con 0x7fd52c152920 2024-09-17T04:29:27.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.992+0000 7fd530e00640 1 -- 172.21.15.100:0/232933258 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd52c1f64c0 con 0x7fd50c078cb0 2024-09-17T04:29:27.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:27.997+0000 7fd522000640 1 -- 172.21.15.100:0/232933258 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252078 (secure 0 0 0) 0x7fd52c1f64c0 con 0x7fd50c078cb0 2024-09-17T04:29:28.000 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:28.002 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:29:28.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.001+0000 7fd530e00640 1 -- 172.21.15.100:0/232933258 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd50c078cb0 msgr2=0x7fd50c07b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:28.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.001+0000 7fd530e00640 1 --2- 172.21.15.100:0/232933258 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd50c078cb0 0x7fd50c07b1c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd51c007a50 tx=0x7fd51c009040 comp rx=0 tx=0).stop 2024-09-17T04:29:28.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.001+0000 7fd530e00640 1 -- 172.21.15.100:0/232933258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd52c152920 msgr2=0x7fd52c1f5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:28.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.001+0000 7fd530e00640 1 --2- 172.21.15.100:0/232933258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd52c152920 0x7fd52c1f5a10 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fd518009950 tx=0x7fd518033c20 comp rx=0 tx=0).stop 2024-09-17T04:29:28.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.001+0000 7fd530e00640 1 -- 172.21.15.100:0/232933258 shutdown_connections 2024-09-17T04:29:28.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.001+0000 7fd530e00640 1 --2- 172.21.15.100:0/232933258 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd50c078cb0 0x7fd50c07b1c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:28.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.001+0000 7fd530e00640 1 --2- 172.21.15.100:0/232933258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd52c15f910 0x7fd52c1f5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:28.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.001+0000 7fd530e00640 1 --2- 172.21.15.100:0/232933258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd52c152920 0x7fd52c1f5a10 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:28.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.001+0000 7fd530e00640 1 --2- 172.21.15.100:0/232933258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd52c0aebc0 0x7fd52c159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:28.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.001+0000 7fd530e00640 1 -- 172.21.15.100:0/232933258 >> 172.21.15.100:0/232933258 conn(0x7fd52c06d730 msgr2=0x7fd52c15a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:28.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.002+0000 7fd530e00640 1 -- 172.21.15.100:0/232933258 shutdown_connections 2024-09-17T04:29:28.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.002+0000 7fd530e00640 1 -- 172.21.15.100:0/232933258 wait complete. 2024-09-17T04:29:28.024 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-17T04:29:27.663377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258992,"kb_used_data":9480,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715728,"statfs":{"total":773094113280,"available":772828905472,"internally_reserved":0,"allocated":9707520,"data_stored":4809676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001343"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.599978+0000","last_change":"2024-09-17T04:29:23.114210+0000","last_active":"2024-09-17T04:29:25.599978+0000","last_peered":"2024-09-17T04:29:25.599978+0000","last_clean":"2024-09-17T04:29:25.599978+0000","last_became_active":"2024-09-17T04:29:23.113410+0000","last_became_peered":"2024-09-17T04:29:23.113410+0000","last_unstale":"2024-09-17T04:29:25.599978+0000","last_undegraded":"2024-09-17T04:29:25.599978+0000","last_fullsized":"2024-09-17T04:29:25.599978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:47.476110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.436471+0000","last_change":"2024-09-17T04:29:23.120097+0000","last_active":"2024-09-17T04:29:24.436471+0000","last_peered":"2024-09-17T04:29:24.436471+0000","last_clean":"2024-09-17T04:29:24.436471+0000","last_became_active":"2024-09-17T04:29:23.119797+0000","last_became_peered":"2024-09-17T04:29:23.119797+0000","last_unstale":"2024-09-17T04:29:24.436471+0000","last_undegraded":"2024-09-17T04:29:24.436471+0000","last_fullsized":"2024-09-17T04:29:24.436471+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:50:19.139152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.124336+0000","last_change":"2024-09-17T04:29:23.121594+0000","last_active":"2024-09-17T04:29:24.124336+0000","last_peered":"2024-09-17T04:29:24.124336+0000","last_clean":"2024-09-17T04:29:24.124336+0000","last_became_active":"2024-09-17T04:29:23.121193+0000","last_became_peered":"2024-09-17T04:29:23.121193+0000","last_unstale":"2024-09-17T04:29:24.124336+0000","last_undegraded":"2024-09-17T04:29:24.124336+0000","last_fullsized":"2024-09-17T04:29:24.124336+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:30:14.565815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.123132+0000","last_change":"2024-09-17T04:29:23.130841+0000","last_active":"2024-09-17T04:29:24.123132+0000","last_peered":"2024-09-17T04:29:24.123132+0000","last_clean":"2024-09-17T04:29:24.123132+0000","last_became_active":"2024-09-17T04:29:23.130545+0000","last_became_peered":"2024-09-17T04:29:23.130545+0000","last_unstale":"2024-09-17T04:29:24.123132+0000","last_undegraded":"2024-09-17T04:29:24.123132+0000","last_fullsized":"2024-09-17T04:29:24.123132+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:57:10.279335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.599108+0000","last_change":"2024-09-17T04:29:23.119310+0000","last_active":"2024-09-17T04:29:25.599108+0000","last_peered":"2024-09-17T04:29:25.599108+0000","last_clean":"2024-09-17T04:29:25.599108+0000","last_became_active":"2024-09-17T04:29:23.118988+0000","last_became_peered":"2024-09-17T04:29:23.118988+0000","last_unstale":"2024-09-17T04:29:25.599108+0000","last_undegraded":"2024-09-17T04:29:25.599108+0000","last_fullsized":"2024-09-17T04:29:25.599108+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:11.388051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.123383+0000","last_change":"2024-09-17T04:29:23.099734+0000","last_active":"2024-09-17T04:29:24.123383+0000","last_peered":"2024-09-17T04:29:24.123383+0000","last_clean":"2024-09-17T04:29:24.123383+0000","last_became_active":"2024-09-17T04:29:23.099373+0000","last_became_peered":"2024-09-17T04:29:23.099373+0000","last_unstale":"2024-09-17T04:29:24.123383+0000","last_undegraded":"2024-09-17T04:29:24.123383+0000","last_fullsized":"2024-09-17T04:29:24.123383+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:00.596413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.122116+0000","last_change":"2024-09-17T04:29:23.110059+0000","last_active":"2024-09-17T04:29:24.122116+0000","last_peered":"2024-09-17T04:29:24.122116+0000","last_clean":"2024-09-17T04:29:24.122116+0000","last_became_active":"2024-09-17T04:29:23.109770+0000","last_became_peered":"2024-09-17T04:29:23.109770+0000","last_unstale":"2024-09-17T04:29:24.122116+0000","last_undegraded":"2024-09-17T04:29:24.122116+0000","last_fullsized":"2024-09-17T04:29:24.122116+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:49:48.031948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.123360+0000","last_change":"2024-09-17T04:29:23.123595+0000","last_active":"2024-09-17T04:29:24.123360+0000","last_peered":"2024-09-17T04:29:24.123360+0000","last_clean":"2024-09-17T04:29:24.123360+0000","last_became_active":"2024-09-17T04:29:23.123217+0000","last_became_peered":"2024-09-17T04:29:23.123217+0000","last_unstale":"2024-09-17T04:29:24.123360+0000","last_undegraded":"2024-09-17T04:29:24.123360+0000","last_fullsized":"2024-09-17T04:29:24.123360+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:23.186671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.125072+0000","last_change":"2024-09-17T04:29:23.112473+0000","last_active":"2024-09-17T04:29:24.125072+0000","last_peered":"2024-09-17T04:29:24.125072+0000","last_clean":"2024-09-17T04:29:24.125072+0000","last_became_active":"2024-09-17T04:29:23.112132+0000","last_became_peered":"2024-09-17T04:29:23.112132+0000","last_unstale":"2024-09-17T04:29:24.125072+0000","last_undegraded":"2024-09-17T04:29:24.125072+0000","last_fullsized":"2024-09-17T04:29:24.125072+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:37:58.222465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.600176+0000","last_change":"2024-09-17T04:29:23.117921+0000","last_active":"2024-09-17T04:29:25.600176+0000","last_peered":"2024-09-17T04:29:25.600176+0000","last_clean":"2024-09-17T04:29:25.600176+0000","last_became_active":"2024-09-17T04:29:23.117602+0000","last_became_peered":"2024-09-17T04:29:23.117602+0000","last_unstale":"2024-09-17T04:29:25.600176+0000","last_undegraded":"2024-09-17T04:29:25.600176+0000","last_fullsized":"2024-09-17T04:29:25.600176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:43.789110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.436535+0000","last_change":"2024-09-17T04:29:23.118212+0000","last_active":"2024-09-17T04:29:24.436535+0000","last_peered":"2024-09-17T04:29:24.436535+0000","last_clean":"2024-09-17T04:29:24.436535+0000","last_became_active":"2024-09-17T04:29:23.115375+0000","last_became_peered":"2024-09-17T04:29:23.115375+0000","last_unstale":"2024-09-17T04:29:24.436535+0000","last_undegraded":"2024-09-17T04:29:24.436535+0000","last_fullsized":"2024-09-17T04:29:24.436535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:03.232815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.600440+0000","last_change":"2024-09-17T04:29:23.122277+0000","last_active":"2024-09-17T04:29:25.600440+0000","last_peered":"2024-09-17T04:29:25.600440+0000","last_clean":"2024-09-17T04:29:25.600440+0000","last_became_active":"2024-09-17T04:29:23.119850+0000","last_became_peered":"2024-09-17T04:29:23.119850+0000","last_unstale":"2024-09-17T04:29:25.600440+0000","last_undegraded":"2024-09-17T04:29:25.600440+0000","last_fullsized":"2024-09-17T04:29:25.600440+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:14.643169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.124012+0000","last_change":"2024-09-17T04:29:23.110369+0000","last_active":"2024-09-17T04:29:24.124012+0000","last_peered":"2024-09-17T04:29:24.124012+0000","last_clean":"2024-09-17T04:29:24.124012+0000","last_became_active":"2024-09-17T04:29:23.109999+0000","last_became_peered":"2024-09-17T04:29:23.109999+0000","last_unstale":"2024-09-17T04:29:24.124012+0000","last_undegraded":"2024-09-17T04:29:24.124012+0000","last_fullsized":"2024-09-17T04:29:24.124012+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:53:12.306254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.122408+0000","last_change":"2024-09-17T04:29:23.128932+0000","last_active":"2024-09-17T04:29:24.122408+0000","last_peered":"2024-09-17T04:29:24.122408+0000","last_clean":"2024-09-17T04:29:24.122408+0000","last_became_active":"2024-09-17T04:29:23.126723+0000","last_became_peered":"2024-09-17T04:29:23.126723+0000","last_unstale":"2024-09-17T04:29:24.122408+0000","last_undegraded":"2024-09-17T04:29:24.122408+0000","last_fullsized":"2024-09-17T04:29:24.122408+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:25:59.683646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.115892+0000","last_change":"2024-09-17T04:29:23.111156+0000","last_active":"2024-09-17T04:29:25.115892+0000","last_peered":"2024-09-17T04:29:25.115892+0000","last_clean":"2024-09-17T04:29:25.115892+0000","last_became_active":"2024-09-17T04:29:23.110822+0000","last_became_peered":"2024-09-17T04:29:23.110822+0000","last_unstale":"2024-09-17T04:29:25.115892+0000","last_undegraded":"2024-09-17T04:29:25.115892+0000","last_fullsized":"2024-09-17T04:29:25.115892+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:11:10.331692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.598866+0000","last_change":"2024-09-17T04:29:23.112457+0000","last_active":"2024-09-17T04:29:25.598866+0000","last_peered":"2024-09-17T04:29:25.598866+0000","last_clean":"2024-09-17T04:29:25.598866+0000","last_became_active":"2024-09-17T04:29:23.112123+0000","last_became_peered":"2024-09-17T04:29:23.112123+0000","last_unstale":"2024-09-17T04:29:25.598866+0000","last_undegraded":"2024-09-17T04:29:25.598866+0000","last_fullsized":"2024-09-17T04:29:25.598866+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:31.737168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.599795+0000","last_change":"2024-09-17T04:29:23.119010+0000","last_active":"2024-09-17T04:29:25.599795+0000","last_peered":"2024-09-17T04:29:25.599795+0000","last_clean":"2024-09-17T04:29:25.599795+0000","last_became_active":"2024-09-17T04:29:23.118696+0000","last_became_peered":"2024-09-17T04:29:23.118696+0000","last_unstale":"2024-09-17T04:29:25.599795+0000","last_undegraded":"2024-09-17T04:29:25.599795+0000","last_fullsized":"2024-09-17T04:29:25.599795+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:37:58.222465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.123033+0000","last_change":"2024-09-17T04:29:23.118081+0000","last_active":"2024-09-17T04:29:24.123033+0000","last_peered":"2024-09-17T04:29:24.123033+0000","last_clean":"2024-09-17T04:29:24.123033+0000","last_became_active":"2024-09-17T04:29:23.117738+0000","last_became_peered":"2024-09-17T04:29:23.117738+0000","last_unstale":"2024-09-17T04:29:24.123033+0000","last_undegraded":"2024-09-17T04:29:24.123033+0000","last_fullsized":"2024-09-17T04:29:24.123033+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:56:37.900254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.344056+0000","last_change":"2024-09-17T04:29:23.111775+0000","last_active":"2024-09-17T04:29:24.344056+0000","last_peered":"2024-09-17T04:29:24.344056+0000","last_clean":"2024-09-17T04:29:24.344056+0000","last_became_active":"2024-09-17T04:29:23.111489+0000","last_became_peered":"2024-09-17T04:29:23.111489+0000","last_unstale":"2024-09-17T04:29:24.344056+0000","last_undegraded":"2024-09-17T04:29:24.344056+0000","last_fullsized":"2024-09-17T04:29:24.344056+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:42.556000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.123161+0000","last_change":"2024-09-17T04:29:23.112120+0000","last_active":"2024-09-17T04:29:24.123161+0000","last_peered":"2024-09-17T04:29:24.123161+0000","last_clean":"2024-09-17T04:29:24.123161+0000","last_became_active":"2024-09-17T04:29:23.111837+0000","last_became_peered":"2024-09-17T04:29:23.111837+0000","last_unstale":"2024-09-17T04:29:24.123161+0000","last_undegraded":"2024-09-17T04:29:24.123161+0000","last_fullsized":"2024-09-17T04:29:24.123161+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:50:06.174128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.123075+0000","last_change":"2024-09-17T04:29:23.111990+0000","last_active":"2024-09-17T04:29:24.123075+0000","last_peered":"2024-09-17T04:29:24.123075+0000","last_clean":"2024-09-17T04:29:24.123075+0000","last_became_active":"2024-09-17T04:29:23.111683+0000","last_became_peered":"2024-09-17T04:29:23.111683+0000","last_unstale":"2024-09-17T04:29:24.123075+0000","last_undegraded":"2024-09-17T04:29:24.123075+0000","last_fullsized":"2024-09-17T04:29:24.123075+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:00.720092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.436235+0000","last_change":"2024-09-17T04:29:23.129584+0000","last_active":"2024-09-17T04:29:24.436235+0000","last_peered":"2024-09-17T04:29:24.436235+0000","last_clean":"2024-09-17T04:29:24.436235+0000","last_became_active":"2024-09-17T04:29:23.127014+0000","last_became_peered":"2024-09-17T04:29:23.127014+0000","last_unstale":"2024-09-17T04:29:24.436235+0000","last_undegraded":"2024-09-17T04:29:24.436235+0000","last_fullsized":"2024-09-17T04:29:24.436235+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:22.015077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.600224+0000","last_change":"2024-09-17T04:29:23.125629+0000","last_active":"2024-09-17T04:29:25.600224+0000","last_peered":"2024-09-17T04:29:25.600224+0000","last_clean":"2024-09-17T04:29:25.600224+0000","last_became_active":"2024-09-17T04:29:23.125234+0000","last_became_peered":"2024-09-17T04:29:23.125234+0000","last_unstale":"2024-09-17T04:29:25.600224+0000","last_undegraded":"2024-09-17T04:29:25.600224+0000","last_fullsized":"2024-09-17T04:29:25.600224+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:05:32.951310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.122122+0000","last_change":"2024-09-17T04:29:23.107682+0000","last_active":"2024-09-17T04:29:24.122122+0000","last_peered":"2024-09-17T04:29:24.122122+0000","last_clean":"2024-09-17T04:29:24.122122+0000","last_became_active":"2024-09-17T04:29:23.107344+0000","last_became_peered":"2024-09-17T04:29:23.107344+0000","last_unstale":"2024-09-17T04:29:24.122122+0000","last_undegraded":"2024-09-17T04:29:24.122122+0000","last_fullsized":"2024-09-17T04:29:24.122122+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:26:02.562268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.436402+0000","last_change":"2024-09-17T04:29:23.135250+0000","last_active":"2024-09-17T04:29:24.436402+0000","last_peered":"2024-09-17T04:29:24.436402+0000","last_clean":"2024-09-17T04:29:24.436402+0000","last_became_active":"2024-09-17T04:29:23.131719+0000","last_became_peered":"2024-09-17T04:29:23.131719+0000","last_unstale":"2024-09-17T04:29:24.436402+0000","last_undegraded":"2024-09-17T04:29:24.436402+0000","last_fullsized":"2024-09-17T04:29:24.436402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:56.761667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.123871+0000","last_change":"2024-09-17T04:29:23.127559+0000","last_active":"2024-09-17T04:29:24.123871+0000","last_peered":"2024-09-17T04:29:24.123871+0000","last_clean":"2024-09-17T04:29:24.123871+0000","last_became_active":"2024-09-17T04:29:23.127310+0000","last_became_peered":"2024-09-17T04:29:23.127310+0000","last_unstale":"2024-09-17T04:29:24.123871+0000","last_undegraded":"2024-09-17T04:29:24.123871+0000","last_fullsized":"2024-09-17T04:29:24.123871+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:43.789110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.598976+0000","last_change":"2024-09-17T04:29:23.102064+0000","last_active":"2024-09-17T04:29:25.598976+0000","last_peered":"2024-09-17T04:29:25.598976+0000","last_clean":"2024-09-17T04:29:25.598976+0000","last_became_active":"2024-09-17T04:29:23.101710+0000","last_became_peered":"2024-09-17T04:29:23.101710+0000","last_unstale":"2024-09-17T04:29:25.598976+0000","last_undegraded":"2024-09-17T04:29:25.598976+0000","last_fullsized":"2024-09-17T04:29:25.598976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:51:09.505443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.600283+0000","last_change":"2024-09-17T04:29:23.111744+0000","last_active":"2024-09-17T04:29:25.600283+0000","last_peered":"2024-09-17T04:29:25.600283+0000","last_clean":"2024-09-17T04:29:25.600283+0000","last_became_active":"2024-09-17T04:29:23.111246+0000","last_became_peered":"2024-09-17T04:29:23.111246+0000","last_unstale":"2024-09-17T04:29:25.600283+0000","last_undegraded":"2024-09-17T04:29:25.600283+0000","last_fullsized":"2024-09-17T04:29:25.600283+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:30.560187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.436600+0000","last_change":"2024-09-17T04:29:23.118270+0000","last_active":"2024-09-17T04:29:24.436600+0000","last_peered":"2024-09-17T04:29:24.436600+0000","last_clean":"2024-09-17T04:29:24.436600+0000","last_became_active":"2024-09-17T04:29:23.117920+0000","last_became_peered":"2024-09-17T04:29:23.117920+0000","last_unstale":"2024-09-17T04:29:24.436600+0000","last_undegraded":"2024-09-17T04:29:24.436600+0000","last_fullsized":"2024-09-17T04:29:24.436600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:19:32.754147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.436177+0000","last_change":"2024-09-17T04:29:23.126546+0000","last_active":"2024-09-17T04:29:24.436177+0000","last_peered":"2024-09-17T04:29:24.436177+0000","last_clean":"2024-09-17T04:29:24.436177+0000","last_became_active":"2024-09-17T04:29:23.125877+0000","last_became_peered":"2024-09-17T04:29:23.125877+0000","last_unstale":"2024-09-17T04:29:24.436177+0000","last_undegraded":"2024-09-17T04:29:24.436177+0000","last_fullsized":"2024-09-17T04:29:24.436177+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:37:07.614480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.599510+0000","last_change":"2024-09-17T04:29:23.110825+0000","last_active":"2024-09-17T04:29:25.599510+0000","last_peered":"2024-09-17T04:29:25.599510+0000","last_clean":"2024-09-17T04:29:25.599510+0000","last_became_active":"2024-09-17T04:29:23.110503+0000","last_became_peered":"2024-09-17T04:29:23.110503+0000","last_unstale":"2024-09-17T04:29:25.599510+0000","last_undegraded":"2024-09-17T04:29:25.599510+0000","last_fullsized":"2024-09-17T04:29:25.599510+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:14.643169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.599527+0000","last_change":"2024-09-17T04:29:23.126436+0000","last_active":"2024-09-17T04:29:25.599527+0000","last_peered":"2024-09-17T04:29:25.599527+0000","last_clean":"2024-09-17T04:29:25.599527+0000","last_became_active":"2024-09-17T04:29:23.126125+0000","last_became_peered":"2024-09-17T04:29:23.126125+0000","last_unstale":"2024-09-17T04:29:25.599527+0000","last_undegraded":"2024-09-17T04:29:25.599527+0000","last_fullsized":"2024-09-17T04:29:25.599527+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:05:33.531403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.599795+0000","last_change":"2024-09-17T04:29:23.118724+0000","last_active":"2024-09-17T04:29:25.599795+0000","last_peered":"2024-09-17T04:29:25.599795+0000","last_clean":"2024-09-17T04:29:25.599795+0000","last_became_active":"2024-09-17T04:29:23.118413+0000","last_became_peered":"2024-09-17T04:29:23.118413+0000","last_unstale":"2024-09-17T04:29:25.599795+0000","last_undegraded":"2024-09-17T04:29:25.599795+0000","last_fullsized":"2024-09-17T04:29:25.599795+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:47.476110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"86'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.600398+0000","last_change":"2024-09-17T04:29:22.114376+0000","last_active":"2024-09-17T04:29:25.600398+0000","last_peered":"2024-09-17T04:29:25.600398+0000","last_clean":"2024-09-17T04:29:25.600398+0000","last_became_active":"2024-09-17T04:29:22.113906+0000","last_became_peered":"2024-09-17T04:29:22.113906+0000","last_unstale":"2024-09-17T04:29:25.600398+0000","last_undegraded":"2024-09-17T04:29:25.600398+0000","last_fullsized":"2024-09-17T04:29:25.600398+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:12.526309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.600272+0000","last_change":"2024-09-17T04:29:23.125955+0000","last_active":"2024-09-17T04:29:25.600272+0000","last_peered":"2024-09-17T04:29:25.600272+0000","last_clean":"2024-09-17T04:29:25.600272+0000","last_became_active":"2024-09-17T04:29:23.125650+0000","last_became_peered":"2024-09-17T04:29:23.125650+0000","last_unstale":"2024-09-17T04:29:25.600272+0000","last_undegraded":"2024-09-17T04:29:25.600272+0000","last_fullsized":"2024-09-17T04:29:25.600272+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:53:12.306254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.600036+0000","last_change":"2024-09-17T04:29:22.118420+0000","last_active":"2024-09-17T04:29:25.600036+0000","last_peered":"2024-09-17T04:29:25.600036+0000","last_clean":"2024-09-17T04:29:25.600036+0000","last_became_active":"2024-09-17T04:29:22.117941+0000","last_became_peered":"2024-09-17T04:29:22.117941+0000","last_unstale":"2024-09-17T04:29:25.600036+0000","last_undegraded":"2024-09-17T04:29:25.600036+0000","last_fullsized":"2024-09-17T04:29:25.600036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:12.526309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.600555+0000","last_change":"2024-09-17T04:29:23.126976+0000","last_active":"2024-09-17T04:29:25.600555+0000","last_peered":"2024-09-17T04:29:25.600555+0000","last_clean":"2024-09-17T04:29:25.600555+0000","last_became_active":"2024-09-17T04:29:23.126291+0000","last_became_peered":"2024-09-17T04:29:23.126291+0000","last_unstale":"2024-09-17T04:29:25.600555+0000","last_undegraded":"2024-09-17T04:29:25.600555+0000","last_fullsized":"2024-09-17T04:29:25.600555+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:27.163511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.343922+0000","last_change":"2024-09-17T04:29:22.114473+0000","last_active":"2024-09-17T04:29:24.343922+0000","last_peered":"2024-09-17T04:29:24.343922+0000","last_clean":"2024-09-17T04:29:24.343922+0000","last_became_active":"2024-09-17T04:29:22.110485+0000","last_became_peered":"2024-09-17T04:29:22.110485+0000","last_unstale":"2024-09-17T04:29:24.343922+0000","last_undegraded":"2024-09-17T04:29:24.343922+0000","last_fullsized":"2024-09-17T04:29:24.343922+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:46.472432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.123875+0000","last_change":"2024-09-17T04:29:23.121593+0000","last_active":"2024-09-17T04:29:24.123875+0000","last_peered":"2024-09-17T04:29:24.123875+0000","last_clean":"2024-09-17T04:29:24.123875+0000","last_became_active":"2024-09-17T04:29:23.121227+0000","last_became_peered":"2024-09-17T04:29:23.121227+0000","last_unstale":"2024-09-17T04:29:24.123875+0000","last_undegraded":"2024-09-17T04:29:24.123875+0000","last_fullsized":"2024-09-17T04:29:24.123875+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:31.361849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.343646+0000","last_change":"2024-09-17T04:29:22.115695+0000","last_active":"2024-09-17T04:29:24.343646+0000","last_peered":"2024-09-17T04:29:24.343646+0000","last_clean":"2024-09-17T04:29:24.343646+0000","last_became_active":"2024-09-17T04:29:22.115405+0000","last_became_peered":"2024-09-17T04:29:22.115405+0000","last_unstale":"2024-09-17T04:29:24.343646+0000","last_undegraded":"2024-09-17T04:29:24.343646+0000","last_fullsized":"2024-09-17T04:29:24.343646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:13.639491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.436525+0000","last_change":"2024-09-17T04:29:23.126842+0000","last_active":"2024-09-17T04:29:24.436525+0000","last_peered":"2024-09-17T04:29:24.436525+0000","last_clean":"2024-09-17T04:29:24.436525+0000","last_became_active":"2024-09-17T04:29:23.126255+0000","last_became_peered":"2024-09-17T04:29:23.126255+0000","last_unstale":"2024-09-17T04:29:24.436525+0000","last_undegraded":"2024-09-17T04:29:24.436525+0000","last_fullsized":"2024-09-17T04:29:24.436525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:55:18.771747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.599196+0000","last_change":"2024-09-17T04:29:22.119978+0000","last_active":"2024-09-17T04:29:25.599196+0000","last_peered":"2024-09-17T04:29:25.599196+0000","last_clean":"2024-09-17T04:29:25.599196+0000","last_became_active":"2024-09-17T04:29:22.119632+0000","last_became_peered":"2024-09-17T04:29:22.119632+0000","last_unstale":"2024-09-17T04:29:25.599196+0000","last_undegraded":"2024-09-17T04:29:25.599196+0000","last_fullsized":"2024-09-17T04:29:25.599196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:51.703195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.599134+0000","last_change":"2024-09-17T04:29:23.105317+0000","last_active":"2024-09-17T04:29:25.599134+0000","last_peered":"2024-09-17T04:29:25.599134+0000","last_clean":"2024-09-17T04:29:25.599134+0000","last_became_active":"2024-09-17T04:29:23.105024+0000","last_became_peered":"2024-09-17T04:29:23.105024+0000","last_unstale":"2024-09-17T04:29:25.599134+0000","last_undegraded":"2024-09-17T04:29:25.599134+0000","last_fullsized":"2024-09-17T04:29:25.599134+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:05:33.531403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.599315+0000","last_change":"2024-09-17T04:29:22.117911+0000","last_active":"2024-09-17T04:29:25.599315+0000","last_peered":"2024-09-17T04:29:25.599315+0000","last_clean":"2024-09-17T04:29:25.599315+0000","last_became_active":"2024-09-17T04:29:22.117425+0000","last_became_peered":"2024-09-17T04:29:22.117425+0000","last_unstale":"2024-09-17T04:29:25.599315+0000","last_undegraded":"2024-09-17T04:29:25.599315+0000","last_fullsized":"2024-09-17T04:29:25.599315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:55.553481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.124759+0000","last_change":"2024-09-17T04:29:23.125611+0000","last_active":"2024-09-17T04:29:24.124759+0000","last_peered":"2024-09-17T04:29:24.124759+0000","last_clean":"2024-09-17T04:29:24.124759+0000","last_became_active":"2024-09-17T04:29:23.124775+0000","last_became_peered":"2024-09-17T04:29:23.124775+0000","last_unstale":"2024-09-17T04:29:24.124759+0000","last_undegraded":"2024-09-17T04:29:24.124759+0000","last_fullsized":"2024-09-17T04:29:24.124759+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:46:39.176607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.343429+0000","last_change":"2024-09-17T04:29:22.111739+0000","last_active":"2024-09-17T04:29:24.343429+0000","last_peered":"2024-09-17T04:29:24.343429+0000","last_clean":"2024-09-17T04:29:24.343429+0000","last_became_active":"2024-09-17T04:29:22.111409+0000","last_became_peered":"2024-09-17T04:29:22.111409+0000","last_unstale":"2024-09-17T04:29:24.343429+0000","last_undegraded":"2024-09-17T04:29:24.343429+0000","last_fullsized":"2024-09-17T04:29:24.343429+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:56.977879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.116074+0000","last_change":"2024-09-17T04:29:23.108131+0000","last_active":"2024-09-17T04:29:25.116074+0000","last_peered":"2024-09-17T04:29:25.116074+0000","last_clean":"2024-09-17T04:29:25.116074+0000","last_became_active":"2024-09-17T04:29:23.107816+0000","last_became_peered":"2024-09-17T04:29:23.107816+0000","last_unstale":"2024-09-17T04:29:25.116074+0000","last_undegraded":"2024-09-17T04:29:25.116074+0000","last_fullsized":"2024-09-17T04:29:25.116074+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:26.157657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.436058+0000","last_change":"2024-09-17T04:29:22.109519+0000","last_active":"2024-09-17T04:29:24.436058+0000","last_peered":"2024-09-17T04:29:24.436058+0000","last_clean":"2024-09-17T04:29:24.436058+0000","last_became_active":"2024-09-17T04:29:22.109162+0000","last_became_peered":"2024-09-17T04:29:22.109162+0000","last_unstale":"2024-09-17T04:29:24.436058+0000","last_undegraded":"2024-09-17T04:29:24.436058+0000","last_fullsized":"2024-09-17T04:29:24.436058+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:47.246605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.116156+0000","last_change":"2024-09-17T04:29:23.115959+0000","last_active":"2024-09-17T04:29:25.116156+0000","last_peered":"2024-09-17T04:29:25.116156+0000","last_clean":"2024-09-17T04:29:25.116156+0000","last_became_active":"2024-09-17T04:29:23.115522+0000","last_became_peered":"2024-09-17T04:29:23.115522+0000","last_unstale":"2024-09-17T04:29:25.116156+0000","last_undegraded":"2024-09-17T04:29:25.116156+0000","last_fullsized":"2024-09-17T04:29:25.116156+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:58.551162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.600518+0000","last_change":"2024-09-17T04:29:22.118937+0000","last_active":"2024-09-17T04:29:25.600518+0000","last_peered":"2024-09-17T04:29:25.600518+0000","last_clean":"2024-09-17T04:29:25.600518+0000","last_became_active":"2024-09-17T04:29:22.118327+0000","last_became_peered":"2024-09-17T04:29:22.118327+0000","last_unstale":"2024-09-17T04:29:25.600518+0000","last_undegraded":"2024-09-17T04:29:25.600518+0000","last_fullsized":"2024-09-17T04:29:25.600518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:05:01.017955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.125161+0000","last_change":"2024-09-17T04:29:23.099661+0000","last_active":"2024-09-17T04:29:24.125161+0000","last_peered":"2024-09-17T04:29:24.125161+0000","last_clean":"2024-09-17T04:29:24.125161+0000","last_became_active":"2024-09-17T04:29:23.099295+0000","last_became_peered":"2024-09-17T04:29:23.099295+0000","last_unstale":"2024-09-17T04:29:24.125161+0000","last_undegraded":"2024-09-17T04:29:24.125161+0000","last_fullsized":"2024-09-17T04:29:24.125161+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:31.361849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.600250+0000","last_change":"2024-09-17T04:29:22.112665+0000","last_active":"2024-09-17T04:29:25.600250+0000","last_peered":"2024-09-17T04:29:25.600250+0000","last_clean":"2024-09-17T04:29:25.600250+0000","last_became_active":"2024-09-17T04:29:22.112212+0000","last_became_peered":"2024-09-17T04:29:22.112212+0000","last_unstale":"2024-09-17T04:29:25.600250+0000","last_undegraded":"2024-09-17T04:29:25.600250+0000","last_fullsized":"2024-09-17T04:29:25.600250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:51.703195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.343714+0000","last_change":"2024-09-17T04:29:23.117076+0000","last_active":"2024-09-17T04:29:24.343714+0000","last_peered":"2024-09-17T04:29:24.343714+0000","last_clean":"2024-09-17T04:29:24.343714+0000","last_became_active":"2024-09-17T04:29:23.116755+0000","last_became_peered":"2024-09-17T04:29:23.116755+0000","last_unstale":"2024-09-17T04:29:24.343714+0000","last_undegraded":"2024-09-17T04:29:24.343714+0000","last_fullsized":"2024-09-17T04:29:24.343714+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:01.221273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"86'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.599656+0000","last_change":"2024-09-17T04:29:22.120323+0000","last_active":"2024-09-17T04:29:25.599656+0000","last_peered":"2024-09-17T04:29:25.599656+0000","last_clean":"2024-09-17T04:29:25.599656+0000","last_became_active":"2024-09-17T04:29:22.119974+0000","last_became_peered":"2024-09-17T04:29:22.119974+0000","last_unstale":"2024-09-17T04:29:25.599656+0000","last_undegraded":"2024-09-17T04:29:25.599656+0000","last_fullsized":"2024-09-17T04:29:25.599656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:56:36.362685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.599627+0000","last_change":"2024-09-17T04:29:23.120535+0000","last_active":"2024-09-17T04:29:25.599627+0000","last_peered":"2024-09-17T04:29:25.599627+0000","last_clean":"2024-09-17T04:29:25.599627+0000","last_became_active":"2024-09-17T04:29:23.117663+0000","last_became_peered":"2024-09-17T04:29:23.117663+0000","last_unstale":"2024-09-17T04:29:25.599627+0000","last_undegraded":"2024-09-17T04:29:25.599627+0000","last_fullsized":"2024-09-17T04:29:25.599627+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:58.174802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"86'6","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.123334+0000","last_change":"2024-09-17T04:29:22.105137+0000","last_active":"2024-09-17T04:29:24.123334+0000","last_peered":"2024-09-17T04:29:24.123334+0000","last_clean":"2024-09-17T04:29:24.123334+0000","last_became_active":"2024-09-17T04:29:22.104043+0000","last_became_peered":"2024-09-17T04:29:22.104043+0000","last_unstale":"2024-09-17T04:29:24.123334+0000","last_undegraded":"2024-09-17T04:29:24.123334+0000","last_fullsized":"2024-09-17T04:29:24.123334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:02.021212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.343647+0000","last_change":"2024-09-17T04:29:23.126683+0000","last_active":"2024-09-17T04:29:24.343647+0000","last_peered":"2024-09-17T04:29:24.343647+0000","last_clean":"2024-09-17T04:29:24.343647+0000","last_became_active":"2024-09-17T04:29:23.126398+0000","last_became_peered":"2024-09-17T04:29:23.126398+0000","last_unstale":"2024-09-17T04:29:24.343647+0000","last_undegraded":"2024-09-17T04:29:24.343647+0000","last_fullsized":"2024-09-17T04:29:24.343647+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:30.560187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.436122+0000","last_change":"2024-09-17T04:29:22.112841+0000","last_active":"2024-09-17T04:29:24.436122+0000","last_peered":"2024-09-17T04:29:24.436122+0000","last_clean":"2024-09-17T04:29:24.436122+0000","last_became_active":"2024-09-17T04:29:22.112483+0000","last_became_peered":"2024-09-17T04:29:22.112483+0000","last_unstale":"2024-09-17T04:29:24.436122+0000","last_undegraded":"2024-09-17T04:29:24.436122+0000","last_fullsized":"2024-09-17T04:29:24.436122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:20.452471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.124275+0000","last_change":"2024-09-17T04:29:23.125951+0000","last_active":"2024-09-17T04:29:24.124275+0000","last_peered":"2024-09-17T04:29:24.124275+0000","last_clean":"2024-09-17T04:29:24.124275+0000","last_became_active":"2024-09-17T04:29:23.125560+0000","last_became_peered":"2024-09-17T04:29:23.125560+0000","last_unstale":"2024-09-17T04:29:24.124275+0000","last_undegraded":"2024-09-17T04:29:24.124275+0000","last_fullsized":"2024-09-17T04:29:24.124275+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:01.221273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.115788+0000","last_change":"2024-09-17T04:29:22.110597+0000","last_active":"2024-09-17T04:29:25.115788+0000","last_peered":"2024-09-17T04:29:25.115788+0000","last_clean":"2024-09-17T04:29:25.115788+0000","last_became_active":"2024-09-17T04:29:22.110258+0000","last_became_peered":"2024-09-17T04:29:22.110258+0000","last_unstale":"2024-09-17T04:29:25.115788+0000","last_undegraded":"2024-09-17T04:29:25.115788+0000","last_fullsized":"2024-09-17T04:29:25.115788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:08.063351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.436239+0000","last_change":"2024-09-17T04:29:23.125417+0000","last_active":"2024-09-17T04:29:24.436239+0000","last_peered":"2024-09-17T04:29:24.436239+0000","last_clean":"2024-09-17T04:29:24.436239+0000","last_became_active":"2024-09-17T04:29:23.125096+0000","last_became_peered":"2024-09-17T04:29:23.125096+0000","last_unstale":"2024-09-17T04:29:24.436239+0000","last_undegraded":"2024-09-17T04:29:24.436239+0000","last_fullsized":"2024-09-17T04:29:24.436239+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:11:45.036272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.124516+0000","last_change":"2024-09-17T04:29:22.110061+0000","last_active":"2024-09-17T04:29:24.124516+0000","last_peered":"2024-09-17T04:29:24.124516+0000","last_clean":"2024-09-17T04:29:24.124516+0000","last_became_active":"2024-09-17T04:29:22.109085+0000","last_became_peered":"2024-09-17T04:29:22.109085+0000","last_unstale":"2024-09-17T04:29:24.124516+0000","last_undegraded":"2024-09-17T04:29:24.124516+0000","last_fullsized":"2024-09-17T04:29:24.124516+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:55.805651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.436196+0000","last_change":"2024-09-17T04:29:23.120461+0000","last_active":"2024-09-17T04:29:24.436196+0000","last_peered":"2024-09-17T04:29:24.436196+0000","last_clean":"2024-09-17T04:29:24.436196+0000","last_became_active":"2024-09-17T04:29:23.120163+0000","last_became_peered":"2024-09-17T04:29:23.120163+0000","last_unstale":"2024-09-17T04:29:24.436196+0000","last_undegraded":"2024-09-17T04:29:24.436196+0000","last_fullsized":"2024-09-17T04:29:24.436196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:16.142896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"86'8","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.343269+0000","last_change":"2024-09-17T04:29:22.110820+0000","last_active":"2024-09-17T04:29:24.343269+0000","last_peered":"2024-09-17T04:29:24.343269+0000","last_clean":"2024-09-17T04:29:24.343269+0000","last_became_active":"2024-09-17T04:29:22.110509+0000","last_became_peered":"2024-09-17T04:29:22.110509+0000","last_unstale":"2024-09-17T04:29:24.343269+0000","last_undegraded":"2024-09-17T04:29:24.343269+0000","last_fullsized":"2024-09-17T04:29:24.343269+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:34.216990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.124089+0000","last_change":"2024-09-17T04:29:23.109415+0000","last_active":"2024-09-17T04:29:24.124089+0000","last_peered":"2024-09-17T04:29:24.124089+0000","last_clean":"2024-09-17T04:29:24.124089+0000","last_became_active":"2024-09-17T04:29:23.108403+0000","last_became_peered":"2024-09-17T04:29:23.108403+0000","last_unstale":"2024-09-17T04:29:24.124089+0000","last_undegraded":"2024-09-17T04:29:24.124089+0000","last_fullsized":"2024-09-17T04:29:24.124089+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:44:26.709262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.600640+0000","last_change":"2024-09-17T04:29:22.108783+0000","last_active":"2024-09-17T04:29:25.600640+0000","last_peered":"2024-09-17T04:29:25.600640+0000","last_clean":"2024-09-17T04:29:25.600640+0000","last_became_active":"2024-09-17T04:29:22.108444+0000","last_became_peered":"2024-09-17T04:29:22.108444+0000","last_unstale":"2024-09-17T04:29:25.600640+0000","last_undegraded":"2024-09-17T04:29:25.600640+0000","last_fullsized":"2024-09-17T04:29:25.600640+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:12:35.682491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.600454+0000","last_change":"2024-09-17T04:29:23.114559+0000","last_active":"2024-09-17T04:29:25.600454+0000","last_peered":"2024-09-17T04:29:25.600454+0000","last_clean":"2024-09-17T04:29:25.600454+0000","last_became_active":"2024-09-17T04:29:23.113850+0000","last_became_peered":"2024-09-17T04:29:23.113850+0000","last_unstale":"2024-09-17T04:29:25.600454+0000","last_undegraded":"2024-09-17T04:29:25.600454+0000","last_fullsized":"2024-09-17T04:29:25.600454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:01.221273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.343965+0000","last_change":"2024-09-17T04:29:22.115510+0000","last_active":"2024-09-17T04:29:24.343965+0000","last_peered":"2024-09-17T04:29:24.343965+0000","last_clean":"2024-09-17T04:29:24.343965+0000","last_became_active":"2024-09-17T04:29:22.113466+0000","last_became_peered":"2024-09-17T04:29:22.113466+0000","last_unstale":"2024-09-17T04:29:24.343965+0000","last_undegraded":"2024-09-17T04:29:24.343965+0000","last_fullsized":"2024-09-17T04:29:24.343965+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:05:31.947632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.599904+0000","last_change":"2024-09-17T04:29:23.120672+0000","last_active":"2024-09-17T04:29:25.599904+0000","last_peered":"2024-09-17T04:29:25.599904+0000","last_clean":"2024-09-17T04:29:25.599904+0000","last_became_active":"2024-09-17T04:29:23.119802+0000","last_became_peered":"2024-09-17T04:29:23.119802+0000","last_unstale":"2024-09-17T04:29:25.599904+0000","last_undegraded":"2024-09-17T04:29:25.599904+0000","last_fullsized":"2024-09-17T04:29:25.599904+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:05:32.951310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.343786+0000","last_change":"2024-09-17T04:29:22.114093+0000","last_active":"2024-09-17T04:29:24.343786+0000","last_peered":"2024-09-17T04:29:24.343786+0000","last_clean":"2024-09-17T04:29:24.343786+0000","last_became_active":"2024-09-17T04:29:22.113050+0000","last_became_peered":"2024-09-17T04:29:22.113050+0000","last_unstale":"2024-09-17T04:29:24.343786+0000","last_undegraded":"2024-09-17T04:29:24.343786+0000","last_fullsized":"2024-09-17T04:29:24.343786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:42.785431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.122645+0000","last_change":"2024-09-17T04:29:23.132305+0000","last_active":"2024-09-17T04:29:24.122645+0000","last_peered":"2024-09-17T04:29:24.122645+0000","last_clean":"2024-09-17T04:29:24.122645+0000","last_became_active":"2024-09-17T04:29:23.125817+0000","last_became_peered":"2024-09-17T04:29:23.125817+0000","last_unstale":"2024-09-17T04:29:24.122645+0000","last_undegraded":"2024-09-17T04:29:24.122645+0000","last_fullsized":"2024-09-17T04:29:24.122645+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:20.433749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.115983+0000","last_change":"2024-09-17T04:29:22.105163+0000","last_active":"2024-09-17T04:29:25.115983+0000","last_peered":"2024-09-17T04:29:25.115983+0000","last_clean":"2024-09-17T04:29:25.115983+0000","last_became_active":"2024-09-17T04:29:22.104907+0000","last_became_peered":"2024-09-17T04:29:22.104907+0000","last_unstale":"2024-09-17T04:29:25.115983+0000","last_undegraded":"2024-09-17T04:29:25.115983+0000","last_fullsized":"2024-09-17T04:29:25.115983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:19:24.223467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.124431+0000","last_change":"2024-09-17T04:29:23.109256+0000","last_active":"2024-09-17T04:29:24.124431+0000","last_peered":"2024-09-17T04:29:24.124431+0000","last_clean":"2024-09-17T04:29:24.124431+0000","last_became_active":"2024-09-17T04:29:23.108311+0000","last_became_peered":"2024-09-17T04:29:23.108311+0000","last_unstale":"2024-09-17T04:29:24.124431+0000","last_undegraded":"2024-09-17T04:29:24.124431+0000","last_fullsized":"2024-09-17T04:29:24.124431+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:05:42.014439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.122758+0000","last_change":"2024-09-17T04:29:22.112493+0000","last_active":"2024-09-17T04:29:24.122758+0000","last_peered":"2024-09-17T04:29:24.122758+0000","last_clean":"2024-09-17T04:29:24.122758+0000","last_became_active":"2024-09-17T04:29:22.112174+0000","last_became_peered":"2024-09-17T04:29:22.112174+0000","last_unstale":"2024-09-17T04:29:24.122758+0000","last_undegraded":"2024-09-17T04:29:24.122758+0000","last_fullsized":"2024-09-17T04:29:24.122758+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:13.517465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.122155+0000","last_change":"2024-09-17T04:29:23.106516+0000","last_active":"2024-09-17T04:29:24.122155+0000","last_peered":"2024-09-17T04:29:24.122155+0000","last_clean":"2024-09-17T04:29:24.122155+0000","last_became_active":"2024-09-17T04:29:23.106189+0000","last_became_peered":"2024-09-17T04:29:23.106189+0000","last_unstale":"2024-09-17T04:29:24.122155+0000","last_undegraded":"2024-09-17T04:29:24.122155+0000","last_fullsized":"2024-09-17T04:29:24.122155+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:10:32.781691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.121976+0000","last_change":"2024-09-17T04:29:22.106490+0000","last_active":"2024-09-17T04:29:24.121976+0000","last_peered":"2024-09-17T04:29:24.121976+0000","last_clean":"2024-09-17T04:29:24.121976+0000","last_became_active":"2024-09-17T04:29:22.104512+0000","last_became_peered":"2024-09-17T04:29:22.104512+0000","last_unstale":"2024-09-17T04:29:24.121976+0000","last_undegraded":"2024-09-17T04:29:24.121976+0000","last_fullsized":"2024-09-17T04:29:24.121976+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:42.849310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.124599+0000","last_change":"2024-09-17T04:29:23.106148+0000","last_active":"2024-09-17T04:29:24.124599+0000","last_peered":"2024-09-17T04:29:24.124599+0000","last_clean":"2024-09-17T04:29:24.124599+0000","last_became_active":"2024-09-17T04:29:23.105123+0000","last_became_peered":"2024-09-17T04:29:23.105123+0000","last_unstale":"2024-09-17T04:29:24.124599+0000","last_undegraded":"2024-09-17T04:29:24.124599+0000","last_fullsized":"2024-09-17T04:29:24.124599+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:41:13.472551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.115614+0000","last_change":"2024-09-17T04:29:22.104235+0000","last_active":"2024-09-17T04:29:25.115614+0000","last_peered":"2024-09-17T04:29:25.115614+0000","last_clean":"2024-09-17T04:29:25.115614+0000","last_became_active":"2024-09-17T04:29:22.103945+0000","last_became_peered":"2024-09-17T04:29:22.103945+0000","last_unstale":"2024-09-17T04:29:25.115614+0000","last_undegraded":"2024-09-17T04:29:25.115614+0000","last_fullsized":"2024-09-17T04:29:25.115614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:11:51.844445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.599990+0000","last_change":"2024-09-17T04:29:23.100821+0000","last_active":"2024-09-17T04:29:25.599990+0000","last_peered":"2024-09-17T04:29:25.599990+0000","last_clean":"2024-09-17T04:29:25.599990+0000","last_became_active":"2024-09-17T04:29:23.100507+0000","last_became_peered":"2024-09-17T04:29:23.100507+0000","last_unstale":"2024-09-17T04:29:25.599990+0000","last_undegraded":"2024-09-17T04:29:25.599990+0000","last_fullsized":"2024-09-17T04:29:25.599990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:53:12.306254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.436344+0000","last_change":"2024-09-17T04:29:22.116131+0000","last_active":"2024-09-17T04:29:24.436344+0000","last_peered":"2024-09-17T04:29:24.436344+0000","last_clean":"2024-09-17T04:29:24.436344+0000","last_became_active":"2024-09-17T04:29:22.115224+0000","last_became_peered":"2024-09-17T04:29:22.115224+0000","last_unstale":"2024-09-17T04:29:24.436344+0000","last_undegraded":"2024-09-17T04:29:24.436344+0000","last_fullsized":"2024-09-17T04:29:24.436344+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:56:11.870729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.124981+0000","last_change":"2024-09-17T04:29:23.120744+0000","last_active":"2024-09-17T04:29:24.124981+0000","last_peered":"2024-09-17T04:29:24.124981+0000","last_clean":"2024-09-17T04:29:24.124981+0000","last_became_active":"2024-09-17T04:29:23.120419+0000","last_became_peered":"2024-09-17T04:29:23.120419+0000","last_unstale":"2024-09-17T04:29:24.124981+0000","last_undegraded":"2024-09-17T04:29:24.124981+0000","last_fullsized":"2024-09-17T04:29:24.124981+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:00:58.554738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.436565+0000","last_change":"2024-09-17T04:29:22.111668+0000","last_active":"2024-09-17T04:29:24.436565+0000","last_peered":"2024-09-17T04:29:24.436565+0000","last_clean":"2024-09-17T04:29:24.436565+0000","last_became_active":"2024-09-17T04:29:22.111361+0000","last_became_peered":"2024-09-17T04:29:22.111361+0000","last_unstale":"2024-09-17T04:29:24.436565+0000","last_undegraded":"2024-09-17T04:29:24.436565+0000","last_fullsized":"2024-09-17T04:29:24.436565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:43:45.542781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.600712+0000","last_change":"2024-09-17T04:29:23.125071+0000","last_active":"2024-09-17T04:29:25.600712+0000","last_peered":"2024-09-17T04:29:25.600712+0000","last_clean":"2024-09-17T04:29:25.600712+0000","last_became_active":"2024-09-17T04:29:23.124674+0000","last_became_peered":"2024-09-17T04:29:23.124674+0000","last_unstale":"2024-09-17T04:29:25.600712+0000","last_undegraded":"2024-09-17T04:29:25.600712+0000","last_fullsized":"2024-09-17T04:29:25.600712+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:30:14.565815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.599910+0000","last_change":"2024-09-17T04:29:22.117370+0000","last_active":"2024-09-17T04:29:25.599910+0000","last_peered":"2024-09-17T04:29:25.599910+0000","last_clean":"2024-09-17T04:29:25.599910+0000","last_became_active":"2024-09-17T04:29:22.116999+0000","last_became_peered":"2024-09-17T04:29:22.116999+0000","last_unstale":"2024-09-17T04:29:25.599910+0000","last_undegraded":"2024-09-17T04:29:25.599910+0000","last_fullsized":"2024-09-17T04:29:25.599910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:55.553481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.600200+0000","last_change":"2024-09-17T04:29:23.112223+0000","last_active":"2024-09-17T04:29:25.600200+0000","last_peered":"2024-09-17T04:29:25.600200+0000","last_clean":"2024-09-17T04:29:25.600200+0000","last_became_active":"2024-09-17T04:29:23.111824+0000","last_became_peered":"2024-09-17T04:29:23.111824+0000","last_unstale":"2024-09-17T04:29:25.600200+0000","last_undegraded":"2024-09-17T04:29:25.600200+0000","last_fullsized":"2024-09-17T04:29:25.600200+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:44:26.709262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.600659+0000","last_change":"2024-09-17T04:29:22.109724+0000","last_active":"2024-09-17T04:29:25.600659+0000","last_peered":"2024-09-17T04:29:25.600659+0000","last_clean":"2024-09-17T04:29:25.600659+0000","last_became_active":"2024-09-17T04:29:22.109396+0000","last_became_peered":"2024-09-17T04:29:22.109396+0000","last_unstale":"2024-09-17T04:29:25.600659+0000","last_undegraded":"2024-09-17T04:29:25.600659+0000","last_fullsized":"2024-09-17T04:29:25.600659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:23.221109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.124679+0000","last_change":"2024-09-17T04:29:23.107656+0000","last_active":"2024-09-17T04:29:24.124679+0000","last_peered":"2024-09-17T04:29:24.124679+0000","last_clean":"2024-09-17T04:29:24.124679+0000","last_became_active":"2024-09-17T04:29:23.106837+0000","last_became_peered":"2024-09-17T04:29:23.106837+0000","last_unstale":"2024-09-17T04:29:24.124679+0000","last_undegraded":"2024-09-17T04:29:24.124679+0000","last_fullsized":"2024-09-17T04:29:24.124679+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:00:29.248833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.343802+0000","last_change":"2024-09-17T04:29:22.113223+0000","last_active":"2024-09-17T04:29:24.343802+0000","last_peered":"2024-09-17T04:29:24.343802+0000","last_clean":"2024-09-17T04:29:24.343802+0000","last_became_active":"2024-09-17T04:29:22.110897+0000","last_became_peered":"2024-09-17T04:29:22.110897+0000","last_unstale":"2024-09-17T04:29:24.343802+0000","last_undegraded":"2024-09-17T04:29:24.343802+0000","last_fullsized":"2024-09-17T04:29:24.343802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:05:32.527725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.343716+0000","last_change":"2024-09-17T04:29:23.111444+0000","last_active":"2024-09-17T04:29:24.343716+0000","last_peered":"2024-09-17T04:29:24.343716+0000","last_clean":"2024-09-17T04:29:24.343716+0000","last_became_active":"2024-09-17T04:29:23.111129+0000","last_became_peered":"2024-09-17T04:29:23.111129+0000","last_unstale":"2024-09-17T04:29:24.343716+0000","last_undegraded":"2024-09-17T04:29:24.343716+0000","last_fullsized":"2024-09-17T04:29:24.343716+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:50:34.402970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.599169+0000","last_change":"2024-09-17T04:29:22.118261+0000","last_active":"2024-09-17T04:29:25.599169+0000","last_peered":"2024-09-17T04:29:25.599169+0000","last_clean":"2024-09-17T04:29:25.599169+0000","last_became_active":"2024-09-17T04:29:22.116999+0000","last_became_peered":"2024-09-17T04:29:22.116999+0000","last_unstale":"2024-09-17T04:29:25.599169+0000","last_undegraded":"2024-09-17T04:29:25.599169+0000","last_fullsized":"2024-09-17T04:29:25.599169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:23.221109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.123615+0000","last_change":"2024-09-17T04:29:23.128614+0000","last_active":"2024-09-17T04:29:24.123615+0000","last_peered":"2024-09-17T04:29:24.123615+0000","last_clean":"2024-09-17T04:29:24.123615+0000","last_became_active":"2024-09-17T04:29:23.128298+0000","last_became_peered":"2024-09-17T04:29:23.128298+0000","last_unstale":"2024-09-17T04:29:24.123615+0000","last_undegraded":"2024-09-17T04:29:24.123615+0000","last_fullsized":"2024-09-17T04:29:24.123615+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:47.476110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.599882+0000","last_change":"2024-09-17T04:29:22.110740+0000","last_active":"2024-09-17T04:29:25.599882+0000","last_peered":"2024-09-17T04:29:25.599882+0000","last_clean":"2024-09-17T04:29:25.599882+0000","last_became_active":"2024-09-17T04:29:22.110373+0000","last_became_peered":"2024-09-17T04:29:22.110373+0000","last_unstale":"2024-09-17T04:29:25.599882+0000","last_undegraded":"2024-09-17T04:29:25.599882+0000","last_fullsized":"2024-09-17T04:29:25.599882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:12.167796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.599690+0000","last_change":"2024-09-17T04:29:23.123463+0000","last_active":"2024-09-17T04:29:25.599690+0000","last_peered":"2024-09-17T04:29:25.599690+0000","last_clean":"2024-09-17T04:29:25.599690+0000","last_became_active":"2024-09-17T04:29:23.122871+0000","last_became_peered":"2024-09-17T04:29:23.122871+0000","last_unstale":"2024-09-17T04:29:25.599690+0000","last_undegraded":"2024-09-17T04:29:25.599690+0000","last_fullsized":"2024-09-17T04:29:25.599690+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:58.174802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:25.598910+0000","last_change":"2024-09-17T04:29:22.110589+0000","last_active":"2024-09-17T04:29:25.598910+0000","last_peered":"2024-09-17T04:29:25.598910+0000","last_clean":"2024-09-17T04:29:25.598910+0000","last_became_active":"2024-09-17T04:29:22.110249+0000","last_became_peered":"2024-09-17T04:29:22.110249+0000","last_unstale":"2024-09-17T04:29:25.598910+0000","last_undegraded":"2024-09-17T04:29:25.598910+0000","last_fullsized":"2024-09-17T04:29:25.598910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:12.167796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.436592+0000","last_change":"2024-09-17T04:29:23.124813+0000","last_active":"2024-09-17T04:29:24.436592+0000","last_peered":"2024-09-17T04:29:24.436592+0000","last_clean":"2024-09-17T04:29:24.436592+0000","last_became_active":"2024-09-17T04:29:23.124484+0000","last_became_peered":"2024-09-17T04:29:23.124484+0000","last_unstale":"2024-09-17T04:29:24.436592+0000","last_undegraded":"2024-09-17T04:29:24.436592+0000","last_fullsized":"2024-09-17T04:29:24.436592+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:22.080485+0000","last_clean_scrub_stamp":"2024-09-17T04:29:22.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:52.744331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.123969+0000","last_change":"2024-09-17T04:29:22.116895+0000","last_active":"2024-09-17T04:29:24.123969+0000","last_peered":"2024-09-17T04:29:24.123969+0000","last_clean":"2024-09-17T04:29:24.123969+0000","last_became_active":"2024-09-17T04:29:22.116525+0000","last_became_peered":"2024-09-17T04:29:22.116525+0000","last_unstale":"2024-09-17T04:29:24.123969+0000","last_undegraded":"2024-09-17T04:29:24.123969+0000","last_fullsized":"2024-09-17T04:29:24.123969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:21.076807+0000","last_clean_scrub_stamp":"2024-09-17T04:29:21.076807+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:51.737893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"76'79","reported_seq":258,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T04:29:24.123727+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:29:24.123727+0000","last_peered":"2024-09-17T04:29:24.123727+0000","last_clean":"2024-09-17T04:29:24.123727+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:29:24.123727+0000","last_undegraded":"2024-09-17T04:29:24.123727+0000","last_fullsized":"2024-09-17T04:29:24.123727+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476806,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32804,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339036,"statfs":{"total":96636764160,"available":96603172864,"internally_reserved":0,"allocated":1654784,"data_stored":1043453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476805,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1069056,"data_stored":453709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476806,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32236,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339604,"statfs":{"total":96636764160,"available":96603754496,"internally_reserved":0,"allocated":1073152,"data_stored":454649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476806,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1056768,"data_stored":453015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542213,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1069056,"data_stored":454627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509509,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1060864,"data_stored":453597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509508,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1069056,"data_stored":453209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509508,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32804,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339036,"statfs":{"total":96636764160,"available":96603172864,"internally_reserved":0,"allocated":1654784,"data_stored":1043417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:29:28.028 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T04:29:28.028 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T04:29:28.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.292+0000 7fe4b9200640 1 -- 172.21.15.100:0/3049007364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4b4145550 msgr2=0x7fe4b406fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:28.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.292+0000 7fe4b9200640 1 --2- 172.21.15.100:0/3049007364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4b4145550 0x7fe4b406fb40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4009830 tx=0x7fe4a402a5a0 comp rx=0 tx=0).stop 2024-09-17T04:29:28.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.292+0000 7fe4b9200640 1 -- 172.21.15.100:0/3049007364 shutdown_connections 2024-09-17T04:29:28.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.292+0000 7fe4b9200640 1 --2- 172.21.15.100:0/3049007364 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4b4145a40 0x7fe4b4073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:28.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.292+0000 7fe4b9200640 1 --2- 172.21.15.100:0/3049007364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4b4145550 0x7fe4b406fb40 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:28.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.292+0000 7fe4b9200640 1 -- 172.21.15.100:0/3049007364 >> 172.21.15.100:0/3049007364 conn(0x7fe4b406d1c0 msgr2=0x7fe4b406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:28.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.292+0000 7fe4b9200640 1 -- 172.21.15.100:0/3049007364 shutdown_connections 2024-09-17T04:29:28.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.292+0000 7fe4b9200640 1 -- 172.21.15.100:0/3049007364 wait complete. 2024-09-17T04:29:28.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.294+0000 7fe4b9200640 1 Processor -- start 2024-09-17T04:29:28.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.295+0000 7fe4b9200640 1 -- start start 2024-09-17T04:29:28.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.296+0000 7fe4b9200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4b4073a80 0x7fe4b414d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:28.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.296+0000 7fe4b9200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4b4145550 0x7fe4b40a97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:28.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.296+0000 7fe4b9200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4b4145a40 0x7fe4b415ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:28.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.296+0000 7fe4b9200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b4160a30 con 0x7fe4b4073a80 2024-09-17T04:29:28.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.296+0000 7fe4b9200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b40ce940 con 0x7fe4b4145a40 2024-09-17T04:29:28.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.296+0000 7fe4b9200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b4160bb0 con 0x7fe4b4145550 2024-09-17T04:29:28.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.296+0000 7fe4b1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4b4073a80 0x7fe4b414d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:28.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.296+0000 7fe4b1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4b4073a80 0x7fe4b414d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33066/0 (socket says 172.21.15.100:33066) 2024-09-17T04:29:28.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.296+0000 7fe4b1e00640 1 -- 172.21.15.100:0/2922750370 learned_addr learned my addr 172.21.15.100:0/2922750370 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:28.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.296+0000 7fe4b1400640 1 --2- 172.21.15.100:0/2922750370 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4b4145550 0x7fe4b40a97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:28.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.296+0000 7fe4b2800640 1 --2- 172.21.15.100:0/2922750370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4b4145a40 0x7fe4b415ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:28.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.297+0000 7fe4b1e00640 1 -- 172.21.15.100:0/2922750370 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4b4145550 msgr2=0x7fe4b40a97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:28.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.297+0000 7fe4b1e00640 1 --2- 172.21.15.100:0/2922750370 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4b4145550 0x7fe4b40a97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:28.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.297+0000 7fe4b1e00640 1 -- 172.21.15.100:0/2922750370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4b4145a40 msgr2=0x7fe4b415ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:28.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.297+0000 7fe4b1e00640 1 --2- 172.21.15.100:0/2922750370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4b4145a40 0x7fe4b415ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:28.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.297+0000 7fe4b1e00640 1 -- 172.21.15.100:0/2922750370 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b414dac0 con 0x7fe4b4073a80 2024-09-17T04:29:28.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.297+0000 7fe4b1e00640 1 --2- 172.21.15.100:0/2922750370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4b4073a80 0x7fe4b414d390 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4000c00 tx=0x7fe4a4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:28.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.297+0000 7fe4aa000640 1 -- 172.21.15.100:0/2922750370 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4031d60 con 0x7fe4b4073a80 2024-09-17T04:29:28.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.298+0000 7fe4aa000640 1 -- 172.21.15.100:0/2922750370 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4a4043070 con 0x7fe4b4073a80 2024-09-17T04:29:28.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.298+0000 7fe4b9200640 1 -- 172.21.15.100:0/2922750370 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b414dda0 con 0x7fe4b4073a80 2024-09-17T04:29:28.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.298+0000 7fe4aa000640 1 -- 172.21.15.100:0/2922750370 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a40332f0 con 0x7fe4b4073a80 2024-09-17T04:29:28.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.298+0000 7fe4b9200640 1 -- 172.21.15.100:0/2922750370 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b414e360 con 0x7fe4b4073a80 2024-09-17T04:29:28.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.299+0000 7fe4b9200640 1 -- 172.21.15.100:0/2922750370 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4b41610f0 con 0x7fe4b4073a80 2024-09-17T04:29:28.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.300+0000 7fe4aa000640 1 -- 172.21.15.100:0/2922750370 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fe4a4033490 con 0x7fe4b4073a80 2024-09-17T04:29:28.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.301+0000 7fe4aa000640 1 --2- 172.21.15.100:0/2922750370 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe494078cb0 0x7fe49407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:28.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.301+0000 7fe4aa000640 1 -- 172.21.15.100:0/2922750370 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fe4a40bbd60 con 0x7fe4b4073a80 2024-09-17T04:29:28.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.301+0000 7fe4b1400640 1 --2- 172.21.15.100:0/2922750370 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe494078cb0 0x7fe49407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:28.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.302+0000 7fe4b1400640 1 --2- 172.21.15.100:0/2922750370 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe494078cb0 0x7fe49407b1c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40afa20 tx=0x7fe4a0009260 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:28.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.305+0000 7fe4aa000640 1 -- 172.21.15.100:0/2922750370 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe4a4083930 con 0x7fe4b4073a80 2024-09-17T04:29:28.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.557+0000 7fe4b9200640 1 -- 172.21.15.100:0/2922750370 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe4b4050ff0 con 0x7fe4b4073a80 2024-09-17T04:29:28.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.560+0000 7fe4aa000640 1 -- 172.21.15.100:0/2922750370 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7fe4a4088360 con 0x7fe4b4073a80 2024-09-17T04:29:28.561 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-09-17T04:29:28.561 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAsBOlmfYg9IBAABP1uRKGOugrA47VxwRLkjA== 2024-09-17T04:29:28.561 INFO:teuthology.orchestra.run.smithi100.stdout: caps mds = "allow" 2024-09-17T04:29:28.561 INFO:teuthology.orchestra.run.smithi100.stdout: caps mon = "allow r" 2024-09-17T04:29:28.561 INFO:teuthology.orchestra.run.smithi100.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T04:29:28.561 INFO:teuthology.orchestra.run.smithi100.stderr:updated caps for client.0 2024-09-17T04:29:28.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.563+0000 7fe4b9200640 1 -- 172.21.15.100:0/2922750370 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe494078cb0 msgr2=0x7fe49407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:28.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.563+0000 7fe4b9200640 1 --2- 172.21.15.100:0/2922750370 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe494078cb0 0x7fe49407b1c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40afa20 tx=0x7fe4a0009260 comp rx=0 tx=0).stop 2024-09-17T04:29:28.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.563+0000 7fe4b9200640 1 -- 172.21.15.100:0/2922750370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4b4073a80 msgr2=0x7fe4b414d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:28.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.563+0000 7fe4b9200640 1 --2- 172.21.15.100:0/2922750370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4b4073a80 0x7fe4b414d390 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4000c00 tx=0x7fe4a4033c20 comp rx=0 tx=0).stop 2024-09-17T04:29:28.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.563+0000 7fe4b9200640 1 -- 172.21.15.100:0/2922750370 shutdown_connections 2024-09-17T04:29:28.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.563+0000 7fe4b9200640 1 --2- 172.21.15.100:0/2922750370 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe494078cb0 0x7fe49407b1c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:28.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.563+0000 7fe4b9200640 1 --2- 172.21.15.100:0/2922750370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4b4145a40 0x7fe4b415ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:28.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.563+0000 7fe4b9200640 1 --2- 172.21.15.100:0/2922750370 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4b4145550 0x7fe4b40a97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:28.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.563+0000 7fe4b9200640 1 --2- 172.21.15.100:0/2922750370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4b4073a80 0x7fe4b414d390 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:28.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.563+0000 7fe4b9200640 1 -- 172.21.15.100:0/2922750370 >> 172.21.15.100:0/2922750370 conn(0x7fe4b406d1c0 msgr2=0x7fe4b4141aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:28.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.563+0000 7fe4b9200640 1 -- 172.21.15.100:0/2922750370 shutdown_connections 2024-09-17T04:29:28.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.563+0000 7fe4b9200640 1 -- 172.21.15.100:0/2922750370 wait complete. 2024-09-17T04:29:28.584 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:29:28.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.804+0000 7faa71400640 1 -- 172.21.15.100:0/4213688266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa6c15d760 msgr2=0x7faa6c1535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:28.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.804+0000 7faa71400640 1 --2- 172.21.15.100:0/4213688266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa6c15d760 0x7faa6c1535c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7faa6c075250 tx=0x7faa5c02a4d0 comp rx=0 tx=0).stop 2024-09-17T04:29:28.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.804+0000 7faa71400640 1 -- 172.21.15.100:0/4213688266 shutdown_connections 2024-09-17T04:29:28.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.804+0000 7faa71400640 1 --2- 172.21.15.100:0/4213688266 >> v2:172.21.15.100:3300/0 conn(0x7faa6c0ad0e0 0x7faa6c0ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:28.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.804+0000 7faa71400640 1 --2- 172.21.15.100:0/4213688266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa6c15d760 0x7faa6c1535c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:28.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.804+0000 7faa71400640 1 -- 172.21.15.100:0/4213688266 >> 172.21.15.100:0/4213688266 conn(0x7faa6c06d730 msgr2=0x7faa6c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:28.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.804+0000 7faa71400640 1 -- 172.21.15.100:0/4213688266 shutdown_connections 2024-09-17T04:29:28.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.804+0000 7faa71400640 1 -- 172.21.15.100:0/4213688266 wait complete. 2024-09-17T04:29:28.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.806+0000 7faa71400640 1 Processor -- start 2024-09-17T04:29:28.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.806+0000 7faa71400640 1 -- start start 2024-09-17T04:29:28.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.807+0000 7faa71400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa6c0ad0e0 0x7faa6c1f6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:28.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.807+0000 7faa71400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa6c15d760 0x7faa6c156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:28.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.807+0000 7faa71400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa6c160310 0x7faa6c149800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:28.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.807+0000 7faa71400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c0b4c20 con 0x7faa6c15d760 2024-09-17T04:29:28.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.807+0000 7faa71400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c153c70 con 0x7faa6c160310 2024-09-17T04:29:28.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.807+0000 7faa71400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faa6c0b4da0 con 0x7faa6c0ad0e0 2024-09-17T04:29:28.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.808+0000 7faa61e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa6c15d760 0x7faa6c156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:28.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.808+0000 7faa61e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa6c15d760 0x7faa6c156bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33092/0 (socket says 172.21.15.100:33092) 2024-09-17T04:29:28.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.808+0000 7faa61e00640 1 -- 172.21.15.100:0/1751312213 learned_addr learned my addr 172.21.15.100:0/1751312213 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:28.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.808+0000 7faa6a800640 1 --2- 172.21.15.100:0/1751312213 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa6c0ad0e0 0x7faa6c1f6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:28.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.808+0000 7faa6b200640 1 --2- 172.21.15.100:0/1751312213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa6c160310 0x7faa6c149800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:28.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.808+0000 7faa61e00640 1 -- 172.21.15.100:0/1751312213 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa6c0ad0e0 msgr2=0x7faa6c1f6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:28.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.808+0000 7faa61e00640 1 --2- 172.21.15.100:0/1751312213 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa6c0ad0e0 0x7faa6c1f6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:28.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.808+0000 7faa61e00640 1 -- 172.21.15.100:0/1751312213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa6c160310 msgr2=0x7faa6c149800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:28.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.808+0000 7faa61e00640 1 --2- 172.21.15.100:0/1751312213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa6c160310 0x7faa6c149800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:28.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.808+0000 7faa61e00640 1 -- 172.21.15.100:0/1751312213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6c0735e0 con 0x7faa6c15d760 2024-09-17T04:29:28.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.808+0000 7faa6b200640 1 --2- 172.21.15.100:0/1751312213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa6c160310 0x7faa6c149800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:28.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.809+0000 7faa61e00640 1 --2- 172.21.15.100:0/1751312213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa6c15d760 0x7faa6c156bf0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7faa5400ea10 tx=0x7faa5400eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:28.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.809+0000 7faa63400640 1 -- 172.21.15.100:0/1751312213 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5400cd80 con 0x7faa6c15d760 2024-09-17T04:29:28.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.809+0000 7faa63400640 1 -- 172.21.15.100:0/1751312213 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa54011560 con 0x7faa6c15d760 2024-09-17T04:29:28.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.809+0000 7faa63400640 1 -- 172.21.15.100:0/1751312213 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa54011970 con 0x7faa6c15d760 2024-09-17T04:29:28.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.809+0000 7faa71400640 1 -- 172.21.15.100:0/1751312213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa6c073a50 con 0x7faa6c15d760 2024-09-17T04:29:28.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.809+0000 7faa71400640 1 -- 172.21.15.100:0/1751312213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa6c1f5650 con 0x7faa6c15d760 2024-09-17T04:29:28.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.810+0000 7faa71400640 1 -- 172.21.15.100:0/1751312213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa6c161010 con 0x7faa6c15d760 2024-09-17T04:29:28.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.811+0000 7faa63400640 1 -- 172.21.15.100:0/1751312213 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7faa54011b10 con 0x7faa6c15d760 2024-09-17T04:29:28.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.812+0000 7faa63400640 1 --2- 172.21.15.100:0/1751312213 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7faa48078cb0 0x7faa4807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:28.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.812+0000 7faa63400640 1 -- 172.21.15.100:0/1751312213 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7faa5409bb90 con 0x7faa6c15d760 2024-09-17T04:29:28.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.812+0000 7faa6a800640 1 --2- 172.21.15.100:0/1751312213 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7faa48078cb0 0x7faa4807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:28.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.813+0000 7faa6a800640 1 --2- 172.21.15.100:0/1751312213 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7faa48078cb0 0x7faa4807b1c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faa5c033000 tx=0x7faa5c004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:28.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:28.816+0000 7faa63400640 1 -- 172.21.15.100:0/1751312213 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7faa54016030 con 0x7faa6c15d760 2024-09-17T04:29:29.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.037+0000 7faa71400640 1 -- 172.21.15.100:0/1751312213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa6c0737b0 con 0x7faa6c15d760 2024-09-17T04:29:29.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.038+0000 7faa63400640 1 -- 172.21.15.100:0/1751312213 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5237 (secure 0 0 0) 0x7faa54063da0 con 0x7faa6c15d760 2024-09-17T04:29:29.038 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:29.038 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":57,"btime":"2024-09-17T04:29:26:129057+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5825,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3480776805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3480776805},{"type":"v1","addr":"172.21.15.184:6835","nonce":3480776805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":6391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/3591753470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3591753470},{"type":"v1","addr":"172.21.15.100:6833","nonce":3591753470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6400,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/2063784289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2063784289},{"type":"v1","addr":"172.21.15.184:6801","nonce":2063784289}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:29:22.583149+0000","modified":"2024-09-17T04:29:25.614111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6606},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6606":{"gid":6606,"name":"c","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/2660333125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2660333125},{"type":"v1","addr":"172.21.15.100:6835","nonce":2660333125}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6606,"qdb_cluster":[6606]},"id":6}]} 2024-09-17T04:29:29.038 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 57 2024-09-17T04:29:29.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.040+0000 7faa71400640 1 -- 172.21.15.100:0/1751312213 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7faa48078cb0 msgr2=0x7faa4807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:29.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.040+0000 7faa71400640 1 --2- 172.21.15.100:0/1751312213 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7faa48078cb0 0x7faa4807b1c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faa5c033000 tx=0x7faa5c004730 comp rx=0 tx=0).stop 2024-09-17T04:29:29.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.040+0000 7faa71400640 1 -- 172.21.15.100:0/1751312213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa6c15d760 msgr2=0x7faa6c156bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:29.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.040+0000 7faa71400640 1 --2- 172.21.15.100:0/1751312213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa6c15d760 0x7faa6c156bf0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7faa5400ea10 tx=0x7faa5400eed0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.040+0000 7faa71400640 1 -- 172.21.15.100:0/1751312213 shutdown_connections 2024-09-17T04:29:29.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.040+0000 7faa71400640 1 --2- 172.21.15.100:0/1751312213 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7faa48078cb0 0x7faa4807b1c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.040+0000 7faa71400640 1 --2- 172.21.15.100:0/1751312213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa6c160310 0x7faa6c149800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.040+0000 7faa71400640 1 --2- 172.21.15.100:0/1751312213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa6c15d760 0x7faa6c156bf0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.040+0000 7faa71400640 1 --2- 172.21.15.100:0/1751312213 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa6c0ad0e0 0x7faa6c1f6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.040+0000 7faa71400640 1 -- 172.21.15.100:0/1751312213 >> 172.21.15.100:0/1751312213 conn(0x7faa6c06d730 msgr2=0x7faa6c15f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:29.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.040+0000 7faa71400640 1 -- 172.21.15.100:0/1751312213 shutdown_connections 2024-09-17T04:29:29.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.040+0000 7faa71400640 1 -- 172.21.15.100:0/1751312213 wait complete. 2024-09-17T04:29:29.467 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T04:29:22.583149+0000', 'modified': '2024-09-17T04:29:25.614111+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6606}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6606': {'gid': 6606, 'name': 'c', 'rank': 0, 'incarnation': 53, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6835/2660333125', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6834', 'nonce': 2660333125}, {'type': 'v1', 'addr': '172.21.15.100:6835', 'nonce': 2660333125}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6606, 'qdb_cluster': [6606]} 2024-09-17T04:29:29.467 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T04:29:29.467 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T04:29:29.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.688+0000 7f932f200640 1 -- 172.21.15.100:0/2279523078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9314004d40 msgr2=0x7f9314005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:29.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.688+0000 7f932f200640 1 --2- 172.21.15.100:0/2279523078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9314004d40 0x7f9314005150 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f9328002230 tx=0x7f9314005830 comp rx=0 tx=0).stop 2024-09-17T04:29:29.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.688+0000 7f932e800640 1 -- 172.21.15.100:0/2279523078 reap_dead start 2024-09-17T04:29:29.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.688+0000 7f932f200640 1 -- 172.21.15.100:0/2279523078 shutdown_connections 2024-09-17T04:29:29.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.688+0000 7f932f200640 1 -- 172.21.15.100:0/2279523078 >> 172.21.15.100:0/2279523078 conn(0x7f93300d1470 msgr2=0x7f9330073670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:29.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.688+0000 7f932f200640 1 -- 172.21.15.100:0/2279523078 shutdown_connections 2024-09-17T04:29:29.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.688+0000 7f932f200640 1 -- 172.21.15.100:0/2279523078 wait complete. 2024-09-17T04:29:29.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.690+0000 7f932f200640 1 Processor -- start 2024-09-17T04:29:29.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.691+0000 7f932f200640 1 -- start start 2024-09-17T04:29:29.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.692+0000 7f932f200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9314004d40 0x7f933014d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:29.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.692+0000 7f932f200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f933015e5c0 0x7f93301f6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:29.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.692+0000 7f932f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9330149420 0x7f933014b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:29.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.692+0000 7f932f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f933015c460 con 0x7f9330149420 2024-09-17T04:29:29.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.692+0000 7f932f200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f933015c2e0 con 0x7f933015e5c0 2024-09-17T04:29:29.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.692+0000 7f932f200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f933015c5e0 con 0x7f9314004d40 2024-09-17T04:29:29.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.692+0000 7f932e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9330149420 0x7f933014b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:29.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.692+0000 7f932e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9330149420 0x7f933014b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33106/0 (socket says 172.21.15.100:33106) 2024-09-17T04:29:29.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.692+0000 7f932e800640 1 -- 172.21.15.100:0/3299367888 learned_addr learned my addr 172.21.15.100:0/3299367888 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:29.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.693+0000 7f9325400640 1 --2- 172.21.15.100:0/3299367888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f933015e5c0 0x7f93301f6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:29.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.693+0000 7f932de00640 1 --2- 172.21.15.100:0/3299367888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9314004d40 0x7f933014d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:29.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.693+0000 7f932e800640 1 -- 172.21.15.100:0/3299367888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9314004d40 msgr2=0x7f933014d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:29.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.693+0000 7f932e800640 1 --2- 172.21.15.100:0/3299367888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9314004d40 0x7f933014d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.693+0000 7f932e800640 1 -- 172.21.15.100:0/3299367888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f933015e5c0 msgr2=0x7f93301f6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:29.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.693+0000 7f932e800640 1 --2- 172.21.15.100:0/3299367888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f933015e5c0 0x7f93301f6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.693+0000 7f932e800640 1 -- 172.21.15.100:0/3299367888 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93301f5550 con 0x7f9330149420 2024-09-17T04:29:29.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.693+0000 7f932e800640 1 --2- 172.21.15.100:0/3299367888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9330149420 0x7f933014b470 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f932800c6d0 tx=0x7f9328002110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:29.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.693+0000 7f9326a00640 1 -- 172.21.15.100:0/3299367888 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932802bc10 con 0x7f9330149420 2024-09-17T04:29:29.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.693+0000 7f9326a00640 1 -- 172.21.15.100:0/3299367888 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9328006070 con 0x7f9330149420 2024-09-17T04:29:29.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.693+0000 7f932f200640 1 -- 172.21.15.100:0/3299367888 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f933014cca0 con 0x7f9330149420 2024-09-17T04:29:29.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.693+0000 7f9326a00640 1 -- 172.21.15.100:0/3299367888 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9328006480 con 0x7f9330149420 2024-09-17T04:29:29.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.694+0000 7f932f200640 1 -- 172.21.15.100:0/3299367888 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93301f5810 con 0x7f9330149420 2024-09-17T04:29:29.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.696+0000 7f9326a00640 1 -- 172.21.15.100:0/3299367888 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f9328006620 con 0x7f9330149420 2024-09-17T04:29:29.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.697+0000 7f9326a00640 1 --2- 172.21.15.100:0/3299367888 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f930c078cb0 0x7f930c07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:29.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.697+0000 7f9326a00640 1 -- 172.21.15.100:0/3299367888 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f93280bb460 con 0x7f9330149420 2024-09-17T04:29:29.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.697+0000 7f932de00640 1 --2- 172.21.15.100:0/3299367888 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f930c078cb0 0x7f930c07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:29.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.698+0000 7f933ba3a740 1 Processor -- start 2024-09-17T04:29:29.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.698+0000 7f933ba3a740 1 -- start start 2024-09-17T04:29:29.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.698+0000 7f932de00640 1 --2- 172.21.15.100:0/3299367888 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f930c078cb0 0x7f930c07b1c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9320009c10 tx=0x7f9320005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:29.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.699+0000 7f933ba3a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e671051270 0x55e671421b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:29.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.699+0000 7f933ba3a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6710283b0 0x55e6714220a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:29.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.699+0000 7f933ba3a740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e671030590 0x55e6713f2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:29.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.699+0000 7f933ba3a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55e67137b430 con 0x55e6710283b0 2024-09-17T04:29:29.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.699+0000 7f933ba3a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e67136b5d0 con 0x55e671051270 2024-09-17T04:29:29.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.699+0000 7f933ba3a740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e67135aa00 con 0x55e671030590 2024-09-17T04:29:29.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.699+0000 7f9325400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6710283b0 0x55e6714220a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:29.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.700+0000 7f9325400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6710283b0 0x55e6714220a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33110/0 (socket says 172.21.15.100:33110) 2024-09-17T04:29:29.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.700+0000 7f9325400640 1 -- 172.21.15.100:0/186396211 learned_addr learned my addr 172.21.15.100:0/186396211 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:29.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.700+0000 7f9325400640 1 -- 172.21.15.100:0/186396211 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e671030590 msgr2=0x55e6713f2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:29:29.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.700+0000 7f9325400640 1 --2- 172.21.15.100:0/186396211 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e671030590 0x55e6713f2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.700+0000 7f9325400640 1 -- 172.21.15.100:0/186396211 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e671051270 msgr2=0x55e671421b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:29:29.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.700+0000 7f932de00640 1 --2- 172.21.15.100:0/186396211 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e671051270 0x55e671421b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:29.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.700+0000 7f9325400640 1 --2- 172.21.15.100:0/186396211 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e671051270 0x55e671421b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.700+0000 7f9325400640 1 -- 172.21.15.100:0/186396211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e671323520 con 0x55e6710283b0 2024-09-17T04:29:29.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.700+0000 7f932de00640 1 --2- 172.21.15.100:0/186396211 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e671051270 0x55e671421b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:29:29.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.700+0000 7f9325400640 1 --2- 172.21.15.100:0/186396211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6710283b0 0x55e6714220a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f9318009b70 tx=0x7f9318012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6734bbbdd1832bf server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:29.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.700+0000 7f931e600640 1 -- 172.21.15.100:0/186396211 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318009d20 con 0x55e6710283b0 2024-09-17T04:29:29.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.700+0000 7f931e600640 1 -- 172.21.15.100:0/186396211 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9318012110 con 0x55e6710283b0 2024-09-17T04:29:29.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.700+0000 7f931e600640 1 -- 172.21.15.100:0/186396211 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318004270 con 0x55e6710283b0 2024-09-17T04:29:29.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.701+0000 7f933ba3a740 1 -- 172.21.15.100:0/186396211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6710283b0 msgr2=0x55e6714220a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:29.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.701+0000 7f933ba3a740 1 --2- 172.21.15.100:0/186396211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6710283b0 0x55e6714220a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f9318009b70 tx=0x7f9318012820 comp rx=0 tx=0).stop 2024-09-17T04:29:29.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.701+0000 7f933ba3a740 1 -- 172.21.15.100:0/186396211 shutdown_connections 2024-09-17T04:29:29.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.701+0000 7f933ba3a740 1 --2- 172.21.15.100:0/186396211 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e671030590 0x55e6713f2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.701+0000 7f933ba3a740 1 --2- 172.21.15.100:0/186396211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6710283b0 0x55e6714220a0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.701+0000 7f933ba3a740 1 --2- 172.21.15.100:0/186396211 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e671051270 0x55e671421b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.701+0000 7f933ba3a740 1 -- 172.21.15.100:0/186396211 >> 172.21.15.100:0/186396211 conn(0x55e6710453c0 msgr2=0x55e6713f7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:29.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.701+0000 7f933ba3a740 1 -- 172.21.15.100:0/186396211 shutdown_connections 2024-09-17T04:29:29.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.702+0000 7f933ba3a740 1 -- 172.21.15.100:0/186396211 wait complete. 2024-09-17T04:29:29.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.702+0000 7f933ba3a740 1 -- start start 2024-09-17T04:29:29.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.703+0000 7f933ba3a740 1 Processor -- start 2024-09-17T04:29:29.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.704+0000 7f933ba3a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6710283b0 0x55e6714184e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:29.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.704+0000 7f933ba3a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e671030590 0x55e67144c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:29.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.704+0000 7f933ba3a740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e671051270 0x55e67144f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:29.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.704+0000 7f933ba3a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55e671301330 con 0x55e6710283b0 2024-09-17T04:29:29.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.704+0000 7f933ba3a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e67135e810 con 0x55e671030590 2024-09-17T04:29:29.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.704+0000 7f933ba3a740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e6710cb620 con 0x55e671051270 2024-09-17T04:29:29.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.704+0000 7f933ba3a740 10 client.0 _ll_register_callbacks cb 0x55e671306420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:29:29.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.704+0000 7f932de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6710283b0 0x55e6714184e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:29.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.704+0000 7f932de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6710283b0 0x55e6714184e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33124/0 (socket says 172.21.15.100:33124) 2024-09-17T04:29:29.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.704+0000 7f932de00640 1 -- 172.21.15.100:0/3296293907 learned_addr learned my addr 172.21.15.100:0/3296293907 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:29.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.705+0000 7f9325400640 1 --2- 172.21.15.100:0/3296293907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e671030590 0x55e67144c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:29.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.705+0000 7f932e800640 1 --2- 172.21.15.100:0/3296293907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e671051270 0x55e67144f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:29.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.705+0000 7f932de00640 1 -- 172.21.15.100:0/3296293907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e671051270 msgr2=0x55e67144f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:29:29.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.705+0000 7f932de00640 1 --2- 172.21.15.100:0/3296293907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e671051270 0x55e67144f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.705+0000 7f932de00640 1 -- 172.21.15.100:0/3296293907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e671030590 msgr2=0x55e67144c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:29:29.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.705+0000 7f932de00640 1 --2- 172.21.15.100:0/3296293907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e671030590 0x55e67144c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.705+0000 7f932de00640 1 -- 172.21.15.100:0/3296293907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6710bc850 con 0x55e6710283b0 2024-09-17T04:29:29.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.705+0000 7f9325400640 1 --2- 172.21.15.100:0/3296293907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e671030590 0x55e67144c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:29.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.705+0000 7f932de00640 1 --2- 172.21.15.100:0/3296293907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6710283b0 0x55e6714184e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f9320009de0 tx=0x7f9320004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3b68472f8c3df9 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:29.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.705+0000 7f9312a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-17T04:29:29.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.705+0000 7f9312a00640 1 -- 172.21.15.100:0/3296293907 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320003d00 con 0x55e6710283b0 2024-09-17T04:29:29.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.705+0000 7f9312a00640 1 -- 172.21.15.100:0/3296293907 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f9320018030 con 0x55e6710283b0 2024-09-17T04:29:29.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.705+0000 7f933ba3a740 1 -- 172.21.15.100:0/3296293907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9318012110 con 0x55e6710283b0 2024-09-17T04:29:29.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.706+0000 7f9312a00640 1 -- 172.21.15.100:0/3296293907 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f932001d410 con 0x55e6710283b0 2024-09-17T04:29:29.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.706+0000 7f933ba3a740 10 client.6723 fetch_fsmap learned FSMap version 57 2024-09-17T04:29:29.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.706+0000 7f933ba3a740 1 -- 172.21.15.100:0/3296293907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x55e6711336b0 con 0x55e6710283b0 2024-09-17T04:29:29.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.706+0000 7f9312a00640 1 -- 172.21.15.100:0/3296293907 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9318012110 con 0x55e6710283b0 2024-09-17T04:29:29.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.706+0000 7f9312a00640 1 -- 172.21.15.100:0/3296293907 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 57) ==== 3502+0+0 (secure 0 0 0) 0x7f9320031050 con 0x55e6710283b0 2024-09-17T04:29:29.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.706+0000 7f933ba3a740 10 client.6723 fetch_fsmap finished waiting for FSMap version 57 2024-09-17T04:29:29.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.706+0000 7f933ba3a740 10 client.6723 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T04:29:29.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.706+0000 7f933ba3a740 20 client.6723 populate_metadata read hostname 'smithi100' 2024-09-17T04:29:29.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.706+0000 7f933ba3a740 1 --2- 172.21.15.100:0/3296293907 >> [v2:172.21.15.100:6834/2660333125,v1:172.21.15.100:6835/2660333125] conn(0x55e67100fe40 0x55e671419f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:29.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.706+0000 7f933ba3a740 4 client.6723 mds_command: new command op to 6606 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T04:29:29.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.706+0000 7f933ba3a740 1 -- 172.21.15.100:0/3296293907 --> [v2:172.21.15.100:6834/2660333125,v1:172.21.15.100:6835/2660333125] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e67136d520 con 0x55e67100fe40 2024-09-17T04:29:29.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.706+0000 7f9325400640 1 --2- 172.21.15.100:0/3296293907 >> [v2:172.21.15.100:6834/2660333125,v1:172.21.15.100:6835/2660333125] conn(0x55e67100fe40 0x55e671419f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:29.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.707+0000 7f9312a00640 10 client.6723 ms_handle_connect on v2:172.21.15.100:6834/2660333125 2024-09-17T04:29:29.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.707+0000 7f9325400640 1 --2- 172.21.15.100:0/3296293907 >> [v2:172.21.15.100:6834/2660333125,v1:172.21.15.100:6835/2660333125] conn(0x55e67100fe40 0x55e671419f90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db2f49c90e70a06b server_cookie=d0cb5e11ebefcddd in_seq=0 out_seq=0 2024-09-17T04:29:29.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.708+0000 7f9312a00640 1 -- 172.21.15.100:0/3296293907 <== mds.0 v2:172.21.15.100:6834/2660333125 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e67136d520 con 0x55e67100fe40 2024-09-17T04:29:29.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.708+0000 7f9312a00640 10 client.6723 handle_command_reply: tid=0 2024-09-17T04:29:29.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.708+0000 7f933ba3a740 1 client.6723 shutdown 2024-09-17T04:29:29.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.708+0000 7f933ba3a740 1 -- 172.21.15.100:0/3296293907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6710283b0 msgr2=0x55e6714184e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:29.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.708+0000 7f933ba3a740 1 --2- 172.21.15.100:0/3296293907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6710283b0 0x55e6714184e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f9320009de0 tx=0x7f9320004730 comp rx=0 tx=0).stop 2024-09-17T04:29:29.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.708+0000 7f933ba3a740 1 -- 172.21.15.100:0/3296293907 shutdown_connections 2024-09-17T04:29:29.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.708+0000 7f933ba3a740 1 --2- 172.21.15.100:0/3296293907 >> [v2:172.21.15.100:6834/2660333125,v1:172.21.15.100:6835/2660333125] conn(0x55e67100fe40 0x55e671419f90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.708+0000 7f933ba3a740 1 --2- 172.21.15.100:0/3296293907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e671051270 0x55e67144f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.708+0000 7f933ba3a740 1 --2- 172.21.15.100:0/3296293907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e671030590 0x55e67144c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.708+0000 7f933ba3a740 1 --2- 172.21.15.100:0/3296293907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6710283b0 0x55e6714184e0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.708+0000 7f9312a00640 0 client.6723 ms_handle_reset on v2:172.21.15.100:6834/2660333125 2024-09-17T04:29:29.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.708+0000 7f933ba3a740 1 -- 172.21.15.100:0/3296293907 >> 172.21.15.100:0/3296293907 conn(0x55e6710453c0 msgr2=0x55e67143fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:29.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.708+0000 7f933ba3a740 1 -- 172.21.15.100:0/3296293907 shutdown_connections 2024-09-17T04:29:29.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.708+0000 7f933ba3a740 1 -- 172.21.15.100:0/3296293907 wait complete. 2024-09-17T04:29:29.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.708+0000 7f933ba3a740 20 client.6723 trim_cache size 0 max 16384 2024-09-17T04:29:29.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.727+0000 7f933ba3a740 1 Processor -- start 2024-09-17T04:29:29.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.727+0000 7f933ba3a740 1 -- start start 2024-09-17T04:29:29.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.728+0000 7f933ba3a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6710283b0 0x55e67143d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:29.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.728+0000 7f933ba3a740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e671030590 0x55e6714400e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:29.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.729+0000 7f933ba3a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e671051270 0x55e671455940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:29.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.729+0000 7f933ba3a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f93302012e0 con 0x55e6710283b0 2024-09-17T04:29:29.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.729+0000 7f933ba3a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f93301f9400 con 0x55e671051270 2024-09-17T04:29:29.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.729+0000 7f933ba3a740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9330200eb0 con 0x55e671030590 2024-09-17T04:29:29.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.729+0000 7f932de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6710283b0 0x55e67143d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:29.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.729+0000 7f932de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6710283b0 0x55e67143d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33136/0 (socket says 172.21.15.100:33136) 2024-09-17T04:29:29.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.729+0000 7f932de00640 1 -- 172.21.15.100:0/819144607 learned_addr learned my addr 172.21.15.100:0/819144607 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:29.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.729+0000 7f932e800640 1 --2- 172.21.15.100:0/819144607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e671051270 0x55e671455940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:29.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.729+0000 7f9325400640 1 --2- 172.21.15.100:0/819144607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e671030590 0x55e6714400e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:29.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.729+0000 7f932de00640 1 -- 172.21.15.100:0/819144607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e671030590 msgr2=0x55e6714400e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:29:29.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.729+0000 7f932de00640 1 --2- 172.21.15.100:0/819144607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e671030590 0x55e6714400e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.729+0000 7f932de00640 1 -- 172.21.15.100:0/819144607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e671051270 msgr2=0x55e671455940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:29:29.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.729+0000 7f932de00640 1 --2- 172.21.15.100:0/819144607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e671051270 0x55e671455940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.729+0000 7f932de00640 1 -- 172.21.15.100:0/819144607 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e67144f750 con 0x55e6710283b0 2024-09-17T04:29:29.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.729+0000 7f932e800640 1 --2- 172.21.15.100:0/819144607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e671051270 0x55e671455940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:29:29.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.729+0000 7f9325400640 1 --2- 172.21.15.100:0/819144607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e671030590 0x55e6714400e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:29:29.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.729+0000 7f932de00640 1 --2- 172.21.15.100:0/819144607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6710283b0 0x55e67143d290 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f9320004880 tx=0x7f93200048b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99694b3039ebf66d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:29.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.730+0000 7f931dc00640 1 -- 172.21.15.100:0/819144607 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932001d5b0 con 0x55e6710283b0 2024-09-17T04:29:29.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.730+0000 7f931dc00640 1 -- 172.21.15.100:0/819144607 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f932001d750 con 0x55e6710283b0 2024-09-17T04:29:29.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.730+0000 7f931dc00640 1 -- 172.21.15.100:0/819144607 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932001d980 con 0x55e6710283b0 2024-09-17T04:29:29.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.730+0000 7f933ba3a740 1 -- 172.21.15.100:0/819144607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6710283b0 msgr2=0x55e67143d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:29.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.730+0000 7f933ba3a740 1 --2- 172.21.15.100:0/819144607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6710283b0 0x55e67143d290 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f9320004880 tx=0x7f93200048b0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.730+0000 7f933ba3a740 1 -- 172.21.15.100:0/819144607 shutdown_connections 2024-09-17T04:29:29.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.730+0000 7f933ba3a740 1 --2- 172.21.15.100:0/819144607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e671051270 0x55e671455940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.730+0000 7f933ba3a740 1 --2- 172.21.15.100:0/819144607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e671030590 0x55e6714400e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.730+0000 7f933ba3a740 1 --2- 172.21.15.100:0/819144607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6710283b0 0x55e67143d290 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.730+0000 7f933ba3a740 1 -- 172.21.15.100:0/819144607 >> 172.21.15.100:0/819144607 conn(0x55e6710453c0 msgr2=0x55e6713ed6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:29.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.730+0000 7f933ba3a740 1 -- 172.21.15.100:0/819144607 shutdown_connections 2024-09-17T04:29:29.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.731+0000 7f933ba3a740 1 -- 172.21.15.100:0/819144607 wait complete. 2024-09-17T04:29:29.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.731+0000 7f933ba3a740 1 -- start start 2024-09-17T04:29:29.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.732+0000 7f933ba3a740 1 Processor -- start 2024-09-17T04:29:29.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.733+0000 7f933ba3a740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e6710283b0 0x55e6714bceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:29.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.733+0000 7f933ba3a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e671030590 0x55e67149ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:29.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.733+0000 7f933ba3a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e671051270 0x55e6714400e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:29.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.733+0000 7f933ba3a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55e67141b670 con 0x55e671051270 2024-09-17T04:29:29.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.733+0000 7f933ba3a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e67144c140 con 0x55e671030590 2024-09-17T04:29:29.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.733+0000 7f933ba3a740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55e67144f920 con 0x55e6710283b0 2024-09-17T04:29:29.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.733+0000 7f933ba3a740 10 client.0 _ll_register_callbacks cb 0x7f93200043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:29:29.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.733+0000 7f932e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e671051270 0x55e6714400e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:29.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.733+0000 7f932e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e671051270 0x55e6714400e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33144/0 (socket says 172.21.15.100:33144) 2024-09-17T04:29:29.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.733+0000 7f932e800640 1 -- 172.21.15.100:0/2401067892 learned_addr learned my addr 172.21.15.100:0/2401067892 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:29.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.733+0000 7f9325400640 1 --2- 172.21.15.100:0/2401067892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e671030590 0x55e67149ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:29.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.734+0000 7f932de00640 1 --2- 172.21.15.100:0/2401067892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e6710283b0 0x55e6714bceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:29.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.734+0000 7f932e800640 1 -- 172.21.15.100:0/2401067892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e6710283b0 msgr2=0x55e6714bceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:29:29.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.734+0000 7f932e800640 1 --2- 172.21.15.100:0/2401067892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e6710283b0 0x55e6714bceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.734+0000 7f932e800640 1 -- 172.21.15.100:0/2401067892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e671030590 msgr2=0x55e67149ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:29:29.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.734+0000 7f932e800640 1 --2- 172.21.15.100:0/2401067892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e671030590 0x55e67149ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.734+0000 7f932e800640 1 -- 172.21.15.100:0/2401067892 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6713f3ee0 con 0x55e671051270 2024-09-17T04:29:29.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.734+0000 7f932e800640 1 --2- 172.21.15.100:0/2401067892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e671051270 0x55e6714400e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f9328004bb0 tx=0x7f9328082780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99d81d280f9ae0e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:29.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.734+0000 7f9312a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-17T04:29:29.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.734+0000 7f9312a00640 1 -- 172.21.15.100:0/2401067892 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9328045030 con 0x55e671051270 2024-09-17T04:29:29.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.734+0000 7f9312a00640 1 -- 172.21.15.100:0/2401067892 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f932807f0c0 con 0x55e671051270 2024-09-17T04:29:29.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.734+0000 7f933ba3a740 1 -- 172.21.15.100:0/2401067892 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f932001d750 con 0x55e671051270 2024-09-17T04:29:29.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.734+0000 7f9312a00640 1 -- 172.21.15.100:0/2401067892 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f9328082ee0 con 0x55e671051270 2024-09-17T04:29:29.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.734+0000 7f933ba3a740 10 client.6729 fetch_fsmap learned FSMap version 57 2024-09-17T04:29:29.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.735+0000 7f933ba3a740 1 -- 172.21.15.100:0/2401067892 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x55e6714a2440 con 0x55e671051270 2024-09-17T04:29:29.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.735+0000 7f9312a00640 1 -- 172.21.15.100:0/2401067892 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f932001d750 con 0x55e671051270 2024-09-17T04:29:29.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.735+0000 7f9312a00640 1 -- 172.21.15.100:0/2401067892 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 57) ==== 3502+0+0 (secure 0 0 0) 0x7f9328048060 con 0x55e671051270 2024-09-17T04:29:29.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.735+0000 7f933ba3a740 10 client.6729 fetch_fsmap finished waiting for FSMap version 57 2024-09-17T04:29:29.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.735+0000 7f933ba3a740 10 client.6729 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T04:29:29.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.735+0000 7f933ba3a740 20 client.6729 populate_metadata read hostname 'smithi100' 2024-09-17T04:29:29.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.735+0000 7f933ba3a740 1 --2- 172.21.15.100:0/2401067892 >> [v2:172.21.15.100:6834/2660333125,v1:172.21.15.100:6835/2660333125] conn(0x55e6714bfd10 0x55e6714c0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:29.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.735+0000 7f933ba3a740 4 client.6729 mds_command: new command op to 6606 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T04:29:29.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.735+0000 7f933ba3a740 1 -- 172.21.15.100:0/2401067892 --> [v2:172.21.15.100:6834/2660333125,v1:172.21.15.100:6835/2660333125] -- command(tid 0: {"prefix": "status"}) -- 0x55e671368fd0 con 0x55e6714bfd10 2024-09-17T04:29:29.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.735+0000 7f932de00640 1 --2- 172.21.15.100:0/2401067892 >> [v2:172.21.15.100:6834/2660333125,v1:172.21.15.100:6835/2660333125] conn(0x55e6714bfd10 0x55e6714c0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:29.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.736+0000 7f932de00640 1 --2- 172.21.15.100:0/2401067892 >> [v2:172.21.15.100:6834/2660333125,v1:172.21.15.100:6835/2660333125] conn(0x55e6714bfd10 0x55e6714c0120 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fc6b277bcc41578 server_cookie=5fcf58c24c5c9899 in_seq=0 out_seq=0 2024-09-17T04:29:29.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.736+0000 7f9312a00640 10 client.6729 ms_handle_connect on v2:172.21.15.100:6834/2660333125 2024-09-17T04:29:29.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.736+0000 7f9312a00640 1 -- 172.21.15.100:0/2401067892 <== mds.0 v2:172.21.15.100:6834/2660333125 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55e671368fd0 con 0x55e6714bfd10 2024-09-17T04:29:29.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.736+0000 7f9312a00640 10 client.6729 handle_command_reply: tid=0 2024-09-17T04:29:29.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.736+0000 7f933ba3a740 1 client.6729 shutdown 2024-09-17T04:29:29.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.736+0000 7f933ba3a740 1 -- 172.21.15.100:0/2401067892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e671051270 msgr2=0x55e6714400e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:29.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.736+0000 7f933ba3a740 1 --2- 172.21.15.100:0/2401067892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e671051270 0x55e6714400e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f9328004bb0 tx=0x7f9328082780 comp rx=0 tx=0).stop 2024-09-17T04:29:29.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.736+0000 7f933ba3a740 1 -- 172.21.15.100:0/2401067892 shutdown_connections 2024-09-17T04:29:29.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.736+0000 7f933ba3a740 1 --2- 172.21.15.100:0/2401067892 >> [v2:172.21.15.100:6834/2660333125,v1:172.21.15.100:6835/2660333125] conn(0x55e6714bfd10 0x55e6714c0120 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.736+0000 7f933ba3a740 1 --2- 172.21.15.100:0/2401067892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e671051270 0x55e6714400e0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.736+0000 7f933ba3a740 1 --2- 172.21.15.100:0/2401067892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e671030590 0x55e67149ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.736+0000 7f933ba3a740 1 --2- 172.21.15.100:0/2401067892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e6710283b0 0x55e6714bceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.736+0000 7f9312a00640 0 client.6729 ms_handle_reset on v2:172.21.15.100:6834/2660333125 2024-09-17T04:29:29.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.736+0000 7f933ba3a740 1 -- 172.21.15.100:0/2401067892 >> 172.21.15.100:0/2401067892 conn(0x55e6710453c0 msgr2=0x55e67143cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:29.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.736+0000 7f933ba3a740 1 -- 172.21.15.100:0/2401067892 shutdown_connections 2024-09-17T04:29:29.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.736+0000 7f933ba3a740 1 -- 172.21.15.100:0/2401067892 wait complete. 2024-09-17T04:29:29.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.736+0000 7f933ba3a740 20 client.6729 trim_cache size 0 max 16384 2024-09-17T04:29:29.740 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:29:29.741 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "8a399842-3a5c-4ea2-9afb-b2704332e59f", 2024-09-17T04:29:29.741 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 0, 2024-09-17T04:29:29.741 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 6606, 2024-09-17T04:29:29.741 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-09-17T04:29:29.742 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-09-17T04:29:29.742 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-09-17T04:29:29.742 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 6.6476973639999999, 2024-09-17T04:29:29.742 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 57, 2024-09-17T04:29:29.742 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 87, 2024-09-17T04:29:29.742 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 87, 2024-09-17T04:29:29.742 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 11.655692397999999 2024-09-17T04:29:29.742 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:29:29.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.737+0000 7f932f200640 1 -- 172.21.15.100:0/3299367888 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f930c078cb0 msgr2=0x7f930c07b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:29.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.737+0000 7f932f200640 1 --2- 172.21.15.100:0/3299367888 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f930c078cb0 0x7f930c07b1c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9320009c10 tx=0x7f9320005f90 comp rx=0 tx=0).stop 2024-09-17T04:29:29.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.737+0000 7f932f200640 1 -- 172.21.15.100:0/3299367888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9330149420 msgr2=0x7f933014b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:29.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.737+0000 7f932f200640 1 --2- 172.21.15.100:0/3299367888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9330149420 0x7f933014b470 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f932800c6d0 tx=0x7f9328002110 comp rx=0 tx=0).stop 2024-09-17T04:29:29.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.737+0000 7f932f200640 1 -- 172.21.15.100:0/3299367888 shutdown_connections 2024-09-17T04:29:29.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.737+0000 7f932f200640 1 --2- 172.21.15.100:0/3299367888 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f930c078cb0 0x7f930c07b1c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.737+0000 7f932f200640 1 --2- 172.21.15.100:0/3299367888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9330149420 0x7f933014b470 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.737+0000 7f932f200640 1 --2- 172.21.15.100:0/3299367888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f933015e5c0 0x7f93301f6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.737+0000 7f932f200640 1 --2- 172.21.15.100:0/3299367888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9314004d40 0x7f933014d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.737+0000 7f932f200640 1 -- 172.21.15.100:0/3299367888 >> 172.21.15.100:0/3299367888 conn(0x7f93300d1470 msgr2=0x7f9330151290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:29.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.737+0000 7f932f200640 1 -- 172.21.15.100:0/3299367888 shutdown_connections 2024-09-17T04:29:29.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.737+0000 7f932f200640 1 -- 172.21.15.100:0/3299367888 wait complete. 2024-09-17T04:29:29.758 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:29:29.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.978+0000 7f2c1c600640 1 -- 172.21.15.100:0/2559252200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c140af630 msgr2=0x7f2c14073500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:29.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.978+0000 7f2c1c600640 1 --2- 172.21.15.100:0/2559252200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c140af630 0x7f2c14073500 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f2c14074d40 tx=0x7f2c0802a530 comp rx=0 tx=0).stop 2024-09-17T04:29:29.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.978+0000 7f2c1c600640 1 -- 172.21.15.100:0/2559252200 shutdown_connections 2024-09-17T04:29:29.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.978+0000 7f2c1c600640 1 --2- 172.21.15.100:0/2559252200 >> v2:172.21.15.184:3300/0 conn(0x7f2c1406f7b0 0x7f2c140b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.978+0000 7f2c1c600640 1 --2- 172.21.15.100:0/2559252200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c140af630 0x7f2c14073500 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.978+0000 7f2c1c600640 1 -- 172.21.15.100:0/2559252200 >> 172.21.15.100:0/2559252200 conn(0x7f2c1406d730 msgr2=0x7f2c140b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:29.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.978+0000 7f2c1c600640 1 -- 172.21.15.100:0/2559252200 shutdown_connections 2024-09-17T04:29:29.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.978+0000 7f2c1c600640 1 -- 172.21.15.100:0/2559252200 wait complete. 2024-09-17T04:29:29.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.981+0000 7f2c1c600640 1 Processor -- start 2024-09-17T04:29:29.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.981+0000 7f2c1c600640 1 -- start start 2024-09-17T04:29:29.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.982+0000 7f2c1c600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c1406f7b0 0x7f2c141f6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:29.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.982+0000 7f2c1c600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c14073a40 0x7f2c141f9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:29.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.982+0000 7f2c1c600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c140af630 0x7f2c141518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:29.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.982+0000 7f2c1c600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2c140b3b60 con 0x7f2c1406f7b0 2024-09-17T04:29:29.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.982+0000 7f2c1c600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2c140b39e0 con 0x7f2c14073a40 2024-09-17T04:29:29.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.982+0000 7f2c1c600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2c140b3ce0 con 0x7f2c140af630 2024-09-17T04:29:29.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.982+0000 7f2c19c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c1406f7b0 0x7f2c141f6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:29.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.983+0000 7f2c19c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c1406f7b0 0x7f2c141f6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33160/0 (socket says 172.21.15.100:33160) 2024-09-17T04:29:29.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.983+0000 7f2c19c00640 1 -- 172.21.15.100:0/3919732729 learned_addr learned my addr 172.21.15.100:0/3919732729 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:29.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.983+0000 7f2c1a600640 1 --2- 172.21.15.100:0/3919732729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c140af630 0x7f2c141518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:29.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.983+0000 7f2c19200640 1 --2- 172.21.15.100:0/3919732729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c14073a40 0x7f2c141f9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:29.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.983+0000 7f2c19c00640 1 -- 172.21.15.100:0/3919732729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c140af630 msgr2=0x7f2c141518f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:29.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.983+0000 7f2c19c00640 1 --2- 172.21.15.100:0/3919732729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c140af630 0x7f2c141518f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.983+0000 7f2c19c00640 1 -- 172.21.15.100:0/3919732729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c14073a40 msgr2=0x7f2c141f9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:29.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.983+0000 7f2c19c00640 1 --2- 172.21.15.100:0/3919732729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c14073a40 0x7f2c141f9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:29.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.983+0000 7f2c19c00640 1 -- 172.21.15.100:0/3919732729 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c141f9fc0 con 0x7f2c1406f7b0 2024-09-17T04:29:29.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.983+0000 7f2c19c00640 1 --2- 172.21.15.100:0/3919732729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c1406f7b0 0x7f2c141f6670 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f2c08009780 tx=0x7f2c08033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:29.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.983+0000 7f2c11600640 1 -- 172.21.15.100:0/3919732729 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c08031d60 con 0x7f2c1406f7b0 2024-09-17T04:29:29.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.984+0000 7f2c11600640 1 -- 172.21.15.100:0/3919732729 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c08033e10 con 0x7f2c1406f7b0 2024-09-17T04:29:29.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.984+0000 7f2c1c600640 1 -- 172.21.15.100:0/3919732729 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c141598d0 con 0x7f2c1406f7b0 2024-09-17T04:29:29.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.984+0000 7f2c11600640 1 -- 172.21.15.100:0/3919732729 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c08005260 con 0x7f2c1406f7b0 2024-09-17T04:29:29.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.984+0000 7f2c1c600640 1 -- 172.21.15.100:0/3919732729 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c141f5b90 con 0x7f2c1406f7b0 2024-09-17T04:29:29.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.985+0000 7f2c1c600640 1 -- 172.21.15.100:0/3919732729 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c140b4200 con 0x7f2c1406f7b0 2024-09-17T04:29:29.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.986+0000 7f2c11600640 1 -- 172.21.15.100:0/3919732729 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f2c08005400 con 0x7f2c1406f7b0 2024-09-17T04:29:29.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.987+0000 7f2c11600640 1 --2- 172.21.15.100:0/3919732729 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2bf4078cb0 0x7f2bf407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:29.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.987+0000 7f2c11600640 1 -- 172.21.15.100:0/3919732729 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f2c080bc1e0 con 0x7f2c1406f7b0 2024-09-17T04:29:29.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.987+0000 7f2c19200640 1 --2- 172.21.15.100:0/3919732729 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2bf4078cb0 0x7f2bf407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:29.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.988+0000 7f2c19200640 1 --2- 172.21.15.100:0/3919732729 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2bf4078cb0 0x7f2bf407b1c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2c00007fd0 tx=0x7f2c00009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:29.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:29.991+0000 7f2c11600640 1 -- 172.21.15.100:0/3919732729 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f2c141f9fc0 con 0x7f2c1406f7b0 2024-09-17T04:29:30.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:30.218+0000 7f2c1c600640 1 -- 172.21.15.100:0/3919732729 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2c1406f2f0 con 0x7f2c1406f7b0 2024-09-17T04:29:30.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:30.218+0000 7f2c11600640 1 -- 172.21.15.100:0/3919732729 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v57) ==== 53+0+73 (secure 0 0 0) 0x7f2c0803f030 con 0x7f2c1406f7b0 2024-09-17T04:29:30.219 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:29:30.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:30.220+0000 7f2c1c600640 1 -- 172.21.15.100:0/3919732729 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2bf4078cb0 msgr2=0x7f2bf407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:30.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:30.220+0000 7f2c1c600640 1 --2- 172.21.15.100:0/3919732729 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2bf4078cb0 0x7f2bf407b1c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2c00007fd0 tx=0x7f2c00009040 comp rx=0 tx=0).stop 2024-09-17T04:29:30.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:30.220+0000 7f2c1c600640 1 -- 172.21.15.100:0/3919732729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c1406f7b0 msgr2=0x7f2c141f6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:30.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:30.220+0000 7f2c1c600640 1 --2- 172.21.15.100:0/3919732729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c1406f7b0 0x7f2c141f6670 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f2c08009780 tx=0x7f2c08033c20 comp rx=0 tx=0).stop 2024-09-17T04:29:30.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:30.221+0000 7f2c1c600640 1 -- 172.21.15.100:0/3919732729 shutdown_connections 2024-09-17T04:29:30.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:30.221+0000 7f2c1c600640 1 --2- 172.21.15.100:0/3919732729 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2bf4078cb0 0x7f2bf407b1c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:30.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:30.221+0000 7f2c1c600640 1 --2- 172.21.15.100:0/3919732729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c140af630 0x7f2c141518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:30.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:30.221+0000 7f2c1c600640 1 --2- 172.21.15.100:0/3919732729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c14073a40 0x7f2c141f9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:30.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:30.221+0000 7f2c1c600640 1 --2- 172.21.15.100:0/3919732729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c1406f7b0 0x7f2c141f6670 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:30.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:30.221+0000 7f2c1c600640 1 -- 172.21.15.100:0/3919732729 >> 172.21.15.100:0/3919732729 conn(0x7f2c1406d730 msgr2=0x7f2c140aeb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:30.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:30.221+0000 7f2c1c600640 1 -- 172.21.15.100:0/3919732729 shutdown_connections 2024-09-17T04:29:30.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:30.221+0000 7f2c1c600640 1 -- 172.21.15.100:0/3919732729 wait complete. 2024-09-17T04:29:30.242 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T04:29:30.242 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:29:30.243 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-09-17T04:29:30.243 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T04:29:30.243 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T04:29:30.243 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T04:29:30.243 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T04:29:30.243 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:29:30.243 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:29:30.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:29:30.243 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-09-17T04:29:30.279 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:29:30.279 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:29:30.279 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:29:30.279 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:29:30.280 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-17T04:29:30.280 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:29:30.280 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:29:30.280 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:29:30.280 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-09-17T04:29:30.280 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:29:30.280 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:29:30.280 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-09-17T04:29:30.280 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:29:30.280 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:29:30.280 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:29:30.280 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:29:30.280 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T04:29:30.281 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42390sec preferred_lft 42390sec 2024-09-17T04:29:30.281 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link noprefixroute 2024-09-17T04:29:30.281 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:29:30.281 INFO:teuthology.orchestra.run.smithi100.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:29:30.281 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:2e:de:8e:a4:7c brd ff:ff:ff:ff:ff:ff 2024-09-17T04:29:30.281 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T04:29:30.281 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:29:30.281 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::cc78:e4ff:fe70:7fbe/64 scope link 2024-09-17T04:29:30.281 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:29:30.281 INFO:teuthology.orchestra.run.smithi100.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:29:30.281 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether ea:8c:45:9f:34:87 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:29:30.281 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::e88c:45ff:fe9f:3487/64 scope link 2024-09-17T04:29:30.281 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:29:30.281 INFO:teuthology.orchestra.run.smithi100.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:29:30.282 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:2e:de:8e:a4:7c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:29:30.282 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::c2e:deff:fe8e:a47c/64 scope link 2024-09-17T04:29:30.282 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:29:30.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:29:30.282 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:29:30.373 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-09-17T04:29:30.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:29:30.377 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-09-17T04:29:30.412 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-09-17T04:29:30.413 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:29:30.413 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T04:29:30.413 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T04:29:30.413 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:29:30.413 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:29:30.413 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-17T04:29:30.413 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T04:29:30.413 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T04:29:30.413 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T04:29:30.414 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-17T04:29:30.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:29:30.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:29:30.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:29:30.610 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-17T04:29:30.645 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T04:29:30.646 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T04:29:30.646 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:29:30.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:29:30.646 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-09-17T04:29:30.742 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:29:30.742 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T04:29:30.777 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:29:30.777 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:29:30.778 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:29:30.852 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T04:29:30.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:29:30.853 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:29:31.048 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:29:31.048 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:29:31.048 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:29:31.068 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:29:31.068 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:29:31.069 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:29:31.069 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:29:31.069 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:29:31.100 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:29:31.100 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:29:31.100 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:29:31.100 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:29:31.101 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:29:31.101 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-09-17T04:29:31.101 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-09-17T04:29:31.101 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8a399842-3a5c-4ea2-9afb-b2704332e59f" will pass to kernel 2024-09-17T04:29:31.102 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-17T04:29:31.102 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-09-17T04:29:31.102 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-17T04:29:31.102 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-09-17T04:29:31.102 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-17T04:29:31.105 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T04:29:31.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:29:31.105 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:29:31.165 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:29:31.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:29:31.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:29:31.262 INFO:teuthology.orchestra.run.smithi100.stdout:client6511 2024-09-17T04:29:31.262 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:29:31.262 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:29:31.298 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:29:31.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.556+0000 7fd673400640 1 -- 172.21.15.100:0/3951370056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd66c0ad0e0 msgr2=0x7fd66c0ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:31.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.556+0000 7fd673400640 1 --2- 172.21.15.100:0/3951370056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd66c0ad0e0 0x7fd66c0ad4f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd66c075250 tx=0x7fd65802a4d0 comp rx=0 tx=0).stop 2024-09-17T04:29:31.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.556+0000 7fd673400640 1 -- 172.21.15.100:0/3951370056 shutdown_connections 2024-09-17T04:29:31.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.556+0000 7fd673400640 1 --2- 172.21.15.100:0/3951370056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd66c160310 0x7fd66c160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:31.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.556+0000 7fd673400640 1 --2- 172.21.15.100:0/3951370056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd66c0ad0e0 0x7fd66c0ad4f0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:31.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.556+0000 7fd673400640 1 --2- 172.21.15.100:0/3951370056 >> v2:172.21.15.184:3300/0 conn(0x7fd66c15d760 0x7fd66c1535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:31.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.556+0000 7fd673400640 1 -- 172.21.15.100:0/3951370056 >> 172.21.15.100:0/3951370056 conn(0x7fd66c06d730 msgr2=0x7fd66c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:31.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.556+0000 7fd673400640 1 -- 172.21.15.100:0/3951370056 shutdown_connections 2024-09-17T04:29:31.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.556+0000 7fd673400640 1 -- 172.21.15.100:0/3951370056 wait complete. 2024-09-17T04:29:31.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.559+0000 7fd673400640 1 Processor -- start 2024-09-17T04:29:31.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.559+0000 7fd673400640 1 -- start start 2024-09-17T04:29:31.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.560+0000 7fd673400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd66c0ad0e0 0x7fd66c15a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:31.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.560+0000 7fd673400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd66c15d760 0x7fd66c1476e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:31.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.560+0000 7fd673400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd66c160310 0x7fd66c0b1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:31.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.560+0000 7fd673400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c0b4da0 con 0x7fd66c160310 2024-09-17T04:29:31.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.560+0000 7fd673400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c0b4c20 con 0x7fd66c15d760 2024-09-17T04:29:31.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.560+0000 7fd673400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd66c160c60 con 0x7fd66c0ad0e0 2024-09-17T04:29:31.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.561+0000 7fd671400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd66c160310 0x7fd66c0b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:31.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.561+0000 7fd671400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd66c160310 0x7fd66c0b1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33174/0 (socket says 172.21.15.100:33174) 2024-09-17T04:29:31.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.561+0000 7fd671400640 1 -- 172.21.15.100:0/2070234251 learned_addr learned my addr 172.21.15.100:0/2070234251 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:31.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.561+0000 7fd670a00640 1 --2- 172.21.15.100:0/2070234251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd66c0ad0e0 0x7fd66c15a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:31.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.561+0000 7fd66be00640 1 --2- 172.21.15.100:0/2070234251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd66c15d760 0x7fd66c1476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:31.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.561+0000 7fd671400640 1 -- 172.21.15.100:0/2070234251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd66c0ad0e0 msgr2=0x7fd66c15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:31.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.561+0000 7fd671400640 1 --2- 172.21.15.100:0/2070234251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd66c0ad0e0 0x7fd66c15a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:31.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.561+0000 7fd671400640 1 -- 172.21.15.100:0/2070234251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd66c15d760 msgr2=0x7fd66c1476e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:31.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.561+0000 7fd671400640 1 --2- 172.21.15.100:0/2070234251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd66c15d760 0x7fd66c1476e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:31.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.561+0000 7fd671400640 1 -- 172.21.15.100:0/2070234251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66c148020 con 0x7fd66c160310 2024-09-17T04:29:31.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.561+0000 7fd671400640 1 --2- 172.21.15.100:0/2070234251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd66c160310 0x7fd66c0b1000 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fd66400cb00 tx=0x7fd66400a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:31.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.562+0000 7fd668c00640 1 -- 172.21.15.100:0/2070234251 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66400ae30 con 0x7fd66c160310 2024-09-17T04:29:31.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.562+0000 7fd668c00640 1 -- 172.21.15.100:0/2070234251 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd66400f450 con 0x7fd66c160310 2024-09-17T04:29:31.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.562+0000 7fd668c00640 1 -- 172.21.15.100:0/2070234251 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66400f8c0 con 0x7fd66c160310 2024-09-17T04:29:31.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.562+0000 7fd673400640 1 -- 172.21.15.100:0/2070234251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd66c158940 con 0x7fd66c160310 2024-09-17T04:29:31.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.562+0000 7fd673400640 1 -- 172.21.15.100:0/2070234251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd66c155710 con 0x7fd66c160310 2024-09-17T04:29:31.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.563+0000 7fd673400640 1 -- 172.21.15.100:0/2070234251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd66c155250 con 0x7fd66c160310 2024-09-17T04:29:31.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.564+0000 7fd668c00640 1 -- 172.21.15.100:0/2070234251 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fd66400fa60 con 0x7fd66c160310 2024-09-17T04:29:31.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.565+0000 7fd668c00640 1 --2- 172.21.15.100:0/2070234251 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd64c078cb0 0x7fd64c07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:31.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.565+0000 7fd668c00640 1 -- 172.21.15.100:0/2070234251 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fd66409b950 con 0x7fd66c160310 2024-09-17T04:29:31.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.565+0000 7fd670a00640 1 --2- 172.21.15.100:0/2070234251 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd64c078cb0 0x7fd64c07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:31.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.566+0000 7fd670a00640 1 --2- 172.21.15.100:0/2070234251 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd64c078cb0 0x7fd64c07b1c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd660005fd0 tx=0x7fd66000a250 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:31.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.569+0000 7fd668c00640 1 -- 172.21.15.100:0/2070234251 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd66c155250 con 0x7fd66c160310 2024-09-17T04:29:31.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.790+0000 7fd673400640 1 -- 172.21.15.100:0/2070234251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd66c1535c0 con 0x7fd66c160310 2024-09-17T04:29:31.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.791+0000 7fd668c00640 1 -- 172.21.15.100:0/2070234251 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v57) ==== 53+0+73 (secure 0 0 0) 0x7fd664014030 con 0x7fd66c160310 2024-09-17T04:29:31.792 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:29:31.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.793+0000 7fd673400640 1 -- 172.21.15.100:0/2070234251 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd64c078cb0 msgr2=0x7fd64c07b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:31.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.793+0000 7fd673400640 1 --2- 172.21.15.100:0/2070234251 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd64c078cb0 0x7fd64c07b1c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd660005fd0 tx=0x7fd66000a250 comp rx=0 tx=0).stop 2024-09-17T04:29:31.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.793+0000 7fd673400640 1 -- 172.21.15.100:0/2070234251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd66c160310 msgr2=0x7fd66c0b1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:31.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.793+0000 7fd673400640 1 --2- 172.21.15.100:0/2070234251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd66c160310 0x7fd66c0b1000 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fd66400cb00 tx=0x7fd66400a250 comp rx=0 tx=0).stop 2024-09-17T04:29:31.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.793+0000 7fd673400640 1 -- 172.21.15.100:0/2070234251 shutdown_connections 2024-09-17T04:29:31.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.793+0000 7fd673400640 1 --2- 172.21.15.100:0/2070234251 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd64c078cb0 0x7fd64c07b1c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:31.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.793+0000 7fd673400640 1 --2- 172.21.15.100:0/2070234251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd66c160310 0x7fd66c0b1000 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:31.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.793+0000 7fd673400640 1 --2- 172.21.15.100:0/2070234251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd66c15d760 0x7fd66c1476e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:31.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.793+0000 7fd673400640 1 --2- 172.21.15.100:0/2070234251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd66c0ad0e0 0x7fd66c15a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:31.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.793+0000 7fd673400640 1 -- 172.21.15.100:0/2070234251 >> 172.21.15.100:0/2070234251 conn(0x7fd66c06d730 msgr2=0x7fd66c1529b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:31.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.793+0000 7fd673400640 1 -- 172.21.15.100:0/2070234251 shutdown_connections 2024-09-17T04:29:31.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:31.793+0000 7fd673400640 1 -- 172.21.15.100:0/2070234251 wait complete. 2024-09-17T04:29:31.814 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T04:29:31.815 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:29:31.815 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2024-09-17T04:29:31.815 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T04:29:31.815 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T04:29:31.815 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T04:29:31.816 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T04:29:31.816 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:29:31.816 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:29:31.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:29:31.816 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2024-09-17T04:29:31.857 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:29:31.857 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:29:31.857 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:29:31.857 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:29:31.857 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-09-17T04:29:31.857 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:29:31.857 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:29:31.858 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:29:31.858 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2024-09-17T04:29:31.858 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:29:31.858 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:29:31.858 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2024-09-17T04:29:31.858 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:29:31.858 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2024-09-17T04:29:31.858 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:29:31.858 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2024-09-17T04:29:31.858 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T04:29:31.858 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42372sec preferred_lft 42372sec 2024-09-17T04:29:31.858 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ec4:7aff:fe88:7b8f/64 scope link noprefixroute 2024-09-17T04:29:31.859 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:29:31.859 INFO:teuthology.orchestra.run.smithi184.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:29:31.859 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 7e:67:e5:69:96:69 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:29:31.859 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T04:29:31.859 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:29:31.859 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::3c9d:aff:fe6e:8fcb/64 scope link 2024-09-17T04:29:31.859 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:29:31.859 INFO:teuthology.orchestra.run.smithi184.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:29:31.859 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether a6:a9:1b:a9:61:ec brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T04:29:31.859 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::a4a9:1bff:fea9:61ec/64 scope link 2024-09-17T04:29:31.859 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:29:31.859 INFO:teuthology.orchestra.run.smithi184.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:29:31.859 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 7e:67:e5:69:96:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T04:29:31.860 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::7c67:e5ff:fe69:9669/64 scope link 2024-09-17T04:29:31.860 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:29:31.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:29:31.860 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:29:31.955 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-09-17T04:29:31.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:29:31.959 DEBUG:teuthology.orchestra.run.smithi184:> route 2024-09-17T04:29:31.997 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2024-09-17T04:29:31.997 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:29:31.998 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T04:29:31.998 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T04:29:31.998 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:29:31.999 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:29:31.999 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-09-17T04:29:31.999 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T04:29:31.999 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T04:29:31.999 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T04:29:32.000 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-09-17T04:29:32.116 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:29:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:29:32.189 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:29:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:29:32.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:29:32.198 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-09-17T04:29:32.234 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T04:29:32.234 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T04:29:32.234 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:29:32.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:29:32.234 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.0 up 2024-09-17T04:29:32.331 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:29:32.331 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T04:29:32.367 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:29:32.367 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:29:32.367 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T04:29:32.442 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T04:29:32.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:29:32.442 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:29:32.635 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:29:32.635 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:29:32.635 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:29:32.655 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:29:32.655 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:29:32.655 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:29:32.655 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:29:32.656 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:29:32.686 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:29:32.687 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:29:32.687 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:29:32.687 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:29:32.687 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:29:32.687 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-09-17T04:29:32.687 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-09-17T04:29:32.687 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8a399842-3a5c-4ea2-9afb-b2704332e59f" will pass to kernel 2024-09-17T04:29:32.688 INFO:teuthology.orchestra.run.smithi184.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-17T04:29:32.688 INFO:teuthology.orchestra.run.smithi184.stdout: You just mounted a file system that supports labels which does not 2024-09-17T04:29:32.688 INFO:teuthology.orchestra.run.smithi184.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-17T04:29:32.688 INFO:teuthology.orchestra.run.smithi184.stdout: applications will generate AVC messages and not be allowed access to 2024-09-17T04:29:32.688 INFO:teuthology.orchestra.run.smithi184.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-17T04:29:32.691 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T04:29:32.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:29:32.692 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T04:29:32.747 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:29:32.826 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:29:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:29:32.829 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:29:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:29:32.837 INFO:teuthology.orchestra.run.smithi184.stdout:client6517 2024-09-17T04:29:32.837 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:29:32.837 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:29:32.873 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T04:29:33.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.104+0000 7fb45cc00640 1 -- 172.21.15.100:0/307294058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4580b56e0 msgr2=0x7fb45806fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:33.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.104+0000 7fb45cc00640 1 --2- 172.21.15.100:0/307294058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4580b56e0 0x7fb45806fd60 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fb45815cd50 tx=0x7fb44402a4a0 comp rx=0 tx=0).stop 2024-09-17T04:29:33.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.105+0000 7fb45cc00640 1 -- 172.21.15.100:0/307294058 shutdown_connections 2024-09-17T04:29:33.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.105+0000 7fb45cc00640 1 --2- 172.21.15.100:0/307294058 >> v2:172.21.15.184:3300/0 conn(0x7fb4580702a0 0x7fb4580ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:33.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.105+0000 7fb45cc00640 1 --2- 172.21.15.100:0/307294058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4580b56e0 0x7fb45806fd60 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:33.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.105+0000 7fb45cc00640 1 -- 172.21.15.100:0/307294058 >> 172.21.15.100:0/307294058 conn(0x7fb4580b2220 msgr2=0x7fb4580d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:33.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.105+0000 7fb45cc00640 1 -- 172.21.15.100:0/307294058 shutdown_connections 2024-09-17T04:29:33.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.105+0000 7fb45cc00640 1 -- 172.21.15.100:0/307294058 wait complete. 2024-09-17T04:29:33.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.107+0000 7fb45cc00640 1 Processor -- start 2024-09-17T04:29:33.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.107+0000 7fb45cc00640 1 -- start start 2024-09-17T04:29:33.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.108+0000 7fb45cc00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4580702a0 0x7fb4581f6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:33.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.108+0000 7fb45cc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4580aec80 0x7fb4581f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:33.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.109+0000 7fb45cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4580b56e0 0x7fb4580c2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:33.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.109+0000 7fb45cc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb4580b4e00 con 0x7fb4580b56e0 2024-09-17T04:29:33.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.109+0000 7fb45cc00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb4580b4c80 con 0x7fb4580aec80 2024-09-17T04:29:33.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.109+0000 7fb45cc00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb4580ae5c0 con 0x7fb4580702a0 2024-09-17T04:29:33.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.109+0000 7fb457e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4580b56e0 0x7fb4580c2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:33.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.109+0000 7fb457e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4580b56e0 0x7fb4580c2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33198/0 (socket says 172.21.15.100:33198) 2024-09-17T04:29:33.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.109+0000 7fb457e00640 1 -- 172.21.15.100:0/2036037639 learned_addr learned my addr 172.21.15.100:0/2036037639 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:33.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.109+0000 7fb456a00640 1 --2- 172.21.15.100:0/2036037639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4580aec80 0x7fb4581f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:33.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.109+0000 7fb457400640 1 --2- 172.21.15.100:0/2036037639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4580702a0 0x7fb4581f6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:33.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.109+0000 7fb457e00640 1 -- 172.21.15.100:0/2036037639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4580702a0 msgr2=0x7fb4581f6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:33.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.109+0000 7fb457e00640 1 --2- 172.21.15.100:0/2036037639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4580702a0 0x7fb4581f6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:33.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.109+0000 7fb457e00640 1 -- 172.21.15.100:0/2036037639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4580aec80 msgr2=0x7fb4581f35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:33.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.109+0000 7fb457e00640 1 --2- 172.21.15.100:0/2036037639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4580aec80 0x7fb4581f35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:33.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.109+0000 7fb457e00640 1 -- 172.21.15.100:0/2036037639 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4581f4330 con 0x7fb4580b56e0 2024-09-17T04:29:33.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.110+0000 7fb457e00640 1 --2- 172.21.15.100:0/2036037639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4580b56e0 0x7fb4580c2560 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fb45000e960 tx=0x7fb45000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:33.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.110+0000 7fb44f400640 1 -- 172.21.15.100:0/2036037639 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450004d20 con 0x7fb4580b56e0 2024-09-17T04:29:33.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.110+0000 7fb44f400640 1 -- 172.21.15.100:0/2036037639 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb45000cb20 con 0x7fb4580b56e0 2024-09-17T04:29:33.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.110+0000 7fb45cc00640 1 -- 172.21.15.100:0/2036037639 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4580c3a50 con 0x7fb4580b56e0 2024-09-17T04:29:33.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.110+0000 7fb44f400640 1 -- 172.21.15.100:0/2036037639 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450005020 con 0x7fb4580b56e0 2024-09-17T04:29:33.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.110+0000 7fb45cc00640 1 -- 172.21.15.100:0/2036037639 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4581f71f0 con 0x7fb4580b56e0 2024-09-17T04:29:33.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.111+0000 7fb45cc00640 1 -- 172.21.15.100:0/2036037639 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4581f40a0 con 0x7fb4580b56e0 2024-09-17T04:29:33.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.112+0000 7fb44f400640 1 -- 172.21.15.100:0/2036037639 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fb4500051c0 con 0x7fb4580b56e0 2024-09-17T04:29:33.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.113+0000 7fb44f400640 1 --2- 172.21.15.100:0/2036037639 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb438078cb0 0x7fb43807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:33.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.113+0000 7fb44f400640 1 -- 172.21.15.100:0/2036037639 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fb45009b8a0 con 0x7fb4580b56e0 2024-09-17T04:29:33.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.113+0000 7fb457400640 1 --2- 172.21.15.100:0/2036037639 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb438078cb0 0x7fb43807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:33.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.114+0000 7fb457400640 1 --2- 172.21.15.100:0/2036037639 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb438078cb0 0x7fb43807b1c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb448007fd0 tx=0x7fb448001370 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:33.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.118+0000 7fb44f400640 1 -- 172.21.15.100:0/2036037639 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb450016030 con 0x7fb4580b56e0 2024-09-17T04:29:33.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.343+0000 7fb45cc00640 1 -- 172.21.15.100:0/2036037639 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fb4580aea00 con 0x7fb4580b56e0 2024-09-17T04:29:33.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.395+0000 7fb44f400640 1 -- 172.21.15.100:0/2036037639 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v58) ==== 136+0+0 (secure 0 0 0) 0x7fb4500634a0 con 0x7fb4580b56e0 2024-09-17T04:29:33.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.398+0000 7fb45cc00640 1 -- 172.21.15.100:0/2036037639 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb438078cb0 msgr2=0x7fb43807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:33.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.398+0000 7fb45cc00640 1 --2- 172.21.15.100:0/2036037639 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb438078cb0 0x7fb43807b1c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb448007fd0 tx=0x7fb448001370 comp rx=0 tx=0).stop 2024-09-17T04:29:33.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.398+0000 7fb45cc00640 1 -- 172.21.15.100:0/2036037639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4580b56e0 msgr2=0x7fb4580c2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:33.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.398+0000 7fb45cc00640 1 --2- 172.21.15.100:0/2036037639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4580b56e0 0x7fb4580c2560 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fb45000e960 tx=0x7fb45000ee20 comp rx=0 tx=0).stop 2024-09-17T04:29:33.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.398+0000 7fb45cc00640 1 -- 172.21.15.100:0/2036037639 shutdown_connections 2024-09-17T04:29:33.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.398+0000 7fb45cc00640 1 --2- 172.21.15.100:0/2036037639 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb438078cb0 0x7fb43807b1c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:33.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.398+0000 7fb45cc00640 1 --2- 172.21.15.100:0/2036037639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4580b56e0 0x7fb4580c2560 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:33.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.398+0000 7fb45cc00640 1 --2- 172.21.15.100:0/2036037639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4580aec80 0x7fb4581f35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:33.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.398+0000 7fb45cc00640 1 --2- 172.21.15.100:0/2036037639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4580702a0 0x7fb4581f6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:33.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.398+0000 7fb45cc00640 1 -- 172.21.15.100:0/2036037639 >> 172.21.15.100:0/2036037639 conn(0x7fb4580b2220 msgr2=0x7fb4580afa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:33.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.398+0000 7fb45cc00640 1 -- 172.21.15.100:0/2036037639 shutdown_connections 2024-09-17T04:29:33.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.398+0000 7fb45cc00640 1 -- 172.21.15.100:0/2036037639 wait complete. 2024-09-17T04:29:33.424 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T04:29:33.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.642+0000 7fbc3f000640 1 -- 172.21.15.100:0/1809615740 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc380cbfc0 msgr2=0x7fbc380af8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:33.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.642+0000 7fbc3f000640 1 --2- 172.21.15.100:0/1809615740 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc380cbfc0 0x7fbc380af8d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbc28009830 tx=0x7fbc2802a480 comp rx=0 tx=0).stop 2024-09-17T04:29:33.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.642+0000 7fbc3f000640 1 -- 172.21.15.100:0/1809615740 shutdown_connections 2024-09-17T04:29:33.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.642+0000 7fbc3f000640 1 --2- 172.21.15.100:0/1809615740 >> v2:172.21.15.184:3300/0 conn(0x7fbc3806f0d0 0x7fbc38070570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:33.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.642+0000 7fbc3f000640 1 --2- 172.21.15.100:0/1809615740 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc380cbfc0 0x7fbc380af8d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:33.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.642+0000 7fbc3f000640 1 -- 172.21.15.100:0/1809615740 >> 172.21.15.100:0/1809615740 conn(0x7fbc3806d3d0 msgr2=0x7fbc380b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:33.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.642+0000 7fbc3f000640 1 -- 172.21.15.100:0/1809615740 shutdown_connections 2024-09-17T04:29:33.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.642+0000 7fbc3f000640 1 -- 172.21.15.100:0/1809615740 wait complete. 2024-09-17T04:29:33.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.644+0000 7fbc3f000640 1 Processor -- start 2024-09-17T04:29:33.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.644+0000 7fbc3f000640 1 -- start start 2024-09-17T04:29:33.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.646+0000 7fbc3f000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc3806f0d0 0x7fbc38158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:33.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.646+0000 7fbc3f000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc38070ab0 0x7fbc38157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:33.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.646+0000 7fbc3f000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc380cbfc0 0x7fbc380c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:33.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.646+0000 7fbc3f000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbc380b3ff0 con 0x7fbc380cbfc0 2024-09-17T04:29:33.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.646+0000 7fbc3f000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbc380b3e70 con 0x7fbc38070ab0 2024-09-17T04:29:33.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.646+0000 7fbc3f000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbc3806f680 con 0x7fbc3806f0d0 2024-09-17T04:29:33.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.646+0000 7fbc3d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc380cbfc0 0x7fbc380c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:33.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.646+0000 7fbc3d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc380cbfc0 0x7fbc380c7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33210/0 (socket says 172.21.15.100:33210) 2024-09-17T04:29:33.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.646+0000 7fbc3d000640 1 -- 172.21.15.100:0/1821082526 learned_addr learned my addr 172.21.15.100:0/1821082526 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:33.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.646+0000 7fbc37e00640 1 --2- 172.21.15.100:0/1821082526 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc3806f0d0 0x7fbc38158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:33.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.646+0000 7fbc37400640 1 --2- 172.21.15.100:0/1821082526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc38070ab0 0x7fbc38157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:33.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.647+0000 7fbc3d000640 1 -- 172.21.15.100:0/1821082526 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc3806f0d0 msgr2=0x7fbc38158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:33.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.647+0000 7fbc3d000640 1 --2- 172.21.15.100:0/1821082526 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc3806f0d0 0x7fbc38158330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:33.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.647+0000 7fbc3d000640 1 -- 172.21.15.100:0/1821082526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc38070ab0 msgr2=0x7fbc38157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:33.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.647+0000 7fbc3d000640 1 --2- 172.21.15.100:0/1821082526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc38070ab0 0x7fbc38157050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:33.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.647+0000 7fbc3d000640 1 -- 172.21.15.100:0/1821082526 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc38157590 con 0x7fbc380cbfc0 2024-09-17T04:29:33.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.647+0000 7fbc3d000640 1 --2- 172.21.15.100:0/1821082526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc380cbfc0 0x7fbc380c7380 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fbc3000c730 tx=0x7fbc3000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:33.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.647+0000 7fbc2fe00640 1 -- 172.21.15.100:0/1821082526 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc30003b70 con 0x7fbc380cbfc0 2024-09-17T04:29:33.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.647+0000 7fbc2fe00640 1 -- 172.21.15.100:0/1821082526 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc30003d10 con 0x7fbc380cbfc0 2024-09-17T04:29:33.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.647+0000 7fbc3f000640 1 -- 172.21.15.100:0/1821082526 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc38158cd0 con 0x7fbc380cbfc0 2024-09-17T04:29:33.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.647+0000 7fbc2fe00640 1 -- 172.21.15.100:0/1821082526 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3000aab0 con 0x7fbc380cbfc0 2024-09-17T04:29:33.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.647+0000 7fbc3f000640 1 -- 172.21.15.100:0/1821082526 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc38097d20 con 0x7fbc380cbfc0 2024-09-17T04:29:33.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.648+0000 7fbc3f000640 1 -- 172.21.15.100:0/1821082526 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc38158a40 con 0x7fbc380cbfc0 2024-09-17T04:29:33.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.650+0000 7fbc2fe00640 1 -- 172.21.15.100:0/1821082526 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fbc3000ac50 con 0x7fbc380cbfc0 2024-09-17T04:29:33.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.651+0000 7fbc2fe00640 1 --2- 172.21.15.100:0/1821082526 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbc18078ce0 0x7fbc1807b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:33.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.651+0000 7fbc2fe00640 1 -- 172.21.15.100:0/1821082526 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fbc3009ab60 con 0x7fbc380cbfc0 2024-09-17T04:29:33.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.651+0000 7fbc37e00640 1 --2- 172.21.15.100:0/1821082526 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbc18078ce0 0x7fbc1807b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:33.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.652+0000 7fbc37e00640 1 --2- 172.21.15.100:0/1821082526 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbc18078ce0 0x7fbc1807b1f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbc28034000 tx=0x7fbc28004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:33.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.655+0000 7fbc2fe00640 1 -- 172.21.15.100:0/1821082526 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fbc38070ec0 con 0x7fbc380cbfc0 2024-09-17T04:29:33.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.861+0000 7fbc3f000640 1 -- 172.21.15.100:0/1821082526 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fbc38070570 con 0x7fbc18078ce0 2024-09-17T04:29:33.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.863+0000 7fbc2fe00640 1 -- 172.21.15.100:0/1821082526 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fbc38070570 con 0x7fbc18078ce0 2024-09-17T04:29:33.863 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-09-17T04:29:33.863 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:33.863 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephfs" 2024-09-17T04:29:33.863 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:33.863 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-09-17T04:29:33.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.865+0000 7fbc3f000640 1 -- 172.21.15.100:0/1821082526 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbc18078ce0 msgr2=0x7fbc1807b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:33.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.865+0000 7fbc3f000640 1 --2- 172.21.15.100:0/1821082526 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbc18078ce0 0x7fbc1807b1f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbc28034000 tx=0x7fbc28004730 comp rx=0 tx=0).stop 2024-09-17T04:29:33.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.865+0000 7fbc3f000640 1 -- 172.21.15.100:0/1821082526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc380cbfc0 msgr2=0x7fbc380c7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:33.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.865+0000 7fbc3f000640 1 --2- 172.21.15.100:0/1821082526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc380cbfc0 0x7fbc380c7380 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fbc3000c730 tx=0x7fbc3000cbf0 comp rx=0 tx=0).stop 2024-09-17T04:29:33.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.865+0000 7fbc3f000640 1 -- 172.21.15.100:0/1821082526 shutdown_connections 2024-09-17T04:29:33.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.865+0000 7fbc3f000640 1 --2- 172.21.15.100:0/1821082526 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbc18078ce0 0x7fbc1807b1f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:33.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.865+0000 7fbc3f000640 1 --2- 172.21.15.100:0/1821082526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc380cbfc0 0x7fbc380c7380 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:33.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.865+0000 7fbc3f000640 1 --2- 172.21.15.100:0/1821082526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc38070ab0 0x7fbc38157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:33.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.865+0000 7fbc3f000640 1 --2- 172.21.15.100:0/1821082526 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc3806f0d0 0x7fbc38158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:33.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.865+0000 7fbc3f000640 1 -- 172.21.15.100:0/1821082526 >> 172.21.15.100:0/1821082526 conn(0x7fbc3806d3d0 msgr2=0x7fbc380b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:33.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.865+0000 7fbc3f000640 1 -- 172.21.15.100:0/1821082526 shutdown_connections 2024-09-17T04:29:33.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:33.865+0000 7fbc3f000640 1 -- 172.21.15.100:0/1821082526 wait complete. 2024-09-17T04:29:33.889 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T04:29:34.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.112+0000 7fb737400640 1 -- 172.21.15.100:0/1594327653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb730145a40 msgr2=0x7fb730073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:34.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.112+0000 7fb737400640 1 --2- 172.21.15.100:0/1594327653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb730145a40 0x7fb730073540 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb71c009830 tx=0x7fb71c02a5a0 comp rx=0 tx=0).stop 2024-09-17T04:29:34.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.112+0000 7fb737400640 1 -- 172.21.15.100:0/1594327653 shutdown_connections 2024-09-17T04:29:34.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.112+0000 7fb737400640 1 --2- 172.21.15.100:0/1594327653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb730073a80 0x7fb7300cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:34.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.112+0000 7fb737400640 1 --2- 172.21.15.100:0/1594327653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb730145a40 0x7fb730073540 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:34.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.112+0000 7fb737400640 1 -- 172.21.15.100:0/1594327653 >> 172.21.15.100:0/1594327653 conn(0x7fb73006d1c0 msgr2=0x7fb73006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:34.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.112+0000 7fb737400640 1 -- 172.21.15.100:0/1594327653 shutdown_connections 2024-09-17T04:29:34.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.113+0000 7fb737400640 1 -- 172.21.15.100:0/1594327653 wait complete. 2024-09-17T04:29:34.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.115+0000 7fb737400640 1 Processor -- start 2024-09-17T04:29:34.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.115+0000 7fb737400640 1 -- start start 2024-09-17T04:29:34.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.116+0000 7fb737400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb730073a80 0x7fb73015b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:34.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.116+0000 7fb737400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb730145550 0x7fb7301e9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:34.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.116+0000 7fb737400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb730145a40 0x7fb7301e9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:34.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.116+0000 7fb737400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb7300ce140 con 0x7fb730145a40 2024-09-17T04:29:34.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.116+0000 7fb737400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb7300cdfc0 con 0x7fb730073a80 2024-09-17T04:29:34.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.116+0000 7fb737400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb7300ce2c0 con 0x7fb730145550 2024-09-17T04:29:34.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.117+0000 7fb735400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb730145a40 0x7fb7301e9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:34.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.117+0000 7fb735400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb730145a40 0x7fb7301e9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33228/0 (socket says 172.21.15.100:33228) 2024-09-17T04:29:34.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.117+0000 7fb735400640 1 -- 172.21.15.100:0/2997638499 learned_addr learned my addr 172.21.15.100:0/2997638499 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:34.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.117+0000 7fb72fe00640 1 --2- 172.21.15.100:0/2997638499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb730145550 0x7fb7301e9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:34.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.117+0000 7fb734a00640 1 --2- 172.21.15.100:0/2997638499 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb730073a80 0x7fb73015b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:34.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.117+0000 7fb735400640 1 -- 172.21.15.100:0/2997638499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb730145550 msgr2=0x7fb7301e9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:34.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.117+0000 7fb735400640 1 --2- 172.21.15.100:0/2997638499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb730145550 0x7fb7301e9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:34.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.117+0000 7fb735400640 1 -- 172.21.15.100:0/2997638499 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb730073a80 msgr2=0x7fb73015b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:34.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.117+0000 7fb735400640 1 --2- 172.21.15.100:0/2997638499 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb730073a80 0x7fb73015b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:34.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.117+0000 7fb735400640 1 -- 172.21.15.100:0/2997638499 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7301e7650 con 0x7fb730145a40 2024-09-17T04:29:34.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.117+0000 7fb72fe00640 1 --2- 172.21.15.100:0/2997638499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb730145550 0x7fb7301e9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:34.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.117+0000 7fb735400640 1 --2- 172.21.15.100:0/2997638499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb730145a40 0x7fb7301e9550 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fb72800e960 tx=0x7fb72800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:34.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.117+0000 7fb72cc00640 1 -- 172.21.15.100:0/2997638499 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728004d20 con 0x7fb730145a40 2024-09-17T04:29:34.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.118+0000 7fb72cc00640 1 -- 172.21.15.100:0/2997638499 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb72800cac0 con 0x7fb730145a40 2024-09-17T04:29:34.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.118+0000 7fb72cc00640 1 -- 172.21.15.100:0/2997638499 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728005020 con 0x7fb730145a40 2024-09-17T04:29:34.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.118+0000 7fb737400640 1 -- 172.21.15.100:0/2997638499 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7301e7930 con 0x7fb730145a40 2024-09-17T04:29:34.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.118+0000 7fb737400640 1 -- 172.21.15.100:0/2997638499 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7301e8af0 con 0x7fb730145a40 2024-09-17T04:29:34.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.119+0000 7fb737400640 1 -- 172.21.15.100:0/2997638499 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7300ce770 con 0x7fb730145a40 2024-09-17T04:29:34.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.120+0000 7fb72cc00640 1 -- 172.21.15.100:0/2997638499 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fb7280051c0 con 0x7fb730145a40 2024-09-17T04:29:34.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.121+0000 7fb72cc00640 1 --2- 172.21.15.100:0/2997638499 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb710078cb0 0x7fb71007b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:34.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.121+0000 7fb72cc00640 1 -- 172.21.15.100:0/2997638499 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fb72809b760 con 0x7fb730145a40 2024-09-17T04:29:34.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.121+0000 7fb734a00640 1 --2- 172.21.15.100:0/2997638499 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb710078cb0 0x7fb71007b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:34.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.122+0000 7fb734a00640 1 --2- 172.21.15.100:0/2997638499 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb710078cb0 0x7fb71007b1c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb724007d60 tx=0x7fb724007cf0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:34.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.125+0000 7fb72cc00640 1 -- 172.21.15.100:0/2997638499 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb728016030 con 0x7fb730145a40 2024-09-17T04:29:34.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.309+0000 7fb737400640 1 -- 172.21.15.100:0/2997638499 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb73015bc10 con 0x7fb730145a40 2024-09-17T04:29:34.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.309+0000 7fb72cc00640 1 -- 172.21.15.100:0/2997638499 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) ==== 122+0+0 (secure 0 0 0) 0x7fb728063450 con 0x7fb730145a40 2024-09-17T04:29:34.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.312+0000 7fb737400640 1 -- 172.21.15.100:0/2997638499 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb710078cb0 msgr2=0x7fb71007b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:34.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.312+0000 7fb737400640 1 --2- 172.21.15.100:0/2997638499 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb710078cb0 0x7fb71007b1c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb724007d60 tx=0x7fb724007cf0 comp rx=0 tx=0).stop 2024-09-17T04:29:34.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.312+0000 7fb737400640 1 -- 172.21.15.100:0/2997638499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb730145a40 msgr2=0x7fb7301e9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:34.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.312+0000 7fb737400640 1 --2- 172.21.15.100:0/2997638499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb730145a40 0x7fb7301e9550 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fb72800e960 tx=0x7fb72800ee20 comp rx=0 tx=0).stop 2024-09-17T04:29:34.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.312+0000 7fb737400640 1 -- 172.21.15.100:0/2997638499 shutdown_connections 2024-09-17T04:29:34.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.312+0000 7fb737400640 1 --2- 172.21.15.100:0/2997638499 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb710078cb0 0x7fb71007b1c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:34.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.312+0000 7fb737400640 1 --2- 172.21.15.100:0/2997638499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb730145a40 0x7fb7301e9550 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:34.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.312+0000 7fb737400640 1 --2- 172.21.15.100:0/2997638499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb730145550 0x7fb7301e9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:34.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.312+0000 7fb737400640 1 --2- 172.21.15.100:0/2997638499 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb730073a80 0x7fb73015b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:34.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.312+0000 7fb737400640 1 -- 172.21.15.100:0/2997638499 >> 172.21.15.100:0/2997638499 conn(0x7fb73006d1c0 msgr2=0x7fb73006f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:34.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.312+0000 7fb737400640 1 -- 172.21.15.100:0/2997638499 shutdown_connections 2024-09-17T04:29:34.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.312+0000 7fb737400640 1 -- 172.21.15.100:0/2997638499 wait complete. 2024-09-17T04:29:34.334 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create 'abcd@#' 2024-09-17T04:29:34.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.557+0000 7fe808a00640 1 -- 172.21.15.100:0/3910064674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8040d0f90 msgr2=0x7fe8040b7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:34.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.557+0000 7fe808a00640 1 --2- 172.21.15.100:0/3910064674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8040d0f90 0x7fe8040b7ac0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc005fd0 tx=0x7fe7fc02a940 comp rx=0 tx=0).stop 2024-09-17T04:29:34.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.557+0000 7fe808a00640 1 -- 172.21.15.100:0/3910064674 shutdown_connections 2024-09-17T04:29:34.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.557+0000 7fe808a00640 1 --2- 172.21.15.100:0/3910064674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8040d0f90 0x7fe8040b7ac0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:34.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.557+0000 7fe808a00640 1 -- 172.21.15.100:0/3910064674 >> 172.21.15.100:0/3910064674 conn(0x7fe8040cede0 msgr2=0x7fe8040cf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:34.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.557+0000 7fe808a00640 1 -- 172.21.15.100:0/3910064674 shutdown_connections 2024-09-17T04:29:34.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.557+0000 7fe808a00640 1 -- 172.21.15.100:0/3910064674 wait complete. 2024-09-17T04:29:34.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.559+0000 7fe808a00640 1 Processor -- start 2024-09-17T04:29:34.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.559+0000 7fe808a00640 1 -- start start 2024-09-17T04:29:34.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.561+0000 7fe808a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8040b1600 0x7fe80415e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:34.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.561+0000 7fe808a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8040b76b0 0x7fe8040af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:34.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.561+0000 7fe808a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8040d0f90 0x7fe8040afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:34.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.561+0000 7fe808a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe804167f40 con 0x7fe8040b76b0 2024-09-17T04:29:34.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.561+0000 7fe808a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe80409bcb0 con 0x7fe8040d0f90 2024-09-17T04:29:34.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.561+0000 7fe808a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe8041680c0 con 0x7fe8040b1600 2024-09-17T04:29:34.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.561+0000 7fe7f9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8040b76b0 0x7fe8040af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:34.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.561+0000 7fe7f9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8040b76b0 0x7fe8040af930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33246/0 (socket says 172.21.15.100:33246) 2024-09-17T04:29:34.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.561+0000 7fe7f9e00640 1 -- 172.21.15.100:0/3295037908 learned_addr learned my addr 172.21.15.100:0/3295037908 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:34.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.561+0000 7fe802800640 1 --2- 172.21.15.100:0/3295037908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8040b1600 0x7fe80415e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:34.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.561+0000 7fe803200640 1 --2- 172.21.15.100:0/3295037908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8040d0f90 0x7fe8040afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:34.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.562+0000 7fe7f9e00640 1 -- 172.21.15.100:0/3295037908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8040b1600 msgr2=0x7fe80415e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:34.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.562+0000 7fe7f9e00640 1 --2- 172.21.15.100:0/3295037908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8040b1600 0x7fe80415e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:34.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.562+0000 7fe7f9e00640 1 -- 172.21.15.100:0/3295037908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8040d0f90 msgr2=0x7fe8040afe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:34.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.562+0000 7fe7f9e00640 1 --2- 172.21.15.100:0/3295037908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8040d0f90 0x7fe8040afe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:34.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.562+0000 7fe7f9e00640 1 -- 172.21.15.100:0/3295037908 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80415eab0 con 0x7fe8040b76b0 2024-09-17T04:29:34.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.562+0000 7fe7f9e00640 1 --2- 172.21.15.100:0/3295037908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8040b76b0 0x7fe8040af930 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00c3f0 tx=0x7fe7ec00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:34.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.562+0000 7fe7fb400640 1 -- 172.21.15.100:0/3295037908 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec017070 con 0x7fe8040b76b0 2024-09-17T04:29:34.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.562+0000 7fe7fb400640 1 -- 172.21.15.100:0/3295037908 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7ec0035f0 con 0x7fe8040b76b0 2024-09-17T04:29:34.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.562+0000 7fe808a00640 1 -- 172.21.15.100:0/3295037908 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8040b64f0 con 0x7fe8040b76b0 2024-09-17T04:29:34.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.562+0000 7fe7fb400640 1 -- 172.21.15.100:0/3295037908 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec003a40 con 0x7fe8040b76b0 2024-09-17T04:29:34.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.562+0000 7fe808a00640 1 -- 172.21.15.100:0/3295037908 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8040b6ab0 con 0x7fe8040b76b0 2024-09-17T04:29:34.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.563+0000 7fe808a00640 1 -- 172.21.15.100:0/3295037908 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe80415ecf0 con 0x7fe8040b76b0 2024-09-17T04:29:34.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.564+0000 7fe7fb400640 1 -- 172.21.15.100:0/3295037908 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fe7ec01a070 con 0x7fe8040b76b0 2024-09-17T04:29:34.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.565+0000 7fe7fb400640 1 --2- 172.21.15.100:0/3295037908 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe7e0078cb0 0x7fe7e007b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:34.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.565+0000 7fe7fb400640 1 -- 172.21.15.100:0/3295037908 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fe7ec09a210 con 0x7fe8040b76b0 2024-09-17T04:29:34.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.566+0000 7fe802800640 1 --2- 172.21.15.100:0/3295037908 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe7e0078cb0 0x7fe7e007b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:34.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.566+0000 7fe802800640 1 --2- 172.21.15.100:0/3295037908 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe7e0078cb0 0x7fe7e007b1c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4000f60 tx=0x7fe7f4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:34.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:34.570+0000 7fe7fb400640 1 -- 172.21.15.100:0/3295037908 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe7ec061d40 con 0x7fe8040b76b0 2024-09-17T04:29:34.779 INFO:teuthology.orchestra.run.smithi100.stderr:Invalid command: invalid chars #@ in abcd@# 2024-09-17T04:29:34.779 INFO:teuthology.orchestra.run.smithi100.stderr:fs volume create [] : Create a CephFS volume 2024-09-17T04:29:34.779 INFO:teuthology.orchestra.run.smithi100.stderr:Error EINVAL: invalid command 2024-09-17T04:29:34.796 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T04:29:34.797 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create testvol 2024-09-17T04:29:35.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.017+0000 7f26a5600640 1 -- 172.21.15.100:0/398275883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26a0145a40 msgr2=0x7f26a0073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.017+0000 7f26a5600640 1 --2- 172.21.15.100:0/398275883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26a0145a40 0x7f26a0073540 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f268c009830 tx=0x7f268c02a5a0 comp rx=0 tx=0).stop 2024-09-17T04:29:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.018+0000 7f26a5600640 1 -- 172.21.15.100:0/398275883 shutdown_connections 2024-09-17T04:29:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.018+0000 7f26a5600640 1 --2- 172.21.15.100:0/398275883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26a0145a40 0x7f26a0073540 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.018+0000 7f26a5600640 1 --2- 172.21.15.100:0/398275883 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26a0145550 0x7f26a006fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.018+0000 7f26a5600640 1 -- 172.21.15.100:0/398275883 >> 172.21.15.100:0/398275883 conn(0x7f26a006d1c0 msgr2=0x7f26a006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.018+0000 7f26a5600640 1 -- 172.21.15.100:0/398275883 shutdown_connections 2024-09-17T04:29:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.018+0000 7f26a5600640 1 -- 172.21.15.100:0/398275883 wait complete. 2024-09-17T04:29:35.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.020+0000 7f26a5600640 1 Processor -- start 2024-09-17T04:29:35.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.020+0000 7f26a5600640 1 -- start start 2024-09-17T04:29:35.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.021+0000 7f26a5600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26a0073a80 0x7f26a015be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:35.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.021+0000 7f26a5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26a0145550 0x7f26a00a9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:35.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.021+0000 7f26a5600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26a0145a40 0x7f26a00a9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:35.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.021+0000 7f26a5600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f26a0160a30 con 0x7f26a0145550 2024-09-17T04:29:35.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.021+0000 7f26a5600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f26a00ce940 con 0x7f26a0145a40 2024-09-17T04:29:35.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.021+0000 7f26a5600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f26a0160bb0 con 0x7f26a0073a80 2024-09-17T04:29:35.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.022+0000 7f269e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26a0145550 0x7f26a00a9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:35.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.022+0000 7f269e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26a0145550 0x7f26a00a9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54342/0 (socket says 172.21.15.100:54342) 2024-09-17T04:29:35.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.022+0000 7f269e000640 1 -- 172.21.15.100:0/2402389412 learned_addr learned my addr 172.21.15.100:0/2402389412 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:35.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.022+0000 7f269f400640 1 --2- 172.21.15.100:0/2402389412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26a0145a40 0x7f26a00a9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:35.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.022+0000 7f269ea00640 1 --2- 172.21.15.100:0/2402389412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26a0073a80 0x7f26a015be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:35.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.022+0000 7f269e000640 1 -- 172.21.15.100:0/2402389412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26a0073a80 msgr2=0x7f26a015be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:35.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.022+0000 7f269e000640 1 --2- 172.21.15.100:0/2402389412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26a0073a80 0x7f26a015be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:35.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.022+0000 7f269e000640 1 -- 172.21.15.100:0/2402389412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26a0145a40 msgr2=0x7f26a00a9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:35.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.022+0000 7f269e000640 1 --2- 172.21.15.100:0/2402389412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26a0145a40 0x7f26a00a9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:35.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.022+0000 7f269e000640 1 -- 172.21.15.100:0/2402389412 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a01f55f0 con 0x7f26a0145550 2024-09-17T04:29:35.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.022+0000 7f269e000640 1 --2- 172.21.15.100:0/2402389412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26a0145550 0x7f26a00a9aa0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f268c009950 tx=0x7f268c033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:35.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.023+0000 7f2696a00640 1 -- 172.21.15.100:0/2402389412 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c005c90 con 0x7f26a0145550 2024-09-17T04:29:35.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.023+0000 7f2696a00640 1 -- 172.21.15.100:0/2402389412 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f268c043070 con 0x7f26a0145550 2024-09-17T04:29:35.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.023+0000 7f2696a00640 1 -- 172.21.15.100:0/2402389412 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c033330 con 0x7f26a0145550 2024-09-17T04:29:35.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.023+0000 7f26a5600640 1 -- 172.21.15.100:0/2402389412 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a01f58d0 con 0x7f26a0145550 2024-09-17T04:29:35.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.023+0000 7f26a5600640 1 -- 172.21.15.100:0/2402389412 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a01f7430 con 0x7f26a0145550 2024-09-17T04:29:35.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.024+0000 7f26a5600640 1 -- 172.21.15.100:0/2402389412 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26a00aa5d0 con 0x7f26a0145550 2024-09-17T04:29:35.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.025+0000 7f2696a00640 1 -- 172.21.15.100:0/2402389412 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f268c0334d0 con 0x7f26a0145550 2024-09-17T04:29:35.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.026+0000 7f2696a00640 1 --2- 172.21.15.100:0/2402389412 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2680078ce0 0x7f268007b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:35.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.026+0000 7f2696a00640 1 -- 172.21.15.100:0/2402389412 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f268c0bbfa0 con 0x7f26a0145550 2024-09-17T04:29:35.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.026+0000 7f269ea00640 1 --2- 172.21.15.100:0/2402389412 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2680078ce0 0x7f268007b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:35.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.027+0000 7f269ea00640 1 --2- 172.21.15.100:0/2402389412 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2680078ce0 0x7f268007b1f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2690005fd0 tx=0x7f2690005850 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:35.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.031+0000 7f2696a00640 1 -- 172.21.15.100:0/2402389412 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f268c084120 con 0x7f26a0145550 2024-09-17T04:29:35.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:35.235+0000 7f26a5600640 1 -- 172.21.15.100:0/2402389412 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "testvol", "target": ["mon-mgr", ""]}) -- 0x7f26a01f5380 con 0x7f2680078ce0 2024-09-17T04:29:35.554 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:29:35.559 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:29:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.447+0000 7f2696a00640 1 -- 172.21.15.100:0/2402389412 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) ==== 60+0+0 (secure 0 0 0) 0x7f26a01f5380 con 0x7f2680078ce0 2024-09-17T04:29:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:Volume created successfully (no MDS daemons created) 2024-09-17T04:29:37.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.450+0000 7f26a5600640 1 -- 172.21.15.100:0/2402389412 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2680078ce0 msgr2=0x7f268007b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:37.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.450+0000 7f26a5600640 1 --2- 172.21.15.100:0/2402389412 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2680078ce0 0x7f268007b1f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2690005fd0 tx=0x7f2690005850 comp rx=0 tx=0).stop 2024-09-17T04:29:37.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.451+0000 7f26a5600640 1 -- 172.21.15.100:0/2402389412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26a0145550 msgr2=0x7f26a00a9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:37.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.451+0000 7f26a5600640 1 --2- 172.21.15.100:0/2402389412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26a0145550 0x7f26a00a9aa0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f268c009950 tx=0x7f268c033c90 comp rx=0 tx=0).stop 2024-09-17T04:29:37.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.451+0000 7f26a5600640 1 -- 172.21.15.100:0/2402389412 shutdown_connections 2024-09-17T04:29:37.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.451+0000 7f26a5600640 1 --2- 172.21.15.100:0/2402389412 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2680078ce0 0x7f268007b1f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:37.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.451+0000 7f26a5600640 1 --2- 172.21.15.100:0/2402389412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26a0145a40 0x7f26a00a9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:37.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.451+0000 7f26a5600640 1 --2- 172.21.15.100:0/2402389412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26a0145550 0x7f26a00a9aa0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:37.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.451+0000 7f26a5600640 1 --2- 172.21.15.100:0/2402389412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26a0073a80 0x7f26a015be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:37.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.451+0000 7f26a5600640 1 -- 172.21.15.100:0/2402389412 >> 172.21.15.100:0/2402389412 conn(0x7f26a006d1c0 msgr2=0x7f26a0142bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:37.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.451+0000 7f26a5600640 1 -- 172.21.15.100:0/2402389412 shutdown_connections 2024-09-17T04:29:37.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.451+0000 7f26a5600640 1 -- 172.21.15.100:0/2402389412 wait complete. 2024-09-17T04:29:37.483 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create testvol 'abcd@#' 2024-09-17T04:29:37.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.709+0000 7f28d9c00640 1 -- 172.21.15.100:0/1080552522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d40b3f30 msgr2=0x7f28d4156ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:37.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.709+0000 7f28d9c00640 1 --2- 172.21.15.100:0/1080552522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d40b3f30 0x7f28d4156ca0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f28d40b3160 tx=0x7f28c402a8b0 comp rx=0 tx=0).stop 2024-09-17T04:29:37.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.709+0000 7f28d9c00640 1 -- 172.21.15.100:0/1080552522 shutdown_connections 2024-09-17T04:29:37.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.709+0000 7f28d9c00640 1 --2- 172.21.15.100:0/1080552522 >> v2:172.21.15.184:3300/0 conn(0x7f28d415d6e0 0x7f28d40ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:37.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.709+0000 7f28d9c00640 1 --2- 172.21.15.100:0/1080552522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d40b3f30 0x7f28d4156ca0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:37.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.709+0000 7f28d9c00640 1 -- 172.21.15.100:0/1080552522 >> 172.21.15.100:0/1080552522 conn(0x7f28d406d620 msgr2=0x7f28d40b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:37.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.709+0000 7f28d9c00640 1 -- 172.21.15.100:0/1080552522 shutdown_connections 2024-09-17T04:29:37.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.709+0000 7f28d9c00640 1 -- 172.21.15.100:0/1080552522 wait complete. 2024-09-17T04:29:37.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.712+0000 7f28d9c00640 1 Processor -- start 2024-09-17T04:29:37.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.712+0000 7f28d9c00640 1 -- start start 2024-09-17T04:29:37.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.713+0000 7f28d9c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28d40b3f30 0x7f28d41f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:37.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.713+0000 7f28d9c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28d40cd000 0x7f28d4145990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:37.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.713+0000 7f28d9c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d415d6e0 0x7f28d415c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:37.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.713+0000 7f28d9c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f28d40af960 con 0x7f28d415d6e0 2024-09-17T04:29:37.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.713+0000 7f28d9c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f28d40af7e0 con 0x7f28d40cd000 2024-09-17T04:29:37.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.713+0000 7f28d9c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f28d40afae0 con 0x7f28d40b3f30 2024-09-17T04:29:37.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.714+0000 7f28d9200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d415d6e0 0x7f28d415c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:37.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.714+0000 7f28d9200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d415d6e0 0x7f28d415c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54362/0 (socket says 172.21.15.100:54362) 2024-09-17T04:29:37.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.714+0000 7f28d9200640 1 -- 172.21.15.100:0/3442407365 learned_addr learned my addr 172.21.15.100:0/3442407365 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:37.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.714+0000 7f28d3e00640 1 --2- 172.21.15.100:0/3442407365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28d40b3f30 0x7f28d41f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:37.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.714+0000 7f28d3400640 1 --2- 172.21.15.100:0/3442407365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28d40cd000 0x7f28d4145990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:37.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.714+0000 7f28d9200640 1 -- 172.21.15.100:0/3442407365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28d40b3f30 msgr2=0x7f28d41f8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:37.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.714+0000 7f28d9200640 1 --2- 172.21.15.100:0/3442407365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28d40b3f30 0x7f28d41f8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:37.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.714+0000 7f28d9200640 1 -- 172.21.15.100:0/3442407365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28d40cd000 msgr2=0x7f28d4145990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:37.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.714+0000 7f28d9200640 1 --2- 172.21.15.100:0/3442407365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28d40cd000 0x7f28d4145990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:37.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.714+0000 7f28d9200640 1 -- 172.21.15.100:0/3442407365 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d41f94a0 con 0x7f28d415d6e0 2024-09-17T04:29:37.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.714+0000 7f28d9200640 1 --2- 172.21.15.100:0/3442407365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d415d6e0 0x7f28d415c110 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f28cc00c3c0 tx=0x7f28cc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:37.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.715+0000 7f28cbe00640 1 -- 172.21.15.100:0/3442407365 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc017070 con 0x7f28d415d6e0 2024-09-17T04:29:37.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.715+0000 7f28cbe00640 1 -- 172.21.15.100:0/3442407365 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28cc0035f0 con 0x7f28d415d6e0 2024-09-17T04:29:37.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.715+0000 7f28d9c00640 1 -- 172.21.15.100:0/3442407365 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d41f9780 con 0x7f28d415d6e0 2024-09-17T04:29:37.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.715+0000 7f28cbe00640 1 -- 172.21.15.100:0/3442407365 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc003a80 con 0x7f28d415d6e0 2024-09-17T04:29:37.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.715+0000 7f28d9c00640 1 -- 172.21.15.100:0/3442407365 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d41f9d40 con 0x7f28d415d6e0 2024-09-17T04:29:37.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.716+0000 7f28d9c00640 1 -- 172.21.15.100:0/3442407365 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28d40b10c0 con 0x7f28d415d6e0 2024-09-17T04:29:37.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.718+0000 7f28cbe00640 1 -- 172.21.15.100:0/3442407365 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f28cc01a070 con 0x7f28d415d6e0 2024-09-17T04:29:37.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.719+0000 7f28cbe00640 1 --2- 172.21.15.100:0/3442407365 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f28b4078cb0 0x7f28b407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:37.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.719+0000 7f28cbe00640 1 -- 172.21.15.100:0/3442407365 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6543+0+0 (secure 0 0 0) 0x7f28cc09a640 con 0x7f28d415d6e0 2024-09-17T04:29:37.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.719+0000 7f28d3e00640 1 --2- 172.21.15.100:0/3442407365 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f28b4078cb0 0x7f28b407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:37.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.720+0000 7f28d3e00640 1 --2- 172.21.15.100:0/3442407365 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f28b4078cb0 0x7f28b407b1c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f28c4034000 tx=0x7f28c4004ab0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:37.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:37.724+0000 7f28cbe00640 1 -- 172.21.15.100:0/3442407365 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f28d40b10c0 con 0x7f28d415d6e0 2024-09-17T04:29:37.933 INFO:teuthology.orchestra.run.smithi100.stderr:Invalid command: invalid chars @# in abcd@# 2024-09-17T04:29:37.934 INFO:teuthology.orchestra.run.smithi100.stderr:fs subvolumegroup create [] [] [] [] [] : Create a CephFS subvolume group in a volume, and optionally, with a specific data pool layout, and a specific numeric mode 2024-09-17T04:29:37.934 INFO:teuthology.orchestra.run.smithi100.stderr:Error EINVAL: invalid command 2024-09-17T04:29:37.951 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T04:29:37.952 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create testvol 'abcd@#' 2024-09-17T04:29:38.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.179+0000 7fb76e200640 1 -- 172.21.15.100:0/738004675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb768073c50 msgr2=0x7fb7681674a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:38.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.179+0000 7fb76e200640 1 --2- 172.21.15.100:0/738004675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb768073c50 0x7fb7681674a0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb760005fd0 tx=0x7fb76002a9d0 comp rx=0 tx=0).stop 2024-09-17T04:29:38.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.179+0000 7fb76e200640 1 -- 172.21.15.100:0/738004675 shutdown_connections 2024-09-17T04:29:38.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.179+0000 7fb76e200640 1 --2- 172.21.15.100:0/738004675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb768073c50 0x7fb7681674a0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:38.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.179+0000 7fb76e200640 1 -- 172.21.15.100:0/738004675 >> 172.21.15.100:0/738004675 conn(0x7fb76806d1c0 msgr2=0x7fb76806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:38.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.180+0000 7fb76e200640 1 -- 172.21.15.100:0/738004675 shutdown_connections 2024-09-17T04:29:38.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.180+0000 7fb76e200640 1 -- 172.21.15.100:0/738004675 wait complete. 2024-09-17T04:29:38.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.182+0000 7fb76e200640 1 Processor -- start 2024-09-17T04:29:38.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.182+0000 7fb76e200640 1 -- start start 2024-09-17T04:29:38.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.183+0000 7fb76e200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb768145550 0x7fb7680aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:38.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.183+0000 7fb76e200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb768145a40 0x7fb7680aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:38.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.183+0000 7fb76e200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7680b3980 0x7fb7680b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:38.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.183+0000 7fb76e200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb768167d70 con 0x7fb7680b3980 2024-09-17T04:29:38.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.183+0000 7fb76e200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb768167bf0 con 0x7fb768145a40 2024-09-17T04:29:38.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.183+0000 7fb76e200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb768167ef0 con 0x7fb768145550 2024-09-17T04:29:38.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.183+0000 7fb767e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7680b3980 0x7fb7680b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:38.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.183+0000 7fb767e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7680b3980 0x7fb7680b3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54380/0 (socket says 172.21.15.100:54380) 2024-09-17T04:29:38.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.183+0000 7fb767e00640 1 -- 172.21.15.100:0/1553930217 learned_addr learned my addr 172.21.15.100:0/1553930217 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:38.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.184+0000 7fb767400640 1 --2- 172.21.15.100:0/1553930217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb768145550 0x7fb7680aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:38.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.184+0000 7fb766a00640 1 --2- 172.21.15.100:0/1553930217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb768145a40 0x7fb7680aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:38.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.184+0000 7fb767e00640 1 -- 172.21.15.100:0/1553930217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb768145550 msgr2=0x7fb7680aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:38.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.184+0000 7fb767e00640 1 --2- 172.21.15.100:0/1553930217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb768145550 0x7fb7680aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:38.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.184+0000 7fb767e00640 1 -- 172.21.15.100:0/1553930217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb768145a40 msgr2=0x7fb7680aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:38.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.184+0000 7fb767e00640 1 --2- 172.21.15.100:0/1553930217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb768145a40 0x7fb7680aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:38.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.184+0000 7fb767e00640 1 -- 172.21.15.100:0/1553930217 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb768159540 con 0x7fb7680b3980 2024-09-17T04:29:38.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.184+0000 7fb767400640 1 --2- 172.21.15.100:0/1553930217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb768145550 0x7fb7680aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:38.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.184+0000 7fb767e00640 1 --2- 172.21.15.100:0/1553930217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7680b3980 0x7fb7680b3ed0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb760009cd0 tx=0x7fb76002f6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:38.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.184+0000 7fb75f400640 1 -- 172.21.15.100:0/1553930217 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb760045070 con 0x7fb7680b3980 2024-09-17T04:29:38.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.184+0000 7fb75f400640 1 -- 172.21.15.100:0/1553930217 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb760003dc0 con 0x7fb7680b3980 2024-09-17T04:29:38.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.185+0000 7fb76e200640 1 -- 172.21.15.100:0/1553930217 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7680adc90 con 0x7fb7680b3980 2024-09-17T04:29:38.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.185+0000 7fb75f400640 1 -- 172.21.15.100:0/1553930217 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76002fc80 con 0x7fb7680b3980 2024-09-17T04:29:38.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.185+0000 7fb76e200640 1 -- 172.21.15.100:0/1553930217 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7680ae250 con 0x7fb7680b3980 2024-09-17T04:29:38.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.186+0000 7fb76e200640 1 -- 172.21.15.100:0/1553930217 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb76806fb40 con 0x7fb7680b3980 2024-09-17T04:29:38.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.187+0000 7fb75f400640 1 -- 172.21.15.100:0/1553930217 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fb760048070 con 0x7fb7680b3980 2024-09-17T04:29:38.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.188+0000 7fb75f400640 1 --2- 172.21.15.100:0/1553930217 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb748078cb0 0x7fb74807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:38.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.188+0000 7fb75f400640 1 -- 172.21.15.100:0/1553930217 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6543+0+0 (secure 0 0 0) 0x7fb7600bbde0 con 0x7fb7680b3980 2024-09-17T04:29:38.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.188+0000 7fb767400640 1 --2- 172.21.15.100:0/1553930217 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb748078cb0 0x7fb74807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:38.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.189+0000 7fb767400640 1 --2- 172.21.15.100:0/1553930217 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb748078cb0 0x7fb74807b1c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb758007fd0 tx=0x7fb758009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:38.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.192+0000 7fb75f400640 1 -- 172.21.15.100:0/1553930217 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb760033020 con 0x7fb7680b3980 2024-09-17T04:29:38.398 INFO:teuthology.orchestra.run.smithi100.stderr:Invalid command: invalid chars #@ in abcd@# 2024-09-17T04:29:38.399 INFO:teuthology.orchestra.run.smithi100.stderr:fs subvolume create [] [] [] [] [] [] [--namespace-isolated] : Create a CephFS subvolume in a volume, and optionally, with a specific size (in bytes), a specific data pool layout, a specific mode, in a specific subvolume group and in separate RADOS namespace 2024-09-17T04:29:38.399 INFO:teuthology.orchestra.run.smithi100.stderr:Error EINVAL: invalid command 2024-09-17T04:29:38.417 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T04:29:38.418 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm testvol --yes-i-really-mean-it 2024-09-17T04:29:38.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.656+0000 7fb902000640 1 -- 172.21.15.100:0/990551039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8fc145450 msgr2=0x7fb8fc06f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:38.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.656+0000 7fb902000640 1 --2- 172.21.15.100:0/990551039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8fc145450 0x7fb8fc06f180 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc071590 tx=0x7fb8ec02a560 comp rx=0 tx=0).stop 2024-09-17T04:29:38.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.656+0000 7fb902000640 1 -- 172.21.15.100:0/990551039 shutdown_connections 2024-09-17T04:29:38.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.656+0000 7fb902000640 1 --2- 172.21.15.100:0/990551039 >> v2:172.21.15.184:3300/0 conn(0x7fb8fc0735e0 0x7fb8fc0739f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:38.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.656+0000 7fb902000640 1 --2- 172.21.15.100:0/990551039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8fc145450 0x7fb8fc06f180 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:38.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.656+0000 7fb902000640 1 -- 172.21.15.100:0/990551039 >> 172.21.15.100:0/990551039 conn(0x7fb8fc06d1c0 msgr2=0x7fb8fc06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:38.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.656+0000 7fb902000640 1 -- 172.21.15.100:0/990551039 shutdown_connections 2024-09-17T04:29:38.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.657+0000 7fb902000640 1 -- 172.21.15.100:0/990551039 wait complete. 2024-09-17T04:29:38.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.659+0000 7fb902000640 1 Processor -- start 2024-09-17T04:29:38.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.659+0000 7fb902000640 1 -- start start 2024-09-17T04:29:38.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.660+0000 7fb902000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8fc0735e0 0x7fb8fc13c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:38.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.660+0000 7fb902000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8fc145450 0x7fb8fc13e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:38.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.660+0000 7fb902000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8fc160320 0x7fb8fc13ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:38.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.660+0000 7fb902000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc0ce620 con 0x7fb8fc160320 2024-09-17T04:29:38.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.660+0000 7fb902000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc071020 con 0x7fb8fc0735e0 2024-09-17T04:29:38.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.660+0000 7fb902000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb8fc0ce7a0 con 0x7fb8fc145450 2024-09-17T04:29:38.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.660+0000 7fb8fbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8fc160320 0x7fb8fc13ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:38.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.660+0000 7fb8fbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8fc160320 0x7fb8fc13ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54402/0 (socket says 172.21.15.100:54402) 2024-09-17T04:29:38.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.661+0000 7fb8fbe00640 1 -- 172.21.15.100:0/421321940 learned_addr learned my addr 172.21.15.100:0/421321940 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:38.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.661+0000 7fb8fb400640 1 --2- 172.21.15.100:0/421321940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8fc0735e0 0x7fb8fc13c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:38.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.661+0000 7fb8fbe00640 1 -- 172.21.15.100:0/421321940 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8fc145450 msgr2=0x7fb8fc13e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:38.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.661+0000 7fb8fbe00640 1 --2- 172.21.15.100:0/421321940 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8fc145450 0x7fb8fc13e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:38.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.661+0000 7fb8fbe00640 1 -- 172.21.15.100:0/421321940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8fc0735e0 msgr2=0x7fb8fc13c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:38.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.661+0000 7fb8fbe00640 1 --2- 172.21.15.100:0/421321940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8fc0735e0 0x7fb8fc13c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:38.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.661+0000 7fb8fbe00640 1 -- 172.21.15.100:0/421321940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8fc13cb50 con 0x7fb8fc160320 2024-09-17T04:29:38.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.661+0000 7fb8fb400640 1 --2- 172.21.15.100:0/421321940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8fc0735e0 0x7fb8fc13c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:29:38.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.661+0000 7fb8fbe00640 1 --2- 172.21.15.100:0/421321940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8fc160320 0x7fb8fc13ed60 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fb8f400b970 tx=0x7fb8f400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:38.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.661+0000 7fb8f3400640 1 -- 172.21.15.100:0/421321940 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4019070 con 0x7fb8fc160320 2024-09-17T04:29:38.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.662+0000 7fb8f3400640 1 -- 172.21.15.100:0/421321940 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8f400cc30 con 0x7fb8fc160320 2024-09-17T04:29:38.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.662+0000 7fb8f3400640 1 -- 172.21.15.100:0/421321940 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f400ce60 con 0x7fb8fc160320 2024-09-17T04:29:38.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.662+0000 7fb902000640 1 -- 172.21.15.100:0/421321940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8fc13d830 con 0x7fb8fc160320 2024-09-17T04:29:38.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.662+0000 7fb902000640 1 -- 172.21.15.100:0/421321940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8fc13ddf0 con 0x7fb8fc160320 2024-09-17T04:29:38.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.663+0000 7fb902000640 1 -- 172.21.15.100:0/421321940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8fc13c8c0 con 0x7fb8fc160320 2024-09-17T04:29:38.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.664+0000 7fb8f3400640 1 -- 172.21.15.100:0/421321940 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fb8f401c070 con 0x7fb8fc160320 2024-09-17T04:29:38.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.665+0000 7fb8f3400640 1 --2- 172.21.15.100:0/421321940 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb8dc078cb0 0x7fb8dc07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:38.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.665+0000 7fb8f3400640 1 -- 172.21.15.100:0/421321940 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6543+0+0 (secure 0 0 0) 0x7fb8f409aec0 con 0x7fb8fc160320 2024-09-17T04:29:38.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.665+0000 7fb8fb400640 1 --2- 172.21.15.100:0/421321940 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb8dc078cb0 0x7fb8dc07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:38.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.666+0000 7fb8fb400640 1 --2- 172.21.15.100:0/421321940 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb8dc078cb0 0x7fb8dc07b1c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec033000 tx=0x7fb8ec004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:38.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.669+0000 7fb8f3400640 1 -- 172.21.15.100:0/421321940 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb8f4062d90 con 0x7fb8fc160320 2024-09-17T04:29:38.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:38.877+0000 7fb902000640 1 -- 172.21.15.100:0/421321940 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "testvol", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) -- 0x7fb8fc160f80 con 0x7fb8dc078cb0 2024-09-17T04:29:38.881 INFO:tasks.ceph.mon.a.smithi100.stderr:2024-09-17T04:29:38.881+0000 7fd3d0a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T04:29:39.006 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-09-17T04:29:41.902 INFO:teuthology.orchestra.run.smithi100.stdout:metadata pool: cephfs.testvol.meta data pool: ['cephfs.testvol.data'] removed 2024-09-17T04:29:41.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:41.902+0000 7fb8f3400640 1 -- 172.21.15.100:0/421321940 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+78 (secure 0 0 0) 0x7fb8fc160f80 con 0x7fb8dc078cb0 2024-09-17T04:29:41.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:41.907+0000 7fb902000640 1 -- 172.21.15.100:0/421321940 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb8dc078cb0 msgr2=0x7fb8dc07b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:41.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:41.907+0000 7fb902000640 1 --2- 172.21.15.100:0/421321940 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb8dc078cb0 0x7fb8dc07b1c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec033000 tx=0x7fb8ec004730 comp rx=0 tx=0).stop 2024-09-17T04:29:41.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:41.907+0000 7fb902000640 1 -- 172.21.15.100:0/421321940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8fc160320 msgr2=0x7fb8fc13ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:41.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:41.907+0000 7fb902000640 1 --2- 172.21.15.100:0/421321940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8fc160320 0x7fb8fc13ed60 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fb8f400b970 tx=0x7fb8f400be30 comp rx=0 tx=0).stop 2024-09-17T04:29:41.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:41.907+0000 7fb902000640 1 -- 172.21.15.100:0/421321940 shutdown_connections 2024-09-17T04:29:41.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:41.907+0000 7fb902000640 1 --2- 172.21.15.100:0/421321940 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb8dc078cb0 0x7fb8dc07b1c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:41.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:41.907+0000 7fb902000640 1 --2- 172.21.15.100:0/421321940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8fc160320 0x7fb8fc13ed60 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:41.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:41.907+0000 7fb902000640 1 --2- 172.21.15.100:0/421321940 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8fc145450 0x7fb8fc13e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:41.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:41.907+0000 7fb902000640 1 --2- 172.21.15.100:0/421321940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8fc0735e0 0x7fb8fc13c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:41.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:41.907+0000 7fb902000640 1 -- 172.21.15.100:0/421321940 >> 172.21.15.100:0/421321940 conn(0x7fb8fc06d1c0 msgr2=0x7fb8fc1449d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:41.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:41.907+0000 7fb902000640 1 -- 172.21.15.100:0/421321940 shutdown_connections 2024-09-17T04:29:41.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:41.908+0000 7fb902000640 1 -- 172.21.15.100:0/421321940 wait complete. 2024-09-17T04:29:41.934 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T04:29:41.934 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:29:42.015 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:29:42.105 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T04:29:42.105 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:29:42.105 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:29:42.105 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:29:42.141 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:29:42.141 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:29:42.216 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:29:42.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.479+0000 7eff1a800640 1 -- 172.21.15.100:0/2393760802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff141518d0 msgr2=0x7eff1406f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:42.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.479+0000 7eff1a800640 1 --2- 172.21.15.100:0/2393760802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff141518d0 0x7eff1406f9f0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7eff140b4ed0 tx=0x7eff0802a540 comp rx=0 tx=0).stop 2024-09-17T04:29:42.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.479+0000 7eff1a800640 1 -- 172.21.15.100:0/2393760802 shutdown_connections 2024-09-17T04:29:42.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.479+0000 7eff1a800640 1 --2- 172.21.15.100:0/2393760802 >> v2:172.21.15.184:3300/0 conn(0x7eff140735c0 0x7eff140739d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:42.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.479+0000 7eff1a800640 1 --2- 172.21.15.100:0/2393760802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff141518d0 0x7eff1406f9f0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:42.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.479+0000 7eff1a800640 1 -- 172.21.15.100:0/2393760802 >> 172.21.15.100:0/2393760802 conn(0x7eff140b1e40 msgr2=0x7eff140b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:42.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.479+0000 7eff1a800640 1 -- 172.21.15.100:0/2393760802 shutdown_connections 2024-09-17T04:29:42.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.479+0000 7eff1a800640 1 -- 172.21.15.100:0/2393760802 wait complete. 2024-09-17T04:29:42.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.483+0000 7eff1a800640 1 Processor -- start 2024-09-17T04:29:42.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.483+0000 7eff1a800640 1 -- start start 2024-09-17T04:29:42.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.486+0000 7eff1a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff140735c0 0x7eff141e67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:42.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.486+0000 7eff1a800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff141518d0 0x7eff14143960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:42.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.486+0000 7eff1a800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff1415e3d0 0x7eff141e9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:42.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.486+0000 7eff1a800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7eff14070f50 con 0x7eff140735c0 2024-09-17T04:29:42.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.486+0000 7eff1a800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7eff14070140 con 0x7eff1415e3d0 2024-09-17T04:29:42.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.486+0000 7eff1a800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7eff140710d0 con 0x7eff141518d0 2024-09-17T04:29:42.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.486+0000 7eff19400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff140735c0 0x7eff141e67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:42.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.486+0000 7eff19400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff140735c0 0x7eff141e67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54424/0 (socket says 172.21.15.100:54424) 2024-09-17T04:29:42.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.486+0000 7eff19400640 1 -- 172.21.15.100:0/2532755586 learned_addr learned my addr 172.21.15.100:0/2532755586 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:42.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.486+0000 7eff19e00640 1 --2- 172.21.15.100:0/2532755586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff1415e3d0 0x7eff141e9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:42.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.487+0000 7eff19e00640 1 -- 172.21.15.100:0/2532755586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff141518d0 msgr2=0x7eff14143960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:42.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.488+0000 7eff19e00640 1 --2- 172.21.15.100:0/2532755586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff141518d0 0x7eff14143960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:42.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.488+0000 7eff19e00640 1 -- 172.21.15.100:0/2532755586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff140735c0 msgr2=0x7eff141e67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:42.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.488+0000 7eff19e00640 1 --2- 172.21.15.100:0/2532755586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff140735c0 0x7eff141e67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:42.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.488+0000 7eff19e00640 1 -- 172.21.15.100:0/2532755586 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1409c360 con 0x7eff1415e3d0 2024-09-17T04:29:42.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.488+0000 7eff19e00640 1 --2- 172.21.15.100:0/2532755586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff1415e3d0 0x7eff141e9980 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7eff1406f940 tx=0x7eff0c00cb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:42.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.489+0000 7eff11600640 1 -- 172.21.15.100:0/2532755586 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c003b70 con 0x7eff1415e3d0 2024-09-17T04:29:42.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.489+0000 7eff11600640 1 -- 172.21.15.100:0/2532755586 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff0c003d30 con 0x7eff1415e3d0 2024-09-17T04:29:42.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.489+0000 7eff11600640 1 -- 172.21.15.100:0/2532755586 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c0052d0 con 0x7eff1415e3d0 2024-09-17T04:29:42.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.489+0000 7eff1a800640 1 -- 172.21.15.100:0/2532755586 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff1409c640 con 0x7eff1415e3d0 2024-09-17T04:29:42.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.489+0000 7eff1a800640 1 -- 172.21.15.100:0/2532755586 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff1409cc00 con 0x7eff1415e3d0 2024-09-17T04:29:42.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.491+0000 7eff1a800640 1 -- 172.21.15.100:0/2532755586 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff1406a610 con 0x7eff1415e3d0 2024-09-17T04:29:42.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.493+0000 7eff11600640 1 -- 172.21.15.100:0/2532755586 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7eff0c005470 con 0x7eff1415e3d0 2024-09-17T04:29:42.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.494+0000 7eff11600640 1 --2- 172.21.15.100:0/2532755586 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7efef4078d00 0x7efef407b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:42.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.494+0000 7eff11600640 1 -- 172.21.15.100:0/2532755586 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5795+0+0 (secure 0 0 0) 0x7eff0c09ab50 con 0x7eff1415e3d0 2024-09-17T04:29:42.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.494+0000 7eff19400640 1 --2- 172.21.15.100:0/2532755586 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7efef4078d00 0x7efef407b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:42.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.495+0000 7eff19400640 1 --2- 172.21.15.100:0/2532755586 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7efef4078d00 0x7efef407b210 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7eff080095f0 tx=0x7eff08004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:42.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.500+0000 7eff11600640 1 -- 172.21.15.100:0/2532755586 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7eff14073f10 con 0x7eff1415e3d0 2024-09-17T04:29:42.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.694+0000 7eff1a800640 1 -- 172.21.15.100:0/2532755586 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7eff140b47d0 con 0x7eff1415e3d0 2024-09-17T04:29:42.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.695+0000 7eff11600640 1 -- 172.21.15.100:0/2532755586 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v94) ==== 80+0+124 (secure 0 0 0) 0x7eff0c062df0 con 0x7eff1415e3d0 2024-09-17T04:29:42.694 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.184:6801/2063784289 2024-09-18T04:29:38.882138+0000 2024-09-17T04:29:42.694 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.184:6800/2063784289 2024-09-18T04:29:38.882138+0000 2024-09-17T04:29:42.694 INFO:teuthology.orchestra.run.smithi100.stderr:listed 2 entries 2024-09-17T04:29:42.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.697+0000 7eff1a800640 1 -- 172.21.15.100:0/2532755586 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7efef4078d00 msgr2=0x7efef407b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:42.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.697+0000 7eff1a800640 1 --2- 172.21.15.100:0/2532755586 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7efef4078d00 0x7efef407b210 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7eff080095f0 tx=0x7eff08004730 comp rx=0 tx=0).stop 2024-09-17T04:29:42.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.697+0000 7eff1a800640 1 -- 172.21.15.100:0/2532755586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff1415e3d0 msgr2=0x7eff141e9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:42.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.697+0000 7eff1a800640 1 --2- 172.21.15.100:0/2532755586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff1415e3d0 0x7eff141e9980 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7eff1406f940 tx=0x7eff0c00cb90 comp rx=0 tx=0).stop 2024-09-17T04:29:42.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.697+0000 7eff1a800640 1 -- 172.21.15.100:0/2532755586 shutdown_connections 2024-09-17T04:29:42.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.697+0000 7eff1a800640 1 --2- 172.21.15.100:0/2532755586 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7efef4078d00 0x7efef407b210 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:42.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.697+0000 7eff1a800640 1 --2- 172.21.15.100:0/2532755586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff1415e3d0 0x7eff141e9980 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:42.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.697+0000 7eff1a800640 1 --2- 172.21.15.100:0/2532755586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff141518d0 0x7eff14143960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:42.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.697+0000 7eff1a800640 1 --2- 172.21.15.100:0/2532755586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff140735c0 0x7eff141e67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:42.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.697+0000 7eff1a800640 1 -- 172.21.15.100:0/2532755586 >> 172.21.15.100:0/2532755586 conn(0x7eff140b1e40 msgr2=0x7eff140b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:42.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.697+0000 7eff1a800640 1 -- 172.21.15.100:0/2532755586 shutdown_connections 2024-09-17T04:29:42.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:42.697+0000 7eff1a800640 1 -- 172.21.15.100:0/2532755586 wait complete. 2024-09-17T04:29:42.722 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T04:29:42.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:29:42.723 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T04:29:42.786 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-09-17T04:29:42.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:29:42.786 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T04:29:42.822 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:29:42.822 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:29:42.822 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:29:42.862 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:29:42.863 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:29:42.940 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:29:43.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.163+0000 7f922ae00640 1 -- 172.21.15.100:0/1566659329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9208005150 msgr2=0x7f920800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:43.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.163+0000 7f922ae00640 1 --2- 172.21.15.100:0/1566659329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9208005150 0x7f920800b9c0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f92240cf780 tx=0x7f9208005830 comp rx=0 tx=0).stop 2024-09-17T04:29:43.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.163+0000 7f9228e00640 1 -- 172.21.15.100:0/1566659329 reap_dead start 2024-09-17T04:29:43.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.163+0000 7f922ae00640 1 -- 172.21.15.100:0/1566659329 shutdown_connections 2024-09-17T04:29:43.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.163+0000 7f922ae00640 1 -- 172.21.15.100:0/1566659329 >> 172.21.15.100:0/1566659329 conn(0x7f922406d3d0 msgr2=0x7f92240b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:43.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.163+0000 7f922ae00640 1 -- 172.21.15.100:0/1566659329 shutdown_connections 2024-09-17T04:29:43.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.163+0000 7f922ae00640 1 -- 172.21.15.100:0/1566659329 wait complete. 2024-09-17T04:29:43.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.165+0000 7f922ae00640 1 Processor -- start 2024-09-17T04:29:43.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.165+0000 7f922ae00640 1 -- start start 2024-09-17T04:29:43.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.167+0000 7f922ae00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9208005150 0x7f92240c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:43.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.167+0000 7f922ae00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92240beec0 0x7f922415b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:43.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.167+0000 7f922ae00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92240bf2d0 0x7f922415be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:43.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.167+0000 7f922ae00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f922415d790 con 0x7f92240bf2d0 2024-09-17T04:29:43.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.167+0000 7f922ae00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f922415d610 con 0x7f9208005150 2024-09-17T04:29:43.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.167+0000 7f922ae00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f922415d910 con 0x7f92240beec0 2024-09-17T04:29:43.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.167+0000 7f9228e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92240bf2d0 0x7f922415be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:43.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.167+0000 7f9228e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92240bf2d0 0x7f922415be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54450/0 (socket says 172.21.15.100:54450) 2024-09-17T04:29:43.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.167+0000 7f9228e00640 1 -- 172.21.15.100:0/3275487295 learned_addr learned my addr 172.21.15.100:0/3275487295 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:43.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.167+0000 7f9223e00640 1 --2- 172.21.15.100:0/3275487295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9208005150 0x7f92240c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:43.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.167+0000 7f921b400640 1 --2- 172.21.15.100:0/3275487295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92240beec0 0x7f922415b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:43.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.167+0000 7f9228e00640 1 -- 172.21.15.100:0/3275487295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92240beec0 msgr2=0x7f922415b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:43.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.167+0000 7f9228e00640 1 --2- 172.21.15.100:0/3275487295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92240beec0 0x7f922415b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:43.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.167+0000 7f9228e00640 1 -- 172.21.15.100:0/3275487295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9208005150 msgr2=0x7f92240c73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:43.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.167+0000 7f9228e00640 1 --2- 172.21.15.100:0/3275487295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9208005150 0x7f92240c73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:43.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.167+0000 7f9228e00640 1 -- 172.21.15.100:0/3275487295 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92240c64f0 con 0x7f92240bf2d0 2024-09-17T04:29:43.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.168+0000 7f9228e00640 1 --2- 172.21.15.100:0/3275487295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92240bf2d0 0x7f922415be40 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f921c02b890 tx=0x7f921c033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:43.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.168+0000 7f9220c00640 1 -- 172.21.15.100:0/3275487295 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921c009c10 con 0x7f92240bf2d0 2024-09-17T04:29:43.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.168+0000 7f9220c00640 1 -- 172.21.15.100:0/3275487295 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f921c044070 con 0x7f92240bf2d0 2024-09-17T04:29:43.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.168+0000 7f9220c00640 1 -- 172.21.15.100:0/3275487295 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921c03d6b0 con 0x7f92240bf2d0 2024-09-17T04:29:43.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.168+0000 7f922ae00640 1 -- 172.21.15.100:0/3275487295 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92240ca4e0 con 0x7f92240bf2d0 2024-09-17T04:29:43.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.168+0000 7f922ae00640 1 -- 172.21.15.100:0/3275487295 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92241f94d0 con 0x7f92240bf2d0 2024-09-17T04:29:43.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.169+0000 7f922ae00640 1 -- 172.21.15.100:0/3275487295 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92240ca250 con 0x7f92240bf2d0 2024-09-17T04:29:43.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.170+0000 7f9220c00640 1 -- 172.21.15.100:0/3275487295 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f921c03d850 con 0x7f92240bf2d0 2024-09-17T04:29:43.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.171+0000 7f9220c00640 1 --2- 172.21.15.100:0/3275487295 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9200078cb0 0x7f920007b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:43.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.171+0000 7f9220c00640 1 -- 172.21.15.100:0/3275487295 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5795+0+0 (secure 0 0 0) 0x7f921c0bc5d0 con 0x7f92240bf2d0 2024-09-17T04:29:43.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.172+0000 7f9223e00640 1 --2- 172.21.15.100:0/3275487295 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9200078cb0 0x7f920007b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:43.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.172+0000 7f9223e00640 1 --2- 172.21.15.100:0/3275487295 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9200078cb0 0x7f920007b1c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f922409e510 tx=0x7f9214005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:43.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.176+0000 7f9220c00640 1 -- 172.21.15.100:0/3275487295 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f921c0840a0 con 0x7f92240bf2d0 2024-09-17T04:29:43.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.370+0000 7f922ae00640 1 -- 172.21.15.100:0/3275487295 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9224156e40 con 0x7f92240bf2d0 2024-09-17T04:29:43.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.370+0000 7f9220c00640 1 -- 172.21.15.100:0/3275487295 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v94) ==== 80+0+124 (secure 0 0 0) 0x7f921c0bae20 con 0x7f92240bf2d0 2024-09-17T04:29:43.370 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.184:6801/2063784289 2024-09-18T04:29:38.882138+0000 2024-09-17T04:29:43.370 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.184:6800/2063784289 2024-09-18T04:29:38.882138+0000 2024-09-17T04:29:43.370 INFO:teuthology.orchestra.run.smithi100.stderr:listed 2 entries 2024-09-17T04:29:43.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.373+0000 7f922ae00640 1 -- 172.21.15.100:0/3275487295 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9200078cb0 msgr2=0x7f920007b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:43.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.373+0000 7f922ae00640 1 --2- 172.21.15.100:0/3275487295 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9200078cb0 0x7f920007b1c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f922409e510 tx=0x7f9214005f90 comp rx=0 tx=0).stop 2024-09-17T04:29:43.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.373+0000 7f922ae00640 1 -- 172.21.15.100:0/3275487295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92240bf2d0 msgr2=0x7f922415be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:43.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.373+0000 7f922ae00640 1 --2- 172.21.15.100:0/3275487295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92240bf2d0 0x7f922415be40 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f921c02b890 tx=0x7f921c033ba0 comp rx=0 tx=0).stop 2024-09-17T04:29:43.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.373+0000 7f922ae00640 1 -- 172.21.15.100:0/3275487295 shutdown_connections 2024-09-17T04:29:43.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.373+0000 7f922ae00640 1 --2- 172.21.15.100:0/3275487295 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9200078cb0 0x7f920007b1c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:43.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.373+0000 7f922ae00640 1 --2- 172.21.15.100:0/3275487295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92240bf2d0 0x7f922415be40 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:43.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.373+0000 7f922ae00640 1 --2- 172.21.15.100:0/3275487295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92240beec0 0x7f922415b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:43.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.373+0000 7f922ae00640 1 --2- 172.21.15.100:0/3275487295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9208005150 0x7f92240c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:43.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.373+0000 7f922ae00640 1 -- 172.21.15.100:0/3275487295 >> 172.21.15.100:0/3275487295 conn(0x7f922406d3d0 msgr2=0x7f922406f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:43.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.373+0000 7f922ae00640 1 -- 172.21.15.100:0/3275487295 shutdown_connections 2024-09-17T04:29:43.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.373+0000 7f922ae00640 1 -- 172.21.15.100:0/3275487295 wait complete. 2024-09-17T04:29:43.394 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-17T04:29:43.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:29:43.395 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-17T04:29:43.464 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2024-09-17T04:29:43.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:29:43.464 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T04:29:43.502 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:29:43.502 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:29:43.502 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:29:43.537 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:29:43.537 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:29:43.537 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:29:43.537 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:29:43.578 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:29:43.578 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:29:43.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.799+0000 7f5745400640 1 -- 172.21.15.100:0/2989337599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57400b3550 msgr2=0x7f57400b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:43.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.799+0000 7f5745400640 1 --2- 172.21.15.100:0/2989337599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57400b3550 0x7f57400b3960 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f5740072d60 tx=0x7f573802a820 comp rx=0 tx=0).stop 2024-09-17T04:29:43.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.800+0000 7f5745400640 1 -- 172.21.15.100:0/2989337599 shutdown_connections 2024-09-17T04:29:43.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.800+0000 7f5745400640 1 --2- 172.21.15.100:0/2989337599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57400b3550 0x7f57400b3960 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:43.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.800+0000 7f5745400640 1 --2- 172.21.15.100:0/2989337599 >> v2:172.21.15.184:3300/0 conn(0x7f57400ad0e0 0x7f57400ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:43.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.800+0000 7f5745400640 1 -- 172.21.15.100:0/2989337599 >> 172.21.15.100:0/2989337599 conn(0x7f574006d730 msgr2=0x7f57400b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:43.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.800+0000 7f5745400640 1 -- 172.21.15.100:0/2989337599 shutdown_connections 2024-09-17T04:29:43.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.800+0000 7f5745400640 1 -- 172.21.15.100:0/2989337599 wait complete. 2024-09-17T04:29:43.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.802+0000 7f5745400640 1 Processor -- start 2024-09-17T04:29:43.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.802+0000 7f5745400640 1 -- start start 2024-09-17T04:29:43.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.803+0000 7f5745400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57400ad0e0 0x7f5740156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:43.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.803+0000 7f5745400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57400b3550 0x7f57401f8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:43.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.803+0000 7f5745400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f574015d760 0x7f57401f93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:43.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.803+0000 7f5745400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f57400b4da0 con 0x7f57400b3550 2024-09-17T04:29:43.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.803+0000 7f5745400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f57400b4c20 con 0x7f57400ad0e0 2024-09-17T04:29:43.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.803+0000 7f5745400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f57400b3ea0 con 0x7f574015d760 2024-09-17T04:29:43.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.804+0000 7f573de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57400b3550 0x7f57401f8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:43.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.804+0000 7f573de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57400b3550 0x7f57401f8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54482/0 (socket says 172.21.15.100:54482) 2024-09-17T04:29:43.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.804+0000 7f573de00640 1 -- 172.21.15.100:0/2905297949 learned_addr learned my addr 172.21.15.100:0/2905297949 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:43.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.804+0000 7f573e800640 1 --2- 172.21.15.100:0/2905297949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57400ad0e0 0x7f5740156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:43.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.804+0000 7f573f200640 1 --2- 172.21.15.100:0/2905297949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f574015d760 0x7f57401f93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:43.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.804+0000 7f573de00640 1 -- 172.21.15.100:0/2905297949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f574015d760 msgr2=0x7f57401f93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:43.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.804+0000 7f573de00640 1 --2- 172.21.15.100:0/2905297949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f574015d760 0x7f57401f93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:43.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.804+0000 7f573de00640 1 -- 172.21.15.100:0/2905297949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57400ad0e0 msgr2=0x7f5740156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:43.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.804+0000 7f573de00640 1 --2- 172.21.15.100:0/2905297949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57400ad0e0 0x7f5740156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:43.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.804+0000 7f573de00640 1 -- 172.21.15.100:0/2905297949 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f574015fb20 con 0x7f57400b3550 2024-09-17T04:29:43.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.804+0000 7f573de00640 1 --2- 172.21.15.100:0/2905297949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57400b3550 0x7f57401f8eb0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f572c00d6d0 tx=0x7f572c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:43.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.805+0000 7f5736a00640 1 -- 172.21.15.100:0/2905297949 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c017070 con 0x7f57400b3550 2024-09-17T04:29:43.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.805+0000 7f5736a00640 1 -- 172.21.15.100:0/2905297949 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f572c004310 con 0x7f57400b3550 2024-09-17T04:29:43.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.805+0000 7f5736a00640 1 -- 172.21.15.100:0/2905297949 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c009a20 con 0x7f57400b3550 2024-09-17T04:29:43.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.805+0000 7f5745400640 1 -- 172.21.15.100:0/2905297949 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5740153f70 con 0x7f57400b3550 2024-09-17T04:29:43.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.805+0000 7f5745400640 1 -- 172.21.15.100:0/2905297949 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57401566a0 con 0x7f57400b3550 2024-09-17T04:29:43.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.806+0000 7f5745400640 1 -- 172.21.15.100:0/2905297949 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57400b43c0 con 0x7f57400b3550 2024-09-17T04:29:43.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.807+0000 7f5736a00640 1 -- 172.21.15.100:0/2905297949 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f572c0035f0 con 0x7f57400b3550 2024-09-17T04:29:43.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.808+0000 7f5736a00640 1 --2- 172.21.15.100:0/2905297949 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5720078cb0 0x7f572007b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:43.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.808+0000 7f5736a00640 1 -- 172.21.15.100:0/2905297949 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5795+0+0 (secure 0 0 0) 0x7f572c09b220 con 0x7f57400b3550 2024-09-17T04:29:43.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.808+0000 7f573e800640 1 --2- 172.21.15.100:0/2905297949 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5720078cb0 0x7f572007b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:43.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.809+0000 7f573e800640 1 --2- 172.21.15.100:0/2905297949 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5720078cb0 0x7f572007b1c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5730000f60 tx=0x7f5730009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:43.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:43.813+0000 7f5736a00640 1 -- 172.21.15.100:0/2905297949 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f572c062d80 con 0x7f57400b3550 2024-09-17T04:29:44.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.032+0000 7f5745400640 1 -- 172.21.15.100:0/2905297949 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57400ad4f0 con 0x7f57400b3550 2024-09-17T04:29:44.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.033+0000 7f5736a00640 1 -- 172.21.15.100:0/2905297949 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+5237 (secure 0 0 0) 0x7f572c0677b0 con 0x7f57400b3550 2024-09-17T04:29:44.033 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:44.033 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":63,"btime":"2024-09-17T04:29:39:884597+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5825,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3480776805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3480776805},{"type":"v1","addr":"172.21.15.184:6835","nonce":3480776805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":5948,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1319632725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1319632725},{"type":"v1","addr":"172.21.15.184:6801","nonce":1319632725}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/3591753470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3591753470},{"type":"v1","addr":"172.21.15.100:6833","nonce":3591753470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:29:22.583149+0000","modified":"2024-09-17T04:29:25.614111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6606},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6606":{"gid":6606,"name":"c","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/2660333125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2660333125},{"type":"v1","addr":"172.21.15.100:6835","nonce":2660333125}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6606,"qdb_cluster":[6606]},"id":6}]} 2024-09-17T04:29:44.033 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 63 2024-09-17T04:29:44.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.035+0000 7f5745400640 1 -- 172.21.15.100:0/2905297949 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5720078cb0 msgr2=0x7f572007b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:44.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.035+0000 7f5745400640 1 --2- 172.21.15.100:0/2905297949 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5720078cb0 0x7f572007b1c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5730000f60 tx=0x7f5730009040 comp rx=0 tx=0).stop 2024-09-17T04:29:44.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.035+0000 7f5745400640 1 -- 172.21.15.100:0/2905297949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57400b3550 msgr2=0x7f57401f8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:44.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.035+0000 7f5745400640 1 --2- 172.21.15.100:0/2905297949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57400b3550 0x7f57401f8eb0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f572c00d6d0 tx=0x7f572c00db90 comp rx=0 tx=0).stop 2024-09-17T04:29:44.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.035+0000 7f5745400640 1 -- 172.21.15.100:0/2905297949 shutdown_connections 2024-09-17T04:29:44.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.035+0000 7f5745400640 1 --2- 172.21.15.100:0/2905297949 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5720078cb0 0x7f572007b1c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:44.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.035+0000 7f5745400640 1 --2- 172.21.15.100:0/2905297949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f574015d760 0x7f57401f93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:44.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.035+0000 7f5745400640 1 --2- 172.21.15.100:0/2905297949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57400b3550 0x7f57401f8eb0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:44.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.035+0000 7f5745400640 1 --2- 172.21.15.100:0/2905297949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57400ad0e0 0x7f5740156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:44.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.035+0000 7f5745400640 1 -- 172.21.15.100:0/2905297949 >> 172.21.15.100:0/2905297949 conn(0x7f574006d730 msgr2=0x7f574006f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:44.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.035+0000 7f5745400640 1 -- 172.21.15.100:0/2905297949 shutdown_connections 2024-09-17T04:29:44.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.036+0000 7f5745400640 1 -- 172.21.15.100:0/2905297949 wait complete. 2024-09-17T04:29:44.056 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:29:44.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.277+0000 7f9deba00640 1 -- 172.21.15.100:0/1072249462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dc8004800 msgr2=0x7f9dc8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:44.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.277+0000 7f9deba00640 1 --2- 172.21.15.100:0/1072249462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dc8004800 0x7f9dc8004c10 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc009b20 tx=0x7f9dc8005830 comp rx=0 tx=0).stop 2024-09-17T04:29:44.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.277+0000 7f9deba00640 1 -- 172.21.15.100:0/1072249462 shutdown_connections 2024-09-17T04:29:44.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.277+0000 7f9deba00640 1 --2- 172.21.15.100:0/1072249462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dc8004800 0x7f9dc8004c10 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:44.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.277+0000 7f9deba00640 1 -- 172.21.15.100:0/1072249462 >> 172.21.15.100:0/1072249462 conn(0x7f9de406d730 msgr2=0x7f9de40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:44.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.277+0000 7f9deba00640 1 -- 172.21.15.100:0/1072249462 shutdown_connections 2024-09-17T04:29:44.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.277+0000 7f9deba00640 1 -- 172.21.15.100:0/1072249462 wait complete. 2024-09-17T04:29:44.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.280+0000 7f9deba00640 1 Processor -- start 2024-09-17T04:29:44.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.280+0000 7f9deba00640 1 -- start start 2024-09-17T04:29:44.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.281+0000 7f9deba00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dc8004800 0x7f9de414c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:44.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.281+0000 7f9deba00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9de41f5da0 0x7f9de4148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:44.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.281+0000 7f9deba00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9de414c930 0x7f9de414cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:44.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.281+0000 7f9deba00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9de406f260 con 0x7f9dc8004800 2024-09-17T04:29:44.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.281+0000 7f9deba00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9de40b4e30 con 0x7f9de41f5da0 2024-09-17T04:29:44.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.281+0000 7f9deba00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9de40ad4f0 con 0x7f9de414c930 2024-09-17T04:29:44.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.281+0000 7f9de9000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dc8004800 0x7f9de414c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:44.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.281+0000 7f9de9000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dc8004800 0x7f9de414c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54500/0 (socket says 172.21.15.100:54500) 2024-09-17T04:29:44.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.281+0000 7f9de9000640 1 -- 172.21.15.100:0/3921273169 learned_addr learned my addr 172.21.15.100:0/3921273169 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:44.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.281+0000 7f9ddbe00640 1 --2- 172.21.15.100:0/3921273169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9de41f5da0 0x7f9de4148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:44.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.282+0000 7f9de9a00640 1 --2- 172.21.15.100:0/3921273169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9de414c930 0x7f9de414cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:44.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.282+0000 7f9de9000640 1 -- 172.21.15.100:0/3921273169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9de414c930 msgr2=0x7f9de414cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:44.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.282+0000 7f9de9000640 1 --2- 172.21.15.100:0/3921273169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9de414c930 0x7f9de414cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:44.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.282+0000 7f9de9000640 1 -- 172.21.15.100:0/3921273169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9de41f5da0 msgr2=0x7f9de4148710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:44.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.282+0000 7f9de9000640 1 --2- 172.21.15.100:0/3921273169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9de41f5da0 0x7f9de4148710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:44.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.282+0000 7f9de9000640 1 -- 172.21.15.100:0/3921273169 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de4149600 con 0x7f9dc8004800 2024-09-17T04:29:44.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.282+0000 7f9de9000640 1 --2- 172.21.15.100:0/3921273169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dc8004800 0x7f9de414c260 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4003de0 tx=0x7f9dd4001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:44.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.282+0000 7f9de1600640 1 -- 172.21.15.100:0/3921273169 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4011640 con 0x7f9dc8004800 2024-09-17T04:29:44.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.282+0000 7f9de1600640 1 -- 172.21.15.100:0/3921273169 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9dd4016020 con 0x7f9dc8004800 2024-09-17T04:29:44.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.282+0000 7f9de1600640 1 -- 172.21.15.100:0/3921273169 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4011a40 con 0x7f9dc8004800 2024-09-17T04:29:44.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.282+0000 7f9deba00640 1 -- 172.21.15.100:0/3921273169 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9de41498e0 con 0x7f9dc8004800 2024-09-17T04:29:44.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.283+0000 7f9deba00640 1 -- 172.21.15.100:0/3921273169 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9de414ba60 con 0x7f9dc8004800 2024-09-17T04:29:44.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.284+0000 7f9deba00640 1 -- 172.21.15.100:0/3921273169 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9de41535c0 con 0x7f9dc8004800 2024-09-17T04:29:44.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.285+0000 7f9de1600640 1 -- 172.21.15.100:0/3921273169 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f9dd4011be0 con 0x7f9dc8004800 2024-09-17T04:29:44.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.286+0000 7f9de1600640 1 --2- 172.21.15.100:0/3921273169 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9dc0078cb0 0x7f9dc007b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:44.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.286+0000 7f9de1600640 1 -- 172.21.15.100:0/3921273169 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5795+0+0 (secure 0 0 0) 0x7f9dd409bb60 con 0x7f9dc8004800 2024-09-17T04:29:44.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.286+0000 7f9ddbe00640 1 --2- 172.21.15.100:0/3921273169 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9dc0078cb0 0x7f9dc007b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:44.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.287+0000 7f9ddbe00640 1 --2- 172.21.15.100:0/3921273169 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9dc0078cb0 0x7f9dc007b1c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc035000 tx=0x7f9dcc0046c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:44.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.290+0000 7f9de1600640 1 -- 172.21.15.100:0/3921273169 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9dd4020020 con 0x7f9dc8004800 2024-09-17T04:29:44.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.506+0000 7f9deba00640 1 -- 172.21.15.100:0/3921273169 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9de40731e0 con 0x7f9dc8004800 2024-09-17T04:29:44.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.507+0000 7f9de1600640 1 -- 172.21.15.100:0/3921273169 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+5237 (secure 0 0 0) 0x7f9dd40636d0 con 0x7f9dc8004800 2024-09-17T04:29:44.506 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:44.507 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":63,"btime":"2024-09-17T04:29:39:884597+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5825,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3480776805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3480776805},{"type":"v1","addr":"172.21.15.184:6835","nonce":3480776805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":5948,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1319632725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1319632725},{"type":"v1","addr":"172.21.15.184:6801","nonce":1319632725}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/3591753470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3591753470},{"type":"v1","addr":"172.21.15.100:6833","nonce":3591753470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:29:22.583149+0000","modified":"2024-09-17T04:29:25.614111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6606},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6606":{"gid":6606,"name":"c","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/2660333125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2660333125},{"type":"v1","addr":"172.21.15.100:6835","nonce":2660333125}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6606,"qdb_cluster":[6606]},"id":6}]} 2024-09-17T04:29:44.507 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 63 2024-09-17T04:29:44.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.509+0000 7f9deba00640 1 -- 172.21.15.100:0/3921273169 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9dc0078cb0 msgr2=0x7f9dc007b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:44.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.509+0000 7f9deba00640 1 --2- 172.21.15.100:0/3921273169 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9dc0078cb0 0x7f9dc007b1c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc035000 tx=0x7f9dcc0046c0 comp rx=0 tx=0).stop 2024-09-17T04:29:44.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.509+0000 7f9deba00640 1 -- 172.21.15.100:0/3921273169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dc8004800 msgr2=0x7f9de414c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:44.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.509+0000 7f9deba00640 1 --2- 172.21.15.100:0/3921273169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dc8004800 0x7f9de414c260 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4003de0 tx=0x7f9dd4001800 comp rx=0 tx=0).stop 2024-09-17T04:29:44.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.509+0000 7f9deba00640 1 -- 172.21.15.100:0/3921273169 shutdown_connections 2024-09-17T04:29:44.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.509+0000 7f9deba00640 1 --2- 172.21.15.100:0/3921273169 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9dc0078cb0 0x7f9dc007b1c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:44.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.509+0000 7f9deba00640 1 --2- 172.21.15.100:0/3921273169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9de414c930 0x7f9de414cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:44.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.509+0000 7f9deba00640 1 --2- 172.21.15.100:0/3921273169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9de41f5da0 0x7f9de4148710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:44.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.509+0000 7f9deba00640 1 --2- 172.21.15.100:0/3921273169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dc8004800 0x7f9de414c260 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:44.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.509+0000 7f9deba00640 1 -- 172.21.15.100:0/3921273169 >> 172.21.15.100:0/3921273169 conn(0x7f9de406d730 msgr2=0x7f9de40d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:44.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.509+0000 7f9deba00640 1 -- 172.21.15.100:0/3921273169 shutdown_connections 2024-09-17T04:29:44.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.509+0000 7f9deba00640 1 -- 172.21.15.100:0/3921273169 wait complete. 2024-09-17T04:29:44.533 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:29:44.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.753+0000 7f6ed5800640 1 -- 172.21.15.100:0/2668934447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ed00ad0e0 msgr2=0x7f6ed00ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:44.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.753+0000 7f6ed5800640 1 --2- 172.21.15.100:0/2668934447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ed00ad0e0 0x7f6ed00ad4f0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8009830 tx=0x7f6eb802a480 comp rx=0 tx=0).stop 2024-09-17T04:29:44.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.753+0000 7f6ed5800640 1 -- 172.21.15.100:0/2668934447 shutdown_connections 2024-09-17T04:29:44.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.753+0000 7f6ed5800640 1 --2- 172.21.15.100:0/2668934447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ed00ad0e0 0x7f6ed00ad4f0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:44.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.753+0000 7f6ed5800640 1 --2- 172.21.15.100:0/2668934447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ed015d760 0x7f6ed01535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:44.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.753+0000 7f6ed5800640 1 -- 172.21.15.100:0/2668934447 >> 172.21.15.100:0/2668934447 conn(0x7f6ed006d730 msgr2=0x7f6ed00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:44.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.753+0000 7f6ed5800640 1 -- 172.21.15.100:0/2668934447 shutdown_connections 2024-09-17T04:29:44.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.753+0000 7f6ed5800640 1 -- 172.21.15.100:0/2668934447 wait complete. 2024-09-17T04:29:44.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.755+0000 7f6ed5800640 1 Processor -- start 2024-09-17T04:29:44.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.756+0000 7f6ed5800640 1 -- start start 2024-09-17T04:29:44.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.757+0000 7f6ed5800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ed00ad0e0 0x7f6ed01e8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:44.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.757+0000 7f6ed5800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ed015d760 0x7f6ed00999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:44.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.757+0000 7f6ed5800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ed0160310 0x7f6ed0074090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:44.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.757+0000 7f6ed5800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed006f220 con 0x7f6ed015d760 2024-09-17T04:29:44.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.757+0000 7f6ed5800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed006f0a0 con 0x7f6ed0160310 2024-09-17T04:29:44.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.757+0000 7f6ed5800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6ed00b4c20 con 0x7f6ed00ad0e0 2024-09-17T04:29:44.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.757+0000 7f6ec6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ed015d760 0x7f6ed00999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:44.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.757+0000 7f6ec6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ed015d760 0x7f6ed00999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54520/0 (socket says 172.21.15.100:54520) 2024-09-17T04:29:44.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.757+0000 7f6ec6000640 1 -- 172.21.15.100:0/3550161301 learned_addr learned my addr 172.21.15.100:0/3550161301 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:44.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.757+0000 7f6ecea00640 1 --2- 172.21.15.100:0/3550161301 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ed00ad0e0 0x7f6ed01e8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:44.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.757+0000 7f6ecf400640 1 --2- 172.21.15.100:0/3550161301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ed0160310 0x7f6ed0074090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:44.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.758+0000 7f6ec6000640 1 -- 172.21.15.100:0/3550161301 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ed00ad0e0 msgr2=0x7f6ed01e8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:44.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.758+0000 7f6ec6000640 1 --2- 172.21.15.100:0/3550161301 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ed00ad0e0 0x7f6ed01e8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:44.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.758+0000 7f6ec6000640 1 -- 172.21.15.100:0/3550161301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ed0160310 msgr2=0x7f6ed0074090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:44.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.758+0000 7f6ec6000640 1 --2- 172.21.15.100:0/3550161301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ed0160310 0x7f6ed0074090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:44.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.758+0000 7f6ec6000640 1 -- 172.21.15.100:0/3550161301 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed0099ee0 con 0x7f6ed015d760 2024-09-17T04:29:44.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.758+0000 7f6ecf400640 1 --2- 172.21.15.100:0/3550161301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ed0160310 0x7f6ed0074090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:44.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.758+0000 7f6ec6000640 1 --2- 172.21.15.100:0/3550161301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ed015d760 0x7f6ed00999a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f6eb802aa70 tx=0x7f6eb8033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:44.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.758+0000 7f6ec7400640 1 -- 172.21.15.100:0/3550161301 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8005c90 con 0x7f6ed015d760 2024-09-17T04:29:44.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.758+0000 7f6ec7400640 1 -- 172.21.15.100:0/3550161301 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6eb8031e50 con 0x7f6ed015d760 2024-09-17T04:29:44.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.758+0000 7f6ed5800640 1 -- 172.21.15.100:0/3550161301 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed015c3d0 con 0x7f6ed015d760 2024-09-17T04:29:44.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.758+0000 7f6ec7400640 1 -- 172.21.15.100:0/3550161301 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8005260 con 0x7f6ed015d760 2024-09-17T04:29:44.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.759+0000 7f6ed5800640 1 -- 172.21.15.100:0/3550161301 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed01f61e0 con 0x7f6ed015d760 2024-09-17T04:29:44.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.759+0000 7f6ed5800640 1 -- 172.21.15.100:0/3550161301 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed0160fa0 con 0x7f6ed015d760 2024-09-17T04:29:44.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.761+0000 7f6ec7400640 1 -- 172.21.15.100:0/3550161301 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f6eb8005400 con 0x7f6ed015d760 2024-09-17T04:29:44.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.762+0000 7f6ec7400640 1 --2- 172.21.15.100:0/3550161301 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6eac078ce0 0x7f6eac07b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:44.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.762+0000 7f6ec7400640 1 -- 172.21.15.100:0/3550161301 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5795+0+0 (secure 0 0 0) 0x7f6eb80bc0e0 con 0x7f6ed015d760 2024-09-17T04:29:44.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.762+0000 7f6ecea00640 1 --2- 172.21.15.100:0/3550161301 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6eac078ce0 0x7f6eac07b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:44.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.763+0000 7f6ecea00640 1 --2- 172.21.15.100:0/3550161301 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6eac078ce0 0x7f6eac07b1f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0009b30 tx=0x7f6ec0005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:44.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.766+0000 7f6ec7400640 1 -- 172.21.15.100:0/3550161301 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6ed0099ee0 con 0x7f6ed015d760 2024-09-17T04:29:44.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.957+0000 7f6ed5800640 1 -- 172.21.15.100:0/3550161301 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ed0152eb0 con 0x7f6ed015d760 2024-09-17T04:29:44.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.958+0000 7f6ec7400640 1 -- 172.21.15.100:0/3550161301 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+16899 (secure 0 0 0) 0x7f6eb803f030 con 0x7f6ed015d760 2024-09-17T04:29:44.958 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:44.959 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":94,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:29:41.890487+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:29:20.485434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-17T04:29:21.553185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.184:6801/2063784289":"2024-09-18T04:29:38.882138+0000","172.21.15.184:6800/2063784289":"2024-09-18T04:29:38.882138+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:29:44.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.961+0000 7f6ed5800640 1 -- 172.21.15.100:0/3550161301 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6eac078ce0 msgr2=0x7f6eac07b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:44.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.961+0000 7f6ed5800640 1 --2- 172.21.15.100:0/3550161301 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6eac078ce0 0x7f6eac07b1f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0009b30 tx=0x7f6ec0005f90 comp rx=0 tx=0).stop 2024-09-17T04:29:44.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.961+0000 7f6ed5800640 1 -- 172.21.15.100:0/3550161301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ed015d760 msgr2=0x7f6ed00999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:44.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.961+0000 7f6ed5800640 1 --2- 172.21.15.100:0/3550161301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ed015d760 0x7f6ed00999a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f6eb802aa70 tx=0x7f6eb8033c20 comp rx=0 tx=0).stop 2024-09-17T04:29:44.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.961+0000 7f6ed5800640 1 -- 172.21.15.100:0/3550161301 shutdown_connections 2024-09-17T04:29:44.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.961+0000 7f6ed5800640 1 --2- 172.21.15.100:0/3550161301 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6eac078ce0 0x7f6eac07b1f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:44.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.961+0000 7f6ed5800640 1 --2- 172.21.15.100:0/3550161301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ed0160310 0x7f6ed0074090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:44.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.961+0000 7f6ed5800640 1 --2- 172.21.15.100:0/3550161301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ed015d760 0x7f6ed00999a0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:44.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.961+0000 7f6ed5800640 1 --2- 172.21.15.100:0/3550161301 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ed00ad0e0 0x7f6ed01e8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:44.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.961+0000 7f6ed5800640 1 -- 172.21.15.100:0/3550161301 >> 172.21.15.100:0/3550161301 conn(0x7f6ed006d730 msgr2=0x7f6ed015f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:44.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.961+0000 7f6ed5800640 1 -- 172.21.15.100:0/3550161301 shutdown_connections 2024-09-17T04:29:44.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:44.961+0000 7f6ed5800640 1 -- 172.21.15.100:0/3550161301 wait complete. 2024-09-17T04:29:44.984 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T04:29:44.984 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:29:45.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.206+0000 7f2bdea00640 1 -- 172.21.15.100:0/2413274030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bd80b9b80 msgr2=0x7f2bd80cfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:45.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.206+0000 7f2bdea00640 1 --2- 172.21.15.100:0/2413274030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bd80b9b80 0x7f2bd80cfd30 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc005fd0 tx=0x7f2bcc02aa90 comp rx=0 tx=0).stop 2024-09-17T04:29:45.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.206+0000 7f2bdea00640 1 -- 172.21.15.100:0/2413274030 shutdown_connections 2024-09-17T04:29:45.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.206+0000 7f2bdea00640 1 --2- 172.21.15.100:0/2413274030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bd81620e0 0x7f2bd80d0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:45.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.206+0000 7f2bdea00640 1 --2- 172.21.15.100:0/2413274030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bd80b9b80 0x7f2bd80cfd30 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:45.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.206+0000 7f2bdea00640 1 -- 172.21.15.100:0/2413274030 >> 172.21.15.100:0/2413274030 conn(0x7f2bd809d820 msgr2=0x7f2bd808bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:45.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.206+0000 7f2bdea00640 1 -- 172.21.15.100:0/2413274030 shutdown_connections 2024-09-17T04:29:45.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.206+0000 7f2bdea00640 1 -- 172.21.15.100:0/2413274030 wait complete. 2024-09-17T04:29:45.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.208+0000 7f2bdea00640 1 Processor -- start 2024-09-17T04:29:45.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.208+0000 7f2bdea00640 1 -- start start 2024-09-17T04:29:45.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.210+0000 7f2bdea00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bd80b9610 0x7f2bd8155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:45.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.210+0000 7f2bdea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bd80b9b80 0x7f2bd81f4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:45.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.210+0000 7f2bdea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bd81620e0 0x7f2bd81528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:45.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.210+0000 7f2bdea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd80b4360 con 0x7f2bd80b9b80 2024-09-17T04:29:45.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.210+0000 7f2bdea00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd80b41e0 con 0x7f2bd81620e0 2024-09-17T04:29:45.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.210+0000 7f2bdea00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2bd80b44e0 con 0x7f2bd80b9610 2024-09-17T04:29:45.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.210+0000 7f2bdcc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bd80b9b80 0x7f2bd81f4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:45.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.210+0000 7f2bdcc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bd80b9b80 0x7f2bd81f4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33134/0 (socket says 172.21.15.100:33134) 2024-09-17T04:29:45.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.210+0000 7f2bdcc00640 1 -- 172.21.15.100:0/3541149962 learned_addr learned my addr 172.21.15.100:0/3541149962 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:45.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.210+0000 7f2bdd600640 1 --2- 172.21.15.100:0/3541149962 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bd80b9610 0x7f2bd8155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:45.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.210+0000 7f2bde000640 1 --2- 172.21.15.100:0/3541149962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bd81620e0 0x7f2bd81528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:45.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.211+0000 7f2bdcc00640 1 -- 172.21.15.100:0/3541149962 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bd80b9610 msgr2=0x7f2bd8155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:45.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.211+0000 7f2bdcc00640 1 --2- 172.21.15.100:0/3541149962 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bd80b9610 0x7f2bd8155430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:45.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.211+0000 7f2bdcc00640 1 -- 172.21.15.100:0/3541149962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bd81620e0 msgr2=0x7f2bd81528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:45.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.211+0000 7f2bdcc00640 1 --2- 172.21.15.100:0/3541149962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bd81620e0 0x7f2bd81528a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:45.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.211+0000 7f2bdcc00640 1 -- 172.21.15.100:0/3541149962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd80b3890 con 0x7f2bd80b9b80 2024-09-17T04:29:45.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.211+0000 7f2bde000640 1 --2- 172.21.15.100:0/3541149962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bd81620e0 0x7f2bd81528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:45.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.211+0000 7f2bdcc00640 1 --2- 172.21.15.100:0/3541149962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bd80b9b80 0x7f2bd81f4d00 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400d720 tx=0x7f2bc400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:45.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.211+0000 7f2bd5600640 1 -- 172.21.15.100:0/3541149962 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4009c30 con 0x7f2bd80b9b80 2024-09-17T04:29:45.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.211+0000 7f2bd5600640 1 -- 172.21.15.100:0/3541149962 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bc400de30 con 0x7f2bd80b9b80 2024-09-17T04:29:45.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.211+0000 7f2bd5600640 1 -- 172.21.15.100:0/3541149962 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4004260 con 0x7f2bd80b9b80 2024-09-17T04:29:45.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.211+0000 7f2bdea00640 1 -- 172.21.15.100:0/3541149962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd808c3a0 con 0x7f2bd80b9b80 2024-09-17T04:29:45.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.211+0000 7f2bdea00640 1 -- 172.21.15.100:0/3541149962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd80b3e10 con 0x7f2bd80b9b80 2024-09-17T04:29:45.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.212+0000 7f2bdea00640 1 -- 172.21.15.100:0/3541149962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bd80d16a0 con 0x7f2bd80b9b80 2024-09-17T04:29:45.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.214+0000 7f2bd5600640 1 -- 172.21.15.100:0/3541149962 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f2bc4005030 con 0x7f2bd80b9b80 2024-09-17T04:29:45.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.215+0000 7f2bd5600640 1 --2- 172.21.15.100:0/3541149962 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2bb8078ce0 0x7f2bb807b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:45.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.215+0000 7f2bd5600640 1 -- 172.21.15.100:0/3541149962 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5795+0+0 (secure 0 0 0) 0x7f2bc409b4f0 con 0x7f2bd80b9b80 2024-09-17T04:29:45.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.215+0000 7f2bdd600640 1 --2- 172.21.15.100:0/3541149962 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2bb8078ce0 0x7f2bb807b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:45.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.216+0000 7f2bdd600640 1 --2- 172.21.15.100:0/3541149962 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2bb8078ce0 0x7f2bb807b1f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc035000 tx=0x7f2bcc038040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:45.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.219+0000 7f2bd5600640 1 -- 172.21.15.100:0/3541149962 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f2bc4020020 con 0x7f2bd80b9b80 2024-09-17T04:29:45.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.437+0000 7f2bdea00640 1 -- 172.21.15.100:0/3541149962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bd806cda0 con 0x7f2bd80b9b80 2024-09-17T04:29:45.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.438+0000 7f2bd5600640 1 -- 172.21.15.100:0/3541149962 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+5237 (secure 0 0 0) 0x7f2bc4017030 con 0x7f2bd80b9b80 2024-09-17T04:29:45.438 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:45.438 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":63,"btime":"2024-09-17T04:29:39:884597+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5825,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3480776805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3480776805},{"type":"v1","addr":"172.21.15.184:6835","nonce":3480776805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":5948,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1319632725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1319632725},{"type":"v1","addr":"172.21.15.184:6801","nonce":1319632725}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/3591753470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3591753470},{"type":"v1","addr":"172.21.15.100:6833","nonce":3591753470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:29:22.583149+0000","modified":"2024-09-17T04:29:25.614111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6606},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6606":{"gid":6606,"name":"c","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/2660333125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2660333125},{"type":"v1","addr":"172.21.15.100:6835","nonce":2660333125}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6606,"qdb_cluster":[6606]},"id":6}]} 2024-09-17T04:29:45.438 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 63 2024-09-17T04:29:45.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.440+0000 7f2bdea00640 1 -- 172.21.15.100:0/3541149962 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2bb8078ce0 msgr2=0x7f2bb807b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:45.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.440+0000 7f2bdea00640 1 --2- 172.21.15.100:0/3541149962 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2bb8078ce0 0x7f2bb807b1f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc035000 tx=0x7f2bcc038040 comp rx=0 tx=0).stop 2024-09-17T04:29:45.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.440+0000 7f2bdea00640 1 -- 172.21.15.100:0/3541149962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bd80b9b80 msgr2=0x7f2bd81f4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:45.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.440+0000 7f2bdea00640 1 --2- 172.21.15.100:0/3541149962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bd80b9b80 0x7f2bd81f4d00 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400d720 tx=0x7f2bc400dbe0 comp rx=0 tx=0).stop 2024-09-17T04:29:45.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.441+0000 7f2bdea00640 1 -- 172.21.15.100:0/3541149962 shutdown_connections 2024-09-17T04:29:45.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.441+0000 7f2bdea00640 1 --2- 172.21.15.100:0/3541149962 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2bb8078ce0 0x7f2bb807b1f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:45.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.441+0000 7f2bdea00640 1 --2- 172.21.15.100:0/3541149962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bd81620e0 0x7f2bd81528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:45.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.441+0000 7f2bdea00640 1 --2- 172.21.15.100:0/3541149962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bd80b9b80 0x7f2bd81f4d00 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:45.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.441+0000 7f2bdea00640 1 --2- 172.21.15.100:0/3541149962 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bd80b9610 0x7f2bd8155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:45.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.441+0000 7f2bdea00640 1 -- 172.21.15.100:0/3541149962 >> 172.21.15.100:0/3541149962 conn(0x7f2bd809d820 msgr2=0x7f2bd80b5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:45.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.441+0000 7f2bdea00640 1 -- 172.21.15.100:0/3541149962 shutdown_connections 2024-09-17T04:29:45.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.441+0000 7f2bdea00640 1 -- 172.21.15.100:0/3541149962 wait complete. 2024-09-17T04:29:45.461 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:29:45.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.683+0000 7f993c000640 1 -- 172.21.15.100:0/3561882063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f993415fae0 msgr2=0x7f99340b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:45.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.683+0000 7f993c000640 1 --2- 172.21.15.100:0/3561882063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f993415fae0 0x7f99340b3750 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f992c005fd0 tx=0x7f992c02a820 comp rx=0 tx=0).stop 2024-09-17T04:29:45.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.684+0000 7f993c000640 1 -- 172.21.15.100:0/3561882063 shutdown_connections 2024-09-17T04:29:45.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.684+0000 7f993c000640 1 --2- 172.21.15.100:0/3561882063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f993415fae0 0x7f99340b3750 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:45.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.684+0000 7f993c000640 1 --2- 172.21.15.100:0/3561882063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99340aebc0 0x7f9934152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:45.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.684+0000 7f993c000640 1 -- 172.21.15.100:0/3561882063 >> 172.21.15.100:0/3561882063 conn(0x7f993406d730 msgr2=0x7f99340b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:45.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.684+0000 7f993c000640 1 -- 172.21.15.100:0/3561882063 shutdown_connections 2024-09-17T04:29:45.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.684+0000 7f993c000640 1 -- 172.21.15.100:0/3561882063 wait complete. 2024-09-17T04:29:45.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.686+0000 7f993c000640 1 Processor -- start 2024-09-17T04:29:45.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.686+0000 7f993c000640 1 -- start start 2024-09-17T04:29:45.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.687+0000 7f993c000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99340aebc0 0x7f9934153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:45.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.687+0000 7f993c000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99340b2750 0x7f99340cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:45.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.687+0000 7f993c000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9934145370 0x7f99341f06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:45.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.688+0000 7f993c000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f99340b4260 con 0x7f9934145370 2024-09-17T04:29:45.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.688+0000 7f993c000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f99340b40e0 con 0x7f99340aebc0 2024-09-17T04:29:45.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.688+0000 7f993c000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f99340b43e0 con 0x7f99340b2750 2024-09-17T04:29:45.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.688+0000 7f993a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9934145370 0x7f99341f06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:45.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.688+0000 7f993a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9934145370 0x7f99341f06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33138/0 (socket says 172.21.15.100:33138) 2024-09-17T04:29:45.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.688+0000 7f993a000640 1 -- 172.21.15.100:0/1169492516 learned_addr learned my addr 172.21.15.100:0/1169492516 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:45.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.688+0000 7f9939600640 1 --2- 172.21.15.100:0/1169492516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99340aebc0 0x7f9934153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:45.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.688+0000 7f9938c00640 1 --2- 172.21.15.100:0/1169492516 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99340b2750 0x7f99340cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:45.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.688+0000 7f993a000640 1 -- 172.21.15.100:0/1169492516 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99340b2750 msgr2=0x7f99340cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:45.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.688+0000 7f993a000640 1 --2- 172.21.15.100:0/1169492516 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99340b2750 0x7f99340cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:45.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.688+0000 7f993a000640 1 -- 172.21.15.100:0/1169492516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99340aebc0 msgr2=0x7f9934153f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:45.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.688+0000 7f993a000640 1 --2- 172.21.15.100:0/1169492516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99340aebc0 0x7f9934153f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:45.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.688+0000 7f993a000640 1 -- 172.21.15.100:0/1169492516 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99341f0c10 con 0x7f9934145370 2024-09-17T04:29:45.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.688+0000 7f9939600640 1 --2- 172.21.15.100:0/1169492516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99340aebc0 0x7f9934153f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:45.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.689+0000 7f993a000640 1 --2- 172.21.15.100:0/1169492516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9934145370 0x7f99341f06d0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f992c009cd0 tx=0x7f992c032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:45.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.689+0000 7f9931600640 1 -- 172.21.15.100:0/1169492516 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c03f020 con 0x7f9934145370 2024-09-17T04:29:45.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.689+0000 7f9931600640 1 -- 172.21.15.100:0/1169492516 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f992c004040 con 0x7f9934145370 2024-09-17T04:29:45.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.689+0000 7f9931600640 1 -- 172.21.15.100:0/1169492516 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c037040 con 0x7f9934145370 2024-09-17T04:29:45.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.689+0000 7f993c000640 1 -- 172.21.15.100:0/1169492516 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99340ce090 con 0x7f9934145370 2024-09-17T04:29:45.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.689+0000 7f993c000640 1 -- 172.21.15.100:0/1169492516 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99340ce650 con 0x7f9934145370 2024-09-17T04:29:45.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.690+0000 7f993c000640 1 -- 172.21.15.100:0/1169492516 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99340b4900 con 0x7f9934145370 2024-09-17T04:29:45.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.691+0000 7f9931600640 1 -- 172.21.15.100:0/1169492516 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f992c042020 con 0x7f9934145370 2024-09-17T04:29:45.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.692+0000 7f9931600640 1 --2- 172.21.15.100:0/1169492516 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9914078cb0 0x7f991407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.692+0000 7f9931600640 1 -- 172.21.15.100:0/1169492516 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5795+0+0 (secure 0 0 0) 0x7f992c0bb740 con 0x7f9934145370 2024-09-17T04:29:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.693+0000 7f9939600640 1 --2- 172.21.15.100:0/1169492516 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9914078cb0 0x7f991407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:45.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.693+0000 7f9939600640 1 --2- 172.21.15.100:0/1169492516 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9914078cb0 0x7f991407b1c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9928009b80 tx=0x7f9928005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:45.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.697+0000 7f9931600640 1 -- 172.21.15.100:0/1169492516 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f992c049030 con 0x7f9934145370 2024-09-17T04:29:45.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.916+0000 7f993c000640 1 -- 172.21.15.100:0/1169492516 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99341457a0 con 0x7f9934145370 2024-09-17T04:29:45.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.917+0000 7f9931600640 1 -- 172.21.15.100:0/1169492516 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+5237 (secure 0 0 0) 0x7f992c083860 con 0x7f9934145370 2024-09-17T04:29:45.917 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:45.917 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":63,"btime":"2024-09-17T04:29:39:884597+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5825,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3480776805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3480776805},{"type":"v1","addr":"172.21.15.184:6835","nonce":3480776805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":5948,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1319632725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1319632725},{"type":"v1","addr":"172.21.15.184:6801","nonce":1319632725}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/3591753470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3591753470},{"type":"v1","addr":"172.21.15.100:6833","nonce":3591753470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:29:22.583149+0000","modified":"2024-09-17T04:29:25.614111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6606},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6606":{"gid":6606,"name":"c","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/2660333125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2660333125},{"type":"v1","addr":"172.21.15.100:6835","nonce":2660333125}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6606,"qdb_cluster":[6606]},"id":6}]} 2024-09-17T04:29:45.917 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 63 2024-09-17T04:29:45.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.919+0000 7f993c000640 1 -- 172.21.15.100:0/1169492516 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9914078cb0 msgr2=0x7f991407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:45.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.919+0000 7f993c000640 1 --2- 172.21.15.100:0/1169492516 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9914078cb0 0x7f991407b1c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9928009b80 tx=0x7f9928005f90 comp rx=0 tx=0).stop 2024-09-17T04:29:45.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.920+0000 7f993c000640 1 -- 172.21.15.100:0/1169492516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9934145370 msgr2=0x7f99341f06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:45.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.920+0000 7f993c000640 1 --2- 172.21.15.100:0/1169492516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9934145370 0x7f99341f06d0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f992c009cd0 tx=0x7f992c032c90 comp rx=0 tx=0).stop 2024-09-17T04:29:45.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.920+0000 7f993c000640 1 -- 172.21.15.100:0/1169492516 shutdown_connections 2024-09-17T04:29:45.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.920+0000 7f993c000640 1 --2- 172.21.15.100:0/1169492516 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9914078cb0 0x7f991407b1c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:45.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.920+0000 7f993c000640 1 --2- 172.21.15.100:0/1169492516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9934145370 0x7f99341f06d0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:45.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.920+0000 7f993c000640 1 --2- 172.21.15.100:0/1169492516 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99340b2750 0x7f99340cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:45.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.920+0000 7f993c000640 1 --2- 172.21.15.100:0/1169492516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99340aebc0 0x7f9934153f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:45.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.920+0000 7f993c000640 1 -- 172.21.15.100:0/1169492516 >> 172.21.15.100:0/1169492516 conn(0x7f993406d730 msgr2=0x7f993415c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:45.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.920+0000 7f993c000640 1 -- 172.21.15.100:0/1169492516 shutdown_connections 2024-09-17T04:29:45.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:45.920+0000 7f993c000640 1 -- 172.21.15.100:0/1169492516 wait complete. 2024-09-17T04:29:45.944 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:29:46.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.164+0000 7f1b2ca00640 1 -- 172.21.15.100:0/261889402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b280b3550 msgr2=0x7f1b280b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:46.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.164+0000 7f1b2ca00640 1 --2- 172.21.15.100:0/261889402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b280b3550 0x7f1b280b3960 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f1b28072d60 tx=0x7f1b2002a820 comp rx=0 tx=0).stop 2024-09-17T04:29:46.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.164+0000 7f1b2ca00640 1 -- 172.21.15.100:0/261889402 shutdown_connections 2024-09-17T04:29:46.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.164+0000 7f1b2ca00640 1 --2- 172.21.15.100:0/261889402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b280b3550 0x7f1b280b3960 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:46.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.164+0000 7f1b2ca00640 1 --2- 172.21.15.100:0/261889402 >> v2:172.21.15.184:3300/0 conn(0x7f1b280ad0e0 0x7f1b280ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:46.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.164+0000 7f1b2ca00640 1 -- 172.21.15.100:0/261889402 >> 172.21.15.100:0/261889402 conn(0x7f1b2806d730 msgr2=0x7f1b280b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:46.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.164+0000 7f1b2ca00640 1 -- 172.21.15.100:0/261889402 shutdown_connections 2024-09-17T04:29:46.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.164+0000 7f1b2ca00640 1 -- 172.21.15.100:0/261889402 wait complete. 2024-09-17T04:29:46.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.167+0000 7f1b2ca00640 1 Processor -- start 2024-09-17T04:29:46.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.167+0000 7f1b2ca00640 1 -- start start 2024-09-17T04:29:46.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.168+0000 7f1b2ca00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b280ad0e0 0x7f1b28146070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:46.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.168+0000 7f1b2ca00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b280b3550 0x7f1b281465b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:46.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.168+0000 7f1b2ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b2815d760 0x7f1b28153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:46.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.168+0000 7f1b2ca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1b280b4da0 con 0x7f1b2815d760 2024-09-17T04:29:46.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.168+0000 7f1b2ca00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1b280b4c20 con 0x7f1b280b3550 2024-09-17T04:29:46.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.168+0000 7f1b2ca00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1b280b3ea0 con 0x7f1b280ad0e0 2024-09-17T04:29:46.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.169+0000 7f1b26800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b2815d760 0x7f1b28153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:46.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.169+0000 7f1b26800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b2815d760 0x7f1b28153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33154/0 (socket says 172.21.15.100:33154) 2024-09-17T04:29:46.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.169+0000 7f1b26800640 1 -- 172.21.15.100:0/2576224405 learned_addr learned my addr 172.21.15.100:0/2576224405 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:46.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.169+0000 7f1b26800640 1 -- 172.21.15.100:0/2576224405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b280ad0e0 msgr2=0x7f1b28146070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:46.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.169+0000 7f1b26800640 1 --2- 172.21.15.100:0/2576224405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b280ad0e0 0x7f1b28146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:46.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.169+0000 7f1b25400640 1 --2- 172.21.15.100:0/2576224405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b280b3550 0x7f1b281465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:46.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.169+0000 7f1b26800640 1 -- 172.21.15.100:0/2576224405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b280b3550 msgr2=0x7f1b281465b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:46.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.169+0000 7f1b26800640 1 --2- 172.21.15.100:0/2576224405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b280b3550 0x7f1b281465b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:46.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.169+0000 7f1b26800640 1 -- 172.21.15.100:0/2576224405 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b28155dc0 con 0x7f1b2815d760 2024-09-17T04:29:46.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.169+0000 7f1b25400640 1 --2- 172.21.15.100:0/2576224405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b280b3550 0x7f1b281465b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:29:46.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.169+0000 7f1b26800640 1 --2- 172.21.15.100:0/2576224405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b2815d760 0x7f1b28153e40 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1b20009ca0 tx=0x7f1b20032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:46.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.170+0000 7f1b1e000640 1 -- 172.21.15.100:0/2576224405 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b20037040 con 0x7f1b2815d760 2024-09-17T04:29:46.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.170+0000 7f1b1e000640 1 -- 172.21.15.100:0/2576224405 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b20042070 con 0x7f1b2815d760 2024-09-17T04:29:46.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.170+0000 7f1b2ca00640 1 -- 172.21.15.100:0/2576224405 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b281560a0 con 0x7f1b2815d760 2024-09-17T04:29:46.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.170+0000 7f1b1e000640 1 -- 172.21.15.100:0/2576224405 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b20003e60 con 0x7f1b2815d760 2024-09-17T04:29:46.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.170+0000 7f1b2ca00640 1 -- 172.21.15.100:0/2576224405 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b28156660 con 0x7f1b2815d760 2024-09-17T04:29:46.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.171+0000 7f1b2ca00640 1 -- 172.21.15.100:0/2576224405 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b281f63c0 con 0x7f1b2815d760 2024-09-17T04:29:46.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.172+0000 7f1b1e000640 1 -- 172.21.15.100:0/2576224405 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f1b200042a0 con 0x7f1b2815d760 2024-09-17T04:29:46.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.173+0000 7f1b1e000640 1 --2- 172.21.15.100:0/2576224405 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1b08078cb0 0x7f1b0807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:46.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.174+0000 7f1b1e000640 1 -- 172.21.15.100:0/2576224405 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5795+0+0 (secure 0 0 0) 0x7f1b200bbd90 con 0x7f1b2815d760 2024-09-17T04:29:46.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.174+0000 7f1b25e00640 1 --2- 172.21.15.100:0/2576224405 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1b08078cb0 0x7f1b0807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:46.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.174+0000 7f1b25e00640 1 --2- 172.21.15.100:0/2576224405 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1b08078cb0 0x7f1b0807b1c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1b180046b0 tx=0x7f1b180012b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:46.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.178+0000 7f1b1e000640 1 -- 172.21.15.100:0/2576224405 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f1b20083f40 con 0x7f1b2815d760 2024-09-17T04:29:46.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.359+0000 7f1b2ca00640 1 -- 172.21.15.100:0/2576224405 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b280b43c0 con 0x7f1b2815d760 2024-09-17T04:29:46.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.360+0000 7f1b1e000640 1 -- 172.21.15.100:0/2576224405 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+16899 (secure 0 0 0) 0x7f1b20083840 con 0x7f1b2815d760 2024-09-17T04:29:46.360 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:46.360 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":94,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:29:41.890487+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:29:20.485434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-17T04:29:21.553185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.184:6801/2063784289":"2024-09-18T04:29:38.882138+0000","172.21.15.184:6800/2063784289":"2024-09-18T04:29:38.882138+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:29:46.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.363+0000 7f1b2ca00640 1 -- 172.21.15.100:0/2576224405 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1b08078cb0 msgr2=0x7f1b0807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:46.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.363+0000 7f1b2ca00640 1 --2- 172.21.15.100:0/2576224405 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1b08078cb0 0x7f1b0807b1c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1b180046b0 tx=0x7f1b180012b0 comp rx=0 tx=0).stop 2024-09-17T04:29:46.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.363+0000 7f1b2ca00640 1 -- 172.21.15.100:0/2576224405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b2815d760 msgr2=0x7f1b28153e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:46.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.363+0000 7f1b2ca00640 1 --2- 172.21.15.100:0/2576224405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b2815d760 0x7f1b28153e40 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1b20009ca0 tx=0x7f1b20032c60 comp rx=0 tx=0).stop 2024-09-17T04:29:46.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.363+0000 7f1b2ca00640 1 -- 172.21.15.100:0/2576224405 shutdown_connections 2024-09-17T04:29:46.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.363+0000 7f1b2ca00640 1 --2- 172.21.15.100:0/2576224405 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1b08078cb0 0x7f1b0807b1c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:46.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.363+0000 7f1b2ca00640 1 --2- 172.21.15.100:0/2576224405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b2815d760 0x7f1b28153e40 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:46.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.363+0000 7f1b2ca00640 1 --2- 172.21.15.100:0/2576224405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b280b3550 0x7f1b281465b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:46.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.363+0000 7f1b2ca00640 1 --2- 172.21.15.100:0/2576224405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b280ad0e0 0x7f1b28146070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:46.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.363+0000 7f1b2ca00640 1 -- 172.21.15.100:0/2576224405 >> 172.21.15.100:0/2576224405 conn(0x7f1b2806d730 msgr2=0x7f1b280b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:46.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.363+0000 7f1b2ca00640 1 -- 172.21.15.100:0/2576224405 shutdown_connections 2024-09-17T04:29:46.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.363+0000 7f1b2ca00640 1 -- 172.21.15.100:0/2576224405 wait complete. 2024-09-17T04:29:46.388 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T04:29:46.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.607+0000 7fec7f600640 1 -- 172.21.15.100:0/2369551723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec60004800 msgr2=0x7fec60004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:46.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.607+0000 7fec7f600640 1 --2- 172.21.15.100:0/2369551723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec60004800 0x7fec60004c10 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fec6400c2e0 tx=0x7fec60005830 comp rx=0 tx=0).stop 2024-09-17T04:29:46.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.607+0000 7fec7d600640 1 -- 172.21.15.100:0/2369551723 reap_dead start 2024-09-17T04:29:46.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.607+0000 7fec7f600640 1 -- 172.21.15.100:0/2369551723 shutdown_connections 2024-09-17T04:29:46.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.607+0000 7fec7f600640 1 -- 172.21.15.100:0/2369551723 >> 172.21.15.100:0/2369551723 conn(0x7fec7806d3d0 msgr2=0x7fec780b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:46.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.608+0000 7fec7f600640 1 -- 172.21.15.100:0/2369551723 shutdown_connections 2024-09-17T04:29:46.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.608+0000 7fec7f600640 1 -- 172.21.15.100:0/2369551723 wait complete. 2024-09-17T04:29:46.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.610+0000 7fec7f600640 1 Processor -- start 2024-09-17T04:29:46.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.610+0000 7fec7f600640 1 -- start start 2024-09-17T04:29:46.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.611+0000 7fec7f600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec60004800 0x7fec781547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:46.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.611+0000 7fec7f600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec780b00b0 0x7fec780c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:46.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.611+0000 7fec7f600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec780ae4f0 0x7fec780ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:46.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.611+0000 7fec7f600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fec7815d740 con 0x7fec60004800 2024-09-17T04:29:46.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.611+0000 7fec7f600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fec7815d5c0 con 0x7fec780ae4f0 2024-09-17T04:29:46.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.611+0000 7fec7f600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fec7815d8c0 con 0x7fec780b00b0 2024-09-17T04:29:46.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.611+0000 7fec7cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec60004800 0x7fec781547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:46.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.612+0000 7fec7cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec60004800 0x7fec781547b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33170/0 (socket says 172.21.15.100:33170) 2024-09-17T04:29:46.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.612+0000 7fec7cc00640 1 -- 172.21.15.100:0/1200782463 learned_addr learned my addr 172.21.15.100:0/1200782463 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:46.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.612+0000 7fec77e00640 1 --2- 172.21.15.100:0/1200782463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec780b00b0 0x7fec780c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:46.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.612+0000 7fec7d600640 1 --2- 172.21.15.100:0/1200782463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec780ae4f0 0x7fec780ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:46.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.612+0000 7fec7cc00640 1 -- 172.21.15.100:0/1200782463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec780b00b0 msgr2=0x7fec780c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:46.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.612+0000 7fec7cc00640 1 --2- 172.21.15.100:0/1200782463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec780b00b0 0x7fec780c5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:46.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.612+0000 7fec7cc00640 1 -- 172.21.15.100:0/1200782463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec780ae4f0 msgr2=0x7fec780ae900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:46.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.612+0000 7fec7cc00640 1 --2- 172.21.15.100:0/1200782463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec780ae4f0 0x7fec780ae900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:46.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.612+0000 7fec7cc00640 1 -- 172.21.15.100:0/1200782463 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec780aee40 con 0x7fec60004800 2024-09-17T04:29:46.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.612+0000 7fec77e00640 1 --2- 172.21.15.100:0/1200782463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec780b00b0 0x7fec780c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:29:46.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.612+0000 7fec7cc00640 1 --2- 172.21.15.100:0/1200782463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec60004800 0x7fec781547b0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fec6c005bc0 tx=0x7fec6c011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:46.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.612+0000 7fec74c00640 1 -- 172.21.15.100:0/1200782463 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c00fc70 con 0x7fec60004800 2024-09-17T04:29:46.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.613+0000 7fec74c00640 1 -- 172.21.15.100:0/1200782463 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec6c00fe10 con 0x7fec60004800 2024-09-17T04:29:46.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.613+0000 7fec7f600640 1 -- 172.21.15.100:0/1200782463 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec781f8d60 con 0x7fec60004800 2024-09-17T04:29:46.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.613+0000 7fec74c00640 1 -- 172.21.15.100:0/1200782463 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c019670 con 0x7fec60004800 2024-09-17T04:29:46.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.613+0000 7fec7f600640 1 -- 172.21.15.100:0/1200782463 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec781f9320 con 0x7fec60004800 2024-09-17T04:29:46.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.614+0000 7fec7f600640 1 -- 172.21.15.100:0/1200782463 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec78156e40 con 0x7fec60004800 2024-09-17T04:29:46.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.615+0000 7fec74c00640 1 -- 172.21.15.100:0/1200782463 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fec6c019810 con 0x7fec60004800 2024-09-17T04:29:46.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.616+0000 7fec74c00640 1 --2- 172.21.15.100:0/1200782463 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fec58078cb0 0x7fec5807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:46.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.616+0000 7fec74c00640 1 -- 172.21.15.100:0/1200782463 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5795+0+0 (secure 0 0 0) 0x7fec6c09baf0 con 0x7fec60004800 2024-09-17T04:29:46.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.616+0000 7fec77e00640 1 --2- 172.21.15.100:0/1200782463 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fec58078cb0 0x7fec5807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:46.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.617+0000 7fec77e00640 1 --2- 172.21.15.100:0/1200782463 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fec58078cb0 0x7fec5807b1c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fec6400c2e0 tx=0x7fec6402d000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:46.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.621+0000 7fec74c00640 1 -- 172.21.15.100:0/1200782463 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fec780aee40 con 0x7fec60004800 2024-09-17T04:29:46.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.841+0000 7fec7f600640 1 -- 172.21.15.100:0/1200782463 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fec780afa70 con 0x7fec60004800 2024-09-17T04:29:46.841 INFO:tasks.ceph.mon.a.smithi100.stderr:2024-09-17T04:29:46.842+0000 7fd3d0a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T04:29:46.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.844+0000 7fec74c00640 1 -- 172.21.15.100:0/1200782463 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v64) ==== 161+0+0 (secure 0 0 0) 0x7fec6c063ca0 con 0x7fec60004800 2024-09-17T04:29:46.843 INFO:teuthology.orchestra.run.smithi100.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T04:29:46.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.847+0000 7fec7f600640 1 -- 172.21.15.100:0/1200782463 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fec58078cb0 msgr2=0x7fec5807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:46.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.847+0000 7fec7f600640 1 --2- 172.21.15.100:0/1200782463 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fec58078cb0 0x7fec5807b1c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fec6400c2e0 tx=0x7fec6402d000 comp rx=0 tx=0).stop 2024-09-17T04:29:46.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.847+0000 7fec7f600640 1 -- 172.21.15.100:0/1200782463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec60004800 msgr2=0x7fec781547b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:46.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.847+0000 7fec7f600640 1 --2- 172.21.15.100:0/1200782463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec60004800 0x7fec781547b0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fec6c005bc0 tx=0x7fec6c011f40 comp rx=0 tx=0).stop 2024-09-17T04:29:46.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.848+0000 7fec7f600640 1 -- 172.21.15.100:0/1200782463 shutdown_connections 2024-09-17T04:29:46.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.848+0000 7fec7f600640 1 --2- 172.21.15.100:0/1200782463 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fec58078cb0 0x7fec5807b1c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:46.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.848+0000 7fec7f600640 1 --2- 172.21.15.100:0/1200782463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec780ae4f0 0x7fec780ae900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:46.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.848+0000 7fec7f600640 1 --2- 172.21.15.100:0/1200782463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec780b00b0 0x7fec780c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:46.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.848+0000 7fec7f600640 1 --2- 172.21.15.100:0/1200782463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec60004800 0x7fec781547b0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:46.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.848+0000 7fec7f600640 1 -- 172.21.15.100:0/1200782463 >> 172.21.15.100:0/1200782463 conn(0x7fec7806d3d0 msgr2=0x7fec780c6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:46.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.848+0000 7fec7f600640 1 -- 172.21.15.100:0/1200782463 shutdown_connections 2024-09-17T04:29:46.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:46.848+0000 7fec7f600640 1 -- 172.21.15.100:0/1200782463 wait complete. 2024-09-17T04:29:46.852 INFO:tasks.ceph.mds.c.smithi100.stderr: -2834> 2024-09-17T04:29:25.070+0000 7fad70400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T04:29:46.873 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T04:29:46.972 INFO:tasks.ceph.mds.c.smithi100.stdout:starting mds.c at 2024-09-17T04:29:47.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.102+0000 7fcd8a400640 1 -- 172.21.15.100:0/3877183449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd84074440 msgr2=0x7fcd8406ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:47.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.102+0000 7fcd8a400640 1 --2- 172.21.15.100:0/3877183449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd84074440 0x7fcd8406ddb0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fcd840cf060 tx=0x7fcd7402a4a0 comp rx=0 tx=0).stop 2024-09-17T04:29:47.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.102+0000 7fcd8a400640 1 -- 172.21.15.100:0/3877183449 shutdown_connections 2024-09-17T04:29:47.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.102+0000 7fcd8a400640 1 --2- 172.21.15.100:0/3877183449 >> v2:172.21.15.184:3300/0 conn(0x7fcd84070a70 0x7fcd8406e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:47.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.102+0000 7fcd8a400640 1 --2- 172.21.15.100:0/3877183449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd84074440 0x7fcd8406ddb0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:47.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.102+0000 7fcd8a400640 1 -- 172.21.15.100:0/3877183449 >> 172.21.15.100:0/3877183449 conn(0x7fcd84073920 msgr2=0x7fcd84073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:47.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.103+0000 7fcd8a400640 1 -- 172.21.15.100:0/3877183449 shutdown_connections 2024-09-17T04:29:47.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.103+0000 7fcd8a400640 1 -- 172.21.15.100:0/3877183449 wait complete. 2024-09-17T04:29:47.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.105+0000 7fcd8a400640 1 Processor -- start 2024-09-17T04:29:47.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.105+0000 7fcd8a400640 1 -- start start 2024-09-17T04:29:47.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.106+0000 7fcd8a400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd84070a70 0x7fcd841f40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:47.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.106+0000 7fcd8a400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd84074440 0x7fcd841435a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:47.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.106+0000 7fcd8a400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd8415b920 0x7fcd841f4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:47.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.106+0000 7fcd8a400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcd840b41b0 con 0x7fcd84070a70 2024-09-17T04:29:47.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.106+0000 7fcd8a400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcd840d0590 con 0x7fcd84074440 2024-09-17T04:29:47.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.107+0000 7fcd8a400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcd840b4330 con 0x7fcd8415b920 2024-09-17T04:29:47.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.107+0000 7fcd83400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd84070a70 0x7fcd841f40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:47.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.107+0000 7fcd83400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd84070a70 0x7fcd841f40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33212/0 (socket says 172.21.15.100:33212) 2024-09-17T04:29:47.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.107+0000 7fcd83400640 1 -- 172.21.15.100:0/1363312721 learned_addr learned my addr 172.21.15.100:0/1363312721 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:47.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.107+0000 7fcd7aa00640 1 --2- 172.21.15.100:0/1363312721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd84074440 0x7fcd841435a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:47.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.107+0000 7fcd83e00640 1 --2- 172.21.15.100:0/1363312721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd8415b920 0x7fcd841f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:47.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.107+0000 7fcd83400640 1 -- 172.21.15.100:0/1363312721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd8415b920 msgr2=0x7fcd841f4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:47.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.107+0000 7fcd83400640 1 --2- 172.21.15.100:0/1363312721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd8415b920 0x7fcd841f4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:47.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.107+0000 7fcd83400640 1 -- 172.21.15.100:0/1363312721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd84074440 msgr2=0x7fcd841435a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:47.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.107+0000 7fcd83400640 1 --2- 172.21.15.100:0/1363312721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd84074440 0x7fcd841435a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:47.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.107+0000 7fcd83400640 1 -- 172.21.15.100:0/1363312721 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd841f75f0 con 0x7fcd84070a70 2024-09-17T04:29:47.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.108+0000 7fcd83400640 1 --2- 172.21.15.100:0/1363312721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd84070a70 0x7fcd841f40a0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fcd7402aa90 tx=0x7fcd74033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:47.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.108+0000 7fcd7be00640 1 -- 172.21.15.100:0/1363312721 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd74005c90 con 0x7fcd84070a70 2024-09-17T04:29:47.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.108+0000 7fcd7be00640 1 -- 172.21.15.100:0/1363312721 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd74031e50 con 0x7fcd84070a70 2024-09-17T04:29:47.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.108+0000 7fcd7be00640 1 -- 172.21.15.100:0/1363312721 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd74005260 con 0x7fcd84070a70 2024-09-17T04:29:47.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.108+0000 7fcd8a400640 1 -- 172.21.15.100:0/1363312721 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd841f78d0 con 0x7fcd84070a70 2024-09-17T04:29:47.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.108+0000 7fcd8a400640 1 -- 172.21.15.100:0/1363312721 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd841f5150 con 0x7fcd84070a70 2024-09-17T04:29:47.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.109+0000 7fcd8a400640 1 -- 172.21.15.100:0/1363312721 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd841f53b0 con 0x7fcd84070a70 2024-09-17T04:29:47.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.110+0000 7fcd7be00640 1 -- 172.21.15.100:0/1363312721 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fcd74005400 con 0x7fcd84070a70 2024-09-17T04:29:47.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.111+0000 7fcd7be00640 1 --2- 172.21.15.100:0/1363312721 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcd60078cb0 0x7fcd6007b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:47.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.111+0000 7fcd7be00640 1 -- 172.21.15.100:0/1363312721 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5881+0+0 (secure 0 0 0) 0x7fcd740bc110 con 0x7fcd84070a70 2024-09-17T04:29:47.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.112+0000 7fcd7aa00640 1 --2- 172.21.15.100:0/1363312721 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcd60078cb0 0x7fcd6007b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:47.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.112+0000 7fcd7aa00640 1 --2- 172.21.15.100:0/1363312721 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcd60078cb0 0x7fcd6007b1c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcd840b2770 tx=0x7fcd6c005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:47.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.116+0000 7fcd7be00640 1 -- 172.21.15.100:0/1363312721 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fcd7403f030 con 0x7fcd84070a70 2024-09-17T04:29:47.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.336+0000 7fcd8a400640 1 -- 172.21.15.100:0/1363312721 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fcd840b4790 con 0x7fcd84070a70 2024-09-17T04:29:47.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.922+0000 7fcd7be00640 1 -- 172.21.15.100:0/1363312721 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v65) ==== 104+0+0 (secure 0 0 0) 0x7fcd74083c20 con 0x7fcd84070a70 2024-09-17T04:29:47.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.925+0000 7fcd8a400640 1 -- 172.21.15.100:0/1363312721 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcd60078cb0 msgr2=0x7fcd6007b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:47.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.925+0000 7fcd8a400640 1 --2- 172.21.15.100:0/1363312721 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcd60078cb0 0x7fcd6007b1c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcd840b2770 tx=0x7fcd6c005f90 comp rx=0 tx=0).stop 2024-09-17T04:29:47.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.925+0000 7fcd8a400640 1 -- 172.21.15.100:0/1363312721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd84070a70 msgr2=0x7fcd841f40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:47.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.925+0000 7fcd8a400640 1 --2- 172.21.15.100:0/1363312721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd84070a70 0x7fcd841f40a0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fcd7402aa90 tx=0x7fcd74033c20 comp rx=0 tx=0).stop 2024-09-17T04:29:47.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.925+0000 7fcd8a400640 1 -- 172.21.15.100:0/1363312721 shutdown_connections 2024-09-17T04:29:47.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.925+0000 7fcd8a400640 1 --2- 172.21.15.100:0/1363312721 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcd60078cb0 0x7fcd6007b1c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:47.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.925+0000 7fcd8a400640 1 --2- 172.21.15.100:0/1363312721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd8415b920 0x7fcd841f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:47.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.925+0000 7fcd8a400640 1 --2- 172.21.15.100:0/1363312721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd84074440 0x7fcd841435a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:47.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.925+0000 7fcd8a400640 1 --2- 172.21.15.100:0/1363312721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd84070a70 0x7fcd841f40a0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:47.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.925+0000 7fcd8a400640 1 -- 172.21.15.100:0/1363312721 >> 172.21.15.100:0/1363312721 conn(0x7fcd84073920 msgr2=0x7fcd8415e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:47.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.925+0000 7fcd8a400640 1 -- 172.21.15.100:0/1363312721 shutdown_connections 2024-09-17T04:29:47.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:47.925+0000 7fcd8a400640 1 -- 172.21.15.100:0/1363312721 wait complete. 2024-09-17T04:29:47.952 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T04:29:48.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.177+0000 7f7f3ce00640 1 -- 172.21.15.100:0/717789136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f380b4620 msgr2=0x7f7f38162ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:48.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.177+0000 7f7f3ce00640 1 --2- 172.21.15.100:0/717789136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f380b4620 0x7f7f38162ff0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f7f24009830 tx=0x7f7f2402a510 comp rx=0 tx=0).stop 2024-09-17T04:29:48.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.177+0000 7f7f3ce00640 1 -- 172.21.15.100:0/717789136 shutdown_connections 2024-09-17T04:29:48.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.177+0000 7f7f3ce00640 1 --2- 172.21.15.100:0/717789136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f380b4620 0x7f7f38162ff0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:48.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.177+0000 7f7f3ce00640 1 -- 172.21.15.100:0/717789136 >> 172.21.15.100:0/717789136 conn(0x7f7f380b2540 msgr2=0x7f7f380d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:48.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.177+0000 7f7f3ce00640 1 -- 172.21.15.100:0/717789136 shutdown_connections 2024-09-17T04:29:48.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.177+0000 7f7f3ce00640 1 -- 172.21.15.100:0/717789136 wait complete. 2024-09-17T04:29:48.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.180+0000 7f7f3ce00640 1 Processor -- start 2024-09-17T04:29:48.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.180+0000 7f7f3ce00640 1 -- start start 2024-09-17T04:29:48.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.181+0000 7f7f3ce00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f3806cda0 0x7f7f38156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:48.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.181+0000 7f7f3ce00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f380b4620 0x7f7f38072f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:48.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.181+0000 7f7f3ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f3815fd30 0x7f7f380cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:48.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.181+0000 7f7f3ce00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7f380af5a0 con 0x7f7f3815fd30 2024-09-17T04:29:48.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.181+0000 7f7f3ce00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7f380af420 con 0x7f7f380b4620 2024-09-17T04:29:48.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.181+0000 7f7f3ce00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7f380af720 con 0x7f7f3806cda0 2024-09-17T04:29:48.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.181+0000 7f7f36800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f3815fd30 0x7f7f380cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:48.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.181+0000 7f7f36800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f3815fd30 0x7f7f380cc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33226/0 (socket says 172.21.15.100:33226) 2024-09-17T04:29:48.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.181+0000 7f7f36800640 1 -- 172.21.15.100:0/3430278527 learned_addr learned my addr 172.21.15.100:0/3430278527 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:48.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.182+0000 7f7f35e00640 1 --2- 172.21.15.100:0/3430278527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f3806cda0 0x7f7f38156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:48.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.182+0000 7f7f35400640 1 --2- 172.21.15.100:0/3430278527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f380b4620 0x7f7f38072f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:48.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.182+0000 7f7f36800640 1 -- 172.21.15.100:0/3430278527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f3806cda0 msgr2=0x7f7f38156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:48.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.182+0000 7f7f36800640 1 --2- 172.21.15.100:0/3430278527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f3806cda0 0x7f7f38156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:48.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.182+0000 7f7f36800640 1 -- 172.21.15.100:0/3430278527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f380b4620 msgr2=0x7f7f38072f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:48.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.182+0000 7f7f36800640 1 --2- 172.21.15.100:0/3430278527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f380b4620 0x7f7f38072f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:48.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.182+0000 7f7f36800640 1 -- 172.21.15.100:0/3430278527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f38157700 con 0x7f7f3815fd30 2024-09-17T04:29:48.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.182+0000 7f7f35e00640 1 --2- 172.21.15.100:0/3430278527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f3806cda0 0x7f7f38156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:48.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.182+0000 7f7f35400640 1 --2- 172.21.15.100:0/3430278527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f380b4620 0x7f7f38072f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:48.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.182+0000 7f7f36800640 1 --2- 172.21.15.100:0/3430278527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f3815fd30 0x7f7f380cc190 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000b9e0 tx=0x7f7f3000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:48.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.182+0000 7f7f2e000640 1 -- 172.21.15.100:0/3430278527 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f30003b70 con 0x7f7f3815fd30 2024-09-17T04:29:48.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.182+0000 7f7f2e000640 1 -- 172.21.15.100:0/3430278527 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f30003e00 con 0x7f7f3815fd30 2024-09-17T04:29:48.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.182+0000 7f7f2e000640 1 -- 172.21.15.100:0/3430278527 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3000cb80 con 0x7f7f3815fd30 2024-09-17T04:29:48.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.182+0000 7f7f3ce00640 1 -- 172.21.15.100:0/3430278527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f38073e50 con 0x7f7f3815fd30 2024-09-17T04:29:48.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.183+0000 7f7f3ce00640 1 -- 172.21.15.100:0/3430278527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f380caa30 con 0x7f7f3815fd30 2024-09-17T04:29:48.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.184+0000 7f7f3ce00640 1 -- 172.21.15.100:0/3430278527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f38162ff0 con 0x7f7f3815fd30 2024-09-17T04:29:48.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.185+0000 7f7f2e000640 1 -- 172.21.15.100:0/3430278527 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f7f30004000 con 0x7f7f3815fd30 2024-09-17T04:29:48.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.186+0000 7f7f2e000640 1 --2- 172.21.15.100:0/3430278527 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7f18078ce0 0x7f7f1807b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:48.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.186+0000 7f7f2e000640 1 -- 172.21.15.100:0/3430278527 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5881+0+0 (secure 0 0 0) 0x7f7f3009af30 con 0x7f7f3815fd30 2024-09-17T04:29:48.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.186+0000 7f7f35e00640 1 --2- 172.21.15.100:0/3430278527 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7f18078ce0 0x7f7f1807b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:48.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.187+0000 7f7f35e00640 1 --2- 172.21.15.100:0/3430278527 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7f18078ce0 0x7f7f1807b1f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7f280013f0 tx=0x7f7f28009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:48.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.190+0000 7f7f2e000640 1 -- 172.21.15.100:0/3430278527 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f7f30063120 con 0x7f7f3815fd30 2024-09-17T04:29:48.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.372+0000 7f7f3ce00640 1 -- 172.21.15.100:0/3430278527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7f38158820 con 0x7f7f3815fd30 2024-09-17T04:29:48.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.930+0000 7f7f2e000640 1 -- 172.21.15.100:0/3430278527 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v96) ==== 181+0+0 (secure 0 0 0) 0x7f7f30062a20 con 0x7f7f3815fd30 2024-09-17T04:29:48.929 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' removed 2024-09-17T04:29:48.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.935+0000 7f7f3ce00640 1 -- 172.21.15.100:0/3430278527 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7f18078ce0 msgr2=0x7f7f1807b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:48.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.935+0000 7f7f3ce00640 1 --2- 172.21.15.100:0/3430278527 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7f18078ce0 0x7f7f1807b1f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7f280013f0 tx=0x7f7f28009040 comp rx=0 tx=0).stop 2024-09-17T04:29:48.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.935+0000 7f7f3ce00640 1 -- 172.21.15.100:0/3430278527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f3815fd30 msgr2=0x7f7f380cc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:48.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.935+0000 7f7f3ce00640 1 --2- 172.21.15.100:0/3430278527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f3815fd30 0x7f7f380cc190 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000b9e0 tx=0x7f7f3000bea0 comp rx=0 tx=0).stop 2024-09-17T04:29:48.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.936+0000 7f7f3ce00640 1 -- 172.21.15.100:0/3430278527 shutdown_connections 2024-09-17T04:29:48.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.936+0000 7f7f3ce00640 1 --2- 172.21.15.100:0/3430278527 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7f18078ce0 0x7f7f1807b1f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:48.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.936+0000 7f7f3ce00640 1 --2- 172.21.15.100:0/3430278527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f3815fd30 0x7f7f380cc190 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:48.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.936+0000 7f7f3ce00640 1 --2- 172.21.15.100:0/3430278527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f380b4620 0x7f7f38072f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:48.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.936+0000 7f7f3ce00640 1 --2- 172.21.15.100:0/3430278527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f3806cda0 0x7f7f38156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:48.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.936+0000 7f7f3ce00640 1 -- 172.21.15.100:0/3430278527 >> 172.21.15.100:0/3430278527 conn(0x7f7f380b2540 msgr2=0x7f7f3806f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:48.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.937+0000 7f7f3ce00640 1 -- 172.21.15.100:0/3430278527 shutdown_connections 2024-09-17T04:29:48.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:48.937+0000 7f7f3ce00640 1 -- 172.21.15.100:0/3430278527 wait complete. 2024-09-17T04:29:48.968 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T04:29:49.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.195+0000 7f29d8e00640 1 -- 172.21.15.100:0/928785321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29d40705b0 msgr2=0x7f29d40af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:49.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.195+0000 7f29d8e00640 1 --2- 172.21.15.100:0/928785321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29d40705b0 0x7f29d40af3c0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f29bc009830 tx=0x7f29bc02a480 comp rx=0 tx=0).stop 2024-09-17T04:29:49.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.195+0000 7f29d8e00640 1 -- 172.21.15.100:0/928785321 shutdown_connections 2024-09-17T04:29:49.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.195+0000 7f29d8e00640 1 --2- 172.21.15.100:0/928785321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29d40705b0 0x7f29d40af3c0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:49.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.195+0000 7f29d8e00640 1 --2- 172.21.15.100:0/928785321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29d40b3ec0 0x7f29d40b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:49.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.195+0000 7f29d8e00640 1 -- 172.21.15.100:0/928785321 >> 172.21.15.100:0/928785321 conn(0x7f29d40b1dd0 msgr2=0x7f29d40b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:49.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.195+0000 7f29d8e00640 1 -- 172.21.15.100:0/928785321 shutdown_connections 2024-09-17T04:29:49.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.195+0000 7f29d8e00640 1 -- 172.21.15.100:0/928785321 wait complete. 2024-09-17T04:29:49.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.197+0000 7f29d8e00640 1 Processor -- start 2024-09-17T04:29:49.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.198+0000 7f29d8e00640 1 -- start start 2024-09-17T04:29:49.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.199+0000 7f29d8e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29d40705b0 0x7f29d409cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:49.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.199+0000 7f29d8e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29d40b3ec0 0x7f29d4158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:49.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.199+0000 7f29d8e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29d41532e0 0x7f29d4157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:49.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.199+0000 7f29d8e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f29d4153a80 con 0x7f29d41532e0 2024-09-17T04:29:49.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.199+0000 7f29d8e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f29d4153900 con 0x7f29d40b3ec0 2024-09-17T04:29:49.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.199+0000 7f29d8e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f29d4153c00 con 0x7f29d40705b0 2024-09-17T04:29:49.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.199+0000 7f29d3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29d41532e0 0x7f29d4157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:49.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.200+0000 7f29d3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29d41532e0 0x7f29d4157380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33258/0 (socket says 172.21.15.100:33258) 2024-09-17T04:29:49.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.200+0000 7f29d3e00640 1 -- 172.21.15.100:0/2080089261 learned_addr learned my addr 172.21.15.100:0/2080089261 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:49.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.200+0000 7f29d3400640 1 --2- 172.21.15.100:0/2080089261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29d40705b0 0x7f29d409cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:49.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.200+0000 7f29caa00640 1 --2- 172.21.15.100:0/2080089261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29d40b3ec0 0x7f29d4158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:49.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.200+0000 7f29d3e00640 1 -- 172.21.15.100:0/2080089261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29d40705b0 msgr2=0x7f29d409cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:49.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.200+0000 7f29d3e00640 1 --2- 172.21.15.100:0/2080089261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29d40705b0 0x7f29d409cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:49.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.200+0000 7f29d3e00640 1 -- 172.21.15.100:0/2080089261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29d40b3ec0 msgr2=0x7f29d4158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:49.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.200+0000 7f29d3e00640 1 --2- 172.21.15.100:0/2080089261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29d40b3ec0 0x7f29d4158a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:49.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.200+0000 7f29d3e00640 1 -- 172.21.15.100:0/2080089261 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d4158f60 con 0x7f29d41532e0 2024-09-17T04:29:49.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.200+0000 7f29d3e00640 1 --2- 172.21.15.100:0/2080089261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29d41532e0 0x7f29d4157380 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f29cc00cad0 tx=0x7f29cc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:49.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.200+0000 7f29cbe00640 1 -- 172.21.15.100:0/2080089261 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc016020 con 0x7f29d41532e0 2024-09-17T04:29:49.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.201+0000 7f29cbe00640 1 -- 172.21.15.100:0/2080089261 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29cc00ad20 con 0x7f29d41532e0 2024-09-17T04:29:49.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.201+0000 7f29d8e00640 1 -- 172.21.15.100:0/2080089261 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d409bef0 con 0x7f29d41532e0 2024-09-17T04:29:49.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.201+0000 7f29cbe00640 1 -- 172.21.15.100:0/2080089261 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc012680 con 0x7f29d41532e0 2024-09-17T04:29:49.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.201+0000 7f29d8e00640 1 -- 172.21.15.100:0/2080089261 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d409c4b0 con 0x7f29d41532e0 2024-09-17T04:29:49.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.202+0000 7f29d8e00640 1 -- 172.21.15.100:0/2080089261 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29d40af3c0 con 0x7f29d41532e0 2024-09-17T04:29:49.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.203+0000 7f29cbe00640 1 -- 172.21.15.100:0/2080089261 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f29cc019020 con 0x7f29d41532e0 2024-09-17T04:29:49.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.204+0000 7f29cbe00640 1 --2- 172.21.15.100:0/2080089261 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f29b0078cb0 0x7f29b007b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:49.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.204+0000 7f29cbe00640 1 -- 172.21.15.100:0/2080089261 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5443+0+0 (secure 0 0 0) 0x7f29cc09b3f0 con 0x7f29d41532e0 2024-09-17T04:29:49.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.204+0000 7f29d3400640 1 --2- 172.21.15.100:0/2080089261 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f29b0078cb0 0x7f29b007b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:49.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.205+0000 7f29d3400640 1 --2- 172.21.15.100:0/2080089261 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f29b0078cb0 0x7f29b007b1c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f29c400bb00 tx=0x7f29c4005e30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:49.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.208+0000 7f29cbe00640 1 -- 172.21.15.100:0/2080089261 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f29cc020020 con 0x7f29d41532e0 2024-09-17T04:29:49.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.389+0000 7f29d8e00640 1 -- 172.21.15.100:0/2080089261 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f29d415e990 con 0x7f29d41532e0 2024-09-17T04:29:49.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.936+0000 7f29cbe00640 1 -- 172.21.15.100:0/2080089261 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v97) ==== 169+0+0 (secure 0 0 0) 0x7f29d40b4ec0 con 0x7f29d41532e0 2024-09-17T04:29:49.936 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' removed 2024-09-17T04:29:49.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.940+0000 7f29d8e00640 1 -- 172.21.15.100:0/2080089261 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f29b0078cb0 msgr2=0x7f29b007b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:49.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.940+0000 7f29d8e00640 1 --2- 172.21.15.100:0/2080089261 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f29b0078cb0 0x7f29b007b1c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f29c400bb00 tx=0x7f29c4005e30 comp rx=0 tx=0).stop 2024-09-17T04:29:49.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.940+0000 7f29d8e00640 1 -- 172.21.15.100:0/2080089261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29d41532e0 msgr2=0x7f29d4157380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:49.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.940+0000 7f29d8e00640 1 --2- 172.21.15.100:0/2080089261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29d41532e0 0x7f29d4157380 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f29cc00cad0 tx=0x7f29cc00cf90 comp rx=0 tx=0).stop 2024-09-17T04:29:49.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.941+0000 7f29d8e00640 1 -- 172.21.15.100:0/2080089261 shutdown_connections 2024-09-17T04:29:49.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.941+0000 7f29d8e00640 1 --2- 172.21.15.100:0/2080089261 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f29b0078cb0 0x7f29b007b1c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:49.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.941+0000 7f29d8e00640 1 --2- 172.21.15.100:0/2080089261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29d41532e0 0x7f29d4157380 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:49.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.941+0000 7f29d8e00640 1 --2- 172.21.15.100:0/2080089261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29d40b3ec0 0x7f29d4158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:49.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.941+0000 7f29d8e00640 1 --2- 172.21.15.100:0/2080089261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29d40705b0 0x7f29d409cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:49.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.941+0000 7f29d8e00640 1 -- 172.21.15.100:0/2080089261 >> 172.21.15.100:0/2080089261 conn(0x7f29d40b1dd0 msgr2=0x7f29d415de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:49.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.941+0000 7f29d8e00640 1 -- 172.21.15.100:0/2080089261 shutdown_connections 2024-09-17T04:29:49.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:49.941+0000 7f29d8e00640 1 -- 172.21.15.100:0/2080089261 wait complete. 2024-09-17T04:29:49.969 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T04:29:50.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.200+0000 7f2309600640 1 -- 172.21.15.100:0/3161316376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23040cfc80 msgr2=0x7f2304156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:50.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.200+0000 7f2309600640 1 --2- 172.21.15.100:0/3161316376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23040cfc80 0x7f2304156e40 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f23040cde30 tx=0x7f22f402a4d0 comp rx=0 tx=0).stop 2024-09-17T04:29:50.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.200+0000 7f2309600640 1 -- 172.21.15.100:0/3161316376 shutdown_connections 2024-09-17T04:29:50.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.200+0000 7f2309600640 1 --2- 172.21.15.100:0/3161316376 >> v2:172.21.15.184:3300/0 conn(0x7f230415ce30 0x7f230415d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:50.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.200+0000 7f2309600640 1 --2- 172.21.15.100:0/3161316376 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23040705b0 0x7f23040709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:50.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.200+0000 7f2309600640 1 --2- 172.21.15.100:0/3161316376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23040cfc80 0x7f2304156e40 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:50.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.200+0000 7f2309600640 1 -- 172.21.15.100:0/3161316376 >> 172.21.15.100:0/3161316376 conn(0x7f230406d3d0 msgr2=0x7f23040b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:50.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.201+0000 7f2309600640 1 -- 172.21.15.100:0/3161316376 shutdown_connections 2024-09-17T04:29:50.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.201+0000 7f2309600640 1 -- 172.21.15.100:0/3161316376 wait complete. 2024-09-17T04:29:50.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.203+0000 7f2309600640 1 Processor -- start 2024-09-17T04:29:50.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.203+0000 7f2309600640 1 -- start start 2024-09-17T04:29:50.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.204+0000 7f2309600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23040705b0 0x7f230415e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:50.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.204+0000 7f2309600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23040cfc80 0x7f2304099aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:50.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.204+0000 7f2309600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f230415ce30 0x7f23041f7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:50.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.204+0000 7f2309600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f23040cce40 con 0x7f230415ce30 2024-09-17T04:29:50.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.204+0000 7f2309600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f23040cccc0 con 0x7f23040cfc80 2024-09-17T04:29:50.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.204+0000 7f2309600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f23040ccfc0 con 0x7f23040705b0 2024-09-17T04:29:50.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.205+0000 7f2303400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f230415ce30 0x7f23041f7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:50.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.205+0000 7f2302a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23040705b0 0x7f230415e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:50.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.205+0000 7f2303400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f230415ce30 0x7f23041f7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33276/0 (socket says 172.21.15.100:33276) 2024-09-17T04:29:50.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.205+0000 7f2303400640 1 -- 172.21.15.100:0/2774251636 learned_addr learned my addr 172.21.15.100:0/2774251636 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:50.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.205+0000 7f2302000640 1 --2- 172.21.15.100:0/2774251636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23040cfc80 0x7f2304099aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:50.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.205+0000 7f2303400640 1 -- 172.21.15.100:0/2774251636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23040705b0 msgr2=0x7f230415e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:50.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.205+0000 7f2303400640 1 --2- 172.21.15.100:0/2774251636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23040705b0 0x7f230415e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:50.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.205+0000 7f2303400640 1 -- 172.21.15.100:0/2774251636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23040cfc80 msgr2=0x7f2304099aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:50.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.205+0000 7f2303400640 1 --2- 172.21.15.100:0/2774251636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23040cfc80 0x7f2304099aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:50.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.205+0000 7f2303400640 1 -- 172.21.15.100:0/2774251636 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23041f88b0 con 0x7f230415ce30 2024-09-17T04:29:50.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.206+0000 7f2303400640 1 --2- 172.21.15.100:0/2774251636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f230415ce30 0x7f23041f7950 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00e9f0 tx=0x7f22fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:50.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.206+0000 7f22faa00640 1 -- 172.21.15.100:0/2774251636 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc004d20 con 0x7f230415ce30 2024-09-17T04:29:50.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.206+0000 7f22faa00640 1 -- 172.21.15.100:0/2774251636 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22fc00cac0 con 0x7f230415ce30 2024-09-17T04:29:50.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.206+0000 7f2309600640 1 -- 172.21.15.100:0/2774251636 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23041f5610 con 0x7f230415ce30 2024-09-17T04:29:50.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.206+0000 7f22faa00640 1 -- 172.21.15.100:0/2774251636 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc005020 con 0x7f230415ce30 2024-09-17T04:29:50.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.206+0000 7f2309600640 1 -- 172.21.15.100:0/2774251636 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23041f5bd0 con 0x7f230415ce30 2024-09-17T04:29:50.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.207+0000 7f2309600640 1 -- 172.21.15.100:0/2774251636 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23040cd550 con 0x7f230415ce30 2024-09-17T04:29:50.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.208+0000 7f22faa00640 1 -- 172.21.15.100:0/2774251636 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f22fc0051c0 con 0x7f230415ce30 2024-09-17T04:29:50.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.209+0000 7f22faa00640 1 --2- 172.21.15.100:0/2774251636 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f22e4078ce0 0x7f22e407b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:50.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.209+0000 7f22faa00640 1 -- 172.21.15.100:0/2774251636 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5029+0+0 (secure 0 0 0) 0x7f22fc09b570 con 0x7f230415ce30 2024-09-17T04:29:50.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.209+0000 7f2302a00640 1 --2- 172.21.15.100:0/2774251636 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f22e4078ce0 0x7f22e407b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:50.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.210+0000 7f2302a00640 1 --2- 172.21.15.100:0/2774251636 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f22e4078ce0 0x7f22e407b1f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f22f4033000 tx=0x7f22f4004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:50.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.214+0000 7f22faa00640 1 -- 172.21.15.100:0/2774251636 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f22fc016030 con 0x7f230415ce30 2024-09-17T04:29:50.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.397+0000 7f2309600640 1 -- 172.21.15.100:0/2774251636 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f23040ca720 con 0x7f230415ce30 2024-09-17T04:29:50.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.402+0000 7f22faa00640 1 -- 172.21.15.100:0/2774251636 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v41) ==== 71+0+0 (secure 0 0 0) 0x7f22fc063340 con 0x7f230415ce30 2024-09-17T04:29:50.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.404+0000 7f2309600640 1 -- 172.21.15.100:0/2774251636 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f22e4078ce0 msgr2=0x7f22e407b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:50.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.404+0000 7f2309600640 1 --2- 172.21.15.100:0/2774251636 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f22e4078ce0 0x7f22e407b1f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f22f4033000 tx=0x7f22f4004730 comp rx=0 tx=0).stop 2024-09-17T04:29:50.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.404+0000 7f2309600640 1 -- 172.21.15.100:0/2774251636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f230415ce30 msgr2=0x7f23041f7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:50.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.404+0000 7f2309600640 1 --2- 172.21.15.100:0/2774251636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f230415ce30 0x7f23041f7950 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00e9f0 tx=0x7f22fc00eeb0 comp rx=0 tx=0).stop 2024-09-17T04:29:50.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.404+0000 7f2309600640 1 -- 172.21.15.100:0/2774251636 shutdown_connections 2024-09-17T04:29:50.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.404+0000 7f2309600640 1 --2- 172.21.15.100:0/2774251636 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f22e4078ce0 0x7f22e407b1f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:50.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.404+0000 7f2309600640 1 --2- 172.21.15.100:0/2774251636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f230415ce30 0x7f23041f7950 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:50.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.404+0000 7f2309600640 1 --2- 172.21.15.100:0/2774251636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23040cfc80 0x7f2304099aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:50.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.404+0000 7f2309600640 1 --2- 172.21.15.100:0/2774251636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23040705b0 0x7f230415e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:50.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.404+0000 7f2309600640 1 -- 172.21.15.100:0/2774251636 >> 172.21.15.100:0/2774251636 conn(0x7f230406d3d0 msgr2=0x7f23040b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:50.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.404+0000 7f2309600640 1 -- 172.21.15.100:0/2774251636 shutdown_connections 2024-09-17T04:29:50.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.405+0000 7f2309600640 1 -- 172.21.15.100:0/2774251636 wait complete. 2024-09-17T04:29:50.426 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2024-09-17T04:29:50.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.646+0000 7f8811400640 1 -- 172.21.15.100:0/3332305363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f880c15d670 msgr2=0x7f880c070530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:50.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.646+0000 7f8811400640 1 --2- 172.21.15.100:0/3332305363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f880c15d670 0x7f880c070530 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f880c0d00a0 tx=0x7f880402a4d0 comp rx=0 tx=0).stop 2024-09-17T04:29:50.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.647+0000 7f8811400640 1 -- 172.21.15.100:0/3332305363 shutdown_connections 2024-09-17T04:29:50.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.647+0000 7f8811400640 1 --2- 172.21.15.100:0/3332305363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f880c15d670 0x7f880c070530 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:50.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.647+0000 7f8811400640 1 --2- 172.21.15.100:0/3332305363 >> v2:172.21.15.184:3300/0 conn(0x7f880c0b2f60 0x7f880c15a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:50.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.647+0000 7f8811400640 1 --2- 172.21.15.100:0/3332305363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f880c0ad9c0 0x7f880c0af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:50.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.647+0000 7f8811400640 1 -- 172.21.15.100:0/3332305363 >> 172.21.15.100:0/3332305363 conn(0x7f880c0b1e40 msgr2=0x7f880c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:50.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.647+0000 7f8811400640 1 -- 172.21.15.100:0/3332305363 shutdown_connections 2024-09-17T04:29:50.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.647+0000 7f8811400640 1 -- 172.21.15.100:0/3332305363 wait complete. 2024-09-17T04:29:50.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.649+0000 7f8811400640 1 Processor -- start 2024-09-17T04:29:50.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.649+0000 7f8811400640 1 -- start start 2024-09-17T04:29:50.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.650+0000 7f8811400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f880c0ad9c0 0x7f880c154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:50.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.651+0000 7f8811400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f880c0b2f60 0x7f880c074730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:50.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.651+0000 7f8811400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f880c15d670 0x7f880c1f8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:50.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.651+0000 7f8811400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f880c06f290 con 0x7f880c0ad9c0 2024-09-17T04:29:50.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.651+0000 7f8811400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f880c070c80 con 0x7f880c0b2f60 2024-09-17T04:29:50.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.651+0000 7f8811400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f880c06f410 con 0x7f880c15d670 2024-09-17T04:29:50.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.651+0000 7f880be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f880c0ad9c0 0x7f880c154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:50.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.651+0000 7f880be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f880c0ad9c0 0x7f880c154de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33290/0 (socket says 172.21.15.100:33290) 2024-09-17T04:29:50.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.651+0000 7f880be00640 1 -- 172.21.15.100:0/2421872623 learned_addr learned my addr 172.21.15.100:0/2421872623 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:50.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.651+0000 7f8803400640 1 --2- 172.21.15.100:0/2421872623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f880c0b2f60 0x7f880c074730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:50.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.651+0000 7f8810a00640 1 --2- 172.21.15.100:0/2421872623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f880c15d670 0x7f880c1f8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:50.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.651+0000 7f880be00640 1 -- 172.21.15.100:0/2421872623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f880c15d670 msgr2=0x7f880c1f8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:50.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.651+0000 7f880be00640 1 --2- 172.21.15.100:0/2421872623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f880c15d670 0x7f880c1f8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:50.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.651+0000 7f880be00640 1 -- 172.21.15.100:0/2421872623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f880c0b2f60 msgr2=0x7f880c074730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:50.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.651+0000 7f880be00640 1 --2- 172.21.15.100:0/2421872623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f880c0b2f60 0x7f880c074730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:50.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.651+0000 7f880be00640 1 -- 172.21.15.100:0/2421872623 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880c073e40 con 0x7f880c0ad9c0 2024-09-17T04:29:50.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.651+0000 7f880be00640 1 --2- 172.21.15.100:0/2421872623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f880c0ad9c0 0x7f880c154de0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00d740 tx=0x7f87fc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:50.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.652+0000 7f8808c00640 1 -- 172.21.15.100:0/2421872623 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc017070 con 0x7f880c0ad9c0 2024-09-17T04:29:50.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.652+0000 7f8808c00640 1 -- 172.21.15.100:0/2421872623 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87fc009d20 con 0x7f880c0ad9c0 2024-09-17T04:29:50.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.652+0000 7f8811400640 1 -- 172.21.15.100:0/2421872623 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f880c074120 con 0x7f880c0ad9c0 2024-09-17T04:29:50.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.652+0000 7f8808c00640 1 -- 172.21.15.100:0/2421872623 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc0106c0 con 0x7f880c0ad9c0 2024-09-17T04:29:50.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.652+0000 7f8811400640 1 -- 172.21.15.100:0/2421872623 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f880c1e7000 con 0x7f880c0ad9c0 2024-09-17T04:29:50.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.653+0000 7f8811400640 1 -- 172.21.15.100:0/2421872623 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f880c1e7260 con 0x7f880c0ad9c0 2024-09-17T04:29:50.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.654+0000 7f8808c00640 1 -- 172.21.15.100:0/2421872623 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f87fc01a070 con 0x7f880c0ad9c0 2024-09-17T04:29:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.655+0000 7f8808c00640 1 --2- 172.21.15.100:0/2421872623 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f87e8078cb0 0x7f87e807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.655+0000 7f8808c00640 1 -- 172.21.15.100:0/2421872623 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5029+0+0 (secure 0 0 0) 0x7f87fc09ae30 con 0x7f880c0ad9c0 2024-09-17T04:29:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.655+0000 7f8803400640 1 --2- 172.21.15.100:0/2421872623 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f87e8078cb0 0x7f87e807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:50.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.656+0000 7f8803400640 1 --2- 172.21.15.100:0/2421872623 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f87e8078cb0 0x7f87e807b1c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f87f4005c40 tx=0x7f87f4005bd0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:50.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.659+0000 7f8808c00640 1 -- 172.21.15.100:0/2421872623 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f87fc063380 con 0x7f880c0ad9c0 2024-09-17T04:29:50.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.912+0000 7f8811400640 1 -- 172.21.15.100:0/2421872623 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) -- 0x7f880c06f930 con 0x7f880c0ad9c0 2024-09-17T04:29:50.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.938+0000 7f8808c00640 1 -- 172.21.15.100:0/2421872623 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v322) ==== 146+0+0 (secure 0 0 0) 0x7f87fc062c80 con 0x7f880c0ad9c0 2024-09-17T04:29:50.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.941+0000 7f8811400640 1 -- 172.21.15.100:0/2421872623 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f87e8078cb0 msgr2=0x7f87e807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:50.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.941+0000 7f8811400640 1 --2- 172.21.15.100:0/2421872623 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f87e8078cb0 0x7f87e807b1c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f87f4005c40 tx=0x7f87f4005bd0 comp rx=0 tx=0).stop 2024-09-17T04:29:50.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.942+0000 7f8811400640 1 -- 172.21.15.100:0/2421872623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f880c0ad9c0 msgr2=0x7f880c154de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:50.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.942+0000 7f8811400640 1 --2- 172.21.15.100:0/2421872623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f880c0ad9c0 0x7f880c154de0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00d740 tx=0x7f87fc00dc00 comp rx=0 tx=0).stop 2024-09-17T04:29:50.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.942+0000 7f8811400640 1 -- 172.21.15.100:0/2421872623 shutdown_connections 2024-09-17T04:29:50.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.942+0000 7f8811400640 1 --2- 172.21.15.100:0/2421872623 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f87e8078cb0 0x7f87e807b1c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:50.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.942+0000 7f8811400640 1 --2- 172.21.15.100:0/2421872623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f880c15d670 0x7f880c1f8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:50.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.942+0000 7f8811400640 1 --2- 172.21.15.100:0/2421872623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f880c0b2f60 0x7f880c074730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:50.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.942+0000 7f8811400640 1 --2- 172.21.15.100:0/2421872623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f880c0ad9c0 0x7f880c154de0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:50.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.942+0000 7f8811400640 1 -- 172.21.15.100:0/2421872623 >> 172.21.15.100:0/2421872623 conn(0x7f880c0b1e40 msgr2=0x7f880c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:50.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.942+0000 7f8811400640 1 -- 172.21.15.100:0/2421872623 shutdown_connections 2024-09-17T04:29:50.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:50.942+0000 7f8811400640 1 -- 172.21.15.100:0/2421872623 wait complete. 2024-09-17T04:29:50.967 INFO:tasks.cephfs_test_runner:Test the creating vols, subvols subvolgroups fails when their names uses ... ok 2024-09-17T04:29:50.967 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-09-17T04:29:50.967 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2024-09-17T04:29:50.967 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:29:51.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.189+0000 7f98eb800640 1 -- 172.21.15.100:0/4258331466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98e415d760 msgr2=0x7f98e41535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:51.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.189+0000 7f98eb800640 1 --2- 172.21.15.100:0/4258331466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98e415d760 0x7f98e41535c0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f98d4005fd0 tx=0x7f98d402a4a0 comp rx=0 tx=0).stop 2024-09-17T04:29:51.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.189+0000 7f98eb800640 1 -- 172.21.15.100:0/4258331466 shutdown_connections 2024-09-17T04:29:51.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.189+0000 7f98eb800640 1 --2- 172.21.15.100:0/4258331466 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98e40b3550 0x7f98e40b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:51.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.189+0000 7f98eb800640 1 --2- 172.21.15.100:0/4258331466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98e415d760 0x7f98e41535c0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:51.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.189+0000 7f98eb800640 1 -- 172.21.15.100:0/4258331466 >> 172.21.15.100:0/4258331466 conn(0x7f98e406d730 msgr2=0x7f98e40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:51.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.189+0000 7f98eb800640 1 -- 172.21.15.100:0/4258331466 shutdown_connections 2024-09-17T04:29:51.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.189+0000 7f98eb800640 1 -- 172.21.15.100:0/4258331466 wait complete. 2024-09-17T04:29:51.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.191+0000 7f98eb800640 1 Processor -- start 2024-09-17T04:29:51.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.191+0000 7f98eb800640 1 -- start start 2024-09-17T04:29:51.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.193+0000 7f98eb800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98e40ad0e0 0x7f98e4074800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:51.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.193+0000 7f98eb800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98e40b3550 0x7f98e4148cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:51.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.193+0000 7f98eb800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98e415d760 0x7f98e415b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:51.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.193+0000 7f98eb800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f98e406f240 con 0x7f98e40ad0e0 2024-09-17T04:29:51.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.193+0000 7f98eb800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f98e40b4e30 con 0x7f98e415d760 2024-09-17T04:29:51.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.193+0000 7f98eb800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f98e40b3ea0 con 0x7f98e40b3550 2024-09-17T04:29:51.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.193+0000 7f98e8e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98e40ad0e0 0x7f98e4074800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:51.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.193+0000 7f98e8e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98e40ad0e0 0x7f98e4074800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33302/0 (socket says 172.21.15.100:33302) 2024-09-17T04:29:51.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.193+0000 7f98e8e00640 1 -- 172.21.15.100:0/1855414512 learned_addr learned my addr 172.21.15.100:0/1855414512 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:51.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.194+0000 7f98dbe00640 1 --2- 172.21.15.100:0/1855414512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98e40b3550 0x7f98e4148cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:51.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.194+0000 7f98e9800640 1 --2- 172.21.15.100:0/1855414512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98e415d760 0x7f98e415b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:51.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.194+0000 7f98e8e00640 1 -- 172.21.15.100:0/1855414512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98e40b3550 msgr2=0x7f98e4148cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:51.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.194+0000 7f98e8e00640 1 --2- 172.21.15.100:0/1855414512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98e40b3550 0x7f98e4148cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:51.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.194+0000 7f98e8e00640 1 -- 172.21.15.100:0/1855414512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98e415d760 msgr2=0x7f98e415b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:51.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.194+0000 7f98e8e00640 1 --2- 172.21.15.100:0/1855414512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98e415d760 0x7f98e415b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:51.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.194+0000 7f98e8e00640 1 -- 172.21.15.100:0/1855414512 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e4074d40 con 0x7f98e40ad0e0 2024-09-17T04:29:51.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.194+0000 7f98dbe00640 1 --2- 172.21.15.100:0/1855414512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98e40b3550 0x7f98e4148cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:29:51.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.194+0000 7f98e9800640 1 --2- 172.21.15.100:0/1855414512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98e415d760 0x7f98e415b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:29:51.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.194+0000 7f98e8e00640 1 --2- 172.21.15.100:0/1855414512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98e40ad0e0 0x7f98e4074800 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f98d4006010 tx=0x7f98d40039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:51.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.194+0000 7f98e1600640 1 -- 172.21.15.100:0/1855414512 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d403e030 con 0x7f98e40ad0e0 2024-09-17T04:29:51.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.194+0000 7f98e1600640 1 -- 172.21.15.100:0/1855414512 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98d40047a0 con 0x7f98e40ad0e0 2024-09-17T04:29:51.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.194+0000 7f98eb800640 1 -- 172.21.15.100:0/1855414512 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e4149210 con 0x7f98e40ad0e0 2024-09-17T04:29:51.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.194+0000 7f98e1600640 1 -- 172.21.15.100:0/1855414512 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4035040 con 0x7f98e40ad0e0 2024-09-17T04:29:51.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.194+0000 7f98eb800640 1 -- 172.21.15.100:0/1855414512 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e41f9ad0 con 0x7f98e40ad0e0 2024-09-17T04:29:51.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.195+0000 7f98eb800640 1 -- 172.21.15.100:0/1855414512 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98e40b4370 con 0x7f98e40ad0e0 2024-09-17T04:29:51.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.197+0000 7f98e1600640 1 -- 172.21.15.100:0/1855414512 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f98d4041030 con 0x7f98e40ad0e0 2024-09-17T04:29:51.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.198+0000 7f98e1600640 1 --2- 172.21.15.100:0/1855414512 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f98c0078ce0 0x7f98c007b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:51.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.198+0000 7f98e1600640 1 -- 172.21.15.100:0/1855414512 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5029+0+0 (secure 0 0 0) 0x7f98d40bb650 con 0x7f98e40ad0e0 2024-09-17T04:29:51.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.198+0000 7f98dbe00640 1 --2- 172.21.15.100:0/1855414512 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f98c0078ce0 0x7f98c007b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:51.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.199+0000 7f98dbe00640 1 --2- 172.21.15.100:0/1855414512 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f98c0078ce0 0x7f98c007b1f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f98e41538e0 tx=0x7f98cc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:51.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.202+0000 7f98e1600640 1 -- 172.21.15.100:0/1855414512 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f98d4083b00 con 0x7f98e40ad0e0 2024-09-17T04:29:51.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.390+0000 7f98eb800640 1 -- 172.21.15.100:0/1855414512 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f98e4149560 con 0x7f98e40ad0e0 2024-09-17T04:29:51.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.391+0000 7f98e1600640 1 -- 172.21.15.100:0/1855414512 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+13215 (secure 0 0 0) 0x7f98d4083400 con 0x7f98e40ad0e0 2024-09-17T04:29:51.391 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:51.391 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":97,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:29:49.930914+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6835/2660333125":"2024-09-18T04:29:46.842375+0000","172.21.15.100:6834/2660333125":"2024-09-18T04:29:46.842375+0000","172.21.15.184:6801/2063784289":"2024-09-18T04:29:38.882138+0000","172.21.15.184:6800/2063784289":"2024-09-18T04:29:38.882138+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:29:51.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.394+0000 7f98eb800640 1 -- 172.21.15.100:0/1855414512 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f98c0078ce0 msgr2=0x7f98c007b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:51.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.394+0000 7f98eb800640 1 --2- 172.21.15.100:0/1855414512 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f98c0078ce0 0x7f98c007b1f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f98e41538e0 tx=0x7f98cc008040 comp rx=0 tx=0).stop 2024-09-17T04:29:51.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.394+0000 7f98eb800640 1 -- 172.21.15.100:0/1855414512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98e40ad0e0 msgr2=0x7f98e4074800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:51.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.394+0000 7f98eb800640 1 --2- 172.21.15.100:0/1855414512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98e40ad0e0 0x7f98e4074800 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f98d4006010 tx=0x7f98d40039e0 comp rx=0 tx=0).stop 2024-09-17T04:29:51.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.394+0000 7f98eb800640 1 -- 172.21.15.100:0/1855414512 shutdown_connections 2024-09-17T04:29:51.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.394+0000 7f98eb800640 1 --2- 172.21.15.100:0/1855414512 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f98c0078ce0 0x7f98c007b1f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:51.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.394+0000 7f98eb800640 1 --2- 172.21.15.100:0/1855414512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98e415d760 0x7f98e415b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:51.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.394+0000 7f98eb800640 1 --2- 172.21.15.100:0/1855414512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98e40b3550 0x7f98e4148cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:51.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.394+0000 7f98eb800640 1 --2- 172.21.15.100:0/1855414512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98e40ad0e0 0x7f98e4074800 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:51.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.394+0000 7f98eb800640 1 -- 172.21.15.100:0/1855414512 >> 172.21.15.100:0/1855414512 conn(0x7f98e406d730 msgr2=0x7f98e40cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:51.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.394+0000 7f98eb800640 1 -- 172.21.15.100:0/1855414512 shutdown_connections 2024-09-17T04:29:51.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.394+0000 7f98eb800640 1 -- 172.21.15.100:0/1855414512 wait complete. 2024-09-17T04:29:51.416 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T04:24:03.995355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T04:29:51.416 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T04:29:51.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.639+0000 7f0739c00640 1 -- 172.21.15.100:0/641721145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07340b4f00 msgr2=0x7f07340704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:51.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.639+0000 7f0739c00640 1 --2- 172.21.15.100:0/641721145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07340b4f00 0x7f07340704a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f0724009830 tx=0x7f072402a510 comp rx=0 tx=0).stop 2024-09-17T04:29:51.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.639+0000 7f0739c00640 1 -- 172.21.15.100:0/641721145 shutdown_connections 2024-09-17T04:29:51.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.639+0000 7f0739c00640 1 --2- 172.21.15.100:0/641721145 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0734160490 0x7f073415d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:51.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.639+0000 7f0739c00640 1 --2- 172.21.15.100:0/641721145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07340b4f00 0x7f07340704a0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:51.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.639+0000 7f0739c00640 1 -- 172.21.15.100:0/641721145 >> 172.21.15.100:0/641721145 conn(0x7f073406d710 msgr2=0x7f07340b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:51.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.639+0000 7f0739c00640 1 -- 172.21.15.100:0/641721145 shutdown_connections 2024-09-17T04:29:51.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.639+0000 7f0739c00640 1 -- 172.21.15.100:0/641721145 wait complete. 2024-09-17T04:29:51.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.641+0000 7f0739c00640 1 Processor -- start 2024-09-17T04:29:51.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.641+0000 7f0739c00640 1 -- start start 2024-09-17T04:29:51.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.643+0000 7f0739c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07340ad3d0 0x7f073409bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:51.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.643+0000 7f0739c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07340b4f00 0x7f0734068e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:51.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.643+0000 7f0739c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0734160490 0x7f07340693c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:51.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.643+0000 7f0739c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f073406f0f0 con 0x7f0734160490 2024-09-17T04:29:51.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.643+0000 7f0739c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f073406ef70 con 0x7f07340ad3d0 2024-09-17T04:29:51.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.643+0000 7f0739c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f073406f270 con 0x7f07340b4f00 2024-09-17T04:29:51.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.643+0000 7f0733400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0734160490 0x7f07340693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:51.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.643+0000 7f0733400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0734160490 0x7f07340693c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33324/0 (socket says 172.21.15.100:33324) 2024-09-17T04:29:51.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.643+0000 7f0733400640 1 -- 172.21.15.100:0/3318054644 learned_addr learned my addr 172.21.15.100:0/3318054644 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:51.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.643+0000 7f0732a00640 1 --2- 172.21.15.100:0/3318054644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07340ad3d0 0x7f073409bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:51.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.643+0000 7f072a000640 1 --2- 172.21.15.100:0/3318054644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07340b4f00 0x7f0734068e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:51.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.643+0000 7f0733400640 1 -- 172.21.15.100:0/3318054644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07340b4f00 msgr2=0x7f0734068e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:51.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.643+0000 7f0733400640 1 --2- 172.21.15.100:0/3318054644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07340b4f00 0x7f0734068e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:51.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.643+0000 7f0733400640 1 -- 172.21.15.100:0/3318054644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07340ad3d0 msgr2=0x7f073409bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:51.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.643+0000 7f0733400640 1 --2- 172.21.15.100:0/3318054644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07340ad3d0 0x7f073409bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:51.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.643+0000 7f0733400640 1 -- 172.21.15.100:0/3318054644 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f073409cff0 con 0x7f0734160490 2024-09-17T04:29:51.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.644+0000 7f0733400640 1 --2- 172.21.15.100:0/3318054644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0734160490 0x7f07340693c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f072c00c730 tx=0x7f072c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:51.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.644+0000 7f072b400640 1 -- 172.21.15.100:0/3318054644 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072c003b70 con 0x7f0734160490 2024-09-17T04:29:51.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.644+0000 7f072b400640 1 -- 172.21.15.100:0/3318054644 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f072c003d30 con 0x7f0734160490 2024-09-17T04:29:51.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.644+0000 7f072b400640 1 -- 172.21.15.100:0/3318054644 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072c0052d0 con 0x7f0734160490 2024-09-17T04:29:51.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.644+0000 7f0739c00640 1 -- 172.21.15.100:0/3318054644 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f073409d2d0 con 0x7f0734160490 2024-09-17T04:29:51.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.644+0000 7f0739c00640 1 -- 172.21.15.100:0/3318054644 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f073409d890 con 0x7f0734160490 2024-09-17T04:29:51.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.645+0000 7f0739c00640 1 -- 172.21.15.100:0/3318054644 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f073406f740 con 0x7f0734160490 2024-09-17T04:29:51.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.647+0000 7f072b400640 1 -- 172.21.15.100:0/3318054644 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f072c005470 con 0x7f0734160490 2024-09-17T04:29:51.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.648+0000 7f072b400640 1 --2- 172.21.15.100:0/3318054644 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0710078cb0 0x7f071007b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:51.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.648+0000 7f072b400640 1 -- 172.21.15.100:0/3318054644 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5029+0+0 (secure 0 0 0) 0x7f072c09a980 con 0x7f0734160490 2024-09-17T04:29:51.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.648+0000 7f0732a00640 1 --2- 172.21.15.100:0/3318054644 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0710078cb0 0x7f071007b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:51.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.649+0000 7f0732a00640 1 --2- 172.21.15.100:0/3318054644 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0710078cb0 0x7f071007b1c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0724034000 tx=0x7f0724004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:51.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.652+0000 7f072b400640 1 -- 172.21.15.100:0/3318054644 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f072c0627f0 con 0x7f0734160490 2024-09-17T04:29:51.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.834+0000 7f0739c00640 1 -- 172.21.15.100:0/3318054644 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f07340732a0 con 0x7f0734160490 2024-09-17T04:29:51.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.834+0000 7f072b400640 1 -- 172.21.15.100:0/3318054644 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f072c067220 con 0x7f0734160490 2024-09-17T04:29:51.835 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-09-17T04:29:51.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.836+0000 7f0739c00640 1 -- 172.21.15.100:0/3318054644 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0710078cb0 msgr2=0x7f071007b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:51.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.836+0000 7f0739c00640 1 --2- 172.21.15.100:0/3318054644 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0710078cb0 0x7f071007b1c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0724034000 tx=0x7f0724004730 comp rx=0 tx=0).stop 2024-09-17T04:29:51.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.836+0000 7f0739c00640 1 -- 172.21.15.100:0/3318054644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0734160490 msgr2=0x7f07340693c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:51.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.836+0000 7f0739c00640 1 --2- 172.21.15.100:0/3318054644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0734160490 0x7f07340693c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f072c00c730 tx=0x7f072c00cbf0 comp rx=0 tx=0).stop 2024-09-17T04:29:51.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.836+0000 7f0739c00640 1 -- 172.21.15.100:0/3318054644 shutdown_connections 2024-09-17T04:29:51.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.836+0000 7f0739c00640 1 --2- 172.21.15.100:0/3318054644 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0710078cb0 0x7f071007b1c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:51.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.836+0000 7f0739c00640 1 --2- 172.21.15.100:0/3318054644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0734160490 0x7f07340693c0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:51.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.836+0000 7f0739c00640 1 --2- 172.21.15.100:0/3318054644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07340b4f00 0x7f0734068e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:51.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.837+0000 7f0739c00640 1 --2- 172.21.15.100:0/3318054644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07340ad3d0 0x7f073409bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:51.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.837+0000 7f0739c00640 1 -- 172.21.15.100:0/3318054644 >> 172.21.15.100:0/3318054644 conn(0x7f073406d710 msgr2=0x7f0734156880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:51.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.837+0000 7f0739c00640 1 -- 172.21.15.100:0/3318054644 shutdown_connections 2024-09-17T04:29:51.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:51.837+0000 7f0739c00640 1 -- 172.21.15.100:0/3318054644 wait complete. 2024-09-17T04:29:51.858 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2024-09-17T04:29:52.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.083+0000 7fed06200640 1 -- 172.21.15.100:0/1225982415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed000b2750 msgr2=0x7fed0015f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:52.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.083+0000 7fed06200640 1 --2- 172.21.15.100:0/1225982415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed000b2750 0x7fed0015f3d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fece8005fd0 tx=0x7fece802a540 comp rx=0 tx=0).stop 2024-09-17T04:29:52.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.083+0000 7fed06200640 1 -- 172.21.15.100:0/1225982415 shutdown_connections 2024-09-17T04:29:52.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.083+0000 7fed06200640 1 --2- 172.21.15.100:0/1225982415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed000b2750 0x7fed0015f3d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:52.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.083+0000 7fed06200640 1 --2- 172.21.15.100:0/1225982415 >> v2:172.21.15.184:3300/0 conn(0x7fed000aebc0 0x7fed00152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:52.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.083+0000 7fed06200640 1 -- 172.21.15.100:0/1225982415 >> 172.21.15.100:0/1225982415 conn(0x7fed0006d730 msgr2=0x7fed000b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:52.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.083+0000 7fed06200640 1 -- 172.21.15.100:0/1225982415 shutdown_connections 2024-09-17T04:29:52.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.083+0000 7fed06200640 1 -- 172.21.15.100:0/1225982415 wait complete. 2024-09-17T04:29:52.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.085+0000 7fed06200640 1 Processor -- start 2024-09-17T04:29:52.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.085+0000 7fed06200640 1 -- start start 2024-09-17T04:29:52.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.087+0000 7fed06200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed000aebc0 0x7fed00157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:52.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.087+0000 7fed06200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed000b2750 0x7fed001f3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:52.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.087+0000 7fed06200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed0015f910 0x7fed00150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:52.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.087+0000 7fed06200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fed0015c4a0 con 0x7fed000aebc0 2024-09-17T04:29:52.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.087+0000 7fed06200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fed000b4a20 con 0x7fed000b2750 2024-09-17T04:29:52.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.087+0000 7fed06200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fed0015c620 con 0x7fed0015f910 2024-09-17T04:29:52.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.087+0000 7fecff400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed000aebc0 0x7fed00157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:52.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.087+0000 7fecff400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed000aebc0 0x7fed00157630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33330/0 (socket says 172.21.15.100:33330) 2024-09-17T04:29:52.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.087+0000 7fecff400640 1 -- 172.21.15.100:0/3632712092 learned_addr learned my addr 172.21.15.100:0/3632712092 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:52.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.087+0000 7fecf6a00640 1 --2- 172.21.15.100:0/3632712092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed000b2750 0x7fed001f3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:52.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.087+0000 7fecffe00640 1 --2- 172.21.15.100:0/3632712092 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed0015f910 0x7fed00150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:52.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.087+0000 7fecff400640 1 -- 172.21.15.100:0/3632712092 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed0015f910 msgr2=0x7fed00150db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:52.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.087+0000 7fecff400640 1 --2- 172.21.15.100:0/3632712092 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed0015f910 0x7fed00150db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:52.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.087+0000 7fecff400640 1 -- 172.21.15.100:0/3632712092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed000b2750 msgr2=0x7fed001f3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:52.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.087+0000 7fecff400640 1 --2- 172.21.15.100:0/3632712092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed000b2750 0x7fed001f3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:52.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.087+0000 7fecff400640 1 -- 172.21.15.100:0/3632712092 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0014eba0 con 0x7fed000aebc0 2024-09-17T04:29:52.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.088+0000 7fecff400640 1 --2- 172.21.15.100:0/3632712092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed000aebc0 0x7fed00157630 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fecf000ea20 tx=0x7fecf000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:52.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.088+0000 7fecf7e00640 1 -- 172.21.15.100:0/3632712092 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf000ce40 con 0x7fed000aebc0 2024-09-17T04:29:52.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.088+0000 7fecf7e00640 1 -- 172.21.15.100:0/3632712092 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecf0004020 con 0x7fed000aebc0 2024-09-17T04:29:52.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.088+0000 7fecf7e00640 1 -- 172.21.15.100:0/3632712092 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf0004470 con 0x7fed000aebc0 2024-09-17T04:29:52.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.088+0000 7fed06200640 1 -- 172.21.15.100:0/3632712092 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed0014ee80 con 0x7fed000aebc0 2024-09-17T04:29:52.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.088+0000 7fed06200640 1 -- 172.21.15.100:0/3632712092 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed00157360 con 0x7fed000aebc0 2024-09-17T04:29:52.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.089+0000 7fed06200640 1 -- 172.21.15.100:0/3632712092 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed0015cb40 con 0x7fed000aebc0 2024-09-17T04:29:52.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.090+0000 7fecf7e00640 1 -- 172.21.15.100:0/3632712092 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fecf00035f0 con 0x7fed000aebc0 2024-09-17T04:29:52.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.091+0000 7fecf7e00640 1 --2- 172.21.15.100:0/3632712092 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fecdc078ce0 0x7fecdc07b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:52.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.091+0000 7fecf7e00640 1 -- 172.21.15.100:0/3632712092 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5029+0+0 (secure 0 0 0) 0x7fecf009b6e0 con 0x7fed000aebc0 2024-09-17T04:29:52.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.092+0000 7fecf6a00640 1 --2- 172.21.15.100:0/3632712092 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fecdc078ce0 0x7fecdc07b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:52.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.092+0000 7fecf6a00640 1 --2- 172.21.15.100:0/3632712092 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fecdc078ce0 0x7fecdc07b1f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fece8033000 tx=0x7fece8004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:52.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.096+0000 7fecf7e00640 1 -- 172.21.15.100:0/3632712092 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fecf0063540 con 0x7fed000aebc0 2024-09-17T04:29:52.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.348+0000 7fed06200640 1 -- 172.21.15.100:0/3632712092 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) -- 0x7fed0015f570 con 0x7fed000aebc0 2024-09-17T04:29:52.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.945+0000 7fecf7e00640 1 -- 172.21.15.100:0/3632712092 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v324) ==== 152+0+0 (secure 0 0 0) 0x7fecf0067f70 con 0x7fed000aebc0 2024-09-17T04:29:52.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.948+0000 7fed06200640 1 -- 172.21.15.100:0/3632712092 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fecdc078ce0 msgr2=0x7fecdc07b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:52.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.948+0000 7fed06200640 1 --2- 172.21.15.100:0/3632712092 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fecdc078ce0 0x7fecdc07b1f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fece8033000 tx=0x7fece8004730 comp rx=0 tx=0).stop 2024-09-17T04:29:52.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.948+0000 7fed06200640 1 -- 172.21.15.100:0/3632712092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed000aebc0 msgr2=0x7fed00157630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:52.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.948+0000 7fed06200640 1 --2- 172.21.15.100:0/3632712092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed000aebc0 0x7fed00157630 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fecf000ea20 tx=0x7fecf000eee0 comp rx=0 tx=0).stop 2024-09-17T04:29:52.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.948+0000 7fed06200640 1 -- 172.21.15.100:0/3632712092 shutdown_connections 2024-09-17T04:29:52.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.948+0000 7fed06200640 1 --2- 172.21.15.100:0/3632712092 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fecdc078ce0 0x7fecdc07b1f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:52.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.948+0000 7fed06200640 1 --2- 172.21.15.100:0/3632712092 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed0015f910 0x7fed00150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:52.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.948+0000 7fed06200640 1 --2- 172.21.15.100:0/3632712092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed000b2750 0x7fed001f3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:52.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.948+0000 7fed06200640 1 --2- 172.21.15.100:0/3632712092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed000aebc0 0x7fed00157630 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:52.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.948+0000 7fed06200640 1 -- 172.21.15.100:0/3632712092 >> 172.21.15.100:0/3632712092 conn(0x7fed0006d730 msgr2=0x7fed000702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:52.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.948+0000 7fed06200640 1 -- 172.21.15.100:0/3632712092 shutdown_connections 2024-09-17T04:29:52.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:52.948+0000 7fed06200640 1 -- 172.21.15.100:0/3632712092 wait complete. 2024-09-17T04:29:52.973 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T04:29:53.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.195+0000 7f2b5ea00640 1 -- 172.21.15.100:0/3441901718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b5815d760 msgr2=0x7f2b581535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:53.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.195+0000 7f2b5ea00640 1 --2- 172.21.15.100:0/3441901718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b5815d760 0x7f2b581535c0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f2b580b3100 tx=0x7f2b4802aa30 comp rx=0 tx=0).stop 2024-09-17T04:29:53.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.195+0000 7f2b5ea00640 1 -- 172.21.15.100:0/3441901718 shutdown_connections 2024-09-17T04:29:53.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.195+0000 7f2b5ea00640 1 --2- 172.21.15.100:0/3441901718 >> v2:172.21.15.184:3300/0 conn(0x7f2b580b3550 0x7f2b580b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:53.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.195+0000 7f2b5ea00640 1 --2- 172.21.15.100:0/3441901718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b5815d760 0x7f2b581535c0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:53.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.195+0000 7f2b5ea00640 1 -- 172.21.15.100:0/3441901718 >> 172.21.15.100:0/3441901718 conn(0x7f2b5806d730 msgr2=0x7f2b580b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:53.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.195+0000 7f2b5ea00640 1 -- 172.21.15.100:0/3441901718 shutdown_connections 2024-09-17T04:29:53.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.195+0000 7f2b5ea00640 1 -- 172.21.15.100:0/3441901718 wait complete. 2024-09-17T04:29:53.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.198+0000 7f2b5ea00640 1 Processor -- start 2024-09-17T04:29:53.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.198+0000 7f2b5ea00640 1 -- start start 2024-09-17T04:29:53.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.199+0000 7f2b5ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b580ad0e0 0x7f2b581f7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:53.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.199+0000 7f2b5ea00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b580b3550 0x7f2b58151450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:53.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.199+0000 7f2b5ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b5815d760 0x7f2b5809d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:53.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.199+0000 7f2b5ea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2b580b4da0 con 0x7f2b580ad0e0 2024-09-17T04:29:53.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.199+0000 7f2b5ea00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2b580b4c20 con 0x7f2b5815d760 2024-09-17T04:29:53.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.199+0000 7f2b5ea00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2b580b3ea0 con 0x7f2b580b3550 2024-09-17T04:29:53.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.199+0000 7f2b57e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b580ad0e0 0x7f2b581f7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:53.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.200+0000 7f2b57e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b580ad0e0 0x7f2b581f7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33338/0 (socket says 172.21.15.100:33338) 2024-09-17T04:29:53.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.200+0000 7f2b57e00640 1 -- 172.21.15.100:0/1032770459 learned_addr learned my addr 172.21.15.100:0/1032770459 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:53.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.200+0000 7f2b57400640 1 --2- 172.21.15.100:0/1032770459 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b580b3550 0x7f2b58151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:53.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.200+0000 7f2b5ca00640 1 --2- 172.21.15.100:0/1032770459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b5815d760 0x7f2b5809d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:53.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.200+0000 7f2b57e00640 1 -- 172.21.15.100:0/1032770459 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b580b3550 msgr2=0x7f2b58151450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:53.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.200+0000 7f2b57e00640 1 --2- 172.21.15.100:0/1032770459 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b580b3550 0x7f2b58151450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:53.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.200+0000 7f2b57e00640 1 -- 172.21.15.100:0/1032770459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b5815d760 msgr2=0x7f2b5809d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:53.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.200+0000 7f2b57e00640 1 --2- 172.21.15.100:0/1032770459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b5815d760 0x7f2b5809d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:53.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.200+0000 7f2b57e00640 1 -- 172.21.15.100:0/1032770459 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b5815a6f0 con 0x7f2b580ad0e0 2024-09-17T04:29:53.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.200+0000 7f2b57e00640 1 --2- 172.21.15.100:0/1032770459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b580ad0e0 0x7f2b581f7840 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f2b48009d60 tx=0x7f2b48003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:53.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.201+0000 7f2b4fe00640 1 -- 172.21.15.100:0/1032770459 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48033050 con 0x7f2b580ad0e0 2024-09-17T04:29:53.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.201+0000 7f2b4fe00640 1 -- 172.21.15.100:0/1032770459 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b480044f0 con 0x7f2b580ad0e0 2024-09-17T04:29:53.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.201+0000 7f2b5ea00640 1 -- 172.21.15.100:0/1032770459 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b5815c040 con 0x7f2b580ad0e0 2024-09-17T04:29:53.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.201+0000 7f2b4fe00640 1 -- 172.21.15.100:0/1032770459 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48038450 con 0x7f2b580ad0e0 2024-09-17T04:29:53.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.201+0000 7f2b5ea00640 1 -- 172.21.15.100:0/1032770459 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b5809a2d0 con 0x7f2b580ad0e0 2024-09-17T04:29:53.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.202+0000 7f2b5ea00640 1 -- 172.21.15.100:0/1032770459 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b580b4360 con 0x7f2b580ad0e0 2024-09-17T04:29:53.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.203+0000 7f2b4fe00640 1 -- 172.21.15.100:0/1032770459 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f2b48036070 con 0x7f2b580ad0e0 2024-09-17T04:29:53.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.204+0000 7f2b4fe00640 1 --2- 172.21.15.100:0/1032770459 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2b38078cb0 0x7f2b3807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:53.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.204+0000 7f2b4fe00640 1 -- 172.21.15.100:0/1032770459 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5029+0+0 (secure 0 0 0) 0x7f2b480bc3b0 con 0x7f2b580ad0e0 2024-09-17T04:29:53.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.204+0000 7f2b57400640 1 --2- 172.21.15.100:0/1032770459 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2b38078cb0 0x7f2b3807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:53.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.205+0000 7f2b57400640 1 --2- 172.21.15.100:0/1032770459 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2b38078cb0 0x7f2b3807b1c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2b5809cd70 tx=0x7f2b44009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:53.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.208+0000 7f2b4fe00640 1 -- 172.21.15.100:0/1032770459 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f2b48084750 con 0x7f2b580ad0e0 2024-09-17T04:29:53.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.390+0000 7f2b5ea00640 1 -- 172.21.15.100:0/1032770459 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f2b580afe80 con 0x7f2b580ad0e0 2024-09-17T04:29:53.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.391+0000 7f2b4fe00640 1 -- 172.21.15.100:0/1032770459 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v41) ==== 59+0+2294 (secure 0 0 0) 0x7f2b48084050 con 0x7f2b580ad0e0 2024-09-17T04:29:53.391 INFO:teuthology.orchestra.run.smithi100.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T04:29:53.391 INFO:teuthology.orchestra.run.smithi100.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T04:29:53.391 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced client_mount_timeout 30 2024-09-17T04:29:53.391 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_client 20/20 2024-09-17T04:29:53.391 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_mgr 20/20 2024-09-17T04:29:53.392 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_ms 1/1 2024-09-17T04:29:53.392 INFO:teuthology.orchestra.run.smithi100.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T04:29:53.392 INFO:teuthology.orchestra.run.smithi100.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T04:29:53.392 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3938.598654 2024-09-17T04:29:53.392 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4008.640154 2024-09-17T04:29:53.392 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3706.588546 2024-09-17T04:29:53.392 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 6299.338665 2024-09-17T04:29:53.392 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3633.876699 2024-09-17T04:29:53.392 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3674.821841 2024-09-17T04:29:53.392 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3789.849984 2024-09-17T04:29:53.392 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3467.457298 2024-09-17T04:29:53.392 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_mds 20/20 2024-09-17T04:29:53.392 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T04:29:53.393 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_ms 2/2 2024-09-17T04:29:53.393 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_debug_frag true 2024-09-17T04:29:53.393 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_debug_scatterstat true 2024-09-17T04:29:53.393 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T04:29:53.393 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_verify_scatter true 2024-09-17T04:29:53.393 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T04:29:53.393 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T04:29:53.393 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T04:29:53.393 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced client_mount_timeout 600 2024-09-17T04:29:53.393 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced debug_client 20/20 2024-09-17T04:29:53.393 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced debug_ms 1/1 2024-09-17T04:29:53.393 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T04:29:53.393 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T04:29:53.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.393+0000 7f2b5ea00640 1 -- 172.21.15.100:0/1032770459 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2b38078cb0 msgr2=0x7f2b3807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:53.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.393+0000 7f2b5ea00640 1 --2- 172.21.15.100:0/1032770459 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2b38078cb0 0x7f2b3807b1c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2b5809cd70 tx=0x7f2b44009040 comp rx=0 tx=0).stop 2024-09-17T04:29:53.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.393+0000 7f2b5ea00640 1 -- 172.21.15.100:0/1032770459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b580ad0e0 msgr2=0x7f2b581f7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:53.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.393+0000 7f2b5ea00640 1 --2- 172.21.15.100:0/1032770459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b580ad0e0 0x7f2b581f7840 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f2b48009d60 tx=0x7f2b48003f10 comp rx=0 tx=0).stop 2024-09-17T04:29:53.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.393+0000 7f2b5ea00640 1 -- 172.21.15.100:0/1032770459 shutdown_connections 2024-09-17T04:29:53.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.393+0000 7f2b5ea00640 1 --2- 172.21.15.100:0/1032770459 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2b38078cb0 0x7f2b3807b1c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:53.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.393+0000 7f2b5ea00640 1 --2- 172.21.15.100:0/1032770459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b5815d760 0x7f2b5809d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:53.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.393+0000 7f2b5ea00640 1 --2- 172.21.15.100:0/1032770459 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b580b3550 0x7f2b58151450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:53.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.393+0000 7f2b5ea00640 1 --2- 172.21.15.100:0/1032770459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b580ad0e0 0x7f2b581f7840 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:53.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.393+0000 7f2b5ea00640 1 -- 172.21.15.100:0/1032770459 >> 172.21.15.100:0/1032770459 conn(0x7f2b5806d730 msgr2=0x7f2b5815e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:53.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.393+0000 7f2b5ea00640 1 -- 172.21.15.100:0/1032770459 shutdown_connections 2024-09-17T04:29:53.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.393+0000 7f2b5ea00640 1 -- 172.21.15.100:0/1032770459 wait complete. 2024-09-17T04:29:53.415 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T04:29:53.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.638+0000 7ff14ba00640 1 -- 172.21.15.100:0/1316413130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff144145550 msgr2=0x7ff14406fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:53.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.638+0000 7ff14ba00640 1 --2- 172.21.15.100:0/1316413130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff144145550 0x7ff14406fb40 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7ff144160280 tx=0x7ff13402a5f0 comp rx=0 tx=0).stop 2024-09-17T04:29:53.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.639+0000 7ff14ba00640 1 -- 172.21.15.100:0/1316413130 shutdown_connections 2024-09-17T04:29:53.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.639+0000 7ff14ba00640 1 --2- 172.21.15.100:0/1316413130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff144073a80 0x7ff144160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:53.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.639+0000 7ff14ba00640 1 --2- 172.21.15.100:0/1316413130 >> v2:172.21.15.184:3300/0 conn(0x7ff144145a40 0x7ff144073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:53.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.639+0000 7ff14ba00640 1 --2- 172.21.15.100:0/1316413130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff144145550 0x7ff14406fb40 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:53.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.639+0000 7ff14ba00640 1 -- 172.21.15.100:0/1316413130 >> 172.21.15.100:0/1316413130 conn(0x7ff14406d1c0 msgr2=0x7ff14406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:53.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.639+0000 7ff14ba00640 1 -- 172.21.15.100:0/1316413130 shutdown_connections 2024-09-17T04:29:53.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.639+0000 7ff14ba00640 1 -- 172.21.15.100:0/1316413130 wait complete. 2024-09-17T04:29:53.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.641+0000 7ff14ba00640 1 Processor -- start 2024-09-17T04:29:53.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.641+0000 7ff14ba00640 1 -- start start 2024-09-17T04:29:53.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.642+0000 7ff14ba00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff144073a80 0x7ff14415fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:53.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.642+0000 7ff14ba00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff144145550 0x7ff1441f7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:53.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.642+0000 7ff14ba00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff144145a40 0x7ff1441f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:53.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.642+0000 7ff14ba00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff144160c30 con 0x7ff144145550 2024-09-17T04:29:53.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.642+0000 7ff14ba00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff144160ab0 con 0x7ff144145a40 2024-09-17T04:29:53.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.642+0000 7ff14ba00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff144160db0 con 0x7ff144073a80 2024-09-17T04:29:53.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.643+0000 7ff13be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff144145550 0x7ff1441f7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:53.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.643+0000 7ff13be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff144145550 0x7ff1441f7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33362/0 (socket says 172.21.15.100:33362) 2024-09-17T04:29:53.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.643+0000 7ff13be00640 1 -- 172.21.15.100:0/2080065965 learned_addr learned my addr 172.21.15.100:0/2080065965 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:53.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.643+0000 7ff13be00640 1 -- 172.21.15.100:0/2080065965 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff144073a80 msgr2=0x7ff14415fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:53.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.643+0000 7ff13be00640 1 --2- 172.21.15.100:0/2080065965 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff144073a80 0x7ff14415fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:53.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.643+0000 7ff13be00640 1 -- 172.21.15.100:0/2080065965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff144145a40 msgr2=0x7ff1441f76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:53.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.643+0000 7ff13be00640 1 --2- 172.21.15.100:0/2080065965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff144145a40 0x7ff1441f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:53.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.643+0000 7ff13be00640 1 -- 172.21.15.100:0/2080065965 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1441f7f30 con 0x7ff144145550 2024-09-17T04:29:53.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.643+0000 7ff13be00640 1 --2- 172.21.15.100:0/2080065965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff144145550 0x7ff1441f7160 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00d700 tx=0x7ff12c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:53.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.643+0000 7ff141600640 1 -- 172.21.15.100:0/2080065965 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c017070 con 0x7ff144145550 2024-09-17T04:29:53.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.644+0000 7ff141600640 1 -- 172.21.15.100:0/2080065965 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff12c004310 con 0x7ff144145550 2024-09-17T04:29:53.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.644+0000 7ff141600640 1 -- 172.21.15.100:0/2080065965 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c009a80 con 0x7ff144145550 2024-09-17T04:29:53.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.644+0000 7ff14ba00640 1 -- 172.21.15.100:0/2080065965 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff144136830 con 0x7ff144145550 2024-09-17T04:29:53.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.644+0000 7ff14ba00640 1 -- 172.21.15.100:0/2080065965 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff144136df0 con 0x7ff144145550 2024-09-17T04:29:53.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.645+0000 7ff14ba00640 1 -- 172.21.15.100:0/2080065965 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff14406fb40 con 0x7ff144145550 2024-09-17T04:29:53.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.646+0000 7ff141600640 1 -- 172.21.15.100:0/2080065965 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7ff12c009c20 con 0x7ff144145550 2024-09-17T04:29:53.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.647+0000 7ff141600640 1 --2- 172.21.15.100:0/2080065965 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff120078cb0 0x7ff12007b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:53.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.647+0000 7ff141600640 1 -- 172.21.15.100:0/2080065965 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5029+0+0 (secure 0 0 0) 0x7ff12c09adb0 con 0x7ff144145550 2024-09-17T04:29:53.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.647+0000 7ff149000640 1 --2- 172.21.15.100:0/2080065965 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff120078cb0 0x7ff12007b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:53.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.648+0000 7ff149000640 1 --2- 172.21.15.100:0/2080065965 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff120078cb0 0x7ff12007b1c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff134033000 tx=0x7ff1340046c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:53.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.651+0000 7ff141600640 1 -- 172.21.15.100:0/2080065965 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff12c063270 con 0x7ff144145550 2024-09-17T04:29:53.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.834+0000 7ff14ba00640 1 -- 172.21.15.100:0/2080065965 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff14413f790 con 0x7ff144145550 2024-09-17T04:29:53.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.839+0000 7ff141600640 1 -- 172.21.15.100:0/2080065965 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v42)=0 v42) ==== 122+0+0 (secure 0 0 0) 0x7ff12c062b70 con 0x7ff144145550 2024-09-17T04:29:53.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.841+0000 7ff14ba00640 1 -- 172.21.15.100:0/2080065965 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff120078cb0 msgr2=0x7ff12007b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:53.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.841+0000 7ff14ba00640 1 --2- 172.21.15.100:0/2080065965 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff120078cb0 0x7ff12007b1c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff134033000 tx=0x7ff1340046c0 comp rx=0 tx=0).stop 2024-09-17T04:29:53.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.841+0000 7ff14ba00640 1 -- 172.21.15.100:0/2080065965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff144145550 msgr2=0x7ff1441f7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:53.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.841+0000 7ff14ba00640 1 --2- 172.21.15.100:0/2080065965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff144145550 0x7ff1441f7160 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00d700 tx=0x7ff12c00dbc0 comp rx=0 tx=0).stop 2024-09-17T04:29:53.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.841+0000 7ff14ba00640 1 -- 172.21.15.100:0/2080065965 shutdown_connections 2024-09-17T04:29:53.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.841+0000 7ff14ba00640 1 --2- 172.21.15.100:0/2080065965 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff120078cb0 0x7ff12007b1c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:53.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.841+0000 7ff14ba00640 1 --2- 172.21.15.100:0/2080065965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff144145a40 0x7ff1441f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:53.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.841+0000 7ff14ba00640 1 --2- 172.21.15.100:0/2080065965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff144145550 0x7ff1441f7160 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:53.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.841+0000 7ff14ba00640 1 --2- 172.21.15.100:0/2080065965 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff144073a80 0x7ff14415fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:53.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.841+0000 7ff14ba00640 1 -- 172.21.15.100:0/2080065965 >> 172.21.15.100:0/2080065965 conn(0x7ff14406d1c0 msgr2=0x7ff14413e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:53.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.841+0000 7ff14ba00640 1 -- 172.21.15.100:0/2080065965 shutdown_connections 2024-09-17T04:29:53.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:53.841+0000 7ff14ba00640 1 -- 172.21.15.100:0/2080065965 wait complete. 2024-09-17T04:29:53.866 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T04:29:53.866 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:29:53.961 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:29:54.063 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T04:29:54.063 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:29:54.063 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:29:54.099 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:29:54.099 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:29:54.100 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:29:54.137 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:29:54.137 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:29:54.137 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:29:54.175 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:29:54.175 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:29:54.175 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:29:54.213 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:29:54.213 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T04:29:54.213 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:29:54.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.433+0000 7f61c1000640 1 -- 172.21.15.100:0/4231748262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61bc0b2f60 msgr2=0x7f61bc15a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:54.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.433+0000 7f61c1000640 1 --2- 172.21.15.100:0/4231748262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61bc0b2f60 0x7f61bc15a0f0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f61bc051590 tx=0x7f61a402a560 comp rx=0 tx=0).stop 2024-09-17T04:29:54.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.433+0000 7f61c1000640 1 -- 172.21.15.100:0/4231748262 shutdown_connections 2024-09-17T04:29:54.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.433+0000 7f61c1000640 1 --2- 172.21.15.100:0/4231748262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61bc0b2f60 0x7f61bc15a0f0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:54.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.433+0000 7f61c1000640 1 --2- 172.21.15.100:0/4231748262 >> v2:172.21.15.184:3300/0 conn(0x7f61bc0ad9c0 0x7f61bc0af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:54.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.433+0000 7f61c1000640 1 -- 172.21.15.100:0/4231748262 >> 172.21.15.100:0/4231748262 conn(0x7f61bc0b1e40 msgr2=0x7f61bc0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:54.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.433+0000 7f61c1000640 1 -- 172.21.15.100:0/4231748262 shutdown_connections 2024-09-17T04:29:54.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.433+0000 7f61c1000640 1 -- 172.21.15.100:0/4231748262 wait complete. 2024-09-17T04:29:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.435+0000 7f61c1000640 1 Processor -- start 2024-09-17T04:29:54.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.435+0000 7f61c1000640 1 -- start start 2024-09-17T04:29:54.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.437+0000 7f61c1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61bc0ad9c0 0x7f61bc146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:54.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.437+0000 7f61c1000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61bc0b2f60 0x7f61bc14a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:54.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.437+0000 7f61c1000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61bc15d670 0x7f61bc1579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:54.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.437+0000 7f61c1000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc06f290 con 0x7f61bc0ad9c0 2024-09-17T04:29:54.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.437+0000 7f61c1000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc070d50 con 0x7f61bc0b2f60 2024-09-17T04:29:54.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.437+0000 7f61c1000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f61bc06f410 con 0x7f61bc15d670 2024-09-17T04:29:54.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.437+0000 7f61bb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61bc0ad9c0 0x7f61bc146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:54.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.437+0000 7f61b2a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61bc0b2f60 0x7f61bc14a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:54.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.437+0000 7f61bb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61bc0ad9c0 0x7f61bc146ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33376/0 (socket says 172.21.15.100:33376) 2024-09-17T04:29:54.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.437+0000 7f61bb400640 1 -- 172.21.15.100:0/3581024832 learned_addr learned my addr 172.21.15.100:0/3581024832 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:54.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.437+0000 7f61bbe00640 1 --2- 172.21.15.100:0/3581024832 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61bc15d670 0x7f61bc1579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:54.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.437+0000 7f61b2a00640 1 -- 172.21.15.100:0/3581024832 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61bc15d670 msgr2=0x7f61bc1579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:54.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.437+0000 7f61b2a00640 1 --2- 172.21.15.100:0/3581024832 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61bc15d670 0x7f61bc1579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:54.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.437+0000 7f61b2a00640 1 -- 172.21.15.100:0/3581024832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61bc0ad9c0 msgr2=0x7f61bc146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:54.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.437+0000 7f61b2a00640 1 --2- 172.21.15.100:0/3581024832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61bc0ad9c0 0x7f61bc146ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:54.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.437+0000 7f61b2a00640 1 -- 172.21.15.100:0/3581024832 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61bc156900 con 0x7f61bc0b2f60 2024-09-17T04:29:54.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.437+0000 7f61bbe00640 1 --2- 172.21.15.100:0/3581024832 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61bc15d670 0x7f61bc1579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:29:54.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.438+0000 7f61b2a00640 1 --2- 172.21.15.100:0/3581024832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61bc0b2f60 0x7f61bc14a7f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f61a4031000 tx=0x7f61a40038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:54.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.438+0000 7f61b3e00640 1 -- 172.21.15.100:0/3581024832 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a403e030 con 0x7f61bc0b2f60 2024-09-17T04:29:54.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.438+0000 7f61b3e00640 1 -- 172.21.15.100:0/3581024832 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61a4003dd0 con 0x7f61bc0b2f60 2024-09-17T04:29:54.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.438+0000 7f61b3e00640 1 -- 172.21.15.100:0/3581024832 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a4035040 con 0x7f61bc0b2f60 2024-09-17T04:29:54.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.438+0000 7f61c1000640 1 -- 172.21.15.100:0/3581024832 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61bc156be0 con 0x7f61bc0b2f60 2024-09-17T04:29:54.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.438+0000 7f61c1000640 1 -- 172.21.15.100:0/3581024832 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61bc1571a0 con 0x7f61bc0b2f60 2024-09-17T04:29:54.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.439+0000 7f61c1000640 1 -- 172.21.15.100:0/3581024832 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61bc06f8a0 con 0x7f61bc0b2f60 2024-09-17T04:29:54.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.441+0000 7f61b3e00640 1 -- 172.21.15.100:0/3581024832 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f61a4041030 con 0x7f61bc0b2f60 2024-09-17T04:29:54.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.441+0000 7f61b3e00640 1 --2- 172.21.15.100:0/3581024832 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6198078ce0 0x7f619807b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:54.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.442+0000 7f61b3e00640 1 -- 172.21.15.100:0/3581024832 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5029+0+0 (secure 0 0 0) 0x7f61a40bbb00 con 0x7f61bc0b2f60 2024-09-17T04:29:54.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.442+0000 7f61bb400640 1 --2- 172.21.15.100:0/3581024832 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6198078ce0 0x7f619807b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:54.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.442+0000 7f61bb400640 1 --2- 172.21.15.100:0/3581024832 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6198078ce0 0x7f619807b1f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f61ac009bf0 tx=0x7f61ac005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:54.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.445+0000 7f61b3e00640 1 -- 172.21.15.100:0/3581024832 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f61a4083960 con 0x7f61bc0b2f60 2024-09-17T04:29:54.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.662+0000 7f61c1000640 1 -- 172.21.15.100:0/3581024832 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61bc156690 con 0x7f61bc0b2f60 2024-09-17T04:29:54.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.662+0000 7f61b3e00640 1 -- 172.21.15.100:0/3581024832 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+3923 (secure 0 0 0) 0x7f61a4088390 con 0x7f61bc0b2f60 2024-09-17T04:29:54.663 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:54.663 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":65,"btime":"2024-09-17T04:29:47:917021+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5825,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3480776805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3480776805},{"type":"v1","addr":"172.21.15.184:6835","nonce":3480776805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":5948,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1319632725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1319632725},{"type":"v1","addr":"172.21.15.184:6801","nonce":1319632725}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/3591753470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3591753470},{"type":"v1","addr":"172.21.15.100:6833","nonce":3591753470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6882,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2070433536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2070433536},{"type":"v1","addr":"172.21.15.100:6835","nonce":2070433536}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[]} 2024-09-17T04:29:54.663 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 65 2024-09-17T04:29:54.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.665+0000 7f61c1000640 1 -- 172.21.15.100:0/3581024832 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6198078ce0 msgr2=0x7f619807b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:54.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.665+0000 7f61c1000640 1 --2- 172.21.15.100:0/3581024832 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6198078ce0 0x7f619807b1f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f61ac009bf0 tx=0x7f61ac005f90 comp rx=0 tx=0).stop 2024-09-17T04:29:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.665+0000 7f61c1000640 1 -- 172.21.15.100:0/3581024832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61bc0b2f60 msgr2=0x7f61bc14a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.665+0000 7f61c1000640 1 --2- 172.21.15.100:0/3581024832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61bc0b2f60 0x7f61bc14a7f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f61a4031000 tx=0x7f61a40038a0 comp rx=0 tx=0).stop 2024-09-17T04:29:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.665+0000 7f61c1000640 1 -- 172.21.15.100:0/3581024832 shutdown_connections 2024-09-17T04:29:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.665+0000 7f61c1000640 1 --2- 172.21.15.100:0/3581024832 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6198078ce0 0x7f619807b1f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.665+0000 7f61c1000640 1 --2- 172.21.15.100:0/3581024832 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61bc15d670 0x7f61bc1579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.665+0000 7f61c1000640 1 --2- 172.21.15.100:0/3581024832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61bc0b2f60 0x7f61bc14a7f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.665+0000 7f61c1000640 1 --2- 172.21.15.100:0/3581024832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61bc0ad9c0 0x7f61bc146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.665+0000 7f61c1000640 1 -- 172.21.15.100:0/3581024832 >> 172.21.15.100:0/3581024832 conn(0x7f61bc0b1e40 msgr2=0x7f61bc0b41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.665+0000 7f61c1000640 1 -- 172.21.15.100:0/3581024832 shutdown_connections 2024-09-17T04:29:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:54.665+0000 7f61c1000640 1 -- 172.21.15.100:0/3581024832 wait complete. 2024-09-17T04:29:54.688 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T04:29:54.688 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T04:29:54.688 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T04:29:54.689 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:29:54.689 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T04:29:54.689 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T04:29:54.689 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T04:29:54.689 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:29:54.689 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T04:29:54.689 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T04:29:54.690 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T04:29:54.690 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:29:54.690 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T04:29:54.690 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T04:29:54.690 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T04:29:54.690 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:29:54.713 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T04:29:54.713 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T04:29:54.714 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T04:29:54.714 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T04:29:54.723 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T04:29:54.723 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T04:29:54.724 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T04:29:54.724 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T04:29:54.759 INFO:tasks.ceph.mds.a:Started 2024-09-17T04:29:54.759 DEBUG:teuthology.parallel:result is None 2024-09-17T04:29:54.762 INFO:tasks.ceph.mds.c:Started 2024-09-17T04:29:54.762 DEBUG:teuthology.parallel:result is None 2024-09-17T04:29:54.768 INFO:tasks.ceph.mds.d:Started 2024-09-17T04:29:54.768 DEBUG:teuthology.parallel:result is None 2024-09-17T04:29:54.770 INFO:tasks.ceph.mds.b:Started 2024-09-17T04:29:54.771 DEBUG:teuthology.parallel:result is None 2024-09-17T04:29:54.771 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T04:29:54.926 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-09-17T04:29:54.956 INFO:tasks.ceph.mds.a.smithi100.stdout:starting mds.a at 2024-09-17T04:29:54.963 INFO:tasks.ceph.mds.c.smithi100.stdout:starting mds.c at 2024-09-17T04:29:54.965 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-09-17T04:29:54.966 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-09-17T04:29:55.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.003+0000 7f7491e00640 1 -- 172.21.15.100:0/2304635654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f748c070cc0 msgr2=0x7f748c156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:55.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.003+0000 7f7491e00640 1 --2- 172.21.15.100:0/2304635654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f748c070cc0 0x7f748c156e40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7478009860 tx=0x7f747802a4f0 comp rx=0 tx=0).stop 2024-09-17T04:29:55.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.004+0000 7f7491e00640 1 -- 172.21.15.100:0/2304635654 shutdown_connections 2024-09-17T04:29:55.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.004+0000 7f7491e00640 1 --2- 172.21.15.100:0/2304635654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f748c070cc0 0x7f748c156e40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:55.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.004+0000 7f7491e00640 1 -- 172.21.15.100:0/2304635654 >> 172.21.15.100:0/2304635654 conn(0x7f748c06d3d0 msgr2=0x7f748c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:55.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.004+0000 7f7491e00640 1 -- 172.21.15.100:0/2304635654 shutdown_connections 2024-09-17T04:29:55.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.004+0000 7f7491e00640 1 -- 172.21.15.100:0/2304635654 wait complete. 2024-09-17T04:29:55.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.006+0000 7f7491e00640 1 Processor -- start 2024-09-17T04:29:55.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.006+0000 7f7491e00640 1 -- start start 2024-09-17T04:29:55.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.008+0000 7f7491e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f748c0cfc80 0x7f748c0b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:55.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.008+0000 7f7491e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f748c15ce30 0x7f748c0b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:55.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.008+0000 7f7491e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f748c159210 0x7f748c159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:55.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.008+0000 7f7491e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f748c0cd100 con 0x7f748c15ce30 2024-09-17T04:29:55.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.008+0000 7f7491e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f748c0ccf80 con 0x7f748c159210 2024-09-17T04:29:55.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.008+0000 7f7491e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f748c0cd280 con 0x7f748c0cfc80 2024-09-17T04:29:55.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.008+0000 7f748a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f748c15ce30 0x7f748c0b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:55.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.008+0000 7f748a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f748c15ce30 0x7f748c0b3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37602/0 (socket says 172.21.15.100:37602) 2024-09-17T04:29:55.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.008+0000 7f748a000640 1 -- 172.21.15.100:0/1178434419 learned_addr learned my addr 172.21.15.100:0/1178434419 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:55.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.008+0000 7f748aa00640 1 --2- 172.21.15.100:0/1178434419 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f748c0cfc80 0x7f748c0b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:55.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.008+0000 7f748b400640 1 --2- 172.21.15.100:0/1178434419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f748c159210 0x7f748c159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:55.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.008+0000 7f748a000640 1 -- 172.21.15.100:0/1178434419 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f748c0cfc80 msgr2=0x7f748c0b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:55.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.008+0000 7f748a000640 1 --2- 172.21.15.100:0/1178434419 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f748c0cfc80 0x7f748c0b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:55.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.008+0000 7f748a000640 1 -- 172.21.15.100:0/1178434419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f748c159210 msgr2=0x7f748c159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:55.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.008+0000 7f748a000640 1 --2- 172.21.15.100:0/1178434419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f748c159210 0x7f748c159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:55.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.008+0000 7f748a000640 1 -- 172.21.15.100:0/1178434419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f748c0b02d0 con 0x7f748c15ce30 2024-09-17T04:29:55.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.009+0000 7f748b400640 1 --2- 172.21.15.100:0/1178434419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f748c159210 0x7f748c159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:29:55.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.009+0000 7f748a000640 1 --2- 172.21.15.100:0/1178434419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f748c15ce30 0x7f748c0b3990 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f747802aae0 tx=0x7f7478031c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:55.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.009+0000 7f7483400640 1 -- 172.21.15.100:0/1178434419 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747803f070 con 0x7f748c15ce30 2024-09-17T04:29:55.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.009+0000 7f7483400640 1 -- 172.21.15.100:0/1178434419 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7478004310 con 0x7f748c15ce30 2024-09-17T04:29:55.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.009+0000 7f7483400640 1 -- 172.21.15.100:0/1178434419 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747803b880 con 0x7f748c15ce30 2024-09-17T04:29:55.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.009+0000 7f7491e00640 1 -- 172.21.15.100:0/1178434419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f748c0b05b0 con 0x7f748c15ce30 2024-09-17T04:29:55.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.009+0000 7f7491e00640 1 -- 172.21.15.100:0/1178434419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f748c0b0b70 con 0x7f748c15ce30 2024-09-17T04:29:55.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.011+0000 7f7491e00640 1 -- 172.21.15.100:0/1178434419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f748c0cd7b0 con 0x7f748c15ce30 2024-09-17T04:29:55.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.012+0000 7f7483400640 1 -- 172.21.15.100:0/1178434419 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f747803ba20 con 0x7f748c15ce30 2024-09-17T04:29:55.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.013+0000 7f7483400640 1 --2- 172.21.15.100:0/1178434419 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f746c078ce0 0x7f746c07b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:55.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.013+0000 7f7483400640 1 -- 172.21.15.100:0/1178434419 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5029+0+0 (secure 0 0 0) 0x7f74780bb9d0 con 0x7f748c15ce30 2024-09-17T04:29:55.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.013+0000 7f748aa00640 1 --2- 172.21.15.100:0/1178434419 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f746c078ce0 0x7f746c07b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:55.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.014+0000 7f748aa00640 1 --2- 172.21.15.100:0/1178434419 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f746c078ce0 0x7f746c07b1f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f747c000f60 tx=0x7f747c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:55.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.017+0000 7f7483400640 1 -- 172.21.15.100:0/1178434419 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f7478083830 con 0x7f748c15ce30 2024-09-17T04:29:55.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.212+0000 7f7491e00640 1 -- 172.21.15.100:0/1178434419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f748c0b0060 con 0x7f748c15ce30 2024-09-17T04:29:55.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.967+0000 7f7483400640 1 -- 172.21.15.100:0/1178434419 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v99) ==== 97+0+0 (secure 0 0 0) 0x7f7478088260 con 0x7f748c15ce30 2024-09-17T04:29:55.968 INFO:teuthology.orchestra.run.smithi100.stderr: removed all blocklist entries 2024-09-17T04:29:55.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.970+0000 7f7491e00640 1 -- 172.21.15.100:0/1178434419 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f746c078ce0 msgr2=0x7f746c07b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:55.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.970+0000 7f7491e00640 1 --2- 172.21.15.100:0/1178434419 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f746c078ce0 0x7f746c07b1f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f747c000f60 tx=0x7f747c009040 comp rx=0 tx=0).stop 2024-09-17T04:29:55.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.971+0000 7f7491e00640 1 -- 172.21.15.100:0/1178434419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f748c15ce30 msgr2=0x7f748c0b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:55.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.971+0000 7f7491e00640 1 --2- 172.21.15.100:0/1178434419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f748c15ce30 0x7f748c0b3990 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f747802aae0 tx=0x7f7478031c20 comp rx=0 tx=0).stop 2024-09-17T04:29:55.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.971+0000 7f7491e00640 1 -- 172.21.15.100:0/1178434419 shutdown_connections 2024-09-17T04:29:55.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.971+0000 7f7491e00640 1 --2- 172.21.15.100:0/1178434419 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f746c078ce0 0x7f746c07b1f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:55.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.971+0000 7f7491e00640 1 --2- 172.21.15.100:0/1178434419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f748c159210 0x7f748c159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:55.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.971+0000 7f7491e00640 1 --2- 172.21.15.100:0/1178434419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f748c15ce30 0x7f748c0b3990 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:55.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.971+0000 7f7491e00640 1 --2- 172.21.15.100:0/1178434419 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f748c0cfc80 0x7f748c0b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:55.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.971+0000 7f7491e00640 1 -- 172.21.15.100:0/1178434419 >> 172.21.15.100:0/1178434419 conn(0x7f748c06d3d0 msgr2=0x7f748c15f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:55.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.971+0000 7f7491e00640 1 -- 172.21.15.100:0/1178434419 shutdown_connections 2024-09-17T04:29:55.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:55.971+0000 7f7491e00640 1 -- 172.21.15.100:0/1178434419 wait complete. 2024-09-17T04:29:55.996 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T04:29:56.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.212+0000 7fe179c00640 1 -- 172.21.15.100:0/2592822849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe17415d760 msgr2=0x7fe1741535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:56.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.212+0000 7fe179c00640 1 --2- 172.21.15.100:0/2592822849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe17415d760 0x7fe1741535c0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fe164005fd0 tx=0x7fe16402a540 comp rx=0 tx=0).stop 2024-09-17T04:29:56.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.213+0000 7fe179c00640 1 -- 172.21.15.100:0/2592822849 shutdown_connections 2024-09-17T04:29:56.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.213+0000 7fe179c00640 1 --2- 172.21.15.100:0/2592822849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1740ad0e0 0x7fe1740ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:56.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.213+0000 7fe179c00640 1 --2- 172.21.15.100:0/2592822849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe17415d760 0x7fe1741535c0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:56.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.213+0000 7fe179c00640 1 -- 172.21.15.100:0/2592822849 >> 172.21.15.100:0/2592822849 conn(0x7fe17406d730 msgr2=0x7fe1740b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:56.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.213+0000 7fe179c00640 1 -- 172.21.15.100:0/2592822849 shutdown_connections 2024-09-17T04:29:56.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.213+0000 7fe179c00640 1 -- 172.21.15.100:0/2592822849 wait complete. 2024-09-17T04:29:56.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.215+0000 7fe179c00640 1 Processor -- start 2024-09-17T04:29:56.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.215+0000 7fe179c00640 1 -- start start 2024-09-17T04:29:56.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.216+0000 7fe179c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1740ad0e0 0x7fe174142030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.216+0000 7fe179c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe17415d760 0x7fe17413ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.216+0000 7fe179c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe174160310 0x7fe1741f95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.216+0000 7fe179c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe17406f220 con 0x7fe17415d760 2024-09-17T04:29:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.216+0000 7fe179c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe17406f0a0 con 0x7fe1740ad0e0 2024-09-17T04:29:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.216+0000 7fe179c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe1740b4c20 con 0x7fe174160310 2024-09-17T04:29:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.217+0000 7fe172000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe17415d760 0x7fe17413ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.217+0000 7fe172000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe17415d760 0x7fe17413ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37614/0 (socket says 172.21.15.100:37614) 2024-09-17T04:29:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.217+0000 7fe172000640 1 -- 172.21.15.100:0/3607036067 learned_addr learned my addr 172.21.15.100:0/3607036067 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.217+0000 7fe172a00640 1 --2- 172.21.15.100:0/3607036067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1740ad0e0 0x7fe174142030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.217+0000 7fe173400640 1 --2- 172.21.15.100:0/3607036067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe174160310 0x7fe1741f95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.217+0000 7fe172000640 1 -- 172.21.15.100:0/3607036067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe174160310 msgr2=0x7fe1741f95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.217+0000 7fe172000640 1 --2- 172.21.15.100:0/3607036067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe174160310 0x7fe1741f95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.217+0000 7fe172000640 1 -- 172.21.15.100:0/3607036067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1740ad0e0 msgr2=0x7fe174142030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:56.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.217+0000 7fe172000640 1 --2- 172.21.15.100:0/3607036067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1740ad0e0 0x7fe174142030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:56.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.217+0000 7fe172000640 1 -- 172.21.15.100:0/3607036067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe174142930 con 0x7fe17415d760 2024-09-17T04:29:56.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.217+0000 7fe172000640 1 --2- 172.21.15.100:0/3607036067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe17415d760 0x7fe17413ffa0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe16000e790 tx=0x7fe16000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:56.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.218+0000 7fe16aa00640 1 -- 172.21.15.100:0/3607036067 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16000eea0 con 0x7fe17415d760 2024-09-17T04:29:56.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.218+0000 7fe16aa00640 1 -- 172.21.15.100:0/3607036067 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe16000ccc0 con 0x7fe17415d760 2024-09-17T04:29:56.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.218+0000 7fe16aa00640 1 -- 172.21.15.100:0/3607036067 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160004280 con 0x7fe17415d760 2024-09-17T04:29:56.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.218+0000 7fe179c00640 1 -- 172.21.15.100:0/3607036067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe174142c10 con 0x7fe17415d760 2024-09-17T04:29:56.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.218+0000 7fe179c00640 1 -- 172.21.15.100:0/3607036067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1741f90b0 con 0x7fe17415d760 2024-09-17T04:29:56.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.219+0000 7fe179c00640 1 -- 172.21.15.100:0/3607036067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1741426a0 con 0x7fe17415d760 2024-09-17T04:29:56.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.220+0000 7fe16aa00640 1 -- 172.21.15.100:0/3607036067 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fe1600035f0 con 0x7fe17415d760 2024-09-17T04:29:56.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.221+0000 7fe16aa00640 1 --2- 172.21.15.100:0/3607036067 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe154078cb0 0x7fe15407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:56.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.221+0000 7fe16aa00640 1 -- 172.21.15.100:0/3607036067 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 4857+0+0 (secure 0 0 0) 0x7fe16009b410 con 0x7fe17415d760 2024-09-17T04:29:56.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.221+0000 7fe172a00640 1 --2- 172.21.15.100:0/3607036067 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe154078cb0 0x7fe15407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:56.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.222+0000 7fe172a00640 1 --2- 172.21.15.100:0/3607036067 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe154078cb0 0x7fe15407b1c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe164033000 tx=0x7fe164004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:56.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.225+0000 7fe16aa00640 1 -- 172.21.15.100:0/3607036067 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe1600633c0 con 0x7fe17415d760 2024-09-17T04:29:56.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.480+0000 7fe179c00640 1 -- 172.21.15.100:0/3607036067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe174160fa0 con 0x7fe17415d760 2024-09-17T04:29:56.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.480+0000 7fe16aa00640 1 -- 172.21.15.100:0/3607036067 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) ==== 80+0+5304 (secure 0 0 0) 0x7fe160067df0 con 0x7fe17415d760 2024-09-17T04:29:56.481 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:56.481 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:29:56.481 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_dump": [ 2024-09-17T04:29:56.481 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:56.481 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.a", 2024-09-17T04:29:56.481 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmV8asJRAAkSgIWqGc1OeSRbs9PEOT8w==", 2024-09-17T04:29:56.481 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:56.481 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:29:56.481 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:29:56.481 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:29:56.481 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:56.482 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:56.482 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:56.482 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:56.482 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.b", 2024-09-17T04:29:56.482 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmpM7ZBRAA1Og785tJwa6NA9zBdmNNQA==", 2024-09-17T04:29:56.482 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:56.482 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:29:56.482 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:29:56.482 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:29:56.482 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:56.482 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:56.482 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:56.482 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:56.482 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.c", 2024-09-17T04:29:56.483 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmXaOGMxAAT+96g/5fAzPW8Cc2HtQUyA==", 2024-09-17T04:29:56.483 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:56.483 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:29:56.483 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:29:56.483 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:29:56.483 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:56.483 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:56.483 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:56.483 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:56.483 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.d", 2024-09-17T04:29:56.483 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmRqC9ExAA23kZ0Ks1lGN2k4GXL8ASrg==", 2024-09-17T04:29:56.483 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:56.483 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:29:56.484 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:29:56.484 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:29:56.484 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:56.484 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:56.484 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:56.484 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:56.484 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.0", 2024-09-17T04:29:56.484 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAwBOlmofFmAxAARerYkU9B2a9MM9vHa3o4LQ==", 2024-09-17T04:29:56.484 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:56.484 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:29:56.484 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:29:56.484 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:56.484 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:56.484 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:56.485 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:56.485 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.1", 2024-09-17T04:29:56.485 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAzBOlmbz6VExAAoVqSqXTc1wArKhdA6+NxnA==", 2024-09-17T04:29:56.485 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:56.485 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:29:56.485 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:29:56.485 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:56.485 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:56.485 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:56.485 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:56.485 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.2", 2024-09-17T04:29:56.485 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA2BOlmfeSAJBAAchY6XhyDRusc+zqwQ6v6wQ==", 2024-09-17T04:29:56.485 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:56.486 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:29:56.486 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:29:56.486 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:56.486 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:56.486 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:56.486 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:56.486 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.3", 2024-09-17T04:29:56.486 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA6BOlm/TQiBBAAHhbs2+RCjAenGR1frhbTmQ==", 2024-09-17T04:29:56.486 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:56.486 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:29:56.487 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:29:56.487 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:56.487 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:56.487 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:56.487 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:56.487 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.4", 2024-09-17T04:29:56.487 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBABOlm7OKvBhAALkaAKEUYN5ChwGejoSF8yA==", 2024-09-17T04:29:56.487 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:56.487 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:29:56.487 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:29:56.487 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:56.487 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:56.487 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:56.487 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:56.488 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.5", 2024-09-17T04:29:56.488 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBDBOlmQpi1FhAAbA47dg6ckK1PFoCZL0SBEQ==", 2024-09-17T04:29:56.488 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:56.488 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:29:56.488 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:29:56.488 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:56.488 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:56.488 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:56.488 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:56.488 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.6", 2024-09-17T04:29:56.488 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBGBOlmx/w0JhAAWcPvZkMYtXD/EMKrg/W/4Q==", 2024-09-17T04:29:56.488 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:56.488 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:29:56.488 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:29:56.489 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:56.489 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:56.489 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:56.489 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:56.489 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.7", 2024-09-17T04:29:56.489 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBJBOlmWHjWNBAAgdul2UKY9g5p7EVi0ZdrFw==", 2024-09-17T04:29:56.489 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:56.489 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:29:56.489 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:29:56.489 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:56.489 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:56.489 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:56.489 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:56.490 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.0", 2024-09-17T04:29:56.490 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmfYg9IBAABP1uRKGOugrA47VxwRLkjA==", 2024-09-17T04:29:56.490 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:56.490 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:29:56.490 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow r", 2024-09-17T04:29:56.490 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T04:29:56.490 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:56.490 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:56.490 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:56.490 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.1", 2024-09-17T04:29:56.490 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmByhhNRAAPGaxFCDd8ywFVwwhprrPbQ==", 2024-09-17T04:29:56.490 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:56.490 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:29:56.490 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:29:56.491 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:29:56.491 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:29:56.491 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:56.491 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:56.491 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:56.491 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.2", 2024-09-17T04:29:56.491 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmgcm0KhAAGF06VlHjBM/AGxafN6/3aA==", 2024-09-17T04:29:56.491 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:56.491 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:29:56.491 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:29:56.491 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:29:56.491 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:29:56.491 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:56.492 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:56.492 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:56.492 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.3", 2024-09-17T04:29:56.492 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtBOlm9VQ6BBAAXcahu35/uDm8Z+f+3SF0Gg==", 2024-09-17T04:29:56.492 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:56.492 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:29:56.492 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:29:56.492 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:29:56.492 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:29:56.492 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:56.492 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:56.492 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:56.492 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.admin", 2024-09-17T04:29:56.492 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAqBOlmZ+bIJBAAG3oaMNlYWNakywGDvAGbGw==", 2024-09-17T04:29:56.493 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:56.493 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:29:56.493 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:29:56.493 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:29:56.493 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:56.493 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:56.493 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:56.493 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:56.493 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mgr.x", 2024-09-17T04:29:56.493 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmfaL7GhAArqLH66RkbBsz+TNOKtPHOA==", 2024-09-17T04:29:56.493 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:56.493 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:29:56.493 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile mgr", 2024-09-17T04:29:56.493 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:56.494 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:56.494 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:29:56.494 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:56.494 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mgr.y", 2024-09-17T04:29:56.494 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmM4D1DhAA5uUWgRVOMujWqbYzOjFp9A==", 2024-09-17T04:29:56.494 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:29:56.494 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:29:56.494 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile mgr", 2024-09-17T04:29:56.494 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:29:56.494 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:56.494 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:56.494 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-17T04:29:56.494 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:29:56.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.483+0000 7fe179c00640 1 -- 172.21.15.100:0/3607036067 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe154078cb0 msgr2=0x7fe15407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:56.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.483+0000 7fe179c00640 1 --2- 172.21.15.100:0/3607036067 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe154078cb0 0x7fe15407b1c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe164033000 tx=0x7fe164004730 comp rx=0 tx=0).stop 2024-09-17T04:29:56.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.483+0000 7fe179c00640 1 -- 172.21.15.100:0/3607036067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe17415d760 msgr2=0x7fe17413ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:56.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.483+0000 7fe179c00640 1 --2- 172.21.15.100:0/3607036067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe17415d760 0x7fe17413ffa0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe16000e790 tx=0x7fe16000ec50 comp rx=0 tx=0).stop 2024-09-17T04:29:56.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.483+0000 7fe179c00640 1 -- 172.21.15.100:0/3607036067 shutdown_connections 2024-09-17T04:29:56.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.483+0000 7fe179c00640 1 --2- 172.21.15.100:0/3607036067 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe154078cb0 0x7fe15407b1c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:56.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.483+0000 7fe179c00640 1 --2- 172.21.15.100:0/3607036067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe174160310 0x7fe1741f95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:56.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.483+0000 7fe179c00640 1 --2- 172.21.15.100:0/3607036067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe17415d760 0x7fe17413ffa0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:56.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.483+0000 7fe179c00640 1 --2- 172.21.15.100:0/3607036067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1740ad0e0 0x7fe174142030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:56.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.483+0000 7fe179c00640 1 -- 172.21.15.100:0/3607036067 >> 172.21.15.100:0/3607036067 conn(0x7fe17406d730 msgr2=0x7fe1740cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:56.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.483+0000 7fe179c00640 1 -- 172.21.15.100:0/3607036067 shutdown_connections 2024-09-17T04:29:56.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.483+0000 7fe179c00640 1 -- 172.21.15.100:0/3607036067 wait complete. 2024-09-17T04:29:56.506 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T04:29:56.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.727+0000 7fc475400640 1 -- 172.21.15.100:0/203383879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc47015a0f0 msgr2=0x7fc47015d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.727+0000 7fc475400640 1 --2- 172.21.15.100:0/203383879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc47015a0f0 0x7fc47015d670 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fc468005fd0 tx=0x7fc46802a820 comp rx=0 tx=0).stop 2024-09-17T04:29:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.727+0000 7fc475400640 1 -- 172.21.15.100:0/203383879 shutdown_connections 2024-09-17T04:29:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.727+0000 7fc475400640 1 --2- 172.21.15.100:0/203383879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc47015a0f0 0x7fc47015d670 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.727+0000 7fc475400640 1 --2- 172.21.15.100:0/203383879 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4700ad9c0 0x7fc4700af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.727+0000 7fc475400640 1 -- 172.21.15.100:0/203383879 >> 172.21.15.100:0/203383879 conn(0x7fc4700b1e40 msgr2=0x7fc4700b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.727+0000 7fc475400640 1 -- 172.21.15.100:0/203383879 shutdown_connections 2024-09-17T04:29:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.727+0000 7fc475400640 1 -- 172.21.15.100:0/203383879 wait complete. 2024-09-17T04:29:56.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.729+0000 7fc475400640 1 Processor -- start 2024-09-17T04:29:56.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.729+0000 7fc475400640 1 -- start start 2024-09-17T04:29:56.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.731+0000 7fc475400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4700ad9c0 0x7fc4700ce510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:56.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.731+0000 7fc475400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4700b2f60 0x7fc47014c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:56.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.731+0000 7fc475400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc47015a0f0 0x7fc470158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:56.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.731+0000 7fc475400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc47006f410 con 0x7fc4700b2f60 2024-09-17T04:29:56.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.731+0000 7fc475400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc47006f290 con 0x7fc47015a0f0 2024-09-17T04:29:56.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.731+0000 7fc475400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc47006f590 con 0x7fc4700ad9c0 2024-09-17T04:29:56.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.731+0000 7fc46f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4700b2f60 0x7fc47014c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:56.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.731+0000 7fc46f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4700b2f60 0x7fc47014c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37634/0 (socket says 172.21.15.100:37634) 2024-09-17T04:29:56.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.731+0000 7fc46f400640 1 -- 172.21.15.100:0/4053243799 learned_addr learned my addr 172.21.15.100:0/4053243799 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:56.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.731+0000 7fc46fe00640 1 --2- 172.21.15.100:0/4053243799 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4700ad9c0 0x7fc4700ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:56.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.731+0000 7fc474a00640 1 --2- 172.21.15.100:0/4053243799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc47015a0f0 0x7fc470158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:56.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.731+0000 7fc46f400640 1 -- 172.21.15.100:0/4053243799 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4700ad9c0 msgr2=0x7fc4700ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:56.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.731+0000 7fc46f400640 1 --2- 172.21.15.100:0/4053243799 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4700ad9c0 0x7fc4700ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:56.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.732+0000 7fc46f400640 1 -- 172.21.15.100:0/4053243799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc47015a0f0 msgr2=0x7fc470158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:56.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.732+0000 7fc46f400640 1 --2- 172.21.15.100:0/4053243799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc47015a0f0 0x7fc470158f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:56.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.732+0000 7fc46f400640 1 -- 172.21.15.100:0/4053243799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4700cd470 con 0x7fc4700b2f60 2024-09-17T04:29:56.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.732+0000 7fc46f400640 1 --2- 172.21.15.100:0/4053243799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4700b2f60 0x7fc47014c0c0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00c400 tx=0x7fc45c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:56.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.732+0000 7fc467e00640 1 -- 172.21.15.100:0/4053243799 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c011790 con 0x7fc4700b2f60 2024-09-17T04:29:56.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.732+0000 7fc467e00640 1 -- 172.21.15.100:0/4053243799 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc45c017070 con 0x7fc4700b2f60 2024-09-17T04:29:56.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.732+0000 7fc467e00640 1 -- 172.21.15.100:0/4053243799 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c011cc0 con 0x7fc4700b2f60 2024-09-17T04:29:56.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.732+0000 7fc475400640 1 -- 172.21.15.100:0/4053243799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4700cd750 con 0x7fc4700b2f60 2024-09-17T04:29:56.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.732+0000 7fc475400640 1 -- 172.21.15.100:0/4053243799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4700cdd10 con 0x7fc4700b2f60 2024-09-17T04:29:56.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.733+0000 7fc475400640 1 -- 172.21.15.100:0/4053243799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc47006fab0 con 0x7fc4700b2f60 2024-09-17T04:29:56.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.734+0000 7fc467e00640 1 -- 172.21.15.100:0/4053243799 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fc45c01c460 con 0x7fc4700b2f60 2024-09-17T04:29:56.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.735+0000 7fc467e00640 1 --2- 172.21.15.100:0/4053243799 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc450078cb0 0x7fc45007b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:56.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.735+0000 7fc467e00640 1 -- 172.21.15.100:0/4053243799 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 4857+0+0 (secure 0 0 0) 0x7fc45c09a910 con 0x7fc4700b2f60 2024-09-17T04:29:56.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.736+0000 7fc46fe00640 1 --2- 172.21.15.100:0/4053243799 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc450078cb0 0x7fc45007b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:56.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.736+0000 7fc46fe00640 1 --2- 172.21.15.100:0/4053243799 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc450078cb0 0x7fc45007b1c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc46000bb50 tx=0x7fc46000b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:56.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.740+0000 7fc467e00640 1 -- 172.21.15.100:0/4053243799 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc45c0628e0 con 0x7fc4700b2f60 2024-09-17T04:29:56.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.924+0000 7fc475400640 1 -- 172.21.15.100:0/4053243799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc4700cd200 con 0x7fc4700b2f60 2024-09-17T04:29:56.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.924+0000 7fc467e00640 1 -- 172.21.15.100:0/4053243799 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v101) ==== 84+0+65 (secure 0 0 0) 0x7fc45c067310 con 0x7fc4700b2f60 2024-09-17T04:29:56.925 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:29:56.925 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-09-17T04:29:56.925 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:29:56.925 INFO:teuthology.orchestra.run.smithi100.stdout: "poolnum": 1, 2024-09-17T04:29:56.925 INFO:teuthology.orchestra.run.smithi100.stdout: "poolname": ".mgr" 2024-09-17T04:29:56.925 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:29:56.925 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-09-17T04:29:56.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.927+0000 7fc475400640 1 -- 172.21.15.100:0/4053243799 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc450078cb0 msgr2=0x7fc45007b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:56.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.927+0000 7fc475400640 1 --2- 172.21.15.100:0/4053243799 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc450078cb0 0x7fc45007b1c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc46000bb50 tx=0x7fc46000b2e0 comp rx=0 tx=0).stop 2024-09-17T04:29:56.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.927+0000 7fc475400640 1 -- 172.21.15.100:0/4053243799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4700b2f60 msgr2=0x7fc47014c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:56.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.927+0000 7fc475400640 1 --2- 172.21.15.100:0/4053243799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4700b2f60 0x7fc47014c0c0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00c400 tx=0x7fc45c00c8c0 comp rx=0 tx=0).stop 2024-09-17T04:29:56.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.927+0000 7fc475400640 1 -- 172.21.15.100:0/4053243799 shutdown_connections 2024-09-17T04:29:56.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.927+0000 7fc475400640 1 --2- 172.21.15.100:0/4053243799 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc450078cb0 0x7fc45007b1c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:56.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.927+0000 7fc475400640 1 --2- 172.21.15.100:0/4053243799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc47015a0f0 0x7fc470158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:56.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.927+0000 7fc475400640 1 --2- 172.21.15.100:0/4053243799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4700b2f60 0x7fc47014c0c0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:56.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.927+0000 7fc475400640 1 --2- 172.21.15.100:0/4053243799 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4700ad9c0 0x7fc4700ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:56.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.927+0000 7fc475400640 1 -- 172.21.15.100:0/4053243799 >> 172.21.15.100:0/4053243799 conn(0x7fc4700b1e40 msgr2=0x7fc4700b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:56.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.927+0000 7fc475400640 1 -- 172.21.15.100:0/4053243799 shutdown_connections 2024-09-17T04:29:56.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:56.927+0000 7fc475400640 1 -- 172.21.15.100:0/4053243799 wait complete. 2024-09-17T04:29:56.950 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T04:29:56.950 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T04:29:57.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.177+0000 7fcabe000640 1 -- 172.21.15.100:0/168419168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca9c0041c0 msgr2=0x7fca9c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:57.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.177+0000 7fcabe000640 1 --2- 172.21.15.100:0/168419168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca9c0041c0 0x7fca9c004800 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40017d0 tx=0x7fca9c005830 comp rx=0 tx=0).stop 2024-09-17T04:29:57.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.177+0000 7fcab7e00640 1 -- 172.21.15.100:0/168419168 reap_dead start 2024-09-17T04:29:57.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.177+0000 7fcabe000640 1 -- 172.21.15.100:0/168419168 shutdown_connections 2024-09-17T04:29:57.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.177+0000 7fcabe000640 1 -- 172.21.15.100:0/168419168 >> 172.21.15.100:0/168419168 conn(0x7fcab80b2540 msgr2=0x7fcab80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:57.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.177+0000 7fcabe000640 1 -- 172.21.15.100:0/168419168 shutdown_connections 2024-09-17T04:29:57.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.177+0000 7fcabe000640 1 -- 172.21.15.100:0/168419168 wait complete. 2024-09-17T04:29:57.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.179+0000 7fcabe000640 1 Processor -- start 2024-09-17T04:29:57.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.179+0000 7fcabe000640 1 -- start start 2024-09-17T04:29:57.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.181+0000 7fcabe000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca9c0041c0 0x7fcab8155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:57.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.181+0000 7fcabe000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcab80b17a0 0x7fcab8157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:57.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.181+0000 7fcabe000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcab8074b40 0x7fcab8157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:57.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.181+0000 7fcabe000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcab80b3a80 con 0x7fcab80b17a0 2024-09-17T04:29:57.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.181+0000 7fcabe000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcab80b3900 con 0x7fcab8074b40 2024-09-17T04:29:57.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.181+0000 7fcabe000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcab806f720 con 0x7fca9c0041c0 2024-09-17T04:29:57.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.181+0000 7fcab6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcab80b17a0 0x7fcab8157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:57.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.181+0000 7fcab6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcab80b17a0 0x7fcab8157520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37650/0 (socket says 172.21.15.100:37650) 2024-09-17T04:29:57.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.181+0000 7fcab6a00640 1 -- 172.21.15.100:0/358259334 learned_addr learned my addr 172.21.15.100:0/358259334 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:57.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.181+0000 7fcab7400640 1 --2- 172.21.15.100:0/358259334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca9c0041c0 0x7fcab8155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:57.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.181+0000 7fcab7e00640 1 --2- 172.21.15.100:0/358259334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcab8074b40 0x7fcab8157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:57.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.182+0000 7fcab6a00640 1 -- 172.21.15.100:0/358259334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca9c0041c0 msgr2=0x7fcab8155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:57.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.182+0000 7fcab6a00640 1 --2- 172.21.15.100:0/358259334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca9c0041c0 0x7fcab8155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:57.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.182+0000 7fcab6a00640 1 -- 172.21.15.100:0/358259334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcab8074b40 msgr2=0x7fcab8157a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:57.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.182+0000 7fcab6a00640 1 --2- 172.21.15.100:0/358259334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcab8074b40 0x7fcab8157a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:57.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.182+0000 7fcab6a00640 1 -- 172.21.15.100:0/358259334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab80b1c30 con 0x7fcab80b17a0 2024-09-17T04:29:57.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.182+0000 7fcab6a00640 1 --2- 172.21.15.100:0/358259334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcab80b17a0 0x7fcab8157520 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fcaa402aa40 tx=0x7fcaa4001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:57.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.182+0000 7fcaaf400640 1 -- 172.21.15.100:0/358259334 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4043070 con 0x7fcab80b17a0 2024-09-17T04:29:57.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.182+0000 7fcaaf400640 1 -- 172.21.15.100:0/358259334 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcaa40038e0 con 0x7fcab80b17a0 2024-09-17T04:29:57.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.182+0000 7fcabe000640 1 -- 172.21.15.100:0/358259334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcab80c94d0 con 0x7fcab80b17a0 2024-09-17T04:29:57.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.182+0000 7fcaaf400640 1 -- 172.21.15.100:0/358259334 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4003d10 con 0x7fcab80b17a0 2024-09-17T04:29:57.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.182+0000 7fcabe000640 1 -- 172.21.15.100:0/358259334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcab80b4a70 con 0x7fcab80b17a0 2024-09-17T04:29:57.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.183+0000 7fcabe000640 1 -- 172.21.15.100:0/358259334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcab806fc80 con 0x7fcab80b17a0 2024-09-17T04:29:57.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.185+0000 7fcaaf400640 1 -- 172.21.15.100:0/358259334 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fcaa4046070 con 0x7fcab80b17a0 2024-09-17T04:29:57.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.186+0000 7fcaaf400640 1 --2- 172.21.15.100:0/358259334 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fca98078ce0 0x7fca9807b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:57.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.186+0000 7fcaaf400640 1 -- 172.21.15.100:0/358259334 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 4857+0+0 (secure 0 0 0) 0x7fcaa40baa50 con 0x7fcab80b17a0 2024-09-17T04:29:57.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.186+0000 7fcab7400640 1 --2- 172.21.15.100:0/358259334 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fca98078ce0 0x7fca9807b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:57.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.186+0000 7fcab7400640 1 --2- 172.21.15.100:0/358259334 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fca98078ce0 0x7fca9807b1f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8001eb0 tx=0x7fcaa8006040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:57.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.190+0000 7fcaaf400640 1 -- 172.21.15.100:0/358259334 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fcab80c94d0 con 0x7fcab80b17a0 2024-09-17T04:29:57.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.374+0000 7fcabe000640 1 -- 172.21.15.100:0/358259334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fcab80ce360 con 0x7fcab80b17a0 2024-09-17T04:29:57.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.979+0000 7fcaaf400640 1 -- 172.21.15.100:0/358259334 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v102) ==== 138+0+0 (secure 0 0 0) 0x7fcaa4082960 con 0x7fcab80b17a0 2024-09-17T04:29:57.979 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' created 2024-09-17T04:29:57.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.981+0000 7fcabe000640 1 -- 172.21.15.100:0/358259334 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fca98078ce0 msgr2=0x7fca9807b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:57.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.981+0000 7fcabe000640 1 --2- 172.21.15.100:0/358259334 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fca98078ce0 0x7fca9807b1f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8001eb0 tx=0x7fcaa8006040 comp rx=0 tx=0).stop 2024-09-17T04:29:57.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.982+0000 7fcabe000640 1 -- 172.21.15.100:0/358259334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcab80b17a0 msgr2=0x7fcab8157520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:57.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.982+0000 7fcabe000640 1 --2- 172.21.15.100:0/358259334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcab80b17a0 0x7fcab8157520 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fcaa402aa40 tx=0x7fcaa4001190 comp rx=0 tx=0).stop 2024-09-17T04:29:57.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.982+0000 7fcabe000640 1 -- 172.21.15.100:0/358259334 shutdown_connections 2024-09-17T04:29:57.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.982+0000 7fcabe000640 1 --2- 172.21.15.100:0/358259334 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fca98078ce0 0x7fca9807b1f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:57.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.982+0000 7fcabe000640 1 --2- 172.21.15.100:0/358259334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcab8074b40 0x7fcab8157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:57.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.982+0000 7fcabe000640 1 --2- 172.21.15.100:0/358259334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcab80b17a0 0x7fcab8157520 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:57.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.982+0000 7fcabe000640 1 --2- 172.21.15.100:0/358259334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca9c0041c0 0x7fcab8155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:57.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.982+0000 7fcabe000640 1 -- 172.21.15.100:0/358259334 >> 172.21.15.100:0/358259334 conn(0x7fcab80b2540 msgr2=0x7fcab80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:57.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.982+0000 7fcabe000640 1 -- 172.21.15.100:0/358259334 shutdown_connections 2024-09-17T04:29:57.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:57.982+0000 7fcabe000640 1 -- 172.21.15.100:0/358259334 wait complete. 2024-09-17T04:29:58.009 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T04:29:58.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.236+0000 7ff12f600640 1 -- 172.21.15.100:0/1314774676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff128073c50 msgr2=0x7ff1280cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:58.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.236+0000 7ff12f600640 1 --2- 172.21.15.100:0/1314774676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff128073c50 0x7ff1280cd610 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff120005fd0 tx=0x7ff12002ab20 comp rx=0 tx=0).stop 2024-09-17T04:29:58.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.236+0000 7ff12f600640 1 -- 172.21.15.100:0/1314774676 shutdown_connections 2024-09-17T04:29:58.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.236+0000 7ff12f600640 1 --2- 172.21.15.100:0/1314774676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff128073c50 0x7ff1280cd610 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:58.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.236+0000 7ff12f600640 1 --2- 172.21.15.100:0/1314774676 >> v2:172.21.15.184:3300/0 conn(0x7ff128145550 0x7ff12806fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:58.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.236+0000 7ff12f600640 1 -- 172.21.15.100:0/1314774676 >> 172.21.15.100:0/1314774676 conn(0x7ff12806d1c0 msgr2=0x7ff12806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:58.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.236+0000 7ff12f600640 1 -- 172.21.15.100:0/1314774676 shutdown_connections 2024-09-17T04:29:58.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.236+0000 7ff12f600640 1 -- 172.21.15.100:0/1314774676 wait complete. 2024-09-17T04:29:58.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.238+0000 7ff12f600640 1 Processor -- start 2024-09-17T04:29:58.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.239+0000 7ff12f600640 1 -- start start 2024-09-17T04:29:58.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.240+0000 7ff12f600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff128145550 0x7ff1281f7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:58.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.240+0000 7ff12f600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff128145a40 0x7ff1280b2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.240+0000 7ff12f600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1280b2cd0 0x7ff12815b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.240+0000 7ff12f600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff1280ce100 con 0x7ff128145550 2024-09-17T04:29:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.240+0000 7ff12f600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff1280cdf80 con 0x7ff1280b2cd0 2024-09-17T04:29:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.240+0000 7ff12f600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff1280ce280 con 0x7ff128145a40 2024-09-17T04:29:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.240+0000 7ff12cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff128145550 0x7ff1281f7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.240+0000 7ff12cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff128145550 0x7ff1281f7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37664/0 (socket says 172.21.15.100:37664) 2024-09-17T04:29:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.240+0000 7ff12cc00640 1 -- 172.21.15.100:0/1622201510 learned_addr learned my addr 172.21.15.100:0/1622201510 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.241+0000 7ff12d600640 1 --2- 172.21.15.100:0/1622201510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1280b2cd0 0x7ff12815b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.241+0000 7ff127e00640 1 --2- 172.21.15.100:0/1622201510 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff128145a40 0x7ff1280b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.241+0000 7ff12cc00640 1 -- 172.21.15.100:0/1622201510 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff128145a40 msgr2=0x7ff1280b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.241+0000 7ff12cc00640 1 --2- 172.21.15.100:0/1622201510 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff128145a40 0x7ff1280b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.241+0000 7ff12cc00640 1 -- 172.21.15.100:0/1622201510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1280b2cd0 msgr2=0x7ff12815b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:58.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.241+0000 7ff12cc00640 1 --2- 172.21.15.100:0/1622201510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1280b2cd0 0x7ff12815b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:58.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.241+0000 7ff12cc00640 1 -- 172.21.15.100:0/1622201510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff128137180 con 0x7ff128145550 2024-09-17T04:29:58.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.241+0000 7ff12cc00640 1 --2- 172.21.15.100:0/1622201510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff128145550 0x7ff1281f7bf0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00e790 tx=0x7ff11c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:58.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.242+0000 7ff124c00640 1 -- 172.21.15.100:0/1622201510 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c00eea0 con 0x7ff128145550 2024-09-17T04:29:58.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.242+0000 7ff12f600640 1 -- 172.21.15.100:0/1622201510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff128137460 con 0x7ff128145550 2024-09-17T04:29:58.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.242+0000 7ff124c00640 1 -- 172.21.15.100:0/1622201510 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff11c00cdb0 con 0x7ff128145550 2024-09-17T04:29:58.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.242+0000 7ff124c00640 1 -- 172.21.15.100:0/1622201510 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c010690 con 0x7ff128145550 2024-09-17T04:29:58.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.242+0000 7ff12f600640 1 -- 172.21.15.100:0/1622201510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff128137a20 con 0x7ff128145550 2024-09-17T04:29:58.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.243+0000 7ff12f600640 1 -- 172.21.15.100:0/1622201510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1280b0150 con 0x7ff128145550 2024-09-17T04:29:58.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.245+0000 7ff124c00640 1 -- 172.21.15.100:0/1622201510 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7ff11c010830 con 0x7ff128145550 2024-09-17T04:29:58.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.246+0000 7ff124c00640 1 --2- 172.21.15.100:0/1622201510 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff108078cb0 0x7ff10807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:58.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.246+0000 7ff124c00640 1 -- 172.21.15.100:0/1622201510 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5227+0+0 (secure 0 0 0) 0x7ff11c09c040 con 0x7ff128145550 2024-09-17T04:29:58.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.246+0000 7ff127e00640 1 --2- 172.21.15.100:0/1622201510 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff108078cb0 0x7ff10807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:58.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.247+0000 7ff127e00640 1 --2- 172.21.15.100:0/1622201510 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff108078cb0 0x7ff10807b1c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff128160230 tx=0x7ff114009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:58.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.249+0000 7ff124c00640 1 -- 172.21.15.100:0/1622201510 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff11c063df0 con 0x7ff128145550 2024-09-17T04:29:58.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.435+0000 7ff12f600640 1 -- 172.21.15.100:0/1622201510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff1280ce7a0 con 0x7ff128145550 2024-09-17T04:29:58.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.982+0000 7ff124c00640 1 -- 172.21.15.100:0/1622201510 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v103) ==== 170+0+0 (secure 0 0 0) 0x7ff11c068820 con 0x7ff128145550 2024-09-17T04:29:58.983 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' created 2024-09-17T04:29:58.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.986+0000 7ff12f600640 1 -- 172.21.15.100:0/1622201510 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff108078cb0 msgr2=0x7ff10807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:58.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.986+0000 7ff12f600640 1 --2- 172.21.15.100:0/1622201510 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff108078cb0 0x7ff10807b1c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff128160230 tx=0x7ff114009040 comp rx=0 tx=0).stop 2024-09-17T04:29:58.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.986+0000 7ff12f600640 1 -- 172.21.15.100:0/1622201510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff128145550 msgr2=0x7ff1281f7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:58.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.986+0000 7ff12f600640 1 --2- 172.21.15.100:0/1622201510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff128145550 0x7ff1281f7bf0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00e790 tx=0x7ff11c00ec50 comp rx=0 tx=0).stop 2024-09-17T04:29:58.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.986+0000 7ff12f600640 1 -- 172.21.15.100:0/1622201510 shutdown_connections 2024-09-17T04:29:58.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.986+0000 7ff12f600640 1 --2- 172.21.15.100:0/1622201510 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff108078cb0 0x7ff10807b1c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:58.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.986+0000 7ff12f600640 1 --2- 172.21.15.100:0/1622201510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1280b2cd0 0x7ff12815b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:58.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.986+0000 7ff12f600640 1 --2- 172.21.15.100:0/1622201510 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff128145a40 0x7ff1280b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:58.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.986+0000 7ff12f600640 1 --2- 172.21.15.100:0/1622201510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff128145550 0x7ff1281f7bf0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:58.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.986+0000 7ff12f600640 1 -- 172.21.15.100:0/1622201510 >> 172.21.15.100:0/1622201510 conn(0x7ff12806d1c0 msgr2=0x7ff12813e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:58.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.986+0000 7ff12f600640 1 -- 172.21.15.100:0/1622201510 shutdown_connections 2024-09-17T04:29:58.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:58.986+0000 7ff12f600640 1 -- 172.21.15.100:0/1622201510 wait complete. 2024-09-17T04:29:59.028 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T04:29:59.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.253+0000 7f8321c00640 1 -- 172.21.15.100:0/3038585844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f831c156ca0 msgr2=0x7f831c0ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:59.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.253+0000 7f8321c00640 1 --2- 172.21.15.100:0/3038585844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f831c156ca0 0x7f831c0ccac0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f8308009830 tx=0x7f830802a5b0 comp rx=0 tx=0).stop 2024-09-17T04:29:59.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.253+0000 7f8321c00640 1 -- 172.21.15.100:0/3038585844 shutdown_connections 2024-09-17T04:29:59.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.253+0000 7f8321c00640 1 --2- 172.21.15.100:0/3038585844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f831c156ca0 0x7f831c0ccac0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:59.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.253+0000 7f8321c00640 1 -- 172.21.15.100:0/3038585844 >> 172.21.15.100:0/3038585844 conn(0x7f831c06d620 msgr2=0x7f831c0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:29:59.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.253+0000 7f8321c00640 1 -- 172.21.15.100:0/3038585844 shutdown_connections 2024-09-17T04:29:59.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.253+0000 7f8321c00640 1 -- 172.21.15.100:0/3038585844 wait complete. 2024-09-17T04:29:59.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.255+0000 7f8321c00640 1 Processor -- start 2024-09-17T04:29:59.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.256+0000 7f8321c00640 1 -- start start 2024-09-17T04:29:59.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.257+0000 7f8321c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f831c0b3f30 0x7f831c1f94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:59.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.257+0000 7f8321c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f831c0cd000 0x7f831c1f9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:59.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.257+0000 7f8321c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f831c156ca0 0x7f831c072eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:59.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.257+0000 7f8321c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f831c0af7e0 con 0x7f831c156ca0 2024-09-17T04:29:59.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.257+0000 7f8321c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f831c070370 con 0x7f831c0cd000 2024-09-17T04:29:59.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.257+0000 7f8321c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f831c0af960 con 0x7f831c0b3f30 2024-09-17T04:29:59.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.257+0000 7f8321200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f831c156ca0 0x7f831c072eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:59.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.257+0000 7f8321200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f831c156ca0 0x7f831c072eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37674/0 (socket says 172.21.15.100:37674) 2024-09-17T04:29:59.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.257+0000 7f8321200640 1 -- 172.21.15.100:0/1252416364 learned_addr learned my addr 172.21.15.100:0/1252416364 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:29:59.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.258+0000 7f831b400640 1 --2- 172.21.15.100:0/1252416364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f831c0cd000 0x7f831c1f9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:59.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.258+0000 7f831be00640 1 --2- 172.21.15.100:0/1252416364 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f831c0b3f30 0x7f831c1f94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:59.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.258+0000 7f8321200640 1 -- 172.21.15.100:0/1252416364 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f831c0b3f30 msgr2=0x7f831c1f94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:59.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.258+0000 7f8321200640 1 --2- 172.21.15.100:0/1252416364 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f831c0b3f30 0x7f831c1f94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:59.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.258+0000 7f8321200640 1 -- 172.21.15.100:0/1252416364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f831c0cd000 msgr2=0x7f831c1f9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:29:59.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.258+0000 7f8321200640 1 --2- 172.21.15.100:0/1252416364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f831c0cd000 0x7f831c1f9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:29:59.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.258+0000 7f8321200640 1 -- 172.21.15.100:0/1252416364 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831c1455a0 con 0x7f831c156ca0 2024-09-17T04:29:59.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.258+0000 7f8321200640 1 --2- 172.21.15.100:0/1252416364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f831c156ca0 0x7f831c072eb0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f831400c420 tx=0x7f831400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:59.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.258+0000 7f8313e00640 1 -- 172.21.15.100:0/1252416364 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314011790 con 0x7f831c156ca0 2024-09-17T04:29:59.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.259+0000 7f8313e00640 1 -- 172.21.15.100:0/1252416364 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8314017070 con 0x7f831c156ca0 2024-09-17T04:29:59.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.259+0000 7f8313e00640 1 -- 172.21.15.100:0/1252416364 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314011ce0 con 0x7f831c156ca0 2024-09-17T04:29:59.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.259+0000 7f8321c00640 1 -- 172.21.15.100:0/1252416364 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831c1fa140 con 0x7f831c156ca0 2024-09-17T04:29:59.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.259+0000 7f8321c00640 1 -- 172.21.15.100:0/1252416364 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f831c157990 con 0x7f831c156ca0 2024-09-17T04:29:59.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.260+0000 7f8321c00640 1 -- 172.21.15.100:0/1252416364 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f831c0ccac0 con 0x7f831c156ca0 2024-09-17T04:29:59.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.261+0000 7f8313e00640 1 -- 172.21.15.100:0/1252416364 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f831401c460 con 0x7f831c156ca0 2024-09-17T04:29:59.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.262+0000 7f8313e00640 1 --2- 172.21.15.100:0/1252416364 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f82fc078cb0 0x7f82fc07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:29:59.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.262+0000 7f8313e00640 1 -- 172.21.15.100:0/1252416364 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5609+0+0 (secure 0 0 0) 0x7f831409abc0 con 0x7f831c156ca0 2024-09-17T04:29:59.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.262+0000 7f831be00640 1 --2- 172.21.15.100:0/1252416364 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f82fc078cb0 0x7f82fc07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:29:59.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.263+0000 7f831be00640 1 --2- 172.21.15.100:0/1252416364 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f82fc078cb0 0x7f82fc07b1c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f830c001e20 tx=0x7f830c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:29:59.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.267+0000 7f8313e00640 1 -- 172.21.15.100:0/1252416364 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8314062760 con 0x7f831c156ca0 2024-09-17T04:29:59.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:29:59.485+0000 7f8321c00640 1 -- 172.21.15.100:0/1252416364 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f831c0b0280 con 0x7f831c156ca0 2024-09-17T04:30:00.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.009+0000 7f8313e00640 1 -- 172.21.15.100:0/1252416364 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v68) ==== 174+0+0 (secure 0 0 0) 0x7f8314067190 con 0x7f831c156ca0 2024-09-17T04:30:00.010 INFO:teuthology.orchestra.run.smithi100.stderr:new fs with metadata pool 16 and data pool 17 2024-09-17T04:30:00.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.016+0000 7f8321c00640 1 -- 172.21.15.100:0/1252416364 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f82fc078cb0 msgr2=0x7f82fc07b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:00.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.016+0000 7f8321c00640 1 --2- 172.21.15.100:0/1252416364 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f82fc078cb0 0x7f82fc07b1c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f830c001e20 tx=0x7f830c009040 comp rx=0 tx=0).stop 2024-09-17T04:30:00.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.016+0000 7f8321c00640 1 -- 172.21.15.100:0/1252416364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f831c156ca0 msgr2=0x7f831c072eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:00.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.016+0000 7f8321c00640 1 --2- 172.21.15.100:0/1252416364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f831c156ca0 0x7f831c072eb0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f831400c420 tx=0x7f831400c8e0 comp rx=0 tx=0).stop 2024-09-17T04:30:00.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.017+0000 7f8321c00640 1 -- 172.21.15.100:0/1252416364 shutdown_connections 2024-09-17T04:30:00.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.017+0000 7f8321c00640 1 --2- 172.21.15.100:0/1252416364 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f82fc078cb0 0x7f82fc07b1c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:00.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.017+0000 7f8321c00640 1 --2- 172.21.15.100:0/1252416364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f831c156ca0 0x7f831c072eb0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:00.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.017+0000 7f8321c00640 1 --2- 172.21.15.100:0/1252416364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f831c0cd000 0x7f831c1f9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:00.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.017+0000 7f8321c00640 1 --2- 172.21.15.100:0/1252416364 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f831c0b3f30 0x7f831c1f94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:00.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.017+0000 7f8321c00640 1 -- 172.21.15.100:0/1252416364 >> 172.21.15.100:0/1252416364 conn(0x7f831c06d620 msgr2=0x7f831c0af0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:00.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.017+0000 7f8321c00640 1 -- 172.21.15.100:0/1252416364 shutdown_connections 2024-09-17T04:30:00.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.018+0000 7f8321c00640 1 -- 172.21.15.100:0/1252416364 wait complete. 2024-09-17T04:30:00.066 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:30:00.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.290+0000 7f0d3e800640 1 -- 172.21.15.100:0/291631189 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d3815d760 msgr2=0x7f0d381535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:00.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.290+0000 7f0d3e800640 1 --2- 172.21.15.100:0/291631189 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d3815d760 0x7f0d381535c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0d28009830 tx=0x7f0d2802a530 comp rx=0 tx=0).stop 2024-09-17T04:30:00.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.290+0000 7f0d3e800640 1 -- 172.21.15.100:0/291631189 shutdown_connections 2024-09-17T04:30:00.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.290+0000 7f0d3e800640 1 --2- 172.21.15.100:0/291631189 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d3815d760 0x7f0d381535c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:00.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.290+0000 7f0d3e800640 1 -- 172.21.15.100:0/291631189 >> 172.21.15.100:0/291631189 conn(0x7f0d3806d730 msgr2=0x7f0d380b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:00.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.290+0000 7f0d3e800640 1 -- 172.21.15.100:0/291631189 shutdown_connections 2024-09-17T04:30:00.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.290+0000 7f0d3e800640 1 -- 172.21.15.100:0/291631189 wait complete. 2024-09-17T04:30:00.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.292+0000 7f0d3e800640 1 Processor -- start 2024-09-17T04:30:00.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.292+0000 7f0d3e800640 1 -- start start 2024-09-17T04:30:00.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.294+0000 7f0d3e800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d380ad0e0 0x7f0d381f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:00.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.294+0000 7f0d3e800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d380b3550 0x7f0d381f7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:00.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.294+0000 7f0d3e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d3815d760 0x7f0d3814bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:00.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.294+0000 7f0d3e800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3806f240 con 0x7f0d3815d760 2024-09-17T04:30:00.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.294+0000 7f0d3e800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0d380b4e30 con 0x7f0d380ad0e0 2024-09-17T04:30:00.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.294+0000 7f0d3e800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0d380b4070 con 0x7f0d380b3550 2024-09-17T04:30:00.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.294+0000 7f0d37400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d380ad0e0 0x7f0d381f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:00.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.294+0000 7f0d37400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d380ad0e0 0x7f0d381f6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.100:42320/0 (socket says 172.21.15.100:42320) 2024-09-17T04:30:00.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.294+0000 7f0d37400640 1 -- 172.21.15.100:0/2180243481 learned_addr learned my addr 172.21.15.100:0/2180243481 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:00.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.294+0000 7f0d37e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d3815d760 0x7f0d3814bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:00.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.294+0000 7f0d2fe00640 1 --2- 172.21.15.100:0/2180243481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d380b3550 0x7f0d381f7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:00.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.295+0000 7f0d37400640 1 -- 172.21.15.100:0/2180243481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d380b3550 msgr2=0x7f0d381f7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:00.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.295+0000 7f0d37400640 1 --2- 172.21.15.100:0/2180243481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d380b3550 0x7f0d381f7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:00.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.295+0000 7f0d37400640 1 -- 172.21.15.100:0/2180243481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d3815d760 msgr2=0x7f0d3814bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:00.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.295+0000 7f0d37400640 1 --2- 172.21.15.100:0/2180243481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d3815d760 0x7f0d3814bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:00.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.295+0000 7f0d37400640 1 -- 172.21.15.100:0/2180243481 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d381499f0 con 0x7f0d380ad0e0 2024-09-17T04:30:00.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.295+0000 7f0d37400640 1 --2- 172.21.15.100:0/2180243481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d380ad0e0 0x7f0d381f6fe0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0d28009950 tx=0x7f0d28033c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:00.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.295+0000 7f0d2f400640 1 -- 172.21.15.100:0/2180243481 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d28031d60 con 0x7f0d380ad0e0 2024-09-17T04:30:00.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.295+0000 7f0d2f400640 1 -- 172.21.15.100:0/2180243481 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d28033e10 con 0x7f0d380ad0e0 2024-09-17T04:30:00.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.295+0000 7f0d2f400640 1 -- 172.21.15.100:0/2180243481 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d28033310 con 0x7f0d380ad0e0 2024-09-17T04:30:00.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.295+0000 7f0d3e800640 1 -- 172.21.15.100:0/2180243481 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d38149cd0 con 0x7f0d380ad0e0 2024-09-17T04:30:00.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.296+0000 7f0d3e800640 1 -- 172.21.15.100:0/2180243481 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d381f7b80 con 0x7f0d380ad0e0 2024-09-17T04:30:00.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.296+0000 7f0d3e800640 1 -- 172.21.15.100:0/2180243481 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d38149f10 con 0x7f0d380ad0e0 2024-09-17T04:30:00.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.298+0000 7f0d2f400640 1 -- 172.21.15.100:0/2180243481 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f0d280334b0 con 0x7f0d380ad0e0 2024-09-17T04:30:00.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.299+0000 7f0d2f400640 1 --2- 172.21.15.100:0/2180243481 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0d14078cb0 0x7f0d1407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:00.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.299+0000 7f0d2f400640 1 -- 172.21.15.100:0/2180243481 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f0d280bc6e0 con 0x7f0d380ad0e0 2024-09-17T04:30:00.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.299+0000 7f0d2fe00640 1 --2- 172.21.15.100:0/2180243481 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0d14078cb0 0x7f0d1407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:00.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.300+0000 7f0d2fe00640 1 --2- 172.21.15.100:0/2180243481 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0d14078cb0 0x7f0d1407b1c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0d38149810 tx=0x7f0d20009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:00.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.303+0000 7f0d2f400640 1 -- 172.21.15.100:0/2180243481 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0d38149f10 con 0x7f0d380ad0e0 2024-09-17T04:30:00.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.488+0000 7f0d3e800640 1 -- 172.21.15.100:0/2180243481 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d38069a50 con 0x7f0d380ad0e0 2024-09-17T04:30:00.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.490+0000 7f0d2f400640 1 -- 172.21.15.100:0/2180243481 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+16748 (secure 0 0 0) 0x7f0d38069a50 con 0x7f0d380ad0e0 2024-09-17T04:30:00.490 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:30:00.491 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":104,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:29:59.487621+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:29:57.388120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-17T04:29:58.449229+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:30:00.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.492+0000 7f0d3e800640 1 -- 172.21.15.100:0/2180243481 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0d14078cb0 msgr2=0x7f0d1407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:00.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.492+0000 7f0d3e800640 1 --2- 172.21.15.100:0/2180243481 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0d14078cb0 0x7f0d1407b1c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0d38149810 tx=0x7f0d20009040 comp rx=0 tx=0).stop 2024-09-17T04:30:00.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.492+0000 7f0d3e800640 1 -- 172.21.15.100:0/2180243481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d380ad0e0 msgr2=0x7f0d381f6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:00.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.492+0000 7f0d3e800640 1 --2- 172.21.15.100:0/2180243481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d380ad0e0 0x7f0d381f6fe0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0d28009950 tx=0x7f0d28033c20 comp rx=0 tx=0).stop 2024-09-17T04:30:00.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.492+0000 7f0d3e800640 1 -- 172.21.15.100:0/2180243481 shutdown_connections 2024-09-17T04:30:00.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.492+0000 7f0d3e800640 1 --2- 172.21.15.100:0/2180243481 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0d14078cb0 0x7f0d1407b1c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:00.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.492+0000 7f0d3e800640 1 --2- 172.21.15.100:0/2180243481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d3815d760 0x7f0d3814bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:00.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.492+0000 7f0d3e800640 1 --2- 172.21.15.100:0/2180243481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d380b3550 0x7f0d381f7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:00.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.492+0000 7f0d3e800640 1 --2- 172.21.15.100:0/2180243481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d380ad0e0 0x7f0d381f6fe0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:00.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.492+0000 7f0d3e800640 1 -- 172.21.15.100:0/2180243481 >> 172.21.15.100:0/2180243481 conn(0x7f0d3806d730 msgr2=0x7f0d380b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:00.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.492+0000 7f0d3e800640 1 -- 172.21.15.100:0/2180243481 shutdown_connections 2024-09-17T04:30:00.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.492+0000 7f0d3e800640 1 -- 172.21.15.100:0/2180243481 wait complete. 2024-09-17T04:30:00.516 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:30:00.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.738+0000 7f5bb3c00640 1 -- 172.21.15.100:0/1629849209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b94004800 msgr2=0x7f5b94004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:00.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.738+0000 7f5bb3c00640 1 --2- 172.21.15.100:0/1629849209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b94004800 0x7f5b94004c10 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f5b98006f60 tx=0x7f5b94005830 comp rx=0 tx=0).stop 2024-09-17T04:30:00.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.738+0000 7f5bb1c00640 1 -- 172.21.15.100:0/1629849209 reap_dead start 2024-09-17T04:30:00.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.739+0000 7f5bb3c00640 1 -- 172.21.15.100:0/1629849209 shutdown_connections 2024-09-17T04:30:00.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.739+0000 7f5bb3c00640 1 -- 172.21.15.100:0/1629849209 >> 172.21.15.100:0/1629849209 conn(0x7f5bac06d730 msgr2=0x7f5bac0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:00.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.739+0000 7f5bb3c00640 1 -- 172.21.15.100:0/1629849209 shutdown_connections 2024-09-17T04:30:00.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.739+0000 7f5bb3c00640 1 -- 172.21.15.100:0/1629849209 wait complete. 2024-09-17T04:30:00.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.741+0000 7f5bb3c00640 1 Processor -- start 2024-09-17T04:30:00.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.741+0000 7f5bb3c00640 1 -- start start 2024-09-17T04:30:00.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.742+0000 7f5bb3c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b94004800 0x7f5bac1475d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:00.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.742+0000 7f5bb3c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5bac148a50 0x7f5bac14a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:00.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.742+0000 7f5bb3c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bac14a8b0 0x7f5bac1f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:00.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.742+0000 7f5bb3c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5bac0b4e30 con 0x7f5bac14a8b0 2024-09-17T04:30:00.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.742+0000 7f5bb3c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5bac165890 con 0x7f5b94004800 2024-09-17T04:30:00.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.742+0000 7f5bb3c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5bac0b1670 con 0x7f5bac148a50 2024-09-17T04:30:00.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.743+0000 7f5bb1c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bac14a8b0 0x7f5bac1f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:00.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.743+0000 7f5bb1c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bac14a8b0 0x7f5bac1f6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37690/0 (socket says 172.21.15.100:37690) 2024-09-17T04:30:00.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.743+0000 7f5bb1c00640 1 -- 172.21.15.100:0/2750359173 learned_addr learned my addr 172.21.15.100:0/2750359173 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:00.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.743+0000 7f5bb1c00640 1 -- 172.21.15.100:0/2750359173 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5bac148a50 msgr2=0x7f5bac14a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:00.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.743+0000 7f5babe00640 1 --2- 172.21.15.100:0/2750359173 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5bac148a50 0x7f5bac14a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:00.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.743+0000 7f5bb1200640 1 --2- 172.21.15.100:0/2750359173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b94004800 0x7f5bac1475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:00.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.743+0000 7f5bb1c00640 1 --2- 172.21.15.100:0/2750359173 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5bac148a50 0x7f5bac14a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:00.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.743+0000 7f5bb1c00640 1 -- 172.21.15.100:0/2750359173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b94004800 msgr2=0x7f5bac1475d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:00.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.743+0000 7f5bb1c00640 1 --2- 172.21.15.100:0/2750359173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b94004800 0x7f5bac1475d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:00.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.743+0000 7f5bb1c00640 1 -- 172.21.15.100:0/2750359173 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bac156bb0 con 0x7f5bac14a8b0 2024-09-17T04:30:00.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.743+0000 7f5babe00640 1 --2- 172.21.15.100:0/2750359173 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5bac148a50 0x7f5bac14a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:30:00.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.743+0000 7f5bb1200640 1 --2- 172.21.15.100:0/2750359173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b94004800 0x7f5bac1475d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:30:00.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.743+0000 7f5bb1c00640 1 --2- 172.21.15.100:0/2750359173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bac14a8b0 0x7f5bac1f6e80 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4005fd0 tx=0x7f5ba400ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:00.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.744+0000 7f5ba8c00640 1 -- 172.21.15.100:0/2750359173 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4004b20 con 0x7f5bac14a8b0 2024-09-17T04:30:00.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.744+0000 7f5ba8c00640 1 -- 172.21.15.100:0/2750359173 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ba4004dd0 con 0x7f5bac14a8b0 2024-09-17T04:30:00.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.744+0000 7f5ba8c00640 1 -- 172.21.15.100:0/2750359173 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba400cd60 con 0x7f5bac14a8b0 2024-09-17T04:30:00.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.744+0000 7f5bb3c00640 1 -- 172.21.15.100:0/2750359173 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bac156e70 con 0x7f5bac14a8b0 2024-09-17T04:30:00.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.744+0000 7f5bb3c00640 1 -- 172.21.15.100:0/2750359173 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bac0b00d0 con 0x7f5bac14a8b0 2024-09-17T04:30:00.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.745+0000 7f5bb3c00640 1 -- 172.21.15.100:0/2750359173 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bac1535c0 con 0x7f5bac14a8b0 2024-09-17T04:30:00.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.746+0000 7f5ba8c00640 1 -- 172.21.15.100:0/2750359173 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f5ba401b030 con 0x7f5bac14a8b0 2024-09-17T04:30:00.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.747+0000 7f5ba8c00640 1 --2- 172.21.15.100:0/2750359173 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5b8c078cb0 0x7f5b8c07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:00.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.747+0000 7f5ba8c00640 1 -- 172.21.15.100:0/2750359173 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f5ba409aa20 con 0x7f5bac14a8b0 2024-09-17T04:30:00.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.747+0000 7f5bb1200640 1 --2- 172.21.15.100:0/2750359173 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5b8c078cb0 0x7f5b8c07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:00.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.748+0000 7f5bb1200640 1 --2- 172.21.15.100:0/2750359173 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5b8c078cb0 0x7f5b8c07b1c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5ba00064d0 tx=0x7f5ba0006b10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:00.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.752+0000 7f5ba8c00640 1 -- 172.21.15.100:0/2750359173 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f5ba4022060 con 0x7f5bac14a8b0 2024-09-17T04:30:00.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.936+0000 7f5bb3c00640 1 -- 172.21.15.100:0/2750359173 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5bac1f4610 con 0x7f5bac14a8b0 2024-09-17T04:30:00.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.937+0000 7f5ba8c00640 1 -- 172.21.15.100:0/2750359173 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+16748 (secure 0 0 0) 0x7f5ba4062cd0 con 0x7f5bac14a8b0 2024-09-17T04:30:00.937 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:30:00.938 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":104,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:29:59.487621+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:29:57.388120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-17T04:29:58.449229+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:30:00.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.939+0000 7f5bb3c00640 1 -- 172.21.15.100:0/2750359173 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5b8c078cb0 msgr2=0x7f5b8c07b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:00.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.939+0000 7f5bb3c00640 1 --2- 172.21.15.100:0/2750359173 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5b8c078cb0 0x7f5b8c07b1c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5ba00064d0 tx=0x7f5ba0006b10 comp rx=0 tx=0).stop 2024-09-17T04:30:00.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.939+0000 7f5bb3c00640 1 -- 172.21.15.100:0/2750359173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bac14a8b0 msgr2=0x7f5bac1f6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:00.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.939+0000 7f5bb3c00640 1 --2- 172.21.15.100:0/2750359173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bac14a8b0 0x7f5bac1f6e80 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4005fd0 tx=0x7f5ba400ee10 comp rx=0 tx=0).stop 2024-09-17T04:30:00.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.939+0000 7f5bb3c00640 1 -- 172.21.15.100:0/2750359173 shutdown_connections 2024-09-17T04:30:00.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.939+0000 7f5bb3c00640 1 --2- 172.21.15.100:0/2750359173 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5b8c078cb0 0x7f5b8c07b1c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:00.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.939+0000 7f5bb3c00640 1 --2- 172.21.15.100:0/2750359173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bac14a8b0 0x7f5bac1f6e80 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:00.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.939+0000 7f5bb3c00640 1 --2- 172.21.15.100:0/2750359173 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5bac148a50 0x7f5bac14a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:00.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.939+0000 7f5bb3c00640 1 --2- 172.21.15.100:0/2750359173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b94004800 0x7f5bac1475d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:00.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.940+0000 7f5bb3c00640 1 -- 172.21.15.100:0/2750359173 >> 172.21.15.100:0/2750359173 conn(0x7f5bac06d730 msgr2=0x7f5bac14e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:00.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.940+0000 7f5bb3c00640 1 -- 172.21.15.100:0/2750359173 shutdown_connections 2024-09-17T04:30:00.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:00.940+0000 7f5bb3c00640 1 -- 172.21.15.100:0/2750359173 wait complete. 2024-09-17T04:30:00.965 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T04:30:01.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.189+0000 7f566a600640 1 -- 172.21.15.100:0/2510817224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56640af4e0 msgr2=0x7f56640cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:01.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.189+0000 7f566a600640 1 --2- 172.21.15.100:0/2510817224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56640af4e0 0x7f56640cf850 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f5664074890 tx=0x7f565802a820 comp rx=0 tx=0).stop 2024-09-17T04:30:01.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.189+0000 7f566a600640 1 -- 172.21.15.100:0/2510817224 shutdown_connections 2024-09-17T04:30:01.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.189+0000 7f566a600640 1 --2- 172.21.15.100:0/2510817224 >> v2:172.21.15.184:3300/0 conn(0x7f56640af8f0 0x7f566406ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:01.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.189+0000 7f566a600640 1 --2- 172.21.15.100:0/2510817224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56640af4e0 0x7f56640cf850 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:01.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.189+0000 7f566a600640 1 -- 172.21.15.100:0/2510817224 >> 172.21.15.100:0/2510817224 conn(0x7f566406d620 msgr2=0x7f56640b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:01.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.189+0000 7f566a600640 1 -- 172.21.15.100:0/2510817224 shutdown_connections 2024-09-17T04:30:01.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.189+0000 7f566a600640 1 -- 172.21.15.100:0/2510817224 wait complete. 2024-09-17T04:30:01.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.192+0000 7f566a600640 1 Processor -- start 2024-09-17T04:30:01.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.192+0000 7f566a600640 1 -- start start 2024-09-17T04:30:01.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.193+0000 7f566a600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56640af4e0 0x7f56641f92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:01.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.193+0000 7f566a600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56640af8f0 0x7f5664149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:01.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.193+0000 7f566a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56640cfd90 0x7f56640b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:01.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.193+0000 7f566a600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f566406f450 con 0x7f56640cfd90 2024-09-17T04:30:01.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.193+0000 7f566a600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f56640711b0 con 0x7f56640af8f0 2024-09-17T04:30:01.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.193+0000 7f566a600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f566406f5d0 con 0x7f56640af4e0 2024-09-17T04:30:01.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.193+0000 7f5669c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56640cfd90 0x7f56640b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:01.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.194+0000 7f5669c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56640cfd90 0x7f56640b08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37718/0 (socket says 172.21.15.100:37718) 2024-09-17T04:30:01.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.194+0000 7f5669c00640 1 -- 172.21.15.100:0/4155132626 learned_addr learned my addr 172.21.15.100:0/4155132626 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:01.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.194+0000 7f5669200640 1 --2- 172.21.15.100:0/4155132626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56640af4e0 0x7f56641f92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:01.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.194+0000 7f5663e00640 1 --2- 172.21.15.100:0/4155132626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56640af8f0 0x7f5664149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:01.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.194+0000 7f5669c00640 1 -- 172.21.15.100:0/4155132626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56640af4e0 msgr2=0x7f56641f92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:01.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.194+0000 7f5669c00640 1 --2- 172.21.15.100:0/4155132626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56640af4e0 0x7f56641f92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:01.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.194+0000 7f5669c00640 1 -- 172.21.15.100:0/4155132626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56640af8f0 msgr2=0x7f5664149570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:01.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.194+0000 7f5669c00640 1 --2- 172.21.15.100:0/4155132626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56640af8f0 0x7f5664149570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:01.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.194+0000 7f5669c00640 1 -- 172.21.15.100:0/4155132626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56641f9830 con 0x7f56640cfd90 2024-09-17T04:30:01.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.194+0000 7f5669c00640 1 --2- 172.21.15.100:0/4155132626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56640cfd90 0x7f56640b08c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f565c00d660 tx=0x7f565c00db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:01.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.194+0000 7f5660c00640 1 -- 172.21.15.100:0/4155132626 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c017070 con 0x7f56640cfd90 2024-09-17T04:30:01.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.195+0000 7f5660c00640 1 -- 172.21.15.100:0/4155132626 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f565c005310 con 0x7f56640cfd90 2024-09-17T04:30:01.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.195+0000 7f566a600640 1 -- 172.21.15.100:0/4155132626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56641f8b10 con 0x7f56640cfd90 2024-09-17T04:30:01.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.195+0000 7f5660c00640 1 -- 172.21.15.100:0/4155132626 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c009820 con 0x7f56640cfd90 2024-09-17T04:30:01.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.195+0000 7f566a600640 1 -- 172.21.15.100:0/4155132626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56640c9940 con 0x7f56640cfd90 2024-09-17T04:30:01.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.196+0000 7f566a600640 1 -- 172.21.15.100:0/4155132626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56640cf850 con 0x7f56640cfd90 2024-09-17T04:30:01.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.197+0000 7f5660c00640 1 -- 172.21.15.100:0/4155132626 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f565c0099c0 con 0x7f56640cfd90 2024-09-17T04:30:01.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.198+0000 7f5660c00640 1 --2- 172.21.15.100:0/4155132626 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5644078cb0 0x7f564407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:01.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.198+0000 7f5660c00640 1 -- 172.21.15.100:0/4155132626 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f565c09b0b0 con 0x7f56640cfd90 2024-09-17T04:30:01.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.198+0000 7f5669200640 1 --2- 172.21.15.100:0/4155132626 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5644078cb0 0x7f564407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:01.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.199+0000 7f5669200640 1 --2- 172.21.15.100:0/4155132626 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5644078cb0 0x7f564407b1c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5658034000 tx=0x7f5658004ab0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:01.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.202+0000 7f5660c00640 1 -- 172.21.15.100:0/4155132626 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f565c062bf0 con 0x7f56640cfd90 2024-09-17T04:30:01.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:01.425+0000 7f566a600640 1 -- 172.21.15.100:0/4155132626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f56640cf240 con 0x7f56640cfd90 2024-09-17T04:30:01.640 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-09-17T04:30:02.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.023+0000 7f5660c00640 1 -- 172.21.15.100:0/4155132626 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v71) ==== 118+0+0 (secure 0 0 0) 0x7f565c067620 con 0x7f56640cfd90 2024-09-17T04:30:02.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.026+0000 7f566a600640 1 -- 172.21.15.100:0/4155132626 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5644078cb0 msgr2=0x7f564407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:02.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.026+0000 7f566a600640 1 --2- 172.21.15.100:0/4155132626 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5644078cb0 0x7f564407b1c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5658034000 tx=0x7f5658004ab0 comp rx=0 tx=0).stop 2024-09-17T04:30:02.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.026+0000 7f566a600640 1 -- 172.21.15.100:0/4155132626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56640cfd90 msgr2=0x7f56640b08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:02.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.026+0000 7f566a600640 1 --2- 172.21.15.100:0/4155132626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56640cfd90 0x7f56640b08c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f565c00d660 tx=0x7f565c00db20 comp rx=0 tx=0).stop 2024-09-17T04:30:02.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.026+0000 7f566a600640 1 -- 172.21.15.100:0/4155132626 shutdown_connections 2024-09-17T04:30:02.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.026+0000 7f566a600640 1 --2- 172.21.15.100:0/4155132626 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5644078cb0 0x7f564407b1c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:02.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.026+0000 7f566a600640 1 --2- 172.21.15.100:0/4155132626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56640cfd90 0x7f56640b08c0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:02.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.026+0000 7f566a600640 1 --2- 172.21.15.100:0/4155132626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56640af8f0 0x7f5664149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:02.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.026+0000 7f566a600640 1 --2- 172.21.15.100:0/4155132626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56640af4e0 0x7f56641f92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:02.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.026+0000 7f566a600640 1 -- 172.21.15.100:0/4155132626 >> 172.21.15.100:0/4155132626 conn(0x7f566406d620 msgr2=0x7f5664070890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:02.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.026+0000 7f566a600640 1 -- 172.21.15.100:0/4155132626 shutdown_connections 2024-09-17T04:30:02.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.026+0000 7f566a600640 1 -- 172.21.15.100:0/4155132626 wait complete. 2024-09-17T04:30:02.051 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T04:30:02.051 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T04:30:02.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.274+0000 7fe022a00640 1 -- 172.21.15.100:0/947663033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe01c0cd000 msgr2=0x7fe01c0af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:02.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.274+0000 7fe022a00640 1 --2- 172.21.15.100:0/947663033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe01c0cd000 0x7fe01c0af0d0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fe01c15de20 tx=0x7fe01402aa00 comp rx=0 tx=0).stop 2024-09-17T04:30:02.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.274+0000 7fe022a00640 1 -- 172.21.15.100:0/947663033 shutdown_connections 2024-09-17T04:30:02.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.274+0000 7fe022a00640 1 --2- 172.21.15.100:0/947663033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe01c0cd000 0x7fe01c0af0d0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:02.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.274+0000 7fe022a00640 1 --2- 172.21.15.100:0/947663033 >> v2:172.21.15.184:3300/0 conn(0x7fe01c156ca0 0x7fe01c0ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:02.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.274+0000 7fe022a00640 1 -- 172.21.15.100:0/947663033 >> 172.21.15.100:0/947663033 conn(0x7fe01c06d620 msgr2=0x7fe01c0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:02.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.274+0000 7fe022a00640 1 -- 172.21.15.100:0/947663033 shutdown_connections 2024-09-17T04:30:02.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.274+0000 7fe022a00640 1 -- 172.21.15.100:0/947663033 wait complete. 2024-09-17T04:30:02.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.276+0000 7fe022a00640 1 Processor -- start 2024-09-17T04:30:02.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.276+0000 7fe022a00640 1 -- start start 2024-09-17T04:30:02.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.278+0000 7fe022a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe01c0b3f30 0x7fe01c145290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:02.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.278+0000 7fe022a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe01c0cd000 0x7fe01c0741a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:02.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.278+0000 7fe022a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe01c156ca0 0x7fe01c0746e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:02.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.278+0000 7fe022a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c0af9a0 con 0x7fe01c156ca0 2024-09-17T04:30:02.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.278+0000 7fe022a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c0af820 con 0x7fe01c0b3f30 2024-09-17T04:30:02.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.278+0000 7fe022a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe01c0afb20 con 0x7fe01c0cd000 2024-09-17T04:30:02.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.278+0000 7fe022000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe01c156ca0 0x7fe01c0746e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:02.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.278+0000 7fe022000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe01c156ca0 0x7fe01c0746e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37740/0 (socket says 172.21.15.100:37740) 2024-09-17T04:30:02.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.278+0000 7fe022000640 1 -- 172.21.15.100:0/1726175052 learned_addr learned my addr 172.21.15.100:0/1726175052 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:02.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.278+0000 7fe021600640 1 --2- 172.21.15.100:0/1726175052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe01c0b3f30 0x7fe01c145290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:02.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.278+0000 7fe020c00640 1 --2- 172.21.15.100:0/1726175052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe01c0cd000 0x7fe01c0741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:02.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.279+0000 7fe022000640 1 -- 172.21.15.100:0/1726175052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe01c0cd000 msgr2=0x7fe01c0741a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:02.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.279+0000 7fe022000640 1 --2- 172.21.15.100:0/1726175052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe01c0cd000 0x7fe01c0741a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:02.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.279+0000 7fe022000640 1 -- 172.21.15.100:0/1726175052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe01c0b3f30 msgr2=0x7fe01c145290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:02.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.279+0000 7fe022000640 1 --2- 172.21.15.100:0/1726175052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe01c0b3f30 0x7fe01c145290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:02.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.279+0000 7fe022000640 1 -- 172.21.15.100:0/1726175052 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01c1e6d10 con 0x7fe01c156ca0 2024-09-17T04:30:02.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.279+0000 7fe021600640 1 --2- 172.21.15.100:0/1726175052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe01c0b3f30 0x7fe01c145290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:30:02.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.279+0000 7fe022000640 1 --2- 172.21.15.100:0/1726175052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe01c156ca0 0x7fe01c0746e0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fe01402aff0 tx=0x7fe014003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:02.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.279+0000 7fe019600640 1 -- 172.21.15.100:0/1726175052 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe014033050 con 0x7fe01c156ca0 2024-09-17T04:30:02.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.279+0000 7fe019600640 1 -- 172.21.15.100:0/1726175052 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe014004450 con 0x7fe01c156ca0 2024-09-17T04:30:02.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.279+0000 7fe019600640 1 -- 172.21.15.100:0/1726175052 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe014038450 con 0x7fe01c156ca0 2024-09-17T04:30:02.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.279+0000 7fe022a00640 1 -- 172.21.15.100:0/1726175052 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe01c073100 con 0x7fe01c156ca0 2024-09-17T04:30:02.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.279+0000 7fe022a00640 1 -- 172.21.15.100:0/1726175052 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe01c0736c0 con 0x7fe01c156ca0 2024-09-17T04:30:02.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.280+0000 7fe022a00640 1 -- 172.21.15.100:0/1726175052 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe01c0af0d0 con 0x7fe01c156ca0 2024-09-17T04:30:02.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.281+0000 7fe019600640 1 -- 172.21.15.100:0/1726175052 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fe014036070 con 0x7fe01c156ca0 2024-09-17T04:30:02.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.282+0000 7fe019600640 1 --2- 172.21.15.100:0/1726175052 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdffc078cb0 0x7fdffc07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:02.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.282+0000 7fe019600640 1 -- 172.21.15.100:0/1726175052 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7fe0140bc470 con 0x7fe01c156ca0 2024-09-17T04:30:02.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.283+0000 7fe021600640 1 --2- 172.21.15.100:0/1726175052 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdffc078cb0 0x7fdffc07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:02.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.283+0000 7fe021600640 1 --2- 172.21.15.100:0/1726175052 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdffc078cb0 0x7fdffc07b1c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe010006fd0 tx=0x7fe010008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:02.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.287+0000 7fe019600640 1 -- 172.21.15.100:0/1726175052 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe014084690 con 0x7fe01c156ca0 2024-09-17T04:30:02.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:02.506+0000 7fe022a00640 1 -- 172.21.15.100:0/1726175052 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe01c15c5e0 con 0x7fe01c156ca0 2024-09-17T04:30:03.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.025+0000 7fe019600640 1 -- 172.21.15.100:0/1726175052 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v72) ==== 122+0+0 (secure 0 0 0) 0x7fe014083f90 con 0x7fe01c156ca0 2024-09-17T04:30:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.028+0000 7fe022a00640 1 -- 172.21.15.100:0/1726175052 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdffc078cb0 msgr2=0x7fdffc07b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.028+0000 7fe022a00640 1 --2- 172.21.15.100:0/1726175052 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdffc078cb0 0x7fdffc07b1c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe010006fd0 tx=0x7fe010008040 comp rx=0 tx=0).stop 2024-09-17T04:30:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.028+0000 7fe022a00640 1 -- 172.21.15.100:0/1726175052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe01c156ca0 msgr2=0x7fe01c0746e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.028+0000 7fe022a00640 1 --2- 172.21.15.100:0/1726175052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe01c156ca0 0x7fe01c0746e0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fe01402aff0 tx=0x7fe014003f10 comp rx=0 tx=0).stop 2024-09-17T04:30:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.028+0000 7fe022a00640 1 -- 172.21.15.100:0/1726175052 shutdown_connections 2024-09-17T04:30:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.028+0000 7fe022a00640 1 --2- 172.21.15.100:0/1726175052 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdffc078cb0 0x7fdffc07b1c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.028+0000 7fe022a00640 1 --2- 172.21.15.100:0/1726175052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe01c156ca0 0x7fe01c0746e0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:03.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.028+0000 7fe022a00640 1 --2- 172.21.15.100:0/1726175052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe01c0cd000 0x7fe01c0741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:03.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.028+0000 7fe022a00640 1 --2- 172.21.15.100:0/1726175052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe01c0b3f30 0x7fe01c145290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:03.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.028+0000 7fe022a00640 1 -- 172.21.15.100:0/1726175052 >> 172.21.15.100:0/1726175052 conn(0x7fe01c06d620 msgr2=0x7fe01c070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:03.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.028+0000 7fe022a00640 1 -- 172.21.15.100:0/1726175052 shutdown_connections 2024-09-17T04:30:03.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.028+0000 7fe022a00640 1 -- 172.21.15.100:0/1726175052 wait complete. 2024-09-17T04:30:03.052 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T04:30:03.052 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:30:03.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.270+0000 7fd9d7c00640 1 -- 172.21.15.100:0/780759697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9d015fae0 msgr2=0x7fd9d00b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:03.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.270+0000 7fd9d7c00640 1 --2- 172.21.15.100:0/780759697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9d015fae0 0x7fd9d00b3750 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8005fd0 tx=0x7fd9c802a820 comp rx=0 tx=0).stop 2024-09-17T04:30:03.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.270+0000 7fd9d7c00640 1 -- 172.21.15.100:0/780759697 shutdown_connections 2024-09-17T04:30:03.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.270+0000 7fd9d7c00640 1 --2- 172.21.15.100:0/780759697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9d015fae0 0x7fd9d00b3750 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:03.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.270+0000 7fd9d7c00640 1 --2- 172.21.15.100:0/780759697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9d00aebc0 0x7fd9d0152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:03.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.270+0000 7fd9d7c00640 1 -- 172.21.15.100:0/780759697 >> 172.21.15.100:0/780759697 conn(0x7fd9d006d730 msgr2=0x7fd9d00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:03.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.270+0000 7fd9d7c00640 1 -- 172.21.15.100:0/780759697 shutdown_connections 2024-09-17T04:30:03.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.270+0000 7fd9d7c00640 1 -- 172.21.15.100:0/780759697 wait complete. 2024-09-17T04:30:03.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.272+0000 7fd9d7c00640 1 Processor -- start 2024-09-17T04:30:03.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.273+0000 7fd9d7c00640 1 -- start start 2024-09-17T04:30:03.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.274+0000 7fd9d7c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9d00aebc0 0x7fd9d00ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:03.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.274+0000 7fd9d7c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9d00b2750 0x7fd9d0073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:03.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.274+0000 7fd9d7c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9d015f1f0 0x7fd9d015f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:03.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.274+0000 7fd9d7c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d00b41b0 con 0x7fd9d00aebc0 2024-09-17T04:30:03.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.274+0000 7fd9d7c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d00b4030 con 0x7fd9d00b2750 2024-09-17T04:30:03.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.274+0000 7fd9d7c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd9d00b4330 con 0x7fd9d015f1f0 2024-09-17T04:30:03.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.274+0000 7fd9d5200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9d00aebc0 0x7fd9d00ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:03.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.274+0000 7fd9d5200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9d00aebc0 0x7fd9d00ca9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37768/0 (socket says 172.21.15.100:37768) 2024-09-17T04:30:03.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.274+0000 7fd9d5200640 1 -- 172.21.15.100:0/1927089290 learned_addr learned my addr 172.21.15.100:0/1927089290 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:03.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.275+0000 7fd9d5c00640 1 --2- 172.21.15.100:0/1927089290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9d015f1f0 0x7fd9d015f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:03.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.275+0000 7fd9d5200640 1 -- 172.21.15.100:0/1927089290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9d015f1f0 msgr2=0x7fd9d015f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:03.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.275+0000 7fd9d5200640 1 --2- 172.21.15.100:0/1927089290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9d015f1f0 0x7fd9d015f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:03.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.275+0000 7fd9d5200640 1 -- 172.21.15.100:0/1927089290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9d00b2750 msgr2=0x7fd9d0073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:03.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.275+0000 7fd9c7e00640 1 --2- 172.21.15.100:0/1927089290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9d00b2750 0x7fd9d0073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:03.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.275+0000 7fd9d5200640 1 --2- 172.21.15.100:0/1927089290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9d00b2750 0x7fd9d0073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:03.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.275+0000 7fd9d5200640 1 -- 172.21.15.100:0/1927089290 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d015fb40 con 0x7fd9d00aebc0 2024-09-17T04:30:03.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.275+0000 7fd9d5c00640 1 --2- 172.21.15.100:0/1927089290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9d015f1f0 0x7fd9d015f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:30:03.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.275+0000 7fd9c7e00640 1 --2- 172.21.15.100:0/1927089290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9d00b2750 0x7fd9d0073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:30:03.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.275+0000 7fd9d5200640 1 --2- 172.21.15.100:0/1927089290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9d00aebc0 0x7fd9d00ca9b0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000e7c0 tx=0x7fd9c000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:03.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.275+0000 7fd9cd600640 1 -- 172.21.15.100:0/1927089290 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c000ccc0 con 0x7fd9d00aebc0 2024-09-17T04:30:03.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.275+0000 7fd9cd600640 1 -- 172.21.15.100:0/1927089290 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9c0010430 con 0x7fd9d00aebc0 2024-09-17T04:30:03.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.275+0000 7fd9d7c00640 1 -- 172.21.15.100:0/1927089290 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9d01591e0 con 0x7fd9d00aebc0 2024-09-17T04:30:03.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.275+0000 7fd9cd600640 1 -- 172.21.15.100:0/1927089290 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0010860 con 0x7fd9d00aebc0 2024-09-17T04:30:03.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.275+0000 7fd9d7c00640 1 -- 172.21.15.100:0/1927089290 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9d00c9bf0 con 0x7fd9d00aebc0 2024-09-17T04:30:03.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.276+0000 7fd9d7c00640 1 -- 172.21.15.100:0/1927089290 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9d00b4d40 con 0x7fd9d00aebc0 2024-09-17T04:30:03.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.278+0000 7fd9cd600640 1 -- 172.21.15.100:0/1927089290 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fd9c0010a00 con 0x7fd9d00aebc0 2024-09-17T04:30:03.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.279+0000 7fd9cd600640 1 --2- 172.21.15.100:0/1927089290 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd9ac078ce0 0x7fd9ac07b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:03.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.279+0000 7fd9cd600640 1 -- 172.21.15.100:0/1927089290 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7fd9c009ba30 con 0x7fd9d00aebc0 2024-09-17T04:30:03.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.279+0000 7fd9c7e00640 1 --2- 172.21.15.100:0/1927089290 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd9ac078ce0 0x7fd9ac07b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:03.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.280+0000 7fd9c7e00640 1 --2- 172.21.15.100:0/1927089290 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd9ac078ce0 0x7fd9ac07b1f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00cc320 tx=0x7fd9b8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:03.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.284+0000 7fd9cd600640 1 -- 172.21.15.100:0/1927089290 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd9d015fb40 con 0x7fd9d00aebc0 2024-09-17T04:30:03.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.500+0000 7fd9d7c00640 1 -- 172.21.15.100:0/1927089290 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9d0069950 con 0x7fd9d00aebc0 2024-09-17T04:30:03.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.501+0000 7fd9cd600640 1 -- 172.21.15.100:0/1927089290 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5234 (secure 0 0 0) 0x7fd9c006a070 con 0x7fd9d00aebc0 2024-09-17T04:30:03.501 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:30:03.502 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":72,"btime":"2024-09-17T04:30:03:024749+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6044,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1891177834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1891177834},{"type":"v1","addr":"172.21.15.100:6835","nonce":1891177834}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":6047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1158083377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1158083377},{"type":"v1","addr":"172.21.15.184:6835","nonce":1158083377}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":6697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/714465116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":714465116},{"type":"v1","addr":"172.21.15.184:6801","nonce":714465116}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:29:59.487577+0000","modified":"2024-09-17T04:30:02.508652+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6972":{"gid":6972,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6833/2666467995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2666467995},{"type":"v1","addr":"172.21.15.100:6833","nonce":2666467995}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6972,"qdb_cluster":[6972]},"id":8}]} 2024-09-17T04:30:03.502 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 72 2024-09-17T04:30:03.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.503+0000 7fd9d7c00640 1 -- 172.21.15.100:0/1927089290 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd9ac078ce0 msgr2=0x7fd9ac07b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:03.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.503+0000 7fd9d7c00640 1 --2- 172.21.15.100:0/1927089290 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd9ac078ce0 0x7fd9ac07b1f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00cc320 tx=0x7fd9b8009040 comp rx=0 tx=0).stop 2024-09-17T04:30:03.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.503+0000 7fd9d7c00640 1 -- 172.21.15.100:0/1927089290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9d00aebc0 msgr2=0x7fd9d00ca9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:03.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.503+0000 7fd9d7c00640 1 --2- 172.21.15.100:0/1927089290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9d00aebc0 0x7fd9d00ca9b0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000e7c0 tx=0x7fd9c000ec80 comp rx=0 tx=0).stop 2024-09-17T04:30:03.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.503+0000 7fd9d7c00640 1 -- 172.21.15.100:0/1927089290 shutdown_connections 2024-09-17T04:30:03.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.503+0000 7fd9d7c00640 1 --2- 172.21.15.100:0/1927089290 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd9ac078ce0 0x7fd9ac07b1f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:03.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.503+0000 7fd9d7c00640 1 --2- 172.21.15.100:0/1927089290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9d015f1f0 0x7fd9d015f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:03.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.503+0000 7fd9d7c00640 1 --2- 172.21.15.100:0/1927089290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9d00b2750 0x7fd9d0073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:03.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.503+0000 7fd9d7c00640 1 --2- 172.21.15.100:0/1927089290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9d00aebc0 0x7fd9d00ca9b0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:03.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.503+0000 7fd9d7c00640 1 -- 172.21.15.100:0/1927089290 >> 172.21.15.100:0/1927089290 conn(0x7fd9d006d730 msgr2=0x7fd9d00b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:03.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.504+0000 7fd9d7c00640 1 -- 172.21.15.100:0/1927089290 shutdown_connections 2024-09-17T04:30:03.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.504+0000 7fd9d7c00640 1 -- 172.21.15.100:0/1927089290 wait complete. 2024-09-17T04:30:03.530 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:30:03.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.747+0000 7fd341400640 1 -- 172.21.15.100:0/3472772951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd33c0b64e0 msgr2=0x7fd33c0b68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:03.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.748+0000 7fd341400640 1 --2- 172.21.15.100:0/3472772951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd33c0b64e0 0x7fd33c0b68f0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fd334009ca0 tx=0x7fd33402aa60 comp rx=0 tx=0).stop 2024-09-17T04:30:03.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.748+0000 7fd341400640 1 -- 172.21.15.100:0/3472772951 shutdown_connections 2024-09-17T04:30:03.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.748+0000 7fd341400640 1 --2- 172.21.15.100:0/3472772951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd33c0b64e0 0x7fd33c0b68f0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:03.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.748+0000 7fd341400640 1 --2- 172.21.15.100:0/3472772951 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd33c0cf6f0 0x7fd33c0b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:03.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.748+0000 7fd341400640 1 --2- 172.21.15.100:0/3472772951 >> v2:172.21.15.184:3300/0 conn(0x7fd33c0b3820 0x7fd33c0cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:03.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.748+0000 7fd341400640 1 -- 172.21.15.100:0/3472772951 >> 172.21.15.100:0/3472772951 conn(0x7fd33c09d8a0 msgr2=0x7fd33c08bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:03.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.748+0000 7fd341400640 1 -- 172.21.15.100:0/3472772951 shutdown_connections 2024-09-17T04:30:03.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.748+0000 7fd341400640 1 -- 172.21.15.100:0/3472772951 wait complete. 2024-09-17T04:30:03.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.750+0000 7fd341400640 1 Processor -- start 2024-09-17T04:30:03.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.750+0000 7fd341400640 1 -- start start 2024-09-17T04:30:03.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.751+0000 7fd341400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd33c0b3820 0x7fd33c160900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:03.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.751+0000 7fd341400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd33c0b64e0 0x7fd33c1f5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:03.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.751+0000 7fd341400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd33c0cf6f0 0x7fd33c0ce120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:03.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.751+0000 7fd341400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd33c0d4340 con 0x7fd33c0b64e0 2024-09-17T04:30:03.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.751+0000 7fd341400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd33c0b4740 con 0x7fd33c0b3820 2024-09-17T04:30:03.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.751+0000 7fd341400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd33c0b3ef0 con 0x7fd33c0cf6f0 2024-09-17T04:30:03.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.752+0000 7fd333400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd33c0b64e0 0x7fd33c1f5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:03.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.752+0000 7fd333400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd33c0b64e0 0x7fd33c1f5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37792/0 (socket says 172.21.15.100:37792) 2024-09-17T04:30:03.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.752+0000 7fd333400640 1 -- 172.21.15.100:0/3014072542 learned_addr learned my addr 172.21.15.100:0/3014072542 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:03.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.752+0000 7fd33be00640 1 --2- 172.21.15.100:0/3014072542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd33c0b3820 0x7fd33c160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:03.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.752+0000 7fd340a00640 1 --2- 172.21.15.100:0/3014072542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd33c0cf6f0 0x7fd33c0ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:03.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.752+0000 7fd333400640 1 -- 172.21.15.100:0/3014072542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd33c0cf6f0 msgr2=0x7fd33c0ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:03.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.752+0000 7fd333400640 1 --2- 172.21.15.100:0/3014072542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd33c0cf6f0 0x7fd33c0ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:03.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.752+0000 7fd333400640 1 -- 172.21.15.100:0/3014072542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd33c0b3820 msgr2=0x7fd33c160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:03.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.752+0000 7fd333400640 1 --2- 172.21.15.100:0/3014072542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd33c0b3820 0x7fd33c160900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:03.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.752+0000 7fd333400640 1 -- 172.21.15.100:0/3014072542 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd33c1f6140 con 0x7fd33c0b64e0 2024-09-17T04:30:03.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.753+0000 7fd333400640 1 --2- 172.21.15.100:0/3014072542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd33c0b64e0 0x7fd33c1f5600 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fd32400d6a0 tx=0x7fd32400db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:03.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.753+0000 7fd338c00640 1 -- 172.21.15.100:0/3014072542 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd324017070 con 0x7fd33c0b64e0 2024-09-17T04:30:03.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.753+0000 7fd338c00640 1 -- 172.21.15.100:0/3014072542 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd324009b10 con 0x7fd33c0b64e0 2024-09-17T04:30:03.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.753+0000 7fd341400640 1 -- 172.21.15.100:0/3014072542 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd33c1f6420 con 0x7fd33c0b64e0 2024-09-17T04:30:03.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.753+0000 7fd338c00640 1 -- 172.21.15.100:0/3014072542 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd324010460 con 0x7fd33c0b64e0 2024-09-17T04:30:03.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.753+0000 7fd341400640 1 -- 172.21.15.100:0/3014072542 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd33c0cade0 con 0x7fd33c0b64e0 2024-09-17T04:30:03.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.754+0000 7fd341400640 1 -- 172.21.15.100:0/3014072542 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd33c1f5eb0 con 0x7fd33c0b64e0 2024-09-17T04:30:03.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.755+0000 7fd338c00640 1 -- 172.21.15.100:0/3014072542 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fd32401a070 con 0x7fd33c0b64e0 2024-09-17T04:30:03.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.756+0000 7fd338c00640 1 --2- 172.21.15.100:0/3014072542 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd318078ce0 0x7fd31807b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:03.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.756+0000 7fd338c00640 1 -- 172.21.15.100:0/3014072542 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7fd32409af70 con 0x7fd33c0b64e0 2024-09-17T04:30:03.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.756+0000 7fd33be00640 1 --2- 172.21.15.100:0/3014072542 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd318078ce0 0x7fd31807b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:03.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.757+0000 7fd33be00640 1 --2- 172.21.15.100:0/3014072542 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd318078ce0 0x7fd31807b1f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd32c005fd0 tx=0x7fd32c00b260 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:03.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.761+0000 7fd338c00640 1 -- 172.21.15.100:0/3014072542 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd324062bd0 con 0x7fd33c0b64e0 2024-09-17T04:30:03.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.944+0000 7fd341400640 1 -- 172.21.15.100:0/3014072542 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd33c0b68f0 con 0x7fd33c0b64e0 2024-09-17T04:30:03.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.946+0000 7fd338c00640 1 -- 172.21.15.100:0/3014072542 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+16735 (secure 0 0 0) 0x7fd324067600 con 0x7fd33c0b64e0 2024-09-17T04:30:03.946 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:30:03.946 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":105,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:30:01.010692+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:29:57.388120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-17T04:29:58.449229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:30:03.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.948+0000 7fd341400640 1 -- 172.21.15.100:0/3014072542 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd318078ce0 msgr2=0x7fd31807b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:03.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.948+0000 7fd341400640 1 --2- 172.21.15.100:0/3014072542 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd318078ce0 0x7fd31807b1f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd32c005fd0 tx=0x7fd32c00b260 comp rx=0 tx=0).stop 2024-09-17T04:30:03.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.948+0000 7fd341400640 1 -- 172.21.15.100:0/3014072542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd33c0b64e0 msgr2=0x7fd33c1f5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:03.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.948+0000 7fd341400640 1 --2- 172.21.15.100:0/3014072542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd33c0b64e0 0x7fd33c1f5600 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fd32400d6a0 tx=0x7fd32400db60 comp rx=0 tx=0).stop 2024-09-17T04:30:03.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.948+0000 7fd341400640 1 -- 172.21.15.100:0/3014072542 shutdown_connections 2024-09-17T04:30:03.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.948+0000 7fd341400640 1 --2- 172.21.15.100:0/3014072542 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd318078ce0 0x7fd31807b1f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:03.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.948+0000 7fd341400640 1 --2- 172.21.15.100:0/3014072542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd33c0cf6f0 0x7fd33c0ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:03.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.948+0000 7fd341400640 1 --2- 172.21.15.100:0/3014072542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd33c0b64e0 0x7fd33c1f5600 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:03.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.948+0000 7fd341400640 1 --2- 172.21.15.100:0/3014072542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd33c0b3820 0x7fd33c160900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:03.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.948+0000 7fd341400640 1 -- 172.21.15.100:0/3014072542 >> 172.21.15.100:0/3014072542 conn(0x7fd33c09d8a0 msgr2=0x7fd33c0d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:03.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.948+0000 7fd341400640 1 -- 172.21.15.100:0/3014072542 shutdown_connections 2024-09-17T04:30:03.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:03.948+0000 7fd341400640 1 -- 172.21.15.100:0/3014072542 wait complete. 2024-09-17T04:30:03.975 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T04:30:03.975 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:30:04.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.196+0000 7f95e4600640 1 -- 172.21.15.100:0/2352326368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95dc0ad3c0 msgr2=0x7f95dc0ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:04.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.196+0000 7f95e4600640 1 --2- 172.21.15.100:0/2352326368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95dc0ad3c0 0x7f95dc0ad7d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f95c800a940 tx=0x7f95c802b7f0 comp rx=0 tx=0).stop 2024-09-17T04:30:04.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.196+0000 7f95e4600640 1 -- 172.21.15.100:0/2352326368 shutdown_connections 2024-09-17T04:30:04.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.196+0000 7f95e4600640 1 --2- 172.21.15.100:0/2352326368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95dc0ad3c0 0x7f95dc0ad7d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:04.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.196+0000 7f95e4600640 1 -- 172.21.15.100:0/2352326368 >> 172.21.15.100:0/2352326368 conn(0x7f95dc06d730 msgr2=0x7f95dc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:04.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.196+0000 7f95e4600640 1 -- 172.21.15.100:0/2352326368 shutdown_connections 2024-09-17T04:30:04.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.196+0000 7f95e4600640 1 -- 172.21.15.100:0/2352326368 wait complete. 2024-09-17T04:30:04.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.198+0000 7f95e4600640 1 Processor -- start 2024-09-17T04:30:04.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.198+0000 7f95e4600640 1 -- start start 2024-09-17T04:30:04.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.199+0000 7f95e4600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95dc0704b0 0x7f95dc1465a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:04.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.200+0000 7f95e4600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95dc0ad3c0 0x7f95dc1f6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:04.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.200+0000 7f95e4600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95dc0add10 0x7f95dc1f7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:04.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.200+0000 7f95e4600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc0712f0 con 0x7f95dc0add10 2024-09-17T04:30:04.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.200+0000 7f95e4600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc071170 con 0x7f95dc0704b0 2024-09-17T04:30:04.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.200+0000 7f95e4600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f95dc165e00 con 0x7f95dc0ad3c0 2024-09-17T04:30:04.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.200+0000 7f95e2600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95dc0add10 0x7f95dc1f7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:04.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.200+0000 7f95e2600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95dc0add10 0x7f95dc1f7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37796/0 (socket says 172.21.15.100:37796) 2024-09-17T04:30:04.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.200+0000 7f95e2600640 1 -- 172.21.15.100:0/3483299637 learned_addr learned my addr 172.21.15.100:0/3483299637 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:04.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.200+0000 7f95e1c00640 1 --2- 172.21.15.100:0/3483299637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95dc0704b0 0x7f95dc1465a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:04.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.200+0000 7f95e1200640 1 --2- 172.21.15.100:0/3483299637 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95dc0ad3c0 0x7f95dc1f6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:04.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.200+0000 7f95e2600640 1 -- 172.21.15.100:0/3483299637 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95dc0ad3c0 msgr2=0x7f95dc1f6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:04.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.200+0000 7f95e2600640 1 --2- 172.21.15.100:0/3483299637 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95dc0ad3c0 0x7f95dc1f6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:04.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.200+0000 7f95e2600640 1 -- 172.21.15.100:0/3483299637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95dc0704b0 msgr2=0x7f95dc1465a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:04.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.200+0000 7f95e2600640 1 --2- 172.21.15.100:0/3483299637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95dc0704b0 0x7f95dc1465a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:04.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.200+0000 7f95e2600640 1 -- 172.21.15.100:0/3483299637 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95dc14c0b0 con 0x7f95dc0add10 2024-09-17T04:30:04.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.201+0000 7f95e2600640 1 --2- 172.21.15.100:0/3483299637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95dc0add10 0x7f95dc1f7360 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f95d400cbd0 tx=0x7f95d400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:04.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.201+0000 7f95d9600640 1 -- 172.21.15.100:0/3483299637 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d4016020 con 0x7f95dc0add10 2024-09-17T04:30:04.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.201+0000 7f95d9600640 1 -- 172.21.15.100:0/3483299637 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95d400ae50 con 0x7f95dc0add10 2024-09-17T04:30:04.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.201+0000 7f95d9600640 1 -- 172.21.15.100:0/3483299637 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d4012680 con 0x7f95dc0add10 2024-09-17T04:30:04.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.201+0000 7f95e4600640 1 -- 172.21.15.100:0/3483299637 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95dc14c390 con 0x7f95dc0add10 2024-09-17T04:30:04.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.201+0000 7f95e4600640 1 -- 172.21.15.100:0/3483299637 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95dc153110 con 0x7f95dc0add10 2024-09-17T04:30:04.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.202+0000 7f95e4600640 1 -- 172.21.15.100:0/3483299637 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95dc153370 con 0x7f95dc0add10 2024-09-17T04:30:04.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.203+0000 7f95d9600640 1 -- 172.21.15.100:0/3483299637 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f95d4019020 con 0x7f95dc0add10 2024-09-17T04:30:04.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.204+0000 7f95d9600640 1 --2- 172.21.15.100:0/3483299637 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f95bc078ce0 0x7f95bc07b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:04.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.204+0000 7f95d9600640 1 -- 172.21.15.100:0/3483299637 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f95d409b9a0 con 0x7f95dc0add10 2024-09-17T04:30:04.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.205+0000 7f95e1c00640 1 --2- 172.21.15.100:0/3483299637 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f95bc078ce0 0x7f95bc07b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:04.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.205+0000 7f95e1c00640 1 --2- 172.21.15.100:0/3483299637 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f95bc078ce0 0x7f95bc07b1f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f95d0001720 tx=0x7f95d0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:04.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.209+0000 7f95d9600640 1 -- 172.21.15.100:0/3483299637 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f95d4020020 con 0x7f95dc0add10 2024-09-17T04:30:04.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.389+0000 7f95e4600640 1 -- 172.21.15.100:0/3483299637 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f95dc0aed90 con 0x7f95bc078ce0 2024-09-17T04:30:04.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.394+0000 7f95d9600640 1 -- 172.21.15.100:0/3483299637 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252473 (secure 0 0 0) 0x7f95d0001d00 con 0x7f95bc078ce0 2024-09-17T04:30:04.398 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:30:04.399 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:30:04.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.398+0000 7f95e4600640 1 -- 172.21.15.100:0/3483299637 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f95bc078ce0 msgr2=0x7f95bc07b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:04.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.398+0000 7f95e4600640 1 --2- 172.21.15.100:0/3483299637 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f95bc078ce0 0x7f95bc07b1f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f95d0001720 tx=0x7f95d0009040 comp rx=0 tx=0).stop 2024-09-17T04:30:04.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.398+0000 7f95e4600640 1 -- 172.21.15.100:0/3483299637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95dc0add10 msgr2=0x7f95dc1f7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:04.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.398+0000 7f95e4600640 1 --2- 172.21.15.100:0/3483299637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95dc0add10 0x7f95dc1f7360 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f95d400cbd0 tx=0x7f95d400a430 comp rx=0 tx=0).stop 2024-09-17T04:30:04.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.398+0000 7f95e4600640 1 -- 172.21.15.100:0/3483299637 shutdown_connections 2024-09-17T04:30:04.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.398+0000 7f95e4600640 1 --2- 172.21.15.100:0/3483299637 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f95bc078ce0 0x7f95bc07b1f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:04.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.398+0000 7f95e4600640 1 --2- 172.21.15.100:0/3483299637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95dc0add10 0x7f95dc1f7360 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:04.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.398+0000 7f95e4600640 1 --2- 172.21.15.100:0/3483299637 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95dc0ad3c0 0x7f95dc1f6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:04.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.398+0000 7f95e4600640 1 --2- 172.21.15.100:0/3483299637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95dc0704b0 0x7f95dc1465a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:04.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.398+0000 7f95e4600640 1 -- 172.21.15.100:0/3483299637 >> 172.21.15.100:0/3483299637 conn(0x7f95dc06d730 msgr2=0x7f95dc1500a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:04.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.398+0000 7f95e4600640 1 -- 172.21.15.100:0/3483299637 shutdown_connections 2024-09-17T04:30:04.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.398+0000 7f95e4600640 1 -- 172.21.15.100:0/3483299637 wait complete. 2024-09-17T04:30:04.421 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2024-09-17T04:30:03.672223+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260804,"kb_used_data":11276,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713916,"statfs":{"total":773094113280,"available":772827049984,"internally_reserved":0,"allocated":11546624,"data_stored":5678704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.691239"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.035978+0000","last_change":"2024-09-17T04:30:00.031045+0000","last_active":"2024-09-17T04:30:00.035978+0000","last_peered":"2024-09-17T04:30:00.035978+0000","last_clean":"2024-09-17T04:30:00.035978+0000","last_became_active":"2024-09-17T04:30:00.029801+0000","last_became_peered":"2024-09-17T04:30:00.029801+0000","last_unstale":"2024-09-17T04:30:00.035978+0000","last_undegraded":"2024-09-17T04:30:00.035978+0000","last_fullsized":"2024-09-17T04:30:00.035978+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:20:30.612554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.023477+0000","last_change":"2024-09-17T04:30:00.030378+0000","last_active":"2024-09-17T04:30:01.023477+0000","last_peered":"2024-09-17T04:30:01.023477+0000","last_clean":"2024-09-17T04:30:01.023477+0000","last_became_active":"2024-09-17T04:30:00.030043+0000","last_became_peered":"2024-09-17T04:30:00.030043+0000","last_unstale":"2024-09-17T04:30:01.023477+0000","last_undegraded":"2024-09-17T04:30:01.023477+0000","last_fullsized":"2024-09-17T04:30:01.023477+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:44.121537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.036815+0000","last_change":"2024-09-17T04:30:00.032305+0000","last_active":"2024-09-17T04:30:00.036815+0000","last_peered":"2024-09-17T04:30:00.036815+0000","last_clean":"2024-09-17T04:30:00.036815+0000","last_became_active":"2024-09-17T04:30:00.031944+0000","last_became_peered":"2024-09-17T04:30:00.031944+0000","last_unstale":"2024-09-17T04:30:00.036815+0000","last_undegraded":"2024-09-17T04:30:00.036815+0000","last_fullsized":"2024-09-17T04:30:00.036815+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:11:10.891912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.034115+0000","last_change":"2024-09-17T04:30:00.025576+0000","last_active":"2024-09-17T04:30:00.034115+0000","last_peered":"2024-09-17T04:30:00.034115+0000","last_clean":"2024-09-17T04:30:00.034115+0000","last_became_active":"2024-09-17T04:30:00.025136+0000","last_became_peered":"2024-09-17T04:30:00.025136+0000","last_unstale":"2024-09-17T04:30:00.034115+0000","last_undegraded":"2024-09-17T04:30:00.034115+0000","last_fullsized":"2024-09-17T04:30:00.034115+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:45:52.659055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.021284+0000","last_change":"2024-09-17T04:30:00.011478+0000","last_active":"2024-09-17T04:30:01.021284+0000","last_peered":"2024-09-17T04:30:01.021284+0000","last_clean":"2024-09-17T04:30:01.021284+0000","last_became_active":"2024-09-17T04:30:00.011137+0000","last_became_peered":"2024-09-17T04:30:00.011137+0000","last_unstale":"2024-09-17T04:30:01.021284+0000","last_undegraded":"2024-09-17T04:30:01.021284+0000","last_fullsized":"2024-09-17T04:30:01.021284+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:51:11.302837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.240767+0000","last_change":"2024-09-17T04:30:00.036444+0000","last_active":"2024-09-17T04:30:01.240767+0000","last_peered":"2024-09-17T04:30:01.240767+0000","last_clean":"2024-09-17T04:30:01.240767+0000","last_became_active":"2024-09-17T04:30:00.035097+0000","last_became_peered":"2024-09-17T04:30:00.035097+0000","last_unstale":"2024-09-17T04:30:01.240767+0000","last_undegraded":"2024-09-17T04:30:01.240767+0000","last_fullsized":"2024-09-17T04:30:01.240767+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:58:26.889907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.023535+0000","last_change":"2024-09-17T04:30:00.023963+0000","last_active":"2024-09-17T04:30:01.023535+0000","last_peered":"2024-09-17T04:30:01.023535+0000","last_clean":"2024-09-17T04:30:01.023535+0000","last_became_active":"2024-09-17T04:30:00.023262+0000","last_became_peered":"2024-09-17T04:30:00.023262+0000","last_unstale":"2024-09-17T04:30:01.023535+0000","last_undegraded":"2024-09-17T04:30:01.023535+0000","last_fullsized":"2024-09-17T04:30:01.023535+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:40:00.038594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.027163+0000","last_change":"2024-09-17T04:30:00.026007+0000","last_active":"2024-09-17T04:30:00.027163+0000","last_peered":"2024-09-17T04:30:00.027163+0000","last_clean":"2024-09-17T04:30:00.027163+0000","last_became_active":"2024-09-17T04:30:00.025689+0000","last_became_peered":"2024-09-17T04:30:00.025689+0000","last_unstale":"2024-09-17T04:30:00.027163+0000","last_undegraded":"2024-09-17T04:30:00.027163+0000","last_fullsized":"2024-09-17T04:30:00.027163+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:50.175832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.047778+0000","last_change":"2024-09-17T04:30:00.038957+0000","last_active":"2024-09-17T04:30:00.047778+0000","last_peered":"2024-09-17T04:30:00.047778+0000","last_clean":"2024-09-17T04:30:00.047778+0000","last_became_active":"2024-09-17T04:30:00.038525+0000","last_became_peered":"2024-09-17T04:30:00.038525+0000","last_unstale":"2024-09-17T04:30:00.047778+0000","last_undegraded":"2024-09-17T04:30:00.047778+0000","last_fullsized":"2024-09-17T04:30:00.047778+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:49.903236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.046821+0000","last_change":"2024-09-17T04:30:00.033765+0000","last_active":"2024-09-17T04:30:00.046821+0000","last_peered":"2024-09-17T04:30:00.046821+0000","last_clean":"2024-09-17T04:30:00.046821+0000","last_became_active":"2024-09-17T04:30:00.033172+0000","last_became_peered":"2024-09-17T04:30:00.033172+0000","last_unstale":"2024-09-17T04:30:00.046821+0000","last_undegraded":"2024-09-17T04:30:00.046821+0000","last_fullsized":"2024-09-17T04:30:00.046821+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:49.787137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.023119+0000","last_change":"2024-09-17T04:30:00.023924+0000","last_active":"2024-09-17T04:30:01.023119+0000","last_peered":"2024-09-17T04:30:01.023119+0000","last_clean":"2024-09-17T04:30:01.023119+0000","last_became_active":"2024-09-17T04:30:00.023409+0000","last_became_peered":"2024-09-17T04:30:00.023409+0000","last_unstale":"2024-09-17T04:30:01.023119+0000","last_undegraded":"2024-09-17T04:30:01.023119+0000","last_fullsized":"2024-09-17T04:30:01.023119+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:05:21.232149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.044201+0000","last_change":"2024-09-17T04:30:00.031748+0000","last_active":"2024-09-17T04:30:00.044201+0000","last_peered":"2024-09-17T04:30:00.044201+0000","last_clean":"2024-09-17T04:30:00.044201+0000","last_became_active":"2024-09-17T04:30:00.031370+0000","last_became_peered":"2024-09-17T04:30:00.031370+0000","last_unstale":"2024-09-17T04:30:00.044201+0000","last_undegraded":"2024-09-17T04:30:00.044201+0000","last_fullsized":"2024-09-17T04:30:00.044201+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:12:57.242715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.038862+0000","last_change":"2024-09-17T04:30:00.033820+0000","last_active":"2024-09-17T04:30:00.038862+0000","last_peered":"2024-09-17T04:30:00.038862+0000","last_clean":"2024-09-17T04:30:00.038862+0000","last_became_active":"2024-09-17T04:30:00.033435+0000","last_became_peered":"2024-09-17T04:30:00.033435+0000","last_unstale":"2024-09-17T04:30:00.038862+0000","last_undegraded":"2024-09-17T04:30:00.038862+0000","last_fullsized":"2024-09-17T04:30:00.038862+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:25.595122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.039465+0000","last_change":"2024-09-17T04:30:00.032263+0000","last_active":"2024-09-17T04:30:00.039465+0000","last_peered":"2024-09-17T04:30:00.039465+0000","last_clean":"2024-09-17T04:30:00.039465+0000","last_became_active":"2024-09-17T04:30:00.031102+0000","last_became_peered":"2024-09-17T04:30:00.031102+0000","last_unstale":"2024-09-17T04:30:00.039465+0000","last_undegraded":"2024-09-17T04:30:00.039465+0000","last_fullsized":"2024-09-17T04:30:00.039465+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:25.300591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.240776+0000","last_change":"2024-09-17T04:30:00.035400+0000","last_active":"2024-09-17T04:30:01.240776+0000","last_peered":"2024-09-17T04:30:01.240776+0000","last_clean":"2024-09-17T04:30:01.240776+0000","last_became_active":"2024-09-17T04:30:00.035073+0000","last_became_peered":"2024-09-17T04:30:00.035073+0000","last_unstale":"2024-09-17T04:30:01.240776+0000","last_undegraded":"2024-09-17T04:30:01.240776+0000","last_fullsized":"2024-09-17T04:30:01.240776+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:18:55.077953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.042374+0000","last_change":"2024-09-17T04:30:00.031905+0000","last_active":"2024-09-17T04:30:00.042374+0000","last_peered":"2024-09-17T04:30:00.042374+0000","last_clean":"2024-09-17T04:30:00.042374+0000","last_became_active":"2024-09-17T04:30:00.030633+0000","last_became_peered":"2024-09-17T04:30:00.030633+0000","last_unstale":"2024-09-17T04:30:00.042374+0000","last_undegraded":"2024-09-17T04:30:00.042374+0000","last_fullsized":"2024-09-17T04:30:00.042374+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:23:39.082595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.023187+0000","last_change":"2024-09-17T04:30:00.032923+0000","last_active":"2024-09-17T04:30:01.023187+0000","last_peered":"2024-09-17T04:30:01.023187+0000","last_clean":"2024-09-17T04:30:01.023187+0000","last_became_active":"2024-09-17T04:30:00.030726+0000","last_became_peered":"2024-09-17T04:30:00.030726+0000","last_unstale":"2024-09-17T04:30:01.023187+0000","last_undegraded":"2024-09-17T04:30:01.023187+0000","last_fullsized":"2024-09-17T04:30:01.023187+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:35.261999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.027335+0000","last_change":"2024-09-17T04:30:00.033675+0000","last_active":"2024-09-17T04:30:01.027335+0000","last_peered":"2024-09-17T04:30:01.027335+0000","last_clean":"2024-09-17T04:30:01.027335+0000","last_became_active":"2024-09-17T04:30:00.033337+0000","last_became_peered":"2024-09-17T04:30:00.033337+0000","last_unstale":"2024-09-17T04:30:01.027335+0000","last_undegraded":"2024-09-17T04:30:01.027335+0000","last_fullsized":"2024-09-17T04:30:01.027335+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:55.361480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.037764+0000","last_change":"2024-09-17T04:30:00.034215+0000","last_active":"2024-09-17T04:30:00.037764+0000","last_peered":"2024-09-17T04:30:00.037764+0000","last_clean":"2024-09-17T04:30:00.037764+0000","last_became_active":"2024-09-17T04:30:00.033859+0000","last_became_peered":"2024-09-17T04:30:00.033859+0000","last_unstale":"2024-09-17T04:30:00.037764+0000","last_undegraded":"2024-09-17T04:30:00.037764+0000","last_fullsized":"2024-09-17T04:30:00.037764+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:53:33.901940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.023592+0000","last_change":"2024-09-17T04:30:00.026637+0000","last_active":"2024-09-17T04:30:01.023592+0000","last_peered":"2024-09-17T04:30:01.023592+0000","last_clean":"2024-09-17T04:30:01.023592+0000","last_became_active":"2024-09-17T04:30:00.024736+0000","last_became_peered":"2024-09-17T04:30:00.024736+0000","last_unstale":"2024-09-17T04:30:01.023592+0000","last_undegraded":"2024-09-17T04:30:01.023592+0000","last_fullsized":"2024-09-17T04:30:01.023592+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:32:09.600854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.023316+0000","last_change":"2024-09-17T04:30:00.013520+0000","last_active":"2024-09-17T04:30:01.023316+0000","last_peered":"2024-09-17T04:30:01.023316+0000","last_clean":"2024-09-17T04:30:01.023316+0000","last_became_active":"2024-09-17T04:30:00.013208+0000","last_became_peered":"2024-09-17T04:30:00.013208+0000","last_unstale":"2024-09-17T04:30:01.023316+0000","last_undegraded":"2024-09-17T04:30:01.023316+0000","last_fullsized":"2024-09-17T04:30:01.023316+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:56:34.201971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.032949+0000","last_change":"2024-09-17T04:30:00.026054+0000","last_active":"2024-09-17T04:30:00.032949+0000","last_peered":"2024-09-17T04:30:00.032949+0000","last_clean":"2024-09-17T04:30:00.032949+0000","last_became_active":"2024-09-17T04:30:00.025677+0000","last_became_peered":"2024-09-17T04:30:00.025677+0000","last_unstale":"2024-09-17T04:30:00.032949+0000","last_undegraded":"2024-09-17T04:30:00.032949+0000","last_fullsized":"2024-09-17T04:30:00.032949+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:41.639339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.022021+0000","last_change":"2024-09-17T04:30:00.026955+0000","last_active":"2024-09-17T04:30:01.022021+0000","last_peered":"2024-09-17T04:30:01.022021+0000","last_clean":"2024-09-17T04:30:01.022021+0000","last_became_active":"2024-09-17T04:30:00.024869+0000","last_became_peered":"2024-09-17T04:30:00.024869+0000","last_unstale":"2024-09-17T04:30:01.022021+0000","last_undegraded":"2024-09-17T04:30:01.022021+0000","last_fullsized":"2024-09-17T04:30:01.022021+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:45.610227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.031583+0000","last_change":"2024-09-17T04:30:00.022611+0000","last_active":"2024-09-17T04:30:00.031583+0000","last_peered":"2024-09-17T04:30:00.031583+0000","last_clean":"2024-09-17T04:30:00.031583+0000","last_became_active":"2024-09-17T04:30:00.021870+0000","last_became_peered":"2024-09-17T04:30:00.021870+0000","last_unstale":"2024-09-17T04:30:00.031583+0000","last_undegraded":"2024-09-17T04:30:00.031583+0000","last_fullsized":"2024-09-17T04:30:00.031583+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:36.864227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.023226+0000","last_change":"2024-09-17T04:30:00.023068+0000","last_active":"2024-09-17T04:30:01.023226+0000","last_peered":"2024-09-17T04:30:01.023226+0000","last_clean":"2024-09-17T04:30:01.023226+0000","last_became_active":"2024-09-17T04:30:00.022718+0000","last_became_peered":"2024-09-17T04:30:00.022718+0000","last_unstale":"2024-09-17T04:30:01.023226+0000","last_undegraded":"2024-09-17T04:30:01.023226+0000","last_fullsized":"2024-09-17T04:30:01.023226+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:54:19.658662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.032965+0000","last_change":"2024-09-17T04:30:00.026274+0000","last_active":"2024-09-17T04:30:00.032965+0000","last_peered":"2024-09-17T04:30:00.032965+0000","last_clean":"2024-09-17T04:30:00.032965+0000","last_became_active":"2024-09-17T04:30:00.025962+0000","last_became_peered":"2024-09-17T04:30:00.025962+0000","last_unstale":"2024-09-17T04:30:00.032965+0000","last_undegraded":"2024-09-17T04:30:00.032965+0000","last_fullsized":"2024-09-17T04:30:00.032965+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:05.524758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.241286+0000","last_change":"2024-09-17T04:30:00.041368+0000","last_active":"2024-09-17T04:30:01.241286+0000","last_peered":"2024-09-17T04:30:01.241286+0000","last_clean":"2024-09-17T04:30:01.241286+0000","last_became_active":"2024-09-17T04:30:00.041015+0000","last_became_peered":"2024-09-17T04:30:00.041015+0000","last_unstale":"2024-09-17T04:30:01.241286+0000","last_undegraded":"2024-09-17T04:30:01.241286+0000","last_fullsized":"2024-09-17T04:30:01.241286+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:07.024004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.034010+0000","last_change":"2024-09-17T04:30:00.029929+0000","last_active":"2024-09-17T04:30:00.034010+0000","last_peered":"2024-09-17T04:30:00.034010+0000","last_clean":"2024-09-17T04:30:00.034010+0000","last_became_active":"2024-09-17T04:30:00.029639+0000","last_became_peered":"2024-09-17T04:30:00.029639+0000","last_unstale":"2024-09-17T04:30:00.034010+0000","last_undegraded":"2024-09-17T04:30:00.034010+0000","last_fullsized":"2024-09-17T04:30:00.034010+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:12:00.589604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.022811+0000","last_change":"2024-09-17T04:30:00.023591+0000","last_active":"2024-09-17T04:30:01.022811+0000","last_peered":"2024-09-17T04:30:01.022811+0000","last_clean":"2024-09-17T04:30:01.022811+0000","last_became_active":"2024-09-17T04:30:00.021925+0000","last_became_peered":"2024-09-17T04:30:00.021925+0000","last_unstale":"2024-09-17T04:30:01.022811+0000","last_undegraded":"2024-09-17T04:30:01.022811+0000","last_fullsized":"2024-09-17T04:30:01.022811+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:00.376670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.024004+0000","last_change":"2024-09-17T04:30:00.013112+0000","last_active":"2024-09-17T04:30:01.024004+0000","last_peered":"2024-09-17T04:30:01.024004+0000","last_clean":"2024-09-17T04:30:01.024004+0000","last_became_active":"2024-09-17T04:30:00.012655+0000","last_became_peered":"2024-09-17T04:30:00.012655+0000","last_unstale":"2024-09-17T04:30:01.024004+0000","last_undegraded":"2024-09-17T04:30:01.024004+0000","last_fullsized":"2024-09-17T04:30:01.024004+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:05:25.837826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.032358+0000","last_change":"2024-09-17T04:30:00.028487+0000","last_active":"2024-09-17T04:30:00.032358+0000","last_peered":"2024-09-17T04:30:00.032358+0000","last_clean":"2024-09-17T04:30:00.032358+0000","last_became_active":"2024-09-17T04:30:00.028125+0000","last_became_peered":"2024-09-17T04:30:00.028125+0000","last_unstale":"2024-09-17T04:30:00.032358+0000","last_undegraded":"2024-09-17T04:30:00.032358+0000","last_fullsized":"2024-09-17T04:30:00.032358+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:38.160220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.240782+0000","last_change":"2024-09-17T04:30:00.028467+0000","last_active":"2024-09-17T04:30:01.240782+0000","last_peered":"2024-09-17T04:30:01.240782+0000","last_clean":"2024-09-17T04:30:01.240782+0000","last_became_active":"2024-09-17T04:30:00.028170+0000","last_became_peered":"2024-09-17T04:30:00.028170+0000","last_unstale":"2024-09-17T04:30:01.240782+0000","last_undegraded":"2024-09-17T04:30:01.240782+0000","last_fullsized":"2024-09-17T04:30:01.240782+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:18:30.492299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.241352+0000","last_change":"2024-09-17T04:30:00.031831+0000","last_active":"2024-09-17T04:30:01.241352+0000","last_peered":"2024-09-17T04:30:01.241352+0000","last_clean":"2024-09-17T04:30:01.241352+0000","last_became_active":"2024-09-17T04:30:00.031530+0000","last_became_peered":"2024-09-17T04:30:00.031530+0000","last_unstale":"2024-09-17T04:30:01.241352+0000","last_undegraded":"2024-09-17T04:30:01.241352+0000","last_fullsized":"2024-09-17T04:30:01.241352+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:06.907531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"104'4","reported_seq":19,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.024060+0000","last_change":"2024-09-17T04:29:59.000936+0000","last_active":"2024-09-17T04:30:01.024060+0000","last_peered":"2024-09-17T04:30:01.024060+0000","last_clean":"2024-09-17T04:30:01.024060+0000","last_became_active":"2024-09-17T04:29:59.000619+0000","last_became_peered":"2024-09-17T04:29:59.000619+0000","last_unstale":"2024-09-17T04:30:01.024060+0000","last_undegraded":"2024-09-17T04:30:01.024060+0000","last_fullsized":"2024-09-17T04:30:01.024060+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:54:35.865664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.027605+0000","last_change":"2024-09-17T04:30:00.025354+0000","last_active":"2024-09-17T04:30:01.027605+0000","last_peered":"2024-09-17T04:30:01.027605+0000","last_clean":"2024-09-17T04:30:01.027605+0000","last_became_active":"2024-09-17T04:30:00.024223+0000","last_became_peered":"2024-09-17T04:30:00.024223+0000","last_unstale":"2024-09-17T04:30:01.027605+0000","last_undegraded":"2024-09-17T04:30:01.027605+0000","last_fullsized":"2024-09-17T04:30:01.027605+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:35.261999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"104'1","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.041962+0000","last_change":"2024-09-17T04:29:59.005274+0000","last_active":"2024-09-17T04:30:00.041962+0000","last_peered":"2024-09-17T04:30:00.041962+0000","last_clean":"2024-09-17T04:30:00.041962+0000","last_became_active":"2024-09-17T04:29:59.004956+0000","last_became_peered":"2024-09-17T04:29:59.004956+0000","last_unstale":"2024-09-17T04:30:00.041962+0000","last_undegraded":"2024-09-17T04:30:00.041962+0000","last_fullsized":"2024-09-17T04:30:00.041962+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:19:02.665241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.241418+0000","last_change":"2024-09-17T04:30:00.037349+0000","last_active":"2024-09-17T04:30:01.241418+0000","last_peered":"2024-09-17T04:30:01.241418+0000","last_clean":"2024-09-17T04:30:01.241418+0000","last_became_active":"2024-09-17T04:30:00.035164+0000","last_became_peered":"2024-09-17T04:30:00.035164+0000","last_unstale":"2024-09-17T04:30:01.241418+0000","last_undegraded":"2024-09-17T04:30:01.241418+0000","last_fullsized":"2024-09-17T04:30:01.241418+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:48.127054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.021128+0000","last_change":"2024-09-17T04:29:59.018202+0000","last_active":"2024-09-17T04:30:01.021128+0000","last_peered":"2024-09-17T04:30:01.021128+0000","last_clean":"2024-09-17T04:30:01.021128+0000","last_became_active":"2024-09-17T04:29:59.017545+0000","last_became_peered":"2024-09-17T04:29:59.017545+0000","last_unstale":"2024-09-17T04:30:01.021128+0000","last_undegraded":"2024-09-17T04:30:01.021128+0000","last_fullsized":"2024-09-17T04:30:01.021128+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:41:49.363538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.037737+0000","last_change":"2024-09-17T04:30:00.034076+0000","last_active":"2024-09-17T04:30:00.037737+0000","last_peered":"2024-09-17T04:30:00.037737+0000","last_clean":"2024-09-17T04:30:00.037737+0000","last_became_active":"2024-09-17T04:30:00.033709+0000","last_became_peered":"2024-09-17T04:30:00.033709+0000","last_unstale":"2024-09-17T04:30:00.037737+0000","last_undegraded":"2024-09-17T04:30:00.037737+0000","last_fullsized":"2024-09-17T04:30:00.037737+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:43:29.810312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.020947+0000","last_change":"2024-09-17T04:29:59.009241+0000","last_active":"2024-09-17T04:30:01.020947+0000","last_peered":"2024-09-17T04:30:01.020947+0000","last_clean":"2024-09-17T04:30:01.020947+0000","last_became_active":"2024-09-17T04:29:59.006633+0000","last_became_peered":"2024-09-17T04:29:59.006633+0000","last_unstale":"2024-09-17T04:30:01.020947+0000","last_undegraded":"2024-09-17T04:30:01.020947+0000","last_fullsized":"2024-09-17T04:30:01.020947+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:01:05.139820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.048802+0000","last_change":"2024-09-17T04:30:00.039327+0000","last_active":"2024-09-17T04:30:00.048802+0000","last_peered":"2024-09-17T04:30:00.048802+0000","last_clean":"2024-09-17T04:30:00.048802+0000","last_became_active":"2024-09-17T04:30:00.039006+0000","last_became_peered":"2024-09-17T04:30:00.039006+0000","last_unstale":"2024-09-17T04:30:00.048802+0000","last_undegraded":"2024-09-17T04:30:00.048802+0000","last_fullsized":"2024-09-17T04:30:00.048802+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:59:46.857615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.023751+0000","last_change":"2024-09-17T04:29:59.010630+0000","last_active":"2024-09-17T04:30:01.023751+0000","last_peered":"2024-09-17T04:30:01.023751+0000","last_clean":"2024-09-17T04:30:01.023751+0000","last_became_active":"2024-09-17T04:29:59.009923+0000","last_became_peered":"2024-09-17T04:29:59.009923+0000","last_unstale":"2024-09-17T04:30:01.023751+0000","last_undegraded":"2024-09-17T04:30:01.023751+0000","last_fullsized":"2024-09-17T04:30:01.023751+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:08:28.309920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.022426+0000","last_change":"2024-09-17T04:30:00.031279+0000","last_active":"2024-09-17T04:30:01.022426+0000","last_peered":"2024-09-17T04:30:01.022426+0000","last_clean":"2024-09-17T04:30:01.022426+0000","last_became_active":"2024-09-17T04:30:00.030957+0000","last_became_peered":"2024-09-17T04:30:00.030957+0000","last_unstale":"2024-09-17T04:30:01.022426+0000","last_undegraded":"2024-09-17T04:30:01.022426+0000","last_fullsized":"2024-09-17T04:30:01.022426+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:19:30.797559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"104'1","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.040233+0000","last_change":"2024-09-17T04:29:59.011407+0000","last_active":"2024-09-17T04:30:00.040233+0000","last_peered":"2024-09-17T04:30:00.040233+0000","last_clean":"2024-09-17T04:30:00.040233+0000","last_became_active":"2024-09-17T04:29:59.007991+0000","last_became_peered":"2024-09-17T04:29:59.007991+0000","last_unstale":"2024-09-17T04:30:00.040233+0000","last_undegraded":"2024-09-17T04:30:00.040233+0000","last_fullsized":"2024-09-17T04:30:00.040233+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:22.125466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.047839+0000","last_change":"2024-09-17T04:30:00.042491+0000","last_active":"2024-09-17T04:30:00.047839+0000","last_peered":"2024-09-17T04:30:00.047839+0000","last_clean":"2024-09-17T04:30:00.047839+0000","last_became_active":"2024-09-17T04:30:00.042160+0000","last_became_peered":"2024-09-17T04:30:00.042160+0000","last_unstale":"2024-09-17T04:30:00.047839+0000","last_undegraded":"2024-09-17T04:30:00.047839+0000","last_fullsized":"2024-09-17T04:30:00.047839+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:24:20.303018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.022344+0000","last_change":"2024-09-17T04:29:59.004824+0000","last_active":"2024-09-17T04:30:01.022344+0000","last_peered":"2024-09-17T04:30:01.022344+0000","last_clean":"2024-09-17T04:30:01.022344+0000","last_became_active":"2024-09-17T04:29:59.003696+0000","last_became_peered":"2024-09-17T04:29:59.003696+0000","last_unstale":"2024-09-17T04:30:01.022344+0000","last_undegraded":"2024-09-17T04:30:01.022344+0000","last_fullsized":"2024-09-17T04:30:01.022344+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:49.166951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.023281+0000","last_change":"2024-09-17T04:30:00.024319+0000","last_active":"2024-09-17T04:30:01.023281+0000","last_peered":"2024-09-17T04:30:01.023281+0000","last_clean":"2024-09-17T04:30:01.023281+0000","last_became_active":"2024-09-17T04:30:00.023658+0000","last_became_peered":"2024-09-17T04:30:00.023658+0000","last_unstale":"2024-09-17T04:30:01.023281+0000","last_undegraded":"2024-09-17T04:30:01.023281+0000","last_fullsized":"2024-09-17T04:30:01.023281+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:19:12.584918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.016788+0000","last_change":"2024-09-17T04:29:59.010122+0000","last_active":"2024-09-17T04:30:00.016788+0000","last_peered":"2024-09-17T04:30:00.016788+0000","last_clean":"2024-09-17T04:30:00.016788+0000","last_became_active":"2024-09-17T04:29:59.009781+0000","last_became_peered":"2024-09-17T04:29:59.009781+0000","last_unstale":"2024-09-17T04:30:00.016788+0000","last_undegraded":"2024-09-17T04:30:00.016788+0000","last_fullsized":"2024-09-17T04:30:00.016788+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:32:08.591973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.024362+0000","last_change":"2024-09-17T04:30:00.013587+0000","last_active":"2024-09-17T04:30:01.024362+0000","last_peered":"2024-09-17T04:30:01.024362+0000","last_clean":"2024-09-17T04:30:01.024362+0000","last_became_active":"2024-09-17T04:30:00.013146+0000","last_became_peered":"2024-09-17T04:30:00.013146+0000","last_unstale":"2024-09-17T04:30:01.024362+0000","last_undegraded":"2024-09-17T04:30:01.024362+0000","last_fullsized":"2024-09-17T04:30:01.024362+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:25:51.689001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.021422+0000","last_change":"2024-09-17T04:29:59.012316+0000","last_active":"2024-09-17T04:30:01.021422+0000","last_peered":"2024-09-17T04:30:01.021422+0000","last_clean":"2024-09-17T04:30:01.021422+0000","last_became_active":"2024-09-17T04:29:59.012001+0000","last_became_peered":"2024-09-17T04:29:59.012001+0000","last_unstale":"2024-09-17T04:30:01.021422+0000","last_undegraded":"2024-09-17T04:30:01.021422+0000","last_fullsized":"2024-09-17T04:30:01.021422+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:06:17.905426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.027752+0000","last_change":"2024-09-17T04:30:00.035515+0000","last_active":"2024-09-17T04:30:01.027752+0000","last_peered":"2024-09-17T04:30:01.027752+0000","last_clean":"2024-09-17T04:30:01.027752+0000","last_became_active":"2024-09-17T04:30:00.035190+0000","last_became_peered":"2024-09-17T04:30:00.035190+0000","last_unstale":"2024-09-17T04:30:01.027752+0000","last_undegraded":"2024-09-17T04:30:01.027752+0000","last_fullsized":"2024-09-17T04:30:01.027752+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:54:19.658662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.009988+0000","last_change":"2024-09-17T04:29:59.012130+0000","last_active":"2024-09-17T04:30:00.009988+0000","last_peered":"2024-09-17T04:30:00.009988+0000","last_clean":"2024-09-17T04:30:00.009988+0000","last_became_active":"2024-09-17T04:29:59.011625+0000","last_became_peered":"2024-09-17T04:29:59.011625+0000","last_unstale":"2024-09-17T04:30:00.009988+0000","last_undegraded":"2024-09-17T04:30:00.009988+0000","last_fullsized":"2024-09-17T04:30:00.009988+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:28:15.481834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.024913+0000","last_change":"2024-09-17T04:30:00.019717+0000","last_active":"2024-09-17T04:30:00.024913+0000","last_peered":"2024-09-17T04:30:00.024913+0000","last_clean":"2024-09-17T04:30:00.024913+0000","last_became_active":"2024-09-17T04:30:00.017527+0000","last_became_peered":"2024-09-17T04:30:00.017527+0000","last_unstale":"2024-09-17T04:30:00.024913+0000","last_undegraded":"2024-09-17T04:30:00.024913+0000","last_fullsized":"2024-09-17T04:30:00.024913+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:28:45.075635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"104'3","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.053077+0000","last_change":"2024-09-17T04:29:59.017863+0000","last_active":"2024-09-17T04:30:00.053077+0000","last_peered":"2024-09-17T04:30:00.053077+0000","last_clean":"2024-09-17T04:30:00.053077+0000","last_became_active":"2024-09-17T04:29:59.017383+0000","last_became_peered":"2024-09-17T04:29:59.017383+0000","last_unstale":"2024-09-17T04:30:00.053077+0000","last_undegraded":"2024-09-17T04:30:00.053077+0000","last_fullsized":"2024-09-17T04:30:00.053077+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:56:33.193090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.024254+0000","last_change":"2024-09-17T04:30:00.019174+0000","last_active":"2024-09-17T04:30:01.024254+0000","last_peered":"2024-09-17T04:30:01.024254+0000","last_clean":"2024-09-17T04:30:01.024254+0000","last_became_active":"2024-09-17T04:30:00.018865+0000","last_became_peered":"2024-09-17T04:30:00.018865+0000","last_unstale":"2024-09-17T04:30:01.024254+0000","last_undegraded":"2024-09-17T04:30:01.024254+0000","last_fullsized":"2024-09-17T04:30:01.024254+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:56.828497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"104'6","reported_seq":19,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.055093+0000","last_change":"2024-09-17T04:29:59.016065+0000","last_active":"2024-09-17T04:30:00.055093+0000","last_peered":"2024-09-17T04:30:00.055093+0000","last_clean":"2024-09-17T04:30:00.055093+0000","last_became_active":"2024-09-17T04:29:59.015764+0000","last_became_peered":"2024-09-17T04:29:59.015764+0000","last_unstale":"2024-09-17T04:30:00.055093+0000","last_undegraded":"2024-09-17T04:30:00.055093+0000","last_fullsized":"2024-09-17T04:30:00.055093+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:13.616467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.016619+0000","last_change":"2024-09-17T04:30:00.013540+0000","last_active":"2024-09-17T04:30:00.016619+0000","last_peered":"2024-09-17T04:30:00.016619+0000","last_clean":"2024-09-17T04:30:00.016619+0000","last_became_active":"2024-09-17T04:30:00.013151+0000","last_became_peered":"2024-09-17T04:30:00.013151+0000","last_unstale":"2024-09-17T04:30:00.016619+0000","last_undegraded":"2024-09-17T04:30:00.016619+0000","last_fullsized":"2024-09-17T04:30:00.016619+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:42.584154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.241058+0000","last_change":"2024-09-17T04:29:59.013264+0000","last_active":"2024-09-17T04:30:01.241058+0000","last_peered":"2024-09-17T04:30:01.241058+0000","last_clean":"2024-09-17T04:30:01.241058+0000","last_became_active":"2024-09-17T04:29:59.012844+0000","last_became_peered":"2024-09-17T04:29:59.012844+0000","last_unstale":"2024-09-17T04:30:01.241058+0000","last_undegraded":"2024-09-17T04:30:01.241058+0000","last_fullsized":"2024-09-17T04:30:01.241058+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:51:10.293956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.032729+0000","last_change":"2024-09-17T04:30:00.026868+0000","last_active":"2024-09-17T04:30:00.032729+0000","last_peered":"2024-09-17T04:30:00.032729+0000","last_clean":"2024-09-17T04:30:00.032729+0000","last_became_active":"2024-09-17T04:30:00.026582+0000","last_became_peered":"2024-09-17T04:30:00.026582+0000","last_unstale":"2024-09-17T04:30:00.032729+0000","last_undegraded":"2024-09-17T04:30:00.032729+0000","last_fullsized":"2024-09-17T04:30:00.032729+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:47:25.619529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.027276+0000","last_change":"2024-09-17T04:29:59.007287+0000","last_active":"2024-09-17T04:30:01.027276+0000","last_peered":"2024-09-17T04:30:01.027276+0000","last_clean":"2024-09-17T04:30:01.027276+0000","last_became_active":"2024-09-17T04:29:59.006446+0000","last_became_peered":"2024-09-17T04:29:59.006446+0000","last_unstale":"2024-09-17T04:30:01.027276+0000","last_undegraded":"2024-09-17T04:30:01.027276+0000","last_fullsized":"2024-09-17T04:30:01.027276+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:53:32.893059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.033321+0000","last_change":"2024-09-17T04:30:00.027527+0000","last_active":"2024-09-17T04:30:00.033321+0000","last_peered":"2024-09-17T04:30:00.033321+0000","last_clean":"2024-09-17T04:30:00.033321+0000","last_became_active":"2024-09-17T04:30:00.023985+0000","last_became_peered":"2024-09-17T04:30:00.023985+0000","last_unstale":"2024-09-17T04:30:00.033321+0000","last_undegraded":"2024-09-17T04:30:00.033321+0000","last_fullsized":"2024-09-17T04:30:00.033321+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:22.339699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"104'8","reported_seq":23,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.020948+0000","last_change":"2024-09-17T04:29:59.011479+0000","last_active":"2024-09-17T04:30:01.020948+0000","last_peered":"2024-09-17T04:30:01.020948+0000","last_clean":"2024-09-17T04:30:01.020948+0000","last_became_active":"2024-09-17T04:29:59.011162+0000","last_became_peered":"2024-09-17T04:29:59.011162+0000","last_unstale":"2024-09-17T04:30:01.020948+0000","last_undegraded":"2024-09-17T04:30:01.020948+0000","last_fullsized":"2024-09-17T04:30:01.020948+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:59.077658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.021351+0000","last_change":"2024-09-17T04:30:00.026452+0000","last_active":"2024-09-17T04:30:01.021351+0000","last_peered":"2024-09-17T04:30:01.021351+0000","last_clean":"2024-09-17T04:30:01.021351+0000","last_became_active":"2024-09-17T04:30:00.026087+0000","last_became_peered":"2024-09-17T04:30:00.026087+0000","last_unstale":"2024-09-17T04:30:01.021351+0000","last_undegraded":"2024-09-17T04:30:01.021351+0000","last_fullsized":"2024-09-17T04:30:01.021351+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:01:35.454605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"104'1","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.048906+0000","last_change":"2024-09-17T04:29:59.013573+0000","last_active":"2024-09-17T04:30:00.048906+0000","last_peered":"2024-09-17T04:30:00.048906+0000","last_clean":"2024-09-17T04:30:00.048906+0000","last_became_active":"2024-09-17T04:29:59.013307+0000","last_became_peered":"2024-09-17T04:29:59.013307+0000","last_unstale":"2024-09-17T04:30:00.048906+0000","last_undegraded":"2024-09-17T04:30:00.048906+0000","last_fullsized":"2024-09-17T04:30:00.048906+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:43.112656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.241276+0000","last_change":"2024-09-17T04:30:00.042113+0000","last_active":"2024-09-17T04:30:01.241276+0000","last_peered":"2024-09-17T04:30:01.241276+0000","last_clean":"2024-09-17T04:30:01.241276+0000","last_became_active":"2024-09-17T04:30:00.041788+0000","last_became_peered":"2024-09-17T04:30:00.041788+0000","last_unstale":"2024-09-17T04:30:01.241276+0000","last_undegraded":"2024-09-17T04:30:01.241276+0000","last_fullsized":"2024-09-17T04:30:01.241276+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:50:54.585046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.021326+0000","last_change":"2024-09-17T04:29:58.999969+0000","last_active":"2024-09-17T04:30:01.021326+0000","last_peered":"2024-09-17T04:30:01.021326+0000","last_clean":"2024-09-17T04:30:01.021326+0000","last_became_active":"2024-09-17T04:29:58.998593+0000","last_became_peered":"2024-09-17T04:29:58.998593+0000","last_unstale":"2024-09-17T04:30:01.021326+0000","last_undegraded":"2024-09-17T04:30:01.021326+0000","last_fullsized":"2024-09-17T04:30:01.021326+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:38:34.113452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.040125+0000","last_change":"2024-09-17T04:30:00.039039+0000","last_active":"2024-09-17T04:30:00.040125+0000","last_peered":"2024-09-17T04:30:00.040125+0000","last_clean":"2024-09-17T04:30:00.040125+0000","last_became_active":"2024-09-17T04:30:00.038305+0000","last_became_peered":"2024-09-17T04:30:00.038305+0000","last_unstale":"2024-09-17T04:30:00.040125+0000","last_undegraded":"2024-09-17T04:30:00.040125+0000","last_fullsized":"2024-09-17T04:30:00.040125+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:57:59.119887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.027348+0000","last_change":"2024-09-17T04:29:59.007492+0000","last_active":"2024-09-17T04:30:01.027348+0000","last_peered":"2024-09-17T04:30:01.027348+0000","last_clean":"2024-09-17T04:30:01.027348+0000","last_became_active":"2024-09-17T04:29:59.006443+0000","last_became_peered":"2024-09-17T04:29:59.006443+0000","last_unstale":"2024-09-17T04:30:01.027348+0000","last_undegraded":"2024-09-17T04:30:01.027348+0000","last_fullsized":"2024-09-17T04:30:01.027348+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:43:28.801432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.037613+0000","last_change":"2024-09-17T04:30:00.032254+0000","last_active":"2024-09-17T04:30:00.037613+0000","last_peered":"2024-09-17T04:30:00.037613+0000","last_clean":"2024-09-17T04:30:00.037613+0000","last_became_active":"2024-09-17T04:30:00.031486+0000","last_became_peered":"2024-09-17T04:30:00.031486+0000","last_unstale":"2024-09-17T04:30:00.037613+0000","last_undegraded":"2024-09-17T04:30:00.037613+0000","last_fullsized":"2024-09-17T04:30:00.037613+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:58.219483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.023919+0000","last_change":"2024-09-17T04:29:59.008591+0000","last_active":"2024-09-17T04:30:01.023919+0000","last_peered":"2024-09-17T04:30:01.023919+0000","last_clean":"2024-09-17T04:30:01.023919+0000","last_became_active":"2024-09-17T04:29:59.008142+0000","last_became_peered":"2024-09-17T04:29:59.008142+0000","last_unstale":"2024-09-17T04:30:01.023919+0000","last_undegraded":"2024-09-17T04:30:01.023919+0000","last_fullsized":"2024-09-17T04:30:01.023919+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:43:28.696763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.039494+0000","last_change":"2024-09-17T04:30:00.035767+0000","last_active":"2024-09-17T04:30:00.039494+0000","last_peered":"2024-09-17T04:30:00.039494+0000","last_clean":"2024-09-17T04:30:00.039494+0000","last_became_active":"2024-09-17T04:30:00.034914+0000","last_became_peered":"2024-09-17T04:30:00.034914+0000","last_unstale":"2024-09-17T04:30:00.039494+0000","last_undegraded":"2024-09-17T04:30:00.039494+0000","last_fullsized":"2024-09-17T04:30:00.039494+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:05:08.139172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.241100+0000","last_change":"2024-09-17T04:29:59.016631+0000","last_active":"2024-09-17T04:30:01.241100+0000","last_peered":"2024-09-17T04:30:01.241100+0000","last_clean":"2024-09-17T04:30:01.241100+0000","last_became_active":"2024-09-17T04:29:59.016246+0000","last_became_peered":"2024-09-17T04:29:59.016246+0000","last_unstale":"2024-09-17T04:30:01.241100+0000","last_undegraded":"2024-09-17T04:30:01.241100+0000","last_fullsized":"2024-09-17T04:30:01.241100+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:01:34.445725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.040463+0000","last_change":"2024-09-17T04:30:00.032092+0000","last_active":"2024-09-17T04:30:00.040463+0000","last_peered":"2024-09-17T04:30:00.040463+0000","last_clean":"2024-09-17T04:30:00.040463+0000","last_became_active":"2024-09-17T04:30:00.031742+0000","last_became_peered":"2024-09-17T04:30:00.031742+0000","last_unstale":"2024-09-17T04:30:00.040463+0000","last_undegraded":"2024-09-17T04:30:00.040463+0000","last_fullsized":"2024-09-17T04:30:00.040463+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:56.769785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.020935+0000","last_change":"2024-09-17T04:29:59.016046+0000","last_active":"2024-09-17T04:30:01.020935+0000","last_peered":"2024-09-17T04:30:01.020935+0000","last_clean":"2024-09-17T04:30:01.020935+0000","last_became_active":"2024-09-17T04:29:59.015762+0000","last_became_peered":"2024-09-17T04:29:59.015762+0000","last_unstale":"2024-09-17T04:30:01.020935+0000","last_undegraded":"2024-09-17T04:30:01.020935+0000","last_fullsized":"2024-09-17T04:30:01.020935+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:19.680096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.039728+0000","last_change":"2024-09-17T04:30:00.033808+0000","last_active":"2024-09-17T04:30:00.039728+0000","last_peered":"2024-09-17T04:30:00.039728+0000","last_clean":"2024-09-17T04:30:00.039728+0000","last_became_active":"2024-09-17T04:30:00.033381+0000","last_became_peered":"2024-09-17T04:30:00.033381+0000","last_unstale":"2024-09-17T04:30:00.039728+0000","last_undegraded":"2024-09-17T04:30:00.039728+0000","last_fullsized":"2024-09-17T04:30:00.039728+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:05:25.837826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.240644+0000","last_change":"2024-09-17T04:29:59.011744+0000","last_active":"2024-09-17T04:30:01.240644+0000","last_peered":"2024-09-17T04:30:01.240644+0000","last_clean":"2024-09-17T04:30:01.240644+0000","last_became_active":"2024-09-17T04:29:59.011460+0000","last_became_peered":"2024-09-17T04:29:59.011460+0000","last_unstale":"2024-09-17T04:30:01.240644+0000","last_undegraded":"2024-09-17T04:30:01.240644+0000","last_fullsized":"2024-09-17T04:30:01.240644+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:49:18.446987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.033348+0000","last_change":"2024-09-17T04:30:00.030573+0000","last_active":"2024-09-17T04:30:00.033348+0000","last_peered":"2024-09-17T04:30:00.033348+0000","last_clean":"2024-09-17T04:30:00.033348+0000","last_became_active":"2024-09-17T04:30:00.030261+0000","last_became_peered":"2024-09-17T04:30:00.030261+0000","last_unstale":"2024-09-17T04:30:00.033348+0000","last_undegraded":"2024-09-17T04:30:00.033348+0000","last_fullsized":"2024-09-17T04:30:00.033348+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:03.908735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.021003+0000","last_change":"2024-09-17T04:29:59.013448+0000","last_active":"2024-09-17T04:30:00.021003+0000","last_peered":"2024-09-17T04:30:00.021003+0000","last_clean":"2024-09-17T04:30:00.021003+0000","last_became_active":"2024-09-17T04:29:59.010565+0000","last_became_peered":"2024-09-17T04:29:59.010565+0000","last_unstale":"2024-09-17T04:30:00.021003+0000","last_undegraded":"2024-09-17T04:30:00.021003+0000","last_fullsized":"2024-09-17T04:30:00.021003+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:25:13.476952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.027384+0000","last_change":"2024-09-17T04:30:00.035250+0000","last_active":"2024-09-17T04:30:01.027384+0000","last_peered":"2024-09-17T04:30:01.027384+0000","last_clean":"2024-09-17T04:30:01.027384+0000","last_became_active":"2024-09-17T04:30:00.029488+0000","last_became_peered":"2024-09-17T04:30:00.029488+0000","last_unstale":"2024-09-17T04:30:01.027384+0000","last_undegraded":"2024-09-17T04:30:01.027384+0000","last_fullsized":"2024-09-17T04:30:01.027384+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:00.376670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"104'2","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.048889+0000","last_change":"2024-09-17T04:29:59.009226+0000","last_active":"2024-09-17T04:30:00.048889+0000","last_peered":"2024-09-17T04:30:00.048889+0000","last_clean":"2024-09-17T04:30:00.048889+0000","last_became_active":"2024-09-17T04:29:59.008934+0000","last_became_peered":"2024-09-17T04:29:59.008934+0000","last_unstale":"2024-09-17T04:30:00.048889+0000","last_undegraded":"2024-09-17T04:30:00.048889+0000","last_fullsized":"2024-09-17T04:30:00.048889+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:17:58.897435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.040218+0000","last_change":"2024-09-17T04:30:00.031413+0000","last_active":"2024-09-17T04:30:00.040218+0000","last_peered":"2024-09-17T04:30:00.040218+0000","last_clean":"2024-09-17T04:30:00.040218+0000","last_became_active":"2024-09-17T04:30:00.029749+0000","last_became_peered":"2024-09-17T04:30:00.029749+0000","last_unstale":"2024-09-17T04:30:00.040218+0000","last_undegraded":"2024-09-17T04:30:00.040218+0000","last_fullsized":"2024-09-17T04:30:00.040218+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:00.354631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"105'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.029126+0000","last_change":"2024-09-17T04:29:58.999706+0000","last_active":"2024-09-17T04:30:01.029126+0000","last_peered":"2024-09-17T04:30:01.029126+0000","last_clean":"2024-09-17T04:30:01.029126+0000","last_became_active":"2024-09-17T04:29:58.999361+0000","last_became_peered":"2024-09-17T04:29:58.999361+0000","last_unstale":"2024-09-17T04:30:01.029126+0000","last_undegraded":"2024-09-17T04:30:01.029126+0000","last_fullsized":"2024-09-17T04:30:01.029126+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:05:07.130292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.048723+0000","last_change":"2024-09-17T04:30:00.034238+0000","last_active":"2024-09-17T04:30:00.048723+0000","last_peered":"2024-09-17T04:30:00.048723+0000","last_clean":"2024-09-17T04:30:00.048723+0000","last_became_active":"2024-09-17T04:30:00.033877+0000","last_became_peered":"2024-09-17T04:30:00.033877+0000","last_unstale":"2024-09-17T04:30:00.048723+0000","last_undegraded":"2024-09-17T04:30:00.048723+0000","last_fullsized":"2024-09-17T04:30:00.048723+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:14:47.053925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.014537+0000","last_change":"2024-09-17T04:29:59.010343+0000","last_active":"2024-09-17T04:30:00.014537+0000","last_peered":"2024-09-17T04:30:00.014537+0000","last_clean":"2024-09-17T04:30:00.014537+0000","last_became_active":"2024-09-17T04:29:59.010035+0000","last_became_peered":"2024-09-17T04:29:59.010035+0000","last_unstale":"2024-09-17T04:30:00.014537+0000","last_undegraded":"2024-09-17T04:30:00.014537+0000","last_fullsized":"2024-09-17T04:30:00.014537+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:41.091741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.027459+0000","last_change":"2024-09-17T04:30:00.033009+0000","last_active":"2024-09-17T04:30:01.027459+0000","last_peered":"2024-09-17T04:30:01.027459+0000","last_clean":"2024-09-17T04:30:01.027459+0000","last_became_active":"2024-09-17T04:30:00.032624+0000","last_became_peered":"2024-09-17T04:30:00.032624+0000","last_unstale":"2024-09-17T04:30:01.027459+0000","last_undegraded":"2024-09-17T04:30:01.027459+0000","last_fullsized":"2024-09-17T04:30:01.027459+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:05:21.232149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.010019+0000","last_change":"2024-09-17T04:29:59.008457+0000","last_active":"2024-09-17T04:30:00.010019+0000","last_peered":"2024-09-17T04:30:00.010019+0000","last_clean":"2024-09-17T04:30:00.010019+0000","last_became_active":"2024-09-17T04:29:59.008080+0000","last_became_peered":"2024-09-17T04:29:59.008080+0000","last_unstale":"2024-09-17T04:30:00.010019+0000","last_undegraded":"2024-09-17T04:30:00.010019+0000","last_fullsized":"2024-09-17T04:30:00.010019+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:06:44.785805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.022592+0000","last_change":"2024-09-17T04:30:00.021360+0000","last_active":"2024-09-17T04:30:01.022592+0000","last_peered":"2024-09-17T04:30:01.022592+0000","last_clean":"2024-09-17T04:30:01.022592+0000","last_became_active":"2024-09-17T04:30:00.020152+0000","last_became_peered":"2024-09-17T04:30:00.020152+0000","last_unstale":"2024-09-17T04:30:01.022592+0000","last_undegraded":"2024-09-17T04:30:01.022592+0000","last_fullsized":"2024-09-17T04:30:01.022592+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:55.361480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.023746+0000","last_change":"2024-09-17T04:29:59.003143+0000","last_active":"2024-09-17T04:30:01.023746+0000","last_peered":"2024-09-17T04:30:01.023746+0000","last_clean":"2024-09-17T04:30:01.023746+0000","last_became_active":"2024-09-17T04:29:59.002678+0000","last_became_peered":"2024-09-17T04:29:59.002678+0000","last_unstale":"2024-09-17T04:30:01.023746+0000","last_undegraded":"2024-09-17T04:30:01.023746+0000","last_fullsized":"2024-09-17T04:30:01.023746+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:10.160946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.023398+0000","last_change":"2024-09-17T04:30:00.017223+0000","last_active":"2024-09-17T04:30:01.023398+0000","last_peered":"2024-09-17T04:30:01.023398+0000","last_clean":"2024-09-17T04:30:01.023398+0000","last_became_active":"2024-09-17T04:30:00.016903+0000","last_became_peered":"2024-09-17T04:30:00.016903+0000","last_unstale":"2024-09-17T04:30:01.023398+0000","last_undegraded":"2024-09-17T04:30:01.023398+0000","last_fullsized":"2024-09-17T04:30:01.023398+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:42.100621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.022452+0000","last_change":"2024-09-17T04:29:59.011982+0000","last_active":"2024-09-17T04:30:01.022452+0000","last_peered":"2024-09-17T04:30:01.022452+0000","last_clean":"2024-09-17T04:30:01.022452+0000","last_became_active":"2024-09-17T04:29:59.011509+0000","last_became_peered":"2024-09-17T04:29:59.011509+0000","last_unstale":"2024-09-17T04:30:01.022452+0000","last_undegraded":"2024-09-17T04:30:01.022452+0000","last_fullsized":"2024-09-17T04:30:01.022452+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:24.291710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.039698+0000","last_change":"2024-09-17T04:30:00.032939+0000","last_active":"2024-09-17T04:30:00.039698+0000","last_peered":"2024-09-17T04:30:00.039698+0000","last_clean":"2024-09-17T04:30:00.039698+0000","last_became_active":"2024-09-17T04:30:00.032065+0000","last_became_peered":"2024-09-17T04:30:00.032065+0000","last_unstale":"2024-09-17T04:30:00.039698+0000","last_undegraded":"2024-09-17T04:30:00.039698+0000","last_fullsized":"2024-09-17T04:30:00.039698+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:38.961501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.020874+0000","last_change":"2024-09-17T04:29:59.016863+0000","last_active":"2024-09-17T04:30:01.020874+0000","last_peered":"2024-09-17T04:30:01.020874+0000","last_clean":"2024-09-17T04:30:01.020874+0000","last_became_active":"2024-09-17T04:29:59.016582+0000","last_became_peered":"2024-09-17T04:29:59.016582+0000","last_unstale":"2024-09-17T04:30:01.020874+0000","last_undegraded":"2024-09-17T04:30:01.020874+0000","last_fullsized":"2024-09-17T04:30:01.020874+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:47.279038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.047891+0000","last_change":"2024-09-17T04:30:00.041495+0000","last_active":"2024-09-17T04:30:00.047891+0000","last_peered":"2024-09-17T04:30:00.047891+0000","last_clean":"2024-09-17T04:30:00.047891+0000","last_became_active":"2024-09-17T04:30:00.040827+0000","last_became_peered":"2024-09-17T04:30:00.040827+0000","last_unstale":"2024-09-17T04:30:00.047891+0000","last_undegraded":"2024-09-17T04:30:00.047891+0000","last_fullsized":"2024-09-17T04:30:00.047891+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:33:52.257861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.017479+0000","last_change":"2024-09-17T04:29:59.014718+0000","last_active":"2024-09-17T04:30:00.017479+0000","last_peered":"2024-09-17T04:30:00.017479+0000","last_clean":"2024-09-17T04:30:00.017479+0000","last_became_active":"2024-09-17T04:29:59.010601+0000","last_became_peered":"2024-09-17T04:29:59.010601+0000","last_unstale":"2024-09-17T04:30:00.017479+0000","last_undegraded":"2024-09-17T04:30:00.017479+0000","last_fullsized":"2024-09-17T04:30:00.017479+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:17.750773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.027209+0000","last_change":"2024-09-17T04:30:00.020727+0000","last_active":"2024-09-17T04:30:00.027209+0000","last_peered":"2024-09-17T04:30:00.027209+0000","last_clean":"2024-09-17T04:30:00.027209+0000","last_became_active":"2024-09-17T04:30:00.020436+0000","last_became_peered":"2024-09-17T04:30:00.020436+0000","last_unstale":"2024-09-17T04:30:00.027209+0000","last_undegraded":"2024-09-17T04:30:00.027209+0000","last_fullsized":"2024-09-17T04:30:00.027209+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:36.288918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.023700+0000","last_change":"2024-09-17T04:29:59.007405+0000","last_active":"2024-09-17T04:30:01.023700+0000","last_peered":"2024-09-17T04:30:01.023700+0000","last_clean":"2024-09-17T04:30:01.023700+0000","last_became_active":"2024-09-17T04:29:59.006905+0000","last_became_peered":"2024-09-17T04:29:59.006905+0000","last_unstale":"2024-09-17T04:30:01.023700+0000","last_undegraded":"2024-09-17T04:30:01.023700+0000","last_fullsized":"2024-09-17T04:30:01.023700+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:55:24.434433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"76'79","reported_seq":289,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.018945+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:30:00.018945+0000","last_peered":"2024-09-17T04:30:00.018945+0000","last_clean":"2024-09-17T04:30:00.018945+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:30:00.018945+0000","last_undegraded":"2024-09-17T04:30:00.018945+0000","last_fullsized":"2024-09-17T04:30:00.018945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476813,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1884160,"data_stored":1150265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476812,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32460,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339380,"statfs":{"total":96636764160,"available":96603525120,"internally_reserved":0,"allocated":1298432,"data_stored":565651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476813,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1290240,"data_stored":559885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476813,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1282048,"data_stored":559281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542220,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32460,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339380,"statfs":{"total":96636764160,"available":96603525120,"internally_reserved":0,"allocated":1298432,"data_stored":560963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509516,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32460,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339380,"statfs":{"total":96636764160,"available":96603525120,"internally_reserved":0,"allocated":1294336,"data_stored":565581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509515,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1302528,"data_stored":565705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509515,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1896448,"data_stored":1151373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:30:04.427 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:30:04.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.681+0000 7fdbffa00640 1 -- 172.21.15.100:0/1474854185 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbf815f910 msgr2=0x7fdbf80b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:04.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.681+0000 7fdbffa00640 1 --2- 172.21.15.100:0/1474854185 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbf815f910 0x7fdbf80b3750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0005fd0 tx=0x7fdbf002a7a0 comp rx=0 tx=0).stop 2024-09-17T04:30:04.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.681+0000 7fdbffa00640 1 -- 172.21.15.100:0/1474854185 shutdown_connections 2024-09-17T04:30:04.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.681+0000 7fdbffa00640 1 --2- 172.21.15.100:0/1474854185 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbf815f910 0x7fdbf80b3750 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:04.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.681+0000 7fdbffa00640 1 --2- 172.21.15.100:0/1474854185 >> v2:172.21.15.184:3300/0 conn(0x7fdbf8152920 0x7fdbf815f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:04.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.681+0000 7fdbffa00640 1 -- 172.21.15.100:0/1474854185 >> 172.21.15.100:0/1474854185 conn(0x7fdbf806d730 msgr2=0x7fdbf80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:04.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.681+0000 7fdbffa00640 1 -- 172.21.15.100:0/1474854185 shutdown_connections 2024-09-17T04:30:04.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.681+0000 7fdbffa00640 1 -- 172.21.15.100:0/1474854185 wait complete. 2024-09-17T04:30:04.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.684+0000 7fdbffa00640 1 Processor -- start 2024-09-17T04:30:04.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.684+0000 7fdbffa00640 1 -- start start 2024-09-17T04:30:04.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.685+0000 7fdbffa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbf80aebc0 0x7fdbf80ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:04.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.685+0000 7fdbffa00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbf8152920 0x7fdbf8151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:04.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.685+0000 7fdbffa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbf815f910 0x7fdbf81556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:04.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.685+0000 7fdbffa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf80b4130 con 0x7fdbf80aebc0 2024-09-17T04:30:04.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.685+0000 7fdbffa00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf80b3fb0 con 0x7fdbf815f910 2024-09-17T04:30:04.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.685+0000 7fdbffa00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdbf80b42b0 con 0x7fdbf8152920 2024-09-17T04:30:04.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.685+0000 7fdbfd000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbf80aebc0 0x7fdbf80ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:04.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.685+0000 7fdbfd000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbf80aebc0 0x7fdbf80ce9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37802/0 (socket says 172.21.15.100:37802) 2024-09-17T04:30:04.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.686+0000 7fdbfd000640 1 -- 172.21.15.100:0/2440416531 learned_addr learned my addr 172.21.15.100:0/2440416531 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:04.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.686+0000 7fdbfda00640 1 --2- 172.21.15.100:0/2440416531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbf815f910 0x7fdbf81556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:04.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.686+0000 7fdbefe00640 1 --2- 172.21.15.100:0/2440416531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbf8152920 0x7fdbf8151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:04.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.686+0000 7fdbfd000640 1 -- 172.21.15.100:0/2440416531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbf8152920 msgr2=0x7fdbf8151910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:04.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.686+0000 7fdbfd000640 1 --2- 172.21.15.100:0/2440416531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbf8152920 0x7fdbf8151910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:04.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.686+0000 7fdbfd000640 1 -- 172.21.15.100:0/2440416531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbf815f910 msgr2=0x7fdbf81556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:04.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.686+0000 7fdbfd000640 1 --2- 172.21.15.100:0/2440416531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbf815f910 0x7fdbf81556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:04.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.686+0000 7fdbfd000640 1 -- 172.21.15.100:0/2440416531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf8151e50 con 0x7fdbf80aebc0 2024-09-17T04:30:04.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.686+0000 7fdbfd000640 1 --2- 172.21.15.100:0/2440416531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbf80aebc0 0x7fdbf80ce9f0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800c530 tx=0x7fdbe800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:04.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.687+0000 7fdbf5600640 1 -- 172.21.15.100:0/2440416531 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8016020 con 0x7fdbf80aebc0 2024-09-17T04:30:04.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.687+0000 7fdbffa00640 1 -- 172.21.15.100:0/2440416531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf80cf360 con 0x7fdbf80aebc0 2024-09-17T04:30:04.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.687+0000 7fdbf5600640 1 -- 172.21.15.100:0/2440416531 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbe8012940 con 0x7fdbf80aebc0 2024-09-17T04:30:04.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.687+0000 7fdbf5600640 1 -- 172.21.15.100:0/2440416531 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8012d90 con 0x7fdbf80aebc0 2024-09-17T04:30:04.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.687+0000 7fdbffa00640 1 -- 172.21.15.100:0/2440416531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf81f3940 con 0x7fdbf80aebc0 2024-09-17T04:30:04.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.688+0000 7fdbffa00640 1 -- 172.21.15.100:0/2440416531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf80b4840 con 0x7fdbf80aebc0 2024-09-17T04:30:04.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.689+0000 7fdbf5600640 1 -- 172.21.15.100:0/2440416531 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fdbe8019020 con 0x7fdbf80aebc0 2024-09-17T04:30:04.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.690+0000 7fdbf5600640 1 --2- 172.21.15.100:0/2440416531 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdbd4078ce0 0x7fdbd407b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:04.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.690+0000 7fdbf5600640 1 -- 172.21.15.100:0/2440416531 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7fdbe809ac90 con 0x7fdbf80aebc0 2024-09-17T04:30:04.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.690+0000 7fdbefe00640 1 --2- 172.21.15.100:0/2440416531 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdbd4078ce0 0x7fdbd407b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:04.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.691+0000 7fdbefe00640 1 --2- 172.21.15.100:0/2440416531 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdbd4078ce0 0x7fdbd407b1f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdbf809e640 tx=0x7fdbe0005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:04.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.694+0000 7fdbf5600640 1 -- 172.21.15.100:0/2440416531 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fdbe8020030 con 0x7fdbf80aebc0 2024-09-17T04:30:04.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.877+0000 7fdbffa00640 1 -- 172.21.15.100:0/2440416531 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdbf80b4b60 con 0x7fdbd4078ce0 2024-09-17T04:30:04.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.881+0000 7fdbf5600640 1 -- 172.21.15.100:0/2440416531 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252473 (secure 0 0 0) 0x7fdbf80b4b60 con 0x7fdbd4078ce0 2024-09-17T04:30:04.885 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:30:04.887 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:30:04.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.885+0000 7fdbffa00640 1 -- 172.21.15.100:0/2440416531 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdbd4078ce0 msgr2=0x7fdbd407b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:04.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.885+0000 7fdbffa00640 1 --2- 172.21.15.100:0/2440416531 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdbd4078ce0 0x7fdbd407b1f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdbf809e640 tx=0x7fdbe0005f90 comp rx=0 tx=0).stop 2024-09-17T04:30:04.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.885+0000 7fdbffa00640 1 -- 172.21.15.100:0/2440416531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbf80aebc0 msgr2=0x7fdbf80ce9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:04.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.885+0000 7fdbffa00640 1 --2- 172.21.15.100:0/2440416531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbf80aebc0 0x7fdbf80ce9f0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800c530 tx=0x7fdbe800c9f0 comp rx=0 tx=0).stop 2024-09-17T04:30:04.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.885+0000 7fdbffa00640 1 -- 172.21.15.100:0/2440416531 shutdown_connections 2024-09-17T04:30:04.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.885+0000 7fdbffa00640 1 --2- 172.21.15.100:0/2440416531 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdbd4078ce0 0x7fdbd407b1f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:04.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.885+0000 7fdbffa00640 1 --2- 172.21.15.100:0/2440416531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbf815f910 0x7fdbf81556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:04.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.885+0000 7fdbffa00640 1 --2- 172.21.15.100:0/2440416531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbf8152920 0x7fdbf8151910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:04.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.885+0000 7fdbffa00640 1 --2- 172.21.15.100:0/2440416531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbf80aebc0 0x7fdbf80ce9f0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:04.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.885+0000 7fdbffa00640 1 -- 172.21.15.100:0/2440416531 >> 172.21.15.100:0/2440416531 conn(0x7fdbf806d730 msgr2=0x7fdbf815e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:04.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.885+0000 7fdbffa00640 1 -- 172.21.15.100:0/2440416531 shutdown_connections 2024-09-17T04:30:04.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:04.885+0000 7fdbffa00640 1 -- 172.21.15.100:0/2440416531 wait complete. 2024-09-17T04:30:04.911 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2024-09-17T04:30:03.672223+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260804,"kb_used_data":11276,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713916,"statfs":{"total":773094113280,"available":772827049984,"internally_reserved":0,"allocated":11546624,"data_stored":5678704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.691239"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.035978+0000","last_change":"2024-09-17T04:30:00.031045+0000","last_active":"2024-09-17T04:30:00.035978+0000","last_peered":"2024-09-17T04:30:00.035978+0000","last_clean":"2024-09-17T04:30:00.035978+0000","last_became_active":"2024-09-17T04:30:00.029801+0000","last_became_peered":"2024-09-17T04:30:00.029801+0000","last_unstale":"2024-09-17T04:30:00.035978+0000","last_undegraded":"2024-09-17T04:30:00.035978+0000","last_fullsized":"2024-09-17T04:30:00.035978+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:20:30.612554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.023477+0000","last_change":"2024-09-17T04:30:00.030378+0000","last_active":"2024-09-17T04:30:01.023477+0000","last_peered":"2024-09-17T04:30:01.023477+0000","last_clean":"2024-09-17T04:30:01.023477+0000","last_became_active":"2024-09-17T04:30:00.030043+0000","last_became_peered":"2024-09-17T04:30:00.030043+0000","last_unstale":"2024-09-17T04:30:01.023477+0000","last_undegraded":"2024-09-17T04:30:01.023477+0000","last_fullsized":"2024-09-17T04:30:01.023477+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:44.121537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.036815+0000","last_change":"2024-09-17T04:30:00.032305+0000","last_active":"2024-09-17T04:30:00.036815+0000","last_peered":"2024-09-17T04:30:00.036815+0000","last_clean":"2024-09-17T04:30:00.036815+0000","last_became_active":"2024-09-17T04:30:00.031944+0000","last_became_peered":"2024-09-17T04:30:00.031944+0000","last_unstale":"2024-09-17T04:30:00.036815+0000","last_undegraded":"2024-09-17T04:30:00.036815+0000","last_fullsized":"2024-09-17T04:30:00.036815+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:11:10.891912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.034115+0000","last_change":"2024-09-17T04:30:00.025576+0000","last_active":"2024-09-17T04:30:00.034115+0000","last_peered":"2024-09-17T04:30:00.034115+0000","last_clean":"2024-09-17T04:30:00.034115+0000","last_became_active":"2024-09-17T04:30:00.025136+0000","last_became_peered":"2024-09-17T04:30:00.025136+0000","last_unstale":"2024-09-17T04:30:00.034115+0000","last_undegraded":"2024-09-17T04:30:00.034115+0000","last_fullsized":"2024-09-17T04:30:00.034115+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:45:52.659055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.021284+0000","last_change":"2024-09-17T04:30:00.011478+0000","last_active":"2024-09-17T04:30:01.021284+0000","last_peered":"2024-09-17T04:30:01.021284+0000","last_clean":"2024-09-17T04:30:01.021284+0000","last_became_active":"2024-09-17T04:30:00.011137+0000","last_became_peered":"2024-09-17T04:30:00.011137+0000","last_unstale":"2024-09-17T04:30:01.021284+0000","last_undegraded":"2024-09-17T04:30:01.021284+0000","last_fullsized":"2024-09-17T04:30:01.021284+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:51:11.302837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.240767+0000","last_change":"2024-09-17T04:30:00.036444+0000","last_active":"2024-09-17T04:30:01.240767+0000","last_peered":"2024-09-17T04:30:01.240767+0000","last_clean":"2024-09-17T04:30:01.240767+0000","last_became_active":"2024-09-17T04:30:00.035097+0000","last_became_peered":"2024-09-17T04:30:00.035097+0000","last_unstale":"2024-09-17T04:30:01.240767+0000","last_undegraded":"2024-09-17T04:30:01.240767+0000","last_fullsized":"2024-09-17T04:30:01.240767+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:58:26.889907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.023535+0000","last_change":"2024-09-17T04:30:00.023963+0000","last_active":"2024-09-17T04:30:01.023535+0000","last_peered":"2024-09-17T04:30:01.023535+0000","last_clean":"2024-09-17T04:30:01.023535+0000","last_became_active":"2024-09-17T04:30:00.023262+0000","last_became_peered":"2024-09-17T04:30:00.023262+0000","last_unstale":"2024-09-17T04:30:01.023535+0000","last_undegraded":"2024-09-17T04:30:01.023535+0000","last_fullsized":"2024-09-17T04:30:01.023535+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:40:00.038594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.027163+0000","last_change":"2024-09-17T04:30:00.026007+0000","last_active":"2024-09-17T04:30:00.027163+0000","last_peered":"2024-09-17T04:30:00.027163+0000","last_clean":"2024-09-17T04:30:00.027163+0000","last_became_active":"2024-09-17T04:30:00.025689+0000","last_became_peered":"2024-09-17T04:30:00.025689+0000","last_unstale":"2024-09-17T04:30:00.027163+0000","last_undegraded":"2024-09-17T04:30:00.027163+0000","last_fullsized":"2024-09-17T04:30:00.027163+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:50.175832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.047778+0000","last_change":"2024-09-17T04:30:00.038957+0000","last_active":"2024-09-17T04:30:00.047778+0000","last_peered":"2024-09-17T04:30:00.047778+0000","last_clean":"2024-09-17T04:30:00.047778+0000","last_became_active":"2024-09-17T04:30:00.038525+0000","last_became_peered":"2024-09-17T04:30:00.038525+0000","last_unstale":"2024-09-17T04:30:00.047778+0000","last_undegraded":"2024-09-17T04:30:00.047778+0000","last_fullsized":"2024-09-17T04:30:00.047778+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:49.903236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.046821+0000","last_change":"2024-09-17T04:30:00.033765+0000","last_active":"2024-09-17T04:30:00.046821+0000","last_peered":"2024-09-17T04:30:00.046821+0000","last_clean":"2024-09-17T04:30:00.046821+0000","last_became_active":"2024-09-17T04:30:00.033172+0000","last_became_peered":"2024-09-17T04:30:00.033172+0000","last_unstale":"2024-09-17T04:30:00.046821+0000","last_undegraded":"2024-09-17T04:30:00.046821+0000","last_fullsized":"2024-09-17T04:30:00.046821+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:49.787137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.023119+0000","last_change":"2024-09-17T04:30:00.023924+0000","last_active":"2024-09-17T04:30:01.023119+0000","last_peered":"2024-09-17T04:30:01.023119+0000","last_clean":"2024-09-17T04:30:01.023119+0000","last_became_active":"2024-09-17T04:30:00.023409+0000","last_became_peered":"2024-09-17T04:30:00.023409+0000","last_unstale":"2024-09-17T04:30:01.023119+0000","last_undegraded":"2024-09-17T04:30:01.023119+0000","last_fullsized":"2024-09-17T04:30:01.023119+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:05:21.232149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.044201+0000","last_change":"2024-09-17T04:30:00.031748+0000","last_active":"2024-09-17T04:30:00.044201+0000","last_peered":"2024-09-17T04:30:00.044201+0000","last_clean":"2024-09-17T04:30:00.044201+0000","last_became_active":"2024-09-17T04:30:00.031370+0000","last_became_peered":"2024-09-17T04:30:00.031370+0000","last_unstale":"2024-09-17T04:30:00.044201+0000","last_undegraded":"2024-09-17T04:30:00.044201+0000","last_fullsized":"2024-09-17T04:30:00.044201+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:12:57.242715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.038862+0000","last_change":"2024-09-17T04:30:00.033820+0000","last_active":"2024-09-17T04:30:00.038862+0000","last_peered":"2024-09-17T04:30:00.038862+0000","last_clean":"2024-09-17T04:30:00.038862+0000","last_became_active":"2024-09-17T04:30:00.033435+0000","last_became_peered":"2024-09-17T04:30:00.033435+0000","last_unstale":"2024-09-17T04:30:00.038862+0000","last_undegraded":"2024-09-17T04:30:00.038862+0000","last_fullsized":"2024-09-17T04:30:00.038862+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:25.595122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.039465+0000","last_change":"2024-09-17T04:30:00.032263+0000","last_active":"2024-09-17T04:30:00.039465+0000","last_peered":"2024-09-17T04:30:00.039465+0000","last_clean":"2024-09-17T04:30:00.039465+0000","last_became_active":"2024-09-17T04:30:00.031102+0000","last_became_peered":"2024-09-17T04:30:00.031102+0000","last_unstale":"2024-09-17T04:30:00.039465+0000","last_undegraded":"2024-09-17T04:30:00.039465+0000","last_fullsized":"2024-09-17T04:30:00.039465+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:25.300591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.240776+0000","last_change":"2024-09-17T04:30:00.035400+0000","last_active":"2024-09-17T04:30:01.240776+0000","last_peered":"2024-09-17T04:30:01.240776+0000","last_clean":"2024-09-17T04:30:01.240776+0000","last_became_active":"2024-09-17T04:30:00.035073+0000","last_became_peered":"2024-09-17T04:30:00.035073+0000","last_unstale":"2024-09-17T04:30:01.240776+0000","last_undegraded":"2024-09-17T04:30:01.240776+0000","last_fullsized":"2024-09-17T04:30:01.240776+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:18:55.077953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.042374+0000","last_change":"2024-09-17T04:30:00.031905+0000","last_active":"2024-09-17T04:30:00.042374+0000","last_peered":"2024-09-17T04:30:00.042374+0000","last_clean":"2024-09-17T04:30:00.042374+0000","last_became_active":"2024-09-17T04:30:00.030633+0000","last_became_peered":"2024-09-17T04:30:00.030633+0000","last_unstale":"2024-09-17T04:30:00.042374+0000","last_undegraded":"2024-09-17T04:30:00.042374+0000","last_fullsized":"2024-09-17T04:30:00.042374+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:23:39.082595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.023187+0000","last_change":"2024-09-17T04:30:00.032923+0000","last_active":"2024-09-17T04:30:01.023187+0000","last_peered":"2024-09-17T04:30:01.023187+0000","last_clean":"2024-09-17T04:30:01.023187+0000","last_became_active":"2024-09-17T04:30:00.030726+0000","last_became_peered":"2024-09-17T04:30:00.030726+0000","last_unstale":"2024-09-17T04:30:01.023187+0000","last_undegraded":"2024-09-17T04:30:01.023187+0000","last_fullsized":"2024-09-17T04:30:01.023187+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:35.261999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.027335+0000","last_change":"2024-09-17T04:30:00.033675+0000","last_active":"2024-09-17T04:30:01.027335+0000","last_peered":"2024-09-17T04:30:01.027335+0000","last_clean":"2024-09-17T04:30:01.027335+0000","last_became_active":"2024-09-17T04:30:00.033337+0000","last_became_peered":"2024-09-17T04:30:00.033337+0000","last_unstale":"2024-09-17T04:30:01.027335+0000","last_undegraded":"2024-09-17T04:30:01.027335+0000","last_fullsized":"2024-09-17T04:30:01.027335+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:55.361480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.037764+0000","last_change":"2024-09-17T04:30:00.034215+0000","last_active":"2024-09-17T04:30:00.037764+0000","last_peered":"2024-09-17T04:30:00.037764+0000","last_clean":"2024-09-17T04:30:00.037764+0000","last_became_active":"2024-09-17T04:30:00.033859+0000","last_became_peered":"2024-09-17T04:30:00.033859+0000","last_unstale":"2024-09-17T04:30:00.037764+0000","last_undegraded":"2024-09-17T04:30:00.037764+0000","last_fullsized":"2024-09-17T04:30:00.037764+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:53:33.901940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.023592+0000","last_change":"2024-09-17T04:30:00.026637+0000","last_active":"2024-09-17T04:30:01.023592+0000","last_peered":"2024-09-17T04:30:01.023592+0000","last_clean":"2024-09-17T04:30:01.023592+0000","last_became_active":"2024-09-17T04:30:00.024736+0000","last_became_peered":"2024-09-17T04:30:00.024736+0000","last_unstale":"2024-09-17T04:30:01.023592+0000","last_undegraded":"2024-09-17T04:30:01.023592+0000","last_fullsized":"2024-09-17T04:30:01.023592+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:32:09.600854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.023316+0000","last_change":"2024-09-17T04:30:00.013520+0000","last_active":"2024-09-17T04:30:01.023316+0000","last_peered":"2024-09-17T04:30:01.023316+0000","last_clean":"2024-09-17T04:30:01.023316+0000","last_became_active":"2024-09-17T04:30:00.013208+0000","last_became_peered":"2024-09-17T04:30:00.013208+0000","last_unstale":"2024-09-17T04:30:01.023316+0000","last_undegraded":"2024-09-17T04:30:01.023316+0000","last_fullsized":"2024-09-17T04:30:01.023316+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:56:34.201971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.032949+0000","last_change":"2024-09-17T04:30:00.026054+0000","last_active":"2024-09-17T04:30:00.032949+0000","last_peered":"2024-09-17T04:30:00.032949+0000","last_clean":"2024-09-17T04:30:00.032949+0000","last_became_active":"2024-09-17T04:30:00.025677+0000","last_became_peered":"2024-09-17T04:30:00.025677+0000","last_unstale":"2024-09-17T04:30:00.032949+0000","last_undegraded":"2024-09-17T04:30:00.032949+0000","last_fullsized":"2024-09-17T04:30:00.032949+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:41.639339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.022021+0000","last_change":"2024-09-17T04:30:00.026955+0000","last_active":"2024-09-17T04:30:01.022021+0000","last_peered":"2024-09-17T04:30:01.022021+0000","last_clean":"2024-09-17T04:30:01.022021+0000","last_became_active":"2024-09-17T04:30:00.024869+0000","last_became_peered":"2024-09-17T04:30:00.024869+0000","last_unstale":"2024-09-17T04:30:01.022021+0000","last_undegraded":"2024-09-17T04:30:01.022021+0000","last_fullsized":"2024-09-17T04:30:01.022021+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:45.610227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.031583+0000","last_change":"2024-09-17T04:30:00.022611+0000","last_active":"2024-09-17T04:30:00.031583+0000","last_peered":"2024-09-17T04:30:00.031583+0000","last_clean":"2024-09-17T04:30:00.031583+0000","last_became_active":"2024-09-17T04:30:00.021870+0000","last_became_peered":"2024-09-17T04:30:00.021870+0000","last_unstale":"2024-09-17T04:30:00.031583+0000","last_undegraded":"2024-09-17T04:30:00.031583+0000","last_fullsized":"2024-09-17T04:30:00.031583+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:36.864227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.023226+0000","last_change":"2024-09-17T04:30:00.023068+0000","last_active":"2024-09-17T04:30:01.023226+0000","last_peered":"2024-09-17T04:30:01.023226+0000","last_clean":"2024-09-17T04:30:01.023226+0000","last_became_active":"2024-09-17T04:30:00.022718+0000","last_became_peered":"2024-09-17T04:30:00.022718+0000","last_unstale":"2024-09-17T04:30:01.023226+0000","last_undegraded":"2024-09-17T04:30:01.023226+0000","last_fullsized":"2024-09-17T04:30:01.023226+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:54:19.658662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.032965+0000","last_change":"2024-09-17T04:30:00.026274+0000","last_active":"2024-09-17T04:30:00.032965+0000","last_peered":"2024-09-17T04:30:00.032965+0000","last_clean":"2024-09-17T04:30:00.032965+0000","last_became_active":"2024-09-17T04:30:00.025962+0000","last_became_peered":"2024-09-17T04:30:00.025962+0000","last_unstale":"2024-09-17T04:30:00.032965+0000","last_undegraded":"2024-09-17T04:30:00.032965+0000","last_fullsized":"2024-09-17T04:30:00.032965+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:05.524758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.241286+0000","last_change":"2024-09-17T04:30:00.041368+0000","last_active":"2024-09-17T04:30:01.241286+0000","last_peered":"2024-09-17T04:30:01.241286+0000","last_clean":"2024-09-17T04:30:01.241286+0000","last_became_active":"2024-09-17T04:30:00.041015+0000","last_became_peered":"2024-09-17T04:30:00.041015+0000","last_unstale":"2024-09-17T04:30:01.241286+0000","last_undegraded":"2024-09-17T04:30:01.241286+0000","last_fullsized":"2024-09-17T04:30:01.241286+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:07.024004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.034010+0000","last_change":"2024-09-17T04:30:00.029929+0000","last_active":"2024-09-17T04:30:00.034010+0000","last_peered":"2024-09-17T04:30:00.034010+0000","last_clean":"2024-09-17T04:30:00.034010+0000","last_became_active":"2024-09-17T04:30:00.029639+0000","last_became_peered":"2024-09-17T04:30:00.029639+0000","last_unstale":"2024-09-17T04:30:00.034010+0000","last_undegraded":"2024-09-17T04:30:00.034010+0000","last_fullsized":"2024-09-17T04:30:00.034010+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:12:00.589604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.022811+0000","last_change":"2024-09-17T04:30:00.023591+0000","last_active":"2024-09-17T04:30:01.022811+0000","last_peered":"2024-09-17T04:30:01.022811+0000","last_clean":"2024-09-17T04:30:01.022811+0000","last_became_active":"2024-09-17T04:30:00.021925+0000","last_became_peered":"2024-09-17T04:30:00.021925+0000","last_unstale":"2024-09-17T04:30:01.022811+0000","last_undegraded":"2024-09-17T04:30:01.022811+0000","last_fullsized":"2024-09-17T04:30:01.022811+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:00.376670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.024004+0000","last_change":"2024-09-17T04:30:00.013112+0000","last_active":"2024-09-17T04:30:01.024004+0000","last_peered":"2024-09-17T04:30:01.024004+0000","last_clean":"2024-09-17T04:30:01.024004+0000","last_became_active":"2024-09-17T04:30:00.012655+0000","last_became_peered":"2024-09-17T04:30:00.012655+0000","last_unstale":"2024-09-17T04:30:01.024004+0000","last_undegraded":"2024-09-17T04:30:01.024004+0000","last_fullsized":"2024-09-17T04:30:01.024004+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:05:25.837826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.032358+0000","last_change":"2024-09-17T04:30:00.028487+0000","last_active":"2024-09-17T04:30:00.032358+0000","last_peered":"2024-09-17T04:30:00.032358+0000","last_clean":"2024-09-17T04:30:00.032358+0000","last_became_active":"2024-09-17T04:30:00.028125+0000","last_became_peered":"2024-09-17T04:30:00.028125+0000","last_unstale":"2024-09-17T04:30:00.032358+0000","last_undegraded":"2024-09-17T04:30:00.032358+0000","last_fullsized":"2024-09-17T04:30:00.032358+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:38.160220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.240782+0000","last_change":"2024-09-17T04:30:00.028467+0000","last_active":"2024-09-17T04:30:01.240782+0000","last_peered":"2024-09-17T04:30:01.240782+0000","last_clean":"2024-09-17T04:30:01.240782+0000","last_became_active":"2024-09-17T04:30:00.028170+0000","last_became_peered":"2024-09-17T04:30:00.028170+0000","last_unstale":"2024-09-17T04:30:01.240782+0000","last_undegraded":"2024-09-17T04:30:01.240782+0000","last_fullsized":"2024-09-17T04:30:01.240782+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:18:30.492299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.241352+0000","last_change":"2024-09-17T04:30:00.031831+0000","last_active":"2024-09-17T04:30:01.241352+0000","last_peered":"2024-09-17T04:30:01.241352+0000","last_clean":"2024-09-17T04:30:01.241352+0000","last_became_active":"2024-09-17T04:30:00.031530+0000","last_became_peered":"2024-09-17T04:30:00.031530+0000","last_unstale":"2024-09-17T04:30:01.241352+0000","last_undegraded":"2024-09-17T04:30:01.241352+0000","last_fullsized":"2024-09-17T04:30:01.241352+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:06.907531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"104'4","reported_seq":19,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.024060+0000","last_change":"2024-09-17T04:29:59.000936+0000","last_active":"2024-09-17T04:30:01.024060+0000","last_peered":"2024-09-17T04:30:01.024060+0000","last_clean":"2024-09-17T04:30:01.024060+0000","last_became_active":"2024-09-17T04:29:59.000619+0000","last_became_peered":"2024-09-17T04:29:59.000619+0000","last_unstale":"2024-09-17T04:30:01.024060+0000","last_undegraded":"2024-09-17T04:30:01.024060+0000","last_fullsized":"2024-09-17T04:30:01.024060+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:54:35.865664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.027605+0000","last_change":"2024-09-17T04:30:00.025354+0000","last_active":"2024-09-17T04:30:01.027605+0000","last_peered":"2024-09-17T04:30:01.027605+0000","last_clean":"2024-09-17T04:30:01.027605+0000","last_became_active":"2024-09-17T04:30:00.024223+0000","last_became_peered":"2024-09-17T04:30:00.024223+0000","last_unstale":"2024-09-17T04:30:01.027605+0000","last_undegraded":"2024-09-17T04:30:01.027605+0000","last_fullsized":"2024-09-17T04:30:01.027605+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:35.261999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"104'1","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.041962+0000","last_change":"2024-09-17T04:29:59.005274+0000","last_active":"2024-09-17T04:30:00.041962+0000","last_peered":"2024-09-17T04:30:00.041962+0000","last_clean":"2024-09-17T04:30:00.041962+0000","last_became_active":"2024-09-17T04:29:59.004956+0000","last_became_peered":"2024-09-17T04:29:59.004956+0000","last_unstale":"2024-09-17T04:30:00.041962+0000","last_undegraded":"2024-09-17T04:30:00.041962+0000","last_fullsized":"2024-09-17T04:30:00.041962+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:19:02.665241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.241418+0000","last_change":"2024-09-17T04:30:00.037349+0000","last_active":"2024-09-17T04:30:01.241418+0000","last_peered":"2024-09-17T04:30:01.241418+0000","last_clean":"2024-09-17T04:30:01.241418+0000","last_became_active":"2024-09-17T04:30:00.035164+0000","last_became_peered":"2024-09-17T04:30:00.035164+0000","last_unstale":"2024-09-17T04:30:01.241418+0000","last_undegraded":"2024-09-17T04:30:01.241418+0000","last_fullsized":"2024-09-17T04:30:01.241418+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:48.127054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.021128+0000","last_change":"2024-09-17T04:29:59.018202+0000","last_active":"2024-09-17T04:30:01.021128+0000","last_peered":"2024-09-17T04:30:01.021128+0000","last_clean":"2024-09-17T04:30:01.021128+0000","last_became_active":"2024-09-17T04:29:59.017545+0000","last_became_peered":"2024-09-17T04:29:59.017545+0000","last_unstale":"2024-09-17T04:30:01.021128+0000","last_undegraded":"2024-09-17T04:30:01.021128+0000","last_fullsized":"2024-09-17T04:30:01.021128+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:41:49.363538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.037737+0000","last_change":"2024-09-17T04:30:00.034076+0000","last_active":"2024-09-17T04:30:00.037737+0000","last_peered":"2024-09-17T04:30:00.037737+0000","last_clean":"2024-09-17T04:30:00.037737+0000","last_became_active":"2024-09-17T04:30:00.033709+0000","last_became_peered":"2024-09-17T04:30:00.033709+0000","last_unstale":"2024-09-17T04:30:00.037737+0000","last_undegraded":"2024-09-17T04:30:00.037737+0000","last_fullsized":"2024-09-17T04:30:00.037737+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:43:29.810312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.020947+0000","last_change":"2024-09-17T04:29:59.009241+0000","last_active":"2024-09-17T04:30:01.020947+0000","last_peered":"2024-09-17T04:30:01.020947+0000","last_clean":"2024-09-17T04:30:01.020947+0000","last_became_active":"2024-09-17T04:29:59.006633+0000","last_became_peered":"2024-09-17T04:29:59.006633+0000","last_unstale":"2024-09-17T04:30:01.020947+0000","last_undegraded":"2024-09-17T04:30:01.020947+0000","last_fullsized":"2024-09-17T04:30:01.020947+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:01:05.139820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.048802+0000","last_change":"2024-09-17T04:30:00.039327+0000","last_active":"2024-09-17T04:30:00.048802+0000","last_peered":"2024-09-17T04:30:00.048802+0000","last_clean":"2024-09-17T04:30:00.048802+0000","last_became_active":"2024-09-17T04:30:00.039006+0000","last_became_peered":"2024-09-17T04:30:00.039006+0000","last_unstale":"2024-09-17T04:30:00.048802+0000","last_undegraded":"2024-09-17T04:30:00.048802+0000","last_fullsized":"2024-09-17T04:30:00.048802+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:59:46.857615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.023751+0000","last_change":"2024-09-17T04:29:59.010630+0000","last_active":"2024-09-17T04:30:01.023751+0000","last_peered":"2024-09-17T04:30:01.023751+0000","last_clean":"2024-09-17T04:30:01.023751+0000","last_became_active":"2024-09-17T04:29:59.009923+0000","last_became_peered":"2024-09-17T04:29:59.009923+0000","last_unstale":"2024-09-17T04:30:01.023751+0000","last_undegraded":"2024-09-17T04:30:01.023751+0000","last_fullsized":"2024-09-17T04:30:01.023751+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:08:28.309920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.022426+0000","last_change":"2024-09-17T04:30:00.031279+0000","last_active":"2024-09-17T04:30:01.022426+0000","last_peered":"2024-09-17T04:30:01.022426+0000","last_clean":"2024-09-17T04:30:01.022426+0000","last_became_active":"2024-09-17T04:30:00.030957+0000","last_became_peered":"2024-09-17T04:30:00.030957+0000","last_unstale":"2024-09-17T04:30:01.022426+0000","last_undegraded":"2024-09-17T04:30:01.022426+0000","last_fullsized":"2024-09-17T04:30:01.022426+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:19:30.797559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"104'1","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.040233+0000","last_change":"2024-09-17T04:29:59.011407+0000","last_active":"2024-09-17T04:30:00.040233+0000","last_peered":"2024-09-17T04:30:00.040233+0000","last_clean":"2024-09-17T04:30:00.040233+0000","last_became_active":"2024-09-17T04:29:59.007991+0000","last_became_peered":"2024-09-17T04:29:59.007991+0000","last_unstale":"2024-09-17T04:30:00.040233+0000","last_undegraded":"2024-09-17T04:30:00.040233+0000","last_fullsized":"2024-09-17T04:30:00.040233+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:22.125466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.047839+0000","last_change":"2024-09-17T04:30:00.042491+0000","last_active":"2024-09-17T04:30:00.047839+0000","last_peered":"2024-09-17T04:30:00.047839+0000","last_clean":"2024-09-17T04:30:00.047839+0000","last_became_active":"2024-09-17T04:30:00.042160+0000","last_became_peered":"2024-09-17T04:30:00.042160+0000","last_unstale":"2024-09-17T04:30:00.047839+0000","last_undegraded":"2024-09-17T04:30:00.047839+0000","last_fullsized":"2024-09-17T04:30:00.047839+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:24:20.303018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.022344+0000","last_change":"2024-09-17T04:29:59.004824+0000","last_active":"2024-09-17T04:30:01.022344+0000","last_peered":"2024-09-17T04:30:01.022344+0000","last_clean":"2024-09-17T04:30:01.022344+0000","last_became_active":"2024-09-17T04:29:59.003696+0000","last_became_peered":"2024-09-17T04:29:59.003696+0000","last_unstale":"2024-09-17T04:30:01.022344+0000","last_undegraded":"2024-09-17T04:30:01.022344+0000","last_fullsized":"2024-09-17T04:30:01.022344+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:49.166951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.023281+0000","last_change":"2024-09-17T04:30:00.024319+0000","last_active":"2024-09-17T04:30:01.023281+0000","last_peered":"2024-09-17T04:30:01.023281+0000","last_clean":"2024-09-17T04:30:01.023281+0000","last_became_active":"2024-09-17T04:30:00.023658+0000","last_became_peered":"2024-09-17T04:30:00.023658+0000","last_unstale":"2024-09-17T04:30:01.023281+0000","last_undegraded":"2024-09-17T04:30:01.023281+0000","last_fullsized":"2024-09-17T04:30:01.023281+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:19:12.584918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.016788+0000","last_change":"2024-09-17T04:29:59.010122+0000","last_active":"2024-09-17T04:30:00.016788+0000","last_peered":"2024-09-17T04:30:00.016788+0000","last_clean":"2024-09-17T04:30:00.016788+0000","last_became_active":"2024-09-17T04:29:59.009781+0000","last_became_peered":"2024-09-17T04:29:59.009781+0000","last_unstale":"2024-09-17T04:30:00.016788+0000","last_undegraded":"2024-09-17T04:30:00.016788+0000","last_fullsized":"2024-09-17T04:30:00.016788+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:32:08.591973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.024362+0000","last_change":"2024-09-17T04:30:00.013587+0000","last_active":"2024-09-17T04:30:01.024362+0000","last_peered":"2024-09-17T04:30:01.024362+0000","last_clean":"2024-09-17T04:30:01.024362+0000","last_became_active":"2024-09-17T04:30:00.013146+0000","last_became_peered":"2024-09-17T04:30:00.013146+0000","last_unstale":"2024-09-17T04:30:01.024362+0000","last_undegraded":"2024-09-17T04:30:01.024362+0000","last_fullsized":"2024-09-17T04:30:01.024362+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:25:51.689001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.021422+0000","last_change":"2024-09-17T04:29:59.012316+0000","last_active":"2024-09-17T04:30:01.021422+0000","last_peered":"2024-09-17T04:30:01.021422+0000","last_clean":"2024-09-17T04:30:01.021422+0000","last_became_active":"2024-09-17T04:29:59.012001+0000","last_became_peered":"2024-09-17T04:29:59.012001+0000","last_unstale":"2024-09-17T04:30:01.021422+0000","last_undegraded":"2024-09-17T04:30:01.021422+0000","last_fullsized":"2024-09-17T04:30:01.021422+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:06:17.905426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.027752+0000","last_change":"2024-09-17T04:30:00.035515+0000","last_active":"2024-09-17T04:30:01.027752+0000","last_peered":"2024-09-17T04:30:01.027752+0000","last_clean":"2024-09-17T04:30:01.027752+0000","last_became_active":"2024-09-17T04:30:00.035190+0000","last_became_peered":"2024-09-17T04:30:00.035190+0000","last_unstale":"2024-09-17T04:30:01.027752+0000","last_undegraded":"2024-09-17T04:30:01.027752+0000","last_fullsized":"2024-09-17T04:30:01.027752+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:54:19.658662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.009988+0000","last_change":"2024-09-17T04:29:59.012130+0000","last_active":"2024-09-17T04:30:00.009988+0000","last_peered":"2024-09-17T04:30:00.009988+0000","last_clean":"2024-09-17T04:30:00.009988+0000","last_became_active":"2024-09-17T04:29:59.011625+0000","last_became_peered":"2024-09-17T04:29:59.011625+0000","last_unstale":"2024-09-17T04:30:00.009988+0000","last_undegraded":"2024-09-17T04:30:00.009988+0000","last_fullsized":"2024-09-17T04:30:00.009988+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:28:15.481834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.024913+0000","last_change":"2024-09-17T04:30:00.019717+0000","last_active":"2024-09-17T04:30:00.024913+0000","last_peered":"2024-09-17T04:30:00.024913+0000","last_clean":"2024-09-17T04:30:00.024913+0000","last_became_active":"2024-09-17T04:30:00.017527+0000","last_became_peered":"2024-09-17T04:30:00.017527+0000","last_unstale":"2024-09-17T04:30:00.024913+0000","last_undegraded":"2024-09-17T04:30:00.024913+0000","last_fullsized":"2024-09-17T04:30:00.024913+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:28:45.075635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"104'3","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.053077+0000","last_change":"2024-09-17T04:29:59.017863+0000","last_active":"2024-09-17T04:30:00.053077+0000","last_peered":"2024-09-17T04:30:00.053077+0000","last_clean":"2024-09-17T04:30:00.053077+0000","last_became_active":"2024-09-17T04:29:59.017383+0000","last_became_peered":"2024-09-17T04:29:59.017383+0000","last_unstale":"2024-09-17T04:30:00.053077+0000","last_undegraded":"2024-09-17T04:30:00.053077+0000","last_fullsized":"2024-09-17T04:30:00.053077+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:56:33.193090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.024254+0000","last_change":"2024-09-17T04:30:00.019174+0000","last_active":"2024-09-17T04:30:01.024254+0000","last_peered":"2024-09-17T04:30:01.024254+0000","last_clean":"2024-09-17T04:30:01.024254+0000","last_became_active":"2024-09-17T04:30:00.018865+0000","last_became_peered":"2024-09-17T04:30:00.018865+0000","last_unstale":"2024-09-17T04:30:01.024254+0000","last_undegraded":"2024-09-17T04:30:01.024254+0000","last_fullsized":"2024-09-17T04:30:01.024254+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:56.828497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"104'6","reported_seq":19,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.055093+0000","last_change":"2024-09-17T04:29:59.016065+0000","last_active":"2024-09-17T04:30:00.055093+0000","last_peered":"2024-09-17T04:30:00.055093+0000","last_clean":"2024-09-17T04:30:00.055093+0000","last_became_active":"2024-09-17T04:29:59.015764+0000","last_became_peered":"2024-09-17T04:29:59.015764+0000","last_unstale":"2024-09-17T04:30:00.055093+0000","last_undegraded":"2024-09-17T04:30:00.055093+0000","last_fullsized":"2024-09-17T04:30:00.055093+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:13.616467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.016619+0000","last_change":"2024-09-17T04:30:00.013540+0000","last_active":"2024-09-17T04:30:00.016619+0000","last_peered":"2024-09-17T04:30:00.016619+0000","last_clean":"2024-09-17T04:30:00.016619+0000","last_became_active":"2024-09-17T04:30:00.013151+0000","last_became_peered":"2024-09-17T04:30:00.013151+0000","last_unstale":"2024-09-17T04:30:00.016619+0000","last_undegraded":"2024-09-17T04:30:00.016619+0000","last_fullsized":"2024-09-17T04:30:00.016619+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:42.584154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.241058+0000","last_change":"2024-09-17T04:29:59.013264+0000","last_active":"2024-09-17T04:30:01.241058+0000","last_peered":"2024-09-17T04:30:01.241058+0000","last_clean":"2024-09-17T04:30:01.241058+0000","last_became_active":"2024-09-17T04:29:59.012844+0000","last_became_peered":"2024-09-17T04:29:59.012844+0000","last_unstale":"2024-09-17T04:30:01.241058+0000","last_undegraded":"2024-09-17T04:30:01.241058+0000","last_fullsized":"2024-09-17T04:30:01.241058+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:51:10.293956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.032729+0000","last_change":"2024-09-17T04:30:00.026868+0000","last_active":"2024-09-17T04:30:00.032729+0000","last_peered":"2024-09-17T04:30:00.032729+0000","last_clean":"2024-09-17T04:30:00.032729+0000","last_became_active":"2024-09-17T04:30:00.026582+0000","last_became_peered":"2024-09-17T04:30:00.026582+0000","last_unstale":"2024-09-17T04:30:00.032729+0000","last_undegraded":"2024-09-17T04:30:00.032729+0000","last_fullsized":"2024-09-17T04:30:00.032729+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:47:25.619529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.027276+0000","last_change":"2024-09-17T04:29:59.007287+0000","last_active":"2024-09-17T04:30:01.027276+0000","last_peered":"2024-09-17T04:30:01.027276+0000","last_clean":"2024-09-17T04:30:01.027276+0000","last_became_active":"2024-09-17T04:29:59.006446+0000","last_became_peered":"2024-09-17T04:29:59.006446+0000","last_unstale":"2024-09-17T04:30:01.027276+0000","last_undegraded":"2024-09-17T04:30:01.027276+0000","last_fullsized":"2024-09-17T04:30:01.027276+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:53:32.893059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.033321+0000","last_change":"2024-09-17T04:30:00.027527+0000","last_active":"2024-09-17T04:30:00.033321+0000","last_peered":"2024-09-17T04:30:00.033321+0000","last_clean":"2024-09-17T04:30:00.033321+0000","last_became_active":"2024-09-17T04:30:00.023985+0000","last_became_peered":"2024-09-17T04:30:00.023985+0000","last_unstale":"2024-09-17T04:30:00.033321+0000","last_undegraded":"2024-09-17T04:30:00.033321+0000","last_fullsized":"2024-09-17T04:30:00.033321+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:22.339699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"104'8","reported_seq":23,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.020948+0000","last_change":"2024-09-17T04:29:59.011479+0000","last_active":"2024-09-17T04:30:01.020948+0000","last_peered":"2024-09-17T04:30:01.020948+0000","last_clean":"2024-09-17T04:30:01.020948+0000","last_became_active":"2024-09-17T04:29:59.011162+0000","last_became_peered":"2024-09-17T04:29:59.011162+0000","last_unstale":"2024-09-17T04:30:01.020948+0000","last_undegraded":"2024-09-17T04:30:01.020948+0000","last_fullsized":"2024-09-17T04:30:01.020948+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:59.077658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.021351+0000","last_change":"2024-09-17T04:30:00.026452+0000","last_active":"2024-09-17T04:30:01.021351+0000","last_peered":"2024-09-17T04:30:01.021351+0000","last_clean":"2024-09-17T04:30:01.021351+0000","last_became_active":"2024-09-17T04:30:00.026087+0000","last_became_peered":"2024-09-17T04:30:00.026087+0000","last_unstale":"2024-09-17T04:30:01.021351+0000","last_undegraded":"2024-09-17T04:30:01.021351+0000","last_fullsized":"2024-09-17T04:30:01.021351+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:01:35.454605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"104'1","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.048906+0000","last_change":"2024-09-17T04:29:59.013573+0000","last_active":"2024-09-17T04:30:00.048906+0000","last_peered":"2024-09-17T04:30:00.048906+0000","last_clean":"2024-09-17T04:30:00.048906+0000","last_became_active":"2024-09-17T04:29:59.013307+0000","last_became_peered":"2024-09-17T04:29:59.013307+0000","last_unstale":"2024-09-17T04:30:00.048906+0000","last_undegraded":"2024-09-17T04:30:00.048906+0000","last_fullsized":"2024-09-17T04:30:00.048906+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:43.112656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.241276+0000","last_change":"2024-09-17T04:30:00.042113+0000","last_active":"2024-09-17T04:30:01.241276+0000","last_peered":"2024-09-17T04:30:01.241276+0000","last_clean":"2024-09-17T04:30:01.241276+0000","last_became_active":"2024-09-17T04:30:00.041788+0000","last_became_peered":"2024-09-17T04:30:00.041788+0000","last_unstale":"2024-09-17T04:30:01.241276+0000","last_undegraded":"2024-09-17T04:30:01.241276+0000","last_fullsized":"2024-09-17T04:30:01.241276+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:50:54.585046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.021326+0000","last_change":"2024-09-17T04:29:58.999969+0000","last_active":"2024-09-17T04:30:01.021326+0000","last_peered":"2024-09-17T04:30:01.021326+0000","last_clean":"2024-09-17T04:30:01.021326+0000","last_became_active":"2024-09-17T04:29:58.998593+0000","last_became_peered":"2024-09-17T04:29:58.998593+0000","last_unstale":"2024-09-17T04:30:01.021326+0000","last_undegraded":"2024-09-17T04:30:01.021326+0000","last_fullsized":"2024-09-17T04:30:01.021326+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:38:34.113452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.040125+0000","last_change":"2024-09-17T04:30:00.039039+0000","last_active":"2024-09-17T04:30:00.040125+0000","last_peered":"2024-09-17T04:30:00.040125+0000","last_clean":"2024-09-17T04:30:00.040125+0000","last_became_active":"2024-09-17T04:30:00.038305+0000","last_became_peered":"2024-09-17T04:30:00.038305+0000","last_unstale":"2024-09-17T04:30:00.040125+0000","last_undegraded":"2024-09-17T04:30:00.040125+0000","last_fullsized":"2024-09-17T04:30:00.040125+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:57:59.119887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.027348+0000","last_change":"2024-09-17T04:29:59.007492+0000","last_active":"2024-09-17T04:30:01.027348+0000","last_peered":"2024-09-17T04:30:01.027348+0000","last_clean":"2024-09-17T04:30:01.027348+0000","last_became_active":"2024-09-17T04:29:59.006443+0000","last_became_peered":"2024-09-17T04:29:59.006443+0000","last_unstale":"2024-09-17T04:30:01.027348+0000","last_undegraded":"2024-09-17T04:30:01.027348+0000","last_fullsized":"2024-09-17T04:30:01.027348+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:43:28.801432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.037613+0000","last_change":"2024-09-17T04:30:00.032254+0000","last_active":"2024-09-17T04:30:00.037613+0000","last_peered":"2024-09-17T04:30:00.037613+0000","last_clean":"2024-09-17T04:30:00.037613+0000","last_became_active":"2024-09-17T04:30:00.031486+0000","last_became_peered":"2024-09-17T04:30:00.031486+0000","last_unstale":"2024-09-17T04:30:00.037613+0000","last_undegraded":"2024-09-17T04:30:00.037613+0000","last_fullsized":"2024-09-17T04:30:00.037613+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:58.219483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.023919+0000","last_change":"2024-09-17T04:29:59.008591+0000","last_active":"2024-09-17T04:30:01.023919+0000","last_peered":"2024-09-17T04:30:01.023919+0000","last_clean":"2024-09-17T04:30:01.023919+0000","last_became_active":"2024-09-17T04:29:59.008142+0000","last_became_peered":"2024-09-17T04:29:59.008142+0000","last_unstale":"2024-09-17T04:30:01.023919+0000","last_undegraded":"2024-09-17T04:30:01.023919+0000","last_fullsized":"2024-09-17T04:30:01.023919+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:43:28.696763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.039494+0000","last_change":"2024-09-17T04:30:00.035767+0000","last_active":"2024-09-17T04:30:00.039494+0000","last_peered":"2024-09-17T04:30:00.039494+0000","last_clean":"2024-09-17T04:30:00.039494+0000","last_became_active":"2024-09-17T04:30:00.034914+0000","last_became_peered":"2024-09-17T04:30:00.034914+0000","last_unstale":"2024-09-17T04:30:00.039494+0000","last_undegraded":"2024-09-17T04:30:00.039494+0000","last_fullsized":"2024-09-17T04:30:00.039494+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:05:08.139172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.241100+0000","last_change":"2024-09-17T04:29:59.016631+0000","last_active":"2024-09-17T04:30:01.241100+0000","last_peered":"2024-09-17T04:30:01.241100+0000","last_clean":"2024-09-17T04:30:01.241100+0000","last_became_active":"2024-09-17T04:29:59.016246+0000","last_became_peered":"2024-09-17T04:29:59.016246+0000","last_unstale":"2024-09-17T04:30:01.241100+0000","last_undegraded":"2024-09-17T04:30:01.241100+0000","last_fullsized":"2024-09-17T04:30:01.241100+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:01:34.445725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.040463+0000","last_change":"2024-09-17T04:30:00.032092+0000","last_active":"2024-09-17T04:30:00.040463+0000","last_peered":"2024-09-17T04:30:00.040463+0000","last_clean":"2024-09-17T04:30:00.040463+0000","last_became_active":"2024-09-17T04:30:00.031742+0000","last_became_peered":"2024-09-17T04:30:00.031742+0000","last_unstale":"2024-09-17T04:30:00.040463+0000","last_undegraded":"2024-09-17T04:30:00.040463+0000","last_fullsized":"2024-09-17T04:30:00.040463+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:56.769785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.020935+0000","last_change":"2024-09-17T04:29:59.016046+0000","last_active":"2024-09-17T04:30:01.020935+0000","last_peered":"2024-09-17T04:30:01.020935+0000","last_clean":"2024-09-17T04:30:01.020935+0000","last_became_active":"2024-09-17T04:29:59.015762+0000","last_became_peered":"2024-09-17T04:29:59.015762+0000","last_unstale":"2024-09-17T04:30:01.020935+0000","last_undegraded":"2024-09-17T04:30:01.020935+0000","last_fullsized":"2024-09-17T04:30:01.020935+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:19.680096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.039728+0000","last_change":"2024-09-17T04:30:00.033808+0000","last_active":"2024-09-17T04:30:00.039728+0000","last_peered":"2024-09-17T04:30:00.039728+0000","last_clean":"2024-09-17T04:30:00.039728+0000","last_became_active":"2024-09-17T04:30:00.033381+0000","last_became_peered":"2024-09-17T04:30:00.033381+0000","last_unstale":"2024-09-17T04:30:00.039728+0000","last_undegraded":"2024-09-17T04:30:00.039728+0000","last_fullsized":"2024-09-17T04:30:00.039728+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:05:25.837826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.240644+0000","last_change":"2024-09-17T04:29:59.011744+0000","last_active":"2024-09-17T04:30:01.240644+0000","last_peered":"2024-09-17T04:30:01.240644+0000","last_clean":"2024-09-17T04:30:01.240644+0000","last_became_active":"2024-09-17T04:29:59.011460+0000","last_became_peered":"2024-09-17T04:29:59.011460+0000","last_unstale":"2024-09-17T04:30:01.240644+0000","last_undegraded":"2024-09-17T04:30:01.240644+0000","last_fullsized":"2024-09-17T04:30:01.240644+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:49:18.446987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.033348+0000","last_change":"2024-09-17T04:30:00.030573+0000","last_active":"2024-09-17T04:30:00.033348+0000","last_peered":"2024-09-17T04:30:00.033348+0000","last_clean":"2024-09-17T04:30:00.033348+0000","last_became_active":"2024-09-17T04:30:00.030261+0000","last_became_peered":"2024-09-17T04:30:00.030261+0000","last_unstale":"2024-09-17T04:30:00.033348+0000","last_undegraded":"2024-09-17T04:30:00.033348+0000","last_fullsized":"2024-09-17T04:30:00.033348+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:03.908735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.021003+0000","last_change":"2024-09-17T04:29:59.013448+0000","last_active":"2024-09-17T04:30:00.021003+0000","last_peered":"2024-09-17T04:30:00.021003+0000","last_clean":"2024-09-17T04:30:00.021003+0000","last_became_active":"2024-09-17T04:29:59.010565+0000","last_became_peered":"2024-09-17T04:29:59.010565+0000","last_unstale":"2024-09-17T04:30:00.021003+0000","last_undegraded":"2024-09-17T04:30:00.021003+0000","last_fullsized":"2024-09-17T04:30:00.021003+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:25:13.476952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.027384+0000","last_change":"2024-09-17T04:30:00.035250+0000","last_active":"2024-09-17T04:30:01.027384+0000","last_peered":"2024-09-17T04:30:01.027384+0000","last_clean":"2024-09-17T04:30:01.027384+0000","last_became_active":"2024-09-17T04:30:00.029488+0000","last_became_peered":"2024-09-17T04:30:00.029488+0000","last_unstale":"2024-09-17T04:30:01.027384+0000","last_undegraded":"2024-09-17T04:30:01.027384+0000","last_fullsized":"2024-09-17T04:30:01.027384+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:00.376670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"104'2","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.048889+0000","last_change":"2024-09-17T04:29:59.009226+0000","last_active":"2024-09-17T04:30:00.048889+0000","last_peered":"2024-09-17T04:30:00.048889+0000","last_clean":"2024-09-17T04:30:00.048889+0000","last_became_active":"2024-09-17T04:29:59.008934+0000","last_became_peered":"2024-09-17T04:29:59.008934+0000","last_unstale":"2024-09-17T04:30:00.048889+0000","last_undegraded":"2024-09-17T04:30:00.048889+0000","last_fullsized":"2024-09-17T04:30:00.048889+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:17:58.897435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.040218+0000","last_change":"2024-09-17T04:30:00.031413+0000","last_active":"2024-09-17T04:30:00.040218+0000","last_peered":"2024-09-17T04:30:00.040218+0000","last_clean":"2024-09-17T04:30:00.040218+0000","last_became_active":"2024-09-17T04:30:00.029749+0000","last_became_peered":"2024-09-17T04:30:00.029749+0000","last_unstale":"2024-09-17T04:30:00.040218+0000","last_undegraded":"2024-09-17T04:30:00.040218+0000","last_fullsized":"2024-09-17T04:30:00.040218+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:00.354631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"105'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.029126+0000","last_change":"2024-09-17T04:29:58.999706+0000","last_active":"2024-09-17T04:30:01.029126+0000","last_peered":"2024-09-17T04:30:01.029126+0000","last_clean":"2024-09-17T04:30:01.029126+0000","last_became_active":"2024-09-17T04:29:58.999361+0000","last_became_peered":"2024-09-17T04:29:58.999361+0000","last_unstale":"2024-09-17T04:30:01.029126+0000","last_undegraded":"2024-09-17T04:30:01.029126+0000","last_fullsized":"2024-09-17T04:30:01.029126+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:05:07.130292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.048723+0000","last_change":"2024-09-17T04:30:00.034238+0000","last_active":"2024-09-17T04:30:00.048723+0000","last_peered":"2024-09-17T04:30:00.048723+0000","last_clean":"2024-09-17T04:30:00.048723+0000","last_became_active":"2024-09-17T04:30:00.033877+0000","last_became_peered":"2024-09-17T04:30:00.033877+0000","last_unstale":"2024-09-17T04:30:00.048723+0000","last_undegraded":"2024-09-17T04:30:00.048723+0000","last_fullsized":"2024-09-17T04:30:00.048723+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:14:47.053925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.014537+0000","last_change":"2024-09-17T04:29:59.010343+0000","last_active":"2024-09-17T04:30:00.014537+0000","last_peered":"2024-09-17T04:30:00.014537+0000","last_clean":"2024-09-17T04:30:00.014537+0000","last_became_active":"2024-09-17T04:29:59.010035+0000","last_became_peered":"2024-09-17T04:29:59.010035+0000","last_unstale":"2024-09-17T04:30:00.014537+0000","last_undegraded":"2024-09-17T04:30:00.014537+0000","last_fullsized":"2024-09-17T04:30:00.014537+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:41.091741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.027459+0000","last_change":"2024-09-17T04:30:00.033009+0000","last_active":"2024-09-17T04:30:01.027459+0000","last_peered":"2024-09-17T04:30:01.027459+0000","last_clean":"2024-09-17T04:30:01.027459+0000","last_became_active":"2024-09-17T04:30:00.032624+0000","last_became_peered":"2024-09-17T04:30:00.032624+0000","last_unstale":"2024-09-17T04:30:01.027459+0000","last_undegraded":"2024-09-17T04:30:01.027459+0000","last_fullsized":"2024-09-17T04:30:01.027459+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:05:21.232149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.010019+0000","last_change":"2024-09-17T04:29:59.008457+0000","last_active":"2024-09-17T04:30:00.010019+0000","last_peered":"2024-09-17T04:30:00.010019+0000","last_clean":"2024-09-17T04:30:00.010019+0000","last_became_active":"2024-09-17T04:29:59.008080+0000","last_became_peered":"2024-09-17T04:29:59.008080+0000","last_unstale":"2024-09-17T04:30:00.010019+0000","last_undegraded":"2024-09-17T04:30:00.010019+0000","last_fullsized":"2024-09-17T04:30:00.010019+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:06:44.785805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.022592+0000","last_change":"2024-09-17T04:30:00.021360+0000","last_active":"2024-09-17T04:30:01.022592+0000","last_peered":"2024-09-17T04:30:01.022592+0000","last_clean":"2024-09-17T04:30:01.022592+0000","last_became_active":"2024-09-17T04:30:00.020152+0000","last_became_peered":"2024-09-17T04:30:00.020152+0000","last_unstale":"2024-09-17T04:30:01.022592+0000","last_undegraded":"2024-09-17T04:30:01.022592+0000","last_fullsized":"2024-09-17T04:30:01.022592+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:55.361480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.023746+0000","last_change":"2024-09-17T04:29:59.003143+0000","last_active":"2024-09-17T04:30:01.023746+0000","last_peered":"2024-09-17T04:30:01.023746+0000","last_clean":"2024-09-17T04:30:01.023746+0000","last_became_active":"2024-09-17T04:29:59.002678+0000","last_became_peered":"2024-09-17T04:29:59.002678+0000","last_unstale":"2024-09-17T04:30:01.023746+0000","last_undegraded":"2024-09-17T04:30:01.023746+0000","last_fullsized":"2024-09-17T04:30:01.023746+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:10.160946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.023398+0000","last_change":"2024-09-17T04:30:00.017223+0000","last_active":"2024-09-17T04:30:01.023398+0000","last_peered":"2024-09-17T04:30:01.023398+0000","last_clean":"2024-09-17T04:30:01.023398+0000","last_became_active":"2024-09-17T04:30:00.016903+0000","last_became_peered":"2024-09-17T04:30:00.016903+0000","last_unstale":"2024-09-17T04:30:01.023398+0000","last_undegraded":"2024-09-17T04:30:01.023398+0000","last_fullsized":"2024-09-17T04:30:01.023398+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:42.100621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.022452+0000","last_change":"2024-09-17T04:29:59.011982+0000","last_active":"2024-09-17T04:30:01.022452+0000","last_peered":"2024-09-17T04:30:01.022452+0000","last_clean":"2024-09-17T04:30:01.022452+0000","last_became_active":"2024-09-17T04:29:59.011509+0000","last_became_peered":"2024-09-17T04:29:59.011509+0000","last_unstale":"2024-09-17T04:30:01.022452+0000","last_undegraded":"2024-09-17T04:30:01.022452+0000","last_fullsized":"2024-09-17T04:30:01.022452+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:24.291710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.039698+0000","last_change":"2024-09-17T04:30:00.032939+0000","last_active":"2024-09-17T04:30:00.039698+0000","last_peered":"2024-09-17T04:30:00.039698+0000","last_clean":"2024-09-17T04:30:00.039698+0000","last_became_active":"2024-09-17T04:30:00.032065+0000","last_became_peered":"2024-09-17T04:30:00.032065+0000","last_unstale":"2024-09-17T04:30:00.039698+0000","last_undegraded":"2024-09-17T04:30:00.039698+0000","last_fullsized":"2024-09-17T04:30:00.039698+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:38.961501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.020874+0000","last_change":"2024-09-17T04:29:59.016863+0000","last_active":"2024-09-17T04:30:01.020874+0000","last_peered":"2024-09-17T04:30:01.020874+0000","last_clean":"2024-09-17T04:30:01.020874+0000","last_became_active":"2024-09-17T04:29:59.016582+0000","last_became_peered":"2024-09-17T04:29:59.016582+0000","last_unstale":"2024-09-17T04:30:01.020874+0000","last_undegraded":"2024-09-17T04:30:01.020874+0000","last_fullsized":"2024-09-17T04:30:01.020874+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:47.279038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.047891+0000","last_change":"2024-09-17T04:30:00.041495+0000","last_active":"2024-09-17T04:30:00.047891+0000","last_peered":"2024-09-17T04:30:00.047891+0000","last_clean":"2024-09-17T04:30:00.047891+0000","last_became_active":"2024-09-17T04:30:00.040827+0000","last_became_peered":"2024-09-17T04:30:00.040827+0000","last_unstale":"2024-09-17T04:30:00.047891+0000","last_undegraded":"2024-09-17T04:30:00.047891+0000","last_fullsized":"2024-09-17T04:30:00.047891+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:33:52.257861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.017479+0000","last_change":"2024-09-17T04:29:59.014718+0000","last_active":"2024-09-17T04:30:00.017479+0000","last_peered":"2024-09-17T04:30:00.017479+0000","last_clean":"2024-09-17T04:30:00.017479+0000","last_became_active":"2024-09-17T04:29:59.010601+0000","last_became_peered":"2024-09-17T04:29:59.010601+0000","last_unstale":"2024-09-17T04:30:00.017479+0000","last_undegraded":"2024-09-17T04:30:00.017479+0000","last_fullsized":"2024-09-17T04:30:00.017479+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:17.750773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.027209+0000","last_change":"2024-09-17T04:30:00.020727+0000","last_active":"2024-09-17T04:30:00.027209+0000","last_peered":"2024-09-17T04:30:00.027209+0000","last_clean":"2024-09-17T04:30:00.027209+0000","last_became_active":"2024-09-17T04:30:00.020436+0000","last_became_peered":"2024-09-17T04:30:00.020436+0000","last_unstale":"2024-09-17T04:30:00.027209+0000","last_undegraded":"2024-09-17T04:30:00.027209+0000","last_fullsized":"2024-09-17T04:30:00.027209+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:58.980352+0000","last_clean_scrub_stamp":"2024-09-17T04:29:58.980352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:36.288918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-17T04:30:01.023700+0000","last_change":"2024-09-17T04:29:59.007405+0000","last_active":"2024-09-17T04:30:01.023700+0000","last_peered":"2024-09-17T04:30:01.023700+0000","last_clean":"2024-09-17T04:30:01.023700+0000","last_became_active":"2024-09-17T04:29:59.006905+0000","last_became_peered":"2024-09-17T04:29:59.006905+0000","last_unstale":"2024-09-17T04:30:01.023700+0000","last_undegraded":"2024-09-17T04:30:01.023700+0000","last_fullsized":"2024-09-17T04:30:01.023700+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:29:57.971472+0000","last_clean_scrub_stamp":"2024-09-17T04:29:57.971472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:55:24.434433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"76'79","reported_seq":289,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T04:30:00.018945+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:30:00.018945+0000","last_peered":"2024-09-17T04:30:00.018945+0000","last_clean":"2024-09-17T04:30:00.018945+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:30:00.018945+0000","last_undegraded":"2024-09-17T04:30:00.018945+0000","last_fullsized":"2024-09-17T04:30:00.018945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476813,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1884160,"data_stored":1150265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476812,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32460,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339380,"statfs":{"total":96636764160,"available":96603525120,"internally_reserved":0,"allocated":1298432,"data_stored":565651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476813,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1290240,"data_stored":559885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476813,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1282048,"data_stored":559281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542220,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32460,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339380,"statfs":{"total":96636764160,"available":96603525120,"internally_reserved":0,"allocated":1298432,"data_stored":560963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509516,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32460,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339380,"statfs":{"total":96636764160,"available":96603525120,"internally_reserved":0,"allocated":1294336,"data_stored":565581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509515,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1302528,"data_stored":565705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509515,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1896448,"data_stored":1151373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:30:04.914 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T04:30:04.915 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T04:30:05.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.180+0000 7f7f38e00640 1 -- 172.21.15.100:0/3151633533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f34145450 msgr2=0x7f7f3406f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:05.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.180+0000 7f7f38e00640 1 --2- 172.21.15.100:0/3151633533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f34145450 0x7f7f3406f180 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f7f3415cd30 tx=0x7f7f2402a540 comp rx=0 tx=0).stop 2024-09-17T04:30:05.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.180+0000 7f7f38e00640 1 -- 172.21.15.100:0/3151633533 shutdown_connections 2024-09-17T04:30:05.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.180+0000 7f7f38e00640 1 --2- 172.21.15.100:0/3151633533 >> v2:172.21.15.184:3300/0 conn(0x7f7f340cd610 0x7f7f340cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:05.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.180+0000 7f7f38e00640 1 --2- 172.21.15.100:0/3151633533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f34145450 0x7f7f3406f180 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:05.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.180+0000 7f7f38e00640 1 -- 172.21.15.100:0/3151633533 >> 172.21.15.100:0/3151633533 conn(0x7f7f3406d1c0 msgr2=0x7f7f3406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:05.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.180+0000 7f7f38e00640 1 -- 172.21.15.100:0/3151633533 shutdown_connections 2024-09-17T04:30:05.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.181+0000 7f7f38e00640 1 -- 172.21.15.100:0/3151633533 wait complete. 2024-09-17T04:30:05.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.183+0000 7f7f38e00640 1 Processor -- start 2024-09-17T04:30:05.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.183+0000 7f7f38e00640 1 -- start start 2024-09-17T04:30:05.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.184+0000 7f7f38e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f340735e0 0x7f7f340ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:05.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.184+0000 7f7f38e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f340cd610 0x7f7f341f8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:05.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.184+0000 7f7f38e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f34145450 0x7f7f341f9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:05.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.184+0000 7f7f38e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7f340ce2f0 con 0x7f7f340cd610 2024-09-17T04:30:05.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.184+0000 7f7f38e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7f340ce170 con 0x7f7f340735e0 2024-09-17T04:30:05.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.184+0000 7f7f38e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7f340ce470 con 0x7f7f34145450 2024-09-17T04:30:05.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.184+0000 7f7f31400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f340cd610 0x7f7f341f8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:05.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.185+0000 7f7f31400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f340cd610 0x7f7f341f8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57246/0 (socket says 172.21.15.100:57246) 2024-09-17T04:30:05.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.185+0000 7f7f31400640 1 -- 172.21.15.100:0/1828636123 learned_addr learned my addr 172.21.15.100:0/1828636123 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:05.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.185+0000 7f7f31e00640 1 --2- 172.21.15.100:0/1828636123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f340735e0 0x7f7f340ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:05.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.185+0000 7f7f32800640 1 --2- 172.21.15.100:0/1828636123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f34145450 0x7f7f341f9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:05.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.185+0000 7f7f31400640 1 -- 172.21.15.100:0/1828636123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f34145450 msgr2=0x7f7f341f9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:05.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.185+0000 7f7f31400640 1 --2- 172.21.15.100:0/1828636123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f34145450 0x7f7f341f9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:05.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.185+0000 7f7f31400640 1 -- 172.21.15.100:0/1828636123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f340735e0 msgr2=0x7f7f340ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:05.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.185+0000 7f7f31400640 1 --2- 172.21.15.100:0/1828636123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f340735e0 0x7f7f340ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:05.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.185+0000 7f7f31400640 1 -- 172.21.15.100:0/1828636123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f340cad50 con 0x7f7f340cd610 2024-09-17T04:30:05.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.185+0000 7f7f31400640 1 --2- 172.21.15.100:0/1828636123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f340cd610 0x7f7f341f8fe0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000c370 tx=0x7f7f2000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:05.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.186+0000 7f7f2a000640 1 -- 172.21.15.100:0/1828636123 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20011790 con 0x7f7f340cd610 2024-09-17T04:30:05.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.186+0000 7f7f2a000640 1 -- 172.21.15.100:0/1828636123 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f20017070 con 0x7f7f340cd610 2024-09-17T04:30:05.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.186+0000 7f7f2a000640 1 -- 172.21.15.100:0/1828636123 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20011c80 con 0x7f7f340cd610 2024-09-17T04:30:05.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.186+0000 7f7f38e00640 1 -- 172.21.15.100:0/1828636123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f340cb030 con 0x7f7f340cd610 2024-09-17T04:30:05.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.186+0000 7f7f38e00640 1 -- 172.21.15.100:0/1828636123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f340cb5f0 con 0x7f7f340cd610 2024-09-17T04:30:05.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.187+0000 7f7f38e00640 1 -- 172.21.15.100:0/1828636123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f340cda20 con 0x7f7f340cd610 2024-09-17T04:30:05.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.188+0000 7f7f2a000640 1 -- 172.21.15.100:0/1828636123 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f7f2001c460 con 0x7f7f340cd610 2024-09-17T04:30:05.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.189+0000 7f7f2a000640 1 --2- 172.21.15.100:0/1828636123 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7f14078cb0 0x7f7f1407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:05.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.189+0000 7f7f2a000640 1 -- 172.21.15.100:0/1828636123 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f7f2009ac10 con 0x7f7f340cd610 2024-09-17T04:30:05.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.189+0000 7f7f31e00640 1 --2- 172.21.15.100:0/1828636123 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7f14078cb0 0x7f7f1407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:05.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.190+0000 7f7f31e00640 1 --2- 172.21.15.100:0/1828636123 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7f14078cb0 0x7f7f1407b1c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7f24034000 tx=0x7f7f24004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:05.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.193+0000 7f7f2a000640 1 -- 172.21.15.100:0/1828636123 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f7f20062860 con 0x7f7f340cd610 2024-09-17T04:30:05.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.447+0000 7f7f38e00640 1 -- 172.21.15.100:0/1828636123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f7f34142370 con 0x7f7f340cd610 2024-09-17T04:30:05.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.450+0000 7f7f2a000640 1 -- 172.21.15.100:0/1828636123 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7f7f20067290 con 0x7f7f340cd610 2024-09-17T04:30:05.451 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-09-17T04:30:05.451 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAsBOlmfYg9IBAABP1uRKGOugrA47VxwRLkjA== 2024-09-17T04:30:05.451 INFO:teuthology.orchestra.run.smithi100.stdout: caps mds = "allow" 2024-09-17T04:30:05.451 INFO:teuthology.orchestra.run.smithi100.stdout: caps mon = "allow r" 2024-09-17T04:30:05.451 INFO:teuthology.orchestra.run.smithi100.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T04:30:05.451 INFO:teuthology.orchestra.run.smithi100.stderr:updated caps for client.0 2024-09-17T04:30:05.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.453+0000 7f7f38e00640 1 -- 172.21.15.100:0/1828636123 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7f14078cb0 msgr2=0x7f7f1407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:05.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.453+0000 7f7f38e00640 1 --2- 172.21.15.100:0/1828636123 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7f14078cb0 0x7f7f1407b1c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7f24034000 tx=0x7f7f24004730 comp rx=0 tx=0).stop 2024-09-17T04:30:05.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.453+0000 7f7f38e00640 1 -- 172.21.15.100:0/1828636123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f340cd610 msgr2=0x7f7f341f8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:05.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.453+0000 7f7f38e00640 1 --2- 172.21.15.100:0/1828636123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f340cd610 0x7f7f341f8fe0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000c370 tx=0x7f7f2000c830 comp rx=0 tx=0).stop 2024-09-17T04:30:05.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.453+0000 7f7f38e00640 1 -- 172.21.15.100:0/1828636123 shutdown_connections 2024-09-17T04:30:05.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.453+0000 7f7f38e00640 1 --2- 172.21.15.100:0/1828636123 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7f14078cb0 0x7f7f1407b1c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:05.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.453+0000 7f7f38e00640 1 --2- 172.21.15.100:0/1828636123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f34145450 0x7f7f341f9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:05.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.453+0000 7f7f38e00640 1 --2- 172.21.15.100:0/1828636123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f340cd610 0x7f7f341f8fe0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:05.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.453+0000 7f7f38e00640 1 --2- 172.21.15.100:0/1828636123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f340735e0 0x7f7f340ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:05.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.453+0000 7f7f38e00640 1 -- 172.21.15.100:0/1828636123 >> 172.21.15.100:0/1828636123 conn(0x7f7f3406d1c0 msgr2=0x7f7f34073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:05.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.453+0000 7f7f38e00640 1 -- 172.21.15.100:0/1828636123 shutdown_connections 2024-09-17T04:30:05.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.453+0000 7f7f38e00640 1 -- 172.21.15.100:0/1828636123 wait complete. 2024-09-17T04:30:05.475 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:30:05.670 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:30:05.674 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:30:05.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.695+0000 7fc157600640 1 -- 172.21.15.100:0/258698448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1500d03b0 msgr2=0x7fc1500d07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:05.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.695+0000 7fc157600640 1 --2- 172.21.15.100:0/258698448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1500d03b0 0x7fc1500d07c0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fc144009820 tx=0x7fc14402a5a0 comp rx=0 tx=0).stop 2024-09-17T04:30:05.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.696+0000 7fc157600640 1 -- 172.21.15.100:0/258698448 shutdown_connections 2024-09-17T04:30:05.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.696+0000 7fc157600640 1 --2- 172.21.15.100:0/258698448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1500d03b0 0x7fc1500d07c0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:05.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.696+0000 7fc157600640 1 -- 172.21.15.100:0/258698448 >> 172.21.15.100:0/258698448 conn(0x7fc1500cef30 msgr2=0x7fc1500cf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:05.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.696+0000 7fc157600640 1 -- 172.21.15.100:0/258698448 shutdown_connections 2024-09-17T04:30:05.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.696+0000 7fc157600640 1 -- 172.21.15.100:0/258698448 wait complete. 2024-09-17T04:30:05.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.698+0000 7fc157600640 1 Processor -- start 2024-09-17T04:30:05.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.698+0000 7fc157600640 1 -- start start 2024-09-17T04:30:05.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.699+0000 7fc157600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1500d03b0 0x7fc150159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:05.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.699+0000 7fc157600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1500d0dc0 0x7fc1500b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:05.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.700+0000 7fc157600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1500d11d0 0x7fc1500b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:05.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.700+0000 7fc157600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc1500b78e0 con 0x7fc1500d03b0 2024-09-17T04:30:05.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.700+0000 7fc157600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc1500b1a20 con 0x7fc1500d0dc0 2024-09-17T04:30:05.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.700+0000 7fc157600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc1500b7a60 con 0x7fc1500d11d0 2024-09-17T04:30:05.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.700+0000 7fc154c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1500d03b0 0x7fc150159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:05.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.700+0000 7fc154c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1500d03b0 0x7fc150159270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57260/0 (socket says 172.21.15.100:57260) 2024-09-17T04:30:05.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.700+0000 7fc154c00640 1 -- 172.21.15.100:0/855216550 learned_addr learned my addr 172.21.15.100:0/855216550 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:05.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.700+0000 7fc155600640 1 --2- 172.21.15.100:0/855216550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1500d11d0 0x7fc1500b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:05.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.700+0000 7fc14fe00640 1 --2- 172.21.15.100:0/855216550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1500d0dc0 0x7fc1500b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:05.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.700+0000 7fc154c00640 1 -- 172.21.15.100:0/855216550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1500d11d0 msgr2=0x7fc1500b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:05.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.700+0000 7fc154c00640 1 --2- 172.21.15.100:0/855216550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1500d11d0 0x7fc1500b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:05.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.700+0000 7fc154c00640 1 -- 172.21.15.100:0/855216550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1500d0dc0 msgr2=0x7fc1500b02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:05.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.700+0000 7fc154c00640 1 --2- 172.21.15.100:0/855216550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1500d0dc0 0x7fc1500b02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:05.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.700+0000 7fc154c00640 1 -- 172.21.15.100:0/855216550 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1501f6cb0 con 0x7fc1500d03b0 2024-09-17T04:30:05.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.701+0000 7fc154c00640 1 --2- 172.21.15.100:0/855216550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1500d03b0 0x7fc150159270 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fc144000c00 tx=0x7fc144005360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:05.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.701+0000 7fc14cc00640 1 -- 172.21.15.100:0/855216550 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14403f070 con 0x7fc1500d03b0 2024-09-17T04:30:05.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.701+0000 7fc14cc00640 1 -- 172.21.15.100:0/855216550 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc144004340 con 0x7fc1500d03b0 2024-09-17T04:30:05.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.701+0000 7fc14cc00640 1 -- 172.21.15.100:0/855216550 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc144031bb0 con 0x7fc1500d03b0 2024-09-17T04:30:05.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.701+0000 7fc157600640 1 -- 172.21.15.100:0/855216550 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1501f9510 con 0x7fc1500d03b0 2024-09-17T04:30:05.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.701+0000 7fc157600640 1 -- 172.21.15.100:0/855216550 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1501f9ad0 con 0x7fc1500d03b0 2024-09-17T04:30:05.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.702+0000 7fc157600640 1 -- 172.21.15.100:0/855216550 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1500d07c0 con 0x7fc1500d03b0 2024-09-17T04:30:05.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.704+0000 7fc14cc00640 1 -- 172.21.15.100:0/855216550 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fc144042070 con 0x7fc1500d03b0 2024-09-17T04:30:05.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.704+0000 7fc14cc00640 1 --2- 172.21.15.100:0/855216550 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc130078cb0 0x7fc13007b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:05.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.705+0000 7fc14cc00640 1 -- 172.21.15.100:0/855216550 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7fc1440bbff0 con 0x7fc1500d03b0 2024-09-17T04:30:05.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.705+0000 7fc14fe00640 1 --2- 172.21.15.100:0/855216550 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc130078cb0 0x7fc13007b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:05.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.706+0000 7fc14fe00640 1 --2- 172.21.15.100:0/855216550 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc130078cb0 0x7fc13007b1c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc13c000f60 tx=0x7fc13c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:05.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.709+0000 7fc14cc00640 1 -- 172.21.15.100:0/855216550 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc144083bd0 con 0x7fc1500d03b0 2024-09-17T04:30:05.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.931+0000 7fc157600640 1 -- 172.21.15.100:0/855216550 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1500d17b0 con 0x7fc1500d03b0 2024-09-17T04:30:05.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.932+0000 7fc14cc00640 1 -- 172.21.15.100:0/855216550 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5234 (secure 0 0 0) 0x7fc144088600 con 0x7fc1500d03b0 2024-09-17T04:30:05.934 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:30:05.934 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":72,"btime":"2024-09-17T04:30:03:024749+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6044,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1891177834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1891177834},{"type":"v1","addr":"172.21.15.100:6835","nonce":1891177834}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":6047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1158083377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1158083377},{"type":"v1","addr":"172.21.15.184:6835","nonce":1158083377}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":6697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/714465116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":714465116},{"type":"v1","addr":"172.21.15.184:6801","nonce":714465116}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:29:59.487577+0000","modified":"2024-09-17T04:30:02.508652+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6972":{"gid":6972,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6833/2666467995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2666467995},{"type":"v1","addr":"172.21.15.100:6833","nonce":2666467995}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6972,"qdb_cluster":[6972]},"id":8}]} 2024-09-17T04:30:05.936 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 72 2024-09-17T04:30:05.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.934+0000 7fc157600640 1 -- 172.21.15.100:0/855216550 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc130078cb0 msgr2=0x7fc13007b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:05.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.934+0000 7fc157600640 1 --2- 172.21.15.100:0/855216550 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc130078cb0 0x7fc13007b1c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc13c000f60 tx=0x7fc13c009040 comp rx=0 tx=0).stop 2024-09-17T04:30:05.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.934+0000 7fc157600640 1 -- 172.21.15.100:0/855216550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1500d03b0 msgr2=0x7fc150159270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:05.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.934+0000 7fc157600640 1 --2- 172.21.15.100:0/855216550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1500d03b0 0x7fc150159270 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fc144000c00 tx=0x7fc144005360 comp rx=0 tx=0).stop 2024-09-17T04:30:05.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.934+0000 7fc157600640 1 -- 172.21.15.100:0/855216550 shutdown_connections 2024-09-17T04:30:05.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.934+0000 7fc157600640 1 --2- 172.21.15.100:0/855216550 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc130078cb0 0x7fc13007b1c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:05.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.934+0000 7fc157600640 1 --2- 172.21.15.100:0/855216550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1500d11d0 0x7fc1500b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:05.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.934+0000 7fc157600640 1 --2- 172.21.15.100:0/855216550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1500d0dc0 0x7fc1500b02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:05.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.934+0000 7fc157600640 1 --2- 172.21.15.100:0/855216550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1500d03b0 0x7fc150159270 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:05.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.934+0000 7fc157600640 1 -- 172.21.15.100:0/855216550 >> 172.21.15.100:0/855216550 conn(0x7fc1500cef30 msgr2=0x7fc150155e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:05.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.934+0000 7fc157600640 1 -- 172.21.15.100:0/855216550 shutdown_connections 2024-09-17T04:30:05.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:05.935+0000 7fc157600640 1 -- 172.21.15.100:0/855216550 wait complete. 2024-09-17T04:30:05.973 INFO:tasks.ceph.mds.a.smithi100.stderr:2024-09-17T04:30:05.973+0000 7f6371600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T04:30:06.366 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T04:29:59.487577+0000', 'modified': '2024-09-17T04:30:02.508652+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6972}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6972': {'gid': 6972, 'name': 'a', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6833/2666467995', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6832', 'nonce': 2666467995}, {'type': 'v1', 'addr': '172.21.15.100:6833', 'nonce': 2666467995}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6972, 'qdb_cluster': [6972]} 2024-09-17T04:30:06.366 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T04:30:06.366 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-17T04:30:06.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.594+0000 7f14e9400640 1 -- 172.21.15.100:0/244737371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14e40705b0 msgr2=0x7f14e40709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:06.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.594+0000 7f14e9400640 1 --2- 172.21.15.100:0/244737371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14e40705b0 0x7f14e40709c0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f14d0005fd0 tx=0x7f14d002a4f0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.594+0000 7f14e9400640 1 -- 172.21.15.100:0/244737371 shutdown_connections 2024-09-17T04:30:06.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.594+0000 7f14e9400640 1 --2- 172.21.15.100:0/244737371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14e40705b0 0x7f14e40709c0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.594+0000 7f14e9400640 1 --2- 172.21.15.100:0/244737371 >> v2:172.21.15.184:3300/0 conn(0x7f14e40cfc80 0x7f14e4156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.594+0000 7f14e9400640 1 -- 172.21.15.100:0/244737371 >> 172.21.15.100:0/244737371 conn(0x7f14e406d3d0 msgr2=0x7f14e40b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:06.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.594+0000 7f14e9400640 1 -- 172.21.15.100:0/244737371 shutdown_connections 2024-09-17T04:30:06.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.594+0000 7f14e9400640 1 -- 172.21.15.100:0/244737371 wait complete. 2024-09-17T04:30:06.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.597+0000 7f14e9400640 1 Processor -- start 2024-09-17T04:30:06.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.597+0000 7f14e9400640 1 -- start start 2024-09-17T04:30:06.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.598+0000 7f14e9400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14e40705b0 0x7f14e40c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:06.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.598+0000 7f14e9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14e40cfc80 0x7f14e40c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:06.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.598+0000 7f14e9400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14e415ce30 0x7f14e40c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:06.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.598+0000 7f14e9400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f14e40cced0 con 0x7f14e40cfc80 2024-09-17T04:30:06.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.598+0000 7f14e9400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f14e40d0170 con 0x7f14e40705b0 2024-09-17T04:30:06.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.598+0000 7f14e9400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f14e40cd050 con 0x7f14e415ce30 2024-09-17T04:30:06.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.598+0000 7f14e1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14e40cfc80 0x7f14e40c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:06.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.598+0000 7f14e1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14e40cfc80 0x7f14e40c9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57282/0 (socket says 172.21.15.100:57282) 2024-09-17T04:30:06.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.598+0000 7f14e1e00640 1 -- 172.21.15.100:0/3294215114 learned_addr learned my addr 172.21.15.100:0/3294215114 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:06.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.599+0000 7f14e2800640 1 --2- 172.21.15.100:0/3294215114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14e40705b0 0x7f14e40c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:06.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.599+0000 7f14e3200640 1 --2- 172.21.15.100:0/3294215114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14e415ce30 0x7f14e40c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:06.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.599+0000 7f14e1e00640 1 -- 172.21.15.100:0/3294215114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14e415ce30 msgr2=0x7f14e40c73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:06.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.599+0000 7f14e1e00640 1 --2- 172.21.15.100:0/3294215114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14e415ce30 0x7f14e40c73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.599+0000 7f14e1e00640 1 -- 172.21.15.100:0/3294215114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14e40705b0 msgr2=0x7f14e40c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:06.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.599+0000 7f14e1e00640 1 --2- 172.21.15.100:0/3294215114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14e40705b0 0x7f14e40c8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.599+0000 7f14e1e00640 1 -- 172.21.15.100:0/3294215114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e4155530 con 0x7f14e40cfc80 2024-09-17T04:30:06.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.599+0000 7f14e2800640 1 --2- 172.21.15.100:0/3294215114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14e40705b0 0x7f14e40c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:30:06.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.599+0000 7f14e1e00640 1 --2- 172.21.15.100:0/3294215114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14e40cfc80 0x7f14e40c9220 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f14d0006010 tx=0x7f14d00039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:06.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.600+0000 7f14daa00640 1 -- 172.21.15.100:0/3294215114 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d003e030 con 0x7f14e40cfc80 2024-09-17T04:30:06.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.600+0000 7f14daa00640 1 -- 172.21.15.100:0/3294215114 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14d00047a0 con 0x7f14e40cfc80 2024-09-17T04:30:06.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.600+0000 7f14daa00640 1 -- 172.21.15.100:0/3294215114 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d0035040 con 0x7f14e40cfc80 2024-09-17T04:30:06.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.600+0000 7f14e9400640 1 -- 172.21.15.100:0/3294215114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e4155810 con 0x7f14e40cfc80 2024-09-17T04:30:06.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.600+0000 7f14e9400640 1 -- 172.21.15.100:0/3294215114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e40ad640 con 0x7f14e40cfc80 2024-09-17T04:30:06.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.603+0000 7f14daa00640 1 -- 172.21.15.100:0/3294215114 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f14d0041030 con 0x7f14e40cfc80 2024-09-17T04:30:06.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.604+0000 7f14daa00640 1 --2- 172.21.15.100:0/3294215114 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f14c4078cb0 0x7f14c407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:06.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.605+0000 7f14daa00640 1 -- 172.21.15.100:0/3294215114 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5767+0+0 (secure 0 0 0) 0x7f14d00bb9c0 con 0x7f14e40cfc80 2024-09-17T04:30:06.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.605+0000 7f14e2800640 1 --2- 172.21.15.100:0/3294215114 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f14c4078cb0 0x7f14c407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:06.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.605+0000 7f14f0472740 1 Processor -- start 2024-09-17T04:30:06.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.605+0000 7f14f0472740 1 -- start start 2024-09-17T04:30:06.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.605+0000 7f14e2800640 1 --2- 172.21.15.100:0/3294215114 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f14c4078cb0 0x7f14c407b1c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f14e409e510 tx=0x7f14d400a540 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:06.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.607+0000 7f14f0472740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f07385270 0x557f07770a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:06.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.607+0000 7f14f0472740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557f0735c3b0 0x557f07725f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:06.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.607+0000 7f14f0472740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f07364590 0x557f077264a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:06.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.607+0000 7f14f0472740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x557f076af430 con 0x557f07364590 2024-09-17T04:30:06.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.607+0000 7f14f0472740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557f0769f5d0 con 0x557f07385270 2024-09-17T04:30:06.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.607+0000 7f14f0472740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x557f0768ea00 con 0x557f0735c3b0 2024-09-17T04:30:06.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.607+0000 7f14e3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f07364590 0x557f077264a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:06.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.607+0000 7f14e3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f07364590 0x557f077264a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57298/0 (socket says 172.21.15.100:57298) 2024-09-17T04:30:06.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.607+0000 7f14e3200640 1 -- 172.21.15.100:0/2868978668 learned_addr learned my addr 172.21.15.100:0/2868978668 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:06.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.607+0000 7f14e1e00640 1 --2- 172.21.15.100:0/2868978668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557f0735c3b0 0x557f07725f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:06.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.607+0000 7f14e2800640 1 --2- 172.21.15.100:0/2868978668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f07385270 0x557f07770a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:06.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.607+0000 7f14e3200640 1 -- 172.21.15.100:0/2868978668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557f0735c3b0 msgr2=0x557f07725f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:30:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.607+0000 7f14e3200640 1 --2- 172.21.15.100:0/2868978668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557f0735c3b0 0x557f07725f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.607+0000 7f14e3200640 1 -- 172.21.15.100:0/2868978668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f07385270 msgr2=0x557f07770a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:30:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.607+0000 7f14e3200640 1 --2- 172.21.15.100:0/2868978668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f07385270 0x557f07770a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.607+0000 7f14e3200640 1 -- 172.21.15.100:0/2868978668 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f07657520 con 0x557f07364590 2024-09-17T04:30:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.607+0000 7f14e2800640 1 --2- 172.21.15.100:0/2868978668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f07385270 0x557f07770a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:30:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.608+0000 7f14e3200640 1 --2- 172.21.15.100:0/2868978668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f07364590 0x557f077264a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009f30 tx=0x7f14dc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d9226c1b62e2ce server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.608+0000 7f14cf000640 1 -- 172.21.15.100:0/2868978668 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc01a070 con 0x557f07364590 2024-09-17T04:30:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.608+0000 7f14cf000640 1 -- 172.21.15.100:0/2868978668 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14dc015410 con 0x557f07364590 2024-09-17T04:30:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.608+0000 7f14cf000640 1 -- 172.21.15.100:0/2868978668 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc015860 con 0x557f07364590 2024-09-17T04:30:06.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.609+0000 7f14f0472740 1 -- 172.21.15.100:0/2868978668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f07364590 msgr2=0x557f077264a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:06.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.609+0000 7f14f0472740 1 --2- 172.21.15.100:0/2868978668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f07364590 0x557f077264a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009f30 tx=0x7f14dc00c8b0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.609+0000 7f14f0472740 1 -- 172.21.15.100:0/2868978668 shutdown_connections 2024-09-17T04:30:06.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.609+0000 7f14f0472740 1 --2- 172.21.15.100:0/2868978668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f07364590 0x557f077264a0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.609+0000 7f14f0472740 1 --2- 172.21.15.100:0/2868978668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557f0735c3b0 0x557f07725f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.609+0000 7f14f0472740 1 --2- 172.21.15.100:0/2868978668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f07385270 0x557f07770a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.609+0000 7f14f0472740 1 -- 172.21.15.100:0/2868978668 >> 172.21.15.100:0/2868978668 conn(0x557f073793c0 msgr2=0x557f0772b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:06.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.609+0000 7f14f0472740 1 -- 172.21.15.100:0/2868978668 shutdown_connections 2024-09-17T04:30:06.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.609+0000 7f14f0472740 1 -- 172.21.15.100:0/2868978668 wait complete. 2024-09-17T04:30:06.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.610+0000 7f14f0472740 1 -- start start 2024-09-17T04:30:06.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.610+0000 7f14f0472740 1 Processor -- start 2024-09-17T04:30:06.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.612+0000 7f14f0472740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557f0735c3b0 0x557f0773fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:06.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.612+0000 7f14f0472740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f07364590 0x557f077401d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:06.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.612+0000 7f14f0472740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f07385270 0x557f07754a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:06.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.612+0000 7f14f0472740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x557f07635330 con 0x557f07364590 2024-09-17T04:30:06.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.612+0000 7f14f0472740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557f07692810 con 0x557f07385270 2024-09-17T04:30:06.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.612+0000 7f14f0472740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x557f073ff620 con 0x557f0735c3b0 2024-09-17T04:30:06.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.612+0000 7f14f0472740 10 client.0 _ll_register_callbacks cb 0x557f0763a420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:30:06.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.612+0000 7f14e1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f07364590 0x557f077401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:06.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.612+0000 7f14e1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f07364590 0x557f077401d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57314/0 (socket says 172.21.15.100:57314) 2024-09-17T04:30:06.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.612+0000 7f14e1e00640 1 -- 172.21.15.100:0/3158384586 learned_addr learned my addr 172.21.15.100:0/3158384586 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:06.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.612+0000 7f14e2800640 1 --2- 172.21.15.100:0/3158384586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557f0735c3b0 0x557f0773fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:06.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.612+0000 7f14e3200640 1 --2- 172.21.15.100:0/3158384586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f07385270 0x557f07754a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:06.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.612+0000 7f14e1e00640 1 -- 172.21.15.100:0/3158384586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557f0735c3b0 msgr2=0x557f0773fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:30:06.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.612+0000 7f14e1e00640 1 --2- 172.21.15.100:0/3158384586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557f0735c3b0 0x557f0773fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.612+0000 7f14e1e00640 1 -- 172.21.15.100:0/3158384586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f07385270 msgr2=0x557f07754a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:30:06.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.612+0000 7f14e1e00640 1 --2- 172.21.15.100:0/3158384586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f07385270 0x557f07754a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.612+0000 7f14e1e00640 1 -- 172.21.15.100:0/3158384586 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f073f0850 con 0x557f07364590 2024-09-17T04:30:06.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.612+0000 7f14e3200640 1 --2- 172.21.15.100:0/3158384586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f07385270 0x557f07754a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:30:06.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.612+0000 7f14e1e00640 1 --2- 172.21.15.100:0/3158384586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f07364590 0x557f077401d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f14d002aae0 tx=0x7f14d00ba0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c013040b4237ab server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:06.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.612+0000 7f14c3400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-17T04:30:06.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.613+0000 7f14c3400640 1 -- 172.21.15.100:0/3158384586 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d0048050 con 0x557f07364590 2024-09-17T04:30:06.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.613+0000 7f14c3400640 1 -- 172.21.15.100:0/3158384586 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 5767+0+0 (secure 0 0 0) 0x7f14d0082660 con 0x557f07364590 2024-09-17T04:30:06.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.613+0000 7f14f0472740 1 -- 172.21.15.100:0/3158384586 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14dc015410 con 0x557f07364590 2024-09-17T04:30:06.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.613+0000 7f14c3400640 1 -- 172.21.15.100:0/3158384586 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f14d0082f10 con 0x557f07364590 2024-09-17T04:30:06.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.613+0000 7f14f0472740 10 client.7107 fetch_fsmap learned FSMap version 72 2024-09-17T04:30:06.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.613+0000 7f14f0472740 1 -- 172.21.15.100:0/3158384586 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x557f074676b0 con 0x557f07364590 2024-09-17T04:30:06.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.613+0000 7f14c3400640 1 -- 172.21.15.100:0/3158384586 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14dc015410 con 0x557f07364590 2024-09-17T04:30:06.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.613+0000 7f14c3400640 1 -- 172.21.15.100:0/3158384586 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 72) ==== 3502+0+0 (secure 0 0 0) 0x7f14d007ed50 con 0x557f07364590 2024-09-17T04:30:06.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.613+0000 7f14f0472740 10 client.7107 fetch_fsmap finished waiting for FSMap version 72 2024-09-17T04:30:06.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.613+0000 7f14f0472740 10 client.7107 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T04:30:06.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.613+0000 7f14f0472740 20 client.7107 populate_metadata read hostname 'smithi100' 2024-09-17T04:30:06.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.613+0000 7f14f0472740 1 --2- 172.21.15.100:0/3158384586 >> [v2:172.21.15.100:6832/2666467995,v1:172.21.15.100:6833/2666467995] conn(0x557f07343e40 0x557f07784a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:06.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.613+0000 7f14f0472740 4 client.7107 mds_command: new command op to 6972 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T04:30:06.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.613+0000 7f14f0472740 1 -- 172.21.15.100:0/3158384586 --> [v2:172.21.15.100:6832/2666467995,v1:172.21.15.100:6833/2666467995] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f076a1520 con 0x557f07343e40 2024-09-17T04:30:06.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.614+0000 7f14e2800640 1 --2- 172.21.15.100:0/3158384586 >> [v2:172.21.15.100:6832/2666467995,v1:172.21.15.100:6833/2666467995] conn(0x557f07343e40 0x557f07784a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:06.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.614+0000 7f14e2800640 1 --2- 172.21.15.100:0/3158384586 >> [v2:172.21.15.100:6832/2666467995,v1:172.21.15.100:6833/2666467995] conn(0x557f07343e40 0x557f07784a30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31172116069e8803 server_cookie=500d80090c6cf650 in_seq=0 out_seq=0 2024-09-17T04:30:06.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.614+0000 7f14c3400640 10 client.7107 ms_handle_connect on v2:172.21.15.100:6832/2666467995 2024-09-17T04:30:06.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.615+0000 7f14c3400640 1 -- 172.21.15.100:0/3158384586 <== mds.0 v2:172.21.15.100:6832/2666467995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557f076a1520 con 0x557f07343e40 2024-09-17T04:30:06.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.615+0000 7f14c3400640 10 client.7107 handle_command_reply: tid=0 2024-09-17T04:30:06.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.615+0000 7f14f0472740 1 client.7107 shutdown 2024-09-17T04:30:06.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.615+0000 7f14f0472740 1 -- 172.21.15.100:0/3158384586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f07364590 msgr2=0x557f077401d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:06.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.615+0000 7f14f0472740 1 --2- 172.21.15.100:0/3158384586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f07364590 0x557f077401d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f14d002aae0 tx=0x7f14d00ba0c0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.615+0000 7f14f0472740 1 -- 172.21.15.100:0/3158384586 shutdown_connections 2024-09-17T04:30:06.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.615+0000 7f14f0472740 1 --2- 172.21.15.100:0/3158384586 >> [v2:172.21.15.100:6832/2666467995,v1:172.21.15.100:6833/2666467995] conn(0x557f07343e40 0x557f07784a30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.615+0000 7f14f0472740 1 --2- 172.21.15.100:0/3158384586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f07385270 0x557f07754a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.615+0000 7f14f0472740 1 --2- 172.21.15.100:0/3158384586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f07364590 0x557f077401d0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.615+0000 7f14c3400640 0 client.7107 ms_handle_reset on v2:172.21.15.100:6832/2666467995 2024-09-17T04:30:06.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.615+0000 7f14f0472740 1 --2- 172.21.15.100:0/3158384586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557f0735c3b0 0x557f0773fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.615+0000 7f14f0472740 1 -- 172.21.15.100:0/3158384586 >> 172.21.15.100:0/3158384586 conn(0x557f073793c0 msgr2=0x557f07752580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:06.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.616+0000 7f14f0472740 1 -- 172.21.15.100:0/3158384586 shutdown_connections 2024-09-17T04:30:06.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.616+0000 7f14f0472740 1 -- 172.21.15.100:0/3158384586 wait complete. 2024-09-17T04:30:06.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.616+0000 7f14f0472740 20 client.7107 trim_cache size 0 max 16384 2024-09-17T04:30:06.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.635+0000 7f14f0472740 1 Processor -- start 2024-09-17T04:30:06.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.635+0000 7f14f0472740 1 -- start start 2024-09-17T04:30:06.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.636+0000 7f14f0472740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557f0735c3b0 0x557f07755570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:06.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.636+0000 7f14f0472740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f07364590 0x557f07755ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:06.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.636+0000 7f14f0472740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f07385270 0x557f07784a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:06.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.636+0000 7f14f0472740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f14e41fdfc0 con 0x557f07385270 2024-09-17T04:30:06.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.636+0000 7f14f0472740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f14e40cbca0 con 0x557f07364590 2024-09-17T04:30:06.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.636+0000 7f14f0472740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f14e40ae8f0 con 0x557f0735c3b0 2024-09-17T04:30:06.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.637+0000 7f14e3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f07385270 0x557f07784a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:06.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.637+0000 7f14e3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f07385270 0x557f07784a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57328/0 (socket says 172.21.15.100:57328) 2024-09-17T04:30:06.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.637+0000 7f14e3200640 1 -- 172.21.15.100:0/3121454816 learned_addr learned my addr 172.21.15.100:0/3121454816 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:06.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.637+0000 7f14e1e00640 1 --2- 172.21.15.100:0/3121454816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f07364590 0x557f07755ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:06.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.637+0000 7f14e2800640 1 --2- 172.21.15.100:0/3121454816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557f0735c3b0 0x557f07755570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:06.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.637+0000 7f14e3200640 1 -- 172.21.15.100:0/3121454816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557f0735c3b0 msgr2=0x557f07755570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:30:06.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.637+0000 7f14e3200640 1 --2- 172.21.15.100:0/3121454816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557f0735c3b0 0x557f07755570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.637+0000 7f14e3200640 1 -- 172.21.15.100:0/3121454816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f07364590 msgr2=0x557f07755ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:30:06.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.637+0000 7f14e3200640 1 --2- 172.21.15.100:0/3121454816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f07364590 0x557f07755ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.637+0000 7f14e3200640 1 -- 172.21.15.100:0/3121454816 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f0772a610 con 0x557f07385270 2024-09-17T04:30:06.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.637+0000 7f14e1e00640 1 --2- 172.21.15.100:0/3121454816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f07364590 0x557f07755ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:30:06.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.637+0000 7f14e2800640 1 --2- 172.21.15.100:0/3121454816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557f0735c3b0 0x557f07755570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:30:06.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.637+0000 7f14e3200640 1 --2- 172.21.15.100:0/3121454816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f07385270 0x557f07784a30 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00cdb0 tx=0x7f14dc004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0ae92b507ba87 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:06.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.638+0000 7f14ce600640 1 -- 172.21.15.100:0/3121454816 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc024030 con 0x557f07385270 2024-09-17T04:30:06.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.638+0000 7f14ce600640 1 -- 172.21.15.100:0/3121454816 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14dc003970 con 0x557f07385270 2024-09-17T04:30:06.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.638+0000 7f14ce600640 1 -- 172.21.15.100:0/3121454816 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc01a040 con 0x557f07385270 2024-09-17T04:30:06.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.638+0000 7f14f0472740 1 -- 172.21.15.100:0/3121454816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f07385270 msgr2=0x557f07784a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:06.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.638+0000 7f14f0472740 1 --2- 172.21.15.100:0/3121454816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f07385270 0x557f07784a30 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00cdb0 tx=0x7f14dc004a30 comp rx=0 tx=0).stop 2024-09-17T04:30:06.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.638+0000 7f14f0472740 1 -- 172.21.15.100:0/3121454816 shutdown_connections 2024-09-17T04:30:06.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.638+0000 7f14f0472740 1 --2- 172.21.15.100:0/3121454816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f07385270 0x557f07784a30 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.638+0000 7f14f0472740 1 --2- 172.21.15.100:0/3121454816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f07364590 0x557f07755ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.638+0000 7f14f0472740 1 --2- 172.21.15.100:0/3121454816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557f0735c3b0 0x557f07755570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.638+0000 7f14f0472740 1 -- 172.21.15.100:0/3121454816 >> 172.21.15.100:0/3121454816 conn(0x557f073793c0 msgr2=0x557f07783e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:06.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.639+0000 7f14f0472740 1 -- 172.21.15.100:0/3121454816 shutdown_connections 2024-09-17T04:30:06.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.639+0000 7f14f0472740 1 -- 172.21.15.100:0/3121454816 wait complete. 2024-09-17T04:30:06.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.639+0000 7f14f0472740 1 -- start start 2024-09-17T04:30:06.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.640+0000 7f14f0472740 1 Processor -- start 2024-09-17T04:30:06.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.641+0000 7f14f0472740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f0735c3b0 0x557f07798760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:06.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.641+0000 7f14f0472740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f07364590 0x557f07755570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:06.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.641+0000 7f14f0472740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557f07385270 0x557f07755ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:06.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.641+0000 7f14f0472740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x557f07787d70 con 0x557f0735c3b0 2024-09-17T04:30:06.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.641+0000 7f14f0472740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f14e40c7e90 con 0x557f07364590 2024-09-17T04:30:06.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.641+0000 7f14f0472740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x557f07788390 con 0x557f07385270 2024-09-17T04:30:06.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.641+0000 7f14f0472740 10 client.0 _ll_register_callbacks cb 0x7f14d007f2a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:30:06.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.641+0000 7f14e2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f0735c3b0 0x557f07798760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:06.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.641+0000 7f14e2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f0735c3b0 0x557f07798760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57336/0 (socket says 172.21.15.100:57336) 2024-09-17T04:30:06.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.641+0000 7f14e2800640 1 -- 172.21.15.100:0/2961666098 learned_addr learned my addr 172.21.15.100:0/2961666098 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:06.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.642+0000 7f14e3200640 1 --2- 172.21.15.100:0/2961666098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557f07385270 0x557f07755ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:06.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.642+0000 7f14e1e00640 1 --2- 172.21.15.100:0/2961666098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f07364590 0x557f07755570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:06.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.642+0000 7f14e2800640 1 -- 172.21.15.100:0/2961666098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557f07385270 msgr2=0x557f07755ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:30:06.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.642+0000 7f14e2800640 1 --2- 172.21.15.100:0/2961666098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557f07385270 0x557f07755ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.642+0000 7f14e2800640 1 -- 172.21.15.100:0/2961666098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f07364590 msgr2=0x557f07755570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:30:06.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.642+0000 7f14e2800640 1 --2- 172.21.15.100:0/2961666098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f07364590 0x557f07755570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.642+0000 7f14e2800640 1 -- 172.21.15.100:0/2961666098 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f0778ec60 con 0x557f0735c3b0 2024-09-17T04:30:06.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.642+0000 7f14e3200640 1 --2- 172.21.15.100:0/2961666098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557f07385270 0x557f07755ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:30:06.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.642+0000 7f14e2800640 1 --2- 172.21.15.100:0/2961666098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f0735c3b0 0x557f07798760 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f14d4010660 tx=0x7f14d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21892031dd1d4881 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:06.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.642+0000 7f14c3400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-17T04:30:06.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.642+0000 7f14c3400640 1 -- 172.21.15.100:0/2961666098 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d4005230 con 0x557f0735c3b0 2024-09-17T04:30:06.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.642+0000 7f14f0472740 1 -- 172.21.15.100:0/2961666098 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14dc003970 con 0x557f0735c3b0 2024-09-17T04:30:06.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.642+0000 7f14c3400640 1 -- 172.21.15.100:0/2961666098 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 5767+0+0 (secure 0 0 0) 0x7f14d40053d0 con 0x557f0735c3b0 2024-09-17T04:30:06.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.642+0000 7f14c3400640 1 -- 172.21.15.100:0/2961666098 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f14d4037050 con 0x557f0735c3b0 2024-09-17T04:30:06.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.643+0000 7f14f0472740 10 client.7113 fetch_fsmap learned FSMap version 72 2024-09-17T04:30:06.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.643+0000 7f14f0472740 1 -- 172.21.15.100:0/2961666098 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x557f077f5ce0 con 0x557f0735c3b0 2024-09-17T04:30:06.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.643+0000 7f14c3400640 1 -- 172.21.15.100:0/2961666098 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14dc003970 con 0x557f0735c3b0 2024-09-17T04:30:06.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.643+0000 7f14c3400640 1 -- 172.21.15.100:0/2961666098 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 72) ==== 3502+0+0 (secure 0 0 0) 0x7f14d403a020 con 0x557f0735c3b0 2024-09-17T04:30:06.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.643+0000 7f14f0472740 10 client.7113 fetch_fsmap finished waiting for FSMap version 72 2024-09-17T04:30:06.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.643+0000 7f14f0472740 10 client.7113 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T04:30:06.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.643+0000 7f14f0472740 20 client.7113 populate_metadata read hostname 'smithi100' 2024-09-17T04:30:06.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.643+0000 7f14f0472740 1 --2- 172.21.15.100:0/2961666098 >> [v2:172.21.15.100:6832/2666467995,v1:172.21.15.100:6833/2666467995] conn(0x557f07798ca0 0x557f077f5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:06.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.643+0000 7f14f0472740 4 client.7113 mds_command: new command op to 6972 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T04:30:06.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.643+0000 7f14f0472740 1 -- 172.21.15.100:0/2961666098 --> [v2:172.21.15.100:6832/2666467995,v1:172.21.15.100:6833/2666467995] -- command(tid 0: {"prefix": "status"}) -- 0x557f0769cfd0 con 0x557f07798ca0 2024-09-17T04:30:06.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.643+0000 7f14e1e00640 1 --2- 172.21.15.100:0/2961666098 >> [v2:172.21.15.100:6832/2666467995,v1:172.21.15.100:6833/2666467995] conn(0x557f07798ca0 0x557f077f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:06.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.644+0000 7f14e1e00640 1 --2- 172.21.15.100:0/2961666098 >> [v2:172.21.15.100:6832/2666467995,v1:172.21.15.100:6833/2666467995] conn(0x557f07798ca0 0x557f077f5670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc1a515b57412bae server_cookie=d1d2ecbc16200a0f in_seq=0 out_seq=0 2024-09-17T04:30:06.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.644+0000 7f14c3400640 10 client.7113 ms_handle_connect on v2:172.21.15.100:6832/2666467995 2024-09-17T04:30:06.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.644+0000 7f14c3400640 1 -- 172.21.15.100:0/2961666098 <== mds.0 v2:172.21.15.100:6832/2666467995 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x557f0769cfd0 con 0x557f07798ca0 2024-09-17T04:30:06.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.644+0000 7f14c3400640 10 client.7113 handle_command_reply: tid=0 2024-09-17T04:30:06.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.644+0000 7f14f0472740 1 client.7113 shutdown 2024-09-17T04:30:06.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.644+0000 7f14f0472740 1 -- 172.21.15.100:0/2961666098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f0735c3b0 msgr2=0x557f07798760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:06.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.644+0000 7f14f0472740 1 --2- 172.21.15.100:0/2961666098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f0735c3b0 0x557f07798760 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f14d4010660 tx=0x7f14d4004730 comp rx=0 tx=0).stop 2024-09-17T04:30:06.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.644+0000 7f14f0472740 1 -- 172.21.15.100:0/2961666098 shutdown_connections 2024-09-17T04:30:06.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.644+0000 7f14f0472740 1 --2- 172.21.15.100:0/2961666098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557f07385270 0x557f07755ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.644+0000 7f14f0472740 1 --2- 172.21.15.100:0/2961666098 >> [v2:172.21.15.100:6832/2666467995,v1:172.21.15.100:6833/2666467995] conn(0x557f07798ca0 0x557f077f5670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.644+0000 7f14f0472740 1 --2- 172.21.15.100:0/2961666098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f07364590 0x557f07755570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.644+0000 7f14c3400640 0 client.7113 ms_handle_reset on v2:172.21.15.100:6832/2666467995 2024-09-17T04:30:06.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.644+0000 7f14f0472740 1 --2- 172.21.15.100:0/2961666098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f0735c3b0 0x557f07798760 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.644+0000 7f14f0472740 1 -- 172.21.15.100:0/2961666098 >> 172.21.15.100:0/2961666098 conn(0x557f073793c0 msgr2=0x557f07767070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:06.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.644+0000 7f14f0472740 1 -- 172.21.15.100:0/2961666098 shutdown_connections 2024-09-17T04:30:06.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.644+0000 7f14f0472740 1 -- 172.21.15.100:0/2961666098 wait complete. 2024-09-17T04:30:06.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.645+0000 7f14f0472740 20 client.7113 trim_cache size 0 max 16384 2024-09-17T04:30:06.651 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:30:06.651 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "8a399842-3a5c-4ea2-9afb-b2704332e59f", 2024-09-17T04:30:06.651 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 0, 2024-09-17T04:30:06.651 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 6972, 2024-09-17T04:30:06.651 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-09-17T04:30:06.651 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-09-17T04:30:06.651 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-09-17T04:30:06.652 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 6.6332194729999996, 2024-09-17T04:30:06.652 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 72, 2024-09-17T04:30:06.652 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 106, 2024-09-17T04:30:06.652 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 105, 2024-09-17T04:30:06.652 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 11.684478137999999 2024-09-17T04:30:06.652 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:30:06.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.645+0000 7f14e9400640 1 -- 172.21.15.100:0/3294215114 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f14c4078cb0 msgr2=0x7f14c407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:06.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.645+0000 7f14e9400640 1 --2- 172.21.15.100:0/3294215114 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f14c4078cb0 0x7f14c407b1c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f14e409e510 tx=0x7f14d400a540 comp rx=0 tx=0).stop 2024-09-17T04:30:06.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.645+0000 7f14e9400640 1 -- 172.21.15.100:0/3294215114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14e40cfc80 msgr2=0x7f14e40c9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:06.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.645+0000 7f14e9400640 1 --2- 172.21.15.100:0/3294215114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14e40cfc80 0x7f14e40c9220 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f14d0006010 tx=0x7f14d00039e0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.645+0000 7f14e9400640 1 -- 172.21.15.100:0/3294215114 shutdown_connections 2024-09-17T04:30:06.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.645+0000 7f14e9400640 1 --2- 172.21.15.100:0/3294215114 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f14c4078cb0 0x7f14c407b1c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.645+0000 7f14e9400640 1 --2- 172.21.15.100:0/3294215114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14e415ce30 0x7f14e40c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.645+0000 7f14e9400640 1 --2- 172.21.15.100:0/3294215114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14e40cfc80 0x7f14e40c9220 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.645+0000 7f14e9400640 1 --2- 172.21.15.100:0/3294215114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14e40705b0 0x7f14e40c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.645+0000 7f14e9400640 1 -- 172.21.15.100:0/3294215114 >> 172.21.15.100:0/3294215114 conn(0x7f14e406d3d0 msgr2=0x7f14e415ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:06.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.645+0000 7f14e9400640 1 -- 172.21.15.100:0/3294215114 shutdown_connections 2024-09-17T04:30:06.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.645+0000 7f14e9400640 1 -- 172.21.15.100:0/3294215114 wait complete. 2024-09-17T04:30:06.669 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:30:06.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.888+0000 7f0da2200640 1 -- 172.21.15.100:0/620758192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d80004d90 msgr2=0x7f0d80005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:06.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.888+0000 7f0da2200640 1 --2- 172.21.15.100:0/620758192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d80004d90 0x7f0d80005220 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f0d94009ed0 tx=0x7f0d80005830 comp rx=0 tx=0).stop 2024-09-17T04:30:06.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.888+0000 7f0d9be00640 1 -- 172.21.15.100:0/620758192 reap_dead start 2024-09-17T04:30:06.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.889+0000 7f0da2200640 1 -- 172.21.15.100:0/620758192 shutdown_connections 2024-09-17T04:30:06.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.889+0000 7f0da2200640 1 -- 172.21.15.100:0/620758192 >> 172.21.15.100:0/620758192 conn(0x7f0d9c06d730 msgr2=0x7f0d9c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:06.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.889+0000 7f0da2200640 1 -- 172.21.15.100:0/620758192 shutdown_connections 2024-09-17T04:30:06.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.889+0000 7f0da2200640 1 -- 172.21.15.100:0/620758192 wait complete. 2024-09-17T04:30:06.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.891+0000 7f0da2200640 1 Processor -- start 2024-09-17T04:30:06.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.891+0000 7f0da2200640 1 -- start start 2024-09-17T04:30:06.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.892+0000 7f0da2200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d80004d90 0x7f0d9c147990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:06.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.892+0000 7f0da2200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d9c14b7c0 0x7f0d9c074940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:06.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.892+0000 7f0da2200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d9c06fae0 0x7f0d9c1f8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:06.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.892+0000 7f0da2200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c06f230 con 0x7f0d9c14b7c0 2024-09-17T04:30:06.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.892+0000 7f0da2200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c0b4e30 con 0x7f0d80004d90 2024-09-17T04:30:06.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.892+0000 7f0da2200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0d9c165890 con 0x7f0d9c06fae0 2024-09-17T04:30:06.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.893+0000 7f0d9aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d9c14b7c0 0x7f0d9c074940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:06.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.893+0000 7f0d9aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d9c14b7c0 0x7f0d9c074940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57364/0 (socket says 172.21.15.100:57364) 2024-09-17T04:30:06.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.893+0000 7f0d9aa00640 1 -- 172.21.15.100:0/4114065373 learned_addr learned my addr 172.21.15.100:0/4114065373 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:06.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.893+0000 7f0d9aa00640 1 -- 172.21.15.100:0/4114065373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d9c06fae0 msgr2=0x7f0d9c1f8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:06.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.893+0000 7f0d9b400640 1 --2- 172.21.15.100:0/4114065373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d80004d90 0x7f0d9c147990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:06.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.893+0000 7f0d9be00640 1 --2- 172.21.15.100:0/4114065373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d9c06fae0 0x7f0d9c1f8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:06.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.893+0000 7f0d9aa00640 1 --2- 172.21.15.100:0/4114065373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d9c06fae0 0x7f0d9c1f8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.893+0000 7f0d9aa00640 1 -- 172.21.15.100:0/4114065373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d80004d90 msgr2=0x7f0d9c147990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:06.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.893+0000 7f0d9aa00640 1 --2- 172.21.15.100:0/4114065373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d80004d90 0x7f0d9c147990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:06.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.893+0000 7f0d9aa00640 1 -- 172.21.15.100:0/4114065373 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9c15b980 con 0x7f0d9c14b7c0 2024-09-17T04:30:06.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.893+0000 7f0d9be00640 1 --2- 172.21.15.100:0/4114065373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d9c06fae0 0x7f0d9c1f8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:30:06.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.893+0000 7f0d9b400640 1 --2- 172.21.15.100:0/4114065373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d80004d90 0x7f0d9c147990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:30:06.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.893+0000 7f0d9aa00640 1 --2- 172.21.15.100:0/4114065373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d9c14b7c0 0x7f0d9c074940 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f0d88006400 tx=0x7f0d880039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:06.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.894+0000 7f0d93400640 1 -- 172.21.15.100:0/4114065373 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d88015070 con 0x7f0d9c14b7c0 2024-09-17T04:30:06.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.894+0000 7f0d93400640 1 -- 172.21.15.100:0/4114065373 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d88011840 con 0x7f0d9c14b7c0 2024-09-17T04:30:06.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.894+0000 7f0d93400640 1 -- 172.21.15.100:0/4114065373 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d88011c90 con 0x7f0d9c14b7c0 2024-09-17T04:30:06.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.894+0000 7f0da2200640 1 -- 172.21.15.100:0/4114065373 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d9c15bc60 con 0x7f0d9c14b7c0 2024-09-17T04:30:06.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.894+0000 7f0da2200640 1 -- 172.21.15.100:0/4114065373 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d9c074e80 con 0x7f0d9c14b7c0 2024-09-17T04:30:06.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.895+0000 7f0da2200640 1 -- 172.21.15.100:0/4114065373 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d9c06f3b0 con 0x7f0d9c14b7c0 2024-09-17T04:30:06.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.896+0000 7f0d93400640 1 -- 172.21.15.100:0/4114065373 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f0d88018070 con 0x7f0d9c14b7c0 2024-09-17T04:30:06.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.897+0000 7f0d93400640 1 --2- 172.21.15.100:0/4114065373 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0d7c078ce0 0x7f0d7c07b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:06.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.897+0000 7f0d93400640 1 -- 172.21.15.100:0/4114065373 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5767+0+0 (secure 0 0 0) 0x7f0d8809bce0 con 0x7f0d9c14b7c0 2024-09-17T04:30:06.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.897+0000 7f0d9b400640 1 --2- 172.21.15.100:0/4114065373 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0d7c078ce0 0x7f0d7c07b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:06.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.898+0000 7f0d9b400640 1 --2- 172.21.15.100:0/4114065373 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0d7c078ce0 0x7f0d7c07b1f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c006550 tx=0x7f0d8c006b10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:06.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:06.902+0000 7f0d93400640 1 -- 172.21.15.100:0/4114065373 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0d88063ec0 con 0x7f0d9c14b7c0 2024-09-17T04:30:07.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:07.125+0000 7f0da2200640 1 -- 172.21.15.100:0/4114065373 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0d9c1535c0 con 0x7f0d9c14b7c0 2024-09-17T04:30:07.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:07.125+0000 7f0d93400640 1 -- 172.21.15.100:0/4114065373 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v72) ==== 53+0+73 (secure 0 0 0) 0x7f0d880637c0 con 0x7f0d9c14b7c0 2024-09-17T04:30:07.126 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:30:07.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:07.127+0000 7f0da2200640 1 -- 172.21.15.100:0/4114065373 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0d7c078ce0 msgr2=0x7f0d7c07b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:07.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:07.127+0000 7f0da2200640 1 --2- 172.21.15.100:0/4114065373 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0d7c078ce0 0x7f0d7c07b1f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c006550 tx=0x7f0d8c006b10 comp rx=0 tx=0).stop 2024-09-17T04:30:07.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:07.127+0000 7f0da2200640 1 -- 172.21.15.100:0/4114065373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d9c14b7c0 msgr2=0x7f0d9c074940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:07.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:07.127+0000 7f0da2200640 1 --2- 172.21.15.100:0/4114065373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d9c14b7c0 0x7f0d9c074940 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f0d88006400 tx=0x7f0d880039c0 comp rx=0 tx=0).stop 2024-09-17T04:30:07.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:07.128+0000 7f0da2200640 1 -- 172.21.15.100:0/4114065373 shutdown_connections 2024-09-17T04:30:07.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:07.128+0000 7f0da2200640 1 --2- 172.21.15.100:0/4114065373 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0d7c078ce0 0x7f0d7c07b1f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:07.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:07.128+0000 7f0da2200640 1 --2- 172.21.15.100:0/4114065373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d9c06fae0 0x7f0d9c1f8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:07.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:07.128+0000 7f0da2200640 1 --2- 172.21.15.100:0/4114065373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d9c14b7c0 0x7f0d9c074940 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:07.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:07.128+0000 7f0da2200640 1 --2- 172.21.15.100:0/4114065373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d80004d90 0x7f0d9c147990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:07.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:07.128+0000 7f0da2200640 1 -- 172.21.15.100:0/4114065373 >> 172.21.15.100:0/4114065373 conn(0x7f0d9c06d730 msgr2=0x7f0d9c14e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:07.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:07.128+0000 7f0da2200640 1 -- 172.21.15.100:0/4114065373 shutdown_connections 2024-09-17T04:30:07.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:07.128+0000 7f0da2200640 1 -- 172.21.15.100:0/4114065373 wait complete. 2024-09-17T04:30:07.150 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T04:30:07.150 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:30:07.150 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-09-17T04:30:07.150 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T04:30:07.150 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T04:30:07.150 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T04:30:07.150 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T04:30:07.150 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:30:07.150 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:30:07.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:07.151 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-09-17T04:30:07.187 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:30:07.187 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:30:07.187 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:30:07.187 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:30:07.187 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-17T04:30:07.187 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:30:07.187 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:30:07.187 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:30:07.187 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-09-17T04:30:07.188 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:30:07.188 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:30:07.188 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-09-17T04:30:07.188 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:30:07.188 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:30:07.188 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:30:07.188 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:30:07.188 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T04:30:07.188 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42353sec preferred_lft 42353sec 2024-09-17T04:30:07.188 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link noprefixroute 2024-09-17T04:30:07.188 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:30:07.188 INFO:teuthology.orchestra.run.smithi100.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:30:07.188 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:2e:de:8e:a4:7c brd ff:ff:ff:ff:ff:ff 2024-09-17T04:30:07.189 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T04:30:07.189 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:30:07.189 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::cc78:e4ff:fe70:7fbe/64 scope link 2024-09-17T04:30:07.189 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:30:07.189 INFO:teuthology.orchestra.run.smithi100.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:30:07.189 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether ea:8c:45:9f:34:87 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:30:07.189 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::e88c:45ff:fe9f:3487/64 scope link 2024-09-17T04:30:07.189 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:30:07.189 INFO:teuthology.orchestra.run.smithi100.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:30:07.189 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:2e:de:8e:a4:7c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:30:07.189 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::c2e:deff:fe8e:a47c/64 scope link 2024-09-17T04:30:07.189 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:30:07.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:07.190 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:30:07.280 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-09-17T04:30:07.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:07.283 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-09-17T04:30:07.319 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-09-17T04:30:07.319 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:30:07.319 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T04:30:07.319 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T04:30:07.319 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:30:07.319 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:30:07.320 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-17T04:30:07.320 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T04:30:07.320 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T04:30:07.320 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T04:30:07.320 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-17T04:30:07.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:30:07.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:30:07.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:07.519 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-17T04:30:07.554 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T04:30:07.554 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T04:30:07.555 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:30:07.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:07.555 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-09-17T04:30:07.652 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:30:07.652 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T04:30:07.687 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:30:07.687 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:30:07.687 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:30:07.762 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T04:30:07.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:07.762 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:30:07.958 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:30:07.958 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:30:07.958 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:30:07.978 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:30:07.978 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:30:07.979 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:30:07.979 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:30:07.979 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:30:08.011 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:30:08.011 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:30:08.011 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:30:08.011 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:30:08.011 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:30:08.011 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-09-17T04:30:08.011 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-09-17T04:30:08.012 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8a399842-3a5c-4ea2-9afb-b2704332e59f" will pass to kernel 2024-09-17T04:30:08.013 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-17T04:30:08.013 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-09-17T04:30:08.013 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-17T04:30:08.013 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-09-17T04:30:08.013 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-17T04:30:08.016 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T04:30:08.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:08.016 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:30:08.074 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:30:08.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:30:08.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:30:08.169 INFO:teuthology.orchestra.run.smithi100.stdout:client6814 2024-09-17T04:30:08.169 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:30:08.169 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:30:08.205 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:30:08.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.467+0000 7f869fc00640 1 -- 172.21.15.100:0/910106693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f867c004800 msgr2=0x7f867c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:08.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.467+0000 7f869fc00640 1 --2- 172.21.15.100:0/910106693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f867c004800 0x7f867c004c10 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f868400c2e0 tx=0x7f867c005830 comp rx=0 tx=0).stop 2024-09-17T04:30:08.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.467+0000 7f869fc00640 1 -- 172.21.15.100:0/910106693 shutdown_connections 2024-09-17T04:30:08.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.467+0000 7f869fc00640 1 --2- 172.21.15.100:0/910106693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f867c004800 0x7f867c004c10 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:08.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.467+0000 7f869fc00640 1 -- 172.21.15.100:0/910106693 >> 172.21.15.100:0/910106693 conn(0x7f869806d730 msgr2=0x7f86980b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:08.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.467+0000 7f869fc00640 1 -- 172.21.15.100:0/910106693 shutdown_connections 2024-09-17T04:30:08.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.467+0000 7f869fc00640 1 -- 172.21.15.100:0/910106693 wait complete. 2024-09-17T04:30:08.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.470+0000 7f869fc00640 1 Processor -- start 2024-09-17T04:30:08.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.470+0000 7f869fc00640 1 -- start start 2024-09-17T04:30:08.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.471+0000 7f869fc00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f867c004800 0x7f8698099b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:08.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.471+0000 7f869fc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8698099600 0x7f8698073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:08.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.471+0000 7f869fc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f869814b0d0 0x7f86980b1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:08.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.471+0000 7f869fc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f86980ad4f0 con 0x7f8698099600 2024-09-17T04:30:08.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.471+0000 7f869fc00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f86980b4e30 con 0x7f869814b0d0 2024-09-17T04:30:08.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.471+0000 7f869fc00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f86980ad670 con 0x7f867c004800 2024-09-17T04:30:08.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.472+0000 7f8697e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8698099600 0x7f8698073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:08.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.472+0000 7f8697e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8698099600 0x7f8698073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57386/0 (socket says 172.21.15.100:57386) 2024-09-17T04:30:08.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.472+0000 7f8697e00640 1 -- 172.21.15.100:0/3665505347 learned_addr learned my addr 172.21.15.100:0/3665505347 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:08.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.472+0000 7f869d200640 1 --2- 172.21.15.100:0/3665505347 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f867c004800 0x7f8698099b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:08.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.472+0000 7f869dc00640 1 --2- 172.21.15.100:0/3665505347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f869814b0d0 0x7f86980b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:08.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.472+0000 7f8697e00640 1 -- 172.21.15.100:0/3665505347 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f867c004800 msgr2=0x7f8698099b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:08.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.472+0000 7f8697e00640 1 --2- 172.21.15.100:0/3665505347 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f867c004800 0x7f8698099b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:08.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.472+0000 7f8697e00640 1 -- 172.21.15.100:0/3665505347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f869814b0d0 msgr2=0x7f86980b1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:08.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.472+0000 7f8697e00640 1 --2- 172.21.15.100:0/3665505347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f869814b0d0 0x7f86980b1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:08.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.472+0000 7f8697e00640 1 -- 172.21.15.100:0/3665505347 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8698073580 con 0x7f8698099600 2024-09-17T04:30:08.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.472+0000 7f8697e00640 1 --2- 172.21.15.100:0/3665505347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8698099600 0x7f8698073040 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f868400c400 tx=0x7f8684001d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:08.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.474+0000 7f8694c00640 1 -- 172.21.15.100:0/3665505347 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684031040 con 0x7f8698099600 2024-09-17T04:30:08.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.474+0000 7f8694c00640 1 -- 172.21.15.100:0/3665505347 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86840035f0 con 0x7f8698099600 2024-09-17T04:30:08.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.474+0000 7f869fc00640 1 -- 172.21.15.100:0/3665505347 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8698148510 con 0x7f8698099600 2024-09-17T04:30:08.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.474+0000 7f8694c00640 1 -- 172.21.15.100:0/3665505347 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684003a20 con 0x7f8698099600 2024-09-17T04:30:08.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.474+0000 7f869fc00640 1 -- 172.21.15.100:0/3665505347 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86980737c0 con 0x7f8698099600 2024-09-17T04:30:08.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.475+0000 7f869fc00640 1 -- 172.21.15.100:0/3665505347 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86981535c0 con 0x7f8698099600 2024-09-17T04:30:08.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.476+0000 7f8694c00640 1 -- 172.21.15.100:0/3665505347 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f868403e030 con 0x7f8698099600 2024-09-17T04:30:08.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.477+0000 7f8694c00640 1 --2- 172.21.15.100:0/3665505347 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8678078cb0 0x7f867807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:08.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.477+0000 7f8694c00640 1 -- 172.21.15.100:0/3665505347 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f86840baaa0 con 0x7f8698099600 2024-09-17T04:30:08.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.477+0000 7f869d200640 1 --2- 172.21.15.100:0/3665505347 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8678078cb0 0x7f867807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:08.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.478+0000 7f869d200640 1 --2- 172.21.15.100:0/3665505347 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8678078cb0 0x7f867807b1c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f86980521d0 tx=0x7f86880049b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:08.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.481+0000 7f8694c00640 1 -- 172.21.15.100:0/3665505347 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8684082750 con 0x7f8698099600 2024-09-17T04:30:08.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.699+0000 7f869fc00640 1 -- 172.21.15.100:0/3665505347 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f869815e340 con 0x7f8698099600 2024-09-17T04:30:08.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.700+0000 7f8694c00640 1 -- 172.21.15.100:0/3665505347 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v72) ==== 53+0+73 (secure 0 0 0) 0x7f8684087180 con 0x7f8698099600 2024-09-17T04:30:08.700 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:30:08.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.702+0000 7f869fc00640 1 -- 172.21.15.100:0/3665505347 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8678078cb0 msgr2=0x7f867807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:08.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.702+0000 7f869fc00640 1 --2- 172.21.15.100:0/3665505347 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8678078cb0 0x7f867807b1c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f86980521d0 tx=0x7f86880049b0 comp rx=0 tx=0).stop 2024-09-17T04:30:08.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.702+0000 7f869fc00640 1 -- 172.21.15.100:0/3665505347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8698099600 msgr2=0x7f8698073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.702+0000 7f869fc00640 1 --2- 172.21.15.100:0/3665505347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8698099600 0x7f8698073040 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f868400c400 tx=0x7f8684001d90 comp rx=0 tx=0).stop 2024-09-17T04:30:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.702+0000 7f869fc00640 1 -- 172.21.15.100:0/3665505347 shutdown_connections 2024-09-17T04:30:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.702+0000 7f869fc00640 1 --2- 172.21.15.100:0/3665505347 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8678078cb0 0x7f867807b1c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.702+0000 7f869fc00640 1 --2- 172.21.15.100:0/3665505347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f869814b0d0 0x7f86980b1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.702+0000 7f869fc00640 1 --2- 172.21.15.100:0/3665505347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8698099600 0x7f8698073040 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.702+0000 7f869fc00640 1 --2- 172.21.15.100:0/3665505347 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f867c004800 0x7f8698099b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.702+0000 7f869fc00640 1 -- 172.21.15.100:0/3665505347 >> 172.21.15.100:0/3665505347 conn(0x7f869806d730 msgr2=0x7f86981602a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.702+0000 7f869fc00640 1 -- 172.21.15.100:0/3665505347 shutdown_connections 2024-09-17T04:30:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:08.702+0000 7f869fc00640 1 -- 172.21.15.100:0/3665505347 wait complete. 2024-09-17T04:30:08.725 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T04:30:08.725 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:30:08.725 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2024-09-17T04:30:08.725 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T04:30:08.725 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T04:30:08.725 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T04:30:08.725 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T04:30:08.725 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:30:08.725 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:30:08.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:08.725 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2024-09-17T04:30:08.767 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:30:08.767 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:30:08.767 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:30:08.767 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:30:08.767 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-09-17T04:30:08.767 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:30:08.767 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:30:08.767 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:30:08.767 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2024-09-17T04:30:08.768 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:30:08.768 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:30:08.768 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2024-09-17T04:30:08.768 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:30:08.768 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2024-09-17T04:30:08.768 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:30:08.768 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2024-09-17T04:30:08.768 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T04:30:08.768 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42335sec preferred_lft 42335sec 2024-09-17T04:30:08.768 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ec4:7aff:fe88:7b8f/64 scope link noprefixroute 2024-09-17T04:30:08.768 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:30:08.768 INFO:teuthology.orchestra.run.smithi184.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:30:08.768 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 7e:67:e5:69:96:69 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:30:08.768 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T04:30:08.769 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:30:08.769 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::3c9d:aff:fe6e:8fcb/64 scope link 2024-09-17T04:30:08.769 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:30:08.769 INFO:teuthology.orchestra.run.smithi184.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:30:08.769 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether a6:a9:1b:a9:61:ec brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T04:30:08.769 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::a4a9:1bff:fea9:61ec/64 scope link 2024-09-17T04:30:08.769 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:30:08.769 INFO:teuthology.orchestra.run.smithi184.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:30:08.769 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 7e:67:e5:69:96:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T04:30:08.769 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::7c67:e5ff:fe69:9669/64 scope link 2024-09-17T04:30:08.769 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:30:08.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:08.770 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:30:08.865 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-09-17T04:30:08.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:08.869 DEBUG:teuthology.orchestra.run.smithi184:> route 2024-09-17T04:30:08.906 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2024-09-17T04:30:08.906 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:30:08.906 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T04:30:08.906 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T04:30:08.906 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:30:08.907 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:30:08.907 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-09-17T04:30:08.907 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T04:30:08.907 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T04:30:08.907 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T04:30:08.907 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-09-17T04:30:09.025 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:30:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:30:09.095 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:30:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:30:09.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:09.103 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-09-17T04:30:09.139 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T04:30:09.139 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T04:30:09.139 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:30:09.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:09.139 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.0 up 2024-09-17T04:30:09.237 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:30:09.237 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T04:30:09.272 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:30:09.273 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:30:09.273 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T04:30:09.348 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T04:30:09.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:09.349 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:30:09.543 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:30:09.543 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:30:09.543 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:30:09.563 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:30:09.563 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:30:09.563 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:30:09.563 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:30:09.563 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:30:09.592 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:30:09.592 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:30:09.593 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:30:09.593 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:30:09.593 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:30:09.593 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-09-17T04:30:09.593 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-09-17T04:30:09.593 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8a399842-3a5c-4ea2-9afb-b2704332e59f" will pass to kernel 2024-09-17T04:30:09.594 INFO:teuthology.orchestra.run.smithi184.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-17T04:30:09.594 INFO:teuthology.orchestra.run.smithi184.stdout: You just mounted a file system that supports labels which does not 2024-09-17T04:30:09.594 INFO:teuthology.orchestra.run.smithi184.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-17T04:30:09.594 INFO:teuthology.orchestra.run.smithi184.stdout: applications will generate AVC messages and not be allowed access to 2024-09-17T04:30:09.594 INFO:teuthology.orchestra.run.smithi184.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-17T04:30:09.597 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T04:30:09.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:09.597 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T04:30:09.653 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:30:09.732 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:30:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:30:09.735 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:30:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:30:09.743 INFO:teuthology.orchestra.run.smithi184.stdout:client6134 2024-09-17T04:30:09.743 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:30:09.743 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:30:09.778 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T04:30:10.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.008+0000 7f7c91200640 1 -- 172.21.15.100:0/3032356706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c8c0b9bc0 msgr2=0x7f7c8c0d2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:10.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.008+0000 7f7c91200640 1 --2- 172.21.15.100:0/3032356706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c8c0b9bc0 0x7f7c8c0d2650 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c0d0f70 tx=0x7f7c7c02a560 comp rx=0 tx=0).stop 2024-09-17T04:30:10.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.009+0000 7f7c91200640 1 -- 172.21.15.100:0/3032356706 shutdown_connections 2024-09-17T04:30:10.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.009+0000 7f7c91200640 1 --2- 172.21.15.100:0/3032356706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c8c0ceb70 0x7f7c8c0cef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:10.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.009+0000 7f7c91200640 1 --2- 172.21.15.100:0/3032356706 >> v2:172.21.15.184:3300/0 conn(0x7f7c8c0b6700 0x7f7c8c0d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:10.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.009+0000 7f7c91200640 1 --2- 172.21.15.100:0/3032356706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c8c0b9bc0 0x7f7c8c0d2650 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:10.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.009+0000 7f7c91200640 1 -- 172.21.15.100:0/3032356706 >> 172.21.15.100:0/3032356706 conn(0x7f7c8c08bfe0 msgr2=0x7f7c8c0d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:10.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.009+0000 7f7c91200640 1 -- 172.21.15.100:0/3032356706 shutdown_connections 2024-09-17T04:30:10.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.009+0000 7f7c91200640 1 -- 172.21.15.100:0/3032356706 wait complete. 2024-09-17T04:30:10.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.011+0000 7f7c91200640 1 Processor -- start 2024-09-17T04:30:10.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.011+0000 7f7c91200640 1 -- start start 2024-09-17T04:30:10.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.013+0000 7f7c91200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c8c0b6700 0x7f7c8c1592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:10.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.013+0000 7f7c91200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c8c0b9bc0 0x7f7c8c0cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:10.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.013+0000 7f7c91200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c8c0ceb70 0x7f7c8c0cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:10.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.013+0000 7f7c91200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c0cf4c0 con 0x7f7c8c0b9bc0 2024-09-17T04:30:10.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.013+0000 7f7c91200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c0d4e40 con 0x7f7c8c0b6700 2024-09-17T04:30:10.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.013+0000 7f7c91200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7c8c0cf640 con 0x7f7c8c0ceb70 2024-09-17T04:30:10.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.013+0000 7f7c8aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c8c0b9bc0 0x7f7c8c0cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:10.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.013+0000 7f7c8aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c8c0b9bc0 0x7f7c8c0cd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57416/0 (socket says 172.21.15.100:57416) 2024-09-17T04:30:10.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.013+0000 7f7c8aa00640 1 -- 172.21.15.100:0/625661683 learned_addr learned my addr 172.21.15.100:0/625661683 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:10.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.013+0000 7f7c8b400640 1 --2- 172.21.15.100:0/625661683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c8c0b6700 0x7f7c8c1592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:10.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.013+0000 7f7c8be00640 1 --2- 172.21.15.100:0/625661683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c8c0ceb70 0x7f7c8c0cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:10.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.013+0000 7f7c8aa00640 1 -- 172.21.15.100:0/625661683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c8c0ceb70 msgr2=0x7f7c8c0cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:10.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.013+0000 7f7c8aa00640 1 --2- 172.21.15.100:0/625661683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c8c0ceb70 0x7f7c8c0cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:10.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.013+0000 7f7c8aa00640 1 -- 172.21.15.100:0/625661683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c8c0b6700 msgr2=0x7f7c8c1592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:10.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.013+0000 7f7c8aa00640 1 --2- 172.21.15.100:0/625661683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c8c0b6700 0x7f7c8c1592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:10.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.013+0000 7f7c8aa00640 1 -- 172.21.15.100:0/625661683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8c0c8770 con 0x7f7c8c0b9bc0 2024-09-17T04:30:10.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.014+0000 7f7c8aa00640 1 --2- 172.21.15.100:0/625661683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c8c0b9bc0 0x7f7c8c0cd340 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800d720 tx=0x7f7c7800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:10.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.014+0000 7f7c83400640 1 -- 172.21.15.100:0/625661683 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c78017070 con 0x7f7c8c0b9bc0 2024-09-17T04:30:10.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.014+0000 7f7c83400640 1 -- 172.21.15.100:0/625661683 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c78009ab0 con 0x7f7c8c0b9bc0 2024-09-17T04:30:10.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.014+0000 7f7c91200640 1 -- 172.21.15.100:0/625661683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c8c0c8a50 con 0x7f7c8c0b9bc0 2024-09-17T04:30:10.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.014+0000 7f7c83400640 1 -- 172.21.15.100:0/625661683 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c78010460 con 0x7f7c8c0b9bc0 2024-09-17T04:30:10.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.014+0000 7f7c91200640 1 -- 172.21.15.100:0/625661683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c8c0c9010 con 0x7f7c8c0b9bc0 2024-09-17T04:30:10.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.015+0000 7f7c91200640 1 -- 172.21.15.100:0/625661683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c8c0cef80 con 0x7f7c8c0b9bc0 2024-09-17T04:30:10.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.017+0000 7f7c83400640 1 -- 172.21.15.100:0/625661683 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f7c7801a070 con 0x7f7c8c0b9bc0 2024-09-17T04:30:10.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.018+0000 7f7c83400640 1 --2- 172.21.15.100:0/625661683 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7c6c078ce0 0x7f7c6c07b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:10.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.019+0000 7f7c83400640 1 -- 172.21.15.100:0/625661683 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f7c7809afe0 con 0x7f7c8c0b9bc0 2024-09-17T04:30:10.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.019+0000 7f7c8b400640 1 --2- 172.21.15.100:0/625661683 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7c6c078ce0 0x7f7c6c07b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:10.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.019+0000 7f7c8b400640 1 --2- 172.21.15.100:0/625661683 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7c6c078ce0 0x7f7c6c07b1f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c033000 tx=0x7f7c7c004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:10.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.023+0000 7f7c83400640 1 -- 172.21.15.100:0/625661683 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f7c78062b70 con 0x7f7c8c0b9bc0 2024-09-17T04:30:10.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.244+0000 7f7c91200640 1 -- 172.21.15.100:0/625661683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f7c8c0b6e50 con 0x7f7c8c0b9bc0 2024-09-17T04:30:10.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.481+0000 7f7c83400640 1 -- 172.21.15.100:0/625661683 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v73) ==== 136+0+0 (secure 0 0 0) 0x7f7c780675a0 con 0x7f7c8c0b9bc0 2024-09-17T04:30:10.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.484+0000 7f7c91200640 1 -- 172.21.15.100:0/625661683 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7c6c078ce0 msgr2=0x7f7c6c07b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:10.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.484+0000 7f7c91200640 1 --2- 172.21.15.100:0/625661683 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7c6c078ce0 0x7f7c6c07b1f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c033000 tx=0x7f7c7c004730 comp rx=0 tx=0).stop 2024-09-17T04:30:10.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.484+0000 7f7c91200640 1 -- 172.21.15.100:0/625661683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c8c0b9bc0 msgr2=0x7f7c8c0cd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:10.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.484+0000 7f7c91200640 1 --2- 172.21.15.100:0/625661683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c8c0b9bc0 0x7f7c8c0cd340 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800d720 tx=0x7f7c7800dbe0 comp rx=0 tx=0).stop 2024-09-17T04:30:10.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.484+0000 7f7c91200640 1 -- 172.21.15.100:0/625661683 shutdown_connections 2024-09-17T04:30:10.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.484+0000 7f7c91200640 1 --2- 172.21.15.100:0/625661683 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7c6c078ce0 0x7f7c6c07b1f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:10.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.484+0000 7f7c91200640 1 --2- 172.21.15.100:0/625661683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c8c0ceb70 0x7f7c8c0cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:10.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.484+0000 7f7c91200640 1 --2- 172.21.15.100:0/625661683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c8c0b9bc0 0x7f7c8c0cd340 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:10.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.484+0000 7f7c91200640 1 --2- 172.21.15.100:0/625661683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c8c0b6700 0x7f7c8c1592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:10.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.484+0000 7f7c91200640 1 -- 172.21.15.100:0/625661683 >> 172.21.15.100:0/625661683 conn(0x7f7c8c08bfe0 msgr2=0x7f7c8c0b3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:10.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.484+0000 7f7c91200640 1 -- 172.21.15.100:0/625661683 shutdown_connections 2024-09-17T04:30:10.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.484+0000 7f7c91200640 1 -- 172.21.15.100:0/625661683 wait complete. 2024-09-17T04:30:10.510 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T04:30:10.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.731+0000 7f2aae800640 1 -- 172.21.15.100:0/482679923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a8c005150 msgr2=0x7f2a8c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:10.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.731+0000 7f2aae800640 1 --2- 172.21.15.100:0/482679923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a8c005150 0x7f2a8c00b9c0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0001d90 tx=0x7f2a8c005830 comp rx=0 tx=0).stop 2024-09-17T04:30:10.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.731+0000 7f2aa7e00640 1 -- 172.21.15.100:0/482679923 reap_dead start 2024-09-17T04:30:10.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.731+0000 7f2aae800640 1 -- 172.21.15.100:0/482679923 shutdown_connections 2024-09-17T04:30:10.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.731+0000 7f2aae800640 1 -- 172.21.15.100:0/482679923 >> 172.21.15.100:0/482679923 conn(0x7f2aa806d3d0 msgr2=0x7f2aa80b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:10.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.731+0000 7f2aae800640 1 -- 172.21.15.100:0/482679923 shutdown_connections 2024-09-17T04:30:10.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.731+0000 7f2aae800640 1 -- 172.21.15.100:0/482679923 wait complete. 2024-09-17T04:30:10.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.733+0000 7f2aae800640 1 Processor -- start 2024-09-17T04:30:10.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.734+0000 7f2aae800640 1 -- start start 2024-09-17T04:30:10.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.735+0000 7f2aae800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2aa80742d0 0x7f2aa81f3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:10.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.735+0000 7f2aae800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2aa80b15e0 0x7f2aa80ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:10.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.735+0000 7f2aae800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2aa80ae750 0x7f2aa81f61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:10.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.735+0000 7f2aae800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa8156fc0 con 0x7f2aa80742d0 2024-09-17T04:30:10.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.735+0000 7f2aae800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa8156e40 con 0x7f2aa80ae750 2024-09-17T04:30:10.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.735+0000 7f2aae800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2aa8157140 con 0x7f2aa80b15e0 2024-09-17T04:30:10.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.735+0000 7f2aa7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2aa80742d0 0x7f2aa81f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:10.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.735+0000 7f2aa7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2aa80742d0 0x7f2aa81f3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57432/0 (socket says 172.21.15.100:57432) 2024-09-17T04:30:10.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.735+0000 7f2aa7400640 1 -- 172.21.15.100:0/481633743 learned_addr learned my addr 172.21.15.100:0/481633743 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:10.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.736+0000 7f2aa7e00640 1 --2- 172.21.15.100:0/481633743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2aa80ae750 0x7f2aa81f61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:10.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.736+0000 7f2a9ea00640 1 --2- 172.21.15.100:0/481633743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2aa80b15e0 0x7f2aa80ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:10.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.736+0000 7f2aa7400640 1 -- 172.21.15.100:0/481633743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2aa80b15e0 msgr2=0x7f2aa80ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:10.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.736+0000 7f2aa7400640 1 --2- 172.21.15.100:0/481633743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2aa80b15e0 0x7f2aa80ae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:10.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.736+0000 7f2aa7400640 1 -- 172.21.15.100:0/481633743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2aa80ae750 msgr2=0x7f2aa81f61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:10.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.736+0000 7f2aa7400640 1 --2- 172.21.15.100:0/481633743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2aa80ae750 0x7f2aa81f61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:10.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.736+0000 7f2aa7400640 1 -- 172.21.15.100:0/481633743 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa8074840 con 0x7f2aa80742d0 2024-09-17T04:30:10.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.736+0000 7f2aa7e00640 1 --2- 172.21.15.100:0/481633743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2aa80ae750 0x7f2aa81f61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:30:10.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.736+0000 7f2aa7400640 1 --2- 172.21.15.100:0/481633743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2aa80742d0 0x7f2aa81f3170 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f2a980066b0 tx=0x7f2a98003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:10.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.736+0000 7f2a9fe00640 1 -- 172.21.15.100:0/481633743 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9800f780 con 0x7f2aa80742d0 2024-09-17T04:30:10.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.736+0000 7f2a9fe00640 1 -- 172.21.15.100:0/481633743 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a98016020 con 0x7f2aa80742d0 2024-09-17T04:30:10.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.736+0000 7f2aae800640 1 -- 172.21.15.100:0/481633743 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aa8074b20 con 0x7f2aa80742d0 2024-09-17T04:30:10.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.736+0000 7f2a9fe00640 1 -- 172.21.15.100:0/481633743 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9800fbe0 con 0x7f2aa80742d0 2024-09-17T04:30:10.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.737+0000 7f2aae800640 1 -- 172.21.15.100:0/481633743 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aa80750e0 con 0x7f2aa80742d0 2024-09-17T04:30:10.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.737+0000 7f2aae800640 1 -- 172.21.15.100:0/481633743 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aa81e94a0 con 0x7f2aa80742d0 2024-09-17T04:30:10.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.739+0000 7f2a9fe00640 1 -- 172.21.15.100:0/481633743 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f2a98011430 con 0x7f2aa80742d0 2024-09-17T04:30:10.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.739+0000 7f2a9fe00640 1 --2- 172.21.15.100:0/481633743 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2a84078cb0 0x7f2a8407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:10.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.740+0000 7f2a9fe00640 1 -- 172.21.15.100:0/481633743 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f2a9809b770 con 0x7f2aa80742d0 2024-09-17T04:30:10.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.740+0000 7f2a9ea00640 1 --2- 172.21.15.100:0/481633743 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2a84078cb0 0x7f2a8407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:10.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.740+0000 7f2a9ea00640 1 --2- 172.21.15.100:0/481633743 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2a84078cb0 0x7f2a8407b1c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2a90009d30 tx=0x7f2a90005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:10.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.744+0000 7f2a9fe00640 1 -- 172.21.15.100:0/481633743 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f2aa81e94a0 con 0x7f2aa80742d0 2024-09-17T04:30:10.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.952+0000 7f2aae800640 1 -- 172.21.15.100:0/481633743 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f2aa80afa70 con 0x7f2a84078cb0 2024-09-17T04:30:10.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.953+0000 7f2a9fe00640 1 -- 172.21.15.100:0/481633743 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f2aa80afa70 con 0x7f2a84078cb0 2024-09-17T04:30:10.953 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-09-17T04:30:10.953 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:30:10.954 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephfs" 2024-09-17T04:30:10.954 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:30:10.954 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-09-17T04:30:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.955+0000 7f2aae800640 1 -- 172.21.15.100:0/481633743 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2a84078cb0 msgr2=0x7f2a8407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.955+0000 7f2aae800640 1 --2- 172.21.15.100:0/481633743 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2a84078cb0 0x7f2a8407b1c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2a90009d30 tx=0x7f2a90005f90 comp rx=0 tx=0).stop 2024-09-17T04:30:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.955+0000 7f2aae800640 1 -- 172.21.15.100:0/481633743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2aa80742d0 msgr2=0x7f2aa81f3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.955+0000 7f2aae800640 1 --2- 172.21.15.100:0/481633743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2aa80742d0 0x7f2aa81f3170 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f2a980066b0 tx=0x7f2a98003aa0 comp rx=0 tx=0).stop 2024-09-17T04:30:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.956+0000 7f2aae800640 1 -- 172.21.15.100:0/481633743 shutdown_connections 2024-09-17T04:30:10.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.956+0000 7f2aae800640 1 --2- 172.21.15.100:0/481633743 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2a84078cb0 0x7f2a8407b1c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:10.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.956+0000 7f2aae800640 1 --2- 172.21.15.100:0/481633743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2aa80ae750 0x7f2aa81f61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:10.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.956+0000 7f2aae800640 1 --2- 172.21.15.100:0/481633743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2aa80b15e0 0x7f2aa80ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:10.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.956+0000 7f2aae800640 1 --2- 172.21.15.100:0/481633743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2aa80742d0 0x7f2aa81f3170 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:10.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.956+0000 7f2aae800640 1 -- 172.21.15.100:0/481633743 >> 172.21.15.100:0/481633743 conn(0x7f2aa806d3d0 msgr2=0x7f2aa8160920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:10.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.956+0000 7f2aae800640 1 -- 172.21.15.100:0/481633743 shutdown_connections 2024-09-17T04:30:10.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:10.956+0000 7f2aae800640 1 -- 172.21.15.100:0/481633743 wait complete. 2024-09-17T04:30:10.979 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T04:30:11.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.201+0000 7fb3f9000640 1 -- 172.21.15.100:0/811166537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3f4145a40 msgr2=0x7fb3f4073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:11.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.201+0000 7fb3f9000640 1 --2- 172.21.15.100:0/811166537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3f4145a40 0x7fb3f4073540 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4072bd0 tx=0x7fb3dc02a680 comp rx=0 tx=0).stop 2024-09-17T04:30:11.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.201+0000 7fb3f9000640 1 -- 172.21.15.100:0/811166537 shutdown_connections 2024-09-17T04:30:11.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.201+0000 7fb3f9000640 1 --2- 172.21.15.100:0/811166537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3f4145a40 0x7fb3f4073540 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:11.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.201+0000 7fb3f9000640 1 --2- 172.21.15.100:0/811166537 >> v2:172.21.15.184:3300/0 conn(0x7fb3f4145550 0x7fb3f406fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:11.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.201+0000 7fb3f9000640 1 -- 172.21.15.100:0/811166537 >> 172.21.15.100:0/811166537 conn(0x7fb3f406d1c0 msgr2=0x7fb3f406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:11.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.201+0000 7fb3f9000640 1 -- 172.21.15.100:0/811166537 shutdown_connections 2024-09-17T04:30:11.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.201+0000 7fb3f9000640 1 -- 172.21.15.100:0/811166537 wait complete. 2024-09-17T04:30:11.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.203+0000 7fb3f9000640 1 Processor -- start 2024-09-17T04:30:11.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.203+0000 7fb3f9000640 1 -- start start 2024-09-17T04:30:11.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.205+0000 7fb3f9000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3f4073a80 0x7fb3f41f3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:11.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.205+0000 7fb3f9000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3f4145550 0x7fb3f4152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:11.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.205+0000 7fb3f9000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3f4145a40 0x7fb3f4153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:11.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.205+0000 7fb3f9000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f4160a30 con 0x7fb3f4145a40 2024-09-17T04:30:11.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.205+0000 7fb3f9000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f40ce870 con 0x7fb3f4073a80 2024-09-17T04:30:11.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.205+0000 7fb3f9000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb3f4160bb0 con 0x7fb3f4145550 2024-09-17T04:30:11.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.205+0000 7fb3f2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3f4145a40 0x7fb3f4153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:11.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.205+0000 7fb3f2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3f4145a40 0x7fb3f4153200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57454/0 (socket says 172.21.15.100:57454) 2024-09-17T04:30:11.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.205+0000 7fb3f2800640 1 -- 172.21.15.100:0/4220320402 learned_addr learned my addr 172.21.15.100:0/4220320402 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:11.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.205+0000 7fb3f1e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3f4073a80 0x7fb3f41f3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:11.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.205+0000 7fb3e9400640 1 --2- 172.21.15.100:0/4220320402 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3f4145550 0x7fb3f4152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:11.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.205+0000 7fb3f2800640 1 -- 172.21.15.100:0/4220320402 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3f4145550 msgr2=0x7fb3f4152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:11.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.205+0000 7fb3f2800640 1 --2- 172.21.15.100:0/4220320402 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3f4145550 0x7fb3f4152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:11.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.205+0000 7fb3f2800640 1 -- 172.21.15.100:0/4220320402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3f4073a80 msgr2=0x7fb3f41f3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:11.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.205+0000 7fb3f2800640 1 --2- 172.21.15.100:0/4220320402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3f4073a80 0x7fb3f41f3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:11.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.205+0000 7fb3f2800640 1 -- 172.21.15.100:0/4220320402 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f4136690 con 0x7fb3f4145a40 2024-09-17T04:30:11.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.206+0000 7fb3f2800640 1 --2- 172.21.15.100:0/4220320402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3f4145a40 0x7fb3f4153200 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00c730 tx=0x7fb3ec00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:11.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.206+0000 7fb3eaa00640 1 -- 172.21.15.100:0/4220320402 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec003b70 con 0x7fb3f4145a40 2024-09-17T04:30:11.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.206+0000 7fb3eaa00640 1 -- 172.21.15.100:0/4220320402 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3ec003d10 con 0x7fb3f4145a40 2024-09-17T04:30:11.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.206+0000 7fb3eaa00640 1 -- 172.21.15.100:0/4220320402 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec005330 con 0x7fb3f4145a40 2024-09-17T04:30:11.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.206+0000 7fb3f9000640 1 -- 172.21.15.100:0/4220320402 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f4136970 con 0x7fb3f4145a40 2024-09-17T04:30:11.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.206+0000 7fb3f9000640 1 -- 172.21.15.100:0/4220320402 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f4136f30 con 0x7fb3f4145a40 2024-09-17T04:30:11.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.207+0000 7fb3f9000640 1 -- 172.21.15.100:0/4220320402 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3f41610d0 con 0x7fb3f4145a40 2024-09-17T04:30:11.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.209+0000 7fb3eaa00640 1 -- 172.21.15.100:0/4220320402 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fb3ec0056e0 con 0x7fb3f4145a40 2024-09-17T04:30:11.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.209+0000 7fb3eaa00640 1 --2- 172.21.15.100:0/4220320402 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb3d0078cb0 0x7fb3d007b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:11.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.210+0000 7fb3eaa00640 1 -- 172.21.15.100:0/4220320402 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7fb3ec09ad50 con 0x7fb3f4145a40 2024-09-17T04:30:11.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.210+0000 7fb3f1e00640 1 --2- 172.21.15.100:0/4220320402 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb3d0078cb0 0x7fb3d007b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:11.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.210+0000 7fb3f1e00640 1 --2- 172.21.15.100:0/4220320402 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb3d0078cb0 0x7fb3d007b1c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4005fd0 tx=0x7fb3e400b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:11.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.214+0000 7fb3eaa00640 1 -- 172.21.15.100:0/4220320402 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb3ec09e030 con 0x7fb3f4145a40 2024-09-17T04:30:11.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.396+0000 7fb3f9000640 1 -- 172.21.15.100:0/4220320402 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb3f4140d20 con 0x7fb3f4145a40 2024-09-17T04:30:11.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.397+0000 7fb3eaa00640 1 -- 172.21.15.100:0/4220320402 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v42)=0 v42) ==== 122+0+0 (secure 0 0 0) 0x7fb3f4073e90 con 0x7fb3f4145a40 2024-09-17T04:30:11.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.399+0000 7fb3f9000640 1 -- 172.21.15.100:0/4220320402 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb3d0078cb0 msgr2=0x7fb3d007b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:11.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.399+0000 7fb3f9000640 1 --2- 172.21.15.100:0/4220320402 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb3d0078cb0 0x7fb3d007b1c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4005fd0 tx=0x7fb3e400b2e0 comp rx=0 tx=0).stop 2024-09-17T04:30:11.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.399+0000 7fb3f9000640 1 -- 172.21.15.100:0/4220320402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3f4145a40 msgr2=0x7fb3f4153200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:11.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.399+0000 7fb3f9000640 1 --2- 172.21.15.100:0/4220320402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3f4145a40 0x7fb3f4153200 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00c730 tx=0x7fb3ec00cbf0 comp rx=0 tx=0).stop 2024-09-17T04:30:11.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.399+0000 7fb3f9000640 1 -- 172.21.15.100:0/4220320402 shutdown_connections 2024-09-17T04:30:11.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.399+0000 7fb3f9000640 1 --2- 172.21.15.100:0/4220320402 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb3d0078cb0 0x7fb3d007b1c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:11.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.399+0000 7fb3f9000640 1 --2- 172.21.15.100:0/4220320402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3f4145a40 0x7fb3f4153200 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:11.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.399+0000 7fb3f9000640 1 --2- 172.21.15.100:0/4220320402 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3f4145550 0x7fb3f4152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:11.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.399+0000 7fb3f9000640 1 --2- 172.21.15.100:0/4220320402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3f4073a80 0x7fb3f41f3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:11.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.399+0000 7fb3f9000640 1 -- 172.21.15.100:0/4220320402 >> 172.21.15.100:0/4220320402 conn(0x7fb3f406d1c0 msgr2=0x7fb3f4143090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:11.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.399+0000 7fb3f9000640 1 -- 172.21.15.100:0/4220320402 shutdown_connections 2024-09-17T04:30:11.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:11.399+0000 7fb3f9000640 1 -- 172.21.15.100:0/4220320402 wait complete. 2024-09-17T04:30:11.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:11.422 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4) 2024-09-17T04:30:11.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:11.490 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/fake) 2024-09-17T04:30:11.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:11.574 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-17T04:30:11.610 INFO:teuthology.orchestra.run.smithi100.stdout:cephfs_data 2024-09-17T04:30:11.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:11.610 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4) 2024-09-17T04:30:11.712 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:30:11.712 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol80190/.meta 2024-09-17T04:30:11.772 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol80190 2024-09-17T04:30:12.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.036+0000 7f973d800640 1 -- 172.21.15.100:0/2334894236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f971c004d40 msgr2=0x7f971c0051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:12.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.036+0000 7f973d800640 1 --2- 172.21.15.100:0/2334894236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f971c004d40 0x7f971c0051f0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f9730009ed0 tx=0x7f971c005830 comp rx=0 tx=0).stop 2024-09-17T04:30:12.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.036+0000 7f973ce00640 1 -- 172.21.15.100:0/2334894236 reap_dead start 2024-09-17T04:30:12.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.036+0000 7f973d800640 1 -- 172.21.15.100:0/2334894236 shutdown_connections 2024-09-17T04:30:12.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.036+0000 7f973d800640 1 -- 172.21.15.100:0/2334894236 >> 172.21.15.100:0/2334894236 conn(0x7f973806d620 msgr2=0x7f97380b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:12.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.036+0000 7f973d800640 1 -- 172.21.15.100:0/2334894236 shutdown_connections 2024-09-17T04:30:12.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.036+0000 7f973d800640 1 -- 172.21.15.100:0/2334894236 wait complete. 2024-09-17T04:30:12.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.038+0000 7f973d800640 1 Processor -- start 2024-09-17T04:30:12.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.039+0000 7f973d800640 1 -- start start 2024-09-17T04:30:12.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.040+0000 7f973d800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f973809b300 0x7f97381f7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:12.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.040+0000 7f973d800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f973809dc60 0x7f973809bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:12.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.040+0000 7f973d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f973809c490 0x7f9738144010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:12.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.040+0000 7f973d800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f97380af0d0 con 0x7f973809c490 2024-09-17T04:30:12.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.040+0000 7f973d800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f97380cd490 con 0x7f973809dc60 2024-09-17T04:30:12.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.040+0000 7f973d800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f97380af250 con 0x7f973809b300 2024-09-17T04:30:12.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.041+0000 7f973ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f973809c490 0x7f9738144010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:12.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.041+0000 7f9737e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f973809b300 0x7f97381f7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:12.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.041+0000 7f9737400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f973809dc60 0x7f973809bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:12.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.041+0000 7f9737e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f973809b300 0x7f97381f7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.100:48840/0 (socket says 172.21.15.100:48840) 2024-09-17T04:30:12.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.041+0000 7f9737e00640 1 -- 172.21.15.100:0/354368653 learned_addr learned my addr 172.21.15.100:0/354368653 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:12.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.041+0000 7f973ce00640 1 -- 172.21.15.100:0/354368653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f973809b300 msgr2=0x7f97381f7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:12.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.041+0000 7f973ce00640 1 --2- 172.21.15.100:0/354368653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f973809b300 0x7f97381f7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:12.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.041+0000 7f973ce00640 1 -- 172.21.15.100:0/354368653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f973809dc60 msgr2=0x7f973809bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:12.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.041+0000 7f973ce00640 1 --2- 172.21.15.100:0/354368653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f973809dc60 0x7f973809bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:12.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.041+0000 7f973ce00640 1 -- 172.21.15.100:0/354368653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f973809a260 con 0x7f973809c490 2024-09-17T04:30:12.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.041+0000 7f973ce00640 1 --2- 172.21.15.100:0/354368653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f973809c490 0x7f9738144010 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f9730032000 tx=0x7f9730033c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:12.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.042+0000 7f972fe00640 1 -- 172.21.15.100:0/354368653 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9730043070 con 0x7f973809c490 2024-09-17T04:30:12.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.042+0000 7f972fe00640 1 -- 172.21.15.100:0/354368653 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9730033d20 con 0x7f973809c490 2024-09-17T04:30:12.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.042+0000 7f973d800640 1 -- 172.21.15.100:0/354368653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f973809a540 con 0x7f973809c490 2024-09-17T04:30:12.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.042+0000 7f972fe00640 1 -- 172.21.15.100:0/354368653 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9730038040 con 0x7f973809c490 2024-09-17T04:30:12.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.042+0000 7f973d800640 1 -- 172.21.15.100:0/354368653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f973809ab00 con 0x7f973809c490 2024-09-17T04:30:12.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.043+0000 7f973d800640 1 -- 172.21.15.100:0/354368653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97380af750 con 0x7f973809c490 2024-09-17T04:30:12.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.044+0000 7f972fe00640 1 -- 172.21.15.100:0/354368653 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f9730003bd0 con 0x7f973809c490 2024-09-17T04:30:12.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.045+0000 7f972fe00640 1 --2- 172.21.15.100:0/354368653 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9718078cb0 0x7f971807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:12.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.045+0000 7f972fe00640 1 -- 172.21.15.100:0/354368653 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f97300bae00 con 0x7f973809c490 2024-09-17T04:30:12.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.045+0000 7f9737e00640 1 --2- 172.21.15.100:0/354368653 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9718078cb0 0x7f971807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:12.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.046+0000 7f9737e00640 1 --2- 172.21.15.100:0/354368653 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9718078cb0 0x7f971807b1c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f97380b1a30 tx=0x7f9728006b70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:12.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.050+0000 7f972fe00640 1 -- 172.21.15.100:0/354368653 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f97300828f0 con 0x7f973809c490 2024-09-17T04:30:12.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.257+0000 7f973d800640 1 -- 172.21.15.100:0/354368653 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol80190", "target": ["mon-mgr", ""]}) -- 0x7f97380d0be0 con 0x7f9718078cb0 2024-09-17T04:30:12.264 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:12.263+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:12.265 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:12.263+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:12.265 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:12.263+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:12.265 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:12.263+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:12.265 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:12.263+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:12.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.306+0000 7f972fe00640 1 -- 172.21.15.100:0/354368653 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f97380d0be0 con 0x7f9718078cb0 2024-09-17T04:30:12.307 INFO:teuthology.orchestra.run.smithi100.stdout:/volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4 2024-09-17T04:30:12.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.309+0000 7f973d800640 1 -- 172.21.15.100:0/354368653 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9718078cb0 msgr2=0x7f971807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:12.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.309+0000 7f973d800640 1 --2- 172.21.15.100:0/354368653 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9718078cb0 0x7f971807b1c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f97380b1a30 tx=0x7f9728006b70 comp rx=0 tx=0).stop 2024-09-17T04:30:12.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.309+0000 7f973d800640 1 -- 172.21.15.100:0/354368653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f973809c490 msgr2=0x7f9738144010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:12.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.309+0000 7f973d800640 1 --2- 172.21.15.100:0/354368653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f973809c490 0x7f9738144010 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f9730032000 tx=0x7f9730033c30 comp rx=0 tx=0).stop 2024-09-17T04:30:12.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.309+0000 7f973d800640 1 -- 172.21.15.100:0/354368653 shutdown_connections 2024-09-17T04:30:12.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.309+0000 7f973d800640 1 --2- 172.21.15.100:0/354368653 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9718078cb0 0x7f971807b1c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:12.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.309+0000 7f973d800640 1 --2- 172.21.15.100:0/354368653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f973809c490 0x7f9738144010 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:12.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.309+0000 7f973d800640 1 --2- 172.21.15.100:0/354368653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f973809dc60 0x7f973809bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:12.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.309+0000 7f973d800640 1 --2- 172.21.15.100:0/354368653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f973809b300 0x7f97381f7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:12.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.309+0000 7f973d800640 1 -- 172.21.15.100:0/354368653 >> 172.21.15.100:0/354368653 conn(0x7f973806d620 msgr2=0x7f97380cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:12.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.309+0000 7f973d800640 1 -- 172.21.15.100:0/354368653 shutdown_connections 2024-09-17T04:30:12.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.309+0000 7f973d800640 1 -- 172.21.15.100:0/354368653 wait complete. 2024-09-17T04:30:12.333 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2024-09-17T04:30:12.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.558+0000 7f8ad7200640 1 -- 172.21.15.100:0/2527761162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8abc003c10 msgr2=0x7f8abc004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:12.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.558+0000 7f8ad7200640 1 --2- 172.21.15.100:0/2527761162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8abc003c10 0x7f8abc004060 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8001830 tx=0x7f8abc005830 comp rx=0 tx=0).stop 2024-09-17T04:30:12.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.558+0000 7f8ad6800640 1 -- 172.21.15.100:0/2527761162 reap_dead start 2024-09-17T04:30:12.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.558+0000 7f8ad7200640 1 -- 172.21.15.100:0/2527761162 shutdown_connections 2024-09-17T04:30:12.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.558+0000 7f8ad7200640 1 -- 172.21.15.100:0/2527761162 >> 172.21.15.100:0/2527761162 conn(0x7f8ad806d620 msgr2=0x7f8ad80b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:12.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.558+0000 7f8ad7200640 1 -- 172.21.15.100:0/2527761162 shutdown_connections 2024-09-17T04:30:12.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.558+0000 7f8ad7200640 1 -- 172.21.15.100:0/2527761162 wait complete. 2024-09-17T04:30:12.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.560+0000 7f8ad7200640 1 Processor -- start 2024-09-17T04:30:12.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.560+0000 7f8ad7200640 1 -- start start 2024-09-17T04:30:12.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.562+0000 7f8ad7200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8abc003c10 0x7f8ad80d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:12.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.562+0000 7f8ad7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ad81f5450 0x7f8ad81525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:12.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.562+0000 7f8ad7200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ad8152b30 0x7f8ad81f9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:12.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.562+0000 7f8ad7200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad80703b0 con 0x7f8ad81f5450 2024-09-17T04:30:12.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.562+0000 7f8ad7200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad81571f0 con 0x7f8abc003c10 2024-09-17T04:30:12.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.562+0000 7f8ad7200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8ad80ccac0 con 0x7f8ad8152b30 2024-09-17T04:30:12.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.562+0000 7f8acd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ad81f5450 0x7f8ad81525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:12.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.562+0000 7f8acd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ad81f5450 0x7f8ad81525f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57510/0 (socket says 172.21.15.100:57510) 2024-09-17T04:30:12.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.562+0000 7f8acd400640 1 -- 172.21.15.100:0/1975494521 learned_addr learned my addr 172.21.15.100:0/1975494521 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:12.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.562+0000 7f8ad5e00640 1 --2- 172.21.15.100:0/1975494521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8abc003c10 0x7f8ad80d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:12.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.562+0000 7f8ad6800640 1 --2- 172.21.15.100:0/1975494521 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ad8152b30 0x7f8ad81f9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:12.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.562+0000 7f8acd400640 1 -- 172.21.15.100:0/1975494521 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ad8152b30 msgr2=0x7f8ad81f9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:12.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.562+0000 7f8acd400640 1 --2- 172.21.15.100:0/1975494521 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ad8152b30 0x7f8ad81f9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:12.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.562+0000 7f8acd400640 1 -- 172.21.15.100:0/1975494521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8abc003c10 msgr2=0x7f8ad80d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:12.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.562+0000 7f8acd400640 1 --2- 172.21.15.100:0/1975494521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8abc003c10 0x7f8ad80d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:12.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.562+0000 7f8acd400640 1 -- 172.21.15.100:0/1975494521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad8152f40 con 0x7f8ad81f5450 2024-09-17T04:30:12.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.562+0000 7f8ad6800640 1 --2- 172.21.15.100:0/1975494521 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ad8152b30 0x7f8ad81f9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:30:12.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.563+0000 7f8acd400640 1 --2- 172.21.15.100:0/1975494521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ad81f5450 0x7f8ad81525f0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f8ac00119b0 tx=0x7f8ac0011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:12.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.563+0000 7f8acea00640 1 -- 172.21.15.100:0/1975494521 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac000fcf0 con 0x7f8ad81f5450 2024-09-17T04:30:12.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.563+0000 7f8acea00640 1 -- 172.21.15.100:0/1975494521 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ac0005020 con 0x7f8ad81f5450 2024-09-17T04:30:12.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.563+0000 7f8ad7200640 1 -- 172.21.15.100:0/1975494521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad8146240 con 0x7f8ad81f5450 2024-09-17T04:30:12.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.563+0000 7f8acea00640 1 -- 172.21.15.100:0/1975494521 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac0005470 con 0x7f8ad81f5450 2024-09-17T04:30:12.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.563+0000 7f8ad7200640 1 -- 172.21.15.100:0/1975494521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad809db20 con 0x7f8ad81f5450 2024-09-17T04:30:12.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.564+0000 7f8ad7200640 1 -- 172.21.15.100:0/1975494521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ad815d6e0 con 0x7f8ad81f5450 2024-09-17T04:30:12.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.565+0000 7f8acea00640 1 -- 172.21.15.100:0/1975494521 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f8ac00047a0 con 0x7f8ad81f5450 2024-09-17T04:30:12.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.566+0000 7f8acea00640 1 --2- 172.21.15.100:0/1975494521 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8ab4078ce0 0x7f8ab407b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:12.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.567+0000 7f8acea00640 1 -- 172.21.15.100:0/1975494521 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f8ac009bca0 con 0x7f8ad81f5450 2024-09-17T04:30:12.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.567+0000 7f8ad5e00640 1 --2- 172.21.15.100:0/1975494521 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8ab4078ce0 0x7f8ab407b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:12.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.567+0000 7f8ad5e00640 1 --2- 172.21.15.100:0/1975494521 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8ab4078ce0 0x7f8ab407b1f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8032000 tx=0x7f8ac8001360 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:12.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.571+0000 7f8acea00640 1 -- 172.21.15.100:0/1975494521 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8ad8152f40 con 0x7f8ad81f5450 2024-09-17T04:30:12.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.777+0000 7f8ad7200640 1 -- 172.21.15.100:0/1975494521 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7f8ad80737f0 con 0x7f8ab4078ce0 2024-09-17T04:30:12.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.781+0000 7f8acea00640 1 -- 172.21.15.100:0/1975494521 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f8ad80737f0 con 0x7f8ab4078ce0 2024-09-17T04:30:12.781 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-09-17T04:30:12.781 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:30:12.781 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "subvol80190" 2024-09-17T04:30:12.781 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:30:12.782 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-09-17T04:30:12.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.783+0000 7f8ad7200640 1 -- 172.21.15.100:0/1975494521 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8ab4078ce0 msgr2=0x7f8ab407b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:12.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.783+0000 7f8ad7200640 1 --2- 172.21.15.100:0/1975494521 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8ab4078ce0 0x7f8ab407b1f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8032000 tx=0x7f8ac8001360 comp rx=0 tx=0).stop 2024-09-17T04:30:12.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.783+0000 7f8ad7200640 1 -- 172.21.15.100:0/1975494521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ad81f5450 msgr2=0x7f8ad81525f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:12.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.783+0000 7f8ad7200640 1 --2- 172.21.15.100:0/1975494521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ad81f5450 0x7f8ad81525f0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f8ac00119b0 tx=0x7f8ac0011e70 comp rx=0 tx=0).stop 2024-09-17T04:30:12.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.783+0000 7f8ad7200640 1 -- 172.21.15.100:0/1975494521 shutdown_connections 2024-09-17T04:30:12.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.783+0000 7f8ad7200640 1 --2- 172.21.15.100:0/1975494521 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8ab4078ce0 0x7f8ab407b1f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:12.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.783+0000 7f8ad7200640 1 --2- 172.21.15.100:0/1975494521 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ad8152b30 0x7f8ad81f9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:12.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.783+0000 7f8ad7200640 1 --2- 172.21.15.100:0/1975494521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ad81f5450 0x7f8ad81525f0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:12.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.783+0000 7f8ad7200640 1 --2- 172.21.15.100:0/1975494521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8abc003c10 0x7f8ad80d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:12.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.783+0000 7f8ad7200640 1 -- 172.21.15.100:0/1975494521 >> 172.21.15.100:0/1975494521 conn(0x7f8ad806d620 msgr2=0x7f8ad8068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:12.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.784+0000 7f8ad7200640 1 -- 172.21.15.100:0/1975494521 shutdown_connections 2024-09-17T04:30:12.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:12.784+0000 7f8ad7200640 1 -- 172.21.15.100:0/1975494521 wait complete. 2024-09-17T04:30:12.806 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol80190 2024-09-17T04:30:13.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.030+0000 7f0040c00640 1 -- 172.21.15.100:0/1091346916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f003c073a80 msgr2=0x7f003c0cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:13.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.030+0000 7f0040c00640 1 --2- 172.21.15.100:0/1091346916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f003c073a80 0x7f003c0cd610 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f0034005fd0 tx=0x7f003402ab20 comp rx=0 tx=0).stop 2024-09-17T04:30:13.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.030+0000 7f0040c00640 1 -- 172.21.15.100:0/1091346916 shutdown_connections 2024-09-17T04:30:13.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.030+0000 7f0040c00640 1 --2- 172.21.15.100:0/1091346916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f003c073a80 0x7f003c0cd610 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:13.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.030+0000 7f0040c00640 1 --2- 172.21.15.100:0/1091346916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f003c145a40 0x7f003c073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:13.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.030+0000 7f0040c00640 1 -- 172.21.15.100:0/1091346916 >> 172.21.15.100:0/1091346916 conn(0x7f003c06d1c0 msgr2=0x7f003c06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:13.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.030+0000 7f0040c00640 1 -- 172.21.15.100:0/1091346916 shutdown_connections 2024-09-17T04:30:13.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.030+0000 7f0040c00640 1 -- 172.21.15.100:0/1091346916 wait complete. 2024-09-17T04:30:13.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.032+0000 7f0040c00640 1 Processor -- start 2024-09-17T04:30:13.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.032+0000 7f0040c00640 1 -- start start 2024-09-17T04:30:13.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.034+0000 7f0040c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f003c073a80 0x7f003c15d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:13.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.034+0000 7f0040c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f003c145550 0x7f003c154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:13.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.034+0000 7f0040c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f003c145a40 0x7f003c0b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:13.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.034+0000 7f0040c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f003c0ce210 con 0x7f003c145a40 2024-09-17T04:30:13.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.034+0000 7f0040c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f003c0ce090 con 0x7f003c073a80 2024-09-17T04:30:13.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.034+0000 7f0040c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f003c0ce390 con 0x7f003c145550 2024-09-17T04:30:13.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.034+0000 7f003a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f003c145a40 0x7f003c0b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:13.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.034+0000 7f003a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f003c145a40 0x7f003c0b22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57528/0 (socket says 172.21.15.100:57528) 2024-09-17T04:30:13.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.034+0000 7f003a800640 1 -- 172.21.15.100:0/3511503187 learned_addr learned my addr 172.21.15.100:0/3511503187 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:13.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.034+0000 7f0039e00640 1 --2- 172.21.15.100:0/3511503187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f003c073a80 0x7f003c15d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:13.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.034+0000 7f0039400640 1 --2- 172.21.15.100:0/3511503187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f003c145550 0x7f003c154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:13.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.034+0000 7f003a800640 1 -- 172.21.15.100:0/3511503187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f003c145550 msgr2=0x7f003c154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:13.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.034+0000 7f003a800640 1 --2- 172.21.15.100:0/3511503187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f003c145550 0x7f003c154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:13.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.034+0000 7f003a800640 1 -- 172.21.15.100:0/3511503187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f003c073a80 msgr2=0x7f003c15d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:13.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.034+0000 7f003a800640 1 --2- 172.21.15.100:0/3511503187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f003c073a80 0x7f003c15d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:13.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.034+0000 7f003a800640 1 -- 172.21.15.100:0/3511503187 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f003c0aeaf0 con 0x7f003c145a40 2024-09-17T04:30:13.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.034+0000 7f0039400640 1 --2- 172.21.15.100:0/3511503187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f003c145550 0x7f003c154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:30:13.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.035+0000 7f003a800640 1 --2- 172.21.15.100:0/3511503187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f003c145a40 0x7f003c0b22c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f0034006010 tx=0x7f0034003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:13.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.035+0000 7f0032000640 1 -- 172.21.15.100:0/3511503187 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034044070 con 0x7f003c145a40 2024-09-17T04:30:13.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.035+0000 7f0032000640 1 -- 172.21.15.100:0/3511503187 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f003402c820 con 0x7f003c145a40 2024-09-17T04:30:13.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.035+0000 7f0040c00640 1 -- 172.21.15.100:0/3511503187 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f003c1f5e00 con 0x7f003c145a40 2024-09-17T04:30:13.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.035+0000 7f0032000640 1 -- 172.21.15.100:0/3511503187 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00340044b0 con 0x7f003c145a40 2024-09-17T04:30:13.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.035+0000 7f0040c00640 1 -- 172.21.15.100:0/3511503187 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f003c1f63c0 con 0x7f003c145a40 2024-09-17T04:30:13.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.036+0000 7f0040c00640 1 -- 172.21.15.100:0/3511503187 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f003c0b35d0 con 0x7f003c145a40 2024-09-17T04:30:13.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.038+0000 7f0032000640 1 -- 172.21.15.100:0/3511503187 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f0034004650 con 0x7f003c145a40 2024-09-17T04:30:13.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.039+0000 7f0032000640 1 --2- 172.21.15.100:0/3511503187 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f001c078cb0 0x7f001c07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:13.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.039+0000 7f0032000640 1 -- 172.21.15.100:0/3511503187 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f00340bb860 con 0x7f003c145a40 2024-09-17T04:30:13.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.039+0000 7f0039e00640 1 --2- 172.21.15.100:0/3511503187 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f001c078cb0 0x7f001c07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:13.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.040+0000 7f0039e00640 1 --2- 172.21.15.100:0/3511503187 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f001c078cb0 0x7f001c07b1c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f002c001720 tx=0x7f002c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:13.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.044+0000 7f0032000640 1 -- 172.21.15.100:0/3511503187 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0034032030 con 0x7f003c145a40 2024-09-17T04:30:13.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.257+0000 7f0040c00640 1 -- 172.21.15.100:0/3511503187 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol80190", "target": ["mon-mgr", ""]}) -- 0x7f003c1fa0f0 con 0x7f001c078cb0 2024-09-17T04:30:13.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.284+0000 7f0032000640 1 -- 172.21.15.100:0/3511503187 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+687 (secure 0 0 0) 0x7f003c1fa0f0 con 0x7f001c078cb0 2024-09-17T04:30:13.285 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:30:13.285 INFO:teuthology.orchestra.run.smithi100.stdout: "atime": "2024-09-17 04:30:11", 2024-09-17T04:30:13.285 INFO:teuthology.orchestra.run.smithi100.stdout: "bytes_pcent": "undefined", 2024-09-17T04:30:13.285 INFO:teuthology.orchestra.run.smithi100.stdout: "bytes_quota": "infinite", 2024-09-17T04:30:13.285 INFO:teuthology.orchestra.run.smithi100.stdout: "bytes_used": 0, 2024-09-17T04:30:13.285 INFO:teuthology.orchestra.run.smithi100.stdout: "created_at": "2024-09-17 04:30:11", 2024-09-17T04:30:13.285 INFO:teuthology.orchestra.run.smithi100.stdout: "ctime": "2024-09-17 04:30:12", 2024-09-17T04:30:13.285 INFO:teuthology.orchestra.run.smithi100.stdout: "data_pool": "cephfs_data", 2024-09-17T04:30:13.285 INFO:teuthology.orchestra.run.smithi100.stdout: "features": [ 2024-09-17T04:30:13.285 INFO:teuthology.orchestra.run.smithi100.stdout: "snapshot-clone", 2024-09-17T04:30:13.286 INFO:teuthology.orchestra.run.smithi100.stdout: "snapshot-autoprotect" 2024-09-17T04:30:13.286 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:30:13.286 INFO:teuthology.orchestra.run.smithi100.stdout: "flavor": 1, 2024-09-17T04:30:13.286 INFO:teuthology.orchestra.run.smithi100.stdout: "gid": 0, 2024-09-17T04:30:13.286 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": 16877, 2024-09-17T04:30:13.286 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_addrs": [ 2024-09-17T04:30:13.286 INFO:teuthology.orchestra.run.smithi100.stdout: "172.21.15.100:6789", 2024-09-17T04:30:13.286 INFO:teuthology.orchestra.run.smithi100.stdout: "172.21.15.184:6789", 2024-09-17T04:30:13.286 INFO:teuthology.orchestra.run.smithi100.stdout: "172.21.15.184:6790" 2024-09-17T04:30:13.286 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:30:13.286 INFO:teuthology.orchestra.run.smithi100.stdout: "mtime": "2024-09-17 04:30:11", 2024-09-17T04:30:13.286 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4", 2024-09-17T04:30:13.286 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_namespace": "", 2024-09-17T04:30:13.287 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "complete", 2024-09-17T04:30:13.287 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "subvolume", 2024-09-17T04:30:13.287 INFO:teuthology.orchestra.run.smithi100.stdout: "uid": 0 2024-09-17T04:30:13.287 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:30:13.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.287+0000 7f0040c00640 1 -- 172.21.15.100:0/3511503187 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f001c078cb0 msgr2=0x7f001c07b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:13.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.287+0000 7f0040c00640 1 --2- 172.21.15.100:0/3511503187 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f001c078cb0 0x7f001c07b1c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f002c001720 tx=0x7f002c009040 comp rx=0 tx=0).stop 2024-09-17T04:30:13.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.287+0000 7f0040c00640 1 -- 172.21.15.100:0/3511503187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f003c145a40 msgr2=0x7f003c0b22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:13.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.287+0000 7f0040c00640 1 --2- 172.21.15.100:0/3511503187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f003c145a40 0x7f003c0b22c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f0034006010 tx=0x7f0034003f10 comp rx=0 tx=0).stop 2024-09-17T04:30:13.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.287+0000 7f0040c00640 1 -- 172.21.15.100:0/3511503187 shutdown_connections 2024-09-17T04:30:13.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.287+0000 7f0040c00640 1 --2- 172.21.15.100:0/3511503187 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f001c078cb0 0x7f001c07b1c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:13.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.287+0000 7f0040c00640 1 --2- 172.21.15.100:0/3511503187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f003c145a40 0x7f003c0b22c0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:13.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.287+0000 7f0040c00640 1 --2- 172.21.15.100:0/3511503187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f003c145550 0x7f003c154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:13.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.287+0000 7f0040c00640 1 --2- 172.21.15.100:0/3511503187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f003c073a80 0x7f003c15d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:13.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.287+0000 7f0040c00640 1 -- 172.21.15.100:0/3511503187 >> 172.21.15.100:0/3511503187 conn(0x7f003c06d1c0 msgr2=0x7f003c1440a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:13.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.287+0000 7f0040c00640 1 -- 172.21.15.100:0/3511503187 shutdown_connections 2024-09-17T04:30:13.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.287+0000 7f0040c00640 1 -- 172.21.15.100:0/3511503187 wait complete. 2024-09-17T04:30:13.310 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize cephfs subvol80190 10485760 2024-09-17T04:30:13.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.539+0000 7fe99d200640 1 -- 172.21.15.100:0/3614076065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9980b59b0 msgr2=0x7fe9980d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:13.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.539+0000 7fe99d200640 1 --2- 172.21.15.100:0/3614076065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9980b59b0 0x7fe9980d29f0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fe988005fd0 tx=0x7fe98802a540 comp rx=0 tx=0).stop 2024-09-17T04:30:13.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.539+0000 7fe99d200640 1 -- 172.21.15.100:0/3614076065 shutdown_connections 2024-09-17T04:30:13.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.539+0000 7fe99d200640 1 --2- 172.21.15.100:0/3614076065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe998163400 0x7fe9980b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:13.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.539+0000 7fe99d200640 1 --2- 172.21.15.100:0/3614076065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9980b59b0 0x7fe9980d29f0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:13.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.539+0000 7fe99d200640 1 -- 172.21.15.100:0/3614076065 >> 172.21.15.100:0/3614076065 conn(0x7fe99808bfe0 msgr2=0x7fe9980d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:13.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.539+0000 7fe99d200640 1 -- 172.21.15.100:0/3614076065 shutdown_connections 2024-09-17T04:30:13.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.539+0000 7fe99d200640 1 -- 172.21.15.100:0/3614076065 wait complete. 2024-09-17T04:30:13.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.541+0000 7fe99d200640 1 Processor -- start 2024-09-17T04:30:13.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.541+0000 7fe99d200640 1 -- start start 2024-09-17T04:30:13.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.543+0000 7fe99d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9980b59b0 0x7fe99815e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:13.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.543+0000 7fe99d200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9980d2f30 0x7fe9980cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:13.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.543+0000 7fe99d200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe998163400 0x7fe9980cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:13.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.543+0000 7fe99d200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe9980b41e0 con 0x7fe9980b59b0 2024-09-17T04:30:13.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.543+0000 7fe99d200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe998163a20 con 0x7fe9980d2f30 2024-09-17T04:30:13.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.543+0000 7fe99d200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe9980b4360 con 0x7fe998163400 2024-09-17T04:30:13.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.543+0000 7fe997400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9980b59b0 0x7fe99815e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:13.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.543+0000 7fe997400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9980b59b0 0x7fe99815e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57544/0 (socket says 172.21.15.100:57544) 2024-09-17T04:30:13.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.543+0000 7fe997400640 1 -- 172.21.15.100:0/2668131604 learned_addr learned my addr 172.21.15.100:0/2668131604 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:13.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.543+0000 7fe98ea00640 1 --2- 172.21.15.100:0/2668131604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9980d2f30 0x7fe9980cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:13.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.543+0000 7fe997e00640 1 --2- 172.21.15.100:0/2668131604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe998163400 0x7fe9980cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:13.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.544+0000 7fe997400640 1 -- 172.21.15.100:0/2668131604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe998163400 msgr2=0x7fe9980cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:13.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.544+0000 7fe997400640 1 --2- 172.21.15.100:0/2668131604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe998163400 0x7fe9980cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:13.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.544+0000 7fe997400640 1 -- 172.21.15.100:0/2668131604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9980d2f30 msgr2=0x7fe9980cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:13.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.544+0000 7fe997400640 1 --2- 172.21.15.100:0/2668131604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9980d2f30 0x7fe9980cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:13.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.544+0000 7fe997400640 1 -- 172.21.15.100:0/2668131604 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe998162220 con 0x7fe9980b59b0 2024-09-17T04:30:13.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.544+0000 7fe997e00640 1 --2- 172.21.15.100:0/2668131604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe998163400 0x7fe9980cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:30:13.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.544+0000 7fe997400640 1 --2- 172.21.15.100:0/2668131604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9980b59b0 0x7fe99815e2b0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fe988006010 tx=0x7fe9880038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:13.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.544+0000 7fe98fe00640 1 -- 172.21.15.100:0/2668131604 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98803e030 con 0x7fe9980b59b0 2024-09-17T04:30:13.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.544+0000 7fe98fe00640 1 -- 172.21.15.100:0/2668131604 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe988003d90 con 0x7fe9980b59b0 2024-09-17T04:30:13.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.544+0000 7fe99d200640 1 -- 172.21.15.100:0/2668131604 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9981f8dc0 con 0x7fe9980b59b0 2024-09-17T04:30:13.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.544+0000 7fe98fe00640 1 -- 172.21.15.100:0/2668131604 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988035040 con 0x7fe9980b59b0 2024-09-17T04:30:13.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.544+0000 7fe99d200640 1 -- 172.21.15.100:0/2668131604 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe998162500 con 0x7fe9980b59b0 2024-09-17T04:30:13.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.546+0000 7fe99d200640 1 -- 172.21.15.100:0/2668131604 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9980d29f0 con 0x7fe9980b59b0 2024-09-17T04:30:13.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.547+0000 7fe98fe00640 1 -- 172.21.15.100:0/2668131604 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 101594+0+0 (secure 0 0 0) 0x7fe988041030 con 0x7fe9980b59b0 2024-09-17T04:30:13.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.548+0000 7fe98fe00640 1 --2- 172.21.15.100:0/2668131604 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe974078d70 0x7fe97407b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:13.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.548+0000 7fe98fe00640 1 -- 172.21.15.100:0/2668131604 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7fe9880bb830 con 0x7fe9980b59b0 2024-09-17T04:30:13.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.548+0000 7fe98ea00640 1 --2- 172.21.15.100:0/2668131604 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe974078d70 0x7fe97407b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:13.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.549+0000 7fe98ea00640 1 --2- 172.21.15.100:0/2668131604 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe974078d70 0x7fe97407b280 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe9981f8c80 tx=0x7fe980009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:13.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.553+0000 7fe98fe00640 1 -- 172.21.15.100:0/2668131604 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe988083240 con 0x7fe9980b59b0 2024-09-17T04:30:13.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.767+0000 7fe99d200640 1 -- 172.21.15.100:0/2668131604 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "cephfs", "sub_name": "subvol80190", "new_size": "10485760", "target": ["mon-mgr", ""]}) -- 0x7fe9980cd240 con 0x7fe974078d70 2024-09-17T04:30:13.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.795+0000 7fe98fe00640 1 -- 172.21.15.100:0/2668131604 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+128 (secure 0 0 0) 0x7fe9980cd240 con 0x7fe974078d70 2024-09-17T04:30:13.796 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-09-17T04:30:13.796 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:30:13.796 INFO:teuthology.orchestra.run.smithi100.stdout: "bytes_used": 0 2024-09-17T04:30:13.796 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:30:13.796 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:30:13.796 INFO:teuthology.orchestra.run.smithi100.stdout: "bytes_quota": 10485760 2024-09-17T04:30:13.796 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:30:13.797 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:30:13.797 INFO:teuthology.orchestra.run.smithi100.stdout: "bytes_pcent": "0.00" 2024-09-17T04:30:13.797 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:30:13.797 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-09-17T04:30:13.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.798+0000 7fe99d200640 1 -- 172.21.15.100:0/2668131604 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe974078d70 msgr2=0x7fe97407b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:13.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.798+0000 7fe99d200640 1 --2- 172.21.15.100:0/2668131604 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe974078d70 0x7fe97407b280 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe9981f8c80 tx=0x7fe980009040 comp rx=0 tx=0).stop 2024-09-17T04:30:13.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.798+0000 7fe99d200640 1 -- 172.21.15.100:0/2668131604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9980b59b0 msgr2=0x7fe99815e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:13.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.798+0000 7fe99d200640 1 --2- 172.21.15.100:0/2668131604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9980b59b0 0x7fe99815e2b0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fe988006010 tx=0x7fe9880038a0 comp rx=0 tx=0).stop 2024-09-17T04:30:13.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.798+0000 7fe99d200640 1 -- 172.21.15.100:0/2668131604 shutdown_connections 2024-09-17T04:30:13.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.798+0000 7fe99d200640 1 --2- 172.21.15.100:0/2668131604 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe974078d70 0x7fe97407b280 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:13.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.798+0000 7fe99d200640 1 --2- 172.21.15.100:0/2668131604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe998163400 0x7fe9980cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:13.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.798+0000 7fe99d200640 1 --2- 172.21.15.100:0/2668131604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9980d2f30 0x7fe9980cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:13.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.798+0000 7fe99d200640 1 --2- 172.21.15.100:0/2668131604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9980b59b0 0x7fe99815e2b0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:13.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.798+0000 7fe99d200640 1 -- 172.21.15.100:0/2668131604 >> 172.21.15.100:0/2668131604 conn(0x7fe99808bfe0 msgr2=0x7fe9980b1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:13.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.798+0000 7fe99d200640 1 -- 172.21.15.100:0/2668131604 shutdown_connections 2024-09-17T04:30:13.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:13.798+0000 7fe99d200640 1 -- 172.21.15.100:0/2668131604 wait complete. 2024-09-17T04:30:13.824 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol80190 2024-09-17T04:30:14.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.046+0000 7f9ce0c00640 1 -- 172.21.15.100:0/1125252586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cdc070160 msgr2=0x7f9cdc0af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:14.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.046+0000 7f9ce0c00640 1 --2- 172.21.15.100:0/1125252586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cdc070160 0x7f9cdc0af0d0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4005fd0 tx=0x7f9cd402ab40 comp rx=0 tx=0).stop 2024-09-17T04:30:14.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.046+0000 7f9ce0c00640 1 -- 172.21.15.100:0/1125252586 shutdown_connections 2024-09-17T04:30:14.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.046+0000 7f9ce0c00640 1 --2- 172.21.15.100:0/1125252586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cdc070160 0x7f9cdc0af0d0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:14.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.046+0000 7f9ce0c00640 1 -- 172.21.15.100:0/1125252586 >> 172.21.15.100:0/1125252586 conn(0x7f9cdc06d620 msgr2=0x7f9cdc0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:14.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.047+0000 7f9ce0c00640 1 -- 172.21.15.100:0/1125252586 shutdown_connections 2024-09-17T04:30:14.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.047+0000 7f9ce0c00640 1 -- 172.21.15.100:0/1125252586 wait complete. 2024-09-17T04:30:14.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.049+0000 7f9ce0c00640 1 Processor -- start 2024-09-17T04:30:14.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.049+0000 7f9ce0c00640 1 -- start start 2024-09-17T04:30:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.050+0000 7f9ce0c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cdc070160 0x7f9cdc099910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.050+0000 7f9ce0c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cdc0b3f30 0x7f9cdc1f7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.050+0000 7f9ce0c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9cdc156ca0 0x7f9cdc1f7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.050+0000 7f9ce0c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9cdc0af790 con 0x7f9cdc070160 2024-09-17T04:30:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.050+0000 7f9ce0c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9cdc0af610 con 0x7f9cdc156ca0 2024-09-17T04:30:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.050+0000 7f9ce0c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9cdc0af910 con 0x7f9cdc0b3f30 2024-09-17T04:30:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.050+0000 7f9cdb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cdc070160 0x7f9cdc099910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.051+0000 7f9cdb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cdc070160 0x7f9cdc099910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57572/0 (socket says 172.21.15.100:57572) 2024-09-17T04:30:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.051+0000 7f9cdb400640 1 -- 172.21.15.100:0/4235815187 learned_addr learned my addr 172.21.15.100:0/4235815187 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:14.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.051+0000 7f9cdaa00640 1 --2- 172.21.15.100:0/4235815187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cdc0b3f30 0x7f9cdc1f7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:14.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.051+0000 7f9cdbe00640 1 --2- 172.21.15.100:0/4235815187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9cdc156ca0 0x7f9cdc1f7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:14.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.051+0000 7f9cdb400640 1 -- 172.21.15.100:0/4235815187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cdc0b3f30 msgr2=0x7f9cdc1f7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:14.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.051+0000 7f9cdb400640 1 --2- 172.21.15.100:0/4235815187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cdc0b3f30 0x7f9cdc1f7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:14.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.051+0000 7f9cdb400640 1 -- 172.21.15.100:0/4235815187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9cdc156ca0 msgr2=0x7f9cdc1f7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:14.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.051+0000 7f9cdb400640 1 --2- 172.21.15.100:0/4235815187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9cdc156ca0 0x7f9cdc1f7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:14.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.051+0000 7f9cdb400640 1 -- 172.21.15.100:0/4235815187 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cdc1f41f0 con 0x7f9cdc070160 2024-09-17T04:30:14.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.051+0000 7f9cdaa00640 1 --2- 172.21.15.100:0/4235815187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cdc0b3f30 0x7f9cdc1f7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:30:14.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.051+0000 7f9cdbe00640 1 --2- 172.21.15.100:0/4235815187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9cdc156ca0 0x7f9cdc1f7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:30:14.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.051+0000 7f9cdb400640 1 --2- 172.21.15.100:0/4235815187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cdc070160 0x7f9cdc099910 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00c3d0 tx=0x7f9ccc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:14.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.052+0000 7f9cd3400640 1 -- 172.21.15.100:0/4235815187 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc017070 con 0x7f9cdc070160 2024-09-17T04:30:14.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.052+0000 7f9cd3400640 1 -- 172.21.15.100:0/4235815187 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ccc0053c0 con 0x7f9cdc070160 2024-09-17T04:30:14.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.052+0000 7f9ce0c00640 1 -- 172.21.15.100:0/4235815187 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cdc1f8540 con 0x7f9cdc070160 2024-09-17T04:30:14.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.052+0000 7f9cd3400640 1 -- 172.21.15.100:0/4235815187 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc003850 con 0x7f9cdc070160 2024-09-17T04:30:14.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.052+0000 7f9ce0c00640 1 -- 172.21.15.100:0/4235815187 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cdc09bdb0 con 0x7f9cdc070160 2024-09-17T04:30:14.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.053+0000 7f9ce0c00640 1 -- 172.21.15.100:0/4235815187 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cdc0af0d0 con 0x7f9cdc070160 2024-09-17T04:30:14.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.054+0000 7f9cd3400640 1 -- 172.21.15.100:0/4235815187 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 101594+0+0 (secure 0 0 0) 0x7f9ccc01a070 con 0x7f9cdc070160 2024-09-17T04:30:14.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.055+0000 7f9cd3400640 1 --2- 172.21.15.100:0/4235815187 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9cbc078d70 0x7f9cbc07b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:14.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.055+0000 7f9cd3400640 1 -- 172.21.15.100:0/4235815187 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f9ccc09a530 con 0x7f9cdc070160 2024-09-17T04:30:14.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.055+0000 7f9cdaa00640 1 --2- 172.21.15.100:0/4235815187 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9cbc078d70 0x7f9cbc07b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:14.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.056+0000 7f9cdaa00640 1 --2- 172.21.15.100:0/4235815187 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9cbc078d70 0x7f9cbc07b280 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc15cc30 tx=0x7f9cc8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:14.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.059+0000 7f9cd3400640 1 -- 172.21.15.100:0/4235815187 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9ccc062590 con 0x7f9cdc070160 2024-09-17T04:30:14.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.267+0000 7f9ce0c00640 1 -- 172.21.15.100:0/4235815187 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol80190", "target": ["mon-mgr", ""]}) -- 0x7f9cdc1f43c0 con 0x7f9cbc078d70 2024-09-17T04:30:14.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.293+0000 7f9cd3400640 1 -- 172.21.15.100:0/4235815187 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+680 (secure 0 0 0) 0x7f9cdc1f43c0 con 0x7f9cbc078d70 2024-09-17T04:30:14.294 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:30:14.294 INFO:teuthology.orchestra.run.smithi100.stdout: "atime": "2024-09-17 04:30:11", 2024-09-17T04:30:14.294 INFO:teuthology.orchestra.run.smithi100.stdout: "bytes_pcent": "0.00", 2024-09-17T04:30:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: "bytes_quota": 10485760, 2024-09-17T04:30:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: "bytes_used": 0, 2024-09-17T04:30:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: "created_at": "2024-09-17 04:30:11", 2024-09-17T04:30:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: "ctime": "2024-09-17 04:30:13", 2024-09-17T04:30:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: "data_pool": "cephfs_data", 2024-09-17T04:30:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: "features": [ 2024-09-17T04:30:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: "snapshot-clone", 2024-09-17T04:30:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: "snapshot-autoprotect" 2024-09-17T04:30:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:30:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: "flavor": 1, 2024-09-17T04:30:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: "gid": 0, 2024-09-17T04:30:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": 16877, 2024-09-17T04:30:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_addrs": [ 2024-09-17T04:30:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: "172.21.15.100:6789", 2024-09-17T04:30:14.296 INFO:teuthology.orchestra.run.smithi100.stdout: "172.21.15.184:6789", 2024-09-17T04:30:14.296 INFO:teuthology.orchestra.run.smithi100.stdout: "172.21.15.184:6790" 2024-09-17T04:30:14.296 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:30:14.296 INFO:teuthology.orchestra.run.smithi100.stdout: "mtime": "2024-09-17 04:30:11", 2024-09-17T04:30:14.296 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4", 2024-09-17T04:30:14.296 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_namespace": "", 2024-09-17T04:30:14.296 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "complete", 2024-09-17T04:30:14.296 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "subvolume", 2024-09-17T04:30:14.296 INFO:teuthology.orchestra.run.smithi100.stdout: "uid": 0 2024-09-17T04:30:14.296 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:30:14.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.295+0000 7f9ce0c00640 1 -- 172.21.15.100:0/4235815187 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9cbc078d70 msgr2=0x7f9cbc07b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:14.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.295+0000 7f9ce0c00640 1 --2- 172.21.15.100:0/4235815187 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9cbc078d70 0x7f9cbc07b280 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc15cc30 tx=0x7f9cc8009040 comp rx=0 tx=0).stop 2024-09-17T04:30:14.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.295+0000 7f9ce0c00640 1 -- 172.21.15.100:0/4235815187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cdc070160 msgr2=0x7f9cdc099910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:14.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.295+0000 7f9ce0c00640 1 --2- 172.21.15.100:0/4235815187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cdc070160 0x7f9cdc099910 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00c3d0 tx=0x7f9ccc00c890 comp rx=0 tx=0).stop 2024-09-17T04:30:14.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.296+0000 7f9ce0c00640 1 -- 172.21.15.100:0/4235815187 shutdown_connections 2024-09-17T04:30:14.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.296+0000 7f9ce0c00640 1 --2- 172.21.15.100:0/4235815187 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9cbc078d70 0x7f9cbc07b280 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:14.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.296+0000 7f9ce0c00640 1 --2- 172.21.15.100:0/4235815187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9cdc156ca0 0x7f9cdc1f7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:14.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.296+0000 7f9ce0c00640 1 --2- 172.21.15.100:0/4235815187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cdc0b3f30 0x7f9cdc1f7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:14.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.296+0000 7f9ce0c00640 1 --2- 172.21.15.100:0/4235815187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cdc070160 0x7f9cdc099910 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:14.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.296+0000 7f9ce0c00640 1 -- 172.21.15.100:0/4235815187 >> 172.21.15.100:0/4235815187 conn(0x7f9cdc06d620 msgr2=0x7f9cdc15f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:14.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.296+0000 7f9ce0c00640 1 -- 172.21.15.100:0/4235815187 shutdown_connections 2024-09-17T04:30:14.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.296+0000 7f9ce0c00640 1 -- 172.21.15.100:0/4235815187 wait complete. 2024-09-17T04:30:14.322 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol80190 --mode 777 --uid 1000 --gid 1000 2024-09-17T04:30:14.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.555+0000 7f06f0800640 1 -- 172.21.15.100:0/2023493557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06d00039e0 msgr2=0x7f06d0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:14.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.555+0000 7f06f0800640 1 --2- 172.21.15.100:0/2023493557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06d00039e0 0x7f06d0003e10 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f06dc00c2e0 tx=0x7f06d0005830 comp rx=0 tx=0).stop 2024-09-17T04:30:14.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.556+0000 7f06ee800640 1 -- 172.21.15.100:0/2023493557 reap_dead start 2024-09-17T04:30:14.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.556+0000 7f06f0800640 1 -- 172.21.15.100:0/2023493557 shutdown_connections 2024-09-17T04:30:14.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.556+0000 7f06f0800640 1 -- 172.21.15.100:0/2023493557 >> 172.21.15.100:0/2023493557 conn(0x7f06e80b2ff0 msgr2=0x7f06e80b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:14.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.556+0000 7f06f0800640 1 -- 172.21.15.100:0/2023493557 shutdown_connections 2024-09-17T04:30:14.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.556+0000 7f06f0800640 1 -- 172.21.15.100:0/2023493557 wait complete. 2024-09-17T04:30:14.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.558+0000 7f06f0800640 1 Processor -- start 2024-09-17T04:30:14.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.558+0000 7f06f0800640 1 -- start start 2024-09-17T04:30:14.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.559+0000 7f06f0800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06d00039e0 0x7f06e809b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:14.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.559+0000 7f06f0800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06e8154520 0x7f06e8069120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:14.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.559+0000 7f06f0800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06e80cad90 0x7f06e80cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:14.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.559+0000 7f06f0800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f06e80c7540 con 0x7f06d00039e0 2024-09-17T04:30:14.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.559+0000 7f06f0800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f06e815e0c0 con 0x7f06e8154520 2024-09-17T04:30:14.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.559+0000 7f06f0800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f06e80c76c0 con 0x7f06e80cad90 2024-09-17T04:30:14.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.560+0000 7f06ede00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06d00039e0 0x7f06e809b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:14.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.560+0000 7f06ede00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06d00039e0 0x7f06e809b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57586/0 (socket says 172.21.15.100:57586) 2024-09-17T04:30:14.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.560+0000 7f06ede00640 1 -- 172.21.15.100:0/1257025669 learned_addr learned my addr 172.21.15.100:0/1257025669 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:14.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.560+0000 7f06ed400640 1 --2- 172.21.15.100:0/1257025669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06e8154520 0x7f06e8069120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:14.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.560+0000 7f06ee800640 1 --2- 172.21.15.100:0/1257025669 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06e80cad90 0x7f06e80cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:14.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.560+0000 7f06ede00640 1 -- 172.21.15.100:0/1257025669 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06e80cad90 msgr2=0x7f06e80cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:14.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.560+0000 7f06ede00640 1 --2- 172.21.15.100:0/1257025669 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06e80cad90 0x7f06e80cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:14.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.560+0000 7f06ede00640 1 -- 172.21.15.100:0/1257025669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06e8154520 msgr2=0x7f06e8069120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:14.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.560+0000 7f06ede00640 1 --2- 172.21.15.100:0/1257025669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06e8154520 0x7f06e8069120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:14.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.560+0000 7f06ede00640 1 -- 172.21.15.100:0/1257025669 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e80c93d0 con 0x7f06d00039e0 2024-09-17T04:30:14.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.560+0000 7f06ee800640 1 --2- 172.21.15.100:0/1257025669 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06e80cad90 0x7f06e80cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:30:14.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.561+0000 7f06ede00640 1 --2- 172.21.15.100:0/1257025669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06d00039e0 0x7f06e809b350 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f06dc00c2e0 tx=0x7f06dc0060e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:14.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.561+0000 7f06e2000640 1 -- 172.21.15.100:0/1257025669 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc032030 con 0x7f06d00039e0 2024-09-17T04:30:14.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.561+0000 7f06e2000640 1 -- 172.21.15.100:0/1257025669 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06dc003b10 con 0x7f06d00039e0 2024-09-17T04:30:14.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.561+0000 7f06f0800640 1 -- 172.21.15.100:0/1257025669 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06e80af880 con 0x7f06d00039e0 2024-09-17T04:30:14.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.561+0000 7f06e2000640 1 -- 172.21.15.100:0/1257025669 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc006580 con 0x7f06d00039e0 2024-09-17T04:30:14.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.561+0000 7f06f0800640 1 -- 172.21.15.100:0/1257025669 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06e80afb20 con 0x7f06d00039e0 2024-09-17T04:30:14.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.562+0000 7f06f0800640 1 -- 172.21.15.100:0/1257025669 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06e80b3b80 con 0x7f06d00039e0 2024-09-17T04:30:14.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.563+0000 7f06e2000640 1 -- 172.21.15.100:0/1257025669 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 101594+0+0 (secure 0 0 0) 0x7f06dc035020 con 0x7f06d00039e0 2024-09-17T04:30:14.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.564+0000 7f06e2000640 1 --2- 172.21.15.100:0/1257025669 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f06cc078d40 0x7f06cc07b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:14.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.564+0000 7f06e2000640 1 -- 172.21.15.100:0/1257025669 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f06dc0bb5d0 con 0x7f06d00039e0 2024-09-17T04:30:14.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.564+0000 7f06ed400640 1 --2- 172.21.15.100:0/1257025669 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f06cc078d40 0x7f06cc07b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:14.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.565+0000 7f06ed400640 1 --2- 172.21.15.100:0/1257025669 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f06cc078d40 0x7f06cc07b250 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f06d8006b90 tx=0x7f06d8006b20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:14.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.568+0000 7f06e2000640 1 -- 172.21.15.100:0/1257025669 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f06dc047030 con 0x7f06d00039e0 2024-09-17T04:30:14.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.777+0000 7f06f0800640 1 -- 172.21.15.100:0/1257025669 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol80190", "mode": "777", "uid": 1000, "gid": 1000, "target": ["mon-mgr", ""]}) -- 0x7f06e80ccc30 con 0x7f06cc078d40 2024-09-17T04:30:14.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.805+0000 7f06e2000640 1 -- 172.21.15.100:0/1257025669 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f06e80ccc30 con 0x7f06cc078d40 2024-09-17T04:30:14.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.807+0000 7f06f0800640 1 -- 172.21.15.100:0/1257025669 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f06cc078d40 msgr2=0x7f06cc07b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:14.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.807+0000 7f06f0800640 1 --2- 172.21.15.100:0/1257025669 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f06cc078d40 0x7f06cc07b250 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f06d8006b90 tx=0x7f06d8006b20 comp rx=0 tx=0).stop 2024-09-17T04:30:14.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.807+0000 7f06f0800640 1 -- 172.21.15.100:0/1257025669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06d00039e0 msgr2=0x7f06e809b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:14.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.807+0000 7f06f0800640 1 --2- 172.21.15.100:0/1257025669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06d00039e0 0x7f06e809b350 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f06dc00c2e0 tx=0x7f06dc0060e0 comp rx=0 tx=0).stop 2024-09-17T04:30:14.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.807+0000 7f06f0800640 1 -- 172.21.15.100:0/1257025669 shutdown_connections 2024-09-17T04:30:14.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.807+0000 7f06f0800640 1 --2- 172.21.15.100:0/1257025669 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f06cc078d40 0x7f06cc07b250 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:14.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.807+0000 7f06f0800640 1 --2- 172.21.15.100:0/1257025669 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06e80cad90 0x7f06e80cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:14.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.807+0000 7f06f0800640 1 --2- 172.21.15.100:0/1257025669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06e8154520 0x7f06e8069120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:14.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.807+0000 7f06f0800640 1 --2- 172.21.15.100:0/1257025669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06d00039e0 0x7f06e809b350 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:14.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.807+0000 7f06f0800640 1 -- 172.21.15.100:0/1257025669 >> 172.21.15.100:0/1257025669 conn(0x7f06e80b2ff0 msgr2=0x7f06e8071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:14.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.807+0000 7f06f0800640 1 -- 172.21.15.100:0/1257025669 shutdown_connections 2024-09-17T04:30:14.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:14.807+0000 7f06f0800640 1 -- 172.21.15.100:0/1257025669 wait complete. 2024-09-17T04:30:14.834 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol80190 2024-09-17T04:30:15.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.060+0000 7f2690400640 1 -- 172.21.15.100:0/873879081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26880aee00 msgr2=0x7f26880af210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:15.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.060+0000 7f2690400640 1 --2- 172.21.15.100:0/873879081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26880aee00 0x7f26880af210 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f2674005fd0 tx=0x7f267402a410 comp rx=0 tx=0).stop 2024-09-17T04:30:15.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.061+0000 7f2690400640 1 -- 172.21.15.100:0/873879081 shutdown_connections 2024-09-17T04:30:15.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.061+0000 7f2690400640 1 --2- 172.21.15.100:0/873879081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26880aee00 0x7f26880af210 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:15.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.061+0000 7f2690400640 1 --2- 172.21.15.100:0/873879081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f268815f9e0 0x7f2688147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:15.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.061+0000 7f2690400640 1 -- 172.21.15.100:0/873879081 >> 172.21.15.100:0/873879081 conn(0x7f268806d1c0 msgr2=0x7f268806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:15.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.061+0000 7f2690400640 1 -- 172.21.15.100:0/873879081 shutdown_connections 2024-09-17T04:30:15.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.061+0000 7f2690400640 1 -- 172.21.15.100:0/873879081 wait complete. 2024-09-17T04:30:15.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.063+0000 7f2690400640 1 Processor -- start 2024-09-17T04:30:15.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.063+0000 7f2690400640 1 -- start start 2024-09-17T04:30:15.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.064+0000 7f2690400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2688073540 0x7f26881510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:15.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.064+0000 7f2690400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26880aee00 0x7f268813fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:15.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.064+0000 7f2690400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f268815f9e0 0x7f268813dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:15.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.064+0000 7f2690400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f268806fb60 con 0x7f2688073540 2024-09-17T04:30:15.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.064+0000 7f2690400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f268806f9e0 con 0x7f268815f9e0 2024-09-17T04:30:15.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.064+0000 7f2690400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f268806fce0 con 0x7f26880aee00 2024-09-17T04:30:15.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.065+0000 7f268da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2688073540 0x7f26881510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:15.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.065+0000 7f268da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2688073540 0x7f26881510d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60372/0 (socket says 172.21.15.100:60372) 2024-09-17T04:30:15.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.065+0000 7f268da00640 1 -- 172.21.15.100:0/53413529 learned_addr learned my addr 172.21.15.100:0/53413529 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:15.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.065+0000 7f268da00640 1 -- 172.21.15.100:0/53413529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26880aee00 msgr2=0x7f268813fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:15.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.065+0000 7f268da00640 1 --2- 172.21.15.100:0/53413529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26880aee00 0x7f268813fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:15.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.065+0000 7f268da00640 1 -- 172.21.15.100:0/53413529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f268815f9e0 msgr2=0x7f268813dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:15.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.065+0000 7f268da00640 1 --2- 172.21.15.100:0/53413529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f268815f9e0 0x7f268813dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:15.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.065+0000 7f268da00640 1 -- 172.21.15.100:0/53413529 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26881403a0 con 0x7f2688073540 2024-09-17T04:30:15.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.065+0000 7f268da00640 1 --2- 172.21.15.100:0/53413529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2688073540 0x7f26881510d0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f267c00d750 tx=0x7f267c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:15.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.065+0000 7f2685600640 1 -- 172.21.15.100:0/53413529 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c017070 con 0x7f2688073540 2024-09-17T04:30:15.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.066+0000 7f2685600640 1 -- 172.21.15.100:0/53413529 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f267c009b20 con 0x7f2688073540 2024-09-17T04:30:15.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.066+0000 7f2690400640 1 -- 172.21.15.100:0/53413529 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2688151900 con 0x7f2688073540 2024-09-17T04:30:15.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.066+0000 7f2685600640 1 -- 172.21.15.100:0/53413529 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c010460 con 0x7f2688073540 2024-09-17T04:30:15.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.066+0000 7f2690400640 1 -- 172.21.15.100:0/53413529 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2688151ec0 con 0x7f2688073540 2024-09-17T04:30:15.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.067+0000 7f2690400640 1 -- 172.21.15.100:0/53413529 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26880af210 con 0x7f2688073540 2024-09-17T04:30:15.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.068+0000 7f2685600640 1 -- 172.21.15.100:0/53413529 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 101594+0+0 (secure 0 0 0) 0x7f267c01a070 con 0x7f2688073540 2024-09-17T04:30:15.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.069+0000 7f2685600640 1 --2- 172.21.15.100:0/53413529 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2668078d40 0x7f266807b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:15.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.069+0000 7f2685600640 1 -- 172.21.15.100:0/53413529 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f267c09b380 con 0x7f2688073540 2024-09-17T04:30:15.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.069+0000 7f268d000640 1 --2- 172.21.15.100:0/53413529 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2668078d40 0x7f266807b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:15.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.070+0000 7f268d000640 1 --2- 172.21.15.100:0/53413529 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2668078d40 0x7f266807b250 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2674033000 tx=0x7f26740046c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:15.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.073+0000 7f2685600640 1 -- 172.21.15.100:0/53413529 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f267c0633d0 con 0x7f2688073540 2024-09-17T04:30:15.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.284+0000 7f2690400640 1 -- 172.21.15.100:0/53413529 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol80190", "target": ["mon-mgr", ""]}) -- 0x7f2688147c40 con 0x7f2668078d40 2024-09-17T04:30:15.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.310+0000 7f2685600640 1 -- 172.21.15.100:0/53413529 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f2688147c40 con 0x7f2668078d40 2024-09-17T04:30:15.310 INFO:teuthology.orchestra.run.smithi100.stdout:/volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4 2024-09-17T04:30:15.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.312+0000 7f2690400640 1 -- 172.21.15.100:0/53413529 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2668078d40 msgr2=0x7f266807b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:15.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.312+0000 7f2690400640 1 --2- 172.21.15.100:0/53413529 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2668078d40 0x7f266807b250 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2674033000 tx=0x7f26740046c0 comp rx=0 tx=0).stop 2024-09-17T04:30:15.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.312+0000 7f2690400640 1 -- 172.21.15.100:0/53413529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2688073540 msgr2=0x7f26881510d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:15.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.312+0000 7f2690400640 1 --2- 172.21.15.100:0/53413529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2688073540 0x7f26881510d0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f267c00d750 tx=0x7f267c00dc10 comp rx=0 tx=0).stop 2024-09-17T04:30:15.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.312+0000 7f2690400640 1 -- 172.21.15.100:0/53413529 shutdown_connections 2024-09-17T04:30:15.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.312+0000 7f2690400640 1 --2- 172.21.15.100:0/53413529 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2668078d40 0x7f266807b250 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:15.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.312+0000 7f2690400640 1 --2- 172.21.15.100:0/53413529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f268815f9e0 0x7f268813dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:15.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.312+0000 7f2690400640 1 --2- 172.21.15.100:0/53413529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26880aee00 0x7f268813fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:15.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.312+0000 7f2690400640 1 --2- 172.21.15.100:0/53413529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2688073540 0x7f26881510d0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:15.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.312+0000 7f2690400640 1 -- 172.21.15.100:0/53413529 >> 172.21.15.100:0/53413529 conn(0x7f268806d1c0 msgr2=0x7f26881600d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:15.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.312+0000 7f2690400640 1 -- 172.21.15.100:0/53413529 shutdown_connections 2024-09-17T04:30:15.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:15.312+0000 7f2690400640 1 -- 172.21.15.100:0/53413529 wait complete. 2024-09-17T04:30:15.334 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvol80190 with 8 files each 1MB size under directory volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4 2024-09-17T04:30:15.334 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:30:15.335 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:30:15.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:15.370 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0) 2024-09-17T04:30:15.468 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-17T04:30:15.468 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-17T04:30:15.468 INFO:teuthology.orchestra.run.smithi100.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214769 s, 48.8 MB/s 2024-09-17T04:30:15.469 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:30:15.470 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:30:15.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:15.505 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0) 2024-09-17T04:30:15.602 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-17T04:30:15.603 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-17T04:30:15.603 INFO:teuthology.orchestra.run.smithi100.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210259 s, 49.9 MB/s 2024-09-17T04:30:15.604 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:30:15.604 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:30:15.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:15.640 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0) 2024-09-17T04:30:15.737 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-17T04:30:15.737 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-17T04:30:15.737 INFO:teuthology.orchestra.run.smithi100.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208482 s, 50.3 MB/s 2024-09-17T04:30:15.738 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:30:15.738 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:30:15.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:15.774 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0) 2024-09-17T04:30:15.869 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-17T04:30:15.869 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-17T04:30:15.869 INFO:teuthology.orchestra.run.smithi100.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184879 s, 56.7 MB/s 2024-09-17T04:30:15.870 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:30:15.870 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:30:15.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:15.906 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0) 2024-09-17T04:30:16.002 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-17T04:30:16.002 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-17T04:30:16.002 INFO:teuthology.orchestra.run.smithi100.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203713 s, 51.5 MB/s 2024-09-17T04:30:16.003 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:30:16.003 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:30:16.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:16.041 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0) 2024-09-17T04:30:16.137 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-17T04:30:16.137 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-17T04:30:16.138 INFO:teuthology.orchestra.run.smithi100.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197874 s, 53.0 MB/s 2024-09-17T04:30:16.139 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:30:16.139 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:30:16.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:16.174 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0) 2024-09-17T04:30:16.268 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-17T04:30:16.268 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-17T04:30:16.269 INFO:teuthology.orchestra.run.smithi100.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182513 s, 57.5 MB/s 2024-09-17T04:30:16.269 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:30:16.270 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:30:16.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:16.306 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0) 2024-09-17T04:30:16.399 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-17T04:30:16.399 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-17T04:30:16.399 INFO:teuthology.orchestra.run.smithi100.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166077 s, 63.1 MB/s 2024-09-17T04:30:16.400 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol80190 subvol_snap99850 2024-09-17T04:30:16.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.626+0000 7fcfda600640 1 -- 172.21.15.100:0/2766561518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfd406cda0 msgr2=0x7fcfd406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:16.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.626+0000 7fcfda600640 1 --2- 172.21.15.100:0/2766561518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfd406cda0 0x7fcfd406d1b0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc005fd0 tx=0x7fcfcc02aa90 comp rx=0 tx=0).stop 2024-09-17T04:30:16.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.626+0000 7fcfda600640 1 -- 172.21.15.100:0/2766561518 shutdown_connections 2024-09-17T04:30:16.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.626+0000 7fcfda600640 1 --2- 172.21.15.100:0/2766561518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfd406cda0 0x7fcfd406d1b0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:16.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.626+0000 7fcfda600640 1 --2- 172.21.15.100:0/2766561518 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfd415fd30 0x7fcfd40b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:16.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.626+0000 7fcfda600640 1 -- 172.21.15.100:0/2766561518 >> 172.21.15.100:0/2766561518 conn(0x7fcfd40b2540 msgr2=0x7fcfd40d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:16.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.626+0000 7fcfda600640 1 -- 172.21.15.100:0/2766561518 shutdown_connections 2024-09-17T04:30:16.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.626+0000 7fcfda600640 1 -- 172.21.15.100:0/2766561518 wait complete. 2024-09-17T04:30:16.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.629+0000 7fcfda600640 1 Processor -- start 2024-09-17T04:30:16.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.629+0000 7fcfda600640 1 -- start start 2024-09-17T04:30:16.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.630+0000 7fcfda600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfd406cda0 0x7fcfd415b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:16.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.630+0000 7fcfda600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfd415fd30 0x7fcfd41f6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:16.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.630+0000 7fcfda600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfd4162ff0 0x7fcfd41f7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:16.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.630+0000 7fcfda600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd40af490 con 0x7fcfd415fd30 2024-09-17T04:30:16.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.630+0000 7fcfda600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd40af310 con 0x7fcfd406cda0 2024-09-17T04:30:16.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.630+0000 7fcfda600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcfd40af610 con 0x7fcfd4162ff0 2024-09-17T04:30:16.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.630+0000 7fcfcaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfd415fd30 0x7fcfd41f6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:16.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.631+0000 7fcfcaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfd415fd30 0x7fcfd41f6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60384/0 (socket says 172.21.15.100:60384) 2024-09-17T04:30:16.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.631+0000 7fcfcaa00640 1 -- 172.21.15.100:0/3965737413 learned_addr learned my addr 172.21.15.100:0/3965737413 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:16.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.631+0000 7fcfd3400640 1 --2- 172.21.15.100:0/3965737413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfd406cda0 0x7fcfd415b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:16.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.631+0000 7fcfd3e00640 1 --2- 172.21.15.100:0/3965737413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfd4162ff0 0x7fcfd41f7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:16.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.631+0000 7fcfcaa00640 1 -- 172.21.15.100:0/3965737413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfd4162ff0 msgr2=0x7fcfd41f7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:16.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.631+0000 7fcfcaa00640 1 --2- 172.21.15.100:0/3965737413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfd4162ff0 0x7fcfd41f7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:16.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.631+0000 7fcfcaa00640 1 -- 172.21.15.100:0/3965737413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfd406cda0 msgr2=0x7fcfd415b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:16.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.631+0000 7fcfcaa00640 1 --2- 172.21.15.100:0/3965737413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfd406cda0 0x7fcfd415b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:16.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.631+0000 7fcfcaa00640 1 -- 172.21.15.100:0/3965737413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd41fa2d0 con 0x7fcfd415fd30 2024-09-17T04:30:16.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.631+0000 7fcfcaa00640 1 --2- 172.21.15.100:0/3965737413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfd415fd30 0x7fcfd41f6d50 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc001eb0 tx=0x7fcfbc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:16.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.631+0000 7fcfcbe00640 1 -- 172.21.15.100:0/3965737413 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc017070 con 0x7fcfd415fd30 2024-09-17T04:30:16.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.631+0000 7fcfcbe00640 1 -- 172.21.15.100:0/3965737413 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfbc0053c0 con 0x7fcfd415fd30 2024-09-17T04:30:16.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.632+0000 7fcfda600640 1 -- 172.21.15.100:0/3965737413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfd41fa5b0 con 0x7fcfd415fd30 2024-09-17T04:30:16.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.632+0000 7fcfcbe00640 1 -- 172.21.15.100:0/3965737413 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc003830 con 0x7fcfd415fd30 2024-09-17T04:30:16.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.632+0000 7fcfda600640 1 -- 172.21.15.100:0/3965737413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfd41fab70 con 0x7fcfd415fd30 2024-09-17T04:30:16.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.633+0000 7fcfda600640 1 -- 172.21.15.100:0/3965737413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfd406d1b0 con 0x7fcfd415fd30 2024-09-17T04:30:16.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.634+0000 7fcfcbe00640 1 -- 172.21.15.100:0/3965737413 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 101594+0+0 (secure 0 0 0) 0x7fcfbc01a070 con 0x7fcfd415fd30 2024-09-17T04:30:16.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.635+0000 7fcfcbe00640 1 --2- 172.21.15.100:0/3965737413 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcfb0078d70 0x7fcfb007b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:16.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.635+0000 7fcfcbe00640 1 -- 172.21.15.100:0/3965737413 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7fcfbc09a530 con 0x7fcfd415fd30 2024-09-17T04:30:16.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.635+0000 7fcfd3400640 1 --2- 172.21.15.100:0/3965737413 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcfb0078d70 0x7fcfb007b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:16.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.636+0000 7fcfd3400640 1 --2- 172.21.15.100:0/3965737413 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcfb0078d70 0x7fcfb007b280 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009ac0 tx=0x7fcfc4005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:16.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.640+0000 7fcfcbe00640 1 -- 172.21.15.100:0/3965737413 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fcfd415efd0 con 0x7fcfd415fd30 2024-09-17T04:30:16.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:16.853+0000 7fcfda600640 1 -- 172.21.15.100:0/3965737413 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol80190", "snap_name": "subvol_snap99850", "target": ["mon-mgr", ""]}) -- 0x7fcfd4159070 con 0x7fcfb0078d70 2024-09-17T04:30:19.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:19.989+0000 7fcfcbe00640 1 -- 172.21.15.100:0/3965737413 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcfd4159070 con 0x7fcfb0078d70 2024-09-17T04:30:19.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:19.991+0000 7fcfda600640 1 -- 172.21.15.100:0/3965737413 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcfb0078d70 msgr2=0x7fcfb007b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:19.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:19.991+0000 7fcfda600640 1 --2- 172.21.15.100:0/3965737413 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcfb0078d70 0x7fcfb007b280 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009ac0 tx=0x7fcfc4005f90 comp rx=0 tx=0).stop 2024-09-17T04:30:19.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:19.991+0000 7fcfda600640 1 -- 172.21.15.100:0/3965737413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfd415fd30 msgr2=0x7fcfd41f6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:19.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:19.991+0000 7fcfda600640 1 --2- 172.21.15.100:0/3965737413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfd415fd30 0x7fcfd41f6d50 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc001eb0 tx=0x7fcfbc00c890 comp rx=0 tx=0).stop 2024-09-17T04:30:19.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:19.992+0000 7fcfda600640 1 -- 172.21.15.100:0/3965737413 shutdown_connections 2024-09-17T04:30:19.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:19.992+0000 7fcfda600640 1 --2- 172.21.15.100:0/3965737413 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcfb0078d70 0x7fcfb007b280 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:19.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:19.992+0000 7fcfda600640 1 --2- 172.21.15.100:0/3965737413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfd4162ff0 0x7fcfd41f7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:19.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:19.992+0000 7fcfda600640 1 --2- 172.21.15.100:0/3965737413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfd415fd30 0x7fcfd41f6d50 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:19.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:19.992+0000 7fcfda600640 1 --2- 172.21.15.100:0/3965737413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfd406cda0 0x7fcfd415b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:19.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:19.992+0000 7fcfda600640 1 -- 172.21.15.100:0/3965737413 >> 172.21.15.100:0/3965737413 conn(0x7fcfd40b2540 msgr2=0x7fcfd406f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:19.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:19.992+0000 7fcfda600640 1 -- 172.21.15.100:0/3965737413 shutdown_connections 2024-09-17T04:30:19.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:19.992+0000 7fcfda600640 1 -- 172.21.15.100:0/3965737413 wait complete. 2024-09-17T04:30:20.020 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol80190 subvol_snap99850 subvol_clone15410 2024-09-17T04:30:20.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.251+0000 7f05f7200640 1 -- 172.21.15.100:0/3851263671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05dc004800 msgr2=0x7f05dc004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:20.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.251+0000 7f05f7200640 1 --2- 172.21.15.100:0/3851263671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05dc004800 0x7f05dc004c10 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f05f8070010 tx=0x7f05dc005830 comp rx=0 tx=0).stop 2024-09-17T04:30:20.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.251+0000 7f05f6800640 1 -- 172.21.15.100:0/3851263671 reap_dead start 2024-09-17T04:30:20.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.251+0000 7f05f7200640 1 -- 172.21.15.100:0/3851263671 shutdown_connections 2024-09-17T04:30:20.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.251+0000 7f05f7200640 1 -- 172.21.15.100:0/3851263671 >> 172.21.15.100:0/3851263671 conn(0x7f05f808bfe0 msgr2=0x7f05f80d4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:20.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.251+0000 7f05f7200640 1 -- 172.21.15.100:0/3851263671 shutdown_connections 2024-09-17T04:30:20.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.251+0000 7f05f7200640 1 -- 172.21.15.100:0/3851263671 wait complete. 2024-09-17T04:30:20.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.254+0000 7f05f7200640 1 Processor -- start 2024-09-17T04:30:20.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.254+0000 7f05f7200640 1 -- start start 2024-09-17T04:30:20.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.255+0000 7f05f7200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05dc004800 0x7f05f81f61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:20.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.255+0000 7f05f7200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05f809ebe0 0x7f05f81f67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:20.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.255+0000 7f05f7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05f80cdda0 0x7f05f815fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:20.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.255+0000 7f05f7200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f05f80d2650 con 0x7f05f80cdda0 2024-09-17T04:30:20.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.255+0000 7f05f7200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f05f80d24d0 con 0x7f05f809ebe0 2024-09-17T04:30:20.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.255+0000 7f05f7200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f05f806dea0 con 0x7f05dc004800 2024-09-17T04:30:20.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.255+0000 7f05f6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05f80cdda0 0x7f05f815fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:20.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.256+0000 7f05f6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05f80cdda0 0x7f05f815fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60396/0 (socket says 172.21.15.100:60396) 2024-09-17T04:30:20.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.256+0000 7f05f6800640 1 -- 172.21.15.100:0/3639435447 learned_addr learned my addr 172.21.15.100:0/3639435447 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:20.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.256+0000 7f05f5400640 1 --2- 172.21.15.100:0/3639435447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05f809ebe0 0x7f05f81f67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:20.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.256+0000 7f05f5e00640 1 --2- 172.21.15.100:0/3639435447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05dc004800 0x7f05f81f61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:20.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.256+0000 7f05f6800640 1 -- 172.21.15.100:0/3639435447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05dc004800 msgr2=0x7f05f81f61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:20.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.256+0000 7f05f6800640 1 --2- 172.21.15.100:0/3639435447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05dc004800 0x7f05f81f61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:20.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.256+0000 7f05f6800640 1 -- 172.21.15.100:0/3639435447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05f809ebe0 msgr2=0x7f05f81f67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:20.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.256+0000 7f05f6800640 1 --2- 172.21.15.100:0/3639435447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05f809ebe0 0x7f05f81f67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:20.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.256+0000 7f05f6800640 1 -- 172.21.15.100:0/3639435447 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f809f360 con 0x7f05f80cdda0 2024-09-17T04:30:20.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.256+0000 7f05f5400640 1 --2- 172.21.15.100:0/3639435447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05f809ebe0 0x7f05f81f67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:30:20.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.256+0000 7f05f6800640 1 --2- 172.21.15.100:0/3639435447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05f80cdda0 0x7f05f815fe50 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f05f808d9d0 tx=0x7f05f00064f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:20.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.256+0000 7f05ee000640 1 -- 172.21.15.100:0/3639435447 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f0015070 con 0x7f05f80cdda0 2024-09-17T04:30:20.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.256+0000 7f05ee000640 1 -- 172.21.15.100:0/3639435447 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05f00119c0 con 0x7f05f80cdda0 2024-09-17T04:30:20.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.256+0000 7f05ee000640 1 -- 172.21.15.100:0/3639435447 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8009710 con 0x7f05f80cdda0 2024-09-17T04:30:20.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.256+0000 7f05f7200640 1 -- 172.21.15.100:0/3639435447 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f80a0a60 con 0x7f05f80cdda0 2024-09-17T04:30:20.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.257+0000 7f05f7200640 1 -- 172.21.15.100:0/3639435447 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f80a1020 con 0x7f05f80cdda0 2024-09-17T04:30:20.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.258+0000 7f05f7200640 1 -- 172.21.15.100:0/3639435447 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05f80b3a80 con 0x7f05f80cdda0 2024-09-17T04:30:20.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.259+0000 7f05ee000640 1 -- 172.21.15.100:0/3639435447 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 101594+0+0 (secure 0 0 0) 0x7f05f0010420 con 0x7f05f80cdda0 2024-09-17T04:30:20.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.260+0000 7f05ee000640 1 --2- 172.21.15.100:0/3639435447 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f05d8078d70 0x7f05d807b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:20.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.261+0000 7f05ee000640 1 -- 172.21.15.100:0/3639435447 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f05f009bc90 con 0x7f05f80cdda0 2024-09-17T04:30:20.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.261+0000 7f05f5e00640 1 --2- 172.21.15.100:0/3639435447 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f05d8078d70 0x7f05d807b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:20.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.261+0000 7f05f5e00640 1 --2- 172.21.15.100:0/3639435447 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f05d8078d70 0x7f05d807b280 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009d20 tx=0x7f05e8005f30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:20.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.265+0000 7f05ee000640 1 -- 172.21.15.100:0/3639435447 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f05f809f360 con 0x7f05f80cdda0 2024-09-17T04:30:20.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.481+0000 7f05f7200640 1 -- 172.21.15.100:0/3639435447 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol80190", "snap_name": "subvol_snap99850", "target_sub_name": "subvol_clone15410", "target": ["mon-mgr", ""]}) -- 0x7f05f80b5fa0 con 0x7f05d8078d70 2024-09-17T04:30:20.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.548+0000 7f05ee000640 1 -- 172.21.15.100:0/3639435447 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f05f80b5fa0 con 0x7f05d8078d70 2024-09-17T04:30:20.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.551+0000 7f05f7200640 1 -- 172.21.15.100:0/3639435447 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f05d8078d70 msgr2=0x7f05d807b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:20.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.551+0000 7f05f7200640 1 --2- 172.21.15.100:0/3639435447 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f05d8078d70 0x7f05d807b280 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009d20 tx=0x7f05e8005f30 comp rx=0 tx=0).stop 2024-09-17T04:30:20.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.551+0000 7f05f7200640 1 -- 172.21.15.100:0/3639435447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05f80cdda0 msgr2=0x7f05f815fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:20.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.551+0000 7f05f7200640 1 --2- 172.21.15.100:0/3639435447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05f80cdda0 0x7f05f815fe50 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f05f808d9d0 tx=0x7f05f00064f0 comp rx=0 tx=0).stop 2024-09-17T04:30:20.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.551+0000 7f05f7200640 1 -- 172.21.15.100:0/3639435447 shutdown_connections 2024-09-17T04:30:20.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.551+0000 7f05f7200640 1 --2- 172.21.15.100:0/3639435447 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f05d8078d70 0x7f05d807b280 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:20.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.551+0000 7f05f7200640 1 --2- 172.21.15.100:0/3639435447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05f80cdda0 0x7f05f815fe50 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:20.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.551+0000 7f05f7200640 1 --2- 172.21.15.100:0/3639435447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05f809ebe0 0x7f05f81f67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:20.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.551+0000 7f05f7200640 1 --2- 172.21.15.100:0/3639435447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05dc004800 0x7f05f81f61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:20.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.551+0000 7f05f7200640 1 -- 172.21.15.100:0/3639435447 >> 172.21.15.100:0/3639435447 conn(0x7f05f808bfe0 msgr2=0x7f05f80cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:20.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.551+0000 7f05f7200640 1 -- 172.21.15.100:0/3639435447 shutdown_connections 2024-09-17T04:30:20.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.551+0000 7f05f7200640 1 -- 172.21.15.100:0/3639435447 wait complete. 2024-09-17T04:30:20.553 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:20.554+0000 7f30ba800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:20.553 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:20.554+0000 7f30ba800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:20.553 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:20.554+0000 7f30ba800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:20.553 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:20.554+0000 7f30ba800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:20.553 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:20.554+0000 7f30ba800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:20.571 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:20.572+0000 7f30b9e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:20.571 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:20.572+0000 7f30b9e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:20.571 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:20.572+0000 7f30b9e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:20.571 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:20.572+0000 7f30b9e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:20.571 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:20.572+0000 7f30b9e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:20.579 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone15410 2024-09-17T04:30:20.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.832+0000 7f402f800640 1 -- 172.21.15.100:0/2662021828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4028145550 msgr2=0x7f402806fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:20.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.832+0000 7f402f800640 1 --2- 172.21.15.100:0/2662021828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4028145550 0x7f402806fb40 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f40281464e0 tx=0x7f401c02a680 comp rx=0 tx=0).stop 2024-09-17T04:30:20.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.833+0000 7f402f800640 1 -- 172.21.15.100:0/2662021828 shutdown_connections 2024-09-17T04:30:20.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.833+0000 7f402f800640 1 --2- 172.21.15.100:0/2662021828 >> v2:172.21.15.184:3300/0 conn(0x7f4028073c50 0x7f40280cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:20.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.833+0000 7f402f800640 1 --2- 172.21.15.100:0/2662021828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4028145550 0x7f402806fb40 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:20.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.833+0000 7f402f800640 1 -- 172.21.15.100:0/2662021828 >> 172.21.15.100:0/2662021828 conn(0x7f402806d1c0 msgr2=0x7f402806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:20.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.833+0000 7f402f800640 1 -- 172.21.15.100:0/2662021828 shutdown_connections 2024-09-17T04:30:20.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.833+0000 7f402f800640 1 -- 172.21.15.100:0/2662021828 wait complete. 2024-09-17T04:30:20.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.835+0000 7f402f800640 1 Processor -- start 2024-09-17T04:30:20.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.835+0000 7f402f800640 1 -- start start 2024-09-17T04:30:20.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.836+0000 7f402f800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4028145550 0x7f402813dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:20.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.836+0000 7f402f800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4028145a40 0x7f402815d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:20.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.836+0000 7f402f800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40281f44f0 0x7f402813f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:20.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.836+0000 7f402f800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f40280ce130 con 0x7f4028145550 2024-09-17T04:30:20.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.836+0000 7f402f800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f40280cdfb0 con 0x7f40281f44f0 2024-09-17T04:30:20.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.836+0000 7f402f800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f40280ce2b0 con 0x7f4028145a40 2024-09-17T04:30:20.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.837+0000 7f402ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4028145550 0x7f402813dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:20.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.837+0000 7f402ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4028145550 0x7f402813dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60462/0 (socket says 172.21.15.100:60462) 2024-09-17T04:30:20.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.837+0000 7f402ce00640 1 -- 172.21.15.100:0/3226635825 learned_addr learned my addr 172.21.15.100:0/3226635825 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:20.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.837+0000 7f402d800640 1 --2- 172.21.15.100:0/3226635825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40281f44f0 0x7f402813f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:20.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.837+0000 7f4027e00640 1 --2- 172.21.15.100:0/3226635825 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4028145a40 0x7f402815d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:20.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.837+0000 7f402ce00640 1 -- 172.21.15.100:0/3226635825 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4028145a40 msgr2=0x7f402815d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:20.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.837+0000 7f402ce00640 1 --2- 172.21.15.100:0/3226635825 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4028145a40 0x7f402815d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:20.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.837+0000 7f402ce00640 1 -- 172.21.15.100:0/3226635825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40281f44f0 msgr2=0x7f402813f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:20.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.837+0000 7f402ce00640 1 --2- 172.21.15.100:0/3226635825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40281f44f0 0x7f402813f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:20.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.837+0000 7f402ce00640 1 -- 172.21.15.100:0/3226635825 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402815dcb0 con 0x7f4028145550 2024-09-17T04:30:20.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.837+0000 7f402d800640 1 --2- 172.21.15.100:0/3226635825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40281f44f0 0x7f402813f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:30:20.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.838+0000 7f402ce00640 1 --2- 172.21.15.100:0/3226635825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4028145550 0x7f402813dd90 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f401c000c00 tx=0x7f401c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:20.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.838+0000 7f4024c00640 1 -- 172.21.15.100:0/3226635825 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c03e030 con 0x7f4028145550 2024-09-17T04:30:20.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.838+0000 7f4024c00640 1 -- 172.21.15.100:0/3226635825 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f401c0047a0 con 0x7f4028145550 2024-09-17T04:30:20.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.838+0000 7f402f800640 1 -- 172.21.15.100:0/3226635825 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f402813e790 con 0x7f4028145550 2024-09-17T04:30:20.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.838+0000 7f4024c00640 1 -- 172.21.15.100:0/3226635825 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c035040 con 0x7f4028145550 2024-09-17T04:30:20.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.838+0000 7f402f800640 1 -- 172.21.15.100:0/3226635825 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f402813ed50 con 0x7f4028145550 2024-09-17T04:30:20.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.839+0000 7f402f800640 1 -- 172.21.15.100:0/3226635825 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f402806f100 con 0x7f4028145550 2024-09-17T04:30:20.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.840+0000 7f4024c00640 1 -- 172.21.15.100:0/3226635825 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 101594+0+0 (secure 0 0 0) 0x7f401c041030 con 0x7f4028145550 2024-09-17T04:30:20.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.841+0000 7f4024c00640 1 --2- 172.21.15.100:0/3226635825 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4008078d40 0x7f400807b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:20.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.841+0000 7f4024c00640 1 -- 172.21.15.100:0/3226635825 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f401c0bba00 con 0x7f4028145550 2024-09-17T04:30:20.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.841+0000 7f4027e00640 1 --2- 172.21.15.100:0/3226635825 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4008078d40 0x7f400807b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:20.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.842+0000 7f4027e00640 1 --2- 172.21.15.100:0/3226635825 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4008078d40 0x7f400807b250 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4028069520 tx=0x7f4014009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:20.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:20.846+0000 7f4024c00640 1 -- 172.21.15.100:0/3226635825 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f401c083b00 con 0x7f4028145550 2024-09-17T04:30:21.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:21.065+0000 7f402f800640 1 -- 172.21.15.100:0/3226635825 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone15410", "target": ["mon-mgr", ""]}) -- 0x7f402813e540 con 0x7f4008078d40 2024-09-17T04:30:21.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:21.087+0000 7f4024c00640 1 -- 172.21.15.100:0/3226635825 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+301 (secure 0 0 0) 0x7f402813e540 con 0x7f4008078d40 2024-09-17T04:30:21.087 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:30:21.087 INFO:teuthology.orchestra.run.smithi100.stdout: "status": { 2024-09-17T04:30:21.087 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "in-progress", 2024-09-17T04:30:21.087 INFO:teuthology.orchestra.run.smithi100.stdout: "source": { 2024-09-17T04:30:21.087 INFO:teuthology.orchestra.run.smithi100.stdout: "volume": "cephfs", 2024-09-17T04:30:21.087 INFO:teuthology.orchestra.run.smithi100.stdout: "subvolume": "subvol80190", 2024-09-17T04:30:21.087 INFO:teuthology.orchestra.run.smithi100.stdout: "snapshot": "subvol_snap99850" 2024-09-17T04:30:21.088 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:30:21.088 INFO:teuthology.orchestra.run.smithi100.stdout: "progress_report": { 2024-09-17T04:30:21.088 INFO:teuthology.orchestra.run.smithi100.stdout: "percentage cloned": "0.003%", 2024-09-17T04:30:21.088 INFO:teuthology.orchestra.run.smithi100.stdout: "amount cloned": "213/8.0M", 2024-09-17T04:30:21.088 INFO:teuthology.orchestra.run.smithi100.stdout: "files cloned": "3/9" 2024-09-17T04:30:21.088 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:30:21.088 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:30:21.088 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:30:21.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:21.090+0000 7f402f800640 1 -- 172.21.15.100:0/3226635825 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4008078d40 msgr2=0x7f400807b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:21.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:21.090+0000 7f402f800640 1 --2- 172.21.15.100:0/3226635825 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4008078d40 0x7f400807b250 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4028069520 tx=0x7f4014009040 comp rx=0 tx=0).stop 2024-09-17T04:30:21.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:21.090+0000 7f402f800640 1 -- 172.21.15.100:0/3226635825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4028145550 msgr2=0x7f402813dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:21.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:21.090+0000 7f402f800640 1 --2- 172.21.15.100:0/3226635825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4028145550 0x7f402813dd90 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f401c000c00 tx=0x7f401c0039e0 comp rx=0 tx=0).stop 2024-09-17T04:30:21.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:21.090+0000 7f402f800640 1 -- 172.21.15.100:0/3226635825 shutdown_connections 2024-09-17T04:30:21.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:21.090+0000 7f402f800640 1 --2- 172.21.15.100:0/3226635825 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4008078d40 0x7f400807b250 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:21.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:21.090+0000 7f402f800640 1 --2- 172.21.15.100:0/3226635825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40281f44f0 0x7f402813f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:21.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:21.090+0000 7f402f800640 1 --2- 172.21.15.100:0/3226635825 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4028145a40 0x7f402815d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:21.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:21.090+0000 7f402f800640 1 --2- 172.21.15.100:0/3226635825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4028145550 0x7f402813dd90 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:21.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:21.090+0000 7f402f800640 1 -- 172.21.15.100:0/3226635825 >> 172.21.15.100:0/3226635825 conn(0x7f402806d1c0 msgr2=0x7f40280cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:21.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:21.090+0000 7f402f800640 1 -- 172.21.15.100:0/3226635825 shutdown_connections 2024-09-17T04:30:21.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:21.090+0000 7f402f800640 1 -- 172.21.15.100:0/3226635825 wait complete. 2024-09-17T04:30:21.113 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-17T04:30:21.555 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:21.556+0000 7f309b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:21.555 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:21.556+0000 7f309b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:21.555 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:21.556+0000 7f309b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:21.556 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:21.556+0000 7f309b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:21.556 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:21.556+0000 7f309b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:22.115 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone15410 2024-09-17T04:30:22.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.339+0000 7fcd58e00640 1 -- 172.21.15.100:0/3184774423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd54073c50 msgr2=0x7fcd540cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:22.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.339+0000 7fcd58e00640 1 --2- 172.21.15.100:0/3184774423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd54073c50 0x7fcd540cd610 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c005fd0 tx=0x7fcd4c02ab20 comp rx=0 tx=0).stop 2024-09-17T04:30:22.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.339+0000 7fcd58e00640 1 -- 172.21.15.100:0/3184774423 shutdown_connections 2024-09-17T04:30:22.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.339+0000 7fcd58e00640 1 --2- 172.21.15.100:0/3184774423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd54073c50 0x7fcd540cd610 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:22.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.339+0000 7fcd58e00640 1 --2- 172.21.15.100:0/3184774423 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd54145550 0x7fcd5406fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:22.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.339+0000 7fcd58e00640 1 -- 172.21.15.100:0/3184774423 >> 172.21.15.100:0/3184774423 conn(0x7fcd5406d1c0 msgr2=0x7fcd5406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:22.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.339+0000 7fcd58e00640 1 -- 172.21.15.100:0/3184774423 shutdown_connections 2024-09-17T04:30:22.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.339+0000 7fcd58e00640 1 -- 172.21.15.100:0/3184774423 wait complete. 2024-09-17T04:30:22.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.341+0000 7fcd58e00640 1 Processor -- start 2024-09-17T04:30:22.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.341+0000 7fcd58e00640 1 -- start start 2024-09-17T04:30:22.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.342+0000 7fcd58e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd54145550 0x7fcd54140da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:22.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.342+0000 7fcd58e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd54145a40 0x7fcd5415fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:22.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.343+0000 7fcd58e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd541412e0 0x7fcd540aec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:22.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.343+0000 7fcd58e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcd540ce1f0 con 0x7fcd541412e0 2024-09-17T04:30:22.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.343+0000 7fcd58e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcd540ce070 con 0x7fcd54145a40 2024-09-17T04:30:22.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.343+0000 7fcd58e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcd540ce370 con 0x7fcd54145550 2024-09-17T04:30:22.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.343+0000 7fcd52800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd541412e0 0x7fcd540aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:22.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.343+0000 7fcd52800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd541412e0 0x7fcd540aec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60484/0 (socket says 172.21.15.100:60484) 2024-09-17T04:30:22.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.343+0000 7fcd52800640 1 -- 172.21.15.100:0/3121072891 learned_addr learned my addr 172.21.15.100:0/3121072891 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:22.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.343+0000 7fcd51400640 1 --2- 172.21.15.100:0/3121072891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd54145a40 0x7fcd5415fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:22.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.343+0000 7fcd51e00640 1 --2- 172.21.15.100:0/3121072891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd54145550 0x7fcd54140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:22.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.343+0000 7fcd52800640 1 -- 172.21.15.100:0/3121072891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd54145550 msgr2=0x7fcd54140da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:22.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.343+0000 7fcd52800640 1 --2- 172.21.15.100:0/3121072891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd54145550 0x7fcd54140da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:22.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.343+0000 7fcd52800640 1 -- 172.21.15.100:0/3121072891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd54145a40 msgr2=0x7fcd5415fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:22.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.343+0000 7fcd52800640 1 --2- 172.21.15.100:0/3121072891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd54145a40 0x7fcd5415fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:22.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.343+0000 7fcd52800640 1 -- 172.21.15.100:0/3121072891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd54160d60 con 0x7fcd541412e0 2024-09-17T04:30:22.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.344+0000 7fcd52800640 1 --2- 172.21.15.100:0/3121072891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd541412e0 0x7fcd540aec30 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c006010 tx=0x7fcd4c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:22.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.344+0000 7fcd4a000640 1 -- 172.21.15.100:0/3121072891 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c044070 con 0x7fcd541412e0 2024-09-17T04:30:22.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.344+0000 7fcd4a000640 1 -- 172.21.15.100:0/3121072891 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd4c02c820 con 0x7fcd541412e0 2024-09-17T04:30:22.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.344+0000 7fcd58e00640 1 -- 172.21.15.100:0/3121072891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd540ade70 con 0x7fcd541412e0 2024-09-17T04:30:22.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.344+0000 7fcd4a000640 1 -- 172.21.15.100:0/3121072891 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c0044b0 con 0x7fcd541412e0 2024-09-17T04:30:22.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.344+0000 7fcd58e00640 1 -- 172.21.15.100:0/3121072891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd540ae430 con 0x7fcd541412e0 2024-09-17T04:30:22.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.345+0000 7fcd58e00640 1 -- 172.21.15.100:0/3121072891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd54160ad0 con 0x7fcd541412e0 2024-09-17T04:30:22.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.346+0000 7fcd4a000640 1 -- 172.21.15.100:0/3121072891 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) ==== 101696+0+0 (secure 0 0 0) 0x7fcd4c004650 con 0x7fcd541412e0 2024-09-17T04:30:22.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.347+0000 7fcd4a000640 1 --2- 172.21.15.100:0/3121072891 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcd34078e60 0x7fcd3407b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:22.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.347+0000 7fcd4a000640 1 -- 172.21.15.100:0/3121072891 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7fcd4c0bb900 con 0x7fcd541412e0 2024-09-17T04:30:22.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.347+0000 7fcd51e00640 1 --2- 172.21.15.100:0/3121072891 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcd34078e60 0x7fcd3407b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:22.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.348+0000 7fcd51e00640 1 --2- 172.21.15.100:0/3121072891 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcd34078e60 0x7fcd3407b370 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400bb50 tx=0x7fcd4400b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:22.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.352+0000 7fcd4a000640 1 -- 172.21.15.100:0/3121072891 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fcd4c032030 con 0x7fcd541412e0 2024-09-17T04:30:22.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.568+0000 7fcd58e00640 1 -- 172.21.15.100:0/3121072891 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone15410", "target": ["mon-mgr", ""]}) -- 0x7fcd540ce8d0 con 0x7fcd34078e60 2024-09-17T04:30:22.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.586+0000 7fcd4a000640 1 -- 172.21.15.100:0/3121072891 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+301 (secure 0 0 0) 0x7fcd540ce8d0 con 0x7fcd34078e60 2024-09-17T04:30:22.586 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:30:22.586 INFO:teuthology.orchestra.run.smithi100.stdout: "status": { 2024-09-17T04:30:22.586 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "in-progress", 2024-09-17T04:30:22.586 INFO:teuthology.orchestra.run.smithi100.stdout: "source": { 2024-09-17T04:30:22.586 INFO:teuthology.orchestra.run.smithi100.stdout: "volume": "cephfs", 2024-09-17T04:30:22.586 INFO:teuthology.orchestra.run.smithi100.stdout: "subvolume": "subvol80190", 2024-09-17T04:30:22.586 INFO:teuthology.orchestra.run.smithi100.stdout: "snapshot": "subvol_snap99850" 2024-09-17T04:30:22.586 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:30:22.586 INFO:teuthology.orchestra.run.smithi100.stdout: "progress_report": { 2024-09-17T04:30:22.586 INFO:teuthology.orchestra.run.smithi100.stdout: "percentage cloned": "0.003%", 2024-09-17T04:30:22.586 INFO:teuthology.orchestra.run.smithi100.stdout: "amount cloned": "213/8.0M", 2024-09-17T04:30:22.587 INFO:teuthology.orchestra.run.smithi100.stdout: "files cloned": "3/9" 2024-09-17T04:30:22.587 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:30:22.587 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:30:22.587 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:30:22.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.588+0000 7fcd58e00640 1 -- 172.21.15.100:0/3121072891 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcd34078e60 msgr2=0x7fcd3407b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:22.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.588+0000 7fcd58e00640 1 --2- 172.21.15.100:0/3121072891 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcd34078e60 0x7fcd3407b370 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400bb50 tx=0x7fcd4400b2e0 comp rx=0 tx=0).stop 2024-09-17T04:30:22.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.588+0000 7fcd58e00640 1 -- 172.21.15.100:0/3121072891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd541412e0 msgr2=0x7fcd540aec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:22.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.588+0000 7fcd58e00640 1 --2- 172.21.15.100:0/3121072891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd541412e0 0x7fcd540aec30 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c006010 tx=0x7fcd4c003f10 comp rx=0 tx=0).stop 2024-09-17T04:30:22.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.588+0000 7fcd58e00640 1 -- 172.21.15.100:0/3121072891 shutdown_connections 2024-09-17T04:30:22.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.589+0000 7fcd58e00640 1 --2- 172.21.15.100:0/3121072891 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcd34078e60 0x7fcd3407b370 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:22.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.589+0000 7fcd58e00640 1 --2- 172.21.15.100:0/3121072891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd541412e0 0x7fcd540aec30 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:22.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.589+0000 7fcd58e00640 1 --2- 172.21.15.100:0/3121072891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd54145a40 0x7fcd5415fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:22.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.589+0000 7fcd58e00640 1 --2- 172.21.15.100:0/3121072891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd54145550 0x7fcd54140da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:22.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.589+0000 7fcd58e00640 1 -- 172.21.15.100:0/3121072891 >> 172.21.15.100:0/3121072891 conn(0x7fcd5406d1c0 msgr2=0x7fcd54143e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:22.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.589+0000 7fcd58e00640 1 -- 172.21.15.100:0/3121072891 shutdown_connections 2024-09-17T04:30:22.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:22.589+0000 7fcd58e00640 1 -- 172.21.15.100:0/3121072891 wait complete. 2024-09-17T04:30:22.614 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-17T04:30:23.615 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone15410 2024-09-17T04:30:23.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.853+0000 7f0afb600640 1 -- 172.21.15.100:0/263816231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0af4145550 msgr2=0x7f0af406fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:23.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.853+0000 7f0afb600640 1 --2- 172.21.15.100:0/263816231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0af4145550 0x7f0af406fb40 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f0af4160280 tx=0x7f0ae402a680 comp rx=0 tx=0).stop 2024-09-17T04:30:23.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.853+0000 7f0afb600640 1 -- 172.21.15.100:0/263816231 shutdown_connections 2024-09-17T04:30:23.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.853+0000 7f0afb600640 1 --2- 172.21.15.100:0/263816231 >> v2:172.21.15.184:3300/0 conn(0x7f0af4145a40 0x7f0af4073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:23.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.853+0000 7f0afb600640 1 --2- 172.21.15.100:0/263816231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0af4145550 0x7f0af406fb40 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:23.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.853+0000 7f0afb600640 1 -- 172.21.15.100:0/263816231 >> 172.21.15.100:0/263816231 conn(0x7f0af406d1c0 msgr2=0x7f0af406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:23.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.853+0000 7f0afb600640 1 -- 172.21.15.100:0/263816231 shutdown_connections 2024-09-17T04:30:23.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.853+0000 7f0afb600640 1 -- 172.21.15.100:0/263816231 wait complete. 2024-09-17T04:30:23.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.856+0000 7f0afb600640 1 Processor -- start 2024-09-17T04:30:23.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.856+0000 7f0afb600640 1 -- start start 2024-09-17T04:30:23.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.857+0000 7f0afb600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0af4073a80 0x7f0af412e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:23.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.857+0000 7f0afb600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0af4145550 0x7f0af41f6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:23.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.857+0000 7f0afb600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0af4145a40 0x7f0af40b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:23.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.857+0000 7f0afb600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0af4160a30 con 0x7f0af4073a80 2024-09-17T04:30:23.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.857+0000 7f0afb600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0af40ce870 con 0x7f0af4145550 2024-09-17T04:30:23.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.857+0000 7f0afb600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0af4160bb0 con 0x7f0af4145a40 2024-09-17T04:30:23.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.857+0000 7f0af8c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0af4073a80 0x7f0af412e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:23.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.857+0000 7f0af8c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0af4073a80 0x7f0af412e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60490/0 (socket says 172.21.15.100:60490) 2024-09-17T04:30:23.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.857+0000 7f0af8c00640 1 -- 172.21.15.100:0/1008091088 learned_addr learned my addr 172.21.15.100:0/1008091088 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:23.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.858+0000 7f0af3e00640 1 --2- 172.21.15.100:0/1008091088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0af4145550 0x7f0af41f6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:23.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.858+0000 7f0af9600640 1 --2- 172.21.15.100:0/1008091088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0af4145a40 0x7f0af40b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:23.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.858+0000 7f0af8c00640 1 -- 172.21.15.100:0/1008091088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0af4145a40 msgr2=0x7f0af40b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:23.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.858+0000 7f0af8c00640 1 --2- 172.21.15.100:0/1008091088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0af4145a40 0x7f0af40b22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:23.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.858+0000 7f0af8c00640 1 -- 172.21.15.100:0/1008091088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0af4145550 msgr2=0x7f0af41f6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:23.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.858+0000 7f0af8c00640 1 --2- 172.21.15.100:0/1008091088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0af4145550 0x7f0af41f6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:23.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.858+0000 7f0af8c00640 1 -- 172.21.15.100:0/1008091088 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af415d380 con 0x7f0af4073a80 2024-09-17T04:30:23.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.858+0000 7f0af8c00640 1 --2- 172.21.15.100:0/1008091088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0af4073a80 0x7f0af412e6c0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4034040 tx=0x7f0ae4003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:23.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.858+0000 7f0af0c00640 1 -- 172.21.15.100:0/1008091088 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae403e030 con 0x7f0af4073a80 2024-09-17T04:30:23.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.858+0000 7f0af0c00640 1 -- 172.21.15.100:0/1008091088 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ae4003e00 con 0x7f0af4073a80 2024-09-17T04:30:23.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.858+0000 7f0af0c00640 1 -- 172.21.15.100:0/1008091088 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4035040 con 0x7f0af4073a80 2024-09-17T04:30:23.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.858+0000 7f0afb600640 1 -- 172.21.15.100:0/1008091088 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0af415d660 con 0x7f0af4073a80 2024-09-17T04:30:23.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.858+0000 7f0afb600640 1 -- 172.21.15.100:0/1008091088 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0af415dc20 con 0x7f0af4073a80 2024-09-17T04:30:23.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.860+0000 7f0afb600640 1 -- 172.21.15.100:0/1008091088 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0af41610d0 con 0x7f0af4073a80 2024-09-17T04:30:23.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.861+0000 7f0af0c00640 1 -- 172.21.15.100:0/1008091088 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7f0ae4041030 con 0x7f0af4073a80 2024-09-17T04:30:23.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.862+0000 7f0af0c00640 1 --2- 172.21.15.100:0/1008091088 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0ad4078ef0 0x7f0ad407b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:23.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.862+0000 7f0af0c00640 1 -- 172.21.15.100:0/1008091088 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f0ae40bbae0 con 0x7f0af4073a80 2024-09-17T04:30:23.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.862+0000 7f0af3e00640 1 --2- 172.21.15.100:0/1008091088 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0ad4078ef0 0x7f0ad407b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:23.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.863+0000 7f0af3e00640 1 --2- 172.21.15.100:0/1008091088 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0ad4078ef0 0x7f0ad407b400 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0005fd0 tx=0x7f0ae0005850 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:23.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:23.866+0000 7f0af0c00640 1 -- 172.21.15.100:0/1008091088 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0ae4083ae0 con 0x7f0af4073a80 2024-09-17T04:30:24.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:24.093+0000 7f0afb600640 1 -- 172.21.15.100:0/1008091088 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone15410", "target": ["mon-mgr", ""]}) -- 0x7f0af4145e90 con 0x7f0ad4078ef0 2024-09-17T04:30:24.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:24.112+0000 7f0af0c00640 1 -- 172.21.15.100:0/1008091088 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+301 (secure 0 0 0) 0x7f0af4145e90 con 0x7f0ad4078ef0 2024-09-17T04:30:24.112 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:30:24.112 INFO:teuthology.orchestra.run.smithi100.stdout: "status": { 2024-09-17T04:30:24.112 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "in-progress", 2024-09-17T04:30:24.112 INFO:teuthology.orchestra.run.smithi100.stdout: "source": { 2024-09-17T04:30:24.113 INFO:teuthology.orchestra.run.smithi100.stdout: "volume": "cephfs", 2024-09-17T04:30:24.113 INFO:teuthology.orchestra.run.smithi100.stdout: "subvolume": "subvol80190", 2024-09-17T04:30:24.113 INFO:teuthology.orchestra.run.smithi100.stdout: "snapshot": "subvol_snap99850" 2024-09-17T04:30:24.113 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:30:24.113 INFO:teuthology.orchestra.run.smithi100.stdout: "progress_report": { 2024-09-17T04:30:24.113 INFO:teuthology.orchestra.run.smithi100.stdout: "percentage cloned": "0.003%", 2024-09-17T04:30:24.113 INFO:teuthology.orchestra.run.smithi100.stdout: "amount cloned": "213/8.0M", 2024-09-17T04:30:24.113 INFO:teuthology.orchestra.run.smithi100.stdout: "files cloned": "3/9" 2024-09-17T04:30:24.113 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:30:24.113 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:30:24.113 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:30:24.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:24.115+0000 7f0afb600640 1 -- 172.21.15.100:0/1008091088 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0ad4078ef0 msgr2=0x7f0ad407b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:24.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:24.115+0000 7f0afb600640 1 --2- 172.21.15.100:0/1008091088 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0ad4078ef0 0x7f0ad407b400 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0005fd0 tx=0x7f0ae0005850 comp rx=0 tx=0).stop 2024-09-17T04:30:24.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:24.115+0000 7f0afb600640 1 -- 172.21.15.100:0/1008091088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0af4073a80 msgr2=0x7f0af412e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:24.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:24.115+0000 7f0afb600640 1 --2- 172.21.15.100:0/1008091088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0af4073a80 0x7f0af412e6c0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4034040 tx=0x7f0ae4003910 comp rx=0 tx=0).stop 2024-09-17T04:30:24.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:24.115+0000 7f0afb600640 1 -- 172.21.15.100:0/1008091088 shutdown_connections 2024-09-17T04:30:24.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:24.115+0000 7f0afb600640 1 --2- 172.21.15.100:0/1008091088 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0ad4078ef0 0x7f0ad407b400 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:24.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:24.115+0000 7f0afb600640 1 --2- 172.21.15.100:0/1008091088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0af4145a40 0x7f0af40b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:24.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:24.115+0000 7f0afb600640 1 --2- 172.21.15.100:0/1008091088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0af4145550 0x7f0af41f6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:24.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:24.115+0000 7f0afb600640 1 --2- 172.21.15.100:0/1008091088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0af4073a80 0x7f0af412e6c0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:24.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:24.115+0000 7f0afb600640 1 -- 172.21.15.100:0/1008091088 >> 172.21.15.100:0/1008091088 conn(0x7f0af406d1c0 msgr2=0x7f0af40ae2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:24.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:24.115+0000 7f0afb600640 1 -- 172.21.15.100:0/1008091088 shutdown_connections 2024-09-17T04:30:24.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:24.115+0000 7f0afb600640 1 -- 172.21.15.100:0/1008091088 wait complete. 2024-09-17T04:30:24.139 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-17T04:30:25.140 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone15410 2024-09-17T04:30:25.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.367+0000 7f6b98e00640 1 -- 172.21.15.100:0/2713043944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b94073c50 msgr2=0x7f6b9406f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:25.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.367+0000 7f6b98e00640 1 --2- 172.21.15.100:0/2713043944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b94073c50 0x7f6b9406f7d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c005fd0 tx=0x7f6b8c02a820 comp rx=0 tx=0).stop 2024-09-17T04:30:25.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.367+0000 7f6b98e00640 1 -- 172.21.15.100:0/2713043944 shutdown_connections 2024-09-17T04:30:25.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.367+0000 7f6b98e00640 1 --2- 172.21.15.100:0/2713043944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b94073c50 0x7f6b9406f7d0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:25.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.367+0000 7f6b98e00640 1 -- 172.21.15.100:0/2713043944 >> 172.21.15.100:0/2713043944 conn(0x7f6b9406d1c0 msgr2=0x7f6b9406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:25.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.367+0000 7f6b98e00640 1 -- 172.21.15.100:0/2713043944 shutdown_connections 2024-09-17T04:30:25.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.367+0000 7f6b98e00640 1 -- 172.21.15.100:0/2713043944 wait complete. 2024-09-17T04:30:25.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.369+0000 7f6b98e00640 1 Processor -- start 2024-09-17T04:30:25.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.370+0000 7f6b98e00640 1 -- start start 2024-09-17T04:30:25.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.371+0000 7f6b98e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b94147b90 0x7f6b941f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:25.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.371+0000 7f6b98e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b9415f9e0 0x7f6b941432b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:25.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.371+0000 7f6b98e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6b9413fdb0 0x7f6b940af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:25.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.371+0000 7f6b98e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9406fe90 con 0x7f6b94147b90 2024-09-17T04:30:25.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.371+0000 7f6b98e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9406fd10 con 0x7f6b9415f9e0 2024-09-17T04:30:25.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.371+0000 7f6b98e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6b94070010 con 0x7f6b9413fdb0 2024-09-17T04:30:25.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.371+0000 7f6b91e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b94147b90 0x7f6b941f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:25.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.371+0000 7f6b91e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b94147b90 0x7f6b941f63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49662/0 (socket says 172.21.15.100:49662) 2024-09-17T04:30:25.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.371+0000 7f6b91e00640 1 -- 172.21.15.100:0/4035874395 learned_addr learned my addr 172.21.15.100:0/4035874395 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:25.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.372+0000 7f6b89400640 1 --2- 172.21.15.100:0/4035874395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b9415f9e0 0x7f6b941432b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:25.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.372+0000 7f6b92800640 1 --2- 172.21.15.100:0/4035874395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6b9413fdb0 0x7f6b940af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:25.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.372+0000 7f6b91e00640 1 -- 172.21.15.100:0/4035874395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6b9413fdb0 msgr2=0x7f6b940af9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:25.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.372+0000 7f6b91e00640 1 --2- 172.21.15.100:0/4035874395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6b9413fdb0 0x7f6b940af9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:25.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.372+0000 7f6b91e00640 1 -- 172.21.15.100:0/4035874395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b9415f9e0 msgr2=0x7f6b941432b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:25.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.372+0000 7f6b91e00640 1 --2- 172.21.15.100:0/4035874395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b9415f9e0 0x7f6b941432b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:25.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.372+0000 7f6b91e00640 1 -- 172.21.15.100:0/4035874395 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b94143ae0 con 0x7f6b94147b90 2024-09-17T04:30:25.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.372+0000 7f6b91e00640 1 --2- 172.21.15.100:0/4035874395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b94147b90 0x7f6b941f63b0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400c3a0 tx=0x7f6b8400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:25.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.372+0000 7f6b8aa00640 1 -- 172.21.15.100:0/4035874395 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84017070 con 0x7f6b94147b90 2024-09-17T04:30:25.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.372+0000 7f6b8aa00640 1 -- 172.21.15.100:0/4035874395 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b840053c0 con 0x7f6b94147b90 2024-09-17T04:30:25.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.372+0000 7f6b98e00640 1 -- 172.21.15.100:0/4035874395 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b94143dc0 con 0x7f6b94147b90 2024-09-17T04:30:25.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.372+0000 7f6b8aa00640 1 -- 172.21.15.100:0/4035874395 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84003850 con 0x7f6b94147b90 2024-09-17T04:30:25.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.372+0000 7f6b98e00640 1 -- 172.21.15.100:0/4035874395 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b94141ff0 con 0x7f6b94147b90 2024-09-17T04:30:25.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.373+0000 7f6b98e00640 1 -- 172.21.15.100:0/4035874395 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b9406f7d0 con 0x7f6b94147b90 2024-09-17T04:30:25.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.375+0000 7f6b8aa00640 1 -- 172.21.15.100:0/4035874395 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7f6b8401a070 con 0x7f6b94147b90 2024-09-17T04:30:25.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.375+0000 7f6b8aa00640 1 --2- 172.21.15.100:0/4035874395 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6b70078e90 0x7f6b7007b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:25.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.376+0000 7f6b8aa00640 1 -- 172.21.15.100:0/4035874395 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f6b8409a720 con 0x7f6b94147b90 2024-09-17T04:30:25.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.376+0000 7f6b89400640 1 --2- 172.21.15.100:0/4035874395 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6b70078e90 0x7f6b7007b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:25.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.376+0000 7f6b89400640 1 --2- 172.21.15.100:0/4035874395 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6b70078e90 0x7f6b7007b3a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6b941478d0 tx=0x7f6b7c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:25.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.380+0000 7f6b8aa00640 1 -- 172.21.15.100:0/4035874395 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6b840625e0 con 0x7f6b94147b90 2024-09-17T04:30:25.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.597+0000 7f6b98e00640 1 -- 172.21.15.100:0/4035874395 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone15410", "target": ["mon-mgr", ""]}) -- 0x7f6b940705d0 con 0x7f6b70078e90 2024-09-17T04:30:25.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.615+0000 7f6b8aa00640 1 -- 172.21.15.100:0/4035874395 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+304 (secure 0 0 0) 0x7f6b940705d0 con 0x7f6b70078e90 2024-09-17T04:30:25.614 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:30:25.614 INFO:teuthology.orchestra.run.smithi100.stdout: "status": { 2024-09-17T04:30:25.614 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "in-progress", 2024-09-17T04:30:25.614 INFO:teuthology.orchestra.run.smithi100.stdout: "source": { 2024-09-17T04:30:25.614 INFO:teuthology.orchestra.run.smithi100.stdout: "volume": "cephfs", 2024-09-17T04:30:25.615 INFO:teuthology.orchestra.run.smithi100.stdout: "subvolume": "subvol80190", 2024-09-17T04:30:25.615 INFO:teuthology.orchestra.run.smithi100.stdout: "snapshot": "subvol_snap99850" 2024-09-17T04:30:25.615 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:30:25.615 INFO:teuthology.orchestra.run.smithi100.stdout: "progress_report": { 2024-09-17T04:30:25.615 INFO:teuthology.orchestra.run.smithi100.stdout: "percentage cloned": "75.003%", 2024-09-17T04:30:25.615 INFO:teuthology.orchestra.run.smithi100.stdout: "amount cloned": "6.0M/8.0M", 2024-09-17T04:30:25.615 INFO:teuthology.orchestra.run.smithi100.stdout: "files cloned": "10/9" 2024-09-17T04:30:25.615 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:30:25.615 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:30:25.615 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:30:25.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.617+0000 7f6b98e00640 1 -- 172.21.15.100:0/4035874395 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6b70078e90 msgr2=0x7f6b7007b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:25.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.617+0000 7f6b98e00640 1 --2- 172.21.15.100:0/4035874395 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6b70078e90 0x7f6b7007b3a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6b941478d0 tx=0x7f6b7c009040 comp rx=0 tx=0).stop 2024-09-17T04:30:25.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.617+0000 7f6b98e00640 1 -- 172.21.15.100:0/4035874395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b94147b90 msgr2=0x7f6b941f63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:25.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.617+0000 7f6b98e00640 1 --2- 172.21.15.100:0/4035874395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b94147b90 0x7f6b941f63b0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400c3a0 tx=0x7f6b8400c860 comp rx=0 tx=0).stop 2024-09-17T04:30:25.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.617+0000 7f6b98e00640 1 -- 172.21.15.100:0/4035874395 shutdown_connections 2024-09-17T04:30:25.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.617+0000 7f6b98e00640 1 --2- 172.21.15.100:0/4035874395 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6b70078e90 0x7f6b7007b3a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:25.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.617+0000 7f6b98e00640 1 --2- 172.21.15.100:0/4035874395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6b9413fdb0 0x7f6b940af9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:25.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.617+0000 7f6b98e00640 1 --2- 172.21.15.100:0/4035874395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b9415f9e0 0x7f6b941432b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:25.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.617+0000 7f6b98e00640 1 --2- 172.21.15.100:0/4035874395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b94147b90 0x7f6b941f63b0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:25.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.617+0000 7f6b98e00640 1 -- 172.21.15.100:0/4035874395 >> 172.21.15.100:0/4035874395 conn(0x7f6b9406d1c0 msgr2=0x7f6b941405d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:25.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.617+0000 7f6b98e00640 1 -- 172.21.15.100:0/4035874395 shutdown_connections 2024-09-17T04:30:25.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:25.617+0000 7f6b98e00640 1 -- 172.21.15.100:0/4035874395 wait complete. 2024-09-17T04:30:25.638 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-17T04:30:26.639 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone15410 2024-09-17T04:30:26.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.872+0000 7f8a8e600640 1 -- 172.21.15.100:0/1487597257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a880aee00 msgr2=0x7f8a880af210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:26.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.872+0000 7f8a8e600640 1 --2- 172.21.15.100:0/1487597257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a880aee00 0x7f8a880af210 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f8a70005fd0 tx=0x7f8a7002a4b0 comp rx=0 tx=0).stop 2024-09-17T04:30:26.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.873+0000 7f8a8e600640 1 -- 172.21.15.100:0/1487597257 shutdown_connections 2024-09-17T04:30:26.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.873+0000 7f8a8e600640 1 --2- 172.21.15.100:0/1487597257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a880aee00 0x7f8a880af210 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:26.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.873+0000 7f8a8e600640 1 --2- 172.21.15.100:0/1487597257 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a8815f9e0 0x7f8a88147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:26.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.873+0000 7f8a8e600640 1 -- 172.21.15.100:0/1487597257 >> 172.21.15.100:0/1487597257 conn(0x7f8a8806d1c0 msgr2=0x7f8a8806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:26.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.873+0000 7f8a8e600640 1 -- 172.21.15.100:0/1487597257 shutdown_connections 2024-09-17T04:30:26.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.873+0000 7f8a8e600640 1 -- 172.21.15.100:0/1487597257 wait complete. 2024-09-17T04:30:26.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.875+0000 7f8a8e600640 1 Processor -- start 2024-09-17T04:30:26.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.875+0000 7f8a8e600640 1 -- start start 2024-09-17T04:30:26.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.877+0000 7f8a8e600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a88073540 0x7f8a88144110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:26.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.877+0000 7f8a8e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a880aee00 0x7f8a8815dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:26.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.877+0000 7f8a8e600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a8815f9e0 0x7f8a8813e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:26.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.877+0000 7f8a8e600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8806fb60 con 0x7f8a880aee00 2024-09-17T04:30:26.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.877+0000 7f8a8e600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8806f9e0 con 0x7f8a8815f9e0 2024-09-17T04:30:26.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.877+0000 7f8a8e600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8a8806fce0 con 0x7f8a88073540 2024-09-17T04:30:26.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.877+0000 7f8a7ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a880aee00 0x7f8a8815dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:26.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.877+0000 7f8a7ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a880aee00 0x7f8a8815dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49682/0 (socket says 172.21.15.100:49682) 2024-09-17T04:30:26.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.877+0000 7f8a7ea00640 1 -- 172.21.15.100:0/2433895909 learned_addr learned my addr 172.21.15.100:0/2433895909 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:26.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.877+0000 7f8a87400640 1 --2- 172.21.15.100:0/2433895909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a88073540 0x7f8a88144110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:26.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.877+0000 7f8a87e00640 1 --2- 172.21.15.100:0/2433895909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a8815f9e0 0x7f8a8813e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:26.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.877+0000 7f8a7ea00640 1 -- 172.21.15.100:0/2433895909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a88073540 msgr2=0x7f8a88144110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:26.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.877+0000 7f8a7ea00640 1 --2- 172.21.15.100:0/2433895909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a88073540 0x7f8a88144110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:26.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.878+0000 7f8a7ea00640 1 -- 172.21.15.100:0/2433895909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a8815f9e0 msgr2=0x7f8a8813e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:26.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.878+0000 7f8a7ea00640 1 --2- 172.21.15.100:0/2433895909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a8815f9e0 0x7f8a8813e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:26.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.878+0000 7f8a7ea00640 1 -- 172.21.15.100:0/2433895909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a88143c30 con 0x7f8a880aee00 2024-09-17T04:30:26.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.878+0000 7f8a7ea00640 1 --2- 172.21.15.100:0/2433895909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a880aee00 0x7f8a8815dae0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f8a70034040 tx=0x7f8a700038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:26.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.878+0000 7f8a7fe00640 1 -- 172.21.15.100:0/2433895909 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7003e030 con 0x7f8a880aee00 2024-09-17T04:30:26.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.878+0000 7f8a7fe00640 1 -- 172.21.15.100:0/2433895909 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a70003d90 con 0x7f8a880aee00 2024-09-17T04:30:26.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.878+0000 7f8a7fe00640 1 -- 172.21.15.100:0/2433895909 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a70035040 con 0x7f8a880aee00 2024-09-17T04:30:26.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.878+0000 7f8a8e600640 1 -- 172.21.15.100:0/2433895909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a88141510 con 0x7f8a880aee00 2024-09-17T04:30:26.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.878+0000 7f8a8e600640 1 -- 172.21.15.100:0/2433895909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a881f9ae0 con 0x7f8a880aee00 2024-09-17T04:30:26.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.879+0000 7f8a8e600640 1 -- 172.21.15.100:0/2433895909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a8813d9a0 con 0x7f8a880aee00 2024-09-17T04:30:26.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.880+0000 7f8a7fe00640 1 -- 172.21.15.100:0/2433895909 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7f8a70041030 con 0x7f8a880aee00 2024-09-17T04:30:26.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.881+0000 7f8a7fe00640 1 --2- 172.21.15.100:0/2433895909 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8a64078ef0 0x7f8a6407b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:26.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.881+0000 7f8a7fe00640 1 -- 172.21.15.100:0/2433895909 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f8a700bbe10 con 0x7f8a880aee00 2024-09-17T04:30:26.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.882+0000 7f8a87400640 1 --2- 172.21.15.100:0/2433895909 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8a64078ef0 0x7f8a6407b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:26.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.882+0000 7f8a87400640 1 --2- 172.21.15.100:0/2433895909 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8a64078ef0 0x7f8a6407b400 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8a880b3240 tx=0x7f8a7800b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:26.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:26.886+0000 7f8a7fe00640 1 -- 172.21.15.100:0/2433895909 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8a700bf030 con 0x7f8a880aee00 2024-09-17T04:30:27.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:27.104+0000 7f8a8e600640 1 -- 172.21.15.100:0/2433895909 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone15410", "target": ["mon-mgr", ""]}) -- 0x7f8a880af210 con 0x7f8a64078ef0 2024-09-17T04:30:27.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:27.121+0000 7f8a7fe00640 1 -- 172.21.15.100:0/2433895909 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+304 (secure 0 0 0) 0x7f8a880af210 con 0x7f8a64078ef0 2024-09-17T04:30:27.120 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:30:27.121 INFO:teuthology.orchestra.run.smithi100.stdout: "status": { 2024-09-17T04:30:27.121 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "in-progress", 2024-09-17T04:30:27.121 INFO:teuthology.orchestra.run.smithi100.stdout: "source": { 2024-09-17T04:30:27.121 INFO:teuthology.orchestra.run.smithi100.stdout: "volume": "cephfs", 2024-09-17T04:30:27.121 INFO:teuthology.orchestra.run.smithi100.stdout: "subvolume": "subvol80190", 2024-09-17T04:30:27.121 INFO:teuthology.orchestra.run.smithi100.stdout: "snapshot": "subvol_snap99850" 2024-09-17T04:30:27.121 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:30:27.121 INFO:teuthology.orchestra.run.smithi100.stdout: "progress_report": { 2024-09-17T04:30:27.121 INFO:teuthology.orchestra.run.smithi100.stdout: "percentage cloned": "75.003%", 2024-09-17T04:30:27.122 INFO:teuthology.orchestra.run.smithi100.stdout: "amount cloned": "6.0M/8.0M", 2024-09-17T04:30:27.122 INFO:teuthology.orchestra.run.smithi100.stdout: "files cloned": "10/9" 2024-09-17T04:30:27.122 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:30:27.122 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:30:27.122 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:30:27.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:27.123+0000 7f8a8e600640 1 -- 172.21.15.100:0/2433895909 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8a64078ef0 msgr2=0x7f8a6407b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:27.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:27.123+0000 7f8a8e600640 1 --2- 172.21.15.100:0/2433895909 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8a64078ef0 0x7f8a6407b400 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8a880b3240 tx=0x7f8a7800b2e0 comp rx=0 tx=0).stop 2024-09-17T04:30:27.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:27.123+0000 7f8a8e600640 1 -- 172.21.15.100:0/2433895909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a880aee00 msgr2=0x7f8a8815dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:27.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:27.123+0000 7f8a8e600640 1 --2- 172.21.15.100:0/2433895909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a880aee00 0x7f8a8815dae0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f8a70034040 tx=0x7f8a700038a0 comp rx=0 tx=0).stop 2024-09-17T04:30:27.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:27.123+0000 7f8a8e600640 1 -- 172.21.15.100:0/2433895909 shutdown_connections 2024-09-17T04:30:27.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:27.123+0000 7f8a8e600640 1 --2- 172.21.15.100:0/2433895909 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8a64078ef0 0x7f8a6407b400 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:27.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:27.123+0000 7f8a8e600640 1 --2- 172.21.15.100:0/2433895909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a8815f9e0 0x7f8a8813e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:27.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:27.123+0000 7f8a8e600640 1 --2- 172.21.15.100:0/2433895909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a880aee00 0x7f8a8815dae0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:27.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:27.123+0000 7f8a8e600640 1 --2- 172.21.15.100:0/2433895909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a88073540 0x7f8a88144110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:27.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:27.123+0000 7f8a8e600640 1 -- 172.21.15.100:0/2433895909 >> 172.21.15.100:0/2433895909 conn(0x7f8a8806d1c0 msgr2=0x7f8a880b3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:27.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:27.123+0000 7f8a8e600640 1 -- 172.21.15.100:0/2433895909 shutdown_connections 2024-09-17T04:30:27.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:27.123+0000 7f8a8e600640 1 -- 172.21.15.100:0/2433895909 wait complete. 2024-09-17T04:30:27.147 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-17T04:30:28.148 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone15410 2024-09-17T04:30:28.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.374+0000 7f35db200640 1 -- 172.21.15.100:0/73910205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35dc156ca0 msgr2=0x7f35dc0ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:28.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.374+0000 7f35db200640 1 --2- 172.21.15.100:0/73910205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35dc156ca0 0x7f35dc0ccac0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f35c4005fd0 tx=0x7f35c402a5d0 comp rx=0 tx=0).stop 2024-09-17T04:30:28.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.374+0000 7f35db200640 1 -- 172.21.15.100:0/73910205 shutdown_connections 2024-09-17T04:30:28.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.374+0000 7f35db200640 1 --2- 172.21.15.100:0/73910205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35dc0cd000 0x7f35dc0af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:28.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.374+0000 7f35db200640 1 --2- 172.21.15.100:0/73910205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35dc156ca0 0x7f35dc0ccac0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:28.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.374+0000 7f35db200640 1 -- 172.21.15.100:0/73910205 >> 172.21.15.100:0/73910205 conn(0x7f35dc06d620 msgr2=0x7f35dc0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:28.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.374+0000 7f35db200640 1 -- 172.21.15.100:0/73910205 shutdown_connections 2024-09-17T04:30:28.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.374+0000 7f35db200640 1 -- 172.21.15.100:0/73910205 wait complete. 2024-09-17T04:30:28.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.376+0000 7f35db200640 1 Processor -- start 2024-09-17T04:30:28.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.376+0000 7f35db200640 1 -- start start 2024-09-17T04:30:28.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.378+0000 7f35db200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35dc0b3f30 0x7f35dc1543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:28.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.378+0000 7f35db200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35dc0cd000 0x7f35dc154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:28.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.378+0000 7f35db200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35dc156ca0 0x7f35dc1456a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:28.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.378+0000 7f35db200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc0af9a0 con 0x7f35dc0b3f30 2024-09-17T04:30:28.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.378+0000 7f35db200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc0af820 con 0x7f35dc0cd000 2024-09-17T04:30:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.378+0000 7f35db200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f35dc0afb20 con 0x7f35dc156ca0 2024-09-17T04:30:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.378+0000 7f35d9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35dc0b3f30 0x7f35dc1543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.378+0000 7f35d9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35dc0b3f30 0x7f35dc1543b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49692/0 (socket says 172.21.15.100:49692) 2024-09-17T04:30:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.378+0000 7f35d9e00640 1 -- 172.21.15.100:0/1565288008 learned_addr learned my addr 172.21.15.100:0/1565288008 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.378+0000 7f35d1400640 1 --2- 172.21.15.100:0/1565288008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35dc0cd000 0x7f35dc154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.378+0000 7f35da800640 1 --2- 172.21.15.100:0/1565288008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35dc156ca0 0x7f35dc1456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.378+0000 7f35d9e00640 1 -- 172.21.15.100:0/1565288008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35dc156ca0 msgr2=0x7f35dc1456a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.378+0000 7f35d9e00640 1 --2- 172.21.15.100:0/1565288008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35dc156ca0 0x7f35dc1456a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.378+0000 7f35d9e00640 1 -- 172.21.15.100:0/1565288008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35dc0cd000 msgr2=0x7f35dc154a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.378+0000 7f35d9e00640 1 --2- 172.21.15.100:0/1565288008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35dc0cd000 0x7f35dc154a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.378+0000 7f35d9e00640 1 -- 172.21.15.100:0/1565288008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35dc155590 con 0x7f35dc0b3f30 2024-09-17T04:30:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.379+0000 7f35d9e00640 1 --2- 172.21.15.100:0/1565288008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35dc0b3f30 0x7f35dc1543b0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00c3a0 tx=0x7f35cc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.379+0000 7f35d2a00640 1 -- 172.21.15.100:0/1565288008 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc0052d0 con 0x7f35dc0b3f30 2024-09-17T04:30:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.379+0000 7f35d2a00640 1 -- 172.21.15.100:0/1565288008 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35cc007020 con 0x7f35dc0b3f30 2024-09-17T04:30:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.379+0000 7f35db200640 1 -- 172.21.15.100:0/1565288008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35dc155870 con 0x7f35dc0b3f30 2024-09-17T04:30:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.379+0000 7f35d2a00640 1 -- 172.21.15.100:0/1565288008 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc007470 con 0x7f35dc0b3f30 2024-09-17T04:30:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.379+0000 7f35db200640 1 -- 172.21.15.100:0/1565288008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35dc155e30 con 0x7f35dc0b3f30 2024-09-17T04:30:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.380+0000 7f35db200640 1 -- 172.21.15.100:0/1565288008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35dc0af0d0 con 0x7f35dc0b3f30 2024-09-17T04:30:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.381+0000 7f35d2a00640 1 -- 172.21.15.100:0/1565288008 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7f35cc0067d0 con 0x7f35dc0b3f30 2024-09-17T04:30:28.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.382+0000 7f35d2a00640 1 --2- 172.21.15.100:0/1565288008 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f35b8078f20 0x7f35b807b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:28.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.382+0000 7f35d2a00640 1 -- 172.21.15.100:0/1565288008 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f35cc09a8a0 con 0x7f35dc0b3f30 2024-09-17T04:30:28.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.383+0000 7f35d1400640 1 --2- 172.21.15.100:0/1565288008 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f35b8078f20 0x7f35b807b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:28.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.383+0000 7f35d1400640 1 --2- 172.21.15.100:0/1565288008 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f35b8078f20 0x7f35b807b430 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f35c4033000 tx=0x7f35c4004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:28.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.387+0000 7f35d2a00640 1 -- 172.21.15.100:0/1565288008 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f35cc062740 con 0x7f35dc0b3f30 2024-09-17T04:30:28.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.605+0000 7f35db200640 1 -- 172.21.15.100:0/1565288008 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone15410", "target": ["mon-mgr", ""]}) -- 0x7f35dc09d220 con 0x7f35b8078f20 2024-09-17T04:30:28.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.627+0000 7f35d2a00640 1 -- 172.21.15.100:0/1565288008 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+304 (secure 0 0 0) 0x7f35dc09d220 con 0x7f35b8078f20 2024-09-17T04:30:28.626 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:30:28.626 INFO:teuthology.orchestra.run.smithi100.stdout: "status": { 2024-09-17T04:30:28.626 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "in-progress", 2024-09-17T04:30:28.627 INFO:teuthology.orchestra.run.smithi100.stdout: "source": { 2024-09-17T04:30:28.627 INFO:teuthology.orchestra.run.smithi100.stdout: "volume": "cephfs", 2024-09-17T04:30:28.627 INFO:teuthology.orchestra.run.smithi100.stdout: "subvolume": "subvol80190", 2024-09-17T04:30:28.627 INFO:teuthology.orchestra.run.smithi100.stdout: "snapshot": "subvol_snap99850" 2024-09-17T04:30:28.627 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:30:28.627 INFO:teuthology.orchestra.run.smithi100.stdout: "progress_report": { 2024-09-17T04:30:28.627 INFO:teuthology.orchestra.run.smithi100.stdout: "percentage cloned": "75.003%", 2024-09-17T04:30:28.627 INFO:teuthology.orchestra.run.smithi100.stdout: "amount cloned": "6.0M/8.0M", 2024-09-17T04:30:28.627 INFO:teuthology.orchestra.run.smithi100.stdout: "files cloned": "10/9" 2024-09-17T04:30:28.627 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:30:28.627 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:30:28.627 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:30:28.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.629+0000 7f35db200640 1 -- 172.21.15.100:0/1565288008 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f35b8078f20 msgr2=0x7f35b807b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:28.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.629+0000 7f35db200640 1 --2- 172.21.15.100:0/1565288008 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f35b8078f20 0x7f35b807b430 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f35c4033000 tx=0x7f35c4004730 comp rx=0 tx=0).stop 2024-09-17T04:30:28.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.629+0000 7f35db200640 1 -- 172.21.15.100:0/1565288008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35dc0b3f30 msgr2=0x7f35dc1543b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:28.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.629+0000 7f35db200640 1 --2- 172.21.15.100:0/1565288008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35dc0b3f30 0x7f35dc1543b0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00c3a0 tx=0x7f35cc00c860 comp rx=0 tx=0).stop 2024-09-17T04:30:28.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.629+0000 7f35db200640 1 -- 172.21.15.100:0/1565288008 shutdown_connections 2024-09-17T04:30:28.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.629+0000 7f35db200640 1 --2- 172.21.15.100:0/1565288008 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f35b8078f20 0x7f35b807b430 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:28.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.629+0000 7f35db200640 1 --2- 172.21.15.100:0/1565288008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35dc156ca0 0x7f35dc1456a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:28.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.629+0000 7f35db200640 1 --2- 172.21.15.100:0/1565288008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35dc0cd000 0x7f35dc154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:28.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.629+0000 7f35db200640 1 --2- 172.21.15.100:0/1565288008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35dc0b3f30 0x7f35dc1543b0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:28.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.629+0000 7f35db200640 1 -- 172.21.15.100:0/1565288008 >> 172.21.15.100:0/1565288008 conn(0x7f35dc06d620 msgr2=0x7f35dc0701e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:28.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.630+0000 7f35db200640 1 -- 172.21.15.100:0/1565288008 shutdown_connections 2024-09-17T04:30:28.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:28.630+0000 7f35db200640 1 -- 172.21.15.100:0/1565288008 wait complete. 2024-09-17T04:30:28.653 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-17T04:30:29.654 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone15410 2024-09-17T04:30:29.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.885+0000 7f3b40600640 1 -- 172.21.15.100:0/1468642967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b38145550 msgr2=0x7f3b3806fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:29.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.885+0000 7f3b40600640 1 --2- 172.21.15.100:0/1468642967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b38145550 0x7f3b3806fb40 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009830 tx=0x7f3b2c02a640 comp rx=0 tx=0).stop 2024-09-17T04:30:29.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.886+0000 7f3b40600640 1 -- 172.21.15.100:0/1468642967 shutdown_connections 2024-09-17T04:30:29.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.886+0000 7f3b40600640 1 --2- 172.21.15.100:0/1468642967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b38145550 0x7f3b3806fb40 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:29.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.886+0000 7f3b40600640 1 -- 172.21.15.100:0/1468642967 >> 172.21.15.100:0/1468642967 conn(0x7f3b3806d1c0 msgr2=0x7f3b3806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:29.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.886+0000 7f3b40600640 1 -- 172.21.15.100:0/1468642967 shutdown_connections 2024-09-17T04:30:29.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.886+0000 7f3b40600640 1 -- 172.21.15.100:0/1468642967 wait complete. 2024-09-17T04:30:29.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.888+0000 7f3b40600640 1 Processor -- start 2024-09-17T04:30:29.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.888+0000 7f3b40600640 1 -- start start 2024-09-17T04:30:29.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.889+0000 7f3b40600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b38145550 0x7f3b381f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:29.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.889+0000 7f3b40600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b38145a40 0x7f3b3815c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:29.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.889+0000 7f3b40600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b380af3c0 0x7f3b3815ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:29.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.889+0000 7f3b40600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3b380ce360 con 0x7f3b38145550 2024-09-17T04:30:29.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.889+0000 7f3b40600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3b380ce1e0 con 0x7f3b380af3c0 2024-09-17T04:30:29.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.889+0000 7f3b40600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3b380ce4e0 con 0x7f3b38145a40 2024-09-17T04:30:29.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.890+0000 7f3b3dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b38145550 0x7f3b381f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:29.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.890+0000 7f3b3dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b38145550 0x7f3b381f53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49696/0 (socket says 172.21.15.100:49696) 2024-09-17T04:30:29.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.890+0000 7f3b3dc00640 1 -- 172.21.15.100:0/5773693 learned_addr learned my addr 172.21.15.100:0/5773693 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:29.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.890+0000 7f3b3dc00640 1 -- 172.21.15.100:0/5773693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b38145a40 msgr2=0x7f3b3815c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:29.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.890+0000 7f3b3dc00640 1 --2- 172.21.15.100:0/5773693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b38145a40 0x7f3b3815c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:29.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.890+0000 7f3b3dc00640 1 -- 172.21.15.100:0/5773693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b380af3c0 msgr2=0x7f3b3815ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:29.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.890+0000 7f3b3dc00640 1 --2- 172.21.15.100:0/5773693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b380af3c0 0x7f3b3815ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:29.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.890+0000 7f3b3dc00640 1 -- 172.21.15.100:0/5773693 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b380ae320 con 0x7f3b38145550 2024-09-17T04:30:29.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.890+0000 7f3b3dc00640 1 --2- 172.21.15.100:0/5773693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b38145550 0x7f3b381f53a0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c02ac30 tx=0x7f3b2c033df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:29.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.891+0000 7f3b35600640 1 -- 172.21.15.100:0/5773693 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c0332e0 con 0x7f3b38145550 2024-09-17T04:30:29.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.891+0000 7f3b35600640 1 -- 172.21.15.100:0/5773693 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b2c033570 con 0x7f3b38145550 2024-09-17T04:30:29.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.891+0000 7f3b40600640 1 -- 172.21.15.100:0/5773693 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b380ae600 con 0x7f3b38145550 2024-09-17T04:30:29.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.891+0000 7f3b35600640 1 -- 172.21.15.100:0/5773693 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c005280 con 0x7f3b38145550 2024-09-17T04:30:29.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.891+0000 7f3b40600640 1 -- 172.21.15.100:0/5773693 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b380aebc0 con 0x7f3b38145550 2024-09-17T04:30:29.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.892+0000 7f3b40600640 1 -- 172.21.15.100:0/5773693 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b3806fb40 con 0x7f3b38145550 2024-09-17T04:30:29.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.893+0000 7f3b35600640 1 -- 172.21.15.100:0/5773693 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7f3b2c005420 con 0x7f3b38145550 2024-09-17T04:30:29.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.894+0000 7f3b35600640 1 --2- 172.21.15.100:0/5773693 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3b18078f20 0x7f3b1807b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:29.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.894+0000 7f3b35600640 1 -- 172.21.15.100:0/5773693 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f3b2c0bc540 con 0x7f3b38145550 2024-09-17T04:30:29.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.894+0000 7f3b3d200640 1 --2- 172.21.15.100:0/5773693 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3b18078f20 0x7f3b1807b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:29.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.895+0000 7f3b3d200640 1 --2- 172.21.15.100:0/5773693 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3b18078f20 0x7f3b1807b430 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3b381f9aa0 tx=0x7f3b24007970 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:29.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:29.899+0000 7f3b35600640 1 -- 172.21.15.100:0/5773693 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3b2c03f030 con 0x7f3b38145550 2024-09-17T04:30:30.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:30.151+0000 7f3b40600640 1 -- 172.21.15.100:0/5773693 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone15410", "target": ["mon-mgr", ""]}) -- 0x7f3b380b22d0 con 0x7f3b18078f20 2024-09-17T04:30:34.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:34.975+0000 7f3b35600640 1 -- 172.21.15.100:0/5773693 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f3b380b22d0 con 0x7f3b18078f20 2024-09-17T04:30:34.976 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:30:34.976 INFO:teuthology.orchestra.run.smithi100.stdout: "status": { 2024-09-17T04:30:34.976 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "complete" 2024-09-17T04:30:34.977 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:30:34.977 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:30:34.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:34.979+0000 7f3b40600640 1 -- 172.21.15.100:0/5773693 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3b18078f20 msgr2=0x7f3b1807b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:34.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:34.979+0000 7f3b40600640 1 --2- 172.21.15.100:0/5773693 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3b18078f20 0x7f3b1807b430 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3b381f9aa0 tx=0x7f3b24007970 comp rx=0 tx=0).stop 2024-09-17T04:30:34.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:34.979+0000 7f3b40600640 1 -- 172.21.15.100:0/5773693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b38145550 msgr2=0x7f3b381f53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:34.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:34.979+0000 7f3b40600640 1 --2- 172.21.15.100:0/5773693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b38145550 0x7f3b381f53a0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c02ac30 tx=0x7f3b2c033df0 comp rx=0 tx=0).stop 2024-09-17T04:30:34.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:34.979+0000 7f3b40600640 1 -- 172.21.15.100:0/5773693 shutdown_connections 2024-09-17T04:30:34.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:34.979+0000 7f3b40600640 1 --2- 172.21.15.100:0/5773693 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3b18078f20 0x7f3b1807b430 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:34.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:34.979+0000 7f3b40600640 1 --2- 172.21.15.100:0/5773693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b380af3c0 0x7f3b3815ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:34.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:34.979+0000 7f3b40600640 1 --2- 172.21.15.100:0/5773693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b38145a40 0x7f3b3815c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:34.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:34.979+0000 7f3b40600640 1 --2- 172.21.15.100:0/5773693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b38145550 0x7f3b381f53a0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:34.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:34.979+0000 7f3b40600640 1 -- 172.21.15.100:0/5773693 >> 172.21.15.100:0/5773693 conn(0x7f3b3806d1c0 msgr2=0x7f3b38142bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:34.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:34.979+0000 7f3b40600640 1 -- 172.21.15.100:0/5773693 shutdown_connections 2024-09-17T04:30:34.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:34.979+0000 7f3b40600640 1 -- 172.21.15.100:0/5773693 wait complete. 2024-09-17T04:30:35.006 DEBUG:tasks.cephfs.test_volumes:current clone state = complete 2024-09-17T04:30:35.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:35.007 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone15410/.meta) 2024-09-17T04:30:35.066 INFO:teuthology.orchestra.run.smithi100.stdout:[GLOBAL] 2024-09-17T04:30:35.066 INFO:teuthology.orchestra.run.smithi100.stdout:version = 2 2024-09-17T04:30:35.066 INFO:teuthology.orchestra.run.smithi100.stdout:type = clone 2024-09-17T04:30:35.066 INFO:teuthology.orchestra.run.smithi100.stdout:path = /volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c 2024-09-17T04:30:35.066 INFO:teuthology.orchestra.run.smithi100.stdout:state = complete 2024-09-17T04:30:35.066 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:30:35.069 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol80190 2024-09-17T04:30:35.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.295+0000 7f8b7e000640 1 -- 172.21.15.100:0/4029751966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b78145a40 msgr2=0x7f8b78073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:35.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.295+0000 7f8b7e000640 1 --2- 172.21.15.100:0/4029751966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b78145a40 0x7f8b78073540 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f8b781464e0 tx=0x7f8b6002a5f0 comp rx=0 tx=0).stop 2024-09-17T04:30:35.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.295+0000 7f8b7e000640 1 -- 172.21.15.100:0/4029751966 shutdown_connections 2024-09-17T04:30:35.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.295+0000 7f8b7e000640 1 --2- 172.21.15.100:0/4029751966 >> v2:172.21.15.184:3300/0 conn(0x7f8b78073a80 0x7f8b780cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:35.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.295+0000 7f8b7e000640 1 --2- 172.21.15.100:0/4029751966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b78145a40 0x7f8b78073540 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:35.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.295+0000 7f8b7e000640 1 -- 172.21.15.100:0/4029751966 >> 172.21.15.100:0/4029751966 conn(0x7f8b7806d1c0 msgr2=0x7f8b7806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:35.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.295+0000 7f8b7e000640 1 -- 172.21.15.100:0/4029751966 shutdown_connections 2024-09-17T04:30:35.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.296+0000 7f8b7e000640 1 -- 172.21.15.100:0/4029751966 wait complete. 2024-09-17T04:30:35.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.298+0000 7f8b7e000640 1 Processor -- start 2024-09-17T04:30:35.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.298+0000 7f8b7e000640 1 -- start start 2024-09-17T04:30:35.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.299+0000 7f8b7e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b78073a80 0x7f8b781f9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:35.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.299+0000 7f8b7e000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b78145550 0x7f8b781409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:35.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.299+0000 7f8b7e000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b78145a40 0x7f8b780aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:35.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.299+0000 7f8b7e000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8b780ce0e0 con 0x7f8b78073a80 2024-09-17T04:30:35.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.299+0000 7f8b7e000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8b780cdf60 con 0x7f8b78145550 2024-09-17T04:30:35.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.299+0000 7f8b7e000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8b780ce260 con 0x7f8b78145a40 2024-09-17T04:30:35.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.299+0000 7f8b77400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b78073a80 0x7f8b781f9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:35.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.299+0000 7f8b77400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b78073a80 0x7f8b781f9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58488/0 (socket says 172.21.15.100:58488) 2024-09-17T04:30:35.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.299+0000 7f8b77400640 1 -- 172.21.15.100:0/3415907765 learned_addr learned my addr 172.21.15.100:0/3415907765 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:35.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.300+0000 7f8b77400640 1 -- 172.21.15.100:0/3415907765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b78145a40 msgr2=0x7f8b780aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:35.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.300+0000 7f8b77400640 1 --2- 172.21.15.100:0/3415907765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b78145a40 0x7f8b780aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:35.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.300+0000 7f8b77400640 1 -- 172.21.15.100:0/3415907765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b78145550 msgr2=0x7f8b781409d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:35.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.300+0000 7f8b77400640 1 --2- 172.21.15.100:0/3415907765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b78145550 0x7f8b781409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:35.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.300+0000 7f8b77400640 1 -- 172.21.15.100:0/3415907765 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b781fa020 con 0x7f8b78073a80 2024-09-17T04:30:35.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.300+0000 7f8b77400640 1 --2- 172.21.15.100:0/3415907765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b78073a80 0x7f8b781f9160 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800b3f0 tx=0x7f8b6800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:35.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.300+0000 7f8b6f400640 1 -- 172.21.15.100:0/3415907765 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68015020 con 0x7f8b78073a80 2024-09-17T04:30:35.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.300+0000 7f8b6f400640 1 -- 172.21.15.100:0/3415907765 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b68011920 con 0x7f8b78073a80 2024-09-17T04:30:35.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.300+0000 7f8b7e000640 1 -- 172.21.15.100:0/3415907765 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b781f9d40 con 0x7f8b78073a80 2024-09-17T04:30:35.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.300+0000 7f8b6f400640 1 -- 172.21.15.100:0/3415907765 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68011d70 con 0x7f8b78073a80 2024-09-17T04:30:35.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.300+0000 7f8b7e000640 1 -- 172.21.15.100:0/3415907765 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b781e7e70 con 0x7f8b78073a80 2024-09-17T04:30:35.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.302+0000 7f8b7e000640 1 -- 172.21.15.100:0/3415907765 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b780ce7a0 con 0x7f8b78073a80 2024-09-17T04:30:35.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.303+0000 7f8b6f400640 1 -- 172.21.15.100:0/3415907765 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7f8b68018020 con 0x7f8b78073a80 2024-09-17T04:30:35.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.304+0000 7f8b6f400640 1 --2- 172.21.15.100:0/3415907765 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8b54078f20 0x7f8b5407b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:35.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.304+0000 7f8b6f400640 1 -- 172.21.15.100:0/3415907765 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f8b6809b000 con 0x7f8b78073a80 2024-09-17T04:30:35.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.305+0000 7f8b6fe00640 1 --2- 172.21.15.100:0/3415907765 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8b54078f20 0x7f8b5407b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:35.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.305+0000 7f8b6fe00640 1 --2- 172.21.15.100:0/3415907765 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8b54078f20 0x7f8b5407b430 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8b60033000 tx=0x7f8b600046c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:35.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.309+0000 7f8b6f400640 1 -- 172.21.15.100:0/3415907765 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8b6801f030 con 0x7f8b78073a80 2024-09-17T04:30:35.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.518+0000 7f8b7e000640 1 -- 172.21.15.100:0/3415907765 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol80190", "target": ["mon-mgr", ""]}) -- 0x7f8b7815ca00 con 0x7f8b54078f20 2024-09-17T04:30:35.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.532+0000 7f8b6f400640 1 -- 172.21.15.100:0/3415907765 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f8b7815ca00 con 0x7f8b54078f20 2024-09-17T04:30:35.532 INFO:teuthology.orchestra.run.smithi100.stdout:/volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4 2024-09-17T04:30:35.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.534+0000 7f8b7e000640 1 -- 172.21.15.100:0/3415907765 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8b54078f20 msgr2=0x7f8b5407b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:35.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.534+0000 7f8b7e000640 1 --2- 172.21.15.100:0/3415907765 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8b54078f20 0x7f8b5407b430 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8b60033000 tx=0x7f8b600046c0 comp rx=0 tx=0).stop 2024-09-17T04:30:35.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.534+0000 7f8b7e000640 1 -- 172.21.15.100:0/3415907765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b78073a80 msgr2=0x7f8b781f9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:35.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.534+0000 7f8b7e000640 1 --2- 172.21.15.100:0/3415907765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b78073a80 0x7f8b781f9160 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800b3f0 tx=0x7f8b6800b8b0 comp rx=0 tx=0).stop 2024-09-17T04:30:35.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.534+0000 7f8b7e000640 1 -- 172.21.15.100:0/3415907765 shutdown_connections 2024-09-17T04:30:35.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.534+0000 7f8b7e000640 1 --2- 172.21.15.100:0/3415907765 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8b54078f20 0x7f8b5407b430 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:35.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.534+0000 7f8b7e000640 1 --2- 172.21.15.100:0/3415907765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b78145a40 0x7f8b780aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:35.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.534+0000 7f8b7e000640 1 --2- 172.21.15.100:0/3415907765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b78145550 0x7f8b781409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:35.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.534+0000 7f8b7e000640 1 --2- 172.21.15.100:0/3415907765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b78073a80 0x7f8b781f9160 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:35.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.534+0000 7f8b7e000640 1 -- 172.21.15.100:0/3415907765 >> 172.21.15.100:0/3415907765 conn(0x7f8b7806d1c0 msgr2=0x7f8b78144100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:35.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.534+0000 7f8b7e000640 1 -- 172.21.15.100:0/3415907765 shutdown_connections 2024-09-17T04:30:35.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.534+0000 7f8b7e000640 1 -- 172.21.15.100:0/3415907765 wait complete. 2024-09-17T04:30:35.557 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol_clone15410 2024-09-17T04:30:35.767 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:30:35.771 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:30:35.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.782+0000 7fa4cfe00640 1 -- 172.21.15.100:0/165047502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4c815ee80 msgr2=0x7fa4c80c7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:35.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.782+0000 7fa4cfe00640 1 --2- 172.21.15.100:0/165047502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4c815ee80 0x7fa4c80c7730 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009860 tx=0x7fa4bc02a5a0 comp rx=0 tx=0).stop 2024-09-17T04:30:35.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.782+0000 7fa4cfe00640 1 -- 172.21.15.100:0/165047502 shutdown_connections 2024-09-17T04:30:35.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.782+0000 7fa4cfe00640 1 --2- 172.21.15.100:0/165047502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4c815ee80 0x7fa4c80c7730 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:35.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.782+0000 7fa4cfe00640 1 -- 172.21.15.100:0/165047502 >> 172.21.15.100:0/165047502 conn(0x7fa4c80b2ff0 msgr2=0x7fa4c80b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:35.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.783+0000 7fa4cfe00640 1 -- 172.21.15.100:0/165047502 shutdown_connections 2024-09-17T04:30:35.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.783+0000 7fa4cfe00640 1 -- 172.21.15.100:0/165047502 wait complete. 2024-09-17T04:30:35.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.785+0000 7fa4cfe00640 1 Processor -- start 2024-09-17T04:30:35.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.785+0000 7fa4cfe00640 1 -- start start 2024-09-17T04:30:35.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.786+0000 7fa4cfe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4c80b6b80 0x7fa4c814d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:35.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.786+0000 7fa4cfe00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4c80d0ee0 0x7fa4c815ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:35.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.787+0000 7fa4cfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4c815ee80 0x7fa4c815b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:35.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.787+0000 7fa4cfe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c80c7df0 con 0x7fa4c815ee80 2024-09-17T04:30:35.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.787+0000 7fa4cfe00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c80c7c70 con 0x7fa4c80b6b80 2024-09-17T04:30:35.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.787+0000 7fa4cfe00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa4c80c7f70 con 0x7fa4c80d0ee0 2024-09-17T04:30:35.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.787+0000 7fa4cde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4c815ee80 0x7fa4c815b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:35.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.787+0000 7fa4cde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4c815ee80 0x7fa4c815b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58502/0 (socket says 172.21.15.100:58502) 2024-09-17T04:30:35.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.787+0000 7fa4cde00640 1 -- 172.21.15.100:0/2369374988 learned_addr learned my addr 172.21.15.100:0/2369374988 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:35.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.787+0000 7fa4cd400640 1 --2- 172.21.15.100:0/2369374988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4c80b6b80 0x7fa4c814d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:35.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.787+0000 7fa4cca00640 1 --2- 172.21.15.100:0/2369374988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4c80d0ee0 0x7fa4c815ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:35.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.787+0000 7fa4cde00640 1 -- 172.21.15.100:0/2369374988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4c80d0ee0 msgr2=0x7fa4c815ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:35.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.787+0000 7fa4cde00640 1 --2- 172.21.15.100:0/2369374988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4c80d0ee0 0x7fa4c815ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:35.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.787+0000 7fa4cde00640 1 -- 172.21.15.100:0/2369374988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4c80b6b80 msgr2=0x7fa4c814d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:35.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.787+0000 7fa4cde00640 1 --2- 172.21.15.100:0/2369374988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4c80b6b80 0x7fa4c814d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:35.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.787+0000 7fa4cde00640 1 -- 172.21.15.100:0/2369374988 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c8161280 con 0x7fa4c815ee80 2024-09-17T04:30:35.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.787+0000 7fa4cca00640 1 --2- 172.21.15.100:0/2369374988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4c80d0ee0 0x7fa4c815ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:30:35.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.788+0000 7fa4cde00640 1 --2- 172.21.15.100:0/2369374988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4c815ee80 0x7fa4c815b2c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0002230 tx=0x7fa4c000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:35.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.788+0000 7fa4c5600640 1 -- 172.21.15.100:0/2369374988 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0016020 con 0x7fa4c815ee80 2024-09-17T04:30:35.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.788+0000 7fa4c5600640 1 -- 172.21.15.100:0/2369374988 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4c000ace0 con 0x7fa4c815ee80 2024-09-17T04:30:35.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.788+0000 7fa4c5600640 1 -- 172.21.15.100:0/2369374988 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0012680 con 0x7fa4c815ee80 2024-09-17T04:30:35.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.788+0000 7fa4cfe00640 1 -- 172.21.15.100:0/2369374988 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4c814d760 con 0x7fa4c815ee80 2024-09-17T04:30:35.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.788+0000 7fa4cfe00640 1 -- 172.21.15.100:0/2369374988 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4c8160fa0 con 0x7fa4c815ee80 2024-09-17T04:30:35.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.789+0000 7fa4cfe00640 1 -- 172.21.15.100:0/2369374988 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4c8160a10 con 0x7fa4c815ee80 2024-09-17T04:30:35.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.791+0000 7fa4c5600640 1 -- 172.21.15.100:0/2369374988 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7fa4c0019020 con 0x7fa4c815ee80 2024-09-17T04:30:35.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.792+0000 7fa4c5600640 1 --2- 172.21.15.100:0/2369374988 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa4a8078ef0 0x7fa4a807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:35.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.792+0000 7fa4c5600640 1 -- 172.21.15.100:0/2369374988 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7fa4c009ba60 con 0x7fa4c815ee80 2024-09-17T04:30:35.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.792+0000 7fa4cd400640 1 --2- 172.21.15.100:0/2369374988 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa4a8078ef0 0x7fa4a807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:35.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.793+0000 7fa4cd400640 1 --2- 172.21.15.100:0/2369374988 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa4a8078ef0 0x7fa4a807b400 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc034000 tx=0x7fa4bc004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:35.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:35.796+0000 7fa4c5600640 1 -- 172.21.15.100:0/2369374988 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa4c0020020 con 0x7fa4c815ee80 2024-09-17T04:30:36.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.011+0000 7fa4cfe00640 1 -- 172.21.15.100:0/2369374988 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol_clone15410", "target": ["mon-mgr", ""]}) -- 0x7fa4c80c7730 con 0x7fa4a8078ef0 2024-09-17T04:30:36.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.016+0000 7fa4c5600640 1 -- 172.21.15.100:0/2369374988 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+73 (secure 0 0 0) 0x7fa4c80c7730 con 0x7fa4a8078ef0 2024-09-17T04:30:36.017 INFO:teuthology.orchestra.run.smithi100.stdout:/volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c 2024-09-17T04:30:36.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.019+0000 7fa4cfe00640 1 -- 172.21.15.100:0/2369374988 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa4a8078ef0 msgr2=0x7fa4a807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:36.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.019+0000 7fa4cfe00640 1 --2- 172.21.15.100:0/2369374988 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa4a8078ef0 0x7fa4a807b400 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc034000 tx=0x7fa4bc004730 comp rx=0 tx=0).stop 2024-09-17T04:30:36.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.019+0000 7fa4cfe00640 1 -- 172.21.15.100:0/2369374988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4c815ee80 msgr2=0x7fa4c815b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:36.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.019+0000 7fa4cfe00640 1 --2- 172.21.15.100:0/2369374988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4c815ee80 0x7fa4c815b2c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0002230 tx=0x7fa4c000cf50 comp rx=0 tx=0).stop 2024-09-17T04:30:36.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.019+0000 7fa4cfe00640 1 -- 172.21.15.100:0/2369374988 shutdown_connections 2024-09-17T04:30:36.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.019+0000 7fa4cfe00640 1 --2- 172.21.15.100:0/2369374988 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa4a8078ef0 0x7fa4a807b400 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:36.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.019+0000 7fa4cfe00640 1 --2- 172.21.15.100:0/2369374988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4c815ee80 0x7fa4c815b2c0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:36.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.019+0000 7fa4cfe00640 1 --2- 172.21.15.100:0/2369374988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4c80d0ee0 0x7fa4c815ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:36.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.019+0000 7fa4cfe00640 1 --2- 172.21.15.100:0/2369374988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4c80b6b80 0x7fa4c814d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:36.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.019+0000 7fa4cfe00640 1 -- 172.21.15.100:0/2369374988 >> 172.21.15.100:0/2369374988 conn(0x7fa4c80b2ff0 msgr2=0x7fa4c815e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:36.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.019+0000 7fa4cfe00640 1 -- 172.21.15.100:0/2369374988 shutdown_connections 2024-09-17T04:30:36.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.019+0000 7fa4cfe00640 1 -- 172.21.15.100:0/2369374988 wait complete. 2024-09-17T04:30:36.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:36.042 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850) 2024-09-17T04:30:36.079 INFO:teuthology.orchestra.run.smithi100.stdout:9 2024-09-17T04:30:36.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:36.080 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c) 2024-09-17T04:30:36.158 INFO:teuthology.orchestra.run.smithi100.stdout:9 2024-09-17T04:30:36.158 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol_clone15410 2024-09-17T04:30:36.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.420+0000 7f78c8a00640 1 -- 172.21.15.100:0/2476980031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78c40cd1d0 msgr2=0x7f78c40af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:36.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.420+0000 7f78c8a00640 1 --2- 172.21.15.100:0/2476980031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78c40cd1d0 0x7f78c40af0d0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f78bc005fd0 tx=0x7f78bc02a8b0 comp rx=0 tx=0).stop 2024-09-17T04:30:36.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.420+0000 7f78c8a00640 1 -- 172.21.15.100:0/2476980031 shutdown_connections 2024-09-17T04:30:36.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.420+0000 7f78c8a00640 1 --2- 172.21.15.100:0/2476980031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78c40cd1d0 0x7f78c40af0d0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:36.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.420+0000 7f78c8a00640 1 --2- 172.21.15.100:0/2476980031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c40b3f30 0x7f78c4156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:36.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.420+0000 7f78c8a00640 1 -- 172.21.15.100:0/2476980031 >> 172.21.15.100:0/2476980031 conn(0x7f78c406d620 msgr2=0x7f78c40b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:36.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.420+0000 7f78c8a00640 1 -- 172.21.15.100:0/2476980031 shutdown_connections 2024-09-17T04:30:36.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.420+0000 7f78c8a00640 1 -- 172.21.15.100:0/2476980031 wait complete. 2024-09-17T04:30:36.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.422+0000 7f78c8a00640 1 Processor -- start 2024-09-17T04:30:36.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.422+0000 7f78c8a00640 1 -- start start 2024-09-17T04:30:36.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.424+0000 7f78c8a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78c40b3f30 0x7f78c41456b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:36.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.424+0000 7f78c8a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78c40cd1d0 0x7f78c415b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:36.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.424+0000 7f78c8a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c415d6e0 0x7f78c415bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:36.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.424+0000 7f78c8a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f78c40af9a0 con 0x7f78c40b3f30 2024-09-17T04:30:36.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.424+0000 7f78c8a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78c40af820 con 0x7f78c40cd1d0 2024-09-17T04:30:36.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.424+0000 7f78c8a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f78c40afb20 con 0x7f78c415d6e0 2024-09-17T04:30:36.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.424+0000 7f78c3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78c40b3f30 0x7f78c41456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:36.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.424+0000 7f78c3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78c40b3f30 0x7f78c41456b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58530/0 (socket says 172.21.15.100:58530) 2024-09-17T04:30:36.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.424+0000 7f78c3400640 1 -- 172.21.15.100:0/1896674200 learned_addr learned my addr 172.21.15.100:0/1896674200 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:36.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.424+0000 7f78c3e00640 1 --2- 172.21.15.100:0/1896674200 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c415d6e0 0x7f78c415bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:36.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.424+0000 7f78c3400640 1 -- 172.21.15.100:0/1896674200 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c415d6e0 msgr2=0x7f78c415bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:36.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.424+0000 7f78c3400640 1 --2- 172.21.15.100:0/1896674200 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c415d6e0 0x7f78c415bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:36.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.424+0000 7f78c3400640 1 -- 172.21.15.100:0/1896674200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78c40cd1d0 msgr2=0x7f78c415b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:36.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.424+0000 7f78baa00640 1 --2- 172.21.15.100:0/1896674200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78c40cd1d0 0x7f78c415b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:36.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.424+0000 7f78c3400640 1 --2- 172.21.15.100:0/1896674200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78c40cd1d0 0x7f78c415b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:36.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.425+0000 7f78c3400640 1 -- 172.21.15.100:0/1896674200 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c415ca30 con 0x7f78c40b3f30 2024-09-17T04:30:36.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.425+0000 7f78baa00640 1 --2- 172.21.15.100:0/1896674200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78c40cd1d0 0x7f78c415b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:30:36.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.425+0000 7f78c3400640 1 --2- 172.21.15.100:0/1896674200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78c40b3f30 0x7f78c41456b0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f78b400d740 tx=0x7f78b400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:36.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.425+0000 7f78bbe00640 1 -- 172.21.15.100:0/1896674200 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4017070 con 0x7f78c40b3f30 2024-09-17T04:30:36.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.425+0000 7f78bbe00640 1 -- 172.21.15.100:0/1896674200 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78b4009cc0 con 0x7f78c40b3f30 2024-09-17T04:30:36.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.425+0000 7f78bbe00640 1 -- 172.21.15.100:0/1896674200 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b40106c0 con 0x7f78c40b3f30 2024-09-17T04:30:36.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.425+0000 7f78c8a00640 1 -- 172.21.15.100:0/1896674200 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c41f5950 con 0x7f78c40b3f30 2024-09-17T04:30:36.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.425+0000 7f78c8a00640 1 -- 172.21.15.100:0/1896674200 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c409ddf0 con 0x7f78c40b3f30 2024-09-17T04:30:36.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.426+0000 7f78c8a00640 1 -- 172.21.15.100:0/1896674200 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c4156ca0 con 0x7f78c40b3f30 2024-09-17T04:30:36.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.427+0000 7f78bbe00640 1 -- 172.21.15.100:0/1896674200 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7f78b401a070 con 0x7f78c40b3f30 2024-09-17T04:30:36.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.428+0000 7f78bbe00640 1 --2- 172.21.15.100:0/1896674200 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f78a0078ef0 0x7f78a007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:36.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.429+0000 7f78bbe00640 1 -- 172.21.15.100:0/1896674200 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f78b409b6c0 con 0x7f78c40b3f30 2024-09-17T04:30:36.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.429+0000 7f78baa00640 1 --2- 172.21.15.100:0/1896674200 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f78a0078ef0 0x7f78a007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:36.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.429+0000 7f78baa00640 1 --2- 172.21.15.100:0/1896674200 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f78a0078ef0 0x7f78a007b400 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f78c415e990 tx=0x7f78ac009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:36.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.433+0000 7f78bbe00640 1 -- 172.21.15.100:0/1896674200 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f78c415ca30 con 0x7f78c40b3f30 2024-09-17T04:30:36.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.644+0000 7f78c8a00640 1 -- 172.21.15.100:0/1896674200 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol_clone15410", "target": ["mon-mgr", ""]}) -- 0x7f78c41566c0 con 0x7f78a0078ef0 2024-09-17T04:30:36.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.652+0000 7f78bbe00640 1 -- 172.21.15.100:0/1896674200 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+725 (secure 0 0 0) 0x7f78c41566c0 con 0x7f78a0078ef0 2024-09-17T04:30:36.653 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:30:36.653 INFO:teuthology.orchestra.run.smithi100.stdout: "atime": "2024-09-17 04:30:11", 2024-09-17T04:30:36.653 INFO:teuthology.orchestra.run.smithi100.stdout: "bytes_pcent": "80.00", 2024-09-17T04:30:36.653 INFO:teuthology.orchestra.run.smithi100.stdout: "bytes_quota": 10485760, 2024-09-17T04:30:36.653 INFO:teuthology.orchestra.run.smithi100.stdout: "bytes_used": 8388608, 2024-09-17T04:30:36.653 INFO:teuthology.orchestra.run.smithi100.stdout: "created_at": "2024-09-17 04:30:20", 2024-09-17T04:30:36.653 INFO:teuthology.orchestra.run.smithi100.stdout: "ctime": "2024-09-17 04:30:29", 2024-09-17T04:30:36.653 INFO:teuthology.orchestra.run.smithi100.stdout: "data_pool": "cephfs_data", 2024-09-17T04:30:36.653 INFO:teuthology.orchestra.run.smithi100.stdout: "features": [ 2024-09-17T04:30:36.653 INFO:teuthology.orchestra.run.smithi100.stdout: "snapshot-clone", 2024-09-17T04:30:36.653 INFO:teuthology.orchestra.run.smithi100.stdout: "snapshot-autoprotect", 2024-09-17T04:30:36.653 INFO:teuthology.orchestra.run.smithi100.stdout: "snapshot-retention" 2024-09-17T04:30:36.653 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:30:36.654 INFO:teuthology.orchestra.run.smithi100.stdout: "flavor": 2, 2024-09-17T04:30:36.654 INFO:teuthology.orchestra.run.smithi100.stdout: "gid": 1000, 2024-09-17T04:30:36.654 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": 16895, 2024-09-17T04:30:36.654 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_addrs": [ 2024-09-17T04:30:36.654 INFO:teuthology.orchestra.run.smithi100.stdout: "172.21.15.100:6789", 2024-09-17T04:30:36.654 INFO:teuthology.orchestra.run.smithi100.stdout: "172.21.15.184:6789", 2024-09-17T04:30:36.654 INFO:teuthology.orchestra.run.smithi100.stdout: "172.21.15.184:6790" 2024-09-17T04:30:36.654 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-17T04:30:36.654 INFO:teuthology.orchestra.run.smithi100.stdout: "mtime": "2024-09-17 04:30:16", 2024-09-17T04:30:36.654 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c", 2024-09-17T04:30:36.654 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_namespace": "", 2024-09-17T04:30:36.654 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "complete", 2024-09-17T04:30:36.654 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "clone", 2024-09-17T04:30:36.654 INFO:teuthology.orchestra.run.smithi100.stdout: "uid": 1000 2024-09-17T04:30:36.655 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:30:36.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.655+0000 7f78c8a00640 1 -- 172.21.15.100:0/1896674200 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f78a0078ef0 msgr2=0x7f78a007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:36.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.655+0000 7f78c8a00640 1 --2- 172.21.15.100:0/1896674200 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f78a0078ef0 0x7f78a007b400 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f78c415e990 tx=0x7f78ac009040 comp rx=0 tx=0).stop 2024-09-17T04:30:36.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.655+0000 7f78c8a00640 1 -- 172.21.15.100:0/1896674200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78c40b3f30 msgr2=0x7f78c41456b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:36.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.655+0000 7f78c8a00640 1 --2- 172.21.15.100:0/1896674200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78c40b3f30 0x7f78c41456b0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f78b400d740 tx=0x7f78b400dc00 comp rx=0 tx=0).stop 2024-09-17T04:30:36.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.655+0000 7f78c8a00640 1 -- 172.21.15.100:0/1896674200 shutdown_connections 2024-09-17T04:30:36.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.655+0000 7f78c8a00640 1 --2- 172.21.15.100:0/1896674200 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f78a0078ef0 0x7f78a007b400 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:36.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.655+0000 7f78c8a00640 1 --2- 172.21.15.100:0/1896674200 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c415d6e0 0x7f78c415bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:36.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.655+0000 7f78c8a00640 1 --2- 172.21.15.100:0/1896674200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78c40cd1d0 0x7f78c415b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:36.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.655+0000 7f78c8a00640 1 --2- 172.21.15.100:0/1896674200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78c40b3f30 0x7f78c41456b0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:36.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.655+0000 7f78c8a00640 1 -- 172.21.15.100:0/1896674200 >> 172.21.15.100:0/1896674200 conn(0x7f78c406d620 msgr2=0x7f78c406f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:36.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.655+0000 7f78c8a00640 1 -- 172.21.15.100:0/1896674200 shutdown_connections 2024-09-17T04:30:36.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:36.655+0000 7f78c8a00640 1 -- 172.21.15.100:0/1896674200 wait complete. 2024-09-17T04:30:36.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:36.679 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.quota.max_bytes volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850) 2024-09-17T04:30:36.715 INFO:teuthology.orchestra.run.smithi100.stderr:volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850: ceph.quota.max_bytes: No such attribute 2024-09-17T04:30:36.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T04:30:36.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:36.716 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850) 2024-09-17T04:30:36.794 INFO:teuthology.orchestra.run.smithi100.stdout:cephfs_data 2024-09-17T04:30:36.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:36.794 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool_namespace volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850) 2024-09-17T04:30:36.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:36.871 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec find volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850) 2024-09-17T04:30:36.949 INFO:teuthology.orchestra.run.smithi100.stdout:volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850 2024-09-17T04:30:36.949 INFO:teuthology.orchestra.run.smithi100.stdout:volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.2 2024-09-17T04:30:36.949 INFO:teuthology.orchestra.run.smithi100.stdout:volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.5 2024-09-17T04:30:36.949 INFO:teuthology.orchestra.run.smithi100.stdout:volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.4 2024-09-17T04:30:36.949 INFO:teuthology.orchestra.run.smithi100.stdout:volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.7 2024-09-17T04:30:36.949 INFO:teuthology.orchestra.run.smithi100.stdout:volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.6 2024-09-17T04:30:36.949 INFO:teuthology.orchestra.run.smithi100.stdout:volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.0 2024-09-17T04:30:36.949 INFO:teuthology.orchestra.run.smithi100.stdout:volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.3 2024-09-17T04:30:36.950 INFO:teuthology.orchestra.run.smithi100.stdout:volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.1 2024-09-17T04:30:36.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:36.951 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850) 2024-09-17T04:30:37.026 INFO:teuthology.orchestra.run.smithi100.stdout:41ff 2024-09-17T04:30:37.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:37.028 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/) 2024-09-17T04:30:37.103 INFO:teuthology.orchestra.run.smithi100.stdout:41ff 2024-09-17T04:30:37.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:37.104 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850) 2024-09-17T04:30:37.179 INFO:teuthology.orchestra.run.smithi100.stdout:1000 2024-09-17T04:30:37.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:37.180 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/) 2024-09-17T04:30:37.255 INFO:teuthology.orchestra.run.smithi100.stdout:1000 2024-09-17T04:30:37.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:37.256 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850) 2024-09-17T04:30:37.331 INFO:teuthology.orchestra.run.smithi100.stdout:1000 2024-09-17T04:30:37.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:37.332 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/) 2024-09-17T04:30:37.407 INFO:teuthology.orchestra.run.smithi100.stdout:1000 2024-09-17T04:30:37.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:37.408 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850) 2024-09-17T04:30:37.483 INFO:teuthology.orchestra.run.smithi100.stdout:1726547416 2024-09-17T04:30:37.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:37.484 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/) 2024-09-17T04:30:37.560 INFO:teuthology.orchestra.run.smithi100.stdout:1726547416 2024-09-17T04:30:37.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:37.561 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.2) 2024-09-17T04:30:37.638 INFO:teuthology.orchestra.run.smithi100.stdout:81a4 2024-09-17T04:30:37.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:37.640 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.2) 2024-09-17T04:30:37.716 INFO:teuthology.orchestra.run.smithi100.stdout:81a4 2024-09-17T04:30:37.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:37.718 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.2) 2024-09-17T04:30:37.793 INFO:teuthology.orchestra.run.smithi100.stdout:1000 2024-09-17T04:30:37.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:37.795 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.2) 2024-09-17T04:30:37.871 INFO:teuthology.orchestra.run.smithi100.stdout:1000 2024-09-17T04:30:37.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:37.872 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.2) 2024-09-17T04:30:37.947 INFO:teuthology.orchestra.run.smithi100.stdout:1333 2024-09-17T04:30:37.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:37.948 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.2) 2024-09-17T04:30:38.023 INFO:teuthology.orchestra.run.smithi100.stdout:1333 2024-09-17T04:30:38.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:38.024 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.2) 2024-09-17T04:30:38.103 INFO:teuthology.orchestra.run.smithi100.stdout:1726547415 2024-09-17T04:30:38.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:38.105 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.2) 2024-09-17T04:30:38.179 INFO:teuthology.orchestra.run.smithi100.stdout:1726547415 2024-09-17T04:30:38.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:38.181 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.5) 2024-09-17T04:30:38.258 INFO:teuthology.orchestra.run.smithi100.stdout:81a4 2024-09-17T04:30:38.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:38.260 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.5) 2024-09-17T04:30:38.336 INFO:teuthology.orchestra.run.smithi100.stdout:81a4 2024-09-17T04:30:38.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:38.337 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.5) 2024-09-17T04:30:38.412 INFO:teuthology.orchestra.run.smithi100.stdout:1000 2024-09-17T04:30:38.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:38.413 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.5) 2024-09-17T04:30:38.487 INFO:teuthology.orchestra.run.smithi100.stdout:1000 2024-09-17T04:30:38.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:38.488 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.5) 2024-09-17T04:30:38.564 INFO:teuthology.orchestra.run.smithi100.stdout:1333 2024-09-17T04:30:38.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:38.565 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.5) 2024-09-17T04:30:38.639 INFO:teuthology.orchestra.run.smithi100.stdout:1333 2024-09-17T04:30:38.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:38.641 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.5) 2024-09-17T04:30:38.718 INFO:teuthology.orchestra.run.smithi100.stdout:1726547416 2024-09-17T04:30:38.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:38.719 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.5) 2024-09-17T04:30:38.796 INFO:teuthology.orchestra.run.smithi100.stdout:1726547416 2024-09-17T04:30:38.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:38.796 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.4) 2024-09-17T04:30:38.873 INFO:teuthology.orchestra.run.smithi100.stdout:81a4 2024-09-17T04:30:38.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:38.874 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.4) 2024-09-17T04:30:38.950 INFO:teuthology.orchestra.run.smithi100.stdout:81a4 2024-09-17T04:30:38.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:38.953 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.4) 2024-09-17T04:30:39.037 INFO:teuthology.orchestra.run.smithi100.stdout:1000 2024-09-17T04:30:39.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:39.041 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.4) 2024-09-17T04:30:39.121 INFO:teuthology.orchestra.run.smithi100.stdout:1000 2024-09-17T04:30:39.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:39.122 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.4) 2024-09-17T04:30:39.199 INFO:teuthology.orchestra.run.smithi100.stdout:1333 2024-09-17T04:30:39.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:39.201 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.4) 2024-09-17T04:30:39.278 INFO:teuthology.orchestra.run.smithi100.stdout:1333 2024-09-17T04:30:39.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:39.278 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.4) 2024-09-17T04:30:39.358 INFO:teuthology.orchestra.run.smithi100.stdout:1726547415 2024-09-17T04:30:39.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:39.359 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.4) 2024-09-17T04:30:39.434 INFO:teuthology.orchestra.run.smithi100.stdout:1726547415 2024-09-17T04:30:39.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:39.436 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.7) 2024-09-17T04:30:39.513 INFO:teuthology.orchestra.run.smithi100.stdout:81a4 2024-09-17T04:30:39.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:39.514 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.7) 2024-09-17T04:30:39.590 INFO:teuthology.orchestra.run.smithi100.stdout:81a4 2024-09-17T04:30:39.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:39.591 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.7) 2024-09-17T04:30:39.667 INFO:teuthology.orchestra.run.smithi100.stdout:1000 2024-09-17T04:30:39.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:39.668 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.7) 2024-09-17T04:30:39.745 INFO:teuthology.orchestra.run.smithi100.stdout:1000 2024-09-17T04:30:39.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:39.745 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.7) 2024-09-17T04:30:39.821 INFO:teuthology.orchestra.run.smithi100.stdout:1333 2024-09-17T04:30:39.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:39.822 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.7) 2024-09-17T04:30:39.897 INFO:teuthology.orchestra.run.smithi100.stdout:1333 2024-09-17T04:30:39.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:39.898 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.7) 2024-09-17T04:30:39.978 INFO:teuthology.orchestra.run.smithi100.stdout:1726547416 2024-09-17T04:30:39.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:39.979 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.7) 2024-09-17T04:30:40.055 INFO:teuthology.orchestra.run.smithi100.stdout:1726547416 2024-09-17T04:30:40.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:40.057 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.6) 2024-09-17T04:30:40.133 INFO:teuthology.orchestra.run.smithi100.stdout:81a4 2024-09-17T04:30:40.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:40.134 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.6) 2024-09-17T04:30:40.209 INFO:teuthology.orchestra.run.smithi100.stdout:81a4 2024-09-17T04:30:40.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:40.211 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.6) 2024-09-17T04:30:40.285 INFO:teuthology.orchestra.run.smithi100.stdout:1000 2024-09-17T04:30:40.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:40.287 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.6) 2024-09-17T04:30:40.362 INFO:teuthology.orchestra.run.smithi100.stdout:1000 2024-09-17T04:30:40.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:40.363 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.6) 2024-09-17T04:30:40.439 INFO:teuthology.orchestra.run.smithi100.stdout:1333 2024-09-17T04:30:40.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:40.440 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.6) 2024-09-17T04:30:40.515 INFO:teuthology.orchestra.run.smithi100.stdout:1333 2024-09-17T04:30:40.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:40.516 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.6) 2024-09-17T04:30:40.594 INFO:teuthology.orchestra.run.smithi100.stdout:1726547416 2024-09-17T04:30:40.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:40.595 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.6) 2024-09-17T04:30:40.670 INFO:teuthology.orchestra.run.smithi100.stdout:1726547416 2024-09-17T04:30:40.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:40.672 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.0) 2024-09-17T04:30:40.747 INFO:teuthology.orchestra.run.smithi100.stdout:81a4 2024-09-17T04:30:40.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:40.749 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.0) 2024-09-17T04:30:40.825 INFO:teuthology.orchestra.run.smithi100.stdout:81a4 2024-09-17T04:30:40.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:40.826 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.0) 2024-09-17T04:30:40.901 INFO:teuthology.orchestra.run.smithi100.stdout:1000 2024-09-17T04:30:40.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:40.902 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.0) 2024-09-17T04:30:40.977 INFO:teuthology.orchestra.run.smithi100.stdout:1000 2024-09-17T04:30:40.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:40.978 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.0) 2024-09-17T04:30:41.053 INFO:teuthology.orchestra.run.smithi100.stdout:1333 2024-09-17T04:30:41.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:41.054 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.0) 2024-09-17T04:30:41.130 INFO:teuthology.orchestra.run.smithi100.stdout:1333 2024-09-17T04:30:41.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:41.131 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.0) 2024-09-17T04:30:41.207 INFO:teuthology.orchestra.run.smithi100.stdout:1726547415 2024-09-17T04:30:41.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:41.207 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.0) 2024-09-17T04:30:41.282 INFO:teuthology.orchestra.run.smithi100.stdout:1726547415 2024-09-17T04:30:41.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:41.283 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.3) 2024-09-17T04:30:41.358 INFO:teuthology.orchestra.run.smithi100.stdout:81a4 2024-09-17T04:30:41.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:41.359 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.3) 2024-09-17T04:30:41.435 INFO:teuthology.orchestra.run.smithi100.stdout:81a4 2024-09-17T04:30:41.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:41.436 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.3) 2024-09-17T04:30:41.511 INFO:teuthology.orchestra.run.smithi100.stdout:1000 2024-09-17T04:30:41.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:41.512 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.3) 2024-09-17T04:30:41.587 INFO:teuthology.orchestra.run.smithi100.stdout:1000 2024-09-17T04:30:41.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:41.588 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.3) 2024-09-17T04:30:41.664 INFO:teuthology.orchestra.run.smithi100.stdout:1333 2024-09-17T04:30:41.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:41.665 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.3) 2024-09-17T04:30:41.741 INFO:teuthology.orchestra.run.smithi100.stdout:1333 2024-09-17T04:30:41.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:41.742 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.3) 2024-09-17T04:30:41.818 INFO:teuthology.orchestra.run.smithi100.stdout:1726547415 2024-09-17T04:30:41.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:41.819 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.3) 2024-09-17T04:30:41.894 INFO:teuthology.orchestra.run.smithi100.stdout:1726547415 2024-09-17T04:30:41.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:41.895 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.1) 2024-09-17T04:30:41.972 INFO:teuthology.orchestra.run.smithi100.stdout:81a4 2024-09-17T04:30:41.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:41.973 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.1) 2024-09-17T04:30:42.048 INFO:teuthology.orchestra.run.smithi100.stdout:81a4 2024-09-17T04:30:42.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:42.050 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.1) 2024-09-17T04:30:42.125 INFO:teuthology.orchestra.run.smithi100.stdout:1000 2024-09-17T04:30:42.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:42.126 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.1) 2024-09-17T04:30:42.201 INFO:teuthology.orchestra.run.smithi100.stdout:1000 2024-09-17T04:30:42.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:42.202 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.1) 2024-09-17T04:30:42.277 INFO:teuthology.orchestra.run.smithi100.stdout:1333 2024-09-17T04:30:42.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:42.278 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.1) 2024-09-17T04:30:42.353 INFO:teuthology.orchestra.run.smithi100.stdout:1333 2024-09-17T04:30:42.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:42.355 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4/.snap/subvol_snap99850/subvolume_file.1) 2024-09-17T04:30:42.430 INFO:teuthology.orchestra.run.smithi100.stdout:1726547415 2024-09-17T04:30:42.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:42.431 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone15410/5c5b1381-26a0-43b5-8819-4cd26a3c864c/subvolume_file.1) 2024-09-17T04:30:42.506 INFO:teuthology.orchestra.run.smithi100.stdout:1726547415 2024-09-17T04:30:42.507 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol80190 fake subvol_clone89001 2024-09-17T04:30:42.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.778+0000 7fba1dc00640 1 -- 172.21.15.100:0/3801013209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba18145a40 msgr2=0x7fba18073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:42.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.778+0000 7fba1dc00640 1 --2- 172.21.15.100:0/3801013209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba18145a40 0x7fba18073540 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fba04009830 tx=0x7fba0402a640 comp rx=0 tx=0).stop 2024-09-17T04:30:42.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.778+0000 7fba1dc00640 1 -- 172.21.15.100:0/3801013209 shutdown_connections 2024-09-17T04:30:42.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.778+0000 7fba1dc00640 1 --2- 172.21.15.100:0/3801013209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba18145a40 0x7fba18073540 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:42.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.778+0000 7fba1dc00640 1 -- 172.21.15.100:0/3801013209 >> 172.21.15.100:0/3801013209 conn(0x7fba1806d1c0 msgr2=0x7fba1806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:42.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.778+0000 7fba1dc00640 1 -- 172.21.15.100:0/3801013209 shutdown_connections 2024-09-17T04:30:42.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.778+0000 7fba1dc00640 1 -- 172.21.15.100:0/3801013209 wait complete. 2024-09-17T04:30:42.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.781+0000 7fba1dc00640 1 Processor -- start 2024-09-17T04:30:42.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.781+0000 7fba1dc00640 1 -- start start 2024-09-17T04:30:42.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.783+0000 7fba1dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba18073a80 0x7fba180aee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:42.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.783+0000 7fba1dc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba18145550 0x7fba180b3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:42.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.783+0000 7fba1dc00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba18145a40 0x7fba181f9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:42.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.783+0000 7fba1dc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fba180ce350 con 0x7fba18073a80 2024-09-17T04:30:42.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.783+0000 7fba1dc00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fba180ce1d0 con 0x7fba18145550 2024-09-17T04:30:42.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.783+0000 7fba1dc00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fba180ce4d0 con 0x7fba18145a40 2024-09-17T04:30:42.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.783+0000 7fba16a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba18073a80 0x7fba180aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:42.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.783+0000 7fba16a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba18073a80 0x7fba180aee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58542/0 (socket says 172.21.15.100:58542) 2024-09-17T04:30:42.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.783+0000 7fba16a00640 1 -- 172.21.15.100:0/2140447481 learned_addr learned my addr 172.21.15.100:0/2140447481 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:42.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.783+0000 7fba16000640 1 --2- 172.21.15.100:0/2140447481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba18145550 0x7fba180b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:42.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.783+0000 7fba17400640 1 --2- 172.21.15.100:0/2140447481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba18145a40 0x7fba181f9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:42.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.784+0000 7fba16a00640 1 -- 172.21.15.100:0/2140447481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba18145a40 msgr2=0x7fba181f9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:42.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.784+0000 7fba16a00640 1 --2- 172.21.15.100:0/2140447481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba18145a40 0x7fba181f9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:42.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.784+0000 7fba16a00640 1 -- 172.21.15.100:0/2140447481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba18145550 msgr2=0x7fba180b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:42.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.784+0000 7fba16a00640 1 --2- 172.21.15.100:0/2140447481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba18145550 0x7fba180b3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:42.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.784+0000 7fba16a00640 1 -- 172.21.15.100:0/2140447481 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba180b3e40 con 0x7fba18073a80 2024-09-17T04:30:42.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.784+0000 7fba16a00640 1 --2- 172.21.15.100:0/2140447481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba18073a80 0x7fba180aee40 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fba0800c400 tx=0x7fba0800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:42.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.784+0000 7fba0ea00640 1 -- 172.21.15.100:0/2140447481 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba08011790 con 0x7fba18073a80 2024-09-17T04:30:42.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.784+0000 7fba0ea00640 1 -- 172.21.15.100:0/2140447481 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba08017070 con 0x7fba18073a80 2024-09-17T04:30:42.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.784+0000 7fba1dc00640 1 -- 172.21.15.100:0/2140447481 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba180b4120 con 0x7fba18073a80 2024-09-17T04:30:42.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.784+0000 7fba0ea00640 1 -- 172.21.15.100:0/2140447481 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba08011ce0 con 0x7fba18073a80 2024-09-17T04:30:42.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.785+0000 7fba1dc00640 1 -- 172.21.15.100:0/2140447481 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba180b46e0 con 0x7fba18073a80 2024-09-17T04:30:42.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.786+0000 7fba1dc00640 1 -- 172.21.15.100:0/2140447481 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba18070350 con 0x7fba18073a80 2024-09-17T04:30:42.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.787+0000 7fba0ea00640 1 -- 172.21.15.100:0/2140447481 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7fba0801c460 con 0x7fba18073a80 2024-09-17T04:30:42.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.788+0000 7fba0ea00640 1 --2- 172.21.15.100:0/2140447481 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb9f8078ef0 0x7fb9f807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:42.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.789+0000 7fba0ea00640 1 -- 172.21.15.100:0/2140447481 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7fba0809b0c0 con 0x7fba18073a80 2024-09-17T04:30:42.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.789+0000 7fba16000640 1 --2- 172.21.15.100:0/2140447481 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb9f8078ef0 0x7fb9f807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:42.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.789+0000 7fba16000640 1 --2- 172.21.15.100:0/2140447481 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb9f8078ef0 0x7fb9f807b400 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fba04034000 tx=0x7fba04004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:42.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:42.793+0000 7fba0ea00640 1 -- 172.21.15.100:0/2140447481 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fba18070350 con 0x7fba18073a80 2024-09-17T04:30:43.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.010+0000 7fba1dc00640 1 -- 172.21.15.100:0/2140447481 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol80190", "snap_name": "fake", "target_sub_name": "subvol_clone89001", "target": ["mon-mgr", ""]}) -- 0x7fba18073540 con 0x7fb9f8078ef0 2024-09-17T04:30:43.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.074+0000 7fba0ea00640 1 -- 172.21.15.100:0/2140447481 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fba18073540 con 0x7fb9f8078ef0 2024-09-17T04:30:43.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.077+0000 7fba1dc00640 1 -- 172.21.15.100:0/2140447481 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb9f8078ef0 msgr2=0x7fb9f807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:43.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.077+0000 7fba1dc00640 1 --2- 172.21.15.100:0/2140447481 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb9f8078ef0 0x7fb9f807b400 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fba04034000 tx=0x7fba04004730 comp rx=0 tx=0).stop 2024-09-17T04:30:43.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.077+0000 7fba1dc00640 1 -- 172.21.15.100:0/2140447481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba18073a80 msgr2=0x7fba180aee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:43.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.077+0000 7fba1dc00640 1 --2- 172.21.15.100:0/2140447481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba18073a80 0x7fba180aee40 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fba0800c400 tx=0x7fba0800c8c0 comp rx=0 tx=0).stop 2024-09-17T04:30:43.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.077+0000 7fba1dc00640 1 -- 172.21.15.100:0/2140447481 shutdown_connections 2024-09-17T04:30:43.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.077+0000 7fba1dc00640 1 --2- 172.21.15.100:0/2140447481 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb9f8078ef0 0x7fb9f807b400 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:43.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.077+0000 7fba1dc00640 1 --2- 172.21.15.100:0/2140447481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba18145a40 0x7fba181f9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:43.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.077+0000 7fba1dc00640 1 --2- 172.21.15.100:0/2140447481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba18145550 0x7fba180b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:43.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.077+0000 7fba1dc00640 1 --2- 172.21.15.100:0/2140447481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba18073a80 0x7fba180aee40 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:43.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.077+0000 7fba1dc00640 1 -- 172.21.15.100:0/2140447481 >> 172.21.15.100:0/2140447481 conn(0x7fba1806d1c0 msgr2=0x7fba1813f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:43.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.077+0000 7fba1dc00640 1 -- 172.21.15.100:0/2140447481 shutdown_connections 2024-09-17T04:30:43.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.078+0000 7fba1dc00640 1 -- 172.21.15.100:0/2140447481 wait complete. 2024-09-17T04:30:43.102 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone89001 2024-09-17T04:30:43.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.326+0000 7fa84b600640 1 -- 172.21.15.100:0/2794486486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8440aee00 msgr2=0x7fa8440af210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:43.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.326+0000 7fa84b600640 1 --2- 172.21.15.100:0/2794486486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8440aee00 0x7fa8440af210 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fa844072bd0 tx=0x7fa83002a440 comp rx=0 tx=0).stop 2024-09-17T04:30:43.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.327+0000 7fa84b600640 1 -- 172.21.15.100:0/2794486486 shutdown_connections 2024-09-17T04:30:43.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.327+0000 7fa84b600640 1 --2- 172.21.15.100:0/2794486486 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa844073540 0x7fa844073950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:43.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.327+0000 7fa84b600640 1 --2- 172.21.15.100:0/2794486486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8440aee00 0x7fa8440af210 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:43.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.327+0000 7fa84b600640 1 --2- 172.21.15.100:0/2794486486 >> v2:172.21.15.184:3300/0 conn(0x7fa84415f9e0 0x7fa844147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:43.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.327+0000 7fa84b600640 1 -- 172.21.15.100:0/2794486486 >> 172.21.15.100:0/2794486486 conn(0x7fa84406d1c0 msgr2=0x7fa84406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:43.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.327+0000 7fa84b600640 1 -- 172.21.15.100:0/2794486486 shutdown_connections 2024-09-17T04:30:43.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.327+0000 7fa84b600640 1 -- 172.21.15.100:0/2794486486 wait complete. 2024-09-17T04:30:43.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.329+0000 7fa84b600640 1 Processor -- start 2024-09-17T04:30:43.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.329+0000 7fa84b600640 1 -- start start 2024-09-17T04:30:43.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.330+0000 7fa84b600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa844073540 0x7fa8441eafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:43.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.330+0000 7fa84b600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8440aee00 0x7fa84412fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:43.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.330+0000 7fa84b600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa84415f9e0 0x7fa844130110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:43.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.330+0000 7fa84b600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa84406f950 con 0x7fa84415f9e0 2024-09-17T04:30:43.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.330+0000 7fa84b600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa84406f7d0 con 0x7fa844073540 2024-09-17T04:30:43.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.330+0000 7fa84b600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa84406fad0 con 0x7fa8440aee00 2024-09-17T04:30:43.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.331+0000 7fa849600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa84415f9e0 0x7fa844130110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:43.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.331+0000 7fa849600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa84415f9e0 0x7fa844130110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58562/0 (socket says 172.21.15.100:58562) 2024-09-17T04:30:43.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.331+0000 7fa849600640 1 -- 172.21.15.100:0/582871926 learned_addr learned my addr 172.21.15.100:0/582871926 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:43.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.331+0000 7fa848c00640 1 --2- 172.21.15.100:0/582871926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa844073540 0x7fa8441eafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:43.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.331+0000 7fa843e00640 1 --2- 172.21.15.100:0/582871926 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8440aee00 0x7fa84412fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:43.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.331+0000 7fa849600640 1 -- 172.21.15.100:0/582871926 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8440aee00 msgr2=0x7fa84412fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:43.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.331+0000 7fa849600640 1 --2- 172.21.15.100:0/582871926 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8440aee00 0x7fa84412fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:43.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.331+0000 7fa849600640 1 -- 172.21.15.100:0/582871926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa844073540 msgr2=0x7fa8441eafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:43.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.331+0000 7fa849600640 1 --2- 172.21.15.100:0/582871926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa844073540 0x7fa8441eafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:43.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.331+0000 7fa849600640 1 -- 172.21.15.100:0/582871926 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84412dce0 con 0x7fa84415f9e0 2024-09-17T04:30:43.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.331+0000 7fa843e00640 1 --2- 172.21.15.100:0/582871926 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8440aee00 0x7fa84412fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:30:43.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.331+0000 7fa849600640 1 --2- 172.21.15.100:0/582871926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa84415f9e0 0x7fa844130110 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00e910 tx=0x7fa83c00edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:43.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.332+0000 7fa840c00640 1 -- 172.21.15.100:0/582871926 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c004d20 con 0x7fa84415f9e0 2024-09-17T04:30:43.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.332+0000 7fa840c00640 1 -- 172.21.15.100:0/582871926 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa83c00cb20 con 0x7fa84415f9e0 2024-09-17T04:30:43.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.332+0000 7fa84b600640 1 -- 172.21.15.100:0/582871926 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa84412dfc0 con 0x7fa84415f9e0 2024-09-17T04:30:43.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.332+0000 7fa840c00640 1 -- 172.21.15.100:0/582871926 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c005020 con 0x7fa84415f9e0 2024-09-17T04:30:43.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.332+0000 7fa84b600640 1 -- 172.21.15.100:0/582871926 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa84412e580 con 0x7fa84415f9e0 2024-09-17T04:30:43.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.333+0000 7fa84b600640 1 -- 172.21.15.100:0/582871926 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8441442e0 con 0x7fa84415f9e0 2024-09-17T04:30:43.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.334+0000 7fa840c00640 1 -- 172.21.15.100:0/582871926 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7fa83c0051c0 con 0x7fa84415f9e0 2024-09-17T04:30:43.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.335+0000 7fa840c00640 1 --2- 172.21.15.100:0/582871926 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa824078f20 0x7fa82407b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:43.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.335+0000 7fa840c00640 1 -- 172.21.15.100:0/582871926 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7fa83c09bc90 con 0x7fa84415f9e0 2024-09-17T04:30:43.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.335+0000 7fa848c00640 1 --2- 172.21.15.100:0/582871926 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa824078f20 0x7fa82407b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:43.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.336+0000 7fa848c00640 1 --2- 172.21.15.100:0/582871926 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa824078f20 0x7fa82407b430 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa8440b3240 tx=0x7fa83800b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:43.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.339+0000 7fa840c00640 1 -- 172.21.15.100:0/582871926 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa83c016030 con 0x7fa84415f9e0 2024-09-17T04:30:43.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:43.556+0000 7fa84b600640 1 -- 172.21.15.100:0/582871926 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone89001", "target": ["mon-mgr", ""]}) -- 0x7fa84406fff0 con 0x7fa824078f20 2024-09-17T04:30:44.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:44.983+0000 7fa840c00640 1 -- 172.21.15.100:0/582871926 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+153 (secure 0 0 0) 0x7fa84406fff0 con 0x7fa824078f20 2024-09-17T04:30:44.984 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:30:44.984 INFO:teuthology.orchestra.run.smithi100.stdout: "status": { 2024-09-17T04:30:44.984 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "pending", 2024-09-17T04:30:44.984 INFO:teuthology.orchestra.run.smithi100.stdout: "source": { 2024-09-17T04:30:44.984 INFO:teuthology.orchestra.run.smithi100.stdout: "volume": "cephfs", 2024-09-17T04:30:44.984 INFO:teuthology.orchestra.run.smithi100.stdout: "subvolume": "subvol80190", 2024-09-17T04:30:44.984 INFO:teuthology.orchestra.run.smithi100.stdout: "snapshot": "fake" 2024-09-17T04:30:44.985 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:30:44.985 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:30:44.985 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:30:44.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:44.986+0000 7fa84b600640 1 -- 172.21.15.100:0/582871926 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa824078f20 msgr2=0x7fa82407b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:44.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:44.986+0000 7fa84b600640 1 --2- 172.21.15.100:0/582871926 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa824078f20 0x7fa82407b430 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa8440b3240 tx=0x7fa83800b2e0 comp rx=0 tx=0).stop 2024-09-17T04:30:44.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:44.986+0000 7fa84b600640 1 -- 172.21.15.100:0/582871926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa84415f9e0 msgr2=0x7fa844130110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:44.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:44.986+0000 7fa84b600640 1 --2- 172.21.15.100:0/582871926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa84415f9e0 0x7fa844130110 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00e910 tx=0x7fa83c00edd0 comp rx=0 tx=0).stop 2024-09-17T04:30:44.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:44.986+0000 7fa84b600640 1 -- 172.21.15.100:0/582871926 shutdown_connections 2024-09-17T04:30:44.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:44.986+0000 7fa84b600640 1 --2- 172.21.15.100:0/582871926 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa824078f20 0x7fa82407b430 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:44.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:44.986+0000 7fa84b600640 1 --2- 172.21.15.100:0/582871926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa84415f9e0 0x7fa844130110 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:44.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:44.986+0000 7fa84b600640 1 --2- 172.21.15.100:0/582871926 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8440aee00 0x7fa84412fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:44.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:44.986+0000 7fa84b600640 1 --2- 172.21.15.100:0/582871926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa844073540 0x7fa8441eafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:44.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:44.986+0000 7fa84b600640 1 -- 172.21.15.100:0/582871926 >> 172.21.15.100:0/582871926 conn(0x7fa84406d1c0 msgr2=0x7fa844068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:44.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:44.986+0000 7fa84b600640 1 -- 172.21.15.100:0/582871926 shutdown_connections 2024-09-17T04:30:44.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:44.986+0000 7fa84b600640 1 -- 172.21.15.100:0/582871926 wait complete. 2024-09-17T04:30:45.019 DEBUG:tasks.cephfs.test_volumes:current clone state = pending 2024-09-17T04:30:46.020 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone89001 2024-09-17T04:30:46.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.249+0000 7fd422c00640 1 -- 172.21.15.100:0/661279477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd41c070160 msgr2=0x7fd41c0af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:46.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.249+0000 7fd422c00640 1 --2- 172.21.15.100:0/661279477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd41c070160 0x7fd41c0af0d0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fd414005fd0 tx=0x7fd41402aa30 comp rx=0 tx=0).stop 2024-09-17T04:30:46.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.249+0000 7fd422c00640 1 -- 172.21.15.100:0/661279477 shutdown_connections 2024-09-17T04:30:46.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.249+0000 7fd422c00640 1 --2- 172.21.15.100:0/661279477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd41c070160 0x7fd41c0af0d0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:46.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.249+0000 7fd422c00640 1 -- 172.21.15.100:0/661279477 >> 172.21.15.100:0/661279477 conn(0x7fd41c06d620 msgr2=0x7fd41c0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:46.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.249+0000 7fd422c00640 1 -- 172.21.15.100:0/661279477 shutdown_connections 2024-09-17T04:30:46.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.249+0000 7fd422c00640 1 -- 172.21.15.100:0/661279477 wait complete. 2024-09-17T04:30:46.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.251+0000 7fd422c00640 1 Processor -- start 2024-09-17T04:30:46.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.251+0000 7fd422c00640 1 -- start start 2024-09-17T04:30:46.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.252+0000 7fd422c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd41c070160 0x7fd41c09dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:46.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.253+0000 7fd422c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd41c0b3f30 0x7fd41c074870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:46.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.253+0000 7fd422c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd41c156ca0 0x7fd41c0b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:46.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.253+0000 7fd422c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c0af790 con 0x7fd41c156ca0 2024-09-17T04:30:46.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.253+0000 7fd422c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c0af610 con 0x7fd41c070160 2024-09-17T04:30:46.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.253+0000 7fd422c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd41c0af910 con 0x7fd41c0b3f30 2024-09-17T04:30:46.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.253+0000 7fd422200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd41c156ca0 0x7fd41c0b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:46.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.253+0000 7fd422200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd41c156ca0 0x7fd41c0b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35906/0 (socket says 172.21.15.100:35906) 2024-09-17T04:30:46.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.253+0000 7fd422200640 1 -- 172.21.15.100:0/1601842703 learned_addr learned my addr 172.21.15.100:0/1601842703 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:46.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.253+0000 7fd421800640 1 --2- 172.21.15.100:0/1601842703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd41c070160 0x7fd41c09dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:46.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.253+0000 7fd420e00640 1 --2- 172.21.15.100:0/1601842703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd41c0b3f30 0x7fd41c074870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:46.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.253+0000 7fd422200640 1 -- 172.21.15.100:0/1601842703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd41c0b3f30 msgr2=0x7fd41c074870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:46.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.253+0000 7fd422200640 1 --2- 172.21.15.100:0/1601842703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd41c0b3f30 0x7fd41c074870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:46.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.253+0000 7fd422200640 1 -- 172.21.15.100:0/1601842703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd41c070160 msgr2=0x7fd41c09dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:46.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.253+0000 7fd422200640 1 --2- 172.21.15.100:0/1601842703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd41c070160 0x7fd41c09dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:46.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.253+0000 7fd422200640 1 -- 172.21.15.100:0/1601842703 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd41c140900 con 0x7fd41c156ca0 2024-09-17T04:30:46.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.253+0000 7fd421800640 1 --2- 172.21.15.100:0/1601842703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd41c070160 0x7fd41c09dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:30:46.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.254+0000 7fd422200640 1 --2- 172.21.15.100:0/1601842703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd41c156ca0 0x7fd41c0b0fd0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fd414009d90 tx=0x7fd414003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:46.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.254+0000 7fd419600640 1 -- 172.21.15.100:0/1601842703 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd414034040 con 0x7fd41c156ca0 2024-09-17T04:30:46.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.254+0000 7fd419600640 1 -- 172.21.15.100:0/1601842703 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4140045c0 con 0x7fd41c156ca0 2024-09-17T04:30:46.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.254+0000 7fd419600640 1 -- 172.21.15.100:0/1601842703 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd414045480 con 0x7fd41c156ca0 2024-09-17T04:30:46.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.254+0000 7fd422c00640 1 -- 172.21.15.100:0/1601842703 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd41c140be0 con 0x7fd41c156ca0 2024-09-17T04:30:46.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.254+0000 7fd422c00640 1 -- 172.21.15.100:0/1601842703 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd41c1411a0 con 0x7fd41c156ca0 2024-09-17T04:30:46.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.255+0000 7fd422c00640 1 -- 172.21.15.100:0/1601842703 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd41c0af0d0 con 0x7fd41c156ca0 2024-09-17T04:30:46.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.256+0000 7fd419600640 1 -- 172.21.15.100:0/1601842703 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7fd414037020 con 0x7fd41c156ca0 2024-09-17T04:30:46.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.257+0000 7fd419600640 1 --2- 172.21.15.100:0/1601842703 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd3fc078ef0 0x7fd3fc07b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:46.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.257+0000 7fd419600640 1 -- 172.21.15.100:0/1601842703 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7fd4140bcd60 con 0x7fd41c156ca0 2024-09-17T04:30:46.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.258+0000 7fd421800640 1 --2- 172.21.15.100:0/1601842703 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd3fc078ef0 0x7fd3fc07b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:46.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.258+0000 7fd421800640 1 --2- 172.21.15.100:0/1601842703 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd3fc078ef0 0x7fd3fc07b400 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd410000f60 tx=0x7fd410009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:46.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.261+0000 7fd419600640 1 -- 172.21.15.100:0/1601842703 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd414084c00 con 0x7fd41c156ca0 2024-09-17T04:30:46.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:46.481+0000 7fd422c00640 1 -- 172.21.15.100:0/1601842703 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone89001", "target": ["mon-mgr", ""]}) -- 0x7fd41c1567e0 con 0x7fd3fc078ef0 2024-09-17T04:30:49.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:49.976+0000 7fd419600640 1 -- 172.21.15.100:0/1601842703 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7fd41c1567e0 con 0x7fd3fc078ef0 2024-09-17T04:30:49.977 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:30:49.978 INFO:teuthology.orchestra.run.smithi100.stdout: "status": { 2024-09-17T04:30:49.978 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "complete" 2024-09-17T04:30:49.978 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:30:49.978 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:30:49.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:49.980+0000 7fd422c00640 1 -- 172.21.15.100:0/1601842703 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd3fc078ef0 msgr2=0x7fd3fc07b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:49.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:49.980+0000 7fd422c00640 1 --2- 172.21.15.100:0/1601842703 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd3fc078ef0 0x7fd3fc07b400 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd410000f60 tx=0x7fd410009040 comp rx=0 tx=0).stop 2024-09-17T04:30:49.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:49.980+0000 7fd422c00640 1 -- 172.21.15.100:0/1601842703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd41c156ca0 msgr2=0x7fd41c0b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:49.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:49.980+0000 7fd422c00640 1 --2- 172.21.15.100:0/1601842703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd41c156ca0 0x7fd41c0b0fd0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fd414009d90 tx=0x7fd414003f90 comp rx=0 tx=0).stop 2024-09-17T04:30:49.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:49.980+0000 7fd422c00640 1 -- 172.21.15.100:0/1601842703 shutdown_connections 2024-09-17T04:30:49.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:49.980+0000 7fd422c00640 1 --2- 172.21.15.100:0/1601842703 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd3fc078ef0 0x7fd3fc07b400 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:49.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:49.980+0000 7fd422c00640 1 --2- 172.21.15.100:0/1601842703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd41c156ca0 0x7fd41c0b0fd0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:49.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:49.980+0000 7fd422c00640 1 --2- 172.21.15.100:0/1601842703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd41c0b3f30 0x7fd41c074870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:49.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:49.980+0000 7fd422c00640 1 --2- 172.21.15.100:0/1601842703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd41c070160 0x7fd41c09dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:49.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:49.980+0000 7fd422c00640 1 -- 172.21.15.100:0/1601842703 >> 172.21.15.100:0/1601842703 conn(0x7fd41c06d620 msgr2=0x7fd41c0ad800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:49.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:49.980+0000 7fd422c00640 1 -- 172.21.15.100:0/1601842703 shutdown_connections 2024-09-17T04:30:49.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:49.980+0000 7fd422c00640 1 -- 172.21.15.100:0/1601842703 wait complete. 2024-09-17T04:30:50.009 DEBUG:tasks.cephfs.test_volumes:current clone state = complete 2024-09-17T04:30:50.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:50.009 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone89001/.meta) 2024-09-17T04:30:50.068 INFO:teuthology.orchestra.run.smithi100.stdout:[GLOBAL] 2024-09-17T04:30:50.068 INFO:teuthology.orchestra.run.smithi100.stdout:version = 2 2024-09-17T04:30:50.068 INFO:teuthology.orchestra.run.smithi100.stdout:type = clone 2024-09-17T04:30:50.068 INFO:teuthology.orchestra.run.smithi100.stdout:path = /volumes/_nogroup/subvol_clone89001/4f183df5-e51a-4ede-b16d-b605f9ab23a1 2024-09-17T04:30:50.069 INFO:teuthology.orchestra.run.smithi100.stdout:state = complete 2024-09-17T04:30:50.069 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:30:50.071 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info cephfs subvol80190 subvol_snap99850 2024-09-17T04:30:50.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.298+0000 7f60b6200640 1 -- 172.21.15.100:0/1302943754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60b006cda0 msgr2=0x7f60b006d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:50.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.298+0000 7f60b6200640 1 --2- 172.21.15.100:0/1302943754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60b006cda0 0x7f60b006d1b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6098005fd0 tx=0x7f609802a5d0 comp rx=0 tx=0).stop 2024-09-17T04:30:50.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.298+0000 7f60b6200640 1 -- 172.21.15.100:0/1302943754 shutdown_connections 2024-09-17T04:30:50.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.298+0000 7f60b6200640 1 --2- 172.21.15.100:0/1302943754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60b006cda0 0x7f60b006d1b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:50.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.298+0000 7f60b6200640 1 --2- 172.21.15.100:0/1302943754 >> v2:172.21.15.184:3300/0 conn(0x7f60b00cc830 0x7f60b00b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:50.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.298+0000 7f60b6200640 1 -- 172.21.15.100:0/1302943754 >> 172.21.15.100:0/1302943754 conn(0x7f60b00b2540 msgr2=0x7f60b00d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:50.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.298+0000 7f60b6200640 1 -- 172.21.15.100:0/1302943754 shutdown_connections 2024-09-17T04:30:50.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.298+0000 7f60b6200640 1 -- 172.21.15.100:0/1302943754 wait complete. 2024-09-17T04:30:50.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.300+0000 7f60b6200640 1 Processor -- start 2024-09-17T04:30:50.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.300+0000 7f60b6200640 1 -- start start 2024-09-17T04:30:50.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.302+0000 7f60b6200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60b006cda0 0x7f60b01f37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:50.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.302+0000 7f60b6200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60b00af100 0x7f60b01f7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:50.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.302+0000 7f60b6200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60b00cc830 0x7f60b01f7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:50.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.302+0000 7f60b6200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f60b00b3670 con 0x7f60b006cda0 2024-09-17T04:30:50.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.302+0000 7f60b6200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f60b00b34f0 con 0x7f60b00af100 2024-09-17T04:30:50.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.302+0000 7f60b6200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f60b00b37f0 con 0x7f60b00cc830 2024-09-17T04:30:50.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.302+0000 7f60af400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60b006cda0 0x7f60b01f37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:50.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.302+0000 7f60af400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60b006cda0 0x7f60b01f37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35918/0 (socket says 172.21.15.100:35918) 2024-09-17T04:30:50.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.302+0000 7f60af400640 1 -- 172.21.15.100:0/473105523 learned_addr learned my addr 172.21.15.100:0/473105523 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:50.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.302+0000 7f60a6a00640 1 --2- 172.21.15.100:0/473105523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60b00af100 0x7f60b01f7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:50.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.302+0000 7f60afe00640 1 --2- 172.21.15.100:0/473105523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60b00cc830 0x7f60b01f7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:50.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.303+0000 7f60af400640 1 -- 172.21.15.100:0/473105523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60b00cc830 msgr2=0x7f60b01f7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:50.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.303+0000 7f60af400640 1 --2- 172.21.15.100:0/473105523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60b00cc830 0x7f60b01f7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:50.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.303+0000 7f60af400640 1 -- 172.21.15.100:0/473105523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60b00af100 msgr2=0x7f60b01f7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:50.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.303+0000 7f60af400640 1 --2- 172.21.15.100:0/473105523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60b00af100 0x7f60b01f7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:50.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.303+0000 7f60af400640 1 -- 172.21.15.100:0/473105523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b0151ef0 con 0x7f60b006cda0 2024-09-17T04:30:50.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.303+0000 7f60af400640 1 --2- 172.21.15.100:0/473105523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60b006cda0 0x7f60b01f37a0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f60a000e9f0 tx=0x7f60a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:50.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.303+0000 7f60a7e00640 1 -- 172.21.15.100:0/473105523 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a000cd80 con 0x7f60b006cda0 2024-09-17T04:30:50.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.303+0000 7f60a7e00640 1 -- 172.21.15.100:0/473105523 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60a0004020 con 0x7f60b006cda0 2024-09-17T04:30:50.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.303+0000 7f60a7e00640 1 -- 172.21.15.100:0/473105523 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a0004470 con 0x7f60b006cda0 2024-09-17T04:30:50.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.303+0000 7f60b6200640 1 -- 172.21.15.100:0/473105523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b01521d0 con 0x7f60b006cda0 2024-09-17T04:30:50.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.303+0000 7f60b6200640 1 -- 172.21.15.100:0/473105523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b0152d90 con 0x7f60b006cda0 2024-09-17T04:30:50.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.304+0000 7f60b6200640 1 -- 172.21.15.100:0/473105523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60b00b3d10 con 0x7f60b006cda0 2024-09-17T04:30:50.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.306+0000 7f60a7e00640 1 -- 172.21.15.100:0/473105523 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7f60a00035f0 con 0x7f60b006cda0 2024-09-17T04:30:50.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.307+0000 7f60a7e00640 1 --2- 172.21.15.100:0/473105523 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f608c078f20 0x7f608c07b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:50.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.307+0000 7f60a7e00640 1 -- 172.21.15.100:0/473105523 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f60a009be30 con 0x7f60b006cda0 2024-09-17T04:30:50.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.308+0000 7f60a6a00640 1 --2- 172.21.15.100:0/473105523 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f608c078f20 0x7f608c07b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:50.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.308+0000 7f60a6a00640 1 --2- 172.21.15.100:0/473105523 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f608c078f20 0x7f608c07b430 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6098033000 tx=0x7f6098004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:50.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.312+0000 7f60a7e00640 1 -- 172.21.15.100:0/473105523 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f60a0063610 con 0x7f60b006cda0 2024-09-17T04:30:50.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.529+0000 7f60b6200640 1 -- 172.21.15.100:0/473105523 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "cephfs", "sub_name": "subvol80190", "snap_name": "subvol_snap99850", "target": ["mon-mgr", ""]}) -- 0x7f60b00d1030 con 0x7f608c078f20 2024-09-17T04:30:50.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.545+0000 7f60a7e00640 1 -- 172.21.15.100:0/473105523 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+115 (secure 0 0 0) 0x7f60b00d1030 con 0x7f608c078f20 2024-09-17T04:30:50.545 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:30:50.545 INFO:teuthology.orchestra.run.smithi100.stdout: "created_at": "2024-09-17 04:30:19.982967", 2024-09-17T04:30:50.546 INFO:teuthology.orchestra.run.smithi100.stdout: "data_pool": "cephfs_data", 2024-09-17T04:30:50.546 INFO:teuthology.orchestra.run.smithi100.stdout: "has_pending_clones": "no" 2024-09-17T04:30:50.546 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:30:50.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.547+0000 7f60b6200640 1 -- 172.21.15.100:0/473105523 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f608c078f20 msgr2=0x7f608c07b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:50.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.547+0000 7f60b6200640 1 --2- 172.21.15.100:0/473105523 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f608c078f20 0x7f608c07b430 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6098033000 tx=0x7f6098004730 comp rx=0 tx=0).stop 2024-09-17T04:30:50.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.547+0000 7f60b6200640 1 -- 172.21.15.100:0/473105523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60b006cda0 msgr2=0x7f60b01f37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:50.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.547+0000 7f60b6200640 1 --2- 172.21.15.100:0/473105523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60b006cda0 0x7f60b01f37a0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f60a000e9f0 tx=0x7f60a000eeb0 comp rx=0 tx=0).stop 2024-09-17T04:30:50.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.547+0000 7f60b6200640 1 -- 172.21.15.100:0/473105523 shutdown_connections 2024-09-17T04:30:50.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.547+0000 7f60b6200640 1 --2- 172.21.15.100:0/473105523 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f608c078f20 0x7f608c07b430 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:50.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.547+0000 7f60b6200640 1 --2- 172.21.15.100:0/473105523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60b00cc830 0x7f60b01f7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:50.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.547+0000 7f60b6200640 1 --2- 172.21.15.100:0/473105523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60b00af100 0x7f60b01f7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:50.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.547+0000 7f60b6200640 1 --2- 172.21.15.100:0/473105523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60b006cda0 0x7f60b01f37a0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:50.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.547+0000 7f60b6200640 1 -- 172.21.15.100:0/473105523 >> 172.21.15.100:0/473105523 conn(0x7f60b00b2540 msgr2=0x7f60b006f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:50.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.547+0000 7f60b6200640 1 -- 172.21.15.100:0/473105523 shutdown_connections 2024-09-17T04:30:50.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.547+0000 7f60b6200640 1 -- 172.21.15.100:0/473105523 wait complete. 2024-09-17T04:30:50.570 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs subvol80190 2024-09-17T04:30:50.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.798+0000 7f1760a00640 1 -- 172.21.15.100:0/2980896766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f175c0cc7d0 msgr2=0x7f175c0b3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:50.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.798+0000 7f1760a00640 1 --2- 172.21.15.100:0/2980896766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f175c0cc7d0 0x7f175c0b3120 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f174c009830 tx=0x7f174c02a3f0 comp rx=0 tx=0).stop 2024-09-17T04:30:50.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.799+0000 7f1760a00640 1 -- 172.21.15.100:0/2980896766 shutdown_connections 2024-09-17T04:30:50.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.799+0000 7f1760a00640 1 --2- 172.21.15.100:0/2980896766 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f175c15ba50 0x7f175c0b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:50.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.799+0000 7f1760a00640 1 --2- 172.21.15.100:0/2980896766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f175c0cc7d0 0x7f175c0b3120 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:50.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.799+0000 7f1760a00640 1 -- 172.21.15.100:0/2980896766 >> 172.21.15.100:0/2980896766 conn(0x7f175c0b2540 msgr2=0x7f175c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:50.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.799+0000 7f1760a00640 1 -- 172.21.15.100:0/2980896766 shutdown_connections 2024-09-17T04:30:50.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.799+0000 7f1760a00640 1 -- 172.21.15.100:0/2980896766 wait complete. 2024-09-17T04:30:50.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.801+0000 7f1760a00640 1 Processor -- start 2024-09-17T04:30:50.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.801+0000 7f1760a00640 1 -- start start 2024-09-17T04:30:50.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.803+0000 7f1760a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f175c06f010 0x7f175c0b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:50.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.803+0000 7f1760a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f175c0cc7d0 0x7f175c1f81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:50.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.803+0000 7f1760a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f175c15ba50 0x7f175c15bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:50.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.803+0000 7f1760a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f175c0cb930 con 0x7f175c15ba50 2024-09-17T04:30:50.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.803+0000 7f1760a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f175c06ff90 con 0x7f175c0cc7d0 2024-09-17T04:30:50.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.803+0000 7f1760a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f175c0cbab0 con 0x7f175c06f010 2024-09-17T04:30:50.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.803+0000 7f175a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f175c15ba50 0x7f175c15bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:50.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.803+0000 7f175a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f175c15ba50 0x7f175c15bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35936/0 (socket says 172.21.15.100:35936) 2024-09-17T04:30:50.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.803+0000 7f175a800640 1 -- 172.21.15.100:0/4229431975 learned_addr learned my addr 172.21.15.100:0/4229431975 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:50.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.803+0000 7f1759e00640 1 --2- 172.21.15.100:0/4229431975 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f175c06f010 0x7f175c0b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:50.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.803+0000 7f1751400640 1 --2- 172.21.15.100:0/4229431975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f175c0cc7d0 0x7f175c1f81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:50.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.803+0000 7f175a800640 1 -- 172.21.15.100:0/4229431975 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f175c06f010 msgr2=0x7f175c0b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:50.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.803+0000 7f175a800640 1 --2- 172.21.15.100:0/4229431975 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f175c06f010 0x7f175c0b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:50.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.803+0000 7f175a800640 1 -- 172.21.15.100:0/4229431975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f175c0cc7d0 msgr2=0x7f175c1f81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:50.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.803+0000 7f175a800640 1 --2- 172.21.15.100:0/4229431975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f175c0cc7d0 0x7f175c1f81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:50.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.803+0000 7f175a800640 1 -- 172.21.15.100:0/4229431975 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f175c0c82b0 con 0x7f175c15ba50 2024-09-17T04:30:50.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.804+0000 7f175a800640 1 --2- 172.21.15.100:0/4229431975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f175c15ba50 0x7f175c15bef0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f175400cb10 tx=0x7f175400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:50.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.804+0000 7f1752a00640 1 -- 172.21.15.100:0/4229431975 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1754016020 con 0x7f175c15ba50 2024-09-17T04:30:50.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.804+0000 7f1752a00640 1 -- 172.21.15.100:0/4229431975 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f175400ad60 con 0x7f175c15ba50 2024-09-17T04:30:50.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.804+0000 7f1760a00640 1 -- 172.21.15.100:0/4229431975 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f175c0c9af0 con 0x7f175c15ba50 2024-09-17T04:30:50.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.804+0000 7f1752a00640 1 -- 172.21.15.100:0/4229431975 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1754012680 con 0x7f175c15ba50 2024-09-17T04:30:50.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.804+0000 7f1760a00640 1 -- 172.21.15.100:0/4229431975 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f175c15ed10 con 0x7f175c15ba50 2024-09-17T04:30:50.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.805+0000 7f1760a00640 1 -- 172.21.15.100:0/4229431975 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f175c0ad130 con 0x7f175c15ba50 2024-09-17T04:30:50.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.806+0000 7f1752a00640 1 -- 172.21.15.100:0/4229431975 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7f1754019020 con 0x7f175c15ba50 2024-09-17T04:30:50.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.807+0000 7f1752a00640 1 --2- 172.21.15.100:0/4229431975 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1738078ef0 0x7f173807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:50.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.807+0000 7f1752a00640 1 -- 172.21.15.100:0/4229431975 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f175409b5e0 con 0x7f175c15ba50 2024-09-17T04:30:50.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.808+0000 7f1759e00640 1 --2- 172.21.15.100:0/4229431975 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1738078ef0 0x7f173807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:50.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.808+0000 7f1759e00640 1 --2- 172.21.15.100:0/4229431975 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1738078ef0 0x7f173807b400 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f174c034000 tx=0x7f174c004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:50.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:50.812+0000 7f1752a00640 1 -- 172.21.15.100:0/4229431975 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f1754020030 con 0x7f175c15ba50 2024-09-17T04:30:51.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.025+0000 7f1760a00640 1 -- 172.21.15.100:0/4229431975 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "subvol80190", "target": ["mon-mgr", ""]}) -- 0x7f175c1f8710 con 0x7f1738078ef0 2024-09-17T04:30:51.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.038+0000 7f1752a00640 1 -- 172.21.15.100:0/4229431975 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+87 (secure 0 0 0) 0x7f175c1f8710 con 0x7f1738078ef0 2024-09-17T04:30:51.039 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-09-17T04:30:51.039 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:30:51.039 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "fake" 2024-09-17T04:30:51.039 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:30:51.039 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:30:51.039 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "subvol_snap99850" 2024-09-17T04:30:51.039 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:30:51.039 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-09-17T04:30:51.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.041+0000 7f1760a00640 1 -- 172.21.15.100:0/4229431975 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1738078ef0 msgr2=0x7f173807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:51.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.041+0000 7f1760a00640 1 --2- 172.21.15.100:0/4229431975 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1738078ef0 0x7f173807b400 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f174c034000 tx=0x7f174c004730 comp rx=0 tx=0).stop 2024-09-17T04:30:51.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.041+0000 7f1760a00640 1 -- 172.21.15.100:0/4229431975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f175c15ba50 msgr2=0x7f175c15bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:51.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.041+0000 7f1760a00640 1 --2- 172.21.15.100:0/4229431975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f175c15ba50 0x7f175c15bef0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f175400cb10 tx=0x7f175400a060 comp rx=0 tx=0).stop 2024-09-17T04:30:51.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.041+0000 7f1760a00640 1 -- 172.21.15.100:0/4229431975 shutdown_connections 2024-09-17T04:30:51.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.041+0000 7f1760a00640 1 --2- 172.21.15.100:0/4229431975 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1738078ef0 0x7f173807b400 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:51.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.041+0000 7f1760a00640 1 --2- 172.21.15.100:0/4229431975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f175c15ba50 0x7f175c15bef0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:51.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.041+0000 7f1760a00640 1 --2- 172.21.15.100:0/4229431975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f175c0cc7d0 0x7f175c1f81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:51.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.041+0000 7f1760a00640 1 --2- 172.21.15.100:0/4229431975 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f175c06f010 0x7f175c0b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:51.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.041+0000 7f1760a00640 1 -- 172.21.15.100:0/4229431975 >> 172.21.15.100:0/4229431975 conn(0x7f175c0b2540 msgr2=0x7f175c0ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:51.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.041+0000 7f1760a00640 1 -- 172.21.15.100:0/4229431975 shutdown_connections 2024-09-17T04:30:51.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.041+0000 7f1760a00640 1 -- 172.21.15.100:0/4229431975 wait complete. 2024-09-17T04:30:51.064 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol80190 subvol_snap99850 2024-09-17T04:30:51.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.290+0000 7fb6cd000640 1 -- 172.21.15.100:0/1429223134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6ac004800 msgr2=0x7fb6ac004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:51.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.290+0000 7fb6cd000640 1 --2- 172.21.15.100:0/1429223134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6ac004800 0x7fb6ac004c10 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8072f60 tx=0x7fb6ac005830 comp rx=0 tx=0).stop 2024-09-17T04:30:51.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.290+0000 7fb6c6800640 1 -- 172.21.15.100:0/1429223134 reap_dead start 2024-09-17T04:30:51.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.290+0000 7fb6cd000640 1 -- 172.21.15.100:0/1429223134 shutdown_connections 2024-09-17T04:30:51.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.290+0000 7fb6cd000640 1 -- 172.21.15.100:0/1429223134 >> 172.21.15.100:0/1429223134 conn(0x7fb6c80b2540 msgr2=0x7fb6c80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:51.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.290+0000 7fb6cd000640 1 -- 172.21.15.100:0/1429223134 shutdown_connections 2024-09-17T04:30:51.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.290+0000 7fb6cd000640 1 -- 172.21.15.100:0/1429223134 wait complete. 2024-09-17T04:30:51.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.292+0000 7fb6cd000640 1 Processor -- start 2024-09-17T04:30:51.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.292+0000 7fb6cd000640 1 -- start start 2024-09-17T04:30:51.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.294+0000 7fb6cd000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6ac004800 0x7fb6c81f2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:51.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.294+0000 7fb6cd000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6c80c99c0 0x7fb6c80c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:51.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.294+0000 7fb6cd000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6c80ca310 0x7fb6c81f8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:51.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.294+0000 7fb6cd000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c80b49b0 con 0x7fb6ac004800 2024-09-17T04:30:51.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.294+0000 7fb6cd000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c80b4830 con 0x7fb6c80c99c0 2024-09-17T04:30:51.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.294+0000 7fb6cd000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb6c806d720 con 0x7fb6c80ca310 2024-09-17T04:30:51.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.294+0000 7fb6c5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6ac004800 0x7fb6c81f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:51.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.294+0000 7fb6c5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6ac004800 0x7fb6c81f2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35960/0 (socket says 172.21.15.100:35960) 2024-09-17T04:30:51.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.294+0000 7fb6c5e00640 1 -- 172.21.15.100:0/1786268231 learned_addr learned my addr 172.21.15.100:0/1786268231 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:51.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.294+0000 7fb6c5e00640 1 -- 172.21.15.100:0/1786268231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6c80ca310 msgr2=0x7fb6c81f8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:51.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.294+0000 7fb6c5e00640 1 --2- 172.21.15.100:0/1786268231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6c80ca310 0x7fb6c81f8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:51.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.294+0000 7fb6c5e00640 1 -- 172.21.15.100:0/1786268231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6c80c99c0 msgr2=0x7fb6c80c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:51.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.294+0000 7fb6c5e00640 1 --2- 172.21.15.100:0/1786268231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6c80c99c0 0x7fb6c80c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:51.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.295+0000 7fb6c5e00640 1 -- 172.21.15.100:0/1786268231 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c81552f0 con 0x7fb6ac004800 2024-09-17T04:30:51.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.295+0000 7fb6c5e00640 1 --2- 172.21.15.100:0/1786268231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6ac004800 0x7fb6c81f2780 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8006650 tx=0x7fb6b8001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:51.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.295+0000 7fb6be000640 1 -- 172.21.15.100:0/1786268231 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8011640 con 0x7fb6ac004800 2024-09-17T04:30:51.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.295+0000 7fb6be000640 1 -- 172.21.15.100:0/1786268231 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6b8016020 con 0x7fb6ac004800 2024-09-17T04:30:51.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.295+0000 7fb6cd000640 1 -- 172.21.15.100:0/1786268231 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c81557f0 con 0x7fb6ac004800 2024-09-17T04:30:51.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.295+0000 7fb6be000640 1 -- 172.21.15.100:0/1786268231 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8011a40 con 0x7fb6ac004800 2024-09-17T04:30:51.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.295+0000 7fb6cd000640 1 -- 172.21.15.100:0/1786268231 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c8155db0 con 0x7fb6ac004800 2024-09-17T04:30:51.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.296+0000 7fb6cd000640 1 -- 172.21.15.100:0/1786268231 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6c80af510 con 0x7fb6ac004800 2024-09-17T04:30:51.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.297+0000 7fb6be000640 1 -- 172.21.15.100:0/1786268231 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7fb6b8011be0 con 0x7fb6ac004800 2024-09-17T04:30:51.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.298+0000 7fb6be000640 1 --2- 172.21.15.100:0/1786268231 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb6a8078ef0 0x7fb6a807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:51.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.298+0000 7fb6be000640 1 -- 172.21.15.100:0/1786268231 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7fb6b809bff0 con 0x7fb6ac004800 2024-09-17T04:30:51.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.299+0000 7fb6c5400640 1 --2- 172.21.15.100:0/1786268231 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb6a8078ef0 0x7fb6a807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:51.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.299+0000 7fb6c5400640 1 --2- 172.21.15.100:0/1786268231 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb6a8078ef0 0x7fb6a807b400 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4034000 tx=0x7fb6b4004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:51.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.303+0000 7fb6be000640 1 -- 172.21.15.100:0/1786268231 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb6b8020020 con 0x7fb6ac004800 2024-09-17T04:30:51.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.519+0000 7fb6cd000640 1 -- 172.21.15.100:0/1786268231 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol80190", "snap_name": "subvol_snap99850", "target": ["mon-mgr", ""]}) -- 0x7fb6c80d1030 con 0x7fb6a8078ef0 2024-09-17T04:30:51.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.554+0000 7fb6be000640 1 -- 172.21.15.100:0/1786268231 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb6c80d1030 con 0x7fb6a8078ef0 2024-09-17T04:30:51.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.556+0000 7fb6cd000640 1 -- 172.21.15.100:0/1786268231 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb6a8078ef0 msgr2=0x7fb6a807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:51.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.556+0000 7fb6cd000640 1 --2- 172.21.15.100:0/1786268231 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb6a8078ef0 0x7fb6a807b400 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4034000 tx=0x7fb6b4004730 comp rx=0 tx=0).stop 2024-09-17T04:30:51.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.556+0000 7fb6cd000640 1 -- 172.21.15.100:0/1786268231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6ac004800 msgr2=0x7fb6c81f2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:51.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.556+0000 7fb6cd000640 1 --2- 172.21.15.100:0/1786268231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6ac004800 0x7fb6c81f2780 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8006650 tx=0x7fb6b8001800 comp rx=0 tx=0).stop 2024-09-17T04:30:51.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.556+0000 7fb6cd000640 1 -- 172.21.15.100:0/1786268231 shutdown_connections 2024-09-17T04:30:51.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.556+0000 7fb6cd000640 1 --2- 172.21.15.100:0/1786268231 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb6a8078ef0 0x7fb6a807b400 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:51.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.556+0000 7fb6cd000640 1 --2- 172.21.15.100:0/1786268231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6c80ca310 0x7fb6c81f8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:51.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.556+0000 7fb6cd000640 1 --2- 172.21.15.100:0/1786268231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6c80c99c0 0x7fb6c80c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:51.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.556+0000 7fb6cd000640 1 --2- 172.21.15.100:0/1786268231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6ac004800 0x7fb6c81f2780 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:51.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.556+0000 7fb6cd000640 1 -- 172.21.15.100:0/1786268231 >> 172.21.15.100:0/1786268231 conn(0x7fb6c80b2540 msgr2=0x7fb6c806f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:51.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.556+0000 7fb6cd000640 1 -- 172.21.15.100:0/1786268231 shutdown_connections 2024-09-17T04:30:51.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.557+0000 7fb6cd000640 1 -- 172.21.15.100:0/1786268231 wait complete. 2024-09-17T04:30:51.580 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol80190 fake 2024-09-17T04:30:51.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.809+0000 7f6373400640 1 -- 172.21.15.100:0/2951456025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f636c15ba50 msgr2=0x7f636c0b3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:51.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.809+0000 7f6373400640 1 --2- 172.21.15.100:0/2951456025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f636c15ba50 0x7f636c0b3120 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f6358009830 tx=0x7f635802a3f0 comp rx=0 tx=0).stop 2024-09-17T04:30:51.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.810+0000 7f6373400640 1 -- 172.21.15.100:0/2951456025 shutdown_connections 2024-09-17T04:30:51.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.810+0000 7f6373400640 1 --2- 172.21.15.100:0/2951456025 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f636c0b36f0 0x7f636c0cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:51.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.810+0000 7f6373400640 1 --2- 172.21.15.100:0/2951456025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f636c15ba50 0x7f636c0b3120 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:51.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.810+0000 7f6373400640 1 -- 172.21.15.100:0/2951456025 >> 172.21.15.100:0/2951456025 conn(0x7f636c0b2540 msgr2=0x7f636c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:51.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.810+0000 7f6373400640 1 -- 172.21.15.100:0/2951456025 shutdown_connections 2024-09-17T04:30:51.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.810+0000 7f6373400640 1 -- 172.21.15.100:0/2951456025 wait complete. 2024-09-17T04:30:51.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.812+0000 7f6373400640 1 Processor -- start 2024-09-17T04:30:51.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.812+0000 7f6373400640 1 -- start start 2024-09-17T04:30:51.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.813+0000 7f6373400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f636c0b36f0 0x7f636c1579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:51.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.813+0000 7f6373400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f636c0cc7d0 0x7f636c1f6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:51.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.813+0000 7f6373400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f636c15ba50 0x7f636c0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:51.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.813+0000 7f6373400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f636c06f8a0 con 0x7f636c0cc7d0 2024-09-17T04:30:51.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.813+0000 7f6373400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f636c06f720 con 0x7f636c0b36f0 2024-09-17T04:30:51.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.813+0000 7f6373400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f636c06fa20 con 0x7f636c15ba50 2024-09-17T04:30:51.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.814+0000 7f636be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f636c0cc7d0 0x7f636c1f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:51.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.814+0000 7f636be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f636c0cc7d0 0x7f636c1f6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35984/0 (socket says 172.21.15.100:35984) 2024-09-17T04:30:51.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.814+0000 7f636be00640 1 -- 172.21.15.100:0/2843291836 learned_addr learned my addr 172.21.15.100:0/2843291836 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:51.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.814+0000 7f6370a00640 1 --2- 172.21.15.100:0/2843291836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f636c0b36f0 0x7f636c1579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:51.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.814+0000 7f6371400640 1 --2- 172.21.15.100:0/2843291836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f636c15ba50 0x7f636c0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:51.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.814+0000 7f636be00640 1 -- 172.21.15.100:0/2843291836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f636c15ba50 msgr2=0x7f636c0c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:51.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.814+0000 7f636be00640 1 --2- 172.21.15.100:0/2843291836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f636c15ba50 0x7f636c0c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:51.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.814+0000 7f636be00640 1 -- 172.21.15.100:0/2843291836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f636c0b36f0 msgr2=0x7f636c1579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:51.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.814+0000 7f636be00640 1 --2- 172.21.15.100:0/2843291836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f636c0b36f0 0x7f636c1579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:51.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.814+0000 7f636be00640 1 -- 172.21.15.100:0/2843291836 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636c1f6d70 con 0x7f636c0cc7d0 2024-09-17T04:30:51.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.814+0000 7f6370a00640 1 --2- 172.21.15.100:0/2843291836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f636c0b36f0 0x7f636c1579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:30:51.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.814+0000 7f6371400640 1 --2- 172.21.15.100:0/2843291836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f636c15ba50 0x7f636c0c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:30:51.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.815+0000 7f636be00640 1 --2- 172.21.15.100:0/2843291836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f636c0cc7d0 0x7f636c1f6830 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f6358009950 tx=0x7f6358033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:51.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.815+0000 7f6368c00640 1 -- 172.21.15.100:0/2843291836 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6358031d60 con 0x7f636c0cc7d0 2024-09-17T04:30:51.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.815+0000 7f6368c00640 1 -- 172.21.15.100:0/2843291836 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6358033e10 con 0x7f636c0cc7d0 2024-09-17T04:30:51.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.815+0000 7f6368c00640 1 -- 172.21.15.100:0/2843291836 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6358005260 con 0x7f636c0cc7d0 2024-09-17T04:30:51.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.815+0000 7f6373400640 1 -- 172.21.15.100:0/2843291836 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f636c1f37b0 con 0x7f636c0cc7d0 2024-09-17T04:30:51.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.815+0000 7f6373400640 1 -- 172.21.15.100:0/2843291836 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f636c156ee0 con 0x7f636c0cc7d0 2024-09-17T04:30:51.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.816+0000 7f6373400640 1 -- 172.21.15.100:0/2843291836 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f636c06a700 con 0x7f636c0cc7d0 2024-09-17T04:30:51.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.817+0000 7f6368c00640 1 -- 172.21.15.100:0/2843291836 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7f6358005400 con 0x7f636c0cc7d0 2024-09-17T04:30:51.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.818+0000 7f6368c00640 1 --2- 172.21.15.100:0/2843291836 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f634c078e60 0x7f634c07b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:51.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.818+0000 7f6368c00640 1 -- 172.21.15.100:0/2843291836 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f63580bc760 con 0x7f636c0cc7d0 2024-09-17T04:30:51.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.818+0000 7f6370a00640 1 --2- 172.21.15.100:0/2843291836 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f634c078e60 0x7f634c07b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:51.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.819+0000 7f6370a00640 1 --2- 172.21.15.100:0/2843291836 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f634c078e60 0x7f634c07b370 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6360000f60 tx=0x7f6360009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:51.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:51.823+0000 7f6368c00640 1 -- 172.21.15.100:0/2843291836 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f636c06a700 con 0x7f636c0cc7d0 2024-09-17T04:30:52.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.038+0000 7f6373400640 1 -- 172.21.15.100:0/2843291836 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol80190", "snap_name": "fake", "target": ["mon-mgr", ""]}) -- 0x7f636c06ff40 con 0x7f634c078e60 2024-09-17T04:30:52.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.071+0000 7f6368c00640 1 -- 172.21.15.100:0/2843291836 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f636c06ff40 con 0x7f634c078e60 2024-09-17T04:30:52.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.073+0000 7f6373400640 1 -- 172.21.15.100:0/2843291836 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f634c078e60 msgr2=0x7f634c07b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:52.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.073+0000 7f6373400640 1 --2- 172.21.15.100:0/2843291836 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f634c078e60 0x7f634c07b370 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6360000f60 tx=0x7f6360009040 comp rx=0 tx=0).stop 2024-09-17T04:30:52.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.073+0000 7f6373400640 1 -- 172.21.15.100:0/2843291836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f636c0cc7d0 msgr2=0x7f636c1f6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:52.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.073+0000 7f6373400640 1 --2- 172.21.15.100:0/2843291836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f636c0cc7d0 0x7f636c1f6830 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f6358009950 tx=0x7f6358033c20 comp rx=0 tx=0).stop 2024-09-17T04:30:52.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.073+0000 7f6373400640 1 -- 172.21.15.100:0/2843291836 shutdown_connections 2024-09-17T04:30:52.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.073+0000 7f6373400640 1 --2- 172.21.15.100:0/2843291836 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f634c078e60 0x7f634c07b370 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:52.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.073+0000 7f6373400640 1 --2- 172.21.15.100:0/2843291836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f636c15ba50 0x7f636c0c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:52.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.073+0000 7f6373400640 1 --2- 172.21.15.100:0/2843291836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f636c0cc7d0 0x7f636c1f6830 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:52.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.073+0000 7f6373400640 1 --2- 172.21.15.100:0/2843291836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f636c0b36f0 0x7f636c1579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:52.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.073+0000 7f6373400640 1 -- 172.21.15.100:0/2843291836 >> 172.21.15.100:0/2843291836 conn(0x7f636c0b2540 msgr2=0x7f636c0ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:52.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.073+0000 7f6373400640 1 -- 172.21.15.100:0/2843291836 shutdown_connections 2024-09-17T04:30:52.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.073+0000 7f6373400640 1 -- 172.21.15.100:0/2843291836 wait complete. 2024-09-17T04:30:52.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:52.100 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol80190/.meta) 2024-09-17T04:30:52.156 INFO:teuthology.orchestra.run.smithi100.stdout:[GLOBAL] 2024-09-17T04:30:52.156 INFO:teuthology.orchestra.run.smithi100.stdout:version = 1 2024-09-17T04:30:52.157 INFO:teuthology.orchestra.run.smithi100.stdout:type = subvolume 2024-09-17T04:30:52.157 INFO:teuthology.orchestra.run.smithi100.stdout:path = /volumes/_nogroup/subvol80190/2e0c5ce8-bd2e-4ac7-93be-29db3fa785a4 2024-09-17T04:30:52.157 INFO:teuthology.orchestra.run.smithi100.stdout:state = complete 2024-09-17T04:30:52.157 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:30:52.160 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol80190 2024-09-17T04:30:52.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.381+0000 7f26e2200640 1 -- 172.21.15.100:0/184977546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26dc145550 msgr2=0x7f26dc06fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:52.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.381+0000 7f26e2200640 1 --2- 172.21.15.100:0/184977546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26dc145550 0x7f26dc06fb40 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f26cc009830 tx=0x7f26cc02a640 comp rx=0 tx=0).stop 2024-09-17T04:30:52.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.381+0000 7f26e2200640 1 -- 172.21.15.100:0/184977546 shutdown_connections 2024-09-17T04:30:52.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.381+0000 7f26e2200640 1 --2- 172.21.15.100:0/184977546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26dc145550 0x7f26dc06fb40 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:52.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.381+0000 7f26e2200640 1 -- 172.21.15.100:0/184977546 >> 172.21.15.100:0/184977546 conn(0x7f26dc06d1c0 msgr2=0x7f26dc06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:52.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.382+0000 7f26e2200640 1 -- 172.21.15.100:0/184977546 shutdown_connections 2024-09-17T04:30:52.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.382+0000 7f26e2200640 1 -- 172.21.15.100:0/184977546 wait complete. 2024-09-17T04:30:52.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.384+0000 7f26e2200640 1 Processor -- start 2024-09-17T04:30:52.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.384+0000 7f26e2200640 1 -- start start 2024-09-17T04:30:52.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.385+0000 7f26e2200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26dc145550 0x7f26dc1f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:52.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.385+0000 7f26e2200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26dc145a40 0x7f26dc15c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:52.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.385+0000 7f26e2200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26dc0af3c0 0x7f26dc15ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:52.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.385+0000 7f26e2200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc0ce360 con 0x7f26dc145550 2024-09-17T04:30:52.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.385+0000 7f26e2200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc0ce1e0 con 0x7f26dc145a40 2024-09-17T04:30:52.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.385+0000 7f26e2200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f26dc0ce4e0 con 0x7f26dc0af3c0 2024-09-17T04:30:52.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.386+0000 7f26db400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26dc145550 0x7f26dc1f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:52.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.386+0000 7f26db400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26dc145550 0x7f26dc1f53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35996/0 (socket says 172.21.15.100:35996) 2024-09-17T04:30:52.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.386+0000 7f26db400640 1 -- 172.21.15.100:0/2362592752 learned_addr learned my addr 172.21.15.100:0/2362592752 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:52.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.386+0000 7f26daa00640 1 --2- 172.21.15.100:0/2362592752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26dc145a40 0x7f26dc15c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:52.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.386+0000 7f26dbe00640 1 --2- 172.21.15.100:0/2362592752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26dc0af3c0 0x7f26dc15ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:52.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.386+0000 7f26db400640 1 -- 172.21.15.100:0/2362592752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26dc0af3c0 msgr2=0x7f26dc15ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:52.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.386+0000 7f26db400640 1 --2- 172.21.15.100:0/2362592752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26dc0af3c0 0x7f26dc15ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:52.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.386+0000 7f26db400640 1 -- 172.21.15.100:0/2362592752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26dc145a40 msgr2=0x7f26dc15c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:52.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.386+0000 7f26db400640 1 --2- 172.21.15.100:0/2362592752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26dc145a40 0x7f26dc15c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:52.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.386+0000 7f26db400640 1 -- 172.21.15.100:0/2362592752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26dc0ae320 con 0x7f26dc145550 2024-09-17T04:30:52.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.386+0000 7f26db400640 1 --2- 172.21.15.100:0/2362592752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26dc145550 0x7f26dc1f53a0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f26cc000c00 tx=0x7f26cc033df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:52.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.387+0000 7f26d3400640 1 -- 172.21.15.100:0/2362592752 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26cc0332e0 con 0x7f26dc145550 2024-09-17T04:30:52.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.387+0000 7f26d3400640 1 -- 172.21.15.100:0/2362592752 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26cc033570 con 0x7f26dc145550 2024-09-17T04:30:52.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.387+0000 7f26d3400640 1 -- 172.21.15.100:0/2362592752 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26cc031a10 con 0x7f26dc145550 2024-09-17T04:30:52.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.387+0000 7f26e2200640 1 -- 172.21.15.100:0/2362592752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26dc0ae600 con 0x7f26dc145550 2024-09-17T04:30:52.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.387+0000 7f26e2200640 1 -- 172.21.15.100:0/2362592752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26dc0aebc0 con 0x7f26dc145550 2024-09-17T04:30:52.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.388+0000 7f26e2200640 1 -- 172.21.15.100:0/2362592752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26dc06fb40 con 0x7f26dc145550 2024-09-17T04:30:52.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.389+0000 7f26d3400640 1 -- 172.21.15.100:0/2362592752 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7f26cc031bb0 con 0x7f26dc145550 2024-09-17T04:30:52.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.390+0000 7f26d3400640 1 --2- 172.21.15.100:0/2362592752 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f26bc078f20 0x7f26bc07b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:52.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.390+0000 7f26d3400640 1 -- 172.21.15.100:0/2362592752 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f26cc0bc060 con 0x7f26dc145550 2024-09-17T04:30:52.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.390+0000 7f26daa00640 1 --2- 172.21.15.100:0/2362592752 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f26bc078f20 0x7f26bc07b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:52.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.391+0000 7f26daa00640 1 --2- 172.21.15.100:0/2362592752 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f26bc078f20 0x7f26bc07b430 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f26dc1f9aa0 tx=0x7f26c8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:52.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.395+0000 7f26d3400640 1 -- 172.21.15.100:0/2362592752 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f26cc04a070 con 0x7f26dc145550 2024-09-17T04:30:52.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.602+0000 7f26e2200640 1 -- 172.21.15.100:0/2362592752 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol80190", "target": ["mon-mgr", ""]}) -- 0x7f26dc0b22d0 con 0x7f26bc078f20 2024-09-17T04:30:52.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.645+0000 7f26d3400640 1 -- 172.21.15.100:0/2362592752 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f26dc0b22d0 con 0x7f26bc078f20 2024-09-17T04:30:52.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.648+0000 7f26e2200640 1 -- 172.21.15.100:0/2362592752 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f26bc078f20 msgr2=0x7f26bc07b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:52.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.648+0000 7f26e2200640 1 --2- 172.21.15.100:0/2362592752 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f26bc078f20 0x7f26bc07b430 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f26dc1f9aa0 tx=0x7f26c8009040 comp rx=0 tx=0).stop 2024-09-17T04:30:52.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.648+0000 7f26e2200640 1 -- 172.21.15.100:0/2362592752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26dc145550 msgr2=0x7f26dc1f53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:52.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.648+0000 7f26e2200640 1 --2- 172.21.15.100:0/2362592752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26dc145550 0x7f26dc1f53a0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f26cc000c00 tx=0x7f26cc033df0 comp rx=0 tx=0).stop 2024-09-17T04:30:52.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.648+0000 7f26e2200640 1 -- 172.21.15.100:0/2362592752 shutdown_connections 2024-09-17T04:30:52.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.648+0000 7f26e2200640 1 --2- 172.21.15.100:0/2362592752 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f26bc078f20 0x7f26bc07b430 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:52.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.648+0000 7f26e2200640 1 --2- 172.21.15.100:0/2362592752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26dc0af3c0 0x7f26dc15ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:52.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.648+0000 7f26e2200640 1 --2- 172.21.15.100:0/2362592752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26dc145a40 0x7f26dc15c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:52.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.648+0000 7f26e2200640 1 --2- 172.21.15.100:0/2362592752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26dc145550 0x7f26dc1f53a0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:52.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.648+0000 7f26e2200640 1 -- 172.21.15.100:0/2362592752 >> 172.21.15.100:0/2362592752 conn(0x7f26dc06d1c0 msgr2=0x7f26dc142bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:52.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.648+0000 7f26e2200640 1 -- 172.21.15.100:0/2362592752 shutdown_connections 2024-09-17T04:30:52.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.648+0000 7f26e2200640 1 -- 172.21.15.100:0/2362592752 wait complete. 2024-09-17T04:30:52.651 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:52.650+0000 7f30b4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:52.651 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:52.650+0000 7f30b4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:52.651 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:52.650+0000 7f30b4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:52.651 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:52.650+0000 7f30b4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:52.651 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:52.650+0000 7f30b4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:52.669 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:52.669+0000 7f30b6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:52.669 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:52.669+0000 7f30b6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:52.669 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:52.669+0000 7f30b6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:52.669 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:52.669+0000 7f30b6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:52.669 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:30:52.669+0000 7f30b6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:30:52.681 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol_clone15410 2024-09-17T04:30:52.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.912+0000 7fb667200640 1 -- 172.21.15.100:0/1705501999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb660073a80 msgr2=0x7fb6600cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:52.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.912+0000 7fb667200640 1 --2- 172.21.15.100:0/1705501999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb660073a80 0x7fb6600cd610 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fb660160280 tx=0x7fb65802a960 comp rx=0 tx=0).stop 2024-09-17T04:30:52.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.912+0000 7fb667200640 1 -- 172.21.15.100:0/1705501999 shutdown_connections 2024-09-17T04:30:52.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.912+0000 7fb667200640 1 --2- 172.21.15.100:0/1705501999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb660073a80 0x7fb6600cd610 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:52.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.912+0000 7fb667200640 1 --2- 172.21.15.100:0/1705501999 >> v2:172.21.15.184:3300/0 conn(0x7fb660145a40 0x7fb660073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:52.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.912+0000 7fb667200640 1 -- 172.21.15.100:0/1705501999 >> 172.21.15.100:0/1705501999 conn(0x7fb66006d1c0 msgr2=0x7fb66006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:52.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.912+0000 7fb667200640 1 -- 172.21.15.100:0/1705501999 shutdown_connections 2024-09-17T04:30:52.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.912+0000 7fb667200640 1 -- 172.21.15.100:0/1705501999 wait complete. 2024-09-17T04:30:52.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.914+0000 7fb667200640 1 Processor -- start 2024-09-17T04:30:52.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.914+0000 7fb667200640 1 -- start start 2024-09-17T04:30:52.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.916+0000 7fb667200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb660073a80 0x7fb6601381f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:52.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.916+0000 7fb667200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb660145550 0x7fb6601393f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:52.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.916+0000 7fb667200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb660145a40 0x7fb660139930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:52.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.916+0000 7fb667200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb6600ce0e0 con 0x7fb660145a40 2024-09-17T04:30:52.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.916+0000 7fb667200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb6600cdf60 con 0x7fb660073a80 2024-09-17T04:30:52.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.916+0000 7fb667200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb6600ce260 con 0x7fb660145550 2024-09-17T04:30:52.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.916+0000 7fb665200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb660145a40 0x7fb660139930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:52.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.916+0000 7fb665200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb660145a40 0x7fb660139930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36064/0 (socket says 172.21.15.100:36064) 2024-09-17T04:30:52.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.916+0000 7fb665200640 1 -- 172.21.15.100:0/2034649692 learned_addr learned my addr 172.21.15.100:0/2034649692 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:52.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.916+0000 7fb65f400640 1 --2- 172.21.15.100:0/2034649692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb660145550 0x7fb6601393f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:52.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.916+0000 7fb65fe00640 1 --2- 172.21.15.100:0/2034649692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb660073a80 0x7fb6601381f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:52.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.916+0000 7fb665200640 1 -- 172.21.15.100:0/2034649692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb660145550 msgr2=0x7fb6601393f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:52.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.916+0000 7fb665200640 1 --2- 172.21.15.100:0/2034649692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb660145550 0x7fb6601393f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:52.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.916+0000 7fb665200640 1 -- 172.21.15.100:0/2034649692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb660073a80 msgr2=0x7fb6601381f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:52.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.916+0000 7fb665200640 1 --2- 172.21.15.100:0/2034649692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb660073a80 0x7fb6601381f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:52.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.916+0000 7fb665200640 1 -- 172.21.15.100:0/2034649692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb66015be10 con 0x7fb660145a40 2024-09-17T04:30:52.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.916+0000 7fb65fe00640 1 --2- 172.21.15.100:0/2034649692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb660073a80 0x7fb6601381f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:30:52.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.917+0000 7fb665200640 1 --2- 172.21.15.100:0/2034649692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb660145a40 0x7fb660139930 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fb65802af50 tx=0x7fb658032bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:52.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.917+0000 7fb657e00640 1 -- 172.21.15.100:0/2034649692 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb658030ae0 con 0x7fb660145a40 2024-09-17T04:30:52.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.917+0000 7fb657e00640 1 -- 172.21.15.100:0/2034649692 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb65803e070 con 0x7fb660145a40 2024-09-17T04:30:52.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.917+0000 7fb667200640 1 -- 172.21.15.100:0/2034649692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb660138930 con 0x7fb660145a40 2024-09-17T04:30:52.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.917+0000 7fb657e00640 1 -- 172.21.15.100:0/2034649692 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6580320d0 con 0x7fb660145a40 2024-09-17T04:30:52.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.917+0000 7fb667200640 1 -- 172.21.15.100:0/2034649692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb660138bf0 con 0x7fb660145a40 2024-09-17T04:30:52.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.918+0000 7fb667200640 1 -- 172.21.15.100:0/2034649692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6600ce780 con 0x7fb660145a40 2024-09-17T04:30:52.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.919+0000 7fb657e00640 1 -- 172.21.15.100:0/2034649692 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7fb658032270 con 0x7fb660145a40 2024-09-17T04:30:52.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.920+0000 7fb657e00640 1 --2- 172.21.15.100:0/2034649692 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb640078ef0 0x7fb64007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:52.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.920+0000 7fb657e00640 1 -- 172.21.15.100:0/2034649692 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7fb6580bcde0 con 0x7fb660145a40 2024-09-17T04:30:52.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.921+0000 7fb65fe00640 1 --2- 172.21.15.100:0/2034649692 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb640078ef0 0x7fb64007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:52.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.921+0000 7fb65fe00640 1 --2- 172.21.15.100:0/2034649692 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb640078ef0 0x7fb64007b400 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb650000f60 tx=0x7fb650009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:52.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:52.925+0000 7fb657e00640 1 -- 172.21.15.100:0/2034649692 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb658084c10 con 0x7fb660145a40 2024-09-17T04:30:53.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.140+0000 7fb667200640 1 -- 172.21.15.100:0/2034649692 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol_clone15410", "target": ["mon-mgr", ""]}) -- 0x7fb660139e70 con 0x7fb640078ef0 2024-09-17T04:30:53.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.160+0000 7fb657e00640 1 -- 172.21.15.100:0/2034649692 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb6500013a0 con 0x7fb640078ef0 2024-09-17T04:30:53.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.162+0000 7fb667200640 1 -- 172.21.15.100:0/2034649692 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb640078ef0 msgr2=0x7fb64007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:53.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.162+0000 7fb667200640 1 --2- 172.21.15.100:0/2034649692 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb640078ef0 0x7fb64007b400 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb650000f60 tx=0x7fb650009040 comp rx=0 tx=0).stop 2024-09-17T04:30:53.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.162+0000 7fb667200640 1 -- 172.21.15.100:0/2034649692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb660145a40 msgr2=0x7fb660139930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:53.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.162+0000 7fb667200640 1 --2- 172.21.15.100:0/2034649692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb660145a40 0x7fb660139930 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fb65802af50 tx=0x7fb658032bb0 comp rx=0 tx=0).stop 2024-09-17T04:30:53.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.162+0000 7fb667200640 1 -- 172.21.15.100:0/2034649692 shutdown_connections 2024-09-17T04:30:53.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.162+0000 7fb667200640 1 --2- 172.21.15.100:0/2034649692 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb640078ef0 0x7fb64007b400 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:53.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.162+0000 7fb667200640 1 --2- 172.21.15.100:0/2034649692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb660145a40 0x7fb660139930 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:53.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.162+0000 7fb667200640 1 --2- 172.21.15.100:0/2034649692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb660145550 0x7fb6601393f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:53.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.162+0000 7fb667200640 1 --2- 172.21.15.100:0/2034649692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb660073a80 0x7fb6601381f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:53.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.162+0000 7fb667200640 1 -- 172.21.15.100:0/2034649692 >> 172.21.15.100:0/2034649692 conn(0x7fb66006d1c0 msgr2=0x7fb660142bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:53.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.163+0000 7fb667200640 1 -- 172.21.15.100:0/2034649692 shutdown_connections 2024-09-17T04:30:53.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.163+0000 7fb667200640 1 -- 172.21.15.100:0/2034649692 wait complete. 2024-09-17T04:30:53.198 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol_clone89001 2024-09-17T04:30:53.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.433+0000 7eff2a400640 1 -- 172.21.15.100:0/3517798012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff240cd1d0 msgr2=0x7eff240af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:53.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.433+0000 7eff2a400640 1 --2- 172.21.15.100:0/3517798012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff240cd1d0 0x7eff240af0d0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7eff1c005fd0 tx=0x7eff1c02aa90 comp rx=0 tx=0).stop 2024-09-17T04:30:53.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.433+0000 7eff2a400640 1 -- 172.21.15.100:0/3517798012 shutdown_connections 2024-09-17T04:30:53.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.433+0000 7eff2a400640 1 --2- 172.21.15.100:0/3517798012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff240cd1d0 0x7eff240af0d0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:53.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.433+0000 7eff2a400640 1 --2- 172.21.15.100:0/3517798012 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff240b3f30 0x7eff24156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:53.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.433+0000 7eff2a400640 1 -- 172.21.15.100:0/3517798012 >> 172.21.15.100:0/3517798012 conn(0x7eff2406d620 msgr2=0x7eff240b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:53.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.433+0000 7eff2a400640 1 -- 172.21.15.100:0/3517798012 shutdown_connections 2024-09-17T04:30:53.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.433+0000 7eff2a400640 1 -- 172.21.15.100:0/3517798012 wait complete. 2024-09-17T04:30:53.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.435+0000 7eff2a400640 1 Processor -- start 2024-09-17T04:30:53.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.436+0000 7eff2a400640 1 -- start start 2024-09-17T04:30:53.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.437+0000 7eff2a400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff240b3f30 0x7eff241f5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:53.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.437+0000 7eff2a400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff240cd1d0 0x7eff24154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:53.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.437+0000 7eff2a400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff2415d6e0 0x7eff2409da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:53.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.437+0000 7eff2a400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7eff240af9a0 con 0x7eff240cd1d0 2024-09-17T04:30:53.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.437+0000 7eff2a400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7eff240af820 con 0x7eff240b3f30 2024-09-17T04:30:53.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.437+0000 7eff2a400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7eff240afb20 con 0x7eff2415d6e0 2024-09-17T04:30:53.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.437+0000 7eff1be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff240cd1d0 0x7eff24154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:53.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.437+0000 7eff1be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff240cd1d0 0x7eff24154bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36076/0 (socket says 172.21.15.100:36076) 2024-09-17T04:30:53.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.437+0000 7eff1be00640 1 -- 172.21.15.100:0/396473179 learned_addr learned my addr 172.21.15.100:0/396473179 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:53.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.438+0000 7eff29000640 1 --2- 172.21.15.100:0/396473179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff240b3f30 0x7eff241f5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:53.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.438+0000 7eff29a00640 1 --2- 172.21.15.100:0/396473179 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff2415d6e0 0x7eff2409da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:53.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.438+0000 7eff1be00640 1 -- 172.21.15.100:0/396473179 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff2415d6e0 msgr2=0x7eff2409da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:53.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.438+0000 7eff1be00640 1 --2- 172.21.15.100:0/396473179 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff2415d6e0 0x7eff2409da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:53.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.438+0000 7eff1be00640 1 -- 172.21.15.100:0/396473179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff240b3f30 msgr2=0x7eff241f5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.438+0000 7eff1be00640 1 --2- 172.21.15.100:0/396473179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff240b3f30 0x7eff241f5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.438+0000 7eff1be00640 1 -- 172.21.15.100:0/396473179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff241f5850 con 0x7eff240cd1d0 2024-09-17T04:30:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.438+0000 7eff1be00640 1 --2- 172.21.15.100:0/396473179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff240cd1d0 0x7eff24154bd0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00c3c0 tx=0x7eff0c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.439+0000 7eff21600640 1 -- 172.21.15.100:0/396473179 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c0052d0 con 0x7eff240cd1d0 2024-09-17T04:30:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.439+0000 7eff21600640 1 -- 172.21.15.100:0/396473179 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff0c007020 con 0x7eff240cd1d0 2024-09-17T04:30:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.439+0000 7eff2a400640 1 -- 172.21.15.100:0/396473179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff2415c400 con 0x7eff240cd1d0 2024-09-17T04:30:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.439+0000 7eff21600640 1 -- 172.21.15.100:0/396473179 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c007470 con 0x7eff240cd1d0 2024-09-17T04:30:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.439+0000 7eff2a400640 1 -- 172.21.15.100:0/396473179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff241f7900 con 0x7eff240cd1d0 2024-09-17T04:30:53.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.440+0000 7eff2a400640 1 -- 172.21.15.100:0/396473179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff24145e40 con 0x7eff240cd1d0 2024-09-17T04:30:53.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.442+0000 7eff21600640 1 -- 172.21.15.100:0/396473179 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7eff0c0067d0 con 0x7eff240cd1d0 2024-09-17T04:30:53.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.443+0000 7eff21600640 1 --2- 172.21.15.100:0/396473179 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7eff00078e60 0x7eff0007b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:53.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.444+0000 7eff21600640 1 -- 172.21.15.100:0/396473179 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7eff0c09a710 con 0x7eff240cd1d0 2024-09-17T04:30:53.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.444+0000 7eff29000640 1 --2- 172.21.15.100:0/396473179 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7eff00078e60 0x7eff0007b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:53.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.444+0000 7eff29000640 1 --2- 172.21.15.100:0/396473179 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7eff00078e60 0x7eff0007b370 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7eff1400bb50 tx=0x7eff1400b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:53.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.452+0000 7eff21600640 1 -- 172.21.15.100:0/396473179 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7eff0c062020 con 0x7eff240cd1d0 2024-09-17T04:30:53.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.670+0000 7eff2a400640 1 -- 172.21.15.100:0/396473179 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol_clone89001", "target": ["mon-mgr", ""]}) -- 0x7eff24156ca0 con 0x7eff00078e60 2024-09-17T04:30:53.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.686+0000 7eff21600640 1 -- 172.21.15.100:0/396473179 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7eff24156ca0 con 0x7eff00078e60 2024-09-17T04:30:53.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.689+0000 7eff2a400640 1 -- 172.21.15.100:0/396473179 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7eff00078e60 msgr2=0x7eff0007b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.689+0000 7eff2a400640 1 --2- 172.21.15.100:0/396473179 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7eff00078e60 0x7eff0007b370 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7eff1400bb50 tx=0x7eff1400b2e0 comp rx=0 tx=0).stop 2024-09-17T04:30:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.689+0000 7eff2a400640 1 -- 172.21.15.100:0/396473179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff240cd1d0 msgr2=0x7eff24154bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.689+0000 7eff2a400640 1 --2- 172.21.15.100:0/396473179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff240cd1d0 0x7eff24154bd0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00c3c0 tx=0x7eff0c00c880 comp rx=0 tx=0).stop 2024-09-17T04:30:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.689+0000 7eff2a400640 1 -- 172.21.15.100:0/396473179 shutdown_connections 2024-09-17T04:30:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.689+0000 7eff2a400640 1 --2- 172.21.15.100:0/396473179 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7eff00078e60 0x7eff0007b370 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.689+0000 7eff2a400640 1 --2- 172.21.15.100:0/396473179 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff2415d6e0 0x7eff2409da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.689+0000 7eff2a400640 1 --2- 172.21.15.100:0/396473179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff240cd1d0 0x7eff24154bd0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.689+0000 7eff2a400640 1 --2- 172.21.15.100:0/396473179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff240b3f30 0x7eff241f5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.689+0000 7eff2a400640 1 -- 172.21.15.100:0/396473179 >> 172.21.15.100:0/396473179 conn(0x7eff2406d620 msgr2=0x7eff2415e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.689+0000 7eff2a400640 1 -- 172.21.15.100:0/396473179 shutdown_connections 2024-09-17T04:30:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53.689+0000 7eff2a400640 1 -- 172.21.15.100:0/396473179 wait complete. 2024-09-17T04:30:53.726 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-17T04:30:53.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:30:53.812 INFO:teuthology.orchestra.run.smithi100.stdout:2 2024-09-17T04:30:53.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:30:53.823 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T04:30:53.824 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:30:53.919 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:30:54.005 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T04:30:54.005 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:30:54.005 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:30:54.005 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:30:54.042 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:30:54.042 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:30:54.118 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:30:54.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.384+0000 7f0ca3200640 1 -- 172.21.15.100:0/1806302718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ca40af170 msgr2=0x7f0ca40b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:54.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.384+0000 7f0ca3200640 1 --2- 172.21.15.100:0/1806302718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ca40af170 0x7f0ca40b2f60 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f0c90009830 tx=0x7f0c9002a4a0 comp rx=0 tx=0).stop 2024-09-17T04:30:54.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.385+0000 7f0ca3200640 1 -- 172.21.15.100:0/1806302718 shutdown_connections 2024-09-17T04:30:54.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.385+0000 7f0ca3200640 1 --2- 172.21.15.100:0/1806302718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ca40af170 0x7f0ca40b2f60 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:54.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.385+0000 7f0ca3200640 1 -- 172.21.15.100:0/1806302718 >> 172.21.15.100:0/1806302718 conn(0x7f0ca40b1e40 msgr2=0x7f0ca40b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:54.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.385+0000 7f0ca3200640 1 -- 172.21.15.100:0/1806302718 shutdown_connections 2024-09-17T04:30:54.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.385+0000 7f0ca3200640 1 -- 172.21.15.100:0/1806302718 wait complete. 2024-09-17T04:30:54.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.387+0000 7f0ca3200640 1 Processor -- start 2024-09-17T04:30:54.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.387+0000 7f0ca3200640 1 -- start start 2024-09-17T04:30:54.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.388+0000 7f0ca3200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ca40ad9c0 0x7f0ca41552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:54.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.388+0000 7f0ca3200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ca40af170 0x7f0ca409dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:54.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.389+0000 7f0ca3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ca415a0f0 0x7f0ca41e8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:54.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.389+0000 7f0ca3200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca406fb20 con 0x7f0ca415a0f0 2024-09-17T04:30:54.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.389+0000 7f0ca3200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca406f9a0 con 0x7f0ca40af170 2024-09-17T04:30:54.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.389+0000 7f0ca3200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0ca415d670 con 0x7f0ca40ad9c0 2024-09-17T04:30:54.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.389+0000 7f0ca2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ca415a0f0 0x7f0ca41e8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:54.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.389+0000 7f0ca2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ca415a0f0 0x7f0ca41e8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36094/0 (socket says 172.21.15.100:36094) 2024-09-17T04:30:54.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.389+0000 7f0ca2800640 1 -- 172.21.15.100:0/3613189152 learned_addr learned my addr 172.21.15.100:0/3613189152 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:54.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.389+0000 7f0ca1e00640 1 --2- 172.21.15.100:0/3613189152 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ca40ad9c0 0x7f0ca41552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:54.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.389+0000 7f0ca1400640 1 --2- 172.21.15.100:0/3613189152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ca40af170 0x7f0ca409dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:54.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.389+0000 7f0ca2800640 1 -- 172.21.15.100:0/3613189152 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ca40ad9c0 msgr2=0x7f0ca41552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:54.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.389+0000 7f0ca2800640 1 --2- 172.21.15.100:0/3613189152 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ca40ad9c0 0x7f0ca41552a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:54.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.389+0000 7f0ca2800640 1 -- 172.21.15.100:0/3613189152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ca40af170 msgr2=0x7f0ca409dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:54.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.389+0000 7f0ca2800640 1 --2- 172.21.15.100:0/3613189152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ca40af170 0x7f0ca409dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:54.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.389+0000 7f0ca2800640 1 -- 172.21.15.100:0/3613189152 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca409aa50 con 0x7f0ca415a0f0 2024-09-17T04:30:54.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.390+0000 7f0ca2800640 1 --2- 172.21.15.100:0/3613189152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ca415a0f0 0x7f0ca41e8630 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c00c730 tx=0x7f0c9c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:54.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.390+0000 7f0c9a000640 1 -- 172.21.15.100:0/3613189152 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c003b70 con 0x7f0ca415a0f0 2024-09-17T04:30:54.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.390+0000 7f0c9a000640 1 -- 172.21.15.100:0/3613189152 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c9c003d10 con 0x7f0ca415a0f0 2024-09-17T04:30:54.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.390+0000 7f0c9a000640 1 -- 172.21.15.100:0/3613189152 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c00aab0 con 0x7f0ca415a0f0 2024-09-17T04:30:54.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.390+0000 7f0ca3200640 1 -- 172.21.15.100:0/3613189152 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ca409aec0 con 0x7f0ca415a0f0 2024-09-17T04:30:54.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.390+0000 7f0ca3200640 1 -- 172.21.15.100:0/3613189152 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ca409b800 con 0x7f0ca415a0f0 2024-09-17T04:30:54.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.391+0000 7f0ca3200640 1 -- 172.21.15.100:0/3613189152 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ca409ac20 con 0x7f0ca415a0f0 2024-09-17T04:30:54.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.392+0000 7f0c9a000640 1 -- 172.21.15.100:0/3613189152 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f0c9c00ac50 con 0x7f0ca415a0f0 2024-09-17T04:30:54.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.393+0000 7f0c9a000640 1 --2- 172.21.15.100:0/3613189152 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0c84079040 0x7f0c8407b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:54.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.393+0000 7f0c9a000640 1 -- 172.21.15.100:0/3613189152 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f0c9c09b1e0 con 0x7f0ca415a0f0 2024-09-17T04:30:54.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.394+0000 7f0ca1e00640 1 --2- 172.21.15.100:0/3613189152 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0c84079040 0x7f0c8407b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:54.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.394+0000 7f0ca1e00640 1 --2- 172.21.15.100:0/3613189152 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0c84079040 0x7f0c8407b550 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0c94000f60 tx=0x7f0c94009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:54.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.398+0000 7f0c9a000640 1 -- 172.21.15.100:0/3613189152 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0ca409ac20 con 0x7f0ca415a0f0 2024-09-17T04:30:54.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.601+0000 7f0ca3200640 1 -- 172.21.15.100:0/3613189152 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0ca40b2f60 con 0x7f0ca415a0f0 2024-09-17T04:30:54.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.602+0000 7f0c9a000640 1 -- 172.21.15.100:0/3613189152 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v107) ==== 80+0+0 (secure 0 0 0) 0x7f0c9c062940 con 0x7f0ca415a0f0 2024-09-17T04:30:54.603 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-09-17T04:30:54.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.604+0000 7f0ca3200640 1 -- 172.21.15.100:0/3613189152 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0c84079040 msgr2=0x7f0c8407b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:54.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.604+0000 7f0ca3200640 1 --2- 172.21.15.100:0/3613189152 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0c84079040 0x7f0c8407b550 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0c94000f60 tx=0x7f0c94009040 comp rx=0 tx=0).stop 2024-09-17T04:30:54.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.604+0000 7f0ca3200640 1 -- 172.21.15.100:0/3613189152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ca415a0f0 msgr2=0x7f0ca41e8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:54.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.604+0000 7f0ca3200640 1 --2- 172.21.15.100:0/3613189152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ca415a0f0 0x7f0ca41e8630 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c00c730 tx=0x7f0c9c00cbf0 comp rx=0 tx=0).stop 2024-09-17T04:30:54.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.604+0000 7f0ca3200640 1 -- 172.21.15.100:0/3613189152 shutdown_connections 2024-09-17T04:30:54.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.604+0000 7f0ca3200640 1 --2- 172.21.15.100:0/3613189152 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0c84079040 0x7f0c8407b550 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:54.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.604+0000 7f0ca3200640 1 --2- 172.21.15.100:0/3613189152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ca415a0f0 0x7f0ca41e8630 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:54.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.604+0000 7f0ca3200640 1 --2- 172.21.15.100:0/3613189152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ca40af170 0x7f0ca409dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:54.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.604+0000 7f0ca3200640 1 --2- 172.21.15.100:0/3613189152 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ca40ad9c0 0x7f0ca41552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:54.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.604+0000 7f0ca3200640 1 -- 172.21.15.100:0/3613189152 >> 172.21.15.100:0/3613189152 conn(0x7f0ca40b1e40 msgr2=0x7f0ca415a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:54.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.605+0000 7f0ca3200640 1 -- 172.21.15.100:0/3613189152 shutdown_connections 2024-09-17T04:30:54.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:54.605+0000 7f0ca3200640 1 -- 172.21.15.100:0/3613189152 wait complete. 2024-09-17T04:30:54.625 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T04:30:54.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:54.626 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T04:30:54.712 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-09-17T04:30:54.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:54.713 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T04:30:54.765 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:30:54.765 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:30:54.765 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:30:54.802 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:30:54.802 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:30:54.879 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:30:55.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.108+0000 7f07f1800640 1 -- 172.21.15.100:0/3361901493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07ec15a0f0 msgr2=0x7f07ec15d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:55.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.108+0000 7f07f1800640 1 --2- 172.21.15.100:0/3361901493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07ec15a0f0 0x7f07ec15d670 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f07e4009cd0 tx=0x7f07e402aa70 comp rx=0 tx=0).stop 2024-09-17T04:30:55.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.108+0000 7f07f1800640 1 -- 172.21.15.100:0/3361901493 shutdown_connections 2024-09-17T04:30:55.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.108+0000 7f07f1800640 1 --2- 172.21.15.100:0/3361901493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07ec15a0f0 0x7f07ec15d670 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:55.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.108+0000 7f07f1800640 1 --2- 172.21.15.100:0/3361901493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07ec0af170 0x7f07ec0b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:55.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.108+0000 7f07f1800640 1 -- 172.21.15.100:0/3361901493 >> 172.21.15.100:0/3361901493 conn(0x7f07ec0b1e40 msgr2=0x7f07ec0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:55.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.108+0000 7f07f1800640 1 -- 172.21.15.100:0/3361901493 shutdown_connections 2024-09-17T04:30:55.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.108+0000 7f07f1800640 1 -- 172.21.15.100:0/3361901493 wait complete. 2024-09-17T04:30:55.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.110+0000 7f07f1800640 1 Processor -- start 2024-09-17T04:30:55.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.111+0000 7f07f1800640 1 -- start start 2024-09-17T04:30:55.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.112+0000 7f07f1800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07ec0ad9c0 0x7f07ec1486b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:55.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.112+0000 7f07f1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07ec0af170 0x7f07ec157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:55.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.112+0000 7f07f1800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07ec15a0f0 0x7f07ec157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:55.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.112+0000 7f07f1800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec06f410 con 0x7f07ec0af170 2024-09-17T04:30:55.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.112+0000 7f07f1800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec06f290 con 0x7f07ec15a0f0 2024-09-17T04:30:55.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.112+0000 7f07f1800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f07ec06f590 con 0x7f07ec0ad9c0 2024-09-17T04:30:55.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.112+0000 7f07e3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07ec0af170 0x7f07ec157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:55.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.112+0000 7f07e3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07ec0af170 0x7f07ec157670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53550/0 (socket says 172.21.15.100:53550) 2024-09-17T04:30:55.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.112+0000 7f07e3e00640 1 -- 172.21.15.100:0/1158379050 learned_addr learned my addr 172.21.15.100:0/1158379050 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:55.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.114+0000 7f07e3e00640 1 -- 172.21.15.100:0/1158379050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07ec0ad9c0 msgr2=0x7f07ec1486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:55.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.114+0000 7f07f0e00640 1 --2- 172.21.15.100:0/1158379050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07ec15a0f0 0x7f07ec157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:55.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.114+0000 7f07ebe00640 1 --2- 172.21.15.100:0/1158379050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07ec0ad9c0 0x7f07ec1486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:55.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.114+0000 7f07e3e00640 1 --2- 172.21.15.100:0/1158379050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07ec0ad9c0 0x7f07ec1486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:55.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.114+0000 7f07e3e00640 1 -- 172.21.15.100:0/1158379050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07ec15a0f0 msgr2=0x7f07ec157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:55.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.114+0000 7f07e3e00640 1 --2- 172.21.15.100:0/1158379050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07ec15a0f0 0x7f07ec157bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:55.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.114+0000 7f07e3e00640 1 -- 172.21.15.100:0/1158379050 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07ec09dab0 con 0x7f07ec0af170 2024-09-17T04:30:55.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.114+0000 7f07f0e00640 1 --2- 172.21.15.100:0/1158379050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07ec15a0f0 0x7f07ec157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:30:55.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.114+0000 7f07ebe00640 1 --2- 172.21.15.100:0/1158379050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07ec0ad9c0 0x7f07ec1486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:30:55.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.114+0000 7f07e3e00640 1 --2- 172.21.15.100:0/1158379050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07ec0af170 0x7f07ec157670 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f07d4009fd0 tx=0x7f07d400ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:55.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.114+0000 7f07e8c00640 1 -- 172.21.15.100:0/1158379050 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d400ccc0 con 0x7f07ec0af170 2024-09-17T04:30:55.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.114+0000 7f07e8c00640 1 -- 172.21.15.100:0/1158379050 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07d400ee70 con 0x7f07ec0af170 2024-09-17T04:30:55.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.114+0000 7f07f1800640 1 -- 172.21.15.100:0/1158379050 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07ec148e20 con 0x7f07ec0af170 2024-09-17T04:30:55.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.114+0000 7f07e8c00640 1 -- 172.21.15.100:0/1158379050 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d400ce60 con 0x7f07ec0af170 2024-09-17T04:30:55.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.114+0000 7f07f1800640 1 -- 172.21.15.100:0/1158379050 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07ec1490c0 con 0x7f07ec0af170 2024-09-17T04:30:55.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.115+0000 7f07f1800640 1 -- 172.21.15.100:0/1158379050 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07ec09dd00 con 0x7f07ec0af170 2024-09-17T04:30:55.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.117+0000 7f07e8c00640 1 -- 172.21.15.100:0/1158379050 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f07d4005260 con 0x7f07ec0af170 2024-09-17T04:30:55.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.118+0000 7f07e8c00640 1 --2- 172.21.15.100:0/1158379050 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f07c8079010 0x7f07c807b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:55.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.118+0000 7f07e8c00640 1 -- 172.21.15.100:0/1158379050 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f07d409c010 con 0x7f07ec0af170 2024-09-17T04:30:55.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.118+0000 7f07ebe00640 1 --2- 172.21.15.100:0/1158379050 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f07c8079010 0x7f07c807b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:55.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.118+0000 7f07ebe00640 1 --2- 172.21.15.100:0/1158379050 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f07c8079010 0x7f07c807b520 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f07dc000f60 tx=0x7f07dc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:55.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.122+0000 7f07e8c00640 1 -- 172.21.15.100:0/1158379050 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f07ec09dab0 con 0x7f07ec0af170 2024-09-17T04:30:55.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.315+0000 7f07f1800640 1 -- 172.21.15.100:0/1158379050 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f07ec06fab0 con 0x7f07ec0af170 2024-09-17T04:30:55.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.316+0000 7f07e8c00640 1 -- 172.21.15.100:0/1158379050 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v107) ==== 80+0+0 (secure 0 0 0) 0x7f07d4063c90 con 0x7f07ec0af170 2024-09-17T04:30:55.315 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-09-17T04:30:55.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.318+0000 7f07f1800640 1 -- 172.21.15.100:0/1158379050 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f07c8079010 msgr2=0x7f07c807b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:55.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.318+0000 7f07f1800640 1 --2- 172.21.15.100:0/1158379050 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f07c8079010 0x7f07c807b520 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f07dc000f60 tx=0x7f07dc009040 comp rx=0 tx=0).stop 2024-09-17T04:30:55.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.318+0000 7f07f1800640 1 -- 172.21.15.100:0/1158379050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07ec0af170 msgr2=0x7f07ec157670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:55.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.318+0000 7f07f1800640 1 --2- 172.21.15.100:0/1158379050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07ec0af170 0x7f07ec157670 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f07d4009fd0 tx=0x7f07d400ec20 comp rx=0 tx=0).stop 2024-09-17T04:30:55.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.318+0000 7f07f1800640 1 -- 172.21.15.100:0/1158379050 shutdown_connections 2024-09-17T04:30:55.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.318+0000 7f07f1800640 1 --2- 172.21.15.100:0/1158379050 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f07c8079010 0x7f07c807b520 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:55.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.318+0000 7f07f1800640 1 --2- 172.21.15.100:0/1158379050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07ec15a0f0 0x7f07ec157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:55.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.318+0000 7f07f1800640 1 --2- 172.21.15.100:0/1158379050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07ec0af170 0x7f07ec157670 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:55.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.318+0000 7f07f1800640 1 --2- 172.21.15.100:0/1158379050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07ec0ad9c0 0x7f07ec1486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:55.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.318+0000 7f07f1800640 1 -- 172.21.15.100:0/1158379050 >> 172.21.15.100:0/1158379050 conn(0x7f07ec0b1e40 msgr2=0x7f07ec0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:55.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.318+0000 7f07f1800640 1 -- 172.21.15.100:0/1158379050 shutdown_connections 2024-09-17T04:30:55.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.318+0000 7f07f1800640 1 -- 172.21.15.100:0/1158379050 wait complete. 2024-09-17T04:30:55.338 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-17T04:30:55.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:55.339 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-17T04:30:55.403 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2024-09-17T04:30:55.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:30:55.403 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T04:30:55.440 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:30:55.440 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:30:55.441 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:30:55.476 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:30:55.477 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:30:55.477 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:30:55.477 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:30:55.516 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:30:55.516 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:30:55.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.742+0000 7f65d1600640 1 -- 172.21.15.100:0/3843696701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65cc15d760 msgr2=0x7f65cc1535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:55.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.742+0000 7f65d1600640 1 --2- 172.21.15.100:0/3843696701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65cc15d760 0x7f65cc1535c0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f65cc072d60 tx=0x7f65bc02a4d0 comp rx=0 tx=0).stop 2024-09-17T04:30:55.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.742+0000 7f65d1600640 1 -- 172.21.15.100:0/3843696701 shutdown_connections 2024-09-17T04:30:55.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.742+0000 7f65d1600640 1 --2- 172.21.15.100:0/3843696701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65cc160310 0x7f65cc160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:55.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.742+0000 7f65d1600640 1 --2- 172.21.15.100:0/3843696701 >> v2:172.21.15.184:3300/0 conn(0x7f65cc0ad0e0 0x7f65cc0ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:55.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.742+0000 7f65d1600640 1 --2- 172.21.15.100:0/3843696701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65cc15d760 0x7f65cc1535c0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:55.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.742+0000 7f65d1600640 1 -- 172.21.15.100:0/3843696701 >> 172.21.15.100:0/3843696701 conn(0x7f65cc06d730 msgr2=0x7f65cc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:55.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.742+0000 7f65d1600640 1 -- 172.21.15.100:0/3843696701 shutdown_connections 2024-09-17T04:30:55.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.742+0000 7f65d1600640 1 -- 172.21.15.100:0/3843696701 wait complete. 2024-09-17T04:30:55.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.744+0000 7f65d1600640 1 Processor -- start 2024-09-17T04:30:55.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.744+0000 7f65d1600640 1 -- start start 2024-09-17T04:30:55.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.746+0000 7f65d1600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65cc0ad0e0 0x7f65cc09a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:55.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.746+0000 7f65d1600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65cc15d760 0x7f65cc148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:55.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.746+0000 7f65d1600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65cc160310 0x7f65cc15b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:55.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.746+0000 7f65d1600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc0b4da0 con 0x7f65cc0ad0e0 2024-09-17T04:30:55.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.746+0000 7f65d1600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc0b4c20 con 0x7f65cc160310 2024-09-17T04:30:55.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.746+0000 7f65d1600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f65cc160c60 con 0x7f65cc15d760 2024-09-17T04:30:55.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.746+0000 7f65caa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65cc0ad0e0 0x7f65cc09a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:55.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.746+0000 7f65caa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65cc0ad0e0 0x7f65cc09a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53562/0 (socket says 172.21.15.100:53562) 2024-09-17T04:30:55.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.746+0000 7f65caa00640 1 -- 172.21.15.100:0/3962131952 learned_addr learned my addr 172.21.15.100:0/3962131952 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:55.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.746+0000 7f65ca000640 1 --2- 172.21.15.100:0/3962131952 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65cc15d760 0x7f65cc148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:55.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.746+0000 7f65cb400640 1 --2- 172.21.15.100:0/3962131952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65cc160310 0x7f65cc15b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:55.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.746+0000 7f65caa00640 1 -- 172.21.15.100:0/3962131952 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65cc15d760 msgr2=0x7f65cc148710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:55.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.746+0000 7f65caa00640 1 --2- 172.21.15.100:0/3962131952 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65cc15d760 0x7f65cc148710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:55.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.746+0000 7f65caa00640 1 -- 172.21.15.100:0/3962131952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65cc160310 msgr2=0x7f65cc15b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:55.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.747+0000 7f65caa00640 1 --2- 172.21.15.100:0/3962131952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65cc160310 0x7f65cc15b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:55.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.747+0000 7f65caa00640 1 -- 172.21.15.100:0/3962131952 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65cc09a810 con 0x7f65cc0ad0e0 2024-09-17T04:30:55.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.747+0000 7f65caa00640 1 --2- 172.21.15.100:0/3962131952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65cc0ad0e0 0x7f65cc09a2d0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f65bc000c00 tx=0x7f65bc0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:55.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.747+0000 7f65c2a00640 1 -- 172.21.15.100:0/3962131952 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc03e030 con 0x7f65cc0ad0e0 2024-09-17T04:30:55.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.747+0000 7f65c2a00640 1 -- 172.21.15.100:0/3962131952 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65bc003ca0 con 0x7f65cc0ad0e0 2024-09-17T04:30:55.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.747+0000 7f65d1600640 1 -- 172.21.15.100:0/3962131952 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65cc1f9a50 con 0x7f65cc0ad0e0 2024-09-17T04:30:55.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.747+0000 7f65c2a00640 1 -- 172.21.15.100:0/3962131952 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc035040 con 0x7f65cc0ad0e0 2024-09-17T04:30:55.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.747+0000 7f65d1600640 1 -- 172.21.15.100:0/3962131952 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65cc156b90 con 0x7f65cc0ad0e0 2024-09-17T04:30:55.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.748+0000 7f65d1600640 1 -- 172.21.15.100:0/3962131952 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65cc074b80 con 0x7f65cc0ad0e0 2024-09-17T04:30:55.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.750+0000 7f65c2a00640 1 -- 172.21.15.100:0/3962131952 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f65bc041030 con 0x7f65cc0ad0e0 2024-09-17T04:30:55.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.751+0000 7f65c2a00640 1 --2- 172.21.15.100:0/3962131952 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f65ac079010 0x7f65ac07b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:55.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.751+0000 7f65c2a00640 1 -- 172.21.15.100:0/3962131952 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f65bc0bbc90 con 0x7f65cc0ad0e0 2024-09-17T04:30:55.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.751+0000 7f65ca000640 1 --2- 172.21.15.100:0/3962131952 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f65ac079010 0x7f65ac07b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:55.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.752+0000 7f65ca000640 1 --2- 172.21.15.100:0/3962131952 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f65ac079010 0x7f65ac07b520 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f65cc0b4660 tx=0x7f65b8005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:55.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.755+0000 7f65c2a00640 1 -- 172.21.15.100:0/3962131952 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f65bc083300 con 0x7f65cc0ad0e0 2024-09-17T04:30:55.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.983+0000 7f65d1600640 1 -- 172.21.15.100:0/3962131952 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65cc15e5d0 con 0x7f65cc0ad0e0 2024-09-17T04:30:55.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.984+0000 7f65c2a00640 1 -- 172.21.15.100:0/3962131952 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5234 (secure 0 0 0) 0x7f65bc087d30 con 0x7f65cc0ad0e0 2024-09-17T04:30:55.983 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:30:55.983 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":73,"btime":"2024-09-17T04:30:10:474753+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6044,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1891177834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1891177834},{"type":"v1","addr":"172.21.15.100:6835","nonce":1891177834}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":6047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1158083377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1158083377},{"type":"v1","addr":"172.21.15.184:6835","nonce":1158083377}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":6697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/714465116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":714465116},{"type":"v1","addr":"172.21.15.184:6801","nonce":714465116}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:29:59.487577+0000","modified":"2024-09-17T04:30:02.508652+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6972":{"gid":6972,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6833/2666467995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2666467995},{"type":"v1","addr":"172.21.15.100:6833","nonce":2666467995}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6972,"qdb_cluster":[6972]},"id":8}]} 2024-09-17T04:30:55.984 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 73 2024-09-17T04:30:55.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.986+0000 7f65d1600640 1 -- 172.21.15.100:0/3962131952 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f65ac079010 msgr2=0x7f65ac07b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:55.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.986+0000 7f65d1600640 1 --2- 172.21.15.100:0/3962131952 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f65ac079010 0x7f65ac07b520 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f65cc0b4660 tx=0x7f65b8005f90 comp rx=0 tx=0).stop 2024-09-17T04:30:55.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.986+0000 7f65d1600640 1 -- 172.21.15.100:0/3962131952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65cc0ad0e0 msgr2=0x7f65cc09a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:55.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.986+0000 7f65d1600640 1 --2- 172.21.15.100:0/3962131952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65cc0ad0e0 0x7f65cc09a2d0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f65bc000c00 tx=0x7f65bc0038a0 comp rx=0 tx=0).stop 2024-09-17T04:30:55.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.986+0000 7f65d1600640 1 -- 172.21.15.100:0/3962131952 shutdown_connections 2024-09-17T04:30:55.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.986+0000 7f65d1600640 1 --2- 172.21.15.100:0/3962131952 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f65ac079010 0x7f65ac07b520 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:55.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.986+0000 7f65d1600640 1 --2- 172.21.15.100:0/3962131952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65cc160310 0x7f65cc15b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:55.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.986+0000 7f65d1600640 1 --2- 172.21.15.100:0/3962131952 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65cc15d760 0x7f65cc148710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:55.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.986+0000 7f65d1600640 1 --2- 172.21.15.100:0/3962131952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65cc0ad0e0 0x7f65cc09a2d0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:55.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.986+0000 7f65d1600640 1 -- 172.21.15.100:0/3962131952 >> 172.21.15.100:0/3962131952 conn(0x7f65cc06d730 msgr2=0x7f65cc15ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:55.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.986+0000 7f65d1600640 1 -- 172.21.15.100:0/3962131952 shutdown_connections 2024-09-17T04:30:55.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:55.986+0000 7f65d1600640 1 -- 172.21.15.100:0/3962131952 wait complete. 2024-09-17T04:30:56.007 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:30:56.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.241+0000 7f651c000640 1 -- 172.21.15.100:0/2455935462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65140ad0e0 msgr2=0x7f65140ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:56.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.241+0000 7f651c000640 1 --2- 172.21.15.100:0/2455935462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65140ad0e0 0x7f65140ad4f0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f6500005fd0 tx=0x7f650002a540 comp rx=0 tx=0).stop 2024-09-17T04:30:56.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.241+0000 7f651c000640 1 -- 172.21.15.100:0/2455935462 shutdown_connections 2024-09-17T04:30:56.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.241+0000 7f651c000640 1 --2- 172.21.15.100:0/2455935462 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65140b3550 0x7f65140b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:56.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.241+0000 7f651c000640 1 --2- 172.21.15.100:0/2455935462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65140ad0e0 0x7f65140ad4f0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:56.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.241+0000 7f651c000640 1 -- 172.21.15.100:0/2455935462 >> 172.21.15.100:0/2455935462 conn(0x7f651406d730 msgr2=0x7f65140b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:56.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.241+0000 7f651c000640 1 -- 172.21.15.100:0/2455935462 shutdown_connections 2024-09-17T04:30:56.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.241+0000 7f651c000640 1 -- 172.21.15.100:0/2455935462 wait complete. 2024-09-17T04:30:56.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.243+0000 7f651c000640 1 Processor -- start 2024-09-17T04:30:56.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.243+0000 7f651c000640 1 -- start start 2024-09-17T04:30:56.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.245+0000 7f651c000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65140ad0e0 0x7f6514156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:56.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.245+0000 7f651c000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65140b3550 0x7f65141f5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:56.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.245+0000 7f651c000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f651415d760 0x7f651415b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:56.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.245+0000 7f651c000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f65140b4da0 con 0x7f65140ad0e0 2024-09-17T04:30:56.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.245+0000 7f651c000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f65140b4c20 con 0x7f651415d760 2024-09-17T04:30:56.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.245+0000 7f651c000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f65140b3ea0 con 0x7f65140b3550 2024-09-17T04:30:56.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.245+0000 7f6519600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65140ad0e0 0x7f6514156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:56.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.245+0000 7f6519600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65140ad0e0 0x7f6514156700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53584/0 (socket says 172.21.15.100:53584) 2024-09-17T04:30:56.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.245+0000 7f6519600640 1 -- 172.21.15.100:0/1619240882 learned_addr learned my addr 172.21.15.100:0/1619240882 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:56.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.245+0000 7f651a000640 1 --2- 172.21.15.100:0/1619240882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f651415d760 0x7f651415b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:56.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.245+0000 7f6518c00640 1 --2- 172.21.15.100:0/1619240882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65140b3550 0x7f65141f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:56.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.245+0000 7f6519600640 1 -- 172.21.15.100:0/1619240882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65140b3550 msgr2=0x7f65141f5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:56.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.245+0000 7f6519600640 1 --2- 172.21.15.100:0/1619240882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65140b3550 0x7f65141f5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:56.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.245+0000 7f6519600640 1 -- 172.21.15.100:0/1619240882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f651415d760 msgr2=0x7f651415b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:56.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.245+0000 7f6519600640 1 --2- 172.21.15.100:0/1619240882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f651415d760 0x7f651415b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:56.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.245+0000 7f6519600640 1 -- 172.21.15.100:0/1619240882 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f651414f4b0 con 0x7f65140ad0e0 2024-09-17T04:30:56.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.246+0000 7f6519600640 1 --2- 172.21.15.100:0/1619240882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65140ad0e0 0x7f6514156700 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f650800c730 tx=0x7f650800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:56.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.246+0000 7f6511600640 1 -- 172.21.15.100:0/1619240882 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6508003b70 con 0x7f65140ad0e0 2024-09-17T04:30:56.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.246+0000 7f6511600640 1 -- 172.21.15.100:0/1619240882 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6508003d30 con 0x7f65140ad0e0 2024-09-17T04:30:56.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.246+0000 7f651c000640 1 -- 172.21.15.100:0/1619240882 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f651414f790 con 0x7f65140ad0e0 2024-09-17T04:30:56.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.246+0000 7f6511600640 1 -- 172.21.15.100:0/1619240882 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650800aa50 con 0x7f65140ad0e0 2024-09-17T04:30:56.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.246+0000 7f651c000640 1 -- 172.21.15.100:0/1619240882 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f651414fd50 con 0x7f65140ad0e0 2024-09-17T04:30:56.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.247+0000 7f651c000640 1 -- 172.21.15.100:0/1619240882 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65140b4370 con 0x7f65140ad0e0 2024-09-17T04:30:56.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.249+0000 7f6511600640 1 -- 172.21.15.100:0/1619240882 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f650800abf0 con 0x7f65140ad0e0 2024-09-17T04:30:56.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.249+0000 7f6511600640 1 --2- 172.21.15.100:0/1619240882 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f64f4079040 0x7f64f407b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:56.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.250+0000 7f6511600640 1 -- 172.21.15.100:0/1619240882 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5823+0+0 (secure 0 0 0) 0x7f650809b1b0 con 0x7f65140ad0e0 2024-09-17T04:30:56.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.250+0000 7f6518c00640 1 --2- 172.21.15.100:0/1619240882 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f64f4079040 0x7f64f407b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:56.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.251+0000 7f6518c00640 1 --2- 172.21.15.100:0/1619240882 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f64f4079040 0x7f64f407b550 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6500033000 tx=0x7f6500004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:56.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.254+0000 7f6511600640 1 -- 172.21.15.100:0/1619240882 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f65140ada30 con 0x7f65140ad0e0 2024-09-17T04:30:56.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.472+0000 7f651c000640 1 -- 172.21.15.100:0/1619240882 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6514069700 con 0x7f65140ad0e0 2024-09-17T04:30:56.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.472+0000 7f6511600640 1 -- 172.21.15.100:0/1619240882 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5234 (secure 0 0 0) 0x7f65080628e0 con 0x7f65140ad0e0 2024-09-17T04:30:56.472 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:30:56.472 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":73,"btime":"2024-09-17T04:30:10:474753+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6044,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1891177834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1891177834},{"type":"v1","addr":"172.21.15.100:6835","nonce":1891177834}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":6047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1158083377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1158083377},{"type":"v1","addr":"172.21.15.184:6835","nonce":1158083377}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":6697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/714465116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":714465116},{"type":"v1","addr":"172.21.15.184:6801","nonce":714465116}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:29:59.487577+0000","modified":"2024-09-17T04:30:02.508652+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6972":{"gid":6972,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6833/2666467995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2666467995},{"type":"v1","addr":"172.21.15.100:6833","nonce":2666467995}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6972,"qdb_cluster":[6972]},"id":8}]} 2024-09-17T04:30:56.472 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 73 2024-09-17T04:30:56.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.475+0000 7f651c000640 1 -- 172.21.15.100:0/1619240882 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f64f4079040 msgr2=0x7f64f407b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:56.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.475+0000 7f651c000640 1 --2- 172.21.15.100:0/1619240882 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f64f4079040 0x7f64f407b550 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6500033000 tx=0x7f6500004730 comp rx=0 tx=0).stop 2024-09-17T04:30:56.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.475+0000 7f651c000640 1 -- 172.21.15.100:0/1619240882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65140ad0e0 msgr2=0x7f6514156700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:56.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.475+0000 7f651c000640 1 --2- 172.21.15.100:0/1619240882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65140ad0e0 0x7f6514156700 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f650800c730 tx=0x7f650800cbf0 comp rx=0 tx=0).stop 2024-09-17T04:30:56.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.475+0000 7f651c000640 1 -- 172.21.15.100:0/1619240882 shutdown_connections 2024-09-17T04:30:56.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.475+0000 7f651c000640 1 --2- 172.21.15.100:0/1619240882 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f64f4079040 0x7f64f407b550 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:56.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.475+0000 7f651c000640 1 --2- 172.21.15.100:0/1619240882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f651415d760 0x7f651415b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:56.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.475+0000 7f651c000640 1 --2- 172.21.15.100:0/1619240882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65140b3550 0x7f65141f5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:56.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.475+0000 7f651c000640 1 --2- 172.21.15.100:0/1619240882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65140ad0e0 0x7f6514156700 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:56.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.475+0000 7f651c000640 1 -- 172.21.15.100:0/1619240882 >> 172.21.15.100:0/1619240882 conn(0x7f651406d730 msgr2=0x7f651415dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:56.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.475+0000 7f651c000640 1 -- 172.21.15.100:0/1619240882 shutdown_connections 2024-09-17T04:30:56.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.475+0000 7f651c000640 1 -- 172.21.15.100:0/1619240882 wait complete. 2024-09-17T04:30:56.497 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:30:56.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.721+0000 7f5d7ec00640 1 -- 172.21.15.100:0/1280666437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d5c005350 msgr2=0x7f5d5c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:56.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.721+0000 7f5d7ec00640 1 --2- 172.21.15.100:0/1280666437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d5c005350 0x7f5d5c00b9c0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f5d70009b20 tx=0x7f5d5c005830 comp rx=0 tx=0).stop 2024-09-17T04:30:56.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.721+0000 7f5d7ec00640 1 -- 172.21.15.100:0/1280666437 shutdown_connections 2024-09-17T04:30:56.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.721+0000 7f5d7ec00640 1 --2- 172.21.15.100:0/1280666437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d5c005350 0x7f5d5c00b9c0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:56.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.721+0000 7f5d7ec00640 1 -- 172.21.15.100:0/1280666437 >> 172.21.15.100:0/1280666437 conn(0x7f5d7806d730 msgr2=0x7f5d780b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:56.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.721+0000 7f5d7ec00640 1 -- 172.21.15.100:0/1280666437 shutdown_connections 2024-09-17T04:30:56.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.721+0000 7f5d7ec00640 1 -- 172.21.15.100:0/1280666437 wait complete. 2024-09-17T04:30:56.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.724+0000 7f5d7ec00640 1 Processor -- start 2024-09-17T04:30:56.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.724+0000 7f5d7ec00640 1 -- start start 2024-09-17T04:30:56.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.725+0000 7f5d7ec00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d5c005350 0x7f5d78073a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:56.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.725+0000 7f5d7ec00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d7814cf50 0x7f5d7814c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:56.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.725+0000 7f5d7ec00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d7814d4a0 0x7f5d781541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:56.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.725+0000 7f5d7ec00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5d780ad4f0 con 0x7f5d7814d4a0 2024-09-17T04:30:56.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.725+0000 7f5d7ec00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5d780b4e30 con 0x7f5d5c005350 2024-09-17T04:30:56.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.725+0000 7f5d7ec00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5d780ad670 con 0x7f5d7814cf50 2024-09-17T04:30:56.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.725+0000 7f5d7cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d7814d4a0 0x7f5d781541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.725+0000 7f5d7cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d7814d4a0 0x7f5d781541c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53594/0 (socket says 172.21.15.100:53594) 2024-09-17T04:30:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.725+0000 7f5d7cc00640 1 -- 172.21.15.100:0/4290983508 learned_addr learned my addr 172.21.15.100:0/4290983508 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.726+0000 7f5d77e00640 1 --2- 172.21.15.100:0/4290983508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d5c005350 0x7f5d78073a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.726+0000 7f5d7cc00640 1 -- 172.21.15.100:0/4290983508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d7814cf50 msgr2=0x7f5d7814c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.726+0000 7f5d7cc00640 1 --2- 172.21.15.100:0/4290983508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d7814cf50 0x7f5d7814c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.726+0000 7f5d7cc00640 1 -- 172.21.15.100:0/4290983508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d5c005350 msgr2=0x7f5d78073a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.726+0000 7f5d7cc00640 1 --2- 172.21.15.100:0/4290983508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d5c005350 0x7f5d78073a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.726+0000 7f5d7cc00640 1 -- 172.21.15.100:0/4290983508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d781f9a40 con 0x7f5d7814d4a0 2024-09-17T04:30:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.726+0000 7f5d7cc00640 1 --2- 172.21.15.100:0/4290983508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d7814d4a0 0x7f5d781541c0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f5d70009c40 tx=0x7f5d700097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.726+0000 7f5d6fe00640 1 -- 172.21.15.100:0/4290983508 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d70041070 con 0x7f5d7814d4a0 2024-09-17T04:30:56.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.726+0000 7f5d6fe00640 1 -- 172.21.15.100:0/4290983508 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d70033650 con 0x7f5d7814d4a0 2024-09-17T04:30:56.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.726+0000 7f5d7ec00640 1 -- 172.21.15.100:0/4290983508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d78153e20 con 0x7f5d7814d4a0 2024-09-17T04:30:56.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.726+0000 7f5d6fe00640 1 -- 172.21.15.100:0/4290983508 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d70033ac0 con 0x7f5d7814d4a0 2024-09-17T04:30:56.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.726+0000 7f5d7ec00640 1 -- 172.21.15.100:0/4290983508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d78073f70 con 0x7f5d7814d4a0 2024-09-17T04:30:56.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.727+0000 7f5d7ec00640 1 -- 172.21.15.100:0/4290983508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d781535c0 con 0x7f5d7814d4a0 2024-09-17T04:30:56.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.729+0000 7f5d6fe00640 1 -- 172.21.15.100:0/4290983508 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f5d70044070 con 0x7f5d7814d4a0 2024-09-17T04:30:56.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.730+0000 7f5d6fe00640 1 --2- 172.21.15.100:0/4290983508 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5d58079040 0x7f5d5807b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:56.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.730+0000 7f5d6fe00640 1 -- 172.21.15.100:0/4290983508 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5823+0+0 (secure 0 0 0) 0x7f5d700bb8c0 con 0x7f5d7814d4a0 2024-09-17T04:30:56.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.730+0000 7f5d77e00640 1 --2- 172.21.15.100:0/4290983508 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5d58079040 0x7f5d5807b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:56.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.730+0000 7f5d77e00640 1 --2- 172.21.15.100:0/4290983508 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5d58079040 0x7f5d5807b550 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5d68001770 tx=0x7f5d680067a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:56.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.734+0000 7f5d6fe00640 1 -- 172.21.15.100:0/4290983508 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f5d700835a0 con 0x7f5d7814d4a0 2024-09-17T04:30:56.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.920+0000 7f5d7ec00640 1 -- 172.21.15.100:0/4290983508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d7806ffb0 con 0x7f5d7814d4a0 2024-09-17T04:30:56.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.921+0000 7f5d6fe00640 1 -- 172.21.15.100:0/4290983508 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+16999 (secure 0 0 0) 0x7f5d70082ea0 con 0x7f5d7814d4a0 2024-09-17T04:30:56.921 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:30:56.921 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":108,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:30:55.813015+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:29:57.388120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-17T04:29:58.449229+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.9","mappings":[{"from":0,"to":5}]},{"pgid":"17.37","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:30:56.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.924+0000 7f5d7ec00640 1 -- 172.21.15.100:0/4290983508 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5d58079040 msgr2=0x7f5d5807b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:56.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.924+0000 7f5d7ec00640 1 --2- 172.21.15.100:0/4290983508 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5d58079040 0x7f5d5807b550 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5d68001770 tx=0x7f5d680067a0 comp rx=0 tx=0).stop 2024-09-17T04:30:56.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.924+0000 7f5d7ec00640 1 -- 172.21.15.100:0/4290983508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d7814d4a0 msgr2=0x7f5d781541c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:56.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.924+0000 7f5d7ec00640 1 --2- 172.21.15.100:0/4290983508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d7814d4a0 0x7f5d781541c0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f5d70009c40 tx=0x7f5d700097a0 comp rx=0 tx=0).stop 2024-09-17T04:30:56.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.924+0000 7f5d7ec00640 1 -- 172.21.15.100:0/4290983508 shutdown_connections 2024-09-17T04:30:56.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.924+0000 7f5d7ec00640 1 --2- 172.21.15.100:0/4290983508 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5d58079040 0x7f5d5807b550 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:56.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.924+0000 7f5d7ec00640 1 --2- 172.21.15.100:0/4290983508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d7814d4a0 0x7f5d781541c0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:56.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.924+0000 7f5d7ec00640 1 --2- 172.21.15.100:0/4290983508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d7814cf50 0x7f5d7814c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:56.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.924+0000 7f5d7ec00640 1 --2- 172.21.15.100:0/4290983508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d5c005350 0x7f5d78073a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:56.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.924+0000 7f5d7ec00640 1 -- 172.21.15.100:0/4290983508 >> 172.21.15.100:0/4290983508 conn(0x7f5d7806d730 msgr2=0x7f5d78070290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:56.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.924+0000 7f5d7ec00640 1 -- 172.21.15.100:0/4290983508 shutdown_connections 2024-09-17T04:30:56.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:56.924+0000 7f5d7ec00640 1 -- 172.21.15.100:0/4290983508 wait complete. 2024-09-17T04:30:56.945 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T04:30:56.945 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:30:57.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.165+0000 7fe684800640 1 -- 172.21.15.100:0/73741681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe67c15fae0 msgr2=0x7fe67c0b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:57.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.165+0000 7fe684800640 1 --2- 172.21.15.100:0/73741681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe67c15fae0 0x7fe67c0b3750 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe678005fd0 tx=0x7fe67802aa10 comp rx=0 tx=0).stop 2024-09-17T04:30:57.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.165+0000 7fe684800640 1 -- 172.21.15.100:0/73741681 shutdown_connections 2024-09-17T04:30:57.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.165+0000 7fe684800640 1 --2- 172.21.15.100:0/73741681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe67c15fae0 0x7fe67c0b3750 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:57.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.165+0000 7fe684800640 1 --2- 172.21.15.100:0/73741681 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe67c0aebc0 0x7fe67c152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:57.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.165+0000 7fe684800640 1 -- 172.21.15.100:0/73741681 >> 172.21.15.100:0/73741681 conn(0x7fe67c06d730 msgr2=0x7fe67c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:57.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.165+0000 7fe684800640 1 -- 172.21.15.100:0/73741681 shutdown_connections 2024-09-17T04:30:57.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.165+0000 7fe684800640 1 -- 172.21.15.100:0/73741681 wait complete. 2024-09-17T04:30:57.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.167+0000 7fe684800640 1 Processor -- start 2024-09-17T04:30:57.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.167+0000 7fe684800640 1 -- start start 2024-09-17T04:30:57.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.169+0000 7fe684800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe67c0aebc0 0x7fe67c074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:57.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.169+0000 7fe684800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe67c0b2750 0x7fe67c1f9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:57.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.169+0000 7fe684800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe67c151620 0x7fe67c0b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:57.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.169+0000 7fe684800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c0b4210 con 0x7fe67c0aebc0 2024-09-17T04:30:57.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.169+0000 7fe684800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c0b4090 con 0x7fe67c0b2750 2024-09-17T04:30:57.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.169+0000 7fe684800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe67c0b4390 con 0x7fe67c151620 2024-09-17T04:30:57.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.169+0000 7fe681e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe67c0aebc0 0x7fe67c074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:57.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.169+0000 7fe681e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe67c0aebc0 0x7fe67c074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53602/0 (socket says 172.21.15.100:53602) 2024-09-17T04:30:57.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.169+0000 7fe681e00640 1 -- 172.21.15.100:0/209543794 learned_addr learned my addr 172.21.15.100:0/209543794 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:57.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.169+0000 7fe682800640 1 --2- 172.21.15.100:0/209543794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe67c151620 0x7fe67c0b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:57.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.169+0000 7fe681400640 1 --2- 172.21.15.100:0/209543794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe67c0b2750 0x7fe67c1f9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:57.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.169+0000 7fe681e00640 1 -- 172.21.15.100:0/209543794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe67c151620 msgr2=0x7fe67c0b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:57.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.169+0000 7fe681e00640 1 --2- 172.21.15.100:0/209543794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe67c151620 0x7fe67c0b0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:57.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.169+0000 7fe681e00640 1 -- 172.21.15.100:0/209543794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe67c0b2750 msgr2=0x7fe67c1f9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:57.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.169+0000 7fe681e00640 1 --2- 172.21.15.100:0/209543794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe67c0b2750 0x7fe67c1f9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:57.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.169+0000 7fe681e00640 1 -- 172.21.15.100:0/209543794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe67c0743d0 con 0x7fe67c0aebc0 2024-09-17T04:30:57.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.169+0000 7fe682800640 1 --2- 172.21.15.100:0/209543794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe67c151620 0x7fe67c0b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:30:57.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.170+0000 7fe681e00640 1 --2- 172.21.15.100:0/209543794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe67c0aebc0 0x7fe67c074c60 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fe67000e9f0 tx=0x7fe67000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:57.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.170+0000 7fe676000640 1 -- 172.21.15.100:0/209543794 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67000cd80 con 0x7fe67c0aebc0 2024-09-17T04:30:57.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.170+0000 7fe676000640 1 -- 172.21.15.100:0/209543794 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe670004020 con 0x7fe67c0aebc0 2024-09-17T04:30:57.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.170+0000 7fe684800640 1 -- 172.21.15.100:0/209543794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe67c0746b0 con 0x7fe67c0aebc0 2024-09-17T04:30:57.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.170+0000 7fe676000640 1 -- 172.21.15.100:0/209543794 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe670004470 con 0x7fe67c0aebc0 2024-09-17T04:30:57.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.170+0000 7fe684800640 1 -- 172.21.15.100:0/209543794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe67c1f8bf0 con 0x7fe67c0aebc0 2024-09-17T04:30:57.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.171+0000 7fe684800640 1 -- 172.21.15.100:0/209543794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe67c0b48b0 con 0x7fe67c0aebc0 2024-09-17T04:30:57.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.173+0000 7fe676000640 1 -- 172.21.15.100:0/209543794 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fe6700035f0 con 0x7fe67c0aebc0 2024-09-17T04:30:57.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.174+0000 7fe676000640 1 --2- 172.21.15.100:0/209543794 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe660079010 0x7fe66007b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:57.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.174+0000 7fe676000640 1 -- 172.21.15.100:0/209543794 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5823+0+0 (secure 0 0 0) 0x7fe67009c220 con 0x7fe67c0aebc0 2024-09-17T04:30:57.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.174+0000 7fe681400640 1 --2- 172.21.15.100:0/209543794 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe660079010 0x7fe66007b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:57.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.175+0000 7fe681400640 1 --2- 172.21.15.100:0/209543794 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe660079010 0x7fe66007b520 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe67c0cf8e0 tx=0x7fe66c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:57.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.179+0000 7fe676000640 1 -- 172.21.15.100:0/209543794 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe670063e80 con 0x7fe67c0aebc0 2024-09-17T04:30:57.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.398+0000 7fe684800640 1 -- 172.21.15.100:0/209543794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe67c0ae690 con 0x7fe67c0aebc0 2024-09-17T04:30:57.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.399+0000 7fe676000640 1 -- 172.21.15.100:0/209543794 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5234 (secure 0 0 0) 0x7fe670063780 con 0x7fe67c0aebc0 2024-09-17T04:30:57.398 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:30:57.399 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":73,"btime":"2024-09-17T04:30:10:474753+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6044,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1891177834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1891177834},{"type":"v1","addr":"172.21.15.100:6835","nonce":1891177834}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":6047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1158083377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1158083377},{"type":"v1","addr":"172.21.15.184:6835","nonce":1158083377}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":6697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/714465116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":714465116},{"type":"v1","addr":"172.21.15.184:6801","nonce":714465116}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:29:59.487577+0000","modified":"2024-09-17T04:30:02.508652+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6972":{"gid":6972,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6833/2666467995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2666467995},{"type":"v1","addr":"172.21.15.100:6833","nonce":2666467995}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6972,"qdb_cluster":[6972]},"id":8}]} 2024-09-17T04:30:57.399 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 73 2024-09-17T04:30:57.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.401+0000 7fe684800640 1 -- 172.21.15.100:0/209543794 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe660079010 msgr2=0x7fe66007b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:57.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.401+0000 7fe684800640 1 --2- 172.21.15.100:0/209543794 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe660079010 0x7fe66007b520 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe67c0cf8e0 tx=0x7fe66c009040 comp rx=0 tx=0).stop 2024-09-17T04:30:57.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.401+0000 7fe684800640 1 -- 172.21.15.100:0/209543794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe67c0aebc0 msgr2=0x7fe67c074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:57.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.401+0000 7fe684800640 1 --2- 172.21.15.100:0/209543794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe67c0aebc0 0x7fe67c074c60 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fe67000e9f0 tx=0x7fe67000eeb0 comp rx=0 tx=0).stop 2024-09-17T04:30:57.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.401+0000 7fe684800640 1 -- 172.21.15.100:0/209543794 shutdown_connections 2024-09-17T04:30:57.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.401+0000 7fe684800640 1 --2- 172.21.15.100:0/209543794 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe660079010 0x7fe66007b520 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:57.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.401+0000 7fe684800640 1 --2- 172.21.15.100:0/209543794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe67c151620 0x7fe67c0b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:57.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.401+0000 7fe684800640 1 --2- 172.21.15.100:0/209543794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe67c0b2750 0x7fe67c1f9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:57.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.401+0000 7fe684800640 1 --2- 172.21.15.100:0/209543794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe67c0aebc0 0x7fe67c074c60 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:57.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.401+0000 7fe684800640 1 -- 172.21.15.100:0/209543794 >> 172.21.15.100:0/209543794 conn(0x7fe67c06d730 msgr2=0x7fe67c15e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:57.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.402+0000 7fe684800640 1 -- 172.21.15.100:0/209543794 shutdown_connections 2024-09-17T04:30:57.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.402+0000 7fe684800640 1 -- 172.21.15.100:0/209543794 wait complete. 2024-09-17T04:30:57.424 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:30:57.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.643+0000 7fb3a0800640 1 -- 172.21.15.100:0/774494953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb39815d760 msgr2=0x7fb3981535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:57.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.643+0000 7fb3a0800640 1 --2- 172.21.15.100:0/774494953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb39815d760 0x7fb3981535c0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fb38c005fd0 tx=0x7fb38c02a4a0 comp rx=0 tx=0).stop 2024-09-17T04:30:57.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.643+0000 7fb3a0800640 1 -- 172.21.15.100:0/774494953 shutdown_connections 2024-09-17T04:30:57.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.643+0000 7fb3a0800640 1 --2- 172.21.15.100:0/774494953 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3980b3550 0x7fb3980b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:57.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.643+0000 7fb3a0800640 1 --2- 172.21.15.100:0/774494953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb39815d760 0x7fb3981535c0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:57.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.643+0000 7fb3a0800640 1 -- 172.21.15.100:0/774494953 >> 172.21.15.100:0/774494953 conn(0x7fb39806d730 msgr2=0x7fb3980b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:57.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.643+0000 7fb3a0800640 1 -- 172.21.15.100:0/774494953 shutdown_connections 2024-09-17T04:30:57.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.643+0000 7fb3a0800640 1 -- 172.21.15.100:0/774494953 wait complete. 2024-09-17T04:30:57.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.645+0000 7fb3a0800640 1 Processor -- start 2024-09-17T04:30:57.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.645+0000 7fb3a0800640 1 -- start start 2024-09-17T04:30:57.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.647+0000 7fb3a0800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3980ad0e0 0x7fb398149860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:57.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.647+0000 7fb3a0800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3980b3550 0x7fb3981516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:57.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.647+0000 7fb3a0800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb39815d760 0x7fb3981f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:57.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.647+0000 7fb3a0800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb39806f240 con 0x7fb3980b3550 2024-09-17T04:30:57.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.647+0000 7fb3a0800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb3980b4e30 con 0x7fb3980ad0e0 2024-09-17T04:30:57.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.647+0000 7fb3a0800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb3980b3ea0 con 0x7fb39815d760 2024-09-17T04:30:57.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.647+0000 7fb39d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3980b3550 0x7fb3981516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:57.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.647+0000 7fb39d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3980b3550 0x7fb3981516a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53616/0 (socket says 172.21.15.100:53616) 2024-09-17T04:30:57.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.647+0000 7fb39d400640 1 -- 172.21.15.100:0/3697892774 learned_addr learned my addr 172.21.15.100:0/3697892774 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:57.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.647+0000 7fb39de00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3980ad0e0 0x7fb398149860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:57.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.647+0000 7fb39e800640 1 --2- 172.21.15.100:0/3697892774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb39815d760 0x7fb3981f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:57.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.647+0000 7fb39d400640 1 -- 172.21.15.100:0/3697892774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb39815d760 msgr2=0x7fb3981f5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:57.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.647+0000 7fb39d400640 1 --2- 172.21.15.100:0/3697892774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb39815d760 0x7fb3981f5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:57.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.647+0000 7fb39d400640 1 -- 172.21.15.100:0/3697892774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3980ad0e0 msgr2=0x7fb398149860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:57.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.647+0000 7fb39d400640 1 --2- 172.21.15.100:0/3697892774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3980ad0e0 0x7fb398149860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:57.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.648+0000 7fb39d400640 1 -- 172.21.15.100:0/3697892774 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb39814c350 con 0x7fb3980b3550 2024-09-17T04:30:57.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.648+0000 7fb39d400640 1 --2- 172.21.15.100:0/3697892774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3980b3550 0x7fb3981516a0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fb38800b3c0 tx=0x7fb38800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:57.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.648+0000 7fb392000640 1 -- 172.21.15.100:0/3697892774 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb388011830 con 0x7fb3980b3550 2024-09-17T04:30:57.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.648+0000 7fb392000640 1 -- 172.21.15.100:0/3697892774 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb388016070 con 0x7fb3980b3550 2024-09-17T04:30:57.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.648+0000 7fb3a0800640 1 -- 172.21.15.100:0/3697892774 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb39814c630 con 0x7fb3980b3550 2024-09-17T04:30:57.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.648+0000 7fb392000640 1 -- 172.21.15.100:0/3697892774 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb388011da0 con 0x7fb3980b3550 2024-09-17T04:30:57.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.648+0000 7fb3a0800640 1 -- 172.21.15.100:0/3697892774 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb398151180 con 0x7fb3980b3550 2024-09-17T04:30:57.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.649+0000 7fb3a0800640 1 -- 172.21.15.100:0/3697892774 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3980718f0 con 0x7fb3980b3550 2024-09-17T04:30:57.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.651+0000 7fb392000640 1 -- 172.21.15.100:0/3697892774 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fb38801b460 con 0x7fb3980b3550 2024-09-17T04:30:57.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.653+0000 7fb392000640 1 --2- 172.21.15.100:0/3697892774 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb37c079010 0x7fb37c07b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:57.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.653+0000 7fb392000640 1 -- 172.21.15.100:0/3697892774 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5823+0+0 (secure 0 0 0) 0x7fb38809b010 con 0x7fb3980b3550 2024-09-17T04:30:57.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.653+0000 7fb39de00640 1 --2- 172.21.15.100:0/3697892774 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb37c079010 0x7fb37c07b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:57.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.654+0000 7fb39de00640 1 --2- 172.21.15.100:0/3697892774 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb37c079010 0x7fb37c07b520 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb38c033000 tx=0x7fb38c004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:57.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.657+0000 7fb392000640 1 -- 172.21.15.100:0/3697892774 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb3980718f0 con 0x7fb3980b3550 2024-09-17T04:30:57.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.876+0000 7fb3a0800640 1 -- 172.21.15.100:0/3697892774 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3980b42f0 con 0x7fb3980b3550 2024-09-17T04:30:57.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.877+0000 7fb392000640 1 -- 172.21.15.100:0/3697892774 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5234 (secure 0 0 0) 0x7fb388020020 con 0x7fb3980b3550 2024-09-17T04:30:57.876 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:30:57.876 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":73,"btime":"2024-09-17T04:30:10:474753+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6044,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1891177834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1891177834},{"type":"v1","addr":"172.21.15.100:6835","nonce":1891177834}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":6047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1158083377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1158083377},{"type":"v1","addr":"172.21.15.184:6835","nonce":1158083377}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":6697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/714465116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":714465116},{"type":"v1","addr":"172.21.15.184:6801","nonce":714465116}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:29:59.487577+0000","modified":"2024-09-17T04:30:02.508652+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6972":{"gid":6972,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6833/2666467995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2666467995},{"type":"v1","addr":"172.21.15.100:6833","nonce":2666467995}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6972,"qdb_cluster":[6972]},"id":8}]} 2024-09-17T04:30:57.876 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 73 2024-09-17T04:30:57.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.879+0000 7fb3a0800640 1 -- 172.21.15.100:0/3697892774 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb37c079010 msgr2=0x7fb37c07b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:57.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.879+0000 7fb3a0800640 1 --2- 172.21.15.100:0/3697892774 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb37c079010 0x7fb37c07b520 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb38c033000 tx=0x7fb38c004730 comp rx=0 tx=0).stop 2024-09-17T04:30:57.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.879+0000 7fb3a0800640 1 -- 172.21.15.100:0/3697892774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3980b3550 msgr2=0x7fb3981516a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:57.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.879+0000 7fb3a0800640 1 --2- 172.21.15.100:0/3697892774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3980b3550 0x7fb3981516a0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fb38800b3c0 tx=0x7fb38800b880 comp rx=0 tx=0).stop 2024-09-17T04:30:57.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.879+0000 7fb3a0800640 1 -- 172.21.15.100:0/3697892774 shutdown_connections 2024-09-17T04:30:57.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.879+0000 7fb3a0800640 1 --2- 172.21.15.100:0/3697892774 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb37c079010 0x7fb37c07b520 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:57.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.879+0000 7fb3a0800640 1 --2- 172.21.15.100:0/3697892774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb39815d760 0x7fb3981f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:57.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.879+0000 7fb3a0800640 1 --2- 172.21.15.100:0/3697892774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3980b3550 0x7fb3981516a0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:57.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.879+0000 7fb3a0800640 1 --2- 172.21.15.100:0/3697892774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3980ad0e0 0x7fb398149860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:57.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.879+0000 7fb3a0800640 1 -- 172.21.15.100:0/3697892774 >> 172.21.15.100:0/3697892774 conn(0x7fb39806d730 msgr2=0x7fb39815ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:57.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.879+0000 7fb3a0800640 1 -- 172.21.15.100:0/3697892774 shutdown_connections 2024-09-17T04:30:57.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:57.879+0000 7fb3a0800640 1 -- 172.21.15.100:0/3697892774 wait complete. 2024-09-17T04:30:57.901 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:30:58.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.121+0000 7f68c6e00640 1 -- 172.21.15.100:0/1241274023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68c00ad0e0 msgr2=0x7f68c00ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:58.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.121+0000 7f68c6e00640 1 --2- 172.21.15.100:0/1241274023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68c00ad0e0 0x7f68c00ad4f0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f68c0072d90 tx=0x7f68a802a4d0 comp rx=0 tx=0).stop 2024-09-17T04:30:58.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.121+0000 7f68c6e00640 1 -- 172.21.15.100:0/1241274023 shutdown_connections 2024-09-17T04:30:58.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.121+0000 7f68c6e00640 1 --2- 172.21.15.100:0/1241274023 >> v2:172.21.15.184:3300/0 conn(0x7f68c0160310 0x7f68c0160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:58.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.121+0000 7f68c6e00640 1 --2- 172.21.15.100:0/1241274023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68c00ad0e0 0x7f68c00ad4f0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:58.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.121+0000 7f68c6e00640 1 --2- 172.21.15.100:0/1241274023 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68c015d760 0x7f68c01535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:58.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.121+0000 7f68c6e00640 1 -- 172.21.15.100:0/1241274023 >> 172.21.15.100:0/1241274023 conn(0x7f68c006d730 msgr2=0x7f68c00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:58.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.121+0000 7f68c6e00640 1 -- 172.21.15.100:0/1241274023 shutdown_connections 2024-09-17T04:30:58.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.121+0000 7f68c6e00640 1 -- 172.21.15.100:0/1241274023 wait complete. 2024-09-17T04:30:58.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.123+0000 7f68c6e00640 1 Processor -- start 2024-09-17T04:30:58.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.123+0000 7f68c6e00640 1 -- start start 2024-09-17T04:30:58.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.125+0000 7f68c6e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68c00ad0e0 0x7f68c01f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:58.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.125+0000 7f68c6e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68c015d760 0x7f68c01487f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:58.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.125+0000 7f68c6e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68c0160310 0x7f68c015b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:58.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.125+0000 7f68c6e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f68c00b4da0 con 0x7f68c0160310 2024-09-17T04:30:58.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.125+0000 7f68c6e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f68c00b4c20 con 0x7f68c00ad0e0 2024-09-17T04:30:58.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.125+0000 7f68c6e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f68c0160c60 con 0x7f68c015d760 2024-09-17T04:30:58.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.125+0000 7f68c4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68c0160310 0x7f68c015b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:58.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.125+0000 7f68c4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68c0160310 0x7f68c015b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53630/0 (socket says 172.21.15.100:53630) 2024-09-17T04:30:58.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.125+0000 7f68c4e00640 1 -- 172.21.15.100:0/3371612559 learned_addr learned my addr 172.21.15.100:0/3371612559 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:58.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.125+0000 7f68b7400640 1 --2- 172.21.15.100:0/3371612559 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68c015d760 0x7f68c01487f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:58.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.125+0000 7f68bfe00640 1 --2- 172.21.15.100:0/3371612559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68c00ad0e0 0x7f68c01f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:58.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.126+0000 7f68c4e00640 1 -- 172.21.15.100:0/3371612559 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68c015d760 msgr2=0x7f68c01487f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:58.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.126+0000 7f68c4e00640 1 --2- 172.21.15.100:0/3371612559 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68c015d760 0x7f68c01487f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:58.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.126+0000 7f68c4e00640 1 -- 172.21.15.100:0/3371612559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68c00ad0e0 msgr2=0x7f68c01f63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:58.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.126+0000 7f68c4e00640 1 --2- 172.21.15.100:0/3371612559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68c00ad0e0 0x7f68c01f63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:58.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.126+0000 7f68c4e00640 1 -- 172.21.15.100:0/3371612559 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c01f6900 con 0x7f68c0160310 2024-09-17T04:30:58.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.126+0000 7f68c4e00640 1 --2- 172.21.15.100:0/3371612559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68c0160310 0x7f68c015b7f0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f68b800e9c0 tx=0x7f68b800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:58.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.126+0000 7f68bcc00640 1 -- 172.21.15.100:0/3371612559 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b8004d20 con 0x7f68c0160310 2024-09-17T04:30:58.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.126+0000 7f68bcc00640 1 -- 172.21.15.100:0/3371612559 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68b800cb20 con 0x7f68c0160310 2024-09-17T04:30:58.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.126+0000 7f68bcc00640 1 -- 172.21.15.100:0/3371612559 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b8005020 con 0x7f68c0160310 2024-09-17T04:30:58.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.126+0000 7f68c6e00640 1 -- 172.21.15.100:0/3371612559 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c0156fc0 con 0x7f68c0160310 2024-09-17T04:30:58.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.126+0000 7f68c6e00640 1 -- 172.21.15.100:0/3371612559 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c0074b70 con 0x7f68c0160310 2024-09-17T04:30:58.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.127+0000 7f68c6e00640 1 -- 172.21.15.100:0/3371612559 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68c0156d30 con 0x7f68c0160310 2024-09-17T04:30:58.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.128+0000 7f68bcc00640 1 -- 172.21.15.100:0/3371612559 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f68b80051c0 con 0x7f68c0160310 2024-09-17T04:30:58.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.129+0000 7f68bcc00640 1 --2- 172.21.15.100:0/3371612559 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f689c079010 0x7f689c07b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:58.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.130+0000 7f68bcc00640 1 -- 172.21.15.100:0/3371612559 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5823+0+0 (secure 0 0 0) 0x7f68b809bea0 con 0x7f68c0160310 2024-09-17T04:30:58.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.130+0000 7f68bfe00640 1 --2- 172.21.15.100:0/3371612559 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f689c079010 0x7f689c07b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:58.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.130+0000 7f68bfe00640 1 --2- 172.21.15.100:0/3371612559 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f689c079010 0x7f689c07b520 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f68b0004630 tx=0x7f68b000b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:58.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.134+0000 7f68bcc00640 1 -- 172.21.15.100:0/3371612559 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f68b8016030 con 0x7f68c0160310 2024-09-17T04:30:58.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.316+0000 7f68c6e00640 1 -- 172.21.15.100:0/3371612559 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f68c01535c0 con 0x7f68c0160310 2024-09-17T04:30:58.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.318+0000 7f68bcc00640 1 -- 172.21.15.100:0/3371612559 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+16999 (secure 0 0 0) 0x7f68b80634a0 con 0x7f68c0160310 2024-09-17T04:30:58.318 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:30:58.318 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":108,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:30:55.813015+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:29:57.388120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-17T04:29:58.449229+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.9","mappings":[{"from":0,"to":5}]},{"pgid":"17.37","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:30:58.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.320+0000 7f68c6e00640 1 -- 172.21.15.100:0/3371612559 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f689c079010 msgr2=0x7f689c07b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:58.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.320+0000 7f68c6e00640 1 --2- 172.21.15.100:0/3371612559 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f689c079010 0x7f689c07b520 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f68b0004630 tx=0x7f68b000b2e0 comp rx=0 tx=0).stop 2024-09-17T04:30:58.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.320+0000 7f68c6e00640 1 -- 172.21.15.100:0/3371612559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68c0160310 msgr2=0x7f68c015b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:58.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.320+0000 7f68c6e00640 1 --2- 172.21.15.100:0/3371612559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68c0160310 0x7f68c015b7f0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f68b800e9c0 tx=0x7f68b800ee80 comp rx=0 tx=0).stop 2024-09-17T04:30:58.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.320+0000 7f68c6e00640 1 -- 172.21.15.100:0/3371612559 shutdown_connections 2024-09-17T04:30:58.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.320+0000 7f68c6e00640 1 --2- 172.21.15.100:0/3371612559 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f689c079010 0x7f689c07b520 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:58.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.320+0000 7f68c6e00640 1 --2- 172.21.15.100:0/3371612559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68c0160310 0x7f68c015b7f0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:58.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.320+0000 7f68c6e00640 1 --2- 172.21.15.100:0/3371612559 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68c015d760 0x7f68c01487f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:58.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.320+0000 7f68c6e00640 1 --2- 172.21.15.100:0/3371612559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68c00ad0e0 0x7f68c01f63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:58.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.320+0000 7f68c6e00640 1 -- 172.21.15.100:0/3371612559 >> 172.21.15.100:0/3371612559 conn(0x7f68c006d730 msgr2=0x7f68c015ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:58.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.320+0000 7f68c6e00640 1 -- 172.21.15.100:0/3371612559 shutdown_connections 2024-09-17T04:30:58.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.320+0000 7f68c6e00640 1 -- 172.21.15.100:0/3371612559 wait complete. 2024-09-17T04:30:58.341 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T04:30:58.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.561+0000 7f6e44000640 1 -- 172.21.15.100:0/1865210478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e3c0cfc80 msgr2=0x7f6e3c156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:58.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.561+0000 7f6e44000640 1 --2- 172.21.15.100:0/1865210478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e3c0cfc80 0x7f6e3c156e40 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009860 tx=0x7f6e3002a4b0 comp rx=0 tx=0).stop 2024-09-17T04:30:58.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.561+0000 7f6e44000640 1 -- 172.21.15.100:0/1865210478 shutdown_connections 2024-09-17T04:30:58.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.561+0000 7f6e44000640 1 --2- 172.21.15.100:0/1865210478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e3c0cfc80 0x7f6e3c156e40 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:58.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.561+0000 7f6e44000640 1 -- 172.21.15.100:0/1865210478 >> 172.21.15.100:0/1865210478 conn(0x7f6e3c06d3d0 msgr2=0x7f6e3c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:58.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.561+0000 7f6e44000640 1 -- 172.21.15.100:0/1865210478 shutdown_connections 2024-09-17T04:30:58.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.561+0000 7f6e44000640 1 -- 172.21.15.100:0/1865210478 wait complete. 2024-09-17T04:30:58.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.563+0000 7f6e44000640 1 Processor -- start 2024-09-17T04:30:58.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.564+0000 7f6e44000640 1 -- start start 2024-09-17T04:30:58.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.565+0000 7f6e44000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e3c0705b0 0x7f6e3c157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:58.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.565+0000 7f6e44000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e3c0cfc80 0x7f6e3c0c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:58.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.565+0000 7f6e44000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e3c15ce30 0x7f6e3c159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:58.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.565+0000 7f6e44000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c0cd100 con 0x7f6e3c0cfc80 2024-09-17T04:30:58.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.565+0000 7f6e44000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c0ccf80 con 0x7f6e3c0705b0 2024-09-17T04:30:58.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.565+0000 7f6e44000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6e3c0cd280 con 0x7f6e3c15ce30 2024-09-17T04:30:58.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.565+0000 7f6e40c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e3c0cfc80 0x7f6e3c0c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:58.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.565+0000 7f6e40c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e3c0cfc80 0x7f6e3c0c5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53656/0 (socket says 172.21.15.100:53656) 2024-09-17T04:30:58.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.565+0000 7f6e40c00640 1 -- 172.21.15.100:0/1839667861 learned_addr learned my addr 172.21.15.100:0/1839667861 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:58.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.565+0000 7f6e41600640 1 --2- 172.21.15.100:0/1839667861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e3c0705b0 0x7f6e3c157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:58.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.565+0000 7f6e42000640 1 --2- 172.21.15.100:0/1839667861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e3c15ce30 0x7f6e3c159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:58.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.566+0000 7f6e40c00640 1 -- 172.21.15.100:0/1839667861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e3c15ce30 msgr2=0x7f6e3c159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:58.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.566+0000 7f6e40c00640 1 --2- 172.21.15.100:0/1839667861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e3c15ce30 0x7f6e3c159910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:58.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.566+0000 7f6e40c00640 1 -- 172.21.15.100:0/1839667861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e3c0705b0 msgr2=0x7f6e3c157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:58.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.566+0000 7f6e40c00640 1 --2- 172.21.15.100:0/1839667861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e3c0705b0 0x7f6e3c157f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:58.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.566+0000 7f6e40c00640 1 -- 172.21.15.100:0/1839667861 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e3c158870 con 0x7f6e3c0cfc80 2024-09-17T04:30:58.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.566+0000 7f6e40c00640 1 --2- 172.21.15.100:0/1839667861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e3c0cfc80 0x7f6e3c0c5d00 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f6e2800c3d0 tx=0x7f6e2800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:58.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.566+0000 7f6e39600640 1 -- 172.21.15.100:0/1839667861 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e28011790 con 0x7f6e3c0cfc80 2024-09-17T04:30:58.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.566+0000 7f6e39600640 1 -- 172.21.15.100:0/1839667861 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e28017070 con 0x7f6e3c0cfc80 2024-09-17T04:30:58.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.566+0000 7f6e44000640 1 -- 172.21.15.100:0/1839667861 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e3c158b50 con 0x7f6e3c0cfc80 2024-09-17T04:30:58.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.566+0000 7f6e39600640 1 -- 172.21.15.100:0/1839667861 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e28011ce0 con 0x7f6e3c0cfc80 2024-09-17T04:30:58.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.567+0000 7f6e44000640 1 -- 172.21.15.100:0/1839667861 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e3c159110 con 0x7f6e3c0cfc80 2024-09-17T04:30:58.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.568+0000 7f6e44000640 1 -- 172.21.15.100:0/1839667861 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e3c0cd7d0 con 0x7f6e3c0cfc80 2024-09-17T04:30:58.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.569+0000 7f6e39600640 1 -- 172.21.15.100:0/1839667861 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f6e2801c460 con 0x7f6e3c0cfc80 2024-09-17T04:30:58.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.570+0000 7f6e39600640 1 --2- 172.21.15.100:0/1839667861 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6e1c079040 0x7f6e1c07b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:58.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.570+0000 7f6e39600640 1 -- 172.21.15.100:0/1839667861 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5823+0+0 (secure 0 0 0) 0x7f6e2809b200 con 0x7f6e3c0cfc80 2024-09-17T04:30:58.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.570+0000 7f6e41600640 1 --2- 172.21.15.100:0/1839667861 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6e1c079040 0x7f6e1c07b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:58.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.570+0000 7f6e41600640 1 --2- 172.21.15.100:0/1839667861 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6e1c079040 0x7f6e1c07b550 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6e30034000 tx=0x7f6e30004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:58.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.574+0000 7f6e39600640 1 -- 172.21.15.100:0/1839667861 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6e28062930 con 0x7f6e3c0cfc80 2024-09-17T04:30:58.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.792+0000 7f6e44000640 1 -- 172.21.15.100:0/1839667861 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6e3c0afa70 con 0x7f6e3c0cfc80 2024-09-17T04:30:58.792 INFO:tasks.ceph.mon.a.smithi100.stderr:2024-09-17T04:30:58.793+0000 7fd3d0a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T04:30:58.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.796+0000 7f6e39600640 1 -- 172.21.15.100:0/1839667861 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v74) ==== 161+0+0 (secure 0 0 0) 0x7f6e28067360 con 0x7f6e3c0cfc80 2024-09-17T04:30:58.795 INFO:teuthology.orchestra.run.smithi100.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T04:30:58.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.799+0000 7f6e44000640 1 -- 172.21.15.100:0/1839667861 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6e1c079040 msgr2=0x7f6e1c07b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:58.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.799+0000 7f6e44000640 1 --2- 172.21.15.100:0/1839667861 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6e1c079040 0x7f6e1c07b550 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6e30034000 tx=0x7f6e30004730 comp rx=0 tx=0).stop 2024-09-17T04:30:58.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.799+0000 7f6e44000640 1 -- 172.21.15.100:0/1839667861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e3c0cfc80 msgr2=0x7f6e3c0c5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:58.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.799+0000 7f6e44000640 1 --2- 172.21.15.100:0/1839667861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e3c0cfc80 0x7f6e3c0c5d00 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f6e2800c3d0 tx=0x7f6e2800c890 comp rx=0 tx=0).stop 2024-09-17T04:30:58.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.799+0000 7f6e44000640 1 -- 172.21.15.100:0/1839667861 shutdown_connections 2024-09-17T04:30:58.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.799+0000 7f6e44000640 1 --2- 172.21.15.100:0/1839667861 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6e1c079040 0x7f6e1c07b550 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:58.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.799+0000 7f6e44000640 1 --2- 172.21.15.100:0/1839667861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e3c15ce30 0x7f6e3c159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:58.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.799+0000 7f6e44000640 1 --2- 172.21.15.100:0/1839667861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e3c0cfc80 0x7f6e3c0c5d00 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:58.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.799+0000 7f6e44000640 1 --2- 172.21.15.100:0/1839667861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e3c0705b0 0x7f6e3c157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:58.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.799+0000 7f6e44000640 1 -- 172.21.15.100:0/1839667861 >> 172.21.15.100:0/1839667861 conn(0x7f6e3c06d3d0 msgr2=0x7f6e3c15dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:58.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.800+0000 7f6e44000640 1 -- 172.21.15.100:0/1839667861 shutdown_connections 2024-09-17T04:30:58.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:58.800+0000 7f6e44000640 1 -- 172.21.15.100:0/1839667861 wait complete. 2024-09-17T04:30:58.824 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T04:30:58.943 INFO:tasks.ceph.mds.a.smithi100.stdout:starting mds.a at 2024-09-17T04:30:59.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.052+0000 7fc65f200640 1 -- 172.21.15.100:0/2297136198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6600b3f30 msgr2=0x7fc660156ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:59.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.052+0000 7fc65f200640 1 --2- 172.21.15.100:0/2297136198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6600b3f30 0x7fc660156ca0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fc6600cfba0 tx=0x7fc65002a530 comp rx=0 tx=0).stop 2024-09-17T04:30:59.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.052+0000 7fc65f200640 1 -- 172.21.15.100:0/2297136198 shutdown_connections 2024-09-17T04:30:59.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.052+0000 7fc65f200640 1 --2- 172.21.15.100:0/2297136198 >> v2:172.21.15.184:3300/0 conn(0x7fc6600cd1d0 0x7fc6600af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:59.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.052+0000 7fc65f200640 1 --2- 172.21.15.100:0/2297136198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6600b3f30 0x7fc660156ca0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:59.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.052+0000 7fc65f200640 1 -- 172.21.15.100:0/2297136198 >> 172.21.15.100:0/2297136198 conn(0x7fc66006d620 msgr2=0x7fc6600b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:59.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.052+0000 7fc65f200640 1 -- 172.21.15.100:0/2297136198 shutdown_connections 2024-09-17T04:30:59.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.052+0000 7fc65f200640 1 -- 172.21.15.100:0/2297136198 wait complete. 2024-09-17T04:30:59.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.054+0000 7fc65f200640 1 Processor -- start 2024-09-17T04:30:59.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.054+0000 7fc65f200640 1 -- start start 2024-09-17T04:30:59.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.056+0000 7fc65f200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6600b3f30 0x7fc6601f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:59.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.056+0000 7fc65f200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6600cd1d0 0x7fc660145190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:59.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.056+0000 7fc65f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc66015d6e0 0x7fc6601456d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:59.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.056+0000 7fc65f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc6600af9a0 con 0x7fc66015d6e0 2024-09-17T04:30:59.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.056+0000 7fc65f200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc6600af820 con 0x7fc6600cd1d0 2024-09-17T04:30:59.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.056+0000 7fc65f200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc6600afb20 con 0x7fc6600b3f30 2024-09-17T04:30:59.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.056+0000 7fc65e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc66015d6e0 0x7fc6601456d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:59.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.056+0000 7fc65e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc66015d6e0 0x7fc6601456d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53698/0 (socket says 172.21.15.100:53698) 2024-09-17T04:30:59.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.056+0000 7fc65e800640 1 -- 172.21.15.100:0/1035155225 learned_addr learned my addr 172.21.15.100:0/1035155225 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:30:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.056+0000 7fc65de00640 1 --2- 172.21.15.100:0/1035155225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6600b3f30 0x7fc6601f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.056+0000 7fc655400640 1 --2- 172.21.15.100:0/1035155225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6600cd1d0 0x7fc660145190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.056+0000 7fc65e800640 1 -- 172.21.15.100:0/1035155225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6600b3f30 msgr2=0x7fc6601f54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.057+0000 7fc65e800640 1 --2- 172.21.15.100:0/1035155225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6600b3f30 0x7fc6601f54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.057+0000 7fc65e800640 1 -- 172.21.15.100:0/1035155225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6600cd1d0 msgr2=0x7fc660145190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.057+0000 7fc65e800640 1 --2- 172.21.15.100:0/1035155225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6600cd1d0 0x7fc660145190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.057+0000 7fc65e800640 1 -- 172.21.15.100:0/1035155225 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc66015b660 con 0x7fc66015d6e0 2024-09-17T04:30:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.057+0000 7fc65e800640 1 --2- 172.21.15.100:0/1035155225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc66015d6e0 0x7fc6601456d0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fc65800e9c0 tx=0x7fc65800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:59.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.057+0000 7fc656a00640 1 -- 172.21.15.100:0/1035155225 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc658004d20 con 0x7fc66015d6e0 2024-09-17T04:30:59.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.057+0000 7fc656a00640 1 -- 172.21.15.100:0/1035155225 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc65800cac0 con 0x7fc66015d6e0 2024-09-17T04:30:59.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.057+0000 7fc65f200640 1 -- 172.21.15.100:0/1035155225 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc66015b940 con 0x7fc66015d6e0 2024-09-17T04:30:59.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.057+0000 7fc656a00640 1 -- 172.21.15.100:0/1035155225 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc658005020 con 0x7fc66015d6e0 2024-09-17T04:30:59.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.057+0000 7fc65f200640 1 -- 172.21.15.100:0/1035155225 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc66015bf00 con 0x7fc66015d6e0 2024-09-17T04:30:59.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.058+0000 7fc65f200640 1 -- 172.21.15.100:0/1035155225 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc660156ca0 con 0x7fc66015d6e0 2024-09-17T04:30:59.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.059+0000 7fc656a00640 1 -- 172.21.15.100:0/1035155225 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fc6580051c0 con 0x7fc66015d6e0 2024-09-17T04:30:59.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.060+0000 7fc656a00640 1 --2- 172.21.15.100:0/1035155225 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc63c079010 0x7fc63c07b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:30:59.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.060+0000 7fc656a00640 1 -- 172.21.15.100:0/1035155225 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5881+0+0 (secure 0 0 0) 0x7fc65809bf70 con 0x7fc66015d6e0 2024-09-17T04:30:59.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.061+0000 7fc65de00640 1 --2- 172.21.15.100:0/1035155225 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc63c079010 0x7fc63c07b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:30:59.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.061+0000 7fc65de00640 1 --2- 172.21.15.100:0/1035155225 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc63c079010 0x7fc63c07b520 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc650034000 tx=0x7fc650004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:30:59.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.065+0000 7fc656a00640 1 -- 172.21.15.100:0/1035155225 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc658016030 con 0x7fc66015d6e0 2024-09-17T04:30:59.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.285+0000 7fc65f200640 1 -- 172.21.15.100:0/1035155225 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc6600d0d60 con 0x7fc66015d6e0 2024-09-17T04:30:59.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.836+0000 7fc656a00640 1 -- 172.21.15.100:0/1035155225 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v75) ==== 104+0+0 (secure 0 0 0) 0x7fc658063530 con 0x7fc66015d6e0 2024-09-17T04:30:59.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.839+0000 7fc65f200640 1 -- 172.21.15.100:0/1035155225 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc63c079010 msgr2=0x7fc63c07b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:59.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.839+0000 7fc65f200640 1 --2- 172.21.15.100:0/1035155225 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc63c079010 0x7fc63c07b520 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc650034000 tx=0x7fc650004730 comp rx=0 tx=0).stop 2024-09-17T04:30:59.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.839+0000 7fc65f200640 1 -- 172.21.15.100:0/1035155225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc66015d6e0 msgr2=0x7fc6601456d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:30:59.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.839+0000 7fc65f200640 1 --2- 172.21.15.100:0/1035155225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc66015d6e0 0x7fc6601456d0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fc65800e9c0 tx=0x7fc65800ee80 comp rx=0 tx=0).stop 2024-09-17T04:30:59.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.839+0000 7fc65f200640 1 -- 172.21.15.100:0/1035155225 shutdown_connections 2024-09-17T04:30:59.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.839+0000 7fc65f200640 1 --2- 172.21.15.100:0/1035155225 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc63c079010 0x7fc63c07b520 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:59.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.839+0000 7fc65f200640 1 --2- 172.21.15.100:0/1035155225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc66015d6e0 0x7fc6601456d0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:59.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.839+0000 7fc65f200640 1 --2- 172.21.15.100:0/1035155225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6600cd1d0 0x7fc660145190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:59.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.839+0000 7fc65f200640 1 --2- 172.21.15.100:0/1035155225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6600b3f30 0x7fc6601f54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:30:59.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.839+0000 7fc65f200640 1 -- 172.21.15.100:0/1035155225 >> 172.21.15.100:0/1035155225 conn(0x7fc66006d620 msgr2=0x7fc6600cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:30:59.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.839+0000 7fc65f200640 1 -- 172.21.15.100:0/1035155225 shutdown_connections 2024-09-17T04:30:59.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:30:59.840+0000 7fc65f200640 1 -- 172.21.15.100:0/1035155225 wait complete. 2024-09-17T04:30:59.863 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T04:31:00.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.088+0000 7f193dc00640 1 -- 172.21.15.100:0/908661200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f191c004800 msgr2=0x7f191c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:00.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.088+0000 7f193dc00640 1 --2- 172.21.15.100:0/908661200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f191c004800 0x7f191c004c10 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f1924009b20 tx=0x7f191c005830 comp rx=0 tx=0).stop 2024-09-17T04:31:00.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.088+0000 7f193dc00640 1 -- 172.21.15.100:0/908661200 shutdown_connections 2024-09-17T04:31:00.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.088+0000 7f193dc00640 1 --2- 172.21.15.100:0/908661200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f191c004800 0x7f191c004c10 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:00.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.088+0000 7f193dc00640 1 -- 172.21.15.100:0/908661200 >> 172.21.15.100:0/908661200 conn(0x7f19380b2540 msgr2=0x7f19380d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:00.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.088+0000 7f193dc00640 1 -- 172.21.15.100:0/908661200 shutdown_connections 2024-09-17T04:31:00.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.088+0000 7f193dc00640 1 -- 172.21.15.100:0/908661200 wait complete. 2024-09-17T04:31:00.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.090+0000 7f193dc00640 1 Processor -- start 2024-09-17T04:31:00.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.090+0000 7f193dc00640 1 -- start start 2024-09-17T04:31:00.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.092+0000 7f193dc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f191c004800 0x7f19381f5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:00.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.092+0000 7f193dc00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19380ca0f0 0x7f1938099910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:00.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.092+0000 7f193dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1938154fc0 0x7f19381e8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:00.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.092+0000 7f193dc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f19380af8a0 con 0x7f1938154fc0 2024-09-17T04:31:00.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.092+0000 7f193dc00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f19380af720 con 0x7f191c004800 2024-09-17T04:31:00.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.092+0000 7f193dc00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f19380afa20 con 0x7f19380ca0f0 2024-09-17T04:31:00.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.092+0000 7f1937400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1938154fc0 0x7f19381e8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:00.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.092+0000 7f1936a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f191c004800 0x7f19381f5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:00.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.092+0000 7f1936000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19380ca0f0 0x7f1938099910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:00.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.092+0000 7f1937400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1938154fc0 0x7f19381e8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53716/0 (socket says 172.21.15.100:53716) 2024-09-17T04:31:00.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.092+0000 7f1937400640 1 -- 172.21.15.100:0/4228945794 learned_addr learned my addr 172.21.15.100:0/4228945794 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:00.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.092+0000 7f1937400640 1 -- 172.21.15.100:0/4228945794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19380ca0f0 msgr2=0x7f1938099910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:00.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.092+0000 7f1937400640 1 --2- 172.21.15.100:0/4228945794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19380ca0f0 0x7f1938099910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:00.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.093+0000 7f1937400640 1 -- 172.21.15.100:0/4228945794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f191c004800 msgr2=0x7f19381f5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:00.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.093+0000 7f1937400640 1 --2- 172.21.15.100:0/4228945794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f191c004800 0x7f19381f5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:00.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.093+0000 7f1937400640 1 -- 172.21.15.100:0/4228945794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19381f62f0 con 0x7f1938154fc0 2024-09-17T04:31:00.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.093+0000 7f1937400640 1 --2- 172.21.15.100:0/4228945794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1938154fc0 0x7f19381e8290 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f19380cac00 tx=0x7f193000eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:00.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.093+0000 7f192ea00640 1 -- 172.21.15.100:0/4228945794 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193000ca10 con 0x7f1938154fc0 2024-09-17T04:31:00.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.093+0000 7f192ea00640 1 -- 172.21.15.100:0/4228945794 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1930018070 con 0x7f1938154fc0 2024-09-17T04:31:00.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.093+0000 7f192ea00640 1 -- 172.21.15.100:0/4228945794 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1930011460 con 0x7f1938154fc0 2024-09-17T04:31:00.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.093+0000 7f193dc00640 1 -- 172.21.15.100:0/4228945794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19381f65d0 con 0x7f1938154fc0 2024-09-17T04:31:00.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.093+0000 7f193dc00640 1 -- 172.21.15.100:0/4228945794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19381e7a90 con 0x7f1938154fc0 2024-09-17T04:31:00.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.094+0000 7f193dc00640 1 -- 172.21.15.100:0/4228945794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19381f6040 con 0x7f1938154fc0 2024-09-17T04:31:00.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.095+0000 7f192ea00640 1 -- 172.21.15.100:0/4228945794 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f1930011600 con 0x7f1938154fc0 2024-09-17T04:31:00.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.096+0000 7f192ea00640 1 --2- 172.21.15.100:0/4228945794 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1918079010 0x7f191807b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:00.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.097+0000 7f192ea00640 1 -- 172.21.15.100:0/4228945794 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5881+0+0 (secure 0 0 0) 0x7f193009b960 con 0x7f1938154fc0 2024-09-17T04:31:00.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.097+0000 7f1936a00640 1 --2- 172.21.15.100:0/4228945794 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1918079010 0x7f191807b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:00.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.097+0000 7f1936a00640 1 --2- 172.21.15.100:0/4228945794 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1918079010 0x7f191807b520 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1928000c00 tx=0x7f1928006d10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:00.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.101+0000 7f192ea00640 1 -- 172.21.15.100:0/4228945794 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f1930062fc0 con 0x7f1938154fc0 2024-09-17T04:31:00.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.284+0000 7f193dc00640 1 -- 172.21.15.100:0/4228945794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f193806cf10 con 0x7f1938154fc0 2024-09-17T04:31:00.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.843+0000 7f192ea00640 1 -- 172.21.15.100:0/4228945794 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v110) ==== 181+0+0 (secure 0 0 0) 0x7f19300679f0 con 0x7f1938154fc0 2024-09-17T04:31:00.843 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' removed 2024-09-17T04:31:00.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.847+0000 7f193dc00640 1 -- 172.21.15.100:0/4228945794 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1918079010 msgr2=0x7f191807b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:00.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.847+0000 7f193dc00640 1 --2- 172.21.15.100:0/4228945794 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1918079010 0x7f191807b520 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1928000c00 tx=0x7f1928006d10 comp rx=0 tx=0).stop 2024-09-17T04:31:00.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.847+0000 7f193dc00640 1 -- 172.21.15.100:0/4228945794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1938154fc0 msgr2=0x7f19381e8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:00.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.847+0000 7f193dc00640 1 --2- 172.21.15.100:0/4228945794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1938154fc0 0x7f19381e8290 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f19380cac00 tx=0x7f193000eb60 comp rx=0 tx=0).stop 2024-09-17T04:31:00.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.847+0000 7f193dc00640 1 -- 172.21.15.100:0/4228945794 shutdown_connections 2024-09-17T04:31:00.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.847+0000 7f193dc00640 1 --2- 172.21.15.100:0/4228945794 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1918079010 0x7f191807b520 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:00.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.847+0000 7f193dc00640 1 --2- 172.21.15.100:0/4228945794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1938154fc0 0x7f19381e8290 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:00.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.847+0000 7f193dc00640 1 --2- 172.21.15.100:0/4228945794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19380ca0f0 0x7f1938099910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:00.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.847+0000 7f193dc00640 1 --2- 172.21.15.100:0/4228945794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f191c004800 0x7f19381f5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:00.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.847+0000 7f193dc00640 1 -- 172.21.15.100:0/4228945794 >> 172.21.15.100:0/4228945794 conn(0x7f19380b2540 msgr2=0x7f19381eaa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:00.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.848+0000 7f193dc00640 1 -- 172.21.15.100:0/4228945794 shutdown_connections 2024-09-17T04:31:00.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:00.848+0000 7f193dc00640 1 -- 172.21.15.100:0/4228945794 wait complete. 2024-09-17T04:31:00.872 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T04:31:01.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.099+0000 7fef3e400640 1 -- 172.21.15.100:0/4268068530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef1c004d90 msgr2=0x7fef1c005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:01.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.099+0000 7fef3e400640 1 --2- 172.21.15.100:0/4268068530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef1c004d90 0x7fef1c005220 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fef30005fd0 tx=0x7fef1c005830 comp rx=0 tx=0).stop 2024-09-17T04:31:01.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.099+0000 7fef37e00640 1 -- 172.21.15.100:0/4268068530 reap_dead start 2024-09-17T04:31:01.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.099+0000 7fef3e400640 1 -- 172.21.15.100:0/4268068530 shutdown_connections 2024-09-17T04:31:01.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.099+0000 7fef3e400640 1 -- 172.21.15.100:0/4268068530 >> 172.21.15.100:0/4268068530 conn(0x7fef380b2540 msgr2=0x7fef380d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:01.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.099+0000 7fef3e400640 1 -- 172.21.15.100:0/4268068530 shutdown_connections 2024-09-17T04:31:01.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.100+0000 7fef3e400640 1 -- 172.21.15.100:0/4268068530 wait complete. 2024-09-17T04:31:01.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.102+0000 7fef3e400640 1 Processor -- start 2024-09-17T04:31:01.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.102+0000 7fef3e400640 1 -- start start 2024-09-17T04:31:01.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.103+0000 7fef3e400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef1c004d90 0x7fef381f6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:01.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.103+0000 7fef3e400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef381e90a0 0x7fef381531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:01.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.103+0000 7fef3e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef380cdfb0 0x7fef380b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:01.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.103+0000 7fef3e400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fef380b32a0 con 0x7fef380cdfb0 2024-09-17T04:31:01.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.103+0000 7fef3e400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fef380b3120 con 0x7fef381e90a0 2024-09-17T04:31:01.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.103+0000 7fef3e400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fef380b3420 con 0x7fef1c004d90 2024-09-17T04:31:01.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.104+0000 7fef37e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef380cdfb0 0x7fef380b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:01.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.104+0000 7fef37e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef380cdfb0 0x7fef380b0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53734/0 (socket says 172.21.15.100:53734) 2024-09-17T04:31:01.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.104+0000 7fef37e00640 1 -- 172.21.15.100:0/3589404436 learned_addr learned my addr 172.21.15.100:0/3589404436 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:01.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.104+0000 7fef36a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef381e90a0 0x7fef381531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:01.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.104+0000 7fef37400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef1c004d90 0x7fef381f6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:01.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.104+0000 7fef37e00640 1 -- 172.21.15.100:0/3589404436 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef1c004d90 msgr2=0x7fef381f6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:01.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.104+0000 7fef37e00640 1 --2- 172.21.15.100:0/3589404436 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef1c004d90 0x7fef381f6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:01.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.104+0000 7fef37e00640 1 -- 172.21.15.100:0/3589404436 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef381e90a0 msgr2=0x7fef381531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:01.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.104+0000 7fef37e00640 1 --2- 172.21.15.100:0/3589404436 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef381e90a0 0x7fef381531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:01.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.104+0000 7fef37e00640 1 -- 172.21.15.100:0/3589404436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef381f6790 con 0x7fef380cdfb0 2024-09-17T04:31:01.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.104+0000 7fef37e00640 1 --2- 172.21.15.100:0/3589404436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef380cdfb0 0x7fef380b0dc0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fef30036040 tx=0x7fef30033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:01.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.105+0000 7fef2f400640 1 -- 172.21.15.100:0/3589404436 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef30042070 con 0x7fef380cdfb0 2024-09-17T04:31:01.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.105+0000 7fef2f400640 1 -- 172.21.15.100:0/3589404436 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef30003970 con 0x7fef380cdfb0 2024-09-17T04:31:01.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.105+0000 7fef3e400640 1 -- 172.21.15.100:0/3589404436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef381ea8c0 con 0x7fef380cdfb0 2024-09-17T04:31:01.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.105+0000 7fef2f400640 1 -- 172.21.15.100:0/3589404436 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef30038040 con 0x7fef380cdfb0 2024-09-17T04:31:01.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.105+0000 7fef3e400640 1 -- 172.21.15.100:0/3589404436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef381eaba0 con 0x7fef380cdfb0 2024-09-17T04:31:01.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.106+0000 7fef3e400640 1 -- 172.21.15.100:0/3589404436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef380b3980 con 0x7fef380cdfb0 2024-09-17T04:31:01.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.107+0000 7fef2f400640 1 -- 172.21.15.100:0/3589404436 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fef30004030 con 0x7fef380cdfb0 2024-09-17T04:31:01.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.108+0000 7fef2f400640 1 --2- 172.21.15.100:0/3589404436 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fef18079010 0x7fef1807b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:01.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.108+0000 7fef2f400640 1 -- 172.21.15.100:0/3589404436 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5443+0+0 (secure 0 0 0) 0x7fef300bb7a0 con 0x7fef380cdfb0 2024-09-17T04:31:01.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.108+0000 7fef37400640 1 --2- 172.21.15.100:0/3589404436 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fef18079010 0x7fef1807b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:01.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.109+0000 7fef37400640 1 --2- 172.21.15.100:0/3589404436 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fef18079010 0x7fef1807b520 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fef28006fd0 tx=0x7fef28006b10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:01.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.113+0000 7fef2f400640 1 -- 172.21.15.100:0/3589404436 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fef30082f10 con 0x7fef380cdfb0 2024-09-17T04:31:01.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.296+0000 7fef3e400640 1 -- 172.21.15.100:0/3589404436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fef380d1030 con 0x7fef380cdfb0 2024-09-17T04:31:01.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.849+0000 7fef2f400640 1 -- 172.21.15.100:0/3589404436 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v111) ==== 169+0+0 (secure 0 0 0) 0x7fef300bb020 con 0x7fef380cdfb0 2024-09-17T04:31:01.849 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' removed 2024-09-17T04:31:01.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.853+0000 7fef3e400640 1 -- 172.21.15.100:0/3589404436 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fef18079010 msgr2=0x7fef1807b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:01.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.853+0000 7fef3e400640 1 --2- 172.21.15.100:0/3589404436 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fef18079010 0x7fef1807b520 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fef28006fd0 tx=0x7fef28006b10 comp rx=0 tx=0).stop 2024-09-17T04:31:01.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.853+0000 7fef3e400640 1 -- 172.21.15.100:0/3589404436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef380cdfb0 msgr2=0x7fef380b0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:01.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.853+0000 7fef3e400640 1 --2- 172.21.15.100:0/3589404436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef380cdfb0 0x7fef380b0dc0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fef30036040 tx=0x7fef30033c10 comp rx=0 tx=0).stop 2024-09-17T04:31:01.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.853+0000 7fef3e400640 1 -- 172.21.15.100:0/3589404436 shutdown_connections 2024-09-17T04:31:01.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.853+0000 7fef3e400640 1 --2- 172.21.15.100:0/3589404436 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fef18079010 0x7fef1807b520 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:01.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.853+0000 7fef3e400640 1 --2- 172.21.15.100:0/3589404436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef380cdfb0 0x7fef380b0dc0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:01.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.853+0000 7fef3e400640 1 --2- 172.21.15.100:0/3589404436 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef381e90a0 0x7fef381531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:01.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.853+0000 7fef3e400640 1 --2- 172.21.15.100:0/3589404436 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef1c004d90 0x7fef381f6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:01.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.853+0000 7fef3e400640 1 -- 172.21.15.100:0/3589404436 >> 172.21.15.100:0/3589404436 conn(0x7fef380b2540 msgr2=0x7fef38074540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:01.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.854+0000 7fef3e400640 1 -- 172.21.15.100:0/3589404436 shutdown_connections 2024-09-17T04:31:01.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:01.854+0000 7fef3e400640 1 -- 172.21.15.100:0/3589404436 wait complete. 2024-09-17T04:31:01.879 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T04:31:02.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.102+0000 7f443b200640 1 -- 172.21.15.100:0/3063701723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44340cfc80 msgr2=0x7f4434156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:02.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.102+0000 7f443b200640 1 --2- 172.21.15.100:0/3063701723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44340cfc80 0x7f4434156e40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4424009860 tx=0x7f442402a4f0 comp rx=0 tx=0).stop 2024-09-17T04:31:02.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.102+0000 7f443b200640 1 -- 172.21.15.100:0/3063701723 shutdown_connections 2024-09-17T04:31:02.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.102+0000 7f443b200640 1 --2- 172.21.15.100:0/3063701723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44340cfc80 0x7f4434156e40 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:02.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.102+0000 7f443b200640 1 -- 172.21.15.100:0/3063701723 >> 172.21.15.100:0/3063701723 conn(0x7f443406d3d0 msgr2=0x7f44340b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:02.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.102+0000 7f443b200640 1 -- 172.21.15.100:0/3063701723 shutdown_connections 2024-09-17T04:31:02.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.102+0000 7f443b200640 1 -- 172.21.15.100:0/3063701723 wait complete. 2024-09-17T04:31:02.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.104+0000 7f443b200640 1 Processor -- start 2024-09-17T04:31:02.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.104+0000 7f443b200640 1 -- start start 2024-09-17T04:31:02.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.106+0000 7f443b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44340705b0 0x7f4434159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:02.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.106+0000 7f443b200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44340cfc80 0x7f44341f6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:02.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.106+0000 7f443b200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f443415ce30 0x7f4434074630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:02.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.106+0000 7f443b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f44340cd100 con 0x7f44340705b0 2024-09-17T04:31:02.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.106+0000 7f443b200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f44340ccf80 con 0x7f443415ce30 2024-09-17T04:31:02.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.106+0000 7f443b200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f44340cd280 con 0x7f44340cfc80 2024-09-17T04:31:02.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.106+0000 7f4433e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44340705b0 0x7f4434159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:02.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.106+0000 7f4433e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44340705b0 0x7f4434159ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53748/0 (socket says 172.21.15.100:53748) 2024-09-17T04:31:02.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.106+0000 7f4433e00640 1 -- 172.21.15.100:0/488887944 learned_addr learned my addr 172.21.15.100:0/488887944 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:02.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.106+0000 7f4433400640 1 --2- 172.21.15.100:0/488887944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44340cfc80 0x7f44341f6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:02.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.106+0000 7f4439200640 1 --2- 172.21.15.100:0/488887944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f443415ce30 0x7f4434074630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:02.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.106+0000 7f4433e00640 1 -- 172.21.15.100:0/488887944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44340cfc80 msgr2=0x7f44341f6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:02.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.106+0000 7f4433e00640 1 --2- 172.21.15.100:0/488887944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44340cfc80 0x7f44341f6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:02.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.106+0000 7f4433e00640 1 -- 172.21.15.100:0/488887944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f443415ce30 msgr2=0x7f4434074630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:02.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.106+0000 7f4433e00640 1 --2- 172.21.15.100:0/488887944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f443415ce30 0x7f4434074630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:02.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.106+0000 7f4433e00640 1 -- 172.21.15.100:0/488887944 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44341f6ed0 con 0x7f44340705b0 2024-09-17T04:31:02.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.106+0000 7f4433400640 1 --2- 172.21.15.100:0/488887944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44340cfc80 0x7f44341f6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:31:02.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.107+0000 7f4433e00640 1 --2- 172.21.15.100:0/488887944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44340705b0 0x7f4434159ea0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f442402aae0 tx=0x7f44240038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:02.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.107+0000 7f442be00640 1 -- 172.21.15.100:0/488887944 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4424003ea0 con 0x7f44340705b0 2024-09-17T04:31:02.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.107+0000 7f442be00640 1 -- 172.21.15.100:0/488887944 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4424005020 con 0x7f44340705b0 2024-09-17T04:31:02.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.107+0000 7f443b200640 1 -- 172.21.15.100:0/488887944 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4434073590 con 0x7f44340705b0 2024-09-17T04:31:02.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.107+0000 7f442be00640 1 -- 172.21.15.100:0/488887944 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4424005490 con 0x7f44340705b0 2024-09-17T04:31:02.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.107+0000 7f443b200640 1 -- 172.21.15.100:0/488887944 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4434073b50 con 0x7f44340705b0 2024-09-17T04:31:02.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.108+0000 7f443b200640 1 -- 172.21.15.100:0/488887944 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44340cd790 con 0x7f44340705b0 2024-09-17T04:31:02.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.109+0000 7f442be00640 1 -- 172.21.15.100:0/488887944 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f44240047a0 con 0x7f44340705b0 2024-09-17T04:31:02.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.111+0000 7f442be00640 1 --2- 172.21.15.100:0/488887944 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4414079040 0x7f441407b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:02.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.111+0000 7f442be00640 1 -- 172.21.15.100:0/488887944 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 4971+0+0 (secure 0 0 0) 0x7f44240bc000 con 0x7f44340705b0 2024-09-17T04:31:02.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.111+0000 7f4433400640 1 --2- 172.21.15.100:0/488887944 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4414079040 0x7f441407b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:02.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.116+0000 7f4433400640 1 --2- 172.21.15.100:0/488887944 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4414079040 0x7f441407b550 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4420007fd0 tx=0x7f4420009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:02.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.119+0000 7f442be00640 1 -- 172.21.15.100:0/488887944 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f442403f030 con 0x7f44340705b0 2024-09-17T04:31:02.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.309+0000 7f443b200640 1 -- 172.21.15.100:0/488887944 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f443415c380 con 0x7f44340705b0 2024-09-17T04:31:02.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.314+0000 7f442be00640 1 -- 172.21.15.100:0/488887944 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v43) ==== 71+0+0 (secure 0 0 0) 0x7f4424084030 con 0x7f44340705b0 2024-09-17T04:31:02.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.317+0000 7f443b200640 1 -- 172.21.15.100:0/488887944 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4414079040 msgr2=0x7f441407b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:02.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.317+0000 7f443b200640 1 --2- 172.21.15.100:0/488887944 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4414079040 0x7f441407b550 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4420007fd0 tx=0x7f4420009040 comp rx=0 tx=0).stop 2024-09-17T04:31:02.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.317+0000 7f443b200640 1 -- 172.21.15.100:0/488887944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44340705b0 msgr2=0x7f4434159ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:02.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.317+0000 7f443b200640 1 --2- 172.21.15.100:0/488887944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44340705b0 0x7f4434159ea0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f442402aae0 tx=0x7f44240038a0 comp rx=0 tx=0).stop 2024-09-17T04:31:02.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.317+0000 7f443b200640 1 -- 172.21.15.100:0/488887944 shutdown_connections 2024-09-17T04:31:02.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.317+0000 7f443b200640 1 --2- 172.21.15.100:0/488887944 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4414079040 0x7f441407b550 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:02.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.317+0000 7f443b200640 1 --2- 172.21.15.100:0/488887944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f443415ce30 0x7f4434074630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:02.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.317+0000 7f443b200640 1 --2- 172.21.15.100:0/488887944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44340cfc80 0x7f44341f6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:02.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.317+0000 7f443b200640 1 --2- 172.21.15.100:0/488887944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44340705b0 0x7f4434159ea0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:02.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.317+0000 7f443b200640 1 -- 172.21.15.100:0/488887944 >> 172.21.15.100:0/488887944 conn(0x7f443406d3d0 msgr2=0x7f443415f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:02.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.317+0000 7f443b200640 1 -- 172.21.15.100:0/488887944 shutdown_connections 2024-09-17T04:31:02.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.317+0000 7f443b200640 1 -- 172.21.15.100:0/488887944 wait complete. 2024-09-17T04:31:02.340 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2024-09-17T04:31:02.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.561+0000 7f0445400640 1 -- 172.21.15.100:0/157772318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0424004800 msgr2=0x7f0424004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:02.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.561+0000 7f0445400640 1 --2- 172.21.15.100:0/157772318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0424004800 0x7f0424004c10 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f0428006fd0 tx=0x7f0424005830 comp rx=0 tx=0).stop 2024-09-17T04:31:02.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.561+0000 7f043f200640 1 -- 172.21.15.100:0/157772318 reap_dead start 2024-09-17T04:31:02.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.561+0000 7f0445400640 1 -- 172.21.15.100:0/157772318 shutdown_connections 2024-09-17T04:31:02.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.561+0000 7f0445400640 1 -- 172.21.15.100:0/157772318 >> 172.21.15.100:0/157772318 conn(0x7f044006d730 msgr2=0x7f04400b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:02.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.561+0000 7f0445400640 1 -- 172.21.15.100:0/157772318 shutdown_connections 2024-09-17T04:31:02.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.561+0000 7f0445400640 1 -- 172.21.15.100:0/157772318 wait complete. 2024-09-17T04:31:02.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.563+0000 7f0445400640 1 Processor -- start 2024-09-17T04:31:02.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.563+0000 7f0445400640 1 -- start start 2024-09-17T04:31:02.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.565+0000 7f0445400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0424004800 0x7f04401f9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:02.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.565+0000 7f0445400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04400742c0 0x7f0440147770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:02.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.565+0000 7f0445400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f044015b7f0 0x7f044015bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:02.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.565+0000 7f0445400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f04400b3ea0 con 0x7f044015b7f0 2024-09-17T04:31:02.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.565+0000 7f0445400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f04400b4e30 con 0x7f0424004800 2024-09-17T04:31:02.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.565+0000 7f0445400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f04400b4020 con 0x7f04400742c0 2024-09-17T04:31:02.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.565+0000 7f043f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f044015b7f0 0x7f044015bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:02.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.565+0000 7f043f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f044015b7f0 0x7f044015bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53764/0 (socket says 172.21.15.100:53764) 2024-09-17T04:31:02.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.565+0000 7f043f200640 1 -- 172.21.15.100:0/953290794 learned_addr learned my addr 172.21.15.100:0/953290794 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:02.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.565+0000 7f043e800640 1 --2- 172.21.15.100:0/953290794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0424004800 0x7f04401f9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:02.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.565+0000 7f0435e00640 1 --2- 172.21.15.100:0/953290794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04400742c0 0x7f0440147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:02.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.565+0000 7f043f200640 1 -- 172.21.15.100:0/953290794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04400742c0 msgr2=0x7f0440147770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:02.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.565+0000 7f043f200640 1 --2- 172.21.15.100:0/953290794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04400742c0 0x7f0440147770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:02.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.565+0000 7f043f200640 1 -- 172.21.15.100:0/953290794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0424004800 msgr2=0x7f04401f9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:02.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.565+0000 7f043f200640 1 --2- 172.21.15.100:0/953290794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0424004800 0x7f04401f9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:02.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.565+0000 7f043f200640 1 -- 172.21.15.100:0/953290794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f044015c140 con 0x7f044015b7f0 2024-09-17T04:31:02.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.566+0000 7f043f200640 1 --2- 172.21.15.100:0/953290794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f044015b7f0 0x7f044015bc00 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f04400ae430 tx=0x7f0438011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:02.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.566+0000 7f0437400640 1 -- 172.21.15.100:0/953290794 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0438004230 con 0x7f044015b7f0 2024-09-17T04:31:02.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.566+0000 7f0437400640 1 -- 172.21.15.100:0/953290794 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04380043f0 con 0x7f044015b7f0 2024-09-17T04:31:02.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.566+0000 7f0437400640 1 -- 172.21.15.100:0/953290794 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043800fb50 con 0x7f044015b7f0 2024-09-17T04:31:02.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.566+0000 7f0445400640 1 -- 172.21.15.100:0/953290794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f044009c0d0 con 0x7f044015b7f0 2024-09-17T04:31:02.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.566+0000 7f0445400640 1 -- 172.21.15.100:0/953290794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04401f91a0 con 0x7f044015b7f0 2024-09-17T04:31:02.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.567+0000 7f0445400640 1 -- 172.21.15.100:0/953290794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04400b3960 con 0x7f044015b7f0 2024-09-17T04:31:02.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.568+0000 7f0437400640 1 -- 172.21.15.100:0/953290794 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f043800fcf0 con 0x7f044015b7f0 2024-09-17T04:31:02.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.569+0000 7f0437400640 1 --2- 172.21.15.100:0/953290794 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f041c079010 0x7f041c07b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:02.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.569+0000 7f0437400640 1 -- 172.21.15.100:0/953290794 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 4971+0+0 (secure 0 0 0) 0x7f043809bb60 con 0x7f044015b7f0 2024-09-17T04:31:02.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.570+0000 7f043e800640 1 --2- 172.21.15.100:0/953290794 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f041c079010 0x7f041c07b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:02.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.570+0000 7f043e800640 1 --2- 172.21.15.100:0/953290794 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f041c079010 0x7f041c07b520 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0430009b30 tx=0x7f0430005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:02.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.574+0000 7f0437400640 1 -- 172.21.15.100:0/953290794 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f044015c140 con 0x7f044015b7f0 2024-09-17T04:31:02.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.824+0000 7f0445400640 1 -- 172.21.15.100:0/953290794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) -- 0x7f044009be60 con 0x7f044015b7f0 2024-09-17T04:31:02.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.851+0000 7f0437400640 1 -- 172.21.15.100:0/953290794 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v384) ==== 149+0+0 (secure 0 0 0) 0x7f0438063a70 con 0x7f044015b7f0 2024-09-17T04:31:02.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.853+0000 7f0445400640 1 -- 172.21.15.100:0/953290794 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f041c079010 msgr2=0x7f041c07b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:02.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.853+0000 7f0445400640 1 --2- 172.21.15.100:0/953290794 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f041c079010 0x7f041c07b520 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0430009b30 tx=0x7f0430005f90 comp rx=0 tx=0).stop 2024-09-17T04:31:02.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.853+0000 7f0445400640 1 -- 172.21.15.100:0/953290794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f044015b7f0 msgr2=0x7f044015bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:02.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.853+0000 7f0445400640 1 --2- 172.21.15.100:0/953290794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f044015b7f0 0x7f044015bc00 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f04400ae430 tx=0x7f0438011ba0 comp rx=0 tx=0).stop 2024-09-17T04:31:02.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.853+0000 7f0445400640 1 -- 172.21.15.100:0/953290794 shutdown_connections 2024-09-17T04:31:02.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.853+0000 7f0445400640 1 --2- 172.21.15.100:0/953290794 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f041c079010 0x7f041c07b520 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:02.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.853+0000 7f0445400640 1 --2- 172.21.15.100:0/953290794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f044015b7f0 0x7f044015bc00 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:02.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.853+0000 7f0445400640 1 --2- 172.21.15.100:0/953290794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04400742c0 0x7f0440147770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:02.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.853+0000 7f0445400640 1 --2- 172.21.15.100:0/953290794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0424004800 0x7f04401f9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:02.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.853+0000 7f0445400640 1 -- 172.21.15.100:0/953290794 >> 172.21.15.100:0/953290794 conn(0x7f044006d730 msgr2=0x7f044015efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:02.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.853+0000 7f0445400640 1 -- 172.21.15.100:0/953290794 shutdown_connections 2024-09-17T04:31:02.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:02.853+0000 7f0445400640 1 -- 172.21.15.100:0/953290794 wait complete. 2024-09-17T04:31:02.876 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-09-17T04:31:02.876 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-09-17T04:31:02.876 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) 2024-09-17T04:31:02.877 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:31:03.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.098+0000 7f33aaa00640 1 -- 172.21.15.100:0/2911499350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a415d760 msgr2=0x7f33a41535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:03.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.098+0000 7f33aaa00640 1 --2- 172.21.15.100:0/2911499350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a415d760 0x7f33a41535c0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f33a4072d60 tx=0x7f339402a560 comp rx=0 tx=0).stop 2024-09-17T04:31:03.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.098+0000 7f33aaa00640 1 -- 172.21.15.100:0/2911499350 shutdown_connections 2024-09-17T04:31:03.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.098+0000 7f33aaa00640 1 --2- 172.21.15.100:0/2911499350 >> v2:172.21.15.184:3300/0 conn(0x7f33a40ad0e0 0x7f33a40ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:03.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.098+0000 7f33aaa00640 1 --2- 172.21.15.100:0/2911499350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a415d760 0x7f33a41535c0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:03.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.098+0000 7f33aaa00640 1 -- 172.21.15.100:0/2911499350 >> 172.21.15.100:0/2911499350 conn(0x7f33a406d730 msgr2=0x7f33a40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:03.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.098+0000 7f33aaa00640 1 -- 172.21.15.100:0/2911499350 shutdown_connections 2024-09-17T04:31:03.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.098+0000 7f33aaa00640 1 -- 172.21.15.100:0/2911499350 wait complete. 2024-09-17T04:31:03.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.101+0000 7f33aaa00640 1 Processor -- start 2024-09-17T04:31:03.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.101+0000 7f33aaa00640 1 -- start start 2024-09-17T04:31:03.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.102+0000 7f33aaa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33a40ad0e0 0x7f33a414e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:03.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.102+0000 7f33aaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a415d760 0x7f33a41416f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:03.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.102+0000 7f33aaa00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33a4160310 0x7f33a4154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:03.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.102+0000 7f33aaa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f33a40b4c20 con 0x7f33a415d760 2024-09-17T04:31:03.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.102+0000 7f33aaa00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f33a4153c70 con 0x7f33a40ad0e0 2024-09-17T04:31:03.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.102+0000 7f33aaa00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f33a40b4da0 con 0x7f33a4160310 2024-09-17T04:31:03.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.102+0000 7f33a3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a415d760 0x7f33a41416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:03.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.102+0000 7f33a3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a415d760 0x7f33a41416f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53790/0 (socket says 172.21.15.100:53790) 2024-09-17T04:31:03.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.102+0000 7f33a3400640 1 -- 172.21.15.100:0/2347843833 learned_addr learned my addr 172.21.15.100:0/2347843833 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:03.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.103+0000 7f33a3e00640 1 --2- 172.21.15.100:0/2347843833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33a40ad0e0 0x7f33a414e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:03.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.103+0000 7f33a8a00640 1 --2- 172.21.15.100:0/2347843833 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33a4160310 0x7f33a4154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:03.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.103+0000 7f33a3400640 1 -- 172.21.15.100:0/2347843833 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33a4160310 msgr2=0x7f33a4154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:03.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.103+0000 7f33a3400640 1 --2- 172.21.15.100:0/2347843833 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33a4160310 0x7f33a4154920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:03.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.103+0000 7f33a3400640 1 -- 172.21.15.100:0/2347843833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33a40ad0e0 msgr2=0x7f33a414e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:03.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.103+0000 7f33a3400640 1 --2- 172.21.15.100:0/2347843833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33a40ad0e0 0x7f33a414e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:03.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.103+0000 7f33a3400640 1 -- 172.21.15.100:0/2347843833 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a414b4e0 con 0x7f33a415d760 2024-09-17T04:31:03.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.103+0000 7f33a8a00640 1 --2- 172.21.15.100:0/2347843833 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33a4160310 0x7f33a4154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:31:03.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.103+0000 7f33a3400640 1 --2- 172.21.15.100:0/2347843833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a415d760 0x7f33a41416f0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f339000d750 tx=0x7f339000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:03.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.103+0000 7f339be00640 1 -- 172.21.15.100:0/2347843833 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3390017070 con 0x7f33a415d760 2024-09-17T04:31:03.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.104+0000 7f339be00640 1 -- 172.21.15.100:0/2347843833 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3390009b20 con 0x7f33a415d760 2024-09-17T04:31:03.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.104+0000 7f33aaa00640 1 -- 172.21.15.100:0/2347843833 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a414b7c0 con 0x7f33a415d760 2024-09-17T04:31:03.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.104+0000 7f339be00640 1 -- 172.21.15.100:0/2347843833 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3390010460 con 0x7f33a415d760 2024-09-17T04:31:03.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.104+0000 7f33aaa00640 1 -- 172.21.15.100:0/2347843833 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a4154120 con 0x7f33a415d760 2024-09-17T04:31:03.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.105+0000 7f33aaa00640 1 -- 172.21.15.100:0/2347843833 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33a414b250 con 0x7f33a415d760 2024-09-17T04:31:03.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.106+0000 7f339be00640 1 -- 172.21.15.100:0/2347843833 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f339001a070 con 0x7f33a415d760 2024-09-17T04:31:03.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.107+0000 7f339be00640 1 --2- 172.21.15.100:0/2347843833 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3384079010 0x7f338407b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:03.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.108+0000 7f339be00640 1 -- 172.21.15.100:0/2347843833 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 4971+0+0 (secure 0 0 0) 0x7f339009b490 con 0x7f33a415d760 2024-09-17T04:31:03.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.108+0000 7f33a3e00640 1 --2- 172.21.15.100:0/2347843833 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3384079010 0x7f338407b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:03.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.108+0000 7f33a3e00640 1 --2- 172.21.15.100:0/2347843833 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3384079010 0x7f338407b520 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3394033000 tx=0x7f3394004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:03.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.112+0000 7f339be00640 1 -- 172.21.15.100:0/2347843833 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3390062db0 con 0x7f33a415d760 2024-09-17T04:31:03.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.297+0000 7f33aaa00640 1 -- 172.21.15.100:0/2347843833 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f33a4160ff0 con 0x7f33a415d760 2024-09-17T04:31:03.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.298+0000 7f339be00640 1 -- 172.21.15.100:0/2347843833 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+13136 (secure 0 0 0) 0x7f33900677e0 con 0x7f33a415d760 2024-09-17T04:31:03.298 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:03.298 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":111,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:31:01.844136+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6833/2666467995":"2024-09-18T04:30:58.793717+0000","172.21.15.100:6832/2666467995":"2024-09-18T04:30:58.793717+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:31:03.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.300+0000 7f33aaa00640 1 -- 172.21.15.100:0/2347843833 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3384079010 msgr2=0x7f338407b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:03.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.300+0000 7f33aaa00640 1 --2- 172.21.15.100:0/2347843833 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3384079010 0x7f338407b520 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3394033000 tx=0x7f3394004730 comp rx=0 tx=0).stop 2024-09-17T04:31:03.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.300+0000 7f33aaa00640 1 -- 172.21.15.100:0/2347843833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a415d760 msgr2=0x7f33a41416f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:03.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.300+0000 7f33aaa00640 1 --2- 172.21.15.100:0/2347843833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a415d760 0x7f33a41416f0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f339000d750 tx=0x7f339000dc10 comp rx=0 tx=0).stop 2024-09-17T04:31:03.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.301+0000 7f33aaa00640 1 -- 172.21.15.100:0/2347843833 shutdown_connections 2024-09-17T04:31:03.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.301+0000 7f33aaa00640 1 --2- 172.21.15.100:0/2347843833 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3384079010 0x7f338407b520 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:03.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.301+0000 7f33aaa00640 1 --2- 172.21.15.100:0/2347843833 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33a4160310 0x7f33a4154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:03.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.301+0000 7f33aaa00640 1 --2- 172.21.15.100:0/2347843833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a415d760 0x7f33a41416f0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:03.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.301+0000 7f33aaa00640 1 --2- 172.21.15.100:0/2347843833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33a40ad0e0 0x7f33a414e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:03.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.301+0000 7f33aaa00640 1 -- 172.21.15.100:0/2347843833 >> 172.21.15.100:0/2347843833 conn(0x7f33a406d730 msgr2=0x7f33a40cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:03.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.301+0000 7f33aaa00640 1 -- 172.21.15.100:0/2347843833 shutdown_connections 2024-09-17T04:31:03.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.301+0000 7f33aaa00640 1 -- 172.21.15.100:0/2347843833 wait complete. 2024-09-17T04:31:03.323 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T04:24:03.995355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T04:31:03.323 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T04:31:03.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.543+0000 7fb38d800640 1 -- 172.21.15.100:0/2221125089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3880ad3d0 msgr2=0x7fb3880ad7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:03.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.543+0000 7fb38d800640 1 --2- 172.21.15.100:0/2221125089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3880ad3d0 0x7fb3880ad7e0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fb388072790 tx=0x7fb38002aa30 comp rx=0 tx=0).stop 2024-09-17T04:31:03.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.543+0000 7fb38d800640 1 -- 172.21.15.100:0/2221125089 shutdown_connections 2024-09-17T04:31:03.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.543+0000 7fb38d800640 1 --2- 172.21.15.100:0/2221125089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3880ad3d0 0x7fb3880ad7e0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:03.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.543+0000 7fb38d800640 1 --2- 172.21.15.100:0/2221125089 >> v2:172.21.15.184:3300/0 conn(0x7fb3880b4f00 0x7fb3880704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:03.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.543+0000 7fb38d800640 1 -- 172.21.15.100:0/2221125089 >> 172.21.15.100:0/2221125089 conn(0x7fb38806d710 msgr2=0x7fb3880b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:03.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.543+0000 7fb38d800640 1 -- 172.21.15.100:0/2221125089 shutdown_connections 2024-09-17T04:31:03.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.543+0000 7fb38d800640 1 -- 172.21.15.100:0/2221125089 wait complete. 2024-09-17T04:31:03.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.545+0000 7fb38d800640 1 Processor -- start 2024-09-17T04:31:03.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.546+0000 7fb38d800640 1 -- start start 2024-09-17T04:31:03.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.547+0000 7fb38d800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3880ad3d0 0x7fb38815b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:03.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.547+0000 7fb38d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3880b4f00 0x7fb3881507b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:03.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.547+0000 7fb38d800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb388160490 0x7fb388150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:03.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.547+0000 7fb38d800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb38806f0f0 con 0x7fb3880b4f00 2024-09-17T04:31:03.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.547+0000 7fb38d800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb38806ef70 con 0x7fb388160490 2024-09-17T04:31:03.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.547+0000 7fb38d800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb38806f270 con 0x7fb3880ad3d0 2024-09-17T04:31:03.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.547+0000 7fb386000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3880b4f00 0x7fb3881507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:03.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.547+0000 7fb386000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3880b4f00 0x7fb3881507b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53804/0 (socket says 172.21.15.100:53804) 2024-09-17T04:31:03.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.547+0000 7fb386000640 1 -- 172.21.15.100:0/4194601707 learned_addr learned my addr 172.21.15.100:0/4194601707 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:03.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.547+0000 7fb387400640 1 --2- 172.21.15.100:0/4194601707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb388160490 0x7fb388150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:03.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.547+0000 7fb386a00640 1 --2- 172.21.15.100:0/4194601707 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3880ad3d0 0x7fb38815b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:03.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.548+0000 7fb386000640 1 -- 172.21.15.100:0/4194601707 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3880ad3d0 msgr2=0x7fb38815b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:03.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.548+0000 7fb386000640 1 --2- 172.21.15.100:0/4194601707 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3880ad3d0 0x7fb38815b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:03.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.548+0000 7fb386000640 1 -- 172.21.15.100:0/4194601707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb388160490 msgr2=0x7fb388150cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:03.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.548+0000 7fb386000640 1 --2- 172.21.15.100:0/4194601707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb388160490 0x7fb388150cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:03.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.548+0000 7fb386000640 1 -- 172.21.15.100:0/4194601707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb38815c250 con 0x7fb3880b4f00 2024-09-17T04:31:03.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.548+0000 7fb386000640 1 --2- 172.21.15.100:0/4194601707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3880b4f00 0x7fb3881507b0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fb37400b3f0 tx=0x7fb37400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:03.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.548+0000 7fb37ea00640 1 -- 172.21.15.100:0/4194601707 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb374015020 con 0x7fb3880b4f00 2024-09-17T04:31:03.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.548+0000 7fb37ea00640 1 -- 172.21.15.100:0/4194601707 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb374011920 con 0x7fb3880b4f00 2024-09-17T04:31:03.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.548+0000 7fb37ea00640 1 -- 172.21.15.100:0/4194601707 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb374011d70 con 0x7fb3880b4f00 2024-09-17T04:31:03.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.548+0000 7fb38d800640 1 -- 172.21.15.100:0/4194601707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3881f5850 con 0x7fb3880b4f00 2024-09-17T04:31:03.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.549+0000 7fb38d800640 1 -- 172.21.15.100:0/4194601707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3881422d0 con 0x7fb3880b4f00 2024-09-17T04:31:03.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.549+0000 7fb38d800640 1 -- 172.21.15.100:0/4194601707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb38815bfc0 con 0x7fb3880b4f00 2024-09-17T04:31:03.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.551+0000 7fb37ea00640 1 -- 172.21.15.100:0/4194601707 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fb374018020 con 0x7fb3880b4f00 2024-09-17T04:31:03.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.552+0000 7fb37ea00640 1 --2- 172.21.15.100:0/4194601707 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb368079010 0x7fb36807b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:03.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.552+0000 7fb37ea00640 1 -- 172.21.15.100:0/4194601707 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 4971+0+0 (secure 0 0 0) 0x7fb37409aeb0 con 0x7fb3880b4f00 2024-09-17T04:31:03.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.552+0000 7fb386a00640 1 --2- 172.21.15.100:0/4194601707 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb368079010 0x7fb36807b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:03.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.553+0000 7fb386a00640 1 --2- 172.21.15.100:0/4194601707 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb368079010 0x7fb36807b520 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb378004630 tx=0x7fb37800b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:03.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.556+0000 7fb37ea00640 1 -- 172.21.15.100:0/4194601707 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb37401f030 con 0x7fb3880b4f00 2024-09-17T04:31:03.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.739+0000 7fb38d800640 1 -- 172.21.15.100:0/4194601707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb38806f7b0 con 0x7fb3880b4f00 2024-09-17T04:31:03.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.739+0000 7fb37ea00640 1 -- 172.21.15.100:0/4194601707 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v111) ==== 93+0+10 (secure 0 0 0) 0x7fb374062800 con 0x7fb3880b4f00 2024-09-17T04:31:03.739 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-09-17T04:31:03.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.741+0000 7fb38d800640 1 -- 172.21.15.100:0/4194601707 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb368079010 msgr2=0x7fb36807b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:03.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.741+0000 7fb38d800640 1 --2- 172.21.15.100:0/4194601707 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb368079010 0x7fb36807b520 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb378004630 tx=0x7fb37800b2e0 comp rx=0 tx=0).stop 2024-09-17T04:31:03.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.741+0000 7fb38d800640 1 -- 172.21.15.100:0/4194601707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3880b4f00 msgr2=0x7fb3881507b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:03.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.741+0000 7fb38d800640 1 --2- 172.21.15.100:0/4194601707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3880b4f00 0x7fb3881507b0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fb37400b3f0 tx=0x7fb37400b8b0 comp rx=0 tx=0).stop 2024-09-17T04:31:03.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.741+0000 7fb38d800640 1 -- 172.21.15.100:0/4194601707 shutdown_connections 2024-09-17T04:31:03.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.741+0000 7fb38d800640 1 --2- 172.21.15.100:0/4194601707 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb368079010 0x7fb36807b520 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:03.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.741+0000 7fb38d800640 1 --2- 172.21.15.100:0/4194601707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb388160490 0x7fb388150cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:03.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.741+0000 7fb38d800640 1 --2- 172.21.15.100:0/4194601707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3880b4f00 0x7fb3881507b0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:03.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.741+0000 7fb38d800640 1 --2- 172.21.15.100:0/4194601707 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3880ad3d0 0x7fb38815b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:03.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.741+0000 7fb38d800640 1 -- 172.21.15.100:0/4194601707 >> 172.21.15.100:0/4194601707 conn(0x7fb38806d710 msgr2=0x7fb38815df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:03.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.741+0000 7fb38d800640 1 -- 172.21.15.100:0/4194601707 shutdown_connections 2024-09-17T04:31:03.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.742+0000 7fb38d800640 1 -- 172.21.15.100:0/4194601707 wait complete. 2024-09-17T04:31:03.763 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2' 2024-09-17T04:31:03.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.987+0000 7fbf45e00640 1 -- 172.21.15.100:0/802441546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf24003a30 msgr2=0x7fbf24003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:03.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.987+0000 7fbf45e00640 1 --2- 172.21.15.100:0/802441546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf24003a30 0x7fbf24003ea0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009ed0 tx=0x7fbf24005830 comp rx=0 tx=0).stop 2024-09-17T04:31:03.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.988+0000 7fbf3f400640 1 -- 172.21.15.100:0/802441546 reap_dead start 2024-09-17T04:31:03.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.988+0000 7fbf45e00640 1 -- 172.21.15.100:0/802441546 shutdown_connections 2024-09-17T04:31:03.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.988+0000 7fbf45e00640 1 -- 172.21.15.100:0/802441546 >> 172.21.15.100:0/802441546 conn(0x7fbf4006d730 msgr2=0x7fbf400b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:03.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.988+0000 7fbf45e00640 1 -- 172.21.15.100:0/802441546 shutdown_connections 2024-09-17T04:31:03.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.988+0000 7fbf45e00640 1 -- 172.21.15.100:0/802441546 wait complete. 2024-09-17T04:31:03.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.990+0000 7fbf45e00640 1 Processor -- start 2024-09-17T04:31:03.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.990+0000 7fbf45e00640 1 -- start start 2024-09-17T04:31:03.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.992+0000 7fbf45e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf24003a30 0x7fbf4014c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:03.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.992+0000 7fbf45e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf4014cc20 0x7fbf4014d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:03.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.992+0000 7fbf45e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf40074050 0x7fbf40074460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:03.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.992+0000 7fbf45e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbf400b41f0 con 0x7fbf40074050 2024-09-17T04:31:03.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.992+0000 7fbf45e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbf400b4070 con 0x7fbf4014cc20 2024-09-17T04:31:03.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.992+0000 7fbf45e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbf400b4370 con 0x7fbf24003a30 2024-09-17T04:31:03.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.992+0000 7fbf3f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf40074050 0x7fbf40074460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:03.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.992+0000 7fbf3f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf40074050 0x7fbf40074460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53826/0 (socket says 172.21.15.100:53826) 2024-09-17T04:31:03.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.992+0000 7fbf3f400640 1 -- 172.21.15.100:0/182839752 learned_addr learned my addr 172.21.15.100:0/182839752 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:03.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.992+0000 7fbf3e000640 1 --2- 172.21.15.100:0/182839752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf4014cc20 0x7fbf4014d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:03.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.992+0000 7fbf3ea00640 1 --2- 172.21.15.100:0/182839752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf24003a30 0x7fbf4014c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:03.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.992+0000 7fbf3f400640 1 -- 172.21.15.100:0/182839752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf24003a30 msgr2=0x7fbf4014c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:03.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.992+0000 7fbf3f400640 1 --2- 172.21.15.100:0/182839752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf24003a30 0x7fbf4014c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:03.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.992+0000 7fbf3f400640 1 -- 172.21.15.100:0/182839752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf4014cc20 msgr2=0x7fbf4014d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:03.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.992+0000 7fbf3f400640 1 --2- 172.21.15.100:0/182839752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf4014cc20 0x7fbf4014d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:03.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.992+0000 7fbf3f400640 1 -- 172.21.15.100:0/182839752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf4014d570 con 0x7fbf40074050 2024-09-17T04:31:03.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.993+0000 7fbf3f400640 1 --2- 172.21.15.100:0/182839752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf40074050 0x7fbf40074460 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fbf40075250 tx=0x7fbf3800e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:03.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.993+0000 7fbf37400640 1 -- 172.21.15.100:0/182839752 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf38011870 con 0x7fbf40074050 2024-09-17T04:31:03.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.993+0000 7fbf37400640 1 -- 172.21.15.100:0/182839752 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf38017070 con 0x7fbf40074050 2024-09-17T04:31:03.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.993+0000 7fbf37400640 1 -- 172.21.15.100:0/182839752 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf38011da0 con 0x7fbf40074050 2024-09-17T04:31:03.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.993+0000 7fbf45e00640 1 -- 172.21.15.100:0/182839752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf400749a0 con 0x7fbf40074050 2024-09-17T04:31:03.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.993+0000 7fbf45e00640 1 -- 172.21.15.100:0/182839752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf4014f290 con 0x7fbf40074050 2024-09-17T04:31:03.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.995+0000 7fbf45e00640 1 -- 172.21.15.100:0/182839752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf4014ed50 con 0x7fbf40074050 2024-09-17T04:31:03.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.996+0000 7fbf37400640 1 -- 172.21.15.100:0/182839752 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fbf3801c460 con 0x7fbf40074050 2024-09-17T04:31:03.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.996+0000 7fbf37400640 1 --2- 172.21.15.100:0/182839752 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbf20079010 0x7fbf2007b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:03.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.997+0000 7fbf37400640 1 -- 172.21.15.100:0/182839752 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 4971+0+0 (secure 0 0 0) 0x7fbf3809b460 con 0x7fbf40074050 2024-09-17T04:31:03.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.997+0000 7fbf3ea00640 1 --2- 172.21.15.100:0/182839752 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbf20079010 0x7fbf2007b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:03.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:03.998+0000 7fbf3ea00640 1 --2- 172.21.15.100:0/182839752 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbf20079010 0x7fbf2007b520 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbf30035000 tx=0x7fbf3002d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:04.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:04.001+0000 7fbf37400640 1 -- 172.21.15.100:0/182839752 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fbf400749a0 con 0x7fbf40074050 2024-09-17T04:31:04.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:04.272+0000 7fbf45e00640 1 -- 172.21.15.100:0/182839752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]} v 0) -- 0x7fbf4014c810 con 0x7fbf40074050 2024-09-17T04:31:04.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:04.859+0000 7fbf37400640 1 -- 172.21.15.100:0/182839752 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]}]=0 v386) ==== 151+0+0 (secure 0 0 0) 0x7fbf4014d570 con 0x7fbf40074050 2024-09-17T04:31:04.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:04.863+0000 7fbf45e00640 1 -- 172.21.15.100:0/182839752 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbf20079010 msgr2=0x7fbf2007b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:04.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:04.863+0000 7fbf45e00640 1 --2- 172.21.15.100:0/182839752 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbf20079010 0x7fbf2007b520 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbf30035000 tx=0x7fbf3002d040 comp rx=0 tx=0).stop 2024-09-17T04:31:04.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:04.863+0000 7fbf45e00640 1 -- 172.21.15.100:0/182839752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf40074050 msgr2=0x7fbf40074460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:04.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:04.863+0000 7fbf45e00640 1 --2- 172.21.15.100:0/182839752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf40074050 0x7fbf40074460 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fbf40075250 tx=0x7fbf3800e770 comp rx=0 tx=0).stop 2024-09-17T04:31:04.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:04.863+0000 7fbf45e00640 1 -- 172.21.15.100:0/182839752 shutdown_connections 2024-09-17T04:31:04.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:04.863+0000 7fbf45e00640 1 --2- 172.21.15.100:0/182839752 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbf20079010 0x7fbf2007b520 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:04.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:04.863+0000 7fbf45e00640 1 --2- 172.21.15.100:0/182839752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf40074050 0x7fbf40074460 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:04.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:04.863+0000 7fbf45e00640 1 --2- 172.21.15.100:0/182839752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf4014cc20 0x7fbf4014d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:04.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:04.863+0000 7fbf45e00640 1 --2- 172.21.15.100:0/182839752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf24003a30 0x7fbf4014c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:04.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:04.863+0000 7fbf45e00640 1 -- 172.21.15.100:0/182839752 >> 172.21.15.100:0/182839752 conn(0x7fbf4006d730 msgr2=0x7fbf4015bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:04.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:04.863+0000 7fbf45e00640 1 -- 172.21.15.100:0/182839752 shutdown_connections 2024-09-17T04:31:04.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:04.863+0000 7fbf45e00640 1 -- 172.21.15.100:0/182839752 wait complete. 2024-09-17T04:31:04.888 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T04:31:05.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.108+0000 7f1391000640 1 -- 172.21.15.100:0/3973835155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f138c0ad0e0 msgr2=0x7f138c0ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:05.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.108+0000 7f1391000640 1 --2- 172.21.15.100:0/3973835155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f138c0ad0e0 0x7f138c0ad4f0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f138c0b3100 tx=0x7f137802a4d0 comp rx=0 tx=0).stop 2024-09-17T04:31:05.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.108+0000 7f1391000640 1 -- 172.21.15.100:0/3973835155 shutdown_connections 2024-09-17T04:31:05.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.109+0000 7f1391000640 1 --2- 172.21.15.100:0/3973835155 >> v2:172.21.15.184:3300/0 conn(0x7f138c0b3550 0x7f138c0b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:05.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.109+0000 7f1391000640 1 --2- 172.21.15.100:0/3973835155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f138c0ad0e0 0x7f138c0ad4f0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:05.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.109+0000 7f1391000640 1 -- 172.21.15.100:0/3973835155 >> 172.21.15.100:0/3973835155 conn(0x7f138c06d730 msgr2=0x7f138c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:05.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.109+0000 7f1391000640 1 -- 172.21.15.100:0/3973835155 shutdown_connections 2024-09-17T04:31:05.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.109+0000 7f1391000640 1 -- 172.21.15.100:0/3973835155 wait complete. 2024-09-17T04:31:05.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.111+0000 7f1391000640 1 Processor -- start 2024-09-17T04:31:05.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.111+0000 7f1391000640 1 -- start start 2024-09-17T04:31:05.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.112+0000 7f1391000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f138c0ad0e0 0x7f138c074bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:05.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.112+0000 7f1391000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f138c0b3550 0x7f138c0ce920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:05.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.112+0000 7f1391000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f138c15d760 0x7f138c15b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:05.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.112+0000 7f1391000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f138c0b4da0 con 0x7f138c0ad0e0 2024-09-17T04:31:05.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.112+0000 7f1391000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f138c0b4c20 con 0x7f138c0b3550 2024-09-17T04:31:05.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.112+0000 7f1391000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f138c0b3ea0 con 0x7f138c15d760 2024-09-17T04:31:05.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.113+0000 7f1389e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f138c0ad0e0 0x7f138c074bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:05.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.113+0000 7f1389e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f138c0ad0e0 0x7f138c074bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56958/0 (socket says 172.21.15.100:56958) 2024-09-17T04:31:05.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.113+0000 7f1389e00640 1 -- 172.21.15.100:0/3481722790 learned_addr learned my addr 172.21.15.100:0/3481722790 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:05.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.113+0000 7f1389400640 1 --2- 172.21.15.100:0/3481722790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f138c0b3550 0x7f138c0ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:05.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.113+0000 7f138a800640 1 --2- 172.21.15.100:0/3481722790 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f138c15d760 0x7f138c15b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:05.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.113+0000 7f1389e00640 1 -- 172.21.15.100:0/3481722790 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f138c15d760 msgr2=0x7f138c15b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:05.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.113+0000 7f1389e00640 1 --2- 172.21.15.100:0/3481722790 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f138c15d760 0x7f138c15b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:05.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.113+0000 7f1389e00640 1 -- 172.21.15.100:0/3481722790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f138c0b3550 msgr2=0x7f138c0ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:05.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.113+0000 7f1389e00640 1 --2- 172.21.15.100:0/3481722790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f138c0b3550 0x7f138c0ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:05.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.113+0000 7f1389e00640 1 -- 172.21.15.100:0/3481722790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138c15a120 con 0x7f138c0ad0e0 2024-09-17T04:31:05.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.113+0000 7f138a800640 1 --2- 172.21.15.100:0/3481722790 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f138c15d760 0x7f138c15b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:31:05.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.113+0000 7f1389e00640 1 --2- 172.21.15.100:0/3481722790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f138c0ad0e0 0x7f138c074bb0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f137c00b3f0 tx=0x7f137c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:05.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.114+0000 7f1382000640 1 -- 172.21.15.100:0/3481722790 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c015020 con 0x7f138c0ad0e0 2024-09-17T04:31:05.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.114+0000 7f1382000640 1 -- 172.21.15.100:0/3481722790 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f137c011920 con 0x7f138c0ad0e0 2024-09-17T04:31:05.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.114+0000 7f1391000640 1 -- 172.21.15.100:0/3481722790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f138c1f5f00 con 0x7f138c0ad0e0 2024-09-17T04:31:05.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.114+0000 7f1382000640 1 -- 172.21.15.100:0/3481722790 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c011d70 con 0x7f138c0ad0e0 2024-09-17T04:31:05.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.114+0000 7f1391000640 1 -- 172.21.15.100:0/3481722790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f138c1f64c0 con 0x7f138c0ad0e0 2024-09-17T04:31:05.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.115+0000 7f1391000640 1 -- 172.21.15.100:0/3481722790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f138c0b43a0 con 0x7f138c0ad0e0 2024-09-17T04:31:05.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.116+0000 7f1382000640 1 -- 172.21.15.100:0/3481722790 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f137c018020 con 0x7f138c0ad0e0 2024-09-17T04:31:05.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.117+0000 7f1382000640 1 --2- 172.21.15.100:0/3481722790 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f136c079010 0x7f136c07b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:05.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.117+0000 7f1382000640 1 -- 172.21.15.100:0/3481722790 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 4971+0+0 (secure 0 0 0) 0x7f137c09afd0 con 0x7f138c0ad0e0 2024-09-17T04:31:05.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.117+0000 7f1389400640 1 --2- 172.21.15.100:0/3481722790 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f136c079010 0x7f136c07b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:05.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.118+0000 7f1389400640 1 --2- 172.21.15.100:0/3481722790 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f136c079010 0x7f136c07b520 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1378033000 tx=0x7f1378004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:05.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.121+0000 7f1382000640 1 -- 172.21.15.100:0/3481722790 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f137c01f030 con 0x7f138c0ad0e0 2024-09-17T04:31:05.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.304+0000 7f1391000640 1 -- 172.21.15.100:0/3481722790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f138c0b4690 con 0x7f138c0ad0e0 2024-09-17T04:31:05.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.304+0000 7f1382000640 1 -- 172.21.15.100:0/3481722790 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v43) ==== 59+0+2294 (secure 0 0 0) 0x7f137c062f70 con 0x7f138c0ad0e0 2024-09-17T04:31:05.304 INFO:teuthology.orchestra.run.smithi100.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T04:31:05.304 INFO:teuthology.orchestra.run.smithi100.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T04:31:05.304 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced client_mount_timeout 30 2024-09-17T04:31:05.304 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_client 20/20 2024-09-17T04:31:05.304 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_mgr 20/20 2024-09-17T04:31:05.304 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_ms 1/1 2024-09-17T04:31:05.304 INFO:teuthology.orchestra.run.smithi100.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T04:31:05.304 INFO:teuthology.orchestra.run.smithi100.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T04:31:05.304 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3938.598654 2024-09-17T04:31:05.304 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4008.640154 2024-09-17T04:31:05.304 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3706.588546 2024-09-17T04:31:05.305 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 6299.338665 2024-09-17T04:31:05.305 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3633.876699 2024-09-17T04:31:05.305 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3674.821841 2024-09-17T04:31:05.305 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3789.849984 2024-09-17T04:31:05.305 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3467.457298 2024-09-17T04:31:05.305 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_mds 20/20 2024-09-17T04:31:05.305 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T04:31:05.305 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_ms 2/2 2024-09-17T04:31:05.305 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_debug_frag true 2024-09-17T04:31:05.305 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_debug_scatterstat true 2024-09-17T04:31:05.305 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T04:31:05.305 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_verify_scatter true 2024-09-17T04:31:05.305 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T04:31:05.306 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T04:31:05.306 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T04:31:05.306 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced client_mount_timeout 600 2024-09-17T04:31:05.306 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced debug_client 20/20 2024-09-17T04:31:05.306 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced debug_ms 1/1 2024-09-17T04:31:05.306 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T04:31:05.306 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T04:31:05.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.307+0000 7f1391000640 1 -- 172.21.15.100:0/3481722790 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f136c079010 msgr2=0x7f136c07b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:05.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.307+0000 7f1391000640 1 --2- 172.21.15.100:0/3481722790 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f136c079010 0x7f136c07b520 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1378033000 tx=0x7f1378004730 comp rx=0 tx=0).stop 2024-09-17T04:31:05.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.307+0000 7f1391000640 1 -- 172.21.15.100:0/3481722790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f138c0ad0e0 msgr2=0x7f138c074bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:05.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.307+0000 7f1391000640 1 --2- 172.21.15.100:0/3481722790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f138c0ad0e0 0x7f138c074bb0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f137c00b3f0 tx=0x7f137c00b8b0 comp rx=0 tx=0).stop 2024-09-17T04:31:05.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.307+0000 7f1391000640 1 -- 172.21.15.100:0/3481722790 shutdown_connections 2024-09-17T04:31:05.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.307+0000 7f1391000640 1 --2- 172.21.15.100:0/3481722790 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f136c079010 0x7f136c07b520 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:05.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.307+0000 7f1391000640 1 --2- 172.21.15.100:0/3481722790 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f138c15d760 0x7f138c15b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:05.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.307+0000 7f1391000640 1 --2- 172.21.15.100:0/3481722790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f138c0b3550 0x7f138c0ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:05.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.307+0000 7f1391000640 1 --2- 172.21.15.100:0/3481722790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f138c0ad0e0 0x7f138c074bb0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:05.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.307+0000 7f1391000640 1 -- 172.21.15.100:0/3481722790 >> 172.21.15.100:0/3481722790 conn(0x7f138c06d730 msgr2=0x7f138c15f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:05.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.307+0000 7f1391000640 1 -- 172.21.15.100:0/3481722790 shutdown_connections 2024-09-17T04:31:05.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.307+0000 7f1391000640 1 -- 172.21.15.100:0/3481722790 wait complete. 2024-09-17T04:31:05.332 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T04:31:05.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.556+0000 7efc65200640 1 -- 172.21.15.100:0/3762456475 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc60145550 msgr2=0x7efc6006fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:05.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.556+0000 7efc65200640 1 --2- 172.21.15.100:0/3762456475 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc60145550 0x7efc6006fb40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efc50009830 tx=0x7efc5002a530 comp rx=0 tx=0).stop 2024-09-17T04:31:05.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.556+0000 7efc65200640 1 -- 172.21.15.100:0/3762456475 shutdown_connections 2024-09-17T04:31:05.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.556+0000 7efc65200640 1 --2- 172.21.15.100:0/3762456475 >> v2:172.21.15.184:3300/0 conn(0x7efc60145a40 0x7efc60073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:05.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.556+0000 7efc65200640 1 --2- 172.21.15.100:0/3762456475 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc60145550 0x7efc6006fb40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:05.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.556+0000 7efc65200640 1 -- 172.21.15.100:0/3762456475 >> 172.21.15.100:0/3762456475 conn(0x7efc6006d1c0 msgr2=0x7efc6006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:05.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.556+0000 7efc65200640 1 -- 172.21.15.100:0/3762456475 shutdown_connections 2024-09-17T04:31:05.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.556+0000 7efc65200640 1 -- 172.21.15.100:0/3762456475 wait complete. 2024-09-17T04:31:05.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.558+0000 7efc65200640 1 Processor -- start 2024-09-17T04:31:05.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.558+0000 7efc65200640 1 -- start start 2024-09-17T04:31:05.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.560+0000 7efc65200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc60073a80 0x7efc6013d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:05.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.560+0000 7efc65200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc60145550 0x7efc601f5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:05.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.560+0000 7efc65200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc60145a40 0x7efc6015fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:05.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.560+0000 7efc65200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efc60160a30 con 0x7efc60145550 2024-09-17T04:31:05.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.560+0000 7efc65200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efc600ce920 con 0x7efc60073a80 2024-09-17T04:31:05.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.560+0000 7efc65200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efc60160bb0 con 0x7efc60145a40 2024-09-17T04:31:05.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.560+0000 7efc55400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc60145550 0x7efc601f5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:05.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.560+0000 7efc55400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc60145550 0x7efc601f5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56972/0 (socket says 172.21.15.100:56972) 2024-09-17T04:31:05.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.560+0000 7efc55400640 1 -- 172.21.15.100:0/4230328285 learned_addr learned my addr 172.21.15.100:0/4230328285 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:05.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.560+0000 7efc5de00640 1 --2- 172.21.15.100:0/4230328285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc60073a80 0x7efc6013d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:05.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.560+0000 7efc5e800640 1 --2- 172.21.15.100:0/4230328285 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc60145a40 0x7efc6015fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:05.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.560+0000 7efc55400640 1 -- 172.21.15.100:0/4230328285 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc60145a40 msgr2=0x7efc6015fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:05.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.560+0000 7efc55400640 1 --2- 172.21.15.100:0/4230328285 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc60145a40 0x7efc6015fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:05.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.560+0000 7efc55400640 1 -- 172.21.15.100:0/4230328285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc60073a80 msgr2=0x7efc6013d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:05.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.560+0000 7efc55400640 1 --2- 172.21.15.100:0/4230328285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc60073a80 0x7efc6013d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:05.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.560+0000 7efc55400640 1 -- 172.21.15.100:0/4230328285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6013d760 con 0x7efc60145550 2024-09-17T04:31:05.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.561+0000 7efc55400640 1 --2- 172.21.15.100:0/4230328285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc60145550 0x7efc601f5980 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7efc4800e6d0 tx=0x7efc4800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:05.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.561+0000 7efc56a00640 1 -- 172.21.15.100:0/4230328285 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4800cab0 con 0x7efc60145550 2024-09-17T04:31:05.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.561+0000 7efc56a00640 1 -- 172.21.15.100:0/4230328285 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc4800cd40 con 0x7efc60145550 2024-09-17T04:31:05.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.561+0000 7efc65200640 1 -- 172.21.15.100:0/4230328285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc6013db00 con 0x7efc60145550 2024-09-17T04:31:05.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.561+0000 7efc56a00640 1 -- 172.21.15.100:0/4230328285 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48010690 con 0x7efc60145550 2024-09-17T04:31:05.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.561+0000 7efc65200640 1 -- 172.21.15.100:0/4230328285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc601f6770 con 0x7efc60145550 2024-09-17T04:31:05.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.562+0000 7efc65200640 1 -- 172.21.15.100:0/4230328285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc601f69b0 con 0x7efc60145550 2024-09-17T04:31:05.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.563+0000 7efc56a00640 1 -- 172.21.15.100:0/4230328285 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7efc48010830 con 0x7efc60145550 2024-09-17T04:31:05.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.564+0000 7efc56a00640 1 --2- 172.21.15.100:0/4230328285 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7efc3c079010 0x7efc3c07b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:05.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.564+0000 7efc56a00640 1 -- 172.21.15.100:0/4230328285 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 4971+0+0 (secure 0 0 0) 0x7efc4809bcb0 con 0x7efc60145550 2024-09-17T04:31:05.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.564+0000 7efc5de00640 1 --2- 172.21.15.100:0/4230328285 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7efc3c079010 0x7efc3c07b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:05.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.565+0000 7efc5de00640 1 --2- 172.21.15.100:0/4230328285 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7efc3c079010 0x7efc3c07b520 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efc50034000 tx=0x7efc50004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:05.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.569+0000 7efc56a00640 1 -- 172.21.15.100:0/4230328285 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7efc601f69b0 con 0x7efc60145550 2024-09-17T04:31:05.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.754+0000 7efc65200640 1 -- 172.21.15.100:0/4230328285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7efc601f6220 con 0x7efc60145550 2024-09-17T04:31:05.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.758+0000 7efc56a00640 1 -- 172.21.15.100:0/4230328285 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v44)=0 v44) ==== 122+0+0 (secure 0 0 0) 0x7efc48063600 con 0x7efc60145550 2024-09-17T04:31:05.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.761+0000 7efc65200640 1 -- 172.21.15.100:0/4230328285 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7efc3c079010 msgr2=0x7efc3c07b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:05.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.761+0000 7efc65200640 1 --2- 172.21.15.100:0/4230328285 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7efc3c079010 0x7efc3c07b520 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efc50034000 tx=0x7efc50004730 comp rx=0 tx=0).stop 2024-09-17T04:31:05.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.761+0000 7efc65200640 1 -- 172.21.15.100:0/4230328285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc60145550 msgr2=0x7efc601f5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:05.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.761+0000 7efc65200640 1 --2- 172.21.15.100:0/4230328285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc60145550 0x7efc601f5980 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7efc4800e6d0 tx=0x7efc4800eb90 comp rx=0 tx=0).stop 2024-09-17T04:31:05.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.761+0000 7efc65200640 1 -- 172.21.15.100:0/4230328285 shutdown_connections 2024-09-17T04:31:05.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.761+0000 7efc65200640 1 --2- 172.21.15.100:0/4230328285 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7efc3c079010 0x7efc3c07b520 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:05.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.761+0000 7efc65200640 1 --2- 172.21.15.100:0/4230328285 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc60145a40 0x7efc6015fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:05.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.761+0000 7efc65200640 1 --2- 172.21.15.100:0/4230328285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc60145550 0x7efc601f5980 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:05.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.761+0000 7efc65200640 1 --2- 172.21.15.100:0/4230328285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc60073a80 0x7efc6013d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:05.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.761+0000 7efc65200640 1 -- 172.21.15.100:0/4230328285 >> 172.21.15.100:0/4230328285 conn(0x7efc6006d1c0 msgr2=0x7efc600b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:05.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.761+0000 7efc65200640 1 -- 172.21.15.100:0/4230328285 shutdown_connections 2024-09-17T04:31:05.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:05.762+0000 7efc65200640 1 -- 172.21.15.100:0/4230328285 wait complete. 2024-09-17T04:31:05.788 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T04:31:05.788 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:31:05.873 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:31:05.875 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:31:05.919 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:31:05.978 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T04:31:05.978 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:31:05.978 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:31:06.014 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:31:06.014 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:31:06.014 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:31:06.078 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:31:06.078 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:31:06.078 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:31:06.114 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:31:06.114 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:31:06.114 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:31:06.154 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:31:06.154 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T04:31:06.154 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:31:06.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.373+0000 7f54dc200640 1 -- 172.21.15.100:0/377574123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54d415d760 msgr2=0x7f54d41535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:06.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.373+0000 7f54dc200640 1 --2- 172.21.15.100:0/377574123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54d415d760 0x7f54d41535c0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f54d4072d60 tx=0x7f54c802a560 comp rx=0 tx=0).stop 2024-09-17T04:31:06.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.373+0000 7f54dc200640 1 -- 172.21.15.100:0/377574123 shutdown_connections 2024-09-17T04:31:06.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.373+0000 7f54dc200640 1 --2- 172.21.15.100:0/377574123 >> v2:172.21.15.184:3300/0 conn(0x7f54d40ad0e0 0x7f54d40ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:06.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.373+0000 7f54dc200640 1 --2- 172.21.15.100:0/377574123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54d415d760 0x7f54d41535c0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:06.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.373+0000 7f54dc200640 1 -- 172.21.15.100:0/377574123 >> 172.21.15.100:0/377574123 conn(0x7f54d406d730 msgr2=0x7f54d40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:06.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.374+0000 7f54dc200640 1 -- 172.21.15.100:0/377574123 shutdown_connections 2024-09-17T04:31:06.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.374+0000 7f54dc200640 1 -- 172.21.15.100:0/377574123 wait complete. 2024-09-17T04:31:06.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.376+0000 7f54dc200640 1 Processor -- start 2024-09-17T04:31:06.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.376+0000 7f54dc200640 1 -- start start 2024-09-17T04:31:06.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.377+0000 7f54dc200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54d40ad0e0 0x7f54d4154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:06.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.377+0000 7f54dc200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54d415d760 0x7f54d414e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:06.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.377+0000 7f54dc200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54d4160310 0x7f54d4156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:06.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.377+0000 7f54dc200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f54d40b4c20 con 0x7f54d40ad0e0 2024-09-17T04:31:06.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.377+0000 7f54dc200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f54d4153c70 con 0x7f54d415d760 2024-09-17T04:31:06.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.377+0000 7f54dc200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f54d40b4da0 con 0x7f54d4160310 2024-09-17T04:31:06.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.377+0000 7f54d9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54d40ad0e0 0x7f54d4154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:06.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.378+0000 7f54d9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54d40ad0e0 0x7f54d4154850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57000/0 (socket says 172.21.15.100:57000) 2024-09-17T04:31:06.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.378+0000 7f54d9800640 1 -- 172.21.15.100:0/2852481434 learned_addr learned my addr 172.21.15.100:0/2852481434 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:06.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.378+0000 7f54d9800640 1 -- 172.21.15.100:0/2852481434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54d4160310 msgr2=0x7f54d4156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:06.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.378+0000 7f54d9800640 1 --2- 172.21.15.100:0/2852481434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54d4160310 0x7f54d4156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:06.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.378+0000 7f54d8e00640 1 --2- 172.21.15.100:0/2852481434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54d415d760 0x7f54d414e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:06.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.378+0000 7f54d9800640 1 -- 172.21.15.100:0/2852481434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54d415d760 msgr2=0x7f54d414e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:06.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.378+0000 7f54d9800640 1 --2- 172.21.15.100:0/2852481434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54d415d760 0x7f54d414e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:06.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.378+0000 7f54d9800640 1 -- 172.21.15.100:0/2852481434 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d4155280 con 0x7f54d40ad0e0 2024-09-17T04:31:06.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.378+0000 7f54d8e00640 1 --2- 172.21.15.100:0/2852481434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54d415d760 0x7f54d414e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:31:06.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.378+0000 7f54d9800640 1 --2- 172.21.15.100:0/2852481434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54d40ad0e0 0x7f54d4154850 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f54c8034040 tx=0x7f54c8003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:06.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.379+0000 7f54d1600640 1 -- 172.21.15.100:0/2852481434 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c803e030 con 0x7f54d40ad0e0 2024-09-17T04:31:06.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.379+0000 7f54d1600640 1 -- 172.21.15.100:0/2852481434 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54c8003d10 con 0x7f54d40ad0e0 2024-09-17T04:31:06.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.379+0000 7f54dc200640 1 -- 172.21.15.100:0/2852481434 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d41557c0 con 0x7f54d40ad0e0 2024-09-17T04:31:06.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.379+0000 7f54d1600640 1 -- 172.21.15.100:0/2852481434 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8035040 con 0x7f54d40ad0e0 2024-09-17T04:31:06.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.379+0000 7f54dc200640 1 -- 172.21.15.100:0/2852481434 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d4155d80 con 0x7f54d40ad0e0 2024-09-17T04:31:06.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.380+0000 7f54dc200640 1 -- 172.21.15.100:0/2852481434 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54d4160ff0 con 0x7f54d40ad0e0 2024-09-17T04:31:06.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.381+0000 7f54d1600640 1 -- 172.21.15.100:0/2852481434 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f54c8041030 con 0x7f54d40ad0e0 2024-09-17T04:31:06.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.382+0000 7f54d1600640 1 --2- 172.21.15.100:0/2852481434 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f54b4079010 0x7f54b407b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:06.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.382+0000 7f54d1600640 1 -- 172.21.15.100:0/2852481434 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 4971+0+0 (secure 0 0 0) 0x7f54c80bbb30 con 0x7f54d40ad0e0 2024-09-17T04:31:06.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.382+0000 7f54d8e00640 1 --2- 172.21.15.100:0/2852481434 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f54b4079010 0x7f54b407b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:06.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.383+0000 7f54d8e00640 1 --2- 172.21.15.100:0/2852481434 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f54b4079010 0x7f54b407b520 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f54d414f850 tx=0x7f54c0005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:06.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.386+0000 7f54d1600640 1 -- 172.21.15.100:0/2852481434 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f54c8083ad0 con 0x7f54d40ad0e0 2024-09-17T04:31:06.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.605+0000 7f54dc200640 1 -- 172.21.15.100:0/2852481434 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54d406f4e0 con 0x7f54d40ad0e0 2024-09-17T04:31:06.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.605+0000 7f54d1600640 1 -- 172.21.15.100:0/2852481434 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+3920 (secure 0 0 0) 0x7f54c80833d0 con 0x7f54d40ad0e0 2024-09-17T04:31:06.607 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:06.607 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":75,"btime":"2024-09-17T04:30:59:830441+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6044,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1891177834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1891177834},{"type":"v1","addr":"172.21.15.100:6835","nonce":1891177834}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":6047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1158083377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1158083377},{"type":"v1","addr":"172.21.15.184:6835","nonce":1158083377}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":6697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/714465116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":714465116},{"type":"v1","addr":"172.21.15.184:6801","nonce":714465116}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":7449,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/3308607200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3308607200},{"type":"v1","addr":"172.21.15.100:6833","nonce":3308607200}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[]} 2024-09-17T04:31:06.609 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 75 2024-09-17T04:31:06.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.607+0000 7f54dc200640 1 -- 172.21.15.100:0/2852481434 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f54b4079010 msgr2=0x7f54b407b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.607+0000 7f54dc200640 1 --2- 172.21.15.100:0/2852481434 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f54b4079010 0x7f54b407b520 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f54d414f850 tx=0x7f54c0005f90 comp rx=0 tx=0).stop 2024-09-17T04:31:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.607+0000 7f54dc200640 1 -- 172.21.15.100:0/2852481434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54d40ad0e0 msgr2=0x7f54d4154850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.607+0000 7f54dc200640 1 --2- 172.21.15.100:0/2852481434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54d40ad0e0 0x7f54d4154850 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f54c8034040 tx=0x7f54c8003820 comp rx=0 tx=0).stop 2024-09-17T04:31:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.608+0000 7f54dc200640 1 -- 172.21.15.100:0/2852481434 shutdown_connections 2024-09-17T04:31:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.608+0000 7f54dc200640 1 --2- 172.21.15.100:0/2852481434 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f54b4079010 0x7f54b407b520 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:06.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.608+0000 7f54dc200640 1 --2- 172.21.15.100:0/2852481434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54d4160310 0x7f54d4156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:06.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.608+0000 7f54dc200640 1 --2- 172.21.15.100:0/2852481434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54d415d760 0x7f54d414e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:06.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.608+0000 7f54dc200640 1 --2- 172.21.15.100:0/2852481434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54d40ad0e0 0x7f54d4154850 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:06.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.608+0000 7f54dc200640 1 -- 172.21.15.100:0/2852481434 >> 172.21.15.100:0/2852481434 conn(0x7f54d406d730 msgr2=0x7f54d40cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:06.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.608+0000 7f54dc200640 1 -- 172.21.15.100:0/2852481434 shutdown_connections 2024-09-17T04:31:06.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.608+0000 7f54dc200640 1 -- 172.21.15.100:0/2852481434 wait complete. 2024-09-17T04:31:06.630 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T04:31:06.630 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T04:31:06.631 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T04:31:06.631 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:31:06.631 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T04:31:06.631 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T04:31:06.631 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T04:31:06.631 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:31:06.631 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T04:31:06.632 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T04:31:06.632 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T04:31:06.632 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:31:06.632 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T04:31:06.632 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T04:31:06.632 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T04:31:06.632 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:31:06.652 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T04:31:06.652 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T04:31:06.656 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T04:31:06.656 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T04:31:06.664 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T04:31:06.665 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T04:31:06.665 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T04:31:06.666 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T04:31:06.700 INFO:tasks.ceph.mds.a:Started 2024-09-17T04:31:06.700 DEBUG:teuthology.parallel:result is None 2024-09-17T04:31:06.703 INFO:tasks.ceph.mds.c:Started 2024-09-17T04:31:06.703 DEBUG:teuthology.parallel:result is None 2024-09-17T04:31:06.709 INFO:tasks.ceph.mds.b:Started 2024-09-17T04:31:06.709 DEBUG:teuthology.parallel:result is None 2024-09-17T04:31:06.711 INFO:tasks.ceph.mds.d:Started 2024-09-17T04:31:06.711 DEBUG:teuthology.parallel:result is None 2024-09-17T04:31:06.712 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T04:31:06.896 INFO:tasks.ceph.mds.a.smithi100.stdout:starting mds.a at 2024-09-17T04:31:06.898 INFO:tasks.ceph.mds.c.smithi100.stdout:starting mds.c at 2024-09-17T04:31:06.906 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-09-17T04:31:06.906 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-09-17T04:31:06.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.947+0000 7fe89d600640 1 -- 172.21.15.100:0/3955015773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe87c004800 msgr2=0x7fe87c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:06.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.947+0000 7fe89d600640 1 --2- 172.21.15.100:0/3955015773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe87c004800 0x7fe87c004c10 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fe88400c290 tx=0x7fe87c005830 comp rx=0 tx=0).stop 2024-09-17T04:31:06.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.947+0000 7fe89d600640 1 -- 172.21.15.100:0/3955015773 shutdown_connections 2024-09-17T04:31:06.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.947+0000 7fe89d600640 1 --2- 172.21.15.100:0/3955015773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe87c004800 0x7fe87c004c10 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:06.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.947+0000 7fe89d600640 1 -- 172.21.15.100:0/3955015773 >> 172.21.15.100:0/3955015773 conn(0x7fe89806dd60 msgr2=0x7fe89806e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:06.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.947+0000 7fe89d600640 1 -- 172.21.15.100:0/3955015773 shutdown_connections 2024-09-17T04:31:06.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.947+0000 7fe89d600640 1 -- 172.21.15.100:0/3955015773 wait complete. 2024-09-17T04:31:06.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.949+0000 7fe89d600640 1 Processor -- start 2024-09-17T04:31:06.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.949+0000 7fe89d600640 1 -- start start 2024-09-17T04:31:06.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.951+0000 7fe89d600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe87c004800 0x7fe89815ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:06.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.951+0000 7fe89d600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8980cd640 0x7fe8981f2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:06.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.951+0000 7fe89d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8981f5b50 0x7fe8981f5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:06.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.951+0000 7fe89d600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe89815eb40 con 0x7fe8981f5b50 2024-09-17T04:31:06.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.951+0000 7fe89d600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe89815e9c0 con 0x7fe87c004800 2024-09-17T04:31:06.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.951+0000 7fe89d600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe89815ecc0 con 0x7fe8980cd640 2024-09-17T04:31:06.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.951+0000 7fe89cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8981f5b50 0x7fe8981f5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:06.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.951+0000 7fe89cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8981f5b50 0x7fe8981f5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57056/0 (socket says 172.21.15.100:57056) 2024-09-17T04:31:06.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.951+0000 7fe89cc00640 1 -- 172.21.15.100:0/3004889739 learned_addr learned my addr 172.21.15.100:0/3004889739 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:06.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.951+0000 7fe897e00640 1 --2- 172.21.15.100:0/3004889739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe87c004800 0x7fe89815ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:06.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.951+0000 7fe897400640 1 --2- 172.21.15.100:0/3004889739 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8980cd640 0x7fe8981f2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:06.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.951+0000 7fe89cc00640 1 -- 172.21.15.100:0/3004889739 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8980cd640 msgr2=0x7fe8981f2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:06.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.951+0000 7fe89cc00640 1 --2- 172.21.15.100:0/3004889739 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8980cd640 0x7fe8981f2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:06.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.951+0000 7fe89cc00640 1 -- 172.21.15.100:0/3004889739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe87c004800 msgr2=0x7fe89815ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:06.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.951+0000 7fe89cc00640 1 --2- 172.21.15.100:0/3004889739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe87c004800 0x7fe89815ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:06.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.951+0000 7fe89cc00640 1 -- 172.21.15.100:0/3004889739 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8981f9110 con 0x7fe8981f5b50 2024-09-17T04:31:06.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.952+0000 7fe89cc00640 1 --2- 172.21.15.100:0/3004889739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8981f5b50 0x7fe8981f5f60 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fe890005fd0 tx=0x7fe890010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:06.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.952+0000 7fe88fe00640 1 -- 172.21.15.100:0/3004889739 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89000ee70 con 0x7fe8981f5b50 2024-09-17T04:31:06.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.952+0000 7fe88fe00640 1 -- 172.21.15.100:0/3004889739 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe890015070 con 0x7fe8981f5b50 2024-09-17T04:31:06.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.952+0000 7fe89d600640 1 -- 172.21.15.100:0/3004889739 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8981f93f0 con 0x7fe8981f5b50 2024-09-17T04:31:06.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.952+0000 7fe88fe00640 1 -- 172.21.15.100:0/3004889739 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890003970 con 0x7fe8981f5b50 2024-09-17T04:31:06.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.952+0000 7fe89d600640 1 -- 172.21.15.100:0/3004889739 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8981f5630 con 0x7fe8981f5b50 2024-09-17T04:31:06.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.953+0000 7fe89d600640 1 -- 172.21.15.100:0/3004889739 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8981f8e80 con 0x7fe8981f5b50 2024-09-17T04:31:06.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.954+0000 7fe88fe00640 1 -- 172.21.15.100:0/3004889739 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fe890003b10 con 0x7fe8981f5b50 2024-09-17T04:31:06.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.955+0000 7fe88fe00640 1 --2- 172.21.15.100:0/3004889739 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe878079010 0x7fe87807b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:06.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.955+0000 7fe88fe00640 1 -- 172.21.15.100:0/3004889739 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 4971+0+0 (secure 0 0 0) 0x7fe89009baf0 con 0x7fe8981f5b50 2024-09-17T04:31:06.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.956+0000 7fe897e00640 1 --2- 172.21.15.100:0/3004889739 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe878079010 0x7fe87807b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:06.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.956+0000 7fe897e00640 1 --2- 172.21.15.100:0/3004889739 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe878079010 0x7fe87807b520 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe888006fd0 tx=0x7fe888006b10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:06.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:06.960+0000 7fe88fe00640 1 -- 172.21.15.100:0/3004889739 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe8900633b0 con 0x7fe8981f5b50 2024-09-17T04:31:07.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:07.154+0000 7fe89d600640 1 -- 172.21.15.100:0/3004889739 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe8980d0f10 con 0x7fe8981f5b50 2024-09-17T04:31:07.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:07.879+0000 7fe88fe00640 1 -- 172.21.15.100:0/3004889739 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v113) ==== 97+0+0 (secure 0 0 0) 0x7fe890067de0 con 0x7fe8981f5b50 2024-09-17T04:31:07.879 INFO:teuthology.orchestra.run.smithi100.stderr: removed all blocklist entries 2024-09-17T04:31:07.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:07.882+0000 7fe89d600640 1 -- 172.21.15.100:0/3004889739 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe878079010 msgr2=0x7fe87807b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:07.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:07.882+0000 7fe89d600640 1 --2- 172.21.15.100:0/3004889739 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe878079010 0x7fe87807b520 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe888006fd0 tx=0x7fe888006b10 comp rx=0 tx=0).stop 2024-09-17T04:31:07.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:07.882+0000 7fe89d600640 1 -- 172.21.15.100:0/3004889739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8981f5b50 msgr2=0x7fe8981f5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:07.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:07.882+0000 7fe89d600640 1 --2- 172.21.15.100:0/3004889739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8981f5b50 0x7fe8981f5f60 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fe890005fd0 tx=0x7fe890010f40 comp rx=0 tx=0).stop 2024-09-17T04:31:07.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:07.882+0000 7fe89d600640 1 -- 172.21.15.100:0/3004889739 shutdown_connections 2024-09-17T04:31:07.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:07.882+0000 7fe89d600640 1 --2- 172.21.15.100:0/3004889739 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe878079010 0x7fe87807b520 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:07.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:07.882+0000 7fe89d600640 1 --2- 172.21.15.100:0/3004889739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8981f5b50 0x7fe8981f5f60 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:07.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:07.882+0000 7fe89d600640 1 --2- 172.21.15.100:0/3004889739 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8980cd640 0x7fe8981f2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:07.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:07.882+0000 7fe89d600640 1 --2- 172.21.15.100:0/3004889739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe87c004800 0x7fe89815ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:07.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:07.882+0000 7fe89d600640 1 -- 172.21.15.100:0/3004889739 >> 172.21.15.100:0/3004889739 conn(0x7fe89806dd60 msgr2=0x7fe8980b5da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:07.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:07.883+0000 7fe89d600640 1 -- 172.21.15.100:0/3004889739 shutdown_connections 2024-09-17T04:31:07.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:07.883+0000 7fe89d600640 1 -- 172.21.15.100:0/3004889739 wait complete. 2024-09-17T04:31:07.910 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T04:31:08.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.133+0000 7f18a1e00640 1 -- 172.21.15.100:0/921414343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1880003a50 msgr2=0x7f1880003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:08.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.133+0000 7f18a1e00640 1 --2- 172.21.15.100:0/921414343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1880003a50 0x7f1880003ec0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f188c001050 tx=0x7f1880005830 comp rx=0 tx=0).stop 2024-09-17T04:31:08.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.133+0000 7f18a1e00640 1 -- 172.21.15.100:0/921414343 shutdown_connections 2024-09-17T04:31:08.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.133+0000 7f18a1e00640 1 --2- 172.21.15.100:0/921414343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1880003a50 0x7f1880003ec0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:08.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.133+0000 7f18a1e00640 1 -- 172.21.15.100:0/921414343 >> 172.21.15.100:0/921414343 conn(0x7f189c0b1e40 msgr2=0x7f189c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:08.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.133+0000 7f18a1e00640 1 -- 172.21.15.100:0/921414343 shutdown_connections 2024-09-17T04:31:08.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.133+0000 7f18a1e00640 1 -- 172.21.15.100:0/921414343 wait complete. 2024-09-17T04:31:08.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.135+0000 7f18a1e00640 1 Processor -- start 2024-09-17T04:31:08.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.135+0000 7f18a1e00640 1 -- start start 2024-09-17T04:31:08.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.137+0000 7f18a1e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1880003a50 0x7f189c150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:08.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.137+0000 7f18a1e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f189c15c470 0x7f189c149490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:08.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.137+0000 7f18a1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f189c0af990 0x7f189c14b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:08.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.137+0000 7f18a1e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f189c072e60 con 0x7f189c0af990 2024-09-17T04:31:08.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.137+0000 7f18a1e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f189c0702c0 con 0x7f189c15c470 2024-09-17T04:31:08.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.137+0000 7f18a1e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f189c072fe0 con 0x7f1880003a50 2024-09-17T04:31:08.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.137+0000 7f18a1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f189c0af990 0x7f189c14b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:08.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.137+0000 7f18a1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f189c0af990 0x7f189c14b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57074/0 (socket says 172.21.15.100:57074) 2024-09-17T04:31:08.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.137+0000 7f18a1400640 1 -- 172.21.15.100:0/1492008667 learned_addr learned my addr 172.21.15.100:0/1492008667 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:08.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.137+0000 7f18a0a00640 1 --2- 172.21.15.100:0/1492008667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1880003a50 0x7f189c150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:08.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.137+0000 7f189be00640 1 --2- 172.21.15.100:0/1492008667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f189c15c470 0x7f189c149490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:08.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.137+0000 7f18a1400640 1 -- 172.21.15.100:0/1492008667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1880003a50 msgr2=0x7f189c150d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:08.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.137+0000 7f18a1400640 1 --2- 172.21.15.100:0/1492008667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1880003a50 0x7f189c150d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:08.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.137+0000 7f18a1400640 1 -- 172.21.15.100:0/1492008667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f189c15c470 msgr2=0x7f189c149490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:08.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.137+0000 7f18a1400640 1 --2- 172.21.15.100:0/1492008667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f189c15c470 0x7f189c149490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:08.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.137+0000 7f18a1400640 1 -- 172.21.15.100:0/1492008667 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189c15b3d0 con 0x7f189c0af990 2024-09-17T04:31:08.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.137+0000 7f18a0a00640 1 --2- 172.21.15.100:0/1492008667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1880003a50 0x7f189c150d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:31:08.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.137+0000 7f189be00640 1 --2- 172.21.15.100:0/1492008667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f189c15c470 0x7f189c149490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:31:08.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.138+0000 7f18a1400640 1 --2- 172.21.15.100:0/1492008667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f189c0af990 0x7f189c14b0a0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f189c0ad830 tx=0x7f18940019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:08.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.138+0000 7f1898c00640 1 -- 172.21.15.100:0/1492008667 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894011870 con 0x7f189c0af990 2024-09-17T04:31:08.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.138+0000 7f1898c00640 1 -- 172.21.15.100:0/1492008667 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1894017070 con 0x7f189c0af990 2024-09-17T04:31:08.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.138+0000 7f1898c00640 1 -- 172.21.15.100:0/1492008667 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894011da0 con 0x7f189c0af990 2024-09-17T04:31:08.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.138+0000 7f18a1e00640 1 -- 172.21.15.100:0/1492008667 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f189c15b6b0 con 0x7f189c0af990 2024-09-17T04:31:08.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.138+0000 7f18a1e00640 1 -- 172.21.15.100:0/1492008667 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f189c15bc70 con 0x7f189c0af990 2024-09-17T04:31:08.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.139+0000 7f18a1e00640 1 -- 172.21.15.100:0/1492008667 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f189c15b140 con 0x7f189c0af990 2024-09-17T04:31:08.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.140+0000 7f1898c00640 1 -- 172.21.15.100:0/1492008667 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f189401c460 con 0x7f189c0af990 2024-09-17T04:31:08.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.141+0000 7f1898c00640 1 --2- 172.21.15.100:0/1492008667 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f187c079070 0x7f187c07b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:08.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.141+0000 7f1898c00640 1 -- 172.21.15.100:0/1492008667 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 4885+0+0 (secure 0 0 0) 0x7f189409b9c0 con 0x7f189c0af990 2024-09-17T04:31:08.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.141+0000 7f18a0a00640 1 --2- 172.21.15.100:0/1492008667 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f187c079070 0x7f187c07b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:08.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.142+0000 7f18a0a00640 1 --2- 172.21.15.100:0/1492008667 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f187c079070 0x7f187c07b580 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f188c033000 tx=0x7f188c0019f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:08.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.146+0000 7f1898c00640 1 -- 172.21.15.100:0/1492008667 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f1894063ae0 con 0x7f189c0af990 2024-09-17T04:31:08.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.397+0000 7f18a1e00640 1 -- 172.21.15.100:0/1492008667 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f189c073540 con 0x7f189c0af990 2024-09-17T04:31:08.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.397+0000 7f1898c00640 1 -- 172.21.15.100:0/1492008667 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7f18940633e0 con 0x7f189c0af990 2024-09-17T04:31:08.398 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:08.398 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:31:08.398 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_dump": [ 2024-09-17T04:31:08.398 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:08.398 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.a", 2024-09-17T04:31:08.398 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmV8asJRAAkSgIWqGc1OeSRbs9PEOT8w==", 2024-09-17T04:31:08.398 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:08.398 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:31:08.399 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:31:08.399 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:31:08.399 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:08.399 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:08.399 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:08.399 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:08.399 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.b", 2024-09-17T04:31:08.399 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmpM7ZBRAA1Og785tJwa6NA9zBdmNNQA==", 2024-09-17T04:31:08.399 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:08.399 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:31:08.399 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:31:08.399 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:31:08.399 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:08.400 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:08.400 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:08.400 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:08.400 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.c", 2024-09-17T04:31:08.400 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmXaOGMxAAT+96g/5fAzPW8Cc2HtQUyA==", 2024-09-17T04:31:08.400 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:08.400 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:31:08.400 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:31:08.400 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:31:08.400 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:08.400 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:08.400 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:08.400 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:08.401 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.d", 2024-09-17T04:31:08.401 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmRqC9ExAA23kZ0Ks1lGN2k4GXL8ASrg==", 2024-09-17T04:31:08.401 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:08.401 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:31:08.401 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:31:08.401 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:31:08.401 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:08.401 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:08.401 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:08.401 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:08.401 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.0", 2024-09-17T04:31:08.401 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAwBOlmofFmAxAARerYkU9B2a9MM9vHa3o4LQ==", 2024-09-17T04:31:08.401 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:08.402 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:31:08.402 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:31:08.402 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:08.402 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:08.402 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:08.402 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:08.402 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.1", 2024-09-17T04:31:08.402 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAzBOlmbz6VExAAoVqSqXTc1wArKhdA6+NxnA==", 2024-09-17T04:31:08.402 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:08.402 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:31:08.402 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:31:08.402 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:08.402 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:08.402 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:08.403 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:08.403 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.2", 2024-09-17T04:31:08.403 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA2BOlmfeSAJBAAchY6XhyDRusc+zqwQ6v6wQ==", 2024-09-17T04:31:08.403 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:08.403 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:31:08.403 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:31:08.403 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:08.403 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:08.403 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:08.403 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:08.403 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.3", 2024-09-17T04:31:08.403 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA6BOlm/TQiBBAAHhbs2+RCjAenGR1frhbTmQ==", 2024-09-17T04:31:08.403 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:08.404 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:31:08.404 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:31:08.404 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:08.404 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:08.404 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:08.404 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:08.404 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.4", 2024-09-17T04:31:08.404 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBABOlm7OKvBhAALkaAKEUYN5ChwGejoSF8yA==", 2024-09-17T04:31:08.404 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:08.405 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:31:08.405 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:31:08.405 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:08.405 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:08.405 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:08.405 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:08.405 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.5", 2024-09-17T04:31:08.405 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBDBOlmQpi1FhAAbA47dg6ckK1PFoCZL0SBEQ==", 2024-09-17T04:31:08.405 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:08.405 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:31:08.405 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:31:08.405 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:08.406 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:08.406 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:08.406 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:08.406 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.6", 2024-09-17T04:31:08.406 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBGBOlmx/w0JhAAWcPvZkMYtXD/EMKrg/W/4Q==", 2024-09-17T04:31:08.406 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:08.406 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:31:08.406 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:31:08.406 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:08.406 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:08.406 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:08.406 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:08.406 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.7", 2024-09-17T04:31:08.406 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBJBOlmWHjWNBAAgdul2UKY9g5p7EVi0ZdrFw==", 2024-09-17T04:31:08.407 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:08.407 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:31:08.407 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:31:08.407 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:08.407 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:08.407 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:08.407 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:08.407 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.0", 2024-09-17T04:31:08.407 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmfYg9IBAABP1uRKGOugrA47VxwRLkjA==", 2024-09-17T04:31:08.408 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:08.408 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:31:08.408 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow r", 2024-09-17T04:31:08.408 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T04:31:08.408 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:08.408 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:08.408 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:08.408 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.1", 2024-09-17T04:31:08.408 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmByhhNRAAPGaxFCDd8ywFVwwhprrPbQ==", 2024-09-17T04:31:08.408 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:08.408 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:31:08.408 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:31:08.408 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:31:08.409 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:31:08.409 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:08.409 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:08.409 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:08.409 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.2", 2024-09-17T04:31:08.409 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmgcm0KhAAGF06VlHjBM/AGxafN6/3aA==", 2024-09-17T04:31:08.409 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:08.409 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:31:08.409 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:31:08.409 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:31:08.409 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:31:08.409 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:08.410 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:08.410 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:08.410 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.3", 2024-09-17T04:31:08.410 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtBOlm9VQ6BBAAXcahu35/uDm8Z+f+3SF0Gg==", 2024-09-17T04:31:08.410 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:08.410 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:31:08.410 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:31:08.410 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:31:08.410 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:31:08.410 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:08.410 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:08.410 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:08.410 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.admin", 2024-09-17T04:31:08.410 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAqBOlmZ+bIJBAAG3oaMNlYWNakywGDvAGbGw==", 2024-09-17T04:31:08.411 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:08.411 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:31:08.411 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:31:08.411 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:31:08.411 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:08.411 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:08.411 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:08.411 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:08.411 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mgr.x", 2024-09-17T04:31:08.411 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmfaL7GhAArqLH66RkbBsz+TNOKtPHOA==", 2024-09-17T04:31:08.411 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:08.411 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:31:08.411 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile mgr", 2024-09-17T04:31:08.411 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:08.412 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:08.412 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:08.412 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:08.412 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mgr.y", 2024-09-17T04:31:08.412 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmM4D1DhAA5uUWgRVOMujWqbYzOjFp9A==", 2024-09-17T04:31:08.412 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:08.412 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:31:08.412 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile mgr", 2024-09-17T04:31:08.412 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:08.412 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:08.412 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:08.412 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-17T04:31:08.413 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:31:08.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.400+0000 7f18a1e00640 1 -- 172.21.15.100:0/1492008667 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f187c079070 msgr2=0x7f187c07b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:08.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.400+0000 7f18a1e00640 1 --2- 172.21.15.100:0/1492008667 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f187c079070 0x7f187c07b580 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f188c033000 tx=0x7f188c0019f0 comp rx=0 tx=0).stop 2024-09-17T04:31:08.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.400+0000 7f18a1e00640 1 -- 172.21.15.100:0/1492008667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f189c0af990 msgr2=0x7f189c14b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:08.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.400+0000 7f18a1e00640 1 --2- 172.21.15.100:0/1492008667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f189c0af990 0x7f189c14b0a0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f189c0ad830 tx=0x7f18940019b0 comp rx=0 tx=0).stop 2024-09-17T04:31:08.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.400+0000 7f18a1e00640 1 -- 172.21.15.100:0/1492008667 shutdown_connections 2024-09-17T04:31:08.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.400+0000 7f18a1e00640 1 --2- 172.21.15.100:0/1492008667 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f187c079070 0x7f187c07b580 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:08.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.400+0000 7f18a1e00640 1 --2- 172.21.15.100:0/1492008667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f189c0af990 0x7f189c14b0a0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:08.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.400+0000 7f18a1e00640 1 --2- 172.21.15.100:0/1492008667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f189c15c470 0x7f189c149490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:08.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.400+0000 7f18a1e00640 1 --2- 172.21.15.100:0/1492008667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1880003a50 0x7f189c150d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:08.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.400+0000 7f18a1e00640 1 -- 172.21.15.100:0/1492008667 >> 172.21.15.100:0/1492008667 conn(0x7f189c0b1e40 msgr2=0x7f189c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:08.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.400+0000 7f18a1e00640 1 -- 172.21.15.100:0/1492008667 shutdown_connections 2024-09-17T04:31:08.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.400+0000 7f18a1e00640 1 -- 172.21.15.100:0/1492008667 wait complete. 2024-09-17T04:31:08.425 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T04:31:08.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.645+0000 7fb0a5800640 1 -- 172.21.15.100:0/814290405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb084004800 msgr2=0x7fb084004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:08.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.645+0000 7fb0a5800640 1 --2- 172.21.15.100:0/814290405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb084004800 0x7fb084004c10 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fb088001800 tx=0x7fb084005830 comp rx=0 tx=0).stop 2024-09-17T04:31:08.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.645+0000 7fb0a5800640 1 -- 172.21.15.100:0/814290405 shutdown_connections 2024-09-17T04:31:08.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.645+0000 7fb0a5800640 1 --2- 172.21.15.100:0/814290405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb084004800 0x7fb084004c10 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:08.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.645+0000 7fb0a5800640 1 -- 172.21.15.100:0/814290405 >> 172.21.15.100:0/814290405 conn(0x7fb0a006d730 msgr2=0x7fb0a00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:08.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.645+0000 7fb0a5800640 1 -- 172.21.15.100:0/814290405 shutdown_connections 2024-09-17T04:31:08.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.645+0000 7fb0a5800640 1 -- 172.21.15.100:0/814290405 wait complete. 2024-09-17T04:31:08.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.647+0000 7fb0a5800640 1 Processor -- start 2024-09-17T04:31:08.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.647+0000 7fb0a5800640 1 -- start start 2024-09-17T04:31:08.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.649+0000 7fb0a5800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb084004800 0x7fb0a01575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:08.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.649+0000 7fb0a5800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0a0149920 0x7fb0a0159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:08.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.649+0000 7fb0a5800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0a015a240 0x7fb0a00aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:08.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.649+0000 7fb0a5800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a0073040 con 0x7fb0a015a240 2024-09-17T04:31:08.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.649+0000 7fb0a5800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a0071340 con 0x7fb0a0149920 2024-09-17T04:31:08.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.649+0000 7fb0a5800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb0a00731c0 con 0x7fb084004800 2024-09-17T04:31:08.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.649+0000 7fb09f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0a015a240 0x7fb0a00aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:08.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.649+0000 7fb09f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0a015a240 0x7fb0a00aefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57088/0 (socket says 172.21.15.100:57088) 2024-09-17T04:31:08.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.649+0000 7fb09f400640 1 -- 172.21.15.100:0/3309512477 learned_addr learned my addr 172.21.15.100:0/3309512477 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:08.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.649+0000 7fb09ea00640 1 --2- 172.21.15.100:0/3309512477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb084004800 0x7fb0a01575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:08.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.649+0000 7fb096000640 1 --2- 172.21.15.100:0/3309512477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0a0149920 0x7fb0a0159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:08.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.650+0000 7fb09f400640 1 -- 172.21.15.100:0/3309512477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb084004800 msgr2=0x7fb0a01575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:08.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.650+0000 7fb09f400640 1 --2- 172.21.15.100:0/3309512477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb084004800 0x7fb0a01575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:08.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.650+0000 7fb09f400640 1 -- 172.21.15.100:0/3309512477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0a0149920 msgr2=0x7fb0a0159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:08.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.650+0000 7fb09f400640 1 --2- 172.21.15.100:0/3309512477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0a0149920 0x7fb0a0159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:08.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.650+0000 7fb09f400640 1 -- 172.21.15.100:0/3309512477 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a0098cd0 con 0x7fb0a015a240 2024-09-17T04:31:08.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.650+0000 7fb09f400640 1 --2- 172.21.15.100:0/3309512477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0a015a240 0x7fb0a00aefa0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fb098001be0 tx=0x7fb09800e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:08.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.650+0000 7fb097400640 1 -- 172.21.15.100:0/3309512477 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb098010890 con 0x7fb0a015a240 2024-09-17T04:31:08.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.650+0000 7fb097400640 1 -- 172.21.15.100:0/3309512477 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb098016070 con 0x7fb0a015a240 2024-09-17T04:31:08.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.650+0000 7fb0a5800640 1 -- 172.21.15.100:0/3309512477 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a015a650 con 0x7fb0a015a240 2024-09-17T04:31:08.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.650+0000 7fb097400640 1 -- 172.21.15.100:0/3309512477 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb098010dc0 con 0x7fb0a015a240 2024-09-17T04:31:08.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.651+0000 7fb0a5800640 1 -- 172.21.15.100:0/3309512477 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a00ae4c0 con 0x7fb0a015a240 2024-09-17T04:31:08.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.651+0000 7fb0a5800640 1 -- 172.21.15.100:0/3309512477 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0a0098510 con 0x7fb0a015a240 2024-09-17T04:31:08.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.653+0000 7fb097400640 1 -- 172.21.15.100:0/3309512477 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fb09801b460 con 0x7fb0a015a240 2024-09-17T04:31:08.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.654+0000 7fb097400640 1 --2- 172.21.15.100:0/3309512477 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb07c079040 0x7fb07c07b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:08.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.654+0000 7fb097400640 1 -- 172.21.15.100:0/3309512477 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 4885+0+0 (secure 0 0 0) 0x7fb09809b670 con 0x7fb0a015a240 2024-09-17T04:31:08.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.654+0000 7fb09ea00640 1 --2- 172.21.15.100:0/3309512477 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb07c079040 0x7fb07c07b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:08.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.655+0000 7fb09ea00640 1 --2- 172.21.15.100:0/3309512477 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb07c079040 0x7fb07c07b550 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb09000aca0 tx=0x7fb090005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:08.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.658+0000 7fb097400640 1 -- 172.21.15.100:0/3309512477 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb0a015a650 con 0x7fb0a015a240 2024-09-17T04:31:08.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.844+0000 7fb0a5800640 1 -- 172.21.15.100:0/3309512477 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb0a00736d0 con 0x7fb0a015a240 2024-09-17T04:31:08.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.845+0000 7fb097400640 1 -- 172.21.15.100:0/3309512477 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v115) ==== 84+0+65 (secure 0 0 0) 0x7fb0a0098cd0 con 0x7fb0a015a240 2024-09-17T04:31:08.845 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:08.845 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-09-17T04:31:08.845 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:08.846 INFO:teuthology.orchestra.run.smithi100.stdout: "poolnum": 1, 2024-09-17T04:31:08.846 INFO:teuthology.orchestra.run.smithi100.stdout: "poolname": ".mgr" 2024-09-17T04:31:08.846 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:08.846 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-09-17T04:31:08.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.847+0000 7fb0a5800640 1 -- 172.21.15.100:0/3309512477 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb07c079040 msgr2=0x7fb07c07b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:08.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.847+0000 7fb0a5800640 1 --2- 172.21.15.100:0/3309512477 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb07c079040 0x7fb07c07b550 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb09000aca0 tx=0x7fb090005f90 comp rx=0 tx=0).stop 2024-09-17T04:31:08.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.847+0000 7fb0a5800640 1 -- 172.21.15.100:0/3309512477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0a015a240 msgr2=0x7fb0a00aefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:08.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.847+0000 7fb0a5800640 1 --2- 172.21.15.100:0/3309512477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0a015a240 0x7fb0a00aefa0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fb098001be0 tx=0x7fb09800e950 comp rx=0 tx=0).stop 2024-09-17T04:31:08.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.847+0000 7fb0a5800640 1 -- 172.21.15.100:0/3309512477 shutdown_connections 2024-09-17T04:31:08.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.847+0000 7fb0a5800640 1 --2- 172.21.15.100:0/3309512477 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb07c079040 0x7fb07c07b550 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:08.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.847+0000 7fb0a5800640 1 --2- 172.21.15.100:0/3309512477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0a015a240 0x7fb0a00aefa0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:08.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.847+0000 7fb0a5800640 1 --2- 172.21.15.100:0/3309512477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0a0149920 0x7fb0a0159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:08.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.847+0000 7fb0a5800640 1 --2- 172.21.15.100:0/3309512477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb084004800 0x7fb0a01575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:08.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.847+0000 7fb0a5800640 1 -- 172.21.15.100:0/3309512477 >> 172.21.15.100:0/3309512477 conn(0x7fb0a006d730 msgr2=0x7fb0a00afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:08.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.847+0000 7fb0a5800640 1 -- 172.21.15.100:0/3309512477 shutdown_connections 2024-09-17T04:31:08.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:08.847+0000 7fb0a5800640 1 -- 172.21.15.100:0/3309512477 wait complete. 2024-09-17T04:31:08.869 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T04:31:08.869 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T04:31:09.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.094+0000 7f8bd9c00640 1 -- 172.21.15.100:0/351150346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bd40af100 msgr2=0x7f8bd40af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:09.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.094+0000 7f8bd9c00640 1 --2- 172.21.15.100:0/351150346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bd40af100 0x7f8bd40af510 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40d0d00 tx=0x7f8bcc02a5f0 comp rx=0 tx=0).stop 2024-09-17T04:31:09.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.094+0000 7f8bd9c00640 1 -- 172.21.15.100:0/351150346 shutdown_connections 2024-09-17T04:31:09.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.095+0000 7f8bd9c00640 1 --2- 172.21.15.100:0/351150346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bd40af100 0x7f8bd40af510 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:09.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.095+0000 7f8bd9c00640 1 --2- 172.21.15.100:0/351150346 >> v2:172.21.15.184:3300/0 conn(0x7f8bd406d4b0 0x7f8bd40b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:09.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.095+0000 7f8bd9c00640 1 -- 172.21.15.100:0/351150346 >> 172.21.15.100:0/351150346 conn(0x7f8bd40b2540 msgr2=0x7f8bd40d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:09.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.095+0000 7f8bd9c00640 1 -- 172.21.15.100:0/351150346 shutdown_connections 2024-09-17T04:31:09.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.095+0000 7f8bd9c00640 1 -- 172.21.15.100:0/351150346 wait complete. 2024-09-17T04:31:09.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.097+0000 7f8bd9c00640 1 Processor -- start 2024-09-17T04:31:09.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.097+0000 7f8bd9c00640 1 -- start start 2024-09-17T04:31:09.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.098+0000 7f8bd9c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bd40af100 0x7f8bd408d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:09.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.098+0000 7f8bd9c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bd40cc830 0x7f8bd408bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:09.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.098+0000 7f8bd9c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bd409df90 0x7f8bd408c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:09.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.098+0000 7f8bd9c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd40b3620 con 0x7f8bd40cc830 2024-09-17T04:31:09.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.098+0000 7f8bd9c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd40b34a0 con 0x7f8bd40af100 2024-09-17T04:31:09.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.098+0000 7f8bd9c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8bd40b37a0 con 0x7f8bd409df90 2024-09-17T04:31:09.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.099+0000 7f8bd2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bd40cc830 0x7f8bd408bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:09.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.099+0000 7f8bd2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bd40cc830 0x7f8bd408bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57100/0 (socket says 172.21.15.100:57100) 2024-09-17T04:31:09.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.099+0000 7f8bd2000640 1 -- 172.21.15.100:0/3805715069 learned_addr learned my addr 172.21.15.100:0/3805715069 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:09.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.099+0000 7f8bd3400640 1 --2- 172.21.15.100:0/3805715069 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bd409df90 0x7f8bd408c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:09.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.099+0000 7f8bd2a00640 1 --2- 172.21.15.100:0/3805715069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bd40af100 0x7f8bd408d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:09.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.099+0000 7f8bd2000640 1 -- 172.21.15.100:0/3805715069 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bd409df90 msgr2=0x7f8bd408c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:09.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.099+0000 7f8bd2000640 1 --2- 172.21.15.100:0/3805715069 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bd409df90 0x7f8bd408c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:09.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.099+0000 7f8bd2000640 1 -- 172.21.15.100:0/3805715069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bd40af100 msgr2=0x7f8bd408d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:09.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.099+0000 7f8bd2000640 1 --2- 172.21.15.100:0/3805715069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bd40af100 0x7f8bd408d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:09.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.099+0000 7f8bd2000640 1 -- 172.21.15.100:0/3805715069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd408dd80 con 0x7f8bd40cc830 2024-09-17T04:31:09.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.099+0000 7f8bd2000640 1 --2- 172.21.15.100:0/3805715069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bd40cc830 0x7f8bd408bfe0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000c9a0 tx=0x7f8bc000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:09.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.100+0000 7f8bcaa00640 1 -- 172.21.15.100:0/3805715069 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0007bb0 con 0x7f8bd40cc830 2024-09-17T04:31:09.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.100+0000 7f8bcaa00640 1 -- 172.21.15.100:0/3805715069 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8bc0007e40 con 0x7f8bd40cc830 2024-09-17T04:31:09.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.100+0000 7f8bd9c00640 1 -- 172.21.15.100:0/3805715069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd409d1d0 con 0x7f8bd40cc830 2024-09-17T04:31:09.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.100+0000 7f8bcaa00640 1 -- 172.21.15.100:0/3805715069 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc000f6b0 con 0x7f8bd40cc830 2024-09-17T04:31:09.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.100+0000 7f8bd9c00640 1 -- 172.21.15.100:0/3805715069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd409d790 con 0x7f8bd40cc830 2024-09-17T04:31:09.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.101+0000 7f8bd9c00640 1 -- 172.21.15.100:0/3805715069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bd40b3cc0 con 0x7f8bd40cc830 2024-09-17T04:31:09.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.102+0000 7f8bcaa00640 1 -- 172.21.15.100:0/3805715069 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f8bc000f850 con 0x7f8bd40cc830 2024-09-17T04:31:09.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.103+0000 7f8bcaa00640 1 --2- 172.21.15.100:0/3805715069 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8bb4079040 0x7f8bb407b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:09.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.103+0000 7f8bcaa00640 1 -- 172.21.15.100:0/3805715069 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 4885+0+0 (secure 0 0 0) 0x7f8bc009bae0 con 0x7f8bd40cc830 2024-09-17T04:31:09.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.103+0000 7f8bd2a00640 1 --2- 172.21.15.100:0/3805715069 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8bb4079040 0x7f8bb407b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:09.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.104+0000 7f8bd2a00640 1 --2- 172.21.15.100:0/3805715069 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8bb4079040 0x7f8bb407b550 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4007fd0 tx=0x7f8bc4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:09.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.107+0000 7f8bcaa00640 1 -- 172.21.15.100:0/3805715069 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8bc0014030 con 0x7f8bd40cc830 2024-09-17T04:31:09.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.289+0000 7f8bd9c00640 1 -- 172.21.15.100:0/3805715069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8bd40aec90 con 0x7f8bd40cc830 2024-09-17T04:31:09.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.888+0000 7f8bcaa00640 1 -- 172.21.15.100:0/3805715069 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v116) ==== 138+0+0 (secure 0 0 0) 0x7f8bc0063a80 con 0x7f8bd40cc830 2024-09-17T04:31:09.954 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' created 2024-09-17T04:31:09.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.892+0000 7f8bd9c00640 1 -- 172.21.15.100:0/3805715069 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8bb4079040 msgr2=0x7f8bb407b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:09.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.892+0000 7f8bd9c00640 1 --2- 172.21.15.100:0/3805715069 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8bb4079040 0x7f8bb407b550 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4007fd0 tx=0x7f8bc4009040 comp rx=0 tx=0).stop 2024-09-17T04:31:09.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.892+0000 7f8bd9c00640 1 -- 172.21.15.100:0/3805715069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bd40cc830 msgr2=0x7f8bd408bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:09.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.892+0000 7f8bd9c00640 1 --2- 172.21.15.100:0/3805715069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bd40cc830 0x7f8bd408bfe0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000c9a0 tx=0x7f8bc000ce60 comp rx=0 tx=0).stop 2024-09-17T04:31:09.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.892+0000 7f8bd9c00640 1 -- 172.21.15.100:0/3805715069 shutdown_connections 2024-09-17T04:31:09.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.892+0000 7f8bd9c00640 1 --2- 172.21.15.100:0/3805715069 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8bb4079040 0x7f8bb407b550 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:09.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.892+0000 7f8bd9c00640 1 --2- 172.21.15.100:0/3805715069 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bd409df90 0x7f8bd408c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:09.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.892+0000 7f8bd9c00640 1 --2- 172.21.15.100:0/3805715069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bd40cc830 0x7f8bd408bfe0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:09.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.892+0000 7f8bd9c00640 1 --2- 172.21.15.100:0/3805715069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bd40af100 0x7f8bd408d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:09.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.892+0000 7f8bd9c00640 1 -- 172.21.15.100:0/3805715069 >> 172.21.15.100:0/3805715069 conn(0x7f8bd40b2540 msgr2=0x7f8bd40b0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:09.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.892+0000 7f8bd9c00640 1 -- 172.21.15.100:0/3805715069 shutdown_connections 2024-09-17T04:31:09.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:09.892+0000 7f8bd9c00640 1 -- 172.21.15.100:0/3805715069 wait complete. 2024-09-17T04:31:09.958 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T04:31:10.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.192+0000 7fae43200640 1 -- 172.21.15.100:0/1217638989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae28004800 msgr2=0x7fae28004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:10.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.192+0000 7fae43200640 1 --2- 172.21.15.100:0/1217638989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae28004800 0x7fae28004c10 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fae3000c4a0 tx=0x7fae28005830 comp rx=0 tx=0).stop 2024-09-17T04:31:10.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.192+0000 7fae42800640 1 -- 172.21.15.100:0/1217638989 reap_dead start 2024-09-17T04:31:10.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.192+0000 7fae43200640 1 -- 172.21.15.100:0/1217638989 shutdown_connections 2024-09-17T04:31:10.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.192+0000 7fae43200640 1 -- 172.21.15.100:0/1217638989 >> 172.21.15.100:0/1217638989 conn(0x7fae440b1dd0 msgr2=0x7fae440b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:10.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.192+0000 7fae43200640 1 -- 172.21.15.100:0/1217638989 shutdown_connections 2024-09-17T04:31:10.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.192+0000 7fae43200640 1 -- 172.21.15.100:0/1217638989 wait complete. 2024-09-17T04:31:10.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.195+0000 7fae43200640 1 Processor -- start 2024-09-17T04:31:10.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.195+0000 7fae43200640 1 -- start start 2024-09-17T04:31:10.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.196+0000 7fae43200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae28004800 0x7fae44157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:10.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.196+0000 7fae43200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae4415bba0 0x7fae441f8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:10.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.196+0000 7fae43200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae4415bfb0 0x7fae441f8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:10.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.196+0000 7fae43200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fae44070c70 con 0x7fae4415bfb0 2024-09-17T04:31:10.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.196+0000 7fae43200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fae440cd450 con 0x7fae4415bba0 2024-09-17T04:31:10.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.196+0000 7fae43200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fae44160780 con 0x7fae28004800 2024-09-17T04:31:10.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.196+0000 7fae42800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae4415bfb0 0x7fae441f8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:10.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.197+0000 7fae42800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae4415bfb0 0x7fae441f8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57108/0 (socket says 172.21.15.100:57108) 2024-09-17T04:31:10.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.197+0000 7fae42800640 1 -- 172.21.15.100:0/4028970067 learned_addr learned my addr 172.21.15.100:0/4028970067 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:10.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.197+0000 7fae41400640 1 --2- 172.21.15.100:0/4028970067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae4415bba0 0x7fae441f8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:10.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.197+0000 7fae41e00640 1 --2- 172.21.15.100:0/4028970067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae28004800 0x7fae44157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:10.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.197+0000 7fae42800640 1 -- 172.21.15.100:0/4028970067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae28004800 msgr2=0x7fae44157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:10.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.197+0000 7fae42800640 1 --2- 172.21.15.100:0/4028970067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae28004800 0x7fae44157ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:10.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.197+0000 7fae42800640 1 -- 172.21.15.100:0/4028970067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae4415bba0 msgr2=0x7fae441f8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:10.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.197+0000 7fae42800640 1 --2- 172.21.15.100:0/4028970067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae4415bba0 0x7fae441f8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:10.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.197+0000 7fae42800640 1 -- 172.21.15.100:0/4028970067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae44154fa0 con 0x7fae4415bfb0 2024-09-17T04:31:10.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.197+0000 7fae41400640 1 --2- 172.21.15.100:0/4028970067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae4415bba0 0x7fae441f8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:31:10.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.197+0000 7fae42800640 1 --2- 172.21.15.100:0/4028970067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae4415bfb0 0x7fae441f8d40 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fae440d00d0 tx=0x7fae3c010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:10.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.197+0000 7fae3a000640 1 -- 172.21.15.100:0/4028970067 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c00ee50 con 0x7fae4415bfb0 2024-09-17T04:31:10.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.198+0000 7fae3a000640 1 -- 172.21.15.100:0/4028970067 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae3c015070 con 0x7fae4415bfb0 2024-09-17T04:31:10.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.198+0000 7fae43200640 1 -- 172.21.15.100:0/4028970067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae44155280 con 0x7fae4415bfb0 2024-09-17T04:31:10.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.198+0000 7fae3a000640 1 -- 172.21.15.100:0/4028970067 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c003d00 con 0x7fae4415bfb0 2024-09-17T04:31:10.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.198+0000 7fae43200640 1 -- 172.21.15.100:0/4028970067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae44155840 con 0x7fae4415bfb0 2024-09-17T04:31:10.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.199+0000 7fae43200640 1 -- 172.21.15.100:0/4028970067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae4415d790 con 0x7fae4415bfb0 2024-09-17T04:31:10.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.200+0000 7fae3a000640 1 -- 172.21.15.100:0/4028970067 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fae3c004b20 con 0x7fae4415bfb0 2024-09-17T04:31:10.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.201+0000 7fae3a000640 1 --2- 172.21.15.100:0/4028970067 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fae24079010 0x7fae2407b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:10.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.201+0000 7fae3a000640 1 -- 172.21.15.100:0/4028970067 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5255+0+0 (secure 0 0 0) 0x7fae3c09c3a0 con 0x7fae4415bfb0 2024-09-17T04:31:10.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.201+0000 7fae41e00640 1 --2- 172.21.15.100:0/4028970067 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fae24079010 0x7fae2407b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:10.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.202+0000 7fae41e00640 1 --2- 172.21.15.100:0/4028970067 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fae24079010 0x7fae2407b520 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fae34009b90 tx=0x7fae34005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:10.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.205+0000 7fae3a000640 1 -- 172.21.15.100:0/4028970067 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fae3c064210 con 0x7fae4415bfb0 2024-09-17T04:31:10.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.390+0000 7fae43200640 1 -- 172.21.15.100:0/4028970067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fae44154d30 con 0x7fae4415bfb0 2024-09-17T04:31:10.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.898+0000 7fae3a000640 1 -- 172.21.15.100:0/4028970067 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v117) ==== 170+0+0 (secure 0 0 0) 0x7fae3c063b10 con 0x7fae4415bfb0 2024-09-17T04:31:10.899 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' created 2024-09-17T04:31:10.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.901+0000 7fae43200640 1 -- 172.21.15.100:0/4028970067 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fae24079010 msgr2=0x7fae2407b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:10.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.901+0000 7fae43200640 1 --2- 172.21.15.100:0/4028970067 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fae24079010 0x7fae2407b520 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fae34009b90 tx=0x7fae34005f90 comp rx=0 tx=0).stop 2024-09-17T04:31:10.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.901+0000 7fae43200640 1 -- 172.21.15.100:0/4028970067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae4415bfb0 msgr2=0x7fae441f8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:10.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.901+0000 7fae43200640 1 --2- 172.21.15.100:0/4028970067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae4415bfb0 0x7fae441f8d40 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fae440d00d0 tx=0x7fae3c010f40 comp rx=0 tx=0).stop 2024-09-17T04:31:10.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.901+0000 7fae43200640 1 -- 172.21.15.100:0/4028970067 shutdown_connections 2024-09-17T04:31:10.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.901+0000 7fae43200640 1 --2- 172.21.15.100:0/4028970067 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fae24079010 0x7fae2407b520 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:10.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.901+0000 7fae43200640 1 --2- 172.21.15.100:0/4028970067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae4415bfb0 0x7fae441f8d40 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:10.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.901+0000 7fae43200640 1 --2- 172.21.15.100:0/4028970067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae4415bba0 0x7fae441f8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:10.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.901+0000 7fae43200640 1 --2- 172.21.15.100:0/4028970067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae28004800 0x7fae44157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:10.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.901+0000 7fae43200640 1 -- 172.21.15.100:0/4028970067 >> 172.21.15.100:0/4028970067 conn(0x7fae440b1dd0 msgr2=0x7fae440b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:10.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.904+0000 7fae43200640 1 -- 172.21.15.100:0/4028970067 shutdown_connections 2024-09-17T04:31:10.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:10.909+0000 7fae43200640 1 -- 172.21.15.100:0/4028970067 wait complete. 2024-09-17T04:31:10.947 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T04:31:11.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.171+0000 7fcf18000640 1 -- 172.21.15.100:0/3225731439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf100d0fc0 msgr2=0x7fcf100cd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:11.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.171+0000 7fcf18000640 1 --2- 172.21.15.100:0/3225731439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf100d0fc0 0x7fcf100cd0e0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fcf08005fd0 tx=0x7fcf0802a4b0 comp rx=0 tx=0).stop 2024-09-17T04:31:11.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.171+0000 7fcf18000640 1 -- 172.21.15.100:0/3225731439 shutdown_connections 2024-09-17T04:31:11.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.171+0000 7fcf18000640 1 --2- 172.21.15.100:0/3225731439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf100d0fc0 0x7fcf100cd0e0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:11.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.171+0000 7fcf18000640 1 --2- 172.21.15.100:0/3225731439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf100b7d70 0x7fcf100d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:11.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.171+0000 7fcf18000640 1 -- 172.21.15.100:0/3225731439 >> 172.21.15.100:0/3225731439 conn(0x7fcf100cef20 msgr2=0x7fcf100cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:11.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.171+0000 7fcf18000640 1 -- 172.21.15.100:0/3225731439 shutdown_connections 2024-09-17T04:31:11.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.171+0000 7fcf18000640 1 -- 172.21.15.100:0/3225731439 wait complete. 2024-09-17T04:31:11.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.173+0000 7fcf18000640 1 Processor -- start 2024-09-17T04:31:11.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.174+0000 7fcf18000640 1 -- start start 2024-09-17T04:31:11.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.175+0000 7fcf18000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf100b7960 0x7fcf101f63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:11.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.175+0000 7fcf18000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf100b7d70 0x7fcf1015f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:11.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.175+0000 7fcf18000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf100d0fc0 0x7fcf10158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:11.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.175+0000 7fcf18000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcf100af900 con 0x7fcf100b7d70 2024-09-17T04:31:11.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.175+0000 7fcf18000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcf100d1480 con 0x7fcf100d0fc0 2024-09-17T04:31:11.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.175+0000 7fcf18000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcf1015e8e0 con 0x7fcf100b7960 2024-09-17T04:31:11.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.175+0000 7fcf14c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf100b7d70 0x7fcf1015f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:11.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.175+0000 7fcf14c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf100b7d70 0x7fcf1015f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57122/0 (socket says 172.21.15.100:57122) 2024-09-17T04:31:11.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.175+0000 7fcf14c00640 1 -- 172.21.15.100:0/2701151929 learned_addr learned my addr 172.21.15.100:0/2701151929 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:11.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.175+0000 7fcf15600640 1 --2- 172.21.15.100:0/2701151929 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf100b7960 0x7fcf101f63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:11.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.175+0000 7fcf16000640 1 --2- 172.21.15.100:0/2701151929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf100d0fc0 0x7fcf10158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:11.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.176+0000 7fcf14c00640 1 -- 172.21.15.100:0/2701151929 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf100b7960 msgr2=0x7fcf101f63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:11.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.176+0000 7fcf14c00640 1 --2- 172.21.15.100:0/2701151929 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf100b7960 0x7fcf101f63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:11.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.176+0000 7fcf14c00640 1 -- 172.21.15.100:0/2701151929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf100d0fc0 msgr2=0x7fcf10158350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:11.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.176+0000 7fcf14c00640 1 --2- 172.21.15.100:0/2701151929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf100d0fc0 0x7fcf10158350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:11.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.176+0000 7fcf14c00640 1 -- 172.21.15.100:0/2701151929 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1015f870 con 0x7fcf100b7d70 2024-09-17T04:31:11.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.176+0000 7fcf14c00640 1 --2- 172.21.15.100:0/2701151929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf100b7d70 0x7fcf1015f330 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00d6d0 tx=0x7fcefc00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:11.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.176+0000 7fcf0d600640 1 -- 172.21.15.100:0/2701151929 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc017070 con 0x7fcf100b7d70 2024-09-17T04:31:11.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.176+0000 7fcf0d600640 1 -- 172.21.15.100:0/2701151929 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcefc009b10 con 0x7fcf100b7d70 2024-09-17T04:31:11.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.176+0000 7fcf18000640 1 -- 172.21.15.100:0/2701151929 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf101567c0 con 0x7fcf100b7d70 2024-09-17T04:31:11.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.176+0000 7fcf0d600640 1 -- 172.21.15.100:0/2701151929 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc010460 con 0x7fcf100b7d70 2024-09-17T04:31:11.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.177+0000 7fcf18000640 1 -- 172.21.15.100:0/2701151929 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf10156d80 con 0x7fcf100b7d70 2024-09-17T04:31:11.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.177+0000 7fcf18000640 1 -- 172.21.15.100:0/2701151929 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf100d0a80 con 0x7fcf100b7d70 2024-09-17T04:31:11.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.179+0000 7fcf0d600640 1 -- 172.21.15.100:0/2701151929 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fcefc01a070 con 0x7fcf100b7d70 2024-09-17T04:31:11.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.180+0000 7fcf0d600640 1 --2- 172.21.15.100:0/2701151929 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcef0079010 0x7fcef007b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:11.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.180+0000 7fcf0d600640 1 -- 172.21.15.100:0/2701151929 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5637+0+0 (secure 0 0 0) 0x7fcefc09baf0 con 0x7fcf100b7d70 2024-09-17T04:31:11.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.180+0000 7fcf15600640 1 --2- 172.21.15.100:0/2701151929 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcef0079010 0x7fcef007b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:11.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.181+0000 7fcf15600640 1 --2- 172.21.15.100:0/2701151929 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcef0079010 0x7fcef007b520 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcf100d2470 tx=0x7fcf04009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:11.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.184+0000 7fcf0d600640 1 -- 172.21.15.100:0/2701151929 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fcefc063220 con 0x7fcf100b7d70 2024-09-17T04:31:11.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.401+0000 7fcf18000640 1 -- 172.21.15.100:0/2701151929 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fcf1006f730 con 0x7fcf100b7d70 2024-09-17T04:31:11.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.913+0000 7fcf0d600640 1 -- 172.21.15.100:0/2701151929 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v78) ==== 174+0+0 (secure 0 0 0) 0x7fcefc067c50 con 0x7fcf100b7d70 2024-09-17T04:31:11.917 INFO:teuthology.orchestra.run.smithi100.stderr:new fs with metadata pool 18 and data pool 19 2024-09-17T04:31:11.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.924+0000 7fcf18000640 1 -- 172.21.15.100:0/2701151929 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcef0079010 msgr2=0x7fcef007b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:11.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.924+0000 7fcf18000640 1 --2- 172.21.15.100:0/2701151929 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcef0079010 0x7fcef007b520 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcf100d2470 tx=0x7fcf04009040 comp rx=0 tx=0).stop 2024-09-17T04:31:11.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.924+0000 7fcf18000640 1 -- 172.21.15.100:0/2701151929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf100b7d70 msgr2=0x7fcf1015f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:11.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.924+0000 7fcf18000640 1 --2- 172.21.15.100:0/2701151929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf100b7d70 0x7fcf1015f330 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00d6d0 tx=0x7fcefc00db90 comp rx=0 tx=0).stop 2024-09-17T04:31:11.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.927+0000 7fcf18000640 1 -- 172.21.15.100:0/2701151929 shutdown_connections 2024-09-17T04:31:11.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.927+0000 7fcf18000640 1 --2- 172.21.15.100:0/2701151929 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcef0079010 0x7fcef007b520 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:11.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.927+0000 7fcf18000640 1 --2- 172.21.15.100:0/2701151929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf100d0fc0 0x7fcf10158350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:11.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.927+0000 7fcf18000640 1 --2- 172.21.15.100:0/2701151929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf100b7d70 0x7fcf1015f330 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:11.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.927+0000 7fcf18000640 1 --2- 172.21.15.100:0/2701151929 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf100b7960 0x7fcf101f63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:11.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.927+0000 7fcf18000640 1 -- 172.21.15.100:0/2701151929 >> 172.21.15.100:0/2701151929 conn(0x7fcf100cef20 msgr2=0x7fcf100cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:11.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.928+0000 7fcf18000640 1 -- 172.21.15.100:0/2701151929 shutdown_connections 2024-09-17T04:31:11.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:11.928+0000 7fcf18000640 1 -- 172.21.15.100:0/2701151929 wait complete. 2024-09-17T04:31:11.965 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:31:12.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.184+0000 7f0a7ce00640 1 -- 172.21.15.100:0/2240562561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a7806d2e0 msgr2=0x7f0a7815c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:12.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.184+0000 7f0a7ce00640 1 --2- 172.21.15.100:0/2240562561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a7806d2e0 0x7f0a7815c770 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f0a70005fd0 tx=0x7f0a7002aa90 comp rx=0 tx=0).stop 2024-09-17T04:31:12.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.184+0000 7f0a7ce00640 1 -- 172.21.15.100:0/2240562561 shutdown_connections 2024-09-17T04:31:12.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.184+0000 7f0a7ce00640 1 --2- 172.21.15.100:0/2240562561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a7806d2e0 0x7f0a7815c770 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:12.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.184+0000 7f0a7ce00640 1 --2- 172.21.15.100:0/2240562561 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a780b2360 0x7f0a7806cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:12.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.185+0000 7f0a7ce00640 1 -- 172.21.15.100:0/2240562561 >> 172.21.15.100:0/2240562561 conn(0x7f0a780b2fd0 msgr2=0x7f0a780b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:12.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.185+0000 7f0a7ce00640 1 -- 172.21.15.100:0/2240562561 shutdown_connections 2024-09-17T04:31:12.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.185+0000 7f0a7ce00640 1 -- 172.21.15.100:0/2240562561 wait complete. 2024-09-17T04:31:12.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.187+0000 7f0a7ce00640 1 Processor -- start 2024-09-17T04:31:12.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.187+0000 7f0a7ce00640 1 -- start start 2024-09-17T04:31:12.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.188+0000 7f0a7ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a7806d2e0 0x7f0a7809d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:12.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.188+0000 7f0a7ce00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a780b2360 0x7f0a7809cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:12.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.188+0000 7f0a7ce00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a780cd8f0 0x7f0a781e6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:12.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.188+0000 7f0a7ce00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0a780b5000 con 0x7f0a7806d2e0 2024-09-17T04:31:12.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.188+0000 7f0a7ce00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0a780b4e80 con 0x7f0a780cd8f0 2024-09-17T04:31:12.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.188+0000 7f0a7ce00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0a780b5180 con 0x7f0a780b2360 2024-09-17T04:31:12.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.189+0000 7f0a75e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a7806d2e0 0x7f0a7809d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:12.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.189+0000 7f0a75e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a7806d2e0 0x7f0a7809d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57140/0 (socket says 172.21.15.100:57140) 2024-09-17T04:31:12.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.189+0000 7f0a75e00640 1 -- 172.21.15.100:0/3201630210 learned_addr learned my addr 172.21.15.100:0/3201630210 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:12.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.189+0000 7f0a75e00640 1 -- 172.21.15.100:0/3201630210 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a780b2360 msgr2=0x7f0a7809cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:12.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.189+0000 7f0a75e00640 1 --2- 172.21.15.100:0/3201630210 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a780b2360 0x7f0a7809cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:12.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.189+0000 7f0a75e00640 1 -- 172.21.15.100:0/3201630210 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a780cd8f0 msgr2=0x7f0a781e6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:12.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.189+0000 7f0a75e00640 1 --2- 172.21.15.100:0/3201630210 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a780cd8f0 0x7f0a781e6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:12.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.189+0000 7f0a75e00640 1 -- 172.21.15.100:0/3201630210 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a781f5f60 con 0x7f0a7806d2e0 2024-09-17T04:31:12.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.189+0000 7f0a75e00640 1 --2- 172.21.15.100:0/3201630210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a7806d2e0 0x7f0a7809d6d0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800d740 tx=0x7f0a6800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:12.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.189+0000 7f0a6e000640 1 -- 172.21.15.100:0/3201630210 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a68017070 con 0x7f0a7806d2e0 2024-09-17T04:31:12.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.190+0000 7f0a6e000640 1 -- 172.21.15.100:0/3201630210 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a68009d20 con 0x7f0a7806d2e0 2024-09-17T04:31:12.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.190+0000 7f0a6e000640 1 -- 172.21.15.100:0/3201630210 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a680106c0 con 0x7f0a7806d2e0 2024-09-17T04:31:12.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.190+0000 7f0a7ce00640 1 -- 172.21.15.100:0/3201630210 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a781f6240 con 0x7f0a7806d2e0 2024-09-17T04:31:12.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.190+0000 7f0a7ce00640 1 -- 172.21.15.100:0/3201630210 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a781f6d20 con 0x7f0a7806d2e0 2024-09-17T04:31:12.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.191+0000 7f0a7ce00640 1 -- 172.21.15.100:0/3201630210 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a7806cda0 con 0x7f0a7806d2e0 2024-09-17T04:31:12.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.192+0000 7f0a6e000640 1 -- 172.21.15.100:0/3201630210 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f0a6801a070 con 0x7f0a7806d2e0 2024-09-17T04:31:12.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.193+0000 7f0a6e000640 1 --2- 172.21.15.100:0/3201630210 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0a58079040 0x7f0a5807b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:12.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.193+0000 7f0a6e000640 1 -- 172.21.15.100:0/3201630210 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5737+0+0 (secure 0 0 0) 0x7f0a6809bcc0 con 0x7f0a7806d2e0 2024-09-17T04:31:12.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.193+0000 7f0a75400640 1 --2- 172.21.15.100:0/3201630210 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0a58079040 0x7f0a5807b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:12.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.194+0000 7f0a75400640 1 --2- 172.21.15.100:0/3201630210 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0a58079040 0x7f0a5807b550 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0a64000f60 tx=0x7f0a640079b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:12.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.197+0000 7f0a6e000640 1 -- 172.21.15.100:0/3201630210 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0a68063990 con 0x7f0a7806d2e0 2024-09-17T04:31:12.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.379+0000 7f0a7ce00640 1 -- 172.21.15.100:0/3201630210 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0a78071330 con 0x7f0a7806d2e0 2024-09-17T04:31:12.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.381+0000 7f0a6e000640 1 -- 172.21.15.100:0/3201630210 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16844 (secure 0 0 0) 0x7f0a68063290 con 0x7f0a7806d2e0 2024-09-17T04:31:12.381 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:12.382 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":118,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:31:11.403009+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:31:09.303673+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-17T04:31:10.403511+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:31:12.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.383+0000 7f0a7ce00640 1 -- 172.21.15.100:0/3201630210 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0a58079040 msgr2=0x7f0a5807b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:12.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.383+0000 7f0a7ce00640 1 --2- 172.21.15.100:0/3201630210 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0a58079040 0x7f0a5807b550 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0a64000f60 tx=0x7f0a640079b0 comp rx=0 tx=0).stop 2024-09-17T04:31:12.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.383+0000 7f0a7ce00640 1 -- 172.21.15.100:0/3201630210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a7806d2e0 msgr2=0x7f0a7809d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:12.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.383+0000 7f0a7ce00640 1 --2- 172.21.15.100:0/3201630210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a7806d2e0 0x7f0a7809d6d0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800d740 tx=0x7f0a6800dc00 comp rx=0 tx=0).stop 2024-09-17T04:31:12.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.383+0000 7f0a7ce00640 1 -- 172.21.15.100:0/3201630210 shutdown_connections 2024-09-17T04:31:12.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.383+0000 7f0a7ce00640 1 --2- 172.21.15.100:0/3201630210 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0a58079040 0x7f0a5807b550 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:12.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.383+0000 7f0a7ce00640 1 --2- 172.21.15.100:0/3201630210 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a780cd8f0 0x7f0a781e6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:12.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.383+0000 7f0a7ce00640 1 --2- 172.21.15.100:0/3201630210 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a780b2360 0x7f0a7809cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:12.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.383+0000 7f0a7ce00640 1 --2- 172.21.15.100:0/3201630210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a7806d2e0 0x7f0a7809d6d0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:12.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.383+0000 7f0a7ce00640 1 -- 172.21.15.100:0/3201630210 >> 172.21.15.100:0/3201630210 conn(0x7f0a780b2fd0 msgr2=0x7f0a780b04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:12.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.383+0000 7f0a7ce00640 1 -- 172.21.15.100:0/3201630210 shutdown_connections 2024-09-17T04:31:12.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.383+0000 7f0a7ce00640 1 -- 172.21.15.100:0/3201630210 wait complete. 2024-09-17T04:31:12.407 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:31:12.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.628+0000 7f303e000640 1 -- 172.21.15.100:0/4075311550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30380b41f0 msgr2=0x7f303806f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:12.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.628+0000 7f303e000640 1 --2- 172.21.15.100:0/4075311550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30380b41f0 0x7f303806f7b0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f3030009be0 tx=0x7f303002a8e0 comp rx=0 tx=0).stop 2024-09-17T04:31:12.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.628+0000 7f303e000640 1 -- 172.21.15.100:0/4075311550 shutdown_connections 2024-09-17T04:31:12.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.628+0000 7f303e000640 1 --2- 172.21.15.100:0/4075311550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30380b41f0 0x7f303806f7b0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:12.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.628+0000 7f303e000640 1 -- 172.21.15.100:0/4075311550 >> 172.21.15.100:0/4075311550 conn(0x7f303806d730 msgr2=0x7f30380b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:12.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.628+0000 7f303e000640 1 -- 172.21.15.100:0/4075311550 shutdown_connections 2024-09-17T04:31:12.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.628+0000 7f303e000640 1 -- 172.21.15.100:0/4075311550 wait complete. 2024-09-17T04:31:12.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.630+0000 7f303e000640 1 Processor -- start 2024-09-17T04:31:12.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.630+0000 7f303e000640 1 -- start start 2024-09-17T04:31:12.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.632+0000 7f303e000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30380af630 0x7f3038145a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:12.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.632+0000 7f303e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30380b41f0 0x7f3038148c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:12.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.632+0000 7f303e000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3038147bb0 0x7f3038157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:12.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.632+0000 7f303e000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3038169da0 con 0x7f30380b41f0 2024-09-17T04:31:12.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.632+0000 7f303e000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3038169c20 con 0x7f30380af630 2024-09-17T04:31:12.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.632+0000 7f303e000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3038169f20 con 0x7f3038147bb0 2024-09-17T04:31:12.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.632+0000 7f302ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30380b41f0 0x7f3038148c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:12.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.632+0000 7f302ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30380b41f0 0x7f3038148c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57156/0 (socket says 172.21.15.100:57156) 2024-09-17T04:31:12.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.632+0000 7f302ea00640 1 -- 172.21.15.100:0/3732428998 learned_addr learned my addr 172.21.15.100:0/3732428998 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:12.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.632+0000 7f3037400640 1 --2- 172.21.15.100:0/3732428998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30380af630 0x7f3038145a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:12.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.632+0000 7f3037e00640 1 --2- 172.21.15.100:0/3732428998 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3038147bb0 0x7f3038157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:12.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.633+0000 7f302ea00640 1 -- 172.21.15.100:0/3732428998 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3038147bb0 msgr2=0x7f3038157340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:12.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.633+0000 7f302ea00640 1 --2- 172.21.15.100:0/3732428998 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3038147bb0 0x7f3038157340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:12.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.633+0000 7f302ea00640 1 -- 172.21.15.100:0/3732428998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30380af630 msgr2=0x7f3038145a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:12.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.633+0000 7f302ea00640 1 --2- 172.21.15.100:0/3732428998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30380af630 0x7f3038145a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:12.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.633+0000 7f302ea00640 1 -- 172.21.15.100:0/3732428998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30381589a0 con 0x7f30380b41f0 2024-09-17T04:31:12.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.633+0000 7f3037400640 1 --2- 172.21.15.100:0/3732428998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30380af630 0x7f3038145a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:31:12.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.633+0000 7f302ea00640 1 --2- 172.21.15.100:0/3732428998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30380b41f0 0x7f3038148c30 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f302000b3f0 tx=0x7f302000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:12.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.633+0000 7f302fe00640 1 -- 172.21.15.100:0/3732428998 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3020015020 con 0x7f30380b41f0 2024-09-17T04:31:12.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.633+0000 7f302fe00640 1 -- 172.21.15.100:0/3732428998 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3020011920 con 0x7f30380b41f0 2024-09-17T04:31:12.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.633+0000 7f303e000640 1 -- 172.21.15.100:0/3732428998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f303814b3c0 con 0x7f30380b41f0 2024-09-17T04:31:12.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.633+0000 7f302fe00640 1 -- 172.21.15.100:0/3732428998 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3020011d70 con 0x7f30380b41f0 2024-09-17T04:31:12.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.633+0000 7f303e000640 1 -- 172.21.15.100:0/3732428998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3038157880 con 0x7f30380b41f0 2024-09-17T04:31:12.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.634+0000 7f303e000640 1 -- 172.21.15.100:0/3732428998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3038147860 con 0x7f30380b41f0 2024-09-17T04:31:12.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.636+0000 7f302fe00640 1 -- 172.21.15.100:0/3732428998 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f3020018020 con 0x7f30380b41f0 2024-09-17T04:31:12.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.637+0000 7f302fe00640 1 --2- 172.21.15.100:0/3732428998 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3014079010 0x7f301407b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:12.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.637+0000 7f302fe00640 1 -- 172.21.15.100:0/3732428998 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5737+0+0 (secure 0 0 0) 0x7f302009b2b0 con 0x7f30380b41f0 2024-09-17T04:31:12.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.637+0000 7f3037400640 1 --2- 172.21.15.100:0/3732428998 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3014079010 0x7f301407b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:12.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.638+0000 7f3037400640 1 --2- 172.21.15.100:0/3732428998 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3014079010 0x7f301407b520 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f30280016a0 tx=0x7f3028009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:12.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.641+0000 7f302fe00640 1 -- 172.21.15.100:0/3732428998 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f302001f030 con 0x7f30380b41f0 2024-09-17T04:31:12.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.826+0000 7f303e000640 1 -- 172.21.15.100:0/3732428998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3038158b70 con 0x7f30380b41f0 2024-09-17T04:31:12.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.827+0000 7f302fe00640 1 -- 172.21.15.100:0/3732428998 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16844 (secure 0 0 0) 0x7f3020062f90 con 0x7f30380b41f0 2024-09-17T04:31:12.828 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:12.828 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":118,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:31:11.403009+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:31:09.303673+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-17T04:31:10.403511+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:31:12.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.829+0000 7f303e000640 1 -- 172.21.15.100:0/3732428998 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3014079010 msgr2=0x7f301407b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:12.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.829+0000 7f303e000640 1 --2- 172.21.15.100:0/3732428998 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3014079010 0x7f301407b520 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f30280016a0 tx=0x7f3028009040 comp rx=0 tx=0).stop 2024-09-17T04:31:12.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.830+0000 7f303e000640 1 -- 172.21.15.100:0/3732428998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30380b41f0 msgr2=0x7f3038148c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:12.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.830+0000 7f303e000640 1 --2- 172.21.15.100:0/3732428998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30380b41f0 0x7f3038148c30 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f302000b3f0 tx=0x7f302000b8b0 comp rx=0 tx=0).stop 2024-09-17T04:31:12.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.830+0000 7f303e000640 1 -- 172.21.15.100:0/3732428998 shutdown_connections 2024-09-17T04:31:12.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.830+0000 7f303e000640 1 --2- 172.21.15.100:0/3732428998 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3014079010 0x7f301407b520 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:12.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.830+0000 7f303e000640 1 --2- 172.21.15.100:0/3732428998 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3038147bb0 0x7f3038157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:12.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.830+0000 7f303e000640 1 --2- 172.21.15.100:0/3732428998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30380b41f0 0x7f3038148c30 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:12.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.830+0000 7f303e000640 1 --2- 172.21.15.100:0/3732428998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30380af630 0x7f3038145a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:12.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.830+0000 7f303e000640 1 -- 172.21.15.100:0/3732428998 >> 172.21.15.100:0/3732428998 conn(0x7f303806d730 msgr2=0x7f30380b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:12.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.830+0000 7f303e000640 1 -- 172.21.15.100:0/3732428998 shutdown_connections 2024-09-17T04:31:12.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:12.830+0000 7f303e000640 1 -- 172.21.15.100:0/3732428998 wait complete. 2024-09-17T04:31:12.852 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T04:31:13.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.077+0000 7fad12000640 1 -- 172.21.15.100:0/1803457498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad0c073c50 msgr2=0x7fad0c1674a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:13.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.077+0000 7fad12000640 1 --2- 172.21.15.100:0/1803457498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad0c073c50 0x7fad0c1674a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fad04009be0 tx=0x7fad0402a940 comp rx=0 tx=0).stop 2024-09-17T04:31:13.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.077+0000 7fad12000640 1 -- 172.21.15.100:0/1803457498 shutdown_connections 2024-09-17T04:31:13.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.077+0000 7fad12000640 1 --2- 172.21.15.100:0/1803457498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad0c073c50 0x7fad0c1674a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:13.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.077+0000 7fad12000640 1 -- 172.21.15.100:0/1803457498 >> 172.21.15.100:0/1803457498 conn(0x7fad0c06d1c0 msgr2=0x7fad0c06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:13.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.077+0000 7fad12000640 1 -- 172.21.15.100:0/1803457498 shutdown_connections 2024-09-17T04:31:13.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.077+0000 7fad12000640 1 -- 172.21.15.100:0/1803457498 wait complete. 2024-09-17T04:31:13.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.079+0000 7fad12000640 1 Processor -- start 2024-09-17T04:31:13.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.080+0000 7fad12000640 1 -- start start 2024-09-17T04:31:13.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.081+0000 7fad12000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad0c145550 0x7fad0c15c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:13.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.081+0000 7fad12000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad0c145a40 0x7fad0c0b0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:13.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.081+0000 7fad12000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad0c0b2ca0 0x7fad0c13b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:13.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.081+0000 7fad12000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c167df0 con 0x7fad0c145a40 2024-09-17T04:31:13.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.081+0000 7fad12000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c167c70 con 0x7fad0c0b2ca0 2024-09-17T04:31:13.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.081+0000 7fad12000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fad0c167f70 con 0x7fad0c145550 2024-09-17T04:31:13.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.081+0000 7fad0aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad0c145a40 0x7fad0c0b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:13.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.081+0000 7fad0aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad0c145a40 0x7fad0c0b0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57172/0 (socket says 172.21.15.100:57172) 2024-09-17T04:31:13.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.081+0000 7fad0aa00640 1 -- 172.21.15.100:0/3333112035 learned_addr learned my addr 172.21.15.100:0/3333112035 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:13.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.082+0000 7fad0be00640 1 --2- 172.21.15.100:0/3333112035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad0c0b2ca0 0x7fad0c13b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:13.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.082+0000 7fad0b400640 1 --2- 172.21.15.100:0/3333112035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad0c145550 0x7fad0c15c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:13.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.082+0000 7fad0aa00640 1 -- 172.21.15.100:0/3333112035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad0c145550 msgr2=0x7fad0c15c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:13.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.082+0000 7fad0aa00640 1 --2- 172.21.15.100:0/3333112035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad0c145550 0x7fad0c15c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:13.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.082+0000 7fad0aa00640 1 -- 172.21.15.100:0/3333112035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad0c0b2ca0 msgr2=0x7fad0c13b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:13.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.082+0000 7fad0aa00640 1 --2- 172.21.15.100:0/3333112035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad0c0b2ca0 0x7fad0c13b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:13.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.082+0000 7fad0aa00640 1 -- 172.21.15.100:0/3333112035 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad0c0b06a0 con 0x7fad0c145a40 2024-09-17T04:31:13.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.082+0000 7fad0b400640 1 --2- 172.21.15.100:0/3333112035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad0c145550 0x7fad0c15c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:31:13.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.082+0000 7fad0aa00640 1 --2- 172.21.15.100:0/3333112035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad0c145a40 0x7fad0c0b0160 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7facf800c700 tx=0x7facf800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:13.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.082+0000 7fad03400640 1 -- 172.21.15.100:0/3333112035 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8016020 con 0x7fad0c145a40 2024-09-17T04:31:13.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.082+0000 7fad03400640 1 -- 172.21.15.100:0/3333112035 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facf800a930 con 0x7fad0c145a40 2024-09-17T04:31:13.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.082+0000 7fad12000640 1 -- 172.21.15.100:0/3333112035 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad0c13a7b0 con 0x7fad0c145a40 2024-09-17T04:31:13.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.082+0000 7fad03400640 1 -- 172.21.15.100:0/3333112035 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf800ad40 con 0x7fad0c145a40 2024-09-17T04:31:13.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.082+0000 7fad12000640 1 -- 172.21.15.100:0/3333112035 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad0c13ad70 con 0x7fad0c145a40 2024-09-17T04:31:13.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.083+0000 7fad12000640 1 -- 172.21.15.100:0/3333112035 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad0c06f100 con 0x7fad0c145a40 2024-09-17T04:31:13.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.085+0000 7fad03400640 1 -- 172.21.15.100:0/3333112035 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7facf8019020 con 0x7fad0c145a40 2024-09-17T04:31:13.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.086+0000 7fad03400640 1 --2- 172.21.15.100:0/3333112035 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7facec079010 0x7facec07b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:13.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.086+0000 7fad03400640 1 -- 172.21.15.100:0/3333112035 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7facf809b400 con 0x7fad0c145a40 2024-09-17T04:31:13.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.086+0000 7fad0b400640 1 --2- 172.21.15.100:0/3333112035 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7facec079010 0x7facec07b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:13.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.086+0000 7fad0b400640 1 --2- 172.21.15.100:0/3333112035 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7facec079010 0x7facec07b520 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7facfc001120 tx=0x7facfc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:13.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.090+0000 7fad03400640 1 -- 172.21.15.100:0/3333112035 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7facf8020030 con 0x7fad0c145a40 2024-09-17T04:31:13.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.307+0000 7fad12000640 1 -- 172.21.15.100:0/3333112035 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fad0c06fb40 con 0x7fad0c145a40 2024-09-17T04:31:13.889 INFO:tasks.ceph.mds.c.smithi100.stderr:2024-09-17T04:31:13.888+0000 7f1e05600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T04:31:13.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.928+0000 7fad03400640 1 -- 172.21.15.100:0/3333112035 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v81) ==== 118+0+0 (secure 0 0 0) 0x7facf8063130 con 0x7fad0c145a40 2024-09-17T04:31:13.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.931+0000 7fad12000640 1 -- 172.21.15.100:0/3333112035 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7facec079010 msgr2=0x7facec07b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:13.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.931+0000 7fad12000640 1 --2- 172.21.15.100:0/3333112035 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7facec079010 0x7facec07b520 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7facfc001120 tx=0x7facfc009040 comp rx=0 tx=0).stop 2024-09-17T04:31:13.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.931+0000 7fad12000640 1 -- 172.21.15.100:0/3333112035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad0c145a40 msgr2=0x7fad0c0b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:13.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.931+0000 7fad12000640 1 --2- 172.21.15.100:0/3333112035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad0c145a40 0x7fad0c0b0160 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7facf800c700 tx=0x7facf800cbc0 comp rx=0 tx=0).stop 2024-09-17T04:31:13.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.931+0000 7fad12000640 1 -- 172.21.15.100:0/3333112035 shutdown_connections 2024-09-17T04:31:13.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.931+0000 7fad12000640 1 --2- 172.21.15.100:0/3333112035 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7facec079010 0x7facec07b520 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:13.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.931+0000 7fad12000640 1 --2- 172.21.15.100:0/3333112035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad0c0b2ca0 0x7fad0c13b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:13.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.931+0000 7fad12000640 1 --2- 172.21.15.100:0/3333112035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad0c145a40 0x7fad0c0b0160 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:13.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.931+0000 7fad12000640 1 --2- 172.21.15.100:0/3333112035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad0c145550 0x7fad0c15c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:13.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.931+0000 7fad12000640 1 -- 172.21.15.100:0/3333112035 >> 172.21.15.100:0/3333112035 conn(0x7fad0c06d1c0 msgr2=0x7fad0c13f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:13.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.931+0000 7fad12000640 1 -- 172.21.15.100:0/3333112035 shutdown_connections 2024-09-17T04:31:13.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:13.931+0000 7fad12000640 1 -- 172.21.15.100:0/3333112035 wait complete. 2024-09-17T04:31:13.956 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T04:31:13.956 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T04:31:14.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.181+0000 7f0966800640 1 -- 172.21.15.100:0/508427982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0960073c50 msgr2=0x7f09600af210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:14.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.181+0000 7f0966800640 1 --2- 172.21.15.100:0/508427982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0960073c50 0x7f09600af210 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f0960072bd0 tx=0x7f095802a9a0 comp rx=0 tx=0).stop 2024-09-17T04:31:14.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.181+0000 7f0966800640 1 -- 172.21.15.100:0/508427982 shutdown_connections 2024-09-17T04:31:14.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.181+0000 7f0966800640 1 --2- 172.21.15.100:0/508427982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0960073c50 0x7f09600af210 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:14.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.181+0000 7f0966800640 1 --2- 172.21.15.100:0/508427982 >> v2:172.21.15.184:3300/0 conn(0x7f096015f9e0 0x7f0960147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:14.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.181+0000 7f0966800640 1 -- 172.21.15.100:0/508427982 >> 172.21.15.100:0/508427982 conn(0x7f096006d1c0 msgr2=0x7f096006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:14.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.181+0000 7f0966800640 1 -- 172.21.15.100:0/508427982 shutdown_connections 2024-09-17T04:31:14.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.182+0000 7f0966800640 1 -- 172.21.15.100:0/508427982 wait complete. 2024-09-17T04:31:14.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.184+0000 7f0966800640 1 Processor -- start 2024-09-17T04:31:14.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.184+0000 7f0966800640 1 -- start start 2024-09-17T04:31:14.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.185+0000 7f0966800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09600aee00 0x7f09601f5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:14.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.185+0000 7f0966800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f096015f9e0 0x7f0960141700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:14.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.185+0000 7f0966800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09600ceda0 0x7f0960143530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:14.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.185+0000 7f0966800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f096006f9e0 con 0x7f096015f9e0 2024-09-17T04:31:14.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.185+0000 7f0966800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f09601484e0 con 0x7f09600aee00 2024-09-17T04:31:14.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.185+0000 7f0966800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f096006fb60 con 0x7f09600ceda0 2024-09-17T04:31:14.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.185+0000 7f095ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f096015f9e0 0x7f0960141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:14.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.186+0000 7f095ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f096015f9e0 0x7f0960141700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57192/0 (socket says 172.21.15.100:57192) 2024-09-17T04:31:14.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.186+0000 7f095ea00640 1 -- 172.21.15.100:0/4000425532 learned_addr learned my addr 172.21.15.100:0/4000425532 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:14.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.186+0000 7f095f400640 1 --2- 172.21.15.100:0/4000425532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09600aee00 0x7f09601f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:14.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.186+0000 7f095ea00640 1 -- 172.21.15.100:0/4000425532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09600ceda0 msgr2=0x7f0960143530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:14.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.186+0000 7f095ea00640 1 --2- 172.21.15.100:0/4000425532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09600ceda0 0x7f0960143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:14.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.186+0000 7f095ea00640 1 -- 172.21.15.100:0/4000425532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09600aee00 msgr2=0x7f09601f5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:14.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.186+0000 7f095ea00640 1 --2- 172.21.15.100:0/4000425532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09600aee00 0x7f09601f5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:14.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.186+0000 7f095ea00640 1 -- 172.21.15.100:0/4000425532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0960141f30 con 0x7f096015f9e0 2024-09-17T04:31:14.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.186+0000 7f095f400640 1 --2- 172.21.15.100:0/4000425532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09600aee00 0x7f09601f5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:31:14.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.186+0000 7f095ea00640 1 --2- 172.21.15.100:0/4000425532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f096015f9e0 0x7f0960141700 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f094c00c3a0 tx=0x7f094c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:14.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.186+0000 7f0957400640 1 -- 172.21.15.100:0/4000425532 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c017070 con 0x7f096015f9e0 2024-09-17T04:31:14.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.186+0000 7f0957400640 1 -- 172.21.15.100:0/4000425532 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f094c0053c0 con 0x7f096015f9e0 2024-09-17T04:31:14.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.186+0000 7f0966800640 1 -- 172.21.15.100:0/4000425532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0960142210 con 0x7f096015f9e0 2024-09-17T04:31:14.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.187+0000 7f0957400640 1 -- 172.21.15.100:0/4000425532 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c003850 con 0x7f096015f9e0 2024-09-17T04:31:14.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.187+0000 7f0966800640 1 -- 172.21.15.100:0/4000425532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09601427d0 con 0x7f096015f9e0 2024-09-17T04:31:14.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.188+0000 7f0966800640 1 -- 172.21.15.100:0/4000425532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0960070050 con 0x7f096015f9e0 2024-09-17T04:31:14.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.189+0000 7f0957400640 1 -- 172.21.15.100:0/4000425532 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f094c01a070 con 0x7f096015f9e0 2024-09-17T04:31:14.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.190+0000 7f0957400640 1 --2- 172.21.15.100:0/4000425532 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0940079010 0x7f094007b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:14.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.190+0000 7f0957400640 1 -- 172.21.15.100:0/4000425532 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7f094c09aa40 con 0x7f096015f9e0 2024-09-17T04:31:14.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.190+0000 7f095f400640 1 --2- 172.21.15.100:0/4000425532 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0940079010 0x7f094007b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:14.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.191+0000 7f095f400640 1 --2- 172.21.15.100:0/4000425532 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0940079010 0x7f094007b520 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f09600b3240 tx=0x7f0950005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:14.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.194+0000 7f0957400640 1 -- 172.21.15.100:0/4000425532 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f094c0626e0 con 0x7f096015f9e0 2024-09-17T04:31:14.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.412+0000 7f0966800640 1 -- 172.21.15.100:0/4000425532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f0960141cc0 con 0x7f096015f9e0 2024-09-17T04:31:14.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.930+0000 7f0957400640 1 -- 172.21.15.100:0/4000425532 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v82) ==== 122+0+0 (secure 0 0 0) 0x7f094c061fe0 con 0x7f096015f9e0 2024-09-17T04:31:14.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.932+0000 7f0966800640 1 -- 172.21.15.100:0/4000425532 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0940079010 msgr2=0x7f094007b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:14.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.932+0000 7f0966800640 1 --2- 172.21.15.100:0/4000425532 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0940079010 0x7f094007b520 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f09600b3240 tx=0x7f0950005f90 comp rx=0 tx=0).stop 2024-09-17T04:31:14.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.932+0000 7f0966800640 1 -- 172.21.15.100:0/4000425532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f096015f9e0 msgr2=0x7f0960141700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:14.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.932+0000 7f0966800640 1 --2- 172.21.15.100:0/4000425532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f096015f9e0 0x7f0960141700 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f094c00c3a0 tx=0x7f094c00c860 comp rx=0 tx=0).stop 2024-09-17T04:31:14.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.932+0000 7f0966800640 1 -- 172.21.15.100:0/4000425532 shutdown_connections 2024-09-17T04:31:14.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.932+0000 7f0966800640 1 --2- 172.21.15.100:0/4000425532 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0940079010 0x7f094007b520 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:14.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.932+0000 7f0966800640 1 --2- 172.21.15.100:0/4000425532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09600ceda0 0x7f0960143530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:14.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.932+0000 7f0966800640 1 --2- 172.21.15.100:0/4000425532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f096015f9e0 0x7f0960141700 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:14.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.932+0000 7f0966800640 1 --2- 172.21.15.100:0/4000425532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09600aee00 0x7f09601f5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:14.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.932+0000 7f0966800640 1 -- 172.21.15.100:0/4000425532 >> 172.21.15.100:0/4000425532 conn(0x7f096006d1c0 msgr2=0x7f0960147840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:14.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.933+0000 7f0966800640 1 -- 172.21.15.100:0/4000425532 shutdown_connections 2024-09-17T04:31:14.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:14.933+0000 7f0966800640 1 -- 172.21.15.100:0/4000425532 wait complete. 2024-09-17T04:31:14.957 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T04:31:14.957 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:31:15.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.177+0000 7fa0f8a00640 1 -- 172.21.15.100:0/3259071297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0d8004d90 msgr2=0x7fa0d8005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:15.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.177+0000 7fa0f8a00640 1 --2- 172.21.15.100:0/3259071297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0d8004d90 0x7fa0d8005220 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec001f00 tx=0x7fa0d8005830 comp rx=0 tx=0).stop 2024-09-17T04:31:15.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.177+0000 7fa0f3200640 1 -- 172.21.15.100:0/3259071297 reap_dead start 2024-09-17T04:31:15.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.178+0000 7fa0f8a00640 1 -- 172.21.15.100:0/3259071297 shutdown_connections 2024-09-17T04:31:15.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.178+0000 7fa0f8a00640 1 -- 172.21.15.100:0/3259071297 >> 172.21.15.100:0/3259071297 conn(0x7fa0f408bfe0 msgr2=0x7fa0f409d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:15.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.178+0000 7fa0f8a00640 1 -- 172.21.15.100:0/3259071297 shutdown_connections 2024-09-17T04:31:15.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.178+0000 7fa0f8a00640 1 -- 172.21.15.100:0/3259071297 wait complete. 2024-09-17T04:31:15.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.180+0000 7fa0f8a00640 1 Processor -- start 2024-09-17T04:31:15.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.180+0000 7fa0f8a00640 1 -- start start 2024-09-17T04:31:15.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.181+0000 7fa0f8a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0d8004d90 0x7fa0f4161840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:15.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.181+0000 7fa0f8a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0f40a1eb0 0x7fa0f41f8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:15.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.181+0000 7fa0f8a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0f41f93c0 0x7fa0f409f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:15.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.182+0000 7fa0f8a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f40d1b10 con 0x7fa0f40a1eb0 2024-09-17T04:31:15.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.182+0000 7fa0f8a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f40d1990 con 0x7fa0d8004d90 2024-09-17T04:31:15.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.182+0000 7fa0f8a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa0f40d1c90 con 0x7fa0f41f93c0 2024-09-17T04:31:15.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.182+0000 7fa0f1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0f40a1eb0 0x7fa0f41f8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:15.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.182+0000 7fa0f1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0f40a1eb0 0x7fa0f41f8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55368/0 (socket says 172.21.15.100:55368) 2024-09-17T04:31:15.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.182+0000 7fa0f1e00640 1 -- 172.21.15.100:0/2441758867 learned_addr learned my addr 172.21.15.100:0/2441758867 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:15.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.182+0000 7fa0f2800640 1 --2- 172.21.15.100:0/2441758867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0d8004d90 0x7fa0f4161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:15.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.182+0000 7fa0f3200640 1 --2- 172.21.15.100:0/2441758867 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0f41f93c0 0x7fa0f409f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:15.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.182+0000 7fa0f1e00640 1 -- 172.21.15.100:0/2441758867 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0f41f93c0 msgr2=0x7fa0f409f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:15.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.182+0000 7fa0f1e00640 1 --2- 172.21.15.100:0/2441758867 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0f41f93c0 0x7fa0f409f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:15.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.182+0000 7fa0f1e00640 1 -- 172.21.15.100:0/2441758867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0d8004d90 msgr2=0x7fa0f4161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:15.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.182+0000 7fa0f1e00640 1 --2- 172.21.15.100:0/2441758867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0d8004d90 0x7fa0f4161840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:15.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.182+0000 7fa0f1e00640 1 -- 172.21.15.100:0/2441758867 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f40cc450 con 0x7fa0f40a1eb0 2024-09-17T04:31:15.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.182+0000 7fa0f3200640 1 --2- 172.21.15.100:0/2441758867 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0f41f93c0 0x7fa0f409f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:31:15.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.182+0000 7fa0f1e00640 1 --2- 172.21.15.100:0/2441758867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0f40a1eb0 0x7fa0f41f8e80 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0001ca0 tx=0x7fa0e00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:15.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.183+0000 7fa0eaa00640 1 -- 172.21.15.100:0/2441758867 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0015070 con 0x7fa0f40a1eb0 2024-09-17T04:31:15.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.183+0000 7fa0eaa00640 1 -- 172.21.15.100:0/2441758867 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0e0011900 con 0x7fa0f40a1eb0 2024-09-17T04:31:15.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.183+0000 7fa0eaa00640 1 -- 172.21.15.100:0/2441758867 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0011d50 con 0x7fa0f40a1eb0 2024-09-17T04:31:15.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.183+0000 7fa0f8a00640 1 -- 172.21.15.100:0/2441758867 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0f40cc730 con 0x7fa0f40a1eb0 2024-09-17T04:31:15.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.183+0000 7fa0f8a00640 1 -- 172.21.15.100:0/2441758867 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0f40cccf0 con 0x7fa0f40a1eb0 2024-09-17T04:31:15.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.184+0000 7fa0f8a00640 1 -- 172.21.15.100:0/2441758867 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0f40cc1a0 con 0x7fa0f40a1eb0 2024-09-17T04:31:15.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.186+0000 7fa0eaa00640 1 -- 172.21.15.100:0/2441758867 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fa0e0018070 con 0x7fa0f40a1eb0 2024-09-17T04:31:15.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.187+0000 7fa0eaa00640 1 --2- 172.21.15.100:0/2441758867 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa0d4079040 0x7fa0d407b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:15.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.187+0000 7fa0eaa00640 1 -- 172.21.15.100:0/2441758867 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7fa0e009c640 con 0x7fa0f40a1eb0 2024-09-17T04:31:15.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.187+0000 7fa0f2800640 1 --2- 172.21.15.100:0/2441758867 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa0d4079040 0x7fa0d407b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:15.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.188+0000 7fa0f2800640 1 --2- 172.21.15.100:0/2441758867 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa0d4079040 0x7fa0d407b550 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009b90 tx=0x7fa0e4005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:15.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.190+0000 7fa0eaa00640 1 -- 172.21.15.100:0/2441758867 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa0e0064390 con 0x7fa0f40a1eb0 2024-09-17T04:31:15.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.411+0000 7fa0f8a00640 1 -- 172.21.15.100:0/2441758867 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0f40d24a0 con 0x7fa0f40a1eb0 2024-09-17T04:31:15.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.411+0000 7fa0eaa00640 1 -- 172.21.15.100:0/2441758867 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5234 (secure 0 0 0) 0x7fa0e0063c90 con 0x7fa0f40a1eb0 2024-09-17T04:31:15.416 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:15.416 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":82,"btime":"2024-09-17T04:31:14:929364+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6410,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1845046345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1845046345},{"type":"v1","addr":"172.21.15.184:6801","nonce":1845046345}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7135,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3970003794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3970003794},{"type":"v1","addr":"172.21.15.184:6835","nonce":3970003794}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/3663836388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3663836388},{"type":"v1","addr":"172.21.15.100:6834","nonce":3663836388}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:31:11.402955+0000","modified":"2024-09-17T04:31:14.414638+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7542":{"gid":7542,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/207356836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":207356836},{"type":"v1","addr":"172.21.15.100:6835","nonce":207356836}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7542,"qdb_cluster":[7542]},"id":9}]} 2024-09-17T04:31:15.416 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 82 2024-09-17T04:31:15.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.414+0000 7fa0f8a00640 1 -- 172.21.15.100:0/2441758867 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa0d4079040 msgr2=0x7fa0d407b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:15.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.414+0000 7fa0f8a00640 1 --2- 172.21.15.100:0/2441758867 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa0d4079040 0x7fa0d407b550 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009b90 tx=0x7fa0e4005f90 comp rx=0 tx=0).stop 2024-09-17T04:31:15.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.414+0000 7fa0f8a00640 1 -- 172.21.15.100:0/2441758867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0f40a1eb0 msgr2=0x7fa0f41f8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:15.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.414+0000 7fa0f8a00640 1 --2- 172.21.15.100:0/2441758867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0f40a1eb0 0x7fa0f41f8e80 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0001ca0 tx=0x7fa0e00039c0 comp rx=0 tx=0).stop 2024-09-17T04:31:15.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.414+0000 7fa0f8a00640 1 -- 172.21.15.100:0/2441758867 shutdown_connections 2024-09-17T04:31:15.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.414+0000 7fa0f8a00640 1 --2- 172.21.15.100:0/2441758867 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa0d4079040 0x7fa0d407b550 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:15.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.414+0000 7fa0f8a00640 1 --2- 172.21.15.100:0/2441758867 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0f41f93c0 0x7fa0f409f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:15.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.414+0000 7fa0f8a00640 1 --2- 172.21.15.100:0/2441758867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0f40a1eb0 0x7fa0f41f8e80 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:15.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.414+0000 7fa0f8a00640 1 --2- 172.21.15.100:0/2441758867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0d8004d90 0x7fa0f4161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:15.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.414+0000 7fa0f8a00640 1 -- 172.21.15.100:0/2441758867 >> 172.21.15.100:0/2441758867 conn(0x7fa0f408bfe0 msgr2=0x7fa0f40cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:15.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.414+0000 7fa0f8a00640 1 -- 172.21.15.100:0/2441758867 shutdown_connections 2024-09-17T04:31:15.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.414+0000 7fa0f8a00640 1 -- 172.21.15.100:0/2441758867 wait complete. 2024-09-17T04:31:15.438 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:31:15.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.658+0000 7f0af3800640 1 -- 172.21.15.100:0/393981636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aec06f5e0 msgr2=0x7f0aec162560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:15.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.658+0000 7f0af3800640 1 --2- 172.21.15.100:0/393981636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aec06f5e0 0x7f0aec162560 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f0aec072810 tx=0x7f0ae402aa10 comp rx=0 tx=0).stop 2024-09-17T04:31:15.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.658+0000 7f0af3800640 1 -- 172.21.15.100:0/393981636 shutdown_connections 2024-09-17T04:31:15.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.658+0000 7f0af3800640 1 --2- 172.21.15.100:0/393981636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aec06f5e0 0x7f0aec162560 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:15.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.658+0000 7f0af3800640 1 --2- 172.21.15.100:0/393981636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0aec073a40 0x7f0aec06f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:15.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.658+0000 7f0af3800640 1 --2- 172.21.15.100:0/393981636 >> v2:172.21.15.184:3300/0 conn(0x7f0aec0af630 0x7f0aec073500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:15.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.658+0000 7f0af3800640 1 -- 172.21.15.100:0/393981636 >> 172.21.15.100:0/393981636 conn(0x7f0aec06d730 msgr2=0x7f0aec0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:15.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.659+0000 7f0af3800640 1 -- 172.21.15.100:0/393981636 shutdown_connections 2024-09-17T04:31:15.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.659+0000 7f0af3800640 1 -- 172.21.15.100:0/393981636 wait complete. 2024-09-17T04:31:15.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.661+0000 7f0af3800640 1 Processor -- start 2024-09-17T04:31:15.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.661+0000 7f0af3800640 1 -- start start 2024-09-17T04:31:15.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.662+0000 7f0af3800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0aec06f5e0 0x7f0aec151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:15.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.662+0000 7f0af3800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aec073a40 0x7f0aec15cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:15.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.662+0000 7f0af3800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0aec0af630 0x7f0aec15d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:15.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.662+0000 7f0af3800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0aec162e30 con 0x7f0aec073a40 2024-09-17T04:31:15.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.662+0000 7f0af3800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0aec162cb0 con 0x7f0aec0af630 2024-09-17T04:31:15.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.662+0000 7f0af3800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0aec162fb0 con 0x7f0aec06f5e0 2024-09-17T04:31:15.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.663+0000 7f0aebe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aec073a40 0x7f0aec15cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:15.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.663+0000 7f0aebe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aec073a40 0x7f0aec15cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55384/0 (socket says 172.21.15.100:55384) 2024-09-17T04:31:15.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.663+0000 7f0aebe00640 1 -- 172.21.15.100:0/3215847443 learned_addr learned my addr 172.21.15.100:0/3215847443 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:15.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.663+0000 7f0af1800640 1 --2- 172.21.15.100:0/3215847443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0aec0af630 0x7f0aec15d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:15.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.663+0000 7f0af0e00640 1 --2- 172.21.15.100:0/3215847443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0aec06f5e0 0x7f0aec151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:15.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.663+0000 7f0aebe00640 1 -- 172.21.15.100:0/3215847443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0aec06f5e0 msgr2=0x7f0aec151a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:15.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.663+0000 7f0aebe00640 1 --2- 172.21.15.100:0/3215847443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0aec06f5e0 0x7f0aec151a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:15.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.663+0000 7f0aebe00640 1 -- 172.21.15.100:0/3215847443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0aec0af630 msgr2=0x7f0aec15d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:15.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.663+0000 7f0aebe00640 1 --2- 172.21.15.100:0/3215847443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0aec0af630 0x7f0aec15d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:15.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.663+0000 7f0aebe00640 1 -- 172.21.15.100:0/3215847443 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aec151f50 con 0x7f0aec073a40 2024-09-17T04:31:15.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.663+0000 7f0aebe00640 1 --2- 172.21.15.100:0/3215847443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aec073a40 0x7f0aec15cf30 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f0ad800c970 tx=0x7f0ad800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:15.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.664+0000 7f0ae8c00640 1 -- 172.21.15.100:0/3215847443 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8007bb0 con 0x7f0aec073a40 2024-09-17T04:31:15.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.664+0000 7f0ae8c00640 1 -- 172.21.15.100:0/3215847443 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ad8007e40 con 0x7f0aec073a40 2024-09-17T04:31:15.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.664+0000 7f0ae8c00640 1 -- 172.21.15.100:0/3215847443 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad800f6b0 con 0x7f0aec073a40 2024-09-17T04:31:15.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.664+0000 7f0af3800640 1 -- 172.21.15.100:0/3215847443 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aec1f8ef0 con 0x7f0aec073a40 2024-09-17T04:31:15.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.664+0000 7f0af3800640 1 -- 172.21.15.100:0/3215847443 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aec156f40 con 0x7f0aec073a40 2024-09-17T04:31:15.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.665+0000 7f0af3800640 1 -- 172.21.15.100:0/3215847443 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aec073820 con 0x7f0aec073a40 2024-09-17T04:31:15.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.666+0000 7f0ae8c00640 1 -- 172.21.15.100:0/3215847443 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f0ad800f850 con 0x7f0aec073a40 2024-09-17T04:31:15.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.667+0000 7f0ae8c00640 1 --2- 172.21.15.100:0/3215847443 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0acc079040 0x7f0acc07b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:15.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.667+0000 7f0ae8c00640 1 -- 172.21.15.100:0/3215847443 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7f0ad809be80 con 0x7f0aec073a40 2024-09-17T04:31:15.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.667+0000 7f0af0e00640 1 --2- 172.21.15.100:0/3215847443 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0acc079040 0x7f0acc07b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:15.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.668+0000 7f0af0e00640 1 --2- 172.21.15.100:0/3215847443 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0acc079040 0x7f0acc07b550 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0ae000ca40 tx=0x7f0ae000c420 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:15.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.672+0000 7f0ae8c00640 1 -- 172.21.15.100:0/3215847443 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0ad8014030 con 0x7f0aec073a40 2024-09-17T04:31:15.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.856+0000 7f0af3800640 1 -- 172.21.15.100:0/3215847443 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0aec0adc40 con 0x7f0aec073a40 2024-09-17T04:31:15.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.857+0000 7f0ae8c00640 1 -- 172.21.15.100:0/3215847443 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+16831 (secure 0 0 0) 0x7f0ad8063b20 con 0x7f0aec073a40 2024-09-17T04:31:15.858 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:15.858 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":119,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:31:12.901747+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:31:09.303673+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-17T04:31:10.403511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:31:15.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.860+0000 7f0af3800640 1 -- 172.21.15.100:0/3215847443 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0acc079040 msgr2=0x7f0acc07b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:15.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.860+0000 7f0af3800640 1 --2- 172.21.15.100:0/3215847443 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0acc079040 0x7f0acc07b550 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0ae000ca40 tx=0x7f0ae000c420 comp rx=0 tx=0).stop 2024-09-17T04:31:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.860+0000 7f0af3800640 1 -- 172.21.15.100:0/3215847443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aec073a40 msgr2=0x7f0aec15cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.860+0000 7f0af3800640 1 --2- 172.21.15.100:0/3215847443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aec073a40 0x7f0aec15cf30 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f0ad800c970 tx=0x7f0ad800ce30 comp rx=0 tx=0).stop 2024-09-17T04:31:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.860+0000 7f0af3800640 1 -- 172.21.15.100:0/3215847443 shutdown_connections 2024-09-17T04:31:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.860+0000 7f0af3800640 1 --2- 172.21.15.100:0/3215847443 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0acc079040 0x7f0acc07b550 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.860+0000 7f0af3800640 1 --2- 172.21.15.100:0/3215847443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0aec0af630 0x7f0aec15d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.860+0000 7f0af3800640 1 --2- 172.21.15.100:0/3215847443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0aec073a40 0x7f0aec15cf30 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.860+0000 7f0af3800640 1 --2- 172.21.15.100:0/3215847443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0aec06f5e0 0x7f0aec151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.860+0000 7f0af3800640 1 -- 172.21.15.100:0/3215847443 >> 172.21.15.100:0/3215847443 conn(0x7f0aec06d730 msgr2=0x7f0aec0b3170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.860+0000 7f0af3800640 1 -- 172.21.15.100:0/3215847443 shutdown_connections 2024-09-17T04:31:15.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:15.860+0000 7f0af3800640 1 -- 172.21.15.100:0/3215847443 wait complete. 2024-09-17T04:31:15.883 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T04:31:15.884 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:31:16.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.106+0000 7f93ca200640 1 -- 172.21.15.100:0/436462095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93a8003c30 msgr2=0x7f93a8004080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:16.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.106+0000 7f93ca200640 1 --2- 172.21.15.100:0/436462095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93a8003c30 0x7f93a8004080 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009b50 tx=0x7f93a8005830 comp rx=0 tx=0).stop 2024-09-17T04:31:16.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.106+0000 7f93ca200640 1 -- 172.21.15.100:0/436462095 shutdown_connections 2024-09-17T04:31:16.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.106+0000 7f93ca200640 1 --2- 172.21.15.100:0/436462095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93a8003c30 0x7f93a8004080 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:16.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.106+0000 7f93ca200640 1 -- 172.21.15.100:0/436462095 >> 172.21.15.100:0/436462095 conn(0x7f93c409d820 msgr2=0x7f93c408bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:16.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.106+0000 7f93ca200640 1 -- 172.21.15.100:0/436462095 shutdown_connections 2024-09-17T04:31:16.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.106+0000 7f93ca200640 1 -- 172.21.15.100:0/436462095 wait complete. 2024-09-17T04:31:16.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.108+0000 7f93ca200640 1 Processor -- start 2024-09-17T04:31:16.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.108+0000 7f93ca200640 1 -- start start 2024-09-17T04:31:16.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.109+0000 7f93ca200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93a8003c30 0x7f93c41f5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:16.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.110+0000 7f93ca200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93c41f49d0 0x7f93c40cb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:16.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.110+0000 7f93ca200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93c41f4de0 0x7f93c40cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:16.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.110+0000 7f93ca200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f93c40cfd30 con 0x7f93c41f4de0 2024-09-17T04:31:16.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.110+0000 7f93ca200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f93c40b9930 con 0x7f93a8003c30 2024-09-17T04:31:16.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.110+0000 7f93ca200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f93c40cfeb0 con 0x7f93c41f49d0 2024-09-17T04:31:16.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.110+0000 7f93c9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93c41f4de0 0x7f93c40cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:16.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.110+0000 7f93c9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93c41f4de0 0x7f93c40cbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55412/0 (socket says 172.21.15.100:55412) 2024-09-17T04:31:16.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.110+0000 7f93c9800640 1 -- 172.21.15.100:0/301437117 learned_addr learned my addr 172.21.15.100:0/301437117 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:16.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.110+0000 7f93c8e00640 1 --2- 172.21.15.100:0/301437117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93a8003c30 0x7f93c41f5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:16.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.110+0000 7f93bbe00640 1 --2- 172.21.15.100:0/301437117 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93c41f49d0 0x7f93c40cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:16.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.110+0000 7f93c9800640 1 -- 172.21.15.100:0/301437117 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93c41f49d0 msgr2=0x7f93c40cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:16.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.110+0000 7f93c9800640 1 --2- 172.21.15.100:0/301437117 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93c41f49d0 0x7f93c40cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:16.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.110+0000 7f93c9800640 1 -- 172.21.15.100:0/301437117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93a8003c30 msgr2=0x7f93c41f5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:16.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.110+0000 7f93c9800640 1 --2- 172.21.15.100:0/301437117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93a8003c30 0x7f93c41f5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:16.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.110+0000 7f93c9800640 1 -- 172.21.15.100:0/301437117 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c40c9190 con 0x7f93c41f4de0 2024-09-17T04:31:16.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.110+0000 7f93c9800640 1 --2- 172.21.15.100:0/301437117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93c41f4de0 0x7f93c40cbec0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f93c408e640 tx=0x7f93bc001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:16.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.111+0000 7f93c1600640 1 -- 172.21.15.100:0/301437117 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc011930 con 0x7f93c41f4de0 2024-09-17T04:31:16.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.111+0000 7f93c1600640 1 -- 172.21.15.100:0/301437117 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93bc017070 con 0x7f93c41f4de0 2024-09-17T04:31:16.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.111+0000 7f93ca200640 1 -- 172.21.15.100:0/301437117 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c40ce040 con 0x7f93c41f4de0 2024-09-17T04:31:16.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.111+0000 7f93c1600640 1 -- 172.21.15.100:0/301437117 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc011e20 con 0x7f93c41f4de0 2024-09-17T04:31:16.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.111+0000 7f93ca200640 1 -- 172.21.15.100:0/301437117 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c40ce600 con 0x7f93c41f4de0 2024-09-17T04:31:16.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.112+0000 7f93ca200640 1 -- 172.21.15.100:0/301437117 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93c40d03b0 con 0x7f93c41f4de0 2024-09-17T04:31:16.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.113+0000 7f93c1600640 1 -- 172.21.15.100:0/301437117 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f93bc01c460 con 0x7f93c41f4de0 2024-09-17T04:31:16.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.114+0000 7f93c1600640 1 --2- 172.21.15.100:0/301437117 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f93a0079010 0x7f93a007b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:16.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.114+0000 7f93c1600640 1 -- 172.21.15.100:0/301437117 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7f93bc09bed0 con 0x7f93c41f4de0 2024-09-17T04:31:16.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.114+0000 7f93c8e00640 1 --2- 172.21.15.100:0/301437117 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f93a0079010 0x7f93a007b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:16.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.115+0000 7f93c8e00640 1 --2- 172.21.15.100:0/301437117 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f93a0079010 0x7f93a007b520 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f93b4035000 tx=0x7f93b40046c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:16.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.118+0000 7f93c1600640 1 -- 172.21.15.100:0/301437117 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f93bc063ca0 con 0x7f93c41f4de0 2024-09-17T04:31:16.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.297+0000 7f93ca200640 1 -- 172.21.15.100:0/301437117 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f93c415d1f0 con 0x7f93a0079010 2024-09-17T04:31:16.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.301+0000 7f93c1600640 1 -- 172.21.15.100:0/301437117 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247116 (secure 0 0 0) 0x7f93c415d1f0 con 0x7f93a0079010 2024-09-17T04:31:16.305 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:16.307 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:31:16.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.305+0000 7f93ca200640 1 -- 172.21.15.100:0/301437117 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f93a0079010 msgr2=0x7f93a007b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:16.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.305+0000 7f93ca200640 1 --2- 172.21.15.100:0/301437117 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f93a0079010 0x7f93a007b520 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f93b4035000 tx=0x7f93b40046c0 comp rx=0 tx=0).stop 2024-09-17T04:31:16.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.305+0000 7f93ca200640 1 -- 172.21.15.100:0/301437117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93c41f4de0 msgr2=0x7f93c40cbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:16.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.305+0000 7f93ca200640 1 --2- 172.21.15.100:0/301437117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93c41f4de0 0x7f93c40cbec0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f93c408e640 tx=0x7f93bc001b80 comp rx=0 tx=0).stop 2024-09-17T04:31:16.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.305+0000 7f93ca200640 1 -- 172.21.15.100:0/301437117 shutdown_connections 2024-09-17T04:31:16.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.305+0000 7f93ca200640 1 --2- 172.21.15.100:0/301437117 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f93a0079010 0x7f93a007b520 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:16.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.305+0000 7f93ca200640 1 --2- 172.21.15.100:0/301437117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93c41f4de0 0x7f93c40cbec0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:16.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.305+0000 7f93ca200640 1 --2- 172.21.15.100:0/301437117 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93c41f49d0 0x7f93c40cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:16.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.305+0000 7f93ca200640 1 --2- 172.21.15.100:0/301437117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93a8003c30 0x7f93c41f5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:16.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.305+0000 7f93ca200640 1 -- 172.21.15.100:0/301437117 >> 172.21.15.100:0/301437117 conn(0x7f93c409d820 msgr2=0x7f93c406de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:16.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.305+0000 7f93ca200640 1 -- 172.21.15.100:0/301437117 shutdown_connections 2024-09-17T04:31:16.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.305+0000 7f93ca200640 1 -- 172.21.15.100:0/301437117 wait complete. 2024-09-17T04:31:16.326 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-09-17T04:31:15.688582+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262008,"kb_used_data":12476,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712712,"statfs":{"total":773094113280,"available":772825817088,"internally_reserved":0,"allocated":12775424,"data_stored":6250897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.795560"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.914190+0000","last_change":"2024-09-17T04:31:10.914190+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.914190+0000","last_undegraded":"2024-09-17T04:31:10.914190+0000","last_fullsized":"2024-09-17T04:31:10.914190+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.904387+0000","last_change":"2024-09-17T04:31:10.904387+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.904387+0000","last_undegraded":"2024-09-17T04:31:10.904387+0000","last_fullsized":"2024-09-17T04:31:10.904387+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.917302+0000","last_change":"2024-09-17T04:31:10.917302+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.917302+0000","last_undegraded":"2024-09-17T04:31:10.917302+0000","last_fullsized":"2024-09-17T04:31:10.917302+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.912667+0000","last_change":"2024-09-17T04:31:10.912667+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.912667+0000","last_undegraded":"2024-09-17T04:31:10.912667+0000","last_fullsized":"2024-09-17T04:31:10.912667+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.910610+0000","last_change":"2024-09-17T04:31:10.910610+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.910610+0000","last_undegraded":"2024-09-17T04:31:10.910610+0000","last_fullsized":"2024-09-17T04:31:10.910610+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.903895+0000","last_change":"2024-09-17T04:31:10.903895+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.903895+0000","last_undegraded":"2024-09-17T04:31:10.903895+0000","last_fullsized":"2024-09-17T04:31:10.903895+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.904370+0000","last_change":"2024-09-17T04:31:10.904370+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.904370+0000","last_undegraded":"2024-09-17T04:31:10.904370+0000","last_fullsized":"2024-09-17T04:31:10.904370+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.912073+0000","last_change":"2024-09-17T04:31:10.912073+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.912073+0000","last_undegraded":"2024-09-17T04:31:10.912073+0000","last_fullsized":"2024-09-17T04:31:10.912073+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.915843+0000","last_change":"2024-09-17T04:31:10.915843+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.915843+0000","last_undegraded":"2024-09-17T04:31:10.915843+0000","last_fullsized":"2024-09-17T04:31:10.915843+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.903841+0000","last_change":"2024-09-17T04:31:10.903841+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.903841+0000","last_undegraded":"2024-09-17T04:31:10.903841+0000","last_fullsized":"2024-09-17T04:31:10.903841+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.912182+0000","last_change":"2024-09-17T04:31:10.912182+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.912182+0000","last_undegraded":"2024-09-17T04:31:10.912182+0000","last_fullsized":"2024-09-17T04:31:10.912182+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.913434+0000","last_change":"2024-09-17T04:31:10.913434+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.913434+0000","last_undegraded":"2024-09-17T04:31:10.913434+0000","last_fullsized":"2024-09-17T04:31:10.913434+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.915532+0000","last_change":"2024-09-17T04:31:10.915532+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.915532+0000","last_undegraded":"2024-09-17T04:31:10.915532+0000","last_fullsized":"2024-09-17T04:31:10.915532+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.903634+0000","last_change":"2024-09-17T04:31:10.903634+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.903634+0000","last_undegraded":"2024-09-17T04:31:10.903634+0000","last_fullsized":"2024-09-17T04:31:10.903634+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.913164+0000","last_change":"2024-09-17T04:31:10.913164+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.913164+0000","last_undegraded":"2024-09-17T04:31:10.913164+0000","last_fullsized":"2024-09-17T04:31:10.913164+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.912271+0000","last_change":"2024-09-17T04:31:10.912271+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.912271+0000","last_undegraded":"2024-09-17T04:31:10.912271+0000","last_fullsized":"2024-09-17T04:31:10.912271+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.904605+0000","last_change":"2024-09-17T04:31:10.904605+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.904605+0000","last_undegraded":"2024-09-17T04:31:10.904605+0000","last_fullsized":"2024-09-17T04:31:10.904605+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.912776+0000","last_change":"2024-09-17T04:31:10.912776+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.912776+0000","last_undegraded":"2024-09-17T04:31:10.912776+0000","last_fullsized":"2024-09-17T04:31:10.912776+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.911756+0000","last_change":"2024-09-17T04:31:10.911756+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.911756+0000","last_undegraded":"2024-09-17T04:31:10.911756+0000","last_fullsized":"2024-09-17T04:31:10.911756+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.911841+0000","last_change":"2024-09-17T04:31:10.911841+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.911841+0000","last_undegraded":"2024-09-17T04:31:10.911841+0000","last_fullsized":"2024-09-17T04:31:10.911841+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.916213+0000","last_change":"2024-09-17T04:31:10.916213+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.916213+0000","last_undegraded":"2024-09-17T04:31:10.916213+0000","last_fullsized":"2024-09-17T04:31:10.916213+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.910761+0000","last_change":"2024-09-17T04:31:10.910761+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.910761+0000","last_undegraded":"2024-09-17T04:31:10.910761+0000","last_fullsized":"2024-09-17T04:31:10.910761+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.914620+0000","last_change":"2024-09-17T04:31:10.914620+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.914620+0000","last_undegraded":"2024-09-17T04:31:10.914620+0000","last_fullsized":"2024-09-17T04:31:10.914620+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.907757+0000","last_change":"2024-09-17T04:31:10.907757+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.907757+0000","last_undegraded":"2024-09-17T04:31:10.907757+0000","last_fullsized":"2024-09-17T04:31:10.907757+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.904983+0000","last_change":"2024-09-17T04:31:10.904983+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.904983+0000","last_undegraded":"2024-09-17T04:31:10.904983+0000","last_fullsized":"2024-09-17T04:31:10.904983+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.912322+0000","last_change":"2024-09-17T04:31:10.912322+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.912322+0000","last_undegraded":"2024-09-17T04:31:10.912322+0000","last_fullsized":"2024-09-17T04:31:10.912322+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909621+0000","last_change":"2024-09-17T04:31:10.909621+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909621+0000","last_undegraded":"2024-09-17T04:31:10.909621+0000","last_fullsized":"2024-09-17T04:31:10.909621+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909027+0000","last_change":"2024-09-17T04:31:10.909027+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909027+0000","last_undegraded":"2024-09-17T04:31:10.909027+0000","last_fullsized":"2024-09-17T04:31:10.909027+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.910178+0000","last_change":"2024-09-17T04:31:10.910178+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.910178+0000","last_undegraded":"2024-09-17T04:31:10.910178+0000","last_fullsized":"2024-09-17T04:31:10.910178+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.911090+0000","last_change":"2024-09-17T04:31:10.911090+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.911090+0000","last_undegraded":"2024-09-17T04:31:10.911090+0000","last_fullsized":"2024-09-17T04:31:10.911090+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909120+0000","last_change":"2024-09-17T04:31:10.909120+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909120+0000","last_undegraded":"2024-09-17T04:31:10.909120+0000","last_fullsized":"2024-09-17T04:31:10.909120+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.903949+0000","last_change":"2024-09-17T04:31:10.903949+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.903949+0000","last_undegraded":"2024-09-17T04:31:10.903949+0000","last_fullsized":"2024-09-17T04:31:10.903949+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.910613+0000","last_change":"2024-09-17T04:31:10.910613+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.910613+0000","last_undegraded":"2024-09-17T04:31:10.910613+0000","last_fullsized":"2024-09-17T04:31:10.910613+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.928277+0000","last_change":"2024-09-17T04:31:10.925886+0000","last_active":"2024-09-17T04:31:10.928277+0000","last_peered":"2024-09-17T04:31:10.928277+0000","last_clean":"2024-09-17T04:31:10.928277+0000","last_became_active":"2024-09-17T04:31:10.925455+0000","last_became_peered":"2024-09-17T04:31:10.925455+0000","last_unstale":"2024-09-17T04:31:10.928277+0000","last_undegraded":"2024-09-17T04:31:10.928277+0000","last_fullsized":"2024-09-17T04:31:10.928277+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:01.185188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.914848+0000","last_change":"2024-09-17T04:31:10.914848+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.914848+0000","last_undegraded":"2024-09-17T04:31:10.914848+0000","last_fullsized":"2024-09-17T04:31:10.914848+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.925524+0000","last_change":"2024-09-17T04:31:10.923497+0000","last_active":"2024-09-17T04:31:10.925524+0000","last_peered":"2024-09-17T04:31:10.925524+0000","last_clean":"2024-09-17T04:31:10.925524+0000","last_became_active":"2024-09-17T04:31:10.923086+0000","last_became_peered":"2024-09-17T04:31:10.923086+0000","last_unstale":"2024-09-17T04:31:10.925524+0000","last_undegraded":"2024-09-17T04:31:10.925524+0000","last_fullsized":"2024-09-17T04:31:10.925524+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:25.527822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909426+0000","last_change":"2024-09-17T04:31:10.909426+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909426+0000","last_undegraded":"2024-09-17T04:31:10.909426+0000","last_fullsized":"2024-09-17T04:31:10.909426+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.923768+0000","last_change":"2024-09-17T04:31:10.922177+0000","last_active":"2024-09-17T04:31:10.923768+0000","last_peered":"2024-09-17T04:31:10.923768+0000","last_clean":"2024-09-17T04:31:10.923768+0000","last_became_active":"2024-09-17T04:31:10.921875+0000","last_became_peered":"2024-09-17T04:31:10.921875+0000","last_unstale":"2024-09-17T04:31:10.923768+0000","last_undegraded":"2024-09-17T04:31:10.923768+0000","last_fullsized":"2024-09-17T04:31:10.923768+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:32.470330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.910133+0000","last_change":"2024-09-17T04:31:10.910133+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.910133+0000","last_undegraded":"2024-09-17T04:31:10.910133+0000","last_fullsized":"2024-09-17T04:31:10.910133+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.919580+0000","last_change":"2024-09-17T04:31:10.917285+0000","last_active":"2024-09-17T04:31:10.919580+0000","last_peered":"2024-09-17T04:31:10.919580+0000","last_clean":"2024-09-17T04:31:10.919580+0000","last_became_active":"2024-09-17T04:31:10.916739+0000","last_became_peered":"2024-09-17T04:31:10.916739+0000","last_unstale":"2024-09-17T04:31:10.919580+0000","last_undegraded":"2024-09-17T04:31:10.919580+0000","last_fullsized":"2024-09-17T04:31:10.919580+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:17.262185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909946+0000","last_change":"2024-09-17T04:31:10.909946+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909946+0000","last_undegraded":"2024-09-17T04:31:10.909946+0000","last_fullsized":"2024-09-17T04:31:10.909946+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.918268+0000","last_change":"2024-09-17T04:31:10.913568+0000","last_active":"2024-09-17T04:31:10.918268+0000","last_peered":"2024-09-17T04:31:10.918268+0000","last_clean":"2024-09-17T04:31:10.918268+0000","last_became_active":"2024-09-17T04:31:10.913273+0000","last_became_peered":"2024-09-17T04:31:10.913273+0000","last_unstale":"2024-09-17T04:31:10.918268+0000","last_undegraded":"2024-09-17T04:31:10.918268+0000","last_fullsized":"2024-09-17T04:31:10.918268+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:10.022361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.905237+0000","last_change":"2024-09-17T04:31:10.905237+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.905237+0000","last_undegraded":"2024-09-17T04:31:10.905237+0000","last_fullsized":"2024-09-17T04:31:10.905237+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.922101+0000","last_change":"2024-09-17T04:31:10.917769+0000","last_active":"2024-09-17T04:31:10.922101+0000","last_peered":"2024-09-17T04:31:10.922101+0000","last_clean":"2024-09-17T04:31:10.922101+0000","last_became_active":"2024-09-17T04:31:10.917493+0000","last_became_peered":"2024-09-17T04:31:10.917493+0000","last_unstale":"2024-09-17T04:31:10.922101+0000","last_undegraded":"2024-09-17T04:31:10.922101+0000","last_fullsized":"2024-09-17T04:31:10.922101+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:40:04.424615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.911438+0000","last_change":"2024-09-17T04:31:10.911438+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.911438+0000","last_undegraded":"2024-09-17T04:31:10.911438+0000","last_fullsized":"2024-09-17T04:31:10.911438+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.927265+0000","last_change":"2024-09-17T04:31:10.919927+0000","last_active":"2024-09-17T04:31:10.927265+0000","last_peered":"2024-09-17T04:31:10.927265+0000","last_clean":"2024-09-17T04:31:10.927265+0000","last_became_active":"2024-09-17T04:31:10.919536+0000","last_became_peered":"2024-09-17T04:31:10.919536+0000","last_unstale":"2024-09-17T04:31:10.927265+0000","last_undegraded":"2024-09-17T04:31:10.927265+0000","last_fullsized":"2024-09-17T04:31:10.927265+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:46.091206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.903852+0000","last_change":"2024-09-17T04:31:10.903852+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.903852+0000","last_undegraded":"2024-09-17T04:31:10.903852+0000","last_fullsized":"2024-09-17T04:31:10.903852+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.927852+0000","last_change":"2024-09-17T04:31:10.926647+0000","last_active":"2024-09-17T04:31:10.927852+0000","last_peered":"2024-09-17T04:31:10.927852+0000","last_clean":"2024-09-17T04:31:10.927852+0000","last_became_active":"2024-09-17T04:31:10.926287+0000","last_became_peered":"2024-09-17T04:31:10.926287+0000","last_unstale":"2024-09-17T04:31:10.927852+0000","last_undegraded":"2024-09-17T04:31:10.927852+0000","last_fullsized":"2024-09-17T04:31:10.927852+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:41.700033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909109+0000","last_change":"2024-09-17T04:31:10.909109+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909109+0000","last_undegraded":"2024-09-17T04:31:10.909109+0000","last_fullsized":"2024-09-17T04:31:10.909109+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.928213+0000","last_change":"2024-09-17T04:31:10.926032+0000","last_active":"2024-09-17T04:31:10.928213+0000","last_peered":"2024-09-17T04:31:10.928213+0000","last_clean":"2024-09-17T04:31:10.928213+0000","last_became_active":"2024-09-17T04:31:10.925736+0000","last_became_peered":"2024-09-17T04:31:10.925736+0000","last_unstale":"2024-09-17T04:31:10.928213+0000","last_undegraded":"2024-09-17T04:31:10.928213+0000","last_fullsized":"2024-09-17T04:31:10.928213+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:40.221385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.912472+0000","last_change":"2024-09-17T04:31:10.912472+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.912472+0000","last_undegraded":"2024-09-17T04:31:10.912472+0000","last_fullsized":"2024-09-17T04:31:10.912472+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.930968+0000","last_change":"2024-09-17T04:31:10.930027+0000","last_active":"2024-09-17T04:31:10.930968+0000","last_peered":"2024-09-17T04:31:10.930968+0000","last_clean":"2024-09-17T04:31:10.930968+0000","last_became_active":"2024-09-17T04:31:10.929648+0000","last_became_peered":"2024-09-17T04:31:10.929648+0000","last_unstale":"2024-09-17T04:31:10.930968+0000","last_undegraded":"2024-09-17T04:31:10.930968+0000","last_fullsized":"2024-09-17T04:31:10.930968+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:18:25.078648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.908795+0000","last_change":"2024-09-17T04:31:10.908795+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.908795+0000","last_undegraded":"2024-09-17T04:31:10.908795+0000","last_fullsized":"2024-09-17T04:31:10.908795+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.920305+0000","last_change":"2024-09-17T04:31:10.917626+0000","last_active":"2024-09-17T04:31:10.920305+0000","last_peered":"2024-09-17T04:31:10.920305+0000","last_clean":"2024-09-17T04:31:10.920305+0000","last_became_active":"2024-09-17T04:31:10.917239+0000","last_became_peered":"2024-09-17T04:31:10.917239+0000","last_unstale":"2024-09-17T04:31:10.920305+0000","last_undegraded":"2024-09-17T04:31:10.920305+0000","last_fullsized":"2024-09-17T04:31:10.920305+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:29.945578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.908678+0000","last_change":"2024-09-17T04:31:10.908678+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.908678+0000","last_undegraded":"2024-09-17T04:31:10.908678+0000","last_fullsized":"2024-09-17T04:31:10.908678+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.931270+0000","last_change":"2024-09-17T04:31:10.929556+0000","last_active":"2024-09-17T04:31:10.931270+0000","last_peered":"2024-09-17T04:31:10.931270+0000","last_clean":"2024-09-17T04:31:10.931270+0000","last_became_active":"2024-09-17T04:31:10.929171+0000","last_became_peered":"2024-09-17T04:31:10.929171+0000","last_unstale":"2024-09-17T04:31:10.931270+0000","last_undegraded":"2024-09-17T04:31:10.931270+0000","last_fullsized":"2024-09-17T04:31:10.931270+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:00.689611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909041+0000","last_change":"2024-09-17T04:31:10.909041+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909041+0000","last_undegraded":"2024-09-17T04:31:10.909041+0000","last_fullsized":"2024-09-17T04:31:10.909041+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.925160+0000","last_change":"2024-09-17T04:31:10.921170+0000","last_active":"2024-09-17T04:31:10.925160+0000","last_peered":"2024-09-17T04:31:10.925160+0000","last_clean":"2024-09-17T04:31:10.925160+0000","last_became_active":"2024-09-17T04:31:10.920873+0000","last_became_peered":"2024-09-17T04:31:10.920873+0000","last_unstale":"2024-09-17T04:31:10.925160+0000","last_undegraded":"2024-09-17T04:31:10.925160+0000","last_fullsized":"2024-09-17T04:31:10.925160+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:09:40.221275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.903934+0000","last_change":"2024-09-17T04:31:10.903934+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.903934+0000","last_undegraded":"2024-09-17T04:31:10.903934+0000","last_fullsized":"2024-09-17T04:31:10.903934+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.924402+0000","last_change":"2024-09-17T04:31:10.920901+0000","last_active":"2024-09-17T04:31:10.924402+0000","last_peered":"2024-09-17T04:31:10.924402+0000","last_clean":"2024-09-17T04:31:10.924402+0000","last_became_active":"2024-09-17T04:31:10.920586+0000","last_became_peered":"2024-09-17T04:31:10.920586+0000","last_unstale":"2024-09-17T04:31:10.924402+0000","last_undegraded":"2024-09-17T04:31:10.924402+0000","last_fullsized":"2024-09-17T04:31:10.924402+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:59.093831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.915262+0000","last_change":"2024-09-17T04:31:10.915262+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.915262+0000","last_undegraded":"2024-09-17T04:31:10.915262+0000","last_fullsized":"2024-09-17T04:31:10.915262+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.929486+0000","last_change":"2024-09-17T04:31:10.924812+0000","last_active":"2024-09-17T04:31:10.929486+0000","last_peered":"2024-09-17T04:31:10.929486+0000","last_clean":"2024-09-17T04:31:10.929486+0000","last_became_active":"2024-09-17T04:31:10.924423+0000","last_became_peered":"2024-09-17T04:31:10.924423+0000","last_unstale":"2024-09-17T04:31:10.929486+0000","last_undegraded":"2024-09-17T04:31:10.929486+0000","last_fullsized":"2024-09-17T04:31:10.929486+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:07.672259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.917600+0000","last_change":"2024-09-17T04:31:10.917600+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.917600+0000","last_undegraded":"2024-09-17T04:31:10.917600+0000","last_fullsized":"2024-09-17T04:31:10.917600+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.926479+0000","last_change":"2024-09-17T04:31:10.924255+0000","last_active":"2024-09-17T04:31:10.926479+0000","last_peered":"2024-09-17T04:31:10.926479+0000","last_clean":"2024-09-17T04:31:10.926479+0000","last_became_active":"2024-09-17T04:31:10.923887+0000","last_became_peered":"2024-09-17T04:31:10.923887+0000","last_unstale":"2024-09-17T04:31:10.926479+0000","last_undegraded":"2024-09-17T04:31:10.926479+0000","last_fullsized":"2024-09-17T04:31:10.926479+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:42.584932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.911484+0000","last_change":"2024-09-17T04:31:10.911484+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.911484+0000","last_undegraded":"2024-09-17T04:31:10.911484+0000","last_fullsized":"2024-09-17T04:31:10.911484+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.930817+0000","last_change":"2024-09-17T04:31:10.930022+0000","last_active":"2024-09-17T04:31:10.930817+0000","last_peered":"2024-09-17T04:31:10.930817+0000","last_clean":"2024-09-17T04:31:10.930817+0000","last_became_active":"2024-09-17T04:31:10.929681+0000","last_became_peered":"2024-09-17T04:31:10.929681+0000","last_unstale":"2024-09-17T04:31:10.930817+0000","last_undegraded":"2024-09-17T04:31:10.930817+0000","last_fullsized":"2024-09-17T04:31:10.930817+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:16.582319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.911356+0000","last_change":"2024-09-17T04:31:10.911356+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.911356+0000","last_undegraded":"2024-09-17T04:31:10.911356+0000","last_fullsized":"2024-09-17T04:31:10.911356+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.918083+0000","last_change":"2024-09-17T04:31:10.910443+0000","last_active":"2024-09-17T04:31:10.918083+0000","last_peered":"2024-09-17T04:31:10.918083+0000","last_clean":"2024-09-17T04:31:10.918083+0000","last_became_active":"2024-09-17T04:31:10.910084+0000","last_became_peered":"2024-09-17T04:31:10.910084+0000","last_unstale":"2024-09-17T04:31:10.918083+0000","last_undegraded":"2024-09-17T04:31:10.918083+0000","last_fullsized":"2024-09-17T04:31:10.918083+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:02.897825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.904544+0000","last_change":"2024-09-17T04:31:10.904544+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.904544+0000","last_undegraded":"2024-09-17T04:31:10.904544+0000","last_fullsized":"2024-09-17T04:31:10.904544+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.921482+0000","last_change":"2024-09-17T04:31:10.918599+0000","last_active":"2024-09-17T04:31:10.921482+0000","last_peered":"2024-09-17T04:31:10.921482+0000","last_clean":"2024-09-17T04:31:10.921482+0000","last_became_active":"2024-09-17T04:31:10.918208+0000","last_became_peered":"2024-09-17T04:31:10.918208+0000","last_unstale":"2024-09-17T04:31:10.921482+0000","last_undegraded":"2024-09-17T04:31:10.921482+0000","last_fullsized":"2024-09-17T04:31:10.921482+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:29.662127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909400+0000","last_change":"2024-09-17T04:31:10.909400+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909400+0000","last_undegraded":"2024-09-17T04:31:10.909400+0000","last_fullsized":"2024-09-17T04:31:10.909400+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.929291+0000","last_change":"2024-09-17T04:31:10.927592+0000","last_active":"2024-09-17T04:31:10.929291+0000","last_peered":"2024-09-17T04:31:10.929291+0000","last_clean":"2024-09-17T04:31:10.929291+0000","last_became_active":"2024-09-17T04:31:10.926639+0000","last_became_peered":"2024-09-17T04:31:10.926639+0000","last_unstale":"2024-09-17T04:31:10.929291+0000","last_undegraded":"2024-09-17T04:31:10.929291+0000","last_fullsized":"2024-09-17T04:31:10.929291+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:50.191404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.904320+0000","last_change":"2024-09-17T04:31:10.904320+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.904320+0000","last_undegraded":"2024-09-17T04:31:10.904320+0000","last_fullsized":"2024-09-17T04:31:10.904320+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.928276+0000","last_change":"2024-09-17T04:31:10.926073+0000","last_active":"2024-09-17T04:31:10.928276+0000","last_peered":"2024-09-17T04:31:10.928276+0000","last_clean":"2024-09-17T04:31:10.928276+0000","last_became_active":"2024-09-17T04:31:10.925750+0000","last_became_peered":"2024-09-17T04:31:10.925750+0000","last_unstale":"2024-09-17T04:31:10.928276+0000","last_undegraded":"2024-09-17T04:31:10.928276+0000","last_fullsized":"2024-09-17T04:31:10.928276+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:10:46.052152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.903377+0000","last_change":"2024-09-17T04:31:10.903377+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.903377+0000","last_undegraded":"2024-09-17T04:31:10.903377+0000","last_fullsized":"2024-09-17T04:31:10.903377+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.923038+0000","last_change":"2024-09-17T04:31:10.917223+0000","last_active":"2024-09-17T04:31:10.923038+0000","last_peered":"2024-09-17T04:31:10.923038+0000","last_clean":"2024-09-17T04:31:10.923038+0000","last_became_active":"2024-09-17T04:31:10.916765+0000","last_became_peered":"2024-09-17T04:31:10.916765+0000","last_unstale":"2024-09-17T04:31:10.923038+0000","last_undegraded":"2024-09-17T04:31:10.923038+0000","last_fullsized":"2024-09-17T04:31:10.923038+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:37.040110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.910300+0000","last_change":"2024-09-17T04:31:10.910300+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.910300+0000","last_undegraded":"2024-09-17T04:31:10.910300+0000","last_fullsized":"2024-09-17T04:31:10.910300+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.927355+0000","last_change":"2024-09-17T04:31:10.926100+0000","last_active":"2024-09-17T04:31:10.927355+0000","last_peered":"2024-09-17T04:31:10.927355+0000","last_clean":"2024-09-17T04:31:10.927355+0000","last_became_active":"2024-09-17T04:31:10.925806+0000","last_became_peered":"2024-09-17T04:31:10.925806+0000","last_unstale":"2024-09-17T04:31:10.927355+0000","last_undegraded":"2024-09-17T04:31:10.927355+0000","last_fullsized":"2024-09-17T04:31:10.927355+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:05.786635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.904697+0000","last_change":"2024-09-17T04:31:10.904697+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.904697+0000","last_undegraded":"2024-09-17T04:31:10.904697+0000","last_fullsized":"2024-09-17T04:31:10.904697+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.924475+0000","last_change":"2024-09-17T04:31:10.919421+0000","last_active":"2024-09-17T04:31:10.924475+0000","last_peered":"2024-09-17T04:31:10.924475+0000","last_clean":"2024-09-17T04:31:10.924475+0000","last_became_active":"2024-09-17T04:31:10.919126+0000","last_became_peered":"2024-09-17T04:31:10.919126+0000","last_unstale":"2024-09-17T04:31:10.924475+0000","last_undegraded":"2024-09-17T04:31:10.924475+0000","last_fullsized":"2024-09-17T04:31:10.924475+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:40.608118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909084+0000","last_change":"2024-09-17T04:31:10.909084+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909084+0000","last_undegraded":"2024-09-17T04:31:10.909084+0000","last_fullsized":"2024-09-17T04:31:10.909084+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.931338+0000","last_change":"2024-09-17T04:31:10.930374+0000","last_active":"2024-09-17T04:31:10.931338+0000","last_peered":"2024-09-17T04:31:10.931338+0000","last_clean":"2024-09-17T04:31:10.931338+0000","last_became_active":"2024-09-17T04:31:10.930043+0000","last_became_peered":"2024-09-17T04:31:10.930043+0000","last_unstale":"2024-09-17T04:31:10.931338+0000","last_undegraded":"2024-09-17T04:31:10.931338+0000","last_fullsized":"2024-09-17T04:31:10.931338+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:02.897825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.904139+0000","last_change":"2024-09-17T04:31:10.904139+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.904139+0000","last_undegraded":"2024-09-17T04:31:10.904139+0000","last_fullsized":"2024-09-17T04:31:10.904139+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.930617+0000","last_change":"2024-09-17T04:31:10.929027+0000","last_active":"2024-09-17T04:31:10.930617+0000","last_peered":"2024-09-17T04:31:10.930617+0000","last_clean":"2024-09-17T04:31:10.930617+0000","last_became_active":"2024-09-17T04:31:10.928767+0000","last_became_peered":"2024-09-17T04:31:10.928767+0000","last_unstale":"2024-09-17T04:31:10.930617+0000","last_undegraded":"2024-09-17T04:31:10.930617+0000","last_fullsized":"2024-09-17T04:31:10.930617+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:06.263954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.905447+0000","last_change":"2024-09-17T04:31:10.905447+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.905447+0000","last_undegraded":"2024-09-17T04:31:10.905447+0000","last_fullsized":"2024-09-17T04:31:10.905447+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.922602+0000","last_change":"2024-09-17T04:31:10.920453+0000","last_active":"2024-09-17T04:31:10.922602+0000","last_peered":"2024-09-17T04:31:10.922602+0000","last_clean":"2024-09-17T04:31:10.922602+0000","last_became_active":"2024-09-17T04:31:10.919363+0000","last_became_peered":"2024-09-17T04:31:10.919363+0000","last_unstale":"2024-09-17T04:31:10.922602+0000","last_undegraded":"2024-09-17T04:31:10.922602+0000","last_fullsized":"2024-09-17T04:31:10.922602+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:04:35.664082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909560+0000","last_change":"2024-09-17T04:31:10.909560+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909560+0000","last_undegraded":"2024-09-17T04:31:10.909560+0000","last_fullsized":"2024-09-17T04:31:10.909560+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.924812+0000","last_change":"2024-09-17T04:31:10.923121+0000","last_active":"2024-09-17T04:31:10.924812+0000","last_peered":"2024-09-17T04:31:10.924812+0000","last_clean":"2024-09-17T04:31:10.924812+0000","last_became_active":"2024-09-17T04:31:10.922374+0000","last_became_peered":"2024-09-17T04:31:10.922374+0000","last_unstale":"2024-09-17T04:31:10.924812+0000","last_undegraded":"2024-09-17T04:31:10.924812+0000","last_fullsized":"2024-09-17T04:31:10.924812+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:23:56.512701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.911942+0000","last_change":"2024-09-17T04:31:10.911942+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.911942+0000","last_undegraded":"2024-09-17T04:31:10.911942+0000","last_fullsized":"2024-09-17T04:31:10.911942+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.927043+0000","last_change":"2024-09-17T04:31:10.925280+0000","last_active":"2024-09-17T04:31:10.927043+0000","last_peered":"2024-09-17T04:31:10.927043+0000","last_clean":"2024-09-17T04:31:10.927043+0000","last_became_active":"2024-09-17T04:31:10.924762+0000","last_became_peered":"2024-09-17T04:31:10.924762+0000","last_unstale":"2024-09-17T04:31:10.927043+0000","last_undegraded":"2024-09-17T04:31:10.927043+0000","last_fullsized":"2024-09-17T04:31:10.927043+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:28.910545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.910074+0000","last_change":"2024-09-17T04:31:10.910074+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.910074+0000","last_undegraded":"2024-09-17T04:31:10.910074+0000","last_fullsized":"2024-09-17T04:31:10.910074+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.928272+0000","last_change":"2024-09-17T04:31:10.926916+0000","last_active":"2024-09-17T04:31:10.928272+0000","last_peered":"2024-09-17T04:31:10.928272+0000","last_clean":"2024-09-17T04:31:10.928272+0000","last_became_active":"2024-09-17T04:31:10.926611+0000","last_became_peered":"2024-09-17T04:31:10.926611+0000","last_unstale":"2024-09-17T04:31:10.928272+0000","last_undegraded":"2024-09-17T04:31:10.928272+0000","last_fullsized":"2024-09-17T04:31:10.928272+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:39.197809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.903334+0000","last_change":"2024-09-17T04:31:10.903334+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.903334+0000","last_undegraded":"2024-09-17T04:31:10.903334+0000","last_fullsized":"2024-09-17T04:31:10.903334+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.921772+0000","last_change":"2024-09-17T04:31:10.918838+0000","last_active":"2024-09-17T04:31:10.921772+0000","last_peered":"2024-09-17T04:31:10.921772+0000","last_clean":"2024-09-17T04:31:10.921772+0000","last_became_active":"2024-09-17T04:31:10.918535+0000","last_became_peered":"2024-09-17T04:31:10.918535+0000","last_unstale":"2024-09-17T04:31:10.921772+0000","last_undegraded":"2024-09-17T04:31:10.921772+0000","last_fullsized":"2024-09-17T04:31:10.921772+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:32.795755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.910191+0000","last_change":"2024-09-17T04:31:10.910191+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.910191+0000","last_undegraded":"2024-09-17T04:31:10.910191+0000","last_fullsized":"2024-09-17T04:31:10.910191+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.928967+0000","last_change":"2024-09-17T04:31:10.927734+0000","last_active":"2024-09-17T04:31:10.928967+0000","last_peered":"2024-09-17T04:31:10.928967+0000","last_clean":"2024-09-17T04:31:10.928967+0000","last_became_active":"2024-09-17T04:31:10.926995+0000","last_became_peered":"2024-09-17T04:31:10.926995+0000","last_unstale":"2024-09-17T04:31:10.928967+0000","last_undegraded":"2024-09-17T04:31:10.928967+0000","last_fullsized":"2024-09-17T04:31:10.928967+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:48:42.671646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"76'79","reported_seq":313,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.917506+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:31:10.917506+0000","last_peered":"2024-09-17T04:31:10.917506+0000","last_clean":"2024-09-17T04:31:10.917506+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:31:10.917506+0000","last_undegraded":"2024-09-17T04:31:10.917506+0000","last_fullsized":"2024-09-17T04:31:10.917506+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476827,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33188,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338652,"statfs":{"total":96636764160,"available":96602779648,"internally_reserved":0,"allocated":2043904,"data_stored":1224926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476826,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1449984,"data_stored":634558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476827,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1449984,"data_stored":634558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476827,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1437696,"data_stored":628255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542234,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32612,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339228,"statfs":{"total":96636764160,"available":96603369472,"internally_reserved":0,"allocated":1449984,"data_stored":634558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509530,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1449984,"data_stored":634558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509529,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1449984,"data_stored":634558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509529,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33188,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338652,"statfs":{"total":96636764160,"available":96602779648,"internally_reserved":0,"allocated":2043904,"data_stored":1224926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:31:16.331 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:31:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.586+0000 7fd529a00640 1 -- 172.21.15.100:0/2585316261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5240ad0e0 msgr2=0x7fd5240ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.586+0000 7fd529a00640 1 --2- 172.21.15.100:0/2585316261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5240ad0e0 0x7fd5240ad4f0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fd50c009830 tx=0x7fd50c02a480 comp rx=0 tx=0).stop 2024-09-17T04:31:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.586+0000 7fd529a00640 1 -- 172.21.15.100:0/2585316261 shutdown_connections 2024-09-17T04:31:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.586+0000 7fd529a00640 1 --2- 172.21.15.100:0/2585316261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5240ad0e0 0x7fd5240ad4f0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.586+0000 7fd529a00640 1 --2- 172.21.15.100:0/2585316261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd52415d760 0x7fd5241535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.586+0000 7fd529a00640 1 -- 172.21.15.100:0/2585316261 >> 172.21.15.100:0/2585316261 conn(0x7fd52406d730 msgr2=0x7fd5240b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.586+0000 7fd529a00640 1 -- 172.21.15.100:0/2585316261 shutdown_connections 2024-09-17T04:31:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.586+0000 7fd529a00640 1 -- 172.21.15.100:0/2585316261 wait complete. 2024-09-17T04:31:16.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.589+0000 7fd529a00640 1 Processor -- start 2024-09-17T04:31:16.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.589+0000 7fd529a00640 1 -- start start 2024-09-17T04:31:16.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.590+0000 7fd529a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5240ad0e0 0x7fd52409d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:16.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.590+0000 7fd529a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd52415d760 0x7fd5240b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:16.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.590+0000 7fd529a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd524160310 0x7fd5240b1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:16.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.590+0000 7fd529a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd52406f220 con 0x7fd52415d760 2024-09-17T04:31:16.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.590+0000 7fd529a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd52406f0a0 con 0x7fd5240ad0e0 2024-09-17T04:31:16.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.590+0000 7fd529a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd5240b4c20 con 0x7fd524160310 2024-09-17T04:31:16.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.591+0000 7fd51a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd52415d760 0x7fd5240b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:16.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.591+0000 7fd51a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd52415d760 0x7fd5240b1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55440/0 (socket says 172.21.15.100:55440) 2024-09-17T04:31:16.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.591+0000 7fd51a000640 1 -- 172.21.15.100:0/3893195435 learned_addr learned my addr 172.21.15.100:0/3893195435 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:16.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.591+0000 7fd522a00640 1 --2- 172.21.15.100:0/3893195435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5240ad0e0 0x7fd52409d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:16.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.591+0000 7fd523400640 1 --2- 172.21.15.100:0/3893195435 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd524160310 0x7fd5240b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:16.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.591+0000 7fd51a000640 1 -- 172.21.15.100:0/3893195435 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd524160310 msgr2=0x7fd5240b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:16.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.591+0000 7fd51a000640 1 --2- 172.21.15.100:0/3893195435 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd524160310 0x7fd5240b1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:16.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.591+0000 7fd51a000640 1 -- 172.21.15.100:0/3893195435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5240ad0e0 msgr2=0x7fd52409d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:16.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.591+0000 7fd51a000640 1 --2- 172.21.15.100:0/3893195435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5240ad0e0 0x7fd52409d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:16.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.591+0000 7fd51a000640 1 -- 172.21.15.100:0/3893195435 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52415a2f0 con 0x7fd52415d760 2024-09-17T04:31:16.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.591+0000 7fd51a000640 1 --2- 172.21.15.100:0/3893195435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd52415d760 0x7fd5240b1010 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fd50c009950 tx=0x7fd50c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:16.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.592+0000 7fd51b400640 1 -- 172.21.15.100:0/3893195435 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50c031d60 con 0x7fd52415d760 2024-09-17T04:31:16.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.592+0000 7fd51b400640 1 -- 172.21.15.100:0/3893195435 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd50c033e10 con 0x7fd52415d760 2024-09-17T04:31:16.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.592+0000 7fd51b400640 1 -- 172.21.15.100:0/3893195435 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50c005260 con 0x7fd52415d760 2024-09-17T04:31:16.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.592+0000 7fd529a00640 1 -- 172.21.15.100:0/3893195435 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5241574f0 con 0x7fd52415d760 2024-09-17T04:31:16.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.592+0000 7fd529a00640 1 -- 172.21.15.100:0/3893195435 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd524157ab0 con 0x7fd52415d760 2024-09-17T04:31:16.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.593+0000 7fd529a00640 1 -- 172.21.15.100:0/3893195435 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd52415a060 con 0x7fd52415d760 2024-09-17T04:31:16.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.594+0000 7fd51b400640 1 -- 172.21.15.100:0/3893195435 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fd50c005400 con 0x7fd52415d760 2024-09-17T04:31:16.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.595+0000 7fd51b400640 1 --2- 172.21.15.100:0/3893195435 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd500079010 0x7fd50007b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:16.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.595+0000 7fd51b400640 1 -- 172.21.15.100:0/3893195435 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7fd50c0bc800 con 0x7fd52415d760 2024-09-17T04:31:16.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.595+0000 7fd522a00640 1 --2- 172.21.15.100:0/3893195435 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd500079010 0x7fd50007b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:16.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.596+0000 7fd522a00640 1 --2- 172.21.15.100:0/3893195435 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd500079010 0x7fd50007b520 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd514004590 tx=0x7fd51400b390 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:16.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.600+0000 7fd51b400640 1 -- 172.21.15.100:0/3893195435 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd50c03f030 con 0x7fd52415d760 2024-09-17T04:31:16.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.785+0000 7fd529a00640 1 -- 172.21.15.100:0/3893195435 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd5241fa1a0 con 0x7fd500079010 2024-09-17T04:31:16.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.790+0000 7fd51b400640 1 -- 172.21.15.100:0/3893195435 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247116 (secure 0 0 0) 0x7fd5140037f0 con 0x7fd500079010 2024-09-17T04:31:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:16.794 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:31:16.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.794+0000 7fd529a00640 1 -- 172.21.15.100:0/3893195435 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd500079010 msgr2=0x7fd50007b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:16.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.794+0000 7fd529a00640 1 --2- 172.21.15.100:0/3893195435 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd500079010 0x7fd50007b520 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd514004590 tx=0x7fd51400b390 comp rx=0 tx=0).stop 2024-09-17T04:31:16.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.794+0000 7fd529a00640 1 -- 172.21.15.100:0/3893195435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd52415d760 msgr2=0x7fd5240b1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:16.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.794+0000 7fd529a00640 1 --2- 172.21.15.100:0/3893195435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd52415d760 0x7fd5240b1010 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fd50c009950 tx=0x7fd50c033c20 comp rx=0 tx=0).stop 2024-09-17T04:31:16.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.794+0000 7fd529a00640 1 -- 172.21.15.100:0/3893195435 shutdown_connections 2024-09-17T04:31:16.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.794+0000 7fd529a00640 1 --2- 172.21.15.100:0/3893195435 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd500079010 0x7fd50007b520 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:16.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.794+0000 7fd529a00640 1 --2- 172.21.15.100:0/3893195435 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd524160310 0x7fd5240b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:16.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.794+0000 7fd529a00640 1 --2- 172.21.15.100:0/3893195435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd52415d760 0x7fd5240b1010 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:16.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.794+0000 7fd529a00640 1 --2- 172.21.15.100:0/3893195435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5240ad0e0 0x7fd52409d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:16.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.794+0000 7fd529a00640 1 -- 172.21.15.100:0/3893195435 >> 172.21.15.100:0/3893195435 conn(0x7fd52406d730 msgr2=0x7fd5240cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:16.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.794+0000 7fd529a00640 1 -- 172.21.15.100:0/3893195435 shutdown_connections 2024-09-17T04:31:16.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:16.794+0000 7fd529a00640 1 -- 172.21.15.100:0/3893195435 wait complete. 2024-09-17T04:31:16.817 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-09-17T04:31:15.688582+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262008,"kb_used_data":12476,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712712,"statfs":{"total":773094113280,"available":772825817088,"internally_reserved":0,"allocated":12775424,"data_stored":6250897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.795560"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.914190+0000","last_change":"2024-09-17T04:31:10.914190+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.914190+0000","last_undegraded":"2024-09-17T04:31:10.914190+0000","last_fullsized":"2024-09-17T04:31:10.914190+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.904387+0000","last_change":"2024-09-17T04:31:10.904387+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.904387+0000","last_undegraded":"2024-09-17T04:31:10.904387+0000","last_fullsized":"2024-09-17T04:31:10.904387+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.917302+0000","last_change":"2024-09-17T04:31:10.917302+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.917302+0000","last_undegraded":"2024-09-17T04:31:10.917302+0000","last_fullsized":"2024-09-17T04:31:10.917302+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.912667+0000","last_change":"2024-09-17T04:31:10.912667+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.912667+0000","last_undegraded":"2024-09-17T04:31:10.912667+0000","last_fullsized":"2024-09-17T04:31:10.912667+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.910610+0000","last_change":"2024-09-17T04:31:10.910610+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.910610+0000","last_undegraded":"2024-09-17T04:31:10.910610+0000","last_fullsized":"2024-09-17T04:31:10.910610+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.903895+0000","last_change":"2024-09-17T04:31:10.903895+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.903895+0000","last_undegraded":"2024-09-17T04:31:10.903895+0000","last_fullsized":"2024-09-17T04:31:10.903895+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.904370+0000","last_change":"2024-09-17T04:31:10.904370+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.904370+0000","last_undegraded":"2024-09-17T04:31:10.904370+0000","last_fullsized":"2024-09-17T04:31:10.904370+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.912073+0000","last_change":"2024-09-17T04:31:10.912073+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.912073+0000","last_undegraded":"2024-09-17T04:31:10.912073+0000","last_fullsized":"2024-09-17T04:31:10.912073+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.915843+0000","last_change":"2024-09-17T04:31:10.915843+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.915843+0000","last_undegraded":"2024-09-17T04:31:10.915843+0000","last_fullsized":"2024-09-17T04:31:10.915843+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.903841+0000","last_change":"2024-09-17T04:31:10.903841+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.903841+0000","last_undegraded":"2024-09-17T04:31:10.903841+0000","last_fullsized":"2024-09-17T04:31:10.903841+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.912182+0000","last_change":"2024-09-17T04:31:10.912182+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.912182+0000","last_undegraded":"2024-09-17T04:31:10.912182+0000","last_fullsized":"2024-09-17T04:31:10.912182+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.913434+0000","last_change":"2024-09-17T04:31:10.913434+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.913434+0000","last_undegraded":"2024-09-17T04:31:10.913434+0000","last_fullsized":"2024-09-17T04:31:10.913434+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.915532+0000","last_change":"2024-09-17T04:31:10.915532+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.915532+0000","last_undegraded":"2024-09-17T04:31:10.915532+0000","last_fullsized":"2024-09-17T04:31:10.915532+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.903634+0000","last_change":"2024-09-17T04:31:10.903634+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.903634+0000","last_undegraded":"2024-09-17T04:31:10.903634+0000","last_fullsized":"2024-09-17T04:31:10.903634+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.913164+0000","last_change":"2024-09-17T04:31:10.913164+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.913164+0000","last_undegraded":"2024-09-17T04:31:10.913164+0000","last_fullsized":"2024-09-17T04:31:10.913164+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.912271+0000","last_change":"2024-09-17T04:31:10.912271+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.912271+0000","last_undegraded":"2024-09-17T04:31:10.912271+0000","last_fullsized":"2024-09-17T04:31:10.912271+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.904605+0000","last_change":"2024-09-17T04:31:10.904605+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.904605+0000","last_undegraded":"2024-09-17T04:31:10.904605+0000","last_fullsized":"2024-09-17T04:31:10.904605+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.912776+0000","last_change":"2024-09-17T04:31:10.912776+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.912776+0000","last_undegraded":"2024-09-17T04:31:10.912776+0000","last_fullsized":"2024-09-17T04:31:10.912776+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.911756+0000","last_change":"2024-09-17T04:31:10.911756+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.911756+0000","last_undegraded":"2024-09-17T04:31:10.911756+0000","last_fullsized":"2024-09-17T04:31:10.911756+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.911841+0000","last_change":"2024-09-17T04:31:10.911841+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.911841+0000","last_undegraded":"2024-09-17T04:31:10.911841+0000","last_fullsized":"2024-09-17T04:31:10.911841+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.916213+0000","last_change":"2024-09-17T04:31:10.916213+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.916213+0000","last_undegraded":"2024-09-17T04:31:10.916213+0000","last_fullsized":"2024-09-17T04:31:10.916213+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.910761+0000","last_change":"2024-09-17T04:31:10.910761+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.910761+0000","last_undegraded":"2024-09-17T04:31:10.910761+0000","last_fullsized":"2024-09-17T04:31:10.910761+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.914620+0000","last_change":"2024-09-17T04:31:10.914620+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.914620+0000","last_undegraded":"2024-09-17T04:31:10.914620+0000","last_fullsized":"2024-09-17T04:31:10.914620+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.907757+0000","last_change":"2024-09-17T04:31:10.907757+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.907757+0000","last_undegraded":"2024-09-17T04:31:10.907757+0000","last_fullsized":"2024-09-17T04:31:10.907757+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.904983+0000","last_change":"2024-09-17T04:31:10.904983+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.904983+0000","last_undegraded":"2024-09-17T04:31:10.904983+0000","last_fullsized":"2024-09-17T04:31:10.904983+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.912322+0000","last_change":"2024-09-17T04:31:10.912322+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.912322+0000","last_undegraded":"2024-09-17T04:31:10.912322+0000","last_fullsized":"2024-09-17T04:31:10.912322+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909621+0000","last_change":"2024-09-17T04:31:10.909621+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909621+0000","last_undegraded":"2024-09-17T04:31:10.909621+0000","last_fullsized":"2024-09-17T04:31:10.909621+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909027+0000","last_change":"2024-09-17T04:31:10.909027+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909027+0000","last_undegraded":"2024-09-17T04:31:10.909027+0000","last_fullsized":"2024-09-17T04:31:10.909027+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.910178+0000","last_change":"2024-09-17T04:31:10.910178+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.910178+0000","last_undegraded":"2024-09-17T04:31:10.910178+0000","last_fullsized":"2024-09-17T04:31:10.910178+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.911090+0000","last_change":"2024-09-17T04:31:10.911090+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.911090+0000","last_undegraded":"2024-09-17T04:31:10.911090+0000","last_fullsized":"2024-09-17T04:31:10.911090+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909120+0000","last_change":"2024-09-17T04:31:10.909120+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909120+0000","last_undegraded":"2024-09-17T04:31:10.909120+0000","last_fullsized":"2024-09-17T04:31:10.909120+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.903949+0000","last_change":"2024-09-17T04:31:10.903949+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.903949+0000","last_undegraded":"2024-09-17T04:31:10.903949+0000","last_fullsized":"2024-09-17T04:31:10.903949+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.910613+0000","last_change":"2024-09-17T04:31:10.910613+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.910613+0000","last_undegraded":"2024-09-17T04:31:10.910613+0000","last_fullsized":"2024-09-17T04:31:10.910613+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.928277+0000","last_change":"2024-09-17T04:31:10.925886+0000","last_active":"2024-09-17T04:31:10.928277+0000","last_peered":"2024-09-17T04:31:10.928277+0000","last_clean":"2024-09-17T04:31:10.928277+0000","last_became_active":"2024-09-17T04:31:10.925455+0000","last_became_peered":"2024-09-17T04:31:10.925455+0000","last_unstale":"2024-09-17T04:31:10.928277+0000","last_undegraded":"2024-09-17T04:31:10.928277+0000","last_fullsized":"2024-09-17T04:31:10.928277+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:01.185188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.914848+0000","last_change":"2024-09-17T04:31:10.914848+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.914848+0000","last_undegraded":"2024-09-17T04:31:10.914848+0000","last_fullsized":"2024-09-17T04:31:10.914848+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.925524+0000","last_change":"2024-09-17T04:31:10.923497+0000","last_active":"2024-09-17T04:31:10.925524+0000","last_peered":"2024-09-17T04:31:10.925524+0000","last_clean":"2024-09-17T04:31:10.925524+0000","last_became_active":"2024-09-17T04:31:10.923086+0000","last_became_peered":"2024-09-17T04:31:10.923086+0000","last_unstale":"2024-09-17T04:31:10.925524+0000","last_undegraded":"2024-09-17T04:31:10.925524+0000","last_fullsized":"2024-09-17T04:31:10.925524+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:25.527822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909426+0000","last_change":"2024-09-17T04:31:10.909426+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909426+0000","last_undegraded":"2024-09-17T04:31:10.909426+0000","last_fullsized":"2024-09-17T04:31:10.909426+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.923768+0000","last_change":"2024-09-17T04:31:10.922177+0000","last_active":"2024-09-17T04:31:10.923768+0000","last_peered":"2024-09-17T04:31:10.923768+0000","last_clean":"2024-09-17T04:31:10.923768+0000","last_became_active":"2024-09-17T04:31:10.921875+0000","last_became_peered":"2024-09-17T04:31:10.921875+0000","last_unstale":"2024-09-17T04:31:10.923768+0000","last_undegraded":"2024-09-17T04:31:10.923768+0000","last_fullsized":"2024-09-17T04:31:10.923768+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:32.470330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.910133+0000","last_change":"2024-09-17T04:31:10.910133+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.910133+0000","last_undegraded":"2024-09-17T04:31:10.910133+0000","last_fullsized":"2024-09-17T04:31:10.910133+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.919580+0000","last_change":"2024-09-17T04:31:10.917285+0000","last_active":"2024-09-17T04:31:10.919580+0000","last_peered":"2024-09-17T04:31:10.919580+0000","last_clean":"2024-09-17T04:31:10.919580+0000","last_became_active":"2024-09-17T04:31:10.916739+0000","last_became_peered":"2024-09-17T04:31:10.916739+0000","last_unstale":"2024-09-17T04:31:10.919580+0000","last_undegraded":"2024-09-17T04:31:10.919580+0000","last_fullsized":"2024-09-17T04:31:10.919580+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:17.262185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909946+0000","last_change":"2024-09-17T04:31:10.909946+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909946+0000","last_undegraded":"2024-09-17T04:31:10.909946+0000","last_fullsized":"2024-09-17T04:31:10.909946+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.918268+0000","last_change":"2024-09-17T04:31:10.913568+0000","last_active":"2024-09-17T04:31:10.918268+0000","last_peered":"2024-09-17T04:31:10.918268+0000","last_clean":"2024-09-17T04:31:10.918268+0000","last_became_active":"2024-09-17T04:31:10.913273+0000","last_became_peered":"2024-09-17T04:31:10.913273+0000","last_unstale":"2024-09-17T04:31:10.918268+0000","last_undegraded":"2024-09-17T04:31:10.918268+0000","last_fullsized":"2024-09-17T04:31:10.918268+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:10.022361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.905237+0000","last_change":"2024-09-17T04:31:10.905237+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.905237+0000","last_undegraded":"2024-09-17T04:31:10.905237+0000","last_fullsized":"2024-09-17T04:31:10.905237+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.922101+0000","last_change":"2024-09-17T04:31:10.917769+0000","last_active":"2024-09-17T04:31:10.922101+0000","last_peered":"2024-09-17T04:31:10.922101+0000","last_clean":"2024-09-17T04:31:10.922101+0000","last_became_active":"2024-09-17T04:31:10.917493+0000","last_became_peered":"2024-09-17T04:31:10.917493+0000","last_unstale":"2024-09-17T04:31:10.922101+0000","last_undegraded":"2024-09-17T04:31:10.922101+0000","last_fullsized":"2024-09-17T04:31:10.922101+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:40:04.424615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.911438+0000","last_change":"2024-09-17T04:31:10.911438+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.911438+0000","last_undegraded":"2024-09-17T04:31:10.911438+0000","last_fullsized":"2024-09-17T04:31:10.911438+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.927265+0000","last_change":"2024-09-17T04:31:10.919927+0000","last_active":"2024-09-17T04:31:10.927265+0000","last_peered":"2024-09-17T04:31:10.927265+0000","last_clean":"2024-09-17T04:31:10.927265+0000","last_became_active":"2024-09-17T04:31:10.919536+0000","last_became_peered":"2024-09-17T04:31:10.919536+0000","last_unstale":"2024-09-17T04:31:10.927265+0000","last_undegraded":"2024-09-17T04:31:10.927265+0000","last_fullsized":"2024-09-17T04:31:10.927265+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:46.091206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.903852+0000","last_change":"2024-09-17T04:31:10.903852+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.903852+0000","last_undegraded":"2024-09-17T04:31:10.903852+0000","last_fullsized":"2024-09-17T04:31:10.903852+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.927852+0000","last_change":"2024-09-17T04:31:10.926647+0000","last_active":"2024-09-17T04:31:10.927852+0000","last_peered":"2024-09-17T04:31:10.927852+0000","last_clean":"2024-09-17T04:31:10.927852+0000","last_became_active":"2024-09-17T04:31:10.926287+0000","last_became_peered":"2024-09-17T04:31:10.926287+0000","last_unstale":"2024-09-17T04:31:10.927852+0000","last_undegraded":"2024-09-17T04:31:10.927852+0000","last_fullsized":"2024-09-17T04:31:10.927852+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:41.700033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909109+0000","last_change":"2024-09-17T04:31:10.909109+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909109+0000","last_undegraded":"2024-09-17T04:31:10.909109+0000","last_fullsized":"2024-09-17T04:31:10.909109+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.928213+0000","last_change":"2024-09-17T04:31:10.926032+0000","last_active":"2024-09-17T04:31:10.928213+0000","last_peered":"2024-09-17T04:31:10.928213+0000","last_clean":"2024-09-17T04:31:10.928213+0000","last_became_active":"2024-09-17T04:31:10.925736+0000","last_became_peered":"2024-09-17T04:31:10.925736+0000","last_unstale":"2024-09-17T04:31:10.928213+0000","last_undegraded":"2024-09-17T04:31:10.928213+0000","last_fullsized":"2024-09-17T04:31:10.928213+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:40.221385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.912472+0000","last_change":"2024-09-17T04:31:10.912472+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.912472+0000","last_undegraded":"2024-09-17T04:31:10.912472+0000","last_fullsized":"2024-09-17T04:31:10.912472+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.930968+0000","last_change":"2024-09-17T04:31:10.930027+0000","last_active":"2024-09-17T04:31:10.930968+0000","last_peered":"2024-09-17T04:31:10.930968+0000","last_clean":"2024-09-17T04:31:10.930968+0000","last_became_active":"2024-09-17T04:31:10.929648+0000","last_became_peered":"2024-09-17T04:31:10.929648+0000","last_unstale":"2024-09-17T04:31:10.930968+0000","last_undegraded":"2024-09-17T04:31:10.930968+0000","last_fullsized":"2024-09-17T04:31:10.930968+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:18:25.078648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.908795+0000","last_change":"2024-09-17T04:31:10.908795+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.908795+0000","last_undegraded":"2024-09-17T04:31:10.908795+0000","last_fullsized":"2024-09-17T04:31:10.908795+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.920305+0000","last_change":"2024-09-17T04:31:10.917626+0000","last_active":"2024-09-17T04:31:10.920305+0000","last_peered":"2024-09-17T04:31:10.920305+0000","last_clean":"2024-09-17T04:31:10.920305+0000","last_became_active":"2024-09-17T04:31:10.917239+0000","last_became_peered":"2024-09-17T04:31:10.917239+0000","last_unstale":"2024-09-17T04:31:10.920305+0000","last_undegraded":"2024-09-17T04:31:10.920305+0000","last_fullsized":"2024-09-17T04:31:10.920305+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:29.945578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.908678+0000","last_change":"2024-09-17T04:31:10.908678+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.908678+0000","last_undegraded":"2024-09-17T04:31:10.908678+0000","last_fullsized":"2024-09-17T04:31:10.908678+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.931270+0000","last_change":"2024-09-17T04:31:10.929556+0000","last_active":"2024-09-17T04:31:10.931270+0000","last_peered":"2024-09-17T04:31:10.931270+0000","last_clean":"2024-09-17T04:31:10.931270+0000","last_became_active":"2024-09-17T04:31:10.929171+0000","last_became_peered":"2024-09-17T04:31:10.929171+0000","last_unstale":"2024-09-17T04:31:10.931270+0000","last_undegraded":"2024-09-17T04:31:10.931270+0000","last_fullsized":"2024-09-17T04:31:10.931270+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:00.689611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909041+0000","last_change":"2024-09-17T04:31:10.909041+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909041+0000","last_undegraded":"2024-09-17T04:31:10.909041+0000","last_fullsized":"2024-09-17T04:31:10.909041+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.925160+0000","last_change":"2024-09-17T04:31:10.921170+0000","last_active":"2024-09-17T04:31:10.925160+0000","last_peered":"2024-09-17T04:31:10.925160+0000","last_clean":"2024-09-17T04:31:10.925160+0000","last_became_active":"2024-09-17T04:31:10.920873+0000","last_became_peered":"2024-09-17T04:31:10.920873+0000","last_unstale":"2024-09-17T04:31:10.925160+0000","last_undegraded":"2024-09-17T04:31:10.925160+0000","last_fullsized":"2024-09-17T04:31:10.925160+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:09:40.221275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.903934+0000","last_change":"2024-09-17T04:31:10.903934+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.903934+0000","last_undegraded":"2024-09-17T04:31:10.903934+0000","last_fullsized":"2024-09-17T04:31:10.903934+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.924402+0000","last_change":"2024-09-17T04:31:10.920901+0000","last_active":"2024-09-17T04:31:10.924402+0000","last_peered":"2024-09-17T04:31:10.924402+0000","last_clean":"2024-09-17T04:31:10.924402+0000","last_became_active":"2024-09-17T04:31:10.920586+0000","last_became_peered":"2024-09-17T04:31:10.920586+0000","last_unstale":"2024-09-17T04:31:10.924402+0000","last_undegraded":"2024-09-17T04:31:10.924402+0000","last_fullsized":"2024-09-17T04:31:10.924402+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:59.093831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.915262+0000","last_change":"2024-09-17T04:31:10.915262+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.915262+0000","last_undegraded":"2024-09-17T04:31:10.915262+0000","last_fullsized":"2024-09-17T04:31:10.915262+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.929486+0000","last_change":"2024-09-17T04:31:10.924812+0000","last_active":"2024-09-17T04:31:10.929486+0000","last_peered":"2024-09-17T04:31:10.929486+0000","last_clean":"2024-09-17T04:31:10.929486+0000","last_became_active":"2024-09-17T04:31:10.924423+0000","last_became_peered":"2024-09-17T04:31:10.924423+0000","last_unstale":"2024-09-17T04:31:10.929486+0000","last_undegraded":"2024-09-17T04:31:10.929486+0000","last_fullsized":"2024-09-17T04:31:10.929486+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:07.672259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.917600+0000","last_change":"2024-09-17T04:31:10.917600+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.917600+0000","last_undegraded":"2024-09-17T04:31:10.917600+0000","last_fullsized":"2024-09-17T04:31:10.917600+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.926479+0000","last_change":"2024-09-17T04:31:10.924255+0000","last_active":"2024-09-17T04:31:10.926479+0000","last_peered":"2024-09-17T04:31:10.926479+0000","last_clean":"2024-09-17T04:31:10.926479+0000","last_became_active":"2024-09-17T04:31:10.923887+0000","last_became_peered":"2024-09-17T04:31:10.923887+0000","last_unstale":"2024-09-17T04:31:10.926479+0000","last_undegraded":"2024-09-17T04:31:10.926479+0000","last_fullsized":"2024-09-17T04:31:10.926479+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:42.584932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.911484+0000","last_change":"2024-09-17T04:31:10.911484+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.911484+0000","last_undegraded":"2024-09-17T04:31:10.911484+0000","last_fullsized":"2024-09-17T04:31:10.911484+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.930817+0000","last_change":"2024-09-17T04:31:10.930022+0000","last_active":"2024-09-17T04:31:10.930817+0000","last_peered":"2024-09-17T04:31:10.930817+0000","last_clean":"2024-09-17T04:31:10.930817+0000","last_became_active":"2024-09-17T04:31:10.929681+0000","last_became_peered":"2024-09-17T04:31:10.929681+0000","last_unstale":"2024-09-17T04:31:10.930817+0000","last_undegraded":"2024-09-17T04:31:10.930817+0000","last_fullsized":"2024-09-17T04:31:10.930817+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:16.582319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.911356+0000","last_change":"2024-09-17T04:31:10.911356+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.911356+0000","last_undegraded":"2024-09-17T04:31:10.911356+0000","last_fullsized":"2024-09-17T04:31:10.911356+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.918083+0000","last_change":"2024-09-17T04:31:10.910443+0000","last_active":"2024-09-17T04:31:10.918083+0000","last_peered":"2024-09-17T04:31:10.918083+0000","last_clean":"2024-09-17T04:31:10.918083+0000","last_became_active":"2024-09-17T04:31:10.910084+0000","last_became_peered":"2024-09-17T04:31:10.910084+0000","last_unstale":"2024-09-17T04:31:10.918083+0000","last_undegraded":"2024-09-17T04:31:10.918083+0000","last_fullsized":"2024-09-17T04:31:10.918083+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:02.897825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.904544+0000","last_change":"2024-09-17T04:31:10.904544+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.904544+0000","last_undegraded":"2024-09-17T04:31:10.904544+0000","last_fullsized":"2024-09-17T04:31:10.904544+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.921482+0000","last_change":"2024-09-17T04:31:10.918599+0000","last_active":"2024-09-17T04:31:10.921482+0000","last_peered":"2024-09-17T04:31:10.921482+0000","last_clean":"2024-09-17T04:31:10.921482+0000","last_became_active":"2024-09-17T04:31:10.918208+0000","last_became_peered":"2024-09-17T04:31:10.918208+0000","last_unstale":"2024-09-17T04:31:10.921482+0000","last_undegraded":"2024-09-17T04:31:10.921482+0000","last_fullsized":"2024-09-17T04:31:10.921482+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:29.662127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909400+0000","last_change":"2024-09-17T04:31:10.909400+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909400+0000","last_undegraded":"2024-09-17T04:31:10.909400+0000","last_fullsized":"2024-09-17T04:31:10.909400+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.929291+0000","last_change":"2024-09-17T04:31:10.927592+0000","last_active":"2024-09-17T04:31:10.929291+0000","last_peered":"2024-09-17T04:31:10.929291+0000","last_clean":"2024-09-17T04:31:10.929291+0000","last_became_active":"2024-09-17T04:31:10.926639+0000","last_became_peered":"2024-09-17T04:31:10.926639+0000","last_unstale":"2024-09-17T04:31:10.929291+0000","last_undegraded":"2024-09-17T04:31:10.929291+0000","last_fullsized":"2024-09-17T04:31:10.929291+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:50.191404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.904320+0000","last_change":"2024-09-17T04:31:10.904320+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.904320+0000","last_undegraded":"2024-09-17T04:31:10.904320+0000","last_fullsized":"2024-09-17T04:31:10.904320+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.928276+0000","last_change":"2024-09-17T04:31:10.926073+0000","last_active":"2024-09-17T04:31:10.928276+0000","last_peered":"2024-09-17T04:31:10.928276+0000","last_clean":"2024-09-17T04:31:10.928276+0000","last_became_active":"2024-09-17T04:31:10.925750+0000","last_became_peered":"2024-09-17T04:31:10.925750+0000","last_unstale":"2024-09-17T04:31:10.928276+0000","last_undegraded":"2024-09-17T04:31:10.928276+0000","last_fullsized":"2024-09-17T04:31:10.928276+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:10:46.052152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.903377+0000","last_change":"2024-09-17T04:31:10.903377+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.903377+0000","last_undegraded":"2024-09-17T04:31:10.903377+0000","last_fullsized":"2024-09-17T04:31:10.903377+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.923038+0000","last_change":"2024-09-17T04:31:10.917223+0000","last_active":"2024-09-17T04:31:10.923038+0000","last_peered":"2024-09-17T04:31:10.923038+0000","last_clean":"2024-09-17T04:31:10.923038+0000","last_became_active":"2024-09-17T04:31:10.916765+0000","last_became_peered":"2024-09-17T04:31:10.916765+0000","last_unstale":"2024-09-17T04:31:10.923038+0000","last_undegraded":"2024-09-17T04:31:10.923038+0000","last_fullsized":"2024-09-17T04:31:10.923038+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:37.040110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.910300+0000","last_change":"2024-09-17T04:31:10.910300+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.910300+0000","last_undegraded":"2024-09-17T04:31:10.910300+0000","last_fullsized":"2024-09-17T04:31:10.910300+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.927355+0000","last_change":"2024-09-17T04:31:10.926100+0000","last_active":"2024-09-17T04:31:10.927355+0000","last_peered":"2024-09-17T04:31:10.927355+0000","last_clean":"2024-09-17T04:31:10.927355+0000","last_became_active":"2024-09-17T04:31:10.925806+0000","last_became_peered":"2024-09-17T04:31:10.925806+0000","last_unstale":"2024-09-17T04:31:10.927355+0000","last_undegraded":"2024-09-17T04:31:10.927355+0000","last_fullsized":"2024-09-17T04:31:10.927355+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:05.786635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.904697+0000","last_change":"2024-09-17T04:31:10.904697+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.904697+0000","last_undegraded":"2024-09-17T04:31:10.904697+0000","last_fullsized":"2024-09-17T04:31:10.904697+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.924475+0000","last_change":"2024-09-17T04:31:10.919421+0000","last_active":"2024-09-17T04:31:10.924475+0000","last_peered":"2024-09-17T04:31:10.924475+0000","last_clean":"2024-09-17T04:31:10.924475+0000","last_became_active":"2024-09-17T04:31:10.919126+0000","last_became_peered":"2024-09-17T04:31:10.919126+0000","last_unstale":"2024-09-17T04:31:10.924475+0000","last_undegraded":"2024-09-17T04:31:10.924475+0000","last_fullsized":"2024-09-17T04:31:10.924475+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:40.608118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909084+0000","last_change":"2024-09-17T04:31:10.909084+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909084+0000","last_undegraded":"2024-09-17T04:31:10.909084+0000","last_fullsized":"2024-09-17T04:31:10.909084+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.931338+0000","last_change":"2024-09-17T04:31:10.930374+0000","last_active":"2024-09-17T04:31:10.931338+0000","last_peered":"2024-09-17T04:31:10.931338+0000","last_clean":"2024-09-17T04:31:10.931338+0000","last_became_active":"2024-09-17T04:31:10.930043+0000","last_became_peered":"2024-09-17T04:31:10.930043+0000","last_unstale":"2024-09-17T04:31:10.931338+0000","last_undegraded":"2024-09-17T04:31:10.931338+0000","last_fullsized":"2024-09-17T04:31:10.931338+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:02.897825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.904139+0000","last_change":"2024-09-17T04:31:10.904139+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.904139+0000","last_undegraded":"2024-09-17T04:31:10.904139+0000","last_fullsized":"2024-09-17T04:31:10.904139+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.930617+0000","last_change":"2024-09-17T04:31:10.929027+0000","last_active":"2024-09-17T04:31:10.930617+0000","last_peered":"2024-09-17T04:31:10.930617+0000","last_clean":"2024-09-17T04:31:10.930617+0000","last_became_active":"2024-09-17T04:31:10.928767+0000","last_became_peered":"2024-09-17T04:31:10.928767+0000","last_unstale":"2024-09-17T04:31:10.930617+0000","last_undegraded":"2024-09-17T04:31:10.930617+0000","last_fullsized":"2024-09-17T04:31:10.930617+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:06.263954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.905447+0000","last_change":"2024-09-17T04:31:10.905447+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.905447+0000","last_undegraded":"2024-09-17T04:31:10.905447+0000","last_fullsized":"2024-09-17T04:31:10.905447+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.922602+0000","last_change":"2024-09-17T04:31:10.920453+0000","last_active":"2024-09-17T04:31:10.922602+0000","last_peered":"2024-09-17T04:31:10.922602+0000","last_clean":"2024-09-17T04:31:10.922602+0000","last_became_active":"2024-09-17T04:31:10.919363+0000","last_became_peered":"2024-09-17T04:31:10.919363+0000","last_unstale":"2024-09-17T04:31:10.922602+0000","last_undegraded":"2024-09-17T04:31:10.922602+0000","last_fullsized":"2024-09-17T04:31:10.922602+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:04:35.664082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909560+0000","last_change":"2024-09-17T04:31:10.909560+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909560+0000","last_undegraded":"2024-09-17T04:31:10.909560+0000","last_fullsized":"2024-09-17T04:31:10.909560+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.924812+0000","last_change":"2024-09-17T04:31:10.923121+0000","last_active":"2024-09-17T04:31:10.924812+0000","last_peered":"2024-09-17T04:31:10.924812+0000","last_clean":"2024-09-17T04:31:10.924812+0000","last_became_active":"2024-09-17T04:31:10.922374+0000","last_became_peered":"2024-09-17T04:31:10.922374+0000","last_unstale":"2024-09-17T04:31:10.924812+0000","last_undegraded":"2024-09-17T04:31:10.924812+0000","last_fullsized":"2024-09-17T04:31:10.924812+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:23:56.512701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.911942+0000","last_change":"2024-09-17T04:31:10.911942+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.911942+0000","last_undegraded":"2024-09-17T04:31:10.911942+0000","last_fullsized":"2024-09-17T04:31:10.911942+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.927043+0000","last_change":"2024-09-17T04:31:10.925280+0000","last_active":"2024-09-17T04:31:10.927043+0000","last_peered":"2024-09-17T04:31:10.927043+0000","last_clean":"2024-09-17T04:31:10.927043+0000","last_became_active":"2024-09-17T04:31:10.924762+0000","last_became_peered":"2024-09-17T04:31:10.924762+0000","last_unstale":"2024-09-17T04:31:10.927043+0000","last_undegraded":"2024-09-17T04:31:10.927043+0000","last_fullsized":"2024-09-17T04:31:10.927043+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:28.910545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.910074+0000","last_change":"2024-09-17T04:31:10.910074+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.910074+0000","last_undegraded":"2024-09-17T04:31:10.910074+0000","last_fullsized":"2024-09-17T04:31:10.910074+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.928272+0000","last_change":"2024-09-17T04:31:10.926916+0000","last_active":"2024-09-17T04:31:10.928272+0000","last_peered":"2024-09-17T04:31:10.928272+0000","last_clean":"2024-09-17T04:31:10.928272+0000","last_became_active":"2024-09-17T04:31:10.926611+0000","last_became_peered":"2024-09-17T04:31:10.926611+0000","last_unstale":"2024-09-17T04:31:10.928272+0000","last_undegraded":"2024-09-17T04:31:10.928272+0000","last_fullsized":"2024-09-17T04:31:10.928272+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:39.197809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.903334+0000","last_change":"2024-09-17T04:31:10.903334+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.903334+0000","last_undegraded":"2024-09-17T04:31:10.903334+0000","last_fullsized":"2024-09-17T04:31:10.903334+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.921772+0000","last_change":"2024-09-17T04:31:10.918838+0000","last_active":"2024-09-17T04:31:10.921772+0000","last_peered":"2024-09-17T04:31:10.921772+0000","last_clean":"2024-09-17T04:31:10.921772+0000","last_became_active":"2024-09-17T04:31:10.918535+0000","last_became_peered":"2024-09-17T04:31:10.918535+0000","last_unstale":"2024-09-17T04:31:10.921772+0000","last_undegraded":"2024-09-17T04:31:10.921772+0000","last_fullsized":"2024-09-17T04:31:10.921772+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:32.795755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.910191+0000","last_change":"2024-09-17T04:31:10.910191+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.910191+0000","last_undegraded":"2024-09-17T04:31:10.910191+0000","last_fullsized":"2024-09-17T04:31:10.910191+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.928967+0000","last_change":"2024-09-17T04:31:10.927734+0000","last_active":"2024-09-17T04:31:10.928967+0000","last_peered":"2024-09-17T04:31:10.928967+0000","last_clean":"2024-09-17T04:31:10.928967+0000","last_became_active":"2024-09-17T04:31:10.926995+0000","last_became_peered":"2024-09-17T04:31:10.926995+0000","last_unstale":"2024-09-17T04:31:10.928967+0000","last_undegraded":"2024-09-17T04:31:10.928967+0000","last_fullsized":"2024-09-17T04:31:10.928967+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:48:42.671646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"76'79","reported_seq":313,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.917506+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:31:10.917506+0000","last_peered":"2024-09-17T04:31:10.917506+0000","last_clean":"2024-09-17T04:31:10.917506+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:31:10.917506+0000","last_undegraded":"2024-09-17T04:31:10.917506+0000","last_fullsized":"2024-09-17T04:31:10.917506+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476827,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33188,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338652,"statfs":{"total":96636764160,"available":96602779648,"internally_reserved":0,"allocated":2043904,"data_stored":1224926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476826,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1449984,"data_stored":634558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476827,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1449984,"data_stored":634558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476827,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1437696,"data_stored":628255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542234,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32612,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339228,"statfs":{"total":96636764160,"available":96603369472,"internally_reserved":0,"allocated":1449984,"data_stored":634558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509530,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1449984,"data_stored":634558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509529,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1449984,"data_stored":634558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509529,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33188,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338652,"statfs":{"total":96636764160,"available":96602779648,"internally_reserved":0,"allocated":2043904,"data_stored":1224926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:31:16.821 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:31:17.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.078+0000 7fcdb8000640 1 -- 172.21.15.100:0/1708966094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdb015d760 msgr2=0x7fcdb01535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:17.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.078+0000 7fcdb8000640 1 --2- 172.21.15.100:0/1708966094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdb015d760 0x7fcdb01535c0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fcda4005fd0 tx=0x7fcda402a540 comp rx=0 tx=0).stop 2024-09-17T04:31:17.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.078+0000 7fcdb8000640 1 -- 172.21.15.100:0/1708966094 shutdown_connections 2024-09-17T04:31:17.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.078+0000 7fcdb8000640 1 --2- 172.21.15.100:0/1708966094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdb015d760 0x7fcdb01535c0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:17.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.078+0000 7fcdb8000640 1 -- 172.21.15.100:0/1708966094 >> 172.21.15.100:0/1708966094 conn(0x7fcdb006d730 msgr2=0x7fcdb00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:17.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.078+0000 7fcdb8000640 1 -- 172.21.15.100:0/1708966094 shutdown_connections 2024-09-17T04:31:17.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.078+0000 7fcdb8000640 1 -- 172.21.15.100:0/1708966094 wait complete. 2024-09-17T04:31:17.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.081+0000 7fcdb8000640 1 Processor -- start 2024-09-17T04:31:17.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.081+0000 7fcdb8000640 1 -- start start 2024-09-17T04:31:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.082+0000 7fcdb8000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdb00ad0e0 0x7fcdb0149760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.082+0000 7fcdb8000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdb00b3550 0x7fcdb01f5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.082+0000 7fcdb8000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdb015d760 0x7fcdb00b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.082+0000 7fcdb8000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb006f240 con 0x7fcdb00b3550 2024-09-17T04:31:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.082+0000 7fcdb8000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb00b4e30 con 0x7fcdb00ad0e0 2024-09-17T04:31:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.082+0000 7fcdb8000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcdb00b4070 con 0x7fcdb015d760 2024-09-17T04:31:17.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.082+0000 7fcdb4c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdb00b3550 0x7fcdb01f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:17.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.082+0000 7fcdb4c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdb00b3550 0x7fcdb01f5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55472/0 (socket says 172.21.15.100:55472) 2024-09-17T04:31:17.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.082+0000 7fcdb4c00640 1 -- 172.21.15.100:0/3791594769 learned_addr learned my addr 172.21.15.100:0/3791594769 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:17.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.083+0000 7fcdb5600640 1 --2- 172.21.15.100:0/3791594769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdb00ad0e0 0x7fcdb0149760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:17.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.083+0000 7fcdb6000640 1 --2- 172.21.15.100:0/3791594769 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdb015d760 0x7fcdb00b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:17.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.083+0000 7fcdb4c00640 1 -- 172.21.15.100:0/3791594769 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdb015d760 msgr2=0x7fcdb00b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:17.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.083+0000 7fcdb4c00640 1 --2- 172.21.15.100:0/3791594769 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdb015d760 0x7fcdb00b0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:17.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.083+0000 7fcdb4c00640 1 -- 172.21.15.100:0/3791594769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdb00ad0e0 msgr2=0x7fcdb0149760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:17.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.083+0000 7fcdb4c00640 1 --2- 172.21.15.100:0/3791594769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdb00ad0e0 0x7fcdb0149760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:17.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.083+0000 7fcdb4c00640 1 -- 172.21.15.100:0/3791594769 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb0141fd0 con 0x7fcdb00b3550 2024-09-17T04:31:17.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.083+0000 7fcdb4c00640 1 --2- 172.21.15.100:0/3791594769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdb00b3550 0x7fcdb01f5650 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c00b3f0 tx=0x7fcd9c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:17.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.083+0000 7fcdad600640 1 -- 172.21.15.100:0/3791594769 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c015020 con 0x7fcdb00b3550 2024-09-17T04:31:17.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.084+0000 7fcdad600640 1 -- 172.21.15.100:0/3791594769 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd9c011920 con 0x7fcdb00b3550 2024-09-17T04:31:17.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.084+0000 7fcdad600640 1 -- 172.21.15.100:0/3791594769 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c011d70 con 0x7fcdb00b3550 2024-09-17T04:31:17.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.084+0000 7fcdb8000640 1 -- 172.21.15.100:0/3791594769 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdb013e1c0 con 0x7fcdb00b3550 2024-09-17T04:31:17.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.084+0000 7fcdb8000640 1 -- 172.21.15.100:0/3791594769 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdb013e780 con 0x7fcdb00b3550 2024-09-17T04:31:17.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.085+0000 7fcdb8000640 1 -- 172.21.15.100:0/3791594769 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdb0149e40 con 0x7fcdb00b3550 2024-09-17T04:31:17.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.086+0000 7fcdad600640 1 -- 172.21.15.100:0/3791594769 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fcd9c018020 con 0x7fcdb00b3550 2024-09-17T04:31:17.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.087+0000 7fcdad600640 1 --2- 172.21.15.100:0/3791594769 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcd90079040 0x7fcd9007b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:17.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.088+0000 7fcdad600640 1 -- 172.21.15.100:0/3791594769 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7fcd9c09b1d0 con 0x7fcdb00b3550 2024-09-17T04:31:17.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.088+0000 7fcdb5600640 1 --2- 172.21.15.100:0/3791594769 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcd90079040 0x7fcd9007b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:17.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.088+0000 7fcdb5600640 1 --2- 172.21.15.100:0/3791594769 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcd90079040 0x7fcd9007b550 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcda4033000 tx=0x7fcda4004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:17.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.092+0000 7fcdad600640 1 -- 172.21.15.100:0/3791594769 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fcd9c01f030 con 0x7fcdb00b3550 2024-09-17T04:31:17.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.273+0000 7fcdb8000640 1 -- 172.21.15.100:0/3791594769 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcdb01535c0 con 0x7fcd90079040 2024-09-17T04:31:17.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.277+0000 7fcdad600640 1 -- 172.21.15.100:0/3791594769 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247116 (secure 0 0 0) 0x7fcdb01535c0 con 0x7fcd90079040 2024-09-17T04:31:17.280 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:17.282 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:31:17.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.281+0000 7fcdb8000640 1 -- 172.21.15.100:0/3791594769 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcd90079040 msgr2=0x7fcd9007b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:17.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.281+0000 7fcdb8000640 1 --2- 172.21.15.100:0/3791594769 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcd90079040 0x7fcd9007b550 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcda4033000 tx=0x7fcda4004730 comp rx=0 tx=0).stop 2024-09-17T04:31:17.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.281+0000 7fcdb8000640 1 -- 172.21.15.100:0/3791594769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdb00b3550 msgr2=0x7fcdb01f5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:17.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.281+0000 7fcdb8000640 1 --2- 172.21.15.100:0/3791594769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdb00b3550 0x7fcdb01f5650 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c00b3f0 tx=0x7fcd9c00b8b0 comp rx=0 tx=0).stop 2024-09-17T04:31:17.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.281+0000 7fcdb8000640 1 -- 172.21.15.100:0/3791594769 shutdown_connections 2024-09-17T04:31:17.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.281+0000 7fcdb8000640 1 --2- 172.21.15.100:0/3791594769 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcd90079040 0x7fcd9007b550 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:17.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.281+0000 7fcdb8000640 1 --2- 172.21.15.100:0/3791594769 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdb015d760 0x7fcdb00b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:17.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.281+0000 7fcdb8000640 1 --2- 172.21.15.100:0/3791594769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdb00b3550 0x7fcdb01f5650 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:17.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.281+0000 7fcdb8000640 1 --2- 172.21.15.100:0/3791594769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdb00ad0e0 0x7fcdb0149760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:17.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.281+0000 7fcdb8000640 1 -- 172.21.15.100:0/3791594769 >> 172.21.15.100:0/3791594769 conn(0x7fcdb006d730 msgr2=0x7fcdb015b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:17.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.281+0000 7fcdb8000640 1 -- 172.21.15.100:0/3791594769 shutdown_connections 2024-09-17T04:31:17.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.281+0000 7fcdb8000640 1 -- 172.21.15.100:0/3791594769 wait complete. 2024-09-17T04:31:17.305 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-09-17T04:31:15.688582+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262008,"kb_used_data":12476,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712712,"statfs":{"total":773094113280,"available":772825817088,"internally_reserved":0,"allocated":12775424,"data_stored":6250897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.795560"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.914190+0000","last_change":"2024-09-17T04:31:10.914190+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.914190+0000","last_undegraded":"2024-09-17T04:31:10.914190+0000","last_fullsized":"2024-09-17T04:31:10.914190+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.904387+0000","last_change":"2024-09-17T04:31:10.904387+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.904387+0000","last_undegraded":"2024-09-17T04:31:10.904387+0000","last_fullsized":"2024-09-17T04:31:10.904387+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.917302+0000","last_change":"2024-09-17T04:31:10.917302+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.917302+0000","last_undegraded":"2024-09-17T04:31:10.917302+0000","last_fullsized":"2024-09-17T04:31:10.917302+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.912667+0000","last_change":"2024-09-17T04:31:10.912667+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.912667+0000","last_undegraded":"2024-09-17T04:31:10.912667+0000","last_fullsized":"2024-09-17T04:31:10.912667+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.910610+0000","last_change":"2024-09-17T04:31:10.910610+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.910610+0000","last_undegraded":"2024-09-17T04:31:10.910610+0000","last_fullsized":"2024-09-17T04:31:10.910610+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.903895+0000","last_change":"2024-09-17T04:31:10.903895+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.903895+0000","last_undegraded":"2024-09-17T04:31:10.903895+0000","last_fullsized":"2024-09-17T04:31:10.903895+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.904370+0000","last_change":"2024-09-17T04:31:10.904370+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.904370+0000","last_undegraded":"2024-09-17T04:31:10.904370+0000","last_fullsized":"2024-09-17T04:31:10.904370+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.912073+0000","last_change":"2024-09-17T04:31:10.912073+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.912073+0000","last_undegraded":"2024-09-17T04:31:10.912073+0000","last_fullsized":"2024-09-17T04:31:10.912073+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.915843+0000","last_change":"2024-09-17T04:31:10.915843+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.915843+0000","last_undegraded":"2024-09-17T04:31:10.915843+0000","last_fullsized":"2024-09-17T04:31:10.915843+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.903841+0000","last_change":"2024-09-17T04:31:10.903841+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.903841+0000","last_undegraded":"2024-09-17T04:31:10.903841+0000","last_fullsized":"2024-09-17T04:31:10.903841+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.912182+0000","last_change":"2024-09-17T04:31:10.912182+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.912182+0000","last_undegraded":"2024-09-17T04:31:10.912182+0000","last_fullsized":"2024-09-17T04:31:10.912182+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.913434+0000","last_change":"2024-09-17T04:31:10.913434+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.913434+0000","last_undegraded":"2024-09-17T04:31:10.913434+0000","last_fullsized":"2024-09-17T04:31:10.913434+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.915532+0000","last_change":"2024-09-17T04:31:10.915532+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.915532+0000","last_undegraded":"2024-09-17T04:31:10.915532+0000","last_fullsized":"2024-09-17T04:31:10.915532+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.903634+0000","last_change":"2024-09-17T04:31:10.903634+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.903634+0000","last_undegraded":"2024-09-17T04:31:10.903634+0000","last_fullsized":"2024-09-17T04:31:10.903634+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.913164+0000","last_change":"2024-09-17T04:31:10.913164+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.913164+0000","last_undegraded":"2024-09-17T04:31:10.913164+0000","last_fullsized":"2024-09-17T04:31:10.913164+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.912271+0000","last_change":"2024-09-17T04:31:10.912271+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.912271+0000","last_undegraded":"2024-09-17T04:31:10.912271+0000","last_fullsized":"2024-09-17T04:31:10.912271+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.904605+0000","last_change":"2024-09-17T04:31:10.904605+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.904605+0000","last_undegraded":"2024-09-17T04:31:10.904605+0000","last_fullsized":"2024-09-17T04:31:10.904605+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.912776+0000","last_change":"2024-09-17T04:31:10.912776+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.912776+0000","last_undegraded":"2024-09-17T04:31:10.912776+0000","last_fullsized":"2024-09-17T04:31:10.912776+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.911756+0000","last_change":"2024-09-17T04:31:10.911756+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.911756+0000","last_undegraded":"2024-09-17T04:31:10.911756+0000","last_fullsized":"2024-09-17T04:31:10.911756+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.911841+0000","last_change":"2024-09-17T04:31:10.911841+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.911841+0000","last_undegraded":"2024-09-17T04:31:10.911841+0000","last_fullsized":"2024-09-17T04:31:10.911841+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.916213+0000","last_change":"2024-09-17T04:31:10.916213+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.916213+0000","last_undegraded":"2024-09-17T04:31:10.916213+0000","last_fullsized":"2024-09-17T04:31:10.916213+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.910761+0000","last_change":"2024-09-17T04:31:10.910761+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.910761+0000","last_undegraded":"2024-09-17T04:31:10.910761+0000","last_fullsized":"2024-09-17T04:31:10.910761+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.914620+0000","last_change":"2024-09-17T04:31:10.914620+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.914620+0000","last_undegraded":"2024-09-17T04:31:10.914620+0000","last_fullsized":"2024-09-17T04:31:10.914620+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.907757+0000","last_change":"2024-09-17T04:31:10.907757+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.907757+0000","last_undegraded":"2024-09-17T04:31:10.907757+0000","last_fullsized":"2024-09-17T04:31:10.907757+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.904983+0000","last_change":"2024-09-17T04:31:10.904983+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.904983+0000","last_undegraded":"2024-09-17T04:31:10.904983+0000","last_fullsized":"2024-09-17T04:31:10.904983+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.912322+0000","last_change":"2024-09-17T04:31:10.912322+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.912322+0000","last_undegraded":"2024-09-17T04:31:10.912322+0000","last_fullsized":"2024-09-17T04:31:10.912322+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909621+0000","last_change":"2024-09-17T04:31:10.909621+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909621+0000","last_undegraded":"2024-09-17T04:31:10.909621+0000","last_fullsized":"2024-09-17T04:31:10.909621+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909027+0000","last_change":"2024-09-17T04:31:10.909027+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909027+0000","last_undegraded":"2024-09-17T04:31:10.909027+0000","last_fullsized":"2024-09-17T04:31:10.909027+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.910178+0000","last_change":"2024-09-17T04:31:10.910178+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.910178+0000","last_undegraded":"2024-09-17T04:31:10.910178+0000","last_fullsized":"2024-09-17T04:31:10.910178+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.911090+0000","last_change":"2024-09-17T04:31:10.911090+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.911090+0000","last_undegraded":"2024-09-17T04:31:10.911090+0000","last_fullsized":"2024-09-17T04:31:10.911090+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909120+0000","last_change":"2024-09-17T04:31:10.909120+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909120+0000","last_undegraded":"2024-09-17T04:31:10.909120+0000","last_fullsized":"2024-09-17T04:31:10.909120+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.903949+0000","last_change":"2024-09-17T04:31:10.903949+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.903949+0000","last_undegraded":"2024-09-17T04:31:10.903949+0000","last_fullsized":"2024-09-17T04:31:10.903949+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.910613+0000","last_change":"2024-09-17T04:31:10.910613+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.910613+0000","last_undegraded":"2024-09-17T04:31:10.910613+0000","last_fullsized":"2024-09-17T04:31:10.910613+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.928277+0000","last_change":"2024-09-17T04:31:10.925886+0000","last_active":"2024-09-17T04:31:10.928277+0000","last_peered":"2024-09-17T04:31:10.928277+0000","last_clean":"2024-09-17T04:31:10.928277+0000","last_became_active":"2024-09-17T04:31:10.925455+0000","last_became_peered":"2024-09-17T04:31:10.925455+0000","last_unstale":"2024-09-17T04:31:10.928277+0000","last_undegraded":"2024-09-17T04:31:10.928277+0000","last_fullsized":"2024-09-17T04:31:10.928277+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:01.185188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.914848+0000","last_change":"2024-09-17T04:31:10.914848+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.914848+0000","last_undegraded":"2024-09-17T04:31:10.914848+0000","last_fullsized":"2024-09-17T04:31:10.914848+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.925524+0000","last_change":"2024-09-17T04:31:10.923497+0000","last_active":"2024-09-17T04:31:10.925524+0000","last_peered":"2024-09-17T04:31:10.925524+0000","last_clean":"2024-09-17T04:31:10.925524+0000","last_became_active":"2024-09-17T04:31:10.923086+0000","last_became_peered":"2024-09-17T04:31:10.923086+0000","last_unstale":"2024-09-17T04:31:10.925524+0000","last_undegraded":"2024-09-17T04:31:10.925524+0000","last_fullsized":"2024-09-17T04:31:10.925524+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:25.527822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909426+0000","last_change":"2024-09-17T04:31:10.909426+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909426+0000","last_undegraded":"2024-09-17T04:31:10.909426+0000","last_fullsized":"2024-09-17T04:31:10.909426+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.923768+0000","last_change":"2024-09-17T04:31:10.922177+0000","last_active":"2024-09-17T04:31:10.923768+0000","last_peered":"2024-09-17T04:31:10.923768+0000","last_clean":"2024-09-17T04:31:10.923768+0000","last_became_active":"2024-09-17T04:31:10.921875+0000","last_became_peered":"2024-09-17T04:31:10.921875+0000","last_unstale":"2024-09-17T04:31:10.923768+0000","last_undegraded":"2024-09-17T04:31:10.923768+0000","last_fullsized":"2024-09-17T04:31:10.923768+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:32.470330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.910133+0000","last_change":"2024-09-17T04:31:10.910133+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.910133+0000","last_undegraded":"2024-09-17T04:31:10.910133+0000","last_fullsized":"2024-09-17T04:31:10.910133+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.919580+0000","last_change":"2024-09-17T04:31:10.917285+0000","last_active":"2024-09-17T04:31:10.919580+0000","last_peered":"2024-09-17T04:31:10.919580+0000","last_clean":"2024-09-17T04:31:10.919580+0000","last_became_active":"2024-09-17T04:31:10.916739+0000","last_became_peered":"2024-09-17T04:31:10.916739+0000","last_unstale":"2024-09-17T04:31:10.919580+0000","last_undegraded":"2024-09-17T04:31:10.919580+0000","last_fullsized":"2024-09-17T04:31:10.919580+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:17.262185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909946+0000","last_change":"2024-09-17T04:31:10.909946+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909946+0000","last_undegraded":"2024-09-17T04:31:10.909946+0000","last_fullsized":"2024-09-17T04:31:10.909946+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.918268+0000","last_change":"2024-09-17T04:31:10.913568+0000","last_active":"2024-09-17T04:31:10.918268+0000","last_peered":"2024-09-17T04:31:10.918268+0000","last_clean":"2024-09-17T04:31:10.918268+0000","last_became_active":"2024-09-17T04:31:10.913273+0000","last_became_peered":"2024-09-17T04:31:10.913273+0000","last_unstale":"2024-09-17T04:31:10.918268+0000","last_undegraded":"2024-09-17T04:31:10.918268+0000","last_fullsized":"2024-09-17T04:31:10.918268+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:10.022361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.905237+0000","last_change":"2024-09-17T04:31:10.905237+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.905237+0000","last_undegraded":"2024-09-17T04:31:10.905237+0000","last_fullsized":"2024-09-17T04:31:10.905237+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.922101+0000","last_change":"2024-09-17T04:31:10.917769+0000","last_active":"2024-09-17T04:31:10.922101+0000","last_peered":"2024-09-17T04:31:10.922101+0000","last_clean":"2024-09-17T04:31:10.922101+0000","last_became_active":"2024-09-17T04:31:10.917493+0000","last_became_peered":"2024-09-17T04:31:10.917493+0000","last_unstale":"2024-09-17T04:31:10.922101+0000","last_undegraded":"2024-09-17T04:31:10.922101+0000","last_fullsized":"2024-09-17T04:31:10.922101+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:40:04.424615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.911438+0000","last_change":"2024-09-17T04:31:10.911438+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.911438+0000","last_undegraded":"2024-09-17T04:31:10.911438+0000","last_fullsized":"2024-09-17T04:31:10.911438+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.927265+0000","last_change":"2024-09-17T04:31:10.919927+0000","last_active":"2024-09-17T04:31:10.927265+0000","last_peered":"2024-09-17T04:31:10.927265+0000","last_clean":"2024-09-17T04:31:10.927265+0000","last_became_active":"2024-09-17T04:31:10.919536+0000","last_became_peered":"2024-09-17T04:31:10.919536+0000","last_unstale":"2024-09-17T04:31:10.927265+0000","last_undegraded":"2024-09-17T04:31:10.927265+0000","last_fullsized":"2024-09-17T04:31:10.927265+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:46.091206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.903852+0000","last_change":"2024-09-17T04:31:10.903852+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.903852+0000","last_undegraded":"2024-09-17T04:31:10.903852+0000","last_fullsized":"2024-09-17T04:31:10.903852+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.927852+0000","last_change":"2024-09-17T04:31:10.926647+0000","last_active":"2024-09-17T04:31:10.927852+0000","last_peered":"2024-09-17T04:31:10.927852+0000","last_clean":"2024-09-17T04:31:10.927852+0000","last_became_active":"2024-09-17T04:31:10.926287+0000","last_became_peered":"2024-09-17T04:31:10.926287+0000","last_unstale":"2024-09-17T04:31:10.927852+0000","last_undegraded":"2024-09-17T04:31:10.927852+0000","last_fullsized":"2024-09-17T04:31:10.927852+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:41.700033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909109+0000","last_change":"2024-09-17T04:31:10.909109+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909109+0000","last_undegraded":"2024-09-17T04:31:10.909109+0000","last_fullsized":"2024-09-17T04:31:10.909109+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.928213+0000","last_change":"2024-09-17T04:31:10.926032+0000","last_active":"2024-09-17T04:31:10.928213+0000","last_peered":"2024-09-17T04:31:10.928213+0000","last_clean":"2024-09-17T04:31:10.928213+0000","last_became_active":"2024-09-17T04:31:10.925736+0000","last_became_peered":"2024-09-17T04:31:10.925736+0000","last_unstale":"2024-09-17T04:31:10.928213+0000","last_undegraded":"2024-09-17T04:31:10.928213+0000","last_fullsized":"2024-09-17T04:31:10.928213+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:40.221385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.912472+0000","last_change":"2024-09-17T04:31:10.912472+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.912472+0000","last_undegraded":"2024-09-17T04:31:10.912472+0000","last_fullsized":"2024-09-17T04:31:10.912472+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.930968+0000","last_change":"2024-09-17T04:31:10.930027+0000","last_active":"2024-09-17T04:31:10.930968+0000","last_peered":"2024-09-17T04:31:10.930968+0000","last_clean":"2024-09-17T04:31:10.930968+0000","last_became_active":"2024-09-17T04:31:10.929648+0000","last_became_peered":"2024-09-17T04:31:10.929648+0000","last_unstale":"2024-09-17T04:31:10.930968+0000","last_undegraded":"2024-09-17T04:31:10.930968+0000","last_fullsized":"2024-09-17T04:31:10.930968+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:18:25.078648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.908795+0000","last_change":"2024-09-17T04:31:10.908795+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.908795+0000","last_undegraded":"2024-09-17T04:31:10.908795+0000","last_fullsized":"2024-09-17T04:31:10.908795+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.920305+0000","last_change":"2024-09-17T04:31:10.917626+0000","last_active":"2024-09-17T04:31:10.920305+0000","last_peered":"2024-09-17T04:31:10.920305+0000","last_clean":"2024-09-17T04:31:10.920305+0000","last_became_active":"2024-09-17T04:31:10.917239+0000","last_became_peered":"2024-09-17T04:31:10.917239+0000","last_unstale":"2024-09-17T04:31:10.920305+0000","last_undegraded":"2024-09-17T04:31:10.920305+0000","last_fullsized":"2024-09-17T04:31:10.920305+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:29.945578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.908678+0000","last_change":"2024-09-17T04:31:10.908678+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.908678+0000","last_undegraded":"2024-09-17T04:31:10.908678+0000","last_fullsized":"2024-09-17T04:31:10.908678+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.931270+0000","last_change":"2024-09-17T04:31:10.929556+0000","last_active":"2024-09-17T04:31:10.931270+0000","last_peered":"2024-09-17T04:31:10.931270+0000","last_clean":"2024-09-17T04:31:10.931270+0000","last_became_active":"2024-09-17T04:31:10.929171+0000","last_became_peered":"2024-09-17T04:31:10.929171+0000","last_unstale":"2024-09-17T04:31:10.931270+0000","last_undegraded":"2024-09-17T04:31:10.931270+0000","last_fullsized":"2024-09-17T04:31:10.931270+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:00.689611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909041+0000","last_change":"2024-09-17T04:31:10.909041+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909041+0000","last_undegraded":"2024-09-17T04:31:10.909041+0000","last_fullsized":"2024-09-17T04:31:10.909041+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.925160+0000","last_change":"2024-09-17T04:31:10.921170+0000","last_active":"2024-09-17T04:31:10.925160+0000","last_peered":"2024-09-17T04:31:10.925160+0000","last_clean":"2024-09-17T04:31:10.925160+0000","last_became_active":"2024-09-17T04:31:10.920873+0000","last_became_peered":"2024-09-17T04:31:10.920873+0000","last_unstale":"2024-09-17T04:31:10.925160+0000","last_undegraded":"2024-09-17T04:31:10.925160+0000","last_fullsized":"2024-09-17T04:31:10.925160+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:09:40.221275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.903934+0000","last_change":"2024-09-17T04:31:10.903934+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.903934+0000","last_undegraded":"2024-09-17T04:31:10.903934+0000","last_fullsized":"2024-09-17T04:31:10.903934+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.924402+0000","last_change":"2024-09-17T04:31:10.920901+0000","last_active":"2024-09-17T04:31:10.924402+0000","last_peered":"2024-09-17T04:31:10.924402+0000","last_clean":"2024-09-17T04:31:10.924402+0000","last_became_active":"2024-09-17T04:31:10.920586+0000","last_became_peered":"2024-09-17T04:31:10.920586+0000","last_unstale":"2024-09-17T04:31:10.924402+0000","last_undegraded":"2024-09-17T04:31:10.924402+0000","last_fullsized":"2024-09-17T04:31:10.924402+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:59.093831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.915262+0000","last_change":"2024-09-17T04:31:10.915262+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.915262+0000","last_undegraded":"2024-09-17T04:31:10.915262+0000","last_fullsized":"2024-09-17T04:31:10.915262+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.929486+0000","last_change":"2024-09-17T04:31:10.924812+0000","last_active":"2024-09-17T04:31:10.929486+0000","last_peered":"2024-09-17T04:31:10.929486+0000","last_clean":"2024-09-17T04:31:10.929486+0000","last_became_active":"2024-09-17T04:31:10.924423+0000","last_became_peered":"2024-09-17T04:31:10.924423+0000","last_unstale":"2024-09-17T04:31:10.929486+0000","last_undegraded":"2024-09-17T04:31:10.929486+0000","last_fullsized":"2024-09-17T04:31:10.929486+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:07.672259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.917600+0000","last_change":"2024-09-17T04:31:10.917600+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.917600+0000","last_undegraded":"2024-09-17T04:31:10.917600+0000","last_fullsized":"2024-09-17T04:31:10.917600+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.926479+0000","last_change":"2024-09-17T04:31:10.924255+0000","last_active":"2024-09-17T04:31:10.926479+0000","last_peered":"2024-09-17T04:31:10.926479+0000","last_clean":"2024-09-17T04:31:10.926479+0000","last_became_active":"2024-09-17T04:31:10.923887+0000","last_became_peered":"2024-09-17T04:31:10.923887+0000","last_unstale":"2024-09-17T04:31:10.926479+0000","last_undegraded":"2024-09-17T04:31:10.926479+0000","last_fullsized":"2024-09-17T04:31:10.926479+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:42.584932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.911484+0000","last_change":"2024-09-17T04:31:10.911484+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.911484+0000","last_undegraded":"2024-09-17T04:31:10.911484+0000","last_fullsized":"2024-09-17T04:31:10.911484+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.930817+0000","last_change":"2024-09-17T04:31:10.930022+0000","last_active":"2024-09-17T04:31:10.930817+0000","last_peered":"2024-09-17T04:31:10.930817+0000","last_clean":"2024-09-17T04:31:10.930817+0000","last_became_active":"2024-09-17T04:31:10.929681+0000","last_became_peered":"2024-09-17T04:31:10.929681+0000","last_unstale":"2024-09-17T04:31:10.930817+0000","last_undegraded":"2024-09-17T04:31:10.930817+0000","last_fullsized":"2024-09-17T04:31:10.930817+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:16.582319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.911356+0000","last_change":"2024-09-17T04:31:10.911356+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.911356+0000","last_undegraded":"2024-09-17T04:31:10.911356+0000","last_fullsized":"2024-09-17T04:31:10.911356+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.918083+0000","last_change":"2024-09-17T04:31:10.910443+0000","last_active":"2024-09-17T04:31:10.918083+0000","last_peered":"2024-09-17T04:31:10.918083+0000","last_clean":"2024-09-17T04:31:10.918083+0000","last_became_active":"2024-09-17T04:31:10.910084+0000","last_became_peered":"2024-09-17T04:31:10.910084+0000","last_unstale":"2024-09-17T04:31:10.918083+0000","last_undegraded":"2024-09-17T04:31:10.918083+0000","last_fullsized":"2024-09-17T04:31:10.918083+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:02.897825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.904544+0000","last_change":"2024-09-17T04:31:10.904544+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.904544+0000","last_undegraded":"2024-09-17T04:31:10.904544+0000","last_fullsized":"2024-09-17T04:31:10.904544+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.921482+0000","last_change":"2024-09-17T04:31:10.918599+0000","last_active":"2024-09-17T04:31:10.921482+0000","last_peered":"2024-09-17T04:31:10.921482+0000","last_clean":"2024-09-17T04:31:10.921482+0000","last_became_active":"2024-09-17T04:31:10.918208+0000","last_became_peered":"2024-09-17T04:31:10.918208+0000","last_unstale":"2024-09-17T04:31:10.921482+0000","last_undegraded":"2024-09-17T04:31:10.921482+0000","last_fullsized":"2024-09-17T04:31:10.921482+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:29.662127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909400+0000","last_change":"2024-09-17T04:31:10.909400+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909400+0000","last_undegraded":"2024-09-17T04:31:10.909400+0000","last_fullsized":"2024-09-17T04:31:10.909400+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.929291+0000","last_change":"2024-09-17T04:31:10.927592+0000","last_active":"2024-09-17T04:31:10.929291+0000","last_peered":"2024-09-17T04:31:10.929291+0000","last_clean":"2024-09-17T04:31:10.929291+0000","last_became_active":"2024-09-17T04:31:10.926639+0000","last_became_peered":"2024-09-17T04:31:10.926639+0000","last_unstale":"2024-09-17T04:31:10.929291+0000","last_undegraded":"2024-09-17T04:31:10.929291+0000","last_fullsized":"2024-09-17T04:31:10.929291+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:50.191404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.904320+0000","last_change":"2024-09-17T04:31:10.904320+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.904320+0000","last_undegraded":"2024-09-17T04:31:10.904320+0000","last_fullsized":"2024-09-17T04:31:10.904320+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.928276+0000","last_change":"2024-09-17T04:31:10.926073+0000","last_active":"2024-09-17T04:31:10.928276+0000","last_peered":"2024-09-17T04:31:10.928276+0000","last_clean":"2024-09-17T04:31:10.928276+0000","last_became_active":"2024-09-17T04:31:10.925750+0000","last_became_peered":"2024-09-17T04:31:10.925750+0000","last_unstale":"2024-09-17T04:31:10.928276+0000","last_undegraded":"2024-09-17T04:31:10.928276+0000","last_fullsized":"2024-09-17T04:31:10.928276+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:10:46.052152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.903377+0000","last_change":"2024-09-17T04:31:10.903377+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.903377+0000","last_undegraded":"2024-09-17T04:31:10.903377+0000","last_fullsized":"2024-09-17T04:31:10.903377+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.923038+0000","last_change":"2024-09-17T04:31:10.917223+0000","last_active":"2024-09-17T04:31:10.923038+0000","last_peered":"2024-09-17T04:31:10.923038+0000","last_clean":"2024-09-17T04:31:10.923038+0000","last_became_active":"2024-09-17T04:31:10.916765+0000","last_became_peered":"2024-09-17T04:31:10.916765+0000","last_unstale":"2024-09-17T04:31:10.923038+0000","last_undegraded":"2024-09-17T04:31:10.923038+0000","last_fullsized":"2024-09-17T04:31:10.923038+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:37.040110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.910300+0000","last_change":"2024-09-17T04:31:10.910300+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.910300+0000","last_undegraded":"2024-09-17T04:31:10.910300+0000","last_fullsized":"2024-09-17T04:31:10.910300+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.927355+0000","last_change":"2024-09-17T04:31:10.926100+0000","last_active":"2024-09-17T04:31:10.927355+0000","last_peered":"2024-09-17T04:31:10.927355+0000","last_clean":"2024-09-17T04:31:10.927355+0000","last_became_active":"2024-09-17T04:31:10.925806+0000","last_became_peered":"2024-09-17T04:31:10.925806+0000","last_unstale":"2024-09-17T04:31:10.927355+0000","last_undegraded":"2024-09-17T04:31:10.927355+0000","last_fullsized":"2024-09-17T04:31:10.927355+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:05.786635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.904697+0000","last_change":"2024-09-17T04:31:10.904697+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.904697+0000","last_undegraded":"2024-09-17T04:31:10.904697+0000","last_fullsized":"2024-09-17T04:31:10.904697+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.924475+0000","last_change":"2024-09-17T04:31:10.919421+0000","last_active":"2024-09-17T04:31:10.924475+0000","last_peered":"2024-09-17T04:31:10.924475+0000","last_clean":"2024-09-17T04:31:10.924475+0000","last_became_active":"2024-09-17T04:31:10.919126+0000","last_became_peered":"2024-09-17T04:31:10.919126+0000","last_unstale":"2024-09-17T04:31:10.924475+0000","last_undegraded":"2024-09-17T04:31:10.924475+0000","last_fullsized":"2024-09-17T04:31:10.924475+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:40.608118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909084+0000","last_change":"2024-09-17T04:31:10.909084+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909084+0000","last_undegraded":"2024-09-17T04:31:10.909084+0000","last_fullsized":"2024-09-17T04:31:10.909084+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.931338+0000","last_change":"2024-09-17T04:31:10.930374+0000","last_active":"2024-09-17T04:31:10.931338+0000","last_peered":"2024-09-17T04:31:10.931338+0000","last_clean":"2024-09-17T04:31:10.931338+0000","last_became_active":"2024-09-17T04:31:10.930043+0000","last_became_peered":"2024-09-17T04:31:10.930043+0000","last_unstale":"2024-09-17T04:31:10.931338+0000","last_undegraded":"2024-09-17T04:31:10.931338+0000","last_fullsized":"2024-09-17T04:31:10.931338+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:02.897825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.904139+0000","last_change":"2024-09-17T04:31:10.904139+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.904139+0000","last_undegraded":"2024-09-17T04:31:10.904139+0000","last_fullsized":"2024-09-17T04:31:10.904139+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.930617+0000","last_change":"2024-09-17T04:31:10.929027+0000","last_active":"2024-09-17T04:31:10.930617+0000","last_peered":"2024-09-17T04:31:10.930617+0000","last_clean":"2024-09-17T04:31:10.930617+0000","last_became_active":"2024-09-17T04:31:10.928767+0000","last_became_peered":"2024-09-17T04:31:10.928767+0000","last_unstale":"2024-09-17T04:31:10.930617+0000","last_undegraded":"2024-09-17T04:31:10.930617+0000","last_fullsized":"2024-09-17T04:31:10.930617+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:06.263954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.905447+0000","last_change":"2024-09-17T04:31:10.905447+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.905447+0000","last_undegraded":"2024-09-17T04:31:10.905447+0000","last_fullsized":"2024-09-17T04:31:10.905447+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.922602+0000","last_change":"2024-09-17T04:31:10.920453+0000","last_active":"2024-09-17T04:31:10.922602+0000","last_peered":"2024-09-17T04:31:10.922602+0000","last_clean":"2024-09-17T04:31:10.922602+0000","last_became_active":"2024-09-17T04:31:10.919363+0000","last_became_peered":"2024-09-17T04:31:10.919363+0000","last_unstale":"2024-09-17T04:31:10.922602+0000","last_undegraded":"2024-09-17T04:31:10.922602+0000","last_fullsized":"2024-09-17T04:31:10.922602+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:04:35.664082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.909560+0000","last_change":"2024-09-17T04:31:10.909560+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.909560+0000","last_undegraded":"2024-09-17T04:31:10.909560+0000","last_fullsized":"2024-09-17T04:31:10.909560+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.924812+0000","last_change":"2024-09-17T04:31:10.923121+0000","last_active":"2024-09-17T04:31:10.924812+0000","last_peered":"2024-09-17T04:31:10.924812+0000","last_clean":"2024-09-17T04:31:10.924812+0000","last_became_active":"2024-09-17T04:31:10.922374+0000","last_became_peered":"2024-09-17T04:31:10.922374+0000","last_unstale":"2024-09-17T04:31:10.924812+0000","last_undegraded":"2024-09-17T04:31:10.924812+0000","last_fullsized":"2024-09-17T04:31:10.924812+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:23:56.512701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.911942+0000","last_change":"2024-09-17T04:31:10.911942+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.911942+0000","last_undegraded":"2024-09-17T04:31:10.911942+0000","last_fullsized":"2024-09-17T04:31:10.911942+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.927043+0000","last_change":"2024-09-17T04:31:10.925280+0000","last_active":"2024-09-17T04:31:10.927043+0000","last_peered":"2024-09-17T04:31:10.927043+0000","last_clean":"2024-09-17T04:31:10.927043+0000","last_became_active":"2024-09-17T04:31:10.924762+0000","last_became_peered":"2024-09-17T04:31:10.924762+0000","last_unstale":"2024-09-17T04:31:10.927043+0000","last_undegraded":"2024-09-17T04:31:10.927043+0000","last_fullsized":"2024-09-17T04:31:10.927043+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:28.910545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.910074+0000","last_change":"2024-09-17T04:31:10.910074+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.910074+0000","last_undegraded":"2024-09-17T04:31:10.910074+0000","last_fullsized":"2024-09-17T04:31:10.910074+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.928272+0000","last_change":"2024-09-17T04:31:10.926916+0000","last_active":"2024-09-17T04:31:10.928272+0000","last_peered":"2024-09-17T04:31:10.928272+0000","last_clean":"2024-09-17T04:31:10.928272+0000","last_became_active":"2024-09-17T04:31:10.926611+0000","last_became_peered":"2024-09-17T04:31:10.926611+0000","last_unstale":"2024-09-17T04:31:10.928272+0000","last_undegraded":"2024-09-17T04:31:10.928272+0000","last_fullsized":"2024-09-17T04:31:10.928272+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:39.197809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.903334+0000","last_change":"2024-09-17T04:31:10.903334+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.903334+0000","last_undegraded":"2024-09-17T04:31:10.903334+0000","last_fullsized":"2024-09-17T04:31:10.903334+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.921772+0000","last_change":"2024-09-17T04:31:10.918838+0000","last_active":"2024-09-17T04:31:10.921772+0000","last_peered":"2024-09-17T04:31:10.921772+0000","last_clean":"2024-09-17T04:31:10.921772+0000","last_became_active":"2024-09-17T04:31:10.918535+0000","last_became_peered":"2024-09-17T04:31:10.918535+0000","last_unstale":"2024-09-17T04:31:10.921772+0000","last_undegraded":"2024-09-17T04:31:10.921772+0000","last_fullsized":"2024-09-17T04:31:10.921772+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:32.795755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-17T04:31:10.910191+0000","last_change":"2024-09-17T04:31:10.910191+0000","last_active":"2024-09-17T04:31:10.890602+0000","last_peered":"2024-09-17T04:31:10.890602+0000","last_clean":"2024-09-17T04:31:10.890602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:31:10.910191+0000","last_undegraded":"2024-09-17T04:31:10.910191+0000","last_fullsized":"2024-09-17T04:31:10.910191+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.928967+0000","last_change":"2024-09-17T04:31:10.927734+0000","last_active":"2024-09-17T04:31:10.928967+0000","last_peered":"2024-09-17T04:31:10.928967+0000","last_clean":"2024-09-17T04:31:10.928967+0000","last_became_active":"2024-09-17T04:31:10.926995+0000","last_became_peered":"2024-09-17T04:31:10.926995+0000","last_unstale":"2024-09-17T04:31:10.928967+0000","last_undegraded":"2024-09-17T04:31:10.928967+0000","last_fullsized":"2024-09-17T04:31:10.928967+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:48:42.671646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"76'79","reported_seq":313,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T04:31:10.917506+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:31:10.917506+0000","last_peered":"2024-09-17T04:31:10.917506+0000","last_clean":"2024-09-17T04:31:10.917506+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:31:10.917506+0000","last_undegraded":"2024-09-17T04:31:10.917506+0000","last_fullsized":"2024-09-17T04:31:10.917506+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476827,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33188,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338652,"statfs":{"total":96636764160,"available":96602779648,"internally_reserved":0,"allocated":2043904,"data_stored":1224926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476826,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1449984,"data_stored":634558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476827,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1449984,"data_stored":634558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476827,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1437696,"data_stored":628255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542234,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32612,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339228,"statfs":{"total":96636764160,"available":96603369472,"internally_reserved":0,"allocated":1449984,"data_stored":634558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509530,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1449984,"data_stored":634558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509529,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1449984,"data_stored":634558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509529,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33188,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338652,"statfs":{"total":96636764160,"available":96602779648,"internally_reserved":0,"allocated":2043904,"data_stored":1224926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:31:17.309 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3f is not active+clean 2024-09-17T04:31:17.309 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.914190+0000', 'last_change': '2024-09-17T04:31:10.914190+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.914190+0000', 'last_undegraded': '2024-09-17T04:31:10.914190+0000', 'last_fullsized': '2024-09-17T04:31:10.914190+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T04:31:17.309 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3e is not active+clean 2024-09-17T04:31:17.309 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.904387+0000', 'last_change': '2024-09-17T04:31:10.904387+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.904387+0000', 'last_undegraded': '2024-09-17T04:31:10.904387+0000', 'last_fullsized': '2024-09-17T04:31:10.904387+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T04:31:17.309 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3d is not active+clean 2024-09-17T04:31:17.310 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.917302+0000', 'last_change': '2024-09-17T04:31:10.917302+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.917302+0000', 'last_undegraded': '2024-09-17T04:31:10.917302+0000', 'last_fullsized': '2024-09-17T04:31:10.917302+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T04:31:17.310 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3c is not active+clean 2024-09-17T04:31:17.310 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.912667+0000', 'last_change': '2024-09-17T04:31:10.912667+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.912667+0000', 'last_undegraded': '2024-09-17T04:31:10.912667+0000', 'last_fullsized': '2024-09-17T04:31:10.912667+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T04:31:17.310 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3b is not active+clean 2024-09-17T04:31:17.310 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.910610+0000', 'last_change': '2024-09-17T04:31:10.910610+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.910610+0000', 'last_undegraded': '2024-09-17T04:31:10.910610+0000', 'last_fullsized': '2024-09-17T04:31:10.910610+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:31:17.310 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3a is not active+clean 2024-09-17T04:31:17.310 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.903895+0000', 'last_change': '2024-09-17T04:31:10.903895+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.903895+0000', 'last_undegraded': '2024-09-17T04:31:10.903895+0000', 'last_fullsized': '2024-09-17T04:31:10.903895+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T04:31:17.311 INFO:tasks.ceph.ceph_manager.ceph:PG 19.39 is not active+clean 2024-09-17T04:31:17.311 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.904370+0000', 'last_change': '2024-09-17T04:31:10.904370+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.904370+0000', 'last_undegraded': '2024-09-17T04:31:10.904370+0000', 'last_fullsized': '2024-09-17T04:31:10.904370+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T04:31:17.311 INFO:tasks.ceph.ceph_manager.ceph:PG 19.38 is not active+clean 2024-09-17T04:31:17.311 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.912073+0000', 'last_change': '2024-09-17T04:31:10.912073+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.912073+0000', 'last_undegraded': '2024-09-17T04:31:10.912073+0000', 'last_fullsized': '2024-09-17T04:31:10.912073+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T04:31:17.311 INFO:tasks.ceph.ceph_manager.ceph:PG 19.37 is not active+clean 2024-09-17T04:31:17.311 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.915843+0000', 'last_change': '2024-09-17T04:31:10.915843+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.915843+0000', 'last_undegraded': '2024-09-17T04:31:10.915843+0000', 'last_fullsized': '2024-09-17T04:31:10.915843+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T04:31:17.311 INFO:tasks.ceph.ceph_manager.ceph:PG 19.36 is not active+clean 2024-09-17T04:31:17.311 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.903841+0000', 'last_change': '2024-09-17T04:31:10.903841+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.903841+0000', 'last_undegraded': '2024-09-17T04:31:10.903841+0000', 'last_fullsized': '2024-09-17T04:31:10.903841+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T04:31:17.311 INFO:tasks.ceph.ceph_manager.ceph:PG 19.35 is not active+clean 2024-09-17T04:31:17.312 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.912182+0000', 'last_change': '2024-09-17T04:31:10.912182+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.912182+0000', 'last_undegraded': '2024-09-17T04:31:10.912182+0000', 'last_fullsized': '2024-09-17T04:31:10.912182+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T04:31:17.312 INFO:tasks.ceph.ceph_manager.ceph:PG 19.34 is not active+clean 2024-09-17T04:31:17.312 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.913434+0000', 'last_change': '2024-09-17T04:31:10.913434+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.913434+0000', 'last_undegraded': '2024-09-17T04:31:10.913434+0000', 'last_fullsized': '2024-09-17T04:31:10.913434+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T04:31:17.312 INFO:tasks.ceph.ceph_manager.ceph:PG 19.33 is not active+clean 2024-09-17T04:31:17.312 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.915532+0000', 'last_change': '2024-09-17T04:31:10.915532+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.915532+0000', 'last_undegraded': '2024-09-17T04:31:10.915532+0000', 'last_fullsized': '2024-09-17T04:31:10.915532+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T04:31:17.312 INFO:tasks.ceph.ceph_manager.ceph:PG 19.32 is not active+clean 2024-09-17T04:31:17.312 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.903634+0000', 'last_change': '2024-09-17T04:31:10.903634+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.903634+0000', 'last_undegraded': '2024-09-17T04:31:10.903634+0000', 'last_fullsized': '2024-09-17T04:31:10.903634+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T04:31:17.313 INFO:tasks.ceph.ceph_manager.ceph:PG 19.31 is not active+clean 2024-09-17T04:31:17.313 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.913164+0000', 'last_change': '2024-09-17T04:31:10.913164+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.913164+0000', 'last_undegraded': '2024-09-17T04:31:10.913164+0000', 'last_fullsized': '2024-09-17T04:31:10.913164+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T04:31:17.313 INFO:tasks.ceph.ceph_manager.ceph:PG 19.30 is not active+clean 2024-09-17T04:31:17.313 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.912271+0000', 'last_change': '2024-09-17T04:31:10.912271+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.912271+0000', 'last_undegraded': '2024-09-17T04:31:10.912271+0000', 'last_fullsized': '2024-09-17T04:31:10.912271+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T04:31:17.313 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2f is not active+clean 2024-09-17T04:31:17.313 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.904605+0000', 'last_change': '2024-09-17T04:31:10.904605+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.904605+0000', 'last_undegraded': '2024-09-17T04:31:10.904605+0000', 'last_fullsized': '2024-09-17T04:31:10.904605+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T04:31:17.313 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2e is not active+clean 2024-09-17T04:31:17.314 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.912776+0000', 'last_change': '2024-09-17T04:31:10.912776+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.912776+0000', 'last_undegraded': '2024-09-17T04:31:10.912776+0000', 'last_fullsized': '2024-09-17T04:31:10.912776+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T04:31:17.314 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2d is not active+clean 2024-09-17T04:31:17.314 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.911756+0000', 'last_change': '2024-09-17T04:31:10.911756+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.911756+0000', 'last_undegraded': '2024-09-17T04:31:10.911756+0000', 'last_fullsized': '2024-09-17T04:31:10.911756+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T04:31:17.314 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2c is not active+clean 2024-09-17T04:31:17.314 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.911841+0000', 'last_change': '2024-09-17T04:31:10.911841+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.911841+0000', 'last_undegraded': '2024-09-17T04:31:10.911841+0000', 'last_fullsized': '2024-09-17T04:31:10.911841+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T04:31:17.314 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2b is not active+clean 2024-09-17T04:31:17.314 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.916213+0000', 'last_change': '2024-09-17T04:31:10.916213+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.916213+0000', 'last_undegraded': '2024-09-17T04:31:10.916213+0000', 'last_fullsized': '2024-09-17T04:31:10.916213+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:31:17.315 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2a is not active+clean 2024-09-17T04:31:17.315 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.910761+0000', 'last_change': '2024-09-17T04:31:10.910761+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.910761+0000', 'last_undegraded': '2024-09-17T04:31:10.910761+0000', 'last_fullsized': '2024-09-17T04:31:10.910761+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T04:31:17.315 INFO:tasks.ceph.ceph_manager.ceph:PG 19.29 is not active+clean 2024-09-17T04:31:17.315 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.914620+0000', 'last_change': '2024-09-17T04:31:10.914620+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.914620+0000', 'last_undegraded': '2024-09-17T04:31:10.914620+0000', 'last_fullsized': '2024-09-17T04:31:10.914620+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T04:31:17.315 INFO:tasks.ceph.ceph_manager.ceph:PG 19.28 is not active+clean 2024-09-17T04:31:17.315 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.907757+0000', 'last_change': '2024-09-17T04:31:10.907757+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.907757+0000', 'last_undegraded': '2024-09-17T04:31:10.907757+0000', 'last_fullsized': '2024-09-17T04:31:10.907757+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T04:31:17.315 INFO:tasks.ceph.ceph_manager.ceph:PG 19.27 is not active+clean 2024-09-17T04:31:17.315 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.904983+0000', 'last_change': '2024-09-17T04:31:10.904983+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.904983+0000', 'last_undegraded': '2024-09-17T04:31:10.904983+0000', 'last_fullsized': '2024-09-17T04:31:10.904983+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T04:31:17.315 INFO:tasks.ceph.ceph_manager.ceph:PG 19.26 is not active+clean 2024-09-17T04:31:17.315 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.912322+0000', 'last_change': '2024-09-17T04:31:10.912322+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.912322+0000', 'last_undegraded': '2024-09-17T04:31:10.912322+0000', 'last_fullsized': '2024-09-17T04:31:10.912322+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T04:31:17.315 INFO:tasks.ceph.ceph_manager.ceph:PG 19.25 is not active+clean 2024-09-17T04:31:17.316 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.909621+0000', 'last_change': '2024-09-17T04:31:10.909621+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.909621+0000', 'last_undegraded': '2024-09-17T04:31:10.909621+0000', 'last_fullsized': '2024-09-17T04:31:10.909621+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T04:31:17.316 INFO:tasks.ceph.ceph_manager.ceph:PG 19.24 is not active+clean 2024-09-17T04:31:17.316 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.909027+0000', 'last_change': '2024-09-17T04:31:10.909027+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.909027+0000', 'last_undegraded': '2024-09-17T04:31:10.909027+0000', 'last_fullsized': '2024-09-17T04:31:10.909027+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T04:31:17.316 INFO:tasks.ceph.ceph_manager.ceph:PG 19.23 is not active+clean 2024-09-17T04:31:17.316 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.910178+0000', 'last_change': '2024-09-17T04:31:10.910178+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.910178+0000', 'last_undegraded': '2024-09-17T04:31:10.910178+0000', 'last_fullsized': '2024-09-17T04:31:10.910178+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T04:31:17.316 INFO:tasks.ceph.ceph_manager.ceph:PG 19.22 is not active+clean 2024-09-17T04:31:17.316 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.911090+0000', 'last_change': '2024-09-17T04:31:10.911090+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.911090+0000', 'last_undegraded': '2024-09-17T04:31:10.911090+0000', 'last_fullsized': '2024-09-17T04:31:10.911090+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T04:31:17.316 INFO:tasks.ceph.ceph_manager.ceph:PG 19.21 is not active+clean 2024-09-17T04:31:17.316 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.909120+0000', 'last_change': '2024-09-17T04:31:10.909120+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.909120+0000', 'last_undegraded': '2024-09-17T04:31:10.909120+0000', 'last_fullsized': '2024-09-17T04:31:10.909120+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:31:17.316 INFO:tasks.ceph.ceph_manager.ceph:PG 19.20 is not active+clean 2024-09-17T04:31:17.317 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.903949+0000', 'last_change': '2024-09-17T04:31:10.903949+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.903949+0000', 'last_undegraded': '2024-09-17T04:31:10.903949+0000', 'last_fullsized': '2024-09-17T04:31:10.903949+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T04:31:17.317 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1e is not active+clean 2024-09-17T04:31:17.317 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.910613+0000', 'last_change': '2024-09-17T04:31:10.910613+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.910613+0000', 'last_undegraded': '2024-09-17T04:31:10.910613+0000', 'last_fullsized': '2024-09-17T04:31:10.910613+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T04:31:17.317 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1f is not active+clean 2024-09-17T04:31:17.317 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.914848+0000', 'last_change': '2024-09-17T04:31:10.914848+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.914848+0000', 'last_undegraded': '2024-09-17T04:31:10.914848+0000', 'last_fullsized': '2024-09-17T04:31:10.914848+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T04:31:17.317 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1c is not active+clean 2024-09-17T04:31:17.317 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.909426+0000', 'last_change': '2024-09-17T04:31:10.909426+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.909426+0000', 'last_undegraded': '2024-09-17T04:31:10.909426+0000', 'last_fullsized': '2024-09-17T04:31:10.909426+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:31:17.317 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1d is not active+clean 2024-09-17T04:31:17.317 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.910133+0000', 'last_change': '2024-09-17T04:31:10.910133+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.910133+0000', 'last_undegraded': '2024-09-17T04:31:10.910133+0000', 'last_fullsized': '2024-09-17T04:31:10.910133+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:31:17.317 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1a is not active+clean 2024-09-17T04:31:17.318 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.909946+0000', 'last_change': '2024-09-17T04:31:10.909946+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.909946+0000', 'last_undegraded': '2024-09-17T04:31:10.909946+0000', 'last_fullsized': '2024-09-17T04:31:10.909946+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T04:31:17.318 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1b is not active+clean 2024-09-17T04:31:17.318 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.905237+0000', 'last_change': '2024-09-17T04:31:10.905237+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.905237+0000', 'last_undegraded': '2024-09-17T04:31:10.905237+0000', 'last_fullsized': '2024-09-17T04:31:10.905237+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T04:31:17.318 INFO:tasks.ceph.ceph_manager.ceph:PG 19.18 is not active+clean 2024-09-17T04:31:17.318 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.911438+0000', 'last_change': '2024-09-17T04:31:10.911438+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.911438+0000', 'last_undegraded': '2024-09-17T04:31:10.911438+0000', 'last_fullsized': '2024-09-17T04:31:10.911438+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T04:31:17.318 INFO:tasks.ceph.ceph_manager.ceph:PG 19.19 is not active+clean 2024-09-17T04:31:17.318 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.903852+0000', 'last_change': '2024-09-17T04:31:10.903852+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.903852+0000', 'last_undegraded': '2024-09-17T04:31:10.903852+0000', 'last_fullsized': '2024-09-17T04:31:10.903852+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T04:31:17.318 INFO:tasks.ceph.ceph_manager.ceph:PG 19.16 is not active+clean 2024-09-17T04:31:17.318 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.909109+0000', 'last_change': '2024-09-17T04:31:10.909109+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.909109+0000', 'last_undegraded': '2024-09-17T04:31:10.909109+0000', 'last_fullsized': '2024-09-17T04:31:10.909109+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T04:31:17.318 INFO:tasks.ceph.ceph_manager.ceph:PG 19.17 is not active+clean 2024-09-17T04:31:17.318 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.912472+0000', 'last_change': '2024-09-17T04:31:10.912472+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.912472+0000', 'last_undegraded': '2024-09-17T04:31:10.912472+0000', 'last_fullsized': '2024-09-17T04:31:10.912472+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T04:31:17.319 INFO:tasks.ceph.ceph_manager.ceph:PG 19.14 is not active+clean 2024-09-17T04:31:17.319 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.908795+0000', 'last_change': '2024-09-17T04:31:10.908795+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.908795+0000', 'last_undegraded': '2024-09-17T04:31:10.908795+0000', 'last_fullsized': '2024-09-17T04:31:10.908795+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:31:17.319 INFO:tasks.ceph.ceph_manager.ceph:PG 19.15 is not active+clean 2024-09-17T04:31:17.319 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.908678+0000', 'last_change': '2024-09-17T04:31:10.908678+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.908678+0000', 'last_undegraded': '2024-09-17T04:31:10.908678+0000', 'last_fullsized': '2024-09-17T04:31:10.908678+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:31:17.319 INFO:tasks.ceph.ceph_manager.ceph:PG 19.13 is not active+clean 2024-09-17T04:31:17.319 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.909041+0000', 'last_change': '2024-09-17T04:31:10.909041+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.909041+0000', 'last_undegraded': '2024-09-17T04:31:10.909041+0000', 'last_fullsized': '2024-09-17T04:31:10.909041+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T04:31:17.319 INFO:tasks.ceph.ceph_manager.ceph:PG 19.10 is not active+clean 2024-09-17T04:31:17.319 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.903934+0000', 'last_change': '2024-09-17T04:31:10.903934+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.903934+0000', 'last_undegraded': '2024-09-17T04:31:10.903934+0000', 'last_fullsized': '2024-09-17T04:31:10.903934+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T04:31:17.319 INFO:tasks.ceph.ceph_manager.ceph:PG 19.11 is not active+clean 2024-09-17T04:31:17.319 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.915262+0000', 'last_change': '2024-09-17T04:31:10.915262+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.915262+0000', 'last_undegraded': '2024-09-17T04:31:10.915262+0000', 'last_fullsized': '2024-09-17T04:31:10.915262+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T04:31:17.320 INFO:tasks.ceph.ceph_manager.ceph:PG 19.e is not active+clean 2024-09-17T04:31:17.320 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.917600+0000', 'last_change': '2024-09-17T04:31:10.917600+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.917600+0000', 'last_undegraded': '2024-09-17T04:31:10.917600+0000', 'last_fullsized': '2024-09-17T04:31:10.917600+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T04:31:17.320 INFO:tasks.ceph.ceph_manager.ceph:PG 19.f is not active+clean 2024-09-17T04:31:17.320 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.911484+0000', 'last_change': '2024-09-17T04:31:10.911484+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.911484+0000', 'last_undegraded': '2024-09-17T04:31:10.911484+0000', 'last_fullsized': '2024-09-17T04:31:10.911484+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T04:31:17.320 INFO:tasks.ceph.ceph_manager.ceph:PG 19.c is not active+clean 2024-09-17T04:31:17.320 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.911356+0000', 'last_change': '2024-09-17T04:31:10.911356+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.911356+0000', 'last_undegraded': '2024-09-17T04:31:10.911356+0000', 'last_fullsized': '2024-09-17T04:31:10.911356+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T04:31:17.320 INFO:tasks.ceph.ceph_manager.ceph:PG 19.d is not active+clean 2024-09-17T04:31:17.320 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.904544+0000', 'last_change': '2024-09-17T04:31:10.904544+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.904544+0000', 'last_undegraded': '2024-09-17T04:31:10.904544+0000', 'last_fullsized': '2024-09-17T04:31:10.904544+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T04:31:17.321 INFO:tasks.ceph.ceph_manager.ceph:PG 19.a is not active+clean 2024-09-17T04:31:17.321 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.909400+0000', 'last_change': '2024-09-17T04:31:10.909400+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.909400+0000', 'last_undegraded': '2024-09-17T04:31:10.909400+0000', 'last_fullsized': '2024-09-17T04:31:10.909400+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T04:31:17.321 INFO:tasks.ceph.ceph_manager.ceph:PG 19.b is not active+clean 2024-09-17T04:31:17.321 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.904320+0000', 'last_change': '2024-09-17T04:31:10.904320+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.904320+0000', 'last_undegraded': '2024-09-17T04:31:10.904320+0000', 'last_fullsized': '2024-09-17T04:31:10.904320+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T04:31:17.321 INFO:tasks.ceph.ceph_manager.ceph:PG 19.8 is not active+clean 2024-09-17T04:31:17.321 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.903377+0000', 'last_change': '2024-09-17T04:31:10.903377+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.903377+0000', 'last_undegraded': '2024-09-17T04:31:10.903377+0000', 'last_fullsized': '2024-09-17T04:31:10.903377+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T04:31:17.321 INFO:tasks.ceph.ceph_manager.ceph:PG 19.9 is not active+clean 2024-09-17T04:31:17.321 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.910300+0000', 'last_change': '2024-09-17T04:31:10.910300+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.910300+0000', 'last_undegraded': '2024-09-17T04:31:10.910300+0000', 'last_fullsized': '2024-09-17T04:31:10.910300+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:31:17.322 INFO:tasks.ceph.ceph_manager.ceph:PG 19.6 is not active+clean 2024-09-17T04:31:17.322 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.904697+0000', 'last_change': '2024-09-17T04:31:10.904697+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.904697+0000', 'last_undegraded': '2024-09-17T04:31:10.904697+0000', 'last_fullsized': '2024-09-17T04:31:10.904697+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T04:31:17.322 INFO:tasks.ceph.ceph_manager.ceph:PG 19.7 is not active+clean 2024-09-17T04:31:17.322 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.909084+0000', 'last_change': '2024-09-17T04:31:10.909084+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.909084+0000', 'last_undegraded': '2024-09-17T04:31:10.909084+0000', 'last_fullsized': '2024-09-17T04:31:10.909084+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T04:31:17.322 INFO:tasks.ceph.ceph_manager.ceph:PG 19.4 is not active+clean 2024-09-17T04:31:17.322 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.904139+0000', 'last_change': '2024-09-17T04:31:10.904139+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.904139+0000', 'last_undegraded': '2024-09-17T04:31:10.904139+0000', 'last_fullsized': '2024-09-17T04:31:10.904139+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T04:31:17.322 INFO:tasks.ceph.ceph_manager.ceph:PG 19.5 is not active+clean 2024-09-17T04:31:17.322 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.905447+0000', 'last_change': '2024-09-17T04:31:10.905447+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.905447+0000', 'last_undegraded': '2024-09-17T04:31:10.905447+0000', 'last_fullsized': '2024-09-17T04:31:10.905447+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T04:31:17.323 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2 is not active+clean 2024-09-17T04:31:17.323 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.909560+0000', 'last_change': '2024-09-17T04:31:10.909560+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.909560+0000', 'last_undegraded': '2024-09-17T04:31:10.909560+0000', 'last_fullsized': '2024-09-17T04:31:10.909560+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:31:17.323 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3 is not active+clean 2024-09-17T04:31:17.323 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.911942+0000', 'last_change': '2024-09-17T04:31:10.911942+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.911942+0000', 'last_undegraded': '2024-09-17T04:31:10.911942+0000', 'last_fullsized': '2024-09-17T04:31:10.911942+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T04:31:17.323 INFO:tasks.ceph.ceph_manager.ceph:PG 19.0 is not active+clean 2024-09-17T04:31:17.323 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.910074+0000', 'last_change': '2024-09-17T04:31:10.910074+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.910074+0000', 'last_undegraded': '2024-09-17T04:31:10.910074+0000', 'last_fullsized': '2024-09-17T04:31:10.910074+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T04:31:17.323 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1 is not active+clean 2024-09-17T04:31:17.324 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.903334+0000', 'last_change': '2024-09-17T04:31:10.903334+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.903334+0000', 'last_undegraded': '2024-09-17T04:31:10.903334+0000', 'last_fullsized': '2024-09-17T04:31:10.903334+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T04:31:17.324 INFO:tasks.ceph.ceph_manager.ceph:PG 19.12 is not active+clean 2024-09-17T04:31:17.324 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-17T04:31:10.910191+0000', 'last_change': '2024-09-17T04:31:10.910191+0000', 'last_active': '2024-09-17T04:31:10.890602+0000', 'last_peered': '2024-09-17T04:31:10.890602+0000', 'last_clean': '2024-09-17T04:31:10.890602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T04:31:10.910191+0000', 'last_undegraded': '2024-09-17T04:31:10.910191+0000', 'last_fullsized': '2024-09-17T04:31:10.910191+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'last_clean_scrub_stamp': '2024-09-17T04:31:10.890602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T04:31:17.324 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T04:31:17.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.564+0000 7fa0d2000640 1 -- 172.21.15.100:0/3784047692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0cc0b48a0 msgr2=0x7fa0cc155370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:17.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.564+0000 7fa0d2000640 1 --2- 172.21.15.100:0/3784047692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0cc0b48a0 0x7fa0cc155370 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4009be0 tx=0x7fa0c402a890 comp rx=0 tx=0).stop 2024-09-17T04:31:17.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.564+0000 7fa0d2000640 1 -- 172.21.15.100:0/3784047692 shutdown_connections 2024-09-17T04:31:17.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.564+0000 7fa0d2000640 1 --2- 172.21.15.100:0/3784047692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0cc0b48a0 0x7fa0cc155370 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:17.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.564+0000 7fa0d2000640 1 -- 172.21.15.100:0/3784047692 >> 172.21.15.100:0/3784047692 conn(0x7fa0cc0b1e40 msgr2=0x7fa0cc0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:17.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.564+0000 7fa0d2000640 1 -- 172.21.15.100:0/3784047692 shutdown_connections 2024-09-17T04:31:17.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.564+0000 7fa0d2000640 1 -- 172.21.15.100:0/3784047692 wait complete. 2024-09-17T04:31:17.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.566+0000 7fa0d2000640 1 Processor -- start 2024-09-17T04:31:17.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.566+0000 7fa0d2000640 1 -- start start 2024-09-17T04:31:17.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.568+0000 7fa0d2000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0cc06f280 0x7fa0cc151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:17.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.568+0000 7fa0d2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0cc0b48a0 0x7fa0cc157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:17.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.568+0000 7fa0d2000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0cc1623a0 0x7fa0cc154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:17.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.568+0000 7fa0d2000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc155d20 con 0x7fa0cc0b48a0 2024-09-17T04:31:17.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.568+0000 7fa0d2000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc155ba0 con 0x7fa0cc06f280 2024-09-17T04:31:17.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.568+0000 7fa0d2000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa0cc169f10 con 0x7fa0cc1623a0 2024-09-17T04:31:17.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.568+0000 7fa0c3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0cc0b48a0 0x7fa0cc157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:17.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.568+0000 7fa0c3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0cc0b48a0 0x7fa0cc157b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55474/0 (socket says 172.21.15.100:55474) 2024-09-17T04:31:17.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.568+0000 7fa0c3e00640 1 -- 172.21.15.100:0/2086636939 learned_addr learned my addr 172.21.15.100:0/2086636939 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:17.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.568+0000 7fa0d0c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0cc06f280 0x7fa0cc151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:17.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.568+0000 7fa0d1600640 1 --2- 172.21.15.100:0/2086636939 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0cc1623a0 0x7fa0cc154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:17.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.569+0000 7fa0c3e00640 1 -- 172.21.15.100:0/2086636939 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0cc1623a0 msgr2=0x7fa0cc154630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:17.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.569+0000 7fa0c3e00640 1 --2- 172.21.15.100:0/2086636939 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0cc1623a0 0x7fa0cc154630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:17.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.569+0000 7fa0c3e00640 1 -- 172.21.15.100:0/2086636939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0cc06f280 msgr2=0x7fa0cc151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:17.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.569+0000 7fa0c3e00640 1 --2- 172.21.15.100:0/2086636939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0cc06f280 0x7fa0cc151db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:17.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.569+0000 7fa0c3e00640 1 -- 172.21.15.100:0/2086636939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0cc156270 con 0x7fa0cc0b48a0 2024-09-17T04:31:17.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.569+0000 7fa0c3e00640 1 --2- 172.21.15.100:0/2086636939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0cc0b48a0 0x7fa0cc157b90 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400c3f0 tx=0x7fa0b400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:17.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.569+0000 7fa0c9600640 1 -- 172.21.15.100:0/2086636939 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4017070 con 0x7fa0cc0b48a0 2024-09-17T04:31:17.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.569+0000 7fa0c9600640 1 -- 172.21.15.100:0/2086636939 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0b40035f0 con 0x7fa0cc0b48a0 2024-09-17T04:31:17.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.569+0000 7fa0d2000640 1 -- 172.21.15.100:0/2086636939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0cc156550 con 0x7fa0cc0b48a0 2024-09-17T04:31:17.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.569+0000 7fa0c9600640 1 -- 172.21.15.100:0/2086636939 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4003a40 con 0x7fa0cc0b48a0 2024-09-17T04:31:17.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.569+0000 7fa0d2000640 1 -- 172.21.15.100:0/2086636939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0cc156b10 con 0x7fa0cc0b48a0 2024-09-17T04:31:17.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.571+0000 7fa0d2000640 1 -- 172.21.15.100:0/2086636939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0cc06fa60 con 0x7fa0cc0b48a0 2024-09-17T04:31:17.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.571+0000 7fa0c9600640 1 -- 172.21.15.100:0/2086636939 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fa0b401a070 con 0x7fa0cc0b48a0 2024-09-17T04:31:17.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.572+0000 7fa0c9600640 1 --2- 172.21.15.100:0/2086636939 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa0a8079010 0x7fa0a807b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:17.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.573+0000 7fa0c9600640 1 -- 172.21.15.100:0/2086636939 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7fa0b409a930 con 0x7fa0cc0b48a0 2024-09-17T04:31:17.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.573+0000 7fa0d0c00640 1 --2- 172.21.15.100:0/2086636939 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa0a8079010 0x7fa0a807b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:17.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.573+0000 7fa0d0c00640 1 --2- 172.21.15.100:0/2086636939 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa0a8079010 0x7fa0a807b520 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc007fd0 tx=0x7fa0bc001370 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:17.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.577+0000 7fa0c9600640 1 -- 172.21.15.100:0/2086636939 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa0b4061f80 con 0x7fa0cc0b48a0 2024-09-17T04:31:17.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.827+0000 7fa0d2000640 1 -- 172.21.15.100:0/2086636939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fa0cc155330 con 0x7fa0cc0b48a0 2024-09-17T04:31:17.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.828+0000 7fa0c9600640 1 -- 172.21.15.100:0/2086636939 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1078 (secure 0 0 0) 0x7fa0b40669b0 con 0x7fa0cc0b48a0 2024-09-17T04:31:17.828 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:17.828 INFO:teuthology.orchestra.run.smithi100.stdout:{"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":450,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":119,"num_osds":8,"num_up_osds":8,"osd_up_since":1726547043,"num_in_osds":8,"osd_in_since":1726547032,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":268296192,"bytes_avail":772825817088,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":82,"btime":"2024-09-17T04:31:14:929364+0000","id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":7542}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-09-17T04:30:59.685414+0000","services":{}},"progress_events":{}} 2024-09-17T04:31:17.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.830+0000 7fa0d2000640 1 -- 172.21.15.100:0/2086636939 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa0a8079010 msgr2=0x7fa0a807b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:17.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.830+0000 7fa0d2000640 1 --2- 172.21.15.100:0/2086636939 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa0a8079010 0x7fa0a807b520 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc007fd0 tx=0x7fa0bc001370 comp rx=0 tx=0).stop 2024-09-17T04:31:17.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.830+0000 7fa0d2000640 1 -- 172.21.15.100:0/2086636939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0cc0b48a0 msgr2=0x7fa0cc157b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:17.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.830+0000 7fa0d2000640 1 --2- 172.21.15.100:0/2086636939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0cc0b48a0 0x7fa0cc157b90 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400c3f0 tx=0x7fa0b400c8b0 comp rx=0 tx=0).stop 2024-09-17T04:31:17.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.830+0000 7fa0d2000640 1 -- 172.21.15.100:0/2086636939 shutdown_connections 2024-09-17T04:31:17.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.830+0000 7fa0d2000640 1 --2- 172.21.15.100:0/2086636939 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa0a8079010 0x7fa0a807b520 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:17.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.830+0000 7fa0d2000640 1 --2- 172.21.15.100:0/2086636939 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0cc1623a0 0x7fa0cc154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:17.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.830+0000 7fa0d2000640 1 --2- 172.21.15.100:0/2086636939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0cc0b48a0 0x7fa0cc157b90 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:17.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.830+0000 7fa0d2000640 1 --2- 172.21.15.100:0/2086636939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0cc06f280 0x7fa0cc151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:17.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.830+0000 7fa0d2000640 1 -- 172.21.15.100:0/2086636939 >> 172.21.15.100:0/2086636939 conn(0x7fa0cc0b1e40 msgr2=0x7fa0cc0aea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:17.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.830+0000 7fa0d2000640 1 -- 172.21.15.100:0/2086636939 shutdown_connections 2024-09-17T04:31:17.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:17.830+0000 7fa0d2000640 1 -- 172.21.15.100:0/2086636939 wait complete. 2024-09-17T04:31:17.857 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T04:31:17.857 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:31:18.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.077+0000 7f9b2fe00640 1 -- 172.21.15.100:0/2715874358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b300ad9c0 msgr2=0x7f9b300af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:18.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.077+0000 7f9b2fe00640 1 --2- 172.21.15.100:0/2715874358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b300ad9c0 0x7f9b300af170 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f9b3006e790 tx=0x7f9b2002a560 comp rx=0 tx=0).stop 2024-09-17T04:31:18.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.077+0000 7f9b2fe00640 1 -- 172.21.15.100:0/2715874358 shutdown_connections 2024-09-17T04:31:18.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.077+0000 7f9b2fe00640 1 --2- 172.21.15.100:0/2715874358 >> v2:172.21.15.184:3300/0 conn(0x7f9b300b2f60 0x7f9b3015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:18.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.077+0000 7f9b2fe00640 1 --2- 172.21.15.100:0/2715874358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b300ad9c0 0x7f9b300af170 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:18.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.077+0000 7f9b2fe00640 1 -- 172.21.15.100:0/2715874358 >> 172.21.15.100:0/2715874358 conn(0x7f9b300b1e40 msgr2=0x7f9b300b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:18.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.077+0000 7f9b2fe00640 1 -- 172.21.15.100:0/2715874358 shutdown_connections 2024-09-17T04:31:18.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.077+0000 7f9b2fe00640 1 -- 172.21.15.100:0/2715874358 wait complete. 2024-09-17T04:31:18.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.080+0000 7f9b2fe00640 1 Processor -- start 2024-09-17T04:31:18.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.080+0000 7f9b2fe00640 1 -- start start 2024-09-17T04:31:18.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.081+0000 7f9b2fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b300ad9c0 0x7f9b3014b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:18.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.081+0000 7f9b2fe00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b300b2f60 0x7f9b3014b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:18.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.081+0000 7f9b2fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b3015d670 0x7f9b301514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:18.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.081+0000 7f9b2fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3006f290 con 0x7f9b300ad9c0 2024-09-17T04:31:18.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.081+0000 7f9b2fe00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9b30070d50 con 0x7f9b3015d670 2024-09-17T04:31:18.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.081+0000 7f9b2fe00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9b3006f410 con 0x7f9b300b2f60 2024-09-17T04:31:18.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.081+0000 7f9b2ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b300ad9c0 0x7f9b3014b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:18.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.081+0000 7f9b2ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b300ad9c0 0x7f9b3014b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55496/0 (socket says 172.21.15.100:55496) 2024-09-17T04:31:18.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.081+0000 7f9b2ea00640 1 -- 172.21.15.100:0/1886994102 learned_addr learned my addr 172.21.15.100:0/1886994102 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:18.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.082+0000 7f9b26000640 1 --2- 172.21.15.100:0/1886994102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b300b2f60 0x7f9b3014b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:18.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.082+0000 7f9b2f400640 1 --2- 172.21.15.100:0/1886994102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b3015d670 0x7f9b301514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:18.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.082+0000 7f9b2ea00640 1 -- 172.21.15.100:0/1886994102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b300b2f60 msgr2=0x7f9b3014b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:18.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.082+0000 7f9b2ea00640 1 --2- 172.21.15.100:0/1886994102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b300b2f60 0x7f9b3014b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:18.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.082+0000 7f9b2ea00640 1 -- 172.21.15.100:0/1886994102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b3015d670 msgr2=0x7f9b301514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:18.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.082+0000 7f9b2ea00640 1 --2- 172.21.15.100:0/1886994102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b3015d670 0x7f9b301514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:18.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.082+0000 7f9b2ea00640 1 -- 172.21.15.100:0/1886994102 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b30152430 con 0x7f9b300ad9c0 2024-09-17T04:31:18.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.082+0000 7f9b2f400640 1 --2- 172.21.15.100:0/1886994102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b3015d670 0x7f9b301514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:31:18.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.082+0000 7f9b2ea00640 1 --2- 172.21.15.100:0/1886994102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b300ad9c0 0x7f9b3014b2d0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f9b20034040 tx=0x7f9b200039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:18.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.082+0000 7f9b27400640 1 -- 172.21.15.100:0/1886994102 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2003e030 con 0x7f9b300ad9c0 2024-09-17T04:31:18.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.082+0000 7f9b27400640 1 -- 172.21.15.100:0/1886994102 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b200047a0 con 0x7f9b300ad9c0 2024-09-17T04:31:18.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.082+0000 7f9b2fe00640 1 -- 172.21.15.100:0/1886994102 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b300ccc40 con 0x7f9b300ad9c0 2024-09-17T04:31:18.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.082+0000 7f9b27400640 1 -- 172.21.15.100:0/1886994102 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b20035040 con 0x7f9b300ad9c0 2024-09-17T04:31:18.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.083+0000 7f9b2fe00640 1 -- 172.21.15.100:0/1886994102 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b3014ea70 con 0x7f9b300ad9c0 2024-09-17T04:31:18.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.083+0000 7f9b2fe00640 1 -- 172.21.15.100:0/1886994102 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b3015c880 con 0x7f9b300ad9c0 2024-09-17T04:31:18.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.085+0000 7f9b27400640 1 -- 172.21.15.100:0/1886994102 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f9b20041030 con 0x7f9b300ad9c0 2024-09-17T04:31:18.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.086+0000 7f9b27400640 1 --2- 172.21.15.100:0/1886994102 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9b0c079010 0x7f9b0c07b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:18.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.086+0000 7f9b27400640 1 -- 172.21.15.100:0/1886994102 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7f9b200bbef0 con 0x7f9b300ad9c0 2024-09-17T04:31:18.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.086+0000 7f9b26000640 1 --2- 172.21.15.100:0/1886994102 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9b0c079010 0x7f9b0c07b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:18.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.087+0000 7f9b26000640 1 --2- 172.21.15.100:0/1886994102 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9b0c079010 0x7f9b0c07b520 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9b18005fd0 tx=0x7f9b18005c50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:18.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.090+0000 7f9b27400640 1 -- 172.21.15.100:0/1886994102 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9b3015c880 con 0x7f9b300ad9c0 2024-09-17T04:31:18.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.271+0000 7f9b2fe00640 1 -- 172.21.15.100:0/1886994102 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9b3006f8a0 con 0x7f9b0c079010 2024-09-17T04:31:18.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.276+0000 7f9b27400640 1 -- 172.21.15.100:0/1886994102 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252478 (secure 0 0 0) 0x7f9b3006f8a0 con 0x7f9b0c079010 2024-09-17T04:31:18.279 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:18.281 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:31:18.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.280+0000 7f9b2fe00640 1 -- 172.21.15.100:0/1886994102 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9b0c079010 msgr2=0x7f9b0c07b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:18.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.280+0000 7f9b2fe00640 1 --2- 172.21.15.100:0/1886994102 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9b0c079010 0x7f9b0c07b520 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9b18005fd0 tx=0x7f9b18005c50 comp rx=0 tx=0).stop 2024-09-17T04:31:18.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.280+0000 7f9b2fe00640 1 -- 172.21.15.100:0/1886994102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b300ad9c0 msgr2=0x7f9b3014b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:18.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.280+0000 7f9b2fe00640 1 --2- 172.21.15.100:0/1886994102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b300ad9c0 0x7f9b3014b2d0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f9b20034040 tx=0x7f9b200039e0 comp rx=0 tx=0).stop 2024-09-17T04:31:18.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.280+0000 7f9b2fe00640 1 -- 172.21.15.100:0/1886994102 shutdown_connections 2024-09-17T04:31:18.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.280+0000 7f9b2fe00640 1 --2- 172.21.15.100:0/1886994102 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9b0c079010 0x7f9b0c07b520 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:18.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.280+0000 7f9b2fe00640 1 --2- 172.21.15.100:0/1886994102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b3015d670 0x7f9b301514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:18.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.280+0000 7f9b2fe00640 1 --2- 172.21.15.100:0/1886994102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b300b2f60 0x7f9b3014b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:18.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.280+0000 7f9b2fe00640 1 --2- 172.21.15.100:0/1886994102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b300ad9c0 0x7f9b3014b2d0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:18.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.280+0000 7f9b2fe00640 1 -- 172.21.15.100:0/1886994102 >> 172.21.15.100:0/1886994102 conn(0x7f9b300b1e40 msgr2=0x7f9b3015a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:18.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.280+0000 7f9b2fe00640 1 -- 172.21.15.100:0/1886994102 shutdown_connections 2024-09-17T04:31:18.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:18.280+0000 7f9b2fe00640 1 -- 172.21.15.100:0/1886994102 wait complete. 2024-09-17T04:31:18.302 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-09-17T04:31:17.688903+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262276,"kb_used_data":12744,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712444,"statfs":{"total":773094113280,"available":772825542656,"internally_reserved":0,"allocated":13049856,"data_stored":6367068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.786092"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.906609+0000","last_change":"2024-09-17T04:31:11.915628+0000","last_active":"2024-09-17T04:31:12.906609+0000","last_peered":"2024-09-17T04:31:12.906609+0000","last_clean":"2024-09-17T04:31:12.906609+0000","last_became_active":"2024-09-17T04:31:11.914741+0000","last_became_peered":"2024-09-17T04:31:11.914741+0000","last_unstale":"2024-09-17T04:31:12.906609+0000","last_undegraded":"2024-09-17T04:31:12.906609+0000","last_fullsized":"2024-09-17T04:31:12.906609+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:06:44.732901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552802+0000","last_change":"2024-09-17T04:31:11.927079+0000","last_active":"2024-09-17T04:31:13.552802+0000","last_peered":"2024-09-17T04:31:13.552802+0000","last_clean":"2024-09-17T04:31:13.552802+0000","last_became_active":"2024-09-17T04:31:11.926754+0000","last_became_peered":"2024-09-17T04:31:11.926754+0000","last_unstale":"2024-09-17T04:31:13.552802+0000","last_undegraded":"2024-09-17T04:31:13.552802+0000","last_fullsized":"2024-09-17T04:31:13.552802+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:43:13.340843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420705+0000","last_change":"2024-09-17T04:31:11.924404+0000","last_active":"2024-09-17T04:31:14.420705+0000","last_peered":"2024-09-17T04:31:14.420705+0000","last_clean":"2024-09-17T04:31:14.420705+0000","last_became_active":"2024-09-17T04:31:11.924123+0000","last_became_peered":"2024-09-17T04:31:11.924123+0000","last_unstale":"2024-09-17T04:31:14.420705+0000","last_undegraded":"2024-09-17T04:31:14.420705+0000","last_fullsized":"2024-09-17T04:31:14.420705+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:31.226598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907370+0000","last_change":"2024-09-17T04:31:11.931535+0000","last_active":"2024-09-17T04:31:12.907370+0000","last_peered":"2024-09-17T04:31:12.907370+0000","last_clean":"2024-09-17T04:31:12.907370+0000","last_became_active":"2024-09-17T04:31:11.930362+0000","last_became_peered":"2024-09-17T04:31:11.930362+0000","last_unstale":"2024-09-17T04:31:12.907370+0000","last_undegraded":"2024-09-17T04:31:12.907370+0000","last_fullsized":"2024-09-17T04:31:12.907370+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:17.964326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.926064+0000","last_change":"2024-09-17T04:31:11.938594+0000","last_active":"2024-09-17T04:31:12.926064+0000","last_peered":"2024-09-17T04:31:12.926064+0000","last_clean":"2024-09-17T04:31:12.926064+0000","last_became_active":"2024-09-17T04:31:11.935855+0000","last_became_peered":"2024-09-17T04:31:11.935855+0000","last_unstale":"2024-09-17T04:31:12.926064+0000","last_undegraded":"2024-09-17T04:31:12.926064+0000","last_fullsized":"2024-09-17T04:31:12.926064+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:48.696810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907944+0000","last_change":"2024-09-17T04:31:11.913994+0000","last_active":"2024-09-17T04:31:12.907944+0000","last_peered":"2024-09-17T04:31:12.907944+0000","last_clean":"2024-09-17T04:31:12.907944+0000","last_became_active":"2024-09-17T04:31:11.913641+0000","last_became_peered":"2024-09-17T04:31:11.913641+0000","last_unstale":"2024-09-17T04:31:12.907944+0000","last_undegraded":"2024-09-17T04:31:12.907944+0000","last_fullsized":"2024-09-17T04:31:12.907944+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:31.426067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907825+0000","last_change":"2024-09-17T04:31:11.931072+0000","last_active":"2024-09-17T04:31:12.907825+0000","last_peered":"2024-09-17T04:31:12.907825+0000","last_clean":"2024-09-17T04:31:12.907825+0000","last_became_active":"2024-09-17T04:31:11.929265+0000","last_became_peered":"2024-09-17T04:31:11.929265+0000","last_unstale":"2024-09-17T04:31:12.907825+0000","last_undegraded":"2024-09-17T04:31:12.907825+0000","last_fullsized":"2024-09-17T04:31:12.907825+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:31.927735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420393+0000","last_change":"2024-09-17T04:31:11.939991+0000","last_active":"2024-09-17T04:31:14.420393+0000","last_peered":"2024-09-17T04:31:14.420393+0000","last_clean":"2024-09-17T04:31:14.420393+0000","last_became_active":"2024-09-17T04:31:11.939519+0000","last_became_peered":"2024-09-17T04:31:11.939519+0000","last_unstale":"2024-09-17T04:31:14.420393+0000","last_undegraded":"2024-09-17T04:31:14.420393+0000","last_fullsized":"2024-09-17T04:31:14.420393+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:53:08.566341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420860+0000","last_change":"2024-09-17T04:31:11.937050+0000","last_active":"2024-09-17T04:31:14.420860+0000","last_peered":"2024-09-17T04:31:14.420860+0000","last_clean":"2024-09-17T04:31:14.420860+0000","last_became_active":"2024-09-17T04:31:11.935171+0000","last_became_peered":"2024-09-17T04:31:11.935171+0000","last_unstale":"2024-09-17T04:31:14.420860+0000","last_undegraded":"2024-09-17T04:31:14.420860+0000","last_fullsized":"2024-09-17T04:31:14.420860+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:19:55.960416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552586+0000","last_change":"2024-09-17T04:31:11.927788+0000","last_active":"2024-09-17T04:31:13.552586+0000","last_peered":"2024-09-17T04:31:13.552586+0000","last_clean":"2024-09-17T04:31:13.552586+0000","last_became_active":"2024-09-17T04:31:11.927474+0000","last_became_peered":"2024-09-17T04:31:11.927474+0000","last_unstale":"2024-09-17T04:31:13.552586+0000","last_undegraded":"2024-09-17T04:31:13.552586+0000","last_fullsized":"2024-09-17T04:31:13.552586+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:44.817656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907978+0000","last_change":"2024-09-17T04:31:11.927761+0000","last_active":"2024-09-17T04:31:12.907978+0000","last_peered":"2024-09-17T04:31:12.907978+0000","last_clean":"2024-09-17T04:31:12.907978+0000","last_became_active":"2024-09-17T04:31:11.927451+0000","last_became_peered":"2024-09-17T04:31:11.927451+0000","last_unstale":"2024-09-17T04:31:12.907978+0000","last_undegraded":"2024-09-17T04:31:12.907978+0000","last_fullsized":"2024-09-17T04:31:12.907978+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:59:30.997201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420696+0000","last_change":"2024-09-17T04:31:11.936942+0000","last_active":"2024-09-17T04:31:14.420696+0000","last_peered":"2024-09-17T04:31:14.420696+0000","last_clean":"2024-09-17T04:31:14.420696+0000","last_became_active":"2024-09-17T04:31:11.935646+0000","last_became_peered":"2024-09-17T04:31:11.935646+0000","last_unstale":"2024-09-17T04:31:14.420696+0000","last_undegraded":"2024-09-17T04:31:14.420696+0000","last_fullsized":"2024-09-17T04:31:14.420696+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:17:35.366008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907590+0000","last_change":"2024-09-17T04:31:11.916650+0000","last_active":"2024-09-17T04:31:12.907590+0000","last_peered":"2024-09-17T04:31:12.907590+0000","last_clean":"2024-09-17T04:31:12.907590+0000","last_became_active":"2024-09-17T04:31:11.916136+0000","last_became_peered":"2024-09-17T04:31:11.916136+0000","last_unstale":"2024-09-17T04:31:12.907590+0000","last_undegraded":"2024-09-17T04:31:12.907590+0000","last_fullsized":"2024-09-17T04:31:12.907590+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:45.053959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552430+0000","last_change":"2024-09-17T04:31:11.933739+0000","last_active":"2024-09-17T04:31:13.552430+0000","last_peered":"2024-09-17T04:31:13.552430+0000","last_clean":"2024-09-17T04:31:13.552430+0000","last_became_active":"2024-09-17T04:31:11.932707+0000","last_became_peered":"2024-09-17T04:31:11.932707+0000","last_unstale":"2024-09-17T04:31:13.552430+0000","last_undegraded":"2024-09-17T04:31:13.552430+0000","last_fullsized":"2024-09-17T04:31:13.552430+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:43.901633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.906214+0000","last_change":"2024-09-17T04:31:11.932007+0000","last_active":"2024-09-17T04:31:12.906214+0000","last_peered":"2024-09-17T04:31:12.906214+0000","last_clean":"2024-09-17T04:31:12.906214+0000","last_became_active":"2024-09-17T04:31:11.931790+0000","last_became_peered":"2024-09-17T04:31:11.931790+0000","last_unstale":"2024-09-17T04:31:12.906214+0000","last_undegraded":"2024-09-17T04:31:12.906214+0000","last_fullsized":"2024-09-17T04:31:12.906214+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:12.445854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.419887+0000","last_change":"2024-09-17T04:31:11.910629+0000","last_active":"2024-09-17T04:31:14.419887+0000","last_peered":"2024-09-17T04:31:14.419887+0000","last_clean":"2024-09-17T04:31:14.419887+0000","last_became_active":"2024-09-17T04:31:11.910301+0000","last_became_peered":"2024-09-17T04:31:11.910301+0000","last_unstale":"2024-09-17T04:31:14.419887+0000","last_undegraded":"2024-09-17T04:31:14.419887+0000","last_fullsized":"2024-09-17T04:31:14.419887+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:57.039960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.908274+0000","last_change":"2024-09-17T04:31:11.927492+0000","last_active":"2024-09-17T04:31:12.908274+0000","last_peered":"2024-09-17T04:31:12.908274+0000","last_clean":"2024-09-17T04:31:12.908274+0000","last_became_active":"2024-09-17T04:31:11.925763+0000","last_became_peered":"2024-09-17T04:31:11.925763+0000","last_unstale":"2024-09-17T04:31:12.908274+0000","last_undegraded":"2024-09-17T04:31:12.908274+0000","last_fullsized":"2024-09-17T04:31:12.908274+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:34.292501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420047+0000","last_change":"2024-09-17T04:31:11.927839+0000","last_active":"2024-09-17T04:31:14.420047+0000","last_peered":"2024-09-17T04:31:14.420047+0000","last_clean":"2024-09-17T04:31:14.420047+0000","last_became_active":"2024-09-17T04:31:11.927504+0000","last_became_peered":"2024-09-17T04:31:11.927504+0000","last_unstale":"2024-09-17T04:31:14.420047+0000","last_undegraded":"2024-09-17T04:31:14.420047+0000","last_fullsized":"2024-09-17T04:31:14.420047+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:52:32.891281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907876+0000","last_change":"2024-09-17T04:31:11.927367+0000","last_active":"2024-09-17T04:31:12.907876+0000","last_peered":"2024-09-17T04:31:12.907876+0000","last_clean":"2024-09-17T04:31:12.907876+0000","last_became_active":"2024-09-17T04:31:11.926983+0000","last_became_peered":"2024-09-17T04:31:11.926983+0000","last_unstale":"2024-09-17T04:31:12.907876+0000","last_undegraded":"2024-09-17T04:31:12.907876+0000","last_fullsized":"2024-09-17T04:31:12.907876+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:43.351066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.419974+0000","last_change":"2024-09-17T04:31:11.932828+0000","last_active":"2024-09-17T04:31:14.419974+0000","last_peered":"2024-09-17T04:31:14.419974+0000","last_clean":"2024-09-17T04:31:14.419974+0000","last_became_active":"2024-09-17T04:31:11.925915+0000","last_became_peered":"2024-09-17T04:31:11.925915+0000","last_unstale":"2024-09-17T04:31:14.419974+0000","last_undegraded":"2024-09-17T04:31:14.419974+0000","last_fullsized":"2024-09-17T04:31:14.419974+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:42.403746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.925760+0000","last_change":"2024-09-17T04:31:11.929745+0000","last_active":"2024-09-17T04:31:12.925760+0000","last_peered":"2024-09-17T04:31:12.925760+0000","last_clean":"2024-09-17T04:31:12.925760+0000","last_became_active":"2024-09-17T04:31:11.929386+0000","last_became_peered":"2024-09-17T04:31:11.929386+0000","last_unstale":"2024-09-17T04:31:12.925760+0000","last_undegraded":"2024-09-17T04:31:12.925760+0000","last_fullsized":"2024-09-17T04:31:12.925760+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:25.555379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907083+0000","last_change":"2024-09-17T04:31:11.933052+0000","last_active":"2024-09-17T04:31:12.907083+0000","last_peered":"2024-09-17T04:31:12.907083+0000","last_clean":"2024-09-17T04:31:12.907083+0000","last_became_active":"2024-09-17T04:31:11.932721+0000","last_became_peered":"2024-09-17T04:31:11.932721+0000","last_unstale":"2024-09-17T04:31:12.907083+0000","last_undegraded":"2024-09-17T04:31:12.907083+0000","last_fullsized":"2024-09-17T04:31:12.907083+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:47:42.414199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420662+0000","last_change":"2024-09-17T04:31:11.940132+0000","last_active":"2024-09-17T04:31:14.420662+0000","last_peered":"2024-09-17T04:31:14.420662+0000","last_clean":"2024-09-17T04:31:14.420662+0000","last_became_active":"2024-09-17T04:31:11.934036+0000","last_became_peered":"2024-09-17T04:31:11.934036+0000","last_unstale":"2024-09-17T04:31:14.420662+0000","last_undegraded":"2024-09-17T04:31:14.420662+0000","last_fullsized":"2024-09-17T04:31:14.420662+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:09.888729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.906917+0000","last_change":"2024-09-17T04:31:11.921074+0000","last_active":"2024-09-17T04:31:12.906917+0000","last_peered":"2024-09-17T04:31:12.906917+0000","last_clean":"2024-09-17T04:31:12.906917+0000","last_became_active":"2024-09-17T04:31:11.920773+0000","last_became_peered":"2024-09-17T04:31:11.920773+0000","last_unstale":"2024-09-17T04:31:12.906917+0000","last_undegraded":"2024-09-17T04:31:12.906917+0000","last_fullsized":"2024-09-17T04:31:12.906917+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:01.695666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552713+0000","last_change":"2024-09-17T04:31:11.930143+0000","last_active":"2024-09-17T04:31:13.552713+0000","last_peered":"2024-09-17T04:31:13.552713+0000","last_clean":"2024-09-17T04:31:13.552713+0000","last_became_active":"2024-09-17T04:31:11.929693+0000","last_became_peered":"2024-09-17T04:31:11.929693+0000","last_unstale":"2024-09-17T04:31:13.552713+0000","last_undegraded":"2024-09-17T04:31:13.552713+0000","last_fullsized":"2024-09-17T04:31:13.552713+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:56:14.273325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420204+0000","last_change":"2024-09-17T04:31:11.911729+0000","last_active":"2024-09-17T04:31:14.420204+0000","last_peered":"2024-09-17T04:31:14.420204+0000","last_clean":"2024-09-17T04:31:14.420204+0000","last_became_active":"2024-09-17T04:31:11.911437+0000","last_became_peered":"2024-09-17T04:31:11.911437+0000","last_unstale":"2024-09-17T04:31:14.420204+0000","last_undegraded":"2024-09-17T04:31:14.420204+0000","last_fullsized":"2024-09-17T04:31:14.420204+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:30:58.089743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.906605+0000","last_change":"2024-09-17T04:31:11.914117+0000","last_active":"2024-09-17T04:31:12.906605+0000","last_peered":"2024-09-17T04:31:12.906605+0000","last_clean":"2024-09-17T04:31:12.906605+0000","last_became_active":"2024-09-17T04:31:11.913390+0000","last_became_peered":"2024-09-17T04:31:11.913390+0000","last_unstale":"2024-09-17T04:31:12.906605+0000","last_undegraded":"2024-09-17T04:31:12.906605+0000","last_fullsized":"2024-09-17T04:31:12.906605+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:23:57.520477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.239989+0000","last_change":"2024-09-17T04:31:11.929489+0000","last_active":"2024-09-17T04:31:13.239989+0000","last_peered":"2024-09-17T04:31:13.239989+0000","last_clean":"2024-09-17T04:31:13.239989+0000","last_became_active":"2024-09-17T04:31:11.929161+0000","last_became_peered":"2024-09-17T04:31:11.929161+0000","last_unstale":"2024-09-17T04:31:13.239989+0000","last_undegraded":"2024-09-17T04:31:13.239989+0000","last_fullsized":"2024-09-17T04:31:13.239989+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:32.213268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.906409+0000","last_change":"2024-09-17T04:31:11.935751+0000","last_active":"2024-09-17T04:31:12.906409+0000","last_peered":"2024-09-17T04:31:12.906409+0000","last_clean":"2024-09-17T04:31:12.906409+0000","last_became_active":"2024-09-17T04:31:11.934477+0000","last_became_peered":"2024-09-17T04:31:11.934477+0000","last_unstale":"2024-09-17T04:31:12.906409+0000","last_undegraded":"2024-09-17T04:31:12.906409+0000","last_fullsized":"2024-09-17T04:31:12.906409+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:32.238876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.906247+0000","last_change":"2024-09-17T04:31:11.934997+0000","last_active":"2024-09-17T04:31:12.906247+0000","last_peered":"2024-09-17T04:31:12.906247+0000","last_clean":"2024-09-17T04:31:12.906247+0000","last_became_active":"2024-09-17T04:31:11.931566+0000","last_became_peered":"2024-09-17T04:31:11.931566+0000","last_unstale":"2024-09-17T04:31:12.906247+0000","last_undegraded":"2024-09-17T04:31:12.906247+0000","last_fullsized":"2024-09-17T04:31:12.906247+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:33:16.150618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.925823+0000","last_change":"2024-09-17T04:31:11.937985+0000","last_active":"2024-09-17T04:31:12.925823+0000","last_peered":"2024-09-17T04:31:12.925823+0000","last_clean":"2024-09-17T04:31:12.925823+0000","last_became_active":"2024-09-17T04:31:11.934427+0000","last_became_peered":"2024-09-17T04:31:11.934427+0000","last_unstale":"2024-09-17T04:31:12.925823+0000","last_undegraded":"2024-09-17T04:31:12.925823+0000","last_fullsized":"2024-09-17T04:31:12.925823+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:17.218054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907697+0000","last_change":"2024-09-17T04:31:11.914266+0000","last_active":"2024-09-17T04:31:12.907697+0000","last_peered":"2024-09-17T04:31:12.907697+0000","last_clean":"2024-09-17T04:31:12.907697+0000","last_became_active":"2024-09-17T04:31:11.913960+0000","last_became_peered":"2024-09-17T04:31:11.913960+0000","last_unstale":"2024-09-17T04:31:12.907697+0000","last_undegraded":"2024-09-17T04:31:12.907697+0000","last_fullsized":"2024-09-17T04:31:12.907697+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:06.248101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420220+0000","last_change":"2024-09-17T04:31:11.923380+0000","last_active":"2024-09-17T04:31:14.420220+0000","last_peered":"2024-09-17T04:31:14.420220+0000","last_clean":"2024-09-17T04:31:14.420220+0000","last_became_active":"2024-09-17T04:31:11.923084+0000","last_became_peered":"2024-09-17T04:31:11.923084+0000","last_unstale":"2024-09-17T04:31:14.420220+0000","last_undegraded":"2024-09-17T04:31:14.420220+0000","last_fullsized":"2024-09-17T04:31:14.420220+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:31.148864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"118'4","reported_seq":19,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.926316+0000","last_change":"2024-09-17T04:31:10.925886+0000","last_active":"2024-09-17T04:31:12.926316+0000","last_peered":"2024-09-17T04:31:12.926316+0000","last_clean":"2024-09-17T04:31:12.926316+0000","last_became_active":"2024-09-17T04:31:10.925455+0000","last_became_peered":"2024-09-17T04:31:10.925455+0000","last_unstale":"2024-09-17T04:31:12.926316+0000","last_undegraded":"2024-09-17T04:31:12.926316+0000","last_fullsized":"2024-09-17T04:31:12.926316+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:40:57.412470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420901+0000","last_change":"2024-09-17T04:31:11.938822+0000","last_active":"2024-09-17T04:31:14.420901+0000","last_peered":"2024-09-17T04:31:14.420901+0000","last_clean":"2024-09-17T04:31:14.420901+0000","last_became_active":"2024-09-17T04:31:11.938412+0000","last_became_peered":"2024-09-17T04:31:11.938412+0000","last_unstale":"2024-09-17T04:31:14.420901+0000","last_undegraded":"2024-09-17T04:31:14.420901+0000","last_fullsized":"2024-09-17T04:31:14.420901+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:10.737466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"118'1","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.240024+0000","last_change":"2024-09-17T04:31:10.923497+0000","last_active":"2024-09-17T04:31:13.240024+0000","last_peered":"2024-09-17T04:31:13.240024+0000","last_clean":"2024-09-17T04:31:13.240024+0000","last_became_active":"2024-09-17T04:31:10.923086+0000","last_became_peered":"2024-09-17T04:31:10.923086+0000","last_unstale":"2024-09-17T04:31:13.240024+0000","last_undegraded":"2024-09-17T04:31:13.240024+0000","last_fullsized":"2024-09-17T04:31:13.240024+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:38:44.426816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.926099+0000","last_change":"2024-09-17T04:31:11.928219+0000","last_active":"2024-09-17T04:31:12.926099+0000","last_peered":"2024-09-17T04:31:12.926099+0000","last_clean":"2024-09-17T04:31:12.926099+0000","last_became_active":"2024-09-17T04:31:11.927796+0000","last_became_peered":"2024-09-17T04:31:11.927796+0000","last_unstale":"2024-09-17T04:31:12.926099+0000","last_undegraded":"2024-09-17T04:31:12.926099+0000","last_fullsized":"2024-09-17T04:31:12.926099+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:25.762200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.908056+0000","last_change":"2024-09-17T04:31:10.922177+0000","last_active":"2024-09-17T04:31:12.908056+0000","last_peered":"2024-09-17T04:31:12.908056+0000","last_clean":"2024-09-17T04:31:12.908056+0000","last_became_active":"2024-09-17T04:31:10.921875+0000","last_became_peered":"2024-09-17T04:31:10.921875+0000","last_unstale":"2024-09-17T04:31:12.908056+0000","last_undegraded":"2024-09-17T04:31:12.908056+0000","last_fullsized":"2024-09-17T04:31:12.908056+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:33:18.323563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.926082+0000","last_change":"2024-09-17T04:31:11.935286+0000","last_active":"2024-09-17T04:31:12.926082+0000","last_peered":"2024-09-17T04:31:12.926082+0000","last_clean":"2024-09-17T04:31:12.926082+0000","last_became_active":"2024-09-17T04:31:11.933218+0000","last_became_peered":"2024-09-17T04:31:11.933218+0000","last_unstale":"2024-09-17T04:31:12.926082+0000","last_undegraded":"2024-09-17T04:31:12.926082+0000","last_fullsized":"2024-09-17T04:31:12.926082+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:12.070463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552714+0000","last_change":"2024-09-17T04:31:10.917285+0000","last_active":"2024-09-17T04:31:13.552714+0000","last_peered":"2024-09-17T04:31:13.552714+0000","last_clean":"2024-09-17T04:31:13.552714+0000","last_became_active":"2024-09-17T04:31:10.916739+0000","last_became_peered":"2024-09-17T04:31:10.916739+0000","last_unstale":"2024-09-17T04:31:13.552714+0000","last_undegraded":"2024-09-17T04:31:13.552714+0000","last_fullsized":"2024-09-17T04:31:13.552714+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:06:32.134623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.419694+0000","last_change":"2024-09-17T04:31:11.914279+0000","last_active":"2024-09-17T04:31:14.419694+0000","last_peered":"2024-09-17T04:31:14.419694+0000","last_clean":"2024-09-17T04:31:14.419694+0000","last_became_active":"2024-09-17T04:31:11.913971+0000","last_became_peered":"2024-09-17T04:31:11.913971+0000","last_unstale":"2024-09-17T04:31:14.419694+0000","last_undegraded":"2024-09-17T04:31:14.419694+0000","last_fullsized":"2024-09-17T04:31:14.419694+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:42:34.939903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"118'1","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552208+0000","last_change":"2024-09-17T04:31:10.913568+0000","last_active":"2024-09-17T04:31:13.552208+0000","last_peered":"2024-09-17T04:31:13.552208+0000","last_clean":"2024-09-17T04:31:13.552208+0000","last_became_active":"2024-09-17T04:31:10.913273+0000","last_became_peered":"2024-09-17T04:31:10.913273+0000","last_unstale":"2024-09-17T04:31:13.552208+0000","last_undegraded":"2024-09-17T04:31:13.552208+0000","last_fullsized":"2024-09-17T04:31:13.552208+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:45.104445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907798+0000","last_change":"2024-09-17T04:31:11.927550+0000","last_active":"2024-09-17T04:31:12.907798+0000","last_peered":"2024-09-17T04:31:12.907798+0000","last_clean":"2024-09-17T04:31:12.907798+0000","last_became_active":"2024-09-17T04:31:11.926417+0000","last_became_peered":"2024-09-17T04:31:11.926417+0000","last_unstale":"2024-09-17T04:31:12.907798+0000","last_undegraded":"2024-09-17T04:31:12.907798+0000","last_fullsized":"2024-09-17T04:31:12.907798+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:13.917612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"118'1","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.419614+0000","last_change":"2024-09-17T04:31:10.917769+0000","last_active":"2024-09-17T04:31:14.419614+0000","last_peered":"2024-09-17T04:31:14.419614+0000","last_clean":"2024-09-17T04:31:14.419614+0000","last_became_active":"2024-09-17T04:31:10.917493+0000","last_became_peered":"2024-09-17T04:31:10.917493+0000","last_unstale":"2024-09-17T04:31:14.419614+0000","last_undegraded":"2024-09-17T04:31:14.419614+0000","last_fullsized":"2024-09-17T04:31:14.419614+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:53.541655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420445+0000","last_change":"2024-09-17T04:31:11.926740+0000","last_active":"2024-09-17T04:31:14.420445+0000","last_peered":"2024-09-17T04:31:14.420445+0000","last_clean":"2024-09-17T04:31:14.420445+0000","last_became_active":"2024-09-17T04:31:11.926396+0000","last_became_peered":"2024-09-17T04:31:11.926396+0000","last_unstale":"2024-09-17T04:31:14.420445+0000","last_undegraded":"2024-09-17T04:31:14.420445+0000","last_fullsized":"2024-09-17T04:31:14.420445+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:42.899760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907484+0000","last_change":"2024-09-17T04:31:10.919927+0000","last_active":"2024-09-17T04:31:12.907484+0000","last_peered":"2024-09-17T04:31:12.907484+0000","last_clean":"2024-09-17T04:31:12.907484+0000","last_became_active":"2024-09-17T04:31:10.919536+0000","last_became_peered":"2024-09-17T04:31:10.919536+0000","last_unstale":"2024-09-17T04:31:12.907484+0000","last_undegraded":"2024-09-17T04:31:12.907484+0000","last_fullsized":"2024-09-17T04:31:12.907484+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:47.465465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552277+0000","last_change":"2024-09-17T04:31:11.924910+0000","last_active":"2024-09-17T04:31:13.552277+0000","last_peered":"2024-09-17T04:31:13.552277+0000","last_clean":"2024-09-17T04:31:13.552277+0000","last_became_active":"2024-09-17T04:31:11.924604+0000","last_became_peered":"2024-09-17T04:31:11.924604+0000","last_unstale":"2024-09-17T04:31:13.552277+0000","last_undegraded":"2024-09-17T04:31:13.552277+0000","last_fullsized":"2024-09-17T04:31:13.552277+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:34.516657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552651+0000","last_change":"2024-09-17T04:31:10.926647+0000","last_active":"2024-09-17T04:31:13.552651+0000","last_peered":"2024-09-17T04:31:13.552651+0000","last_clean":"2024-09-17T04:31:13.552651+0000","last_became_active":"2024-09-17T04:31:10.926287+0000","last_became_peered":"2024-09-17T04:31:10.926287+0000","last_unstale":"2024-09-17T04:31:13.552651+0000","last_undegraded":"2024-09-17T04:31:13.552651+0000","last_fullsized":"2024-09-17T04:31:13.552651+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:11.279144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907050+0000","last_change":"2024-09-17T04:31:11.933200+0000","last_active":"2024-09-17T04:31:12.907050+0000","last_peered":"2024-09-17T04:31:12.907050+0000","last_clean":"2024-09-17T04:31:12.907050+0000","last_became_active":"2024-09-17T04:31:11.930452+0000","last_became_peered":"2024-09-17T04:31:11.930452+0000","last_unstale":"2024-09-17T04:31:12.907050+0000","last_undegraded":"2024-09-17T04:31:12.907050+0000","last_fullsized":"2024-09-17T04:31:12.907050+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:42:33.591948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.240289+0000","last_change":"2024-09-17T04:31:10.926032+0000","last_active":"2024-09-17T04:31:13.240289+0000","last_peered":"2024-09-17T04:31:13.240289+0000","last_clean":"2024-09-17T04:31:13.240289+0000","last_became_active":"2024-09-17T04:31:10.925736+0000","last_became_peered":"2024-09-17T04:31:10.925736+0000","last_unstale":"2024-09-17T04:31:13.240289+0000","last_undegraded":"2024-09-17T04:31:13.240289+0000","last_fullsized":"2024-09-17T04:31:13.240289+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:33.508882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.240159+0000","last_change":"2024-09-17T04:31:11.929279+0000","last_active":"2024-09-17T04:31:13.240159+0000","last_peered":"2024-09-17T04:31:13.240159+0000","last_clean":"2024-09-17T04:31:13.240159+0000","last_became_active":"2024-09-17T04:31:11.928813+0000","last_became_peered":"2024-09-17T04:31:11.928813+0000","last_unstale":"2024-09-17T04:31:13.240159+0000","last_undegraded":"2024-09-17T04:31:13.240159+0000","last_fullsized":"2024-09-17T04:31:13.240159+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:29.918321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420321+0000","last_change":"2024-09-17T04:31:10.930027+0000","last_active":"2024-09-17T04:31:14.420321+0000","last_peered":"2024-09-17T04:31:14.420321+0000","last_clean":"2024-09-17T04:31:14.420321+0000","last_became_active":"2024-09-17T04:31:10.929648+0000","last_became_peered":"2024-09-17T04:31:10.929648+0000","last_unstale":"2024-09-17T04:31:14.420321+0000","last_undegraded":"2024-09-17T04:31:14.420321+0000","last_fullsized":"2024-09-17T04:31:14.420321+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:46.988963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.926236+0000","last_change":"2024-09-17T04:31:11.937498+0000","last_active":"2024-09-17T04:31:12.926236+0000","last_peered":"2024-09-17T04:31:12.926236+0000","last_clean":"2024-09-17T04:31:12.926236+0000","last_became_active":"2024-09-17T04:31:11.937168+0000","last_became_peered":"2024-09-17T04:31:11.937168+0000","last_unstale":"2024-09-17T04:31:12.926236+0000","last_undegraded":"2024-09-17T04:31:12.926236+0000","last_fullsized":"2024-09-17T04:31:12.926236+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:53:17.261085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"118'3","reported_seq":18,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.239991+0000","last_change":"2024-09-17T04:31:10.917626+0000","last_active":"2024-09-17T04:31:13.239991+0000","last_peered":"2024-09-17T04:31:13.239991+0000","last_clean":"2024-09-17T04:31:13.239991+0000","last_became_active":"2024-09-17T04:31:10.917239+0000","last_became_peered":"2024-09-17T04:31:10.917239+0000","last_unstale":"2024-09-17T04:31:13.239991+0000","last_undegraded":"2024-09-17T04:31:13.239991+0000","last_fullsized":"2024-09-17T04:31:13.239991+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:49:42.911365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.926467+0000","last_change":"2024-09-17T04:31:11.933050+0000","last_active":"2024-09-17T04:31:12.926467+0000","last_peered":"2024-09-17T04:31:12.926467+0000","last_clean":"2024-09-17T04:31:12.926467+0000","last_became_active":"2024-09-17T04:31:11.932673+0000","last_became_peered":"2024-09-17T04:31:11.932673+0000","last_unstale":"2024-09-17T04:31:12.926467+0000","last_undegraded":"2024-09-17T04:31:12.926467+0000","last_fullsized":"2024-09-17T04:31:12.926467+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:20.703359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"118'6","reported_seq":21,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.421089+0000","last_change":"2024-09-17T04:31:10.929556+0000","last_active":"2024-09-17T04:31:14.421089+0000","last_peered":"2024-09-17T04:31:14.421089+0000","last_clean":"2024-09-17T04:31:14.421089+0000","last_became_active":"2024-09-17T04:31:10.929171+0000","last_became_peered":"2024-09-17T04:31:10.929171+0000","last_unstale":"2024-09-17T04:31:14.421089+0000","last_undegraded":"2024-09-17T04:31:14.421089+0000","last_fullsized":"2024-09-17T04:31:14.421089+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:46:00.805710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.906665+0000","last_change":"2024-09-17T04:31:11.922654+0000","last_active":"2024-09-17T04:31:12.906665+0000","last_peered":"2024-09-17T04:31:12.906665+0000","last_clean":"2024-09-17T04:31:12.906665+0000","last_became_active":"2024-09-17T04:31:11.922366+0000","last_became_peered":"2024-09-17T04:31:11.922366+0000","last_unstale":"2024-09-17T04:31:12.906665+0000","last_undegraded":"2024-09-17T04:31:12.906665+0000","last_fullsized":"2024-09-17T04:31:12.906665+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:02.214226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.906647+0000","last_change":"2024-09-17T04:31:10.921170+0000","last_active":"2024-09-17T04:31:12.906647+0000","last_peered":"2024-09-17T04:31:12.906647+0000","last_clean":"2024-09-17T04:31:12.906647+0000","last_became_active":"2024-09-17T04:31:10.920873+0000","last_became_peered":"2024-09-17T04:31:10.920873+0000","last_unstale":"2024-09-17T04:31:12.906647+0000","last_undegraded":"2024-09-17T04:31:12.906647+0000","last_fullsized":"2024-09-17T04:31:12.906647+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:37.745557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552564+0000","last_change":"2024-09-17T04:31:11.930818+0000","last_active":"2024-09-17T04:31:13.552564+0000","last_peered":"2024-09-17T04:31:13.552564+0000","last_clean":"2024-09-17T04:31:13.552564+0000","last_became_active":"2024-09-17T04:31:11.930515+0000","last_became_peered":"2024-09-17T04:31:11.930515+0000","last_unstale":"2024-09-17T04:31:13.552564+0000","last_undegraded":"2024-09-17T04:31:13.552564+0000","last_fullsized":"2024-09-17T04:31:13.552564+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:58:29.490878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552485+0000","last_change":"2024-09-17T04:31:10.920901+0000","last_active":"2024-09-17T04:31:13.552485+0000","last_peered":"2024-09-17T04:31:13.552485+0000","last_clean":"2024-09-17T04:31:13.552485+0000","last_became_active":"2024-09-17T04:31:10.920586+0000","last_became_peered":"2024-09-17T04:31:10.920586+0000","last_unstale":"2024-09-17T04:31:13.552485+0000","last_undegraded":"2024-09-17T04:31:13.552485+0000","last_fullsized":"2024-09-17T04:31:13.552485+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:46.164473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420618+0000","last_change":"2024-09-17T04:31:11.940323+0000","last_active":"2024-09-17T04:31:14.420618+0000","last_peered":"2024-09-17T04:31:14.420618+0000","last_clean":"2024-09-17T04:31:14.420618+0000","last_became_active":"2024-09-17T04:31:11.934452+0000","last_became_peered":"2024-09-17T04:31:11.934452+0000","last_unstale":"2024-09-17T04:31:14.420618+0000","last_undegraded":"2024-09-17T04:31:14.420618+0000","last_fullsized":"2024-09-17T04:31:14.420618+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:55:12.099316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"118'8","reported_seq":23,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420821+0000","last_change":"2024-09-17T04:31:10.924812+0000","last_active":"2024-09-17T04:31:14.420821+0000","last_peered":"2024-09-17T04:31:14.420821+0000","last_clean":"2024-09-17T04:31:14.420821+0000","last_became_active":"2024-09-17T04:31:10.924423+0000","last_became_peered":"2024-09-17T04:31:10.924423+0000","last_unstale":"2024-09-17T04:31:14.420821+0000","last_undegraded":"2024-09-17T04:31:14.420821+0000","last_fullsized":"2024-09-17T04:31:14.420821+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:15:57.956399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420978+0000","last_change":"2024-09-17T04:31:11.937551+0000","last_active":"2024-09-17T04:31:14.420978+0000","last_peered":"2024-09-17T04:31:14.420978+0000","last_clean":"2024-09-17T04:31:14.420978+0000","last_became_active":"2024-09-17T04:31:11.937225+0000","last_became_peered":"2024-09-17T04:31:11.937225+0000","last_unstale":"2024-09-17T04:31:14.420978+0000","last_undegraded":"2024-09-17T04:31:14.420978+0000","last_fullsized":"2024-09-17T04:31:14.420978+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:47.220402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"118'1","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.240336+0000","last_change":"2024-09-17T04:31:10.924255+0000","last_active":"2024-09-17T04:31:13.240336+0000","last_peered":"2024-09-17T04:31:13.240336+0000","last_clean":"2024-09-17T04:31:13.240336+0000","last_became_active":"2024-09-17T04:31:10.923887+0000","last_became_peered":"2024-09-17T04:31:10.923887+0000","last_unstale":"2024-09-17T04:31:13.240336+0000","last_undegraded":"2024-09-17T04:31:13.240336+0000","last_fullsized":"2024-09-17T04:31:13.240336+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:10.663359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.240197+0000","last_change":"2024-09-17T04:31:11.925272+0000","last_active":"2024-09-17T04:31:13.240197+0000","last_peered":"2024-09-17T04:31:13.240197+0000","last_clean":"2024-09-17T04:31:13.240197+0000","last_became_active":"2024-09-17T04:31:11.922597+0000","last_became_peered":"2024-09-17T04:31:11.922597+0000","last_unstale":"2024-09-17T04:31:13.240197+0000","last_undegraded":"2024-09-17T04:31:13.240197+0000","last_fullsized":"2024-09-17T04:31:13.240197+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:02.192964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907801+0000","last_change":"2024-09-17T04:31:10.930022+0000","last_active":"2024-09-17T04:31:12.907801+0000","last_peered":"2024-09-17T04:31:12.907801+0000","last_clean":"2024-09-17T04:31:12.907801+0000","last_became_active":"2024-09-17T04:31:10.929681+0000","last_became_peered":"2024-09-17T04:31:10.929681+0000","last_unstale":"2024-09-17T04:31:12.907801+0000","last_undegraded":"2024-09-17T04:31:12.907801+0000","last_fullsized":"2024-09-17T04:31:12.907801+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:56:34.444783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420034+0000","last_change":"2024-09-17T04:31:11.913668+0000","last_active":"2024-09-17T04:31:14.420034+0000","last_peered":"2024-09-17T04:31:14.420034+0000","last_clean":"2024-09-17T04:31:14.420034+0000","last_became_active":"2024-09-17T04:31:11.913365+0000","last_became_peered":"2024-09-17T04:31:11.913365+0000","last_unstale":"2024-09-17T04:31:14.420034+0000","last_undegraded":"2024-09-17T04:31:14.420034+0000","last_fullsized":"2024-09-17T04:31:14.420034+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:26.062554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"118'1","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907988+0000","last_change":"2024-09-17T04:31:10.910443+0000","last_active":"2024-09-17T04:31:12.907988+0000","last_peered":"2024-09-17T04:31:12.907988+0000","last_clean":"2024-09-17T04:31:12.907988+0000","last_became_active":"2024-09-17T04:31:10.910084+0000","last_became_peered":"2024-09-17T04:31:10.910084+0000","last_unstale":"2024-09-17T04:31:12.907988+0000","last_undegraded":"2024-09-17T04:31:12.907988+0000","last_fullsized":"2024-09-17T04:31:12.907988+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:23:55.674833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552564+0000","last_change":"2024-09-17T04:31:11.927237+0000","last_active":"2024-09-17T04:31:13.552564+0000","last_peered":"2024-09-17T04:31:13.552564+0000","last_clean":"2024-09-17T04:31:13.552564+0000","last_became_active":"2024-09-17T04:31:11.926932+0000","last_became_peered":"2024-09-17T04:31:11.926932+0000","last_unstale":"2024-09-17T04:31:13.552564+0000","last_undegraded":"2024-09-17T04:31:13.552564+0000","last_fullsized":"2024-09-17T04:31:13.552564+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:00:58.767864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.239894+0000","last_change":"2024-09-17T04:31:10.918599+0000","last_active":"2024-09-17T04:31:13.239894+0000","last_peered":"2024-09-17T04:31:13.239894+0000","last_clean":"2024-09-17T04:31:13.239894+0000","last_became_active":"2024-09-17T04:31:10.918208+0000","last_became_peered":"2024-09-17T04:31:10.918208+0000","last_unstale":"2024-09-17T04:31:13.239894+0000","last_undegraded":"2024-09-17T04:31:13.239894+0000","last_fullsized":"2024-09-17T04:31:13.239894+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:02:45.443822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.906485+0000","last_change":"2024-09-17T04:31:11.914162+0000","last_active":"2024-09-17T04:31:12.906485+0000","last_peered":"2024-09-17T04:31:12.906485+0000","last_clean":"2024-09-17T04:31:12.906485+0000","last_became_active":"2024-09-17T04:31:11.913790+0000","last_became_peered":"2024-09-17T04:31:11.913790+0000","last_unstale":"2024-09-17T04:31:12.906485+0000","last_undegraded":"2024-09-17T04:31:12.906485+0000","last_fullsized":"2024-09-17T04:31:12.906485+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:48.924309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.239879+0000","last_change":"2024-09-17T04:31:10.927592+0000","last_active":"2024-09-17T04:31:13.239879+0000","last_peered":"2024-09-17T04:31:13.239879+0000","last_clean":"2024-09-17T04:31:13.239879+0000","last_became_active":"2024-09-17T04:31:10.926639+0000","last_became_peered":"2024-09-17T04:31:10.926639+0000","last_unstale":"2024-09-17T04:31:13.239879+0000","last_undegraded":"2024-09-17T04:31:13.239879+0000","last_fullsized":"2024-09-17T04:31:13.239879+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:48.244439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.908066+0000","last_change":"2024-09-17T04:31:11.930701+0000","last_active":"2024-09-17T04:31:12.908066+0000","last_peered":"2024-09-17T04:31:12.908066+0000","last_clean":"2024-09-17T04:31:12.908066+0000","last_became_active":"2024-09-17T04:31:11.930311+0000","last_became_peered":"2024-09-17T04:31:11.930311+0000","last_unstale":"2024-09-17T04:31:12.908066+0000","last_undegraded":"2024-09-17T04:31:12.908066+0000","last_fullsized":"2024-09-17T04:31:12.908066+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:47.972740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.419760+0000","last_change":"2024-09-17T04:31:10.926073+0000","last_active":"2024-09-17T04:31:14.419760+0000","last_peered":"2024-09-17T04:31:14.419760+0000","last_clean":"2024-09-17T04:31:14.419760+0000","last_became_active":"2024-09-17T04:31:10.925750+0000","last_became_peered":"2024-09-17T04:31:10.925750+0000","last_unstale":"2024-09-17T04:31:14.419760+0000","last_undegraded":"2024-09-17T04:31:14.419760+0000","last_fullsized":"2024-09-17T04:31:14.419760+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:44:44.167314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552269+0000","last_change":"2024-09-17T04:31:11.924135+0000","last_active":"2024-09-17T04:31:13.552269+0000","last_peered":"2024-09-17T04:31:13.552269+0000","last_clean":"2024-09-17T04:31:13.552269+0000","last_became_active":"2024-09-17T04:31:11.923823+0000","last_became_peered":"2024-09-17T04:31:11.923823+0000","last_unstale":"2024-09-17T04:31:13.552269+0000","last_undegraded":"2024-09-17T04:31:13.552269+0000","last_fullsized":"2024-09-17T04:31:13.552269+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:38:45.434591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.419613+0000","last_change":"2024-09-17T04:31:10.917223+0000","last_active":"2024-09-17T04:31:14.419613+0000","last_peered":"2024-09-17T04:31:14.419613+0000","last_clean":"2024-09-17T04:31:14.419613+0000","last_became_active":"2024-09-17T04:31:10.916765+0000","last_became_peered":"2024-09-17T04:31:10.916765+0000","last_unstale":"2024-09-17T04:31:14.419613+0000","last_undegraded":"2024-09-17T04:31:14.419613+0000","last_fullsized":"2024-09-17T04:31:14.419613+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:59:40.427551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.925664+0000","last_change":"2024-09-17T04:31:11.937920+0000","last_active":"2024-09-17T04:31:12.925664+0000","last_peered":"2024-09-17T04:31:12.925664+0000","last_clean":"2024-09-17T04:31:12.925664+0000","last_became_active":"2024-09-17T04:31:11.935350+0000","last_became_peered":"2024-09-17T04:31:11.935350+0000","last_unstale":"2024-09-17T04:31:12.925664+0000","last_undegraded":"2024-09-17T04:31:12.925664+0000","last_fullsized":"2024-09-17T04:31:12.925664+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:11.764380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907098+0000","last_change":"2024-09-17T04:31:10.926100+0000","last_active":"2024-09-17T04:31:12.907098+0000","last_peered":"2024-09-17T04:31:12.907098+0000","last_clean":"2024-09-17T04:31:12.907098+0000","last_became_active":"2024-09-17T04:31:10.925806+0000","last_became_peered":"2024-09-17T04:31:10.925806+0000","last_unstale":"2024-09-17T04:31:12.907098+0000","last_undegraded":"2024-09-17T04:31:12.907098+0000","last_fullsized":"2024-09-17T04:31:12.907098+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:34.222663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.908486+0000","last_change":"2024-09-17T04:31:11.919463+0000","last_active":"2024-09-17T04:31:12.908486+0000","last_peered":"2024-09-17T04:31:12.908486+0000","last_clean":"2024-09-17T04:31:12.908486+0000","last_became_active":"2024-09-17T04:31:11.919149+0000","last_became_peered":"2024-09-17T04:31:11.919149+0000","last_unstale":"2024-09-17T04:31:12.908486+0000","last_undegraded":"2024-09-17T04:31:12.908486+0000","last_fullsized":"2024-09-17T04:31:12.908486+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:45:26.764073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.906999+0000","last_change":"2024-09-17T04:31:10.919421+0000","last_active":"2024-09-17T04:31:12.906999+0000","last_peered":"2024-09-17T04:31:12.906999+0000","last_clean":"2024-09-17T04:31:12.906999+0000","last_became_active":"2024-09-17T04:31:10.919126+0000","last_became_peered":"2024-09-17T04:31:10.919126+0000","last_unstale":"2024-09-17T04:31:12.906999+0000","last_undegraded":"2024-09-17T04:31:12.906999+0000","last_fullsized":"2024-09-17T04:31:12.906999+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:12.002189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.906828+0000","last_change":"2024-09-17T04:31:11.932606+0000","last_active":"2024-09-17T04:31:12.906828+0000","last_peered":"2024-09-17T04:31:12.906828+0000","last_clean":"2024-09-17T04:31:12.906828+0000","last_became_active":"2024-09-17T04:31:11.931658+0000","last_became_peered":"2024-09-17T04:31:11.931658+0000","last_unstale":"2024-09-17T04:31:12.906828+0000","last_undegraded":"2024-09-17T04:31:12.906828+0000","last_fullsized":"2024-09-17T04:31:12.906828+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:33.903268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"119'1","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.925922+0000","last_change":"2024-09-17T04:31:10.930374+0000","last_active":"2024-09-17T04:31:12.925922+0000","last_peered":"2024-09-17T04:31:12.925922+0000","last_clean":"2024-09-17T04:31:12.925922+0000","last_became_active":"2024-09-17T04:31:10.930043+0000","last_became_peered":"2024-09-17T04:31:10.930043+0000","last_unstale":"2024-09-17T04:31:12.925922+0000","last_undegraded":"2024-09-17T04:31:12.925922+0000","last_fullsized":"2024-09-17T04:31:12.925922+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:23:55.674833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552639+0000","last_change":"2024-09-17T04:31:11.927579+0000","last_active":"2024-09-17T04:31:13.552639+0000","last_peered":"2024-09-17T04:31:13.552639+0000","last_clean":"2024-09-17T04:31:13.552639+0000","last_became_active":"2024-09-17T04:31:11.927251+0000","last_became_peered":"2024-09-17T04:31:11.927251+0000","last_unstale":"2024-09-17T04:31:13.552639+0000","last_undegraded":"2024-09-17T04:31:13.552639+0000","last_fullsized":"2024-09-17T04:31:13.552639+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:13:44.105379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552594+0000","last_change":"2024-09-17T04:31:10.929027+0000","last_active":"2024-09-17T04:31:13.552594+0000","last_peered":"2024-09-17T04:31:13.552594+0000","last_clean":"2024-09-17T04:31:13.552594+0000","last_became_active":"2024-09-17T04:31:10.928767+0000","last_became_peered":"2024-09-17T04:31:10.928767+0000","last_unstale":"2024-09-17T04:31:13.552594+0000","last_undegraded":"2024-09-17T04:31:13.552594+0000","last_fullsized":"2024-09-17T04:31:13.552594+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:55:30.561136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907943+0000","last_change":"2024-09-17T04:31:11.923172+0000","last_active":"2024-09-17T04:31:12.907943+0000","last_peered":"2024-09-17T04:31:12.907943+0000","last_clean":"2024-09-17T04:31:12.907943+0000","last_became_active":"2024-09-17T04:31:11.922622+0000","last_became_peered":"2024-09-17T04:31:11.922622+0000","last_unstale":"2024-09-17T04:31:12.907943+0000","last_undegraded":"2024-09-17T04:31:12.907943+0000","last_fullsized":"2024-09-17T04:31:12.907943+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:11.969541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907869+0000","last_change":"2024-09-17T04:31:10.920453+0000","last_active":"2024-09-17T04:31:12.907869+0000","last_peered":"2024-09-17T04:31:12.907869+0000","last_clean":"2024-09-17T04:31:12.907869+0000","last_became_active":"2024-09-17T04:31:10.919363+0000","last_became_peered":"2024-09-17T04:31:10.919363+0000","last_unstale":"2024-09-17T04:31:12.907869+0000","last_undegraded":"2024-09-17T04:31:12.907869+0000","last_fullsized":"2024-09-17T04:31:12.907869+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:42:44.035420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.925768+0000","last_change":"2024-09-17T04:31:11.931401+0000","last_active":"2024-09-17T04:31:12.925768+0000","last_peered":"2024-09-17T04:31:12.925768+0000","last_clean":"2024-09-17T04:31:12.925768+0000","last_became_active":"2024-09-17T04:31:11.931104+0000","last_became_peered":"2024-09-17T04:31:11.931104+0000","last_unstale":"2024-09-17T04:31:12.925768+0000","last_undegraded":"2024-09-17T04:31:12.925768+0000","last_fullsized":"2024-09-17T04:31:12.925768+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:36.616761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552236+0000","last_change":"2024-09-17T04:31:10.923121+0000","last_active":"2024-09-17T04:31:13.552236+0000","last_peered":"2024-09-17T04:31:13.552236+0000","last_clean":"2024-09-17T04:31:13.552236+0000","last_became_active":"2024-09-17T04:31:10.922374+0000","last_became_peered":"2024-09-17T04:31:10.922374+0000","last_unstale":"2024-09-17T04:31:13.552236+0000","last_undegraded":"2024-09-17T04:31:13.552236+0000","last_fullsized":"2024-09-17T04:31:13.552236+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:45:53.003095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.240215+0000","last_change":"2024-09-17T04:31:11.914013+0000","last_active":"2024-09-17T04:31:13.240215+0000","last_peered":"2024-09-17T04:31:13.240215+0000","last_clean":"2024-09-17T04:31:13.240215+0000","last_became_active":"2024-09-17T04:31:11.913628+0000","last_became_peered":"2024-09-17T04:31:11.913628+0000","last_unstale":"2024-09-17T04:31:13.240215+0000","last_undegraded":"2024-09-17T04:31:13.240215+0000","last_fullsized":"2024-09-17T04:31:13.240215+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:48:43.679422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.925957+0000","last_change":"2024-09-17T04:31:10.925280+0000","last_active":"2024-09-17T04:31:12.925957+0000","last_peered":"2024-09-17T04:31:12.925957+0000","last_clean":"2024-09-17T04:31:12.925957+0000","last_became_active":"2024-09-17T04:31:10.924762+0000","last_became_peered":"2024-09-17T04:31:10.924762+0000","last_unstale":"2024-09-17T04:31:12.925957+0000","last_undegraded":"2024-09-17T04:31:12.925957+0000","last_fullsized":"2024-09-17T04:31:12.925957+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:46.091206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.419876+0000","last_change":"2024-09-17T04:31:11.932486+0000","last_active":"2024-09-17T04:31:14.419876+0000","last_peered":"2024-09-17T04:31:14.419876+0000","last_clean":"2024-09-17T04:31:14.419876+0000","last_became_active":"2024-09-17T04:31:11.932167+0000","last_became_peered":"2024-09-17T04:31:11.932167+0000","last_unstale":"2024-09-17T04:31:14.419876+0000","last_undegraded":"2024-09-17T04:31:14.419876+0000","last_fullsized":"2024-09-17T04:31:14.419876+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:16.991781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.419684+0000","last_change":"2024-09-17T04:31:10.926916+0000","last_active":"2024-09-17T04:31:14.419684+0000","last_peered":"2024-09-17T04:31:14.419684+0000","last_clean":"2024-09-17T04:31:14.419684+0000","last_became_active":"2024-09-17T04:31:10.926611+0000","last_became_peered":"2024-09-17T04:31:10.926611+0000","last_unstale":"2024-09-17T04:31:14.419684+0000","last_undegraded":"2024-09-17T04:31:14.419684+0000","last_fullsized":"2024-09-17T04:31:14.419684+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:09.894586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552388+0000","last_change":"2024-09-17T04:31:11.930046+0000","last_active":"2024-09-17T04:31:13.552388+0000","last_peered":"2024-09-17T04:31:13.552388+0000","last_clean":"2024-09-17T04:31:13.552388+0000","last_became_active":"2024-09-17T04:31:11.929718+0000","last_became_peered":"2024-09-17T04:31:11.929718+0000","last_unstale":"2024-09-17T04:31:13.552388+0000","last_undegraded":"2024-09-17T04:31:13.552388+0000","last_fullsized":"2024-09-17T04:31:13.552388+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:11.671135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907160+0000","last_change":"2024-09-17T04:31:10.918838+0000","last_active":"2024-09-17T04:31:12.907160+0000","last_peered":"2024-09-17T04:31:12.907160+0000","last_clean":"2024-09-17T04:31:12.907160+0000","last_became_active":"2024-09-17T04:31:10.918535+0000","last_became_peered":"2024-09-17T04:31:10.918535+0000","last_unstale":"2024-09-17T04:31:12.907160+0000","last_undegraded":"2024-09-17T04:31:12.907160+0000","last_fullsized":"2024-09-17T04:31:12.907160+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:06.468845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.925714+0000","last_change":"2024-09-17T04:31:11.936638+0000","last_active":"2024-09-17T04:31:12.925714+0000","last_peered":"2024-09-17T04:31:12.925714+0000","last_clean":"2024-09-17T04:31:12.925714+0000","last_became_active":"2024-09-17T04:31:11.932931+0000","last_became_peered":"2024-09-17T04:31:11.932931+0000","last_unstale":"2024-09-17T04:31:12.925714+0000","last_undegraded":"2024-09-17T04:31:12.925714+0000","last_fullsized":"2024-09-17T04:31:12.925714+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:11.969541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.926155+0000","last_change":"2024-09-17T04:31:10.927734+0000","last_active":"2024-09-17T04:31:12.926155+0000","last_peered":"2024-09-17T04:31:12.926155+0000","last_clean":"2024-09-17T04:31:12.926155+0000","last_became_active":"2024-09-17T04:31:10.926995+0000","last_became_peered":"2024-09-17T04:31:10.926995+0000","last_unstale":"2024-09-17T04:31:12.926155+0000","last_undegraded":"2024-09-17T04:31:12.926155+0000","last_fullsized":"2024-09-17T04:31:12.926155+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:47.465465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"76'79","reported_seq":317,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.925541+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:31:12.925541+0000","last_peered":"2024-09-17T04:31:12.925541+0000","last_clean":"2024-09-17T04:31:12.925541+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:31:12.925541+0000","last_undegraded":"2024-09-17T04:31:12.925541+0000","last_fullsized":"2024-09-17T04:31:12.925541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476828,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2093056,"data_stored":1240307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476827,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32640,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339200,"statfs":{"total":96636764160,"available":96603340800,"internally_reserved":0,"allocated":1486848,"data_stored":649301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476828,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32640,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339200,"statfs":{"total":96636764160,"available":96603340800,"internally_reserved":0,"allocated":1482752,"data_stored":647801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476828,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32636,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1478656,"data_stored":647759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542235,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32636,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1482752,"data_stored":647849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509531,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32644,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339196,"statfs":{"total":96636764160,"available":96603336704,"internally_reserved":0,"allocated":1482752,"data_stored":648305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509530,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1474560,"data_stored":647689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509530,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2068480,"data_stored":1238057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:31:21.307 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:31:21.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.536+0000 7f7443c00640 1 -- 172.21.15.100:0/3466843169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f743c0ad0e0 msgr2=0x7f743c0ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:21.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.536+0000 7f7443c00640 1 --2- 172.21.15.100:0/3466843169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f743c0ad0e0 0x7f743c0ad4f0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f7424005fd0 tx=0x7f742402a540 comp rx=0 tx=0).stop 2024-09-17T04:31:21.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.536+0000 7f7443c00640 1 -- 172.21.15.100:0/3466843169 shutdown_connections 2024-09-17T04:31:21.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.536+0000 7f7443c00640 1 --2- 172.21.15.100:0/3466843169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f743c0b3550 0x7f743c0b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:21.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.536+0000 7f7443c00640 1 --2- 172.21.15.100:0/3466843169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f743c0ad0e0 0x7f743c0ad4f0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:21.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.536+0000 7f7443c00640 1 -- 172.21.15.100:0/3466843169 >> 172.21.15.100:0/3466843169 conn(0x7f743c06d730 msgr2=0x7f743c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:21.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.536+0000 7f7443c00640 1 -- 172.21.15.100:0/3466843169 shutdown_connections 2024-09-17T04:31:21.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.536+0000 7f7443c00640 1 -- 172.21.15.100:0/3466843169 wait complete. 2024-09-17T04:31:21.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.538+0000 7f7443c00640 1 Processor -- start 2024-09-17T04:31:21.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.539+0000 7f7443c00640 1 -- start start 2024-09-17T04:31:21.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.540+0000 7f7443c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f743c0ad0e0 0x7f743c156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:21.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.540+0000 7f7443c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f743c0b3550 0x7f743c1ec5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:21.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.540+0000 7f7443c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f743c15d760 0x7f743c1f5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:21.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.540+0000 7f7443c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f743c0b4da0 con 0x7f743c15d760 2024-09-17T04:31:21.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.540+0000 7f7443c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f743c0b4c20 con 0x7f743c0ad0e0 2024-09-17T04:31:21.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.540+0000 7f7443c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f743c0b3ea0 con 0x7f743c0b3550 2024-09-17T04:31:21.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.540+0000 7f7441c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f743c15d760 0x7f743c1f5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:21.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.540+0000 7f7441c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f743c15d760 0x7f743c1f5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55514/0 (socket says 172.21.15.100:55514) 2024-09-17T04:31:21.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.540+0000 7f7441c00640 1 -- 172.21.15.100:0/955800022 learned_addr learned my addr 172.21.15.100:0/955800022 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:21.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.541+0000 7f7433e00640 1 --2- 172.21.15.100:0/955800022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f743c0b3550 0x7f743c1ec5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:21.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.541+0000 7f7441200640 1 --2- 172.21.15.100:0/955800022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f743c0ad0e0 0x7f743c156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:21.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.541+0000 7f7441c00640 1 -- 172.21.15.100:0/955800022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f743c0b3550 msgr2=0x7f743c1ec5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:21.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.541+0000 7f7441c00640 1 --2- 172.21.15.100:0/955800022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f743c0b3550 0x7f743c1ec5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:21.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.541+0000 7f7441c00640 1 -- 172.21.15.100:0/955800022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f743c0ad0e0 msgr2=0x7f743c156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:21.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.541+0000 7f7441c00640 1 --2- 172.21.15.100:0/955800022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f743c0ad0e0 0x7f743c156a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:21.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.541+0000 7f7441c00640 1 -- 172.21.15.100:0/955800022 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743c14da00 con 0x7f743c15d760 2024-09-17T04:31:21.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.541+0000 7f7441200640 1 --2- 172.21.15.100:0/955800022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f743c0ad0e0 0x7f743c156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:31:21.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.541+0000 7f7441c00640 1 --2- 172.21.15.100:0/955800022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f743c15d760 0x7f743c1f5b60 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f743400e960 tx=0x7f743400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:21.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.541+0000 7f7439600640 1 -- 172.21.15.100:0/955800022 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7434004d20 con 0x7f743c15d760 2024-09-17T04:31:21.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.541+0000 7f7439600640 1 -- 172.21.15.100:0/955800022 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f743400cb20 con 0x7f743c15d760 2024-09-17T04:31:21.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.541+0000 7f7439600640 1 -- 172.21.15.100:0/955800022 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7434005020 con 0x7f743c15d760 2024-09-17T04:31:21.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.541+0000 7f7443c00640 1 -- 172.21.15.100:0/955800022 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f743c14ce70 con 0x7f743c15d760 2024-09-17T04:31:21.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.541+0000 7f7443c00640 1 -- 172.21.15.100:0/955800022 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f743c14d430 con 0x7f743c15d760 2024-09-17T04:31:21.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.542+0000 7f7443c00640 1 -- 172.21.15.100:0/955800022 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f743c0b4370 con 0x7f743c15d760 2024-09-17T04:31:21.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.544+0000 7f7439600640 1 -- 172.21.15.100:0/955800022 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f74340051c0 con 0x7f743c15d760 2024-09-17T04:31:21.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.545+0000 7f7439600640 1 --2- 172.21.15.100:0/955800022 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7418079010 0x7f741807b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:21.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.545+0000 7f7439600640 1 -- 172.21.15.100:0/955800022 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7f743409bf50 con 0x7f743c15d760 2024-09-17T04:31:21.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.545+0000 7f7441200640 1 --2- 172.21.15.100:0/955800022 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7418079010 0x7f741807b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:21.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.545+0000 7f7441200640 1 --2- 172.21.15.100:0/955800022 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7418079010 0x7f741807b520 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f742c000f60 tx=0x7f742c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:21.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.549+0000 7f7439600640 1 -- 172.21.15.100:0/955800022 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f7434016030 con 0x7f743c15d760 2024-09-17T04:31:21.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.731+0000 7f7443c00640 1 -- 172.21.15.100:0/955800022 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f743c14e480 con 0x7f7418079010 2024-09-17T04:31:21.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.735+0000 7f7439600640 1 -- 172.21.15.100:0/955800022 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252366 (secure 0 0 0) 0x7f743c14e480 con 0x7f7418079010 2024-09-17T04:31:21.738 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:21.739 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:31:21.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.739+0000 7f7443c00640 1 -- 172.21.15.100:0/955800022 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7418079010 msgr2=0x7f741807b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:21.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.739+0000 7f7443c00640 1 --2- 172.21.15.100:0/955800022 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7418079010 0x7f741807b520 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f742c000f60 tx=0x7f742c009040 comp rx=0 tx=0).stop 2024-09-17T04:31:21.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.739+0000 7f7443c00640 1 -- 172.21.15.100:0/955800022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f743c15d760 msgr2=0x7f743c1f5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:21.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.739+0000 7f7443c00640 1 --2- 172.21.15.100:0/955800022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f743c15d760 0x7f743c1f5b60 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f743400e960 tx=0x7f743400ee20 comp rx=0 tx=0).stop 2024-09-17T04:31:21.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.739+0000 7f7443c00640 1 -- 172.21.15.100:0/955800022 shutdown_connections 2024-09-17T04:31:21.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.739+0000 7f7443c00640 1 --2- 172.21.15.100:0/955800022 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7418079010 0x7f741807b520 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:21.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.739+0000 7f7443c00640 1 --2- 172.21.15.100:0/955800022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f743c15d760 0x7f743c1f5b60 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:21.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.739+0000 7f7443c00640 1 --2- 172.21.15.100:0/955800022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f743c0b3550 0x7f743c1ec5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:21.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.739+0000 7f7443c00640 1 --2- 172.21.15.100:0/955800022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f743c0ad0e0 0x7f743c156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:21.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.739+0000 7f7443c00640 1 -- 172.21.15.100:0/955800022 >> 172.21.15.100:0/955800022 conn(0x7f743c06d730 msgr2=0x7f743c15f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:21.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.740+0000 7f7443c00640 1 -- 172.21.15.100:0/955800022 shutdown_connections 2024-09-17T04:31:21.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:21.740+0000 7f7443c00640 1 -- 172.21.15.100:0/955800022 wait complete. 2024-09-17T04:31:21.764 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-09-17T04:31:21.689922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262276,"kb_used_data":12744,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712444,"statfs":{"total":773094113280,"available":772825542656,"internally_reserved":0,"allocated":13049856,"data_stored":6367068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.787410"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.906609+0000","last_change":"2024-09-17T04:31:11.915628+0000","last_active":"2024-09-17T04:31:12.906609+0000","last_peered":"2024-09-17T04:31:12.906609+0000","last_clean":"2024-09-17T04:31:12.906609+0000","last_became_active":"2024-09-17T04:31:11.914741+0000","last_became_peered":"2024-09-17T04:31:11.914741+0000","last_unstale":"2024-09-17T04:31:12.906609+0000","last_undegraded":"2024-09-17T04:31:12.906609+0000","last_fullsized":"2024-09-17T04:31:12.906609+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:06:44.732901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552802+0000","last_change":"2024-09-17T04:31:11.927079+0000","last_active":"2024-09-17T04:31:13.552802+0000","last_peered":"2024-09-17T04:31:13.552802+0000","last_clean":"2024-09-17T04:31:13.552802+0000","last_became_active":"2024-09-17T04:31:11.926754+0000","last_became_peered":"2024-09-17T04:31:11.926754+0000","last_unstale":"2024-09-17T04:31:13.552802+0000","last_undegraded":"2024-09-17T04:31:13.552802+0000","last_fullsized":"2024-09-17T04:31:13.552802+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:43:13.340843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420705+0000","last_change":"2024-09-17T04:31:11.924404+0000","last_active":"2024-09-17T04:31:14.420705+0000","last_peered":"2024-09-17T04:31:14.420705+0000","last_clean":"2024-09-17T04:31:14.420705+0000","last_became_active":"2024-09-17T04:31:11.924123+0000","last_became_peered":"2024-09-17T04:31:11.924123+0000","last_unstale":"2024-09-17T04:31:14.420705+0000","last_undegraded":"2024-09-17T04:31:14.420705+0000","last_fullsized":"2024-09-17T04:31:14.420705+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:31.226598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907370+0000","last_change":"2024-09-17T04:31:11.931535+0000","last_active":"2024-09-17T04:31:12.907370+0000","last_peered":"2024-09-17T04:31:12.907370+0000","last_clean":"2024-09-17T04:31:12.907370+0000","last_became_active":"2024-09-17T04:31:11.930362+0000","last_became_peered":"2024-09-17T04:31:11.930362+0000","last_unstale":"2024-09-17T04:31:12.907370+0000","last_undegraded":"2024-09-17T04:31:12.907370+0000","last_fullsized":"2024-09-17T04:31:12.907370+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:17.964326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.926064+0000","last_change":"2024-09-17T04:31:11.938594+0000","last_active":"2024-09-17T04:31:12.926064+0000","last_peered":"2024-09-17T04:31:12.926064+0000","last_clean":"2024-09-17T04:31:12.926064+0000","last_became_active":"2024-09-17T04:31:11.935855+0000","last_became_peered":"2024-09-17T04:31:11.935855+0000","last_unstale":"2024-09-17T04:31:12.926064+0000","last_undegraded":"2024-09-17T04:31:12.926064+0000","last_fullsized":"2024-09-17T04:31:12.926064+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:48.696810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907944+0000","last_change":"2024-09-17T04:31:11.913994+0000","last_active":"2024-09-17T04:31:12.907944+0000","last_peered":"2024-09-17T04:31:12.907944+0000","last_clean":"2024-09-17T04:31:12.907944+0000","last_became_active":"2024-09-17T04:31:11.913641+0000","last_became_peered":"2024-09-17T04:31:11.913641+0000","last_unstale":"2024-09-17T04:31:12.907944+0000","last_undegraded":"2024-09-17T04:31:12.907944+0000","last_fullsized":"2024-09-17T04:31:12.907944+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:31.426067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907825+0000","last_change":"2024-09-17T04:31:11.931072+0000","last_active":"2024-09-17T04:31:12.907825+0000","last_peered":"2024-09-17T04:31:12.907825+0000","last_clean":"2024-09-17T04:31:12.907825+0000","last_became_active":"2024-09-17T04:31:11.929265+0000","last_became_peered":"2024-09-17T04:31:11.929265+0000","last_unstale":"2024-09-17T04:31:12.907825+0000","last_undegraded":"2024-09-17T04:31:12.907825+0000","last_fullsized":"2024-09-17T04:31:12.907825+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:31.927735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420393+0000","last_change":"2024-09-17T04:31:11.939991+0000","last_active":"2024-09-17T04:31:14.420393+0000","last_peered":"2024-09-17T04:31:14.420393+0000","last_clean":"2024-09-17T04:31:14.420393+0000","last_became_active":"2024-09-17T04:31:11.939519+0000","last_became_peered":"2024-09-17T04:31:11.939519+0000","last_unstale":"2024-09-17T04:31:14.420393+0000","last_undegraded":"2024-09-17T04:31:14.420393+0000","last_fullsized":"2024-09-17T04:31:14.420393+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:53:08.566341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420860+0000","last_change":"2024-09-17T04:31:11.937050+0000","last_active":"2024-09-17T04:31:14.420860+0000","last_peered":"2024-09-17T04:31:14.420860+0000","last_clean":"2024-09-17T04:31:14.420860+0000","last_became_active":"2024-09-17T04:31:11.935171+0000","last_became_peered":"2024-09-17T04:31:11.935171+0000","last_unstale":"2024-09-17T04:31:14.420860+0000","last_undegraded":"2024-09-17T04:31:14.420860+0000","last_fullsized":"2024-09-17T04:31:14.420860+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:19:55.960416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552586+0000","last_change":"2024-09-17T04:31:11.927788+0000","last_active":"2024-09-17T04:31:13.552586+0000","last_peered":"2024-09-17T04:31:13.552586+0000","last_clean":"2024-09-17T04:31:13.552586+0000","last_became_active":"2024-09-17T04:31:11.927474+0000","last_became_peered":"2024-09-17T04:31:11.927474+0000","last_unstale":"2024-09-17T04:31:13.552586+0000","last_undegraded":"2024-09-17T04:31:13.552586+0000","last_fullsized":"2024-09-17T04:31:13.552586+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:44.817656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907978+0000","last_change":"2024-09-17T04:31:11.927761+0000","last_active":"2024-09-17T04:31:12.907978+0000","last_peered":"2024-09-17T04:31:12.907978+0000","last_clean":"2024-09-17T04:31:12.907978+0000","last_became_active":"2024-09-17T04:31:11.927451+0000","last_became_peered":"2024-09-17T04:31:11.927451+0000","last_unstale":"2024-09-17T04:31:12.907978+0000","last_undegraded":"2024-09-17T04:31:12.907978+0000","last_fullsized":"2024-09-17T04:31:12.907978+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:59:30.997201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420696+0000","last_change":"2024-09-17T04:31:11.936942+0000","last_active":"2024-09-17T04:31:14.420696+0000","last_peered":"2024-09-17T04:31:14.420696+0000","last_clean":"2024-09-17T04:31:14.420696+0000","last_became_active":"2024-09-17T04:31:11.935646+0000","last_became_peered":"2024-09-17T04:31:11.935646+0000","last_unstale":"2024-09-17T04:31:14.420696+0000","last_undegraded":"2024-09-17T04:31:14.420696+0000","last_fullsized":"2024-09-17T04:31:14.420696+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:17:35.366008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907590+0000","last_change":"2024-09-17T04:31:11.916650+0000","last_active":"2024-09-17T04:31:12.907590+0000","last_peered":"2024-09-17T04:31:12.907590+0000","last_clean":"2024-09-17T04:31:12.907590+0000","last_became_active":"2024-09-17T04:31:11.916136+0000","last_became_peered":"2024-09-17T04:31:11.916136+0000","last_unstale":"2024-09-17T04:31:12.907590+0000","last_undegraded":"2024-09-17T04:31:12.907590+0000","last_fullsized":"2024-09-17T04:31:12.907590+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:45.053959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552430+0000","last_change":"2024-09-17T04:31:11.933739+0000","last_active":"2024-09-17T04:31:13.552430+0000","last_peered":"2024-09-17T04:31:13.552430+0000","last_clean":"2024-09-17T04:31:13.552430+0000","last_became_active":"2024-09-17T04:31:11.932707+0000","last_became_peered":"2024-09-17T04:31:11.932707+0000","last_unstale":"2024-09-17T04:31:13.552430+0000","last_undegraded":"2024-09-17T04:31:13.552430+0000","last_fullsized":"2024-09-17T04:31:13.552430+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:43.901633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.906214+0000","last_change":"2024-09-17T04:31:11.932007+0000","last_active":"2024-09-17T04:31:12.906214+0000","last_peered":"2024-09-17T04:31:12.906214+0000","last_clean":"2024-09-17T04:31:12.906214+0000","last_became_active":"2024-09-17T04:31:11.931790+0000","last_became_peered":"2024-09-17T04:31:11.931790+0000","last_unstale":"2024-09-17T04:31:12.906214+0000","last_undegraded":"2024-09-17T04:31:12.906214+0000","last_fullsized":"2024-09-17T04:31:12.906214+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:12.445854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.419887+0000","last_change":"2024-09-17T04:31:11.910629+0000","last_active":"2024-09-17T04:31:14.419887+0000","last_peered":"2024-09-17T04:31:14.419887+0000","last_clean":"2024-09-17T04:31:14.419887+0000","last_became_active":"2024-09-17T04:31:11.910301+0000","last_became_peered":"2024-09-17T04:31:11.910301+0000","last_unstale":"2024-09-17T04:31:14.419887+0000","last_undegraded":"2024-09-17T04:31:14.419887+0000","last_fullsized":"2024-09-17T04:31:14.419887+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:57.039960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.908274+0000","last_change":"2024-09-17T04:31:11.927492+0000","last_active":"2024-09-17T04:31:12.908274+0000","last_peered":"2024-09-17T04:31:12.908274+0000","last_clean":"2024-09-17T04:31:12.908274+0000","last_became_active":"2024-09-17T04:31:11.925763+0000","last_became_peered":"2024-09-17T04:31:11.925763+0000","last_unstale":"2024-09-17T04:31:12.908274+0000","last_undegraded":"2024-09-17T04:31:12.908274+0000","last_fullsized":"2024-09-17T04:31:12.908274+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:34.292501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420047+0000","last_change":"2024-09-17T04:31:11.927839+0000","last_active":"2024-09-17T04:31:14.420047+0000","last_peered":"2024-09-17T04:31:14.420047+0000","last_clean":"2024-09-17T04:31:14.420047+0000","last_became_active":"2024-09-17T04:31:11.927504+0000","last_became_peered":"2024-09-17T04:31:11.927504+0000","last_unstale":"2024-09-17T04:31:14.420047+0000","last_undegraded":"2024-09-17T04:31:14.420047+0000","last_fullsized":"2024-09-17T04:31:14.420047+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:52:32.891281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907876+0000","last_change":"2024-09-17T04:31:11.927367+0000","last_active":"2024-09-17T04:31:12.907876+0000","last_peered":"2024-09-17T04:31:12.907876+0000","last_clean":"2024-09-17T04:31:12.907876+0000","last_became_active":"2024-09-17T04:31:11.926983+0000","last_became_peered":"2024-09-17T04:31:11.926983+0000","last_unstale":"2024-09-17T04:31:12.907876+0000","last_undegraded":"2024-09-17T04:31:12.907876+0000","last_fullsized":"2024-09-17T04:31:12.907876+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:43.351066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.419974+0000","last_change":"2024-09-17T04:31:11.932828+0000","last_active":"2024-09-17T04:31:14.419974+0000","last_peered":"2024-09-17T04:31:14.419974+0000","last_clean":"2024-09-17T04:31:14.419974+0000","last_became_active":"2024-09-17T04:31:11.925915+0000","last_became_peered":"2024-09-17T04:31:11.925915+0000","last_unstale":"2024-09-17T04:31:14.419974+0000","last_undegraded":"2024-09-17T04:31:14.419974+0000","last_fullsized":"2024-09-17T04:31:14.419974+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:42.403746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.925760+0000","last_change":"2024-09-17T04:31:11.929745+0000","last_active":"2024-09-17T04:31:12.925760+0000","last_peered":"2024-09-17T04:31:12.925760+0000","last_clean":"2024-09-17T04:31:12.925760+0000","last_became_active":"2024-09-17T04:31:11.929386+0000","last_became_peered":"2024-09-17T04:31:11.929386+0000","last_unstale":"2024-09-17T04:31:12.925760+0000","last_undegraded":"2024-09-17T04:31:12.925760+0000","last_fullsized":"2024-09-17T04:31:12.925760+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:25.555379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907083+0000","last_change":"2024-09-17T04:31:11.933052+0000","last_active":"2024-09-17T04:31:12.907083+0000","last_peered":"2024-09-17T04:31:12.907083+0000","last_clean":"2024-09-17T04:31:12.907083+0000","last_became_active":"2024-09-17T04:31:11.932721+0000","last_became_peered":"2024-09-17T04:31:11.932721+0000","last_unstale":"2024-09-17T04:31:12.907083+0000","last_undegraded":"2024-09-17T04:31:12.907083+0000","last_fullsized":"2024-09-17T04:31:12.907083+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:47:42.414199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420662+0000","last_change":"2024-09-17T04:31:11.940132+0000","last_active":"2024-09-17T04:31:14.420662+0000","last_peered":"2024-09-17T04:31:14.420662+0000","last_clean":"2024-09-17T04:31:14.420662+0000","last_became_active":"2024-09-17T04:31:11.934036+0000","last_became_peered":"2024-09-17T04:31:11.934036+0000","last_unstale":"2024-09-17T04:31:14.420662+0000","last_undegraded":"2024-09-17T04:31:14.420662+0000","last_fullsized":"2024-09-17T04:31:14.420662+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:09.888729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.906917+0000","last_change":"2024-09-17T04:31:11.921074+0000","last_active":"2024-09-17T04:31:12.906917+0000","last_peered":"2024-09-17T04:31:12.906917+0000","last_clean":"2024-09-17T04:31:12.906917+0000","last_became_active":"2024-09-17T04:31:11.920773+0000","last_became_peered":"2024-09-17T04:31:11.920773+0000","last_unstale":"2024-09-17T04:31:12.906917+0000","last_undegraded":"2024-09-17T04:31:12.906917+0000","last_fullsized":"2024-09-17T04:31:12.906917+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:01.695666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552713+0000","last_change":"2024-09-17T04:31:11.930143+0000","last_active":"2024-09-17T04:31:13.552713+0000","last_peered":"2024-09-17T04:31:13.552713+0000","last_clean":"2024-09-17T04:31:13.552713+0000","last_became_active":"2024-09-17T04:31:11.929693+0000","last_became_peered":"2024-09-17T04:31:11.929693+0000","last_unstale":"2024-09-17T04:31:13.552713+0000","last_undegraded":"2024-09-17T04:31:13.552713+0000","last_fullsized":"2024-09-17T04:31:13.552713+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:56:14.273325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420204+0000","last_change":"2024-09-17T04:31:11.911729+0000","last_active":"2024-09-17T04:31:14.420204+0000","last_peered":"2024-09-17T04:31:14.420204+0000","last_clean":"2024-09-17T04:31:14.420204+0000","last_became_active":"2024-09-17T04:31:11.911437+0000","last_became_peered":"2024-09-17T04:31:11.911437+0000","last_unstale":"2024-09-17T04:31:14.420204+0000","last_undegraded":"2024-09-17T04:31:14.420204+0000","last_fullsized":"2024-09-17T04:31:14.420204+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:30:58.089743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.906605+0000","last_change":"2024-09-17T04:31:11.914117+0000","last_active":"2024-09-17T04:31:12.906605+0000","last_peered":"2024-09-17T04:31:12.906605+0000","last_clean":"2024-09-17T04:31:12.906605+0000","last_became_active":"2024-09-17T04:31:11.913390+0000","last_became_peered":"2024-09-17T04:31:11.913390+0000","last_unstale":"2024-09-17T04:31:12.906605+0000","last_undegraded":"2024-09-17T04:31:12.906605+0000","last_fullsized":"2024-09-17T04:31:12.906605+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:23:57.520477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.239989+0000","last_change":"2024-09-17T04:31:11.929489+0000","last_active":"2024-09-17T04:31:13.239989+0000","last_peered":"2024-09-17T04:31:13.239989+0000","last_clean":"2024-09-17T04:31:13.239989+0000","last_became_active":"2024-09-17T04:31:11.929161+0000","last_became_peered":"2024-09-17T04:31:11.929161+0000","last_unstale":"2024-09-17T04:31:13.239989+0000","last_undegraded":"2024-09-17T04:31:13.239989+0000","last_fullsized":"2024-09-17T04:31:13.239989+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:32.213268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.906409+0000","last_change":"2024-09-17T04:31:11.935751+0000","last_active":"2024-09-17T04:31:12.906409+0000","last_peered":"2024-09-17T04:31:12.906409+0000","last_clean":"2024-09-17T04:31:12.906409+0000","last_became_active":"2024-09-17T04:31:11.934477+0000","last_became_peered":"2024-09-17T04:31:11.934477+0000","last_unstale":"2024-09-17T04:31:12.906409+0000","last_undegraded":"2024-09-17T04:31:12.906409+0000","last_fullsized":"2024-09-17T04:31:12.906409+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:32.238876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.906247+0000","last_change":"2024-09-17T04:31:11.934997+0000","last_active":"2024-09-17T04:31:12.906247+0000","last_peered":"2024-09-17T04:31:12.906247+0000","last_clean":"2024-09-17T04:31:12.906247+0000","last_became_active":"2024-09-17T04:31:11.931566+0000","last_became_peered":"2024-09-17T04:31:11.931566+0000","last_unstale":"2024-09-17T04:31:12.906247+0000","last_undegraded":"2024-09-17T04:31:12.906247+0000","last_fullsized":"2024-09-17T04:31:12.906247+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:33:16.150618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.925823+0000","last_change":"2024-09-17T04:31:11.937985+0000","last_active":"2024-09-17T04:31:12.925823+0000","last_peered":"2024-09-17T04:31:12.925823+0000","last_clean":"2024-09-17T04:31:12.925823+0000","last_became_active":"2024-09-17T04:31:11.934427+0000","last_became_peered":"2024-09-17T04:31:11.934427+0000","last_unstale":"2024-09-17T04:31:12.925823+0000","last_undegraded":"2024-09-17T04:31:12.925823+0000","last_fullsized":"2024-09-17T04:31:12.925823+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:17.218054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907697+0000","last_change":"2024-09-17T04:31:11.914266+0000","last_active":"2024-09-17T04:31:12.907697+0000","last_peered":"2024-09-17T04:31:12.907697+0000","last_clean":"2024-09-17T04:31:12.907697+0000","last_became_active":"2024-09-17T04:31:11.913960+0000","last_became_peered":"2024-09-17T04:31:11.913960+0000","last_unstale":"2024-09-17T04:31:12.907697+0000","last_undegraded":"2024-09-17T04:31:12.907697+0000","last_fullsized":"2024-09-17T04:31:12.907697+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:06.248101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420220+0000","last_change":"2024-09-17T04:31:11.923380+0000","last_active":"2024-09-17T04:31:14.420220+0000","last_peered":"2024-09-17T04:31:14.420220+0000","last_clean":"2024-09-17T04:31:14.420220+0000","last_became_active":"2024-09-17T04:31:11.923084+0000","last_became_peered":"2024-09-17T04:31:11.923084+0000","last_unstale":"2024-09-17T04:31:14.420220+0000","last_undegraded":"2024-09-17T04:31:14.420220+0000","last_fullsized":"2024-09-17T04:31:14.420220+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:31.148864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"118'4","reported_seq":19,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.926316+0000","last_change":"2024-09-17T04:31:10.925886+0000","last_active":"2024-09-17T04:31:12.926316+0000","last_peered":"2024-09-17T04:31:12.926316+0000","last_clean":"2024-09-17T04:31:12.926316+0000","last_became_active":"2024-09-17T04:31:10.925455+0000","last_became_peered":"2024-09-17T04:31:10.925455+0000","last_unstale":"2024-09-17T04:31:12.926316+0000","last_undegraded":"2024-09-17T04:31:12.926316+0000","last_fullsized":"2024-09-17T04:31:12.926316+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:40:57.412470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420901+0000","last_change":"2024-09-17T04:31:11.938822+0000","last_active":"2024-09-17T04:31:14.420901+0000","last_peered":"2024-09-17T04:31:14.420901+0000","last_clean":"2024-09-17T04:31:14.420901+0000","last_became_active":"2024-09-17T04:31:11.938412+0000","last_became_peered":"2024-09-17T04:31:11.938412+0000","last_unstale":"2024-09-17T04:31:14.420901+0000","last_undegraded":"2024-09-17T04:31:14.420901+0000","last_fullsized":"2024-09-17T04:31:14.420901+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:10.737466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"118'1","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.240024+0000","last_change":"2024-09-17T04:31:10.923497+0000","last_active":"2024-09-17T04:31:13.240024+0000","last_peered":"2024-09-17T04:31:13.240024+0000","last_clean":"2024-09-17T04:31:13.240024+0000","last_became_active":"2024-09-17T04:31:10.923086+0000","last_became_peered":"2024-09-17T04:31:10.923086+0000","last_unstale":"2024-09-17T04:31:13.240024+0000","last_undegraded":"2024-09-17T04:31:13.240024+0000","last_fullsized":"2024-09-17T04:31:13.240024+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:38:44.426816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.926099+0000","last_change":"2024-09-17T04:31:11.928219+0000","last_active":"2024-09-17T04:31:12.926099+0000","last_peered":"2024-09-17T04:31:12.926099+0000","last_clean":"2024-09-17T04:31:12.926099+0000","last_became_active":"2024-09-17T04:31:11.927796+0000","last_became_peered":"2024-09-17T04:31:11.927796+0000","last_unstale":"2024-09-17T04:31:12.926099+0000","last_undegraded":"2024-09-17T04:31:12.926099+0000","last_fullsized":"2024-09-17T04:31:12.926099+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:25.762200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.908056+0000","last_change":"2024-09-17T04:31:10.922177+0000","last_active":"2024-09-17T04:31:12.908056+0000","last_peered":"2024-09-17T04:31:12.908056+0000","last_clean":"2024-09-17T04:31:12.908056+0000","last_became_active":"2024-09-17T04:31:10.921875+0000","last_became_peered":"2024-09-17T04:31:10.921875+0000","last_unstale":"2024-09-17T04:31:12.908056+0000","last_undegraded":"2024-09-17T04:31:12.908056+0000","last_fullsized":"2024-09-17T04:31:12.908056+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:33:18.323563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.926082+0000","last_change":"2024-09-17T04:31:11.935286+0000","last_active":"2024-09-17T04:31:12.926082+0000","last_peered":"2024-09-17T04:31:12.926082+0000","last_clean":"2024-09-17T04:31:12.926082+0000","last_became_active":"2024-09-17T04:31:11.933218+0000","last_became_peered":"2024-09-17T04:31:11.933218+0000","last_unstale":"2024-09-17T04:31:12.926082+0000","last_undegraded":"2024-09-17T04:31:12.926082+0000","last_fullsized":"2024-09-17T04:31:12.926082+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:12.070463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552714+0000","last_change":"2024-09-17T04:31:10.917285+0000","last_active":"2024-09-17T04:31:13.552714+0000","last_peered":"2024-09-17T04:31:13.552714+0000","last_clean":"2024-09-17T04:31:13.552714+0000","last_became_active":"2024-09-17T04:31:10.916739+0000","last_became_peered":"2024-09-17T04:31:10.916739+0000","last_unstale":"2024-09-17T04:31:13.552714+0000","last_undegraded":"2024-09-17T04:31:13.552714+0000","last_fullsized":"2024-09-17T04:31:13.552714+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:06:32.134623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.419694+0000","last_change":"2024-09-17T04:31:11.914279+0000","last_active":"2024-09-17T04:31:14.419694+0000","last_peered":"2024-09-17T04:31:14.419694+0000","last_clean":"2024-09-17T04:31:14.419694+0000","last_became_active":"2024-09-17T04:31:11.913971+0000","last_became_peered":"2024-09-17T04:31:11.913971+0000","last_unstale":"2024-09-17T04:31:14.419694+0000","last_undegraded":"2024-09-17T04:31:14.419694+0000","last_fullsized":"2024-09-17T04:31:14.419694+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:42:34.939903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"118'1","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552208+0000","last_change":"2024-09-17T04:31:10.913568+0000","last_active":"2024-09-17T04:31:13.552208+0000","last_peered":"2024-09-17T04:31:13.552208+0000","last_clean":"2024-09-17T04:31:13.552208+0000","last_became_active":"2024-09-17T04:31:10.913273+0000","last_became_peered":"2024-09-17T04:31:10.913273+0000","last_unstale":"2024-09-17T04:31:13.552208+0000","last_undegraded":"2024-09-17T04:31:13.552208+0000","last_fullsized":"2024-09-17T04:31:13.552208+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:45.104445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907798+0000","last_change":"2024-09-17T04:31:11.927550+0000","last_active":"2024-09-17T04:31:12.907798+0000","last_peered":"2024-09-17T04:31:12.907798+0000","last_clean":"2024-09-17T04:31:12.907798+0000","last_became_active":"2024-09-17T04:31:11.926417+0000","last_became_peered":"2024-09-17T04:31:11.926417+0000","last_unstale":"2024-09-17T04:31:12.907798+0000","last_undegraded":"2024-09-17T04:31:12.907798+0000","last_fullsized":"2024-09-17T04:31:12.907798+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:13.917612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"118'1","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.419614+0000","last_change":"2024-09-17T04:31:10.917769+0000","last_active":"2024-09-17T04:31:14.419614+0000","last_peered":"2024-09-17T04:31:14.419614+0000","last_clean":"2024-09-17T04:31:14.419614+0000","last_became_active":"2024-09-17T04:31:10.917493+0000","last_became_peered":"2024-09-17T04:31:10.917493+0000","last_unstale":"2024-09-17T04:31:14.419614+0000","last_undegraded":"2024-09-17T04:31:14.419614+0000","last_fullsized":"2024-09-17T04:31:14.419614+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:53.541655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420445+0000","last_change":"2024-09-17T04:31:11.926740+0000","last_active":"2024-09-17T04:31:14.420445+0000","last_peered":"2024-09-17T04:31:14.420445+0000","last_clean":"2024-09-17T04:31:14.420445+0000","last_became_active":"2024-09-17T04:31:11.926396+0000","last_became_peered":"2024-09-17T04:31:11.926396+0000","last_unstale":"2024-09-17T04:31:14.420445+0000","last_undegraded":"2024-09-17T04:31:14.420445+0000","last_fullsized":"2024-09-17T04:31:14.420445+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:42.899760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907484+0000","last_change":"2024-09-17T04:31:10.919927+0000","last_active":"2024-09-17T04:31:12.907484+0000","last_peered":"2024-09-17T04:31:12.907484+0000","last_clean":"2024-09-17T04:31:12.907484+0000","last_became_active":"2024-09-17T04:31:10.919536+0000","last_became_peered":"2024-09-17T04:31:10.919536+0000","last_unstale":"2024-09-17T04:31:12.907484+0000","last_undegraded":"2024-09-17T04:31:12.907484+0000","last_fullsized":"2024-09-17T04:31:12.907484+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:47.465465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552277+0000","last_change":"2024-09-17T04:31:11.924910+0000","last_active":"2024-09-17T04:31:13.552277+0000","last_peered":"2024-09-17T04:31:13.552277+0000","last_clean":"2024-09-17T04:31:13.552277+0000","last_became_active":"2024-09-17T04:31:11.924604+0000","last_became_peered":"2024-09-17T04:31:11.924604+0000","last_unstale":"2024-09-17T04:31:13.552277+0000","last_undegraded":"2024-09-17T04:31:13.552277+0000","last_fullsized":"2024-09-17T04:31:13.552277+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:34.516657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552651+0000","last_change":"2024-09-17T04:31:10.926647+0000","last_active":"2024-09-17T04:31:13.552651+0000","last_peered":"2024-09-17T04:31:13.552651+0000","last_clean":"2024-09-17T04:31:13.552651+0000","last_became_active":"2024-09-17T04:31:10.926287+0000","last_became_peered":"2024-09-17T04:31:10.926287+0000","last_unstale":"2024-09-17T04:31:13.552651+0000","last_undegraded":"2024-09-17T04:31:13.552651+0000","last_fullsized":"2024-09-17T04:31:13.552651+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:11.279144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907050+0000","last_change":"2024-09-17T04:31:11.933200+0000","last_active":"2024-09-17T04:31:12.907050+0000","last_peered":"2024-09-17T04:31:12.907050+0000","last_clean":"2024-09-17T04:31:12.907050+0000","last_became_active":"2024-09-17T04:31:11.930452+0000","last_became_peered":"2024-09-17T04:31:11.930452+0000","last_unstale":"2024-09-17T04:31:12.907050+0000","last_undegraded":"2024-09-17T04:31:12.907050+0000","last_fullsized":"2024-09-17T04:31:12.907050+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:42:33.591948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.240289+0000","last_change":"2024-09-17T04:31:10.926032+0000","last_active":"2024-09-17T04:31:13.240289+0000","last_peered":"2024-09-17T04:31:13.240289+0000","last_clean":"2024-09-17T04:31:13.240289+0000","last_became_active":"2024-09-17T04:31:10.925736+0000","last_became_peered":"2024-09-17T04:31:10.925736+0000","last_unstale":"2024-09-17T04:31:13.240289+0000","last_undegraded":"2024-09-17T04:31:13.240289+0000","last_fullsized":"2024-09-17T04:31:13.240289+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:33.508882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.240159+0000","last_change":"2024-09-17T04:31:11.929279+0000","last_active":"2024-09-17T04:31:13.240159+0000","last_peered":"2024-09-17T04:31:13.240159+0000","last_clean":"2024-09-17T04:31:13.240159+0000","last_became_active":"2024-09-17T04:31:11.928813+0000","last_became_peered":"2024-09-17T04:31:11.928813+0000","last_unstale":"2024-09-17T04:31:13.240159+0000","last_undegraded":"2024-09-17T04:31:13.240159+0000","last_fullsized":"2024-09-17T04:31:13.240159+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:29.918321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420321+0000","last_change":"2024-09-17T04:31:10.930027+0000","last_active":"2024-09-17T04:31:14.420321+0000","last_peered":"2024-09-17T04:31:14.420321+0000","last_clean":"2024-09-17T04:31:14.420321+0000","last_became_active":"2024-09-17T04:31:10.929648+0000","last_became_peered":"2024-09-17T04:31:10.929648+0000","last_unstale":"2024-09-17T04:31:14.420321+0000","last_undegraded":"2024-09-17T04:31:14.420321+0000","last_fullsized":"2024-09-17T04:31:14.420321+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:46.988963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.926236+0000","last_change":"2024-09-17T04:31:11.937498+0000","last_active":"2024-09-17T04:31:12.926236+0000","last_peered":"2024-09-17T04:31:12.926236+0000","last_clean":"2024-09-17T04:31:12.926236+0000","last_became_active":"2024-09-17T04:31:11.937168+0000","last_became_peered":"2024-09-17T04:31:11.937168+0000","last_unstale":"2024-09-17T04:31:12.926236+0000","last_undegraded":"2024-09-17T04:31:12.926236+0000","last_fullsized":"2024-09-17T04:31:12.926236+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:53:17.261085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"118'3","reported_seq":18,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.239991+0000","last_change":"2024-09-17T04:31:10.917626+0000","last_active":"2024-09-17T04:31:13.239991+0000","last_peered":"2024-09-17T04:31:13.239991+0000","last_clean":"2024-09-17T04:31:13.239991+0000","last_became_active":"2024-09-17T04:31:10.917239+0000","last_became_peered":"2024-09-17T04:31:10.917239+0000","last_unstale":"2024-09-17T04:31:13.239991+0000","last_undegraded":"2024-09-17T04:31:13.239991+0000","last_fullsized":"2024-09-17T04:31:13.239991+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:49:42.911365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.926467+0000","last_change":"2024-09-17T04:31:11.933050+0000","last_active":"2024-09-17T04:31:12.926467+0000","last_peered":"2024-09-17T04:31:12.926467+0000","last_clean":"2024-09-17T04:31:12.926467+0000","last_became_active":"2024-09-17T04:31:11.932673+0000","last_became_peered":"2024-09-17T04:31:11.932673+0000","last_unstale":"2024-09-17T04:31:12.926467+0000","last_undegraded":"2024-09-17T04:31:12.926467+0000","last_fullsized":"2024-09-17T04:31:12.926467+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:20.703359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"118'6","reported_seq":21,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.421089+0000","last_change":"2024-09-17T04:31:10.929556+0000","last_active":"2024-09-17T04:31:14.421089+0000","last_peered":"2024-09-17T04:31:14.421089+0000","last_clean":"2024-09-17T04:31:14.421089+0000","last_became_active":"2024-09-17T04:31:10.929171+0000","last_became_peered":"2024-09-17T04:31:10.929171+0000","last_unstale":"2024-09-17T04:31:14.421089+0000","last_undegraded":"2024-09-17T04:31:14.421089+0000","last_fullsized":"2024-09-17T04:31:14.421089+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:46:00.805710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.906665+0000","last_change":"2024-09-17T04:31:11.922654+0000","last_active":"2024-09-17T04:31:12.906665+0000","last_peered":"2024-09-17T04:31:12.906665+0000","last_clean":"2024-09-17T04:31:12.906665+0000","last_became_active":"2024-09-17T04:31:11.922366+0000","last_became_peered":"2024-09-17T04:31:11.922366+0000","last_unstale":"2024-09-17T04:31:12.906665+0000","last_undegraded":"2024-09-17T04:31:12.906665+0000","last_fullsized":"2024-09-17T04:31:12.906665+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:02.214226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.906647+0000","last_change":"2024-09-17T04:31:10.921170+0000","last_active":"2024-09-17T04:31:12.906647+0000","last_peered":"2024-09-17T04:31:12.906647+0000","last_clean":"2024-09-17T04:31:12.906647+0000","last_became_active":"2024-09-17T04:31:10.920873+0000","last_became_peered":"2024-09-17T04:31:10.920873+0000","last_unstale":"2024-09-17T04:31:12.906647+0000","last_undegraded":"2024-09-17T04:31:12.906647+0000","last_fullsized":"2024-09-17T04:31:12.906647+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:37.745557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552564+0000","last_change":"2024-09-17T04:31:11.930818+0000","last_active":"2024-09-17T04:31:13.552564+0000","last_peered":"2024-09-17T04:31:13.552564+0000","last_clean":"2024-09-17T04:31:13.552564+0000","last_became_active":"2024-09-17T04:31:11.930515+0000","last_became_peered":"2024-09-17T04:31:11.930515+0000","last_unstale":"2024-09-17T04:31:13.552564+0000","last_undegraded":"2024-09-17T04:31:13.552564+0000","last_fullsized":"2024-09-17T04:31:13.552564+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:58:29.490878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552485+0000","last_change":"2024-09-17T04:31:10.920901+0000","last_active":"2024-09-17T04:31:13.552485+0000","last_peered":"2024-09-17T04:31:13.552485+0000","last_clean":"2024-09-17T04:31:13.552485+0000","last_became_active":"2024-09-17T04:31:10.920586+0000","last_became_peered":"2024-09-17T04:31:10.920586+0000","last_unstale":"2024-09-17T04:31:13.552485+0000","last_undegraded":"2024-09-17T04:31:13.552485+0000","last_fullsized":"2024-09-17T04:31:13.552485+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:46.164473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420618+0000","last_change":"2024-09-17T04:31:11.940323+0000","last_active":"2024-09-17T04:31:14.420618+0000","last_peered":"2024-09-17T04:31:14.420618+0000","last_clean":"2024-09-17T04:31:14.420618+0000","last_became_active":"2024-09-17T04:31:11.934452+0000","last_became_peered":"2024-09-17T04:31:11.934452+0000","last_unstale":"2024-09-17T04:31:14.420618+0000","last_undegraded":"2024-09-17T04:31:14.420618+0000","last_fullsized":"2024-09-17T04:31:14.420618+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:55:12.099316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"118'8","reported_seq":23,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420821+0000","last_change":"2024-09-17T04:31:10.924812+0000","last_active":"2024-09-17T04:31:14.420821+0000","last_peered":"2024-09-17T04:31:14.420821+0000","last_clean":"2024-09-17T04:31:14.420821+0000","last_became_active":"2024-09-17T04:31:10.924423+0000","last_became_peered":"2024-09-17T04:31:10.924423+0000","last_unstale":"2024-09-17T04:31:14.420821+0000","last_undegraded":"2024-09-17T04:31:14.420821+0000","last_fullsized":"2024-09-17T04:31:14.420821+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:15:57.956399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420978+0000","last_change":"2024-09-17T04:31:11.937551+0000","last_active":"2024-09-17T04:31:14.420978+0000","last_peered":"2024-09-17T04:31:14.420978+0000","last_clean":"2024-09-17T04:31:14.420978+0000","last_became_active":"2024-09-17T04:31:11.937225+0000","last_became_peered":"2024-09-17T04:31:11.937225+0000","last_unstale":"2024-09-17T04:31:14.420978+0000","last_undegraded":"2024-09-17T04:31:14.420978+0000","last_fullsized":"2024-09-17T04:31:14.420978+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:47.220402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"118'1","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.240336+0000","last_change":"2024-09-17T04:31:10.924255+0000","last_active":"2024-09-17T04:31:13.240336+0000","last_peered":"2024-09-17T04:31:13.240336+0000","last_clean":"2024-09-17T04:31:13.240336+0000","last_became_active":"2024-09-17T04:31:10.923887+0000","last_became_peered":"2024-09-17T04:31:10.923887+0000","last_unstale":"2024-09-17T04:31:13.240336+0000","last_undegraded":"2024-09-17T04:31:13.240336+0000","last_fullsized":"2024-09-17T04:31:13.240336+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:10.663359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.240197+0000","last_change":"2024-09-17T04:31:11.925272+0000","last_active":"2024-09-17T04:31:13.240197+0000","last_peered":"2024-09-17T04:31:13.240197+0000","last_clean":"2024-09-17T04:31:13.240197+0000","last_became_active":"2024-09-17T04:31:11.922597+0000","last_became_peered":"2024-09-17T04:31:11.922597+0000","last_unstale":"2024-09-17T04:31:13.240197+0000","last_undegraded":"2024-09-17T04:31:13.240197+0000","last_fullsized":"2024-09-17T04:31:13.240197+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:02.192964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907801+0000","last_change":"2024-09-17T04:31:10.930022+0000","last_active":"2024-09-17T04:31:12.907801+0000","last_peered":"2024-09-17T04:31:12.907801+0000","last_clean":"2024-09-17T04:31:12.907801+0000","last_became_active":"2024-09-17T04:31:10.929681+0000","last_became_peered":"2024-09-17T04:31:10.929681+0000","last_unstale":"2024-09-17T04:31:12.907801+0000","last_undegraded":"2024-09-17T04:31:12.907801+0000","last_fullsized":"2024-09-17T04:31:12.907801+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:56:34.444783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.420034+0000","last_change":"2024-09-17T04:31:11.913668+0000","last_active":"2024-09-17T04:31:14.420034+0000","last_peered":"2024-09-17T04:31:14.420034+0000","last_clean":"2024-09-17T04:31:14.420034+0000","last_became_active":"2024-09-17T04:31:11.913365+0000","last_became_peered":"2024-09-17T04:31:11.913365+0000","last_unstale":"2024-09-17T04:31:14.420034+0000","last_undegraded":"2024-09-17T04:31:14.420034+0000","last_fullsized":"2024-09-17T04:31:14.420034+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:26.062554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"118'1","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907988+0000","last_change":"2024-09-17T04:31:10.910443+0000","last_active":"2024-09-17T04:31:12.907988+0000","last_peered":"2024-09-17T04:31:12.907988+0000","last_clean":"2024-09-17T04:31:12.907988+0000","last_became_active":"2024-09-17T04:31:10.910084+0000","last_became_peered":"2024-09-17T04:31:10.910084+0000","last_unstale":"2024-09-17T04:31:12.907988+0000","last_undegraded":"2024-09-17T04:31:12.907988+0000","last_fullsized":"2024-09-17T04:31:12.907988+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:23:55.674833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552564+0000","last_change":"2024-09-17T04:31:11.927237+0000","last_active":"2024-09-17T04:31:13.552564+0000","last_peered":"2024-09-17T04:31:13.552564+0000","last_clean":"2024-09-17T04:31:13.552564+0000","last_became_active":"2024-09-17T04:31:11.926932+0000","last_became_peered":"2024-09-17T04:31:11.926932+0000","last_unstale":"2024-09-17T04:31:13.552564+0000","last_undegraded":"2024-09-17T04:31:13.552564+0000","last_fullsized":"2024-09-17T04:31:13.552564+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:00:58.767864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.239894+0000","last_change":"2024-09-17T04:31:10.918599+0000","last_active":"2024-09-17T04:31:13.239894+0000","last_peered":"2024-09-17T04:31:13.239894+0000","last_clean":"2024-09-17T04:31:13.239894+0000","last_became_active":"2024-09-17T04:31:10.918208+0000","last_became_peered":"2024-09-17T04:31:10.918208+0000","last_unstale":"2024-09-17T04:31:13.239894+0000","last_undegraded":"2024-09-17T04:31:13.239894+0000","last_fullsized":"2024-09-17T04:31:13.239894+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:02:45.443822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.906485+0000","last_change":"2024-09-17T04:31:11.914162+0000","last_active":"2024-09-17T04:31:12.906485+0000","last_peered":"2024-09-17T04:31:12.906485+0000","last_clean":"2024-09-17T04:31:12.906485+0000","last_became_active":"2024-09-17T04:31:11.913790+0000","last_became_peered":"2024-09-17T04:31:11.913790+0000","last_unstale":"2024-09-17T04:31:12.906485+0000","last_undegraded":"2024-09-17T04:31:12.906485+0000","last_fullsized":"2024-09-17T04:31:12.906485+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:48.924309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.239879+0000","last_change":"2024-09-17T04:31:10.927592+0000","last_active":"2024-09-17T04:31:13.239879+0000","last_peered":"2024-09-17T04:31:13.239879+0000","last_clean":"2024-09-17T04:31:13.239879+0000","last_became_active":"2024-09-17T04:31:10.926639+0000","last_became_peered":"2024-09-17T04:31:10.926639+0000","last_unstale":"2024-09-17T04:31:13.239879+0000","last_undegraded":"2024-09-17T04:31:13.239879+0000","last_fullsized":"2024-09-17T04:31:13.239879+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:48.244439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.908066+0000","last_change":"2024-09-17T04:31:11.930701+0000","last_active":"2024-09-17T04:31:12.908066+0000","last_peered":"2024-09-17T04:31:12.908066+0000","last_clean":"2024-09-17T04:31:12.908066+0000","last_became_active":"2024-09-17T04:31:11.930311+0000","last_became_peered":"2024-09-17T04:31:11.930311+0000","last_unstale":"2024-09-17T04:31:12.908066+0000","last_undegraded":"2024-09-17T04:31:12.908066+0000","last_fullsized":"2024-09-17T04:31:12.908066+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:47.972740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.419760+0000","last_change":"2024-09-17T04:31:10.926073+0000","last_active":"2024-09-17T04:31:14.419760+0000","last_peered":"2024-09-17T04:31:14.419760+0000","last_clean":"2024-09-17T04:31:14.419760+0000","last_became_active":"2024-09-17T04:31:10.925750+0000","last_became_peered":"2024-09-17T04:31:10.925750+0000","last_unstale":"2024-09-17T04:31:14.419760+0000","last_undegraded":"2024-09-17T04:31:14.419760+0000","last_fullsized":"2024-09-17T04:31:14.419760+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:44:44.167314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552269+0000","last_change":"2024-09-17T04:31:11.924135+0000","last_active":"2024-09-17T04:31:13.552269+0000","last_peered":"2024-09-17T04:31:13.552269+0000","last_clean":"2024-09-17T04:31:13.552269+0000","last_became_active":"2024-09-17T04:31:11.923823+0000","last_became_peered":"2024-09-17T04:31:11.923823+0000","last_unstale":"2024-09-17T04:31:13.552269+0000","last_undegraded":"2024-09-17T04:31:13.552269+0000","last_fullsized":"2024-09-17T04:31:13.552269+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:38:45.434591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.419613+0000","last_change":"2024-09-17T04:31:10.917223+0000","last_active":"2024-09-17T04:31:14.419613+0000","last_peered":"2024-09-17T04:31:14.419613+0000","last_clean":"2024-09-17T04:31:14.419613+0000","last_became_active":"2024-09-17T04:31:10.916765+0000","last_became_peered":"2024-09-17T04:31:10.916765+0000","last_unstale":"2024-09-17T04:31:14.419613+0000","last_undegraded":"2024-09-17T04:31:14.419613+0000","last_fullsized":"2024-09-17T04:31:14.419613+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:59:40.427551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.925664+0000","last_change":"2024-09-17T04:31:11.937920+0000","last_active":"2024-09-17T04:31:12.925664+0000","last_peered":"2024-09-17T04:31:12.925664+0000","last_clean":"2024-09-17T04:31:12.925664+0000","last_became_active":"2024-09-17T04:31:11.935350+0000","last_became_peered":"2024-09-17T04:31:11.935350+0000","last_unstale":"2024-09-17T04:31:12.925664+0000","last_undegraded":"2024-09-17T04:31:12.925664+0000","last_fullsized":"2024-09-17T04:31:12.925664+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:11.764380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907098+0000","last_change":"2024-09-17T04:31:10.926100+0000","last_active":"2024-09-17T04:31:12.907098+0000","last_peered":"2024-09-17T04:31:12.907098+0000","last_clean":"2024-09-17T04:31:12.907098+0000","last_became_active":"2024-09-17T04:31:10.925806+0000","last_became_peered":"2024-09-17T04:31:10.925806+0000","last_unstale":"2024-09-17T04:31:12.907098+0000","last_undegraded":"2024-09-17T04:31:12.907098+0000","last_fullsized":"2024-09-17T04:31:12.907098+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:34.222663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.908486+0000","last_change":"2024-09-17T04:31:11.919463+0000","last_active":"2024-09-17T04:31:12.908486+0000","last_peered":"2024-09-17T04:31:12.908486+0000","last_clean":"2024-09-17T04:31:12.908486+0000","last_became_active":"2024-09-17T04:31:11.919149+0000","last_became_peered":"2024-09-17T04:31:11.919149+0000","last_unstale":"2024-09-17T04:31:12.908486+0000","last_undegraded":"2024-09-17T04:31:12.908486+0000","last_fullsized":"2024-09-17T04:31:12.908486+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:45:26.764073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.906999+0000","last_change":"2024-09-17T04:31:10.919421+0000","last_active":"2024-09-17T04:31:12.906999+0000","last_peered":"2024-09-17T04:31:12.906999+0000","last_clean":"2024-09-17T04:31:12.906999+0000","last_became_active":"2024-09-17T04:31:10.919126+0000","last_became_peered":"2024-09-17T04:31:10.919126+0000","last_unstale":"2024-09-17T04:31:12.906999+0000","last_undegraded":"2024-09-17T04:31:12.906999+0000","last_fullsized":"2024-09-17T04:31:12.906999+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:12.002189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.906828+0000","last_change":"2024-09-17T04:31:11.932606+0000","last_active":"2024-09-17T04:31:12.906828+0000","last_peered":"2024-09-17T04:31:12.906828+0000","last_clean":"2024-09-17T04:31:12.906828+0000","last_became_active":"2024-09-17T04:31:11.931658+0000","last_became_peered":"2024-09-17T04:31:11.931658+0000","last_unstale":"2024-09-17T04:31:12.906828+0000","last_undegraded":"2024-09-17T04:31:12.906828+0000","last_fullsized":"2024-09-17T04:31:12.906828+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:33.903268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"119'1","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.925922+0000","last_change":"2024-09-17T04:31:10.930374+0000","last_active":"2024-09-17T04:31:12.925922+0000","last_peered":"2024-09-17T04:31:12.925922+0000","last_clean":"2024-09-17T04:31:12.925922+0000","last_became_active":"2024-09-17T04:31:10.930043+0000","last_became_peered":"2024-09-17T04:31:10.930043+0000","last_unstale":"2024-09-17T04:31:12.925922+0000","last_undegraded":"2024-09-17T04:31:12.925922+0000","last_fullsized":"2024-09-17T04:31:12.925922+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:23:55.674833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552639+0000","last_change":"2024-09-17T04:31:11.927579+0000","last_active":"2024-09-17T04:31:13.552639+0000","last_peered":"2024-09-17T04:31:13.552639+0000","last_clean":"2024-09-17T04:31:13.552639+0000","last_became_active":"2024-09-17T04:31:11.927251+0000","last_became_peered":"2024-09-17T04:31:11.927251+0000","last_unstale":"2024-09-17T04:31:13.552639+0000","last_undegraded":"2024-09-17T04:31:13.552639+0000","last_fullsized":"2024-09-17T04:31:13.552639+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:13:44.105379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552594+0000","last_change":"2024-09-17T04:31:10.929027+0000","last_active":"2024-09-17T04:31:13.552594+0000","last_peered":"2024-09-17T04:31:13.552594+0000","last_clean":"2024-09-17T04:31:13.552594+0000","last_became_active":"2024-09-17T04:31:10.928767+0000","last_became_peered":"2024-09-17T04:31:10.928767+0000","last_unstale":"2024-09-17T04:31:13.552594+0000","last_undegraded":"2024-09-17T04:31:13.552594+0000","last_fullsized":"2024-09-17T04:31:13.552594+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:55:30.561136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907943+0000","last_change":"2024-09-17T04:31:11.923172+0000","last_active":"2024-09-17T04:31:12.907943+0000","last_peered":"2024-09-17T04:31:12.907943+0000","last_clean":"2024-09-17T04:31:12.907943+0000","last_became_active":"2024-09-17T04:31:11.922622+0000","last_became_peered":"2024-09-17T04:31:11.922622+0000","last_unstale":"2024-09-17T04:31:12.907943+0000","last_undegraded":"2024-09-17T04:31:12.907943+0000","last_fullsized":"2024-09-17T04:31:12.907943+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:11.969541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907869+0000","last_change":"2024-09-17T04:31:10.920453+0000","last_active":"2024-09-17T04:31:12.907869+0000","last_peered":"2024-09-17T04:31:12.907869+0000","last_clean":"2024-09-17T04:31:12.907869+0000","last_became_active":"2024-09-17T04:31:10.919363+0000","last_became_peered":"2024-09-17T04:31:10.919363+0000","last_unstale":"2024-09-17T04:31:12.907869+0000","last_undegraded":"2024-09-17T04:31:12.907869+0000","last_fullsized":"2024-09-17T04:31:12.907869+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:42:44.035420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.925768+0000","last_change":"2024-09-17T04:31:11.931401+0000","last_active":"2024-09-17T04:31:12.925768+0000","last_peered":"2024-09-17T04:31:12.925768+0000","last_clean":"2024-09-17T04:31:12.925768+0000","last_became_active":"2024-09-17T04:31:11.931104+0000","last_became_peered":"2024-09-17T04:31:11.931104+0000","last_unstale":"2024-09-17T04:31:12.925768+0000","last_undegraded":"2024-09-17T04:31:12.925768+0000","last_fullsized":"2024-09-17T04:31:12.925768+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:36.616761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552236+0000","last_change":"2024-09-17T04:31:10.923121+0000","last_active":"2024-09-17T04:31:13.552236+0000","last_peered":"2024-09-17T04:31:13.552236+0000","last_clean":"2024-09-17T04:31:13.552236+0000","last_became_active":"2024-09-17T04:31:10.922374+0000","last_became_peered":"2024-09-17T04:31:10.922374+0000","last_unstale":"2024-09-17T04:31:13.552236+0000","last_undegraded":"2024-09-17T04:31:13.552236+0000","last_fullsized":"2024-09-17T04:31:13.552236+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:45:53.003095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.240215+0000","last_change":"2024-09-17T04:31:11.914013+0000","last_active":"2024-09-17T04:31:13.240215+0000","last_peered":"2024-09-17T04:31:13.240215+0000","last_clean":"2024-09-17T04:31:13.240215+0000","last_became_active":"2024-09-17T04:31:11.913628+0000","last_became_peered":"2024-09-17T04:31:11.913628+0000","last_unstale":"2024-09-17T04:31:13.240215+0000","last_undegraded":"2024-09-17T04:31:13.240215+0000","last_fullsized":"2024-09-17T04:31:13.240215+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:48:43.679422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.925957+0000","last_change":"2024-09-17T04:31:10.925280+0000","last_active":"2024-09-17T04:31:12.925957+0000","last_peered":"2024-09-17T04:31:12.925957+0000","last_clean":"2024-09-17T04:31:12.925957+0000","last_became_active":"2024-09-17T04:31:10.924762+0000","last_became_peered":"2024-09-17T04:31:10.924762+0000","last_unstale":"2024-09-17T04:31:12.925957+0000","last_undegraded":"2024-09-17T04:31:12.925957+0000","last_fullsized":"2024-09-17T04:31:12.925957+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:46.091206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.419876+0000","last_change":"2024-09-17T04:31:11.932486+0000","last_active":"2024-09-17T04:31:14.419876+0000","last_peered":"2024-09-17T04:31:14.419876+0000","last_clean":"2024-09-17T04:31:14.419876+0000","last_became_active":"2024-09-17T04:31:11.932167+0000","last_became_peered":"2024-09-17T04:31:11.932167+0000","last_unstale":"2024-09-17T04:31:14.419876+0000","last_undegraded":"2024-09-17T04:31:14.419876+0000","last_fullsized":"2024-09-17T04:31:14.419876+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:16.991781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:14.419684+0000","last_change":"2024-09-17T04:31:10.926916+0000","last_active":"2024-09-17T04:31:14.419684+0000","last_peered":"2024-09-17T04:31:14.419684+0000","last_clean":"2024-09-17T04:31:14.419684+0000","last_became_active":"2024-09-17T04:31:10.926611+0000","last_became_peered":"2024-09-17T04:31:10.926611+0000","last_unstale":"2024-09-17T04:31:14.419684+0000","last_undegraded":"2024-09-17T04:31:14.419684+0000","last_fullsized":"2024-09-17T04:31:14.419684+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:09.894586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:13.552388+0000","last_change":"2024-09-17T04:31:11.930046+0000","last_active":"2024-09-17T04:31:13.552388+0000","last_peered":"2024-09-17T04:31:13.552388+0000","last_clean":"2024-09-17T04:31:13.552388+0000","last_became_active":"2024-09-17T04:31:11.929718+0000","last_became_peered":"2024-09-17T04:31:11.929718+0000","last_unstale":"2024-09-17T04:31:13.552388+0000","last_undegraded":"2024-09-17T04:31:13.552388+0000","last_fullsized":"2024-09-17T04:31:13.552388+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:11.671135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.907160+0000","last_change":"2024-09-17T04:31:10.918838+0000","last_active":"2024-09-17T04:31:12.907160+0000","last_peered":"2024-09-17T04:31:12.907160+0000","last_clean":"2024-09-17T04:31:12.907160+0000","last_became_active":"2024-09-17T04:31:10.918535+0000","last_became_peered":"2024-09-17T04:31:10.918535+0000","last_unstale":"2024-09-17T04:31:12.907160+0000","last_undegraded":"2024-09-17T04:31:12.907160+0000","last_fullsized":"2024-09-17T04:31:12.907160+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:06.468845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.925714+0000","last_change":"2024-09-17T04:31:11.936638+0000","last_active":"2024-09-17T04:31:12.925714+0000","last_peered":"2024-09-17T04:31:12.925714+0000","last_clean":"2024-09-17T04:31:12.925714+0000","last_became_active":"2024-09-17T04:31:11.932931+0000","last_became_peered":"2024-09-17T04:31:11.932931+0000","last_unstale":"2024-09-17T04:31:12.925714+0000","last_undegraded":"2024-09-17T04:31:12.925714+0000","last_fullsized":"2024-09-17T04:31:12.925714+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:10.890602+0000","last_clean_scrub_stamp":"2024-09-17T04:31:10.890602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:11.969541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.926155+0000","last_change":"2024-09-17T04:31:10.927734+0000","last_active":"2024-09-17T04:31:12.926155+0000","last_peered":"2024-09-17T04:31:12.926155+0000","last_clean":"2024-09-17T04:31:12.926155+0000","last_became_active":"2024-09-17T04:31:10.926995+0000","last_became_peered":"2024-09-17T04:31:10.926995+0000","last_unstale":"2024-09-17T04:31:12.926155+0000","last_undegraded":"2024-09-17T04:31:12.926155+0000","last_fullsized":"2024-09-17T04:31:12.926155+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:09.882826+0000","last_clean_scrub_stamp":"2024-09-17T04:31:09.882826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:47.465465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"76'79","reported_seq":317,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T04:31:12.925541+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:31:12.925541+0000","last_peered":"2024-09-17T04:31:12.925541+0000","last_clean":"2024-09-17T04:31:12.925541+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:31:12.925541+0000","last_undegraded":"2024-09-17T04:31:12.925541+0000","last_fullsized":"2024-09-17T04:31:12.925541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476829,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2093056,"data_stored":1240307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476828,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32640,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339200,"statfs":{"total":96636764160,"available":96603340800,"internally_reserved":0,"allocated":1486848,"data_stored":649301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476829,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32640,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339200,"statfs":{"total":96636764160,"available":96603340800,"internally_reserved":0,"allocated":1482752,"data_stored":647801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476829,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32636,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1478656,"data_stored":647759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542236,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32636,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1482752,"data_stored":647849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509532,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32644,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339196,"statfs":{"total":96636764160,"available":96603336704,"internally_reserved":0,"allocated":1482752,"data_stored":648305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509531,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1474560,"data_stored":647689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509531,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2068480,"data_stored":1238057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:31:21.769 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T04:31:21.769 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T04:31:22.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.036+0000 7f28c0000640 1 -- 172.21.15.100:0/2178055566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28b8145a40 msgr2=0x7f28b8073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:22.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.036+0000 7f28c0000640 1 --2- 172.21.15.100:0/2178055566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28b8145a40 0x7f28b8073540 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f28b81464e0 tx=0x7f28a402a5f0 comp rx=0 tx=0).stop 2024-09-17T04:31:22.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.036+0000 7f28c0000640 1 -- 172.21.15.100:0/2178055566 shutdown_connections 2024-09-17T04:31:22.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.036+0000 7f28c0000640 1 --2- 172.21.15.100:0/2178055566 >> v2:172.21.15.184:3300/0 conn(0x7f28b8073a80 0x7f28b8160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:22.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.036+0000 7f28c0000640 1 --2- 172.21.15.100:0/2178055566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28b8145a40 0x7f28b8073540 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:22.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.036+0000 7f28c0000640 1 --2- 172.21.15.100:0/2178055566 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28b8145550 0x7f28b806fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:22.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.036+0000 7f28c0000640 1 -- 172.21.15.100:0/2178055566 >> 172.21.15.100:0/2178055566 conn(0x7f28b806d1c0 msgr2=0x7f28b806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:22.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.036+0000 7f28c0000640 1 -- 172.21.15.100:0/2178055566 shutdown_connections 2024-09-17T04:31:22.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.036+0000 7f28c0000640 1 -- 172.21.15.100:0/2178055566 wait complete. 2024-09-17T04:31:22.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.038+0000 7f28c0000640 1 Processor -- start 2024-09-17T04:31:22.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.038+0000 7f28c0000640 1 -- start start 2024-09-17T04:31:22.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.039+0000 7f28c0000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28b8073a80 0x7f28b813db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:22.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.040+0000 7f28c0000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28b8145550 0x7f28b81f9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:22.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.040+0000 7f28c0000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28b8145a40 0x7f28b81f98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:22.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.040+0000 7f28c0000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f28b8160c20 con 0x7f28b8145550 2024-09-17T04:31:22.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.040+0000 7f28c0000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f28b8160aa0 con 0x7f28b8145a40 2024-09-17T04:31:22.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.040+0000 7f28c0000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f28b8160da0 con 0x7f28b8073a80 2024-09-17T04:31:22.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.040+0000 7f28bcc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28b8145550 0x7f28b81f9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:22.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.040+0000 7f28bcc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28b8145550 0x7f28b81f9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55530/0 (socket says 172.21.15.100:55530) 2024-09-17T04:31:22.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.040+0000 7f28bcc00640 1 -- 172.21.15.100:0/847089779 learned_addr learned my addr 172.21.15.100:0/847089779 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:22.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.040+0000 7f28bd600640 1 --2- 172.21.15.100:0/847089779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28b8073a80 0x7f28b813db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:22.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.040+0000 7f28be000640 1 --2- 172.21.15.100:0/847089779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28b8145a40 0x7f28b81f98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:22.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.040+0000 7f28bcc00640 1 -- 172.21.15.100:0/847089779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28b8073a80 msgr2=0x7f28b813db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:22.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.040+0000 7f28bcc00640 1 --2- 172.21.15.100:0/847089779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28b8073a80 0x7f28b813db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:22.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.040+0000 7f28bcc00640 1 -- 172.21.15.100:0/847089779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28b8145a40 msgr2=0x7f28b81f98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:22.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.040+0000 7f28bcc00640 1 --2- 172.21.15.100:0/847089779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28b8145a40 0x7f28b81f98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:22.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.040+0000 7f28bcc00640 1 -- 172.21.15.100:0/847089779 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b81fa1d0 con 0x7f28b8145550 2024-09-17T04:31:22.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.041+0000 7f28bcc00640 1 --2- 172.21.15.100:0/847089779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28b8145550 0x7f28b81f9370 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f28a40099c0 tx=0x7f28a40039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:22.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.041+0000 7f28b5600640 1 -- 172.21.15.100:0/847089779 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a403e030 con 0x7f28b8145550 2024-09-17T04:31:22.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.041+0000 7f28b5600640 1 -- 172.21.15.100:0/847089779 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28a40047a0 con 0x7f28b8145550 2024-09-17T04:31:22.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.041+0000 7f28c0000640 1 -- 172.21.15.100:0/847089779 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b80ae850 con 0x7f28b8145550 2024-09-17T04:31:22.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.041+0000 7f28b5600640 1 -- 172.21.15.100:0/847089779 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a4035040 con 0x7f28b8145550 2024-09-17T04:31:22.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.041+0000 7f28c0000640 1 -- 172.21.15.100:0/847089779 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b80aee10 con 0x7f28b8145550 2024-09-17T04:31:22.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.042+0000 7f28c0000640 1 -- 172.21.15.100:0/847089779 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b806fb40 con 0x7f28b8145550 2024-09-17T04:31:22.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.043+0000 7f28b5600640 1 -- 172.21.15.100:0/847089779 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f28a4041030 con 0x7f28b8145550 2024-09-17T04:31:22.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.044+0000 7f28b5600640 1 --2- 172.21.15.100:0/847089779 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2898079010 0x7f289807b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:22.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.044+0000 7f28b5600640 1 -- 172.21.15.100:0/847089779 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7f28a40bbe60 con 0x7f28b8145550 2024-09-17T04:31:22.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.044+0000 7f28bd600640 1 --2- 172.21.15.100:0/847089779 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2898079010 0x7f289807b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:22.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.045+0000 7f28bd600640 1 --2- 172.21.15.100:0/847089779 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2898079010 0x7f289807b520 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f28ac005fd0 tx=0x7f28ac005850 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:22.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.049+0000 7f28b5600640 1 -- 172.21.15.100:0/847089779 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f28b81fa1d0 con 0x7f28b8145550 2024-09-17T04:31:22.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.299+0000 7f28c0000640 1 -- 172.21.15.100:0/847089779 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f28b8141430 con 0x7f28b8145550 2024-09-17T04:31:22.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.302+0000 7f28b5600640 1 -- 172.21.15.100:0/847089779 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7f28a4083b00 con 0x7f28b8145550 2024-09-17T04:31:22.302 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-09-17T04:31:22.302 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAsBOlmfYg9IBAABP1uRKGOugrA47VxwRLkjA== 2024-09-17T04:31:22.303 INFO:teuthology.orchestra.run.smithi100.stdout: caps mds = "allow" 2024-09-17T04:31:22.303 INFO:teuthology.orchestra.run.smithi100.stdout: caps mon = "allow r" 2024-09-17T04:31:22.303 INFO:teuthology.orchestra.run.smithi100.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T04:31:22.303 INFO:teuthology.orchestra.run.smithi100.stderr:updated caps for client.0 2024-09-17T04:31:22.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.304+0000 7f28c0000640 1 -- 172.21.15.100:0/847089779 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2898079010 msgr2=0x7f289807b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:22.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.304+0000 7f28c0000640 1 --2- 172.21.15.100:0/847089779 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2898079010 0x7f289807b520 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f28ac005fd0 tx=0x7f28ac005850 comp rx=0 tx=0).stop 2024-09-17T04:31:22.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.304+0000 7f28c0000640 1 -- 172.21.15.100:0/847089779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28b8145550 msgr2=0x7f28b81f9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:22.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.304+0000 7f28c0000640 1 --2- 172.21.15.100:0/847089779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28b8145550 0x7f28b81f9370 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f28a40099c0 tx=0x7f28a40039e0 comp rx=0 tx=0).stop 2024-09-17T04:31:22.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.304+0000 7f28c0000640 1 -- 172.21.15.100:0/847089779 shutdown_connections 2024-09-17T04:31:22.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.304+0000 7f28c0000640 1 --2- 172.21.15.100:0/847089779 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2898079010 0x7f289807b520 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:22.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.304+0000 7f28c0000640 1 --2- 172.21.15.100:0/847089779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28b8145a40 0x7f28b81f98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:22.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.304+0000 7f28c0000640 1 --2- 172.21.15.100:0/847089779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28b8145550 0x7f28b81f9370 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:22.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.304+0000 7f28c0000640 1 --2- 172.21.15.100:0/847089779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28b8073a80 0x7f28b813db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:22.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.304+0000 7f28c0000640 1 -- 172.21.15.100:0/847089779 >> 172.21.15.100:0/847089779 conn(0x7f28b806d1c0 msgr2=0x7f28b80b00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:22.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.305+0000 7f28c0000640 1 -- 172.21.15.100:0/847089779 shutdown_connections 2024-09-17T04:31:22.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.305+0000 7f28c0000640 1 -- 172.21.15.100:0/847089779 wait complete. 2024-09-17T04:31:22.326 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:31:22.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.545+0000 7f05b2c00640 1 -- 172.21.15.100:0/2168983045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0594003a00 msgr2=0x7f0594003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:22.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.545+0000 7f05b2c00640 1 --2- 172.21.15.100:0/2168983045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0594003a00 0x7f0594003e70 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f05a0009b50 tx=0x7f0594005830 comp rx=0 tx=0).stop 2024-09-17T04:31:22.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.545+0000 7f05b2200640 1 -- 172.21.15.100:0/2168983045 reap_dead start 2024-09-17T04:31:22.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.545+0000 7f05b2c00640 1 -- 172.21.15.100:0/2168983045 shutdown_connections 2024-09-17T04:31:22.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.545+0000 7f05b2c00640 1 -- 172.21.15.100:0/2168983045 >> 172.21.15.100:0/2168983045 conn(0x7f05ac0b1e40 msgr2=0x7f05ac0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:22.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.545+0000 7f05b2c00640 1 -- 172.21.15.100:0/2168983045 shutdown_connections 2024-09-17T04:31:22.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.545+0000 7f05b2c00640 1 -- 172.21.15.100:0/2168983045 wait complete. 2024-09-17T04:31:22.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.547+0000 7f05b2c00640 1 Processor -- start 2024-09-17T04:31:22.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.548+0000 7f05b2c00640 1 -- start start 2024-09-17T04:31:22.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.549+0000 7f05b2c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0594003a00 0x7f05ac1f8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:22.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.549+0000 7f05b2c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05ac148fd0 0x7f05ac149cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:22.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.549+0000 7f05b2c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05ac14b750 0x7f05ac1f5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:22.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.549+0000 7f05b2c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac163320 con 0x7f0594003a00 2024-09-17T04:31:22.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.549+0000 7f05b2c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac1631a0 con 0x7f05ac148fd0 2024-09-17T04:31:22.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.549+0000 7f05b2c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f05ac1634a0 con 0x7f05ac14b750 2024-09-17T04:31:22.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.549+0000 7f05b1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0594003a00 0x7f05ac1f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:22.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.549+0000 7f05b1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0594003a00 0x7f05ac1f8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55550/0 (socket says 172.21.15.100:55550) 2024-09-17T04:31:22.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.549+0000 7f05b1800640 1 -- 172.21.15.100:0/901392085 learned_addr learned my addr 172.21.15.100:0/901392085 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:22.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.549+0000 7f05b0e00640 1 --2- 172.21.15.100:0/901392085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05ac148fd0 0x7f05ac149cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:22.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.550+0000 7f05b2200640 1 --2- 172.21.15.100:0/901392085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05ac14b750 0x7f05ac1f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:22.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.550+0000 7f05b1800640 1 -- 172.21.15.100:0/901392085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05ac14b750 msgr2=0x7f05ac1f5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:22.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.550+0000 7f05b1800640 1 --2- 172.21.15.100:0/901392085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05ac14b750 0x7f05ac1f5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:22.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.550+0000 7f05b1800640 1 -- 172.21.15.100:0/901392085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05ac148fd0 msgr2=0x7f05ac149cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:22.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.550+0000 7f05b1800640 1 --2- 172.21.15.100:0/901392085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05ac148fd0 0x7f05ac149cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:22.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.550+0000 7f05b1800640 1 -- 172.21.15.100:0/901392085 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ac1f8ec0 con 0x7f0594003a00 2024-09-17T04:31:22.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.550+0000 7f05b0e00640 1 --2- 172.21.15.100:0/901392085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05ac148fd0 0x7f05ac149cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:31:22.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.550+0000 7f05b1800640 1 --2- 172.21.15.100:0/901392085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0594003a00 0x7f05ac1f8680 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f05a0009a70 tx=0x7f05a0033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:22.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.550+0000 7f05a9600640 1 -- 172.21.15.100:0/901392085 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a0038040 con 0x7f0594003a00 2024-09-17T04:31:22.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.550+0000 7f05a9600640 1 -- 172.21.15.100:0/901392085 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05a0004330 con 0x7f0594003a00 2024-09-17T04:31:22.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.550+0000 7f05a9600640 1 -- 172.21.15.100:0/901392085 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a0004520 con 0x7f0594003a00 2024-09-17T04:31:22.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.550+0000 7f05b2c00640 1 -- 172.21.15.100:0/901392085 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05ac1f91a0 con 0x7f0594003a00 2024-09-17T04:31:22.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.551+0000 7f05b2c00640 1 -- 172.21.15.100:0/901392085 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05ac1f5420 con 0x7f0594003a00 2024-09-17T04:31:22.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.551+0000 7f05b2c00640 1 -- 172.21.15.100:0/901392085 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05ac0ae080 con 0x7f0594003a00 2024-09-17T04:31:22.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.553+0000 7f05a9600640 1 -- 172.21.15.100:0/901392085 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f05a0045060 con 0x7f0594003a00 2024-09-17T04:31:22.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.554+0000 7f05a9600640 1 --2- 172.21.15.100:0/901392085 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f058c079010 0x7f058c07b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:22.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.554+0000 7f05a9600640 1 -- 172.21.15.100:0/901392085 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7f05a00bbc30 con 0x7f0594003a00 2024-09-17T04:31:22.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.554+0000 7f05b0e00640 1 --2- 172.21.15.100:0/901392085 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f058c079010 0x7f058c07b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:22.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.554+0000 7f05b0e00640 1 --2- 172.21.15.100:0/901392085 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f058c079010 0x7f058c07b520 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0598001860 tx=0x7f0598001cd0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:22.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.558+0000 7f05a9600640 1 -- 172.21.15.100:0/901392085 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f05a0083840 con 0x7f0594003a00 2024-09-17T04:31:22.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.774+0000 7f05b2c00640 1 -- 172.21.15.100:0/901392085 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05ac06de60 con 0x7f0594003a00 2024-09-17T04:31:22.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.775+0000 7f05a9600640 1 -- 172.21.15.100:0/901392085 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5234 (secure 0 0 0) 0x7f05a0083140 con 0x7f0594003a00 2024-09-17T04:31:22.776 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:22.776 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":82,"btime":"2024-09-17T04:31:14:929364+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6410,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1845046345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1845046345},{"type":"v1","addr":"172.21.15.184:6801","nonce":1845046345}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7135,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3970003794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3970003794},{"type":"v1","addr":"172.21.15.184:6835","nonce":3970003794}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/3663836388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3663836388},{"type":"v1","addr":"172.21.15.100:6834","nonce":3663836388}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:31:11.402955+0000","modified":"2024-09-17T04:31:14.414638+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7542":{"gid":7542,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/207356836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":207356836},{"type":"v1","addr":"172.21.15.100:6835","nonce":207356836}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7542,"qdb_cluster":[7542]},"id":9}]} 2024-09-17T04:31:22.776 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 82 2024-09-17T04:31:22.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.778+0000 7f05b2c00640 1 -- 172.21.15.100:0/901392085 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f058c079010 msgr2=0x7f058c07b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:22.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.778+0000 7f05b2c00640 1 --2- 172.21.15.100:0/901392085 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f058c079010 0x7f058c07b520 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0598001860 tx=0x7f0598001cd0 comp rx=0 tx=0).stop 2024-09-17T04:31:22.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.778+0000 7f05b2c00640 1 -- 172.21.15.100:0/901392085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0594003a00 msgr2=0x7f05ac1f8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:22.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.778+0000 7f05b2c00640 1 --2- 172.21.15.100:0/901392085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0594003a00 0x7f05ac1f8680 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f05a0009a70 tx=0x7f05a0033670 comp rx=0 tx=0).stop 2024-09-17T04:31:22.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.778+0000 7f05b2c00640 1 -- 172.21.15.100:0/901392085 shutdown_connections 2024-09-17T04:31:22.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.778+0000 7f05b2c00640 1 --2- 172.21.15.100:0/901392085 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f058c079010 0x7f058c07b520 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:22.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.778+0000 7f05b2c00640 1 --2- 172.21.15.100:0/901392085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05ac14b750 0x7f05ac1f5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:22.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.778+0000 7f05b2c00640 1 --2- 172.21.15.100:0/901392085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05ac148fd0 0x7f05ac149cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:22.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.778+0000 7f05b2c00640 1 --2- 172.21.15.100:0/901392085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0594003a00 0x7f05ac1f8680 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:22.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.778+0000 7f05b2c00640 1 -- 172.21.15.100:0/901392085 >> 172.21.15.100:0/901392085 conn(0x7f05ac0b1e40 msgr2=0x7f05ac0b0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:22.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.778+0000 7f05b2c00640 1 -- 172.21.15.100:0/901392085 shutdown_connections 2024-09-17T04:31:22.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:22.778+0000 7f05b2c00640 1 -- 172.21.15.100:0/901392085 wait complete. 2024-09-17T04:31:23.206 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 82, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T04:31:11.402955+0000', 'modified': '2024-09-17T04:31:14.414638+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7542}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7542': {'gid': 7542, 'name': 'c', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6835/207356836', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6833', 'nonce': 207356836}, {'type': 'v1', 'addr': '172.21.15.100:6835', 'nonce': 207356836}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7542, 'qdb_cluster': [7542]} 2024-09-17T04:31:23.206 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T04:31:23.206 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T04:31:23.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.426+0000 7f77c5c00640 1 -- 172.21.15.100:0/1486882445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77c00705b0 msgr2=0x7f77c00709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:23.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.426+0000 7f77c5c00640 1 --2- 172.21.15.100:0/1486882445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77c00705b0 0x7f77c00709c0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f77c00cde30 tx=0x7f77ac02a4d0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.427+0000 7f77c5c00640 1 -- 172.21.15.100:0/1486882445 shutdown_connections 2024-09-17T04:31:23.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.427+0000 7f77c5c00640 1 --2- 172.21.15.100:0/1486882445 >> v2:172.21.15.184:3300/0 conn(0x7f77c015ce30 0x7f77c015d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.427+0000 7f77c5c00640 1 --2- 172.21.15.100:0/1486882445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77c00705b0 0x7f77c00709c0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.427+0000 7f77c5c00640 1 --2- 172.21.15.100:0/1486882445 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77c00cfc80 0x7f77c0156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.427+0000 7f77c5c00640 1 -- 172.21.15.100:0/1486882445 >> 172.21.15.100:0/1486882445 conn(0x7f77c006d3d0 msgr2=0x7f77c00b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:23.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.427+0000 7f77c5c00640 1 -- 172.21.15.100:0/1486882445 shutdown_connections 2024-09-17T04:31:23.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.427+0000 7f77c5c00640 1 -- 172.21.15.100:0/1486882445 wait complete. 2024-09-17T04:31:23.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.429+0000 7f77c5c00640 1 Processor -- start 2024-09-17T04:31:23.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.429+0000 7f77c5c00640 1 -- start start 2024-09-17T04:31:23.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.430+0000 7f77c5c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77c00705b0 0x7f77c009a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:23.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.430+0000 7f77c5c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77c00cfc80 0x7f77c01f5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:23.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.430+0000 7f77c5c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77c015ce30 0x7f77c0073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:23.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.430+0000 7f77c5c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f77c00cce40 con 0x7f77c00705b0 2024-09-17T04:31:23.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.430+0000 7f77c5c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f77c00cccc0 con 0x7f77c00cfc80 2024-09-17T04:31:23.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.430+0000 7f77c5c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f77c00ccfc0 con 0x7f77c015ce30 2024-09-17T04:31:23.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.431+0000 7f77bea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77c00705b0 0x7f77c009a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:23.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.431+0000 7f77bea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77c00705b0 0x7f77c009a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55566/0 (socket says 172.21.15.100:55566) 2024-09-17T04:31:23.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.431+0000 7f77bea00640 1 -- 172.21.15.100:0/3406518820 learned_addr learned my addr 172.21.15.100:0/3406518820 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:23.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.431+0000 7f77bf400640 1 --2- 172.21.15.100:0/3406518820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77c015ce30 0x7f77c0073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:23.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.431+0000 7f77be000640 1 --2- 172.21.15.100:0/3406518820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77c00cfc80 0x7f77c01f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:23.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.431+0000 7f77bea00640 1 -- 172.21.15.100:0/3406518820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77c015ce30 msgr2=0x7f77c0073f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:23.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.431+0000 7f77bea00640 1 --2- 172.21.15.100:0/3406518820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77c015ce30 0x7f77c0073f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.431+0000 7f77bea00640 1 -- 172.21.15.100:0/3406518820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77c00cfc80 msgr2=0x7f77c01f5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:23.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.431+0000 7f77bea00640 1 --2- 172.21.15.100:0/3406518820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77c00cfc80 0x7f77c01f5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.431+0000 7f77bea00640 1 -- 172.21.15.100:0/3406518820 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c00c6d50 con 0x7f77c00705b0 2024-09-17T04:31:23.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.431+0000 7f77bf400640 1 --2- 172.21.15.100:0/3406518820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77c015ce30 0x7f77c0073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:31:23.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.431+0000 7f77be000640 1 --2- 172.21.15.100:0/3406518820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77c00cfc80 0x7f77c01f5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:31:23.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.431+0000 7f77bea00640 1 --2- 172.21.15.100:0/3406518820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77c00705b0 0x7f77c009a2d0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f77b000d6b0 tx=0x7f77b000db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:23.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.432+0000 7f77b6a00640 1 -- 172.21.15.100:0/3406518820 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b0017020 con 0x7f77c00705b0 2024-09-17T04:31:23.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.432+0000 7f77b6a00640 1 -- 172.21.15.100:0/3406518820 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77b0004320 con 0x7f77c00705b0 2024-09-17T04:31:23.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.432+0000 7f77b6a00640 1 -- 172.21.15.100:0/3406518820 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b0009a80 con 0x7f77c00705b0 2024-09-17T04:31:23.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.432+0000 7f77c5c00640 1 -- 172.21.15.100:0/3406518820 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c0075030 con 0x7f77c00705b0 2024-09-17T04:31:23.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.432+0000 7f77c5c00640 1 -- 172.21.15.100:0/3406518820 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c009c930 con 0x7f77c00705b0 2024-09-17T04:31:23.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.434+0000 7f77b6a00640 1 -- 172.21.15.100:0/3406518820 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f77b001a020 con 0x7f77c00705b0 2024-09-17T04:31:23.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.435+0000 7f77b6a00640 1 --2- 172.21.15.100:0/3406518820 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f77a0079010 0x7f77a007b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:23.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.435+0000 7f77b6a00640 1 -- 172.21.15.100:0/3406518820 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7f77b009ba10 con 0x7f77c00705b0 2024-09-17T04:31:23.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.435+0000 7f77be000640 1 --2- 172.21.15.100:0/3406518820 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f77a0079010 0x7f77a007b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:23.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.436+0000 7f77cce45740 1 Processor -- start 2024-09-17T04:31:23.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.436+0000 7f77cce45740 1 -- start start 2024-09-17T04:31:23.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.436+0000 7f77be000640 1 --2- 172.21.15.100:0/3406518820 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f77a0079010 0x7f77a007b520 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f77ac033000 tx=0x7f77ac004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:23.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.437+0000 7f77cce45740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5560782e6270 0x5560786b6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:23.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.437+0000 7f77cce45740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560782bd3b0 0x5560786b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:23.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.437+0000 7f77cce45740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560782c5590 0x556078686f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:23.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.437+0000 7f77cce45740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556078610430 con 0x5560782bd3b0 2024-09-17T04:31:23.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.437+0000 7f77cce45740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5560786005d0 con 0x5560782c5590 2024-09-17T04:31:23.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.437+0000 7f77cce45740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5560785efa00 con 0x5560782e6270 2024-09-17T04:31:23.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.438+0000 7f77be000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560782bd3b0 0x5560786b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:23.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.438+0000 7f77be000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560782bd3b0 0x5560786b6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55572/0 (socket says 172.21.15.100:55572) 2024-09-17T04:31:23.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.438+0000 7f77be000640 1 -- 172.21.15.100:0/3360821638 learned_addr learned my addr 172.21.15.100:0/3360821638 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:23.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.438+0000 7f77bea00640 1 --2- 172.21.15.100:0/3360821638 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5560782e6270 0x5560786b6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:23.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.438+0000 7f77bf400640 1 --2- 172.21.15.100:0/3360821638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560782c5590 0x556078686f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:23.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.438+0000 7f77be000640 1 -- 172.21.15.100:0/3360821638 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5560782e6270 msgr2=0x5560786b6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:31:23.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.438+0000 7f77be000640 1 --2- 172.21.15.100:0/3360821638 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5560782e6270 0x5560786b6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.438+0000 7f77be000640 1 -- 172.21.15.100:0/3360821638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560782c5590 msgr2=0x556078686f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:31:23.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.438+0000 7f77be000640 1 --2- 172.21.15.100:0/3360821638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560782c5590 0x556078686f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.438+0000 7f77be000640 1 -- 172.21.15.100:0/3360821638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560785b8520 con 0x5560782bd3b0 2024-09-17T04:31:23.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.438+0000 7f77be000640 1 --2- 172.21.15.100:0/3360821638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560782bd3b0 0x5560786b6eb0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f77ac000c00 tx=0x7f77ac03b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163d92f5f000b593 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:23.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.438+0000 7f77ab400640 1 -- 172.21.15.100:0/3360821638 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac035050 con 0x5560782bd3b0 2024-09-17T04:31:23.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.439+0000 7f77ab400640 1 -- 172.21.15.100:0/3360821638 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77ac04c070 con 0x5560782bd3b0 2024-09-17T04:31:23.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.439+0000 7f77ab400640 1 -- 172.21.15.100:0/3360821638 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac003be0 con 0x5560782bd3b0 2024-09-17T04:31:23.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.439+0000 7f77cce45740 1 -- 172.21.15.100:0/3360821638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560782bd3b0 msgr2=0x5560786b6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:23.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.439+0000 7f77cce45740 1 --2- 172.21.15.100:0/3360821638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560782bd3b0 0x5560786b6eb0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f77ac000c00 tx=0x7f77ac03b000 comp rx=0 tx=0).stop 2024-09-17T04:31:23.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.439+0000 7f77cce45740 1 -- 172.21.15.100:0/3360821638 shutdown_connections 2024-09-17T04:31:23.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.439+0000 7f77cce45740 1 --2- 172.21.15.100:0/3360821638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560782c5590 0x556078686f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.439+0000 7f77cce45740 1 --2- 172.21.15.100:0/3360821638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560782bd3b0 0x5560786b6eb0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.439+0000 7f77cce45740 1 --2- 172.21.15.100:0/3360821638 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5560782e6270 0x5560786b6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.439+0000 7f77cce45740 1 -- 172.21.15.100:0/3360821638 >> 172.21.15.100:0/3360821638 conn(0x5560782da3c0 msgr2=0x55607868c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:23.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.439+0000 7f77cce45740 1 -- 172.21.15.100:0/3360821638 shutdown_connections 2024-09-17T04:31:23.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.440+0000 7f77cce45740 1 -- 172.21.15.100:0/3360821638 wait complete. 2024-09-17T04:31:23.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.440+0000 7f77cce45740 1 -- start start 2024-09-17T04:31:23.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.441+0000 7f77cce45740 1 Processor -- start 2024-09-17T04:31:23.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.442+0000 7f77cce45740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560782bd3b0 0x5560786cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:23.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.442+0000 7f77cce45740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5560782c5590 0x5560786ad280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:23.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.442+0000 7f77cce45740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560782e6270 0x5560786e4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:23.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.442+0000 7f77cce45740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556078596330 con 0x5560782bd3b0 2024-09-17T04:31:23.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.442+0000 7f77cce45740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5560785f3810 con 0x5560782e6270 2024-09-17T04:31:23.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.442+0000 7f77cce45740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x556078360620 con 0x5560782c5590 2024-09-17T04:31:23.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.442+0000 7f77cce45740 10 client.0 _ll_register_callbacks cb 0x55607859b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:31:23.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.442+0000 7f77bea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560782bd3b0 0x5560786cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:23.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.443+0000 7f77bea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560782bd3b0 0x5560786cbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55582/0 (socket says 172.21.15.100:55582) 2024-09-17T04:31:23.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.443+0000 7f77bea00640 1 -- 172.21.15.100:0/760667411 learned_addr learned my addr 172.21.15.100:0/760667411 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:23.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.443+0000 7f77be000640 1 --2- 172.21.15.100:0/760667411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5560782c5590 0x5560786ad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:23.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.443+0000 7f77bf400640 1 --2- 172.21.15.100:0/760667411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560782e6270 0x5560786e4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:23.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.443+0000 7f77bea00640 1 -- 172.21.15.100:0/760667411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5560782c5590 msgr2=0x5560786ad280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:31:23.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.443+0000 7f77bea00640 1 --2- 172.21.15.100:0/760667411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5560782c5590 0x5560786ad280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.443+0000 7f77bea00640 1 -- 172.21.15.100:0/760667411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560782e6270 msgr2=0x5560786e4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:31:23.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.443+0000 7f77bea00640 1 --2- 172.21.15.100:0/760667411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560782e6270 0x5560786e4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.443+0000 7f77bea00640 1 -- 172.21.15.100:0/760667411 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556078351850 con 0x5560782bd3b0 2024-09-17T04:31:23.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.443+0000 7f77bea00640 1 --2- 172.21.15.100:0/760667411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560782bd3b0 0x5560786cbd40 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f77b0009610 tx=0x7f77b0062a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1633f509816909ac server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:23.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.443+0000 7f779fe00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-17T04:31:23.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.444+0000 7f779fe00640 1 -- 172.21.15.100:0/760667411 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b00636b0 con 0x5560782bd3b0 2024-09-17T04:31:23.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.444+0000 7f779fe00640 1 -- 172.21.15.100:0/760667411 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7f77b0089120 con 0x5560782bd3b0 2024-09-17T04:31:23.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.444+0000 7f77cce45740 1 -- 172.21.15.100:0/760667411 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77ac04c070 con 0x5560782bd3b0 2024-09-17T04:31:23.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.444+0000 7f779fe00640 1 -- 172.21.15.100:0/760667411 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f77b0062fb0 con 0x5560782bd3b0 2024-09-17T04:31:23.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.444+0000 7f77cce45740 10 client.7698 fetch_fsmap learned FSMap version 82 2024-09-17T04:31:23.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.444+0000 7f77cce45740 1 -- 172.21.15.100:0/760667411 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x5560783c86b0 con 0x5560782bd3b0 2024-09-17T04:31:23.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.444+0000 7f779fe00640 1 -- 172.21.15.100:0/760667411 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77ac04c070 con 0x5560782bd3b0 2024-09-17T04:31:23.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.444+0000 7f779fe00640 1 -- 172.21.15.100:0/760667411 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 82) ==== 3502+0+0 (secure 0 0 0) 0x7f77b0062210 con 0x5560782bd3b0 2024-09-17T04:31:23.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.444+0000 7f77cce45740 10 client.7698 fetch_fsmap finished waiting for FSMap version 82 2024-09-17T04:31:23.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.444+0000 7f77cce45740 10 client.7698 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T04:31:23.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.444+0000 7f77cce45740 20 client.7698 populate_metadata read hostname 'smithi100' 2024-09-17T04:31:23.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.444+0000 7f77cce45740 1 --2- 172.21.15.100:0/760667411 >> [v2:172.21.15.100:6833/207356836,v1:172.21.15.100:6835/207356836] conn(0x5560782a4e40 0x5560786e0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:23.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.444+0000 7f77cce45740 4 client.7698 mds_command: new command op to 7542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T04:31:23.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.444+0000 7f77cce45740 1 -- 172.21.15.100:0/760667411 --> [v2:172.21.15.100:6833/207356836,v1:172.21.15.100:6835/207356836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556078602520 con 0x5560782a4e40 2024-09-17T04:31:23.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.444+0000 7f77be000640 1 --2- 172.21.15.100:0/760667411 >> [v2:172.21.15.100:6833/207356836,v1:172.21.15.100:6835/207356836] conn(0x5560782a4e40 0x5560786e0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:23.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.445+0000 7f77be000640 1 --2- 172.21.15.100:0/760667411 >> [v2:172.21.15.100:6833/207356836,v1:172.21.15.100:6835/207356836] conn(0x5560782a4e40 0x5560786e0cd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fdc25b3bcef18e6 server_cookie=b64aa1b66255ea6e in_seq=0 out_seq=0 2024-09-17T04:31:23.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.445+0000 7f779fe00640 10 client.7698 ms_handle_connect on v2:172.21.15.100:6833/207356836 2024-09-17T04:31:23.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.446+0000 7f779fe00640 1 -- 172.21.15.100:0/760667411 <== mds.0 v2:172.21.15.100:6833/207356836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556078602520 con 0x5560782a4e40 2024-09-17T04:31:23.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.446+0000 7f779fe00640 10 client.7698 handle_command_reply: tid=0 2024-09-17T04:31:23.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.446+0000 7f77cce45740 1 client.7698 shutdown 2024-09-17T04:31:23.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.446+0000 7f77cce45740 1 -- 172.21.15.100:0/760667411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560782bd3b0 msgr2=0x5560786cbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:23.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.446+0000 7f77cce45740 1 --2- 172.21.15.100:0/760667411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560782bd3b0 0x5560786cbd40 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f77b0009610 tx=0x7f77b0062a00 comp rx=0 tx=0).stop 2024-09-17T04:31:23.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.446+0000 7f77cce45740 1 -- 172.21.15.100:0/760667411 shutdown_connections 2024-09-17T04:31:23.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.446+0000 7f77cce45740 1 --2- 172.21.15.100:0/760667411 >> [v2:172.21.15.100:6833/207356836,v1:172.21.15.100:6835/207356836] conn(0x5560782a4e40 0x5560786e0cd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.446+0000 7f77cce45740 1 --2- 172.21.15.100:0/760667411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560782e6270 0x5560786e4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.446+0000 7f77cce45740 1 --2- 172.21.15.100:0/760667411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5560782c5590 0x5560786ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.446+0000 7f77cce45740 1 --2- 172.21.15.100:0/760667411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560782bd3b0 0x5560786cbd40 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.446+0000 7f779fe00640 0 client.7698 ms_handle_reset on v2:172.21.15.100:6833/207356836 2024-09-17T04:31:23.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.446+0000 7f77cce45740 1 -- 172.21.15.100:0/760667411 >> 172.21.15.100:0/760667411 conn(0x5560782da3c0 msgr2=0x5560786d38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:23.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.446+0000 7f77cce45740 1 -- 172.21.15.100:0/760667411 shutdown_connections 2024-09-17T04:31:23.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.447+0000 7f77cce45740 1 -- 172.21.15.100:0/760667411 wait complete. 2024-09-17T04:31:23.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.447+0000 7f77cce45740 20 client.7698 trim_cache size 0 max 16384 2024-09-17T04:31:23.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.465+0000 7f77cce45740 1 Processor -- start 2024-09-17T04:31:23.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.466+0000 7f77cce45740 1 -- start start 2024-09-17T04:31:23.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.467+0000 7f77cce45740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560782bd3b0 0x556078687f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:23.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.467+0000 7f77cce45740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5560782c5590 0x5560786884c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:23.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.467+0000 7f77cce45740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560782e6270 0x5560786d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:23.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.467+0000 7f77cce45740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f77c00b1300 con 0x5560782e6270 2024-09-17T04:31:23.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.467+0000 7f77cce45740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f77c00b0660 con 0x5560782bd3b0 2024-09-17T04:31:23.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.467+0000 7f77cce45740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f77c00b0cb0 con 0x5560782c5590 2024-09-17T04:31:23.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.467+0000 7f77bf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560782e6270 0x5560786d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:23.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.467+0000 7f77bf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560782e6270 0x5560786d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55596/0 (socket says 172.21.15.100:55596) 2024-09-17T04:31:23.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.467+0000 7f77bea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560782bd3b0 0x556078687f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:23.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.467+0000 7f77be000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5560782c5590 0x5560786884c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:23.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.467+0000 7f77bf400640 1 -- 172.21.15.100:0/3421891179 learned_addr learned my addr 172.21.15.100:0/3421891179 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:23.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.468+0000 7f77bf400640 1 -- 172.21.15.100:0/3421891179 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5560782c5590 msgr2=0x5560786884c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:31:23.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.468+0000 7f77bf400640 1 --2- 172.21.15.100:0/3421891179 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5560782c5590 0x5560786884c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.468+0000 7f77bf400640 1 -- 172.21.15.100:0/3421891179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560782bd3b0 msgr2=0x556078687f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:31:23.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.468+0000 7f77bf400640 1 --2- 172.21.15.100:0/3421891179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560782bd3b0 0x556078687f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.468+0000 7f77bf400640 1 -- 172.21.15.100:0/3421891179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560786afc00 con 0x5560782e6270 2024-09-17T04:31:23.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.468+0000 7f77bf400640 1 --2- 172.21.15.100:0/3421891179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560782e6270 0x5560786d2820 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009f90 tx=0x7f77b80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ed8d0347d758d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:23.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.468+0000 7f77aaa00640 1 -- 172.21.15.100:0/3421891179 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8004b20 con 0x5560782e6270 2024-09-17T04:31:23.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.468+0000 7f77aaa00640 1 -- 172.21.15.100:0/3421891179 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77b8004cc0 con 0x5560782e6270 2024-09-17T04:31:23.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.468+0000 7f77aaa00640 1 -- 172.21.15.100:0/3421891179 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8005280 con 0x5560782e6270 2024-09-17T04:31:23.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.469+0000 7f77cce45740 1 -- 172.21.15.100:0/3421891179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560782e6270 msgr2=0x5560786d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:23.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.469+0000 7f77cce45740 1 --2- 172.21.15.100:0/3421891179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560782e6270 0x5560786d2820 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009f90 tx=0x7f77b80118d0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.469+0000 7f77cce45740 1 -- 172.21.15.100:0/3421891179 shutdown_connections 2024-09-17T04:31:23.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.469+0000 7f77cce45740 1 --2- 172.21.15.100:0/3421891179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560782e6270 0x5560786d2820 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.469+0000 7f77cce45740 1 --2- 172.21.15.100:0/3421891179 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5560782c5590 0x5560786884c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.469+0000 7f77cce45740 1 --2- 172.21.15.100:0/3421891179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560782bd3b0 0x556078687f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.469+0000 7f77cce45740 1 -- 172.21.15.100:0/3421891179 >> 172.21.15.100:0/3421891179 conn(0x5560782da3c0 msgr2=0x5560786b66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:23.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.469+0000 7f77cce45740 1 -- 172.21.15.100:0/3421891179 shutdown_connections 2024-09-17T04:31:23.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.469+0000 7f77cce45740 1 -- 172.21.15.100:0/3421891179 wait complete. 2024-09-17T04:31:23.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.470+0000 7f77cce45740 1 -- start start 2024-09-17T04:31:23.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.470+0000 7f77cce45740 1 Processor -- start 2024-09-17T04:31:23.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.472+0000 7f77cce45740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5560782bd3b0 0x5560786ab170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:23.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.472+0000 7f77cce45740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560782c5590 0x556078688170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:23.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.472+0000 7f77cce45740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560782e6270 0x556078739ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:23.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.472+0000 7f77cce45740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5560786e4690 con 0x5560782e6270 2024-09-17T04:31:23.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.472+0000 7f77cce45740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5560786b24e0 con 0x5560782c5590 2024-09-17T04:31:23.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.472+0000 7f77cce45740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5560786b0c60 con 0x5560782bd3b0 2024-09-17T04:31:23.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.472+0000 7f77cce45740 10 client.0 _ll_register_callbacks cb 0x7f77b005f450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:31:23.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.472+0000 7f77bf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560782e6270 0x556078739ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:23.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.472+0000 7f77bf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560782e6270 0x556078739ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55600/0 (socket says 172.21.15.100:55600) 2024-09-17T04:31:23.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.472+0000 7f77bf400640 1 -- 172.21.15.100:0/673078239 learned_addr learned my addr 172.21.15.100:0/673078239 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:23.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.472+0000 7f77be000640 1 --2- 172.21.15.100:0/673078239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560782c5590 0x556078688170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:23.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.472+0000 7f77bea00640 1 --2- 172.21.15.100:0/673078239 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5560782bd3b0 0x5560786ab170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:23.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.472+0000 7f77bf400640 1 -- 172.21.15.100:0/673078239 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5560782bd3b0 msgr2=0x5560786ab170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:31:23.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.472+0000 7f77bf400640 1 --2- 172.21.15.100:0/673078239 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5560782bd3b0 0x5560786ab170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.472+0000 7f77bf400640 1 -- 172.21.15.100:0/673078239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560782c5590 msgr2=0x556078688170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:31:23.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.472+0000 7f77bf400640 1 --2- 172.21.15.100:0/673078239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560782c5590 0x556078688170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.473+0000 7f77bf400640 1 -- 172.21.15.100:0/673078239 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556078685c60 con 0x5560782e6270 2024-09-17T04:31:23.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.473+0000 7f77bf400640 1 --2- 172.21.15.100:0/673078239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560782e6270 0x556078739ca0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f77b8018000 tx=0x7f77b8010720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5316d23ab22e103c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:23.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.473+0000 7f779fe00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-17T04:31:23.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.473+0000 7f779fe00640 1 -- 172.21.15.100:0/673078239 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8004b20 con 0x5560782e6270 2024-09-17T04:31:23.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.473+0000 7f779fe00640 1 -- 172.21.15.100:0/673078239 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7f77b8017040 con 0x5560782e6270 2024-09-17T04:31:23.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.473+0000 7f77cce45740 1 -- 172.21.15.100:0/673078239 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77b8004cc0 con 0x5560782e6270 2024-09-17T04:31:23.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.473+0000 7f779fe00640 1 -- 172.21.15.100:0/673078239 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f77b8009bc0 con 0x5560782e6270 2024-09-17T04:31:23.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.473+0000 7f77cce45740 10 client.7704 fetch_fsmap learned FSMap version 82 2024-09-17T04:31:23.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.473+0000 7f77cce45740 1 -- 172.21.15.100:0/673078239 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x556078688770 con 0x5560782e6270 2024-09-17T04:31:23.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.474+0000 7f779fe00640 1 -- 172.21.15.100:0/673078239 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77b8004cc0 con 0x5560782e6270 2024-09-17T04:31:23.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.474+0000 7f779fe00640 1 -- 172.21.15.100:0/673078239 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 82) ==== 3502+0+0 (secure 0 0 0) 0x7f77b802e050 con 0x5560782e6270 2024-09-17T04:31:23.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.474+0000 7f77cce45740 10 client.7704 fetch_fsmap finished waiting for FSMap version 82 2024-09-17T04:31:23.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.474+0000 7f77cce45740 10 client.7704 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T04:31:23.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.474+0000 7f77cce45740 20 client.7704 populate_metadata read hostname 'smithi100' 2024-09-17T04:31:23.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.474+0000 7f77cce45740 1 --2- 172.21.15.100:0/673078239 >> [v2:172.21.15.100:6833/207356836,v1:172.21.15.100:6835/207356836] conn(0x55607873a360 0x556078754660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:23.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.474+0000 7f77cce45740 4 client.7704 mds_command: new command op to 7542 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T04:31:23.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.474+0000 7f77cce45740 1 -- 172.21.15.100:0/673078239 --> [v2:172.21.15.100:6833/207356836,v1:172.21.15.100:6835/207356836] -- command(tid 0: {"prefix": "status"}) -- 0x55607868bfb0 con 0x55607873a360 2024-09-17T04:31:23.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.474+0000 7f77bea00640 1 --2- 172.21.15.100:0/673078239 >> [v2:172.21.15.100:6833/207356836,v1:172.21.15.100:6835/207356836] conn(0x55607873a360 0x556078754660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:23.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.475+0000 7f77bea00640 1 --2- 172.21.15.100:0/673078239 >> [v2:172.21.15.100:6833/207356836,v1:172.21.15.100:6835/207356836] conn(0x55607873a360 0x556078754660 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9b73d1d9341226d server_cookie=4b536607f352ea2d in_seq=0 out_seq=0 2024-09-17T04:31:23.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.475+0000 7f779fe00640 10 client.7704 ms_handle_connect on v2:172.21.15.100:6833/207356836 2024-09-17T04:31:23.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.475+0000 7f779fe00640 1 -- 172.21.15.100:0/673078239 <== mds.0 v2:172.21.15.100:6833/207356836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55607868bfb0 con 0x55607873a360 2024-09-17T04:31:23.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.475+0000 7f779fe00640 10 client.7704 handle_command_reply: tid=0 2024-09-17T04:31:23.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.475+0000 7f77cce45740 1 client.7704 shutdown 2024-09-17T04:31:23.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.475+0000 7f77cce45740 1 -- 172.21.15.100:0/673078239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560782e6270 msgr2=0x556078739ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:23.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.475+0000 7f77cce45740 1 --2- 172.21.15.100:0/673078239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560782e6270 0x556078739ca0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f77b8018000 tx=0x7f77b8010720 comp rx=0 tx=0).stop 2024-09-17T04:31:23.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.475+0000 7f77cce45740 1 -- 172.21.15.100:0/673078239 shutdown_connections 2024-09-17T04:31:23.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.475+0000 7f77cce45740 1 --2- 172.21.15.100:0/673078239 >> [v2:172.21.15.100:6833/207356836,v1:172.21.15.100:6835/207356836] conn(0x55607873a360 0x556078754660 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.475+0000 7f77cce45740 1 --2- 172.21.15.100:0/673078239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560782e6270 0x556078739ca0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.475+0000 7f77cce45740 1 --2- 172.21.15.100:0/673078239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560782c5590 0x556078688170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.475+0000 7f77cce45740 1 --2- 172.21.15.100:0/673078239 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5560782bd3b0 0x5560786ab170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.475+0000 7f779fe00640 0 client.7704 ms_handle_reset on v2:172.21.15.100:6833/207356836 2024-09-17T04:31:23.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.475+0000 7f77cce45740 1 -- 172.21.15.100:0/673078239 >> 172.21.15.100:0/673078239 conn(0x5560782da3c0 msgr2=0x5560786d38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:23.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.475+0000 7f77cce45740 1 -- 172.21.15.100:0/673078239 shutdown_connections 2024-09-17T04:31:23.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.475+0000 7f77cce45740 1 -- 172.21.15.100:0/673078239 wait complete. 2024-09-17T04:31:23.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.475+0000 7f77cce45740 20 client.7704 trim_cache size 0 max 16384 2024-09-17T04:31:23.479 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:31:23.479 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "8a399842-3a5c-4ea2-9afb-b2704332e59f", 2024-09-17T04:31:23.479 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 0, 2024-09-17T04:31:23.480 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 7542, 2024-09-17T04:31:23.480 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-09-17T04:31:23.480 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-09-17T04:31:23.480 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-09-17T04:31:23.480 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 11.560287018, 2024-09-17T04:31:23.480 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 82, 2024-09-17T04:31:23.480 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 119, 2024-09-17T04:31:23.480 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 119, 2024-09-17T04:31:23.480 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 16.572434746999999 2024-09-17T04:31:23.480 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:31:23.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.476+0000 7f77c5c00640 1 -- 172.21.15.100:0/3406518820 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f77a0079010 msgr2=0x7f77a007b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:23.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.476+0000 7f77c5c00640 1 --2- 172.21.15.100:0/3406518820 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f77a0079010 0x7f77a007b520 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f77ac033000 tx=0x7f77ac004730 comp rx=0 tx=0).stop 2024-09-17T04:31:23.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.476+0000 7f77c5c00640 1 -- 172.21.15.100:0/3406518820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77c00705b0 msgr2=0x7f77c009a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:23.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.476+0000 7f77c5c00640 1 --2- 172.21.15.100:0/3406518820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77c00705b0 0x7f77c009a2d0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f77b000d6b0 tx=0x7f77b000db70 comp rx=0 tx=0).stop 2024-09-17T04:31:23.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.476+0000 7f77c5c00640 1 -- 172.21.15.100:0/3406518820 shutdown_connections 2024-09-17T04:31:23.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.476+0000 7f77c5c00640 1 --2- 172.21.15.100:0/3406518820 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f77a0079010 0x7f77a007b520 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.476+0000 7f77c5c00640 1 --2- 172.21.15.100:0/3406518820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77c015ce30 0x7f77c0073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.476+0000 7f77c5c00640 1 --2- 172.21.15.100:0/3406518820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77c00cfc80 0x7f77c01f5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.476+0000 7f77c5c00640 1 --2- 172.21.15.100:0/3406518820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77c00705b0 0x7f77c009a2d0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.476+0000 7f77c5c00640 1 -- 172.21.15.100:0/3406518820 >> 172.21.15.100:0/3406518820 conn(0x7f77c006d3d0 msgr2=0x7f77c00b3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:23.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.476+0000 7f77c5c00640 1 -- 172.21.15.100:0/3406518820 shutdown_connections 2024-09-17T04:31:23.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.476+0000 7f77c5c00640 1 -- 172.21.15.100:0/3406518820 wait complete. 2024-09-17T04:31:23.501 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:31:23.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.722+0000 7fcee8a00640 1 -- 172.21.15.100:0/2431521595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcee40b35c0 msgr2=0x7fcee40d2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:23.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.722+0000 7fcee8a00640 1 --2- 172.21.15.100:0/2431521595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcee40b35c0 0x7fcee40d2310 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fcee406d760 tx=0x7fcecc02a4a0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.722+0000 7fcee8a00640 1 -- 172.21.15.100:0/2431521595 shutdown_connections 2024-09-17T04:31:23.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.722+0000 7fcee8a00640 1 --2- 172.21.15.100:0/2431521595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcee40b35c0 0x7fcee40d2310 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.722+0000 7fcee8a00640 1 --2- 172.21.15.100:0/2431521595 >> v2:172.21.15.184:3300/0 conn(0x7fcee40b9840 0x7fcee40d4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.722+0000 7fcee8a00640 1 -- 172.21.15.100:0/2431521595 >> 172.21.15.100:0/2431521595 conn(0x7fcee40d44a0 msgr2=0x7fcee40d48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:23.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.722+0000 7fcee8a00640 1 -- 172.21.15.100:0/2431521595 shutdown_connections 2024-09-17T04:31:23.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.722+0000 7fcee8a00640 1 -- 172.21.15.100:0/2431521595 wait complete. 2024-09-17T04:31:23.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.724+0000 7fcee8a00640 1 Processor -- start 2024-09-17T04:31:23.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.724+0000 7fcee8a00640 1 -- start start 2024-09-17T04:31:23.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.726+0000 7fcee8a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcee40b35c0 0x7fcee40b6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:23.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.726+0000 7fcee8a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcee40b9840 0x7fcee40ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:23.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.726+0000 7fcee8a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcee40d2850 0x7fcee40ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:23.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.726+0000 7fcee8a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcee40d0e90 con 0x7fcee40b9840 2024-09-17T04:31:23.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.726+0000 7fcee8a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcee40d0d10 con 0x7fcee40d2850 2024-09-17T04:31:23.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.726+0000 7fcee8a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcee40d1010 con 0x7fcee40b35c0 2024-09-17T04:31:23.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.726+0000 7fcedaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcee40b9840 0x7fcee40ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:23.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.726+0000 7fcedaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcee40b9840 0x7fcee40ca0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55612/0 (socket says 172.21.15.100:55612) 2024-09-17T04:31:23.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.726+0000 7fcedaa00640 1 -- 172.21.15.100:0/2034397433 learned_addr learned my addr 172.21.15.100:0/2034397433 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:23.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.727+0000 7fcee3400640 1 --2- 172.21.15.100:0/2034397433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcee40b35c0 0x7fcee40b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:23.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.727+0000 7fcee3e00640 1 --2- 172.21.15.100:0/2034397433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcee40d2850 0x7fcee40ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:23.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.727+0000 7fcedaa00640 1 -- 172.21.15.100:0/2034397433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcee40b35c0 msgr2=0x7fcee40b6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:23.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.727+0000 7fcedaa00640 1 --2- 172.21.15.100:0/2034397433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcee40b35c0 0x7fcee40b6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.727+0000 7fcedaa00640 1 -- 172.21.15.100:0/2034397433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcee40d2850 msgr2=0x7fcee40ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:23.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.727+0000 7fcedaa00640 1 --2- 172.21.15.100:0/2034397433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcee40d2850 0x7fcee40ca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.727+0000 7fcedaa00640 1 -- 172.21.15.100:0/2034397433 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee4156d20 con 0x7fcee40b9840 2024-09-17T04:31:23.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.727+0000 7fcedaa00640 1 --2- 172.21.15.100:0/2034397433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcee40b9840 0x7fcee40ca0b0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0098f0 tx=0x7fcecc033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:23.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.727+0000 7fcedbe00640 1 -- 172.21.15.100:0/2034397433 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc031d60 con 0x7fcee40b9840 2024-09-17T04:31:23.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.727+0000 7fcedbe00640 1 -- 172.21.15.100:0/2034397433 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcecc033e10 con 0x7fcee40b9840 2024-09-17T04:31:23.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.727+0000 7fcedbe00640 1 -- 172.21.15.100:0/2034397433 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc005260 con 0x7fcee40b9840 2024-09-17T04:31:23.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.727+0000 7fcee8a00640 1 -- 172.21.15.100:0/2034397433 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee40b40e0 con 0x7fcee40b9840 2024-09-17T04:31:23.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.728+0000 7fcee8a00640 1 -- 172.21.15.100:0/2034397433 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee40c9bb0 con 0x7fcee40b9840 2024-09-17T04:31:23.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.728+0000 7fcee8a00640 1 -- 172.21.15.100:0/2034397433 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcee40b4600 con 0x7fcee40b9840 2024-09-17T04:31:23.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.730+0000 7fcedbe00640 1 -- 172.21.15.100:0/2034397433 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fcecc005400 con 0x7fcee40b9840 2024-09-17T04:31:23.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.731+0000 7fcedbe00640 1 --2- 172.21.15.100:0/2034397433 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcec0079010 0x7fcec007b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:23.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.731+0000 7fcedbe00640 1 -- 172.21.15.100:0/2034397433 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7fcecc0bc810 con 0x7fcee40b9840 2024-09-17T04:31:23.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.731+0000 7fcee3400640 1 --2- 172.21.15.100:0/2034397433 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcec0079010 0x7fcec007b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:23.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.732+0000 7fcee3400640 1 --2- 172.21.15.100:0/2034397433 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcec0079010 0x7fcec007b520 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fced4005fd0 tx=0x7fced40093f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:23.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.736+0000 7fcedbe00640 1 -- 172.21.15.100:0/2034397433 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fcee40b4600 con 0x7fcee40b9840 2024-09-17T04:31:23.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.954+0000 7fcee8a00640 1 -- 172.21.15.100:0/2034397433 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcee408cc70 con 0x7fcee40b9840 2024-09-17T04:31:23.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.955+0000 7fcedbe00640 1 -- 172.21.15.100:0/2034397433 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) ==== 53+0+73 (secure 0 0 0) 0x7fcee408cc70 con 0x7fcee40b9840 2024-09-17T04:31:23.955 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:31:23.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.957+0000 7fcee8a00640 1 -- 172.21.15.100:0/2034397433 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcec0079010 msgr2=0x7fcec007b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:23.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.957+0000 7fcee8a00640 1 --2- 172.21.15.100:0/2034397433 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcec0079010 0x7fcec007b520 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fced4005fd0 tx=0x7fced40093f0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.957+0000 7fcee8a00640 1 -- 172.21.15.100:0/2034397433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcee40b9840 msgr2=0x7fcee40ca0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:23.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.957+0000 7fcee8a00640 1 --2- 172.21.15.100:0/2034397433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcee40b9840 0x7fcee40ca0b0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0098f0 tx=0x7fcecc033c20 comp rx=0 tx=0).stop 2024-09-17T04:31:23.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.957+0000 7fcee8a00640 1 -- 172.21.15.100:0/2034397433 shutdown_connections 2024-09-17T04:31:23.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.957+0000 7fcee8a00640 1 --2- 172.21.15.100:0/2034397433 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcec0079010 0x7fcec007b520 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.957+0000 7fcee8a00640 1 --2- 172.21.15.100:0/2034397433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcee40d2850 0x7fcee40ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.957+0000 7fcee8a00640 1 --2- 172.21.15.100:0/2034397433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcee40b9840 0x7fcee40ca0b0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.957+0000 7fcee8a00640 1 --2- 172.21.15.100:0/2034397433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcee40b35c0 0x7fcee40b6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:23.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.957+0000 7fcee8a00640 1 -- 172.21.15.100:0/2034397433 >> 172.21.15.100:0/2034397433 conn(0x7fcee40d44a0 msgr2=0x7fcee40d48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:23.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.957+0000 7fcee8a00640 1 -- 172.21.15.100:0/2034397433 shutdown_connections 2024-09-17T04:31:23.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:23.957+0000 7fcee8a00640 1 -- 172.21.15.100:0/2034397433 wait complete. 2024-09-17T04:31:23.984 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T04:31:23.984 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:31:23.984 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-09-17T04:31:23.984 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T04:31:23.984 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T04:31:23.984 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T04:31:23.984 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T04:31:23.984 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:31:23.985 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:31:23.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:23.985 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-09-17T04:31:24.020 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:31:24.021 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:31:24.021 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:31:24.021 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:31:24.021 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-17T04:31:24.021 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:31:24.021 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:31:24.021 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:31:24.021 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-09-17T04:31:24.021 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:31:24.021 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:31:24.021 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-09-17T04:31:24.021 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:31:24.021 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:31:24.022 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:31:24.022 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:31:24.022 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T04:31:24.022 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42276sec preferred_lft 42276sec 2024-09-17T04:31:24.022 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link noprefixroute 2024-09-17T04:31:24.022 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:31:24.022 INFO:teuthology.orchestra.run.smithi100.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:31:24.022 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:2e:de:8e:a4:7c brd ff:ff:ff:ff:ff:ff 2024-09-17T04:31:24.022 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T04:31:24.022 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:31:24.022 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::cc78:e4ff:fe70:7fbe/64 scope link 2024-09-17T04:31:24.022 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:31:24.022 INFO:teuthology.orchestra.run.smithi100.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:31:24.022 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether ea:8c:45:9f:34:87 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:31:24.023 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::e88c:45ff:fe9f:3487/64 scope link 2024-09-17T04:31:24.023 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:31:24.023 INFO:teuthology.orchestra.run.smithi100.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:31:24.023 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:2e:de:8e:a4:7c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:31:24.023 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::c2e:deff:fe8e:a47c/64 scope link 2024-09-17T04:31:24.023 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:31:24.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:24.023 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:31:24.114 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-09-17T04:31:24.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:24.117 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-09-17T04:31:24.153 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-09-17T04:31:24.153 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:31:24.154 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T04:31:24.154 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T04:31:24.154 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:31:24.154 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:31:24.154 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-17T04:31:24.154 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T04:31:24.154 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T04:31:24.154 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T04:31:24.154 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-17T04:31:24.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:31:24.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:31:24.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:24.352 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-17T04:31:24.387 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T04:31:24.388 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T04:31:24.388 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:31:24.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:24.388 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-09-17T04:31:24.484 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:31:24.484 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T04:31:24.520 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:31:24.520 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:31:24.520 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:31:24.595 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T04:31:24.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:24.595 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:31:24.790 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:31:24.791 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:31:24.791 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:31:24.811 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:31:24.811 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:31:24.811 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:31:24.811 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:31:24.811 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:31:24.842 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:31:24.842 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:31:24.842 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:31:24.842 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:31:24.842 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:31:24.842 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-09-17T04:31:24.842 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-09-17T04:31:24.842 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8a399842-3a5c-4ea2-9afb-b2704332e59f" will pass to kernel 2024-09-17T04:31:24.844 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-17T04:31:24.844 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-09-17T04:31:24.844 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-17T04:31:24.844 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-09-17T04:31:24.844 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-17T04:31:24.846 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T04:31:24.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:24.847 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:31:24.903 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:31:24.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:31:24.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:31:24.998 INFO:teuthology.orchestra.run.smithi100.stdout:client7261 2024-09-17T04:31:24.999 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:31:24.999 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:31:25.035 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:31:25.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.294+0000 7fa097200640 1 -- 172.21.15.100:0/2021060217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0980ad9c0 msgr2=0x7fa0980af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:25.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.294+0000 7fa097200640 1 --2- 172.21.15.100:0/2021060217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0980ad9c0 0x7fa0980af170 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa088009830 tx=0x7fa08802a480 comp rx=0 tx=0).stop 2024-09-17T04:31:25.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.294+0000 7fa097200640 1 -- 172.21.15.100:0/2021060217 shutdown_connections 2024-09-17T04:31:25.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.294+0000 7fa097200640 1 --2- 172.21.15.100:0/2021060217 >> v2:172.21.15.184:3300/0 conn(0x7fa0980b2f60 0x7fa09815a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:25.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.294+0000 7fa097200640 1 --2- 172.21.15.100:0/2021060217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0980ad9c0 0x7fa0980af170 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:25.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.294+0000 7fa097200640 1 -- 172.21.15.100:0/2021060217 >> 172.21.15.100:0/2021060217 conn(0x7fa0980b1e40 msgr2=0x7fa0980b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:25.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.294+0000 7fa097200640 1 -- 172.21.15.100:0/2021060217 shutdown_connections 2024-09-17T04:31:25.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.294+0000 7fa097200640 1 -- 172.21.15.100:0/2021060217 wait complete. 2024-09-17T04:31:25.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.296+0000 7fa097200640 1 Processor -- start 2024-09-17T04:31:25.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.297+0000 7fa097200640 1 -- start start 2024-09-17T04:31:25.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.298+0000 7fa097200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0980ad9c0 0x7fa09814b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:25.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.298+0000 7fa097200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0980b2f60 0x7fa09814e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:25.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.298+0000 7fa097200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa09815d670 0x7fa09814e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:25.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.298+0000 7fa097200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa09806f410 con 0x7fa0980b2f60 2024-09-17T04:31:25.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.298+0000 7fa097200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa09806f290 con 0x7fa09815d670 2024-09-17T04:31:25.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.298+0000 7fa097200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa09806f590 con 0x7fa0980ad9c0 2024-09-17T04:31:25.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.298+0000 7fa08d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0980b2f60 0x7fa09814e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:25.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.298+0000 7fa08d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0980b2f60 0x7fa09814e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49318/0 (socket says 172.21.15.100:49318) 2024-09-17T04:31:25.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.298+0000 7fa08d400640 1 -- 172.21.15.100:0/4146013820 learned_addr learned my addr 172.21.15.100:0/4146013820 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:25.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.299+0000 7fa096800640 1 --2- 172.21.15.100:0/4146013820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa09815d670 0x7fa09814e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:25.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.299+0000 7fa095e00640 1 --2- 172.21.15.100:0/4146013820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0980ad9c0 0x7fa09814b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:25.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.299+0000 7fa08d400640 1 -- 172.21.15.100:0/4146013820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0980ad9c0 msgr2=0x7fa09814b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:25.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.299+0000 7fa08d400640 1 --2- 172.21.15.100:0/4146013820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0980ad9c0 0x7fa09814b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:25.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.299+0000 7fa08d400640 1 -- 172.21.15.100:0/4146013820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa09815d670 msgr2=0x7fa09814e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:25.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.299+0000 7fa08d400640 1 --2- 172.21.15.100:0/4146013820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa09815d670 0x7fa09814e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:25.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.299+0000 7fa08d400640 1 -- 172.21.15.100:0/4146013820 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa09814c340 con 0x7fa0980b2f60 2024-09-17T04:31:25.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.299+0000 7fa08d400640 1 --2- 172.21.15.100:0/4146013820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0980b2f60 0x7fa09814e210 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fa08000e770 tx=0x7fa08000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:25.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.299+0000 7fa08ea00640 1 -- 172.21.15.100:0/4146013820 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08000ccc0 con 0x7fa0980b2f60 2024-09-17T04:31:25.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.300+0000 7fa08ea00640 1 -- 172.21.15.100:0/4146013820 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa08000ee80 con 0x7fa0980b2f60 2024-09-17T04:31:25.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.300+0000 7fa097200640 1 -- 172.21.15.100:0/4146013820 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa09814c620 con 0x7fa0980b2f60 2024-09-17T04:31:25.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.300+0000 7fa08ea00640 1 -- 172.21.15.100:0/4146013820 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa080010690 con 0x7fa0980b2f60 2024-09-17T04:31:25.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.300+0000 7fa097200640 1 -- 172.21.15.100:0/4146013820 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa09809aae0 con 0x7fa0980b2f60 2024-09-17T04:31:25.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.301+0000 7fa097200640 1 -- 172.21.15.100:0/4146013820 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa09806fa20 con 0x7fa0980b2f60 2024-09-17T04:31:25.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.302+0000 7fa08ea00640 1 -- 172.21.15.100:0/4146013820 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fa080010830 con 0x7fa0980b2f60 2024-09-17T04:31:25.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.303+0000 7fa08ea00640 1 --2- 172.21.15.100:0/4146013820 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa074079040 0x7fa07407b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:25.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.304+0000 7fa08ea00640 1 -- 172.21.15.100:0/4146013820 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7fa08009bfb0 con 0x7fa0980b2f60 2024-09-17T04:31:25.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.304+0000 7fa095e00640 1 --2- 172.21.15.100:0/4146013820 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa074079040 0x7fa07407b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:25.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.304+0000 7fa095e00640 1 --2- 172.21.15.100:0/4146013820 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa074079040 0x7fa07407b550 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa088034000 tx=0x7fa088004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:25.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.308+0000 7fa08ea00640 1 -- 172.21.15.100:0/4146013820 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa080063600 con 0x7fa0980b2f60 2024-09-17T04:31:25.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.527+0000 7fa097200640 1 -- 172.21.15.100:0/4146013820 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa09815ac00 con 0x7fa0980b2f60 2024-09-17T04:31:25.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.528+0000 7fa08ea00640 1 -- 172.21.15.100:0/4146013820 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) ==== 53+0+73 (secure 0 0 0) 0x7fa080068030 con 0x7fa0980b2f60 2024-09-17T04:31:25.529 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:31:25.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.530+0000 7fa097200640 1 -- 172.21.15.100:0/4146013820 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa074079040 msgr2=0x7fa07407b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:25.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.530+0000 7fa097200640 1 --2- 172.21.15.100:0/4146013820 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa074079040 0x7fa07407b550 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa088034000 tx=0x7fa088004730 comp rx=0 tx=0).stop 2024-09-17T04:31:25.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.530+0000 7fa097200640 1 -- 172.21.15.100:0/4146013820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0980b2f60 msgr2=0x7fa09814e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:25.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.530+0000 7fa097200640 1 --2- 172.21.15.100:0/4146013820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0980b2f60 0x7fa09814e210 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fa08000e770 tx=0x7fa08000ec30 comp rx=0 tx=0).stop 2024-09-17T04:31:25.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.530+0000 7fa097200640 1 -- 172.21.15.100:0/4146013820 shutdown_connections 2024-09-17T04:31:25.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.530+0000 7fa097200640 1 --2- 172.21.15.100:0/4146013820 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa074079040 0x7fa07407b550 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:25.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.530+0000 7fa097200640 1 --2- 172.21.15.100:0/4146013820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa09815d670 0x7fa09814e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:25.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.530+0000 7fa097200640 1 --2- 172.21.15.100:0/4146013820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0980b2f60 0x7fa09814e210 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:25.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.530+0000 7fa097200640 1 --2- 172.21.15.100:0/4146013820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0980ad9c0 0x7fa09814b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:25.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.531+0000 7fa097200640 1 -- 172.21.15.100:0/4146013820 >> 172.21.15.100:0/4146013820 conn(0x7fa0980b1e40 msgr2=0x7fa0980b1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:25.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.531+0000 7fa097200640 1 -- 172.21.15.100:0/4146013820 shutdown_connections 2024-09-17T04:31:25.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:25.531+0000 7fa097200640 1 -- 172.21.15.100:0/4146013820 wait complete. 2024-09-17T04:31:25.552 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T04:31:25.553 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:31:25.553 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2024-09-17T04:31:25.553 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T04:31:25.553 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T04:31:25.553 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T04:31:25.553 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T04:31:25.553 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:31:25.553 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:31:25.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:25.553 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2024-09-17T04:31:25.595 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:31:25.595 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:31:25.595 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:31:25.595 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:31:25.595 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-09-17T04:31:25.595 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:31:25.595 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:31:25.595 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:31:25.596 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2024-09-17T04:31:25.596 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:31:25.596 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:31:25.596 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2024-09-17T04:31:25.596 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:31:25.596 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2024-09-17T04:31:25.596 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:31:25.596 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2024-09-17T04:31:25.596 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T04:31:25.596 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42258sec preferred_lft 42258sec 2024-09-17T04:31:25.596 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ec4:7aff:fe88:7b8f/64 scope link noprefixroute 2024-09-17T04:31:25.596 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:31:25.596 INFO:teuthology.orchestra.run.smithi184.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:31:25.597 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 7e:67:e5:69:96:69 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:31:25.597 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T04:31:25.597 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:31:25.597 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::3c9d:aff:fe6e:8fcb/64 scope link 2024-09-17T04:31:25.597 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:31:25.597 INFO:teuthology.orchestra.run.smithi184.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:31:25.597 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether a6:a9:1b:a9:61:ec brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T04:31:25.597 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::a4a9:1bff:fea9:61ec/64 scope link 2024-09-17T04:31:25.597 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:31:25.597 INFO:teuthology.orchestra.run.smithi184.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:31:25.597 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 7e:67:e5:69:96:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T04:31:25.597 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::7c67:e5ff:fe69:9669/64 scope link 2024-09-17T04:31:25.597 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:31:25.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:25.598 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:31:25.694 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-09-17T04:31:25.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:25.697 DEBUG:teuthology.orchestra.run.smithi184:> route 2024-09-17T04:31:25.734 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2024-09-17T04:31:25.734 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:31:25.734 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T04:31:25.734 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T04:31:25.734 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:31:25.735 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:31:25.735 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-09-17T04:31:25.735 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T04:31:25.735 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T04:31:25.735 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T04:31:25.735 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-09-17T04:31:25.852 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:31:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:31:25.923 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:31:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:31:25.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:25.931 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-09-17T04:31:25.967 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T04:31:25.967 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T04:31:25.967 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:31:25.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:25.967 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.0 up 2024-09-17T04:31:26.063 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:31:26.063 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T04:31:26.099 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:31:26.099 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:31:26.100 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T04:31:26.174 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T04:31:26.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:26.175 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:31:26.368 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:31:26.368 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:31:26.368 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:31:26.388 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:31:26.388 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:31:26.388 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:31:26.388 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:31:26.388 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:31:26.416 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:31:26.416 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:31:26.416 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:31:26.416 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:31:26.416 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:31:26.416 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-09-17T04:31:26.416 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-09-17T04:31:26.416 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8a399842-3a5c-4ea2-9afb-b2704332e59f" will pass to kernel 2024-09-17T04:31:26.417 INFO:teuthology.orchestra.run.smithi184.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-17T04:31:26.417 INFO:teuthology.orchestra.run.smithi184.stdout: You just mounted a file system that supports labels which does not 2024-09-17T04:31:26.418 INFO:teuthology.orchestra.run.smithi184.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-17T04:31:26.418 INFO:teuthology.orchestra.run.smithi184.stdout: applications will generate AVC messages and not be allowed access to 2024-09-17T04:31:26.418 INFO:teuthology.orchestra.run.smithi184.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-17T04:31:26.421 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T04:31:26.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:26.421 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T04:31:26.478 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:31:26.556 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:31:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:31:26.559 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:31:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:31:26.567 INFO:teuthology.orchestra.run.smithi184.stdout:client7273 2024-09-17T04:31:26.568 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:31:26.568 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:31:26.605 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T04:31:26.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.833+0000 7fcda9800640 1 -- 172.21.15.100:0/3503161642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda406cda0 msgr2=0x7fcda406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:26.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.833+0000 7fcda9800640 1 --2- 172.21.15.100:0/3503161642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda406cda0 0x7fcda406d1b0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c005fd0 tx=0x7fcd8c02a5d0 comp rx=0 tx=0).stop 2024-09-17T04:31:26.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.833+0000 7fcda9800640 1 -- 172.21.15.100:0/3503161642 shutdown_connections 2024-09-17T04:31:26.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.833+0000 7fcda9800640 1 --2- 172.21.15.100:0/3503161642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda406cda0 0x7fcda406d1b0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:26.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.833+0000 7fcda9800640 1 --2- 172.21.15.100:0/3503161642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcda40cc830 0x7fcda40b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:26.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.833+0000 7fcda9800640 1 -- 172.21.15.100:0/3503161642 >> 172.21.15.100:0/3503161642 conn(0x7fcda40b2540 msgr2=0x7fcda40d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:26.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.833+0000 7fcda9800640 1 -- 172.21.15.100:0/3503161642 shutdown_connections 2024-09-17T04:31:26.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.833+0000 7fcda9800640 1 -- 172.21.15.100:0/3503161642 wait complete. 2024-09-17T04:31:26.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.835+0000 7fcda9800640 1 Processor -- start 2024-09-17T04:31:26.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.835+0000 7fcda9800640 1 -- start start 2024-09-17T04:31:26.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.837+0000 7fcda9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda406cda0 0x7fcda41568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:26.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.837+0000 7fcda9800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcda40af100 0x7fcda41f31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:26.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.837+0000 7fcda9800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcda40cc830 0x7fcda40cf130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:26.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.837+0000 7fcda9800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcda40b3600 con 0x7fcda406cda0 2024-09-17T04:31:26.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.837+0000 7fcda9800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcda40b3480 con 0x7fcda40cc830 2024-09-17T04:31:26.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.837+0000 7fcda9800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcda40b3780 con 0x7fcda40af100 2024-09-17T04:31:26.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.837+0000 7fcda2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda406cda0 0x7fcda41568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:26.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.837+0000 7fcda2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda406cda0 0x7fcda41568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49330/0 (socket says 172.21.15.100:49330) 2024-09-17T04:31:26.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.837+0000 7fcda2a00640 1 -- 172.21.15.100:0/1489239117 learned_addr learned my addr 172.21.15.100:0/1489239117 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:26.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.837+0000 7fcda3400640 1 --2- 172.21.15.100:0/1489239117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcda40cc830 0x7fcda40cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:26.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.837+0000 7fcd9a000640 1 --2- 172.21.15.100:0/1489239117 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcda40af100 0x7fcda41f31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:26.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.837+0000 7fcda2a00640 1 -- 172.21.15.100:0/1489239117 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcda40af100 msgr2=0x7fcda41f31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:26.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.837+0000 7fcda2a00640 1 --2- 172.21.15.100:0/1489239117 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcda40af100 0x7fcda41f31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:26.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.837+0000 7fcda2a00640 1 -- 172.21.15.100:0/1489239117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcda40cc830 msgr2=0x7fcda40cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:26.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.837+0000 7fcda2a00640 1 --2- 172.21.15.100:0/1489239117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcda40cc830 0x7fcda40cf130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:26.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.837+0000 7fcda2a00640 1 -- 172.21.15.100:0/1489239117 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda41f9930 con 0x7fcda406cda0 2024-09-17T04:31:26.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.838+0000 7fcda2a00640 1 --2- 172.21.15.100:0/1489239117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda406cda0 0x7fcda41568a0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fcd9400e790 tx=0x7fcd9400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:26.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.838+0000 7fcd9b400640 1 -- 172.21.15.100:0/1489239117 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9400eea0 con 0x7fcda406cda0 2024-09-17T04:31:26.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.838+0000 7fcd9b400640 1 -- 172.21.15.100:0/1489239117 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd9400cdb0 con 0x7fcda406cda0 2024-09-17T04:31:26.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.838+0000 7fcda9800640 1 -- 172.21.15.100:0/1489239117 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda41f9c10 con 0x7fcda406cda0 2024-09-17T04:31:26.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.838+0000 7fcd9b400640 1 -- 172.21.15.100:0/1489239117 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd94010690 con 0x7fcda406cda0 2024-09-17T04:31:26.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.838+0000 7fcda9800640 1 -- 172.21.15.100:0/1489239117 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda41fa1d0 con 0x7fcda406cda0 2024-09-17T04:31:26.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.839+0000 7fcda9800640 1 -- 172.21.15.100:0/1489239117 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcda40b3ca0 con 0x7fcda406cda0 2024-09-17T04:31:26.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.840+0000 7fcd9b400640 1 -- 172.21.15.100:0/1489239117 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fcd94010830 con 0x7fcda406cda0 2024-09-17T04:31:26.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.842+0000 7fcd9b400640 1 --2- 172.21.15.100:0/1489239117 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcd80079040 0x7fcd8007b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:26.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.842+0000 7fcd9b400640 1 -- 172.21.15.100:0/1489239117 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7fcd9409bfd0 con 0x7fcda406cda0 2024-09-17T04:31:26.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.842+0000 7fcd9a000640 1 --2- 172.21.15.100:0/1489239117 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcd80079040 0x7fcd8007b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:26.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.843+0000 7fcd9a000640 1 --2- 172.21.15.100:0/1489239117 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcd80079040 0x7fcd8007b550 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c033000 tx=0x7fcd8c004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:26.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:26.846+0000 7fcd9b400640 1 -- 172.21.15.100:0/1489239117 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fcd940635d0 con 0x7fcda406cda0 2024-09-17T04:31:27.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.071+0000 7fcda9800640 1 -- 172.21.15.100:0/1489239117 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fcda40d1030 con 0x7fcda406cda0 2024-09-17T04:31:27.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.597+0000 7fcd9b400640 1 -- 172.21.15.100:0/1489239117 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v83) ==== 136+0+0 (secure 0 0 0) 0x7fcd94068000 con 0x7fcda406cda0 2024-09-17T04:31:27.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.600+0000 7fcda9800640 1 -- 172.21.15.100:0/1489239117 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcd80079040 msgr2=0x7fcd8007b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:27.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.600+0000 7fcda9800640 1 --2- 172.21.15.100:0/1489239117 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcd80079040 0x7fcd8007b550 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c033000 tx=0x7fcd8c004730 comp rx=0 tx=0).stop 2024-09-17T04:31:27.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.600+0000 7fcda9800640 1 -- 172.21.15.100:0/1489239117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda406cda0 msgr2=0x7fcda41568a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:27.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.600+0000 7fcda9800640 1 --2- 172.21.15.100:0/1489239117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda406cda0 0x7fcda41568a0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fcd9400e790 tx=0x7fcd9400ec50 comp rx=0 tx=0).stop 2024-09-17T04:31:27.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.600+0000 7fcda9800640 1 -- 172.21.15.100:0/1489239117 shutdown_connections 2024-09-17T04:31:27.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.600+0000 7fcda9800640 1 --2- 172.21.15.100:0/1489239117 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcd80079040 0x7fcd8007b550 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:27.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.600+0000 7fcda9800640 1 --2- 172.21.15.100:0/1489239117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcda40cc830 0x7fcda40cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:27.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.600+0000 7fcda9800640 1 --2- 172.21.15.100:0/1489239117 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcda40af100 0x7fcda41f31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:27.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.600+0000 7fcda9800640 1 --2- 172.21.15.100:0/1489239117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda406cda0 0x7fcda41568a0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:27.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.600+0000 7fcda9800640 1 -- 172.21.15.100:0/1489239117 >> 172.21.15.100:0/1489239117 conn(0x7fcda40b2540 msgr2=0x7fcda40b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:27.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.600+0000 7fcda9800640 1 -- 172.21.15.100:0/1489239117 shutdown_connections 2024-09-17T04:31:27.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.600+0000 7fcda9800640 1 -- 172.21.15.100:0/1489239117 wait complete. 2024-09-17T04:31:27.627 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T04:31:27.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.847+0000 7f05b3000640 1 -- 172.21.15.100:0/295340967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05ac0cfc80 msgr2=0x7f05ac156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:27.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.847+0000 7f05b3000640 1 --2- 172.21.15.100:0/295340967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05ac0cfc80 0x7f05ac156e40 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f059c005fd0 tx=0x7f059c02a540 comp rx=0 tx=0).stop 2024-09-17T04:31:27.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.847+0000 7f05b3000640 1 -- 172.21.15.100:0/295340967 shutdown_connections 2024-09-17T04:31:27.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.847+0000 7f05b3000640 1 --2- 172.21.15.100:0/295340967 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05ac0705b0 0x7f05ac0709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:27.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.847+0000 7f05b3000640 1 --2- 172.21.15.100:0/295340967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05ac0cfc80 0x7f05ac156e40 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:27.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.847+0000 7f05b3000640 1 -- 172.21.15.100:0/295340967 >> 172.21.15.100:0/295340967 conn(0x7f05ac06d3d0 msgr2=0x7f05ac0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:27.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.847+0000 7f05b3000640 1 -- 172.21.15.100:0/295340967 shutdown_connections 2024-09-17T04:31:27.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.847+0000 7f05b3000640 1 -- 172.21.15.100:0/295340967 wait complete. 2024-09-17T04:31:27.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.850+0000 7f05b3000640 1 Processor -- start 2024-09-17T04:31:27.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.850+0000 7f05b3000640 1 -- start start 2024-09-17T04:31:27.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.851+0000 7f05b3000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05ac0705b0 0x7f05ac157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:27.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.851+0000 7f05b3000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05ac0cfc80 0x7f05ac0743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:27.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.851+0000 7f05b3000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05ac15ce30 0x7f05ac155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:27.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.851+0000 7f05b3000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac0cced0 con 0x7f05ac0705b0 2024-09-17T04:31:27.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.851+0000 7f05b3000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac0d0170 con 0x7f05ac0cfc80 2024-09-17T04:31:27.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.851+0000 7f05b3000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f05ac0cd050 con 0x7f05ac15ce30 2024-09-17T04:31:27.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.851+0000 7f05abe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05ac0705b0 0x7f05ac157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:27.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.851+0000 7f05abe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05ac0705b0 0x7f05ac157be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49344/0 (socket says 172.21.15.100:49344) 2024-09-17T04:31:27.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.852+0000 7f05abe00640 1 -- 172.21.15.100:0/3329590504 learned_addr learned my addr 172.21.15.100:0/3329590504 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:27.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.852+0000 7f05a3e00640 1 --2- 172.21.15.100:0/3329590504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05ac0cfc80 0x7f05ac0743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:27.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.852+0000 7f05b1000640 1 --2- 172.21.15.100:0/3329590504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05ac15ce30 0x7f05ac155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:27.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.852+0000 7f05abe00640 1 -- 172.21.15.100:0/3329590504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05ac15ce30 msgr2=0x7f05ac155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:27.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.852+0000 7f05abe00640 1 --2- 172.21.15.100:0/3329590504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05ac15ce30 0x7f05ac155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:27.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.852+0000 7f05abe00640 1 -- 172.21.15.100:0/3329590504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05ac0cfc80 msgr2=0x7f05ac0743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:27.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.852+0000 7f05abe00640 1 --2- 172.21.15.100:0/3329590504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05ac0cfc80 0x7f05ac0743b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:27.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.852+0000 7f05abe00640 1 -- 172.21.15.100:0/3329590504 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ac0c52b0 con 0x7f05ac0705b0 2024-09-17T04:31:27.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.852+0000 7f05abe00640 1 --2- 172.21.15.100:0/3329590504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05ac0705b0 0x7f05ac157be0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f059c034040 tx=0x7f059c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:27.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.853+0000 7f05a8c00640 1 -- 172.21.15.100:0/3329590504 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c03e030 con 0x7f05ac0705b0 2024-09-17T04:31:27.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.853+0000 7f05a8c00640 1 -- 172.21.15.100:0/3329590504 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f059c003db0 con 0x7f05ac0705b0 2024-09-17T04:31:27.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.853+0000 7f05a8c00640 1 -- 172.21.15.100:0/3329590504 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c035040 con 0x7f05ac0705b0 2024-09-17T04:31:27.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.853+0000 7f05b3000640 1 -- 172.21.15.100:0/3329590504 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05ac1586a0 con 0x7f05ac0705b0 2024-09-17T04:31:27.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.853+0000 7f05b3000640 1 -- 172.21.15.100:0/3329590504 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05ac158c60 con 0x7f05ac0705b0 2024-09-17T04:31:27.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.854+0000 7f05b3000640 1 -- 172.21.15.100:0/3329590504 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05ac0cd570 con 0x7f05ac0705b0 2024-09-17T04:31:27.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.855+0000 7f05a8c00640 1 -- 172.21.15.100:0/3329590504 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f059c041030 con 0x7f05ac0705b0 2024-09-17T04:31:27.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.856+0000 7f05a8c00640 1 --2- 172.21.15.100:0/3329590504 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0588079010 0x7f058807b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:27.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.856+0000 7f05a8c00640 1 -- 172.21.15.100:0/3329590504 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7f059c0bbd10 con 0x7f05ac0705b0 2024-09-17T04:31:27.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.857+0000 7f05a3e00640 1 --2- 172.21.15.100:0/3329590504 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0588079010 0x7f058807b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:27.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.857+0000 7f05a3e00640 1 --2- 172.21.15.100:0/3329590504 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0588079010 0x7f058807b520 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0c5020 tx=0x7f0594005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:27.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:27.861+0000 7f05a8c00640 1 -- 172.21.15.100:0/3329590504 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f059c083310 con 0x7f05ac0705b0 2024-09-17T04:31:28.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.074+0000 7f05b3000640 1 -- 172.21.15.100:0/3329590504 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f05ac15c870 con 0x7f0588079010 2024-09-17T04:31:28.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.076+0000 7f05a8c00640 1 -- 172.21.15.100:0/3329590504 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f05ac15c870 con 0x7f0588079010 2024-09-17T04:31:28.076 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-09-17T04:31:28.076 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:28.077 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephfs" 2024-09-17T04:31:28.077 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:28.077 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-09-17T04:31:28.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.079+0000 7f05b3000640 1 -- 172.21.15.100:0/3329590504 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0588079010 msgr2=0x7f058807b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:28.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.079+0000 7f05b3000640 1 --2- 172.21.15.100:0/3329590504 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0588079010 0x7f058807b520 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0c5020 tx=0x7f0594005f90 comp rx=0 tx=0).stop 2024-09-17T04:31:28.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.079+0000 7f05b3000640 1 -- 172.21.15.100:0/3329590504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05ac0705b0 msgr2=0x7f05ac157be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:28.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.079+0000 7f05b3000640 1 --2- 172.21.15.100:0/3329590504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05ac0705b0 0x7f05ac157be0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f059c034040 tx=0x7f059c0038a0 comp rx=0 tx=0).stop 2024-09-17T04:31:28.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.079+0000 7f05b3000640 1 -- 172.21.15.100:0/3329590504 shutdown_connections 2024-09-17T04:31:28.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.079+0000 7f05b3000640 1 --2- 172.21.15.100:0/3329590504 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0588079010 0x7f058807b520 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:28.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.079+0000 7f05b3000640 1 --2- 172.21.15.100:0/3329590504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05ac15ce30 0x7f05ac155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:28.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.079+0000 7f05b3000640 1 --2- 172.21.15.100:0/3329590504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05ac0cfc80 0x7f05ac0743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:28.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.079+0000 7f05b3000640 1 --2- 172.21.15.100:0/3329590504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05ac0705b0 0x7f05ac157be0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:28.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.079+0000 7f05b3000640 1 -- 172.21.15.100:0/3329590504 >> 172.21.15.100:0/3329590504 conn(0x7f05ac06d3d0 msgr2=0x7f05ac0ad7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:28.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.079+0000 7f05b3000640 1 -- 172.21.15.100:0/3329590504 shutdown_connections 2024-09-17T04:31:28.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.079+0000 7f05b3000640 1 -- 172.21.15.100:0/3329590504 wait complete. 2024-09-17T04:31:28.101 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T04:31:28.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.323+0000 7fb59d400640 1 -- 172.21.15.100:0/1936270128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb59815f9e0 msgr2=0x7fb598147b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:28.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.323+0000 7fb59d400640 1 --2- 172.21.15.100:0/1936270128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb59815f9e0 0x7fb598147b90 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fb588009830 tx=0x7fb58802a3f0 comp rx=0 tx=0).stop 2024-09-17T04:31:28.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.324+0000 7fb59d400640 1 -- 172.21.15.100:0/1936270128 shutdown_connections 2024-09-17T04:31:28.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.324+0000 7fb59d400640 1 --2- 172.21.15.100:0/1936270128 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5980aee00 0x7fb5980af210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:28.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.324+0000 7fb59d400640 1 --2- 172.21.15.100:0/1936270128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb59815f9e0 0x7fb598147b90 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:28.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.324+0000 7fb59d400640 1 -- 172.21.15.100:0/1936270128 >> 172.21.15.100:0/1936270128 conn(0x7fb59806d1c0 msgr2=0x7fb59806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:28.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.324+0000 7fb59d400640 1 -- 172.21.15.100:0/1936270128 shutdown_connections 2024-09-17T04:31:28.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.324+0000 7fb59d400640 1 -- 172.21.15.100:0/1936270128 wait complete. 2024-09-17T04:31:28.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.326+0000 7fb59d400640 1 Processor -- start 2024-09-17T04:31:28.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.326+0000 7fb59d400640 1 -- start start 2024-09-17T04:31:28.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.327+0000 7fb59d400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb598073540 0x7fb598141ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:28.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.327+0000 7fb59d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5980aee00 0x7fb598143e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:28.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.327+0000 7fb59d400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb59815f9e0 0x7fb598142dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:28.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.327+0000 7fb59d400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb59806fb60 con 0x7fb5980aee00 2024-09-17T04:31:28.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.327+0000 7fb59d400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb59806f9e0 con 0x7fb598073540 2024-09-17T04:31:28.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.327+0000 7fb59d400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb59806fce0 con 0x7fb59815f9e0 2024-09-17T04:31:28.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.327+0000 7fb58fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5980aee00 0x7fb598143e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:28.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.328+0000 7fb58fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5980aee00 0x7fb598143e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49360/0 (socket says 172.21.15.100:49360) 2024-09-17T04:31:28.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.328+0000 7fb58fe00640 1 -- 172.21.15.100:0/838434094 learned_addr learned my addr 172.21.15.100:0/838434094 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:28.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.328+0000 7fb596800640 1 --2- 172.21.15.100:0/838434094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb598073540 0x7fb598141ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:28.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.328+0000 7fb597200640 1 --2- 172.21.15.100:0/838434094 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb59815f9e0 0x7fb598142dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:28.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.328+0000 7fb58fe00640 1 -- 172.21.15.100:0/838434094 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb59815f9e0 msgr2=0x7fb598142dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:28.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.328+0000 7fb58fe00640 1 --2- 172.21.15.100:0/838434094 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb59815f9e0 0x7fb598142dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:28.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.328+0000 7fb58fe00640 1 -- 172.21.15.100:0/838434094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb598073540 msgr2=0x7fb598141ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:28.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.328+0000 7fb58fe00640 1 --2- 172.21.15.100:0/838434094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb598073540 0x7fb598141ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:28.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.328+0000 7fb58fe00640 1 -- 172.21.15.100:0/838434094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb598143300 con 0x7fb5980aee00 2024-09-17T04:31:28.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.328+0000 7fb58fe00640 1 --2- 172.21.15.100:0/838434094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5980aee00 0x7fb598143e60 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fb58000e790 tx=0x7fb58000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:28.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.328+0000 7fb58ea00640 1 -- 172.21.15.100:0/838434094 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58000eea0 con 0x7fb5980aee00 2024-09-17T04:31:28.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.328+0000 7fb58ea00640 1 -- 172.21.15.100:0/838434094 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb58000ccc0 con 0x7fb5980aee00 2024-09-17T04:31:28.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.328+0000 7fb58ea00640 1 -- 172.21.15.100:0/838434094 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580005280 con 0x7fb5980aee00 2024-09-17T04:31:28.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.328+0000 7fb59d400640 1 -- 172.21.15.100:0/838434094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5981f5d20 con 0x7fb5980aee00 2024-09-17T04:31:28.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.329+0000 7fb59d400640 1 -- 172.21.15.100:0/838434094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5981f62e0 con 0x7fb5980aee00 2024-09-17T04:31:28.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.330+0000 7fb59d400640 1 -- 172.21.15.100:0/838434094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5981f5a90 con 0x7fb5980aee00 2024-09-17T04:31:28.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.331+0000 7fb58ea00640 1 -- 172.21.15.100:0/838434094 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fb580005420 con 0x7fb5980aee00 2024-09-17T04:31:28.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.332+0000 7fb58ea00640 1 --2- 172.21.15.100:0/838434094 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb574079010 0x7fb57407b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:28.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.332+0000 7fb58ea00640 1 -- 172.21.15.100:0/838434094 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7fb58009bea0 con 0x7fb5980aee00 2024-09-17T04:31:28.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.332+0000 7fb596800640 1 --2- 172.21.15.100:0/838434094 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb574079010 0x7fb57407b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:28.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.333+0000 7fb596800640 1 --2- 172.21.15.100:0/838434094 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb574079010 0x7fb57407b520 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb588034000 tx=0x7fb588004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:28.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.337+0000 7fb58ea00640 1 -- 172.21.15.100:0/838434094 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb598143300 con 0x7fb5980aee00 2024-09-17T04:31:28.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.520+0000 7fb59d400640 1 -- 172.21.15.100:0/838434094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb5980af210 con 0x7fb5980aee00 2024-09-17T04:31:28.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.521+0000 7fb58ea00640 1 -- 172.21.15.100:0/838434094 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v44)=0 v44) ==== 122+0+0 (secure 0 0 0) 0x7fb580063460 con 0x7fb5980aee00 2024-09-17T04:31:28.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.523+0000 7fb59d400640 1 -- 172.21.15.100:0/838434094 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb574079010 msgr2=0x7fb57407b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:28.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.523+0000 7fb59d400640 1 --2- 172.21.15.100:0/838434094 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb574079010 0x7fb57407b520 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb588034000 tx=0x7fb588004730 comp rx=0 tx=0).stop 2024-09-17T04:31:28.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.523+0000 7fb59d400640 1 -- 172.21.15.100:0/838434094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5980aee00 msgr2=0x7fb598143e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:28.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.523+0000 7fb59d400640 1 --2- 172.21.15.100:0/838434094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5980aee00 0x7fb598143e60 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fb58000e790 tx=0x7fb58000ec50 comp rx=0 tx=0).stop 2024-09-17T04:31:28.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.523+0000 7fb59d400640 1 -- 172.21.15.100:0/838434094 shutdown_connections 2024-09-17T04:31:28.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.523+0000 7fb59d400640 1 --2- 172.21.15.100:0/838434094 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb574079010 0x7fb57407b520 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:28.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.523+0000 7fb59d400640 1 --2- 172.21.15.100:0/838434094 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb59815f9e0 0x7fb598142dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:28.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.523+0000 7fb59d400640 1 --2- 172.21.15.100:0/838434094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5980aee00 0x7fb598143e60 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:28.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.523+0000 7fb59d400640 1 --2- 172.21.15.100:0/838434094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb598073540 0x7fb598141ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:28.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.523+0000 7fb59d400640 1 -- 172.21.15.100:0/838434094 >> 172.21.15.100:0/838434094 conn(0x7fb59806d1c0 msgr2=0x7fb598068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:28.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.523+0000 7fb59d400640 1 -- 172.21.15.100:0/838434094 shutdown_connections 2024-09-17T04:31:28.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:28.523+0000 7fb59d400640 1 -- 172.21.15.100:0/838434094 wait complete. 2024-09-17T04:31:28.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:28.545 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol39090/23dc80c1-362e-475b-a414-7ba4509a30ff) 2024-09-17T04:31:28.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:28.611 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol39090/23dc80c1-362e-475b-a414-7ba4509a30ff/.snap/fake) 2024-09-17T04:31:28.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:28.698 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-17T04:31:28.734 INFO:teuthology.orchestra.run.smithi100.stdout:cephfs_data 2024-09-17T04:31:28.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:28.734 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol39090/23dc80c1-362e-475b-a414-7ba4509a30ff) 2024-09-17T04:31:28.836 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:31:28.836 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol39090/.meta 2024-09-17T04:31:28.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:28.896 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp33350/subvol11481/0d0b12bb-592d-48f7-b31f-ded447325aee) 2024-09-17T04:31:28.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:28.996 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp33350/subvol11481/0d0b12bb-592d-48f7-b31f-ded447325aee/.snap/fake) 2024-09-17T04:31:29.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:29.081 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-17T04:31:29.116 INFO:teuthology.orchestra.run.smithi100.stdout:cephfs_data 2024-09-17T04:31:29.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:29.117 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/subvol_grp33350/subvol11481/0d0b12bb-592d-48f7-b31f-ded447325aee) 2024-09-17T04:31:29.218 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:31:29.218 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/subvol_grp33350/subvol11481/.meta 2024-09-17T04:31:29.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:29.274 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol41972/ed59f3df-09cd-4703-820c-9463bc08905f) 2024-09-17T04:31:29.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:29.373 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-17T04:31:29.409 INFO:teuthology.orchestra.run.smithi100.stdout:cephfs_data 2024-09-17T04:31:29.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:29.409 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol41972/ed59f3df-09cd-4703-820c-9463bc08905f) 2024-09-17T04:31:29.510 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:31:29.510 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol41972/.meta 2024-09-17T04:31:29.567 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol39090 2024-09-17T04:31:29.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.833+0000 7f9fc6400640 1 -- 172.21.15.100:0/1157974888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fa40041c0 msgr2=0x7f9fa4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:29.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.833+0000 7f9fc6400640 1 --2- 172.21.15.100:0/1157974888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fa40041c0 0x7f9fa4004800 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f9fc01613e0 tx=0x7f9fa4005830 comp rx=0 tx=0).stop 2024-09-17T04:31:29.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.833+0000 7f9fbfe00640 1 -- 172.21.15.100:0/1157974888 reap_dead start 2024-09-17T04:31:29.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.833+0000 7f9fc6400640 1 -- 172.21.15.100:0/1157974888 shutdown_connections 2024-09-17T04:31:29.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.833+0000 7f9fc6400640 1 -- 172.21.15.100:0/1157974888 >> 172.21.15.100:0/1157974888 conn(0x7f9fc006d1c0 msgr2=0x7f9fc006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:29.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.833+0000 7f9fc6400640 1 -- 172.21.15.100:0/1157974888 shutdown_connections 2024-09-17T04:31:29.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.833+0000 7f9fc6400640 1 -- 172.21.15.100:0/1157974888 wait complete. 2024-09-17T04:31:29.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.836+0000 7f9fc6400640 1 Processor -- start 2024-09-17T04:31:29.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.836+0000 7f9fc6400640 1 -- start start 2024-09-17T04:31:29.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.837+0000 7f9fc6400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fa40041c0 0x7f9fc00a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:29.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.837+0000 7f9fc6400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc015c030 0x7f9fc0144720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:29.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.837+0000 7f9fc6400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fc0144c60 0x7f9fc01f3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:29.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.837+0000 7f9fc6400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc00cdea0 con 0x7f9fc015c030 2024-09-17T04:31:29.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.837+0000 7f9fc6400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc00cdd20 con 0x7f9fa40041c0 2024-09-17T04:31:29.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.837+0000 7f9fc6400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc00ce020 con 0x7f9fc0144c60 2024-09-17T04:31:29.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.837+0000 7f9fb6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc015c030 0x7f9fc0144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:29.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.837+0000 7f9fb6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc015c030 0x7f9fc0144720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49374/0 (socket says 172.21.15.100:49374) 2024-09-17T04:31:29.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.837+0000 7f9fb6a00640 1 -- 172.21.15.100:0/1257120167 learned_addr learned my addr 172.21.15.100:0/1257120167 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:29.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.837+0000 7f9fbfe00640 1 --2- 172.21.15.100:0/1257120167 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fc0144c60 0x7f9fc01f3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:29.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.837+0000 7f9fbf400640 1 --2- 172.21.15.100:0/1257120167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fa40041c0 0x7f9fc00a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:29.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.838+0000 7f9fb6a00640 1 -- 172.21.15.100:0/1257120167 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fc0144c60 msgr2=0x7f9fc01f3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:29.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.838+0000 7f9fb6a00640 1 --2- 172.21.15.100:0/1257120167 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fc0144c60 0x7f9fc01f3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:29.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.838+0000 7f9fb6a00640 1 -- 172.21.15.100:0/1257120167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fa40041c0 msgr2=0x7f9fc00a95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:29.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.838+0000 7f9fb6a00640 1 --2- 172.21.15.100:0/1257120167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fa40041c0 0x7f9fc00a95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:29.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.838+0000 7f9fb6a00640 1 -- 172.21.15.100:0/1257120167 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc0143960 con 0x7f9fc015c030 2024-09-17T04:31:29.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.838+0000 7f9fb6a00640 1 --2- 172.21.15.100:0/1257120167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc015c030 0x7f9fc0144720 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f9fa802a990 tx=0x7f9fa8006310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:29.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.838+0000 7f9fb7e00640 1 -- 172.21.15.100:0/1257120167 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8038070 con 0x7f9fc015c030 2024-09-17T04:31:29.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.838+0000 7f9fb7e00640 1 -- 172.21.15.100:0/1257120167 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fa8003af0 con 0x7f9fc015c030 2024-09-17T04:31:29.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.838+0000 7f9fb7e00640 1 -- 172.21.15.100:0/1257120167 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8033420 con 0x7f9fc015c030 2024-09-17T04:31:29.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.838+0000 7f9fc6400640 1 -- 172.21.15.100:0/1257120167 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc0143c40 con 0x7f9fc015c030 2024-09-17T04:31:29.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.839+0000 7f9fc6400640 1 -- 172.21.15.100:0/1257120167 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc00a9cf0 con 0x7f9fc015c030 2024-09-17T04:31:29.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.840+0000 7f9fc6400640 1 -- 172.21.15.100:0/1257120167 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fc00ce5b0 con 0x7f9fc015c030 2024-09-17T04:31:29.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.841+0000 7f9fb7e00640 1 -- 172.21.15.100:0/1257120167 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f9fa8043020 con 0x7f9fc015c030 2024-09-17T04:31:29.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.842+0000 7f9fb7e00640 1 --2- 172.21.15.100:0/1257120167 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9f9c079010 0x7f9f9c07b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:29.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.842+0000 7f9fb7e00640 1 -- 172.21.15.100:0/1257120167 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7f9fa80bac30 con 0x7f9fc015c030 2024-09-17T04:31:29.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.842+0000 7f9fbf400640 1 --2- 172.21.15.100:0/1257120167 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9f9c079010 0x7f9f9c07b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:29.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.842+0000 7f9fbf400640 1 --2- 172.21.15.100:0/1257120167 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9f9c079010 0x7f9f9c07b520 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009bb0 tx=0x7f9fb0005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:29.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:29.846+0000 7f9fb7e00640 1 -- 172.21.15.100:0/1257120167 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9fa8082ac0 con 0x7f9fc015c030 2024-09-17T04:31:30.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.055+0000 7f9fc6400640 1 -- 172.21.15.100:0/1257120167 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol39090", "target": ["mon-mgr", ""]}) -- 0x7f9fc00a9b20 con 0x7f9f9c079010 2024-09-17T04:31:30.063 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:31:30.063+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:31:30.063 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:31:30.063+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:31:30.064 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:31:30.063+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:31:30.064 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:31:30.063+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:31:30.064 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:31:30.063+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:31:30.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.108+0000 7f9fb7e00640 1 -- 172.21.15.100:0/1257120167 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f9fc00a9b20 con 0x7f9f9c079010 2024-09-17T04:31:30.109 INFO:teuthology.orchestra.run.smithi100.stdout:/volumes/_nogroup/subvol39090/23dc80c1-362e-475b-a414-7ba4509a30ff 2024-09-17T04:31:30.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.110+0000 7f9fc6400640 1 -- 172.21.15.100:0/1257120167 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9f9c079010 msgr2=0x7f9f9c07b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:30.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.110+0000 7f9fc6400640 1 --2- 172.21.15.100:0/1257120167 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9f9c079010 0x7f9f9c07b520 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009bb0 tx=0x7f9fb0005f90 comp rx=0 tx=0).stop 2024-09-17T04:31:30.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.111+0000 7f9fc6400640 1 -- 172.21.15.100:0/1257120167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc015c030 msgr2=0x7f9fc0144720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:30.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.111+0000 7f9fc6400640 1 --2- 172.21.15.100:0/1257120167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc015c030 0x7f9fc0144720 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f9fa802a990 tx=0x7f9fa8006310 comp rx=0 tx=0).stop 2024-09-17T04:31:30.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.111+0000 7f9fc6400640 1 -- 172.21.15.100:0/1257120167 shutdown_connections 2024-09-17T04:31:30.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.111+0000 7f9fc6400640 1 --2- 172.21.15.100:0/1257120167 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9f9c079010 0x7f9f9c07b520 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:30.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.111+0000 7f9fc6400640 1 --2- 172.21.15.100:0/1257120167 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fc0144c60 0x7f9fc01f3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:30.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.111+0000 7f9fc6400640 1 --2- 172.21.15.100:0/1257120167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc015c030 0x7f9fc0144720 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:30.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.111+0000 7f9fc6400640 1 --2- 172.21.15.100:0/1257120167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fa40041c0 0x7f9fc00a95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:30.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.111+0000 7f9fc6400640 1 -- 172.21.15.100:0/1257120167 >> 172.21.15.100:0/1257120167 conn(0x7f9fc006d1c0 msgr2=0x7f9fc00b2430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:30.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.111+0000 7f9fc6400640 1 -- 172.21.15.100:0/1257120167 shutdown_connections 2024-09-17T04:31:30.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.111+0000 7f9fc6400640 1 -- 172.21.15.100:0/1257120167 wait complete. 2024-09-17T04:31:30.135 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol11481 subvol_grp33350 2024-09-17T04:31:30.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.359+0000 7f8565200640 1 -- 172.21.15.100:0/4042238761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f856006cda0 msgr2=0x7f856006d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:30.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.359+0000 7f8565200640 1 --2- 172.21.15.100:0/4042238761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f856006cda0 0x7f856006d1b0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f85600d01c0 tx=0x7f854802a560 comp rx=0 tx=0).stop 2024-09-17T04:31:30.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.359+0000 7f8565200640 1 -- 172.21.15.100:0/4042238761 shutdown_connections 2024-09-17T04:31:30.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.359+0000 7f8565200640 1 --2- 172.21.15.100:0/4042238761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f856006cda0 0x7f856006d1b0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:30.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.359+0000 7f8565200640 1 --2- 172.21.15.100:0/4042238761 >> v2:172.21.15.184:3300/0 conn(0x7f85600cc830 0x7f85600b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:30.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.359+0000 7f8565200640 1 -- 172.21.15.100:0/4042238761 >> 172.21.15.100:0/4042238761 conn(0x7f85600b2540 msgr2=0x7f85600d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:30.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.359+0000 7f8565200640 1 -- 172.21.15.100:0/4042238761 shutdown_connections 2024-09-17T04:31:30.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.359+0000 7f8565200640 1 -- 172.21.15.100:0/4042238761 wait complete. 2024-09-17T04:31:30.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.361+0000 7f8565200640 1 Processor -- start 2024-09-17T04:31:30.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.361+0000 7f8565200640 1 -- start start 2024-09-17T04:31:30.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.363+0000 7f8565200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f856006cda0 0x7f85600985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:30.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.363+0000 7f8565200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85600af100 0x7f85601e6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:30.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.363+0000 7f8565200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85600cc830 0x7f85601e6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:30.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.363+0000 7f8565200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f85600b3620 con 0x7f85600cc830 2024-09-17T04:31:30.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.363+0000 7f8565200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f85600b34a0 con 0x7f85600af100 2024-09-17T04:31:30.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.363+0000 7f8565200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f85600b37a0 con 0x7f856006cda0 2024-09-17T04:31:30.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.363+0000 7f855e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85600cc830 0x7f85601e6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:30.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.363+0000 7f855e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85600cc830 0x7f85601e6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49410/0 (socket says 172.21.15.100:49410) 2024-09-17T04:31:30.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.363+0000 7f855e800640 1 -- 172.21.15.100:0/2277571941 learned_addr learned my addr 172.21.15.100:0/2277571941 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:30.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.363+0000 7f8555400640 1 --2- 172.21.15.100:0/2277571941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85600af100 0x7f85601e6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:30.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.363+0000 7f855de00640 1 --2- 172.21.15.100:0/2277571941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f856006cda0 0x7f85600985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:30.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.363+0000 7f855e800640 1 -- 172.21.15.100:0/2277571941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f856006cda0 msgr2=0x7f85600985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:30.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.363+0000 7f855e800640 1 --2- 172.21.15.100:0/2277571941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f856006cda0 0x7f85600985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:30.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.363+0000 7f855e800640 1 -- 172.21.15.100:0/2277571941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85600af100 msgr2=0x7f85601e6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:30.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.363+0000 7f855e800640 1 --2- 172.21.15.100:0/2277571941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85600af100 0x7f85601e6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:30.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.363+0000 7f855e800640 1 -- 172.21.15.100:0/2277571941 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8560098b30 con 0x7f85600cc830 2024-09-17T04:31:30.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.364+0000 7f855e800640 1 --2- 172.21.15.100:0/2277571941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85600cc830 0x7f85601e6ba0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f855800ba10 tx=0x7f855800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:30.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.364+0000 7f8556a00640 1 -- 172.21.15.100:0/2277571941 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8558019070 con 0x7f85600cc830 2024-09-17T04:31:30.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.364+0000 7f8556a00640 1 -- 172.21.15.100:0/2277571941 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f855800cc90 con 0x7f85600cc830 2024-09-17T04:31:30.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.364+0000 7f8556a00640 1 -- 172.21.15.100:0/2277571941 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85580146a0 con 0x7f85600cc830 2024-09-17T04:31:30.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.364+0000 7f8565200640 1 -- 172.21.15.100:0/2277571941 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85601f47c0 con 0x7f85600cc830 2024-09-17T04:31:30.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.364+0000 7f8565200640 1 -- 172.21.15.100:0/2277571941 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8560156730 con 0x7f85600cc830 2024-09-17T04:31:30.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.365+0000 7f8565200640 1 -- 172.21.15.100:0/2277571941 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85600b3d40 con 0x7f85600cc830 2024-09-17T04:31:30.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.366+0000 7f8556a00640 1 -- 172.21.15.100:0/2277571941 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f855801c070 con 0x7f85600cc830 2024-09-17T04:31:30.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.367+0000 7f8556a00640 1 --2- 172.21.15.100:0/2277571941 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f853c079010 0x7f853c07b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:30.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.367+0000 7f8556a00640 1 -- 172.21.15.100:0/2277571941 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7f855809b3e0 con 0x7f85600cc830 2024-09-17T04:31:30.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.368+0000 7f855de00640 1 --2- 172.21.15.100:0/2277571941 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f853c079010 0x7f853c07b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:30.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.368+0000 7f855de00640 1 --2- 172.21.15.100:0/2277571941 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f853c079010 0x7f853c07b520 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f85500098e0 tx=0x7f8550009060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:30.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.372+0000 7f8556a00640 1 -- 172.21.15.100:0/2277571941 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8558062a30 con 0x7f85600cc830 2024-09-17T04:31:30.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.579+0000 7f8565200640 1 -- 172.21.15.100:0/2277571941 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol11481", "group_name": "subvol_grp33350", "target": ["mon-mgr", ""]}) -- 0x7f85601e76c0 con 0x7f853c079010 2024-09-17T04:31:30.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.618+0000 7f8556a00640 1 -- 172.21.15.100:0/2277571941 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+74 (secure 0 0 0) 0x7f85601e76c0 con 0x7f853c079010 2024-09-17T04:31:30.618 INFO:teuthology.orchestra.run.smithi100.stdout:/volumes/subvol_grp33350/subvol11481/0d0b12bb-592d-48f7-b31f-ded447325aee 2024-09-17T04:31:30.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.620+0000 7f8565200640 1 -- 172.21.15.100:0/2277571941 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f853c079010 msgr2=0x7f853c07b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:30.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.620+0000 7f8565200640 1 --2- 172.21.15.100:0/2277571941 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f853c079010 0x7f853c07b520 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f85500098e0 tx=0x7f8550009060 comp rx=0 tx=0).stop 2024-09-17T04:31:30.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.620+0000 7f8565200640 1 -- 172.21.15.100:0/2277571941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85600cc830 msgr2=0x7f85601e6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:30.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.620+0000 7f8565200640 1 --2- 172.21.15.100:0/2277571941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85600cc830 0x7f85601e6ba0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f855800ba10 tx=0x7f855800bed0 comp rx=0 tx=0).stop 2024-09-17T04:31:30.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.620+0000 7f8565200640 1 -- 172.21.15.100:0/2277571941 shutdown_connections 2024-09-17T04:31:30.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.620+0000 7f8565200640 1 --2- 172.21.15.100:0/2277571941 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f853c079010 0x7f853c07b520 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:30.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.620+0000 7f8565200640 1 --2- 172.21.15.100:0/2277571941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85600cc830 0x7f85601e6ba0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:30.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.620+0000 7f8565200640 1 --2- 172.21.15.100:0/2277571941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85600af100 0x7f85601e6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:30.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.620+0000 7f8565200640 1 --2- 172.21.15.100:0/2277571941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f856006cda0 0x7f85600985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:30.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.620+0000 7f8565200640 1 -- 172.21.15.100:0/2277571941 >> 172.21.15.100:0/2277571941 conn(0x7f85600b2540 msgr2=0x7f8560073780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:30.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.620+0000 7f8565200640 1 -- 172.21.15.100:0/2277571941 shutdown_connections 2024-09-17T04:31:30.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.620+0000 7f8565200640 1 -- 172.21.15.100:0/2277571941 wait complete. 2024-09-17T04:31:30.643 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol41972 2024-09-17T04:31:30.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.869+0000 7f1933200640 1 -- 172.21.15.100:0/1673793166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f192c0cd610 msgr2=0x7f192c0cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:30.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.869+0000 7f1933200640 1 --2- 172.21.15.100:0/1673793166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f192c0cd610 0x7f192c0cda20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1924009830 tx=0x7f192402a510 comp rx=0 tx=0).stop 2024-09-17T04:31:30.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.869+0000 7f1933200640 1 -- 172.21.15.100:0/1673793166 shutdown_connections 2024-09-17T04:31:30.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.869+0000 7f1933200640 1 --2- 172.21.15.100:0/1673793166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f192c0cd610 0x7f192c0cda20 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:30.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.869+0000 7f1933200640 1 --2- 172.21.15.100:0/1673793166 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f192c145450 0x7f192c06f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:30.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.869+0000 7f1933200640 1 -- 172.21.15.100:0/1673793166 >> 172.21.15.100:0/1673793166 conn(0x7f192c06d1c0 msgr2=0x7f192c06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:30.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.869+0000 7f1933200640 1 -- 172.21.15.100:0/1673793166 shutdown_connections 2024-09-17T04:31:30.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.869+0000 7f1933200640 1 -- 172.21.15.100:0/1673793166 wait complete. 2024-09-17T04:31:30.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.872+0000 7f1933200640 1 Processor -- start 2024-09-17T04:31:30.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.872+0000 7f1933200640 1 -- start start 2024-09-17T04:31:30.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.873+0000 7f1933200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f192c0735e0 0x7f192c15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:30.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.873+0000 7f1933200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f192c0cd610 0x7f192c13cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:30.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.873+0000 7f1933200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f192c145450 0x7f192c13d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:30.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.873+0000 7f1933200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f192c0ce2f0 con 0x7f192c145450 2024-09-17T04:31:30.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.873+0000 7f1933200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f192c0ce170 con 0x7f192c0735e0 2024-09-17T04:31:30.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.873+0000 7f1933200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f192c0ce470 con 0x7f192c0cd610 2024-09-17T04:31:30.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.874+0000 7f1931200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f192c145450 0x7f192c13d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:30.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.874+0000 7f1931200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f192c145450 0x7f192c13d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49422/0 (socket says 172.21.15.100:49422) 2024-09-17T04:31:30.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.874+0000 7f1931200640 1 -- 172.21.15.100:0/968271731 learned_addr learned my addr 172.21.15.100:0/968271731 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:30.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.874+0000 7f192be00640 1 --2- 172.21.15.100:0/968271731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f192c0735e0 0x7f192c15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:30.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.874+0000 7f192b400640 1 --2- 172.21.15.100:0/968271731 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f192c0cd610 0x7f192c13cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:30.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.874+0000 7f1931200640 1 -- 172.21.15.100:0/968271731 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f192c0cd610 msgr2=0x7f192c13cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:30.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.874+0000 7f1931200640 1 --2- 172.21.15.100:0/968271731 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f192c0cd610 0x7f192c13cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:30.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.874+0000 7f1931200640 1 -- 172.21.15.100:0/968271731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f192c0735e0 msgr2=0x7f192c15c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:30.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.874+0000 7f1931200640 1 --2- 172.21.15.100:0/968271731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f192c0735e0 0x7f192c15c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:30.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.874+0000 7f1931200640 1 -- 172.21.15.100:0/968271731 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f192c13ba70 con 0x7f192c145450 2024-09-17T04:31:30.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.874+0000 7f1931200640 1 --2- 172.21.15.100:0/968271731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f192c145450 0x7f192c13d2a0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f1924009950 tx=0x7f1924033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:30.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.875+0000 7f1923e00640 1 -- 172.21.15.100:0/968271731 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924031b50 con 0x7f192c145450 2024-09-17T04:31:30.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.875+0000 7f1923e00640 1 -- 172.21.15.100:0/968271731 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1924031de0 con 0x7f192c145450 2024-09-17T04:31:30.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.875+0000 7f1933200640 1 -- 172.21.15.100:0/968271731 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f192c13bd50 con 0x7f192c145450 2024-09-17T04:31:30.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.875+0000 7f1923e00640 1 -- 172.21.15.100:0/968271731 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924005280 con 0x7f192c145450 2024-09-17T04:31:30.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.875+0000 7f1933200640 1 -- 172.21.15.100:0/968271731 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f192c13c310 con 0x7f192c145450 2024-09-17T04:31:30.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.876+0000 7f1933200640 1 -- 172.21.15.100:0/968271731 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f192c0cda20 con 0x7f192c145450 2024-09-17T04:31:30.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.877+0000 7f1923e00640 1 -- 172.21.15.100:0/968271731 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f1924005420 con 0x7f192c145450 2024-09-17T04:31:30.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.878+0000 7f1923e00640 1 --2- 172.21.15.100:0/968271731 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f190c079010 0x7f190c07b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:30.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.878+0000 7f1923e00640 1 -- 172.21.15.100:0/968271731 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7f19240bc9c0 con 0x7f192c145450 2024-09-17T04:31:30.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.879+0000 7f192be00640 1 --2- 172.21.15.100:0/968271731 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f190c079010 0x7f190c07b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:30.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.879+0000 7f192be00640 1 --2- 172.21.15.100:0/968271731 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f190c079010 0x7f190c07b520 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f191c009a60 tx=0x7f191c010040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:30.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:30.883+0000 7f1923e00640 1 -- 172.21.15.100:0/968271731 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f192403f030 con 0x7f192c145450 2024-09-17T04:31:31.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.102+0000 7f1933200640 1 -- 172.21.15.100:0/968271731 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol41972", "target": ["mon-mgr", ""]}) -- 0x7f192c13c680 con 0x7f190c079010 2024-09-17T04:31:31.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.133+0000 7f1923e00640 1 -- 172.21.15.100:0/968271731 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+144 (secure 0 0 0) 0x7f192c13c680 con 0x7f190c079010 2024-09-17T04:31:31.133 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:31:31.133 INFO:teuthology.orchestra.run.smithi100.stdout: "status": { 2024-09-17T04:31:31.134 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "pending", 2024-09-17T04:31:31.134 INFO:teuthology.orchestra.run.smithi100.stdout: "source": { 2024-09-17T04:31:31.134 INFO:teuthology.orchestra.run.smithi100.stdout: "volume": "fake", 2024-09-17T04:31:31.134 INFO:teuthology.orchestra.run.smithi100.stdout: "subvolume": "fake", 2024-09-17T04:31:31.134 INFO:teuthology.orchestra.run.smithi100.stdout: "snapshot": "fake" 2024-09-17T04:31:31.134 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:31.134 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:31.134 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:31:31.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.135+0000 7f1933200640 1 -- 172.21.15.100:0/968271731 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f190c079010 msgr2=0x7f190c07b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:31.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.135+0000 7f1933200640 1 --2- 172.21.15.100:0/968271731 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f190c079010 0x7f190c07b520 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f191c009a60 tx=0x7f191c010040 comp rx=0 tx=0).stop 2024-09-17T04:31:31.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.135+0000 7f1933200640 1 -- 172.21.15.100:0/968271731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f192c145450 msgr2=0x7f192c13d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:31.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.135+0000 7f1933200640 1 --2- 172.21.15.100:0/968271731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f192c145450 0x7f192c13d2a0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f1924009950 tx=0x7f1924033ba0 comp rx=0 tx=0).stop 2024-09-17T04:31:31.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.135+0000 7f1933200640 1 -- 172.21.15.100:0/968271731 shutdown_connections 2024-09-17T04:31:31.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.136+0000 7f1933200640 1 --2- 172.21.15.100:0/968271731 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f190c079010 0x7f190c07b520 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:31.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.136+0000 7f1933200640 1 --2- 172.21.15.100:0/968271731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f192c145450 0x7f192c13d2a0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:31.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.136+0000 7f1933200640 1 --2- 172.21.15.100:0/968271731 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f192c0cd610 0x7f192c13cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:31.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.136+0000 7f1933200640 1 --2- 172.21.15.100:0/968271731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f192c0735e0 0x7f192c15c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:31.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.136+0000 7f1933200640 1 -- 172.21.15.100:0/968271731 >> 172.21.15.100:0/968271731 conn(0x7f192c06d1c0 msgr2=0x7f192c0b3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:31.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.136+0000 7f1933200640 1 -- 172.21.15.100:0/968271731 shutdown_connections 2024-09-17T04:31:31.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.136+0000 7f1933200640 1 -- 172.21.15.100:0/968271731 wait complete. 2024-09-17T04:31:31.158 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol39090 fake 2024-09-17T04:31:31.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.381+0000 7f7ba1800640 1 -- 172.21.15.100:0/471474770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b9c0af100 msgr2=0x7f7b9c0af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:31.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.381+0000 7f7ba1800640 1 --2- 172.21.15.100:0/471474770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b9c0af100 0x7f7b9c0af510 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f7b94005fd0 tx=0x7f7b9402aa90 comp rx=0 tx=0).stop 2024-09-17T04:31:31.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.381+0000 7f7ba1800640 1 -- 172.21.15.100:0/471474770 shutdown_connections 2024-09-17T04:31:31.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.381+0000 7f7ba1800640 1 --2- 172.21.15.100:0/471474770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b9c0af100 0x7f7b9c0af510 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:31.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.381+0000 7f7ba1800640 1 --2- 172.21.15.100:0/471474770 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b9c0cc830 0x7f7b9c0b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:31.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.381+0000 7f7ba1800640 1 -- 172.21.15.100:0/471474770 >> 172.21.15.100:0/471474770 conn(0x7f7b9c0b2540 msgr2=0x7f7b9c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:31.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.381+0000 7f7ba1800640 1 -- 172.21.15.100:0/471474770 shutdown_connections 2024-09-17T04:31:31.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.381+0000 7f7ba1800640 1 -- 172.21.15.100:0/471474770 wait complete. 2024-09-17T04:31:31.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.383+0000 7f7ba1800640 1 Processor -- start 2024-09-17T04:31:31.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.383+0000 7f7ba1800640 1 -- start start 2024-09-17T04:31:31.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.385+0000 7f7ba1800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b9c06cda0 0x7f7b9c098bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:31.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.385+0000 7f7ba1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b9c0af100 0x7f7b9c1e9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:31.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.385+0000 7f7ba1800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b9c0cc830 0x7f7b9c1e9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:31.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.385+0000 7f7ba1800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c0b36a0 con 0x7f7b9c0af100 2024-09-17T04:31:31.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.385+0000 7f7ba1800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c0b3520 con 0x7f7b9c0cc830 2024-09-17T04:31:31.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.385+0000 7f7ba1800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7b9c0b3820 con 0x7f7b9c06cda0 2024-09-17T04:31:31.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.385+0000 7f7b92000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b9c0af100 0x7f7b9c1e9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:31.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.385+0000 7f7b92000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b9c0af100 0x7f7b9c1e9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49434/0 (socket says 172.21.15.100:49434) 2024-09-17T04:31:31.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.385+0000 7f7b92000640 1 -- 172.21.15.100:0/1486555376 learned_addr learned my addr 172.21.15.100:0/1486555376 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:31.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.385+0000 7f7b9aa00640 1 --2- 172.21.15.100:0/1486555376 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b9c06cda0 0x7f7b9c098bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:31.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.385+0000 7f7b9b400640 1 --2- 172.21.15.100:0/1486555376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b9c0cc830 0x7f7b9c1e9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:31.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.385+0000 7f7b92000640 1 -- 172.21.15.100:0/1486555376 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b9c06cda0 msgr2=0x7f7b9c098bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:31.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.385+0000 7f7b92000640 1 --2- 172.21.15.100:0/1486555376 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b9c06cda0 0x7f7b9c098bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:31.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.385+0000 7f7b92000640 1 -- 172.21.15.100:0/1486555376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b9c0cc830 msgr2=0x7f7b9c1e9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:31.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.385+0000 7f7b92000640 1 --2- 172.21.15.100:0/1486555376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b9c0cc830 0x7f7b9c1e9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:31.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.385+0000 7f7b92000640 1 -- 172.21.15.100:0/1486555376 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9c09e230 con 0x7f7b9c0af100 2024-09-17T04:31:31.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.386+0000 7f7b92000640 1 --2- 172.21.15.100:0/1486555376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b9c0af100 0x7f7b9c1e9510 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400c430 tx=0x7f7b8400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:31.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.386+0000 7f7b93400640 1 -- 172.21.15.100:0/1486555376 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84017070 con 0x7f7b9c0af100 2024-09-17T04:31:31.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.386+0000 7f7b93400640 1 -- 172.21.15.100:0/1486555376 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b840053c0 con 0x7f7b9c0af100 2024-09-17T04:31:31.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.386+0000 7f7ba1800640 1 -- 172.21.15.100:0/1486555376 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b9c156040 con 0x7f7b9c0af100 2024-09-17T04:31:31.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.386+0000 7f7b93400640 1 -- 172.21.15.100:0/1486555376 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84003850 con 0x7f7b9c0af100 2024-09-17T04:31:31.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.386+0000 7f7ba1800640 1 -- 172.21.15.100:0/1486555376 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b9c156600 con 0x7f7b9c0af100 2024-09-17T04:31:31.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.387+0000 7f7ba1800640 1 -- 172.21.15.100:0/1486555376 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b9c06a700 con 0x7f7b9c0af100 2024-09-17T04:31:31.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.389+0000 7f7b93400640 1 -- 172.21.15.100:0/1486555376 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f7b8401a070 con 0x7f7b9c0af100 2024-09-17T04:31:31.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.389+0000 7f7b93400640 1 --2- 172.21.15.100:0/1486555376 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7b78079010 0x7f7b7807b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:31.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.390+0000 7f7b93400640 1 -- 172.21.15.100:0/1486555376 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7f7b8409aa20 con 0x7f7b9c0af100 2024-09-17T04:31:31.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.390+0000 7f7b9aa00640 1 --2- 172.21.15.100:0/1486555376 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7b78079010 0x7f7b7807b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:31.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.390+0000 7f7b9aa00640 1 --2- 172.21.15.100:0/1486555376 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7b78079010 0x7f7b7807b520 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009b30 tx=0x7f7b8c005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:31.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.394+0000 7f7b93400640 1 -- 172.21.15.100:0/1486555376 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f7b9c06a700 con 0x7f7b9c0af100 2024-09-17T04:31:31.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.606+0000 7f7ba1800640 1 -- 172.21.15.100:0/1486555376 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol39090", "snap_name": "fake", "target": ["mon-mgr", ""]}) -- 0x7f7b9c0b3d40 con 0x7f7b78079010 2024-09-17T04:31:31.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.625+0000 7f7b93400640 1 -- 172.21.15.100:0/1486555376 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 22) ==== 101900+0+0 (secure 0 0 0) 0x7f7b8405e250 con 0x7f7b9c0af100 2024-09-17T04:31:31.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.658+0000 7f7b93400640 1 -- 172.21.15.100:0/1486555376 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7b9c0b3d40 con 0x7f7b78079010 2024-09-17T04:31:31.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.661+0000 7f7ba1800640 1 -- 172.21.15.100:0/1486555376 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7b78079010 msgr2=0x7f7b7807b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:31.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.661+0000 7f7ba1800640 1 --2- 172.21.15.100:0/1486555376 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7b78079010 0x7f7b7807b520 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009b30 tx=0x7f7b8c005f90 comp rx=0 tx=0).stop 2024-09-17T04:31:31.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.661+0000 7f7ba1800640 1 -- 172.21.15.100:0/1486555376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b9c0af100 msgr2=0x7f7b9c1e9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:31.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.661+0000 7f7ba1800640 1 --2- 172.21.15.100:0/1486555376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b9c0af100 0x7f7b9c1e9510 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400c430 tx=0x7f7b8400c8f0 comp rx=0 tx=0).stop 2024-09-17T04:31:31.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.661+0000 7f7ba1800640 1 -- 172.21.15.100:0/1486555376 shutdown_connections 2024-09-17T04:31:31.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.661+0000 7f7ba1800640 1 --2- 172.21.15.100:0/1486555376 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7b78079010 0x7f7b7807b520 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:31.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.661+0000 7f7ba1800640 1 --2- 172.21.15.100:0/1486555376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b9c0cc830 0x7f7b9c1e9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:31.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.661+0000 7f7ba1800640 1 --2- 172.21.15.100:0/1486555376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b9c0af100 0x7f7b9c1e9510 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:31.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.661+0000 7f7ba1800640 1 --2- 172.21.15.100:0/1486555376 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b9c06cda0 0x7f7b9c098bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:31.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.661+0000 7f7ba1800640 1 -- 172.21.15.100:0/1486555376 >> 172.21.15.100:0/1486555376 conn(0x7f7b9c0b2540 msgr2=0x7f7b9c0747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:31.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.661+0000 7f7ba1800640 1 -- 172.21.15.100:0/1486555376 shutdown_connections 2024-09-17T04:31:31.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.661+0000 7f7ba1800640 1 -- 172.21.15.100:0/1486555376 wait complete. 2024-09-17T04:31:31.682 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol11481 fake subvol_grp33350 2024-09-17T04:31:31.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.908+0000 7f8a89000640 1 -- 172.21.15.100:0/4171591002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a84145450 msgr2=0x7f8a8406f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:31.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.908+0000 7f8a89000640 1 --2- 172.21.15.100:0/4171591002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a84145450 0x7f8a8406f180 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f8a84071590 tx=0x7f8a7402a560 comp rx=0 tx=0).stop 2024-09-17T04:31:31.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.908+0000 7f8a89000640 1 -- 172.21.15.100:0/4171591002 shutdown_connections 2024-09-17T04:31:31.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.908+0000 7f8a89000640 1 --2- 172.21.15.100:0/4171591002 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a84160320 0x7f8a84160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:31.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.908+0000 7f8a89000640 1 --2- 172.21.15.100:0/4171591002 >> v2:172.21.15.184:3300/0 conn(0x7f8a840735e0 0x7f8a840739f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:31.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.908+0000 7f8a89000640 1 --2- 172.21.15.100:0/4171591002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a84145450 0x7f8a8406f180 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:31.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.908+0000 7f8a89000640 1 -- 172.21.15.100:0/4171591002 >> 172.21.15.100:0/4171591002 conn(0x7f8a8406d1c0 msgr2=0x7f8a8406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:31.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.908+0000 7f8a89000640 1 -- 172.21.15.100:0/4171591002 shutdown_connections 2024-09-17T04:31:31.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.908+0000 7f8a89000640 1 -- 172.21.15.100:0/4171591002 wait complete. 2024-09-17T04:31:31.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.910+0000 7f8a89000640 1 Processor -- start 2024-09-17T04:31:31.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.910+0000 7f8a89000640 1 -- start start 2024-09-17T04:31:31.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.913+0000 7f8a89000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a840735e0 0x7f8a840adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:31.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.913+0000 7f8a89000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a84145450 0x7f8a8415edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:31.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.913+0000 7f8a89000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a84160320 0x7f8a840b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:31.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.913+0000 7f8a89000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8a84160c70 con 0x7f8a84160320 2024-09-17T04:31:31.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.913+0000 7f8a89000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8a84071020 con 0x7f8a840735e0 2024-09-17T04:31:31.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.913+0000 7f8a89000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8a84160df0 con 0x7f8a84145450 2024-09-17T04:31:31.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.913+0000 7f8a82800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a84160320 0x7f8a840b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:31.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.913+0000 7f8a82800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a84160320 0x7f8a840b3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49460/0 (socket says 172.21.15.100:49460) 2024-09-17T04:31:31.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.913+0000 7f8a82800640 1 -- 172.21.15.100:0/482597751 learned_addr learned my addr 172.21.15.100:0/482597751 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:31.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.913+0000 7f8a81e00640 1 --2- 172.21.15.100:0/482597751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a840735e0 0x7f8a840adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:31.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.913+0000 7f8a81400640 1 --2- 172.21.15.100:0/482597751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a84145450 0x7f8a8415edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:31.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.913+0000 7f8a82800640 1 -- 172.21.15.100:0/482597751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a84145450 msgr2=0x7f8a8415edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:31.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.913+0000 7f8a82800640 1 --2- 172.21.15.100:0/482597751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a84145450 0x7f8a8415edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:31.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.913+0000 7f8a82800640 1 -- 172.21.15.100:0/482597751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a840735e0 msgr2=0x7f8a840adbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:31.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.913+0000 7f8a82800640 1 --2- 172.21.15.100:0/482597751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a840735e0 0x7f8a840adbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:31.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.913+0000 7f8a82800640 1 -- 172.21.15.100:0/482597751 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a840ae420 con 0x7f8a84160320 2024-09-17T04:31:31.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.913+0000 7f8a82800640 1 --2- 172.21.15.100:0/482597751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a84160320 0x7f8a840b3a50 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c00e990 tx=0x7f8a7c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:31.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.913+0000 7f8a7a000640 1 -- 172.21.15.100:0/482597751 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c004d20 con 0x7f8a84160320 2024-09-17T04:31:31.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.913+0000 7f8a7a000640 1 -- 172.21.15.100:0/482597751 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a7c00caa0 con 0x7f8a84160320 2024-09-17T04:31:31.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.913+0000 7f8a89000640 1 -- 172.21.15.100:0/482597751 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a840ae700 con 0x7f8a84160320 2024-09-17T04:31:31.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.913+0000 7f8a7a000640 1 -- 172.21.15.100:0/482597751 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c005020 con 0x7f8a84160320 2024-09-17T04:31:31.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.913+0000 7f8a89000640 1 -- 172.21.15.100:0/482597751 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a840aecc0 con 0x7f8a84160320 2024-09-17T04:31:31.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.914+0000 7f8a89000640 1 -- 172.21.15.100:0/482597751 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a84144c90 con 0x7f8a84160320 2024-09-17T04:31:31.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.915+0000 7f8a7a000640 1 -- 172.21.15.100:0/482597751 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 101900+0+0 (secure 0 0 0) 0x7f8a7c0051c0 con 0x7f8a84160320 2024-09-17T04:31:31.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.916+0000 7f8a7a000640 1 --2- 172.21.15.100:0/482597751 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8a640790a0 0x7f8a6407b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:31.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.917+0000 7f8a7a000640 1 -- 172.21.15.100:0/482597751 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7f8a7c09bf80 con 0x7f8a84160320 2024-09-17T04:31:31.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.917+0000 7f8a81e00640 1 --2- 172.21.15.100:0/482597751 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8a640790a0 0x7f8a6407b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:31.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.917+0000 7f8a81e00640 1 --2- 172.21.15.100:0/482597751 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8a640790a0 0x7f8a6407b5b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8a74033000 tx=0x7f8a74004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:31.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:31.921+0000 7f8a7a000640 1 -- 172.21.15.100:0/482597751 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8a7c016030 con 0x7f8a84160320 2024-09-17T04:31:32.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.135+0000 7f8a89000640 1 -- 172.21.15.100:0/482597751 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol11481", "snap_name": "fake", "group_name": "subvol_grp33350", "target": ["mon-mgr", ""]}) -- 0x7f8a84143330 con 0x7f8a640790a0 2024-09-17T04:31:32.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.178+0000 7f8a7a000640 1 -- 172.21.15.100:0/482597751 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8a84143330 con 0x7f8a640790a0 2024-09-17T04:31:32.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.182+0000 7f8a89000640 1 -- 172.21.15.100:0/482597751 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8a640790a0 msgr2=0x7f8a6407b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:32.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.182+0000 7f8a89000640 1 --2- 172.21.15.100:0/482597751 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8a640790a0 0x7f8a6407b5b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8a74033000 tx=0x7f8a74004730 comp rx=0 tx=0).stop 2024-09-17T04:31:32.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.182+0000 7f8a89000640 1 -- 172.21.15.100:0/482597751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a84160320 msgr2=0x7f8a840b3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:32.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.182+0000 7f8a89000640 1 --2- 172.21.15.100:0/482597751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a84160320 0x7f8a840b3a50 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c00e990 tx=0x7f8a7c00ee50 comp rx=0 tx=0).stop 2024-09-17T04:31:32.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.182+0000 7f8a89000640 1 -- 172.21.15.100:0/482597751 shutdown_connections 2024-09-17T04:31:32.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.182+0000 7f8a89000640 1 --2- 172.21.15.100:0/482597751 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8a640790a0 0x7f8a6407b5b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:32.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.182+0000 7f8a89000640 1 --2- 172.21.15.100:0/482597751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a84160320 0x7f8a840b3a50 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:32.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.182+0000 7f8a89000640 1 --2- 172.21.15.100:0/482597751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a84145450 0x7f8a8415edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:32.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.182+0000 7f8a89000640 1 --2- 172.21.15.100:0/482597751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a840735e0 0x7f8a840adbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:32.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.182+0000 7f8a89000640 1 -- 172.21.15.100:0/482597751 >> 172.21.15.100:0/482597751 conn(0x7f8a8406d1c0 msgr2=0x7f8a840b0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:32.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.182+0000 7f8a89000640 1 -- 172.21.15.100:0/482597751 shutdown_connections 2024-09-17T04:31:32.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.182+0000 7f8a89000640 1 -- 172.21.15.100:0/482597751 wait complete. 2024-09-17T04:31:32.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:32.207 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol39090/.meta) 2024-09-17T04:31:32.263 INFO:teuthology.orchestra.run.smithi100.stdout:[GLOBAL] 2024-09-17T04:31:32.264 INFO:teuthology.orchestra.run.smithi100.stdout:version = 1 2024-09-17T04:31:32.264 INFO:teuthology.orchestra.run.smithi100.stdout:type = subvolume 2024-09-17T04:31:32.264 INFO:teuthology.orchestra.run.smithi100.stdout:path = /volumes/_nogroup/subvol39090/23dc80c1-362e-475b-a414-7ba4509a30ff 2024-09-17T04:31:32.264 INFO:teuthology.orchestra.run.smithi100.stdout:state = complete 2024-09-17T04:31:32.264 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:32.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:32.267 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/subvol_grp33350/subvol11481/.meta) 2024-09-17T04:31:32.323 INFO:teuthology.orchestra.run.smithi100.stdout:[GLOBAL] 2024-09-17T04:31:32.323 INFO:teuthology.orchestra.run.smithi100.stdout:version = 1 2024-09-17T04:31:32.323 INFO:teuthology.orchestra.run.smithi100.stdout:type = subvolume 2024-09-17T04:31:32.324 INFO:teuthology.orchestra.run.smithi100.stdout:path = /volumes/subvol_grp33350/subvol11481/0d0b12bb-592d-48f7-b31f-ded447325aee 2024-09-17T04:31:32.324 INFO:teuthology.orchestra.run.smithi100.stdout:state = complete 2024-09-17T04:31:32.324 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:32.326 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol39090 2024-09-17T04:31:32.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.549+0000 7efc84600640 1 -- 172.21.15.100:0/3534708060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc7c15ccf0 msgr2=0x7efc7c15d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:32.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.549+0000 7efc84600640 1 --2- 172.21.15.100:0/3534708060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc7c15ccf0 0x7efc7c15d100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efc74005fd0 tx=0x7efc7402a820 comp rx=0 tx=0).stop 2024-09-17T04:31:32.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.549+0000 7efc84600640 1 -- 172.21.15.100:0/3534708060 shutdown_connections 2024-09-17T04:31:32.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.549+0000 7efc84600640 1 --2- 172.21.15.100:0/3534708060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc7c15ccf0 0x7efc7c15d100 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:32.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.549+0000 7efc84600640 1 --2- 172.21.15.100:0/3534708060 >> v2:172.21.15.184:3300/0 conn(0x7efc7c073630 0x7efc7c073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:32.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.549+0000 7efc84600640 1 -- 172.21.15.100:0/3534708060 >> 172.21.15.100:0/3534708060 conn(0x7efc7c06d1c0 msgr2=0x7efc7c06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:32.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.549+0000 7efc84600640 1 -- 172.21.15.100:0/3534708060 shutdown_connections 2024-09-17T04:31:32.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.549+0000 7efc84600640 1 -- 172.21.15.100:0/3534708060 wait complete. 2024-09-17T04:31:32.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.551+0000 7efc84600640 1 Processor -- start 2024-09-17T04:31:32.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.551+0000 7efc84600640 1 -- start start 2024-09-17T04:31:32.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.553+0000 7efc84600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc7c073630 0x7efc7c1413e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:32.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.553+0000 7efc84600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc7c0ccc50 0x7efc7c141920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:32.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.553+0000 7efc84600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc7c15ccf0 0x7efc7c147540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:32.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.553+0000 7efc84600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efc7c15d9d0 con 0x7efc7c0ccc50 2024-09-17T04:31:32.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.553+0000 7efc84600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efc7c15d850 con 0x7efc7c15ccf0 2024-09-17T04:31:32.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.553+0000 7efc84600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efc7c15db50 con 0x7efc7c073630 2024-09-17T04:31:32.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.553+0000 7efc81200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc7c0ccc50 0x7efc7c141920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:32.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.553+0000 7efc81200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc7c0ccc50 0x7efc7c141920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49466/0 (socket says 172.21.15.100:49466) 2024-09-17T04:31:32.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.553+0000 7efc81200640 1 -- 172.21.15.100:0/3118136991 learned_addr learned my addr 172.21.15.100:0/3118136991 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:32.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.553+0000 7efc82600640 1 --2- 172.21.15.100:0/3118136991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc7c15ccf0 0x7efc7c147540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:32.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.553+0000 7efc81c00640 1 --2- 172.21.15.100:0/3118136991 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc7c073630 0x7efc7c1413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:32.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.554+0000 7efc81200640 1 -- 172.21.15.100:0/3118136991 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc7c073630 msgr2=0x7efc7c1413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:32.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.554+0000 7efc81200640 1 --2- 172.21.15.100:0/3118136991 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc7c073630 0x7efc7c1413e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:32.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.554+0000 7efc81200640 1 -- 172.21.15.100:0/3118136991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc7c15ccf0 msgr2=0x7efc7c147540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:32.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.554+0000 7efc81200640 1 --2- 172.21.15.100:0/3118136991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc7c15ccf0 0x7efc7c147540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:32.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.554+0000 7efc81200640 1 -- 172.21.15.100:0/3118136991 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc7c1483d0 con 0x7efc7c0ccc50 2024-09-17T04:31:32.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.554+0000 7efc81200640 1 --2- 172.21.15.100:0/3118136991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc7c0ccc50 0x7efc7c141920 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7efc6800e9f0 tx=0x7efc6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:32.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.554+0000 7efc79600640 1 -- 172.21.15.100:0/3118136991 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6800cd80 con 0x7efc7c0ccc50 2024-09-17T04:31:32.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.554+0000 7efc79600640 1 -- 172.21.15.100:0/3118136991 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc68004020 con 0x7efc7c0ccc50 2024-09-17T04:31:32.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.554+0000 7efc79600640 1 -- 172.21.15.100:0/3118136991 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc68004470 con 0x7efc7c0ccc50 2024-09-17T04:31:32.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.554+0000 7efc84600640 1 -- 172.21.15.100:0/3118136991 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc7c0a9a20 con 0x7efc7c0ccc50 2024-09-17T04:31:32.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.554+0000 7efc84600640 1 -- 172.21.15.100:0/3118136991 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc7c158e90 con 0x7efc7c0ccc50 2024-09-17T04:31:32.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.555+0000 7efc84600640 1 -- 172.21.15.100:0/3118136991 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc7c0743e0 con 0x7efc7c0ccc50 2024-09-17T04:31:32.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.557+0000 7efc79600640 1 -- 172.21.15.100:0/3118136991 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 101900+0+0 (secure 0 0 0) 0x7efc680035f0 con 0x7efc7c0ccc50 2024-09-17T04:31:32.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.557+0000 7efc79600640 1 --2- 172.21.15.100:0/3118136991 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7efc5c0790a0 0x7efc5c07b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:32.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.558+0000 7efc79600640 1 -- 172.21.15.100:0/3118136991 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7efc6809c050 con 0x7efc7c0ccc50 2024-09-17T04:31:32.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.558+0000 7efc81c00640 1 --2- 172.21.15.100:0/3118136991 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7efc5c0790a0 0x7efc5c07b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:32.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.558+0000 7efc81c00640 1 --2- 172.21.15.100:0/3118136991 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7efc5c0790a0 0x7efc5c07b5b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efc70007d60 tx=0x7efc70007cf0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:32.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.562+0000 7efc79600640 1 -- 172.21.15.100:0/3118136991 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7efc7c0743e0 con 0x7efc7c0ccc50 2024-09-17T04:31:32.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.771+0000 7efc84600640 1 -- 172.21.15.100:0/3118136991 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol39090", "target": ["mon-mgr", ""]}) -- 0x7efc7c073a40 con 0x7efc5c0790a0 2024-09-17T04:31:32.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.809+0000 7efc79600640 1 -- 172.21.15.100:0/3118136991 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efc7c073a40 con 0x7efc5c0790a0 2024-09-17T04:31:32.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.811+0000 7efc84600640 1 -- 172.21.15.100:0/3118136991 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7efc5c0790a0 msgr2=0x7efc5c07b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:32.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.811+0000 7efc84600640 1 --2- 172.21.15.100:0/3118136991 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7efc5c0790a0 0x7efc5c07b5b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efc70007d60 tx=0x7efc70007cf0 comp rx=0 tx=0).stop 2024-09-17T04:31:32.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.811+0000 7efc84600640 1 -- 172.21.15.100:0/3118136991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc7c0ccc50 msgr2=0x7efc7c141920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:32.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.811+0000 7efc84600640 1 --2- 172.21.15.100:0/3118136991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc7c0ccc50 0x7efc7c141920 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7efc6800e9f0 tx=0x7efc6800eeb0 comp rx=0 tx=0).stop 2024-09-17T04:31:32.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.811+0000 7efc84600640 1 -- 172.21.15.100:0/3118136991 shutdown_connections 2024-09-17T04:31:32.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.811+0000 7efc84600640 1 --2- 172.21.15.100:0/3118136991 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7efc5c0790a0 0x7efc5c07b5b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:32.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.811+0000 7efc84600640 1 --2- 172.21.15.100:0/3118136991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc7c15ccf0 0x7efc7c147540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:32.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.811+0000 7efc84600640 1 --2- 172.21.15.100:0/3118136991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc7c0ccc50 0x7efc7c141920 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:32.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.811+0000 7efc84600640 1 --2- 172.21.15.100:0/3118136991 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc7c073630 0x7efc7c1413e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:32.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.811+0000 7efc84600640 1 -- 172.21.15.100:0/3118136991 >> 172.21.15.100:0/3118136991 conn(0x7efc7c06d1c0 msgr2=0x7efc7c068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:32.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.811+0000 7efc84600640 1 -- 172.21.15.100:0/3118136991 shutdown_connections 2024-09-17T04:31:32.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:32.811+0000 7efc84600640 1 -- 172.21.15.100:0/3118136991 wait complete. 2024-09-17T04:31:32.816 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:31:32.817+0000 7f30b5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:31:32.816 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:31:32.817+0000 7f30b5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:31:32.816 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:31:32.817+0000 7f30b5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:31:32.816 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:31:32.817+0000 7f30b5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:31:32.816 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:31:32.817+0000 7f30b5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:31:32.834 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:31:32.835+0000 7f30b6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:31:32.834 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:31:32.835+0000 7f30b6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:31:32.834 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:31:32.835+0000 7f30b6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:31:32.834 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:31:32.835+0000 7f30b6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:31:32.835 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:31:32.835+0000 7f30b6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:31:32.839 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol11481 subvol_grp33350 2024-09-17T04:31:33.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.067+0000 7f36d2a00640 1 -- 172.21.15.100:0/2699017030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36cc0705b0 msgr2=0x7f36cc0af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:33.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.067+0000 7f36d2a00640 1 --2- 172.21.15.100:0/2699017030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36cc0705b0 0x7f36cc0af3c0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f36cc06ddf0 tx=0x7f36b802a4d0 comp rx=0 tx=0).stop 2024-09-17T04:31:33.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.067+0000 7f36d2a00640 1 -- 172.21.15.100:0/2699017030 shutdown_connections 2024-09-17T04:31:33.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.067+0000 7f36d2a00640 1 --2- 172.21.15.100:0/2699017030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36cc1532e0 0x7f36cc1536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:33.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.067+0000 7f36d2a00640 1 --2- 172.21.15.100:0/2699017030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36cc0705b0 0x7f36cc0af3c0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:33.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.067+0000 7f36d2a00640 1 --2- 172.21.15.100:0/2699017030 >> v2:172.21.15.184:3300/0 conn(0x7f36cc0b3ec0 0x7f36cc0b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:33.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.067+0000 7f36d2a00640 1 -- 172.21.15.100:0/2699017030 >> 172.21.15.100:0/2699017030 conn(0x7f36cc0b1dd0 msgr2=0x7f36cc0b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:33.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.068+0000 7f36d2a00640 1 -- 172.21.15.100:0/2699017030 shutdown_connections 2024-09-17T04:31:33.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.068+0000 7f36d2a00640 1 -- 172.21.15.100:0/2699017030 wait complete. 2024-09-17T04:31:33.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.070+0000 7f36d2a00640 1 Processor -- start 2024-09-17T04:31:33.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.070+0000 7f36d2a00640 1 -- start start 2024-09-17T04:31:33.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.071+0000 7f36d2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36cc0705b0 0x7f36cc1522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:33.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.071+0000 7f36d2a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36cc0b3ec0 0x7f36cc15cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:33.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.071+0000 7f36d2a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36cc1532e0 0x7f36cc15cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:33.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.071+0000 7f36d2a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc160860 con 0x7f36cc0705b0 2024-09-17T04:31:33.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.071+0000 7f36d2a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc1606e0 con 0x7f36cc1532e0 2024-09-17T04:31:33.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.071+0000 7f36d2a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f36cc1609e0 con 0x7f36cc0b3ec0 2024-09-17T04:31:33.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.071+0000 7f36d1600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36cc0705b0 0x7f36cc1522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:33.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.072+0000 7f36d1600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36cc0705b0 0x7f36cc1522d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49524/0 (socket says 172.21.15.100:49524) 2024-09-17T04:31:33.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.072+0000 7f36d1600640 1 -- 172.21.15.100:0/2639153961 learned_addr learned my addr 172.21.15.100:0/2639153961 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:33.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.072+0000 7f36d0c00640 1 --2- 172.21.15.100:0/2639153961 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36cc0b3ec0 0x7f36cc15cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:33.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.072+0000 7f36d2000640 1 --2- 172.21.15.100:0/2639153961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36cc1532e0 0x7f36cc15cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:33.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.072+0000 7f36d1600640 1 -- 172.21.15.100:0/2639153961 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36cc0b3ec0 msgr2=0x7f36cc15cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:33.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.072+0000 7f36d1600640 1 --2- 172.21.15.100:0/2639153961 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36cc0b3ec0 0x7f36cc15cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:33.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.072+0000 7f36d1600640 1 -- 172.21.15.100:0/2639153961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36cc1532e0 msgr2=0x7f36cc15cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:33.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.072+0000 7f36d1600640 1 --2- 172.21.15.100:0/2639153961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36cc1532e0 0x7f36cc15cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:33.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.072+0000 7f36d1600640 1 -- 172.21.15.100:0/2639153961 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36cc1f5500 con 0x7f36cc0705b0 2024-09-17T04:31:33.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.072+0000 7f36d1600640 1 --2- 172.21.15.100:0/2639153961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36cc0705b0 0x7f36cc1522d0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f36c000eb30 tx=0x7f36c0009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:33.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.072+0000 7f36c9600640 1 -- 172.21.15.100:0/2639153961 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c0010430 con 0x7f36cc0705b0 2024-09-17T04:31:33.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.072+0000 7f36c9600640 1 -- 172.21.15.100:0/2639153961 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36c00105d0 con 0x7f36cc0705b0 2024-09-17T04:31:33.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.072+0000 7f36c9600640 1 -- 172.21.15.100:0/2639153961 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c00109e0 con 0x7f36cc0705b0 2024-09-17T04:31:33.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.072+0000 7f36d2a00640 1 -- 172.21.15.100:0/2639153961 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36cc1f57f0 con 0x7f36cc0705b0 2024-09-17T04:31:33.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.073+0000 7f36d2a00640 1 -- 172.21.15.100:0/2639153961 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36cc1f5db0 con 0x7f36cc0705b0 2024-09-17T04:31:33.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.074+0000 7f36d2a00640 1 -- 172.21.15.100:0/2639153961 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36cc0b47c0 con 0x7f36cc0705b0 2024-09-17T04:31:33.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.075+0000 7f36c9600640 1 -- 172.21.15.100:0/2639153961 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 101900+0+0 (secure 0 0 0) 0x7f36c0010b80 con 0x7f36cc0705b0 2024-09-17T04:31:33.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.076+0000 7f36c9600640 1 --2- 172.21.15.100:0/2639153961 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f36ac0790a0 0x7f36ac07b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:33.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.076+0000 7f36c9600640 1 -- 172.21.15.100:0/2639153961 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7f36c009c650 con 0x7f36cc0705b0 2024-09-17T04:31:33.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.076+0000 7f36d0c00640 1 --2- 172.21.15.100:0/2639153961 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f36ac0790a0 0x7f36ac07b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:33.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.077+0000 7f36d0c00640 1 --2- 172.21.15.100:0/2639153961 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f36ac0790a0 0x7f36ac07b5b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f36b8033000 tx=0x7f36b8004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:33.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.080+0000 7f36c9600640 1 -- 172.21.15.100:0/2639153961 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f36c0064250 con 0x7f36cc0705b0 2024-09-17T04:31:33.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.289+0000 7f36d2a00640 1 -- 172.21.15.100:0/2639153961 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol11481", "group_name": "subvol_grp33350", "target": ["mon-mgr", ""]}) -- 0x7f36cc158920 con 0x7f36ac0790a0 2024-09-17T04:31:33.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.335+0000 7f36c9600640 1 -- 172.21.15.100:0/2639153961 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f36cc158920 con 0x7f36ac0790a0 2024-09-17T04:31:33.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.337+0000 7f36d2a00640 1 -- 172.21.15.100:0/2639153961 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f36ac0790a0 msgr2=0x7f36ac07b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:33.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.337+0000 7f36d2a00640 1 --2- 172.21.15.100:0/2639153961 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f36ac0790a0 0x7f36ac07b5b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f36b8033000 tx=0x7f36b8004730 comp rx=0 tx=0).stop 2024-09-17T04:31:33.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.337+0000 7f36d2a00640 1 -- 172.21.15.100:0/2639153961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36cc0705b0 msgr2=0x7f36cc1522d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:33.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.337+0000 7f36d2a00640 1 --2- 172.21.15.100:0/2639153961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36cc0705b0 0x7f36cc1522d0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f36c000eb30 tx=0x7f36c0009f70 comp rx=0 tx=0).stop 2024-09-17T04:31:33.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.337+0000 7f36d2a00640 1 -- 172.21.15.100:0/2639153961 shutdown_connections 2024-09-17T04:31:33.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.337+0000 7f36d2a00640 1 --2- 172.21.15.100:0/2639153961 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f36ac0790a0 0x7f36ac07b5b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:33.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.337+0000 7f36d2a00640 1 --2- 172.21.15.100:0/2639153961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36cc1532e0 0x7f36cc15cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:33.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.337+0000 7f36d2a00640 1 --2- 172.21.15.100:0/2639153961 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36cc0b3ec0 0x7f36cc15cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:33.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.337+0000 7f36d2a00640 1 --2- 172.21.15.100:0/2639153961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36cc0705b0 0x7f36cc1522d0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:33.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.337+0000 7f36d2a00640 1 -- 172.21.15.100:0/2639153961 >> 172.21.15.100:0/2639153961 conn(0x7f36cc0b1dd0 msgr2=0x7f36cc156be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:33.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.337+0000 7f36d2a00640 1 -- 172.21.15.100:0/2639153961 shutdown_connections 2024-09-17T04:31:33.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.338+0000 7f36d2a00640 1 -- 172.21.15.100:0/2639153961 wait complete. 2024-09-17T04:31:33.362 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol41972 2024-09-17T04:31:33.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.586+0000 7f9e07200640 1 -- 172.21.15.100:0/997147738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dec004800 msgr2=0x7f9dec004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:33.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.586+0000 7f9e07200640 1 --2- 172.21.15.100:0/997147738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dec004800 0x7f9dec004c10 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f9df000c290 tx=0x7f9dec005830 comp rx=0 tx=0).stop 2024-09-17T04:31:33.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.586+0000 7f9e06800640 1 -- 172.21.15.100:0/997147738 reap_dead start 2024-09-17T04:31:33.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.586+0000 7f9e07200640 1 -- 172.21.15.100:0/997147738 shutdown_connections 2024-09-17T04:31:33.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.586+0000 7f9e07200640 1 -- 172.21.15.100:0/997147738 >> 172.21.15.100:0/997147738 conn(0x7f9e0808dd50 msgr2=0x7f9e0809d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:33.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.586+0000 7f9e07200640 1 -- 172.21.15.100:0/997147738 shutdown_connections 2024-09-17T04:31:33.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.586+0000 7f9e07200640 1 -- 172.21.15.100:0/997147738 wait complete. 2024-09-17T04:31:33.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.588+0000 7f9e07200640 1 Processor -- start 2024-09-17T04:31:33.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.588+0000 7f9e07200640 1 -- start start 2024-09-17T04:31:33.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.590+0000 7f9e07200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9dec004800 0x7f9e080b8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:33.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.590+0000 7f9e07200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e080b89d0 0x7f9e080cd620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:33.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.590+0000 7f9e07200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e080cdb60 0x7f9e081f8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:33.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.590+0000 7f9e07200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9e080d2b40 con 0x7f9e080cdb60 2024-09-17T04:31:33.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.590+0000 7f9e07200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9e080d29c0 con 0x7f9dec004800 2024-09-17T04:31:33.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.590+0000 7f9e07200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9e080d2cc0 con 0x7f9e080b89d0 2024-09-17T04:31:33.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.590+0000 7f9e06800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e080cdb60 0x7f9e081f8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:33.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.590+0000 7f9e06800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e080cdb60 0x7f9e081f8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49544/0 (socket says 172.21.15.100:49544) 2024-09-17T04:31:33.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.590+0000 7f9e06800640 1 -- 172.21.15.100:0/577810696 learned_addr learned my addr 172.21.15.100:0/577810696 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:33.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.590+0000 7f9e05e00640 1 --2- 172.21.15.100:0/577810696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9dec004800 0x7f9e080b8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:33.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.590+0000 7f9dfd400640 1 --2- 172.21.15.100:0/577810696 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e080b89d0 0x7f9e080cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:33.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.590+0000 7f9e06800640 1 -- 172.21.15.100:0/577810696 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e080b89d0 msgr2=0x7f9e080cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:33.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.590+0000 7f9e06800640 1 --2- 172.21.15.100:0/577810696 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e080b89d0 0x7f9e080cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:33.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.591+0000 7f9e06800640 1 -- 172.21.15.100:0/577810696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9dec004800 msgr2=0x7f9e080b8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:33.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.591+0000 7f9e06800640 1 --2- 172.21.15.100:0/577810696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9dec004800 0x7f9e080b8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:33.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.591+0000 7f9e06800640 1 -- 172.21.15.100:0/577810696 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e0814f800 con 0x7f9e080cdb60 2024-09-17T04:31:33.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.591+0000 7f9e06800640 1 --2- 172.21.15.100:0/577810696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e080cdb60 0x7f9e081f8b30 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f9e00005fd0 tx=0x7f9e0000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:33.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.591+0000 7f9dfea00640 1 -- 172.21.15.100:0/577810696 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e00003d80 con 0x7f9e080cdb60 2024-09-17T04:31:33.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.591+0000 7f9dfea00640 1 -- 172.21.15.100:0/577810696 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e00004b20 con 0x7f9e080cdb60 2024-09-17T04:31:33.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.591+0000 7f9e07200640 1 -- 172.21.15.100:0/577810696 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e080b8de0 con 0x7f9e080cdb60 2024-09-17T04:31:33.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.591+0000 7f9dfea00640 1 -- 172.21.15.100:0/577810696 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e00004030 con 0x7f9e080cdb60 2024-09-17T04:31:33.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.591+0000 7f9e07200640 1 -- 172.21.15.100:0/577810696 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e08151ef0 con 0x7f9e080cdb60 2024-09-17T04:31:33.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.592+0000 7f9e07200640 1 -- 172.21.15.100:0/577810696 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e080cf830 con 0x7f9e080cdb60 2024-09-17T04:31:33.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.594+0000 7f9dfea00640 1 -- 172.21.15.100:0/577810696 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 101900+0+0 (secure 0 0 0) 0x7f9e000041d0 con 0x7f9e080cdb60 2024-09-17T04:31:33.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.595+0000 7f9dfea00640 1 --2- 172.21.15.100:0/577810696 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9de40790a0 0x7f9de407b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:33.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.595+0000 7f9dfea00640 1 -- 172.21.15.100:0/577810696 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7f9e0009c590 con 0x7f9e080cdb60 2024-09-17T04:31:33.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.595+0000 7f9e05e00640 1 --2- 172.21.15.100:0/577810696 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9de40790a0 0x7f9de407b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:33.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.596+0000 7f9e05e00640 1 --2- 172.21.15.100:0/577810696 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9de40790a0 0x7f9de407b5b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9df8005e50 tx=0x7f9df8005f30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:33.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.600+0000 7f9dfea00640 1 -- 172.21.15.100:0/577810696 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9e080b8de0 con 0x7f9e080cdb60 2024-09-17T04:31:33.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.636+0000 7f9dfea00640 1 -- 172.21.15.100:0/577810696 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f9e0005fd30 con 0x7f9e080cdb60 2024-09-17T04:31:33.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.816+0000 7f9e07200640 1 -- 172.21.15.100:0/577810696 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol41972", "target": ["mon-mgr", ""]}) -- 0x7f9e080b3550 con 0x7f9de40790a0 2024-09-17T04:31:33.837 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:31:33.837+0000 7f30b3000640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvolume 'subvol41972' is not ready for operation rm (use --force to override) 2024-09-17T04:31:33.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:33.838+0000 7f9dfea00640 1 -- 172.21.15.100:0/577810696 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: -11 subvolume 'subvol41972' is not ready for operation rm (use --force to override)) ==== 87+0+0 (secure 0 0 0) 0x7f9e080b3550 con 0x7f9de40790a0 2024-09-17T04:31:33.838 INFO:teuthology.orchestra.run.smithi100.stderr:Error EAGAIN: subvolume 'subvol41972' is not ready for operation rm (use --force to override) 2024-09-17T04:31:33.854 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-09-17T04:31:33.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:33.855 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol41972/.meta) 2024-09-17T04:31:33.909 INFO:teuthology.orchestra.run.smithi100.stdout:[GLOBAL] 2024-09-17T04:31:33.909 INFO:teuthology.orchestra.run.smithi100.stdout:version = 1 2024-09-17T04:31:33.910 INFO:teuthology.orchestra.run.smithi100.stdout:type = clone 2024-09-17T04:31:33.910 INFO:teuthology.orchestra.run.smithi100.stdout:path = /volumes/_nogroup/subvol41972/ed59f3df-09cd-4703-820c-9463bc08905f 2024-09-17T04:31:33.910 INFO:teuthology.orchestra.run.smithi100.stdout:state = pending 2024-09-17T04:31:33.910 INFO:teuthology.orchestra.run.smithi100.stdout:[source] 2024-09-17T04:31:33.910 INFO:teuthology.orchestra.run.smithi100.stdout:volume = fake 2024-09-17T04:31:33.910 INFO:teuthology.orchestra.run.smithi100.stdout:subvolume = fake 2024-09-17T04:31:33.910 INFO:teuthology.orchestra.run.smithi100.stdout:snapshot = fake 2024-09-17T04:31:33.912 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol41972 --force 2024-09-17T04:31:34.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.139+0000 7f2faf200640 1 -- 172.21.15.100:0/2309685356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fa8156db0 msgr2=0x7f2fa80b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:34.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.139+0000 7f2faf200640 1 --2- 172.21.15.100:0/2309685356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fa8156db0 0x7f2fa80b5f40 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8075550 tx=0x7f2f9802a580 comp rx=0 tx=0).stop 2024-09-17T04:31:34.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.139+0000 7f2faf200640 1 -- 172.21.15.100:0/2309685356 shutdown_connections 2024-09-17T04:31:34.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.140+0000 7f2faf200640 1 --2- 172.21.15.100:0/2309685356 >> v2:172.21.15.184:3300/0 conn(0x7f2fa806ddb0 0x7f2fa806e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:34.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.140+0000 7f2faf200640 1 --2- 172.21.15.100:0/2309685356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2fa80745c0 0x7f2fa815d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:34.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.140+0000 7f2faf200640 1 --2- 172.21.15.100:0/2309685356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fa8156db0 0x7f2fa80b5f40 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:34.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.140+0000 7f2faf200640 1 -- 172.21.15.100:0/2309685356 >> 172.21.15.100:0/2309685356 conn(0x7f2fa8073890 msgr2=0x7f2fa8073ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:34.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.140+0000 7f2faf200640 1 -- 172.21.15.100:0/2309685356 shutdown_connections 2024-09-17T04:31:34.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.140+0000 7f2faf200640 1 -- 172.21.15.100:0/2309685356 wait complete. 2024-09-17T04:31:34.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.142+0000 7f2faf200640 1 Processor -- start 2024-09-17T04:31:34.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.142+0000 7f2faf200640 1 -- start start 2024-09-17T04:31:34.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.143+0000 7f2faf200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fa806ddb0 0x7f2fa8071590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:34.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.143+0000 7f2faf200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2fa80745c0 0x7f2fa814f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:34.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.143+0000 7f2faf200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fa8156db0 0x7f2fa8158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:34.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.143+0000 7f2faf200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa80ce810 con 0x7f2fa8156db0 2024-09-17T04:31:34.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.143+0000 7f2faf200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa80ce690 con 0x7f2fa806ddb0 2024-09-17T04:31:34.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.143+0000 7f2faf200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2fa80ce990 con 0x7f2fa80745c0 2024-09-17T04:31:34.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.144+0000 7f2fad200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fa8156db0 0x7f2fa8158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:34.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.144+0000 7f2fad200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fa8156db0 0x7f2fa8158380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49562/0 (socket says 172.21.15.100:49562) 2024-09-17T04:31:34.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.144+0000 7f2fad200640 1 -- 172.21.15.100:0/168189585 learned_addr learned my addr 172.21.15.100:0/168189585 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:34.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.144+0000 7f2fa7e00640 1 --2- 172.21.15.100:0/168189585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fa806ddb0 0x7f2fa8071590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:34.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.144+0000 7f2f9f400640 1 --2- 172.21.15.100:0/168189585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2fa80745c0 0x7f2fa814f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:34.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.144+0000 7f2fad200640 1 -- 172.21.15.100:0/168189585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2fa80745c0 msgr2=0x7f2fa814f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:34.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.144+0000 7f2fad200640 1 --2- 172.21.15.100:0/168189585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2fa80745c0 0x7f2fa814f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:34.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.144+0000 7f2fad200640 1 -- 172.21.15.100:0/168189585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fa806ddb0 msgr2=0x7f2fa8071590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:34.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.144+0000 7f2fad200640 1 --2- 172.21.15.100:0/168189585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fa806ddb0 0x7f2fa8071590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:34.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.144+0000 7f2fad200640 1 -- 172.21.15.100:0/168189585 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa80cc500 con 0x7f2fa8156db0 2024-09-17T04:31:34.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.144+0000 7f2fad200640 1 --2- 172.21.15.100:0/168189585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fa8156db0 0x7f2fa8158380 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f2fa000e910 tx=0x7f2fa000edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:34.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.145+0000 7f2fa4c00640 1 -- 172.21.15.100:0/168189585 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0004d20 con 0x7f2fa8156db0 2024-09-17T04:31:34.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.145+0000 7f2fa4c00640 1 -- 172.21.15.100:0/168189585 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fa000cb20 con 0x7f2fa8156db0 2024-09-17T04:31:34.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.145+0000 7f2fa4c00640 1 -- 172.21.15.100:0/168189585 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0005020 con 0x7f2fa8156db0 2024-09-17T04:31:34.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.145+0000 7f2faf200640 1 -- 172.21.15.100:0/168189585 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fa80cc7e0 con 0x7f2fa8156db0 2024-09-17T04:31:34.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.145+0000 7f2faf200640 1 -- 172.21.15.100:0/168189585 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fa814fcf0 con 0x7f2fa8156db0 2024-09-17T04:31:34.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.146+0000 7f2faf200640 1 -- 172.21.15.100:0/168189585 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fa80d00a0 con 0x7f2fa8156db0 2024-09-17T04:31:34.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.147+0000 7f2fa4c00640 1 -- 172.21.15.100:0/168189585 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f2fa00051c0 con 0x7f2fa8156db0 2024-09-17T04:31:34.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.148+0000 7f2fa4c00640 1 --2- 172.21.15.100:0/168189585 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2f840791c0 0x7f2f8407b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:34.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.148+0000 7f2fa4c00640 1 -- 172.21.15.100:0/168189585 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7f2fa009c1a0 con 0x7f2fa8156db0 2024-09-17T04:31:34.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.148+0000 7f2fa7e00640 1 --2- 172.21.15.100:0/168189585 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2f840791c0 0x7f2f8407b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:34.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.149+0000 7f2fa7e00640 1 --2- 172.21.15.100:0/168189585 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2f840791c0 0x7f2f8407b6d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2f98033000 tx=0x7f2f98004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:34.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.153+0000 7f2fa4c00640 1 -- 172.21.15.100:0/168189585 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f2fa0016030 con 0x7f2fa8156db0 2024-09-17T04:31:34.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.364+0000 7f2faf200640 1 -- 172.21.15.100:0/168189585 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol41972", "force": true, "target": ["mon-mgr", ""]}) -- 0x7f2fa8073e20 con 0x7f2f840791c0 2024-09-17T04:31:34.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.391+0000 7f2fa4c00640 1 -- 172.21.15.100:0/168189585 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2fa8073e20 con 0x7f2f840791c0 2024-09-17T04:31:34.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.393+0000 7f2faf200640 1 -- 172.21.15.100:0/168189585 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2f840791c0 msgr2=0x7f2f8407b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:34.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.393+0000 7f2faf200640 1 --2- 172.21.15.100:0/168189585 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2f840791c0 0x7f2f8407b6d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2f98033000 tx=0x7f2f98004730 comp rx=0 tx=0).stop 2024-09-17T04:31:34.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.393+0000 7f2faf200640 1 -- 172.21.15.100:0/168189585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fa8156db0 msgr2=0x7f2fa8158380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:34.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.393+0000 7f2faf200640 1 --2- 172.21.15.100:0/168189585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fa8156db0 0x7f2fa8158380 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f2fa000e910 tx=0x7f2fa000edd0 comp rx=0 tx=0).stop 2024-09-17T04:31:34.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.393+0000 7f2faf200640 1 -- 172.21.15.100:0/168189585 shutdown_connections 2024-09-17T04:31:34.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.393+0000 7f2faf200640 1 --2- 172.21.15.100:0/168189585 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2f840791c0 0x7f2f8407b6d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:34.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.393+0000 7f2faf200640 1 --2- 172.21.15.100:0/168189585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fa8156db0 0x7f2fa8158380 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:34.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.393+0000 7f2faf200640 1 --2- 172.21.15.100:0/168189585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2fa80745c0 0x7f2fa814f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:34.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.393+0000 7f2faf200640 1 --2- 172.21.15.100:0/168189585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fa806ddb0 0x7f2fa8071590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:34.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.394+0000 7f2faf200640 1 -- 172.21.15.100:0/168189585 >> 172.21.15.100:0/168189585 conn(0x7f2fa8073890 msgr2=0x7f2fa80b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:34.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.394+0000 7f2faf200640 1 -- 172.21.15.100:0/168189585 shutdown_connections 2024-09-17T04:31:34.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.394+0000 7f2faf200640 1 -- 172.21.15.100:0/168189585 wait complete. 2024-09-17T04:31:34.421 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2024-09-17T04:31:34.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.643+0000 7f0415a00640 1 -- 172.21.15.100:0/3708905305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0410145a40 msgr2=0x7f0410073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:34.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.643+0000 7f0415a00640 1 --2- 172.21.15.100:0/3708905305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0410145a40 0x7f0410073540 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f0410072bd0 tx=0x7f03f802a680 comp rx=0 tx=0).stop 2024-09-17T04:31:34.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.644+0000 7f0415a00640 1 -- 172.21.15.100:0/3708905305 shutdown_connections 2024-09-17T04:31:34.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.644+0000 7f0415a00640 1 --2- 172.21.15.100:0/3708905305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0410145a40 0x7f0410073540 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:34.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.644+0000 7f0415a00640 1 --2- 172.21.15.100:0/3708905305 >> v2:172.21.15.184:3300/0 conn(0x7f0410145550 0x7f041006fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:34.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.644+0000 7f0415a00640 1 -- 172.21.15.100:0/3708905305 >> 172.21.15.100:0/3708905305 conn(0x7f041006d1c0 msgr2=0x7f041006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:34.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.644+0000 7f0415a00640 1 -- 172.21.15.100:0/3708905305 shutdown_connections 2024-09-17T04:31:34.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.644+0000 7f0415a00640 1 -- 172.21.15.100:0/3708905305 wait complete. 2024-09-17T04:31:34.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.646+0000 7f0415a00640 1 Processor -- start 2024-09-17T04:31:34.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.646+0000 7f0415a00640 1 -- start start 2024-09-17T04:31:34.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.648+0000 7f0415a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0410073a80 0x7f041015bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:34.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.648+0000 7f0415a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0410145550 0x7f04100b2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:34.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.648+0000 7f0415a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0410145a40 0x7f04100b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:34.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.648+0000 7f0415a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0410160a30 con 0x7f0410145a40 2024-09-17T04:31:34.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.648+0000 7f0415a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f04100ce870 con 0x7f0410073a80 2024-09-17T04:31:34.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.648+0000 7f0415a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0410160bb0 con 0x7f0410145550 2024-09-17T04:31:34.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.648+0000 7f040f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0410145a40 0x7f04100b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:34.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.648+0000 7f040ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0410073a80 0x7f041015bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:34.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.648+0000 7f040f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0410145a40 0x7f04100b2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49574/0 (socket says 172.21.15.100:49574) 2024-09-17T04:31:34.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.648+0000 7f040f400640 1 -- 172.21.15.100:0/1521657557 learned_addr learned my addr 172.21.15.100:0/1521657557 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:34.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.648+0000 7f0407e00640 1 --2- 172.21.15.100:0/1521657557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0410145550 0x7f04100b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:34.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.648+0000 7f040f400640 1 -- 172.21.15.100:0/1521657557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0410145550 msgr2=0x7f04100b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:34.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.648+0000 7f040f400640 1 --2- 172.21.15.100:0/1521657557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0410145550 0x7f04100b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:34.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.649+0000 7f040f400640 1 -- 172.21.15.100:0/1521657557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0410073a80 msgr2=0x7f041015bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:34.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.649+0000 7f040f400640 1 --2- 172.21.15.100:0/1521657557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0410073a80 0x7f041015bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:34.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.649+0000 7f040f400640 1 -- 172.21.15.100:0/1521657557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0410138870 con 0x7f0410145a40 2024-09-17T04:31:34.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.649+0000 7f040f400640 1 --2- 172.21.15.100:0/1521657557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0410145a40 0x7f04100b2a70 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f040800c6b0 tx=0x7f040800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:34.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.649+0000 7f0406a00640 1 -- 172.21.15.100:0/1521657557 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0408003b70 con 0x7f0410145a40 2024-09-17T04:31:34.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.649+0000 7f0406a00640 1 -- 172.21.15.100:0/1521657557 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0408003d10 con 0x7f0410145a40 2024-09-17T04:31:34.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.649+0000 7f0406a00640 1 -- 172.21.15.100:0/1521657557 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0408005330 con 0x7f0410145a40 2024-09-17T04:31:34.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.649+0000 7f0415a00640 1 -- 172.21.15.100:0/1521657557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0410138b30 con 0x7f0410145a40 2024-09-17T04:31:34.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.649+0000 7f0415a00640 1 -- 172.21.15.100:0/1521657557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f041013a710 con 0x7f0410145a40 2024-09-17T04:31:34.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.650+0000 7f0415a00640 1 -- 172.21.15.100:0/1521657557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04101f7ea0 con 0x7f0410145a40 2024-09-17T04:31:34.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.652+0000 7f0406a00640 1 -- 172.21.15.100:0/1521657557 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f04080054d0 con 0x7f0410145a40 2024-09-17T04:31:34.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.653+0000 7f0406a00640 1 --2- 172.21.15.100:0/1521657557 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f03ec0791c0 0x7f03ec07b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:34.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.653+0000 7f0406a00640 1 -- 172.21.15.100:0/1521657557 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7f040809b630 con 0x7f0410145a40 2024-09-17T04:31:34.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.653+0000 7f040ea00640 1 --2- 172.21.15.100:0/1521657557 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f03ec0791c0 0x7f03ec07b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:34.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.654+0000 7f040ea00640 1 --2- 172.21.15.100:0/1521657557 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f03ec0791c0 0x7f03ec07b6d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f040000bb50 tx=0x7f040000b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:34.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.659+0000 7f0406a00640 1 -- 172.21.15.100:0/1521657557 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0410073e90 con 0x7f0410145a40 2024-09-17T04:31:34.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.870+0000 7f0415a00640 1 -- 172.21.15.100:0/1521657557 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7f041013a9b0 con 0x7f03ec0791c0 2024-09-17T04:31:34.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.874+0000 7f0406a00640 1 -- 172.21.15.100:0/1521657557 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f041013a9b0 con 0x7f03ec0791c0 2024-09-17T04:31:34.873 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-09-17T04:31:34.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.876+0000 7f0415a00640 1 -- 172.21.15.100:0/1521657557 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f03ec0791c0 msgr2=0x7f03ec07b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:34.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.876+0000 7f0415a00640 1 --2- 172.21.15.100:0/1521657557 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f03ec0791c0 0x7f03ec07b6d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f040000bb50 tx=0x7f040000b2e0 comp rx=0 tx=0).stop 2024-09-17T04:31:34.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.876+0000 7f0415a00640 1 -- 172.21.15.100:0/1521657557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0410145a40 msgr2=0x7f04100b2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:34.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.876+0000 7f0415a00640 1 --2- 172.21.15.100:0/1521657557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0410145a40 0x7f04100b2a70 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f040800c6b0 tx=0x7f040800cb70 comp rx=0 tx=0).stop 2024-09-17T04:31:34.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.876+0000 7f0415a00640 1 -- 172.21.15.100:0/1521657557 shutdown_connections 2024-09-17T04:31:34.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.876+0000 7f0415a00640 1 --2- 172.21.15.100:0/1521657557 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f03ec0791c0 0x7f03ec07b6d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:34.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.876+0000 7f0415a00640 1 --2- 172.21.15.100:0/1521657557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0410145a40 0x7f04100b2a70 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:34.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.876+0000 7f0415a00640 1 --2- 172.21.15.100:0/1521657557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0410145550 0x7f04100b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:34.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.876+0000 7f0415a00640 1 --2- 172.21.15.100:0/1521657557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0410073a80 0x7f041015bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:34.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.876+0000 7f0415a00640 1 -- 172.21.15.100:0/1521657557 >> 172.21.15.100:0/1521657557 conn(0x7f041006d1c0 msgr2=0x7f04101412c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:34.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.876+0000 7f0415a00640 1 -- 172.21.15.100:0/1521657557 shutdown_connections 2024-09-17T04:31:34.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34.876+0000 7f0415a00640 1 -- 172.21.15.100:0/1521657557 wait complete. 2024-09-17T04:31:34.899 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-17T04:31:34.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:31:34.988 INFO:teuthology.orchestra.run.smithi100.stdout:2 2024-09-17T04:31:34.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:31:34.996 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T04:31:34.996 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:31:35.093 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:31:35.192 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T04:31:35.192 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:31:35.192 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:31:35.192 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:31:35.228 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:31:35.228 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:31:35.303 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:31:35.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.566+0000 7f4078000640 1 -- 172.21.15.100:0/581268464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40700cfc80 msgr2=0x7f4070156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:35.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.566+0000 7f4078000640 1 --2- 172.21.15.100:0/581268464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40700cfc80 0x7f4070156e40 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f4064005fd0 tx=0x7f406402a820 comp rx=0 tx=0).stop 2024-09-17T04:31:35.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.566+0000 7f4078000640 1 -- 172.21.15.100:0/581268464 shutdown_connections 2024-09-17T04:31:35.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.566+0000 7f4078000640 1 --2- 172.21.15.100:0/581268464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40700705b0 0x7f40700709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:35.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.566+0000 7f4078000640 1 --2- 172.21.15.100:0/581268464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40700cfc80 0x7f4070156e40 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:35.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.566+0000 7f4078000640 1 -- 172.21.15.100:0/581268464 >> 172.21.15.100:0/581268464 conn(0x7f407006d3d0 msgr2=0x7f40700b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:35.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.566+0000 7f4078000640 1 -- 172.21.15.100:0/581268464 shutdown_connections 2024-09-17T04:31:35.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.566+0000 7f4078000640 1 -- 172.21.15.100:0/581268464 wait complete. 2024-09-17T04:31:35.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.568+0000 7f4078000640 1 Processor -- start 2024-09-17T04:31:35.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.568+0000 7f4078000640 1 -- start start 2024-09-17T04:31:35.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.570+0000 7f4078000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40700705b0 0x7f4070155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:35.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.570+0000 7f4078000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40700cfc80 0x7f40701f5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:35.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.570+0000 7f4078000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f407015ce30 0x7f407015b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:35.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.570+0000 7f4078000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f40700cce40 con 0x7f40700cfc80 2024-09-17T04:31:35.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.570+0000 7f4078000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f40700cccc0 con 0x7f40700705b0 2024-09-17T04:31:35.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.570+0000 7f4078000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f40700ccfc0 con 0x7f407015ce30 2024-09-17T04:31:35.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.570+0000 7f4074c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40700cfc80 0x7f40701f5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:35.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.570+0000 7f4074c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40700cfc80 0x7f40701f5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39926/0 (socket says 172.21.15.100:39926) 2024-09-17T04:31:35.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.570+0000 7f4074c00640 1 -- 172.21.15.100:0/1098054408 learned_addr learned my addr 172.21.15.100:0/1098054408 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:35.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.570+0000 7f4076000640 1 --2- 172.21.15.100:0/1098054408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f407015ce30 0x7f407015b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:35.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.570+0000 7f4075600640 1 --2- 172.21.15.100:0/1098054408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40700705b0 0x7f4070155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:35.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.570+0000 7f4074c00640 1 -- 172.21.15.100:0/1098054408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f407015ce30 msgr2=0x7f407015b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:35.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.570+0000 7f4074c00640 1 --2- 172.21.15.100:0/1098054408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f407015ce30 0x7f407015b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:35.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.570+0000 7f4074c00640 1 -- 172.21.15.100:0/1098054408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40700705b0 msgr2=0x7f4070155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:35.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.570+0000 7f4074c00640 1 --2- 172.21.15.100:0/1098054408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40700705b0 0x7f4070155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:35.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.570+0000 7f4074c00640 1 -- 172.21.15.100:0/1098054408 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40701f6a10 con 0x7f40700cfc80 2024-09-17T04:31:35.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.571+0000 7f4074c00640 1 --2- 172.21.15.100:0/1098054408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40700cfc80 0x7f40701f5ef0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f405c00d740 tx=0x7f405c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:35.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.571+0000 7f406d600640 1 -- 172.21.15.100:0/1098054408 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405c017070 con 0x7f40700cfc80 2024-09-17T04:31:35.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.571+0000 7f406d600640 1 -- 172.21.15.100:0/1098054408 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f405c009d20 con 0x7f40700cfc80 2024-09-17T04:31:35.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.571+0000 7f4078000640 1 -- 172.21.15.100:0/1098054408 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40701f6cf0 con 0x7f40700cfc80 2024-09-17T04:31:35.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.571+0000 7f406d600640 1 -- 172.21.15.100:0/1098054408 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405c0106c0 con 0x7f40700cfc80 2024-09-17T04:31:35.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.571+0000 7f4078000640 1 -- 172.21.15.100:0/1098054408 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40700c7680 con 0x7f40700cfc80 2024-09-17T04:31:35.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.572+0000 7f4078000640 1 -- 172.21.15.100:0/1098054408 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40700709c0 con 0x7f40700cfc80 2024-09-17T04:31:35.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.573+0000 7f406d600640 1 -- 172.21.15.100:0/1098054408 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f405c01a070 con 0x7f40700cfc80 2024-09-17T04:31:35.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.574+0000 7f406d600640 1 --2- 172.21.15.100:0/1098054408 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f40500791f0 0x7f405007b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:35.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.575+0000 7f406d600640 1 -- 172.21.15.100:0/1098054408 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7f405c09ba30 con 0x7f40700cfc80 2024-09-17T04:31:35.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.575+0000 7f4075600640 1 --2- 172.21.15.100:0/1098054408 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f40500791f0 0x7f405007b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:35.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.575+0000 7f4075600640 1 --2- 172.21.15.100:0/1098054408 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f40500791f0 0x7f405007b700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4064034000 tx=0x7f4064004ab0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:35.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.579+0000 7f406d600640 1 -- 172.21.15.100:0/1098054408 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f405c062e00 con 0x7f40700cfc80 2024-09-17T04:31:35.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.777+0000 7f4078000640 1 -- 172.21.15.100:0/1098054408 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f40700afa70 con 0x7f40700cfc80 2024-09-17T04:31:35.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.777+0000 7f406d600640 1 -- 172.21.15.100:0/1098054408 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v119) ==== 80+0+0 (secure 0 0 0) 0x7f405c067830 con 0x7f40700cfc80 2024-09-17T04:31:35.777 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-09-17T04:31:35.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.779+0000 7f4078000640 1 -- 172.21.15.100:0/1098054408 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f40500791f0 msgr2=0x7f405007b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:35.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.779+0000 7f4078000640 1 --2- 172.21.15.100:0/1098054408 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f40500791f0 0x7f405007b700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4064034000 tx=0x7f4064004ab0 comp rx=0 tx=0).stop 2024-09-17T04:31:35.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.779+0000 7f4078000640 1 -- 172.21.15.100:0/1098054408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40700cfc80 msgr2=0x7f40701f5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:35.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.780+0000 7f4078000640 1 --2- 172.21.15.100:0/1098054408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40700cfc80 0x7f40701f5ef0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f405c00d740 tx=0x7f405c00dc00 comp rx=0 tx=0).stop 2024-09-17T04:31:35.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.780+0000 7f4078000640 1 -- 172.21.15.100:0/1098054408 shutdown_connections 2024-09-17T04:31:35.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.780+0000 7f4078000640 1 --2- 172.21.15.100:0/1098054408 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f40500791f0 0x7f405007b700 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:35.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.780+0000 7f4078000640 1 --2- 172.21.15.100:0/1098054408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f407015ce30 0x7f407015b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:35.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.780+0000 7f4078000640 1 --2- 172.21.15.100:0/1098054408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40700cfc80 0x7f40701f5ef0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:35.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.780+0000 7f4078000640 1 --2- 172.21.15.100:0/1098054408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40700705b0 0x7f4070155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:35.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.780+0000 7f4078000640 1 -- 172.21.15.100:0/1098054408 >> 172.21.15.100:0/1098054408 conn(0x7f407006d3d0 msgr2=0x7f407015f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:35.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.780+0000 7f4078000640 1 -- 172.21.15.100:0/1098054408 shutdown_connections 2024-09-17T04:31:35.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:35.780+0000 7f4078000640 1 -- 172.21.15.100:0/1098054408 wait complete. 2024-09-17T04:31:35.803 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T04:31:35.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:35.804 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T04:31:35.894 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-09-17T04:31:35.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:35.895 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T04:31:35.949 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:31:35.949 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:31:35.949 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:31:35.986 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:31:35.986 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:31:36.003 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:31:36.007 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:31:36.064 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:31:36.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.286+0000 7f4769e00640 1 -- 172.21.15.100:0/2308508077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47640cfc80 msgr2=0x7f4764156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:36.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.286+0000 7f4769e00640 1 --2- 172.21.15.100:0/2308508077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47640cfc80 0x7f4764156e40 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f4754009860 tx=0x7f475402a560 comp rx=0 tx=0).stop 2024-09-17T04:31:36.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.286+0000 7f4769e00640 1 -- 172.21.15.100:0/2308508077 shutdown_connections 2024-09-17T04:31:36.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.286+0000 7f4769e00640 1 --2- 172.21.15.100:0/2308508077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47640cfc80 0x7f4764156e40 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:36.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.286+0000 7f4769e00640 1 -- 172.21.15.100:0/2308508077 >> 172.21.15.100:0/2308508077 conn(0x7f476406d3d0 msgr2=0x7f47640b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:36.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.286+0000 7f4769e00640 1 -- 172.21.15.100:0/2308508077 shutdown_connections 2024-09-17T04:31:36.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.286+0000 7f4769e00640 1 -- 172.21.15.100:0/2308508077 wait complete. 2024-09-17T04:31:36.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.289+0000 7f4769e00640 1 Processor -- start 2024-09-17T04:31:36.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.289+0000 7f4769e00640 1 -- start start 2024-09-17T04:31:36.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.290+0000 7f4769e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47640705b0 0x7f47640c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:36.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.290+0000 7f4769e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47640cfc80 0x7f47640caff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:36.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.290+0000 7f4769e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f476415ce30 0x7f47640b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:36.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.290+0000 7f4769e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f47640cd100 con 0x7f47640cfc80 2024-09-17T04:31:36.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.290+0000 7f4769e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f47640ccf80 con 0x7f47640705b0 2024-09-17T04:31:36.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.290+0000 7f4769e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f47640cd280 con 0x7f476415ce30 2024-09-17T04:31:36.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.290+0000 7f4762000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47640cfc80 0x7f47640caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:36.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.290+0000 7f4762000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47640cfc80 0x7f47640caff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39946/0 (socket says 172.21.15.100:39946) 2024-09-17T04:31:36.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.290+0000 7f4762000640 1 -- 172.21.15.100:0/3721475496 learned_addr learned my addr 172.21.15.100:0/3721475496 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:36.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.291+0000 7f4762a00640 1 --2- 172.21.15.100:0/3721475496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47640705b0 0x7f47640c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:36.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.291+0000 7f4763400640 1 --2- 172.21.15.100:0/3721475496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f476415ce30 0x7f47640b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:36.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.291+0000 7f4762000640 1 -- 172.21.15.100:0/3721475496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f476415ce30 msgr2=0x7f47640b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:36.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.291+0000 7f4762000640 1 --2- 172.21.15.100:0/3721475496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f476415ce30 0x7f47640b39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:36.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.291+0000 7f4762000640 1 -- 172.21.15.100:0/3721475496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47640705b0 msgr2=0x7f47640c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:36.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.291+0000 7f4762000640 1 --2- 172.21.15.100:0/3721475496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47640705b0 0x7f47640c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:36.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.291+0000 7f4762000640 1 -- 172.21.15.100:0/3721475496 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47641f5830 con 0x7f47640cfc80 2024-09-17T04:31:36.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.291+0000 7f4762000640 1 --2- 172.21.15.100:0/3721475496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47640cfc80 0x7f47640caff0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f475000c370 tx=0x7f475000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:36.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.291+0000 7f475b400640 1 -- 172.21.15.100:0/3721475496 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4750011790 con 0x7f47640cfc80 2024-09-17T04:31:36.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.292+0000 7f475b400640 1 -- 172.21.15.100:0/3721475496 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4750017070 con 0x7f47640cfc80 2024-09-17T04:31:36.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.292+0000 7f475b400640 1 -- 172.21.15.100:0/3721475496 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4750011cc0 con 0x7f47640cfc80 2024-09-17T04:31:36.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.292+0000 7f4769e00640 1 -- 172.21.15.100:0/3721475496 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47640cb910 con 0x7f47640cfc80 2024-09-17T04:31:36.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.292+0000 7f4769e00640 1 -- 172.21.15.100:0/3721475496 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47641f5df0 con 0x7f47640cfc80 2024-09-17T04:31:36.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.293+0000 7f4769e00640 1 -- 172.21.15.100:0/3721475496 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47640cd770 con 0x7f47640cfc80 2024-09-17T04:31:36.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.294+0000 7f475b400640 1 -- 172.21.15.100:0/3721475496 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f475001c460 con 0x7f47640cfc80 2024-09-17T04:31:36.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.295+0000 7f475b400640 1 --2- 172.21.15.100:0/3721475496 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f47440791c0 0x7f474407b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:36.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.295+0000 7f475b400640 1 -- 172.21.15.100:0/3721475496 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7f475009b620 con 0x7f47640cfc80 2024-09-17T04:31:36.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.295+0000 7f4762a00640 1 --2- 172.21.15.100:0/3721475496 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f47440791c0 0x7f474407b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:36.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.296+0000 7f4762a00640 1 --2- 172.21.15.100:0/3721475496 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f47440791c0 0x7f474407b6d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4754034000 tx=0x7f4754004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:36.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.299+0000 7f475b400640 1 -- 172.21.15.100:0/3721475496 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f47500630f0 con 0x7f47640cfc80 2024-09-17T04:31:36.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.494+0000 7f4769e00640 1 -- 172.21.15.100:0/3721475496 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f47640afa70 con 0x7f47640cfc80 2024-09-17T04:31:36.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.494+0000 7f475b400640 1 -- 172.21.15.100:0/3721475496 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v119) ==== 80+0+0 (secure 0 0 0) 0x7f47500629f0 con 0x7f47640cfc80 2024-09-17T04:31:36.494 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-09-17T04:31:36.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.496+0000 7f4769e00640 1 -- 172.21.15.100:0/3721475496 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f47440791c0 msgr2=0x7f474407b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:36.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.497+0000 7f4769e00640 1 --2- 172.21.15.100:0/3721475496 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f47440791c0 0x7f474407b6d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4754034000 tx=0x7f4754004730 comp rx=0 tx=0).stop 2024-09-17T04:31:36.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.497+0000 7f4769e00640 1 -- 172.21.15.100:0/3721475496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47640cfc80 msgr2=0x7f47640caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:36.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.497+0000 7f4769e00640 1 --2- 172.21.15.100:0/3721475496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47640cfc80 0x7f47640caff0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f475000c370 tx=0x7f475000c830 comp rx=0 tx=0).stop 2024-09-17T04:31:36.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.497+0000 7f4769e00640 1 -- 172.21.15.100:0/3721475496 shutdown_connections 2024-09-17T04:31:36.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.497+0000 7f4769e00640 1 --2- 172.21.15.100:0/3721475496 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f47440791c0 0x7f474407b6d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:36.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.497+0000 7f4769e00640 1 --2- 172.21.15.100:0/3721475496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f476415ce30 0x7f47640b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:36.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.497+0000 7f4769e00640 1 --2- 172.21.15.100:0/3721475496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47640cfc80 0x7f47640caff0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:36.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.497+0000 7f4769e00640 1 --2- 172.21.15.100:0/3721475496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47640705b0 0x7f47640c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:36.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.497+0000 7f4769e00640 1 -- 172.21.15.100:0/3721475496 >> 172.21.15.100:0/3721475496 conn(0x7f476406d3d0 msgr2=0x7f47640b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:36.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.497+0000 7f4769e00640 1 -- 172.21.15.100:0/3721475496 shutdown_connections 2024-09-17T04:31:36.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.497+0000 7f4769e00640 1 -- 172.21.15.100:0/3721475496 wait complete. 2024-09-17T04:31:36.521 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-17T04:31:36.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:36.521 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-17T04:31:36.585 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2024-09-17T04:31:36.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:31:36.585 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T04:31:36.622 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:31:36.622 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:31:36.622 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:31:36.659 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:31:36.659 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:31:36.659 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:31:36.659 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:31:36.698 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:31:36.699 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:31:36.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.920+0000 7f37eb200640 1 -- 172.21.15.100:0/994744670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37ec0b58b0 msgr2=0x7f37ec0b46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:36.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.921+0000 7f37eb200640 1 --2- 172.21.15.100:0/994744670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37ec0b58b0 0x7f37ec0b46e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f37d4009850 tx=0x7f37d402a5a0 comp rx=0 tx=0).stop 2024-09-17T04:31:36.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.921+0000 7f37eb200640 1 -- 172.21.15.100:0/994744670 shutdown_connections 2024-09-17T04:31:36.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.921+0000 7f37eb200640 1 --2- 172.21.15.100:0/994744670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37ec0b58b0 0x7f37ec0b46e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:36.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.921+0000 7f37eb200640 1 -- 172.21.15.100:0/994744670 >> 172.21.15.100:0/994744670 conn(0x7f37ec09d820 msgr2=0x7f37ec08bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:36.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.921+0000 7f37eb200640 1 -- 172.21.15.100:0/994744670 shutdown_connections 2024-09-17T04:31:36.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.921+0000 7f37eb200640 1 -- 172.21.15.100:0/994744670 wait complete. 2024-09-17T04:31:36.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.923+0000 7f37eb200640 1 Processor -- start 2024-09-17T04:31:36.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.923+0000 7f37eb200640 1 -- start start 2024-09-17T04:31:36.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.924+0000 7f37eb200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37ec0b58b0 0x7f37ec09e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:36.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.924+0000 7f37eb200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37ec0b98d0 0x7f37ec154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:36.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.925+0000 7f37eb200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37ec0d0e70 0x7f37ec0cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:36.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.925+0000 7f37eb200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec0cfec0 con 0x7f37ec0d0e70 2024-09-17T04:31:36.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.925+0000 7f37eb200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec0cfd40 con 0x7f37ec0b58b0 2024-09-17T04:31:36.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.925+0000 7f37eb200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f37ec0d0040 con 0x7f37ec0b98d0 2024-09-17T04:31:36.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.925+0000 7f37ea800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37ec0d0e70 0x7f37ec0cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:36.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.925+0000 7f37ea800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37ec0d0e70 0x7f37ec0cdf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39954/0 (socket says 172.21.15.100:39954) 2024-09-17T04:31:36.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.925+0000 7f37ea800640 1 -- 172.21.15.100:0/212848694 learned_addr learned my addr 172.21.15.100:0/212848694 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:36.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.925+0000 7f37e1400640 1 --2- 172.21.15.100:0/212848694 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37ec0b98d0 0x7f37ec154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:36.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.925+0000 7f37e9e00640 1 --2- 172.21.15.100:0/212848694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37ec0b58b0 0x7f37ec09e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:36.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.925+0000 7f37ea800640 1 -- 172.21.15.100:0/212848694 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37ec0b98d0 msgr2=0x7f37ec154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:36.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.925+0000 7f37ea800640 1 --2- 172.21.15.100:0/212848694 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37ec0b98d0 0x7f37ec154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:36.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.925+0000 7f37ea800640 1 -- 172.21.15.100:0/212848694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37ec0b58b0 msgr2=0x7f37ec09e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:36.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.925+0000 7f37ea800640 1 --2- 172.21.15.100:0/212848694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37ec0b58b0 0x7f37ec09e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:36.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.925+0000 7f37ea800640 1 -- 172.21.15.100:0/212848694 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ec0c9f10 con 0x7f37ec0d0e70 2024-09-17T04:31:36.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.926+0000 7f37ea800640 1 --2- 172.21.15.100:0/212848694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37ec0d0e70 0x7f37ec0cdf00 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f37e400c730 tx=0x7f37e400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:36.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.926+0000 7f37e2a00640 1 -- 172.21.15.100:0/212848694 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e4003b70 con 0x7f37ec0d0e70 2024-09-17T04:31:36.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.926+0000 7f37e2a00640 1 -- 172.21.15.100:0/212848694 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37e4003d30 con 0x7f37ec0d0e70 2024-09-17T04:31:36.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.926+0000 7f37e2a00640 1 -- 172.21.15.100:0/212848694 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e40052d0 con 0x7f37ec0d0e70 2024-09-17T04:31:36.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.926+0000 7f37eb200640 1 -- 172.21.15.100:0/212848694 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37ec0ca1f0 con 0x7f37ec0d0e70 2024-09-17T04:31:36.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.926+0000 7f37eb200640 1 -- 172.21.15.100:0/212848694 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37ec0ca7b0 con 0x7f37ec0d0e70 2024-09-17T04:31:36.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.927+0000 7f37eb200640 1 -- 172.21.15.100:0/212848694 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37ec06f010 con 0x7f37ec0d0e70 2024-09-17T04:31:36.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.929+0000 7f37e2a00640 1 -- 172.21.15.100:0/212848694 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f37e4005680 con 0x7f37ec0d0e70 2024-09-17T04:31:36.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.929+0000 7f37e2a00640 1 --2- 172.21.15.100:0/212848694 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f37c80791c0 0x7f37c807b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:36.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.930+0000 7f37e2a00640 1 -- 172.21.15.100:0/212848694 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7f37e409b830 con 0x7f37ec0d0e70 2024-09-17T04:31:36.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.930+0000 7f37e9e00640 1 --2- 172.21.15.100:0/212848694 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f37c80791c0 0x7f37c807b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:36.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.930+0000 7f37e9e00640 1 --2- 172.21.15.100:0/212848694 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f37c80791c0 0x7f37c807b6d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f37dc007fd0 tx=0x7f37dc001370 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:36.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:36.934+0000 7f37e2a00640 1 -- 172.21.15.100:0/212848694 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f37e4062c00 con 0x7f37ec0d0e70 2024-09-17T04:31:37.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.153+0000 7f37eb200640 1 -- 172.21.15.100:0/212848694 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37ec0cf730 con 0x7f37ec0d0e70 2024-09-17T04:31:37.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.154+0000 7f37e2a00640 1 -- 172.21.15.100:0/212848694 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5234 (secure 0 0 0) 0x7f37e4067630 con 0x7f37ec0d0e70 2024-09-17T04:31:37.153 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:37.153 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":83,"btime":"2024-09-17T04:31:27:587819+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6410,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1845046345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1845046345},{"type":"v1","addr":"172.21.15.184:6801","nonce":1845046345}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7135,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3970003794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3970003794},{"type":"v1","addr":"172.21.15.184:6835","nonce":3970003794}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/3663836388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3663836388},{"type":"v1","addr":"172.21.15.100:6834","nonce":3663836388}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:31:11.402955+0000","modified":"2024-09-17T04:31:14.414638+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7542":{"gid":7542,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/207356836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":207356836},{"type":"v1","addr":"172.21.15.100:6835","nonce":207356836}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7542,"qdb_cluster":[7542]},"id":9}]} 2024-09-17T04:31:37.153 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 83 2024-09-17T04:31:37.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.156+0000 7f37eb200640 1 -- 172.21.15.100:0/212848694 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f37c80791c0 msgr2=0x7f37c807b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:37.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.156+0000 7f37eb200640 1 --2- 172.21.15.100:0/212848694 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f37c80791c0 0x7f37c807b6d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f37dc007fd0 tx=0x7f37dc001370 comp rx=0 tx=0).stop 2024-09-17T04:31:37.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.156+0000 7f37eb200640 1 -- 172.21.15.100:0/212848694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37ec0d0e70 msgr2=0x7f37ec0cdf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:37.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.156+0000 7f37eb200640 1 --2- 172.21.15.100:0/212848694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37ec0d0e70 0x7f37ec0cdf00 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f37e400c730 tx=0x7f37e400cbf0 comp rx=0 tx=0).stop 2024-09-17T04:31:37.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.156+0000 7f37eb200640 1 -- 172.21.15.100:0/212848694 shutdown_connections 2024-09-17T04:31:37.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.156+0000 7f37eb200640 1 --2- 172.21.15.100:0/212848694 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f37c80791c0 0x7f37c807b6d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:37.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.156+0000 7f37eb200640 1 --2- 172.21.15.100:0/212848694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37ec0d0e70 0x7f37ec0cdf00 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:37.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.156+0000 7f37eb200640 1 --2- 172.21.15.100:0/212848694 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37ec0b98d0 0x7f37ec154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:37.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.156+0000 7f37eb200640 1 --2- 172.21.15.100:0/212848694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37ec0b58b0 0x7f37ec09e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:37.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.156+0000 7f37eb200640 1 -- 172.21.15.100:0/212848694 >> 172.21.15.100:0/212848694 conn(0x7f37ec09d820 msgr2=0x7f37ec161cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:37.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.156+0000 7f37eb200640 1 -- 172.21.15.100:0/212848694 shutdown_connections 2024-09-17T04:31:37.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.156+0000 7f37eb200640 1 -- 172.21.15.100:0/212848694 wait complete. 2024-09-17T04:31:37.182 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:31:37.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.405+0000 7f525d600640 1 -- 172.21.15.100:0/1669930220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f523c004800 msgr2=0x7f523c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:37.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.405+0000 7f525d600640 1 --2- 172.21.15.100:0/1669930220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f523c004800 0x7f523c004c10 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f5240009b50 tx=0x7f523c005830 comp rx=0 tx=0).stop 2024-09-17T04:31:37.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.405+0000 7f5257400640 1 -- 172.21.15.100:0/1669930220 reap_dead start 2024-09-17T04:31:37.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.405+0000 7f525d600640 1 -- 172.21.15.100:0/1669930220 shutdown_connections 2024-09-17T04:31:37.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.405+0000 7f525d600640 1 -- 172.21.15.100:0/1669930220 >> 172.21.15.100:0/1669930220 conn(0x7f525806d730 msgr2=0x7f52580b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:37.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.405+0000 7f525d600640 1 -- 172.21.15.100:0/1669930220 shutdown_connections 2024-09-17T04:31:37.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.405+0000 7f525d600640 1 -- 172.21.15.100:0/1669930220 wait complete. 2024-09-17T04:31:37.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.407+0000 7f525d600640 1 Processor -- start 2024-09-17T04:31:37.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.407+0000 7f525d600640 1 -- start start 2024-09-17T04:31:37.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.409+0000 7f525d600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f523c004800 0x7f5258153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:37.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.409+0000 7f525d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52581f56d0 0x7f52581f5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:37.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.409+0000 7f525d600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5258154ca0 0x7f52581550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:37.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.409+0000 7f525d600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f52580b3c90 con 0x7f52581f56d0 2024-09-17T04:31:37.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.409+0000 7f525d600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f525815f750 con 0x7f523c004800 2024-09-17T04:31:37.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.409+0000 7f525d600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f52580b3e10 con 0x7f5258154ca0 2024-09-17T04:31:37.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.409+0000 7f524e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52581f56d0 0x7f52581f5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:37.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.409+0000 7f524e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52581f56d0 0x7f52581f5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39978/0 (socket says 172.21.15.100:39978) 2024-09-17T04:31:37.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.409+0000 7f524e000640 1 -- 172.21.15.100:0/2990780697 learned_addr learned my addr 172.21.15.100:0/2990780697 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:37.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.409+0000 7f5256a00640 1 --2- 172.21.15.100:0/2990780697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f523c004800 0x7f5258153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:37.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.409+0000 7f5257400640 1 --2- 172.21.15.100:0/2990780697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5258154ca0 0x7f52581550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:37.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.410+0000 7f524e000640 1 -- 172.21.15.100:0/2990780697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5258154ca0 msgr2=0x7f52581550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:37.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.410+0000 7f524e000640 1 --2- 172.21.15.100:0/2990780697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5258154ca0 0x7f52581550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:37.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.410+0000 7f524e000640 1 -- 172.21.15.100:0/2990780697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f523c004800 msgr2=0x7f5258153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:37.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.410+0000 7f524e000640 1 --2- 172.21.15.100:0/2990780697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f523c004800 0x7f5258153300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:37.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.410+0000 7f524e000640 1 -- 172.21.15.100:0/2990780697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52581f6440 con 0x7f52581f56d0 2024-09-17T04:31:37.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.410+0000 7f5256a00640 1 --2- 172.21.15.100:0/2990780697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f523c004800 0x7f5258153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:31:37.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.410+0000 7f5257400640 1 --2- 172.21.15.100:0/2990780697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5258154ca0 0x7f52581550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:31:37.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.410+0000 7f524e000640 1 --2- 172.21.15.100:0/2990780697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52581f56d0 0x7f52581f5ae0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f5240036040 tx=0x7f5240033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:37.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.410+0000 7f524f400640 1 -- 172.21.15.100:0/2990780697 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5240043070 con 0x7f52581f56d0 2024-09-17T04:31:37.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.410+0000 7f524f400640 1 -- 172.21.15.100:0/2990780697 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5240004320 con 0x7f52581f56d0 2024-09-17T04:31:37.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.410+0000 7f524f400640 1 -- 172.21.15.100:0/2990780697 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5240038040 con 0x7f52581f56d0 2024-09-17T04:31:37.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.410+0000 7f525d600640 1 -- 172.21.15.100:0/2990780697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5258153840 con 0x7f52581f56d0 2024-09-17T04:31:37.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.410+0000 7f525d600640 1 -- 172.21.15.100:0/2990780697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5258073170 con 0x7f52581f56d0 2024-09-17T04:31:37.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.412+0000 7f525d600640 1 -- 172.21.15.100:0/2990780697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52580b3750 con 0x7f52581f56d0 2024-09-17T04:31:37.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.413+0000 7f524f400640 1 -- 172.21.15.100:0/2990780697 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f524002b8a0 con 0x7f52581f56d0 2024-09-17T04:31:37.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.414+0000 7f524f400640 1 --2- 172.21.15.100:0/2990780697 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f52340791c0 0x7f523407b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:37.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.414+0000 7f524f400640 1 -- 172.21.15.100:0/2990780697 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5737+0+0 (secure 0 0 0) 0x7f52400bbf60 con 0x7f52581f56d0 2024-09-17T04:31:37.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.414+0000 7f5256a00640 1 --2- 172.21.15.100:0/2990780697 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f52340791c0 0x7f523407b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:37.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.414+0000 7f5256a00640 1 --2- 172.21.15.100:0/2990780697 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f52340791c0 0x7f523407b6d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5248001d50 tx=0x7f5248001c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:37.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.418+0000 7f524f400640 1 -- 172.21.15.100:0/2990780697 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f5258153840 con 0x7f52581f56d0 2024-09-17T04:31:37.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.636+0000 7f525d600640 1 -- 172.21.15.100:0/2990780697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52580d0330 con 0x7f52581f56d0 2024-09-17T04:31:37.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.637+0000 7f524f400640 1 -- 172.21.15.100:0/2990780697 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5234 (secure 0 0 0) 0x7f52581f6440 con 0x7f52581f56d0 2024-09-17T04:31:37.637 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:37.637 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":83,"btime":"2024-09-17T04:31:27:587819+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6410,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1845046345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1845046345},{"type":"v1","addr":"172.21.15.184:6801","nonce":1845046345}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7135,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3970003794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3970003794},{"type":"v1","addr":"172.21.15.184:6835","nonce":3970003794}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/3663836388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3663836388},{"type":"v1","addr":"172.21.15.100:6834","nonce":3663836388}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:31:11.402955+0000","modified":"2024-09-17T04:31:14.414638+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7542":{"gid":7542,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/207356836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":207356836},{"type":"v1","addr":"172.21.15.100:6835","nonce":207356836}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7542,"qdb_cluster":[7542]},"id":9}]} 2024-09-17T04:31:37.637 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 83 2024-09-17T04:31:37.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.639+0000 7f525d600640 1 -- 172.21.15.100:0/2990780697 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f52340791c0 msgr2=0x7f523407b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:37.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.639+0000 7f525d600640 1 --2- 172.21.15.100:0/2990780697 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f52340791c0 0x7f523407b6d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5248001d50 tx=0x7f5248001c00 comp rx=0 tx=0).stop 2024-09-17T04:31:37.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.639+0000 7f525d600640 1 -- 172.21.15.100:0/2990780697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52581f56d0 msgr2=0x7f52581f5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:37.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.639+0000 7f525d600640 1 --2- 172.21.15.100:0/2990780697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52581f56d0 0x7f52581f5ae0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f5240036040 tx=0x7f5240033670 comp rx=0 tx=0).stop 2024-09-17T04:31:37.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.639+0000 7f525d600640 1 -- 172.21.15.100:0/2990780697 shutdown_connections 2024-09-17T04:31:37.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.639+0000 7f525d600640 1 --2- 172.21.15.100:0/2990780697 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f52340791c0 0x7f523407b6d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:37.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.639+0000 7f525d600640 1 --2- 172.21.15.100:0/2990780697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5258154ca0 0x7f52581550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:37.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.639+0000 7f525d600640 1 --2- 172.21.15.100:0/2990780697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52581f56d0 0x7f52581f5ae0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:37.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.639+0000 7f525d600640 1 --2- 172.21.15.100:0/2990780697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f523c004800 0x7f5258153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:37.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.639+0000 7f525d600640 1 -- 172.21.15.100:0/2990780697 >> 172.21.15.100:0/2990780697 conn(0x7f525806d730 msgr2=0x7f525815dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:37.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.640+0000 7f525d600640 1 -- 172.21.15.100:0/2990780697 shutdown_connections 2024-09-17T04:31:37.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.640+0000 7f525d600640 1 -- 172.21.15.100:0/2990780697 wait complete. 2024-09-17T04:31:37.665 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:31:37.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.900+0000 7f1745e00640 1 -- 172.21.15.100:0/2175279566 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f174015d760 msgr2=0x7f17401535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:37.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.900+0000 7f1745e00640 1 --2- 172.21.15.100:0/2175279566 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f174015d760 0x7f17401535c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1730009830 tx=0x7f173002a480 comp rx=0 tx=0).stop 2024-09-17T04:31:37.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.900+0000 7f1745e00640 1 -- 172.21.15.100:0/2175279566 shutdown_connections 2024-09-17T04:31:37.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.900+0000 7f1745e00640 1 --2- 172.21.15.100:0/2175279566 >> v2:172.21.15.184:3300/0 conn(0x7f17400b3550 0x7f17400b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:37.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.900+0000 7f1745e00640 1 --2- 172.21.15.100:0/2175279566 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f174015d760 0x7f17401535c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:37.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.901+0000 7f1745e00640 1 -- 172.21.15.100:0/2175279566 >> 172.21.15.100:0/2175279566 conn(0x7f174006d730 msgr2=0x7f17400b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:37.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.901+0000 7f1745e00640 1 -- 172.21.15.100:0/2175279566 shutdown_connections 2024-09-17T04:31:37.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.901+0000 7f1745e00640 1 -- 172.21.15.100:0/2175279566 wait complete. 2024-09-17T04:31:37.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.903+0000 7f1745e00640 1 Processor -- start 2024-09-17T04:31:37.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.903+0000 7f1745e00640 1 -- start start 2024-09-17T04:31:37.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.904+0000 7f1745e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17400ad0e0 0x7f1740148780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:37.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.904+0000 7f1745e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17400b3550 0x7f174015b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:37.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.904+0000 7f1745e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f174015d760 0x7f174015bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:37.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.904+0000 7f1745e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f17400b4da0 con 0x7f174015d760 2024-09-17T04:31:37.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.904+0000 7f1745e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f17400b4c20 con 0x7f17400ad0e0 2024-09-17T04:31:37.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.904+0000 7f1745e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f17400b3ea0 con 0x7f17400b3550 2024-09-17T04:31:37.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.905+0000 7f173f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f174015d760 0x7f174015bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:37.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.905+0000 7f173f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f174015d760 0x7f174015bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39980/0 (socket says 172.21.15.100:39980) 2024-09-17T04:31:37.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.905+0000 7f173f400640 1 -- 172.21.15.100:0/1617543828 learned_addr learned my addr 172.21.15.100:0/1617543828 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:37.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.905+0000 7f173ea00640 1 --2- 172.21.15.100:0/1617543828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17400ad0e0 0x7f1740148780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:37.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.905+0000 7f1737e00640 1 --2- 172.21.15.100:0/1617543828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17400b3550 0x7f174015b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:37.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.905+0000 7f173f400640 1 -- 172.21.15.100:0/1617543828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17400b3550 msgr2=0x7f174015b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:37.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.905+0000 7f173f400640 1 --2- 172.21.15.100:0/1617543828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17400b3550 0x7f174015b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:37.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.905+0000 7f173f400640 1 -- 172.21.15.100:0/1617543828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17400ad0e0 msgr2=0x7f1740148780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:37.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.905+0000 7f173f400640 1 --2- 172.21.15.100:0/1617543828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17400ad0e0 0x7f1740148780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:37.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.905+0000 7f173f400640 1 -- 172.21.15.100:0/1617543828 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1740070290 con 0x7f174015d760 2024-09-17T04:31:37.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.906+0000 7f173f400640 1 --2- 172.21.15.100:0/1617543828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f174015d760 0x7f174015bd30 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f173800c3a0 tx=0x7f173800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:37.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.906+0000 7f1737400640 1 -- 172.21.15.100:0/1617543828 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1738003970 con 0x7f174015d760 2024-09-17T04:31:37.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.906+0000 7f1737400640 1 -- 172.21.15.100:0/1617543828 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1738003c00 con 0x7f174015d760 2024-09-17T04:31:37.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.906+0000 7f1737400640 1 -- 172.21.15.100:0/1617543828 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1738004020 con 0x7f174015d760 2024-09-17T04:31:37.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.906+0000 7f1745e00640 1 -- 172.21.15.100:0/1617543828 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17400730e0 con 0x7f174015d760 2024-09-17T04:31:37.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.906+0000 7f1745e00640 1 -- 172.21.15.100:0/1617543828 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17400736a0 con 0x7f174015d760 2024-09-17T04:31:37.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.907+0000 7f1745e00640 1 -- 172.21.15.100:0/1617543828 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1740070000 con 0x7f174015d760 2024-09-17T04:31:37.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.908+0000 7f1737400640 1 -- 172.21.15.100:0/1617543828 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f17380041c0 con 0x7f174015d760 2024-09-17T04:31:37.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.909+0000 7f1737400640 1 --2- 172.21.15.100:0/1617543828 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f171c0791f0 0x7f171c07b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:37.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.909+0000 7f1737400640 1 -- 172.21.15.100:0/1617543828 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5793+0+0 (secure 0 0 0) 0x7f173809c1b0 con 0x7f174015d760 2024-09-17T04:31:37.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.909+0000 7f173ea00640 1 --2- 172.21.15.100:0/1617543828 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f171c0791f0 0x7f171c07b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:37.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.910+0000 7f173ea00640 1 --2- 172.21.15.100:0/1617543828 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f171c0791f0 0x7f171c07b700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1730034000 tx=0x7f1730004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:37.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:37.914+0000 7f1737400640 1 -- 172.21.15.100:0/1617543828 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f1738063580 con 0x7f174015d760 2024-09-17T04:31:38.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.100+0000 7f1745e00640 1 -- 172.21.15.100:0/1617543828 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f17400b4330 con 0x7f174015d760 2024-09-17T04:31:38.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.101+0000 7f1737400640 1 -- 172.21.15.100:0/1617543828 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) ==== 74+0+16943 (secure 0 0 0) 0x7f1738067fb0 con 0x7f174015d760 2024-09-17T04:31:38.102 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:38.102 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":120,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:31:37.660728+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:31:09.303673+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-17T04:31:10.403511+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":120,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:31:38.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.104+0000 7f1745e00640 1 -- 172.21.15.100:0/1617543828 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f171c0791f0 msgr2=0x7f171c07b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:38.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.104+0000 7f1745e00640 1 --2- 172.21.15.100:0/1617543828 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f171c0791f0 0x7f171c07b700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1730034000 tx=0x7f1730004730 comp rx=0 tx=0).stop 2024-09-17T04:31:38.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.104+0000 7f1745e00640 1 -- 172.21.15.100:0/1617543828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f174015d760 msgr2=0x7f174015bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:38.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.104+0000 7f1745e00640 1 --2- 172.21.15.100:0/1617543828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f174015d760 0x7f174015bd30 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f173800c3a0 tx=0x7f173800c860 comp rx=0 tx=0).stop 2024-09-17T04:31:38.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.104+0000 7f1745e00640 1 -- 172.21.15.100:0/1617543828 shutdown_connections 2024-09-17T04:31:38.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.104+0000 7f1745e00640 1 --2- 172.21.15.100:0/1617543828 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f171c0791f0 0x7f171c07b700 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:38.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.104+0000 7f1745e00640 1 --2- 172.21.15.100:0/1617543828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f174015d760 0x7f174015bd30 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:38.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.104+0000 7f1745e00640 1 --2- 172.21.15.100:0/1617543828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17400b3550 0x7f174015b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:38.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.104+0000 7f1745e00640 1 --2- 172.21.15.100:0/1617543828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17400ad0e0 0x7f1740148780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:38.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.104+0000 7f1745e00640 1 -- 172.21.15.100:0/1617543828 >> 172.21.15.100:0/1617543828 conn(0x7f174006d730 msgr2=0x7f17400cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:38.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.104+0000 7f1745e00640 1 -- 172.21.15.100:0/1617543828 shutdown_connections 2024-09-17T04:31:38.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.104+0000 7f1745e00640 1 -- 172.21.15.100:0/1617543828 wait complete. 2024-09-17T04:31:38.127 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T04:31:38.127 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:31:38.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.348+0000 7fbc93200640 1 -- 172.21.15.100:0/1097077832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc8c1518d0 msgr2=0x7fbc8c06f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:38.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.348+0000 7fbc93200640 1 --2- 172.21.15.100:0/1097077832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc8c1518d0 0x7fbc8c06f9f0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fbc8000a940 tx=0x7fbc8002b8f0 comp rx=0 tx=0).stop 2024-09-17T04:31:38.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.348+0000 7fbc93200640 1 -- 172.21.15.100:0/1097077832 shutdown_connections 2024-09-17T04:31:38.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.348+0000 7fbc93200640 1 --2- 172.21.15.100:0/1097077832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc8c1518d0 0x7fbc8c06f9f0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:38.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.348+0000 7fbc93200640 1 -- 172.21.15.100:0/1097077832 >> 172.21.15.100:0/1097077832 conn(0x7fbc8c0b1e40 msgr2=0x7fbc8c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:38.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.348+0000 7fbc93200640 1 -- 172.21.15.100:0/1097077832 shutdown_connections 2024-09-17T04:31:38.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.348+0000 7fbc93200640 1 -- 172.21.15.100:0/1097077832 wait complete. 2024-09-17T04:31:38.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.350+0000 7fbc93200640 1 Processor -- start 2024-09-17T04:31:38.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.351+0000 7fbc93200640 1 -- start start 2024-09-17T04:31:38.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.352+0000 7fbc93200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc8c0735c0 0x7fbc8c0afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:38.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.352+0000 7fbc93200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc8c0b3520 0x7fbc8c1f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:38.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.352+0000 7fbc93200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc8c1518d0 0x7fbc8c1f70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:38.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.352+0000 7fbc93200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c070130 con 0x7fbc8c1518d0 2024-09-17T04:31:38.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.352+0000 7fbc93200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c071160 con 0x7fbc8c0b3520 2024-09-17T04:31:38.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.352+0000 7fbc93200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbc8c0b4040 con 0x7fbc8c0735c0 2024-09-17T04:31:38.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.352+0000 7fbc92800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc8c1518d0 0x7fbc8c1f70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:38.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.352+0000 7fbc91e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc8c0735c0 0x7fbc8c0afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:38.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.352+0000 7fbc92800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc8c1518d0 0x7fbc8c1f70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39998/0 (socket says 172.21.15.100:39998) 2024-09-17T04:31:38.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.352+0000 7fbc92800640 1 -- 172.21.15.100:0/4221487312 learned_addr learned my addr 172.21.15.100:0/4221487312 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:38.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.352+0000 7fbc91400640 1 --2- 172.21.15.100:0/4221487312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc8c0b3520 0x7fbc8c1f6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:38.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.353+0000 7fbc92800640 1 -- 172.21.15.100:0/4221487312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc8c0735c0 msgr2=0x7fbc8c0afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:38.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.353+0000 7fbc92800640 1 --2- 172.21.15.100:0/4221487312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc8c0735c0 0x7fbc8c0afe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:38.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.353+0000 7fbc92800640 1 -- 172.21.15.100:0/4221487312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc8c0b3520 msgr2=0x7fbc8c1f6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:38.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.353+0000 7fbc92800640 1 --2- 172.21.15.100:0/4221487312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc8c0b3520 0x7fbc8c1f6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:38.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.353+0000 7fbc92800640 1 -- 172.21.15.100:0/4221487312 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8c0b0800 con 0x7fbc8c1518d0 2024-09-17T04:31:38.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.353+0000 7fbc92800640 1 --2- 172.21.15.100:0/4221487312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc8c1518d0 0x7fbc8c1f70d0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800caa0 tx=0x7fbc8800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:38.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.353+0000 7fbc86000640 1 -- 172.21.15.100:0/4221487312 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc88016020 con 0x7fbc8c1518d0 2024-09-17T04:31:38.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.354+0000 7fbc86000640 1 -- 172.21.15.100:0/4221487312 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc8800acf0 con 0x7fbc8c1518d0 2024-09-17T04:31:38.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.354+0000 7fbc86000640 1 -- 172.21.15.100:0/4221487312 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc88012680 con 0x7fbc8c1518d0 2024-09-17T04:31:38.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.354+0000 7fbc93200640 1 -- 172.21.15.100:0/4221487312 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc8c0b0ae0 con 0x7fbc8c1518d0 2024-09-17T04:31:38.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.354+0000 7fbc93200640 1 -- 172.21.15.100:0/4221487312 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc8c0b10a0 con 0x7fbc8c1518d0 2024-09-17T04:31:38.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.355+0000 7fbc93200640 1 -- 172.21.15.100:0/4221487312 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc8c0b0570 con 0x7fbc8c1518d0 2024-09-17T04:31:38.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.356+0000 7fbc86000640 1 -- 172.21.15.100:0/4221487312 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fbc88019020 con 0x7fbc8c1518d0 2024-09-17T04:31:38.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.357+0000 7fbc86000640 1 --2- 172.21.15.100:0/4221487312 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbc7007d600 0x7fbc7007fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:38.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.357+0000 7fbc86000640 1 -- 172.21.15.100:0/4221487312 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5793+0+0 (secure 0 0 0) 0x7fbc8809bd70 con 0x7fbc8c1518d0 2024-09-17T04:31:38.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.357+0000 7fbc91e00640 1 --2- 172.21.15.100:0/4221487312 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbc7007d600 0x7fbc7007fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:38.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.358+0000 7fbc91e00640 1 --2- 172.21.15.100:0/4221487312 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbc7007d600 0x7fbc7007fb10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbc80036000 tx=0x7fbc80004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:38.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.361+0000 7fbc86000640 1 -- 172.21.15.100:0/4221487312 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fbc88020020 con 0x7fbc8c1518d0 2024-09-17T04:31:38.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.577+0000 7fbc93200640 1 -- 172.21.15.100:0/4221487312 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc8c0b45d0 con 0x7fbc8c1518d0 2024-09-17T04:31:38.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.578+0000 7fbc86000640 1 -- 172.21.15.100:0/4221487312 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5234 (secure 0 0 0) 0x7fbc8806ae20 con 0x7fbc8c1518d0 2024-09-17T04:31:38.577 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:38.577 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":83,"btime":"2024-09-17T04:31:27:587819+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6410,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1845046345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1845046345},{"type":"v1","addr":"172.21.15.184:6801","nonce":1845046345}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7135,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3970003794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3970003794},{"type":"v1","addr":"172.21.15.184:6835","nonce":3970003794}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/3663836388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3663836388},{"type":"v1","addr":"172.21.15.100:6834","nonce":3663836388}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:31:11.402955+0000","modified":"2024-09-17T04:31:14.414638+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7542":{"gid":7542,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/207356836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":207356836},{"type":"v1","addr":"172.21.15.100:6835","nonce":207356836}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7542,"qdb_cluster":[7542]},"id":9}]} 2024-09-17T04:31:38.578 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 83 2024-09-17T04:31:38.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.581+0000 7fbc93200640 1 -- 172.21.15.100:0/4221487312 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbc7007d600 msgr2=0x7fbc7007fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:38.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.581+0000 7fbc93200640 1 --2- 172.21.15.100:0/4221487312 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbc7007d600 0x7fbc7007fb10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbc80036000 tx=0x7fbc80004730 comp rx=0 tx=0).stop 2024-09-17T04:31:38.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.581+0000 7fbc93200640 1 -- 172.21.15.100:0/4221487312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc8c1518d0 msgr2=0x7fbc8c1f70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:38.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.581+0000 7fbc93200640 1 --2- 172.21.15.100:0/4221487312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc8c1518d0 0x7fbc8c1f70d0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800caa0 tx=0x7fbc8800cf60 comp rx=0 tx=0).stop 2024-09-17T04:31:38.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.581+0000 7fbc93200640 1 -- 172.21.15.100:0/4221487312 shutdown_connections 2024-09-17T04:31:38.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.581+0000 7fbc93200640 1 --2- 172.21.15.100:0/4221487312 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbc7007d600 0x7fbc7007fb10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:38.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.581+0000 7fbc93200640 1 --2- 172.21.15.100:0/4221487312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc8c1518d0 0x7fbc8c1f70d0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:38.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.581+0000 7fbc93200640 1 --2- 172.21.15.100:0/4221487312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc8c0b3520 0x7fbc8c1f6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:38.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.581+0000 7fbc93200640 1 --2- 172.21.15.100:0/4221487312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc8c0735c0 0x7fbc8c0afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:38.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.581+0000 7fbc93200640 1 -- 172.21.15.100:0/4221487312 >> 172.21.15.100:0/4221487312 conn(0x7fbc8c0b1e40 msgr2=0x7fbc8c0739d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:38.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.581+0000 7fbc93200640 1 -- 172.21.15.100:0/4221487312 shutdown_connections 2024-09-17T04:31:38.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.581+0000 7fbc93200640 1 -- 172.21.15.100:0/4221487312 wait complete. 2024-09-17T04:31:38.602 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:31:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.824+0000 7fba24600640 1 -- 172.21.15.100:0/3614391415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba1c160310 msgr2=0x7fba1c160720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.824+0000 7fba24600640 1 --2- 172.21.15.100:0/3614391415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba1c160310 0x7fba1c160720 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fba1c075250 tx=0x7fba1402a4d0 comp rx=0 tx=0).stop 2024-09-17T04:31:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.824+0000 7fba24600640 1 -- 172.21.15.100:0/3614391415 shutdown_connections 2024-09-17T04:31:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.824+0000 7fba24600640 1 --2- 172.21.15.100:0/3614391415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba1c160310 0x7fba1c160720 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.824+0000 7fba24600640 1 --2- 172.21.15.100:0/3614391415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba1c0ad0e0 0x7fba1c0ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.824+0000 7fba24600640 1 --2- 172.21.15.100:0/3614391415 >> v2:172.21.15.184:3300/0 conn(0x7fba1c15d760 0x7fba1c1535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.824+0000 7fba24600640 1 -- 172.21.15.100:0/3614391415 >> 172.21.15.100:0/3614391415 conn(0x7fba1c06d730 msgr2=0x7fba1c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.824+0000 7fba24600640 1 -- 172.21.15.100:0/3614391415 shutdown_connections 2024-09-17T04:31:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.824+0000 7fba24600640 1 -- 172.21.15.100:0/3614391415 wait complete. 2024-09-17T04:31:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.826+0000 7fba24600640 1 Processor -- start 2024-09-17T04:31:38.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.827+0000 7fba24600640 1 -- start start 2024-09-17T04:31:38.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.828+0000 7fba24600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba1c0ad0e0 0x7fba1c1f6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:38.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.828+0000 7fba24600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba1c15d760 0x7fba1c13f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:38.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.828+0000 7fba24600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba1c160310 0x7fba1c13ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:38.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.828+0000 7fba24600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c0b4da0 con 0x7fba1c160310 2024-09-17T04:31:38.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.828+0000 7fba24600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c0b4c20 con 0x7fba1c0ad0e0 2024-09-17T04:31:38.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.828+0000 7fba24600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fba1c160c60 con 0x7fba1c15d760 2024-09-17T04:31:38.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.828+0000 7fba22600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba1c160310 0x7fba1c13ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:38.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.828+0000 7fba22600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba1c160310 0x7fba1c13ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40018/0 (socket says 172.21.15.100:40018) 2024-09-17T04:31:38.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.828+0000 7fba22600640 1 -- 172.21.15.100:0/1294445572 learned_addr learned my addr 172.21.15.100:0/1294445572 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:38.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.829+0000 7fba21200640 1 --2- 172.21.15.100:0/1294445572 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba1c15d760 0x7fba1c13f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:38.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.829+0000 7fba21c00640 1 --2- 172.21.15.100:0/1294445572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba1c0ad0e0 0x7fba1c1f6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:38.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.829+0000 7fba22600640 1 -- 172.21.15.100:0/1294445572 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba1c15d760 msgr2=0x7fba1c13f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:38.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.829+0000 7fba22600640 1 --2- 172.21.15.100:0/1294445572 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba1c15d760 0x7fba1c13f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:38.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.829+0000 7fba22600640 1 -- 172.21.15.100:0/1294445572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba1c0ad0e0 msgr2=0x7fba1c1f6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:38.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.829+0000 7fba22600640 1 --2- 172.21.15.100:0/1294445572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba1c0ad0e0 0x7fba1c1f6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:38.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.829+0000 7fba22600640 1 -- 172.21.15.100:0/1294445572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1c15a6f0 con 0x7fba1c160310 2024-09-17T04:31:38.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.829+0000 7fba22600640 1 --2- 172.21.15.100:0/1294445572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba1c160310 0x7fba1c13ff00 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fba14034040 tx=0x7fba14003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:38.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.829+0000 7fba19600640 1 -- 172.21.15.100:0/1294445572 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1403e030 con 0x7fba1c160310 2024-09-17T04:31:38.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.830+0000 7fba19600640 1 -- 172.21.15.100:0/1294445572 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba14003e40 con 0x7fba1c160310 2024-09-17T04:31:38.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.830+0000 7fba19600640 1 -- 172.21.15.100:0/1294445572 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba14035040 con 0x7fba1c160310 2024-09-17T04:31:38.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.830+0000 7fba24600640 1 -- 172.21.15.100:0/1294445572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba1c15a130 con 0x7fba1c160310 2024-09-17T04:31:38.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.830+0000 7fba24600640 1 -- 172.21.15.100:0/1294445572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba1c157510 con 0x7fba1c160310 2024-09-17T04:31:38.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.831+0000 7fba24600640 1 -- 172.21.15.100:0/1294445572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba1c1535c0 con 0x7fba1c160310 2024-09-17T04:31:38.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.832+0000 7fba19600640 1 -- 172.21.15.100:0/1294445572 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fba14041030 con 0x7fba1c160310 2024-09-17T04:31:38.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.833+0000 7fba19600640 1 --2- 172.21.15.100:0/1294445572 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb9fc0791c0 0x7fb9fc07b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:38.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.833+0000 7fba19600640 1 -- 172.21.15.100:0/1294445572 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5793+0+0 (secure 0 0 0) 0x7fba140bbfa0 con 0x7fba1c160310 2024-09-17T04:31:38.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.833+0000 7fba21c00640 1 --2- 172.21.15.100:0/1294445572 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb9fc0791c0 0x7fb9fc07b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:38.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.834+0000 7fba21c00640 1 --2- 172.21.15.100:0/1294445572 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb9fc0791c0 0x7fb9fc07b6d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fba10005fd0 tx=0x7fba100095e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:38.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:38.838+0000 7fba19600640 1 -- 172.21.15.100:0/1294445572 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fba14083530 con 0x7fba1c160310 2024-09-17T04:31:39.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.057+0000 7fba24600640 1 -- 172.21.15.100:0/1294445572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba1c1611c0 con 0x7fba1c160310 2024-09-17T04:31:39.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.057+0000 7fba19600640 1 -- 172.21.15.100:0/1294445572 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5234 (secure 0 0 0) 0x7fba14087f60 con 0x7fba1c160310 2024-09-17T04:31:39.057 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:39.057 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":83,"btime":"2024-09-17T04:31:27:587819+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6410,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1845046345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1845046345},{"type":"v1","addr":"172.21.15.184:6801","nonce":1845046345}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7135,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3970003794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3970003794},{"type":"v1","addr":"172.21.15.184:6835","nonce":3970003794}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/3663836388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3663836388},{"type":"v1","addr":"172.21.15.100:6834","nonce":3663836388}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:31:11.402955+0000","modified":"2024-09-17T04:31:14.414638+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7542":{"gid":7542,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/207356836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":207356836},{"type":"v1","addr":"172.21.15.100:6835","nonce":207356836}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7542,"qdb_cluster":[7542]},"id":9}]} 2024-09-17T04:31:39.057 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 83 2024-09-17T04:31:39.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.059+0000 7fba24600640 1 -- 172.21.15.100:0/1294445572 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb9fc0791c0 msgr2=0x7fb9fc07b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:39.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.059+0000 7fba24600640 1 --2- 172.21.15.100:0/1294445572 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb9fc0791c0 0x7fb9fc07b6d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fba10005fd0 tx=0x7fba100095e0 comp rx=0 tx=0).stop 2024-09-17T04:31:39.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.059+0000 7fba24600640 1 -- 172.21.15.100:0/1294445572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba1c160310 msgr2=0x7fba1c13ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:39.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.059+0000 7fba24600640 1 --2- 172.21.15.100:0/1294445572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba1c160310 0x7fba1c13ff00 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fba14034040 tx=0x7fba14003910 comp rx=0 tx=0).stop 2024-09-17T04:31:39.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.060+0000 7fba24600640 1 -- 172.21.15.100:0/1294445572 shutdown_connections 2024-09-17T04:31:39.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.060+0000 7fba24600640 1 --2- 172.21.15.100:0/1294445572 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb9fc0791c0 0x7fb9fc07b6d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:39.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.060+0000 7fba24600640 1 --2- 172.21.15.100:0/1294445572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba1c160310 0x7fba1c13ff00 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:39.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.060+0000 7fba24600640 1 --2- 172.21.15.100:0/1294445572 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba1c15d760 0x7fba1c13f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:39.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.060+0000 7fba24600640 1 --2- 172.21.15.100:0/1294445572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba1c0ad0e0 0x7fba1c1f6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:39.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.060+0000 7fba24600640 1 -- 172.21.15.100:0/1294445572 >> 172.21.15.100:0/1294445572 conn(0x7fba1c06d730 msgr2=0x7fba1c15dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:39.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.060+0000 7fba24600640 1 -- 172.21.15.100:0/1294445572 shutdown_connections 2024-09-17T04:31:39.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.060+0000 7fba24600640 1 -- 172.21.15.100:0/1294445572 wait complete. 2024-09-17T04:31:39.081 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:31:39.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.302+0000 7f4601000640 1 -- 172.21.15.100:0/2247891313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45fc0ad9c0 msgr2=0x7f45fc0af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:39.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.302+0000 7f4601000640 1 --2- 172.21.15.100:0/2247891313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45fc0ad9c0 0x7f45fc0af170 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f45fc06e790 tx=0x7f45ec02a4b0 comp rx=0 tx=0).stop 2024-09-17T04:31:39.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.302+0000 7f4601000640 1 -- 172.21.15.100:0/2247891313 shutdown_connections 2024-09-17T04:31:39.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.302+0000 7f4601000640 1 --2- 172.21.15.100:0/2247891313 >> v2:172.21.15.184:3300/0 conn(0x7f45fc0b2f60 0x7f45fc15a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:39.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.302+0000 7f4601000640 1 --2- 172.21.15.100:0/2247891313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45fc0ad9c0 0x7f45fc0af170 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:39.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.302+0000 7f4601000640 1 -- 172.21.15.100:0/2247891313 >> 172.21.15.100:0/2247891313 conn(0x7f45fc0b1e40 msgr2=0x7f45fc0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:39.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.302+0000 7f4601000640 1 -- 172.21.15.100:0/2247891313 shutdown_connections 2024-09-17T04:31:39.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.302+0000 7f4601000640 1 -- 172.21.15.100:0/2247891313 wait complete. 2024-09-17T04:31:39.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.304+0000 7f4601000640 1 Processor -- start 2024-09-17T04:31:39.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.304+0000 7f4601000640 1 -- start start 2024-09-17T04:31:39.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.306+0000 7f4601000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45fc0ad9c0 0x7f45fc0c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:39.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.306+0000 7f4601000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45fc0b2f60 0x7f45fc1f35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:39.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.306+0000 7f4601000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45fc15d670 0x7f45fc1f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:39.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.306+0000 7f4601000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc06f290 con 0x7f45fc0ad9c0 2024-09-17T04:31:39.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.306+0000 7f4601000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc070d50 con 0x7f45fc0b2f60 2024-09-17T04:31:39.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.306+0000 7f4601000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f45fc06f410 con 0x7f45fc15d670 2024-09-17T04:31:39.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.306+0000 7f45fb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45fc0ad9c0 0x7f45fc0c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:39.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.306+0000 7f45fb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45fc0ad9c0 0x7f45fc0c4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40030/0 (socket says 172.21.15.100:40030) 2024-09-17T04:31:39.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.306+0000 7f45fb400640 1 -- 172.21.15.100:0/3225845615 learned_addr learned my addr 172.21.15.100:0/3225845615 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:39.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.306+0000 7f45f2a00640 1 --2- 172.21.15.100:0/3225845615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45fc0b2f60 0x7f45fc1f35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:39.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.306+0000 7f45fbe00640 1 --2- 172.21.15.100:0/3225845615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45fc15d670 0x7f45fc1f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:39.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.307+0000 7f45fb400640 1 -- 172.21.15.100:0/3225845615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45fc15d670 msgr2=0x7f45fc1f3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:39.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.307+0000 7f45fb400640 1 --2- 172.21.15.100:0/3225845615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45fc15d670 0x7f45fc1f3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:39.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.307+0000 7f45fb400640 1 -- 172.21.15.100:0/3225845615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45fc0b2f60 msgr2=0x7f45fc1f35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:39.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.307+0000 7f45fb400640 1 --2- 172.21.15.100:0/3225845615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45fc0b2f60 0x7f45fc1f35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:39.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.307+0000 7f45fb400640 1 -- 172.21.15.100:0/3225845615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45fc1f7930 con 0x7f45fc0ad9c0 2024-09-17T04:31:39.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.307+0000 7f45fb400640 1 --2- 172.21.15.100:0/3225845615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45fc0ad9c0 0x7f45fc0c4f10 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009780 tx=0x7f45ec033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:39.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.307+0000 7f45f3e00640 1 -- 172.21.15.100:0/3225845615 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec031d60 con 0x7f45fc0ad9c0 2024-09-17T04:31:39.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.307+0000 7f45f3e00640 1 -- 172.21.15.100:0/3225845615 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45ec033e10 con 0x7f45fc0ad9c0 2024-09-17T04:31:39.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.307+0000 7f4601000640 1 -- 172.21.15.100:0/3225845615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45fc1581e0 con 0x7f45fc0ad9c0 2024-09-17T04:31:39.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.307+0000 7f45f3e00640 1 -- 172.21.15.100:0/3225845615 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec005260 con 0x7f45fc0ad9c0 2024-09-17T04:31:39.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.307+0000 7f4601000640 1 -- 172.21.15.100:0/3225845615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45fc1587a0 con 0x7f45fc0ad9c0 2024-09-17T04:31:39.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.308+0000 7f4601000640 1 -- 172.21.15.100:0/3225845615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45fc06f8a0 con 0x7f45fc0ad9c0 2024-09-17T04:31:39.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.310+0000 7f45f3e00640 1 -- 172.21.15.100:0/3225845615 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f45ec005400 con 0x7f45fc0ad9c0 2024-09-17T04:31:39.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.311+0000 7f45f3e00640 1 --2- 172.21.15.100:0/3225845615 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f45d80791f0 0x7f45d807b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:39.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.311+0000 7f45f3e00640 1 -- 172.21.15.100:0/3225845615 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5793+0+0 (secure 0 0 0) 0x7f45ec0bc8a0 con 0x7f45fc0ad9c0 2024-09-17T04:31:39.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.311+0000 7f45f2a00640 1 --2- 172.21.15.100:0/3225845615 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f45d80791f0 0x7f45d807b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:39.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.312+0000 7f45f2a00640 1 --2- 172.21.15.100:0/3225845615 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f45d80791f0 0x7f45d807b700 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f45fc0b43d0 tx=0x7f45e4005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:39.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.315+0000 7f45f3e00640 1 -- 172.21.15.100:0/3225845615 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f45ec03f030 con 0x7f45fc0ad9c0 2024-09-17T04:31:39.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.501+0000 7f4601000640 1 -- 172.21.15.100:0/3225845615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45fc0b4120 con 0x7f45fc0ad9c0 2024-09-17T04:31:39.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.502+0000 7f45f3e00640 1 -- 172.21.15.100:0/3225845615 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) ==== 74+0+16943 (secure 0 0 0) 0x7f45ec083d50 con 0x7f45fc0ad9c0 2024-09-17T04:31:39.502 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:39.502 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":120,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:31:37.660728+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:31:09.303673+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-17T04:31:10.403511+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":120,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:31:39.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.505+0000 7f4601000640 1 -- 172.21.15.100:0/3225845615 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f45d80791f0 msgr2=0x7f45d807b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:39.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.505+0000 7f4601000640 1 --2- 172.21.15.100:0/3225845615 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f45d80791f0 0x7f45d807b700 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f45fc0b43d0 tx=0x7f45e4005f90 comp rx=0 tx=0).stop 2024-09-17T04:31:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.505+0000 7f4601000640 1 -- 172.21.15.100:0/3225845615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45fc0ad9c0 msgr2=0x7f45fc0c4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.505+0000 7f4601000640 1 --2- 172.21.15.100:0/3225845615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45fc0ad9c0 0x7f45fc0c4f10 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009780 tx=0x7f45ec033c20 comp rx=0 tx=0).stop 2024-09-17T04:31:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.505+0000 7f4601000640 1 -- 172.21.15.100:0/3225845615 shutdown_connections 2024-09-17T04:31:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.505+0000 7f4601000640 1 --2- 172.21.15.100:0/3225845615 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f45d80791f0 0x7f45d807b700 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.505+0000 7f4601000640 1 --2- 172.21.15.100:0/3225845615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45fc15d670 0x7f45fc1f3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.505+0000 7f4601000640 1 --2- 172.21.15.100:0/3225845615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45fc0b2f60 0x7f45fc1f35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.505+0000 7f4601000640 1 --2- 172.21.15.100:0/3225845615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45fc0ad9c0 0x7f45fc0c4f10 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.505+0000 7f4601000640 1 -- 172.21.15.100:0/3225845615 >> 172.21.15.100:0/3225845615 conn(0x7f45fc0b1e40 msgr2=0x7f45fc15b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.505+0000 7f4601000640 1 -- 172.21.15.100:0/3225845615 shutdown_connections 2024-09-17T04:31:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.505+0000 7f4601000640 1 -- 172.21.15.100:0/3225845615 wait complete. 2024-09-17T04:31:39.530 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T04:31:39.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.753+0000 7f2ee3200640 1 -- 172.21.15.100:0/3109550548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ee406f8b0 msgr2=0x7f2ee40b88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:39.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.753+0000 7f2ee3200640 1 --2- 172.21.15.100:0/3109550548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ee406f8b0 0x7f2ee40b88b0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40b70c0 tx=0x7f2edc02aa30 comp rx=0 tx=0).stop 2024-09-17T04:31:39.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.753+0000 7f2ee3200640 1 -- 172.21.15.100:0/3109550548 shutdown_connections 2024-09-17T04:31:39.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.753+0000 7f2ee3200640 1 --2- 172.21.15.100:0/3109550548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ee406f8b0 0x7f2ee40b88b0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:39.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.753+0000 7f2ee3200640 1 --2- 172.21.15.100:0/3109550548 >> v2:172.21.15.184:3300/0 conn(0x7f2ee40b9570 0x7f2ee40d28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:39.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.753+0000 7f2ee3200640 1 -- 172.21.15.100:0/3109550548 >> 172.21.15.100:0/3109550548 conn(0x7f2ee409d880 msgr2=0x7f2ee408bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:39.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.753+0000 7f2ee3200640 1 -- 172.21.15.100:0/3109550548 shutdown_connections 2024-09-17T04:31:39.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.753+0000 7f2ee3200640 1 -- 172.21.15.100:0/3109550548 wait complete. 2024-09-17T04:31:39.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.755+0000 7f2ee3200640 1 Processor -- start 2024-09-17T04:31:39.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.755+0000 7f2ee3200640 1 -- start start 2024-09-17T04:31:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.756+0000 7f2ee3200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ee406f8b0 0x7f2ee4158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.756+0000 7f2ee3200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ee40b9570 0x7f2ee40ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.756+0000 7f2ee3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ee40d2de0 0x7f2ee415c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.757+0000 7f2ee3200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee4162350 con 0x7f2ee40d2de0 2024-09-17T04:31:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.757+0000 7f2ee3200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee41621d0 con 0x7f2ee40b9570 2024-09-17T04:31:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.757+0000 7f2ee3200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2ee40b8df0 con 0x7f2ee406f8b0 2024-09-17T04:31:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.757+0000 7f2ee2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ee40d2de0 0x7f2ee415c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.757+0000 7f2ee2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ee40d2de0 0x7f2ee415c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40062/0 (socket says 172.21.15.100:40062) 2024-09-17T04:31:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.757+0000 7f2ee2800640 1 -- 172.21.15.100:0/2940339560 learned_addr learned my addr 172.21.15.100:0/2940339560 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.757+0000 7f2ed9400640 1 --2- 172.21.15.100:0/2940339560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ee40b9570 0x7f2ee40ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.757+0000 7f2ee1e00640 1 --2- 172.21.15.100:0/2940339560 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ee406f8b0 0x7f2ee4158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.757+0000 7f2ee2800640 1 -- 172.21.15.100:0/2940339560 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ee406f8b0 msgr2=0x7f2ee4158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.757+0000 7f2ee2800640 1 --2- 172.21.15.100:0/2940339560 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ee406f8b0 0x7f2ee4158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.757+0000 7f2ee2800640 1 -- 172.21.15.100:0/2940339560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ee40b9570 msgr2=0x7f2ee40ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.757+0000 7f2ee2800640 1 --2- 172.21.15.100:0/2940339560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ee40b9570 0x7f2ee40ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.757+0000 7f2ee2800640 1 -- 172.21.15.100:0/2940339560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee4159f70 con 0x7f2ee40d2de0 2024-09-17T04:31:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.757+0000 7f2ee1e00640 1 --2- 172.21.15.100:0/2940339560 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ee406f8b0 0x7f2ee4158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:31:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.758+0000 7f2ee2800640 1 --2- 172.21.15.100:0/2940339560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ee40d2de0 0x7f2ee415c540 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f2edc02b020 tx=0x7f2edc003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.758+0000 7f2edaa00640 1 -- 172.21.15.100:0/2940339560 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc033050 con 0x7f2ee40d2de0 2024-09-17T04:31:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.758+0000 7f2edaa00640 1 -- 172.21.15.100:0/2940339560 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2edc004450 con 0x7f2ee40d2de0 2024-09-17T04:31:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.758+0000 7f2ee3200640 1 -- 172.21.15.100:0/2940339560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee4158a90 con 0x7f2ee40d2de0 2024-09-17T04:31:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.758+0000 7f2edaa00640 1 -- 172.21.15.100:0/2940339560 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc004840 con 0x7f2ee40d2de0 2024-09-17T04:31:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.758+0000 7f2ee3200640 1 -- 172.21.15.100:0/2940339560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee41594c0 con 0x7f2ee40d2de0 2024-09-17T04:31:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.759+0000 7f2ee3200640 1 -- 172.21.15.100:0/2940339560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ee40b92f0 con 0x7f2ee40d2de0 2024-09-17T04:31:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.760+0000 7f2edaa00640 1 -- 172.21.15.100:0/2940339560 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f2edc036070 con 0x7f2ee40d2de0 2024-09-17T04:31:39.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.761+0000 7f2edaa00640 1 --2- 172.21.15.100:0/2940339560 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2ec00791c0 0x7f2ec007b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:39.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.761+0000 7f2edaa00640 1 -- 172.21.15.100:0/2940339560 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5793+0+0 (secure 0 0 0) 0x7f2edc0bcec0 con 0x7f2ee40d2de0 2024-09-17T04:31:39.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.762+0000 7f2ee1e00640 1 --2- 172.21.15.100:0/2940339560 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2ec00791c0 0x7f2ec007b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:39.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.762+0000 7f2ee1e00640 1 --2- 172.21.15.100:0/2940339560 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2ec00791c0 0x7f2ec007b6d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40ba230 tx=0x7f2ed400b590 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:39.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.766+0000 7f2edaa00640 1 -- 172.21.15.100:0/2940339560 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f2ee40d31f0 con 0x7f2ee40d2de0 2024-09-17T04:31:39.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.985+0000 7f2ee3200640 1 -- 172.21.15.100:0/2940339560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2ee40ce5f0 con 0x7f2ee40d2de0 2024-09-17T04:31:39.985 INFO:tasks.ceph.mon.a.smithi100.stderr:2024-09-17T04:31:39.986+0000 7fd3d0a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T04:31:39.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.990+0000 7f2edaa00640 1 -- 172.21.15.100:0/2940339560 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) ==== 161+0+0 (secure 0 0 0) 0x7f2edc0848e0 con 0x7f2ee40d2de0 2024-09-17T04:31:39.989 INFO:teuthology.orchestra.run.smithi100.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T04:31:39.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.994+0000 7f2ee3200640 1 -- 172.21.15.100:0/2940339560 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2ec00791c0 msgr2=0x7f2ec007b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:39.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.994+0000 7f2ee3200640 1 --2- 172.21.15.100:0/2940339560 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2ec00791c0 0x7f2ec007b6d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40ba230 tx=0x7f2ed400b590 comp rx=0 tx=0).stop 2024-09-17T04:31:39.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.994+0000 7f2ee3200640 1 -- 172.21.15.100:0/2940339560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ee40d2de0 msgr2=0x7f2ee415c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:39.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.994+0000 7f2ee3200640 1 --2- 172.21.15.100:0/2940339560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ee40d2de0 0x7f2ee415c540 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f2edc02b020 tx=0x7f2edc003f10 comp rx=0 tx=0).stop 2024-09-17T04:31:39.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.994+0000 7f2ee3200640 1 -- 172.21.15.100:0/2940339560 shutdown_connections 2024-09-17T04:31:39.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.994+0000 7f2ee3200640 1 --2- 172.21.15.100:0/2940339560 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2ec00791c0 0x7f2ec007b6d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:39.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.994+0000 7f2ee3200640 1 --2- 172.21.15.100:0/2940339560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ee40d2de0 0x7f2ee415c540 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:39.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.994+0000 7f2ee3200640 1 --2- 172.21.15.100:0/2940339560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ee40b9570 0x7f2ee40ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:39.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.994+0000 7f2ee3200640 1 --2- 172.21.15.100:0/2940339560 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ee406f8b0 0x7f2ee4158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:39.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.994+0000 7f2ee3200640 1 -- 172.21.15.100:0/2940339560 >> 172.21.15.100:0/2940339560 conn(0x7f2ee409d880 msgr2=0x7f2ee408bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:39.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.994+0000 7f2ee3200640 1 -- 172.21.15.100:0/2940339560 shutdown_connections 2024-09-17T04:31:39.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:39.994+0000 7f2ee3200640 1 -- 172.21.15.100:0/2940339560 wait complete. 2024-09-17T04:31:40.024 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T04:31:40.134 INFO:tasks.ceph.mds.c.smithi100.stdout:starting mds.c at 2024-09-17T04:31:40.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.255+0000 7f5aab200640 1 -- 172.21.15.100:0/3431996569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a90004800 msgr2=0x7f5a90004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:40.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.255+0000 7f5aab200640 1 --2- 172.21.15.100:0/3431996569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a90004800 0x7f5a90004c10 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f5a98009b20 tx=0x7f5a90005830 comp rx=0 tx=0).stop 2024-09-17T04:31:40.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.255+0000 7f5aaa800640 1 -- 172.21.15.100:0/3431996569 reap_dead start 2024-09-17T04:31:40.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.255+0000 7f5aab200640 1 -- 172.21.15.100:0/3431996569 shutdown_connections 2024-09-17T04:31:40.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.255+0000 7f5aab200640 1 -- 172.21.15.100:0/3431996569 >> 172.21.15.100:0/3431996569 conn(0x7f5aac06d620 msgr2=0x7f5aac0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:40.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.256+0000 7f5aab200640 1 -- 172.21.15.100:0/3431996569 shutdown_connections 2024-09-17T04:31:40.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.256+0000 7f5aab200640 1 -- 172.21.15.100:0/3431996569 wait complete. 2024-09-17T04:31:40.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.258+0000 7f5aab200640 1 Processor -- start 2024-09-17T04:31:40.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.258+0000 7f5aab200640 1 -- start start 2024-09-17T04:31:40.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.259+0000 7f5aab200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a90004800 0x7f5aac1e56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:40.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.259+0000 7f5aab200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5aac099880 0x7f5aac152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:40.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.259+0000 7f5aab200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5aac153c30 0x7f5aac154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:40.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.259+0000 7f5aab200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac0ccc40 con 0x7f5aac153c30 2024-09-17T04:31:40.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.259+0000 7f5aab200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac0ccac0 con 0x7f5a90004800 2024-09-17T04:31:40.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.259+0000 7f5aab200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5aac0ccdc0 con 0x7f5aac099880 2024-09-17T04:31:40.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.260+0000 7f5aaa800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5aac153c30 0x7f5aac154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:40.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.260+0000 7f5aa9e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a90004800 0x7f5aac1e56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:40.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.260+0000 7f5aaa800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5aac153c30 0x7f5aac154040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40086/0 (socket says 172.21.15.100:40086) 2024-09-17T04:31:40.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.260+0000 7f5aaa800640 1 -- 172.21.15.100:0/1675644869 learned_addr learned my addr 172.21.15.100:0/1675644869 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:40.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.260+0000 7f5aa9400640 1 --2- 172.21.15.100:0/1675644869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5aac099880 0x7f5aac152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:40.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.260+0000 7f5aaa800640 1 -- 172.21.15.100:0/1675644869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5aac099880 msgr2=0x7f5aac152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:40.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.260+0000 7f5aaa800640 1 --2- 172.21.15.100:0/1675644869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5aac099880 0x7f5aac152e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:40.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.260+0000 7f5aaa800640 1 -- 172.21.15.100:0/1675644869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a90004800 msgr2=0x7f5aac1e56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:40.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.260+0000 7f5aaa800640 1 --2- 172.21.15.100:0/1675644869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a90004800 0x7f5aac1e56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:40.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.260+0000 7f5aaa800640 1 -- 172.21.15.100:0/1675644869 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aac154580 con 0x7f5aac153c30 2024-09-17T04:31:40.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.260+0000 7f5aaa800640 1 --2- 172.21.15.100:0/1675644869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5aac153c30 0x7f5aac154040 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0cfb40 tx=0x7f5aa4006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:40.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.261+0000 7f5aa2000640 1 -- 172.21.15.100:0/1675644869 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa400fa80 con 0x7f5aac153c30 2024-09-17T04:31:40.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.261+0000 7f5aa2000640 1 -- 172.21.15.100:0/1675644869 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5aa400fc20 con 0x7f5aac153c30 2024-09-17T04:31:40.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.261+0000 7f5aa2000640 1 -- 172.21.15.100:0/1675644869 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4011430 con 0x7f5aac153c30 2024-09-17T04:31:40.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.261+0000 7f5aab200640 1 -- 172.21.15.100:0/1675644869 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aac1e7ff0 con 0x7f5aac153c30 2024-09-17T04:31:40.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.261+0000 7f5aab200640 1 -- 172.21.15.100:0/1675644869 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aac1e85b0 con 0x7f5aac153c30 2024-09-17T04:31:40.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.262+0000 7f5aab200640 1 -- 172.21.15.100:0/1675644869 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aac0afad0 con 0x7f5aac153c30 2024-09-17T04:31:40.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.263+0000 7f5aa2000640 1 -- 172.21.15.100:0/1675644869 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f5aa40115d0 con 0x7f5aac153c30 2024-09-17T04:31:40.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.264+0000 7f5aa2000640 1 --2- 172.21.15.100:0/1675644869 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5a8c0791c0 0x7f5a8c07b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:40.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.265+0000 7f5aa2000640 1 -- 172.21.15.100:0/1675644869 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5851+0+0 (secure 0 0 0) 0x7f5aa409c1f0 con 0x7f5aac153c30 2024-09-17T04:31:40.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.265+0000 7f5aa9e00640 1 --2- 172.21.15.100:0/1675644869 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5a8c0791c0 0x7f5a8c07b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:40.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.265+0000 7f5aa9e00640 1 --2- 172.21.15.100:0/1675644869 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5a8c0791c0 0x7f5a8c07b6d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c009c50 tx=0x7f5a9c005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:40.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.269+0000 7f5aa2000640 1 -- 172.21.15.100:0/1675644869 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f5aac154580 con 0x7f5aac153c30 2024-09-17T04:31:40.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.485+0000 7f5aab200640 1 -- 172.21.15.100:0/1675644869 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5aac1e5c80 con 0x7f5aac153c30 2024-09-17T04:31:40.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.992+0000 7f5aa2000640 1 -- 172.21.15.100:0/1675644869 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v85) ==== 104+0+0 (secure 0 0 0) 0x7f5aa4063ba0 con 0x7f5aac153c30 2024-09-17T04:31:41.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.995+0000 7f5aab200640 1 -- 172.21.15.100:0/1675644869 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5a8c0791c0 msgr2=0x7f5a8c07b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:41.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.995+0000 7f5aab200640 1 --2- 172.21.15.100:0/1675644869 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5a8c0791c0 0x7f5a8c07b6d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c009c50 tx=0x7f5a9c005f90 comp rx=0 tx=0).stop 2024-09-17T04:31:41.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.995+0000 7f5aab200640 1 -- 172.21.15.100:0/1675644869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5aac153c30 msgr2=0x7f5aac154040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:41.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.995+0000 7f5aab200640 1 --2- 172.21.15.100:0/1675644869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5aac153c30 0x7f5aac154040 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0cfb40 tx=0x7f5aa4006660 comp rx=0 tx=0).stop 2024-09-17T04:31:41.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.995+0000 7f5aab200640 1 -- 172.21.15.100:0/1675644869 shutdown_connections 2024-09-17T04:31:41.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.995+0000 7f5aab200640 1 --2- 172.21.15.100:0/1675644869 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5a8c0791c0 0x7f5a8c07b6d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:41.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.995+0000 7f5aab200640 1 --2- 172.21.15.100:0/1675644869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5aac153c30 0x7f5aac154040 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:41.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.995+0000 7f5aab200640 1 --2- 172.21.15.100:0/1675644869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5aac099880 0x7f5aac152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:41.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.995+0000 7f5aab200640 1 --2- 172.21.15.100:0/1675644869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a90004800 0x7f5aac1e56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:41.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.995+0000 7f5aab200640 1 -- 172.21.15.100:0/1675644869 >> 172.21.15.100:0/1675644869 conn(0x7f5aac06d620 msgr2=0x7f5aac06f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:41.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.995+0000 7f5aab200640 1 -- 172.21.15.100:0/1675644869 shutdown_connections 2024-09-17T04:31:41.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:40.995+0000 7f5aab200640 1 -- 172.21.15.100:0/1675644869 wait complete. 2024-09-17T04:31:41.583 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T04:31:41.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.815+0000 7fc888000640 1 -- 172.21.15.100:0/3459281554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8800af100 msgr2=0x7fc8800af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:41.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.815+0000 7fc888000640 1 --2- 172.21.15.100:0/3459281554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8800af100 0x7fc8800af510 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc878005fd0 tx=0x7fc87802a8b0 comp rx=0 tx=0).stop 2024-09-17T04:31:41.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.815+0000 7fc888000640 1 -- 172.21.15.100:0/3459281554 shutdown_connections 2024-09-17T04:31:41.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.815+0000 7fc888000640 1 --2- 172.21.15.100:0/3459281554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8800af100 0x7fc8800af510 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:41.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.815+0000 7fc888000640 1 --2- 172.21.15.100:0/3459281554 >> v2:172.21.15.184:3300/0 conn(0x7fc88006d4b0 0x7fc8800b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:41.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.815+0000 7fc888000640 1 -- 172.21.15.100:0/3459281554 >> 172.21.15.100:0/3459281554 conn(0x7fc8800b2540 msgr2=0x7fc8800d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:41.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.815+0000 7fc888000640 1 -- 172.21.15.100:0/3459281554 shutdown_connections 2024-09-17T04:31:41.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.815+0000 7fc888000640 1 -- 172.21.15.100:0/3459281554 wait complete. 2024-09-17T04:31:41.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.817+0000 7fc888000640 1 Processor -- start 2024-09-17T04:31:41.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.817+0000 7fc888000640 1 -- start start 2024-09-17T04:31:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.819+0000 7fc888000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8800af100 0x7fc8800cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.819+0000 7fc888000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8800cc830 0x7fc8800ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.819+0000 7fc888000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc880069640 0x7fc8801f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.819+0000 7fc888000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc8800b36a0 con 0x7fc880069640 2024-09-17T04:31:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.819+0000 7fc888000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc8800b3520 con 0x7fc8800af100 2024-09-17T04:31:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.819+0000 7fc888000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc8800b3820 con 0x7fc8800cc830 2024-09-17T04:31:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.819+0000 7fc886000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc880069640 0x7fc8801f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.819+0000 7fc886000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc880069640 0x7fc8801f5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40092/0 (socket says 172.21.15.100:40092) 2024-09-17T04:31:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.819+0000 7fc886000640 1 -- 172.21.15.100:0/1850801677 learned_addr learned my addr 172.21.15.100:0/1850801677 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.819+0000 7fc884c00640 1 --2- 172.21.15.100:0/1850801677 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8800cc830 0x7fc8800ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:41.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.819+0000 7fc885600640 1 --2- 172.21.15.100:0/1850801677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8800af100 0x7fc8800cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:41.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.820+0000 7fc886000640 1 -- 172.21.15.100:0/1850801677 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8800cc830 msgr2=0x7fc8800ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:41.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.820+0000 7fc886000640 1 --2- 172.21.15.100:0/1850801677 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8800cc830 0x7fc8800ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:41.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.820+0000 7fc886000640 1 -- 172.21.15.100:0/1850801677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8800af100 msgr2=0x7fc8800cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:41.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.820+0000 7fc886000640 1 --2- 172.21.15.100:0/1850801677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8800af100 0x7fc8800cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:41.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.820+0000 7fc886000640 1 -- 172.21.15.100:0/1850801677 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc880156720 con 0x7fc880069640 2024-09-17T04:31:41.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.820+0000 7fc886000640 1 --2- 172.21.15.100:0/1850801677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc880069640 0x7fc8801f5220 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fc87802aea0 tx=0x7fc8780316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:41.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.820+0000 7fc87d600640 1 -- 172.21.15.100:0/1850801677 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc878031da0 con 0x7fc880069640 2024-09-17T04:31:41.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.820+0000 7fc888000640 1 -- 172.21.15.100:0/1850801677 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8800b4180 con 0x7fc880069640 2024-09-17T04:31:41.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.820+0000 7fc87d600640 1 -- 172.21.15.100:0/1850801677 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc878003a60 con 0x7fc880069640 2024-09-17T04:31:41.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.820+0000 7fc87d600640 1 -- 172.21.15.100:0/1850801677 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc878033070 con 0x7fc880069640 2024-09-17T04:31:41.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.820+0000 7fc888000640 1 -- 172.21.15.100:0/1850801677 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8801f6660 con 0x7fc880069640 2024-09-17T04:31:41.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.821+0000 7fc888000640 1 -- 172.21.15.100:0/1850801677 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8800b3d40 con 0x7fc880069640 2024-09-17T04:31:41.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.823+0000 7fc87d600640 1 -- 172.21.15.100:0/1850801677 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fc878033210 con 0x7fc880069640 2024-09-17T04:31:41.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.823+0000 7fc87d600640 1 --2- 172.21.15.100:0/1850801677 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc8600791c0 0x7fc86007b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:41.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.824+0000 7fc87d600640 1 -- 172.21.15.100:0/1850801677 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5851+0+0 (secure 0 0 0) 0x7fc8780bc970 con 0x7fc880069640 2024-09-17T04:31:41.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.824+0000 7fc885600640 1 --2- 172.21.15.100:0/1850801677 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc8600791c0 0x7fc86007b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:41.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.825+0000 7fc885600640 1 --2- 172.21.15.100:0/1850801677 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc8600791c0 0x7fc86007b6d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc874007fd0 tx=0x7fc874001370 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:41.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:41.828+0000 7fc87d600640 1 -- 172.21.15.100:0/1850801677 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc87804a030 con 0x7fc880069640 2024-09-17T04:31:42.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.014+0000 7fc888000640 1 -- 172.21.15.100:0/1850801677 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc8800d1030 con 0x7fc880069640 2024-09-17T04:31:42.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.694+0000 7fc87d600640 1 -- 172.21.15.100:0/1850801677 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v122) ==== 181+0+0 (secure 0 0 0) 0x7fc87803f030 con 0x7fc880069640 2024-09-17T04:31:42.694 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' removed 2024-09-17T04:31:42.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.698+0000 7fc888000640 1 -- 172.21.15.100:0/1850801677 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc8600791c0 msgr2=0x7fc86007b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:42.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.698+0000 7fc888000640 1 --2- 172.21.15.100:0/1850801677 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc8600791c0 0x7fc86007b6d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc874007fd0 tx=0x7fc874001370 comp rx=0 tx=0).stop 2024-09-17T04:31:42.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.698+0000 7fc888000640 1 -- 172.21.15.100:0/1850801677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc880069640 msgr2=0x7fc8801f5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:42.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.698+0000 7fc888000640 1 --2- 172.21.15.100:0/1850801677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc880069640 0x7fc8801f5220 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fc87802aea0 tx=0x7fc8780316f0 comp rx=0 tx=0).stop 2024-09-17T04:31:42.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.698+0000 7fc888000640 1 -- 172.21.15.100:0/1850801677 shutdown_connections 2024-09-17T04:31:42.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.698+0000 7fc888000640 1 --2- 172.21.15.100:0/1850801677 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc8600791c0 0x7fc86007b6d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:42.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.698+0000 7fc888000640 1 --2- 172.21.15.100:0/1850801677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc880069640 0x7fc8801f5220 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:42.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.698+0000 7fc888000640 1 --2- 172.21.15.100:0/1850801677 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8800cc830 0x7fc8800ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:42.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.698+0000 7fc888000640 1 --2- 172.21.15.100:0/1850801677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8800af100 0x7fc8800cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:42.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.698+0000 7fc888000640 1 -- 172.21.15.100:0/1850801677 >> 172.21.15.100:0/1850801677 conn(0x7fc8800b2540 msgr2=0x7fc8800998b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:42.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.699+0000 7fc888000640 1 -- 172.21.15.100:0/1850801677 shutdown_connections 2024-09-17T04:31:42.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.699+0000 7fc888000640 1 -- 172.21.15.100:0/1850801677 wait complete. 2024-09-17T04:31:42.726 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T04:31:42.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.959+0000 7fb24ec00640 1 -- 172.21.15.100:0/2921481252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb230003a30 msgr2=0x7fb230003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:42.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.959+0000 7fb24ec00640 1 --2- 172.21.15.100:0/2921481252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb230003a30 0x7fb230003ea0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0016c0 tx=0x7fb230005830 comp rx=0 tx=0).stop 2024-09-17T04:31:42.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.959+0000 7fb24e200640 1 -- 172.21.15.100:0/2921481252 reap_dead start 2024-09-17T04:31:42.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.959+0000 7fb24ec00640 1 -- 172.21.15.100:0/2921481252 shutdown_connections 2024-09-17T04:31:42.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.959+0000 7fb24ec00640 1 -- 172.21.15.100:0/2921481252 >> 172.21.15.100:0/2921481252 conn(0x7fb2480b1dd0 msgr2=0x7fb2480b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:42.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.959+0000 7fb24ec00640 1 -- 172.21.15.100:0/2921481252 shutdown_connections 2024-09-17T04:31:42.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.959+0000 7fb24ec00640 1 -- 172.21.15.100:0/2921481252 wait complete. 2024-09-17T04:31:42.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.961+0000 7fb24ec00640 1 Processor -- start 2024-09-17T04:31:42.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.962+0000 7fb24ec00640 1 -- start start 2024-09-17T04:31:42.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.963+0000 7fb24ec00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb230003a30 0x7fb24814ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:42.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.963+0000 7fb24ec00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb248072e20 0x7fb24815ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:42.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.963+0000 7fb24ec00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb24815f4b0 0x7fb2481f88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:42.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.963+0000 7fb24ec00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb2480b4940 con 0x7fb230003a30 2024-09-17T04:31:42.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.963+0000 7fb24ec00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb2480b47c0 con 0x7fb24815f4b0 2024-09-17T04:31:42.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.963+0000 7fb24ec00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb2480b4ac0 con 0x7fb248072e20 2024-09-17T04:31:42.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.963+0000 7fb24d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb230003a30 0x7fb24814ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:42.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.963+0000 7fb24d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb230003a30 0x7fb24814ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40114/0 (socket says 172.21.15.100:40114) 2024-09-17T04:31:42.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.963+0000 7fb24d800640 1 -- 172.21.15.100:0/107620528 learned_addr learned my addr 172.21.15.100:0/107620528 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:42.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.963+0000 7fb24ce00640 1 --2- 172.21.15.100:0/107620528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb248072e20 0x7fb24815ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:42.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.963+0000 7fb24e200640 1 --2- 172.21.15.100:0/107620528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb24815f4b0 0x7fb2481f88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:42.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.964+0000 7fb24d800640 1 -- 172.21.15.100:0/107620528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb248072e20 msgr2=0x7fb24815ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:42.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.964+0000 7fb24d800640 1 --2- 172.21.15.100:0/107620528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb248072e20 0x7fb24815ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:42.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.964+0000 7fb24d800640 1 -- 172.21.15.100:0/107620528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb24815f4b0 msgr2=0x7fb2481f88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:42.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.964+0000 7fb24d800640 1 --2- 172.21.15.100:0/107620528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb24815f4b0 0x7fb2481f88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:42.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.964+0000 7fb24d800640 1 -- 172.21.15.100:0/107620528 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb24815f8c0 con 0x7fb230003a30 2024-09-17T04:31:42.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.964+0000 7fb24d800640 1 --2- 172.21.15.100:0/107620528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb230003a30 0x7fb24814ffe0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fb23c02a930 tx=0x7fb23c006310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:42.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.964+0000 7fb245600640 1 -- 172.21.15.100:0/107620528 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c041070 con 0x7fb230003a30 2024-09-17T04:31:42.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.964+0000 7fb245600640 1 -- 172.21.15.100:0/107620528 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb23c003af0 con 0x7fb230003a30 2024-09-17T04:31:42.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.964+0000 7fb24ec00640 1 -- 172.21.15.100:0/107620528 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb248150e90 con 0x7fb230003a30 2024-09-17T04:31:42.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.964+0000 7fb245600640 1 -- 172.21.15.100:0/107620528 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c038070 con 0x7fb230003a30 2024-09-17T04:31:42.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.965+0000 7fb24ec00640 1 -- 172.21.15.100:0/107620528 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb24809dfa0 con 0x7fb230003a30 2024-09-17T04:31:42.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.966+0000 7fb24ec00640 1 -- 172.21.15.100:0/107620528 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb24809dab0 con 0x7fb230003a30 2024-09-17T04:31:42.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.967+0000 7fb245600640 1 -- 172.21.15.100:0/107620528 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fb23c044070 con 0x7fb230003a30 2024-09-17T04:31:42.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.968+0000 7fb245600640 1 --2- 172.21.15.100:0/107620528 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb2280791f0 0x7fb22807b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:42.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.968+0000 7fb245600640 1 -- 172.21.15.100:0/107620528 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5413+0+0 (secure 0 0 0) 0x7fb23c0bb010 con 0x7fb230003a30 2024-09-17T04:31:42.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.968+0000 7fb24ce00640 1 --2- 172.21.15.100:0/107620528 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb2280791f0 0x7fb22807b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:42.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.969+0000 7fb24ce00640 1 --2- 172.21.15.100:0/107620528 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb2280791f0 0x7fb22807b700 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb234006fd0 tx=0x7fb234006950 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:42.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:42.972+0000 7fb245600640 1 -- 172.21.15.100:0/107620528 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb24815f8c0 con 0x7fb230003a30 2024-09-17T04:31:43.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.156+0000 7fb24ec00640 1 -- 172.21.15.100:0/107620528 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb24815fb40 con 0x7fb230003a30 2024-09-17T04:31:43.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.700+0000 7fb245600640 1 -- 172.21.15.100:0/107620528 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v123) ==== 169+0+0 (secure 0 0 0) 0x7fb23c006500 con 0x7fb230003a30 2024-09-17T04:31:43.701 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' removed 2024-09-17T04:31:43.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.703+0000 7fb24ec00640 1 -- 172.21.15.100:0/107620528 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb2280791f0 msgr2=0x7fb22807b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:43.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.703+0000 7fb24ec00640 1 --2- 172.21.15.100:0/107620528 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb2280791f0 0x7fb22807b700 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb234006fd0 tx=0x7fb234006950 comp rx=0 tx=0).stop 2024-09-17T04:31:43.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.703+0000 7fb24ec00640 1 -- 172.21.15.100:0/107620528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb230003a30 msgr2=0x7fb24814ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:43.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.703+0000 7fb24ec00640 1 --2- 172.21.15.100:0/107620528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb230003a30 0x7fb24814ffe0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fb23c02a930 tx=0x7fb23c006310 comp rx=0 tx=0).stop 2024-09-17T04:31:43.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.703+0000 7fb24ec00640 1 -- 172.21.15.100:0/107620528 shutdown_connections 2024-09-17T04:31:43.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.703+0000 7fb24ec00640 1 --2- 172.21.15.100:0/107620528 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb2280791f0 0x7fb22807b700 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:43.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.703+0000 7fb24ec00640 1 --2- 172.21.15.100:0/107620528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb24815f4b0 0x7fb2481f88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:43.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.703+0000 7fb24ec00640 1 --2- 172.21.15.100:0/107620528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb248072e20 0x7fb24815ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:43.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.703+0000 7fb24ec00640 1 --2- 172.21.15.100:0/107620528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb230003a30 0x7fb24814ffe0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:43.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.703+0000 7fb24ec00640 1 -- 172.21.15.100:0/107620528 >> 172.21.15.100:0/107620528 conn(0x7fb2480b1dd0 msgr2=0x7fb248153da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:43.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.703+0000 7fb24ec00640 1 -- 172.21.15.100:0/107620528 shutdown_connections 2024-09-17T04:31:43.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.703+0000 7fb24ec00640 1 -- 172.21.15.100:0/107620528 wait complete. 2024-09-17T04:31:43.740 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T04:31:43.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.963+0000 7f0330e00640 1 -- 172.21.15.100:0/3095968414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f032c155370 msgr2=0x7f032c155780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:43.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.963+0000 7f0330e00640 1 --2- 172.21.15.100:0/3095968414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f032c155370 0x7f032c155780 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f032c0d1190 tx=0x7f032402a9e0 comp rx=0 tx=0).stop 2024-09-17T04:31:43.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.963+0000 7f0330e00640 1 -- 172.21.15.100:0/3095968414 shutdown_connections 2024-09-17T04:31:43.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.963+0000 7f0330e00640 1 --2- 172.21.15.100:0/3095968414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f032c155370 0x7f032c155780 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:43.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.963+0000 7f0330e00640 1 --2- 172.21.15.100:0/3095968414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f032c1623a0 0x7f032c1627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:43.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.963+0000 7f0330e00640 1 --2- 172.21.15.100:0/3095968414 >> v2:172.21.15.184:3300/0 conn(0x7f032c06f280 0x7f032c06f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:43.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.963+0000 7f0330e00640 1 -- 172.21.15.100:0/3095968414 >> 172.21.15.100:0/3095968414 conn(0x7f032c0b1e40 msgr2=0x7f032c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:43.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.963+0000 7f0330e00640 1 -- 172.21.15.100:0/3095968414 shutdown_connections 2024-09-17T04:31:43.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.963+0000 7f0330e00640 1 -- 172.21.15.100:0/3095968414 wait complete. 2024-09-17T04:31:43.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.965+0000 7f0330e00640 1 Processor -- start 2024-09-17T04:31:43.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.965+0000 7f0330e00640 1 -- start start 2024-09-17T04:31:43.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.967+0000 7f0330e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f032c06f280 0x7f032c09c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:43.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.967+0000 7f0330e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f032c155370 0x7f032c153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:43.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.967+0000 7f0330e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f032c1623a0 0x7f032c0b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:43.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.967+0000 7f0330e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f032c15ed60 con 0x7f032c1623a0 2024-09-17T04:31:43.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.967+0000 7f0330e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f032c073e30 con 0x7f032c155370 2024-09-17T04:31:43.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.967+0000 7f0330e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f032c15eee0 con 0x7f032c06f280 2024-09-17T04:31:43.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.967+0000 7f032be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f032c1623a0 0x7f032c0b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:43.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.967+0000 7f032b400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f032c06f280 0x7f032c09c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:43.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.967+0000 7f032be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f032c1623a0 0x7f032c0b0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40136/0 (socket says 172.21.15.100:40136) 2024-09-17T04:31:43.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.967+0000 7f032be00640 1 -- 172.21.15.100:0/1393519662 learned_addr learned my addr 172.21.15.100:0/1393519662 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:43.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.967+0000 7f0322a00640 1 --2- 172.21.15.100:0/1393519662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f032c155370 0x7f032c153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:43.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.967+0000 7f032be00640 1 -- 172.21.15.100:0/1393519662 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f032c06f280 msgr2=0x7f032c09c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:43.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.967+0000 7f032be00640 1 --2- 172.21.15.100:0/1393519662 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f032c06f280 0x7f032c09c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:43.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.967+0000 7f032be00640 1 -- 172.21.15.100:0/1393519662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f032c155370 msgr2=0x7f032c153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:43.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.967+0000 7f032be00640 1 --2- 172.21.15.100:0/1393519662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f032c155370 0x7f032c153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:43.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.967+0000 7f032be00640 1 -- 172.21.15.100:0/1393519662 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032c09d460 con 0x7f032c1623a0 2024-09-17T04:31:43.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.968+0000 7f032be00640 1 --2- 172.21.15.100:0/1393519662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f032c1623a0 0x7f032c0b0820 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f0324009d90 tx=0x7f0324003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:43.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.968+0000 7f0323e00640 1 -- 172.21.15.100:0/1393519662 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324033050 con 0x7f032c1623a0 2024-09-17T04:31:43.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.968+0000 7f0323e00640 1 -- 172.21.15.100:0/1393519662 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0324004450 con 0x7f032c1623a0 2024-09-17T04:31:43.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.968+0000 7f0323e00640 1 -- 172.21.15.100:0/1393519662 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324004860 con 0x7f032c1623a0 2024-09-17T04:31:43.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.968+0000 7f0330e00640 1 -- 172.21.15.100:0/1393519662 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f032c1f9c20 con 0x7f032c1623a0 2024-09-17T04:31:43.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.968+0000 7f0330e00640 1 -- 172.21.15.100:0/1393519662 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f032c1fa1e0 con 0x7f032c1623a0 2024-09-17T04:31:43.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.969+0000 7f0330e00640 1 -- 172.21.15.100:0/1393519662 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f032c09d1d0 con 0x7f032c1623a0 2024-09-17T04:31:43.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.971+0000 7f0323e00640 1 -- 172.21.15.100:0/1393519662 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f0324036070 con 0x7f032c1623a0 2024-09-17T04:31:43.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.972+0000 7f0323e00640 1 --2- 172.21.15.100:0/1393519662 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f03080791c0 0x7f030807b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:43.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.972+0000 7f0323e00640 1 -- 172.21.15.100:0/1393519662 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4999+0+0 (secure 0 0 0) 0x7f03240bc750 con 0x7f032c1623a0 2024-09-17T04:31:43.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.972+0000 7f032b400640 1 --2- 172.21.15.100:0/1393519662 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f03080791c0 0x7f030807b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:43.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.973+0000 7f032b400640 1 --2- 172.21.15.100:0/1393519662 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f03080791c0 0x7f030807b6d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f031c00bb50 tx=0x7f031c00b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:43.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:43.976+0000 7f0323e00640 1 -- 172.21.15.100:0/1393519662 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f032c162cf0 con 0x7f032c1623a0 2024-09-17T04:31:44.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.159+0000 7f0330e00640 1 -- 172.21.15.100:0/1393519662 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f032c069840 con 0x7f032c1623a0 2024-09-17T04:31:44.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.164+0000 7f0323e00640 1 -- 172.21.15.100:0/1393519662 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v45) ==== 71+0+0 (secure 0 0 0) 0x7f032c069840 con 0x7f032c1623a0 2024-09-17T04:31:44.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.166+0000 7f0330e00640 1 -- 172.21.15.100:0/1393519662 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f03080791c0 msgr2=0x7f030807b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:44.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.166+0000 7f0330e00640 1 --2- 172.21.15.100:0/1393519662 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f03080791c0 0x7f030807b6d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f031c00bb50 tx=0x7f031c00b2e0 comp rx=0 tx=0).stop 2024-09-17T04:31:44.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.166+0000 7f0330e00640 1 -- 172.21.15.100:0/1393519662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f032c1623a0 msgr2=0x7f032c0b0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:44.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.166+0000 7f0330e00640 1 --2- 172.21.15.100:0/1393519662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f032c1623a0 0x7f032c0b0820 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f0324009d90 tx=0x7f0324003f10 comp rx=0 tx=0).stop 2024-09-17T04:31:44.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.166+0000 7f0330e00640 1 -- 172.21.15.100:0/1393519662 shutdown_connections 2024-09-17T04:31:44.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.166+0000 7f0330e00640 1 --2- 172.21.15.100:0/1393519662 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f03080791c0 0x7f030807b6d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:44.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.166+0000 7f0330e00640 1 --2- 172.21.15.100:0/1393519662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f032c1623a0 0x7f032c0b0820 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:44.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.166+0000 7f0330e00640 1 --2- 172.21.15.100:0/1393519662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f032c155370 0x7f032c153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:44.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.166+0000 7f0330e00640 1 --2- 172.21.15.100:0/1393519662 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f032c06f280 0x7f032c09c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:44.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.166+0000 7f0330e00640 1 -- 172.21.15.100:0/1393519662 >> 172.21.15.100:0/1393519662 conn(0x7f032c0b1e40 msgr2=0x7f032c1617b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:44.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.167+0000 7f0330e00640 1 -- 172.21.15.100:0/1393519662 shutdown_connections 2024-09-17T04:31:44.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.167+0000 7f0330e00640 1 -- 172.21.15.100:0/1393519662 wait complete. 2024-09-17T04:31:44.189 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2' 2024-09-17T04:31:44.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.409+0000 7f4184800640 1 -- 172.21.15.100:0/3872666684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f417c0704b0 msgr2=0x7f417c073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:44.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.409+0000 7f4184800640 1 --2- 172.21.15.100:0/3872666684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f417c0704b0 0x7f417c073040 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f417c0afd40 tx=0x7f417002ad50 comp rx=0 tx=0).stop 2024-09-17T04:31:44.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.409+0000 7f4184800640 1 -- 172.21.15.100:0/3872666684 shutdown_connections 2024-09-17T04:31:44.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.409+0000 7f4184800640 1 --2- 172.21.15.100:0/3872666684 >> v2:172.21.15.184:3300/0 conn(0x7f417c0add10 0x7f417c0b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:44.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.409+0000 7f4184800640 1 --2- 172.21.15.100:0/3872666684 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f417c0ad3c0 0x7f417c0ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:44.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.409+0000 7f4184800640 1 --2- 172.21.15.100:0/3872666684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f417c0704b0 0x7f417c073040 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:44.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.409+0000 7f4184800640 1 -- 172.21.15.100:0/3872666684 >> 172.21.15.100:0/3872666684 conn(0x7f417c06d730 msgr2=0x7f417c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:44.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.409+0000 7f4184800640 1 -- 172.21.15.100:0/3872666684 shutdown_connections 2024-09-17T04:31:44.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.409+0000 7f4184800640 1 -- 172.21.15.100:0/3872666684 wait complete. 2024-09-17T04:31:44.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.411+0000 7f4184800640 1 Processor -- start 2024-09-17T04:31:44.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.411+0000 7f4184800640 1 -- start start 2024-09-17T04:31:44.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.413+0000 7f4184800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f417c0704b0 0x7f417c153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:44.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.413+0000 7f4184800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f417c0ad3c0 0x7f417c15be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:44.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.413+0000 7f4184800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f417c0add10 0x7f417c1f4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.413+0000 7f4184800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f417c0712b0 con 0x7f417c0ad3c0 2024-09-17T04:31:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.413+0000 7f4184800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f417c071130 con 0x7f417c0add10 2024-09-17T04:31:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.413+0000 7f4184800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f417c0b4350 con 0x7f417c0704b0 2024-09-17T04:31:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.413+0000 7f4181400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f417c0ad3c0 0x7f417c15be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.413+0000 7f4181e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f417c0704b0 0x7f417c153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.413+0000 7f4181400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f417c0ad3c0 0x7f417c15be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40162/0 (socket says 172.21.15.100:40162) 2024-09-17T04:31:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.413+0000 7f4181400640 1 -- 172.21.15.100:0/3211281852 learned_addr learned my addr 172.21.15.100:0/3211281852 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.413+0000 7f4182800640 1 --2- 172.21.15.100:0/3211281852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f417c0add10 0x7f417c1f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.413+0000 7f4181400640 1 -- 172.21.15.100:0/3211281852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f417c0704b0 msgr2=0x7f417c153f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.413+0000 7f4181400640 1 --2- 172.21.15.100:0/3211281852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f417c0704b0 0x7f417c153f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.413+0000 7f4181400640 1 -- 172.21.15.100:0/3211281852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f417c0add10 msgr2=0x7f417c1f4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.413+0000 7f4181400640 1 --2- 172.21.15.100:0/3211281852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f417c0add10 0x7f417c1f4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.414+0000 7f4181400640 1 -- 172.21.15.100:0/3211281852 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f417c1f3ad0 con 0x7f417c0ad3c0 2024-09-17T04:31:44.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.414+0000 7f4182800640 1 --2- 172.21.15.100:0/3211281852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f417c0add10 0x7f417c1f4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:31:44.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.414+0000 7f4181400640 1 --2- 172.21.15.100:0/3211281852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f417c0ad3c0 0x7f417c15be50 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f416c00b410 tx=0x7f416c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:44.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.414+0000 7f4176000640 1 -- 172.21.15.100:0/3211281852 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c015020 con 0x7f417c0ad3c0 2024-09-17T04:31:44.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.414+0000 7f4176000640 1 -- 172.21.15.100:0/3211281852 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f416c011940 con 0x7f417c0ad3c0 2024-09-17T04:31:44.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.414+0000 7f4176000640 1 -- 172.21.15.100:0/3211281852 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c011d90 con 0x7f417c0ad3c0 2024-09-17T04:31:44.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.414+0000 7f4184800640 1 -- 172.21.15.100:0/3211281852 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f417c1f78a0 con 0x7f417c0ad3c0 2024-09-17T04:31:44.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.414+0000 7f4184800640 1 -- 172.21.15.100:0/3211281852 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f417c1f7e60 con 0x7f417c0ad3c0 2024-09-17T04:31:44.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.415+0000 7f4184800640 1 -- 172.21.15.100:0/3211281852 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f417c0ad7d0 con 0x7f417c0ad3c0 2024-09-17T04:31:44.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.416+0000 7f4176000640 1 -- 172.21.15.100:0/3211281852 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f416c018020 con 0x7f417c0ad3c0 2024-09-17T04:31:44.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.417+0000 7f4176000640 1 --2- 172.21.15.100:0/3211281852 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f41600791c0 0x7f416007b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:44.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.417+0000 7f4176000640 1 -- 172.21.15.100:0/3211281852 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4999+0+0 (secure 0 0 0) 0x7f416c09b5b0 con 0x7f417c0ad3c0 2024-09-17T04:31:44.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.418+0000 7f4181e00640 1 --2- 172.21.15.100:0/3211281852 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f41600791c0 0x7f416007b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:44.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.418+0000 7f4181e00640 1 --2- 172.21.15.100:0/3211281852 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f41600791c0 0x7f416007b6d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4170034000 tx=0x7f4170004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:44.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.422+0000 7f4176000640 1 -- 172.21.15.100:0/3211281852 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f416c01f020 con 0x7f417c0ad3c0 2024-09-17T04:31:44.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.674+0000 7f4184800640 1 -- 172.21.15.100:0/3211281852 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]} v 0) -- 0x7f417c1f7630 con 0x7f417c0ad3c0 2024-09-17T04:31:44.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.702+0000 7f4176000640 1 -- 172.21.15.100:0/3211281852 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]}]=0 v425) ==== 148+0+0 (secure 0 0 0) 0x7f417c1f3ad0 con 0x7f417c0ad3c0 2024-09-17T04:31:44.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.705+0000 7f4184800640 1 -- 172.21.15.100:0/3211281852 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f41600791c0 msgr2=0x7f416007b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:44.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.705+0000 7f4184800640 1 --2- 172.21.15.100:0/3211281852 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f41600791c0 0x7f416007b6d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4170034000 tx=0x7f4170004730 comp rx=0 tx=0).stop 2024-09-17T04:31:44.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.705+0000 7f4184800640 1 -- 172.21.15.100:0/3211281852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f417c0ad3c0 msgr2=0x7f417c15be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:44.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.705+0000 7f4184800640 1 --2- 172.21.15.100:0/3211281852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f417c0ad3c0 0x7f417c15be50 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f416c00b410 tx=0x7f416c00b8d0 comp rx=0 tx=0).stop 2024-09-17T04:31:44.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.705+0000 7f4184800640 1 -- 172.21.15.100:0/3211281852 shutdown_connections 2024-09-17T04:31:44.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.705+0000 7f4184800640 1 --2- 172.21.15.100:0/3211281852 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f41600791c0 0x7f416007b6d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:44.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.705+0000 7f4184800640 1 --2- 172.21.15.100:0/3211281852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f417c0add10 0x7f417c1f4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:44.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.705+0000 7f4184800640 1 --2- 172.21.15.100:0/3211281852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f417c0ad3c0 0x7f417c15be50 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:44.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.705+0000 7f4184800640 1 --2- 172.21.15.100:0/3211281852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f417c0704b0 0x7f417c153f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:44.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.705+0000 7f4184800640 1 -- 172.21.15.100:0/3211281852 >> 172.21.15.100:0/3211281852 conn(0x7f417c06d730 msgr2=0x7f417c0b1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:44.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.705+0000 7f4184800640 1 -- 172.21.15.100:0/3211281852 shutdown_connections 2024-09-17T04:31:44.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.705+0000 7f4184800640 1 -- 172.21.15.100:0/3211281852 wait complete. 2024-09-17T04:31:44.728 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-09-17T04:31:44.728 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_ops_on_nonexistent_vol (tasks.cephfs.test_volumes.TestMisc) 2024-09-17T04:31:44.728 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:31:44.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.947+0000 7f532b200640 1 -- 172.21.15.100:0/3519375011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53240b3550 msgr2=0x7f53240b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:44.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.947+0000 7f532b200640 1 --2- 172.21.15.100:0/3519375011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53240b3550 0x7f53240b3960 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f5324072d60 tx=0x7f531c02aa30 comp rx=0 tx=0).stop 2024-09-17T04:31:44.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.947+0000 7f532b200640 1 -- 172.21.15.100:0/3519375011 shutdown_connections 2024-09-17T04:31:44.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.947+0000 7f532b200640 1 --2- 172.21.15.100:0/3519375011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53240b3550 0x7f53240b3960 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:44.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.947+0000 7f532b200640 1 --2- 172.21.15.100:0/3519375011 >> v2:172.21.15.184:3300/0 conn(0x7f53240ad0e0 0x7f53240ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:44.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.947+0000 7f532b200640 1 -- 172.21.15.100:0/3519375011 >> 172.21.15.100:0/3519375011 conn(0x7f532406d730 msgr2=0x7f53240b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:44.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.947+0000 7f532b200640 1 -- 172.21.15.100:0/3519375011 shutdown_connections 2024-09-17T04:31:44.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.948+0000 7f532b200640 1 -- 172.21.15.100:0/3519375011 wait complete. 2024-09-17T04:31:44.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.950+0000 7f532b200640 1 Processor -- start 2024-09-17T04:31:44.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.950+0000 7f532b200640 1 -- start start 2024-09-17T04:31:44.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.951+0000 7f532b200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53240ad0e0 0x7f532414bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:44.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.951+0000 7f532b200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53240b3550 0x7f5324150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:44.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.951+0000 7f532b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f532415d760 0x7f53240b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:44.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.951+0000 7f532b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f53240b4da0 con 0x7f532415d760 2024-09-17T04:31:44.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.951+0000 7f532b200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f53240b4c20 con 0x7f53240b3550 2024-09-17T04:31:44.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.951+0000 7f532b200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f53240b3ea0 con 0x7f53240ad0e0 2024-09-17T04:31:44.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.951+0000 7f5329200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f532415d760 0x7f53240b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:44.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.952+0000 7f5329200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f532415d760 0x7f53240b1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34208/0 (socket says 172.21.15.100:34208) 2024-09-17T04:31:44.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.952+0000 7f5329200640 1 -- 172.21.15.100:0/143797720 learned_addr learned my addr 172.21.15.100:0/143797720 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:44.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.952+0000 7f531be00640 1 --2- 172.21.15.100:0/143797720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53240b3550 0x7f5324150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:44.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.952+0000 7f5323e00640 1 --2- 172.21.15.100:0/143797720 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53240ad0e0 0x7f532414bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:44.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.952+0000 7f5329200640 1 -- 172.21.15.100:0/143797720 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53240ad0e0 msgr2=0x7f532414bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:44.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.952+0000 7f5329200640 1 --2- 172.21.15.100:0/143797720 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53240ad0e0 0x7f532414bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:44.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.952+0000 7f5329200640 1 -- 172.21.15.100:0/143797720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53240b3550 msgr2=0x7f5324150650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:44.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.952+0000 7f5329200640 1 --2- 172.21.15.100:0/143797720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53240b3550 0x7f5324150650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:44.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.952+0000 7f5329200640 1 -- 172.21.15.100:0/143797720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5324150b90 con 0x7f532415d760 2024-09-17T04:31:44.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.952+0000 7f5329200640 1 --2- 172.21.15.100:0/143797720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f532415d760 0x7f53240b1860 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f531c02b020 tx=0x7f531c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:44.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.953+0000 7f5320c00640 1 -- 172.21.15.100:0/143797720 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c033050 con 0x7f532415d760 2024-09-17T04:31:44.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.953+0000 7f5320c00640 1 -- 172.21.15.100:0/143797720 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f531c004450 con 0x7f532415d760 2024-09-17T04:31:44.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.953+0000 7f5320c00640 1 -- 172.21.15.100:0/143797720 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c004840 con 0x7f532415d760 2024-09-17T04:31:44.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.953+0000 7f532b200640 1 -- 172.21.15.100:0/143797720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5324150e60 con 0x7f532415d760 2024-09-17T04:31:44.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.953+0000 7f532b200640 1 -- 172.21.15.100:0/143797720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53240b0240 con 0x7f532415d760 2024-09-17T04:31:44.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.954+0000 7f532b200640 1 -- 172.21.15.100:0/143797720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53240afd50 con 0x7f532415d760 2024-09-17T04:31:44.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.955+0000 7f5320c00640 1 -- 172.21.15.100:0/143797720 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f531c036070 con 0x7f532415d760 2024-09-17T04:31:44.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.956+0000 7f5320c00640 1 --2- 172.21.15.100:0/143797720 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f53000791c0 0x7f530007b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:44.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.956+0000 7f5320c00640 1 -- 172.21.15.100:0/143797720 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4999+0+0 (secure 0 0 0) 0x7f531c0bcb10 con 0x7f532415d760 2024-09-17T04:31:44.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.956+0000 7f5323e00640 1 --2- 172.21.15.100:0/143797720 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f53000791c0 0x7f530007b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:44.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.957+0000 7f5323e00640 1 --2- 172.21.15.100:0/143797720 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f53000791c0 0x7f530007b6d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5314001720 tx=0x7f5314009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:44.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:44.961+0000 7f5320c00640 1 -- 172.21.15.100:0/143797720 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f531c084130 con 0x7f532415d760 2024-09-17T04:31:45.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.145+0000 7f532b200640 1 -- 172.21.15.100:0/143797720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53240b4360 con 0x7f532415d760 2024-09-17T04:31:45.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.146+0000 7f5320c00640 1 -- 172.21.15.100:0/143797720 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) ==== 74+0+13179 (secure 0 0 0) 0x7f531c088b60 con 0x7f532415d760 2024-09-17T04:31:45.147 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:45.147 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":123,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:31:43.695503+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6835/207356836":"2024-09-18T04:31:39.986634+0000","172.21.15.100:6833/207356836":"2024-09-18T04:31:39.986634+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:31:45.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.148+0000 7f532b200640 1 -- 172.21.15.100:0/143797720 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f53000791c0 msgr2=0x7f530007b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:45.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.148+0000 7f532b200640 1 --2- 172.21.15.100:0/143797720 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f53000791c0 0x7f530007b6d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5314001720 tx=0x7f5314009040 comp rx=0 tx=0).stop 2024-09-17T04:31:45.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.148+0000 7f532b200640 1 -- 172.21.15.100:0/143797720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f532415d760 msgr2=0x7f53240b1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:45.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.148+0000 7f532b200640 1 --2- 172.21.15.100:0/143797720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f532415d760 0x7f53240b1860 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f531c02b020 tx=0x7f531c003f10 comp rx=0 tx=0).stop 2024-09-17T04:31:45.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.149+0000 7f532b200640 1 -- 172.21.15.100:0/143797720 shutdown_connections 2024-09-17T04:31:45.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.149+0000 7f532b200640 1 --2- 172.21.15.100:0/143797720 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f53000791c0 0x7f530007b6d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:45.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.149+0000 7f532b200640 1 --2- 172.21.15.100:0/143797720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f532415d760 0x7f53240b1860 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:45.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.149+0000 7f532b200640 1 --2- 172.21.15.100:0/143797720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53240b3550 0x7f5324150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:45.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.149+0000 7f532b200640 1 --2- 172.21.15.100:0/143797720 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53240ad0e0 0x7f532414bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:45.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.149+0000 7f532b200640 1 -- 172.21.15.100:0/143797720 >> 172.21.15.100:0/143797720 conn(0x7f532406d730 msgr2=0x7f53240cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:45.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.149+0000 7f532b200640 1 -- 172.21.15.100:0/143797720 shutdown_connections 2024-09-17T04:31:45.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.149+0000 7f532b200640 1 -- 172.21.15.100:0/143797720 wait complete. 2024-09-17T04:31:45.171 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T04:24:03.995355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T04:31:45.171 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T04:31:45.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.393+0000 7fac80400640 1 -- 172.21.15.100:0/2355707555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac78160490 msgr2=0x7fac7815d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:45.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.393+0000 7fac80400640 1 --2- 172.21.15.100:0/2355707555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac78160490 0x7fac7815d740 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fac7815d430 tx=0x7fac6402a500 comp rx=0 tx=0).stop 2024-09-17T04:31:45.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.393+0000 7fac80400640 1 -- 172.21.15.100:0/2355707555 shutdown_connections 2024-09-17T04:31:45.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.393+0000 7fac80400640 1 --2- 172.21.15.100:0/2355707555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac780ad3d0 0x7fac780ad7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:45.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.393+0000 7fac80400640 1 --2- 172.21.15.100:0/2355707555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac78160490 0x7fac7815d740 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:45.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.393+0000 7fac80400640 1 -- 172.21.15.100:0/2355707555 >> 172.21.15.100:0/2355707555 conn(0x7fac7806d710 msgr2=0x7fac780b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:45.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.393+0000 7fac80400640 1 -- 172.21.15.100:0/2355707555 shutdown_connections 2024-09-17T04:31:45.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.393+0000 7fac80400640 1 -- 172.21.15.100:0/2355707555 wait complete. 2024-09-17T04:31:45.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.396+0000 7fac80400640 1 Processor -- start 2024-09-17T04:31:45.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.396+0000 7fac80400640 1 -- start start 2024-09-17T04:31:45.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.397+0000 7fac80400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac780ad3d0 0x7fac7809da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:45.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.397+0000 7fac80400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac780b4f00 0x7fac7814a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:45.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.397+0000 7fac80400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac78160490 0x7fac78146fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:45.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.397+0000 7fac80400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fac7806f0f0 con 0x7fac780b4f00 2024-09-17T04:31:45.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.397+0000 7fac80400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fac7806ef70 con 0x7fac78160490 2024-09-17T04:31:45.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.397+0000 7fac80400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fac7806f270 con 0x7fac780ad3d0 2024-09-17T04:31:45.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.397+0000 7fac7d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac780b4f00 0x7fac7814a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:45.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.397+0000 7fac7d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac780b4f00 0x7fac7814a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34230/0 (socket says 172.21.15.100:34230) 2024-09-17T04:31:45.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.397+0000 7fac7d000640 1 -- 172.21.15.100:0/3825506792 learned_addr learned my addr 172.21.15.100:0/3825506792 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:45.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.398+0000 7fac7da00640 1 --2- 172.21.15.100:0/3825506792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac780ad3d0 0x7fac7809da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:45.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.398+0000 7fac7e400640 1 --2- 172.21.15.100:0/3825506792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac78160490 0x7fac78146fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:45.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.398+0000 7fac7d000640 1 -- 172.21.15.100:0/3825506792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac780ad3d0 msgr2=0x7fac7809da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:45.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.398+0000 7fac7d000640 1 --2- 172.21.15.100:0/3825506792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac780ad3d0 0x7fac7809da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:45.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.398+0000 7fac7d000640 1 -- 172.21.15.100:0/3825506792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac78160490 msgr2=0x7fac78146fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:45.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.398+0000 7fac7d000640 1 --2- 172.21.15.100:0/3825506792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac78160490 0x7fac78146fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:45.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.398+0000 7fac7d000640 1 -- 172.21.15.100:0/3825506792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac781f9750 con 0x7fac780b4f00 2024-09-17T04:31:45.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.398+0000 7fac7e400640 1 --2- 172.21.15.100:0/3825506792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac78160490 0x7fac78146fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:31:45.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.398+0000 7fac7da00640 1 --2- 172.21.15.100:0/3825506792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac780ad3d0 0x7fac7809da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:31:45.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.398+0000 7fac7d000640 1 --2- 172.21.15.100:0/3825506792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac780b4f00 0x7fac7814a300 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fac640098f0 tx=0x7fac64033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:45.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.399+0000 7fac75600640 1 -- 172.21.15.100:0/3825506792 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64005c90 con 0x7fac780b4f00 2024-09-17T04:31:45.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.399+0000 7fac75600640 1 -- 172.21.15.100:0/3825506792 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac64031e50 con 0x7fac780b4f00 2024-09-17T04:31:45.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.399+0000 7fac80400640 1 -- 172.21.15.100:0/3825506792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac7814a840 con 0x7fac780b4f00 2024-09-17T04:31:45.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.399+0000 7fac75600640 1 -- 172.21.15.100:0/3825506792 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64005260 con 0x7fac780b4f00 2024-09-17T04:31:45.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.399+0000 7fac80400640 1 -- 172.21.15.100:0/3825506792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac7809cc40 con 0x7fac780b4f00 2024-09-17T04:31:45.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.400+0000 7fac80400640 1 -- 172.21.15.100:0/3825506792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac7809c6d0 con 0x7fac780b4f00 2024-09-17T04:31:45.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.401+0000 7fac75600640 1 -- 172.21.15.100:0/3825506792 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fac64005400 con 0x7fac780b4f00 2024-09-17T04:31:45.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.402+0000 7fac75600640 1 --2- 172.21.15.100:0/3825506792 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fac580791f0 0x7fac5807b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:45.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.402+0000 7fac75600640 1 -- 172.21.15.100:0/3825506792 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4999+0+0 (secure 0 0 0) 0x7fac640bc790 con 0x7fac780b4f00 2024-09-17T04:31:45.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.402+0000 7fac7da00640 1 --2- 172.21.15.100:0/3825506792 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fac580791f0 0x7fac5807b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:45.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.403+0000 7fac7da00640 1 --2- 172.21.15.100:0/3825506792 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fac580791f0 0x7fac5807b700 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fac6c007a50 tx=0x7fac6c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:45.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.407+0000 7fac75600640 1 -- 172.21.15.100:0/3825506792 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fac6403f030 con 0x7fac780b4f00 2024-09-17T04:31:45.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.588+0000 7fac80400640 1 -- 172.21.15.100:0/3825506792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fac7806f6f0 con 0x7fac780b4f00 2024-09-17T04:31:45.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.589+0000 7fac75600640 1 -- 172.21.15.100:0/3825506792 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) ==== 93+0+10 (secure 0 0 0) 0x7fac64084570 con 0x7fac780b4f00 2024-09-17T04:31:45.589 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-09-17T04:31:45.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.591+0000 7fac80400640 1 -- 172.21.15.100:0/3825506792 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fac580791f0 msgr2=0x7fac5807b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:45.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.591+0000 7fac80400640 1 --2- 172.21.15.100:0/3825506792 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fac580791f0 0x7fac5807b700 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fac6c007a50 tx=0x7fac6c009040 comp rx=0 tx=0).stop 2024-09-17T04:31:45.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.591+0000 7fac80400640 1 -- 172.21.15.100:0/3825506792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac780b4f00 msgr2=0x7fac7814a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:45.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.591+0000 7fac80400640 1 --2- 172.21.15.100:0/3825506792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac780b4f00 0x7fac7814a300 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fac640098f0 tx=0x7fac64033c20 comp rx=0 tx=0).stop 2024-09-17T04:31:45.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.591+0000 7fac80400640 1 -- 172.21.15.100:0/3825506792 shutdown_connections 2024-09-17T04:31:45.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.591+0000 7fac80400640 1 --2- 172.21.15.100:0/3825506792 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fac580791f0 0x7fac5807b700 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:45.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.591+0000 7fac80400640 1 --2- 172.21.15.100:0/3825506792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac78160490 0x7fac78146fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:45.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.591+0000 7fac80400640 1 --2- 172.21.15.100:0/3825506792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac780b4f00 0x7fac7814a300 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:45.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.591+0000 7fac80400640 1 --2- 172.21.15.100:0/3825506792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac780ad3d0 0x7fac7809da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:45.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.591+0000 7fac80400640 1 -- 172.21.15.100:0/3825506792 >> 172.21.15.100:0/3825506792 conn(0x7fac7806d710 msgr2=0x7fac780b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:45.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.591+0000 7fac80400640 1 -- 172.21.15.100:0/3825506792 shutdown_connections 2024-09-17T04:31:45.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.591+0000 7fac80400640 1 -- 172.21.15.100:0/3825506792 wait complete. 2024-09-17T04:31:45.614 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol' 2024-09-17T04:31:45.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.838+0000 7fb07c200640 1 -- 172.21.15.100:0/4182662278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb05c004d40 msgr2=0x7fb05c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:45.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.838+0000 7fb07c200640 1 --2- 172.21.15.100:0/4182662278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb05c004d40 0x7fb05c0051d0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fb074159930 tx=0x7fb05c005830 comp rx=0 tx=0).stop 2024-09-17T04:31:45.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.838+0000 7fb07a200640 1 -- 172.21.15.100:0/4182662278 reap_dead start 2024-09-17T04:31:45.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.838+0000 7fb07c200640 1 -- 172.21.15.100:0/4182662278 shutdown_connections 2024-09-17T04:31:45.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.838+0000 7fb07c200640 1 -- 172.21.15.100:0/4182662278 >> 172.21.15.100:0/4182662278 conn(0x7fb0740d0b50 msgr2=0x7fb0740d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:45.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.838+0000 7fb07c200640 1 -- 172.21.15.100:0/4182662278 shutdown_connections 2024-09-17T04:31:45.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.838+0000 7fb07c200640 1 -- 172.21.15.100:0/4182662278 wait complete. 2024-09-17T04:31:45.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.840+0000 7fb07c200640 1 Processor -- start 2024-09-17T04:31:45.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.840+0000 7fb07c200640 1 -- start start 2024-09-17T04:31:45.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.842+0000 7fb07c200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb05c004d40 0x7fb07413ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:45.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.842+0000 7fb07c200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb074156630 0x7fb074156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:45.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.842+0000 7fb07c200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb07413e3b0 0x7fb07413e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:45.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.842+0000 7fb07c200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb07406cf20 con 0x7fb05c004d40 2024-09-17T04:31:45.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.842+0000 7fb07c200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb07406cda0 con 0x7fb074156630 2024-09-17T04:31:45.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.842+0000 7fb07c200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb07406d0a0 con 0x7fb07413e3b0 2024-09-17T04:31:45.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.842+0000 7fb079800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb05c004d40 0x7fb07413ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:45.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.842+0000 7fb079800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb05c004d40 0x7fb07413ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34254/0 (socket says 172.21.15.100:34254) 2024-09-17T04:31:45.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.842+0000 7fb079800640 1 -- 172.21.15.100:0/1481744378 learned_addr learned my addr 172.21.15.100:0/1481744378 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:45.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.842+0000 7fb07a200640 1 --2- 172.21.15.100:0/1481744378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb07413e3b0 0x7fb07413e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:45.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.842+0000 7fb078e00640 1 --2- 172.21.15.100:0/1481744378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb074156630 0x7fb074156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:45.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.843+0000 7fb079800640 1 -- 172.21.15.100:0/1481744378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb07413e3b0 msgr2=0x7fb07413e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:45.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.843+0000 7fb079800640 1 --2- 172.21.15.100:0/1481744378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb07413e3b0 0x7fb07413e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:45.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.843+0000 7fb079800640 1 -- 172.21.15.100:0/1481744378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb074156630 msgr2=0x7fb074156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:45.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.843+0000 7fb079800640 1 --2- 172.21.15.100:0/1481744378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb074156630 0x7fb074156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:45.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.843+0000 7fb079800640 1 -- 172.21.15.100:0/1481744378 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb07413ed00 con 0x7fb05c004d40 2024-09-17T04:31:45.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.843+0000 7fb078e00640 1 --2- 172.21.15.100:0/1481744378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb074156630 0x7fb074156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:31:45.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.843+0000 7fb079800640 1 --2- 172.21.15.100:0/1481744378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb05c004d40 0x7fb07413ef90 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fb06800e6d0 tx=0x7fb06800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:45.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.843+0000 7fb071600640 1 -- 172.21.15.100:0/1481744378 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06800eea0 con 0x7fb05c004d40 2024-09-17T04:31:45.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.843+0000 7fb071600640 1 -- 172.21.15.100:0/1481744378 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb068016070 con 0x7fb05c004d40 2024-09-17T04:31:45.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.843+0000 7fb07c200640 1 -- 172.21.15.100:0/1481744378 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0741556f0 con 0x7fb05c004d40 2024-09-17T04:31:45.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.843+0000 7fb071600640 1 -- 172.21.15.100:0/1481744378 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06800caa0 con 0x7fb05c004d40 2024-09-17T04:31:45.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.843+0000 7fb07c200640 1 -- 172.21.15.100:0/1481744378 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb07413c600 con 0x7fb05c004d40 2024-09-17T04:31:45.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.844+0000 7fb07c200640 1 -- 172.21.15.100:0/1481744378 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb07406d600 con 0x7fb05c004d40 2024-09-17T04:31:45.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.846+0000 7fb071600640 1 -- 172.21.15.100:0/1481744378 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fb06800cc40 con 0x7fb05c004d40 2024-09-17T04:31:45.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.847+0000 7fb071600640 1 --2- 172.21.15.100:0/1481744378 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb0540791c0 0x7fb05407b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:45.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.847+0000 7fb071600640 1 -- 172.21.15.100:0/1481744378 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4999+0+0 (secure 0 0 0) 0x7fb06809bb30 con 0x7fb05c004d40 2024-09-17T04:31:45.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.847+0000 7fb078e00640 1 --2- 172.21.15.100:0/1481744378 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb0540791c0 0x7fb05407b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:45.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.848+0000 7fb078e00640 1 --2- 172.21.15.100:0/1481744378 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb0540791c0 0x7fb05407b6d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb060009c10 tx=0x7fb060005e30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:45.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:45.851+0000 7fb071600640 1 -- 172.21.15.100:0/1481744378 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb07413ed00 con 0x7fb05c004d40 2024-09-17T04:31:46.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.104+0000 7fb07c200640 1 -- 172.21.15.100:0/1481744378 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]} v 0) -- 0x7fb07414f4f0 con 0x7fb05c004d40 2024-09-17T04:31:46.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.712+0000 7fb071600640 1 -- 172.21.15.100:0/1481744378 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]}]=0 v427) ==== 154+0+0 (secure 0 0 0) 0x7fb068063830 con 0x7fb05c004d40 2024-09-17T04:31:46.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.714+0000 7fb07c200640 1 -- 172.21.15.100:0/1481744378 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb0540791c0 msgr2=0x7fb05407b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:46.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.714+0000 7fb07c200640 1 --2- 172.21.15.100:0/1481744378 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb0540791c0 0x7fb05407b6d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb060009c10 tx=0x7fb060005e30 comp rx=0 tx=0).stop 2024-09-17T04:31:46.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.714+0000 7fb07c200640 1 -- 172.21.15.100:0/1481744378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb05c004d40 msgr2=0x7fb07413ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:46.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.714+0000 7fb07c200640 1 --2- 172.21.15.100:0/1481744378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb05c004d40 0x7fb07413ef90 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fb06800e6d0 tx=0x7fb06800eb90 comp rx=0 tx=0).stop 2024-09-17T04:31:46.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.715+0000 7fb07c200640 1 -- 172.21.15.100:0/1481744378 shutdown_connections 2024-09-17T04:31:46.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.715+0000 7fb07c200640 1 --2- 172.21.15.100:0/1481744378 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb0540791c0 0x7fb05407b6d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:46.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.715+0000 7fb07c200640 1 --2- 172.21.15.100:0/1481744378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb07413e3b0 0x7fb07413e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:46.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.715+0000 7fb07c200640 1 --2- 172.21.15.100:0/1481744378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb074156630 0x7fb074156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:46.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.715+0000 7fb07c200640 1 --2- 172.21.15.100:0/1481744378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb05c004d40 0x7fb07413ef90 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:46.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.715+0000 7fb07c200640 1 -- 172.21.15.100:0/1481744378 >> 172.21.15.100:0/1481744378 conn(0x7fb0740d0b50 msgr2=0x7fb07414e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:46.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.715+0000 7fb07c200640 1 -- 172.21.15.100:0/1481744378 shutdown_connections 2024-09-17T04:31:46.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.715+0000 7fb07c200640 1 -- 172.21.15.100:0/1481744378 wait complete. 2024-09-17T04:31:46.738 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T04:31:46.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.959+0000 7f0b9e600640 1 -- 172.21.15.100:0/2967848099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b980d29e0 msgr2=0x7f0b980d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:46.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.959+0000 7f0b9e600640 1 --2- 172.21.15.100:0/2967848099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b980d29e0 0x7f0b980d1580 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f0b9808d440 tx=0x7f0b9002a980 comp rx=0 tx=0).stop 2024-09-17T04:31:46.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.959+0000 7f0b9e600640 1 -- 172.21.15.100:0/2967848099 shutdown_connections 2024-09-17T04:31:46.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.960+0000 7f0b9e600640 1 --2- 172.21.15.100:0/2967848099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b980d29e0 0x7f0b980d1580 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:46.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.960+0000 7f0b9e600640 1 --2- 172.21.15.100:0/2967848099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b980b3160 0x7f0b980d24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:46.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.960+0000 7f0b9e600640 1 --2- 172.21.15.100:0/2967848099 >> v2:172.21.15.184:3300/0 conn(0x7f0b980b9e00 0x7f0b98162a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:46.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.960+0000 7f0b9e600640 1 -- 172.21.15.100:0/2967848099 >> 172.21.15.100:0/2967848099 conn(0x7f0b9808bfe0 msgr2=0x7f0b9809d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:46.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.960+0000 7f0b9e600640 1 -- 172.21.15.100:0/2967848099 shutdown_connections 2024-09-17T04:31:46.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.960+0000 7f0b9e600640 1 -- 172.21.15.100:0/2967848099 wait complete. 2024-09-17T04:31:46.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.962+0000 7f0b9e600640 1 Processor -- start 2024-09-17T04:31:46.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.962+0000 7f0b9e600640 1 -- start start 2024-09-17T04:31:46.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.963+0000 7f0b9e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b980b3160 0x7f0b9815df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:46.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.963+0000 7f0b9e600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b980b9e00 0x7f0b98160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:46.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.963+0000 7f0b9e600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b980d29e0 0x7f0b98161000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:46.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.963+0000 7f0b9e600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0b980d4330 con 0x7f0b980b3160 2024-09-17T04:31:46.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.963+0000 7f0b9e600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0b980d1e10 con 0x7f0b980d29e0 2024-09-17T04:31:46.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.963+0000 7f0b9e600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0b980d44b0 con 0x7f0b980b9e00 2024-09-17T04:31:46.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.964+0000 7f0b9d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b980b3160 0x7f0b9815df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:46.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.964+0000 7f0b9d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b980b3160 0x7f0b9815df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34274/0 (socket says 172.21.15.100:34274) 2024-09-17T04:31:46.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.964+0000 7f0b9d200640 1 -- 172.21.15.100:0/3301068792 learned_addr learned my addr 172.21.15.100:0/3301068792 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:46.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.964+0000 7f0b9dc00640 1 --2- 172.21.15.100:0/3301068792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b980d29e0 0x7f0b98161000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:46.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.964+0000 7f0b97e00640 1 --2- 172.21.15.100:0/3301068792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b980b9e00 0x7f0b98160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:46.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.964+0000 7f0b9d200640 1 -- 172.21.15.100:0/3301068792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b980b9e00 msgr2=0x7f0b98160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:46.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.964+0000 7f0b9d200640 1 --2- 172.21.15.100:0/3301068792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b980b9e00 0x7f0b98160ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:46.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.964+0000 7f0b9d200640 1 -- 172.21.15.100:0/3301068792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b980d29e0 msgr2=0x7f0b98161000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:46.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.964+0000 7f0b9d200640 1 --2- 172.21.15.100:0/3301068792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b980d29e0 0x7f0b98161000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:46.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.964+0000 7f0b9d200640 1 -- 172.21.15.100:0/3301068792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b9815e4b0 con 0x7f0b980b3160 2024-09-17T04:31:46.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.964+0000 7f0b9d200640 1 --2- 172.21.15.100:0/3301068792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b980b3160 0x7f0b9815df70 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f0b8800b440 tx=0x7f0b8800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:46.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.965+0000 7f0b94c00640 1 -- 172.21.15.100:0/3301068792 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b88015020 con 0x7f0b980b3160 2024-09-17T04:31:46.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.965+0000 7f0b94c00640 1 -- 172.21.15.100:0/3301068792 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b88011980 con 0x7f0b980b3160 2024-09-17T04:31:46.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.965+0000 7f0b9e600640 1 -- 172.21.15.100:0/3301068792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b9809eba0 con 0x7f0b980b3160 2024-09-17T04:31:46.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.965+0000 7f0b94c00640 1 -- 172.21.15.100:0/3301068792 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b88011dd0 con 0x7f0b980b3160 2024-09-17T04:31:46.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.965+0000 7f0b9e600640 1 -- 172.21.15.100:0/3301068792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b980ce490 con 0x7f0b980b3160 2024-09-17T04:31:46.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.966+0000 7f0b9e600640 1 -- 172.21.15.100:0/3301068792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b980d4a30 con 0x7f0b980b3160 2024-09-17T04:31:46.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.967+0000 7f0b94c00640 1 -- 172.21.15.100:0/3301068792 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f0b88018020 con 0x7f0b980b3160 2024-09-17T04:31:46.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.968+0000 7f0b94c00640 1 --2- 172.21.15.100:0/3301068792 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0b780791c0 0x7f0b7807b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:46.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.968+0000 7f0b94c00640 1 -- 172.21.15.100:0/3301068792 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4999+0+0 (secure 0 0 0) 0x7f0b8809b690 con 0x7f0b980b3160 2024-09-17T04:31:46.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.968+0000 7f0b97e00640 1 --2- 172.21.15.100:0/3301068792 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0b780791c0 0x7f0b7807b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:46.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.969+0000 7f0b97e00640 1 --2- 172.21.15.100:0/3301068792 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0b780791c0 0x7f0b7807b6d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0b98162e10 tx=0x7f0b84009060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:46.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:46.973+0000 7f0b94c00640 1 -- 172.21.15.100:0/3301068792 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0b8801f020 con 0x7f0b980b3160 2024-09-17T04:31:47.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.156+0000 7f0b9e600640 1 -- 172.21.15.100:0/3301068792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f0b980b4620 con 0x7f0b980b3160 2024-09-17T04:31:47.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.156+0000 7f0b94c00640 1 -- 172.21.15.100:0/3301068792 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v45) ==== 59+0+2294 (secure 0 0 0) 0x7f0b8800d040 con 0x7f0b980b3160 2024-09-17T04:31:47.157 INFO:teuthology.orchestra.run.smithi100.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T04:31:47.157 INFO:teuthology.orchestra.run.smithi100.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T04:31:47.157 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced client_mount_timeout 30 2024-09-17T04:31:47.157 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_client 20/20 2024-09-17T04:31:47.157 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_mgr 20/20 2024-09-17T04:31:47.157 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_ms 1/1 2024-09-17T04:31:47.157 INFO:teuthology.orchestra.run.smithi100.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T04:31:47.157 INFO:teuthology.orchestra.run.smithi100.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T04:31:47.158 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3938.598654 2024-09-17T04:31:47.158 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4008.640154 2024-09-17T04:31:47.158 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3706.588546 2024-09-17T04:31:47.158 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 6299.338665 2024-09-17T04:31:47.158 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3633.876699 2024-09-17T04:31:47.158 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3674.821841 2024-09-17T04:31:47.158 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3789.849984 2024-09-17T04:31:47.158 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3467.457298 2024-09-17T04:31:47.158 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_mds 20/20 2024-09-17T04:31:47.158 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T04:31:47.158 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_ms 2/2 2024-09-17T04:31:47.158 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_debug_frag true 2024-09-17T04:31:47.158 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_debug_scatterstat true 2024-09-17T04:31:47.159 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T04:31:47.159 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_verify_scatter true 2024-09-17T04:31:47.159 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T04:31:47.159 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T04:31:47.159 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T04:31:47.159 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced client_mount_timeout 600 2024-09-17T04:31:47.159 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced debug_client 20/20 2024-09-17T04:31:47.159 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced debug_ms 1/1 2024-09-17T04:31:47.159 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T04:31:47.159 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T04:31:47.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.159+0000 7f0b9e600640 1 -- 172.21.15.100:0/3301068792 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0b780791c0 msgr2=0x7f0b7807b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:47.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.159+0000 7f0b9e600640 1 --2- 172.21.15.100:0/3301068792 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0b780791c0 0x7f0b7807b6d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0b98162e10 tx=0x7f0b84009060 comp rx=0 tx=0).stop 2024-09-17T04:31:47.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.159+0000 7f0b9e600640 1 -- 172.21.15.100:0/3301068792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b980b3160 msgr2=0x7f0b9815df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:47.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.159+0000 7f0b9e600640 1 --2- 172.21.15.100:0/3301068792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b980b3160 0x7f0b9815df70 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f0b8800b440 tx=0x7f0b8800b900 comp rx=0 tx=0).stop 2024-09-17T04:31:47.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.159+0000 7f0b9e600640 1 -- 172.21.15.100:0/3301068792 shutdown_connections 2024-09-17T04:31:47.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.159+0000 7f0b9e600640 1 --2- 172.21.15.100:0/3301068792 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0b780791c0 0x7f0b7807b6d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:47.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.159+0000 7f0b9e600640 1 --2- 172.21.15.100:0/3301068792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b980d29e0 0x7f0b98161000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:47.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.159+0000 7f0b9e600640 1 --2- 172.21.15.100:0/3301068792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b980b9e00 0x7f0b98160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:47.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.159+0000 7f0b9e600640 1 --2- 172.21.15.100:0/3301068792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b980b3160 0x7f0b9815df70 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:47.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.159+0000 7f0b9e600640 1 -- 172.21.15.100:0/3301068792 >> 172.21.15.100:0/3301068792 conn(0x7f0b9808bfe0 msgr2=0x7f0b980cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:47.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.159+0000 7f0b9e600640 1 -- 172.21.15.100:0/3301068792 shutdown_connections 2024-09-17T04:31:47.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.159+0000 7f0b9e600640 1 -- 172.21.15.100:0/3301068792 wait complete. 2024-09-17T04:31:47.180 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T04:31:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.403+0000 7f264ea00640 1 -- 172.21.15.100:0/486422424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f262c004800 msgr2=0x7f262c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.403+0000 7f264ea00640 1 --2- 172.21.15.100:0/486422424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f262c004800 0x7f262c004c10 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f2630001690 tx=0x7f262c005830 comp rx=0 tx=0).stop 2024-09-17T04:31:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.403+0000 7f264ea00640 1 -- 172.21.15.100:0/486422424 shutdown_connections 2024-09-17T04:31:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.403+0000 7f264ea00640 1 --2- 172.21.15.100:0/486422424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f262c004800 0x7f262c004c10 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.403+0000 7f264ea00640 1 -- 172.21.15.100:0/486422424 >> 172.21.15.100:0/486422424 conn(0x7f264806d1c0 msgr2=0x7f264806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.403+0000 7f264ea00640 1 -- 172.21.15.100:0/486422424 shutdown_connections 2024-09-17T04:31:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.403+0000 7f264ea00640 1 -- 172.21.15.100:0/486422424 wait complete. 2024-09-17T04:31:47.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.405+0000 7f264ea00640 1 Processor -- start 2024-09-17T04:31:47.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.405+0000 7f264ea00640 1 -- start start 2024-09-17T04:31:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.406+0000 7f264ea00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f262c004800 0x7f264815c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.406+0000 7f264ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26481f5430 0x7f26481f5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.406+0000 7f264ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f264815ef30 0x7f264815f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.406+0000 7f264ea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f264806f510 con 0x7f264815ef30 2024-09-17T04:31:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.406+0000 7f264ea00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f264806f390 con 0x7f26481f5430 2024-09-17T04:31:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.406+0000 7f264ea00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f264806f690 con 0x7f262c004800 2024-09-17T04:31:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.407+0000 7f264ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f264815ef30 0x7f264815f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:47.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.407+0000 7f264ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f264815ef30 0x7f264815f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34292/0 (socket says 172.21.15.100:34292) 2024-09-17T04:31:47.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.407+0000 7f264ca00640 1 -- 172.21.15.100:0/1355883500 learned_addr learned my addr 172.21.15.100:0/1355883500 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:47.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.407+0000 7f263f400640 1 --2- 172.21.15.100:0/1355883500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26481f5430 0x7f26481f5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:47.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.407+0000 7f2647e00640 1 --2- 172.21.15.100:0/1355883500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f262c004800 0x7f264815c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:47.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.407+0000 7f264ca00640 1 -- 172.21.15.100:0/1355883500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f262c004800 msgr2=0x7f264815c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:47.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.407+0000 7f264ca00640 1 --2- 172.21.15.100:0/1355883500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f262c004800 0x7f264815c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:47.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.407+0000 7f264ca00640 1 -- 172.21.15.100:0/1355883500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26481f5430 msgr2=0x7f26481f5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:47.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.407+0000 7f264ca00640 1 --2- 172.21.15.100:0/1355883500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26481f5430 0x7f26481f5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:47.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.407+0000 7f264ca00640 1 -- 172.21.15.100:0/1355883500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264815f880 con 0x7f264815ef30 2024-09-17T04:31:47.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.407+0000 7f2647e00640 1 --2- 172.21.15.100:0/1355883500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f262c004800 0x7f264815c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:31:47.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.407+0000 7f263f400640 1 --2- 172.21.15.100:0/1355883500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26481f5430 0x7f26481f5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:31:47.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.407+0000 7f264ca00640 1 --2- 172.21.15.100:0/1355883500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f264815ef30 0x7f264815f340 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f2640005fd0 tx=0x7f264000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:47.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.408+0000 7f2644c00640 1 -- 172.21.15.100:0/1355883500 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640003d80 con 0x7f264815ef30 2024-09-17T04:31:47.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.408+0000 7f2644c00640 1 -- 172.21.15.100:0/1355883500 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2640004b20 con 0x7f264815ef30 2024-09-17T04:31:47.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.408+0000 7f264ea00640 1 -- 172.21.15.100:0/1355883500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2648137bf0 con 0x7f264815ef30 2024-09-17T04:31:47.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.408+0000 7f2644c00640 1 -- 172.21.15.100:0/1355883500 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640004030 con 0x7f264815ef30 2024-09-17T04:31:47.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.408+0000 7f264ea00640 1 -- 172.21.15.100:0/1355883500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26481381b0 con 0x7f264815ef30 2024-09-17T04:31:47.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.409+0000 7f264ea00640 1 -- 172.21.15.100:0/1355883500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2648073c70 con 0x7f264815ef30 2024-09-17T04:31:47.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.410+0000 7f2644c00640 1 -- 172.21.15.100:0/1355883500 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f26400041d0 con 0x7f264815ef30 2024-09-17T04:31:47.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.411+0000 7f2644c00640 1 --2- 172.21.15.100:0/1355883500 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f26240791f0 0x7f262407b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:47.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.411+0000 7f2644c00640 1 -- 172.21.15.100:0/1355883500 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4999+0+0 (secure 0 0 0) 0x7f264009c690 con 0x7f264815ef30 2024-09-17T04:31:47.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.411+0000 7f2647e00640 1 --2- 172.21.15.100:0/1355883500 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f26240791f0 0x7f262407b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:47.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.412+0000 7f2647e00640 1 --2- 172.21.15.100:0/1355883500 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f26240791f0 0x7f262407b700 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2638000fb0 tx=0x7f26380067a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:47.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.416+0000 7f2644c00640 1 -- 172.21.15.100:0/1355883500 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f264815f880 con 0x7f264815ef30 2024-09-17T04:31:47.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.601+0000 7f264ea00640 1 -- 172.21.15.100:0/1355883500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f26480ceef0 con 0x7f264815ef30 2024-09-17T04:31:47.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.605+0000 7f2644c00640 1 -- 172.21.15.100:0/1355883500 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7f2640064390 con 0x7f264815ef30 2024-09-17T04:31:47.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.608+0000 7f264ea00640 1 -- 172.21.15.100:0/1355883500 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f26240791f0 msgr2=0x7f262407b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:47.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.608+0000 7f264ea00640 1 --2- 172.21.15.100:0/1355883500 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f26240791f0 0x7f262407b700 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2638000fb0 tx=0x7f26380067a0 comp rx=0 tx=0).stop 2024-09-17T04:31:47.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.608+0000 7f264ea00640 1 -- 172.21.15.100:0/1355883500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f264815ef30 msgr2=0x7f264815f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:47.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.608+0000 7f264ea00640 1 --2- 172.21.15.100:0/1355883500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f264815ef30 0x7f264815f340 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f2640005fd0 tx=0x7f264000f790 comp rx=0 tx=0).stop 2024-09-17T04:31:47.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.608+0000 7f264ea00640 1 -- 172.21.15.100:0/1355883500 shutdown_connections 2024-09-17T04:31:47.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.608+0000 7f264ea00640 1 --2- 172.21.15.100:0/1355883500 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f26240791f0 0x7f262407b700 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:47.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.608+0000 7f264ea00640 1 --2- 172.21.15.100:0/1355883500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f264815ef30 0x7f264815f340 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:47.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.608+0000 7f264ea00640 1 --2- 172.21.15.100:0/1355883500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26481f5430 0x7f26481f5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:47.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.608+0000 7f264ea00640 1 --2- 172.21.15.100:0/1355883500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f262c004800 0x7f264815c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:47.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.608+0000 7f264ea00640 1 -- 172.21.15.100:0/1355883500 >> 172.21.15.100:0/1355883500 conn(0x7f264806d1c0 msgr2=0x7f26480aef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:47.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.608+0000 7f264ea00640 1 -- 172.21.15.100:0/1355883500 shutdown_connections 2024-09-17T04:31:47.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:47.608+0000 7f264ea00640 1 -- 172.21.15.100:0/1355883500 wait complete. 2024-09-17T04:31:47.631 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T04:31:47.631 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:31:47.719 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:31:47.802 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T04:31:47.802 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:31:47.802 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:31:47.838 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:31:47.838 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:31:47.838 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:31:47.875 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:31:47.876 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:31:47.876 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:31:47.913 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:31:47.913 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:31:47.913 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:31:47.951 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:31:47.951 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T04:31:47.952 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:31:48.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.171+0000 7fb336a00640 1 -- 172.21.15.100:0/2472406990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb314004800 msgr2=0x7fb314004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:48.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.171+0000 7fb336a00640 1 --2- 172.21.15.100:0/2472406990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb314004800 0x7fb314004c10 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fb31c006fc0 tx=0x7fb314005830 comp rx=0 tx=0).stop 2024-09-17T04:31:48.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.171+0000 7fb334a00640 1 -- 172.21.15.100:0/2472406990 reap_dead start 2024-09-17T04:31:48.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.171+0000 7fb336a00640 1 -- 172.21.15.100:0/2472406990 shutdown_connections 2024-09-17T04:31:48.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.171+0000 7fb336a00640 1 -- 172.21.15.100:0/2472406990 >> 172.21.15.100:0/2472406990 conn(0x7fb33006d730 msgr2=0x7fb3300b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:48.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.171+0000 7fb336a00640 1 -- 172.21.15.100:0/2472406990 shutdown_connections 2024-09-17T04:31:48.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.171+0000 7fb336a00640 1 -- 172.21.15.100:0/2472406990 wait complete. 2024-09-17T04:31:48.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.173+0000 7fb336a00640 1 Processor -- start 2024-09-17T04:31:48.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.173+0000 7fb336a00640 1 -- start start 2024-09-17T04:31:48.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.175+0000 7fb336a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb314004800 0x7fb33014fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:48.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.175+0000 7fb336a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3300d0dc0 0x7fb330153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:48.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.175+0000 7fb336a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb330153980 0x7fb3300b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:48.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.175+0000 7fb336a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb330164540 con 0x7fb330153980 2024-09-17T04:31:48.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.175+0000 7fb336a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb3300b4970 con 0x7fb3300d0dc0 2024-09-17T04:31:48.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.175+0000 7fb336a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb3301646c0 con 0x7fb314004800 2024-09-17T04:31:48.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.175+0000 7fb334a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb330153980 0x7fb3300b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:48.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.175+0000 7fb334a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb330153980 0x7fb3300b0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34304/0 (socket says 172.21.15.100:34304) 2024-09-17T04:31:48.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.175+0000 7fb334a00640 1 -- 172.21.15.100:0/3693992006 learned_addr learned my addr 172.21.15.100:0/3693992006 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:48.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.175+0000 7fb32fe00640 1 --2- 172.21.15.100:0/3693992006 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb314004800 0x7fb33014fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:48.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.175+0000 7fb32f400640 1 --2- 172.21.15.100:0/3693992006 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3300d0dc0 0x7fb330153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:48.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.175+0000 7fb334a00640 1 -- 172.21.15.100:0/3693992006 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb314004800 msgr2=0x7fb33014fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:48.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.175+0000 7fb334a00640 1 --2- 172.21.15.100:0/3693992006 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb314004800 0x7fb33014fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:48.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.175+0000 7fb334a00640 1 -- 172.21.15.100:0/3693992006 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3300d0dc0 msgr2=0x7fb330153440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:48.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.175+0000 7fb334a00640 1 --2- 172.21.15.100:0/3693992006 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3300d0dc0 0x7fb330153440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:48.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.175+0000 7fb334a00640 1 -- 172.21.15.100:0/3693992006 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3301f3be0 con 0x7fb330153980 2024-09-17T04:31:48.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.176+0000 7fb334a00640 1 --2- 172.21.15.100:0/3693992006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb330153980 0x7fb3300b0df0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fb328005fd0 tx=0x7fb32800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:48.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.176+0000 7fb327e00640 1 -- 172.21.15.100:0/3693992006 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb328004b20 con 0x7fb330153980 2024-09-17T04:31:48.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.176+0000 7fb327e00640 1 -- 172.21.15.100:0/3693992006 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb328004cc0 con 0x7fb330153980 2024-09-17T04:31:48.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.176+0000 7fb336a00640 1 -- 172.21.15.100:0/3693992006 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3301f3ec0 con 0x7fb330153980 2024-09-17T04:31:48.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.176+0000 7fb327e00640 1 -- 172.21.15.100:0/3693992006 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb328003bd0 con 0x7fb330153980 2024-09-17T04:31:48.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.176+0000 7fb336a00640 1 -- 172.21.15.100:0/3693992006 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb330152f40 con 0x7fb330153980 2024-09-17T04:31:48.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.177+0000 7fb336a00640 1 -- 172.21.15.100:0/3693992006 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3301f3950 con 0x7fb330153980 2024-09-17T04:31:48.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.178+0000 7fb327e00640 1 -- 172.21.15.100:0/3693992006 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fb328004030 con 0x7fb330153980 2024-09-17T04:31:48.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.179+0000 7fb327e00640 1 --2- 172.21.15.100:0/3693992006 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb3100791f0 0x7fb31007b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:48.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.179+0000 7fb327e00640 1 -- 172.21.15.100:0/3693992006 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4999+0+0 (secure 0 0 0) 0x7fb32809bbc0 con 0x7fb330153980 2024-09-17T04:31:48.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.180+0000 7fb32fe00640 1 --2- 172.21.15.100:0/3693992006 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb3100791f0 0x7fb31007b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:48.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.180+0000 7fb32fe00640 1 --2- 172.21.15.100:0/3693992006 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb3100791f0 0x7fb31007b700 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb320000c00 tx=0x7fb320006b70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:48.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.184+0000 7fb327e00640 1 -- 172.21.15.100:0/3693992006 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb328063220 con 0x7fb330153980 2024-09-17T04:31:48.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.405+0000 7fb336a00640 1 -- 172.21.15.100:0/3693992006 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb330152d30 con 0x7fb330153980 2024-09-17T04:31:48.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.405+0000 7fb327e00640 1 -- 172.21.15.100:0/3693992006 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+3923 (secure 0 0 0) 0x7fb328067c50 con 0x7fb330153980 2024-09-17T04:31:48.406 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:48.406 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":85,"btime":"2024-09-17T04:31:40:990476+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6410,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1845046345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1845046345},{"type":"v1","addr":"172.21.15.184:6801","nonce":1845046345}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7135,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3970003794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3970003794},{"type":"v1","addr":"172.21.15.184:6835","nonce":3970003794}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/3663836388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3663836388},{"type":"v1","addr":"172.21.15.100:6834","nonce":3663836388}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2736874186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":2736874186},{"type":"v1","addr":"172.21.15.100:6835","nonce":2736874186}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[]} 2024-09-17T04:31:48.406 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 85 2024-09-17T04:31:48.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.408+0000 7fb336a00640 1 -- 172.21.15.100:0/3693992006 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb3100791f0 msgr2=0x7fb31007b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:48.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.408+0000 7fb336a00640 1 --2- 172.21.15.100:0/3693992006 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb3100791f0 0x7fb31007b700 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb320000c00 tx=0x7fb320006b70 comp rx=0 tx=0).stop 2024-09-17T04:31:48.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.408+0000 7fb336a00640 1 -- 172.21.15.100:0/3693992006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb330153980 msgr2=0x7fb3300b0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:48.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.408+0000 7fb336a00640 1 --2- 172.21.15.100:0/3693992006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb330153980 0x7fb3300b0df0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fb328005fd0 tx=0x7fb32800f3c0 comp rx=0 tx=0).stop 2024-09-17T04:31:48.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.408+0000 7fb336a00640 1 -- 172.21.15.100:0/3693992006 shutdown_connections 2024-09-17T04:31:48.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.408+0000 7fb336a00640 1 --2- 172.21.15.100:0/3693992006 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb3100791f0 0x7fb31007b700 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:48.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.408+0000 7fb336a00640 1 --2- 172.21.15.100:0/3693992006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb330153980 0x7fb3300b0df0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:48.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.408+0000 7fb336a00640 1 --2- 172.21.15.100:0/3693992006 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3300d0dc0 0x7fb330153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:48.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.408+0000 7fb336a00640 1 --2- 172.21.15.100:0/3693992006 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb314004800 0x7fb33014fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:48.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.408+0000 7fb336a00640 1 -- 172.21.15.100:0/3693992006 >> 172.21.15.100:0/3693992006 conn(0x7fb33006d730 msgr2=0x7fb33015ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:48.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.408+0000 7fb336a00640 1 -- 172.21.15.100:0/3693992006 shutdown_connections 2024-09-17T04:31:48.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.408+0000 7fb336a00640 1 -- 172.21.15.100:0/3693992006 wait complete. 2024-09-17T04:31:48.431 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T04:31:48.431 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T04:31:48.432 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T04:31:48.432 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:31:48.432 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T04:31:48.432 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T04:31:48.432 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T04:31:48.432 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:31:48.432 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T04:31:48.433 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T04:31:48.433 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T04:31:48.433 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:31:48.433 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T04:31:48.433 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T04:31:48.433 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T04:31:48.433 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:31:48.453 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T04:31:48.453 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T04:31:48.456 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T04:31:48.456 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T04:31:48.457 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T04:31:48.458 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T04:31:48.458 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T04:31:48.458 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T04:31:48.500 INFO:tasks.ceph.mds.c:Started 2024-09-17T04:31:48.501 DEBUG:teuthology.parallel:result is None 2024-09-17T04:31:48.502 INFO:tasks.ceph.mds.d:Started 2024-09-17T04:31:48.503 DEBUG:teuthology.parallel:result is None 2024-09-17T04:31:48.503 INFO:tasks.ceph.mds.a:Started 2024-09-17T04:31:48.504 DEBUG:teuthology.parallel:result is None 2024-09-17T04:31:48.505 INFO:tasks.ceph.mds.b:Started 2024-09-17T04:31:48.505 DEBUG:teuthology.parallel:result is None 2024-09-17T04:31:48.505 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T04:31:48.696 INFO:tasks.ceph.mds.a.smithi100.stdout:starting mds.a at 2024-09-17T04:31:48.697 INFO:tasks.ceph.mds.c.smithi100.stdout:starting mds.c at 2024-09-17T04:31:48.701 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-09-17T04:31:48.701 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-09-17T04:31:48.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.740+0000 7fe584600640 1 -- 172.21.15.100:0/1663164485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe57c0af6a0 msgr2=0x7fe57c0ad1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:48.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.740+0000 7fe584600640 1 --2- 172.21.15.100:0/1663164485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe57c0af6a0 0x7fe57c0ad1c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe568009830 tx=0x7fe56802a4a0 comp rx=0 tx=0).stop 2024-09-17T04:31:48.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.740+0000 7fe584600640 1 -- 172.21.15.100:0/1663164485 shutdown_connections 2024-09-17T04:31:48.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.740+0000 7fe584600640 1 --2- 172.21.15.100:0/1663164485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe57c0af6a0 0x7fe57c0ad1c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:48.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.740+0000 7fe584600640 1 -- 172.21.15.100:0/1663164485 >> 172.21.15.100:0/1663164485 conn(0x7fe57c06d3d0 msgr2=0x7fe57c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:48.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.740+0000 7fe584600640 1 -- 172.21.15.100:0/1663164485 shutdown_connections 2024-09-17T04:31:48.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.740+0000 7fe584600640 1 -- 172.21.15.100:0/1663164485 wait complete. 2024-09-17T04:31:48.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.742+0000 7fe584600640 1 Processor -- start 2024-09-17T04:31:48.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.743+0000 7fe584600640 1 -- start start 2024-09-17T04:31:48.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.744+0000 7fe584600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe57c070660 0x7fe57c159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:48.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.744+0000 7fe584600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe57c0ad700 0x7fe57c0b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:48.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.744+0000 7fe584600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe57c0af6a0 0x7fe57c0b3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:48.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.744+0000 7fe584600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c0cb950 con 0x7fe57c0af6a0 2024-09-17T04:31:48.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.744+0000 7fe584600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c0afab0 con 0x7fe57c070660 2024-09-17T04:31:48.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.744+0000 7fe584600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe57c0cbad0 con 0x7fe57c0ad700 2024-09-17T04:31:48.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.745+0000 7fe581200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe57c0ad700 0x7fe57c0b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:48.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.745+0000 7fe582600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe57c0af6a0 0x7fe57c0b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:48.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.745+0000 7fe581200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe57c0ad700 0x7fe57c0b31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.100:57962/0 (socket says 172.21.15.100:57962) 2024-09-17T04:31:48.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.745+0000 7fe581200640 1 -- 172.21.15.100:0/2090117919 learned_addr learned my addr 172.21.15.100:0/2090117919 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:48.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.745+0000 7fe581c00640 1 --2- 172.21.15.100:0/2090117919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe57c070660 0x7fe57c159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:48.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.745+0000 7fe582600640 1 -- 172.21.15.100:0/2090117919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe57c0ad700 msgr2=0x7fe57c0b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:48.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.745+0000 7fe582600640 1 --2- 172.21.15.100:0/2090117919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe57c0ad700 0x7fe57c0b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:48.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.745+0000 7fe582600640 1 -- 172.21.15.100:0/2090117919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe57c070660 msgr2=0x7fe57c159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:48.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.745+0000 7fe582600640 1 --2- 172.21.15.100:0/2090117919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe57c070660 0x7fe57c159920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:48.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.745+0000 7fe582600640 1 -- 172.21.15.100:0/2090117919 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57c0c6540 con 0x7fe57c0af6a0 2024-09-17T04:31:48.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.746+0000 7fe582600640 1 --2- 172.21.15.100:0/2090117919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe57c0af6a0 0x7fe57c0b3720 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fe57400c9f0 tx=0x7fe57400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:48.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.746+0000 7fe579600640 1 -- 172.21.15.100:0/2090117919 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574016020 con 0x7fe57c0af6a0 2024-09-17T04:31:48.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.746+0000 7fe579600640 1 -- 172.21.15.100:0/2090117919 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe57400ac60 con 0x7fe57c0af6a0 2024-09-17T04:31:48.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.746+0000 7fe579600640 1 -- 172.21.15.100:0/2090117919 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574012680 con 0x7fe57c0af6a0 2024-09-17T04:31:48.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.746+0000 7fe584600640 1 -- 172.21.15.100:0/2090117919 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe57c0c3ec0 con 0x7fe57c0af6a0 2024-09-17T04:31:48.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.746+0000 7fe584600640 1 -- 172.21.15.100:0/2090117919 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe57c0c4480 con 0x7fe57c0af6a0 2024-09-17T04:31:48.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.747+0000 7fe584600640 1 -- 172.21.15.100:0/2090117919 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe57c0add90 con 0x7fe57c0af6a0 2024-09-17T04:31:48.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.748+0000 7fe579600640 1 -- 172.21.15.100:0/2090117919 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fe574019020 con 0x7fe57c0af6a0 2024-09-17T04:31:48.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.749+0000 7fe579600640 1 --2- 172.21.15.100:0/2090117919 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe55c0791c0 0x7fe55c07b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:48.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.749+0000 7fe579600640 1 -- 172.21.15.100:0/2090117919 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 4999+0+0 (secure 0 0 0) 0x7fe57409b8e0 con 0x7fe57c0af6a0 2024-09-17T04:31:48.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.749+0000 7fe581c00640 1 --2- 172.21.15.100:0/2090117919 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe55c0791c0 0x7fe55c07b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:48.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.750+0000 7fe581c00640 1 --2- 172.21.15.100:0/2090117919 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe55c0791c0 0x7fe55c07b6d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe57c072000 tx=0x7fe570009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:48.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.753+0000 7fe579600640 1 -- 172.21.15.100:0/2090117919 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe574020030 con 0x7fe57c0af6a0 2024-09-17T04:31:48.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:48.950+0000 7fe584600640 1 -- 172.21.15.100:0/2090117919 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe57c0c33d0 con 0x7fe57c0af6a0 2024-09-17T04:31:49.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.742+0000 7fe579600640 1 -- 172.21.15.100:0/2090117919 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v128) ==== 97+0+0 (secure 0 0 0) 0x7fe574062e80 con 0x7fe57c0af6a0 2024-09-17T04:31:49.743 INFO:teuthology.orchestra.run.smithi100.stderr: removed all blocklist entries 2024-09-17T04:31:49.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.745+0000 7fe584600640 1 -- 172.21.15.100:0/2090117919 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe55c0791c0 msgr2=0x7fe55c07b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:49.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.745+0000 7fe584600640 1 --2- 172.21.15.100:0/2090117919 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe55c0791c0 0x7fe55c07b6d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe57c072000 tx=0x7fe570009040 comp rx=0 tx=0).stop 2024-09-17T04:31:49.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.745+0000 7fe584600640 1 -- 172.21.15.100:0/2090117919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe57c0af6a0 msgr2=0x7fe57c0b3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:49.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.745+0000 7fe584600640 1 --2- 172.21.15.100:0/2090117919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe57c0af6a0 0x7fe57c0b3720 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fe57400c9f0 tx=0x7fe57400ceb0 comp rx=0 tx=0).stop 2024-09-17T04:31:49.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.745+0000 7fe584600640 1 -- 172.21.15.100:0/2090117919 shutdown_connections 2024-09-17T04:31:49.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.745+0000 7fe584600640 1 --2- 172.21.15.100:0/2090117919 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe55c0791c0 0x7fe55c07b6d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:49.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.745+0000 7fe584600640 1 --2- 172.21.15.100:0/2090117919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe57c0af6a0 0x7fe57c0b3720 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:49.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.745+0000 7fe584600640 1 --2- 172.21.15.100:0/2090117919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe57c0ad700 0x7fe57c0b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:49.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.745+0000 7fe584600640 1 --2- 172.21.15.100:0/2090117919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe57c070660 0x7fe57c159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:49.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.745+0000 7fe584600640 1 -- 172.21.15.100:0/2090117919 >> 172.21.15.100:0/2090117919 conn(0x7fe57c06d3d0 msgr2=0x7fe57c0b4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:49.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.746+0000 7fe584600640 1 -- 172.21.15.100:0/2090117919 shutdown_connections 2024-09-17T04:31:49.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.746+0000 7fe584600640 1 -- 172.21.15.100:0/2090117919 wait complete. 2024-09-17T04:31:49.772 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T04:31:49.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.995+0000 7fd375800640 1 -- 172.21.15.100:0/222985922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd354003a50 msgr2=0x7fd354003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:49.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.995+0000 7fd375800640 1 --2- 172.21.15.100:0/222985922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd354003a50 0x7fd354003ec0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fd360009b50 tx=0x7fd354005830 comp rx=0 tx=0).stop 2024-09-17T04:31:49.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.995+0000 7fd36f400640 1 -- 172.21.15.100:0/222985922 reap_dead start 2024-09-17T04:31:49.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.995+0000 7fd375800640 1 -- 172.21.15.100:0/222985922 shutdown_connections 2024-09-17T04:31:49.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.995+0000 7fd375800640 1 -- 172.21.15.100:0/222985922 >> 172.21.15.100:0/222985922 conn(0x7fd37006d730 msgr2=0x7fd3700b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:49.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.996+0000 7fd375800640 1 -- 172.21.15.100:0/222985922 shutdown_connections 2024-09-17T04:31:49.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.996+0000 7fd375800640 1 -- 172.21.15.100:0/222985922 wait complete. 2024-09-17T04:31:49.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.998+0000 7fd375800640 1 Processor -- start 2024-09-17T04:31:49.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.998+0000 7fd375800640 1 -- start start 2024-09-17T04:31:50.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.999+0000 7fd375800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd354003a50 0x7fd37014d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:50.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.999+0000 7fd375800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3701564a0 0x7fd37014f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:50.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.999+0000 7fd375800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd37014f730 0x7fd3700b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:50.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.999+0000 7fd375800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd370152920 con 0x7fd3701564a0 2024-09-17T04:31:50.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.999+0000 7fd375800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd3700b4970 con 0x7fd37014f730 2024-09-17T04:31:50.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.999+0000 7fd375800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd370152aa0 con 0x7fd354003a50 2024-09-17T04:31:50.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:49.999+0000 7fd367e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3701564a0 0x7fd37014f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:50.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.000+0000 7fd367e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3701564a0 0x7fd37014f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34382/0 (socket says 172.21.15.100:34382) 2024-09-17T04:31:50.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.000+0000 7fd367e00640 1 -- 172.21.15.100:0/1320482026 learned_addr learned my addr 172.21.15.100:0/1320482026 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:50.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.000+0000 7fd36ea00640 1 --2- 172.21.15.100:0/1320482026 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd354003a50 0x7fd37014d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:50.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.000+0000 7fd36f400640 1 --2- 172.21.15.100:0/1320482026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd37014f730 0x7fd3700b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:50.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.000+0000 7fd367e00640 1 -- 172.21.15.100:0/1320482026 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd354003a50 msgr2=0x7fd37014d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:50.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.000+0000 7fd367e00640 1 --2- 172.21.15.100:0/1320482026 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd354003a50 0x7fd37014d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:50.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.000+0000 7fd367e00640 1 -- 172.21.15.100:0/1320482026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd37014f730 msgr2=0x7fd3700b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:50.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.000+0000 7fd367e00640 1 --2- 172.21.15.100:0/1320482026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd37014f730 0x7fd3700b0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:50.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.000+0000 7fd367e00640 1 -- 172.21.15.100:0/1320482026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3700afd10 con 0x7fd3701564a0 2024-09-17T04:31:50.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.000+0000 7fd367e00640 1 --2- 172.21.15.100:0/1320482026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3701564a0 0x7fd37014f1f0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fd358011a50 tx=0x7fd358011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:50.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.000+0000 7fd366a00640 1 -- 172.21.15.100:0/1320482026 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35801f070 con 0x7fd3701564a0 2024-09-17T04:31:50.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.000+0000 7fd366a00640 1 -- 172.21.15.100:0/1320482026 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd35800fd60 con 0x7fd3701564a0 2024-09-17T04:31:50.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.000+0000 7fd375800640 1 -- 172.21.15.100:0/1320482026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd37014fb40 con 0x7fd3701564a0 2024-09-17T04:31:50.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.001+0000 7fd366a00640 1 -- 172.21.15.100:0/1320482026 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3580126b0 con 0x7fd3701564a0 2024-09-17T04:31:50.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.001+0000 7fd375800640 1 -- 172.21.15.100:0/1320482026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3700aff80 con 0x7fd3701564a0 2024-09-17T04:31:50.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.002+0000 7fd375800640 1 -- 172.21.15.100:0/1320482026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd370069920 con 0x7fd3701564a0 2024-09-17T04:31:50.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.003+0000 7fd366a00640 1 -- 172.21.15.100:0/1320482026 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fd358022070 con 0x7fd3701564a0 2024-09-17T04:31:50.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.004+0000 7fd366a00640 1 --2- 172.21.15.100:0/1320482026 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd34c0791f0 0x7fd34c07b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:50.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.004+0000 7fd366a00640 1 -- 172.21.15.100:0/1320482026 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4913+0+0 (secure 0 0 0) 0x7fd35809bf00 con 0x7fd3701564a0 2024-09-17T04:31:50.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.004+0000 7fd36ea00640 1 --2- 172.21.15.100:0/1320482026 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd34c0791f0 0x7fd34c07b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:50.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.005+0000 7fd36ea00640 1 --2- 172.21.15.100:0/1320482026 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd34c0791f0 0x7fd34c07b700 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd360035000 tx=0x7fd36002d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:50.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.009+0000 7fd366a00640 1 -- 172.21.15.100:0/1320482026 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd370069920 con 0x7fd3701564a0 2024-09-17T04:31:50.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.260+0000 7fd375800640 1 -- 172.21.15.100:0/1320482026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fd370157850 con 0x7fd3701564a0 2024-09-17T04:31:50.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.261+0000 7fd366a00640 1 -- 172.21.15.100:0/1320482026 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) ==== 80+0+5304 (secure 0 0 0) 0x7fd37014fb40 con 0x7fd3701564a0 2024-09-17T04:31:50.261 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:50.262 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:31:50.262 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_dump": [ 2024-09-17T04:31:50.262 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:50.262 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.a", 2024-09-17T04:31:50.262 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmV8asJRAAkSgIWqGc1OeSRbs9PEOT8w==", 2024-09-17T04:31:50.262 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:50.262 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:31:50.262 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:31:50.262 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:31:50.262 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:50.262 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:50.263 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:50.263 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:50.263 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.b", 2024-09-17T04:31:50.263 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmpM7ZBRAA1Og785tJwa6NA9zBdmNNQA==", 2024-09-17T04:31:50.263 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:50.263 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:31:50.263 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:31:50.263 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:31:50.263 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:50.263 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:50.263 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:50.263 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:50.263 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.c", 2024-09-17T04:31:50.263 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmXaOGMxAAT+96g/5fAzPW8Cc2HtQUyA==", 2024-09-17T04:31:50.264 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:50.264 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:31:50.264 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:31:50.264 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:31:50.264 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:50.264 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:50.264 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:50.264 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:50.264 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.d", 2024-09-17T04:31:50.264 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmRqC9ExAA23kZ0Ks1lGN2k4GXL8ASrg==", 2024-09-17T04:31:50.264 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:50.264 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:31:50.264 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:31:50.264 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:31:50.265 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:50.265 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:50.265 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:50.265 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:50.265 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.0", 2024-09-17T04:31:50.265 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAwBOlmofFmAxAARerYkU9B2a9MM9vHa3o4LQ==", 2024-09-17T04:31:50.265 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:50.265 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:31:50.265 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:31:50.265 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:50.265 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:50.265 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:50.265 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:50.266 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.1", 2024-09-17T04:31:50.266 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAzBOlmbz6VExAAoVqSqXTc1wArKhdA6+NxnA==", 2024-09-17T04:31:50.266 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:50.266 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:31:50.266 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:31:50.266 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:50.266 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:50.266 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:50.266 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:50.266 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.2", 2024-09-17T04:31:50.266 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA2BOlmfeSAJBAAchY6XhyDRusc+zqwQ6v6wQ==", 2024-09-17T04:31:50.266 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:50.266 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:31:50.266 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:31:50.266 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:50.267 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:50.267 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:50.267 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:50.267 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.3", 2024-09-17T04:31:50.267 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA6BOlm/TQiBBAAHhbs2+RCjAenGR1frhbTmQ==", 2024-09-17T04:31:50.267 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:50.267 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:31:50.267 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:31:50.267 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:50.267 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:50.267 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:50.267 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:50.267 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.4", 2024-09-17T04:31:50.267 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBABOlm7OKvBhAALkaAKEUYN5ChwGejoSF8yA==", 2024-09-17T04:31:50.268 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:50.268 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:31:50.268 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:31:50.268 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:50.268 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:50.268 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:50.268 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:50.268 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.5", 2024-09-17T04:31:50.268 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBDBOlmQpi1FhAAbA47dg6ckK1PFoCZL0SBEQ==", 2024-09-17T04:31:50.268 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:50.268 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:31:50.268 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:31:50.268 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:50.269 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:50.269 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:50.269 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:50.269 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.6", 2024-09-17T04:31:50.269 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBGBOlmx/w0JhAAWcPvZkMYtXD/EMKrg/W/4Q==", 2024-09-17T04:31:50.269 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:50.269 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:31:50.269 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:31:50.269 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:50.269 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:50.269 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:50.269 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:50.269 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.7", 2024-09-17T04:31:50.269 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBJBOlmWHjWNBAAgdul2UKY9g5p7EVi0ZdrFw==", 2024-09-17T04:31:50.270 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:50.270 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:31:50.270 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:31:50.270 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:50.270 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:50.270 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:50.270 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:50.270 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.0", 2024-09-17T04:31:50.270 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmfYg9IBAABP1uRKGOugrA47VxwRLkjA==", 2024-09-17T04:31:50.270 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:50.270 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:31:50.270 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow r", 2024-09-17T04:31:50.270 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T04:31:50.270 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:50.270 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:50.271 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:50.271 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.1", 2024-09-17T04:31:50.271 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmByhhNRAAPGaxFCDd8ywFVwwhprrPbQ==", 2024-09-17T04:31:50.271 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:50.271 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:31:50.271 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:31:50.271 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:31:50.271 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:31:50.271 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:50.271 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:50.271 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:50.271 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.2", 2024-09-17T04:31:50.271 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmgcm0KhAAGF06VlHjBM/AGxafN6/3aA==", 2024-09-17T04:31:50.272 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:50.272 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:31:50.272 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:31:50.272 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:31:50.272 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:31:50.272 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:50.272 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:50.272 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:50.272 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.3", 2024-09-17T04:31:50.272 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtBOlm9VQ6BBAAXcahu35/uDm8Z+f+3SF0Gg==", 2024-09-17T04:31:50.272 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:50.272 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:31:50.272 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:31:50.272 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:31:50.273 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:31:50.273 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:50.273 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:50.273 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:50.273 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.admin", 2024-09-17T04:31:50.273 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAqBOlmZ+bIJBAAG3oaMNlYWNakywGDvAGbGw==", 2024-09-17T04:31:50.273 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:50.273 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:31:50.273 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:31:50.273 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:31:50.273 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:50.273 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:50.273 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:50.273 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:50.273 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mgr.x", 2024-09-17T04:31:50.274 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmfaL7GhAArqLH66RkbBsz+TNOKtPHOA==", 2024-09-17T04:31:50.274 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:50.274 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:31:50.274 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile mgr", 2024-09-17T04:31:50.274 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:50.274 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:50.274 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:31:50.274 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:50.274 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mgr.y", 2024-09-17T04:31:50.274 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmM4D1DhAA5uUWgRVOMujWqbYzOjFp9A==", 2024-09-17T04:31:50.274 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:31:50.274 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:31:50.274 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile mgr", 2024-09-17T04:31:50.274 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:31:50.275 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:50.275 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:50.275 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-17T04:31:50.275 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:31:50.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.263+0000 7fd375800640 1 -- 172.21.15.100:0/1320482026 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd34c0791f0 msgr2=0x7fd34c07b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:50.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.263+0000 7fd375800640 1 --2- 172.21.15.100:0/1320482026 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd34c0791f0 0x7fd34c07b700 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd360035000 tx=0x7fd36002d040 comp rx=0 tx=0).stop 2024-09-17T04:31:50.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.263+0000 7fd375800640 1 -- 172.21.15.100:0/1320482026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3701564a0 msgr2=0x7fd37014f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:50.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.263+0000 7fd375800640 1 --2- 172.21.15.100:0/1320482026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3701564a0 0x7fd37014f1f0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fd358011a50 tx=0x7fd358011f10 comp rx=0 tx=0).stop 2024-09-17T04:31:50.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.264+0000 7fd375800640 1 -- 172.21.15.100:0/1320482026 shutdown_connections 2024-09-17T04:31:50.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.264+0000 7fd375800640 1 --2- 172.21.15.100:0/1320482026 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd34c0791f0 0x7fd34c07b700 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:50.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.264+0000 7fd375800640 1 --2- 172.21.15.100:0/1320482026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd37014f730 0x7fd3700b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:50.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.264+0000 7fd375800640 1 --2- 172.21.15.100:0/1320482026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3701564a0 0x7fd37014f1f0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:50.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.264+0000 7fd375800640 1 --2- 172.21.15.100:0/1320482026 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd354003a50 0x7fd37014d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:50.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.264+0000 7fd375800640 1 -- 172.21.15.100:0/1320482026 >> 172.21.15.100:0/1320482026 conn(0x7fd37006d730 msgr2=0x7fd370073370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:50.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.264+0000 7fd375800640 1 -- 172.21.15.100:0/1320482026 shutdown_connections 2024-09-17T04:31:50.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.264+0000 7fd375800640 1 -- 172.21.15.100:0/1320482026 wait complete. 2024-09-17T04:31:50.286 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T04:31:50.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.502+0000 7f4a7ea00640 1 -- 172.21.15.100:0/1357511864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a78160310 msgr2=0x7f4a78160720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:50.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.502+0000 7f4a7ea00640 1 --2- 172.21.15.100:0/1357511864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a78160310 0x7f4a78160720 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f4a78075250 tx=0x7f4a7002a980 comp rx=0 tx=0).stop 2024-09-17T04:31:50.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.502+0000 7f4a7ea00640 1 -- 172.21.15.100:0/1357511864 shutdown_connections 2024-09-17T04:31:50.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.502+0000 7f4a7ea00640 1 --2- 172.21.15.100:0/1357511864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a78160310 0x7f4a78160720 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:50.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.502+0000 7f4a7ea00640 1 --2- 172.21.15.100:0/1357511864 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a780ad0e0 0x7f4a780ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:50.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.502+0000 7f4a7ea00640 1 --2- 172.21.15.100:0/1357511864 >> v2:172.21.15.184:3300/0 conn(0x7f4a7815d760 0x7f4a781535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:50.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.502+0000 7f4a7ea00640 1 -- 172.21.15.100:0/1357511864 >> 172.21.15.100:0/1357511864 conn(0x7f4a7806d730 msgr2=0x7f4a780b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:50.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.503+0000 7f4a7ea00640 1 -- 172.21.15.100:0/1357511864 shutdown_connections 2024-09-17T04:31:50.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.503+0000 7f4a7ea00640 1 -- 172.21.15.100:0/1357511864 wait complete. 2024-09-17T04:31:50.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.505+0000 7f4a7ea00640 1 Processor -- start 2024-09-17T04:31:50.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.505+0000 7f4a7ea00640 1 -- start start 2024-09-17T04:31:50.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.506+0000 7f4a7ea00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a780ad0e0 0x7f4a78155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:50.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.506+0000 7f4a7ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a7815d760 0x7f4a7809dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:50.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.506+0000 7f4a7ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a78160310 0x7f4a781f4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:50.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.506+0000 7f4a7ea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4a780b4da0 con 0x7f4a7815d760 2024-09-17T04:31:50.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.506+0000 7f4a7ea00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4a780b4c20 con 0x7f4a78160310 2024-09-17T04:31:50.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.506+0000 7f4a7ea00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4a78160c60 con 0x7f4a780ad0e0 2024-09-17T04:31:50.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.506+0000 7f4a77400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a7815d760 0x7f4a7809dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:50.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.507+0000 7f4a77400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a7815d760 0x7f4a7809dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34398/0 (socket says 172.21.15.100:34398) 2024-09-17T04:31:50.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.507+0000 7f4a77400640 1 -- 172.21.15.100:0/732122177 learned_addr learned my addr 172.21.15.100:0/732122177 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:50.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.507+0000 7f4a77e00640 1 --2- 172.21.15.100:0/732122177 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a780ad0e0 0x7f4a78155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:50.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.507+0000 7f4a7ca00640 1 --2- 172.21.15.100:0/732122177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a78160310 0x7f4a781f4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:50.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.507+0000 7f4a77400640 1 -- 172.21.15.100:0/732122177 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a780ad0e0 msgr2=0x7f4a78155c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:50.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.507+0000 7f4a77400640 1 --2- 172.21.15.100:0/732122177 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a780ad0e0 0x7f4a78155c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:50.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.507+0000 7f4a77400640 1 -- 172.21.15.100:0/732122177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a78160310 msgr2=0x7f4a781f4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:50.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.507+0000 7f4a77400640 1 --2- 172.21.15.100:0/732122177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a78160310 0x7f4a781f4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:50.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.507+0000 7f4a77400640 1 -- 172.21.15.100:0/732122177 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a781566f0 con 0x7f4a7815d760 2024-09-17T04:31:50.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.507+0000 7f4a77400640 1 --2- 172.21.15.100:0/732122177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a7815d760 0x7f4a7809dab0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400d700 tx=0x7f4a6400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:50.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.508+0000 7f4a6fe00640 1 -- 172.21.15.100:0/732122177 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a64017070 con 0x7f4a7815d760 2024-09-17T04:31:50.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.508+0000 7f4a6fe00640 1 -- 172.21.15.100:0/732122177 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a64004310 con 0x7f4a7815d760 2024-09-17T04:31:50.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.508+0000 7f4a6fe00640 1 -- 172.21.15.100:0/732122177 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a64009a80 con 0x7f4a7815d760 2024-09-17T04:31:50.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.508+0000 7f4a7ea00640 1 -- 172.21.15.100:0/732122177 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a781f8dc0 con 0x7f4a7815d760 2024-09-17T04:31:50.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.508+0000 7f4a7ea00640 1 -- 172.21.15.100:0/732122177 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a781f9380 con 0x7f4a7815d760 2024-09-17T04:31:50.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.509+0000 7f4a7ea00640 1 -- 172.21.15.100:0/732122177 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a78156460 con 0x7f4a7815d760 2024-09-17T04:31:50.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.510+0000 7f4a6fe00640 1 -- 172.21.15.100:0/732122177 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f4a64009c20 con 0x7f4a7815d760 2024-09-17T04:31:50.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.511+0000 7f4a6fe00640 1 --2- 172.21.15.100:0/732122177 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4a580791c0 0x7f4a5807b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:50.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.511+0000 7f4a6fe00640 1 -- 172.21.15.100:0/732122177 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4913+0+0 (secure 0 0 0) 0x7f4a6409b630 con 0x7f4a7815d760 2024-09-17T04:31:50.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.511+0000 7f4a77e00640 1 --2- 172.21.15.100:0/732122177 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4a580791c0 0x7f4a5807b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:50.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.512+0000 7f4a77e00640 1 --2- 172.21.15.100:0/732122177 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4a580791c0 0x7f4a5807b6d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800bad0 tx=0x7f4a6800b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:50.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.515+0000 7f4a6fe00640 1 -- 172.21.15.100:0/732122177 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4a64062d70 con 0x7f4a7815d760 2024-09-17T04:31:50.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.703+0000 7f4a7ea00640 1 -- 172.21.15.100:0/732122177 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f4a781535c0 con 0x7f4a7815d760 2024-09-17T04:31:50.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.703+0000 7f4a6fe00640 1 -- 172.21.15.100:0/732122177 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v128) ==== 84+0+65 (secure 0 0 0) 0x7f4a640677a0 con 0x7f4a7815d760 2024-09-17T04:31:50.704 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:50.704 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-09-17T04:31:50.704 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:31:50.704 INFO:teuthology.orchestra.run.smithi100.stdout: "poolnum": 1, 2024-09-17T04:31:50.704 INFO:teuthology.orchestra.run.smithi100.stdout: "poolname": ".mgr" 2024-09-17T04:31:50.704 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:31:50.704 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-09-17T04:31:50.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.706+0000 7f4a7ea00640 1 -- 172.21.15.100:0/732122177 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4a580791c0 msgr2=0x7f4a5807b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:50.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.706+0000 7f4a7ea00640 1 --2- 172.21.15.100:0/732122177 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4a580791c0 0x7f4a5807b6d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800bad0 tx=0x7f4a6800b2e0 comp rx=0 tx=0).stop 2024-09-17T04:31:50.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.706+0000 7f4a7ea00640 1 -- 172.21.15.100:0/732122177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a7815d760 msgr2=0x7f4a7809dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:50.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.706+0000 7f4a7ea00640 1 --2- 172.21.15.100:0/732122177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a7815d760 0x7f4a7809dab0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400d700 tx=0x7f4a6400dbc0 comp rx=0 tx=0).stop 2024-09-17T04:31:50.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.706+0000 7f4a7ea00640 1 -- 172.21.15.100:0/732122177 shutdown_connections 2024-09-17T04:31:50.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.706+0000 7f4a7ea00640 1 --2- 172.21.15.100:0/732122177 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4a580791c0 0x7f4a5807b6d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:50.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.706+0000 7f4a7ea00640 1 --2- 172.21.15.100:0/732122177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a78160310 0x7f4a781f4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:50.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.706+0000 7f4a7ea00640 1 --2- 172.21.15.100:0/732122177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a7815d760 0x7f4a7809dab0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:50.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.706+0000 7f4a7ea00640 1 --2- 172.21.15.100:0/732122177 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a780ad0e0 0x7f4a78155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:50.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.706+0000 7f4a7ea00640 1 -- 172.21.15.100:0/732122177 >> 172.21.15.100:0/732122177 conn(0x7f4a7806d730 msgr2=0x7f4a7815b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:50.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.706+0000 7f4a7ea00640 1 -- 172.21.15.100:0/732122177 shutdown_connections 2024-09-17T04:31:50.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.706+0000 7f4a7ea00640 1 -- 172.21.15.100:0/732122177 wait complete. 2024-09-17T04:31:50.728 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T04:31:50.728 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T04:31:50.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.954+0000 7f5243400640 1 -- 172.21.15.100:0/226361011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f523c06cda0 msgr2=0x7f523c06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:50.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.954+0000 7f5243400640 1 --2- 172.21.15.100:0/226361011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f523c06cda0 0x7f523c06d1b0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f523c06d8a0 tx=0x7f522802a580 comp rx=0 tx=0).stop 2024-09-17T04:31:50.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.954+0000 7f5243400640 1 -- 172.21.15.100:0/226361011 shutdown_connections 2024-09-17T04:31:50.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.954+0000 7f5243400640 1 --2- 172.21.15.100:0/226361011 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f523c0af100 0x7f523c0af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:50.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.954+0000 7f5243400640 1 --2- 172.21.15.100:0/226361011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f523c06cda0 0x7f523c06d1b0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:50.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.954+0000 7f5243400640 1 --2- 172.21.15.100:0/226361011 >> v2:172.21.15.184:3300/0 conn(0x7f523c0cc830 0x7f523c0b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:50.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.954+0000 7f5243400640 1 -- 172.21.15.100:0/226361011 >> 172.21.15.100:0/226361011 conn(0x7f523c0b2540 msgr2=0x7f523c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:50.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.954+0000 7f5243400640 1 -- 172.21.15.100:0/226361011 shutdown_connections 2024-09-17T04:31:50.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.954+0000 7f5243400640 1 -- 172.21.15.100:0/226361011 wait complete. 2024-09-17T04:31:50.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.956+0000 7f5243400640 1 Processor -- start 2024-09-17T04:31:50.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.956+0000 7f5243400640 1 -- start start 2024-09-17T04:31:50.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.958+0000 7f5243400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f523c06cda0 0x7f523c15d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:50.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.958+0000 7f5243400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f523c0af100 0x7f523c15dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:50.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.958+0000 7f5243400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f523c0cc830 0x7f523c1f7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:50.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.958+0000 7f5243400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f523c0b34b0 con 0x7f523c0af100 2024-09-17T04:31:50.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.958+0000 7f5243400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f523c0b3330 con 0x7f523c0cc830 2024-09-17T04:31:50.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.958+0000 7f5243400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f523c0b3630 con 0x7f523c06cda0 2024-09-17T04:31:50.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.958+0000 7f523be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f523c0af100 0x7f523c15dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:50.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.958+0000 7f523be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f523c0af100 0x7f523c15dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34424/0 (socket says 172.21.15.100:34424) 2024-09-17T04:31:50.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.958+0000 7f523be00640 1 -- 172.21.15.100:0/1870505180 learned_addr learned my addr 172.21.15.100:0/1870505180 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:50.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.958+0000 7f5240a00640 1 --2- 172.21.15.100:0/1870505180 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f523c06cda0 0x7f523c15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:50.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.958+0000 7f5241400640 1 --2- 172.21.15.100:0/1870505180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f523c0cc830 0x7f523c1f7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:50.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.959+0000 7f523be00640 1 -- 172.21.15.100:0/1870505180 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f523c06cda0 msgr2=0x7f523c15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:50.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.959+0000 7f523be00640 1 --2- 172.21.15.100:0/1870505180 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f523c06cda0 0x7f523c15d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:50.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.959+0000 7f523be00640 1 -- 172.21.15.100:0/1870505180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f523c0cc830 msgr2=0x7f523c1f7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:50.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.959+0000 7f523be00640 1 --2- 172.21.15.100:0/1870505180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f523c0cc830 0x7f523c1f7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:50.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.959+0000 7f523be00640 1 -- 172.21.15.100:0/1870505180 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523c15fb40 con 0x7f523c0af100 2024-09-17T04:31:50.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.959+0000 7f523be00640 1 --2- 172.21.15.100:0/1870505180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f523c0af100 0x7f523c15dda0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f5228034040 tx=0x7f52280038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:50.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.959+0000 7f5238c00640 1 -- 172.21.15.100:0/1870505180 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522803e030 con 0x7f523c0af100 2024-09-17T04:31:50.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.959+0000 7f5238c00640 1 -- 172.21.15.100:0/1870505180 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5228003d90 con 0x7f523c0af100 2024-09-17T04:31:50.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.959+0000 7f5243400640 1 -- 172.21.15.100:0/1870505180 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f523c15caa0 con 0x7f523c0af100 2024-09-17T04:31:50.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.959+0000 7f5238c00640 1 -- 172.21.15.100:0/1870505180 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5228035040 con 0x7f523c0af100 2024-09-17T04:31:50.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.959+0000 7f5243400640 1 -- 172.21.15.100:0/1870505180 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f523c15d060 con 0x7f523c0af100 2024-09-17T04:31:50.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.960+0000 7f5243400640 1 -- 172.21.15.100:0/1870505180 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f523c0af510 con 0x7f523c0af100 2024-09-17T04:31:50.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.962+0000 7f5238c00640 1 -- 172.21.15.100:0/1870505180 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f5228041030 con 0x7f523c0af100 2024-09-17T04:31:50.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.963+0000 7f5238c00640 1 --2- 172.21.15.100:0/1870505180 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f521c0791c0 0x7f521c07b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:50.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.963+0000 7f5238c00640 1 -- 172.21.15.100:0/1870505180 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4913+0+0 (secure 0 0 0) 0x7f52280bbc60 con 0x7f523c0af100 2024-09-17T04:31:50.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.963+0000 7f5240a00640 1 --2- 172.21.15.100:0/1870505180 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f521c0791c0 0x7f521c07b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:50.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.964+0000 7f5240a00640 1 --2- 172.21.15.100:0/1870505180 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f521c0791c0 0x7f521c07b6d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f522c00bb50 tx=0x7f522c00b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:50.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:50.967+0000 7f5238c00640 1 -- 172.21.15.100:0/1870505180 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f5228083480 con 0x7f523c0af100 2024-09-17T04:31:51.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:51.151+0000 7f5243400640 1 -- 172.21.15.100:0/1870505180 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f523c15c710 con 0x7f523c0af100 2024-09-17T04:31:51.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:51.784+0000 7f5238c00640 1 -- 172.21.15.100:0/1870505180 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v129) ==== 138+0+0 (secure 0 0 0) 0x7f5228087eb0 con 0x7f523c0af100 2024-09-17T04:31:51.785 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' created 2024-09-17T04:31:51.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:51.788+0000 7f5243400640 1 -- 172.21.15.100:0/1870505180 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f521c0791c0 msgr2=0x7f521c07b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:51.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:51.788+0000 7f5243400640 1 --2- 172.21.15.100:0/1870505180 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f521c0791c0 0x7f521c07b6d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f522c00bb50 tx=0x7f522c00b2e0 comp rx=0 tx=0).stop 2024-09-17T04:31:51.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:51.788+0000 7f5243400640 1 -- 172.21.15.100:0/1870505180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f523c0af100 msgr2=0x7f523c15dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:51.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:51.788+0000 7f5243400640 1 --2- 172.21.15.100:0/1870505180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f523c0af100 0x7f523c15dda0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f5228034040 tx=0x7f52280038a0 comp rx=0 tx=0).stop 2024-09-17T04:31:51.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:51.788+0000 7f5243400640 1 -- 172.21.15.100:0/1870505180 shutdown_connections 2024-09-17T04:31:51.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:51.788+0000 7f5243400640 1 --2- 172.21.15.100:0/1870505180 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f521c0791c0 0x7f521c07b6d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:51.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:51.788+0000 7f5243400640 1 --2- 172.21.15.100:0/1870505180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f523c0cc830 0x7f523c1f7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:51.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:51.788+0000 7f5243400640 1 --2- 172.21.15.100:0/1870505180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f523c0af100 0x7f523c15dda0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:51.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:51.788+0000 7f5243400640 1 --2- 172.21.15.100:0/1870505180 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f523c06cda0 0x7f523c15d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:51.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:51.788+0000 7f5243400640 1 -- 172.21.15.100:0/1870505180 >> 172.21.15.100:0/1870505180 conn(0x7f523c0b2540 msgr2=0x7f523c0b0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:51.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:51.788+0000 7f5243400640 1 -- 172.21.15.100:0/1870505180 shutdown_connections 2024-09-17T04:31:51.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:51.788+0000 7f5243400640 1 -- 172.21.15.100:0/1870505180 wait complete. 2024-09-17T04:31:51.818 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T04:31:52.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.047+0000 7fa0d3000640 1 -- 172.21.15.100:0/294146488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0cc145450 msgr2=0x7fa0cc06f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:52.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.047+0000 7fa0d3000640 1 --2- 172.21.15.100:0/294146488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0cc145450 0x7fa0cc06f180 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009830 tx=0x7fa0bc02a530 comp rx=0 tx=0).stop 2024-09-17T04:31:52.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.047+0000 7fa0d3000640 1 -- 172.21.15.100:0/294146488 shutdown_connections 2024-09-17T04:31:52.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.047+0000 7fa0d3000640 1 --2- 172.21.15.100:0/294146488 >> v2:172.21.15.184:3300/0 conn(0x7fa0cc0735e0 0x7fa0cc0739f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:52.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.047+0000 7fa0d3000640 1 --2- 172.21.15.100:0/294146488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0cc145450 0x7fa0cc06f180 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:52.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.047+0000 7fa0d3000640 1 -- 172.21.15.100:0/294146488 >> 172.21.15.100:0/294146488 conn(0x7fa0cc06d1c0 msgr2=0x7fa0cc06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:52.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.048+0000 7fa0d3000640 1 -- 172.21.15.100:0/294146488 shutdown_connections 2024-09-17T04:31:52.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.048+0000 7fa0d3000640 1 -- 172.21.15.100:0/294146488 wait complete. 2024-09-17T04:31:52.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.050+0000 7fa0d3000640 1 Processor -- start 2024-09-17T04:31:52.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.050+0000 7fa0d3000640 1 -- start start 2024-09-17T04:31:52.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.051+0000 7fa0d3000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0cc0735e0 0x7fa0cc15b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:52.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.051+0000 7fa0d3000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0cc145450 0x7fa0cc0aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:52.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.051+0000 7fa0d3000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0cc160320 0x7fa0cc0af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:52.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.051+0000 7fa0d3000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc0ce620 con 0x7fa0cc0735e0 2024-09-17T04:31:52.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.051+0000 7fa0d3000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc071020 con 0x7fa0cc160320 2024-09-17T04:31:52.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.051+0000 7fa0d3000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa0cc0ce7a0 con 0x7fa0cc145450 2024-09-17T04:31:52.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.052+0000 7fa0cbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0cc0735e0 0x7fa0cc15b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:52.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.052+0000 7fa0cbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0cc0735e0 0x7fa0cc15b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34430/0 (socket says 172.21.15.100:34430) 2024-09-17T04:31:52.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.052+0000 7fa0cbe00640 1 -- 172.21.15.100:0/2752481883 learned_addr learned my addr 172.21.15.100:0/2752481883 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:52.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.052+0000 7fa0d1000640 1 --2- 172.21.15.100:0/2752481883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0cc160320 0x7fa0cc0af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:52.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.052+0000 7fa0c3400640 1 --2- 172.21.15.100:0/2752481883 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0cc145450 0x7fa0cc0aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:52.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.052+0000 7fa0cbe00640 1 -- 172.21.15.100:0/2752481883 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0cc145450 msgr2=0x7fa0cc0aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:52.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.052+0000 7fa0cbe00640 1 --2- 172.21.15.100:0/2752481883 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0cc145450 0x7fa0cc0aebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:52.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.052+0000 7fa0cbe00640 1 -- 172.21.15.100:0/2752481883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0cc160320 msgr2=0x7fa0cc0af110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:52.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.052+0000 7fa0cbe00640 1 --2- 172.21.15.100:0/2752481883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0cc160320 0x7fa0cc0af110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:52.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.052+0000 7fa0cbe00640 1 -- 172.21.15.100:0/2752481883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0cc15f130 con 0x7fa0cc0735e0 2024-09-17T04:31:52.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.053+0000 7fa0cbe00640 1 --2- 172.21.15.100:0/2752481883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0cc0735e0 0x7fa0cc15b8b0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc000c00 tx=0x7fa0bc033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:52.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.053+0000 7fa0c8c00640 1 -- 172.21.15.100:0/2752481883 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc031d60 con 0x7fa0cc0735e0 2024-09-17T04:31:52.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.053+0000 7fa0c8c00640 1 -- 172.21.15.100:0/2752481883 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0bc033e10 con 0x7fa0cc0735e0 2024-09-17T04:31:52.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.053+0000 7fa0d3000640 1 -- 172.21.15.100:0/2752481883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0cc15f410 con 0x7fa0cc0735e0 2024-09-17T04:31:52.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.053+0000 7fa0c8c00640 1 -- 172.21.15.100:0/2752481883 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc033310 con 0x7fa0cc0735e0 2024-09-17T04:31:52.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.053+0000 7fa0d3000640 1 -- 172.21.15.100:0/2752481883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0cc15f9d0 con 0x7fa0cc0735e0 2024-09-17T04:31:52.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.054+0000 7fa0d3000640 1 -- 172.21.15.100:0/2752481883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0cc15ee90 con 0x7fa0cc0735e0 2024-09-17T04:31:52.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.057+0000 7fa0c8c00640 1 -- 172.21.15.100:0/2752481883 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fa0bc0334b0 con 0x7fa0cc0735e0 2024-09-17T04:31:52.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.058+0000 7fa0c8c00640 1 --2- 172.21.15.100:0/2752481883 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa0a80791c0 0x7fa0a807b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:52.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.058+0000 7fa0c8c00640 1 -- 172.21.15.100:0/2752481883 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5283+0+0 (secure 0 0 0) 0x7fa0bc0bcf50 con 0x7fa0cc0735e0 2024-09-17T04:31:52.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.058+0000 7fa0c3400640 1 --2- 172.21.15.100:0/2752481883 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa0a80791c0 0x7fa0a807b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:52.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.059+0000 7fa0c3400640 1 --2- 172.21.15.100:0/2752481883 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa0a80791c0 0x7fa0a807b6d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40096c0 tx=0x7fa0b4009260 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:52.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.062+0000 7fa0c8c00640 1 -- 172.21.15.100:0/2752481883 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa0cc15ee90 con 0x7fa0cc0735e0 2024-09-17T04:31:52.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.245+0000 7fa0d3000640 1 -- 172.21.15.100:0/2752481883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa0cc161050 con 0x7fa0cc0735e0 2024-09-17T04:31:52.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.788+0000 7fa0c8c00640 1 -- 172.21.15.100:0/2752481883 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v130) ==== 170+0+0 (secure 0 0 0) 0x7fa0bc051020 con 0x7fa0cc0735e0 2024-09-17T04:31:52.788 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' created 2024-09-17T04:31:52.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.791+0000 7fa0d3000640 1 -- 172.21.15.100:0/2752481883 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa0a80791c0 msgr2=0x7fa0a807b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:52.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.791+0000 7fa0d3000640 1 --2- 172.21.15.100:0/2752481883 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa0a80791c0 0x7fa0a807b6d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40096c0 tx=0x7fa0b4009260 comp rx=0 tx=0).stop 2024-09-17T04:31:52.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.791+0000 7fa0d3000640 1 -- 172.21.15.100:0/2752481883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0cc0735e0 msgr2=0x7fa0cc15b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:52.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.791+0000 7fa0d3000640 1 --2- 172.21.15.100:0/2752481883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0cc0735e0 0x7fa0cc15b8b0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc000c00 tx=0x7fa0bc033c20 comp rx=0 tx=0).stop 2024-09-17T04:31:52.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.791+0000 7fa0d3000640 1 -- 172.21.15.100:0/2752481883 shutdown_connections 2024-09-17T04:31:52.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.791+0000 7fa0d3000640 1 --2- 172.21.15.100:0/2752481883 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa0a80791c0 0x7fa0a807b6d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:52.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.791+0000 7fa0d3000640 1 --2- 172.21.15.100:0/2752481883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0cc160320 0x7fa0cc0af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:52.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.791+0000 7fa0d3000640 1 --2- 172.21.15.100:0/2752481883 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0cc145450 0x7fa0cc0aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:52.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.791+0000 7fa0d3000640 1 --2- 172.21.15.100:0/2752481883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0cc0735e0 0x7fa0cc15b8b0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:52.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.791+0000 7fa0d3000640 1 -- 172.21.15.100:0/2752481883 >> 172.21.15.100:0/2752481883 conn(0x7fa0cc06d1c0 msgr2=0x7fa0cc0b00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:52.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.792+0000 7fa0d3000640 1 -- 172.21.15.100:0/2752481883 shutdown_connections 2024-09-17T04:31:52.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:52.792+0000 7fa0d3000640 1 -- 172.21.15.100:0/2752481883 wait complete. 2024-09-17T04:31:52.833 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T04:31:53.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.058+0000 7feb47600640 1 -- 172.21.15.100:0/3431290730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb400d5150 msgr2=0x7feb400b9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:53.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.058+0000 7feb47600640 1 --2- 172.21.15.100:0/3431290730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb400d5150 0x7feb400b9b10 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7feb4008e950 tx=0x7feb3002a540 comp rx=0 tx=0).stop 2024-09-17T04:31:53.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.059+0000 7feb47600640 1 -- 172.21.15.100:0/3431290730 shutdown_connections 2024-09-17T04:31:53.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.059+0000 7feb47600640 1 --2- 172.21.15.100:0/3431290730 >> v2:172.21.15.184:3300/0 conn(0x7feb4006dea0 0x7feb400d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:53.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.059+0000 7feb47600640 1 --2- 172.21.15.100:0/3431290730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb400d5150 0x7feb400b9b10 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:53.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.059+0000 7feb47600640 1 -- 172.21.15.100:0/3431290730 >> 172.21.15.100:0/3431290730 conn(0x7feb4008bfe0 msgr2=0x7feb400d46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:53.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.059+0000 7feb47600640 1 -- 172.21.15.100:0/3431290730 shutdown_connections 2024-09-17T04:31:53.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.059+0000 7feb47600640 1 -- 172.21.15.100:0/3431290730 wait complete. 2024-09-17T04:31:53.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.061+0000 7feb47600640 1 Processor -- start 2024-09-17T04:31:53.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.061+0000 7feb47600640 1 -- start start 2024-09-17T04:31:53.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.063+0000 7feb47600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb4006dea0 0x7feb4015c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:53.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.063+0000 7feb47600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb4006e2b0 0x7feb401f9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:53.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.063+0000 7feb47600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb400d5150 0x7feb400a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:53.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.063+0000 7feb47600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7feb400d3150 con 0x7feb400d5150 2024-09-17T04:31:53.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.063+0000 7feb47600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feb400d2fd0 con 0x7feb4006e2b0 2024-09-17T04:31:53.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.063+0000 7feb47600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7feb400b6870 con 0x7feb4006dea0 2024-09-17T04:31:53.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.063+0000 7feb45600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb400d5150 0x7feb400a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:53.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.063+0000 7feb45600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb400d5150 0x7feb400a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34442/0 (socket says 172.21.15.100:34442) 2024-09-17T04:31:53.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.063+0000 7feb45600640 1 -- 172.21.15.100:0/1211308515 learned_addr learned my addr 172.21.15.100:0/1211308515 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:53.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.063+0000 7feb44c00640 1 --2- 172.21.15.100:0/1211308515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb4006dea0 0x7feb4015c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:53.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.063+0000 7feb37e00640 1 --2- 172.21.15.100:0/1211308515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb4006e2b0 0x7feb401f9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:53.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.063+0000 7feb45600640 1 -- 172.21.15.100:0/1211308515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb4006dea0 msgr2=0x7feb4015c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:53.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.063+0000 7feb45600640 1 --2- 172.21.15.100:0/1211308515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb4006dea0 0x7feb4015c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:53.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.063+0000 7feb45600640 1 -- 172.21.15.100:0/1211308515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb4006e2b0 msgr2=0x7feb401f9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:53.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.063+0000 7feb45600640 1 --2- 172.21.15.100:0/1211308515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb4006e2b0 0x7feb401f9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:53.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.063+0000 7feb45600640 1 -- 172.21.15.100:0/1211308515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb400cc9e0 con 0x7feb400d5150 2024-09-17T04:31:53.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.064+0000 7feb45600640 1 --2- 172.21.15.100:0/1211308515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb400d5150 0x7feb400a17d0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7feb3800b9e0 tx=0x7feb3800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:53.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.064+0000 7feb3d600640 1 -- 172.21.15.100:0/1211308515 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb38019070 con 0x7feb400d5150 2024-09-17T04:31:53.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.064+0000 7feb3d600640 1 -- 172.21.15.100:0/1211308515 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb3800cc10 con 0x7feb400d5150 2024-09-17T04:31:53.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.064+0000 7feb47600640 1 -- 172.21.15.100:0/1211308515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb400ccca0 con 0x7feb400d5150 2024-09-17T04:31:53.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.064+0000 7feb3d600640 1 -- 172.21.15.100:0/1211308515 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb380146a0 con 0x7feb400d5150 2024-09-17T04:31:53.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.064+0000 7feb47600640 1 -- 172.21.15.100:0/1211308515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb400cd780 con 0x7feb400d5150 2024-09-17T04:31:53.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.065+0000 7feb47600640 1 -- 172.21.15.100:0/1211308515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb40159940 con 0x7feb400d5150 2024-09-17T04:31:53.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.067+0000 7feb3d600640 1 -- 172.21.15.100:0/1211308515 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7feb3801c070 con 0x7feb400d5150 2024-09-17T04:31:53.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.068+0000 7feb3d600640 1 --2- 172.21.15.100:0/1211308515 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7feb1c0791c0 0x7feb1c07b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:53.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.068+0000 7feb3d600640 1 -- 172.21.15.100:0/1211308515 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5665+0+0 (secure 0 0 0) 0x7feb3809b680 con 0x7feb400d5150 2024-09-17T04:31:53.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.068+0000 7feb44c00640 1 --2- 172.21.15.100:0/1211308515 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7feb1c0791c0 0x7feb1c07b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:53.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.069+0000 7feb44c00640 1 --2- 172.21.15.100:0/1211308515 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7feb1c0791c0 0x7feb1c07b6d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7feb30034000 tx=0x7feb30004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:53.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.073+0000 7feb3d600640 1 -- 172.21.15.100:0/1211308515 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7feb40159940 con 0x7feb400d5150 2024-09-17T04:31:53.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.291+0000 7feb47600640 1 -- 172.21.15.100:0/1211308515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7feb400d0660 con 0x7feb400d5150 2024-09-17T04:31:53.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.812+0000 7feb3d600640 1 -- 172.21.15.100:0/1211308515 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v87) ==== 174+0+0 (secure 0 0 0) 0x7feb38062a00 con 0x7feb400d5150 2024-09-17T04:31:53.812 INFO:teuthology.orchestra.run.smithi100.stderr:new fs with metadata pool 20 and data pool 21 2024-09-17T04:31:53.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.820+0000 7feb47600640 1 -- 172.21.15.100:0/1211308515 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7feb1c0791c0 msgr2=0x7feb1c07b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:53.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.820+0000 7feb47600640 1 --2- 172.21.15.100:0/1211308515 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7feb1c0791c0 0x7feb1c07b6d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7feb30034000 tx=0x7feb30004730 comp rx=0 tx=0).stop 2024-09-17T04:31:53.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.820+0000 7feb47600640 1 -- 172.21.15.100:0/1211308515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb400d5150 msgr2=0x7feb400a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:53.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.820+0000 7feb47600640 1 --2- 172.21.15.100:0/1211308515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb400d5150 0x7feb400a17d0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7feb3800b9e0 tx=0x7feb3800bea0 comp rx=0 tx=0).stop 2024-09-17T04:31:53.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.821+0000 7feb47600640 1 -- 172.21.15.100:0/1211308515 shutdown_connections 2024-09-17T04:31:53.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.821+0000 7feb47600640 1 --2- 172.21.15.100:0/1211308515 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7feb1c0791c0 0x7feb1c07b6d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:53.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.821+0000 7feb47600640 1 --2- 172.21.15.100:0/1211308515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb400d5150 0x7feb400a17d0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:53.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.821+0000 7feb47600640 1 --2- 172.21.15.100:0/1211308515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb4006e2b0 0x7feb401f9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:53.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.821+0000 7feb47600640 1 --2- 172.21.15.100:0/1211308515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb4006dea0 0x7feb4015c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:53.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.821+0000 7feb47600640 1 -- 172.21.15.100:0/1211308515 >> 172.21.15.100:0/1211308515 conn(0x7feb4008bfe0 msgr2=0x7feb400b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:53.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.822+0000 7feb47600640 1 -- 172.21.15.100:0/1211308515 shutdown_connections 2024-09-17T04:31:53.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:53.822+0000 7feb47600640 1 -- 172.21.15.100:0/1211308515 wait complete. 2024-09-17T04:31:53.862 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:31:54.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.084+0000 7f592ae00640 1 -- 172.21.15.100:0/372894054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59240d14c0 msgr2=0x7f59240d51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:54.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.084+0000 7f592ae00640 1 --2- 172.21.15.100:0/372894054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59240d14c0 0x7f59240d51a0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f5910009830 tx=0x7f591002a510 comp rx=0 tx=0).stop 2024-09-17T04:31:54.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.084+0000 7f592ae00640 1 -- 172.21.15.100:0/372894054 shutdown_connections 2024-09-17T04:31:54.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.084+0000 7f592ae00640 1 --2- 172.21.15.100:0/372894054 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59240d31e0 0x7f59240cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:54.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.084+0000 7f592ae00640 1 --2- 172.21.15.100:0/372894054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59240d14c0 0x7f59240d51a0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:54.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.084+0000 7f592ae00640 1 -- 172.21.15.100:0/372894054 >> 172.21.15.100:0/372894054 conn(0x7f59240d4680 msgr2=0x7f59240d4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:54.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.084+0000 7f592ae00640 1 -- 172.21.15.100:0/372894054 shutdown_connections 2024-09-17T04:31:54.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.084+0000 7f592ae00640 1 -- 172.21.15.100:0/372894054 wait complete. 2024-09-17T04:31:54.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.086+0000 7f592ae00640 1 Processor -- start 2024-09-17T04:31:54.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.086+0000 7f592ae00640 1 -- start start 2024-09-17T04:31:54.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.088+0000 7f592ae00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59240b9b40 0x7f59240b8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:54.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.088+0000 7f592ae00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59240d31e0 0x7f59240b8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:54.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.088+0000 7f592ae00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59240b74d0 0x7f592415ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:54.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.088+0000 7f592ae00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f59240cf7d0 con 0x7f59240b74d0 2024-09-17T04:31:54.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.088+0000 7f592ae00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f59240b3ce0 con 0x7f59240d31e0 2024-09-17T04:31:54.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.088+0000 7f592ae00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f59240cf950 con 0x7f59240b9b40 2024-09-17T04:31:54.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.088+0000 7f5928e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59240b74d0 0x7f592415ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:54.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.088+0000 7f5928e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59240b74d0 0x7f592415ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34458/0 (socket says 172.21.15.100:34458) 2024-09-17T04:31:54.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.088+0000 7f5928e00640 1 -- 172.21.15.100:0/2279933561 learned_addr learned my addr 172.21.15.100:0/2279933561 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:54.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.088+0000 7f5923e00640 1 --2- 172.21.15.100:0/2279933561 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59240b9b40 0x7f59240b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:54.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.088+0000 7f5923400640 1 --2- 172.21.15.100:0/2279933561 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59240d31e0 0x7f59240b8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:54.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.089+0000 7f5928e00640 1 -- 172.21.15.100:0/2279933561 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59240b9b40 msgr2=0x7f59240b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:54.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.089+0000 7f5928e00640 1 --2- 172.21.15.100:0/2279933561 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59240b9b40 0x7f59240b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:54.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.089+0000 7f5928e00640 1 -- 172.21.15.100:0/2279933561 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59240d31e0 msgr2=0x7f59240b8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:54.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.089+0000 7f5928e00640 1 --2- 172.21.15.100:0/2279933561 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59240d31e0 0x7f59240b8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:54.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.089+0000 7f5928e00640 1 -- 172.21.15.100:0/2279933561 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59241f9370 con 0x7f59240b74d0 2024-09-17T04:31:54.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.089+0000 7f5928e00640 1 --2- 172.21.15.100:0/2279933561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59240b74d0 0x7f592415ee50 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f591c00e700 tx=0x7f591c00ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:54.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.089+0000 7f591be00640 1 -- 172.21.15.100:0/2279933561 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c00cab0 con 0x7f59240b74d0 2024-09-17T04:31:54.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.089+0000 7f591be00640 1 -- 172.21.15.100:0/2279933561 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f591c00cc50 con 0x7f59240b74d0 2024-09-17T04:31:54.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.089+0000 7f592ae00640 1 -- 172.21.15.100:0/2279933561 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59241f9650 con 0x7f59240b74d0 2024-09-17T04:31:54.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.089+0000 7f591be00640 1 -- 172.21.15.100:0/2279933561 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c005020 con 0x7f59240b74d0 2024-09-17T04:31:54.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.090+0000 7f592ae00640 1 -- 172.21.15.100:0/2279933561 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59241f9c10 con 0x7f59240b74d0 2024-09-17T04:31:54.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.091+0000 7f592ae00640 1 -- 172.21.15.100:0/2279933561 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59240cfe70 con 0x7f59240b74d0 2024-09-17T04:31:54.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.092+0000 7f591be00640 1 -- 172.21.15.100:0/2279933561 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f591c0051c0 con 0x7f59240b74d0 2024-09-17T04:31:54.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.093+0000 7f591be00640 1 --2- 172.21.15.100:0/2279933561 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f59040791c0 0x7f590407b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:54.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.094+0000 7f591be00640 1 -- 172.21.15.100:0/2279933561 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5765+0+0 (secure 0 0 0) 0x7f591c09c160 con 0x7f59240b74d0 2024-09-17T04:31:54.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.094+0000 7f5923e00640 1 --2- 172.21.15.100:0/2279933561 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f59040791c0 0x7f590407b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:54.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.094+0000 7f5923e00640 1 --2- 172.21.15.100:0/2279933561 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f59040791c0 0x7f590407b6d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f59140012e0 tx=0x7f5914009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:54.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.098+0000 7f591be00640 1 -- 172.21.15.100:0/2279933561 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f591c063550 con 0x7f59240b74d0 2024-09-17T04:31:54.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.283+0000 7f592ae00640 1 -- 172.21.15.100:0/2279933561 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59241f9100 con 0x7f59240b74d0 2024-09-17T04:31:54.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.284+0000 7f591be00640 1 -- 172.21.15.100:0/2279933561 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) ==== 74+0+16889 (secure 0 0 0) 0x7f591c067f80 con 0x7f59240b74d0 2024-09-17T04:31:54.285 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:54.285 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":131,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:31:53.292970+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:31:51.164879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-17T04:31:52.259089+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:31:54.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.286+0000 7f592ae00640 1 -- 172.21.15.100:0/2279933561 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f59040791c0 msgr2=0x7f590407b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:54.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.286+0000 7f592ae00640 1 --2- 172.21.15.100:0/2279933561 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f59040791c0 0x7f590407b6d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f59140012e0 tx=0x7f5914009040 comp rx=0 tx=0).stop 2024-09-17T04:31:54.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.286+0000 7f592ae00640 1 -- 172.21.15.100:0/2279933561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59240b74d0 msgr2=0x7f592415ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:54.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.286+0000 7f592ae00640 1 --2- 172.21.15.100:0/2279933561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59240b74d0 0x7f592415ee50 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f591c00e700 tx=0x7f591c00ebc0 comp rx=0 tx=0).stop 2024-09-17T04:31:54.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.286+0000 7f592ae00640 1 -- 172.21.15.100:0/2279933561 shutdown_connections 2024-09-17T04:31:54.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.286+0000 7f592ae00640 1 --2- 172.21.15.100:0/2279933561 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f59040791c0 0x7f590407b6d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:54.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.286+0000 7f592ae00640 1 --2- 172.21.15.100:0/2279933561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59240b74d0 0x7f592415ee50 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:54.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.286+0000 7f592ae00640 1 --2- 172.21.15.100:0/2279933561 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59240d31e0 0x7f59240b8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:54.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.286+0000 7f592ae00640 1 --2- 172.21.15.100:0/2279933561 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59240b9b40 0x7f59240b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:54.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.286+0000 7f592ae00640 1 -- 172.21.15.100:0/2279933561 >> 172.21.15.100:0/2279933561 conn(0x7f59240d4680 msgr2=0x7f5924160190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:54.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.287+0000 7f592ae00640 1 -- 172.21.15.100:0/2279933561 shutdown_connections 2024-09-17T04:31:54.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.287+0000 7f592ae00640 1 -- 172.21.15.100:0/2279933561 wait complete. 2024-09-17T04:31:54.313 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:31:54.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.531+0000 7f3e9ca00640 1 -- 172.21.15.100:0/1285322131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e980ad9c0 msgr2=0x7f3e980af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:54.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.531+0000 7f3e9ca00640 1 --2- 172.21.15.100:0/1285322131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e980ad9c0 0x7f3e980af170 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3e88005fd0 tx=0x7f3e8802a820 comp rx=0 tx=0).stop 2024-09-17T04:31:54.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.531+0000 7f3e9ca00640 1 -- 172.21.15.100:0/1285322131 shutdown_connections 2024-09-17T04:31:54.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.531+0000 7f3e9ca00640 1 --2- 172.21.15.100:0/1285322131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e9815a0f0 0x7f3e9815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:54.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.531+0000 7f3e9ca00640 1 --2- 172.21.15.100:0/1285322131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e980ad9c0 0x7f3e980af170 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:54.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.531+0000 7f3e9ca00640 1 -- 172.21.15.100:0/1285322131 >> 172.21.15.100:0/1285322131 conn(0x7f3e980b1e40 msgr2=0x7f3e980b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:54.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.531+0000 7f3e9ca00640 1 -- 172.21.15.100:0/1285322131 shutdown_connections 2024-09-17T04:31:54.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.531+0000 7f3e9ca00640 1 -- 172.21.15.100:0/1285322131 wait complete. 2024-09-17T04:31:54.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.533+0000 7f3e9ca00640 1 Processor -- start 2024-09-17T04:31:54.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.534+0000 7f3e9ca00640 1 -- start start 2024-09-17T04:31:54.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.535+0000 7f3e9ca00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e980ad9c0 0x7f3e98145ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:54.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.535+0000 7f3e9ca00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e980b2f60 0x7f3e9809d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:54.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.535+0000 7f3e9ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e9815a0f0 0x7f3e981f66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:54.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.535+0000 7f3e9ca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9806f290 con 0x7f3e9815a0f0 2024-09-17T04:31:54.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.535+0000 7f3e9ca00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9815a5e0 con 0x7f3e980ad9c0 2024-09-17T04:31:54.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.535+0000 7f3e9ca00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3e9806f410 con 0x7f3e980b2f60 2024-09-17T04:31:54.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.535+0000 7f3e97200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e9815a0f0 0x7f3e981f66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:54.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.535+0000 7f3e97200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e9815a0f0 0x7f3e981f66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34472/0 (socket says 172.21.15.100:34472) 2024-09-17T04:31:54.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.535+0000 7f3e97200640 1 -- 172.21.15.100:0/3402863670 learned_addr learned my addr 172.21.15.100:0/3402863670 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:54.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.535+0000 7f3e96800640 1 --2- 172.21.15.100:0/3402863670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e980ad9c0 0x7f3e98145ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:54.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.535+0000 7f3e95e00640 1 --2- 172.21.15.100:0/3402863670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e980b2f60 0x7f3e9809d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:54.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.536+0000 7f3e97200640 1 -- 172.21.15.100:0/3402863670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e980b2f60 msgr2=0x7f3e9809d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:54.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.536+0000 7f3e97200640 1 --2- 172.21.15.100:0/3402863670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e980b2f60 0x7f3e9809d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:54.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.536+0000 7f3e97200640 1 -- 172.21.15.100:0/3402863670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e980ad9c0 msgr2=0x7f3e98145ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:54.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.536+0000 7f3e97200640 1 --2- 172.21.15.100:0/3402863670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e980ad9c0 0x7f3e98145ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:54.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.536+0000 7f3e97200640 1 -- 172.21.15.100:0/3402863670 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e981483b0 con 0x7f3e9815a0f0 2024-09-17T04:31:54.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.536+0000 7f3e97200640 1 --2- 172.21.15.100:0/3402863670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e9815a0f0 0x7f3e981f66d0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000e700 tx=0x7f3e9000ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:54.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.536+0000 7f3e8ea00640 1 -- 172.21.15.100:0/3402863670 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9000cab0 con 0x7f3e9815a0f0 2024-09-17T04:31:54.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.536+0000 7f3e8ea00640 1 -- 172.21.15.100:0/3402863670 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e9000cc50 con 0x7f3e9815a0f0 2024-09-17T04:31:54.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.536+0000 7f3e9ca00640 1 -- 172.21.15.100:0/3402863670 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e98148620 con 0x7f3e9815a0f0 2024-09-17T04:31:54.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.536+0000 7f3e8ea00640 1 -- 172.21.15.100:0/3402863670 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e90005020 con 0x7f3e9815a0f0 2024-09-17T04:31:54.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.537+0000 7f3e9ca00640 1 -- 172.21.15.100:0/3402863670 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e98148be0 con 0x7f3e9815a0f0 2024-09-17T04:31:54.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.537+0000 7f3e9ca00640 1 -- 172.21.15.100:0/3402863670 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e9809e270 con 0x7f3e9815a0f0 2024-09-17T04:31:54.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.539+0000 7f3e8ea00640 1 -- 172.21.15.100:0/3402863670 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f3e900051c0 con 0x7f3e9815a0f0 2024-09-17T04:31:54.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.540+0000 7f3e8ea00640 1 --2- 172.21.15.100:0/3402863670 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3e780791c0 0x7f3e7807b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:54.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.540+0000 7f3e8ea00640 1 -- 172.21.15.100:0/3402863670 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5765+0+0 (secure 0 0 0) 0x7f3e9009c1f0 con 0x7f3e9815a0f0 2024-09-17T04:31:54.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.541+0000 7f3e96800640 1 --2- 172.21.15.100:0/3402863670 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3e780791c0 0x7f3e7807b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:54.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.541+0000 7f3e96800640 1 --2- 172.21.15.100:0/3402863670 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3e780791c0 0x7f3e7807b6d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3e88034000 tx=0x7f3e88004ab0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:54.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.545+0000 7f3e8ea00640 1 -- 172.21.15.100:0/3402863670 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3e90063510 con 0x7f3e9815a0f0 2024-09-17T04:31:54.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.730+0000 7f3e9ca00640 1 -- 172.21.15.100:0/3402863670 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e9806f930 con 0x7f3e9815a0f0 2024-09-17T04:31:54.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.732+0000 7f3e8ea00640 1 -- 172.21.15.100:0/3402863670 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) ==== 74+0+16889 (secure 0 0 0) 0x7f3e90067f40 con 0x7f3e9815a0f0 2024-09-17T04:31:54.732 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:54.732 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":131,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:31:53.292970+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:31:51.164879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-17T04:31:52.259089+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:31:54.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.734+0000 7f3e9ca00640 1 -- 172.21.15.100:0/3402863670 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3e780791c0 msgr2=0x7f3e7807b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:54.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.734+0000 7f3e9ca00640 1 --2- 172.21.15.100:0/3402863670 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3e780791c0 0x7f3e7807b6d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3e88034000 tx=0x7f3e88004ab0 comp rx=0 tx=0).stop 2024-09-17T04:31:54.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.734+0000 7f3e9ca00640 1 -- 172.21.15.100:0/3402863670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e9815a0f0 msgr2=0x7f3e981f66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:54.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.734+0000 7f3e9ca00640 1 --2- 172.21.15.100:0/3402863670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e9815a0f0 0x7f3e981f66d0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000e700 tx=0x7f3e9000ebc0 comp rx=0 tx=0).stop 2024-09-17T04:31:54.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.734+0000 7f3e9ca00640 1 -- 172.21.15.100:0/3402863670 shutdown_connections 2024-09-17T04:31:54.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.734+0000 7f3e9ca00640 1 --2- 172.21.15.100:0/3402863670 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3e780791c0 0x7f3e7807b6d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:54.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.734+0000 7f3e9ca00640 1 --2- 172.21.15.100:0/3402863670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e9815a0f0 0x7f3e981f66d0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:54.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.734+0000 7f3e9ca00640 1 --2- 172.21.15.100:0/3402863670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e980b2f60 0x7f3e9809d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:54.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.734+0000 7f3e9ca00640 1 --2- 172.21.15.100:0/3402863670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e980ad9c0 0x7f3e98145ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:54.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.734+0000 7f3e9ca00640 1 -- 172.21.15.100:0/3402863670 >> 172.21.15.100:0/3402863670 conn(0x7f3e980b1e40 msgr2=0x7f3e98153470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:54.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.734+0000 7f3e9ca00640 1 -- 172.21.15.100:0/3402863670 shutdown_connections 2024-09-17T04:31:54.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.734+0000 7f3e9ca00640 1 -- 172.21.15.100:0/3402863670 wait complete. 2024-09-17T04:31:54.761 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T04:31:54.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.992+0000 7fd122e00640 1 -- 172.21.15.100:0/2030740522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd11c070a70 msgr2=0x7fd11c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:54.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.992+0000 7fd122e00640 1 --2- 172.21.15.100:0/2030740522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd11c070a70 0x7fd11c06e2f0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fd108009830 tx=0x7fd10802a480 comp rx=0 tx=0).stop 2024-09-17T04:31:54.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.992+0000 7fd122e00640 1 -- 172.21.15.100:0/2030740522 shutdown_connections 2024-09-17T04:31:54.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.992+0000 7fd122e00640 1 --2- 172.21.15.100:0/2030740522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd11c070a70 0x7fd11c06e2f0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:54.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.992+0000 7fd122e00640 1 --2- 172.21.15.100:0/2030740522 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd11c074440 0x7fd11c06ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:54.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.992+0000 7fd122e00640 1 -- 172.21.15.100:0/2030740522 >> 172.21.15.100:0/2030740522 conn(0x7fd11c073920 msgr2=0x7fd11c073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:54.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.992+0000 7fd122e00640 1 -- 172.21.15.100:0/2030740522 shutdown_connections 2024-09-17T04:31:54.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.992+0000 7fd122e00640 1 -- 172.21.15.100:0/2030740522 wait complete. 2024-09-17T04:31:54.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.994+0000 7fd122e00640 1 Processor -- start 2024-09-17T04:31:54.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.995+0000 7fd122e00640 1 -- start start 2024-09-17T04:31:54.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.996+0000 7fd122e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd11c070a70 0x7fd11c1f3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:54.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.996+0000 7fd122e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd11c074440 0x7fd11c153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:54.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.996+0000 7fd122e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd11c15b920 0x7fd11c1536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:54.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.996+0000 7fd122e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd11c0b41b0 con 0x7fd11c15b920 2024-09-17T04:31:54.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.996+0000 7fd122e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd11c0d0590 con 0x7fd11c074440 2024-09-17T04:31:54.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.996+0000 7fd122e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd11c0b4330 con 0x7fd11c070a70 2024-09-17T04:31:54.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.996+0000 7fd120e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd11c15b920 0x7fd11c1536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:54.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.996+0000 7fd120e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd11c15b920 0x7fd11c1536d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48758/0 (socket says 172.21.15.100:48758) 2024-09-17T04:31:54.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.996+0000 7fd120e00640 1 -- 172.21.15.100:0/2428981626 learned_addr learned my addr 172.21.15.100:0/2428981626 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:54.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.997+0000 7fd11b400640 1 --2- 172.21.15.100:0/2428981626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd11c074440 0x7fd11c153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:54.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.997+0000 7fd11be00640 1 --2- 172.21.15.100:0/2428981626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd11c070a70 0x7fd11c1f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:54.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.997+0000 7fd120e00640 1 -- 172.21.15.100:0/2428981626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd11c070a70 msgr2=0x7fd11c1f3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:54.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.997+0000 7fd120e00640 1 --2- 172.21.15.100:0/2428981626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd11c070a70 0x7fd11c1f3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:54.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.997+0000 7fd120e00640 1 -- 172.21.15.100:0/2428981626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd11c074440 msgr2=0x7fd11c153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:54.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.997+0000 7fd120e00640 1 --2- 172.21.15.100:0/2428981626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd11c074440 0x7fd11c153190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:54.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.997+0000 7fd120e00640 1 -- 172.21.15.100:0/2428981626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd11c1f41e0 con 0x7fd11c15b920 2024-09-17T04:31:54.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.997+0000 7fd11be00640 1 --2- 172.21.15.100:0/2428981626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd11c070a70 0x7fd11c1f3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:31:54.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.997+0000 7fd11b400640 1 --2- 172.21.15.100:0/2428981626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd11c074440 0x7fd11c153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:31:54.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.997+0000 7fd120e00640 1 --2- 172.21.15.100:0/2428981626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd11c15b920 0x7fd11c1536d0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fd11400c3f0 tx=0x7fd11400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:54.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.997+0000 7fd113e00640 1 -- 172.21.15.100:0/2428981626 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd114017070 con 0x7fd11c15b920 2024-09-17T04:31:54.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.997+0000 7fd113e00640 1 -- 172.21.15.100:0/2428981626 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1140035f0 con 0x7fd11c15b920 2024-09-17T04:31:54.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.997+0000 7fd113e00640 1 -- 172.21.15.100:0/2428981626 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd114003a40 con 0x7fd11c15b920 2024-09-17T04:31:54.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.997+0000 7fd122e00640 1 -- 172.21.15.100:0/2428981626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd11c1523d0 con 0x7fd11c15b920 2024-09-17T04:31:54.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.998+0000 7fd122e00640 1 -- 172.21.15.100:0/2428981626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd11c152990 con 0x7fd11c15b920 2024-09-17T04:31:55.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:54.999+0000 7fd122e00640 1 -- 172.21.15.100:0/2428981626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd11c0b4880 con 0x7fd11c15b920 2024-09-17T04:31:55.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:55.000+0000 7fd113e00640 1 -- 172.21.15.100:0/2428981626 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fd11401a070 con 0x7fd11c15b920 2024-09-17T04:31:55.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:55.002+0000 7fd113e00640 1 --2- 172.21.15.100:0/2428981626 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd0fc0791c0 0x7fd0fc07b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:55.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:55.002+0000 7fd113e00640 1 -- 172.21.15.100:0/2428981626 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7fd11409ab90 con 0x7fd11c15b920 2024-09-17T04:31:55.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:55.002+0000 7fd11be00640 1 --2- 172.21.15.100:0/2428981626 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd0fc0791c0 0x7fd0fc07b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:55.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:55.003+0000 7fd11be00640 1 --2- 172.21.15.100:0/2428981626 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd0fc0791c0 0x7fd0fc07b6d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd10c005fd0 tx=0x7fd10c009480 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:55.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:55.006+0000 7fd113e00640 1 -- 172.21.15.100:0/2428981626 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd114062590 con 0x7fd11c15b920 2024-09-17T04:31:55.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:55.224+0000 7fd122e00640 1 -- 172.21.15.100:0/2428981626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd11c0cfde0 con 0x7fd11c15b920 2024-09-17T04:31:55.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:55.834+0000 7fd113e00640 1 -- 172.21.15.100:0/2428981626 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v90) ==== 118+0+0 (secure 0 0 0) 0x7fd114061e90 con 0x7fd11c15b920 2024-09-17T04:31:55.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:55.836+0000 7fd122e00640 1 -- 172.21.15.100:0/2428981626 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd0fc0791c0 msgr2=0x7fd0fc07b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:55.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:55.836+0000 7fd122e00640 1 --2- 172.21.15.100:0/2428981626 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd0fc0791c0 0x7fd0fc07b6d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd10c005fd0 tx=0x7fd10c009480 comp rx=0 tx=0).stop 2024-09-17T04:31:55.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:55.836+0000 7fd122e00640 1 -- 172.21.15.100:0/2428981626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd11c15b920 msgr2=0x7fd11c1536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:55.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:55.836+0000 7fd122e00640 1 --2- 172.21.15.100:0/2428981626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd11c15b920 0x7fd11c1536d0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fd11400c3f0 tx=0x7fd11400c8b0 comp rx=0 tx=0).stop 2024-09-17T04:31:55.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:55.837+0000 7fd122e00640 1 -- 172.21.15.100:0/2428981626 shutdown_connections 2024-09-17T04:31:55.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:55.837+0000 7fd122e00640 1 --2- 172.21.15.100:0/2428981626 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd0fc0791c0 0x7fd0fc07b6d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:55.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:55.837+0000 7fd122e00640 1 --2- 172.21.15.100:0/2428981626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd11c15b920 0x7fd11c1536d0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:55.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:55.837+0000 7fd122e00640 1 --2- 172.21.15.100:0/2428981626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd11c074440 0x7fd11c153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:55.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:55.837+0000 7fd122e00640 1 --2- 172.21.15.100:0/2428981626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd11c070a70 0x7fd11c1f3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:55.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:55.837+0000 7fd122e00640 1 -- 172.21.15.100:0/2428981626 >> 172.21.15.100:0/2428981626 conn(0x7fd11c073920 msgr2=0x7fd11c15c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:55.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:55.837+0000 7fd122e00640 1 -- 172.21.15.100:0/2428981626 shutdown_connections 2024-09-17T04:31:55.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:55.837+0000 7fd122e00640 1 -- 172.21.15.100:0/2428981626 wait complete. 2024-09-17T04:31:55.861 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T04:31:55.862 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T04:31:56.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.086+0000 7f6c2f600640 1 -- 172.21.15.100:0/1307825439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c28145550 msgr2=0x7f6c2806fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:56.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.086+0000 7f6c2f600640 1 --2- 172.21.15.100:0/1307825439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c28145550 0x7f6c2806fb40 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009830 tx=0x7f6c1c02a5a0 comp rx=0 tx=0).stop 2024-09-17T04:31:56.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.086+0000 7f6c2f600640 1 -- 172.21.15.100:0/1307825439 shutdown_connections 2024-09-17T04:31:56.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.086+0000 7f6c2f600640 1 --2- 172.21.15.100:0/1307825439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c28145a40 0x7f6c28073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:56.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.086+0000 7f6c2f600640 1 --2- 172.21.15.100:0/1307825439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c28145550 0x7f6c2806fb40 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:56.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.086+0000 7f6c2f600640 1 -- 172.21.15.100:0/1307825439 >> 172.21.15.100:0/1307825439 conn(0x7f6c2806d1c0 msgr2=0x7f6c2806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:56.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.086+0000 7f6c2f600640 1 -- 172.21.15.100:0/1307825439 shutdown_connections 2024-09-17T04:31:56.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.086+0000 7f6c2f600640 1 -- 172.21.15.100:0/1307825439 wait complete. 2024-09-17T04:31:56.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.088+0000 7f6c2f600640 1 Processor -- start 2024-09-17T04:31:56.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.088+0000 7f6c2f600640 1 -- start start 2024-09-17T04:31:56.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.090+0000 7f6c2f600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c28073a80 0x7f6c280b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:56.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.090+0000 7f6c2f600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c28145550 0x7f6c2813acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:56.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.090+0000 7f6c2f600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c28145a40 0x7f6c281f7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:56.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.090+0000 7f6c2f600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6c28160a30 con 0x7f6c28145a40 2024-09-17T04:31:56.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.090+0000 7f6c2f600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6c280ce940 con 0x7f6c28073a80 2024-09-17T04:31:56.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.090+0000 7f6c2f600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6c28160bb0 con 0x7f6c28145550 2024-09-17T04:31:56.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.090+0000 7f6c2d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c28145a40 0x7f6c281f7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:56.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.090+0000 7f6c2d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c28145a40 0x7f6c281f7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48766/0 (socket says 172.21.15.100:48766) 2024-09-17T04:31:56.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.090+0000 7f6c2d600640 1 -- 172.21.15.100:0/1930152355 learned_addr learned my addr 172.21.15.100:0/1930152355 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:56.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.090+0000 7f6c27e00640 1 --2- 172.21.15.100:0/1930152355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c28145550 0x7f6c2813acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:56.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.090+0000 7f6c2cc00640 1 --2- 172.21.15.100:0/1930152355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c28073a80 0x7f6c280b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:56.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.090+0000 7f6c2d600640 1 -- 172.21.15.100:0/1930152355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c28145550 msgr2=0x7f6c2813acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:56.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.090+0000 7f6c2d600640 1 --2- 172.21.15.100:0/1930152355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c28145550 0x7f6c2813acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:56.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.090+0000 7f6c2d600640 1 -- 172.21.15.100:0/1930152355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c28073a80 msgr2=0x7f6c280b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:56.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.090+0000 7f6c2d600640 1 --2- 172.21.15.100:0/1930152355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c28073a80 0x7f6c280b28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:56.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.090+0000 7f6c2d600640 1 -- 172.21.15.100:0/1930152355 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2815c6d0 con 0x7f6c28145a40 2024-09-17T04:31:56.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.091+0000 7f6c2d600640 1 --2- 172.21.15.100:0/1930152355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c28145a40 0x7f6c281f7190 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000c730 tx=0x7f6c2000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:56.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.091+0000 7f6c24c00640 1 -- 172.21.15.100:0/1930152355 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c20017070 con 0x7f6c28145a40 2024-09-17T04:31:56.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.091+0000 7f6c24c00640 1 -- 172.21.15.100:0/1930152355 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c20003c60 con 0x7f6c28145a40 2024-09-17T04:31:56.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.091+0000 7f6c2f600640 1 -- 172.21.15.100:0/1930152355 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c280b1a90 con 0x7f6c28145a40 2024-09-17T04:31:56.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.091+0000 7f6c24c00640 1 -- 172.21.15.100:0/1930152355 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c200052d0 con 0x7f6c28145a40 2024-09-17T04:31:56.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.091+0000 7f6c2f600640 1 -- 172.21.15.100:0/1930152355 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c2815cf00 con 0x7f6c28145a40 2024-09-17T04:31:56.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.092+0000 7f6c2f600640 1 -- 172.21.15.100:0/1930152355 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c280b17e0 con 0x7f6c28145a40 2024-09-17T04:31:56.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.093+0000 7f6c24c00640 1 -- 172.21.15.100:0/1930152355 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f6c2001a070 con 0x7f6c28145a40 2024-09-17T04:31:56.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.094+0000 7f6c24c00640 1 --2- 172.21.15.100:0/1930152355 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6c080791f0 0x7f6c0807b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:56.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.094+0000 7f6c24c00640 1 -- 172.21.15.100:0/1930152355 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7f6c2009b0d0 con 0x7f6c28145a40 2024-09-17T04:31:56.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.095+0000 7f6c2cc00640 1 --2- 172.21.15.100:0/1930152355 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6c080791f0 0x7f6c0807b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:56.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.095+0000 7f6c2cc00640 1 --2- 172.21.15.100:0/1930152355 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6c080791f0 0x7f6c0807b700 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c034000 tx=0x7f6c1c004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:56.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.099+0000 7f6c24c00640 1 -- 172.21.15.100:0/1930152355 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6c28073e90 con 0x7f6c28145a40 2024-09-17T04:31:56.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.318+0000 7f6c2f600640 1 -- 172.21.15.100:0/1930152355 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f6c2806fb40 con 0x7f6c28145a40 2024-09-17T04:31:56.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.843+0000 7f6c24c00640 1 -- 172.21.15.100:0/1930152355 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v91) ==== 122+0+0 (secure 0 0 0) 0x7f6c20062480 con 0x7f6c28145a40 2024-09-17T04:31:56.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.847+0000 7f6c2f600640 1 -- 172.21.15.100:0/1930152355 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6c080791f0 msgr2=0x7f6c0807b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:56.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.847+0000 7f6c2f600640 1 --2- 172.21.15.100:0/1930152355 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6c080791f0 0x7f6c0807b700 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c034000 tx=0x7f6c1c004730 comp rx=0 tx=0).stop 2024-09-17T04:31:56.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.847+0000 7f6c2f600640 1 -- 172.21.15.100:0/1930152355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c28145a40 msgr2=0x7f6c281f7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:56.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.847+0000 7f6c2f600640 1 --2- 172.21.15.100:0/1930152355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c28145a40 0x7f6c281f7190 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000c730 tx=0x7f6c2000cbf0 comp rx=0 tx=0).stop 2024-09-17T04:31:56.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.847+0000 7f6c2f600640 1 -- 172.21.15.100:0/1930152355 shutdown_connections 2024-09-17T04:31:56.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.847+0000 7f6c2f600640 1 --2- 172.21.15.100:0/1930152355 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6c080791f0 0x7f6c0807b700 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:56.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.847+0000 7f6c2f600640 1 --2- 172.21.15.100:0/1930152355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c28145a40 0x7f6c281f7190 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:56.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.847+0000 7f6c2f600640 1 --2- 172.21.15.100:0/1930152355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c28145550 0x7f6c2813acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:56.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.847+0000 7f6c2f600640 1 --2- 172.21.15.100:0/1930152355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c28073a80 0x7f6c280b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:56.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.847+0000 7f6c2f600640 1 -- 172.21.15.100:0/1930152355 >> 172.21.15.100:0/1930152355 conn(0x7f6c2806d1c0 msgr2=0x7f6c2813ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:56.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.847+0000 7f6c2f600640 1 -- 172.21.15.100:0/1930152355 shutdown_connections 2024-09-17T04:31:56.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:56.847+0000 7f6c2f600640 1 -- 172.21.15.100:0/1930152355 wait complete. 2024-09-17T04:31:56.870 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T04:31:56.870 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:31:57.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.092+0000 7f3313c00640 1 -- 172.21.15.100:0/3047583473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32f40052b0 msgr2=0x7f32f400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:57.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.092+0000 7f3313c00640 1 --2- 172.21.15.100:0/3047583473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32f40052b0 0x7f32f400b9c0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f330400c610 tx=0x7f32f4005830 comp rx=0 tx=0).stop 2024-09-17T04:31:57.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.092+0000 7f3311c00640 1 -- 172.21.15.100:0/3047583473 reap_dead start 2024-09-17T04:31:57.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.092+0000 7f3313c00640 1 -- 172.21.15.100:0/3047583473 shutdown_connections 2024-09-17T04:31:57.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.092+0000 7f3313c00640 1 -- 172.21.15.100:0/3047583473 >> 172.21.15.100:0/3047583473 conn(0x7f330c06d730 msgr2=0x7f330c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:57.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.093+0000 7f3313c00640 1 -- 172.21.15.100:0/3047583473 shutdown_connections 2024-09-17T04:31:57.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.093+0000 7f3313c00640 1 -- 172.21.15.100:0/3047583473 wait complete. 2024-09-17T04:31:57.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.095+0000 7f3313c00640 1 Processor -- start 2024-09-17T04:31:57.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.095+0000 7f3313c00640 1 -- start start 2024-09-17T04:31:57.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.096+0000 7f3313c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32f40052b0 0x7f330c1e3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:57.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.096+0000 7f3313c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f330c1e4e50 0x7f330c1f0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:57.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.096+0000 7f3313c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f330c156780 0x7f330c156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:57.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.096+0000 7f3313c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f330c0712b0 con 0x7f330c1e4e50 2024-09-17T04:31:57.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.096+0000 7f3313c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f330c071130 con 0x7f32f40052b0 2024-09-17T04:31:57.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.096+0000 7f3313c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f330c0ad7d0 con 0x7f330c156780 2024-09-17T04:31:57.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.097+0000 7f330be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f330c1e4e50 0x7f330c1f0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:57.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.097+0000 7f330be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f330c1e4e50 0x7f330c1f0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48788/0 (socket says 172.21.15.100:48788) 2024-09-17T04:31:57.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.097+0000 7f330be00640 1 -- 172.21.15.100:0/4220147853 learned_addr learned my addr 172.21.15.100:0/4220147853 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:57.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.097+0000 7f3311c00640 1 --2- 172.21.15.100:0/4220147853 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f330c156780 0x7f330c156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:57.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.097+0000 7f3311200640 1 --2- 172.21.15.100:0/4220147853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32f40052b0 0x7f330c1e3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:57.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.097+0000 7f330be00640 1 -- 172.21.15.100:0/4220147853 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f330c156780 msgr2=0x7f330c156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:57.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.097+0000 7f330be00640 1 --2- 172.21.15.100:0/4220147853 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f330c156780 0x7f330c156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:57.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.097+0000 7f330be00640 1 -- 172.21.15.100:0/4220147853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32f40052b0 msgr2=0x7f330c1e3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:57.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.097+0000 7f330be00640 1 --2- 172.21.15.100:0/4220147853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32f40052b0 0x7f330c1e3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:57.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.097+0000 7f330be00640 1 -- 172.21.15.100:0/4220147853 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330c1570d0 con 0x7f330c1e4e50 2024-09-17T04:31:57.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.097+0000 7f3311200640 1 --2- 172.21.15.100:0/4220147853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32f40052b0 0x7f330c1e3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:31:57.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.097+0000 7f330be00640 1 --2- 172.21.15.100:0/4220147853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f330c1e4e50 0x7f330c1f0ed0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f32f800e9c0 tx=0x7f32f800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:57.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.098+0000 7f3308c00640 1 -- 172.21.15.100:0/4220147853 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f801c070 con 0x7f330c1e4e50 2024-09-17T04:31:57.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.098+0000 7f3308c00640 1 -- 172.21.15.100:0/4220147853 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32f800cc80 con 0x7f330c1e4e50 2024-09-17T04:31:57.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.098+0000 7f3308c00640 1 -- 172.21.15.100:0/4220147853 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f8017650 con 0x7f330c1e4e50 2024-09-17T04:31:57.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.098+0000 7f3313c00640 1 -- 172.21.15.100:0/4220147853 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f330c13a7c0 con 0x7f330c1e4e50 2024-09-17T04:31:57.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.098+0000 7f3313c00640 1 -- 172.21.15.100:0/4220147853 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f330c13ad80 con 0x7f330c1e4e50 2024-09-17T04:31:57.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.099+0000 7f3313c00640 1 -- 172.21.15.100:0/4220147853 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f330c073040 con 0x7f330c1e4e50 2024-09-17T04:31:57.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.100+0000 7f3308c00640 1 -- 172.21.15.100:0/4220147853 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f32f801f070 con 0x7f330c1e4e50 2024-09-17T04:31:57.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.101+0000 7f3308c00640 1 --2- 172.21.15.100:0/4220147853 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f32ec0791f0 0x7f32ec07b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:57.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.101+0000 7f3308c00640 1 -- 172.21.15.100:0/4220147853 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7f32f809bc40 con 0x7f330c1e4e50 2024-09-17T04:31:57.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.101+0000 7f3311200640 1 --2- 172.21.15.100:0/4220147853 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f32ec0791f0 0x7f32ec07b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:57.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.102+0000 7f3311200640 1 --2- 172.21.15.100:0/4220147853 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f32ec0791f0 0x7f32ec07b700 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3300006550 tx=0x7f3300006b10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:57.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.106+0000 7f3308c00640 1 -- 172.21.15.100:0/4220147853 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f330c1570d0 con 0x7f330c1e4e50 2024-09-17T04:31:57.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.322+0000 7f3313c00640 1 -- 172.21.15.100:0/4220147853 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f330c15e420 con 0x7f330c1e4e50 2024-09-17T04:31:57.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.323+0000 7f3308c00640 1 -- 172.21.15.100:0/4220147853 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5239 (secure 0 0 0) 0x7f32f8062f60 con 0x7f330c1e4e50 2024-09-17T04:31:57.324 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:57.324 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":91,"btime":"2024-09-17T04:31:56:835735+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6719,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1392663867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1392663867},{"type":"v1","addr":"172.21.15.184:6801","nonce":1392663867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":7516,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3257629487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3257629487},{"type":"v1","addr":"172.21.15.184:6835","nonce":3257629487}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":7962,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/2131565869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2131565869},{"type":"v1","addr":"172.21.15.100:6834","nonce":2131565869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:31:53.292918+0000","modified":"2024-09-17T04:31:56.320263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7965},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7965":{"gid":7965,"name":"c","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3047460644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":3047460644},{"type":"v1","addr":"172.21.15.100:6835","nonce":3047460644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7965,"qdb_cluster":[7965]},"id":10}]} 2024-09-17T04:31:57.324 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 91 2024-09-17T04:31:57.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.325+0000 7f3313c00640 1 -- 172.21.15.100:0/4220147853 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f32ec0791f0 msgr2=0x7f32ec07b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:57.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.325+0000 7f3313c00640 1 --2- 172.21.15.100:0/4220147853 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f32ec0791f0 0x7f32ec07b700 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3300006550 tx=0x7f3300006b10 comp rx=0 tx=0).stop 2024-09-17T04:31:57.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.325+0000 7f3313c00640 1 -- 172.21.15.100:0/4220147853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f330c1e4e50 msgr2=0x7f330c1f0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:57.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.325+0000 7f3313c00640 1 --2- 172.21.15.100:0/4220147853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f330c1e4e50 0x7f330c1f0ed0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f32f800e9c0 tx=0x7f32f800ee80 comp rx=0 tx=0).stop 2024-09-17T04:31:57.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.325+0000 7f3313c00640 1 -- 172.21.15.100:0/4220147853 shutdown_connections 2024-09-17T04:31:57.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.325+0000 7f3313c00640 1 --2- 172.21.15.100:0/4220147853 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f32ec0791f0 0x7f32ec07b700 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:57.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.325+0000 7f3313c00640 1 --2- 172.21.15.100:0/4220147853 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f330c156780 0x7f330c156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:57.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.325+0000 7f3313c00640 1 --2- 172.21.15.100:0/4220147853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f330c1e4e50 0x7f330c1f0ed0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:57.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.325+0000 7f3313c00640 1 --2- 172.21.15.100:0/4220147853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32f40052b0 0x7f330c1e3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:57.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.325+0000 7f3313c00640 1 -- 172.21.15.100:0/4220147853 >> 172.21.15.100:0/4220147853 conn(0x7f330c06d730 msgr2=0x7f330c073c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:57.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.325+0000 7f3313c00640 1 -- 172.21.15.100:0/4220147853 shutdown_connections 2024-09-17T04:31:57.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.325+0000 7f3313c00640 1 -- 172.21.15.100:0/4220147853 wait complete. 2024-09-17T04:31:57.346 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:31:57.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.565+0000 7f3bb5c00640 1 -- 172.21.15.100:0/1385340742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bb00cd8f0 msgr2=0x7f3bb006d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:57.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.565+0000 7f3bb5c00640 1 --2- 172.21.15.100:0/1385340742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bb00cd8f0 0x7f3bb006d2e0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0073c90 tx=0x7f3b9802a530 comp rx=0 tx=0).stop 2024-09-17T04:31:57.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.565+0000 7f3bb5c00640 1 -- 172.21.15.100:0/1385340742 shutdown_connections 2024-09-17T04:31:57.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.565+0000 7f3bb5c00640 1 --2- 172.21.15.100:0/1385340742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bb00cd8f0 0x7f3bb006d2e0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:57.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.565+0000 7f3bb5c00640 1 --2- 172.21.15.100:0/1385340742 >> v2:172.21.15.184:3300/0 conn(0x7f3bb00b2360 0x7f3bb006cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:57.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.565+0000 7f3bb5c00640 1 -- 172.21.15.100:0/1385340742 >> 172.21.15.100:0/1385340742 conn(0x7f3bb00b2fd0 msgr2=0x7f3bb00b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:57.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.566+0000 7f3bb5c00640 1 -- 172.21.15.100:0/1385340742 shutdown_connections 2024-09-17T04:31:57.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.566+0000 7f3bb5c00640 1 -- 172.21.15.100:0/1385340742 wait complete. 2024-09-17T04:31:57.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.568+0000 7f3bb5c00640 1 Processor -- start 2024-09-17T04:31:57.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.568+0000 7f3bb5c00640 1 -- start start 2024-09-17T04:31:57.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.569+0000 7f3bb5c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3bb00b2360 0x7f3bb00cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:57.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.569+0000 7f3bb5c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bb00cd8f0 0x7f3bb01f93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:57.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.569+0000 7f3bb5c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bb015c060 0x7f3bb015acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:57.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.569+0000 7f3bb5c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb00b5000 con 0x7f3bb015c060 2024-09-17T04:31:57.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.569+0000 7f3bb5c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb00b4e80 con 0x7f3bb00cd8f0 2024-09-17T04:31:57.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.569+0000 7f3bb5c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3bb00b5180 con 0x7f3bb00b2360 2024-09-17T04:31:57.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.570+0000 7f3baf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bb015c060 0x7f3bb015acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:57.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.570+0000 7f3baf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bb015c060 0x7f3bb015acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48808/0 (socket says 172.21.15.100:48808) 2024-09-17T04:31:57.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.570+0000 7f3baf400640 1 -- 172.21.15.100:0/1902557163 learned_addr learned my addr 172.21.15.100:0/1902557163 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:57.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.570+0000 7f3ba7e00640 1 --2- 172.21.15.100:0/1902557163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bb00cd8f0 0x7f3bb01f93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:57.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.570+0000 7f3baea00640 1 --2- 172.21.15.100:0/1902557163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3bb00b2360 0x7f3bb00cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:57.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.570+0000 7f3baf400640 1 -- 172.21.15.100:0/1902557163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3bb00b2360 msgr2=0x7f3bb00cadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:57.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.570+0000 7f3baf400640 1 --2- 172.21.15.100:0/1902557163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3bb00b2360 0x7f3bb00cadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:57.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.570+0000 7f3baf400640 1 -- 172.21.15.100:0/1902557163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bb00cd8f0 msgr2=0x7f3bb01f93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:57.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.570+0000 7f3baf400640 1 --2- 172.21.15.100:0/1902557163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bb00cd8f0 0x7f3bb01f93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:57.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.570+0000 7f3baf400640 1 -- 172.21.15.100:0/1902557163 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb01f9900 con 0x7f3bb015c060 2024-09-17T04:31:57.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.570+0000 7f3ba7e00640 1 --2- 172.21.15.100:0/1902557163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bb00cd8f0 0x7f3bb01f93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:31:57.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.570+0000 7f3baea00640 1 --2- 172.21.15.100:0/1902557163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3bb00b2360 0x7f3bb00cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:31:57.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.570+0000 7f3baf400640 1 --2- 172.21.15.100:0/1902557163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bb015c060 0x7f3bb015acd0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800cae0 tx=0x7f3ba800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:57.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.571+0000 7f3ba6a00640 1 -- 172.21.15.100:0/1902557163 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8016020 con 0x7f3bb015c060 2024-09-17T04:31:57.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.571+0000 7f3ba6a00640 1 -- 172.21.15.100:0/1902557163 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ba800ad30 con 0x7f3bb015c060 2024-09-17T04:31:57.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.571+0000 7f3bb5c00640 1 -- 172.21.15.100:0/1902557163 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb00d0b20 con 0x7f3bb015c060 2024-09-17T04:31:57.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.571+0000 7f3ba6a00640 1 -- 172.21.15.100:0/1902557163 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8012680 con 0x7f3bb015c060 2024-09-17T04:31:57.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.571+0000 7f3bb5c00640 1 -- 172.21.15.100:0/1902557163 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb01f8bc0 con 0x7f3bb015c060 2024-09-17T04:31:57.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.572+0000 7f3bb5c00640 1 -- 172.21.15.100:0/1902557163 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb00cf420 con 0x7f3bb015c060 2024-09-17T04:31:57.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.573+0000 7f3ba6a00640 1 -- 172.21.15.100:0/1902557163 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f3ba8019020 con 0x7f3bb015c060 2024-09-17T04:31:57.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.574+0000 7f3ba6a00640 1 --2- 172.21.15.100:0/1902557163 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3b8c0791c0 0x7f3b8c07b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:57.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.574+0000 7f3ba6a00640 1 -- 172.21.15.100:0/1902557163 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7f3ba809c010 con 0x7f3bb015c060 2024-09-17T04:31:57.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.574+0000 7f3baea00640 1 --2- 172.21.15.100:0/1902557163 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3b8c0791c0 0x7f3b8c07b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:57.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.575+0000 7f3baea00640 1 --2- 172.21.15.100:0/1902557163 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3b8c0791c0 0x7f3b8c07b6d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009b80 tx=0x7f3ba0005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:57.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.579+0000 7f3ba6a00640 1 -- 172.21.15.100:0/1902557163 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3ba8020020 con 0x7f3bb015c060 2024-09-17T04:31:57.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.763+0000 7f3bb5c00640 1 -- 172.21.15.100:0/1902557163 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3bb00d1040 con 0x7f3bb015c060 2024-09-17T04:31:57.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.764+0000 7f3ba6a00640 1 -- 172.21.15.100:0/1902557163 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) ==== 74+0+16876 (secure 0 0 0) 0x7f3ba8063980 con 0x7f3bb015c060 2024-09-17T04:31:57.765 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:57.765 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":132,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:31:54.801374+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:31:51.164879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-17T04:31:52.259089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:31:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.767+0000 7f3bb5c00640 1 -- 172.21.15.100:0/1902557163 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3b8c0791c0 msgr2=0x7f3b8c07b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.767+0000 7f3bb5c00640 1 --2- 172.21.15.100:0/1902557163 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3b8c0791c0 0x7f3b8c07b6d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009b80 tx=0x7f3ba0005f90 comp rx=0 tx=0).stop 2024-09-17T04:31:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.767+0000 7f3bb5c00640 1 -- 172.21.15.100:0/1902557163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bb015c060 msgr2=0x7f3bb015acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.767+0000 7f3bb5c00640 1 --2- 172.21.15.100:0/1902557163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bb015c060 0x7f3bb015acd0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800cae0 tx=0x7f3ba800a060 comp rx=0 tx=0).stop 2024-09-17T04:31:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.767+0000 7f3bb5c00640 1 -- 172.21.15.100:0/1902557163 shutdown_connections 2024-09-17T04:31:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.767+0000 7f3bb5c00640 1 --2- 172.21.15.100:0/1902557163 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3b8c0791c0 0x7f3b8c07b6d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.767+0000 7f3bb5c00640 1 --2- 172.21.15.100:0/1902557163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bb015c060 0x7f3bb015acd0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.767+0000 7f3bb5c00640 1 --2- 172.21.15.100:0/1902557163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bb00cd8f0 0x7f3bb01f93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.767+0000 7f3bb5c00640 1 --2- 172.21.15.100:0/1902557163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3bb00b2360 0x7f3bb00cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.767+0000 7f3bb5c00640 1 -- 172.21.15.100:0/1902557163 >> 172.21.15.100:0/1902557163 conn(0x7f3bb00b2fd0 msgr2=0x7f3bb0071240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.767+0000 7f3bb5c00640 1 -- 172.21.15.100:0/1902557163 shutdown_connections 2024-09-17T04:31:57.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:57.767+0000 7f3bb5c00640 1 -- 172.21.15.100:0/1902557163 wait complete. 2024-09-17T04:31:57.792 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T04:31:57.793 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:31:58.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.016+0000 7f2e8e600640 1 -- 172.21.15.100:0/1656305395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e881623a0 msgr2=0x7f2e881627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:58.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.016+0000 7f2e8e600640 1 --2- 172.21.15.100:0/1656305395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e881623a0 0x7f2e881627b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2e74009830 tx=0x7f2e7402a410 comp rx=0 tx=0).stop 2024-09-17T04:31:58.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.016+0000 7f2e8e600640 1 -- 172.21.15.100:0/1656305395 shutdown_connections 2024-09-17T04:31:58.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.016+0000 7f2e8e600640 1 --2- 172.21.15.100:0/1656305395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e881623a0 0x7f2e881627b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:58.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.016+0000 7f2e8e600640 1 --2- 172.21.15.100:0/1656305395 >> v2:172.21.15.184:3300/0 conn(0x7f2e8806f280 0x7f2e8806f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:58.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.016+0000 7f2e8e600640 1 -- 172.21.15.100:0/1656305395 >> 172.21.15.100:0/1656305395 conn(0x7f2e880b1e40 msgr2=0x7f2e880b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:58.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.016+0000 7f2e8e600640 1 -- 172.21.15.100:0/1656305395 shutdown_connections 2024-09-17T04:31:58.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.016+0000 7f2e8e600640 1 -- 172.21.15.100:0/1656305395 wait complete. 2024-09-17T04:31:58.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.018+0000 7f2e8e600640 1 Processor -- start 2024-09-17T04:31:58.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.018+0000 7f2e8e600640 1 -- start start 2024-09-17T04:31:58.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.019+0000 7f2e8e600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e8806f280 0x7f2e8814fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:58.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.019+0000 7f2e8e600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e88155370 0x7f2e881f91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:58.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.019+0000 7f2e8e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e881623a0 0x7f2e880cff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:58.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.019+0000 7f2e8e600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2e880b3a10 con 0x7f2e881623a0 2024-09-17T04:31:58.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.019+0000 7f2e8e600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2e880b3890 con 0x7f2e8806f280 2024-09-17T04:31:58.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.019+0000 7f2e8e600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2e880b3b90 con 0x7f2e88155370 2024-09-17T04:31:58.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.020+0000 7f2e8dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e881623a0 0x7f2e880cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:58.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.020+0000 7f2e8dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e881623a0 0x7f2e880cff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48810/0 (socket says 172.21.15.100:48810) 2024-09-17T04:31:58.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.020+0000 7f2e8dc00640 1 -- 172.21.15.100:0/2905780576 learned_addr learned my addr 172.21.15.100:0/2905780576 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:58.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.020+0000 7f2e87e00640 1 --2- 172.21.15.100:0/2905780576 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e88155370 0x7f2e881f91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:58.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.020+0000 7f2e8d200640 1 --2- 172.21.15.100:0/2905780576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e8806f280 0x7f2e8814fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:58.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.020+0000 7f2e8dc00640 1 -- 172.21.15.100:0/2905780576 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e88155370 msgr2=0x7f2e881f91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:58.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.020+0000 7f2e8dc00640 1 --2- 172.21.15.100:0/2905780576 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e88155370 0x7f2e881f91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:58.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.020+0000 7f2e8dc00640 1 -- 172.21.15.100:0/2905780576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e8806f280 msgr2=0x7f2e8814fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:58.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.020+0000 7f2e8dc00640 1 --2- 172.21.15.100:0/2905780576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e8806f280 0x7f2e8814fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:58.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.020+0000 7f2e8dc00640 1 -- 172.21.15.100:0/2905780576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e881f9730 con 0x7f2e881623a0 2024-09-17T04:31:58.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.020+0000 7f2e8d200640 1 --2- 172.21.15.100:0/2905780576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e8806f280 0x7f2e8814fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:31:58.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.020+0000 7f2e8dc00640 1 --2- 172.21.15.100:0/2905780576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e881623a0 0x7f2e880cff40 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000cbb0 tx=0x7f2e8000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:58.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.021+0000 7f2e84c00640 1 -- 172.21.15.100:0/2905780576 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e80016020 con 0x7f2e881623a0 2024-09-17T04:31:58.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.021+0000 7f2e84c00640 1 -- 172.21.15.100:0/2905780576 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e8000ae50 con 0x7f2e881623a0 2024-09-17T04:31:58.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.021+0000 7f2e8e600640 1 -- 172.21.15.100:0/2905780576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e88153be0 con 0x7f2e881623a0 2024-09-17T04:31:58.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.021+0000 7f2e84c00640 1 -- 172.21.15.100:0/2905780576 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e80012680 con 0x7f2e881623a0 2024-09-17T04:31:58.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.021+0000 7f2e8e600640 1 -- 172.21.15.100:0/2905780576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e881f5670 con 0x7f2e881623a0 2024-09-17T04:31:58.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.022+0000 7f2e8e600640 1 -- 172.21.15.100:0/2905780576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e88153950 con 0x7f2e881623a0 2024-09-17T04:31:58.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.023+0000 7f2e84c00640 1 -- 172.21.15.100:0/2905780576 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f2e80019020 con 0x7f2e881623a0 2024-09-17T04:31:58.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.024+0000 7f2e84c00640 1 --2- 172.21.15.100:0/2905780576 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2e680791c0 0x7f2e6807b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:58.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.024+0000 7f2e84c00640 1 -- 172.21.15.100:0/2905780576 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7f2e8009c090 con 0x7f2e881623a0 2024-09-17T04:31:58.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.024+0000 7f2e8d200640 1 --2- 172.21.15.100:0/2905780576 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2e680791c0 0x7f2e6807b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:58.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.025+0000 7f2e8d200640 1 --2- 172.21.15.100:0/2905780576 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2e680791c0 0x7f2e6807b6d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c009b80 tx=0x7f2e7c005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:58.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.028+0000 7f2e84c00640 1 -- 172.21.15.100:0/2905780576 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f2e80020020 con 0x7f2e881623a0 2024-09-17T04:31:58.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.206+0000 7f2e8e600640 1 -- 172.21.15.100:0/2905780576 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2e88153730 con 0x7f2e680791c0 2024-09-17T04:31:58.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.211+0000 7f2e84c00640 1 -- 172.21.15.100:0/2905780576 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252481 (secure 0 0 0) 0x7f2e88153730 con 0x7f2e680791c0 2024-09-17T04:31:58.215 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:58.217 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:31:58.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.214+0000 7f2e8e600640 1 -- 172.21.15.100:0/2905780576 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2e680791c0 msgr2=0x7f2e6807b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:58.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.214+0000 7f2e8e600640 1 --2- 172.21.15.100:0/2905780576 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2e680791c0 0x7f2e6807b6d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c009b80 tx=0x7f2e7c005f90 comp rx=0 tx=0).stop 2024-09-17T04:31:58.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.214+0000 7f2e8e600640 1 -- 172.21.15.100:0/2905780576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e881623a0 msgr2=0x7f2e880cff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:58.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.214+0000 7f2e8e600640 1 --2- 172.21.15.100:0/2905780576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e881623a0 0x7f2e880cff40 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000cbb0 tx=0x7f2e8000a430 comp rx=0 tx=0).stop 2024-09-17T04:31:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.214+0000 7f2e8e600640 1 -- 172.21.15.100:0/2905780576 shutdown_connections 2024-09-17T04:31:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.214+0000 7f2e8e600640 1 --2- 172.21.15.100:0/2905780576 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2e680791c0 0x7f2e6807b6d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.214+0000 7f2e8e600640 1 --2- 172.21.15.100:0/2905780576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e881623a0 0x7f2e880cff40 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.214+0000 7f2e8e600640 1 --2- 172.21.15.100:0/2905780576 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e88155370 0x7f2e881f91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.214+0000 7f2e8e600640 1 --2- 172.21.15.100:0/2905780576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e8806f280 0x7f2e8814fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.214+0000 7f2e8e600640 1 -- 172.21.15.100:0/2905780576 >> 172.21.15.100:0/2905780576 conn(0x7f2e880b1e40 msgr2=0x7f2e880b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.215+0000 7f2e8e600640 1 -- 172.21.15.100:0/2905780576 shutdown_connections 2024-09-17T04:31:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.215+0000 7f2e8e600640 1 -- 172.21.15.100:0/2905780576 wait complete. 2024-09-17T04:31:58.241 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2024-09-17T04:31:57.698199+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":147,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263584,"kb_used_data":14056,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711136,"statfs":{"total":773094113280,"available":772824203264,"internally_reserved":0,"allocated":14393344,"data_stored":6971860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.909245"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.810355+0000","last_change":"2024-09-17T04:31:53.830592+0000","last_active":"2024-09-17T04:31:54.810355+0000","last_peered":"2024-09-17T04:31:54.810355+0000","last_clean":"2024-09-17T04:31:54.810355+0000","last_became_active":"2024-09-17T04:31:53.830274+0000","last_became_peered":"2024-09-17T04:31:53.830274+0000","last_unstale":"2024-09-17T04:31:54.810355+0000","last_undegraded":"2024-09-17T04:31:54.810355+0000","last_fullsized":"2024-09-17T04:31:54.810355+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:21.313063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.806256+0000","last_change":"2024-09-17T04:31:53.831696+0000","last_active":"2024-09-17T04:31:54.806256+0000","last_peered":"2024-09-17T04:31:54.806256+0000","last_clean":"2024-09-17T04:31:54.806256+0000","last_became_active":"2024-09-17T04:31:53.831353+0000","last_became_peered":"2024-09-17T04:31:53.831353+0000","last_unstale":"2024-09-17T04:31:54.806256+0000","last_undegraded":"2024-09-17T04:31:54.806256+0000","last_fullsized":"2024-09-17T04:31:54.806256+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:11.579919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.201015+0000","last_change":"2024-09-17T04:31:53.811930+0000","last_active":"2024-09-17T04:31:55.201015+0000","last_peered":"2024-09-17T04:31:55.201015+0000","last_clean":"2024-09-17T04:31:55.201015+0000","last_became_active":"2024-09-17T04:31:53.811139+0000","last_became_peered":"2024-09-17T04:31:53.811139+0000","last_unstale":"2024-09-17T04:31:55.201015+0000","last_undegraded":"2024-09-17T04:31:55.201015+0000","last_fullsized":"2024-09-17T04:31:55.201015+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:51.541898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.810655+0000","last_change":"2024-09-17T04:31:53.819868+0000","last_active":"2024-09-17T04:31:54.810655+0000","last_peered":"2024-09-17T04:31:54.810655+0000","last_clean":"2024-09-17T04:31:54.810655+0000","last_became_active":"2024-09-17T04:31:53.818589+0000","last_became_peered":"2024-09-17T04:31:53.818589+0000","last_unstale":"2024-09-17T04:31:54.810655+0000","last_undegraded":"2024-09-17T04:31:54.810655+0000","last_fullsized":"2024-09-17T04:31:54.810655+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:11.977100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.201331+0000","last_change":"2024-09-17T04:31:53.840709+0000","last_active":"2024-09-17T04:31:55.201331+0000","last_peered":"2024-09-17T04:31:55.201331+0000","last_clean":"2024-09-17T04:31:55.201331+0000","last_became_active":"2024-09-17T04:31:53.839975+0000","last_became_peered":"2024-09-17T04:31:53.839975+0000","last_unstale":"2024-09-17T04:31:55.201331+0000","last_undegraded":"2024-09-17T04:31:55.201331+0000","last_fullsized":"2024-09-17T04:31:55.201331+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:01:24.938949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.809438+0000","last_change":"2024-09-17T04:31:53.823162+0000","last_active":"2024-09-17T04:31:54.809438+0000","last_peered":"2024-09-17T04:31:54.809438+0000","last_clean":"2024-09-17T04:31:54.809438+0000","last_became_active":"2024-09-17T04:31:53.822666+0000","last_became_peered":"2024-09-17T04:31:53.822666+0000","last_unstale":"2024-09-17T04:31:54.809438+0000","last_undegraded":"2024-09-17T04:31:54.809438+0000","last_fullsized":"2024-09-17T04:31:54.809438+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:04:30.245191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.862626+0000","last_change":"2024-09-17T04:31:53.826943+0000","last_active":"2024-09-17T04:31:54.862626+0000","last_peered":"2024-09-17T04:31:54.862626+0000","last_clean":"2024-09-17T04:31:54.862626+0000","last_became_active":"2024-09-17T04:31:53.826631+0000","last_became_peered":"2024-09-17T04:31:53.826631+0000","last_unstale":"2024-09-17T04:31:54.862626+0000","last_undegraded":"2024-09-17T04:31:54.862626+0000","last_fullsized":"2024-09-17T04:31:54.862626+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:54.356962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.809162+0000","last_change":"2024-09-17T04:31:53.837611+0000","last_active":"2024-09-17T04:31:54.809162+0000","last_peered":"2024-09-17T04:31:54.809162+0000","last_clean":"2024-09-17T04:31:54.809162+0000","last_became_active":"2024-09-17T04:31:53.837319+0000","last_became_peered":"2024-09-17T04:31:53.837319+0000","last_unstale":"2024-09-17T04:31:54.809162+0000","last_undegraded":"2024-09-17T04:31:54.809162+0000","last_fullsized":"2024-09-17T04:31:54.809162+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:29:59.500142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.806298+0000","last_change":"2024-09-17T04:31:53.822891+0000","last_active":"2024-09-17T04:31:54.806298+0000","last_peered":"2024-09-17T04:31:54.806298+0000","last_clean":"2024-09-17T04:31:54.806298+0000","last_became_active":"2024-09-17T04:31:53.822552+0000","last_became_peered":"2024-09-17T04:31:53.822552+0000","last_unstale":"2024-09-17T04:31:54.806298+0000","last_undegraded":"2024-09-17T04:31:54.806298+0000","last_fullsized":"2024-09-17T04:31:54.806298+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:02:38.832265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.834466+0000","last_change":"2024-09-17T04:31:53.826556+0000","last_active":"2024-09-17T04:31:54.834466+0000","last_peered":"2024-09-17T04:31:54.834466+0000","last_clean":"2024-09-17T04:31:54.834466+0000","last_became_active":"2024-09-17T04:31:53.825875+0000","last_became_peered":"2024-09-17T04:31:53.825875+0000","last_unstale":"2024-09-17T04:31:54.834466+0000","last_undegraded":"2024-09-17T04:31:54.834466+0000","last_fullsized":"2024-09-17T04:31:54.834466+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:15:08.129660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.810285+0000","last_change":"2024-09-17T04:31:53.839488+0000","last_active":"2024-09-17T04:31:54.810285+0000","last_peered":"2024-09-17T04:31:54.810285+0000","last_clean":"2024-09-17T04:31:54.810285+0000","last_became_active":"2024-09-17T04:31:53.839189+0000","last_became_peered":"2024-09-17T04:31:53.839189+0000","last_unstale":"2024-09-17T04:31:54.810285+0000","last_undegraded":"2024-09-17T04:31:54.810285+0000","last_fullsized":"2024-09-17T04:31:54.810285+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:45.195188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.833836+0000","last_change":"2024-09-17T04:31:53.836665+0000","last_active":"2024-09-17T04:31:54.833836+0000","last_peered":"2024-09-17T04:31:54.833836+0000","last_clean":"2024-09-17T04:31:54.833836+0000","last_became_active":"2024-09-17T04:31:53.835906+0000","last_became_peered":"2024-09-17T04:31:53.835906+0000","last_unstale":"2024-09-17T04:31:54.833836+0000","last_undegraded":"2024-09-17T04:31:54.833836+0000","last_fullsized":"2024-09-17T04:31:54.833836+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:39.300901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.201144+0000","last_change":"2024-09-17T04:31:53.833952+0000","last_active":"2024-09-17T04:31:55.201144+0000","last_peered":"2024-09-17T04:31:55.201144+0000","last_clean":"2024-09-17T04:31:55.201144+0000","last_became_active":"2024-09-17T04:31:53.833499+0000","last_became_peered":"2024-09-17T04:31:53.833499+0000","last_unstale":"2024-09-17T04:31:55.201144+0000","last_undegraded":"2024-09-17T04:31:55.201144+0000","last_fullsized":"2024-09-17T04:31:55.201144+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:54:59.003417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.810085+0000","last_change":"2024-09-17T04:31:53.828724+0000","last_active":"2024-09-17T04:31:54.810085+0000","last_peered":"2024-09-17T04:31:54.810085+0000","last_clean":"2024-09-17T04:31:54.810085+0000","last_became_active":"2024-09-17T04:31:53.828263+0000","last_became_peered":"2024-09-17T04:31:53.828263+0000","last_unstale":"2024-09-17T04:31:54.810085+0000","last_undegraded":"2024-09-17T04:31:54.810085+0000","last_fullsized":"2024-09-17T04:31:54.810085+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:00:53.550048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.833588+0000","last_change":"2024-09-17T04:31:53.828087+0000","last_active":"2024-09-17T04:31:54.833588+0000","last_peered":"2024-09-17T04:31:54.833588+0000","last_clean":"2024-09-17T04:31:54.833588+0000","last_became_active":"2024-09-17T04:31:53.827800+0000","last_became_peered":"2024-09-17T04:31:53.827800+0000","last_unstale":"2024-09-17T04:31:54.833588+0000","last_undegraded":"2024-09-17T04:31:54.833588+0000","last_fullsized":"2024-09-17T04:31:54.833588+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:34:39.879340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.835379+0000","last_change":"2024-09-17T04:31:53.823709+0000","last_active":"2024-09-17T04:31:54.835379+0000","last_peered":"2024-09-17T04:31:54.835379+0000","last_clean":"2024-09-17T04:31:54.835379+0000","last_became_active":"2024-09-17T04:31:53.823370+0000","last_became_peered":"2024-09-17T04:31:53.823370+0000","last_unstale":"2024-09-17T04:31:54.835379+0000","last_undegraded":"2024-09-17T04:31:54.835379+0000","last_fullsized":"2024-09-17T04:31:54.835379+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:20:37.855904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.834954+0000","last_change":"2024-09-17T04:31:53.837541+0000","last_active":"2024-09-17T04:31:54.834954+0000","last_peered":"2024-09-17T04:31:54.834954+0000","last_clean":"2024-09-17T04:31:54.834954+0000","last_became_active":"2024-09-17T04:31:53.837134+0000","last_became_peered":"2024-09-17T04:31:53.837134+0000","last_unstale":"2024-09-17T04:31:54.834954+0000","last_undegraded":"2024-09-17T04:31:54.834954+0000","last_fullsized":"2024-09-17T04:31:54.834954+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:01:03.617937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.200697+0000","last_change":"2024-09-17T04:31:53.816290+0000","last_active":"2024-09-17T04:31:55.200697+0000","last_peered":"2024-09-17T04:31:55.200697+0000","last_clean":"2024-09-17T04:31:55.200697+0000","last_became_active":"2024-09-17T04:31:53.815904+0000","last_became_peered":"2024-09-17T04:31:53.815904+0000","last_unstale":"2024-09-17T04:31:55.200697+0000","last_undegraded":"2024-09-17T04:31:55.200697+0000","last_fullsized":"2024-09-17T04:31:55.200697+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:20.432704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.809950+0000","last_change":"2024-09-17T04:31:53.826413+0000","last_active":"2024-09-17T04:31:54.809950+0000","last_peered":"2024-09-17T04:31:54.809950+0000","last_clean":"2024-09-17T04:31:54.809950+0000","last_became_active":"2024-09-17T04:31:53.825654+0000","last_became_peered":"2024-09-17T04:31:53.825654+0000","last_unstale":"2024-09-17T04:31:54.809950+0000","last_undegraded":"2024-09-17T04:31:54.809950+0000","last_fullsized":"2024-09-17T04:31:54.809950+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:43:26.938684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.835738+0000","last_change":"2024-09-17T04:31:53.810221+0000","last_active":"2024-09-17T04:31:54.835738+0000","last_peered":"2024-09-17T04:31:54.835738+0000","last_clean":"2024-09-17T04:31:54.835738+0000","last_became_active":"2024-09-17T04:31:53.809858+0000","last_became_peered":"2024-09-17T04:31:53.809858+0000","last_unstale":"2024-09-17T04:31:54.835738+0000","last_undegraded":"2024-09-17T04:31:54.835738+0000","last_fullsized":"2024-09-17T04:31:54.835738+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:04.510738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.834666+0000","last_change":"2024-09-17T04:31:53.829308+0000","last_active":"2024-09-17T04:31:54.834666+0000","last_peered":"2024-09-17T04:31:54.834666+0000","last_clean":"2024-09-17T04:31:54.834666+0000","last_became_active":"2024-09-17T04:31:53.828993+0000","last_became_peered":"2024-09-17T04:31:53.828993+0000","last_unstale":"2024-09-17T04:31:54.834666+0000","last_undegraded":"2024-09-17T04:31:54.834666+0000","last_fullsized":"2024-09-17T04:31:54.834666+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:43:26.093268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.810154+0000","last_change":"2024-09-17T04:31:53.815644+0000","last_active":"2024-09-17T04:31:54.810154+0000","last_peered":"2024-09-17T04:31:54.810154+0000","last_clean":"2024-09-17T04:31:54.810154+0000","last_became_active":"2024-09-17T04:31:53.812350+0000","last_became_peered":"2024-09-17T04:31:53.812350+0000","last_unstale":"2024-09-17T04:31:54.810154+0000","last_undegraded":"2024-09-17T04:31:54.810154+0000","last_fullsized":"2024-09-17T04:31:54.810154+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:50.305186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.809857+0000","last_change":"2024-09-17T04:31:53.831549+0000","last_active":"2024-09-17T04:31:54.809857+0000","last_peered":"2024-09-17T04:31:54.809857+0000","last_clean":"2024-09-17T04:31:54.809857+0000","last_became_active":"2024-09-17T04:31:53.830866+0000","last_became_peered":"2024-09-17T04:31:53.830866+0000","last_unstale":"2024-09-17T04:31:54.809857+0000","last_undegraded":"2024-09-17T04:31:54.809857+0000","last_fullsized":"2024-09-17T04:31:54.809857+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:59.465298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.833529+0000","last_change":"2024-09-17T04:31:53.831401+0000","last_active":"2024-09-17T04:31:54.833529+0000","last_peered":"2024-09-17T04:31:54.833529+0000","last_clean":"2024-09-17T04:31:54.833529+0000","last_became_active":"2024-09-17T04:31:53.830298+0000","last_became_peered":"2024-09-17T04:31:53.830298+0000","last_unstale":"2024-09-17T04:31:54.833529+0000","last_undegraded":"2024-09-17T04:31:54.833529+0000","last_fullsized":"2024-09-17T04:31:54.833529+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:11:07.418895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.835869+0000","last_change":"2024-09-17T04:31:53.817133+0000","last_active":"2024-09-17T04:31:54.835869+0000","last_peered":"2024-09-17T04:31:54.835869+0000","last_clean":"2024-09-17T04:31:54.835869+0000","last_became_active":"2024-09-17T04:31:53.816793+0000","last_became_peered":"2024-09-17T04:31:53.816793+0000","last_unstale":"2024-09-17T04:31:54.835869+0000","last_undegraded":"2024-09-17T04:31:54.835869+0000","last_fullsized":"2024-09-17T04:31:54.835869+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:24:41.034403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.810334+0000","last_change":"2024-09-17T04:31:53.824497+0000","last_active":"2024-09-17T04:31:54.810334+0000","last_peered":"2024-09-17T04:31:54.810334+0000","last_clean":"2024-09-17T04:31:54.810334+0000","last_became_active":"2024-09-17T04:31:53.824170+0000","last_became_peered":"2024-09-17T04:31:53.824170+0000","last_unstale":"2024-09-17T04:31:54.810334+0000","last_undegraded":"2024-09-17T04:31:54.810334+0000","last_fullsized":"2024-09-17T04:31:54.810334+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:11.415094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.835730+0000","last_change":"2024-09-17T04:31:53.814196+0000","last_active":"2024-09-17T04:31:54.835730+0000","last_peered":"2024-09-17T04:31:54.835730+0000","last_clean":"2024-09-17T04:31:54.835730+0000","last_became_active":"2024-09-17T04:31:53.813831+0000","last_became_peered":"2024-09-17T04:31:53.813831+0000","last_unstale":"2024-09-17T04:31:54.835730+0000","last_undegraded":"2024-09-17T04:31:54.835730+0000","last_fullsized":"2024-09-17T04:31:54.835730+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:29.115890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.862513+0000","last_change":"2024-09-17T04:31:53.832885+0000","last_active":"2024-09-17T04:31:54.862513+0000","last_peered":"2024-09-17T04:31:54.862513+0000","last_clean":"2024-09-17T04:31:54.862513+0000","last_became_active":"2024-09-17T04:31:53.832521+0000","last_became_peered":"2024-09-17T04:31:53.832521+0000","last_unstale":"2024-09-17T04:31:54.862513+0000","last_undegraded":"2024-09-17T04:31:54.862513+0000","last_fullsized":"2024-09-17T04:31:54.862513+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:31:26.782803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.809413+0000","last_change":"2024-09-17T04:31:53.820157+0000","last_active":"2024-09-17T04:31:54.809413+0000","last_peered":"2024-09-17T04:31:54.809413+0000","last_clean":"2024-09-17T04:31:54.809413+0000","last_became_active":"2024-09-17T04:31:53.819866+0000","last_became_peered":"2024-09-17T04:31:53.819866+0000","last_unstale":"2024-09-17T04:31:54.809413+0000","last_undegraded":"2024-09-17T04:31:54.809413+0000","last_fullsized":"2024-09-17T04:31:54.809413+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:19.747207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.810495+0000","last_change":"2024-09-17T04:31:53.826969+0000","last_active":"2024-09-17T04:31:54.810495+0000","last_peered":"2024-09-17T04:31:54.810495+0000","last_clean":"2024-09-17T04:31:54.810495+0000","last_became_active":"2024-09-17T04:31:53.826485+0000","last_became_peered":"2024-09-17T04:31:53.826485+0000","last_unstale":"2024-09-17T04:31:54.810495+0000","last_undegraded":"2024-09-17T04:31:54.810495+0000","last_fullsized":"2024-09-17T04:31:54.810495+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:58:52.207651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.806089+0000","last_change":"2024-09-17T04:31:53.814804+0000","last_active":"2024-09-17T04:31:54.806089+0000","last_peered":"2024-09-17T04:31:54.806089+0000","last_clean":"2024-09-17T04:31:54.806089+0000","last_became_active":"2024-09-17T04:31:53.813106+0000","last_became_peered":"2024-09-17T04:31:53.813106+0000","last_unstale":"2024-09-17T04:31:54.806089+0000","last_undegraded":"2024-09-17T04:31:54.806089+0000","last_fullsized":"2024-09-17T04:31:54.806089+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:18:42.930314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.835457+0000","last_change":"2024-09-17T04:31:53.833697+0000","last_active":"2024-09-17T04:31:54.835457+0000","last_peered":"2024-09-17T04:31:54.835457+0000","last_clean":"2024-09-17T04:31:54.835457+0000","last_became_active":"2024-09-17T04:31:53.833393+0000","last_became_peered":"2024-09-17T04:31:53.833393+0000","last_unstale":"2024-09-17T04:31:54.835457+0000","last_undegraded":"2024-09-17T04:31:54.835457+0000","last_fullsized":"2024-09-17T04:31:54.835457+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:52.632954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.834481+0000","last_change":"2024-09-17T04:31:53.833787+0000","last_active":"2024-09-17T04:31:54.834481+0000","last_peered":"2024-09-17T04:31:54.834481+0000","last_clean":"2024-09-17T04:31:54.834481+0000","last_became_active":"2024-09-17T04:31:53.833467+0000","last_became_peered":"2024-09-17T04:31:53.833467+0000","last_unstale":"2024-09-17T04:31:54.834481+0000","last_undegraded":"2024-09-17T04:31:54.834481+0000","last_fullsized":"2024-09-17T04:31:54.834481+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:35:45.421975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"131'4","reported_seq":19,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.834841+0000","last_change":"2024-09-17T04:31:52.821727+0000","last_active":"2024-09-17T04:31:54.834841+0000","last_peered":"2024-09-17T04:31:54.834841+0000","last_clean":"2024-09-17T04:31:54.834841+0000","last_became_active":"2024-09-17T04:31:52.821463+0000","last_became_peered":"2024-09-17T04:31:52.821463+0000","last_unstale":"2024-09-17T04:31:54.834841+0000","last_undegraded":"2024-09-17T04:31:54.834841+0000","last_fullsized":"2024-09-17T04:31:54.834841+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:56.045866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.806341+0000","last_change":"2024-09-17T04:31:53.831239+0000","last_active":"2024-09-17T04:31:54.806341+0000","last_peered":"2024-09-17T04:31:54.806341+0000","last_clean":"2024-09-17T04:31:54.806341+0000","last_became_active":"2024-09-17T04:31:53.830910+0000","last_became_peered":"2024-09-17T04:31:53.830910+0000","last_unstale":"2024-09-17T04:31:54.806341+0000","last_undegraded":"2024-09-17T04:31:54.806341+0000","last_fullsized":"2024-09-17T04:31:54.806341+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:21:32.999335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.806311+0000","last_change":"2024-09-17T04:31:52.818664+0000","last_active":"2024-09-17T04:31:54.806311+0000","last_peered":"2024-09-17T04:31:54.806311+0000","last_clean":"2024-09-17T04:31:54.806311+0000","last_became_active":"2024-09-17T04:31:52.818284+0000","last_became_peered":"2024-09-17T04:31:52.818284+0000","last_unstale":"2024-09-17T04:31:54.806311+0000","last_undegraded":"2024-09-17T04:31:54.806311+0000","last_fullsized":"2024-09-17T04:31:54.806311+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:15.782451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.809389+0000","last_change":"2024-09-17T04:31:53.828826+0000","last_active":"2024-09-17T04:31:54.809389+0000","last_peered":"2024-09-17T04:31:54.809389+0000","last_clean":"2024-09-17T04:31:54.809389+0000","last_became_active":"2024-09-17T04:31:53.825022+0000","last_became_peered":"2024-09-17T04:31:53.825022+0000","last_unstale":"2024-09-17T04:31:54.809389+0000","last_undegraded":"2024-09-17T04:31:54.809389+0000","last_fullsized":"2024-09-17T04:31:54.809389+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:44:22.026471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.810701+0000","last_change":"2024-09-17T04:31:52.808269+0000","last_active":"2024-09-17T04:31:54.810701+0000","last_peered":"2024-09-17T04:31:54.810701+0000","last_clean":"2024-09-17T04:31:54.810701+0000","last_became_active":"2024-09-17T04:31:52.807984+0000","last_became_peered":"2024-09-17T04:31:52.807984+0000","last_unstale":"2024-09-17T04:31:54.810701+0000","last_undegraded":"2024-09-17T04:31:54.810701+0000","last_fullsized":"2024-09-17T04:31:54.810701+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:58.445527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.834599+0000","last_change":"2024-09-17T04:31:53.836445+0000","last_active":"2024-09-17T04:31:54.834599+0000","last_peered":"2024-09-17T04:31:54.834599+0000","last_clean":"2024-09-17T04:31:54.834599+0000","last_became_active":"2024-09-17T04:31:53.836101+0000","last_became_peered":"2024-09-17T04:31:53.836101+0000","last_unstale":"2024-09-17T04:31:54.834599+0000","last_undegraded":"2024-09-17T04:31:54.834599+0000","last_fullsized":"2024-09-17T04:31:54.834599+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:04.612966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.810531+0000","last_change":"2024-09-17T04:31:52.815226+0000","last_active":"2024-09-17T04:31:54.810531+0000","last_peered":"2024-09-17T04:31:54.810531+0000","last_clean":"2024-09-17T04:31:54.810531+0000","last_became_active":"2024-09-17T04:31:52.814650+0000","last_became_peered":"2024-09-17T04:31:52.814650+0000","last_unstale":"2024-09-17T04:31:54.810531+0000","last_undegraded":"2024-09-17T04:31:54.810531+0000","last_fullsized":"2024-09-17T04:31:54.810531+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:16.085871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.806223+0000","last_change":"2024-09-17T04:31:53.814739+0000","last_active":"2024-09-17T04:31:54.806223+0000","last_peered":"2024-09-17T04:31:54.806223+0000","last_clean":"2024-09-17T04:31:54.806223+0000","last_became_active":"2024-09-17T04:31:53.814402+0000","last_became_peered":"2024-09-17T04:31:53.814402+0000","last_unstale":"2024-09-17T04:31:54.806223+0000","last_undegraded":"2024-09-17T04:31:54.806223+0000","last_fullsized":"2024-09-17T04:31:54.806223+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:33:14.071639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.835473+0000","last_change":"2024-09-17T04:31:52.814054+0000","last_active":"2024-09-17T04:31:54.835473+0000","last_peered":"2024-09-17T04:31:54.835473+0000","last_clean":"2024-09-17T04:31:54.835473+0000","last_became_active":"2024-09-17T04:31:52.813783+0000","last_became_peered":"2024-09-17T04:31:52.813783+0000","last_unstale":"2024-09-17T04:31:54.835473+0000","last_undegraded":"2024-09-17T04:31:54.835473+0000","last_fullsized":"2024-09-17T04:31:54.835473+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:56:55.128758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.810556+0000","last_change":"2024-09-17T04:31:53.817343+0000","last_active":"2024-09-17T04:31:54.810556+0000","last_peered":"2024-09-17T04:31:54.810556+0000","last_clean":"2024-09-17T04:31:54.810556+0000","last_became_active":"2024-09-17T04:31:53.816981+0000","last_became_peered":"2024-09-17T04:31:53.816981+0000","last_unstale":"2024-09-17T04:31:54.810556+0000","last_undegraded":"2024-09-17T04:31:54.810556+0000","last_fullsized":"2024-09-17T04:31:54.810556+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:23.163852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.201252+0000","last_change":"2024-09-17T04:31:52.814668+0000","last_active":"2024-09-17T04:31:55.201252+0000","last_peered":"2024-09-17T04:31:55.201252+0000","last_clean":"2024-09-17T04:31:55.201252+0000","last_became_active":"2024-09-17T04:31:52.814236+0000","last_became_peered":"2024-09-17T04:31:52.814236+0000","last_unstale":"2024-09-17T04:31:55.201252+0000","last_undegraded":"2024-09-17T04:31:55.201252+0000","last_fullsized":"2024-09-17T04:31:55.201252+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:47.103533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.805856+0000","last_change":"2024-09-17T04:31:53.833527+0000","last_active":"2024-09-17T04:31:54.805856+0000","last_peered":"2024-09-17T04:31:54.805856+0000","last_clean":"2024-09-17T04:31:54.805856+0000","last_became_active":"2024-09-17T04:31:53.833171+0000","last_became_peered":"2024-09-17T04:31:53.833171+0000","last_unstale":"2024-09-17T04:31:54.805856+0000","last_undegraded":"2024-09-17T04:31:54.805856+0000","last_fullsized":"2024-09-17T04:31:54.805856+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:57.441621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.835530+0000","last_change":"2024-09-17T04:31:52.808009+0000","last_active":"2024-09-17T04:31:54.835530+0000","last_peered":"2024-09-17T04:31:54.835530+0000","last_clean":"2024-09-17T04:31:54.835530+0000","last_became_active":"2024-09-17T04:31:52.807695+0000","last_became_peered":"2024-09-17T04:31:52.807695+0000","last_unstale":"2024-09-17T04:31:54.835530+0000","last_undegraded":"2024-09-17T04:31:54.835530+0000","last_fullsized":"2024-09-17T04:31:54.835530+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:43:54.196275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.201298+0000","last_change":"2024-09-17T04:31:53.833923+0000","last_active":"2024-09-17T04:31:55.201298+0000","last_peered":"2024-09-17T04:31:55.201298+0000","last_clean":"2024-09-17T04:31:55.201298+0000","last_became_active":"2024-09-17T04:31:53.833587+0000","last_became_peered":"2024-09-17T04:31:53.833587+0000","last_unstale":"2024-09-17T04:31:55.201298+0000","last_undegraded":"2024-09-17T04:31:55.201298+0000","last_fullsized":"2024-09-17T04:31:55.201298+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:27.074253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.809697+0000","last_change":"2024-09-17T04:31:52.812508+0000","last_active":"2024-09-17T04:31:54.809697+0000","last_peered":"2024-09-17T04:31:54.809697+0000","last_clean":"2024-09-17T04:31:54.809697+0000","last_became_active":"2024-09-17T04:31:52.812218+0000","last_became_peered":"2024-09-17T04:31:52.812218+0000","last_unstale":"2024-09-17T04:31:54.809697+0000","last_undegraded":"2024-09-17T04:31:54.809697+0000","last_fullsized":"2024-09-17T04:31:54.809697+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:47.649843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.201335+0000","last_change":"2024-09-17T04:31:53.829505+0000","last_active":"2024-09-17T04:31:55.201335+0000","last_peered":"2024-09-17T04:31:55.201335+0000","last_clean":"2024-09-17T04:31:55.201335+0000","last_became_active":"2024-09-17T04:31:53.827433+0000","last_became_peered":"2024-09-17T04:31:53.827433+0000","last_unstale":"2024-09-17T04:31:55.201335+0000","last_undegraded":"2024-09-17T04:31:55.201335+0000","last_fullsized":"2024-09-17T04:31:55.201335+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:20.296448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.806404+0000","last_change":"2024-09-17T04:31:52.820598+0000","last_active":"2024-09-17T04:31:54.806404+0000","last_peered":"2024-09-17T04:31:54.806404+0000","last_clean":"2024-09-17T04:31:54.806404+0000","last_became_active":"2024-09-17T04:31:52.820305+0000","last_became_peered":"2024-09-17T04:31:52.820305+0000","last_unstale":"2024-09-17T04:31:54.806404+0000","last_undegraded":"2024-09-17T04:31:54.806404+0000","last_fullsized":"2024-09-17T04:31:54.806404+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:14.825479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.834781+0000","last_change":"2024-09-17T04:31:53.832664+0000","last_active":"2024-09-17T04:31:54.834781+0000","last_peered":"2024-09-17T04:31:54.834781+0000","last_clean":"2024-09-17T04:31:54.834781+0000","last_became_active":"2024-09-17T04:31:53.832366+0000","last_became_peered":"2024-09-17T04:31:53.832366+0000","last_unstale":"2024-09-17T04:31:54.834781+0000","last_undegraded":"2024-09-17T04:31:54.834781+0000","last_fullsized":"2024-09-17T04:31:54.834781+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:50:40.198578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.834649+0000","last_change":"2024-09-17T04:31:52.817353+0000","last_active":"2024-09-17T04:31:54.834649+0000","last_peered":"2024-09-17T04:31:54.834649+0000","last_clean":"2024-09-17T04:31:54.834649+0000","last_became_active":"2024-09-17T04:31:52.816563+0000","last_became_peered":"2024-09-17T04:31:52.816563+0000","last_unstale":"2024-09-17T04:31:54.834649+0000","last_undegraded":"2024-09-17T04:31:54.834649+0000","last_fullsized":"2024-09-17T04:31:54.834649+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:48:23.269631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.835865+0000","last_change":"2024-09-17T04:31:53.828049+0000","last_active":"2024-09-17T04:31:54.835865+0000","last_peered":"2024-09-17T04:31:54.835865+0000","last_clean":"2024-09-17T04:31:54.835865+0000","last_became_active":"2024-09-17T04:31:53.827756+0000","last_became_peered":"2024-09-17T04:31:53.827756+0000","last_unstale":"2024-09-17T04:31:54.835865+0000","last_undegraded":"2024-09-17T04:31:54.835865+0000","last_fullsized":"2024-09-17T04:31:54.835865+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:08:00.850010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"131'6","reported_seq":21,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.862647+0000","last_change":"2024-09-17T04:31:52.813285+0000","last_active":"2024-09-17T04:31:54.862647+0000","last_peered":"2024-09-17T04:31:54.862647+0000","last_clean":"2024-09-17T04:31:54.862647+0000","last_became_active":"2024-09-17T04:31:52.812960+0000","last_became_peered":"2024-09-17T04:31:52.812960+0000","last_unstale":"2024-09-17T04:31:54.862647+0000","last_undegraded":"2024-09-17T04:31:54.862647+0000","last_fullsized":"2024-09-17T04:31:54.862647+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:38.260845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.201117+0000","last_change":"2024-09-17T04:31:53.821854+0000","last_active":"2024-09-17T04:31:55.201117+0000","last_peered":"2024-09-17T04:31:55.201117+0000","last_clean":"2024-09-17T04:31:55.201117+0000","last_became_active":"2024-09-17T04:31:53.821203+0000","last_became_peered":"2024-09-17T04:31:53.821203+0000","last_unstale":"2024-09-17T04:31:55.201117+0000","last_undegraded":"2024-09-17T04:31:55.201117+0000","last_fullsized":"2024-09-17T04:31:55.201117+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:50.305186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.200568+0000","last_change":"2024-09-17T04:31:52.809827+0000","last_active":"2024-09-17T04:31:55.200568+0000","last_peered":"2024-09-17T04:31:55.200568+0000","last_clean":"2024-09-17T04:31:55.200568+0000","last_became_active":"2024-09-17T04:31:52.808401+0000","last_became_peered":"2024-09-17T04:31:52.808401+0000","last_unstale":"2024-09-17T04:31:55.200568+0000","last_undegraded":"2024-09-17T04:31:55.200568+0000","last_fullsized":"2024-09-17T04:31:55.200568+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:14.333538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.806071+0000","last_change":"2024-09-17T04:31:53.833489+0000","last_active":"2024-09-17T04:31:54.806071+0000","last_peered":"2024-09-17T04:31:54.806071+0000","last_clean":"2024-09-17T04:31:54.806071+0000","last_became_active":"2024-09-17T04:31:53.833168+0000","last_became_peered":"2024-09-17T04:31:53.833168+0000","last_unstale":"2024-09-17T04:31:54.806071+0000","last_undegraded":"2024-09-17T04:31:54.806071+0000","last_fullsized":"2024-09-17T04:31:54.806071+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:18:47.551599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.200568+0000","last_change":"2024-09-17T04:31:52.805253+0000","last_active":"2024-09-17T04:31:55.200568+0000","last_peered":"2024-09-17T04:31:55.200568+0000","last_clean":"2024-09-17T04:31:55.200568+0000","last_became_active":"2024-09-17T04:31:52.804865+0000","last_became_peered":"2024-09-17T04:31:52.804865+0000","last_unstale":"2024-09-17T04:31:55.200568+0000","last_undegraded":"2024-09-17T04:31:55.200568+0000","last_fullsized":"2024-09-17T04:31:55.200568+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:05:17.527290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.200516+0000","last_change":"2024-09-17T04:31:53.839026+0000","last_active":"2024-09-17T04:31:55.200516+0000","last_peered":"2024-09-17T04:31:55.200516+0000","last_clean":"2024-09-17T04:31:55.200516+0000","last_became_active":"2024-09-17T04:31:53.838725+0000","last_became_peered":"2024-09-17T04:31:53.838725+0000","last_unstale":"2024-09-17T04:31:55.200516+0000","last_undegraded":"2024-09-17T04:31:55.200516+0000","last_fullsized":"2024-09-17T04:31:55.200516+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:06:39.666825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.809038+0000","last_change":"2024-09-17T04:31:52.810090+0000","last_active":"2024-09-17T04:31:54.809038+0000","last_peered":"2024-09-17T04:31:54.809038+0000","last_clean":"2024-09-17T04:31:54.809038+0000","last_became_active":"2024-09-17T04:31:52.809785+0000","last_became_peered":"2024-09-17T04:31:52.809785+0000","last_unstale":"2024-09-17T04:31:54.809038+0000","last_undegraded":"2024-09-17T04:31:54.809038+0000","last_fullsized":"2024-09-17T04:31:54.809038+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:06:38.626770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.200306+0000","last_change":"2024-09-17T04:31:53.811615+0000","last_active":"2024-09-17T04:31:55.200306+0000","last_peered":"2024-09-17T04:31:55.200306+0000","last_clean":"2024-09-17T04:31:55.200306+0000","last_became_active":"2024-09-17T04:31:53.811272+0000","last_became_peered":"2024-09-17T04:31:53.811272+0000","last_unstale":"2024-09-17T04:31:55.200306+0000","last_undegraded":"2024-09-17T04:31:55.200306+0000","last_fullsized":"2024-09-17T04:31:55.200306+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:29.647612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"131'8","reported_seq":23,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.835520+0000","last_change":"2024-09-17T04:31:52.800071+0000","last_active":"2024-09-17T04:31:54.835520+0000","last_peered":"2024-09-17T04:31:54.835520+0000","last_clean":"2024-09-17T04:31:54.835520+0000","last_became_active":"2024-09-17T04:31:52.799773+0000","last_became_peered":"2024-09-17T04:31:52.799773+0000","last_unstale":"2024-09-17T04:31:54.835520+0000","last_undegraded":"2024-09-17T04:31:54.835520+0000","last_fullsized":"2024-09-17T04:31:54.835520+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:59:10.346310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.834745+0000","last_change":"2024-09-17T04:31:53.826915+0000","last_active":"2024-09-17T04:31:54.834745+0000","last_peered":"2024-09-17T04:31:54.834745+0000","last_clean":"2024-09-17T04:31:54.834745+0000","last_became_active":"2024-09-17T04:31:53.826249+0000","last_became_peered":"2024-09-17T04:31:53.826249+0000","last_unstale":"2024-09-17T04:31:54.834745+0000","last_undegraded":"2024-09-17T04:31:54.834745+0000","last_fullsized":"2024-09-17T04:31:54.834745+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:03.187030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.200907+0000","last_change":"2024-09-17T04:31:52.806854+0000","last_active":"2024-09-17T04:31:55.200907+0000","last_peered":"2024-09-17T04:31:55.200907+0000","last_clean":"2024-09-17T04:31:55.200907+0000","last_became_active":"2024-09-17T04:31:52.806381+0000","last_became_peered":"2024-09-17T04:31:52.806381+0000","last_unstale":"2024-09-17T04:31:55.200907+0000","last_undegraded":"2024-09-17T04:31:55.200907+0000","last_fullsized":"2024-09-17T04:31:55.200907+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:54.773045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.836104+0000","last_change":"2024-09-17T04:31:53.826533+0000","last_active":"2024-09-17T04:31:54.836104+0000","last_peered":"2024-09-17T04:31:54.836104+0000","last_clean":"2024-09-17T04:31:54.836104+0000","last_became_active":"2024-09-17T04:31:53.826273+0000","last_became_peered":"2024-09-17T04:31:53.826273+0000","last_unstale":"2024-09-17T04:31:54.836104+0000","last_undegraded":"2024-09-17T04:31:54.836104+0000","last_fullsized":"2024-09-17T04:31:54.836104+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:53.659868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.806402+0000","last_change":"2024-09-17T04:31:52.817476+0000","last_active":"2024-09-17T04:31:54.806402+0000","last_peered":"2024-09-17T04:31:54.806402+0000","last_clean":"2024-09-17T04:31:54.806402+0000","last_became_active":"2024-09-17T04:31:52.817119+0000","last_became_peered":"2024-09-17T04:31:52.817119+0000","last_unstale":"2024-09-17T04:31:54.806402+0000","last_undegraded":"2024-09-17T04:31:54.806402+0000","last_fullsized":"2024-09-17T04:31:54.806402+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:54:51.742786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.201090+0000","last_change":"2024-09-17T04:31:53.825946+0000","last_active":"2024-09-17T04:31:55.201090+0000","last_peered":"2024-09-17T04:31:55.201090+0000","last_clean":"2024-09-17T04:31:55.201090+0000","last_became_active":"2024-09-17T04:31:53.825601+0000","last_became_peered":"2024-09-17T04:31:53.825601+0000","last_unstale":"2024-09-17T04:31:55.201090+0000","last_undegraded":"2024-09-17T04:31:55.201090+0000","last_fullsized":"2024-09-17T04:31:55.201090+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:59.465298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.835929+0000","last_change":"2024-09-17T04:31:52.811887+0000","last_active":"2024-09-17T04:31:54.835929+0000","last_peered":"2024-09-17T04:31:54.835929+0000","last_clean":"2024-09-17T04:31:54.835929+0000","last_became_active":"2024-09-17T04:31:52.811595+0000","last_became_peered":"2024-09-17T04:31:52.811595+0000","last_unstale":"2024-09-17T04:31:54.835929+0000","last_undegraded":"2024-09-17T04:31:54.835929+0000","last_fullsized":"2024-09-17T04:31:54.835929+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:04.452476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.809526+0000","last_change":"2024-09-17T04:31:53.827808+0000","last_active":"2024-09-17T04:31:54.809526+0000","last_peered":"2024-09-17T04:31:54.809526+0000","last_clean":"2024-09-17T04:31:54.809526+0000","last_became_active":"2024-09-17T04:31:53.822678+0000","last_became_peered":"2024-09-17T04:31:53.822678+0000","last_unstale":"2024-09-17T04:31:54.809526+0000","last_undegraded":"2024-09-17T04:31:54.809526+0000","last_fullsized":"2024-09-17T04:31:54.809526+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:14:26.496106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.806009+0000","last_change":"2024-09-17T04:31:52.819566+0000","last_active":"2024-09-17T04:31:54.806009+0000","last_peered":"2024-09-17T04:31:54.806009+0000","last_clean":"2024-09-17T04:31:54.806009+0000","last_became_active":"2024-09-17T04:31:52.815635+0000","last_became_peered":"2024-09-17T04:31:52.815635+0000","last_unstale":"2024-09-17T04:31:54.806009+0000","last_undegraded":"2024-09-17T04:31:54.806009+0000","last_fullsized":"2024-09-17T04:31:54.806009+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:47:49.612979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.810526+0000","last_change":"2024-09-17T04:31:53.825307+0000","last_active":"2024-09-17T04:31:54.810526+0000","last_peered":"2024-09-17T04:31:54.810526+0000","last_clean":"2024-09-17T04:31:54.810526+0000","last_became_active":"2024-09-17T04:31:53.824981+0000","last_became_peered":"2024-09-17T04:31:53.824981+0000","last_unstale":"2024-09-17T04:31:54.810526+0000","last_undegraded":"2024-09-17T04:31:54.810526+0000","last_fullsized":"2024-09-17T04:31:54.810526+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:32:50.351790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.862490+0000","last_change":"2024-09-17T04:31:52.811819+0000","last_active":"2024-09-17T04:31:54.862490+0000","last_peered":"2024-09-17T04:31:54.862490+0000","last_clean":"2024-09-17T04:31:54.862490+0000","last_became_active":"2024-09-17T04:31:52.811531+0000","last_became_peered":"2024-09-17T04:31:52.811531+0000","last_unstale":"2024-09-17T04:31:54.862490+0000","last_undegraded":"2024-09-17T04:31:54.862490+0000","last_fullsized":"2024-09-17T04:31:54.862490+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:15:07.089604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.862692+0000","last_change":"2024-09-17T04:31:53.831422+0000","last_active":"2024-09-17T04:31:54.862692+0000","last_peered":"2024-09-17T04:31:54.862692+0000","last_clean":"2024-09-17T04:31:54.862692+0000","last_became_active":"2024-09-17T04:31:53.830057+0000","last_became_peered":"2024-09-17T04:31:53.830057+0000","last_unstale":"2024-09-17T04:31:54.862692+0000","last_undegraded":"2024-09-17T04:31:54.862692+0000","last_fullsized":"2024-09-17T04:31:54.862692+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:37:26.939620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.833536+0000","last_change":"2024-09-17T04:31:52.820609+0000","last_active":"2024-09-17T04:31:54.833536+0000","last_peered":"2024-09-17T04:31:54.833536+0000","last_clean":"2024-09-17T04:31:54.833536+0000","last_became_active":"2024-09-17T04:31:52.820280+0000","last_became_peered":"2024-09-17T04:31:52.820280+0000","last_unstale":"2024-09-17T04:31:54.833536+0000","last_undegraded":"2024-09-17T04:31:54.833536+0000","last_fullsized":"2024-09-17T04:31:54.833536+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:00:11.852634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.809825+0000","last_change":"2024-09-17T04:31:53.831147+0000","last_active":"2024-09-17T04:31:54.809825+0000","last_peered":"2024-09-17T04:31:54.809825+0000","last_clean":"2024-09-17T04:31:54.809825+0000","last_became_active":"2024-09-17T04:31:53.830801+0000","last_became_peered":"2024-09-17T04:31:53.830801+0000","last_unstale":"2024-09-17T04:31:54.809825+0000","last_undegraded":"2024-09-17T04:31:54.809825+0000","last_fullsized":"2024-09-17T04:31:54.809825+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:01.396758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.834190+0000","last_change":"2024-09-17T04:31:52.820832+0000","last_active":"2024-09-17T04:31:54.834190+0000","last_peered":"2024-09-17T04:31:54.834190+0000","last_clean":"2024-09-17T04:31:54.834190+0000","last_became_active":"2024-09-17T04:31:52.820498+0000","last_became_peered":"2024-09-17T04:31:52.820498+0000","last_unstale":"2024-09-17T04:31:54.834190+0000","last_undegraded":"2024-09-17T04:31:54.834190+0000","last_fullsized":"2024-09-17T04:31:54.834190+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:42.551098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.862454+0000","last_change":"2024-09-17T04:31:53.834522+0000","last_active":"2024-09-17T04:31:54.862454+0000","last_peered":"2024-09-17T04:31:54.862454+0000","last_clean":"2024-09-17T04:31:54.862454+0000","last_became_active":"2024-09-17T04:31:53.830872+0000","last_became_peered":"2024-09-17T04:31:53.830872+0000","last_unstale":"2024-09-17T04:31:54.862454+0000","last_undegraded":"2024-09-17T04:31:54.862454+0000","last_fullsized":"2024-09-17T04:31:54.862454+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:54:13.505240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.199978+0000","last_change":"2024-09-17T04:31:52.812657+0000","last_active":"2024-09-17T04:31:55.199978+0000","last_peered":"2024-09-17T04:31:55.199978+0000","last_clean":"2024-09-17T04:31:55.199978+0000","last_became_active":"2024-09-17T04:31:52.812349+0000","last_became_peered":"2024-09-17T04:31:52.812349+0000","last_unstale":"2024-09-17T04:31:55.199978+0000","last_undegraded":"2024-09-17T04:31:55.199978+0000","last_fullsized":"2024-09-17T04:31:55.199978+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:53:14.658963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.201089+0000","last_change":"2024-09-17T04:31:53.840953+0000","last_active":"2024-09-17T04:31:55.201089+0000","last_peered":"2024-09-17T04:31:55.201089+0000","last_clean":"2024-09-17T04:31:55.201089+0000","last_became_active":"2024-09-17T04:31:53.840535+0000","last_became_peered":"2024-09-17T04:31:53.840535+0000","last_unstale":"2024-09-17T04:31:55.201089+0000","last_undegraded":"2024-09-17T04:31:55.201089+0000","last_fullsized":"2024-09-17T04:31:55.201089+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:59:15.662092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.834913+0000","last_change":"2024-09-17T04:31:52.818964+0000","last_active":"2024-09-17T04:31:54.834913+0000","last_peered":"2024-09-17T04:31:54.834913+0000","last_clean":"2024-09-17T04:31:54.834913+0000","last_became_active":"2024-09-17T04:31:52.818639+0000","last_became_peered":"2024-09-17T04:31:52.818639+0000","last_unstale":"2024-09-17T04:31:54.834913+0000","last_undegraded":"2024-09-17T04:31:54.834913+0000","last_fullsized":"2024-09-17T04:31:54.834913+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:49.265130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.201060+0000","last_change":"2024-09-17T04:31:53.824675+0000","last_active":"2024-09-17T04:31:55.201060+0000","last_peered":"2024-09-17T04:31:55.201060+0000","last_clean":"2024-09-17T04:31:55.201060+0000","last_became_active":"2024-09-17T04:31:53.823472+0000","last_became_peered":"2024-09-17T04:31:53.823472+0000","last_unstale":"2024-09-17T04:31:55.201060+0000","last_undegraded":"2024-09-17T04:31:55.201060+0000","last_fullsized":"2024-09-17T04:31:55.201060+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:01.396758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"132'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.830905+0000","last_change":"2024-09-17T04:31:52.818353+0000","last_active":"2024-09-17T04:31:54.830905+0000","last_peered":"2024-09-17T04:31:54.830905+0000","last_clean":"2024-09-17T04:31:54.830905+0000","last_became_active":"2024-09-17T04:31:52.818069+0000","last_became_peered":"2024-09-17T04:31:52.818069+0000","last_unstale":"2024-09-17T04:31:54.830905+0000","last_undegraded":"2024-09-17T04:31:54.830905+0000","last_fullsized":"2024-09-17T04:31:54.830905+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:31.484410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.806025+0000","last_change":"2024-09-17T04:31:53.817802+0000","last_active":"2024-09-17T04:31:54.806025+0000","last_peered":"2024-09-17T04:31:54.806025+0000","last_clean":"2024-09-17T04:31:54.806025+0000","last_became_active":"2024-09-17T04:31:53.816398+0000","last_became_peered":"2024-09-17T04:31:53.816398+0000","last_unstale":"2024-09-17T04:31:54.806025+0000","last_undegraded":"2024-09-17T04:31:54.806025+0000","last_fullsized":"2024-09-17T04:31:54.806025+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:36.110338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.810059+0000","last_change":"2024-09-17T04:31:52.816351+0000","last_active":"2024-09-17T04:31:54.810059+0000","last_peered":"2024-09-17T04:31:54.810059+0000","last_clean":"2024-09-17T04:31:54.810059+0000","last_became_active":"2024-09-17T04:31:52.815589+0000","last_became_peered":"2024-09-17T04:31:52.815589+0000","last_unstale":"2024-09-17T04:31:54.810059+0000","last_undegraded":"2024-09-17T04:31:54.810059+0000","last_fullsized":"2024-09-17T04:31:54.810059+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:23:18.409482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.809745+0000","last_change":"2024-09-17T04:31:53.826718+0000","last_active":"2024-09-17T04:31:54.809745+0000","last_peered":"2024-09-17T04:31:54.809745+0000","last_clean":"2024-09-17T04:31:54.809745+0000","last_became_active":"2024-09-17T04:31:53.826049+0000","last_became_peered":"2024-09-17T04:31:53.826049+0000","last_unstale":"2024-09-17T04:31:54.809745+0000","last_undegraded":"2024-09-17T04:31:54.809745+0000","last_fullsized":"2024-09-17T04:31:54.809745+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:01:05.856066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.809416+0000","last_change":"2024-09-17T04:31:52.818269+0000","last_active":"2024-09-17T04:31:54.809416+0000","last_peered":"2024-09-17T04:31:54.809416+0000","last_clean":"2024-09-17T04:31:54.809416+0000","last_became_active":"2024-09-17T04:31:52.815665+0000","last_became_peered":"2024-09-17T04:31:52.815665+0000","last_unstale":"2024-09-17T04:31:54.809416+0000","last_undegraded":"2024-09-17T04:31:54.809416+0000","last_fullsized":"2024-09-17T04:31:54.809416+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:46:07.619505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.835499+0000","last_change":"2024-09-17T04:31:53.814349+0000","last_active":"2024-09-17T04:31:54.835499+0000","last_peered":"2024-09-17T04:31:54.835499+0000","last_clean":"2024-09-17T04:31:54.835499+0000","last_became_active":"2024-09-17T04:31:53.813970+0000","last_became_peered":"2024-09-17T04:31:53.813970+0000","last_unstale":"2024-09-17T04:31:54.835499+0000","last_undegraded":"2024-09-17T04:31:54.835499+0000","last_fullsized":"2024-09-17T04:31:54.835499+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:18:17.261496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.862617+0000","last_change":"2024-09-17T04:31:52.812029+0000","last_active":"2024-09-17T04:31:54.862617+0000","last_peered":"2024-09-17T04:31:54.862617+0000","last_clean":"2024-09-17T04:31:54.862617+0000","last_became_active":"2024-09-17T04:31:52.811665+0000","last_became_peered":"2024-09-17T04:31:52.811665+0000","last_unstale":"2024-09-17T04:31:54.862617+0000","last_undegraded":"2024-09-17T04:31:54.862617+0000","last_fullsized":"2024-09-17T04:31:54.862617+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:23:28.995121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.201415+0000","last_change":"2024-09-17T04:31:53.831041+0000","last_active":"2024-09-17T04:31:55.201415+0000","last_peered":"2024-09-17T04:31:55.201415+0000","last_clean":"2024-09-17T04:31:55.201415+0000","last_became_active":"2024-09-17T04:31:53.830698+0000","last_became_peered":"2024-09-17T04:31:53.830698+0000","last_unstale":"2024-09-17T04:31:55.201415+0000","last_undegraded":"2024-09-17T04:31:55.201415+0000","last_fullsized":"2024-09-17T04:31:55.201415+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:28.360388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.834431+0000","last_change":"2024-09-17T04:31:52.819271+0000","last_active":"2024-09-17T04:31:54.834431+0000","last_peered":"2024-09-17T04:31:54.834431+0000","last_clean":"2024-09-17T04:31:54.834431+0000","last_became_active":"2024-09-17T04:31:52.818943+0000","last_became_peered":"2024-09-17T04:31:52.818943+0000","last_unstale":"2024-09-17T04:31:54.834431+0000","last_undegraded":"2024-09-17T04:31:54.834431+0000","last_fullsized":"2024-09-17T04:31:54.834431+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:04:48.413800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.833582+0000","last_change":"2024-09-17T04:31:53.826661+0000","last_active":"2024-09-17T04:31:54.833582+0000","last_peered":"2024-09-17T04:31:54.833582+0000","last_clean":"2024-09-17T04:31:54.833582+0000","last_became_active":"2024-09-17T04:31:53.826318+0000","last_became_peered":"2024-09-17T04:31:53.826318+0000","last_unstale":"2024-09-17T04:31:54.833582+0000","last_undegraded":"2024-09-17T04:31:54.833582+0000","last_fullsized":"2024-09-17T04:31:54.833582+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:45:50.457360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.200569+0000","last_change":"2024-09-17T04:31:52.808661+0000","last_active":"2024-09-17T04:31:55.200569+0000","last_peered":"2024-09-17T04:31:55.200569+0000","last_clean":"2024-09-17T04:31:55.200569+0000","last_became_active":"2024-09-17T04:31:52.808355+0000","last_became_peered":"2024-09-17T04:31:52.808355+0000","last_unstale":"2024-09-17T04:31:55.200569+0000","last_undegraded":"2024-09-17T04:31:55.200569+0000","last_fullsized":"2024-09-17T04:31:55.200569+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:47.649843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.201337+0000","last_change":"2024-09-17T04:31:53.827755+0000","last_active":"2024-09-17T04:31:55.201337+0000","last_peered":"2024-09-17T04:31:55.201337+0000","last_clean":"2024-09-17T04:31:55.201337+0000","last_became_active":"2024-09-17T04:31:53.825215+0000","last_became_peered":"2024-09-17T04:31:53.825215+0000","last_unstale":"2024-09-17T04:31:55.201337+0000","last_undegraded":"2024-09-17T04:31:55.201337+0000","last_fullsized":"2024-09-17T04:31:55.201337+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:03.852438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.806173+0000","last_change":"2024-09-17T04:31:52.817238+0000","last_active":"2024-09-17T04:31:54.806173+0000","last_peered":"2024-09-17T04:31:54.806173+0000","last_clean":"2024-09-17T04:31:54.806173+0000","last_became_active":"2024-09-17T04:31:52.816605+0000","last_became_peered":"2024-09-17T04:31:52.816605+0000","last_unstale":"2024-09-17T04:31:54.806173+0000","last_undegraded":"2024-09-17T04:31:54.806173+0000","last_fullsized":"2024-09-17T04:31:54.806173+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:04.753249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.862809+0000","last_change":"2024-09-17T04:31:53.834167+0000","last_active":"2024-09-17T04:31:54.862809+0000","last_peered":"2024-09-17T04:31:54.862809+0000","last_clean":"2024-09-17T04:31:54.862809+0000","last_became_active":"2024-09-17T04:31:53.831570+0000","last_became_peered":"2024-09-17T04:31:53.831570+0000","last_unstale":"2024-09-17T04:31:54.862809+0000","last_undegraded":"2024-09-17T04:31:54.862809+0000","last_fullsized":"2024-09-17T04:31:54.862809+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:49.369165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"131'3","reported_seq":18,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.200787+0000","last_change":"2024-09-17T04:31:52.815532+0000","last_active":"2024-09-17T04:31:55.200787+0000","last_peered":"2024-09-17T04:31:55.200787+0000","last_clean":"2024-09-17T04:31:55.200787+0000","last_became_active":"2024-09-17T04:31:52.814081+0000","last_became_peered":"2024-09-17T04:31:52.814081+0000","last_unstale":"2024-09-17T04:31:55.200787+0000","last_undegraded":"2024-09-17T04:31:55.200787+0000","last_fullsized":"2024-09-17T04:31:55.200787+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:58.445527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"76'79","reported_seq":341,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.833756+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:31:54.833756+0000","last_peered":"2024-09-17T04:31:54.833756+0000","last_clean":"2024-09-17T04:31:54.833756+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:31:54.833756+0000","last_undegraded":"2024-09-17T04:31:54.833756+0000","last_fullsized":"2024-09-17T04:31:54.833756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476836,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33380,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338460,"statfs":{"total":96636764160,"available":96602583040,"internally_reserved":0,"allocated":2244608,"data_stored":1314308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476835,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32804,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339036,"statfs":{"total":96636764160,"available":96603172864,"internally_reserved":0,"allocated":1654784,"data_stored":723434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476836,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1646592,"data_stored":723310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476836,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32796,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339044,"statfs":{"total":96636764160,"available":96603181056,"internally_reserved":0,"allocated":1642496,"data_stored":723288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542243,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1646592,"data_stored":723378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509539,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1658880,"data_stored":724970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509538,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1646592,"data_stored":723870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509538,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33392,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338448,"statfs":{"total":96636764160,"available":96602570752,"internally_reserved":0,"allocated":2252800,"data_stored":1315302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:31:58.246 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:31:58.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.504+0000 7f4da7000640 1 -- 172.21.15.100:0/1321588601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4da00af170 msgr2=0x7f4da00b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:58.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.504+0000 7f4da7000640 1 --2- 172.21.15.100:0/1321588601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4da00af170 0x7f4da00b2f60 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c005fd0 tx=0x7f4d8c02a4b0 comp rx=0 tx=0).stop 2024-09-17T04:31:58.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.504+0000 7f4da7000640 1 -- 172.21.15.100:0/1321588601 shutdown_connections 2024-09-17T04:31:58.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.504+0000 7f4da7000640 1 --2- 172.21.15.100:0/1321588601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4da00af170 0x7f4da00b2f60 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:58.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.504+0000 7f4da7000640 1 -- 172.21.15.100:0/1321588601 >> 172.21.15.100:0/1321588601 conn(0x7f4da00b1e40 msgr2=0x7f4da00b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:58.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.504+0000 7f4da7000640 1 -- 172.21.15.100:0/1321588601 shutdown_connections 2024-09-17T04:31:58.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.504+0000 7f4da7000640 1 -- 172.21.15.100:0/1321588601 wait complete. 2024-09-17T04:31:58.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.506+0000 7f4da7000640 1 Processor -- start 2024-09-17T04:31:58.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.506+0000 7f4da7000640 1 -- start start 2024-09-17T04:31:58.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.508+0000 7f4da7000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4da00ad9c0 0x7f4da01f3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:58.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.508+0000 7f4da7000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4da00af170 0x7f4da0149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:58.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.508+0000 7f4da7000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4da015a0f0 0x7f4da01f6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:58.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.508+0000 7f4da7000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4da006fb20 con 0x7f4da00af170 2024-09-17T04:31:58.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.508+0000 7f4da7000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4da006f9a0 con 0x7f4da015a0f0 2024-09-17T04:31:58.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.508+0000 7f4da7000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4da015d670 con 0x7f4da00ad9c0 2024-09-17T04:31:58.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.508+0000 7f4da5200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4da00af170 0x7f4da0149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:58.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.508+0000 7f4da5200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4da00af170 0x7f4da0149040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48818/0 (socket says 172.21.15.100:48818) 2024-09-17T04:31:58.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.508+0000 7f4da5200640 1 -- 172.21.15.100:0/2329783439 learned_addr learned my addr 172.21.15.100:0/2329783439 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:58.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.508+0000 7f4da5c00640 1 --2- 172.21.15.100:0/2329783439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4da00ad9c0 0x7f4da01f3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:58.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.508+0000 7f4da6600640 1 --2- 172.21.15.100:0/2329783439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4da015a0f0 0x7f4da01f6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.508+0000 7f4da5200640 1 -- 172.21.15.100:0/2329783439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4da00ad9c0 msgr2=0x7f4da01f3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.508+0000 7f4da5200640 1 --2- 172.21.15.100:0/2329783439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4da00ad9c0 0x7f4da01f3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.508+0000 7f4da5200640 1 -- 172.21.15.100:0/2329783439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4da015a0f0 msgr2=0x7f4da01f6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.508+0000 7f4da5200640 1 --2- 172.21.15.100:0/2329783439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4da015a0f0 0x7f4da01f6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.508+0000 7f4da5200640 1 -- 172.21.15.100:0/2329783439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da0157a50 con 0x7f4da00af170 2024-09-17T04:31:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.508+0000 7f4da5c00640 1 --2- 172.21.15.100:0/2329783439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4da00ad9c0 0x7f4da01f3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:31:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.509+0000 7f4da5200640 1 --2- 172.21.15.100:0/2329783439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4da00af170 0x7f4da0149040 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c000c00 tx=0x7f4d8c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.509+0000 7f4d9d600640 1 -- 172.21.15.100:0/2329783439 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c03e030 con 0x7f4da00af170 2024-09-17T04:31:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.509+0000 7f4d9d600640 1 -- 172.21.15.100:0/2329783439 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d8c003e60 con 0x7f4da00af170 2024-09-17T04:31:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.509+0000 7f4d9d600640 1 -- 172.21.15.100:0/2329783439 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c035040 con 0x7f4da00af170 2024-09-17T04:31:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.509+0000 7f4da7000640 1 -- 172.21.15.100:0/2329783439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da0157d30 con 0x7f4da00af170 2024-09-17T04:31:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.509+0000 7f4da7000640 1 -- 172.21.15.100:0/2329783439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da01582f0 con 0x7f4da00af170 2024-09-17T04:31:58.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.510+0000 7f4da7000640 1 -- 172.21.15.100:0/2329783439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da00b2f60 con 0x7f4da00af170 2024-09-17T04:31:58.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.512+0000 7f4d9d600640 1 -- 172.21.15.100:0/2329783439 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f4d8c041030 con 0x7f4da00af170 2024-09-17T04:31:58.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.513+0000 7f4d9d600640 1 --2- 172.21.15.100:0/2329783439 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4d800791c0 0x7f4d8007b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:58.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.513+0000 7f4d9d600640 1 -- 172.21.15.100:0/2329783439 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7f4d8c0bc070 con 0x7f4da00af170 2024-09-17T04:31:58.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.513+0000 7f4da5c00640 1 --2- 172.21.15.100:0/2329783439 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4d800791c0 0x7f4d8007b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:58.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.514+0000 7f4da5c00640 1 --2- 172.21.15.100:0/2329783439 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4d800791c0 0x7f4d8007b6d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4d94007fd0 tx=0x7f4d94009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:58.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.517+0000 7f4d9d600640 1 -- 172.21.15.100:0/2329783439 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4d8c083c70 con 0x7f4da00af170 2024-09-17T04:31:58.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.697+0000 7f4da7000640 1 -- 172.21.15.100:0/2329783439 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4da00b0230 con 0x7f4d800791c0 2024-09-17T04:31:58.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.701+0000 7f4d9d600640 1 -- 172.21.15.100:0/2329783439 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252481 (secure 0 0 0) 0x7f4da00b0230 con 0x7f4d800791c0 2024-09-17T04:31:58.706 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:58.708 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:31:58.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.706+0000 7f4da7000640 1 -- 172.21.15.100:0/2329783439 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4d800791c0 msgr2=0x7f4d8007b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:58.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.706+0000 7f4da7000640 1 --2- 172.21.15.100:0/2329783439 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4d800791c0 0x7f4d8007b6d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4d94007fd0 tx=0x7f4d94009040 comp rx=0 tx=0).stop 2024-09-17T04:31:58.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.706+0000 7f4da7000640 1 -- 172.21.15.100:0/2329783439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4da00af170 msgr2=0x7f4da0149040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:58.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.706+0000 7f4da7000640 1 --2- 172.21.15.100:0/2329783439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4da00af170 0x7f4da0149040 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c000c00 tx=0x7f4d8c003970 comp rx=0 tx=0).stop 2024-09-17T04:31:58.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.706+0000 7f4da7000640 1 -- 172.21.15.100:0/2329783439 shutdown_connections 2024-09-17T04:31:58.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.706+0000 7f4da7000640 1 --2- 172.21.15.100:0/2329783439 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4d800791c0 0x7f4d8007b6d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:58.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.706+0000 7f4da7000640 1 --2- 172.21.15.100:0/2329783439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4da015a0f0 0x7f4da01f6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:58.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.706+0000 7f4da7000640 1 --2- 172.21.15.100:0/2329783439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4da00af170 0x7f4da0149040 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:58.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.706+0000 7f4da7000640 1 --2- 172.21.15.100:0/2329783439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4da00ad9c0 0x7f4da01f3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:58.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.706+0000 7f4da7000640 1 -- 172.21.15.100:0/2329783439 >> 172.21.15.100:0/2329783439 conn(0x7f4da00b1e40 msgr2=0x7f4da00b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:58.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.706+0000 7f4da7000640 1 -- 172.21.15.100:0/2329783439 shutdown_connections 2024-09-17T04:31:58.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.706+0000 7f4da7000640 1 -- 172.21.15.100:0/2329783439 wait complete. 2024-09-17T04:31:58.732 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2024-09-17T04:31:57.698199+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":147,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263584,"kb_used_data":14056,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711136,"statfs":{"total":773094113280,"available":772824203264,"internally_reserved":0,"allocated":14393344,"data_stored":6971860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.909245"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.810355+0000","last_change":"2024-09-17T04:31:53.830592+0000","last_active":"2024-09-17T04:31:54.810355+0000","last_peered":"2024-09-17T04:31:54.810355+0000","last_clean":"2024-09-17T04:31:54.810355+0000","last_became_active":"2024-09-17T04:31:53.830274+0000","last_became_peered":"2024-09-17T04:31:53.830274+0000","last_unstale":"2024-09-17T04:31:54.810355+0000","last_undegraded":"2024-09-17T04:31:54.810355+0000","last_fullsized":"2024-09-17T04:31:54.810355+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:21.313063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.806256+0000","last_change":"2024-09-17T04:31:53.831696+0000","last_active":"2024-09-17T04:31:54.806256+0000","last_peered":"2024-09-17T04:31:54.806256+0000","last_clean":"2024-09-17T04:31:54.806256+0000","last_became_active":"2024-09-17T04:31:53.831353+0000","last_became_peered":"2024-09-17T04:31:53.831353+0000","last_unstale":"2024-09-17T04:31:54.806256+0000","last_undegraded":"2024-09-17T04:31:54.806256+0000","last_fullsized":"2024-09-17T04:31:54.806256+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:11.579919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.201015+0000","last_change":"2024-09-17T04:31:53.811930+0000","last_active":"2024-09-17T04:31:55.201015+0000","last_peered":"2024-09-17T04:31:55.201015+0000","last_clean":"2024-09-17T04:31:55.201015+0000","last_became_active":"2024-09-17T04:31:53.811139+0000","last_became_peered":"2024-09-17T04:31:53.811139+0000","last_unstale":"2024-09-17T04:31:55.201015+0000","last_undegraded":"2024-09-17T04:31:55.201015+0000","last_fullsized":"2024-09-17T04:31:55.201015+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:51.541898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.810655+0000","last_change":"2024-09-17T04:31:53.819868+0000","last_active":"2024-09-17T04:31:54.810655+0000","last_peered":"2024-09-17T04:31:54.810655+0000","last_clean":"2024-09-17T04:31:54.810655+0000","last_became_active":"2024-09-17T04:31:53.818589+0000","last_became_peered":"2024-09-17T04:31:53.818589+0000","last_unstale":"2024-09-17T04:31:54.810655+0000","last_undegraded":"2024-09-17T04:31:54.810655+0000","last_fullsized":"2024-09-17T04:31:54.810655+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:11.977100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.201331+0000","last_change":"2024-09-17T04:31:53.840709+0000","last_active":"2024-09-17T04:31:55.201331+0000","last_peered":"2024-09-17T04:31:55.201331+0000","last_clean":"2024-09-17T04:31:55.201331+0000","last_became_active":"2024-09-17T04:31:53.839975+0000","last_became_peered":"2024-09-17T04:31:53.839975+0000","last_unstale":"2024-09-17T04:31:55.201331+0000","last_undegraded":"2024-09-17T04:31:55.201331+0000","last_fullsized":"2024-09-17T04:31:55.201331+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:01:24.938949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.809438+0000","last_change":"2024-09-17T04:31:53.823162+0000","last_active":"2024-09-17T04:31:54.809438+0000","last_peered":"2024-09-17T04:31:54.809438+0000","last_clean":"2024-09-17T04:31:54.809438+0000","last_became_active":"2024-09-17T04:31:53.822666+0000","last_became_peered":"2024-09-17T04:31:53.822666+0000","last_unstale":"2024-09-17T04:31:54.809438+0000","last_undegraded":"2024-09-17T04:31:54.809438+0000","last_fullsized":"2024-09-17T04:31:54.809438+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:04:30.245191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.862626+0000","last_change":"2024-09-17T04:31:53.826943+0000","last_active":"2024-09-17T04:31:54.862626+0000","last_peered":"2024-09-17T04:31:54.862626+0000","last_clean":"2024-09-17T04:31:54.862626+0000","last_became_active":"2024-09-17T04:31:53.826631+0000","last_became_peered":"2024-09-17T04:31:53.826631+0000","last_unstale":"2024-09-17T04:31:54.862626+0000","last_undegraded":"2024-09-17T04:31:54.862626+0000","last_fullsized":"2024-09-17T04:31:54.862626+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:54.356962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.809162+0000","last_change":"2024-09-17T04:31:53.837611+0000","last_active":"2024-09-17T04:31:54.809162+0000","last_peered":"2024-09-17T04:31:54.809162+0000","last_clean":"2024-09-17T04:31:54.809162+0000","last_became_active":"2024-09-17T04:31:53.837319+0000","last_became_peered":"2024-09-17T04:31:53.837319+0000","last_unstale":"2024-09-17T04:31:54.809162+0000","last_undegraded":"2024-09-17T04:31:54.809162+0000","last_fullsized":"2024-09-17T04:31:54.809162+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:29:59.500142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.806298+0000","last_change":"2024-09-17T04:31:53.822891+0000","last_active":"2024-09-17T04:31:54.806298+0000","last_peered":"2024-09-17T04:31:54.806298+0000","last_clean":"2024-09-17T04:31:54.806298+0000","last_became_active":"2024-09-17T04:31:53.822552+0000","last_became_peered":"2024-09-17T04:31:53.822552+0000","last_unstale":"2024-09-17T04:31:54.806298+0000","last_undegraded":"2024-09-17T04:31:54.806298+0000","last_fullsized":"2024-09-17T04:31:54.806298+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:02:38.832265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.834466+0000","last_change":"2024-09-17T04:31:53.826556+0000","last_active":"2024-09-17T04:31:54.834466+0000","last_peered":"2024-09-17T04:31:54.834466+0000","last_clean":"2024-09-17T04:31:54.834466+0000","last_became_active":"2024-09-17T04:31:53.825875+0000","last_became_peered":"2024-09-17T04:31:53.825875+0000","last_unstale":"2024-09-17T04:31:54.834466+0000","last_undegraded":"2024-09-17T04:31:54.834466+0000","last_fullsized":"2024-09-17T04:31:54.834466+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:15:08.129660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.810285+0000","last_change":"2024-09-17T04:31:53.839488+0000","last_active":"2024-09-17T04:31:54.810285+0000","last_peered":"2024-09-17T04:31:54.810285+0000","last_clean":"2024-09-17T04:31:54.810285+0000","last_became_active":"2024-09-17T04:31:53.839189+0000","last_became_peered":"2024-09-17T04:31:53.839189+0000","last_unstale":"2024-09-17T04:31:54.810285+0000","last_undegraded":"2024-09-17T04:31:54.810285+0000","last_fullsized":"2024-09-17T04:31:54.810285+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:45.195188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.833836+0000","last_change":"2024-09-17T04:31:53.836665+0000","last_active":"2024-09-17T04:31:54.833836+0000","last_peered":"2024-09-17T04:31:54.833836+0000","last_clean":"2024-09-17T04:31:54.833836+0000","last_became_active":"2024-09-17T04:31:53.835906+0000","last_became_peered":"2024-09-17T04:31:53.835906+0000","last_unstale":"2024-09-17T04:31:54.833836+0000","last_undegraded":"2024-09-17T04:31:54.833836+0000","last_fullsized":"2024-09-17T04:31:54.833836+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:39.300901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.201144+0000","last_change":"2024-09-17T04:31:53.833952+0000","last_active":"2024-09-17T04:31:55.201144+0000","last_peered":"2024-09-17T04:31:55.201144+0000","last_clean":"2024-09-17T04:31:55.201144+0000","last_became_active":"2024-09-17T04:31:53.833499+0000","last_became_peered":"2024-09-17T04:31:53.833499+0000","last_unstale":"2024-09-17T04:31:55.201144+0000","last_undegraded":"2024-09-17T04:31:55.201144+0000","last_fullsized":"2024-09-17T04:31:55.201144+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:54:59.003417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.810085+0000","last_change":"2024-09-17T04:31:53.828724+0000","last_active":"2024-09-17T04:31:54.810085+0000","last_peered":"2024-09-17T04:31:54.810085+0000","last_clean":"2024-09-17T04:31:54.810085+0000","last_became_active":"2024-09-17T04:31:53.828263+0000","last_became_peered":"2024-09-17T04:31:53.828263+0000","last_unstale":"2024-09-17T04:31:54.810085+0000","last_undegraded":"2024-09-17T04:31:54.810085+0000","last_fullsized":"2024-09-17T04:31:54.810085+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:00:53.550048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.833588+0000","last_change":"2024-09-17T04:31:53.828087+0000","last_active":"2024-09-17T04:31:54.833588+0000","last_peered":"2024-09-17T04:31:54.833588+0000","last_clean":"2024-09-17T04:31:54.833588+0000","last_became_active":"2024-09-17T04:31:53.827800+0000","last_became_peered":"2024-09-17T04:31:53.827800+0000","last_unstale":"2024-09-17T04:31:54.833588+0000","last_undegraded":"2024-09-17T04:31:54.833588+0000","last_fullsized":"2024-09-17T04:31:54.833588+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:34:39.879340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.835379+0000","last_change":"2024-09-17T04:31:53.823709+0000","last_active":"2024-09-17T04:31:54.835379+0000","last_peered":"2024-09-17T04:31:54.835379+0000","last_clean":"2024-09-17T04:31:54.835379+0000","last_became_active":"2024-09-17T04:31:53.823370+0000","last_became_peered":"2024-09-17T04:31:53.823370+0000","last_unstale":"2024-09-17T04:31:54.835379+0000","last_undegraded":"2024-09-17T04:31:54.835379+0000","last_fullsized":"2024-09-17T04:31:54.835379+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:20:37.855904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.834954+0000","last_change":"2024-09-17T04:31:53.837541+0000","last_active":"2024-09-17T04:31:54.834954+0000","last_peered":"2024-09-17T04:31:54.834954+0000","last_clean":"2024-09-17T04:31:54.834954+0000","last_became_active":"2024-09-17T04:31:53.837134+0000","last_became_peered":"2024-09-17T04:31:53.837134+0000","last_unstale":"2024-09-17T04:31:54.834954+0000","last_undegraded":"2024-09-17T04:31:54.834954+0000","last_fullsized":"2024-09-17T04:31:54.834954+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:01:03.617937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.200697+0000","last_change":"2024-09-17T04:31:53.816290+0000","last_active":"2024-09-17T04:31:55.200697+0000","last_peered":"2024-09-17T04:31:55.200697+0000","last_clean":"2024-09-17T04:31:55.200697+0000","last_became_active":"2024-09-17T04:31:53.815904+0000","last_became_peered":"2024-09-17T04:31:53.815904+0000","last_unstale":"2024-09-17T04:31:55.200697+0000","last_undegraded":"2024-09-17T04:31:55.200697+0000","last_fullsized":"2024-09-17T04:31:55.200697+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:20.432704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.809950+0000","last_change":"2024-09-17T04:31:53.826413+0000","last_active":"2024-09-17T04:31:54.809950+0000","last_peered":"2024-09-17T04:31:54.809950+0000","last_clean":"2024-09-17T04:31:54.809950+0000","last_became_active":"2024-09-17T04:31:53.825654+0000","last_became_peered":"2024-09-17T04:31:53.825654+0000","last_unstale":"2024-09-17T04:31:54.809950+0000","last_undegraded":"2024-09-17T04:31:54.809950+0000","last_fullsized":"2024-09-17T04:31:54.809950+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:43:26.938684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.835738+0000","last_change":"2024-09-17T04:31:53.810221+0000","last_active":"2024-09-17T04:31:54.835738+0000","last_peered":"2024-09-17T04:31:54.835738+0000","last_clean":"2024-09-17T04:31:54.835738+0000","last_became_active":"2024-09-17T04:31:53.809858+0000","last_became_peered":"2024-09-17T04:31:53.809858+0000","last_unstale":"2024-09-17T04:31:54.835738+0000","last_undegraded":"2024-09-17T04:31:54.835738+0000","last_fullsized":"2024-09-17T04:31:54.835738+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:04.510738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.834666+0000","last_change":"2024-09-17T04:31:53.829308+0000","last_active":"2024-09-17T04:31:54.834666+0000","last_peered":"2024-09-17T04:31:54.834666+0000","last_clean":"2024-09-17T04:31:54.834666+0000","last_became_active":"2024-09-17T04:31:53.828993+0000","last_became_peered":"2024-09-17T04:31:53.828993+0000","last_unstale":"2024-09-17T04:31:54.834666+0000","last_undegraded":"2024-09-17T04:31:54.834666+0000","last_fullsized":"2024-09-17T04:31:54.834666+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:43:26.093268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.810154+0000","last_change":"2024-09-17T04:31:53.815644+0000","last_active":"2024-09-17T04:31:54.810154+0000","last_peered":"2024-09-17T04:31:54.810154+0000","last_clean":"2024-09-17T04:31:54.810154+0000","last_became_active":"2024-09-17T04:31:53.812350+0000","last_became_peered":"2024-09-17T04:31:53.812350+0000","last_unstale":"2024-09-17T04:31:54.810154+0000","last_undegraded":"2024-09-17T04:31:54.810154+0000","last_fullsized":"2024-09-17T04:31:54.810154+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:50.305186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.809857+0000","last_change":"2024-09-17T04:31:53.831549+0000","last_active":"2024-09-17T04:31:54.809857+0000","last_peered":"2024-09-17T04:31:54.809857+0000","last_clean":"2024-09-17T04:31:54.809857+0000","last_became_active":"2024-09-17T04:31:53.830866+0000","last_became_peered":"2024-09-17T04:31:53.830866+0000","last_unstale":"2024-09-17T04:31:54.809857+0000","last_undegraded":"2024-09-17T04:31:54.809857+0000","last_fullsized":"2024-09-17T04:31:54.809857+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:59.465298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.833529+0000","last_change":"2024-09-17T04:31:53.831401+0000","last_active":"2024-09-17T04:31:54.833529+0000","last_peered":"2024-09-17T04:31:54.833529+0000","last_clean":"2024-09-17T04:31:54.833529+0000","last_became_active":"2024-09-17T04:31:53.830298+0000","last_became_peered":"2024-09-17T04:31:53.830298+0000","last_unstale":"2024-09-17T04:31:54.833529+0000","last_undegraded":"2024-09-17T04:31:54.833529+0000","last_fullsized":"2024-09-17T04:31:54.833529+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:11:07.418895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.835869+0000","last_change":"2024-09-17T04:31:53.817133+0000","last_active":"2024-09-17T04:31:54.835869+0000","last_peered":"2024-09-17T04:31:54.835869+0000","last_clean":"2024-09-17T04:31:54.835869+0000","last_became_active":"2024-09-17T04:31:53.816793+0000","last_became_peered":"2024-09-17T04:31:53.816793+0000","last_unstale":"2024-09-17T04:31:54.835869+0000","last_undegraded":"2024-09-17T04:31:54.835869+0000","last_fullsized":"2024-09-17T04:31:54.835869+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:24:41.034403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.810334+0000","last_change":"2024-09-17T04:31:53.824497+0000","last_active":"2024-09-17T04:31:54.810334+0000","last_peered":"2024-09-17T04:31:54.810334+0000","last_clean":"2024-09-17T04:31:54.810334+0000","last_became_active":"2024-09-17T04:31:53.824170+0000","last_became_peered":"2024-09-17T04:31:53.824170+0000","last_unstale":"2024-09-17T04:31:54.810334+0000","last_undegraded":"2024-09-17T04:31:54.810334+0000","last_fullsized":"2024-09-17T04:31:54.810334+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:11.415094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.835730+0000","last_change":"2024-09-17T04:31:53.814196+0000","last_active":"2024-09-17T04:31:54.835730+0000","last_peered":"2024-09-17T04:31:54.835730+0000","last_clean":"2024-09-17T04:31:54.835730+0000","last_became_active":"2024-09-17T04:31:53.813831+0000","last_became_peered":"2024-09-17T04:31:53.813831+0000","last_unstale":"2024-09-17T04:31:54.835730+0000","last_undegraded":"2024-09-17T04:31:54.835730+0000","last_fullsized":"2024-09-17T04:31:54.835730+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:29.115890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.862513+0000","last_change":"2024-09-17T04:31:53.832885+0000","last_active":"2024-09-17T04:31:54.862513+0000","last_peered":"2024-09-17T04:31:54.862513+0000","last_clean":"2024-09-17T04:31:54.862513+0000","last_became_active":"2024-09-17T04:31:53.832521+0000","last_became_peered":"2024-09-17T04:31:53.832521+0000","last_unstale":"2024-09-17T04:31:54.862513+0000","last_undegraded":"2024-09-17T04:31:54.862513+0000","last_fullsized":"2024-09-17T04:31:54.862513+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:31:26.782803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.809413+0000","last_change":"2024-09-17T04:31:53.820157+0000","last_active":"2024-09-17T04:31:54.809413+0000","last_peered":"2024-09-17T04:31:54.809413+0000","last_clean":"2024-09-17T04:31:54.809413+0000","last_became_active":"2024-09-17T04:31:53.819866+0000","last_became_peered":"2024-09-17T04:31:53.819866+0000","last_unstale":"2024-09-17T04:31:54.809413+0000","last_undegraded":"2024-09-17T04:31:54.809413+0000","last_fullsized":"2024-09-17T04:31:54.809413+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:19.747207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.810495+0000","last_change":"2024-09-17T04:31:53.826969+0000","last_active":"2024-09-17T04:31:54.810495+0000","last_peered":"2024-09-17T04:31:54.810495+0000","last_clean":"2024-09-17T04:31:54.810495+0000","last_became_active":"2024-09-17T04:31:53.826485+0000","last_became_peered":"2024-09-17T04:31:53.826485+0000","last_unstale":"2024-09-17T04:31:54.810495+0000","last_undegraded":"2024-09-17T04:31:54.810495+0000","last_fullsized":"2024-09-17T04:31:54.810495+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:58:52.207651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.806089+0000","last_change":"2024-09-17T04:31:53.814804+0000","last_active":"2024-09-17T04:31:54.806089+0000","last_peered":"2024-09-17T04:31:54.806089+0000","last_clean":"2024-09-17T04:31:54.806089+0000","last_became_active":"2024-09-17T04:31:53.813106+0000","last_became_peered":"2024-09-17T04:31:53.813106+0000","last_unstale":"2024-09-17T04:31:54.806089+0000","last_undegraded":"2024-09-17T04:31:54.806089+0000","last_fullsized":"2024-09-17T04:31:54.806089+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:18:42.930314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.835457+0000","last_change":"2024-09-17T04:31:53.833697+0000","last_active":"2024-09-17T04:31:54.835457+0000","last_peered":"2024-09-17T04:31:54.835457+0000","last_clean":"2024-09-17T04:31:54.835457+0000","last_became_active":"2024-09-17T04:31:53.833393+0000","last_became_peered":"2024-09-17T04:31:53.833393+0000","last_unstale":"2024-09-17T04:31:54.835457+0000","last_undegraded":"2024-09-17T04:31:54.835457+0000","last_fullsized":"2024-09-17T04:31:54.835457+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:52.632954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.834481+0000","last_change":"2024-09-17T04:31:53.833787+0000","last_active":"2024-09-17T04:31:54.834481+0000","last_peered":"2024-09-17T04:31:54.834481+0000","last_clean":"2024-09-17T04:31:54.834481+0000","last_became_active":"2024-09-17T04:31:53.833467+0000","last_became_peered":"2024-09-17T04:31:53.833467+0000","last_unstale":"2024-09-17T04:31:54.834481+0000","last_undegraded":"2024-09-17T04:31:54.834481+0000","last_fullsized":"2024-09-17T04:31:54.834481+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:35:45.421975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"131'4","reported_seq":19,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.834841+0000","last_change":"2024-09-17T04:31:52.821727+0000","last_active":"2024-09-17T04:31:54.834841+0000","last_peered":"2024-09-17T04:31:54.834841+0000","last_clean":"2024-09-17T04:31:54.834841+0000","last_became_active":"2024-09-17T04:31:52.821463+0000","last_became_peered":"2024-09-17T04:31:52.821463+0000","last_unstale":"2024-09-17T04:31:54.834841+0000","last_undegraded":"2024-09-17T04:31:54.834841+0000","last_fullsized":"2024-09-17T04:31:54.834841+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:56.045866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.806341+0000","last_change":"2024-09-17T04:31:53.831239+0000","last_active":"2024-09-17T04:31:54.806341+0000","last_peered":"2024-09-17T04:31:54.806341+0000","last_clean":"2024-09-17T04:31:54.806341+0000","last_became_active":"2024-09-17T04:31:53.830910+0000","last_became_peered":"2024-09-17T04:31:53.830910+0000","last_unstale":"2024-09-17T04:31:54.806341+0000","last_undegraded":"2024-09-17T04:31:54.806341+0000","last_fullsized":"2024-09-17T04:31:54.806341+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:21:32.999335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.806311+0000","last_change":"2024-09-17T04:31:52.818664+0000","last_active":"2024-09-17T04:31:54.806311+0000","last_peered":"2024-09-17T04:31:54.806311+0000","last_clean":"2024-09-17T04:31:54.806311+0000","last_became_active":"2024-09-17T04:31:52.818284+0000","last_became_peered":"2024-09-17T04:31:52.818284+0000","last_unstale":"2024-09-17T04:31:54.806311+0000","last_undegraded":"2024-09-17T04:31:54.806311+0000","last_fullsized":"2024-09-17T04:31:54.806311+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:15.782451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.809389+0000","last_change":"2024-09-17T04:31:53.828826+0000","last_active":"2024-09-17T04:31:54.809389+0000","last_peered":"2024-09-17T04:31:54.809389+0000","last_clean":"2024-09-17T04:31:54.809389+0000","last_became_active":"2024-09-17T04:31:53.825022+0000","last_became_peered":"2024-09-17T04:31:53.825022+0000","last_unstale":"2024-09-17T04:31:54.809389+0000","last_undegraded":"2024-09-17T04:31:54.809389+0000","last_fullsized":"2024-09-17T04:31:54.809389+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:44:22.026471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.810701+0000","last_change":"2024-09-17T04:31:52.808269+0000","last_active":"2024-09-17T04:31:54.810701+0000","last_peered":"2024-09-17T04:31:54.810701+0000","last_clean":"2024-09-17T04:31:54.810701+0000","last_became_active":"2024-09-17T04:31:52.807984+0000","last_became_peered":"2024-09-17T04:31:52.807984+0000","last_unstale":"2024-09-17T04:31:54.810701+0000","last_undegraded":"2024-09-17T04:31:54.810701+0000","last_fullsized":"2024-09-17T04:31:54.810701+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:58.445527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.834599+0000","last_change":"2024-09-17T04:31:53.836445+0000","last_active":"2024-09-17T04:31:54.834599+0000","last_peered":"2024-09-17T04:31:54.834599+0000","last_clean":"2024-09-17T04:31:54.834599+0000","last_became_active":"2024-09-17T04:31:53.836101+0000","last_became_peered":"2024-09-17T04:31:53.836101+0000","last_unstale":"2024-09-17T04:31:54.834599+0000","last_undegraded":"2024-09-17T04:31:54.834599+0000","last_fullsized":"2024-09-17T04:31:54.834599+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:04.612966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.810531+0000","last_change":"2024-09-17T04:31:52.815226+0000","last_active":"2024-09-17T04:31:54.810531+0000","last_peered":"2024-09-17T04:31:54.810531+0000","last_clean":"2024-09-17T04:31:54.810531+0000","last_became_active":"2024-09-17T04:31:52.814650+0000","last_became_peered":"2024-09-17T04:31:52.814650+0000","last_unstale":"2024-09-17T04:31:54.810531+0000","last_undegraded":"2024-09-17T04:31:54.810531+0000","last_fullsized":"2024-09-17T04:31:54.810531+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:16.085871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.806223+0000","last_change":"2024-09-17T04:31:53.814739+0000","last_active":"2024-09-17T04:31:54.806223+0000","last_peered":"2024-09-17T04:31:54.806223+0000","last_clean":"2024-09-17T04:31:54.806223+0000","last_became_active":"2024-09-17T04:31:53.814402+0000","last_became_peered":"2024-09-17T04:31:53.814402+0000","last_unstale":"2024-09-17T04:31:54.806223+0000","last_undegraded":"2024-09-17T04:31:54.806223+0000","last_fullsized":"2024-09-17T04:31:54.806223+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:33:14.071639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.835473+0000","last_change":"2024-09-17T04:31:52.814054+0000","last_active":"2024-09-17T04:31:54.835473+0000","last_peered":"2024-09-17T04:31:54.835473+0000","last_clean":"2024-09-17T04:31:54.835473+0000","last_became_active":"2024-09-17T04:31:52.813783+0000","last_became_peered":"2024-09-17T04:31:52.813783+0000","last_unstale":"2024-09-17T04:31:54.835473+0000","last_undegraded":"2024-09-17T04:31:54.835473+0000","last_fullsized":"2024-09-17T04:31:54.835473+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:56:55.128758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.810556+0000","last_change":"2024-09-17T04:31:53.817343+0000","last_active":"2024-09-17T04:31:54.810556+0000","last_peered":"2024-09-17T04:31:54.810556+0000","last_clean":"2024-09-17T04:31:54.810556+0000","last_became_active":"2024-09-17T04:31:53.816981+0000","last_became_peered":"2024-09-17T04:31:53.816981+0000","last_unstale":"2024-09-17T04:31:54.810556+0000","last_undegraded":"2024-09-17T04:31:54.810556+0000","last_fullsized":"2024-09-17T04:31:54.810556+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:23.163852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.201252+0000","last_change":"2024-09-17T04:31:52.814668+0000","last_active":"2024-09-17T04:31:55.201252+0000","last_peered":"2024-09-17T04:31:55.201252+0000","last_clean":"2024-09-17T04:31:55.201252+0000","last_became_active":"2024-09-17T04:31:52.814236+0000","last_became_peered":"2024-09-17T04:31:52.814236+0000","last_unstale":"2024-09-17T04:31:55.201252+0000","last_undegraded":"2024-09-17T04:31:55.201252+0000","last_fullsized":"2024-09-17T04:31:55.201252+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:47.103533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.805856+0000","last_change":"2024-09-17T04:31:53.833527+0000","last_active":"2024-09-17T04:31:54.805856+0000","last_peered":"2024-09-17T04:31:54.805856+0000","last_clean":"2024-09-17T04:31:54.805856+0000","last_became_active":"2024-09-17T04:31:53.833171+0000","last_became_peered":"2024-09-17T04:31:53.833171+0000","last_unstale":"2024-09-17T04:31:54.805856+0000","last_undegraded":"2024-09-17T04:31:54.805856+0000","last_fullsized":"2024-09-17T04:31:54.805856+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:57.441621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.835530+0000","last_change":"2024-09-17T04:31:52.808009+0000","last_active":"2024-09-17T04:31:54.835530+0000","last_peered":"2024-09-17T04:31:54.835530+0000","last_clean":"2024-09-17T04:31:54.835530+0000","last_became_active":"2024-09-17T04:31:52.807695+0000","last_became_peered":"2024-09-17T04:31:52.807695+0000","last_unstale":"2024-09-17T04:31:54.835530+0000","last_undegraded":"2024-09-17T04:31:54.835530+0000","last_fullsized":"2024-09-17T04:31:54.835530+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:43:54.196275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.201298+0000","last_change":"2024-09-17T04:31:53.833923+0000","last_active":"2024-09-17T04:31:55.201298+0000","last_peered":"2024-09-17T04:31:55.201298+0000","last_clean":"2024-09-17T04:31:55.201298+0000","last_became_active":"2024-09-17T04:31:53.833587+0000","last_became_peered":"2024-09-17T04:31:53.833587+0000","last_unstale":"2024-09-17T04:31:55.201298+0000","last_undegraded":"2024-09-17T04:31:55.201298+0000","last_fullsized":"2024-09-17T04:31:55.201298+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:27.074253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.809697+0000","last_change":"2024-09-17T04:31:52.812508+0000","last_active":"2024-09-17T04:31:54.809697+0000","last_peered":"2024-09-17T04:31:54.809697+0000","last_clean":"2024-09-17T04:31:54.809697+0000","last_became_active":"2024-09-17T04:31:52.812218+0000","last_became_peered":"2024-09-17T04:31:52.812218+0000","last_unstale":"2024-09-17T04:31:54.809697+0000","last_undegraded":"2024-09-17T04:31:54.809697+0000","last_fullsized":"2024-09-17T04:31:54.809697+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:47.649843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.201335+0000","last_change":"2024-09-17T04:31:53.829505+0000","last_active":"2024-09-17T04:31:55.201335+0000","last_peered":"2024-09-17T04:31:55.201335+0000","last_clean":"2024-09-17T04:31:55.201335+0000","last_became_active":"2024-09-17T04:31:53.827433+0000","last_became_peered":"2024-09-17T04:31:53.827433+0000","last_unstale":"2024-09-17T04:31:55.201335+0000","last_undegraded":"2024-09-17T04:31:55.201335+0000","last_fullsized":"2024-09-17T04:31:55.201335+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:20.296448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.806404+0000","last_change":"2024-09-17T04:31:52.820598+0000","last_active":"2024-09-17T04:31:54.806404+0000","last_peered":"2024-09-17T04:31:54.806404+0000","last_clean":"2024-09-17T04:31:54.806404+0000","last_became_active":"2024-09-17T04:31:52.820305+0000","last_became_peered":"2024-09-17T04:31:52.820305+0000","last_unstale":"2024-09-17T04:31:54.806404+0000","last_undegraded":"2024-09-17T04:31:54.806404+0000","last_fullsized":"2024-09-17T04:31:54.806404+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:14.825479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.834781+0000","last_change":"2024-09-17T04:31:53.832664+0000","last_active":"2024-09-17T04:31:54.834781+0000","last_peered":"2024-09-17T04:31:54.834781+0000","last_clean":"2024-09-17T04:31:54.834781+0000","last_became_active":"2024-09-17T04:31:53.832366+0000","last_became_peered":"2024-09-17T04:31:53.832366+0000","last_unstale":"2024-09-17T04:31:54.834781+0000","last_undegraded":"2024-09-17T04:31:54.834781+0000","last_fullsized":"2024-09-17T04:31:54.834781+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:50:40.198578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.834649+0000","last_change":"2024-09-17T04:31:52.817353+0000","last_active":"2024-09-17T04:31:54.834649+0000","last_peered":"2024-09-17T04:31:54.834649+0000","last_clean":"2024-09-17T04:31:54.834649+0000","last_became_active":"2024-09-17T04:31:52.816563+0000","last_became_peered":"2024-09-17T04:31:52.816563+0000","last_unstale":"2024-09-17T04:31:54.834649+0000","last_undegraded":"2024-09-17T04:31:54.834649+0000","last_fullsized":"2024-09-17T04:31:54.834649+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:48:23.269631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.835865+0000","last_change":"2024-09-17T04:31:53.828049+0000","last_active":"2024-09-17T04:31:54.835865+0000","last_peered":"2024-09-17T04:31:54.835865+0000","last_clean":"2024-09-17T04:31:54.835865+0000","last_became_active":"2024-09-17T04:31:53.827756+0000","last_became_peered":"2024-09-17T04:31:53.827756+0000","last_unstale":"2024-09-17T04:31:54.835865+0000","last_undegraded":"2024-09-17T04:31:54.835865+0000","last_fullsized":"2024-09-17T04:31:54.835865+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:08:00.850010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"131'6","reported_seq":21,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.862647+0000","last_change":"2024-09-17T04:31:52.813285+0000","last_active":"2024-09-17T04:31:54.862647+0000","last_peered":"2024-09-17T04:31:54.862647+0000","last_clean":"2024-09-17T04:31:54.862647+0000","last_became_active":"2024-09-17T04:31:52.812960+0000","last_became_peered":"2024-09-17T04:31:52.812960+0000","last_unstale":"2024-09-17T04:31:54.862647+0000","last_undegraded":"2024-09-17T04:31:54.862647+0000","last_fullsized":"2024-09-17T04:31:54.862647+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:38.260845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.201117+0000","last_change":"2024-09-17T04:31:53.821854+0000","last_active":"2024-09-17T04:31:55.201117+0000","last_peered":"2024-09-17T04:31:55.201117+0000","last_clean":"2024-09-17T04:31:55.201117+0000","last_became_active":"2024-09-17T04:31:53.821203+0000","last_became_peered":"2024-09-17T04:31:53.821203+0000","last_unstale":"2024-09-17T04:31:55.201117+0000","last_undegraded":"2024-09-17T04:31:55.201117+0000","last_fullsized":"2024-09-17T04:31:55.201117+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:50.305186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.200568+0000","last_change":"2024-09-17T04:31:52.809827+0000","last_active":"2024-09-17T04:31:55.200568+0000","last_peered":"2024-09-17T04:31:55.200568+0000","last_clean":"2024-09-17T04:31:55.200568+0000","last_became_active":"2024-09-17T04:31:52.808401+0000","last_became_peered":"2024-09-17T04:31:52.808401+0000","last_unstale":"2024-09-17T04:31:55.200568+0000","last_undegraded":"2024-09-17T04:31:55.200568+0000","last_fullsized":"2024-09-17T04:31:55.200568+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:14.333538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.806071+0000","last_change":"2024-09-17T04:31:53.833489+0000","last_active":"2024-09-17T04:31:54.806071+0000","last_peered":"2024-09-17T04:31:54.806071+0000","last_clean":"2024-09-17T04:31:54.806071+0000","last_became_active":"2024-09-17T04:31:53.833168+0000","last_became_peered":"2024-09-17T04:31:53.833168+0000","last_unstale":"2024-09-17T04:31:54.806071+0000","last_undegraded":"2024-09-17T04:31:54.806071+0000","last_fullsized":"2024-09-17T04:31:54.806071+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:18:47.551599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.200568+0000","last_change":"2024-09-17T04:31:52.805253+0000","last_active":"2024-09-17T04:31:55.200568+0000","last_peered":"2024-09-17T04:31:55.200568+0000","last_clean":"2024-09-17T04:31:55.200568+0000","last_became_active":"2024-09-17T04:31:52.804865+0000","last_became_peered":"2024-09-17T04:31:52.804865+0000","last_unstale":"2024-09-17T04:31:55.200568+0000","last_undegraded":"2024-09-17T04:31:55.200568+0000","last_fullsized":"2024-09-17T04:31:55.200568+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:05:17.527290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.200516+0000","last_change":"2024-09-17T04:31:53.839026+0000","last_active":"2024-09-17T04:31:55.200516+0000","last_peered":"2024-09-17T04:31:55.200516+0000","last_clean":"2024-09-17T04:31:55.200516+0000","last_became_active":"2024-09-17T04:31:53.838725+0000","last_became_peered":"2024-09-17T04:31:53.838725+0000","last_unstale":"2024-09-17T04:31:55.200516+0000","last_undegraded":"2024-09-17T04:31:55.200516+0000","last_fullsized":"2024-09-17T04:31:55.200516+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:06:39.666825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.809038+0000","last_change":"2024-09-17T04:31:52.810090+0000","last_active":"2024-09-17T04:31:54.809038+0000","last_peered":"2024-09-17T04:31:54.809038+0000","last_clean":"2024-09-17T04:31:54.809038+0000","last_became_active":"2024-09-17T04:31:52.809785+0000","last_became_peered":"2024-09-17T04:31:52.809785+0000","last_unstale":"2024-09-17T04:31:54.809038+0000","last_undegraded":"2024-09-17T04:31:54.809038+0000","last_fullsized":"2024-09-17T04:31:54.809038+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:06:38.626770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.200306+0000","last_change":"2024-09-17T04:31:53.811615+0000","last_active":"2024-09-17T04:31:55.200306+0000","last_peered":"2024-09-17T04:31:55.200306+0000","last_clean":"2024-09-17T04:31:55.200306+0000","last_became_active":"2024-09-17T04:31:53.811272+0000","last_became_peered":"2024-09-17T04:31:53.811272+0000","last_unstale":"2024-09-17T04:31:55.200306+0000","last_undegraded":"2024-09-17T04:31:55.200306+0000","last_fullsized":"2024-09-17T04:31:55.200306+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:29.647612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"131'8","reported_seq":23,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.835520+0000","last_change":"2024-09-17T04:31:52.800071+0000","last_active":"2024-09-17T04:31:54.835520+0000","last_peered":"2024-09-17T04:31:54.835520+0000","last_clean":"2024-09-17T04:31:54.835520+0000","last_became_active":"2024-09-17T04:31:52.799773+0000","last_became_peered":"2024-09-17T04:31:52.799773+0000","last_unstale":"2024-09-17T04:31:54.835520+0000","last_undegraded":"2024-09-17T04:31:54.835520+0000","last_fullsized":"2024-09-17T04:31:54.835520+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:59:10.346310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.834745+0000","last_change":"2024-09-17T04:31:53.826915+0000","last_active":"2024-09-17T04:31:54.834745+0000","last_peered":"2024-09-17T04:31:54.834745+0000","last_clean":"2024-09-17T04:31:54.834745+0000","last_became_active":"2024-09-17T04:31:53.826249+0000","last_became_peered":"2024-09-17T04:31:53.826249+0000","last_unstale":"2024-09-17T04:31:54.834745+0000","last_undegraded":"2024-09-17T04:31:54.834745+0000","last_fullsized":"2024-09-17T04:31:54.834745+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:03.187030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.200907+0000","last_change":"2024-09-17T04:31:52.806854+0000","last_active":"2024-09-17T04:31:55.200907+0000","last_peered":"2024-09-17T04:31:55.200907+0000","last_clean":"2024-09-17T04:31:55.200907+0000","last_became_active":"2024-09-17T04:31:52.806381+0000","last_became_peered":"2024-09-17T04:31:52.806381+0000","last_unstale":"2024-09-17T04:31:55.200907+0000","last_undegraded":"2024-09-17T04:31:55.200907+0000","last_fullsized":"2024-09-17T04:31:55.200907+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:54.773045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.836104+0000","last_change":"2024-09-17T04:31:53.826533+0000","last_active":"2024-09-17T04:31:54.836104+0000","last_peered":"2024-09-17T04:31:54.836104+0000","last_clean":"2024-09-17T04:31:54.836104+0000","last_became_active":"2024-09-17T04:31:53.826273+0000","last_became_peered":"2024-09-17T04:31:53.826273+0000","last_unstale":"2024-09-17T04:31:54.836104+0000","last_undegraded":"2024-09-17T04:31:54.836104+0000","last_fullsized":"2024-09-17T04:31:54.836104+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:53.659868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.806402+0000","last_change":"2024-09-17T04:31:52.817476+0000","last_active":"2024-09-17T04:31:54.806402+0000","last_peered":"2024-09-17T04:31:54.806402+0000","last_clean":"2024-09-17T04:31:54.806402+0000","last_became_active":"2024-09-17T04:31:52.817119+0000","last_became_peered":"2024-09-17T04:31:52.817119+0000","last_unstale":"2024-09-17T04:31:54.806402+0000","last_undegraded":"2024-09-17T04:31:54.806402+0000","last_fullsized":"2024-09-17T04:31:54.806402+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:54:51.742786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.201090+0000","last_change":"2024-09-17T04:31:53.825946+0000","last_active":"2024-09-17T04:31:55.201090+0000","last_peered":"2024-09-17T04:31:55.201090+0000","last_clean":"2024-09-17T04:31:55.201090+0000","last_became_active":"2024-09-17T04:31:53.825601+0000","last_became_peered":"2024-09-17T04:31:53.825601+0000","last_unstale":"2024-09-17T04:31:55.201090+0000","last_undegraded":"2024-09-17T04:31:55.201090+0000","last_fullsized":"2024-09-17T04:31:55.201090+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:59.465298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.835929+0000","last_change":"2024-09-17T04:31:52.811887+0000","last_active":"2024-09-17T04:31:54.835929+0000","last_peered":"2024-09-17T04:31:54.835929+0000","last_clean":"2024-09-17T04:31:54.835929+0000","last_became_active":"2024-09-17T04:31:52.811595+0000","last_became_peered":"2024-09-17T04:31:52.811595+0000","last_unstale":"2024-09-17T04:31:54.835929+0000","last_undegraded":"2024-09-17T04:31:54.835929+0000","last_fullsized":"2024-09-17T04:31:54.835929+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:04.452476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.809526+0000","last_change":"2024-09-17T04:31:53.827808+0000","last_active":"2024-09-17T04:31:54.809526+0000","last_peered":"2024-09-17T04:31:54.809526+0000","last_clean":"2024-09-17T04:31:54.809526+0000","last_became_active":"2024-09-17T04:31:53.822678+0000","last_became_peered":"2024-09-17T04:31:53.822678+0000","last_unstale":"2024-09-17T04:31:54.809526+0000","last_undegraded":"2024-09-17T04:31:54.809526+0000","last_fullsized":"2024-09-17T04:31:54.809526+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:14:26.496106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.806009+0000","last_change":"2024-09-17T04:31:52.819566+0000","last_active":"2024-09-17T04:31:54.806009+0000","last_peered":"2024-09-17T04:31:54.806009+0000","last_clean":"2024-09-17T04:31:54.806009+0000","last_became_active":"2024-09-17T04:31:52.815635+0000","last_became_peered":"2024-09-17T04:31:52.815635+0000","last_unstale":"2024-09-17T04:31:54.806009+0000","last_undegraded":"2024-09-17T04:31:54.806009+0000","last_fullsized":"2024-09-17T04:31:54.806009+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:47:49.612979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.810526+0000","last_change":"2024-09-17T04:31:53.825307+0000","last_active":"2024-09-17T04:31:54.810526+0000","last_peered":"2024-09-17T04:31:54.810526+0000","last_clean":"2024-09-17T04:31:54.810526+0000","last_became_active":"2024-09-17T04:31:53.824981+0000","last_became_peered":"2024-09-17T04:31:53.824981+0000","last_unstale":"2024-09-17T04:31:54.810526+0000","last_undegraded":"2024-09-17T04:31:54.810526+0000","last_fullsized":"2024-09-17T04:31:54.810526+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:32:50.351790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.862490+0000","last_change":"2024-09-17T04:31:52.811819+0000","last_active":"2024-09-17T04:31:54.862490+0000","last_peered":"2024-09-17T04:31:54.862490+0000","last_clean":"2024-09-17T04:31:54.862490+0000","last_became_active":"2024-09-17T04:31:52.811531+0000","last_became_peered":"2024-09-17T04:31:52.811531+0000","last_unstale":"2024-09-17T04:31:54.862490+0000","last_undegraded":"2024-09-17T04:31:54.862490+0000","last_fullsized":"2024-09-17T04:31:54.862490+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:15:07.089604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.862692+0000","last_change":"2024-09-17T04:31:53.831422+0000","last_active":"2024-09-17T04:31:54.862692+0000","last_peered":"2024-09-17T04:31:54.862692+0000","last_clean":"2024-09-17T04:31:54.862692+0000","last_became_active":"2024-09-17T04:31:53.830057+0000","last_became_peered":"2024-09-17T04:31:53.830057+0000","last_unstale":"2024-09-17T04:31:54.862692+0000","last_undegraded":"2024-09-17T04:31:54.862692+0000","last_fullsized":"2024-09-17T04:31:54.862692+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:37:26.939620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.833536+0000","last_change":"2024-09-17T04:31:52.820609+0000","last_active":"2024-09-17T04:31:54.833536+0000","last_peered":"2024-09-17T04:31:54.833536+0000","last_clean":"2024-09-17T04:31:54.833536+0000","last_became_active":"2024-09-17T04:31:52.820280+0000","last_became_peered":"2024-09-17T04:31:52.820280+0000","last_unstale":"2024-09-17T04:31:54.833536+0000","last_undegraded":"2024-09-17T04:31:54.833536+0000","last_fullsized":"2024-09-17T04:31:54.833536+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:00:11.852634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.809825+0000","last_change":"2024-09-17T04:31:53.831147+0000","last_active":"2024-09-17T04:31:54.809825+0000","last_peered":"2024-09-17T04:31:54.809825+0000","last_clean":"2024-09-17T04:31:54.809825+0000","last_became_active":"2024-09-17T04:31:53.830801+0000","last_became_peered":"2024-09-17T04:31:53.830801+0000","last_unstale":"2024-09-17T04:31:54.809825+0000","last_undegraded":"2024-09-17T04:31:54.809825+0000","last_fullsized":"2024-09-17T04:31:54.809825+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:01.396758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.834190+0000","last_change":"2024-09-17T04:31:52.820832+0000","last_active":"2024-09-17T04:31:54.834190+0000","last_peered":"2024-09-17T04:31:54.834190+0000","last_clean":"2024-09-17T04:31:54.834190+0000","last_became_active":"2024-09-17T04:31:52.820498+0000","last_became_peered":"2024-09-17T04:31:52.820498+0000","last_unstale":"2024-09-17T04:31:54.834190+0000","last_undegraded":"2024-09-17T04:31:54.834190+0000","last_fullsized":"2024-09-17T04:31:54.834190+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:42.551098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.862454+0000","last_change":"2024-09-17T04:31:53.834522+0000","last_active":"2024-09-17T04:31:54.862454+0000","last_peered":"2024-09-17T04:31:54.862454+0000","last_clean":"2024-09-17T04:31:54.862454+0000","last_became_active":"2024-09-17T04:31:53.830872+0000","last_became_peered":"2024-09-17T04:31:53.830872+0000","last_unstale":"2024-09-17T04:31:54.862454+0000","last_undegraded":"2024-09-17T04:31:54.862454+0000","last_fullsized":"2024-09-17T04:31:54.862454+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:54:13.505240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.199978+0000","last_change":"2024-09-17T04:31:52.812657+0000","last_active":"2024-09-17T04:31:55.199978+0000","last_peered":"2024-09-17T04:31:55.199978+0000","last_clean":"2024-09-17T04:31:55.199978+0000","last_became_active":"2024-09-17T04:31:52.812349+0000","last_became_peered":"2024-09-17T04:31:52.812349+0000","last_unstale":"2024-09-17T04:31:55.199978+0000","last_undegraded":"2024-09-17T04:31:55.199978+0000","last_fullsized":"2024-09-17T04:31:55.199978+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:53:14.658963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.201089+0000","last_change":"2024-09-17T04:31:53.840953+0000","last_active":"2024-09-17T04:31:55.201089+0000","last_peered":"2024-09-17T04:31:55.201089+0000","last_clean":"2024-09-17T04:31:55.201089+0000","last_became_active":"2024-09-17T04:31:53.840535+0000","last_became_peered":"2024-09-17T04:31:53.840535+0000","last_unstale":"2024-09-17T04:31:55.201089+0000","last_undegraded":"2024-09-17T04:31:55.201089+0000","last_fullsized":"2024-09-17T04:31:55.201089+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:59:15.662092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.834913+0000","last_change":"2024-09-17T04:31:52.818964+0000","last_active":"2024-09-17T04:31:54.834913+0000","last_peered":"2024-09-17T04:31:54.834913+0000","last_clean":"2024-09-17T04:31:54.834913+0000","last_became_active":"2024-09-17T04:31:52.818639+0000","last_became_peered":"2024-09-17T04:31:52.818639+0000","last_unstale":"2024-09-17T04:31:54.834913+0000","last_undegraded":"2024-09-17T04:31:54.834913+0000","last_fullsized":"2024-09-17T04:31:54.834913+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:49.265130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.201060+0000","last_change":"2024-09-17T04:31:53.824675+0000","last_active":"2024-09-17T04:31:55.201060+0000","last_peered":"2024-09-17T04:31:55.201060+0000","last_clean":"2024-09-17T04:31:55.201060+0000","last_became_active":"2024-09-17T04:31:53.823472+0000","last_became_peered":"2024-09-17T04:31:53.823472+0000","last_unstale":"2024-09-17T04:31:55.201060+0000","last_undegraded":"2024-09-17T04:31:55.201060+0000","last_fullsized":"2024-09-17T04:31:55.201060+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:01.396758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"132'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.830905+0000","last_change":"2024-09-17T04:31:52.818353+0000","last_active":"2024-09-17T04:31:54.830905+0000","last_peered":"2024-09-17T04:31:54.830905+0000","last_clean":"2024-09-17T04:31:54.830905+0000","last_became_active":"2024-09-17T04:31:52.818069+0000","last_became_peered":"2024-09-17T04:31:52.818069+0000","last_unstale":"2024-09-17T04:31:54.830905+0000","last_undegraded":"2024-09-17T04:31:54.830905+0000","last_fullsized":"2024-09-17T04:31:54.830905+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:31.484410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.806025+0000","last_change":"2024-09-17T04:31:53.817802+0000","last_active":"2024-09-17T04:31:54.806025+0000","last_peered":"2024-09-17T04:31:54.806025+0000","last_clean":"2024-09-17T04:31:54.806025+0000","last_became_active":"2024-09-17T04:31:53.816398+0000","last_became_peered":"2024-09-17T04:31:53.816398+0000","last_unstale":"2024-09-17T04:31:54.806025+0000","last_undegraded":"2024-09-17T04:31:54.806025+0000","last_fullsized":"2024-09-17T04:31:54.806025+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:36.110338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.810059+0000","last_change":"2024-09-17T04:31:52.816351+0000","last_active":"2024-09-17T04:31:54.810059+0000","last_peered":"2024-09-17T04:31:54.810059+0000","last_clean":"2024-09-17T04:31:54.810059+0000","last_became_active":"2024-09-17T04:31:52.815589+0000","last_became_peered":"2024-09-17T04:31:52.815589+0000","last_unstale":"2024-09-17T04:31:54.810059+0000","last_undegraded":"2024-09-17T04:31:54.810059+0000","last_fullsized":"2024-09-17T04:31:54.810059+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:23:18.409482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.809745+0000","last_change":"2024-09-17T04:31:53.826718+0000","last_active":"2024-09-17T04:31:54.809745+0000","last_peered":"2024-09-17T04:31:54.809745+0000","last_clean":"2024-09-17T04:31:54.809745+0000","last_became_active":"2024-09-17T04:31:53.826049+0000","last_became_peered":"2024-09-17T04:31:53.826049+0000","last_unstale":"2024-09-17T04:31:54.809745+0000","last_undegraded":"2024-09-17T04:31:54.809745+0000","last_fullsized":"2024-09-17T04:31:54.809745+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:01:05.856066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.809416+0000","last_change":"2024-09-17T04:31:52.818269+0000","last_active":"2024-09-17T04:31:54.809416+0000","last_peered":"2024-09-17T04:31:54.809416+0000","last_clean":"2024-09-17T04:31:54.809416+0000","last_became_active":"2024-09-17T04:31:52.815665+0000","last_became_peered":"2024-09-17T04:31:52.815665+0000","last_unstale":"2024-09-17T04:31:54.809416+0000","last_undegraded":"2024-09-17T04:31:54.809416+0000","last_fullsized":"2024-09-17T04:31:54.809416+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:46:07.619505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.835499+0000","last_change":"2024-09-17T04:31:53.814349+0000","last_active":"2024-09-17T04:31:54.835499+0000","last_peered":"2024-09-17T04:31:54.835499+0000","last_clean":"2024-09-17T04:31:54.835499+0000","last_became_active":"2024-09-17T04:31:53.813970+0000","last_became_peered":"2024-09-17T04:31:53.813970+0000","last_unstale":"2024-09-17T04:31:54.835499+0000","last_undegraded":"2024-09-17T04:31:54.835499+0000","last_fullsized":"2024-09-17T04:31:54.835499+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:18:17.261496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.862617+0000","last_change":"2024-09-17T04:31:52.812029+0000","last_active":"2024-09-17T04:31:54.862617+0000","last_peered":"2024-09-17T04:31:54.862617+0000","last_clean":"2024-09-17T04:31:54.862617+0000","last_became_active":"2024-09-17T04:31:52.811665+0000","last_became_peered":"2024-09-17T04:31:52.811665+0000","last_unstale":"2024-09-17T04:31:54.862617+0000","last_undegraded":"2024-09-17T04:31:54.862617+0000","last_fullsized":"2024-09-17T04:31:54.862617+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:23:28.995121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.201415+0000","last_change":"2024-09-17T04:31:53.831041+0000","last_active":"2024-09-17T04:31:55.201415+0000","last_peered":"2024-09-17T04:31:55.201415+0000","last_clean":"2024-09-17T04:31:55.201415+0000","last_became_active":"2024-09-17T04:31:53.830698+0000","last_became_peered":"2024-09-17T04:31:53.830698+0000","last_unstale":"2024-09-17T04:31:55.201415+0000","last_undegraded":"2024-09-17T04:31:55.201415+0000","last_fullsized":"2024-09-17T04:31:55.201415+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:28.360388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.834431+0000","last_change":"2024-09-17T04:31:52.819271+0000","last_active":"2024-09-17T04:31:54.834431+0000","last_peered":"2024-09-17T04:31:54.834431+0000","last_clean":"2024-09-17T04:31:54.834431+0000","last_became_active":"2024-09-17T04:31:52.818943+0000","last_became_peered":"2024-09-17T04:31:52.818943+0000","last_unstale":"2024-09-17T04:31:54.834431+0000","last_undegraded":"2024-09-17T04:31:54.834431+0000","last_fullsized":"2024-09-17T04:31:54.834431+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:04:48.413800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.833582+0000","last_change":"2024-09-17T04:31:53.826661+0000","last_active":"2024-09-17T04:31:54.833582+0000","last_peered":"2024-09-17T04:31:54.833582+0000","last_clean":"2024-09-17T04:31:54.833582+0000","last_became_active":"2024-09-17T04:31:53.826318+0000","last_became_peered":"2024-09-17T04:31:53.826318+0000","last_unstale":"2024-09-17T04:31:54.833582+0000","last_undegraded":"2024-09-17T04:31:54.833582+0000","last_fullsized":"2024-09-17T04:31:54.833582+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:45:50.457360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.200569+0000","last_change":"2024-09-17T04:31:52.808661+0000","last_active":"2024-09-17T04:31:55.200569+0000","last_peered":"2024-09-17T04:31:55.200569+0000","last_clean":"2024-09-17T04:31:55.200569+0000","last_became_active":"2024-09-17T04:31:52.808355+0000","last_became_peered":"2024-09-17T04:31:52.808355+0000","last_unstale":"2024-09-17T04:31:55.200569+0000","last_undegraded":"2024-09-17T04:31:55.200569+0000","last_fullsized":"2024-09-17T04:31:55.200569+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:47.649843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.201337+0000","last_change":"2024-09-17T04:31:53.827755+0000","last_active":"2024-09-17T04:31:55.201337+0000","last_peered":"2024-09-17T04:31:55.201337+0000","last_clean":"2024-09-17T04:31:55.201337+0000","last_became_active":"2024-09-17T04:31:53.825215+0000","last_became_peered":"2024-09-17T04:31:53.825215+0000","last_unstale":"2024-09-17T04:31:55.201337+0000","last_undegraded":"2024-09-17T04:31:55.201337+0000","last_fullsized":"2024-09-17T04:31:55.201337+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:03.852438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.806173+0000","last_change":"2024-09-17T04:31:52.817238+0000","last_active":"2024-09-17T04:31:54.806173+0000","last_peered":"2024-09-17T04:31:54.806173+0000","last_clean":"2024-09-17T04:31:54.806173+0000","last_became_active":"2024-09-17T04:31:52.816605+0000","last_became_peered":"2024-09-17T04:31:52.816605+0000","last_unstale":"2024-09-17T04:31:54.806173+0000","last_undegraded":"2024-09-17T04:31:54.806173+0000","last_fullsized":"2024-09-17T04:31:54.806173+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:04.753249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.862809+0000","last_change":"2024-09-17T04:31:53.834167+0000","last_active":"2024-09-17T04:31:54.862809+0000","last_peered":"2024-09-17T04:31:54.862809+0000","last_clean":"2024-09-17T04:31:54.862809+0000","last_became_active":"2024-09-17T04:31:53.831570+0000","last_became_peered":"2024-09-17T04:31:53.831570+0000","last_unstale":"2024-09-17T04:31:54.862809+0000","last_undegraded":"2024-09-17T04:31:54.862809+0000","last_fullsized":"2024-09-17T04:31:54.862809+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:52.786090+0000","last_clean_scrub_stamp":"2024-09-17T04:31:52.786090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:49.369165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"131'3","reported_seq":18,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:55.200787+0000","last_change":"2024-09-17T04:31:52.815532+0000","last_active":"2024-09-17T04:31:55.200787+0000","last_peered":"2024-09-17T04:31:55.200787+0000","last_clean":"2024-09-17T04:31:55.200787+0000","last_became_active":"2024-09-17T04:31:52.814081+0000","last_became_peered":"2024-09-17T04:31:52.814081+0000","last_unstale":"2024-09-17T04:31:55.200787+0000","last_undegraded":"2024-09-17T04:31:55.200787+0000","last_fullsized":"2024-09-17T04:31:55.200787+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:31:51.746034+0000","last_clean_scrub_stamp":"2024-09-17T04:31:51.746034+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:58.445527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"76'79","reported_seq":341,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T04:31:54.833756+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:31:54.833756+0000","last_peered":"2024-09-17T04:31:54.833756+0000","last_clean":"2024-09-17T04:31:54.833756+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:31:54.833756+0000","last_undegraded":"2024-09-17T04:31:54.833756+0000","last_fullsized":"2024-09-17T04:31:54.833756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476836,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33380,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338460,"statfs":{"total":96636764160,"available":96602583040,"internally_reserved":0,"allocated":2244608,"data_stored":1314308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476835,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32804,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339036,"statfs":{"total":96636764160,"available":96603172864,"internally_reserved":0,"allocated":1654784,"data_stored":723434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476836,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1646592,"data_stored":723310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476836,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32796,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339044,"statfs":{"total":96636764160,"available":96603181056,"internally_reserved":0,"allocated":1642496,"data_stored":723288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542243,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1646592,"data_stored":723378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509539,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1658880,"data_stored":724970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509538,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1646592,"data_stored":723870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509538,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33392,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338448,"statfs":{"total":96636764160,"available":96602570752,"internally_reserved":0,"allocated":2252800,"data_stored":1315302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:31:58.736 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T04:31:58.736 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T04:31:59.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.999+0000 7fe693400640 1 -- 172.21.15.100:0/3290458774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c074440 msgr2=0x7fe68c06ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:59.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:58.999+0000 7fe693400640 1 --2- 172.21.15.100:0/3290458774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c074440 0x7fe68c06ddb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009860 tx=0x7fe67c02a510 comp rx=0 tx=0).stop 2024-09-17T04:31:59.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.000+0000 7fe693400640 1 -- 172.21.15.100:0/3290458774 shutdown_connections 2024-09-17T04:31:59.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.000+0000 7fe693400640 1 --2- 172.21.15.100:0/3290458774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c074440 0x7fe68c06ddb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:59.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.000+0000 7fe693400640 1 -- 172.21.15.100:0/3290458774 >> 172.21.15.100:0/3290458774 conn(0x7fe68c073920 msgr2=0x7fe68c073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:59.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.000+0000 7fe693400640 1 -- 172.21.15.100:0/3290458774 shutdown_connections 2024-09-17T04:31:59.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.000+0000 7fe693400640 1 -- 172.21.15.100:0/3290458774 wait complete. 2024-09-17T04:31:59.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.002+0000 7fe693400640 1 Processor -- start 2024-09-17T04:31:59.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.002+0000 7fe693400640 1 -- start start 2024-09-17T04:31:59.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.003+0000 7fe693400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe68c070a70 0x7fe68c1e4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:59.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.003+0000 7fe693400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe68c074440 0x7fe68c13d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:59.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.003+0000 7fe693400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c0d0380 0x7fe68c13d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:59.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.003+0000 7fe693400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c15bcb0 con 0x7fe68c070a70 2024-09-17T04:31:59.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.003+0000 7fe693400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c15bb30 con 0x7fe68c074440 2024-09-17T04:31:59.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.003+0000 7fe693400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe68c06e2f0 con 0x7fe68c0d0380 2024-09-17T04:31:59.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.004+0000 7fe690a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe68c070a70 0x7fe68c1e4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:59.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.004+0000 7fe690a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe68c070a70 0x7fe68c1e4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48824/0 (socket says 172.21.15.100:48824) 2024-09-17T04:31:59.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.004+0000 7fe690a00640 1 -- 172.21.15.100:0/1827733445 learned_addr learned my addr 172.21.15.100:0/1827733445 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:59.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.004+0000 7fe691400640 1 --2- 172.21.15.100:0/1827733445 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c0d0380 0x7fe68c13d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:59.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.004+0000 7fe683e00640 1 --2- 172.21.15.100:0/1827733445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe68c074440 0x7fe68c13d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:59.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.004+0000 7fe690a00640 1 -- 172.21.15.100:0/1827733445 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c0d0380 msgr2=0x7fe68c13d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:59.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.004+0000 7fe690a00640 1 --2- 172.21.15.100:0/1827733445 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c0d0380 0x7fe68c13d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:59.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.004+0000 7fe690a00640 1 -- 172.21.15.100:0/1827733445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe68c074440 msgr2=0x7fe68c13d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:59.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.004+0000 7fe690a00640 1 --2- 172.21.15.100:0/1827733445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe68c074440 0x7fe68c13d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:59.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.004+0000 7fe690a00640 1 -- 172.21.15.100:0/1827733445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68c13dd70 con 0x7fe68c070a70 2024-09-17T04:31:59.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.005+0000 7fe690a00640 1 --2- 172.21.15.100:0/1827733445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe68c070a70 0x7fe68c1e4390 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009980 tx=0x7fe67c033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:59.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.005+0000 7fe689600640 1 -- 172.21.15.100:0/1827733445 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c031b50 con 0x7fe68c070a70 2024-09-17T04:31:59.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.005+0000 7fe693400640 1 -- 172.21.15.100:0/1827733445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68c1526c0 con 0x7fe68c070a70 2024-09-17T04:31:59.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.005+0000 7fe689600640 1 -- 172.21.15.100:0/1827733445 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe67c031de0 con 0x7fe68c070a70 2024-09-17T04:31:59.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.005+0000 7fe689600640 1 -- 172.21.15.100:0/1827733445 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c033330 con 0x7fe68c070a70 2024-09-17T04:31:59.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.005+0000 7fe693400640 1 -- 172.21.15.100:0/1827733445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe68c152c40 con 0x7fe68c070a70 2024-09-17T04:31:59.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.006+0000 7fe693400640 1 -- 172.21.15.100:0/1827733445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe68c06ddb0 con 0x7fe68c070a70 2024-09-17T04:31:59.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.007+0000 7fe689600640 1 -- 172.21.15.100:0/1827733445 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fe67c0334d0 con 0x7fe68c070a70 2024-09-17T04:31:59.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.008+0000 7fe689600640 1 --2- 172.21.15.100:0/1827733445 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe6680791c0 0x7fe66807b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:59.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.008+0000 7fe689600640 1 -- 172.21.15.100:0/1827733445 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7fe67c0bca00 con 0x7fe68c070a70 2024-09-17T04:31:59.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.009+0000 7fe683e00640 1 --2- 172.21.15.100:0/1827733445 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe6680791c0 0x7fe66807b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:59.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.009+0000 7fe683e00640 1 --2- 172.21.15.100:0/1827733445 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe6680791c0 0x7fe66807b6d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe674007fd0 tx=0x7fe674007eb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:59.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.013+0000 7fe689600640 1 -- 172.21.15.100:0/1827733445 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe67c03f030 con 0x7fe68c070a70 2024-09-17T04:31:59.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.265+0000 7fe693400640 1 -- 172.21.15.100:0/1827733445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe68c15eae0 con 0x7fe68c070a70 2024-09-17T04:31:59.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.268+0000 7fe689600640 1 -- 172.21.15.100:0/1827733445 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) ==== 190+0+147 (secure 0 0 0) 0x7fe67c031770 con 0x7fe68c070a70 2024-09-17T04:31:59.269 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-09-17T04:31:59.269 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAsBOlmfYg9IBAABP1uRKGOugrA47VxwRLkjA== 2024-09-17T04:31:59.269 INFO:teuthology.orchestra.run.smithi100.stdout: caps mds = "allow" 2024-09-17T04:31:59.269 INFO:teuthology.orchestra.run.smithi100.stdout: caps mon = "allow r" 2024-09-17T04:31:59.269 INFO:teuthology.orchestra.run.smithi100.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T04:31:59.269 INFO:teuthology.orchestra.run.smithi100.stderr:updated caps for client.0 2024-09-17T04:31:59.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.271+0000 7fe693400640 1 -- 172.21.15.100:0/1827733445 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe6680791c0 msgr2=0x7fe66807b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:59.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.271+0000 7fe693400640 1 --2- 172.21.15.100:0/1827733445 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe6680791c0 0x7fe66807b6d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe674007fd0 tx=0x7fe674007eb0 comp rx=0 tx=0).stop 2024-09-17T04:31:59.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.271+0000 7fe693400640 1 -- 172.21.15.100:0/1827733445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe68c070a70 msgr2=0x7fe68c1e4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:59.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.271+0000 7fe693400640 1 --2- 172.21.15.100:0/1827733445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe68c070a70 0x7fe68c1e4390 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009980 tx=0x7fe67c033ba0 comp rx=0 tx=0).stop 2024-09-17T04:31:59.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.271+0000 7fe693400640 1 -- 172.21.15.100:0/1827733445 shutdown_connections 2024-09-17T04:31:59.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.271+0000 7fe693400640 1 --2- 172.21.15.100:0/1827733445 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe6680791c0 0x7fe66807b6d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:59.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.271+0000 7fe693400640 1 --2- 172.21.15.100:0/1827733445 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c0d0380 0x7fe68c13d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:59.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.271+0000 7fe693400640 1 --2- 172.21.15.100:0/1827733445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe68c074440 0x7fe68c13d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:59.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.271+0000 7fe693400640 1 --2- 172.21.15.100:0/1827733445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe68c070a70 0x7fe68c1e4390 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:59.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.271+0000 7fe693400640 1 -- 172.21.15.100:0/1827733445 >> 172.21.15.100:0/1827733445 conn(0x7fe68c073920 msgr2=0x7fe68c1569a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:59.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.271+0000 7fe693400640 1 -- 172.21.15.100:0/1827733445 shutdown_connections 2024-09-17T04:31:59.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.271+0000 7fe693400640 1 -- 172.21.15.100:0/1827733445 wait complete. 2024-09-17T04:31:59.294 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:31:59.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.513+0000 7f69cb400640 1 -- 172.21.15.100:0/2560822446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69ac003a50 msgr2=0x7f69ac003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:59.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.513+0000 7f69cb400640 1 --2- 172.21.15.100:0/2560822446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69ac003a50 0x7f69ac003ec0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f69b8009d00 tx=0x7f69ac005830 comp rx=0 tx=0).stop 2024-09-17T04:31:59.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.513+0000 7f69cb400640 1 -- 172.21.15.100:0/2560822446 shutdown_connections 2024-09-17T04:31:59.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.513+0000 7f69cb400640 1 --2- 172.21.15.100:0/2560822446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69ac003a50 0x7f69ac003ec0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:59.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.513+0000 7f69cb400640 1 -- 172.21.15.100:0/2560822446 >> 172.21.15.100:0/2560822446 conn(0x7f69c406d730 msgr2=0x7f69c40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:59.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.514+0000 7f69cb400640 1 -- 172.21.15.100:0/2560822446 shutdown_connections 2024-09-17T04:31:59.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.514+0000 7f69cb400640 1 -- 172.21.15.100:0/2560822446 wait complete. 2024-09-17T04:31:59.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.516+0000 7f69cb400640 1 Processor -- start 2024-09-17T04:31:59.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.516+0000 7f69cb400640 1 -- start start 2024-09-17T04:31:59.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.517+0000 7f69cb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69ac003a50 0x7f69c41f6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:59.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.517+0000 7f69cb400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69c415bc40 0x7f69c409c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:59.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.517+0000 7f69cb400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69c4155400 0x7f69c415b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:59.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.517+0000 7f69cb400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f69c4073890 con 0x7f69ac003a50 2024-09-17T04:31:59.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.517+0000 7f69cb400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f69c4073710 con 0x7f69c4155400 2024-09-17T04:31:59.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.517+0000 7f69cb400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f69c406f0a0 con 0x7f69c415bc40 2024-09-17T04:31:59.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.517+0000 7f69c8a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69ac003a50 0x7f69c41f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:59.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.518+0000 7f69c8a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69ac003a50 0x7f69c41f6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48840/0 (socket says 172.21.15.100:48840) 2024-09-17T04:31:59.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.518+0000 7f69c8a00640 1 -- 172.21.15.100:0/2854270411 learned_addr learned my addr 172.21.15.100:0/2854270411 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:31:59.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.518+0000 7f69c9400640 1 --2- 172.21.15.100:0/2854270411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69c4155400 0x7f69c415b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:59.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.518+0000 7f69c3e00640 1 --2- 172.21.15.100:0/2854270411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69c415bc40 0x7f69c409c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:59.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.518+0000 7f69c8a00640 1 -- 172.21.15.100:0/2854270411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69c415bc40 msgr2=0x7f69c409c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:59.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.518+0000 7f69c8a00640 1 --2- 172.21.15.100:0/2854270411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69c415bc40 0x7f69c409c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:59.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.518+0000 7f69c8a00640 1 -- 172.21.15.100:0/2854270411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69c4155400 msgr2=0x7f69c415b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:59.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.518+0000 7f69c8a00640 1 --2- 172.21.15.100:0/2854270411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69c4155400 0x7f69c415b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:59.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.518+0000 7f69c8a00640 1 -- 172.21.15.100:0/2854270411 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c415c450 con 0x7f69ac003a50 2024-09-17T04:31:59.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.518+0000 7f69c8a00640 1 --2- 172.21.15.100:0/2854270411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69ac003a50 0x7f69c41f6680 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f69b8009e20 tx=0x7f69b80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:59.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.518+0000 7f69c0c00640 1 -- 172.21.15.100:0/2854270411 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b8038040 con 0x7f69ac003a50 2024-09-17T04:31:59.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.519+0000 7f69c0c00640 1 -- 172.21.15.100:0/2854270411 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69b802b9f0 con 0x7f69ac003a50 2024-09-17T04:31:59.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.519+0000 7f69cb400640 1 -- 172.21.15.100:0/2854270411 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69c415a270 con 0x7f69ac003a50 2024-09-17T04:31:59.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.519+0000 7f69c0c00640 1 -- 172.21.15.100:0/2854270411 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b8003cf0 con 0x7f69ac003a50 2024-09-17T04:31:59.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.519+0000 7f69cb400640 1 -- 172.21.15.100:0/2854270411 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69c415a830 con 0x7f69ac003a50 2024-09-17T04:31:59.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.520+0000 7f69cb400640 1 -- 172.21.15.100:0/2854270411 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69c4162560 con 0x7f69ac003a50 2024-09-17T04:31:59.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.521+0000 7f69c0c00640 1 -- 172.21.15.100:0/2854270411 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f69b8043020 con 0x7f69ac003a50 2024-09-17T04:31:59.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.522+0000 7f69c0c00640 1 --2- 172.21.15.100:0/2854270411 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f69a40791f0 0x7f69a407b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:31:59.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.522+0000 7f69c0c00640 1 -- 172.21.15.100:0/2854270411 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7f69b80bbf70 con 0x7f69ac003a50 2024-09-17T04:31:59.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.522+0000 7f69c3e00640 1 --2- 172.21.15.100:0/2854270411 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f69a40791f0 0x7f69a407b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:31:59.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.523+0000 7f69c3e00640 1 --2- 172.21.15.100:0/2854270411 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f69a40791f0 0x7f69a407b700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f69c409e5e0 tx=0x7f69b0006950 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:31:59.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.527+0000 7f69c0c00640 1 -- 172.21.15.100:0/2854270411 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f69b804a020 con 0x7f69ac003a50 2024-09-17T04:31:59.617 INFO:tasks.ceph.mds.c.smithi100.stderr:2024-09-17T04:31:59.617+0000 7fb7b7c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T04:31:59.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.746+0000 7f69cb400640 1 -- 172.21.15.100:0/2854270411 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69c40b4020 con 0x7f69ac003a50 2024-09-17T04:31:59.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.747+0000 7f69c0c00640 1 -- 172.21.15.100:0/2854270411 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5239 (secure 0 0 0) 0x7f69b802bbc0 con 0x7f69ac003a50 2024-09-17T04:31:59.748 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:31:59.748 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":91,"btime":"2024-09-17T04:31:56:835735+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6719,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1392663867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1392663867},{"type":"v1","addr":"172.21.15.184:6801","nonce":1392663867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":7516,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3257629487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3257629487},{"type":"v1","addr":"172.21.15.184:6835","nonce":3257629487}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":7962,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/2131565869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2131565869},{"type":"v1","addr":"172.21.15.100:6834","nonce":2131565869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:31:53.292918+0000","modified":"2024-09-17T04:31:56.320263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7965},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7965":{"gid":7965,"name":"c","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3047460644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":3047460644},{"type":"v1","addr":"172.21.15.100:6835","nonce":3047460644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7965,"qdb_cluster":[7965]},"id":10}]} 2024-09-17T04:31:59.748 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 91 2024-09-17T04:31:59.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.749+0000 7f69cb400640 1 -- 172.21.15.100:0/2854270411 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f69a40791f0 msgr2=0x7f69a407b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:59.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.749+0000 7f69cb400640 1 --2- 172.21.15.100:0/2854270411 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f69a40791f0 0x7f69a407b700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f69c409e5e0 tx=0x7f69b0006950 comp rx=0 tx=0).stop 2024-09-17T04:31:59.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.749+0000 7f69cb400640 1 -- 172.21.15.100:0/2854270411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69ac003a50 msgr2=0x7f69c41f6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:31:59.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.749+0000 7f69cb400640 1 --2- 172.21.15.100:0/2854270411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69ac003a50 0x7f69c41f6680 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f69b8009e20 tx=0x7f69b80038a0 comp rx=0 tx=0).stop 2024-09-17T04:31:59.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.749+0000 7f69cb400640 1 -- 172.21.15.100:0/2854270411 shutdown_connections 2024-09-17T04:31:59.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.749+0000 7f69cb400640 1 --2- 172.21.15.100:0/2854270411 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f69a40791f0 0x7f69a407b700 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:59.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.749+0000 7f69cb400640 1 --2- 172.21.15.100:0/2854270411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69c4155400 0x7f69c415b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:59.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.750+0000 7f69cb400640 1 --2- 172.21.15.100:0/2854270411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69c415bc40 0x7f69c409c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:59.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.750+0000 7f69cb400640 1 --2- 172.21.15.100:0/2854270411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69ac003a50 0x7f69c41f6680 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:31:59.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.750+0000 7f69cb400640 1 -- 172.21.15.100:0/2854270411 >> 172.21.15.100:0/2854270411 conn(0x7f69c406d730 msgr2=0x7f69c40b4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:31:59.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.750+0000 7f69cb400640 1 -- 172.21.15.100:0/2854270411 shutdown_connections 2024-09-17T04:31:59.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:31:59.750+0000 7f69cb400640 1 -- 172.21.15.100:0/2854270411 wait complete. 2024-09-17T04:32:00.174 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 91, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T04:31:53.292918+0000', 'modified': '2024-09-17T04:31:56.320263+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7965}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7965': {'gid': 7965, 'name': 'c', 'rank': 0, 'incarnation': 87, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6835/3047460644', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6833', 'nonce': 3047460644}, {'type': 'v1', 'addr': '172.21.15.100:6835', 'nonce': 3047460644}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7965, 'qdb_cluster': [7965]} 2024-09-17T04:32:00.174 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T04:32:00.175 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T04:32:00.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.395+0000 7fadb2e00640 1 -- 172.21.15.100:0/1654719536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadac15d540 msgr2=0x7fadac156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:00.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.395+0000 7fadb2e00640 1 --2- 172.21.15.100:0/1654719536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadac15d540 0x7fadac156e40 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fada4009be0 tx=0x7fada402a8d0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.396+0000 7fadb2e00640 1 -- 172.21.15.100:0/1654719536 shutdown_connections 2024-09-17T04:32:00.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.396+0000 7fadb2e00640 1 --2- 172.21.15.100:0/1654719536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadac15d540 0x7fadac156e40 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.396+0000 7fadb2e00640 1 -- 172.21.15.100:0/1654719536 >> 172.21.15.100:0/1654719536 conn(0x7fadac06d3d0 msgr2=0x7fadac0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:00.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.396+0000 7fadb2e00640 1 -- 172.21.15.100:0/1654719536 shutdown_connections 2024-09-17T04:32:00.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.396+0000 7fadb2e00640 1 -- 172.21.15.100:0/1654719536 wait complete. 2024-09-17T04:32:00.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.398+0000 7fadb2e00640 1 Processor -- start 2024-09-17T04:32:00.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.398+0000 7fadb2e00640 1 -- start start 2024-09-17T04:32:00.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.399+0000 7fadb2e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadac0cfc80 0x7fadac1e9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:00.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.399+0000 7fadb2e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadac15d540 0x7fadac0b0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:00.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.399+0000 7fadb2e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadac1f61a0 0x7fadac156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:00.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.400+0000 7fadb2e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fadac0cd100 con 0x7fadac0cfc80 2024-09-17T04:32:00.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.400+0000 7fadb2e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fadac0ccf80 con 0x7fadac1f61a0 2024-09-17T04:32:00.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.400+0000 7fadb2e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fadac0cd280 con 0x7fadac15d540 2024-09-17T04:32:00.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.400+0000 7fadabe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadac0cfc80 0x7fadac1e9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:00.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.400+0000 7fadabe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadac0cfc80 0x7fadac1e9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48848/0 (socket says 172.21.15.100:48848) 2024-09-17T04:32:00.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.400+0000 7fadabe00640 1 -- 172.21.15.100:0/3978124622 learned_addr learned my addr 172.21.15.100:0/3978124622 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:00.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.400+0000 7fadab400640 1 --2- 172.21.15.100:0/3978124622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadac15d540 0x7fadac0b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:00.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.400+0000 7fadb0e00640 1 --2- 172.21.15.100:0/3978124622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadac1f61a0 0x7fadac156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:00.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.400+0000 7fadabe00640 1 -- 172.21.15.100:0/3978124622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadac15d540 msgr2=0x7fadac0b0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:00.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.400+0000 7fadabe00640 1 --2- 172.21.15.100:0/3978124622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadac15d540 0x7fadac0b0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.400+0000 7fadabe00640 1 -- 172.21.15.100:0/3978124622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadac1f61a0 msgr2=0x7fadac156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:00.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.400+0000 7fadabe00640 1 --2- 172.21.15.100:0/3978124622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadac1f61a0 0x7fadac156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.400+0000 7fadabe00640 1 -- 172.21.15.100:0/3978124622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadac0aff10 con 0x7fadac0cfc80 2024-09-17T04:32:00.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.400+0000 7fadabe00640 1 --2- 172.21.15.100:0/3978124622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadac0cfc80 0x7fadac1e9d00 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fad9c001eb0 tx=0x7fad9c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:00.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.401+0000 7fada3e00640 1 -- 172.21.15.100:0/3978124622 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c017070 con 0x7fadac0cfc80 2024-09-17T04:32:00.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.401+0000 7fada3e00640 1 -- 172.21.15.100:0/3978124622 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad9c0053c0 con 0x7fadac0cfc80 2024-09-17T04:32:00.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.401+0000 7fada3e00640 1 -- 172.21.15.100:0/3978124622 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c003830 con 0x7fadac0cfc80 2024-09-17T04:32:00.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.401+0000 7fadb2e00640 1 -- 172.21.15.100:0/3978124622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadac0ad260 con 0x7fadac0cfc80 2024-09-17T04:32:00.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.401+0000 7fadb2e00640 1 -- 172.21.15.100:0/3978124622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadac0ad820 con 0x7fadac0cfc80 2024-09-17T04:32:00.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.403+0000 7fada3e00640 1 -- 172.21.15.100:0/3978124622 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fad9c01a070 con 0x7fadac0cfc80 2024-09-17T04:32:00.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.404+0000 7fada3e00640 1 --2- 172.21.15.100:0/3978124622 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fad8c0791c0 0x7fad8c07b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:00.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.404+0000 7fada3e00640 1 -- 172.21.15.100:0/3978124622 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7fad9c09ace0 con 0x7fadac0cfc80 2024-09-17T04:32:00.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.404+0000 7fadab400640 1 --2- 172.21.15.100:0/3978124622 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fad8c0791c0 0x7fad8c07b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:00.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.405+0000 7fadb9e02740 1 Processor -- start 2024-09-17T04:32:00.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.405+0000 7fadab400640 1 --2- 172.21.15.100:0/3978124622 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fad8c0791c0 0x7fad8c07b6d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fadac0b01d0 tx=0x7fad98009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:00.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.405+0000 7fadb9e02740 1 -- start start 2024-09-17T04:32:00.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.406+0000 7fadb9e02740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f757f83270 0x55f758353970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:00.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.406+0000 7fadb9e02740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f757f5a3b0 0x55f758353eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:00.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.406+0000 7fadb9e02740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f757f62590 0x55f758323f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:00.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.406+0000 7fadb9e02740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f7582ad430 con 0x55f757f5a3b0 2024-09-17T04:32:00.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.406+0000 7fadb9e02740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f75829d5d0 con 0x55f757f83270 2024-09-17T04:32:00.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.406+0000 7fadb9e02740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55f75828ca00 con 0x55f757f62590 2024-09-17T04:32:00.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.407+0000 7fadab400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f757f5a3b0 0x55f758353eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:00.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.407+0000 7fadab400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f757f5a3b0 0x55f758353eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48850/0 (socket says 172.21.15.100:48850) 2024-09-17T04:32:00.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.407+0000 7fadab400640 1 -- 172.21.15.100:0/1095161603 learned_addr learned my addr 172.21.15.100:0/1095161603 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:00.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.407+0000 7fadabe00640 1 --2- 172.21.15.100:0/1095161603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f757f83270 0x55f758353970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:00.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.407+0000 7fadb0e00640 1 --2- 172.21.15.100:0/1095161603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f757f62590 0x55f758323f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:00.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.407+0000 7fadab400640 1 -- 172.21.15.100:0/1095161603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f757f62590 msgr2=0x55f758323f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:32:00.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.407+0000 7fadab400640 1 --2- 172.21.15.100:0/1095161603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f757f62590 0x55f758323f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.407+0000 7fadab400640 1 -- 172.21.15.100:0/1095161603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f757f83270 msgr2=0x55f758353970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:32:00.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.407+0000 7fadab400640 1 --2- 172.21.15.100:0/1095161603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f757f83270 0x55f758353970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.407+0000 7fadab400640 1 -- 172.21.15.100:0/1095161603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f758255520 con 0x55f757f5a3b0 2024-09-17T04:32:00.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.407+0000 7fadb0e00640 1 --2- 172.21.15.100:0/1095161603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f757f62590 0x55f758323f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:00.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.407+0000 7fadab400640 1 --2- 172.21.15.100:0/1095161603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f757f5a3b0 0x55f758353eb0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fad9800e5e0 tx=0x7fad98011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b578dd23ce1c1fbe server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:00.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.408+0000 7fada0c00640 1 -- 172.21.15.100:0/1095161603 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98025070 con 0x55f757f5a3b0 2024-09-17T04:32:00.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.408+0000 7fada0c00640 1 -- 172.21.15.100:0/1095161603 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad980157f0 con 0x55f757f5a3b0 2024-09-17T04:32:00.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.408+0000 7fada0c00640 1 -- 172.21.15.100:0/1095161603 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98015bc0 con 0x55f757f5a3b0 2024-09-17T04:32:00.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.408+0000 7fadb9e02740 1 -- 172.21.15.100:0/1095161603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f757f5a3b0 msgr2=0x55f758353eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:00.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.408+0000 7fadb9e02740 1 --2- 172.21.15.100:0/1095161603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f757f5a3b0 0x55f758353eb0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fad9800e5e0 tx=0x7fad98011000 comp rx=0 tx=0).stop 2024-09-17T04:32:00.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.409+0000 7fadb9e02740 1 -- 172.21.15.100:0/1095161603 shutdown_connections 2024-09-17T04:32:00.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.409+0000 7fadb9e02740 1 --2- 172.21.15.100:0/1095161603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f757f62590 0x55f758323f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.409+0000 7fadb9e02740 1 --2- 172.21.15.100:0/1095161603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f757f5a3b0 0x55f758353eb0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.409+0000 7fadb9e02740 1 --2- 172.21.15.100:0/1095161603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f757f83270 0x55f758353970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.409+0000 7fadb9e02740 1 -- 172.21.15.100:0/1095161603 >> 172.21.15.100:0/1095161603 conn(0x55f757f773c0 msgr2=0x55f758329970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:00.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.409+0000 7fadb9e02740 1 -- 172.21.15.100:0/1095161603 shutdown_connections 2024-09-17T04:32:00.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.409+0000 7fadb9e02740 1 -- 172.21.15.100:0/1095161603 wait complete. 2024-09-17T04:32:00.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.410+0000 7fadb9e02740 1 -- start start 2024-09-17T04:32:00.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.410+0000 7fadb9e02740 1 Processor -- start 2024-09-17T04:32:00.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.411+0000 7fadb9e02740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f757f5a3b0 0x55f75834d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:00.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.411+0000 7fadb9e02740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f757f62590 0x55f7583821e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:00.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.411+0000 7fadb9e02740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f757f83270 0x55f758349470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:00.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.411+0000 7fadb9e02740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f758233330 con 0x55f757f62590 2024-09-17T04:32:00.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.411+0000 7fadb9e02740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f758290810 con 0x55f757f5a3b0 2024-09-17T04:32:00.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.411+0000 7fadb9e02740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55f757ffd620 con 0x55f757f83270 2024-09-17T04:32:00.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.411+0000 7fadb9e02740 10 client.0 _ll_register_callbacks cb 0x55f758238420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:32:00.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.412+0000 7fadab400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f757f62590 0x55f7583821e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:00.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.412+0000 7fadab400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f757f62590 0x55f7583821e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48862/0 (socket says 172.21.15.100:48862) 2024-09-17T04:32:00.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.412+0000 7fadab400640 1 -- 172.21.15.100:0/2817433177 learned_addr learned my addr 172.21.15.100:0/2817433177 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:00.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.412+0000 7fadab400640 1 -- 172.21.15.100:0/2817433177 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f757f83270 msgr2=0x55f758349470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:32:00.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.412+0000 7fadab400640 1 --2- 172.21.15.100:0/2817433177 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f757f83270 0x55f758349470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.412+0000 7fadab400640 1 -- 172.21.15.100:0/2817433177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f757f5a3b0 msgr2=0x55f75834d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:32:00.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.412+0000 7fadab400640 1 --2- 172.21.15.100:0/2817433177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f757f5a3b0 0x55f75834d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.412+0000 7fadab400640 1 -- 172.21.15.100:0/2817433177 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f757fee850 con 0x55f757f62590 2024-09-17T04:32:00.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.412+0000 7fadab400640 1 --2- 172.21.15.100:0/2817433177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f757f62590 0x55f7583821e0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fad98007c20 tx=0x7fad98016d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96cff73c03cacdd server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:00.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.412+0000 7fad95600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-17T04:32:00.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.412+0000 7fad95600640 1 -- 172.21.15.100:0/2817433177 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98025040 con 0x55f757f62590 2024-09-17T04:32:00.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.412+0000 7fad95600640 1 -- 172.21.15.100:0/2817433177 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7fad98029620 con 0x55f757f62590 2024-09-17T04:32:00.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.412+0000 7fadb9e02740 1 -- 172.21.15.100:0/2817433177 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad980157f0 con 0x55f757f62590 2024-09-17T04:32:00.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.413+0000 7fad95600640 1 -- 172.21.15.100:0/2817433177 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fad98035050 con 0x55f757f62590 2024-09-17T04:32:00.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.413+0000 7fadb9e02740 10 client.8088 fetch_fsmap learned FSMap version 91 2024-09-17T04:32:00.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.413+0000 7fadb9e02740 1 -- 172.21.15.100:0/2817433177 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55f7580656b0 con 0x55f757f62590 2024-09-17T04:32:00.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.413+0000 7fad95600640 1 -- 172.21.15.100:0/2817433177 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad980157f0 con 0x55f757f62590 2024-09-17T04:32:00.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.413+0000 7fad95600640 1 -- 172.21.15.100:0/2817433177 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 91) ==== 3502+0+0 (secure 0 0 0) 0x7fad98038020 con 0x55f757f62590 2024-09-17T04:32:00.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.413+0000 7fadb9e02740 10 client.8088 fetch_fsmap finished waiting for FSMap version 91 2024-09-17T04:32:00.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.413+0000 7fadb9e02740 10 client.8088 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T04:32:00.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.413+0000 7fadb9e02740 20 client.8088 populate_metadata read hostname 'smithi100' 2024-09-17T04:32:00.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.413+0000 7fadb9e02740 1 --2- 172.21.15.100:0/2817433177 >> [v2:172.21.15.100:6833/3047460644,v1:172.21.15.100:6835/3047460644] conn(0x55f757f41e40 0x55f75837e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:00.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.413+0000 7fadb9e02740 4 client.8088 mds_command: new command op to 7965 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T04:32:00.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.413+0000 7fadb9e02740 1 -- 172.21.15.100:0/2817433177 --> [v2:172.21.15.100:6833/3047460644,v1:172.21.15.100:6835/3047460644] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f75829f520 con 0x55f757f41e40 2024-09-17T04:32:00.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.413+0000 7fadabe00640 1 --2- 172.21.15.100:0/2817433177 >> [v2:172.21.15.100:6833/3047460644,v1:172.21.15.100:6835/3047460644] conn(0x55f757f41e40 0x55f75837e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:00.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.414+0000 7fad95600640 10 client.8088 ms_handle_connect on v2:172.21.15.100:6833/3047460644 2024-09-17T04:32:00.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.414+0000 7fadabe00640 1 --2- 172.21.15.100:0/2817433177 >> [v2:172.21.15.100:6833/3047460644,v1:172.21.15.100:6835/3047460644] conn(0x55f757f41e40 0x55f75837e2d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3129b6560e5d6c96 server_cookie=cf3682234a5d2dcd in_seq=0 out_seq=0 2024-09-17T04:32:00.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.415+0000 7fad95600640 1 -- 172.21.15.100:0/2817433177 <== mds.0 v2:172.21.15.100:6833/3047460644 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f75829f520 con 0x55f757f41e40 2024-09-17T04:32:00.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.415+0000 7fad95600640 10 client.8088 handle_command_reply: tid=0 2024-09-17T04:32:00.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.415+0000 7fadb9e02740 1 client.8088 shutdown 2024-09-17T04:32:00.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.415+0000 7fadb9e02740 1 -- 172.21.15.100:0/2817433177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f757f62590 msgr2=0x55f7583821e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:00.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.415+0000 7fadb9e02740 1 --2- 172.21.15.100:0/2817433177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f757f62590 0x55f7583821e0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fad98007c20 tx=0x7fad98016d30 comp rx=0 tx=0).stop 2024-09-17T04:32:00.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.415+0000 7fadb9e02740 1 -- 172.21.15.100:0/2817433177 shutdown_connections 2024-09-17T04:32:00.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.415+0000 7fadb9e02740 1 --2- 172.21.15.100:0/2817433177 >> [v2:172.21.15.100:6833/3047460644,v1:172.21.15.100:6835/3047460644] conn(0x55f757f41e40 0x55f75837e2d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.415+0000 7fadb9e02740 1 --2- 172.21.15.100:0/2817433177 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f757f83270 0x55f758349470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.415+0000 7fadb9e02740 1 --2- 172.21.15.100:0/2817433177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f757f62590 0x55f7583821e0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.415+0000 7fadb9e02740 1 --2- 172.21.15.100:0/2817433177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f757f5a3b0 0x55f75834d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.415+0000 7fad95600640 0 client.8088 ms_handle_reset on v2:172.21.15.100:6833/3047460644 2024-09-17T04:32:00.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.415+0000 7fadb9e02740 1 -- 172.21.15.100:0/2817433177 >> 172.21.15.100:0/2817433177 conn(0x55f757f773c0 msgr2=0x55f75836f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:00.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.415+0000 7fadb9e02740 1 -- 172.21.15.100:0/2817433177 shutdown_connections 2024-09-17T04:32:00.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.415+0000 7fadb9e02740 1 -- 172.21.15.100:0/2817433177 wait complete. 2024-09-17T04:32:00.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.415+0000 7fadb9e02740 20 client.8088 trim_cache size 0 max 16384 2024-09-17T04:32:00.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.434+0000 7fadb9e02740 1 Processor -- start 2024-09-17T04:32:00.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.434+0000 7fadb9e02740 1 -- start start 2024-09-17T04:32:00.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.435+0000 7fadb9e02740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f757f5a3b0 0x55f758367f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:00.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.435+0000 7fadb9e02740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f757f62590 0x55f7583526d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:00.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.435+0000 7fadb9e02740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f757f83270 0x55f758352c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:00.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.435+0000 7fadb9e02740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fadac201180 con 0x55f757f83270 2024-09-17T04:32:00.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.435+0000 7fadb9e02740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fadac1f9ad0 con 0x55f757f62590 2024-09-17T04:32:00.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.435+0000 7fadb9e02740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fadac1e8170 con 0x55f757f5a3b0 2024-09-17T04:32:00.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.436+0000 7fadb0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f757f83270 0x55f758352c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:00.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.436+0000 7fadb0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f757f83270 0x55f758352c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48878/0 (socket says 172.21.15.100:48878) 2024-09-17T04:32:00.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.436+0000 7fadb0e00640 1 -- 172.21.15.100:0/3060633433 learned_addr learned my addr 172.21.15.100:0/3060633433 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:00.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.436+0000 7fadabe00640 1 --2- 172.21.15.100:0/3060633433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f757f5a3b0 0x55f758367f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:00.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.436+0000 7fadab400640 1 --2- 172.21.15.100:0/3060633433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f757f62590 0x55f7583526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:00.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.436+0000 7fadb0e00640 1 -- 172.21.15.100:0/3060633433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f757f5a3b0 msgr2=0x55f758367f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:32:00.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.436+0000 7fadb0e00640 1 --2- 172.21.15.100:0/3060633433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f757f5a3b0 0x55f758367f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.436+0000 7fadb0e00640 1 -- 172.21.15.100:0/3060633433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f757f62590 msgr2=0x55f7583526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:32:00.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.436+0000 7fadb0e00640 1 --2- 172.21.15.100:0/3060633433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f757f62590 0x55f7583526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.436+0000 7fadb0e00640 1 -- 172.21.15.100:0/3060633433 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f75834abd0 con 0x55f757f83270 2024-09-17T04:32:00.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.436+0000 7fadb0e00640 1 --2- 172.21.15.100:0/3060633433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f757f83270 0x55f758352c10 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fada4009cd0 tx=0x7fada4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=648dffadbae3600a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:00.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.437+0000 7fad97e00640 1 -- 172.21.15.100:0/3060633433 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4003dd0 con 0x55f757f83270 2024-09-17T04:32:00.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.437+0000 7fad97e00640 1 -- 172.21.15.100:0/3060633433 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fada4004130 con 0x55f757f83270 2024-09-17T04:32:00.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.437+0000 7fad97e00640 1 -- 172.21.15.100:0/3060633433 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4004560 con 0x55f757f83270 2024-09-17T04:32:00.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.437+0000 7fadb9e02740 1 -- 172.21.15.100:0/3060633433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f757f83270 msgr2=0x55f758352c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:00.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.437+0000 7fadb9e02740 1 --2- 172.21.15.100:0/3060633433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f757f83270 0x55f758352c10 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fada4009cd0 tx=0x7fada4003970 comp rx=0 tx=0).stop 2024-09-17T04:32:00.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.437+0000 7fadb9e02740 1 -- 172.21.15.100:0/3060633433 shutdown_connections 2024-09-17T04:32:00.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.437+0000 7fadb9e02740 1 --2- 172.21.15.100:0/3060633433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f757f83270 0x55f758352c10 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.437+0000 7fadb9e02740 1 --2- 172.21.15.100:0/3060633433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f757f62590 0x55f7583526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.437+0000 7fadb9e02740 1 --2- 172.21.15.100:0/3060633433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f757f5a3b0 0x55f758367f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.437+0000 7fadb9e02740 1 -- 172.21.15.100:0/3060633433 >> 172.21.15.100:0/3060633433 conn(0x55f757f773c0 msgr2=0x55f758381fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:00.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.438+0000 7fadb9e02740 1 -- 172.21.15.100:0/3060633433 shutdown_connections 2024-09-17T04:32:00.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.438+0000 7fadb9e02740 1 -- 172.21.15.100:0/3060633433 wait complete. 2024-09-17T04:32:00.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.438+0000 7fadb9e02740 1 -- start start 2024-09-17T04:32:00.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.439+0000 7fadb9e02740 1 Processor -- start 2024-09-17T04:32:00.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.440+0000 7fadb9e02740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f757f5a3b0 0x55f75834fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:00.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.440+0000 7fadb9e02740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f757f62590 0x55f758353b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:00.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.440+0000 7fadb9e02740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f757f83270 0x55f7583ef2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:00.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.440+0000 7fadb9e02740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f758366cc0 con 0x55f757f83270 2024-09-17T04:32:00.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.440+0000 7fadb9e02740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f758367340 con 0x55f757f62590 2024-09-17T04:32:00.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.440+0000 7fadb9e02740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55f758324c70 con 0x55f757f5a3b0 2024-09-17T04:32:00.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.440+0000 7fadb9e02740 10 client.0 _ll_register_callbacks cb 0x7fad9800e4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:32:00.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.440+0000 7fadb0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f757f83270 0x55f7583ef2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:00.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.440+0000 7fadb0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f757f83270 0x55f7583ef2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48884/0 (socket says 172.21.15.100:48884) 2024-09-17T04:32:00.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.440+0000 7fadb0e00640 1 -- 172.21.15.100:0/468179500 learned_addr learned my addr 172.21.15.100:0/468179500 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:00.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.440+0000 7fadab400640 1 --2- 172.21.15.100:0/468179500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f757f62590 0x55f758353b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:00.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.440+0000 7fadabe00640 1 --2- 172.21.15.100:0/468179500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f757f5a3b0 0x55f75834fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:00.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.441+0000 7fadb0e00640 1 -- 172.21.15.100:0/468179500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f757f5a3b0 msgr2=0x55f75834fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:32:00.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.441+0000 7fadb0e00640 1 --2- 172.21.15.100:0/468179500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f757f5a3b0 0x55f75834fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.441+0000 7fadb0e00640 1 -- 172.21.15.100:0/468179500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f757f62590 msgr2=0x55f758353b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:32:00.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.441+0000 7fadb0e00640 1 --2- 172.21.15.100:0/468179500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f757f62590 0x55f758353b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.441+0000 7fadb0e00640 1 -- 172.21.15.100:0/468179500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f75831f350 con 0x55f757f83270 2024-09-17T04:32:00.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.441+0000 7fadabe00640 1 --2- 172.21.15.100:0/468179500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f757f5a3b0 0x55f75834fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:00.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.441+0000 7fadb0e00640 1 --2- 172.21.15.100:0/468179500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f757f83270 0x55f7583ef2e0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fada4009d00 tx=0x7fada4003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588df6393e09fa8e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:00.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.441+0000 7fad95600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-17T04:32:00.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.441+0000 7fad95600640 1 -- 172.21.15.100:0/468179500 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada40044e0 con 0x55f757f83270 2024-09-17T04:32:00.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.441+0000 7fad95600640 1 -- 172.21.15.100:0/468179500 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7fada4004680 con 0x55f757f83270 2024-09-17T04:32:00.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.441+0000 7fadb9e02740 1 -- 172.21.15.100:0/468179500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fada4004130 con 0x55f757f83270 2024-09-17T04:32:00.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.442+0000 7fad95600640 1 -- 172.21.15.100:0/468179500 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fada4058050 con 0x55f757f83270 2024-09-17T04:32:00.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.442+0000 7fadb9e02740 10 client.8094 fetch_fsmap learned FSMap version 91 2024-09-17T04:32:00.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.442+0000 7fadb9e02740 1 -- 172.21.15.100:0/468179500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55f75832a650 con 0x55f757f83270 2024-09-17T04:32:00.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.442+0000 7fad95600640 1 -- 172.21.15.100:0/468179500 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fada4004130 con 0x55f757f83270 2024-09-17T04:32:00.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.442+0000 7fad95600640 1 -- 172.21.15.100:0/468179500 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 91) ==== 3502+0+0 (secure 0 0 0) 0x7fada405b020 con 0x55f757f83270 2024-09-17T04:32:00.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.442+0000 7fadb9e02740 10 client.8094 fetch_fsmap finished waiting for FSMap version 91 2024-09-17T04:32:00.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.442+0000 7fadb9e02740 10 client.8094 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T04:32:00.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.442+0000 7fadb9e02740 20 client.8094 populate_metadata read hostname 'smithi100' 2024-09-17T04:32:00.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.442+0000 7fadb9e02740 1 --2- 172.21.15.100:0/468179500 >> [v2:172.21.15.100:6833/3047460644,v1:172.21.15.100:6835/3047460644] conn(0x55f758351920 0x55f758351d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:00.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.442+0000 7fadb9e02740 4 client.8094 mds_command: new command op to 7965 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T04:32:00.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.442+0000 7fadb9e02740 1 -- 172.21.15.100:0/468179500 --> [v2:172.21.15.100:6833/3047460644,v1:172.21.15.100:6835/3047460644] -- command(tid 0: {"prefix": "status"}) -- 0x55f75834f430 con 0x55f758351920 2024-09-17T04:32:00.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.442+0000 7fadabe00640 1 --2- 172.21.15.100:0/468179500 >> [v2:172.21.15.100:6833/3047460644,v1:172.21.15.100:6835/3047460644] conn(0x55f758351920 0x55f758351d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:00.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.443+0000 7fad95600640 10 client.8094 ms_handle_connect on v2:172.21.15.100:6833/3047460644 2024-09-17T04:32:00.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.443+0000 7fadabe00640 1 --2- 172.21.15.100:0/468179500 >> [v2:172.21.15.100:6833/3047460644,v1:172.21.15.100:6835/3047460644] conn(0x55f758351920 0x55f758351d30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9034d8823a137b0a server_cookie=9c2d43adceba060a in_seq=0 out_seq=0 2024-09-17T04:32:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.443+0000 7fad95600640 1 -- 172.21.15.100:0/468179500 <== mds.0 v2:172.21.15.100:6833/3047460644 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f75834f430 con 0x55f758351920 2024-09-17T04:32:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.443+0000 7fad95600640 10 client.8094 handle_command_reply: tid=0 2024-09-17T04:32:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.443+0000 7fadb9e02740 1 client.8094 shutdown 2024-09-17T04:32:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.443+0000 7fadb9e02740 1 -- 172.21.15.100:0/468179500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f757f83270 msgr2=0x55f7583ef2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.443+0000 7fadb9e02740 1 --2- 172.21.15.100:0/468179500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f757f83270 0x55f7583ef2e0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fada4009d00 tx=0x7fada4003e00 comp rx=0 tx=0).stop 2024-09-17T04:32:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.443+0000 7fadb9e02740 1 -- 172.21.15.100:0/468179500 shutdown_connections 2024-09-17T04:32:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.443+0000 7fadb9e02740 1 --2- 172.21.15.100:0/468179500 >> [v2:172.21.15.100:6833/3047460644,v1:172.21.15.100:6835/3047460644] conn(0x55f758351920 0x55f758351d30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.443+0000 7fadb9e02740 1 --2- 172.21.15.100:0/468179500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f757f83270 0x55f7583ef2e0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.443+0000 7fadb9e02740 1 --2- 172.21.15.100:0/468179500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f757f62590 0x55f758353b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.443+0000 7fad95600640 0 client.8094 ms_handle_reset on v2:172.21.15.100:6833/3047460644 2024-09-17T04:32:00.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.443+0000 7fadb9e02740 1 --2- 172.21.15.100:0/468179500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f757f5a3b0 0x55f75834fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.443+0000 7fadb9e02740 1 -- 172.21.15.100:0/468179500 >> 172.21.15.100:0/468179500 conn(0x55f757f773c0 msgr2=0x55f7583683c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:00.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.443+0000 7fadb9e02740 1 -- 172.21.15.100:0/468179500 shutdown_connections 2024-09-17T04:32:00.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.443+0000 7fadb9e02740 1 -- 172.21.15.100:0/468179500 wait complete. 2024-09-17T04:32:00.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.444+0000 7fadb9e02740 20 client.8094 trim_cache size 0 max 16384 2024-09-17T04:32:00.449 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:32:00.449 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "8a399842-3a5c-4ea2-9afb-b2704332e59f", 2024-09-17T04:32:00.449 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 0, 2024-09-17T04:32:00.450 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 7965, 2024-09-17T04:32:00.450 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-09-17T04:32:00.450 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-09-17T04:32:00.450 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-09-17T04:32:00.450 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 6.6279205589999997, 2024-09-17T04:32:00.450 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 91, 2024-09-17T04:32:00.450 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 132, 2024-09-17T04:32:00.450 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 132, 2024-09-17T04:32:00.450 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 11.743349034 2024-09-17T04:32:00.450 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:32:00.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.444+0000 7fadb2e00640 1 -- 172.21.15.100:0/3978124622 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fad8c0791c0 msgr2=0x7fad8c07b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:00.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.444+0000 7fadb2e00640 1 --2- 172.21.15.100:0/3978124622 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fad8c0791c0 0x7fad8c07b6d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fadac0b01d0 tx=0x7fad98009040 comp rx=0 tx=0).stop 2024-09-17T04:32:00.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.444+0000 7fadb2e00640 1 -- 172.21.15.100:0/3978124622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadac0cfc80 msgr2=0x7fadac1e9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:00.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.444+0000 7fadb2e00640 1 --2- 172.21.15.100:0/3978124622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadac0cfc80 0x7fadac1e9d00 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fad9c001eb0 tx=0x7fad9c00c890 comp rx=0 tx=0).stop 2024-09-17T04:32:00.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.444+0000 7fadb2e00640 1 -- 172.21.15.100:0/3978124622 shutdown_connections 2024-09-17T04:32:00.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.444+0000 7fadb2e00640 1 --2- 172.21.15.100:0/3978124622 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fad8c0791c0 0x7fad8c07b6d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.444+0000 7fadb2e00640 1 --2- 172.21.15.100:0/3978124622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadac1f61a0 0x7fadac156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.444+0000 7fadb2e00640 1 --2- 172.21.15.100:0/3978124622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadac15d540 0x7fadac0b0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.444+0000 7fadb2e00640 1 --2- 172.21.15.100:0/3978124622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadac0cfc80 0x7fadac1e9d00 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.444+0000 7fadb2e00640 1 -- 172.21.15.100:0/3978124622 >> 172.21.15.100:0/3978124622 conn(0x7fadac06d3d0 msgr2=0x7fadac15ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:00.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.444+0000 7fadb2e00640 1 -- 172.21.15.100:0/3978124622 shutdown_connections 2024-09-17T04:32:00.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.444+0000 7fadb2e00640 1 -- 172.21.15.100:0/3978124622 wait complete. 2024-09-17T04:32:00.470 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:32:00.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.690+0000 7fc2f5c00640 1 -- 172.21.15.100:0/3149724815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2f00ad9c0 msgr2=0x7fc2f00af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:00.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.690+0000 7fc2f5c00640 1 --2- 172.21.15.100:0/3149724815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2f00ad9c0 0x7fc2f00af170 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0005fd0 tx=0x7fc2e002aa10 comp rx=0 tx=0).stop 2024-09-17T04:32:00.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.690+0000 7fc2f5c00640 1 -- 172.21.15.100:0/3149724815 shutdown_connections 2024-09-17T04:32:00.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.690+0000 7fc2f5c00640 1 --2- 172.21.15.100:0/3149724815 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2f015a0f0 0x7fc2f015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.690+0000 7fc2f5c00640 1 --2- 172.21.15.100:0/3149724815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2f00ad9c0 0x7fc2f00af170 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.690+0000 7fc2f5c00640 1 -- 172.21.15.100:0/3149724815 >> 172.21.15.100:0/3149724815 conn(0x7fc2f00b1e40 msgr2=0x7fc2f00b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:00.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.690+0000 7fc2f5c00640 1 -- 172.21.15.100:0/3149724815 shutdown_connections 2024-09-17T04:32:00.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.690+0000 7fc2f5c00640 1 -- 172.21.15.100:0/3149724815 wait complete. 2024-09-17T04:32:00.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.692+0000 7fc2f5c00640 1 Processor -- start 2024-09-17T04:32:00.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.692+0000 7fc2f5c00640 1 -- start start 2024-09-17T04:32:00.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.694+0000 7fc2f5c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2f00ad9c0 0x7fc2f01589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:00.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.694+0000 7fc2f5c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2f00b2f60 0x7fc2f00ca800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:00.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.694+0000 7fc2f5c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2f015a0f0 0x7fc2f00cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:00.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.694+0000 7fc2f5c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f006f410 con 0x7fc2f015a0f0 2024-09-17T04:32:00.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.694+0000 7fc2f5c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f006f290 con 0x7fc2f00ad9c0 2024-09-17T04:32:00.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.694+0000 7fc2f5c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc2f006f590 con 0x7fc2f00b2f60 2024-09-17T04:32:00.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.694+0000 7fc2f5200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2f015a0f0 0x7fc2f00cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:00.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.694+0000 7fc2f5200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2f015a0f0 0x7fc2f00cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48906/0 (socket says 172.21.15.100:48906) 2024-09-17T04:32:00.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.694+0000 7fc2f5200640 1 -- 172.21.15.100:0/2951778930 learned_addr learned my addr 172.21.15.100:0/2951778930 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:00.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.694+0000 7fc2ef400640 1 --2- 172.21.15.100:0/2951778930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2f00b2f60 0x7fc2f00ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:00.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.694+0000 7fc2efe00640 1 --2- 172.21.15.100:0/2951778930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2f00ad9c0 0x7fc2f01589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:00.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.695+0000 7fc2f5200640 1 -- 172.21.15.100:0/2951778930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2f00b2f60 msgr2=0x7fc2f00ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:00.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.695+0000 7fc2f5200640 1 --2- 172.21.15.100:0/2951778930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2f00b2f60 0x7fc2f00ca800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.695+0000 7fc2f5200640 1 -- 172.21.15.100:0/2951778930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2f00ad9c0 msgr2=0x7fc2f01589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:00.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.695+0000 7fc2f5200640 1 --2- 172.21.15.100:0/2951778930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2f00ad9c0 0x7fc2f01589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.695+0000 7fc2f5200640 1 -- 172.21.15.100:0/2951778930 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f00c9530 con 0x7fc2f015a0f0 2024-09-17T04:32:00.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.695+0000 7fc2f5200640 1 --2- 172.21.15.100:0/2951778930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2f015a0f0 0x7fc2f00cad40 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800e700 tx=0x7fc2e800ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:00.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.695+0000 7fc2e7e00640 1 -- 172.21.15.100:0/2951778930 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e800cab0 con 0x7fc2f015a0f0 2024-09-17T04:32:00.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.695+0000 7fc2e7e00640 1 -- 172.21.15.100:0/2951778930 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2e800cc50 con 0x7fc2f015a0f0 2024-09-17T04:32:00.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.695+0000 7fc2f5c00640 1 -- 172.21.15.100:0/2951778930 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f00ca5b0 con 0x7fc2f015a0f0 2024-09-17T04:32:00.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.695+0000 7fc2e7e00640 1 -- 172.21.15.100:0/2951778930 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8005020 con 0x7fc2f015a0f0 2024-09-17T04:32:00.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.695+0000 7fc2f5c00640 1 -- 172.21.15.100:0/2951778930 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f00c9af0 con 0x7fc2f015a0f0 2024-09-17T04:32:00.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.696+0000 7fc2f5c00640 1 -- 172.21.15.100:0/2951778930 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2f006fab0 con 0x7fc2f015a0f0 2024-09-17T04:32:00.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.698+0000 7fc2e7e00640 1 -- 172.21.15.100:0/2951778930 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fc2e80051c0 con 0x7fc2f015a0f0 2024-09-17T04:32:00.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.698+0000 7fc2e7e00640 1 --2- 172.21.15.100:0/2951778930 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc2d00791c0 0x7fc2d007b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:00.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.699+0000 7fc2e7e00640 1 -- 172.21.15.100:0/2951778930 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7fc2e809c1f0 con 0x7fc2f015a0f0 2024-09-17T04:32:00.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.699+0000 7fc2efe00640 1 --2- 172.21.15.100:0/2951778930 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc2d00791c0 0x7fc2d007b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:00.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.699+0000 7fc2efe00640 1 --2- 172.21.15.100:0/2951778930 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc2d00791c0 0x7fc2d007b6d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0035000 tx=0x7fc2e003b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:00.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.703+0000 7fc2e7e00640 1 -- 172.21.15.100:0/2951778930 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc2e80635a0 con 0x7fc2f015a0f0 2024-09-17T04:32:00.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.925+0000 7fc2f5c00640 1 -- 172.21.15.100:0/2951778930 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc2f0070780 con 0x7fc2f015a0f0 2024-09-17T04:32:00.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.926+0000 7fc2e7e00640 1 -- 172.21.15.100:0/2951778930 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v91) ==== 53+0+73 (secure 0 0 0) 0x7fc2e8067fd0 con 0x7fc2f015a0f0 2024-09-17T04:32:00.927 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:32:00.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.928+0000 7fc2f5c00640 1 -- 172.21.15.100:0/2951778930 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc2d00791c0 msgr2=0x7fc2d007b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:00.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.928+0000 7fc2f5c00640 1 --2- 172.21.15.100:0/2951778930 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc2d00791c0 0x7fc2d007b6d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0035000 tx=0x7fc2e003b040 comp rx=0 tx=0).stop 2024-09-17T04:32:00.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.928+0000 7fc2f5c00640 1 -- 172.21.15.100:0/2951778930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2f015a0f0 msgr2=0x7fc2f00cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:00.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.928+0000 7fc2f5c00640 1 --2- 172.21.15.100:0/2951778930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2f015a0f0 0x7fc2f00cad40 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800e700 tx=0x7fc2e800ebc0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.929+0000 7fc2f5c00640 1 -- 172.21.15.100:0/2951778930 shutdown_connections 2024-09-17T04:32:00.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.929+0000 7fc2f5c00640 1 --2- 172.21.15.100:0/2951778930 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc2d00791c0 0x7fc2d007b6d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.929+0000 7fc2f5c00640 1 --2- 172.21.15.100:0/2951778930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2f015a0f0 0x7fc2f00cad40 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.929+0000 7fc2f5c00640 1 --2- 172.21.15.100:0/2951778930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2f00b2f60 0x7fc2f00ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.929+0000 7fc2f5c00640 1 --2- 172.21.15.100:0/2951778930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2f00ad9c0 0x7fc2f01589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:00.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.929+0000 7fc2f5c00640 1 -- 172.21.15.100:0/2951778930 >> 172.21.15.100:0/2951778930 conn(0x7fc2f00b1e40 msgr2=0x7fc2f015c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:00.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.929+0000 7fc2f5c00640 1 -- 172.21.15.100:0/2951778930 shutdown_connections 2024-09-17T04:32:00.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:00.929+0000 7fc2f5c00640 1 -- 172.21.15.100:0/2951778930 wait complete. 2024-09-17T04:32:00.951 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T04:32:00.951 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:32:00.951 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-09-17T04:32:00.951 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T04:32:00.951 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T04:32:00.951 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T04:32:00.952 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T04:32:00.952 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:32:00.952 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:32:00.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:00.952 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-09-17T04:32:00.988 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:32:00.988 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:32:00.988 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:32:00.988 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:32:00.988 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-17T04:32:00.988 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:32:00.988 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:32:00.988 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:32:00.988 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-09-17T04:32:00.988 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:32:00.988 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:32:00.989 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-09-17T04:32:00.989 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:32:00.989 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:32:00.989 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:32:00.989 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:32:00.989 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T04:32:00.989 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42239sec preferred_lft 42239sec 2024-09-17T04:32:00.989 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link noprefixroute 2024-09-17T04:32:00.989 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:32:00.989 INFO:teuthology.orchestra.run.smithi100.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:32:00.989 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:2e:de:8e:a4:7c brd ff:ff:ff:ff:ff:ff 2024-09-17T04:32:00.989 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T04:32:00.990 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:32:00.990 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::cc78:e4ff:fe70:7fbe/64 scope link 2024-09-17T04:32:00.990 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:32:00.990 INFO:teuthology.orchestra.run.smithi100.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:32:00.990 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether ea:8c:45:9f:34:87 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:32:00.990 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::e88c:45ff:fe9f:3487/64 scope link 2024-09-17T04:32:00.990 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:32:00.990 INFO:teuthology.orchestra.run.smithi100.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:32:00.990 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:2e:de:8e:a4:7c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:32:00.990 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::c2e:deff:fe8e:a47c/64 scope link 2024-09-17T04:32:00.990 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:32:00.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:00.991 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:32:01.081 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-09-17T04:32:01.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:01.084 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-09-17T04:32:01.120 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-09-17T04:32:01.120 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:32:01.120 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T04:32:01.120 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T04:32:01.121 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:32:01.121 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:32:01.121 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-17T04:32:01.121 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T04:32:01.121 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T04:32:01.121 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T04:32:01.121 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-17T04:32:01.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:32:01.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:32:01.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:01.316 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-17T04:32:01.352 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T04:32:01.352 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T04:32:01.352 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:32:01.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:01.352 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-09-17T04:32:01.448 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:32:01.448 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T04:32:01.483 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:32:01.483 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:32:01.483 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:32:01.558 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T04:32:01.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:01.558 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:32:01.752 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:32:01.752 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:32:01.753 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:32:01.773 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:32:01.773 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:32:01.773 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:32:01.773 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:32:01.773 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:32:01.802 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:32:01.802 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:32:01.802 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:32:01.802 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:32:01.802 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:32:01.802 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-09-17T04:32:01.803 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-09-17T04:32:01.803 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8a399842-3a5c-4ea2-9afb-b2704332e59f" will pass to kernel 2024-09-17T04:32:01.804 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-17T04:32:01.804 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-09-17T04:32:01.804 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-17T04:32:01.804 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-09-17T04:32:01.804 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-17T04:32:01.807 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T04:32:01.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:01.808 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:32:01.868 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:32:01.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:32:01.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:32:01.964 INFO:teuthology.orchestra.run.smithi100.stdout:client7627 2024-09-17T04:32:01.964 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:32:01.964 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:32:01.999 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:32:02.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.259+0000 7ff37aa00640 1 -- 172.21.15.100:0/2669114015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff358003c10 msgr2=0x7ff358004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:02.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.259+0000 7ff37aa00640 1 --2- 172.21.15.100:0/2669114015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff358003c10 0x7ff358004060 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7ff36400ac60 tx=0x7ff358005830 comp rx=0 tx=0).stop 2024-09-17T04:32:02.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.259+0000 7ff378a00640 1 -- 172.21.15.100:0/2669114015 reap_dead start 2024-09-17T04:32:02.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.259+0000 7ff37aa00640 1 -- 172.21.15.100:0/2669114015 shutdown_connections 2024-09-17T04:32:02.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.259+0000 7ff37aa00640 1 -- 172.21.15.100:0/2669114015 >> 172.21.15.100:0/2669114015 conn(0x7ff37406d730 msgr2=0x7ff3740b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:02.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.260+0000 7ff37aa00640 1 -- 172.21.15.100:0/2669114015 shutdown_connections 2024-09-17T04:32:02.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.260+0000 7ff37aa00640 1 -- 172.21.15.100:0/2669114015 wait complete. 2024-09-17T04:32:02.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.262+0000 7ff37aa00640 1 Processor -- start 2024-09-17T04:32:02.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.262+0000 7ff37aa00640 1 -- start start 2024-09-17T04:32:02.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.263+0000 7ff37aa00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff358003c10 0x7ff37409bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:02.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.263+0000 7ff37aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3741f91f0 0x7ff37409dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:02.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.263+0000 7ff37aa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff37414df80 0x7ff3741f6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:02.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.263+0000 7ff37aa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff3740712b0 con 0x7ff3741f91f0 2024-09-17T04:32:02.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.263+0000 7ff37aa00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff374071130 con 0x7ff37414df80 2024-09-17T04:32:02.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.263+0000 7ff37aa00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff374073040 con 0x7ff358003c10 2024-09-17T04:32:02.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.264+0000 7ff373400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3741f91f0 0x7ff37409dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:02.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.264+0000 7ff373400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3741f91f0 0x7ff37409dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48918/0 (socket says 172.21.15.100:48918) 2024-09-17T04:32:02.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.264+0000 7ff373400640 1 -- 172.21.15.100:0/1649574760 learned_addr learned my addr 172.21.15.100:0/1649574760 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:02.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.264+0000 7ff373e00640 1 --2- 172.21.15.100:0/1649574760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff358003c10 0x7ff37409bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:02.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.264+0000 7ff378a00640 1 --2- 172.21.15.100:0/1649574760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff37414df80 0x7ff3741f6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:02.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.264+0000 7ff373400640 1 -- 172.21.15.100:0/1649574760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff358003c10 msgr2=0x7ff37409bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:02.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.264+0000 7ff373400640 1 --2- 172.21.15.100:0/1649574760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff358003c10 0x7ff37409bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:02.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.264+0000 7ff373400640 1 -- 172.21.15.100:0/1649574760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff37414df80 msgr2=0x7ff3741f6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:02.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.264+0000 7ff373400640 1 --2- 172.21.15.100:0/1649574760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff37414df80 0x7ff3741f6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:02.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.264+0000 7ff373400640 1 -- 172.21.15.100:0/1649574760 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3741f9960 con 0x7ff3741f91f0 2024-09-17T04:32:02.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.264+0000 7ff378a00640 1 --2- 172.21.15.100:0/1649574760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff37414df80 0x7ff3741f6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:02.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.264+0000 7ff373400640 1 --2- 172.21.15.100:0/1649574760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3741f91f0 0x7ff37409dc50 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7ff36000e5a0 tx=0x7ff36000e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:02.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.265+0000 7ff36be00640 1 -- 172.21.15.100:0/1649574760 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff360015070 con 0x7ff3741f91f0 2024-09-17T04:32:02.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.265+0000 7ff36be00640 1 -- 172.21.15.100:0/1649574760 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff360010980 con 0x7ff3741f91f0 2024-09-17T04:32:02.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.265+0000 7ff37aa00640 1 -- 172.21.15.100:0/1649574760 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3741f9c40 con 0x7ff3741f91f0 2024-09-17T04:32:02.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.265+0000 7ff36be00640 1 -- 172.21.15.100:0/1649574760 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff360010d30 con 0x7ff3741f91f0 2024-09-17T04:32:02.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.265+0000 7ff37aa00640 1 -- 172.21.15.100:0/1649574760 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3741f5df0 con 0x7ff3741f91f0 2024-09-17T04:32:02.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.266+0000 7ff37aa00640 1 -- 172.21.15.100:0/1649574760 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3741f96d0 con 0x7ff3741f91f0 2024-09-17T04:32:02.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.267+0000 7ff36be00640 1 -- 172.21.15.100:0/1649574760 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7ff360018070 con 0x7ff3741f91f0 2024-09-17T04:32:02.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.268+0000 7ff36be00640 1 --2- 172.21.15.100:0/1649574760 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff354079220 0x7ff35407b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:02.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.268+0000 7ff36be00640 1 -- 172.21.15.100:0/1649574760 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7ff36009bd20 con 0x7ff3741f91f0 2024-09-17T04:32:02.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.268+0000 7ff373e00640 1 --2- 172.21.15.100:0/1649574760 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff354079220 0x7ff35407b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:02.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.269+0000 7ff373e00640 1 --2- 172.21.15.100:0/1649574760 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff354079220 0x7ff35407b730 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff364036000 tx=0x7ff3640046c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:02.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.273+0000 7ff36be00640 1 -- 172.21.15.100:0/1649574760 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff360063050 con 0x7ff3741f91f0 2024-09-17T04:32:02.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.489+0000 7ff37aa00640 1 -- 172.21.15.100:0/1649574760 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff3741f5c00 con 0x7ff3741f91f0 2024-09-17T04:32:02.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.490+0000 7ff36be00640 1 -- 172.21.15.100:0/1649574760 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v91) ==== 53+0+73 (secure 0 0 0) 0x7ff360083e20 con 0x7ff3741f91f0 2024-09-17T04:32:02.490 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:32:02.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.492+0000 7ff37aa00640 1 -- 172.21.15.100:0/1649574760 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff354079220 msgr2=0x7ff35407b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:02.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.492+0000 7ff37aa00640 1 --2- 172.21.15.100:0/1649574760 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff354079220 0x7ff35407b730 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff364036000 tx=0x7ff3640046c0 comp rx=0 tx=0).stop 2024-09-17T04:32:02.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.492+0000 7ff37aa00640 1 -- 172.21.15.100:0/1649574760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3741f91f0 msgr2=0x7ff37409dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:02.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.492+0000 7ff37aa00640 1 --2- 172.21.15.100:0/1649574760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3741f91f0 0x7ff37409dc50 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7ff36000e5a0 tx=0x7ff36000e9a0 comp rx=0 tx=0).stop 2024-09-17T04:32:02.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.492+0000 7ff37aa00640 1 -- 172.21.15.100:0/1649574760 shutdown_connections 2024-09-17T04:32:02.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.492+0000 7ff37aa00640 1 --2- 172.21.15.100:0/1649574760 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff354079220 0x7ff35407b730 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:02.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.492+0000 7ff37aa00640 1 --2- 172.21.15.100:0/1649574760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff37414df80 0x7ff3741f6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:02.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.492+0000 7ff37aa00640 1 --2- 172.21.15.100:0/1649574760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3741f91f0 0x7ff37409dc50 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:02.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.492+0000 7ff37aa00640 1 --2- 172.21.15.100:0/1649574760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff358003c10 0x7ff37409bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:02.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.492+0000 7ff37aa00640 1 -- 172.21.15.100:0/1649574760 >> 172.21.15.100:0/1649574760 conn(0x7ff37406d730 msgr2=0x7ff374157e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:02.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.492+0000 7ff37aa00640 1 -- 172.21.15.100:0/1649574760 shutdown_connections 2024-09-17T04:32:02.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:02.492+0000 7ff37aa00640 1 -- 172.21.15.100:0/1649574760 wait complete. 2024-09-17T04:32:02.515 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T04:32:02.515 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:32:02.515 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2024-09-17T04:32:02.515 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T04:32:02.515 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T04:32:02.515 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T04:32:02.515 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T04:32:02.515 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:32:02.516 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:32:02.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:02.516 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2024-09-17T04:32:02.557 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:32:02.557 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:32:02.557 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:32:02.557 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:32:02.557 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-09-17T04:32:02.557 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:32:02.557 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:32:02.558 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:32:02.558 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2024-09-17T04:32:02.558 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:32:02.558 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:32:02.558 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2024-09-17T04:32:02.558 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:32:02.558 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2024-09-17T04:32:02.558 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:32:02.558 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2024-09-17T04:32:02.558 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T04:32:02.558 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42221sec preferred_lft 42221sec 2024-09-17T04:32:02.558 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ec4:7aff:fe88:7b8f/64 scope link noprefixroute 2024-09-17T04:32:02.558 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:32:02.558 INFO:teuthology.orchestra.run.smithi184.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:32:02.559 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 7e:67:e5:69:96:69 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:32:02.559 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T04:32:02.559 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:32:02.559 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::3c9d:aff:fe6e:8fcb/64 scope link 2024-09-17T04:32:02.559 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:32:02.559 INFO:teuthology.orchestra.run.smithi184.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:32:02.559 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether a6:a9:1b:a9:61:ec brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T04:32:02.559 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::a4a9:1bff:fea9:61ec/64 scope link 2024-09-17T04:32:02.559 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:32:02.559 INFO:teuthology.orchestra.run.smithi184.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:32:02.559 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 7e:67:e5:69:96:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T04:32:02.559 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::7c67:e5ff:fe69:9669/64 scope link 2024-09-17T04:32:02.559 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:32:02.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:02.560 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:32:02.655 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-09-17T04:32:02.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:02.658 DEBUG:teuthology.orchestra.run.smithi184:> route 2024-09-17T04:32:02.695 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2024-09-17T04:32:02.696 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:32:02.696 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T04:32:02.696 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T04:32:02.696 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:32:02.696 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:32:02.696 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-09-17T04:32:02.696 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T04:32:02.697 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T04:32:02.697 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T04:32:02.697 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-09-17T04:32:02.814 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:32:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:32:02.886 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:32:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:32:02.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:02.894 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-09-17T04:32:02.929 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T04:32:02.929 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T04:32:02.929 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:32:02.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:02.930 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.0 up 2024-09-17T04:32:03.025 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:32:03.025 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T04:32:03.060 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:32:03.060 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:32:03.060 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T04:32:03.135 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T04:32:03.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:03.135 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:32:03.330 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:32:03.330 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:32:03.330 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:32:03.350 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:32:03.350 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:32:03.350 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:32:03.350 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:32:03.350 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:32:03.380 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:32:03.381 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:32:03.381 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:32:03.381 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:32:03.381 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:32:03.381 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-09-17T04:32:03.381 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-09-17T04:32:03.381 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8a399842-3a5c-4ea2-9afb-b2704332e59f" will pass to kernel 2024-09-17T04:32:03.382 INFO:teuthology.orchestra.run.smithi184.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-17T04:32:03.382 INFO:teuthology.orchestra.run.smithi184.stdout: You just mounted a file system that supports labels which does not 2024-09-17T04:32:03.382 INFO:teuthology.orchestra.run.smithi184.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-17T04:32:03.382 INFO:teuthology.orchestra.run.smithi184.stdout: applications will generate AVC messages and not be allowed access to 2024-09-17T04:32:03.383 INFO:teuthology.orchestra.run.smithi184.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-17T04:32:03.385 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T04:32:03.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:03.386 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T04:32:03.441 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:32:03.519 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:32:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:32:03.522 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:32:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:32:03.529 INFO:teuthology.orchestra.run.smithi184.stdout:client6824 2024-09-17T04:32:03.530 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:32:03.530 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:32:03.565 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T04:32:03.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.800+0000 7f37ff400640 1 -- 172.21.15.100:0/2807204660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37f815fd30 msgr2=0x7f37f80b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:03.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.800+0000 7f37ff400640 1 --2- 172.21.15.100:0/2807204660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37f815fd30 0x7f37f80b4620 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f37ec009830 tx=0x7f37ec02a510 comp rx=0 tx=0).stop 2024-09-17T04:32:03.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.800+0000 7f37ff400640 1 -- 172.21.15.100:0/2807204660 shutdown_connections 2024-09-17T04:32:03.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.800+0000 7f37ff400640 1 --2- 172.21.15.100:0/2807204660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37f806cda0 0x7f37f806d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:03.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.800+0000 7f37ff400640 1 --2- 172.21.15.100:0/2807204660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37f815fd30 0x7f37f80b4620 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:03.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.800+0000 7f37ff400640 1 -- 172.21.15.100:0/2807204660 >> 172.21.15.100:0/2807204660 conn(0x7f37f80b2540 msgr2=0x7f37f80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:03.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.801+0000 7f37ff400640 1 -- 172.21.15.100:0/2807204660 shutdown_connections 2024-09-17T04:32:03.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.801+0000 7f37ff400640 1 -- 172.21.15.100:0/2807204660 wait complete. 2024-09-17T04:32:03.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.803+0000 7f37ff400640 1 Processor -- start 2024-09-17T04:32:03.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.803+0000 7f37ff400640 1 -- start start 2024-09-17T04:32:03.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.804+0000 7f37ff400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37f806cda0 0x7f37f815cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:03.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.804+0000 7f37ff400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37f815fd30 0x7f37f81f6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:03.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.804+0000 7f37ff400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37f8162ff0 0x7f37f80743e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:03.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.804+0000 7f37ff400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f37f80af490 con 0x7f37f806cda0 2024-09-17T04:32:03.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.804+0000 7f37ff400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37f80af310 con 0x7f37f8162ff0 2024-09-17T04:32:03.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.804+0000 7f37ff400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f37f80af610 con 0x7f37f815fd30 2024-09-17T04:32:03.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.804+0000 7f37fca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37f806cda0 0x7f37f815cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:03.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.805+0000 7f37fca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37f806cda0 0x7f37f815cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48926/0 (socket says 172.21.15.100:48926) 2024-09-17T04:32:03.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.805+0000 7f37fca00640 1 -- 172.21.15.100:0/638793603 learned_addr learned my addr 172.21.15.100:0/638793603 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:03.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.805+0000 7f37f7e00640 1 --2- 172.21.15.100:0/638793603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37f815fd30 0x7f37f81f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:03.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.805+0000 7f37fd400640 1 --2- 172.21.15.100:0/638793603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37f8162ff0 0x7f37f80743e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:03.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.805+0000 7f37fca00640 1 -- 172.21.15.100:0/638793603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37f815fd30 msgr2=0x7f37f81f6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:03.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.805+0000 7f37fca00640 1 --2- 172.21.15.100:0/638793603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37f815fd30 0x7f37f81f6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:03.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.805+0000 7f37fca00640 1 -- 172.21.15.100:0/638793603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37f8162ff0 msgr2=0x7f37f80743e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:03.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.805+0000 7f37fca00640 1 --2- 172.21.15.100:0/638793603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37f8162ff0 0x7f37f80743e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:03.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.805+0000 7f37fca00640 1 -- 172.21.15.100:0/638793603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f81f6680 con 0x7f37f806cda0 2024-09-17T04:32:03.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.805+0000 7f37f7e00640 1 --2- 172.21.15.100:0/638793603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37f815fd30 0x7f37f81f6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:03.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.805+0000 7f37fd400640 1 --2- 172.21.15.100:0/638793603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37f8162ff0 0x7f37f80743e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:03.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.805+0000 7f37fca00640 1 --2- 172.21.15.100:0/638793603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37f806cda0 0x7f37f815cbf0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f37ec009950 tx=0x7f37ec033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:03.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.806+0000 7f37f4c00640 1 -- 172.21.15.100:0/638793603 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec005c90 con 0x7f37f806cda0 2024-09-17T04:32:03.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.806+0000 7f37f4c00640 1 -- 172.21.15.100:0/638793603 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37ec031e50 con 0x7f37f806cda0 2024-09-17T04:32:03.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.806+0000 7f37f4c00640 1 -- 172.21.15.100:0/638793603 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec005260 con 0x7f37f806cda0 2024-09-17T04:32:03.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.806+0000 7f37ff400640 1 -- 172.21.15.100:0/638793603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37f80b0e30 con 0x7f37f806cda0 2024-09-17T04:32:03.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.806+0000 7f37ff400640 1 -- 172.21.15.100:0/638793603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37f80b13f0 con 0x7f37f806cda0 2024-09-17T04:32:03.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.807+0000 7f37ff400640 1 -- 172.21.15.100:0/638793603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37f80afb30 con 0x7f37f806cda0 2024-09-17T04:32:03.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.808+0000 7f37f4c00640 1 -- 172.21.15.100:0/638793603 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f37ec005400 con 0x7f37f806cda0 2024-09-17T04:32:03.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.809+0000 7f37f4c00640 1 --2- 172.21.15.100:0/638793603 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f37d80791c0 0x7f37d807b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:03.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.810+0000 7f37f4c00640 1 -- 172.21.15.100:0/638793603 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7f37ec0bc9f0 con 0x7f37f806cda0 2024-09-17T04:32:03.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.810+0000 7f37f7e00640 1 --2- 172.21.15.100:0/638793603 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f37d80791c0 0x7f37d807b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:03.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.810+0000 7f37f7e00640 1 --2- 172.21.15.100:0/638793603 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f37d80791c0 0x7f37d807b6d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f37e4007fd0 tx=0x7f37e4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:03.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:03.813+0000 7f37f4c00640 1 -- 172.21.15.100:0/638793603 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f37ec03f030 con 0x7f37f806cda0 2024-09-17T04:32:04.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:04.038+0000 7f37ff400640 1 -- 172.21.15.100:0/638793603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f37f80b3cd0 con 0x7f37f806cda0 2024-09-17T04:32:04.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:04.870+0000 7f37f4c00640 1 -- 172.21.15.100:0/638793603 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v92) ==== 136+0+0 (secure 0 0 0) 0x7f37f806d6f0 con 0x7f37f806cda0 2024-09-17T04:32:04.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:04.873+0000 7f37ff400640 1 -- 172.21.15.100:0/638793603 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f37d80791c0 msgr2=0x7f37d807b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:04.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:04.873+0000 7f37ff400640 1 --2- 172.21.15.100:0/638793603 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f37d80791c0 0x7f37d807b6d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f37e4007fd0 tx=0x7f37e4009040 comp rx=0 tx=0).stop 2024-09-17T04:32:04.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:04.873+0000 7f37ff400640 1 -- 172.21.15.100:0/638793603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37f806cda0 msgr2=0x7f37f815cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:04.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:04.873+0000 7f37ff400640 1 --2- 172.21.15.100:0/638793603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37f806cda0 0x7f37f815cbf0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f37ec009950 tx=0x7f37ec033c20 comp rx=0 tx=0).stop 2024-09-17T04:32:04.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:04.873+0000 7f37ff400640 1 -- 172.21.15.100:0/638793603 shutdown_connections 2024-09-17T04:32:04.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:04.873+0000 7f37ff400640 1 --2- 172.21.15.100:0/638793603 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f37d80791c0 0x7f37d807b6d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:04.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:04.873+0000 7f37ff400640 1 --2- 172.21.15.100:0/638793603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37f8162ff0 0x7f37f80743e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:04.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:04.873+0000 7f37ff400640 1 --2- 172.21.15.100:0/638793603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37f815fd30 0x7f37f81f6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:04.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:04.873+0000 7f37ff400640 1 --2- 172.21.15.100:0/638793603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37f806cda0 0x7f37f815cbf0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:04.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:04.873+0000 7f37ff400640 1 -- 172.21.15.100:0/638793603 >> 172.21.15.100:0/638793603 conn(0x7f37f80b2540 msgr2=0x7f37f80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:04.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:04.873+0000 7f37ff400640 1 -- 172.21.15.100:0/638793603 shutdown_connections 2024-09-17T04:32:04.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:04.873+0000 7f37ff400640 1 -- 172.21.15.100:0/638793603 wait complete. 2024-09-17T04:32:04.902 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T04:32:05.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.126+0000 7f4626c00640 1 -- 172.21.15.100:0/1932741464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4604004800 msgr2=0x7f4604004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:05.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.126+0000 7f4626c00640 1 --2- 172.21.15.100:0/1932741464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4604004800 0x7f4604004c50 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f460c006fb0 tx=0x7f4604005830 comp rx=0 tx=0).stop 2024-09-17T04:32:05.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.126+0000 7f4624c00640 1 -- 172.21.15.100:0/1932741464 reap_dead start 2024-09-17T04:32:05.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.126+0000 7f4626c00640 1 -- 172.21.15.100:0/1932741464 shutdown_connections 2024-09-17T04:32:05.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.126+0000 7f4626c00640 1 -- 172.21.15.100:0/1932741464 >> 172.21.15.100:0/1932741464 conn(0x7f462006d3d0 msgr2=0x7f46200b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:05.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.126+0000 7f4626c00640 1 -- 172.21.15.100:0/1932741464 shutdown_connections 2024-09-17T04:32:05.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.126+0000 7f4626c00640 1 -- 172.21.15.100:0/1932741464 wait complete. 2024-09-17T04:32:05.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.128+0000 7f4626c00640 1 Processor -- start 2024-09-17T04:32:05.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.129+0000 7f4626c00640 1 -- start start 2024-09-17T04:32:05.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.130+0000 7f4626c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4604004800 0x7f46200ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:05.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.130+0000 7f4626c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46200ad700 0x7f4620074510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:05.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.130+0000 7f4626c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4620074a50 0x7f46200afd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:05.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.130+0000 7f4626c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f46200cccc0 con 0x7f46200ad700 2024-09-17T04:32:05.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.130+0000 7f4626c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4620157290 con 0x7f4620074a50 2024-09-17T04:32:05.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.130+0000 7f4626c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f46200cce40 con 0x7f4604004800 2024-09-17T04:32:05.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.130+0000 7f461f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46200ad700 0x7f4620074510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:05.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.130+0000 7f461f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46200ad700 0x7f4620074510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51108/0 (socket says 172.21.15.100:51108) 2024-09-17T04:32:05.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.130+0000 7f461f400640 1 -- 172.21.15.100:0/2036867180 learned_addr learned my addr 172.21.15.100:0/2036867180 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:05.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.130+0000 7f461fe00640 1 --2- 172.21.15.100:0/2036867180 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4604004800 0x7f46200ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:05.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.130+0000 7f4624c00640 1 --2- 172.21.15.100:0/2036867180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4620074a50 0x7f46200afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:05.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.131+0000 7f461f400640 1 -- 172.21.15.100:0/2036867180 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4604004800 msgr2=0x7f46200ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:05.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.131+0000 7f461f400640 1 --2- 172.21.15.100:0/2036867180 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4604004800 0x7f46200ad1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:05.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.131+0000 7f461f400640 1 -- 172.21.15.100:0/2036867180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4620074a50 msgr2=0x7f46200afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:05.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.131+0000 7f461f400640 1 --2- 172.21.15.100:0/2036867180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4620074a50 0x7f46200afd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:05.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.131+0000 7f461f400640 1 -- 172.21.15.100:0/2036867180 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4620074e60 con 0x7f46200ad700 2024-09-17T04:32:05.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.131+0000 7f461f400640 1 --2- 172.21.15.100:0/2036867180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46200ad700 0x7f4620074510 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f460c031040 tx=0x7f460c0010a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:05.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.131+0000 7f4617e00640 1 -- 172.21.15.100:0/2036867180 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c0037f0 con 0x7f46200ad700 2024-09-17T04:32:05.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.131+0000 7f4617e00640 1 -- 172.21.15.100:0/2036867180 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f460c003a80 con 0x7f46200ad700 2024-09-17T04:32:05.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.131+0000 7f4617e00640 1 -- 172.21.15.100:0/2036867180 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c02e480 con 0x7f46200ad700 2024-09-17T04:32:05.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.131+0000 7f4626c00640 1 -- 172.21.15.100:0/2036867180 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46200ca4b0 con 0x7f46200ad700 2024-09-17T04:32:05.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.131+0000 7f4626c00640 1 -- 172.21.15.100:0/2036867180 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46200ae1d0 con 0x7f46200ad700 2024-09-17T04:32:05.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.133+0000 7f4626c00640 1 -- 172.21.15.100:0/2036867180 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46200adc40 con 0x7f46200ad700 2024-09-17T04:32:05.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.134+0000 7f4617e00640 1 -- 172.21.15.100:0/2036867180 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f460c035020 con 0x7f46200ad700 2024-09-17T04:32:05.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.135+0000 7f4617e00640 1 --2- 172.21.15.100:0/2036867180 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f46000791c0 0x7f460007b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:05.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.135+0000 7f4617e00640 1 -- 172.21.15.100:0/2036867180 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7f460c0bbde0 con 0x7f46200ad700 2024-09-17T04:32:05.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.135+0000 7f461fe00640 1 --2- 172.21.15.100:0/2036867180 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f46000791c0 0x7f460007b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:05.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.136+0000 7f461fe00640 1 --2- 172.21.15.100:0/2036867180 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f46000791c0 0x7f460007b6d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4610006fd0 tx=0x7f4610006bf0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:05.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.139+0000 7f4617e00640 1 -- 172.21.15.100:0/2036867180 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f460c047030 con 0x7f46200ad700 2024-09-17T04:32:05.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.347+0000 7f4626c00640 1 -- 172.21.15.100:0/2036867180 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f46200cd3d0 con 0x7f46000791c0 2024-09-17T04:32:05.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.348+0000 7f4617e00640 1 -- 172.21.15.100:0/2036867180 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f46200cd3d0 con 0x7f46000791c0 2024-09-17T04:32:05.350 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-09-17T04:32:05.350 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:32:05.351 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephfs" 2024-09-17T04:32:05.351 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:32:05.351 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-09-17T04:32:05.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.351+0000 7f4626c00640 1 -- 172.21.15.100:0/2036867180 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f46000791c0 msgr2=0x7f460007b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:05.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.351+0000 7f4626c00640 1 --2- 172.21.15.100:0/2036867180 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f46000791c0 0x7f460007b6d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4610006fd0 tx=0x7f4610006bf0 comp rx=0 tx=0).stop 2024-09-17T04:32:05.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.351+0000 7f4626c00640 1 -- 172.21.15.100:0/2036867180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46200ad700 msgr2=0x7f4620074510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:05.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.351+0000 7f4626c00640 1 --2- 172.21.15.100:0/2036867180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46200ad700 0x7f4620074510 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f460c031040 tx=0x7f460c0010a0 comp rx=0 tx=0).stop 2024-09-17T04:32:05.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.351+0000 7f4626c00640 1 -- 172.21.15.100:0/2036867180 shutdown_connections 2024-09-17T04:32:05.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.351+0000 7f4626c00640 1 --2- 172.21.15.100:0/2036867180 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f46000791c0 0x7f460007b6d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:05.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.351+0000 7f4626c00640 1 --2- 172.21.15.100:0/2036867180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4620074a50 0x7f46200afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:05.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.351+0000 7f4626c00640 1 --2- 172.21.15.100:0/2036867180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46200ad700 0x7f4620074510 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:05.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.351+0000 7f4626c00640 1 --2- 172.21.15.100:0/2036867180 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4604004800 0x7f46200ad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:05.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.351+0000 7f4626c00640 1 -- 172.21.15.100:0/2036867180 >> 172.21.15.100:0/2036867180 conn(0x7f462006d3d0 msgr2=0x7f46201603b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:05.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.351+0000 7f4626c00640 1 -- 172.21.15.100:0/2036867180 shutdown_connections 2024-09-17T04:32:05.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.351+0000 7f4626c00640 1 -- 172.21.15.100:0/2036867180 wait complete. 2024-09-17T04:32:05.377 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T04:32:05.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.602+0000 7fb335c00640 1 -- 172.21.15.100:0/1947172139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb33015b4a0 msgr2=0x7fb3300d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:05.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.602+0000 7fb335c00640 1 --2- 172.21.15.100:0/1947172139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb33015b4a0 0x7fb3300d0ab0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb318009830 tx=0x7fb31802a410 comp rx=0 tx=0).stop 2024-09-17T04:32:05.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.602+0000 7fb335c00640 1 -- 172.21.15.100:0/1947172139 shutdown_connections 2024-09-17T04:32:05.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.602+0000 7fb335c00640 1 --2- 172.21.15.100:0/1947172139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb33015b4a0 0x7fb3300d0ab0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:05.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.602+0000 7fb335c00640 1 --2- 172.21.15.100:0/1947172139 >> v2:172.21.15.184:3300/0 conn(0x7fb3300b50d0 0x7fb33006fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:05.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.602+0000 7fb335c00640 1 -- 172.21.15.100:0/1947172139 >> 172.21.15.100:0/1947172139 conn(0x7fb33006cf80 msgr2=0x7fb33006d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:05.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.602+0000 7fb335c00640 1 -- 172.21.15.100:0/1947172139 shutdown_connections 2024-09-17T04:32:05.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.602+0000 7fb335c00640 1 -- 172.21.15.100:0/1947172139 wait complete. 2024-09-17T04:32:05.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.604+0000 7fb335c00640 1 Processor -- start 2024-09-17T04:32:05.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.605+0000 7fb335c00640 1 -- start start 2024-09-17T04:32:05.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.606+0000 7fb335c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3300b50d0 0x7fb330148370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:05.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.606+0000 7fb335c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3300d0ff0 0x7fb3301f6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:05.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.606+0000 7fb335c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb33015b4a0 0x7fb3300b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:05.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.606+0000 7fb335c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb33015ba30 con 0x7fb3300d0ff0 2024-09-17T04:32:05.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.606+0000 7fb335c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb33015b8b0 con 0x7fb3300b50d0 2024-09-17T04:32:05.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.606+0000 7fb335c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb33015f910 con 0x7fb33015b4a0 2024-09-17T04:32:05.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.606+0000 7fb327e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3300d0ff0 0x7fb3301f6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:05.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.606+0000 7fb327e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3300d0ff0 0x7fb3301f6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51112/0 (socket says 172.21.15.100:51112) 2024-09-17T04:32:05.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.606+0000 7fb327e00640 1 -- 172.21.15.100:0/2033831049 learned_addr learned my addr 172.21.15.100:0/2033831049 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:05.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.606+0000 7fb32fe00640 1 --2- 172.21.15.100:0/2033831049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3300b50d0 0x7fb330148370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:05.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.607+0000 7fb335200640 1 --2- 172.21.15.100:0/2033831049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb33015b4a0 0x7fb3300b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:05.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.607+0000 7fb327e00640 1 -- 172.21.15.100:0/2033831049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb33015b4a0 msgr2=0x7fb3300b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:05.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.607+0000 7fb327e00640 1 --2- 172.21.15.100:0/2033831049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb33015b4a0 0x7fb3300b2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:05.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.607+0000 7fb327e00640 1 -- 172.21.15.100:0/2033831049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3300b50d0 msgr2=0x7fb330148370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:05.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.607+0000 7fb327e00640 1 --2- 172.21.15.100:0/2033831049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3300b50d0 0x7fb330148370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:05.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.607+0000 7fb327e00640 1 -- 172.21.15.100:0/2033831049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3301f3760 con 0x7fb3300d0ff0 2024-09-17T04:32:05.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.607+0000 7fb327e00640 1 --2- 172.21.15.100:0/2033831049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3300d0ff0 0x7fb3301f6d90 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fb31802aa00 tx=0x7fb318003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:05.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.607+0000 7fb32cc00640 1 -- 172.21.15.100:0/2033831049 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31803f030 con 0x7fb3300d0ff0 2024-09-17T04:32:05.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.607+0000 7fb32cc00640 1 -- 172.21.15.100:0/2033831049 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb318004510 con 0x7fb3300d0ff0 2024-09-17T04:32:05.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.607+0000 7fb32cc00640 1 -- 172.21.15.100:0/2033831049 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb318037040 con 0x7fb3300d0ff0 2024-09-17T04:32:05.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.607+0000 7fb335c00640 1 -- 172.21.15.100:0/2033831049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3301f3a40 con 0x7fb3300d0ff0 2024-09-17T04:32:05.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.608+0000 7fb335c00640 1 -- 172.21.15.100:0/2033831049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3301f4000 con 0x7fb3300d0ff0 2024-09-17T04:32:05.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.609+0000 7fb335c00640 1 -- 172.21.15.100:0/2033831049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3301f7870 con 0x7fb3300d0ff0 2024-09-17T04:32:05.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.610+0000 7fb32cc00640 1 -- 172.21.15.100:0/2033831049 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fb318042070 con 0x7fb3300d0ff0 2024-09-17T04:32:05.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.611+0000 7fb32cc00640 1 --2- 172.21.15.100:0/2033831049 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb30c0791c0 0x7fb30c07b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:05.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.611+0000 7fb32cc00640 1 -- 172.21.15.100:0/2033831049 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7fb3180bb470 con 0x7fb3300d0ff0 2024-09-17T04:32:05.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.612+0000 7fb32fe00640 1 --2- 172.21.15.100:0/2033831049 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb30c0791c0 0x7fb30c07b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:05.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.612+0000 7fb32fe00640 1 --2- 172.21.15.100:0/2033831049 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb30c0791c0 0x7fb30c07b6d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb320009b30 tx=0x7fb320005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:05.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.616+0000 7fb32cc00640 1 -- 172.21.15.100:0/2033831049 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb318049070 con 0x7fb3300d0ff0 2024-09-17T04:32:05.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.802+0000 7fb335c00640 1 -- 172.21.15.100:0/2033831049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb33015fdc0 con 0x7fb3300d0ff0 2024-09-17T04:32:05.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.803+0000 7fb32cc00640 1 -- 172.21.15.100:0/2033831049 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7fb3180828b0 con 0x7fb3300d0ff0 2024-09-17T04:32:05.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.805+0000 7fb335c00640 1 -- 172.21.15.100:0/2033831049 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb30c0791c0 msgr2=0x7fb30c07b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:05.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.805+0000 7fb335c00640 1 --2- 172.21.15.100:0/2033831049 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb30c0791c0 0x7fb30c07b6d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb320009b30 tx=0x7fb320005f90 comp rx=0 tx=0).stop 2024-09-17T04:32:05.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.805+0000 7fb335c00640 1 -- 172.21.15.100:0/2033831049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3300d0ff0 msgr2=0x7fb3301f6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:05.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.805+0000 7fb335c00640 1 --2- 172.21.15.100:0/2033831049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3300d0ff0 0x7fb3301f6d90 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fb31802aa00 tx=0x7fb318003b60 comp rx=0 tx=0).stop 2024-09-17T04:32:05.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.805+0000 7fb335c00640 1 -- 172.21.15.100:0/2033831049 shutdown_connections 2024-09-17T04:32:05.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.805+0000 7fb335c00640 1 --2- 172.21.15.100:0/2033831049 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb30c0791c0 0x7fb30c07b6d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:05.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.805+0000 7fb335c00640 1 --2- 172.21.15.100:0/2033831049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb33015b4a0 0x7fb3300b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:05.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.805+0000 7fb335c00640 1 --2- 172.21.15.100:0/2033831049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3300d0ff0 0x7fb3301f6d90 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:05.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.805+0000 7fb335c00640 1 --2- 172.21.15.100:0/2033831049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3300b50d0 0x7fb330148370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:05.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.805+0000 7fb335c00640 1 -- 172.21.15.100:0/2033831049 >> 172.21.15.100:0/2033831049 conn(0x7fb33006cf80 msgr2=0x7fb3300733e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:05.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.805+0000 7fb335c00640 1 -- 172.21.15.100:0/2033831049 shutdown_connections 2024-09-17T04:32:05.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:05.805+0000 7fb335c00640 1 -- 172.21.15.100:0/2033831049 wait complete. 2024-09-17T04:32:05.832 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create non_existent_subvolume subvolume_1 2024-09-17T04:32:06.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.056+0000 7f18b5000640 1 -- 172.21.15.100:0/1319737340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b0145550 msgr2=0x7f18b006fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:06.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.057+0000 7f18b5000640 1 --2- 172.21.15.100:0/1319737340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b0145550 0x7f18b006fb40 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f18a0005fd0 tx=0x7f18a002a670 comp rx=0 tx=0).stop 2024-09-17T04:32:06.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.057+0000 7f18b5000640 1 -- 172.21.15.100:0/1319737340 shutdown_connections 2024-09-17T04:32:06.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.057+0000 7f18b5000640 1 --2- 172.21.15.100:0/1319737340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b0145550 0x7f18b006fb40 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:06.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.057+0000 7f18b5000640 1 -- 172.21.15.100:0/1319737340 >> 172.21.15.100:0/1319737340 conn(0x7f18b006d1c0 msgr2=0x7f18b006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:06.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.057+0000 7f18b5000640 1 -- 172.21.15.100:0/1319737340 shutdown_connections 2024-09-17T04:32:06.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.057+0000 7f18b5000640 1 -- 172.21.15.100:0/1319737340 wait complete. 2024-09-17T04:32:06.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.059+0000 7f18b5000640 1 Processor -- start 2024-09-17T04:32:06.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.059+0000 7f18b5000640 1 -- start start 2024-09-17T04:32:06.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.060+0000 7f18b5000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18b0145550 0x7f18b013b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:06.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.060+0000 7f18b5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b0145a40 0x7f18b00a97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:06.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.060+0000 7f18b5000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18b00b1fe0 0x7f18b00b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:06.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.060+0000 7f18b5000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f18b00ce360 con 0x7f18b0145a40 2024-09-17T04:32:06.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.060+0000 7f18b5000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f18b00ce1e0 con 0x7f18b00b1fe0 2024-09-17T04:32:06.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.060+0000 7f18b5000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f18b00ce4e0 con 0x7f18b0145550 2024-09-17T04:32:06.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.061+0000 7f18ad400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b0145a40 0x7f18b00a97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:06.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.061+0000 7f18ad400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b0145a40 0x7f18b00a97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51122/0 (socket says 172.21.15.100:51122) 2024-09-17T04:32:06.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.061+0000 7f18ad400640 1 -- 172.21.15.100:0/532384375 learned_addr learned my addr 172.21.15.100:0/532384375 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:06.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.061+0000 7f18ae800640 1 --2- 172.21.15.100:0/532384375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18b00b1fe0 0x7f18b00b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:06.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.061+0000 7f18ade00640 1 --2- 172.21.15.100:0/532384375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18b0145550 0x7f18b013b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:06.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.061+0000 7f18ad400640 1 -- 172.21.15.100:0/532384375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18b0145550 msgr2=0x7f18b013b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:06.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.061+0000 7f18ad400640 1 --2- 172.21.15.100:0/532384375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18b0145550 0x7f18b013b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:06.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.061+0000 7f18ad400640 1 -- 172.21.15.100:0/532384375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18b00b1fe0 msgr2=0x7f18b00b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:06.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.061+0000 7f18ad400640 1 --2- 172.21.15.100:0/532384375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18b00b1fe0 0x7f18b00b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:06.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.061+0000 7f18ad400640 1 -- 172.21.15.100:0/532384375 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b00b2930 con 0x7f18b0145a40 2024-09-17T04:32:06.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.062+0000 7f18ad400640 1 --2- 172.21.15.100:0/532384375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b0145a40 0x7f18b00a97e0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f189c00b3f0 tx=0x7f189c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:06.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.062+0000 7f18a6000640 1 -- 172.21.15.100:0/532384375 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c015020 con 0x7f18b0145a40 2024-09-17T04:32:06.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.062+0000 7f18a6000640 1 -- 172.21.15.100:0/532384375 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f189c011920 con 0x7f18b0145a40 2024-09-17T04:32:06.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.062+0000 7f18a6000640 1 -- 172.21.15.100:0/532384375 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c011d70 con 0x7f18b0145a40 2024-09-17T04:32:06.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.062+0000 7f18b5000640 1 -- 172.21.15.100:0/532384375 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b015ddd0 con 0x7f18b0145a40 2024-09-17T04:32:06.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.062+0000 7f18b5000640 1 -- 172.21.15.100:0/532384375 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b015ede0 con 0x7f18b0145a40 2024-09-17T04:32:06.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.063+0000 7f18b5000640 1 -- 172.21.15.100:0/532384375 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18b006fb40 con 0x7f18b0145a40 2024-09-17T04:32:06.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.064+0000 7f18a6000640 1 -- 172.21.15.100:0/532384375 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f189c018020 con 0x7f18b0145a40 2024-09-17T04:32:06.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.065+0000 7f18a6000640 1 --2- 172.21.15.100:0/532384375 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f18900791f0 0x7f189007b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:06.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.065+0000 7f18a6000640 1 -- 172.21.15.100:0/532384375 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7f189c09b580 con 0x7f18b0145a40 2024-09-17T04:32:06.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.066+0000 7f18ade00640 1 --2- 172.21.15.100:0/532384375 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f18900791f0 0x7f189007b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:06.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.067+0000 7f18ade00640 1 --2- 172.21.15.100:0/532384375 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f18900791f0 0x7f189007b700 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f18a0033000 tx=0x7f18a0004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:06.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.070+0000 7f18a6000640 1 -- 172.21.15.100:0/532384375 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f189c01f030 con 0x7f18b0145a40 2024-09-17T04:32:06.125 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:32:06.129 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:32:06.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.284+0000 7f18b5000640 1 -- 172.21.15.100:0/532384375 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f18b01411d0 con 0x7f18900791f0 2024-09-17T04:32:06.285 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:06.285+0000 7f30b3000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-17T04:32:06.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.285+0000 7f18a6000640 1 -- 172.21.15.100:0/532384375 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f18b01411d0 con 0x7f18900791f0 2024-09-17T04:32:06.286 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-17T04:32:06.305 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T04:32:06.306 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm non_existent_subvolume subvolume_1 2024-09-17T04:32:06.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.530+0000 7f76f7200640 1 -- 172.21.15.100:0/3995016394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76dc004d40 msgr2=0x7f76dc005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:06.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.530+0000 7f76f7200640 1 --2- 172.21.15.100:0/3995016394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76dc004d40 0x7f76dc005150 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f76f81598e0 tx=0x7f76dc005830 comp rx=0 tx=0).stop 2024-09-17T04:32:06.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.530+0000 7f76f6800640 1 -- 172.21.15.100:0/3995016394 reap_dead start 2024-09-17T04:32:06.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.530+0000 7f76f7200640 1 -- 172.21.15.100:0/3995016394 shutdown_connections 2024-09-17T04:32:06.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.530+0000 7f76f7200640 1 -- 172.21.15.100:0/3995016394 >> 172.21.15.100:0/3995016394 conn(0x7f76f80b6640 msgr2=0x7f76f80b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:06.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.530+0000 7f76f7200640 1 -- 172.21.15.100:0/3995016394 shutdown_connections 2024-09-17T04:32:06.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.530+0000 7f76f7200640 1 -- 172.21.15.100:0/3995016394 wait complete. 2024-09-17T04:32:06.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.533+0000 7f76f7200640 1 Processor -- start 2024-09-17T04:32:06.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.533+0000 7f76f7200640 1 -- start start 2024-09-17T04:32:06.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.534+0000 7f76f7200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76dc004d40 0x7f76f814edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:06.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.534+0000 7f76f7200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76f81f8970 0x7f76f8070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:06.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.534+0000 7f76f7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76f8070990 0x7f76f8160fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:06.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.534+0000 7f76f7200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f76f8163300 con 0x7f76f8070990 2024-09-17T04:32:06.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.534+0000 7f76f7200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f76f815caa0 con 0x7f76dc004d40 2024-09-17T04:32:06.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.534+0000 7f76f7200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f76f8163480 con 0x7f76f81f8970 2024-09-17T04:32:06.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.535+0000 7f76f6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76f8070990 0x7f76f8160fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:06.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.535+0000 7f76f6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76f8070990 0x7f76f8160fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51128/0 (socket says 172.21.15.100:51128) 2024-09-17T04:32:06.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.535+0000 7f76f6800640 1 -- 172.21.15.100:0/2686826885 learned_addr learned my addr 172.21.15.100:0/2686826885 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:06.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.535+0000 7f76ed400640 1 --2- 172.21.15.100:0/2686826885 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76f81f8970 0x7f76f8070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:06.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.535+0000 7f76f5e00640 1 --2- 172.21.15.100:0/2686826885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76dc004d40 0x7f76f814edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:06.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.535+0000 7f76f6800640 1 -- 172.21.15.100:0/2686826885 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76f81f8970 msgr2=0x7f76f8070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:06.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.535+0000 7f76f6800640 1 --2- 172.21.15.100:0/2686826885 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76f81f8970 0x7f76f8070450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:06.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.535+0000 7f76f6800640 1 -- 172.21.15.100:0/2686826885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76dc004d40 msgr2=0x7f76f814edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:06.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.535+0000 7f76f6800640 1 --2- 172.21.15.100:0/2686826885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76dc004d40 0x7f76f814edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:06.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.535+0000 7f76f6800640 1 -- 172.21.15.100:0/2686826885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f8070da0 con 0x7f76f8070990 2024-09-17T04:32:06.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.535+0000 7f76f6800640 1 --2- 172.21.15.100:0/2686826885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76f8070990 0x7f76f8160fd0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f76f002b810 tx=0x7f76f0033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:06.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.536+0000 7f76eea00640 1 -- 172.21.15.100:0/2686826885 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f0042070 con 0x7f76f8070990 2024-09-17T04:32:06.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.536+0000 7f76eea00640 1 -- 172.21.15.100:0/2686826885 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76f0003970 con 0x7f76f8070990 2024-09-17T04:32:06.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.536+0000 7f76eea00640 1 -- 172.21.15.100:0/2686826885 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f0038040 con 0x7f76f8070990 2024-09-17T04:32:06.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.536+0000 7f76f7200640 1 -- 172.21.15.100:0/2686826885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f814f330 con 0x7f76f8070990 2024-09-17T04:32:06.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.536+0000 7f76f7200640 1 -- 172.21.15.100:0/2686826885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f8154a30 con 0x7f76f8070990 2024-09-17T04:32:06.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.537+0000 7f76f7200640 1 -- 172.21.15.100:0/2686826885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76f80b44f0 con 0x7f76f8070990 2024-09-17T04:32:06.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.538+0000 7f76eea00640 1 -- 172.21.15.100:0/2686826885 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f76f0004030 con 0x7f76f8070990 2024-09-17T04:32:06.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.539+0000 7f76eea00640 1 --2- 172.21.15.100:0/2686826885 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f76d40791c0 0x7f76d407b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:06.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.539+0000 7f76eea00640 1 -- 172.21.15.100:0/2686826885 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7f76f00bbec0 con 0x7f76f8070990 2024-09-17T04:32:06.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.539+0000 7f76f5e00640 1 --2- 172.21.15.100:0/2686826885 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f76d40791c0 0x7f76d407b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:06.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.540+0000 7f76f5e00640 1 --2- 172.21.15.100:0/2686826885 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f76d40791c0 0x7f76d407b6d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f76f8073780 tx=0x7f76e8005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:06.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.544+0000 7f76eea00640 1 -- 172.21.15.100:0/2686826885 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f76f8070da0 con 0x7f76f8070990 2024-09-17T04:32:06.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.754+0000 7f76f7200640 1 -- 172.21.15.100:0/2686826885 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f76f806d9b0 con 0x7f76d40791c0 2024-09-17T04:32:06.755 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:06.755+0000 7f30b3000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-17T04:32:06.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:06.755+0000 7f76eea00640 1 -- 172.21.15.100:0/2686826885 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f76f806d9b0 con 0x7f76d40791c0 2024-09-17T04:32:06.756 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-17T04:32:06.777 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T04:32:06.778 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath non_existent_subvolume subvolume_1 2024-09-17T04:32:07.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.005+0000 7fd9a1a00640 1 -- 172.21.15.100:0/426926093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd99c1604a0 msgr2=0x7fd99c0cca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:07.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.005+0000 7fd9a1a00640 1 --2- 172.21.15.100:0/426926093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd99c1604a0 0x7fd99c0cca10 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fd99c072040 tx=0x7fd98c02a560 comp rx=0 tx=0).stop 2024-09-17T04:32:07.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.006+0000 7fd9a1a00640 1 -- 172.21.15.100:0/426926093 shutdown_connections 2024-09-17T04:32:07.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.006+0000 7fd9a1a00640 1 --2- 172.21.15.100:0/426926093 >> v2:172.21.15.184:3300/0 conn(0x7fd99c0d0c90 0x7fd99c0d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:07.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.006+0000 7fd9a1a00640 1 --2- 172.21.15.100:0/426926093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd99c1604a0 0x7fd99c0cca10 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:07.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.006+0000 7fd9a1a00640 1 -- 172.21.15.100:0/426926093 >> 172.21.15.100:0/426926093 conn(0x7fd99c0b2400 msgr2=0x7fd99c0b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:07.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.006+0000 7fd9a1a00640 1 -- 172.21.15.100:0/426926093 shutdown_connections 2024-09-17T04:32:07.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.006+0000 7fd9a1a00640 1 -- 172.21.15.100:0/426926093 wait complete. 2024-09-17T04:32:07.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.008+0000 7fd9a1a00640 1 Processor -- start 2024-09-17T04:32:07.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.008+0000 7fd9a1a00640 1 -- start start 2024-09-17T04:32:07.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.009+0000 7fd9a1a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd99c0d0c90 0x7fd99c13dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:07.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.009+0000 7fd9a1a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd99c146110 0x7fd99c1f8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:07.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.009+0000 7fd9a1a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd99c1604a0 0x7fd99c1f90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:07.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.009+0000 7fd9a1a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c071400 con 0x7fd99c0d0c90 2024-09-17T04:32:07.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.009+0000 7fd9a1a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c071280 con 0x7fd99c146110 2024-09-17T04:32:07.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.009+0000 7fd9a1a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd99c071580 con 0x7fd99c1604a0 2024-09-17T04:32:07.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.010+0000 7fd99aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd99c0d0c90 0x7fd99c13dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:07.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.010+0000 7fd99aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd99c0d0c90 0x7fd99c13dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51138/0 (socket says 172.21.15.100:51138) 2024-09-17T04:32:07.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.010+0000 7fd99aa00640 1 -- 172.21.15.100:0/1022754578 learned_addr learned my addr 172.21.15.100:0/1022754578 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:07.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.010+0000 7fd99a000640 1 --2- 172.21.15.100:0/1022754578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd99c146110 0x7fd99c1f8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:07.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.010+0000 7fd99b400640 1 --2- 172.21.15.100:0/1022754578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd99c1604a0 0x7fd99c1f90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:07.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.010+0000 7fd99aa00640 1 -- 172.21.15.100:0/1022754578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd99c1604a0 msgr2=0x7fd99c1f90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:07.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.010+0000 7fd99aa00640 1 --2- 172.21.15.100:0/1022754578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd99c1604a0 0x7fd99c1f90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:07.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.010+0000 7fd99aa00640 1 -- 172.21.15.100:0/1022754578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd99c146110 msgr2=0x7fd99c1f8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:07.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.010+0000 7fd99aa00640 1 --2- 172.21.15.100:0/1022754578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd99c146110 0x7fd99c1f8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:07.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.010+0000 7fd99aa00640 1 -- 172.21.15.100:0/1022754578 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd99c13e2c0 con 0x7fd99c0d0c90 2024-09-17T04:32:07.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.010+0000 7fd99b400640 1 --2- 172.21.15.100:0/1022754578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd99c1604a0 0x7fd99c1f90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:07.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.010+0000 7fd99aa00640 1 --2- 172.21.15.100:0/1022754578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd99c0d0c90 0x7fd99c13dd80 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fd98c034040 tx=0x7fd98c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:07.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.011+0000 7fd992a00640 1 -- 172.21.15.100:0/1022754578 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c03e030 con 0x7fd99c0d0c90 2024-09-17T04:32:07.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.011+0000 7fd992a00640 1 -- 172.21.15.100:0/1022754578 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd98c003e00 con 0x7fd99c0d0c90 2024-09-17T04:32:07.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.011+0000 7fd992a00640 1 -- 172.21.15.100:0/1022754578 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c035040 con 0x7fd99c0d0c90 2024-09-17T04:32:07.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.011+0000 7fd9a1a00640 1 -- 172.21.15.100:0/1022754578 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd99c1f98e0 con 0x7fd99c0d0c90 2024-09-17T04:32:07.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.011+0000 7fd9a1a00640 1 -- 172.21.15.100:0/1022754578 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd99c1f6390 con 0x7fd99c0d0c90 2024-09-17T04:32:07.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.012+0000 7fd9a1a00640 1 -- 172.21.15.100:0/1022754578 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd99c071aa0 con 0x7fd99c0d0c90 2024-09-17T04:32:07.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.013+0000 7fd992a00640 1 -- 172.21.15.100:0/1022754578 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fd98c041030 con 0x7fd99c0d0c90 2024-09-17T04:32:07.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.014+0000 7fd992a00640 1 --2- 172.21.15.100:0/1022754578 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd97c0791c0 0x7fd97c07b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:07.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.014+0000 7fd992a00640 1 -- 172.21.15.100:0/1022754578 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5794+0+0 (secure 0 0 0) 0x7fd98c0bc080 con 0x7fd99c0d0c90 2024-09-17T04:32:07.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.014+0000 7fd99a000640 1 --2- 172.21.15.100:0/1022754578 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd97c0791c0 0x7fd97c07b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:07.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.015+0000 7fd99a000640 1 --2- 172.21.15.100:0/1022754578 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd97c0791c0 0x7fd97c07b6d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd99c1efaa0 tx=0x7fd988009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:07.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.019+0000 7fd992a00640 1 -- 172.21.15.100:0/1022754578 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd99c13e2c0 con 0x7fd99c0d0c90 2024-09-17T04:32:07.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.228+0000 7fd9a1a00640 1 -- 172.21.15.100:0/1022754578 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7fd99c0d1370 con 0x7fd97c0791c0 2024-09-17T04:32:07.229 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:07.229+0000 7f30b3000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-17T04:32:07.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.229+0000 7fd992a00640 1 -- 172.21.15.100:0/1022754578 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fd99c0d1370 con 0x7fd97c0791c0 2024-09-17T04:32:07.230 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-17T04:32:07.245 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T04:32:07.246 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info non_existent_subvolume subvolume_1 2024-09-17T04:32:07.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.468+0000 7f5333800640 1 -- 172.21.15.100:0/3372969622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f532c145550 msgr2=0x7f532c06fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:07.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.468+0000 7f5333800640 1 --2- 172.21.15.100:0/3372969622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f532c145550 0x7f532c06fb40 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f531c005fd0 tx=0x7f531c02a660 comp rx=0 tx=0).stop 2024-09-17T04:32:07.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.468+0000 7f5333800640 1 -- 172.21.15.100:0/3372969622 shutdown_connections 2024-09-17T04:32:07.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.468+0000 7f5333800640 1 --2- 172.21.15.100:0/3372969622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f532c145a40 0x7f532c073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:07.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.468+0000 7f5333800640 1 --2- 172.21.15.100:0/3372969622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f532c145550 0x7f532c06fb40 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:07.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.468+0000 7f5333800640 1 -- 172.21.15.100:0/3372969622 >> 172.21.15.100:0/3372969622 conn(0x7f532c06d1c0 msgr2=0x7f532c06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:07.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.468+0000 7f5333800640 1 -- 172.21.15.100:0/3372969622 shutdown_connections 2024-09-17T04:32:07.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.468+0000 7f5333800640 1 -- 172.21.15.100:0/3372969622 wait complete. 2024-09-17T04:32:07.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.470+0000 7f5333800640 1 Processor -- start 2024-09-17T04:32:07.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.471+0000 7f5333800640 1 -- start start 2024-09-17T04:32:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.472+0000 7f5333800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f532c073a80 0x7f532c15bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.472+0000 7f5333800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f532c145550 0x7f532c15de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.472+0000 7f5333800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f532c145a40 0x7f532c1f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.472+0000 7f5333800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f532c160a30 con 0x7f532c145550 2024-09-17T04:32:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.472+0000 7f5333800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f532c0ce940 con 0x7f532c145a40 2024-09-17T04:32:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.472+0000 7f5333800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f532c160bb0 con 0x7f532c073a80 2024-09-17T04:32:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.472+0000 7f5323e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f532c145550 0x7f532c15de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.472+0000 7f5323e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f532c145550 0x7f532c15de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51164/0 (socket says 172.21.15.100:51164) 2024-09-17T04:32:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.472+0000 7f5323e00640 1 -- 172.21.15.100:0/3690565484 learned_addr learned my addr 172.21.15.100:0/3690565484 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.472+0000 7f5330e00640 1 --2- 172.21.15.100:0/3690565484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f532c073a80 0x7f532c15bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:07.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.472+0000 7f5331800640 1 --2- 172.21.15.100:0/3690565484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f532c145a40 0x7f532c1f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:07.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.473+0000 7f5323e00640 1 -- 172.21.15.100:0/3690565484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f532c073a80 msgr2=0x7f532c15bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:07.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.473+0000 7f5323e00640 1 --2- 172.21.15.100:0/3690565484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f532c073a80 0x7f532c15bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:07.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.473+0000 7f5323e00640 1 -- 172.21.15.100:0/3690565484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f532c145a40 msgr2=0x7f532c1f3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:07.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.473+0000 7f5323e00640 1 --2- 172.21.15.100:0/3690565484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f532c145a40 0x7f532c1f3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:07.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.473+0000 7f5323e00640 1 -- 172.21.15.100:0/3690565484 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f532c15e360 con 0x7f532c145550 2024-09-17T04:32:07.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.473+0000 7f5323e00640 1 --2- 172.21.15.100:0/3690565484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f532c145550 0x7f532c15de20 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f531400d6b0 tx=0x7f531400db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:07.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.473+0000 7f5329600640 1 -- 172.21.15.100:0/3690565484 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5314017070 con 0x7f532c145550 2024-09-17T04:32:07.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.473+0000 7f5329600640 1 -- 172.21.15.100:0/3690565484 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5314005310 con 0x7f532c145550 2024-09-17T04:32:07.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.473+0000 7f5333800640 1 -- 172.21.15.100:0/3690565484 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f532c15e870 con 0x7f532c145550 2024-09-17T04:32:07.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.473+0000 7f5329600640 1 -- 172.21.15.100:0/3690565484 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53140097c0 con 0x7f532c145550 2024-09-17T04:32:07.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.474+0000 7f5333800640 1 -- 172.21.15.100:0/3690565484 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f532c15ee30 con 0x7f532c145550 2024-09-17T04:32:07.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.475+0000 7f5333800640 1 -- 172.21.15.100:0/3690565484 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f532c15e5e0 con 0x7f532c145550 2024-09-17T04:32:07.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.476+0000 7f5329600640 1 -- 172.21.15.100:0/3690565484 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f5314009960 con 0x7f532c145550 2024-09-17T04:32:07.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.477+0000 7f5329600640 1 --2- 172.21.15.100:0/3690565484 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f53080791f0 0x7f530807b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:07.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.477+0000 7f5329600640 1 -- 172.21.15.100:0/3690565484 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5794+0+0 (secure 0 0 0) 0x7f531409b8f0 con 0x7f532c145550 2024-09-17T04:32:07.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.477+0000 7f5330e00640 1 --2- 172.21.15.100:0/3690565484 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f53080791f0 0x7f530807b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:07.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.478+0000 7f5330e00640 1 --2- 172.21.15.100:0/3690565484 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f53080791f0 0x7f530807b700 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f531c033000 tx=0x7f531c004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:07.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.481+0000 7f5329600640 1 -- 172.21.15.100:0/3690565484 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f5314062c80 con 0x7f532c145550 2024-09-17T04:32:07.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.692+0000 7f5333800640 1 -- 172.21.15.100:0/3690565484 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f532c06a7f0 con 0x7f53080791f0 2024-09-17T04:32:07.694 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:07.694+0000 7f30b3000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-17T04:32:07.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.694+0000 7f5329600640 1 -- 172.21.15.100:0/3690565484 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f532c06a7f0 con 0x7f53080791f0 2024-09-17T04:32:07.694 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-17T04:32:07.713 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T04:32:07.714 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize non_existent_subvolume subvolname_1 inf 2024-09-17T04:32:07.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.942+0000 7fde8f200640 1 -- 172.21.15.100:0/613479301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde74004800 msgr2=0x7fde74004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:07.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.942+0000 7fde8f200640 1 --2- 172.21.15.100:0/613479301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde74004800 0x7fde74004c90 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fde900b4f10 tx=0x7fde74005830 comp rx=0 tx=0).stop 2024-09-17T04:32:07.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.942+0000 7fde8e800640 1 -- 172.21.15.100:0/613479301 reap_dead start 2024-09-17T04:32:07.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.942+0000 7fde8f200640 1 -- 172.21.15.100:0/613479301 shutdown_connections 2024-09-17T04:32:07.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.942+0000 7fde8f200640 1 -- 172.21.15.100:0/613479301 >> 172.21.15.100:0/613479301 conn(0x7fde9006d190 msgr2=0x7fde9006d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:07.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.942+0000 7fde8f200640 1 -- 172.21.15.100:0/613479301 shutdown_connections 2024-09-17T04:32:07.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.942+0000 7fde8f200640 1 -- 172.21.15.100:0/613479301 wait complete. 2024-09-17T04:32:07.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.945+0000 7fde8f200640 1 Processor -- start 2024-09-17T04:32:07.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.945+0000 7fde8f200640 1 -- start start 2024-09-17T04:32:07.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.946+0000 7fde8f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde74004800 0x7fde9015ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:07.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.946+0000 7fde8f200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde901f5760 0x7fde9015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:07.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.946+0000 7fde8f200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde9015db10 0x7fde9014d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:07.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.946+0000 7fde8f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fde9015e870 con 0x7fde74004800 2024-09-17T04:32:07.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.946+0000 7fde8f200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fde900b4b90 con 0x7fde901f5760 2024-09-17T04:32:07.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.946+0000 7fde8f200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fde9015e9f0 con 0x7fde9015db10 2024-09-17T04:32:07.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.946+0000 7fde8de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde74004800 0x7fde9015ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:07.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.946+0000 7fde8de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde74004800 0x7fde9015ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51186/0 (socket says 172.21.15.100:51186) 2024-09-17T04:32:07.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.946+0000 7fde8de00640 1 -- 172.21.15.100:0/293095974 learned_addr learned my addr 172.21.15.100:0/293095974 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:07.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.947+0000 7fde85400640 1 --2- 172.21.15.100:0/293095974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde901f5760 0x7fde9015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:07.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.947+0000 7fde8e800640 1 --2- 172.21.15.100:0/293095974 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde9015db10 0x7fde9014d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:07.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.947+0000 7fde8de00640 1 -- 172.21.15.100:0/293095974 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde9015db10 msgr2=0x7fde9014d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:07.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.947+0000 7fde8de00640 1 --2- 172.21.15.100:0/293095974 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde9015db10 0x7fde9014d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:07.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.947+0000 7fde8de00640 1 -- 172.21.15.100:0/293095974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde901f5760 msgr2=0x7fde9015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:07.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.947+0000 7fde8de00640 1 --2- 172.21.15.100:0/293095974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde901f5760 0x7fde9015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:07.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.947+0000 7fde8de00640 1 -- 172.21.15.100:0/293095974 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde90158b20 con 0x7fde74004800 2024-09-17T04:32:07.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.947+0000 7fde85400640 1 --2- 172.21.15.100:0/293095974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde901f5760 0x7fde9015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:07.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.947+0000 7fde8de00640 1 --2- 172.21.15.100:0/293095974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde74004800 0x7fde9015ed50 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fde80003db0 tx=0x7fde80001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:07.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.947+0000 7fde86a00640 1 -- 172.21.15.100:0/293095974 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde80011430 con 0x7fde74004800 2024-09-17T04:32:07.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.947+0000 7fde86a00640 1 -- 172.21.15.100:0/293095974 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde80016020 con 0x7fde74004800 2024-09-17T04:32:07.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.947+0000 7fde8f200640 1 -- 172.21.15.100:0/293095974 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde900b65f0 con 0x7fde74004800 2024-09-17T04:32:07.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.947+0000 7fde86a00640 1 -- 172.21.15.100:0/293095974 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde80011890 con 0x7fde74004800 2024-09-17T04:32:07.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.948+0000 7fde8f200640 1 -- 172.21.15.100:0/293095974 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde900b6bb0 con 0x7fde74004800 2024-09-17T04:32:07.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.949+0000 7fde8f200640 1 -- 172.21.15.100:0/293095974 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde9006edd0 con 0x7fde74004800 2024-09-17T04:32:07.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.950+0000 7fde86a00640 1 -- 172.21.15.100:0/293095974 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fde80011a30 con 0x7fde74004800 2024-09-17T04:32:07.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.951+0000 7fde86a00640 1 --2- 172.21.15.100:0/293095974 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fde6c0791c0 0x7fde6c07b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:07.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.951+0000 7fde86a00640 1 -- 172.21.15.100:0/293095974 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5794+0+0 (secure 0 0 0) 0x7fde8009c300 con 0x7fde74004800 2024-09-17T04:32:07.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.951+0000 7fde85400640 1 --2- 172.21.15.100:0/293095974 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fde6c0791c0 0x7fde6c07b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:07.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.952+0000 7fde85400640 1 --2- 172.21.15.100:0/293095974 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fde6c0791c0 0x7fde6c07b6d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fde78035000 tx=0x7fde780046c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:07.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:07.955+0000 7fde86a00640 1 -- 172.21.15.100:0/293095974 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fde80020030 con 0x7fde74004800 2024-09-17T04:32:08.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.171+0000 7fde8f200640 1 -- 172.21.15.100:0/293095974 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "non_existent_subvolume", "sub_name": "subvolname_1", "new_size": "inf", "target": ["mon-mgr", ""]}) -- 0x7fde9006f8b0 con 0x7fde6c0791c0 2024-09-17T04:32:08.172 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:08.172+0000 7f30b3000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-17T04:32:08.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.172+0000 7fde86a00640 1 -- 172.21.15.100:0/293095974 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fde9006f8b0 con 0x7fde6c0791c0 2024-09-17T04:32:08.173 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-17T04:32:08.189 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T04:32:08.190 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume pin non_existent_subvolume subvolname_1 export 1 2024-09-17T04:32:08.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.415+0000 7f32dfc00640 1 -- 172.21.15.100:0/1454848004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32d80cc830 msgr2=0x7f32d80b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:08.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.415+0000 7f32dfc00640 1 --2- 172.21.15.100:0/1454848004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32d80cc830 0x7f32d80b4620 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f32c8005fd0 tx=0x7f32c802a5d0 comp rx=0 tx=0).stop 2024-09-17T04:32:08.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.415+0000 7f32dfc00640 1 -- 172.21.15.100:0/1454848004 shutdown_connections 2024-09-17T04:32:08.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.415+0000 7f32dfc00640 1 --2- 172.21.15.100:0/1454848004 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32d80af100 0x7f32d80af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:08.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.415+0000 7f32dfc00640 1 --2- 172.21.15.100:0/1454848004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32d80cc830 0x7f32d80b4620 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:08.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.415+0000 7f32dfc00640 1 -- 172.21.15.100:0/1454848004 >> 172.21.15.100:0/1454848004 conn(0x7f32d80b2540 msgr2=0x7f32d80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:08.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.415+0000 7f32dfc00640 1 -- 172.21.15.100:0/1454848004 shutdown_connections 2024-09-17T04:32:08.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.415+0000 7f32dfc00640 1 -- 172.21.15.100:0/1454848004 wait complete. 2024-09-17T04:32:08.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.417+0000 7f32dfc00640 1 Processor -- start 2024-09-17T04:32:08.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.417+0000 7f32dfc00640 1 -- start start 2024-09-17T04:32:08.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.419+0000 7f32dfc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32d806cda0 0x7f32d80c31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:08.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.419+0000 7f32dfc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32d80af100 0x7f32d809a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:08.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.419+0000 7f32dfc00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32d80cc830 0x7f32d809a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:08.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.419+0000 7f32dfc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f32d80b36a0 con 0x7f32d806cda0 2024-09-17T04:32:08.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.419+0000 7f32dfc00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f32d80b3520 con 0x7f32d80af100 2024-09-17T04:32:08.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.419+0000 7f32dfc00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f32d80b3820 con 0x7f32d80cc830 2024-09-17T04:32:08.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.419+0000 7f32dd200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32d806cda0 0x7f32d80c31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:08.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.419+0000 7f32dd200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32d806cda0 0x7f32d80c31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51202/0 (socket says 172.21.15.100:51202) 2024-09-17T04:32:08.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.419+0000 7f32dd200640 1 -- 172.21.15.100:0/3211695909 learned_addr learned my addr 172.21.15.100:0/3211695909 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:08.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.419+0000 7f32ddc00640 1 --2- 172.21.15.100:0/3211695909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32d80cc830 0x7f32d809a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:08.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.419+0000 7f32cfe00640 1 --2- 172.21.15.100:0/3211695909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32d80af100 0x7f32d809a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:08.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.419+0000 7f32dd200640 1 -- 172.21.15.100:0/3211695909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32d80cc830 msgr2=0x7f32d809a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:08.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.419+0000 7f32dd200640 1 --2- 172.21.15.100:0/3211695909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32d80cc830 0x7f32d809a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:08.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.419+0000 7f32dd200640 1 -- 172.21.15.100:0/3211695909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32d80af100 msgr2=0x7f32d809a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:08.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.419+0000 7f32dd200640 1 --2- 172.21.15.100:0/3211695909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32d80af100 0x7f32d809a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:08.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.419+0000 7f32dd200640 1 -- 172.21.15.100:0/3211695909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d809e240 con 0x7f32d806cda0 2024-09-17T04:32:08.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.420+0000 7f32dd200640 1 --2- 172.21.15.100:0/3211695909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32d806cda0 0x7f32d80c31c0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f32c8034040 tx=0x7f32c80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:08.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.420+0000 7f32d5600640 1 -- 172.21.15.100:0/3211695909 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c803e030 con 0x7f32d806cda0 2024-09-17T04:32:08.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.420+0000 7f32d5600640 1 -- 172.21.15.100:0/3211695909 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32c8003ca0 con 0x7f32d806cda0 2024-09-17T04:32:08.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.420+0000 7f32dfc00640 1 -- 172.21.15.100:0/3211695909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32d809af30 con 0x7f32d806cda0 2024-09-17T04:32:08.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.420+0000 7f32d5600640 1 -- 172.21.15.100:0/3211695909 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c8035040 con 0x7f32d806cda0 2024-09-17T04:32:08.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.420+0000 7f32dfc00640 1 -- 172.21.15.100:0/3211695909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32d809b4f0 con 0x7f32d806cda0 2024-09-17T04:32:08.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.421+0000 7f32dfc00640 1 -- 172.21.15.100:0/3211695909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32d80b3d40 con 0x7f32d806cda0 2024-09-17T04:32:08.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.423+0000 7f32d5600640 1 -- 172.21.15.100:0/3211695909 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f32c8041030 con 0x7f32d806cda0 2024-09-17T04:32:08.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.423+0000 7f32d5600640 1 --2- 172.21.15.100:0/3211695909 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f32b40791c0 0x7f32b407b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:08.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.424+0000 7f32d5600640 1 -- 172.21.15.100:0/3211695909 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5794+0+0 (secure 0 0 0) 0x7f32c80bbdf0 con 0x7f32d806cda0 2024-09-17T04:32:08.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.424+0000 7f32cfe00640 1 --2- 172.21.15.100:0/3211695909 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f32b40791c0 0x7f32b407b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:08.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.424+0000 7f32cfe00640 1 --2- 172.21.15.100:0/3211695909 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f32b40791c0 0x7f32b407b6d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f32d80ae7b0 tx=0x7f32c0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:08.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.428+0000 7f32d5600640 1 -- 172.21.15.100:0/3211695909 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f32c8083330 con 0x7f32d806cda0 2024-09-17T04:32:08.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.644+0000 7f32dfc00640 1 -- 172.21.15.100:0/3211695909 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume pin", "vol_name": "non_existent_subvolume", "sub_name": "subvolname_1", "pin_type": "export", "pin_setting": "1", "target": ["mon-mgr", ""]}) -- 0x7f32d8098e00 con 0x7f32b40791c0 2024-09-17T04:32:08.645 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:08.645+0000 7f30b3000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-17T04:32:08.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.645+0000 7f32d5600640 1 -- 172.21.15.100:0/3211695909 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f32d8098e00 con 0x7f32b40791c0 2024-09-17T04:32:08.646 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-17T04:32:08.661 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T04:32:08.662 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls non_existent_subvolume 2024-09-17T04:32:08.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.888+0000 7fc73aa00640 1 -- 172.21.15.100:0/749799208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc718003c30 msgr2=0x7fc718004080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:08.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.888+0000 7fc73aa00640 1 --2- 172.21.15.100:0/749799208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc718003c30 0x7fc718004080 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fc72400c740 tx=0x7fc718005830 comp rx=0 tx=0).stop 2024-09-17T04:32:08.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.888+0000 7fc73aa00640 1 -- 172.21.15.100:0/749799208 shutdown_connections 2024-09-17T04:32:08.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.889+0000 7fc73aa00640 1 --2- 172.21.15.100:0/749799208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc718003c30 0x7fc718004080 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:08.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.889+0000 7fc73aa00640 1 -- 172.21.15.100:0/749799208 >> 172.21.15.100:0/749799208 conn(0x7fc73406d1c0 msgr2=0x7fc73406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:08.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.889+0000 7fc73aa00640 1 -- 172.21.15.100:0/749799208 shutdown_connections 2024-09-17T04:32:08.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.889+0000 7fc73aa00640 1 -- 172.21.15.100:0/749799208 wait complete. 2024-09-17T04:32:08.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.891+0000 7fc73aa00640 1 Processor -- start 2024-09-17T04:32:08.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.891+0000 7fc73aa00640 1 -- start start 2024-09-17T04:32:08.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.892+0000 7fc73aa00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc718003c30 0x7fc7340c5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:08.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.892+0000 7fc73aa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7341f5c00 0x7fc7341f6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:08.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.892+0000 7fc73aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc734144240 0x7fc734144650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:08.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.892+0000 7fc73aa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc73406fcc0 con 0x7fc734144240 2024-09-17T04:32:08.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.892+0000 7fc73aa00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc73406fb40 con 0x7fc7341f5c00 2024-09-17T04:32:08.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.892+0000 7fc73aa00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc73406fe40 con 0x7fc718003c30 2024-09-17T04:32:08.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.893+0000 7fc738a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc734144240 0x7fc734144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:08.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.893+0000 7fc738a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc734144240 0x7fc734144650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51226/0 (socket says 172.21.15.100:51226) 2024-09-17T04:32:08.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.893+0000 7fc738a00640 1 -- 172.21.15.100:0/3454847655 learned_addr learned my addr 172.21.15.100:0/3454847655 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:08.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.893+0000 7fc72b400640 1 --2- 172.21.15.100:0/3454847655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7341f5c00 0x7fc7341f6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:08.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.893+0000 7fc733e00640 1 --2- 172.21.15.100:0/3454847655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc718003c30 0x7fc7340c5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:08.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.893+0000 7fc738a00640 1 -- 172.21.15.100:0/3454847655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc718003c30 msgr2=0x7fc7340c5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:08.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.893+0000 7fc738a00640 1 --2- 172.21.15.100:0/3454847655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc718003c30 0x7fc7340c5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:08.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.893+0000 7fc738a00640 1 -- 172.21.15.100:0/3454847655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7341f5c00 msgr2=0x7fc7341f6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:08.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.893+0000 7fc738a00640 1 --2- 172.21.15.100:0/3454847655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7341f5c00 0x7fc7341f6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:08.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.893+0000 7fc738a00640 1 -- 172.21.15.100:0/3454847655 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc734144b90 con 0x7fc734144240 2024-09-17T04:32:08.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.893+0000 7fc72b400640 1 --2- 172.21.15.100:0/3454847655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7341f5c00 0x7fc7341f6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:08.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.893+0000 7fc738a00640 1 --2- 172.21.15.100:0/3454847655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc734144240 0x7fc734144650 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fc7340730d0 tx=0x7fc72c001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:08.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.894+0000 7fc730c00640 1 -- 172.21.15.100:0/3454847655 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c011840 con 0x7fc734144240 2024-09-17T04:32:08.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.894+0000 7fc730c00640 1 -- 172.21.15.100:0/3454847655 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc72c018070 con 0x7fc734144240 2024-09-17T04:32:08.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.894+0000 7fc73aa00640 1 -- 172.21.15.100:0/3454847655 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7340adc00 con 0x7fc734144240 2024-09-17T04:32:08.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.894+0000 7fc730c00640 1 -- 172.21.15.100:0/3454847655 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c011c60 con 0x7fc734144240 2024-09-17T04:32:08.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.894+0000 7fc73aa00640 1 -- 172.21.15.100:0/3454847655 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7340ae1c0 con 0x7fc734144240 2024-09-17T04:32:08.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.895+0000 7fc73aa00640 1 -- 172.21.15.100:0/3454847655 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc734160320 con 0x7fc734144240 2024-09-17T04:32:08.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.896+0000 7fc730c00640 1 -- 172.21.15.100:0/3454847655 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fc72c01d460 con 0x7fc734144240 2024-09-17T04:32:08.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.897+0000 7fc730c00640 1 --2- 172.21.15.100:0/3454847655 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc7100791f0 0x7fc71007b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:08.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.897+0000 7fc730c00640 1 -- 172.21.15.100:0/3454847655 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5794+0+0 (secure 0 0 0) 0x7fc72c09bfd0 con 0x7fc734144240 2024-09-17T04:32:08.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.897+0000 7fc733e00640 1 --2- 172.21.15.100:0/3454847655 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc7100791f0 0x7fc71007b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:08.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.898+0000 7fc733e00640 1 --2- 172.21.15.100:0/3454847655 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc7100791f0 0x7fc71007b700 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc724033000 tx=0x7fc724038040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:08.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:08.901+0000 7fc730c00640 1 -- 172.21.15.100:0/3454847655 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc734144b90 con 0x7fc734144240 2024-09-17T04:32:09.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.112+0000 7fc73aa00640 1 -- 172.21.15.100:0/3454847655 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "non_existent_subvolume", "target": ["mon-mgr", ""]}) -- 0x7fc7340aeca0 con 0x7fc7100791f0 2024-09-17T04:32:09.113 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:09.113+0000 7f30b3000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-17T04:32:09.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.113+0000 7fc730c00640 1 -- 172.21.15.100:0/3454847655 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fc724001c10 con 0x7fc7100791f0 2024-09-17T04:32:09.114 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-17T04:32:09.133 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T04:32:09.133 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create non_existent_subvolume subvolume_1 snapshot_1 2024-09-17T04:32:09.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.357+0000 7f8572a00640 1 -- 172.21.15.100:0/3294891636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f856c0b3ec0 msgr2=0x7f856c0b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:09.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.357+0000 7f8572a00640 1 --2- 172.21.15.100:0/3294891636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f856c0b3ec0 0x7f856c0b47c0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f856c06e750 tx=0x7f856002a4d0 comp rx=0 tx=0).stop 2024-09-17T04:32:09.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.357+0000 7f8572a00640 1 -- 172.21.15.100:0/3294891636 shutdown_connections 2024-09-17T04:32:09.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.357+0000 7f8572a00640 1 --2- 172.21.15.100:0/3294891636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f856c1532e0 0x7f856c1536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:09.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.357+0000 7f8572a00640 1 --2- 172.21.15.100:0/3294891636 >> v2:172.21.15.184:3300/0 conn(0x7f856c0705b0 0x7f856c0af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:09.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.357+0000 7f8572a00640 1 --2- 172.21.15.100:0/3294891636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f856c0b3ec0 0x7f856c0b47c0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:09.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.357+0000 7f8572a00640 1 -- 172.21.15.100:0/3294891636 >> 172.21.15.100:0/3294891636 conn(0x7f856c0b1dd0 msgr2=0x7f856c0b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:09.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.357+0000 7f8572a00640 1 -- 172.21.15.100:0/3294891636 shutdown_connections 2024-09-17T04:32:09.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.357+0000 7f8572a00640 1 -- 172.21.15.100:0/3294891636 wait complete. 2024-09-17T04:32:09.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.360+0000 7f8572a00640 1 Processor -- start 2024-09-17T04:32:09.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.360+0000 7f8572a00640 1 -- start start 2024-09-17T04:32:09.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.361+0000 7f8572a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f856c0705b0 0x7f856c1f8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:09.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.361+0000 7f8572a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f856c0b3ec0 0x7f856c155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:09.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.361+0000 7f8572a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f856c1532e0 0x7f856c148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:09.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.361+0000 7f8572a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f856c160860 con 0x7f856c0b3ec0 2024-09-17T04:32:09.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.361+0000 7f8572a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f856c1606e0 con 0x7f856c0705b0 2024-09-17T04:32:09.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.361+0000 7f8572a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f856c1609e0 con 0x7f856c1532e0 2024-09-17T04:32:09.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.361+0000 7f8570c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f856c0b3ec0 0x7f856c155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:09.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.361+0000 7f8570c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f856c0b3ec0 0x7f856c155400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51256/0 (socket says 172.21.15.100:51256) 2024-09-17T04:32:09.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.361+0000 7f8570c00640 1 -- 172.21.15.100:0/1420446456 learned_addr learned my addr 172.21.15.100:0/1420446456 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:09.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.362+0000 7f8571600640 1 --2- 172.21.15.100:0/1420446456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f856c0705b0 0x7f856c1f8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:09.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.362+0000 7f8570c00640 1 -- 172.21.15.100:0/1420446456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f856c1532e0 msgr2=0x7f856c148710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:09.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.362+0000 7f8572000640 1 --2- 172.21.15.100:0/1420446456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f856c1532e0 0x7f856c148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:09.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.362+0000 7f8570c00640 1 --2- 172.21.15.100:0/1420446456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f856c1532e0 0x7f856c148710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:09.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.362+0000 7f8570c00640 1 -- 172.21.15.100:0/1420446456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f856c0705b0 msgr2=0x7f856c1f8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:09.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.362+0000 7f8570c00640 1 --2- 172.21.15.100:0/1420446456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f856c0705b0 0x7f856c1f8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:09.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.362+0000 7f8570c00640 1 -- 172.21.15.100:0/1420446456 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856c1f9650 con 0x7f856c0b3ec0 2024-09-17T04:32:09.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.362+0000 7f8571600640 1 --2- 172.21.15.100:0/1420446456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f856c0705b0 0x7f856c1f8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:32:09.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.362+0000 7f8572000640 1 --2- 172.21.15.100:0/1420446456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f856c1532e0 0x7f856c148710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:32:09.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.362+0000 7f8570c00640 1 --2- 172.21.15.100:0/1420446456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f856c0b3ec0 0x7f856c155400 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f855800c940 tx=0x7f855800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:09.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.362+0000 7f8569600640 1 -- 172.21.15.100:0/1420446456 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8558007bb0 con 0x7f856c0b3ec0 2024-09-17T04:32:09.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.362+0000 7f8569600640 1 -- 172.21.15.100:0/1420446456 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8558007e40 con 0x7f856c0b3ec0 2024-09-17T04:32:09.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.363+0000 7f8569600640 1 -- 172.21.15.100:0/1420446456 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855800f670 con 0x7f856c0b3ec0 2024-09-17T04:32:09.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.363+0000 7f8572a00640 1 -- 172.21.15.100:0/1420446456 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f856c1f9930 con 0x7f856c0b3ec0 2024-09-17T04:32:09.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.363+0000 7f8572a00640 1 -- 172.21.15.100:0/1420446456 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f856c09bdf0 con 0x7f856c0b3ec0 2024-09-17T04:32:09.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.364+0000 7f8572a00640 1 -- 172.21.15.100:0/1420446456 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f856c1f93c0 con 0x7f856c0b3ec0 2024-09-17T04:32:09.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.365+0000 7f8569600640 1 -- 172.21.15.100:0/1420446456 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f855800f810 con 0x7f856c0b3ec0 2024-09-17T04:32:09.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.366+0000 7f8569600640 1 --2- 172.21.15.100:0/1420446456 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f854c0791c0 0x7f854c07b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:09.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.366+0000 7f8569600640 1 -- 172.21.15.100:0/1420446456 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5794+0+0 (secure 0 0 0) 0x7f855809c120 con 0x7f856c0b3ec0 2024-09-17T04:32:09.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.366+0000 7f8571600640 1 --2- 172.21.15.100:0/1420446456 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f854c0791c0 0x7f854c07b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:09.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.367+0000 7f8571600640 1 --2- 172.21.15.100:0/1420446456 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f854c0791c0 0x7f854c07b6d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8560033000 tx=0x7f8560004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:09.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.370+0000 7f8569600640 1 -- 172.21.15.100:0/1420446456 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8558014030 con 0x7f856c0b3ec0 2024-09-17T04:32:09.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.582+0000 7f8572a00640 1 -- 172.21.15.100:0/1420446456 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f856c0b47c0 con 0x7f854c0791c0 2024-09-17T04:32:09.583 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:09.583+0000 7f30b3000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-17T04:32:09.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.584+0000 7f8569600640 1 -- 172.21.15.100:0/1420446456 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f856c0b47c0 con 0x7f854c0791c0 2024-09-17T04:32:09.584 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-17T04:32:09.603 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T04:32:09.604 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm non_existent_subvolume subvolume_1 snapshot_1 2024-09-17T04:32:09.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.833+0000 7f0063200640 1 -- 172.21.15.100:0/3822667957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0064163230 msgr2=0x7f0064163640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:09.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.833+0000 7f0063200640 1 --2- 172.21.15.100:0/3822667957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0064163230 0x7f0064163640 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f006408e7a0 tx=0x7f005c02a980 comp rx=0 tx=0).stop 2024-09-17T04:32:09.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.834+0000 7f0063200640 1 -- 172.21.15.100:0/3822667957 shutdown_connections 2024-09-17T04:32:09.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.834+0000 7f0063200640 1 --2- 172.21.15.100:0/3822667957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0064163230 0x7f0064163640 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:09.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.834+0000 7f0063200640 1 --2- 172.21.15.100:0/3822667957 >> v2:172.21.15.184:3300/0 conn(0x7f00640d2f30 0x7f0064162cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:09.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.834+0000 7f0063200640 1 --2- 172.21.15.100:0/3822667957 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00640b59b0 0x7f00640d29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:09.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.834+0000 7f0063200640 1 -- 172.21.15.100:0/3822667957 >> 172.21.15.100:0/3822667957 conn(0x7f006408bfe0 msgr2=0x7f00640d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:09.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.834+0000 7f0063200640 1 -- 172.21.15.100:0/3822667957 shutdown_connections 2024-09-17T04:32:09.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.834+0000 7f0063200640 1 -- 172.21.15.100:0/3822667957 wait complete. 2024-09-17T04:32:09.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.836+0000 7f0063200640 1 Processor -- start 2024-09-17T04:32:09.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.836+0000 7f0063200640 1 -- start start 2024-09-17T04:32:09.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.837+0000 7f0063200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00640b59b0 0x7f006414e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:09.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.837+0000 7f0063200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00640d2f30 0x7f00640cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:09.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.838+0000 7f0063200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0064163230 0x7f0064150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:09.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.838+0000 7f0063200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f00640cefb0 con 0x7f00640d2f30 2024-09-17T04:32:09.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.838+0000 7f0063200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f00640cee30 con 0x7f0064163230 2024-09-17T04:32:09.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.838+0000 7f0063200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f00640cf130 con 0x7f00640b59b0 2024-09-17T04:32:09.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.838+0000 7f005be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00640d2f30 0x7f00640cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:09.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.838+0000 7f005be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00640d2f30 0x7f00640cb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51282/0 (socket says 172.21.15.100:51282) 2024-09-17T04:32:09.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.838+0000 7f005be00640 1 -- 172.21.15.100:0/2156883551 learned_addr learned my addr 172.21.15.100:0/2156883551 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:09.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.838+0000 7f0061e00640 1 --2- 172.21.15.100:0/2156883551 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00640b59b0 0x7f006414e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:09.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.838+0000 7f0062800640 1 --2- 172.21.15.100:0/2156883551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0064163230 0x7f0064150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:09.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.838+0000 7f005be00640 1 -- 172.21.15.100:0/2156883551 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00640b59b0 msgr2=0x7f006414e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:09.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.838+0000 7f005be00640 1 --2- 172.21.15.100:0/2156883551 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00640b59b0 0x7f006414e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:09.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.838+0000 7f005be00640 1 -- 172.21.15.100:0/2156883551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0064163230 msgr2=0x7f0064150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:09.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.838+0000 7f005be00640 1 --2- 172.21.15.100:0/2156883551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0064163230 0x7f0064150eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:09.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.838+0000 7f005be00640 1 -- 172.21.15.100:0/2156883551 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00640cdf50 con 0x7f00640d2f30 2024-09-17T04:32:09.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.839+0000 7f005be00640 1 --2- 172.21.15.100:0/2156883551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00640d2f30 0x7f00640cb560 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f004c00d750 tx=0x7f004c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:09.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.839+0000 7f005a000640 1 -- 172.21.15.100:0/2156883551 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c017070 con 0x7f00640d2f30 2024-09-17T04:32:09.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.839+0000 7f005a000640 1 -- 172.21.15.100:0/2156883551 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f004c004310 con 0x7f00640d2f30 2024-09-17T04:32:09.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.839+0000 7f0063200640 1 -- 172.21.15.100:0/2156883551 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00640ce230 con 0x7f00640d2f30 2024-09-17T04:32:09.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.839+0000 7f005a000640 1 -- 172.21.15.100:0/2156883551 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c009c30 con 0x7f00640d2f30 2024-09-17T04:32:09.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.839+0000 7f0063200640 1 -- 172.21.15.100:0/2156883551 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00640ce7f0 con 0x7f00640d2f30 2024-09-17T04:32:09.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.840+0000 7f0063200640 1 -- 172.21.15.100:0/2156883551 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0064162cf0 con 0x7f00640d2f30 2024-09-17T04:32:09.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.841+0000 7f005a000640 1 -- 172.21.15.100:0/2156883551 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f004c005020 con 0x7f00640d2f30 2024-09-17T04:32:09.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.842+0000 7f005a000640 1 --2- 172.21.15.100:0/2156883551 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f00400791c0 0x7f004007b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:09.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.842+0000 7f005a000640 1 -- 172.21.15.100:0/2156883551 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5794+0+0 (secure 0 0 0) 0x7f004c09bdb0 con 0x7f00640d2f30 2024-09-17T04:32:09.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.842+0000 7f0061e00640 1 --2- 172.21.15.100:0/2156883551 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f00400791c0 0x7f004007b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:09.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.843+0000 7f0061e00640 1 --2- 172.21.15.100:0/2156883551 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f00400791c0 0x7f004007b6d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0054009a40 tx=0x7f0054005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:09.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:09.847+0000 7f005a000640 1 -- 172.21.15.100:0/2156883551 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f004c020020 con 0x7f00640d2f30 2024-09-17T04:32:10.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.062+0000 7f0063200640 1 -- 172.21.15.100:0/2156883551 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f00640cdd00 con 0x7f00400791c0 2024-09-17T04:32:10.062 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:10.063+0000 7f30b3000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-17T04:32:10.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.063+0000 7f005a000640 1 -- 172.21.15.100:0/2156883551 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f00640cdd00 con 0x7f00400791c0 2024-09-17T04:32:10.063 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-17T04:32:10.079 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T04:32:10.080 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info non_existent_subvolume subvolume_1 snapshot_1 2024-09-17T04:32:10.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.305+0000 7fe5ff200640 1 -- 172.21.15.100:0/3265393689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6001532e0 msgr2=0x7fe6001536f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:10.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.305+0000 7fe5ff200640 1 --2- 172.21.15.100:0/3265393689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6001532e0 0x7fe6001536f0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fe60006a110 tx=0x7fe5f802a980 comp rx=0 tx=0).stop 2024-09-17T04:32:10.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.305+0000 7fe5ff200640 1 -- 172.21.15.100:0/3265393689 shutdown_connections 2024-09-17T04:32:10.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.305+0000 7fe5ff200640 1 --2- 172.21.15.100:0/3265393689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6001532e0 0x7fe6001536f0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:10.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.305+0000 7fe5ff200640 1 --2- 172.21.15.100:0/3265393689 >> v2:172.21.15.184:3300/0 conn(0x7fe6000705b0 0x7fe6000af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:10.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.305+0000 7fe5ff200640 1 --2- 172.21.15.100:0/3265393689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6000b3ec0 0x7fe6000b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:10.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.305+0000 7fe5ff200640 1 -- 172.21.15.100:0/3265393689 >> 172.21.15.100:0/3265393689 conn(0x7fe6000b1dd0 msgr2=0x7fe6000b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:10.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.306+0000 7fe5ff200640 1 -- 172.21.15.100:0/3265393689 shutdown_connections 2024-09-17T04:32:10.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.306+0000 7fe5ff200640 1 -- 172.21.15.100:0/3265393689 wait complete. 2024-09-17T04:32:10.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.308+0000 7fe5ff200640 1 Processor -- start 2024-09-17T04:32:10.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.308+0000 7fe5ff200640 1 -- start start 2024-09-17T04:32:10.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.309+0000 7fe5ff200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6000705b0 0x7fe60015b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:10.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.309+0000 7fe5ff200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6000b3ec0 0x7fe60009c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:10.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.309+0000 7fe5ff200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6001532e0 0x7fe600147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:10.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.309+0000 7fe5ff200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe600160860 con 0x7fe6000705b0 2024-09-17T04:32:10.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.309+0000 7fe5ff200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe6001606e0 con 0x7fe6001532e0 2024-09-17T04:32:10.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.309+0000 7fe5ff200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe6001609e0 con 0x7fe6000b3ec0 2024-09-17T04:32:10.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.309+0000 7fe5fde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6000705b0 0x7fe60015b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:10.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.309+0000 7fe5fde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6000705b0 0x7fe60015b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51298/0 (socket says 172.21.15.100:51298) 2024-09-17T04:32:10.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.310+0000 7fe5fde00640 1 -- 172.21.15.100:0/341051730 learned_addr learned my addr 172.21.15.100:0/341051730 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:10.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.310+0000 7fe5fd400640 1 --2- 172.21.15.100:0/341051730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6000b3ec0 0x7fe60009c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:10.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.310+0000 7fe5fe800640 1 --2- 172.21.15.100:0/341051730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6001532e0 0x7fe600147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:10.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.310+0000 7fe5fde00640 1 -- 172.21.15.100:0/341051730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6000b3ec0 msgr2=0x7fe60009c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:10.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.310+0000 7fe5fde00640 1 --2- 172.21.15.100:0/341051730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6000b3ec0 0x7fe60009c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:10.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.310+0000 7fe5fde00640 1 -- 172.21.15.100:0/341051730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6001532e0 msgr2=0x7fe600147f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:10.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.310+0000 7fe5fde00640 1 --2- 172.21.15.100:0/341051730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6001532e0 0x7fe600147f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:10.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.310+0000 7fe5fde00640 1 -- 172.21.15.100:0/341051730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60009cab0 con 0x7fe6000705b0 2024-09-17T04:32:10.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.310+0000 7fe5fe800640 1 --2- 172.21.15.100:0/341051730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6001532e0 0x7fe600147f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:10.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.310+0000 7fe5fde00640 1 --2- 172.21.15.100:0/341051730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6000705b0 0x7fe60015b040 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000d6d0 tx=0x7fe5f000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:10.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.310+0000 7fe5f6000640 1 -- 172.21.15.100:0/341051730 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0017070 con 0x7fe6000705b0 2024-09-17T04:32:10.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.311+0000 7fe5f6000640 1 -- 172.21.15.100:0/341051730 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe5f0005330 con 0x7fe6000705b0 2024-09-17T04:32:10.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.311+0000 7fe5ff200640 1 -- 172.21.15.100:0/341051730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe600158e80 con 0x7fe6000705b0 2024-09-17T04:32:10.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.311+0000 7fe5f6000640 1 -- 172.21.15.100:0/341051730 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0009820 con 0x7fe6000705b0 2024-09-17T04:32:10.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.311+0000 7fe5ff200640 1 -- 172.21.15.100:0/341051730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe600159120 con 0x7fe6000705b0 2024-09-17T04:32:10.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.312+0000 7fe5ff200640 1 -- 172.21.15.100:0/341051730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6000afe00 con 0x7fe6000705b0 2024-09-17T04:32:10.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.313+0000 7fe5f6000640 1 -- 172.21.15.100:0/341051730 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fe5f0009bd0 con 0x7fe6000705b0 2024-09-17T04:32:10.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.314+0000 7fe5f6000640 1 --2- 172.21.15.100:0/341051730 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe5e00791c0 0x7fe5e007b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:10.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.314+0000 7fe5f6000640 1 -- 172.21.15.100:0/341051730 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5794+0+0 (secure 0 0 0) 0x7fe5f009bd10 con 0x7fe6000705b0 2024-09-17T04:32:10.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.314+0000 7fe5fd400640 1 --2- 172.21.15.100:0/341051730 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe5e00791c0 0x7fe5e007b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:10.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.315+0000 7fe5fd400640 1 --2- 172.21.15.100:0/341051730 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe5e00791c0 0x7fe5e007b6d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe60015a7d0 tx=0x7fe5ec00b260 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:10.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.318+0000 7fe5f6000640 1 -- 172.21.15.100:0/341051730 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe5f0063010 con 0x7fe6000705b0 2024-09-17T04:32:10.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.528+0000 7fe5ff200640 1 -- 172.21.15.100:0/341051730 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7fe60006de20 con 0x7fe5e00791c0 2024-09-17T04:32:10.528 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:10.529+0000 7f30b3000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-17T04:32:10.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.529+0000 7fe5f6000640 1 -- 172.21.15.100:0/341051730 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fe60006de20 con 0x7fe5e00791c0 2024-09-17T04:32:10.529 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-17T04:32:10.549 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T04:32:10.549 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot protect non_existent_subvolume subvolume_1 snapshot_1 2024-09-17T04:32:10.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.774+0000 7efdc8000640 1 -- 172.21.15.100:0/1696366308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdc00af100 msgr2=0x7efdc00af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:10.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.774+0000 7efdc8000640 1 --2- 172.21.15.100:0/1696366308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdc00af100 0x7efdc00af510 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7efdc00d01c0 tx=0x7efdb802aa30 comp rx=0 tx=0).stop 2024-09-17T04:32:10.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.774+0000 7efdc8000640 1 -- 172.21.15.100:0/1696366308 shutdown_connections 2024-09-17T04:32:10.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.774+0000 7efdc8000640 1 --2- 172.21.15.100:0/1696366308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdc00af100 0x7efdc00af510 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:10.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.774+0000 7efdc8000640 1 --2- 172.21.15.100:0/1696366308 >> v2:172.21.15.184:3300/0 conn(0x7efdc00cc830 0x7efdc00b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:10.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.774+0000 7efdc8000640 1 -- 172.21.15.100:0/1696366308 >> 172.21.15.100:0/1696366308 conn(0x7efdc00b2540 msgr2=0x7efdc00d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:10.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.774+0000 7efdc8000640 1 -- 172.21.15.100:0/1696366308 shutdown_connections 2024-09-17T04:32:10.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.774+0000 7efdc8000640 1 -- 172.21.15.100:0/1696366308 wait complete. 2024-09-17T04:32:10.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.776+0000 7efdc8000640 1 Processor -- start 2024-09-17T04:32:10.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.776+0000 7efdc8000640 1 -- start start 2024-09-17T04:32:10.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.777+0000 7efdc8000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdc006cda0 0x7efdc01532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:10.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.777+0000 7efdc8000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdc00af100 0x7efdc01f6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:10.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.777+0000 7efdc8000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdc00cc830 0x7efdc01f6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:10.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.778+0000 7efdc8000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efdc00b3620 con 0x7efdc00cc830 2024-09-17T04:32:10.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.778+0000 7efdc8000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efdc00b34a0 con 0x7efdc00af100 2024-09-17T04:32:10.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.778+0000 7efdc8000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efdc00b37a0 con 0x7efdc006cda0 2024-09-17T04:32:10.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.778+0000 7efdc6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdc00cc830 0x7efdc01f6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:10.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.778+0000 7efdc6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdc00cc830 0x7efdc01f6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51314/0 (socket says 172.21.15.100:51314) 2024-09-17T04:32:10.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.778+0000 7efdc6000640 1 -- 172.21.15.100:0/3114544947 learned_addr learned my addr 172.21.15.100:0/3114544947 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:10.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.778+0000 7efdc5600640 1 --2- 172.21.15.100:0/3114544947 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdc006cda0 0x7efdc01532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:10.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.778+0000 7efdc4c00640 1 --2- 172.21.15.100:0/3114544947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdc00af100 0x7efdc01f6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:10.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.778+0000 7efdc6000640 1 -- 172.21.15.100:0/3114544947 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdc006cda0 msgr2=0x7efdc01532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:10.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.778+0000 7efdc6000640 1 --2- 172.21.15.100:0/3114544947 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdc006cda0 0x7efdc01532f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:10.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.778+0000 7efdc6000640 1 -- 172.21.15.100:0/3114544947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdc00af100 msgr2=0x7efdc01f6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:10.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.778+0000 7efdc6000640 1 --2- 172.21.15.100:0/3114544947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdc00af100 0x7efdc01f6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:10.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.778+0000 7efdc6000640 1 -- 172.21.15.100:0/3114544947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc009b320 con 0x7efdc00cc830 2024-09-17T04:32:10.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.779+0000 7efdc6000640 1 --2- 172.21.15.100:0/3114544947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdc00cc830 0x7efdc01f6e70 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7efdb8005fd0 tx=0x7efdb8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:10.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.779+0000 7efdbd600640 1 -- 172.21.15.100:0/3114544947 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb8033050 con 0x7efdc00cc830 2024-09-17T04:32:10.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.779+0000 7efdbd600640 1 -- 172.21.15.100:0/3114544947 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efdb8004450 con 0x7efdc00cc830 2024-09-17T04:32:10.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.779+0000 7efdbd600640 1 -- 172.21.15.100:0/3114544947 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb8004840 con 0x7efdc00cc830 2024-09-17T04:32:10.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.779+0000 7efdc8000640 1 -- 172.21.15.100:0/3114544947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc00c9da0 con 0x7efdc00cc830 2024-09-17T04:32:10.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.779+0000 7efdc8000640 1 -- 172.21.15.100:0/3114544947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc01f7b40 con 0x7efdc00cc830 2024-09-17T04:32:10.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.780+0000 7efdc8000640 1 -- 172.21.15.100:0/3114544947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdc015ff00 con 0x7efdc00cc830 2024-09-17T04:32:10.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.781+0000 7efdbd600640 1 -- 172.21.15.100:0/3114544947 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7efdb8036070 con 0x7efdc00cc830 2024-09-17T04:32:10.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.782+0000 7efdbd600640 1 --2- 172.21.15.100:0/3114544947 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7efda00791c0 0x7efda007b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:10.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.782+0000 7efdbd600640 1 -- 172.21.15.100:0/3114544947 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5794+0+0 (secure 0 0 0) 0x7efdb80bce30 con 0x7efdc00cc830 2024-09-17T04:32:10.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.782+0000 7efdc5600640 1 --2- 172.21.15.100:0/3114544947 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7efda00791c0 0x7efda007b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:10.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.783+0000 7efdc5600640 1 --2- 172.21.15.100:0/3114544947 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7efda00791c0 0x7efda007b6d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efdb400bb50 tx=0x7efdb400b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:10.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:10.787+0000 7efdbd600640 1 -- 172.21.15.100:0/3114544947 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7efdc015ff00 con 0x7efdc00cc830 2024-09-17T04:32:11.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.006+0000 7efdc8000640 1 -- 172.21.15.100:0/3114544947 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot protect", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7efdc00c9b50 con 0x7efda00791c0 2024-09-17T04:32:11.006 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:11.007+0000 7f30b3000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-17T04:32:11.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.007+0000 7efdbd600640 1 -- 172.21.15.100:0/3114544947 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7efdc00c9b50 con 0x7efda00791c0 2024-09-17T04:32:11.007 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-17T04:32:11.022 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T04:32:11.023 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot unprotect non_existent_subvolume subvolume_1 snapshot_1 2024-09-17T04:32:11.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.250+0000 7f1bb6a00640 1 -- 172.21.15.100:0/4227174946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b98004800 msgr2=0x7f1b98004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:11.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.250+0000 7f1bb6a00640 1 --2- 172.21.15.100:0/4227174946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b98004800 0x7f1b98004c10 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c00c4a0 tx=0x7f1b98005830 comp rx=0 tx=0).stop 2024-09-17T04:32:11.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.250+0000 7f1bb6000640 1 -- 172.21.15.100:0/4227174946 reap_dead start 2024-09-17T04:32:11.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.250+0000 7f1bb6a00640 1 -- 172.21.15.100:0/4227174946 shutdown_connections 2024-09-17T04:32:11.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.250+0000 7f1bb6a00640 1 -- 172.21.15.100:0/4227174946 >> 172.21.15.100:0/4227174946 conn(0x7f1bb0071090 msgr2=0x7f1bb00714a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:11.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.250+0000 7f1bb6a00640 1 -- 172.21.15.100:0/4227174946 shutdown_connections 2024-09-17T04:32:11.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.250+0000 7f1bb6a00640 1 -- 172.21.15.100:0/4227174946 wait complete. 2024-09-17T04:32:11.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.252+0000 7f1bb6a00640 1 Processor -- start 2024-09-17T04:32:11.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.252+0000 7f1bb6a00640 1 -- start start 2024-09-17T04:32:11.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.253+0000 7f1bb6a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b98004800 0x7f1bb0147320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:11.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.254+0000 7f1bb6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bb0147860 0x7f1bb0153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:11.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.254+0000 7f1bb6a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bb00b7c30 0x7f1bb01538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:11.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.254+0000 7f1bb6a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb006f120 con 0x7f1bb0147860 2024-09-17T04:32:11.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.254+0000 7f1bb6a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb00d05e0 con 0x7f1b98004800 2024-09-17T04:32:11.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.254+0000 7f1bb6a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1bb006f2a0 con 0x7f1bb00b7c30 2024-09-17T04:32:11.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.254+0000 7f1bb4c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bb0147860 0x7f1bb0153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:11.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.254+0000 7f1bb4c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bb0147860 0x7f1bb0153380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51326/0 (socket says 172.21.15.100:51326) 2024-09-17T04:32:11.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.254+0000 7f1bb4c00640 1 -- 172.21.15.100:0/1579353191 learned_addr learned my addr 172.21.15.100:0/1579353191 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:11.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.254+0000 7f1bb4c00640 1 -- 172.21.15.100:0/1579353191 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bb00b7c30 msgr2=0x7f1bb01538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:11.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.254+0000 7f1bb6000640 1 --2- 172.21.15.100:0/1579353191 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bb00b7c30 0x7f1bb01538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:11.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.254+0000 7f1bb4c00640 1 --2- 172.21.15.100:0/1579353191 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bb00b7c30 0x7f1bb01538c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:11.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.254+0000 7f1bb5600640 1 --2- 172.21.15.100:0/1579353191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b98004800 0x7f1bb0147320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:11.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.254+0000 7f1bb4c00640 1 -- 172.21.15.100:0/1579353191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b98004800 msgr2=0x7f1bb0147320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:11.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.254+0000 7f1bb4c00640 1 --2- 172.21.15.100:0/1579353191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b98004800 0x7f1bb0147320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:11.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.254+0000 7f1bb4c00640 1 -- 172.21.15.100:0/1579353191 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb01527a0 con 0x7f1bb0147860 2024-09-17T04:32:11.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.254+0000 7f1bb6000640 1 --2- 172.21.15.100:0/1579353191 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bb00b7c30 0x7f1bb01538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:32:11.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.255+0000 7f1bb4c00640 1 --2- 172.21.15.100:0/1579353191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bb0147860 0x7f1bb0153380 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c000fb0 tx=0x7f1b9c0012c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:11.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.255+0000 7f1bad600640 1 -- 172.21.15.100:0/1579353191 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c042070 con 0x7f1bb0147860 2024-09-17T04:32:11.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.255+0000 7f1bad600640 1 -- 172.21.15.100:0/1579353191 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b9c02b650 con 0x7f1bb0147860 2024-09-17T04:32:11.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.255+0000 7f1bb6a00640 1 -- 172.21.15.100:0/1579353191 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb0151c20 con 0x7f1bb0147860 2024-09-17T04:32:11.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.255+0000 7f1bad600640 1 -- 172.21.15.100:0/1579353191 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c039070 con 0x7f1bb0147860 2024-09-17T04:32:11.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.255+0000 7f1bb6a00640 1 -- 172.21.15.100:0/1579353191 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb01521e0 con 0x7f1bb0147860 2024-09-17T04:32:11.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.256+0000 7f1bb6a00640 1 -- 172.21.15.100:0/1579353191 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bb006ebe0 con 0x7f1bb0147860 2024-09-17T04:32:11.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.257+0000 7f1bad600640 1 -- 172.21.15.100:0/1579353191 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f1b9c02baa0 con 0x7f1bb0147860 2024-09-17T04:32:11.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.258+0000 7f1bad600640 1 --2- 172.21.15.100:0/1579353191 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1b900791c0 0x7f1b9007b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:11.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.258+0000 7f1bad600640 1 -- 172.21.15.100:0/1579353191 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5794+0+0 (secure 0 0 0) 0x7f1b9c0bb460 con 0x7f1bb0147860 2024-09-17T04:32:11.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.258+0000 7f1bb5600640 1 --2- 172.21.15.100:0/1579353191 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1b900791c0 0x7f1b9007b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:11.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.259+0000 7f1bb5600640 1 --2- 172.21.15.100:0/1579353191 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1b900791c0 0x7f1b9007b6d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1ba40099f0 tx=0x7f1ba4005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:11.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.263+0000 7f1bad600640 1 -- 172.21.15.100:0/1579353191 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f1bb00d0bb0 con 0x7f1bb0147860 2024-09-17T04:32:11.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.480+0000 7f1bb6a00640 1 -- 172.21.15.100:0/1579353191 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot unprotect", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f1bb0161a70 con 0x7f1b900791c0 2024-09-17T04:32:11.481 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:11.482+0000 7f30b3000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-17T04:32:11.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.482+0000 7f1bad600640 1 -- 172.21.15.100:0/1579353191 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f1bb0161a70 con 0x7f1b900791c0 2024-09-17T04:32:11.481 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-17T04:32:11.498 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T04:32:11.498 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls non_existent_subvolume subvolume_1 2024-09-17T04:32:11.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.723+0000 7fe447000640 1 -- 172.21.15.100:0/1734267172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4400af100 msgr2=0x7fe4400af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:11.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.723+0000 7fe447000640 1 --2- 172.21.15.100:0/1734267172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4400af100 0x7fe4400af510 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fe4400d0d00 tx=0x7fe43802aa30 comp rx=0 tx=0).stop 2024-09-17T04:32:11.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.723+0000 7fe447000640 1 -- 172.21.15.100:0/1734267172 shutdown_connections 2024-09-17T04:32:11.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.723+0000 7fe447000640 1 --2- 172.21.15.100:0/1734267172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4400af100 0x7fe4400af510 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:11.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.723+0000 7fe447000640 1 --2- 172.21.15.100:0/1734267172 >> v2:172.21.15.184:3300/0 conn(0x7fe44006d4b0 0x7fe4400b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:11.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.723+0000 7fe447000640 1 -- 172.21.15.100:0/1734267172 >> 172.21.15.100:0/1734267172 conn(0x7fe4400b2540 msgr2=0x7fe4400d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:11.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.723+0000 7fe447000640 1 -- 172.21.15.100:0/1734267172 shutdown_connections 2024-09-17T04:32:11.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.723+0000 7fe447000640 1 -- 172.21.15.100:0/1734267172 wait complete. 2024-09-17T04:32:11.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.725+0000 7fe447000640 1 Processor -- start 2024-09-17T04:32:11.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.725+0000 7fe447000640 1 -- start start 2024-09-17T04:32:11.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.727+0000 7fe447000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4400af100 0x7fe4400b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:11.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.727+0000 7fe447000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4400cc830 0x7fe440098cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:11.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.727+0000 7fe447000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe440153f90 0x7fe4400cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:11.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.727+0000 7fe447000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe4400b3620 con 0x7fe4400af100 2024-09-17T04:32:11.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.727+0000 7fe447000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe4400b34a0 con 0x7fe440153f90 2024-09-17T04:32:11.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.727+0000 7fe447000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe4400b37a0 con 0x7fe4400cc830 2024-09-17T04:32:11.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.727+0000 7fe43fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4400af100 0x7fe4400b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:11.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.727+0000 7fe43fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4400af100 0x7fe4400b3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51356/0 (socket says 172.21.15.100:51356) 2024-09-17T04:32:11.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.727+0000 7fe43fe00640 1 -- 172.21.15.100:0/2060876844 learned_addr learned my addr 172.21.15.100:0/2060876844 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:11.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.727+0000 7fe445000640 1 --2- 172.21.15.100:0/2060876844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe440153f90 0x7fe4400cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:11.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.727+0000 7fe437400640 1 --2- 172.21.15.100:0/2060876844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4400cc830 0x7fe440098cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:11.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.727+0000 7fe43fe00640 1 -- 172.21.15.100:0/2060876844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4400cc830 msgr2=0x7fe440098cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:11.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.727+0000 7fe43fe00640 1 --2- 172.21.15.100:0/2060876844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4400cc830 0x7fe440098cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:11.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.727+0000 7fe43fe00640 1 -- 172.21.15.100:0/2060876844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe440153f90 msgr2=0x7fe4400cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:11.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.727+0000 7fe43fe00640 1 --2- 172.21.15.100:0/2060876844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe440153f90 0x7fe4400cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:11.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.727+0000 7fe43fe00640 1 -- 172.21.15.100:0/2060876844 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe440155b20 con 0x7fe4400af100 2024-09-17T04:32:11.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.728+0000 7fe43fe00640 1 --2- 172.21.15.100:0/2060876844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4400af100 0x7fe4400b3ef0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fe43000b3f0 tx=0x7fe43000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:11.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.728+0000 7fe43cc00640 1 -- 172.21.15.100:0/2060876844 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430015020 con 0x7fe4400af100 2024-09-17T04:32:11.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.728+0000 7fe43cc00640 1 -- 172.21.15.100:0/2060876844 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe430011920 con 0x7fe4400af100 2024-09-17T04:32:11.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.728+0000 7fe447000640 1 -- 172.21.15.100:0/2060876844 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe440155de0 con 0x7fe4400af100 2024-09-17T04:32:11.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.728+0000 7fe43cc00640 1 -- 172.21.15.100:0/2060876844 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430011d70 con 0x7fe4400af100 2024-09-17T04:32:11.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.728+0000 7fe447000640 1 -- 172.21.15.100:0/2060876844 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4401563a0 con 0x7fe4400af100 2024-09-17T04:32:11.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.729+0000 7fe447000640 1 -- 172.21.15.100:0/2060876844 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4400b3ce0 con 0x7fe4400af100 2024-09-17T04:32:11.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.730+0000 7fe43cc00640 1 -- 172.21.15.100:0/2060876844 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fe430018020 con 0x7fe4400af100 2024-09-17T04:32:11.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.731+0000 7fe43cc00640 1 --2- 172.21.15.100:0/2060876844 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe41c0791c0 0x7fe41c07b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:11.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.731+0000 7fe43cc00640 1 -- 172.21.15.100:0/2060876844 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5794+0+0 (secure 0 0 0) 0x7fe43009b650 con 0x7fe4400af100 2024-09-17T04:32:11.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.732+0000 7fe437400640 1 --2- 172.21.15.100:0/2060876844 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe41c0791c0 0x7fe41c07b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:11.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.732+0000 7fe437400640 1 --2- 172.21.15.100:0/2060876844 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe41c0791c0 0x7fe41c07b6d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe4400d0190 tx=0x7fe42800a250 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:11.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.736+0000 7fe43cc00640 1 -- 172.21.15.100:0/2060876844 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe43001f030 con 0x7fe4400af100 2024-09-17T04:32:11.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.953+0000 7fe447000640 1 -- 172.21.15.100:0/2060876844 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7fe4400d1030 con 0x7fe41c0791c0 2024-09-17T04:32:12.322 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:11.954+0000 7f30b3000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-17T04:32:12.323 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T04:32:12.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:11.954+0000 7fe43cc00640 1 -- 172.21.15.100:0/2060876844 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fe4400d1030 con 0x7fe41c0791c0 2024-09-17T04:32:12.324 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-17T04:32:12.325 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone non_existent_subvolume subvolume_1 snapshot_1 clone_1 2024-09-17T04:32:12.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.551+0000 7fc5eb000640 1 -- 172.21.15.100:0/2334555192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5e40cd240 msgr2=0x7fc5e40b3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:12.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.551+0000 7fc5eb000640 1 --2- 172.21.15.100:0/2334555192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5e40cd240 0x7fc5e40b3fc0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc005fd0 tx=0x7fc5dc02aa90 comp rx=0 tx=0).stop 2024-09-17T04:32:12.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.551+0000 7fc5eb000640 1 -- 172.21.15.100:0/2334555192 shutdown_connections 2024-09-17T04:32:12.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.551+0000 7fc5eb000640 1 --2- 172.21.15.100:0/2334555192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5e40cd240 0x7fc5e40b3fc0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:12.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.551+0000 7fc5eb000640 1 --2- 172.21.15.100:0/2334555192 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc5e40af4e0 0x7fc5e415d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:12.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.551+0000 7fc5eb000640 1 -- 172.21.15.100:0/2334555192 >> 172.21.15.100:0/2334555192 conn(0x7fc5e40b1dd0 msgr2=0x7fc5e40b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:12.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.552+0000 7fc5eb000640 1 -- 172.21.15.100:0/2334555192 shutdown_connections 2024-09-17T04:32:12.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.552+0000 7fc5eb000640 1 -- 172.21.15.100:0/2334555192 wait complete. 2024-09-17T04:32:12.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.554+0000 7fc5eb000640 1 Processor -- start 2024-09-17T04:32:12.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.554+0000 7fc5eb000640 1 -- start start 2024-09-17T04:32:12.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.555+0000 7fc5eb000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc5e40af4e0 0x7fc5e41f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:12.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.555+0000 7fc5eb000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc5e4160370 0x7fc5e4157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:12.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.555+0000 7fc5eb000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5e41578d0 0x7fc5e415ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:12.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.555+0000 7fc5eb000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e40b4500 con 0x7fc5e41578d0 2024-09-17T04:32:12.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.555+0000 7fc5eb000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e4160990 con 0x7fc5e4160370 2024-09-17T04:32:12.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.555+0000 7fc5eb000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc5e40b4680 con 0x7fc5e40af4e0 2024-09-17T04:32:12.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.556+0000 7fc5ea600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5e41578d0 0x7fc5e415ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:12.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.556+0000 7fc5ea600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5e41578d0 0x7fc5e415ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51368/0 (socket says 172.21.15.100:51368) 2024-09-17T04:32:12.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.556+0000 7fc5ea600640 1 -- 172.21.15.100:0/3172250907 learned_addr learned my addr 172.21.15.100:0/3172250907 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:12.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.556+0000 7fc5e9200640 1 --2- 172.21.15.100:0/3172250907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc5e4160370 0x7fc5e4157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:12.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.556+0000 7fc5e9c00640 1 --2- 172.21.15.100:0/3172250907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc5e40af4e0 0x7fc5e41f71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:12.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.556+0000 7fc5ea600640 1 -- 172.21.15.100:0/3172250907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc5e40af4e0 msgr2=0x7fc5e41f71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:12.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.556+0000 7fc5ea600640 1 --2- 172.21.15.100:0/3172250907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc5e40af4e0 0x7fc5e41f71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:12.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.556+0000 7fc5ea600640 1 -- 172.21.15.100:0/3172250907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc5e4160370 msgr2=0x7fc5e4157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:12.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.556+0000 7fc5ea600640 1 --2- 172.21.15.100:0/3172250907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc5e4160370 0x7fc5e4157390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:12.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.556+0000 7fc5ea600640 1 -- 172.21.15.100:0/3172250907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e41f36e0 con 0x7fc5e41578d0 2024-09-17T04:32:12.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.557+0000 7fc5ea600640 1 --2- 172.21.15.100:0/3172250907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5e41578d0 0x7fc5e415ef00 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc006010 tx=0x7fc5dc003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:12.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.557+0000 7fc5e1600640 1 -- 172.21.15.100:0/3172250907 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc044070 con 0x7fc5e41578d0 2024-09-17T04:32:12.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.557+0000 7fc5e1600640 1 -- 172.21.15.100:0/3172250907 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5dc02e7f0 con 0x7fc5e41578d0 2024-09-17T04:32:12.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.557+0000 7fc5eb000640 1 -- 172.21.15.100:0/3172250907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5e41f39c0 con 0x7fc5e41578d0 2024-09-17T04:32:12.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.557+0000 7fc5e1600640 1 -- 172.21.15.100:0/3172250907 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc0044b0 con 0x7fc5e41578d0 2024-09-17T04:32:12.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.557+0000 7fc5eb000640 1 -- 172.21.15.100:0/3172250907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5e41f3f80 con 0x7fc5e41578d0 2024-09-17T04:32:12.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.558+0000 7fc5eb000640 1 -- 172.21.15.100:0/3172250907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5e4157d50 con 0x7fc5e41578d0 2024-09-17T04:32:12.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.559+0000 7fc5e1600640 1 -- 172.21.15.100:0/3172250907 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fc5dc047070 con 0x7fc5e41578d0 2024-09-17T04:32:12.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.560+0000 7fc5e1600640 1 --2- 172.21.15.100:0/3172250907 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc5c40791c0 0x7fc5c407b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:12.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.560+0000 7fc5e1600640 1 -- 172.21.15.100:0/3172250907 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5794+0+0 (secure 0 0 0) 0x7fc5dc0bc920 con 0x7fc5e41578d0 2024-09-17T04:32:12.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.561+0000 7fc5e9c00640 1 --2- 172.21.15.100:0/3172250907 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc5c40791c0 0x7fc5c407b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:12.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.561+0000 7fc5e9c00640 1 --2- 172.21.15.100:0/3172250907 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc5c40791c0 0x7fc5c407b6d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8005fd0 tx=0x7fc5d8005850 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:12.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.565+0000 7fc5e1600640 1 -- 172.21.15.100:0/3172250907 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc5dc032030 con 0x7fc5e41578d0 2024-09-17T04:32:12.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.783+0000 7fc5eb000640 1 -- 172.21.15.100:0/3172250907 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target_sub_name": "clone_1", "target": ["mon-mgr", ""]}) -- 0x7fc5e40b4b80 con 0x7fc5c40791c0 2024-09-17T04:32:12.789 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:12.790+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:32:12.789 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:12.790+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:32:12.789 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:12.790+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:32:12.789 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:12.790+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:32:12.789 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:12.790+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:32:12.796 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:12.796+0000 7f30b3000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-17T04:32:12.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:12.797+0000 7fc5e1600640 1 -- 172.21.15.100:0/3172250907 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fc5e40b4b80 con 0x7fc5c40791c0 2024-09-17T04:32:12.796 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-17T04:32:12.816 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T04:32:12.817 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status non_existent_subvolume clone_1 2024-09-17T04:32:13.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.045+0000 7f6704800640 1 -- 172.21.15.100:0/1702091556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66fc145a40 msgr2=0x7f66fc073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:13.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.045+0000 7f6704800640 1 --2- 172.21.15.100:0/1702091556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66fc145a40 0x7f66fc073540 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f66ec009830 tx=0x7f66ec02a5d0 comp rx=0 tx=0).stop 2024-09-17T04:32:13.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.045+0000 7f6704800640 1 -- 172.21.15.100:0/1702091556 shutdown_connections 2024-09-17T04:32:13.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.045+0000 7f6704800640 1 --2- 172.21.15.100:0/1702091556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66fc145a40 0x7f66fc073540 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:13.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.045+0000 7f6704800640 1 -- 172.21.15.100:0/1702091556 >> 172.21.15.100:0/1702091556 conn(0x7f66fc06d1c0 msgr2=0x7f66fc06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:13.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.045+0000 7f6704800640 1 -- 172.21.15.100:0/1702091556 shutdown_connections 2024-09-17T04:32:13.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.045+0000 7f6704800640 1 -- 172.21.15.100:0/1702091556 wait complete. 2024-09-17T04:32:13.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.048+0000 7f6704800640 1 Processor -- start 2024-09-17T04:32:13.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.048+0000 7f6704800640 1 -- start start 2024-09-17T04:32:13.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.049+0000 7f6704800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66fc073a80 0x7f66fc0b3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:13.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.049+0000 7f6704800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66fc145550 0x7f66fc13b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:13.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.049+0000 7f6704800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66fc145a40 0x7f66fc13b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:13.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.049+0000 7f6704800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc0ce350 con 0x7f66fc073a80 2024-09-17T04:32:13.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.049+0000 7f6704800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc0ce1d0 con 0x7f66fc145a40 2024-09-17T04:32:13.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.049+0000 7f6704800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f66fc0ce4d0 con 0x7f66fc145550 2024-09-17T04:32:13.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.049+0000 7f6701e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66fc073a80 0x7f66fc0b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:13.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.050+0000 7f6701e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66fc073a80 0x7f66fc0b3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51402/0 (socket says 172.21.15.100:51402) 2024-09-17T04:32:13.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.050+0000 7f6701e00640 1 -- 172.21.15.100:0/2765074584 learned_addr learned my addr 172.21.15.100:0/2765074584 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:13.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.050+0000 7f6702800640 1 --2- 172.21.15.100:0/2765074584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66fc145a40 0x7f66fc13b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:13.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.050+0000 7f6701400640 1 --2- 172.21.15.100:0/2765074584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66fc145550 0x7f66fc13b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:13.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.050+0000 7f6701e00640 1 -- 172.21.15.100:0/2765074584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66fc145550 msgr2=0x7f66fc13b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:13.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.050+0000 7f6701e00640 1 --2- 172.21.15.100:0/2765074584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66fc145550 0x7f66fc13b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:13.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.050+0000 7f6701e00640 1 -- 172.21.15.100:0/2765074584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66fc145a40 msgr2=0x7f66fc13b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:13.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.050+0000 7f6701e00640 1 --2- 172.21.15.100:0/2765074584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66fc145a40 0x7f66fc13b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:13.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.050+0000 7f6701e00640 1 -- 172.21.15.100:0/2765074584 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66fc0b4270 con 0x7f66fc073a80 2024-09-17T04:32:13.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.050+0000 7f6701e00640 1 --2- 172.21.15.100:0/2765074584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66fc073a80 0x7f66fc0b3580 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f66f000c430 tx=0x7f66f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:13.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.051+0000 7f66f6000640 1 -- 172.21.15.100:0/2765074584 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f0017070 con 0x7f66fc073a80 2024-09-17T04:32:13.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.051+0000 7f66f6000640 1 -- 172.21.15.100:0/2765074584 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66f00053e0 con 0x7f66fc073a80 2024-09-17T04:32:13.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.051+0000 7f6704800640 1 -- 172.21.15.100:0/2765074584 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66fc0b4550 con 0x7f66fc073a80 2024-09-17T04:32:13.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.051+0000 7f66f6000640 1 -- 172.21.15.100:0/2765074584 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f0003850 con 0x7f66fc073a80 2024-09-17T04:32:13.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.051+0000 7f6704800640 1 -- 172.21.15.100:0/2765074584 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66fc0b4b10 con 0x7f66fc073a80 2024-09-17T04:32:13.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.052+0000 7f6704800640 1 -- 172.21.15.100:0/2765074584 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66fc073540 con 0x7f66fc073a80 2024-09-17T04:32:13.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.053+0000 7f66f6000640 1 -- 172.21.15.100:0/2765074584 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f66f001a070 con 0x7f66fc073a80 2024-09-17T04:32:13.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.054+0000 7f66f6000640 1 --2- 172.21.15.100:0/2765074584 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f66e0079250 0x7f66e007b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:13.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.054+0000 7f66f6000640 1 -- 172.21.15.100:0/2765074584 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5794+0+0 (secure 0 0 0) 0x7f66f009aeb0 con 0x7f66fc073a80 2024-09-17T04:32:13.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.054+0000 7f6701400640 1 --2- 172.21.15.100:0/2765074584 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f66e0079250 0x7f66e007b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:13.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.055+0000 7f6701400640 1 --2- 172.21.15.100:0/2765074584 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f66e0079250 0x7f66e007b760 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f66ec034000 tx=0x7f66ec004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:13.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.059+0000 7f66f6000640 1 -- 172.21.15.100:0/2765074584 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f66f00620e0 con 0x7f66fc073a80 2024-09-17T04:32:13.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.274+0000 7f6704800640 1 -- 172.21.15.100:0/2765074584 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "non_existent_subvolume", "clone_name": "clone_1", "target": ["mon-mgr", ""]}) -- 0x7f66fc06ffe0 con 0x7f66e0079250 2024-09-17T04:32:13.275 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:13.275+0000 7f30b3000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-17T04:32:13.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.276+0000 7f66f6000640 1 -- 172.21.15.100:0/2765074584 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f66fc06ffe0 con 0x7f66e0079250 2024-09-17T04:32:13.275 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-17T04:32:13.291 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T04:32:13.292 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create non_existent_subvolume group_1 2024-09-17T04:32:13.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.517+0000 7f0effe00640 1 -- 172.21.15.100:0/1661843223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ee0004800 msgr2=0x7f0ee0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:13.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.517+0000 7f0effe00640 1 --2- 172.21.15.100:0/1661843223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ee0004800 0x7f0ee0004c10 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f0ef81613e0 tx=0x7f0ee0005830 comp rx=0 tx=0).stop 2024-09-17T04:32:13.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.517+0000 7f0efde00640 1 -- 172.21.15.100:0/1661843223 reap_dead start 2024-09-17T04:32:13.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.517+0000 7f0effe00640 1 -- 172.21.15.100:0/1661843223 shutdown_connections 2024-09-17T04:32:13.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.517+0000 7f0effe00640 1 -- 172.21.15.100:0/1661843223 >> 172.21.15.100:0/1661843223 conn(0x7f0ef806d1c0 msgr2=0x7f0ef806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:13.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.517+0000 7f0effe00640 1 -- 172.21.15.100:0/1661843223 shutdown_connections 2024-09-17T04:32:13.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.517+0000 7f0effe00640 1 -- 172.21.15.100:0/1661843223 wait complete. 2024-09-17T04:32:13.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.520+0000 7f0effe00640 1 Processor -- start 2024-09-17T04:32:13.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.520+0000 7f0effe00640 1 -- start start 2024-09-17T04:32:13.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.521+0000 7f0effe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ee0004800 0x7f0ef80b3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:13.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.521+0000 7f0effe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ef80b3ca0 0x7f0ef8154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:13.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.521+0000 7f0effe00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ef8155590 0x7f0ef8154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:13.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.521+0000 7f0effe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef8073540 con 0x7f0ef80b3ca0 2024-09-17T04:32:13.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.521+0000 7f0effe00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef80ce940 con 0x7f0ee0004800 2024-09-17T04:32:13.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.521+0000 7f0effe00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0ef80736c0 con 0x7f0ef8155590 2024-09-17T04:32:13.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.521+0000 7f0efca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ef80b3ca0 0x7f0ef8154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:13.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.522+0000 7f0efca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ef80b3ca0 0x7f0ef8154a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51420/0 (socket says 172.21.15.100:51420) 2024-09-17T04:32:13.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.522+0000 7f0efca00640 1 -- 172.21.15.100:0/899614234 learned_addr learned my addr 172.21.15.100:0/899614234 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:13.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.522+0000 7f0efd400640 1 --2- 172.21.15.100:0/899614234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ee0004800 0x7f0ef80b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:13.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.522+0000 7f0efde00640 1 --2- 172.21.15.100:0/899614234 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ef8155590 0x7f0ef8154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:13.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.522+0000 7f0efca00640 1 -- 172.21.15.100:0/899614234 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ef8155590 msgr2=0x7f0ef8154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:13.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.522+0000 7f0efca00640 1 --2- 172.21.15.100:0/899614234 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ef8155590 0x7f0ef8154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:13.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.522+0000 7f0efca00640 1 -- 172.21.15.100:0/899614234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ee0004800 msgr2=0x7f0ef80b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:13.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.522+0000 7f0efca00640 1 --2- 172.21.15.100:0/899614234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ee0004800 0x7f0ef80b3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:13.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.522+0000 7f0efca00640 1 -- 172.21.15.100:0/899614234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef81f9670 con 0x7f0ef80b3ca0 2024-09-17T04:32:13.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.522+0000 7f0efde00640 1 --2- 172.21.15.100:0/899614234 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ef8155590 0x7f0ef8154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:13.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.522+0000 7f0efca00640 1 --2- 172.21.15.100:0/899614234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ef80b3ca0 0x7f0ef8154a00 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4036040 tx=0x7f0ee4033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:13.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.523+0000 7f0ef5600640 1 -- 172.21.15.100:0/899614234 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4043070 con 0x7f0ef80b3ca0 2024-09-17T04:32:13.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.523+0000 7f0ef5600640 1 -- 172.21.15.100:0/899614234 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ee4004140 con 0x7f0ef80b3ca0 2024-09-17T04:32:13.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.523+0000 7f0ef5600640 1 -- 172.21.15.100:0/899614234 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4038040 con 0x7f0ef80b3ca0 2024-09-17T04:32:13.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.523+0000 7f0effe00640 1 -- 172.21.15.100:0/899614234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef81f9950 con 0x7f0ef80b3ca0 2024-09-17T04:32:13.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.523+0000 7f0effe00640 1 -- 172.21.15.100:0/899614234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef81f9f10 con 0x7f0ef80b3ca0 2024-09-17T04:32:13.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.524+0000 7f0effe00640 1 -- 172.21.15.100:0/899614234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ef81f93e0 con 0x7f0ef80b3ca0 2024-09-17T04:32:13.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.525+0000 7f0ef5600640 1 -- 172.21.15.100:0/899614234 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f0ee4046070 con 0x7f0ef80b3ca0 2024-09-17T04:32:13.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.526+0000 7f0ef5600640 1 --2- 172.21.15.100:0/899614234 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0ed8079250 0x7f0ed807b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:13.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.526+0000 7f0ef5600640 1 -- 172.21.15.100:0/899614234 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5794+0+0 (secure 0 0 0) 0x7f0ee40bbef0 con 0x7f0ef80b3ca0 2024-09-17T04:32:13.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.526+0000 7f0efd400640 1 --2- 172.21.15.100:0/899614234 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0ed8079250 0x7f0ed807b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:13.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.527+0000 7f0efd400640 1 --2- 172.21.15.100:0/899614234 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0ed8079250 0x7f0ed807b760 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0eec009c50 tx=0x7f0eec005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:13.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.531+0000 7f0ef5600640 1 -- 172.21.15.100:0/899614234 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0ee4083370 con 0x7f0ef80b3ca0 2024-09-17T04:32:13.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.737+0000 7f0effe00640 1 -- 172.21.15.100:0/899614234 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7f0ef806fb40 con 0x7f0ed8079250 2024-09-17T04:32:13.741 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:13.739+0000 7f30b3000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-17T04:32:13.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.739+0000 7f0ef5600640 1 -- 172.21.15.100:0/899614234 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f0ef806fb40 con 0x7f0ed8079250 2024-09-17T04:32:13.741 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-17T04:32:13.753 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T04:32:13.754 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm non_existent_subvolume group_1 2024-09-17T04:32:13.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.980+0000 7f98ab800640 1 -- 172.21.15.100:0/2292039593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98a40735e0 msgr2=0x7f98a40739f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:13.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.980+0000 7f98ab800640 1 --2- 172.21.15.100:0/2292039593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98a40735e0 0x7f98a40739f0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f98a41463e0 tx=0x7f989002a560 comp rx=0 tx=0).stop 2024-09-17T04:32:13.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.980+0000 7f98ab800640 1 -- 172.21.15.100:0/2292039593 shutdown_connections 2024-09-17T04:32:13.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.980+0000 7f98ab800640 1 --2- 172.21.15.100:0/2292039593 >> v2:172.21.15.184:3300/0 conn(0x7f98a4160320 0x7f98a4160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:13.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.980+0000 7f98ab800640 1 --2- 172.21.15.100:0/2292039593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98a40735e0 0x7f98a40739f0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:13.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.980+0000 7f98ab800640 1 --2- 172.21.15.100:0/2292039593 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98a4145450 0x7f98a406f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:13.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.980+0000 7f98ab800640 1 -- 172.21.15.100:0/2292039593 >> 172.21.15.100:0/2292039593 conn(0x7f98a406d1c0 msgr2=0x7f98a406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:13.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.981+0000 7f98ab800640 1 -- 172.21.15.100:0/2292039593 shutdown_connections 2024-09-17T04:32:13.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.981+0000 7f98ab800640 1 -- 172.21.15.100:0/2292039593 wait complete. 2024-09-17T04:32:13.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.983+0000 7f98ab800640 1 Processor -- start 2024-09-17T04:32:13.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.983+0000 7f98ab800640 1 -- start start 2024-09-17T04:32:13.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.984+0000 7f98ab800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98a40735e0 0x7f98a40ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:13.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.984+0000 7f98ab800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98a4145450 0x7f98a40ad550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:13.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.984+0000 7f98ab800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98a4160320 0x7f98a40ada90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:13.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.984+0000 7f98ab800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f98a4160c70 con 0x7f98a40735e0 2024-09-17T04:32:13.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.984+0000 7f98ab800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f98a4071020 con 0x7f98a4145450 2024-09-17T04:32:13.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.984+0000 7f98ab800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f98a4160df0 con 0x7f98a4160320 2024-09-17T04:32:13.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.985+0000 7f98a8e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98a40735e0 0x7f98a40ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:13.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.985+0000 7f98a8e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98a40735e0 0x7f98a40ab8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51434/0 (socket says 172.21.15.100:51434) 2024-09-17T04:32:13.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.985+0000 7f98a8e00640 1 -- 172.21.15.100:0/3476573932 learned_addr learned my addr 172.21.15.100:0/3476573932 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:13.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.985+0000 7f98a9800640 1 --2- 172.21.15.100:0/3476573932 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98a4160320 0x7f98a40ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:13.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.985+0000 7f98a3e00640 1 --2- 172.21.15.100:0/3476573932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98a4145450 0x7f98a40ad550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:13.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.985+0000 7f98a8e00640 1 -- 172.21.15.100:0/3476573932 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98a4160320 msgr2=0x7f98a40ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:13.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.985+0000 7f98a8e00640 1 --2- 172.21.15.100:0/3476573932 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98a4160320 0x7f98a40ada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:13.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.985+0000 7f98a8e00640 1 -- 172.21.15.100:0/3476573932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98a4145450 msgr2=0x7f98a40ad550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:13.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.985+0000 7f98a8e00640 1 --2- 172.21.15.100:0/3476573932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98a4145450 0x7f98a40ad550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:13.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.985+0000 7f98a8e00640 1 -- 172.21.15.100:0/3476573932 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a40ac4b0 con 0x7f98a40735e0 2024-09-17T04:32:13.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.985+0000 7f98a8e00640 1 --2- 172.21.15.100:0/3476573932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98a40735e0 0x7f98a40ab8a0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f989800c970 tx=0x7f989800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:13.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.986+0000 7f98a0c00640 1 -- 172.21.15.100:0/3476573932 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9898007b50 con 0x7f98a40735e0 2024-09-17T04:32:13.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.986+0000 7f98a0c00640 1 -- 172.21.15.100:0/3476573932 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9898007de0 con 0x7f98a40735e0 2024-09-17T04:32:13.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.986+0000 7f98ab800640 1 -- 172.21.15.100:0/3476573932 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a40ac790 con 0x7f98a40735e0 2024-09-17T04:32:13.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.986+0000 7f98a0c00640 1 -- 172.21.15.100:0/3476573932 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989800f6b0 con 0x7f98a40735e0 2024-09-17T04:32:13.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.986+0000 7f98ab800640 1 -- 172.21.15.100:0/3476573932 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a40acd50 con 0x7f98a40735e0 2024-09-17T04:32:13.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.987+0000 7f98ab800640 1 -- 172.21.15.100:0/3476573932 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a4070bd0 con 0x7f98a40735e0 2024-09-17T04:32:13.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.989+0000 7f98a0c00640 1 -- 172.21.15.100:0/3476573932 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f989800f850 con 0x7f98a40735e0 2024-09-17T04:32:13.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.989+0000 7f98a0c00640 1 --2- 172.21.15.100:0/3476573932 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9884079250 0x7f988407b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:13.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.990+0000 7f98a0c00640 1 -- 172.21.15.100:0/3476573932 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5794+0+0 (secure 0 0 0) 0x7f989809c190 con 0x7f98a40735e0 2024-09-17T04:32:13.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.990+0000 7f98a3e00640 1 --2- 172.21.15.100:0/3476573932 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9884079250 0x7f988407b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:13.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.990+0000 7f98a3e00640 1 --2- 172.21.15.100:0/3476573932 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9884079250 0x7f988407b760 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9890033000 tx=0x7f9890004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:13.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:13.994+0000 7f98a0c00640 1 -- 172.21.15.100:0/3476573932 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9898014030 con 0x7f98a40735e0 2024-09-17T04:32:14.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.201+0000 7f98ab800640 1 -- 172.21.15.100:0/3476573932 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7f98a415c800 con 0x7f9884079250 2024-09-17T04:32:14.202 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:14.203+0000 7f30b3000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-17T04:32:14.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.203+0000 7f98a0c00640 1 -- 172.21.15.100:0/3476573932 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f9890004040 con 0x7f9884079250 2024-09-17T04:32:14.202 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-17T04:32:14.219 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T04:32:14.219 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup getpath non_existent_subvolume group_1 2024-09-17T04:32:14.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.444+0000 7f8029800640 1 -- 172.21.15.100:0/2298191450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80240cd610 msgr2=0x7f80240cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:14.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.444+0000 7f8029800640 1 --2- 172.21.15.100:0/2298191450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80240cd610 0x7f80240cda20 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f8024071590 tx=0x7f801c02aac0 comp rx=0 tx=0).stop 2024-09-17T04:32:14.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.444+0000 7f8029800640 1 -- 172.21.15.100:0/2298191450 shutdown_connections 2024-09-17T04:32:14.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.444+0000 7f8029800640 1 --2- 172.21.15.100:0/2298191450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80240cd610 0x7f80240cda20 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:14.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.444+0000 7f8029800640 1 --2- 172.21.15.100:0/2298191450 >> v2:172.21.15.184:3300/0 conn(0x7f80240735e0 0x7f80240739f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:14.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.444+0000 7f8029800640 1 -- 172.21.15.100:0/2298191450 >> 172.21.15.100:0/2298191450 conn(0x7f802406d1c0 msgr2=0x7f802406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:14.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.445+0000 7f8029800640 1 -- 172.21.15.100:0/2298191450 shutdown_connections 2024-09-17T04:32:14.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.445+0000 7f8029800640 1 -- 172.21.15.100:0/2298191450 wait complete. 2024-09-17T04:32:14.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.447+0000 7f8029800640 1 Processor -- start 2024-09-17T04:32:14.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.447+0000 7f8029800640 1 -- start start 2024-09-17T04:32:14.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.448+0000 7f8029800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80240735e0 0x7f80241f8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:14.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.448+0000 7f8029800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80240cd610 0x7f80240b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:14.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.448+0000 7f8029800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8024145450 0x7f8024144060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:14.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.448+0000 7f8029800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f80240ce2f0 con 0x7f80240735e0 2024-09-17T04:32:14.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.448+0000 7f8029800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f80240ce170 con 0x7f8024145450 2024-09-17T04:32:14.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.448+0000 7f8029800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f80240ce470 con 0x7f80240cd610 2024-09-17T04:32:14.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.449+0000 7f8022a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80240735e0 0x7f80241f8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:14.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.449+0000 7f8022a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80240735e0 0x7f80241f8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51458/0 (socket says 172.21.15.100:51458) 2024-09-17T04:32:14.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.449+0000 7f8022a00640 1 -- 172.21.15.100:0/1618952150 learned_addr learned my addr 172.21.15.100:0/1618952150 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:14.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.449+0000 7f8023400640 1 --2- 172.21.15.100:0/1618952150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8024145450 0x7f8024144060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:14.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.449+0000 7f8022a00640 1 -- 172.21.15.100:0/1618952150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80240cd610 msgr2=0x7f80240b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:14.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.449+0000 7f801be00640 1 --2- 172.21.15.100:0/1618952150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80240cd610 0x7f80240b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:14.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.449+0000 7f8022a00640 1 --2- 172.21.15.100:0/1618952150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80240cd610 0x7f80240b4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:14.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.449+0000 7f8022a00640 1 -- 172.21.15.100:0/1618952150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8024145450 msgr2=0x7f8024144060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:14.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.449+0000 7f8022a00640 1 --2- 172.21.15.100:0/1618952150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8024145450 0x7f8024144060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:14.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.449+0000 7f8022a00640 1 -- 172.21.15.100:0/1618952150 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80241f5990 con 0x7f80240735e0 2024-09-17T04:32:14.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.449+0000 7f8023400640 1 --2- 172.21.15.100:0/1618952150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8024145450 0x7f8024144060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:32:14.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.449+0000 7f801be00640 1 --2- 172.21.15.100:0/1618952150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80240cd610 0x7f80240b4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:32:14.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.449+0000 7f8022a00640 1 --2- 172.21.15.100:0/1618952150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80240735e0 0x7f80241f8700 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f801400c3a0 tx=0x7f801400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:14.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.450+0000 7f801aa00640 1 -- 172.21.15.100:0/1618952150 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8014017070 con 0x7f80240735e0 2024-09-17T04:32:14.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.450+0000 7f801aa00640 1 -- 172.21.15.100:0/1618952150 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80140053c0 con 0x7f80240735e0 2024-09-17T04:32:14.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.450+0000 7f8029800640 1 -- 172.21.15.100:0/1618952150 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80241f5c70 con 0x7f80240735e0 2024-09-17T04:32:14.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.450+0000 7f801aa00640 1 -- 172.21.15.100:0/1618952150 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8014003850 con 0x7f80240735e0 2024-09-17T04:32:14.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.450+0000 7f8029800640 1 -- 172.21.15.100:0/1618952150 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80241f6230 con 0x7f80240735e0 2024-09-17T04:32:14.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.451+0000 7f8029800640 1 -- 172.21.15.100:0/1618952150 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f802406f100 con 0x7f80240735e0 2024-09-17T04:32:14.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.452+0000 7f801aa00640 1 -- 172.21.15.100:0/1618952150 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f801401a070 con 0x7f80240735e0 2024-09-17T04:32:14.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.453+0000 7f801aa00640 1 --2- 172.21.15.100:0/1618952150 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8000079250 0x7f800007b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:14.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.453+0000 7f801aa00640 1 -- 172.21.15.100:0/1618952150 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5794+0+0 (secure 0 0 0) 0x7f801409af40 con 0x7f80240735e0 2024-09-17T04:32:14.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.453+0000 7f801be00640 1 --2- 172.21.15.100:0/1618952150 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8000079250 0x7f800007b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:14.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.454+0000 7f801be00640 1 --2- 172.21.15.100:0/1618952150 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8000079250 0x7f800007b760 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8024143fa0 tx=0x7f800c005df0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:14.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.457+0000 7f801aa00640 1 -- 172.21.15.100:0/1618952150 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8024073f30 con 0x7f80240735e0 2024-09-17T04:32:14.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.666+0000 7f8029800640 1 -- 172.21.15.100:0/1618952150 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup getpath", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7f80240739f0 con 0x7f8000079250 2024-09-17T04:32:14.667 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:14.668+0000 7f30b3000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-17T04:32:14.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.668+0000 7f801aa00640 1 -- 172.21.15.100:0/1618952150 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f80240739f0 con 0x7f8000079250 2024-09-17T04:32:14.668 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-17T04:32:14.687 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T04:32:14.687 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup pin non_existent_subvolume group_1 export 0 2024-09-17T04:32:14.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.913+0000 7f9c2d600640 1 -- 172.21.15.100:0/2913593836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c280af100 msgr2=0x7f9c280af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:14.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.913+0000 7f9c2d600640 1 --2- 172.21.15.100:0/2913593836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c280af100 0x7f9c280af510 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f9c20005fd0 tx=0x7f9c2002aa90 comp rx=0 tx=0).stop 2024-09-17T04:32:14.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.913+0000 7f9c2d600640 1 -- 172.21.15.100:0/2913593836 shutdown_connections 2024-09-17T04:32:14.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.913+0000 7f9c2d600640 1 --2- 172.21.15.100:0/2913593836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c280af100 0x7f9c280af510 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:14.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.913+0000 7f9c2d600640 1 --2- 172.21.15.100:0/2913593836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c2806d4b0 0x7f9c280b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:14.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.913+0000 7f9c2d600640 1 -- 172.21.15.100:0/2913593836 >> 172.21.15.100:0/2913593836 conn(0x7f9c280b2540 msgr2=0x7f9c280d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:14.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.913+0000 7f9c2d600640 1 -- 172.21.15.100:0/2913593836 shutdown_connections 2024-09-17T04:32:14.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.913+0000 7f9c2d600640 1 -- 172.21.15.100:0/2913593836 wait complete. 2024-09-17T04:32:14.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.915+0000 7f9c2d600640 1 Processor -- start 2024-09-17T04:32:14.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.915+0000 7f9c2d600640 1 -- start start 2024-09-17T04:32:14.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.917+0000 7f9c2d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c280af100 0x7f9c28153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:14.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.917+0000 7f9c2d600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c280cc830 0x7f9c281f5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:14.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.917+0000 7f9c2d600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c28153300 0x7f9c280ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:14.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.917+0000 7f9c2d600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9c280b35f0 con 0x7f9c280af100 2024-09-17T04:32:14.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.917+0000 7f9c2d600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9c280b3470 con 0x7f9c280cc830 2024-09-17T04:32:14.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.917+0000 7f9c2d600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9c280b3770 con 0x7f9c28153300 2024-09-17T04:32:14.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.917+0000 7f9c26a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c280af100 0x7f9c28153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:14.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.917+0000 7f9c26a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c280af100 0x7f9c28153f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40788/0 (socket says 172.21.15.100:40788) 2024-09-17T04:32:14.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.917+0000 7f9c26a00640 1 -- 172.21.15.100:0/1471095368 learned_addr learned my addr 172.21.15.100:0/1471095368 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:14.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.917+0000 7f9c27400640 1 --2- 172.21.15.100:0/1471095368 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c28153300 0x7f9c280ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:14.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.917+0000 7f9c26000640 1 --2- 172.21.15.100:0/1471095368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c280cc830 0x7f9c281f5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:14.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.917+0000 7f9c26a00640 1 -- 172.21.15.100:0/1471095368 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c28153300 msgr2=0x7f9c280ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:14.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.917+0000 7f9c26a00640 1 --2- 172.21.15.100:0/1471095368 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c28153300 0x7f9c280ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:14.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.917+0000 7f9c26a00640 1 -- 172.21.15.100:0/1471095368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c280cc830 msgr2=0x7f9c281f5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:14.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.917+0000 7f9c26a00640 1 --2- 172.21.15.100:0/1471095368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c280cc830 0x7f9c281f5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:14.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.917+0000 7f9c26a00640 1 -- 172.21.15.100:0/1471095368 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c281f64a0 con 0x7f9c280af100 2024-09-17T04:32:14.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.918+0000 7f9c26a00640 1 --2- 172.21.15.100:0/1471095368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c280af100 0x7f9c28153f90 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800c3d0 tx=0x7f9c1800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:14.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.918+0000 7f9c1ea00640 1 -- 172.21.15.100:0/1471095368 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c18017070 con 0x7f9c280af100 2024-09-17T04:32:14.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.918+0000 7f9c1ea00640 1 -- 172.21.15.100:0/1471095368 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c180053c0 con 0x7f9c280af100 2024-09-17T04:32:14.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.918+0000 7f9c1ea00640 1 -- 172.21.15.100:0/1471095368 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c18003850 con 0x7f9c280af100 2024-09-17T04:32:14.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.918+0000 7f9c2d600640 1 -- 172.21.15.100:0/1471095368 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c280ce0e0 con 0x7f9c280af100 2024-09-17T04:32:14.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.918+0000 7f9c2d600640 1 -- 172.21.15.100:0/1471095368 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c280ce6a0 con 0x7f9c280af100 2024-09-17T04:32:14.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.919+0000 7f9c2d600640 1 -- 172.21.15.100:0/1471095368 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c280b3c90 con 0x7f9c280af100 2024-09-17T04:32:14.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.920+0000 7f9c1ea00640 1 -- 172.21.15.100:0/1471095368 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f9c1801a070 con 0x7f9c280af100 2024-09-17T04:32:14.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.921+0000 7f9c1ea00640 1 --2- 172.21.15.100:0/1471095368 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9c08079250 0x7f9c0807b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:14.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.921+0000 7f9c1ea00640 1 -- 172.21.15.100:0/1471095368 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5794+0+0 (secure 0 0 0) 0x7f9c1809ad90 con 0x7f9c280af100 2024-09-17T04:32:14.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.922+0000 7f9c26000640 1 --2- 172.21.15.100:0/1471095368 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9c08079250 0x7f9c0807b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:14.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.922+0000 7f9c26000640 1 --2- 172.21.15.100:0/1471095368 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9c08079250 0x7f9c0807b760 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9c14005fd0 tx=0x7f9c14005850 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:14.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:14.926+0000 7f9c1ea00640 1 -- 172.21.15.100:0/1471095368 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9c180a1080 con 0x7f9c280af100 2024-09-17T04:32:15.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.136+0000 7f9c2d600640 1 -- 172.21.15.100:0/1471095368 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "non_existent_subvolume", "group_name": "group_1", "pin_type": "export", "pin_setting": "0", "target": ["mon-mgr", ""]}) -- 0x7f9c281f69a0 con 0x7f9c08079250 2024-09-17T04:32:15.136 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:15.137+0000 7f30b3000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-17T04:32:15.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.137+0000 7f9c1ea00640 1 -- 172.21.15.100:0/1471095368 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f9c281f69a0 con 0x7f9c08079250 2024-09-17T04:32:15.137 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-17T04:32:15.153 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T04:32:15.153 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup ls non_existent_subvolume 2024-09-17T04:32:15.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.375+0000 7f3b3b400640 1 -- 172.21.15.100:0/3703181558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b3406fe70 msgr2=0x7f3b34151950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:15.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.375+0000 7f3b3b400640 1 --2- 172.21.15.100:0/3703181558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b3406fe70 0x7f3b34151950 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400b3f0 tx=0x7f3b2402ae50 comp rx=0 tx=0).stop 2024-09-17T04:32:15.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.375+0000 7f3b3b400640 1 -- 172.21.15.100:0/3703181558 shutdown_connections 2024-09-17T04:32:15.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.375+0000 7f3b3b400640 1 --2- 172.21.15.100:0/3703181558 >> v2:172.21.15.184:3300/0 conn(0x7f3b34073630 0x7f3b34073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:15.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.375+0000 7f3b3b400640 1 --2- 172.21.15.100:0/3703181558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b3406fe70 0x7f3b34151950 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:15.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.375+0000 7f3b3b400640 1 -- 172.21.15.100:0/3703181558 >> 172.21.15.100:0/3703181558 conn(0x7f3b3406d1c0 msgr2=0x7f3b3406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:15.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.375+0000 7f3b3b400640 1 -- 172.21.15.100:0/3703181558 shutdown_connections 2024-09-17T04:32:15.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.375+0000 7f3b3b400640 1 -- 172.21.15.100:0/3703181558 wait complete. 2024-09-17T04:32:15.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.377+0000 7f3b3b400640 1 Processor -- start 2024-09-17T04:32:15.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.378+0000 7f3b3b400640 1 -- start start 2024-09-17T04:32:15.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.379+0000 7f3b3b400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b34073630 0x7f3b341e49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:15.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.379+0000 7f3b3b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b3415af90 0x7f3b3414f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:15.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.379+0000 7f3b3b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b3413ff40 0x7f3b3414fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:15.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.379+0000 7f3b3b400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3b340b3b20 con 0x7f3b3413ff40 2024-09-17T04:32:15.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.379+0000 7f3b3b400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3b340b39a0 con 0x7f3b3415af90 2024-09-17T04:32:15.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.379+0000 7f3b3b400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3b340b3ca0 con 0x7f3b34073630 2024-09-17T04:32:15.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.379+0000 7f3b39400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b3413ff40 0x7f3b3414fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:15.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.379+0000 7f3b39400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b3413ff40 0x7f3b3414fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40814/0 (socket says 172.21.15.100:40814) 2024-09-17T04:32:15.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.379+0000 7f3b39400640 1 -- 172.21.15.100:0/2126133097 learned_addr learned my addr 172.21.15.100:0/2126133097 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:15.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.380+0000 7f3b2be00640 1 --2- 172.21.15.100:0/2126133097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b3415af90 0x7f3b3414f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:15.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.380+0000 7f3b38a00640 1 --2- 172.21.15.100:0/2126133097 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b34073630 0x7f3b341e49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:15.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.380+0000 7f3b39400640 1 -- 172.21.15.100:0/2126133097 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b34073630 msgr2=0x7f3b341e49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:15.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.380+0000 7f3b39400640 1 --2- 172.21.15.100:0/2126133097 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b34073630 0x7f3b341e49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:15.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.380+0000 7f3b39400640 1 -- 172.21.15.100:0/2126133097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b3415af90 msgr2=0x7f3b3414f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:15.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.380+0000 7f3b39400640 1 --2- 172.21.15.100:0/2126133097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b3415af90 0x7f3b3414f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:15.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.380+0000 7f3b39400640 1 -- 172.21.15.100:0/2126133097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b341f6fd0 con 0x7f3b3413ff40 2024-09-17T04:32:15.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.380+0000 7f3b39400640 1 --2- 172.21.15.100:0/2126133097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b3413ff40 0x7f3b3414fa00 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c00b9e0 tx=0x7f3b2c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:15.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.380+0000 7f3b31600640 1 -- 172.21.15.100:0/2126133097 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c019070 con 0x7f3b3413ff40 2024-09-17T04:32:15.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.380+0000 7f3b31600640 1 -- 172.21.15.100:0/2126133097 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b2c00cc10 con 0x7f3b3413ff40 2024-09-17T04:32:15.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.380+0000 7f3b3b400640 1 -- 172.21.15.100:0/2126133097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b341f72b0 con 0x7f3b3413ff40 2024-09-17T04:32:15.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.380+0000 7f3b31600640 1 -- 172.21.15.100:0/2126133097 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c014440 con 0x7f3b3413ff40 2024-09-17T04:32:15.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.381+0000 7f3b3b400640 1 -- 172.21.15.100:0/2126133097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b341f7870 con 0x7f3b3413ff40 2024-09-17T04:32:15.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.382+0000 7f3b3b400640 1 -- 172.21.15.100:0/2126133097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b3406fe70 con 0x7f3b3413ff40 2024-09-17T04:32:15.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.383+0000 7f3b31600640 1 -- 172.21.15.100:0/2126133097 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f3b2c01c070 con 0x7f3b3413ff40 2024-09-17T04:32:15.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.384+0000 7f3b31600640 1 --2- 172.21.15.100:0/2126133097 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3b10079250 0x7f3b1007b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:15.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.384+0000 7f3b31600640 1 -- 172.21.15.100:0/2126133097 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5794+0+0 (secure 0 0 0) 0x7f3b2c09b5e0 con 0x7f3b3413ff40 2024-09-17T04:32:15.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.384+0000 7f3b38a00640 1 --2- 172.21.15.100:0/2126133097 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3b10079250 0x7f3b1007b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:15.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.384+0000 7f3b38a00640 1 --2- 172.21.15.100:0/2126133097 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3b10079250 0x7f3b1007b760 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3b24034000 tx=0x7f3b24004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:15.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.388+0000 7f3b31600640 1 -- 172.21.15.100:0/2126133097 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3b2c062a20 con 0x7f3b3413ff40 2024-09-17T04:32:15.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.594+0000 7f3b3b400640 1 -- 172.21.15.100:0/2126133097 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup ls", "vol_name": "non_existent_subvolume", "target": ["mon-mgr", ""]}) -- 0x7f3b3415aa90 con 0x7f3b10079250 2024-09-17T04:32:15.595 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:15.596+0000 7f30b3000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-17T04:32:15.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.596+0000 7f3b31600640 1 -- 172.21.15.100:0/2126133097 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f3b3415aa90 con 0x7f3b10079250 2024-09-17T04:32:15.596 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-17T04:32:15.612 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T04:32:15.612 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot create non_existent_subvolume group_1 snapshot_1 2024-09-17T04:32:15.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.840+0000 7efe3f200640 1 -- 172.21.15.100:0/3811400521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe380af810 msgr2=0x7efe380b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:15.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.840+0000 7efe3f200640 1 --2- 172.21.15.100:0/3811400521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe380af810 0x7efe380b4620 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7efe30005fd0 tx=0x7efe3002a940 comp rx=0 tx=0).stop 2024-09-17T04:32:15.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.840+0000 7efe3f200640 1 -- 172.21.15.100:0/3811400521 shutdown_connections 2024-09-17T04:32:15.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.840+0000 7efe3f200640 1 --2- 172.21.15.100:0/3811400521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe380af810 0x7efe380b4620 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:15.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.840+0000 7efe3f200640 1 -- 172.21.15.100:0/3811400521 >> 172.21.15.100:0/3811400521 conn(0x7efe380b2540 msgr2=0x7efe380d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:15.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.840+0000 7efe3f200640 1 -- 172.21.15.100:0/3811400521 shutdown_connections 2024-09-17T04:32:15.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.840+0000 7efe3f200640 1 -- 172.21.15.100:0/3811400521 wait complete. 2024-09-17T04:32:15.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.843+0000 7efe3f200640 1 Processor -- start 2024-09-17T04:32:15.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.843+0000 7efe3f200640 1 -- start start 2024-09-17T04:32:15.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.844+0000 7efe3f200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe380af810 0x7efe3815bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:15.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.844+0000 7efe3f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe380cc830 0x7efe38154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:15.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.844+0000 7efe3f200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe3815c060 0x7efe38154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:15.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.844+0000 7efe3f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efe380b3780 con 0x7efe380cc830 2024-09-17T04:32:15.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.844+0000 7efe3f200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efe380b3600 con 0x7efe3815c060 2024-09-17T04:32:15.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.844+0000 7efe3f200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efe380b3900 con 0x7efe380af810 2024-09-17T04:32:15.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.844+0000 7efe2f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe380cc830 0x7efe38154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:15.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.844+0000 7efe2f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe380cc830 0x7efe38154510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40844/0 (socket says 172.21.15.100:40844) 2024-09-17T04:32:15.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.844+0000 7efe2f400640 1 -- 172.21.15.100:0/2948202921 learned_addr learned my addr 172.21.15.100:0/2948202921 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:15.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.845+0000 7efe37e00640 1 --2- 172.21.15.100:0/2948202921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe380af810 0x7efe3815bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:15.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.845+0000 7efe3d200640 1 --2- 172.21.15.100:0/2948202921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe3815c060 0x7efe38154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:15.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.845+0000 7efe2f400640 1 -- 172.21.15.100:0/2948202921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe380af810 msgr2=0x7efe3815bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:15.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.845+0000 7efe2f400640 1 --2- 172.21.15.100:0/2948202921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe380af810 0x7efe3815bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:15.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.845+0000 7efe2f400640 1 -- 172.21.15.100:0/2948202921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe3815c060 msgr2=0x7efe38154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:15.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.845+0000 7efe2f400640 1 --2- 172.21.15.100:0/2948202921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe3815c060 0x7efe38154a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:15.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.845+0000 7efe2f400640 1 -- 172.21.15.100:0/2948202921 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe381f9900 con 0x7efe380cc830 2024-09-17T04:32:15.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.845+0000 7efe37e00640 1 --2- 172.21.15.100:0/2948202921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe380af810 0x7efe3815bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:15.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.845+0000 7efe2f400640 1 --2- 172.21.15.100:0/2948202921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe380cc830 0x7efe38154510 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7efe2000c370 tx=0x7efe2000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:15.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.845+0000 7efe34c00640 1 -- 172.21.15.100:0/2948202921 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe20017070 con 0x7efe380cc830 2024-09-17T04:32:15.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.845+0000 7efe34c00640 1 -- 172.21.15.100:0/2948202921 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe200053c0 con 0x7efe380cc830 2024-09-17T04:32:15.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.846+0000 7efe34c00640 1 -- 172.21.15.100:0/2948202921 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe20003850 con 0x7efe380cc830 2024-09-17T04:32:15.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.846+0000 7efe3f200640 1 -- 172.21.15.100:0/2948202921 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe381f9b70 con 0x7efe380cc830 2024-09-17T04:32:15.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.846+0000 7efe3f200640 1 -- 172.21.15.100:0/2948202921 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe381fa130 con 0x7efe380cc830 2024-09-17T04:32:15.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.847+0000 7efe3f200640 1 -- 172.21.15.100:0/2948202921 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe380b3db0 con 0x7efe380cc830 2024-09-17T04:32:15.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.848+0000 7efe34c00640 1 -- 172.21.15.100:0/2948202921 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7efe2001a070 con 0x7efe380cc830 2024-09-17T04:32:15.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.849+0000 7efe34c00640 1 --2- 172.21.15.100:0/2948202921 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7efe14079250 0x7efe1407b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:15.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.849+0000 7efe34c00640 1 -- 172.21.15.100:0/2948202921 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5794+0+0 (secure 0 0 0) 0x7efe2009ae10 con 0x7efe380cc830 2024-09-17T04:32:15.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.849+0000 7efe37e00640 1 --2- 172.21.15.100:0/2948202921 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7efe14079250 0x7efe1407b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:15.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.850+0000 7efe37e00640 1 --2- 172.21.15.100:0/2948202921 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7efe14079250 0x7efe1407b760 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efe280046b0 tx=0x7efe28009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:15.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:15.853+0000 7efe34c00640 1 -- 172.21.15.100:0/2948202921 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7efe20062030 con 0x7efe380cc830 2024-09-17T04:32:16.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.066+0000 7efe3f200640 1 -- 172.21.15.100:0/2948202921 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot create", "vol_name": "non_existent_subvolume", "group_name": "group_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7efe380d1030 con 0x7efe14079250 2024-09-17T04:32:16.066 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:16.067+0000 7f30b3000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-17T04:32:16.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.067+0000 7efe34c00640 1 -- 172.21.15.100:0/2948202921 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7efe380d1030 con 0x7efe14079250 2024-09-17T04:32:16.067 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-17T04:32:16.082 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T04:32:16.082 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot rm non_existent_subvolume group_1 snapshot_1 2024-09-17T04:32:16.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.307+0000 7f79b1600640 1 -- 172.21.15.100:0/3627881564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79ac0cc830 msgr2=0x7f79ac0b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:16.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.307+0000 7f79b1600640 1 --2- 172.21.15.100:0/3627881564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79ac0cc830 0x7f79ac0b4620 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0d0d00 tx=0x7f799c02a5f0 comp rx=0 tx=0).stop 2024-09-17T04:32:16.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.307+0000 7f79b1600640 1 -- 172.21.15.100:0/3627881564 shutdown_connections 2024-09-17T04:32:16.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.307+0000 7f79b1600640 1 --2- 172.21.15.100:0/3627881564 >> v2:172.21.15.184:3300/0 conn(0x7f79ac06cda0 0x7f79ac06d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:16.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.307+0000 7f79b1600640 1 --2- 172.21.15.100:0/3627881564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79ac0cc830 0x7f79ac0b4620 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:16.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.307+0000 7f79b1600640 1 -- 172.21.15.100:0/3627881564 >> 172.21.15.100:0/3627881564 conn(0x7f79ac0b2540 msgr2=0x7f79ac0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:16.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.307+0000 7f79b1600640 1 -- 172.21.15.100:0/3627881564 shutdown_connections 2024-09-17T04:32:16.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.307+0000 7f79b1600640 1 -- 172.21.15.100:0/3627881564 wait complete. 2024-09-17T04:32:16.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.310+0000 7f79b1600640 1 Processor -- start 2024-09-17T04:32:16.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.310+0000 7f79b1600640 1 -- start start 2024-09-17T04:32:16.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.311+0000 7f79b1600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79ac06cda0 0x7f79ac098a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:16.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.311+0000 7f79b1600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79ac0af100 0x7f79ac1559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:16.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.311+0000 7f79b1600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79ac0cc830 0x7f79ac09aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:16.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.311+0000 7f79b1600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac0b35f0 con 0x7f79ac0cc830 2024-09-17T04:32:16.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.311+0000 7f79b1600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac0b3470 con 0x7f79ac0af100 2024-09-17T04:32:16.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.311+0000 7f79b1600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f79ac0b3770 con 0x7f79ac06cda0 2024-09-17T04:32:16.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.311+0000 7f79ab400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79ac0cc830 0x7f79ac09aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:16.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.311+0000 7f79ab400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79ac0cc830 0x7f79ac09aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40868/0 (socket says 172.21.15.100:40868) 2024-09-17T04:32:16.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.311+0000 7f79ab400640 1 -- 172.21.15.100:0/1017118609 learned_addr learned my addr 172.21.15.100:0/1017118609 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:16.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.312+0000 7f79aaa00640 1 --2- 172.21.15.100:0/1017118609 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79ac06cda0 0x7f79ac098a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:16.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.312+0000 7f79aa000640 1 --2- 172.21.15.100:0/1017118609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79ac0af100 0x7f79ac1559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:16.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.312+0000 7f79ab400640 1 -- 172.21.15.100:0/1017118609 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79ac06cda0 msgr2=0x7f79ac098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:16.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.312+0000 7f79ab400640 1 --2- 172.21.15.100:0/1017118609 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79ac06cda0 0x7f79ac098a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:16.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.312+0000 7f79ab400640 1 -- 172.21.15.100:0/1017118609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79ac0af100 msgr2=0x7f79ac1559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:16.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.312+0000 7f79ab400640 1 --2- 172.21.15.100:0/1017118609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79ac0af100 0x7f79ac1559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:16.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.312+0000 7f79ab400640 1 -- 172.21.15.100:0/1017118609 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79ac098f40 con 0x7f79ac0cc830 2024-09-17T04:32:16.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.312+0000 7f79ab400640 1 --2- 172.21.15.100:0/1017118609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79ac0cc830 0x7f79ac09aca0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f79a400c760 tx=0x7f79a400cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:16.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.312+0000 7f79a2a00640 1 -- 172.21.15.100:0/1017118609 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a4003b70 con 0x7f79ac0cc830 2024-09-17T04:32:16.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.313+0000 7f79a2a00640 1 -- 172.21.15.100:0/1017118609 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79a4003d10 con 0x7f79ac0cc830 2024-09-17T04:32:16.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.313+0000 7f79a2a00640 1 -- 172.21.15.100:0/1017118609 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a4005330 con 0x7f79ac0cc830 2024-09-17T04:32:16.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.313+0000 7f79b1600640 1 -- 172.21.15.100:0/1017118609 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79ac155f00 con 0x7f79ac0cc830 2024-09-17T04:32:16.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.313+0000 7f79b1600640 1 -- 172.21.15.100:0/1017118609 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79ac069670 con 0x7f79ac0cc830 2024-09-17T04:32:16.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.314+0000 7f79b1600640 1 -- 172.21.15.100:0/1017118609 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79ac0b3c90 con 0x7f79ac0cc830 2024-09-17T04:32:16.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.315+0000 7f79a2a00640 1 -- 172.21.15.100:0/1017118609 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f79a40054d0 con 0x7f79ac0cc830 2024-09-17T04:32:16.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.316+0000 7f79a2a00640 1 --2- 172.21.15.100:0/1017118609 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f798c079280 0x7f798c07b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:16.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.316+0000 7f79a2a00640 1 -- 172.21.15.100:0/1017118609 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5794+0+0 (secure 0 0 0) 0x7f79a409b5c0 con 0x7f79ac0cc830 2024-09-17T04:32:16.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.316+0000 7f79aaa00640 1 --2- 172.21.15.100:0/1017118609 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f798c079280 0x7f798c07b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:16.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.317+0000 7f79aaa00640 1 --2- 172.21.15.100:0/1017118609 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f798c079280 0x7f798c07b790 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f799c033000 tx=0x7f799c004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:16.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.321+0000 7f79a2a00640 1 -- 172.21.15.100:0/1017118609 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f79ac098f40 con 0x7f79ac0cc830 2024-09-17T04:32:16.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.532+0000 7f79b1600640 1 -- 172.21.15.100:0/1017118609 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot rm", "vol_name": "non_existent_subvolume", "group_name": "group_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f79ac069840 con 0x7f798c079280 2024-09-17T04:32:16.532 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:16.533+0000 7f30b3000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-17T04:32:16.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.533+0000 7f79a2a00640 1 -- 172.21.15.100:0/1017118609 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f79ac069840 con 0x7f798c079280 2024-09-17T04:32:16.533 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-17T04:32:16.550 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T04:32:16.550 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot ls non_existent_subvolume group_1 2024-09-17T04:32:16.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.776+0000 7fbf45c00640 1 -- 172.21.15.100:0/925557856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf240039e0 msgr2=0x7fbf24003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:16.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.776+0000 7fbf45c00640 1 --2- 172.21.15.100:0/925557856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf240039e0 0x7fbf24003e10 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009b50 tx=0x7fbf24005830 comp rx=0 tx=0).stop 2024-09-17T04:32:16.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.776+0000 7fbf3f400640 1 -- 172.21.15.100:0/925557856 reap_dead start 2024-09-17T04:32:16.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.776+0000 7fbf45c00640 1 -- 172.21.15.100:0/925557856 shutdown_connections 2024-09-17T04:32:16.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.776+0000 7fbf45c00640 1 -- 172.21.15.100:0/925557856 >> 172.21.15.100:0/925557856 conn(0x7fbf400b2540 msgr2=0x7fbf400d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:16.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.776+0000 7fbf45c00640 1 -- 172.21.15.100:0/925557856 shutdown_connections 2024-09-17T04:32:16.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.776+0000 7fbf45c00640 1 -- 172.21.15.100:0/925557856 wait complete. 2024-09-17T04:32:16.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.779+0000 7fbf45c00640 1 Processor -- start 2024-09-17T04:32:16.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.779+0000 7fbf45c00640 1 -- start start 2024-09-17T04:32:16.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.780+0000 7fbf45c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf240039e0 0x7fbf401f2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:16.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.780+0000 7fbf45c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf40073250 0x7fbf400c4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:16.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.780+0000 7fbf45c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf400c49c0 0x7fbf4009b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:16.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.780+0000 7fbf45c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbf400b32a0 con 0x7fbf40073250 2024-09-17T04:32:16.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.780+0000 7fbf45c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbf400b3120 con 0x7fbf240039e0 2024-09-17T04:32:16.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.780+0000 7fbf45c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbf400b3420 con 0x7fbf400c49c0 2024-09-17T04:32:16.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.780+0000 7fbf3e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf40073250 0x7fbf400c4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:16.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.780+0000 7fbf3e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf40073250 0x7fbf400c4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40880/0 (socket says 172.21.15.100:40880) 2024-09-17T04:32:16.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.780+0000 7fbf3e000640 1 -- 172.21.15.100:0/938387966 learned_addr learned my addr 172.21.15.100:0/938387966 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:16.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.781+0000 7fbf3e000640 1 -- 172.21.15.100:0/938387966 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf400c49c0 msgr2=0x7fbf4009b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:16.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.781+0000 7fbf3e000640 1 --2- 172.21.15.100:0/938387966 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf400c49c0 0x7fbf4009b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:16.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.781+0000 7fbf3e000640 1 -- 172.21.15.100:0/938387966 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf240039e0 msgr2=0x7fbf401f2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:16.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.781+0000 7fbf3e000640 1 --2- 172.21.15.100:0/938387966 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf240039e0 0x7fbf401f2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:16.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.781+0000 7fbf3e000640 1 -- 172.21.15.100:0/938387966 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf400c4dd0 con 0x7fbf40073250 2024-09-17T04:32:16.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.781+0000 7fbf3e000640 1 --2- 172.21.15.100:0/938387966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf40073250 0x7fbf400c4480 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c006630 tx=0x7fbf2c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:16.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.781+0000 7fbf36a00640 1 -- 172.21.15.100:0/938387966 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c015070 con 0x7fbf40073250 2024-09-17T04:32:16.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.781+0000 7fbf36a00640 1 -- 172.21.15.100:0/938387966 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf2c011900 con 0x7fbf40073250 2024-09-17T04:32:16.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.781+0000 7fbf36a00640 1 -- 172.21.15.100:0/938387966 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c011d30 con 0x7fbf40073250 2024-09-17T04:32:16.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.781+0000 7fbf45c00640 1 -- 172.21.15.100:0/938387966 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf4009daf0 con 0x7fbf40073250 2024-09-17T04:32:16.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.782+0000 7fbf45c00640 1 -- 172.21.15.100:0/938387966 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf401f3d20 con 0x7fbf40073250 2024-09-17T04:32:16.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.783+0000 7fbf45c00640 1 -- 172.21.15.100:0/938387966 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf401f3b10 con 0x7fbf40073250 2024-09-17T04:32:16.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.784+0000 7fbf36a00640 1 -- 172.21.15.100:0/938387966 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7fbf2c018070 con 0x7fbf40073250 2024-09-17T04:32:16.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.785+0000 7fbf36a00640 1 --2- 172.21.15.100:0/938387966 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbf20079250 0x7fbf2007b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:16.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.785+0000 7fbf36a00640 1 -- 172.21.15.100:0/938387966 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5794+0+0 (secure 0 0 0) 0x7fbf2c09c6c0 con 0x7fbf40073250 2024-09-17T04:32:16.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.785+0000 7fbf3ea00640 1 --2- 172.21.15.100:0/938387966 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbf20079250 0x7fbf2007b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:16.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.786+0000 7fbf3ea00640 1 --2- 172.21.15.100:0/938387966 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbf20079250 0x7fbf2007b760 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbf30035000 tx=0x7fbf30004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:16.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:16.789+0000 7fbf36a00640 1 -- 172.21.15.100:0/938387966 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fbf400c4dd0 con 0x7fbf40073250 2024-09-17T04:32:17.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.002+0000 7fbf45c00640 1 -- 172.21.15.100:0/938387966 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot ls", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7fbf4009e250 con 0x7fbf20079250 2024-09-17T04:32:17.003 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:17.003+0000 7f30b3000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-17T04:32:17.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.004+0000 7fbf36a00640 1 -- 172.21.15.100:0/938387966 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fbf4009e250 con 0x7fbf20079250 2024-09-17T04:32:17.003 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-17T04:32:17.023 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T04:32:17.023 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T04:32:17.024 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:32:17.103 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:32:17.198 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T04:32:17.198 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:32:17.198 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:32:17.198 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:32:17.234 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:32:17.234 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:32:17.310 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:32:17.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.572+0000 7f7b83c00640 1 -- 172.21.15.100:0/2200503509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b7c070cc0 msgr2=0x7f7b7c156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:17.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.572+0000 7f7b83c00640 1 --2- 172.21.15.100:0/2200503509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b7c070cc0 0x7f7b7c156e40 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009900 tx=0x7f7b6402a610 comp rx=0 tx=0).stop 2024-09-17T04:32:17.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.572+0000 7f7b83c00640 1 -- 172.21.15.100:0/2200503509 shutdown_connections 2024-09-17T04:32:17.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.572+0000 7f7b83c00640 1 --2- 172.21.15.100:0/2200503509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b7c070cc0 0x7f7b7c156e40 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:17.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.572+0000 7f7b83c00640 1 -- 172.21.15.100:0/2200503509 >> 172.21.15.100:0/2200503509 conn(0x7f7b7c06d3d0 msgr2=0x7f7b7c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:17.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.572+0000 7f7b83c00640 1 -- 172.21.15.100:0/2200503509 shutdown_connections 2024-09-17T04:32:17.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.572+0000 7f7b83c00640 1 -- 172.21.15.100:0/2200503509 wait complete. 2024-09-17T04:32:17.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.574+0000 7f7b83c00640 1 Processor -- start 2024-09-17T04:32:17.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.575+0000 7f7b83c00640 1 -- start start 2024-09-17T04:32:17.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.576+0000 7f7b83c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b7c0cfc80 0x7f7b7c0aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:17.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.576+0000 7f7b83c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b7c15ce30 0x7f7b7c1f3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:17.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.576+0000 7f7b83c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b7c0b1c70 0x7f7b7c0b0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:17.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.576+0000 7f7b83c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c0cd100 con 0x7f7b7c0b1c70 2024-09-17T04:32:17.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.576+0000 7f7b83c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c0ccf80 con 0x7f7b7c0cfc80 2024-09-17T04:32:17.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.576+0000 7f7b83c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7b7c0cd280 con 0x7f7b7c15ce30 2024-09-17T04:32:17.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.576+0000 7f7b81c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b7c0b1c70 0x7f7b7c0b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:17.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.576+0000 7f7b81c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b7c0b1c70 0x7f7b7c0b0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40898/0 (socket says 172.21.15.100:40898) 2024-09-17T04:32:17.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.576+0000 7f7b81c00640 1 -- 172.21.15.100:0/2905959804 learned_addr learned my addr 172.21.15.100:0/2905959804 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:17.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.577+0000 7f7b73e00640 1 --2- 172.21.15.100:0/2905959804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b7c15ce30 0x7f7b7c1f3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:17.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.577+0000 7f7b81200640 1 --2- 172.21.15.100:0/2905959804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b7c0cfc80 0x7f7b7c0aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:17.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.577+0000 7f7b81c00640 1 -- 172.21.15.100:0/2905959804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b7c15ce30 msgr2=0x7f7b7c1f3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:17.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.577+0000 7f7b81c00640 1 --2- 172.21.15.100:0/2905959804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b7c15ce30 0x7f7b7c1f3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:17.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.577+0000 7f7b81c00640 1 -- 172.21.15.100:0/2905959804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b7c0cfc80 msgr2=0x7f7b7c0aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:17.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.577+0000 7f7b81c00640 1 --2- 172.21.15.100:0/2905959804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b7c0cfc80 0x7f7b7c0aeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:17.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.577+0000 7f7b81c00640 1 -- 172.21.15.100:0/2905959804 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7c0ca030 con 0x7f7b7c0b1c70 2024-09-17T04:32:17.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.577+0000 7f7b81c00640 1 --2- 172.21.15.100:0/2905959804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b7c0b1c70 0x7f7b7c0b0690 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f7b74002230 tx=0x7f7b7400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:17.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.577+0000 7f7b79600640 1 -- 172.21.15.100:0/2905959804 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b74003b70 con 0x7f7b7c0b1c70 2024-09-17T04:32:17.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.577+0000 7f7b79600640 1 -- 172.21.15.100:0/2905959804 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b74003d10 con 0x7f7b7c0b1c70 2024-09-17T04:32:17.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.577+0000 7f7b79600640 1 -- 172.21.15.100:0/2905959804 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b74005330 con 0x7f7b7c0b1c70 2024-09-17T04:32:17.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.578+0000 7f7b83c00640 1 -- 172.21.15.100:0/2905959804 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b7c0adae0 con 0x7f7b7c0b1c70 2024-09-17T04:32:17.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.578+0000 7f7b83c00640 1 -- 172.21.15.100:0/2905959804 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b7c0ae0a0 con 0x7f7b7c0b1c70 2024-09-17T04:32:17.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.579+0000 7f7b83c00640 1 -- 172.21.15.100:0/2905959804 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b7c0cd770 con 0x7f7b7c0b1c70 2024-09-17T04:32:17.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.580+0000 7f7b79600640 1 -- 172.21.15.100:0/2905959804 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f7b740054d0 con 0x7f7b7c0b1c70 2024-09-17T04:32:17.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.581+0000 7f7b79600640 1 --2- 172.21.15.100:0/2905959804 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7b58079250 0x7f7b5807b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:17.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.581+0000 7f7b79600640 1 -- 172.21.15.100:0/2905959804 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5794+0+0 (secure 0 0 0) 0x7f7b7409b880 con 0x7f7b7c0b1c70 2024-09-17T04:32:17.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.581+0000 7f7b81200640 1 --2- 172.21.15.100:0/2905959804 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7b58079250 0x7f7b5807b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:17.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.582+0000 7f7b81200640 1 --2- 172.21.15.100:0/2905959804 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7b58079250 0x7f7b5807b760 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c000f60 tx=0x7f7b6c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:17.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.586+0000 7f7b79600640 1 -- 172.21.15.100:0/2905959804 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f7b74062a20 con 0x7f7b7c0b1c70 2024-09-17T04:32:17.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.782+0000 7f7b83c00640 1 -- 172.21.15.100:0/2905959804 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7b7c0afa70 con 0x7f7b7c0b1c70 2024-09-17T04:32:17.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.782+0000 7f7b79600640 1 -- 172.21.15.100:0/2905959804 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v134) ==== 80+0+0 (secure 0 0 0) 0x7f7b74067450 con 0x7f7b7c0b1c70 2024-09-17T04:32:17.782 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-09-17T04:32:17.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.784+0000 7f7b83c00640 1 -- 172.21.15.100:0/2905959804 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7b58079250 msgr2=0x7f7b5807b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:17.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.784+0000 7f7b83c00640 1 --2- 172.21.15.100:0/2905959804 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7b58079250 0x7f7b5807b760 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c000f60 tx=0x7f7b6c009040 comp rx=0 tx=0).stop 2024-09-17T04:32:17.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.784+0000 7f7b83c00640 1 -- 172.21.15.100:0/2905959804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b7c0b1c70 msgr2=0x7f7b7c0b0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:17.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.784+0000 7f7b83c00640 1 --2- 172.21.15.100:0/2905959804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b7c0b1c70 0x7f7b7c0b0690 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f7b74002230 tx=0x7f7b7400cb90 comp rx=0 tx=0).stop 2024-09-17T04:32:17.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.784+0000 7f7b83c00640 1 -- 172.21.15.100:0/2905959804 shutdown_connections 2024-09-17T04:32:17.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.784+0000 7f7b83c00640 1 --2- 172.21.15.100:0/2905959804 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7b58079250 0x7f7b5807b760 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:17.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.784+0000 7f7b83c00640 1 --2- 172.21.15.100:0/2905959804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b7c0b1c70 0x7f7b7c0b0690 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:17.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.784+0000 7f7b83c00640 1 --2- 172.21.15.100:0/2905959804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b7c15ce30 0x7f7b7c1f3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:17.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.784+0000 7f7b83c00640 1 --2- 172.21.15.100:0/2905959804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b7c0cfc80 0x7f7b7c0aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:17.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.784+0000 7f7b83c00640 1 -- 172.21.15.100:0/2905959804 >> 172.21.15.100:0/2905959804 conn(0x7f7b7c06d3d0 msgr2=0x7f7b7c0b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:17.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.785+0000 7f7b83c00640 1 -- 172.21.15.100:0/2905959804 shutdown_connections 2024-09-17T04:32:17.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:17.785+0000 7f7b83c00640 1 -- 172.21.15.100:0/2905959804 wait complete. 2024-09-17T04:32:17.808 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T04:32:17.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:17.809 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T04:32:17.873 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-09-17T04:32:17.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:17.873 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T04:32:17.909 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:32:17.909 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:32:17.909 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:32:17.950 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:32:17.950 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:32:18.029 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:32:18.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.250+0000 7f4cbe600640 1 -- 172.21.15.100:0/1870760641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cb80cfc80 msgr2=0x7f4cb8156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:18.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.250+0000 7f4cbe600640 1 --2- 172.21.15.100:0/1870760641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cb80cfc80 0x7f4cb8156e40 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8009900 tx=0x7f4ca802a610 comp rx=0 tx=0).stop 2024-09-17T04:32:18.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.250+0000 7f4cbe600640 1 -- 172.21.15.100:0/1870760641 shutdown_connections 2024-09-17T04:32:18.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.250+0000 7f4cbe600640 1 --2- 172.21.15.100:0/1870760641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cb80cfc80 0x7f4cb8156e40 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:18.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.250+0000 7f4cbe600640 1 -- 172.21.15.100:0/1870760641 >> 172.21.15.100:0/1870760641 conn(0x7f4cb806d3d0 msgr2=0x7f4cb80b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:18.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.250+0000 7f4cbe600640 1 -- 172.21.15.100:0/1870760641 shutdown_connections 2024-09-17T04:32:18.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.250+0000 7f4cbe600640 1 -- 172.21.15.100:0/1870760641 wait complete. 2024-09-17T04:32:18.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.252+0000 7f4cbe600640 1 Processor -- start 2024-09-17T04:32:18.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.252+0000 7f4cbe600640 1 -- start start 2024-09-17T04:32:18.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.254+0000 7f4cbe600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cb80705b0 0x7f4cb80c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:18.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.254+0000 7f4cbe600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4cb80cfc80 0x7f4cb80c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:18.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.254+0000 7f4cbe600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4cb815ce30 0x7f4cb80b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:18.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.254+0000 7f4cbe600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb80cd100 con 0x7f4cb80705b0 2024-09-17T04:32:18.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.254+0000 7f4cbe600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb80ccf80 con 0x7f4cb815ce30 2024-09-17T04:32:18.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.254+0000 7f4cbe600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4cb80cd280 con 0x7f4cb80cfc80 2024-09-17T04:32:18.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.254+0000 7f4cb7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cb80705b0 0x7f4cb80c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:18.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.254+0000 7f4cb7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cb80705b0 0x7f4cb80c5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40916/0 (socket says 172.21.15.100:40916) 2024-09-17T04:32:18.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.254+0000 7f4cb7400640 1 -- 172.21.15.100:0/1247544861 learned_addr learned my addr 172.21.15.100:0/1247544861 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:18.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.254+0000 7f4cb7400640 1 -- 172.21.15.100:0/1247544861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4cb80cfc80 msgr2=0x7f4cb80c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:18.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.254+0000 7f4cb7400640 1 --2- 172.21.15.100:0/1247544861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4cb80cfc80 0x7f4cb80c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:18.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.254+0000 7f4cb7400640 1 -- 172.21.15.100:0/1247544861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4cb815ce30 msgr2=0x7f4cb80b0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:18.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.254+0000 7f4cb7e00640 1 --2- 172.21.15.100:0/1247544861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4cb815ce30 0x7f4cb80b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:18.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.255+0000 7f4cb7400640 1 --2- 172.21.15.100:0/1247544861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4cb815ce30 0x7f4cb80b0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:18.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.255+0000 7f4cb7400640 1 -- 172.21.15.100:0/1247544861 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb80c8e70 con 0x7f4cb80705b0 2024-09-17T04:32:18.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.255+0000 7f4cb7e00640 1 --2- 172.21.15.100:0/1247544861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4cb815ce30 0x7f4cb80b0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:32:18.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.255+0000 7f4cb7400640 1 --2- 172.21.15.100:0/1247544861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cb80705b0 0x7f4cb80c5fc0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8034040 tx=0x7f4ca80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:18.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.255+0000 7f4cafe00640 1 -- 172.21.15.100:0/1247544861 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca803e030 con 0x7f4cb80705b0 2024-09-17T04:32:18.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.255+0000 7f4cafe00640 1 -- 172.21.15.100:0/1247544861 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ca8003ca0 con 0x7f4cb80705b0 2024-09-17T04:32:18.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.255+0000 7f4cafe00640 1 -- 172.21.15.100:0/1247544861 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca8035040 con 0x7f4cb80705b0 2024-09-17T04:32:18.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.255+0000 7f4cbe600640 1 -- 172.21.15.100:0/1247544861 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cb80c9380 con 0x7f4cb80705b0 2024-09-17T04:32:18.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.255+0000 7f4cbe600640 1 -- 172.21.15.100:0/1247544861 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cb80c9940 con 0x7f4cb80705b0 2024-09-17T04:32:18.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.256+0000 7f4cbe600640 1 -- 172.21.15.100:0/1247544861 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cb80cd770 con 0x7f4cb80705b0 2024-09-17T04:32:18.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.258+0000 7f4cafe00640 1 -- 172.21.15.100:0/1247544861 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f4ca8041030 con 0x7f4cb80705b0 2024-09-17T04:32:18.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.259+0000 7f4cafe00640 1 --2- 172.21.15.100:0/1247544861 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4c94079250 0x7f4c9407b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:18.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.259+0000 7f4cafe00640 1 -- 172.21.15.100:0/1247544861 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5794+0+0 (secure 0 0 0) 0x7f4ca80bc040 con 0x7f4cb80705b0 2024-09-17T04:32:18.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.259+0000 7f4caea00640 1 --2- 172.21.15.100:0/1247544861 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4c94079250 0x7f4c9407b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:18.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.260+0000 7f4caea00640 1 --2- 172.21.15.100:0/1247544861 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4c94079250 0x7f4c9407b760 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0000f60 tx=0x7f4ca00079b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:18.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.264+0000 7f4cafe00640 1 -- 172.21.15.100:0/1247544861 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4ca8083a80 con 0x7f4cb80705b0 2024-09-17T04:32:18.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.459+0000 7f4cbe600640 1 -- 172.21.15.100:0/1247544861 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4cb80afa70 con 0x7f4cb80705b0 2024-09-17T04:32:18.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.459+0000 7f4cafe00640 1 -- 172.21.15.100:0/1247544861 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v134) ==== 80+0+0 (secure 0 0 0) 0x7f4ca8083380 con 0x7f4cb80705b0 2024-09-17T04:32:18.459 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-09-17T04:32:18.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.462+0000 7f4cbe600640 1 -- 172.21.15.100:0/1247544861 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4c94079250 msgr2=0x7f4c9407b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:18.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.462+0000 7f4cbe600640 1 --2- 172.21.15.100:0/1247544861 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4c94079250 0x7f4c9407b760 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0000f60 tx=0x7f4ca00079b0 comp rx=0 tx=0).stop 2024-09-17T04:32:18.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.462+0000 7f4cbe600640 1 -- 172.21.15.100:0/1247544861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cb80705b0 msgr2=0x7f4cb80c5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:18.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.462+0000 7f4cbe600640 1 --2- 172.21.15.100:0/1247544861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cb80705b0 0x7f4cb80c5fc0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8034040 tx=0x7f4ca80038a0 comp rx=0 tx=0).stop 2024-09-17T04:32:18.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.462+0000 7f4cbe600640 1 -- 172.21.15.100:0/1247544861 shutdown_connections 2024-09-17T04:32:18.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.462+0000 7f4cbe600640 1 --2- 172.21.15.100:0/1247544861 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4c94079250 0x7f4c9407b760 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:18.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.462+0000 7f4cbe600640 1 --2- 172.21.15.100:0/1247544861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4cb815ce30 0x7f4cb80b0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:18.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.462+0000 7f4cbe600640 1 --2- 172.21.15.100:0/1247544861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4cb80cfc80 0x7f4cb80c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:18.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.462+0000 7f4cbe600640 1 --2- 172.21.15.100:0/1247544861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cb80705b0 0x7f4cb80c5fc0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:18.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.462+0000 7f4cbe600640 1 -- 172.21.15.100:0/1247544861 >> 172.21.15.100:0/1247544861 conn(0x7f4cb806d3d0 msgr2=0x7f4cb80b3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:18.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.462+0000 7f4cbe600640 1 -- 172.21.15.100:0/1247544861 shutdown_connections 2024-09-17T04:32:18.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.462+0000 7f4cbe600640 1 -- 172.21.15.100:0/1247544861 wait complete. 2024-09-17T04:32:18.483 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-17T04:32:18.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:18.483 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-17T04:32:18.547 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2024-09-17T04:32:18.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:18.547 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T04:32:18.584 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:32:18.585 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:32:18.585 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:32:18.620 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:32:18.620 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:32:18.620 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:32:18.621 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:32:18.661 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:32:18.661 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:32:18.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.882+0000 7f921f600640 1 -- 172.21.15.100:0/224866820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91fc0041e0 msgr2=0x7f91fc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:18.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.882+0000 7f921f600640 1 --2- 172.21.15.100:0/224866820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91fc0041e0 0x7f91fc004800 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f920000ac60 tx=0x7f91fc005830 comp rx=0 tx=0).stop 2024-09-17T04:32:18.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.882+0000 7f921d600640 1 -- 172.21.15.100:0/224866820 reap_dead start 2024-09-17T04:32:18.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.882+0000 7f921f600640 1 -- 172.21.15.100:0/224866820 shutdown_connections 2024-09-17T04:32:18.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.882+0000 7f921f600640 1 -- 172.21.15.100:0/224866820 >> 172.21.15.100:0/224866820 conn(0x7f921806d730 msgr2=0x7f92180b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:18.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.882+0000 7f921f600640 1 -- 172.21.15.100:0/224866820 shutdown_connections 2024-09-17T04:32:18.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.882+0000 7f921f600640 1 -- 172.21.15.100:0/224866820 wait complete. 2024-09-17T04:32:18.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.885+0000 7f921f600640 1 Processor -- start 2024-09-17T04:32:18.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.885+0000 7f921f600640 1 -- start start 2024-09-17T04:32:18.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.886+0000 7f921f600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91fc0041e0 0x7f9218158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:18.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.886+0000 7f921f600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f921814fdb0 0x7f92181f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:18.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.886+0000 7f921f600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92181f4ce0 0x7f92181f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:18.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.886+0000 7f921f600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f92180712b0 con 0x7f91fc0041e0 2024-09-17T04:32:18.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.886+0000 7f921f600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9218071130 con 0x7f92181f4ce0 2024-09-17T04:32:18.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.886+0000 7f921f600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f92180ad7d0 con 0x7f921814fdb0 2024-09-17T04:32:18.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.887+0000 7f921cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91fc0041e0 0x7f9218158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:18.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.887+0000 7f921cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91fc0041e0 0x7f9218158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40924/0 (socket says 172.21.15.100:40924) 2024-09-17T04:32:18.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.887+0000 7f921cc00640 1 -- 172.21.15.100:0/2468290116 learned_addr learned my addr 172.21.15.100:0/2468290116 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:18.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.887+0000 7f920fe00640 1 --2- 172.21.15.100:0/2468290116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f921814fdb0 0x7f92181f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:18.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.887+0000 7f921d600640 1 --2- 172.21.15.100:0/2468290116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92181f4ce0 0x7f92181f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:18.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.887+0000 7f921cc00640 1 -- 172.21.15.100:0/2468290116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f921814fdb0 msgr2=0x7f92181f47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:18.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.887+0000 7f921cc00640 1 --2- 172.21.15.100:0/2468290116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f921814fdb0 0x7f92181f47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:18.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.887+0000 7f921cc00640 1 -- 172.21.15.100:0/2468290116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92181f4ce0 msgr2=0x7f92181f6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:18.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.887+0000 7f921cc00640 1 --2- 172.21.15.100:0/2468290116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92181f4ce0 0x7f92181f6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:18.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.887+0000 7f921cc00640 1 -- 172.21.15.100:0/2468290116 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92181f50f0 con 0x7f91fc0041e0 2024-09-17T04:32:18.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.887+0000 7f921d600640 1 --2- 172.21.15.100:0/2468290116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92181f4ce0 0x7f92181f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:32:18.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.887+0000 7f921cc00640 1 --2- 172.21.15.100:0/2468290116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91fc0041e0 0x7f9218158e40 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f9208003e40 tx=0x7f9208001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:18.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.887+0000 7f9215600640 1 -- 172.21.15.100:0/2468290116 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208011700 con 0x7f91fc0041e0 2024-09-17T04:32:18.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.888+0000 7f9215600640 1 -- 172.21.15.100:0/2468290116 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9208016020 con 0x7f91fc0041e0 2024-09-17T04:32:18.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.888+0000 7f9215600640 1 -- 172.21.15.100:0/2468290116 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208011b40 con 0x7f91fc0041e0 2024-09-17T04:32:18.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.888+0000 7f921f600640 1 -- 172.21.15.100:0/2468290116 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9218158080 con 0x7f91fc0041e0 2024-09-17T04:32:18.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.888+0000 7f921f600640 1 -- 172.21.15.100:0/2468290116 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9218158640 con 0x7f91fc0041e0 2024-09-17T04:32:18.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.889+0000 7f921f600640 1 -- 172.21.15.100:0/2468290116 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9218073040 con 0x7f91fc0041e0 2024-09-17T04:32:18.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.890+0000 7f9215600640 1 -- 172.21.15.100:0/2468290116 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f9208011ce0 con 0x7f91fc0041e0 2024-09-17T04:32:18.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.891+0000 7f9215600640 1 --2- 172.21.15.100:0/2468290116 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f91f4079250 0x7f91f407b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:18.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.892+0000 7f9215600640 1 -- 172.21.15.100:0/2468290116 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5794+0+0 (secure 0 0 0) 0x7f920809c920 con 0x7f91fc0041e0 2024-09-17T04:32:18.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.892+0000 7f920fe00640 1 --2- 172.21.15.100:0/2468290116 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f91f4079250 0x7f91f407b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:18.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.892+0000 7f920fe00640 1 --2- 172.21.15.100:0/2468290116 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f91f4079250 0x7f91f407b760 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9200036000 tx=0x7f92000046c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:18.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:18.896+0000 7f9215600640 1 -- 172.21.15.100:0/2468290116 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9208020020 con 0x7f91fc0041e0 2024-09-17T04:32:19.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.116+0000 7f921f600640 1 -- 172.21.15.100:0/2468290116 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f921815e420 con 0x7f91fc0041e0 2024-09-17T04:32:19.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.116+0000 7f9215600640 1 -- 172.21.15.100:0/2468290116 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5239 (secure 0 0 0) 0x7f9208064190 con 0x7f91fc0041e0 2024-09-17T04:32:19.116 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:32:19.116 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":92,"btime":"2024-09-17T04:32:04:868771+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6719,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1392663867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1392663867},{"type":"v1","addr":"172.21.15.184:6801","nonce":1392663867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":7516,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3257629487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3257629487},{"type":"v1","addr":"172.21.15.184:6835","nonce":3257629487}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":7962,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/2131565869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2131565869},{"type":"v1","addr":"172.21.15.100:6834","nonce":2131565869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:31:53.292918+0000","modified":"2024-09-17T04:31:56.320263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7965},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7965":{"gid":7965,"name":"c","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3047460644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":3047460644},{"type":"v1","addr":"172.21.15.100:6835","nonce":3047460644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7965,"qdb_cluster":[7965]},"id":10}]} 2024-09-17T04:32:19.116 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 92 2024-09-17T04:32:19.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.119+0000 7f921f600640 1 -- 172.21.15.100:0/2468290116 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f91f4079250 msgr2=0x7f91f407b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:19.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.119+0000 7f921f600640 1 --2- 172.21.15.100:0/2468290116 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f91f4079250 0x7f91f407b760 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9200036000 tx=0x7f92000046c0 comp rx=0 tx=0).stop 2024-09-17T04:32:19.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.119+0000 7f921f600640 1 -- 172.21.15.100:0/2468290116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91fc0041e0 msgr2=0x7f9218158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:19.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.119+0000 7f921f600640 1 --2- 172.21.15.100:0/2468290116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91fc0041e0 0x7f9218158e40 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f9208003e40 tx=0x7f9208001800 comp rx=0 tx=0).stop 2024-09-17T04:32:19.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.119+0000 7f921f600640 1 -- 172.21.15.100:0/2468290116 shutdown_connections 2024-09-17T04:32:19.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.119+0000 7f921f600640 1 --2- 172.21.15.100:0/2468290116 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f91f4079250 0x7f91f407b760 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:19.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.119+0000 7f921f600640 1 --2- 172.21.15.100:0/2468290116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92181f4ce0 0x7f92181f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:19.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.119+0000 7f921f600640 1 --2- 172.21.15.100:0/2468290116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f921814fdb0 0x7f92181f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:19.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.119+0000 7f921f600640 1 --2- 172.21.15.100:0/2468290116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91fc0041e0 0x7f9218158e40 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:19.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.119+0000 7f921f600640 1 -- 172.21.15.100:0/2468290116 >> 172.21.15.100:0/2468290116 conn(0x7f921806d730 msgr2=0x7f9218152fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:19.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.119+0000 7f921f600640 1 -- 172.21.15.100:0/2468290116 shutdown_connections 2024-09-17T04:32:19.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.119+0000 7f921f600640 1 -- 172.21.15.100:0/2468290116 wait complete. 2024-09-17T04:32:19.144 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:32:19.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.365+0000 7f680b400640 1 -- 172.21.15.100:0/3762897804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f680415d760 msgr2=0x7f68041535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:19.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.365+0000 7f680b400640 1 --2- 172.21.15.100:0/3762897804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f680415d760 0x7f68041535c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6804075250 tx=0x7f67f402a560 comp rx=0 tx=0).stop 2024-09-17T04:32:19.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.365+0000 7f680b400640 1 -- 172.21.15.100:0/3762897804 shutdown_connections 2024-09-17T04:32:19.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.365+0000 7f680b400640 1 --2- 172.21.15.100:0/3762897804 >> v2:172.21.15.100:3300/0 conn(0x7f68040b3550 0x7f68040b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:19.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.365+0000 7f680b400640 1 --2- 172.21.15.100:0/3762897804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f680415d760 0x7f68041535c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:19.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.365+0000 7f680b400640 1 -- 172.21.15.100:0/3762897804 >> 172.21.15.100:0/3762897804 conn(0x7f680406d730 msgr2=0x7f68040b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:19.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.365+0000 7f680b400640 1 -- 172.21.15.100:0/3762897804 shutdown_connections 2024-09-17T04:32:19.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.365+0000 7f680b400640 1 -- 172.21.15.100:0/3762897804 wait complete. 2024-09-17T04:32:19.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.368+0000 7f680b400640 1 Processor -- start 2024-09-17T04:32:19.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.368+0000 7f680b400640 1 -- start start 2024-09-17T04:32:19.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.369+0000 7f680b400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68040ad0e0 0x7f680409de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:19.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.369+0000 7f680b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68040b3550 0x7f6804156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:19.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.369+0000 7f680b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f680415d760 0x7f68041f4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:19.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.369+0000 7f680b400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f68040b4da0 con 0x7f68040b3550 2024-09-17T04:32:19.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.369+0000 7f680b400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f68040b4c20 con 0x7f680415d760 2024-09-17T04:32:19.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.369+0000 7f680b400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f68040b3ea0 con 0x7f68040ad0e0 2024-09-17T04:32:19.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.370+0000 7f6803e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68040b3550 0x7f6804156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:19.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.370+0000 7f6803e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68040b3550 0x7f6804156610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40946/0 (socket says 172.21.15.100:40946) 2024-09-17T04:32:19.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.370+0000 7f6803e00640 1 -- 172.21.15.100:0/2768462349 learned_addr learned my addr 172.21.15.100:0/2768462349 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:19.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.370+0000 7f6809400640 1 --2- 172.21.15.100:0/2768462349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f680415d760 0x7f68041f4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:19.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.370+0000 7f6808a00640 1 --2- 172.21.15.100:0/2768462349 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68040ad0e0 0x7f680409de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:19.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.370+0000 7f6803e00640 1 -- 172.21.15.100:0/2768462349 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68040ad0e0 msgr2=0x7f680409de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:19.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.370+0000 7f6803e00640 1 --2- 172.21.15.100:0/2768462349 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68040ad0e0 0x7f680409de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:19.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.370+0000 7f6803e00640 1 -- 172.21.15.100:0/2768462349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f680415d760 msgr2=0x7f68041f4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:19.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.370+0000 7f6803e00640 1 --2- 172.21.15.100:0/2768462349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f680415d760 0x7f68041f4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:19.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.370+0000 7f6803e00640 1 -- 172.21.15.100:0/2768462349 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680414a260 con 0x7f68040b3550 2024-09-17T04:32:19.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.370+0000 7f6803e00640 1 --2- 172.21.15.100:0/2768462349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68040b3550 0x7f6804156610 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f67f0001eb0 tx=0x7f67f000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:19.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.371+0000 7f6800c00640 1 -- 172.21.15.100:0/2768462349 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f0017070 con 0x7f68040b3550 2024-09-17T04:32:19.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.371+0000 7f6800c00640 1 -- 172.21.15.100:0/2768462349 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67f00053c0 con 0x7f68040b3550 2024-09-17T04:32:19.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.371+0000 7f6800c00640 1 -- 172.21.15.100:0/2768462349 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f0003830 con 0x7f68040b3550 2024-09-17T04:32:19.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.371+0000 7f680b400640 1 -- 172.21.15.100:0/2768462349 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68041f9310 con 0x7f68040b3550 2024-09-17T04:32:19.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.371+0000 7f680b400640 1 -- 172.21.15.100:0/2768462349 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68041f98d0 con 0x7f68040b3550 2024-09-17T04:32:19.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.372+0000 7f680b400640 1 -- 172.21.15.100:0/2768462349 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68040b4360 con 0x7f68040b3550 2024-09-17T04:32:19.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.373+0000 7f6800c00640 1 -- 172.21.15.100:0/2768462349 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f67f001a070 con 0x7f68040b3550 2024-09-17T04:32:19.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.375+0000 7f6800c00640 1 --2- 172.21.15.100:0/2768462349 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f67e4079250 0x7f67e407b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:19.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.375+0000 7f6800c00640 1 -- 172.21.15.100:0/2768462349 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5794+0+0 (secure 0 0 0) 0x7f67f009ae80 con 0x7f68040b3550 2024-09-17T04:32:19.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.375+0000 7f6808a00640 1 --2- 172.21.15.100:0/2768462349 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f67e4079250 0x7f67e407b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:19.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.376+0000 7f6808a00640 1 --2- 172.21.15.100:0/2768462349 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f67e4079250 0x7f67e407b760 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f67f4033000 tx=0x7f67f4004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:19.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.379+0000 7f6800c00640 1 -- 172.21.15.100:0/2768462349 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f67f009f070 con 0x7f68040b3550 2024-09-17T04:32:19.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.598+0000 7f680b400640 1 -- 172.21.15.100:0/2768462349 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68041f90a0 con 0x7f68040b3550 2024-09-17T04:32:19.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.599+0000 7f6800c00640 1 -- 172.21.15.100:0/2768462349 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5239 (secure 0 0 0) 0x7f680414a260 con 0x7f68040b3550 2024-09-17T04:32:19.598 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:32:19.599 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":92,"btime":"2024-09-17T04:32:04:868771+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6719,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1392663867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1392663867},{"type":"v1","addr":"172.21.15.184:6801","nonce":1392663867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":7516,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3257629487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3257629487},{"type":"v1","addr":"172.21.15.184:6835","nonce":3257629487}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":7962,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/2131565869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2131565869},{"type":"v1","addr":"172.21.15.100:6834","nonce":2131565869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:31:53.292918+0000","modified":"2024-09-17T04:31:56.320263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7965},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7965":{"gid":7965,"name":"c","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3047460644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":3047460644},{"type":"v1","addr":"172.21.15.100:6835","nonce":3047460644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7965,"qdb_cluster":[7965]},"id":10}]} 2024-09-17T04:32:19.599 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 92 2024-09-17T04:32:19.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.601+0000 7f680b400640 1 -- 172.21.15.100:0/2768462349 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f67e4079250 msgr2=0x7f67e407b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:19.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.601+0000 7f680b400640 1 --2- 172.21.15.100:0/2768462349 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f67e4079250 0x7f67e407b760 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f67f4033000 tx=0x7f67f4004730 comp rx=0 tx=0).stop 2024-09-17T04:32:19.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.601+0000 7f680b400640 1 -- 172.21.15.100:0/2768462349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68040b3550 msgr2=0x7f6804156610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:19.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.601+0000 7f680b400640 1 --2- 172.21.15.100:0/2768462349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68040b3550 0x7f6804156610 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f67f0001eb0 tx=0x7f67f000c890 comp rx=0 tx=0).stop 2024-09-17T04:32:19.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.601+0000 7f680b400640 1 -- 172.21.15.100:0/2768462349 shutdown_connections 2024-09-17T04:32:19.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.601+0000 7f680b400640 1 --2- 172.21.15.100:0/2768462349 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f67e4079250 0x7f67e407b760 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:19.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.601+0000 7f680b400640 1 --2- 172.21.15.100:0/2768462349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f680415d760 0x7f68041f4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:19.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.601+0000 7f680b400640 1 --2- 172.21.15.100:0/2768462349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68040b3550 0x7f6804156610 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:19.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.601+0000 7f680b400640 1 --2- 172.21.15.100:0/2768462349 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68040ad0e0 0x7f680409de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:19.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.601+0000 7f680b400640 1 -- 172.21.15.100:0/2768462349 >> 172.21.15.100:0/2768462349 conn(0x7f680406d730 msgr2=0x7f680406f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:19.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.602+0000 7f680b400640 1 -- 172.21.15.100:0/2768462349 shutdown_connections 2024-09-17T04:32:19.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.602+0000 7f680b400640 1 -- 172.21.15.100:0/2768462349 wait complete. 2024-09-17T04:32:19.627 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:32:19.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.849+0000 7ff3a4e00640 1 -- 172.21.15.100:0/887287739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3a0070540 msgr2=0x7ff3a01651b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:19.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.849+0000 7ff3a4e00640 1 --2- 172.21.15.100:0/887287739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3a0070540 0x7ff3a01651b0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7ff398005fd0 tx=0x7ff39802a8b0 comp rx=0 tx=0).stop 2024-09-17T04:32:19.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.849+0000 7ff3a4e00640 1 -- 172.21.15.100:0/887287739 shutdown_connections 2024-09-17T04:32:19.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.849+0000 7ff3a4e00640 1 --2- 172.21.15.100:0/887287739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3a0070540 0x7ff3a01651b0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:19.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.849+0000 7ff3a4e00640 1 --2- 172.21.15.100:0/887287739 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3a00ae080 0x7ff3a0070000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:19.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.849+0000 7ff3a4e00640 1 -- 172.21.15.100:0/887287739 >> 172.21.15.100:0/887287739 conn(0x7ff3a00b1e40 msgr2=0x7ff3a00b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:19.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.849+0000 7ff3a4e00640 1 -- 172.21.15.100:0/887287739 shutdown_connections 2024-09-17T04:32:19.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.849+0000 7ff3a4e00640 1 -- 172.21.15.100:0/887287739 wait complete. 2024-09-17T04:32:19.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.851+0000 7ff3a4e00640 1 Processor -- start 2024-09-17T04:32:19.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.851+0000 7ff3a4e00640 1 -- start start 2024-09-17T04:32:19.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.853+0000 7ff3a4e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3a0070540 0x7ff3a0144a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:19.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.853+0000 7ff3a4e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3a00ae080 0x7ff3a0153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:19.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.853+0000 7ff3a4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3a00af050 0x7ff3a0154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:19.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.853+0000 7ff3a4e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a0165a80 con 0x7ff3a00af050 2024-09-17T04:32:19.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.853+0000 7ff3a4e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a0165900 con 0x7ff3a00ae080 2024-09-17T04:32:19.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.853+0000 7ff3a4e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff3a0165c00 con 0x7ff3a0070540 2024-09-17T04:32:19.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.853+0000 7ff39fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3a00af050 0x7ff3a0154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:19.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.853+0000 7ff39ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3a00ae080 0x7ff3a0153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:19.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.853+0000 7ff39fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3a00af050 0x7ff3a0154250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40960/0 (socket says 172.21.15.100:40960) 2024-09-17T04:32:19.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.853+0000 7ff39fe00640 1 -- 172.21.15.100:0/3425534931 learned_addr learned my addr 172.21.15.100:0/3425534931 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:19.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.853+0000 7ff39f400640 1 --2- 172.21.15.100:0/3425534931 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3a0070540 0x7ff3a0144a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:19.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.853+0000 7ff39fe00640 1 -- 172.21.15.100:0/3425534931 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3a0070540 msgr2=0x7ff3a0144a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:19.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.853+0000 7ff39fe00640 1 --2- 172.21.15.100:0/3425534931 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3a0070540 0x7ff3a0144a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:19.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.853+0000 7ff39fe00640 1 -- 172.21.15.100:0/3425534931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3a00ae080 msgr2=0x7ff3a0153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:19.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.853+0000 7ff39fe00640 1 --2- 172.21.15.100:0/3425534931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3a00ae080 0x7ff3a0153d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:19.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.853+0000 7ff39fe00640 1 -- 172.21.15.100:0/3425534931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a0152c70 con 0x7ff3a00af050 2024-09-17T04:32:19.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.853+0000 7ff39ea00640 1 --2- 172.21.15.100:0/3425534931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3a00ae080 0x7ff3a0153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:32:19.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.854+0000 7ff39fe00640 1 --2- 172.21.15.100:0/3425534931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3a00af050 0x7ff3a0154250 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7ff398009cd0 tx=0x7ff398003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:19.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.854+0000 7ff397400640 1 -- 172.21.15.100:0/3425534931 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff398031990 con 0x7ff3a00af050 2024-09-17T04:32:19.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.854+0000 7ff397400640 1 -- 172.21.15.100:0/3425534931 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff398031c20 con 0x7ff3a00af050 2024-09-17T04:32:19.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.854+0000 7ff3a4e00640 1 -- 172.21.15.100:0/3425534931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3a0152f50 con 0x7ff3a00af050 2024-09-17T04:32:19.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.854+0000 7ff397400640 1 -- 172.21.15.100:0/3425534931 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff398033070 con 0x7ff3a00af050 2024-09-17T04:32:19.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.854+0000 7ff3a4e00640 1 -- 172.21.15.100:0/3425534931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3a0153510 con 0x7ff3a00af050 2024-09-17T04:32:19.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.855+0000 7ff3a4e00640 1 -- 172.21.15.100:0/3425534931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3a0070000 con 0x7ff3a00af050 2024-09-17T04:32:19.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.857+0000 7ff397400640 1 -- 172.21.15.100:0/3425534931 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7ff398033210 con 0x7ff3a00af050 2024-09-17T04:32:19.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.857+0000 7ff397400640 1 --2- 172.21.15.100:0/3425534931 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff380079270 0x7ff38007b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:19.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.858+0000 7ff397400640 1 -- 172.21.15.100:0/3425534931 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5794+0+0 (secure 0 0 0) 0x7ff3980bcef0 con 0x7ff3a00af050 2024-09-17T04:32:19.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.858+0000 7ff39f400640 1 --2- 172.21.15.100:0/3425534931 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff380079270 0x7ff38007b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:19.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.858+0000 7ff39f400640 1 --2- 172.21.15.100:0/3425534931 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff380079270 0x7ff38007b780 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff390001720 tx=0x7ff390007e90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:19.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:19.862+0000 7ff397400640 1 -- 172.21.15.100:0/3425534931 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff39803f030 con 0x7ff3a00af050 2024-09-17T04:32:20.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.050+0000 7ff3a4e00640 1 -- 172.21.15.100:0/3425534931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff3a0164040 con 0x7ff3a00af050 2024-09-17T04:32:20.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.051+0000 7ff397400640 1 -- 172.21.15.100:0/3425534931 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+16926 (secure 0 0 0) 0x7ff3980317b0 con 0x7ff3a00af050 2024-09-17T04:32:20.051 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:32:20.051 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":134,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:32:07.884372+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:31:51.164879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-17T04:31:52.259089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.1b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:32:20.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.054+0000 7ff3a4e00640 1 -- 172.21.15.100:0/3425534931 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff380079270 msgr2=0x7ff38007b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:20.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.054+0000 7ff3a4e00640 1 --2- 172.21.15.100:0/3425534931 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff380079270 0x7ff38007b780 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff390001720 tx=0x7ff390007e90 comp rx=0 tx=0).stop 2024-09-17T04:32:20.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.054+0000 7ff3a4e00640 1 -- 172.21.15.100:0/3425534931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3a00af050 msgr2=0x7ff3a0154250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:20.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.054+0000 7ff3a4e00640 1 --2- 172.21.15.100:0/3425534931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3a00af050 0x7ff3a0154250 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7ff398009cd0 tx=0x7ff398003970 comp rx=0 tx=0).stop 2024-09-17T04:32:20.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.054+0000 7ff3a4e00640 1 -- 172.21.15.100:0/3425534931 shutdown_connections 2024-09-17T04:32:20.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.054+0000 7ff3a4e00640 1 --2- 172.21.15.100:0/3425534931 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff380079270 0x7ff38007b780 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:20.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.054+0000 7ff3a4e00640 1 --2- 172.21.15.100:0/3425534931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3a00af050 0x7ff3a0154250 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:20.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.054+0000 7ff3a4e00640 1 --2- 172.21.15.100:0/3425534931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3a00ae080 0x7ff3a0153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:20.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.054+0000 7ff3a4e00640 1 --2- 172.21.15.100:0/3425534931 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3a0070540 0x7ff3a0144a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:20.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.054+0000 7ff3a4e00640 1 -- 172.21.15.100:0/3425534931 >> 172.21.15.100:0/3425534931 conn(0x7ff3a00b1e40 msgr2=0x7ff3a00b0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:20.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.054+0000 7ff3a4e00640 1 -- 172.21.15.100:0/3425534931 shutdown_connections 2024-09-17T04:32:20.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.054+0000 7ff3a4e00640 1 -- 172.21.15.100:0/3425534931 wait complete. 2024-09-17T04:32:20.075 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T04:32:20.075 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:32:20.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.297+0000 7ff813e00640 1 -- 172.21.15.100:0/4110747233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff80c160310 msgr2=0x7ff80c160720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:20.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.297+0000 7ff813e00640 1 --2- 172.21.15.100:0/4110747233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff80c160310 0x7ff80c160720 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7ff80c075250 tx=0x7ff80402a4d0 comp rx=0 tx=0).stop 2024-09-17T04:32:20.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.297+0000 7ff813e00640 1 -- 172.21.15.100:0/4110747233 shutdown_connections 2024-09-17T04:32:20.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.297+0000 7ff813e00640 1 --2- 172.21.15.100:0/4110747233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff80c160310 0x7ff80c160720 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:20.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.297+0000 7ff813e00640 1 --2- 172.21.15.100:0/4110747233 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff80c0ad0e0 0x7ff80c0ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:20.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.297+0000 7ff813e00640 1 --2- 172.21.15.100:0/4110747233 >> v2:172.21.15.184:3300/0 conn(0x7ff80c15d760 0x7ff80c1535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:20.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.297+0000 7ff813e00640 1 -- 172.21.15.100:0/4110747233 >> 172.21.15.100:0/4110747233 conn(0x7ff80c06d730 msgr2=0x7ff80c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:20.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.297+0000 7ff813e00640 1 -- 172.21.15.100:0/4110747233 shutdown_connections 2024-09-17T04:32:20.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.297+0000 7ff813e00640 1 -- 172.21.15.100:0/4110747233 wait complete. 2024-09-17T04:32:20.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.299+0000 7ff813e00640 1 Processor -- start 2024-09-17T04:32:20.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.300+0000 7ff813e00640 1 -- start start 2024-09-17T04:32:20.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.301+0000 7ff813e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff80c0ad0e0 0x7ff80c153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:20.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.301+0000 7ff813e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff80c15d760 0x7ff80c15b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:20.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.301+0000 7ff813e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff80c160310 0x7ff80c15bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:20.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.301+0000 7ff813e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c0b4da0 con 0x7ff80c15d760 2024-09-17T04:32:20.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.301+0000 7ff813e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c0b4c20 con 0x7ff80c0ad0e0 2024-09-17T04:32:20.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.301+0000 7ff813e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff80c160c60 con 0x7ff80c160310 2024-09-17T04:32:20.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.301+0000 7ff810a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff80c15d760 0x7ff80c15b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:20.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.301+0000 7ff810a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff80c15d760 0x7ff80c15b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40988/0 (socket says 172.21.15.100:40988) 2024-09-17T04:32:20.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.301+0000 7ff810a00640 1 -- 172.21.15.100:0/256036944 learned_addr learned my addr 172.21.15.100:0/256036944 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:20.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.302+0000 7ff811e00640 1 --2- 172.21.15.100:0/256036944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff80c160310 0x7ff80c15bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:20.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.302+0000 7ff811400640 1 --2- 172.21.15.100:0/256036944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff80c0ad0e0 0x7ff80c153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:20.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.302+0000 7ff810a00640 1 -- 172.21.15.100:0/256036944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff80c160310 msgr2=0x7ff80c15bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:20.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.302+0000 7ff810a00640 1 --2- 172.21.15.100:0/256036944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff80c160310 0x7ff80c15bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:20.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.302+0000 7ff810a00640 1 -- 172.21.15.100:0/256036944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff80c0ad0e0 msgr2=0x7ff80c153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:20.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.302+0000 7ff810a00640 1 --2- 172.21.15.100:0/256036944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff80c0ad0e0 0x7ff80c153cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:20.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.302+0000 7ff810a00640 1 -- 172.21.15.100:0/256036944 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff80c1f90a0 con 0x7ff80c15d760 2024-09-17T04:32:20.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.302+0000 7ff810a00640 1 --2- 172.21.15.100:0/256036944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff80c15d760 0x7ff80c15b630 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8009fd0 tx=0x7ff7f800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:20.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.302+0000 7ff809600640 1 -- 172.21.15.100:0/256036944 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f800ccc0 con 0x7ff80c15d760 2024-09-17T04:32:20.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.302+0000 7ff809600640 1 -- 172.21.15.100:0/256036944 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7f800ee70 con 0x7ff80c15d760 2024-09-17T04:32:20.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.302+0000 7ff813e00640 1 -- 172.21.15.100:0/256036944 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff80c1f9380 con 0x7ff80c15d760 2024-09-17T04:32:20.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.302+0000 7ff809600640 1 -- 172.21.15.100:0/256036944 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f800ce60 con 0x7ff80c15d760 2024-09-17T04:32:20.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.302+0000 7ff813e00640 1 -- 172.21.15.100:0/256036944 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff80c1f9940 con 0x7ff80c15d760 2024-09-17T04:32:20.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.303+0000 7ff813e00640 1 -- 172.21.15.100:0/256036944 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff80c1535c0 con 0x7ff80c15d760 2024-09-17T04:32:20.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.305+0000 7ff809600640 1 -- 172.21.15.100:0/256036944 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7ff7f8004280 con 0x7ff80c15d760 2024-09-17T04:32:20.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.306+0000 7ff809600640 1 --2- 172.21.15.100:0/256036944 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff7ec079250 0x7ff7ec07b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:20.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.306+0000 7ff809600640 1 -- 172.21.15.100:0/256036944 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5794+0+0 (secure 0 0 0) 0x7ff7f809c3f0 con 0x7ff80c15d760 2024-09-17T04:32:20.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.306+0000 7ff811400640 1 --2- 172.21.15.100:0/256036944 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff7ec079250 0x7ff7ec07b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:20.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.307+0000 7ff811400640 1 --2- 172.21.15.100:0/256036944 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff7ec079250 0x7ff7ec07b760 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff800005fd0 tx=0x7ff800009500 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:20.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.310+0000 7ff809600640 1 -- 172.21.15.100:0/256036944 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff7f8063e30 con 0x7ff80c15d760 2024-09-17T04:32:20.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.528+0000 7ff813e00640 1 -- 172.21.15.100:0/256036944 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff80c156b80 con 0x7ff80c15d760 2024-09-17T04:32:20.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.528+0000 7ff809600640 1 -- 172.21.15.100:0/256036944 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5239 (secure 0 0 0) 0x7ff7f8063730 con 0x7ff80c15d760 2024-09-17T04:32:20.529 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:32:20.529 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":92,"btime":"2024-09-17T04:32:04:868771+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6719,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1392663867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1392663867},{"type":"v1","addr":"172.21.15.184:6801","nonce":1392663867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":7516,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3257629487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3257629487},{"type":"v1","addr":"172.21.15.184:6835","nonce":3257629487}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":7962,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/2131565869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2131565869},{"type":"v1","addr":"172.21.15.100:6834","nonce":2131565869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:31:53.292918+0000","modified":"2024-09-17T04:31:56.320263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7965},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7965":{"gid":7965,"name":"c","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3047460644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":3047460644},{"type":"v1","addr":"172.21.15.100:6835","nonce":3047460644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7965,"qdb_cluster":[7965]},"id":10}]} 2024-09-17T04:32:20.530 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 92 2024-09-17T04:32:20.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.531+0000 7ff813e00640 1 -- 172.21.15.100:0/256036944 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff7ec079250 msgr2=0x7ff7ec07b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:20.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.531+0000 7ff813e00640 1 --2- 172.21.15.100:0/256036944 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff7ec079250 0x7ff7ec07b760 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff800005fd0 tx=0x7ff800009500 comp rx=0 tx=0).stop 2024-09-17T04:32:20.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.531+0000 7ff813e00640 1 -- 172.21.15.100:0/256036944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff80c15d760 msgr2=0x7ff80c15b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:20.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.531+0000 7ff813e00640 1 --2- 172.21.15.100:0/256036944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff80c15d760 0x7ff80c15b630 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8009fd0 tx=0x7ff7f800ec20 comp rx=0 tx=0).stop 2024-09-17T04:32:20.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.531+0000 7ff813e00640 1 -- 172.21.15.100:0/256036944 shutdown_connections 2024-09-17T04:32:20.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.531+0000 7ff813e00640 1 --2- 172.21.15.100:0/256036944 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff7ec079250 0x7ff7ec07b760 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:20.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.532+0000 7ff813e00640 1 --2- 172.21.15.100:0/256036944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff80c160310 0x7ff80c15bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:20.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.532+0000 7ff813e00640 1 --2- 172.21.15.100:0/256036944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff80c15d760 0x7ff80c15b630 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:20.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.532+0000 7ff813e00640 1 --2- 172.21.15.100:0/256036944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff80c0ad0e0 0x7ff80c153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:20.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.532+0000 7ff813e00640 1 -- 172.21.15.100:0/256036944 >> 172.21.15.100:0/256036944 conn(0x7ff80c06d730 msgr2=0x7ff80c15a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:20.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.532+0000 7ff813e00640 1 -- 172.21.15.100:0/256036944 shutdown_connections 2024-09-17T04:32:20.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.532+0000 7ff813e00640 1 -- 172.21.15.100:0/256036944 wait complete. 2024-09-17T04:32:20.556 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:32:20.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.776+0000 7f3db0400640 1 -- 172.21.15.100:0/4126254463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3da80ad0e0 msgr2=0x7f3da80ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:20.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.777+0000 7f3db0400640 1 --2- 172.21.15.100:0/4126254463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3da80ad0e0 0x7f3da80ad4f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009830 tx=0x7f3d9402a520 comp rx=0 tx=0).stop 2024-09-17T04:32:20.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.777+0000 7f3db0400640 1 -- 172.21.15.100:0/4126254463 shutdown_connections 2024-09-17T04:32:20.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.777+0000 7f3db0400640 1 --2- 172.21.15.100:0/4126254463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3da80ad0e0 0x7f3da80ad4f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:20.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.777+0000 7f3db0400640 1 -- 172.21.15.100:0/4126254463 >> 172.21.15.100:0/4126254463 conn(0x7f3da806d730 msgr2=0x7f3da80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:20.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.777+0000 7f3db0400640 1 -- 172.21.15.100:0/4126254463 shutdown_connections 2024-09-17T04:32:20.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.777+0000 7f3db0400640 1 -- 172.21.15.100:0/4126254463 wait complete. 2024-09-17T04:32:20.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.779+0000 7f3db0400640 1 Processor -- start 2024-09-17T04:32:20.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.779+0000 7f3db0400640 1 -- start start 2024-09-17T04:32:20.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.781+0000 7f3db0400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3da80ad0e0 0x7f3da81513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:20.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.781+0000 7f3db0400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3da80b3550 0x7f3da81f8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:20.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.781+0000 7f3db0400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3da815d760 0x7f3da8156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:20.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.781+0000 7f3db0400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3da80b4da0 con 0x7f3da815d760 2024-09-17T04:32:20.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.781+0000 7f3db0400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3da80b4c20 con 0x7f3da80b3550 2024-09-17T04:32:20.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.781+0000 7f3db0400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3da80b4070 con 0x7f3da80ad0e0 2024-09-17T04:32:20.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.781+0000 7f3dae400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3da815d760 0x7f3da8156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:20.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.781+0000 7f3dae400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3da815d760 0x7f3da8156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40996/0 (socket says 172.21.15.100:40996) 2024-09-17T04:32:20.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.781+0000 7f3dae400640 1 -- 172.21.15.100:0/3569440527 learned_addr learned my addr 172.21.15.100:0/3569440527 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:20.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.781+0000 7f3dad000640 1 --2- 172.21.15.100:0/3569440527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3da80b3550 0x7f3da81f8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:20.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.781+0000 7f3dada00640 1 --2- 172.21.15.100:0/3569440527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3da80ad0e0 0x7f3da81513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:20.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.781+0000 7f3dae400640 1 -- 172.21.15.100:0/3569440527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3da80ad0e0 msgr2=0x7f3da81513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:20.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.781+0000 7f3dae400640 1 --2- 172.21.15.100:0/3569440527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3da80ad0e0 0x7f3da81513c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:20.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.781+0000 7f3dae400640 1 -- 172.21.15.100:0/3569440527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3da80b3550 msgr2=0x7f3da81f8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:20.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.781+0000 7f3dae400640 1 --2- 172.21.15.100:0/3569440527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3da80b3550 0x7f3da81f8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:20.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.781+0000 7f3dae400640 1 -- 172.21.15.100:0/3569440527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da809afe0 con 0x7f3da815d760 2024-09-17T04:32:20.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.781+0000 7f3dad000640 1 --2- 172.21.15.100:0/3569440527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3da80b3550 0x7f3da81f8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:32:20.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.781+0000 7f3dada00640 1 --2- 172.21.15.100:0/3569440527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3da80ad0e0 0x7f3da81513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:32:20.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.782+0000 7f3dae400640 1 --2- 172.21.15.100:0/3569440527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3da815d760 0x7f3da8156010 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f3da000c700 tx=0x7f3da000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:20.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.782+0000 7f3da5600640 1 -- 172.21.15.100:0/3569440527 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da0003b70 con 0x7f3da815d760 2024-09-17T04:32:20.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.782+0000 7f3da5600640 1 -- 172.21.15.100:0/3569440527 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3da0003d10 con 0x7f3da815d760 2024-09-17T04:32:20.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.782+0000 7f3db0400640 1 -- 172.21.15.100:0/3569440527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da81f9700 con 0x7f3da815d760 2024-09-17T04:32:20.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.782+0000 7f3da5600640 1 -- 172.21.15.100:0/3569440527 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da000aab0 con 0x7f3da815d760 2024-09-17T04:32:20.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.782+0000 7f3db0400640 1 -- 172.21.15.100:0/3569440527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da81f59b0 con 0x7f3da815d760 2024-09-17T04:32:20.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.783+0000 7f3db0400640 1 -- 172.21.15.100:0/3569440527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3da81f9470 con 0x7f3da815d760 2024-09-17T04:32:20.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.784+0000 7f3da5600640 1 -- 172.21.15.100:0/3569440527 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f3da000ac50 con 0x7f3da815d760 2024-09-17T04:32:20.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.785+0000 7f3da5600640 1 --2- 172.21.15.100:0/3569440527 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3d88079250 0x7f3d8807b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:20.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.785+0000 7f3da5600640 1 -- 172.21.15.100:0/3569440527 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5794+0+0 (secure 0 0 0) 0x7f3da009b700 con 0x7f3da815d760 2024-09-17T04:32:20.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.785+0000 7f3dada00640 1 --2- 172.21.15.100:0/3569440527 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3d88079250 0x7f3d8807b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:20.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.786+0000 7f3dada00640 1 --2- 172.21.15.100:0/3569440527 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3d88079250 0x7f3d8807b760 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0013f0 tx=0x7f3d9c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:20.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:20.790+0000 7f3da5600640 1 -- 172.21.15.100:0/3569440527 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3da809afe0 con 0x7f3da815d760 2024-09-17T04:32:21.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.012+0000 7f3db0400640 1 -- 172.21.15.100:0/3569440527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3da80ad4f0 con 0x7f3da815d760 2024-09-17T04:32:21.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.013+0000 7f3da5600640 1 -- 172.21.15.100:0/3569440527 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5239 (secure 0 0 0) 0x7f3da00631d0 con 0x7f3da815d760 2024-09-17T04:32:21.013 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:32:21.014 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":92,"btime":"2024-09-17T04:32:04:868771+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6719,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1392663867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1392663867},{"type":"v1","addr":"172.21.15.184:6801","nonce":1392663867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":7516,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3257629487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3257629487},{"type":"v1","addr":"172.21.15.184:6835","nonce":3257629487}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":7962,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/2131565869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2131565869},{"type":"v1","addr":"172.21.15.100:6834","nonce":2131565869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:31:53.292918+0000","modified":"2024-09-17T04:31:56.320263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7965},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7965":{"gid":7965,"name":"c","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3047460644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":3047460644},{"type":"v1","addr":"172.21.15.100:6835","nonce":3047460644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7965,"qdb_cluster":[7965]},"id":10}]} 2024-09-17T04:32:21.014 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 92 2024-09-17T04:32:21.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.015+0000 7f3db0400640 1 -- 172.21.15.100:0/3569440527 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3d88079250 msgr2=0x7f3d8807b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:21.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.015+0000 7f3db0400640 1 --2- 172.21.15.100:0/3569440527 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3d88079250 0x7f3d8807b760 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0013f0 tx=0x7f3d9c009040 comp rx=0 tx=0).stop 2024-09-17T04:32:21.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.015+0000 7f3db0400640 1 -- 172.21.15.100:0/3569440527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3da815d760 msgr2=0x7f3da8156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:21.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.015+0000 7f3db0400640 1 --2- 172.21.15.100:0/3569440527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3da815d760 0x7f3da8156010 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f3da000c700 tx=0x7f3da000cbc0 comp rx=0 tx=0).stop 2024-09-17T04:32:21.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.015+0000 7f3db0400640 1 -- 172.21.15.100:0/3569440527 shutdown_connections 2024-09-17T04:32:21.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.015+0000 7f3db0400640 1 --2- 172.21.15.100:0/3569440527 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3d88079250 0x7f3d8807b760 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:21.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.015+0000 7f3db0400640 1 --2- 172.21.15.100:0/3569440527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3da815d760 0x7f3da8156010 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:21.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.015+0000 7f3db0400640 1 --2- 172.21.15.100:0/3569440527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3da80b3550 0x7f3da81f8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:21.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.015+0000 7f3db0400640 1 --2- 172.21.15.100:0/3569440527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3da80ad0e0 0x7f3da81513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:21.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.015+0000 7f3db0400640 1 -- 172.21.15.100:0/3569440527 >> 172.21.15.100:0/3569440527 conn(0x7f3da806d730 msgr2=0x7f3da80b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:21.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.015+0000 7f3db0400640 1 -- 172.21.15.100:0/3569440527 shutdown_connections 2024-09-17T04:32:21.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.015+0000 7f3db0400640 1 -- 172.21.15.100:0/3569440527 wait complete. 2024-09-17T04:32:21.037 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:32:21.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.255+0000 7f7882800640 1 -- 172.21.15.100:0/1298449354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7864005250 msgr2=0x7f786400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:21.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.255+0000 7f7882800640 1 --2- 172.21.15.100:0/1298449354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7864005250 0x7f786400b9c0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f7874001dc0 tx=0x7f7864005830 comp rx=0 tx=0).stop 2024-09-17T04:32:21.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.255+0000 7f7882800640 1 -- 172.21.15.100:0/1298449354 shutdown_connections 2024-09-17T04:32:21.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.256+0000 7f7882800640 1 --2- 172.21.15.100:0/1298449354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7864005250 0x7f786400b9c0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:21.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.256+0000 7f7882800640 1 -- 172.21.15.100:0/1298449354 >> 172.21.15.100:0/1298449354 conn(0x7f787c0b1e40 msgr2=0x7f787c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:21.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.256+0000 7f7882800640 1 -- 172.21.15.100:0/1298449354 shutdown_connections 2024-09-17T04:32:21.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.256+0000 7f7882800640 1 -- 172.21.15.100:0/1298449354 wait complete. 2024-09-17T04:32:21.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.258+0000 7f7882800640 1 Processor -- start 2024-09-17T04:32:21.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.258+0000 7f7882800640 1 -- start start 2024-09-17T04:32:21.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.259+0000 7f7882800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7864005250 0x7f787c14dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:21.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.259+0000 7f7882800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f787c15d570 0x7f787c140750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:21.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.259+0000 7f7882800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f787c0b08b0 0x7f787c0b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:21.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.259+0000 7f7882800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f787c072e60 con 0x7f7864005250 2024-09-17T04:32:21.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.259+0000 7f7882800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f787c0702c0 con 0x7f787c15d570 2024-09-17T04:32:21.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.259+0000 7f7882800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f787c072fe0 con 0x7f787c0b08b0 2024-09-17T04:32:21.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.260+0000 7f7881400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7864005250 0x7f787c14dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:21.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.260+0000 7f7881400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7864005250 0x7f787c14dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41002/0 (socket says 172.21.15.100:41002) 2024-09-17T04:32:21.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.260+0000 7f7881400640 1 -- 172.21.15.100:0/2515657291 learned_addr learned my addr 172.21.15.100:0/2515657291 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:21.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.260+0000 7f7880a00640 1 --2- 172.21.15.100:0/2515657291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f787c15d570 0x7f787c140750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:21.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.260+0000 7f7881e00640 1 --2- 172.21.15.100:0/2515657291 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f787c0b08b0 0x7f787c0b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:21.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.260+0000 7f7881400640 1 -- 172.21.15.100:0/2515657291 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f787c0b08b0 msgr2=0x7f787c0b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:21.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.260+0000 7f7881400640 1 --2- 172.21.15.100:0/2515657291 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f787c0b08b0 0x7f787c0b0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:21.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.260+0000 7f7881400640 1 -- 172.21.15.100:0/2515657291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f787c15d570 msgr2=0x7f787c140750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:21.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.260+0000 7f7881400640 1 --2- 172.21.15.100:0/2515657291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f787c15d570 0x7f787c140750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:21.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.260+0000 7f7881400640 1 -- 172.21.15.100:0/2515657291 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787c0b1200 con 0x7f7864005250 2024-09-17T04:32:21.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.260+0000 7f7881400640 1 --2- 172.21.15.100:0/2515657291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7864005250 0x7f787c14dbe0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f7870006680 tx=0x7f78700039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:21.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.260+0000 7f7879600640 1 -- 172.21.15.100:0/2515657291 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870011700 con 0x7f7864005250 2024-09-17T04:32:21.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.260+0000 7f7879600640 1 -- 172.21.15.100:0/2515657291 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7870016020 con 0x7f7864005250 2024-09-17T04:32:21.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.261+0000 7f7879600640 1 -- 172.21.15.100:0/2515657291 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870011aa0 con 0x7f7864005250 2024-09-17T04:32:21.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.261+0000 7f7882800640 1 -- 172.21.15.100:0/2515657291 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f787c15dd80 con 0x7f7864005250 2024-09-17T04:32:21.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.261+0000 7f7882800640 1 -- 172.21.15.100:0/2515657291 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f787c15e340 con 0x7f7864005250 2024-09-17T04:32:21.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.262+0000 7f7882800640 1 -- 172.21.15.100:0/2515657291 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f787c073540 con 0x7f7864005250 2024-09-17T04:32:21.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.263+0000 7f7879600640 1 -- 172.21.15.100:0/2515657291 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f7870011c40 con 0x7f7864005250 2024-09-17T04:32:21.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.264+0000 7f7879600640 1 --2- 172.21.15.100:0/2515657291 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f785c079250 0x7f785c07b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:21.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.264+0000 7f7879600640 1 -- 172.21.15.100:0/2515657291 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5794+0+0 (secure 0 0 0) 0x7f787009c260 con 0x7f7864005250 2024-09-17T04:32:21.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.264+0000 7f7880a00640 1 --2- 172.21.15.100:0/2515657291 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f785c079250 0x7f785c07b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:21.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.265+0000 7f7880a00640 1 --2- 172.21.15.100:0/2515657291 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f785c079250 0x7f785c07b760 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f787c153370 tx=0x7f7868005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:21.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.268+0000 7f7879600640 1 -- 172.21.15.100:0/2515657291 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f7870020030 con 0x7f7864005250 2024-09-17T04:32:21.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.453+0000 7f7882800640 1 -- 172.21.15.100:0/2515657291 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f787c1530c0 con 0x7f7864005250 2024-09-17T04:32:21.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.454+0000 7f7879600640 1 -- 172.21.15.100:0/2515657291 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+16926 (secure 0 0 0) 0x7f787c0b1200 con 0x7f7864005250 2024-09-17T04:32:21.454 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:32:21.455 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":134,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:32:07.884372+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:31:51.164879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-17T04:31:52.259089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.1b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:32:21.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.456+0000 7f7882800640 1 -- 172.21.15.100:0/2515657291 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f785c079250 msgr2=0x7f785c07b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:21.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.456+0000 7f7882800640 1 --2- 172.21.15.100:0/2515657291 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f785c079250 0x7f785c07b760 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f787c153370 tx=0x7f7868005f90 comp rx=0 tx=0).stop 2024-09-17T04:32:21.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.456+0000 7f7882800640 1 -- 172.21.15.100:0/2515657291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7864005250 msgr2=0x7f787c14dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:21.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.456+0000 7f7882800640 1 --2- 172.21.15.100:0/2515657291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7864005250 0x7f787c14dbe0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f7870006680 tx=0x7f78700039c0 comp rx=0 tx=0).stop 2024-09-17T04:32:21.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.456+0000 7f7882800640 1 -- 172.21.15.100:0/2515657291 shutdown_connections 2024-09-17T04:32:21.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.456+0000 7f7882800640 1 --2- 172.21.15.100:0/2515657291 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f785c079250 0x7f785c07b760 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:21.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.456+0000 7f7882800640 1 --2- 172.21.15.100:0/2515657291 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f787c0b08b0 0x7f787c0b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:21.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.456+0000 7f7882800640 1 --2- 172.21.15.100:0/2515657291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f787c15d570 0x7f787c140750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:21.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.456+0000 7f7882800640 1 --2- 172.21.15.100:0/2515657291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7864005250 0x7f787c14dbe0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:21.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.456+0000 7f7882800640 1 -- 172.21.15.100:0/2515657291 >> 172.21.15.100:0/2515657291 conn(0x7f787c0b1e40 msgr2=0x7f787c153430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:21.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.456+0000 7f7882800640 1 -- 172.21.15.100:0/2515657291 shutdown_connections 2024-09-17T04:32:21.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.457+0000 7f7882800640 1 -- 172.21.15.100:0/2515657291 wait complete. 2024-09-17T04:32:21.483 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T04:32:21.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.706+0000 7f1e23e00640 1 -- 172.21.15.100:0/1432427988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e240b2f60 msgr2=0x7f1e2415a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:21.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.706+0000 7f1e23e00640 1 --2- 172.21.15.100:0/1432427988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e240b2f60 0x7f1e2415a0f0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f1e240aea00 tx=0x7f1e0c02a4d0 comp rx=0 tx=0).stop 2024-09-17T04:32:21.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.706+0000 7f1e23e00640 1 -- 172.21.15.100:0/1432427988 shutdown_connections 2024-09-17T04:32:21.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.706+0000 7f1e23e00640 1 --2- 172.21.15.100:0/1432427988 >> v2:172.21.15.184:3300/0 conn(0x7f1e2415d670 0x7f1e24070530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:21.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.706+0000 7f1e23e00640 1 --2- 172.21.15.100:0/1432427988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e240b2f60 0x7f1e2415a0f0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:21.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.706+0000 7f1e23e00640 1 --2- 172.21.15.100:0/1432427988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e240ad9c0 0x7f1e240af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:21.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.706+0000 7f1e23e00640 1 -- 172.21.15.100:0/1432427988 >> 172.21.15.100:0/1432427988 conn(0x7f1e240b1e40 msgr2=0x7f1e240b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:21.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.706+0000 7f1e23e00640 1 -- 172.21.15.100:0/1432427988 shutdown_connections 2024-09-17T04:32:21.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.706+0000 7f1e23e00640 1 -- 172.21.15.100:0/1432427988 wait complete. 2024-09-17T04:32:21.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.708+0000 7f1e23e00640 1 Processor -- start 2024-09-17T04:32:21.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.709+0000 7f1e23e00640 1 -- start start 2024-09-17T04:32:21.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.710+0000 7f1e23e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e240ad9c0 0x7f1e24154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:21.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.710+0000 7f1e23e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e240b2f60 0x7f1e240736c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:21.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.710+0000 7f1e23e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e2415d670 0x7f1e24073c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:21.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.710+0000 7f1e23e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2406f290 con 0x7f1e240ad9c0 2024-09-17T04:32:21.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.710+0000 7f1e23e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1e24070c80 con 0x7f1e240b2f60 2024-09-17T04:32:21.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.710+0000 7f1e23e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1e2406f410 con 0x7f1e2415d670 2024-09-17T04:32:21.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.710+0000 7f1e22a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e240ad9c0 0x7f1e24154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:21.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.710+0000 7f1e22a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e240ad9c0 0x7f1e24154d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41028/0 (socket says 172.21.15.100:41028) 2024-09-17T04:32:21.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.710+0000 7f1e22a00640 1 -- 172.21.15.100:0/1799312059 learned_addr learned my addr 172.21.15.100:0/1799312059 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:21.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.711+0000 7f1e23400640 1 --2- 172.21.15.100:0/1799312059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e2415d670 0x7f1e24073c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:21.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.711+0000 7f1e1a000640 1 --2- 172.21.15.100:0/1799312059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e240b2f60 0x7f1e240736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:21.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.711+0000 7f1e22a00640 1 -- 172.21.15.100:0/1799312059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e2415d670 msgr2=0x7f1e24073c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:21.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.711+0000 7f1e22a00640 1 --2- 172.21.15.100:0/1799312059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e2415d670 0x7f1e24073c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:21.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.711+0000 7f1e22a00640 1 -- 172.21.15.100:0/1799312059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e240b2f60 msgr2=0x7f1e240736c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:21.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.711+0000 7f1e22a00640 1 --2- 172.21.15.100:0/1799312059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e240b2f60 0x7f1e240736c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:21.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.711+0000 7f1e22a00640 1 -- 172.21.15.100:0/1799312059 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e24074480 con 0x7f1e240ad9c0 2024-09-17T04:32:21.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.711+0000 7f1e22a00640 1 --2- 172.21.15.100:0/1799312059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e240ad9c0 0x7f1e24154d60 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400d6d0 tx=0x7f1e1400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:21.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.711+0000 7f1e1be00640 1 -- 172.21.15.100:0/1799312059 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14017070 con 0x7f1e240ad9c0 2024-09-17T04:32:21.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.711+0000 7f1e1be00640 1 -- 172.21.15.100:0/1799312059 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e14005330 con 0x7f1e240ad9c0 2024-09-17T04:32:21.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.711+0000 7f1e23e00640 1 -- 172.21.15.100:0/1799312059 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e241f5290 con 0x7f1e240ad9c0 2024-09-17T04:32:21.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.711+0000 7f1e1be00640 1 -- 172.21.15.100:0/1799312059 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14009820 con 0x7f1e240ad9c0 2024-09-17T04:32:21.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.711+0000 7f1e23e00640 1 -- 172.21.15.100:0/1799312059 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e241577a0 con 0x7f1e240ad9c0 2024-09-17T04:32:21.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.712+0000 7f1e23e00640 1 -- 172.21.15.100:0/1799312059 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e2406f930 con 0x7f1e240ad9c0 2024-09-17T04:32:21.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.714+0000 7f1e1be00640 1 -- 172.21.15.100:0/1799312059 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f1e14009bd0 con 0x7f1e240ad9c0 2024-09-17T04:32:21.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.715+0000 7f1e1be00640 1 --2- 172.21.15.100:0/1799312059 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1e00079250 0x7f1e0007b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:21.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.715+0000 7f1e1be00640 1 -- 172.21.15.100:0/1799312059 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5794+0+0 (secure 0 0 0) 0x7f1e1409bda0 con 0x7f1e240ad9c0 2024-09-17T04:32:21.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.715+0000 7f1e1a000640 1 --2- 172.21.15.100:0/1799312059 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1e00079250 0x7f1e0007b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:21.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.715+0000 7f1e1a000640 1 --2- 172.21.15.100:0/1799312059 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1e00079250 0x7f1e0007b760 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c033000 tx=0x7f1e0c004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:21.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.719+0000 7f1e1be00640 1 -- 172.21.15.100:0/1799312059 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f1e24074480 con 0x7f1e240ad9c0 2024-09-17T04:32:21.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.941+0000 7f1e23e00640 1 -- 172.21.15.100:0/1799312059 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1e240cac20 con 0x7f1e240ad9c0 2024-09-17T04:32:21.942 INFO:tasks.ceph.mon.a.smithi100.stderr:2024-09-17T04:32:21.942+0000 7fd3d0a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T04:32:21.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.943+0000 7f1e1be00640 1 -- 172.21.15.100:0/1799312059 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v93) ==== 161+0+0 (secure 0 0 0) 0x7f1e140636e0 con 0x7f1e240ad9c0 2024-09-17T04:32:21.944 INFO:teuthology.orchestra.run.smithi100.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T04:32:21.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.947+0000 7f1e23e00640 1 -- 172.21.15.100:0/1799312059 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1e00079250 msgr2=0x7f1e0007b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:21.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.947+0000 7f1e23e00640 1 --2- 172.21.15.100:0/1799312059 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1e00079250 0x7f1e0007b760 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c033000 tx=0x7f1e0c004730 comp rx=0 tx=0).stop 2024-09-17T04:32:21.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.947+0000 7f1e23e00640 1 -- 172.21.15.100:0/1799312059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e240ad9c0 msgr2=0x7f1e24154d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:21.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.947+0000 7f1e23e00640 1 --2- 172.21.15.100:0/1799312059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e240ad9c0 0x7f1e24154d60 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400d6d0 tx=0x7f1e1400db90 comp rx=0 tx=0).stop 2024-09-17T04:32:21.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.947+0000 7f1e23e00640 1 -- 172.21.15.100:0/1799312059 shutdown_connections 2024-09-17T04:32:21.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.947+0000 7f1e23e00640 1 --2- 172.21.15.100:0/1799312059 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1e00079250 0x7f1e0007b760 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:21.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.947+0000 7f1e23e00640 1 --2- 172.21.15.100:0/1799312059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e2415d670 0x7f1e24073c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:21.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.947+0000 7f1e23e00640 1 --2- 172.21.15.100:0/1799312059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e240b2f60 0x7f1e240736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:21.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.947+0000 7f1e23e00640 1 --2- 172.21.15.100:0/1799312059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e240ad9c0 0x7f1e24154d60 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:21.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.947+0000 7f1e23e00640 1 -- 172.21.15.100:0/1799312059 >> 172.21.15.100:0/1799312059 conn(0x7f1e240b1e40 msgr2=0x7f1e2415bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:21.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.947+0000 7f1e23e00640 1 -- 172.21.15.100:0/1799312059 shutdown_connections 2024-09-17T04:32:21.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:21.947+0000 7f1e23e00640 1 -- 172.21.15.100:0/1799312059 wait complete. 2024-09-17T04:32:21.954 INFO:tasks.ceph.mds.c.smithi100.stderr: -3152> 2024-09-17T04:31:59.617+0000 7fb7b7c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T04:32:21.973 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T04:32:22.082 INFO:tasks.ceph.mds.c.smithi100.stdout:starting mds.c at 2024-09-17T04:32:22.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.199+0000 7fccd9600640 1 -- 172.21.15.100:0/2065631823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccd4145550 msgr2=0x7fccd406fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:22.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.199+0000 7fccd9600640 1 --2- 172.21.15.100:0/2065631823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccd4145550 0x7fccd406fb40 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fccc4005fd0 tx=0x7fccc402abc0 comp rx=0 tx=0).stop 2024-09-17T04:32:22.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.199+0000 7fccd9600640 1 -- 172.21.15.100:0/2065631823 shutdown_connections 2024-09-17T04:32:22.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.199+0000 7fccd9600640 1 --2- 172.21.15.100:0/2065631823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccd4145550 0x7fccd406fb40 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:22.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.199+0000 7fccd9600640 1 -- 172.21.15.100:0/2065631823 >> 172.21.15.100:0/2065631823 conn(0x7fccd406d1c0 msgr2=0x7fccd406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:22.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.200+0000 7fccd9600640 1 -- 172.21.15.100:0/2065631823 shutdown_connections 2024-09-17T04:32:22.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.200+0000 7fccd9600640 1 -- 172.21.15.100:0/2065631823 wait complete. 2024-09-17T04:32:22.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.202+0000 7fccd9600640 1 Processor -- start 2024-09-17T04:32:22.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.202+0000 7fccd9600640 1 -- start start 2024-09-17T04:32:22.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.203+0000 7fccd9600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccd4145550 0x7fccd415c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:22.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.203+0000 7fccd9600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccd4145a40 0x7fccd4160040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:22.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.203+0000 7fccd9600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccd413f5c0 0x7fccd4160580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:22.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.203+0000 7fccd9600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fccd40ce360 con 0x7fccd4145a40 2024-09-17T04:32:22.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.203+0000 7fccd9600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fccd40ce1e0 con 0x7fccd413f5c0 2024-09-17T04:32:22.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.203+0000 7fccd9600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fccd40ce4e0 con 0x7fccd4145550 2024-09-17T04:32:22.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.203+0000 7fccd2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccd4145a40 0x7fccd4160040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:22.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.203+0000 7fccd2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccd4145a40 0x7fccd4160040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41074/0 (socket says 172.21.15.100:41074) 2024-09-17T04:32:22.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.204+0000 7fccd2000640 1 -- 172.21.15.100:0/1685427718 learned_addr learned my addr 172.21.15.100:0/1685427718 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:22.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.204+0000 7fccd3400640 1 --2- 172.21.15.100:0/1685427718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccd413f5c0 0x7fccd4160580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:22.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.204+0000 7fccd2a00640 1 --2- 172.21.15.100:0/1685427718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccd4145550 0x7fccd415c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.204+0000 7fccd2000640 1 -- 172.21.15.100:0/1685427718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccd4145550 msgr2=0x7fccd415c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.204+0000 7fccd2000640 1 --2- 172.21.15.100:0/1685427718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccd4145550 0x7fccd415c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.204+0000 7fccd2000640 1 -- 172.21.15.100:0/1685427718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccd413f5c0 msgr2=0x7fccd4160580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.204+0000 7fccd2000640 1 --2- 172.21.15.100:0/1685427718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccd413f5c0 0x7fccd4160580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.204+0000 7fccd2000640 1 -- 172.21.15.100:0/1685427718 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd40adbf0 con 0x7fccd4145a40 2024-09-17T04:32:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.204+0000 7fccd2000640 1 --2- 172.21.15.100:0/1685427718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccd4145a40 0x7fccd4160040 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fccc000c3a0 tx=0x7fccc000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.205+0000 7fcccaa00640 1 -- 172.21.15.100:0/1685427718 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0017070 con 0x7fccd4145a40 2024-09-17T04:32:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.205+0000 7fcccaa00640 1 -- 172.21.15.100:0/1685427718 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fccc00053c0 con 0x7fccd4145a40 2024-09-17T04:32:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.205+0000 7fcccaa00640 1 -- 172.21.15.100:0/1685427718 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0003850 con 0x7fccd4145a40 2024-09-17T04:32:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.205+0000 7fccd9600640 1 -- 172.21.15.100:0/1685427718 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd40ae0f0 con 0x7fccd4145a40 2024-09-17T04:32:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.205+0000 7fccd9600640 1 -- 172.21.15.100:0/1685427718 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd40ae6b0 con 0x7fccd4145a40 2024-09-17T04:32:22.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.206+0000 7fccd9600640 1 -- 172.21.15.100:0/1685427718 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccd40ade60 con 0x7fccd4145a40 2024-09-17T04:32:22.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.207+0000 7fcccaa00640 1 -- 172.21.15.100:0/1685427718 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7fccc001a070 con 0x7fccd4145a40 2024-09-17T04:32:22.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.208+0000 7fcccaa00640 1 --2- 172.21.15.100:0/1685427718 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fccb4079250 0x7fccb407b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:22.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.208+0000 7fcccaa00640 1 -- 172.21.15.100:0/1685427718 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5880+0+0 (secure 0 0 0) 0x7fccc009ae20 con 0x7fccd4145a40 2024-09-17T04:32:22.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.208+0000 7fccd2a00640 1 --2- 172.21.15.100:0/1685427718 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fccb4079250 0x7fccb407b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:22.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.209+0000 7fccd2a00640 1 --2- 172.21.15.100:0/1685427718 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fccb4079250 0x7fccb407b760 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fccc4035000 tx=0x7fccc4004ab0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:22.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.213+0000 7fcccaa00640 1 -- 172.21.15.100:0/1685427718 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fccc0062080 con 0x7fccd4145a40 2024-09-17T04:32:22.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.430+0000 7fccd9600640 1 -- 172.21.15.100:0/1685427718 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fccd406fb40 con 0x7fccd4145a40 2024-09-17T04:32:22.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.977+0000 7fcccaa00640 1 -- 172.21.15.100:0/1685427718 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v94) ==== 104+0+0 (secure 0 0 0) 0x7fccc0066ab0 con 0x7fccd4145a40 2024-09-17T04:32:22.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.980+0000 7fccd9600640 1 -- 172.21.15.100:0/1685427718 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fccb4079250 msgr2=0x7fccb407b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:22.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.980+0000 7fccd9600640 1 --2- 172.21.15.100:0/1685427718 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fccb4079250 0x7fccb407b760 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fccc4035000 tx=0x7fccc4004ab0 comp rx=0 tx=0).stop 2024-09-17T04:32:22.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.980+0000 7fccd9600640 1 -- 172.21.15.100:0/1685427718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccd4145a40 msgr2=0x7fccd4160040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:22.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.980+0000 7fccd9600640 1 --2- 172.21.15.100:0/1685427718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccd4145a40 0x7fccd4160040 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fccc000c3a0 tx=0x7fccc000c860 comp rx=0 tx=0).stop 2024-09-17T04:32:22.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.980+0000 7fccd9600640 1 -- 172.21.15.100:0/1685427718 shutdown_connections 2024-09-17T04:32:22.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.980+0000 7fccd9600640 1 --2- 172.21.15.100:0/1685427718 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fccb4079250 0x7fccb407b760 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:22.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.980+0000 7fccd9600640 1 --2- 172.21.15.100:0/1685427718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccd413f5c0 0x7fccd4160580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:22.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.980+0000 7fccd9600640 1 --2- 172.21.15.100:0/1685427718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccd4145a40 0x7fccd4160040 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:22.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.980+0000 7fccd9600640 1 --2- 172.21.15.100:0/1685427718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccd4145550 0x7fccd415c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:22.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.980+0000 7fccd9600640 1 -- 172.21.15.100:0/1685427718 >> 172.21.15.100:0/1685427718 conn(0x7fccd406d1c0 msgr2=0x7fccd41445d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:22.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.980+0000 7fccd9600640 1 -- 172.21.15.100:0/1685427718 shutdown_connections 2024-09-17T04:32:22.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:22.980+0000 7fccd9600640 1 -- 172.21.15.100:0/1685427718 wait complete. 2024-09-17T04:32:23.005 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T04:32:23.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.231+0000 7f3b16400640 1 -- 172.21.15.100:0/532319066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b1006d4b0 msgr2=0x7f3b100b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:23.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.231+0000 7f3b16400640 1 --2- 172.21.15.100:0/532319066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b1006d4b0 0x7f3b100b4620 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f3b10071040 tx=0x7f3af802a5f0 comp rx=0 tx=0).stop 2024-09-17T04:32:23.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.231+0000 7f3b16400640 1 -- 172.21.15.100:0/532319066 shutdown_connections 2024-09-17T04:32:23.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.231+0000 7f3b16400640 1 --2- 172.21.15.100:0/532319066 >> v2:172.21.15.184:3300/0 conn(0x7f3b100af100 0x7f3b100af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:23.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.231+0000 7f3b16400640 1 --2- 172.21.15.100:0/532319066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b1006d4b0 0x7f3b100b4620 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:23.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.231+0000 7f3b16400640 1 -- 172.21.15.100:0/532319066 >> 172.21.15.100:0/532319066 conn(0x7f3b100b2540 msgr2=0x7f3b100d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:23.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.231+0000 7f3b16400640 1 -- 172.21.15.100:0/532319066 shutdown_connections 2024-09-17T04:32:23.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.231+0000 7f3b16400640 1 -- 172.21.15.100:0/532319066 wait complete. 2024-09-17T04:32:23.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.233+0000 7f3b16400640 1 Processor -- start 2024-09-17T04:32:23.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.234+0000 7f3b16400640 1 -- start start 2024-09-17T04:32:23.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.235+0000 7f3b16400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b100af100 0x7f3b101f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:23.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.235+0000 7f3b16400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b100cc830 0x7f3b10073690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:23.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.235+0000 7f3b16400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b10074ae0 0x7f3b10073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:23.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.235+0000 7f3b16400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3b100b3620 con 0x7f3b100cc830 2024-09-17T04:32:23.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.235+0000 7f3b16400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3b100b34a0 con 0x7f3b100af100 2024-09-17T04:32:23.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.235+0000 7f3b16400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3b100b37a0 con 0x7f3b10074ae0 2024-09-17T04:32:23.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.235+0000 7f3b07e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b100cc830 0x7f3b10073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:23.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.235+0000 7f3b07e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b100cc830 0x7f3b10073690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41094/0 (socket says 172.21.15.100:41094) 2024-09-17T04:32:23.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.235+0000 7f3b07e00640 1 -- 172.21.15.100:0/1856023988 learned_addr learned my addr 172.21.15.100:0/1856023988 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:23.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.235+0000 7f3b0f400640 1 --2- 172.21.15.100:0/1856023988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b100af100 0x7f3b101f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:23.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.235+0000 7f3b0fe00640 1 --2- 172.21.15.100:0/1856023988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b10074ae0 0x7f3b10073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:23.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.236+0000 7f3b07e00640 1 -- 172.21.15.100:0/1856023988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b10074ae0 msgr2=0x7f3b10073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:23.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.236+0000 7f3b07e00640 1 --2- 172.21.15.100:0/1856023988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b10074ae0 0x7f3b10073bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:23.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.236+0000 7f3b07e00640 1 -- 172.21.15.100:0/1856023988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b100af100 msgr2=0x7f3b101f7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:23.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.236+0000 7f3b07e00640 1 --2- 172.21.15.100:0/1856023988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b100af100 0x7f3b101f7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:23.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.236+0000 7f3b07e00640 1 -- 172.21.15.100:0/1856023988 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b10156de0 con 0x7f3b100cc830 2024-09-17T04:32:23.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.236+0000 7f3b0f400640 1 --2- 172.21.15.100:0/1856023988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b100af100 0x7f3b101f7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:23.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.236+0000 7f3b07e00640 1 --2- 172.21.15.100:0/1856023988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b100cc830 0x7f3b10073690 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f3af8000c00 tx=0x7f3af8003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:23.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.236+0000 7f3b07400640 1 -- 172.21.15.100:0/1856023988 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af803e030 con 0x7f3b100cc830 2024-09-17T04:32:23.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.236+0000 7f3b07400640 1 -- 172.21.15.100:0/1856023988 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3af8003e00 con 0x7f3b100cc830 2024-09-17T04:32:23.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.236+0000 7f3b07400640 1 -- 172.21.15.100:0/1856023988 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8035040 con 0x7f3b100cc830 2024-09-17T04:32:23.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.236+0000 7f3b16400640 1 -- 172.21.15.100:0/1856023988 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b101570c0 con 0x7f3b100cc830 2024-09-17T04:32:23.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.236+0000 7f3b16400640 1 -- 172.21.15.100:0/1856023988 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b1009a800 con 0x7f3b100cc830 2024-09-17T04:32:23.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.238+0000 7f3b16400640 1 -- 172.21.15.100:0/1856023988 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b100b3cc0 con 0x7f3b100cc830 2024-09-17T04:32:23.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.239+0000 7f3b07400640 1 -- 172.21.15.100:0/1856023988 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f3af8041030 con 0x7f3b100cc830 2024-09-17T04:32:23.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.240+0000 7f3b07400640 1 --2- 172.21.15.100:0/1856023988 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3aec079250 0x7f3aec07b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:23.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.240+0000 7f3b07400640 1 -- 172.21.15.100:0/1856023988 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5880+0+0 (secure 0 0 0) 0x7f3af80bc280 con 0x7f3b100cc830 2024-09-17T04:32:23.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.240+0000 7f3b0f400640 1 --2- 172.21.15.100:0/1856023988 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3aec079250 0x7f3aec07b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:23.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.241+0000 7f3b0f400640 1 --2- 172.21.15.100:0/1856023988 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3aec079250 0x7f3aec07b760 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3b00007fd0 tx=0x7f3b00009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:23.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.244+0000 7f3b07400640 1 -- 172.21.15.100:0/1856023988 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3af8083c00 con 0x7f3b100cc830 2024-09-17T04:32:23.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.429+0000 7f3b16400640 1 -- 172.21.15.100:0/1856023988 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3b1015c7b0 con 0x7f3b100cc830 2024-09-17T04:32:23.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.982+0000 7f3b07400640 1 -- 172.21.15.100:0/1856023988 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v136) ==== 181+0+0 (secure 0 0 0) 0x7f3af8083500 con 0x7f3b100cc830 2024-09-17T04:32:23.983 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' removed 2024-09-17T04:32:23.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.986+0000 7f3b16400640 1 -- 172.21.15.100:0/1856023988 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3aec079250 msgr2=0x7f3aec07b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:23.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.986+0000 7f3b16400640 1 --2- 172.21.15.100:0/1856023988 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3aec079250 0x7f3aec07b760 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3b00007fd0 tx=0x7f3b00009040 comp rx=0 tx=0).stop 2024-09-17T04:32:23.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.986+0000 7f3b16400640 1 -- 172.21.15.100:0/1856023988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b100cc830 msgr2=0x7f3b10073690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:23.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.986+0000 7f3b16400640 1 --2- 172.21.15.100:0/1856023988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b100cc830 0x7f3b10073690 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f3af8000c00 tx=0x7f3af8003910 comp rx=0 tx=0).stop 2024-09-17T04:32:23.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.986+0000 7f3b16400640 1 -- 172.21.15.100:0/1856023988 shutdown_connections 2024-09-17T04:32:23.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.986+0000 7f3b16400640 1 --2- 172.21.15.100:0/1856023988 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3aec079250 0x7f3aec07b760 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:23.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.986+0000 7f3b16400640 1 --2- 172.21.15.100:0/1856023988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b10074ae0 0x7f3b10073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:23.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.986+0000 7f3b16400640 1 --2- 172.21.15.100:0/1856023988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b100cc830 0x7f3b10073690 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:23.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.986+0000 7f3b16400640 1 --2- 172.21.15.100:0/1856023988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b100af100 0x7f3b101f7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:23.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.986+0000 7f3b16400640 1 -- 172.21.15.100:0/1856023988 >> 172.21.15.100:0/1856023988 conn(0x7f3b100b2540 msgr2=0x7f3b10068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:23.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.987+0000 7f3b16400640 1 -- 172.21.15.100:0/1856023988 shutdown_connections 2024-09-17T04:32:23.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:23.987+0000 7f3b16400640 1 -- 172.21.15.100:0/1856023988 wait complete. 2024-09-17T04:32:24.014 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T04:32:24.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.246+0000 7f818c000640 1 -- 172.21.15.100:0/2337537954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81840cc7d0 msgr2=0x7f81840b3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:24.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.246+0000 7f818c000640 1 --2- 172.21.15.100:0/2337537954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81840cc7d0 0x7f81840b3120 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f8178005fd0 tx=0x7f817802a4b0 comp rx=0 tx=0).stop 2024-09-17T04:32:24.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.246+0000 7f818c000640 1 -- 172.21.15.100:0/2337537954 shutdown_connections 2024-09-17T04:32:24.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.246+0000 7f818c000640 1 --2- 172.21.15.100:0/2337537954 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f818415ba50 0x7f81840b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:24.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.246+0000 7f818c000640 1 --2- 172.21.15.100:0/2337537954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81840cc7d0 0x7f81840b3120 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:24.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.246+0000 7f818c000640 1 -- 172.21.15.100:0/2337537954 >> 172.21.15.100:0/2337537954 conn(0x7f81840b2540 msgr2=0x7f81840d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:24.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.246+0000 7f818c000640 1 -- 172.21.15.100:0/2337537954 shutdown_connections 2024-09-17T04:32:24.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.246+0000 7f818c000640 1 -- 172.21.15.100:0/2337537954 wait complete. 2024-09-17T04:32:24.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.248+0000 7f818c000640 1 Processor -- start 2024-09-17T04:32:24.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.248+0000 7f818c000640 1 -- start start 2024-09-17T04:32:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.250+0000 7f818c000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f818406f010 0x7f81841f4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.250+0000 7f818c000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81840cc7d0 0x7f81840c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.250+0000 7f818c000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f818415ba50 0x7f81840c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.250+0000 7f818c000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f81840cb930 con 0x7f818415ba50 2024-09-17T04:32:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.250+0000 7f818c000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f818406ff90 con 0x7f818406f010 2024-09-17T04:32:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.250+0000 7f818c000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f81840cbab0 con 0x7f81840cc7d0 2024-09-17T04:32:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.250+0000 7f818a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f818415ba50 0x7f81840c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.250+0000 7f818a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f818415ba50 0x7f81840c8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41100/0 (socket says 172.21.15.100:41100) 2024-09-17T04:32:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.250+0000 7f818a000640 1 -- 172.21.15.100:0/1210386672 learned_addr learned my addr 172.21.15.100:0/1210386672 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.250+0000 7f8188c00640 1 --2- 172.21.15.100:0/1210386672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81840cc7d0 0x7f81840c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.250+0000 7f8189600640 1 --2- 172.21.15.100:0/1210386672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f818406f010 0x7f81841f4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:24.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.250+0000 7f818a000640 1 -- 172.21.15.100:0/1210386672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81840cc7d0 msgr2=0x7f81840c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:24.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.250+0000 7f818a000640 1 --2- 172.21.15.100:0/1210386672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81840cc7d0 0x7f81840c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:24.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.250+0000 7f818a000640 1 -- 172.21.15.100:0/1210386672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f818406f010 msgr2=0x7f81841f4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:24.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.250+0000 7f818a000640 1 --2- 172.21.15.100:0/1210386672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f818406f010 0x7f81841f4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:24.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.250+0000 7f818a000640 1 -- 172.21.15.100:0/1210386672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81841569e0 con 0x7f818415ba50 2024-09-17T04:32:24.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.251+0000 7f818a000640 1 --2- 172.21.15.100:0/1210386672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f818415ba50 0x7f81840c8750 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f817c002230 tx=0x7f817c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:24.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.251+0000 7f8181600640 1 -- 172.21.15.100:0/1210386672 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817c003b70 con 0x7f818415ba50 2024-09-17T04:32:24.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.251+0000 7f8181600640 1 -- 172.21.15.100:0/1210386672 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f817c003d10 con 0x7f818415ba50 2024-09-17T04:32:24.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.251+0000 7f8181600640 1 -- 172.21.15.100:0/1210386672 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817c005330 con 0x7f818415ba50 2024-09-17T04:32:24.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.251+0000 7f818c000640 1 -- 172.21.15.100:0/1210386672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81840745b0 con 0x7f818415ba50 2024-09-17T04:32:24.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.251+0000 7f818c000640 1 -- 172.21.15.100:0/1210386672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8184157170 con 0x7f818415ba50 2024-09-17T04:32:24.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.252+0000 7f818c000640 1 -- 172.21.15.100:0/1210386672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81840cbfd0 con 0x7f818415ba50 2024-09-17T04:32:24.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.253+0000 7f8181600640 1 -- 172.21.15.100:0/1210386672 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f817c0056e0 con 0x7f818415ba50 2024-09-17T04:32:24.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.254+0000 7f8181600640 1 --2- 172.21.15.100:0/1210386672 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8164079250 0x7f816407b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:24.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.254+0000 7f8181600640 1 -- 172.21.15.100:0/1210386672 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5442+0+0 (secure 0 0 0) 0x7f817c09b900 con 0x7f818415ba50 2024-09-17T04:32:24.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.254+0000 7f8189600640 1 --2- 172.21.15.100:0/1210386672 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8164079250 0x7f816407b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:24.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.255+0000 7f8189600640 1 --2- 172.21.15.100:0/1210386672 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8164079250 0x7f816407b760 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8178033000 tx=0x7f8178004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:24.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.259+0000 7f8181600640 1 -- 172.21.15.100:0/1210386672 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f817c063460 con 0x7f818415ba50 2024-09-17T04:32:24.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.441+0000 7f818c000640 1 -- 172.21.15.100:0/1210386672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f81840ccef0 con 0x7f818415ba50 2024-09-17T04:32:24.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.984+0000 7f8181600640 1 -- 172.21.15.100:0/1210386672 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v137) ==== 169+0+0 (secure 0 0 0) 0x7f817c062d60 con 0x7f818415ba50 2024-09-17T04:32:24.985 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' removed 2024-09-17T04:32:24.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.987+0000 7f818c000640 1 -- 172.21.15.100:0/1210386672 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8164079250 msgr2=0x7f816407b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:24.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.987+0000 7f818c000640 1 --2- 172.21.15.100:0/1210386672 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8164079250 0x7f816407b760 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8178033000 tx=0x7f8178004730 comp rx=0 tx=0).stop 2024-09-17T04:32:24.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.987+0000 7f818c000640 1 -- 172.21.15.100:0/1210386672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f818415ba50 msgr2=0x7f81840c8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:24.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.987+0000 7f818c000640 1 --2- 172.21.15.100:0/1210386672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f818415ba50 0x7f81840c8750 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f817c002230 tx=0x7f817c00cb90 comp rx=0 tx=0).stop 2024-09-17T04:32:24.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.988+0000 7f818c000640 1 -- 172.21.15.100:0/1210386672 shutdown_connections 2024-09-17T04:32:24.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.988+0000 7f818c000640 1 --2- 172.21.15.100:0/1210386672 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8164079250 0x7f816407b760 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:24.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.988+0000 7f818c000640 1 --2- 172.21.15.100:0/1210386672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f818415ba50 0x7f81840c8750 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:24.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.988+0000 7f818c000640 1 --2- 172.21.15.100:0/1210386672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81840cc7d0 0x7f81840c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:24.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.988+0000 7f818c000640 1 --2- 172.21.15.100:0/1210386672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f818406f010 0x7f81841f4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:24.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.988+0000 7f818c000640 1 -- 172.21.15.100:0/1210386672 >> 172.21.15.100:0/1210386672 conn(0x7f81840b2540 msgr2=0x7f818415a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:24.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.989+0000 7f818c000640 1 -- 172.21.15.100:0/1210386672 shutdown_connections 2024-09-17T04:32:24.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:24.989+0000 7f818c000640 1 -- 172.21.15.100:0/1210386672 wait complete. 2024-09-17T04:32:25.015 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T04:32:25.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.243+0000 7f82f7600640 1 -- 172.21.15.100:0/166781203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82d8005270 msgr2=0x7f82d800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:25.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.243+0000 7f82f7600640 1 --2- 172.21.15.100:0/166781203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82d8005270 0x7f82d800b9c0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f82f00cf7a0 tx=0x7f82d8005830 comp rx=0 tx=0).stop 2024-09-17T04:32:25.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.243+0000 7f82f5600640 1 -- 172.21.15.100:0/166781203 reap_dead start 2024-09-17T04:32:25.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.243+0000 7f82f7600640 1 -- 172.21.15.100:0/166781203 shutdown_connections 2024-09-17T04:32:25.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.243+0000 7f82f7600640 1 -- 172.21.15.100:0/166781203 >> 172.21.15.100:0/166781203 conn(0x7f82f006d3d0 msgr2=0x7f82f00b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:25.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.243+0000 7f82f7600640 1 -- 172.21.15.100:0/166781203 shutdown_connections 2024-09-17T04:32:25.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.243+0000 7f82f7600640 1 -- 172.21.15.100:0/166781203 wait complete. 2024-09-17T04:32:25.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.246+0000 7f82f7600640 1 Processor -- start 2024-09-17T04:32:25.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.246+0000 7f82f7600640 1 -- start start 2024-09-17T04:32:25.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.247+0000 7f82f7600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82f0156750 0x7f82f01e6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:25.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.247+0000 7f82f7600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82f0072f20 0x7f82f00c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:25.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.247+0000 7f82f7600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82f00c5110 0x7f82f00ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:25.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.247+0000 7f82f7600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f82f0156fc0 con 0x7f82f0156750 2024-09-17T04:32:25.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.247+0000 7f82f7600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f82f0156e40 con 0x7f82f00c5110 2024-09-17T04:32:25.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.247+0000 7f82f7600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f82f0157140 con 0x7f82f0072f20 2024-09-17T04:32:25.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.247+0000 7f82f4c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82f0156750 0x7f82f01e6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:25.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.247+0000 7f82f4c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82f0156750 0x7f82f01e6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38274/0 (socket says 172.21.15.100:38274) 2024-09-17T04:32:25.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.247+0000 7f82f4c00640 1 -- 172.21.15.100:0/2255750612 learned_addr learned my addr 172.21.15.100:0/2255750612 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:25.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.248+0000 7f82f5600640 1 --2- 172.21.15.100:0/2255750612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82f00c5110 0x7f82f00ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:25.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.248+0000 7f82f4c00640 1 -- 172.21.15.100:0/2255750612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82f0072f20 msgr2=0x7f82f00c4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:25.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.248+0000 7f82efe00640 1 --2- 172.21.15.100:0/2255750612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82f0072f20 0x7f82f00c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:25.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.248+0000 7f82f4c00640 1 --2- 172.21.15.100:0/2255750612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82f0072f20 0x7f82f00c4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:25.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.248+0000 7f82f4c00640 1 -- 172.21.15.100:0/2255750612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82f00c5110 msgr2=0x7f82f00ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:25.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.248+0000 7f82f4c00640 1 --2- 172.21.15.100:0/2255750612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82f00c5110 0x7f82f00ae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:25.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.248+0000 7f82f4c00640 1 -- 172.21.15.100:0/2255750612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f00c69b0 con 0x7f82f0156750 2024-09-17T04:32:25.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.248+0000 7f82f5600640 1 --2- 172.21.15.100:0/2255750612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82f00c5110 0x7f82f00ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:32:25.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.248+0000 7f82f4c00640 1 --2- 172.21.15.100:0/2255750612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82f0156750 0x7f82f01e6a20 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f82e4003f20 tx=0x7f82e40016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:25.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.248+0000 7f82ecc00640 1 -- 172.21.15.100:0/2255750612 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e4011640 con 0x7f82f0156750 2024-09-17T04:32:25.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.248+0000 7f82ecc00640 1 -- 172.21.15.100:0/2255750612 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82e4016020 con 0x7f82f0156750 2024-09-17T04:32:25.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.248+0000 7f82f7600640 1 -- 172.21.15.100:0/2255750612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f00c6f30 con 0x7f82f0156750 2024-09-17T04:32:25.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.248+0000 7f82ecc00640 1 -- 172.21.15.100:0/2255750612 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e4011a80 con 0x7f82f0156750 2024-09-17T04:32:25.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.249+0000 7f82f7600640 1 -- 172.21.15.100:0/2255750612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f01554d0 con 0x7f82f0156750 2024-09-17T04:32:25.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.250+0000 7f82f7600640 1 -- 172.21.15.100:0/2255750612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f00afa70 con 0x7f82f0156750 2024-09-17T04:32:25.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.251+0000 7f82ecc00640 1 -- 172.21.15.100:0/2255750612 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f82e4011c20 con 0x7f82f0156750 2024-09-17T04:32:25.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.252+0000 7f82ecc00640 1 --2- 172.21.15.100:0/2255750612 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f82d0079250 0x7f82d007b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:25.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.252+0000 7f82ecc00640 1 -- 172.21.15.100:0/2255750612 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4999+0+0 (secure 0 0 0) 0x7f82e409c3e0 con 0x7f82f0156750 2024-09-17T04:32:25.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.252+0000 7f82efe00640 1 --2- 172.21.15.100:0/2255750612 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f82d0079250 0x7f82d007b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:25.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.253+0000 7f82efe00640 1 --2- 172.21.15.100:0/2255750612 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f82d0079250 0x7f82d007b760 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0045b0 tx=0x7f82dc00b260 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:25.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.256+0000 7f82ecc00640 1 -- 172.21.15.100:0/2255750612 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f82e4020020 con 0x7f82f0156750 2024-09-17T04:32:25.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.440+0000 7f82f7600640 1 -- 172.21.15.100:0/2255750612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f82f006fbc0 con 0x7f82f0156750 2024-09-17T04:32:25.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.444+0000 7f82ecc00640 1 -- 172.21.15.100:0/2255750612 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v47) ==== 71+0+0 (secure 0 0 0) 0x7f82e40640d0 con 0x7f82f0156750 2024-09-17T04:32:25.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.447+0000 7f82f7600640 1 -- 172.21.15.100:0/2255750612 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f82d0079250 msgr2=0x7f82d007b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.447+0000 7f82f7600640 1 --2- 172.21.15.100:0/2255750612 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f82d0079250 0x7f82d007b760 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0045b0 tx=0x7f82dc00b260 comp rx=0 tx=0).stop 2024-09-17T04:32:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.447+0000 7f82f7600640 1 -- 172.21.15.100:0/2255750612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82f0156750 msgr2=0x7f82f01e6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.447+0000 7f82f7600640 1 --2- 172.21.15.100:0/2255750612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82f0156750 0x7f82f01e6a20 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f82e4003f20 tx=0x7f82e40016f0 comp rx=0 tx=0).stop 2024-09-17T04:32:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.447+0000 7f82f7600640 1 -- 172.21.15.100:0/2255750612 shutdown_connections 2024-09-17T04:32:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.447+0000 7f82f7600640 1 --2- 172.21.15.100:0/2255750612 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f82d0079250 0x7f82d007b760 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.447+0000 7f82f7600640 1 --2- 172.21.15.100:0/2255750612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82f00c5110 0x7f82f00ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.447+0000 7f82f7600640 1 --2- 172.21.15.100:0/2255750612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82f0072f20 0x7f82f00c4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.447+0000 7f82f7600640 1 --2- 172.21.15.100:0/2255750612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82f0156750 0x7f82f01e6a20 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.447+0000 7f82f7600640 1 -- 172.21.15.100:0/2255750612 >> 172.21.15.100:0/2255750612 conn(0x7f82f006d3d0 msgr2=0x7f82f015d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.447+0000 7f82f7600640 1 -- 172.21.15.100:0/2255750612 shutdown_connections 2024-09-17T04:32:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.447+0000 7f82f7600640 1 -- 172.21.15.100:0/2255750612 wait complete. 2024-09-17T04:32:25.469 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol' 2024-09-17T04:32:25.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.689+0000 7fe867e00640 1 -- 172.21.15.100:0/4072095761 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8600aebc0 msgr2=0x7fe860152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:25.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.689+0000 7fe867e00640 1 --2- 172.21.15.100:0/4072095761 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8600aebc0 0x7fe860152920 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe854005fd0 tx=0x7fe85402a540 comp rx=0 tx=0).stop 2024-09-17T04:32:25.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.689+0000 7fe867e00640 1 -- 172.21.15.100:0/4072095761 shutdown_connections 2024-09-17T04:32:25.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.689+0000 7fe867e00640 1 --2- 172.21.15.100:0/4072095761 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe86015fae0 0x7fe8600b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:25.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.689+0000 7fe867e00640 1 --2- 172.21.15.100:0/4072095761 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8600aebc0 0x7fe860152920 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:25.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.689+0000 7fe867e00640 1 -- 172.21.15.100:0/4072095761 >> 172.21.15.100:0/4072095761 conn(0x7fe86006d730 msgr2=0x7fe8600b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:25.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.690+0000 7fe867e00640 1 -- 172.21.15.100:0/4072095761 shutdown_connections 2024-09-17T04:32:25.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.690+0000 7fe867e00640 1 -- 172.21.15.100:0/4072095761 wait complete. 2024-09-17T04:32:25.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.692+0000 7fe867e00640 1 Processor -- start 2024-09-17T04:32:25.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.692+0000 7fe867e00640 1 -- start start 2024-09-17T04:32:25.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.693+0000 7fe867e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8600aebc0 0x7fe8601f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:25.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.693+0000 7fe867e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8600b2750 0x7fe860149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:25.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.693+0000 7fe867e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8601584d0 0x7fe8601f8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:25.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.693+0000 7fe867e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe8600b4130 con 0x7fe8600aebc0 2024-09-17T04:32:25.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.693+0000 7fe867e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe8600b3fb0 con 0x7fe8601584d0 2024-09-17T04:32:25.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.693+0000 7fe867e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe8600b42b0 con 0x7fe8600b2750 2024-09-17T04:32:25.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.693+0000 7fe865400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8600aebc0 0x7fe8601f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:25.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.693+0000 7fe865400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8600aebc0 0x7fe8601f64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38276/0 (socket says 172.21.15.100:38276) 2024-09-17T04:32:25.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.693+0000 7fe865400640 1 -- 172.21.15.100:0/2502579065 learned_addr learned my addr 172.21.15.100:0/2502579065 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:25.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.694+0000 7fe865e00640 1 --2- 172.21.15.100:0/2502579065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8601584d0 0x7fe8601f8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:25.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.694+0000 7fe864a00640 1 --2- 172.21.15.100:0/2502579065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8600b2750 0x7fe860149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:25.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.694+0000 7fe865400640 1 -- 172.21.15.100:0/2502579065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8600b2750 msgr2=0x7fe860149890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:25.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.694+0000 7fe865400640 1 --2- 172.21.15.100:0/2502579065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8600b2750 0x7fe860149890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:25.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.694+0000 7fe865400640 1 -- 172.21.15.100:0/2502579065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8601584d0 msgr2=0x7fe8601f8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:25.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.694+0000 7fe865400640 1 --2- 172.21.15.100:0/2502579065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8601584d0 0x7fe8601f8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:25.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.694+0000 7fe865400640 1 -- 172.21.15.100:0/2502579065 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe860149dd0 con 0x7fe8600aebc0 2024-09-17T04:32:25.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.694+0000 7fe865400640 1 --2- 172.21.15.100:0/2502579065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8600aebc0 0x7fe8601f64d0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fe854034040 tx=0x7fe8540038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:25.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.695+0000 7fe85d600640 1 -- 172.21.15.100:0/2502579065 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85403e030 con 0x7fe8600aebc0 2024-09-17T04:32:25.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.695+0000 7fe85d600640 1 -- 172.21.15.100:0/2502579065 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe854003d90 con 0x7fe8600aebc0 2024-09-17T04:32:25.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.695+0000 7fe85d600640 1 -- 172.21.15.100:0/2502579065 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe854035040 con 0x7fe8600aebc0 2024-09-17T04:32:25.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.695+0000 7fe867e00640 1 -- 172.21.15.100:0/2502579065 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe860158fd0 con 0x7fe8600aebc0 2024-09-17T04:32:25.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.695+0000 7fe867e00640 1 -- 172.21.15.100:0/2502579065 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8601f5cd0 con 0x7fe8600aebc0 2024-09-17T04:32:25.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.696+0000 7fe867e00640 1 -- 172.21.15.100:0/2502579065 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8600b4760 con 0x7fe8600aebc0 2024-09-17T04:32:25.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.697+0000 7fe85d600640 1 -- 172.21.15.100:0/2502579065 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7fe854041030 con 0x7fe8600aebc0 2024-09-17T04:32:25.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.699+0000 7fe85d600640 1 --2- 172.21.15.100:0/2502579065 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe840079250 0x7fe84007b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:25.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.699+0000 7fe85d600640 1 -- 172.21.15.100:0/2502579065 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4999+0+0 (secure 0 0 0) 0x7fe8540bbe10 con 0x7fe8600aebc0 2024-09-17T04:32:25.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.699+0000 7fe864a00640 1 --2- 172.21.15.100:0/2502579065 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe840079250 0x7fe84007b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:25.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.700+0000 7fe864a00640 1 --2- 172.21.15.100:0/2502579065 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe840079250 0x7fe84007b760 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe86015d180 tx=0x7fe84c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:25.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.703+0000 7fe85d600640 1 -- 172.21.15.100:0/2502579065 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe854083420 con 0x7fe8600aebc0 2024-09-17T04:32:25.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.956+0000 7fe867e00640 1 -- 172.21.15.100:0/2502579065 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]} v 0) -- 0x7fe86015dd30 con 0x7fe8600aebc0 2024-09-17T04:32:25.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.994+0000 7fe85d600640 1 -- 172.21.15.100:0/2502579065 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]}]=0 v465) ==== 151+0+0 (secure 0 0 0) 0x7fe854087e50 con 0x7fe8600aebc0 2024-09-17T04:32:25.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.996+0000 7fe867e00640 1 -- 172.21.15.100:0/2502579065 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe840079250 msgr2=0x7fe84007b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:25.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.996+0000 7fe867e00640 1 --2- 172.21.15.100:0/2502579065 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe840079250 0x7fe84007b760 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe86015d180 tx=0x7fe84c009040 comp rx=0 tx=0).stop 2024-09-17T04:32:25.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.996+0000 7fe867e00640 1 -- 172.21.15.100:0/2502579065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8600aebc0 msgr2=0x7fe8601f64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:25.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.996+0000 7fe867e00640 1 --2- 172.21.15.100:0/2502579065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8600aebc0 0x7fe8601f64d0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fe854034040 tx=0x7fe8540038a0 comp rx=0 tx=0).stop 2024-09-17T04:32:25.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.996+0000 7fe867e00640 1 -- 172.21.15.100:0/2502579065 shutdown_connections 2024-09-17T04:32:25.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.996+0000 7fe867e00640 1 --2- 172.21.15.100:0/2502579065 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe840079250 0x7fe84007b760 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:25.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.996+0000 7fe867e00640 1 --2- 172.21.15.100:0/2502579065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8601584d0 0x7fe8601f8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:25.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.996+0000 7fe867e00640 1 --2- 172.21.15.100:0/2502579065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8600b2750 0x7fe860149890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:25.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.996+0000 7fe867e00640 1 --2- 172.21.15.100:0/2502579065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8600aebc0 0x7fe8601f64d0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:25.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.996+0000 7fe867e00640 1 -- 172.21.15.100:0/2502579065 >> 172.21.15.100:0/2502579065 conn(0x7fe86006d730 msgr2=0x7fe86015f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:25.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.996+0000 7fe867e00640 1 -- 172.21.15.100:0/2502579065 shutdown_connections 2024-09-17T04:32:25.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:25.997+0000 7fe867e00640 1 -- 172.21.15.100:0/2502579065 wait complete. 2024-09-17T04:32:26.019 INFO:tasks.cephfs_test_runner:test_subvolume_ops_on_nonexistent_vol (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-09-17T04:32:26.019 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_upgrade_legacy_to_v1 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- rather than going through a full upgrade cycle, 2024-09-17T04:32:26.019 INFO:tasks.cephfs_test_runner:test_subvolume_upgrade_legacy_to_v1 (tasks.cephfs.test_volumes.TestMisc) 2024-09-17T04:32:26.019 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:32:26.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.239+0000 7f6aae800640 1 -- 172.21.15.100:0/2354951129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a8c003a50 msgr2=0x7f6a8c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:26.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.239+0000 7f6aae800640 1 --2- 172.21.15.100:0/2354951129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a8c003a50 0x7f6a8c003ec0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800c310 tx=0x7f6a8c005830 comp rx=0 tx=0).stop 2024-09-17T04:32:26.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.239+0000 7f6aae800640 1 -- 172.21.15.100:0/2354951129 shutdown_connections 2024-09-17T04:32:26.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.239+0000 7f6aae800640 1 --2- 172.21.15.100:0/2354951129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a8c003a50 0x7f6a8c003ec0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:26.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.239+0000 7f6aae800640 1 -- 172.21.15.100:0/2354951129 >> 172.21.15.100:0/2354951129 conn(0x7f6aa806d730 msgr2=0x7f6aa80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:26.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.239+0000 7f6aae800640 1 -- 172.21.15.100:0/2354951129 shutdown_connections 2024-09-17T04:32:26.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.239+0000 7f6aae800640 1 -- 172.21.15.100:0/2354951129 wait complete. 2024-09-17T04:32:26.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.242+0000 7f6aae800640 1 Processor -- start 2024-09-17T04:32:26.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.242+0000 7f6aae800640 1 -- start start 2024-09-17T04:32:26.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.243+0000 7f6aae800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a8c003a50 0x7f6aa81f87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:26.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.243+0000 7f6aae800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa8150b10 0x7f6aa8150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:26.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.243+0000 7f6aae800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6aa81f7c00 0x7f6aa81f8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:26.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.243+0000 7f6aae800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa8073500 con 0x7f6aa8150b10 2024-09-17T04:32:26.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.243+0000 7f6aae800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa806f360 con 0x7f6a8c003a50 2024-09-17T04:32:26.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.243+0000 7f6aae800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6aa8073680 con 0x7f6aa81f7c00 2024-09-17T04:32:26.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.243+0000 7f6aa6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa8150b10 0x7f6aa8150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:26.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.243+0000 7f6aa6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa8150b10 0x7f6aa8150f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38296/0 (socket says 172.21.15.100:38296) 2024-09-17T04:32:26.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.243+0000 7f6aa6a00640 1 -- 172.21.15.100:0/2679034015 learned_addr learned my addr 172.21.15.100:0/2679034015 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:26.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.244+0000 7f6aa6a00640 1 -- 172.21.15.100:0/2679034015 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6aa81f7c00 msgr2=0x7f6aa81f8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:26.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.244+0000 7f6aa7400640 1 --2- 172.21.15.100:0/2679034015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a8c003a50 0x7f6aa81f87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:26.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.244+0000 7f6aa6a00640 1 --2- 172.21.15.100:0/2679034015 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6aa81f7c00 0x7f6aa81f8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:26.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.244+0000 7f6aa6a00640 1 -- 172.21.15.100:0/2679034015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a8c003a50 msgr2=0x7f6aa81f87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:26.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.244+0000 7f6aa6a00640 1 --2- 172.21.15.100:0/2679034015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a8c003a50 0x7f6aa81f87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:26.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.244+0000 7f6aa6a00640 1 -- 172.21.15.100:0/2679034015 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa81e8590 con 0x7f6aa8150b10 2024-09-17T04:32:26.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.244+0000 7f6aa7400640 1 --2- 172.21.15.100:0/2679034015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a8c003a50 0x7f6aa81f87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:32:26.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.244+0000 7f6aa6a00640 1 --2- 172.21.15.100:0/2679034015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa8150b10 0x7f6aa8150f20 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f6a94011a40 tx=0x7f6a94011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:26.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.244+0000 7f6a9f400640 1 -- 172.21.15.100:0/2679034015 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a94004460 con 0x7f6aa8150b10 2024-09-17T04:32:26.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.244+0000 7f6a9f400640 1 -- 172.21.15.100:0/2679034015 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a9400f9a0 con 0x7f6aa8150b10 2024-09-17T04:32:26.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.244+0000 7f6aae800640 1 -- 172.21.15.100:0/2679034015 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa81f8550 con 0x7f6aa8150b10 2024-09-17T04:32:26.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.244+0000 7f6a9f400640 1 -- 172.21.15.100:0/2679034015 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9400fdb0 con 0x7f6aa8150b10 2024-09-17T04:32:26.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.245+0000 7f6aae800640 1 -- 172.21.15.100:0/2679034015 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa81f9340 con 0x7f6aa8150b10 2024-09-17T04:32:26.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.246+0000 7f6aae800640 1 -- 172.21.15.100:0/2679034015 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aa8073b70 con 0x7f6aa8150b10 2024-09-17T04:32:26.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.247+0000 7f6a9f400640 1 -- 172.21.15.100:0/2679034015 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f6a94005020 con 0x7f6aa8150b10 2024-09-17T04:32:26.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.248+0000 7f6a9f400640 1 --2- 172.21.15.100:0/2679034015 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6a88079250 0x7f6a8807b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:26.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.248+0000 7f6a9f400640 1 -- 172.21.15.100:0/2679034015 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4999+0+0 (secure 0 0 0) 0x7f6a9409c550 con 0x7f6aa8150b10 2024-09-17T04:32:26.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.248+0000 7f6aa7400640 1 --2- 172.21.15.100:0/2679034015 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6a88079250 0x7f6a8807b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:26.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.249+0000 7f6aa7400640 1 --2- 172.21.15.100:0/2679034015 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6a88079250 0x7f6a8807b760 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6a98033000 tx=0x7f6aa806ffa0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:26.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.252+0000 7f6a9f400640 1 -- 172.21.15.100:0/2679034015 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6a94064170 con 0x7f6aa8150b10 2024-09-17T04:32:26.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.436+0000 7f6aae800640 1 -- 172.21.15.100:0/2679034015 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6aa81e5510 con 0x7f6aa8150b10 2024-09-17T04:32:26.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.437+0000 7f6a9f400640 1 -- 172.21.15.100:0/2679034015 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) ==== 74+0+13181 (secure 0 0 0) 0x7f6a94063a70 con 0x7f6aa8150b10 2024-09-17T04:32:26.437 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:32:26.437 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":137,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:32:24.984003+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6835/3047460644":"2024-09-18T04:32:21.943001+0000","172.21.15.100:6833/3047460644":"2024-09-18T04:32:21.943001+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:32:26.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.439+0000 7f6aae800640 1 -- 172.21.15.100:0/2679034015 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6a88079250 msgr2=0x7f6a8807b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:26.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.439+0000 7f6aae800640 1 --2- 172.21.15.100:0/2679034015 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6a88079250 0x7f6a8807b760 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6a98033000 tx=0x7f6aa806ffa0 comp rx=0 tx=0).stop 2024-09-17T04:32:26.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.439+0000 7f6aae800640 1 -- 172.21.15.100:0/2679034015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa8150b10 msgr2=0x7f6aa8150f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:26.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.439+0000 7f6aae800640 1 --2- 172.21.15.100:0/2679034015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa8150b10 0x7f6aa8150f20 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f6a94011a40 tx=0x7f6a94011f00 comp rx=0 tx=0).stop 2024-09-17T04:32:26.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.439+0000 7f6aae800640 1 -- 172.21.15.100:0/2679034015 shutdown_connections 2024-09-17T04:32:26.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.440+0000 7f6aae800640 1 --2- 172.21.15.100:0/2679034015 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6a88079250 0x7f6a8807b760 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:26.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.440+0000 7f6aae800640 1 --2- 172.21.15.100:0/2679034015 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6aa81f7c00 0x7f6aa81f8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:26.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.440+0000 7f6aae800640 1 --2- 172.21.15.100:0/2679034015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa8150b10 0x7f6aa8150f20 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:26.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.440+0000 7f6aae800640 1 --2- 172.21.15.100:0/2679034015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a8c003a50 0x7f6aa81f87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:26.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.440+0000 7f6aae800640 1 -- 172.21.15.100:0/2679034015 >> 172.21.15.100:0/2679034015 conn(0x7f6aa806d730 msgr2=0x7f6aa80b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:26.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.440+0000 7f6aae800640 1 -- 172.21.15.100:0/2679034015 shutdown_connections 2024-09-17T04:32:26.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.440+0000 7f6aae800640 1 -- 172.21.15.100:0/2679034015 wait complete. 2024-09-17T04:32:26.461 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T04:24:03.995355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T04:32:26.462 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T04:32:26.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.685+0000 7f4994800640 1 -- 172.21.15.100:0/3883450204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f498c152060 msgr2=0x7f498c152470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:26.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.685+0000 7f4994800640 1 --2- 172.21.15.100:0/3883450204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f498c152060 0x7f498c152470 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f4988005fd0 tx=0x7f498802aa90 comp rx=0 tx=0).stop 2024-09-17T04:32:26.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.685+0000 7f4994800640 1 -- 172.21.15.100:0/3883450204 shutdown_connections 2024-09-17T04:32:26.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.685+0000 7f4994800640 1 --2- 172.21.15.100:0/3883450204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f498c152060 0x7f498c152470 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:26.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.685+0000 7f4994800640 1 --2- 172.21.15.100:0/3883450204 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f498c072fc0 0x7f498c0733d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:26.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.685+0000 7f4994800640 1 -- 172.21.15.100:0/3883450204 >> 172.21.15.100:0/3883450204 conn(0x7f498c06d710 msgr2=0x7f498c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:26.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.686+0000 7f4994800640 1 -- 172.21.15.100:0/3883450204 shutdown_connections 2024-09-17T04:32:26.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.686+0000 7f4994800640 1 -- 172.21.15.100:0/3883450204 wait complete. 2024-09-17T04:32:26.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.688+0000 7f4994800640 1 Processor -- start 2024-09-17T04:32:26.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.688+0000 7f4994800640 1 -- start start 2024-09-17T04:32:26.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.689+0000 7f4994800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f498c072fc0 0x7f498c1557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:26.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.689+0000 7f4994800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f498c152060 0x7f498c1548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:26.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.689+0000 7f4994800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f498c15f250 0x7f498c14e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:26.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.689+0000 7f4994800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f498c0aec60 con 0x7f498c072fc0 2024-09-17T04:32:26.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.689+0000 7f4994800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f498c0aeae0 con 0x7f498c152060 2024-09-17T04:32:26.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.689+0000 7f4994800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f498c0aede0 con 0x7f498c15f250 2024-09-17T04:32:26.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.689+0000 7f4991e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f498c072fc0 0x7f498c1557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:26.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.690+0000 7f4991e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f498c072fc0 0x7f498c1557e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38308/0 (socket says 172.21.15.100:38308) 2024-09-17T04:32:26.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.690+0000 7f4991e00640 1 -- 172.21.15.100:0/3338314123 learned_addr learned my addr 172.21.15.100:0/3338314123 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:26.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.690+0000 7f4991400640 1 --2- 172.21.15.100:0/3338314123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f498c152060 0x7f498c1548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:26.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.690+0000 7f4992800640 1 --2- 172.21.15.100:0/3338314123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f498c15f250 0x7f498c14e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:26.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.690+0000 7f4991e00640 1 -- 172.21.15.100:0/3338314123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f498c15f250 msgr2=0x7f498c14e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:26.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.690+0000 7f4991e00640 1 --2- 172.21.15.100:0/3338314123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f498c15f250 0x7f498c14e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:26.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.690+0000 7f4991e00640 1 -- 172.21.15.100:0/3338314123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f498c152060 msgr2=0x7f498c1548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:26.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.690+0000 7f4991e00640 1 --2- 172.21.15.100:0/3338314123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f498c152060 0x7f498c1548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:26.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.690+0000 7f4991e00640 1 -- 172.21.15.100:0/3338314123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498c158fd0 con 0x7f498c072fc0 2024-09-17T04:32:26.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.690+0000 7f4991e00640 1 --2- 172.21.15.100:0/3338314123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f498c072fc0 0x7f498c1557e0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f498000c3a0 tx=0x7f498000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:26.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.691+0000 7f4986000640 1 -- 172.21.15.100:0/3338314123 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4980017070 con 0x7f498c072fc0 2024-09-17T04:32:26.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.691+0000 7f4986000640 1 -- 172.21.15.100:0/3338314123 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49800053c0 con 0x7f498c072fc0 2024-09-17T04:32:26.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.691+0000 7f4994800640 1 -- 172.21.15.100:0/3338314123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f498c153b20 con 0x7f498c072fc0 2024-09-17T04:32:26.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.691+0000 7f4986000640 1 -- 172.21.15.100:0/3338314123 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4980003850 con 0x7f498c072fc0 2024-09-17T04:32:26.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.691+0000 7f4994800640 1 -- 172.21.15.100:0/3338314123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f498c1540e0 con 0x7f498c072fc0 2024-09-17T04:32:26.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.692+0000 7f4994800640 1 -- 172.21.15.100:0/3338314123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f498c0733d0 con 0x7f498c072fc0 2024-09-17T04:32:26.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.693+0000 7f4986000640 1 -- 172.21.15.100:0/3338314123 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f498001a070 con 0x7f498c072fc0 2024-09-17T04:32:26.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.694+0000 7f4986000640 1 --2- 172.21.15.100:0/3338314123 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4970079250 0x7f497007b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:26.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.694+0000 7f4986000640 1 -- 172.21.15.100:0/3338314123 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4999+0+0 (secure 0 0 0) 0x7f498009abd0 con 0x7f498c072fc0 2024-09-17T04:32:26.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.694+0000 7f4991400640 1 --2- 172.21.15.100:0/3338314123 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4970079250 0x7f497007b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:26.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.695+0000 7f4991400640 1 --2- 172.21.15.100:0/3338314123 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4970079250 0x7f497007b760 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f498c15b8a0 tx=0x7f497c00b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:26.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.698+0000 7f4986000640 1 -- 172.21.15.100:0/3338314123 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4980062090 con 0x7f498c072fc0 2024-09-17T04:32:26.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.884+0000 7f4994800640 1 -- 172.21.15.100:0/3338314123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f498c1eb2a0 con 0x7f498c072fc0 2024-09-17T04:32:26.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.884+0000 7f4986000640 1 -- 172.21.15.100:0/3338314123 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v137) ==== 93+0+10 (secure 0 0 0) 0x7f4980066ac0 con 0x7f498c072fc0 2024-09-17T04:32:26.885 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-09-17T04:32:26.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.887+0000 7f4994800640 1 -- 172.21.15.100:0/3338314123 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4970079250 msgr2=0x7f497007b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:26.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.887+0000 7f4994800640 1 --2- 172.21.15.100:0/3338314123 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4970079250 0x7f497007b760 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f498c15b8a0 tx=0x7f497c00b2e0 comp rx=0 tx=0).stop 2024-09-17T04:32:26.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.887+0000 7f4994800640 1 -- 172.21.15.100:0/3338314123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f498c072fc0 msgr2=0x7f498c1557e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:26.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.887+0000 7f4994800640 1 --2- 172.21.15.100:0/3338314123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f498c072fc0 0x7f498c1557e0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f498000c3a0 tx=0x7f498000c860 comp rx=0 tx=0).stop 2024-09-17T04:32:26.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.887+0000 7f4994800640 1 -- 172.21.15.100:0/3338314123 shutdown_connections 2024-09-17T04:32:26.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.887+0000 7f4994800640 1 --2- 172.21.15.100:0/3338314123 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4970079250 0x7f497007b760 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:26.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.887+0000 7f4994800640 1 --2- 172.21.15.100:0/3338314123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f498c15f250 0x7f498c14e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:26.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.887+0000 7f4994800640 1 --2- 172.21.15.100:0/3338314123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f498c152060 0x7f498c1548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:26.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.887+0000 7f4994800640 1 --2- 172.21.15.100:0/3338314123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f498c072fc0 0x7f498c1557e0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:26.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.887+0000 7f4994800640 1 -- 172.21.15.100:0/3338314123 >> 172.21.15.100:0/3338314123 conn(0x7f498c06d710 msgr2=0x7f498c151830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:26.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.887+0000 7f4994800640 1 -- 172.21.15.100:0/3338314123 shutdown_connections 2024-09-17T04:32:26.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:26.887+0000 7f4994800640 1 -- 172.21.15.100:0/3338314123 wait complete. 2024-09-17T04:32:26.911 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1' 2024-09-17T04:32:27.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.133+0000 7ff0fbe00640 1 -- 172.21.15.100:0/1028408129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0fc15a0f0 msgr2=0x7ff0fc15d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:27.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.133+0000 7ff0fbe00640 1 --2- 172.21.15.100:0/1028408129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0fc15a0f0 0x7ff0fc15d670 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4009cd0 tx=0x7ff0f402a9c0 comp rx=0 tx=0).stop 2024-09-17T04:32:27.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.134+0000 7ff0fbe00640 1 -- 172.21.15.100:0/1028408129 shutdown_connections 2024-09-17T04:32:27.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.134+0000 7ff0fbe00640 1 --2- 172.21.15.100:0/1028408129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0fc15a0f0 0x7ff0fc15d670 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:27.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.134+0000 7ff0fbe00640 1 --2- 172.21.15.100:0/1028408129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0fc0af170 0x7ff0fc0b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:27.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.134+0000 7ff0fbe00640 1 -- 172.21.15.100:0/1028408129 >> 172.21.15.100:0/1028408129 conn(0x7ff0fc0b1e40 msgr2=0x7ff0fc0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:27.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.134+0000 7ff0fbe00640 1 -- 172.21.15.100:0/1028408129 shutdown_connections 2024-09-17T04:32:27.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.134+0000 7ff0fbe00640 1 -- 172.21.15.100:0/1028408129 wait complete. 2024-09-17T04:32:27.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.136+0000 7ff0fbe00640 1 Processor -- start 2024-09-17T04:32:27.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.136+0000 7ff0fbe00640 1 -- start start 2024-09-17T04:32:27.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.138+0000 7ff0fbe00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0fc0ad9c0 0x7ff0fc155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:27.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.138+0000 7ff0fbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0fc0af170 0x7ff0fc1e9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:27.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.138+0000 7ff0fbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0fc15a0f0 0x7ff0fc1e9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:27.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.138+0000 7ff0fbe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc06f410 con 0x7ff0fc15a0f0 2024-09-17T04:32:27.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.138+0000 7ff0fbe00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc06f290 con 0x7ff0fc0af170 2024-09-17T04:32:27.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.138+0000 7ff0fbe00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff0fc06f590 con 0x7ff0fc0ad9c0 2024-09-17T04:32:27.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.138+0000 7ff0fb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0fc15a0f0 0x7ff0fc1e9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:27.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.138+0000 7ff0fb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0fc15a0f0 0x7ff0fc1e9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38324/0 (socket says 172.21.15.100:38324) 2024-09-17T04:32:27.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.138+0000 7ff0fb400640 1 -- 172.21.15.100:0/2324783730 learned_addr learned my addr 172.21.15.100:0/2324783730 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:27.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.138+0000 7ff0faa00640 1 --2- 172.21.15.100:0/2324783730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0fc0ad9c0 0x7ff0fc155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:27.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.138+0000 7ff0f2000640 1 --2- 172.21.15.100:0/2324783730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0fc0af170 0x7ff0fc1e9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:27.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.138+0000 7ff0fb400640 1 -- 172.21.15.100:0/2324783730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0fc0ad9c0 msgr2=0x7ff0fc155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:27.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.138+0000 7ff0fb400640 1 --2- 172.21.15.100:0/2324783730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0fc0ad9c0 0x7ff0fc155110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:27.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.138+0000 7ff0fb400640 1 -- 172.21.15.100:0/2324783730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0fc0af170 msgr2=0x7ff0fc1e9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:27.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.138+0000 7ff0fb400640 1 --2- 172.21.15.100:0/2324783730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0fc0af170 0x7ff0fc1e9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:27.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.139+0000 7ff0fb400640 1 -- 172.21.15.100:0/2324783730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0fc09be90 con 0x7ff0fc15a0f0 2024-09-17T04:32:27.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.139+0000 7ff0f2000640 1 --2- 172.21.15.100:0/2324783730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0fc0af170 0x7ff0fc1e9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:27.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.139+0000 7ff0fb400640 1 --2- 172.21.15.100:0/2324783730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0fc15a0f0 0x7ff0fc1e9570 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4006010 tx=0x7ff0f4003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:27.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.139+0000 7ff0f3400640 1 -- 172.21.15.100:0/2324783730 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4034040 con 0x7ff0fc15a0f0 2024-09-17T04:32:27.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.139+0000 7ff0f3400640 1 -- 172.21.15.100:0/2324783730 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0f40045c0 con 0x7ff0fc15a0f0 2024-09-17T04:32:27.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.139+0000 7ff0fbe00640 1 -- 172.21.15.100:0/2324783730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0fc09bbb0 con 0x7ff0fc15a0f0 2024-09-17T04:32:27.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.139+0000 7ff0f3400640 1 -- 172.21.15.100:0/2324783730 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4045480 con 0x7ff0fc15a0f0 2024-09-17T04:32:27.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.139+0000 7ff0fbe00640 1 -- 172.21.15.100:0/2324783730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0fc09d880 con 0x7ff0fc15a0f0 2024-09-17T04:32:27.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.141+0000 7ff0fbe00640 1 -- 172.21.15.100:0/2324783730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0fc070fe0 con 0x7ff0fc15a0f0 2024-09-17T04:32:27.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.142+0000 7ff0f3400640 1 -- 172.21.15.100:0/2324783730 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7ff0f4037020 con 0x7ff0fc15a0f0 2024-09-17T04:32:27.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.143+0000 7ff0f3400640 1 --2- 172.21.15.100:0/2324783730 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff0d8079250 0x7ff0d807b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:27.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.143+0000 7ff0f3400640 1 -- 172.21.15.100:0/2324783730 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4999+0+0 (secure 0 0 0) 0x7ff0f40bcec0 con 0x7ff0fc15a0f0 2024-09-17T04:32:27.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.143+0000 7ff0faa00640 1 --2- 172.21.15.100:0/2324783730 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff0d8079250 0x7ff0d807b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:27.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.144+0000 7ff0faa00640 1 --2- 172.21.15.100:0/2324783730 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff0d8079250 0x7ff0d807b760 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec006fd0 tx=0x7ff0ec008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:27.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.147+0000 7ff0f3400640 1 -- 172.21.15.100:0/2324783730 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff0fc070fe0 con 0x7ff0fc15a0f0 2024-09-17T04:32:27.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:27.400+0000 7ff0fbe00640 1 -- 172.21.15.100:0/2324783730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]} v 0) -- 0x7ff0fc06fad0 con 0x7ff0fc15a0f0 2024-09-17T04:32:28.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.005+0000 7ff0f3400640 1 -- 172.21.15.100:0/2324783730 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]}]=0 v467) ==== 152+0+0 (secure 0 0 0) 0x7ff0f4084c80 con 0x7ff0fc15a0f0 2024-09-17T04:32:28.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.008+0000 7ff0fbe00640 1 -- 172.21.15.100:0/2324783730 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff0d8079250 msgr2=0x7ff0d807b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:28.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.008+0000 7ff0fbe00640 1 --2- 172.21.15.100:0/2324783730 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff0d8079250 0x7ff0d807b760 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec006fd0 tx=0x7ff0ec008040 comp rx=0 tx=0).stop 2024-09-17T04:32:28.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.008+0000 7ff0fbe00640 1 -- 172.21.15.100:0/2324783730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0fc15a0f0 msgr2=0x7ff0fc1e9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:28.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.008+0000 7ff0fbe00640 1 --2- 172.21.15.100:0/2324783730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0fc15a0f0 0x7ff0fc1e9570 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4006010 tx=0x7ff0f4003f90 comp rx=0 tx=0).stop 2024-09-17T04:32:28.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.008+0000 7ff0fbe00640 1 -- 172.21.15.100:0/2324783730 shutdown_connections 2024-09-17T04:32:28.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.008+0000 7ff0fbe00640 1 --2- 172.21.15.100:0/2324783730 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff0d8079250 0x7ff0d807b760 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:28.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.008+0000 7ff0fbe00640 1 --2- 172.21.15.100:0/2324783730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0fc15a0f0 0x7ff0fc1e9570 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:28.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.008+0000 7ff0fbe00640 1 --2- 172.21.15.100:0/2324783730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0fc0af170 0x7ff0fc1e9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:28.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.008+0000 7ff0fbe00640 1 --2- 172.21.15.100:0/2324783730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0fc0ad9c0 0x7ff0fc155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:28.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.008+0000 7ff0fbe00640 1 -- 172.21.15.100:0/2324783730 >> 172.21.15.100:0/2324783730 conn(0x7ff0fc0b1e40 msgr2=0x7ff0fc0b4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:28.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.008+0000 7ff0fbe00640 1 -- 172.21.15.100:0/2324783730 shutdown_connections 2024-09-17T04:32:28.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.009+0000 7ff0fbe00640 1 -- 172.21.15.100:0/2324783730 wait complete. 2024-09-17T04:32:28.032 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T04:32:28.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.251+0000 7f39d8000640 1 -- 172.21.15.100:0/3528467224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39d00ad3c0 msgr2=0x7f39d00ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:28.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.251+0000 7f39d8000640 1 --2- 172.21.15.100:0/3528467224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39d00ad3c0 0x7f39d00ad7d0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f39bc005fd0 tx=0x7f39bc02b910 comp rx=0 tx=0).stop 2024-09-17T04:32:28.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.251+0000 7f39d8000640 1 -- 172.21.15.100:0/3528467224 shutdown_connections 2024-09-17T04:32:28.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.251+0000 7f39d8000640 1 --2- 172.21.15.100:0/3528467224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39d00ad3c0 0x7f39d00ad7d0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:28.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.251+0000 7f39d8000640 1 -- 172.21.15.100:0/3528467224 >> 172.21.15.100:0/3528467224 conn(0x7f39d006d730 msgr2=0x7f39d00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:28.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.251+0000 7f39d8000640 1 -- 172.21.15.100:0/3528467224 shutdown_connections 2024-09-17T04:32:28.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.252+0000 7f39d8000640 1 -- 172.21.15.100:0/3528467224 wait complete. 2024-09-17T04:32:28.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.254+0000 7f39d8000640 1 Processor -- start 2024-09-17T04:32:28.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.254+0000 7f39d8000640 1 -- start start 2024-09-17T04:32:28.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.255+0000 7f39d8000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39d00704b0 0x7f39d0148c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:28.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.255+0000 7f39d8000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39d00ad3c0 0x7f39d0162a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:28.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.255+0000 7f39d8000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39d00add10 0x7f39d0162f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:28.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.255+0000 7f39d8000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f39d00712f0 con 0x7f39d00add10 2024-09-17T04:32:28.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.255+0000 7f39d8000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f39d0071170 con 0x7f39d00ad3c0 2024-09-17T04:32:28.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.255+0000 7f39d8000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f39d0165e00 con 0x7f39d00704b0 2024-09-17T04:32:28.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.256+0000 7f39d6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39d00add10 0x7f39d0162f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:28.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.256+0000 7f39d6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39d00add10 0x7f39d0162f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38346/0 (socket says 172.21.15.100:38346) 2024-09-17T04:32:28.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.256+0000 7f39d6000640 1 -- 172.21.15.100:0/2815468417 learned_addr learned my addr 172.21.15.100:0/2815468417 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:28.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.256+0000 7f39d4c00640 1 --2- 172.21.15.100:0/2815468417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39d00ad3c0 0x7f39d0162a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:28.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.256+0000 7f39d5600640 1 --2- 172.21.15.100:0/2815468417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39d00704b0 0x7f39d0148c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:28.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.256+0000 7f39d6000640 1 -- 172.21.15.100:0/2815468417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39d00704b0 msgr2=0x7f39d0148c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:28.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.256+0000 7f39d6000640 1 --2- 172.21.15.100:0/2815468417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39d00704b0 0x7f39d0148c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:28.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.256+0000 7f39d6000640 1 -- 172.21.15.100:0/2815468417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39d00ad3c0 msgr2=0x7f39d0162a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:28.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.256+0000 7f39d6000640 1 --2- 172.21.15.100:0/2815468417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39d00ad3c0 0x7f39d0162a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:28.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.256+0000 7f39d6000640 1 -- 172.21.15.100:0/2815468417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d01516f0 con 0x7f39d00add10 2024-09-17T04:32:28.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.256+0000 7f39d4c00640 1 --2- 172.21.15.100:0/2815468417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39d00ad3c0 0x7f39d0162a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:28.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.256+0000 7f39d6000640 1 --2- 172.21.15.100:0/2815468417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39d00add10 0x7f39d0162f90 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f39c8002230 tx=0x7f39c800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:28.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.257+0000 7f39cd600640 1 -- 172.21.15.100:0/2815468417 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8003b70 con 0x7f39d00add10 2024-09-17T04:32:28.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.257+0000 7f39cd600640 1 -- 172.21.15.100:0/2815468417 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39c8003d10 con 0x7f39d00add10 2024-09-17T04:32:28.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.257+0000 7f39cd600640 1 -- 172.21.15.100:0/2815468417 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8005330 con 0x7f39d00add10 2024-09-17T04:32:28.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.257+0000 7f39d8000640 1 -- 172.21.15.100:0/2815468417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d01519d0 con 0x7f39d00add10 2024-09-17T04:32:28.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.257+0000 7f39d8000640 1 -- 172.21.15.100:0/2815468417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d0151f90 con 0x7f39d00add10 2024-09-17T04:32:28.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.258+0000 7f39d8000640 1 -- 172.21.15.100:0/2815468417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d00ad7d0 con 0x7f39d00add10 2024-09-17T04:32:28.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.259+0000 7f39cd600640 1 -- 172.21.15.100:0/2815468417 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f39c80056e0 con 0x7f39d00add10 2024-09-17T04:32:28.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.260+0000 7f39cd600640 1 --2- 172.21.15.100:0/2815468417 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f39b0079280 0x7f39b007b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:28.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.260+0000 7f39cd600640 1 -- 172.21.15.100:0/2815468417 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4999+0+0 (secure 0 0 0) 0x7f39c809b680 con 0x7f39d00add10 2024-09-17T04:32:28.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.261+0000 7f39d5600640 1 --2- 172.21.15.100:0/2815468417 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f39b0079280 0x7f39b007b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:28.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.261+0000 7f39d5600640 1 --2- 172.21.15.100:0/2815468417 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f39b0079280 0x7f39b007b790 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f39c40015e0 tx=0x7f39c4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:28.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.265+0000 7f39cd600640 1 -- 172.21.15.100:0/2815468417 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f39c80633a0 con 0x7f39d00add10 2024-09-17T04:32:28.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.447+0000 7f39d8000640 1 -- 172.21.15.100:0/2815468417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f39d015e420 con 0x7f39d00add10 2024-09-17T04:32:28.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.448+0000 7f39cd600640 1 -- 172.21.15.100:0/2815468417 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v47) ==== 59+0+2294 (secure 0 0 0) 0x7f39c8062ca0 con 0x7f39d00add10 2024-09-17T04:32:28.448 INFO:teuthology.orchestra.run.smithi100.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T04:32:28.448 INFO:teuthology.orchestra.run.smithi100.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T04:32:28.448 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced client_mount_timeout 30 2024-09-17T04:32:28.448 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_client 20/20 2024-09-17T04:32:28.448 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_mgr 20/20 2024-09-17T04:32:28.448 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_ms 1/1 2024-09-17T04:32:28.449 INFO:teuthology.orchestra.run.smithi100.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T04:32:28.449 INFO:teuthology.orchestra.run.smithi100.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T04:32:28.449 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3938.598654 2024-09-17T04:32:28.449 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4008.640154 2024-09-17T04:32:28.449 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3706.588546 2024-09-17T04:32:28.449 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 6299.338665 2024-09-17T04:32:28.449 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3633.876699 2024-09-17T04:32:28.449 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3674.821841 2024-09-17T04:32:28.449 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3789.849984 2024-09-17T04:32:28.449 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3467.457298 2024-09-17T04:32:28.449 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_mds 20/20 2024-09-17T04:32:28.449 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T04:32:28.449 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_ms 2/2 2024-09-17T04:32:28.449 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_debug_frag true 2024-09-17T04:32:28.450 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_debug_scatterstat true 2024-09-17T04:32:28.450 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T04:32:28.450 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_verify_scatter true 2024-09-17T04:32:28.450 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T04:32:28.450 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T04:32:28.450 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T04:32:28.450 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced client_mount_timeout 600 2024-09-17T04:32:28.450 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced debug_client 20/20 2024-09-17T04:32:28.450 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced debug_ms 1/1 2024-09-17T04:32:28.450 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T04:32:28.450 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T04:32:28.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.450+0000 7f39d8000640 1 -- 172.21.15.100:0/2815468417 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f39b0079280 msgr2=0x7f39b007b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:28.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.450+0000 7f39d8000640 1 --2- 172.21.15.100:0/2815468417 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f39b0079280 0x7f39b007b790 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f39c40015e0 tx=0x7f39c4009040 comp rx=0 tx=0).stop 2024-09-17T04:32:28.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.450+0000 7f39d8000640 1 -- 172.21.15.100:0/2815468417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39d00add10 msgr2=0x7f39d0162f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:28.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.450+0000 7f39d8000640 1 --2- 172.21.15.100:0/2815468417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39d00add10 0x7f39d0162f90 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f39c8002230 tx=0x7f39c800cb90 comp rx=0 tx=0).stop 2024-09-17T04:32:28.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.450+0000 7f39d8000640 1 -- 172.21.15.100:0/2815468417 shutdown_connections 2024-09-17T04:32:28.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.450+0000 7f39d8000640 1 --2- 172.21.15.100:0/2815468417 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f39b0079280 0x7f39b007b790 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:28.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.450+0000 7f39d8000640 1 --2- 172.21.15.100:0/2815468417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39d00add10 0x7f39d0162f90 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:28.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.450+0000 7f39d8000640 1 --2- 172.21.15.100:0/2815468417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39d00ad3c0 0x7f39d0162a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:28.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.450+0000 7f39d8000640 1 --2- 172.21.15.100:0/2815468417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39d00704b0 0x7f39d0148c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:28.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.450+0000 7f39d8000640 1 -- 172.21.15.100:0/2815468417 >> 172.21.15.100:0/2815468417 conn(0x7f39d006d730 msgr2=0x7f39d01546a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:28.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.450+0000 7f39d8000640 1 -- 172.21.15.100:0/2815468417 shutdown_connections 2024-09-17T04:32:28.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.450+0000 7f39d8000640 1 -- 172.21.15.100:0/2815468417 wait complete. 2024-09-17T04:32:28.473 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T04:32:28.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.695+0000 7f58dce00640 1 -- 172.21.15.100:0/458808977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d8145550 msgr2=0x7f58d806fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:28.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.695+0000 7f58dce00640 1 --2- 172.21.15.100:0/458808977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d8145550 0x7f58d806fb40 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f58c8005fd0 tx=0x7f58c802a670 comp rx=0 tx=0).stop 2024-09-17T04:32:28.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.695+0000 7f58dce00640 1 -- 172.21.15.100:0/458808977 shutdown_connections 2024-09-17T04:32:28.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.695+0000 7f58dce00640 1 --2- 172.21.15.100:0/458808977 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58d8145a40 0x7f58d8073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:28.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.695+0000 7f58dce00640 1 --2- 172.21.15.100:0/458808977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d8145550 0x7f58d806fb40 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:28.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.695+0000 7f58dce00640 1 -- 172.21.15.100:0/458808977 >> 172.21.15.100:0/458808977 conn(0x7f58d806d1c0 msgr2=0x7f58d806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:28.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.695+0000 7f58dce00640 1 -- 172.21.15.100:0/458808977 shutdown_connections 2024-09-17T04:32:28.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.695+0000 7f58dce00640 1 -- 172.21.15.100:0/458808977 wait complete. 2024-09-17T04:32:28.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.697+0000 7f58dce00640 1 Processor -- start 2024-09-17T04:32:28.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.697+0000 7f58dce00640 1 -- start start 2024-09-17T04:32:28.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.699+0000 7f58dce00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58d8073a80 0x7f58d80c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:28.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.699+0000 7f58dce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d8145550 0x7f58d815be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:28.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.699+0000 7f58dce00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58d8145a40 0x7f58d80c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:28.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.699+0000 7f58dce00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f58d8160a30 con 0x7f58d8145550 2024-09-17T04:32:28.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.699+0000 7f58dce00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f58d80ce940 con 0x7f58d8073a80 2024-09-17T04:32:28.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.699+0000 7f58dce00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f58d8160bb0 con 0x7f58d8145a40 2024-09-17T04:32:28.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.699+0000 7f58cfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d8145550 0x7f58d815be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:28.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.699+0000 7f58cfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d8145550 0x7f58d815be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38354/0 (socket says 172.21.15.100:38354) 2024-09-17T04:32:28.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.699+0000 7f58cfe00640 1 -- 172.21.15.100:0/2820900338 learned_addr learned my addr 172.21.15.100:0/2820900338 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:28.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.699+0000 7f58d5e00640 1 --2- 172.21.15.100:0/2820900338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58d8073a80 0x7f58d80c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:28.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.699+0000 7f58d6800640 1 --2- 172.21.15.100:0/2820900338 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58d8145a40 0x7f58d80c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:28.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.699+0000 7f58cfe00640 1 -- 172.21.15.100:0/2820900338 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58d8145a40 msgr2=0x7f58d80c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:28.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.699+0000 7f58cfe00640 1 --2- 172.21.15.100:0/2820900338 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58d8145a40 0x7f58d80c5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:28.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.699+0000 7f58cfe00640 1 -- 172.21.15.100:0/2820900338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58d8073a80 msgr2=0x7f58d80c5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:28.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.699+0000 7f58cfe00640 1 --2- 172.21.15.100:0/2820900338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58d8073a80 0x7f58d80c5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:28.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.699+0000 7f58cfe00640 1 -- 172.21.15.100:0/2820900338 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d80aeba0 con 0x7f58d8145550 2024-09-17T04:32:28.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.699+0000 7f58d5e00640 1 --2- 172.21.15.100:0/2820900338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58d8073a80 0x7f58d80c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:28.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.700+0000 7f58cfe00640 1 --2- 172.21.15.100:0/2820900338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d8145550 0x7f58d815be00 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f58c000d680 tx=0x7f58c000db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:28.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.700+0000 7f58ce000640 1 -- 172.21.15.100:0/2820900338 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c0017070 con 0x7f58d8145550 2024-09-17T04:32:28.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.700+0000 7f58ce000640 1 -- 172.21.15.100:0/2820900338 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58c0005310 con 0x7f58d8145550 2024-09-17T04:32:28.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.700+0000 7f58dce00640 1 -- 172.21.15.100:0/2820900338 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d80aee80 con 0x7f58d8145550 2024-09-17T04:32:28.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.700+0000 7f58ce000640 1 -- 172.21.15.100:0/2820900338 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c0009820 con 0x7f58d8145550 2024-09-17T04:32:28.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.700+0000 7f58dce00640 1 -- 172.21.15.100:0/2820900338 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d80af440 con 0x7f58d8145550 2024-09-17T04:32:28.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.701+0000 7f58dce00640 1 -- 172.21.15.100:0/2820900338 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58d81610f0 con 0x7f58d8145550 2024-09-17T04:32:28.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.702+0000 7f58ce000640 1 -- 172.21.15.100:0/2820900338 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f58c00099c0 con 0x7f58d8145550 2024-09-17T04:32:28.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.703+0000 7f58ce000640 1 --2- 172.21.15.100:0/2820900338 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f58b4079280 0x7f58b407b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:28.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.704+0000 7f58ce000640 1 -- 172.21.15.100:0/2820900338 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4999+0+0 (secure 0 0 0) 0x7f58c009b880 con 0x7f58d8145550 2024-09-17T04:32:28.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.704+0000 7f58d5e00640 1 --2- 172.21.15.100:0/2820900338 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f58b4079280 0x7f58b407b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:28.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.704+0000 7f58d5e00640 1 --2- 172.21.15.100:0/2820900338 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f58b4079280 0x7f58b407b790 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f58c8033000 tx=0x7f58c8004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:28.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.708+0000 7f58ce000640 1 -- 172.21.15.100:0/2820900338 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f58c0063410 con 0x7f58d8145550 2024-09-17T04:32:28.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.893+0000 7f58dce00640 1 -- 172.21.15.100:0/2820900338 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f58d80c5ac0 con 0x7f58d8145550 2024-09-17T04:32:28.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.898+0000 7f58ce000640 1 -- 172.21.15.100:0/2820900338 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v48)=0 v48) ==== 122+0+0 (secure 0 0 0) 0x7f58c0062d10 con 0x7f58d8145550 2024-09-17T04:32:28.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.900+0000 7f58dce00640 1 -- 172.21.15.100:0/2820900338 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f58b4079280 msgr2=0x7f58b407b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:28.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.900+0000 7f58dce00640 1 --2- 172.21.15.100:0/2820900338 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f58b4079280 0x7f58b407b790 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f58c8033000 tx=0x7f58c8004730 comp rx=0 tx=0).stop 2024-09-17T04:32:28.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.900+0000 7f58dce00640 1 -- 172.21.15.100:0/2820900338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d8145550 msgr2=0x7f58d815be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:28.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.901+0000 7f58dce00640 1 --2- 172.21.15.100:0/2820900338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d8145550 0x7f58d815be00 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f58c000d680 tx=0x7f58c000db40 comp rx=0 tx=0).stop 2024-09-17T04:32:28.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.901+0000 7f58dce00640 1 -- 172.21.15.100:0/2820900338 shutdown_connections 2024-09-17T04:32:28.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.901+0000 7f58dce00640 1 --2- 172.21.15.100:0/2820900338 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f58b4079280 0x7f58b407b790 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:28.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.901+0000 7f58dce00640 1 --2- 172.21.15.100:0/2820900338 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58d8145a40 0x7f58d80c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:28.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.901+0000 7f58dce00640 1 --2- 172.21.15.100:0/2820900338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d8145550 0x7f58d815be00 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:28.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.901+0000 7f58dce00640 1 --2- 172.21.15.100:0/2820900338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58d8073a80 0x7f58d80c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:28.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.901+0000 7f58dce00640 1 -- 172.21.15.100:0/2820900338 >> 172.21.15.100:0/2820900338 conn(0x7f58d806d1c0 msgr2=0x7f58d806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:28.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.901+0000 7f58dce00640 1 -- 172.21.15.100:0/2820900338 shutdown_connections 2024-09-17T04:32:28.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:28.901+0000 7f58dce00640 1 -- 172.21.15.100:0/2820900338 wait complete. 2024-09-17T04:32:28.926 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T04:32:28.926 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:32:29.015 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:32:29.117 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T04:32:29.117 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:32:29.117 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:32:29.153 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:32:29.153 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:32:29.153 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:32:29.190 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:32:29.190 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:32:29.190 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:32:29.228 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:32:29.228 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:32:29.228 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:32:29.266 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:32:29.266 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T04:32:29.267 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:32:29.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.487+0000 7f429be00640 1 -- 172.21.15.100:0/2996972431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f429415d760 msgr2=0x7f42941535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:29.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.487+0000 7f429be00640 1 --2- 172.21.15.100:0/2996972431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f429415d760 0x7f42941535c0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f4288005fd0 tx=0x7f428802a540 comp rx=0 tx=0).stop 2024-09-17T04:32:29.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.487+0000 7f429be00640 1 -- 172.21.15.100:0/2996972431 shutdown_connections 2024-09-17T04:32:29.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.487+0000 7f429be00640 1 --2- 172.21.15.100:0/2996972431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42940b3550 0x7f42940b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:29.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.487+0000 7f429be00640 1 --2- 172.21.15.100:0/2996972431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f429415d760 0x7f42941535c0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:29.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.487+0000 7f429be00640 1 -- 172.21.15.100:0/2996972431 >> 172.21.15.100:0/2996972431 conn(0x7f429406d730 msgr2=0x7f42940b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:29.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.487+0000 7f429be00640 1 -- 172.21.15.100:0/2996972431 shutdown_connections 2024-09-17T04:32:29.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.487+0000 7f429be00640 1 -- 172.21.15.100:0/2996972431 wait complete. 2024-09-17T04:32:29.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.490+0000 7f429be00640 1 Processor -- start 2024-09-17T04:32:29.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.490+0000 7f429be00640 1 -- start start 2024-09-17T04:32:29.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.491+0000 7f429be00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42940ad0e0 0x7f42941f5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:29.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.491+0000 7f429be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42940b3550 0x7f42941ef780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:29.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.491+0000 7f429be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f429415d760 0x7f429415b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:29.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.491+0000 7f429be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f429406f240 con 0x7f429415d760 2024-09-17T04:32:29.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.491+0000 7f429be00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f42940b4e30 con 0x7f42940b3550 2024-09-17T04:32:29.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.491+0000 7f429be00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f42940b3ea0 con 0x7f42940ad0e0 2024-09-17T04:32:29.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.492+0000 7f4299e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f429415d760 0x7f429415b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:29.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.492+0000 7f4299e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f429415d760 0x7f429415b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38370/0 (socket says 172.21.15.100:38370) 2024-09-17T04:32:29.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.492+0000 7f4299e00640 1 -- 172.21.15.100:0/2294738800 learned_addr learned my addr 172.21.15.100:0/2294738800 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:29.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.492+0000 7f4299400640 1 --2- 172.21.15.100:0/2294738800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42940ad0e0 0x7f42941f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:29.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.492+0000 7f4298a00640 1 --2- 172.21.15.100:0/2294738800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42940b3550 0x7f42941ef780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:29.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.492+0000 7f4299e00640 1 -- 172.21.15.100:0/2294738800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42940ad0e0 msgr2=0x7f42941f5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:29.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.492+0000 7f4299e00640 1 --2- 172.21.15.100:0/2294738800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42940ad0e0 0x7f42941f5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:29.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.492+0000 7f4299e00640 1 -- 172.21.15.100:0/2294738800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42940b3550 msgr2=0x7f42941ef780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:29.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.492+0000 7f4299e00640 1 --2- 172.21.15.100:0/2294738800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42940b3550 0x7f42941ef780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:29.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.492+0000 7f4299e00640 1 -- 172.21.15.100:0/2294738800 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f429414aef0 con 0x7f429415d760 2024-09-17T04:32:29.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.492+0000 7f4298a00640 1 --2- 172.21.15.100:0/2294738800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42940b3550 0x7f42941ef780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:29.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.492+0000 7f4299e00640 1 --2- 172.21.15.100:0/2294738800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f429415d760 0x7f429415b7f0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f428c00e960 tx=0x7f428c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:29.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.493+0000 7f4291600640 1 -- 172.21.15.100:0/2294738800 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c004d20 con 0x7f429415d760 2024-09-17T04:32:29.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.493+0000 7f4291600640 1 -- 172.21.15.100:0/2294738800 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f428c00cb20 con 0x7f429415d760 2024-09-17T04:32:29.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.493+0000 7f4291600640 1 -- 172.21.15.100:0/2294738800 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c005020 con 0x7f429415d760 2024-09-17T04:32:29.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.493+0000 7f429be00640 1 -- 172.21.15.100:0/2294738800 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42941552b0 con 0x7f429415d760 2024-09-17T04:32:29.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.493+0000 7f429be00640 1 -- 172.21.15.100:0/2294738800 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4294155870 con 0x7f429415d760 2024-09-17T04:32:29.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.494+0000 7f429be00640 1 -- 172.21.15.100:0/2294738800 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42941efcc0 con 0x7f429415d760 2024-09-17T04:32:29.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.495+0000 7f4291600640 1 -- 172.21.15.100:0/2294738800 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f428c0051c0 con 0x7f429415d760 2024-09-17T04:32:29.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.496+0000 7f4291600640 1 --2- 172.21.15.100:0/2294738800 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4274079280 0x7f427407b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:29.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.496+0000 7f4291600640 1 -- 172.21.15.100:0/2294738800 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4999+0+0 (secure 0 0 0) 0x7f428c09c0c0 con 0x7f429415d760 2024-09-17T04:32:29.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.496+0000 7f4299400640 1 --2- 172.21.15.100:0/2294738800 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4274079280 0x7f427407b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:29.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.497+0000 7f4299400640 1 --2- 172.21.15.100:0/2294738800 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4274079280 0x7f427407b790 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4288033000 tx=0x7f4288004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:29.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.500+0000 7f4291600640 1 -- 172.21.15.100:0/2294738800 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f429414aef0 con 0x7f429415d760 2024-09-17T04:32:29.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.720+0000 7f429be00640 1 -- 172.21.15.100:0/2294738800 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42940b42f0 con 0x7f429415d760 2024-09-17T04:32:29.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.721+0000 7f4291600640 1 -- 172.21.15.100:0/2294738800 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+3923 (secure 0 0 0) 0x7f428c016030 con 0x7f429415d760 2024-09-17T04:32:29.722 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:32:29.722 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":94,"btime":"2024-09-17T04:32:22:968997+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6719,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1392663867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1392663867},{"type":"v1","addr":"172.21.15.184:6801","nonce":1392663867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":7516,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3257629487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3257629487},{"type":"v1","addr":"172.21.15.184:6835","nonce":3257629487}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":7962,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/2131565869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2131565869},{"type":"v1","addr":"172.21.15.100:6834","nonce":2131565869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":8340,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1224245630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6833","nonce":1224245630},{"type":"v1","addr":"172.21.15.100:6835","nonce":1224245630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[]} 2024-09-17T04:32:29.722 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 94 2024-09-17T04:32:29.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.724+0000 7f429be00640 1 -- 172.21.15.100:0/2294738800 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4274079280 msgr2=0x7f427407b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:29.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.724+0000 7f429be00640 1 --2- 172.21.15.100:0/2294738800 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4274079280 0x7f427407b790 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4288033000 tx=0x7f4288004730 comp rx=0 tx=0).stop 2024-09-17T04:32:29.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.724+0000 7f429be00640 1 -- 172.21.15.100:0/2294738800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f429415d760 msgr2=0x7f429415b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:29.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.724+0000 7f429be00640 1 --2- 172.21.15.100:0/2294738800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f429415d760 0x7f429415b7f0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f428c00e960 tx=0x7f428c00ee20 comp rx=0 tx=0).stop 2024-09-17T04:32:29.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.724+0000 7f429be00640 1 -- 172.21.15.100:0/2294738800 shutdown_connections 2024-09-17T04:32:29.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.724+0000 7f429be00640 1 --2- 172.21.15.100:0/2294738800 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4274079280 0x7f427407b790 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:29.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.724+0000 7f429be00640 1 --2- 172.21.15.100:0/2294738800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f429415d760 0x7f429415b7f0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:29.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.724+0000 7f429be00640 1 --2- 172.21.15.100:0/2294738800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42940b3550 0x7f42941ef780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:29.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.724+0000 7f429be00640 1 --2- 172.21.15.100:0/2294738800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42940ad0e0 0x7f42941f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:29.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.724+0000 7f429be00640 1 -- 172.21.15.100:0/2294738800 >> 172.21.15.100:0/2294738800 conn(0x7f429406d730 msgr2=0x7f429415eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:29.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.724+0000 7f429be00640 1 -- 172.21.15.100:0/2294738800 shutdown_connections 2024-09-17T04:32:29.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:29.724+0000 7f429be00640 1 -- 172.21.15.100:0/2294738800 wait complete. 2024-09-17T04:32:29.746 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T04:32:29.747 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T04:32:29.747 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T04:32:29.747 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:32:29.747 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T04:32:29.747 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T04:32:29.748 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T04:32:29.748 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:32:29.748 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T04:32:29.748 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T04:32:29.748 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T04:32:29.748 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:32:29.748 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T04:32:29.748 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T04:32:29.749 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T04:32:29.749 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:32:29.773 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T04:32:29.773 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T04:32:29.775 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T04:32:29.775 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T04:32:29.781 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T04:32:29.781 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T04:32:29.782 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T04:32:29.782 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T04:32:29.817 INFO:tasks.ceph.mds.c:Started 2024-09-17T04:32:29.818 DEBUG:teuthology.parallel:result is None 2024-09-17T04:32:29.820 INFO:tasks.ceph.mds.a:Started 2024-09-17T04:32:29.820 DEBUG:teuthology.parallel:result is None 2024-09-17T04:32:29.825 INFO:tasks.ceph.mds.b:Started 2024-09-17T04:32:29.825 DEBUG:teuthology.parallel:result is None 2024-09-17T04:32:29.828 INFO:tasks.ceph.mds.d:Started 2024-09-17T04:32:29.828 DEBUG:teuthology.parallel:result is None 2024-09-17T04:32:29.828 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T04:32:30.012 INFO:tasks.ceph.mds.c.smithi100.stdout:starting mds.c at 2024-09-17T04:32:30.016 INFO:tasks.ceph.mds.a.smithi100.stdout:starting mds.a at 2024-09-17T04:32:30.022 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-09-17T04:32:30.025 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-09-17T04:32:30.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.064+0000 7f4f3b200640 1 -- 172.21.15.100:0/253840928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f1c004d90 msgr2=0x7f4f1c005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:30.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.064+0000 7f4f3b200640 1 --2- 172.21.15.100:0/253840928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f1c004d90 0x7f4f1c005220 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f4f34160160 tx=0x7f4f1c005830 comp rx=0 tx=0).stop 2024-09-17T04:32:30.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.064+0000 7f4f3a800640 1 -- 172.21.15.100:0/253840928 reap_dead start 2024-09-17T04:32:30.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.064+0000 7f4f3b200640 1 -- 172.21.15.100:0/253840928 shutdown_connections 2024-09-17T04:32:30.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.064+0000 7f4f3b200640 1 -- 172.21.15.100:0/253840928 >> 172.21.15.100:0/253840928 conn(0x7f4f3406d1c0 msgr2=0x7f4f3406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:30.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.064+0000 7f4f3b200640 1 -- 172.21.15.100:0/253840928 shutdown_connections 2024-09-17T04:32:30.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.064+0000 7f4f3b200640 1 -- 172.21.15.100:0/253840928 wait complete. 2024-09-17T04:32:30.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.066+0000 7f4f3b200640 1 Processor -- start 2024-09-17T04:32:30.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.066+0000 7f4f3b200640 1 -- start start 2024-09-17T04:32:30.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.068+0000 7f4f3b200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f1c004d90 0x7f4f34141fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:30.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.068+0000 7f4f3b200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f341f9ab0 0x7f4f341f5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:30.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.068+0000 7f4f3b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f340b7510 0x7f4f340b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:30.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.068+0000 7f4f3b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4f340d2610 con 0x7f4f340b7510 2024-09-17T04:32:30.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.068+0000 7f4f3b200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4f340d2490 con 0x7f4f1c004d90 2024-09-17T04:32:30.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.068+0000 7f4f3b200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4f3415d5f0 con 0x7f4f341f9ab0 2024-09-17T04:32:30.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.068+0000 7f4f3a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f340b7510 0x7f4f340b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:30.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.068+0000 7f4f3a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f340b7510 0x7f4f340b7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38402/0 (socket says 172.21.15.100:38402) 2024-09-17T04:32:30.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.068+0000 7f4f3a800640 1 -- 172.21.15.100:0/1452806169 learned_addr learned my addr 172.21.15.100:0/1452806169 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:30.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.068+0000 7f4f39e00640 1 --2- 172.21.15.100:0/1452806169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f1c004d90 0x7f4f34141fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:30.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.068+0000 7f4f39400640 1 --2- 172.21.15.100:0/1452806169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f341f9ab0 0x7f4f341f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:30.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.068+0000 7f4f3a800640 1 -- 172.21.15.100:0/1452806169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f341f9ab0 msgr2=0x7f4f341f5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:30.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.068+0000 7f4f3a800640 1 --2- 172.21.15.100:0/1452806169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f341f9ab0 0x7f4f341f5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:30.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.068+0000 7f4f3a800640 1 -- 172.21.15.100:0/1452806169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f1c004d90 msgr2=0x7f4f34141fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:30.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.068+0000 7f4f3a800640 1 --2- 172.21.15.100:0/1452806169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f1c004d90 0x7f4f34141fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:30.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.068+0000 7f4f3a800640 1 -- 172.21.15.100:0/1452806169 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f340b6d50 con 0x7f4f340b7510 2024-09-17T04:32:30.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.068+0000 7f4f39400640 1 --2- 172.21.15.100:0/1452806169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f341f9ab0 0x7f4f341f5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:30.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.069+0000 7f4f3a800640 1 --2- 172.21.15.100:0/1452806169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f340b7510 0x7f4f340b7920 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f4f3002af50 tx=0x7f4f300062e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:30.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.069+0000 7f4f2e000640 1 -- 172.21.15.100:0/1452806169 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30042070 con 0x7f4f340b7510 2024-09-17T04:32:30.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.069+0000 7f4f2e000640 1 -- 172.21.15.100:0/1452806169 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f30037040 con 0x7f4f340b7510 2024-09-17T04:32:30.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.069+0000 7f4f3b200640 1 -- 172.21.15.100:0/1452806169 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f3414e600 con 0x7f4f340b7510 2024-09-17T04:32:30.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.069+0000 7f4f2e000640 1 -- 172.21.15.100:0/1452806169 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30036070 con 0x7f4f340b7510 2024-09-17T04:32:30.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.069+0000 7f4f3b200640 1 -- 172.21.15.100:0/1452806169 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f340b6ff0 con 0x7f4f340b7510 2024-09-17T04:32:30.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.070+0000 7f4f3b200640 1 -- 172.21.15.100:0/1452806169 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f340d0ce0 con 0x7f4f340b7510 2024-09-17T04:32:30.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.071+0000 7f4f2e000640 1 -- 172.21.15.100:0/1452806169 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f4f30045070 con 0x7f4f340b7510 2024-09-17T04:32:30.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.072+0000 7f4f2e000640 1 --2- 172.21.15.100:0/1452806169 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4f18079280 0x7f4f1807b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:30.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.072+0000 7f4f2e000640 1 -- 172.21.15.100:0/1452806169 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4999+0+0 (secure 0 0 0) 0x7f4f300baf80 con 0x7f4f340b7510 2024-09-17T04:32:30.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.072+0000 7f4f39e00640 1 --2- 172.21.15.100:0/1452806169 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4f18079280 0x7f4f1807b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:30.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.073+0000 7f4f39e00640 1 --2- 172.21.15.100:0/1452806169 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4f18079280 0x7f4f1807b790 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009c50 tx=0x7f4f28005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:30.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.077+0000 7f4f2e000640 1 -- 172.21.15.100:0/1452806169 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4f340d2bd0 con 0x7f4f340b7510 2024-09-17T04:32:30.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:30.272+0000 7f4f3b200640 1 -- 172.21.15.100:0/1452806169 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f4f340d2010 con 0x7f4f340b7510 2024-09-17T04:32:31.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.029+0000 7f4f2e000640 1 -- 172.21.15.100:0/1452806169 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v139) ==== 97+0+0 (secure 0 0 0) 0x7f4f30082550 con 0x7f4f340b7510 2024-09-17T04:32:31.029 INFO:teuthology.orchestra.run.smithi100.stderr: removed all blocklist entries 2024-09-17T04:32:31.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.032+0000 7f4f3b200640 1 -- 172.21.15.100:0/1452806169 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4f18079280 msgr2=0x7f4f1807b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:31.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.032+0000 7f4f3b200640 1 --2- 172.21.15.100:0/1452806169 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4f18079280 0x7f4f1807b790 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009c50 tx=0x7f4f28005f90 comp rx=0 tx=0).stop 2024-09-17T04:32:31.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.032+0000 7f4f3b200640 1 -- 172.21.15.100:0/1452806169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f340b7510 msgr2=0x7f4f340b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:31.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.032+0000 7f4f3b200640 1 --2- 172.21.15.100:0/1452806169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f340b7510 0x7f4f340b7920 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f4f3002af50 tx=0x7f4f300062e0 comp rx=0 tx=0).stop 2024-09-17T04:32:31.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.032+0000 7f4f3b200640 1 -- 172.21.15.100:0/1452806169 shutdown_connections 2024-09-17T04:32:31.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.032+0000 7f4f3b200640 1 --2- 172.21.15.100:0/1452806169 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4f18079280 0x7f4f1807b790 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:31.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.032+0000 7f4f3b200640 1 --2- 172.21.15.100:0/1452806169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f340b7510 0x7f4f340b7920 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:31.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.032+0000 7f4f3b200640 1 --2- 172.21.15.100:0/1452806169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f341f9ab0 0x7f4f341f5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:31.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.032+0000 7f4f3b200640 1 --2- 172.21.15.100:0/1452806169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f1c004d90 0x7f4f34141fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:31.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.032+0000 7f4f3b200640 1 -- 172.21.15.100:0/1452806169 >> 172.21.15.100:0/1452806169 conn(0x7f4f3406d1c0 msgr2=0x7f4f34155520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:31.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.032+0000 7f4f3b200640 1 -- 172.21.15.100:0/1452806169 shutdown_connections 2024-09-17T04:32:31.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.032+0000 7f4f3b200640 1 -- 172.21.15.100:0/1452806169 wait complete. 2024-09-17T04:32:31.058 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T04:32:31.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.279+0000 7fa67e600640 1 -- 172.21.15.100:0/3791257802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa65c003c60 msgr2=0x7fa65c0040b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:31.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.279+0000 7fa67e600640 1 --2- 172.21.15.100:0/3791257802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa65c003c60 0x7fa65c0040b0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fa668001830 tx=0x7fa65c005830 comp rx=0 tx=0).stop 2024-09-17T04:32:31.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.279+0000 7fa677e00640 1 -- 172.21.15.100:0/3791257802 reap_dead start 2024-09-17T04:32:31.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.279+0000 7fa67e600640 1 -- 172.21.15.100:0/3791257802 shutdown_connections 2024-09-17T04:32:31.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.279+0000 7fa67e600640 1 -- 172.21.15.100:0/3791257802 >> 172.21.15.100:0/3791257802 conn(0x7fa67806d730 msgr2=0x7fa6780b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:31.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.279+0000 7fa67e600640 1 -- 172.21.15.100:0/3791257802 shutdown_connections 2024-09-17T04:32:31.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.279+0000 7fa67e600640 1 -- 172.21.15.100:0/3791257802 wait complete. 2024-09-17T04:32:31.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.281+0000 7fa67e600640 1 Processor -- start 2024-09-17T04:32:31.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.282+0000 7fa67e600640 1 -- start start 2024-09-17T04:32:31.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.283+0000 7fa67e600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa65c003c60 0x7fa678149ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:31.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.283+0000 7fa67e600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa67814dd60 0x7fa67814e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:31.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.283+0000 7fa67e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa67814f9b0 0x7fa67814fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:31.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.283+0000 7fa67e600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa6781678e0 con 0x7fa67814f9b0 2024-09-17T04:32:31.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.283+0000 7fa67e600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa67806fb10 con 0x7fa67814dd60 2024-09-17T04:32:31.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.283+0000 7fa67e600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa678167a60 con 0x7fa65c003c60 2024-09-17T04:32:31.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.283+0000 7fa677e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa67814f9b0 0x7fa67814fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:31.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.283+0000 7fa677e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa67814f9b0 0x7fa67814fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38408/0 (socket says 172.21.15.100:38408) 2024-09-17T04:32:31.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.283+0000 7fa677e00640 1 -- 172.21.15.100:0/2634393250 learned_addr learned my addr 172.21.15.100:0/2634393250 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:31.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.283+0000 7fa66ea00640 1 --2- 172.21.15.100:0/2634393250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa67814dd60 0x7fa67814e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:31.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.283+0000 7fa677400640 1 --2- 172.21.15.100:0/2634393250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa65c003c60 0x7fa678149ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:31.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.284+0000 7fa677e00640 1 -- 172.21.15.100:0/2634393250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa65c003c60 msgr2=0x7fa678149ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:31.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.284+0000 7fa677e00640 1 --2- 172.21.15.100:0/2634393250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa65c003c60 0x7fa678149ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:31.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.284+0000 7fa677e00640 1 -- 172.21.15.100:0/2634393250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa67814dd60 msgr2=0x7fa67814e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:31.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.284+0000 7fa677e00640 1 --2- 172.21.15.100:0/2634393250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa67814dd60 0x7fa67814e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:31.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.284+0000 7fa677e00640 1 -- 172.21.15.100:0/2634393250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa678150300 con 0x7fa67814f9b0 2024-09-17T04:32:31.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.284+0000 7fa66ea00640 1 --2- 172.21.15.100:0/2634393250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa67814dd60 0x7fa67814e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:31.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.284+0000 7fa677e00640 1 --2- 172.21.15.100:0/2634393250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa67814f9b0 0x7fa67814fdc0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fa678159570 tx=0x7fa67000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:31.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.284+0000 7fa66fe00640 1 -- 172.21.15.100:0/2634393250 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670003970 con 0x7fa67814f9b0 2024-09-17T04:32:31.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.284+0000 7fa66fe00640 1 -- 172.21.15.100:0/2634393250 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa670003b10 con 0x7fa67814f9b0 2024-09-17T04:32:31.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.284+0000 7fa66fe00640 1 -- 172.21.15.100:0/2634393250 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670004030 con 0x7fa67814f9b0 2024-09-17T04:32:31.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.284+0000 7fa67e600640 1 -- 172.21.15.100:0/2634393250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa67814e990 con 0x7fa67814f9b0 2024-09-17T04:32:31.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.284+0000 7fa67e600640 1 -- 172.21.15.100:0/2634393250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa67814ec70 con 0x7fa67814f9b0 2024-09-17T04:32:31.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.285+0000 7fa67e600640 1 -- 172.21.15.100:0/2634393250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6780b16d0 con 0x7fa67814f9b0 2024-09-17T04:32:31.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.287+0000 7fa66fe00640 1 -- 172.21.15.100:0/2634393250 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7fa6700041d0 con 0x7fa67814f9b0 2024-09-17T04:32:31.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.288+0000 7fa66fe00640 1 --2- 172.21.15.100:0/2634393250 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa654079280 0x7fa65407b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:31.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.288+0000 7fa66fe00640 1 -- 172.21.15.100:0/2634393250 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4913+0+0 (secure 0 0 0) 0x7fa67009c150 con 0x7fa67814f9b0 2024-09-17T04:32:31.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.288+0000 7fa677400640 1 --2- 172.21.15.100:0/2634393250 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa654079280 0x7fa65407b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:31.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.289+0000 7fa677400640 1 --2- 172.21.15.100:0/2634393250 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa654079280 0x7fa65407b790 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa668032000 tx=0x7fa668001360 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:31.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.291+0000 7fa66fe00640 1 -- 172.21.15.100:0/2634393250 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa678150300 con 0x7fa67814f9b0 2024-09-17T04:32:31.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.540+0000 7fa67e600640 1 -- 172.21.15.100:0/2634393250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fa67814e6e0 con 0x7fa67814f9b0 2024-09-17T04:32:31.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.540+0000 7fa66fe00640 1 -- 172.21.15.100:0/2634393250 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) ==== 80+0+5304 (secure 0 0 0) 0x7fa670063db0 con 0x7fa67814f9b0 2024-09-17T04:32:31.541 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:32:31.541 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:32:31.541 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_dump": [ 2024-09-17T04:32:31.541 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:32:31.541 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.a", 2024-09-17T04:32:31.541 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmV8asJRAAkSgIWqGc1OeSRbs9PEOT8w==", 2024-09-17T04:32:31.541 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:32:31.541 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:32:31.541 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:32:31.541 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:32:31.541 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:32:31.542 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:32:31.542 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:32:31.542 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:32:31.542 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.b", 2024-09-17T04:32:31.542 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmpM7ZBRAA1Og785tJwa6NA9zBdmNNQA==", 2024-09-17T04:32:31.542 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:32:31.542 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:32:31.542 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:32:31.542 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:32:31.542 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:32:31.542 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:32:31.542 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:32:31.543 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:32:31.543 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.c", 2024-09-17T04:32:31.543 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmXaOGMxAAT+96g/5fAzPW8Cc2HtQUyA==", 2024-09-17T04:32:31.543 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:32:31.543 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:32:31.543 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:32:31.543 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:32:31.543 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:32:31.543 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:32:31.543 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:32:31.543 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:32:31.543 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.d", 2024-09-17T04:32:31.543 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmRqC9ExAA23kZ0Ks1lGN2k4GXL8ASrg==", 2024-09-17T04:32:31.544 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:32:31.544 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:32:31.544 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:32:31.544 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:32:31.544 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:32:31.544 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:32:31.544 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:32:31.544 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:32:31.544 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.0", 2024-09-17T04:32:31.544 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAwBOlmofFmAxAARerYkU9B2a9MM9vHa3o4LQ==", 2024-09-17T04:32:31.544 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:32:31.544 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:32:31.544 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:32:31.545 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:32:31.545 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:32:31.545 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:32:31.545 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:32:31.545 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.1", 2024-09-17T04:32:31.545 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAzBOlmbz6VExAAoVqSqXTc1wArKhdA6+NxnA==", 2024-09-17T04:32:31.545 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:32:31.545 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:32:31.545 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:32:31.545 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:32:31.545 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:32:31.545 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:32:31.545 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:32:31.546 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.2", 2024-09-17T04:32:31.546 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA2BOlmfeSAJBAAchY6XhyDRusc+zqwQ6v6wQ==", 2024-09-17T04:32:31.546 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:32:31.546 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:32:31.546 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:32:31.546 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:32:31.546 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:32:31.546 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:32:31.546 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:32:31.546 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.3", 2024-09-17T04:32:31.546 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA6BOlm/TQiBBAAHhbs2+RCjAenGR1frhbTmQ==", 2024-09-17T04:32:31.546 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:32:31.546 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:32:31.547 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:32:31.547 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:32:31.547 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:32:31.547 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:32:31.547 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:32:31.547 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.4", 2024-09-17T04:32:31.547 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBABOlm7OKvBhAALkaAKEUYN5ChwGejoSF8yA==", 2024-09-17T04:32:31.547 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:32:31.547 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:32:31.547 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:32:31.547 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:32:31.547 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:32:31.547 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:32:31.547 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:32:31.548 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.5", 2024-09-17T04:32:31.548 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBDBOlmQpi1FhAAbA47dg6ckK1PFoCZL0SBEQ==", 2024-09-17T04:32:31.548 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:32:31.548 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:32:31.548 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:32:31.548 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:32:31.548 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:32:31.548 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:32:31.548 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:32:31.548 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.6", 2024-09-17T04:32:31.548 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBGBOlmx/w0JhAAWcPvZkMYtXD/EMKrg/W/4Q==", 2024-09-17T04:32:31.548 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:32:31.548 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:32:31.549 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:32:31.549 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:32:31.549 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:32:31.549 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:32:31.549 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:32:31.549 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.7", 2024-09-17T04:32:31.549 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBJBOlmWHjWNBAAgdul2UKY9g5p7EVi0ZdrFw==", 2024-09-17T04:32:31.549 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:32:31.549 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:32:31.549 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:32:31.549 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:32:31.549 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:32:31.549 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:32:31.550 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:32:31.550 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.0", 2024-09-17T04:32:31.550 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmfYg9IBAABP1uRKGOugrA47VxwRLkjA==", 2024-09-17T04:32:31.550 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:32:31.550 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:32:31.550 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow r", 2024-09-17T04:32:31.550 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T04:32:31.550 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:32:31.550 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:32:31.550 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:32:31.550 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.1", 2024-09-17T04:32:31.550 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmByhhNRAAPGaxFCDd8ywFVwwhprrPbQ==", 2024-09-17T04:32:31.550 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:32:31.550 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:32:31.551 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:32:31.551 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:32:31.551 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:32:31.551 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:32:31.551 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:32:31.551 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:32:31.551 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.2", 2024-09-17T04:32:31.551 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmgcm0KhAAGF06VlHjBM/AGxafN6/3aA==", 2024-09-17T04:32:31.551 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:32:31.551 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:32:31.551 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:32:31.551 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:32:31.551 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:32:31.552 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:32:31.552 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:32:31.552 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:32:31.552 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.3", 2024-09-17T04:32:31.552 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtBOlm9VQ6BBAAXcahu35/uDm8Z+f+3SF0Gg==", 2024-09-17T04:32:31.552 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:32:31.552 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:32:31.552 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:32:31.552 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:32:31.552 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:32:31.552 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:32:31.552 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:32:31.552 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:32:31.553 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.admin", 2024-09-17T04:32:31.553 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAqBOlmZ+bIJBAAG3oaMNlYWNakywGDvAGbGw==", 2024-09-17T04:32:31.553 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:32:31.553 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:32:31.553 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:32:31.553 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:32:31.553 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:32:31.553 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:32:31.553 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:32:31.553 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:32:31.553 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mgr.x", 2024-09-17T04:32:31.553 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmfaL7GhAArqLH66RkbBsz+TNOKtPHOA==", 2024-09-17T04:32:31.553 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:32:31.554 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:32:31.554 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile mgr", 2024-09-17T04:32:31.554 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:32:31.554 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:32:31.554 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:32:31.554 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:32:31.554 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mgr.y", 2024-09-17T04:32:31.554 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmM4D1DhAA5uUWgRVOMujWqbYzOjFp9A==", 2024-09-17T04:32:31.554 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:32:31.554 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:32:31.554 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile mgr", 2024-09-17T04:32:31.554 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:32:31.554 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:32:31.554 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:32:31.555 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-17T04:32:31.555 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:32:31.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.543+0000 7fa67e600640 1 -- 172.21.15.100:0/2634393250 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa654079280 msgr2=0x7fa65407b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:31.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.543+0000 7fa67e600640 1 --2- 172.21.15.100:0/2634393250 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa654079280 0x7fa65407b790 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa668032000 tx=0x7fa668001360 comp rx=0 tx=0).stop 2024-09-17T04:32:31.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.543+0000 7fa67e600640 1 -- 172.21.15.100:0/2634393250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa67814f9b0 msgr2=0x7fa67814fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:31.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.543+0000 7fa67e600640 1 --2- 172.21.15.100:0/2634393250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa67814f9b0 0x7fa67814fdc0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fa678159570 tx=0x7fa67000f3c0 comp rx=0 tx=0).stop 2024-09-17T04:32:31.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.543+0000 7fa67e600640 1 -- 172.21.15.100:0/2634393250 shutdown_connections 2024-09-17T04:32:31.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.543+0000 7fa67e600640 1 --2- 172.21.15.100:0/2634393250 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa654079280 0x7fa65407b790 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:31.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.543+0000 7fa67e600640 1 --2- 172.21.15.100:0/2634393250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa67814f9b0 0x7fa67814fdc0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:31.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.543+0000 7fa67e600640 1 --2- 172.21.15.100:0/2634393250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa67814dd60 0x7fa67814e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:31.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.543+0000 7fa67e600640 1 --2- 172.21.15.100:0/2634393250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa65c003c60 0x7fa678149ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:31.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.543+0000 7fa67e600640 1 -- 172.21.15.100:0/2634393250 >> 172.21.15.100:0/2634393250 conn(0x7fa67806d730 msgr2=0x7fa6780b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:31.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.543+0000 7fa67e600640 1 -- 172.21.15.100:0/2634393250 shutdown_connections 2024-09-17T04:32:31.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.543+0000 7fa67e600640 1 -- 172.21.15.100:0/2634393250 wait complete. 2024-09-17T04:32:31.570 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T04:32:31.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.794+0000 7f5cc1800640 1 -- 172.21.15.100:0/2397534977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca0003a50 msgr2=0x7f5ca0003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:31.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.794+0000 7f5cc1800640 1 --2- 172.21.15.100:0/2397534977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca0003a50 0x7f5ca0003ec0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f5cac009b50 tx=0x7f5ca0005830 comp rx=0 tx=0).stop 2024-09-17T04:32:31.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.794+0000 7f5cc1800640 1 -- 172.21.15.100:0/2397534977 shutdown_connections 2024-09-17T04:32:31.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.794+0000 7f5cc1800640 1 --2- 172.21.15.100:0/2397534977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca0003a50 0x7f5ca0003ec0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:31.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.794+0000 7f5cc1800640 1 -- 172.21.15.100:0/2397534977 >> 172.21.15.100:0/2397534977 conn(0x7f5cbc06d730 msgr2=0x7f5cbc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:31.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.794+0000 7f5cc1800640 1 -- 172.21.15.100:0/2397534977 shutdown_connections 2024-09-17T04:32:31.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.794+0000 7f5cc1800640 1 -- 172.21.15.100:0/2397534977 wait complete. 2024-09-17T04:32:31.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.796+0000 7f5cc1800640 1 Processor -- start 2024-09-17T04:32:31.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.797+0000 7f5cc1800640 1 -- start start 2024-09-17T04:32:31.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.798+0000 7f5cc1800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ca0003a50 0x7f5cbc1f5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:31.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.798+0000 7f5cc1800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cbc14df60 0x7f5cbc14e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:31.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.798+0000 7f5cc1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cbc1f8bd0 0x7f5cbc1f8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:31.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.798+0000 7f5cc1800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc0ad4f0 con 0x7f5cbc1f8bd0 2024-09-17T04:32:31.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.798+0000 7f5cc1800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc0ad670 con 0x7f5ca0003a50 2024-09-17T04:32:31.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.798+0000 7f5cc1800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5cbc0ad7f0 con 0x7f5cbc14df60 2024-09-17T04:32:31.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.798+0000 7f5cbb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cbc1f8bd0 0x7f5cbc1f8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:31.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.798+0000 7f5cbb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cbc1f8bd0 0x7f5cbc1f8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38434/0 (socket says 172.21.15.100:38434) 2024-09-17T04:32:31.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.798+0000 7f5cbb400640 1 -- 172.21.15.100:0/411183147 learned_addr learned my addr 172.21.15.100:0/411183147 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:31.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.799+0000 7f5cbaa00640 1 --2- 172.21.15.100:0/411183147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ca0003a50 0x7f5cbc1f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:31.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.799+0000 7f5cb2000640 1 --2- 172.21.15.100:0/411183147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cbc14df60 0x7f5cbc14e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:31.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.799+0000 7f5cbb400640 1 -- 172.21.15.100:0/411183147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cbc14df60 msgr2=0x7f5cbc14e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:31.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.799+0000 7f5cbb400640 1 --2- 172.21.15.100:0/411183147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cbc14df60 0x7f5cbc14e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:31.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.799+0000 7f5cbb400640 1 -- 172.21.15.100:0/411183147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ca0003a50 msgr2=0x7f5cbc1f5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:31.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.799+0000 7f5cbb400640 1 --2- 172.21.15.100:0/411183147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ca0003a50 0x7f5cbc1f5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:31.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.799+0000 7f5cbb400640 1 -- 172.21.15.100:0/411183147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cbc1f6560 con 0x7f5cbc1f8bd0 2024-09-17T04:32:31.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.799+0000 7f5cbaa00640 1 --2- 172.21.15.100:0/411183147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ca0003a50 0x7f5cbc1f5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:31.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.799+0000 7f5cbb400640 1 --2- 172.21.15.100:0/411183147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cbc1f8bd0 0x7f5cbc1f8fe0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc0afb70 tx=0x7f5cb4006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:31.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.799+0000 7f5cb3400640 1 -- 172.21.15.100:0/411183147 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4011900 con 0x7f5cbc1f8bd0 2024-09-17T04:32:31.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.799+0000 7f5cb3400640 1 -- 172.21.15.100:0/411183147 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5cb4018070 con 0x7f5cbc1f8bd0 2024-09-17T04:32:31.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.799+0000 7f5cc1800640 1 -- 172.21.15.100:0/411183147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cbc1f5fa0 con 0x7f5cbc1f8bd0 2024-09-17T04:32:31.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.799+0000 7f5cb3400640 1 -- 172.21.15.100:0/411183147 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4011d60 con 0x7f5cbc1f8bd0 2024-09-17T04:32:31.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.800+0000 7f5cc1800640 1 -- 172.21.15.100:0/411183147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cbc1508d0 con 0x7f5cbc1f8bd0 2024-09-17T04:32:31.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.800+0000 7f5cc1800640 1 -- 172.21.15.100:0/411183147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cbc1416d0 con 0x7f5cbc1f8bd0 2024-09-17T04:32:31.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.802+0000 7f5cb3400640 1 -- 172.21.15.100:0/411183147 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f5cb401d460 con 0x7f5cbc1f8bd0 2024-09-17T04:32:31.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.803+0000 7f5cb3400640 1 --2- 172.21.15.100:0/411183147 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5c98079250 0x7f5c9807b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:31.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.803+0000 7f5cb3400640 1 -- 172.21.15.100:0/411183147 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4913+0+0 (secure 0 0 0) 0x7f5cb409bfb0 con 0x7f5cbc1f8bd0 2024-09-17T04:32:31.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.803+0000 7f5cbaa00640 1 --2- 172.21.15.100:0/411183147 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5c98079250 0x7f5c9807b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:31.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.804+0000 7f5cbaa00640 1 --2- 172.21.15.100:0/411183147 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5c98079250 0x7f5c9807b760 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5cac035000 tx=0x7f5cac0046c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:31.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.807+0000 7f5cb3400640 1 -- 172.21.15.100:0/411183147 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f5cb4063d20 con 0x7f5cbc1f8bd0 2024-09-17T04:32:32.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.999+0000 7f5cc1800640 1 -- 172.21.15.100:0/411183147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5cbc153730 con 0x7f5cbc1f8bd0 2024-09-17T04:32:32.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:31.999+0000 7f5cb3400640 1 -- 172.21.15.100:0/411183147 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v141) ==== 84+0+65 (secure 0 0 0) 0x7f5cb409d030 con 0x7f5cbc1f8bd0 2024-09-17T04:32:32.000 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:32:32.000 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-09-17T04:32:32.000 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:32:32.000 INFO:teuthology.orchestra.run.smithi100.stdout: "poolnum": 1, 2024-09-17T04:32:32.000 INFO:teuthology.orchestra.run.smithi100.stdout: "poolname": ".mgr" 2024-09-17T04:32:32.000 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:32:32.001 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-09-17T04:32:32.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.002+0000 7f5cc1800640 1 -- 172.21.15.100:0/411183147 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5c98079250 msgr2=0x7f5c9807b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:32.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.002+0000 7f5cc1800640 1 --2- 172.21.15.100:0/411183147 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5c98079250 0x7f5c9807b760 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5cac035000 tx=0x7f5cac0046c0 comp rx=0 tx=0).stop 2024-09-17T04:32:32.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.002+0000 7f5cc1800640 1 -- 172.21.15.100:0/411183147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cbc1f8bd0 msgr2=0x7f5cbc1f8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:32.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.002+0000 7f5cc1800640 1 --2- 172.21.15.100:0/411183147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cbc1f8bd0 0x7f5cbc1f8fe0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc0afb70 tx=0x7f5cb4006430 comp rx=0 tx=0).stop 2024-09-17T04:32:32.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.002+0000 7f5cc1800640 1 -- 172.21.15.100:0/411183147 shutdown_connections 2024-09-17T04:32:32.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.002+0000 7f5cc1800640 1 --2- 172.21.15.100:0/411183147 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5c98079250 0x7f5c9807b760 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:32.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.002+0000 7f5cc1800640 1 --2- 172.21.15.100:0/411183147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cbc1f8bd0 0x7f5cbc1f8fe0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:32.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.002+0000 7f5cc1800640 1 --2- 172.21.15.100:0/411183147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cbc14df60 0x7f5cbc14e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:32.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.002+0000 7f5cc1800640 1 --2- 172.21.15.100:0/411183147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ca0003a50 0x7f5cbc1f5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:32.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.002+0000 7f5cc1800640 1 -- 172.21.15.100:0/411183147 >> 172.21.15.100:0/411183147 conn(0x7f5cbc06d730 msgr2=0x7f5cbc15eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:32.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.002+0000 7f5cc1800640 1 -- 172.21.15.100:0/411183147 shutdown_connections 2024-09-17T04:32:32.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.002+0000 7f5cc1800640 1 -- 172.21.15.100:0/411183147 wait complete. 2024-09-17T04:32:32.024 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T04:32:32.025 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T04:32:32.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.251+0000 7f1d22200640 1 -- 172.21.15.100:0/1045335995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d04004800 msgr2=0x7f1d04004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:32.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.251+0000 7f1d22200640 1 --2- 172.21.15.100:0/1045335995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d04004800 0x7f1d04004c10 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f1d080016c0 tx=0x7f1d04005830 comp rx=0 tx=0).stop 2024-09-17T04:32:32.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.251+0000 7f1d22200640 1 -- 172.21.15.100:0/1045335995 shutdown_connections 2024-09-17T04:32:32.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.251+0000 7f1d22200640 1 --2- 172.21.15.100:0/1045335995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d04004800 0x7f1d04004c10 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:32.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.251+0000 7f1d22200640 1 -- 172.21.15.100:0/1045335995 >> 172.21.15.100:0/1045335995 conn(0x7f1d1c0b1dd0 msgr2=0x7f1d1c0b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:32.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.251+0000 7f1d22200640 1 -- 172.21.15.100:0/1045335995 shutdown_connections 2024-09-17T04:32:32.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.251+0000 7f1d22200640 1 -- 172.21.15.100:0/1045335995 wait complete. 2024-09-17T04:32:32.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.253+0000 7f1d22200640 1 Processor -- start 2024-09-17T04:32:32.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.254+0000 7f1d22200640 1 -- start start 2024-09-17T04:32:32.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.255+0000 7f1d22200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d04004800 0x7f1d1c09da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:32.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.255+0000 7f1d22200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d1c152490 0x7f1d1c09aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:32.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.255+0000 7f1d22200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d1c153d50 0x7f1d1c154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:32.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.255+0000 7f1d22200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c153460 con 0x7f1d04004800 2024-09-17T04:32:32.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.255+0000 7f1d22200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c1532e0 con 0x7f1d1c153d50 2024-09-17T04:32:32.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.255+0000 7f1d22200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1d1c1535e0 con 0x7f1d1c152490 2024-09-17T04:32:32.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.255+0000 7f1d20e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d04004800 0x7f1d1c09da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:32.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.255+0000 7f1d20e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d04004800 0x7f1d1c09da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38452/0 (socket says 172.21.15.100:38452) 2024-09-17T04:32:32.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.255+0000 7f1d20e00640 1 -- 172.21.15.100:0/156583342 learned_addr learned my addr 172.21.15.100:0/156583342 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:32.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.255+0000 7f1d1be00640 1 --2- 172.21.15.100:0/156583342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d1c152490 0x7f1d1c09aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:32.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.255+0000 7f1d21800640 1 --2- 172.21.15.100:0/156583342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d1c153d50 0x7f1d1c154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:32.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.256+0000 7f1d20e00640 1 -- 172.21.15.100:0/156583342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d1c152490 msgr2=0x7f1d1c09aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:32.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.256+0000 7f1d20e00640 1 --2- 172.21.15.100:0/156583342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d1c152490 0x7f1d1c09aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:32.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.256+0000 7f1d20e00640 1 -- 172.21.15.100:0/156583342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d1c153d50 msgr2=0x7f1d1c154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:32.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.256+0000 7f1d20e00640 1 --2- 172.21.15.100:0/156583342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d1c153d50 0x7f1d1c154160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:32.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.256+0000 7f1d20e00640 1 -- 172.21.15.100:0/156583342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d1c159e50 con 0x7f1d04004800 2024-09-17T04:32:32.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.256+0000 7f1d20e00640 1 --2- 172.21.15.100:0/156583342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d04004800 0x7f1d1c09da40 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f1d10011900 tx=0x7f1d10011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:32.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.256+0000 7f1d18c00640 1 -- 172.21.15.100:0/156583342 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1000fe10 con 0x7f1d04004800 2024-09-17T04:32:32.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.256+0000 7f1d18c00640 1 -- 172.21.15.100:0/156583342 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d10004030 con 0x7f1d04004800 2024-09-17T04:32:32.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.256+0000 7f1d22200640 1 -- 172.21.15.100:0/156583342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d1c15a130 con 0x7f1d04004800 2024-09-17T04:32:32.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.256+0000 7f1d18c00640 1 -- 172.21.15.100:0/156583342 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d10004440 con 0x7f1d04004800 2024-09-17T04:32:32.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.256+0000 7f1d22200640 1 -- 172.21.15.100:0/156583342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d1c09af90 con 0x7f1d04004800 2024-09-17T04:32:32.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.257+0000 7f1d22200640 1 -- 172.21.15.100:0/156583342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d1c153b40 con 0x7f1d04004800 2024-09-17T04:32:32.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.259+0000 7f1d18c00640 1 -- 172.21.15.100:0/156583342 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f1d100035f0 con 0x7f1d04004800 2024-09-17T04:32:32.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.260+0000 7f1d18c00640 1 --2- 172.21.15.100:0/156583342 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1cfc079250 0x7f1cfc07b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:32.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.260+0000 7f1d18c00640 1 -- 172.21.15.100:0/156583342 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4913+0+0 (secure 0 0 0) 0x7f1d1009c330 con 0x7f1d04004800 2024-09-17T04:32:32.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.260+0000 7f1d1be00640 1 --2- 172.21.15.100:0/156583342 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1cfc079250 0x7f1cfc07b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:32.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.260+0000 7f1d1be00640 1 --2- 172.21.15.100:0/156583342 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1cfc079250 0x7f1cfc07b760 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1d08032000 tx=0x7f1d08001190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:32.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.264+0000 7f1d18c00640 1 -- 172.21.15.100:0/156583342 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f1d100641f0 con 0x7f1d04004800 2024-09-17T04:32:32.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:32.447+0000 7f1d22200640 1 -- 172.21.15.100:0/156583342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f1d1c0b2ef0 con 0x7f1d04004800 2024-09-17T04:32:33.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.048+0000 7f1d18c00640 1 -- 172.21.15.100:0/156583342 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v142) ==== 138+0+0 (secure 0 0 0) 0x7f1d10063af0 con 0x7f1d04004800 2024-09-17T04:32:33.049 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' created 2024-09-17T04:32:33.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.051+0000 7f1d22200640 1 -- 172.21.15.100:0/156583342 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1cfc079250 msgr2=0x7f1cfc07b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:33.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.051+0000 7f1d22200640 1 --2- 172.21.15.100:0/156583342 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1cfc079250 0x7f1cfc07b760 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1d08032000 tx=0x7f1d08001190 comp rx=0 tx=0).stop 2024-09-17T04:32:33.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.051+0000 7f1d22200640 1 -- 172.21.15.100:0/156583342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d04004800 msgr2=0x7f1d1c09da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:33.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.051+0000 7f1d22200640 1 --2- 172.21.15.100:0/156583342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d04004800 0x7f1d1c09da40 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f1d10011900 tx=0x7f1d10011dc0 comp rx=0 tx=0).stop 2024-09-17T04:32:33.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.051+0000 7f1d22200640 1 -- 172.21.15.100:0/156583342 shutdown_connections 2024-09-17T04:32:33.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.051+0000 7f1d22200640 1 --2- 172.21.15.100:0/156583342 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1cfc079250 0x7f1cfc07b760 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:33.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.051+0000 7f1d22200640 1 --2- 172.21.15.100:0/156583342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d1c153d50 0x7f1d1c154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:33.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.051+0000 7f1d22200640 1 --2- 172.21.15.100:0/156583342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d1c152490 0x7f1d1c09aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:33.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.051+0000 7f1d22200640 1 --2- 172.21.15.100:0/156583342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d04004800 0x7f1d1c09da40 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:33.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.051+0000 7f1d22200640 1 -- 172.21.15.100:0/156583342 >> 172.21.15.100:0/156583342 conn(0x7f1d1c0b1dd0 msgr2=0x7f1d1c0b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:33.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.051+0000 7f1d22200640 1 -- 172.21.15.100:0/156583342 shutdown_connections 2024-09-17T04:32:33.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.051+0000 7f1d22200640 1 -- 172.21.15.100:0/156583342 wait complete. 2024-09-17T04:32:33.078 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T04:32:33.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.304+0000 7f5de7000640 1 -- 172.21.15.100:0/1559985453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5de00af4e0 msgr2=0x7f5de015d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:33.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.304+0000 7f5de7000640 1 --2- 172.21.15.100:0/1559985453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5de00af4e0 0x7f5de015d790 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4005fd0 tx=0x7f5dd402a540 comp rx=0 tx=0).stop 2024-09-17T04:32:33.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.304+0000 7f5de7000640 1 -- 172.21.15.100:0/1559985453 shutdown_connections 2024-09-17T04:32:33.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.304+0000 7f5de7000640 1 --2- 172.21.15.100:0/1559985453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5de0160370 0x7f5de00ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:33.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.304+0000 7f5de7000640 1 --2- 172.21.15.100:0/1559985453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5de00af4e0 0x7f5de015d790 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:33.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.304+0000 7f5de7000640 1 -- 172.21.15.100:0/1559985453 >> 172.21.15.100:0/1559985453 conn(0x7f5de00b1dd0 msgr2=0x7f5de00b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:33.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.304+0000 7f5de7000640 1 -- 172.21.15.100:0/1559985453 shutdown_connections 2024-09-17T04:32:33.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.304+0000 7f5de7000640 1 -- 172.21.15.100:0/1559985453 wait complete. 2024-09-17T04:32:33.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.306+0000 7f5de7000640 1 Processor -- start 2024-09-17T04:32:33.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.306+0000 7f5de7000640 1 -- start start 2024-09-17T04:32:33.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.308+0000 7f5de7000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de00af4e0 0x7f5de0156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:33.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.308+0000 7f5de7000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5de0160370 0x7f5de015b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:33.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.308+0000 7f5de7000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5de0160780 0x7f5de015bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:33.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.308+0000 7f5de7000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5de00cd400 con 0x7f5de00af4e0 2024-09-17T04:32:33.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.308+0000 7f5de7000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5de00cd280 con 0x7f5de0160780 2024-09-17T04:32:33.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.308+0000 7f5de7000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5de00b46d0 con 0x7f5de0160370 2024-09-17T04:32:33.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.308+0000 7f5de5c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de00af4e0 0x7f5de0156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:33.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.308+0000 7f5de5c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de00af4e0 0x7f5de0156330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38454/0 (socket says 172.21.15.100:38454) 2024-09-17T04:32:33.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.308+0000 7f5de5c00640 1 -- 172.21.15.100:0/1006333803 learned_addr learned my addr 172.21.15.100:0/1006333803 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:33.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.308+0000 7f5de5200640 1 --2- 172.21.15.100:0/1006333803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5de0160370 0x7f5de015b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:33.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.308+0000 7f5de6600640 1 --2- 172.21.15.100:0/1006333803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5de0160780 0x7f5de015bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:33.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.309+0000 7f5de5c00640 1 -- 172.21.15.100:0/1006333803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5de0160370 msgr2=0x7f5de015b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:33.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.309+0000 7f5de5c00640 1 --2- 172.21.15.100:0/1006333803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5de0160370 0x7f5de015b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:33.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.309+0000 7f5de5c00640 1 -- 172.21.15.100:0/1006333803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5de0160780 msgr2=0x7f5de015bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:33.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.309+0000 7f5de5c00640 1 --2- 172.21.15.100:0/1006333803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5de0160780 0x7f5de015bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:33.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.309+0000 7f5de5c00640 1 -- 172.21.15.100:0/1006333803 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de01fae80 con 0x7f5de00af4e0 2024-09-17T04:32:33.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.309+0000 7f5de5c00640 1 --2- 172.21.15.100:0/1006333803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de00af4e0 0x7f5de0156330 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4034040 tx=0x7f5dd40038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:33.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.309+0000 7f5ddd600640 1 -- 172.21.15.100:0/1006333803 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd403e030 con 0x7f5de00af4e0 2024-09-17T04:32:33.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.309+0000 7f5ddd600640 1 -- 172.21.15.100:0/1006333803 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5dd4003ca0 con 0x7f5de00af4e0 2024-09-17T04:32:33.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.309+0000 7f5ddd600640 1 -- 172.21.15.100:0/1006333803 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4035040 con 0x7f5de00af4e0 2024-09-17T04:32:33.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.309+0000 7f5de7000640 1 -- 172.21.15.100:0/1006333803 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de009b1b0 con 0x7f5de00af4e0 2024-09-17T04:32:33.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.310+0000 7f5de7000640 1 -- 172.21.15.100:0/1006333803 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de009b770 con 0x7f5de00af4e0 2024-09-17T04:32:33.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.311+0000 7f5de7000640 1 -- 172.21.15.100:0/1006333803 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5de00b4bf0 con 0x7f5de00af4e0 2024-09-17T04:32:33.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.312+0000 7f5ddd600640 1 -- 172.21.15.100:0/1006333803 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f5dd4041030 con 0x7f5de00af4e0 2024-09-17T04:32:33.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.313+0000 7f5ddd600640 1 --2- 172.21.15.100:0/1006333803 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5dc0079280 0x7f5dc007b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:33.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.313+0000 7f5ddd600640 1 -- 172.21.15.100:0/1006333803 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5283+0+0 (secure 0 0 0) 0x7f5dd40bbe30 con 0x7f5de00af4e0 2024-09-17T04:32:33.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.313+0000 7f5de5200640 1 --2- 172.21.15.100:0/1006333803 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5dc0079280 0x7f5dc007b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:33.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.314+0000 7f5de5200640 1 --2- 172.21.15.100:0/1006333803 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5dc0079280 0x7f5dc007b790 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5de01fd210 tx=0x7f5dcc005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:33.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.318+0000 7f5ddd600640 1 -- 172.21.15.100:0/1006333803 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f5dd40c1020 con 0x7f5de00af4e0 2024-09-17T04:32:33.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:33.503+0000 7f5de7000640 1 -- 172.21.15.100:0/1006333803 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5de00b2490 con 0x7f5de00af4e0 2024-09-17T04:32:34.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.055+0000 7f5ddd600640 1 -- 172.21.15.100:0/1006333803 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v143) ==== 170+0+0 (secure 0 0 0) 0x7f5dd4083370 con 0x7f5de00af4e0 2024-09-17T04:32:34.056 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' created 2024-09-17T04:32:34.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.059+0000 7f5de7000640 1 -- 172.21.15.100:0/1006333803 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5dc0079280 msgr2=0x7f5dc007b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:34.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.059+0000 7f5de7000640 1 --2- 172.21.15.100:0/1006333803 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5dc0079280 0x7f5dc007b790 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5de01fd210 tx=0x7f5dcc005f90 comp rx=0 tx=0).stop 2024-09-17T04:32:34.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.059+0000 7f5de7000640 1 -- 172.21.15.100:0/1006333803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de00af4e0 msgr2=0x7f5de0156330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:34.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.059+0000 7f5de7000640 1 --2- 172.21.15.100:0/1006333803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de00af4e0 0x7f5de0156330 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4034040 tx=0x7f5dd40038a0 comp rx=0 tx=0).stop 2024-09-17T04:32:34.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.059+0000 7f5de7000640 1 -- 172.21.15.100:0/1006333803 shutdown_connections 2024-09-17T04:32:34.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.059+0000 7f5de7000640 1 --2- 172.21.15.100:0/1006333803 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5dc0079280 0x7f5dc007b790 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:34.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.059+0000 7f5de7000640 1 --2- 172.21.15.100:0/1006333803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5de0160780 0x7f5de015bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:34.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.059+0000 7f5de7000640 1 --2- 172.21.15.100:0/1006333803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5de0160370 0x7f5de015b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:34.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.059+0000 7f5de7000640 1 --2- 172.21.15.100:0/1006333803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de00af4e0 0x7f5de0156330 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:34.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.059+0000 7f5de7000640 1 -- 172.21.15.100:0/1006333803 >> 172.21.15.100:0/1006333803 conn(0x7f5de00b1dd0 msgr2=0x7f5de006fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:34.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.059+0000 7f5de7000640 1 -- 172.21.15.100:0/1006333803 shutdown_connections 2024-09-17T04:32:34.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.059+0000 7f5de7000640 1 -- 172.21.15.100:0/1006333803 wait complete. 2024-09-17T04:32:34.099 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T04:32:34.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.322+0000 7ff030000640 1 -- 172.21.15.100:0/3215984532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff028145450 msgr2=0x7ff02806f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:34.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.322+0000 7ff030000640 1 --2- 172.21.15.100:0/3215984532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff028145450 0x7ff02806f180 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7ff028071590 tx=0x7ff01c02a560 comp rx=0 tx=0).stop 2024-09-17T04:32:34.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.322+0000 7ff030000640 1 -- 172.21.15.100:0/3215984532 shutdown_connections 2024-09-17T04:32:34.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.322+0000 7ff030000640 1 --2- 172.21.15.100:0/3215984532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff028160320 0x7ff028160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:34.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.322+0000 7ff030000640 1 --2- 172.21.15.100:0/3215984532 >> v2:172.21.15.184:3300/0 conn(0x7ff0280735e0 0x7ff0280739f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:34.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.322+0000 7ff030000640 1 --2- 172.21.15.100:0/3215984532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff028145450 0x7ff02806f180 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:34.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.322+0000 7ff030000640 1 -- 172.21.15.100:0/3215984532 >> 172.21.15.100:0/3215984532 conn(0x7ff02806d1c0 msgr2=0x7ff02806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:34.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.322+0000 7ff030000640 1 -- 172.21.15.100:0/3215984532 shutdown_connections 2024-09-17T04:32:34.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.322+0000 7ff030000640 1 -- 172.21.15.100:0/3215984532 wait complete. 2024-09-17T04:32:34.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.324+0000 7ff030000640 1 Processor -- start 2024-09-17T04:32:34.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.324+0000 7ff030000640 1 -- start start 2024-09-17T04:32:34.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.326+0000 7ff030000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0280735e0 0x7ff02815c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:34.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.326+0000 7ff030000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff028145450 0x7ff0280c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:34.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.326+0000 7ff030000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff028160320 0x7ff0280c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:34.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.326+0000 7ff030000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff028160c70 con 0x7ff028160320 2024-09-17T04:32:34.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.326+0000 7ff030000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff028071020 con 0x7ff0280735e0 2024-09-17T04:32:34.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.326+0000 7ff030000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff028160df0 con 0x7ff028145450 2024-09-17T04:32:34.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.326+0000 7ff02e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff028160320 0x7ff0280c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:34.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.326+0000 7ff02e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff028160320 0x7ff0280c6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38476/0 (socket says 172.21.15.100:38476) 2024-09-17T04:32:34.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.326+0000 7ff02e000640 1 -- 172.21.15.100:0/2596924453 learned_addr learned my addr 172.21.15.100:0/2596924453 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:34.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.326+0000 7ff02cc00640 1 --2- 172.21.15.100:0/2596924453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff028145450 0x7ff0280c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:34.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.326+0000 7ff02d600640 1 --2- 172.21.15.100:0/2596924453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0280735e0 0x7ff02815c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:34.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.326+0000 7ff02e000640 1 -- 172.21.15.100:0/2596924453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff028145450 msgr2=0x7ff0280c53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:34.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.326+0000 7ff02e000640 1 --2- 172.21.15.100:0/2596924453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff028145450 0x7ff0280c53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:34.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.326+0000 7ff02e000640 1 -- 172.21.15.100:0/2596924453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0280735e0 msgr2=0x7ff02815c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.326+0000 7ff02e000640 1 --2- 172.21.15.100:0/2596924453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0280735e0 0x7ff02815c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.326+0000 7ff02e000640 1 -- 172.21.15.100:0/2596924453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0280c5d90 con 0x7ff028160320 2024-09-17T04:32:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.327+0000 7ff02e000640 1 --2- 172.21.15.100:0/2596924453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff028160320 0x7ff0280c6ec0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7ff02000e990 tx=0x7ff02000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.327+0000 7ff025600640 1 -- 172.21.15.100:0/2596924453 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff020004d20 con 0x7ff028160320 2024-09-17T04:32:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.327+0000 7ff025600640 1 -- 172.21.15.100:0/2596924453 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff02000ca40 con 0x7ff028160320 2024-09-17T04:32:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.327+0000 7ff030000640 1 -- 172.21.15.100:0/2596924453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0280c6070 con 0x7ff028160320 2024-09-17T04:32:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.327+0000 7ff025600640 1 -- 172.21.15.100:0/2596924453 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff020005020 con 0x7ff028160320 2024-09-17T04:32:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.327+0000 7ff030000640 1 -- 172.21.15.100:0/2596924453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0280c6630 con 0x7ff028160320 2024-09-17T04:32:34.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.328+0000 7ff030000640 1 -- 172.21.15.100:0/2596924453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff028160a50 con 0x7ff028160320 2024-09-17T04:32:34.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.329+0000 7ff025600640 1 -- 172.21.15.100:0/2596924453 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7ff0200051c0 con 0x7ff028160320 2024-09-17T04:32:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.330+0000 7ff025600640 1 --2- 172.21.15.100:0/2596924453 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff008079280 0x7ff00807b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.330+0000 7ff025600640 1 -- 172.21.15.100:0/2596924453 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5665+0+0 (secure 0 0 0) 0x7ff02009c280 con 0x7ff028160320 2024-09-17T04:32:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.331+0000 7ff02d600640 1 --2- 172.21.15.100:0/2596924453 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff008079280 0x7ff00807b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.331+0000 7ff02d600640 1 --2- 172.21.15.100:0/2596924453 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff008079280 0x7ff00807b790 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff01c033000 tx=0x7ff01c004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:34.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.335+0000 7ff025600640 1 -- 172.21.15.100:0/2596924453 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff020016030 con 0x7ff028160320 2024-09-17T04:32:34.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:34.555+0000 7ff030000640 1 -- 172.21.15.100:0/2596924453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff02806f100 con 0x7ff028160320 2024-09-17T04:32:35.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.058+0000 7ff025600640 1 -- 172.21.15.100:0/2596924453 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v97) ==== 174+0+0 (secure 0 0 0) 0x7ff0200635b0 con 0x7ff028160320 2024-09-17T04:32:35.058 INFO:teuthology.orchestra.run.smithi100.stderr:new fs with metadata pool 22 and data pool 23 2024-09-17T04:32:35.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.062+0000 7ff030000640 1 -- 172.21.15.100:0/2596924453 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff008079280 msgr2=0x7ff00807b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:35.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.062+0000 7ff030000640 1 --2- 172.21.15.100:0/2596924453 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff008079280 0x7ff00807b790 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff01c033000 tx=0x7ff01c004730 comp rx=0 tx=0).stop 2024-09-17T04:32:35.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.062+0000 7ff030000640 1 -- 172.21.15.100:0/2596924453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff028160320 msgr2=0x7ff0280c6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:35.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.062+0000 7ff030000640 1 --2- 172.21.15.100:0/2596924453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff028160320 0x7ff0280c6ec0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7ff02000e990 tx=0x7ff02000ee50 comp rx=0 tx=0).stop 2024-09-17T04:32:35.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.062+0000 7ff030000640 1 -- 172.21.15.100:0/2596924453 shutdown_connections 2024-09-17T04:32:35.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.062+0000 7ff030000640 1 --2- 172.21.15.100:0/2596924453 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff008079280 0x7ff00807b790 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:35.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.062+0000 7ff030000640 1 --2- 172.21.15.100:0/2596924453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff028160320 0x7ff0280c6ec0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:35.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.062+0000 7ff030000640 1 --2- 172.21.15.100:0/2596924453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff028145450 0x7ff0280c53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:35.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.062+0000 7ff030000640 1 --2- 172.21.15.100:0/2596924453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0280735e0 0x7ff02815c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:35.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.062+0000 7ff030000640 1 -- 172.21.15.100:0/2596924453 >> 172.21.15.100:0/2596924453 conn(0x7ff02806d1c0 msgr2=0x7ff0280b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:35.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.063+0000 7ff030000640 1 -- 172.21.15.100:0/2596924453 shutdown_connections 2024-09-17T04:32:35.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.063+0000 7ff030000640 1 -- 172.21.15.100:0/2596924453 wait complete. 2024-09-17T04:32:35.106 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:32:35.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.324+0000 7f1519600640 1 -- 172.21.15.100:0/1301879935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15140704b0 msgr2=0x7f1514073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:35.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.324+0000 7f1519600640 1 --2- 172.21.15.100:0/1301879935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15140704b0 0x7f1514073040 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f151409e430 tx=0x7f150402a4d0 comp rx=0 tx=0).stop 2024-09-17T04:32:35.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.324+0000 7f1519600640 1 -- 172.21.15.100:0/1301879935 shutdown_connections 2024-09-17T04:32:35.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.324+0000 7f1519600640 1 --2- 172.21.15.100:0/1301879935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15140add10 0x7f15140b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:35.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.324+0000 7f1519600640 1 --2- 172.21.15.100:0/1301879935 >> v2:172.21.15.184:3300/0 conn(0x7f15140ad3c0 0x7f15140ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:35.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.324+0000 7f1519600640 1 --2- 172.21.15.100:0/1301879935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15140704b0 0x7f1514073040 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:35.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.324+0000 7f1519600640 1 -- 172.21.15.100:0/1301879935 >> 172.21.15.100:0/1301879935 conn(0x7f151406d730 msgr2=0x7f15140b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:35.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.324+0000 7f1519600640 1 -- 172.21.15.100:0/1301879935 shutdown_connections 2024-09-17T04:32:35.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.324+0000 7f1519600640 1 -- 172.21.15.100:0/1301879935 wait complete. 2024-09-17T04:32:35.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.327+0000 7f1519600640 1 Processor -- start 2024-09-17T04:32:35.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.327+0000 7f1519600640 1 -- start start 2024-09-17T04:32:35.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.328+0000 7f1519600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15140704b0 0x7f15141559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:35.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.328+0000 7f1519600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15140ad3c0 0x7f151409ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:35.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.328+0000 7f1519600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15140add10 0x7f151409b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:35.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.328+0000 7f1519600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f15140712b0 con 0x7f15140ad3c0 2024-09-17T04:32:35.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.328+0000 7f1519600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1514071130 con 0x7f15140704b0 2024-09-17T04:32:35.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.328+0000 7f1519600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f15140b4350 con 0x7f15140add10 2024-09-17T04:32:35.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.329+0000 7f1512000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15140ad3c0 0x7f151409ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:35.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.329+0000 7f1512000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15140ad3c0 0x7f151409ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39048/0 (socket says 172.21.15.100:39048) 2024-09-17T04:32:35.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.329+0000 7f1512000640 1 -- 172.21.15.100:0/2504931023 learned_addr learned my addr 172.21.15.100:0/2504931023 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:35.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.329+0000 7f1512a00640 1 --2- 172.21.15.100:0/2504931023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15140704b0 0x7f15141559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:35.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.329+0000 7f1513400640 1 --2- 172.21.15.100:0/2504931023 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15140add10 0x7f151409b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:35.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.329+0000 7f1512000640 1 -- 172.21.15.100:0/2504931023 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15140add10 msgr2=0x7f151409b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:35.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.329+0000 7f1512000640 1 --2- 172.21.15.100:0/2504931023 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15140add10 0x7f151409b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:35.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.329+0000 7f1512000640 1 -- 172.21.15.100:0/2504931023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15140704b0 msgr2=0x7f15141559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:35.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.329+0000 7f1512000640 1 --2- 172.21.15.100:0/2504931023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15140704b0 0x7f15141559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:35.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.329+0000 7f1512000640 1 -- 172.21.15.100:0/2504931023 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151409d1b0 con 0x7f15140ad3c0 2024-09-17T04:32:35.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.329+0000 7f1512000640 1 --2- 172.21.15.100:0/2504931023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15140ad3c0 0x7f151409ad80 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f150000da80 tx=0x7f150000df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:35.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.330+0000 7f150aa00640 1 -- 172.21.15.100:0/2504931023 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150001b070 con 0x7f15140ad3c0 2024-09-17T04:32:35.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.330+0000 7f150aa00640 1 -- 172.21.15.100:0/2504931023 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f150000e450 con 0x7f15140ad3c0 2024-09-17T04:32:35.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.330+0000 7f150aa00640 1 -- 172.21.15.100:0/2504931023 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150000e860 con 0x7f15140ad3c0 2024-09-17T04:32:35.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.330+0000 7f1519600640 1 -- 172.21.15.100:0/2504931023 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15141f4b30 con 0x7f15140ad3c0 2024-09-17T04:32:35.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.330+0000 7f1519600640 1 -- 172.21.15.100:0/2504931023 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15141f6690 con 0x7f15140ad3c0 2024-09-17T04:32:35.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.331+0000 7f1519600640 1 -- 172.21.15.100:0/2504931023 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15140ad7d0 con 0x7f15140ad3c0 2024-09-17T04:32:35.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.332+0000 7f150aa00640 1 -- 172.21.15.100:0/2504931023 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f150001e070 con 0x7f15140ad3c0 2024-09-17T04:32:35.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.333+0000 7f150aa00640 1 --2- 172.21.15.100:0/2504931023 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f14f40792b0 0x7f14f407b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:35.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.333+0000 7f150aa00640 1 -- 172.21.15.100:0/2504931023 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5765+0+0 (secure 0 0 0) 0x7f150009bbc0 con 0x7f15140ad3c0 2024-09-17T04:32:35.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.333+0000 7f1512a00640 1 --2- 172.21.15.100:0/2504931023 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f14f40792b0 0x7f14f407b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:35.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.334+0000 7f1512a00640 1 --2- 172.21.15.100:0/2504931023 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f14f40792b0 0x7f14f407b7c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1504033000 tx=0x7f1504004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:35.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.338+0000 7f150aa00640 1 -- 172.21.15.100:0/2504931023 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f1500062f90 con 0x7f15140ad3c0 2024-09-17T04:32:35.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.524+0000 7f1519600640 1 -- 172.21.15.100:0/2504931023 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f151415e420 con 0x7f15140ad3c0 2024-09-17T04:32:35.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.526+0000 7f150aa00640 1 -- 172.21.15.100:0/2504931023 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+16881 (secure 0 0 0) 0x7f15000679c0 con 0x7f15140ad3c0 2024-09-17T04:32:35.527 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:32:35.527 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":144,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:32:34.557610+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:32:32.462209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-17T04:32:33.516803+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:32:35.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.528+0000 7f1519600640 1 -- 172.21.15.100:0/2504931023 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f14f40792b0 msgr2=0x7f14f407b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:35.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.528+0000 7f1519600640 1 --2- 172.21.15.100:0/2504931023 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f14f40792b0 0x7f14f407b7c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1504033000 tx=0x7f1504004730 comp rx=0 tx=0).stop 2024-09-17T04:32:35.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.528+0000 7f1519600640 1 -- 172.21.15.100:0/2504931023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15140ad3c0 msgr2=0x7f151409ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:35.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.528+0000 7f1519600640 1 --2- 172.21.15.100:0/2504931023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15140ad3c0 0x7f151409ad80 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f150000da80 tx=0x7f150000df40 comp rx=0 tx=0).stop 2024-09-17T04:32:35.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.528+0000 7f1519600640 1 -- 172.21.15.100:0/2504931023 shutdown_connections 2024-09-17T04:32:35.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.528+0000 7f1519600640 1 --2- 172.21.15.100:0/2504931023 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f14f40792b0 0x7f14f407b7c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:35.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.528+0000 7f1519600640 1 --2- 172.21.15.100:0/2504931023 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15140add10 0x7f151409b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:35.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.528+0000 7f1519600640 1 --2- 172.21.15.100:0/2504931023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15140ad3c0 0x7f151409ad80 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:35.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.528+0000 7f1519600640 1 --2- 172.21.15.100:0/2504931023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15140704b0 0x7f15141559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:35.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.528+0000 7f1519600640 1 -- 172.21.15.100:0/2504931023 >> 172.21.15.100:0/2504931023 conn(0x7f151406d730 msgr2=0x7f151415d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:35.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.528+0000 7f1519600640 1 -- 172.21.15.100:0/2504931023 shutdown_connections 2024-09-17T04:32:35.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.528+0000 7f1519600640 1 -- 172.21.15.100:0/2504931023 wait complete. 2024-09-17T04:32:35.551 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:32:35.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.771+0000 7f592ec00640 1 -- 172.21.15.100:0/3742044480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f592815c700 msgr2=0x7f59280d1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:35.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.771+0000 7f592ec00640 1 --2- 172.21.15.100:0/3742044480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f592815c700 0x7f59280d1a00 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f592815cc90 tx=0x7f591c02a4b0 comp rx=0 tx=0).stop 2024-09-17T04:32:35.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.771+0000 7f592ec00640 1 -- 172.21.15.100:0/3742044480 shutdown_connections 2024-09-17T04:32:35.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.771+0000 7f592ec00640 1 --2- 172.21.15.100:0/3742044480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f592815c700 0x7f59280d1a00 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:35.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.771+0000 7f592ec00640 1 -- 172.21.15.100:0/3742044480 >> 172.21.15.100:0/3742044480 conn(0x7f5928071070 msgr2=0x7f5928071480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:35.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.771+0000 7f592ec00640 1 -- 172.21.15.100:0/3742044480 shutdown_connections 2024-09-17T04:32:35.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.771+0000 7f592ec00640 1 -- 172.21.15.100:0/3742044480 wait complete. 2024-09-17T04:32:35.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.773+0000 7f592ec00640 1 Processor -- start 2024-09-17T04:32:35.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.773+0000 7f592ec00640 1 -- start start 2024-09-17T04:32:35.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.775+0000 7f592ec00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59280b5150 0x7f59280ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:35.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.775+0000 7f592ec00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f592815c700 0x7f59280cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:35.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.775+0000 7f592ec00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f592815ad10 0x7f592815b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:35.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.775+0000 7f592ec00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5928160c60 con 0x7f59280b5150 2024-09-17T04:32:35.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.775+0000 7f592ec00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f59280b5770 con 0x7f592815ad10 2024-09-17T04:32:35.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.775+0000 7f592ec00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5928160de0 con 0x7f592815c700 2024-09-17T04:32:35.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.775+0000 7f592d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59280b5150 0x7f59280ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:35.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.775+0000 7f592d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59280b5150 0x7f59280ca200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39072/0 (socket says 172.21.15.100:39072) 2024-09-17T04:32:35.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.775+0000 7f592d800640 1 -- 172.21.15.100:0/3151077772 learned_addr learned my addr 172.21.15.100:0/3151077772 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:35.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.775+0000 7f592ce00640 1 --2- 172.21.15.100:0/3151077772 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f592815c700 0x7f59280cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:35.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.775+0000 7f592e200640 1 --2- 172.21.15.100:0/3151077772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f592815ad10 0x7f592815b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:35.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.775+0000 7f592d800640 1 -- 172.21.15.100:0/3151077772 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f592815c700 msgr2=0x7f59280cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:35.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.775+0000 7f592d800640 1 --2- 172.21.15.100:0/3151077772 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f592815c700 0x7f59280cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:35.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.776+0000 7f592d800640 1 -- 172.21.15.100:0/3151077772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f592815ad10 msgr2=0x7f592815b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:35.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.776+0000 7f592d800640 1 --2- 172.21.15.100:0/3151077772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f592815ad10 0x7f592815b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:35.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.776+0000 7f592d800640 1 -- 172.21.15.100:0/3151077772 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59280ca740 con 0x7f59280b5150 2024-09-17T04:32:35.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.776+0000 7f592d800640 1 --2- 172.21.15.100:0/3151077772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59280b5150 0x7f59280ca200 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f591c034040 tx=0x7f591c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:35.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.776+0000 7f5925600640 1 -- 172.21.15.100:0/3151077772 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c03e030 con 0x7f59280b5150 2024-09-17T04:32:35.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.776+0000 7f5925600640 1 -- 172.21.15.100:0/3151077772 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f591c0047a0 con 0x7f59280b5150 2024-09-17T04:32:35.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.776+0000 7f5925600640 1 -- 172.21.15.100:0/3151077772 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c035040 con 0x7f59280b5150 2024-09-17T04:32:35.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.776+0000 7f592ec00640 1 -- 172.21.15.100:0/3151077772 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59281f98a0 con 0x7f59280b5150 2024-09-17T04:32:35.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.776+0000 7f592ec00640 1 -- 172.21.15.100:0/3151077772 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59281f9e60 con 0x7f59280b5150 2024-09-17T04:32:35.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.777+0000 7f592ec00640 1 -- 172.21.15.100:0/3151077772 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59280d1a00 con 0x7f59280b5150 2024-09-17T04:32:35.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.779+0000 7f5925600640 1 -- 172.21.15.100:0/3151077772 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f591c041030 con 0x7f59280b5150 2024-09-17T04:32:35.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.780+0000 7f5925600640 1 --2- 172.21.15.100:0/3151077772 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5908079250 0x7f590807b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:35.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.780+0000 7f5925600640 1 -- 172.21.15.100:0/3151077772 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5765+0+0 (secure 0 0 0) 0x7f591c0bc2e0 con 0x7f59280b5150 2024-09-17T04:32:35.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.780+0000 7f592ce00640 1 --2- 172.21.15.100:0/3151077772 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5908079250 0x7f590807b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:35.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.780+0000 7f592ce00640 1 --2- 172.21.15.100:0/3151077772 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5908079250 0x7f590807b760 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5914006fd0 tx=0x7f5914008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:35.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.784+0000 7f5925600640 1 -- 172.21.15.100:0/3151077772 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f591c083b70 con 0x7f59280b5150 2024-09-17T04:32:35.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.966+0000 7f592ec00640 1 -- 172.21.15.100:0/3151077772 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59280b6cd0 con 0x7f59280b5150 2024-09-17T04:32:35.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.968+0000 7f5925600640 1 -- 172.21.15.100:0/3151077772 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+16881 (secure 0 0 0) 0x7f591c083470 con 0x7f59280b5150 2024-09-17T04:32:35.969 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:32:35.969 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":144,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:32:34.557610+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:32:32.462209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-17T04:32:33.516803+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:32:35.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.970+0000 7f592ec00640 1 -- 172.21.15.100:0/3151077772 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5908079250 msgr2=0x7f590807b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:35.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.970+0000 7f592ec00640 1 --2- 172.21.15.100:0/3151077772 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5908079250 0x7f590807b760 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5914006fd0 tx=0x7f5914008040 comp rx=0 tx=0).stop 2024-09-17T04:32:35.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.970+0000 7f592ec00640 1 -- 172.21.15.100:0/3151077772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59280b5150 msgr2=0x7f59280ca200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:35.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.970+0000 7f592ec00640 1 --2- 172.21.15.100:0/3151077772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59280b5150 0x7f59280ca200 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f591c034040 tx=0x7f591c0039e0 comp rx=0 tx=0).stop 2024-09-17T04:32:35.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.970+0000 7f592ec00640 1 -- 172.21.15.100:0/3151077772 shutdown_connections 2024-09-17T04:32:35.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.970+0000 7f592ec00640 1 --2- 172.21.15.100:0/3151077772 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5908079250 0x7f590807b760 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:35.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.970+0000 7f592ec00640 1 --2- 172.21.15.100:0/3151077772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f592815ad10 0x7f592815b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:35.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.970+0000 7f592ec00640 1 --2- 172.21.15.100:0/3151077772 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f592815c700 0x7f59280cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:35.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.970+0000 7f592ec00640 1 --2- 172.21.15.100:0/3151077772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59280b5150 0x7f59280ca200 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:35.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.970+0000 7f592ec00640 1 -- 172.21.15.100:0/3151077772 >> 172.21.15.100:0/3151077772 conn(0x7f5928071070 msgr2=0x7f59280b5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:35.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.970+0000 7f592ec00640 1 -- 172.21.15.100:0/3151077772 shutdown_connections 2024-09-17T04:32:35.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:35.970+0000 7f592ec00640 1 -- 172.21.15.100:0/3151077772 wait complete. 2024-09-17T04:32:35.995 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T04:32:36.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.222+0000 7ff351000640 1 -- 172.21.15.100:0/2645631280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff34c0cd000 msgr2=0x7ff34c0af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:36.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.222+0000 7ff351000640 1 --2- 172.21.15.100:0/2645631280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff34c0cd000 0x7ff34c0af0d0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0b3160 tx=0x7ff34402aac0 comp rx=0 tx=0).stop 2024-09-17T04:32:36.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.222+0000 7ff351000640 1 -- 172.21.15.100:0/2645631280 shutdown_connections 2024-09-17T04:32:36.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.222+0000 7ff351000640 1 --2- 172.21.15.100:0/2645631280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff34c0cd000 0x7ff34c0af0d0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:36.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.222+0000 7ff351000640 1 --2- 172.21.15.100:0/2645631280 >> v2:172.21.15.184:3300/0 conn(0x7ff34c156ca0 0x7ff34c0ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:36.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.222+0000 7ff351000640 1 -- 172.21.15.100:0/2645631280 >> 172.21.15.100:0/2645631280 conn(0x7ff34c06d620 msgr2=0x7ff34c0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:36.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.222+0000 7ff351000640 1 -- 172.21.15.100:0/2645631280 shutdown_connections 2024-09-17T04:32:36.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.222+0000 7ff351000640 1 -- 172.21.15.100:0/2645631280 wait complete. 2024-09-17T04:32:36.224 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:32:36.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.224+0000 7ff351000640 1 Processor -- start 2024-09-17T04:32:36.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.224+0000 7ff351000640 1 -- start start 2024-09-17T04:32:36.229 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:32:36.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.226+0000 7ff351000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff34c0b3f30 0x7ff34c1f36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:36.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.226+0000 7ff351000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff34c0cd000 0x7ff34c09c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:36.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.226+0000 7ff351000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff34c156ca0 0x7ff34c09c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:36.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.226+0000 7ff351000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c0af9a0 con 0x7ff34c0cd000 2024-09-17T04:32:36.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.226+0000 7ff351000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c0af820 con 0x7ff34c0b3f30 2024-09-17T04:32:36.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.226+0000 7ff351000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff34c0afb20 con 0x7ff34c156ca0 2024-09-17T04:32:36.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.226+0000 7ff342a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff34c0cd000 0x7ff34c09c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:36.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.226+0000 7ff342a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff34c0cd000 0x7ff34c09c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39092/0 (socket says 172.21.15.100:39092) 2024-09-17T04:32:36.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.226+0000 7ff342a00640 1 -- 172.21.15.100:0/2480738020 learned_addr learned my addr 172.21.15.100:0/2480738020 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:36.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.226+0000 7ff34b400640 1 --2- 172.21.15.100:0/2480738020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff34c0b3f30 0x7ff34c1f36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:36.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.226+0000 7ff34be00640 1 --2- 172.21.15.100:0/2480738020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff34c156ca0 0x7ff34c09c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:36.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.226+0000 7ff342a00640 1 -- 172.21.15.100:0/2480738020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff34c156ca0 msgr2=0x7ff34c09c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:36.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.226+0000 7ff342a00640 1 --2- 172.21.15.100:0/2480738020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff34c156ca0 0x7ff34c09c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:36.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.226+0000 7ff342a00640 1 -- 172.21.15.100:0/2480738020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff34c0b3f30 msgr2=0x7ff34c1f36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:36.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.226+0000 7ff342a00640 1 --2- 172.21.15.100:0/2480738020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff34c0b3f30 0x7ff34c1f36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:36.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.226+0000 7ff342a00640 1 -- 172.21.15.100:0/2480738020 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34c09b2f0 con 0x7ff34c0cd000 2024-09-17T04:32:36.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.226+0000 7ff34be00640 1 --2- 172.21.15.100:0/2480738020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff34c156ca0 0x7ff34c09c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:36.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.227+0000 7ff342a00640 1 --2- 172.21.15.100:0/2480738020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff34c0cd000 0x7ff34c09c3f0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7ff33400d6d0 tx=0x7ff33400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:36.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.227+0000 7ff343e00640 1 -- 172.21.15.100:0/2480738020 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334017070 con 0x7ff34c0cd000 2024-09-17T04:32:36.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.227+0000 7ff343e00640 1 -- 172.21.15.100:0/2480738020 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff334009ab0 con 0x7ff34c0cd000 2024-09-17T04:32:36.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.227+0000 7ff351000640 1 -- 172.21.15.100:0/2480738020 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff34c09b5d0 con 0x7ff34c0cd000 2024-09-17T04:32:36.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.227+0000 7ff343e00640 1 -- 172.21.15.100:0/2480738020 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334010460 con 0x7ff34c0cd000 2024-09-17T04:32:36.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.227+0000 7ff351000640 1 -- 172.21.15.100:0/2480738020 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff34c09bb90 con 0x7ff34c0cd000 2024-09-17T04:32:36.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.228+0000 7ff351000640 1 -- 172.21.15.100:0/2480738020 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff34c0af0d0 con 0x7ff34c0cd000 2024-09-17T04:32:36.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.229+0000 7ff343e00640 1 -- 172.21.15.100:0/2480738020 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7ff33401a070 con 0x7ff34c0cd000 2024-09-17T04:32:36.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.230+0000 7ff343e00640 1 --2- 172.21.15.100:0/2480738020 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff328079280 0x7ff32807b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:36.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.231+0000 7ff343e00640 1 -- 172.21.15.100:0/2480738020 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7ff33409bc70 con 0x7ff34c0cd000 2024-09-17T04:32:36.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.231+0000 7ff34b400640 1 --2- 172.21.15.100:0/2480738020 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff328079280 0x7ff32807b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:36.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.231+0000 7ff34b400640 1 --2- 172.21.15.100:0/2480738020 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff328079280 0x7ff32807b790 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0cf590 tx=0x7ff33c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:36.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.235+0000 7ff343e00640 1 -- 172.21.15.100:0/2480738020 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff334062f80 con 0x7ff34c0cd000 2024-09-17T04:32:36.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:36.462+0000 7ff351000640 1 -- 172.21.15.100:0/2480738020 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff34c09a470 con 0x7ff34c0cd000 2024-09-17T04:32:37.036 INFO:tasks.ceph.mds.a.smithi100.stderr:2024-09-17T04:32:37.036+0000 7f5559e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T04:32:37.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.083+0000 7ff343e00640 1 -- 172.21.15.100:0/2480738020 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v100) ==== 118+0+0 (secure 0 0 0) 0x7ff3340679b0 con 0x7ff34c0cd000 2024-09-17T04:32:37.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.085+0000 7ff351000640 1 -- 172.21.15.100:0/2480738020 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff328079280 msgr2=0x7ff32807b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:37.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.085+0000 7ff351000640 1 --2- 172.21.15.100:0/2480738020 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff328079280 0x7ff32807b790 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0cf590 tx=0x7ff33c009040 comp rx=0 tx=0).stop 2024-09-17T04:32:37.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.085+0000 7ff351000640 1 -- 172.21.15.100:0/2480738020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff34c0cd000 msgr2=0x7ff34c09c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:37.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.085+0000 7ff351000640 1 --2- 172.21.15.100:0/2480738020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff34c0cd000 0x7ff34c09c3f0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7ff33400d6d0 tx=0x7ff33400db90 comp rx=0 tx=0).stop 2024-09-17T04:32:37.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.086+0000 7ff351000640 1 -- 172.21.15.100:0/2480738020 shutdown_connections 2024-09-17T04:32:37.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.086+0000 7ff351000640 1 --2- 172.21.15.100:0/2480738020 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff328079280 0x7ff32807b790 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:37.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.086+0000 7ff351000640 1 --2- 172.21.15.100:0/2480738020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff34c156ca0 0x7ff34c09c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:37.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.086+0000 7ff351000640 1 --2- 172.21.15.100:0/2480738020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff34c0cd000 0x7ff34c09c3f0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:37.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.086+0000 7ff351000640 1 --2- 172.21.15.100:0/2480738020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff34c0b3f30 0x7ff34c1f36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:37.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.086+0000 7ff351000640 1 -- 172.21.15.100:0/2480738020 >> 172.21.15.100:0/2480738020 conn(0x7ff34c06d620 msgr2=0x7ff34c0ad0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:37.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.086+0000 7ff351000640 1 -- 172.21.15.100:0/2480738020 shutdown_connections 2024-09-17T04:32:37.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.086+0000 7ff351000640 1 -- 172.21.15.100:0/2480738020 wait complete. 2024-09-17T04:32:37.109 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T04:32:37.109 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T04:32:37.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.336+0000 7fd567200640 1 -- 172.21.15.100:0/4254775513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd56815d6e0 msgr2=0x7fd5680ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:37.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.336+0000 7fd567200640 1 --2- 172.21.15.100:0/4254775513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd56815d6e0 0x7fd5680ccac0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd550005fd0 tx=0x7fd55002a5d0 comp rx=0 tx=0).stop 2024-09-17T04:32:37.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.336+0000 7fd567200640 1 -- 172.21.15.100:0/4254775513 shutdown_connections 2024-09-17T04:32:37.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.336+0000 7fd567200640 1 --2- 172.21.15.100:0/4254775513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd56815d6e0 0x7fd5680ccac0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:37.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.336+0000 7fd567200640 1 --2- 172.21.15.100:0/4254775513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5680b3f30 0x7fd568156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:37.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.336+0000 7fd567200640 1 -- 172.21.15.100:0/4254775513 >> 172.21.15.100:0/4254775513 conn(0x7fd56806d620 msgr2=0x7fd5680b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:37.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.336+0000 7fd567200640 1 -- 172.21.15.100:0/4254775513 shutdown_connections 2024-09-17T04:32:37.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.336+0000 7fd567200640 1 -- 172.21.15.100:0/4254775513 wait complete. 2024-09-17T04:32:37.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.338+0000 7fd567200640 1 Processor -- start 2024-09-17T04:32:37.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.338+0000 7fd567200640 1 -- start start 2024-09-17T04:32:37.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.340+0000 7fd567200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5680b3f30 0x7fd568148b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:37.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.340+0000 7fd567200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5680cd000 0x7fd5681451a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:37.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.340+0000 7fd567200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd56815d6e0 0x7fd5681456e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:37.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.340+0000 7fd567200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd5680af960 con 0x7fd5680b3f30 2024-09-17T04:32:37.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.340+0000 7fd567200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd5680af7e0 con 0x7fd56815d6e0 2024-09-17T04:32:37.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.340+0000 7fd567200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd5680afae0 con 0x7fd5680cd000 2024-09-17T04:32:37.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.340+0000 7fd565e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5680b3f30 0x7fd568148b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:37.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.340+0000 7fd565e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5680b3f30 0x7fd568148b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39104/0 (socket says 172.21.15.100:39104) 2024-09-17T04:32:37.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.340+0000 7fd565e00640 1 -- 172.21.15.100:0/597467825 learned_addr learned my addr 172.21.15.100:0/597467825 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:37.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.340+0000 7fd55d400640 1 --2- 172.21.15.100:0/597467825 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5680cd000 0x7fd5681451a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:37.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.340+0000 7fd566800640 1 --2- 172.21.15.100:0/597467825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd56815d6e0 0x7fd5681456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:37.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.340+0000 7fd565e00640 1 -- 172.21.15.100:0/597467825 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5680cd000 msgr2=0x7fd5681451a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:37.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.340+0000 7fd565e00640 1 --2- 172.21.15.100:0/597467825 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5680cd000 0x7fd5681451a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:37.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.340+0000 7fd565e00640 1 -- 172.21.15.100:0/597467825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd56815d6e0 msgr2=0x7fd5681456e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:37.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.340+0000 7fd565e00640 1 --2- 172.21.15.100:0/597467825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd56815d6e0 0x7fd5681456e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:37.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.340+0000 7fd565e00640 1 -- 172.21.15.100:0/597467825 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5681432b0 con 0x7fd5680b3f30 2024-09-17T04:32:37.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.340+0000 7fd566800640 1 --2- 172.21.15.100:0/597467825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd56815d6e0 0x7fd5681456e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:32:37.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.341+0000 7fd565e00640 1 --2- 172.21.15.100:0/597467825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5680b3f30 0x7fd568148b00 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fd55800ea20 tx=0x7fd55800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:37.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.341+0000 7fd55ea00640 1 -- 172.21.15.100:0/597467825 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55800ce40 con 0x7fd5680b3f30 2024-09-17T04:32:37.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.341+0000 7fd55ea00640 1 -- 172.21.15.100:0/597467825 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd558004020 con 0x7fd5680b3f30 2024-09-17T04:32:37.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.341+0000 7fd567200640 1 -- 172.21.15.100:0/597467825 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5681e5bf0 con 0x7fd5680b3f30 2024-09-17T04:32:37.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.341+0000 7fd55ea00640 1 -- 172.21.15.100:0/597467825 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd558004470 con 0x7fd5680b3f30 2024-09-17T04:32:37.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.341+0000 7fd567200640 1 -- 172.21.15.100:0/597467825 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd568143830 con 0x7fd5680b3f30 2024-09-17T04:32:37.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.342+0000 7fd567200640 1 -- 172.21.15.100:0/597467825 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd568156ca0 con 0x7fd5680b3f30 2024-09-17T04:32:37.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.343+0000 7fd55ea00640 1 -- 172.21.15.100:0/597467825 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7fd5580035f0 con 0x7fd5680b3f30 2024-09-17T04:32:37.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.344+0000 7fd55ea00640 1 --2- 172.21.15.100:0/597467825 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd544079250 0x7fd54407b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:37.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.344+0000 7fd55ea00640 1 -- 172.21.15.100:0/597467825 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7fd55809c3e0 con 0x7fd5680b3f30 2024-09-17T04:32:37.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.345+0000 7fd55d400640 1 --2- 172.21.15.100:0/597467825 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd544079250 0x7fd54407b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:37.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.345+0000 7fd55d400640 1 --2- 172.21.15.100:0/597467825 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd544079250 0x7fd54407b760 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd550033000 tx=0x7fd550004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:37.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.349+0000 7fd55ea00640 1 -- 172.21.15.100:0/597467825 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd558063e10 con 0x7fd5680b3f30 2024-09-17T04:32:37.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:37.566+0000 7fd567200640 1 -- 172.21.15.100:0/597467825 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd5681e5dc0 con 0x7fd5680b3f30 2024-09-17T04:32:38.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.090+0000 7fd55ea00640 1 -- 172.21.15.100:0/597467825 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v101) ==== 122+0+0 (secure 0 0 0) 0x7fd558063710 con 0x7fd5680b3f30 2024-09-17T04:32:38.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.093+0000 7fd567200640 1 -- 172.21.15.100:0/597467825 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd544079250 msgr2=0x7fd54407b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:38.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.093+0000 7fd567200640 1 --2- 172.21.15.100:0/597467825 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd544079250 0x7fd54407b760 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd550033000 tx=0x7fd550004730 comp rx=0 tx=0).stop 2024-09-17T04:32:38.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.093+0000 7fd567200640 1 -- 172.21.15.100:0/597467825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5680b3f30 msgr2=0x7fd568148b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:38.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.093+0000 7fd567200640 1 --2- 172.21.15.100:0/597467825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5680b3f30 0x7fd568148b00 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fd55800ea20 tx=0x7fd55800eee0 comp rx=0 tx=0).stop 2024-09-17T04:32:38.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.093+0000 7fd567200640 1 -- 172.21.15.100:0/597467825 shutdown_connections 2024-09-17T04:32:38.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.093+0000 7fd567200640 1 --2- 172.21.15.100:0/597467825 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd544079250 0x7fd54407b760 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:38.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.093+0000 7fd567200640 1 --2- 172.21.15.100:0/597467825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd56815d6e0 0x7fd5681456e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:38.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.093+0000 7fd567200640 1 --2- 172.21.15.100:0/597467825 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5680cd000 0x7fd5681451a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:38.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.093+0000 7fd567200640 1 --2- 172.21.15.100:0/597467825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5680b3f30 0x7fd568148b00 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:38.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.093+0000 7fd567200640 1 -- 172.21.15.100:0/597467825 >> 172.21.15.100:0/597467825 conn(0x7fd56806d620 msgr2=0x7fd5680ae330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:38.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.094+0000 7fd567200640 1 -- 172.21.15.100:0/597467825 shutdown_connections 2024-09-17T04:32:38.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.094+0000 7fd567200640 1 -- 172.21.15.100:0/597467825 wait complete. 2024-09-17T04:32:38.119 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T04:32:38.119 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:32:38.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.339+0000 7f3d3d800640 1 -- 172.21.15.100:0/2335113156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d3815d760 msgr2=0x7f3d381535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:38.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.339+0000 7f3d3d800640 1 --2- 172.21.15.100:0/2335113156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d3815d760 0x7f3d381535c0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f3d38072d90 tx=0x7f3d2802a980 comp rx=0 tx=0).stop 2024-09-17T04:32:38.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.339+0000 7f3d3d800640 1 -- 172.21.15.100:0/2335113156 shutdown_connections 2024-09-17T04:32:38.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.339+0000 7f3d3d800640 1 --2- 172.21.15.100:0/2335113156 >> v2:172.21.15.184:3300/0 conn(0x7f3d38160310 0x7f3d38160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:38.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.339+0000 7f3d3d800640 1 --2- 172.21.15.100:0/2335113156 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d380ad0e0 0x7f3d380ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:38.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.339+0000 7f3d3d800640 1 --2- 172.21.15.100:0/2335113156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d3815d760 0x7f3d381535c0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:38.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.339+0000 7f3d3d800640 1 -- 172.21.15.100:0/2335113156 >> 172.21.15.100:0/2335113156 conn(0x7f3d3806d730 msgr2=0x7f3d380b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:38.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.339+0000 7f3d3d800640 1 -- 172.21.15.100:0/2335113156 shutdown_connections 2024-09-17T04:32:38.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.339+0000 7f3d3d800640 1 -- 172.21.15.100:0/2335113156 wait complete. 2024-09-17T04:32:38.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.342+0000 7f3d3d800640 1 Processor -- start 2024-09-17T04:32:38.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.342+0000 7f3d3d800640 1 -- start start 2024-09-17T04:32:38.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.343+0000 7f3d3d800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d380ad0e0 0x7f3d380afd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:38.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.343+0000 7f3d3d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d3815d760 0x7f3d381f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:38.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.343+0000 7f3d3d800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d38160310 0x7f3d38155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:38.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.343+0000 7f3d3d800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3d380b4da0 con 0x7f3d3815d760 2024-09-17T04:32:38.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.343+0000 7f3d3d800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3d380b4c20 con 0x7f3d380ad0e0 2024-09-17T04:32:38.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.343+0000 7f3d3d800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3d38160c60 con 0x7f3d38160310 2024-09-17T04:32:38.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.344+0000 7f3d36000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d3815d760 0x7f3d381f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:38.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.344+0000 7f3d36000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d3815d760 0x7f3d381f4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39124/0 (socket says 172.21.15.100:39124) 2024-09-17T04:32:38.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.344+0000 7f3d36000640 1 -- 172.21.15.100:0/1802295769 learned_addr learned my addr 172.21.15.100:0/1802295769 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:38.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.344+0000 7f3d37400640 1 --2- 172.21.15.100:0/1802295769 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d38160310 0x7f3d38155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:38.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.344+0000 7f3d36a00640 1 --2- 172.21.15.100:0/1802295769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d380ad0e0 0x7f3d380afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:38.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.344+0000 7f3d36000640 1 -- 172.21.15.100:0/1802295769 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d38160310 msgr2=0x7f3d38155960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:38.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.344+0000 7f3d36000640 1 --2- 172.21.15.100:0/1802295769 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d38160310 0x7f3d38155960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:38.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.344+0000 7f3d36000640 1 -- 172.21.15.100:0/1802295769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d380ad0e0 msgr2=0x7f3d380afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:38.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.344+0000 7f3d36000640 1 --2- 172.21.15.100:0/1802295769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d380ad0e0 0x7f3d380afd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:38.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.344+0000 7f3d36000640 1 -- 172.21.15.100:0/1802295769 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d380b06c0 con 0x7f3d3815d760 2024-09-17T04:32:38.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.344+0000 7f3d36000640 1 --2- 172.21.15.100:0/1802295769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d3815d760 0x7f3d381f4fe0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f3d2400e740 tx=0x7f3d2400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:38.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.345+0000 7f3d2ea00640 1 -- 172.21.15.100:0/1802295769 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2400ccc0 con 0x7f3d3815d760 2024-09-17T04:32:38.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.345+0000 7f3d2ea00640 1 -- 172.21.15.100:0/1802295769 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d2400ee50 con 0x7f3d3815d760 2024-09-17T04:32:38.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.345+0000 7f3d2ea00640 1 -- 172.21.15.100:0/1802295769 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24004290 con 0x7f3d3815d760 2024-09-17T04:32:38.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.345+0000 7f3d3d800640 1 -- 172.21.15.100:0/1802295769 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d380b09a0 con 0x7f3d3815d760 2024-09-17T04:32:38.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.345+0000 7f3d3d800640 1 -- 172.21.15.100:0/1802295769 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d380b0f60 con 0x7f3d3815d760 2024-09-17T04:32:38.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.346+0000 7f3d3d800640 1 -- 172.21.15.100:0/1802295769 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d381535c0 con 0x7f3d3815d760 2024-09-17T04:32:38.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.347+0000 7f3d2ea00640 1 -- 172.21.15.100:0/1802295769 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f3d240035f0 con 0x7f3d3815d760 2024-09-17T04:32:38.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.348+0000 7f3d2ea00640 1 --2- 172.21.15.100:0/1802295769 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3d18079250 0x7f3d1807b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:38.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.348+0000 7f3d2ea00640 1 -- 172.21.15.100:0/1802295769 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7f3d2409c220 con 0x7f3d3815d760 2024-09-17T04:32:38.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.348+0000 7f3d36a00640 1 --2- 172.21.15.100:0/1802295769 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3d18079250 0x7f3d1807b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:38.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.349+0000 7f3d36a00640 1 --2- 172.21.15.100:0/1802295769 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3d18079250 0x7f3d1807b760 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3d28035000 tx=0x7f3d28004ab0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:38.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.352+0000 7f3d2ea00640 1 -- 172.21.15.100:0/1802295769 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3d24063630 con 0x7f3d3815d760 2024-09-17T04:32:38.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.574+0000 7f3d3d800640 1 -- 172.21.15.100:0/1802295769 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d38157060 con 0x7f3d3815d760 2024-09-17T04:32:38.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.575+0000 7f3d2ea00640 1 -- 172.21.15.100:0/1802295769 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+5241 (secure 0 0 0) 0x7f3d24068060 con 0x7f3d3815d760 2024-09-17T04:32:38.575 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:32:38.575 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":101,"btime":"2024-09-17T04:32:38:084767+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7028,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1529630991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1529630991},{"type":"v1","addr":"172.21.15.184:6801","nonce":1529630991}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7906,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1516666057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1516666057},{"type":"v1","addr":"172.21.15.184:6835","nonce":1516666057}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2252725120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2252725120},{"type":"v1","addr":"172.21.15.100:6833","nonce":2252725120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:32:34.557571+0000","modified":"2024-09-17T04:32:37.568034+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8427":{"gid":8427,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/2425159512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2425159512},{"type":"v1","addr":"172.21.15.100:6835","nonce":2425159512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8427,"qdb_cluster":[8427]},"id":11}]} 2024-09-17T04:32:38.575 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 101 2024-09-17T04:32:38.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.577+0000 7f3d3d800640 1 -- 172.21.15.100:0/1802295769 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3d18079250 msgr2=0x7f3d1807b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:38.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.577+0000 7f3d3d800640 1 --2- 172.21.15.100:0/1802295769 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3d18079250 0x7f3d1807b760 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3d28035000 tx=0x7f3d28004ab0 comp rx=0 tx=0).stop 2024-09-17T04:32:38.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.577+0000 7f3d3d800640 1 -- 172.21.15.100:0/1802295769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d3815d760 msgr2=0x7f3d381f4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:38.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.577+0000 7f3d3d800640 1 --2- 172.21.15.100:0/1802295769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d3815d760 0x7f3d381f4fe0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f3d2400e740 tx=0x7f3d2400ec00 comp rx=0 tx=0).stop 2024-09-17T04:32:38.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.577+0000 7f3d3d800640 1 -- 172.21.15.100:0/1802295769 shutdown_connections 2024-09-17T04:32:38.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.577+0000 7f3d3d800640 1 --2- 172.21.15.100:0/1802295769 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3d18079250 0x7f3d1807b760 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:38.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.577+0000 7f3d3d800640 1 --2- 172.21.15.100:0/1802295769 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d38160310 0x7f3d38155960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:38.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.577+0000 7f3d3d800640 1 --2- 172.21.15.100:0/1802295769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d3815d760 0x7f3d381f4fe0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:38.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.577+0000 7f3d3d800640 1 --2- 172.21.15.100:0/1802295769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d380ad0e0 0x7f3d380afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:38.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.577+0000 7f3d3d800640 1 -- 172.21.15.100:0/1802295769 >> 172.21.15.100:0/1802295769 conn(0x7f3d3806d730 msgr2=0x7f3d380cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:38.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.578+0000 7f3d3d800640 1 -- 172.21.15.100:0/1802295769 shutdown_connections 2024-09-17T04:32:38.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.578+0000 7f3d3d800640 1 -- 172.21.15.100:0/1802295769 wait complete. 2024-09-17T04:32:38.600 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:32:38.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.819+0000 7fb9cc400640 1 -- 172.21.15.100:0/2846325285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9c4073a40 msgr2=0x7fb9c406f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:38.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.819+0000 7fb9cc400640 1 --2- 172.21.15.100:0/2846325285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9c4073a40 0x7fb9c406f0a0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4072810 tx=0x7fb9b002a5f0 comp rx=0 tx=0).stop 2024-09-17T04:32:38.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.819+0000 7fb9cc400640 1 -- 172.21.15.100:0/2846325285 shutdown_connections 2024-09-17T04:32:38.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.819+0000 7fb9cc400640 1 --2- 172.21.15.100:0/2846325285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9c4073a40 0x7fb9c406f0a0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:38.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.819+0000 7fb9cc400640 1 --2- 172.21.15.100:0/2846325285 >> v2:172.21.15.184:3300/0 conn(0x7fb9c40af630 0x7fb9c4073500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:38.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.819+0000 7fb9cc400640 1 -- 172.21.15.100:0/2846325285 >> 172.21.15.100:0/2846325285 conn(0x7fb9c406d730 msgr2=0x7fb9c40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:38.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.819+0000 7fb9cc400640 1 -- 172.21.15.100:0/2846325285 shutdown_connections 2024-09-17T04:32:38.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.819+0000 7fb9cc400640 1 -- 172.21.15.100:0/2846325285 wait complete. 2024-09-17T04:32:38.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.821+0000 7fb9cc400640 1 Processor -- start 2024-09-17T04:32:38.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.822+0000 7fb9cc400640 1 -- start start 2024-09-17T04:32:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.823+0000 7fb9cc400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9c406f5e0 0x7fb9c409dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.823+0000 7fb9cc400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9c4073a40 0x7fb9c415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.823+0000 7fb9cc400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9c40af630 0x7fb9c409b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.823+0000 7fb9cc400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c4162c70 con 0x7fb9c40af630 2024-09-17T04:32:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.823+0000 7fb9cc400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c40b4400 con 0x7fb9c406f5e0 2024-09-17T04:32:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.823+0000 7fb9cc400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb9c4162df0 con 0x7fb9c4073a40 2024-09-17T04:32:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.823+0000 7fb9ca400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9c40af630 0x7fb9c409b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.823+0000 7fb9ca400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9c40af630 0x7fb9c409b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39146/0 (socket says 172.21.15.100:39146) 2024-09-17T04:32:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.824+0000 7fb9ca400640 1 -- 172.21.15.100:0/3195974415 learned_addr learned my addr 172.21.15.100:0/3195974415 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.824+0000 7fb9c9a00640 1 --2- 172.21.15.100:0/3195974415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9c406f5e0 0x7fb9c409dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.824+0000 7fb9c9000640 1 --2- 172.21.15.100:0/3195974415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9c4073a40 0x7fb9c415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.824+0000 7fb9ca400640 1 -- 172.21.15.100:0/3195974415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9c4073a40 msgr2=0x7fb9c415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.824+0000 7fb9ca400640 1 --2- 172.21.15.100:0/3195974415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9c4073a40 0x7fb9c415dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.824+0000 7fb9ca400640 1 -- 172.21.15.100:0/3195974415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9c406f5e0 msgr2=0x7fb9c409dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.824+0000 7fb9ca400640 1 --2- 172.21.15.100:0/3195974415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9c406f5e0 0x7fb9c409dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.824+0000 7fb9ca400640 1 -- 172.21.15.100:0/3195974415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c409c1a0 con 0x7fb9c40af630 2024-09-17T04:32:38.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.824+0000 7fb9ca400640 1 --2- 172.21.15.100:0/3195974415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9c40af630 0x7fb9c409b470 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc00c730 tx=0x7fb9bc00cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:38.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.824+0000 7fb9c1600640 1 -- 172.21.15.100:0/3195974415 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc003b70 con 0x7fb9c40af630 2024-09-17T04:32:38.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.825+0000 7fb9c1600640 1 -- 172.21.15.100:0/3195974415 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9bc003d10 con 0x7fb9c40af630 2024-09-17T04:32:38.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.825+0000 7fb9cc400640 1 -- 172.21.15.100:0/3195974415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c409ce00 con 0x7fb9c40af630 2024-09-17T04:32:38.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.825+0000 7fb9c1600640 1 -- 172.21.15.100:0/3195974415 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc005330 con 0x7fb9c40af630 2024-09-17T04:32:38.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.825+0000 7fb9cc400640 1 -- 172.21.15.100:0/3195974415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c409d3c0 con 0x7fb9c40af630 2024-09-17T04:32:38.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.826+0000 7fb9cc400640 1 -- 172.21.15.100:0/3195974415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9c409bf10 con 0x7fb9c40af630 2024-09-17T04:32:38.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.827+0000 7fb9c1600640 1 -- 172.21.15.100:0/3195974415 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7fb9bc0054d0 con 0x7fb9c40af630 2024-09-17T04:32:38.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.828+0000 7fb9c1600640 1 --2- 172.21.15.100:0/3195974415 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb9a4079280 0x7fb9a407b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:38.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.828+0000 7fb9c1600640 1 -- 172.21.15.100:0/3195974415 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7fb9bc09b630 con 0x7fb9c40af630 2024-09-17T04:32:38.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.828+0000 7fb9c9a00640 1 --2- 172.21.15.100:0/3195974415 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb9a4079280 0x7fb9a407b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:38.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.829+0000 7fb9c9a00640 1 --2- 172.21.15.100:0/3195974415 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb9a4079280 0x7fb9a407b790 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb9b80098e0 tx=0x7fb9b8009060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:38.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:38.832+0000 7fb9c1600640 1 -- 172.21.15.100:0/3195974415 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb9c4073e50 con 0x7fb9c40af630 2024-09-17T04:32:39.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.017+0000 7fb9cc400640 1 -- 172.21.15.100:0/3195974415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9c40ada10 con 0x7fb9c40af630 2024-09-17T04:32:39.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.018+0000 7fb9c1600640 1 -- 172.21.15.100:0/3195974415 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16868 (secure 0 0 0) 0x7fb9bc07ce20 con 0x7fb9c40af630 2024-09-17T04:32:39.019 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:32:39.019 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":145,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:32:36.070033+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:32:32.462209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-17T04:32:33.516803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:32:39.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.020+0000 7fb9cc400640 1 -- 172.21.15.100:0/3195974415 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb9a4079280 msgr2=0x7fb9a407b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:39.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.020+0000 7fb9cc400640 1 --2- 172.21.15.100:0/3195974415 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb9a4079280 0x7fb9a407b790 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb9b80098e0 tx=0x7fb9b8009060 comp rx=0 tx=0).stop 2024-09-17T04:32:39.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.020+0000 7fb9cc400640 1 -- 172.21.15.100:0/3195974415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9c40af630 msgr2=0x7fb9c409b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:39.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.020+0000 7fb9cc400640 1 --2- 172.21.15.100:0/3195974415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9c40af630 0x7fb9c409b470 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc00c730 tx=0x7fb9bc00cb30 comp rx=0 tx=0).stop 2024-09-17T04:32:39.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.020+0000 7fb9cc400640 1 -- 172.21.15.100:0/3195974415 shutdown_connections 2024-09-17T04:32:39.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.020+0000 7fb9cc400640 1 --2- 172.21.15.100:0/3195974415 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb9a4079280 0x7fb9a407b790 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:39.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.020+0000 7fb9cc400640 1 --2- 172.21.15.100:0/3195974415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9c40af630 0x7fb9c409b470 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:39.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.020+0000 7fb9cc400640 1 --2- 172.21.15.100:0/3195974415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9c4073a40 0x7fb9c415dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:39.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.020+0000 7fb9cc400640 1 --2- 172.21.15.100:0/3195974415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9c406f5e0 0x7fb9c409dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:39.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.020+0000 7fb9cc400640 1 -- 172.21.15.100:0/3195974415 >> 172.21.15.100:0/3195974415 conn(0x7fb9c406d730 msgr2=0x7fb9c40b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:39.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.021+0000 7fb9cc400640 1 -- 172.21.15.100:0/3195974415 shutdown_connections 2024-09-17T04:32:39.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.021+0000 7fb9cc400640 1 -- 172.21.15.100:0/3195974415 wait complete. 2024-09-17T04:32:39.047 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T04:32:39.047 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:32:39.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.267+0000 7f2955000640 1 -- 172.21.15.100:0/2300607146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2950160310 msgr2=0x7f2950160720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:39.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.267+0000 7f2955000640 1 --2- 172.21.15.100:0/2300607146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2950160310 0x7f2950160720 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f2950075440 tx=0x7f294802a980 comp rx=0 tx=0).stop 2024-09-17T04:32:39.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.267+0000 7f2955000640 1 -- 172.21.15.100:0/2300607146 shutdown_connections 2024-09-17T04:32:39.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.267+0000 7f2955000640 1 --2- 172.21.15.100:0/2300607146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2950160310 0x7f2950160720 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:39.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.267+0000 7f2955000640 1 --2- 172.21.15.100:0/2300607146 >> v2:172.21.15.184:3300/0 conn(0x7f29500ad0e0 0x7f29500ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:39.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.267+0000 7f2955000640 1 --2- 172.21.15.100:0/2300607146 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f295015d760 0x7f29501535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:39.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.267+0000 7f2955000640 1 -- 172.21.15.100:0/2300607146 >> 172.21.15.100:0/2300607146 conn(0x7f295006d730 msgr2=0x7f29500b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:39.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.267+0000 7f2955000640 1 -- 172.21.15.100:0/2300607146 shutdown_connections 2024-09-17T04:32:39.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.267+0000 7f2955000640 1 -- 172.21.15.100:0/2300607146 wait complete. 2024-09-17T04:32:39.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.270+0000 7f2955000640 1 Processor -- start 2024-09-17T04:32:39.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.270+0000 7f2955000640 1 -- start start 2024-09-17T04:32:39.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.271+0000 7f2955000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29500ad0e0 0x7f29501499b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:39.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.271+0000 7f2955000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f295015d760 0x7f2950151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:39.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.271+0000 7f2955000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2950160310 0x7f2950151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:39.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.271+0000 7f2955000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f29500b4da0 con 0x7f295015d760 2024-09-17T04:32:39.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.271+0000 7f2955000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f29500b4c20 con 0x7f29500ad0e0 2024-09-17T04:32:39.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.271+0000 7f2955000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2950160c60 con 0x7f2950160310 2024-09-17T04:32:39.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.271+0000 7f2945400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f295015d760 0x7f2950151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:39.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.271+0000 7f2945400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f295015d760 0x7f2950151660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39168/0 (socket says 172.21.15.100:39168) 2024-09-17T04:32:39.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.271+0000 7f2945400640 1 -- 172.21.15.100:0/2617892818 learned_addr learned my addr 172.21.15.100:0/2617892818 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:39.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.272+0000 7f294de00640 1 --2- 172.21.15.100:0/2617892818 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29500ad0e0 0x7f29501499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:39.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.272+0000 7f294e800640 1 --2- 172.21.15.100:0/2617892818 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2950160310 0x7f2950151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:39.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.272+0000 7f2945400640 1 -- 172.21.15.100:0/2617892818 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2950160310 msgr2=0x7f2950151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:39.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.272+0000 7f2945400640 1 --2- 172.21.15.100:0/2617892818 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2950160310 0x7f2950151ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:39.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.272+0000 7f2945400640 1 -- 172.21.15.100:0/2617892818 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29500ad0e0 msgr2=0x7f29501499b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:39.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.272+0000 7f2945400640 1 --2- 172.21.15.100:0/2617892818 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29500ad0e0 0x7f29501499b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:39.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.272+0000 7f2945400640 1 -- 172.21.15.100:0/2617892818 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2950153e20 con 0x7f295015d760 2024-09-17T04:32:39.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.272+0000 7f2945400640 1 --2- 172.21.15.100:0/2617892818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f295015d760 0x7f2950151660 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f293800d700 tx=0x7f293800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:39.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.272+0000 7f2946a00640 1 -- 172.21.15.100:0/2617892818 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938017070 con 0x7f295015d760 2024-09-17T04:32:39.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.272+0000 7f2946a00640 1 -- 172.21.15.100:0/2617892818 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2938004310 con 0x7f295015d760 2024-09-17T04:32:39.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.272+0000 7f2955000640 1 -- 172.21.15.100:0/2617892818 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2950154100 con 0x7f295015d760 2024-09-17T04:32:39.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.272+0000 7f2946a00640 1 -- 172.21.15.100:0/2617892818 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938009a80 con 0x7f295015d760 2024-09-17T04:32:39.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.273+0000 7f2955000640 1 -- 172.21.15.100:0/2617892818 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29501546c0 con 0x7f295015d760 2024-09-17T04:32:39.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.274+0000 7f2955000640 1 -- 172.21.15.100:0/2617892818 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29501535c0 con 0x7f295015d760 2024-09-17T04:32:39.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.275+0000 7f2946a00640 1 -- 172.21.15.100:0/2617892818 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f2938009c20 con 0x7f295015d760 2024-09-17T04:32:39.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.276+0000 7f2946a00640 1 --2- 172.21.15.100:0/2617892818 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f292c079250 0x7f292c07b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:39.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.276+0000 7f2946a00640 1 -- 172.21.15.100:0/2617892818 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7f293809bae0 con 0x7f295015d760 2024-09-17T04:32:39.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.276+0000 7f294de00640 1 --2- 172.21.15.100:0/2617892818 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f292c079250 0x7f292c07b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:39.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.277+0000 7f294de00640 1 --2- 172.21.15.100:0/2617892818 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f292c079250 0x7f292c07b760 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f29400045b0 tx=0x7f294000b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:39.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.280+0000 7f2946a00640 1 -- 172.21.15.100:0/2617892818 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f2938062db0 con 0x7f295015d760 2024-09-17T04:32:39.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.460+0000 7f2955000640 1 -- 172.21.15.100:0/2617892818 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2950155c60 con 0x7f292c079250 2024-09-17T04:32:39.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.464+0000 7f2946a00640 1 -- 172.21.15.100:0/2617892818 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252475 (secure 0 0 0) 0x7f294000b780 con 0x7f292c079250 2024-09-17T04:32:39.467 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:32:39.469 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:32:39.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.468+0000 7f2955000640 1 -- 172.21.15.100:0/2617892818 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f292c079250 msgr2=0x7f292c07b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:39.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.468+0000 7f2955000640 1 --2- 172.21.15.100:0/2617892818 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f292c079250 0x7f292c07b760 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f29400045b0 tx=0x7f294000b2e0 comp rx=0 tx=0).stop 2024-09-17T04:32:39.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.468+0000 7f2955000640 1 -- 172.21.15.100:0/2617892818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f295015d760 msgr2=0x7f2950151660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:39.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.468+0000 7f2955000640 1 --2- 172.21.15.100:0/2617892818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f295015d760 0x7f2950151660 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f293800d700 tx=0x7f293800dbc0 comp rx=0 tx=0).stop 2024-09-17T04:32:39.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.468+0000 7f2955000640 1 -- 172.21.15.100:0/2617892818 shutdown_connections 2024-09-17T04:32:39.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.468+0000 7f2955000640 1 --2- 172.21.15.100:0/2617892818 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f292c079250 0x7f292c07b760 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:39.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.468+0000 7f2955000640 1 --2- 172.21.15.100:0/2617892818 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2950160310 0x7f2950151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:39.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.468+0000 7f2955000640 1 --2- 172.21.15.100:0/2617892818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f295015d760 0x7f2950151660 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:39.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.468+0000 7f2955000640 1 --2- 172.21.15.100:0/2617892818 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29500ad0e0 0x7f29501499b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:39.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.468+0000 7f2955000640 1 -- 172.21.15.100:0/2617892818 >> 172.21.15.100:0/2617892818 conn(0x7f295006d730 msgr2=0x7f29500b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:39.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.468+0000 7f2955000640 1 -- 172.21.15.100:0/2617892818 shutdown_connections 2024-09-17T04:32:39.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.468+0000 7f2955000640 1 -- 172.21.15.100:0/2617892818 wait complete. 2024-09-17T04:32:39.491 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2024-09-17T04:32:37.706604+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":157,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":125,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264828,"kb_used_data":15284,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754709892,"statfs":{"total":773094113280,"available":772822929408,"internally_reserved":0,"allocated":15650816,"data_stored":7534696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.654015"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.095347+0000","last_change":"2024-09-17T04:32:35.090715+0000","last_active":"2024-09-17T04:32:35.095347+0000","last_peered":"2024-09-17T04:32:35.095347+0000","last_clean":"2024-09-17T04:32:35.095347+0000","last_became_active":"2024-09-17T04:32:35.090348+0000","last_became_peered":"2024-09-17T04:32:35.090348+0000","last_unstale":"2024-09-17T04:32:35.095347+0000","last_undegraded":"2024-09-17T04:32:35.095347+0000","last_fullsized":"2024-09-17T04:32:35.095347+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:35.901594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.636805+0000","last_change":"2024-09-17T04:32:35.089497+0000","last_active":"2024-09-17T04:32:36.636805+0000","last_peered":"2024-09-17T04:32:36.636805+0000","last_clean":"2024-09-17T04:32:36.636805+0000","last_became_active":"2024-09-17T04:32:35.089171+0000","last_became_peered":"2024-09-17T04:32:35.089171+0000","last_unstale":"2024-09-17T04:32:36.636805+0000","last_undegraded":"2024-09-17T04:32:36.636805+0000","last_fullsized":"2024-09-17T04:32:36.636805+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:08.356019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080338+0000","last_change":"2024-09-17T04:32:35.081484+0000","last_active":"2024-09-17T04:32:36.080338+0000","last_peered":"2024-09-17T04:32:36.080338+0000","last_clean":"2024-09-17T04:32:36.080338+0000","last_became_active":"2024-09-17T04:32:35.081154+0000","last_became_peered":"2024-09-17T04:32:35.081154+0000","last_unstale":"2024-09-17T04:32:36.080338+0000","last_undegraded":"2024-09-17T04:32:36.080338+0000","last_fullsized":"2024-09-17T04:32:36.080338+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:10.057486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.106158+0000","last_change":"2024-09-17T04:32:35.105304+0000","last_active":"2024-09-17T04:32:35.106158+0000","last_peered":"2024-09-17T04:32:35.106158+0000","last_clean":"2024-09-17T04:32:35.106158+0000","last_became_active":"2024-09-17T04:32:35.105046+0000","last_became_peered":"2024-09-17T04:32:35.105046+0000","last_unstale":"2024-09-17T04:32:35.106158+0000","last_undegraded":"2024-09-17T04:32:35.106158+0000","last_fullsized":"2024-09-17T04:32:35.106158+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:06:11.856341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.636646+0000","last_change":"2024-09-17T04:32:35.088118+0000","last_active":"2024-09-17T04:32:36.636646+0000","last_peered":"2024-09-17T04:32:36.636646+0000","last_clean":"2024-09-17T04:32:36.636646+0000","last_became_active":"2024-09-17T04:32:35.087805+0000","last_became_peered":"2024-09-17T04:32:35.087805+0000","last_unstale":"2024-09-17T04:32:36.636646+0000","last_undegraded":"2024-09-17T04:32:36.636646+0000","last_fullsized":"2024-09-17T04:32:36.636646+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:10.890012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.078703+0000","last_change":"2024-09-17T04:32:35.091693+0000","last_active":"2024-09-17T04:32:36.078703+0000","last_peered":"2024-09-17T04:32:36.078703+0000","last_clean":"2024-09-17T04:32:36.078703+0000","last_became_active":"2024-09-17T04:32:35.091369+0000","last_became_peered":"2024-09-17T04:32:35.091369+0000","last_unstale":"2024-09-17T04:32:36.078703+0000","last_undegraded":"2024-09-17T04:32:36.078703+0000","last_fullsized":"2024-09-17T04:32:36.078703+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:58.390151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.087953+0000","last_change":"2024-09-17T04:32:35.085803+0000","last_active":"2024-09-17T04:32:35.087953+0000","last_peered":"2024-09-17T04:32:35.087953+0000","last_clean":"2024-09-17T04:32:35.087953+0000","last_became_active":"2024-09-17T04:32:35.085478+0000","last_became_peered":"2024-09-17T04:32:35.085478+0000","last_unstale":"2024-09-17T04:32:35.087953+0000","last_undegraded":"2024-09-17T04:32:35.087953+0000","last_fullsized":"2024-09-17T04:32:35.087953+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:25:42.410141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.636450+0000","last_change":"2024-09-17T04:32:35.086333+0000","last_active":"2024-09-17T04:32:36.636450+0000","last_peered":"2024-09-17T04:32:36.636450+0000","last_clean":"2024-09-17T04:32:36.636450+0000","last_became_active":"2024-09-17T04:32:35.085194+0000","last_became_peered":"2024-09-17T04:32:35.085194+0000","last_unstale":"2024-09-17T04:32:36.636450+0000","last_undegraded":"2024-09-17T04:32:36.636450+0000","last_fullsized":"2024-09-17T04:32:36.636450+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:51:21.462803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.636957+0000","last_change":"2024-09-17T04:32:35.089949+0000","last_active":"2024-09-17T04:32:36.636957+0000","last_peered":"2024-09-17T04:32:36.636957+0000","last_clean":"2024-09-17T04:32:36.636957+0000","last_became_active":"2024-09-17T04:32:35.089004+0000","last_became_peered":"2024-09-17T04:32:35.089004+0000","last_unstale":"2024-09-17T04:32:36.636957+0000","last_undegraded":"2024-09-17T04:32:36.636957+0000","last_fullsized":"2024-09-17T04:32:36.636957+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:03:17.726639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.636880+0000","last_change":"2024-09-17T04:32:35.079425+0000","last_active":"2024-09-17T04:32:36.636880+0000","last_peered":"2024-09-17T04:32:36.636880+0000","last_clean":"2024-09-17T04:32:36.636880+0000","last_became_active":"2024-09-17T04:32:35.078944+0000","last_became_peered":"2024-09-17T04:32:35.078944+0000","last_unstale":"2024-09-17T04:32:36.636880+0000","last_undegraded":"2024-09-17T04:32:36.636880+0000","last_fullsized":"2024-09-17T04:32:36.636880+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:17:31.875959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.636710+0000","last_change":"2024-09-17T04:32:35.089585+0000","last_active":"2024-09-17T04:32:36.636710+0000","last_peered":"2024-09-17T04:32:36.636710+0000","last_clean":"2024-09-17T04:32:36.636710+0000","last_became_active":"2024-09-17T04:32:35.085301+0000","last_became_peered":"2024-09-17T04:32:35.085301+0000","last_unstale":"2024-09-17T04:32:36.636710+0000","last_undegraded":"2024-09-17T04:32:36.636710+0000","last_fullsized":"2024-09-17T04:32:36.636710+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:16.000604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.636426+0000","last_change":"2024-09-17T04:32:35.082150+0000","last_active":"2024-09-17T04:32:36.636426+0000","last_peered":"2024-09-17T04:32:36.636426+0000","last_clean":"2024-09-17T04:32:36.636426+0000","last_became_active":"2024-09-17T04:32:35.081423+0000","last_became_peered":"2024-09-17T04:32:35.081423+0000","last_unstale":"2024-09-17T04:32:36.636426+0000","last_undegraded":"2024-09-17T04:32:36.636426+0000","last_fullsized":"2024-09-17T04:32:36.636426+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:00:28.386090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080659+0000","last_change":"2024-09-17T04:32:35.083646+0000","last_active":"2024-09-17T04:32:36.080659+0000","last_peered":"2024-09-17T04:32:36.080659+0000","last_clean":"2024-09-17T04:32:36.080659+0000","last_became_active":"2024-09-17T04:32:35.082348+0000","last_became_peered":"2024-09-17T04:32:35.082348+0000","last_unstale":"2024-09-17T04:32:36.080659+0000","last_undegraded":"2024-09-17T04:32:36.080659+0000","last_fullsized":"2024-09-17T04:32:36.080659+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:57:50.170842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080474+0000","last_change":"2024-09-17T04:32:35.082355+0000","last_active":"2024-09-17T04:32:36.080474+0000","last_peered":"2024-09-17T04:32:36.080474+0000","last_clean":"2024-09-17T04:32:36.080474+0000","last_became_active":"2024-09-17T04:32:35.082043+0000","last_became_peered":"2024-09-17T04:32:35.082043+0000","last_unstale":"2024-09-17T04:32:36.080474+0000","last_undegraded":"2024-09-17T04:32:36.080474+0000","last_fullsized":"2024-09-17T04:32:36.080474+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:02:44.641636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.092762+0000","last_change":"2024-09-17T04:32:35.083470+0000","last_active":"2024-09-17T04:32:35.092762+0000","last_peered":"2024-09-17T04:32:35.092762+0000","last_clean":"2024-09-17T04:32:35.092762+0000","last_became_active":"2024-09-17T04:32:35.083166+0000","last_became_peered":"2024-09-17T04:32:35.083166+0000","last_unstale":"2024-09-17T04:32:35.092762+0000","last_undegraded":"2024-09-17T04:32:35.092762+0000","last_fullsized":"2024-09-17T04:32:35.092762+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:26:25.443388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.094963+0000","last_change":"2024-09-17T04:32:35.090551+0000","last_active":"2024-09-17T04:32:35.094963+0000","last_peered":"2024-09-17T04:32:35.094963+0000","last_clean":"2024-09-17T04:32:35.094963+0000","last_became_active":"2024-09-17T04:32:35.090216+0000","last_became_peered":"2024-09-17T04:32:35.090216+0000","last_unstale":"2024-09-17T04:32:35.094963+0000","last_undegraded":"2024-09-17T04:32:35.094963+0000","last_fullsized":"2024-09-17T04:32:35.094963+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:43:59.061916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.079373+0000","last_change":"2024-09-17T04:32:35.087309+0000","last_active":"2024-09-17T04:32:36.079373+0000","last_peered":"2024-09-17T04:32:36.079373+0000","last_clean":"2024-09-17T04:32:36.079373+0000","last_became_active":"2024-09-17T04:32:35.087000+0000","last_became_peered":"2024-09-17T04:32:35.087000+0000","last_unstale":"2024-09-17T04:32:36.079373+0000","last_undegraded":"2024-09-17T04:32:36.079373+0000","last_fullsized":"2024-09-17T04:32:36.079373+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:53:03.745647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.094032+0000","last_change":"2024-09-17T04:32:35.087977+0000","last_active":"2024-09-17T04:32:35.094032+0000","last_peered":"2024-09-17T04:32:35.094032+0000","last_clean":"2024-09-17T04:32:35.094032+0000","last_became_active":"2024-09-17T04:32:35.087710+0000","last_became_peered":"2024-09-17T04:32:35.087710+0000","last_unstale":"2024-09-17T04:32:35.094032+0000","last_undegraded":"2024-09-17T04:32:35.094032+0000","last_fullsized":"2024-09-17T04:32:35.094032+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:41.604068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.079076+0000","last_change":"2024-09-17T04:32:35.071247+0000","last_active":"2024-09-17T04:32:36.079076+0000","last_peered":"2024-09-17T04:32:36.079076+0000","last_clean":"2024-09-17T04:32:36.079076+0000","last_became_active":"2024-09-17T04:32:35.070946+0000","last_became_peered":"2024-09-17T04:32:35.070946+0000","last_unstale":"2024-09-17T04:32:36.079076+0000","last_undegraded":"2024-09-17T04:32:36.079076+0000","last_fullsized":"2024-09-17T04:32:36.079076+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:57:58.612271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080103+0000","last_change":"2024-09-17T04:32:35.082505+0000","last_active":"2024-09-17T04:32:36.080103+0000","last_peered":"2024-09-17T04:32:36.080103+0000","last_clean":"2024-09-17T04:32:36.080103+0000","last_became_active":"2024-09-17T04:32:35.080875+0000","last_became_peered":"2024-09-17T04:32:35.080875+0000","last_unstale":"2024-09-17T04:32:36.080103+0000","last_undegraded":"2024-09-17T04:32:36.080103+0000","last_fullsized":"2024-09-17T04:32:36.080103+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:28:07.076135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.079030+0000","last_change":"2024-09-17T04:32:35.081404+0000","last_active":"2024-09-17T04:32:36.079030+0000","last_peered":"2024-09-17T04:32:36.079030+0000","last_clean":"2024-09-17T04:32:36.079030+0000","last_became_active":"2024-09-17T04:32:35.080899+0000","last_became_peered":"2024-09-17T04:32:35.080899+0000","last_unstale":"2024-09-17T04:32:36.079030+0000","last_undegraded":"2024-09-17T04:32:36.079030+0000","last_fullsized":"2024-09-17T04:32:36.079030+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:53.337598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.636608+0000","last_change":"2024-09-17T04:32:35.091973+0000","last_active":"2024-09-17T04:32:36.636608+0000","last_peered":"2024-09-17T04:32:36.636608+0000","last_clean":"2024-09-17T04:32:36.636608+0000","last_became_active":"2024-09-17T04:32:35.091665+0000","last_became_peered":"2024-09-17T04:32:35.091665+0000","last_unstale":"2024-09-17T04:32:36.636608+0000","last_undegraded":"2024-09-17T04:32:36.636608+0000","last_fullsized":"2024-09-17T04:32:36.636608+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:57:58.856735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.078440+0000","last_change":"2024-09-17T04:32:35.085017+0000","last_active":"2024-09-17T04:32:36.078440+0000","last_peered":"2024-09-17T04:32:36.078440+0000","last_clean":"2024-09-17T04:32:36.078440+0000","last_became_active":"2024-09-17T04:32:35.084007+0000","last_became_peered":"2024-09-17T04:32:35.084007+0000","last_unstale":"2024-09-17T04:32:36.078440+0000","last_undegraded":"2024-09-17T04:32:36.078440+0000","last_fullsized":"2024-09-17T04:32:36.078440+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:56.637818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.636501+0000","last_change":"2024-09-17T04:32:35.088288+0000","last_active":"2024-09-17T04:32:36.636501+0000","last_peered":"2024-09-17T04:32:36.636501+0000","last_clean":"2024-09-17T04:32:36.636501+0000","last_became_active":"2024-09-17T04:32:35.086037+0000","last_became_peered":"2024-09-17T04:32:35.086037+0000","last_unstale":"2024-09-17T04:32:36.636501+0000","last_undegraded":"2024-09-17T04:32:36.636501+0000","last_fullsized":"2024-09-17T04:32:36.636501+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:04.393262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080710+0000","last_change":"2024-09-17T04:32:35.076267+0000","last_active":"2024-09-17T04:32:36.080710+0000","last_peered":"2024-09-17T04:32:36.080710+0000","last_clean":"2024-09-17T04:32:36.080710+0000","last_became_active":"2024-09-17T04:32:35.075177+0000","last_became_peered":"2024-09-17T04:32:35.075177+0000","last_unstale":"2024-09-17T04:32:36.080710+0000","last_undegraded":"2024-09-17T04:32:36.080710+0000","last_fullsized":"2024-09-17T04:32:36.080710+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:42.003911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.104483+0000","last_change":"2024-09-17T04:32:35.103352+0000","last_active":"2024-09-17T04:32:35.104483+0000","last_peered":"2024-09-17T04:32:35.104483+0000","last_clean":"2024-09-17T04:32:35.104483+0000","last_became_active":"2024-09-17T04:32:35.103116+0000","last_became_peered":"2024-09-17T04:32:35.103116+0000","last_unstale":"2024-09-17T04:32:35.104483+0000","last_undegraded":"2024-09-17T04:32:35.104483+0000","last_fullsized":"2024-09-17T04:32:35.104483+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:54:39.150606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080721+0000","last_change":"2024-09-17T04:32:35.078622+0000","last_active":"2024-09-17T04:32:36.080721+0000","last_peered":"2024-09-17T04:32:36.080721+0000","last_clean":"2024-09-17T04:32:36.080721+0000","last_became_active":"2024-09-17T04:32:35.076584+0000","last_became_peered":"2024-09-17T04:32:35.076584+0000","last_unstale":"2024-09-17T04:32:36.080721+0000","last_undegraded":"2024-09-17T04:32:36.080721+0000","last_fullsized":"2024-09-17T04:32:36.080721+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:58.781510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.104965+0000","last_change":"2024-09-17T04:32:35.102530+0000","last_active":"2024-09-17T04:32:35.104965+0000","last_peered":"2024-09-17T04:32:35.104965+0000","last_clean":"2024-09-17T04:32:35.104965+0000","last_became_active":"2024-09-17T04:32:35.096082+0000","last_became_peered":"2024-09-17T04:32:35.096082+0000","last_unstale":"2024-09-17T04:32:35.104965+0000","last_undegraded":"2024-09-17T04:32:35.104965+0000","last_fullsized":"2024-09-17T04:32:35.104965+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:24.355505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080737+0000","last_change":"2024-09-17T04:32:35.079899+0000","last_active":"2024-09-17T04:32:36.080737+0000","last_peered":"2024-09-17T04:32:36.080737+0000","last_clean":"2024-09-17T04:32:36.080737+0000","last_became_active":"2024-09-17T04:32:35.079586+0000","last_became_peered":"2024-09-17T04:32:35.079586+0000","last_unstale":"2024-09-17T04:32:36.080737+0000","last_undegraded":"2024-09-17T04:32:36.080737+0000","last_fullsized":"2024-09-17T04:32:36.080737+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:30.305087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.081622+0000","last_change":"2024-09-17T04:32:35.080098+0000","last_active":"2024-09-17T04:32:36.081622+0000","last_peered":"2024-09-17T04:32:36.081622+0000","last_clean":"2024-09-17T04:32:36.081622+0000","last_became_active":"2024-09-17T04:32:35.077456+0000","last_became_peered":"2024-09-17T04:32:35.077456+0000","last_unstale":"2024-09-17T04:32:36.081622+0000","last_undegraded":"2024-09-17T04:32:36.081622+0000","last_fullsized":"2024-09-17T04:32:36.081622+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:03:17.726639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080146+0000","last_change":"2024-09-17T04:32:35.085846+0000","last_active":"2024-09-17T04:32:36.080146+0000","last_peered":"2024-09-17T04:32:36.080146+0000","last_clean":"2024-09-17T04:32:36.080146+0000","last_became_active":"2024-09-17T04:32:35.085491+0000","last_became_peered":"2024-09-17T04:32:35.085491+0000","last_unstale":"2024-09-17T04:32:36.080146+0000","last_undegraded":"2024-09-17T04:32:36.080146+0000","last_fullsized":"2024-09-17T04:32:36.080146+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:17:31.875959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.092929+0000","last_change":"2024-09-17T04:32:35.088238+0000","last_active":"2024-09-17T04:32:35.092929+0000","last_peered":"2024-09-17T04:32:35.092929+0000","last_clean":"2024-09-17T04:32:35.092929+0000","last_became_active":"2024-09-17T04:32:35.087644+0000","last_became_peered":"2024-09-17T04:32:35.087644+0000","last_unstale":"2024-09-17T04:32:35.092929+0000","last_undegraded":"2024-09-17T04:32:35.092929+0000","last_fullsized":"2024-09-17T04:32:35.092929+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:09.506332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.104456+0000","last_change":"2024-09-17T04:32:35.100948+0000","last_active":"2024-09-17T04:32:35.104456+0000","last_peered":"2024-09-17T04:32:35.104456+0000","last_clean":"2024-09-17T04:32:35.104456+0000","last_became_active":"2024-09-17T04:32:35.100532+0000","last_became_peered":"2024-09-17T04:32:35.100532+0000","last_unstale":"2024-09-17T04:32:35.104456+0000","last_undegraded":"2024-09-17T04:32:35.104456+0000","last_fullsized":"2024-09-17T04:32:35.104456+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:15:30.142426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"144'4","reported_seq":17,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.105378+0000","last_change":"2024-09-17T04:32:34.073631+0000","last_active":"2024-09-17T04:32:35.105378+0000","last_peered":"2024-09-17T04:32:35.105378+0000","last_clean":"2024-09-17T04:32:35.105378+0000","last_became_active":"2024-09-17T04:32:34.071463+0000","last_became_peered":"2024-09-17T04:32:34.071463+0000","last_unstale":"2024-09-17T04:32:35.105378+0000","last_undegraded":"2024-09-17T04:32:35.105378+0000","last_fullsized":"2024-09-17T04:32:35.105378+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:21:37.338931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.082502+0000","last_change":"2024-09-17T04:32:35.099925+0000","last_active":"2024-09-17T04:32:36.082502+0000","last_peered":"2024-09-17T04:32:36.082502+0000","last_clean":"2024-09-17T04:32:36.082502+0000","last_became_active":"2024-09-17T04:32:35.098049+0000","last_became_peered":"2024-09-17T04:32:35.098049+0000","last_unstale":"2024-09-17T04:32:36.082502+0000","last_undegraded":"2024-09-17T04:32:36.082502+0000","last_fullsized":"2024-09-17T04:32:36.082502+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:56.877315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.082003+0000","last_change":"2024-09-17T04:32:34.089003+0000","last_active":"2024-09-17T04:32:36.082003+0000","last_peered":"2024-09-17T04:32:36.082003+0000","last_clean":"2024-09-17T04:32:36.082003+0000","last_became_active":"2024-09-17T04:32:34.088705+0000","last_became_peered":"2024-09-17T04:32:34.088705+0000","last_unstale":"2024-09-17T04:32:36.082003+0000","last_undegraded":"2024-09-17T04:32:36.082003+0000","last_fullsized":"2024-09-17T04:32:36.082003+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:06:10.852486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.078968+0000","last_change":"2024-09-17T04:32:35.086724+0000","last_active":"2024-09-17T04:32:36.078968+0000","last_peered":"2024-09-17T04:32:36.078968+0000","last_clean":"2024-09-17T04:32:36.078968+0000","last_became_active":"2024-09-17T04:32:35.086414+0000","last_became_peered":"2024-09-17T04:32:35.086414+0000","last_unstale":"2024-09-17T04:32:36.078968+0000","last_undegraded":"2024-09-17T04:32:36.078968+0000","last_fullsized":"2024-09-17T04:32:36.078968+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:30.636333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"144'2","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.100002+0000","last_change":"2024-09-17T04:32:34.082468+0000","last_active":"2024-09-17T04:32:35.100002+0000","last_peered":"2024-09-17T04:32:35.100002+0000","last_clean":"2024-09-17T04:32:35.100002+0000","last_became_active":"2024-09-17T04:32:34.082122+0000","last_became_peered":"2024-09-17T04:32:34.082122+0000","last_unstale":"2024-09-17T04:32:35.100002+0000","last_undegraded":"2024-09-17T04:32:35.100002+0000","last_fullsized":"2024-09-17T04:32:35.100002+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:41.000056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080990+0000","last_change":"2024-09-17T04:32:35.074526+0000","last_active":"2024-09-17T04:32:36.080990+0000","last_peered":"2024-09-17T04:32:36.080990+0000","last_clean":"2024-09-17T04:32:36.080990+0000","last_became_active":"2024-09-17T04:32:35.074180+0000","last_became_peered":"2024-09-17T04:32:35.074180+0000","last_unstale":"2024-09-17T04:32:36.080990+0000","last_undegraded":"2024-09-17T04:32:36.080990+0000","last_fullsized":"2024-09-17T04:32:36.080990+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:21.143564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"144'2","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.103250+0000","last_change":"2024-09-17T04:32:34.078337+0000","last_active":"2024-09-17T04:32:35.103250+0000","last_peered":"2024-09-17T04:32:35.103250+0000","last_clean":"2024-09-17T04:32:35.103250+0000","last_became_active":"2024-09-17T04:32:34.077663+0000","last_became_peered":"2024-09-17T04:32:34.077663+0000","last_unstale":"2024-09-17T04:32:35.103250+0000","last_undegraded":"2024-09-17T04:32:35.103250+0000","last_fullsized":"2024-09-17T04:32:35.103250+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:20.139709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.099081+0000","last_change":"2024-09-17T04:32:35.097272+0000","last_active":"2024-09-17T04:32:35.099081+0000","last_peered":"2024-09-17T04:32:35.099081+0000","last_clean":"2024-09-17T04:32:35.099081+0000","last_became_active":"2024-09-17T04:32:35.096085+0000","last_became_peered":"2024-09-17T04:32:35.096085+0000","last_unstale":"2024-09-17T04:32:35.099081+0000","last_undegraded":"2024-09-17T04:32:35.099081+0000","last_fullsized":"2024-09-17T04:32:35.099081+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:05.879557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080812+0000","last_change":"2024-09-17T04:32:34.089042+0000","last_active":"2024-09-17T04:32:36.080812+0000","last_peered":"2024-09-17T04:32:36.080812+0000","last_clean":"2024-09-17T04:32:36.080812+0000","last_became_active":"2024-09-17T04:32:34.088339+0000","last_became_peered":"2024-09-17T04:32:34.088339+0000","last_unstale":"2024-09-17T04:32:36.080812+0000","last_undegraded":"2024-09-17T04:32:36.080812+0000","last_fullsized":"2024-09-17T04:32:36.080812+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:52.333743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.099835+0000","last_change":"2024-09-17T04:32:35.092302+0000","last_active":"2024-09-17T04:32:35.099835+0000","last_peered":"2024-09-17T04:32:35.099835+0000","last_clean":"2024-09-17T04:32:35.099835+0000","last_became_active":"2024-09-17T04:32:35.092021+0000","last_became_peered":"2024-09-17T04:32:35.092021+0000","last_unstale":"2024-09-17T04:32:35.099835+0000","last_undegraded":"2024-09-17T04:32:35.099835+0000","last_fullsized":"2024-09-17T04:32:35.099835+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:36:26.686200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"144'1","reported_seq":14,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.104383+0000","last_change":"2024-09-17T04:32:34.085874+0000","last_active":"2024-09-17T04:32:35.104383+0000","last_peered":"2024-09-17T04:32:35.104383+0000","last_clean":"2024-09-17T04:32:35.104383+0000","last_became_active":"2024-09-17T04:32:34.085387+0000","last_became_peered":"2024-09-17T04:32:34.085387+0000","last_unstale":"2024-09-17T04:32:35.104383+0000","last_undegraded":"2024-09-17T04:32:35.104383+0000","last_fullsized":"2024-09-17T04:32:35.104383+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:28:06.072280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.099032+0000","last_change":"2024-09-17T04:32:35.084585+0000","last_active":"2024-09-17T04:32:35.099032+0000","last_peered":"2024-09-17T04:32:35.099032+0000","last_clean":"2024-09-17T04:32:35.099032+0000","last_became_active":"2024-09-17T04:32:35.084264+0000","last_became_peered":"2024-09-17T04:32:35.084264+0000","last_unstale":"2024-09-17T04:32:35.099032+0000","last_undegraded":"2024-09-17T04:32:35.099032+0000","last_fullsized":"2024-09-17T04:32:35.099032+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:51.828184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.636376+0000","last_change":"2024-09-17T04:32:34.081485+0000","last_active":"2024-09-17T04:32:36.636376+0000","last_peered":"2024-09-17T04:32:36.636376+0000","last_clean":"2024-09-17T04:32:36.636376+0000","last_became_active":"2024-09-17T04:32:34.081051+0000","last_became_peered":"2024-09-17T04:32:34.081051+0000","last_unstale":"2024-09-17T04:32:36.636376+0000","last_undegraded":"2024-09-17T04:32:36.636376+0000","last_fullsized":"2024-09-17T04:32:36.636376+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:02:43.637781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.078466+0000","last_change":"2024-09-17T04:32:35.085683+0000","last_active":"2024-09-17T04:32:36.078466+0000","last_peered":"2024-09-17T04:32:36.078466+0000","last_clean":"2024-09-17T04:32:36.078466+0000","last_became_active":"2024-09-17T04:32:35.085405+0000","last_became_peered":"2024-09-17T04:32:35.085405+0000","last_unstale":"2024-09-17T04:32:36.078466+0000","last_undegraded":"2024-09-17T04:32:36.078466+0000","last_fullsized":"2024-09-17T04:32:36.078466+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:29.954123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.075495+0000","last_change":"2024-09-17T04:32:34.083756+0000","last_active":"2024-09-17T04:32:35.075495+0000","last_peered":"2024-09-17T04:32:35.075495+0000","last_clean":"2024-09-17T04:32:35.075495+0000","last_became_active":"2024-09-17T04:32:34.083169+0000","last_became_peered":"2024-09-17T04:32:34.083169+0000","last_unstale":"2024-09-17T04:32:35.075495+0000","last_undegraded":"2024-09-17T04:32:35.075495+0000","last_fullsized":"2024-09-17T04:32:35.075495+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:52:26.507084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.099967+0000","last_change":"2024-09-17T04:32:35.093537+0000","last_active":"2024-09-17T04:32:35.099967+0000","last_peered":"2024-09-17T04:32:35.099967+0000","last_clean":"2024-09-17T04:32:35.099967+0000","last_became_active":"2024-09-17T04:32:35.084065+0000","last_became_peered":"2024-09-17T04:32:35.084065+0000","last_unstale":"2024-09-17T04:32:35.099967+0000","last_undegraded":"2024-09-17T04:32:35.099967+0000","last_fullsized":"2024-09-17T04:32:35.099967+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:41.480232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.079331+0000","last_change":"2024-09-17T04:32:34.080730+0000","last_active":"2024-09-17T04:32:36.079331+0000","last_peered":"2024-09-17T04:32:36.079331+0000","last_clean":"2024-09-17T04:32:36.079331+0000","last_became_active":"2024-09-17T04:32:34.080430+0000","last_became_peered":"2024-09-17T04:32:34.080430+0000","last_unstale":"2024-09-17T04:32:36.079331+0000","last_undegraded":"2024-09-17T04:32:36.079331+0000","last_fullsized":"2024-09-17T04:32:36.079331+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:59:55.922462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.096583+0000","last_change":"2024-09-17T04:32:35.092708+0000","last_active":"2024-09-17T04:32:35.096583+0000","last_peered":"2024-09-17T04:32:35.096583+0000","last_clean":"2024-09-17T04:32:35.096583+0000","last_became_active":"2024-09-17T04:32:35.088854+0000","last_became_peered":"2024-09-17T04:32:35.088854+0000","last_unstale":"2024-09-17T04:32:35.096583+0000","last_undegraded":"2024-09-17T04:32:35.096583+0000","last_fullsized":"2024-09-17T04:32:35.096583+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:00.452058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"144'3","reported_seq":18,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.079231+0000","last_change":"2024-09-17T04:32:34.080016+0000","last_active":"2024-09-17T04:32:36.079231+0000","last_peered":"2024-09-17T04:32:36.079231+0000","last_clean":"2024-09-17T04:32:36.079231+0000","last_became_active":"2024-09-17T04:32:34.079772+0000","last_became_peered":"2024-09-17T04:32:34.079772+0000","last_unstale":"2024-09-17T04:32:36.079231+0000","last_undegraded":"2024-09-17T04:32:36.079231+0000","last_fullsized":"2024-09-17T04:32:36.079231+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:55:39.263787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.082251+0000","last_change":"2024-09-17T04:32:35.093833+0000","last_active":"2024-09-17T04:32:36.082251+0000","last_peered":"2024-09-17T04:32:36.082251+0000","last_clean":"2024-09-17T04:32:36.082251+0000","last_became_active":"2024-09-17T04:32:35.092339+0000","last_became_peered":"2024-09-17T04:32:35.092339+0000","last_unstale":"2024-09-17T04:32:36.082251+0000","last_undegraded":"2024-09-17T04:32:36.082251+0000","last_fullsized":"2024-09-17T04:32:36.082251+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:22.581126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"144'6","reported_seq":21,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.081645+0000","last_change":"2024-09-17T04:32:34.086974+0000","last_active":"2024-09-17T04:32:36.081645+0000","last_peered":"2024-09-17T04:32:36.081645+0000","last_clean":"2024-09-17T04:32:36.081645+0000","last_became_active":"2024-09-17T04:32:34.086635+0000","last_became_peered":"2024-09-17T04:32:34.086635+0000","last_unstale":"2024-09-17T04:32:36.081645+0000","last_undegraded":"2024-09-17T04:32:36.081645+0000","last_fullsized":"2024-09-17T04:32:36.081645+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:24.074500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.092605+0000","last_change":"2024-09-17T04:32:35.085770+0000","last_active":"2024-09-17T04:32:35.092605+0000","last_peered":"2024-09-17T04:32:35.092605+0000","last_clean":"2024-09-17T04:32:35.092605+0000","last_became_active":"2024-09-17T04:32:35.079222+0000","last_became_peered":"2024-09-17T04:32:35.079222+0000","last_unstale":"2024-09-17T04:32:35.092605+0000","last_undegraded":"2024-09-17T04:32:35.092605+0000","last_fullsized":"2024-09-17T04:32:35.092605+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:46:08.231777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"144'2","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.099463+0000","last_change":"2024-09-17T04:32:34.075629+0000","last_active":"2024-09-17T04:32:35.099463+0000","last_peered":"2024-09-17T04:32:35.099463+0000","last_clean":"2024-09-17T04:32:35.099463+0000","last_became_active":"2024-09-17T04:32:34.074708+0000","last_became_peered":"2024-09-17T04:32:34.074708+0000","last_unstale":"2024-09-17T04:32:35.099463+0000","last_undegraded":"2024-09-17T04:32:35.099463+0000","last_fullsized":"2024-09-17T04:32:35.099463+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:33.198633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.094576+0000","last_change":"2024-09-17T04:32:35.089783+0000","last_active":"2024-09-17T04:32:35.094576+0000","last_peered":"2024-09-17T04:32:35.094576+0000","last_clean":"2024-09-17T04:32:35.094576+0000","last_became_active":"2024-09-17T04:32:35.089401+0000","last_became_peered":"2024-09-17T04:32:35.089401+0000","last_unstale":"2024-09-17T04:32:35.094576+0000","last_undegraded":"2024-09-17T04:32:35.094576+0000","last_fullsized":"2024-09-17T04:32:35.094576+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:01:00.636968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.074199+0000","last_change":"2024-09-17T04:32:34.072836+0000","last_active":"2024-09-17T04:32:35.074199+0000","last_peered":"2024-09-17T04:32:35.074199+0000","last_clean":"2024-09-17T04:32:35.074199+0000","last_became_active":"2024-09-17T04:32:34.072208+0000","last_became_peered":"2024-09-17T04:32:34.072208+0000","last_unstale":"2024-09-17T04:32:35.074199+0000","last_undegraded":"2024-09-17T04:32:35.074199+0000","last_fullsized":"2024-09-17T04:32:35.074199+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:59.636197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.101795+0000","last_change":"2024-09-17T04:32:35.098774+0000","last_active":"2024-09-17T04:32:35.101795+0000","last_peered":"2024-09-17T04:32:35.101795+0000","last_clean":"2024-09-17T04:32:35.101795+0000","last_became_active":"2024-09-17T04:32:35.088507+0000","last_became_peered":"2024-09-17T04:32:35.088507+0000","last_unstale":"2024-09-17T04:32:35.101795+0000","last_undegraded":"2024-09-17T04:32:35.101795+0000","last_fullsized":"2024-09-17T04:32:35.101795+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:24:11.299402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.070960+0000","last_change":"2024-09-17T04:32:34.086441+0000","last_active":"2024-09-17T04:32:35.070960+0000","last_peered":"2024-09-17T04:32:35.070960+0000","last_clean":"2024-09-17T04:32:35.070960+0000","last_became_active":"2024-09-17T04:32:34.084649+0000","last_became_peered":"2024-09-17T04:32:34.084649+0000","last_unstale":"2024-09-17T04:32:35.070960+0000","last_undegraded":"2024-09-17T04:32:35.070960+0000","last_fullsized":"2024-09-17T04:32:35.070960+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:02:43.637781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080078+0000","last_change":"2024-09-17T04:32:35.086623+0000","last_active":"2024-09-17T04:32:36.080078+0000","last_peered":"2024-09-17T04:32:36.080078+0000","last_clean":"2024-09-17T04:32:36.080078+0000","last_became_active":"2024-09-17T04:32:35.086253+0000","last_became_peered":"2024-09-17T04:32:35.086253+0000","last_unstale":"2024-09-17T04:32:36.080078+0000","last_undegraded":"2024-09-17T04:32:36.080078+0000","last_fullsized":"2024-09-17T04:32:36.080078+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:48:31.157613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"144'8","reported_seq":23,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080093+0000","last_change":"2024-09-17T04:32:34.079075+0000","last_active":"2024-09-17T04:32:36.080093+0000","last_peered":"2024-09-17T04:32:36.080093+0000","last_clean":"2024-09-17T04:32:36.080093+0000","last_became_active":"2024-09-17T04:32:34.078757+0000","last_became_peered":"2024-09-17T04:32:34.078757+0000","last_unstale":"2024-09-17T04:32:36.080093+0000","last_undegraded":"2024-09-17T04:32:36.080093+0000","last_fullsized":"2024-09-17T04:32:36.080093+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:36:25.682345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.102544+0000","last_change":"2024-09-17T04:32:35.099506+0000","last_active":"2024-09-17T04:32:35.102544+0000","last_peered":"2024-09-17T04:32:35.102544+0000","last_clean":"2024-09-17T04:32:35.102544+0000","last_became_active":"2024-09-17T04:32:35.099154+0000","last_became_peered":"2024-09-17T04:32:35.099154+0000","last_unstale":"2024-09-17T04:32:35.102544+0000","last_undegraded":"2024-09-17T04:32:35.102544+0000","last_fullsized":"2024-09-17T04:32:35.102544+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:25.078355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080831+0000","last_change":"2024-09-17T04:32:34.077079+0000","last_active":"2024-09-17T04:32:36.080831+0000","last_peered":"2024-09-17T04:32:36.080831+0000","last_clean":"2024-09-17T04:32:36.080831+0000","last_became_active":"2024-09-17T04:32:34.076742+0000","last_became_peered":"2024-09-17T04:32:34.076742+0000","last_unstale":"2024-09-17T04:32:36.080831+0000","last_undegraded":"2024-09-17T04:32:36.080831+0000","last_fullsized":"2024-09-17T04:32:36.080831+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:53:02.741792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.091725+0000","last_change":"2024-09-17T04:32:35.084574+0000","last_active":"2024-09-17T04:32:35.091725+0000","last_peered":"2024-09-17T04:32:35.091725+0000","last_clean":"2024-09-17T04:32:35.091725+0000","last_became_active":"2024-09-17T04:32:35.083717+0000","last_became_peered":"2024-09-17T04:32:35.083717+0000","last_unstale":"2024-09-17T04:32:35.091725+0000","last_undegraded":"2024-09-17T04:32:35.091725+0000","last_fullsized":"2024-09-17T04:32:35.091725+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:17.467815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.081259+0000","last_change":"2024-09-17T04:32:34.075041+0000","last_active":"2024-09-17T04:32:36.081259+0000","last_peered":"2024-09-17T04:32:36.081259+0000","last_clean":"2024-09-17T04:32:36.081259+0000","last_became_active":"2024-09-17T04:32:34.074404+0000","last_became_peered":"2024-09-17T04:32:34.074404+0000","last_unstale":"2024-09-17T04:32:36.081259+0000","last_undegraded":"2024-09-17T04:32:36.081259+0000","last_fullsized":"2024-09-17T04:32:36.081259+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:57:57.608417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.102833+0000","last_change":"2024-09-17T04:32:35.086420+0000","last_active":"2024-09-17T04:32:35.102833+0000","last_peered":"2024-09-17T04:32:35.102833+0000","last_clean":"2024-09-17T04:32:35.102833+0000","last_became_active":"2024-09-17T04:32:35.086112+0000","last_became_peered":"2024-09-17T04:32:35.086112+0000","last_unstale":"2024-09-17T04:32:35.102833+0000","last_undegraded":"2024-09-17T04:32:35.102833+0000","last_fullsized":"2024-09-17T04:32:35.102833+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:11:27.736791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"144'1","reported_seq":14,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.103180+0000","last_change":"2024-09-17T04:32:34.085040+0000","last_active":"2024-09-17T04:32:35.103180+0000","last_peered":"2024-09-17T04:32:35.103180+0000","last_clean":"2024-09-17T04:32:35.103180+0000","last_became_active":"2024-09-17T04:32:34.084555+0000","last_became_peered":"2024-09-17T04:32:34.084555+0000","last_unstale":"2024-09-17T04:32:35.103180+0000","last_undegraded":"2024-09-17T04:32:35.103180+0000","last_fullsized":"2024-09-17T04:32:35.103180+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:05:45.418015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.094914+0000","last_change":"2024-09-17T04:32:35.091067+0000","last_active":"2024-09-17T04:32:35.094914+0000","last_peered":"2024-09-17T04:32:35.094914+0000","last_clean":"2024-09-17T04:32:35.094914+0000","last_became_active":"2024-09-17T04:32:35.090802+0000","last_became_peered":"2024-09-17T04:32:35.090802+0000","last_unstale":"2024-09-17T04:32:35.094914+0000","last_undegraded":"2024-09-17T04:32:35.094914+0000","last_fullsized":"2024-09-17T04:32:35.094914+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:59.596899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.072529+0000","last_change":"2024-09-17T04:32:34.077805+0000","last_active":"2024-09-17T04:32:35.072529+0000","last_peered":"2024-09-17T04:32:35.072529+0000","last_clean":"2024-09-17T04:32:35.072529+0000","last_became_active":"2024-09-17T04:32:34.076979+0000","last_became_peered":"2024-09-17T04:32:34.076979+0000","last_unstale":"2024-09-17T04:32:35.072529+0000","last_undegraded":"2024-09-17T04:32:35.072529+0000","last_fullsized":"2024-09-17T04:32:35.072529+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:47.070510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080008+0000","last_change":"2024-09-17T04:32:35.086830+0000","last_active":"2024-09-17T04:32:36.080008+0000","last_peered":"2024-09-17T04:32:36.080008+0000","last_clean":"2024-09-17T04:32:36.080008+0000","last_became_active":"2024-09-17T04:32:35.086479+0000","last_became_peered":"2024-09-17T04:32:35.086479+0000","last_unstale":"2024-09-17T04:32:36.080008+0000","last_undegraded":"2024-09-17T04:32:36.080008+0000","last_fullsized":"2024-09-17T04:32:36.080008+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:53.423214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"144'2","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.101896+0000","last_change":"2024-09-17T04:32:34.085366+0000","last_active":"2024-09-17T04:32:35.101896+0000","last_peered":"2024-09-17T04:32:35.101896+0000","last_clean":"2024-09-17T04:32:35.101896+0000","last_became_active":"2024-09-17T04:32:34.084709+0000","last_became_peered":"2024-09-17T04:32:34.084709+0000","last_unstale":"2024-09-17T04:32:35.101896+0000","last_undegraded":"2024-09-17T04:32:35.101896+0000","last_fullsized":"2024-09-17T04:32:35.101896+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:29.301232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.096782+0000","last_change":"2024-09-17T04:32:35.095661+0000","last_active":"2024-09-17T04:32:35.096782+0000","last_peered":"2024-09-17T04:32:35.096782+0000","last_clean":"2024-09-17T04:32:35.096782+0000","last_became_active":"2024-09-17T04:32:35.095387+0000","last_became_peered":"2024-09-17T04:32:35.095387+0000","last_unstale":"2024-09-17T04:32:35.096782+0000","last_undegraded":"2024-09-17T04:32:35.096782+0000","last_fullsized":"2024-09-17T04:32:35.096782+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:23:25.283146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.070249+0000","last_change":"2024-09-17T04:32:34.081342+0000","last_active":"2024-09-17T04:32:35.070249+0000","last_peered":"2024-09-17T04:32:35.070249+0000","last_clean":"2024-09-17T04:32:35.070249+0000","last_became_active":"2024-09-17T04:32:34.080204+0000","last_became_peered":"2024-09-17T04:32:34.080204+0000","last_unstale":"2024-09-17T04:32:35.070249+0000","last_undegraded":"2024-09-17T04:32:35.070249+0000","last_fullsized":"2024-09-17T04:32:35.070249+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:44:07.700336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.086334+0000","last_change":"2024-09-17T04:32:35.080905+0000","last_active":"2024-09-17T04:32:35.086334+0000","last_peered":"2024-09-17T04:32:35.086334+0000","last_clean":"2024-09-17T04:32:35.086334+0000","last_became_active":"2024-09-17T04:32:35.080463+0000","last_became_peered":"2024-09-17T04:32:35.080463+0000","last_unstale":"2024-09-17T04:32:35.086334+0000","last_undegraded":"2024-09-17T04:32:35.086334+0000","last_fullsized":"2024-09-17T04:32:35.086334+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:07:12.753089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.071174+0000","last_change":"2024-09-17T04:32:34.078179+0000","last_active":"2024-09-17T04:32:35.071174+0000","last_peered":"2024-09-17T04:32:35.071174+0000","last_clean":"2024-09-17T04:32:35.071174+0000","last_became_active":"2024-09-17T04:32:34.077856+0000","last_became_peered":"2024-09-17T04:32:34.077856+0000","last_unstale":"2024-09-17T04:32:35.071174+0000","last_undegraded":"2024-09-17T04:32:35.071174+0000","last_fullsized":"2024-09-17T04:32:35.071174+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:57:49.166987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.078630+0000","last_change":"2024-09-17T04:32:35.080413+0000","last_active":"2024-09-17T04:32:36.078630+0000","last_peered":"2024-09-17T04:32:36.078630+0000","last_clean":"2024-09-17T04:32:36.078630+0000","last_became_active":"2024-09-17T04:32:35.079997+0000","last_became_peered":"2024-09-17T04:32:35.079997+0000","last_unstale":"2024-09-17T04:32:36.078630+0000","last_undegraded":"2024-09-17T04:32:36.078630+0000","last_fullsized":"2024-09-17T04:32:36.078630+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:39:40.749807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.071279+0000","last_change":"2024-09-17T04:32:34.082227+0000","last_active":"2024-09-17T04:32:35.071279+0000","last_peered":"2024-09-17T04:32:35.071279+0000","last_clean":"2024-09-17T04:32:35.071279+0000","last_became_active":"2024-09-17T04:32:34.081636+0000","last_became_peered":"2024-09-17T04:32:34.081636+0000","last_unstale":"2024-09-17T04:32:35.071279+0000","last_undegraded":"2024-09-17T04:32:35.071279+0000","last_fullsized":"2024-09-17T04:32:35.071279+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:48.701058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.087606+0000","last_change":"2024-09-17T04:32:35.080401+0000","last_active":"2024-09-17T04:32:35.087606+0000","last_peered":"2024-09-17T04:32:35.087606+0000","last_clean":"2024-09-17T04:32:35.087606+0000","last_became_active":"2024-09-17T04:32:35.079764+0000","last_became_peered":"2024-09-17T04:32:35.079764+0000","last_unstale":"2024-09-17T04:32:35.087606+0000","last_undegraded":"2024-09-17T04:32:35.087606+0000","last_fullsized":"2024-09-17T04:32:35.087606+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:54.461711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"144'2","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.100356+0000","last_change":"2024-09-17T04:32:34.077843+0000","last_active":"2024-09-17T04:32:35.100356+0000","last_peered":"2024-09-17T04:32:35.100356+0000","last_clean":"2024-09-17T04:32:35.100356+0000","last_became_active":"2024-09-17T04:32:34.076279+0000","last_became_peered":"2024-09-17T04:32:34.076279+0000","last_unstale":"2024-09-17T04:32:35.100356+0000","last_undegraded":"2024-09-17T04:32:35.100356+0000","last_fullsized":"2024-09-17T04:32:35.100356+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:37:54.092137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.094943+0000","last_change":"2024-09-17T04:32:35.088684+0000","last_active":"2024-09-17T04:32:35.094943+0000","last_peered":"2024-09-17T04:32:35.094943+0000","last_clean":"2024-09-17T04:32:35.094943+0000","last_became_active":"2024-09-17T04:32:35.088345+0000","last_became_peered":"2024-09-17T04:32:35.088345+0000","last_unstale":"2024-09-17T04:32:35.094943+0000","last_undegraded":"2024-09-17T04:32:35.094943+0000","last_fullsized":"2024-09-17T04:32:35.094943+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:39:37.472466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.081276+0000","last_change":"2024-09-17T04:32:34.083917+0000","last_active":"2024-09-17T04:32:36.081276+0000","last_peered":"2024-09-17T04:32:36.081276+0000","last_clean":"2024-09-17T04:32:36.081276+0000","last_became_active":"2024-09-17T04:32:34.083495+0000","last_became_peered":"2024-09-17T04:32:34.083495+0000","last_unstale":"2024-09-17T04:32:36.081276+0000","last_undegraded":"2024-09-17T04:32:36.081276+0000","last_fullsized":"2024-09-17T04:32:36.081276+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:41.487197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.079038+0000","last_change":"2024-09-17T04:32:35.088206+0000","last_active":"2024-09-17T04:32:36.079038+0000","last_peered":"2024-09-17T04:32:36.079038+0000","last_clean":"2024-09-17T04:32:36.079038+0000","last_became_active":"2024-09-17T04:32:35.087850+0000","last_became_peered":"2024-09-17T04:32:35.087850+0000","last_unstale":"2024-09-17T04:32:36.079038+0000","last_undegraded":"2024-09-17T04:32:36.079038+0000","last_fullsized":"2024-09-17T04:32:36.079038+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:28:26.459412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.636399+0000","last_change":"2024-09-17T04:32:34.077316+0000","last_active":"2024-09-17T04:32:36.636399+0000","last_peered":"2024-09-17T04:32:36.636399+0000","last_clean":"2024-09-17T04:32:36.636399+0000","last_became_active":"2024-09-17T04:32:34.076801+0000","last_became_peered":"2024-09-17T04:32:34.076801+0000","last_unstale":"2024-09-17T04:32:36.636399+0000","last_undegraded":"2024-09-17T04:32:36.636399+0000","last_fullsized":"2024-09-17T04:32:36.636399+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:41.000056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.094000+0000","last_change":"2024-09-17T04:32:35.087253+0000","last_active":"2024-09-17T04:32:35.094000+0000","last_peered":"2024-09-17T04:32:35.094000+0000","last_clean":"2024-09-17T04:32:35.094000+0000","last_became_active":"2024-09-17T04:32:35.086540+0000","last_became_peered":"2024-09-17T04:32:35.086540+0000","last_unstale":"2024-09-17T04:32:35.094000+0000","last_undegraded":"2024-09-17T04:32:35.094000+0000","last_fullsized":"2024-09-17T04:32:35.094000+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:09:37.690440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.073461+0000","last_change":"2024-09-17T04:32:34.068661+0000","last_active":"2024-09-17T04:32:35.073461+0000","last_peered":"2024-09-17T04:32:35.073461+0000","last_clean":"2024-09-17T04:32:35.073461+0000","last_became_active":"2024-09-17T04:32:34.068140+0000","last_became_peered":"2024-09-17T04:32:34.068140+0000","last_unstale":"2024-09-17T04:32:35.073461+0000","last_undegraded":"2024-09-17T04:32:35.073461+0000","last_fullsized":"2024-09-17T04:32:35.073461+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:54:17.375406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.099177+0000","last_change":"2024-09-17T04:32:35.093784+0000","last_active":"2024-09-17T04:32:35.099177+0000","last_peered":"2024-09-17T04:32:35.099177+0000","last_clean":"2024-09-17T04:32:35.099177+0000","last_became_active":"2024-09-17T04:32:35.092761+0000","last_became_peered":"2024-09-17T04:32:35.092761+0000","last_unstale":"2024-09-17T04:32:35.099177+0000","last_undegraded":"2024-09-17T04:32:35.099177+0000","last_fullsized":"2024-09-17T04:32:35.099177+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:53.193952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.081777+0000","last_change":"2024-09-17T04:32:34.083476+0000","last_active":"2024-09-17T04:32:36.081777+0000","last_peered":"2024-09-17T04:32:36.081777+0000","last_clean":"2024-09-17T04:32:36.081777+0000","last_became_active":"2024-09-17T04:32:34.082708+0000","last_became_peered":"2024-09-17T04:32:34.082708+0000","last_unstale":"2024-09-17T04:32:36.081777+0000","last_undegraded":"2024-09-17T04:32:36.081777+0000","last_fullsized":"2024-09-17T04:32:36.081777+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:23.351650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.081694+0000","last_change":"2024-09-17T04:32:35.092655+0000","last_active":"2024-09-17T04:32:36.081694+0000","last_peered":"2024-09-17T04:32:36.081694+0000","last_clean":"2024-09-17T04:32:36.081694+0000","last_became_active":"2024-09-17T04:32:35.092257+0000","last_became_peered":"2024-09-17T04:32:35.092257+0000","last_unstale":"2024-09-17T04:32:36.081694+0000","last_undegraded":"2024-09-17T04:32:36.081694+0000","last_fullsized":"2024-09-17T04:32:36.081694+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:32:23.805138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.070897+0000","last_change":"2024-09-17T04:32:34.081964+0000","last_active":"2024-09-17T04:32:35.070897+0000","last_peered":"2024-09-17T04:32:35.070897+0000","last_clean":"2024-09-17T04:32:35.070897+0000","last_became_active":"2024-09-17T04:32:34.081652+0000","last_became_peered":"2024-09-17T04:32:34.081652+0000","last_unstale":"2024-09-17T04:32:35.070897+0000","last_undegraded":"2024-09-17T04:32:35.070897+0000","last_fullsized":"2024-09-17T04:32:35.070897+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:09.053631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.094989+0000","last_change":"2024-09-17T04:32:35.090858+0000","last_active":"2024-09-17T04:32:35.094989+0000","last_peered":"2024-09-17T04:32:35.094989+0000","last_clean":"2024-09-17T04:32:35.094989+0000","last_became_active":"2024-09-17T04:32:35.090544+0000","last_became_peered":"2024-09-17T04:32:35.090544+0000","last_unstale":"2024-09-17T04:32:35.094989+0000","last_undegraded":"2024-09-17T04:32:35.094989+0000","last_fullsized":"2024-09-17T04:32:35.094989+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:46.749781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.636519+0000","last_change":"2024-09-17T04:32:34.077612+0000","last_active":"2024-09-17T04:32:36.636519+0000","last_peered":"2024-09-17T04:32:36.636519+0000","last_clean":"2024-09-17T04:32:36.636519+0000","last_became_active":"2024-09-17T04:32:34.077255+0000","last_became_peered":"2024-09-17T04:32:34.077255+0000","last_unstale":"2024-09-17T04:32:36.636519+0000","last_undegraded":"2024-09-17T04:32:36.636519+0000","last_fullsized":"2024-09-17T04:32:36.636519+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:57:49.166987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.079949+0000","last_change":"2024-09-17T04:32:35.075354+0000","last_active":"2024-09-17T04:32:36.079949+0000","last_peered":"2024-09-17T04:32:36.079949+0000","last_clean":"2024-09-17T04:32:36.079949+0000","last_became_active":"2024-09-17T04:32:35.074232+0000","last_became_peered":"2024-09-17T04:32:35.074232+0000","last_unstale":"2024-09-17T04:32:36.079949+0000","last_undegraded":"2024-09-17T04:32:36.079949+0000","last_fullsized":"2024-09-17T04:32:36.079949+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:52:30.108425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080232+0000","last_change":"2024-09-17T04:32:34.085197+0000","last_active":"2024-09-17T04:32:36.080232+0000","last_peered":"2024-09-17T04:32:36.080232+0000","last_clean":"2024-09-17T04:32:36.080232+0000","last_became_active":"2024-09-17T04:32:34.084828+0000","last_became_peered":"2024-09-17T04:32:34.084828+0000","last_unstale":"2024-09-17T04:32:36.080232+0000","last_undegraded":"2024-09-17T04:32:36.080232+0000","last_fullsized":"2024-09-17T04:32:36.080232+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:24:10.295547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.087575+0000","last_change":"2024-09-17T04:32:35.080054+0000","last_active":"2024-09-17T04:32:35.087575+0000","last_peered":"2024-09-17T04:32:35.087575+0000","last_clean":"2024-09-17T04:32:35.087575+0000","last_became_active":"2024-09-17T04:32:35.079266+0000","last_became_peered":"2024-09-17T04:32:35.079266+0000","last_unstale":"2024-09-17T04:32:35.087575+0000","last_undegraded":"2024-09-17T04:32:35.087575+0000","last_fullsized":"2024-09-17T04:32:35.087575+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:09:17.649085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.082384+0000","last_change":"2024-09-17T04:32:34.087884+0000","last_active":"2024-09-17T04:32:36.082384+0000","last_peered":"2024-09-17T04:32:36.082384+0000","last_clean":"2024-09-17T04:32:36.082384+0000","last_became_active":"2024-09-17T04:32:34.086902+0000","last_became_peered":"2024-09-17T04:32:34.086902+0000","last_unstale":"2024-09-17T04:32:36.082384+0000","last_undegraded":"2024-09-17T04:32:36.082384+0000","last_fullsized":"2024-09-17T04:32:36.082384+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:00:27.382235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"76'79","reported_seq":363,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.071324+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:32:35.071324+0000","last_peered":"2024-09-17T04:32:35.071324+0000","last_clean":"2024-09-17T04:32:35.071324+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:32:35.071324+0000","last_undegraded":"2024-09-17T04:32:35.071324+0000","last_fullsized":"2024-09-17T04:32:35.071324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476844,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33552,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338288,"statfs":{"total":96636764160,"available":96602406912,"internally_reserved":0,"allocated":2416640,"data_stored":1385570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476843,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32952,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338888,"statfs":{"total":96636764160,"available":96603021312,"internally_reserved":0,"allocated":1802240,"data_stored":792870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476844,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32964,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338876,"statfs":{"total":96636764160,"available":96603009024,"internally_reserved":0,"allocated":1818624,"data_stored":799808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476844,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32968,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338872,"statfs":{"total":96636764160,"available":96603004928,"internally_reserved":0,"allocated":1814528,"data_stored":794530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542251,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32956,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338884,"statfs":{"total":96636764160,"available":96603017216,"internally_reserved":0,"allocated":1802240,"data_stored":792882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509547,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32948,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338892,"statfs":{"total":96636764160,"available":96603025408,"internally_reserved":0,"allocated":1802240,"data_stored":792938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509546,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32956,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338884,"statfs":{"total":96636764160,"available":96603017216,"internally_reserved":0,"allocated":1802240,"data_stored":792882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509546,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33532,"kb_used_data":2336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338308,"statfs":{"total":96636764160,"available":96602427392,"internally_reserved":0,"allocated":2392064,"data_stored":1383216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:32:39.496 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:32:39.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.751+0000 7fb722e00640 1 -- 172.21.15.100:0/3605630139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb71c0d2a30 msgr2=0x7fb71c0d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:39.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.751+0000 7fb722e00640 1 --2- 172.21.15.100:0/3605630139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb71c0d2a30 0x7fb71c0d2e40 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fb714005fd0 tx=0x7fb71402ab40 comp rx=0 tx=0).stop 2024-09-17T04:32:39.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.752+0000 7fb722e00640 1 -- 172.21.15.100:0/3605630139 shutdown_connections 2024-09-17T04:32:39.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.752+0000 7fb722e00640 1 --2- 172.21.15.100:0/3605630139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb71c0d2a30 0x7fb71c0d2e40 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:39.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.752+0000 7fb722e00640 1 -- 172.21.15.100:0/3605630139 >> 172.21.15.100:0/3605630139 conn(0x7fb71c09d8a0 msgr2=0x7fb71c08bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:39.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.752+0000 7fb722e00640 1 -- 172.21.15.100:0/3605630139 shutdown_connections 2024-09-17T04:32:39.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.752+0000 7fb722e00640 1 -- 172.21.15.100:0/3605630139 wait complete. 2024-09-17T04:32:39.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.754+0000 7fb722e00640 1 Processor -- start 2024-09-17T04:32:39.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.754+0000 7fb722e00640 1 -- start start 2024-09-17T04:32:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.755+0000 7fb722e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb71c0b3820 0x7fb71c0cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.755+0000 7fb722e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb71c0d2a30 0x7fb71c0c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.755+0000 7fb722e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb71c1f8c30 0x7fb71c1f9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.755+0000 7fb722e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c0d34e0 con 0x7fb71c0b3820 2024-09-17T04:32:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.755+0000 7fb722e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c0b4740 con 0x7fb71c0d2a30 2024-09-17T04:32:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.755+0000 7fb722e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb71c0b3ef0 con 0x7fb71c1f8c30 2024-09-17T04:32:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.756+0000 7fb721a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb71c0b3820 0x7fb71c0cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.756+0000 7fb721a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb71c0b3820 0x7fb71c0cab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39182/0 (socket says 172.21.15.100:39182) 2024-09-17T04:32:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.756+0000 7fb721a00640 1 -- 172.21.15.100:0/4021799117 learned_addr learned my addr 172.21.15.100:0/4021799117 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.756+0000 7fb721000640 1 --2- 172.21.15.100:0/4021799117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb71c0d2a30 0x7fb71c0c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.756+0000 7fb722400640 1 --2- 172.21.15.100:0/4021799117 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb71c1f8c30 0x7fb71c1f9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.756+0000 7fb721a00640 1 -- 172.21.15.100:0/4021799117 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb71c1f8c30 msgr2=0x7fb71c1f9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.756+0000 7fb721a00640 1 --2- 172.21.15.100:0/4021799117 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb71c1f8c30 0x7fb71c1f9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.756+0000 7fb721a00640 1 -- 172.21.15.100:0/4021799117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb71c0d2a30 msgr2=0x7fb71c0c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.756+0000 7fb721a00640 1 --2- 172.21.15.100:0/4021799117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb71c0d2a30 0x7fb71c0c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.756+0000 7fb721a00640 1 -- 172.21.15.100:0/4021799117 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71c1f9580 con 0x7fb71c0b3820 2024-09-17T04:32:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.756+0000 7fb721000640 1 --2- 172.21.15.100:0/4021799117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb71c0d2a30 0x7fb71c0c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.756+0000 7fb721a00640 1 --2- 172.21.15.100:0/4021799117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb71c0b3820 0x7fb71c0cab70 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fb71000c430 tx=0x7fb71000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.757+0000 7fb719600640 1 -- 172.21.15.100:0/4021799117 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb710017070 con 0x7fb71c0b3820 2024-09-17T04:32:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.757+0000 7fb719600640 1 -- 172.21.15.100:0/4021799117 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7100053c0 con 0x7fb71c0b3820 2024-09-17T04:32:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.757+0000 7fb722e00640 1 -- 172.21.15.100:0/4021799117 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb71c0c9b40 con 0x7fb71c0b3820 2024-09-17T04:32:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.757+0000 7fb719600640 1 -- 172.21.15.100:0/4021799117 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb710003850 con 0x7fb71c0b3820 2024-09-17T04:32:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.757+0000 7fb722e00640 1 -- 172.21.15.100:0/4021799117 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb71c0c8aa0 con 0x7fb71c0b3820 2024-09-17T04:32:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.758+0000 7fb722e00640 1 -- 172.21.15.100:0/4021799117 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb71c0b8570 con 0x7fb71c0b3820 2024-09-17T04:32:39.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.759+0000 7fb719600640 1 -- 172.21.15.100:0/4021799117 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7fb71001a070 con 0x7fb71c0b3820 2024-09-17T04:32:39.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.760+0000 7fb719600640 1 --2- 172.21.15.100:0/4021799117 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb6fc079280 0x7fb6fc07b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:39.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.760+0000 7fb719600640 1 -- 172.21.15.100:0/4021799117 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7fb71009aee0 con 0x7fb71c0b3820 2024-09-17T04:32:39.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.760+0000 7fb721000640 1 --2- 172.21.15.100:0/4021799117 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb6fc079280 0x7fb6fc07b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:39.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.761+0000 7fb721000640 1 --2- 172.21.15.100:0/4021799117 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb6fc079280 0x7fb6fc07b790 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0c7e50 tx=0x7fb708009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:39.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.765+0000 7fb719600640 1 -- 172.21.15.100:0/4021799117 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb71c0b8570 con 0x7fb71c0b3820 2024-09-17T04:32:39.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.943+0000 7fb722e00640 1 -- 172.21.15.100:0/4021799117 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb71c0d2e40 con 0x7fb6fc079280 2024-09-17T04:32:39.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.948+0000 7fb719600640 1 -- 172.21.15.100:0/4021799117 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252475 (secure 0 0 0) 0x7fb71c0d2e40 con 0x7fb6fc079280 2024-09-17T04:32:39.951 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:32:39.952 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:32:39.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.952+0000 7fb722e00640 1 -- 172.21.15.100:0/4021799117 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb6fc079280 msgr2=0x7fb6fc07b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:39.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.952+0000 7fb722e00640 1 --2- 172.21.15.100:0/4021799117 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb6fc079280 0x7fb6fc07b790 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0c7e50 tx=0x7fb708009040 comp rx=0 tx=0).stop 2024-09-17T04:32:39.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.952+0000 7fb722e00640 1 -- 172.21.15.100:0/4021799117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb71c0b3820 msgr2=0x7fb71c0cab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:39.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.952+0000 7fb722e00640 1 --2- 172.21.15.100:0/4021799117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb71c0b3820 0x7fb71c0cab70 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fb71000c430 tx=0x7fb71000c8f0 comp rx=0 tx=0).stop 2024-09-17T04:32:39.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.952+0000 7fb722e00640 1 -- 172.21.15.100:0/4021799117 shutdown_connections 2024-09-17T04:32:39.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.952+0000 7fb722e00640 1 --2- 172.21.15.100:0/4021799117 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb6fc079280 0x7fb6fc07b790 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:39.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.952+0000 7fb722e00640 1 --2- 172.21.15.100:0/4021799117 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb71c1f8c30 0x7fb71c1f9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:39.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.952+0000 7fb722e00640 1 --2- 172.21.15.100:0/4021799117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb71c0d2a30 0x7fb71c0c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:39.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.952+0000 7fb722e00640 1 --2- 172.21.15.100:0/4021799117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb71c0b3820 0x7fb71c0cab70 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:39.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.952+0000 7fb722e00640 1 -- 172.21.15.100:0/4021799117 >> 172.21.15.100:0/4021799117 conn(0x7fb71c09d8a0 msgr2=0x7fb71c0b7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:39.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.952+0000 7fb722e00640 1 -- 172.21.15.100:0/4021799117 shutdown_connections 2024-09-17T04:32:39.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:39.952+0000 7fb722e00640 1 -- 172.21.15.100:0/4021799117 wait complete. 2024-09-17T04:32:39.975 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2024-09-17T04:32:39.707121+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":157,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":125,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264828,"kb_used_data":15284,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754709892,"statfs":{"total":773094113280,"available":772822929408,"internally_reserved":0,"allocated":15650816,"data_stored":7534696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001327"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.095347+0000","last_change":"2024-09-17T04:32:35.090715+0000","last_active":"2024-09-17T04:32:35.095347+0000","last_peered":"2024-09-17T04:32:35.095347+0000","last_clean":"2024-09-17T04:32:35.095347+0000","last_became_active":"2024-09-17T04:32:35.090348+0000","last_became_peered":"2024-09-17T04:32:35.090348+0000","last_unstale":"2024-09-17T04:32:35.095347+0000","last_undegraded":"2024-09-17T04:32:35.095347+0000","last_fullsized":"2024-09-17T04:32:35.095347+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:35.901594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.636805+0000","last_change":"2024-09-17T04:32:35.089497+0000","last_active":"2024-09-17T04:32:36.636805+0000","last_peered":"2024-09-17T04:32:36.636805+0000","last_clean":"2024-09-17T04:32:36.636805+0000","last_became_active":"2024-09-17T04:32:35.089171+0000","last_became_peered":"2024-09-17T04:32:35.089171+0000","last_unstale":"2024-09-17T04:32:36.636805+0000","last_undegraded":"2024-09-17T04:32:36.636805+0000","last_fullsized":"2024-09-17T04:32:36.636805+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:08.356019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080338+0000","last_change":"2024-09-17T04:32:35.081484+0000","last_active":"2024-09-17T04:32:36.080338+0000","last_peered":"2024-09-17T04:32:36.080338+0000","last_clean":"2024-09-17T04:32:36.080338+0000","last_became_active":"2024-09-17T04:32:35.081154+0000","last_became_peered":"2024-09-17T04:32:35.081154+0000","last_unstale":"2024-09-17T04:32:36.080338+0000","last_undegraded":"2024-09-17T04:32:36.080338+0000","last_fullsized":"2024-09-17T04:32:36.080338+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:10.057486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.106158+0000","last_change":"2024-09-17T04:32:35.105304+0000","last_active":"2024-09-17T04:32:35.106158+0000","last_peered":"2024-09-17T04:32:35.106158+0000","last_clean":"2024-09-17T04:32:35.106158+0000","last_became_active":"2024-09-17T04:32:35.105046+0000","last_became_peered":"2024-09-17T04:32:35.105046+0000","last_unstale":"2024-09-17T04:32:35.106158+0000","last_undegraded":"2024-09-17T04:32:35.106158+0000","last_fullsized":"2024-09-17T04:32:35.106158+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:06:11.856341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.636646+0000","last_change":"2024-09-17T04:32:35.088118+0000","last_active":"2024-09-17T04:32:36.636646+0000","last_peered":"2024-09-17T04:32:36.636646+0000","last_clean":"2024-09-17T04:32:36.636646+0000","last_became_active":"2024-09-17T04:32:35.087805+0000","last_became_peered":"2024-09-17T04:32:35.087805+0000","last_unstale":"2024-09-17T04:32:36.636646+0000","last_undegraded":"2024-09-17T04:32:36.636646+0000","last_fullsized":"2024-09-17T04:32:36.636646+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:10.890012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.078703+0000","last_change":"2024-09-17T04:32:35.091693+0000","last_active":"2024-09-17T04:32:36.078703+0000","last_peered":"2024-09-17T04:32:36.078703+0000","last_clean":"2024-09-17T04:32:36.078703+0000","last_became_active":"2024-09-17T04:32:35.091369+0000","last_became_peered":"2024-09-17T04:32:35.091369+0000","last_unstale":"2024-09-17T04:32:36.078703+0000","last_undegraded":"2024-09-17T04:32:36.078703+0000","last_fullsized":"2024-09-17T04:32:36.078703+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:58.390151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.087953+0000","last_change":"2024-09-17T04:32:35.085803+0000","last_active":"2024-09-17T04:32:35.087953+0000","last_peered":"2024-09-17T04:32:35.087953+0000","last_clean":"2024-09-17T04:32:35.087953+0000","last_became_active":"2024-09-17T04:32:35.085478+0000","last_became_peered":"2024-09-17T04:32:35.085478+0000","last_unstale":"2024-09-17T04:32:35.087953+0000","last_undegraded":"2024-09-17T04:32:35.087953+0000","last_fullsized":"2024-09-17T04:32:35.087953+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:25:42.410141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.636450+0000","last_change":"2024-09-17T04:32:35.086333+0000","last_active":"2024-09-17T04:32:36.636450+0000","last_peered":"2024-09-17T04:32:36.636450+0000","last_clean":"2024-09-17T04:32:36.636450+0000","last_became_active":"2024-09-17T04:32:35.085194+0000","last_became_peered":"2024-09-17T04:32:35.085194+0000","last_unstale":"2024-09-17T04:32:36.636450+0000","last_undegraded":"2024-09-17T04:32:36.636450+0000","last_fullsized":"2024-09-17T04:32:36.636450+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:51:21.462803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.636957+0000","last_change":"2024-09-17T04:32:35.089949+0000","last_active":"2024-09-17T04:32:36.636957+0000","last_peered":"2024-09-17T04:32:36.636957+0000","last_clean":"2024-09-17T04:32:36.636957+0000","last_became_active":"2024-09-17T04:32:35.089004+0000","last_became_peered":"2024-09-17T04:32:35.089004+0000","last_unstale":"2024-09-17T04:32:36.636957+0000","last_undegraded":"2024-09-17T04:32:36.636957+0000","last_fullsized":"2024-09-17T04:32:36.636957+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:03:17.726639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.636880+0000","last_change":"2024-09-17T04:32:35.079425+0000","last_active":"2024-09-17T04:32:36.636880+0000","last_peered":"2024-09-17T04:32:36.636880+0000","last_clean":"2024-09-17T04:32:36.636880+0000","last_became_active":"2024-09-17T04:32:35.078944+0000","last_became_peered":"2024-09-17T04:32:35.078944+0000","last_unstale":"2024-09-17T04:32:36.636880+0000","last_undegraded":"2024-09-17T04:32:36.636880+0000","last_fullsized":"2024-09-17T04:32:36.636880+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:17:31.875959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.636710+0000","last_change":"2024-09-17T04:32:35.089585+0000","last_active":"2024-09-17T04:32:36.636710+0000","last_peered":"2024-09-17T04:32:36.636710+0000","last_clean":"2024-09-17T04:32:36.636710+0000","last_became_active":"2024-09-17T04:32:35.085301+0000","last_became_peered":"2024-09-17T04:32:35.085301+0000","last_unstale":"2024-09-17T04:32:36.636710+0000","last_undegraded":"2024-09-17T04:32:36.636710+0000","last_fullsized":"2024-09-17T04:32:36.636710+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:16.000604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.636426+0000","last_change":"2024-09-17T04:32:35.082150+0000","last_active":"2024-09-17T04:32:36.636426+0000","last_peered":"2024-09-17T04:32:36.636426+0000","last_clean":"2024-09-17T04:32:36.636426+0000","last_became_active":"2024-09-17T04:32:35.081423+0000","last_became_peered":"2024-09-17T04:32:35.081423+0000","last_unstale":"2024-09-17T04:32:36.636426+0000","last_undegraded":"2024-09-17T04:32:36.636426+0000","last_fullsized":"2024-09-17T04:32:36.636426+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:00:28.386090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080659+0000","last_change":"2024-09-17T04:32:35.083646+0000","last_active":"2024-09-17T04:32:36.080659+0000","last_peered":"2024-09-17T04:32:36.080659+0000","last_clean":"2024-09-17T04:32:36.080659+0000","last_became_active":"2024-09-17T04:32:35.082348+0000","last_became_peered":"2024-09-17T04:32:35.082348+0000","last_unstale":"2024-09-17T04:32:36.080659+0000","last_undegraded":"2024-09-17T04:32:36.080659+0000","last_fullsized":"2024-09-17T04:32:36.080659+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:57:50.170842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080474+0000","last_change":"2024-09-17T04:32:35.082355+0000","last_active":"2024-09-17T04:32:36.080474+0000","last_peered":"2024-09-17T04:32:36.080474+0000","last_clean":"2024-09-17T04:32:36.080474+0000","last_became_active":"2024-09-17T04:32:35.082043+0000","last_became_peered":"2024-09-17T04:32:35.082043+0000","last_unstale":"2024-09-17T04:32:36.080474+0000","last_undegraded":"2024-09-17T04:32:36.080474+0000","last_fullsized":"2024-09-17T04:32:36.080474+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:02:44.641636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.092762+0000","last_change":"2024-09-17T04:32:35.083470+0000","last_active":"2024-09-17T04:32:35.092762+0000","last_peered":"2024-09-17T04:32:35.092762+0000","last_clean":"2024-09-17T04:32:35.092762+0000","last_became_active":"2024-09-17T04:32:35.083166+0000","last_became_peered":"2024-09-17T04:32:35.083166+0000","last_unstale":"2024-09-17T04:32:35.092762+0000","last_undegraded":"2024-09-17T04:32:35.092762+0000","last_fullsized":"2024-09-17T04:32:35.092762+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:26:25.443388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.094963+0000","last_change":"2024-09-17T04:32:35.090551+0000","last_active":"2024-09-17T04:32:35.094963+0000","last_peered":"2024-09-17T04:32:35.094963+0000","last_clean":"2024-09-17T04:32:35.094963+0000","last_became_active":"2024-09-17T04:32:35.090216+0000","last_became_peered":"2024-09-17T04:32:35.090216+0000","last_unstale":"2024-09-17T04:32:35.094963+0000","last_undegraded":"2024-09-17T04:32:35.094963+0000","last_fullsized":"2024-09-17T04:32:35.094963+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:43:59.061916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.079373+0000","last_change":"2024-09-17T04:32:35.087309+0000","last_active":"2024-09-17T04:32:36.079373+0000","last_peered":"2024-09-17T04:32:36.079373+0000","last_clean":"2024-09-17T04:32:36.079373+0000","last_became_active":"2024-09-17T04:32:35.087000+0000","last_became_peered":"2024-09-17T04:32:35.087000+0000","last_unstale":"2024-09-17T04:32:36.079373+0000","last_undegraded":"2024-09-17T04:32:36.079373+0000","last_fullsized":"2024-09-17T04:32:36.079373+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:53:03.745647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.094032+0000","last_change":"2024-09-17T04:32:35.087977+0000","last_active":"2024-09-17T04:32:35.094032+0000","last_peered":"2024-09-17T04:32:35.094032+0000","last_clean":"2024-09-17T04:32:35.094032+0000","last_became_active":"2024-09-17T04:32:35.087710+0000","last_became_peered":"2024-09-17T04:32:35.087710+0000","last_unstale":"2024-09-17T04:32:35.094032+0000","last_undegraded":"2024-09-17T04:32:35.094032+0000","last_fullsized":"2024-09-17T04:32:35.094032+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:41.604068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.079076+0000","last_change":"2024-09-17T04:32:35.071247+0000","last_active":"2024-09-17T04:32:36.079076+0000","last_peered":"2024-09-17T04:32:36.079076+0000","last_clean":"2024-09-17T04:32:36.079076+0000","last_became_active":"2024-09-17T04:32:35.070946+0000","last_became_peered":"2024-09-17T04:32:35.070946+0000","last_unstale":"2024-09-17T04:32:36.079076+0000","last_undegraded":"2024-09-17T04:32:36.079076+0000","last_fullsized":"2024-09-17T04:32:36.079076+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:57:58.612271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080103+0000","last_change":"2024-09-17T04:32:35.082505+0000","last_active":"2024-09-17T04:32:36.080103+0000","last_peered":"2024-09-17T04:32:36.080103+0000","last_clean":"2024-09-17T04:32:36.080103+0000","last_became_active":"2024-09-17T04:32:35.080875+0000","last_became_peered":"2024-09-17T04:32:35.080875+0000","last_unstale":"2024-09-17T04:32:36.080103+0000","last_undegraded":"2024-09-17T04:32:36.080103+0000","last_fullsized":"2024-09-17T04:32:36.080103+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:28:07.076135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.079030+0000","last_change":"2024-09-17T04:32:35.081404+0000","last_active":"2024-09-17T04:32:36.079030+0000","last_peered":"2024-09-17T04:32:36.079030+0000","last_clean":"2024-09-17T04:32:36.079030+0000","last_became_active":"2024-09-17T04:32:35.080899+0000","last_became_peered":"2024-09-17T04:32:35.080899+0000","last_unstale":"2024-09-17T04:32:36.079030+0000","last_undegraded":"2024-09-17T04:32:36.079030+0000","last_fullsized":"2024-09-17T04:32:36.079030+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:53.337598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.636608+0000","last_change":"2024-09-17T04:32:35.091973+0000","last_active":"2024-09-17T04:32:36.636608+0000","last_peered":"2024-09-17T04:32:36.636608+0000","last_clean":"2024-09-17T04:32:36.636608+0000","last_became_active":"2024-09-17T04:32:35.091665+0000","last_became_peered":"2024-09-17T04:32:35.091665+0000","last_unstale":"2024-09-17T04:32:36.636608+0000","last_undegraded":"2024-09-17T04:32:36.636608+0000","last_fullsized":"2024-09-17T04:32:36.636608+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:57:58.856735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.078440+0000","last_change":"2024-09-17T04:32:35.085017+0000","last_active":"2024-09-17T04:32:36.078440+0000","last_peered":"2024-09-17T04:32:36.078440+0000","last_clean":"2024-09-17T04:32:36.078440+0000","last_became_active":"2024-09-17T04:32:35.084007+0000","last_became_peered":"2024-09-17T04:32:35.084007+0000","last_unstale":"2024-09-17T04:32:36.078440+0000","last_undegraded":"2024-09-17T04:32:36.078440+0000","last_fullsized":"2024-09-17T04:32:36.078440+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:56.637818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.636501+0000","last_change":"2024-09-17T04:32:35.088288+0000","last_active":"2024-09-17T04:32:36.636501+0000","last_peered":"2024-09-17T04:32:36.636501+0000","last_clean":"2024-09-17T04:32:36.636501+0000","last_became_active":"2024-09-17T04:32:35.086037+0000","last_became_peered":"2024-09-17T04:32:35.086037+0000","last_unstale":"2024-09-17T04:32:36.636501+0000","last_undegraded":"2024-09-17T04:32:36.636501+0000","last_fullsized":"2024-09-17T04:32:36.636501+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:04.393262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080710+0000","last_change":"2024-09-17T04:32:35.076267+0000","last_active":"2024-09-17T04:32:36.080710+0000","last_peered":"2024-09-17T04:32:36.080710+0000","last_clean":"2024-09-17T04:32:36.080710+0000","last_became_active":"2024-09-17T04:32:35.075177+0000","last_became_peered":"2024-09-17T04:32:35.075177+0000","last_unstale":"2024-09-17T04:32:36.080710+0000","last_undegraded":"2024-09-17T04:32:36.080710+0000","last_fullsized":"2024-09-17T04:32:36.080710+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:42.003911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.104483+0000","last_change":"2024-09-17T04:32:35.103352+0000","last_active":"2024-09-17T04:32:35.104483+0000","last_peered":"2024-09-17T04:32:35.104483+0000","last_clean":"2024-09-17T04:32:35.104483+0000","last_became_active":"2024-09-17T04:32:35.103116+0000","last_became_peered":"2024-09-17T04:32:35.103116+0000","last_unstale":"2024-09-17T04:32:35.104483+0000","last_undegraded":"2024-09-17T04:32:35.104483+0000","last_fullsized":"2024-09-17T04:32:35.104483+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:54:39.150606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080721+0000","last_change":"2024-09-17T04:32:35.078622+0000","last_active":"2024-09-17T04:32:36.080721+0000","last_peered":"2024-09-17T04:32:36.080721+0000","last_clean":"2024-09-17T04:32:36.080721+0000","last_became_active":"2024-09-17T04:32:35.076584+0000","last_became_peered":"2024-09-17T04:32:35.076584+0000","last_unstale":"2024-09-17T04:32:36.080721+0000","last_undegraded":"2024-09-17T04:32:36.080721+0000","last_fullsized":"2024-09-17T04:32:36.080721+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:58.781510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.104965+0000","last_change":"2024-09-17T04:32:35.102530+0000","last_active":"2024-09-17T04:32:35.104965+0000","last_peered":"2024-09-17T04:32:35.104965+0000","last_clean":"2024-09-17T04:32:35.104965+0000","last_became_active":"2024-09-17T04:32:35.096082+0000","last_became_peered":"2024-09-17T04:32:35.096082+0000","last_unstale":"2024-09-17T04:32:35.104965+0000","last_undegraded":"2024-09-17T04:32:35.104965+0000","last_fullsized":"2024-09-17T04:32:35.104965+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:24.355505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080737+0000","last_change":"2024-09-17T04:32:35.079899+0000","last_active":"2024-09-17T04:32:36.080737+0000","last_peered":"2024-09-17T04:32:36.080737+0000","last_clean":"2024-09-17T04:32:36.080737+0000","last_became_active":"2024-09-17T04:32:35.079586+0000","last_became_peered":"2024-09-17T04:32:35.079586+0000","last_unstale":"2024-09-17T04:32:36.080737+0000","last_undegraded":"2024-09-17T04:32:36.080737+0000","last_fullsized":"2024-09-17T04:32:36.080737+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:30.305087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.081622+0000","last_change":"2024-09-17T04:32:35.080098+0000","last_active":"2024-09-17T04:32:36.081622+0000","last_peered":"2024-09-17T04:32:36.081622+0000","last_clean":"2024-09-17T04:32:36.081622+0000","last_became_active":"2024-09-17T04:32:35.077456+0000","last_became_peered":"2024-09-17T04:32:35.077456+0000","last_unstale":"2024-09-17T04:32:36.081622+0000","last_undegraded":"2024-09-17T04:32:36.081622+0000","last_fullsized":"2024-09-17T04:32:36.081622+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:03:17.726639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080146+0000","last_change":"2024-09-17T04:32:35.085846+0000","last_active":"2024-09-17T04:32:36.080146+0000","last_peered":"2024-09-17T04:32:36.080146+0000","last_clean":"2024-09-17T04:32:36.080146+0000","last_became_active":"2024-09-17T04:32:35.085491+0000","last_became_peered":"2024-09-17T04:32:35.085491+0000","last_unstale":"2024-09-17T04:32:36.080146+0000","last_undegraded":"2024-09-17T04:32:36.080146+0000","last_fullsized":"2024-09-17T04:32:36.080146+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:17:31.875959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.092929+0000","last_change":"2024-09-17T04:32:35.088238+0000","last_active":"2024-09-17T04:32:35.092929+0000","last_peered":"2024-09-17T04:32:35.092929+0000","last_clean":"2024-09-17T04:32:35.092929+0000","last_became_active":"2024-09-17T04:32:35.087644+0000","last_became_peered":"2024-09-17T04:32:35.087644+0000","last_unstale":"2024-09-17T04:32:35.092929+0000","last_undegraded":"2024-09-17T04:32:35.092929+0000","last_fullsized":"2024-09-17T04:32:35.092929+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:09.506332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.104456+0000","last_change":"2024-09-17T04:32:35.100948+0000","last_active":"2024-09-17T04:32:35.104456+0000","last_peered":"2024-09-17T04:32:35.104456+0000","last_clean":"2024-09-17T04:32:35.104456+0000","last_became_active":"2024-09-17T04:32:35.100532+0000","last_became_peered":"2024-09-17T04:32:35.100532+0000","last_unstale":"2024-09-17T04:32:35.104456+0000","last_undegraded":"2024-09-17T04:32:35.104456+0000","last_fullsized":"2024-09-17T04:32:35.104456+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:15:30.142426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"144'4","reported_seq":17,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.105378+0000","last_change":"2024-09-17T04:32:34.073631+0000","last_active":"2024-09-17T04:32:35.105378+0000","last_peered":"2024-09-17T04:32:35.105378+0000","last_clean":"2024-09-17T04:32:35.105378+0000","last_became_active":"2024-09-17T04:32:34.071463+0000","last_became_peered":"2024-09-17T04:32:34.071463+0000","last_unstale":"2024-09-17T04:32:35.105378+0000","last_undegraded":"2024-09-17T04:32:35.105378+0000","last_fullsized":"2024-09-17T04:32:35.105378+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:21:37.338931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.082502+0000","last_change":"2024-09-17T04:32:35.099925+0000","last_active":"2024-09-17T04:32:36.082502+0000","last_peered":"2024-09-17T04:32:36.082502+0000","last_clean":"2024-09-17T04:32:36.082502+0000","last_became_active":"2024-09-17T04:32:35.098049+0000","last_became_peered":"2024-09-17T04:32:35.098049+0000","last_unstale":"2024-09-17T04:32:36.082502+0000","last_undegraded":"2024-09-17T04:32:36.082502+0000","last_fullsized":"2024-09-17T04:32:36.082502+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:56.877315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.082003+0000","last_change":"2024-09-17T04:32:34.089003+0000","last_active":"2024-09-17T04:32:36.082003+0000","last_peered":"2024-09-17T04:32:36.082003+0000","last_clean":"2024-09-17T04:32:36.082003+0000","last_became_active":"2024-09-17T04:32:34.088705+0000","last_became_peered":"2024-09-17T04:32:34.088705+0000","last_unstale":"2024-09-17T04:32:36.082003+0000","last_undegraded":"2024-09-17T04:32:36.082003+0000","last_fullsized":"2024-09-17T04:32:36.082003+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:06:10.852486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.078968+0000","last_change":"2024-09-17T04:32:35.086724+0000","last_active":"2024-09-17T04:32:36.078968+0000","last_peered":"2024-09-17T04:32:36.078968+0000","last_clean":"2024-09-17T04:32:36.078968+0000","last_became_active":"2024-09-17T04:32:35.086414+0000","last_became_peered":"2024-09-17T04:32:35.086414+0000","last_unstale":"2024-09-17T04:32:36.078968+0000","last_undegraded":"2024-09-17T04:32:36.078968+0000","last_fullsized":"2024-09-17T04:32:36.078968+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:30.636333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"144'2","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.100002+0000","last_change":"2024-09-17T04:32:34.082468+0000","last_active":"2024-09-17T04:32:35.100002+0000","last_peered":"2024-09-17T04:32:35.100002+0000","last_clean":"2024-09-17T04:32:35.100002+0000","last_became_active":"2024-09-17T04:32:34.082122+0000","last_became_peered":"2024-09-17T04:32:34.082122+0000","last_unstale":"2024-09-17T04:32:35.100002+0000","last_undegraded":"2024-09-17T04:32:35.100002+0000","last_fullsized":"2024-09-17T04:32:35.100002+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:41.000056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080990+0000","last_change":"2024-09-17T04:32:35.074526+0000","last_active":"2024-09-17T04:32:36.080990+0000","last_peered":"2024-09-17T04:32:36.080990+0000","last_clean":"2024-09-17T04:32:36.080990+0000","last_became_active":"2024-09-17T04:32:35.074180+0000","last_became_peered":"2024-09-17T04:32:35.074180+0000","last_unstale":"2024-09-17T04:32:36.080990+0000","last_undegraded":"2024-09-17T04:32:36.080990+0000","last_fullsized":"2024-09-17T04:32:36.080990+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:21.143564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"144'2","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.103250+0000","last_change":"2024-09-17T04:32:34.078337+0000","last_active":"2024-09-17T04:32:35.103250+0000","last_peered":"2024-09-17T04:32:35.103250+0000","last_clean":"2024-09-17T04:32:35.103250+0000","last_became_active":"2024-09-17T04:32:34.077663+0000","last_became_peered":"2024-09-17T04:32:34.077663+0000","last_unstale":"2024-09-17T04:32:35.103250+0000","last_undegraded":"2024-09-17T04:32:35.103250+0000","last_fullsized":"2024-09-17T04:32:35.103250+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:20.139709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.099081+0000","last_change":"2024-09-17T04:32:35.097272+0000","last_active":"2024-09-17T04:32:35.099081+0000","last_peered":"2024-09-17T04:32:35.099081+0000","last_clean":"2024-09-17T04:32:35.099081+0000","last_became_active":"2024-09-17T04:32:35.096085+0000","last_became_peered":"2024-09-17T04:32:35.096085+0000","last_unstale":"2024-09-17T04:32:35.099081+0000","last_undegraded":"2024-09-17T04:32:35.099081+0000","last_fullsized":"2024-09-17T04:32:35.099081+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:05.879557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080812+0000","last_change":"2024-09-17T04:32:34.089042+0000","last_active":"2024-09-17T04:32:36.080812+0000","last_peered":"2024-09-17T04:32:36.080812+0000","last_clean":"2024-09-17T04:32:36.080812+0000","last_became_active":"2024-09-17T04:32:34.088339+0000","last_became_peered":"2024-09-17T04:32:34.088339+0000","last_unstale":"2024-09-17T04:32:36.080812+0000","last_undegraded":"2024-09-17T04:32:36.080812+0000","last_fullsized":"2024-09-17T04:32:36.080812+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:52.333743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.099835+0000","last_change":"2024-09-17T04:32:35.092302+0000","last_active":"2024-09-17T04:32:35.099835+0000","last_peered":"2024-09-17T04:32:35.099835+0000","last_clean":"2024-09-17T04:32:35.099835+0000","last_became_active":"2024-09-17T04:32:35.092021+0000","last_became_peered":"2024-09-17T04:32:35.092021+0000","last_unstale":"2024-09-17T04:32:35.099835+0000","last_undegraded":"2024-09-17T04:32:35.099835+0000","last_fullsized":"2024-09-17T04:32:35.099835+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:36:26.686200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"144'1","reported_seq":14,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.104383+0000","last_change":"2024-09-17T04:32:34.085874+0000","last_active":"2024-09-17T04:32:35.104383+0000","last_peered":"2024-09-17T04:32:35.104383+0000","last_clean":"2024-09-17T04:32:35.104383+0000","last_became_active":"2024-09-17T04:32:34.085387+0000","last_became_peered":"2024-09-17T04:32:34.085387+0000","last_unstale":"2024-09-17T04:32:35.104383+0000","last_undegraded":"2024-09-17T04:32:35.104383+0000","last_fullsized":"2024-09-17T04:32:35.104383+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:28:06.072280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.099032+0000","last_change":"2024-09-17T04:32:35.084585+0000","last_active":"2024-09-17T04:32:35.099032+0000","last_peered":"2024-09-17T04:32:35.099032+0000","last_clean":"2024-09-17T04:32:35.099032+0000","last_became_active":"2024-09-17T04:32:35.084264+0000","last_became_peered":"2024-09-17T04:32:35.084264+0000","last_unstale":"2024-09-17T04:32:35.099032+0000","last_undegraded":"2024-09-17T04:32:35.099032+0000","last_fullsized":"2024-09-17T04:32:35.099032+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:51.828184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.636376+0000","last_change":"2024-09-17T04:32:34.081485+0000","last_active":"2024-09-17T04:32:36.636376+0000","last_peered":"2024-09-17T04:32:36.636376+0000","last_clean":"2024-09-17T04:32:36.636376+0000","last_became_active":"2024-09-17T04:32:34.081051+0000","last_became_peered":"2024-09-17T04:32:34.081051+0000","last_unstale":"2024-09-17T04:32:36.636376+0000","last_undegraded":"2024-09-17T04:32:36.636376+0000","last_fullsized":"2024-09-17T04:32:36.636376+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:02:43.637781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.078466+0000","last_change":"2024-09-17T04:32:35.085683+0000","last_active":"2024-09-17T04:32:36.078466+0000","last_peered":"2024-09-17T04:32:36.078466+0000","last_clean":"2024-09-17T04:32:36.078466+0000","last_became_active":"2024-09-17T04:32:35.085405+0000","last_became_peered":"2024-09-17T04:32:35.085405+0000","last_unstale":"2024-09-17T04:32:36.078466+0000","last_undegraded":"2024-09-17T04:32:36.078466+0000","last_fullsized":"2024-09-17T04:32:36.078466+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:29.954123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.075495+0000","last_change":"2024-09-17T04:32:34.083756+0000","last_active":"2024-09-17T04:32:35.075495+0000","last_peered":"2024-09-17T04:32:35.075495+0000","last_clean":"2024-09-17T04:32:35.075495+0000","last_became_active":"2024-09-17T04:32:34.083169+0000","last_became_peered":"2024-09-17T04:32:34.083169+0000","last_unstale":"2024-09-17T04:32:35.075495+0000","last_undegraded":"2024-09-17T04:32:35.075495+0000","last_fullsized":"2024-09-17T04:32:35.075495+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:52:26.507084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.099967+0000","last_change":"2024-09-17T04:32:35.093537+0000","last_active":"2024-09-17T04:32:35.099967+0000","last_peered":"2024-09-17T04:32:35.099967+0000","last_clean":"2024-09-17T04:32:35.099967+0000","last_became_active":"2024-09-17T04:32:35.084065+0000","last_became_peered":"2024-09-17T04:32:35.084065+0000","last_unstale":"2024-09-17T04:32:35.099967+0000","last_undegraded":"2024-09-17T04:32:35.099967+0000","last_fullsized":"2024-09-17T04:32:35.099967+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:41.480232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.079331+0000","last_change":"2024-09-17T04:32:34.080730+0000","last_active":"2024-09-17T04:32:36.079331+0000","last_peered":"2024-09-17T04:32:36.079331+0000","last_clean":"2024-09-17T04:32:36.079331+0000","last_became_active":"2024-09-17T04:32:34.080430+0000","last_became_peered":"2024-09-17T04:32:34.080430+0000","last_unstale":"2024-09-17T04:32:36.079331+0000","last_undegraded":"2024-09-17T04:32:36.079331+0000","last_fullsized":"2024-09-17T04:32:36.079331+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:59:55.922462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.096583+0000","last_change":"2024-09-17T04:32:35.092708+0000","last_active":"2024-09-17T04:32:35.096583+0000","last_peered":"2024-09-17T04:32:35.096583+0000","last_clean":"2024-09-17T04:32:35.096583+0000","last_became_active":"2024-09-17T04:32:35.088854+0000","last_became_peered":"2024-09-17T04:32:35.088854+0000","last_unstale":"2024-09-17T04:32:35.096583+0000","last_undegraded":"2024-09-17T04:32:35.096583+0000","last_fullsized":"2024-09-17T04:32:35.096583+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:00.452058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"144'3","reported_seq":18,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.079231+0000","last_change":"2024-09-17T04:32:34.080016+0000","last_active":"2024-09-17T04:32:36.079231+0000","last_peered":"2024-09-17T04:32:36.079231+0000","last_clean":"2024-09-17T04:32:36.079231+0000","last_became_active":"2024-09-17T04:32:34.079772+0000","last_became_peered":"2024-09-17T04:32:34.079772+0000","last_unstale":"2024-09-17T04:32:36.079231+0000","last_undegraded":"2024-09-17T04:32:36.079231+0000","last_fullsized":"2024-09-17T04:32:36.079231+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:55:39.263787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.082251+0000","last_change":"2024-09-17T04:32:35.093833+0000","last_active":"2024-09-17T04:32:36.082251+0000","last_peered":"2024-09-17T04:32:36.082251+0000","last_clean":"2024-09-17T04:32:36.082251+0000","last_became_active":"2024-09-17T04:32:35.092339+0000","last_became_peered":"2024-09-17T04:32:35.092339+0000","last_unstale":"2024-09-17T04:32:36.082251+0000","last_undegraded":"2024-09-17T04:32:36.082251+0000","last_fullsized":"2024-09-17T04:32:36.082251+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:22.581126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"144'6","reported_seq":21,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.081645+0000","last_change":"2024-09-17T04:32:34.086974+0000","last_active":"2024-09-17T04:32:36.081645+0000","last_peered":"2024-09-17T04:32:36.081645+0000","last_clean":"2024-09-17T04:32:36.081645+0000","last_became_active":"2024-09-17T04:32:34.086635+0000","last_became_peered":"2024-09-17T04:32:34.086635+0000","last_unstale":"2024-09-17T04:32:36.081645+0000","last_undegraded":"2024-09-17T04:32:36.081645+0000","last_fullsized":"2024-09-17T04:32:36.081645+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:24.074500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.092605+0000","last_change":"2024-09-17T04:32:35.085770+0000","last_active":"2024-09-17T04:32:35.092605+0000","last_peered":"2024-09-17T04:32:35.092605+0000","last_clean":"2024-09-17T04:32:35.092605+0000","last_became_active":"2024-09-17T04:32:35.079222+0000","last_became_peered":"2024-09-17T04:32:35.079222+0000","last_unstale":"2024-09-17T04:32:35.092605+0000","last_undegraded":"2024-09-17T04:32:35.092605+0000","last_fullsized":"2024-09-17T04:32:35.092605+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:46:08.231777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"144'2","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.099463+0000","last_change":"2024-09-17T04:32:34.075629+0000","last_active":"2024-09-17T04:32:35.099463+0000","last_peered":"2024-09-17T04:32:35.099463+0000","last_clean":"2024-09-17T04:32:35.099463+0000","last_became_active":"2024-09-17T04:32:34.074708+0000","last_became_peered":"2024-09-17T04:32:34.074708+0000","last_unstale":"2024-09-17T04:32:35.099463+0000","last_undegraded":"2024-09-17T04:32:35.099463+0000","last_fullsized":"2024-09-17T04:32:35.099463+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:33.198633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.094576+0000","last_change":"2024-09-17T04:32:35.089783+0000","last_active":"2024-09-17T04:32:35.094576+0000","last_peered":"2024-09-17T04:32:35.094576+0000","last_clean":"2024-09-17T04:32:35.094576+0000","last_became_active":"2024-09-17T04:32:35.089401+0000","last_became_peered":"2024-09-17T04:32:35.089401+0000","last_unstale":"2024-09-17T04:32:35.094576+0000","last_undegraded":"2024-09-17T04:32:35.094576+0000","last_fullsized":"2024-09-17T04:32:35.094576+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:01:00.636968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.074199+0000","last_change":"2024-09-17T04:32:34.072836+0000","last_active":"2024-09-17T04:32:35.074199+0000","last_peered":"2024-09-17T04:32:35.074199+0000","last_clean":"2024-09-17T04:32:35.074199+0000","last_became_active":"2024-09-17T04:32:34.072208+0000","last_became_peered":"2024-09-17T04:32:34.072208+0000","last_unstale":"2024-09-17T04:32:35.074199+0000","last_undegraded":"2024-09-17T04:32:35.074199+0000","last_fullsized":"2024-09-17T04:32:35.074199+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:59.636197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.101795+0000","last_change":"2024-09-17T04:32:35.098774+0000","last_active":"2024-09-17T04:32:35.101795+0000","last_peered":"2024-09-17T04:32:35.101795+0000","last_clean":"2024-09-17T04:32:35.101795+0000","last_became_active":"2024-09-17T04:32:35.088507+0000","last_became_peered":"2024-09-17T04:32:35.088507+0000","last_unstale":"2024-09-17T04:32:35.101795+0000","last_undegraded":"2024-09-17T04:32:35.101795+0000","last_fullsized":"2024-09-17T04:32:35.101795+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:24:11.299402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.070960+0000","last_change":"2024-09-17T04:32:34.086441+0000","last_active":"2024-09-17T04:32:35.070960+0000","last_peered":"2024-09-17T04:32:35.070960+0000","last_clean":"2024-09-17T04:32:35.070960+0000","last_became_active":"2024-09-17T04:32:34.084649+0000","last_became_peered":"2024-09-17T04:32:34.084649+0000","last_unstale":"2024-09-17T04:32:35.070960+0000","last_undegraded":"2024-09-17T04:32:35.070960+0000","last_fullsized":"2024-09-17T04:32:35.070960+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:02:43.637781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080078+0000","last_change":"2024-09-17T04:32:35.086623+0000","last_active":"2024-09-17T04:32:36.080078+0000","last_peered":"2024-09-17T04:32:36.080078+0000","last_clean":"2024-09-17T04:32:36.080078+0000","last_became_active":"2024-09-17T04:32:35.086253+0000","last_became_peered":"2024-09-17T04:32:35.086253+0000","last_unstale":"2024-09-17T04:32:36.080078+0000","last_undegraded":"2024-09-17T04:32:36.080078+0000","last_fullsized":"2024-09-17T04:32:36.080078+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:48:31.157613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"144'8","reported_seq":23,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080093+0000","last_change":"2024-09-17T04:32:34.079075+0000","last_active":"2024-09-17T04:32:36.080093+0000","last_peered":"2024-09-17T04:32:36.080093+0000","last_clean":"2024-09-17T04:32:36.080093+0000","last_became_active":"2024-09-17T04:32:34.078757+0000","last_became_peered":"2024-09-17T04:32:34.078757+0000","last_unstale":"2024-09-17T04:32:36.080093+0000","last_undegraded":"2024-09-17T04:32:36.080093+0000","last_fullsized":"2024-09-17T04:32:36.080093+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:36:25.682345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.102544+0000","last_change":"2024-09-17T04:32:35.099506+0000","last_active":"2024-09-17T04:32:35.102544+0000","last_peered":"2024-09-17T04:32:35.102544+0000","last_clean":"2024-09-17T04:32:35.102544+0000","last_became_active":"2024-09-17T04:32:35.099154+0000","last_became_peered":"2024-09-17T04:32:35.099154+0000","last_unstale":"2024-09-17T04:32:35.102544+0000","last_undegraded":"2024-09-17T04:32:35.102544+0000","last_fullsized":"2024-09-17T04:32:35.102544+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:25.078355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080831+0000","last_change":"2024-09-17T04:32:34.077079+0000","last_active":"2024-09-17T04:32:36.080831+0000","last_peered":"2024-09-17T04:32:36.080831+0000","last_clean":"2024-09-17T04:32:36.080831+0000","last_became_active":"2024-09-17T04:32:34.076742+0000","last_became_peered":"2024-09-17T04:32:34.076742+0000","last_unstale":"2024-09-17T04:32:36.080831+0000","last_undegraded":"2024-09-17T04:32:36.080831+0000","last_fullsized":"2024-09-17T04:32:36.080831+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:53:02.741792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.091725+0000","last_change":"2024-09-17T04:32:35.084574+0000","last_active":"2024-09-17T04:32:35.091725+0000","last_peered":"2024-09-17T04:32:35.091725+0000","last_clean":"2024-09-17T04:32:35.091725+0000","last_became_active":"2024-09-17T04:32:35.083717+0000","last_became_peered":"2024-09-17T04:32:35.083717+0000","last_unstale":"2024-09-17T04:32:35.091725+0000","last_undegraded":"2024-09-17T04:32:35.091725+0000","last_fullsized":"2024-09-17T04:32:35.091725+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:17.467815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.081259+0000","last_change":"2024-09-17T04:32:34.075041+0000","last_active":"2024-09-17T04:32:36.081259+0000","last_peered":"2024-09-17T04:32:36.081259+0000","last_clean":"2024-09-17T04:32:36.081259+0000","last_became_active":"2024-09-17T04:32:34.074404+0000","last_became_peered":"2024-09-17T04:32:34.074404+0000","last_unstale":"2024-09-17T04:32:36.081259+0000","last_undegraded":"2024-09-17T04:32:36.081259+0000","last_fullsized":"2024-09-17T04:32:36.081259+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:57:57.608417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.102833+0000","last_change":"2024-09-17T04:32:35.086420+0000","last_active":"2024-09-17T04:32:35.102833+0000","last_peered":"2024-09-17T04:32:35.102833+0000","last_clean":"2024-09-17T04:32:35.102833+0000","last_became_active":"2024-09-17T04:32:35.086112+0000","last_became_peered":"2024-09-17T04:32:35.086112+0000","last_unstale":"2024-09-17T04:32:35.102833+0000","last_undegraded":"2024-09-17T04:32:35.102833+0000","last_fullsized":"2024-09-17T04:32:35.102833+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:11:27.736791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"144'1","reported_seq":14,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.103180+0000","last_change":"2024-09-17T04:32:34.085040+0000","last_active":"2024-09-17T04:32:35.103180+0000","last_peered":"2024-09-17T04:32:35.103180+0000","last_clean":"2024-09-17T04:32:35.103180+0000","last_became_active":"2024-09-17T04:32:34.084555+0000","last_became_peered":"2024-09-17T04:32:34.084555+0000","last_unstale":"2024-09-17T04:32:35.103180+0000","last_undegraded":"2024-09-17T04:32:35.103180+0000","last_fullsized":"2024-09-17T04:32:35.103180+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:05:45.418015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.094914+0000","last_change":"2024-09-17T04:32:35.091067+0000","last_active":"2024-09-17T04:32:35.094914+0000","last_peered":"2024-09-17T04:32:35.094914+0000","last_clean":"2024-09-17T04:32:35.094914+0000","last_became_active":"2024-09-17T04:32:35.090802+0000","last_became_peered":"2024-09-17T04:32:35.090802+0000","last_unstale":"2024-09-17T04:32:35.094914+0000","last_undegraded":"2024-09-17T04:32:35.094914+0000","last_fullsized":"2024-09-17T04:32:35.094914+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:59.596899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.072529+0000","last_change":"2024-09-17T04:32:34.077805+0000","last_active":"2024-09-17T04:32:35.072529+0000","last_peered":"2024-09-17T04:32:35.072529+0000","last_clean":"2024-09-17T04:32:35.072529+0000","last_became_active":"2024-09-17T04:32:34.076979+0000","last_became_peered":"2024-09-17T04:32:34.076979+0000","last_unstale":"2024-09-17T04:32:35.072529+0000","last_undegraded":"2024-09-17T04:32:35.072529+0000","last_fullsized":"2024-09-17T04:32:35.072529+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:47.070510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080008+0000","last_change":"2024-09-17T04:32:35.086830+0000","last_active":"2024-09-17T04:32:36.080008+0000","last_peered":"2024-09-17T04:32:36.080008+0000","last_clean":"2024-09-17T04:32:36.080008+0000","last_became_active":"2024-09-17T04:32:35.086479+0000","last_became_peered":"2024-09-17T04:32:35.086479+0000","last_unstale":"2024-09-17T04:32:36.080008+0000","last_undegraded":"2024-09-17T04:32:36.080008+0000","last_fullsized":"2024-09-17T04:32:36.080008+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:53.423214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"144'2","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.101896+0000","last_change":"2024-09-17T04:32:34.085366+0000","last_active":"2024-09-17T04:32:35.101896+0000","last_peered":"2024-09-17T04:32:35.101896+0000","last_clean":"2024-09-17T04:32:35.101896+0000","last_became_active":"2024-09-17T04:32:34.084709+0000","last_became_peered":"2024-09-17T04:32:34.084709+0000","last_unstale":"2024-09-17T04:32:35.101896+0000","last_undegraded":"2024-09-17T04:32:35.101896+0000","last_fullsized":"2024-09-17T04:32:35.101896+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:29.301232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.096782+0000","last_change":"2024-09-17T04:32:35.095661+0000","last_active":"2024-09-17T04:32:35.096782+0000","last_peered":"2024-09-17T04:32:35.096782+0000","last_clean":"2024-09-17T04:32:35.096782+0000","last_became_active":"2024-09-17T04:32:35.095387+0000","last_became_peered":"2024-09-17T04:32:35.095387+0000","last_unstale":"2024-09-17T04:32:35.096782+0000","last_undegraded":"2024-09-17T04:32:35.096782+0000","last_fullsized":"2024-09-17T04:32:35.096782+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:23:25.283146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.070249+0000","last_change":"2024-09-17T04:32:34.081342+0000","last_active":"2024-09-17T04:32:35.070249+0000","last_peered":"2024-09-17T04:32:35.070249+0000","last_clean":"2024-09-17T04:32:35.070249+0000","last_became_active":"2024-09-17T04:32:34.080204+0000","last_became_peered":"2024-09-17T04:32:34.080204+0000","last_unstale":"2024-09-17T04:32:35.070249+0000","last_undegraded":"2024-09-17T04:32:35.070249+0000","last_fullsized":"2024-09-17T04:32:35.070249+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:44:07.700336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.086334+0000","last_change":"2024-09-17T04:32:35.080905+0000","last_active":"2024-09-17T04:32:35.086334+0000","last_peered":"2024-09-17T04:32:35.086334+0000","last_clean":"2024-09-17T04:32:35.086334+0000","last_became_active":"2024-09-17T04:32:35.080463+0000","last_became_peered":"2024-09-17T04:32:35.080463+0000","last_unstale":"2024-09-17T04:32:35.086334+0000","last_undegraded":"2024-09-17T04:32:35.086334+0000","last_fullsized":"2024-09-17T04:32:35.086334+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:07:12.753089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.071174+0000","last_change":"2024-09-17T04:32:34.078179+0000","last_active":"2024-09-17T04:32:35.071174+0000","last_peered":"2024-09-17T04:32:35.071174+0000","last_clean":"2024-09-17T04:32:35.071174+0000","last_became_active":"2024-09-17T04:32:34.077856+0000","last_became_peered":"2024-09-17T04:32:34.077856+0000","last_unstale":"2024-09-17T04:32:35.071174+0000","last_undegraded":"2024-09-17T04:32:35.071174+0000","last_fullsized":"2024-09-17T04:32:35.071174+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:57:49.166987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.078630+0000","last_change":"2024-09-17T04:32:35.080413+0000","last_active":"2024-09-17T04:32:36.078630+0000","last_peered":"2024-09-17T04:32:36.078630+0000","last_clean":"2024-09-17T04:32:36.078630+0000","last_became_active":"2024-09-17T04:32:35.079997+0000","last_became_peered":"2024-09-17T04:32:35.079997+0000","last_unstale":"2024-09-17T04:32:36.078630+0000","last_undegraded":"2024-09-17T04:32:36.078630+0000","last_fullsized":"2024-09-17T04:32:36.078630+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:39:40.749807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.071279+0000","last_change":"2024-09-17T04:32:34.082227+0000","last_active":"2024-09-17T04:32:35.071279+0000","last_peered":"2024-09-17T04:32:35.071279+0000","last_clean":"2024-09-17T04:32:35.071279+0000","last_became_active":"2024-09-17T04:32:34.081636+0000","last_became_peered":"2024-09-17T04:32:34.081636+0000","last_unstale":"2024-09-17T04:32:35.071279+0000","last_undegraded":"2024-09-17T04:32:35.071279+0000","last_fullsized":"2024-09-17T04:32:35.071279+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:48.701058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.087606+0000","last_change":"2024-09-17T04:32:35.080401+0000","last_active":"2024-09-17T04:32:35.087606+0000","last_peered":"2024-09-17T04:32:35.087606+0000","last_clean":"2024-09-17T04:32:35.087606+0000","last_became_active":"2024-09-17T04:32:35.079764+0000","last_became_peered":"2024-09-17T04:32:35.079764+0000","last_unstale":"2024-09-17T04:32:35.087606+0000","last_undegraded":"2024-09-17T04:32:35.087606+0000","last_fullsized":"2024-09-17T04:32:35.087606+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:54.461711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"144'2","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.100356+0000","last_change":"2024-09-17T04:32:34.077843+0000","last_active":"2024-09-17T04:32:35.100356+0000","last_peered":"2024-09-17T04:32:35.100356+0000","last_clean":"2024-09-17T04:32:35.100356+0000","last_became_active":"2024-09-17T04:32:34.076279+0000","last_became_peered":"2024-09-17T04:32:34.076279+0000","last_unstale":"2024-09-17T04:32:35.100356+0000","last_undegraded":"2024-09-17T04:32:35.100356+0000","last_fullsized":"2024-09-17T04:32:35.100356+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:37:54.092137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.094943+0000","last_change":"2024-09-17T04:32:35.088684+0000","last_active":"2024-09-17T04:32:35.094943+0000","last_peered":"2024-09-17T04:32:35.094943+0000","last_clean":"2024-09-17T04:32:35.094943+0000","last_became_active":"2024-09-17T04:32:35.088345+0000","last_became_peered":"2024-09-17T04:32:35.088345+0000","last_unstale":"2024-09-17T04:32:35.094943+0000","last_undegraded":"2024-09-17T04:32:35.094943+0000","last_fullsized":"2024-09-17T04:32:35.094943+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:39:37.472466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.081276+0000","last_change":"2024-09-17T04:32:34.083917+0000","last_active":"2024-09-17T04:32:36.081276+0000","last_peered":"2024-09-17T04:32:36.081276+0000","last_clean":"2024-09-17T04:32:36.081276+0000","last_became_active":"2024-09-17T04:32:34.083495+0000","last_became_peered":"2024-09-17T04:32:34.083495+0000","last_unstale":"2024-09-17T04:32:36.081276+0000","last_undegraded":"2024-09-17T04:32:36.081276+0000","last_fullsized":"2024-09-17T04:32:36.081276+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:41.487197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.079038+0000","last_change":"2024-09-17T04:32:35.088206+0000","last_active":"2024-09-17T04:32:36.079038+0000","last_peered":"2024-09-17T04:32:36.079038+0000","last_clean":"2024-09-17T04:32:36.079038+0000","last_became_active":"2024-09-17T04:32:35.087850+0000","last_became_peered":"2024-09-17T04:32:35.087850+0000","last_unstale":"2024-09-17T04:32:36.079038+0000","last_undegraded":"2024-09-17T04:32:36.079038+0000","last_fullsized":"2024-09-17T04:32:36.079038+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:28:26.459412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.636399+0000","last_change":"2024-09-17T04:32:34.077316+0000","last_active":"2024-09-17T04:32:36.636399+0000","last_peered":"2024-09-17T04:32:36.636399+0000","last_clean":"2024-09-17T04:32:36.636399+0000","last_became_active":"2024-09-17T04:32:34.076801+0000","last_became_peered":"2024-09-17T04:32:34.076801+0000","last_unstale":"2024-09-17T04:32:36.636399+0000","last_undegraded":"2024-09-17T04:32:36.636399+0000","last_fullsized":"2024-09-17T04:32:36.636399+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:41.000056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.094000+0000","last_change":"2024-09-17T04:32:35.087253+0000","last_active":"2024-09-17T04:32:35.094000+0000","last_peered":"2024-09-17T04:32:35.094000+0000","last_clean":"2024-09-17T04:32:35.094000+0000","last_became_active":"2024-09-17T04:32:35.086540+0000","last_became_peered":"2024-09-17T04:32:35.086540+0000","last_unstale":"2024-09-17T04:32:35.094000+0000","last_undegraded":"2024-09-17T04:32:35.094000+0000","last_fullsized":"2024-09-17T04:32:35.094000+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:09:37.690440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.073461+0000","last_change":"2024-09-17T04:32:34.068661+0000","last_active":"2024-09-17T04:32:35.073461+0000","last_peered":"2024-09-17T04:32:35.073461+0000","last_clean":"2024-09-17T04:32:35.073461+0000","last_became_active":"2024-09-17T04:32:34.068140+0000","last_became_peered":"2024-09-17T04:32:34.068140+0000","last_unstale":"2024-09-17T04:32:35.073461+0000","last_undegraded":"2024-09-17T04:32:35.073461+0000","last_fullsized":"2024-09-17T04:32:35.073461+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:54:17.375406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.099177+0000","last_change":"2024-09-17T04:32:35.093784+0000","last_active":"2024-09-17T04:32:35.099177+0000","last_peered":"2024-09-17T04:32:35.099177+0000","last_clean":"2024-09-17T04:32:35.099177+0000","last_became_active":"2024-09-17T04:32:35.092761+0000","last_became_peered":"2024-09-17T04:32:35.092761+0000","last_unstale":"2024-09-17T04:32:35.099177+0000","last_undegraded":"2024-09-17T04:32:35.099177+0000","last_fullsized":"2024-09-17T04:32:35.099177+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:53.193952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.081777+0000","last_change":"2024-09-17T04:32:34.083476+0000","last_active":"2024-09-17T04:32:36.081777+0000","last_peered":"2024-09-17T04:32:36.081777+0000","last_clean":"2024-09-17T04:32:36.081777+0000","last_became_active":"2024-09-17T04:32:34.082708+0000","last_became_peered":"2024-09-17T04:32:34.082708+0000","last_unstale":"2024-09-17T04:32:36.081777+0000","last_undegraded":"2024-09-17T04:32:36.081777+0000","last_fullsized":"2024-09-17T04:32:36.081777+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:23.351650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.081694+0000","last_change":"2024-09-17T04:32:35.092655+0000","last_active":"2024-09-17T04:32:36.081694+0000","last_peered":"2024-09-17T04:32:36.081694+0000","last_clean":"2024-09-17T04:32:36.081694+0000","last_became_active":"2024-09-17T04:32:35.092257+0000","last_became_peered":"2024-09-17T04:32:35.092257+0000","last_unstale":"2024-09-17T04:32:36.081694+0000","last_undegraded":"2024-09-17T04:32:36.081694+0000","last_fullsized":"2024-09-17T04:32:36.081694+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:32:23.805138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.070897+0000","last_change":"2024-09-17T04:32:34.081964+0000","last_active":"2024-09-17T04:32:35.070897+0000","last_peered":"2024-09-17T04:32:35.070897+0000","last_clean":"2024-09-17T04:32:35.070897+0000","last_became_active":"2024-09-17T04:32:34.081652+0000","last_became_peered":"2024-09-17T04:32:34.081652+0000","last_unstale":"2024-09-17T04:32:35.070897+0000","last_undegraded":"2024-09-17T04:32:35.070897+0000","last_fullsized":"2024-09-17T04:32:35.070897+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:09.053631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.094989+0000","last_change":"2024-09-17T04:32:35.090858+0000","last_active":"2024-09-17T04:32:35.094989+0000","last_peered":"2024-09-17T04:32:35.094989+0000","last_clean":"2024-09-17T04:32:35.094989+0000","last_became_active":"2024-09-17T04:32:35.090544+0000","last_became_peered":"2024-09-17T04:32:35.090544+0000","last_unstale":"2024-09-17T04:32:35.094989+0000","last_undegraded":"2024-09-17T04:32:35.094989+0000","last_fullsized":"2024-09-17T04:32:35.094989+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:46.749781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.636519+0000","last_change":"2024-09-17T04:32:34.077612+0000","last_active":"2024-09-17T04:32:36.636519+0000","last_peered":"2024-09-17T04:32:36.636519+0000","last_clean":"2024-09-17T04:32:36.636519+0000","last_became_active":"2024-09-17T04:32:34.077255+0000","last_became_peered":"2024-09-17T04:32:34.077255+0000","last_unstale":"2024-09-17T04:32:36.636519+0000","last_undegraded":"2024-09-17T04:32:36.636519+0000","last_fullsized":"2024-09-17T04:32:36.636519+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:57:49.166987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.079949+0000","last_change":"2024-09-17T04:32:35.075354+0000","last_active":"2024-09-17T04:32:36.079949+0000","last_peered":"2024-09-17T04:32:36.079949+0000","last_clean":"2024-09-17T04:32:36.079949+0000","last_became_active":"2024-09-17T04:32:35.074232+0000","last_became_peered":"2024-09-17T04:32:35.074232+0000","last_unstale":"2024-09-17T04:32:36.079949+0000","last_undegraded":"2024-09-17T04:32:36.079949+0000","last_fullsized":"2024-09-17T04:32:36.079949+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:52:30.108425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.080232+0000","last_change":"2024-09-17T04:32:34.085197+0000","last_active":"2024-09-17T04:32:36.080232+0000","last_peered":"2024-09-17T04:32:36.080232+0000","last_clean":"2024-09-17T04:32:36.080232+0000","last_became_active":"2024-09-17T04:32:34.084828+0000","last_became_peered":"2024-09-17T04:32:34.084828+0000","last_unstale":"2024-09-17T04:32:36.080232+0000","last_undegraded":"2024-09-17T04:32:36.080232+0000","last_fullsized":"2024-09-17T04:32:36.080232+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:24:10.295547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.087575+0000","last_change":"2024-09-17T04:32:35.080054+0000","last_active":"2024-09-17T04:32:35.087575+0000","last_peered":"2024-09-17T04:32:35.087575+0000","last_clean":"2024-09-17T04:32:35.087575+0000","last_became_active":"2024-09-17T04:32:35.079266+0000","last_became_peered":"2024-09-17T04:32:35.079266+0000","last_unstale":"2024-09-17T04:32:35.087575+0000","last_undegraded":"2024-09-17T04:32:35.087575+0000","last_fullsized":"2024-09-17T04:32:35.087575+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:34.050314+0000","last_clean_scrub_stamp":"2024-09-17T04:32:34.050314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:09:17.649085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T04:32:36.082384+0000","last_change":"2024-09-17T04:32:34.087884+0000","last_active":"2024-09-17T04:32:36.082384+0000","last_peered":"2024-09-17T04:32:36.082384+0000","last_clean":"2024-09-17T04:32:36.082384+0000","last_became_active":"2024-09-17T04:32:34.086902+0000","last_became_peered":"2024-09-17T04:32:34.086902+0000","last_unstale":"2024-09-17T04:32:36.082384+0000","last_undegraded":"2024-09-17T04:32:36.082384+0000","last_fullsized":"2024-09-17T04:32:36.082384+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:32:33.046459+0000","last_clean_scrub_stamp":"2024-09-17T04:32:33.046459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:00:27.382235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"76'79","reported_seq":363,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-17T04:32:35.071324+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:32:35.071324+0000","last_peered":"2024-09-17T04:32:35.071324+0000","last_clean":"2024-09-17T04:32:35.071324+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:32:35.071324+0000","last_undegraded":"2024-09-17T04:32:35.071324+0000","last_fullsized":"2024-09-17T04:32:35.071324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476844,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33552,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338288,"statfs":{"total":96636764160,"available":96602406912,"internally_reserved":0,"allocated":2416640,"data_stored":1385570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476843,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32952,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338888,"statfs":{"total":96636764160,"available":96603021312,"internally_reserved":0,"allocated":1802240,"data_stored":792870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476844,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32964,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338876,"statfs":{"total":96636764160,"available":96603009024,"internally_reserved":0,"allocated":1818624,"data_stored":799808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476844,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32968,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338872,"statfs":{"total":96636764160,"available":96603004928,"internally_reserved":0,"allocated":1814528,"data_stored":794530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542251,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32956,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338884,"statfs":{"total":96636764160,"available":96603017216,"internally_reserved":0,"allocated":1802240,"data_stored":792882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509547,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32948,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338892,"statfs":{"total":96636764160,"available":96603025408,"internally_reserved":0,"allocated":1802240,"data_stored":792938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509546,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32956,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338884,"statfs":{"total":96636764160,"available":96603017216,"internally_reserved":0,"allocated":1802240,"data_stored":792882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509546,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33532,"kb_used_data":2336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338308,"statfs":{"total":96636764160,"available":96602427392,"internally_reserved":0,"allocated":2392064,"data_stored":1383216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:32:39.980 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T04:32:39.981 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T04:32:40.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.242+0000 7f9d4d600640 1 -- 172.21.15.100:0/1838315268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d480d0ee0 msgr2=0x7f9d480cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:40.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.242+0000 7f9d4d600640 1 --2- 172.21.15.100:0/1838315268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d480d0ee0 0x7f9d480cef70 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f9d30005fd0 tx=0x7f9d3002a5d0 comp rx=0 tx=0).stop 2024-09-17T04:32:40.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.242+0000 7f9d4d600640 1 -- 172.21.15.100:0/1838315268 shutdown_connections 2024-09-17T04:32:40.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.242+0000 7f9d4d600640 1 --2- 172.21.15.100:0/1838315268 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d480b6b80 0x7f9d480cf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:40.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.242+0000 7f9d4d600640 1 --2- 172.21.15.100:0/1838315268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d480d0ee0 0x7f9d480cef70 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:40.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.242+0000 7f9d4d600640 1 -- 172.21.15.100:0/1838315268 >> 172.21.15.100:0/1838315268 conn(0x7f9d480b2ff0 msgr2=0x7f9d480b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:40.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.242+0000 7f9d4d600640 1 -- 172.21.15.100:0/1838315268 shutdown_connections 2024-09-17T04:32:40.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.242+0000 7f9d4d600640 1 -- 172.21.15.100:0/1838315268 wait complete. 2024-09-17T04:32:40.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.244+0000 7f9d4d600640 1 Processor -- start 2024-09-17T04:32:40.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.245+0000 7f9d4d600640 1 -- start start 2024-09-17T04:32:40.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.246+0000 7f9d4d600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d480b6b80 0x7f9d480cb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:40.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.246+0000 7f9d4d600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d480d0ee0 0x7f9d4814d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:40.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.246+0000 7f9d4d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d4815ee80 0x7f9d481505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:40.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.246+0000 7f9d4d600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9d480c7fc0 con 0x7f9d4815ee80 2024-09-17T04:32:40.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.246+0000 7f9d4d600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9d480c7e40 con 0x7f9d480b6b80 2024-09-17T04:32:40.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.246+0000 7f9d4d600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9d480c8140 con 0x7f9d480d0ee0 2024-09-17T04:32:40.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.246+0000 7f9d47400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d4815ee80 0x7f9d481505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:40.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.246+0000 7f9d47400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d4815ee80 0x7f9d481505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39208/0 (socket says 172.21.15.100:39208) 2024-09-17T04:32:40.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.246+0000 7f9d47400640 1 -- 172.21.15.100:0/2327768346 learned_addr learned my addr 172.21.15.100:0/2327768346 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:40.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.247+0000 7f9d3fe00640 1 --2- 172.21.15.100:0/2327768346 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d480d0ee0 0x7f9d4814d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:40.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.247+0000 7f9d46a00640 1 --2- 172.21.15.100:0/2327768346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d480b6b80 0x7f9d480cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:40.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.247+0000 7f9d47400640 1 -- 172.21.15.100:0/2327768346 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d480d0ee0 msgr2=0x7f9d4814d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:40.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.247+0000 7f9d47400640 1 --2- 172.21.15.100:0/2327768346 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d480d0ee0 0x7f9d4814d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:40.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.247+0000 7f9d47400640 1 -- 172.21.15.100:0/2327768346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d480b6b80 msgr2=0x7f9d480cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:40.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.247+0000 7f9d47400640 1 --2- 172.21.15.100:0/2327768346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d480b6b80 0x7f9d480cb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:40.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.247+0000 7f9d47400640 1 -- 172.21.15.100:0/2327768346 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d480af940 con 0x7f9d4815ee80 2024-09-17T04:32:40.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.247+0000 7f9d46a00640 1 --2- 172.21.15.100:0/2327768346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d480b6b80 0x7f9d480cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:32:40.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.247+0000 7f9d47400640 1 --2- 172.21.15.100:0/2327768346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d4815ee80 0x7f9d481505e0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f9d4000e960 tx=0x7f9d4000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:40.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.247+0000 7f9d3ea00640 1 -- 172.21.15.100:0/2327768346 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d40004d20 con 0x7f9d4815ee80 2024-09-17T04:32:40.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.247+0000 7f9d3ea00640 1 -- 172.21.15.100:0/2327768346 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d4000cac0 con 0x7f9d4815ee80 2024-09-17T04:32:40.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.247+0000 7f9d4d600640 1 -- 172.21.15.100:0/2327768346 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d480afc20 con 0x7f9d4815ee80 2024-09-17T04:32:40.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.247+0000 7f9d3ea00640 1 -- 172.21.15.100:0/2327768346 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d40005020 con 0x7f9d4815ee80 2024-09-17T04:32:40.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.247+0000 7f9d4d600640 1 -- 172.21.15.100:0/2327768346 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d480b01e0 con 0x7f9d4815ee80 2024-09-17T04:32:40.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.248+0000 7f9d4d600640 1 -- 172.21.15.100:0/2327768346 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d480cfc70 con 0x7f9d4815ee80 2024-09-17T04:32:40.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.250+0000 7f9d3ea00640 1 -- 172.21.15.100:0/2327768346 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f9d400051c0 con 0x7f9d4815ee80 2024-09-17T04:32:40.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.251+0000 7f9d3ea00640 1 --2- 172.21.15.100:0/2327768346 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9d24079250 0x7f9d2407b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:40.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.251+0000 7f9d3ea00640 1 -- 172.21.15.100:0/2327768346 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7f9d4009c3a0 con 0x7f9d4815ee80 2024-09-17T04:32:40.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.251+0000 7f9d46a00640 1 --2- 172.21.15.100:0/2327768346 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9d24079250 0x7f9d2407b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:40.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.251+0000 7f9d46a00640 1 --2- 172.21.15.100:0/2327768346 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9d24079250 0x7f9d2407b760 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9d380016a0 tx=0x7f9d38009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:40.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.255+0000 7f9d3ea00640 1 -- 172.21.15.100:0/2327768346 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9d40016030 con 0x7f9d4815ee80 2024-09-17T04:32:40.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.503+0000 7f9d4d600640 1 -- 172.21.15.100:0/2327768346 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f9d480af6d0 con 0x7f9d4815ee80 2024-09-17T04:32:40.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.507+0000 7f9d3ea00640 1 -- 172.21.15.100:0/2327768346 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) ==== 190+0+147 (secure 0 0 0) 0x7f9d40063d50 con 0x7f9d4815ee80 2024-09-17T04:32:40.507 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-09-17T04:32:40.508 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAsBOlmfYg9IBAABP1uRKGOugrA47VxwRLkjA== 2024-09-17T04:32:40.508 INFO:teuthology.orchestra.run.smithi100.stdout: caps mds = "allow" 2024-09-17T04:32:40.508 INFO:teuthology.orchestra.run.smithi100.stdout: caps mon = "allow r" 2024-09-17T04:32:40.508 INFO:teuthology.orchestra.run.smithi100.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T04:32:40.508 INFO:teuthology.orchestra.run.smithi100.stderr:updated caps for client.0 2024-09-17T04:32:40.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.509+0000 7f9d4d600640 1 -- 172.21.15.100:0/2327768346 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9d24079250 msgr2=0x7f9d2407b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:40.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.509+0000 7f9d4d600640 1 --2- 172.21.15.100:0/2327768346 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9d24079250 0x7f9d2407b760 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9d380016a0 tx=0x7f9d38009040 comp rx=0 tx=0).stop 2024-09-17T04:32:40.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.509+0000 7f9d4d600640 1 -- 172.21.15.100:0/2327768346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d4815ee80 msgr2=0x7f9d481505e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:40.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.509+0000 7f9d4d600640 1 --2- 172.21.15.100:0/2327768346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d4815ee80 0x7f9d481505e0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f9d4000e960 tx=0x7f9d4000ee20 comp rx=0 tx=0).stop 2024-09-17T04:32:40.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.510+0000 7f9d4d600640 1 -- 172.21.15.100:0/2327768346 shutdown_connections 2024-09-17T04:32:40.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.510+0000 7f9d4d600640 1 --2- 172.21.15.100:0/2327768346 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9d24079250 0x7f9d2407b760 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:40.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.510+0000 7f9d4d600640 1 --2- 172.21.15.100:0/2327768346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d4815ee80 0x7f9d481505e0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:40.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.510+0000 7f9d4d600640 1 --2- 172.21.15.100:0/2327768346 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d480d0ee0 0x7f9d4814d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:40.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.510+0000 7f9d4d600640 1 --2- 172.21.15.100:0/2327768346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d480b6b80 0x7f9d480cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:40.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.510+0000 7f9d4d600640 1 -- 172.21.15.100:0/2327768346 >> 172.21.15.100:0/2327768346 conn(0x7f9d480b2ff0 msgr2=0x7f9d480c76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:40.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.510+0000 7f9d4d600640 1 -- 172.21.15.100:0/2327768346 shutdown_connections 2024-09-17T04:32:40.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.510+0000 7f9d4d600640 1 -- 172.21.15.100:0/2327768346 wait complete. 2024-09-17T04:32:40.537 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:32:40.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.756+0000 7f4ad5e00640 1 -- 172.21.15.100:0/2903338049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ab4003a30 msgr2=0x7f4ab4003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:40.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.756+0000 7f4ad5e00640 1 --2- 172.21.15.100:0/2903338049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ab4003a30 0x7f4ab4003ea0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009ed0 tx=0x7f4ab4005830 comp rx=0 tx=0).stop 2024-09-17T04:32:40.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.756+0000 7f4acf400640 1 -- 172.21.15.100:0/2903338049 reap_dead start 2024-09-17T04:32:40.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.756+0000 7f4ad5e00640 1 -- 172.21.15.100:0/2903338049 shutdown_connections 2024-09-17T04:32:40.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.756+0000 7f4ad5e00640 1 -- 172.21.15.100:0/2903338049 >> 172.21.15.100:0/2903338049 conn(0x7f4ad006d730 msgr2=0x7f4ad00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:40.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.756+0000 7f4ad5e00640 1 -- 172.21.15.100:0/2903338049 shutdown_connections 2024-09-17T04:32:40.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.757+0000 7f4ad5e00640 1 -- 172.21.15.100:0/2903338049 wait complete. 2024-09-17T04:32:40.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.759+0000 7f4ad5e00640 1 Processor -- start 2024-09-17T04:32:40.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.759+0000 7f4ad5e00640 1 -- start start 2024-09-17T04:32:40.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.760+0000 7f4ad5e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ab4003a30 0x7f4ad01f28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:40.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.760+0000 7f4ad5e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ad015b2e0 0x7f4ad015b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:40.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.760+0000 7f4ad5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ad009d830 0x7f4ad009dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:40.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.760+0000 7f4ad5e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad00b41f0 con 0x7f4ad009d830 2024-09-17T04:32:40.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.760+0000 7f4ad5e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad00b4070 con 0x7f4ad015b2e0 2024-09-17T04:32:40.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.760+0000 7f4ad5e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4ad00b4370 con 0x7f4ab4003a30 2024-09-17T04:32:40.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.761+0000 7f4acf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ad009d830 0x7f4ad009dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:40.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.761+0000 7f4acf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ad009d830 0x7f4ad009dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39222/0 (socket says 172.21.15.100:39222) 2024-09-17T04:32:40.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.761+0000 7f4acf400640 1 -- 172.21.15.100:0/3482497642 learned_addr learned my addr 172.21.15.100:0/3482497642 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:40.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.761+0000 7f4acea00640 1 --2- 172.21.15.100:0/3482497642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ab4003a30 0x7f4ad01f28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:40.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.761+0000 7f4ace000640 1 --2- 172.21.15.100:0/3482497642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ad015b2e0 0x7f4ad015b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:40.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.761+0000 7f4acf400640 1 -- 172.21.15.100:0/3482497642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ab4003a30 msgr2=0x7f4ad01f28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:40.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.761+0000 7f4acf400640 1 --2- 172.21.15.100:0/3482497642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ab4003a30 0x7f4ad01f28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:40.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.761+0000 7f4acf400640 1 -- 172.21.15.100:0/3482497642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ad015b2e0 msgr2=0x7f4ad015b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:40.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.761+0000 7f4acf400640 1 --2- 172.21.15.100:0/3482497642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ad015b2e0 0x7f4ad015b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:40.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.761+0000 7f4acf400640 1 -- 172.21.15.100:0/3482497642 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad01f2df0 con 0x7f4ad009d830 2024-09-17T04:32:40.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.761+0000 7f4acf400640 1 --2- 172.21.15.100:0/3482497642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ad009d830 0x7f4ad009dc40 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0075250 tx=0x7f4ac8001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:40.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.762+0000 7f4ac7400640 1 -- 172.21.15.100:0/3482497642 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8015070 con 0x7f4ad009d830 2024-09-17T04:32:40.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.762+0000 7f4ac7400640 1 -- 172.21.15.100:0/3482497642 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ac8011650 con 0x7f4ad009d830 2024-09-17T04:32:40.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.762+0000 7f4ad5e00640 1 -- 172.21.15.100:0/3482497642 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad01f5580 con 0x7f4ad009d830 2024-09-17T04:32:40.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.762+0000 7f4ac7400640 1 -- 172.21.15.100:0/3482497642 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8011aa0 con 0x7f4ad009d830 2024-09-17T04:32:40.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.762+0000 7f4ad5e00640 1 -- 172.21.15.100:0/3482497642 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad01f5b40 con 0x7f4ad009d830 2024-09-17T04:32:40.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.763+0000 7f4ad5e00640 1 -- 172.21.15.100:0/3482497642 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ad00b3960 con 0x7f4ad009d830 2024-09-17T04:32:40.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.764+0000 7f4ac7400640 1 -- 172.21.15.100:0/3482497642 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f4ac8018070 con 0x7f4ad009d830 2024-09-17T04:32:40.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.765+0000 7f4ac7400640 1 --2- 172.21.15.100:0/3482497642 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4ab0079250 0x7f4ab007b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:40.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.765+0000 7f4ac7400640 1 -- 172.21.15.100:0/3482497642 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7f4ac809c320 con 0x7f4ad009d830 2024-09-17T04:32:40.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.765+0000 7f4acea00640 1 --2- 172.21.15.100:0/3482497642 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4ab0079250 0x7f4ab007b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:40.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.766+0000 7f4acea00640 1 --2- 172.21.15.100:0/3482497642 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4ab0079250 0x7f4ab007b760 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0035000 tx=0x7f4ac002d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:40.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.769+0000 7f4ac7400640 1 -- 172.21.15.100:0/3482497642 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4ac8063680 con 0x7f4ad009d830 2024-09-17T04:32:40.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.989+0000 7f4ad5e00640 1 -- 172.21.15.100:0/3482497642 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ad00d0f70 con 0x7f4ad009d830 2024-09-17T04:32:40.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.990+0000 7f4ac7400640 1 -- 172.21.15.100:0/3482497642 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+5241 (secure 0 0 0) 0x7f4ac80680b0 con 0x7f4ad009d830 2024-09-17T04:32:40.991 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:32:40.991 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":101,"btime":"2024-09-17T04:32:38:084767+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7028,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1529630991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1529630991},{"type":"v1","addr":"172.21.15.184:6801","nonce":1529630991}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7906,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1516666057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1516666057},{"type":"v1","addr":"172.21.15.184:6835","nonce":1516666057}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2252725120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2252725120},{"type":"v1","addr":"172.21.15.100:6833","nonce":2252725120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:32:34.557571+0000","modified":"2024-09-17T04:32:37.568034+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8427":{"gid":8427,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/2425159512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2425159512},{"type":"v1","addr":"172.21.15.100:6835","nonce":2425159512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8427,"qdb_cluster":[8427]},"id":11}]} 2024-09-17T04:32:40.991 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 101 2024-09-17T04:32:40.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.993+0000 7f4ad5e00640 1 -- 172.21.15.100:0/3482497642 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4ab0079250 msgr2=0x7f4ab007b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:40.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.993+0000 7f4ad5e00640 1 --2- 172.21.15.100:0/3482497642 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4ab0079250 0x7f4ab007b760 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0035000 tx=0x7f4ac002d040 comp rx=0 tx=0).stop 2024-09-17T04:32:40.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.993+0000 7f4ad5e00640 1 -- 172.21.15.100:0/3482497642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ad009d830 msgr2=0x7f4ad009dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:40.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.993+0000 7f4ad5e00640 1 --2- 172.21.15.100:0/3482497642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ad009d830 0x7f4ad009dc40 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0075250 tx=0x7f4ac8001630 comp rx=0 tx=0).stop 2024-09-17T04:32:40.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.993+0000 7f4ad5e00640 1 -- 172.21.15.100:0/3482497642 shutdown_connections 2024-09-17T04:32:40.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.993+0000 7f4ad5e00640 1 --2- 172.21.15.100:0/3482497642 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4ab0079250 0x7f4ab007b760 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:40.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.993+0000 7f4ad5e00640 1 --2- 172.21.15.100:0/3482497642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ad009d830 0x7f4ad009dc40 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:40.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.993+0000 7f4ad5e00640 1 --2- 172.21.15.100:0/3482497642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ad015b2e0 0x7f4ad015b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:40.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.993+0000 7f4ad5e00640 1 --2- 172.21.15.100:0/3482497642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ab4003a30 0x7f4ad01f28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:40.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.993+0000 7f4ad5e00640 1 -- 172.21.15.100:0/3482497642 >> 172.21.15.100:0/3482497642 conn(0x7f4ad006d730 msgr2=0x7f4ad00cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:40.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.993+0000 7f4ad5e00640 1 -- 172.21.15.100:0/3482497642 shutdown_connections 2024-09-17T04:32:40.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:40.993+0000 7f4ad5e00640 1 -- 172.21.15.100:0/3482497642 wait complete. 2024-09-17T04:32:41.417 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 101, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T04:32:34.557571+0000', 'modified': '2024-09-17T04:32:37.568034+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8427}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8427': {'gid': 8427, 'name': 'a', 'rank': 0, 'incarnation': 97, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6835/2425159512', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6834', 'nonce': 2425159512}, {'type': 'v1', 'addr': '172.21.15.100:6835', 'nonce': 2425159512}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8427, 'qdb_cluster': [8427]} 2024-09-17T04:32:41.418 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T04:32:41.418 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-17T04:32:41.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.639+0000 7fa887e00640 1 -- 172.21.15.100:0/3566348264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8880ad9c0 msgr2=0x7fa8880af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:41.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.639+0000 7fa887e00640 1 --2- 172.21.15.100:0/3566348264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8880ad9c0 0x7fa8880af170 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fa88806e790 tx=0x7fa87802a560 comp rx=0 tx=0).stop 2024-09-17T04:32:41.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.639+0000 7fa887e00640 1 -- 172.21.15.100:0/3566348264 shutdown_connections 2024-09-17T04:32:41.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.639+0000 7fa887e00640 1 --2- 172.21.15.100:0/3566348264 >> v2:172.21.15.184:3300/0 conn(0x7fa8880b2f60 0x7fa88815a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.639+0000 7fa887e00640 1 --2- 172.21.15.100:0/3566348264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8880ad9c0 0x7fa8880af170 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.639+0000 7fa887e00640 1 -- 172.21.15.100:0/3566348264 >> 172.21.15.100:0/3566348264 conn(0x7fa8880b1e40 msgr2=0x7fa8880b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:41.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.640+0000 7fa887e00640 1 -- 172.21.15.100:0/3566348264 shutdown_connections 2024-09-17T04:32:41.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.640+0000 7fa887e00640 1 -- 172.21.15.100:0/3566348264 wait complete. 2024-09-17T04:32:41.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.642+0000 7fa887e00640 1 Processor -- start 2024-09-17T04:32:41.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.642+0000 7fa887e00640 1 -- start start 2024-09-17T04:32:41.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.643+0000 7fa887e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8880ad9c0 0x7fa888154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:41.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.643+0000 7fa887e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8880b2f60 0x7fa888154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:41.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.643+0000 7fa887e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa88815d670 0x7fa88814a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:41.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.643+0000 7fa887e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa88806f290 con 0x7fa88815d670 2024-09-17T04:32:41.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.643+0000 7fa887e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa888070d50 con 0x7fa8880b2f60 2024-09-17T04:32:41.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.643+0000 7fa887e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa88806f410 con 0x7fa8880ad9c0 2024-09-17T04:32:41.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.644+0000 7fa887400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa88815d670 0x7fa88814a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:41.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.644+0000 7fa887400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa88815d670 0x7fa88814a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39248/0 (socket says 172.21.15.100:39248) 2024-09-17T04:32:41.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.644+0000 7fa887400640 1 -- 172.21.15.100:0/1745048342 learned_addr learned my addr 172.21.15.100:0/1745048342 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:41.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.644+0000 7fa886a00640 1 --2- 172.21.15.100:0/1745048342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8880ad9c0 0x7fa888154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:41.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.644+0000 7fa87e000640 1 --2- 172.21.15.100:0/1745048342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8880b2f60 0x7fa888154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:41.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.644+0000 7fa887400640 1 -- 172.21.15.100:0/1745048342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8880ad9c0 msgr2=0x7fa888154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:41.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.644+0000 7fa887400640 1 --2- 172.21.15.100:0/1745048342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8880ad9c0 0x7fa888154a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.644+0000 7fa887400640 1 -- 172.21.15.100:0/1745048342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8880b2f60 msgr2=0x7fa888154f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:41.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.644+0000 7fa887400640 1 --2- 172.21.15.100:0/1745048342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8880b2f60 0x7fa888154f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.644+0000 7fa887400640 1 -- 172.21.15.100:0/1745048342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88814d2e0 con 0x7fa88815d670 2024-09-17T04:32:41.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.645+0000 7fa887400640 1 --2- 172.21.15.100:0/1745048342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa88815d670 0x7fa88814a2c0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fa88000c6b0 tx=0x7fa88000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:41.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.645+0000 7fa87f400640 1 -- 172.21.15.100:0/1745048342 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa880003b70 con 0x7fa88815d670 2024-09-17T04:32:41.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.645+0000 7fa87f400640 1 -- 172.21.15.100:0/1745048342 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa880003d10 con 0x7fa88815d670 2024-09-17T04:32:41.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.645+0000 7fa887e00640 1 -- 172.21.15.100:0/1745048342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa88814d5c0 con 0x7fa88815d670 2024-09-17T04:32:41.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.645+0000 7fa87f400640 1 -- 172.21.15.100:0/1745048342 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa880005330 con 0x7fa88815d670 2024-09-17T04:32:41.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.645+0000 7fa887e00640 1 -- 172.21.15.100:0/1745048342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa88814db80 con 0x7fa88815d670 2024-09-17T04:32:41.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.647+0000 7fa87f400640 1 -- 172.21.15.100:0/1745048342 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7fa8800054d0 con 0x7fa88815d670 2024-09-17T04:32:41.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.648+0000 7fa87f400640 1 --2- 172.21.15.100:0/1745048342 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa864079250 0x7fa86407b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:41.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.649+0000 7fa87f400640 1 -- 172.21.15.100:0/1745048342 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7fa88009b5f0 con 0x7fa88815d670 2024-09-17T04:32:41.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.649+0000 7fa886a00640 1 --2- 172.21.15.100:0/1745048342 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa864079250 0x7fa86407b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:41.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.649+0000 7fa894c86740 1 Processor -- start 2024-09-17T04:32:41.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.649+0000 7fa894c86740 1 -- start start 2024-09-17T04:32:41.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.649+0000 7fa886a00640 1 --2- 172.21.15.100:0/1745048342 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa864079250 0x7fa86407b760 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa878033000 tx=0x7fa878004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:41.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.650+0000 7fa894c86740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614b7737270 0x5614b7b07b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:41.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.650+0000 7fa894c86740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614b770e3b0 0x5614b7b080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:41.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.650+0000 7fa894c86740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5614b7716590 0x5614b7ad8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:41.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.650+0000 7fa894c86740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5614b7a61430 con 0x5614b7737270 2024-09-17T04:32:41.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.650+0000 7fa894c86740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5614b7a515d0 con 0x5614b770e3b0 2024-09-17T04:32:41.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.650+0000 7fa894c86740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5614b7a40a00 con 0x5614b7716590 2024-09-17T04:32:41.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.651+0000 7fa886a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614b7737270 0x5614b7b07b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:41.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.651+0000 7fa886a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614b7737270 0x5614b7b07b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39262/0 (socket says 172.21.15.100:39262) 2024-09-17T04:32:41.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.651+0000 7fa886a00640 1 -- 172.21.15.100:0/2684587604 learned_addr learned my addr 172.21.15.100:0/2684587604 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:41.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.651+0000 7fa887400640 1 --2- 172.21.15.100:0/2684587604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5614b7716590 0x5614b7ad8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:41.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.651+0000 7fa87e000640 1 --2- 172.21.15.100:0/2684587604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614b770e3b0 0x5614b7b080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:41.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.651+0000 7fa886a00640 1 -- 172.21.15.100:0/2684587604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5614b7716590 msgr2=0x5614b7ad8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:32:41.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.651+0000 7fa886a00640 1 --2- 172.21.15.100:0/2684587604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5614b7716590 0x5614b7ad8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.651+0000 7fa886a00640 1 -- 172.21.15.100:0/2684587604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614b770e3b0 msgr2=0x5614b7b080a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:32:41.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.651+0000 7fa886a00640 1 --2- 172.21.15.100:0/2684587604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614b770e3b0 0x5614b7b080a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.651+0000 7fa886a00640 1 -- 172.21.15.100:0/2684587604 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614b7a09520 con 0x5614b7737270 2024-09-17T04:32:41.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.651+0000 7fa87e000640 1 --2- 172.21.15.100:0/2684587604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614b770e3b0 0x5614b7b080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:41.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.651+0000 7fa886a00640 1 --2- 172.21.15.100:0/2684587604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614b7737270 0x5614b7b07b60 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fa878000c00 tx=0x7fa87803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e5ae9870bfef87 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:41.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.652+0000 7fa877400640 1 -- 172.21.15.100:0/2684587604 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878035040 con 0x5614b7737270 2024-09-17T04:32:41.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.652+0000 7fa877400640 1 -- 172.21.15.100:0/2684587604 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa87804e070 con 0x5614b7737270 2024-09-17T04:32:41.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.652+0000 7fa877400640 1 -- 172.21.15.100:0/2684587604 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878003c00 con 0x5614b7737270 2024-09-17T04:32:41.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.652+0000 7fa894c86740 1 -- 172.21.15.100:0/2684587604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614b7737270 msgr2=0x5614b7b07b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:41.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.652+0000 7fa894c86740 1 --2- 172.21.15.100:0/2684587604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614b7737270 0x5614b7b07b60 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fa878000c00 tx=0x7fa87803d000 comp rx=0 tx=0).stop 2024-09-17T04:32:41.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.653+0000 7fa894c86740 1 -- 172.21.15.100:0/2684587604 shutdown_connections 2024-09-17T04:32:41.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.653+0000 7fa894c86740 1 --2- 172.21.15.100:0/2684587604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5614b7716590 0x5614b7ad8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.653+0000 7fa894c86740 1 --2- 172.21.15.100:0/2684587604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614b770e3b0 0x5614b7b080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.653+0000 7fa894c86740 1 --2- 172.21.15.100:0/2684587604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614b7737270 0x5614b7b07b60 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.653+0000 7fa894c86740 1 -- 172.21.15.100:0/2684587604 >> 172.21.15.100:0/2684587604 conn(0x5614b772b3c0 msgr2=0x5614b7addb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:41.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.653+0000 7fa894c86740 1 -- 172.21.15.100:0/2684587604 shutdown_connections 2024-09-17T04:32:41.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.653+0000 7fa894c86740 1 -- 172.21.15.100:0/2684587604 wait complete. 2024-09-17T04:32:41.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.654+0000 7fa894c86740 1 -- start start 2024-09-17T04:32:41.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.654+0000 7fa894c86740 1 Processor -- start 2024-09-17T04:32:41.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.655+0000 7fa894c86740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614b770e3b0 0x5614b7b1d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:41.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.655+0000 7fa894c86740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5614b7716590 0x5614b7b1cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:41.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.655+0000 7fa894c86740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614b7737270 0x5614b7b32d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:41.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.655+0000 7fa894c86740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5614b79e7330 con 0x5614b7737270 2024-09-17T04:32:41.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.655+0000 7fa894c86740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5614b7a44810 con 0x5614b770e3b0 2024-09-17T04:32:41.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.655+0000 7fa894c86740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5614b77b1620 con 0x5614b7716590 2024-09-17T04:32:41.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.655+0000 7fa894c86740 10 client.0 _ll_register_callbacks cb 0x5614b79ec420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:32:41.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.656+0000 7fa887400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614b7737270 0x5614b7b32d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:41.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.656+0000 7fa887400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614b7737270 0x5614b7b32d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39264/0 (socket says 172.21.15.100:39264) 2024-09-17T04:32:41.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.656+0000 7fa887400640 1 -- 172.21.15.100:0/2083062643 learned_addr learned my addr 172.21.15.100:0/2083062643 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:41.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.656+0000 7fa886a00640 1 --2- 172.21.15.100:0/2083062643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614b770e3b0 0x5614b7b1d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:41.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.656+0000 7fa87e000640 1 --2- 172.21.15.100:0/2083062643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5614b7716590 0x5614b7b1cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:41.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.656+0000 7fa887400640 1 -- 172.21.15.100:0/2083062643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5614b7716590 msgr2=0x5614b7b1cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:32:41.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.656+0000 7fa887400640 1 --2- 172.21.15.100:0/2083062643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5614b7716590 0x5614b7b1cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.656+0000 7fa887400640 1 -- 172.21.15.100:0/2083062643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614b770e3b0 msgr2=0x5614b7b1d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:32:41.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.656+0000 7fa887400640 1 --2- 172.21.15.100:0/2083062643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614b770e3b0 0x5614b7b1d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.656+0000 7fa887400640 1 -- 172.21.15.100:0/2083062643 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614b77a2850 con 0x5614b7737270 2024-09-17T04:32:41.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.656+0000 7fa887400640 1 --2- 172.21.15.100:0/2083062643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614b7737270 0x5614b7b32d50 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fa88005fc40 tx=0x7fa8800625a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25be252db5fdb1aa server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:41.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.656+0000 7fa86be00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-17T04:32:41.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.657+0000 7fa86be00640 1 -- 172.21.15.100:0/2083062643 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa880067070 con 0x5614b7737270 2024-09-17T04:32:41.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.657+0000 7fa86be00640 1 -- 172.21.15.100:0/2083062643 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7fa88005e320 con 0x5614b7737270 2024-09-17T04:32:41.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.657+0000 7fa894c86740 1 -- 172.21.15.100:0/2083062643 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa87804e070 con 0x5614b7737270 2024-09-17T04:32:41.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.657+0000 7fa86be00640 1 -- 172.21.15.100:0/2083062643 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7fa88006c2b0 con 0x5614b7737270 2024-09-17T04:32:41.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.657+0000 7fa894c86740 10 client.8541 fetch_fsmap learned FSMap version 101 2024-09-17T04:32:41.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.657+0000 7fa894c86740 1 -- 172.21.15.100:0/2083062643 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=2+}) -- 0x5614b78196b0 con 0x5614b7737270 2024-09-17T04:32:41.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.657+0000 7fa86be00640 1 -- 172.21.15.100:0/2083062643 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa87804e070 con 0x5614b7737270 2024-09-17T04:32:41.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.657+0000 7fa86be00640 1 -- 172.21.15.100:0/2083062643 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 101) ==== 3502+0+0 (secure 0 0 0) 0x7fa88005e520 con 0x5614b7737270 2024-09-17T04:32:41.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.657+0000 7fa894c86740 10 client.8541 fetch_fsmap finished waiting for FSMap version 101 2024-09-17T04:32:41.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.657+0000 7fa894c86740 10 client.8541 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T04:32:41.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.657+0000 7fa894c86740 20 client.8541 populate_metadata read hostname 'smithi100' 2024-09-17T04:32:41.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.657+0000 7fa894c86740 1 --2- 172.21.15.100:0/2083062643 >> [v2:172.21.15.100:6834/2425159512,v1:172.21.15.100:6835/2425159512] conn(0x5614b76f5e40 0x5614b7b3f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:41.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.657+0000 7fa894c86740 4 client.8541 mds_command: new command op to 8427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T04:32:41.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.657+0000 7fa894c86740 1 -- 172.21.15.100:0/2083062643 --> [v2:172.21.15.100:6834/2425159512,v1:172.21.15.100:6835/2425159512] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614b7a53520 con 0x5614b76f5e40 2024-09-17T04:32:41.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.658+0000 7fa886a00640 1 --2- 172.21.15.100:0/2083062643 >> [v2:172.21.15.100:6834/2425159512,v1:172.21.15.100:6835/2425159512] conn(0x5614b76f5e40 0x5614b7b3f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:41.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.658+0000 7fa886a00640 1 --2- 172.21.15.100:0/2083062643 >> [v2:172.21.15.100:6834/2425159512,v1:172.21.15.100:6835/2425159512] conn(0x5614b76f5e40 0x5614b7b3f6a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e402293903bbd17e server_cookie=da5868b1db24c9e9 in_seq=0 out_seq=0 2024-09-17T04:32:41.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.658+0000 7fa86be00640 10 client.8541 ms_handle_connect on v2:172.21.15.100:6834/2425159512 2024-09-17T04:32:41.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.659+0000 7fa86be00640 1 -- 172.21.15.100:0/2083062643 <== mds.0 v2:172.21.15.100:6834/2425159512 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5614b7a53520 con 0x5614b76f5e40 2024-09-17T04:32:41.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.659+0000 7fa86be00640 10 client.8541 handle_command_reply: tid=0 2024-09-17T04:32:41.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.659+0000 7fa894c86740 1 client.8541 shutdown 2024-09-17T04:32:41.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.659+0000 7fa894c86740 1 -- 172.21.15.100:0/2083062643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614b7737270 msgr2=0x5614b7b32d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:41.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.659+0000 7fa894c86740 1 --2- 172.21.15.100:0/2083062643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614b7737270 0x5614b7b32d50 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fa88005fc40 tx=0x7fa8800625a0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.659+0000 7fa894c86740 1 -- 172.21.15.100:0/2083062643 shutdown_connections 2024-09-17T04:32:41.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.659+0000 7fa894c86740 1 --2- 172.21.15.100:0/2083062643 >> [v2:172.21.15.100:6834/2425159512,v1:172.21.15.100:6835/2425159512] conn(0x5614b76f5e40 0x5614b7b3f6a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.659+0000 7fa894c86740 1 --2- 172.21.15.100:0/2083062643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614b7737270 0x5614b7b32d50 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.659+0000 7fa894c86740 1 --2- 172.21.15.100:0/2083062643 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5614b7716590 0x5614b7b1cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.659+0000 7fa894c86740 1 --2- 172.21.15.100:0/2083062643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614b770e3b0 0x5614b7b1d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.659+0000 7fa86be00640 0 client.8541 ms_handle_reset on v2:172.21.15.100:6834/2425159512 2024-09-17T04:32:41.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.659+0000 7fa894c86740 1 -- 172.21.15.100:0/2083062643 >> 172.21.15.100:0/2083062643 conn(0x5614b772b3c0 msgr2=0x5614b7b25630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:41.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.660+0000 7fa894c86740 1 -- 172.21.15.100:0/2083062643 shutdown_connections 2024-09-17T04:32:41.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.660+0000 7fa894c86740 1 -- 172.21.15.100:0/2083062643 wait complete. 2024-09-17T04:32:41.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.660+0000 7fa894c86740 20 client.8541 trim_cache size 0 max 16384 2024-09-17T04:32:41.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.679+0000 7fa894c86740 1 Processor -- start 2024-09-17T04:32:41.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.679+0000 7fa894c86740 1 -- start start 2024-09-17T04:32:41.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.681+0000 7fa894c86740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614b770e3b0 0x5614b7b1e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:41.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.681+0000 7fa894c86740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614b7716590 0x5614b7b1e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:41.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.681+0000 7fa894c86740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5614b7737270 0x5614b7afd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:41.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.681+0000 7fa894c86740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa888201030 con 0x5614b7716590 2024-09-17T04:32:41.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.681+0000 7fa894c86740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa8881f9220 con 0x5614b770e3b0 2024-09-17T04:32:41.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.681+0000 7fa894c86740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa888200a80 con 0x5614b7737270 2024-09-17T04:32:41.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.681+0000 7fa87e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614b7716590 0x5614b7b1e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:41.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.681+0000 7fa87e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614b7716590 0x5614b7b1e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39272/0 (socket says 172.21.15.100:39272) 2024-09-17T04:32:41.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.681+0000 7fa87e000640 1 -- 172.21.15.100:0/4209925347 learned_addr learned my addr 172.21.15.100:0/4209925347 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:41.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.681+0000 7fa887400640 1 --2- 172.21.15.100:0/4209925347 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5614b7737270 0x5614b7afd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:41.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.681+0000 7fa886a00640 1 --2- 172.21.15.100:0/4209925347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614b770e3b0 0x5614b7b1e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:41.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.682+0000 7fa87e000640 1 -- 172.21.15.100:0/4209925347 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5614b7737270 msgr2=0x5614b7afd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:32:41.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.682+0000 7fa87e000640 1 --2- 172.21.15.100:0/4209925347 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5614b7737270 0x5614b7afd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.682+0000 7fa87e000640 1 -- 172.21.15.100:0/4209925347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614b770e3b0 msgr2=0x5614b7b1e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:32:41.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.682+0000 7fa87e000640 1 --2- 172.21.15.100:0/4209925347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614b770e3b0 0x5614b7b1e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.682+0000 7fa87e000640 1 -- 172.21.15.100:0/4209925347 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614b7b1d2a0 con 0x5614b7716590 2024-09-17T04:32:41.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.682+0000 7fa87e000640 1 --2- 172.21.15.100:0/4209925347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614b7716590 0x5614b7b1e820 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fa8881516e0 tx=0x7fa870005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7991dc1aca1a41 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:41.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.682+0000 7fa876a00640 1 -- 172.21.15.100:0/4209925347 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870009d60 con 0x5614b7716590 2024-09-17T04:32:41.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.682+0000 7fa876a00640 1 -- 172.21.15.100:0/4209925347 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8700120d0 con 0x5614b7716590 2024-09-17T04:32:41.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.682+0000 7fa876a00640 1 -- 172.21.15.100:0/4209925347 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870005280 con 0x5614b7716590 2024-09-17T04:32:41.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.683+0000 7fa894c86740 1 -- 172.21.15.100:0/4209925347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614b7716590 msgr2=0x5614b7b1e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:41.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.683+0000 7fa894c86740 1 --2- 172.21.15.100:0/4209925347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614b7716590 0x5614b7b1e820 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fa8881516e0 tx=0x7fa870005f90 comp rx=0 tx=0).stop 2024-09-17T04:32:41.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.683+0000 7fa894c86740 1 -- 172.21.15.100:0/4209925347 shutdown_connections 2024-09-17T04:32:41.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.683+0000 7fa894c86740 1 --2- 172.21.15.100:0/4209925347 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5614b7737270 0x5614b7afd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.683+0000 7fa894c86740 1 --2- 172.21.15.100:0/4209925347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614b7716590 0x5614b7b1e820 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.683+0000 7fa894c86740 1 --2- 172.21.15.100:0/4209925347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614b770e3b0 0x5614b7b1e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.683+0000 7fa894c86740 1 -- 172.21.15.100:0/4209925347 >> 172.21.15.100:0/4209925347 conn(0x5614b772b3c0 msgr2=0x5614b7ada140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:41.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.683+0000 7fa894c86740 1 -- 172.21.15.100:0/4209925347 shutdown_connections 2024-09-17T04:32:41.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.683+0000 7fa894c86740 1 -- 172.21.15.100:0/4209925347 wait complete. 2024-09-17T04:32:41.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.684+0000 7fa894c86740 1 -- start start 2024-09-17T04:32:41.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.684+0000 7fa894c86740 1 Processor -- start 2024-09-17T04:32:41.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.686+0000 7fa894c86740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614b770e3b0 0x5614b7b99340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:41.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.686+0000 7fa894c86740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5614b7716590 0x5614b7b1e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:41.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.686+0000 7fa894c86740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614b7737270 0x5614b7b1e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:41.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.686+0000 7fa894c86740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5614b7b33e20 con 0x5614b7737270 2024-09-17T04:32:41.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.686+0000 7fa894c86740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5614b7b020f0 con 0x5614b770e3b0 2024-09-17T04:32:41.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.686+0000 7fa894c86740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5614b7b32180 con 0x5614b7716590 2024-09-17T04:32:41.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.686+0000 7fa894c86740 10 client.0 _ll_register_callbacks cb 0x7fa880062140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:32:41.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.686+0000 7fa887400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614b7737270 0x5614b7b1e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:41.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.686+0000 7fa887400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614b7737270 0x5614b7b1e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39280/0 (socket says 172.21.15.100:39280) 2024-09-17T04:32:41.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.686+0000 7fa887400640 1 -- 172.21.15.100:0/2560078585 learned_addr learned my addr 172.21.15.100:0/2560078585 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:41.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.686+0000 7fa886a00640 1 --2- 172.21.15.100:0/2560078585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614b770e3b0 0x5614b7b99340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:41.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.686+0000 7fa887400640 1 -- 172.21.15.100:0/2560078585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5614b7716590 msgr2=0x5614b7b1e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:32:41.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.686+0000 7fa87e000640 1 --2- 172.21.15.100:0/2560078585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5614b7716590 0x5614b7b1e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:41.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.686+0000 7fa887400640 1 --2- 172.21.15.100:0/2560078585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5614b7716590 0x5614b7b1e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.686+0000 7fa887400640 1 -- 172.21.15.100:0/2560078585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614b770e3b0 msgr2=0x5614b7b99340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:32:41.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.686+0000 7fa887400640 1 --2- 172.21.15.100:0/2560078585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614b770e3b0 0x5614b7b99340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.686+0000 7fa887400640 1 -- 172.21.15.100:0/2560078585 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614b7adb5c0 con 0x5614b7737270 2024-09-17T04:32:41.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.686+0000 7fa886a00640 1 --2- 172.21.15.100:0/2560078585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614b770e3b0 0x5614b7b99340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:32:41.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.686+0000 7fa87e000640 1 --2- 172.21.15.100:0/2560078585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5614b7716590 0x5614b7b1e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:32:41.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.687+0000 7fa887400640 1 --2- 172.21.15.100:0/2560078585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614b7737270 0x5614b7b1e690 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fa880001740 tx=0x7fa88005eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5e3a8c95352eaa server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:41.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.687+0000 7fa86be00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-17T04:32:41.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.687+0000 7fa86be00640 1 -- 172.21.15.100:0/2560078585 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa880067040 con 0x5614b7737270 2024-09-17T04:32:41.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.687+0000 7fa86be00640 1 -- 172.21.15.100:0/2560078585 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7fa8800616a0 con 0x5614b7737270 2024-09-17T04:32:41.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.687+0000 7fa894c86740 1 -- 172.21.15.100:0/2560078585 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8700120d0 con 0x5614b7737270 2024-09-17T04:32:41.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.687+0000 7fa86be00640 1 -- 172.21.15.100:0/2560078585 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7fa8800671e0 con 0x5614b7737270 2024-09-17T04:32:41.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.687+0000 7fa894c86740 10 client.8547 fetch_fsmap learned FSMap version 101 2024-09-17T04:32:41.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.687+0000 7fa894c86740 1 -- 172.21.15.100:0/2560078585 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=2+}) -- 0x5614b7adb5c0 con 0x5614b7737270 2024-09-17T04:32:41.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.687+0000 7fa86be00640 1 -- 172.21.15.100:0/2560078585 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8700120d0 con 0x5614b7737270 2024-09-17T04:32:41.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.687+0000 7fa86be00640 1 -- 172.21.15.100:0/2560078585 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 101) ==== 3502+0+0 (secure 0 0 0) 0x7fa8800618a0 con 0x5614b7737270 2024-09-17T04:32:41.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.687+0000 7fa894c86740 10 client.8547 fetch_fsmap finished waiting for FSMap version 101 2024-09-17T04:32:41.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.687+0000 7fa894c86740 10 client.8547 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T04:32:41.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.687+0000 7fa894c86740 20 client.8547 populate_metadata read hostname 'smithi100' 2024-09-17T04:32:41.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.687+0000 7fa894c86740 1 --2- 172.21.15.100:0/2560078585 >> [v2:172.21.15.100:6834/2425159512,v1:172.21.15.100:6835/2425159512] conn(0x5614b7ba6670 0x5614b7b3a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:41.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.688+0000 7fa894c86740 4 client.8547 mds_command: new command op to 8427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T04:32:41.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.688+0000 7fa894c86740 1 -- 172.21.15.100:0/2560078585 --> [v2:172.21.15.100:6834/2425159512,v1:172.21.15.100:6835/2425159512] -- command(tid 0: {"prefix": "status"}) -- 0x5614b7b33990 con 0x5614b7ba6670 2024-09-17T04:32:41.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.688+0000 7fa886a00640 1 --2- 172.21.15.100:0/2560078585 >> [v2:172.21.15.100:6834/2425159512,v1:172.21.15.100:6835/2425159512] conn(0x5614b7ba6670 0x5614b7b3a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:41.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.688+0000 7fa86be00640 10 client.8547 ms_handle_connect on v2:172.21.15.100:6834/2425159512 2024-09-17T04:32:41.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.688+0000 7fa886a00640 1 --2- 172.21.15.100:0/2560078585 >> [v2:172.21.15.100:6834/2425159512,v1:172.21.15.100:6835/2425159512] conn(0x5614b7ba6670 0x5614b7b3a6e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=783320a8687f6dbf server_cookie=db1856e14981f2e9 in_seq=0 out_seq=0 2024-09-17T04:32:41.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.689+0000 7fa86be00640 1 -- 172.21.15.100:0/2560078585 <== mds.0 v2:172.21.15.100:6834/2425159512 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x5614b7b33990 con 0x5614b7ba6670 2024-09-17T04:32:41.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.689+0000 7fa86be00640 10 client.8547 handle_command_reply: tid=0 2024-09-17T04:32:41.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.689+0000 7fa894c86740 1 client.8547 shutdown 2024-09-17T04:32:41.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.689+0000 7fa894c86740 1 -- 172.21.15.100:0/2560078585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614b7737270 msgr2=0x5614b7b1e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:41.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.689+0000 7fa894c86740 1 --2- 172.21.15.100:0/2560078585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614b7737270 0x5614b7b1e690 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fa880001740 tx=0x7fa88005eec0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.689+0000 7fa894c86740 1 -- 172.21.15.100:0/2560078585 shutdown_connections 2024-09-17T04:32:41.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.689+0000 7fa894c86740 1 --2- 172.21.15.100:0/2560078585 >> [v2:172.21.15.100:6834/2425159512,v1:172.21.15.100:6835/2425159512] conn(0x5614b7ba6670 0x5614b7b3a6e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.689+0000 7fa894c86740 1 --2- 172.21.15.100:0/2560078585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614b7737270 0x5614b7b1e690 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.689+0000 7fa894c86740 1 --2- 172.21.15.100:0/2560078585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5614b7716590 0x5614b7b1e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.689+0000 7fa894c86740 1 --2- 172.21.15.100:0/2560078585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614b770e3b0 0x5614b7b99340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.689+0000 7fa86be00640 0 client.8547 ms_handle_reset on v2:172.21.15.100:6834/2425159512 2024-09-17T04:32:41.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.689+0000 7fa894c86740 1 -- 172.21.15.100:0/2560078585 >> 172.21.15.100:0/2560078585 conn(0x5614b772b3c0 msgr2=0x5614b7ada140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:41.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.689+0000 7fa894c86740 1 -- 172.21.15.100:0/2560078585 shutdown_connections 2024-09-17T04:32:41.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.689+0000 7fa894c86740 1 -- 172.21.15.100:0/2560078585 wait complete. 2024-09-17T04:32:41.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.689+0000 7fa894c86740 20 client.8547 trim_cache size 0 max 16384 2024-09-17T04:32:41.693 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:32:41.693 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "8a399842-3a5c-4ea2-9afb-b2704332e59f", 2024-09-17T04:32:41.694 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 0, 2024-09-17T04:32:41.694 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 8427, 2024-09-17T04:32:41.694 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-09-17T04:32:41.694 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-09-17T04:32:41.694 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-09-17T04:32:41.694 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 6.6300286230000003, 2024-09-17T04:32:41.694 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 101, 2024-09-17T04:32:41.694 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 145, 2024-09-17T04:32:41.694 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 145, 2024-09-17T04:32:41.694 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 11.667419747 2024-09-17T04:32:41.694 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:32:41.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.690+0000 7fa887e00640 1 -- 172.21.15.100:0/1745048342 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa864079250 msgr2=0x7fa86407b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:41.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.690+0000 7fa887e00640 1 --2- 172.21.15.100:0/1745048342 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa864079250 0x7fa86407b760 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa878033000 tx=0x7fa878004730 comp rx=0 tx=0).stop 2024-09-17T04:32:41.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.690+0000 7fa887e00640 1 -- 172.21.15.100:0/1745048342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa88815d670 msgr2=0x7fa88814a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:41.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.690+0000 7fa887e00640 1 --2- 172.21.15.100:0/1745048342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa88815d670 0x7fa88814a2c0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fa88000c6b0 tx=0x7fa88000cb70 comp rx=0 tx=0).stop 2024-09-17T04:32:41.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.690+0000 7fa887e00640 1 -- 172.21.15.100:0/1745048342 shutdown_connections 2024-09-17T04:32:41.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.690+0000 7fa887e00640 1 --2- 172.21.15.100:0/1745048342 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa864079250 0x7fa86407b760 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.690+0000 7fa887e00640 1 --2- 172.21.15.100:0/1745048342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa88815d670 0x7fa88814a2c0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.690+0000 7fa887e00640 1 --2- 172.21.15.100:0/1745048342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8880b2f60 0x7fa888154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.690+0000 7fa887e00640 1 --2- 172.21.15.100:0/1745048342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8880ad9c0 0x7fa888154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.690+0000 7fa887e00640 1 -- 172.21.15.100:0/1745048342 >> 172.21.15.100:0/1745048342 conn(0x7fa8880b1e40 msgr2=0x7fa88815c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:41.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.690+0000 7fa887e00640 1 -- 172.21.15.100:0/1745048342 shutdown_connections 2024-09-17T04:32:41.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.690+0000 7fa887e00640 1 -- 172.21.15.100:0/1745048342 wait complete. 2024-09-17T04:32:41.714 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:32:41.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.932+0000 7fcba2c00640 1 -- 172.21.15.100:0/4223983422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb9c152920 msgr2=0x7fcb9c15f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:41.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.932+0000 7fcba2c00640 1 --2- 172.21.15.100:0/4223983422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb9c152920 0x7fcb9c15f3d0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fcb88005fd0 tx=0x7fcb8802a4a0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.933+0000 7fcba2c00640 1 -- 172.21.15.100:0/4223983422 shutdown_connections 2024-09-17T04:32:41.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.933+0000 7fcba2c00640 1 --2- 172.21.15.100:0/4223983422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb9c15f910 0x7fcb9c0b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.933+0000 7fcba2c00640 1 --2- 172.21.15.100:0/4223983422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb9c152920 0x7fcb9c15f3d0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.933+0000 7fcba2c00640 1 -- 172.21.15.100:0/4223983422 >> 172.21.15.100:0/4223983422 conn(0x7fcb9c06d730 msgr2=0x7fcb9c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:41.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.933+0000 7fcba2c00640 1 -- 172.21.15.100:0/4223983422 shutdown_connections 2024-09-17T04:32:41.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.933+0000 7fcba2c00640 1 -- 172.21.15.100:0/4223983422 wait complete. 2024-09-17T04:32:41.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.935+0000 7fcba2c00640 1 Processor -- start 2024-09-17T04:32:41.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.935+0000 7fcba2c00640 1 -- start start 2024-09-17T04:32:41.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.936+0000 7fcba2c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb9c0aebc0 0x7fcb9c151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:41.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.936+0000 7fcba2c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb9c152920 0x7fcb9c0afc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:41.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.936+0000 7fcba2c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb9c15f910 0x7fcb9c0b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:41.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.936+0000 7fcba2c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c0b4190 con 0x7fcb9c152920 2024-09-17T04:32:41.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.936+0000 7fcba2c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c0b4010 con 0x7fcb9c0aebc0 2024-09-17T04:32:41.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.937+0000 7fcba2c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcb9c0b4310 con 0x7fcb9c15f910 2024-09-17T04:32:41.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.937+0000 7fcb9b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb9c152920 0x7fcb9c0afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:41.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.937+0000 7fcb9b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb9c152920 0x7fcb9c0afc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39298/0 (socket says 172.21.15.100:39298) 2024-09-17T04:32:41.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.937+0000 7fcb9b400640 1 -- 172.21.15.100:0/3182455729 learned_addr learned my addr 172.21.15.100:0/3182455729 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:41.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.937+0000 7fcb9be00640 1 --2- 172.21.15.100:0/3182455729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb9c0aebc0 0x7fcb9c151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:41.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.937+0000 7fcba0c00640 1 --2- 172.21.15.100:0/3182455729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb9c15f910 0x7fcb9c0b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:41.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.937+0000 7fcb9b400640 1 -- 172.21.15.100:0/3182455729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb9c15f910 msgr2=0x7fcb9c0b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:41.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.937+0000 7fcb9b400640 1 --2- 172.21.15.100:0/3182455729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb9c15f910 0x7fcb9c0b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.937+0000 7fcb9b400640 1 -- 172.21.15.100:0/3182455729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb9c0aebc0 msgr2=0x7fcb9c151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:41.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.937+0000 7fcb9b400640 1 --2- 172.21.15.100:0/3182455729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb9c0aebc0 0x7fcb9c151170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:41.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.937+0000 7fcb9b400640 1 -- 172.21.15.100:0/3182455729 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb9c1516b0 con 0x7fcb9c152920 2024-09-17T04:32:41.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.938+0000 7fcb9b400640 1 --2- 172.21.15.100:0/3182455729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb9c152920 0x7fcb9c0afc70 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fcb88006010 tx=0x7fcb880039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:41.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.938+0000 7fcb93e00640 1 -- 172.21.15.100:0/3182455729 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8803e030 con 0x7fcb9c152920 2024-09-17T04:32:41.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.938+0000 7fcb93e00640 1 -- 172.21.15.100:0/3182455729 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb880047a0 con 0x7fcb9c152920 2024-09-17T04:32:41.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.938+0000 7fcb93e00640 1 -- 172.21.15.100:0/3182455729 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb88035040 con 0x7fcb9c152920 2024-09-17T04:32:41.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.938+0000 7fcba2c00640 1 -- 172.21.15.100:0/3182455729 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb9c0c9710 con 0x7fcb9c152920 2024-09-17T04:32:41.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.938+0000 7fcba2c00640 1 -- 172.21.15.100:0/3182455729 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb9c1e6fd0 con 0x7fcb9c152920 2024-09-17T04:32:41.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.939+0000 7fcba2c00640 1 -- 172.21.15.100:0/3182455729 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb9c1e6ae0 con 0x7fcb9c152920 2024-09-17T04:32:41.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.941+0000 7fcb93e00640 1 -- 172.21.15.100:0/3182455729 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7fcb88041030 con 0x7fcb9c152920 2024-09-17T04:32:41.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.942+0000 7fcb93e00640 1 --2- 172.21.15.100:0/3182455729 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcb7c079280 0x7fcb7c07b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:41.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.942+0000 7fcb93e00640 1 -- 172.21.15.100:0/3182455729 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7fcb880bc720 con 0x7fcb9c152920 2024-09-17T04:32:41.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.942+0000 7fcb9be00640 1 --2- 172.21.15.100:0/3182455729 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcb7c079280 0x7fcb7c07b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:41.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.943+0000 7fcb9be00640 1 --2- 172.21.15.100:0/3182455729 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcb7c079280 0x7fcb7c07b790 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c006fd0 tx=0x7fcb8c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:41.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:41.946+0000 7fcb93e00640 1 -- 172.21.15.100:0/3182455729 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fcb9c15fd20 con 0x7fcb9c152920 2024-09-17T04:32:42.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:42.177+0000 7fcba2c00640 1 -- 172.21.15.100:0/3182455729 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcb9c0b48a0 con 0x7fcb9c152920 2024-09-17T04:32:42.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:42.177+0000 7fcb93e00640 1 -- 172.21.15.100:0/3182455729 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v101) ==== 53+0+73 (secure 0 0 0) 0x7fcb88084180 con 0x7fcb9c152920 2024-09-17T04:32:42.178 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:32:42.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:42.180+0000 7fcba2c00640 1 -- 172.21.15.100:0/3182455729 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcb7c079280 msgr2=0x7fcb7c07b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:42.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:42.180+0000 7fcba2c00640 1 --2- 172.21.15.100:0/3182455729 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcb7c079280 0x7fcb7c07b790 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c006fd0 tx=0x7fcb8c008040 comp rx=0 tx=0).stop 2024-09-17T04:32:42.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:42.180+0000 7fcba2c00640 1 -- 172.21.15.100:0/3182455729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb9c152920 msgr2=0x7fcb9c0afc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:42.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:42.180+0000 7fcba2c00640 1 --2- 172.21.15.100:0/3182455729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb9c152920 0x7fcb9c0afc70 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fcb88006010 tx=0x7fcb880039e0 comp rx=0 tx=0).stop 2024-09-17T04:32:42.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:42.180+0000 7fcba2c00640 1 -- 172.21.15.100:0/3182455729 shutdown_connections 2024-09-17T04:32:42.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:42.180+0000 7fcba2c00640 1 --2- 172.21.15.100:0/3182455729 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcb7c079280 0x7fcb7c07b790 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:42.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:42.180+0000 7fcba2c00640 1 --2- 172.21.15.100:0/3182455729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb9c15f910 0x7fcb9c0b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:42.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:42.180+0000 7fcba2c00640 1 --2- 172.21.15.100:0/3182455729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb9c152920 0x7fcb9c0afc70 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:42.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:42.180+0000 7fcba2c00640 1 --2- 172.21.15.100:0/3182455729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb9c0aebc0 0x7fcb9c151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:42.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:42.180+0000 7fcba2c00640 1 -- 172.21.15.100:0/3182455729 >> 172.21.15.100:0/3182455729 conn(0x7fcb9c06d730 msgr2=0x7fcb9c15e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:42.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:42.180+0000 7fcba2c00640 1 -- 172.21.15.100:0/3182455729 shutdown_connections 2024-09-17T04:32:42.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:42.180+0000 7fcba2c00640 1 -- 172.21.15.100:0/3182455729 wait complete. 2024-09-17T04:32:42.202 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T04:32:42.202 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:32:42.202 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-09-17T04:32:42.202 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T04:32:42.202 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T04:32:42.202 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T04:32:42.202 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T04:32:42.202 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:32:42.202 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:32:42.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:42.203 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-09-17T04:32:42.238 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:32:42.238 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:32:42.238 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:32:42.238 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:32:42.239 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-17T04:32:42.239 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:32:42.239 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:32:42.239 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:32:42.239 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-09-17T04:32:42.239 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:32:42.239 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:32:42.239 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-09-17T04:32:42.239 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:32:42.239 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:32:42.239 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:32:42.239 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:32:42.239 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T04:32:42.240 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42198sec preferred_lft 42198sec 2024-09-17T04:32:42.240 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link noprefixroute 2024-09-17T04:32:42.240 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:32:42.240 INFO:teuthology.orchestra.run.smithi100.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:32:42.240 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:2e:de:8e:a4:7c brd ff:ff:ff:ff:ff:ff 2024-09-17T04:32:42.240 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T04:32:42.240 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:32:42.240 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::cc78:e4ff:fe70:7fbe/64 scope link 2024-09-17T04:32:42.240 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:32:42.240 INFO:teuthology.orchestra.run.smithi100.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:32:42.240 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether ea:8c:45:9f:34:87 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:32:42.240 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::e88c:45ff:fe9f:3487/64 scope link 2024-09-17T04:32:42.240 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:32:42.240 INFO:teuthology.orchestra.run.smithi100.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:32:42.241 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:2e:de:8e:a4:7c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:32:42.241 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::c2e:deff:fe8e:a47c/64 scope link 2024-09-17T04:32:42.241 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:32:42.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:42.241 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:32:42.331 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-09-17T04:32:42.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:42.334 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-09-17T04:32:42.370 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-09-17T04:32:42.370 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:32:42.371 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T04:32:42.371 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T04:32:42.371 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:32:42.371 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:32:42.371 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-17T04:32:42.371 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T04:32:42.371 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T04:32:42.371 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T04:32:42.371 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-17T04:32:42.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:32:42.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:32:42.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:42.567 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-17T04:32:42.602 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T04:32:42.602 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T04:32:42.603 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:32:42.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:42.603 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-09-17T04:32:42.698 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:32:42.698 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T04:32:42.733 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:32:42.734 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:32:42.734 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:32:42.809 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T04:32:42.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:42.809 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:32:43.003 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:32:43.003 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:32:43.003 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:32:43.023 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:32:43.023 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:32:43.024 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:32:43.024 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:32:43.024 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:32:43.055 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:32:43.055 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:32:43.055 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:32:43.055 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:32:43.056 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:32:43.056 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-09-17T04:32:43.056 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-09-17T04:32:43.056 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8a399842-3a5c-4ea2-9afb-b2704332e59f" will pass to kernel 2024-09-17T04:32:43.057 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-17T04:32:43.057 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-09-17T04:32:43.057 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-17T04:32:43.057 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-09-17T04:32:43.057 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-17T04:32:43.060 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T04:32:43.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:43.060 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:32:43.119 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:32:43.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:32:43.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:32:43.214 INFO:teuthology.orchestra.run.smithi100.stdout:client8556 2024-09-17T04:32:43.215 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:32:43.215 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:32:43.250 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:32:43.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.509+0000 7f9f75200640 1 -- 172.21.15.100:0/977957515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f700ae080 msgr2=0x7f9f70070000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:43.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.509+0000 7f9f75200640 1 --2- 172.21.15.100:0/977957515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f700ae080 0x7f9f70070000 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f9f700ad8e0 tx=0x7f9f5c02a5f0 comp rx=0 tx=0).stop 2024-09-17T04:32:43.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.509+0000 7f9f75200640 1 -- 172.21.15.100:0/977957515 shutdown_connections 2024-09-17T04:32:43.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.509+0000 7f9f75200640 1 --2- 172.21.15.100:0/977957515 >> v2:172.21.15.184:3300/0 conn(0x7f9f70070540 0x7f9f701651b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:43.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.509+0000 7f9f75200640 1 --2- 172.21.15.100:0/977957515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f700ae080 0x7f9f70070000 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:43.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.509+0000 7f9f75200640 1 -- 172.21.15.100:0/977957515 >> 172.21.15.100:0/977957515 conn(0x7f9f700b1e40 msgr2=0x7f9f700b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:43.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.509+0000 7f9f75200640 1 -- 172.21.15.100:0/977957515 shutdown_connections 2024-09-17T04:32:43.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.510+0000 7f9f75200640 1 -- 172.21.15.100:0/977957515 wait complete. 2024-09-17T04:32:43.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.512+0000 7f9f75200640 1 Processor -- start 2024-09-17T04:32:43.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.512+0000 7f9f75200640 1 -- start start 2024-09-17T04:32:43.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.513+0000 7f9f75200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f70070540 0x7f9f70145680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:43.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.513+0000 7f9f75200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f700ae080 0x7f9f701f8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:43.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.513+0000 7f9f75200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f700af050 0x7f9f701f8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:43.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.513+0000 7f9f75200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9f70165870 con 0x7f9f700af050 2024-09-17T04:32:43.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.513+0000 7f9f75200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9f701656f0 con 0x7f9f70070540 2024-09-17T04:32:43.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.513+0000 7f9f75200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9f701659f0 con 0x7f9f700ae080 2024-09-17T04:32:43.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.513+0000 7f9f6fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f700af050 0x7f9f701f8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:43.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.514+0000 7f9f6fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f700af050 0x7f9f701f8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39320/0 (socket says 172.21.15.100:39320) 2024-09-17T04:32:43.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.514+0000 7f9f6fe00640 1 -- 172.21.15.100:0/4038448678 learned_addr learned my addr 172.21.15.100:0/4038448678 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:43.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.514+0000 7f9f6ea00640 1 --2- 172.21.15.100:0/4038448678 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f700ae080 0x7f9f701f8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:43.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.514+0000 7f9f6f400640 1 --2- 172.21.15.100:0/4038448678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f70070540 0x7f9f70145680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:43.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.514+0000 7f9f6fe00640 1 -- 172.21.15.100:0/4038448678 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f700ae080 msgr2=0x7f9f701f8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:43.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.514+0000 7f9f6fe00640 1 --2- 172.21.15.100:0/4038448678 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f700ae080 0x7f9f701f8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:43.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.514+0000 7f9f6fe00640 1 -- 172.21.15.100:0/4038448678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f70070540 msgr2=0x7f9f70145680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:43.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.514+0000 7f9f6fe00640 1 --2- 172.21.15.100:0/4038448678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f70070540 0x7f9f70145680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:43.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.514+0000 7f9f6fe00640 1 -- 172.21.15.100:0/4038448678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f701f9740 con 0x7f9f700af050 2024-09-17T04:32:43.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.514+0000 7f9f6fe00640 1 --2- 172.21.15.100:0/4038448678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f700af050 0x7f9f701f8e70 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f9f6800cad0 tx=0x7f9f6800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:43.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.514+0000 7f9f67400640 1 -- 172.21.15.100:0/4038448678 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6800ae20 con 0x7f9f700af050 2024-09-17T04:32:43.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.515+0000 7f9f67400640 1 -- 172.21.15.100:0/4038448678 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f6800f450 con 0x7f9f700af050 2024-09-17T04:32:43.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.515+0000 7f9f67400640 1 -- 172.21.15.100:0/4038448678 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6800f8c0 con 0x7f9f700af050 2024-09-17T04:32:43.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.515+0000 7f9f75200640 1 -- 172.21.15.100:0/4038448678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f700b1910 con 0x7f9f700af050 2024-09-17T04:32:43.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.515+0000 7f9f75200640 1 -- 172.21.15.100:0/4038448678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f700b1110 con 0x7f9f700af050 2024-09-17T04:32:43.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.516+0000 7f9f75200640 1 -- 172.21.15.100:0/4038448678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f70070000 con 0x7f9f700af050 2024-09-17T04:32:43.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.517+0000 7f9f67400640 1 -- 172.21.15.100:0/4038448678 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f9f6800fa60 con 0x7f9f700af050 2024-09-17T04:32:43.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.518+0000 7f9f67400640 1 --2- 172.21.15.100:0/4038448678 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9f50079250 0x7f9f5007b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:43.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.518+0000 7f9f67400640 1 -- 172.21.15.100:0/4038448678 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7f9f6809c590 con 0x7f9f700af050 2024-09-17T04:32:43.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.518+0000 7f9f6f400640 1 --2- 172.21.15.100:0/4038448678 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9f50079250 0x7f9f5007b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:43.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.519+0000 7f9f6f400640 1 --2- 172.21.15.100:0/4038448678 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9f50079250 0x7f9f5007b760 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9f60000f60 tx=0x7f9f60009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:43.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.523+0000 7f9f67400640 1 -- 172.21.15.100:0/4038448678 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9f68014030 con 0x7f9f700af050 2024-09-17T04:32:43.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.741+0000 7f9f75200640 1 -- 172.21.15.100:0/4038448678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9f701f94d0 con 0x7f9f700af050 2024-09-17T04:32:43.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.742+0000 7f9f67400640 1 -- 172.21.15.100:0/4038448678 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v101) ==== 53+0+73 (secure 0 0 0) 0x7f9f68063810 con 0x7f9f700af050 2024-09-17T04:32:43.742 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:32:43.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.744+0000 7f9f75200640 1 -- 172.21.15.100:0/4038448678 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9f50079250 msgr2=0x7f9f5007b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:43.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.744+0000 7f9f75200640 1 --2- 172.21.15.100:0/4038448678 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9f50079250 0x7f9f5007b760 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9f60000f60 tx=0x7f9f60009040 comp rx=0 tx=0).stop 2024-09-17T04:32:43.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.744+0000 7f9f75200640 1 -- 172.21.15.100:0/4038448678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f700af050 msgr2=0x7f9f701f8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:43.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.744+0000 7f9f75200640 1 --2- 172.21.15.100:0/4038448678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f700af050 0x7f9f701f8e70 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f9f6800cad0 tx=0x7f9f6800cf90 comp rx=0 tx=0).stop 2024-09-17T04:32:43.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.744+0000 7f9f75200640 1 -- 172.21.15.100:0/4038448678 shutdown_connections 2024-09-17T04:32:43.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.744+0000 7f9f75200640 1 --2- 172.21.15.100:0/4038448678 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9f50079250 0x7f9f5007b760 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:43.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.744+0000 7f9f75200640 1 --2- 172.21.15.100:0/4038448678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f700af050 0x7f9f701f8e70 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:43.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.744+0000 7f9f75200640 1 --2- 172.21.15.100:0/4038448678 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f700ae080 0x7f9f701f8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:43.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.744+0000 7f9f75200640 1 --2- 172.21.15.100:0/4038448678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f70070540 0x7f9f70145680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:43.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.744+0000 7f9f75200640 1 -- 172.21.15.100:0/4038448678 >> 172.21.15.100:0/4038448678 conn(0x7f9f700b1e40 msgr2=0x7f9f701463a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:43.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.744+0000 7f9f75200640 1 -- 172.21.15.100:0/4038448678 shutdown_connections 2024-09-17T04:32:43.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:43.744+0000 7f9f75200640 1 -- 172.21.15.100:0/4038448678 wait complete. 2024-09-17T04:32:43.767 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T04:32:43.767 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:32:43.767 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2024-09-17T04:32:43.767 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T04:32:43.768 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T04:32:43.768 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T04:32:43.768 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T04:32:43.768 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:32:43.768 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:32:43.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:43.768 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2024-09-17T04:32:43.808 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:32:43.808 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:32:43.808 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:32:43.808 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:32:43.809 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-09-17T04:32:43.809 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:32:43.809 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:32:43.809 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:32:43.809 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2024-09-17T04:32:43.809 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:32:43.810 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:32:43.810 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2024-09-17T04:32:43.810 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:32:43.810 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2024-09-17T04:32:43.810 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:32:43.810 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2024-09-17T04:32:43.810 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T04:32:43.810 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42180sec preferred_lft 42180sec 2024-09-17T04:32:43.810 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ec4:7aff:fe88:7b8f/64 scope link noprefixroute 2024-09-17T04:32:43.810 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:32:43.810 INFO:teuthology.orchestra.run.smithi184.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:32:43.811 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 7e:67:e5:69:96:69 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:32:43.811 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T04:32:43.811 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:32:43.811 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::3c9d:aff:fe6e:8fcb/64 scope link 2024-09-17T04:32:43.811 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:32:43.811 INFO:teuthology.orchestra.run.smithi184.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:32:43.811 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether a6:a9:1b:a9:61:ec brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T04:32:43.811 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::a4a9:1bff:fea9:61ec/64 scope link 2024-09-17T04:32:43.811 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:32:43.811 INFO:teuthology.orchestra.run.smithi184.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:32:43.811 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 7e:67:e5:69:96:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T04:32:43.811 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::7c67:e5ff:fe69:9669/64 scope link 2024-09-17T04:32:43.811 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:32:43.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:43.813 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:32:43.910 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-09-17T04:32:43.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:43.913 DEBUG:teuthology.orchestra.run.smithi184:> route 2024-09-17T04:32:43.950 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2024-09-17T04:32:43.951 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:32:43.951 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T04:32:43.951 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T04:32:43.951 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:32:43.951 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:32:43.951 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-09-17T04:32:43.951 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T04:32:43.952 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T04:32:43.952 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T04:32:43.952 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-09-17T04:32:44.069 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:32:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:32:44.142 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:32:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:32:44.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:44.149 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-09-17T04:32:44.185 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T04:32:44.185 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T04:32:44.185 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:32:44.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:44.185 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.0 up 2024-09-17T04:32:44.282 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:32:44.283 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T04:32:44.317 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:32:44.317 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:32:44.317 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T04:32:44.393 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T04:32:44.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:44.393 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:32:44.587 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:32:44.587 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:32:44.587 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:32:44.607 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:32:44.607 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:32:44.607 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:32:44.607 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:32:44.607 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:32:44.636 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:32:44.636 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:32:44.636 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:32:44.636 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:32:44.636 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:32:44.636 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-09-17T04:32:44.636 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-09-17T04:32:44.636 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8a399842-3a5c-4ea2-9afb-b2704332e59f" will pass to kernel 2024-09-17T04:32:44.637 INFO:teuthology.orchestra.run.smithi184.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-17T04:32:44.637 INFO:teuthology.orchestra.run.smithi184.stdout: You just mounted a file system that supports labels which does not 2024-09-17T04:32:44.638 INFO:teuthology.orchestra.run.smithi184.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-17T04:32:44.638 INFO:teuthology.orchestra.run.smithi184.stdout: applications will generate AVC messages and not be allowed access to 2024-09-17T04:32:44.638 INFO:teuthology.orchestra.run.smithi184.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-17T04:32:44.640 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T04:32:44.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:44.641 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T04:32:44.697 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:32:44.774 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:32:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:32:44.778 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:32:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:32:44.786 INFO:teuthology.orchestra.run.smithi184.stdout:client8568 2024-09-17T04:32:44.786 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:32:44.786 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:32:44.822 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T04:32:45.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.073+0000 7fd494200640 1 -- 172.21.15.100:0/1890227704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd474003a00 msgr2=0x7fd474003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:45.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.073+0000 7fd494200640 1 --2- 172.21.15.100:0/1890227704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd474003a00 0x7fd474003e70 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fd480009b50 tx=0x7fd474005830 comp rx=0 tx=0).stop 2024-09-17T04:32:45.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.073+0000 7fd492200640 1 -- 172.21.15.100:0/1890227704 reap_dead start 2024-09-17T04:32:45.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.073+0000 7fd494200640 1 -- 172.21.15.100:0/1890227704 shutdown_connections 2024-09-17T04:32:45.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.073+0000 7fd494200640 1 -- 172.21.15.100:0/1890227704 >> 172.21.15.100:0/1890227704 conn(0x7fd48c0b2540 msgr2=0x7fd48c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:45.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.073+0000 7fd494200640 1 -- 172.21.15.100:0/1890227704 shutdown_connections 2024-09-17T04:32:45.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.073+0000 7fd494200640 1 -- 172.21.15.100:0/1890227704 wait complete. 2024-09-17T04:32:45.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.075+0000 7fd494200640 1 Processor -- start 2024-09-17T04:32:45.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.076+0000 7fd494200640 1 -- start start 2024-09-17T04:32:45.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.077+0000 7fd494200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd474003a00 0x7fd48c155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:45.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.077+0000 7fd494200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd48c098cf0 0x7fd48c1556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:45.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.077+0000 7fd494200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd48c099240 0x7fd48c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:45.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.077+0000 7fd494200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c0b3120 con 0x7fd48c099240 2024-09-17T04:32:45.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.077+0000 7fd494200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c0b48f0 con 0x7fd48c098cf0 2024-09-17T04:32:45.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.077+0000 7fd494200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd48c0b32a0 con 0x7fd474003a00 2024-09-17T04:32:45.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.077+0000 7fd492200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd48c099240 0x7fd48c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:45.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.077+0000 7fd491800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd474003a00 0x7fd48c155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:45.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.077+0000 7fd491800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd474003a00 0x7fd48c155190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.100:52946/0 (socket says 172.21.15.100:52946) 2024-09-17T04:32:45.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.077+0000 7fd491800640 1 -- 172.21.15.100:0/4025443184 learned_addr learned my addr 172.21.15.100:0/4025443184 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:45.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.077+0000 7fd490e00640 1 --2- 172.21.15.100:0/4025443184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd48c098cf0 0x7fd48c1556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:45.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.078+0000 7fd491800640 1 -- 172.21.15.100:0/4025443184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd48c098cf0 msgr2=0x7fd48c1556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:45.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.078+0000 7fd491800640 1 --2- 172.21.15.100:0/4025443184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd48c098cf0 0x7fd48c1556d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:45.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.078+0000 7fd491800640 1 -- 172.21.15.100:0/4025443184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd48c099240 msgr2=0x7fd48c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:45.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.078+0000 7fd491800640 1 --2- 172.21.15.100:0/4025443184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd48c099240 0x7fd48c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:45.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.078+0000 7fd491800640 1 -- 172.21.15.100:0/4025443184 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c156fe0 con 0x7fd474003a00 2024-09-17T04:32:45.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.078+0000 7fd492200640 1 --2- 172.21.15.100:0/4025443184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd48c099240 0x7fd48c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:32:45.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.078+0000 7fd491800640 1 --2- 172.21.15.100:0/4025443184 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd474003a00 0x7fd48c155190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd480036040 tx=0x7fd480033670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:45.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.078+0000 7fd489600640 1 -- 172.21.15.100:0/4025443184 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd480038050 con 0x7fd474003a00 2024-09-17T04:32:45.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.079+0000 7fd489600640 1 -- 172.21.15.100:0/4025443184 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd480039040 con 0x7fd474003a00 2024-09-17T04:32:45.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.079+0000 7fd494200640 1 -- 172.21.15.100:0/4025443184 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48c153170 con 0x7fd474003a00 2024-09-17T04:32:45.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.079+0000 7fd489600640 1 -- 172.21.15.100:0/4025443184 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4800339f0 con 0x7fd474003a00 2024-09-17T04:32:45.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.079+0000 7fd494200640 1 -- 172.21.15.100:0/4025443184 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48c156210 con 0x7fd474003a00 2024-09-17T04:32:45.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.080+0000 7fd494200640 1 -- 172.21.15.100:0/4025443184 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd48c156470 con 0x7fd474003a00 2024-09-17T04:32:45.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.081+0000 7fd489600640 1 -- 172.21.15.100:0/4025443184 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7fd480046020 con 0x7fd474003a00 2024-09-17T04:32:45.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.082+0000 7fd489600640 1 --2- 172.21.15.100:0/4025443184 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd46c079250 0x7fd46c07b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:45.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.082+0000 7fd489600640 1 -- 172.21.15.100:0/4025443184 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7fd4800bc2c0 con 0x7fd474003a00 2024-09-17T04:32:45.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.082+0000 7fd490e00640 1 --2- 172.21.15.100:0/4025443184 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd46c079250 0x7fd46c07b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:45.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.083+0000 7fd490e00640 1 --2- 172.21.15.100:0/4025443184 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd46c079250 0x7fd46c07b760 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd48c0699f0 tx=0x7fd478005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:45.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.088+0000 7fd489600640 1 -- 172.21.15.100:0/4025443184 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd48004d020 con 0x7fd474003a00 2024-09-17T04:32:45.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.304+0000 7fd494200640 1 -- 172.21.15.100:0/4025443184 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fd48c06d1b0 con 0x7fd474003a00 2024-09-17T04:32:45.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.360+0000 7fd489600640 1 -- 172.21.15.100:0/4025443184 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v102) ==== 136+0+0 (secure 0 0 0) 0x7fd4800836d0 con 0x7fd474003a00 2024-09-17T04:32:45.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.363+0000 7fd494200640 1 -- 172.21.15.100:0/4025443184 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd46c079250 msgr2=0x7fd46c07b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.363+0000 7fd494200640 1 --2- 172.21.15.100:0/4025443184 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd46c079250 0x7fd46c07b760 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd48c0699f0 tx=0x7fd478005f90 comp rx=0 tx=0).stop 2024-09-17T04:32:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.363+0000 7fd494200640 1 -- 172.21.15.100:0/4025443184 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd474003a00 msgr2=0x7fd48c155190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.363+0000 7fd494200640 1 --2- 172.21.15.100:0/4025443184 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd474003a00 0x7fd48c155190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd480036040 tx=0x7fd480033670 comp rx=0 tx=0).stop 2024-09-17T04:32:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.363+0000 7fd494200640 1 -- 172.21.15.100:0/4025443184 shutdown_connections 2024-09-17T04:32:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.363+0000 7fd494200640 1 --2- 172.21.15.100:0/4025443184 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd46c079250 0x7fd46c07b760 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.363+0000 7fd494200640 1 --2- 172.21.15.100:0/4025443184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd48c099240 0x7fd48c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.363+0000 7fd494200640 1 --2- 172.21.15.100:0/4025443184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd48c098cf0 0x7fd48c1556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.363+0000 7fd494200640 1 --2- 172.21.15.100:0/4025443184 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd474003a00 0x7fd48c155190 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.363+0000 7fd494200640 1 -- 172.21.15.100:0/4025443184 >> 172.21.15.100:0/4025443184 conn(0x7fd48c0b2540 msgr2=0x7fd48c06f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:45.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.363+0000 7fd494200640 1 -- 172.21.15.100:0/4025443184 shutdown_connections 2024-09-17T04:32:45.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.363+0000 7fd494200640 1 -- 172.21.15.100:0/4025443184 wait complete. 2024-09-17T04:32:45.386 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T04:32:45.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.606+0000 7f91b2a00640 1 -- 172.21.15.100:0/1226225728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9190005270 msgr2=0x7f919000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:45.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.606+0000 7f91b2a00640 1 --2- 172.21.15.100:0/1226225728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9190005270 0x7f919000b9c0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f91a4005fd0 tx=0x7f9190005830 comp rx=0 tx=0).stop 2024-09-17T04:32:45.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.606+0000 7f91b2a00640 1 -- 172.21.15.100:0/1226225728 shutdown_connections 2024-09-17T04:32:45.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.606+0000 7f91b2a00640 1 --2- 172.21.15.100:0/1226225728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9190005270 0x7f919000b9c0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:45.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.606+0000 7f91b2a00640 1 -- 172.21.15.100:0/1226225728 >> 172.21.15.100:0/1226225728 conn(0x7f91ac06d3d0 msgr2=0x7f91ac0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:45.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.606+0000 7f91b2a00640 1 -- 172.21.15.100:0/1226225728 shutdown_connections 2024-09-17T04:32:45.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.606+0000 7f91b2a00640 1 -- 172.21.15.100:0/1226225728 wait complete. 2024-09-17T04:32:45.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.609+0000 7f91b2a00640 1 Processor -- start 2024-09-17T04:32:45.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.609+0000 7f91b2a00640 1 -- start start 2024-09-17T04:32:45.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.610+0000 7f91b2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9190005270 0x7f91ac1f7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:45.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.610+0000 7f91b2a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91ac1e8310 0x7f91ac0c51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:45.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.610+0000 7f91b2a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91ac0c5710 0x7f91ac1e6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:45.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.610+0000 7f91b2a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac15ce30 con 0x7f9190005270 2024-09-17T04:32:45.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.610+0000 7f91b2a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac157210 con 0x7f91ac1e8310 2024-09-17T04:32:45.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.610+0000 7f91b2a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f91ac15cfb0 con 0x7f91ac0c5710 2024-09-17T04:32:45.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.610+0000 7f91abe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9190005270 0x7f91ac1f7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:45.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.610+0000 7f91abe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9190005270 0x7f91ac1f7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34946/0 (socket says 172.21.15.100:34946) 2024-09-17T04:32:45.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.610+0000 7f91abe00640 1 -- 172.21.15.100:0/90039 learned_addr learned my addr 172.21.15.100:0/90039 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:45.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.611+0000 7f91ab400640 1 --2- 172.21.15.100:0/90039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91ac1e8310 0x7f91ac0c51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:45.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.611+0000 7f91b0a00640 1 --2- 172.21.15.100:0/90039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91ac0c5710 0x7f91ac1e6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:45.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.611+0000 7f91abe00640 1 -- 172.21.15.100:0/90039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91ac0c5710 msgr2=0x7f91ac1e6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:45.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.611+0000 7f91abe00640 1 --2- 172.21.15.100:0/90039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91ac0c5710 0x7f91ac1e6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:45.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.611+0000 7f91abe00640 1 -- 172.21.15.100:0/90039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91ac1e8310 msgr2=0x7f91ac0c51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:45.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.611+0000 7f91abe00640 1 --2- 172.21.15.100:0/90039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91ac1e8310 0x7f91ac0c51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:45.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.611+0000 7f91abe00640 1 -- 172.21.15.100:0/90039 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91ac1e7c30 con 0x7f9190005270 2024-09-17T04:32:45.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.611+0000 7f91abe00640 1 --2- 172.21.15.100:0/90039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9190005270 0x7f91ac1f7fd0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f919c000c00 tx=0x7f919c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:45.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.611+0000 7f91a3e00640 1 -- 172.21.15.100:0/90039 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c015070 con 0x7f9190005270 2024-09-17T04:32:45.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.611+0000 7f91a3e00640 1 -- 172.21.15.100:0/90039 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f919c011860 con 0x7f9190005270 2024-09-17T04:32:45.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.611+0000 7f91b2a00640 1 -- 172.21.15.100:0/90039 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91ac0c5b20 con 0x7f9190005270 2024-09-17T04:32:45.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.612+0000 7f91a3e00640 1 -- 172.21.15.100:0/90039 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c011cb0 con 0x7f9190005270 2024-09-17T04:32:45.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.612+0000 7f91b2a00640 1 -- 172.21.15.100:0/90039 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91ac0afe30 con 0x7f9190005270 2024-09-17T04:32:45.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.613+0000 7f91b2a00640 1 -- 172.21.15.100:0/90039 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91ac0ca5d0 con 0x7f9190005270 2024-09-17T04:32:45.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.614+0000 7f91a3e00640 1 -- 172.21.15.100:0/90039 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f919c018070 con 0x7f9190005270 2024-09-17T04:32:45.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.615+0000 7f91a3e00640 1 --2- 172.21.15.100:0/90039 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f918c079250 0x7f918c07b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:45.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.615+0000 7f91a3e00640 1 -- 172.21.15.100:0/90039 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7f919c09c630 con 0x7f9190005270 2024-09-17T04:32:45.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.615+0000 7f91ab400640 1 --2- 172.21.15.100:0/90039 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f918c079250 0x7f918c07b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:45.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.616+0000 7f91ab400640 1 --2- 172.21.15.100:0/90039 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f918c079250 0x7f918c07b760 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f91ac156330 tx=0x7f91980067d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:45.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.619+0000 7f91a3e00640 1 -- 172.21.15.100:0/90039 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f91ac0ca5d0 con 0x7f9190005270 2024-09-17T04:32:45.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.825+0000 7f91b2a00640 1 -- 172.21.15.100:0/90039 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f91ac0c3be0 con 0x7f918c079250 2024-09-17T04:32:45.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.827+0000 7f91a3e00640 1 -- 172.21.15.100:0/90039 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f91ac0c3be0 con 0x7f918c079250 2024-09-17T04:32:45.827 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-09-17T04:32:45.827 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:32:45.827 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephfs" 2024-09-17T04:32:45.827 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:32:45.827 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-09-17T04:32:45.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.829+0000 7f91b2a00640 1 -- 172.21.15.100:0/90039 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f918c079250 msgr2=0x7f918c07b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:45.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.829+0000 7f91b2a00640 1 --2- 172.21.15.100:0/90039 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f918c079250 0x7f918c07b760 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f91ac156330 tx=0x7f91980067d0 comp rx=0 tx=0).stop 2024-09-17T04:32:45.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.829+0000 7f91b2a00640 1 -- 172.21.15.100:0/90039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9190005270 msgr2=0x7f91ac1f7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:45.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.829+0000 7f91b2a00640 1 --2- 172.21.15.100:0/90039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9190005270 0x7f91ac1f7fd0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f919c000c00 tx=0x7f919c001710 comp rx=0 tx=0).stop 2024-09-17T04:32:45.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.829+0000 7f91b2a00640 1 -- 172.21.15.100:0/90039 shutdown_connections 2024-09-17T04:32:45.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.829+0000 7f91b2a00640 1 --2- 172.21.15.100:0/90039 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f918c079250 0x7f918c07b760 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:45.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.829+0000 7f91b2a00640 1 --2- 172.21.15.100:0/90039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91ac0c5710 0x7f91ac1e6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:45.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.829+0000 7f91b2a00640 1 --2- 172.21.15.100:0/90039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91ac1e8310 0x7f91ac0c51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:45.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.829+0000 7f91b2a00640 1 --2- 172.21.15.100:0/90039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9190005270 0x7f91ac1f7fd0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:45.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.829+0000 7f91b2a00640 1 -- 172.21.15.100:0/90039 >> 172.21.15.100:0/90039 conn(0x7f91ac06d3d0 msgr2=0x7f91ac1564c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:45.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.829+0000 7f91b2a00640 1 -- 172.21.15.100:0/90039 shutdown_connections 2024-09-17T04:32:45.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:45.829+0000 7f91b2a00640 1 -- 172.21.15.100:0/90039 wait complete. 2024-09-17T04:32:45.856 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T04:32:46.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.081+0000 7fab15e00640 1 -- 172.21.15.100:0/3465787729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab10073c50 msgr2=0x7fab1006f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:46.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.081+0000 7fab15e00640 1 --2- 172.21.15.100:0/3465787729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab10073c50 0x7fab1006f7d0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fab08005fd0 tx=0x7fab0802a780 comp rx=0 tx=0).stop 2024-09-17T04:32:46.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.081+0000 7fab15e00640 1 -- 172.21.15.100:0/3465787729 shutdown_connections 2024-09-17T04:32:46.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.081+0000 7fab15e00640 1 --2- 172.21.15.100:0/3465787729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab10073c50 0x7fab1006f7d0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:46.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.081+0000 7fab15e00640 1 -- 172.21.15.100:0/3465787729 >> 172.21.15.100:0/3465787729 conn(0x7fab1006d1c0 msgr2=0x7fab1006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:46.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.081+0000 7fab15e00640 1 -- 172.21.15.100:0/3465787729 shutdown_connections 2024-09-17T04:32:46.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.081+0000 7fab15e00640 1 -- 172.21.15.100:0/3465787729 wait complete. 2024-09-17T04:32:46.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.083+0000 7fab15e00640 1 Processor -- start 2024-09-17T04:32:46.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.083+0000 7fab15e00640 1 -- start start 2024-09-17T04:32:46.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.084+0000 7fab15e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab10147b90 0x7fab100cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:46.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.084+0000 7fab15e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab1015f9e0 0x7fab1015dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:46.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.084+0000 7fab15e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab1013e340 0x7fab1013ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:46.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.084+0000 7fab15e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fab1006fe90 con 0x7fab10147b90 2024-09-17T04:32:46.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.084+0000 7fab15e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fab1006fd10 con 0x7fab1013e340 2024-09-17T04:32:46.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.084+0000 7fab15e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fab10070010 con 0x7fab1015f9e0 2024-09-17T04:32:46.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.085+0000 7fab0ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab10147b90 0x7fab100cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:46.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.085+0000 7fab0ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab10147b90 0x7fab100cedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34960/0 (socket says 172.21.15.100:34960) 2024-09-17T04:32:46.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.085+0000 7fab0ea00640 1 -- 172.21.15.100:0/1209835686 learned_addr learned my addr 172.21.15.100:0/1209835686 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:46.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.085+0000 7fab0f400640 1 --2- 172.21.15.100:0/1209835686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab1013e340 0x7fab1013ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:46.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.085+0000 7fab0e000640 1 --2- 172.21.15.100:0/1209835686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab1015f9e0 0x7fab1015dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:46.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.085+0000 7fab0ea00640 1 -- 172.21.15.100:0/1209835686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab1015f9e0 msgr2=0x7fab1015dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:46.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.085+0000 7fab0ea00640 1 --2- 172.21.15.100:0/1209835686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab1015f9e0 0x7fab1015dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:46.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.085+0000 7fab0ea00640 1 -- 172.21.15.100:0/1209835686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab1013e340 msgr2=0x7fab1013ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:46.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.085+0000 7fab0ea00640 1 --2- 172.21.15.100:0/1209835686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab1013e340 0x7fab1013ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:46.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.085+0000 7fab0ea00640 1 -- 172.21.15.100:0/1209835686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab10143cf0 con 0x7fab10147b90 2024-09-17T04:32:46.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.085+0000 7fab0e000640 1 --2- 172.21.15.100:0/1209835686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab1015f9e0 0x7fab1015dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:46.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.085+0000 7fab0ea00640 1 --2- 172.21.15.100:0/1209835686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab10147b90 0x7fab100cedc0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fab0000c3c0 tx=0x7fab0000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:46.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.086+0000 7fab07400640 1 -- 172.21.15.100:0/1209835686 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab00017070 con 0x7fab10147b90 2024-09-17T04:32:46.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.086+0000 7fab07400640 1 -- 172.21.15.100:0/1209835686 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab000035f0 con 0x7fab10147b90 2024-09-17T04:32:46.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.086+0000 7fab07400640 1 -- 172.21.15.100:0/1209835686 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab00003a20 con 0x7fab10147b90 2024-09-17T04:32:46.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.086+0000 7fab15e00640 1 -- 172.21.15.100:0/1209835686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab10143fd0 con 0x7fab10147b90 2024-09-17T04:32:46.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.086+0000 7fab15e00640 1 -- 172.21.15.100:0/1209835686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab1013de20 con 0x7fab10147b90 2024-09-17T04:32:46.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.087+0000 7fab15e00640 1 -- 172.21.15.100:0/1209835686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab1006f7d0 con 0x7fab10147b90 2024-09-17T04:32:46.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.088+0000 7fab07400640 1 -- 172.21.15.100:0/1209835686 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7fab0001a070 con 0x7fab10147b90 2024-09-17T04:32:46.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.089+0000 7fab07400640 1 --2- 172.21.15.100:0/1209835686 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7faaf0079280 0x7faaf007b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:46.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.089+0000 7fab07400640 1 -- 172.21.15.100:0/1209835686 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7fab0009acc0 con 0x7fab10147b90 2024-09-17T04:32:46.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.089+0000 7fab0e000640 1 --2- 172.21.15.100:0/1209835686 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7faaf0079280 0x7faaf007b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:46.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.090+0000 7fab0e000640 1 --2- 172.21.15.100:0/1209835686 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7faaf0079280 0x7faaf007b790 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7faafc006fd0 tx=0x7faafc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:46.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.095+0000 7fab07400640 1 -- 172.21.15.100:0/1209835686 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fab00061f00 con 0x7fab10147b90 2024-09-17T04:32:46.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.277+0000 7fab15e00640 1 -- 172.21.15.100:0/1209835686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fab100b3020 con 0x7fab10147b90 2024-09-17T04:32:46.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.278+0000 7fab07400640 1 -- 172.21.15.100:0/1209835686 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v48)=0 v48) ==== 122+0+0 (secure 0 0 0) 0x7fab00066930 con 0x7fab10147b90 2024-09-17T04:32:46.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.281+0000 7fab15e00640 1 -- 172.21.15.100:0/1209835686 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7faaf0079280 msgr2=0x7faaf007b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:46.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.281+0000 7fab15e00640 1 --2- 172.21.15.100:0/1209835686 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7faaf0079280 0x7faaf007b790 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7faafc006fd0 tx=0x7faafc008040 comp rx=0 tx=0).stop 2024-09-17T04:32:46.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.281+0000 7fab15e00640 1 -- 172.21.15.100:0/1209835686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab10147b90 msgr2=0x7fab100cedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:46.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.281+0000 7fab15e00640 1 --2- 172.21.15.100:0/1209835686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab10147b90 0x7fab100cedc0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fab0000c3c0 tx=0x7fab0000c880 comp rx=0 tx=0).stop 2024-09-17T04:32:46.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.281+0000 7fab15e00640 1 -- 172.21.15.100:0/1209835686 shutdown_connections 2024-09-17T04:32:46.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.281+0000 7fab15e00640 1 --2- 172.21.15.100:0/1209835686 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7faaf0079280 0x7faaf007b790 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:46.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.281+0000 7fab15e00640 1 --2- 172.21.15.100:0/1209835686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab1013e340 0x7fab1013ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:46.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.281+0000 7fab15e00640 1 --2- 172.21.15.100:0/1209835686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab1015f9e0 0x7fab1015dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:46.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.281+0000 7fab15e00640 1 --2- 172.21.15.100:0/1209835686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab10147b90 0x7fab100cedc0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:46.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.281+0000 7fab15e00640 1 -- 172.21.15.100:0/1209835686 >> 172.21.15.100:0/1209835686 conn(0x7fab1006d1c0 msgr2=0x7fab10160190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:46.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.281+0000 7fab15e00640 1 -- 172.21.15.100:0/1209835686 shutdown_connections 2024-09-17T04:32:46.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.281+0000 7fab15e00640 1 -- 172.21.15.100:0/1209835686 wait complete. 2024-09-17T04:32:46.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:46.302 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvol57370) 2024-09-17T04:32:46.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:46.368 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp92870/subvol83781) 2024-09-17T04:32:46.427 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol57370 2024-09-17T04:32:46.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.650+0000 7f618b200640 1 -- 172.21.15.100:0/2726246120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61840b3f30 msgr2=0x7f6184156ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:46.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.650+0000 7f618b200640 1 --2- 172.21.15.100:0/2726246120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61840b3f30 0x7f6184156ca0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6178009830 tx=0x7f617802a4a0 comp rx=0 tx=0).stop 2024-09-17T04:32:46.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.650+0000 7f618b200640 1 -- 172.21.15.100:0/2726246120 shutdown_connections 2024-09-17T04:32:46.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.650+0000 7f618b200640 1 --2- 172.21.15.100:0/2726246120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61840cd1d0 0x7f61840af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:46.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.650+0000 7f618b200640 1 --2- 172.21.15.100:0/2726246120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61840b3f30 0x7f6184156ca0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:46.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.650+0000 7f618b200640 1 -- 172.21.15.100:0/2726246120 >> 172.21.15.100:0/2726246120 conn(0x7f618406d620 msgr2=0x7f61840b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:46.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.650+0000 7f618b200640 1 -- 172.21.15.100:0/2726246120 shutdown_connections 2024-09-17T04:32:46.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.650+0000 7f618b200640 1 -- 172.21.15.100:0/2726246120 wait complete. 2024-09-17T04:32:46.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.653+0000 7f618b200640 1 Processor -- start 2024-09-17T04:32:46.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.653+0000 7f618b200640 1 -- start start 2024-09-17T04:32:46.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.655+0000 7f618b200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61840b3f30 0x7f618415b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:46.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.655+0000 7f618b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61840cd1d0 0x7f618415bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:46.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.655+0000 7f618b200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f618415d6e0 0x7f61841f58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:46.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.655+0000 7f618b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f61840af9a0 con 0x7f61840cd1d0 2024-09-17T04:32:46.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.655+0000 7f618b200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f61840af820 con 0x7f61840b3f30 2024-09-17T04:32:46.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.655+0000 7f618b200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f61840afb20 con 0x7f618415d6e0 2024-09-17T04:32:46.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.655+0000 7f6189400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61840cd1d0 0x7f618415bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:46.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.655+0000 7f6189400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61840cd1d0 0x7f618415bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34970/0 (socket says 172.21.15.100:34970) 2024-09-17T04:32:46.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.655+0000 7f6189400640 1 -- 172.21.15.100:0/2951391100 learned_addr learned my addr 172.21.15.100:0/2951391100 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:46.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.655+0000 7f618a800640 1 --2- 172.21.15.100:0/2951391100 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f618415d6e0 0x7f61841f58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:46.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.655+0000 7f6189e00640 1 --2- 172.21.15.100:0/2951391100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61840b3f30 0x7f618415b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:46.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.655+0000 7f6189400640 1 -- 172.21.15.100:0/2951391100 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f618415d6e0 msgr2=0x7f61841f58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:46.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.655+0000 7f6189400640 1 --2- 172.21.15.100:0/2951391100 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f618415d6e0 0x7f61841f58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:46.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.655+0000 7f6189400640 1 -- 172.21.15.100:0/2951391100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61840b3f30 msgr2=0x7f618415b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:46.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.655+0000 7f6189400640 1 --2- 172.21.15.100:0/2951391100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61840b3f30 0x7f618415b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:46.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.655+0000 7f6189400640 1 -- 172.21.15.100:0/2951391100 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6184144a40 con 0x7f61840cd1d0 2024-09-17T04:32:46.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.655+0000 7f6189400640 1 --2- 172.21.15.100:0/2951391100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61840cd1d0 0x7f618415bcc0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f617400c500 tx=0x7f617400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:46.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.655+0000 7f617e000640 1 -- 172.21.15.100:0/2951391100 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6174011790 con 0x7f61840cd1d0 2024-09-17T04:32:46.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.656+0000 7f617e000640 1 -- 172.21.15.100:0/2951391100 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6174017070 con 0x7f61840cd1d0 2024-09-17T04:32:46.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.656+0000 7f618b200640 1 -- 172.21.15.100:0/2951391100 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6184144d20 con 0x7f61840cd1d0 2024-09-17T04:32:46.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.656+0000 7f617e000640 1 -- 172.21.15.100:0/2951391100 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6174011cc0 con 0x7f61840cd1d0 2024-09-17T04:32:46.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.656+0000 7f618b200640 1 -- 172.21.15.100:0/2951391100 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61841452e0 con 0x7f61840cd1d0 2024-09-17T04:32:46.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.657+0000 7f618b200640 1 -- 172.21.15.100:0/2951391100 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6184156ca0 con 0x7f61840cd1d0 2024-09-17T04:32:46.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.658+0000 7f617e000640 1 -- 172.21.15.100:0/2951391100 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f617401c460 con 0x7f61840cd1d0 2024-09-17T04:32:46.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.659+0000 7f617e000640 1 --2- 172.21.15.100:0/2951391100 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6168079250 0x7f616807b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:46.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.659+0000 7f617e000640 1 -- 172.21.15.100:0/2951391100 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7f617409b6e0 con 0x7f61840cd1d0 2024-09-17T04:32:46.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.660+0000 7f6189e00640 1 --2- 172.21.15.100:0/2951391100 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6168079250 0x7f616807b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:46.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.660+0000 7f6189e00640 1 --2- 172.21.15.100:0/2951391100 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6168079250 0x7f616807b760 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6178034000 tx=0x7f6178004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:46.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.663+0000 7f617e000640 1 -- 172.21.15.100:0/2951391100 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6174063140 con 0x7f61840cd1d0 2024-09-17T04:32:46.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.880+0000 7f618b200640 1 -- 172.21.15.100:0/2951391100 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol57370", "target": ["mon-mgr", ""]}) -- 0x7f6184144790 con 0x7f6168079250 2024-09-17T04:32:46.886 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:46.887+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:32:46.887 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:46.887+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:32:46.887 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:46.887+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:32:46.887 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:46.887+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:32:46.887 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:46.887+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:32:46.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.917+0000 7f617e000640 1 -- 172.21.15.100:0/2951391100 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7f6184144790 con 0x7f6168079250 2024-09-17T04:32:46.917 INFO:teuthology.orchestra.run.smithi100.stdout:/volumes/_nogroup/subvol57370 2024-09-17T04:32:46.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.920+0000 7f618b200640 1 -- 172.21.15.100:0/2951391100 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6168079250 msgr2=0x7f616807b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:46.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.920+0000 7f618b200640 1 --2- 172.21.15.100:0/2951391100 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6168079250 0x7f616807b760 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6178034000 tx=0x7f6178004730 comp rx=0 tx=0).stop 2024-09-17T04:32:46.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.920+0000 7f618b200640 1 -- 172.21.15.100:0/2951391100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61840cd1d0 msgr2=0x7f618415bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:46.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.920+0000 7f618b200640 1 --2- 172.21.15.100:0/2951391100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61840cd1d0 0x7f618415bcc0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f617400c500 tx=0x7f617400c900 comp rx=0 tx=0).stop 2024-09-17T04:32:46.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.920+0000 7f618b200640 1 -- 172.21.15.100:0/2951391100 shutdown_connections 2024-09-17T04:32:46.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.920+0000 7f618b200640 1 --2- 172.21.15.100:0/2951391100 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6168079250 0x7f616807b760 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:46.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.920+0000 7f618b200640 1 --2- 172.21.15.100:0/2951391100 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f618415d6e0 0x7f61841f58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:46.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.920+0000 7f618b200640 1 --2- 172.21.15.100:0/2951391100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61840cd1d0 0x7f618415bcc0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:46.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.920+0000 7f618b200640 1 --2- 172.21.15.100:0/2951391100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61840b3f30 0x7f618415b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:46.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.920+0000 7f618b200640 1 -- 172.21.15.100:0/2951391100 >> 172.21.15.100:0/2951391100 conn(0x7f618406d620 msgr2=0x7f61840afe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:46.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.920+0000 7f618b200640 1 -- 172.21.15.100:0/2951391100 shutdown_connections 2024-09-17T04:32:46.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:46.920+0000 7f618b200640 1 -- 172.21.15.100:0/2951391100 wait complete. 2024-09-17T04:32:46.941 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol83781 subvol_grp92870 2024-09-17T04:32:47.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.167+0000 7fe421a00640 1 -- 172.21.15.100:0/832589333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe41c06e2f0 msgr2=0x7fe41c0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:47.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.167+0000 7fe421a00640 1 --2- 172.21.15.100:0/832589333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe41c06e2f0 0x7fe41c0b4de0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fe41c074cf0 tx=0x7fe41402a790 comp rx=0 tx=0).stop 2024-09-17T04:32:47.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.168+0000 7fe421a00640 1 -- 172.21.15.100:0/832589333 shutdown_connections 2024-09-17T04:32:47.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.168+0000 7fe421a00640 1 --2- 172.21.15.100:0/832589333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe41c06e2f0 0x7fe41c0b4de0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:47.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.168+0000 7fe421a00640 1 --2- 172.21.15.100:0/832589333 >> v2:172.21.15.184:3300/0 conn(0x7fe41c15c740 0x7fe41c06ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:47.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.168+0000 7fe421a00640 1 -- 172.21.15.100:0/832589333 >> 172.21.15.100:0/832589333 conn(0x7fe41c073890 msgr2=0x7fe41c073ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:47.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.168+0000 7fe421a00640 1 -- 172.21.15.100:0/832589333 shutdown_connections 2024-09-17T04:32:47.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.168+0000 7fe421a00640 1 -- 172.21.15.100:0/832589333 wait complete. 2024-09-17T04:32:47.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.170+0000 7fe421a00640 1 Processor -- start 2024-09-17T04:32:47.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.170+0000 7fe421a00640 1 -- start start 2024-09-17T04:32:47.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.172+0000 7fe421a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe41c06e2f0 0x7fe41c1e48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:47.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.172+0000 7fe421a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe41c152990 0x7fe41c140450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:47.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.172+0000 7fe421a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe41c15c740 0x7fe41c157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:47.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.172+0000 7fe421a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c0b54a0 con 0x7fe41c15c740 2024-09-17T04:32:47.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.172+0000 7fe421a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c0b5320 con 0x7fe41c152990 2024-09-17T04:32:47.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.172+0000 7fe421a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe41c0b5620 con 0x7fe41c06e2f0 2024-09-17T04:32:47.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.172+0000 7fe41b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe41c15c740 0x7fe41c157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:47.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.172+0000 7fe41b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe41c15c740 0x7fe41c157f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35030/0 (socket says 172.21.15.100:35030) 2024-09-17T04:32:47.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.172+0000 7fe41b400640 1 -- 172.21.15.100:0/3139637176 learned_addr learned my addr 172.21.15.100:0/3139637176 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:47.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.172+0000 7fe41aa00640 1 --2- 172.21.15.100:0/3139637176 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe41c06e2f0 0x7fe41c1e48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:47.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.172+0000 7fe41a000640 1 --2- 172.21.15.100:0/3139637176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe41c152990 0x7fe41c140450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:47.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.172+0000 7fe41b400640 1 -- 172.21.15.100:0/3139637176 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe41c06e2f0 msgr2=0x7fe41c1e48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:47.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.172+0000 7fe41b400640 1 --2- 172.21.15.100:0/3139637176 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe41c06e2f0 0x7fe41c1e48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:47.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.172+0000 7fe41b400640 1 -- 172.21.15.100:0/3139637176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe41c152990 msgr2=0x7fe41c140450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:47.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.172+0000 7fe41b400640 1 --2- 172.21.15.100:0/3139637176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe41c152990 0x7fe41c140450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:47.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.172+0000 7fe41b400640 1 -- 172.21.15.100:0/3139637176 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41c1e4df0 con 0x7fe41c15c740 2024-09-17T04:32:47.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.173+0000 7fe41b400640 1 --2- 172.21.15.100:0/3139637176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe41c15c740 0x7fe41c157f60 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fe414034040 tx=0x7fe414032e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:47.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.173+0000 7fe412a00640 1 -- 172.21.15.100:0/3139637176 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414041020 con 0x7fe41c15c740 2024-09-17T04:32:47.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.173+0000 7fe412a00640 1 -- 172.21.15.100:0/3139637176 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe414004130 con 0x7fe41c15c740 2024-09-17T04:32:47.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.173+0000 7fe412a00640 1 -- 172.21.15.100:0/3139637176 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414036040 con 0x7fe41c15c740 2024-09-17T04:32:47.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.173+0000 7fe421a00640 1 -- 172.21.15.100:0/3139637176 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe41c153c00 con 0x7fe41c15c740 2024-09-17T04:32:47.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.173+0000 7fe421a00640 1 -- 172.21.15.100:0/3139637176 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe41c1e43d0 con 0x7fe41c15c740 2024-09-17T04:32:47.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.174+0000 7fe421a00640 1 -- 172.21.15.100:0/3139637176 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe41c0b4de0 con 0x7fe41c15c740 2024-09-17T04:32:47.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.175+0000 7fe412a00640 1 -- 172.21.15.100:0/3139637176 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7fe414044020 con 0x7fe41c15c740 2024-09-17T04:32:47.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.176+0000 7fe412a00640 1 --2- 172.21.15.100:0/3139637176 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe3fc079250 0x7fe3fc07b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:47.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.176+0000 7fe412a00640 1 -- 172.21.15.100:0/3139637176 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7fe4140bc9b0 con 0x7fe41c15c740 2024-09-17T04:32:47.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.177+0000 7fe41aa00640 1 --2- 172.21.15.100:0/3139637176 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe3fc079250 0x7fe3fc07b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:47.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.177+0000 7fe41aa00640 1 --2- 172.21.15.100:0/3139637176 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe3fc079250 0x7fe3fc07b760 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe40c000f60 tx=0x7fe40c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:47.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.181+0000 7fe412a00640 1 -- 172.21.15.100:0/3139637176 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe41404b030 con 0x7fe41c15c740 2024-09-17T04:32:47.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.391+0000 7fe421a00640 1 -- 172.21.15.100:0/3139637176 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol83781", "group_name": "subvol_grp92870", "target": ["mon-mgr", ""]}) -- 0x7fe41c071670 con 0x7fe3fc079250 2024-09-17T04:32:47.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.409+0000 7fe412a00640 1 -- 172.21.15.100:0/3139637176 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe41c071670 con 0x7fe3fc079250 2024-09-17T04:32:47.409 INFO:teuthology.orchestra.run.smithi100.stdout:/volumes/subvol_grp92870/subvol83781 2024-09-17T04:32:47.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.412+0000 7fe421a00640 1 -- 172.21.15.100:0/3139637176 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe3fc079250 msgr2=0x7fe3fc07b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:47.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.412+0000 7fe421a00640 1 --2- 172.21.15.100:0/3139637176 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe3fc079250 0x7fe3fc07b760 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe40c000f60 tx=0x7fe40c009040 comp rx=0 tx=0).stop 2024-09-17T04:32:47.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.412+0000 7fe421a00640 1 -- 172.21.15.100:0/3139637176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe41c15c740 msgr2=0x7fe41c157f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:47.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.412+0000 7fe421a00640 1 --2- 172.21.15.100:0/3139637176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe41c15c740 0x7fe41c157f60 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fe414034040 tx=0x7fe414032e90 comp rx=0 tx=0).stop 2024-09-17T04:32:47.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.412+0000 7fe421a00640 1 -- 172.21.15.100:0/3139637176 shutdown_connections 2024-09-17T04:32:47.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.412+0000 7fe421a00640 1 --2- 172.21.15.100:0/3139637176 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe3fc079250 0x7fe3fc07b760 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:47.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.412+0000 7fe421a00640 1 --2- 172.21.15.100:0/3139637176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe41c15c740 0x7fe41c157f60 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:47.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.412+0000 7fe421a00640 1 --2- 172.21.15.100:0/3139637176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe41c152990 0x7fe41c140450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:47.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.412+0000 7fe421a00640 1 --2- 172.21.15.100:0/3139637176 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe41c06e2f0 0x7fe41c1e48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:47.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.412+0000 7fe421a00640 1 -- 172.21.15.100:0/3139637176 >> 172.21.15.100:0/3139637176 conn(0x7fe41c073890 msgr2=0x7fe41c070500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:47.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.412+0000 7fe421a00640 1 -- 172.21.15.100:0/3139637176 shutdown_connections 2024-09-17T04:32:47.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.412+0000 7fe421a00640 1 -- 172.21.15.100:0/3139637176 wait complete. 2024-09-17T04:32:47.433 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol57370 2024-09-17T04:32:47.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.658+0000 7f6529800640 1 -- 172.21.15.100:0/2896806514 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6524073a80 msgr2=0x7f65240cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:47.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.658+0000 7f6529800640 1 --2- 172.21.15.100:0/2896806514 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6524073a80 0x7f65240cd610 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f651c009830 tx=0x7f651c02a5a0 comp rx=0 tx=0).stop 2024-09-17T04:32:47.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.658+0000 7f6529800640 1 -- 172.21.15.100:0/2896806514 shutdown_connections 2024-09-17T04:32:47.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.658+0000 7f6529800640 1 --2- 172.21.15.100:0/2896806514 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6524073a80 0x7f65240cd610 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:47.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.658+0000 7f6529800640 1 --2- 172.21.15.100:0/2896806514 >> v2:172.21.15.184:3300/0 conn(0x7f6524145a40 0x7f6524073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:47.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.658+0000 7f6529800640 1 -- 172.21.15.100:0/2896806514 >> 172.21.15.100:0/2896806514 conn(0x7f652406d1c0 msgr2=0x7f652406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:47.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.658+0000 7f6529800640 1 -- 172.21.15.100:0/2896806514 shutdown_connections 2024-09-17T04:32:47.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.659+0000 7f6529800640 1 -- 172.21.15.100:0/2896806514 wait complete. 2024-09-17T04:32:47.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.661+0000 7f6529800640 1 Processor -- start 2024-09-17T04:32:47.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.661+0000 7f6529800640 1 -- start start 2024-09-17T04:32:47.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.662+0000 7f6529800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6524073a80 0x7f65241e6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:47.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.662+0000 7f6529800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6524145550 0x7f652413b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:47.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.662+0000 7f6529800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6524145a40 0x7f652415dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:47.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.662+0000 7f6529800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f65240ce160 con 0x7f6524145a40 2024-09-17T04:32:47.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.662+0000 7f6529800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f65240cdfe0 con 0x7f6524145550 2024-09-17T04:32:47.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.662+0000 7f6529800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f65240ce2e0 con 0x7f6524073a80 2024-09-17T04:32:47.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.663+0000 7f6523400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6524145a40 0x7f652415dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:47.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.663+0000 7f6523400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6524145a40 0x7f652415dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35036/0 (socket says 172.21.15.100:35036) 2024-09-17T04:32:47.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.663+0000 7f6523400640 1 -- 172.21.15.100:0/29122453 learned_addr learned my addr 172.21.15.100:0/29122453 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:47.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.663+0000 7f651a000640 1 --2- 172.21.15.100:0/29122453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6524145550 0x7f652413b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:47.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.663+0000 7f6522a00640 1 --2- 172.21.15.100:0/29122453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6524073a80 0x7f65241e6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:47.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.663+0000 7f6523400640 1 -- 172.21.15.100:0/29122453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6524073a80 msgr2=0x7f65241e6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:47.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.663+0000 7f6523400640 1 --2- 172.21.15.100:0/29122453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6524073a80 0x7f65241e6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:47.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.663+0000 7f6523400640 1 -- 172.21.15.100:0/29122453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6524145550 msgr2=0x7f652413b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:47.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.663+0000 7f6523400640 1 --2- 172.21.15.100:0/29122453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6524145550 0x7f652413b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:47.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.663+0000 7f6523400640 1 -- 172.21.15.100:0/29122453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65241e8aa0 con 0x7f6524145a40 2024-09-17T04:32:47.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.663+0000 7f6523400640 1 --2- 172.21.15.100:0/29122453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6524145a40 0x7f652415dfe0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f651c009950 tx=0x7f651c033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:47.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.664+0000 7f651b400640 1 -- 172.21.15.100:0/29122453 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c031b50 con 0x7f6524145a40 2024-09-17T04:32:47.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.664+0000 7f6529800640 1 -- 172.21.15.100:0/29122453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65241e8d80 con 0x7f6524145a40 2024-09-17T04:32:47.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.664+0000 7f651b400640 1 -- 172.21.15.100:0/29122453 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f651c031de0 con 0x7f6524145a40 2024-09-17T04:32:47.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.664+0000 7f651b400640 1 -- 172.21.15.100:0/29122453 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c005280 con 0x7f6524145a40 2024-09-17T04:32:47.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.664+0000 7f6529800640 1 -- 172.21.15.100:0/29122453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f652415d7e0 con 0x7f6524145a40 2024-09-17T04:32:47.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.665+0000 7f6529800640 1 -- 172.21.15.100:0/29122453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65240ce790 con 0x7f6524145a40 2024-09-17T04:32:47.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.666+0000 7f651b400640 1 -- 172.21.15.100:0/29122453 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f651c005420 con 0x7f6524145a40 2024-09-17T04:32:47.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.667+0000 7f651b400640 1 --2- 172.21.15.100:0/29122453 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6500079250 0x7f650007b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:47.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.667+0000 7f651b400640 1 -- 172.21.15.100:0/29122453 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7f651c0bce10 con 0x7f6524145a40 2024-09-17T04:32:47.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.667+0000 7f6522a00640 1 --2- 172.21.15.100:0/29122453 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6500079250 0x7f650007b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:47.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.668+0000 7f6522a00640 1 --2- 172.21.15.100:0/29122453 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6500079250 0x7f650007b760 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f65140020b0 tx=0x7f6514009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:47.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.671+0000 7f651b400640 1 -- 172.21.15.100:0/29122453 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f651c03f030 con 0x7f6524145a40 2024-09-17T04:32:47.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.882+0000 7f6529800640 1 -- 172.21.15.100:0/29122453 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol57370", "target": ["mon-mgr", ""]}) -- 0x7f6524140bd0 con 0x7f6500079250 2024-09-17T04:32:47.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.888+0000 7f651b400640 1 -- 172.21.15.100:0/29122453 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7f6524140bd0 con 0x7f6500079250 2024-09-17T04:32:47.888 INFO:teuthology.orchestra.run.smithi100.stdout:/volumes/_nogroup/subvol57370 2024-09-17T04:32:47.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.891+0000 7f6529800640 1 -- 172.21.15.100:0/29122453 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6500079250 msgr2=0x7f650007b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:47.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.891+0000 7f6529800640 1 --2- 172.21.15.100:0/29122453 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6500079250 0x7f650007b760 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f65140020b0 tx=0x7f6514009040 comp rx=0 tx=0).stop 2024-09-17T04:32:47.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.891+0000 7f6529800640 1 -- 172.21.15.100:0/29122453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6524145a40 msgr2=0x7f652415dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:47.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.891+0000 7f6529800640 1 --2- 172.21.15.100:0/29122453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6524145a40 0x7f652415dfe0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f651c009950 tx=0x7f651c033ba0 comp rx=0 tx=0).stop 2024-09-17T04:32:47.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.891+0000 7f6529800640 1 -- 172.21.15.100:0/29122453 shutdown_connections 2024-09-17T04:32:47.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.891+0000 7f6529800640 1 --2- 172.21.15.100:0/29122453 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6500079250 0x7f650007b760 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:47.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.891+0000 7f6529800640 1 --2- 172.21.15.100:0/29122453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6524145a40 0x7f652415dfe0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:47.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.891+0000 7f6529800640 1 --2- 172.21.15.100:0/29122453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6524145550 0x7f652413b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:47.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.891+0000 7f6529800640 1 --2- 172.21.15.100:0/29122453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6524073a80 0x7f65241e6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:47.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.891+0000 7f6529800640 1 -- 172.21.15.100:0/29122453 >> 172.21.15.100:0/29122453 conn(0x7f652406d1c0 msgr2=0x7f65240b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:47.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.891+0000 7f6529800640 1 -- 172.21.15.100:0/29122453 shutdown_connections 2024-09-17T04:32:47.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:47.891+0000 7f6529800640 1 -- 172.21.15.100:0/29122453 wait complete. 2024-09-17T04:32:47.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:47.913 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/83da34169df7ac4935cb36f80e400952.meta) 2024-09-17T04:32:50.035 INFO:teuthology.orchestra.run.smithi100.stdout:[GLOBAL] 2024-09-17T04:32:50.035 INFO:teuthology.orchestra.run.smithi100.stdout:version = 1 2024-09-17T04:32:50.035 INFO:teuthology.orchestra.run.smithi100.stdout:type = subvolume 2024-09-17T04:32:50.035 INFO:teuthology.orchestra.run.smithi100.stdout:path = /volumes/_nogroup/subvol57370 2024-09-17T04:32:50.035 INFO:teuthology.orchestra.run.smithi100.stdout:state = complete 2024-09-17T04:32:50.035 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:32:50.039 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol83781 subvol_grp92870 2024-09-17T04:32:50.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.274+0000 7fcbbca00640 1 -- 172.21.15.100:0/1499643609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbb80705b0 msgr2=0x7fcbb80b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:50.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.274+0000 7fcbbca00640 1 --2- 172.21.15.100:0/1499643609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbb80705b0 0x7fcbb80b47c0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80d0050 tx=0x7fcba402a4a0 comp rx=0 tx=0).stop 2024-09-17T04:32:50.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.274+0000 7fcbbca00640 1 -- 172.21.15.100:0/1499643609 shutdown_connections 2024-09-17T04:32:50.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.274+0000 7fcbbca00640 1 --2- 172.21.15.100:0/1499643609 >> v2:172.21.15.184:3300/0 conn(0x7fcbb80af3c0 0x7fcbb81602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:50.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.274+0000 7fcbbca00640 1 --2- 172.21.15.100:0/1499643609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbb80705b0 0x7fcbb80b47c0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:50.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.274+0000 7fcbbca00640 1 -- 172.21.15.100:0/1499643609 >> 172.21.15.100:0/1499643609 conn(0x7fcbb80b1dd0 msgr2=0x7fcbb80b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:50.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.274+0000 7fcbbca00640 1 -- 172.21.15.100:0/1499643609 shutdown_connections 2024-09-17T04:32:50.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.274+0000 7fcbbca00640 1 -- 172.21.15.100:0/1499643609 wait complete. 2024-09-17T04:32:50.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.276+0000 7fcbbca00640 1 Processor -- start 2024-09-17T04:32:50.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.277+0000 7fcbbca00640 1 -- start start 2024-09-17T04:32:50.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.278+0000 7fcbbca00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbb80705b0 0x7fcbb8145bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:50.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.278+0000 7fcbbca00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbb80af3c0 0x7fcbb809c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:50.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.278+0000 7fcbbca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbb80b3ec0 0x7fcbb8155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:50.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.278+0000 7fcbbca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb8160b80 con 0x7fcbb80b3ec0 2024-09-17T04:32:50.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.278+0000 7fcbbca00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb8160a00 con 0x7fcbb80af3c0 2024-09-17T04:32:50.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.278+0000 7fcbbca00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcbb815cc60 con 0x7fcbb80705b0 2024-09-17T04:32:50.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.278+0000 7fcbb7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbb80b3ec0 0x7fcbb8155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:50.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.278+0000 7fcbb7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbb80b3ec0 0x7fcbb8155e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35044/0 (socket says 172.21.15.100:35044) 2024-09-17T04:32:50.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.278+0000 7fcbb7200640 1 -- 172.21.15.100:0/143731463 learned_addr learned my addr 172.21.15.100:0/143731463 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:50.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.278+0000 7fcbb5e00640 1 --2- 172.21.15.100:0/143731463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbb80af3c0 0x7fcbb809c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:50.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.278+0000 7fcbb6800640 1 --2- 172.21.15.100:0/143731463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbb80705b0 0x7fcbb8145bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:50.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.279+0000 7fcbb7200640 1 -- 172.21.15.100:0/143731463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbb80705b0 msgr2=0x7fcbb8145bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:50.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.279+0000 7fcbb7200640 1 --2- 172.21.15.100:0/143731463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbb80705b0 0x7fcbb8145bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:50.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.279+0000 7fcbb7200640 1 -- 172.21.15.100:0/143731463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbb80af3c0 msgr2=0x7fcbb809c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:50.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.279+0000 7fcbb7200640 1 --2- 172.21.15.100:0/143731463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbb80af3c0 0x7fcbb809c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:50.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.279+0000 7fcbb7200640 1 -- 172.21.15.100:0/143731463 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb81f6f10 con 0x7fcbb80b3ec0 2024-09-17T04:32:50.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.279+0000 7fcbb7200640 1 --2- 172.21.15.100:0/143731463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbb80b3ec0 0x7fcbb8155e00 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0005d00 tx=0x7fcbb000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:50.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.279+0000 7fcbaea00640 1 -- 172.21.15.100:0/143731463 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0017070 con 0x7fcbb80b3ec0 2024-09-17T04:32:50.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.279+0000 7fcbaea00640 1 -- 172.21.15.100:0/143731463 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcbb0005330 con 0x7fcbb80b3ec0 2024-09-17T04:32:50.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.279+0000 7fcbaea00640 1 -- 172.21.15.100:0/143731463 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb00047a0 con 0x7fcbb80b3ec0 2024-09-17T04:32:50.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.279+0000 7fcbbca00640 1 -- 172.21.15.100:0/143731463 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb81f71f0 con 0x7fcbb80b3ec0 2024-09-17T04:32:50.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.279+0000 7fcbbca00640 1 -- 172.21.15.100:0/143731463 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb81f77b0 con 0x7fcbb80b3ec0 2024-09-17T04:32:50.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.280+0000 7fcbbca00640 1 -- 172.21.15.100:0/143731463 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbb81f6c80 con 0x7fcbb80b3ec0 2024-09-17T04:32:50.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.282+0000 7fcbaea00640 1 -- 172.21.15.100:0/143731463 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 102104+0+0 (secure 0 0 0) 0x7fcbb0004940 con 0x7fcbb80b3ec0 2024-09-17T04:32:50.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.283+0000 7fcbaea00640 1 --2- 172.21.15.100:0/143731463 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcb980792e0 0x7fcb9807b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:50.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.283+0000 7fcbaea00640 1 -- 172.21.15.100:0/143731463 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7fcbb009bff0 con 0x7fcbb80b3ec0 2024-09-17T04:32:50.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.283+0000 7fcbb6800640 1 --2- 172.21.15.100:0/143731463 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcb980792e0 0x7fcb9807b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:50.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.284+0000 7fcbb6800640 1 --2- 172.21.15.100:0/143731463 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcb980792e0 0x7fcb9807b7f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcba8007fd0 tx=0x7fcba8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:50.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.288+0000 7fcbaea00640 1 -- 172.21.15.100:0/143731463 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fcbb0062de0 con 0x7fcbb80b3ec0 2024-09-17T04:32:50.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.499+0000 7fcbbca00640 1 -- 172.21.15.100:0/143731463 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol83781", "group_name": "subvol_grp92870", "target": ["mon-mgr", ""]}) -- 0x7fcbb80b47c0 con 0x7fcb980792e0 2024-09-17T04:32:50.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.503+0000 7fcbaea00640 1 -- 172.21.15.100:0/143731463 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcbb80b47c0 con 0x7fcb980792e0 2024-09-17T04:32:50.503 INFO:teuthology.orchestra.run.smithi100.stdout:/volumes/subvol_grp92870/subvol83781 2024-09-17T04:32:50.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.505+0000 7fcbbca00640 1 -- 172.21.15.100:0/143731463 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcb980792e0 msgr2=0x7fcb9807b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:50.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.505+0000 7fcbbca00640 1 --2- 172.21.15.100:0/143731463 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcb980792e0 0x7fcb9807b7f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcba8007fd0 tx=0x7fcba8009040 comp rx=0 tx=0).stop 2024-09-17T04:32:50.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.505+0000 7fcbbca00640 1 -- 172.21.15.100:0/143731463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbb80b3ec0 msgr2=0x7fcbb8155e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:50.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.505+0000 7fcbbca00640 1 --2- 172.21.15.100:0/143731463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbb80b3ec0 0x7fcbb8155e00 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0005d00 tx=0x7fcbb000dae0 comp rx=0 tx=0).stop 2024-09-17T04:32:50.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.506+0000 7fcbbca00640 1 -- 172.21.15.100:0/143731463 shutdown_connections 2024-09-17T04:32:50.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.506+0000 7fcbbca00640 1 --2- 172.21.15.100:0/143731463 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcb980792e0 0x7fcb9807b7f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:50.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.506+0000 7fcbbca00640 1 --2- 172.21.15.100:0/143731463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbb80b3ec0 0x7fcbb8155e00 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:50.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.506+0000 7fcbbca00640 1 --2- 172.21.15.100:0/143731463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbb80af3c0 0x7fcbb809c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:50.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.506+0000 7fcbbca00640 1 --2- 172.21.15.100:0/143731463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbb80705b0 0x7fcbb8145bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:50.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.506+0000 7fcbbca00640 1 -- 172.21.15.100:0/143731463 >> 172.21.15.100:0/143731463 conn(0x7fcbb80b1dd0 msgr2=0x7fcbb80d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:50.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.506+0000 7fcbbca00640 1 -- 172.21.15.100:0/143731463 shutdown_connections 2024-09-17T04:32:50.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:50.506+0000 7fcbbca00640 1 -- 172.21.15.100:0/143731463 wait complete. 2024-09-17T04:32:50.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:50.529 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/9009a9ed1fdfb564a56b277d851b6390.meta) 2024-09-17T04:32:55.032 INFO:teuthology.orchestra.run.smithi100.stdout:[GLOBAL] 2024-09-17T04:32:55.032 INFO:teuthology.orchestra.run.smithi100.stdout:version = 1 2024-09-17T04:32:55.032 INFO:teuthology.orchestra.run.smithi100.stdout:type = subvolume 2024-09-17T04:32:55.032 INFO:teuthology.orchestra.run.smithi100.stdout:path = /volumes/subvol_grp92870/subvol83781 2024-09-17T04:32:55.032 INFO:teuthology.orchestra.run.smithi100.stdout:state = complete 2024-09-17T04:32:55.032 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:32:55.036 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol57370 2024-09-17T04:32:55.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.268+0000 7fa8df200640 1 -- 172.21.15.100:0/3483936513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8d8073c50 msgr2=0x7fa8d80cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.268+0000 7fa8df200640 1 --2- 172.21.15.100:0/3483936513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8d8073c50 0x7fa8d80cd610 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0005fd0 tx=0x7fa8d002a930 comp rx=0 tx=0).stop 2024-09-17T04:32:55.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.268+0000 7fa8df200640 1 -- 172.21.15.100:0/3483936513 shutdown_connections 2024-09-17T04:32:55.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.268+0000 7fa8df200640 1 --2- 172.21.15.100:0/3483936513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8d8073c50 0x7fa8d80cd610 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.268+0000 7fa8df200640 1 --2- 172.21.15.100:0/3483936513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8d8145550 0x7fa8d806fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.268+0000 7fa8df200640 1 -- 172.21.15.100:0/3483936513 >> 172.21.15.100:0/3483936513 conn(0x7fa8d806d1c0 msgr2=0x7fa8d806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:55.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.268+0000 7fa8df200640 1 -- 172.21.15.100:0/3483936513 shutdown_connections 2024-09-17T04:32:55.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.268+0000 7fa8df200640 1 -- 172.21.15.100:0/3483936513 wait complete. 2024-09-17T04:32:55.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.270+0000 7fa8df200640 1 Processor -- start 2024-09-17T04:32:55.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.270+0000 7fa8df200640 1 -- start start 2024-09-17T04:32:55.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.272+0000 7fa8df200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8d8145550 0x7fa8d81f3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:55.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.272+0000 7fa8df200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8d8145a40 0x7fa8d8152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:55.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.272+0000 7fa8df200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8d813c600 0x7fa8d815e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:55.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.272+0000 7fa8df200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d80ce1e0 con 0x7fa8d8145a40 2024-09-17T04:32:55.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.272+0000 7fa8df200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d80ce060 con 0x7fa8d8145550 2024-09-17T04:32:55.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.272+0000 7fa8df200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa8d80ce360 con 0x7fa8d813c600 2024-09-17T04:32:55.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.272+0000 7fa8d7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8d8145a40 0x7fa8d8152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:55.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.272+0000 7fa8d7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8d8145a40 0x7fa8d8152e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56666/0 (socket says 172.21.15.100:56666) 2024-09-17T04:32:55.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.272+0000 7fa8d7400640 1 -- 172.21.15.100:0/693036821 learned_addr learned my addr 172.21.15.100:0/693036821 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:55.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.272+0000 7fa8dd200640 1 --2- 172.21.15.100:0/693036821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8d813c600 0x7fa8d815e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:55.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.272+0000 7fa8d7e00640 1 --2- 172.21.15.100:0/693036821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8d8145550 0x7fa8d81f3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:55.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.273+0000 7fa8d7400640 1 -- 172.21.15.100:0/693036821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8d813c600 msgr2=0x7fa8d815e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.273+0000 7fa8d7400640 1 --2- 172.21.15.100:0/693036821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8d813c600 0x7fa8d815e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.273+0000 7fa8d7400640 1 -- 172.21.15.100:0/693036821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8d8145550 msgr2=0x7fa8d81f3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.273+0000 7fa8d7400640 1 --2- 172.21.15.100:0/693036821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8d8145550 0x7fa8d81f3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.273+0000 7fa8d7400640 1 -- 172.21.15.100:0/693036821 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d81f3c50 con 0x7fa8d8145a40 2024-09-17T04:32:55.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.273+0000 7fa8d7400640 1 --2- 172.21.15.100:0/693036821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8d8145a40 0x7fa8d8152e10 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400c3f0 tx=0x7fa8c400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:55.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.273+0000 7fa8cfe00640 1 -- 172.21.15.100:0/693036821 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4011790 con 0x7fa8d8145a40 2024-09-17T04:32:55.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.273+0000 7fa8cfe00640 1 -- 172.21.15.100:0/693036821 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8c4017070 con 0x7fa8d8145a40 2024-09-17T04:32:55.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.273+0000 7fa8cfe00640 1 -- 172.21.15.100:0/693036821 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4011ce0 con 0x7fa8d8145a40 2024-09-17T04:32:55.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.273+0000 7fa8df200640 1 -- 172.21.15.100:0/693036821 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8d815cfb0 con 0x7fa8d8145a40 2024-09-17T04:32:55.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.273+0000 7fa8df200640 1 -- 172.21.15.100:0/693036821 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8d815d570 con 0x7fa8d8145a40 2024-09-17T04:32:55.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.274+0000 7fa8df200640 1 -- 172.21.15.100:0/693036821 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8d806a7f0 con 0x7fa8d8145a40 2024-09-17T04:32:55.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.275+0000 7fa8cfe00640 1 -- 172.21.15.100:0/693036821 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 102104+0+0 (secure 0 0 0) 0x7fa8c401c460 con 0x7fa8d8145a40 2024-09-17T04:32:55.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.276+0000 7fa8cfe00640 1 --2- 172.21.15.100:0/693036821 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa8b80792e0 0x7fa8b807b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:55.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.277+0000 7fa8cfe00640 1 -- 172.21.15.100:0/693036821 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7fa8c409bba0 con 0x7fa8d8145a40 2024-09-17T04:32:55.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.277+0000 7fa8d7e00640 1 --2- 172.21.15.100:0/693036821 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa8b80792e0 0x7fa8b807b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:55.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.277+0000 7fa8d7e00640 1 --2- 172.21.15.100:0/693036821 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa8b80792e0 0x7fa8b807b7f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa8c80095f0 tx=0x7fa8c8005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:55.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.281+0000 7fa8cfe00640 1 -- 172.21.15.100:0/693036821 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa8c4062990 con 0x7fa8d8145a40 2024-09-17T04:32:55.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.489+0000 7fa8df200640 1 -- 172.21.15.100:0/693036821 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol57370", "target": ["mon-mgr", ""]}) -- 0x7fa8d815cd60 con 0x7fa8b80792e0 2024-09-17T04:32:55.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.517+0000 7fa8cfe00640 1 -- 172.21.15.100:0/693036821 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa8d815cd60 con 0x7fa8b80792e0 2024-09-17T04:32:55.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.520+0000 7fa8df200640 1 -- 172.21.15.100:0/693036821 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa8b80792e0 msgr2=0x7fa8b807b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.520+0000 7fa8df200640 1 --2- 172.21.15.100:0/693036821 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa8b80792e0 0x7fa8b807b7f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa8c80095f0 tx=0x7fa8c8005f90 comp rx=0 tx=0).stop 2024-09-17T04:32:55.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.520+0000 7fa8df200640 1 -- 172.21.15.100:0/693036821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8d8145a40 msgr2=0x7fa8d8152e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.520+0000 7fa8df200640 1 --2- 172.21.15.100:0/693036821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8d8145a40 0x7fa8d8152e10 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400c3f0 tx=0x7fa8c400c8b0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.520+0000 7fa8df200640 1 -- 172.21.15.100:0/693036821 shutdown_connections 2024-09-17T04:32:55.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.520+0000 7fa8df200640 1 --2- 172.21.15.100:0/693036821 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa8b80792e0 0x7fa8b807b7f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.520+0000 7fa8df200640 1 --2- 172.21.15.100:0/693036821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8d813c600 0x7fa8d815e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.520+0000 7fa8df200640 1 --2- 172.21.15.100:0/693036821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8d8145a40 0x7fa8d8152e10 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.520+0000 7fa8df200640 1 --2- 172.21.15.100:0/693036821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8d8145550 0x7fa8d81f3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.520+0000 7fa8df200640 1 -- 172.21.15.100:0/693036821 >> 172.21.15.100:0/693036821 conn(0x7fa8d806d1c0 msgr2=0x7fa8d813fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:55.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.520+0000 7fa8df200640 1 -- 172.21.15.100:0/693036821 shutdown_connections 2024-09-17T04:32:55.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.520+0000 7fa8df200640 1 -- 172.21.15.100:0/693036821 wait complete. 2024-09-17T04:32:55.521 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:55.522+0000 7f30b4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:32:55.521 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:55.522+0000 7f30b4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:32:55.521 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:55.522+0000 7f30b4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:32:55.521 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:55.522+0000 7f30b4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:32:55.521 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:55.522+0000 7f30b4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:32:55.541 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:55.542+0000 7f30b5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:32:55.541 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:55.542+0000 7f30b5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:32:55.542 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:55.542+0000 7f30b5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:32:55.542 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:55.542+0000 7f30b5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:32:55.542 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:32:55.542+0000 7f30b5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:32:55.547 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol83781 subvol_grp92870 2024-09-17T04:32:55.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.774+0000 7f79c0800640 1 -- 172.21.15.100:0/689572495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79b80d01b0 msgr2=0x7f79b80d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.774+0000 7f79c0800640 1 --2- 172.21.15.100:0/689572495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79b80d01b0 0x7f79b80d05c0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f79b8051f20 tx=0x7f79ac02a540 comp rx=0 tx=0).stop 2024-09-17T04:32:55.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.775+0000 7f79c0800640 1 -- 172.21.15.100:0/689572495 shutdown_connections 2024-09-17T04:32:55.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.775+0000 7f79c0800640 1 --2- 172.21.15.100:0/689572495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79b80d0ee0 0x7f79b80d12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.775+0000 7f79c0800640 1 --2- 172.21.15.100:0/689572495 >> v2:172.21.15.184:3300/0 conn(0x7f79b80b5250 0x7f79b80b5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.775+0000 7f79c0800640 1 --2- 172.21.15.100:0/689572495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79b80d01b0 0x7f79b80d05c0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.775+0000 7f79c0800640 1 -- 172.21.15.100:0/689572495 >> 172.21.15.100:0/689572495 conn(0x7f79b80cee80 msgr2=0x7f79b80cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:55.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.775+0000 7f79c0800640 1 -- 172.21.15.100:0/689572495 shutdown_connections 2024-09-17T04:32:55.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.775+0000 7f79c0800640 1 -- 172.21.15.100:0/689572495 wait complete. 2024-09-17T04:32:55.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.777+0000 7f79c0800640 1 Processor -- start 2024-09-17T04:32:55.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.777+0000 7f79c0800640 1 -- start start 2024-09-17T04:32:55.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.779+0000 7f79c0800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79b80b5250 0x7f79b81f6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:55.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.779+0000 7f79c0800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79b80d01b0 0x7f79b81536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:55.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.779+0000 7f79c0800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79b80d0ee0 0x7f79b81f8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:55.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.779+0000 7f79c0800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f79b809bc70 con 0x7f79b80b5250 2024-09-17T04:32:55.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.779+0000 7f79c0800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f79b809baf0 con 0x7f79b80d01b0 2024-09-17T04:32:55.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.779+0000 7f79c0800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f79b8160a30 con 0x7f79b80d0ee0 2024-09-17T04:32:55.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.779+0000 7f79bde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79b80b5250 0x7f79b81f6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:55.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.779+0000 7f79bde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79b80b5250 0x7f79b81f6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56728/0 (socket says 172.21.15.100:56728) 2024-09-17T04:32:55.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.779+0000 7f79bde00640 1 -- 172.21.15.100:0/354460026 learned_addr learned my addr 172.21.15.100:0/354460026 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:55.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.779+0000 7f79be800640 1 --2- 172.21.15.100:0/354460026 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79b80d0ee0 0x7f79b81f8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:55.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.779+0000 7f79bd400640 1 --2- 172.21.15.100:0/354460026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79b80d01b0 0x7f79b81536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:55.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.780+0000 7f79bde00640 1 -- 172.21.15.100:0/354460026 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79b80d0ee0 msgr2=0x7f79b81f8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.780+0000 7f79bde00640 1 --2- 172.21.15.100:0/354460026 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79b80d0ee0 0x7f79b81f8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.780+0000 7f79bde00640 1 -- 172.21.15.100:0/354460026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79b80d01b0 msgr2=0x7f79b81536b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.780+0000 7f79bde00640 1 --2- 172.21.15.100:0/354460026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79b80d01b0 0x7f79b81536b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.780+0000 7f79bde00640 1 -- 172.21.15.100:0/354460026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b81f68a0 con 0x7f79b80b5250 2024-09-17T04:32:55.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.780+0000 7f79be800640 1 --2- 172.21.15.100:0/354460026 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79b80d0ee0 0x7f79b81f8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:55.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.780+0000 7f79bde00640 1 --2- 172.21.15.100:0/354460026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79b80b5250 0x7f79b81f6360 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0099d0 tx=0x7f79ac0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:55.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.780+0000 7f79b2000640 1 -- 172.21.15.100:0/354460026 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac005cf0 con 0x7f79b80b5250 2024-09-17T04:32:55.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.780+0000 7f79b2000640 1 -- 172.21.15.100:0/354460026 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79ac031c40 con 0x7f79b80b5250 2024-09-17T04:32:55.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.780+0000 7f79c0800640 1 -- 172.21.15.100:0/354460026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b80b73c0 con 0x7f79b80b5250 2024-09-17T04:32:55.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.780+0000 7f79b2000640 1 -- 172.21.15.100:0/354460026 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac033330 con 0x7f79b80b5250 2024-09-17T04:32:55.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.780+0000 7f79c0800640 1 -- 172.21.15.100:0/354460026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b81604d0 con 0x7f79b80b5250 2024-09-17T04:32:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.782+0000 7f79c0800640 1 -- 172.21.15.100:0/354460026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79b80d05c0 con 0x7f79b80b5250 2024-09-17T04:32:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.783+0000 7f79b2000640 1 -- 172.21.15.100:0/354460026 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 102104+0+0 (secure 0 0 0) 0x7f79ac0334d0 con 0x7f79b80b5250 2024-09-17T04:32:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.785+0000 7f79b2000640 1 --2- 172.21.15.100:0/354460026 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f799c0792e0 0x7f799c07b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.785+0000 7f79b2000640 1 -- 172.21.15.100:0/354460026 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7f79ac0bd430 con 0x7f79b80b5250 2024-09-17T04:32:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.785+0000 7f79bd400640 1 --2- 172.21.15.100:0/354460026 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f799c0792e0 0x7f799c07b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.786+0000 7f79bd400640 1 --2- 172.21.15.100:0/354460026 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f799c0792e0 0x7f799c07b7f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f79b815eb50 tx=0x7f79a8005d70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:55.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.789+0000 7f79b2000640 1 -- 172.21.15.100:0/354460026 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f79b81f68a0 con 0x7f79b80b5250 2024-09-17T04:32:55.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:55.998+0000 7f79c0800640 1 -- 172.21.15.100:0/354460026 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol83781", "group_name": "subvol_grp92870", "target": ["mon-mgr", ""]}) -- 0x7f79b806e050 con 0x7f799c0792e0 2024-09-17T04:32:56.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.027+0000 7f79b2000640 1 -- 172.21.15.100:0/354460026 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f79b806e050 con 0x7f799c0792e0 2024-09-17T04:32:56.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.029+0000 7f79c0800640 1 -- 172.21.15.100:0/354460026 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f799c0792e0 msgr2=0x7f799c07b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:56.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.029+0000 7f79c0800640 1 --2- 172.21.15.100:0/354460026 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f799c0792e0 0x7f799c07b7f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f79b815eb50 tx=0x7f79a8005d70 comp rx=0 tx=0).stop 2024-09-17T04:32:56.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.029+0000 7f79c0800640 1 -- 172.21.15.100:0/354460026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79b80b5250 msgr2=0x7f79b81f6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:56.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.029+0000 7f79c0800640 1 --2- 172.21.15.100:0/354460026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79b80b5250 0x7f79b81f6360 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0099d0 tx=0x7f79ac0052f0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.029+0000 7f79c0800640 1 -- 172.21.15.100:0/354460026 shutdown_connections 2024-09-17T04:32:56.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.029+0000 7f79c0800640 1 --2- 172.21.15.100:0/354460026 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f799c0792e0 0x7f799c07b7f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.029+0000 7f79c0800640 1 --2- 172.21.15.100:0/354460026 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79b80d0ee0 0x7f79b81f8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.029+0000 7f79c0800640 1 --2- 172.21.15.100:0/354460026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79b80d01b0 0x7f79b81536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.029+0000 7f79c0800640 1 --2- 172.21.15.100:0/354460026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79b80b5250 0x7f79b81f6360 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.029+0000 7f79c0800640 1 -- 172.21.15.100:0/354460026 >> 172.21.15.100:0/354460026 conn(0x7f79b80cee80 msgr2=0x7f79b81557c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:56.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.030+0000 7f79c0800640 1 -- 172.21.15.100:0/354460026 shutdown_connections 2024-09-17T04:32:56.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.030+0000 7f79c0800640 1 -- 172.21.15.100:0/354460026 wait complete. 2024-09-17T04:32:56.055 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-17T04:32:56.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:32:56.140 INFO:teuthology.orchestra.run.smithi100.stdout:2 2024-09-17T04:32:56.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:32:56.149 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp92870 2024-09-17T04:32:56.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.372+0000 7f8c18c00640 1 -- 172.21.15.100:0/2479536670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c14145a40 msgr2=0x7f8c14073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:56.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.372+0000 7f8c18c00640 1 --2- 172.21.15.100:0/2479536670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c14145a40 0x7f8c14073540 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8c00005fd0 tx=0x7f8c0002a660 comp rx=0 tx=0).stop 2024-09-17T04:32:56.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.372+0000 7f8c18c00640 1 -- 172.21.15.100:0/2479536670 shutdown_connections 2024-09-17T04:32:56.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.372+0000 7f8c18c00640 1 --2- 172.21.15.100:0/2479536670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c14073a80 0x7f8c140cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.372+0000 7f8c18c00640 1 --2- 172.21.15.100:0/2479536670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c14145a40 0x7f8c14073540 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.372+0000 7f8c18c00640 1 -- 172.21.15.100:0/2479536670 >> 172.21.15.100:0/2479536670 conn(0x7f8c1406d1c0 msgr2=0x7f8c1406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:56.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.372+0000 7f8c18c00640 1 -- 172.21.15.100:0/2479536670 shutdown_connections 2024-09-17T04:32:56.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.372+0000 7f8c18c00640 1 -- 172.21.15.100:0/2479536670 wait complete. 2024-09-17T04:32:56.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.374+0000 7f8c18c00640 1 Processor -- start 2024-09-17T04:32:56.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.375+0000 7f8c18c00640 1 -- start start 2024-09-17T04:32:56.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.376+0000 7f8c18c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c14073a80 0x7f8c141f9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:56.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.376+0000 7f8c18c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c14145550 0x7f8c141f9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:56.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.376+0000 7f8c18c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c14145a40 0x7f8c140aea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:56.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.376+0000 7f8c18c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8c140ce160 con 0x7f8c14145550 2024-09-17T04:32:56.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.376+0000 7f8c18c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8c140cdfe0 con 0x7f8c14073a80 2024-09-17T04:32:56.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.376+0000 7f8c18c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8c140ce2e0 con 0x7f8c14145a40 2024-09-17T04:32:56.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.376+0000 7f8c11400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c14145550 0x7f8c141f9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:56.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.376+0000 7f8c11400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c14145550 0x7f8c141f9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56742/0 (socket says 172.21.15.100:56742) 2024-09-17T04:32:56.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.376+0000 7f8c11400640 1 -- 172.21.15.100:0/349897710 learned_addr learned my addr 172.21.15.100:0/349897710 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:56.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.377+0000 7f8c12800640 1 --2- 172.21.15.100:0/349897710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c14145a40 0x7f8c140aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:56.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.377+0000 7f8c11e00640 1 --2- 172.21.15.100:0/349897710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c14073a80 0x7f8c141f9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:56.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.377+0000 7f8c11400640 1 -- 172.21.15.100:0/349897710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c14145a40 msgr2=0x7f8c140aea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:56.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.377+0000 7f8c11400640 1 --2- 172.21.15.100:0/349897710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c14145a40 0x7f8c140aea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.377+0000 7f8c11400640 1 -- 172.21.15.100:0/349897710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c14073a80 msgr2=0x7f8c141f9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:56.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.377+0000 7f8c11400640 1 --2- 172.21.15.100:0/349897710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c14073a80 0x7f8c141f9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.377+0000 7f8c11400640 1 -- 172.21.15.100:0/349897710 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c141fa540 con 0x7f8c14145550 2024-09-17T04:32:56.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.377+0000 7f8c11e00640 1 --2- 172.21.15.100:0/349897710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c14073a80 0x7f8c141f9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:56.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.377+0000 7f8c11400640 1 --2- 172.21.15.100:0/349897710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c14145550 0x7f8c141f9d90 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f8c00006010 tx=0x7f8c000038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:56.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.377+0000 7f8c0a000640 1 -- 172.21.15.100:0/349897710 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0003e030 con 0x7f8c14145550 2024-09-17T04:32:56.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.378+0000 7f8c0a000640 1 -- 172.21.15.100:0/349897710 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c00003d90 con 0x7f8c14145550 2024-09-17T04:32:56.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.378+0000 7f8c0a000640 1 -- 172.21.15.100:0/349897710 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c00035040 con 0x7f8c14145550 2024-09-17T04:32:56.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.378+0000 7f8c18c00640 1 -- 172.21.15.100:0/349897710 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c141fa820 con 0x7f8c14145550 2024-09-17T04:32:56.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.378+0000 7f8c18c00640 1 -- 172.21.15.100:0/349897710 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c141fade0 con 0x7f8c14145550 2024-09-17T04:32:56.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.379+0000 7f8c18c00640 1 -- 172.21.15.100:0/349897710 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c140ce790 con 0x7f8c14145550 2024-09-17T04:32:56.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.380+0000 7f8c0a000640 1 -- 172.21.15.100:0/349897710 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 102104+0+0 (secure 0 0 0) 0x7f8c00041030 con 0x7f8c14145550 2024-09-17T04:32:56.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.381+0000 7f8c0a000640 1 --2- 172.21.15.100:0/349897710 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8bf4079310 0x7f8bf407b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:56.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.381+0000 7f8c0a000640 1 -- 172.21.15.100:0/349897710 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7f8c000bc620 con 0x7f8c14145550 2024-09-17T04:32:56.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.381+0000 7f8c11e00640 1 --2- 172.21.15.100:0/349897710 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8bf4079310 0x7f8bf407b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:56.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.382+0000 7f8c11e00640 1 --2- 172.21.15.100:0/349897710 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8bf4079310 0x7f8bf407b820 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8c04001da0 tx=0x7f8c04009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:56.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.385+0000 7f8c0a000640 1 -- 172.21.15.100:0/349897710 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8c00083410 con 0x7f8c14145550 2024-09-17T04:32:56.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.593+0000 7f8c18c00640 1 -- 172.21.15.100:0/349897710 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp92870", "target": ["mon-mgr", ""]}) -- 0x7f8c141fa2d0 con 0x7f8bf4079310 2024-09-17T04:32:56.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.609+0000 7f8c0a000640 1 -- 172.21.15.100:0/349897710 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8c141fa2d0 con 0x7f8bf4079310 2024-09-17T04:32:56.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.611+0000 7f8c18c00640 1 -- 172.21.15.100:0/349897710 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8bf4079310 msgr2=0x7f8bf407b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:56.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.611+0000 7f8c18c00640 1 --2- 172.21.15.100:0/349897710 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8bf4079310 0x7f8bf407b820 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8c04001da0 tx=0x7f8c04009040 comp rx=0 tx=0).stop 2024-09-17T04:32:56.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.611+0000 7f8c18c00640 1 -- 172.21.15.100:0/349897710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c14145550 msgr2=0x7f8c141f9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:56.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.611+0000 7f8c18c00640 1 --2- 172.21.15.100:0/349897710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c14145550 0x7f8c141f9d90 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f8c00006010 tx=0x7f8c000038a0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.611+0000 7f8c18c00640 1 -- 172.21.15.100:0/349897710 shutdown_connections 2024-09-17T04:32:56.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.611+0000 7f8c18c00640 1 --2- 172.21.15.100:0/349897710 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8bf4079310 0x7f8bf407b820 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.611+0000 7f8c18c00640 1 --2- 172.21.15.100:0/349897710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c14145a40 0x7f8c140aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.611+0000 7f8c18c00640 1 --2- 172.21.15.100:0/349897710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c14145550 0x7f8c141f9d90 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.611+0000 7f8c18c00640 1 --2- 172.21.15.100:0/349897710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c14073a80 0x7f8c141f9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.611+0000 7f8c18c00640 1 -- 172.21.15.100:0/349897710 >> 172.21.15.100:0/349897710 conn(0x7f8c1406d1c0 msgr2=0x7f8c1406f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:56.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.611+0000 7f8c18c00640 1 -- 172.21.15.100:0/349897710 shutdown_connections 2024-09-17T04:32:56.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:56.611+0000 7f8c18c00640 1 -- 172.21.15.100:0/349897710 wait complete. 2024-09-17T04:32:56.633 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T04:32:56.633 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:32:56.715 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:32:56.816 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T04:32:56.816 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:32:56.816 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:32:56.816 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:32:56.852 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:32:56.852 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:32:56.928 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:32:57.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.193+0000 7f0d63e00640 1 -- 172.21.15.100:0/3290758316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d5c0cfc80 msgr2=0x7f0d5c156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:57.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.193+0000 7f0d63e00640 1 --2- 172.21.15.100:0/3290758316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d5c0cfc80 0x7f0d5c156e40 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009900 tx=0x7f0d5002a610 comp rx=0 tx=0).stop 2024-09-17T04:32:57.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.193+0000 7f0d63e00640 1 -- 172.21.15.100:0/3290758316 shutdown_connections 2024-09-17T04:32:57.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.193+0000 7f0d63e00640 1 --2- 172.21.15.100:0/3290758316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d5c0cfc80 0x7f0d5c156e40 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.193+0000 7f0d63e00640 1 -- 172.21.15.100:0/3290758316 >> 172.21.15.100:0/3290758316 conn(0x7f0d5c06d3d0 msgr2=0x7f0d5c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:57.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.194+0000 7f0d63e00640 1 -- 172.21.15.100:0/3290758316 shutdown_connections 2024-09-17T04:32:57.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.194+0000 7f0d63e00640 1 -- 172.21.15.100:0/3290758316 wait complete. 2024-09-17T04:32:57.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.196+0000 7f0d63e00640 1 Processor -- start 2024-09-17T04:32:57.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.196+0000 7f0d63e00640 1 -- start start 2024-09-17T04:32:57.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.197+0000 7f0d63e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d5c0705b0 0x7f0d5c074d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:57.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.197+0000 7f0d63e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d5c0cfc80 0x7f0d5c155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:57.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.197+0000 7f0d63e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d5c15ce30 0x7f0d5c1f99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:57.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.197+0000 7f0d63e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c0cd100 con 0x7f0d5c15ce30 2024-09-17T04:32:57.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.197+0000 7f0d63e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c0ccf80 con 0x7f0d5c0705b0 2024-09-17T04:32:57.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.197+0000 7f0d63e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0d5c0cd280 con 0x7f0d5c0cfc80 2024-09-17T04:32:57.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.198+0000 7f0d61e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d5c15ce30 0x7f0d5c1f99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:57.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.198+0000 7f0d61e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d5c15ce30 0x7f0d5c1f99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56764/0 (socket says 172.21.15.100:56764) 2024-09-17T04:32:57.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.198+0000 7f0d61e00640 1 -- 172.21.15.100:0/2252045615 learned_addr learned my addr 172.21.15.100:0/2252045615 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:57.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.198+0000 7f0d60a00640 1 --2- 172.21.15.100:0/2252045615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d5c0cfc80 0x7f0d5c155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:57.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.198+0000 7f0d61400640 1 --2- 172.21.15.100:0/2252045615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d5c0705b0 0x7f0d5c074d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:57.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.198+0000 7f0d61e00640 1 -- 172.21.15.100:0/2252045615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d5c0cfc80 msgr2=0x7f0d5c155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:57.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.198+0000 7f0d61e00640 1 --2- 172.21.15.100:0/2252045615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d5c0cfc80 0x7f0d5c155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.198+0000 7f0d61e00640 1 -- 172.21.15.100:0/2252045615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d5c0705b0 msgr2=0x7f0d5c074d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:57.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.198+0000 7f0d61e00640 1 --2- 172.21.15.100:0/2252045615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d5c0705b0 0x7f0d5c074d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.198+0000 7f0d61e00640 1 -- 172.21.15.100:0/2252045615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5c1f68a0 con 0x7f0d5c15ce30 2024-09-17T04:32:57.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.198+0000 7f0d61e00640 1 --2- 172.21.15.100:0/2252045615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d5c15ce30 0x7f0d5c1f99d0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f0d5400c700 tx=0x7f0d5400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:57.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.199+0000 7f0d59600640 1 -- 172.21.15.100:0/2252045615 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54003b70 con 0x7f0d5c15ce30 2024-09-17T04:32:57.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.199+0000 7f0d59600640 1 -- 172.21.15.100:0/2252045615 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d54003d30 con 0x7f0d5c15ce30 2024-09-17T04:32:57.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.199+0000 7f0d59600640 1 -- 172.21.15.100:0/2252045615 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54005330 con 0x7f0d5c15ce30 2024-09-17T04:32:57.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.199+0000 7f0d63e00640 1 -- 172.21.15.100:0/2252045615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d5c1f5ae0 con 0x7f0d5c15ce30 2024-09-17T04:32:57.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.199+0000 7f0d63e00640 1 -- 172.21.15.100:0/2252045615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d5c1f60a0 con 0x7f0d5c15ce30 2024-09-17T04:32:57.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.200+0000 7f0d63e00640 1 -- 172.21.15.100:0/2252045615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d5c0cd770 con 0x7f0d5c15ce30 2024-09-17T04:32:57.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.201+0000 7f0d59600640 1 -- 172.21.15.100:0/2252045615 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 102104+0+0 (secure 0 0 0) 0x7f0d540054d0 con 0x7f0d5c15ce30 2024-09-17T04:32:57.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.202+0000 7f0d59600640 1 --2- 172.21.15.100:0/2252045615 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0d3c079310 0x7f0d3c07b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:57.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.202+0000 7f0d59600640 1 -- 172.21.15.100:0/2252045615 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7f0d5409bc70 con 0x7f0d5c15ce30 2024-09-17T04:32:57.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.202+0000 7f0d61400640 1 --2- 172.21.15.100:0/2252045615 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0d3c079310 0x7f0d3c07b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:57.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.203+0000 7f0d61400640 1 --2- 172.21.15.100:0/2252045615 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0d3c079310 0x7f0d3c07b820 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0d50033000 tx=0x7f0d50004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:57.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.206+0000 7f0d59600640 1 -- 172.21.15.100:0/2252045615 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0d540629e0 con 0x7f0d5c15ce30 2024-09-17T04:32:57.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.398+0000 7f0d63e00640 1 -- 172.21.15.100:0/2252045615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0d5c15e7c0 con 0x7f0d5c15ce30 2024-09-17T04:32:57.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.399+0000 7f0d59600640 1 -- 172.21.15.100:0/2252045615 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v145) ==== 80+0+0 (secure 0 0 0) 0x7f0d54067410 con 0x7f0d5c15ce30 2024-09-17T04:32:57.398 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-09-17T04:32:57.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.401+0000 7f0d63e00640 1 -- 172.21.15.100:0/2252045615 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0d3c079310 msgr2=0x7f0d3c07b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:57.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.401+0000 7f0d63e00640 1 --2- 172.21.15.100:0/2252045615 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0d3c079310 0x7f0d3c07b820 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0d50033000 tx=0x7f0d50004730 comp rx=0 tx=0).stop 2024-09-17T04:32:57.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.401+0000 7f0d63e00640 1 -- 172.21.15.100:0/2252045615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d5c15ce30 msgr2=0x7f0d5c1f99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:57.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.401+0000 7f0d63e00640 1 --2- 172.21.15.100:0/2252045615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d5c15ce30 0x7f0d5c1f99d0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f0d5400c700 tx=0x7f0d5400cbc0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.401+0000 7f0d63e00640 1 -- 172.21.15.100:0/2252045615 shutdown_connections 2024-09-17T04:32:57.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.401+0000 7f0d63e00640 1 --2- 172.21.15.100:0/2252045615 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0d3c079310 0x7f0d3c07b820 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.401+0000 7f0d63e00640 1 --2- 172.21.15.100:0/2252045615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d5c15ce30 0x7f0d5c1f99d0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.401+0000 7f0d63e00640 1 --2- 172.21.15.100:0/2252045615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d5c0cfc80 0x7f0d5c155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.401+0000 7f0d63e00640 1 --2- 172.21.15.100:0/2252045615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d5c0705b0 0x7f0d5c074d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.401+0000 7f0d63e00640 1 -- 172.21.15.100:0/2252045615 >> 172.21.15.100:0/2252045615 conn(0x7f0d5c06d3d0 msgr2=0x7f0d5c0b2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:57.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.401+0000 7f0d63e00640 1 -- 172.21.15.100:0/2252045615 shutdown_connections 2024-09-17T04:32:57.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.401+0000 7f0d63e00640 1 -- 172.21.15.100:0/2252045615 wait complete. 2024-09-17T04:32:57.427 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T04:32:57.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:57.427 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T04:32:57.490 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-09-17T04:32:57.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:57.490 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T04:32:57.527 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:32:57.527 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:32:57.527 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:32:57.567 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:32:57.567 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:32:57.646 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:32:57.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.869+0000 7f47a5200640 1 -- 172.21.15.100:0/837025430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47a015d670 msgr2=0x7f47a0070530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:57.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.869+0000 7f47a5200640 1 --2- 172.21.15.100:0/837025430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47a015d670 0x7f47a0070530 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f47a0051590 tx=0x7f479802a980 comp rx=0 tx=0).stop 2024-09-17T04:32:57.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.869+0000 7f47a5200640 1 -- 172.21.15.100:0/837025430 shutdown_connections 2024-09-17T04:32:57.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.869+0000 7f47a5200640 1 --2- 172.21.15.100:0/837025430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47a015d670 0x7f47a0070530 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.869+0000 7f47a5200640 1 --2- 172.21.15.100:0/837025430 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47a00b2f60 0x7f47a015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.869+0000 7f47a5200640 1 --2- 172.21.15.100:0/837025430 >> v2:172.21.15.184:3300/0 conn(0x7f47a00ad9c0 0x7f47a00af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.869+0000 7f47a5200640 1 -- 172.21.15.100:0/837025430 >> 172.21.15.100:0/837025430 conn(0x7f47a00b1e40 msgr2=0x7f47a00b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:57.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.869+0000 7f47a5200640 1 -- 172.21.15.100:0/837025430 shutdown_connections 2024-09-17T04:32:57.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.869+0000 7f47a5200640 1 -- 172.21.15.100:0/837025430 wait complete. 2024-09-17T04:32:57.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.871+0000 7f47a5200640 1 Processor -- start 2024-09-17T04:32:57.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.871+0000 7f47a5200640 1 -- start start 2024-09-17T04:32:57.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.873+0000 7f47a5200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47a00ad9c0 0x7f47a0148480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:57.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.873+0000 7f47a5200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47a00b2f60 0x7f47a0148df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:57.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.873+0000 7f47a5200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47a015d670 0x7f47a0149330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:57.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.873+0000 7f47a5200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f47a006f290 con 0x7f47a00ad9c0 2024-09-17T04:32:57.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.873+0000 7f47a5200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f47a0070c80 con 0x7f47a015d670 2024-09-17T04:32:57.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.873+0000 7f47a5200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f47a006f410 con 0x7f47a00b2f60 2024-09-17T04:32:57.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.873+0000 7f479f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47a00ad9c0 0x7f47a0148480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:57.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.873+0000 7f479f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47a00ad9c0 0x7f47a0148480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56784/0 (socket says 172.21.15.100:56784) 2024-09-17T04:32:57.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.873+0000 7f479f400640 1 -- 172.21.15.100:0/305182605 learned_addr learned my addr 172.21.15.100:0/305182605 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:57.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.873+0000 7f4796a00640 1 --2- 172.21.15.100:0/305182605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47a00b2f60 0x7f47a0148df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:57.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.873+0000 7f479fe00640 1 --2- 172.21.15.100:0/305182605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47a015d670 0x7f47a0149330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:57.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.874+0000 7f479f400640 1 -- 172.21.15.100:0/305182605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47a00b2f60 msgr2=0x7f47a0148df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:57.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.874+0000 7f479f400640 1 --2- 172.21.15.100:0/305182605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47a00b2f60 0x7f47a0148df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.874+0000 7f479f400640 1 -- 172.21.15.100:0/305182605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47a015d670 msgr2=0x7f47a0149330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:57.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.874+0000 7f479f400640 1 --2- 172.21.15.100:0/305182605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47a015d670 0x7f47a0149330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.874+0000 7f479f400640 1 -- 172.21.15.100:0/305182605 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a01551b0 con 0x7f47a00ad9c0 2024-09-17T04:32:57.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.874+0000 7f479f400640 1 --2- 172.21.15.100:0/305182605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47a00ad9c0 0x7f47a0148480 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f479000d770 tx=0x7f479000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:57.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.874+0000 7f4797e00640 1 -- 172.21.15.100:0/305182605 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790017070 con 0x7f47a00ad9c0 2024-09-17T04:32:57.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.874+0000 7f4797e00640 1 -- 172.21.15.100:0/305182605 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4790009cc0 con 0x7f47a00ad9c0 2024-09-17T04:32:57.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.874+0000 7f4797e00640 1 -- 172.21.15.100:0/305182605 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47900106c0 con 0x7f47a00ad9c0 2024-09-17T04:32:57.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.874+0000 7f47a5200640 1 -- 172.21.15.100:0/305182605 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a009e0b0 con 0x7f47a00ad9c0 2024-09-17T04:32:57.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.874+0000 7f47a5200640 1 -- 172.21.15.100:0/305182605 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a0158500 con 0x7f47a00ad9c0 2024-09-17T04:32:57.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.875+0000 7f47a5200640 1 -- 172.21.15.100:0/305182605 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a006f930 con 0x7f47a00ad9c0 2024-09-17T04:32:57.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.877+0000 7f4797e00640 1 -- 172.21.15.100:0/305182605 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f479001a070 con 0x7f47a00ad9c0 2024-09-17T04:32:57.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.878+0000 7f4797e00640 1 --2- 172.21.15.100:0/305182605 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f477c079400 0x7f477c07b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:57.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.878+0000 7f4797e00640 1 -- 172.21.15.100:0/305182605 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7f479009c110 con 0x7f47a00ad9c0 2024-09-17T04:32:57.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.878+0000 7f4796a00640 1 --2- 172.21.15.100:0/305182605 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f477c079400 0x7f477c07b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:57.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.879+0000 7f4796a00640 1 --2- 172.21.15.100:0/305182605 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f477c079400 0x7f477c07b910 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f47a015a7f0 tx=0x7f478800b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:57.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:57.881+0000 7f4797e00640 1 -- 172.21.15.100:0/305182605 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f47a01551b0 con 0x7f47a00ad9c0 2024-09-17T04:32:58.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.075+0000 7f47a5200640 1 -- 172.21.15.100:0/305182605 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f47a015a820 con 0x7f47a00ad9c0 2024-09-17T04:32:58.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.075+0000 7f4797e00640 1 -- 172.21.15.100:0/305182605 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v145) ==== 80+0+0 (secure 0 0 0) 0x7f47900633b0 con 0x7f47a00ad9c0 2024-09-17T04:32:58.076 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-09-17T04:32:58.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.078+0000 7f47a5200640 1 -- 172.21.15.100:0/305182605 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f477c079400 msgr2=0x7f477c07b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:58.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.078+0000 7f47a5200640 1 --2- 172.21.15.100:0/305182605 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f477c079400 0x7f477c07b910 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f47a015a7f0 tx=0x7f478800b2e0 comp rx=0 tx=0).stop 2024-09-17T04:32:58.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.078+0000 7f47a5200640 1 -- 172.21.15.100:0/305182605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47a00ad9c0 msgr2=0x7f47a0148480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:58.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.078+0000 7f47a5200640 1 --2- 172.21.15.100:0/305182605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47a00ad9c0 0x7f47a0148480 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f479000d770 tx=0x7f479000dc30 comp rx=0 tx=0).stop 2024-09-17T04:32:58.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.078+0000 7f47a5200640 1 -- 172.21.15.100:0/305182605 shutdown_connections 2024-09-17T04:32:58.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.078+0000 7f47a5200640 1 --2- 172.21.15.100:0/305182605 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f477c079400 0x7f477c07b910 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:58.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.078+0000 7f47a5200640 1 --2- 172.21.15.100:0/305182605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47a015d670 0x7f47a0149330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:58.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.078+0000 7f47a5200640 1 --2- 172.21.15.100:0/305182605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47a00b2f60 0x7f47a0148df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:58.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.078+0000 7f47a5200640 1 --2- 172.21.15.100:0/305182605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47a00ad9c0 0x7f47a0148480 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:58.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.078+0000 7f47a5200640 1 -- 172.21.15.100:0/305182605 >> 172.21.15.100:0/305182605 conn(0x7f47a00b1e40 msgr2=0x7f47a00cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:58.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.078+0000 7f47a5200640 1 -- 172.21.15.100:0/305182605 shutdown_connections 2024-09-17T04:32:58.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.078+0000 7f47a5200640 1 -- 172.21.15.100:0/305182605 wait complete. 2024-09-17T04:32:58.102 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-17T04:32:58.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:58.103 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-17T04:32:58.170 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2024-09-17T04:32:58.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:32:58.171 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T04:32:58.207 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:32:58.207 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:32:58.207 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:32:58.243 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:32:58.243 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:32:58.243 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:32:58.243 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:32:58.283 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:32:58.283 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:32:58.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.502+0000 7fde0d600640 1 -- 172.21.15.100:0/3664188632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde0815d760 msgr2=0x7fde081535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:58.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.502+0000 7fde0d600640 1 --2- 172.21.15.100:0/3664188632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde0815d760 0x7fde081535c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009830 tx=0x7fddf802a530 comp rx=0 tx=0).stop 2024-09-17T04:32:58.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.503+0000 7fde0d600640 1 -- 172.21.15.100:0/3664188632 shutdown_connections 2024-09-17T04:32:58.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.503+0000 7fde0d600640 1 --2- 172.21.15.100:0/3664188632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde0815d760 0x7fde081535c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:58.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.503+0000 7fde0d600640 1 -- 172.21.15.100:0/3664188632 >> 172.21.15.100:0/3664188632 conn(0x7fde0806d730 msgr2=0x7fde080b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:58.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.503+0000 7fde0d600640 1 -- 172.21.15.100:0/3664188632 shutdown_connections 2024-09-17T04:32:58.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.503+0000 7fde0d600640 1 -- 172.21.15.100:0/3664188632 wait complete. 2024-09-17T04:32:58.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.505+0000 7fde0d600640 1 Processor -- start 2024-09-17T04:32:58.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.505+0000 7fde0d600640 1 -- start start 2024-09-17T04:32:58.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.506+0000 7fde0d600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde080ad0e0 0x7fde0815ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:58.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.506+0000 7fde0d600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde080b3550 0x7fde08073600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:58.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.506+0000 7fde0d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde0815d760 0x7fde08073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:58.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.506+0000 7fde0d600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fde0806f240 con 0x7fde0815d760 2024-09-17T04:32:58.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.506+0000 7fde0d600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fde080b4e30 con 0x7fde080ad0e0 2024-09-17T04:32:58.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.506+0000 7fde0d600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fde080b4070 con 0x7fde080b3550 2024-09-17T04:32:58.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.507+0000 7fde07400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde0815d760 0x7fde08073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:58.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.507+0000 7fde07400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde0815d760 0x7fde08073b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56800/0 (socket says 172.21.15.100:56800) 2024-09-17T04:32:58.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.507+0000 7fde07400640 1 -- 172.21.15.100:0/3385509665 learned_addr learned my addr 172.21.15.100:0/3385509665 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:58.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.507+0000 7fde06a00640 1 --2- 172.21.15.100:0/3385509665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde080ad0e0 0x7fde0815ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:58.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.507+0000 7fde07400640 1 -- 172.21.15.100:0/3385509665 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde080b3550 msgr2=0x7fde08073600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:58.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.507+0000 7fde06000640 1 --2- 172.21.15.100:0/3385509665 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde080b3550 0x7fde08073600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:58.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.507+0000 7fde07400640 1 --2- 172.21.15.100:0/3385509665 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde080b3550 0x7fde08073600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:58.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.507+0000 7fde07400640 1 -- 172.21.15.100:0/3385509665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde080ad0e0 msgr2=0x7fde0815ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.507+0000 7fde07400640 1 --2- 172.21.15.100:0/3385509665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde080ad0e0 0x7fde0815ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.507+0000 7fde07400640 1 -- 172.21.15.100:0/3385509665 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde08074080 con 0x7fde0815d760 2024-09-17T04:32:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.507+0000 7fde06000640 1 --2- 172.21.15.100:0/3385509665 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde080b3550 0x7fde08073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:32:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.508+0000 7fde07400640 1 --2- 172.21.15.100:0/3385509665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde0815d760 0x7fde08073b40 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fde00002230 tx=0x7fde0000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.508+0000 7fddfea00640 1 -- 172.21.15.100:0/3385509665 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde00016020 con 0x7fde0815d760 2024-09-17T04:32:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.508+0000 7fddfea00640 1 -- 172.21.15.100:0/3385509665 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde0000ad40 con 0x7fde0815d760 2024-09-17T04:32:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.508+0000 7fde0d600640 1 -- 172.21.15.100:0/3385509665 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde081f6120 con 0x7fde0815d760 2024-09-17T04:32:58.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.508+0000 7fddfea00640 1 -- 172.21.15.100:0/3385509665 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde00012680 con 0x7fde0815d760 2024-09-17T04:32:58.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.508+0000 7fde0d600640 1 -- 172.21.15.100:0/3385509665 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde081f66e0 con 0x7fde0815d760 2024-09-17T04:32:58.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.509+0000 7fde0d600640 1 -- 172.21.15.100:0/3385509665 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde081535c0 con 0x7fde0815d760 2024-09-17T04:32:58.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.511+0000 7fddfea00640 1 -- 172.21.15.100:0/3385509665 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7fde00019020 con 0x7fde0815d760 2024-09-17T04:32:58.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.511+0000 7fddfea00640 1 --2- 172.21.15.100:0/3385509665 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdde8079400 0x7fdde807b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:58.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.512+0000 7fddfea00640 1 -- 172.21.15.100:0/3385509665 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7fde00099b60 con 0x7fde0815d760 2024-09-17T04:32:58.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.512+0000 7fde06a00640 1 --2- 172.21.15.100:0/3385509665 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdde8079400 0x7fdde807b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:58.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.512+0000 7fde06a00640 1 --2- 172.21.15.100:0/3385509665 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdde8079400 0x7fdde807b910 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fddf8034000 tx=0x7fddf8004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:58.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.516+0000 7fddfea00640 1 -- 172.21.15.100:0/3385509665 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fde00020020 con 0x7fde0815d760 2024-09-17T04:32:58.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.735+0000 7fde0d600640 1 -- 172.21.15.100:0/3385509665 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde080b07b0 con 0x7fde0815d760 2024-09-17T04:32:58.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.735+0000 7fddfea00640 1 -- 172.21.15.100:0/3385509665 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+5241 (secure 0 0 0) 0x7fde00063ac0 con 0x7fde0815d760 2024-09-17T04:32:58.736 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:32:58.736 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":102,"btime":"2024-09-17T04:32:45:357279+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7028,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1529630991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1529630991},{"type":"v1","addr":"172.21.15.184:6801","nonce":1529630991}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7906,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1516666057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1516666057},{"type":"v1","addr":"172.21.15.184:6835","nonce":1516666057}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2252725120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2252725120},{"type":"v1","addr":"172.21.15.100:6833","nonce":2252725120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:32:34.557571+0000","modified":"2024-09-17T04:32:37.568034+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8427":{"gid":8427,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/2425159512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2425159512},{"type":"v1","addr":"172.21.15.100:6835","nonce":2425159512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8427,"qdb_cluster":[8427]},"id":11}]} 2024-09-17T04:32:58.736 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 102 2024-09-17T04:32:58.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.738+0000 7fde0d600640 1 -- 172.21.15.100:0/3385509665 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdde8079400 msgr2=0x7fdde807b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:58.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.738+0000 7fde0d600640 1 --2- 172.21.15.100:0/3385509665 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdde8079400 0x7fdde807b910 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fddf8034000 tx=0x7fddf8004730 comp rx=0 tx=0).stop 2024-09-17T04:32:58.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.738+0000 7fde0d600640 1 -- 172.21.15.100:0/3385509665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde0815d760 msgr2=0x7fde08073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:58.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.738+0000 7fde0d600640 1 --2- 172.21.15.100:0/3385509665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde0815d760 0x7fde08073b40 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fde00002230 tx=0x7fde0000cf70 comp rx=0 tx=0).stop 2024-09-17T04:32:58.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.738+0000 7fde0d600640 1 -- 172.21.15.100:0/3385509665 shutdown_connections 2024-09-17T04:32:58.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.738+0000 7fde0d600640 1 --2- 172.21.15.100:0/3385509665 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdde8079400 0x7fdde807b910 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:58.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.738+0000 7fde0d600640 1 --2- 172.21.15.100:0/3385509665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde0815d760 0x7fde08073b40 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:58.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.738+0000 7fde0d600640 1 --2- 172.21.15.100:0/3385509665 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde080b3550 0x7fde08073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:58.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.738+0000 7fde0d600640 1 --2- 172.21.15.100:0/3385509665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde080ad0e0 0x7fde0815ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:58.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.738+0000 7fde0d600640 1 -- 172.21.15.100:0/3385509665 >> 172.21.15.100:0/3385509665 conn(0x7fde0806d730 msgr2=0x7fde0815ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:58.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.738+0000 7fde0d600640 1 -- 172.21.15.100:0/3385509665 shutdown_connections 2024-09-17T04:32:58.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.738+0000 7fde0d600640 1 -- 172.21.15.100:0/3385509665 wait complete. 2024-09-17T04:32:58.762 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:32:58.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.981+0000 7fe623e00640 1 -- 172.21.15.100:0/662708596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe61c0b3550 msgr2=0x7fe61c0b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:58.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.981+0000 7fe623e00640 1 --2- 172.21.15.100:0/662708596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe61c0b3550 0x7fe61c0b3960 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe614009830 tx=0x7fe61402a480 comp rx=0 tx=0).stop 2024-09-17T04:32:58.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.981+0000 7fe623e00640 1 -- 172.21.15.100:0/662708596 shutdown_connections 2024-09-17T04:32:58.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.981+0000 7fe623e00640 1 --2- 172.21.15.100:0/662708596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe61c0b3550 0x7fe61c0b3960 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:58.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.981+0000 7fe623e00640 1 --2- 172.21.15.100:0/662708596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe61c0ad0e0 0x7fe61c0ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:58.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.981+0000 7fe623e00640 1 -- 172.21.15.100:0/662708596 >> 172.21.15.100:0/662708596 conn(0x7fe61c06d730 msgr2=0x7fe61c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:58.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.981+0000 7fe623e00640 1 -- 172.21.15.100:0/662708596 shutdown_connections 2024-09-17T04:32:58.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.981+0000 7fe623e00640 1 -- 172.21.15.100:0/662708596 wait complete. 2024-09-17T04:32:58.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.983+0000 7fe623e00640 1 Processor -- start 2024-09-17T04:32:58.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.983+0000 7fe623e00640 1 -- start start 2024-09-17T04:32:58.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.985+0000 7fe623e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe61c0ad0e0 0x7fe61c156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:58.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.985+0000 7fe623e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe61c0b3550 0x7fe61c1f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:58.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.985+0000 7fe623e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe61c15d760 0x7fe61c0b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:58.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.985+0000 7fe623e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c0b4da0 con 0x7fe61c0ad0e0 2024-09-17T04:32:58.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.985+0000 7fe623e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c0b4c20 con 0x7fe61c0b3550 2024-09-17T04:32:58.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.985+0000 7fe623e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe61c0b3ea0 con 0x7fe61c15d760 2024-09-17T04:32:58.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.985+0000 7fe621400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe61c0ad0e0 0x7fe61c156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:58.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.985+0000 7fe621400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe61c0ad0e0 0x7fe61c156850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56812/0 (socket says 172.21.15.100:56812) 2024-09-17T04:32:58.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.985+0000 7fe621400640 1 -- 172.21.15.100:0/3534639730 learned_addr learned my addr 172.21.15.100:0/3534639730 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:58.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.985+0000 7fe620a00640 1 --2- 172.21.15.100:0/3534639730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe61c0b3550 0x7fe61c1f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:58.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.985+0000 7fe621e00640 1 --2- 172.21.15.100:0/3534639730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe61c15d760 0x7fe61c0b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:58.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.985+0000 7fe621400640 1 -- 172.21.15.100:0/3534639730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe61c15d760 msgr2=0x7fe61c0b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:58.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.986+0000 7fe621400640 1 --2- 172.21.15.100:0/3534639730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe61c15d760 0x7fe61c0b0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:58.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.986+0000 7fe621400640 1 -- 172.21.15.100:0/3534639730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe61c0b3550 msgr2=0x7fe61c1f5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:58.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.986+0000 7fe621400640 1 --2- 172.21.15.100:0/3534639730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe61c0b3550 0x7fe61c1f5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:58.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.986+0000 7fe621400640 1 -- 172.21.15.100:0/3534639730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61c157080 con 0x7fe61c0ad0e0 2024-09-17T04:32:58.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.986+0000 7fe620a00640 1 --2- 172.21.15.100:0/3534639730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe61c0b3550 0x7fe61c1f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:32:58.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.986+0000 7fe621400640 1 --2- 172.21.15.100:0/3534639730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe61c0ad0e0 0x7fe61c156850 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fe61000cb50 tx=0x7fe61000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:58.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.986+0000 7fe619600640 1 -- 172.21.15.100:0/3534639730 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610016020 con 0x7fe61c0ad0e0 2024-09-17T04:32:58.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.986+0000 7fe619600640 1 -- 172.21.15.100:0/3534639730 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe61000ae30 con 0x7fe61c0ad0e0 2024-09-17T04:32:58.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.986+0000 7fe619600640 1 -- 172.21.15.100:0/3534639730 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610012660 con 0x7fe61c0ad0e0 2024-09-17T04:32:58.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.986+0000 7fe623e00640 1 -- 172.21.15.100:0/3534639730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe61c1f6930 con 0x7fe61c0ad0e0 2024-09-17T04:32:58.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.986+0000 7fe623e00640 1 -- 172.21.15.100:0/3534639730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe61c09c600 con 0x7fe61c0ad0e0 2024-09-17T04:32:58.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.987+0000 7fe623e00640 1 -- 172.21.15.100:0/3534639730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe61c06a7e0 con 0x7fe61c0ad0e0 2024-09-17T04:32:58.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.988+0000 7fe619600640 1 -- 172.21.15.100:0/3534639730 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7fe610019020 con 0x7fe61c0ad0e0 2024-09-17T04:32:58.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.990+0000 7fe619600640 1 --2- 172.21.15.100:0/3534639730 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe5fc079400 0x7fe5fc07b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:58.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.990+0000 7fe619600640 1 -- 172.21.15.100:0/3534639730 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7fe6100999f0 con 0x7fe61c0ad0e0 2024-09-17T04:32:58.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.990+0000 7fe620a00640 1 --2- 172.21.15.100:0/3534639730 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe5fc079400 0x7fe5fc07b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:58.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.991+0000 7fe620a00640 1 --2- 172.21.15.100:0/3534639730 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe5fc079400 0x7fe5fc07b910 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe61c09e040 tx=0x7fe608005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:58.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:58.995+0000 7fe619600640 1 -- 172.21.15.100:0/3534639730 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe61c06a7e0 con 0x7fe61c0ad0e0 2024-09-17T04:32:59.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.218+0000 7fe623e00640 1 -- 172.21.15.100:0/3534639730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe61c0b4330 con 0x7fe61c0ad0e0 2024-09-17T04:32:59.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.218+0000 7fe619600640 1 -- 172.21.15.100:0/3534639730 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+5241 (secure 0 0 0) 0x7fe610020020 con 0x7fe61c0ad0e0 2024-09-17T04:32:59.219 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:32:59.219 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":102,"btime":"2024-09-17T04:32:45:357279+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7028,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1529630991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1529630991},{"type":"v1","addr":"172.21.15.184:6801","nonce":1529630991}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7906,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1516666057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1516666057},{"type":"v1","addr":"172.21.15.184:6835","nonce":1516666057}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2252725120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2252725120},{"type":"v1","addr":"172.21.15.100:6833","nonce":2252725120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:32:34.557571+0000","modified":"2024-09-17T04:32:37.568034+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8427":{"gid":8427,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/2425159512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2425159512},{"type":"v1","addr":"172.21.15.100:6835","nonce":2425159512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8427,"qdb_cluster":[8427]},"id":11}]} 2024-09-17T04:32:59.220 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 102 2024-09-17T04:32:59.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.221+0000 7fe623e00640 1 -- 172.21.15.100:0/3534639730 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe5fc079400 msgr2=0x7fe5fc07b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:59.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.221+0000 7fe623e00640 1 --2- 172.21.15.100:0/3534639730 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe5fc079400 0x7fe5fc07b910 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe61c09e040 tx=0x7fe608005f90 comp rx=0 tx=0).stop 2024-09-17T04:32:59.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.221+0000 7fe623e00640 1 -- 172.21.15.100:0/3534639730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe61c0ad0e0 msgr2=0x7fe61c156850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:59.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.221+0000 7fe623e00640 1 --2- 172.21.15.100:0/3534639730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe61c0ad0e0 0x7fe61c156850 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fe61000cb50 tx=0x7fe61000a430 comp rx=0 tx=0).stop 2024-09-17T04:32:59.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.221+0000 7fe623e00640 1 -- 172.21.15.100:0/3534639730 shutdown_connections 2024-09-17T04:32:59.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.221+0000 7fe623e00640 1 --2- 172.21.15.100:0/3534639730 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fe5fc079400 0x7fe5fc07b910 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:59.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.221+0000 7fe623e00640 1 --2- 172.21.15.100:0/3534639730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe61c15d760 0x7fe61c0b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:59.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.221+0000 7fe623e00640 1 --2- 172.21.15.100:0/3534639730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe61c0b3550 0x7fe61c1f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:59.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.221+0000 7fe623e00640 1 --2- 172.21.15.100:0/3534639730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe61c0ad0e0 0x7fe61c156850 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:59.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.221+0000 7fe623e00640 1 -- 172.21.15.100:0/3534639730 >> 172.21.15.100:0/3534639730 conn(0x7fe61c06d730 msgr2=0x7fe61c0cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:59.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.221+0000 7fe623e00640 1 -- 172.21.15.100:0/3534639730 shutdown_connections 2024-09-17T04:32:59.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.221+0000 7fe623e00640 1 -- 172.21.15.100:0/3534639730 wait complete. 2024-09-17T04:32:59.247 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:32:59.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.467+0000 7fca49400640 1 -- 172.21.15.100:0/2519672176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca28004d40 msgr2=0x7fca280051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:59.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.467+0000 7fca49400640 1 --2- 172.21.15.100:0/2519672176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca28004d40 0x7fca280051f0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fca440aea60 tx=0x7fca28005830 comp rx=0 tx=0).stop 2024-09-17T04:32:59.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.467+0000 7fca43200640 1 -- 172.21.15.100:0/2519672176 reap_dead start 2024-09-17T04:32:59.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.467+0000 7fca49400640 1 -- 172.21.15.100:0/2519672176 shutdown_connections 2024-09-17T04:32:59.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.467+0000 7fca49400640 1 -- 172.21.15.100:0/2519672176 >> 172.21.15.100:0/2519672176 conn(0x7fca4406d730 msgr2=0x7fca440b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:59.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.467+0000 7fca49400640 1 -- 172.21.15.100:0/2519672176 shutdown_connections 2024-09-17T04:32:59.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.468+0000 7fca49400640 1 -- 172.21.15.100:0/2519672176 wait complete. 2024-09-17T04:32:59.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.470+0000 7fca49400640 1 Processor -- start 2024-09-17T04:32:59.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.470+0000 7fca49400640 1 -- start start 2024-09-17T04:32:59.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.471+0000 7fca49400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca28004d40 0x7fca441f5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:59.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.471+0000 7fca49400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca441f9fb0 0x7fca44144380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:59.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.471+0000 7fca49400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca441448c0 0x7fca441f9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:59.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.471+0000 7fca49400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fca440b41f0 con 0x7fca28004d40 2024-09-17T04:32:59.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.471+0000 7fca49400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fca440b4070 con 0x7fca441f9fb0 2024-09-17T04:32:59.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.471+0000 7fca49400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fca440b4370 con 0x7fca441448c0 2024-09-17T04:32:59.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.471+0000 7fca42800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca28004d40 0x7fca441f5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:59.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.472+0000 7fca42800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca28004d40 0x7fca441f5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56826/0 (socket says 172.21.15.100:56826) 2024-09-17T04:32:59.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.472+0000 7fca42800640 1 -- 172.21.15.100:0/2325946451 learned_addr learned my addr 172.21.15.100:0/2325946451 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:59.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.472+0000 7fca3be00640 1 --2- 172.21.15.100:0/2325946451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca441f9fb0 0x7fca44144380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:59.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.472+0000 7fca43200640 1 --2- 172.21.15.100:0/2325946451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca441448c0 0x7fca441f9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:59.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.472+0000 7fca42800640 1 -- 172.21.15.100:0/2325946451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca441448c0 msgr2=0x7fca441f9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:59.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.472+0000 7fca42800640 1 --2- 172.21.15.100:0/2325946451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca441448c0 0x7fca441f9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:59.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.472+0000 7fca42800640 1 -- 172.21.15.100:0/2325946451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca441f9fb0 msgr2=0x7fca44144380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:59.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.472+0000 7fca42800640 1 --2- 172.21.15.100:0/2325946451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca441f9fb0 0x7fca44144380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:59.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.472+0000 7fca42800640 1 -- 172.21.15.100:0/2325946451 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca441f6720 con 0x7fca28004d40 2024-09-17T04:32:59.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.472+0000 7fca42800640 1 --2- 172.21.15.100:0/2325946451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca28004d40 0x7fca441f5db0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fca34011bf0 tx=0x7fca3400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:59.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.472+0000 7fca3aa00640 1 -- 172.21.15.100:0/2325946451 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3401e070 con 0x7fca28004d40 2024-09-17T04:32:59.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.472+0000 7fca3aa00640 1 -- 172.21.15.100:0/2325946451 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca3400e7c0 con 0x7fca28004d40 2024-09-17T04:32:59.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.472+0000 7fca49400640 1 -- 172.21.15.100:0/2325946451 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca441f6a00 con 0x7fca28004d40 2024-09-17T04:32:59.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.472+0000 7fca3aa00640 1 -- 172.21.15.100:0/2325946451 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3400ebf0 con 0x7fca28004d40 2024-09-17T04:32:59.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.473+0000 7fca49400640 1 -- 172.21.15.100:0/2325946451 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca4409b430 con 0x7fca28004d40 2024-09-17T04:32:59.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.474+0000 7fca49400640 1 -- 172.21.15.100:0/2325946451 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca4415c6e0 con 0x7fca28004d40 2024-09-17T04:32:59.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.475+0000 7fca3aa00640 1 -- 172.21.15.100:0/2325946451 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7fca34021070 con 0x7fca28004d40 2024-09-17T04:32:59.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.476+0000 7fca3aa00640 1 --2- 172.21.15.100:0/2325946451 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fca20079400 0x7fca2007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:59.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.476+0000 7fca3aa00640 1 -- 172.21.15.100:0/2325946451 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7fca3409cf20 con 0x7fca28004d40 2024-09-17T04:32:59.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.476+0000 7fca3be00640 1 --2- 172.21.15.100:0/2325946451 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fca20079400 0x7fca2007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:59.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.477+0000 7fca3be00640 1 --2- 172.21.15.100:0/2325946451 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fca20079400 0x7fca2007b910 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fca441531f0 tx=0x7fca2c006b70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:59.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.480+0000 7fca3aa00640 1 -- 172.21.15.100:0/2325946451 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fca4415c6e0 con 0x7fca28004d40 2024-09-17T04:32:59.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.666+0000 7fca49400640 1 -- 172.21.15.100:0/2325946451 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca440b3960 con 0x7fca28004d40 2024-09-17T04:32:59.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.667+0000 7fca3aa00640 1 -- 172.21.15.100:0/2325946451 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16868 (secure 0 0 0) 0x7fca34063dc0 con 0x7fca28004d40 2024-09-17T04:32:59.668 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:32:59.668 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":145,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:32:36.070033+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:32:32.462209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-17T04:32:33.516803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:32:59.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.669+0000 7fca49400640 1 -- 172.21.15.100:0/2325946451 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fca20079400 msgr2=0x7fca2007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:59.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.669+0000 7fca49400640 1 --2- 172.21.15.100:0/2325946451 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fca20079400 0x7fca2007b910 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fca441531f0 tx=0x7fca2c006b70 comp rx=0 tx=0).stop 2024-09-17T04:32:59.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.669+0000 7fca49400640 1 -- 172.21.15.100:0/2325946451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca28004d40 msgr2=0x7fca441f5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:59.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.669+0000 7fca49400640 1 --2- 172.21.15.100:0/2325946451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca28004d40 0x7fca441f5db0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fca34011bf0 tx=0x7fca3400f790 comp rx=0 tx=0).stop 2024-09-17T04:32:59.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.669+0000 7fca49400640 1 -- 172.21.15.100:0/2325946451 shutdown_connections 2024-09-17T04:32:59.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.669+0000 7fca49400640 1 --2- 172.21.15.100:0/2325946451 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fca20079400 0x7fca2007b910 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:59.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.669+0000 7fca49400640 1 --2- 172.21.15.100:0/2325946451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca441448c0 0x7fca441f9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:59.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.669+0000 7fca49400640 1 --2- 172.21.15.100:0/2325946451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca441f9fb0 0x7fca44144380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:59.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.669+0000 7fca49400640 1 --2- 172.21.15.100:0/2325946451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca28004d40 0x7fca441f5db0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:59.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.669+0000 7fca49400640 1 -- 172.21.15.100:0/2325946451 >> 172.21.15.100:0/2325946451 conn(0x7fca4406d730 msgr2=0x7fca4415bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:59.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.670+0000 7fca49400640 1 -- 172.21.15.100:0/2325946451 shutdown_connections 2024-09-17T04:32:59.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.670+0000 7fca49400640 1 -- 172.21.15.100:0/2325946451 wait complete. 2024-09-17T04:32:59.697 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T04:32:59.697 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:32:59.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.919+0000 7fcbbdc00640 1 -- 172.21.15.100:0/1941639682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbb815d760 msgr2=0x7fcbb81535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:59.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.919+0000 7fcbbdc00640 1 --2- 172.21.15.100:0/1941639682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbb815d760 0x7fcbb81535c0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fcba8005fd0 tx=0x7fcba802aa10 comp rx=0 tx=0).stop 2024-09-17T04:32:59.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.919+0000 7fcbbdc00640 1 -- 172.21.15.100:0/1941639682 shutdown_connections 2024-09-17T04:32:59.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.919+0000 7fcbbdc00640 1 --2- 172.21.15.100:0/1941639682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbb80ad0e0 0x7fcbb80ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:59.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.919+0000 7fcbbdc00640 1 --2- 172.21.15.100:0/1941639682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbb815d760 0x7fcbb81535c0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:59.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.919+0000 7fcbbdc00640 1 -- 172.21.15.100:0/1941639682 >> 172.21.15.100:0/1941639682 conn(0x7fcbb806d730 msgr2=0x7fcbb80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:59.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.919+0000 7fcbbdc00640 1 -- 172.21.15.100:0/1941639682 shutdown_connections 2024-09-17T04:32:59.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.919+0000 7fcbbdc00640 1 -- 172.21.15.100:0/1941639682 wait complete. 2024-09-17T04:32:59.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.921+0000 7fcbbdc00640 1 Processor -- start 2024-09-17T04:32:59.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.921+0000 7fcbbdc00640 1 -- start start 2024-09-17T04:32:59.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.923+0000 7fcbbdc00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbb80ad0e0 0x7fcbb81f2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:59.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.923+0000 7fcbbdc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbb815d760 0x7fcbb81558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:59.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.923+0000 7fcbbdc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbb8160310 0x7fcbb81408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:59.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.923+0000 7fcbbdc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb806f220 con 0x7fcbb815d760 2024-09-17T04:32:59.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.923+0000 7fcbbdc00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb806f0a0 con 0x7fcbb8160310 2024-09-17T04:32:59.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.923+0000 7fcbbdc00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcbb80b4c20 con 0x7fcbb80ad0e0 2024-09-17T04:32:59.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.923+0000 7fcbb6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbb815d760 0x7fcbb81558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:59.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.923+0000 7fcbb6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbb815d760 0x7fcbb81558a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56838/0 (socket says 172.21.15.100:56838) 2024-09-17T04:32:59.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.923+0000 7fcbb6000640 1 -- 172.21.15.100:0/1988134672 learned_addr learned my addr 172.21.15.100:0/1988134672 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:32:59.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.923+0000 7fcbb6a00640 1 --2- 172.21.15.100:0/1988134672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbb80ad0e0 0x7fcbb81f2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:59.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.923+0000 7fcbb7400640 1 --2- 172.21.15.100:0/1988134672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbb8160310 0x7fcbb81408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:59.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.923+0000 7fcbb6000640 1 -- 172.21.15.100:0/1988134672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbb80ad0e0 msgr2=0x7fcbb81f2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:59.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.923+0000 7fcbb6000640 1 --2- 172.21.15.100:0/1988134672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbb80ad0e0 0x7fcbb81f2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:59.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.923+0000 7fcbb6000640 1 -- 172.21.15.100:0/1988134672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbb8160310 msgr2=0x7fcbb81408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:59.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.923+0000 7fcbb6000640 1 --2- 172.21.15.100:0/1988134672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbb8160310 0x7fcbb81408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:59.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.923+0000 7fcbb6000640 1 -- 172.21.15.100:0/1988134672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb81f3520 con 0x7fcbb815d760 2024-09-17T04:32:59.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.923+0000 7fcbb7400640 1 --2- 172.21.15.100:0/1988134672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbb8160310 0x7fcbb81408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:59.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.924+0000 7fcbb6000640 1 --2- 172.21.15.100:0/1988134672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbb815d760 0x7fcbb81558a0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fcba400d700 tx=0x7fcba400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:59.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.924+0000 7fcbaea00640 1 -- 172.21.15.100:0/1988134672 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba4017070 con 0x7fcbb815d760 2024-09-17T04:32:59.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.924+0000 7fcbaea00640 1 -- 172.21.15.100:0/1988134672 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcba4004310 con 0x7fcbb815d760 2024-09-17T04:32:59.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.924+0000 7fcbbdc00640 1 -- 172.21.15.100:0/1988134672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb8156360 con 0x7fcbb815d760 2024-09-17T04:32:59.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.924+0000 7fcbaea00640 1 -- 172.21.15.100:0/1988134672 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba4009a80 con 0x7fcbb815d760 2024-09-17T04:32:59.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.924+0000 7fcbbdc00640 1 -- 172.21.15.100:0/1988134672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb8156920 con 0x7fcbb815d760 2024-09-17T04:32:59.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.925+0000 7fcbbdc00640 1 -- 172.21.15.100:0/1988134672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbb81560d0 con 0x7fcbb815d760 2024-09-17T04:32:59.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.927+0000 7fcbaea00640 1 -- 172.21.15.100:0/1988134672 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7fcba4009c20 con 0x7fcbb815d760 2024-09-17T04:32:59.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.927+0000 7fcbaea00640 1 --2- 172.21.15.100:0/1988134672 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcb98079430 0x7fcb9807b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:59.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.928+0000 7fcbaea00640 1 -- 172.21.15.100:0/1988134672 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7fcba4099520 con 0x7fcbb815d760 2024-09-17T04:32:59.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.928+0000 7fcbb6a00640 1 --2- 172.21.15.100:0/1988134672 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcb98079430 0x7fcb9807b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:59.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.929+0000 7fcbb6a00640 1 --2- 172.21.15.100:0/1988134672 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcb98079430 0x7fcb9807b940 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcba8035000 tx=0x7fcba803b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:59.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:32:59.932+0000 7fcbaea00640 1 -- 172.21.15.100:0/1988134672 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fcba4062dc0 con 0x7fcbb815d760 2024-09-17T04:33:00.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.150+0000 7fcbbdc00640 1 -- 172.21.15.100:0/1988134672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcbb8160fa0 con 0x7fcbb815d760 2024-09-17T04:33:00.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.151+0000 7fcbaea00640 1 -- 172.21.15.100:0/1988134672 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+5241 (secure 0 0 0) 0x7fcba40677f0 con 0x7fcbb815d760 2024-09-17T04:33:00.151 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:00.151 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":102,"btime":"2024-09-17T04:32:45:357279+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7028,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1529630991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1529630991},{"type":"v1","addr":"172.21.15.184:6801","nonce":1529630991}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7906,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1516666057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1516666057},{"type":"v1","addr":"172.21.15.184:6835","nonce":1516666057}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2252725120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2252725120},{"type":"v1","addr":"172.21.15.100:6833","nonce":2252725120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:32:34.557571+0000","modified":"2024-09-17T04:32:37.568034+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8427":{"gid":8427,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/2425159512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2425159512},{"type":"v1","addr":"172.21.15.100:6835","nonce":2425159512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8427,"qdb_cluster":[8427]},"id":11}]} 2024-09-17T04:33:00.152 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 102 2024-09-17T04:33:00.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.153+0000 7fcbbdc00640 1 -- 172.21.15.100:0/1988134672 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcb98079430 msgr2=0x7fcb9807b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:00.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.153+0000 7fcbbdc00640 1 --2- 172.21.15.100:0/1988134672 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcb98079430 0x7fcb9807b940 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcba8035000 tx=0x7fcba803b040 comp rx=0 tx=0).stop 2024-09-17T04:33:00.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.153+0000 7fcbbdc00640 1 -- 172.21.15.100:0/1988134672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbb815d760 msgr2=0x7fcbb81558a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:00.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.153+0000 7fcbbdc00640 1 --2- 172.21.15.100:0/1988134672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbb815d760 0x7fcbb81558a0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fcba400d700 tx=0x7fcba400dbc0 comp rx=0 tx=0).stop 2024-09-17T04:33:00.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.153+0000 7fcbbdc00640 1 -- 172.21.15.100:0/1988134672 shutdown_connections 2024-09-17T04:33:00.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.153+0000 7fcbbdc00640 1 --2- 172.21.15.100:0/1988134672 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcb98079430 0x7fcb9807b940 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:00.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.153+0000 7fcbbdc00640 1 --2- 172.21.15.100:0/1988134672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbb8160310 0x7fcbb81408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:00.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.153+0000 7fcbbdc00640 1 --2- 172.21.15.100:0/1988134672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbb815d760 0x7fcbb81558a0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:00.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.153+0000 7fcbbdc00640 1 --2- 172.21.15.100:0/1988134672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbb80ad0e0 0x7fcbb81f2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:00.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.153+0000 7fcbbdc00640 1 -- 172.21.15.100:0/1988134672 >> 172.21.15.100:0/1988134672 conn(0x7fcbb806d730 msgr2=0x7fcbb8160720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:00.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.153+0000 7fcbbdc00640 1 -- 172.21.15.100:0/1988134672 shutdown_connections 2024-09-17T04:33:00.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.153+0000 7fcbbdc00640 1 -- 172.21.15.100:0/1988134672 wait complete. 2024-09-17T04:33:00.180 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:33:00.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.399+0000 7f6cc7e00640 1 -- 172.21.15.100:0/2367733171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cc015d760 msgr2=0x7f6cc01535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:00.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.399+0000 7f6cc7e00640 1 --2- 172.21.15.100:0/2367733171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cc015d760 0x7f6cc01535c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4005fd0 tx=0x7f6cb402aa10 comp rx=0 tx=0).stop 2024-09-17T04:33:00.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.399+0000 7f6cc7e00640 1 -- 172.21.15.100:0/2367733171 shutdown_connections 2024-09-17T04:33:00.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.399+0000 7f6cc7e00640 1 --2- 172.21.15.100:0/2367733171 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cc00b3550 0x7f6cc00b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:00.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.399+0000 7f6cc7e00640 1 --2- 172.21.15.100:0/2367733171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cc015d760 0x7f6cc01535c0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:00.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.399+0000 7f6cc7e00640 1 -- 172.21.15.100:0/2367733171 >> 172.21.15.100:0/2367733171 conn(0x7f6cc006d730 msgr2=0x7f6cc00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:00.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.399+0000 7f6cc7e00640 1 -- 172.21.15.100:0/2367733171 shutdown_connections 2024-09-17T04:33:00.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.399+0000 7f6cc7e00640 1 -- 172.21.15.100:0/2367733171 wait complete. 2024-09-17T04:33:00.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.402+0000 7f6cc7e00640 1 Processor -- start 2024-09-17T04:33:00.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.402+0000 7f6cc7e00640 1 -- start start 2024-09-17T04:33:00.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.403+0000 7f6cc7e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cc00ad0e0 0x7f6cc01f6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:00.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.403+0000 7f6cc7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cc00b3550 0x7f6cc014eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:00.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.403+0000 7f6cc7e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cc015d760 0x7f6cc01f6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:00.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.403+0000 7f6cc7e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc006f240 con 0x7f6cc00b3550 2024-09-17T04:33:00.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.403+0000 7f6cc7e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc00b4e30 con 0x7f6cc015d760 2024-09-17T04:33:00.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.403+0000 7f6cc7e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6cc00b3ea0 con 0x7f6cc00ad0e0 2024-09-17T04:33:00.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.404+0000 7f6cc4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cc00b3550 0x7f6cc014eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:00.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.404+0000 7f6cc4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cc00b3550 0x7f6cc014eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56842/0 (socket says 172.21.15.100:56842) 2024-09-17T04:33:00.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.404+0000 7f6cc4a00640 1 -- 172.21.15.100:0/2697422962 learned_addr learned my addr 172.21.15.100:0/2697422962 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:00.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.404+0000 7f6cc5400640 1 --2- 172.21.15.100:0/2697422962 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cc00ad0e0 0x7f6cc01f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:00.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.404+0000 7f6cc5e00640 1 --2- 172.21.15.100:0/2697422962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cc015d760 0x7f6cc01f6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:00.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.404+0000 7f6cc4a00640 1 -- 172.21.15.100:0/2697422962 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cc00ad0e0 msgr2=0x7f6cc01f6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:00.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.404+0000 7f6cc4a00640 1 --2- 172.21.15.100:0/2697422962 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cc00ad0e0 0x7f6cc01f6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:00.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.404+0000 7f6cc4a00640 1 -- 172.21.15.100:0/2697422962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cc015d760 msgr2=0x7f6cc01f6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:00.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.404+0000 7f6cc4a00640 1 --2- 172.21.15.100:0/2697422962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cc015d760 0x7f6cc01f6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:00.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.404+0000 7f6cc4a00640 1 -- 172.21.15.100:0/2697422962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc01f69a0 con 0x7f6cc00b3550 2024-09-17T04:33:00.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.405+0000 7f6cc4a00640 1 --2- 172.21.15.100:0/2697422962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cc00b3550 0x7f6cc014eda0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f6cac00c3a0 tx=0x7f6cac00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:00.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.405+0000 7f6cbd600640 1 -- 172.21.15.100:0/2697422962 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cac017070 con 0x7f6cc00b3550 2024-09-17T04:33:00.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.405+0000 7f6cbd600640 1 -- 172.21.15.100:0/2697422962 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6cac0053c0 con 0x7f6cc00b3550 2024-09-17T04:33:00.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.405+0000 7f6cbd600640 1 -- 172.21.15.100:0/2697422962 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cac003850 con 0x7f6cc00b3550 2024-09-17T04:33:00.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.405+0000 7f6cc7e00640 1 -- 172.21.15.100:0/2697422962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cc0154f50 con 0x7f6cc00b3550 2024-09-17T04:33:00.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.405+0000 7f6cc7e00640 1 -- 172.21.15.100:0/2697422962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cc0155510 con 0x7f6cc00b3550 2024-09-17T04:33:00.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.406+0000 7f6cc7e00640 1 -- 172.21.15.100:0/2697422962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cc014f610 con 0x7f6cc00b3550 2024-09-17T04:33:00.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.407+0000 7f6cbd600640 1 -- 172.21.15.100:0/2697422962 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f6cac01a070 con 0x7f6cc00b3550 2024-09-17T04:33:00.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.408+0000 7f6cbd600640 1 --2- 172.21.15.100:0/2697422962 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6ca0079400 0x7f6ca007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:00.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.408+0000 7f6cbd600640 1 -- 172.21.15.100:0/2697422962 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7f6cac09b300 con 0x7f6cc00b3550 2024-09-17T04:33:00.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.408+0000 7f6cc5400640 1 --2- 172.21.15.100:0/2697422962 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6ca0079400 0x7f6ca007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:00.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.409+0000 7f6cc5400640 1 --2- 172.21.15.100:0/2697422962 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6ca0079400 0x7f6ca007b910 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4035000 tx=0x7f6cb403b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:00.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.412+0000 7f6cbd600640 1 -- 172.21.15.100:0/2697422962 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6cac062110 con 0x7f6cc00b3550 2024-09-17T04:33:00.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.631+0000 7f6cc7e00640 1 -- 172.21.15.100:0/2697422962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6cc00b42f0 con 0x7f6cc00b3550 2024-09-17T04:33:00.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.632+0000 7f6cbd600640 1 -- 172.21.15.100:0/2697422962 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+5241 (secure 0 0 0) 0x7f6cac066b40 con 0x7f6cc00b3550 2024-09-17T04:33:00.633 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:00.633 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":102,"btime":"2024-09-17T04:32:45:357279+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7028,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1529630991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1529630991},{"type":"v1","addr":"172.21.15.184:6801","nonce":1529630991}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7906,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1516666057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1516666057},{"type":"v1","addr":"172.21.15.184:6835","nonce":1516666057}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2252725120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2252725120},{"type":"v1","addr":"172.21.15.100:6833","nonce":2252725120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:32:34.557571+0000","modified":"2024-09-17T04:32:37.568034+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8427":{"gid":8427,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/2425159512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2425159512},{"type":"v1","addr":"172.21.15.100:6835","nonce":2425159512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8427,"qdb_cluster":[8427]},"id":11}]} 2024-09-17T04:33:00.633 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 102 2024-09-17T04:33:00.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.635+0000 7f6cc7e00640 1 -- 172.21.15.100:0/2697422962 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6ca0079400 msgr2=0x7f6ca007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:00.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.635+0000 7f6cc7e00640 1 --2- 172.21.15.100:0/2697422962 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6ca0079400 0x7f6ca007b910 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4035000 tx=0x7f6cb403b040 comp rx=0 tx=0).stop 2024-09-17T04:33:00.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.635+0000 7f6cc7e00640 1 -- 172.21.15.100:0/2697422962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cc00b3550 msgr2=0x7f6cc014eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:00.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.635+0000 7f6cc7e00640 1 --2- 172.21.15.100:0/2697422962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cc00b3550 0x7f6cc014eda0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f6cac00c3a0 tx=0x7f6cac00c860 comp rx=0 tx=0).stop 2024-09-17T04:33:00.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.635+0000 7f6cc7e00640 1 -- 172.21.15.100:0/2697422962 shutdown_connections 2024-09-17T04:33:00.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.635+0000 7f6cc7e00640 1 --2- 172.21.15.100:0/2697422962 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6ca0079400 0x7f6ca007b910 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:00.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.635+0000 7f6cc7e00640 1 --2- 172.21.15.100:0/2697422962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cc015d760 0x7f6cc01f6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:00.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.635+0000 7f6cc7e00640 1 --2- 172.21.15.100:0/2697422962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cc00b3550 0x7f6cc014eda0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:00.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.635+0000 7f6cc7e00640 1 --2- 172.21.15.100:0/2697422962 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cc00ad0e0 0x7f6cc01f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:00.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.635+0000 7f6cc7e00640 1 -- 172.21.15.100:0/2697422962 >> 172.21.15.100:0/2697422962 conn(0x7f6cc006d730 msgr2=0x7f6cc015eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:00.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.635+0000 7f6cc7e00640 1 -- 172.21.15.100:0/2697422962 shutdown_connections 2024-09-17T04:33:00.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.635+0000 7f6cc7e00640 1 -- 172.21.15.100:0/2697422962 wait complete. 2024-09-17T04:33:00.657 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:33:00.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.880+0000 7f876ce00640 1 -- 172.21.15.100:0/3817229241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87680b9840 msgr2=0x7f87680d4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:00.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.880+0000 7f876ce00640 1 --2- 172.21.15.100:0/3817229241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87680b9840 0x7f87680d4fc0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f87580098e0 tx=0x7f875802a5d0 comp rx=0 tx=0).stop 2024-09-17T04:33:00.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.880+0000 7f876ce00640 1 -- 172.21.15.100:0/3817229241 shutdown_connections 2024-09-17T04:33:00.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.880+0000 7f876ce00640 1 --2- 172.21.15.100:0/3817229241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87680b9840 0x7f87680d4fc0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:00.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.880+0000 7f876ce00640 1 -- 172.21.15.100:0/3817229241 >> 172.21.15.100:0/3817229241 conn(0x7f87680d44a0 msgr2=0x7f87680d48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:00.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.880+0000 7f876ce00640 1 -- 172.21.15.100:0/3817229241 shutdown_connections 2024-09-17T04:33:00.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.880+0000 7f876ce00640 1 -- 172.21.15.100:0/3817229241 wait complete. 2024-09-17T04:33:00.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.882+0000 7f876ce00640 1 Processor -- start 2024-09-17T04:33:00.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.883+0000 7f876ce00640 1 -- start start 2024-09-17T04:33:00.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.884+0000 7f876ce00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87680b35c0 0x7f87680c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:00.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.884+0000 7f876ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87680b9840 0x7f87680cb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:00.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.884+0000 7f876ce00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87680cfd40 0x7f87680cb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:00.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.884+0000 7f876ce00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f87680d0150 con 0x7f87680b9840 2024-09-17T04:33:00.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.884+0000 7f876ce00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f87680d1630 con 0x7f87680cfd40 2024-09-17T04:33:00.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.884+0000 7f876ce00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f87680d02d0 con 0x7f87680b35c0 2024-09-17T04:33:00.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.884+0000 7f875ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87680b9840 0x7f87680cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:00.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.884+0000 7f875ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87680b9840 0x7f87680cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56868/0 (socket says 172.21.15.100:56868) 2024-09-17T04:33:00.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.884+0000 7f875ea00640 1 -- 172.21.15.100:0/486294666 learned_addr learned my addr 172.21.15.100:0/486294666 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:00.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.884+0000 7f8767400640 1 --2- 172.21.15.100:0/486294666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87680b35c0 0x7f87680c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:00.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.884+0000 7f8767e00640 1 --2- 172.21.15.100:0/486294666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87680cfd40 0x7f87680cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:00.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.885+0000 7f875ea00640 1 -- 172.21.15.100:0/486294666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87680b35c0 msgr2=0x7f87680c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:00.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.885+0000 7f875ea00640 1 --2- 172.21.15.100:0/486294666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87680b35c0 0x7f87680c9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:00.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.885+0000 7f875ea00640 1 -- 172.21.15.100:0/486294666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87680cfd40 msgr2=0x7f87680cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:00.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.885+0000 7f875ea00640 1 --2- 172.21.15.100:0/486294666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87680cfd40 0x7f87680cb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:00.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.885+0000 7f875ea00640 1 -- 172.21.15.100:0/486294666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87680ce290 con 0x7f87680b9840 2024-09-17T04:33:00.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.885+0000 7f875ea00640 1 --2- 172.21.15.100:0/486294666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87680b9840 0x7f87680cb230 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f875000c3c0 tx=0x7f875000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:00.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.885+0000 7f875fe00640 1 -- 172.21.15.100:0/486294666 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750017070 con 0x7f87680b9840 2024-09-17T04:33:00.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.885+0000 7f875fe00640 1 -- 172.21.15.100:0/486294666 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87500053e0 con 0x7f87680b9840 2024-09-17T04:33:00.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.885+0000 7f875fe00640 1 -- 172.21.15.100:0/486294666 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750003870 con 0x7f87680b9840 2024-09-17T04:33:00.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.885+0000 7f876ce00640 1 -- 172.21.15.100:0/486294666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87680cd710 con 0x7f87680b9840 2024-09-17T04:33:00.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.886+0000 7f876ce00640 1 -- 172.21.15.100:0/486294666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87680cdcd0 con 0x7f87680b9840 2024-09-17T04:33:00.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.887+0000 7f876ce00640 1 -- 172.21.15.100:0/486294666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f876808cc70 con 0x7f87680b9840 2024-09-17T04:33:00.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.888+0000 7f875fe00640 1 -- 172.21.15.100:0/486294666 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f875001a070 con 0x7f87680b9840 2024-09-17T04:33:00.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.889+0000 7f875fe00640 1 --2- 172.21.15.100:0/486294666 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8744079400 0x7f874407b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:00.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.889+0000 7f875fe00640 1 -- 172.21.15.100:0/486294666 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7f8750098810 con 0x7f87680b9840 2024-09-17T04:33:00.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.889+0000 7f8767400640 1 --2- 172.21.15.100:0/486294666 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8744079400 0x7f874407b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:00.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.890+0000 7f8767400640 1 --2- 172.21.15.100:0/486294666 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8744079400 0x7f874407b910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8758034000 tx=0x7f8758004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:00.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:00.893+0000 7f875fe00640 1 -- 172.21.15.100:0/486294666 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f876808cc70 con 0x7f87680b9840 2024-09-17T04:33:01.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.078+0000 7f876ce00640 1 -- 172.21.15.100:0/486294666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f87680d4fc0 con 0x7f87680b9840 2024-09-17T04:33:01.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.079+0000 7f875fe00640 1 -- 172.21.15.100:0/486294666 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16868 (secure 0 0 0) 0x7f87500620b0 con 0x7f87680b9840 2024-09-17T04:33:01.080 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:01.080 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":145,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:32:36.070033+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:32:32.462209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-17T04:32:33.516803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:33:01.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.082+0000 7f876ce00640 1 -- 172.21.15.100:0/486294666 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8744079400 msgr2=0x7f874407b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:01.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.082+0000 7f876ce00640 1 --2- 172.21.15.100:0/486294666 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8744079400 0x7f874407b910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8758034000 tx=0x7f8758004730 comp rx=0 tx=0).stop 2024-09-17T04:33:01.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.082+0000 7f876ce00640 1 -- 172.21.15.100:0/486294666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87680b9840 msgr2=0x7f87680cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:01.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.082+0000 7f876ce00640 1 --2- 172.21.15.100:0/486294666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87680b9840 0x7f87680cb230 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f875000c3c0 tx=0x7f875000c880 comp rx=0 tx=0).stop 2024-09-17T04:33:01.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.082+0000 7f876ce00640 1 -- 172.21.15.100:0/486294666 shutdown_connections 2024-09-17T04:33:01.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.082+0000 7f876ce00640 1 --2- 172.21.15.100:0/486294666 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8744079400 0x7f874407b910 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:01.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.082+0000 7f876ce00640 1 --2- 172.21.15.100:0/486294666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87680cfd40 0x7f87680cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:01.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.082+0000 7f876ce00640 1 --2- 172.21.15.100:0/486294666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87680b9840 0x7f87680cb230 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:01.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.082+0000 7f876ce00640 1 --2- 172.21.15.100:0/486294666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87680b35c0 0x7f87680c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:01.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.082+0000 7f876ce00640 1 -- 172.21.15.100:0/486294666 >> 172.21.15.100:0/486294666 conn(0x7f87680d44a0 msgr2=0x7f87680d09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:01.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.082+0000 7f876ce00640 1 -- 172.21.15.100:0/486294666 shutdown_connections 2024-09-17T04:33:01.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.082+0000 7f876ce00640 1 -- 172.21.15.100:0/486294666 wait complete. 2024-09-17T04:33:01.106 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T04:33:01.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.325+0000 7f43fde00640 1 -- 172.21.15.100:0/3081097214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43f80ad1c0 msgr2=0x7f43f80ad5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:01.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.325+0000 7f43fde00640 1 --2- 172.21.15.100:0/3081097214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43f80ad1c0 0x7f43f80ad5d0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f43f806dd40 tx=0x7f43f002a8f0 comp rx=0 tx=0).stop 2024-09-17T04:33:01.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.325+0000 7f43fde00640 1 -- 172.21.15.100:0/3081097214 shutdown_connections 2024-09-17T04:33:01.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.325+0000 7f43fde00640 1 --2- 172.21.15.100:0/3081097214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43f80ad1c0 0x7f43f80ad5d0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:01.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.325+0000 7f43fde00640 1 --2- 172.21.15.100:0/3081097214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43f80ce490 0x7f43f80ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:01.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.325+0000 7f43fde00640 1 --2- 172.21.15.100:0/3081097214 >> v2:172.21.15.184:3300/0 conn(0x7f43f8070660 0x7f43f80af6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:01.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.326+0000 7f43fde00640 1 -- 172.21.15.100:0/3081097214 >> 172.21.15.100:0/3081097214 conn(0x7f43f806d3d0 msgr2=0x7f43f80b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:01.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.326+0000 7f43fde00640 1 -- 172.21.15.100:0/3081097214 shutdown_connections 2024-09-17T04:33:01.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.326+0000 7f43fde00640 1 -- 172.21.15.100:0/3081097214 wait complete. 2024-09-17T04:33:01.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.328+0000 7f43fde00640 1 Processor -- start 2024-09-17T04:33:01.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.328+0000 7f43fde00640 1 -- start start 2024-09-17T04:33:01.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.329+0000 7f43fde00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43f8070660 0x7f43f80c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:01.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.329+0000 7f43fde00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43f80ad1c0 0x7f43f80caab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:01.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.329+0000 7f43fde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43f80ce490 0x7f43f80caff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:01.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.329+0000 7f43fde00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f43f80adea0 con 0x7f43f80ce490 2024-09-17T04:33:01.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.329+0000 7f43fde00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f43f80add20 con 0x7f43f80ad1c0 2024-09-17T04:33:01.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.329+0000 7f43fde00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f43f80cca80 con 0x7f43f8070660 2024-09-17T04:33:01.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.330+0000 7f43f7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43f80ce490 0x7f43f80caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:01.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.330+0000 7f43f7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43f80ce490 0x7f43f80caff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56888/0 (socket says 172.21.15.100:56888) 2024-09-17T04:33:01.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.330+0000 7f43f7400640 1 -- 172.21.15.100:0/3837619209 learned_addr learned my addr 172.21.15.100:0/3837619209 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:01.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.330+0000 7f43f6a00640 1 --2- 172.21.15.100:0/3837619209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43f8070660 0x7f43f80c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:01.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.330+0000 7f43f6000640 1 --2- 172.21.15.100:0/3837619209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43f80ad1c0 0x7f43f80caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:01.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.330+0000 7f43f7400640 1 -- 172.21.15.100:0/3837619209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43f8070660 msgr2=0x7f43f80c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:01.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.330+0000 7f43f7400640 1 --2- 172.21.15.100:0/3837619209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43f8070660 0x7f43f80c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:01.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.330+0000 7f43f7400640 1 -- 172.21.15.100:0/3837619209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43f80ad1c0 msgr2=0x7f43f80caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:01.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.330+0000 7f43f7400640 1 --2- 172.21.15.100:0/3837619209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43f80ad1c0 0x7f43f80caab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:01.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.330+0000 7f43f7400640 1 -- 172.21.15.100:0/3837619209 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f80c3ec0 con 0x7f43f80ce490 2024-09-17T04:33:01.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.330+0000 7f43f7400640 1 --2- 172.21.15.100:0/3837619209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43f80ce490 0x7f43f80caff0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f43f0009d60 tx=0x7f43f0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:01.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.331+0000 7f43ef400640 1 -- 172.21.15.100:0/3837619209 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f0004040 con 0x7f43f80ce490 2024-09-17T04:33:01.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.331+0000 7f43ef400640 1 -- 172.21.15.100:0/3837619209 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43f002d070 con 0x7f43f80ce490 2024-09-17T04:33:01.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.331+0000 7f43fde00640 1 -- 172.21.15.100:0/3837619209 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f8074370 con 0x7f43f80ce490 2024-09-17T04:33:01.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.331+0000 7f43ef400640 1 -- 172.21.15.100:0/3837619209 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f0004460 con 0x7f43f80ce490 2024-09-17T04:33:01.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.331+0000 7f43fde00640 1 -- 172.21.15.100:0/3837619209 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f81f77f0 con 0x7f43f80ce490 2024-09-17T04:33:01.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.332+0000 7f43fde00640 1 -- 172.21.15.100:0/3837619209 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43f80ad5d0 con 0x7f43f80ce490 2024-09-17T04:33:01.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.333+0000 7f43ef400640 1 -- 172.21.15.100:0/3837619209 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f43f0004600 con 0x7f43f80ce490 2024-09-17T04:33:01.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.334+0000 7f43ef400640 1 --2- 172.21.15.100:0/3837619209 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f43d8079400 0x7f43d807b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:01.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.334+0000 7f43ef400640 1 -- 172.21.15.100:0/3837619209 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5765+0+0 (secure 0 0 0) 0x7f43f00bd410 con 0x7f43f80ce490 2024-09-17T04:33:01.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.335+0000 7f43f6a00640 1 --2- 172.21.15.100:0/3837619209 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f43d8079400 0x7f43d807b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:01.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.335+0000 7f43f6a00640 1 --2- 172.21.15.100:0/3837619209 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f43d8079400 0x7f43d807b910 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f43f8072000 tx=0x7f43e8005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:01.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.339+0000 7f43ef400640 1 -- 172.21.15.100:0/3837619209 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f43f0084d10 con 0x7f43f80ce490 2024-09-17T04:33:01.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.554+0000 7f43fde00640 1 -- 172.21.15.100:0/3837619209 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f43f81f72e0 con 0x7f43f80ce490 2024-09-17T04:33:01.555 INFO:tasks.ceph.mon.a.smithi100.stderr:2024-09-17T04:33:01.555+0000 7fd3d0a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T04:33:01.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.557+0000 7f43ef400640 1 -- 172.21.15.100:0/3837619209 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v103) ==== 161+0+0 (secure 0 0 0) 0x7f43f0084610 con 0x7f43f80ce490 2024-09-17T04:33:01.558 INFO:teuthology.orchestra.run.smithi100.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T04:33:01.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.559+0000 7f43fde00640 1 -- 172.21.15.100:0/3837619209 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f43d8079400 msgr2=0x7f43d807b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:01.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.559+0000 7f43fde00640 1 --2- 172.21.15.100:0/3837619209 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f43d8079400 0x7f43d807b910 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f43f8072000 tx=0x7f43e8005f90 comp rx=0 tx=0).stop 2024-09-17T04:33:01.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.560+0000 7f43fde00640 1 -- 172.21.15.100:0/3837619209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43f80ce490 msgr2=0x7f43f80caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:01.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.560+0000 7f43fde00640 1 --2- 172.21.15.100:0/3837619209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43f80ce490 0x7f43f80caff0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f43f0009d60 tx=0x7f43f0003ba0 comp rx=0 tx=0).stop 2024-09-17T04:33:01.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.560+0000 7f43fde00640 1 -- 172.21.15.100:0/3837619209 shutdown_connections 2024-09-17T04:33:01.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.560+0000 7f43fde00640 1 --2- 172.21.15.100:0/3837619209 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f43d8079400 0x7f43d807b910 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:01.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.560+0000 7f43fde00640 1 --2- 172.21.15.100:0/3837619209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43f80ce490 0x7f43f80caff0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:01.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.560+0000 7f43fde00640 1 --2- 172.21.15.100:0/3837619209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43f80ad1c0 0x7f43f80caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:01.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.560+0000 7f43fde00640 1 --2- 172.21.15.100:0/3837619209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43f8070660 0x7f43f80c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:01.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.560+0000 7f43fde00640 1 -- 172.21.15.100:0/3837619209 >> 172.21.15.100:0/3837619209 conn(0x7f43f806d3d0 msgr2=0x7f43f80b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:01.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.560+0000 7f43fde00640 1 -- 172.21.15.100:0/3837619209 shutdown_connections 2024-09-17T04:33:01.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.560+0000 7f43fde00640 1 -- 172.21.15.100:0/3837619209 wait complete. 2024-09-17T04:33:01.584 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T04:33:01.697 INFO:tasks.ceph.mds.a.smithi100.stdout:starting mds.a at 2024-09-17T04:33:01.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.814+0000 7f950e800640 1 -- 172.21.15.100:0/3037217995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95080b42a0 msgr2=0x7f95081624b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:01.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.814+0000 7f950e800640 1 --2- 172.21.15.100:0/3037217995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95080b42a0 0x7f95081624b0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f94f4005fd0 tx=0x7f94f402a540 comp rx=0 tx=0).stop 2024-09-17T04:33:01.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.814+0000 7f950e800640 1 -- 172.21.15.100:0/3037217995 shutdown_connections 2024-09-17T04:33:01.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.814+0000 7f950e800640 1 --2- 172.21.15.100:0/3037217995 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95080cf2c0 0x7f95080cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:01.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.814+0000 7f950e800640 1 --2- 172.21.15.100:0/3037217995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95080b42a0 0x7f95081624b0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:01.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.814+0000 7f950e800640 1 -- 172.21.15.100:0/3037217995 >> 172.21.15.100:0/3037217995 conn(0x7f950806d620 msgr2=0x7f95080b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:01.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.814+0000 7f950e800640 1 -- 172.21.15.100:0/3037217995 shutdown_connections 2024-09-17T04:33:01.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.814+0000 7f950e800640 1 -- 172.21.15.100:0/3037217995 wait complete. 2024-09-17T04:33:01.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.816+0000 7f950e800640 1 Processor -- start 2024-09-17T04:33:01.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.816+0000 7f950e800640 1 -- start start 2024-09-17T04:33:01.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.817+0000 7f950e800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95080742b0 0x7f95080998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:01.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.818+0000 7f950e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95080b42a0 0x7f95081f9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:01.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.818+0000 7f950e800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95080cf2c0 0x7f95080ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:01.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.818+0000 7f950e800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9508073240 con 0x7f95080b42a0 2024-09-17T04:33:01.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.818+0000 7f950e800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f95080730c0 con 0x7f95080742b0 2024-09-17T04:33:01.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.818+0000 7f950e800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f95080733c0 con 0x7f95080cf2c0 2024-09-17T04:33:01.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.818+0000 7f950ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95080b42a0 0x7f95081f9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:01.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.818+0000 7f950ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95080b42a0 0x7f95081f9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56914/0 (socket says 172.21.15.100:56914) 2024-09-17T04:33:01.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.818+0000 7f950ca00640 1 -- 172.21.15.100:0/809320611 learned_addr learned my addr 172.21.15.100:0/809320611 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:01.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.818+0000 7f950d400640 1 --2- 172.21.15.100:0/809320611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95080742b0 0x7f95080998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:01.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.818+0000 7f950de00640 1 --2- 172.21.15.100:0/809320611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95080cf2c0 0x7f95080ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:01.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.818+0000 7f950ca00640 1 -- 172.21.15.100:0/809320611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95080cf2c0 msgr2=0x7f95080ae0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:01.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.818+0000 7f950ca00640 1 --2- 172.21.15.100:0/809320611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95080cf2c0 0x7f95080ae0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:01.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.818+0000 7f950ca00640 1 -- 172.21.15.100:0/809320611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95080742b0 msgr2=0x7f95080998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:01.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.818+0000 7f950ca00640 1 --2- 172.21.15.100:0/809320611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95080742b0 0x7f95080998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:01.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.818+0000 7f950ca00640 1 -- 172.21.15.100:0/809320611 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95081f9e30 con 0x7f95080b42a0 2024-09-17T04:33:01.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.819+0000 7f950ca00640 1 --2- 172.21.15.100:0/809320611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95080b42a0 0x7f95081f9100 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f94f4034040 tx=0x7f94f40038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:01.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.819+0000 7f9505600640 1 -- 172.21.15.100:0/809320611 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f403e030 con 0x7f95080b42a0 2024-09-17T04:33:01.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.819+0000 7f9505600640 1 -- 172.21.15.100:0/809320611 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94f4003d90 con 0x7f95080b42a0 2024-09-17T04:33:01.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.819+0000 7f950e800640 1 -- 172.21.15.100:0/809320611 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9508154490 con 0x7f95080b42a0 2024-09-17T04:33:01.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.819+0000 7f9505600640 1 -- 172.21.15.100:0/809320611 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f4035040 con 0x7f95080b42a0 2024-09-17T04:33:01.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.819+0000 7f950e800640 1 -- 172.21.15.100:0/809320611 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9508099de0 con 0x7f95080b42a0 2024-09-17T04:33:01.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.820+0000 7f950e800640 1 -- 172.21.15.100:0/809320611 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95081624b0 con 0x7f95080b42a0 2024-09-17T04:33:01.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.821+0000 7f9505600640 1 -- 172.21.15.100:0/809320611 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f94f4041030 con 0x7f95080b42a0 2024-09-17T04:33:01.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.823+0000 7f9505600640 1 --2- 172.21.15.100:0/809320611 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f94e8079400 0x7f94e807b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:01.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.823+0000 7f9505600640 1 -- 172.21.15.100:0/809320611 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5851+0+0 (secure 0 0 0) 0x7f94f40bc6c0 con 0x7f95080b42a0 2024-09-17T04:33:01.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.823+0000 7f950d400640 1 --2- 172.21.15.100:0/809320611 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f94e8079400 0x7f94e807b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:01.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.824+0000 7f950d400640 1 --2- 172.21.15.100:0/809320611 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f94e8079400 0x7f94e807b910 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f94fc001720 tx=0x7f94fc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:01.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:01.827+0000 7f9505600640 1 -- 172.21.15.100:0/809320611 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f94f4083540 con 0x7f95080b42a0 2024-09-17T04:33:02.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.050+0000 7f950e800640 1 -- 172.21.15.100:0/809320611 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f95081f9bc0 con 0x7f95080b42a0 2024-09-17T04:33:02.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.563+0000 7f9505600640 1 -- 172.21.15.100:0/809320611 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v104) ==== 104+0+0 (secure 0 0 0) 0x7f94f4087f70 con 0x7f95080b42a0 2024-09-17T04:33:02.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.566+0000 7f950e800640 1 -- 172.21.15.100:0/809320611 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f94e8079400 msgr2=0x7f94e807b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:02.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.566+0000 7f950e800640 1 --2- 172.21.15.100:0/809320611 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f94e8079400 0x7f94e807b910 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f94fc001720 tx=0x7f94fc009040 comp rx=0 tx=0).stop 2024-09-17T04:33:02.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.566+0000 7f950e800640 1 -- 172.21.15.100:0/809320611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95080b42a0 msgr2=0x7f95081f9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:02.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.566+0000 7f950e800640 1 --2- 172.21.15.100:0/809320611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95080b42a0 0x7f95081f9100 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f94f4034040 tx=0x7f94f40038a0 comp rx=0 tx=0).stop 2024-09-17T04:33:02.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.566+0000 7f950e800640 1 -- 172.21.15.100:0/809320611 shutdown_connections 2024-09-17T04:33:02.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.566+0000 7f950e800640 1 --2- 172.21.15.100:0/809320611 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f94e8079400 0x7f94e807b910 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:02.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.566+0000 7f950e800640 1 --2- 172.21.15.100:0/809320611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95080cf2c0 0x7f95080ae0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:02.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.566+0000 7f950e800640 1 --2- 172.21.15.100:0/809320611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95080b42a0 0x7f95081f9100 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:02.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.566+0000 7f950e800640 1 --2- 172.21.15.100:0/809320611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95080742b0 0x7f95080998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:02.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.566+0000 7f950e800640 1 -- 172.21.15.100:0/809320611 >> 172.21.15.100:0/809320611 conn(0x7f950806d620 msgr2=0x7f9508161b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:02.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.566+0000 7f950e800640 1 -- 172.21.15.100:0/809320611 shutdown_connections 2024-09-17T04:33:02.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.566+0000 7f950e800640 1 -- 172.21.15.100:0/809320611 wait complete. 2024-09-17T04:33:02.594 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T04:33:02.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.821+0000 7fa8fbc00640 1 -- 172.21.15.100:0/510035230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8dc003a00 msgr2=0x7fa8dc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:02.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.821+0000 7fa8fbc00640 1 --2- 172.21.15.100:0/510035230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8dc003a00 0x7fa8dc003e70 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fa8e800c310 tx=0x7fa8dc005830 comp rx=0 tx=0).stop 2024-09-17T04:33:02.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.821+0000 7fa8f9c00640 1 -- 172.21.15.100:0/510035230 reap_dead start 2024-09-17T04:33:02.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.822+0000 7fa8fbc00640 1 -- 172.21.15.100:0/510035230 shutdown_connections 2024-09-17T04:33:02.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.822+0000 7fa8fbc00640 1 -- 172.21.15.100:0/510035230 >> 172.21.15.100:0/510035230 conn(0x7fa8f40b2540 msgr2=0x7fa8f40d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:02.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.822+0000 7fa8fbc00640 1 -- 172.21.15.100:0/510035230 shutdown_connections 2024-09-17T04:33:02.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.822+0000 7fa8fbc00640 1 -- 172.21.15.100:0/510035230 wait complete. 2024-09-17T04:33:02.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.824+0000 7fa8fbc00640 1 Processor -- start 2024-09-17T04:33:02.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.824+0000 7fa8fbc00640 1 -- start start 2024-09-17T04:33:02.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.825+0000 7fa8fbc00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8dc003a00 0x7fa8f4156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:02.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.825+0000 7fa8fbc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8f41e2460 0x7fa8f41e75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:02.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.825+0000 7fa8fbc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f41e7ae0 0x7fa8f41e3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:02.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.825+0000 7fa8fbc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f40b3120 con 0x7fa8f41e7ae0 2024-09-17T04:33:02.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.825+0000 7fa8fbc00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f40afbb0 con 0x7fa8f41e2460 2024-09-17T04:33:02.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.825+0000 7fa8fbc00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa8f40b32a0 con 0x7fa8dc003a00 2024-09-17T04:33:02.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.826+0000 7fa8f9c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f41e7ae0 0x7fa8f41e3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:02.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.826+0000 7fa8f9c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f41e7ae0 0x7fa8f41e3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56930/0 (socket says 172.21.15.100:56930) 2024-09-17T04:33:02.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.826+0000 7fa8f9c00640 1 -- 172.21.15.100:0/2111640715 learned_addr learned my addr 172.21.15.100:0/2111640715 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:02.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.826+0000 7fa8f3e00640 1 --2- 172.21.15.100:0/2111640715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8f41e2460 0x7fa8f41e75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:02.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.826+0000 7fa8f9200640 1 --2- 172.21.15.100:0/2111640715 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8dc003a00 0x7fa8f4156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:02.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.826+0000 7fa8f9c00640 1 -- 172.21.15.100:0/2111640715 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8dc003a00 msgr2=0x7fa8f4156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:02.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.826+0000 7fa8f9c00640 1 --2- 172.21.15.100:0/2111640715 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8dc003a00 0x7fa8f4156d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:02.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.826+0000 7fa8f9c00640 1 -- 172.21.15.100:0/2111640715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8f41e2460 msgr2=0x7fa8f41e75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:02.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.826+0000 7fa8f9c00640 1 --2- 172.21.15.100:0/2111640715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8f41e2460 0x7fa8f41e75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:02.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.826+0000 7fa8f9c00640 1 -- 172.21.15.100:0/2111640715 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f41565d0 con 0x7fa8f41e7ae0 2024-09-17T04:33:02.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.826+0000 7fa8f9200640 1 --2- 172.21.15.100:0/2111640715 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8dc003a00 0x7fa8f4156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:02.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.826+0000 7fa8f9c00640 1 --2- 172.21.15.100:0/2111640715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f41e7ae0 0x7fa8f41e3cb0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec005fd0 tx=0x7fa8ec011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:02.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.827+0000 7fa8f0c00640 1 -- 172.21.15.100:0/2111640715 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec00fd70 con 0x7fa8f41e7ae0 2024-09-17T04:33:02.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.827+0000 7fa8f0c00640 1 -- 172.21.15.100:0/2111640715 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8ec004b20 con 0x7fa8f41e7ae0 2024-09-17T04:33:02.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.827+0000 7fa8f0c00640 1 -- 172.21.15.100:0/2111640715 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec003970 con 0x7fa8f41e7ae0 2024-09-17T04:33:02.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.827+0000 7fa8fbc00640 1 -- 172.21.15.100:0/2111640715 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f4069620 con 0x7fa8f41e7ae0 2024-09-17T04:33:02.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.827+0000 7fa8fbc00640 1 -- 172.21.15.100:0/2111640715 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f406f6e0 con 0x7fa8f41e7ae0 2024-09-17T04:33:02.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.828+0000 7fa8fbc00640 1 -- 172.21.15.100:0/2111640715 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8f40d1030 con 0x7fa8f41e7ae0 2024-09-17T04:33:02.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.829+0000 7fa8f0c00640 1 -- 172.21.15.100:0/2111640715 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7fa8ec020020 con 0x7fa8f41e7ae0 2024-09-17T04:33:02.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.830+0000 7fa8f0c00640 1 --2- 172.21.15.100:0/2111640715 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa8d4079400 0x7fa8d407b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:02.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.830+0000 7fa8f0c00640 1 -- 172.21.15.100:0/2111640715 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5851+0+0 (secure 0 0 0) 0x7fa8ec099e00 con 0x7fa8f41e7ae0 2024-09-17T04:33:02.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.830+0000 7fa8f9200640 1 --2- 172.21.15.100:0/2111640715 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa8d4079400 0x7fa8d407b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:02.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.831+0000 7fa8f9200640 1 --2- 172.21.15.100:0/2111640715 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa8d4079400 0x7fa8d407b910 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8033000 tx=0x7fa8e802d000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:02.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:02.835+0000 7fa8f0c00640 1 -- 172.21.15.100:0/2111640715 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa8ec063d00 con 0x7fa8f41e7ae0 2024-09-17T04:33:03.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:03.020+0000 7fa8fbc00640 1 -- 172.21.15.100:0/2111640715 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa8f41567a0 con 0x7fa8f41e7ae0 2024-09-17T04:33:03.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:03.804+0000 7fa8f0c00640 1 -- 172.21.15.100:0/2111640715 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v147) ==== 181+0+0 (secure 0 0 0) 0x7fa8ec063600 con 0x7fa8f41e7ae0 2024-09-17T04:33:03.805 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' removed 2024-09-17T04:33:03.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:03.807+0000 7fa8fbc00640 1 -- 172.21.15.100:0/2111640715 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa8d4079400 msgr2=0x7fa8d407b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:03.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:03.807+0000 7fa8fbc00640 1 --2- 172.21.15.100:0/2111640715 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa8d4079400 0x7fa8d407b910 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8033000 tx=0x7fa8e802d000 comp rx=0 tx=0).stop 2024-09-17T04:33:03.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:03.807+0000 7fa8fbc00640 1 -- 172.21.15.100:0/2111640715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f41e7ae0 msgr2=0x7fa8f41e3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:03.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:03.807+0000 7fa8fbc00640 1 --2- 172.21.15.100:0/2111640715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f41e7ae0 0x7fa8f41e3cb0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec005fd0 tx=0x7fa8ec011f90 comp rx=0 tx=0).stop 2024-09-17T04:33:03.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:03.807+0000 7fa8fbc00640 1 -- 172.21.15.100:0/2111640715 shutdown_connections 2024-09-17T04:33:03.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:03.807+0000 7fa8fbc00640 1 --2- 172.21.15.100:0/2111640715 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa8d4079400 0x7fa8d407b910 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:03.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:03.807+0000 7fa8fbc00640 1 --2- 172.21.15.100:0/2111640715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f41e7ae0 0x7fa8f41e3cb0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:03.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:03.807+0000 7fa8fbc00640 1 --2- 172.21.15.100:0/2111640715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8f41e2460 0x7fa8f41e75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:03.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:03.807+0000 7fa8fbc00640 1 --2- 172.21.15.100:0/2111640715 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8dc003a00 0x7fa8f4156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:03.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:03.807+0000 7fa8fbc00640 1 -- 172.21.15.100:0/2111640715 >> 172.21.15.100:0/2111640715 conn(0x7fa8f40b2540 msgr2=0x7fa8f40ae9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:03.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:03.808+0000 7fa8fbc00640 1 -- 172.21.15.100:0/2111640715 shutdown_connections 2024-09-17T04:33:03.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:03.808+0000 7fa8fbc00640 1 -- 172.21.15.100:0/2111640715 wait complete. 2024-09-17T04:33:03.836 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T04:33:04.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.061+0000 7f4fd3200640 1 -- 172.21.15.100:0/7772388 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fcc0af100 msgr2=0x7f4fcc0af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:04.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.061+0000 7f4fd3200640 1 --2- 172.21.15.100:0/7772388 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fcc0af100 0x7f4fcc0af510 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4005fd0 tx=0x7f4fc402a7a0 comp rx=0 tx=0).stop 2024-09-17T04:33:04.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.061+0000 7f4fd3200640 1 -- 172.21.15.100:0/7772388 shutdown_connections 2024-09-17T04:33:04.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.061+0000 7f4fd3200640 1 --2- 172.21.15.100:0/7772388 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fcc0af100 0x7f4fcc0af510 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:04.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.061+0000 7f4fd3200640 1 --2- 172.21.15.100:0/7772388 >> v2:172.21.15.184:3300/0 conn(0x7f4fcc06d4b0 0x7f4fcc0b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:04.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.061+0000 7f4fd3200640 1 -- 172.21.15.100:0/7772388 >> 172.21.15.100:0/7772388 conn(0x7f4fcc0b2540 msgr2=0x7f4fcc0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:04.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.061+0000 7f4fd3200640 1 -- 172.21.15.100:0/7772388 shutdown_connections 2024-09-17T04:33:04.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.062+0000 7f4fd3200640 1 -- 172.21.15.100:0/7772388 wait complete. 2024-09-17T04:33:04.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.064+0000 7f4fd3200640 1 Processor -- start 2024-09-17T04:33:04.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.064+0000 7f4fd3200640 1 -- start start 2024-09-17T04:33:04.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.065+0000 7f4fd3200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fcc0af100 0x7f4fcc098d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:04.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.065+0000 7f4fd3200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fcc0cc830 0x7f4fcc1e6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:04.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.065+0000 7f4fd3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fcc15f780 0x7f4fcc1e6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:04.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.065+0000 7f4fd3200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc0b3700 con 0x7f4fcc15f780 2024-09-17T04:33:04.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.065+0000 7f4fd3200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc0b3580 con 0x7f4fcc0af100 2024-09-17T04:33:04.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.065+0000 7f4fd3200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4fcc0b3880 con 0x7f4fcc0cc830 2024-09-17T04:33:04.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.065+0000 7f4fd1200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fcc15f780 0x7f4fcc1e6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:04.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.066+0000 7f4fd1200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fcc15f780 0x7f4fcc1e6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56938/0 (socket says 172.21.15.100:56938) 2024-09-17T04:33:04.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.066+0000 7f4fd1200640 1 -- 172.21.15.100:0/2111748422 learned_addr learned my addr 172.21.15.100:0/2111748422 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:04.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.066+0000 7f4fcbe00640 1 --2- 172.21.15.100:0/2111748422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fcc0af100 0x7f4fcc098d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:04.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.066+0000 7f4fcb400640 1 --2- 172.21.15.100:0/2111748422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fcc0cc830 0x7f4fcc1e6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:04.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.066+0000 7f4fd1200640 1 -- 172.21.15.100:0/2111748422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fcc0cc830 msgr2=0x7f4fcc1e6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:04.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.066+0000 7f4fd1200640 1 --2- 172.21.15.100:0/2111748422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fcc0cc830 0x7f4fcc1e6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:04.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.066+0000 7f4fd1200640 1 -- 172.21.15.100:0/2111748422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fcc0af100 msgr2=0x7f4fcc098d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:04.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.066+0000 7f4fd1200640 1 --2- 172.21.15.100:0/2111748422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fcc0af100 0x7f4fcc098d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:04.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.066+0000 7f4fd1200640 1 -- 172.21.15.100:0/2111748422 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fcc0cf210 con 0x7f4fcc15f780 2024-09-17T04:33:04.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.066+0000 7f4fd1200640 1 --2- 172.21.15.100:0/2111748422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fcc15f780 0x7f4fcc1e6690 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4006010 tx=0x7f4fc4032e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:04.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.067+0000 7f4fc3e00640 1 -- 172.21.15.100:0/2111748422 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4041020 con 0x7f4fcc15f780 2024-09-17T04:33:04.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.067+0000 7f4fc3e00640 1 -- 172.21.15.100:0/2111748422 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4fc4004040 con 0x7f4fcc15f780 2024-09-17T04:33:04.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.067+0000 7f4fd3200640 1 -- 172.21.15.100:0/2111748422 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fcc0cf4f0 con 0x7f4fcc15f780 2024-09-17T04:33:04.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.067+0000 7f4fc3e00640 1 -- 172.21.15.100:0/2111748422 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4036040 con 0x7f4fcc15f780 2024-09-17T04:33:04.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.067+0000 7f4fd3200640 1 -- 172.21.15.100:0/2111748422 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fcc1e6bd0 con 0x7f4fcc15f780 2024-09-17T04:33:04.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.068+0000 7f4fd3200640 1 -- 172.21.15.100:0/2111748422 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fcc0b3da0 con 0x7f4fcc15f780 2024-09-17T04:33:04.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.069+0000 7f4fc3e00640 1 -- 172.21.15.100:0/2111748422 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f4fc4044020 con 0x7f4fcc15f780 2024-09-17T04:33:04.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.070+0000 7f4fc3e00640 1 --2- 172.21.15.100:0/2111748422 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4fac079400 0x7f4fac07b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:04.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.070+0000 7f4fc3e00640 1 -- 172.21.15.100:0/2111748422 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5413+0+0 (secure 0 0 0) 0x7f4fc40bcb80 con 0x7f4fcc15f780 2024-09-17T04:33:04.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.070+0000 7f4fcbe00640 1 --2- 172.21.15.100:0/2111748422 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4fac079400 0x7f4fac07b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:04.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.071+0000 7f4fcbe00640 1 --2- 172.21.15.100:0/2111748422 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4fac079400 0x7f4fac07b910 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc0018b0 tx=0x7f4fbc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:04.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.075+0000 7f4fc3e00640 1 -- 172.21.15.100:0/2111748422 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4fc404b070 con 0x7f4fcc15f780 2024-09-17T04:33:04.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.257+0000 7f4fd3200640 1 -- 172.21.15.100:0/2111748422 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4fcc1e6ea0 con 0x7f4fcc15f780 2024-09-17T04:33:04.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.807+0000 7f4fc3e00640 1 -- 172.21.15.100:0/2111748422 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v148) ==== 169+0+0 (secure 0 0 0) 0x7f4fc4083a60 con 0x7f4fcc15f780 2024-09-17T04:33:04.807 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' removed 2024-09-17T04:33:04.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.812+0000 7f4fd3200640 1 -- 172.21.15.100:0/2111748422 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4fac079400 msgr2=0x7f4fac07b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:04.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.812+0000 7f4fd3200640 1 --2- 172.21.15.100:0/2111748422 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4fac079400 0x7f4fac07b910 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc0018b0 tx=0x7f4fbc009040 comp rx=0 tx=0).stop 2024-09-17T04:33:04.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.812+0000 7f4fd3200640 1 -- 172.21.15.100:0/2111748422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fcc15f780 msgr2=0x7f4fcc1e6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:04.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.812+0000 7f4fd3200640 1 --2- 172.21.15.100:0/2111748422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fcc15f780 0x7f4fcc1e6690 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4006010 tx=0x7f4fc4032e90 comp rx=0 tx=0).stop 2024-09-17T04:33:04.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.813+0000 7f4fd3200640 1 -- 172.21.15.100:0/2111748422 shutdown_connections 2024-09-17T04:33:04.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.813+0000 7f4fd3200640 1 --2- 172.21.15.100:0/2111748422 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4fac079400 0x7f4fac07b910 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:04.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.813+0000 7f4fd3200640 1 --2- 172.21.15.100:0/2111748422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fcc15f780 0x7f4fcc1e6690 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:04.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.813+0000 7f4fd3200640 1 --2- 172.21.15.100:0/2111748422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fcc0cc830 0x7f4fcc1e6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:04.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.813+0000 7f4fd3200640 1 --2- 172.21.15.100:0/2111748422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fcc0af100 0x7f4fcc098d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:04.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.813+0000 7f4fd3200640 1 -- 172.21.15.100:0/2111748422 >> 172.21.15.100:0/2111748422 conn(0x7f4fcc0b2540 msgr2=0x7f4fcc1e7a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:04.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.814+0000 7f4fd3200640 1 -- 172.21.15.100:0/2111748422 shutdown_connections 2024-09-17T04:33:04.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:04.814+0000 7f4fd3200640 1 -- 172.21.15.100:0/2111748422 wait complete. 2024-09-17T04:33:04.841 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T04:33:05.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.064+0000 7f1696e00640 1 -- 172.21.15.100:0/978306606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16900cfc80 msgr2=0x7f1690156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:05.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.064+0000 7f1696e00640 1 --2- 172.21.15.100:0/978306606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16900cfc80 0x7f1690156e40 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f1680009900 tx=0x7f168002a580 comp rx=0 tx=0).stop 2024-09-17T04:33:05.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.064+0000 7f1696e00640 1 -- 172.21.15.100:0/978306606 shutdown_connections 2024-09-17T04:33:05.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.064+0000 7f1696e00640 1 --2- 172.21.15.100:0/978306606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16900cfc80 0x7f1690156e40 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:05.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.064+0000 7f1696e00640 1 -- 172.21.15.100:0/978306606 >> 172.21.15.100:0/978306606 conn(0x7f169006d3d0 msgr2=0x7f16900b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:05.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.064+0000 7f1696e00640 1 -- 172.21.15.100:0/978306606 shutdown_connections 2024-09-17T04:33:05.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.064+0000 7f1696e00640 1 -- 172.21.15.100:0/978306606 wait complete. 2024-09-17T04:33:05.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.067+0000 7f1696e00640 1 Processor -- start 2024-09-17T04:33:05.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.067+0000 7f1696e00640 1 -- start start 2024-09-17T04:33:05.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.068+0000 7f1696e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16900705b0 0x7f16901f5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:05.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.068+0000 7f1696e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16900cfc80 0x7f16900c5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:05.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.068+0000 7f1696e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f169015ce30 0x7f16900c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:05.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.068+0000 7f1696e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f16900cd100 con 0x7f16900705b0 2024-09-17T04:33:05.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.068+0000 7f1696e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f16900ccf80 con 0x7f16900cfc80 2024-09-17T04:33:05.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.068+0000 7f1696e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f16900cd280 con 0x7f169015ce30 2024-09-17T04:33:05.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.068+0000 7f168fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16900705b0 0x7f16901f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:05.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.068+0000 7f168fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16900705b0 0x7f16901f5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55314/0 (socket says 172.21.15.100:55314) 2024-09-17T04:33:05.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.068+0000 7f168fe00640 1 -- 172.21.15.100:0/3695709371 learned_addr learned my addr 172.21.15.100:0/3695709371 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:05.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.069+0000 7f1694e00640 1 --2- 172.21.15.100:0/3695709371 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f169015ce30 0x7f16900c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:05.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.069+0000 7f168f400640 1 --2- 172.21.15.100:0/3695709371 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16900cfc80 0x7f16900c5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:05.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.069+0000 7f168fe00640 1 -- 172.21.15.100:0/3695709371 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f169015ce30 msgr2=0x7f16900c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:05.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.069+0000 7f168fe00640 1 --2- 172.21.15.100:0/3695709371 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f169015ce30 0x7f16900c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:05.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.069+0000 7f168fe00640 1 -- 172.21.15.100:0/3695709371 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16900cfc80 msgr2=0x7f16900c5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:05.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.069+0000 7f168fe00640 1 --2- 172.21.15.100:0/3695709371 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16900cfc80 0x7f16900c5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:05.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.069+0000 7f168fe00640 1 -- 172.21.15.100:0/3695709371 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16901568c0 con 0x7f16900705b0 2024-09-17T04:33:05.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.069+0000 7f168fe00640 1 --2- 172.21.15.100:0/3695709371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16900705b0 0x7f16901f5530 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f1680009230 tx=0x7f16800039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:05.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.070+0000 7f1687e00640 1 -- 172.21.15.100:0/3695709371 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168003e030 con 0x7f16900705b0 2024-09-17T04:33:05.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.070+0000 7f1687e00640 1 -- 172.21.15.100:0/3695709371 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16800047a0 con 0x7f16900705b0 2024-09-17T04:33:05.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.070+0000 7f1696e00640 1 -- 172.21.15.100:0/3695709371 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16900c6380 con 0x7f16900705b0 2024-09-17T04:33:05.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.070+0000 7f1687e00640 1 -- 172.21.15.100:0/3695709371 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1680035040 con 0x7f16900705b0 2024-09-17T04:33:05.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.070+0000 7f1696e00640 1 -- 172.21.15.100:0/3695709371 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16900c6c20 con 0x7f16900705b0 2024-09-17T04:33:05.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.071+0000 7f1696e00640 1 -- 172.21.15.100:0/3695709371 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16900cd7d0 con 0x7f16900705b0 2024-09-17T04:33:05.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.072+0000 7f1687e00640 1 -- 172.21.15.100:0/3695709371 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f1680041030 con 0x7f16900705b0 2024-09-17T04:33:05.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.074+0000 7f1687e00640 1 --2- 172.21.15.100:0/3695709371 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1670079400 0x7f167007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:05.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.074+0000 7f1687e00640 1 -- 172.21.15.100:0/3695709371 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4999+0+0 (secure 0 0 0) 0x7f16800bc5a0 con 0x7f16900705b0 2024-09-17T04:33:05.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.074+0000 7f168f400640 1 --2- 172.21.15.100:0/3695709371 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1670079400 0x7f167007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:05.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.074+0000 7f168f400640 1 --2- 172.21.15.100:0/3695709371 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1670079400 0x7f167007b910 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1690069880 tx=0x7f167c009060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:05.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.078+0000 7f1687e00640 1 -- 172.21.15.100:0/3695709371 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f16800836b0 con 0x7f16900705b0 2024-09-17T04:33:05.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.261+0000 7f1696e00640 1 -- 172.21.15.100:0/3695709371 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f16900afa70 con 0x7f16900705b0 2024-09-17T04:33:05.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.265+0000 7f1687e00640 1 -- 172.21.15.100:0/3695709371 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v49) ==== 71+0+0 (secure 0 0 0) 0x7f16800880e0 con 0x7f16900705b0 2024-09-17T04:33:05.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.268+0000 7f1696e00640 1 -- 172.21.15.100:0/3695709371 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1670079400 msgr2=0x7f167007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:05.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.268+0000 7f1696e00640 1 --2- 172.21.15.100:0/3695709371 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1670079400 0x7f167007b910 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1690069880 tx=0x7f167c009060 comp rx=0 tx=0).stop 2024-09-17T04:33:05.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.268+0000 7f1696e00640 1 -- 172.21.15.100:0/3695709371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16900705b0 msgr2=0x7f16901f5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:05.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.268+0000 7f1696e00640 1 --2- 172.21.15.100:0/3695709371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16900705b0 0x7f16901f5530 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f1680009230 tx=0x7f16800039e0 comp rx=0 tx=0).stop 2024-09-17T04:33:05.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.268+0000 7f1696e00640 1 -- 172.21.15.100:0/3695709371 shutdown_connections 2024-09-17T04:33:05.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.268+0000 7f1696e00640 1 --2- 172.21.15.100:0/3695709371 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1670079400 0x7f167007b910 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:05.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.268+0000 7f1696e00640 1 --2- 172.21.15.100:0/3695709371 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f169015ce30 0x7f16900c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:05.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.268+0000 7f1696e00640 1 --2- 172.21.15.100:0/3695709371 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16900cfc80 0x7f16900c5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:05.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.268+0000 7f1696e00640 1 --2- 172.21.15.100:0/3695709371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16900705b0 0x7f16901f5530 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:05.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.268+0000 7f1696e00640 1 -- 172.21.15.100:0/3695709371 >> 172.21.15.100:0/3695709371 conn(0x7f169006d3d0 msgr2=0x7f169015e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:05.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.268+0000 7f1696e00640 1 -- 172.21.15.100:0/3695709371 shutdown_connections 2024-09-17T04:33:05.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.268+0000 7f1696e00640 1 -- 172.21.15.100:0/3695709371 wait complete. 2024-09-17T04:33:05.289 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1' 2024-09-17T04:33:05.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.513+0000 7f2026c00640 1 -- 172.21.15.100:0/3688869632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20200ad9c0 msgr2=0x7f20200af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:05.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.513+0000 7f2026c00640 1 --2- 172.21.15.100:0/3688869632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20200ad9c0 0x7f20200af170 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f2014009830 tx=0x7f201402a480 comp rx=0 tx=0).stop 2024-09-17T04:33:05.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.513+0000 7f2026c00640 1 -- 172.21.15.100:0/3688869632 shutdown_connections 2024-09-17T04:33:05.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.513+0000 7f2026c00640 1 --2- 172.21.15.100:0/3688869632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20200ad9c0 0x7f20200af170 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:05.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.513+0000 7f2026c00640 1 -- 172.21.15.100:0/3688869632 >> 172.21.15.100:0/3688869632 conn(0x7f20200b1e40 msgr2=0x7f20200b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:05.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.513+0000 7f2026c00640 1 -- 172.21.15.100:0/3688869632 shutdown_connections 2024-09-17T04:33:05.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.513+0000 7f2026c00640 1 -- 172.21.15.100:0/3688869632 wait complete. 2024-09-17T04:33:05.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.515+0000 7f2026c00640 1 Processor -- start 2024-09-17T04:33:05.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.516+0000 7f2026c00640 1 -- start start 2024-09-17T04:33:05.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.517+0000 7f2026c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20200ad9c0 0x7f202009b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:05.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.517+0000 7f2026c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20200b2f60 0x7f20201f3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:05.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.517+0000 7f2026c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f202015a0f0 0x7f202014fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:05.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.517+0000 7f2026c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f202006fb20 con 0x7f20200ad9c0 2024-09-17T04:33:05.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.517+0000 7f2026c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f202006f9a0 con 0x7f202015a0f0 2024-09-17T04:33:05.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.517+0000 7f2026c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f202015d670 con 0x7f20200b2f60 2024-09-17T04:33:05.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.517+0000 7f2025800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20200ad9c0 0x7f202009b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:05.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.517+0000 7f2025800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20200ad9c0 0x7f202009b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55338/0 (socket says 172.21.15.100:55338) 2024-09-17T04:33:05.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.517+0000 7f2025800640 1 -- 172.21.15.100:0/1193021626 learned_addr learned my addr 172.21.15.100:0/1193021626 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:05.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.518+0000 7f2026200640 1 --2- 172.21.15.100:0/1193021626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f202015a0f0 0x7f202014fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:05.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.518+0000 7f2024e00640 1 --2- 172.21.15.100:0/1193021626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20200b2f60 0x7f20201f3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:05.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.518+0000 7f2025800640 1 -- 172.21.15.100:0/1193021626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20200b2f60 msgr2=0x7f20201f3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:05.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.518+0000 7f2025800640 1 --2- 172.21.15.100:0/1193021626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20200b2f60 0x7f20201f3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:05.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.518+0000 7f2025800640 1 -- 172.21.15.100:0/1193021626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f202015a0f0 msgr2=0x7f202014fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:05.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.518+0000 7f2025800640 1 --2- 172.21.15.100:0/1193021626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f202015a0f0 0x7f202014fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:05.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.518+0000 7f2025800640 1 -- 172.21.15.100:0/1193021626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20201f3ff0 con 0x7f20200ad9c0 2024-09-17T04:33:05.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.518+0000 7f2024e00640 1 --2- 172.21.15.100:0/1193021626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20200b2f60 0x7f20201f3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:05.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.518+0000 7f2025800640 1 --2- 172.21.15.100:0/1193021626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20200ad9c0 0x7f202009b5d0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f201402aa70 tx=0x7f2014033ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:05.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.518+0000 7f201d600640 1 -- 172.21.15.100:0/1193021626 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2014031d60 con 0x7f20200ad9c0 2024-09-17T04:33:05.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.518+0000 7f201d600640 1 -- 172.21.15.100:0/1193021626 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2014005020 con 0x7f20200ad9c0 2024-09-17T04:33:05.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.518+0000 7f2026c00640 1 -- 172.21.15.100:0/1193021626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20201502d0 con 0x7f20200ad9c0 2024-09-17T04:33:05.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.519+0000 7f201d600640 1 -- 172.21.15.100:0/1193021626 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2014005470 con 0x7f20200ad9c0 2024-09-17T04:33:05.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.519+0000 7f2026c00640 1 -- 172.21.15.100:0/1193021626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20200c5d00 con 0x7f20200ad9c0 2024-09-17T04:33:05.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.520+0000 7f2026c00640 1 -- 172.21.15.100:0/1193021626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20200af170 con 0x7f20200ad9c0 2024-09-17T04:33:05.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.521+0000 7f201d600640 1 -- 172.21.15.100:0/1193021626 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f20140047a0 con 0x7f20200ad9c0 2024-09-17T04:33:05.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.522+0000 7f201d600640 1 --2- 172.21.15.100:0/1193021626 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2000079400 0x7f200007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:05.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.522+0000 7f201d600640 1 -- 172.21.15.100:0/1193021626 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4999+0+0 (secure 0 0 0) 0x7f20140ba790 con 0x7f20200ad9c0 2024-09-17T04:33:05.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.522+0000 7f2024e00640 1 --2- 172.21.15.100:0/1193021626 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2000079400 0x7f200007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:05.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.523+0000 7f2024e00640 1 --2- 172.21.15.100:0/1193021626 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2000079400 0x7f200007b910 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f200c006fd0 tx=0x7f200c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:05.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.527+0000 7f201d600640 1 -- 172.21.15.100:0/1193021626 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f20201502d0 con 0x7f20200ad9c0 2024-09-17T04:33:05.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.777+0000 7f2026c00640 1 -- 172.21.15.100:0/1193021626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]} v 0) -- 0x7f20200c5fe0 con 0x7f20200ad9c0 2024-09-17T04:33:05.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.816+0000 7f201d600640 1 -- 172.21.15.100:0/1193021626 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]}]=0 v501) ==== 149+0+0 (secure 0 0 0) 0x7f20201f3ff0 con 0x7f20200ad9c0 2024-09-17T04:33:05.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.819+0000 7f2026c00640 1 -- 172.21.15.100:0/1193021626 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2000079400 msgr2=0x7f200007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:05.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.819+0000 7f2026c00640 1 --2- 172.21.15.100:0/1193021626 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2000079400 0x7f200007b910 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f200c006fd0 tx=0x7f200c008040 comp rx=0 tx=0).stop 2024-09-17T04:33:05.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.819+0000 7f2026c00640 1 -- 172.21.15.100:0/1193021626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20200ad9c0 msgr2=0x7f202009b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:05.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.819+0000 7f2026c00640 1 --2- 172.21.15.100:0/1193021626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20200ad9c0 0x7f202009b5d0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f201402aa70 tx=0x7f2014033ce0 comp rx=0 tx=0).stop 2024-09-17T04:33:05.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.819+0000 7f2026c00640 1 -- 172.21.15.100:0/1193021626 shutdown_connections 2024-09-17T04:33:05.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.819+0000 7f2026c00640 1 --2- 172.21.15.100:0/1193021626 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2000079400 0x7f200007b910 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:05.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.819+0000 7f2026c00640 1 --2- 172.21.15.100:0/1193021626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f202015a0f0 0x7f202014fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:05.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.819+0000 7f2026c00640 1 --2- 172.21.15.100:0/1193021626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20200b2f60 0x7f20201f3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:05.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.819+0000 7f2026c00640 1 --2- 172.21.15.100:0/1193021626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20200ad9c0 0x7f202009b5d0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:05.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.819+0000 7f2026c00640 1 -- 172.21.15.100:0/1193021626 >> 172.21.15.100:0/1193021626 conn(0x7f20200b1e40 msgr2=0x7f20200cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:05.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.819+0000 7f2026c00640 1 -- 172.21.15.100:0/1193021626 shutdown_connections 2024-09-17T04:33:05.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:05.819+0000 7f2026c00640 1 -- 172.21.15.100:0/1193021626 wait complete. 2024-09-17T04:33:05.843 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- rather than going through a full upgrade cycle, ... ok 2024-09-17T04:33:05.843 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-09-17T04:33:05.843 INFO:tasks.cephfs_test_runner:test_subvolume_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) 2024-09-17T04:33:05.843 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:33:06.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.064+0000 7f1a4d000640 1 -- 172.21.15.100:0/1179190992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a4815d760 msgr2=0x7f1a481535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:06.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.064+0000 7f1a4d000640 1 --2- 172.21.15.100:0/1179190992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a4815d760 0x7f1a481535c0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f1a48072d60 tx=0x7f1a3802a4d0 comp rx=0 tx=0).stop 2024-09-17T04:33:06.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.064+0000 7f1a4d000640 1 -- 172.21.15.100:0/1179190992 shutdown_connections 2024-09-17T04:33:06.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.064+0000 7f1a4d000640 1 --2- 172.21.15.100:0/1179190992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a48160310 0x7f1a48160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:06.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.064+0000 7f1a4d000640 1 --2- 172.21.15.100:0/1179190992 >> v2:172.21.15.184:3300/0 conn(0x7f1a480ad0e0 0x7f1a480ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:06.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.064+0000 7f1a4d000640 1 --2- 172.21.15.100:0/1179190992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a4815d760 0x7f1a481535c0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:06.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.064+0000 7f1a4d000640 1 -- 172.21.15.100:0/1179190992 >> 172.21.15.100:0/1179190992 conn(0x7f1a4806d730 msgr2=0x7f1a480b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:06.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.064+0000 7f1a4d000640 1 -- 172.21.15.100:0/1179190992 shutdown_connections 2024-09-17T04:33:06.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.064+0000 7f1a4d000640 1 -- 172.21.15.100:0/1179190992 wait complete. 2024-09-17T04:33:06.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.066+0000 7f1a4d000640 1 Processor -- start 2024-09-17T04:33:06.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.067+0000 7f1a4d000640 1 -- start start 2024-09-17T04:33:06.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.068+0000 7f1a4d000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a480ad0e0 0x7f1a48150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:06.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.068+0000 7f1a4d000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a4815d760 0x7f1a481f5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:06.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.068+0000 7f1a4d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a48160310 0x7f1a4809d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:06.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.068+0000 7f1a4d000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1a480b4da0 con 0x7f1a48160310 2024-09-17T04:33:06.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.068+0000 7f1a4d000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1a480b4c20 con 0x7f1a480ad0e0 2024-09-17T04:33:06.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.068+0000 7f1a4d000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1a48160c60 con 0x7f1a4815d760 2024-09-17T04:33:06.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.068+0000 7f1a46800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a48160310 0x7f1a4809d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:06.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.068+0000 7f1a46800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a48160310 0x7f1a4809d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55358/0 (socket says 172.21.15.100:55358) 2024-09-17T04:33:06.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.068+0000 7f1a46800640 1 -- 172.21.15.100:0/1849340639 learned_addr learned my addr 172.21.15.100:0/1849340639 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:06.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.068+0000 7f1a45400640 1 --2- 172.21.15.100:0/1849340639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a4815d760 0x7f1a481f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:06.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.068+0000 7f1a45e00640 1 --2- 172.21.15.100:0/1849340639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a480ad0e0 0x7f1a48150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:06.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.069+0000 7f1a46800640 1 -- 172.21.15.100:0/1849340639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a4815d760 msgr2=0x7f1a481f5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:06.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.069+0000 7f1a46800640 1 --2- 172.21.15.100:0/1849340639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a4815d760 0x7f1a481f5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.069+0000 7f1a46800640 1 -- 172.21.15.100:0/1849340639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a480ad0e0 msgr2=0x7f1a48150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.069+0000 7f1a46800640 1 --2- 172.21.15.100:0/1849340639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a480ad0e0 0x7f1a48150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.069+0000 7f1a46800640 1 -- 172.21.15.100:0/1849340639 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4809a2d0 con 0x7f1a48160310 2024-09-17T04:33:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.069+0000 7f1a46800640 1 --2- 172.21.15.100:0/1849340639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a48160310 0x7f1a4809d710 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000da90 tx=0x7f1a4000df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.069+0000 7f1a3e000640 1 -- 172.21.15.100:0/1849340639 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40017070 con 0x7f1a48160310 2024-09-17T04:33:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.069+0000 7f1a3e000640 1 -- 172.21.15.100:0/1849340639 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a40003970 con 0x7f1a48160310 2024-09-17T04:33:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.069+0000 7f1a4d000640 1 -- 172.21.15.100:0/1849340639 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a48155790 con 0x7f1a48160310 2024-09-17T04:33:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.069+0000 7f1a3e000640 1 -- 172.21.15.100:0/1849340639 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40003dc0 con 0x7f1a48160310 2024-09-17T04:33:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.070+0000 7f1a4d000640 1 -- 172.21.15.100:0/1849340639 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a481f6660 con 0x7f1a48160310 2024-09-17T04:33:06.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.070+0000 7f1a4d000640 1 -- 172.21.15.100:0/1849340639 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a481f60f0 con 0x7f1a48160310 2024-09-17T04:33:06.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.072+0000 7f1a3e000640 1 -- 172.21.15.100:0/1849340639 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f1a40003f60 con 0x7f1a48160310 2024-09-17T04:33:06.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.073+0000 7f1a3e000640 1 --2- 172.21.15.100:0/1849340639 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1a28079430 0x7f1a2807b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:06.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.073+0000 7f1a3e000640 1 -- 172.21.15.100:0/1849340639 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4999+0+0 (secure 0 0 0) 0x7f1a4009c060 con 0x7f1a48160310 2024-09-17T04:33:06.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.073+0000 7f1a45e00640 1 --2- 172.21.15.100:0/1849340639 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1a28079430 0x7f1a2807b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:06.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.074+0000 7f1a45e00640 1 --2- 172.21.15.100:0/1849340639 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1a28079430 0x7f1a2807b940 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1a38033000 tx=0x7f1a38004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:06.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.077+0000 7f1a3e000640 1 -- 172.21.15.100:0/1849340639 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f1a40063170 con 0x7f1a48160310 2024-09-17T04:33:06.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.260+0000 7f1a4d000640 1 -- 172.21.15.100:0/1849340639 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1a481535c0 con 0x7f1a48160310 2024-09-17T04:33:06.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.261+0000 7f1a3e000640 1 -- 172.21.15.100:0/1849340639 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) ==== 74+0+13181 (secure 0 0 0) 0x7f1a40067ba0 con 0x7f1a48160310 2024-09-17T04:33:06.262 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:06.262 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":148,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:33:04.805903+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6835/2425159512":"2024-09-18T04:33:01.556254+0000","172.21.15.100:6834/2425159512":"2024-09-18T04:33:01.556254+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:33:06.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.263+0000 7f1a4d000640 1 -- 172.21.15.100:0/1849340639 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1a28079430 msgr2=0x7f1a2807b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:06.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.263+0000 7f1a4d000640 1 --2- 172.21.15.100:0/1849340639 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1a28079430 0x7f1a2807b940 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1a38033000 tx=0x7f1a38004730 comp rx=0 tx=0).stop 2024-09-17T04:33:06.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.263+0000 7f1a4d000640 1 -- 172.21.15.100:0/1849340639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a48160310 msgr2=0x7f1a4809d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:06.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.263+0000 7f1a4d000640 1 --2- 172.21.15.100:0/1849340639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a48160310 0x7f1a4809d710 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000da90 tx=0x7f1a4000df50 comp rx=0 tx=0).stop 2024-09-17T04:33:06.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.264+0000 7f1a4d000640 1 -- 172.21.15.100:0/1849340639 shutdown_connections 2024-09-17T04:33:06.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.264+0000 7f1a4d000640 1 --2- 172.21.15.100:0/1849340639 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1a28079430 0x7f1a2807b940 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:06.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.264+0000 7f1a4d000640 1 --2- 172.21.15.100:0/1849340639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a48160310 0x7f1a4809d710 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:06.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.264+0000 7f1a4d000640 1 --2- 172.21.15.100:0/1849340639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a4815d760 0x7f1a481f5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:06.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.264+0000 7f1a4d000640 1 --2- 172.21.15.100:0/1849340639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a480ad0e0 0x7f1a48150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:06.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.264+0000 7f1a4d000640 1 -- 172.21.15.100:0/1849340639 >> 172.21.15.100:0/1849340639 conn(0x7f1a4806d730 msgr2=0x7f1a4815e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:06.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.264+0000 7f1a4d000640 1 -- 172.21.15.100:0/1849340639 shutdown_connections 2024-09-17T04:33:06.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.264+0000 7f1a4d000640 1 -- 172.21.15.100:0/1849340639 wait complete. 2024-09-17T04:33:06.290 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T04:24:03.995355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T04:33:06.291 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T04:33:06.338 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:33:06.342 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:33:06.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.516+0000 7f0354400640 1 -- 172.21.15.100:0/3443219685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f034c0ad3d0 msgr2=0x7f034c0ad7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:06.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.516+0000 7f0354400640 1 --2- 172.21.15.100:0/3443219685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f034c0ad3d0 0x7f034c0ad7e0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f0344005fd0 tx=0x7f034402a8a0 comp rx=0 tx=0).stop 2024-09-17T04:33:06.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.516+0000 7f0354400640 1 -- 172.21.15.100:0/3443219685 shutdown_connections 2024-09-17T04:33:06.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.516+0000 7f0354400640 1 --2- 172.21.15.100:0/3443219685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f034c0ad3d0 0x7f034c0ad7e0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:06.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.516+0000 7f0354400640 1 --2- 172.21.15.100:0/3443219685 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f034c0b4f00 0x7f034c0704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:06.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.516+0000 7f0354400640 1 -- 172.21.15.100:0/3443219685 >> 172.21.15.100:0/3443219685 conn(0x7f034c06d710 msgr2=0x7f034c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:06.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.516+0000 7f0354400640 1 -- 172.21.15.100:0/3443219685 shutdown_connections 2024-09-17T04:33:06.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.516+0000 7f0354400640 1 -- 172.21.15.100:0/3443219685 wait complete. 2024-09-17T04:33:06.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.518+0000 7f0354400640 1 Processor -- start 2024-09-17T04:33:06.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.518+0000 7f0354400640 1 -- start start 2024-09-17T04:33:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.520+0000 7f0354400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f034c0ad3d0 0x7f034c09d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.520+0000 7f0354400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f034c0b4f00 0x7f034c09a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.520+0000 7f0354400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f034c160490 0x7f034c09a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.520+0000 7f0354400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f034c06f0f0 con 0x7f034c0ad3d0 2024-09-17T04:33:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.520+0000 7f0354400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f034c06ef70 con 0x7f034c160490 2024-09-17T04:33:06.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.520+0000 7f0354400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f034c06f270 con 0x7f034c0b4f00 2024-09-17T04:33:06.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.520+0000 7f0351a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f034c0ad3d0 0x7f034c09d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:06.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.520+0000 7f0351a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f034c0ad3d0 0x7f034c09d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55382/0 (socket says 172.21.15.100:55382) 2024-09-17T04:33:06.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.520+0000 7f0351a00640 1 -- 172.21.15.100:0/2541479394 learned_addr learned my addr 172.21.15.100:0/2541479394 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:06.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.520+0000 7f0351000640 1 --2- 172.21.15.100:0/2541479394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f034c0b4f00 0x7f034c09a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:06.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.520+0000 7f0352400640 1 --2- 172.21.15.100:0/2541479394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f034c160490 0x7f034c09a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:06.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.520+0000 7f0351a00640 1 -- 172.21.15.100:0/2541479394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f034c0b4f00 msgr2=0x7f034c09a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:06.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.520+0000 7f0351a00640 1 --2- 172.21.15.100:0/2541479394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f034c0b4f00 0x7f034c09a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:06.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.520+0000 7f0351a00640 1 -- 172.21.15.100:0/2541479394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f034c160490 msgr2=0x7f034c09a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:06.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.520+0000 7f0351a00640 1 --2- 172.21.15.100:0/2541479394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f034c160490 0x7f034c09a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:06.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.520+0000 7f0351a00640 1 -- 172.21.15.100:0/2541479394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034c14ca60 con 0x7f034c0ad3d0 2024-09-17T04:33:06.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.520+0000 7f0352400640 1 --2- 172.21.15.100:0/2541479394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f034c160490 0x7f034c09a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:06.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.521+0000 7f0351a00640 1 --2- 172.21.15.100:0/2541479394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f034c0ad3d0 0x7f034c09d880 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f034000d740 tx=0x7f034000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:06.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.521+0000 7f0349600640 1 -- 172.21.15.100:0/2541479394 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340017070 con 0x7f034c0ad3d0 2024-09-17T04:33:06.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.521+0000 7f0349600640 1 -- 172.21.15.100:0/2541479394 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0340009d20 con 0x7f034c0ad3d0 2024-09-17T04:33:06.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.521+0000 7f0354400640 1 -- 172.21.15.100:0/2541479394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f034c152ba0 con 0x7f034c0ad3d0 2024-09-17T04:33:06.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.521+0000 7f0349600640 1 -- 172.21.15.100:0/2541479394 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03400106c0 con 0x7f034c0ad3d0 2024-09-17T04:33:06.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.521+0000 7f0354400640 1 -- 172.21.15.100:0/2541479394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f034c09aea0 con 0x7f034c0ad3d0 2024-09-17T04:33:06.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.522+0000 7f0354400640 1 -- 172.21.15.100:0/2541479394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f034c06f760 con 0x7f034c0ad3d0 2024-09-17T04:33:06.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.523+0000 7f0349600640 1 -- 172.21.15.100:0/2541479394 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f034001a070 con 0x7f034c0ad3d0 2024-09-17T04:33:06.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.524+0000 7f0349600640 1 --2- 172.21.15.100:0/2541479394 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f032c079430 0x7f032c07b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:06.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.524+0000 7f0349600640 1 -- 172.21.15.100:0/2541479394 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4999+0+0 (secure 0 0 0) 0x7f034009bdb0 con 0x7f034c0ad3d0 2024-09-17T04:33:06.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.525+0000 7f0351000640 1 --2- 172.21.15.100:0/2541479394 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f032c079430 0x7f032c07b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:06.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.525+0000 7f0351000640 1 --2- 172.21.15.100:0/2541479394 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f032c079430 0x7f032c07b940 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f034c0b2430 tx=0x7f0338007e90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:06.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.529+0000 7f0349600640 1 -- 172.21.15.100:0/2541479394 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0340062ec0 con 0x7f034c0ad3d0 2024-09-17T04:33:06.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.712+0000 7f0354400640 1 -- 172.21.15.100:0/2541479394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f034c0cc560 con 0x7f034c0ad3d0 2024-09-17T04:33:06.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.712+0000 7f0349600640 1 -- 172.21.15.100:0/2541479394 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v148) ==== 93+0+10 (secure 0 0 0) 0x7f03400678f0 con 0x7f034c0ad3d0 2024-09-17T04:33:06.713 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-09-17T04:33:06.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.714+0000 7f0354400640 1 -- 172.21.15.100:0/2541479394 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f032c079430 msgr2=0x7f032c07b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:06.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.714+0000 7f0354400640 1 --2- 172.21.15.100:0/2541479394 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f032c079430 0x7f032c07b940 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f034c0b2430 tx=0x7f0338007e90 comp rx=0 tx=0).stop 2024-09-17T04:33:06.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.714+0000 7f0354400640 1 -- 172.21.15.100:0/2541479394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f034c0ad3d0 msgr2=0x7f034c09d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:06.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.714+0000 7f0354400640 1 --2- 172.21.15.100:0/2541479394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f034c0ad3d0 0x7f034c09d880 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f034000d740 tx=0x7f034000dc00 comp rx=0 tx=0).stop 2024-09-17T04:33:06.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.715+0000 7f0354400640 1 -- 172.21.15.100:0/2541479394 shutdown_connections 2024-09-17T04:33:06.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.715+0000 7f0354400640 1 --2- 172.21.15.100:0/2541479394 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f032c079430 0x7f032c07b940 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:06.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.715+0000 7f0354400640 1 --2- 172.21.15.100:0/2541479394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f034c160490 0x7f034c09a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:06.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.715+0000 7f0354400640 1 --2- 172.21.15.100:0/2541479394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f034c0b4f00 0x7f034c09a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:06.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.715+0000 7f0354400640 1 --2- 172.21.15.100:0/2541479394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f034c0ad3d0 0x7f034c09d880 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:06.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.715+0000 7f0354400640 1 -- 172.21.15.100:0/2541479394 >> 172.21.15.100:0/2541479394 conn(0x7f034c06d710 msgr2=0x7f034c1566d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:06.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.715+0000 7f0354400640 1 -- 172.21.15.100:0/2541479394 shutdown_connections 2024-09-17T04:33:06.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.715+0000 7f0354400640 1 -- 172.21.15.100:0/2541479394 wait complete. 2024-09-17T04:33:06.739 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2' 2024-09-17T04:33:06.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.960+0000 7f4f4e800640 1 -- 172.21.15.100:0/4007506995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f4815a0f0 msgr2=0x7f4f4815d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:06.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.960+0000 7f4f4e800640 1 --2- 172.21.15.100:0/4007506995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f4815a0f0 0x7f4f4815d670 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f4f48051590 tx=0x7f4f4002a820 comp rx=0 tx=0).stop 2024-09-17T04:33:06.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.960+0000 7f4f4e800640 1 -- 172.21.15.100:0/4007506995 shutdown_connections 2024-09-17T04:33:06.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.960+0000 7f4f4e800640 1 --2- 172.21.15.100:0/4007506995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f4815a0f0 0x7f4f4815d670 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:06.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.960+0000 7f4f4e800640 1 --2- 172.21.15.100:0/4007506995 >> v2:172.21.15.184:3300/0 conn(0x7f4f480ad9c0 0x7f4f480af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:06.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.960+0000 7f4f4e800640 1 -- 172.21.15.100:0/4007506995 >> 172.21.15.100:0/4007506995 conn(0x7f4f480b1e40 msgr2=0x7f4f480b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:06.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.960+0000 7f4f4e800640 1 -- 172.21.15.100:0/4007506995 shutdown_connections 2024-09-17T04:33:06.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.960+0000 7f4f4e800640 1 -- 172.21.15.100:0/4007506995 wait complete. 2024-09-17T04:33:06.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.962+0000 7f4f4e800640 1 Processor -- start 2024-09-17T04:33:06.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.962+0000 7f4f4e800640 1 -- start start 2024-09-17T04:33:06.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.964+0000 7f4f4e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f480ad9c0 0x7f4f481f50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:06.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.964+0000 7f4f4e800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f480b2f60 0x7f4f480cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:06.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.964+0000 7f4f4e800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f4815a0f0 0x7f4f48073790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:06.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.964+0000 7f4f4e800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4806f290 con 0x7f4f480ad9c0 2024-09-17T04:33:06.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.964+0000 7f4f4e800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4815a5b0 con 0x7f4f4815a0f0 2024-09-17T04:33:06.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.964+0000 7f4f4e800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4f4806f410 con 0x7f4f480b2f60 2024-09-17T04:33:06.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.964+0000 7f4f4d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f480ad9c0 0x7f4f481f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:06.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.964+0000 7f4f4d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f480ad9c0 0x7f4f481f50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55400/0 (socket says 172.21.15.100:55400) 2024-09-17T04:33:06.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.964+0000 7f4f4d400640 1 -- 172.21.15.100:0/892300334 learned_addr learned my addr 172.21.15.100:0/892300334 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:06.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.964+0000 7f4f4ca00640 1 --2- 172.21.15.100:0/892300334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f480b2f60 0x7f4f480cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:06.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.964+0000 7f4f4de00640 1 --2- 172.21.15.100:0/892300334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f4815a0f0 0x7f4f48073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:06.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.964+0000 7f4f4d400640 1 -- 172.21.15.100:0/892300334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f480b2f60 msgr2=0x7f4f480cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:06.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.964+0000 7f4f4d400640 1 --2- 172.21.15.100:0/892300334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f480b2f60 0x7f4f480cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:06.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.964+0000 7f4f4d400640 1 -- 172.21.15.100:0/892300334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f4815a0f0 msgr2=0x7f4f48073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:06.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.964+0000 7f4f4d400640 1 --2- 172.21.15.100:0/892300334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f4815a0f0 0x7f4f48073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:06.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.964+0000 7f4f4d400640 1 -- 172.21.15.100:0/892300334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f481f6000 con 0x7f4f480ad9c0 2024-09-17T04:33:06.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.964+0000 7f4f4de00640 1 --2- 172.21.15.100:0/892300334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f4815a0f0 0x7f4f48073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:06.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.965+0000 7f4f4d400640 1 --2- 172.21.15.100:0/892300334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f480ad9c0 0x7f4f481f50d0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c00d770 tx=0x7f4f3c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:06.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.965+0000 7f4f45600640 1 -- 172.21.15.100:0/892300334 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c017070 con 0x7f4f480ad9c0 2024-09-17T04:33:06.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.965+0000 7f4f45600640 1 -- 172.21.15.100:0/892300334 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f3c009cc0 con 0x7f4f480ad9c0 2024-09-17T04:33:06.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.965+0000 7f4f45600640 1 -- 172.21.15.100:0/892300334 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c0106c0 con 0x7f4f480ad9c0 2024-09-17T04:33:06.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.965+0000 7f4f4e800640 1 -- 172.21.15.100:0/892300334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f481f5a40 con 0x7f4f480ad9c0 2024-09-17T04:33:06.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.965+0000 7f4f4e800640 1 -- 172.21.15.100:0/892300334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f48072f90 con 0x7f4f480ad9c0 2024-09-17T04:33:06.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.966+0000 7f4f4e800640 1 -- 172.21.15.100:0/892300334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f480b4c80 con 0x7f4f480ad9c0 2024-09-17T04:33:06.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.968+0000 7f4f45600640 1 -- 172.21.15.100:0/892300334 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f4f3c01a070 con 0x7f4f480ad9c0 2024-09-17T04:33:06.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.969+0000 7f4f45600640 1 --2- 172.21.15.100:0/892300334 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4f28079430 0x7f4f2807b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:06.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.969+0000 7f4f45600640 1 -- 172.21.15.100:0/892300334 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4999+0+0 (secure 0 0 0) 0x7f4f3c09bdd0 con 0x7f4f480ad9c0 2024-09-17T04:33:06.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.969+0000 7f4f4ca00640 1 --2- 172.21.15.100:0/892300334 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4f28079430 0x7f4f2807b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:06.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.970+0000 7f4f4ca00640 1 --2- 172.21.15.100:0/892300334 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4f28079430 0x7f4f2807b940 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4f480ca530 tx=0x7f4f34009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:06.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:06.974+0000 7f4f45600640 1 -- 172.21.15.100:0/892300334 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4f480b4c80 con 0x7f4f480ad9c0 2024-09-17T04:33:07.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:07.226+0000 7f4f4e800640 1 -- 172.21.15.100:0/892300334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]} v 0) -- 0x7f4f4806f8e0 con 0x7f4f480ad9c0 2024-09-17T04:33:07.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:07.825+0000 7f4f45600640 1 -- 172.21.15.100:0/892300334 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]}]=0 v503) ==== 148+0+0 (secure 0 0 0) 0x7f4f481f6000 con 0x7f4f480ad9c0 2024-09-17T04:33:07.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:07.827+0000 7f4f4e800640 1 -- 172.21.15.100:0/892300334 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4f28079430 msgr2=0x7f4f2807b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:07.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:07.827+0000 7f4f4e800640 1 --2- 172.21.15.100:0/892300334 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4f28079430 0x7f4f2807b940 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4f480ca530 tx=0x7f4f34009040 comp rx=0 tx=0).stop 2024-09-17T04:33:07.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:07.827+0000 7f4f4e800640 1 -- 172.21.15.100:0/892300334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f480ad9c0 msgr2=0x7f4f481f50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:07.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:07.827+0000 7f4f4e800640 1 --2- 172.21.15.100:0/892300334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f480ad9c0 0x7f4f481f50d0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c00d770 tx=0x7f4f3c00dc30 comp rx=0 tx=0).stop 2024-09-17T04:33:07.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:07.827+0000 7f4f4e800640 1 -- 172.21.15.100:0/892300334 shutdown_connections 2024-09-17T04:33:07.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:07.827+0000 7f4f4e800640 1 --2- 172.21.15.100:0/892300334 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4f28079430 0x7f4f2807b940 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:07.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:07.827+0000 7f4f4e800640 1 --2- 172.21.15.100:0/892300334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f4815a0f0 0x7f4f48073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:07.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:07.827+0000 7f4f4e800640 1 --2- 172.21.15.100:0/892300334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f480b2f60 0x7f4f480cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:07.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:07.827+0000 7f4f4e800640 1 --2- 172.21.15.100:0/892300334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f480ad9c0 0x7f4f481f50d0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:07.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:07.827+0000 7f4f4e800640 1 -- 172.21.15.100:0/892300334 >> 172.21.15.100:0/892300334 conn(0x7f4f480b1e40 msgr2=0x7f4f480afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:07.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:07.827+0000 7f4f4e800640 1 -- 172.21.15.100:0/892300334 shutdown_connections 2024-09-17T04:33:07.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:07.827+0000 7f4f4e800640 1 -- 172.21.15.100:0/892300334 wait complete. 2024-09-17T04:33:07.851 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T04:33:08.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.072+0000 7fa968200640 1 -- 172.21.15.100:0/2159528980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9600704b0 msgr2=0x7fa960073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:08.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.072+0000 7fa968200640 1 --2- 172.21.15.100:0/2159528980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9600704b0 0x7fa960073040 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fa9600afd40 tx=0x7fa95402a4d0 comp rx=0 tx=0).stop 2024-09-17T04:33:08.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.072+0000 7fa968200640 1 -- 172.21.15.100:0/2159528980 shutdown_connections 2024-09-17T04:33:08.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.072+0000 7fa968200640 1 --2- 172.21.15.100:0/2159528980 >> v2:172.21.15.184:3300/0 conn(0x7fa9600add10 0x7fa9600b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:08.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.072+0000 7fa968200640 1 --2- 172.21.15.100:0/2159528980 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9600ad3c0 0x7fa9600ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:08.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.072+0000 7fa968200640 1 --2- 172.21.15.100:0/2159528980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9600704b0 0x7fa960073040 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:08.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.072+0000 7fa968200640 1 -- 172.21.15.100:0/2159528980 >> 172.21.15.100:0/2159528980 conn(0x7fa96006d730 msgr2=0x7fa9600b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:08.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.072+0000 7fa968200640 1 -- 172.21.15.100:0/2159528980 shutdown_connections 2024-09-17T04:33:08.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.072+0000 7fa968200640 1 -- 172.21.15.100:0/2159528980 wait complete. 2024-09-17T04:33:08.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.074+0000 7fa968200640 1 Processor -- start 2024-09-17T04:33:08.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.075+0000 7fa968200640 1 -- start start 2024-09-17T04:33:08.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.076+0000 7fa968200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9600704b0 0x7fa960159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:08.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.076+0000 7fa968200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9600ad3c0 0x7fa9601f58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:08.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.076+0000 7fa968200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9600add10 0x7fa960150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:08.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.076+0000 7fa968200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa9600712b0 con 0x7fa9600704b0 2024-09-17T04:33:08.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.076+0000 7fa968200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa960071130 con 0x7fa9600add10 2024-09-17T04:33:08.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.076+0000 7fa968200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa9600b4350 con 0x7fa9600ad3c0 2024-09-17T04:33:08.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.076+0000 7fa965800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9600704b0 0x7fa960159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:08.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.076+0000 7fa965800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9600704b0 0x7fa960159e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55406/0 (socket says 172.21.15.100:55406) 2024-09-17T04:33:08.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.076+0000 7fa965800640 1 -- 172.21.15.100:0/3029568837 learned_addr learned my addr 172.21.15.100:0/3029568837 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:08.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.076+0000 7fa964e00640 1 --2- 172.21.15.100:0/3029568837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9600ad3c0 0x7fa9601f58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:08.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.076+0000 7fa966200640 1 --2- 172.21.15.100:0/3029568837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9600add10 0x7fa960150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:08.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.077+0000 7fa965800640 1 -- 172.21.15.100:0/3029568837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9600ad3c0 msgr2=0x7fa9601f58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:08.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.077+0000 7fa965800640 1 --2- 172.21.15.100:0/3029568837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9600ad3c0 0x7fa9601f58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:08.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.077+0000 7fa965800640 1 -- 172.21.15.100:0/3029568837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9600add10 msgr2=0x7fa960150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:08.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.077+0000 7fa965800640 1 --2- 172.21.15.100:0/3029568837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9600add10 0x7fa960150fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:08.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.077+0000 7fa965800640 1 -- 172.21.15.100:0/3029568837 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9601538b0 con 0x7fa9600704b0 2024-09-17T04:33:08.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.077+0000 7fa966200640 1 --2- 172.21.15.100:0/3029568837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9600add10 0x7fa960150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:08.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.077+0000 7fa965800640 1 --2- 172.21.15.100:0/3029568837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9600704b0 0x7fa960159e30 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fa954009340 tx=0x7fa954003880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:08.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.077+0000 7fa95d600640 1 -- 172.21.15.100:0/3029568837 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95403f070 con 0x7fa9600704b0 2024-09-17T04:33:08.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.077+0000 7fa95d600640 1 -- 172.21.15.100:0/3029568837 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9540047a0 con 0x7fa9600704b0 2024-09-17T04:33:08.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.077+0000 7fa968200640 1 -- 172.21.15.100:0/3029568837 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9601f5e10 con 0x7fa9600704b0 2024-09-17T04:33:08.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.077+0000 7fa95d600640 1 -- 172.21.15.100:0/3029568837 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa954035030 con 0x7fa9600704b0 2024-09-17T04:33:08.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.078+0000 7fa968200640 1 -- 172.21.15.100:0/3029568837 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa960159070 con 0x7fa9600704b0 2024-09-17T04:33:08.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.079+0000 7fa968200640 1 -- 172.21.15.100:0/3029568837 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa960157380 con 0x7fa9600704b0 2024-09-17T04:33:08.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.080+0000 7fa95d600640 1 -- 172.21.15.100:0/3029568837 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7fa954042070 con 0x7fa9600704b0 2024-09-17T04:33:08.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.081+0000 7fa95d600640 1 --2- 172.21.15.100:0/3029568837 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa940079400 0x7fa94007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:08.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.081+0000 7fa95d600640 1 -- 172.21.15.100:0/3029568837 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4999+0+0 (secure 0 0 0) 0x7fa9540bcc30 con 0x7fa9600704b0 2024-09-17T04:33:08.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.081+0000 7fa964e00640 1 --2- 172.21.15.100:0/3029568837 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa940079400 0x7fa94007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:08.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.082+0000 7fa964e00640 1 --2- 172.21.15.100:0/3029568837 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa940079400 0x7fa94007b910 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa96009e640 tx=0x7fa94c00b360 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:08.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.085+0000 7fa95d600640 1 -- 172.21.15.100:0/3029568837 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa960157380 con 0x7fa9600704b0 2024-09-17T04:33:08.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.270+0000 7fa968200640 1 -- 172.21.15.100:0/3029568837 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fa9600ad7d0 con 0x7fa9600704b0 2024-09-17T04:33:08.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.271+0000 7fa95d600640 1 -- 172.21.15.100:0/3029568837 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v49) ==== 59+0+2294 (secure 0 0 0) 0x7fa9601538b0 con 0x7fa9600704b0 2024-09-17T04:33:08.272 INFO:teuthology.orchestra.run.smithi100.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T04:33:08.272 INFO:teuthology.orchestra.run.smithi100.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T04:33:08.272 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced client_mount_timeout 30 2024-09-17T04:33:08.272 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_client 20/20 2024-09-17T04:33:08.272 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_mgr 20/20 2024-09-17T04:33:08.272 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_ms 1/1 2024-09-17T04:33:08.272 INFO:teuthology.orchestra.run.smithi100.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T04:33:08.272 INFO:teuthology.orchestra.run.smithi100.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T04:33:08.272 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3938.598654 2024-09-17T04:33:08.272 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4008.640154 2024-09-17T04:33:08.272 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3706.588546 2024-09-17T04:33:08.272 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 6299.338665 2024-09-17T04:33:08.273 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3633.876699 2024-09-17T04:33:08.273 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3674.821841 2024-09-17T04:33:08.273 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3789.849984 2024-09-17T04:33:08.273 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3467.457298 2024-09-17T04:33:08.273 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_mds 20/20 2024-09-17T04:33:08.273 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T04:33:08.273 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_ms 2/2 2024-09-17T04:33:08.273 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_debug_frag true 2024-09-17T04:33:08.273 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_debug_scatterstat true 2024-09-17T04:33:08.273 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T04:33:08.273 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_verify_scatter true 2024-09-17T04:33:08.273 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T04:33:08.273 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T04:33:08.274 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T04:33:08.274 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced client_mount_timeout 600 2024-09-17T04:33:08.274 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced debug_client 20/20 2024-09-17T04:33:08.274 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced debug_ms 1/1 2024-09-17T04:33:08.274 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T04:33:08.274 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T04:33:08.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.274+0000 7fa968200640 1 -- 172.21.15.100:0/3029568837 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa940079400 msgr2=0x7fa94007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:08.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.274+0000 7fa968200640 1 --2- 172.21.15.100:0/3029568837 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa940079400 0x7fa94007b910 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa96009e640 tx=0x7fa94c00b360 comp rx=0 tx=0).stop 2024-09-17T04:33:08.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.274+0000 7fa968200640 1 -- 172.21.15.100:0/3029568837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9600704b0 msgr2=0x7fa960159e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:08.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.274+0000 7fa968200640 1 --2- 172.21.15.100:0/3029568837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9600704b0 0x7fa960159e30 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fa954009340 tx=0x7fa954003880 comp rx=0 tx=0).stop 2024-09-17T04:33:08.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.274+0000 7fa968200640 1 -- 172.21.15.100:0/3029568837 shutdown_connections 2024-09-17T04:33:08.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.274+0000 7fa968200640 1 --2- 172.21.15.100:0/3029568837 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa940079400 0x7fa94007b910 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:08.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.274+0000 7fa968200640 1 --2- 172.21.15.100:0/3029568837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9600add10 0x7fa960150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:08.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.274+0000 7fa968200640 1 --2- 172.21.15.100:0/3029568837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9600ad3c0 0x7fa9601f58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:08.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.274+0000 7fa968200640 1 --2- 172.21.15.100:0/3029568837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9600704b0 0x7fa960159e30 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:08.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.274+0000 7fa968200640 1 -- 172.21.15.100:0/3029568837 >> 172.21.15.100:0/3029568837 conn(0x7fa96006d730 msgr2=0x7fa9600b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:08.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.274+0000 7fa968200640 1 -- 172.21.15.100:0/3029568837 shutdown_connections 2024-09-17T04:33:08.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.274+0000 7fa968200640 1 -- 172.21.15.100:0/3029568837 wait complete. 2024-09-17T04:33:08.300 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T04:33:08.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.522+0000 7f0a8a400640 1 -- 172.21.15.100:0/1980930317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a84145a40 msgr2=0x7f0a84073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:08.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.522+0000 7f0a8a400640 1 --2- 172.21.15.100:0/1980930317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a84145a40 0x7f0a84073540 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f0a70005fd0 tx=0x7f0a7002a670 comp rx=0 tx=0).stop 2024-09-17T04:33:08.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.522+0000 7f0a8a400640 1 -- 172.21.15.100:0/1980930317 shutdown_connections 2024-09-17T04:33:08.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.522+0000 7f0a8a400640 1 --2- 172.21.15.100:0/1980930317 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a84073a80 0x7f0a840cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:08.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.522+0000 7f0a8a400640 1 --2- 172.21.15.100:0/1980930317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a84145a40 0x7f0a84073540 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:08.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.522+0000 7f0a8a400640 1 -- 172.21.15.100:0/1980930317 >> 172.21.15.100:0/1980930317 conn(0x7f0a8406d1c0 msgr2=0x7f0a8406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:08.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.522+0000 7f0a8a400640 1 -- 172.21.15.100:0/1980930317 shutdown_connections 2024-09-17T04:33:08.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.522+0000 7f0a8a400640 1 -- 172.21.15.100:0/1980930317 wait complete. 2024-09-17T04:33:08.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.524+0000 7f0a8a400640 1 Processor -- start 2024-09-17T04:33:08.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.524+0000 7f0a8a400640 1 -- start start 2024-09-17T04:33:08.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.525+0000 7f0a8a400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a84073a80 0x7f0a841f9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:08.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.525+0000 7f0a8a400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a84145550 0x7f0a840af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:08.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.525+0000 7f0a8a400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a84145a40 0x7f0a840b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:08.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.525+0000 7f0a8a400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0a840ce160 con 0x7f0a84145550 2024-09-17T04:33:08.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.525+0000 7f0a8a400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0a840cdfe0 con 0x7f0a84145a40 2024-09-17T04:33:08.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.525+0000 7f0a8a400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0a840ce2e0 con 0x7f0a84073a80 2024-09-17T04:33:08.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.526+0000 7f0a82a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a84145550 0x7f0a840af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:08.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.526+0000 7f0a82a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a84145550 0x7f0a840af4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55416/0 (socket says 172.21.15.100:55416) 2024-09-17T04:33:08.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.526+0000 7f0a83400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a84073a80 0x7f0a841f9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:08.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.526+0000 7f0a82a00640 1 -- 172.21.15.100:0/2385931145 learned_addr learned my addr 172.21.15.100:0/2385931145 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:08.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.526+0000 7f0a83e00640 1 --2- 172.21.15.100:0/2385931145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a84145a40 0x7f0a840b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:08.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.526+0000 7f0a82a00640 1 -- 172.21.15.100:0/2385931145 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a84073a80 msgr2=0x7f0a841f9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:08.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.526+0000 7f0a82a00640 1 --2- 172.21.15.100:0/2385931145 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a84073a80 0x7f0a841f9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:08.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.526+0000 7f0a82a00640 1 -- 172.21.15.100:0/2385931145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a84145a40 msgr2=0x7f0a840b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:08.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.526+0000 7f0a82a00640 1 --2- 172.21.15.100:0/2385931145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a84145a40 0x7f0a840b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:08.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.526+0000 7f0a82a00640 1 -- 172.21.15.100:0/2385931145 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a840af9f0 con 0x7f0a84145550 2024-09-17T04:33:08.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.527+0000 7f0a82a00640 1 --2- 172.21.15.100:0/2385931145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a84145550 0x7f0a840af4b0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f0a70034040 tx=0x7f0a700038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:08.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.527+0000 7f0a7b400640 1 -- 172.21.15.100:0/2385931145 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7003e030 con 0x7f0a84145550 2024-09-17T04:33:08.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.527+0000 7f0a7b400640 1 -- 172.21.15.100:0/2385931145 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a70003d90 con 0x7f0a84145550 2024-09-17T04:33:08.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.527+0000 7f0a7b400640 1 -- 172.21.15.100:0/2385931145 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a70035040 con 0x7f0a84145550 2024-09-17T04:33:08.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.527+0000 7f0a8a400640 1 -- 172.21.15.100:0/2385931145 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a841f5460 con 0x7f0a84145550 2024-09-17T04:33:08.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.527+0000 7f0a8a400640 1 -- 172.21.15.100:0/2385931145 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a841f5720 con 0x7f0a84145550 2024-09-17T04:33:08.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.528+0000 7f0a8a400640 1 -- 172.21.15.100:0/2385931145 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a8413a370 con 0x7f0a84145550 2024-09-17T04:33:08.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.529+0000 7f0a7b400640 1 -- 172.21.15.100:0/2385931145 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f0a70041030 con 0x7f0a84145550 2024-09-17T04:33:08.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.530+0000 7f0a7b400640 1 --2- 172.21.15.100:0/2385931145 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0a64079430 0x7f0a6407b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:08.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.530+0000 7f0a7b400640 1 -- 172.21.15.100:0/2385931145 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4999+0+0 (secure 0 0 0) 0x7f0a700bc340 con 0x7f0a84145550 2024-09-17T04:33:08.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.531+0000 7f0a83400640 1 --2- 172.21.15.100:0/2385931145 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0a64079430 0x7f0a6407b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:08.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.531+0000 7f0a83400640 1 --2- 172.21.15.100:0/2385931145 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0a64079430 0x7f0a6407b940 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0a74001720 tx=0x7f0a74009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:08.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.535+0000 7f0a7b400640 1 -- 172.21.15.100:0/2385931145 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0a70083590 con 0x7f0a84145550 2024-09-17T04:33:08.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.719+0000 7f0a8a400640 1 -- 172.21.15.100:0/2385931145 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f0a840ce790 con 0x7f0a84145550 2024-09-17T04:33:08.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.724+0000 7f0a7b400640 1 -- 172.21.15.100:0/2385931145 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v50)=0 v50) ==== 122+0+0 (secure 0 0 0) 0x7f0a70087fc0 con 0x7f0a84145550 2024-09-17T04:33:08.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.726+0000 7f0a8a400640 1 -- 172.21.15.100:0/2385931145 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0a64079430 msgr2=0x7f0a6407b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:08.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.726+0000 7f0a8a400640 1 --2- 172.21.15.100:0/2385931145 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0a64079430 0x7f0a6407b940 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0a74001720 tx=0x7f0a74009040 comp rx=0 tx=0).stop 2024-09-17T04:33:08.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.726+0000 7f0a8a400640 1 -- 172.21.15.100:0/2385931145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a84145550 msgr2=0x7f0a840af4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:08.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.726+0000 7f0a8a400640 1 --2- 172.21.15.100:0/2385931145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a84145550 0x7f0a840af4b0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f0a70034040 tx=0x7f0a700038a0 comp rx=0 tx=0).stop 2024-09-17T04:33:08.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.727+0000 7f0a8a400640 1 -- 172.21.15.100:0/2385931145 shutdown_connections 2024-09-17T04:33:08.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.727+0000 7f0a8a400640 1 --2- 172.21.15.100:0/2385931145 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0a64079430 0x7f0a6407b940 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:08.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.727+0000 7f0a8a400640 1 --2- 172.21.15.100:0/2385931145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a84145a40 0x7f0a840b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:08.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.727+0000 7f0a8a400640 1 --2- 172.21.15.100:0/2385931145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a84145550 0x7f0a840af4b0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:08.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.727+0000 7f0a8a400640 1 --2- 172.21.15.100:0/2385931145 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a84073a80 0x7f0a841f9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:08.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.727+0000 7f0a8a400640 1 -- 172.21.15.100:0/2385931145 >> 172.21.15.100:0/2385931145 conn(0x7f0a8406d1c0 msgr2=0x7f0a84144250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:08.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.727+0000 7f0a8a400640 1 -- 172.21.15.100:0/2385931145 shutdown_connections 2024-09-17T04:33:08.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:08.727+0000 7f0a8a400640 1 -- 172.21.15.100:0/2385931145 wait complete. 2024-09-17T04:33:08.749 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T04:33:08.749 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:33:08.835 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:33:08.930 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T04:33:08.930 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:33:08.930 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:33:08.966 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:33:08.966 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:33:08.966 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:33:09.003 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:33:09.003 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:33:09.003 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:33:09.041 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:33:09.041 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:33:09.041 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:33:09.079 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:33:09.079 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T04:33:09.080 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:33:09.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.299+0000 7f7b97c00640 1 -- 172.21.15.100:0/3243236578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b900ad3c0 msgr2=0x7f7b900ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:09.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.299+0000 7f7b97c00640 1 --2- 172.21.15.100:0/3243236578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b900ad3c0 0x7f7b900ad7d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c009830 tx=0x7f7b7c02a480 comp rx=0 tx=0).stop 2024-09-17T04:33:09.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.300+0000 7f7b97c00640 1 -- 172.21.15.100:0/3243236578 shutdown_connections 2024-09-17T04:33:09.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.300+0000 7f7b97c00640 1 --2- 172.21.15.100:0/3243236578 >> v2:172.21.15.184:3300/0 conn(0x7f7b900add10 0x7f7b901646e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:09.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.300+0000 7f7b97c00640 1 --2- 172.21.15.100:0/3243236578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b900ad3c0 0x7f7b900ad7d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:09.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.300+0000 7f7b97c00640 1 -- 172.21.15.100:0/3243236578 >> 172.21.15.100:0/3243236578 conn(0x7f7b9006d730 msgr2=0x7f7b900b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:09.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.300+0000 7f7b97c00640 1 -- 172.21.15.100:0/3243236578 shutdown_connections 2024-09-17T04:33:09.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.300+0000 7f7b97c00640 1 -- 172.21.15.100:0/3243236578 wait complete. 2024-09-17T04:33:09.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.302+0000 7f7b97c00640 1 Processor -- start 2024-09-17T04:33:09.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.302+0000 7f7b97c00640 1 -- start start 2024-09-17T04:33:09.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.303+0000 7f7b97c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b900704b0 0x7f7b901f99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:09.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.303+0000 7f7b97c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b900ad3c0 0x7f7b9014c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:09.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.303+0000 7f7b97c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b900add10 0x7f7b9015d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:09.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.303+0000 7f7b97c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7b900712b0 con 0x7f7b900704b0 2024-09-17T04:33:09.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.303+0000 7f7b97c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7b90071130 con 0x7f7b900add10 2024-09-17T04:33:09.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.303+0000 7f7b97c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7b90164c20 con 0x7f7b900ad3c0 2024-09-17T04:33:09.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.304+0000 7f7b95200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b900704b0 0x7f7b901f99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:09.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.304+0000 7f7b95200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b900704b0 0x7f7b901f99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55422/0 (socket says 172.21.15.100:55422) 2024-09-17T04:33:09.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.304+0000 7f7b95200640 1 -- 172.21.15.100:0/826183907 learned_addr learned my addr 172.21.15.100:0/826183907 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:09.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.304+0000 7f7b8fe00640 1 --2- 172.21.15.100:0/826183907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b900ad3c0 0x7f7b9014c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:09.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.304+0000 7f7b95c00640 1 --2- 172.21.15.100:0/826183907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b900add10 0x7f7b9015d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:09.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.304+0000 7f7b95200640 1 -- 172.21.15.100:0/826183907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b900ad3c0 msgr2=0x7f7b9014c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:09.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.304+0000 7f7b95200640 1 --2- 172.21.15.100:0/826183907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b900ad3c0 0x7f7b9014c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:09.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.304+0000 7f7b95200640 1 -- 172.21.15.100:0/826183907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b900add10 msgr2=0x7f7b9015d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:09.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.304+0000 7f7b95200640 1 --2- 172.21.15.100:0/826183907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b900add10 0x7f7b9015d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:09.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.304+0000 7f7b95200640 1 -- 172.21.15.100:0/826183907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b900afaa0 con 0x7f7b900704b0 2024-09-17T04:33:09.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.304+0000 7f7b95200640 1 --2- 172.21.15.100:0/826183907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b900704b0 0x7f7b901f99e0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400c430 tx=0x7f7b8400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:09.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.305+0000 7f7b8cc00640 1 -- 172.21.15.100:0/826183907 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84011790 con 0x7f7b900704b0 2024-09-17T04:33:09.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.305+0000 7f7b8cc00640 1 -- 172.21.15.100:0/826183907 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b84017070 con 0x7f7b900704b0 2024-09-17T04:33:09.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.305+0000 7f7b8cc00640 1 -- 172.21.15.100:0/826183907 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84011ce0 con 0x7f7b900704b0 2024-09-17T04:33:09.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.305+0000 7f7b97c00640 1 -- 172.21.15.100:0/826183907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b901f9f20 con 0x7f7b900704b0 2024-09-17T04:33:09.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.305+0000 7f7b97c00640 1 -- 172.21.15.100:0/826183907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b900b0990 con 0x7f7b900704b0 2024-09-17T04:33:09.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.306+0000 7f7b97c00640 1 -- 172.21.15.100:0/826183907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b900ad7d0 con 0x7f7b900704b0 2024-09-17T04:33:09.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.307+0000 7f7b8cc00640 1 -- 172.21.15.100:0/826183907 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f7b8401c460 con 0x7f7b900704b0 2024-09-17T04:33:09.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.308+0000 7f7b8cc00640 1 --2- 172.21.15.100:0/826183907 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7b70079400 0x7f7b7007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:09.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.308+0000 7f7b8cc00640 1 -- 172.21.15.100:0/826183907 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4999+0+0 (secure 0 0 0) 0x7f7b84099260 con 0x7f7b900704b0 2024-09-17T04:33:09.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.308+0000 7f7b8fe00640 1 --2- 172.21.15.100:0/826183907 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7b70079400 0x7f7b7007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:09.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.309+0000 7f7b8fe00640 1 --2- 172.21.15.100:0/826183907 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7b70079400 0x7f7b7007b910 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c034000 tx=0x7f7b7c004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:09.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.312+0000 7f7b8cc00640 1 -- 172.21.15.100:0/826183907 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f7b84063150 con 0x7f7b900704b0 2024-09-17T04:33:09.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.530+0000 7f7b97c00640 1 -- 172.21.15.100:0/826183907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b9015e420 con 0x7f7b900704b0 2024-09-17T04:33:09.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.530+0000 7f7b8cc00640 1 -- 172.21.15.100:0/826183907 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+3925 (secure 0 0 0) 0x7f7b84062a50 con 0x7f7b900704b0 2024-09-17T04:33:09.531 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:09.531 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":104,"btime":"2024-09-17T04:33:02:559023+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7028,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1529630991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1529630991},{"type":"v1","addr":"172.21.15.184:6801","nonce":1529630991}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7906,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1516666057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1516666057},{"type":"v1","addr":"172.21.15.184:6835","nonce":1516666057}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2252725120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2252725120},{"type":"v1","addr":"172.21.15.100:6833","nonce":2252725120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3936722157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3936722157},{"type":"v1","addr":"172.21.15.100:6835","nonce":3936722157}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[]} 2024-09-17T04:33:09.531 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 104 2024-09-17T04:33:09.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.533+0000 7f7b97c00640 1 -- 172.21.15.100:0/826183907 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7b70079400 msgr2=0x7f7b7007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:09.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.533+0000 7f7b97c00640 1 --2- 172.21.15.100:0/826183907 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7b70079400 0x7f7b7007b910 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c034000 tx=0x7f7b7c004730 comp rx=0 tx=0).stop 2024-09-17T04:33:09.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.533+0000 7f7b97c00640 1 -- 172.21.15.100:0/826183907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b900704b0 msgr2=0x7f7b901f99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:09.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.533+0000 7f7b97c00640 1 --2- 172.21.15.100:0/826183907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b900704b0 0x7f7b901f99e0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400c430 tx=0x7f7b8400c830 comp rx=0 tx=0).stop 2024-09-17T04:33:09.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.533+0000 7f7b97c00640 1 -- 172.21.15.100:0/826183907 shutdown_connections 2024-09-17T04:33:09.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.533+0000 7f7b97c00640 1 --2- 172.21.15.100:0/826183907 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7b70079400 0x7f7b7007b910 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:09.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.533+0000 7f7b97c00640 1 --2- 172.21.15.100:0/826183907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b900add10 0x7f7b9015d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:09.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.533+0000 7f7b97c00640 1 --2- 172.21.15.100:0/826183907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b900ad3c0 0x7f7b9014c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:09.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.533+0000 7f7b97c00640 1 --2- 172.21.15.100:0/826183907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b900704b0 0x7f7b901f99e0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:09.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.533+0000 7f7b97c00640 1 -- 172.21.15.100:0/826183907 >> 172.21.15.100:0/826183907 conn(0x7f7b9006d730 msgr2=0x7f7b900d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:09.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.533+0000 7f7b97c00640 1 -- 172.21.15.100:0/826183907 shutdown_connections 2024-09-17T04:33:09.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.533+0000 7f7b97c00640 1 -- 172.21.15.100:0/826183907 wait complete. 2024-09-17T04:33:09.555 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T04:33:09.555 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T04:33:09.556 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T04:33:09.556 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:33:09.556 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T04:33:09.556 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T04:33:09.556 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T04:33:09.556 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:33:09.557 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T04:33:09.557 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T04:33:09.557 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T04:33:09.557 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:33:09.557 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T04:33:09.557 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T04:33:09.558 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T04:33:09.558 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:33:09.581 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T04:33:09.582 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T04:33:09.583 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T04:33:09.584 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T04:33:09.592 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T04:33:09.592 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T04:33:09.593 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T04:33:09.593 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T04:33:09.628 INFO:tasks.ceph.mds.c:Started 2024-09-17T04:33:09.628 DEBUG:teuthology.parallel:result is None 2024-09-17T04:33:09.631 INFO:tasks.ceph.mds.a:Started 2024-09-17T04:33:09.631 DEBUG:teuthology.parallel:result is None 2024-09-17T04:33:09.637 INFO:tasks.ceph.mds.d:Started 2024-09-17T04:33:09.637 DEBUG:teuthology.parallel:result is None 2024-09-17T04:33:09.639 INFO:tasks.ceph.mds.b:Started 2024-09-17T04:33:09.639 DEBUG:teuthology.parallel:result is None 2024-09-17T04:33:09.640 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T04:33:09.794 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-09-17T04:33:09.795 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-09-17T04:33:09.819 INFO:tasks.ceph.mds.c.smithi100.stdout:starting mds.c at 2024-09-17T04:33:09.826 INFO:tasks.ceph.mds.a.smithi100.stdout:starting mds.a at 2024-09-17T04:33:09.834 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-09-17T04:33:09.837 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-09-17T04:33:09.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.875+0000 7fb0c6800640 1 -- 172.21.15.100:0/671213587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0c00b7f40 msgr2=0x7fb0c00b5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:09.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.875+0000 7fb0c6800640 1 --2- 172.21.15.100:0/671213587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0c00b7f40 0x7fb0c00b5f70 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8005fd0 tx=0x7fb0b802aa10 comp rx=0 tx=0).stop 2024-09-17T04:33:09.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.875+0000 7fb0c6800640 1 -- 172.21.15.100:0/671213587 shutdown_connections 2024-09-17T04:33:09.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.875+0000 7fb0c6800640 1 --2- 172.21.15.100:0/671213587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0c00b7f40 0x7fb0c00b5f70 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:09.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.875+0000 7fb0c6800640 1 --2- 172.21.15.100:0/671213587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0c015e770 0x7fb0c00b7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:09.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.875+0000 7fb0c6800640 1 -- 172.21.15.100:0/671213587 >> 172.21.15.100:0/671213587 conn(0x7fb0c00d44f0 msgr2=0x7fb0c00d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:09.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.875+0000 7fb0c6800640 1 -- 172.21.15.100:0/671213587 shutdown_connections 2024-09-17T04:33:09.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.875+0000 7fb0c6800640 1 -- 172.21.15.100:0/671213587 wait complete. 2024-09-17T04:33:09.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.878+0000 7fb0c6800640 1 Processor -- start 2024-09-17T04:33:09.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.878+0000 7fb0c6800640 1 -- start start 2024-09-17T04:33:09.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.879+0000 7fb0c6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0c00b7f40 0x7fb0c01f4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:09.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.879+0000 7fb0c6800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0c015e770 0x7fb0c01e9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:09.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.879+0000 7fb0c6800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0c014d130 0x7fb0c00ac140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:09.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.879+0000 7fb0c6800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c00d39a0 con 0x7fb0c00b7f40 2024-09-17T04:33:09.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.879+0000 7fb0c6800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c00d3820 con 0x7fb0c014d130 2024-09-17T04:33:09.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.879+0000 7fb0c6800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb0c00d3b20 con 0x7fb0c015e770 2024-09-17T04:33:09.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.880+0000 7fb0c5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0c00b7f40 0x7fb0c01f4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:09.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.880+0000 7fb0c5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0c00b7f40 0x7fb0c01f4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55472/0 (socket says 172.21.15.100:55472) 2024-09-17T04:33:09.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.880+0000 7fb0c5400640 1 -- 172.21.15.100:0/1708680827 learned_addr learned my addr 172.21.15.100:0/1708680827 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:09.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.880+0000 7fb0c4a00640 1 --2- 172.21.15.100:0/1708680827 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0c015e770 0x7fb0c01e9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:09.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.880+0000 7fb0c5e00640 1 --2- 172.21.15.100:0/1708680827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0c014d130 0x7fb0c00ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:09.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.880+0000 7fb0c5400640 1 -- 172.21.15.100:0/1708680827 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0c015e770 msgr2=0x7fb0c01e9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:09.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.880+0000 7fb0c5400640 1 --2- 172.21.15.100:0/1708680827 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0c015e770 0x7fb0c01e9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:09.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.880+0000 7fb0c5400640 1 -- 172.21.15.100:0/1708680827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0c014d130 msgr2=0x7fb0c00ac140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:09.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.880+0000 7fb0c5400640 1 --2- 172.21.15.100:0/1708680827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0c014d130 0x7fb0c00ac140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:09.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.880+0000 7fb0c5400640 1 -- 172.21.15.100:0/1708680827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c00ab0a0 con 0x7fb0c00b7f40 2024-09-17T04:33:09.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.880+0000 7fb0c5e00640 1 --2- 172.21.15.100:0/1708680827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0c014d130 0x7fb0c00ac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:09.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.880+0000 7fb0c5400640 1 --2- 172.21.15.100:0/1708680827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0c00b7f40 0x7fb0c01f4f10 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400c3d0 tx=0x7fb0b400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:09.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.886+0000 7fb0bd600640 1 -- 172.21.15.100:0/1708680827 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4017070 con 0x7fb0c00b7f40 2024-09-17T04:33:09.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.886+0000 7fb0c6800640 1 -- 172.21.15.100:0/1708680827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c00ab380 con 0x7fb0c00b7f40 2024-09-17T04:33:09.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.886+0000 7fb0bd600640 1 -- 172.21.15.100:0/1708680827 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0b40053c0 con 0x7fb0c00b7f40 2024-09-17T04:33:09.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.886+0000 7fb0bd600640 1 -- 172.21.15.100:0/1708680827 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4003850 con 0x7fb0c00b7f40 2024-09-17T04:33:09.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.886+0000 7fb0c6800640 1 -- 172.21.15.100:0/1708680827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c00ab940 con 0x7fb0c00b7f40 2024-09-17T04:33:09.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.888+0000 7fb0c6800640 1 -- 172.21.15.100:0/1708680827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0c00cf6c0 con 0x7fb0c00b7f40 2024-09-17T04:33:09.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.889+0000 7fb0bd600640 1 -- 172.21.15.100:0/1708680827 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7fb0b401a070 con 0x7fb0c00b7f40 2024-09-17T04:33:09.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.891+0000 7fb0bd600640 1 --2- 172.21.15.100:0/1708680827 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb0a0079400 0x7fb0a007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:09.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.891+0000 7fb0bd600640 1 -- 172.21.15.100:0/1708680827 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 4999+0+0 (secure 0 0 0) 0x7fb0b409b130 con 0x7fb0c00b7f40 2024-09-17T04:33:09.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.891+0000 7fb0c4a00640 1 --2- 172.21.15.100:0/1708680827 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb0a0079400 0x7fb0a007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:09.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.891+0000 7fb0c4a00640 1 --2- 172.21.15.100:0/1708680827 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb0a0079400 0x7fb0a007b910 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00d4b60 tx=0x7fb0ac005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:09.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:09.896+0000 7fb0bd600640 1 -- 172.21.15.100:0/1708680827 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb0b4062130 con 0x7fb0c00b7f40 2024-09-17T04:33:10.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:10.091+0000 7fb0c6800640 1 -- 172.21.15.100:0/1708680827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fb0c00b5ea0 con 0x7fb0c00b7f40 2024-09-17T04:33:10.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:10.846+0000 7fb0bd600640 1 -- 172.21.15.100:0/1708680827 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v151) ==== 97+0+0 (secure 0 0 0) 0x7fb0b4066b60 con 0x7fb0c00b7f40 2024-09-17T04:33:10.847 INFO:teuthology.orchestra.run.smithi100.stderr: removed all blocklist entries 2024-09-17T04:33:10.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:10.849+0000 7fb0c6800640 1 -- 172.21.15.100:0/1708680827 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb0a0079400 msgr2=0x7fb0a007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:10.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:10.849+0000 7fb0c6800640 1 --2- 172.21.15.100:0/1708680827 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb0a0079400 0x7fb0a007b910 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00d4b60 tx=0x7fb0ac005f90 comp rx=0 tx=0).stop 2024-09-17T04:33:10.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:10.849+0000 7fb0c6800640 1 -- 172.21.15.100:0/1708680827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0c00b7f40 msgr2=0x7fb0c01f4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:10.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:10.849+0000 7fb0c6800640 1 --2- 172.21.15.100:0/1708680827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0c00b7f40 0x7fb0c01f4f10 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400c3d0 tx=0x7fb0b400c890 comp rx=0 tx=0).stop 2024-09-17T04:33:10.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:10.849+0000 7fb0c6800640 1 -- 172.21.15.100:0/1708680827 shutdown_connections 2024-09-17T04:33:10.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:10.849+0000 7fb0c6800640 1 --2- 172.21.15.100:0/1708680827 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb0a0079400 0x7fb0a007b910 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:10.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:10.849+0000 7fb0c6800640 1 --2- 172.21.15.100:0/1708680827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0c014d130 0x7fb0c00ac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:10.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:10.849+0000 7fb0c6800640 1 --2- 172.21.15.100:0/1708680827 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0c015e770 0x7fb0c01e9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:10.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:10.849+0000 7fb0c6800640 1 --2- 172.21.15.100:0/1708680827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0c00b7f40 0x7fb0c01f4f10 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:10.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:10.849+0000 7fb0c6800640 1 -- 172.21.15.100:0/1708680827 >> 172.21.15.100:0/1708680827 conn(0x7fb0c00d44f0 msgr2=0x7fb0c015fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:10.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:10.849+0000 7fb0c6800640 1 -- 172.21.15.100:0/1708680827 shutdown_connections 2024-09-17T04:33:10.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:10.849+0000 7fb0c6800640 1 -- 172.21.15.100:0/1708680827 wait complete. 2024-09-17T04:33:10.874 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T04:33:11.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.093+0000 7ff0cb000640 1 -- 172.21.15.100:0/2275570960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0c40ad0e0 msgr2=0x7ff0c40ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:11.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.093+0000 7ff0cb000640 1 --2- 172.21.15.100:0/2275570960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0c40ad0e0 0x7ff0c40ad4f0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0005fd0 tx=0x7ff0b002a540 comp rx=0 tx=0).stop 2024-09-17T04:33:11.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.093+0000 7ff0cb000640 1 -- 172.21.15.100:0/2275570960 shutdown_connections 2024-09-17T04:33:11.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.093+0000 7ff0cb000640 1 --2- 172.21.15.100:0/2275570960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0c40ad0e0 0x7ff0c40ad4f0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:11.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.093+0000 7ff0cb000640 1 -- 172.21.15.100:0/2275570960 >> 172.21.15.100:0/2275570960 conn(0x7ff0c406d730 msgr2=0x7ff0c40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:11.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.093+0000 7ff0cb000640 1 -- 172.21.15.100:0/2275570960 shutdown_connections 2024-09-17T04:33:11.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.093+0000 7ff0cb000640 1 -- 172.21.15.100:0/2275570960 wait complete. 2024-09-17T04:33:11.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.095+0000 7ff0cb000640 1 Processor -- start 2024-09-17T04:33:11.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.095+0000 7ff0cb000640 1 -- start start 2024-09-17T04:33:11.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.097+0000 7ff0cb000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0c40ad0e0 0x7ff0c4156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:11.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.097+0000 7ff0cb000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0c40b3550 0x7ff0c41466d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:11.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.097+0000 7ff0cb000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0c415d760 0x7ff0c4159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:11.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.097+0000 7ff0cb000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c40b4da0 con 0x7ff0c415d760 2024-09-17T04:33:11.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.097+0000 7ff0cb000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c40b4c20 con 0x7ff0c40b3550 2024-09-17T04:33:11.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.097+0000 7ff0cb000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff0c40b4070 con 0x7ff0c40ad0e0 2024-09-17T04:33:11.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.097+0000 7ff0c9000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0c415d760 0x7ff0c4159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:11.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.097+0000 7ff0c9000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0c415d760 0x7ff0c4159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55490/0 (socket says 172.21.15.100:55490) 2024-09-17T04:33:11.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.097+0000 7ff0c9000640 1 -- 172.21.15.100:0/4074314266 learned_addr learned my addr 172.21.15.100:0/4074314266 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:11.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.097+0000 7ff0c3e00640 1 --2- 172.21.15.100:0/4074314266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0c40ad0e0 0x7ff0c4156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:11.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.097+0000 7ff0c3400640 1 --2- 172.21.15.100:0/4074314266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0c40b3550 0x7ff0c41466d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:11.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.097+0000 7ff0c9000640 1 -- 172.21.15.100:0/4074314266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0c40ad0e0 msgr2=0x7ff0c4156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:11.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.097+0000 7ff0c9000640 1 --2- 172.21.15.100:0/4074314266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0c40ad0e0 0x7ff0c4156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:11.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.097+0000 7ff0c9000640 1 -- 172.21.15.100:0/4074314266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0c40b3550 msgr2=0x7ff0c41466d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:11.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.097+0000 7ff0c9000640 1 --2- 172.21.15.100:0/4074314266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0c40b3550 0x7ff0c41466d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:11.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.098+0000 7ff0c9000640 1 -- 172.21.15.100:0/4074314266 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c4155170 con 0x7ff0c415d760 2024-09-17T04:33:11.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.098+0000 7ff0c9000640 1 --2- 172.21.15.100:0/4074314266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0c415d760 0x7ff0c4159c20 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00ba10 tx=0x7ff0bc00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:11.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.098+0000 7ff0bbe00640 1 -- 172.21.15.100:0/4074314266 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc019070 con 0x7ff0c415d760 2024-09-17T04:33:11.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.098+0000 7ff0bbe00640 1 -- 172.21.15.100:0/4074314266 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0bc00cc30 con 0x7ff0c415d760 2024-09-17T04:33:11.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.098+0000 7ff0bbe00640 1 -- 172.21.15.100:0/4074314266 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc00ce60 con 0x7ff0c415d760 2024-09-17T04:33:11.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.098+0000 7ff0cb000640 1 -- 172.21.15.100:0/4074314266 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0c4155450 con 0x7ff0c415d760 2024-09-17T04:33:11.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.098+0000 7ff0cb000640 1 -- 172.21.15.100:0/4074314266 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0c4155a10 con 0x7ff0c415d760 2024-09-17T04:33:11.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.099+0000 7ff0cb000640 1 -- 172.21.15.100:0/4074314266 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0c40ad4f0 con 0x7ff0c415d760 2024-09-17T04:33:11.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.101+0000 7ff0bbe00640 1 -- 172.21.15.100:0/4074314266 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7ff0bc01c070 con 0x7ff0c415d760 2024-09-17T04:33:11.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.102+0000 7ff0bbe00640 1 --2- 172.21.15.100:0/4074314266 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff0a4079400 0x7ff0a407b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:11.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.102+0000 7ff0bbe00640 1 -- 172.21.15.100:0/4074314266 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4913+0+0 (secure 0 0 0) 0x7ff0bc099110 con 0x7ff0c415d760 2024-09-17T04:33:11.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.102+0000 7ff0c3e00640 1 --2- 172.21.15.100:0/4074314266 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff0a4079400 0x7ff0a407b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:11.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.103+0000 7ff0c3e00640 1 --2- 172.21.15.100:0/4074314266 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff0a4079400 0x7ff0a407b910 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4000f60 tx=0x7ff0b4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:11.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.107+0000 7ff0bbe00640 1 -- 172.21.15.100:0/4074314266 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff0bc062990 con 0x7ff0c415d760 2024-09-17T04:33:11.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.360+0000 7ff0cb000640 1 -- 172.21.15.100:0/4074314266 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7ff0c4154e90 con 0x7ff0c415d760 2024-09-17T04:33:11.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.361+0000 7ff0bbe00640 1 -- 172.21.15.100:0/4074314266 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) ==== 80+0+5304 (secure 0 0 0) 0x7ff0bc0673c0 con 0x7ff0c415d760 2024-09-17T04:33:11.361 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:11.361 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:33:11.361 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_dump": [ 2024-09-17T04:33:11.361 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:11.362 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.a", 2024-09-17T04:33:11.362 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmV8asJRAAkSgIWqGc1OeSRbs9PEOT8w==", 2024-09-17T04:33:11.362 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:11.362 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:33:11.362 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:33:11.362 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:33:11.362 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:11.362 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:11.362 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:11.362 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:11.362 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.b", 2024-09-17T04:33:11.362 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmpM7ZBRAA1Og785tJwa6NA9zBdmNNQA==", 2024-09-17T04:33:11.362 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:11.363 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:33:11.363 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:33:11.363 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:33:11.363 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:11.363 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:11.363 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:11.363 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:11.363 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.c", 2024-09-17T04:33:11.363 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmXaOGMxAAT+96g/5fAzPW8Cc2HtQUyA==", 2024-09-17T04:33:11.364 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:11.364 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:33:11.364 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:33:11.364 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:33:11.364 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:11.364 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:11.364 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:11.364 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:11.364 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.d", 2024-09-17T04:33:11.364 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmRqC9ExAA23kZ0Ks1lGN2k4GXL8ASrg==", 2024-09-17T04:33:11.364 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:11.364 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:33:11.364 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:33:11.365 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:33:11.365 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:11.365 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:11.365 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:11.365 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:11.365 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.0", 2024-09-17T04:33:11.365 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAwBOlmofFmAxAARerYkU9B2a9MM9vHa3o4LQ==", 2024-09-17T04:33:11.365 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:11.365 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:33:11.365 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:33:11.365 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:11.365 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:11.365 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:11.365 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:11.366 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.1", 2024-09-17T04:33:11.366 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAzBOlmbz6VExAAoVqSqXTc1wArKhdA6+NxnA==", 2024-09-17T04:33:11.366 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:11.366 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:33:11.366 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:33:11.366 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:11.366 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:11.366 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:11.366 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:11.366 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.2", 2024-09-17T04:33:11.366 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA2BOlmfeSAJBAAchY6XhyDRusc+zqwQ6v6wQ==", 2024-09-17T04:33:11.366 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:11.366 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:33:11.366 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:33:11.366 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:11.367 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:11.367 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:11.367 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:11.367 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.3", 2024-09-17T04:33:11.367 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA6BOlm/TQiBBAAHhbs2+RCjAenGR1frhbTmQ==", 2024-09-17T04:33:11.367 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:11.367 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:33:11.367 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:33:11.367 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:11.367 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:11.367 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:11.367 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:11.367 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.4", 2024-09-17T04:33:11.367 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBABOlm7OKvBhAALkaAKEUYN5ChwGejoSF8yA==", 2024-09-17T04:33:11.367 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:11.367 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:33:11.368 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:33:11.368 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:11.368 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:11.368 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:11.368 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:11.368 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.5", 2024-09-17T04:33:11.368 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBDBOlmQpi1FhAAbA47dg6ckK1PFoCZL0SBEQ==", 2024-09-17T04:33:11.368 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:11.368 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:33:11.368 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:33:11.368 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:11.368 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:11.368 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:11.368 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:11.368 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.6", 2024-09-17T04:33:11.369 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBGBOlmx/w0JhAAWcPvZkMYtXD/EMKrg/W/4Q==", 2024-09-17T04:33:11.369 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:11.369 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:33:11.369 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:33:11.369 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:11.369 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:11.369 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:11.369 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:11.369 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.7", 2024-09-17T04:33:11.369 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBJBOlmWHjWNBAAgdul2UKY9g5p7EVi0ZdrFw==", 2024-09-17T04:33:11.369 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:11.369 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:33:11.369 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:33:11.369 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:11.370 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:11.370 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:11.370 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:11.370 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.0", 2024-09-17T04:33:11.370 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmfYg9IBAABP1uRKGOugrA47VxwRLkjA==", 2024-09-17T04:33:11.370 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:11.370 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:33:11.370 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow r", 2024-09-17T04:33:11.370 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T04:33:11.370 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:11.370 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:11.370 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:11.370 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.1", 2024-09-17T04:33:11.370 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmByhhNRAAPGaxFCDd8ywFVwwhprrPbQ==", 2024-09-17T04:33:11.370 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:11.370 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:33:11.371 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:33:11.371 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:33:11.371 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:33:11.371 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:11.371 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:11.371 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:11.371 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.2", 2024-09-17T04:33:11.371 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmgcm0KhAAGF06VlHjBM/AGxafN6/3aA==", 2024-09-17T04:33:11.371 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:11.371 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:33:11.371 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:33:11.371 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:33:11.371 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:33:11.371 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:11.372 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:11.372 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:11.372 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.3", 2024-09-17T04:33:11.372 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtBOlm9VQ6BBAAXcahu35/uDm8Z+f+3SF0Gg==", 2024-09-17T04:33:11.372 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:11.372 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:33:11.372 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:33:11.372 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:33:11.372 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:33:11.372 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:11.372 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:11.372 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:11.372 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.admin", 2024-09-17T04:33:11.372 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAqBOlmZ+bIJBAAG3oaMNlYWNakywGDvAGbGw==", 2024-09-17T04:33:11.373 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:11.373 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:33:11.373 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:33:11.373 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:33:11.373 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:11.373 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:11.373 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:11.373 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:11.373 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mgr.x", 2024-09-17T04:33:11.373 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmfaL7GhAArqLH66RkbBsz+TNOKtPHOA==", 2024-09-17T04:33:11.373 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:11.373 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:33:11.373 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile mgr", 2024-09-17T04:33:11.373 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:11.374 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:11.374 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:11.374 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:11.374 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mgr.y", 2024-09-17T04:33:11.374 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmM4D1DhAA5uUWgRVOMujWqbYzOjFp9A==", 2024-09-17T04:33:11.374 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:11.374 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:33:11.374 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile mgr", 2024-09-17T04:33:11.374 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:11.374 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:11.374 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:11.374 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-17T04:33:11.374 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:33:11.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.363+0000 7ff0cb000640 1 -- 172.21.15.100:0/4074314266 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff0a4079400 msgr2=0x7ff0a407b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:11.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.363+0000 7ff0cb000640 1 --2- 172.21.15.100:0/4074314266 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff0a4079400 0x7ff0a407b910 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4000f60 tx=0x7ff0b4009040 comp rx=0 tx=0).stop 2024-09-17T04:33:11.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.363+0000 7ff0cb000640 1 -- 172.21.15.100:0/4074314266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0c415d760 msgr2=0x7ff0c4159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:11.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.363+0000 7ff0cb000640 1 --2- 172.21.15.100:0/4074314266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0c415d760 0x7ff0c4159c20 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00ba10 tx=0x7ff0bc00bed0 comp rx=0 tx=0).stop 2024-09-17T04:33:11.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.363+0000 7ff0cb000640 1 -- 172.21.15.100:0/4074314266 shutdown_connections 2024-09-17T04:33:11.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.363+0000 7ff0cb000640 1 --2- 172.21.15.100:0/4074314266 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff0a4079400 0x7ff0a407b910 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:11.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.363+0000 7ff0cb000640 1 --2- 172.21.15.100:0/4074314266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0c415d760 0x7ff0c4159c20 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:11.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.363+0000 7ff0cb000640 1 --2- 172.21.15.100:0/4074314266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0c40b3550 0x7ff0c41466d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:11.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.363+0000 7ff0cb000640 1 --2- 172.21.15.100:0/4074314266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0c40ad0e0 0x7ff0c4156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:11.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.363+0000 7ff0cb000640 1 -- 172.21.15.100:0/4074314266 >> 172.21.15.100:0/4074314266 conn(0x7ff0c406d730 msgr2=0x7ff0c415eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:11.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.363+0000 7ff0cb000640 1 -- 172.21.15.100:0/4074314266 shutdown_connections 2024-09-17T04:33:11.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.363+0000 7ff0cb000640 1 -- 172.21.15.100:0/4074314266 wait complete. 2024-09-17T04:33:11.385 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T04:33:11.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.605+0000 7f22aa400640 1 -- 172.21.15.100:0/834237350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22a4165480 msgr2=0x7f22a41678a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:11.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.605+0000 7f22aa400640 1 --2- 172.21.15.100:0/834237350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22a4165480 0x7f22a41678a0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f229c005fd0 tx=0x7f229c02aa10 comp rx=0 tx=0).stop 2024-09-17T04:33:11.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.605+0000 7f22aa400640 1 -- 172.21.15.100:0/834237350 shutdown_connections 2024-09-17T04:33:11.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.605+0000 7f22aa400640 1 --2- 172.21.15.100:0/834237350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22a4165480 0x7f22a41678a0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:11.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.605+0000 7f22aa400640 1 -- 172.21.15.100:0/834237350 >> 172.21.15.100:0/834237350 conn(0x7f22a406d730 msgr2=0x7f22a40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:11.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.605+0000 7f22aa400640 1 -- 172.21.15.100:0/834237350 shutdown_connections 2024-09-17T04:33:11.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.605+0000 7f22aa400640 1 -- 172.21.15.100:0/834237350 wait complete. 2024-09-17T04:33:11.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.608+0000 7f22aa400640 1 Processor -- start 2024-09-17T04:33:11.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.608+0000 7f22aa400640 1 -- start start 2024-09-17T04:33:11.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.609+0000 7f22aa400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22a40ad0e0 0x7f22a4147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:11.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.609+0000 7f22aa400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22a415d760 0x7f22a4159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:11.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.609+0000 7f22aa400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22a4165480 0x7f22a40afd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:11.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.609+0000 7f22aa400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f22a4167de0 con 0x7f22a40ad0e0 2024-09-17T04:33:11.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.609+0000 7f22aa400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f22a40b4e30 con 0x7f22a4165480 2024-09-17T04:33:11.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.609+0000 7f22aa400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f22a4167f60 con 0x7f22a415d760 2024-09-17T04:33:11.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.609+0000 7f22a3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22a40ad0e0 0x7f22a4147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:11.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.609+0000 7f22a3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22a40ad0e0 0x7f22a4147c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55510/0 (socket says 172.21.15.100:55510) 2024-09-17T04:33:11.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.609+0000 7f22a3400640 1 -- 172.21.15.100:0/263431954 learned_addr learned my addr 172.21.15.100:0/263431954 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:11.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.610+0000 7f22a3400640 1 -- 172.21.15.100:0/263431954 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22a415d760 msgr2=0x7f22a4159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:11.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.610+0000 7f229aa00640 1 --2- 172.21.15.100:0/263431954 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22a415d760 0x7f22a4159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:11.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.610+0000 7f22a3e00640 1 --2- 172.21.15.100:0/263431954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22a4165480 0x7f22a40afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:11.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.610+0000 7f22a3400640 1 --2- 172.21.15.100:0/263431954 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22a415d760 0x7f22a4159cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:11.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.610+0000 7f22a3400640 1 -- 172.21.15.100:0/263431954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22a4165480 msgr2=0x7f22a40afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:11.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.610+0000 7f22a3400640 1 --2- 172.21.15.100:0/263431954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22a4165480 0x7f22a40afd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:11.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.610+0000 7f22a3400640 1 -- 172.21.15.100:0/263431954 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a4074930 con 0x7f22a40ad0e0 2024-09-17T04:33:11.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.610+0000 7f229aa00640 1 --2- 172.21.15.100:0/263431954 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22a415d760 0x7f22a4159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:11.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.610+0000 7f22a3e00640 1 --2- 172.21.15.100:0/263431954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22a4165480 0x7f22a40afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:11.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.610+0000 7f22a3400640 1 --2- 172.21.15.100:0/263431954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22a40ad0e0 0x7f22a4147c50 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f229400b3f0 tx=0x7f229400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:11.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.610+0000 7f229be00640 1 -- 172.21.15.100:0/263431954 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294015020 con 0x7f22a40ad0e0 2024-09-17T04:33:11.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.610+0000 7f229be00640 1 -- 172.21.15.100:0/263431954 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2294011920 con 0x7f22a40ad0e0 2024-09-17T04:33:11.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.610+0000 7f229be00640 1 -- 172.21.15.100:0/263431954 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294011d70 con 0x7f22a40ad0e0 2024-09-17T04:33:11.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.611+0000 7f22aa400640 1 -- 172.21.15.100:0/263431954 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a4074650 con 0x7f22a40ad0e0 2024-09-17T04:33:11.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.611+0000 7f22aa400640 1 -- 172.21.15.100:0/263431954 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a41594f0 con 0x7f22a40ad0e0 2024-09-17T04:33:11.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.612+0000 7f22aa400640 1 -- 172.21.15.100:0/263431954 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22a41535c0 con 0x7f22a40ad0e0 2024-09-17T04:33:11.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.613+0000 7f229be00640 1 -- 172.21.15.100:0/263431954 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f2294018020 con 0x7f22a40ad0e0 2024-09-17T04:33:11.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.614+0000 7f229be00640 1 --2- 172.21.15.100:0/263431954 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2280079400 0x7f228007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:11.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.614+0000 7f229be00640 1 -- 172.21.15.100:0/263431954 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4913+0+0 (secure 0 0 0) 0x7f2294099130 con 0x7f22a40ad0e0 2024-09-17T04:33:11.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.614+0000 7f229aa00640 1 --2- 172.21.15.100:0/263431954 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2280079400 0x7f228007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:11.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.615+0000 7f229aa00640 1 --2- 172.21.15.100:0/263431954 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2280079400 0x7f228007b910 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f22a4148d50 tx=0x7f228c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:11.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.618+0000 7f229be00640 1 -- 172.21.15.100:0/263431954 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f229401f030 con 0x7f22a40ad0e0 2024-09-17T04:33:11.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.805+0000 7f22aa400640 1 -- 172.21.15.100:0/263431954 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f22a4153a80 con 0x7f22a40ad0e0 2024-09-17T04:33:11.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.806+0000 7f229be00640 1 -- 172.21.15.100:0/263431954 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v152) ==== 84+0+65 (secure 0 0 0) 0x7f22940630b0 con 0x7f22a40ad0e0 2024-09-17T04:33:11.806 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:11.806 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-09-17T04:33:11.806 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:11.806 INFO:teuthology.orchestra.run.smithi100.stdout: "poolnum": 1, 2024-09-17T04:33:11.806 INFO:teuthology.orchestra.run.smithi100.stdout: "poolname": ".mgr" 2024-09-17T04:33:11.807 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:11.807 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-09-17T04:33:11.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.808+0000 7f22aa400640 1 -- 172.21.15.100:0/263431954 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2280079400 msgr2=0x7f228007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:11.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.808+0000 7f22aa400640 1 --2- 172.21.15.100:0/263431954 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2280079400 0x7f228007b910 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f22a4148d50 tx=0x7f228c009040 comp rx=0 tx=0).stop 2024-09-17T04:33:11.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.808+0000 7f22aa400640 1 -- 172.21.15.100:0/263431954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22a40ad0e0 msgr2=0x7f22a4147c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:11.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.808+0000 7f22aa400640 1 --2- 172.21.15.100:0/263431954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22a40ad0e0 0x7f22a4147c50 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f229400b3f0 tx=0x7f229400b8b0 comp rx=0 tx=0).stop 2024-09-17T04:33:11.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.808+0000 7f22aa400640 1 -- 172.21.15.100:0/263431954 shutdown_connections 2024-09-17T04:33:11.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.808+0000 7f22aa400640 1 --2- 172.21.15.100:0/263431954 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2280079400 0x7f228007b910 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:11.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.808+0000 7f22aa400640 1 --2- 172.21.15.100:0/263431954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22a4165480 0x7f22a40afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:11.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.808+0000 7f22aa400640 1 --2- 172.21.15.100:0/263431954 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22a415d760 0x7f22a4159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:11.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.808+0000 7f22aa400640 1 --2- 172.21.15.100:0/263431954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22a40ad0e0 0x7f22a4147c50 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:11.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.808+0000 7f22aa400640 1 -- 172.21.15.100:0/263431954 >> 172.21.15.100:0/263431954 conn(0x7f22a406d730 msgr2=0x7f22a414b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:11.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.808+0000 7f22aa400640 1 -- 172.21.15.100:0/263431954 shutdown_connections 2024-09-17T04:33:11.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:11.808+0000 7f22aa400640 1 -- 172.21.15.100:0/263431954 wait complete. 2024-09-17T04:33:11.833 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T04:33:11.834 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T04:33:12.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.059+0000 7f9e30c00640 1 -- 172.21.15.100:0/4261868651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e2c0b9bc0 msgr2=0x7f9e2c0d2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:12.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.059+0000 7f9e30c00640 1 --2- 172.21.15.100:0/4261868651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e2c0b9bc0 0x7f9e2c0d2650 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c0d0f70 tx=0x7f9e1c02a560 comp rx=0 tx=0).stop 2024-09-17T04:33:12.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.059+0000 7f9e30c00640 1 -- 172.21.15.100:0/4261868651 shutdown_connections 2024-09-17T04:33:12.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.060+0000 7f9e30c00640 1 --2- 172.21.15.100:0/4261868651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e2c0ceb70 0x7f9e2c0cef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:12.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.060+0000 7f9e30c00640 1 --2- 172.21.15.100:0/4261868651 >> v2:172.21.15.184:3300/0 conn(0x7f9e2c0b6700 0x7f9e2c0d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:12.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.060+0000 7f9e30c00640 1 --2- 172.21.15.100:0/4261868651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e2c0b9bc0 0x7f9e2c0d2650 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:12.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.060+0000 7f9e30c00640 1 -- 172.21.15.100:0/4261868651 >> 172.21.15.100:0/4261868651 conn(0x7f9e2c08bfe0 msgr2=0x7f9e2c0d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:12.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.060+0000 7f9e30c00640 1 -- 172.21.15.100:0/4261868651 shutdown_connections 2024-09-17T04:33:12.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.060+0000 7f9e30c00640 1 -- 172.21.15.100:0/4261868651 wait complete. 2024-09-17T04:33:12.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.062+0000 7f9e30c00640 1 Processor -- start 2024-09-17T04:33:12.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.062+0000 7f9e30c00640 1 -- start start 2024-09-17T04:33:12.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.063+0000 7f9e30c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e2c0b6700 0x7f9e2c1592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:12.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.063+0000 7f9e30c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e2c0b9bc0 0x7f9e2c0cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:12.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.063+0000 7f9e30c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e2c0ceb70 0x7f9e2c0cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:12.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.063+0000 7f9e30c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c0cf4c0 con 0x7f9e2c0b9bc0 2024-09-17T04:33:12.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.063+0000 7f9e30c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c0d4e40 con 0x7f9e2c0ceb70 2024-09-17T04:33:12.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.063+0000 7f9e30c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9e2c0cf640 con 0x7f9e2c0b6700 2024-09-17T04:33:12.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.064+0000 7f9e22a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e2c0b9bc0 0x7f9e2c0cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:12.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.064+0000 7f9e22a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e2c0b9bc0 0x7f9e2c0cd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55528/0 (socket says 172.21.15.100:55528) 2024-09-17T04:33:12.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.064+0000 7f9e22a00640 1 -- 172.21.15.100:0/4133800063 learned_addr learned my addr 172.21.15.100:0/4133800063 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:12.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.064+0000 7f9e2b400640 1 --2- 172.21.15.100:0/4133800063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e2c0b6700 0x7f9e2c1592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:12.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.064+0000 7f9e2be00640 1 --2- 172.21.15.100:0/4133800063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e2c0ceb70 0x7f9e2c0cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:12.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.064+0000 7f9e22a00640 1 -- 172.21.15.100:0/4133800063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e2c0b6700 msgr2=0x7f9e2c1592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:12.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.064+0000 7f9e22a00640 1 --2- 172.21.15.100:0/4133800063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e2c0b6700 0x7f9e2c1592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:12.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.064+0000 7f9e22a00640 1 -- 172.21.15.100:0/4133800063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e2c0ceb70 msgr2=0x7f9e2c0cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:12.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.064+0000 7f9e22a00640 1 --2- 172.21.15.100:0/4133800063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e2c0ceb70 0x7f9e2c0cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:12.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.064+0000 7f9e22a00640 1 -- 172.21.15.100:0/4133800063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e2c0c8770 con 0x7f9e2c0b9bc0 2024-09-17T04:33:12.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.064+0000 7f9e2b400640 1 --2- 172.21.15.100:0/4133800063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e2c0b6700 0x7f9e2c1592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:12.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.064+0000 7f9e2be00640 1 --2- 172.21.15.100:0/4133800063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e2c0ceb70 0x7f9e2c0cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:12.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.064+0000 7f9e22a00640 1 --2- 172.21.15.100:0/4133800063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e2c0b9bc0 0x7f9e2c0cd340 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f9e1400d720 tx=0x7f9e1400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:12.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.064+0000 7f9e23e00640 1 -- 172.21.15.100:0/4133800063 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e14017070 con 0x7f9e2c0b9bc0 2024-09-17T04:33:12.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.065+0000 7f9e30c00640 1 -- 172.21.15.100:0/4133800063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e2c0c8a50 con 0x7f9e2c0b9bc0 2024-09-17T04:33:12.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.065+0000 7f9e23e00640 1 -- 172.21.15.100:0/4133800063 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e14009ab0 con 0x7f9e2c0b9bc0 2024-09-17T04:33:12.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.065+0000 7f9e23e00640 1 -- 172.21.15.100:0/4133800063 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e14010460 con 0x7f9e2c0b9bc0 2024-09-17T04:33:12.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.065+0000 7f9e30c00640 1 -- 172.21.15.100:0/4133800063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e2c0c9010 con 0x7f9e2c0b9bc0 2024-09-17T04:33:12.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.066+0000 7f9e30c00640 1 -- 172.21.15.100:0/4133800063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e2c0cef80 con 0x7f9e2c0b9bc0 2024-09-17T04:33:12.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.067+0000 7f9e23e00640 1 -- 172.21.15.100:0/4133800063 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f9e1401a070 con 0x7f9e2c0b9bc0 2024-09-17T04:33:12.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.068+0000 7f9e23e00640 1 --2- 172.21.15.100:0/4133800063 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9e08079430 0x7f9e0807b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:12.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.069+0000 7f9e23e00640 1 -- 172.21.15.100:0/4133800063 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4913+0+0 (secure 0 0 0) 0x7f9e14099690 con 0x7f9e2c0b9bc0 2024-09-17T04:33:12.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.069+0000 7f9e2b400640 1 --2- 172.21.15.100:0/4133800063 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9e08079430 0x7f9e0807b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:12.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.069+0000 7f9e2b400640 1 --2- 172.21.15.100:0/4133800063 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9e08079430 0x7f9e0807b940 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c033000 tx=0x7f9e1c004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:12.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.073+0000 7f9e23e00640 1 -- 172.21.15.100:0/4133800063 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9e14063500 con 0x7f9e2c0b9bc0 2024-09-17T04:33:12.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.257+0000 7f9e30c00640 1 -- 172.21.15.100:0/4133800063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f9e2c0b6e50 con 0x7f9e2c0b9bc0 2024-09-17T04:33:12.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.859+0000 7f9e23e00640 1 -- 172.21.15.100:0/4133800063 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v153) ==== 138+0+0 (secure 0 0 0) 0x7f9e14062e00 con 0x7f9e2c0b9bc0 2024-09-17T04:33:12.859 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' created 2024-09-17T04:33:12.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.862+0000 7f9e30c00640 1 -- 172.21.15.100:0/4133800063 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9e08079430 msgr2=0x7f9e0807b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:12.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.862+0000 7f9e30c00640 1 --2- 172.21.15.100:0/4133800063 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9e08079430 0x7f9e0807b940 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c033000 tx=0x7f9e1c004730 comp rx=0 tx=0).stop 2024-09-17T04:33:12.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.862+0000 7f9e30c00640 1 -- 172.21.15.100:0/4133800063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e2c0b9bc0 msgr2=0x7f9e2c0cd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:12.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.862+0000 7f9e30c00640 1 --2- 172.21.15.100:0/4133800063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e2c0b9bc0 0x7f9e2c0cd340 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f9e1400d720 tx=0x7f9e1400dbe0 comp rx=0 tx=0).stop 2024-09-17T04:33:12.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.862+0000 7f9e30c00640 1 -- 172.21.15.100:0/4133800063 shutdown_connections 2024-09-17T04:33:12.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.862+0000 7f9e30c00640 1 --2- 172.21.15.100:0/4133800063 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9e08079430 0x7f9e0807b940 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:12.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.862+0000 7f9e30c00640 1 --2- 172.21.15.100:0/4133800063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e2c0ceb70 0x7f9e2c0cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:12.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.862+0000 7f9e30c00640 1 --2- 172.21.15.100:0/4133800063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e2c0b9bc0 0x7f9e2c0cd340 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:12.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.862+0000 7f9e30c00640 1 --2- 172.21.15.100:0/4133800063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e2c0b6700 0x7f9e2c1592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:12.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.862+0000 7f9e30c00640 1 -- 172.21.15.100:0/4133800063 >> 172.21.15.100:0/4133800063 conn(0x7f9e2c08bfe0 msgr2=0x7f9e2c0b3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:12.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.862+0000 7f9e30c00640 1 -- 172.21.15.100:0/4133800063 shutdown_connections 2024-09-17T04:33:12.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:12.862+0000 7f9e30c00640 1 -- 172.21.15.100:0/4133800063 wait complete. 2024-09-17T04:33:12.894 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T04:33:13.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.121+0000 7f8745a00640 1 -- 172.21.15.100:0/3455374329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8740145a40 msgr2=0x7f8740073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:13.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.121+0000 7f8745a00640 1 --2- 172.21.15.100:0/3455374329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8740145a40 0x7f8740073540 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f8728005fd0 tx=0x7f872802a670 comp rx=0 tx=0).stop 2024-09-17T04:33:13.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.121+0000 7f8745a00640 1 -- 172.21.15.100:0/3455374329 shutdown_connections 2024-09-17T04:33:13.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.121+0000 7f8745a00640 1 --2- 172.21.15.100:0/3455374329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8740145a40 0x7f8740073540 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:13.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.121+0000 7f8745a00640 1 --2- 172.21.15.100:0/3455374329 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8740145550 0x7f874006fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:13.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.121+0000 7f8745a00640 1 -- 172.21.15.100:0/3455374329 >> 172.21.15.100:0/3455374329 conn(0x7f874006d1c0 msgr2=0x7f874006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:13.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.121+0000 7f8745a00640 1 -- 172.21.15.100:0/3455374329 shutdown_connections 2024-09-17T04:33:13.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.121+0000 7f8745a00640 1 -- 172.21.15.100:0/3455374329 wait complete. 2024-09-17T04:33:13.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.123+0000 7f8745a00640 1 Processor -- start 2024-09-17T04:33:13.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.124+0000 7f8745a00640 1 -- start start 2024-09-17T04:33:13.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.125+0000 7f8745a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8740073a80 0x7f874015d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:13.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.125+0000 7f8745a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8740145550 0x7f87401f4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:13.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.125+0000 7f8745a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8740145a40 0x7f874015b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:13.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.125+0000 7f8745a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8740160a30 con 0x7f8740073a80 2024-09-17T04:33:13.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.125+0000 7f8745a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f87400ce940 con 0x7f8740145550 2024-09-17T04:33:13.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.125+0000 7f8745a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8740160bb0 con 0x7f8740145a40 2024-09-17T04:33:13.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.125+0000 7f873ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8740073a80 0x7f874015d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:13.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.125+0000 7f873ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8740073a80 0x7f874015d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55538/0 (socket says 172.21.15.100:55538) 2024-09-17T04:33:13.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.125+0000 7f873ea00640 1 -- 172.21.15.100:0/313683920 learned_addr learned my addr 172.21.15.100:0/313683920 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:13.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.126+0000 7f873f400640 1 --2- 172.21.15.100:0/313683920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8740145a40 0x7f874015b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:13.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.126+0000 7f8736000640 1 --2- 172.21.15.100:0/313683920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8740145550 0x7f87401f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:13.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.126+0000 7f873ea00640 1 -- 172.21.15.100:0/313683920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8740145a40 msgr2=0x7f874015b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:13.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.126+0000 7f873ea00640 1 --2- 172.21.15.100:0/313683920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8740145a40 0x7f874015b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:13.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.126+0000 7f873ea00640 1 -- 172.21.15.100:0/313683920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8740145550 msgr2=0x7f87401f4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:13.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.126+0000 7f873ea00640 1 --2- 172.21.15.100:0/313683920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8740145550 0x7f87401f4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:13.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.126+0000 7f873ea00640 1 -- 172.21.15.100:0/313683920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f874015e560 con 0x7f8740073a80 2024-09-17T04:33:13.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.126+0000 7f873f400640 1 --2- 172.21.15.100:0/313683920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8740145a40 0x7f874015b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:13.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.126+0000 7f8736000640 1 --2- 172.21.15.100:0/313683920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8740145550 0x7f87401f4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:13.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.126+0000 7f873ea00640 1 --2- 172.21.15.100:0/313683920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8740073a80 0x7f874015d840 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f873000d6d0 tx=0x7f873000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:13.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.126+0000 7f8737400640 1 -- 172.21.15.100:0/313683920 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8730017070 con 0x7f8740073a80 2024-09-17T04:33:13.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.126+0000 7f8737400640 1 -- 172.21.15.100:0/313683920 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8730005330 con 0x7f8740073a80 2024-09-17T04:33:13.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.127+0000 7f8745a00640 1 -- 172.21.15.100:0/313683920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f874015e840 con 0x7f8740073a80 2024-09-17T04:33:13.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.127+0000 7f8737400640 1 -- 172.21.15.100:0/313683920 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8730009820 con 0x7f8740073a80 2024-09-17T04:33:13.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.127+0000 7f8745a00640 1 -- 172.21.15.100:0/313683920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f874015d040 con 0x7f8740073a80 2024-09-17T04:33:13.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.128+0000 7f8745a00640 1 -- 172.21.15.100:0/313683920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87401610d0 con 0x7f8740073a80 2024-09-17T04:33:13.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.129+0000 7f8737400640 1 -- 172.21.15.100:0/313683920 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f8730009bd0 con 0x7f8740073a80 2024-09-17T04:33:13.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.130+0000 7f8737400640 1 --2- 172.21.15.100:0/313683920 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f871c079400 0x7f871c07b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:13.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.130+0000 7f8737400640 1 -- 172.21.15.100:0/313683920 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5283+0+0 (secure 0 0 0) 0x7f8730099d90 con 0x7f8740073a80 2024-09-17T04:33:13.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.131+0000 7f8736000640 1 --2- 172.21.15.100:0/313683920 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f871c079400 0x7f871c07b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:13.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.131+0000 7f8736000640 1 --2- 172.21.15.100:0/313683920 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f871c079400 0x7f871c07b910 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8728033000 tx=0x7f8728004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:13.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.134+0000 7f8737400640 1 -- 172.21.15.100:0/313683920 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8730063c60 con 0x7f8740073a80 2024-09-17T04:33:13.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.320+0000 7f8745a00640 1 -- 172.21.15.100:0/313683920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8740144160 con 0x7f8740073a80 2024-09-17T04:33:13.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.866+0000 7f8737400640 1 -- 172.21.15.100:0/313683920 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v154) ==== 170+0+0 (secure 0 0 0) 0x7f8730063560 con 0x7f8740073a80 2024-09-17T04:33:13.867 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' created 2024-09-17T04:33:13.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.869+0000 7f8745a00640 1 -- 172.21.15.100:0/313683920 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f871c079400 msgr2=0x7f871c07b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:13.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.869+0000 7f8745a00640 1 --2- 172.21.15.100:0/313683920 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f871c079400 0x7f871c07b910 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8728033000 tx=0x7f8728004730 comp rx=0 tx=0).stop 2024-09-17T04:33:13.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.869+0000 7f8745a00640 1 -- 172.21.15.100:0/313683920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8740073a80 msgr2=0x7f874015d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:13.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.869+0000 7f8745a00640 1 --2- 172.21.15.100:0/313683920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8740073a80 0x7f874015d840 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f873000d6d0 tx=0x7f873000db90 comp rx=0 tx=0).stop 2024-09-17T04:33:13.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.869+0000 7f8745a00640 1 -- 172.21.15.100:0/313683920 shutdown_connections 2024-09-17T04:33:13.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.869+0000 7f8745a00640 1 --2- 172.21.15.100:0/313683920 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f871c079400 0x7f871c07b910 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:13.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.869+0000 7f8745a00640 1 --2- 172.21.15.100:0/313683920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8740145a40 0x7f874015b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:13.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.869+0000 7f8745a00640 1 --2- 172.21.15.100:0/313683920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8740145550 0x7f87401f4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:13.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.869+0000 7f8745a00640 1 --2- 172.21.15.100:0/313683920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8740073a80 0x7f874015d840 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:13.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.869+0000 7f8745a00640 1 -- 172.21.15.100:0/313683920 >> 172.21.15.100:0/313683920 conn(0x7f874006d1c0 msgr2=0x7f874015ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:13.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.869+0000 7f8745a00640 1 -- 172.21.15.100:0/313683920 shutdown_connections 2024-09-17T04:33:13.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:13.869+0000 7f8745a00640 1 -- 172.21.15.100:0/313683920 wait complete. 2024-09-17T04:33:13.908 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T04:33:14.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.129+0000 7f6437e00640 1 -- 172.21.15.100:0/880260393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f643815d6e0 msgr2=0x7f64380ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:14.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.129+0000 7f6437e00640 1 --2- 172.21.15.100:0/880260393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f643815d6e0 0x7f64380ccac0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f6438075230 tx=0x7f642002a5f0 comp rx=0 tx=0).stop 2024-09-17T04:33:14.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.129+0000 7f6437e00640 1 -- 172.21.15.100:0/880260393 shutdown_connections 2024-09-17T04:33:14.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.129+0000 7f6437e00640 1 --2- 172.21.15.100:0/880260393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f643815d6e0 0x7f64380ccac0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:14.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.129+0000 7f6437e00640 1 --2- 172.21.15.100:0/880260393 >> v2:172.21.15.184:3300/0 conn(0x7f64380b3f30 0x7f6438156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:14.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.129+0000 7f6437e00640 1 -- 172.21.15.100:0/880260393 >> 172.21.15.100:0/880260393 conn(0x7f643806d620 msgr2=0x7f64380b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:14.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.129+0000 7f6437e00640 1 -- 172.21.15.100:0/880260393 shutdown_connections 2024-09-17T04:33:14.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.129+0000 7f6437e00640 1 -- 172.21.15.100:0/880260393 wait complete. 2024-09-17T04:33:14.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.132+0000 7f6437e00640 1 Processor -- start 2024-09-17T04:33:14.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.132+0000 7f6437e00640 1 -- start start 2024-09-17T04:33:14.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.133+0000 7f6437e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64380b3f30 0x7f6438156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:14.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.133+0000 7f6437e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64380cd000 0x7f64381eaa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:14.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.133+0000 7f6437e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f643815d6e0 0x7f64381eaf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:14.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.133+0000 7f6437e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f64380af960 con 0x7f64380cd000 2024-09-17T04:33:14.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.133+0000 7f6437e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f64380af7e0 con 0x7f64380b3f30 2024-09-17T04:33:14.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.133+0000 7f6437e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f64380afae0 con 0x7f643815d6e0 2024-09-17T04:33:14.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.134+0000 7f642fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64380cd000 0x7f64381eaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:14.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.134+0000 7f642fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64380cd000 0x7f64381eaa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55552/0 (socket says 172.21.15.100:55552) 2024-09-17T04:33:14.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.134+0000 7f642fe00640 1 -- 172.21.15.100:0/2293176824 learned_addr learned my addr 172.21.15.100:0/2293176824 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:14.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.134+0000 7f6436a00640 1 --2- 172.21.15.100:0/2293176824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64380b3f30 0x7f6438156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:14.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.134+0000 7f6437400640 1 --2- 172.21.15.100:0/2293176824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f643815d6e0 0x7f64381eaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:14.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.134+0000 7f642fe00640 1 -- 172.21.15.100:0/2293176824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f643815d6e0 msgr2=0x7f64381eaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:14.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.134+0000 7f642fe00640 1 --2- 172.21.15.100:0/2293176824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f643815d6e0 0x7f64381eaf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:14.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.134+0000 7f642fe00640 1 -- 172.21.15.100:0/2293176824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64380b3f30 msgr2=0x7f6438156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:14.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.134+0000 7f642fe00640 1 --2- 172.21.15.100:0/2293176824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64380b3f30 0x7f6438156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:14.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.134+0000 7f642fe00640 1 -- 172.21.15.100:0/2293176824 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6438143fb0 con 0x7f64380cd000 2024-09-17T04:33:14.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.134+0000 7f642fe00640 1 --2- 172.21.15.100:0/2293176824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64380cd000 0x7f64381eaa30 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f6420034040 tx=0x7f6420003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:14.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.135+0000 7f642ea00640 1 -- 172.21.15.100:0/2293176824 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642003e030 con 0x7f64380cd000 2024-09-17T04:33:14.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.135+0000 7f642ea00640 1 -- 172.21.15.100:0/2293176824 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6420003d10 con 0x7f64380cd000 2024-09-17T04:33:14.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.135+0000 7f6437e00640 1 -- 172.21.15.100:0/2293176824 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6438144290 con 0x7f64380cd000 2024-09-17T04:33:14.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.135+0000 7f642ea00640 1 -- 172.21.15.100:0/2293176824 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6420035040 con 0x7f64380cd000 2024-09-17T04:33:14.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.135+0000 7f6437e00640 1 -- 172.21.15.100:0/2293176824 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6438147a90 con 0x7f64380cd000 2024-09-17T04:33:14.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.136+0000 7f6437e00640 1 -- 172.21.15.100:0/2293176824 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6438143d20 con 0x7f64380cd000 2024-09-17T04:33:14.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.137+0000 7f642ea00640 1 -- 172.21.15.100:0/2293176824 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f6420041030 con 0x7f64380cd000 2024-09-17T04:33:14.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.138+0000 7f642ea00640 1 --2- 172.21.15.100:0/2293176824 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6414079430 0x7f641407b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:14.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.138+0000 7f642ea00640 1 -- 172.21.15.100:0/2293176824 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5665+0+0 (secure 0 0 0) 0x7f64200bc660 con 0x7f64380cd000 2024-09-17T04:33:14.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.139+0000 7f6436a00640 1 --2- 172.21.15.100:0/2293176824 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6414079430 0x7f641407b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:14.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.139+0000 7f6436a00640 1 --2- 172.21.15.100:0/2293176824 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6414079430 0x7f641407b940 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6428009b30 tx=0x7f6428005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:14.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.142+0000 7f642ea00640 1 -- 172.21.15.100:0/2293176824 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6420083a20 con 0x7f64380cd000 2024-09-17T04:33:14.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.361+0000 7f6437e00640 1 -- 172.21.15.100:0/2293176824 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6438156ca0 con 0x7f64380cd000 2024-09-17T04:33:14.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.868+0000 7f642ea00640 1 -- 172.21.15.100:0/2293176824 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v107) ==== 174+0+0 (secure 0 0 0) 0x7f6420088450 con 0x7f64380cd000 2024-09-17T04:33:14.870 INFO:teuthology.orchestra.run.smithi100.stderr:new fs with metadata pool 24 and data pool 25 2024-09-17T04:33:14.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.872+0000 7f6437e00640 1 -- 172.21.15.100:0/2293176824 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6414079430 msgr2=0x7f641407b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:14.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.872+0000 7f6437e00640 1 --2- 172.21.15.100:0/2293176824 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6414079430 0x7f641407b940 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6428009b30 tx=0x7f6428005f90 comp rx=0 tx=0).stop 2024-09-17T04:33:14.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.872+0000 7f6437e00640 1 -- 172.21.15.100:0/2293176824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64380cd000 msgr2=0x7f64381eaa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:14.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.872+0000 7f6437e00640 1 --2- 172.21.15.100:0/2293176824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64380cd000 0x7f64381eaa30 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f6420034040 tx=0x7f6420003820 comp rx=0 tx=0).stop 2024-09-17T04:33:14.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.873+0000 7f6437e00640 1 -- 172.21.15.100:0/2293176824 shutdown_connections 2024-09-17T04:33:14.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.873+0000 7f6437e00640 1 --2- 172.21.15.100:0/2293176824 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6414079430 0x7f641407b940 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:14.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.873+0000 7f6437e00640 1 --2- 172.21.15.100:0/2293176824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f643815d6e0 0x7f64381eaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:14.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.873+0000 7f6437e00640 1 --2- 172.21.15.100:0/2293176824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64380cd000 0x7f64381eaa30 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:14.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.873+0000 7f6437e00640 1 --2- 172.21.15.100:0/2293176824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64380b3f30 0x7f6438156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:14.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.873+0000 7f6437e00640 1 -- 172.21.15.100:0/2293176824 >> 172.21.15.100:0/2293176824 conn(0x7f643806d620 msgr2=0x7f64380b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:14.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.873+0000 7f6437e00640 1 -- 172.21.15.100:0/2293176824 shutdown_connections 2024-09-17T04:33:14.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:14.874+0000 7f6437e00640 1 -- 172.21.15.100:0/2293176824 wait complete. 2024-09-17T04:33:14.921 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:33:15.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.141+0000 7f21fb600640 1 -- 172.21.15.100:0/1254298682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21f40ad0e0 msgr2=0x7f21f40ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:15.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.141+0000 7f21fb600640 1 --2- 172.21.15.100:0/1254298682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21f40ad0e0 0x7f21f40ad4f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f21dc005fd0 tx=0x7f21dc02a540 comp rx=0 tx=0).stop 2024-09-17T04:33:15.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.141+0000 7f21fb600640 1 -- 172.21.15.100:0/1254298682 shutdown_connections 2024-09-17T04:33:15.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.141+0000 7f21fb600640 1 --2- 172.21.15.100:0/1254298682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21f40ad0e0 0x7f21f40ad4f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:15.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.141+0000 7f21fb600640 1 --2- 172.21.15.100:0/1254298682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21f415d760 0x7f21f41535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:15.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.141+0000 7f21fb600640 1 -- 172.21.15.100:0/1254298682 >> 172.21.15.100:0/1254298682 conn(0x7f21f406d730 msgr2=0x7f21f40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:15.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.142+0000 7f21fb600640 1 -- 172.21.15.100:0/1254298682 shutdown_connections 2024-09-17T04:33:15.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.142+0000 7f21fb600640 1 -- 172.21.15.100:0/1254298682 wait complete. 2024-09-17T04:33:15.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.144+0000 7f21fb600640 1 Processor -- start 2024-09-17T04:33:15.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.144+0000 7f21fb600640 1 -- start start 2024-09-17T04:33:15.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.145+0000 7f21fb600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21f40ad0e0 0x7f21f4099910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:15.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.145+0000 7f21fb600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21f415d760 0x7f21f41f6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:15.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.145+0000 7f21fb600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21f4160310 0x7f21f41f7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:15.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.145+0000 7f21fb600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f21f406f220 con 0x7f21f415d760 2024-09-17T04:33:15.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.145+0000 7f21fb600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f21f406f0a0 con 0x7f21f4160310 2024-09-17T04:33:15.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.145+0000 7f21fb600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f21f40b4c20 con 0x7f21f40ad0e0 2024-09-17T04:33:15.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.146+0000 7f21ebe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21f415d760 0x7f21f41f6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:15.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.146+0000 7f21ebe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21f415d760 0x7f21f41f6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42916/0 (socket says 172.21.15.100:42916) 2024-09-17T04:33:15.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.146+0000 7f21ebe00640 1 -- 172.21.15.100:0/2497396822 learned_addr learned my addr 172.21.15.100:0/2497396822 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:15.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.146+0000 7f21f8c00640 1 --2- 172.21.15.100:0/2497396822 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21f40ad0e0 0x7f21f4099910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:15.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.146+0000 7f21f9600640 1 --2- 172.21.15.100:0/2497396822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21f4160310 0x7f21f41f7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:15.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.146+0000 7f21ebe00640 1 -- 172.21.15.100:0/2497396822 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21f40ad0e0 msgr2=0x7f21f4099910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:15.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.146+0000 7f21ebe00640 1 --2- 172.21.15.100:0/2497396822 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21f40ad0e0 0x7f21f4099910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:15.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.146+0000 7f21ebe00640 1 -- 172.21.15.100:0/2497396822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21f4160310 msgr2=0x7f21f41f7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:15.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.146+0000 7f21ebe00640 1 --2- 172.21.15.100:0/2497396822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21f4160310 0x7f21f41f7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:15.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.146+0000 7f21ebe00640 1 -- 172.21.15.100:0/2497396822 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f41f7850 con 0x7f21f415d760 2024-09-17T04:33:15.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.147+0000 7f21ebe00640 1 --2- 172.21.15.100:0/2497396822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21f415d760 0x7f21f41f6dd0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f21dc034040 tx=0x7f21dc0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:15.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.147+0000 7f21f1600640 1 -- 172.21.15.100:0/2497396822 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc03e030 con 0x7f21f415d760 2024-09-17T04:33:15.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.147+0000 7f21f1600640 1 -- 172.21.15.100:0/2497396822 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21dc003d90 con 0x7f21f415d760 2024-09-17T04:33:15.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.147+0000 7f21fb600640 1 -- 172.21.15.100:0/2497396822 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f41f7cc0 con 0x7f21f415d760 2024-09-17T04:33:15.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.147+0000 7f21f1600640 1 -- 172.21.15.100:0/2497396822 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc035040 con 0x7f21f415d760 2024-09-17T04:33:15.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.147+0000 7f21fb600640 1 -- 172.21.15.100:0/2497396822 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f41541c0 con 0x7f21f415d760 2024-09-17T04:33:15.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.148+0000 7f21fb600640 1 -- 172.21.15.100:0/2497396822 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21f41f7a20 con 0x7f21f415d760 2024-09-17T04:33:15.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.149+0000 7f21f1600640 1 -- 172.21.15.100:0/2497396822 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f21dc041030 con 0x7f21f415d760 2024-09-17T04:33:15.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.150+0000 7f21f1600640 1 --2- 172.21.15.100:0/2497396822 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f21d0079430 0x7f21d007b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:15.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.150+0000 7f21f1600640 1 -- 172.21.15.100:0/2497396822 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5765+0+0 (secure 0 0 0) 0x7f21dc0b9ed0 con 0x7f21f415d760 2024-09-17T04:33:15.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.150+0000 7f21f8c00640 1 --2- 172.21.15.100:0/2497396822 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f21d0079430 0x7f21d007b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:15.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.151+0000 7f21f8c00640 1 --2- 172.21.15.100:0/2497396822 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f21d0079430 0x7f21d007b940 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009b30 tx=0x7f21e4005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:15.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.155+0000 7f21f1600640 1 -- 172.21.15.100:0/2497396822 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f21f41f7a20 con 0x7f21f415d760 2024-09-17T04:33:15.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.340+0000 7f21fb600640 1 -- 172.21.15.100:0/2497396822 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21f4160fa0 con 0x7f21f415d760 2024-09-17T04:33:15.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.342+0000 7f21f1600640 1 -- 172.21.15.100:0/2497396822 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) ==== 74+0+16889 (secure 0 0 0) 0x7f21dc083800 con 0x7f21f415d760 2024-09-17T04:33:15.342 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:15.343 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":155,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:33:14.363149+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:33:12.271566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:13.334445+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:33:15.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.344+0000 7f21fb600640 1 -- 172.21.15.100:0/2497396822 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f21d0079430 msgr2=0x7f21d007b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:15.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.344+0000 7f21fb600640 1 --2- 172.21.15.100:0/2497396822 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f21d0079430 0x7f21d007b940 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009b30 tx=0x7f21e4005f90 comp rx=0 tx=0).stop 2024-09-17T04:33:15.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.344+0000 7f21fb600640 1 -- 172.21.15.100:0/2497396822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21f415d760 msgr2=0x7f21f41f6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:15.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.344+0000 7f21fb600640 1 --2- 172.21.15.100:0/2497396822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21f415d760 0x7f21f41f6dd0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f21dc034040 tx=0x7f21dc0038a0 comp rx=0 tx=0).stop 2024-09-17T04:33:15.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.344+0000 7f21fb600640 1 -- 172.21.15.100:0/2497396822 shutdown_connections 2024-09-17T04:33:15.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.344+0000 7f21fb600640 1 --2- 172.21.15.100:0/2497396822 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f21d0079430 0x7f21d007b940 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:15.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.344+0000 7f21fb600640 1 --2- 172.21.15.100:0/2497396822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21f4160310 0x7f21f41f7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:15.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.344+0000 7f21fb600640 1 --2- 172.21.15.100:0/2497396822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21f415d760 0x7f21f41f6dd0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:15.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.344+0000 7f21fb600640 1 --2- 172.21.15.100:0/2497396822 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21f40ad0e0 0x7f21f4099910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:15.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.344+0000 7f21fb600640 1 -- 172.21.15.100:0/2497396822 >> 172.21.15.100:0/2497396822 conn(0x7f21f406d730 msgr2=0x7f21f414eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:15.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.344+0000 7f21fb600640 1 -- 172.21.15.100:0/2497396822 shutdown_connections 2024-09-17T04:33:15.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.344+0000 7f21fb600640 1 -- 172.21.15.100:0/2497396822 wait complete. 2024-09-17T04:33:15.370 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:33:15.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.589+0000 7f52bd600640 1 -- 172.21.15.100:0/2973748898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52b80ad0e0 msgr2=0x7f52b80ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:15.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.589+0000 7f52bd600640 1 --2- 172.21.15.100:0/2973748898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52b80ad0e0 0x7f52b80ad4f0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f52b8075250 tx=0x7f52a402a4d0 comp rx=0 tx=0).stop 2024-09-17T04:33:15.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.589+0000 7f52bd600640 1 -- 172.21.15.100:0/2973748898 shutdown_connections 2024-09-17T04:33:15.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.589+0000 7f52bd600640 1 --2- 172.21.15.100:0/2973748898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52b80ad0e0 0x7f52b80ad4f0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:15.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.589+0000 7f52bd600640 1 --2- 172.21.15.100:0/2973748898 >> v2:172.21.15.184:3300/0 conn(0x7f52b815d760 0x7f52b81535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:15.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.589+0000 7f52bd600640 1 -- 172.21.15.100:0/2973748898 >> 172.21.15.100:0/2973748898 conn(0x7f52b806d730 msgr2=0x7f52b80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:15.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.589+0000 7f52bd600640 1 -- 172.21.15.100:0/2973748898 shutdown_connections 2024-09-17T04:33:15.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.589+0000 7f52bd600640 1 -- 172.21.15.100:0/2973748898 wait complete. 2024-09-17T04:33:15.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.591+0000 7f52bd600640 1 Processor -- start 2024-09-17T04:33:15.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.592+0000 7f52bd600640 1 -- start start 2024-09-17T04:33:15.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.593+0000 7f52bd600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52b80ad0e0 0x7f52b81f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:15.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.593+0000 7f52bd600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52b815d760 0x7f52b814f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:15.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.593+0000 7f52bd600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52b8160310 0x7f52b80b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:15.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.593+0000 7f52bd600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f52b80b4c20 con 0x7f52b80ad0e0 2024-09-17T04:33:15.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.593+0000 7f52bd600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f52b8153c70 con 0x7f52b815d760 2024-09-17T04:33:15.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.593+0000 7f52bd600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f52b80b4da0 con 0x7f52b8160310 2024-09-17T04:33:15.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.593+0000 7f52b6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52b80ad0e0 0x7f52b81f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:15.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.593+0000 7f52b6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52b80ad0e0 0x7f52b81f7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42926/0 (socket says 172.21.15.100:42926) 2024-09-17T04:33:15.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.593+0000 7f52b6a00640 1 -- 172.21.15.100:0/1173902360 learned_addr learned my addr 172.21.15.100:0/1173902360 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:15.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.594+0000 7f52b6000640 1 --2- 172.21.15.100:0/1173902360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52b815d760 0x7f52b814f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:15.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.594+0000 7f52b7400640 1 --2- 172.21.15.100:0/1173902360 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52b8160310 0x7f52b80b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:15.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.594+0000 7f52b6a00640 1 -- 172.21.15.100:0/1173902360 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52b8160310 msgr2=0x7f52b80b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:15.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.594+0000 7f52b6a00640 1 --2- 172.21.15.100:0/1173902360 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52b8160310 0x7f52b80b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:15.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.594+0000 7f52b6a00640 1 -- 172.21.15.100:0/1173902360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52b815d760 msgr2=0x7f52b814f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:15.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.594+0000 7f52b6a00640 1 --2- 172.21.15.100:0/1173902360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52b815d760 0x7f52b814f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:15.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.594+0000 7f52b6a00640 1 -- 172.21.15.100:0/1173902360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b81f4090 con 0x7f52b80ad0e0 2024-09-17T04:33:15.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.594+0000 7f52b6000640 1 --2- 172.21.15.100:0/1173902360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52b815d760 0x7f52b814f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:15.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.594+0000 7f52b6a00640 1 --2- 172.21.15.100:0/1173902360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52b80ad0e0 0x7f52b81f7e70 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f52a800d770 tx=0x7f52a800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:15.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.594+0000 7f52aea00640 1 -- 172.21.15.100:0/1173902360 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8017070 con 0x7f52b80ad0e0 2024-09-17T04:33:15.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.594+0000 7f52aea00640 1 -- 172.21.15.100:0/1173902360 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52a8009d20 con 0x7f52b80ad0e0 2024-09-17T04:33:15.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.594+0000 7f52bd600640 1 -- 172.21.15.100:0/1173902360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b81f4370 con 0x7f52b80ad0e0 2024-09-17T04:33:15.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.594+0000 7f52aea00640 1 -- 172.21.15.100:0/1173902360 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a80106c0 con 0x7f52b80ad0e0 2024-09-17T04:33:15.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.595+0000 7f52bd600640 1 -- 172.21.15.100:0/1173902360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b81f4930 con 0x7f52b80ad0e0 2024-09-17T04:33:15.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.595+0000 7f52bd600640 1 -- 172.21.15.100:0/1173902360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52b8161010 con 0x7f52b80ad0e0 2024-09-17T04:33:15.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.597+0000 7f52aea00640 1 -- 172.21.15.100:0/1173902360 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f52a801a070 con 0x7f52b80ad0e0 2024-09-17T04:33:15.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.598+0000 7f52aea00640 1 --2- 172.21.15.100:0/1173902360 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5298079400 0x7f529807b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:15.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.598+0000 7f52aea00640 1 -- 172.21.15.100:0/1173902360 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5765+0+0 (secure 0 0 0) 0x7f52a8099660 con 0x7f52b80ad0e0 2024-09-17T04:33:15.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.598+0000 7f52b6000640 1 --2- 172.21.15.100:0/1173902360 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5298079400 0x7f529807b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:15.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.599+0000 7f52b6000640 1 --2- 172.21.15.100:0/1173902360 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5298079400 0x7f529807b910 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f52a4033000 tx=0x7f52a4004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:15.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.602+0000 7f52aea00640 1 -- 172.21.15.100:0/1173902360 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f52a80635e0 con 0x7f52b80ad0e0 2024-09-17T04:33:15.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.788+0000 7f52bd600640 1 -- 172.21.15.100:0/1173902360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52b8069950 con 0x7f52b80ad0e0 2024-09-17T04:33:15.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.789+0000 7f52aea00640 1 -- 172.21.15.100:0/1173902360 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) ==== 74+0+16889 (secure 0 0 0) 0x7f52a8062ee0 con 0x7f52b80ad0e0 2024-09-17T04:33:15.790 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:15.790 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":155,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:33:14.363149+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:33:12.271566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:13.334445+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:33:15.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.791+0000 7f52bd600640 1 -- 172.21.15.100:0/1173902360 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5298079400 msgr2=0x7f529807b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:15.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.791+0000 7f52bd600640 1 --2- 172.21.15.100:0/1173902360 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5298079400 0x7f529807b910 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f52a4033000 tx=0x7f52a4004730 comp rx=0 tx=0).stop 2024-09-17T04:33:15.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.792+0000 7f52bd600640 1 -- 172.21.15.100:0/1173902360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52b80ad0e0 msgr2=0x7f52b81f7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:15.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.792+0000 7f52bd600640 1 --2- 172.21.15.100:0/1173902360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52b80ad0e0 0x7f52b81f7e70 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f52a800d770 tx=0x7f52a800dc30 comp rx=0 tx=0).stop 2024-09-17T04:33:15.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.792+0000 7f52bd600640 1 -- 172.21.15.100:0/1173902360 shutdown_connections 2024-09-17T04:33:15.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.792+0000 7f52bd600640 1 --2- 172.21.15.100:0/1173902360 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5298079400 0x7f529807b910 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:15.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.792+0000 7f52bd600640 1 --2- 172.21.15.100:0/1173902360 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52b8160310 0x7f52b80b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:15.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.792+0000 7f52bd600640 1 --2- 172.21.15.100:0/1173902360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52b815d760 0x7f52b814f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:15.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.792+0000 7f52bd600640 1 --2- 172.21.15.100:0/1173902360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52b80ad0e0 0x7f52b81f7e70 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:15.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.792+0000 7f52bd600640 1 -- 172.21.15.100:0/1173902360 >> 172.21.15.100:0/1173902360 conn(0x7f52b806d730 msgr2=0x7f52b80cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:15.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.792+0000 7f52bd600640 1 -- 172.21.15.100:0/1173902360 shutdown_connections 2024-09-17T04:33:15.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:15.792+0000 7f52bd600640 1 -- 172.21.15.100:0/1173902360 wait complete. 2024-09-17T04:33:15.814 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T04:33:16.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.041+0000 7fdb0d600640 1 -- 172.21.15.100:0/1970542940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdaec005250 msgr2=0x7fdaec00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:16.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.041+0000 7fdb0d600640 1 --2- 172.21.15.100:0/1970542940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdaec005250 0x7fdaec00b9c0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fdb080b0070 tx=0x7fdaec005830 comp rx=0 tx=0).stop 2024-09-17T04:33:16.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.041+0000 7fdb0d600640 1 -- 172.21.15.100:0/1970542940 shutdown_connections 2024-09-17T04:33:16.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.041+0000 7fdb0d600640 1 --2- 172.21.15.100:0/1970542940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdaec005250 0x7fdaec00b9c0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.041+0000 7fdb0d600640 1 -- 172.21.15.100:0/1970542940 >> 172.21.15.100:0/1970542940 conn(0x7fdb0806d1c0 msgr2=0x7fdb0806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:16.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.041+0000 7fdb0d600640 1 -- 172.21.15.100:0/1970542940 shutdown_connections 2024-09-17T04:33:16.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.041+0000 7fdb0d600640 1 -- 172.21.15.100:0/1970542940 wait complete. 2024-09-17T04:33:16.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.043+0000 7fdb0d600640 1 Processor -- start 2024-09-17T04:33:16.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.043+0000 7fdb0d600640 1 -- start start 2024-09-17T04:33:16.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.045+0000 7fdb0d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdaec005250 0x7fdb0812ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:16.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.045+0000 7fdb0d600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb0813f180 0x7fdb0813f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:16.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.045+0000 7fdb0d600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb0815b900 0x7fdb0815bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:16.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.045+0000 7fdb0d600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdb08073540 con 0x7fdaec005250 2024-09-17T04:33:16.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.045+0000 7fdb0d600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdb08070180 con 0x7fdb0815b900 2024-09-17T04:33:16.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.045+0000 7fdb0d600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdb080736c0 con 0x7fdb0813f180 2024-09-17T04:33:16.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.045+0000 7fdb06a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdaec005250 0x7fdb0812ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:16.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.045+0000 7fdb06a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdaec005250 0x7fdb0812ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42946/0 (socket says 172.21.15.100:42946) 2024-09-17T04:33:16.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.045+0000 7fdb06a00640 1 -- 172.21.15.100:0/1741678093 learned_addr learned my addr 172.21.15.100:0/1741678093 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:16.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.045+0000 7fdb06000640 1 --2- 172.21.15.100:0/1741678093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb0813f180 0x7fdb0813f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:16.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.045+0000 7fdb07400640 1 --2- 172.21.15.100:0/1741678093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb0815b900 0x7fdb0815bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:16.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.045+0000 7fdb06a00640 1 -- 172.21.15.100:0/1741678093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb0813f180 msgr2=0x7fdb0813f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:16.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.045+0000 7fdb06a00640 1 --2- 172.21.15.100:0/1741678093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb0813f180 0x7fdb0813f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.045+0000 7fdb06a00640 1 -- 172.21.15.100:0/1741678093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb0815b900 msgr2=0x7fdb0815bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:16.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.045+0000 7fdb06a00640 1 --2- 172.21.15.100:0/1741678093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb0815b900 0x7fdb0815bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.045+0000 7fdb06a00640 1 -- 172.21.15.100:0/1741678093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb081f6380 con 0x7fdaec005250 2024-09-17T04:33:16.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.045+0000 7fdb07400640 1 --2- 172.21.15.100:0/1741678093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb0815b900 0x7fdb0815bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:16.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.046+0000 7fdb06a00640 1 --2- 172.21.15.100:0/1741678093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdaec005250 0x7fdb0812ab60 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8011ab0 tx=0x7fdaf8011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:16.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.046+0000 7fdafea00640 1 -- 172.21.15.100:0/1741678093 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf801e070 con 0x7fdaec005250 2024-09-17T04:33:16.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.046+0000 7fdafea00640 1 -- 172.21.15.100:0/1741678093 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdaf800fe70 con 0x7fdaec005250 2024-09-17T04:33:16.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.046+0000 7fdafea00640 1 -- 172.21.15.100:0/1741678093 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf80042b0 con 0x7fdaec005250 2024-09-17T04:33:16.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.046+0000 7fdb0d600640 1 -- 172.21.15.100:0/1741678093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb08142930 con 0x7fdaec005250 2024-09-17T04:33:16.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.046+0000 7fdb0d600640 1 -- 172.21.15.100:0/1741678093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb08142090 con 0x7fdaec005250 2024-09-17T04:33:16.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.047+0000 7fdb0d600640 1 -- 172.21.15.100:0/1741678093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb08073c20 con 0x7fdaec005250 2024-09-17T04:33:16.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.048+0000 7fdafea00640 1 -- 172.21.15.100:0/1741678093 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7fdaf8021070 con 0x7fdaec005250 2024-09-17T04:33:16.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.049+0000 7fdafea00640 1 --2- 172.21.15.100:0/1741678093 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdae8079400 0x7fdae807b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:16.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.049+0000 7fdafea00640 1 -- 172.21.15.100:0/1741678093 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7fdaf809cb20 con 0x7fdaec005250 2024-09-17T04:33:16.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.049+0000 7fdb06000640 1 --2- 172.21.15.100:0/1741678093 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdae8079400 0x7fdae807b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:16.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.050+0000 7fdb06000640 1 --2- 172.21.15.100:0/1741678093 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdae8079400 0x7fdae807b910 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdb08141410 tx=0x7fdaf4006720 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:16.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.054+0000 7fdafea00640 1 -- 172.21.15.100:0/1741678093 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fdaf80638a0 con 0x7fdaec005250 2024-09-17T04:33:16.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.271+0000 7fdb0d600640 1 -- 172.21.15.100:0/1741678093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fdb080cc660 con 0x7fdaec005250 2024-09-17T04:33:16.509 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-09-17T04:33:16.509 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-09-17T04:33:16.850 INFO:tasks.ceph.mds.b.smithi184.stderr:2024-09-17T04:33:16.850+0000 7f86cfa00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T04:33:16.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.895+0000 7fdafea00640 1 -- 172.21.15.100:0/1741678093 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v110) ==== 118+0+0 (secure 0 0 0) 0x7fdaf80682d0 con 0x7fdaec005250 2024-09-17T04:33:16.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.897+0000 7fdb0d600640 1 -- 172.21.15.100:0/1741678093 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdae8079400 msgr2=0x7fdae807b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:16.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.897+0000 7fdb0d600640 1 --2- 172.21.15.100:0/1741678093 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdae8079400 0x7fdae807b910 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdb08141410 tx=0x7fdaf4006720 comp rx=0 tx=0).stop 2024-09-17T04:33:16.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.897+0000 7fdb0d600640 1 -- 172.21.15.100:0/1741678093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdaec005250 msgr2=0x7fdb0812ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:16.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.897+0000 7fdb0d600640 1 --2- 172.21.15.100:0/1741678093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdaec005250 0x7fdb0812ab60 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8011ab0 tx=0x7fdaf8011f70 comp rx=0 tx=0).stop 2024-09-17T04:33:16.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.898+0000 7fdb0d600640 1 -- 172.21.15.100:0/1741678093 shutdown_connections 2024-09-17T04:33:16.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.898+0000 7fdb0d600640 1 --2- 172.21.15.100:0/1741678093 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdae8079400 0x7fdae807b910 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.898+0000 7fdb0d600640 1 --2- 172.21.15.100:0/1741678093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb0815b900 0x7fdb0815bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.898+0000 7fdb0d600640 1 --2- 172.21.15.100:0/1741678093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb0813f180 0x7fdb0813f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.898+0000 7fdb0d600640 1 --2- 172.21.15.100:0/1741678093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdaec005250 0x7fdb0812ab60 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.898+0000 7fdb0d600640 1 -- 172.21.15.100:0/1741678093 >> 172.21.15.100:0/1741678093 conn(0x7fdb0806d1c0 msgr2=0x7fdb08140510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:16.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.898+0000 7fdb0d600640 1 -- 172.21.15.100:0/1741678093 shutdown_connections 2024-09-17T04:33:16.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:16.898+0000 7fdb0d600640 1 -- 172.21.15.100:0/1741678093 wait complete. 2024-09-17T04:33:16.921 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T04:33:16.922 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T04:33:17.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.144+0000 7fb817a00640 1 -- 172.21.15.100:0/577522915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb810145a40 msgr2=0x7fb810073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:17.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.144+0000 7fb817a00640 1 --2- 172.21.15.100:0/577522915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb810145a40 0x7fb810073540 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8005fd0 tx=0x7fb7f802a670 comp rx=0 tx=0).stop 2024-09-17T04:33:17.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.144+0000 7fb817a00640 1 -- 172.21.15.100:0/577522915 shutdown_connections 2024-09-17T04:33:17.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.144+0000 7fb817a00640 1 --2- 172.21.15.100:0/577522915 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb810073a80 0x7fb8100cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:17.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.144+0000 7fb817a00640 1 --2- 172.21.15.100:0/577522915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb810145a40 0x7fb810073540 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:17.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.144+0000 7fb817a00640 1 -- 172.21.15.100:0/577522915 >> 172.21.15.100:0/577522915 conn(0x7fb81006d1c0 msgr2=0x7fb81006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:17.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.144+0000 7fb817a00640 1 -- 172.21.15.100:0/577522915 shutdown_connections 2024-09-17T04:33:17.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.144+0000 7fb817a00640 1 -- 172.21.15.100:0/577522915 wait complete. 2024-09-17T04:33:17.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.146+0000 7fb817a00640 1 Processor -- start 2024-09-17T04:33:17.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.146+0000 7fb817a00640 1 -- start start 2024-09-17T04:33:17.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.148+0000 7fb817a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb810073a80 0x7fb81015e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:17.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.148+0000 7fb817a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb810145550 0x7fb8101e8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:17.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.148+0000 7fb817a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb810145a40 0x7fb8101e7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:17.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.148+0000 7fb817a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb8100ce140 con 0x7fb810145a40 2024-09-17T04:33:17.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.148+0000 7fb817a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8100cdfc0 con 0x7fb810145550 2024-09-17T04:33:17.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.148+0000 7fb817a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb8100ce2c0 con 0x7fb810073a80 2024-09-17T04:33:17.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.148+0000 7fb815a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb810145a40 0x7fb8101e7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:17.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.148+0000 7fb815a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb810145a40 0x7fb8101e7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42958/0 (socket says 172.21.15.100:42958) 2024-09-17T04:33:17.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.148+0000 7fb815a00640 1 -- 172.21.15.100:0/509817705 learned_addr learned my addr 172.21.15.100:0/509817705 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:17.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.148+0000 7fb815000640 1 --2- 172.21.15.100:0/509817705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb810073a80 0x7fb81015e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:17.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.148+0000 7fb807e00640 1 --2- 172.21.15.100:0/509817705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb810145550 0x7fb8101e8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:17.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.148+0000 7fb815a00640 1 -- 172.21.15.100:0/509817705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb810073a80 msgr2=0x7fb81015e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:17.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.148+0000 7fb815a00640 1 --2- 172.21.15.100:0/509817705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb810073a80 0x7fb81015e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:17.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.148+0000 7fb815a00640 1 -- 172.21.15.100:0/509817705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb810145550 msgr2=0x7fb8101e8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:17.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.148+0000 7fb815a00640 1 --2- 172.21.15.100:0/509817705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb810145550 0x7fb8101e8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:17.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.148+0000 7fb815a00640 1 -- 172.21.15.100:0/509817705 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb81015d4e0 con 0x7fb810145a40 2024-09-17T04:33:17.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.149+0000 7fb815a00640 1 --2- 172.21.15.100:0/509817705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb810145a40 0x7fb8101e7220 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fb80800e960 tx=0x7fb80800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:17.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.149+0000 7fb80d600640 1 -- 172.21.15.100:0/509817705 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb808004d20 con 0x7fb810145a40 2024-09-17T04:33:17.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.149+0000 7fb80d600640 1 -- 172.21.15.100:0/509817705 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb80800cac0 con 0x7fb810145a40 2024-09-17T04:33:17.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.149+0000 7fb80d600640 1 -- 172.21.15.100:0/509817705 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb808005020 con 0x7fb810145a40 2024-09-17T04:33:17.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.149+0000 7fb817a00640 1 -- 172.21.15.100:0/509817705 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb81015d7c0 con 0x7fb810145a40 2024-09-17T04:33:17.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.149+0000 7fb817a00640 1 -- 172.21.15.100:0/509817705 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb81015dd80 con 0x7fb810145a40 2024-09-17T04:33:17.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.150+0000 7fb817a00640 1 -- 172.21.15.100:0/509817705 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8100ce790 con 0x7fb810145a40 2024-09-17T04:33:17.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.152+0000 7fb80d600640 1 -- 172.21.15.100:0/509817705 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7fb8080051c0 con 0x7fb810145a40 2024-09-17T04:33:17.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.152+0000 7fb80d600640 1 --2- 172.21.15.100:0/509817705 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb7ec079430 0x7fb7ec07b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:17.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.153+0000 7fb80d600640 1 -- 172.21.15.100:0/509817705 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7fb808099e10 con 0x7fb810145a40 2024-09-17T04:33:17.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.153+0000 7fb815000640 1 --2- 172.21.15.100:0/509817705 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb7ec079430 0x7fb7ec07b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:17.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.153+0000 7fb815000640 1 --2- 172.21.15.100:0/509817705 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb7ec079430 0x7fb7ec07b940 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb800000f60 tx=0x7fb800009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:17.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.157+0000 7fb80d600640 1 -- 172.21.15.100:0/509817705 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb808016030 con 0x7fb810145a40 2024-09-17T04:33:17.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.376+0000 7fb817a00640 1 -- 172.21.15.100:0/509817705 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb81015d270 con 0x7fb810145a40 2024-09-17T04:33:17.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.903+0000 7fb80d600640 1 -- 172.21.15.100:0/509817705 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v111) ==== 122+0+0 (secure 0 0 0) 0x7fb8080636b0 con 0x7fb810145a40 2024-09-17T04:33:17.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.906+0000 7fb817a00640 1 -- 172.21.15.100:0/509817705 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb7ec079430 msgr2=0x7fb7ec07b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:17.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.906+0000 7fb817a00640 1 --2- 172.21.15.100:0/509817705 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb7ec079430 0x7fb7ec07b940 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb800000f60 tx=0x7fb800009040 comp rx=0 tx=0).stop 2024-09-17T04:33:17.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.906+0000 7fb817a00640 1 -- 172.21.15.100:0/509817705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb810145a40 msgr2=0x7fb8101e7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:17.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.906+0000 7fb817a00640 1 --2- 172.21.15.100:0/509817705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb810145a40 0x7fb8101e7220 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fb80800e960 tx=0x7fb80800ee20 comp rx=0 tx=0).stop 2024-09-17T04:33:17.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.906+0000 7fb817a00640 1 -- 172.21.15.100:0/509817705 shutdown_connections 2024-09-17T04:33:17.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.906+0000 7fb817a00640 1 --2- 172.21.15.100:0/509817705 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fb7ec079430 0x7fb7ec07b940 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:17.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.906+0000 7fb817a00640 1 --2- 172.21.15.100:0/509817705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb810145a40 0x7fb8101e7220 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:17.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.906+0000 7fb817a00640 1 --2- 172.21.15.100:0/509817705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb810145550 0x7fb8101e8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:17.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.906+0000 7fb817a00640 1 --2- 172.21.15.100:0/509817705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb810073a80 0x7fb81015e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:17.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.906+0000 7fb817a00640 1 -- 172.21.15.100:0/509817705 >> 172.21.15.100:0/509817705 conn(0x7fb81006d1c0 msgr2=0x7fb81013f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:17.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.906+0000 7fb817a00640 1 -- 172.21.15.100:0/509817705 shutdown_connections 2024-09-17T04:33:17.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:17.906+0000 7fb817a00640 1 -- 172.21.15.100:0/509817705 wait complete. 2024-09-17T04:33:17.934 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T04:33:17.935 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:33:18.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.154+0000 7f36df800640 1 -- 172.21.15.100:0/3016747170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36d80b3550 msgr2=0x7f36d80b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:18.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.154+0000 7f36df800640 1 --2- 172.21.15.100:0/3016747170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36d80b3550 0x7f36d80b3960 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f36d0005fd0 tx=0x7f36d002aa10 comp rx=0 tx=0).stop 2024-09-17T04:33:18.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.154+0000 7f36df800640 1 -- 172.21.15.100:0/3016747170 shutdown_connections 2024-09-17T04:33:18.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.154+0000 7f36df800640 1 --2- 172.21.15.100:0/3016747170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36d80b3550 0x7f36d80b3960 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:18.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.154+0000 7f36df800640 1 --2- 172.21.15.100:0/3016747170 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36d80ad0e0 0x7f36d80ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:18.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.154+0000 7f36df800640 1 -- 172.21.15.100:0/3016747170 >> 172.21.15.100:0/3016747170 conn(0x7f36d806d730 msgr2=0x7f36d80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:18.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.154+0000 7f36df800640 1 -- 172.21.15.100:0/3016747170 shutdown_connections 2024-09-17T04:33:18.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.155+0000 7f36df800640 1 -- 172.21.15.100:0/3016747170 wait complete. 2024-09-17T04:33:18.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.157+0000 7f36df800640 1 Processor -- start 2024-09-17T04:33:18.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.157+0000 7f36df800640 1 -- start start 2024-09-17T04:33:18.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.158+0000 7f36df800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36d80ad0e0 0x7f36d81f8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:18.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.158+0000 7f36df800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36d80b3550 0x7f36d814a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:18.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.158+0000 7f36df800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36d815d760 0x7f36d81f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:18.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.158+0000 7f36df800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f36d80b4da0 con 0x7f36d80b3550 2024-09-17T04:33:18.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.158+0000 7f36df800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f36d80b4c20 con 0x7f36d80ad0e0 2024-09-17T04:33:18.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.158+0000 7f36df800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f36d80b3ea0 con 0x7f36d815d760 2024-09-17T04:33:18.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.158+0000 7f36d7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36d80b3550 0x7f36d814a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:18.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.159+0000 7f36d7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36d80b3550 0x7f36d814a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42982/0 (socket says 172.21.15.100:42982) 2024-09-17T04:33:18.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.159+0000 7f36d7e00640 1 -- 172.21.15.100:0/599404620 learned_addr learned my addr 172.21.15.100:0/599404620 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:18.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.159+0000 7f36dd800640 1 --2- 172.21.15.100:0/599404620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36d815d760 0x7f36d81f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:18.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.159+0000 7f36dce00640 1 --2- 172.21.15.100:0/599404620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36d80ad0e0 0x7f36d81f8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:18.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.159+0000 7f36d7e00640 1 -- 172.21.15.100:0/599404620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36d815d760 msgr2=0x7f36d81f60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:18.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.159+0000 7f36d7e00640 1 --2- 172.21.15.100:0/599404620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36d815d760 0x7f36d81f60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:18.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.159+0000 7f36d7e00640 1 -- 172.21.15.100:0/599404620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36d80ad0e0 msgr2=0x7f36d81f8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:18.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.159+0000 7f36d7e00640 1 --2- 172.21.15.100:0/599404620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36d80ad0e0 0x7f36d81f8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:18.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.159+0000 7f36d7e00640 1 -- 172.21.15.100:0/599404620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d81f6600 con 0x7f36d80b3550 2024-09-17T04:33:18.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.159+0000 7f36dce00640 1 --2- 172.21.15.100:0/599404620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36d80ad0e0 0x7f36d81f8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:18.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.159+0000 7f36d7e00640 1 --2- 172.21.15.100:0/599404620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36d80b3550 0x7f36d814a8a0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f36c400d6d0 tx=0x7f36c400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:18.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.160+0000 7f36d4c00640 1 -- 172.21.15.100:0/599404620 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4017070 con 0x7f36d80b3550 2024-09-17T04:33:18.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.160+0000 7f36d4c00640 1 -- 172.21.15.100:0/599404620 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36c4004310 con 0x7f36d80b3550 2024-09-17T04:33:18.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.160+0000 7f36df800640 1 -- 172.21.15.100:0/599404620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36d81f9610 con 0x7f36d80b3550 2024-09-17T04:33:18.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.160+0000 7f36d4c00640 1 -- 172.21.15.100:0/599404620 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4009a80 con 0x7f36d80b3550 2024-09-17T04:33:18.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.160+0000 7f36df800640 1 -- 172.21.15.100:0/599404620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36d81f58c0 con 0x7f36d80b3550 2024-09-17T04:33:18.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.161+0000 7f36df800640 1 -- 172.21.15.100:0/599404620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36d81f9330 con 0x7f36d80b3550 2024-09-17T04:33:18.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.162+0000 7f36d4c00640 1 -- 172.21.15.100:0/599404620 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f36c40035f0 con 0x7f36d80b3550 2024-09-17T04:33:18.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.163+0000 7f36d4c00640 1 --2- 172.21.15.100:0/599404620 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f36b8079400 0x7f36b807b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:18.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.163+0000 7f36d4c00640 1 -- 172.21.15.100:0/599404620 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7f36c409c260 con 0x7f36d80b3550 2024-09-17T04:33:18.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.163+0000 7f36dce00640 1 --2- 172.21.15.100:0/599404620 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f36b8079400 0x7f36b807b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:18.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.164+0000 7f36dce00640 1 --2- 172.21.15.100:0/599404620 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f36b8079400 0x7f36b807b910 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f36cc007fd0 tx=0x7f36cc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:18.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.167+0000 7f36d4c00640 1 -- 172.21.15.100:0/599404620 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f36c4063070 con 0x7f36d80b3550 2024-09-17T04:33:18.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.385+0000 7f36df800640 1 -- 172.21.15.100:0/599404620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36d8152cb0 con 0x7f36d80b3550 2024-09-17T04:33:18.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.386+0000 7f36d4c00640 1 -- 172.21.15.100:0/599404620 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5245 (secure 0 0 0) 0x7f36c4067aa0 con 0x7f36d80b3550 2024-09-17T04:33:18.386 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:18.386 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":111,"btime":"2024-09-17T04:33:17:896778+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7295,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1131376012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1131376012},{"type":"v1","addr":"172.21.15.184:6801","nonce":1131376012}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":8230,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/3930702908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3930702908},{"type":"v1","addr":"172.21.15.100:6833","nonce":3930702908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":8808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2238795078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2238795078},{"type":"v1","addr":"172.21.15.100:6835","nonce":2238795078}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:14.363111+0000","modified":"2024-09-17T04:33:17.378067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8814":{"gid":8814,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/2965326201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2965326201},{"type":"v1","addr":"172.21.15.184:6835","nonce":2965326201}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8814,"qdb_cluster":[8814]},"id":12}]} 2024-09-17T04:33:18.386 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 111 2024-09-17T04:33:18.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.388+0000 7f36df800640 1 -- 172.21.15.100:0/599404620 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f36b8079400 msgr2=0x7f36b807b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:18.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.388+0000 7f36df800640 1 --2- 172.21.15.100:0/599404620 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f36b8079400 0x7f36b807b910 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f36cc007fd0 tx=0x7f36cc009040 comp rx=0 tx=0).stop 2024-09-17T04:33:18.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.388+0000 7f36df800640 1 -- 172.21.15.100:0/599404620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36d80b3550 msgr2=0x7f36d814a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:18.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.388+0000 7f36df800640 1 --2- 172.21.15.100:0/599404620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36d80b3550 0x7f36d814a8a0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f36c400d6d0 tx=0x7f36c400db90 comp rx=0 tx=0).stop 2024-09-17T04:33:18.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.388+0000 7f36df800640 1 -- 172.21.15.100:0/599404620 shutdown_connections 2024-09-17T04:33:18.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.388+0000 7f36df800640 1 --2- 172.21.15.100:0/599404620 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f36b8079400 0x7f36b807b910 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:18.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.388+0000 7f36df800640 1 --2- 172.21.15.100:0/599404620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36d815d760 0x7f36d81f60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:18.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.388+0000 7f36df800640 1 --2- 172.21.15.100:0/599404620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36d80b3550 0x7f36d814a8a0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:18.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.388+0000 7f36df800640 1 --2- 172.21.15.100:0/599404620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36d80ad0e0 0x7f36d81f8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:18.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.388+0000 7f36df800640 1 -- 172.21.15.100:0/599404620 >> 172.21.15.100:0/599404620 conn(0x7f36d806d730 msgr2=0x7f36d81535c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:18.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.388+0000 7f36df800640 1 -- 172.21.15.100:0/599404620 shutdown_connections 2024-09-17T04:33:18.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.388+0000 7f36df800640 1 -- 172.21.15.100:0/599404620 wait complete. 2024-09-17T04:33:18.411 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:33:18.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.630+0000 7f12aea00640 1 -- 172.21.15.100:0/3554034188 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12a8155370 msgr2=0x7f12a8155780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:18.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.630+0000 7f12aea00640 1 --2- 172.21.15.100:0/3554034188 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12a8155370 0x7f12a8155780 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f12a0005fd0 tx=0x7f12a002aa10 comp rx=0 tx=0).stop 2024-09-17T04:33:18.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.630+0000 7f12aea00640 1 -- 172.21.15.100:0/3554034188 shutdown_connections 2024-09-17T04:33:18.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.630+0000 7f12aea00640 1 --2- 172.21.15.100:0/3554034188 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12a8155370 0x7f12a8155780 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:18.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.630+0000 7f12aea00640 1 --2- 172.21.15.100:0/3554034188 >> v2:172.21.15.184:3300/0 conn(0x7f12a806f280 0x7f12a806f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:18.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.630+0000 7f12aea00640 1 -- 172.21.15.100:0/3554034188 >> 172.21.15.100:0/3554034188 conn(0x7f12a80b1e40 msgr2=0x7f12a80b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:18.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.630+0000 7f12aea00640 1 -- 172.21.15.100:0/3554034188 shutdown_connections 2024-09-17T04:33:18.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.630+0000 7f12aea00640 1 -- 172.21.15.100:0/3554034188 wait complete. 2024-09-17T04:33:18.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.632+0000 7f12aea00640 1 Processor -- start 2024-09-17T04:33:18.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.632+0000 7f12aea00640 1 -- start start 2024-09-17T04:33:18.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.633+0000 7f12aea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12a806f280 0x7f12a8144e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:18.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.634+0000 7f12aea00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12a8155370 0x7f12a8153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:18.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.634+0000 7f12aea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12a81623a0 0x7f12a8153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:18.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.634+0000 7f12aea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f12a80b3a10 con 0x7f12a806f280 2024-09-17T04:33:18.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.634+0000 7f12aea00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f12a80b3890 con 0x7f12a81623a0 2024-09-17T04:33:18.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.634+0000 7f12aea00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f12a80b3b90 con 0x7f12a8155370 2024-09-17T04:33:18.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.634+0000 7f12ad600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12a806f280 0x7f12a8144e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:18.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.634+0000 7f12ad600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12a806f280 0x7f12a8144e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42988/0 (socket says 172.21.15.100:42988) 2024-09-17T04:33:18.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.634+0000 7f12ad600640 1 -- 172.21.15.100:0/295126065 learned_addr learned my addr 172.21.15.100:0/295126065 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:18.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.634+0000 7f12acc00640 1 --2- 172.21.15.100:0/295126065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12a8155370 0x7f12a8153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:18.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.634+0000 7f12ae000640 1 --2- 172.21.15.100:0/295126065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12a81623a0 0x7f12a8153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:18.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.634+0000 7f12ad600640 1 -- 172.21.15.100:0/295126065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12a8155370 msgr2=0x7f12a8153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:18.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.634+0000 7f12ad600640 1 --2- 172.21.15.100:0/295126065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12a8155370 0x7f12a8153590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:18.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.634+0000 7f12ad600640 1 -- 172.21.15.100:0/295126065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12a81623a0 msgr2=0x7f12a8153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:18.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.634+0000 7f12ad600640 1 --2- 172.21.15.100:0/295126065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12a81623a0 0x7f12a8153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:18.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.634+0000 7f12ad600640 1 -- 172.21.15.100:0/295126065 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a8152160 con 0x7f12a806f280 2024-09-17T04:33:18.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.635+0000 7f12ad600640 1 --2- 172.21.15.100:0/295126065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12a806f280 0x7f12a8144e10 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f129c00e790 tx=0x7f129c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:18.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.635+0000 7f12a5600640 1 -- 172.21.15.100:0/295126065 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c00eea0 con 0x7f12a806f280 2024-09-17T04:33:18.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.635+0000 7f12a5600640 1 -- 172.21.15.100:0/295126065 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f129c00cd50 con 0x7f12a806f280 2024-09-17T04:33:18.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.635+0000 7f12a5600640 1 -- 172.21.15.100:0/295126065 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c010690 con 0x7f12a806f280 2024-09-17T04:33:18.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.635+0000 7f12aea00640 1 -- 172.21.15.100:0/295126065 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12a81568b0 con 0x7f12a806f280 2024-09-17T04:33:18.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.635+0000 7f12aea00640 1 -- 172.21.15.100:0/295126065 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12a81e8ce0 con 0x7f12a806f280 2024-09-17T04:33:18.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.636+0000 7f12aea00640 1 -- 172.21.15.100:0/295126065 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12a8142340 con 0x7f12a806f280 2024-09-17T04:33:18.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.637+0000 7f12a5600640 1 -- 172.21.15.100:0/295126065 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f129c010830 con 0x7f12a806f280 2024-09-17T04:33:18.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.638+0000 7f12a5600640 1 --2- 172.21.15.100:0/295126065 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1288079400 0x7f128807b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:18.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.638+0000 7f12a5600640 1 -- 172.21.15.100:0/295126065 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7f129c099f00 con 0x7f12a806f280 2024-09-17T04:33:18.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.639+0000 7f12acc00640 1 --2- 172.21.15.100:0/295126065 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1288079400 0x7f128807b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:18.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.639+0000 7f12acc00640 1 --2- 172.21.15.100:0/295126065 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1288079400 0x7f128807b910 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f12a81eb410 tx=0x7f1294009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:18.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.643+0000 7f12a5600640 1 -- 172.21.15.100:0/295126065 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f12a8142340 con 0x7f12a806f280 2024-09-17T04:33:18.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.826+0000 7f12aea00640 1 -- 172.21.15.100:0/295126065 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f12a806f690 con 0x7f12a806f280 2024-09-17T04:33:18.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.827+0000 7f12a5600640 1 -- 172.21.15.100:0/295126065 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+16876 (secure 0 0 0) 0x7f129c0637a0 con 0x7f12a806f280 2024-09-17T04:33:18.828 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:18.828 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":156,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:33:15.880224+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:33:12.271566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:13.334445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:33:18.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.830+0000 7f12aea00640 1 -- 172.21.15.100:0/295126065 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1288079400 msgr2=0x7f128807b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:18.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.830+0000 7f12aea00640 1 --2- 172.21.15.100:0/295126065 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1288079400 0x7f128807b910 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f12a81eb410 tx=0x7f1294009040 comp rx=0 tx=0).stop 2024-09-17T04:33:18.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.830+0000 7f12aea00640 1 -- 172.21.15.100:0/295126065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12a806f280 msgr2=0x7f12a8144e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:18.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.830+0000 7f12aea00640 1 --2- 172.21.15.100:0/295126065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12a806f280 0x7f12a8144e10 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f129c00e790 tx=0x7f129c00ec50 comp rx=0 tx=0).stop 2024-09-17T04:33:18.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.830+0000 7f12aea00640 1 -- 172.21.15.100:0/295126065 shutdown_connections 2024-09-17T04:33:18.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.830+0000 7f12aea00640 1 --2- 172.21.15.100:0/295126065 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1288079400 0x7f128807b910 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:18.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.830+0000 7f12aea00640 1 --2- 172.21.15.100:0/295126065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12a81623a0 0x7f12a8153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:18.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.830+0000 7f12aea00640 1 --2- 172.21.15.100:0/295126065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12a8155370 0x7f12a8153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:18.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.830+0000 7f12aea00640 1 --2- 172.21.15.100:0/295126065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12a806f280 0x7f12a8144e10 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:18.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.830+0000 7f12aea00640 1 -- 172.21.15.100:0/295126065 >> 172.21.15.100:0/295126065 conn(0x7f12a80b1e40 msgr2=0x7f12a8154bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:18.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.830+0000 7f12aea00640 1 -- 172.21.15.100:0/295126065 shutdown_connections 2024-09-17T04:33:18.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:18.830+0000 7f12aea00640 1 -- 172.21.15.100:0/295126065 wait complete. 2024-09-17T04:33:18.853 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T04:33:18.853 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:33:19.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.074+0000 7f8203e00640 1 -- 172.21.15.100:0/409254331 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81fc15d760 msgr2=0x7f81fc1535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:19.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.074+0000 7f8203e00640 1 --2- 172.21.15.100:0/409254331 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81fc15d760 0x7f81fc1535c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f81f0005fd0 tx=0x7f81f002a540 comp rx=0 tx=0).stop 2024-09-17T04:33:19.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.074+0000 7f8203e00640 1 -- 172.21.15.100:0/409254331 shutdown_connections 2024-09-17T04:33:19.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.074+0000 7f8203e00640 1 --2- 172.21.15.100:0/409254331 >> v2:172.21.15.184:3300/0 conn(0x7f81fc0b3550 0x7f81fc0b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:19.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.074+0000 7f8203e00640 1 --2- 172.21.15.100:0/409254331 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81fc15d760 0x7f81fc1535c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:19.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.074+0000 7f8203e00640 1 -- 172.21.15.100:0/409254331 >> 172.21.15.100:0/409254331 conn(0x7f81fc06d730 msgr2=0x7f81fc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:19.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.074+0000 7f8203e00640 1 -- 172.21.15.100:0/409254331 shutdown_connections 2024-09-17T04:33:19.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.074+0000 7f8203e00640 1 -- 172.21.15.100:0/409254331 wait complete. 2024-09-17T04:33:19.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.076+0000 7f8203e00640 1 Processor -- start 2024-09-17T04:33:19.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.076+0000 7f8203e00640 1 -- start start 2024-09-17T04:33:19.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.077+0000 7f8203e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81fc0ad0e0 0x7f81fc148780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:19.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.078+0000 7f8203e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81fc0b3550 0x7f81fc15b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:19.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.078+0000 7f8203e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81fc15d760 0x7f81fc15bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:19.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.078+0000 7f8203e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc0b4da0 con 0x7f81fc15d760 2024-09-17T04:33:19.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.078+0000 7f8203e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc0b4c20 con 0x7f81fc0ad0e0 2024-09-17T04:33:19.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.078+0000 7f8203e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f81fc0b3ea0 con 0x7f81fc0b3550 2024-09-17T04:33:19.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.078+0000 7f8201e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81fc15d760 0x7f81fc15bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:19.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.078+0000 7f8201e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81fc15d760 0x7f81fc15bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42990/0 (socket says 172.21.15.100:42990) 2024-09-17T04:33:19.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.078+0000 7f8201e00640 1 -- 172.21.15.100:0/169467713 learned_addr learned my addr 172.21.15.100:0/169467713 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:19.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.078+0000 7f8200a00640 1 --2- 172.21.15.100:0/169467713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81fc0b3550 0x7f81fc15b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:19.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.078+0000 7f8201400640 1 --2- 172.21.15.100:0/169467713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81fc0ad0e0 0x7f81fc148780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:19.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.078+0000 7f8201e00640 1 -- 172.21.15.100:0/169467713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81fc0b3550 msgr2=0x7f81fc15b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:19.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.078+0000 7f8201e00640 1 --2- 172.21.15.100:0/169467713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81fc0b3550 0x7f81fc15b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:19.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.078+0000 7f8201e00640 1 -- 172.21.15.100:0/169467713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81fc0ad0e0 msgr2=0x7f81fc148780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:19.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.078+0000 7f8201e00640 1 --2- 172.21.15.100:0/169467713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81fc0ad0e0 0x7f81fc148780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:19.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.078+0000 7f8201e00640 1 -- 172.21.15.100:0/169467713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81fc070290 con 0x7f81fc15d760 2024-09-17T04:33:19.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.079+0000 7f8201e00640 1 --2- 172.21.15.100:0/169467713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81fc15d760 0x7f81fc15bd30 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f81f400ec20 tx=0x7f81f400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:19.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.079+0000 7f81f9600640 1 -- 172.21.15.100:0/169467713 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4003b70 con 0x7f81fc15d760 2024-09-17T04:33:19.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.079+0000 7f81f9600640 1 -- 172.21.15.100:0/169467713 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81f4003d10 con 0x7f81fc15d760 2024-09-17T04:33:19.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.079+0000 7f81f9600640 1 -- 172.21.15.100:0/169467713 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4004280 con 0x7f81fc15d760 2024-09-17T04:33:19.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.079+0000 7f8203e00640 1 -- 172.21.15.100:0/169467713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81fc0730e0 con 0x7f81fc15d760 2024-09-17T04:33:19.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.079+0000 7f8203e00640 1 -- 172.21.15.100:0/169467713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81fc0736a0 con 0x7f81fc15d760 2024-09-17T04:33:19.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.080+0000 7f8203e00640 1 -- 172.21.15.100:0/169467713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81fc070000 con 0x7f81fc15d760 2024-09-17T04:33:19.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.082+0000 7f81f9600640 1 -- 172.21.15.100:0/169467713 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f81f4004420 con 0x7f81fc15d760 2024-09-17T04:33:19.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.082+0000 7f81f9600640 1 --2- 172.21.15.100:0/169467713 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f81dc079430 0x7f81dc07b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:19.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.083+0000 7f81f9600640 1 -- 172.21.15.100:0/169467713 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7f81f409a180 con 0x7f81fc15d760 2024-09-17T04:33:19.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.083+0000 7f8201400640 1 --2- 172.21.15.100:0/169467713 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f81dc079430 0x7f81dc07b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:19.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.084+0000 7f8201400640 1 --2- 172.21.15.100:0/169467713 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f81dc079430 0x7f81dc07b940 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f81f0033000 tx=0x7f81f0004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:19.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.087+0000 7f81f9600640 1 -- 172.21.15.100:0/169467713 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f81f4064100 con 0x7f81fc15d760 2024-09-17T04:33:19.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.269+0000 7f8203e00640 1 -- 172.21.15.100:0/169467713 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f81fc0b4330 con 0x7f81dc079430 2024-09-17T04:33:19.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.274+0000 7f81f9600640 1 -- 172.21.15.100:0/169467713 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252475 (secure 0 0 0) 0x7f81fc0b4330 con 0x7f81dc079430 2024-09-17T04:33:19.277 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:19.279 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:33:19.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.278+0000 7f8203e00640 1 -- 172.21.15.100:0/169467713 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f81dc079430 msgr2=0x7f81dc07b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:19.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.278+0000 7f8203e00640 1 --2- 172.21.15.100:0/169467713 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f81dc079430 0x7f81dc07b940 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f81f0033000 tx=0x7f81f0004730 comp rx=0 tx=0).stop 2024-09-17T04:33:19.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.278+0000 7f8203e00640 1 -- 172.21.15.100:0/169467713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81fc15d760 msgr2=0x7f81fc15bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:19.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.278+0000 7f8203e00640 1 --2- 172.21.15.100:0/169467713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81fc15d760 0x7f81fc15bd30 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f81f400ec20 tx=0x7f81f400c6a0 comp rx=0 tx=0).stop 2024-09-17T04:33:19.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.278+0000 7f8203e00640 1 -- 172.21.15.100:0/169467713 shutdown_connections 2024-09-17T04:33:19.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.278+0000 7f8203e00640 1 --2- 172.21.15.100:0/169467713 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f81dc079430 0x7f81dc07b940 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:19.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.278+0000 7f8203e00640 1 --2- 172.21.15.100:0/169467713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81fc15d760 0x7f81fc15bd30 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:19.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.278+0000 7f8203e00640 1 --2- 172.21.15.100:0/169467713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81fc0b3550 0x7f81fc15b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:19.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.278+0000 7f8203e00640 1 --2- 172.21.15.100:0/169467713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81fc0ad0e0 0x7f81fc148780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:19.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.278+0000 7f8203e00640 1 -- 172.21.15.100:0/169467713 >> 172.21.15.100:0/169467713 conn(0x7f81fc06d730 msgr2=0x7f81fc0cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:19.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.278+0000 7f8203e00640 1 -- 172.21.15.100:0/169467713 shutdown_connections 2024-09-17T04:33:19.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.278+0000 7f8203e00640 1 -- 172.21.15.100:0/169467713 wait complete. 2024-09-17T04:33:19.304 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":210,"stamp":"2024-09-17T04:33:17.715728+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":157,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":107,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265944,"kb_used_data":16428,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754708776,"statfs":{"total":773094113280,"available":772821786624,"internally_reserved":0,"allocated":16822272,"data_stored":8058624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.854659"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892114+0000","last_change":"2024-09-17T04:33:14.906930+0000","last_active":"2024-09-17T04:33:15.892114+0000","last_peered":"2024-09-17T04:33:15.892114+0000","last_clean":"2024-09-17T04:33:15.892114+0000","last_became_active":"2024-09-17T04:33:14.906541+0000","last_became_peered":"2024-09-17T04:33:14.906541+0000","last_unstale":"2024-09-17T04:33:15.892114+0000","last_undegraded":"2024-09-17T04:33:15.892114+0000","last_fullsized":"2024-09-17T04:33:15.892114+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:56:01.470574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.908429+0000","last_change":"2024-09-17T04:33:14.903915+0000","last_active":"2024-09-17T04:33:14.908429+0000","last_peered":"2024-09-17T04:33:14.908429+0000","last_clean":"2024-09-17T04:33:14.908429+0000","last_became_active":"2024-09-17T04:33:14.903166+0000","last_became_peered":"2024-09-17T04:33:14.903166+0000","last_unstale":"2024-09-17T04:33:14.908429+0000","last_undegraded":"2024-09-17T04:33:14.908429+0000","last_fullsized":"2024-09-17T04:33:14.908429+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:48.770077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.910476+0000","last_change":"2024-09-17T04:33:14.903513+0000","last_active":"2024-09-17T04:33:14.910476+0000","last_peered":"2024-09-17T04:33:14.910476+0000","last_clean":"2024-09-17T04:33:14.910476+0000","last_became_active":"2024-09-17T04:33:14.901689+0000","last_became_peered":"2024-09-17T04:33:14.901689+0000","last_unstale":"2024-09-17T04:33:14.910476+0000","last_undegraded":"2024-09-17T04:33:14.910476+0000","last_fullsized":"2024-09-17T04:33:14.910476+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:14.702026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.905176+0000","last_change":"2024-09-17T04:33:14.901979+0000","last_active":"2024-09-17T04:33:14.905176+0000","last_peered":"2024-09-17T04:33:14.905176+0000","last_clean":"2024-09-17T04:33:14.905176+0000","last_became_active":"2024-09-17T04:33:14.901677+0000","last_became_peered":"2024-09-17T04:33:14.901677+0000","last_unstale":"2024-09-17T04:33:14.905176+0000","last_undegraded":"2024-09-17T04:33:14.905176+0000","last_fullsized":"2024-09-17T04:33:14.905176+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:13.713209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.891868+0000","last_change":"2024-09-17T04:33:14.886229+0000","last_active":"2024-09-17T04:33:15.891868+0000","last_peered":"2024-09-17T04:33:15.891868+0000","last_clean":"2024-09-17T04:33:15.891868+0000","last_became_active":"2024-09-17T04:33:14.885439+0000","last_became_peered":"2024-09-17T04:33:14.885439+0000","last_unstale":"2024-09-17T04:33:15.891868+0000","last_undegraded":"2024-09-17T04:33:15.891868+0000","last_fullsized":"2024-09-17T04:33:15.891868+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:44.203868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.909928+0000","last_change":"2024-09-17T04:33:14.903177+0000","last_active":"2024-09-17T04:33:14.909928+0000","last_peered":"2024-09-17T04:33:14.909928+0000","last_clean":"2024-09-17T04:33:14.909928+0000","last_became_active":"2024-09-17T04:33:14.902892+0000","last_became_peered":"2024-09-17T04:33:14.902892+0000","last_unstale":"2024-09-17T04:33:14.909928+0000","last_undegraded":"2024-09-17T04:33:14.909928+0000","last_fullsized":"2024-09-17T04:33:14.909928+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:22.890760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.898616+0000","last_change":"2024-09-17T04:33:14.896118+0000","last_active":"2024-09-17T04:33:14.898616+0000","last_peered":"2024-09-17T04:33:14.898616+0000","last_clean":"2024-09-17T04:33:14.898616+0000","last_became_active":"2024-09-17T04:33:14.895463+0000","last_became_peered":"2024-09-17T04:33:14.895463+0000","last_unstale":"2024-09-17T04:33:14.898616+0000","last_undegraded":"2024-09-17T04:33:14.898616+0000","last_fullsized":"2024-09-17T04:33:14.898616+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:45.690163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:16.646558+0000","last_change":"2024-09-17T04:33:14.890865+0000","last_active":"2024-09-17T04:33:16.646558+0000","last_peered":"2024-09-17T04:33:16.646558+0000","last_clean":"2024-09-17T04:33:16.646558+0000","last_became_active":"2024-09-17T04:33:14.890373+0000","last_became_peered":"2024-09-17T04:33:14.890373+0000","last_unstale":"2024-09-17T04:33:16.646558+0000","last_undegraded":"2024-09-17T04:33:16.646558+0000","last_fullsized":"2024-09-17T04:33:16.646558+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:44:46.774914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.893418+0000","last_change":"2024-09-17T04:33:14.893833+0000","last_active":"2024-09-17T04:33:15.893418+0000","last_peered":"2024-09-17T04:33:15.893418+0000","last_clean":"2024-09-17T04:33:15.893418+0000","last_became_active":"2024-09-17T04:33:14.893517+0000","last_became_peered":"2024-09-17T04:33:14.893517+0000","last_unstale":"2024-09-17T04:33:15.893418+0000","last_undegraded":"2024-09-17T04:33:15.893418+0000","last_fullsized":"2024-09-17T04:33:15.893418+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:15.712200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.914974+0000","last_change":"2024-09-17T04:33:14.911595+0000","last_active":"2024-09-17T04:33:14.914974+0000","last_peered":"2024-09-17T04:33:14.914974+0000","last_clean":"2024-09-17T04:33:14.914974+0000","last_became_active":"2024-09-17T04:33:14.907336+0000","last_became_peered":"2024-09-17T04:33:14.907336+0000","last_unstale":"2024-09-17T04:33:14.914974+0000","last_undegraded":"2024-09-17T04:33:14.914974+0000","last_fullsized":"2024-09-17T04:33:14.914974+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:06.975187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.906077+0000","last_change":"2024-09-17T04:33:14.902358+0000","last_active":"2024-09-17T04:33:14.906077+0000","last_peered":"2024-09-17T04:33:14.906077+0000","last_clean":"2024-09-17T04:33:14.906077+0000","last_became_active":"2024-09-17T04:33:14.902016+0000","last_became_peered":"2024-09-17T04:33:14.902016+0000","last_unstale":"2024-09-17T04:33:14.906077+0000","last_undegraded":"2024-09-17T04:33:14.906077+0000","last_fullsized":"2024-09-17T04:33:14.906077+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:16.442070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.912798+0000","last_change":"2024-09-17T04:33:14.909611+0000","last_active":"2024-09-17T04:33:14.912798+0000","last_peered":"2024-09-17T04:33:14.912798+0000","last_clean":"2024-09-17T04:33:14.912798+0000","last_became_active":"2024-09-17T04:33:14.909269+0000","last_became_peered":"2024-09-17T04:33:14.909269+0000","last_unstale":"2024-09-17T04:33:14.912798+0000","last_undegraded":"2024-09-17T04:33:14.912798+0000","last_fullsized":"2024-09-17T04:33:14.912798+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:59:41.788193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.903502+0000","last_change":"2024-09-17T04:33:14.895306+0000","last_active":"2024-09-17T04:33:14.903502+0000","last_peered":"2024-09-17T04:33:14.903502+0000","last_clean":"2024-09-17T04:33:14.903502+0000","last_became_active":"2024-09-17T04:33:14.894985+0000","last_became_peered":"2024-09-17T04:33:14.894985+0000","last_unstale":"2024-09-17T04:33:14.903502+0000","last_undegraded":"2024-09-17T04:33:14.903502+0000","last_fullsized":"2024-09-17T04:33:14.903502+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:15.712200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892057+0000","last_change":"2024-09-17T04:33:14.900625+0000","last_active":"2024-09-17T04:33:15.892057+0000","last_peered":"2024-09-17T04:33:15.892057+0000","last_clean":"2024-09-17T04:33:15.892057+0000","last_became_active":"2024-09-17T04:33:14.898662+0000","last_became_peered":"2024-09-17T04:33:14.898662+0000","last_unstale":"2024-09-17T04:33:15.892057+0000","last_undegraded":"2024-09-17T04:33:15.892057+0000","last_fullsized":"2024-09-17T04:33:15.892057+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:27.884972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.913019+0000","last_change":"2024-09-17T04:33:14.907567+0000","last_active":"2024-09-17T04:33:14.913019+0000","last_peered":"2024-09-17T04:33:14.913019+0000","last_clean":"2024-09-17T04:33:14.913019+0000","last_became_active":"2024-09-17T04:33:14.907245+0000","last_became_peered":"2024-09-17T04:33:14.907245+0000","last_unstale":"2024-09-17T04:33:14.913019+0000","last_undegraded":"2024-09-17T04:33:14.913019+0000","last_fullsized":"2024-09-17T04:33:14.913019+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:53:35.444966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:16.646532+0000","last_change":"2024-09-17T04:33:14.889844+0000","last_active":"2024-09-17T04:33:16.646532+0000","last_peered":"2024-09-17T04:33:16.646532+0000","last_clean":"2024-09-17T04:33:16.646532+0000","last_became_active":"2024-09-17T04:33:14.889493+0000","last_became_peered":"2024-09-17T04:33:14.889493+0000","last_unstale":"2024-09-17T04:33:16.646532+0000","last_undegraded":"2024-09-17T04:33:16.646532+0000","last_fullsized":"2024-09-17T04:33:16.646532+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:05:43.089153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892410+0000","last_change":"2024-09-17T04:33:14.900519+0000","last_active":"2024-09-17T04:33:15.892410+0000","last_peered":"2024-09-17T04:33:15.892410+0000","last_clean":"2024-09-17T04:33:15.892410+0000","last_became_active":"2024-09-17T04:33:14.898843+0000","last_became_peered":"2024-09-17T04:33:14.898843+0000","last_unstale":"2024-09-17T04:33:15.892410+0000","last_undegraded":"2024-09-17T04:33:15.892410+0000","last_fullsized":"2024-09-17T04:33:15.892410+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:55.811210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892445+0000","last_change":"2024-09-17T04:33:14.906869+0000","last_active":"2024-09-17T04:33:15.892445+0000","last_peered":"2024-09-17T04:33:15.892445+0000","last_clean":"2024-09-17T04:33:15.892445+0000","last_became_active":"2024-09-17T04:33:14.906517+0000","last_became_peered":"2024-09-17T04:33:14.906517+0000","last_unstale":"2024-09-17T04:33:15.892445+0000","last_undegraded":"2024-09-17T04:33:15.892445+0000","last_fullsized":"2024-09-17T04:33:15.892445+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:11.543891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.905888+0000","last_change":"2024-09-17T04:33:14.898753+0000","last_active":"2024-09-17T04:33:14.905888+0000","last_peered":"2024-09-17T04:33:14.905888+0000","last_clean":"2024-09-17T04:33:14.905888+0000","last_became_active":"2024-09-17T04:33:14.898414+0000","last_became_peered":"2024-09-17T04:33:14.898414+0000","last_unstale":"2024-09-17T04:33:14.905888+0000","last_undegraded":"2024-09-17T04:33:14.905888+0000","last_fullsized":"2024-09-17T04:33:14.905888+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:37:41.250395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892186+0000","last_change":"2024-09-17T04:33:14.896399+0000","last_active":"2024-09-17T04:33:15.892186+0000","last_peered":"2024-09-17T04:33:15.892186+0000","last_clean":"2024-09-17T04:33:15.892186+0000","last_became_active":"2024-09-17T04:33:14.895477+0000","last_became_peered":"2024-09-17T04:33:14.895477+0000","last_unstale":"2024-09-17T04:33:15.892186+0000","last_undegraded":"2024-09-17T04:33:15.892186+0000","last_fullsized":"2024-09-17T04:33:15.892186+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:38.667341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892421+0000","last_change":"2024-09-17T04:33:14.886594+0000","last_active":"2024-09-17T04:33:15.892421+0000","last_peered":"2024-09-17T04:33:15.892421+0000","last_clean":"2024-09-17T04:33:15.892421+0000","last_became_active":"2024-09-17T04:33:14.885920+0000","last_became_peered":"2024-09-17T04:33:14.885920+0000","last_unstale":"2024-09-17T04:33:15.892421+0000","last_undegraded":"2024-09-17T04:33:15.892421+0000","last_fullsized":"2024-09-17T04:33:15.892421+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:01:10.443996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.914979+0000","last_change":"2024-09-17T04:33:14.912168+0000","last_active":"2024-09-17T04:33:14.914979+0000","last_peered":"2024-09-17T04:33:14.914979+0000","last_clean":"2024-09-17T04:33:14.914979+0000","last_became_active":"2024-09-17T04:33:14.910838+0000","last_became_peered":"2024-09-17T04:33:14.910838+0000","last_unstale":"2024-09-17T04:33:14.914979+0000","last_undegraded":"2024-09-17T04:33:14.914979+0000","last_fullsized":"2024-09-17T04:33:14.914979+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:50:31.637640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.906556+0000","last_change":"2024-09-17T04:33:14.899644+0000","last_active":"2024-09-17T04:33:14.906556+0000","last_peered":"2024-09-17T04:33:14.906556+0000","last_clean":"2024-09-17T04:33:14.906556+0000","last_became_active":"2024-09-17T04:33:14.898489+0000","last_became_peered":"2024-09-17T04:33:14.898489+0000","last_unstale":"2024-09-17T04:33:14.906556+0000","last_undegraded":"2024-09-17T04:33:14.906556+0000","last_fullsized":"2024-09-17T04:33:14.906556+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:45:58.483864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:16.646852+0000","last_change":"2024-09-17T04:33:14.898156+0000","last_active":"2024-09-17T04:33:16.646852+0000","last_peered":"2024-09-17T04:33:16.646852+0000","last_clean":"2024-09-17T04:33:16.646852+0000","last_became_active":"2024-09-17T04:33:14.897857+0000","last_became_peered":"2024-09-17T04:33:14.897857+0000","last_unstale":"2024-09-17T04:33:16.646852+0000","last_undegraded":"2024-09-17T04:33:16.646852+0000","last_fullsized":"2024-09-17T04:33:16.646852+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:31.269464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.904192+0000","last_change":"2024-09-17T04:33:14.897442+0000","last_active":"2024-09-17T04:33:14.904192+0000","last_peered":"2024-09-17T04:33:14.904192+0000","last_clean":"2024-09-17T04:33:14.904192+0000","last_became_active":"2024-09-17T04:33:14.897154+0000","last_became_peered":"2024-09-17T04:33:14.897154+0000","last_unstale":"2024-09-17T04:33:14.904192+0000","last_undegraded":"2024-09-17T04:33:14.904192+0000","last_fullsized":"2024-09-17T04:33:14.904192+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:08.510677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.905718+0000","last_change":"2024-09-17T04:33:14.901678+0000","last_active":"2024-09-17T04:33:14.905718+0000","last_peered":"2024-09-17T04:33:14.905718+0000","last_clean":"2024-09-17T04:33:14.905718+0000","last_became_active":"2024-09-17T04:33:14.901166+0000","last_became_peered":"2024-09-17T04:33:14.901166+0000","last_unstale":"2024-09-17T04:33:14.905718+0000","last_undegraded":"2024-09-17T04:33:14.905718+0000","last_fullsized":"2024-09-17T04:33:14.905718+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:40.543340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.910612+0000","last_change":"2024-09-17T04:33:14.907390+0000","last_active":"2024-09-17T04:33:14.910612+0000","last_peered":"2024-09-17T04:33:14.910612+0000","last_clean":"2024-09-17T04:33:14.910612+0000","last_became_active":"2024-09-17T04:33:14.902094+0000","last_became_peered":"2024-09-17T04:33:14.902094+0000","last_unstale":"2024-09-17T04:33:14.910612+0000","last_undegraded":"2024-09-17T04:33:14.910612+0000","last_fullsized":"2024-09-17T04:33:14.910612+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:25:41.401137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892417+0000","last_change":"2024-09-17T04:33:14.902408+0000","last_active":"2024-09-17T04:33:15.892417+0000","last_peered":"2024-09-17T04:33:15.892417+0000","last_clean":"2024-09-17T04:33:15.892417+0000","last_became_active":"2024-09-17T04:33:14.899145+0000","last_became_peered":"2024-09-17T04:33:14.899145+0000","last_unstale":"2024-09-17T04:33:15.892417+0000","last_undegraded":"2024-09-17T04:33:15.892417+0000","last_fullsized":"2024-09-17T04:33:15.892417+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:15.431794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.913709+0000","last_change":"2024-09-17T04:33:14.911225+0000","last_active":"2024-09-17T04:33:14.913709+0000","last_peered":"2024-09-17T04:33:14.913709+0000","last_clean":"2024-09-17T04:33:14.913709+0000","last_became_active":"2024-09-17T04:33:14.910900+0000","last_became_peered":"2024-09-17T04:33:14.910900+0000","last_unstale":"2024-09-17T04:33:14.913709+0000","last_undegraded":"2024-09-17T04:33:14.913709+0000","last_fullsized":"2024-09-17T04:33:14.913709+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:22.890760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892211+0000","last_change":"2024-09-17T04:33:14.900343+0000","last_active":"2024-09-17T04:33:15.892211+0000","last_peered":"2024-09-17T04:33:15.892211+0000","last_clean":"2024-09-17T04:33:15.892211+0000","last_became_active":"2024-09-17T04:33:14.896961+0000","last_became_peered":"2024-09-17T04:33:14.896961+0000","last_unstale":"2024-09-17T04:33:15.892211+0000","last_undegraded":"2024-09-17T04:33:15.892211+0000","last_fullsized":"2024-09-17T04:33:15.892211+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:22:18.153393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.910122+0000","last_change":"2024-09-17T04:33:14.906255+0000","last_active":"2024-09-17T04:33:14.910122+0000","last_peered":"2024-09-17T04:33:14.910122+0000","last_clean":"2024-09-17T04:33:14.910122+0000","last_became_active":"2024-09-17T04:33:14.905984+0000","last_became_peered":"2024-09-17T04:33:14.905984+0000","last_unstale":"2024-09-17T04:33:14.910122+0000","last_undegraded":"2024-09-17T04:33:14.910122+0000","last_fullsized":"2024-09-17T04:33:14.910122+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:32.006003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.891820+0000","last_change":"2024-09-17T04:33:14.886812+0000","last_active":"2024-09-17T04:33:15.891820+0000","last_peered":"2024-09-17T04:33:15.891820+0000","last_clean":"2024-09-17T04:33:15.891820+0000","last_became_active":"2024-09-17T04:33:14.886455+0000","last_became_peered":"2024-09-17T04:33:14.886455+0000","last_unstale":"2024-09-17T04:33:15.891820+0000","last_undegraded":"2024-09-17T04:33:15.891820+0000","last_fullsized":"2024-09-17T04:33:15.891820+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:52:01.273409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:16.647003+0000","last_change":"2024-09-17T04:33:14.903777+0000","last_active":"2024-09-17T04:33:16.647003+0000","last_peered":"2024-09-17T04:33:16.647003+0000","last_clean":"2024-09-17T04:33:16.647003+0000","last_became_active":"2024-09-17T04:33:14.900770+0000","last_became_peered":"2024-09-17T04:33:14.900770+0000","last_unstale":"2024-09-17T04:33:16.647003+0000","last_undegraded":"2024-09-17T04:33:16.647003+0000","last_fullsized":"2024-09-17T04:33:16.647003+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:22:40.765007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"155'4","reported_seq":19,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892431+0000","last_change":"2024-09-17T04:33:13.895022+0000","last_active":"2024-09-17T04:33:15.892431+0000","last_peered":"2024-09-17T04:33:15.892431+0000","last_clean":"2024-09-17T04:33:15.892431+0000","last_became_active":"2024-09-17T04:33:13.894653+0000","last_became_peered":"2024-09-17T04:33:13.894653+0000","last_unstale":"2024-09-17T04:33:15.892431+0000","last_undegraded":"2024-09-17T04:33:15.892431+0000","last_fullsized":"2024-09-17T04:33:15.892431+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:48.865527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.910670+0000","last_change":"2024-09-17T04:33:14.907300+0000","last_active":"2024-09-17T04:33:14.910670+0000","last_peered":"2024-09-17T04:33:14.910670+0000","last_clean":"2024-09-17T04:33:14.910670+0000","last_became_active":"2024-09-17T04:33:14.906616+0000","last_became_peered":"2024-09-17T04:33:14.906616+0000","last_unstale":"2024-09-17T04:33:14.910670+0000","last_undegraded":"2024-09-17T04:33:14.910670+0000","last_fullsized":"2024-09-17T04:33:14.910670+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:51.037084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"155'1","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.893469+0000","last_change":"2024-09-17T04:33:13.883712+0000","last_active":"2024-09-17T04:33:15.893469+0000","last_peered":"2024-09-17T04:33:15.893469+0000","last_clean":"2024-09-17T04:33:15.893469+0000","last_became_active":"2024-09-17T04:33:13.883390+0000","last_became_peered":"2024-09-17T04:33:13.883390+0000","last_unstale":"2024-09-17T04:33:15.893469+0000","last_undegraded":"2024-09-17T04:33:15.893469+0000","last_fullsized":"2024-09-17T04:33:15.893469+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:37.664776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.894724+0000","last_change":"2024-09-17T04:33:14.886562+0000","last_active":"2024-09-17T04:33:14.894724+0000","last_peered":"2024-09-17T04:33:14.894724+0000","last_clean":"2024-09-17T04:33:14.894724+0000","last_became_active":"2024-09-17T04:33:14.886252+0000","last_became_peered":"2024-09-17T04:33:14.886252+0000","last_unstale":"2024-09-17T04:33:14.894724+0000","last_undegraded":"2024-09-17T04:33:14.894724+0000","last_fullsized":"2024-09-17T04:33:14.894724+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:40:17.283073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"155'2","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.903327+0000","last_change":"2024-09-17T04:33:13.901062+0000","last_active":"2024-09-17T04:33:14.903327+0000","last_peered":"2024-09-17T04:33:14.903327+0000","last_clean":"2024-09-17T04:33:14.903327+0000","last_became_active":"2024-09-17T04:33:13.897758+0000","last_became_peered":"2024-09-17T04:33:13.897758+0000","last_unstale":"2024-09-17T04:33:14.903327+0000","last_undegraded":"2024-09-17T04:33:14.903327+0000","last_fullsized":"2024-09-17T04:33:14.903327+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:25:14.106132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.893335+0000","last_change":"2024-09-17T04:33:14.895456+0000","last_active":"2024-09-17T04:33:15.893335+0000","last_peered":"2024-09-17T04:33:15.893335+0000","last_clean":"2024-09-17T04:33:15.893335+0000","last_became_active":"2024-09-17T04:33:14.895115+0000","last_became_peered":"2024-09-17T04:33:14.895115+0000","last_unstale":"2024-09-17T04:33:15.893335+0000","last_undegraded":"2024-09-17T04:33:15.893335+0000","last_fullsized":"2024-09-17T04:33:15.893335+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:05.253995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"155'2","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.912733+0000","last_change":"2024-09-17T04:33:13.893084+0000","last_active":"2024-09-17T04:33:14.912733+0000","last_peered":"2024-09-17T04:33:14.912733+0000","last_clean":"2024-09-17T04:33:14.912733+0000","last_became_active":"2024-09-17T04:33:13.892722+0000","last_became_peered":"2024-09-17T04:33:13.892722+0000","last_unstale":"2024-09-17T04:33:14.912733+0000","last_undegraded":"2024-09-17T04:33:14.912733+0000","last_fullsized":"2024-09-17T04:33:14.912733+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:22:17.150828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.914939+0000","last_change":"2024-09-17T04:33:14.911548+0000","last_active":"2024-09-17T04:33:14.914939+0000","last_peered":"2024-09-17T04:33:14.914939+0000","last_clean":"2024-09-17T04:33:14.914939+0000","last_became_active":"2024-09-17T04:33:14.909119+0000","last_became_peered":"2024-09-17T04:33:14.909119+0000","last_unstale":"2024-09-17T04:33:14.914939+0000","last_undegraded":"2024-09-17T04:33:14.914939+0000","last_fullsized":"2024-09-17T04:33:14.914939+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:56:25.048828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"155'1","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892037+0000","last_change":"2024-09-17T04:33:13.890313+0000","last_active":"2024-09-17T04:33:15.892037+0000","last_peered":"2024-09-17T04:33:15.892037+0000","last_clean":"2024-09-17T04:33:15.892037+0000","last_became_active":"2024-09-17T04:33:13.890006+0000","last_became_peered":"2024-09-17T04:33:13.890006+0000","last_unstale":"2024-09-17T04:33:15.892037+0000","last_undegraded":"2024-09-17T04:33:15.892037+0000","last_fullsized":"2024-09-17T04:33:15.892037+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:28:45.884177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:16.647037+0000","last_change":"2024-09-17T04:33:14.891705+0000","last_active":"2024-09-17T04:33:16.647037+0000","last_peered":"2024-09-17T04:33:16.647037+0000","last_clean":"2024-09-17T04:33:16.647037+0000","last_became_active":"2024-09-17T04:33:14.891421+0000","last_became_peered":"2024-09-17T04:33:14.891421+0000","last_unstale":"2024-09-17T04:33:16.647037+0000","last_undegraded":"2024-09-17T04:33:16.647037+0000","last_fullsized":"2024-09-17T04:33:16.647037+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:07:17.534423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"155'1","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:16.646613+0000","last_change":"2024-09-17T04:33:13.900576+0000","last_active":"2024-09-17T04:33:16.646613+0000","last_peered":"2024-09-17T04:33:16.646613+0000","last_clean":"2024-09-17T04:33:16.646613+0000","last_became_active":"2024-09-17T04:33:13.898861+0000","last_became_peered":"2024-09-17T04:33:13.898861+0000","last_unstale":"2024-09-17T04:33:16.646613+0000","last_undegraded":"2024-09-17T04:33:16.646613+0000","last_fullsized":"2024-09-17T04:33:16.646613+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:35.386254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.906238+0000","last_change":"2024-09-17T04:33:14.902213+0000","last_active":"2024-09-17T04:33:14.906238+0000","last_peered":"2024-09-17T04:33:14.906238+0000","last_clean":"2024-09-17T04:33:14.906238+0000","last_became_active":"2024-09-17T04:33:14.901308+0000","last_became_peered":"2024-09-17T04:33:14.901308+0000","last_unstale":"2024-09-17T04:33:14.906238+0000","last_undegraded":"2024-09-17T04:33:14.906238+0000","last_fullsized":"2024-09-17T04:33:14.906238+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:10:11.078915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.891693+0000","last_change":"2024-09-17T04:33:13.879534+0000","last_active":"2024-09-17T04:33:15.891693+0000","last_peered":"2024-09-17T04:33:15.891693+0000","last_clean":"2024-09-17T04:33:15.891693+0000","last_became_active":"2024-09-17T04:33:13.879210+0000","last_became_peered":"2024-09-17T04:33:13.879210+0000","last_unstale":"2024-09-17T04:33:15.891693+0000","last_undegraded":"2024-09-17T04:33:15.891693+0000","last_fullsized":"2024-09-17T04:33:15.891693+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:20:36.296439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.910108+0000","last_change":"2024-09-17T04:33:14.902151+0000","last_active":"2024-09-17T04:33:14.910108+0000","last_peered":"2024-09-17T04:33:14.910108+0000","last_clean":"2024-09-17T04:33:14.910108+0000","last_became_active":"2024-09-17T04:33:14.901854+0000","last_became_peered":"2024-09-17T04:33:14.901854+0000","last_unstale":"2024-09-17T04:33:14.910108+0000","last_undegraded":"2024-09-17T04:33:14.910108+0000","last_fullsized":"2024-09-17T04:33:14.910108+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:59:41.788193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.884167+0000","last_change":"2024-09-17T04:33:13.891681+0000","last_active":"2024-09-17T04:33:14.884167+0000","last_peered":"2024-09-17T04:33:14.884167+0000","last_clean":"2024-09-17T04:33:14.884167+0000","last_became_active":"2024-09-17T04:33:13.890541+0000","last_became_peered":"2024-09-17T04:33:13.890541+0000","last_unstale":"2024-09-17T04:33:14.884167+0000","last_undegraded":"2024-09-17T04:33:14.884167+0000","last_fullsized":"2024-09-17T04:33:14.884167+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:25.557822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:16.646947+0000","last_change":"2024-09-17T04:33:14.901777+0000","last_active":"2024-09-17T04:33:16.646947+0000","last_peered":"2024-09-17T04:33:16.646947+0000","last_clean":"2024-09-17T04:33:16.646947+0000","last_became_active":"2024-09-17T04:33:14.901487+0000","last_became_peered":"2024-09-17T04:33:14.901487+0000","last_unstale":"2024-09-17T04:33:16.646947+0000","last_undegraded":"2024-09-17T04:33:16.646947+0000","last_fullsized":"2024-09-17T04:33:16.646947+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:08.254493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892196+0000","last_change":"2024-09-17T04:33:13.895279+0000","last_active":"2024-09-17T04:33:15.892196+0000","last_peered":"2024-09-17T04:33:15.892196+0000","last_clean":"2024-09-17T04:33:15.892196+0000","last_became_active":"2024-09-17T04:33:13.894999+0000","last_became_peered":"2024-09-17T04:33:13.894999+0000","last_unstale":"2024-09-17T04:33:15.892196+0000","last_undegraded":"2024-09-17T04:33:15.892196+0000","last_fullsized":"2024-09-17T04:33:15.892196+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:49:09.965654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.907321+0000","last_change":"2024-09-17T04:33:14.900688+0000","last_active":"2024-09-17T04:33:14.907321+0000","last_peered":"2024-09-17T04:33:14.907321+0000","last_clean":"2024-09-17T04:33:14.907321+0000","last_became_active":"2024-09-17T04:33:14.899848+0000","last_became_peered":"2024-09-17T04:33:14.899848+0000","last_unstale":"2024-09-17T04:33:14.907321+0000","last_undegraded":"2024-09-17T04:33:14.907321+0000","last_fullsized":"2024-09-17T04:33:14.907321+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:55:18.961213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"155'3","reported_seq":18,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892170+0000","last_change":"2024-09-17T04:33:13.888601+0000","last_active":"2024-09-17T04:33:15.892170+0000","last_peered":"2024-09-17T04:33:15.892170+0000","last_clean":"2024-09-17T04:33:15.892170+0000","last_became_active":"2024-09-17T04:33:13.887279+0000","last_became_peered":"2024-09-17T04:33:13.887279+0000","last_unstale":"2024-09-17T04:33:15.892170+0000","last_undegraded":"2024-09-17T04:33:15.892170+0000","last_fullsized":"2024-09-17T04:33:15.892170+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:03:23.449677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.905917+0000","last_change":"2024-09-17T04:33:14.902064+0000","last_active":"2024-09-17T04:33:14.905917+0000","last_peered":"2024-09-17T04:33:14.905917+0000","last_clean":"2024-09-17T04:33:14.905917+0000","last_became_active":"2024-09-17T04:33:14.901762+0000","last_became_peered":"2024-09-17T04:33:14.901762+0000","last_unstale":"2024-09-17T04:33:14.905917+0000","last_undegraded":"2024-09-17T04:33:14.905917+0000","last_fullsized":"2024-09-17T04:33:14.905917+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:47:41.016028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"155'6","reported_seq":19,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.916133+0000","last_change":"2024-09-17T04:33:13.899987+0000","last_active":"2024-09-17T04:33:14.916133+0000","last_peered":"2024-09-17T04:33:14.916133+0000","last_clean":"2024-09-17T04:33:14.916133+0000","last_became_active":"2024-09-17T04:33:13.899673+0000","last_became_peered":"2024-09-17T04:33:13.899673+0000","last_unstale":"2024-09-17T04:33:14.916133+0000","last_undegraded":"2024-09-17T04:33:14.916133+0000","last_fullsized":"2024-09-17T04:33:14.916133+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:47.767511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.903381+0000","last_change":"2024-09-17T04:33:14.897603+0000","last_active":"2024-09-17T04:33:14.903381+0000","last_peered":"2024-09-17T04:33:14.903381+0000","last_clean":"2024-09-17T04:33:14.903381+0000","last_became_active":"2024-09-17T04:33:14.897308+0000","last_became_peered":"2024-09-17T04:33:14.897308+0000","last_unstale":"2024-09-17T04:33:14.903381+0000","last_undegraded":"2024-09-17T04:33:14.903381+0000","last_fullsized":"2024-09-17T04:33:14.903381+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:53:35.444966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"155'2","reported_seq":17,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892762+0000","last_change":"2024-09-17T04:33:13.890634+0000","last_active":"2024-09-17T04:33:15.892762+0000","last_peered":"2024-09-17T04:33:15.892762+0000","last_clean":"2024-09-17T04:33:15.892762+0000","last_became_active":"2024-09-17T04:33:13.889614+0000","last_became_peered":"2024-09-17T04:33:13.889614+0000","last_unstale":"2024-09-17T04:33:15.892762+0000","last_undegraded":"2024-09-17T04:33:15.892762+0000","last_fullsized":"2024-09-17T04:33:15.892762+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:07.977950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.899129+0000","last_change":"2024-09-17T04:33:14.896891+0000","last_active":"2024-09-17T04:33:14.899129+0000","last_peered":"2024-09-17T04:33:14.899129+0000","last_clean":"2024-09-17T04:33:14.899129+0000","last_became_active":"2024-09-17T04:33:14.896538+0000","last_became_peered":"2024-09-17T04:33:14.896538+0000","last_unstale":"2024-09-17T04:33:14.899129+0000","last_undegraded":"2024-09-17T04:33:14.899129+0000","last_fullsized":"2024-09-17T04:33:14.899129+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:16:09.953033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.889732+0000","last_change":"2024-09-17T04:33:13.896584+0000","last_active":"2024-09-17T04:33:14.889732+0000","last_peered":"2024-09-17T04:33:14.889732+0000","last_clean":"2024-09-17T04:33:14.889732+0000","last_became_active":"2024-09-17T04:33:13.896299+0000","last_became_peered":"2024-09-17T04:33:13.896299+0000","last_unstale":"2024-09-17T04:33:14.889732+0000","last_undegraded":"2024-09-17T04:33:14.889732+0000","last_fullsized":"2024-09-17T04:33:14.889732+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:35:23.692931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.891996+0000","last_change":"2024-09-17T04:33:14.885206+0000","last_active":"2024-09-17T04:33:15.891996+0000","last_peered":"2024-09-17T04:33:15.891996+0000","last_clean":"2024-09-17T04:33:15.891996+0000","last_became_active":"2024-09-17T04:33:14.884817+0000","last_became_peered":"2024-09-17T04:33:14.884817+0000","last_unstale":"2024-09-17T04:33:15.891996+0000","last_undegraded":"2024-09-17T04:33:15.891996+0000","last_fullsized":"2024-09-17T04:33:15.891996+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:53:07.321546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:16.646436+0000","last_change":"2024-09-17T04:33:13.890400+0000","last_active":"2024-09-17T04:33:16.646436+0000","last_peered":"2024-09-17T04:33:16.646436+0000","last_clean":"2024-09-17T04:33:16.646436+0000","last_became_active":"2024-09-17T04:33:13.890066+0000","last_became_peered":"2024-09-17T04:33:13.890066+0000","last_unstale":"2024-09-17T04:33:16.646436+0000","last_undegraded":"2024-09-17T04:33:16.646436+0000","last_fullsized":"2024-09-17T04:33:16.646436+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:50.665148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.905956+0000","last_change":"2024-09-17T04:33:14.900505+0000","last_active":"2024-09-17T04:33:14.905956+0000","last_peered":"2024-09-17T04:33:14.905956+0000","last_clean":"2024-09-17T04:33:14.905956+0000","last_became_active":"2024-09-17T04:33:14.900146+0000","last_became_peered":"2024-09-17T04:33:14.900146+0000","last_unstale":"2024-09-17T04:33:14.905956+0000","last_undegraded":"2024-09-17T04:33:14.905956+0000","last_fullsized":"2024-09-17T04:33:14.905956+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:11:00.168476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"155'8","reported_seq":23,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.891516+0000","last_change":"2024-09-17T04:33:13.882596+0000","last_active":"2024-09-17T04:33:15.891516+0000","last_peered":"2024-09-17T04:33:15.891516+0000","last_clean":"2024-09-17T04:33:15.891516+0000","last_became_active":"2024-09-17T04:33:13.881415+0000","last_became_peered":"2024-09-17T04:33:13.881415+0000","last_unstale":"2024-09-17T04:33:15.891516+0000","last_undegraded":"2024-09-17T04:33:15.891516+0000","last_fullsized":"2024-09-17T04:33:15.891516+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:24:05.580922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.899322+0000","last_change":"2024-09-17T04:33:14.897529+0000","last_active":"2024-09-17T04:33:14.899322+0000","last_peered":"2024-09-17T04:33:14.899322+0000","last_clean":"2024-09-17T04:33:14.899322+0000","last_became_active":"2024-09-17T04:33:14.897216+0000","last_became_peered":"2024-09-17T04:33:14.897216+0000","last_unstale":"2024-09-17T04:33:14.899322+0000","last_undegraded":"2024-09-17T04:33:14.899322+0000","last_fullsized":"2024-09-17T04:33:14.899322+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:12:58.993006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"155'1","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892505+0000","last_change":"2024-09-17T04:33:13.896035+0000","last_active":"2024-09-17T04:33:15.892505+0000","last_peered":"2024-09-17T04:33:15.892505+0000","last_clean":"2024-09-17T04:33:15.892505+0000","last_became_active":"2024-09-17T04:33:13.895688+0000","last_became_peered":"2024-09-17T04:33:13.895688+0000","last_unstale":"2024-09-17T04:33:15.892505+0000","last_undegraded":"2024-09-17T04:33:15.892505+0000","last_fullsized":"2024-09-17T04:33:15.892505+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:58:28.978883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.893492+0000","last_change":"2024-09-17T04:33:14.902656+0000","last_active":"2024-09-17T04:33:15.893492+0000","last_peered":"2024-09-17T04:33:15.893492+0000","last_clean":"2024-09-17T04:33:15.893492+0000","last_became_active":"2024-09-17T04:33:14.900151+0000","last_became_peered":"2024-09-17T04:33:14.900151+0000","last_unstale":"2024-09-17T04:33:15.893492+0000","last_undegraded":"2024-09-17T04:33:15.893492+0000","last_fullsized":"2024-09-17T04:33:15.893492+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:08.510677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"155'2","reported_seq":17,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:16.646931+0000","last_change":"2024-09-17T04:33:13.891610+0000","last_active":"2024-09-17T04:33:16.646931+0000","last_peered":"2024-09-17T04:33:16.646931+0000","last_clean":"2024-09-17T04:33:16.646931+0000","last_became_active":"2024-09-17T04:33:13.891205+0000","last_became_peered":"2024-09-17T04:33:13.891205+0000","last_unstale":"2024-09-17T04:33:16.646931+0000","last_undegraded":"2024-09-17T04:33:16.646931+0000","last_fullsized":"2024-09-17T04:33:16.646931+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:10.284497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892919+0000","last_change":"2024-09-17T04:33:14.892923+0000","last_active":"2024-09-17T04:33:15.892919+0000","last_peered":"2024-09-17T04:33:15.892919+0000","last_clean":"2024-09-17T04:33:15.892919+0000","last_became_active":"2024-09-17T04:33:14.892556+0000","last_became_peered":"2024-09-17T04:33:14.892556+0000","last_unstale":"2024-09-17T04:33:15.892919+0000","last_undegraded":"2024-09-17T04:33:15.892919+0000","last_fullsized":"2024-09-17T04:33:15.892919+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:07:52.563695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"155'1","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.891918+0000","last_change":"2024-09-17T04:33:13.881840+0000","last_active":"2024-09-17T04:33:15.891918+0000","last_peered":"2024-09-17T04:33:15.891918+0000","last_clean":"2024-09-17T04:33:15.891918+0000","last_became_active":"2024-09-17T04:33:13.881516+0000","last_became_peered":"2024-09-17T04:33:13.881516+0000","last_unstale":"2024-09-17T04:33:15.891918+0000","last_undegraded":"2024-09-17T04:33:15.891918+0000","last_fullsized":"2024-09-17T04:33:15.891918+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:13.368126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.908379+0000","last_change":"2024-09-17T04:33:14.903092+0000","last_active":"2024-09-17T04:33:14.908379+0000","last_peered":"2024-09-17T04:33:14.908379+0000","last_clean":"2024-09-17T04:33:14.908379+0000","last_became_active":"2024-09-17T04:33:14.902180+0000","last_became_peered":"2024-09-17T04:33:14.902180+0000","last_unstale":"2024-09-17T04:33:14.908379+0000","last_undegraded":"2024-09-17T04:33:14.908379+0000","last_fullsized":"2024-09-17T04:33:14.908379+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:25:15.108698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.885632+0000","last_change":"2024-09-17T04:33:13.892940+0000","last_active":"2024-09-17T04:33:14.885632+0000","last_peered":"2024-09-17T04:33:14.885632+0000","last_clean":"2024-09-17T04:33:14.885632+0000","last_became_active":"2024-09-17T04:33:13.892623+0000","last_became_peered":"2024-09-17T04:33:13.892623+0000","last_unstale":"2024-09-17T04:33:14.885632+0000","last_undegraded":"2024-09-17T04:33:14.885632+0000","last_fullsized":"2024-09-17T04:33:14.885632+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:38.404940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.914967+0000","last_change":"2024-09-17T04:33:14.911872+0000","last_active":"2024-09-17T04:33:14.914967+0000","last_peered":"2024-09-17T04:33:14.914967+0000","last_clean":"2024-09-17T04:33:14.914967+0000","last_became_active":"2024-09-17T04:33:14.909649+0000","last_became_peered":"2024-09-17T04:33:14.909649+0000","last_unstale":"2024-09-17T04:33:14.914967+0000","last_undegraded":"2024-09-17T04:33:14.914967+0000","last_fullsized":"2024-09-17T04:33:14.914967+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:18:18.668141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"155'2","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.909390+0000","last_change":"2024-09-17T04:33:13.899555+0000","last_active":"2024-09-17T04:33:14.909390+0000","last_peered":"2024-09-17T04:33:14.909390+0000","last_clean":"2024-09-17T04:33:14.909390+0000","last_became_active":"2024-09-17T04:33:13.899212+0000","last_became_peered":"2024-09-17T04:33:13.899212+0000","last_unstale":"2024-09-17T04:33:14.909390+0000","last_undegraded":"2024-09-17T04:33:14.909390+0000","last_fullsized":"2024-09-17T04:33:14.909390+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:07.508112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.897146+0000","last_change":"2024-09-17T04:33:14.890803+0000","last_active":"2024-09-17T04:33:14.897146+0000","last_peered":"2024-09-17T04:33:14.897146+0000","last_clean":"2024-09-17T04:33:14.897146+0000","last_became_active":"2024-09-17T04:33:14.890483+0000","last_became_peered":"2024-09-17T04:33:14.890483+0000","last_unstale":"2024-09-17T04:33:14.897146+0000","last_undegraded":"2024-09-17T04:33:14.897146+0000","last_fullsized":"2024-09-17T04:33:14.897146+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:24:05.093752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.891886+0000","last_change":"2024-09-17T04:33:13.892630+0000","last_active":"2024-09-17T04:33:15.891886+0000","last_peered":"2024-09-17T04:33:15.891886+0000","last_clean":"2024-09-17T04:33:15.891886+0000","last_became_active":"2024-09-17T04:33:13.892060+0000","last_became_peered":"2024-09-17T04:33:13.892060+0000","last_unstale":"2024-09-17T04:33:15.891886+0000","last_undegraded":"2024-09-17T04:33:15.891886+0000","last_fullsized":"2024-09-17T04:33:15.891886+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:54:32.231255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.903261+0000","last_change":"2024-09-17T04:33:14.895000+0000","last_active":"2024-09-17T04:33:14.903261+0000","last_peered":"2024-09-17T04:33:14.903261+0000","last_clean":"2024-09-17T04:33:14.903261+0000","last_became_active":"2024-09-17T04:33:14.894695+0000","last_became_peered":"2024-09-17T04:33:14.894695+0000","last_unstale":"2024-09-17T04:33:14.903261+0000","last_undegraded":"2024-09-17T04:33:14.903261+0000","last_fullsized":"2024-09-17T04:33:14.903261+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:05.253995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.880687+0000","last_change":"2024-09-17T04:33:13.899571+0000","last_active":"2024-09-17T04:33:14.880687+0000","last_peered":"2024-09-17T04:33:14.880687+0000","last_clean":"2024-09-17T04:33:14.880687+0000","last_became_active":"2024-09-17T04:33:13.898599+0000","last_became_peered":"2024-09-17T04:33:13.898599+0000","last_unstale":"2024-09-17T04:33:14.880687+0000","last_undegraded":"2024-09-17T04:33:14.880687+0000","last_fullsized":"2024-09-17T04:33:14.880687+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:14.709635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.891772+0000","last_change":"2024-09-17T04:33:14.886577+0000","last_active":"2024-09-17T04:33:15.891772+0000","last_peered":"2024-09-17T04:33:15.891772+0000","last_clean":"2024-09-17T04:33:15.891772+0000","last_became_active":"2024-09-17T04:33:14.886221+0000","last_became_peered":"2024-09-17T04:33:14.886221+0000","last_unstale":"2024-09-17T04:33:15.891772+0000","last_undegraded":"2024-09-17T04:33:15.891772+0000","last_fullsized":"2024-09-17T04:33:15.891772+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:24.261861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.891864+0000","last_change":"2024-09-17T04:33:13.892911+0000","last_active":"2024-09-17T04:33:15.891864+0000","last_peered":"2024-09-17T04:33:15.891864+0000","last_clean":"2024-09-17T04:33:15.891864+0000","last_became_active":"2024-09-17T04:33:13.892269+0000","last_became_peered":"2024-09-17T04:33:13.892269+0000","last_unstale":"2024-09-17T04:33:15.891864+0000","last_undegraded":"2024-09-17T04:33:15.891864+0000","last_fullsized":"2024-09-17T04:33:15.891864+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:49.698053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892214+0000","last_change":"2024-09-17T04:33:14.889230+0000","last_active":"2024-09-17T04:33:15.892214+0000","last_peered":"2024-09-17T04:33:15.892214+0000","last_clean":"2024-09-17T04:33:15.892214+0000","last_became_active":"2024-09-17T04:33:14.888681+0000","last_became_peered":"2024-09-17T04:33:14.888681+0000","last_unstale":"2024-09-17T04:33:15.892214+0000","last_undegraded":"2024-09-17T04:33:15.892214+0000","last_fullsized":"2024-09-17T04:33:15.892214+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:50.700618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"155'2","reported_seq":17,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892277+0000","last_change":"2024-09-17T04:33:13.895768+0000","last_active":"2024-09-17T04:33:15.892277+0000","last_peered":"2024-09-17T04:33:15.892277+0000","last_clean":"2024-09-17T04:33:15.892277+0000","last_became_active":"2024-09-17T04:33:13.895491+0000","last_became_peered":"2024-09-17T04:33:13.895491+0000","last_unstale":"2024-09-17T04:33:15.892277+0000","last_undegraded":"2024-09-17T04:33:15.892277+0000","last_fullsized":"2024-09-17T04:33:15.892277+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:02.920831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.911835+0000","last_change":"2024-09-17T04:33:14.909774+0000","last_active":"2024-09-17T04:33:14.911835+0000","last_peered":"2024-09-17T04:33:14.911835+0000","last_clean":"2024-09-17T04:33:14.911835+0000","last_became_active":"2024-09-17T04:33:14.909456+0000","last_became_peered":"2024-09-17T04:33:14.909456+0000","last_unstale":"2024-09-17T04:33:14.911835+0000","last_undegraded":"2024-09-17T04:33:14.911835+0000","last_fullsized":"2024-09-17T04:33:14.911835+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:12:18.578443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.889651+0000","last_change":"2024-09-17T04:33:13.901401+0000","last_active":"2024-09-17T04:33:14.889651+0000","last_peered":"2024-09-17T04:33:14.889651+0000","last_clean":"2024-09-17T04:33:14.889651+0000","last_became_active":"2024-09-17T04:33:13.901001+0000","last_became_peered":"2024-09-17T04:33:13.901001+0000","last_unstale":"2024-09-17T04:33:14.889651+0000","last_undegraded":"2024-09-17T04:33:14.889651+0000","last_fullsized":"2024-09-17T04:33:14.889651+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:48.103792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:16.646905+0000","last_change":"2024-09-17T04:33:14.897223+0000","last_active":"2024-09-17T04:33:16.646905+0000","last_peered":"2024-09-17T04:33:16.646905+0000","last_clean":"2024-09-17T04:33:16.646905+0000","last_became_active":"2024-09-17T04:33:14.896945+0000","last_became_peered":"2024-09-17T04:33:14.896945+0000","last_unstale":"2024-09-17T04:33:16.646905+0000","last_undegraded":"2024-09-17T04:33:16.646905+0000","last_fullsized":"2024-09-17T04:33:16.646905+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:11.647670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.888818+0000","last_change":"2024-09-17T04:33:13.896962+0000","last_active":"2024-09-17T04:33:14.888818+0000","last_peered":"2024-09-17T04:33:14.888818+0000","last_clean":"2024-09-17T04:33:14.888818+0000","last_became_active":"2024-09-17T04:33:13.896698+0000","last_became_peered":"2024-09-17T04:33:13.896698+0000","last_unstale":"2024-09-17T04:33:14.888818+0000","last_undegraded":"2024-09-17T04:33:14.888818+0000","last_fullsized":"2024-09-17T04:33:14.888818+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:51.426171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:16.647112+0000","last_change":"2024-09-17T04:33:14.889928+0000","last_active":"2024-09-17T04:33:16.647112+0000","last_peered":"2024-09-17T04:33:16.647112+0000","last_clean":"2024-09-17T04:33:16.647112+0000","last_became_active":"2024-09-17T04:33:14.889010+0000","last_became_peered":"2024-09-17T04:33:14.889010+0000","last_unstale":"2024-09-17T04:33:16.647112+0000","last_undegraded":"2024-09-17T04:33:16.647112+0000","last_fullsized":"2024-09-17T04:33:16.647112+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:23:36.301615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:16.646964+0000","last_change":"2024-09-17T04:33:13.894840+0000","last_active":"2024-09-17T04:33:16.646964+0000","last_peered":"2024-09-17T04:33:16.646964+0000","last_clean":"2024-09-17T04:33:16.646964+0000","last_became_active":"2024-09-17T04:33:13.893661+0000","last_became_peered":"2024-09-17T04:33:13.893661+0000","last_unstale":"2024-09-17T04:33:16.646964+0000","last_undegraded":"2024-09-17T04:33:16.646964+0000","last_fullsized":"2024-09-17T04:33:16.646964+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:37.002449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892363+0000","last_change":"2024-09-17T04:33:14.910533+0000","last_active":"2024-09-17T04:33:15.892363+0000","last_peered":"2024-09-17T04:33:15.892363+0000","last_clean":"2024-09-17T04:33:15.892363+0000","last_became_active":"2024-09-17T04:33:14.910196+0000","last_became_peered":"2024-09-17T04:33:14.910196+0000","last_unstale":"2024-09-17T04:33:15.892363+0000","last_undegraded":"2024-09-17T04:33:15.892363+0000","last_fullsized":"2024-09-17T04:33:15.892363+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:32.340488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892617+0000","last_change":"2024-09-17T04:33:13.889252+0000","last_active":"2024-09-17T04:33:15.892617+0000","last_peered":"2024-09-17T04:33:15.892617+0000","last_clean":"2024-09-17T04:33:15.892617+0000","last_became_active":"2024-09-17T04:33:13.888943+0000","last_became_peered":"2024-09-17T04:33:13.888943+0000","last_unstale":"2024-09-17T04:33:15.892617+0000","last_undegraded":"2024-09-17T04:33:15.892617+0000","last_fullsized":"2024-09-17T04:33:15.892617+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:47.164060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892837+0000","last_change":"2024-09-17T04:33:14.908784+0000","last_active":"2024-09-17T04:33:15.892837+0000","last_peered":"2024-09-17T04:33:15.892837+0000","last_clean":"2024-09-17T04:33:15.892837+0000","last_became_active":"2024-09-17T04:33:14.908069+0000","last_became_peered":"2024-09-17T04:33:14.908069+0000","last_unstale":"2024-09-17T04:33:15.892837+0000","last_undegraded":"2024-09-17T04:33:15.892837+0000","last_fullsized":"2024-09-17T04:33:15.892837+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:49:26.623232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892176+0000","last_change":"2024-09-17T04:33:13.891080+0000","last_active":"2024-09-17T04:33:15.892176+0000","last_peered":"2024-09-17T04:33:15.892176+0000","last_clean":"2024-09-17T04:33:15.892176+0000","last_became_active":"2024-09-17T04:33:13.890381+0000","last_became_peered":"2024-09-17T04:33:13.890381+0000","last_unstale":"2024-09-17T04:33:15.892176+0000","last_undegraded":"2024-09-17T04:33:15.892176+0000","last_fullsized":"2024-09-17T04:33:15.892176+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:54.808645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.912673+0000","last_change":"2024-09-17T04:33:14.909599+0000","last_active":"2024-09-17T04:33:14.912673+0000","last_peered":"2024-09-17T04:33:14.912673+0000","last_clean":"2024-09-17T04:33:14.912673+0000","last_became_active":"2024-09-17T04:33:14.909168+0000","last_became_peered":"2024-09-17T04:33:14.909168+0000","last_unstale":"2024-09-17T04:33:14.912673+0000","last_undegraded":"2024-09-17T04:33:14.912673+0000","last_fullsized":"2024-09-17T04:33:14.912673+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:45:58.483864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.884717+0000","last_change":"2024-09-17T04:33:13.889911+0000","last_active":"2024-09-17T04:33:14.884717+0000","last_peered":"2024-09-17T04:33:14.884717+0000","last_clean":"2024-09-17T04:33:14.884717+0000","last_became_active":"2024-09-17T04:33:13.889533+0000","last_became_peered":"2024-09-17T04:33:13.889533+0000","last_unstale":"2024-09-17T04:33:14.884717+0000","last_undegraded":"2024-09-17T04:33:14.884717+0000","last_fullsized":"2024-09-17T04:33:14.884717+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:07:51.561130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.891839+0000","last_change":"2024-09-17T04:33:14.893431+0000","last_active":"2024-09-17T04:33:15.891839+0000","last_peered":"2024-09-17T04:33:15.891839+0000","last_clean":"2024-09-17T04:33:15.891839+0000","last_became_active":"2024-09-17T04:33:14.893115+0000","last_became_peered":"2024-09-17T04:33:14.893115+0000","last_unstale":"2024-09-17T04:33:15.891839+0000","last_undegraded":"2024-09-17T04:33:15.891839+0000","last_fullsized":"2024-09-17T04:33:15.891839+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:02:24.692768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.881064+0000","last_change":"2024-09-17T04:33:13.900366+0000","last_active":"2024-09-17T04:33:14.881064+0000","last_peered":"2024-09-17T04:33:14.881064+0000","last_clean":"2024-09-17T04:33:14.881064+0000","last_became_active":"2024-09-17T04:33:13.899454+0000","last_became_peered":"2024-09-17T04:33:13.899454+0000","last_unstale":"2024-09-17T04:33:14.881064+0000","last_undegraded":"2024-09-17T04:33:14.881064+0000","last_fullsized":"2024-09-17T04:33:14.881064+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:12.710644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.895447+0000","last_change":"2024-09-17T04:33:14.890132+0000","last_active":"2024-09-17T04:33:14.895447+0000","last_peered":"2024-09-17T04:33:14.895447+0000","last_clean":"2024-09-17T04:33:14.895447+0000","last_became_active":"2024-09-17T04:33:14.889693+0000","last_became_peered":"2024-09-17T04:33:14.889693+0000","last_unstale":"2024-09-17T04:33:14.895447+0000","last_undegraded":"2024-09-17T04:33:14.895447+0000","last_fullsized":"2024-09-17T04:33:14.895447+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:35:38.838913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.891536+0000","last_change":"2024-09-17T04:33:13.888354+0000","last_active":"2024-09-17T04:33:15.891536+0000","last_peered":"2024-09-17T04:33:15.891536+0000","last_clean":"2024-09-17T04:33:15.891536+0000","last_became_active":"2024-09-17T04:33:13.888017+0000","last_became_peered":"2024-09-17T04:33:13.888017+0000","last_unstale":"2024-09-17T04:33:15.891536+0000","last_undegraded":"2024-09-17T04:33:15.891536+0000","last_fullsized":"2024-09-17T04:33:15.891536+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:18:53.377553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"76'79","reported_seq":385,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.891603+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:33:15.891603+0000","last_peered":"2024-09-17T04:33:15.891603+0000","last_clean":"2024-09-17T04:33:15.891603+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:33:15.891603+0000","last_undegraded":"2024-09-17T04:33:15.891603+0000","last_fullsized":"2024-09-17T04:33:15.891603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33684,"kb_used_data":2496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338156,"statfs":{"total":96636764160,"available":96602271744,"internally_reserved":0,"allocated":2555904,"data_stored":1453578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1941504,"data_stored":856750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476852,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33096,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338744,"statfs":{"total":96636764160,"available":96602873856,"internally_reserved":0,"allocated":1953792,"data_stored":863106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33100,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338740,"statfs":{"total":96636764160,"available":96602869760,"internally_reserved":0,"allocated":1957888,"data_stored":858944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542259,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33096,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338744,"statfs":{"total":96636764160,"available":96602873856,"internally_reserved":0,"allocated":1953792,"data_stored":856944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509555,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33112,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338728,"statfs":{"total":96636764160,"available":96602857472,"internally_reserved":0,"allocated":1966080,"data_stored":863800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509554,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33096,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338744,"statfs":{"total":96636764160,"available":96602873856,"internally_reserved":0,"allocated":1953792,"data_stored":858362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509554,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33668,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338172,"statfs":{"total":96636764160,"available":96602288128,"internally_reserved":0,"allocated":2539520,"data_stored":1447140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:33:19.309 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:33:19.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.564+0000 7fd62b200640 1 -- 172.21.15.100:0/4036034312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6240ad0e0 msgr2=0x7fd6240ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:19.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.564+0000 7fd62b200640 1 --2- 172.21.15.100:0/4036034312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6240ad0e0 0x7fd6240ad4f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd610009860 tx=0x7fd61002a510 comp rx=0 tx=0).stop 2024-09-17T04:33:19.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.564+0000 7fd62b200640 1 -- 172.21.15.100:0/4036034312 shutdown_connections 2024-09-17T04:33:19.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.564+0000 7fd62b200640 1 --2- 172.21.15.100:0/4036034312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6240ad0e0 0x7fd6240ad4f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:19.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.564+0000 7fd62b200640 1 -- 172.21.15.100:0/4036034312 >> 172.21.15.100:0/4036034312 conn(0x7fd62406d730 msgr2=0x7fd6240b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:19.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.564+0000 7fd62b200640 1 -- 172.21.15.100:0/4036034312 shutdown_connections 2024-09-17T04:33:19.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.564+0000 7fd62b200640 1 -- 172.21.15.100:0/4036034312 wait complete. 2024-09-17T04:33:19.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.566+0000 7fd62b200640 1 Processor -- start 2024-09-17T04:33:19.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.566+0000 7fd62b200640 1 -- start start 2024-09-17T04:33:19.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.567+0000 7fd62b200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6240ad0e0 0x7fd6241f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:19.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.568+0000 7fd62b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6240b3550 0x7fd6241573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:19.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.568+0000 7fd62b200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd62415d760 0x7fd624099980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:19.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.568+0000 7fd62b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd6240b4da0 con 0x7fd6240b3550 2024-09-17T04:33:19.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.568+0000 7fd62b200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd6240b4c20 con 0x7fd6240ad0e0 2024-09-17T04:33:19.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.568+0000 7fd62b200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd6240b4070 con 0x7fd62415d760 2024-09-17T04:33:19.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.568+0000 7fd623400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6240b3550 0x7fd6241573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:19.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.568+0000 7fd623400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6240b3550 0x7fd6241573e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43004/0 (socket says 172.21.15.100:43004) 2024-09-17T04:33:19.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.568+0000 7fd623400640 1 -- 172.21.15.100:0/949516980 learned_addr learned my addr 172.21.15.100:0/949516980 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:19.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.568+0000 7fd623e00640 1 --2- 172.21.15.100:0/949516980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6240ad0e0 0x7fd6241f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:19.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.568+0000 7fd629200640 1 --2- 172.21.15.100:0/949516980 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd62415d760 0x7fd624099980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:19.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.568+0000 7fd623400640 1 -- 172.21.15.100:0/949516980 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd62415d760 msgr2=0x7fd624099980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:19.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.568+0000 7fd623400640 1 --2- 172.21.15.100:0/949516980 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd62415d760 0x7fd624099980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:19.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.568+0000 7fd623400640 1 -- 172.21.15.100:0/949516980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6240ad0e0 msgr2=0x7fd6241f6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:19.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.568+0000 7fd623400640 1 --2- 172.21.15.100:0/949516980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6240ad0e0 0x7fd6241f6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:19.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.568+0000 7fd623400640 1 -- 172.21.15.100:0/949516980 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd624099ec0 con 0x7fd6240b3550 2024-09-17T04:33:19.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.568+0000 7fd629200640 1 --2- 172.21.15.100:0/949516980 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd62415d760 0x7fd624099980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:19.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.569+0000 7fd623400640 1 --2- 172.21.15.100:0/949516980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6240b3550 0x7fd6241573e0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fd610009980 tx=0x7fd610033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:19.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.569+0000 7fd61be00640 1 -- 172.21.15.100:0/949516980 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610031b50 con 0x7fd6240b3550 2024-09-17T04:33:19.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.569+0000 7fd61be00640 1 -- 172.21.15.100:0/949516980 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd610031de0 con 0x7fd6240b3550 2024-09-17T04:33:19.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.569+0000 7fd61be00640 1 -- 172.21.15.100:0/949516980 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610005280 con 0x7fd6240b3550 2024-09-17T04:33:19.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.569+0000 7fd62b200640 1 -- 172.21.15.100:0/949516980 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6241f3770 con 0x7fd6240b3550 2024-09-17T04:33:19.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.569+0000 7fd62b200640 1 -- 172.21.15.100:0/949516980 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6241f7850 con 0x7fd6240b3550 2024-09-17T04:33:19.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.570+0000 7fd62b200640 1 -- 172.21.15.100:0/949516980 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6240ad4f0 con 0x7fd6240b3550 2024-09-17T04:33:19.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.571+0000 7fd61be00640 1 -- 172.21.15.100:0/949516980 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7fd610005420 con 0x7fd6240b3550 2024-09-17T04:33:19.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.572+0000 7fd61be00640 1 --2- 172.21.15.100:0/949516980 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd604079430 0x7fd60407b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:19.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.573+0000 7fd61be00640 1 -- 172.21.15.100:0/949516980 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7fd6100ba8f0 con 0x7fd6240b3550 2024-09-17T04:33:19.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.573+0000 7fd623e00640 1 --2- 172.21.15.100:0/949516980 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd604079430 0x7fd60407b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:19.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.573+0000 7fd623e00640 1 --2- 172.21.15.100:0/949516980 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd604079430 0x7fd60407b940 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd6140012e0 tx=0x7fd614009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:19.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.577+0000 7fd61be00640 1 -- 172.21.15.100:0/949516980 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd61003f030 con 0x7fd6240b3550 2024-09-17T04:33:19.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.757+0000 7fd62b200640 1 -- 172.21.15.100:0/949516980 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd6241f7300 con 0x7fd604079430 2024-09-17T04:33:19.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.761+0000 7fd61be00640 1 -- 172.21.15.100:0/949516980 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252475 (secure 0 0 0) 0x7fd6241f7300 con 0x7fd604079430 2024-09-17T04:33:19.764 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:19.766 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:33:19.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.765+0000 7fd62b200640 1 -- 172.21.15.100:0/949516980 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd604079430 msgr2=0x7fd60407b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:19.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.765+0000 7fd62b200640 1 --2- 172.21.15.100:0/949516980 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd604079430 0x7fd60407b940 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd6140012e0 tx=0x7fd614009040 comp rx=0 tx=0).stop 2024-09-17T04:33:19.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.765+0000 7fd62b200640 1 -- 172.21.15.100:0/949516980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6240b3550 msgr2=0x7fd6241573e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:19.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.765+0000 7fd62b200640 1 --2- 172.21.15.100:0/949516980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6240b3550 0x7fd6241573e0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fd610009980 tx=0x7fd610033ba0 comp rx=0 tx=0).stop 2024-09-17T04:33:19.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.766+0000 7fd62b200640 1 -- 172.21.15.100:0/949516980 shutdown_connections 2024-09-17T04:33:19.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.766+0000 7fd62b200640 1 --2- 172.21.15.100:0/949516980 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd604079430 0x7fd60407b940 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:19.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.766+0000 7fd62b200640 1 --2- 172.21.15.100:0/949516980 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd62415d760 0x7fd624099980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:19.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.766+0000 7fd62b200640 1 --2- 172.21.15.100:0/949516980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6240b3550 0x7fd6241573e0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:19.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.766+0000 7fd62b200640 1 --2- 172.21.15.100:0/949516980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6240ad0e0 0x7fd6241f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:19.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.766+0000 7fd62b200640 1 -- 172.21.15.100:0/949516980 >> 172.21.15.100:0/949516980 conn(0x7fd62406d730 msgr2=0x7fd6241535c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:19.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.766+0000 7fd62b200640 1 -- 172.21.15.100:0/949516980 shutdown_connections 2024-09-17T04:33:19.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:19.766+0000 7fd62b200640 1 -- 172.21.15.100:0/949516980 wait complete. 2024-09-17T04:33:19.791 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":211,"stamp":"2024-09-17T04:33:19.716213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":157,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":107,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265944,"kb_used_data":16428,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754708776,"statfs":{"total":773094113280,"available":772821786624,"internally_reserved":0,"allocated":16822272,"data_stored":8058624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001288"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892114+0000","last_change":"2024-09-17T04:33:14.906930+0000","last_active":"2024-09-17T04:33:15.892114+0000","last_peered":"2024-09-17T04:33:15.892114+0000","last_clean":"2024-09-17T04:33:15.892114+0000","last_became_active":"2024-09-17T04:33:14.906541+0000","last_became_peered":"2024-09-17T04:33:14.906541+0000","last_unstale":"2024-09-17T04:33:15.892114+0000","last_undegraded":"2024-09-17T04:33:15.892114+0000","last_fullsized":"2024-09-17T04:33:15.892114+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:56:01.470574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.908429+0000","last_change":"2024-09-17T04:33:14.903915+0000","last_active":"2024-09-17T04:33:14.908429+0000","last_peered":"2024-09-17T04:33:14.908429+0000","last_clean":"2024-09-17T04:33:14.908429+0000","last_became_active":"2024-09-17T04:33:14.903166+0000","last_became_peered":"2024-09-17T04:33:14.903166+0000","last_unstale":"2024-09-17T04:33:14.908429+0000","last_undegraded":"2024-09-17T04:33:14.908429+0000","last_fullsized":"2024-09-17T04:33:14.908429+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:48.770077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.910476+0000","last_change":"2024-09-17T04:33:14.903513+0000","last_active":"2024-09-17T04:33:14.910476+0000","last_peered":"2024-09-17T04:33:14.910476+0000","last_clean":"2024-09-17T04:33:14.910476+0000","last_became_active":"2024-09-17T04:33:14.901689+0000","last_became_peered":"2024-09-17T04:33:14.901689+0000","last_unstale":"2024-09-17T04:33:14.910476+0000","last_undegraded":"2024-09-17T04:33:14.910476+0000","last_fullsized":"2024-09-17T04:33:14.910476+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:14.702026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.905176+0000","last_change":"2024-09-17T04:33:14.901979+0000","last_active":"2024-09-17T04:33:14.905176+0000","last_peered":"2024-09-17T04:33:14.905176+0000","last_clean":"2024-09-17T04:33:14.905176+0000","last_became_active":"2024-09-17T04:33:14.901677+0000","last_became_peered":"2024-09-17T04:33:14.901677+0000","last_unstale":"2024-09-17T04:33:14.905176+0000","last_undegraded":"2024-09-17T04:33:14.905176+0000","last_fullsized":"2024-09-17T04:33:14.905176+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:13.713209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.891868+0000","last_change":"2024-09-17T04:33:14.886229+0000","last_active":"2024-09-17T04:33:15.891868+0000","last_peered":"2024-09-17T04:33:15.891868+0000","last_clean":"2024-09-17T04:33:15.891868+0000","last_became_active":"2024-09-17T04:33:14.885439+0000","last_became_peered":"2024-09-17T04:33:14.885439+0000","last_unstale":"2024-09-17T04:33:15.891868+0000","last_undegraded":"2024-09-17T04:33:15.891868+0000","last_fullsized":"2024-09-17T04:33:15.891868+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:44.203868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.909928+0000","last_change":"2024-09-17T04:33:14.903177+0000","last_active":"2024-09-17T04:33:14.909928+0000","last_peered":"2024-09-17T04:33:14.909928+0000","last_clean":"2024-09-17T04:33:14.909928+0000","last_became_active":"2024-09-17T04:33:14.902892+0000","last_became_peered":"2024-09-17T04:33:14.902892+0000","last_unstale":"2024-09-17T04:33:14.909928+0000","last_undegraded":"2024-09-17T04:33:14.909928+0000","last_fullsized":"2024-09-17T04:33:14.909928+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:22.890760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.898616+0000","last_change":"2024-09-17T04:33:14.896118+0000","last_active":"2024-09-17T04:33:14.898616+0000","last_peered":"2024-09-17T04:33:14.898616+0000","last_clean":"2024-09-17T04:33:14.898616+0000","last_became_active":"2024-09-17T04:33:14.895463+0000","last_became_peered":"2024-09-17T04:33:14.895463+0000","last_unstale":"2024-09-17T04:33:14.898616+0000","last_undegraded":"2024-09-17T04:33:14.898616+0000","last_fullsized":"2024-09-17T04:33:14.898616+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:45.690163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:16.646558+0000","last_change":"2024-09-17T04:33:14.890865+0000","last_active":"2024-09-17T04:33:16.646558+0000","last_peered":"2024-09-17T04:33:16.646558+0000","last_clean":"2024-09-17T04:33:16.646558+0000","last_became_active":"2024-09-17T04:33:14.890373+0000","last_became_peered":"2024-09-17T04:33:14.890373+0000","last_unstale":"2024-09-17T04:33:16.646558+0000","last_undegraded":"2024-09-17T04:33:16.646558+0000","last_fullsized":"2024-09-17T04:33:16.646558+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:44:46.774914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.893418+0000","last_change":"2024-09-17T04:33:14.893833+0000","last_active":"2024-09-17T04:33:15.893418+0000","last_peered":"2024-09-17T04:33:15.893418+0000","last_clean":"2024-09-17T04:33:15.893418+0000","last_became_active":"2024-09-17T04:33:14.893517+0000","last_became_peered":"2024-09-17T04:33:14.893517+0000","last_unstale":"2024-09-17T04:33:15.893418+0000","last_undegraded":"2024-09-17T04:33:15.893418+0000","last_fullsized":"2024-09-17T04:33:15.893418+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:15.712200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.914974+0000","last_change":"2024-09-17T04:33:14.911595+0000","last_active":"2024-09-17T04:33:14.914974+0000","last_peered":"2024-09-17T04:33:14.914974+0000","last_clean":"2024-09-17T04:33:14.914974+0000","last_became_active":"2024-09-17T04:33:14.907336+0000","last_became_peered":"2024-09-17T04:33:14.907336+0000","last_unstale":"2024-09-17T04:33:14.914974+0000","last_undegraded":"2024-09-17T04:33:14.914974+0000","last_fullsized":"2024-09-17T04:33:14.914974+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:06.975187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.906077+0000","last_change":"2024-09-17T04:33:14.902358+0000","last_active":"2024-09-17T04:33:14.906077+0000","last_peered":"2024-09-17T04:33:14.906077+0000","last_clean":"2024-09-17T04:33:14.906077+0000","last_became_active":"2024-09-17T04:33:14.902016+0000","last_became_peered":"2024-09-17T04:33:14.902016+0000","last_unstale":"2024-09-17T04:33:14.906077+0000","last_undegraded":"2024-09-17T04:33:14.906077+0000","last_fullsized":"2024-09-17T04:33:14.906077+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:16.442070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.912798+0000","last_change":"2024-09-17T04:33:14.909611+0000","last_active":"2024-09-17T04:33:14.912798+0000","last_peered":"2024-09-17T04:33:14.912798+0000","last_clean":"2024-09-17T04:33:14.912798+0000","last_became_active":"2024-09-17T04:33:14.909269+0000","last_became_peered":"2024-09-17T04:33:14.909269+0000","last_unstale":"2024-09-17T04:33:14.912798+0000","last_undegraded":"2024-09-17T04:33:14.912798+0000","last_fullsized":"2024-09-17T04:33:14.912798+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:59:41.788193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.903502+0000","last_change":"2024-09-17T04:33:14.895306+0000","last_active":"2024-09-17T04:33:14.903502+0000","last_peered":"2024-09-17T04:33:14.903502+0000","last_clean":"2024-09-17T04:33:14.903502+0000","last_became_active":"2024-09-17T04:33:14.894985+0000","last_became_peered":"2024-09-17T04:33:14.894985+0000","last_unstale":"2024-09-17T04:33:14.903502+0000","last_undegraded":"2024-09-17T04:33:14.903502+0000","last_fullsized":"2024-09-17T04:33:14.903502+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:15.712200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892057+0000","last_change":"2024-09-17T04:33:14.900625+0000","last_active":"2024-09-17T04:33:15.892057+0000","last_peered":"2024-09-17T04:33:15.892057+0000","last_clean":"2024-09-17T04:33:15.892057+0000","last_became_active":"2024-09-17T04:33:14.898662+0000","last_became_peered":"2024-09-17T04:33:14.898662+0000","last_unstale":"2024-09-17T04:33:15.892057+0000","last_undegraded":"2024-09-17T04:33:15.892057+0000","last_fullsized":"2024-09-17T04:33:15.892057+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:27.884972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.913019+0000","last_change":"2024-09-17T04:33:14.907567+0000","last_active":"2024-09-17T04:33:14.913019+0000","last_peered":"2024-09-17T04:33:14.913019+0000","last_clean":"2024-09-17T04:33:14.913019+0000","last_became_active":"2024-09-17T04:33:14.907245+0000","last_became_peered":"2024-09-17T04:33:14.907245+0000","last_unstale":"2024-09-17T04:33:14.913019+0000","last_undegraded":"2024-09-17T04:33:14.913019+0000","last_fullsized":"2024-09-17T04:33:14.913019+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:53:35.444966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:16.646532+0000","last_change":"2024-09-17T04:33:14.889844+0000","last_active":"2024-09-17T04:33:16.646532+0000","last_peered":"2024-09-17T04:33:16.646532+0000","last_clean":"2024-09-17T04:33:16.646532+0000","last_became_active":"2024-09-17T04:33:14.889493+0000","last_became_peered":"2024-09-17T04:33:14.889493+0000","last_unstale":"2024-09-17T04:33:16.646532+0000","last_undegraded":"2024-09-17T04:33:16.646532+0000","last_fullsized":"2024-09-17T04:33:16.646532+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:05:43.089153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892410+0000","last_change":"2024-09-17T04:33:14.900519+0000","last_active":"2024-09-17T04:33:15.892410+0000","last_peered":"2024-09-17T04:33:15.892410+0000","last_clean":"2024-09-17T04:33:15.892410+0000","last_became_active":"2024-09-17T04:33:14.898843+0000","last_became_peered":"2024-09-17T04:33:14.898843+0000","last_unstale":"2024-09-17T04:33:15.892410+0000","last_undegraded":"2024-09-17T04:33:15.892410+0000","last_fullsized":"2024-09-17T04:33:15.892410+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:55.811210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892445+0000","last_change":"2024-09-17T04:33:14.906869+0000","last_active":"2024-09-17T04:33:15.892445+0000","last_peered":"2024-09-17T04:33:15.892445+0000","last_clean":"2024-09-17T04:33:15.892445+0000","last_became_active":"2024-09-17T04:33:14.906517+0000","last_became_peered":"2024-09-17T04:33:14.906517+0000","last_unstale":"2024-09-17T04:33:15.892445+0000","last_undegraded":"2024-09-17T04:33:15.892445+0000","last_fullsized":"2024-09-17T04:33:15.892445+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:11.543891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.905888+0000","last_change":"2024-09-17T04:33:14.898753+0000","last_active":"2024-09-17T04:33:14.905888+0000","last_peered":"2024-09-17T04:33:14.905888+0000","last_clean":"2024-09-17T04:33:14.905888+0000","last_became_active":"2024-09-17T04:33:14.898414+0000","last_became_peered":"2024-09-17T04:33:14.898414+0000","last_unstale":"2024-09-17T04:33:14.905888+0000","last_undegraded":"2024-09-17T04:33:14.905888+0000","last_fullsized":"2024-09-17T04:33:14.905888+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:37:41.250395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892186+0000","last_change":"2024-09-17T04:33:14.896399+0000","last_active":"2024-09-17T04:33:15.892186+0000","last_peered":"2024-09-17T04:33:15.892186+0000","last_clean":"2024-09-17T04:33:15.892186+0000","last_became_active":"2024-09-17T04:33:14.895477+0000","last_became_peered":"2024-09-17T04:33:14.895477+0000","last_unstale":"2024-09-17T04:33:15.892186+0000","last_undegraded":"2024-09-17T04:33:15.892186+0000","last_fullsized":"2024-09-17T04:33:15.892186+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:38.667341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892421+0000","last_change":"2024-09-17T04:33:14.886594+0000","last_active":"2024-09-17T04:33:15.892421+0000","last_peered":"2024-09-17T04:33:15.892421+0000","last_clean":"2024-09-17T04:33:15.892421+0000","last_became_active":"2024-09-17T04:33:14.885920+0000","last_became_peered":"2024-09-17T04:33:14.885920+0000","last_unstale":"2024-09-17T04:33:15.892421+0000","last_undegraded":"2024-09-17T04:33:15.892421+0000","last_fullsized":"2024-09-17T04:33:15.892421+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:01:10.443996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.914979+0000","last_change":"2024-09-17T04:33:14.912168+0000","last_active":"2024-09-17T04:33:14.914979+0000","last_peered":"2024-09-17T04:33:14.914979+0000","last_clean":"2024-09-17T04:33:14.914979+0000","last_became_active":"2024-09-17T04:33:14.910838+0000","last_became_peered":"2024-09-17T04:33:14.910838+0000","last_unstale":"2024-09-17T04:33:14.914979+0000","last_undegraded":"2024-09-17T04:33:14.914979+0000","last_fullsized":"2024-09-17T04:33:14.914979+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:50:31.637640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.906556+0000","last_change":"2024-09-17T04:33:14.899644+0000","last_active":"2024-09-17T04:33:14.906556+0000","last_peered":"2024-09-17T04:33:14.906556+0000","last_clean":"2024-09-17T04:33:14.906556+0000","last_became_active":"2024-09-17T04:33:14.898489+0000","last_became_peered":"2024-09-17T04:33:14.898489+0000","last_unstale":"2024-09-17T04:33:14.906556+0000","last_undegraded":"2024-09-17T04:33:14.906556+0000","last_fullsized":"2024-09-17T04:33:14.906556+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:45:58.483864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:16.646852+0000","last_change":"2024-09-17T04:33:14.898156+0000","last_active":"2024-09-17T04:33:16.646852+0000","last_peered":"2024-09-17T04:33:16.646852+0000","last_clean":"2024-09-17T04:33:16.646852+0000","last_became_active":"2024-09-17T04:33:14.897857+0000","last_became_peered":"2024-09-17T04:33:14.897857+0000","last_unstale":"2024-09-17T04:33:16.646852+0000","last_undegraded":"2024-09-17T04:33:16.646852+0000","last_fullsized":"2024-09-17T04:33:16.646852+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:31.269464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.904192+0000","last_change":"2024-09-17T04:33:14.897442+0000","last_active":"2024-09-17T04:33:14.904192+0000","last_peered":"2024-09-17T04:33:14.904192+0000","last_clean":"2024-09-17T04:33:14.904192+0000","last_became_active":"2024-09-17T04:33:14.897154+0000","last_became_peered":"2024-09-17T04:33:14.897154+0000","last_unstale":"2024-09-17T04:33:14.904192+0000","last_undegraded":"2024-09-17T04:33:14.904192+0000","last_fullsized":"2024-09-17T04:33:14.904192+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:08.510677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.905718+0000","last_change":"2024-09-17T04:33:14.901678+0000","last_active":"2024-09-17T04:33:14.905718+0000","last_peered":"2024-09-17T04:33:14.905718+0000","last_clean":"2024-09-17T04:33:14.905718+0000","last_became_active":"2024-09-17T04:33:14.901166+0000","last_became_peered":"2024-09-17T04:33:14.901166+0000","last_unstale":"2024-09-17T04:33:14.905718+0000","last_undegraded":"2024-09-17T04:33:14.905718+0000","last_fullsized":"2024-09-17T04:33:14.905718+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:40.543340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.910612+0000","last_change":"2024-09-17T04:33:14.907390+0000","last_active":"2024-09-17T04:33:14.910612+0000","last_peered":"2024-09-17T04:33:14.910612+0000","last_clean":"2024-09-17T04:33:14.910612+0000","last_became_active":"2024-09-17T04:33:14.902094+0000","last_became_peered":"2024-09-17T04:33:14.902094+0000","last_unstale":"2024-09-17T04:33:14.910612+0000","last_undegraded":"2024-09-17T04:33:14.910612+0000","last_fullsized":"2024-09-17T04:33:14.910612+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:25:41.401137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892417+0000","last_change":"2024-09-17T04:33:14.902408+0000","last_active":"2024-09-17T04:33:15.892417+0000","last_peered":"2024-09-17T04:33:15.892417+0000","last_clean":"2024-09-17T04:33:15.892417+0000","last_became_active":"2024-09-17T04:33:14.899145+0000","last_became_peered":"2024-09-17T04:33:14.899145+0000","last_unstale":"2024-09-17T04:33:15.892417+0000","last_undegraded":"2024-09-17T04:33:15.892417+0000","last_fullsized":"2024-09-17T04:33:15.892417+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:15.431794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.913709+0000","last_change":"2024-09-17T04:33:14.911225+0000","last_active":"2024-09-17T04:33:14.913709+0000","last_peered":"2024-09-17T04:33:14.913709+0000","last_clean":"2024-09-17T04:33:14.913709+0000","last_became_active":"2024-09-17T04:33:14.910900+0000","last_became_peered":"2024-09-17T04:33:14.910900+0000","last_unstale":"2024-09-17T04:33:14.913709+0000","last_undegraded":"2024-09-17T04:33:14.913709+0000","last_fullsized":"2024-09-17T04:33:14.913709+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:22.890760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892211+0000","last_change":"2024-09-17T04:33:14.900343+0000","last_active":"2024-09-17T04:33:15.892211+0000","last_peered":"2024-09-17T04:33:15.892211+0000","last_clean":"2024-09-17T04:33:15.892211+0000","last_became_active":"2024-09-17T04:33:14.896961+0000","last_became_peered":"2024-09-17T04:33:14.896961+0000","last_unstale":"2024-09-17T04:33:15.892211+0000","last_undegraded":"2024-09-17T04:33:15.892211+0000","last_fullsized":"2024-09-17T04:33:15.892211+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:22:18.153393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.910122+0000","last_change":"2024-09-17T04:33:14.906255+0000","last_active":"2024-09-17T04:33:14.910122+0000","last_peered":"2024-09-17T04:33:14.910122+0000","last_clean":"2024-09-17T04:33:14.910122+0000","last_became_active":"2024-09-17T04:33:14.905984+0000","last_became_peered":"2024-09-17T04:33:14.905984+0000","last_unstale":"2024-09-17T04:33:14.910122+0000","last_undegraded":"2024-09-17T04:33:14.910122+0000","last_fullsized":"2024-09-17T04:33:14.910122+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:32.006003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.891820+0000","last_change":"2024-09-17T04:33:14.886812+0000","last_active":"2024-09-17T04:33:15.891820+0000","last_peered":"2024-09-17T04:33:15.891820+0000","last_clean":"2024-09-17T04:33:15.891820+0000","last_became_active":"2024-09-17T04:33:14.886455+0000","last_became_peered":"2024-09-17T04:33:14.886455+0000","last_unstale":"2024-09-17T04:33:15.891820+0000","last_undegraded":"2024-09-17T04:33:15.891820+0000","last_fullsized":"2024-09-17T04:33:15.891820+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:52:01.273409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:16.647003+0000","last_change":"2024-09-17T04:33:14.903777+0000","last_active":"2024-09-17T04:33:16.647003+0000","last_peered":"2024-09-17T04:33:16.647003+0000","last_clean":"2024-09-17T04:33:16.647003+0000","last_became_active":"2024-09-17T04:33:14.900770+0000","last_became_peered":"2024-09-17T04:33:14.900770+0000","last_unstale":"2024-09-17T04:33:16.647003+0000","last_undegraded":"2024-09-17T04:33:16.647003+0000","last_fullsized":"2024-09-17T04:33:16.647003+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:22:40.765007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"155'4","reported_seq":19,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892431+0000","last_change":"2024-09-17T04:33:13.895022+0000","last_active":"2024-09-17T04:33:15.892431+0000","last_peered":"2024-09-17T04:33:15.892431+0000","last_clean":"2024-09-17T04:33:15.892431+0000","last_became_active":"2024-09-17T04:33:13.894653+0000","last_became_peered":"2024-09-17T04:33:13.894653+0000","last_unstale":"2024-09-17T04:33:15.892431+0000","last_undegraded":"2024-09-17T04:33:15.892431+0000","last_fullsized":"2024-09-17T04:33:15.892431+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:48.865527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.910670+0000","last_change":"2024-09-17T04:33:14.907300+0000","last_active":"2024-09-17T04:33:14.910670+0000","last_peered":"2024-09-17T04:33:14.910670+0000","last_clean":"2024-09-17T04:33:14.910670+0000","last_became_active":"2024-09-17T04:33:14.906616+0000","last_became_peered":"2024-09-17T04:33:14.906616+0000","last_unstale":"2024-09-17T04:33:14.910670+0000","last_undegraded":"2024-09-17T04:33:14.910670+0000","last_fullsized":"2024-09-17T04:33:14.910670+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:51.037084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"155'1","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.893469+0000","last_change":"2024-09-17T04:33:13.883712+0000","last_active":"2024-09-17T04:33:15.893469+0000","last_peered":"2024-09-17T04:33:15.893469+0000","last_clean":"2024-09-17T04:33:15.893469+0000","last_became_active":"2024-09-17T04:33:13.883390+0000","last_became_peered":"2024-09-17T04:33:13.883390+0000","last_unstale":"2024-09-17T04:33:15.893469+0000","last_undegraded":"2024-09-17T04:33:15.893469+0000","last_fullsized":"2024-09-17T04:33:15.893469+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:37.664776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.894724+0000","last_change":"2024-09-17T04:33:14.886562+0000","last_active":"2024-09-17T04:33:14.894724+0000","last_peered":"2024-09-17T04:33:14.894724+0000","last_clean":"2024-09-17T04:33:14.894724+0000","last_became_active":"2024-09-17T04:33:14.886252+0000","last_became_peered":"2024-09-17T04:33:14.886252+0000","last_unstale":"2024-09-17T04:33:14.894724+0000","last_undegraded":"2024-09-17T04:33:14.894724+0000","last_fullsized":"2024-09-17T04:33:14.894724+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:40:17.283073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"155'2","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.903327+0000","last_change":"2024-09-17T04:33:13.901062+0000","last_active":"2024-09-17T04:33:14.903327+0000","last_peered":"2024-09-17T04:33:14.903327+0000","last_clean":"2024-09-17T04:33:14.903327+0000","last_became_active":"2024-09-17T04:33:13.897758+0000","last_became_peered":"2024-09-17T04:33:13.897758+0000","last_unstale":"2024-09-17T04:33:14.903327+0000","last_undegraded":"2024-09-17T04:33:14.903327+0000","last_fullsized":"2024-09-17T04:33:14.903327+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:25:14.106132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.893335+0000","last_change":"2024-09-17T04:33:14.895456+0000","last_active":"2024-09-17T04:33:15.893335+0000","last_peered":"2024-09-17T04:33:15.893335+0000","last_clean":"2024-09-17T04:33:15.893335+0000","last_became_active":"2024-09-17T04:33:14.895115+0000","last_became_peered":"2024-09-17T04:33:14.895115+0000","last_unstale":"2024-09-17T04:33:15.893335+0000","last_undegraded":"2024-09-17T04:33:15.893335+0000","last_fullsized":"2024-09-17T04:33:15.893335+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:05.253995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"155'2","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.912733+0000","last_change":"2024-09-17T04:33:13.893084+0000","last_active":"2024-09-17T04:33:14.912733+0000","last_peered":"2024-09-17T04:33:14.912733+0000","last_clean":"2024-09-17T04:33:14.912733+0000","last_became_active":"2024-09-17T04:33:13.892722+0000","last_became_peered":"2024-09-17T04:33:13.892722+0000","last_unstale":"2024-09-17T04:33:14.912733+0000","last_undegraded":"2024-09-17T04:33:14.912733+0000","last_fullsized":"2024-09-17T04:33:14.912733+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:22:17.150828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.914939+0000","last_change":"2024-09-17T04:33:14.911548+0000","last_active":"2024-09-17T04:33:14.914939+0000","last_peered":"2024-09-17T04:33:14.914939+0000","last_clean":"2024-09-17T04:33:14.914939+0000","last_became_active":"2024-09-17T04:33:14.909119+0000","last_became_peered":"2024-09-17T04:33:14.909119+0000","last_unstale":"2024-09-17T04:33:14.914939+0000","last_undegraded":"2024-09-17T04:33:14.914939+0000","last_fullsized":"2024-09-17T04:33:14.914939+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:56:25.048828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"155'1","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892037+0000","last_change":"2024-09-17T04:33:13.890313+0000","last_active":"2024-09-17T04:33:15.892037+0000","last_peered":"2024-09-17T04:33:15.892037+0000","last_clean":"2024-09-17T04:33:15.892037+0000","last_became_active":"2024-09-17T04:33:13.890006+0000","last_became_peered":"2024-09-17T04:33:13.890006+0000","last_unstale":"2024-09-17T04:33:15.892037+0000","last_undegraded":"2024-09-17T04:33:15.892037+0000","last_fullsized":"2024-09-17T04:33:15.892037+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:28:45.884177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:16.647037+0000","last_change":"2024-09-17T04:33:14.891705+0000","last_active":"2024-09-17T04:33:16.647037+0000","last_peered":"2024-09-17T04:33:16.647037+0000","last_clean":"2024-09-17T04:33:16.647037+0000","last_became_active":"2024-09-17T04:33:14.891421+0000","last_became_peered":"2024-09-17T04:33:14.891421+0000","last_unstale":"2024-09-17T04:33:16.647037+0000","last_undegraded":"2024-09-17T04:33:16.647037+0000","last_fullsized":"2024-09-17T04:33:16.647037+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:07:17.534423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"155'1","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:16.646613+0000","last_change":"2024-09-17T04:33:13.900576+0000","last_active":"2024-09-17T04:33:16.646613+0000","last_peered":"2024-09-17T04:33:16.646613+0000","last_clean":"2024-09-17T04:33:16.646613+0000","last_became_active":"2024-09-17T04:33:13.898861+0000","last_became_peered":"2024-09-17T04:33:13.898861+0000","last_unstale":"2024-09-17T04:33:16.646613+0000","last_undegraded":"2024-09-17T04:33:16.646613+0000","last_fullsized":"2024-09-17T04:33:16.646613+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:35.386254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.906238+0000","last_change":"2024-09-17T04:33:14.902213+0000","last_active":"2024-09-17T04:33:14.906238+0000","last_peered":"2024-09-17T04:33:14.906238+0000","last_clean":"2024-09-17T04:33:14.906238+0000","last_became_active":"2024-09-17T04:33:14.901308+0000","last_became_peered":"2024-09-17T04:33:14.901308+0000","last_unstale":"2024-09-17T04:33:14.906238+0000","last_undegraded":"2024-09-17T04:33:14.906238+0000","last_fullsized":"2024-09-17T04:33:14.906238+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:10:11.078915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.891693+0000","last_change":"2024-09-17T04:33:13.879534+0000","last_active":"2024-09-17T04:33:15.891693+0000","last_peered":"2024-09-17T04:33:15.891693+0000","last_clean":"2024-09-17T04:33:15.891693+0000","last_became_active":"2024-09-17T04:33:13.879210+0000","last_became_peered":"2024-09-17T04:33:13.879210+0000","last_unstale":"2024-09-17T04:33:15.891693+0000","last_undegraded":"2024-09-17T04:33:15.891693+0000","last_fullsized":"2024-09-17T04:33:15.891693+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:20:36.296439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.910108+0000","last_change":"2024-09-17T04:33:14.902151+0000","last_active":"2024-09-17T04:33:14.910108+0000","last_peered":"2024-09-17T04:33:14.910108+0000","last_clean":"2024-09-17T04:33:14.910108+0000","last_became_active":"2024-09-17T04:33:14.901854+0000","last_became_peered":"2024-09-17T04:33:14.901854+0000","last_unstale":"2024-09-17T04:33:14.910108+0000","last_undegraded":"2024-09-17T04:33:14.910108+0000","last_fullsized":"2024-09-17T04:33:14.910108+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:59:41.788193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.884167+0000","last_change":"2024-09-17T04:33:13.891681+0000","last_active":"2024-09-17T04:33:14.884167+0000","last_peered":"2024-09-17T04:33:14.884167+0000","last_clean":"2024-09-17T04:33:14.884167+0000","last_became_active":"2024-09-17T04:33:13.890541+0000","last_became_peered":"2024-09-17T04:33:13.890541+0000","last_unstale":"2024-09-17T04:33:14.884167+0000","last_undegraded":"2024-09-17T04:33:14.884167+0000","last_fullsized":"2024-09-17T04:33:14.884167+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:25.557822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:16.646947+0000","last_change":"2024-09-17T04:33:14.901777+0000","last_active":"2024-09-17T04:33:16.646947+0000","last_peered":"2024-09-17T04:33:16.646947+0000","last_clean":"2024-09-17T04:33:16.646947+0000","last_became_active":"2024-09-17T04:33:14.901487+0000","last_became_peered":"2024-09-17T04:33:14.901487+0000","last_unstale":"2024-09-17T04:33:16.646947+0000","last_undegraded":"2024-09-17T04:33:16.646947+0000","last_fullsized":"2024-09-17T04:33:16.646947+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:08.254493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892196+0000","last_change":"2024-09-17T04:33:13.895279+0000","last_active":"2024-09-17T04:33:15.892196+0000","last_peered":"2024-09-17T04:33:15.892196+0000","last_clean":"2024-09-17T04:33:15.892196+0000","last_became_active":"2024-09-17T04:33:13.894999+0000","last_became_peered":"2024-09-17T04:33:13.894999+0000","last_unstale":"2024-09-17T04:33:15.892196+0000","last_undegraded":"2024-09-17T04:33:15.892196+0000","last_fullsized":"2024-09-17T04:33:15.892196+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:49:09.965654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.907321+0000","last_change":"2024-09-17T04:33:14.900688+0000","last_active":"2024-09-17T04:33:14.907321+0000","last_peered":"2024-09-17T04:33:14.907321+0000","last_clean":"2024-09-17T04:33:14.907321+0000","last_became_active":"2024-09-17T04:33:14.899848+0000","last_became_peered":"2024-09-17T04:33:14.899848+0000","last_unstale":"2024-09-17T04:33:14.907321+0000","last_undegraded":"2024-09-17T04:33:14.907321+0000","last_fullsized":"2024-09-17T04:33:14.907321+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:55:18.961213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"155'3","reported_seq":18,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892170+0000","last_change":"2024-09-17T04:33:13.888601+0000","last_active":"2024-09-17T04:33:15.892170+0000","last_peered":"2024-09-17T04:33:15.892170+0000","last_clean":"2024-09-17T04:33:15.892170+0000","last_became_active":"2024-09-17T04:33:13.887279+0000","last_became_peered":"2024-09-17T04:33:13.887279+0000","last_unstale":"2024-09-17T04:33:15.892170+0000","last_undegraded":"2024-09-17T04:33:15.892170+0000","last_fullsized":"2024-09-17T04:33:15.892170+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:03:23.449677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.905917+0000","last_change":"2024-09-17T04:33:14.902064+0000","last_active":"2024-09-17T04:33:14.905917+0000","last_peered":"2024-09-17T04:33:14.905917+0000","last_clean":"2024-09-17T04:33:14.905917+0000","last_became_active":"2024-09-17T04:33:14.901762+0000","last_became_peered":"2024-09-17T04:33:14.901762+0000","last_unstale":"2024-09-17T04:33:14.905917+0000","last_undegraded":"2024-09-17T04:33:14.905917+0000","last_fullsized":"2024-09-17T04:33:14.905917+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:47:41.016028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"155'6","reported_seq":19,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.916133+0000","last_change":"2024-09-17T04:33:13.899987+0000","last_active":"2024-09-17T04:33:14.916133+0000","last_peered":"2024-09-17T04:33:14.916133+0000","last_clean":"2024-09-17T04:33:14.916133+0000","last_became_active":"2024-09-17T04:33:13.899673+0000","last_became_peered":"2024-09-17T04:33:13.899673+0000","last_unstale":"2024-09-17T04:33:14.916133+0000","last_undegraded":"2024-09-17T04:33:14.916133+0000","last_fullsized":"2024-09-17T04:33:14.916133+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:47.767511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.903381+0000","last_change":"2024-09-17T04:33:14.897603+0000","last_active":"2024-09-17T04:33:14.903381+0000","last_peered":"2024-09-17T04:33:14.903381+0000","last_clean":"2024-09-17T04:33:14.903381+0000","last_became_active":"2024-09-17T04:33:14.897308+0000","last_became_peered":"2024-09-17T04:33:14.897308+0000","last_unstale":"2024-09-17T04:33:14.903381+0000","last_undegraded":"2024-09-17T04:33:14.903381+0000","last_fullsized":"2024-09-17T04:33:14.903381+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:53:35.444966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"155'2","reported_seq":17,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892762+0000","last_change":"2024-09-17T04:33:13.890634+0000","last_active":"2024-09-17T04:33:15.892762+0000","last_peered":"2024-09-17T04:33:15.892762+0000","last_clean":"2024-09-17T04:33:15.892762+0000","last_became_active":"2024-09-17T04:33:13.889614+0000","last_became_peered":"2024-09-17T04:33:13.889614+0000","last_unstale":"2024-09-17T04:33:15.892762+0000","last_undegraded":"2024-09-17T04:33:15.892762+0000","last_fullsized":"2024-09-17T04:33:15.892762+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:07.977950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.899129+0000","last_change":"2024-09-17T04:33:14.896891+0000","last_active":"2024-09-17T04:33:14.899129+0000","last_peered":"2024-09-17T04:33:14.899129+0000","last_clean":"2024-09-17T04:33:14.899129+0000","last_became_active":"2024-09-17T04:33:14.896538+0000","last_became_peered":"2024-09-17T04:33:14.896538+0000","last_unstale":"2024-09-17T04:33:14.899129+0000","last_undegraded":"2024-09-17T04:33:14.899129+0000","last_fullsized":"2024-09-17T04:33:14.899129+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:16:09.953033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.889732+0000","last_change":"2024-09-17T04:33:13.896584+0000","last_active":"2024-09-17T04:33:14.889732+0000","last_peered":"2024-09-17T04:33:14.889732+0000","last_clean":"2024-09-17T04:33:14.889732+0000","last_became_active":"2024-09-17T04:33:13.896299+0000","last_became_peered":"2024-09-17T04:33:13.896299+0000","last_unstale":"2024-09-17T04:33:14.889732+0000","last_undegraded":"2024-09-17T04:33:14.889732+0000","last_fullsized":"2024-09-17T04:33:14.889732+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:35:23.692931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.891996+0000","last_change":"2024-09-17T04:33:14.885206+0000","last_active":"2024-09-17T04:33:15.891996+0000","last_peered":"2024-09-17T04:33:15.891996+0000","last_clean":"2024-09-17T04:33:15.891996+0000","last_became_active":"2024-09-17T04:33:14.884817+0000","last_became_peered":"2024-09-17T04:33:14.884817+0000","last_unstale":"2024-09-17T04:33:15.891996+0000","last_undegraded":"2024-09-17T04:33:15.891996+0000","last_fullsized":"2024-09-17T04:33:15.891996+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:53:07.321546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:16.646436+0000","last_change":"2024-09-17T04:33:13.890400+0000","last_active":"2024-09-17T04:33:16.646436+0000","last_peered":"2024-09-17T04:33:16.646436+0000","last_clean":"2024-09-17T04:33:16.646436+0000","last_became_active":"2024-09-17T04:33:13.890066+0000","last_became_peered":"2024-09-17T04:33:13.890066+0000","last_unstale":"2024-09-17T04:33:16.646436+0000","last_undegraded":"2024-09-17T04:33:16.646436+0000","last_fullsized":"2024-09-17T04:33:16.646436+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:50.665148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.905956+0000","last_change":"2024-09-17T04:33:14.900505+0000","last_active":"2024-09-17T04:33:14.905956+0000","last_peered":"2024-09-17T04:33:14.905956+0000","last_clean":"2024-09-17T04:33:14.905956+0000","last_became_active":"2024-09-17T04:33:14.900146+0000","last_became_peered":"2024-09-17T04:33:14.900146+0000","last_unstale":"2024-09-17T04:33:14.905956+0000","last_undegraded":"2024-09-17T04:33:14.905956+0000","last_fullsized":"2024-09-17T04:33:14.905956+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:11:00.168476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"155'8","reported_seq":23,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.891516+0000","last_change":"2024-09-17T04:33:13.882596+0000","last_active":"2024-09-17T04:33:15.891516+0000","last_peered":"2024-09-17T04:33:15.891516+0000","last_clean":"2024-09-17T04:33:15.891516+0000","last_became_active":"2024-09-17T04:33:13.881415+0000","last_became_peered":"2024-09-17T04:33:13.881415+0000","last_unstale":"2024-09-17T04:33:15.891516+0000","last_undegraded":"2024-09-17T04:33:15.891516+0000","last_fullsized":"2024-09-17T04:33:15.891516+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:24:05.580922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.899322+0000","last_change":"2024-09-17T04:33:14.897529+0000","last_active":"2024-09-17T04:33:14.899322+0000","last_peered":"2024-09-17T04:33:14.899322+0000","last_clean":"2024-09-17T04:33:14.899322+0000","last_became_active":"2024-09-17T04:33:14.897216+0000","last_became_peered":"2024-09-17T04:33:14.897216+0000","last_unstale":"2024-09-17T04:33:14.899322+0000","last_undegraded":"2024-09-17T04:33:14.899322+0000","last_fullsized":"2024-09-17T04:33:14.899322+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:12:58.993006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"155'1","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892505+0000","last_change":"2024-09-17T04:33:13.896035+0000","last_active":"2024-09-17T04:33:15.892505+0000","last_peered":"2024-09-17T04:33:15.892505+0000","last_clean":"2024-09-17T04:33:15.892505+0000","last_became_active":"2024-09-17T04:33:13.895688+0000","last_became_peered":"2024-09-17T04:33:13.895688+0000","last_unstale":"2024-09-17T04:33:15.892505+0000","last_undegraded":"2024-09-17T04:33:15.892505+0000","last_fullsized":"2024-09-17T04:33:15.892505+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:58:28.978883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.893492+0000","last_change":"2024-09-17T04:33:14.902656+0000","last_active":"2024-09-17T04:33:15.893492+0000","last_peered":"2024-09-17T04:33:15.893492+0000","last_clean":"2024-09-17T04:33:15.893492+0000","last_became_active":"2024-09-17T04:33:14.900151+0000","last_became_peered":"2024-09-17T04:33:14.900151+0000","last_unstale":"2024-09-17T04:33:15.893492+0000","last_undegraded":"2024-09-17T04:33:15.893492+0000","last_fullsized":"2024-09-17T04:33:15.893492+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:08.510677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"155'2","reported_seq":17,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:16.646931+0000","last_change":"2024-09-17T04:33:13.891610+0000","last_active":"2024-09-17T04:33:16.646931+0000","last_peered":"2024-09-17T04:33:16.646931+0000","last_clean":"2024-09-17T04:33:16.646931+0000","last_became_active":"2024-09-17T04:33:13.891205+0000","last_became_peered":"2024-09-17T04:33:13.891205+0000","last_unstale":"2024-09-17T04:33:16.646931+0000","last_undegraded":"2024-09-17T04:33:16.646931+0000","last_fullsized":"2024-09-17T04:33:16.646931+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:10.284497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892919+0000","last_change":"2024-09-17T04:33:14.892923+0000","last_active":"2024-09-17T04:33:15.892919+0000","last_peered":"2024-09-17T04:33:15.892919+0000","last_clean":"2024-09-17T04:33:15.892919+0000","last_became_active":"2024-09-17T04:33:14.892556+0000","last_became_peered":"2024-09-17T04:33:14.892556+0000","last_unstale":"2024-09-17T04:33:15.892919+0000","last_undegraded":"2024-09-17T04:33:15.892919+0000","last_fullsized":"2024-09-17T04:33:15.892919+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:07:52.563695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"155'1","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.891918+0000","last_change":"2024-09-17T04:33:13.881840+0000","last_active":"2024-09-17T04:33:15.891918+0000","last_peered":"2024-09-17T04:33:15.891918+0000","last_clean":"2024-09-17T04:33:15.891918+0000","last_became_active":"2024-09-17T04:33:13.881516+0000","last_became_peered":"2024-09-17T04:33:13.881516+0000","last_unstale":"2024-09-17T04:33:15.891918+0000","last_undegraded":"2024-09-17T04:33:15.891918+0000","last_fullsized":"2024-09-17T04:33:15.891918+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:13.368126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.908379+0000","last_change":"2024-09-17T04:33:14.903092+0000","last_active":"2024-09-17T04:33:14.908379+0000","last_peered":"2024-09-17T04:33:14.908379+0000","last_clean":"2024-09-17T04:33:14.908379+0000","last_became_active":"2024-09-17T04:33:14.902180+0000","last_became_peered":"2024-09-17T04:33:14.902180+0000","last_unstale":"2024-09-17T04:33:14.908379+0000","last_undegraded":"2024-09-17T04:33:14.908379+0000","last_fullsized":"2024-09-17T04:33:14.908379+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:25:15.108698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.885632+0000","last_change":"2024-09-17T04:33:13.892940+0000","last_active":"2024-09-17T04:33:14.885632+0000","last_peered":"2024-09-17T04:33:14.885632+0000","last_clean":"2024-09-17T04:33:14.885632+0000","last_became_active":"2024-09-17T04:33:13.892623+0000","last_became_peered":"2024-09-17T04:33:13.892623+0000","last_unstale":"2024-09-17T04:33:14.885632+0000","last_undegraded":"2024-09-17T04:33:14.885632+0000","last_fullsized":"2024-09-17T04:33:14.885632+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:38.404940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.914967+0000","last_change":"2024-09-17T04:33:14.911872+0000","last_active":"2024-09-17T04:33:14.914967+0000","last_peered":"2024-09-17T04:33:14.914967+0000","last_clean":"2024-09-17T04:33:14.914967+0000","last_became_active":"2024-09-17T04:33:14.909649+0000","last_became_peered":"2024-09-17T04:33:14.909649+0000","last_unstale":"2024-09-17T04:33:14.914967+0000","last_undegraded":"2024-09-17T04:33:14.914967+0000","last_fullsized":"2024-09-17T04:33:14.914967+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:18:18.668141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"155'2","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.909390+0000","last_change":"2024-09-17T04:33:13.899555+0000","last_active":"2024-09-17T04:33:14.909390+0000","last_peered":"2024-09-17T04:33:14.909390+0000","last_clean":"2024-09-17T04:33:14.909390+0000","last_became_active":"2024-09-17T04:33:13.899212+0000","last_became_peered":"2024-09-17T04:33:13.899212+0000","last_unstale":"2024-09-17T04:33:14.909390+0000","last_undegraded":"2024-09-17T04:33:14.909390+0000","last_fullsized":"2024-09-17T04:33:14.909390+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:07.508112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.897146+0000","last_change":"2024-09-17T04:33:14.890803+0000","last_active":"2024-09-17T04:33:14.897146+0000","last_peered":"2024-09-17T04:33:14.897146+0000","last_clean":"2024-09-17T04:33:14.897146+0000","last_became_active":"2024-09-17T04:33:14.890483+0000","last_became_peered":"2024-09-17T04:33:14.890483+0000","last_unstale":"2024-09-17T04:33:14.897146+0000","last_undegraded":"2024-09-17T04:33:14.897146+0000","last_fullsized":"2024-09-17T04:33:14.897146+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:24:05.093752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.891886+0000","last_change":"2024-09-17T04:33:13.892630+0000","last_active":"2024-09-17T04:33:15.891886+0000","last_peered":"2024-09-17T04:33:15.891886+0000","last_clean":"2024-09-17T04:33:15.891886+0000","last_became_active":"2024-09-17T04:33:13.892060+0000","last_became_peered":"2024-09-17T04:33:13.892060+0000","last_unstale":"2024-09-17T04:33:15.891886+0000","last_undegraded":"2024-09-17T04:33:15.891886+0000","last_fullsized":"2024-09-17T04:33:15.891886+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:54:32.231255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.903261+0000","last_change":"2024-09-17T04:33:14.895000+0000","last_active":"2024-09-17T04:33:14.903261+0000","last_peered":"2024-09-17T04:33:14.903261+0000","last_clean":"2024-09-17T04:33:14.903261+0000","last_became_active":"2024-09-17T04:33:14.894695+0000","last_became_peered":"2024-09-17T04:33:14.894695+0000","last_unstale":"2024-09-17T04:33:14.903261+0000","last_undegraded":"2024-09-17T04:33:14.903261+0000","last_fullsized":"2024-09-17T04:33:14.903261+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:05.253995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.880687+0000","last_change":"2024-09-17T04:33:13.899571+0000","last_active":"2024-09-17T04:33:14.880687+0000","last_peered":"2024-09-17T04:33:14.880687+0000","last_clean":"2024-09-17T04:33:14.880687+0000","last_became_active":"2024-09-17T04:33:13.898599+0000","last_became_peered":"2024-09-17T04:33:13.898599+0000","last_unstale":"2024-09-17T04:33:14.880687+0000","last_undegraded":"2024-09-17T04:33:14.880687+0000","last_fullsized":"2024-09-17T04:33:14.880687+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:14.709635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.891772+0000","last_change":"2024-09-17T04:33:14.886577+0000","last_active":"2024-09-17T04:33:15.891772+0000","last_peered":"2024-09-17T04:33:15.891772+0000","last_clean":"2024-09-17T04:33:15.891772+0000","last_became_active":"2024-09-17T04:33:14.886221+0000","last_became_peered":"2024-09-17T04:33:14.886221+0000","last_unstale":"2024-09-17T04:33:15.891772+0000","last_undegraded":"2024-09-17T04:33:15.891772+0000","last_fullsized":"2024-09-17T04:33:15.891772+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:24.261861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.891864+0000","last_change":"2024-09-17T04:33:13.892911+0000","last_active":"2024-09-17T04:33:15.891864+0000","last_peered":"2024-09-17T04:33:15.891864+0000","last_clean":"2024-09-17T04:33:15.891864+0000","last_became_active":"2024-09-17T04:33:13.892269+0000","last_became_peered":"2024-09-17T04:33:13.892269+0000","last_unstale":"2024-09-17T04:33:15.891864+0000","last_undegraded":"2024-09-17T04:33:15.891864+0000","last_fullsized":"2024-09-17T04:33:15.891864+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:49.698053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892214+0000","last_change":"2024-09-17T04:33:14.889230+0000","last_active":"2024-09-17T04:33:15.892214+0000","last_peered":"2024-09-17T04:33:15.892214+0000","last_clean":"2024-09-17T04:33:15.892214+0000","last_became_active":"2024-09-17T04:33:14.888681+0000","last_became_peered":"2024-09-17T04:33:14.888681+0000","last_unstale":"2024-09-17T04:33:15.892214+0000","last_undegraded":"2024-09-17T04:33:15.892214+0000","last_fullsized":"2024-09-17T04:33:15.892214+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:50.700618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"155'2","reported_seq":17,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892277+0000","last_change":"2024-09-17T04:33:13.895768+0000","last_active":"2024-09-17T04:33:15.892277+0000","last_peered":"2024-09-17T04:33:15.892277+0000","last_clean":"2024-09-17T04:33:15.892277+0000","last_became_active":"2024-09-17T04:33:13.895491+0000","last_became_peered":"2024-09-17T04:33:13.895491+0000","last_unstale":"2024-09-17T04:33:15.892277+0000","last_undegraded":"2024-09-17T04:33:15.892277+0000","last_fullsized":"2024-09-17T04:33:15.892277+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:02.920831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.911835+0000","last_change":"2024-09-17T04:33:14.909774+0000","last_active":"2024-09-17T04:33:14.911835+0000","last_peered":"2024-09-17T04:33:14.911835+0000","last_clean":"2024-09-17T04:33:14.911835+0000","last_became_active":"2024-09-17T04:33:14.909456+0000","last_became_peered":"2024-09-17T04:33:14.909456+0000","last_unstale":"2024-09-17T04:33:14.911835+0000","last_undegraded":"2024-09-17T04:33:14.911835+0000","last_fullsized":"2024-09-17T04:33:14.911835+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:12:18.578443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.889651+0000","last_change":"2024-09-17T04:33:13.901401+0000","last_active":"2024-09-17T04:33:14.889651+0000","last_peered":"2024-09-17T04:33:14.889651+0000","last_clean":"2024-09-17T04:33:14.889651+0000","last_became_active":"2024-09-17T04:33:13.901001+0000","last_became_peered":"2024-09-17T04:33:13.901001+0000","last_unstale":"2024-09-17T04:33:14.889651+0000","last_undegraded":"2024-09-17T04:33:14.889651+0000","last_fullsized":"2024-09-17T04:33:14.889651+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:48.103792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:16.646905+0000","last_change":"2024-09-17T04:33:14.897223+0000","last_active":"2024-09-17T04:33:16.646905+0000","last_peered":"2024-09-17T04:33:16.646905+0000","last_clean":"2024-09-17T04:33:16.646905+0000","last_became_active":"2024-09-17T04:33:14.896945+0000","last_became_peered":"2024-09-17T04:33:14.896945+0000","last_unstale":"2024-09-17T04:33:16.646905+0000","last_undegraded":"2024-09-17T04:33:16.646905+0000","last_fullsized":"2024-09-17T04:33:16.646905+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:11.647670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.888818+0000","last_change":"2024-09-17T04:33:13.896962+0000","last_active":"2024-09-17T04:33:14.888818+0000","last_peered":"2024-09-17T04:33:14.888818+0000","last_clean":"2024-09-17T04:33:14.888818+0000","last_became_active":"2024-09-17T04:33:13.896698+0000","last_became_peered":"2024-09-17T04:33:13.896698+0000","last_unstale":"2024-09-17T04:33:14.888818+0000","last_undegraded":"2024-09-17T04:33:14.888818+0000","last_fullsized":"2024-09-17T04:33:14.888818+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:51.426171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:16.647112+0000","last_change":"2024-09-17T04:33:14.889928+0000","last_active":"2024-09-17T04:33:16.647112+0000","last_peered":"2024-09-17T04:33:16.647112+0000","last_clean":"2024-09-17T04:33:16.647112+0000","last_became_active":"2024-09-17T04:33:14.889010+0000","last_became_peered":"2024-09-17T04:33:14.889010+0000","last_unstale":"2024-09-17T04:33:16.647112+0000","last_undegraded":"2024-09-17T04:33:16.647112+0000","last_fullsized":"2024-09-17T04:33:16.647112+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:23:36.301615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:16.646964+0000","last_change":"2024-09-17T04:33:13.894840+0000","last_active":"2024-09-17T04:33:16.646964+0000","last_peered":"2024-09-17T04:33:16.646964+0000","last_clean":"2024-09-17T04:33:16.646964+0000","last_became_active":"2024-09-17T04:33:13.893661+0000","last_became_peered":"2024-09-17T04:33:13.893661+0000","last_unstale":"2024-09-17T04:33:16.646964+0000","last_undegraded":"2024-09-17T04:33:16.646964+0000","last_fullsized":"2024-09-17T04:33:16.646964+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:37.002449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892363+0000","last_change":"2024-09-17T04:33:14.910533+0000","last_active":"2024-09-17T04:33:15.892363+0000","last_peered":"2024-09-17T04:33:15.892363+0000","last_clean":"2024-09-17T04:33:15.892363+0000","last_became_active":"2024-09-17T04:33:14.910196+0000","last_became_peered":"2024-09-17T04:33:14.910196+0000","last_unstale":"2024-09-17T04:33:15.892363+0000","last_undegraded":"2024-09-17T04:33:15.892363+0000","last_fullsized":"2024-09-17T04:33:15.892363+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:32.340488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892617+0000","last_change":"2024-09-17T04:33:13.889252+0000","last_active":"2024-09-17T04:33:15.892617+0000","last_peered":"2024-09-17T04:33:15.892617+0000","last_clean":"2024-09-17T04:33:15.892617+0000","last_became_active":"2024-09-17T04:33:13.888943+0000","last_became_peered":"2024-09-17T04:33:13.888943+0000","last_unstale":"2024-09-17T04:33:15.892617+0000","last_undegraded":"2024-09-17T04:33:15.892617+0000","last_fullsized":"2024-09-17T04:33:15.892617+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:47.164060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892837+0000","last_change":"2024-09-17T04:33:14.908784+0000","last_active":"2024-09-17T04:33:15.892837+0000","last_peered":"2024-09-17T04:33:15.892837+0000","last_clean":"2024-09-17T04:33:15.892837+0000","last_became_active":"2024-09-17T04:33:14.908069+0000","last_became_peered":"2024-09-17T04:33:14.908069+0000","last_unstale":"2024-09-17T04:33:15.892837+0000","last_undegraded":"2024-09-17T04:33:15.892837+0000","last_fullsized":"2024-09-17T04:33:15.892837+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:49:26.623232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.892176+0000","last_change":"2024-09-17T04:33:13.891080+0000","last_active":"2024-09-17T04:33:15.892176+0000","last_peered":"2024-09-17T04:33:15.892176+0000","last_clean":"2024-09-17T04:33:15.892176+0000","last_became_active":"2024-09-17T04:33:13.890381+0000","last_became_peered":"2024-09-17T04:33:13.890381+0000","last_unstale":"2024-09-17T04:33:15.892176+0000","last_undegraded":"2024-09-17T04:33:15.892176+0000","last_fullsized":"2024-09-17T04:33:15.892176+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:54.808645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.912673+0000","last_change":"2024-09-17T04:33:14.909599+0000","last_active":"2024-09-17T04:33:14.912673+0000","last_peered":"2024-09-17T04:33:14.912673+0000","last_clean":"2024-09-17T04:33:14.912673+0000","last_became_active":"2024-09-17T04:33:14.909168+0000","last_became_peered":"2024-09-17T04:33:14.909168+0000","last_unstale":"2024-09-17T04:33:14.912673+0000","last_undegraded":"2024-09-17T04:33:14.912673+0000","last_fullsized":"2024-09-17T04:33:14.912673+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:45:58.483864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.884717+0000","last_change":"2024-09-17T04:33:13.889911+0000","last_active":"2024-09-17T04:33:14.884717+0000","last_peered":"2024-09-17T04:33:14.884717+0000","last_clean":"2024-09-17T04:33:14.884717+0000","last_became_active":"2024-09-17T04:33:13.889533+0000","last_became_peered":"2024-09-17T04:33:13.889533+0000","last_unstale":"2024-09-17T04:33:14.884717+0000","last_undegraded":"2024-09-17T04:33:14.884717+0000","last_fullsized":"2024-09-17T04:33:14.884717+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:07:51.561130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.891839+0000","last_change":"2024-09-17T04:33:14.893431+0000","last_active":"2024-09-17T04:33:15.891839+0000","last_peered":"2024-09-17T04:33:15.891839+0000","last_clean":"2024-09-17T04:33:15.891839+0000","last_became_active":"2024-09-17T04:33:14.893115+0000","last_became_peered":"2024-09-17T04:33:14.893115+0000","last_unstale":"2024-09-17T04:33:15.891839+0000","last_undegraded":"2024-09-17T04:33:15.891839+0000","last_fullsized":"2024-09-17T04:33:15.891839+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:02:24.692768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.881064+0000","last_change":"2024-09-17T04:33:13.900366+0000","last_active":"2024-09-17T04:33:14.881064+0000","last_peered":"2024-09-17T04:33:14.881064+0000","last_clean":"2024-09-17T04:33:14.881064+0000","last_became_active":"2024-09-17T04:33:13.899454+0000","last_became_peered":"2024-09-17T04:33:13.899454+0000","last_unstale":"2024-09-17T04:33:14.881064+0000","last_undegraded":"2024-09-17T04:33:14.881064+0000","last_fullsized":"2024-09-17T04:33:14.881064+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:12.710644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T04:33:14.895447+0000","last_change":"2024-09-17T04:33:14.890132+0000","last_active":"2024-09-17T04:33:14.895447+0000","last_peered":"2024-09-17T04:33:14.895447+0000","last_clean":"2024-09-17T04:33:14.895447+0000","last_became_active":"2024-09-17T04:33:14.889693+0000","last_became_peered":"2024-09-17T04:33:14.889693+0000","last_unstale":"2024-09-17T04:33:14.895447+0000","last_undegraded":"2024-09-17T04:33:14.895447+0000","last_fullsized":"2024-09-17T04:33:14.895447+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:13.860921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:13.860921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:35:38.838913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.891536+0000","last_change":"2024-09-17T04:33:13.888354+0000","last_active":"2024-09-17T04:33:15.891536+0000","last_peered":"2024-09-17T04:33:15.891536+0000","last_clean":"2024-09-17T04:33:15.891536+0000","last_became_active":"2024-09-17T04:33:13.888017+0000","last_became_peered":"2024-09-17T04:33:13.888017+0000","last_unstale":"2024-09-17T04:33:15.891536+0000","last_undegraded":"2024-09-17T04:33:15.891536+0000","last_fullsized":"2024-09-17T04:33:15.891536+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:12.858356+0000","last_clean_scrub_stamp":"2024-09-17T04:33:12.858356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:18:53.377553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"76'79","reported_seq":385,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T04:33:15.891603+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:33:15.891603+0000","last_peered":"2024-09-17T04:33:15.891603+0000","last_clean":"2024-09-17T04:33:15.891603+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:33:15.891603+0000","last_undegraded":"2024-09-17T04:33:15.891603+0000","last_fullsized":"2024-09-17T04:33:15.891603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33684,"kb_used_data":2496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338156,"statfs":{"total":96636764160,"available":96602271744,"internally_reserved":0,"allocated":2555904,"data_stored":1453578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1941504,"data_stored":856750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476852,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33096,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338744,"statfs":{"total":96636764160,"available":96602873856,"internally_reserved":0,"allocated":1953792,"data_stored":863106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33100,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338740,"statfs":{"total":96636764160,"available":96602869760,"internally_reserved":0,"allocated":1957888,"data_stored":858944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542259,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33096,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338744,"statfs":{"total":96636764160,"available":96602873856,"internally_reserved":0,"allocated":1953792,"data_stored":856944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509555,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33112,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338728,"statfs":{"total":96636764160,"available":96602857472,"internally_reserved":0,"allocated":1966080,"data_stored":863800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509554,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33096,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338744,"statfs":{"total":96636764160,"available":96602873856,"internally_reserved":0,"allocated":1953792,"data_stored":858362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509554,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33668,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338172,"statfs":{"total":96636764160,"available":96602288128,"internally_reserved":0,"allocated":2539520,"data_stored":1447140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:33:19.795 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T04:33:19.796 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T04:33:20.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.063+0000 7f736d200640 1 -- 172.21.15.100:0/4022239644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7368160370 msgr2=0x7f73680ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:20.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.063+0000 7f736d200640 1 --2- 172.21.15.100:0/4022239644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7368160370 0x7f73680ccb30 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f7350005fd0 tx=0x7f735002a540 comp rx=0 tx=0).stop 2024-09-17T04:33:20.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.063+0000 7f736d200640 1 -- 172.21.15.100:0/4022239644 shutdown_connections 2024-09-17T04:33:20.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.063+0000 7f736d200640 1 --2- 172.21.15.100:0/4022239644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7368160370 0x7f73680ccb30 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.063+0000 7f736d200640 1 --2- 172.21.15.100:0/4022239644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73680af4e0 0x7f736815d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.063+0000 7f736d200640 1 -- 172.21.15.100:0/4022239644 >> 172.21.15.100:0/4022239644 conn(0x7f73680b1dd0 msgr2=0x7f73680b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:20.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.063+0000 7f736d200640 1 -- 172.21.15.100:0/4022239644 shutdown_connections 2024-09-17T04:33:20.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.063+0000 7f736d200640 1 -- 172.21.15.100:0/4022239644 wait complete. 2024-09-17T04:33:20.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.065+0000 7f736d200640 1 Processor -- start 2024-09-17T04:33:20.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.066+0000 7f736d200640 1 -- start start 2024-09-17T04:33:20.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.067+0000 7f736d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73680af4e0 0x7f73681f8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:20.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.067+0000 7f736d200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7368160370 0x7f73681496b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:20.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.067+0000 7f736d200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7368160780 0x7f736809d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:20.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.067+0000 7f736d200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f73680cd400 con 0x7f73680af4e0 2024-09-17T04:33:20.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.067+0000 7f736d200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f73680cd280 con 0x7f7368160780 2024-09-17T04:33:20.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.067+0000 7f736d200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f73680b46d0 con 0x7f7368160370 2024-09-17T04:33:20.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.068+0000 7f7367400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73680af4e0 0x7f73681f8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:20.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.068+0000 7f7367400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73680af4e0 0x7f73681f8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43012/0 (socket says 172.21.15.100:43012) 2024-09-17T04:33:20.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.068+0000 7f7367400640 1 -- 172.21.15.100:0/4044656131 learned_addr learned my addr 172.21.15.100:0/4044656131 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:20.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.068+0000 7f735ea00640 1 --2- 172.21.15.100:0/4044656131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7368160370 0x7f73681496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:20.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.068+0000 7f7367e00640 1 --2- 172.21.15.100:0/4044656131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7368160780 0x7f736809d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:20.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.068+0000 7f7367400640 1 -- 172.21.15.100:0/4044656131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7368160370 msgr2=0x7f73681496b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:20.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.068+0000 7f7367400640 1 --2- 172.21.15.100:0/4044656131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7368160370 0x7f73681496b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.068+0000 7f7367400640 1 -- 172.21.15.100:0/4044656131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7368160780 msgr2=0x7f736809d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:20.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.068+0000 7f7367400640 1 --2- 172.21.15.100:0/4044656131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7368160780 0x7f736809d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.068+0000 7f7367400640 1 -- 172.21.15.100:0/4044656131 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73681f9220 con 0x7f73680af4e0 2024-09-17T04:33:20.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.068+0000 7f7367400640 1 --2- 172.21.15.100:0/4044656131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73680af4e0 0x7f73681f8bd0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f735800ea20 tx=0x7f735800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:20.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.069+0000 7f735fe00640 1 -- 172.21.15.100:0/4044656131 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735800cd80 con 0x7f73680af4e0 2024-09-17T04:33:20.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.069+0000 7f735fe00640 1 -- 172.21.15.100:0/4044656131 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7358011560 con 0x7f73680af4e0 2024-09-17T04:33:20.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.069+0000 7f736d200640 1 -- 172.21.15.100:0/4044656131 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73681f9500 con 0x7f73680af4e0 2024-09-17T04:33:20.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.069+0000 7f735fe00640 1 -- 172.21.15.100:0/4044656131 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7358011990 con 0x7f73680af4e0 2024-09-17T04:33:20.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.069+0000 7f736d200640 1 -- 172.21.15.100:0/4044656131 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73681f9ac0 con 0x7f73680af4e0 2024-09-17T04:33:20.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.070+0000 7f736d200640 1 -- 172.21.15.100:0/4044656131 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73680b4bf0 con 0x7f73680af4e0 2024-09-17T04:33:20.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.071+0000 7f735fe00640 1 -- 172.21.15.100:0/4044656131 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f7358011b30 con 0x7f73680af4e0 2024-09-17T04:33:20.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.072+0000 7f735fe00640 1 --2- 172.21.15.100:0/4044656131 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7344079400 0x7f734407b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:20.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.073+0000 7f735fe00640 1 -- 172.21.15.100:0/4044656131 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7f735809cad0 con 0x7f73680af4e0 2024-09-17T04:33:20.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.073+0000 7f735ea00640 1 --2- 172.21.15.100:0/4044656131 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7344079400 0x7f734407b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:20.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.073+0000 7f735ea00640 1 --2- 172.21.15.100:0/4044656131 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7344079400 0x7f734407b910 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7350033000 tx=0x7f7350004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:20.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.077+0000 7f735fe00640 1 -- 172.21.15.100:0/4044656131 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f7358016030 con 0x7f73680af4e0 2024-09-17T04:33:20.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.325+0000 7f736d200640 1 -- 172.21.15.100:0/4044656131 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f73680704f0 con 0x7f73680af4e0 2024-09-17T04:33:20.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.328+0000 7f735fe00640 1 -- 172.21.15.100:0/4044656131 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) ==== 190+0+147 (secure 0 0 0) 0x7f7358011e20 con 0x7f73680af4e0 2024-09-17T04:33:20.329 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-09-17T04:33:20.329 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAsBOlmfYg9IBAABP1uRKGOugrA47VxwRLkjA== 2024-09-17T04:33:20.329 INFO:teuthology.orchestra.run.smithi100.stdout: caps mds = "allow" 2024-09-17T04:33:20.329 INFO:teuthology.orchestra.run.smithi100.stdout: caps mon = "allow r" 2024-09-17T04:33:20.329 INFO:teuthology.orchestra.run.smithi100.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T04:33:20.329 INFO:teuthology.orchestra.run.smithi100.stderr:updated caps for client.0 2024-09-17T04:33:20.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.331+0000 7f736d200640 1 -- 172.21.15.100:0/4044656131 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7344079400 msgr2=0x7f734407b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:20.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.331+0000 7f736d200640 1 --2- 172.21.15.100:0/4044656131 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7344079400 0x7f734407b910 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7350033000 tx=0x7f7350004730 comp rx=0 tx=0).stop 2024-09-17T04:33:20.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.331+0000 7f736d200640 1 -- 172.21.15.100:0/4044656131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73680af4e0 msgr2=0x7f73681f8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:20.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.331+0000 7f736d200640 1 --2- 172.21.15.100:0/4044656131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73680af4e0 0x7f73681f8bd0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f735800ea20 tx=0x7f735800eee0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.331+0000 7f736d200640 1 -- 172.21.15.100:0/4044656131 shutdown_connections 2024-09-17T04:33:20.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.331+0000 7f736d200640 1 --2- 172.21.15.100:0/4044656131 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7344079400 0x7f734407b910 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.331+0000 7f736d200640 1 --2- 172.21.15.100:0/4044656131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7368160780 0x7f736809d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.331+0000 7f736d200640 1 --2- 172.21.15.100:0/4044656131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7368160370 0x7f73681496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.331+0000 7f736d200640 1 --2- 172.21.15.100:0/4044656131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73680af4e0 0x7f73681f8bd0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.331+0000 7f736d200640 1 -- 172.21.15.100:0/4044656131 >> 172.21.15.100:0/4044656131 conn(0x7f73680b1dd0 msgr2=0x7f73680b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:20.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.331+0000 7f736d200640 1 -- 172.21.15.100:0/4044656131 shutdown_connections 2024-09-17T04:33:20.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.331+0000 7f736d200640 1 -- 172.21.15.100:0/4044656131 wait complete. 2024-09-17T04:33:20.356 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:33:20.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.575+0000 7fbcc1000640 1 -- 172.21.15.100:0/1254640201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcbc0707b0 msgr2=0x7fbcbc06f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:20.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.575+0000 7fbcc1000640 1 --2- 172.21.15.100:0/1254640201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcbc0707b0 0x7fbcbc06f930 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fbcac005fd0 tx=0x7fbcac02a540 comp rx=0 tx=0).stop 2024-09-17T04:33:20.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.575+0000 7fbcc1000640 1 -- 172.21.15.100:0/1254640201 shutdown_connections 2024-09-17T04:33:20.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.575+0000 7fbcc1000640 1 --2- 172.21.15.100:0/1254640201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcbc0b4110 0x7fbcbc06fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.575+0000 7fbcc1000640 1 --2- 172.21.15.100:0/1254640201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcbc0707b0 0x7fbcbc06f930 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.575+0000 7fbcc1000640 1 -- 172.21.15.100:0/1254640201 >> 172.21.15.100:0/1254640201 conn(0x7fbcbc06d730 msgr2=0x7fbcbc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:20.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.575+0000 7fbcc1000640 1 -- 172.21.15.100:0/1254640201 shutdown_connections 2024-09-17T04:33:20.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.575+0000 7fbcc1000640 1 -- 172.21.15.100:0/1254640201 wait complete. 2024-09-17T04:33:20.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.578+0000 7fbcc1000640 1 Processor -- start 2024-09-17T04:33:20.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.578+0000 7fbcc1000640 1 -- start start 2024-09-17T04:33:20.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.579+0000 7fbcc1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcbc0707b0 0x7fbcbc1f5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:20.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.579+0000 7fbcc1000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcbc0730b0 0x7fbcbc151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:20.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.579+0000 7fbcc1000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcbc0b4110 0x7fbcbc1e8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:20.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.579+0000 7fbcc1000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbcbc06f0a0 con 0x7fbcbc0707b0 2024-09-17T04:33:20.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.579+0000 7fbcc1000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbcbc15b2b0 con 0x7fbcbc0730b0 2024-09-17T04:33:20.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.579+0000 7fbcc1000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbcbc06f220 con 0x7fbcbc0b4110 2024-09-17T04:33:20.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.580+0000 7fbcb9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcbc0707b0 0x7fbcbc1f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:20.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.580+0000 7fbcb9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcbc0707b0 0x7fbcbc1f5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43036/0 (socket says 172.21.15.100:43036) 2024-09-17T04:33:20.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.580+0000 7fbcb9e00640 1 -- 172.21.15.100:0/803754582 learned_addr learned my addr 172.21.15.100:0/803754582 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:20.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.580+0000 7fbcba800640 1 --2- 172.21.15.100:0/803754582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcbc0b4110 0x7fbcbc1e8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:20.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.580+0000 7fbcb1400640 1 --2- 172.21.15.100:0/803754582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcbc0730b0 0x7fbcbc151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:20.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.580+0000 7fbcb9e00640 1 -- 172.21.15.100:0/803754582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcbc0b4110 msgr2=0x7fbcbc1e8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:20.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.580+0000 7fbcb9e00640 1 --2- 172.21.15.100:0/803754582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcbc0b4110 0x7fbcbc1e8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.580+0000 7fbcb9e00640 1 -- 172.21.15.100:0/803754582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcbc0730b0 msgr2=0x7fbcbc151b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:20.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.580+0000 7fbcb9e00640 1 --2- 172.21.15.100:0/803754582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcbc0730b0 0x7fbcbc151b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.580+0000 7fbcb9e00640 1 -- 172.21.15.100:0/803754582 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcbc1f57b0 con 0x7fbcbc0707b0 2024-09-17T04:33:20.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.580+0000 7fbcb9e00640 1 --2- 172.21.15.100:0/803754582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcbc0707b0 0x7fbcbc1f5270 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fbcac034040 tx=0x7fbcac0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:20.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.581+0000 7fbcb2a00640 1 -- 172.21.15.100:0/803754582 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcac03e030 con 0x7fbcbc0707b0 2024-09-17T04:33:20.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.581+0000 7fbcb2a00640 1 -- 172.21.15.100:0/803754582 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcac003d90 con 0x7fbcbc0707b0 2024-09-17T04:33:20.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.581+0000 7fbcc1000640 1 -- 172.21.15.100:0/803754582 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcbc13d780 con 0x7fbcbc0707b0 2024-09-17T04:33:20.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.581+0000 7fbcb2a00640 1 -- 172.21.15.100:0/803754582 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcac035040 con 0x7fbcbc0707b0 2024-09-17T04:33:20.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.581+0000 7fbcc1000640 1 -- 172.21.15.100:0/803754582 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcbc13dd40 con 0x7fbcbc0707b0 2024-09-17T04:33:20.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.582+0000 7fbcc1000640 1 -- 172.21.15.100:0/803754582 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcbc06f6d0 con 0x7fbcbc0707b0 2024-09-17T04:33:20.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.583+0000 7fbcb2a00640 1 -- 172.21.15.100:0/803754582 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7fbcac041030 con 0x7fbcbc0707b0 2024-09-17T04:33:20.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.584+0000 7fbcb2a00640 1 --2- 172.21.15.100:0/803754582 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbc98079430 0x7fbc9807b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:20.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.584+0000 7fbcb2a00640 1 -- 172.21.15.100:0/803754582 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7fbcac0ba120 con 0x7fbcbc0707b0 2024-09-17T04:33:20.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.584+0000 7fbcb1400640 1 --2- 172.21.15.100:0/803754582 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbc98079430 0x7fbc9807b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:20.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.585+0000 7fbcb1400640 1 --2- 172.21.15.100:0/803754582 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbc98079430 0x7fbc9807b940 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc155030 tx=0x7fbca4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:20.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.588+0000 7fbcb2a00640 1 -- 172.21.15.100:0/803754582 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fbcac0bee20 con 0x7fbcbc0707b0 2024-09-17T04:33:20.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.806+0000 7fbcc1000640 1 -- 172.21.15.100:0/803754582 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbcbc1569c0 con 0x7fbcbc0707b0 2024-09-17T04:33:20.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.807+0000 7fbcb2a00640 1 -- 172.21.15.100:0/803754582 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5245 (secure 0 0 0) 0x7fbcac084010 con 0x7fbcbc0707b0 2024-09-17T04:33:20.807 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:20.807 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":111,"btime":"2024-09-17T04:33:17:896778+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7295,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1131376012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1131376012},{"type":"v1","addr":"172.21.15.184:6801","nonce":1131376012}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":8230,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/3930702908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3930702908},{"type":"v1","addr":"172.21.15.100:6833","nonce":3930702908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":8808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2238795078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2238795078},{"type":"v1","addr":"172.21.15.100:6835","nonce":2238795078}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:14.363111+0000","modified":"2024-09-17T04:33:17.378067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8814":{"gid":8814,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/2965326201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2965326201},{"type":"v1","addr":"172.21.15.184:6835","nonce":2965326201}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8814,"qdb_cluster":[8814]},"id":12}]} 2024-09-17T04:33:20.808 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 111 2024-09-17T04:33:20.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.809+0000 7fbcc1000640 1 -- 172.21.15.100:0/803754582 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbc98079430 msgr2=0x7fbc9807b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:20.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.809+0000 7fbcc1000640 1 --2- 172.21.15.100:0/803754582 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbc98079430 0x7fbc9807b940 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc155030 tx=0x7fbca4009040 comp rx=0 tx=0).stop 2024-09-17T04:33:20.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.809+0000 7fbcc1000640 1 -- 172.21.15.100:0/803754582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcbc0707b0 msgr2=0x7fbcbc1f5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:20.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.809+0000 7fbcc1000640 1 --2- 172.21.15.100:0/803754582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcbc0707b0 0x7fbcbc1f5270 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fbcac034040 tx=0x7fbcac0038a0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.809+0000 7fbcc1000640 1 -- 172.21.15.100:0/803754582 shutdown_connections 2024-09-17T04:33:20.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.809+0000 7fbcc1000640 1 --2- 172.21.15.100:0/803754582 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbc98079430 0x7fbc9807b940 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.809+0000 7fbcc1000640 1 --2- 172.21.15.100:0/803754582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcbc0b4110 0x7fbcbc1e8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.809+0000 7fbcc1000640 1 --2- 172.21.15.100:0/803754582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcbc0730b0 0x7fbcbc151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.809+0000 7fbcc1000640 1 --2- 172.21.15.100:0/803754582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcbc0707b0 0x7fbcbc1f5270 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.809+0000 7fbcc1000640 1 -- 172.21.15.100:0/803754582 >> 172.21.15.100:0/803754582 conn(0x7fbcbc06d730 msgr2=0x7fbcbc0af080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:20.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.809+0000 7fbcc1000640 1 -- 172.21.15.100:0/803754582 shutdown_connections 2024-09-17T04:33:20.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:20.809+0000 7fbcc1000640 1 -- 172.21.15.100:0/803754582 wait complete. 2024-09-17T04:33:21.234 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 111, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T04:33:14.363111+0000', 'modified': '2024-09-17T04:33:17.378067+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8814}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8814': {'gid': 8814, 'name': 'b', 'rank': 0, 'incarnation': 107, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.184:6835/2965326201', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6834', 'nonce': 2965326201}, {'type': 'v1', 'addr': '172.21.15.184:6835', 'nonce': 2965326201}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8814, 'qdb_cluster': [8814]} 2024-09-17T04:33:21.234 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T04:33:21.234 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-17T04:33:21.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.455+0000 7fd136c00640 1 -- 172.21.15.100:0/2770507774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd130070cc0 msgr2=0x7fd130156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:21.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.455+0000 7fd136c00640 1 --2- 172.21.15.100:0/2770507774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd130070cc0 0x7fd130156e40 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fd118009900 tx=0x7fd11802a4d0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.455+0000 7fd136c00640 1 -- 172.21.15.100:0/2770507774 shutdown_connections 2024-09-17T04:33:21.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.455+0000 7fd136c00640 1 --2- 172.21.15.100:0/2770507774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd13015ce30 0x7fd13015d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.455+0000 7fd136c00640 1 --2- 172.21.15.100:0/2770507774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd130070cc0 0x7fd130156e40 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.455+0000 7fd136c00640 1 -- 172.21.15.100:0/2770507774 >> 172.21.15.100:0/2770507774 conn(0x7fd13006d3d0 msgr2=0x7fd1300b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:21.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.455+0000 7fd136c00640 1 -- 172.21.15.100:0/2770507774 shutdown_connections 2024-09-17T04:33:21.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.455+0000 7fd136c00640 1 -- 172.21.15.100:0/2770507774 wait complete. 2024-09-17T04:33:21.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.457+0000 7fd136c00640 1 Processor -- start 2024-09-17T04:33:21.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.457+0000 7fd136c00640 1 -- start start 2024-09-17T04:33:21.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.459+0000 7fd136c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1300cfc80 0x7fd1300c6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:21.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.459+0000 7fd136c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd13015ce30 0x7fd1300c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:21.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.459+0000 7fd136c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1301f3b40 0x7fd1301f78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:21.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.459+0000 7fd136c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd1300cd050 con 0x7fd1300cfc80 2024-09-17T04:33:21.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.459+0000 7fd136c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd1300cced0 con 0x7fd1301f3b40 2024-09-17T04:33:21.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.459+0000 7fd136c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd1300cd1d0 con 0x7fd13015ce30 2024-09-17T04:33:21.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.459+0000 7fd12fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1300cfc80 0x7fd1300c6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:21.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.459+0000 7fd12fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1300cfc80 0x7fd1300c6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43048/0 (socket says 172.21.15.100:43048) 2024-09-17T04:33:21.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.459+0000 7fd12fe00640 1 -- 172.21.15.100:0/1992600791 learned_addr learned my addr 172.21.15.100:0/1992600791 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:21.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.459+0000 7fd127400640 1 --2- 172.21.15.100:0/1992600791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd13015ce30 0x7fd1300c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:21.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.459+0000 7fd134c00640 1 --2- 172.21.15.100:0/1992600791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1301f3b40 0x7fd1301f78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:21.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.459+0000 7fd12fe00640 1 -- 172.21.15.100:0/1992600791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd13015ce30 msgr2=0x7fd1300c73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:21.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.459+0000 7fd12fe00640 1 --2- 172.21.15.100:0/1992600791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd13015ce30 0x7fd1300c73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.459+0000 7fd12fe00640 1 -- 172.21.15.100:0/1992600791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1301f3b40 msgr2=0x7fd1301f78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:21.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.459+0000 7fd12fe00640 1 --2- 172.21.15.100:0/1992600791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1301f3b40 0x7fd1301f78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.460+0000 7fd12fe00640 1 -- 172.21.15.100:0/1992600791 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1300c78e0 con 0x7fd1300cfc80 2024-09-17T04:33:21.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.460+0000 7fd134c00640 1 --2- 172.21.15.100:0/1992600791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1301f3b40 0x7fd1301f78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:21.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.460+0000 7fd12fe00640 1 --2- 172.21.15.100:0/1992600791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1300cfc80 0x7fd1300c6250 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fd12000b3f0 tx=0x7fd12000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:21.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.460+0000 7fd12cc00640 1 -- 172.21.15.100:0/1992600791 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120015020 con 0x7fd1300cfc80 2024-09-17T04:33:21.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.460+0000 7fd12cc00640 1 -- 172.21.15.100:0/1992600791 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd120011920 con 0x7fd1300cfc80 2024-09-17T04:33:21.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.460+0000 7fd12cc00640 1 -- 172.21.15.100:0/1992600791 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120011d50 con 0x7fd1300cfc80 2024-09-17T04:33:21.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.460+0000 7fd136c00640 1 -- 172.21.15.100:0/1992600791 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1301f4360 con 0x7fd1300cfc80 2024-09-17T04:33:21.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.460+0000 7fd136c00640 1 -- 172.21.15.100:0/1992600791 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1301f4920 con 0x7fd1300cfc80 2024-09-17T04:33:21.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.463+0000 7fd12cc00640 1 -- 172.21.15.100:0/1992600791 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7fd120018020 con 0x7fd1300cfc80 2024-09-17T04:33:21.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.463+0000 7fd12cc00640 1 --2- 172.21.15.100:0/1992600791 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd10c079400 0x7fd10c07b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:21.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.464+0000 7fd12cc00640 1 -- 172.21.15.100:0/1992600791 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7fd12009c020 con 0x7fd1300cfc80 2024-09-17T04:33:21.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.464+0000 7fd127400640 1 --2- 172.21.15.100:0/1992600791 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd10c079400 0x7fd10c07b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:21.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.464+0000 7fd13dbca740 1 Processor -- start 2024-09-17T04:33:21.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.464+0000 7fd13dbca740 1 -- start start 2024-09-17T04:33:21.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.464+0000 7fd127400640 1 --2- 172.21.15.100:0/1992600791 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd10c079400 0x7fd10c07b910 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd118033000 tx=0x7fd118004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:21.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.466+0000 7fd13dbca740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a2d8e3270 0x561a2dcb3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:21.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.466+0000 7fd13dbca740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561a2d8ba3b0 0x561a2dcb3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:21.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.466+0000 7fd13dbca740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a2d8c2590 0x561a2dc83f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:21.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.466+0000 7fd13dbca740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561a2dc0d430 con 0x561a2d8e3270 2024-09-17T04:33:21.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.466+0000 7fd13dbca740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561a2dbfd5d0 con 0x561a2d8c2590 2024-09-17T04:33:21.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.466+0000 7fd13dbca740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x561a2dbeca00 con 0x561a2d8ba3b0 2024-09-17T04:33:21.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.466+0000 7fd12fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a2d8e3270 0x561a2dcb3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:21.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.466+0000 7fd12fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a2d8e3270 0x561a2dcb3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43050/0 (socket says 172.21.15.100:43050) 2024-09-17T04:33:21.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.466+0000 7fd12fe00640 1 -- 172.21.15.100:0/2492534435 learned_addr learned my addr 172.21.15.100:0/2492534435 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:21.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.466+0000 7fd127400640 1 --2- 172.21.15.100:0/2492534435 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561a2d8ba3b0 0x561a2dcb3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:21.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.466+0000 7fd134c00640 1 --2- 172.21.15.100:0/2492534435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a2d8c2590 0x561a2dc83f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:21.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.466+0000 7fd12fe00640 1 -- 172.21.15.100:0/2492534435 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561a2d8ba3b0 msgr2=0x561a2dcb3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:33:21.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.466+0000 7fd12fe00640 1 --2- 172.21.15.100:0/2492534435 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561a2d8ba3b0 0x561a2dcb3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.466+0000 7fd12fe00640 1 -- 172.21.15.100:0/2492534435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a2d8c2590 msgr2=0x561a2dc83f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:33:21.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.466+0000 7fd12fe00640 1 --2- 172.21.15.100:0/2492534435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a2d8c2590 0x561a2dc83f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.466+0000 7fd12fe00640 1 -- 172.21.15.100:0/2492534435 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a2dbb5520 con 0x561a2d8e3270 2024-09-17T04:33:21.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.466+0000 7fd12fe00640 1 --2- 172.21.15.100:0/2492534435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a2d8e3270 0x561a2dcb3970 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fd12000b3c0 tx=0x7fd120062800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a349c349724382c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:21.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.467+0000 7fd11fe00640 1 -- 172.21.15.100:0/2492534435 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120089030 con 0x561a2d8e3270 2024-09-17T04:33:21.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.467+0000 7fd11fe00640 1 -- 172.21.15.100:0/2492534435 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd12006ae40 con 0x561a2d8e3270 2024-09-17T04:33:21.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.467+0000 7fd11fe00640 1 -- 172.21.15.100:0/2492534435 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120062170 con 0x561a2d8e3270 2024-09-17T04:33:21.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.467+0000 7fd13dbca740 1 -- 172.21.15.100:0/2492534435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a2d8e3270 msgr2=0x561a2dcb3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:21.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.467+0000 7fd13dbca740 1 --2- 172.21.15.100:0/2492534435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a2d8e3270 0x561a2dcb3970 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fd12000b3c0 tx=0x7fd120062800 comp rx=0 tx=0).stop 2024-09-17T04:33:21.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.468+0000 7fd13dbca740 1 -- 172.21.15.100:0/2492534435 shutdown_connections 2024-09-17T04:33:21.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.468+0000 7fd13dbca740 1 --2- 172.21.15.100:0/2492534435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a2d8c2590 0x561a2dc83f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.468+0000 7fd13dbca740 1 --2- 172.21.15.100:0/2492534435 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561a2d8ba3b0 0x561a2dcb3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.468+0000 7fd13dbca740 1 --2- 172.21.15.100:0/2492534435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a2d8e3270 0x561a2dcb3970 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.468+0000 7fd13dbca740 1 -- 172.21.15.100:0/2492534435 >> 172.21.15.100:0/2492534435 conn(0x561a2d8d73c0 msgr2=0x561a2dc89970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:21.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.468+0000 7fd13dbca740 1 -- 172.21.15.100:0/2492534435 shutdown_connections 2024-09-17T04:33:21.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.468+0000 7fd13dbca740 1 -- 172.21.15.100:0/2492534435 wait complete. 2024-09-17T04:33:21.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.469+0000 7fd13dbca740 1 -- start start 2024-09-17T04:33:21.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.469+0000 7fd13dbca740 1 Processor -- start 2024-09-17T04:33:21.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.470+0000 7fd13dbca740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a2d8ba3b0 0x561a2dca9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:21.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.470+0000 7fd13dbca740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a2d8c2590 0x561a2dcc76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:21.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.470+0000 7fd13dbca740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561a2d8e3270 0x561a2dcd1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:21.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.470+0000 7fd13dbca740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561a2db93330 con 0x561a2d8ba3b0 2024-09-17T04:33:21.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.470+0000 7fd13dbca740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561a2dbf0810 con 0x561a2d8c2590 2024-09-17T04:33:21.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.470+0000 7fd13dbca740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x561a2d95d620 con 0x561a2d8e3270 2024-09-17T04:33:21.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.470+0000 7fd13dbca740 10 client.0 _ll_register_callbacks cb 0x561a2db98420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:33:21.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.471+0000 7fd12fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a2d8ba3b0 0x561a2dca9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:21.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.471+0000 7fd12fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a2d8ba3b0 0x561a2dca9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43052/0 (socket says 172.21.15.100:43052) 2024-09-17T04:33:21.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.471+0000 7fd12fe00640 1 -- 172.21.15.100:0/2758092468 learned_addr learned my addr 172.21.15.100:0/2758092468 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:21.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.471+0000 7fd134c00640 1 --2- 172.21.15.100:0/2758092468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561a2d8e3270 0x561a2dcd1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:21.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.471+0000 7fd127400640 1 --2- 172.21.15.100:0/2758092468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a2d8c2590 0x561a2dcc76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:21.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.471+0000 7fd12fe00640 1 -- 172.21.15.100:0/2758092468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561a2d8e3270 msgr2=0x561a2dcd1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:33:21.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.471+0000 7fd12fe00640 1 --2- 172.21.15.100:0/2758092468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561a2d8e3270 0x561a2dcd1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.471+0000 7fd12fe00640 1 -- 172.21.15.100:0/2758092468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a2d8c2590 msgr2=0x561a2dcc76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:33:21.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.471+0000 7fd12fe00640 1 --2- 172.21.15.100:0/2758092468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a2d8c2590 0x561a2dcc76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.471+0000 7fd12fe00640 1 -- 172.21.15.100:0/2758092468 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a2d94e850 con 0x561a2d8ba3b0 2024-09-17T04:33:21.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.471+0000 7fd127400640 1 --2- 172.21.15.100:0/2758092468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a2d8c2590 0x561a2dcc76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:21.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.471+0000 7fd12fe00640 1 --2- 172.21.15.100:0/2758092468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a2d8ba3b0 0x561a2dca9ca0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fd120099600 tx=0x7fd12005f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457f076c9785f8bf server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:21.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.471+0000 7fd11cc00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-17T04:33:21.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.472+0000 7fd11cc00640 1 -- 172.21.15.100:0/2758092468 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12007d040 con 0x561a2d8ba3b0 2024-09-17T04:33:21.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.472+0000 7fd11cc00640 1 -- 172.21.15.100:0/2758092468 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7fd12006ab50 con 0x561a2d8ba3b0 2024-09-17T04:33:21.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.472+0000 7fd13dbca740 1 -- 172.21.15.100:0/2758092468 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561a2dbff520 con 0x561a2d8ba3b0 2024-09-17T04:33:21.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.472+0000 7fd11cc00640 1 -- 172.21.15.100:0/2758092468 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) ==== 24+0+0 (secure 0 0 0) 0x7fd120085220 con 0x561a2d8ba3b0 2024-09-17T04:33:21.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.472+0000 7fd13dbca740 10 client.8925 fetch_fsmap learned FSMap version 111 2024-09-17T04:33:21.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.472+0000 7fd13dbca740 1 -- 172.21.15.100:0/2758092468 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) -- 0x561a2d9c56b0 con 0x561a2d8ba3b0 2024-09-17T04:33:21.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.472+0000 7fd11cc00640 1 -- 172.21.15.100:0/2758092468 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x561a2dbff520 con 0x561a2d8ba3b0 2024-09-17T04:33:21.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.472+0000 7fd11cc00640 1 -- 172.21.15.100:0/2758092468 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 111) ==== 3502+0+0 (secure 0 0 0) 0x7fd12005e640 con 0x561a2d8ba3b0 2024-09-17T04:33:21.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.472+0000 7fd13dbca740 10 client.8925 fetch_fsmap finished waiting for FSMap version 111 2024-09-17T04:33:21.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.472+0000 7fd13dbca740 10 client.8925 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-17T04:33:21.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.472+0000 7fd13dbca740 20 client.8925 populate_metadata read hostname 'smithi100' 2024-09-17T04:33:21.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.472+0000 7fd13dbca740 1 --2- 172.21.15.100:0/2758092468 >> [v2:172.21.15.184:6834/2965326201,v1:172.21.15.184:6835/2965326201] conn(0x561a2d8a1e40 0x561a2dcc57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:21.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.472+0000 7fd13dbca740 4 client.8925 mds_command: new command op to 8814 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T04:33:21.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.472+0000 7fd13dbca740 1 -- 172.21.15.100:0/2758092468 --> [v2:172.21.15.184:6834/2965326201,v1:172.21.15.184:6835/2965326201] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a2dbfafd0 con 0x561a2d8a1e40 2024-09-17T04:33:21.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.473+0000 7fd127400640 1 --2- 172.21.15.100:0/2758092468 >> [v2:172.21.15.184:6834/2965326201,v1:172.21.15.184:6835/2965326201] conn(0x561a2d8a1e40 0x561a2dcc57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:21.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.473+0000 7fd127400640 1 --2- 172.21.15.100:0/2758092468 >> [v2:172.21.15.184:6834/2965326201,v1:172.21.15.184:6835/2965326201] conn(0x561a2d8a1e40 0x561a2dcc57a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec57d8dfc562d3ab server_cookie=bb3503403dffbf5d in_seq=0 out_seq=0 2024-09-17T04:33:21.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.473+0000 7fd11cc00640 10 client.8925 ms_handle_connect on v2:172.21.15.184:6834/2965326201 2024-09-17T04:33:21.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.475+0000 7fd11cc00640 1 -- 172.21.15.100:0/2758092468 <== mds.0 v2:172.21.15.184:6834/2965326201 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561a2dbfafd0 con 0x561a2d8a1e40 2024-09-17T04:33:21.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.475+0000 7fd11cc00640 10 client.8925 handle_command_reply: tid=0 2024-09-17T04:33:21.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.475+0000 7fd13dbca740 1 client.8925 shutdown 2024-09-17T04:33:21.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.475+0000 7fd13dbca740 1 -- 172.21.15.100:0/2758092468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a2d8ba3b0 msgr2=0x561a2dca9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:21.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.475+0000 7fd13dbca740 1 --2- 172.21.15.100:0/2758092468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a2d8ba3b0 0x561a2dca9ca0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fd120099600 tx=0x7fd12005f1e0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.475+0000 7fd13dbca740 1 -- 172.21.15.100:0/2758092468 shutdown_connections 2024-09-17T04:33:21.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.475+0000 7fd13dbca740 1 --2- 172.21.15.100:0/2758092468 >> [v2:172.21.15.184:6834/2965326201,v1:172.21.15.184:6835/2965326201] conn(0x561a2d8a1e40 0x561a2dcc57a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.475+0000 7fd13dbca740 1 --2- 172.21.15.100:0/2758092468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561a2d8e3270 0x561a2dcd1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.475+0000 7fd13dbca740 1 --2- 172.21.15.100:0/2758092468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a2d8c2590 0x561a2dcc76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.475+0000 7fd11cc00640 0 client.8925 ms_handle_reset on v2:172.21.15.184:6834/2965326201 2024-09-17T04:33:21.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.475+0000 7fd13dbca740 1 --2- 172.21.15.100:0/2758092468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a2d8ba3b0 0x561a2dca9ca0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.475+0000 7fd13dbca740 1 -- 172.21.15.100:0/2758092468 >> 172.21.15.100:0/2758092468 conn(0x561a2d8d73c0 msgr2=0x561a2dcc4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:21.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.475+0000 7fd13dbca740 1 -- 172.21.15.100:0/2758092468 shutdown_connections 2024-09-17T04:33:21.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.475+0000 7fd13dbca740 1 -- 172.21.15.100:0/2758092468 wait complete. 2024-09-17T04:33:21.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.475+0000 7fd13dbca740 20 client.8925 trim_cache size 0 max 16384 2024-09-17T04:33:21.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.494+0000 7fd13dbca740 1 Processor -- start 2024-09-17T04:33:21.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.494+0000 7fd13dbca740 1 -- start start 2024-09-17T04:33:21.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.496+0000 7fd13dbca740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561a2d8ba3b0 0x561a2dcd1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:21.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.496+0000 7fd13dbca740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a2d8c2590 0x561a2dcd1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:21.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.496+0000 7fd13dbca740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a2d8e3270 0x561a2dcaf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:21.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.496+0000 7fd13dbca740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd1301ff4c0 con 0x561a2d8c2590 2024-09-17T04:33:21.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.496+0000 7fd13dbca740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd130073250 con 0x561a2d8e3270 2024-09-17T04:33:21.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.496+0000 7fd13dbca740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd1300738a0 con 0x561a2d8ba3b0 2024-09-17T04:33:21.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.496+0000 7fd127400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a2d8c2590 0x561a2dcd1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:21.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.496+0000 7fd127400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a2d8c2590 0x561a2dcd1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43056/0 (socket says 172.21.15.100:43056) 2024-09-17T04:33:21.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.496+0000 7fd127400640 1 -- 172.21.15.100:0/3420414336 learned_addr learned my addr 172.21.15.100:0/3420414336 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:21.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.496+0000 7fd12fe00640 1 --2- 172.21.15.100:0/3420414336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561a2d8ba3b0 0x561a2dcd1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:21.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.496+0000 7fd134c00640 1 --2- 172.21.15.100:0/3420414336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a2d8e3270 0x561a2dcaf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:21.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.496+0000 7fd127400640 1 -- 172.21.15.100:0/3420414336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561a2d8ba3b0 msgr2=0x561a2dcd1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:33:21.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.496+0000 7fd127400640 1 --2- 172.21.15.100:0/3420414336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561a2d8ba3b0 0x561a2dcd1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.496+0000 7fd127400640 1 -- 172.21.15.100:0/3420414336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a2d8e3270 msgr2=0x561a2dcaf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:33:21.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.496+0000 7fd127400640 1 --2- 172.21.15.100:0/3420414336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a2d8e3270 0x561a2dcaf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.496+0000 7fd127400640 1 -- 172.21.15.100:0/3420414336 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a2dc8a650 con 0x561a2d8c2590 2024-09-17T04:33:21.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.496+0000 7fd134c00640 1 --2- 172.21.15.100:0/3420414336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a2d8e3270 0x561a2dcaf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:21.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.497+0000 7fd127400640 1 --2- 172.21.15.100:0/3420414336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a2d8c2590 0x561a2dcd1c50 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fd118006010 tx=0x7fd11803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8725468f2417f73 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:21.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.497+0000 7fd11f400640 1 -- 172.21.15.100:0/3420414336 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118035040 con 0x561a2d8c2590 2024-09-17T04:33:21.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.497+0000 7fd11f400640 1 -- 172.21.15.100:0/3420414336 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd118004bf0 con 0x561a2d8c2590 2024-09-17T04:33:21.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.497+0000 7fd11f400640 1 -- 172.21.15.100:0/3420414336 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118004dc0 con 0x561a2d8c2590 2024-09-17T04:33:21.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.498+0000 7fd13dbca740 1 -- 172.21.15.100:0/3420414336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a2d8c2590 msgr2=0x561a2dcd1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:21.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.498+0000 7fd13dbca740 1 --2- 172.21.15.100:0/3420414336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a2d8c2590 0x561a2dcd1c50 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fd118006010 tx=0x7fd11803d000 comp rx=0 tx=0).stop 2024-09-17T04:33:21.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.498+0000 7fd13dbca740 1 -- 172.21.15.100:0/3420414336 shutdown_connections 2024-09-17T04:33:21.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.498+0000 7fd13dbca740 1 --2- 172.21.15.100:0/3420414336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a2d8e3270 0x561a2dcaf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.498+0000 7fd13dbca740 1 --2- 172.21.15.100:0/3420414336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a2d8c2590 0x561a2dcd1c50 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.498+0000 7fd13dbca740 1 --2- 172.21.15.100:0/3420414336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561a2d8ba3b0 0x561a2dcd1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.498+0000 7fd13dbca740 1 -- 172.21.15.100:0/3420414336 >> 172.21.15.100:0/3420414336 conn(0x561a2d8d73c0 msgr2=0x561a2dcc83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:21.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.498+0000 7fd13dbca740 1 -- 172.21.15.100:0/3420414336 shutdown_connections 2024-09-17T04:33:21.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.498+0000 7fd13dbca740 1 -- 172.21.15.100:0/3420414336 wait complete. 2024-09-17T04:33:21.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.499+0000 7fd13dbca740 1 -- start start 2024-09-17T04:33:21.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.499+0000 7fd13dbca740 1 Processor -- start 2024-09-17T04:33:21.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.500+0000 7fd13dbca740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a2d8ba3b0 0x561a2dd48350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:21.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.500+0000 7fd13dbca740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a2d8c2590 0x561a2dd51720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:21.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.500+0000 7fd13dbca740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561a2d8e3270 0x561a2dca9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:21.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.500+0000 7fd13dbca740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561a2db93330 con 0x561a2d8ba3b0 2024-09-17T04:33:21.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.500+0000 7fd13dbca740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561a2dbf0810 con 0x561a2d8c2590 2024-09-17T04:33:21.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.500+0000 7fd13dbca740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x561a2dcaef70 con 0x561a2d8e3270 2024-09-17T04:33:21.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.500+0000 7fd13dbca740 10 client.0 _ll_register_callbacks cb 0x7fd12005ec20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:33:21.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.501+0000 7fd12fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a2d8ba3b0 0x561a2dd48350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:21.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.501+0000 7fd12fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a2d8ba3b0 0x561a2dd48350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43058/0 (socket says 172.21.15.100:43058) 2024-09-17T04:33:21.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.501+0000 7fd12fe00640 1 -- 172.21.15.100:0/2848694564 learned_addr learned my addr 172.21.15.100:0/2848694564 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:21.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.501+0000 7fd127400640 1 --2- 172.21.15.100:0/2848694564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a2d8c2590 0x561a2dd51720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:21.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.501+0000 7fd12fe00640 1 -- 172.21.15.100:0/2848694564 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561a2d8e3270 msgr2=0x561a2dca9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:33:21.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.501+0000 7fd134c00640 1 --2- 172.21.15.100:0/2848694564 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561a2d8e3270 0x561a2dca9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:21.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.501+0000 7fd12fe00640 1 --2- 172.21.15.100:0/2848694564 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561a2d8e3270 0x561a2dca9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.501+0000 7fd12fe00640 1 -- 172.21.15.100:0/2848694564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a2d8c2590 msgr2=0x561a2dd51720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:33:21.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.501+0000 7fd12fe00640 1 --2- 172.21.15.100:0/2848694564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a2d8c2590 0x561a2dd51720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.501+0000 7fd12fe00640 1 -- 172.21.15.100:0/2848694564 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a2dcb2ad0 con 0x561a2d8ba3b0 2024-09-17T04:33:21.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.501+0000 7fd134c00640 1 --2- 172.21.15.100:0/2848694564 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561a2d8e3270 0x561a2dca9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:21.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.501+0000 7fd12fe00640 1 --2- 172.21.15.100:0/2848694564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a2d8ba3b0 0x561a2dd48350 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fd120099820 tx=0x7fd120062390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb51d360e4fc398 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:21.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.501+0000 7fd11cc00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-17T04:33:21.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.502+0000 7fd11cc00640 1 -- 172.21.15.100:0/2848694564 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12005f340 con 0x561a2d8ba3b0 2024-09-17T04:33:21.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.502+0000 7fd11cc00640 1 -- 172.21.15.100:0/2848694564 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7fd12006ab50 con 0x561a2d8ba3b0 2024-09-17T04:33:21.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.502+0000 7fd13dbca740 1 -- 172.21.15.100:0/2848694564 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd118004bf0 con 0x561a2d8ba3b0 2024-09-17T04:33:21.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.502+0000 7fd11cc00640 1 -- 172.21.15.100:0/2848694564 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) ==== 24+0+0 (secure 0 0 0) 0x7fd1200633f0 con 0x561a2d8ba3b0 2024-09-17T04:33:21.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.502+0000 7fd13dbca740 10 client.8931 fetch_fsmap learned FSMap version 111 2024-09-17T04:33:21.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.502+0000 7fd13dbca740 1 -- 172.21.15.100:0/2848694564 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) -- 0x561a2dcb2ad0 con 0x561a2d8ba3b0 2024-09-17T04:33:21.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.502+0000 7fd11cc00640 1 -- 172.21.15.100:0/2848694564 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd118004bf0 con 0x561a2d8ba3b0 2024-09-17T04:33:21.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.502+0000 7fd11cc00640 1 -- 172.21.15.100:0/2848694564 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 111) ==== 3502+0+0 (secure 0 0 0) 0x7fd1200619c0 con 0x561a2d8ba3b0 2024-09-17T04:33:21.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.502+0000 7fd13dbca740 10 client.8931 fetch_fsmap finished waiting for FSMap version 111 2024-09-17T04:33:21.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.502+0000 7fd13dbca740 10 client.8931 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-17T04:33:21.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.502+0000 7fd13dbca740 20 client.8931 populate_metadata read hostname 'smithi100' 2024-09-17T04:33:21.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.502+0000 7fd13dbca740 1 --2- 172.21.15.100:0/2848694564 >> [v2:172.21.15.184:6834/2965326201,v1:172.21.15.184:6835/2965326201] conn(0x561a2dcf1c30 0x561a2dd556f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:21.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.502+0000 7fd13dbca740 4 client.8931 mds_command: new command op to 8814 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T04:33:21.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.502+0000 7fd13dbca740 1 -- 172.21.15.100:0/2848694564 --> [v2:172.21.15.184:6834/2965326201,v1:172.21.15.184:6835/2965326201] -- command(tid 0: {"prefix": "status"}) -- 0x561a2dbf8400 con 0x561a2dcf1c30 2024-09-17T04:33:21.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.503+0000 7fd127400640 1 --2- 172.21.15.100:0/2848694564 >> [v2:172.21.15.184:6834/2965326201,v1:172.21.15.184:6835/2965326201] conn(0x561a2dcf1c30 0x561a2dd556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:21.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.503+0000 7fd127400640 1 --2- 172.21.15.100:0/2848694564 >> [v2:172.21.15.184:6834/2965326201,v1:172.21.15.184:6835/2965326201] conn(0x561a2dcf1c30 0x561a2dd556f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebbadaacc1ed3f4c server_cookie=a18db01734135e8d in_seq=0 out_seq=0 2024-09-17T04:33:21.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.503+0000 7fd11cc00640 10 client.8931 ms_handle_connect on v2:172.21.15.184:6834/2965326201 2024-09-17T04:33:21.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.504+0000 7fd11cc00640 1 -- 172.21.15.100:0/2848694564 <== mds.0 v2:172.21.15.184:6834/2965326201 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x561a2dbf8400 con 0x561a2dcf1c30 2024-09-17T04:33:21.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.504+0000 7fd11cc00640 10 client.8931 handle_command_reply: tid=0 2024-09-17T04:33:21.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.504+0000 7fd13dbca740 1 client.8931 shutdown 2024-09-17T04:33:21.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.504+0000 7fd13dbca740 1 -- 172.21.15.100:0/2848694564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a2d8ba3b0 msgr2=0x561a2dd48350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:21.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.504+0000 7fd13dbca740 1 --2- 172.21.15.100:0/2848694564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a2d8ba3b0 0x561a2dd48350 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fd120099820 tx=0x7fd120062390 comp rx=0 tx=0).stop 2024-09-17T04:33:21.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.504+0000 7fd13dbca740 1 -- 172.21.15.100:0/2848694564 shutdown_connections 2024-09-17T04:33:21.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.504+0000 7fd13dbca740 1 --2- 172.21.15.100:0/2848694564 >> [v2:172.21.15.184:6834/2965326201,v1:172.21.15.184:6835/2965326201] conn(0x561a2dcf1c30 0x561a2dd556f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.504+0000 7fd13dbca740 1 --2- 172.21.15.100:0/2848694564 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561a2d8e3270 0x561a2dca9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.504+0000 7fd13dbca740 1 --2- 172.21.15.100:0/2848694564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a2d8c2590 0x561a2dd51720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.504+0000 7fd13dbca740 1 --2- 172.21.15.100:0/2848694564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a2d8ba3b0 0x561a2dd48350 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.504+0000 7fd11cc00640 0 client.8931 ms_handle_reset on v2:172.21.15.184:6834/2965326201 2024-09-17T04:33:21.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.504+0000 7fd13dbca740 1 -- 172.21.15.100:0/2848694564 >> 172.21.15.100:0/2848694564 conn(0x561a2d8d73c0 msgr2=0x561a2dcd1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:21.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.504+0000 7fd13dbca740 1 -- 172.21.15.100:0/2848694564 shutdown_connections 2024-09-17T04:33:21.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.504+0000 7fd13dbca740 1 -- 172.21.15.100:0/2848694564 wait complete. 2024-09-17T04:33:21.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.504+0000 7fd13dbca740 20 client.8931 trim_cache size 0 max 16384 2024-09-17T04:33:21.510 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:33:21.510 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "8a399842-3a5c-4ea2-9afb-b2704332e59f", 2024-09-17T04:33:21.510 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 0, 2024-09-17T04:33:21.510 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 8814, 2024-09-17T04:33:21.510 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-09-17T04:33:21.510 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-09-17T04:33:21.510 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-09-17T04:33:21.510 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 6.6339259400000001, 2024-09-17T04:33:21.510 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 111, 2024-09-17T04:33:21.510 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 156, 2024-09-17T04:33:21.510 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 156, 2024-09-17T04:33:21.510 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 11.662950711000001 2024-09-17T04:33:21.510 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:33:21.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.505+0000 7fd136c00640 1 -- 172.21.15.100:0/1992600791 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd10c079400 msgr2=0x7fd10c07b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:21.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.505+0000 7fd136c00640 1 --2- 172.21.15.100:0/1992600791 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd10c079400 0x7fd10c07b910 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd118033000 tx=0x7fd118004730 comp rx=0 tx=0).stop 2024-09-17T04:33:21.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.505+0000 7fd136c00640 1 -- 172.21.15.100:0/1992600791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1300cfc80 msgr2=0x7fd1300c6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:21.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.505+0000 7fd136c00640 1 --2- 172.21.15.100:0/1992600791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1300cfc80 0x7fd1300c6250 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fd12000b3f0 tx=0x7fd12000b8b0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.505+0000 7fd136c00640 1 -- 172.21.15.100:0/1992600791 shutdown_connections 2024-09-17T04:33:21.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.505+0000 7fd136c00640 1 --2- 172.21.15.100:0/1992600791 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd10c079400 0x7fd10c07b910 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.505+0000 7fd136c00640 1 --2- 172.21.15.100:0/1992600791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1301f3b40 0x7fd1301f78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.505+0000 7fd136c00640 1 --2- 172.21.15.100:0/1992600791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd13015ce30 0x7fd1300c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.505+0000 7fd136c00640 1 --2- 172.21.15.100:0/1992600791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1300cfc80 0x7fd1300c6250 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.505+0000 7fd136c00640 1 -- 172.21.15.100:0/1992600791 >> 172.21.15.100:0/1992600791 conn(0x7fd13006d3d0 msgr2=0x7fd13015ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:21.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.505+0000 7fd136c00640 1 -- 172.21.15.100:0/1992600791 shutdown_connections 2024-09-17T04:33:21.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.505+0000 7fd136c00640 1 -- 172.21.15.100:0/1992600791 wait complete. 2024-09-17T04:33:21.529 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:33:21.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.749+0000 7f78c6400640 1 -- 172.21.15.100:0/165365643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78c015d760 msgr2=0x7f78c01535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:21.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.749+0000 7f78c6400640 1 --2- 172.21.15.100:0/165365643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78c015d760 0x7f78c01535c0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009830 tx=0x7f78b002a530 comp rx=0 tx=0).stop 2024-09-17T04:33:21.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.749+0000 7f78c6400640 1 -- 172.21.15.100:0/165365643 shutdown_connections 2024-09-17T04:33:21.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.749+0000 7f78c6400640 1 --2- 172.21.15.100:0/165365643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78c015d760 0x7f78c01535c0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.749+0000 7f78c6400640 1 -- 172.21.15.100:0/165365643 >> 172.21.15.100:0/165365643 conn(0x7f78c006d730 msgr2=0x7f78c00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:21.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.749+0000 7f78c6400640 1 -- 172.21.15.100:0/165365643 shutdown_connections 2024-09-17T04:33:21.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.749+0000 7f78c6400640 1 -- 172.21.15.100:0/165365643 wait complete. 2024-09-17T04:33:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.752+0000 7f78c6400640 1 Processor -- start 2024-09-17T04:33:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.752+0000 7f78c6400640 1 -- start start 2024-09-17T04:33:21.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.753+0000 7f78c6400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78c00ad0e0 0x7f78c015f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:21.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.753+0000 7f78c6400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c00b3550 0x7f78c01ef920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:21.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.753+0000 7f78c6400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78c015d760 0x7f78c014e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:21.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.753+0000 7f78c6400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f78c006f240 con 0x7f78c00ad0e0 2024-09-17T04:33:21.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.753+0000 7f78c6400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78c00b4e30 con 0x7f78c015d760 2024-09-17T04:33:21.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.753+0000 7f78c6400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f78c00b4070 con 0x7f78c00b3550 2024-09-17T04:33:21.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.753+0000 7f78bf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78c00ad0e0 0x7f78c015f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:21.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.753+0000 7f78bf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78c00ad0e0 0x7f78c015f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43066/0 (socket says 172.21.15.100:43066) 2024-09-17T04:33:21.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.753+0000 7f78bf400640 1 -- 172.21.15.100:0/2487979954 learned_addr learned my addr 172.21.15.100:0/2487979954 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:21.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.754+0000 7f78bea00640 1 --2- 172.21.15.100:0/2487979954 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c00b3550 0x7f78c01ef920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:21.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.754+0000 7f78bfe00640 1 --2- 172.21.15.100:0/2487979954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78c015d760 0x7f78c014e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:21.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.754+0000 7f78bf400640 1 -- 172.21.15.100:0/2487979954 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c00b3550 msgr2=0x7f78c01ef920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:21.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.754+0000 7f78bf400640 1 --2- 172.21.15.100:0/2487979954 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c00b3550 0x7f78c01ef920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.754+0000 7f78bf400640 1 -- 172.21.15.100:0/2487979954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78c015d760 msgr2=0x7f78c014e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:21.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.754+0000 7f78bf400640 1 --2- 172.21.15.100:0/2487979954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78c015d760 0x7f78c014e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.754+0000 7f78bf400640 1 -- 172.21.15.100:0/2487979954 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c01efe60 con 0x7f78c00ad0e0 2024-09-17T04:33:21.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.754+0000 7f78bfe00640 1 --2- 172.21.15.100:0/2487979954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78c015d760 0x7f78c014e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:21.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.754+0000 7f78bf400640 1 --2- 172.21.15.100:0/2487979954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78c00ad0e0 0x7f78c015f950 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009950 tx=0x7f78b0033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:21.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.754+0000 7f78b7400640 1 -- 172.21.15.100:0/2487979954 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b0031b60 con 0x7f78c00ad0e0 2024-09-17T04:33:21.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.754+0000 7f78b7400640 1 -- 172.21.15.100:0/2487979954 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78b0031df0 con 0x7f78c00ad0e0 2024-09-17T04:33:21.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.754+0000 7f78b7400640 1 -- 172.21.15.100:0/2487979954 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b0005280 con 0x7f78c00ad0e0 2024-09-17T04:33:21.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.754+0000 7f78c6400640 1 -- 172.21.15.100:0/2487979954 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c014e6c0 con 0x7f78c00ad0e0 2024-09-17T04:33:21.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.754+0000 7f78c6400640 1 -- 172.21.15.100:0/2487979954 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c014d6a0 con 0x7f78c00ad0e0 2024-09-17T04:33:21.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.755+0000 7f78c6400640 1 -- 172.21.15.100:0/2487979954 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c01535c0 con 0x7f78c00ad0e0 2024-09-17T04:33:21.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.757+0000 7f78b7400640 1 -- 172.21.15.100:0/2487979954 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f78b0005420 con 0x7f78c00ad0e0 2024-09-17T04:33:21.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.758+0000 7f78b7400640 1 --2- 172.21.15.100:0/2487979954 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f78a0079400 0x7f78a007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:21.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.758+0000 7f78b7400640 1 -- 172.21.15.100:0/2487979954 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7f78b00ba7c0 con 0x7f78c00ad0e0 2024-09-17T04:33:21.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.758+0000 7f78bea00640 1 --2- 172.21.15.100:0/2487979954 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f78a0079400 0x7f78a007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:21.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.759+0000 7f78bea00640 1 --2- 172.21.15.100:0/2487979954 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f78a0079400 0x7f78a007b910 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f78c015d390 tx=0x7f78ac009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:21.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.762+0000 7f78b7400640 1 -- 172.21.15.100:0/2487979954 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f78c01efe60 con 0x7f78c00ad0e0 2024-09-17T04:33:21.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.989+0000 7f78c6400640 1 -- 172.21.15.100:0/2487979954 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f78c015bf00 con 0x7f78c00ad0e0 2024-09-17T04:33:21.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.989+0000 7f78b7400640 1 -- 172.21.15.100:0/2487979954 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v111) ==== 53+0+73 (secure 0 0 0) 0x7f78b003f030 con 0x7f78c00ad0e0 2024-09-17T04:33:21.990 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:33:21.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.992+0000 7f78c6400640 1 -- 172.21.15.100:0/2487979954 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f78a0079400 msgr2=0x7f78a007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:21.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.992+0000 7f78c6400640 1 --2- 172.21.15.100:0/2487979954 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f78a0079400 0x7f78a007b910 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f78c015d390 tx=0x7f78ac009040 comp rx=0 tx=0).stop 2024-09-17T04:33:21.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.992+0000 7f78c6400640 1 -- 172.21.15.100:0/2487979954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78c00ad0e0 msgr2=0x7f78c015f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:21.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.992+0000 7f78c6400640 1 --2- 172.21.15.100:0/2487979954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78c00ad0e0 0x7f78c015f950 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009950 tx=0x7f78b0033c10 comp rx=0 tx=0).stop 2024-09-17T04:33:21.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.992+0000 7f78c6400640 1 -- 172.21.15.100:0/2487979954 shutdown_connections 2024-09-17T04:33:21.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.992+0000 7f78c6400640 1 --2- 172.21.15.100:0/2487979954 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f78a0079400 0x7f78a007b910 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.992+0000 7f78c6400640 1 --2- 172.21.15.100:0/2487979954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78c015d760 0x7f78c014e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.992+0000 7f78c6400640 1 --2- 172.21.15.100:0/2487979954 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c00b3550 0x7f78c01ef920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.992+0000 7f78c6400640 1 --2- 172.21.15.100:0/2487979954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78c00ad0e0 0x7f78c015f950 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.992+0000 7f78c6400640 1 -- 172.21.15.100:0/2487979954 >> 172.21.15.100:0/2487979954 conn(0x7f78c006d730 msgr2=0x7f78c01602a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:21.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.992+0000 7f78c6400640 1 -- 172.21.15.100:0/2487979954 shutdown_connections 2024-09-17T04:33:21.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:21.992+0000 7f78c6400640 1 -- 172.21.15.100:0/2487979954 wait complete. 2024-09-17T04:33:22.018 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T04:33:22.018 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:33:22.018 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-09-17T04:33:22.018 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T04:33:22.018 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T04:33:22.018 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T04:33:22.019 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T04:33:22.019 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:33:22.019 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:33:22.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:22.019 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-09-17T04:33:22.055 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:33:22.056 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:33:22.056 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:33:22.056 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:22.056 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-17T04:33:22.056 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:22.056 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:33:22.056 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:33:22.056 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-09-17T04:33:22.056 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:33:22.056 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:33:22.056 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-09-17T04:33:22.056 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:33:22.056 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:33:22.057 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:33:22.057 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:33:22.057 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T04:33:22.057 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42158sec preferred_lft 42158sec 2024-09-17T04:33:22.057 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link noprefixroute 2024-09-17T04:33:22.057 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:22.057 INFO:teuthology.orchestra.run.smithi100.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:33:22.057 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:2e:de:8e:a4:7c brd ff:ff:ff:ff:ff:ff 2024-09-17T04:33:22.057 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T04:33:22.057 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:22.057 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::cc78:e4ff:fe70:7fbe/64 scope link 2024-09-17T04:33:22.057 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:22.058 INFO:teuthology.orchestra.run.smithi100.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:33:22.058 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether ea:8c:45:9f:34:87 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:33:22.058 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::e88c:45ff:fe9f:3487/64 scope link 2024-09-17T04:33:22.058 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:22.058 INFO:teuthology.orchestra.run.smithi100.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:33:22.058 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:2e:de:8e:a4:7c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:33:22.058 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::c2e:deff:fe8e:a47c/64 scope link 2024-09-17T04:33:22.058 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:22.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:22.058 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:33:22.150 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-09-17T04:33:22.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:22.153 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-09-17T04:33:22.190 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-09-17T04:33:22.190 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:33:22.190 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T04:33:22.190 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T04:33:22.190 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:33:22.190 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:33:22.190 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-17T04:33:22.191 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T04:33:22.191 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T04:33:22.191 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T04:33:22.191 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-17T04:33:22.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:33:22.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:33:22.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:22.386 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-17T04:33:22.422 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T04:33:22.422 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T04:33:22.422 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:33:22.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:22.422 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-09-17T04:33:22.518 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:33:22.518 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T04:33:22.553 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:33:22.553 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:33:22.554 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:33:22.628 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T04:33:22.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:22.629 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:33:22.823 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:33:22.823 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:33:22.824 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:33:22.844 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:33:22.844 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:33:22.844 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:33:22.844 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:33:22.845 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:33:22.874 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:33:22.874 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:33:22.874 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:33:22.874 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:33:22.875 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:33:22.875 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-09-17T04:33:22.875 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-09-17T04:33:22.875 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8a399842-3a5c-4ea2-9afb-b2704332e59f" will pass to kernel 2024-09-17T04:33:22.876 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-17T04:33:22.876 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-09-17T04:33:22.876 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-17T04:33:22.876 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-09-17T04:33:22.876 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-17T04:33:22.879 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T04:33:22.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:22.879 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:33:22.938 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:33:23.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:33:23.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:33:23.040 INFO:teuthology.orchestra.run.smithi100.stdout:client8943 2024-09-17T04:33:23.040 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:33:23.040 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:33:23.076 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:33:23.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.336+0000 7f1369e00640 1 -- 172.21.15.100:0/3525015359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f136415d760 msgr2=0x7f13641535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:23.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.336+0000 7f1369e00640 1 --2- 172.21.15.100:0/3525015359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f136415d760 0x7f13641535c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1354009830 tx=0x7f135402a480 comp rx=0 tx=0).stop 2024-09-17T04:33:23.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.336+0000 7f1369e00640 1 -- 172.21.15.100:0/3525015359 shutdown_connections 2024-09-17T04:33:23.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.336+0000 7f1369e00640 1 --2- 172.21.15.100:0/3525015359 >> v2:172.21.15.184:3300/0 conn(0x7f13640ad0e0 0x7f13640ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.336+0000 7f1369e00640 1 --2- 172.21.15.100:0/3525015359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f136415d760 0x7f13641535c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.336+0000 7f1369e00640 1 -- 172.21.15.100:0/3525015359 >> 172.21.15.100:0/3525015359 conn(0x7f136406d730 msgr2=0x7f13640b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:23.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.336+0000 7f1369e00640 1 -- 172.21.15.100:0/3525015359 shutdown_connections 2024-09-17T04:33:23.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.336+0000 7f1369e00640 1 -- 172.21.15.100:0/3525015359 wait complete. 2024-09-17T04:33:23.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.338+0000 7f1369e00640 1 Processor -- start 2024-09-17T04:33:23.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.338+0000 7f1369e00640 1 -- start start 2024-09-17T04:33:23.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.340+0000 7f1369e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13640ad0e0 0x7f1364156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:23.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.340+0000 7f1369e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f136415d760 0x7f136414f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:23.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.340+0000 7f1369e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1364160310 0x7f136415b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:23.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.340+0000 7f1369e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f136406f220 con 0x7f13640ad0e0 2024-09-17T04:33:23.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.340+0000 7f1369e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f136406f0a0 con 0x7f136415d760 2024-09-17T04:33:23.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.340+0000 7f1369e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f13640b4c20 con 0x7f1364160310 2024-09-17T04:33:23.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.340+0000 7f1362a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13640ad0e0 0x7f1364156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:23.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.340+0000 7f1362a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13640ad0e0 0x7f1364156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43082/0 (socket says 172.21.15.100:43082) 2024-09-17T04:33:23.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.340+0000 7f1362a00640 1 -- 172.21.15.100:0/3271868508 learned_addr learned my addr 172.21.15.100:0/3271868508 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:23.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.340+0000 7f135a000640 1 --2- 172.21.15.100:0/3271868508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f136415d760 0x7f136414f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:23.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.340+0000 7f1363400640 1 --2- 172.21.15.100:0/3271868508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1364160310 0x7f136415b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:23.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.340+0000 7f1362a00640 1 -- 172.21.15.100:0/3271868508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1364160310 msgr2=0x7f136415b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:23.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.340+0000 7f1362a00640 1 --2- 172.21.15.100:0/3271868508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1364160310 0x7f136415b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.340+0000 7f1362a00640 1 -- 172.21.15.100:0/3271868508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f136415d760 msgr2=0x7f136414f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:23.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.340+0000 7f1362a00640 1 --2- 172.21.15.100:0/3271868508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f136415d760 0x7f136414f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.341+0000 7f1362a00640 1 -- 172.21.15.100:0/3271868508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13641f66b0 con 0x7f13640ad0e0 2024-09-17T04:33:23.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.341+0000 7f135a000640 1 --2- 172.21.15.100:0/3271868508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f136415d760 0x7f136414f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:33:23.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.341+0000 7f1362a00640 1 --2- 172.21.15.100:0/3271868508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13640ad0e0 0x7f1364156250 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f1354000c00 tx=0x7f1354033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:23.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.341+0000 7f135be00640 1 -- 172.21.15.100:0/3271868508 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1354031d60 con 0x7f13640ad0e0 2024-09-17T04:33:23.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.341+0000 7f135be00640 1 -- 172.21.15.100:0/3271868508 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1354033e10 con 0x7f13640ad0e0 2024-09-17T04:33:23.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.341+0000 7f135be00640 1 -- 172.21.15.100:0/3271868508 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1354033310 con 0x7f13640ad0e0 2024-09-17T04:33:23.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.341+0000 7f1369e00640 1 -- 172.21.15.100:0/3271868508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13641f9000 con 0x7f13640ad0e0 2024-09-17T04:33:23.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.341+0000 7f1369e00640 1 -- 172.21.15.100:0/3271868508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13641f95c0 con 0x7f13640ad0e0 2024-09-17T04:33:23.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.342+0000 7f1369e00640 1 -- 172.21.15.100:0/3271868508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1364160fa0 con 0x7f13640ad0e0 2024-09-17T04:33:23.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.344+0000 7f135be00640 1 -- 172.21.15.100:0/3271868508 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f13540334b0 con 0x7f13640ad0e0 2024-09-17T04:33:23.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.345+0000 7f135be00640 1 --2- 172.21.15.100:0/3271868508 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1340079400 0x7f134007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:23.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.345+0000 7f135be00640 1 -- 172.21.15.100:0/3271868508 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7f13540bd1e0 con 0x7f13640ad0e0 2024-09-17T04:33:23.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.345+0000 7f135a000640 1 --2- 172.21.15.100:0/3271868508 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1340079400 0x7f134007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:23.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.346+0000 7f135a000640 1 --2- 172.21.15.100:0/3271868508 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1340079400 0x7f134007b910 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f134c0097a0 tx=0x7f134c009260 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:23.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.350+0000 7f135be00640 1 -- 172.21.15.100:0/3271868508 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f135403f030 con 0x7f13640ad0e0 2024-09-17T04:33:23.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.569+0000 7f1369e00640 1 -- 172.21.15.100:0/3271868508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f136406e060 con 0x7f13640ad0e0 2024-09-17T04:33:23.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.569+0000 7f135be00640 1 -- 172.21.15.100:0/3271868508 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v111) ==== 53+0+73 (secure 0 0 0) 0x7f1354084080 con 0x7f13640ad0e0 2024-09-17T04:33:23.570 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:33:23.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.573+0000 7f1369e00640 1 -- 172.21.15.100:0/3271868508 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1340079400 msgr2=0x7f134007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:23.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.573+0000 7f1369e00640 1 --2- 172.21.15.100:0/3271868508 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1340079400 0x7f134007b910 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f134c0097a0 tx=0x7f134c009260 comp rx=0 tx=0).stop 2024-09-17T04:33:23.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.573+0000 7f1369e00640 1 -- 172.21.15.100:0/3271868508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13640ad0e0 msgr2=0x7f1364156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:23.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.573+0000 7f1369e00640 1 --2- 172.21.15.100:0/3271868508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13640ad0e0 0x7f1364156250 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f1354000c00 tx=0x7f1354033c20 comp rx=0 tx=0).stop 2024-09-17T04:33:23.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.573+0000 7f1369e00640 1 -- 172.21.15.100:0/3271868508 shutdown_connections 2024-09-17T04:33:23.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.573+0000 7f1369e00640 1 --2- 172.21.15.100:0/3271868508 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1340079400 0x7f134007b910 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.573+0000 7f1369e00640 1 --2- 172.21.15.100:0/3271868508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1364160310 0x7f136415b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.573+0000 7f1369e00640 1 --2- 172.21.15.100:0/3271868508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f136415d760 0x7f136414f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.573+0000 7f1369e00640 1 --2- 172.21.15.100:0/3271868508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13640ad0e0 0x7f1364156250 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.573+0000 7f1369e00640 1 -- 172.21.15.100:0/3271868508 >> 172.21.15.100:0/3271868508 conn(0x7f136406d730 msgr2=0x7f1364160720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:23.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.573+0000 7f1369e00640 1 -- 172.21.15.100:0/3271868508 shutdown_connections 2024-09-17T04:33:23.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:23.573+0000 7f1369e00640 1 -- 172.21.15.100:0/3271868508 wait complete. 2024-09-17T04:33:23.595 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T04:33:23.596 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:33:23.596 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2024-09-17T04:33:23.596 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T04:33:23.596 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T04:33:23.596 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T04:33:23.596 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T04:33:23.596 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:33:23.596 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:33:23.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:23.596 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2024-09-17T04:33:23.637 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:33:23.637 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:33:23.637 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:33:23.637 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:23.637 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-09-17T04:33:23.637 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:23.637 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:33:23.637 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:33:23.637 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2024-09-17T04:33:23.637 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:33:23.637 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:33:23.637 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2024-09-17T04:33:23.638 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:33:23.638 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2024-09-17T04:33:23.638 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:33:23.638 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2024-09-17T04:33:23.638 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T04:33:23.638 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42140sec preferred_lft 42140sec 2024-09-17T04:33:23.638 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ec4:7aff:fe88:7b8f/64 scope link noprefixroute 2024-09-17T04:33:23.638 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:23.638 INFO:teuthology.orchestra.run.smithi184.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:33:23.638 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 7e:67:e5:69:96:69 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:33:23.638 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T04:33:23.638 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:23.638 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::3c9d:aff:fe6e:8fcb/64 scope link 2024-09-17T04:33:23.639 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:23.639 INFO:teuthology.orchestra.run.smithi184.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:33:23.639 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether a6:a9:1b:a9:61:ec brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T04:33:23.639 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::a4a9:1bff:fea9:61ec/64 scope link 2024-09-17T04:33:23.639 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:23.639 INFO:teuthology.orchestra.run.smithi184.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:33:23.639 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 7e:67:e5:69:96:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T04:33:23.639 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::7c67:e5ff:fe69:9669/64 scope link 2024-09-17T04:33:23.639 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:23.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:23.640 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:33:23.738 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-09-17T04:33:23.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:23.741 DEBUG:teuthology.orchestra.run.smithi184:> route 2024-09-17T04:33:23.778 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2024-09-17T04:33:23.778 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:33:23.778 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T04:33:23.779 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T04:33:23.780 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:33:23.780 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:33:23.780 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-09-17T04:33:23.781 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T04:33:23.781 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T04:33:23.781 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T04:33:23.781 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-09-17T04:33:23.898 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:33:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:33:23.970 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:33:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:33:23.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:23.978 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-09-17T04:33:24.013 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T04:33:24.013 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T04:33:24.014 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:33:24.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:24.015 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.0 up 2024-09-17T04:33:24.110 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:33:24.111 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T04:33:24.146 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:33:24.146 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:33:24.146 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T04:33:24.222 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T04:33:24.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:24.222 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:33:24.417 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:33:24.418 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:33:24.418 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:33:24.437 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:33:24.437 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:33:24.437 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:33:24.437 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:33:24.437 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:33:24.465 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:33:24.465 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:33:24.465 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:33:24.465 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:33:24.465 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:33:24.466 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-09-17T04:33:24.466 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-09-17T04:33:24.466 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8a399842-3a5c-4ea2-9afb-b2704332e59f" will pass to kernel 2024-09-17T04:33:24.467 INFO:teuthology.orchestra.run.smithi184.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-17T04:33:24.467 INFO:teuthology.orchestra.run.smithi184.stdout: You just mounted a file system that supports labels which does not 2024-09-17T04:33:24.467 INFO:teuthology.orchestra.run.smithi184.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-17T04:33:24.467 INFO:teuthology.orchestra.run.smithi184.stdout: applications will generate AVC messages and not be allowed access to 2024-09-17T04:33:24.467 INFO:teuthology.orchestra.run.smithi184.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-17T04:33:24.470 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T04:33:24.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:24.470 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T04:33:24.526 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:33:24.603 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:33:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:33:24.606 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:33:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:33:24.614 INFO:teuthology.orchestra.run.smithi184.stdout:client8356 2024-09-17T04:33:24.614 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:33:24.614 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:33:24.650 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T04:33:24.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.887+0000 7f7200c00640 1 -- 172.21.15.100:0/1961806351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71e0005350 msgr2=0x7f71e000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:24.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.887+0000 7f7200c00640 1 --2- 172.21.15.100:0/1961806351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71e0005350 0x7f71e000b9c0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f71f4005fd0 tx=0x7f71e0005830 comp rx=0 tx=0).stop 2024-09-17T04:33:24.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.887+0000 7f7200c00640 1 -- 172.21.15.100:0/1961806351 shutdown_connections 2024-09-17T04:33:24.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.887+0000 7f7200c00640 1 --2- 172.21.15.100:0/1961806351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71e0005350 0x7f71e000b9c0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:24.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.887+0000 7f7200c00640 1 -- 172.21.15.100:0/1961806351 >> 172.21.15.100:0/1961806351 conn(0x7f71fc08bfe0 msgr2=0x7f71fc0d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:24.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.887+0000 7f7200c00640 1 -- 172.21.15.100:0/1961806351 shutdown_connections 2024-09-17T04:33:24.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.887+0000 7f7200c00640 1 -- 172.21.15.100:0/1961806351 wait complete. 2024-09-17T04:33:24.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.889+0000 7f7200c00640 1 Processor -- start 2024-09-17T04:33:24.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.889+0000 7f7200c00640 1 -- start start 2024-09-17T04:33:24.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.890+0000 7f7200c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71e0005350 0x7f71fc0ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:24.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.890+0000 7f7200c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71fc154580 0x7f71fc15cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:24.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.891+0000 7f7200c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71fc15d510 0x7f71fc0c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:24.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.891+0000 7f7200c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f71fc1620b0 con 0x7f71fc154580 2024-09-17T04:33:24.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.891+0000 7f7200c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f71fc161f30 con 0x7f71e0005350 2024-09-17T04:33:24.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.891+0000 7f7200c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f71fc162230 con 0x7f71fc15d510 2024-09-17T04:33:24.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.891+0000 7f71f2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71fc154580 0x7f71fc15cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:24.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.891+0000 7f71f2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71fc154580 0x7f71fc15cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42008/0 (socket says 172.21.15.100:42008) 2024-09-17T04:33:24.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.891+0000 7f71f2a00640 1 -- 172.21.15.100:0/854667385 learned_addr learned my addr 172.21.15.100:0/854667385 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:24.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.891+0000 7f71fb400640 1 --2- 172.21.15.100:0/854667385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71e0005350 0x7f71fc0ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:24.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.891+0000 7f71fbe00640 1 --2- 172.21.15.100:0/854667385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71fc15d510 0x7f71fc0c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:24.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.891+0000 7f71f2a00640 1 -- 172.21.15.100:0/854667385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71fc15d510 msgr2=0x7f71fc0c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:24.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.891+0000 7f71f2a00640 1 --2- 172.21.15.100:0/854667385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71fc15d510 0x7f71fc0c8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:24.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.891+0000 7f71f2a00640 1 -- 172.21.15.100:0/854667385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71e0005350 msgr2=0x7f71fc0ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:24.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.891+0000 7f71f2a00640 1 --2- 172.21.15.100:0/854667385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71e0005350 0x7f71fc0ca510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:24.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.891+0000 7f71f2a00640 1 -- 172.21.15.100:0/854667385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71fc15d920 con 0x7f71fc154580 2024-09-17T04:33:24.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.892+0000 7f71f2a00640 1 --2- 172.21.15.100:0/854667385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71fc154580 0x7f71fc15cfd0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f71e4003f00 tx=0x7f71e4001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:24.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.892+0000 7f71f3e00640 1 -- 172.21.15.100:0/854667385 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4015070 con 0x7f71fc154580 2024-09-17T04:33:24.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.892+0000 7f71f3e00640 1 -- 172.21.15.100:0/854667385 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71e4011840 con 0x7f71fc154580 2024-09-17T04:33:24.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.892+0000 7f7200c00640 1 -- 172.21.15.100:0/854667385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71fc0caf90 con 0x7f71fc154580 2024-09-17T04:33:24.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.892+0000 7f71f3e00640 1 -- 172.21.15.100:0/854667385 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4011c90 con 0x7f71fc154580 2024-09-17T04:33:24.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.892+0000 7f7200c00640 1 -- 172.21.15.100:0/854667385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71fc1f55f0 con 0x7f71fc154580 2024-09-17T04:33:24.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.893+0000 7f7200c00640 1 -- 172.21.15.100:0/854667385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71fc0cace0 con 0x7f71fc154580 2024-09-17T04:33:24.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.894+0000 7f71f3e00640 1 -- 172.21.15.100:0/854667385 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f71e4018070 con 0x7f71fc154580 2024-09-17T04:33:24.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.895+0000 7f71f3e00640 1 --2- 172.21.15.100:0/854667385 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f71d8079400 0x7f71d807b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:24.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.895+0000 7f71f3e00640 1 -- 172.21.15.100:0/854667385 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7f71e409cc60 con 0x7f71fc154580 2024-09-17T04:33:24.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.895+0000 7f71fb400640 1 --2- 172.21.15.100:0/854667385 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f71d8079400 0x7f71d807b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:24.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.896+0000 7f71fb400640 1 --2- 172.21.15.100:0/854667385 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f71d8079400 0x7f71d807b910 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f71ec001860 tx=0x7f71ec006040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:24.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:24.900+0000 7f71f3e00640 1 -- 172.21.15.100:0/854667385 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f71fc15d920 con 0x7f71fc154580 2024-09-17T04:33:25.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:25.119+0000 7f7200c00640 1 -- 172.21.15.100:0/854667385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f71fc0b6df0 con 0x7f71fc154580 2024-09-17T04:33:25.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:25.932+0000 7f71f3e00640 1 -- 172.21.15.100:0/854667385 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v112) ==== 136+0+0 (secure 0 0 0) 0x7f71e4063a70 con 0x7f71fc154580 2024-09-17T04:33:25.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:25.935+0000 7f7200c00640 1 -- 172.21.15.100:0/854667385 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f71d8079400 msgr2=0x7f71d807b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:25.935+0000 7f7200c00640 1 --2- 172.21.15.100:0/854667385 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f71d8079400 0x7f71d807b910 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f71ec001860 tx=0x7f71ec006040 comp rx=0 tx=0).stop 2024-09-17T04:33:25.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:25.935+0000 7f7200c00640 1 -- 172.21.15.100:0/854667385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71fc154580 msgr2=0x7f71fc15cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:25.935+0000 7f7200c00640 1 --2- 172.21.15.100:0/854667385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71fc154580 0x7f71fc15cfd0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f71e4003f00 tx=0x7f71e4001630 comp rx=0 tx=0).stop 2024-09-17T04:33:25.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:25.935+0000 7f7200c00640 1 -- 172.21.15.100:0/854667385 shutdown_connections 2024-09-17T04:33:25.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:25.935+0000 7f7200c00640 1 --2- 172.21.15.100:0/854667385 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f71d8079400 0x7f71d807b910 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:25.935+0000 7f7200c00640 1 --2- 172.21.15.100:0/854667385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71fc15d510 0x7f71fc0c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:25.935+0000 7f7200c00640 1 --2- 172.21.15.100:0/854667385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71fc154580 0x7f71fc15cfd0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:25.935+0000 7f7200c00640 1 --2- 172.21.15.100:0/854667385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71e0005350 0x7f71fc0ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:25.935+0000 7f7200c00640 1 -- 172.21.15.100:0/854667385 >> 172.21.15.100:0/854667385 conn(0x7f71fc08bfe0 msgr2=0x7f71fc0b4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:25.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:25.936+0000 7f7200c00640 1 -- 172.21.15.100:0/854667385 shutdown_connections 2024-09-17T04:33:25.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:25.936+0000 7f7200c00640 1 -- 172.21.15.100:0/854667385 wait complete. 2024-09-17T04:33:25.963 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T04:33:26.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.189+0000 7f8fdbe00640 1 -- 172.21.15.100:0/3278248764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fc0003a50 msgr2=0x7f8fc0003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.189+0000 7f8fdbe00640 1 --2- 172.21.15.100:0/3278248764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fc0003a50 0x7f8fc0003ec0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00c310 tx=0x7f8fc0005830 comp rx=0 tx=0).stop 2024-09-17T04:33:26.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.189+0000 7f8fdbe00640 1 -- 172.21.15.100:0/3278248764 shutdown_connections 2024-09-17T04:33:26.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.189+0000 7f8fdbe00640 1 --2- 172.21.15.100:0/3278248764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fc0003a50 0x7f8fc0003ec0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.189+0000 7f8fdbe00640 1 -- 172.21.15.100:0/3278248764 >> 172.21.15.100:0/3278248764 conn(0x7f8fdc0b1e40 msgr2=0x7f8fdc0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:26.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.189+0000 7f8fdbe00640 1 -- 172.21.15.100:0/3278248764 shutdown_connections 2024-09-17T04:33:26.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.189+0000 7f8fdbe00640 1 -- 172.21.15.100:0/3278248764 wait complete. 2024-09-17T04:33:26.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.192+0000 7f8fdbe00640 1 Processor -- start 2024-09-17T04:33:26.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.192+0000 7f8fdbe00640 1 -- start start 2024-09-17T04:33:26.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.193+0000 7f8fdbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fc0003a50 0x7f8fdc13cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.193+0000 7f8fdbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fdc142f40 0x7f8fdc13c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.193+0000 7f8fdbe00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8fdc14cbc0 0x7f8fdc14cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.193+0000 7f8fdbe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8fdc072e60 con 0x7f8fc0003a50 2024-09-17T04:33:26.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.193+0000 7f8fdbe00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8fdc0704e0 con 0x7f8fdc142f40 2024-09-17T04:33:26.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.193+0000 7f8fdbe00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8fdc072fe0 con 0x7f8fdc14cbc0 2024-09-17T04:33:26.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.194+0000 7f8fdaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fc0003a50 0x7f8fdc13cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.194+0000 7f8fdaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fc0003a50 0x7f8fdc13cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42028/0 (socket says 172.21.15.100:42028) 2024-09-17T04:33:26.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.194+0000 7f8fdaa00640 1 -- 172.21.15.100:0/528347335 learned_addr learned my addr 172.21.15.100:0/528347335 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:26.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.194+0000 7f8fda000640 1 --2- 172.21.15.100:0/528347335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fdc142f40 0x7f8fdc13c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.194+0000 7f8fdb400640 1 --2- 172.21.15.100:0/528347335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8fdc14cbc0 0x7f8fdc14cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.194+0000 7f8fdaa00640 1 -- 172.21.15.100:0/528347335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8fdc14cbc0 msgr2=0x7f8fdc14cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.194+0000 7f8fdaa00640 1 --2- 172.21.15.100:0/528347335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8fdc14cbc0 0x7f8fdc14cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.194+0000 7f8fdaa00640 1 -- 172.21.15.100:0/528347335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fdc142f40 msgr2=0x7f8fdc13c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.194+0000 7f8fdaa00640 1 --2- 172.21.15.100:0/528347335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fdc142f40 0x7f8fdc13c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.194+0000 7f8fdaa00640 1 -- 172.21.15.100:0/528347335 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fdc14a210 con 0x7f8fc0003a50 2024-09-17T04:33:26.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.194+0000 7f8fdb400640 1 --2- 172.21.15.100:0/528347335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8fdc14cbc0 0x7f8fdc14cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:26.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.194+0000 7f8fdaa00640 1 --2- 172.21.15.100:0/528347335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fc0003a50 0x7f8fdc13cea0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00c430 tx=0x7f8fcc004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:26.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.195+0000 7f8fd3400640 1 -- 172.21.15.100:0/528347335 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc044070 con 0x7f8fc0003a50 2024-09-17T04:33:26.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.195+0000 7f8fd3400640 1 -- 172.21.15.100:0/528347335 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fcc006420 con 0x7f8fc0003a50 2024-09-17T04:33:26.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.195+0000 7f8fd3400640 1 -- 172.21.15.100:0/528347335 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc031040 con 0x7f8fc0003a50 2024-09-17T04:33:26.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.195+0000 7f8fdbe00640 1 -- 172.21.15.100:0/528347335 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fdc14bb20 con 0x7f8fc0003a50 2024-09-17T04:33:26.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.195+0000 7f8fdbe00640 1 -- 172.21.15.100:0/528347335 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fdc14c0e0 con 0x7f8fc0003a50 2024-09-17T04:33:26.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.196+0000 7f8fdbe00640 1 -- 172.21.15.100:0/528347335 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fdc073540 con 0x7f8fc0003a50 2024-09-17T04:33:26.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.197+0000 7f8fd3400640 1 -- 172.21.15.100:0/528347335 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f8fcc034070 con 0x7f8fc0003a50 2024-09-17T04:33:26.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.198+0000 7f8fd3400640 1 --2- 172.21.15.100:0/528347335 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8fbc079430 0x7f8fbc07b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.198+0000 7f8fd3400640 1 -- 172.21.15.100:0/528347335 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7f8fcc0b9050 con 0x7f8fc0003a50 2024-09-17T04:33:26.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.198+0000 7f8fda000640 1 --2- 172.21.15.100:0/528347335 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8fbc079430 0x7f8fbc07b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.199+0000 7f8fda000640 1 --2- 172.21.15.100:0/528347335 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8fbc079430 0x7f8fbc07b940 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc1435b0 tx=0x7f8fc8006b70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:26.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.203+0000 7f8fd3400640 1 -- 172.21.15.100:0/528347335 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8fcc082f40 con 0x7f8fc0003a50 2024-09-17T04:33:26.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.408+0000 7f8fdbe00640 1 -- 172.21.15.100:0/528347335 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f8fdc15eb20 con 0x7f8fbc079430 2024-09-17T04:33:26.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.410+0000 7f8fd3400640 1 -- 172.21.15.100:0/528347335 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f8fdc15eb20 con 0x7f8fbc079430 2024-09-17T04:33:26.409 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-09-17T04:33:26.409 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:26.409 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephfs" 2024-09-17T04:33:26.409 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:26.409 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-09-17T04:33:26.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.412+0000 7f8fdbe00640 1 -- 172.21.15.100:0/528347335 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8fbc079430 msgr2=0x7f8fbc07b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.412+0000 7f8fdbe00640 1 --2- 172.21.15.100:0/528347335 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8fbc079430 0x7f8fbc07b940 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc1435b0 tx=0x7f8fc8006b70 comp rx=0 tx=0).stop 2024-09-17T04:33:26.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.412+0000 7f8fdbe00640 1 -- 172.21.15.100:0/528347335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fc0003a50 msgr2=0x7f8fdc13cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.412+0000 7f8fdbe00640 1 --2- 172.21.15.100:0/528347335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fc0003a50 0x7f8fdc13cea0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00c430 tx=0x7f8fcc004be0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.412+0000 7f8fdbe00640 1 -- 172.21.15.100:0/528347335 shutdown_connections 2024-09-17T04:33:26.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.412+0000 7f8fdbe00640 1 --2- 172.21.15.100:0/528347335 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8fbc079430 0x7f8fbc07b940 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.412+0000 7f8fdbe00640 1 --2- 172.21.15.100:0/528347335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8fdc14cbc0 0x7f8fdc14cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.412+0000 7f8fdbe00640 1 --2- 172.21.15.100:0/528347335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fdc142f40 0x7f8fdc13c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.412+0000 7f8fdbe00640 1 --2- 172.21.15.100:0/528347335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fc0003a50 0x7f8fdc13cea0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.412+0000 7f8fdbe00640 1 -- 172.21.15.100:0/528347335 >> 172.21.15.100:0/528347335 conn(0x7f8fdc0b1e40 msgr2=0x7f8fdc15e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:26.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.412+0000 7f8fdbe00640 1 -- 172.21.15.100:0/528347335 shutdown_connections 2024-09-17T04:33:26.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.412+0000 7f8fdbe00640 1 -- 172.21.15.100:0/528347335 wait complete. 2024-09-17T04:33:26.438 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T04:33:26.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.662+0000 7f8b27e00640 1 -- 172.21.15.100:0/3830218043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b281624b0 msgr2=0x7f8b2815f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.662+0000 7f8b27e00640 1 --2- 172.21.15.100:0/3830218043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b281624b0 0x7f8b2815f6a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8b10005fd0 tx=0x7f8b1002a540 comp rx=0 tx=0).stop 2024-09-17T04:33:26.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.662+0000 7f8b27e00640 1 -- 172.21.15.100:0/3830218043 shutdown_connections 2024-09-17T04:33:26.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.662+0000 7f8b27e00640 1 --2- 172.21.15.100:0/3830218043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b281624b0 0x7f8b2815f6a0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.662+0000 7f8b27e00640 1 --2- 172.21.15.100:0/3830218043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b280742b0 0x7f8b280b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.662+0000 7f8b27e00640 1 -- 172.21.15.100:0/3830218043 >> 172.21.15.100:0/3830218043 conn(0x7f8b2806d620 msgr2=0x7f8b280b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:26.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.662+0000 7f8b27e00640 1 -- 172.21.15.100:0/3830218043 shutdown_connections 2024-09-17T04:33:26.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.662+0000 7f8b27e00640 1 -- 172.21.15.100:0/3830218043 wait complete. 2024-09-17T04:33:26.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.664+0000 7f8b27e00640 1 Processor -- start 2024-09-17T04:33:26.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.664+0000 7f8b27e00640 1 -- start start 2024-09-17T04:33:26.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.666+0000 7f8b27e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b280742b0 0x7f8b281f96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.666+0000 7f8b27e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b2815fbe0 0x7f8b2809b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.666+0000 7f8b27e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b281624b0 0x7f8b2809d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.666+0000 7f8b27e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8b28073070 con 0x7f8b281624b0 2024-09-17T04:33:26.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.666+0000 7f8b27e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8b28072ef0 con 0x7f8b280742b0 2024-09-17T04:33:26.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.666+0000 7f8b27e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8b280731f0 con 0x7f8b2815fbe0 2024-09-17T04:33:26.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.666+0000 7f8b27400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b281624b0 0x7f8b2809d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.666+0000 7f8b26a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b280742b0 0x7f8b281f96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.666+0000 7f8b1e000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b2815fbe0 0x7f8b2809b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.666+0000 7f8b26a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b280742b0 0x7f8b281f96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.100:46344/0 (socket says 172.21.15.100:46344) 2024-09-17T04:33:26.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.666+0000 7f8b26a00640 1 -- 172.21.15.100:0/2938959104 learned_addr learned my addr 172.21.15.100:0/2938959104 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:26.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.666+0000 7f8b26a00640 1 -- 172.21.15.100:0/2938959104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b2815fbe0 msgr2=0x7f8b2809b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.667+0000 7f8b26a00640 1 --2- 172.21.15.100:0/2938959104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b2815fbe0 0x7f8b2809b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.667+0000 7f8b26a00640 1 -- 172.21.15.100:0/2938959104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b281624b0 msgr2=0x7f8b2809d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.667+0000 7f8b26a00640 1 --2- 172.21.15.100:0/2938959104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b281624b0 0x7f8b2809d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.667+0000 7f8b26a00640 1 -- 172.21.15.100:0/2938959104 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b281f9c20 con 0x7f8b280742b0 2024-09-17T04:33:26.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.667+0000 7f8b1e000640 1 --2- 172.21.15.100:0/2938959104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b2815fbe0 0x7f8b2809b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:33:26.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.667+0000 7f8b27400640 1 --2- 172.21.15.100:0/2938959104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b281624b0 0x7f8b2809d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:33:26.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.667+0000 7f8b26a00640 1 --2- 172.21.15.100:0/2938959104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b280742b0 0x7f8b281f96e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8b1800eb60 tx=0x7f8b1800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:26.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.667+0000 7f8b1fe00640 1 -- 172.21.15.100:0/2938959104 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b18010630 con 0x7f8b280742b0 2024-09-17T04:33:26.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.667+0000 7f8b1fe00640 1 -- 172.21.15.100:0/2938959104 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b180108c0 con 0x7f8b280742b0 2024-09-17T04:33:26.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.667+0000 7f8b27e00640 1 -- 172.21.15.100:0/2938959104 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b280cc3f0 con 0x7f8b280742b0 2024-09-17T04:33:26.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.667+0000 7f8b1fe00640 1 -- 172.21.15.100:0/2938959104 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b18010d10 con 0x7f8b280742b0 2024-09-17T04:33:26.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.668+0000 7f8b27e00640 1 -- 172.21.15.100:0/2938959104 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b280cc9b0 con 0x7f8b280742b0 2024-09-17T04:33:26.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.669+0000 7f8b27e00640 1 -- 172.21.15.100:0/2938959104 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b28073710 con 0x7f8b280742b0 2024-09-17T04:33:26.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.670+0000 7f8b1fe00640 1 -- 172.21.15.100:0/2938959104 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f8b18018420 con 0x7f8b280742b0 2024-09-17T04:33:26.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.671+0000 7f8b1fe00640 1 --2- 172.21.15.100:0/2938959104 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8b04079430 0x7f8b0407b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.672+0000 7f8b1fe00640 1 -- 172.21.15.100:0/2938959104 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7f8b1809d530 con 0x7f8b280742b0 2024-09-17T04:33:26.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.672+0000 7f8b1e000640 1 --2- 172.21.15.100:0/2938959104 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8b04079430 0x7f8b0407b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.673+0000 7f8b1e000640 1 --2- 172.21.15.100:0/2938959104 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8b04079430 0x7f8b0407b940 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8b10033000 tx=0x7f8b10004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:26.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.675+0000 7f8b1fe00640 1 -- 172.21.15.100:0/2938959104 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8b1800ee60 con 0x7f8b280742b0 2024-09-17T04:33:26.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.860+0000 7f8b27e00640 1 -- 172.21.15.100:0/2938959104 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f8b280b2ab0 con 0x7f8b280742b0 2024-09-17T04:33:26.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.862+0000 7f8b1fe00640 1 -- 172.21.15.100:0/2938959104 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v50)=0 v50) ==== 122+0+0 (secure 0 0 0) 0x7f8b180643c0 con 0x7f8b280742b0 2024-09-17T04:33:26.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.864+0000 7f8b27e00640 1 -- 172.21.15.100:0/2938959104 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8b04079430 msgr2=0x7f8b0407b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.864+0000 7f8b27e00640 1 --2- 172.21.15.100:0/2938959104 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8b04079430 0x7f8b0407b940 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8b10033000 tx=0x7f8b10004730 comp rx=0 tx=0).stop 2024-09-17T04:33:26.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.864+0000 7f8b27e00640 1 -- 172.21.15.100:0/2938959104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b280742b0 msgr2=0x7f8b281f96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.864+0000 7f8b27e00640 1 --2- 172.21.15.100:0/2938959104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b280742b0 0x7f8b281f96e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8b1800eb60 tx=0x7f8b1800c6a0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.864+0000 7f8b27e00640 1 -- 172.21.15.100:0/2938959104 shutdown_connections 2024-09-17T04:33:26.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.864+0000 7f8b27e00640 1 --2- 172.21.15.100:0/2938959104 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8b04079430 0x7f8b0407b940 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.864+0000 7f8b27e00640 1 --2- 172.21.15.100:0/2938959104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b281624b0 0x7f8b2809d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.864+0000 7f8b27e00640 1 --2- 172.21.15.100:0/2938959104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b2815fbe0 0x7f8b2809b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.864+0000 7f8b27e00640 1 --2- 172.21.15.100:0/2938959104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b280742b0 0x7f8b281f96e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.864+0000 7f8b27e00640 1 -- 172.21.15.100:0/2938959104 >> 172.21.15.100:0/2938959104 conn(0x7f8b2806d620 msgr2=0x7f8b280d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:26.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.864+0000 7f8b27e00640 1 -- 172.21.15.100:0/2938959104 shutdown_connections 2024-09-17T04:33:26.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:26.864+0000 7f8b27e00640 1 -- 172.21.15.100:0/2938959104 wait complete. 2024-09-17T04:33:26.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:26.887 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol13490/6f44f848-4769-4473-bd83-b75f86fee9d1) 2024-09-17T04:33:26.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:26.954 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-17T04:33:26.991 INFO:teuthology.orchestra.run.smithi100.stdout:cephfs_data 2024-09-17T04:33:26.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:26.992 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol13490/6f44f848-4769-4473-bd83-b75f86fee9d1) 2024-09-17T04:33:27.098 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:33:27.098 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol13490/.meta 2024-09-17T04:33:27.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:27.157 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp19350/subvol8751/23886d8d-e3ad-405e-8bd1-d28257e1f979) 2024-09-17T04:33:27.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:27.257 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-17T04:33:27.294 INFO:teuthology.orchestra.run.smithi100.stdout:cephfs_data 2024-09-17T04:33:27.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:27.295 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/subvol_grp19350/subvol8751/23886d8d-e3ad-405e-8bd1-d28257e1f979) 2024-09-17T04:33:27.396 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:33:27.396 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/subvol_grp19350/subvol8751/.meta 2024-09-17T04:33:27.452 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol13490 2024-09-17T04:33:27.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.716+0000 7fabdb800640 1 -- 172.21.15.100:0/4241319993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabd415f9e0 msgr2=0x7fabd4147b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:27.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.716+0000 7fabdb800640 1 --2- 172.21.15.100:0/4241319993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabd415f9e0 0x7fabd4147b90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fabc4005fd0 tx=0x7fabc402a4b0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.716+0000 7fabdb800640 1 -- 172.21.15.100:0/4241319993 shutdown_connections 2024-09-17T04:33:27.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.716+0000 7fabdb800640 1 --2- 172.21.15.100:0/4241319993 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fabd40aee00 0x7fabd40af210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.716+0000 7fabdb800640 1 --2- 172.21.15.100:0/4241319993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabd415f9e0 0x7fabd4147b90 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.716+0000 7fabdb800640 1 -- 172.21.15.100:0/4241319993 >> 172.21.15.100:0/4241319993 conn(0x7fabd406d1c0 msgr2=0x7fabd406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:27.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.716+0000 7fabdb800640 1 -- 172.21.15.100:0/4241319993 shutdown_connections 2024-09-17T04:33:27.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.716+0000 7fabdb800640 1 -- 172.21.15.100:0/4241319993 wait complete. 2024-09-17T04:33:27.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.718+0000 7fabdb800640 1 Processor -- start 2024-09-17T04:33:27.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.718+0000 7fabdb800640 1 -- start start 2024-09-17T04:33:27.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.720+0000 7fabdb800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabd4073540 0x7fabd41f9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.720+0000 7fabdb800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fabd40aee00 0x7fabd40ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.720+0000 7fabdb800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabd415f9e0 0x7fabd41f5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.720+0000 7fabdb800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fabd406fb60 con 0x7fabd415f9e0 2024-09-17T04:33:27.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.720+0000 7fabdb800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fabd406f9e0 con 0x7fabd4073540 2024-09-17T04:33:27.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.720+0000 7fabdb800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fabd406fce0 con 0x7fabd40aee00 2024-09-17T04:33:27.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.720+0000 7fabd9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabd415f9e0 0x7fabd41f5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.720+0000 7fabd9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabd415f9e0 0x7fabd41f5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42040/0 (socket says 172.21.15.100:42040) 2024-09-17T04:33:27.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.720+0000 7fabd9800640 1 -- 172.21.15.100:0/1617733619 learned_addr learned my addr 172.21.15.100:0/1617733619 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:27.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.720+0000 7fabd8e00640 1 --2- 172.21.15.100:0/1617733619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabd4073540 0x7fabd41f9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.720+0000 7fabcbe00640 1 --2- 172.21.15.100:0/1617733619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fabd40aee00 0x7fabd40ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.720+0000 7fabd9800640 1 -- 172.21.15.100:0/1617733619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fabd40aee00 msgr2=0x7fabd40ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:27.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.720+0000 7fabd9800640 1 --2- 172.21.15.100:0/1617733619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fabd40aee00 0x7fabd40ceb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.720+0000 7fabd9800640 1 -- 172.21.15.100:0/1617733619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabd4073540 msgr2=0x7fabd41f9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:27.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.720+0000 7fabd9800640 1 --2- 172.21.15.100:0/1617733619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabd4073540 0x7fabd41f9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.720+0000 7fabd9800640 1 -- 172.21.15.100:0/1617733619 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd414f2d0 con 0x7fabd415f9e0 2024-09-17T04:33:27.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.721+0000 7fabd9800640 1 --2- 172.21.15.100:0/1617733619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabd415f9e0 0x7fabd41f5bf0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00c6b0 tx=0x7fabcc00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:27.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.721+0000 7fabd1600640 1 -- 172.21.15.100:0/1617733619 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabcc003b70 con 0x7fabd415f9e0 2024-09-17T04:33:27.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.721+0000 7fabd1600640 1 -- 172.21.15.100:0/1617733619 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabcc003d30 con 0x7fabd415f9e0 2024-09-17T04:33:27.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.721+0000 7fabdb800640 1 -- 172.21.15.100:0/1617733619 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabd4144950 con 0x7fabd415f9e0 2024-09-17T04:33:27.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.721+0000 7fabd1600640 1 -- 172.21.15.100:0/1617733619 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabcc005330 con 0x7fabd415f9e0 2024-09-17T04:33:27.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.721+0000 7fabdb800640 1 -- 172.21.15.100:0/1617733619 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabd413f770 con 0x7fabd415f9e0 2024-09-17T04:33:27.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.722+0000 7fabdb800640 1 -- 172.21.15.100:0/1617733619 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabd414f4a0 con 0x7fabd415f9e0 2024-09-17T04:33:27.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.723+0000 7fabd1600640 1 -- 172.21.15.100:0/1617733619 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7fabcc0054d0 con 0x7fabd415f9e0 2024-09-17T04:33:27.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.724+0000 7fabd1600640 1 --2- 172.21.15.100:0/1617733619 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fabb0079430 0x7fabb007b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.724+0000 7fabd1600640 1 -- 172.21.15.100:0/1617733619 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7fabcc09bc30 con 0x7fabd415f9e0 2024-09-17T04:33:27.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.725+0000 7fabd8e00640 1 --2- 172.21.15.100:0/1617733619 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fabb0079430 0x7fabb007b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.725+0000 7fabd8e00640 1 --2- 172.21.15.100:0/1617733619 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fabb0079430 0x7fabb007b940 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fabc4033000 tx=0x7fabc4004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:27.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.730+0000 7fabd1600640 1 -- 172.21.15.100:0/1617733619 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fabcc062a40 con 0x7fabd415f9e0 2024-09-17T04:33:27.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:27.942+0000 7fabdb800640 1 -- 172.21.15.100:0/1617733619 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol13490", "target": ["mon-mgr", ""]}) -- 0x7fabd40af210 con 0x7fabb0079430 2024-09-17T04:33:27.949 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:33:27.950+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:33:27.950 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:33:27.950+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:33:27.950 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:33:27.950+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:33:27.950 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:33:27.950+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:33:27.950 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:33:27.950+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:33:28.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.014+0000 7fabd1600640 1 -- 172.21.15.100:0/1617733619 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7fabd40af210 con 0x7fabb0079430 2024-09-17T04:33:28.013 INFO:teuthology.orchestra.run.smithi100.stdout:/volumes/_nogroup/subvol13490/6f44f848-4769-4473-bd83-b75f86fee9d1 2024-09-17T04:33:28.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.016+0000 7fabdb800640 1 -- 172.21.15.100:0/1617733619 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fabb0079430 msgr2=0x7fabb007b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.016+0000 7fabdb800640 1 --2- 172.21.15.100:0/1617733619 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fabb0079430 0x7fabb007b940 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fabc4033000 tx=0x7fabc4004730 comp rx=0 tx=0).stop 2024-09-17T04:33:28.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.016+0000 7fabdb800640 1 -- 172.21.15.100:0/1617733619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabd415f9e0 msgr2=0x7fabd41f5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.016+0000 7fabdb800640 1 --2- 172.21.15.100:0/1617733619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabd415f9e0 0x7fabd41f5bf0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00c6b0 tx=0x7fabcc00cb70 comp rx=0 tx=0).stop 2024-09-17T04:33:28.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.016+0000 7fabdb800640 1 -- 172.21.15.100:0/1617733619 shutdown_connections 2024-09-17T04:33:28.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.016+0000 7fabdb800640 1 --2- 172.21.15.100:0/1617733619 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fabb0079430 0x7fabb007b940 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.016+0000 7fabdb800640 1 --2- 172.21.15.100:0/1617733619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabd415f9e0 0x7fabd41f5bf0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.016+0000 7fabdb800640 1 --2- 172.21.15.100:0/1617733619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fabd40aee00 0x7fabd40ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.016+0000 7fabdb800640 1 --2- 172.21.15.100:0/1617733619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabd4073540 0x7fabd41f9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.016+0000 7fabdb800640 1 -- 172.21.15.100:0/1617733619 >> 172.21.15.100:0/1617733619 conn(0x7fabd406d1c0 msgr2=0x7fabd40b2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:28.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.017+0000 7fabdb800640 1 -- 172.21.15.100:0/1617733619 shutdown_connections 2024-09-17T04:33:28.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.017+0000 7fabdb800640 1 -- 172.21.15.100:0/1617733619 wait complete. 2024-09-17T04:33:28.039 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol8751 subvol_grp19350 2024-09-17T04:33:28.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.264+0000 7f1b92400640 1 -- 172.21.15.100:0/1133031641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b8c0af100 msgr2=0x7f1b8c0af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.264+0000 7f1b92400640 1 --2- 172.21.15.100:0/1133031641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b8c0af100 0x7f1b8c0af510 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c0d01c0 tx=0x7f1b8402aac0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.264+0000 7f1b92400640 1 -- 172.21.15.100:0/1133031641 shutdown_connections 2024-09-17T04:33:28.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.264+0000 7f1b92400640 1 --2- 172.21.15.100:0/1133031641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b8c0af100 0x7f1b8c0af510 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.264+0000 7f1b92400640 1 --2- 172.21.15.100:0/1133031641 >> v2:172.21.15.184:3300/0 conn(0x7f1b8c0cc830 0x7f1b8c0b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.264+0000 7f1b92400640 1 -- 172.21.15.100:0/1133031641 >> 172.21.15.100:0/1133031641 conn(0x7f1b8c0b2540 msgr2=0x7f1b8c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:28.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.264+0000 7f1b92400640 1 -- 172.21.15.100:0/1133031641 shutdown_connections 2024-09-17T04:33:28.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.264+0000 7f1b92400640 1 -- 172.21.15.100:0/1133031641 wait complete. 2024-09-17T04:33:28.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.266+0000 7f1b92400640 1 Processor -- start 2024-09-17T04:33:28.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.266+0000 7f1b92400640 1 -- start start 2024-09-17T04:33:28.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.268+0000 7f1b92400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b8c06cda0 0x7f1b8c1f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:28.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.268+0000 7f1b92400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b8c0af100 0x7f1b8c0c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:28.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.268+0000 7f1b92400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b8c0cc830 0x7f1b8c0c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:28.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.268+0000 7f1b92400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8c0b3670 con 0x7f1b8c0cc830 2024-09-17T04:33:28.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.268+0000 7f1b92400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8c0b34f0 con 0x7f1b8c0af100 2024-09-17T04:33:28.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.268+0000 7f1b92400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1b8c0b37f0 con 0x7f1b8c06cda0 2024-09-17T04:33:28.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.268+0000 7f1b8be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b8c0cc830 0x7f1b8c0c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:28.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.268+0000 7f1b8be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b8c0cc830 0x7f1b8c0c42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42070/0 (socket says 172.21.15.100:42070) 2024-09-17T04:33:28.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.268+0000 7f1b8be00640 1 -- 172.21.15.100:0/1033458363 learned_addr learned my addr 172.21.15.100:0/1033458363 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:28.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.268+0000 7f1b8b400640 1 --2- 172.21.15.100:0/1033458363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b8c06cda0 0x7f1b8c1f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:28.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.268+0000 7f1b8aa00640 1 --2- 172.21.15.100:0/1033458363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b8c0af100 0x7f1b8c0c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:28.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.268+0000 7f1b8be00640 1 -- 172.21.15.100:0/1033458363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b8c06cda0 msgr2=0x7f1b8c1f36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.268+0000 7f1b8be00640 1 --2- 172.21.15.100:0/1033458363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b8c06cda0 0x7f1b8c1f36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.268+0000 7f1b8be00640 1 -- 172.21.15.100:0/1033458363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b8c0af100 msgr2=0x7f1b8c0c3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.268+0000 7f1b8be00640 1 --2- 172.21.15.100:0/1033458363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b8c0af100 0x7f1b8c0c3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.268+0000 7f1b8be00640 1 -- 172.21.15.100:0/1033458363 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b8c0736f0 con 0x7f1b8c0cc830 2024-09-17T04:33:28.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.268+0000 7f1b8aa00640 1 --2- 172.21.15.100:0/1033458363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b8c0af100 0x7f1b8c0c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:28.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.269+0000 7f1b8be00640 1 --2- 172.21.15.100:0/1033458363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b8c0cc830 0x7f1b8c0c42d0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f1b8402b0b0 tx=0x7f1b84003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:28.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.269+0000 7f1b83400640 1 -- 172.21.15.100:0/1033458363 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b84003e50 con 0x7f1b8c0cc830 2024-09-17T04:33:28.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.269+0000 7f1b83400640 1 -- 172.21.15.100:0/1033458363 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b84004250 con 0x7f1b8c0cc830 2024-09-17T04:33:28.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.269+0000 7f1b83400640 1 -- 172.21.15.100:0/1033458363 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b84004680 con 0x7f1b8c0cc830 2024-09-17T04:33:28.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.269+0000 7f1b92400640 1 -- 172.21.15.100:0/1033458363 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b8c1f4050 con 0x7f1b8c0cc830 2024-09-17T04:33:28.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.269+0000 7f1b92400640 1 -- 172.21.15.100:0/1033458363 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b8c156d40 con 0x7f1b8c0cc830 2024-09-17T04:33:28.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.270+0000 7f1b92400640 1 -- 172.21.15.100:0/1033458363 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b8c0b3d10 con 0x7f1b8c0cc830 2024-09-17T04:33:28.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.271+0000 7f1b83400640 1 -- 172.21.15.100:0/1033458363 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f1b84049430 con 0x7f1b8c0cc830 2024-09-17T04:33:28.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.272+0000 7f1b83400640 1 --2- 172.21.15.100:0/1033458363 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1b6c0795d0 0x7f1b6c07bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:28.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.273+0000 7f1b83400640 1 -- 172.21.15.100:0/1033458363 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7f1b840ba1e0 con 0x7f1b8c0cc830 2024-09-17T04:33:28.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.273+0000 7f1b8b400640 1 --2- 172.21.15.100:0/1033458363 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1b6c0795d0 0x7f1b6c07bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:28.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.273+0000 7f1b8b400640 1 --2- 172.21.15.100:0/1033458363 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1b6c0795d0 0x7f1b6c07bae0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c009b30 tx=0x7f1b7c005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:28.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.277+0000 7f1b83400640 1 -- 172.21.15.100:0/1033458363 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f1b84032030 con 0x7f1b8c0cc830 2024-09-17T04:33:28.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.485+0000 7f1b92400640 1 -- 172.21.15.100:0/1033458363 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol8751", "group_name": "subvol_grp19350", "target": ["mon-mgr", ""]}) -- 0x7f1b8c0d1030 con 0x7f1b6c0795d0 2024-09-17T04:33:28.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.546+0000 7f1b83400640 1 -- 172.21.15.100:0/1033458363 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+73 (secure 0 0 0) 0x7f1b8c0d1030 con 0x7f1b6c0795d0 2024-09-17T04:33:28.546 INFO:teuthology.orchestra.run.smithi100.stdout:/volumes/subvol_grp19350/subvol8751/23886d8d-e3ad-405e-8bd1-d28257e1f979 2024-09-17T04:33:28.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.549+0000 7f1b92400640 1 -- 172.21.15.100:0/1033458363 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1b6c0795d0 msgr2=0x7f1b6c07bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.549+0000 7f1b92400640 1 --2- 172.21.15.100:0/1033458363 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1b6c0795d0 0x7f1b6c07bae0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c009b30 tx=0x7f1b7c005f90 comp rx=0 tx=0).stop 2024-09-17T04:33:28.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.549+0000 7f1b92400640 1 -- 172.21.15.100:0/1033458363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b8c0cc830 msgr2=0x7f1b8c0c42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.549+0000 7f1b92400640 1 --2- 172.21.15.100:0/1033458363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b8c0cc830 0x7f1b8c0c42d0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f1b8402b0b0 tx=0x7f1b84003ce0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.549+0000 7f1b92400640 1 -- 172.21.15.100:0/1033458363 shutdown_connections 2024-09-17T04:33:28.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.549+0000 7f1b92400640 1 --2- 172.21.15.100:0/1033458363 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1b6c0795d0 0x7f1b6c07bae0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.549+0000 7f1b92400640 1 --2- 172.21.15.100:0/1033458363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b8c0cc830 0x7f1b8c0c42d0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.549+0000 7f1b92400640 1 --2- 172.21.15.100:0/1033458363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b8c0af100 0x7f1b8c0c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.549+0000 7f1b92400640 1 --2- 172.21.15.100:0/1033458363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b8c06cda0 0x7f1b8c1f36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.549+0000 7f1b92400640 1 -- 172.21.15.100:0/1033458363 >> 172.21.15.100:0/1033458363 conn(0x7f1b8c0b2540 msgr2=0x7f1b8c06fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:28.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.549+0000 7f1b92400640 1 -- 172.21.15.100:0/1033458363 shutdown_connections 2024-09-17T04:33:28.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.549+0000 7f1b92400640 1 -- 172.21.15.100:0/1033458363 wait complete. 2024-09-17T04:33:28.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:28.576 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol13490/.meta) 2024-09-17T04:33:28.631 INFO:teuthology.orchestra.run.smithi100.stdout:[GLOBAL] 2024-09-17T04:33:28.631 INFO:teuthology.orchestra.run.smithi100.stdout:version = 2 2024-09-17T04:33:28.631 INFO:teuthology.orchestra.run.smithi100.stdout:type = subvolume 2024-09-17T04:33:28.631 INFO:teuthology.orchestra.run.smithi100.stdout:path = /volumes/_nogroup/subvol13490/6f44f848-4769-4473-bd83-b75f86fee9d1 2024-09-17T04:33:28.631 INFO:teuthology.orchestra.run.smithi100.stdout:state = complete 2024-09-17T04:33:28.631 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:28.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:28.634 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/subvol_grp19350/subvol8751/.meta) 2024-09-17T04:33:28.692 INFO:teuthology.orchestra.run.smithi100.stdout:[GLOBAL] 2024-09-17T04:33:28.692 INFO:teuthology.orchestra.run.smithi100.stdout:version = 2 2024-09-17T04:33:28.692 INFO:teuthology.orchestra.run.smithi100.stdout:type = subvolume 2024-09-17T04:33:28.692 INFO:teuthology.orchestra.run.smithi100.stdout:path = /volumes/subvol_grp19350/subvol8751/23886d8d-e3ad-405e-8bd1-d28257e1f979 2024-09-17T04:33:28.692 INFO:teuthology.orchestra.run.smithi100.stdout:state = complete 2024-09-17T04:33:28.692 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:28.695 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol13490 2024-09-17T04:33:28.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.920+0000 7f9d8d400640 1 -- 172.21.15.100:0/806439866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d6c003fd0 msgr2=0x7f9d6c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.920+0000 7f9d8d400640 1 --2- 172.21.15.100:0/806439866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d6c003fd0 0x7f9d6c004800 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f9d74009b20 tx=0x7f9d6c005830 comp rx=0 tx=0).stop 2024-09-17T04:33:28.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.920+0000 7f9d8ca00640 1 -- 172.21.15.100:0/806439866 reap_dead start 2024-09-17T04:33:28.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.920+0000 7f9d8d400640 1 -- 172.21.15.100:0/806439866 shutdown_connections 2024-09-17T04:33:28.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.920+0000 7f9d8d400640 1 -- 172.21.15.100:0/806439866 >> 172.21.15.100:0/806439866 conn(0x7f9d8806d620 msgr2=0x7f9d880b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:28.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.920+0000 7f9d8d400640 1 -- 172.21.15.100:0/806439866 shutdown_connections 2024-09-17T04:33:28.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.920+0000 7f9d8d400640 1 -- 172.21.15.100:0/806439866 wait complete. 2024-09-17T04:33:28.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.922+0000 7f9d8d400640 1 Processor -- start 2024-09-17T04:33:28.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.922+0000 7f9d8d400640 1 -- start start 2024-09-17T04:33:28.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.924+0000 7f9d8d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d6c003fd0 0x7f9d880b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:28.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.924+0000 7f9d8d400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d8815ea60 0x7f9d881f6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:28.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.924+0000 7f9d8d400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d881f8c90 0x7f9d8809d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:28.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.924+0000 7f9d8d400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9d880ccc40 con 0x7f9d6c003fd0 2024-09-17T04:33:28.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.924+0000 7f9d8d400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9d880ccac0 con 0x7f9d881f8c90 2024-09-17T04:33:28.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.924+0000 7f9d8d400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9d880ccdc0 con 0x7f9d8815ea60 2024-09-17T04:33:28.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.924+0000 7f9d87e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d6c003fd0 0x7f9d880b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:28.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.924+0000 7f9d87e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d6c003fd0 0x7f9d880b2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42082/0 (socket says 172.21.15.100:42082) 2024-09-17T04:33:28.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.924+0000 7f9d87e00640 1 -- 172.21.15.100:0/3914458 learned_addr learned my addr 172.21.15.100:0/3914458 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:28.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.924+0000 7f9d87400640 1 --2- 172.21.15.100:0/3914458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d8815ea60 0x7f9d881f6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:28.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.924+0000 7f9d8ca00640 1 --2- 172.21.15.100:0/3914458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d881f8c90 0x7f9d8809d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:28.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.924+0000 7f9d87e00640 1 -- 172.21.15.100:0/3914458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d8815ea60 msgr2=0x7f9d881f6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.924+0000 7f9d87e00640 1 --2- 172.21.15.100:0/3914458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d8815ea60 0x7f9d881f6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.924+0000 7f9d87e00640 1 -- 172.21.15.100:0/3914458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d881f8c90 msgr2=0x7f9d8809d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.924+0000 7f9d87e00640 1 --2- 172.21.15.100:0/3914458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d881f8c90 0x7f9d8809d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.924+0000 7f9d87e00640 1 -- 172.21.15.100:0/3914458 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d88073df0 con 0x7f9d6c003fd0 2024-09-17T04:33:28.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.924+0000 7f9d87400640 1 --2- 172.21.15.100:0/3914458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d8815ea60 0x7f9d881f6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:28.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.925+0000 7f9d87e00640 1 --2- 172.21.15.100:0/3914458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d6c003fd0 0x7f9d880b2100 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f9d78011b10 tx=0x7f9d78009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:28.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.925+0000 7f9d7fe00640 1 -- 172.21.15.100:0/3914458 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7800fce0 con 0x7f9d6c003fd0 2024-09-17T04:33:28.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.925+0000 7f9d7fe00640 1 -- 172.21.15.100:0/3914458 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d7800fe80 con 0x7f9d6c003fd0 2024-09-17T04:33:28.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.925+0000 7f9d8d400640 1 -- 172.21.15.100:0/3914458 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d88074870 con 0x7f9d6c003fd0 2024-09-17T04:33:28.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.925+0000 7f9d7fe00640 1 -- 172.21.15.100:0/3914458 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d78004280 con 0x7f9d6c003fd0 2024-09-17T04:33:28.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.925+0000 7f9d8d400640 1 -- 172.21.15.100:0/3914458 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d88074e80 con 0x7f9d6c003fd0 2024-09-17T04:33:28.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.926+0000 7f9d8d400640 1 -- 172.21.15.100:0/3914458 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d880afa10 con 0x7f9d6c003fd0 2024-09-17T04:33:28.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.928+0000 7f9d7fe00640 1 -- 172.21.15.100:0/3914458 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f9d780035f0 con 0x7f9d6c003fd0 2024-09-17T04:33:28.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.928+0000 7f9d7fe00640 1 --2- 172.21.15.100:0/3914458 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9d68079430 0x7f9d6807b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:28.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.929+0000 7f9d7fe00640 1 -- 172.21.15.100:0/3914458 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7f9d7809d390 con 0x7f9d6c003fd0 2024-09-17T04:33:28.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.929+0000 7f9d87400640 1 --2- 172.21.15.100:0/3914458 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9d68079430 0x7f9d6807b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:28.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.930+0000 7f9d87400640 1 --2- 172.21.15.100:0/3914458 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9d68079430 0x7f9d6807b940 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9d74035000 tx=0x7f9d740046c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:28.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.932+0000 7f9d7fe00640 1 -- 172.21.15.100:0/3914458 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9d78064870 con 0x7f9d6c003fd0 2024-09-17T04:33:28.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:28.959+0000 7f9d7fe00640 1 -- 172.21.15.100:0/3914458 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 27) ==== 102257+0+0 (secure 0 0 0) 0x7f9d780603e0 con 0x7f9d6c003fd0 2024-09-17T04:33:29.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.146+0000 7f9d8d400640 1 -- 172.21.15.100:0/3914458 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol13490", "target": ["mon-mgr", ""]}) -- 0x7f9d881567e0 con 0x7f9d68079430 2024-09-17T04:33:29.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.162+0000 7f9d7fe00640 1 -- 172.21.15.100:0/3914458 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9d881567e0 con 0x7f9d68079430 2024-09-17T04:33:29.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.164+0000 7f9d8d400640 1 -- 172.21.15.100:0/3914458 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9d68079430 msgr2=0x7f9d6807b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:29.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.164+0000 7f9d8d400640 1 --2- 172.21.15.100:0/3914458 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9d68079430 0x7f9d6807b940 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9d74035000 tx=0x7f9d740046c0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.164+0000 7f9d8d400640 1 -- 172.21.15.100:0/3914458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d6c003fd0 msgr2=0x7f9d880b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:29.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.164+0000 7f9d8d400640 1 --2- 172.21.15.100:0/3914458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d6c003fd0 0x7f9d880b2100 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f9d78011b10 tx=0x7f9d78009f10 comp rx=0 tx=0).stop 2024-09-17T04:33:29.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.164+0000 7f9d8d400640 1 -- 172.21.15.100:0/3914458 shutdown_connections 2024-09-17T04:33:29.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.165+0000 7f9d8d400640 1 --2- 172.21.15.100:0/3914458 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9d68079430 0x7f9d6807b940 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.165+0000 7f9d8d400640 1 --2- 172.21.15.100:0/3914458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d881f8c90 0x7f9d8809d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.165+0000 7f9d8d400640 1 --2- 172.21.15.100:0/3914458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d8815ea60 0x7f9d881f6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.165+0000 7f9d8d400640 1 --2- 172.21.15.100:0/3914458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d6c003fd0 0x7f9d880b2100 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.165+0000 7f9d8d400640 1 -- 172.21.15.100:0/3914458 >> 172.21.15.100:0/3914458 conn(0x7f9d8806d620 msgr2=0x7f9d88068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:29.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.165+0000 7f9d8d400640 1 -- 172.21.15.100:0/3914458 shutdown_connections 2024-09-17T04:33:29.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.165+0000 7f9d8d400640 1 -- 172.21.15.100:0/3914458 wait complete. 2024-09-17T04:33:29.167 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:33:29.168+0000 7f30b6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:33:29.167 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:33:29.168+0000 7f30b6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:33:29.168 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:33:29.168+0000 7f30b6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:33:29.168 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:33:29.168+0000 7f30b6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:33:29.168 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:33:29.168+0000 7f30b6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:33:29.186 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:33:29.187+0000 7f30b6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:33:29.186 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:33:29.187+0000 7f30b6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:33:29.186 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:33:29.187+0000 7f30b6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:33:29.186 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:33:29.187+0000 7f30b6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:33:29.187 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:33:29.187+0000 7f30b6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:33:29.192 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol8751 subvol_grp19350 2024-09-17T04:33:29.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.419+0000 7f4b25a00640 1 -- 172.21.15.100:0/3721721027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b04003c10 msgr2=0x7f4b04004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:29.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.419+0000 7f4b25a00640 1 --2- 172.21.15.100:0/3721721027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b04003c10 0x7f4b04004060 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f4b10001830 tx=0x7f4b04005830 comp rx=0 tx=0).stop 2024-09-17T04:33:29.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.419+0000 7f4b25000640 1 -- 172.21.15.100:0/3721721027 reap_dead start 2024-09-17T04:33:29.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.420+0000 7f4b25a00640 1 -- 172.21.15.100:0/3721721027 shutdown_connections 2024-09-17T04:33:29.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.420+0000 7f4b25a00640 1 -- 172.21.15.100:0/3721721027 >> 172.21.15.100:0/3721721027 conn(0x7f4b200b1dd0 msgr2=0x7f4b200b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:29.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.420+0000 7f4b25a00640 1 -- 172.21.15.100:0/3721721027 shutdown_connections 2024-09-17T04:33:29.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.420+0000 7f4b25a00640 1 -- 172.21.15.100:0/3721721027 wait complete. 2024-09-17T04:33:29.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.422+0000 7f4b25a00640 1 Processor -- start 2024-09-17T04:33:29.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.422+0000 7f4b25a00640 1 -- start start 2024-09-17T04:33:29.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.423+0000 7f4b25a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b04003c10 0x7f4b20154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:29.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.423+0000 7f4b25a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b2014e900 0x7f4b20155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:29.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.423+0000 7f4b25a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b2014b260 0x7f4b2014b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:29.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.423+0000 7f4b25a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4b200b4940 con 0x7f4b2014e900 2024-09-17T04:33:29.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.423+0000 7f4b25a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4b200b47c0 con 0x7f4b2014b260 2024-09-17T04:33:29.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.423+0000 7f4b25a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4b200b4ac0 con 0x7f4b04003c10 2024-09-17T04:33:29.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.424+0000 7f4b17400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b2014e900 0x7f4b20155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:29.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.424+0000 7f4b17400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b2014e900 0x7f4b20155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42138/0 (socket says 172.21.15.100:42138) 2024-09-17T04:33:29.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.424+0000 7f4b17400640 1 -- 172.21.15.100:0/3514743112 learned_addr learned my addr 172.21.15.100:0/3514743112 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:29.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.424+0000 7f4b1fe00640 1 --2- 172.21.15.100:0/3514743112 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b04003c10 0x7f4b20154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:29.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.424+0000 7f4b25000640 1 --2- 172.21.15.100:0/3514743112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b2014b260 0x7f4b2014b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:29.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.424+0000 7f4b17400640 1 -- 172.21.15.100:0/3514743112 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b04003c10 msgr2=0x7f4b20154000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:29.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.424+0000 7f4b17400640 1 --2- 172.21.15.100:0/3514743112 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b04003c10 0x7f4b20154000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.424+0000 7f4b17400640 1 -- 172.21.15.100:0/3514743112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b2014b260 msgr2=0x7f4b2014b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:29.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.424+0000 7f4b17400640 1 --2- 172.21.15.100:0/3514743112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b2014b260 0x7f4b2014b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.424+0000 7f4b17400640 1 -- 172.21.15.100:0/3514743112 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b20154540 con 0x7f4b2014e900 2024-09-17T04:33:29.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.424+0000 7f4b1fe00640 1 --2- 172.21.15.100:0/3514743112 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b04003c10 0x7f4b20154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:29.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.424+0000 7f4b25000640 1 --2- 172.21.15.100:0/3514743112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b2014b260 0x7f4b2014b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:29.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.424+0000 7f4b17400640 1 --2- 172.21.15.100:0/3514743112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b2014e900 0x7f4b20155ba0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f4b080066b0 tx=0x7f4b08003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:29.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.425+0000 7f4b1cc00640 1 -- 172.21.15.100:0/3514743112 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b08015070 con 0x7f4b2014e900 2024-09-17T04:33:29.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.425+0000 7f4b1cc00640 1 -- 172.21.15.100:0/3514743112 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b0800f9e0 con 0x7f4b2014e900 2024-09-17T04:33:29.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.425+0000 7f4b25a00640 1 -- 172.21.15.100:0/3514743112 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b200ce580 con 0x7f4b2014e900 2024-09-17T04:33:29.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.425+0000 7f4b1cc00640 1 -- 172.21.15.100:0/3514743112 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0800fe30 con 0x7f4b2014e900 2024-09-17T04:33:29.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.425+0000 7f4b25a00640 1 -- 172.21.15.100:0/3514743112 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b200ce820 con 0x7f4b2014e900 2024-09-17T04:33:29.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.426+0000 7f4b25a00640 1 -- 172.21.15.100:0/3514743112 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b200af5f0 con 0x7f4b2014e900 2024-09-17T04:33:29.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.427+0000 7f4b1cc00640 1 -- 172.21.15.100:0/3514743112 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) ==== 102257+0+0 (secure 0 0 0) 0x7f4b08011430 con 0x7f4b2014e900 2024-09-17T04:33:29.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.428+0000 7f4b1cc00640 1 --2- 172.21.15.100:0/3514743112 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4afc079490 0x7f4afc07b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:29.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.428+0000 7f4b1cc00640 1 -- 172.21.15.100:0/3514743112 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7f4b0809a420 con 0x7f4b2014e900 2024-09-17T04:33:29.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.428+0000 7f4b1fe00640 1 --2- 172.21.15.100:0/3514743112 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4afc079490 0x7f4afc07b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:29.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.429+0000 7f4b1fe00640 1 --2- 172.21.15.100:0/3514743112 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4afc079490 0x7f4afc07b9a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4b10032000 tx=0x7f4b10001360 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:29.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.433+0000 7f4b1cc00640 1 -- 172.21.15.100:0/3514743112 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4b080a1070 con 0x7f4b2014e900 2024-09-17T04:33:29.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:29.641+0000 7f4b25a00640 1 -- 172.21.15.100:0/3514743112 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol8751", "group_name": "subvol_grp19350", "target": ["mon-mgr", ""]}) -- 0x7f4b200cd8b0 con 0x7f4afc079490 2024-09-17T04:33:30.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:30.977+0000 7f4b1cc00640 1 -- 172.21.15.100:0/3514743112 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f4b0805fe90 con 0x7f4b2014e900 2024-09-17T04:33:34.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:34.859+0000 7f4b1cc00640 1 -- 172.21.15.100:0/3514743112 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4b200cd8b0 con 0x7f4afc079490 2024-09-17T04:33:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:34.862+0000 7f4b25a00640 1 -- 172.21.15.100:0/3514743112 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4afc079490 msgr2=0x7f4afc07b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:34.862+0000 7f4b25a00640 1 --2- 172.21.15.100:0/3514743112 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4afc079490 0x7f4afc07b9a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4b10032000 tx=0x7f4b10001360 comp rx=0 tx=0).stop 2024-09-17T04:33:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:34.862+0000 7f4b25a00640 1 -- 172.21.15.100:0/3514743112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b2014e900 msgr2=0x7f4b20155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:34.862+0000 7f4b25a00640 1 --2- 172.21.15.100:0/3514743112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b2014e900 0x7f4b20155ba0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f4b080066b0 tx=0x7f4b08003a80 comp rx=0 tx=0).stop 2024-09-17T04:33:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:34.863+0000 7f4b25a00640 1 -- 172.21.15.100:0/3514743112 shutdown_connections 2024-09-17T04:33:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:34.863+0000 7f4b25a00640 1 --2- 172.21.15.100:0/3514743112 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4afc079490 0x7f4afc07b9a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:34.863+0000 7f4b25a00640 1 --2- 172.21.15.100:0/3514743112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b2014b260 0x7f4b2014b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:34.863+0000 7f4b25a00640 1 --2- 172.21.15.100:0/3514743112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b2014e900 0x7f4b20155ba0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:34.863+0000 7f4b25a00640 1 --2- 172.21.15.100:0/3514743112 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b04003c10 0x7f4b20154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:34.863+0000 7f4b25a00640 1 -- 172.21.15.100:0/3514743112 >> 172.21.15.100:0/3514743112 conn(0x7f4b200b1dd0 msgr2=0x7f4b20073830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:34.863+0000 7f4b25a00640 1 -- 172.21.15.100:0/3514743112 shutdown_connections 2024-09-17T04:33:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:34.863+0000 7f4b25a00640 1 -- 172.21.15.100:0/3514743112 wait complete. 2024-09-17T04:33:34.900 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-17T04:33:34.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:33:34.989 INFO:teuthology.orchestra.run.smithi100.stdout:2 2024-09-17T04:33:34.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:33:34.997 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T04:33:34.998 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:33:35.077 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:33:35.172 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T04:33:35.172 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:33:35.172 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:33:35.172 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:33:35.208 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:33:35.208 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:33:35.284 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:33:35.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.545+0000 7f3140a00640 1 -- 172.21.15.100:0/2030809406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f313c15ce30 msgr2=0x7f313c15d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:35.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.545+0000 7f3140a00640 1 --2- 172.21.15.100:0/2030809406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f313c15ce30 0x7f313c15d240 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f313c0b4fa0 tx=0x7f313402aa00 comp rx=0 tx=0).stop 2024-09-17T04:33:35.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.545+0000 7f3140a00640 1 -- 172.21.15.100:0/2030809406 shutdown_connections 2024-09-17T04:33:35.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.545+0000 7f3140a00640 1 --2- 172.21.15.100:0/2030809406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f313c15ce30 0x7f313c15d240 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:35.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.545+0000 7f3140a00640 1 --2- 172.21.15.100:0/2030809406 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f313c0cfc80 0x7f313c156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:35.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.545+0000 7f3140a00640 1 -- 172.21.15.100:0/2030809406 >> 172.21.15.100:0/2030809406 conn(0x7f313c06d3d0 msgr2=0x7f313c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:35.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.545+0000 7f3140a00640 1 -- 172.21.15.100:0/2030809406 shutdown_connections 2024-09-17T04:33:35.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.545+0000 7f3140a00640 1 -- 172.21.15.100:0/2030809406 wait complete. 2024-09-17T04:33:35.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.547+0000 7f3140a00640 1 Processor -- start 2024-09-17T04:33:35.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.547+0000 7f3140a00640 1 -- start start 2024-09-17T04:33:35.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.549+0000 7f3140a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f313c0705b0 0x7f313c155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:35.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.549+0000 7f3140a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f313c0cfc80 0x7f313c0c4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:35.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.549+0000 7f3140a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f313c15ce30 0x7f313c0c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:35.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.549+0000 7f3140a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f313c0cd050 con 0x7f313c0705b0 2024-09-17T04:33:35.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.549+0000 7f3140a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f313c0cced0 con 0x7f313c15ce30 2024-09-17T04:33:35.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.549+0000 7f3140a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f313c0cd1d0 con 0x7f313c0cfc80 2024-09-17T04:33:35.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.549+0000 7f3139e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f313c0705b0 0x7f313c155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:35.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.549+0000 7f3139e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f313c0705b0 0x7f313c155620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55580/0 (socket says 172.21.15.100:55580) 2024-09-17T04:33:35.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.549+0000 7f3139e00640 1 -- 172.21.15.100:0/2892588846 learned_addr learned my addr 172.21.15.100:0/2892588846 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:35.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.549+0000 7f3139400640 1 --2- 172.21.15.100:0/2892588846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f313c0cfc80 0x7f313c0c4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:35.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.549+0000 7f313a800640 1 --2- 172.21.15.100:0/2892588846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f313c15ce30 0x7f313c0c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:35.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.550+0000 7f3139e00640 1 -- 172.21.15.100:0/2892588846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f313c0cfc80 msgr2=0x7f313c0c4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:35.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.550+0000 7f3139e00640 1 --2- 172.21.15.100:0/2892588846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f313c0cfc80 0x7f313c0c4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:35.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.550+0000 7f3139e00640 1 -- 172.21.15.100:0/2892588846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f313c15ce30 msgr2=0x7f313c0c4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:35.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.550+0000 7f3139e00640 1 --2- 172.21.15.100:0/2892588846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f313c15ce30 0x7f313c0c4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:35.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.550+0000 7f3139e00640 1 -- 172.21.15.100:0/2892588846 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313c0c7be0 con 0x7f313c0705b0 2024-09-17T04:33:35.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.550+0000 7f313a800640 1 --2- 172.21.15.100:0/2892588846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f313c15ce30 0x7f313c0c4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:35.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.550+0000 7f3139e00640 1 --2- 172.21.15.100:0/2892588846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f313c0705b0 0x7f313c155620 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f312c00d700 tx=0x7f312c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:35.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.550+0000 7f3132000640 1 -- 172.21.15.100:0/2892588846 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c017070 con 0x7f313c0705b0 2024-09-17T04:33:35.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.550+0000 7f3132000640 1 -- 172.21.15.100:0/2892588846 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f312c004310 con 0x7f313c0705b0 2024-09-17T04:33:35.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.550+0000 7f3140a00640 1 -- 172.21.15.100:0/2892588846 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313c1e6390 con 0x7f313c0705b0 2024-09-17T04:33:35.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.550+0000 7f3132000640 1 -- 172.21.15.100:0/2892588846 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c009a80 con 0x7f313c0705b0 2024-09-17T04:33:35.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.550+0000 7f3140a00640 1 -- 172.21.15.100:0/2892588846 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f313c0c7df0 con 0x7f313c0705b0 2024-09-17T04:33:35.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.551+0000 7f3140a00640 1 -- 172.21.15.100:0/2892588846 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f313c0cd6f0 con 0x7f313c0705b0 2024-09-17T04:33:35.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.553+0000 7f3132000640 1 -- 172.21.15.100:0/2892588846 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f312c009c20 con 0x7f313c0705b0 2024-09-17T04:33:35.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.553+0000 7f3132000640 1 --2- 172.21.15.100:0/2892588846 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f311c0795e0 0x7f311c07baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:35.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.554+0000 7f3132000640 1 -- 172.21.15.100:0/2892588846 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7f312c099800 con 0x7f313c0705b0 2024-09-17T04:33:35.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.554+0000 7f3139400640 1 --2- 172.21.15.100:0/2892588846 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f311c0795e0 0x7f311c07baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:35.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.555+0000 7f3139400640 1 --2- 172.21.15.100:0/2892588846 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f311c0795e0 0x7f311c07baf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f313c0c6ed0 tx=0x7f3128009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:35.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.558+0000 7f3132000640 1 -- 172.21.15.100:0/2892588846 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f312c062e30 con 0x7f313c0705b0 2024-09-17T04:33:35.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.752+0000 7f3140a00640 1 -- 172.21.15.100:0/2892588846 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f313c0afa70 con 0x7f313c0705b0 2024-09-17T04:33:35.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.753+0000 7f3132000640 1 -- 172.21.15.100:0/2892588846 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v156) ==== 80+0+0 (secure 0 0 0) 0x7f312c067860 con 0x7f313c0705b0 2024-09-17T04:33:35.754 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-09-17T04:33:35.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.755+0000 7f3140a00640 1 -- 172.21.15.100:0/2892588846 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f311c0795e0 msgr2=0x7f311c07baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:35.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.755+0000 7f3140a00640 1 --2- 172.21.15.100:0/2892588846 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f311c0795e0 0x7f311c07baf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f313c0c6ed0 tx=0x7f3128009040 comp rx=0 tx=0).stop 2024-09-17T04:33:35.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.755+0000 7f3140a00640 1 -- 172.21.15.100:0/2892588846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f313c0705b0 msgr2=0x7f313c155620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:35.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.755+0000 7f3140a00640 1 --2- 172.21.15.100:0/2892588846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f313c0705b0 0x7f313c155620 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f312c00d700 tx=0x7f312c00dbc0 comp rx=0 tx=0).stop 2024-09-17T04:33:35.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.755+0000 7f3140a00640 1 -- 172.21.15.100:0/2892588846 shutdown_connections 2024-09-17T04:33:35.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.755+0000 7f3140a00640 1 --2- 172.21.15.100:0/2892588846 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f311c0795e0 0x7f311c07baf0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:35.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.755+0000 7f3140a00640 1 --2- 172.21.15.100:0/2892588846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f313c15ce30 0x7f313c0c4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:35.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.755+0000 7f3140a00640 1 --2- 172.21.15.100:0/2892588846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f313c0cfc80 0x7f313c0c4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:35.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.755+0000 7f3140a00640 1 --2- 172.21.15.100:0/2892588846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f313c0705b0 0x7f313c155620 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:35.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.755+0000 7f3140a00640 1 -- 172.21.15.100:0/2892588846 >> 172.21.15.100:0/2892588846 conn(0x7f313c06d3d0 msgr2=0x7f313c06f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:35.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.756+0000 7f3140a00640 1 -- 172.21.15.100:0/2892588846 shutdown_connections 2024-09-17T04:33:35.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:35.756+0000 7f3140a00640 1 -- 172.21.15.100:0/2892588846 wait complete. 2024-09-17T04:33:35.782 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T04:33:35.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:35.783 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T04:33:35.875 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-09-17T04:33:35.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:35.875 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T04:33:35.924 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:33:35.924 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:33:35.924 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:33:35.962 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:33:35.962 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:33:36.039 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:33:36.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.258+0000 7f4c52800640 1 -- 172.21.15.100:0/2339467721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c4c0cfc80 msgr2=0x7f4c4c156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:36.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.258+0000 7f4c52800640 1 --2- 172.21.15.100:0/2339467721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c4c0cfc80 0x7f4c4c156e40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009830 tx=0x7f4c3c02a480 comp rx=0 tx=0).stop 2024-09-17T04:33:36.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.259+0000 7f4c52800640 1 -- 172.21.15.100:0/2339467721 shutdown_connections 2024-09-17T04:33:36.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.259+0000 7f4c52800640 1 --2- 172.21.15.100:0/2339467721 >> v2:172.21.15.184:3300/0 conn(0x7f4c4c0705b0 0x7f4c4c0709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:36.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.259+0000 7f4c52800640 1 --2- 172.21.15.100:0/2339467721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c4c0cfc80 0x7f4c4c156e40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:36.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.259+0000 7f4c52800640 1 -- 172.21.15.100:0/2339467721 >> 172.21.15.100:0/2339467721 conn(0x7f4c4c06d3d0 msgr2=0x7f4c4c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:36.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.259+0000 7f4c52800640 1 -- 172.21.15.100:0/2339467721 shutdown_connections 2024-09-17T04:33:36.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.259+0000 7f4c52800640 1 -- 172.21.15.100:0/2339467721 wait complete. 2024-09-17T04:33:36.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.261+0000 7f4c52800640 1 Processor -- start 2024-09-17T04:33:36.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.261+0000 7f4c52800640 1 -- start start 2024-09-17T04:33:36.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.262+0000 7f4c52800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c4c0705b0 0x7f4c4c1f5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:36.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.262+0000 7f4c52800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c4c0cfc80 0x7f4c4c0b1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:36.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.262+0000 7f4c52800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c4c15ce30 0x7f4c4c15b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:36.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.262+0000 7f4c52800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4c0cced0 con 0x7f4c4c0705b0 2024-09-17T04:33:36.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.262+0000 7f4c52800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4c0d0170 con 0x7f4c4c15ce30 2024-09-17T04:33:36.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.262+0000 7f4c52800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4c4c0cd050 con 0x7f4c4c0cfc80 2024-09-17T04:33:36.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.263+0000 7f4c4b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c4c0705b0 0x7f4c4c1f5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:36.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.263+0000 7f4c4b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c4c0705b0 0x7f4c4c1f5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55590/0 (socket says 172.21.15.100:55590) 2024-09-17T04:33:36.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.263+0000 7f4c4b400640 1 -- 172.21.15.100:0/3821289445 learned_addr learned my addr 172.21.15.100:0/3821289445 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:36.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.263+0000 7f4c4aa00640 1 --2- 172.21.15.100:0/3821289445 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c4c0cfc80 0x7f4c4c0b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:36.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.263+0000 7f4c4be00640 1 --2- 172.21.15.100:0/3821289445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c4c15ce30 0x7f4c4c15b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:36.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.263+0000 7f4c4b400640 1 -- 172.21.15.100:0/3821289445 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c4c0cfc80 msgr2=0x7f4c4c0b1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:36.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.263+0000 7f4c4b400640 1 --2- 172.21.15.100:0/3821289445 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c4c0cfc80 0x7f4c4c0b1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:36.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.263+0000 7f4c4b400640 1 -- 172.21.15.100:0/3821289445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c4c15ce30 msgr2=0x7f4c4c15b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:36.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.263+0000 7f4c4b400640 1 --2- 172.21.15.100:0/3821289445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c4c15ce30 0x7f4c4c15b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:36.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.263+0000 7f4c4b400640 1 -- 172.21.15.100:0/3821289445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4c1f9130 con 0x7f4c4c0705b0 2024-09-17T04:33:36.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.264+0000 7f4c4b400640 1 --2- 172.21.15.100:0/3821289445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c4c0705b0 0x7f4c4c1f5350 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c02aa70 tx=0x7f4c3c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:36.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.264+0000 7f4c43400640 1 -- 172.21.15.100:0/3821289445 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c005c90 con 0x7f4c4c0705b0 2024-09-17T04:33:36.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.264+0000 7f4c43400640 1 -- 172.21.15.100:0/3821289445 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c3c031e50 con 0x7f4c4c0705b0 2024-09-17T04:33:36.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.264+0000 7f4c52800640 1 -- 172.21.15.100:0/3821289445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c4c0b1ec0 con 0x7f4c4c0705b0 2024-09-17T04:33:36.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.264+0000 7f4c43400640 1 -- 172.21.15.100:0/3821289445 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c005260 con 0x7f4c4c0705b0 2024-09-17T04:33:36.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.264+0000 7f4c52800640 1 -- 172.21.15.100:0/3821289445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c4c1f96d0 con 0x7f4c4c0705b0 2024-09-17T04:33:36.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.265+0000 7f4c52800640 1 -- 172.21.15.100:0/3821289445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c4c0cd570 con 0x7f4c4c0705b0 2024-09-17T04:33:36.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.266+0000 7f4c43400640 1 -- 172.21.15.100:0/3821289445 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f4c3c005400 con 0x7f4c4c0705b0 2024-09-17T04:33:36.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.267+0000 7f4c43400640 1 --2- 172.21.15.100:0/3821289445 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4c2c0795b0 0x7f4c2c07bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:36.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.267+0000 7f4c43400640 1 -- 172.21.15.100:0/3821289445 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7f4c3c0ba870 con 0x7f4c4c0705b0 2024-09-17T04:33:36.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.267+0000 7f4c4aa00640 1 --2- 172.21.15.100:0/3821289445 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4c2c0795b0 0x7f4c2c07bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:36.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.268+0000 7f4c4aa00640 1 --2- 172.21.15.100:0/3821289445 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4c2c0795b0 0x7f4c2c07bac0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4c380095f0 tx=0x7f4c38005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:36.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.271+0000 7f4c43400640 1 -- 172.21.15.100:0/3821289445 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4c3c03f030 con 0x7f4c4c0705b0 2024-09-17T04:33:36.427 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:33:36.431 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:33:36.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.469+0000 7f4c52800640 1 -- 172.21.15.100:0/3821289445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4c4c0afa70 con 0x7f4c4c0705b0 2024-09-17T04:33:36.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.469+0000 7f4c43400640 1 -- 172.21.15.100:0/3821289445 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v156) ==== 80+0+0 (secure 0 0 0) 0x7f4c3c083e90 con 0x7f4c4c0705b0 2024-09-17T04:33:36.470 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-09-17T04:33:36.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.471+0000 7f4c52800640 1 -- 172.21.15.100:0/3821289445 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4c2c0795b0 msgr2=0x7f4c2c07bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:36.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.471+0000 7f4c52800640 1 --2- 172.21.15.100:0/3821289445 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4c2c0795b0 0x7f4c2c07bac0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4c380095f0 tx=0x7f4c38005f90 comp rx=0 tx=0).stop 2024-09-17T04:33:36.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.471+0000 7f4c52800640 1 -- 172.21.15.100:0/3821289445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c4c0705b0 msgr2=0x7f4c4c1f5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:36.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.471+0000 7f4c52800640 1 --2- 172.21.15.100:0/3821289445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c4c0705b0 0x7f4c4c1f5350 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c02aa70 tx=0x7f4c3c033c20 comp rx=0 tx=0).stop 2024-09-17T04:33:36.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.471+0000 7f4c52800640 1 -- 172.21.15.100:0/3821289445 shutdown_connections 2024-09-17T04:33:36.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.471+0000 7f4c52800640 1 --2- 172.21.15.100:0/3821289445 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4c2c0795b0 0x7f4c2c07bac0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:36.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.471+0000 7f4c52800640 1 --2- 172.21.15.100:0/3821289445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c4c15ce30 0x7f4c4c15b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:36.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.471+0000 7f4c52800640 1 --2- 172.21.15.100:0/3821289445 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c4c0cfc80 0x7f4c4c0b1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:36.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.471+0000 7f4c52800640 1 --2- 172.21.15.100:0/3821289445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c4c0705b0 0x7f4c4c1f5350 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:36.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.471+0000 7f4c52800640 1 -- 172.21.15.100:0/3821289445 >> 172.21.15.100:0/3821289445 conn(0x7f4c4c06d3d0 msgr2=0x7f4c4c0ad5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:36.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.472+0000 7f4c52800640 1 -- 172.21.15.100:0/3821289445 shutdown_connections 2024-09-17T04:33:36.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.472+0000 7f4c52800640 1 -- 172.21.15.100:0/3821289445 wait complete. 2024-09-17T04:33:36.497 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-17T04:33:36.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:36.497 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-17T04:33:36.559 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2024-09-17T04:33:36.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:36.560 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T04:33:36.635 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:33:36.635 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:33:36.636 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:33:36.671 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:33:36.671 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:33:36.671 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:33:36.671 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:33:36.710 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:33:36.710 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:33:36.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.933+0000 7ff7f2a00640 1 -- 172.21.15.100:0/200619367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7ec0b2750 msgr2=0x7ff7ec15f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:36.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.933+0000 7ff7f2a00640 1 --2- 172.21.15.100:0/200619367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7ec0b2750 0x7ff7ec15f3d0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0b3230 tx=0x7ff7d802a4d0 comp rx=0 tx=0).stop 2024-09-17T04:33:36.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.933+0000 7ff7f2a00640 1 -- 172.21.15.100:0/200619367 shutdown_connections 2024-09-17T04:33:36.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.933+0000 7ff7f2a00640 1 --2- 172.21.15.100:0/200619367 >> v2:172.21.15.184:3300/0 conn(0x7ff7ec15f910 0x7ff7ec15bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:36.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.933+0000 7ff7f2a00640 1 --2- 172.21.15.100:0/200619367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7ec0b2750 0x7ff7ec15f3d0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:36.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.933+0000 7ff7f2a00640 1 --2- 172.21.15.100:0/200619367 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7ec0aebc0 0x7ff7ec152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:36.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.933+0000 7ff7f2a00640 1 -- 172.21.15.100:0/200619367 >> 172.21.15.100:0/200619367 conn(0x7ff7ec06d730 msgr2=0x7ff7ec0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:36.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.933+0000 7ff7f2a00640 1 -- 172.21.15.100:0/200619367 shutdown_connections 2024-09-17T04:33:36.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.933+0000 7ff7f2a00640 1 -- 172.21.15.100:0/200619367 wait complete. 2024-09-17T04:33:36.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.935+0000 7ff7f2a00640 1 Processor -- start 2024-09-17T04:33:36.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.936+0000 7ff7f2a00640 1 -- start start 2024-09-17T04:33:36.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.937+0000 7ff7f2a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7ec0aebc0 0x7ff7ec14b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:36.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.937+0000 7ff7f2a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7ec0b2750 0x7ff7ec0ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:36.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.937+0000 7ff7f2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7ec15f910 0x7ff7ec0b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:36.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.937+0000 7ff7f2a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec15c660 con 0x7ff7ec15f910 2024-09-17T04:33:36.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.937+0000 7ff7f2a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec15c4e0 con 0x7ff7ec0b2750 2024-09-17T04:33:36.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.937+0000 7ff7f2a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff7ec15c7e0 con 0x7ff7ec0aebc0 2024-09-17T04:33:36.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.937+0000 7ff7f0a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7ec15f910 0x7ff7ec0b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:36.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.937+0000 7ff7f0a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7ec15f910 0x7ff7ec0b0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55594/0 (socket says 172.21.15.100:55594) 2024-09-17T04:33:36.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.937+0000 7ff7f0a00640 1 -- 172.21.15.100:0/3998881073 learned_addr learned my addr 172.21.15.100:0/3998881073 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:36.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.938+0000 7ff7ebe00640 1 --2- 172.21.15.100:0/3998881073 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7ec0aebc0 0x7ff7ec14b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:36.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.938+0000 7ff7eb400640 1 --2- 172.21.15.100:0/3998881073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7ec0b2750 0x7ff7ec0ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:36.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.938+0000 7ff7f0a00640 1 -- 172.21.15.100:0/3998881073 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7ec0aebc0 msgr2=0x7ff7ec14b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:36.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.938+0000 7ff7f0a00640 1 --2- 172.21.15.100:0/3998881073 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7ec0aebc0 0x7ff7ec14b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:36.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.938+0000 7ff7f0a00640 1 -- 172.21.15.100:0/3998881073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7ec0b2750 msgr2=0x7ff7ec0ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:36.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.938+0000 7ff7f0a00640 1 --2- 172.21.15.100:0/3998881073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7ec0b2750 0x7ff7ec0ca7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:36.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.938+0000 7ff7f0a00640 1 -- 172.21.15.100:0/3998881073 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7ec1f8f80 con 0x7ff7ec15f910 2024-09-17T04:33:36.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.938+0000 7ff7f0a00640 1 --2- 172.21.15.100:0/3998881073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7ec15f910 0x7ff7ec0b0cc0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7ff7e400eab0 tx=0x7ff7e400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:36.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.938+0000 7ff7e3e00640 1 -- 172.21.15.100:0/3998881073 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e400cea0 con 0x7ff7ec15f910 2024-09-17T04:33:36.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.938+0000 7ff7e3e00640 1 -- 172.21.15.100:0/3998881073 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7e4004b20 con 0x7ff7ec15f910 2024-09-17T04:33:36.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.938+0000 7ff7f2a00640 1 -- 172.21.15.100:0/3998881073 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7ec0cb440 con 0x7ff7ec15f910 2024-09-17T04:33:36.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.938+0000 7ff7e3e00640 1 -- 172.21.15.100:0/3998881073 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4005020 con 0x7ff7ec15f910 2024-09-17T04:33:36.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.939+0000 7ff7f2a00640 1 -- 172.21.15.100:0/3998881073 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7ec1f9500 con 0x7ff7ec15f910 2024-09-17T04:33:36.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.940+0000 7ff7f2a00640 1 -- 172.21.15.100:0/3998881073 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7ec0cb1b0 con 0x7ff7ec15f910 2024-09-17T04:33:36.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.941+0000 7ff7e3e00640 1 -- 172.21.15.100:0/3998881073 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7ff7e40051c0 con 0x7ff7ec15f910 2024-09-17T04:33:36.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.942+0000 7ff7e3e00640 1 --2- 172.21.15.100:0/3998881073 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff7cc0795e0 0x7ff7cc07baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:36.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.943+0000 7ff7e3e00640 1 -- 172.21.15.100:0/3998881073 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7ff7e409a5a0 con 0x7ff7ec15f910 2024-09-17T04:33:36.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.943+0000 7ff7ebe00640 1 --2- 172.21.15.100:0/3998881073 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff7cc0795e0 0x7ff7cc07baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:36.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.943+0000 7ff7ebe00640 1 --2- 172.21.15.100:0/3998881073 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff7cc0795e0 0x7ff7cc07baf0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc009980 tx=0x7ff7dc005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:36.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:36.947+0000 7ff7e3e00640 1 -- 172.21.15.100:0/3998881073 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff7e4064210 con 0x7ff7ec15f910 2024-09-17T04:33:37.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.167+0000 7ff7f2a00640 1 -- 172.21.15.100:0/3998881073 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7ec15f3d0 con 0x7ff7ec15f910 2024-09-17T04:33:37.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.168+0000 7ff7e3e00640 1 -- 172.21.15.100:0/3998881073 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5245 (secure 0 0 0) 0x7ff7e4063b10 con 0x7ff7ec15f910 2024-09-17T04:33:37.169 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:37.169 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":112,"btime":"2024-09-17T04:33:25:922164+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7295,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1131376012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1131376012},{"type":"v1","addr":"172.21.15.184:6801","nonce":1131376012}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":8230,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/3930702908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3930702908},{"type":"v1","addr":"172.21.15.100:6833","nonce":3930702908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":8808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2238795078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2238795078},{"type":"v1","addr":"172.21.15.100:6835","nonce":2238795078}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:14.363111+0000","modified":"2024-09-17T04:33:17.378067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8814":{"gid":8814,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/2965326201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2965326201},{"type":"v1","addr":"172.21.15.184:6835","nonce":2965326201}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8814,"qdb_cluster":[8814]},"id":12}]} 2024-09-17T04:33:37.169 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 112 2024-09-17T04:33:37.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.171+0000 7ff7f2a00640 1 -- 172.21.15.100:0/3998881073 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff7cc0795e0 msgr2=0x7ff7cc07baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:37.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.171+0000 7ff7f2a00640 1 --2- 172.21.15.100:0/3998881073 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff7cc0795e0 0x7ff7cc07baf0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc009980 tx=0x7ff7dc005f90 comp rx=0 tx=0).stop 2024-09-17T04:33:37.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.171+0000 7ff7f2a00640 1 -- 172.21.15.100:0/3998881073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7ec15f910 msgr2=0x7ff7ec0b0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:37.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.171+0000 7ff7f2a00640 1 --2- 172.21.15.100:0/3998881073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7ec15f910 0x7ff7ec0b0cc0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7ff7e400eab0 tx=0x7ff7e400ef70 comp rx=0 tx=0).stop 2024-09-17T04:33:37.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.171+0000 7ff7f2a00640 1 -- 172.21.15.100:0/3998881073 shutdown_connections 2024-09-17T04:33:37.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.171+0000 7ff7f2a00640 1 --2- 172.21.15.100:0/3998881073 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff7cc0795e0 0x7ff7cc07baf0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:37.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.171+0000 7ff7f2a00640 1 --2- 172.21.15.100:0/3998881073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7ec15f910 0x7ff7ec0b0cc0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:37.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.171+0000 7ff7f2a00640 1 --2- 172.21.15.100:0/3998881073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7ec0b2750 0x7ff7ec0ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:37.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.171+0000 7ff7f2a00640 1 --2- 172.21.15.100:0/3998881073 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7ec0aebc0 0x7ff7ec14b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:37.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.171+0000 7ff7f2a00640 1 -- 172.21.15.100:0/3998881073 >> 172.21.15.100:0/3998881073 conn(0x7ff7ec06d730 msgr2=0x7ff7ec0701a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:37.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.171+0000 7ff7f2a00640 1 -- 172.21.15.100:0/3998881073 shutdown_connections 2024-09-17T04:33:37.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.171+0000 7ff7f2a00640 1 -- 172.21.15.100:0/3998881073 wait complete. 2024-09-17T04:33:37.194 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:33:37.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.414+0000 7f16a2c00640 1 -- 172.21.15.100:0/3520916697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f169c0af170 msgr2=0x7f169c0b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:37.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.414+0000 7f16a2c00640 1 --2- 172.21.15.100:0/3520916697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f169c0af170 0x7f169c0b2f60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1688009890 tx=0x7f168802a530 comp rx=0 tx=0).stop 2024-09-17T04:33:37.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.414+0000 7f16a2c00640 1 -- 172.21.15.100:0/3520916697 shutdown_connections 2024-09-17T04:33:37.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.414+0000 7f16a2c00640 1 --2- 172.21.15.100:0/3520916697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f169c0af170 0x7f169c0b2f60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:37.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.414+0000 7f16a2c00640 1 -- 172.21.15.100:0/3520916697 >> 172.21.15.100:0/3520916697 conn(0x7f169c0b1e40 msgr2=0x7f169c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:37.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.414+0000 7f16a2c00640 1 -- 172.21.15.100:0/3520916697 shutdown_connections 2024-09-17T04:33:37.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.414+0000 7f16a2c00640 1 -- 172.21.15.100:0/3520916697 wait complete. 2024-09-17T04:33:37.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.417+0000 7f16a2c00640 1 Processor -- start 2024-09-17T04:33:37.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.417+0000 7f16a2c00640 1 -- start start 2024-09-17T04:33:37.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.418+0000 7f16a2c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f169c0ad9c0 0x7f169c1f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:37.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.418+0000 7f16a2c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f169c0af170 0x7f169c0ce290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:37.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.418+0000 7f16a2c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f169c15a0f0 0x7f169c09c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:37.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.418+0000 7f16a2c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f169c070d00 con 0x7f169c0af170 2024-09-17T04:33:37.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.418+0000 7f16a2c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f169c06fbb0 con 0x7f169c15a0f0 2024-09-17T04:33:37.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.418+0000 7f16a2c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f169c15d670 con 0x7f169c0ad9c0 2024-09-17T04:33:37.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.418+0000 7f16a0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f169c0af170 0x7f169c0ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:37.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.418+0000 7f16a0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f169c0af170 0x7f169c0ce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55606/0 (socket says 172.21.15.100:55606) 2024-09-17T04:33:37.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.418+0000 7f16a0e00640 1 -- 172.21.15.100:0/4206126056 learned_addr learned my addr 172.21.15.100:0/4206126056 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:37.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.419+0000 7f16a1800640 1 --2- 172.21.15.100:0/4206126056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f169c0ad9c0 0x7f169c1f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:37.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.419+0000 7f16a2200640 1 --2- 172.21.15.100:0/4206126056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f169c15a0f0 0x7f169c09c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:37.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.419+0000 7f16a0e00640 1 -- 172.21.15.100:0/4206126056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f169c0ad9c0 msgr2=0x7f169c1f5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:37.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.419+0000 7f16a0e00640 1 --2- 172.21.15.100:0/4206126056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f169c0ad9c0 0x7f169c1f5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:37.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.419+0000 7f16a0e00640 1 -- 172.21.15.100:0/4206126056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f169c15a0f0 msgr2=0x7f169c09c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:37.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.419+0000 7f16a0e00640 1 --2- 172.21.15.100:0/4206126056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f169c15a0f0 0x7f169c09c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:37.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.419+0000 7f16a0e00640 1 -- 172.21.15.100:0/4206126056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169c09bdd0 con 0x7f169c0af170 2024-09-17T04:33:37.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.419+0000 7f16a0e00640 1 --2- 172.21.15.100:0/4206126056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f169c0af170 0x7f169c0ce290 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f168802ab20 tx=0x7f1688033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:37.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.420+0000 7f1699600640 1 -- 172.21.15.100:0/4206126056 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688005c90 con 0x7f169c0af170 2024-09-17T04:33:37.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.420+0000 7f1699600640 1 -- 172.21.15.100:0/4206126056 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1688031e50 con 0x7f169c0af170 2024-09-17T04:33:37.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.420+0000 7f1699600640 1 -- 172.21.15.100:0/4206126056 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688005260 con 0x7f169c0af170 2024-09-17T04:33:37.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.420+0000 7f16a2c00640 1 -- 172.21.15.100:0/4206126056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f169c09ad30 con 0x7f169c0af170 2024-09-17T04:33:37.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.420+0000 7f16a2c00640 1 -- 172.21.15.100:0/4206126056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f169c09b2f0 con 0x7f169c0af170 2024-09-17T04:33:37.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.421+0000 7f16a2c00640 1 -- 172.21.15.100:0/4206126056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f169c09aaa0 con 0x7f169c0af170 2024-09-17T04:33:37.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.422+0000 7f1699600640 1 -- 172.21.15.100:0/4206126056 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f168802ad10 con 0x7f169c0af170 2024-09-17T04:33:37.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.424+0000 7f1699600640 1 --2- 172.21.15.100:0/4206126056 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f167c0795e0 0x7f167c07baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:37.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.424+0000 7f1699600640 1 -- 172.21.15.100:0/4206126056 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7f16880ba540 con 0x7f169c0af170 2024-09-17T04:33:37.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.424+0000 7f16a1800640 1 --2- 172.21.15.100:0/4206126056 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f167c0795e0 0x7f167c07baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:37.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.424+0000 7f16a1800640 1 --2- 172.21.15.100:0/4206126056 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f167c0795e0 0x7f167c07baf0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1690001120 tx=0x7f1690009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:37.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.428+0000 7f1699600640 1 -- 172.21.15.100:0/4206126056 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f168803f030 con 0x7f169c0af170 2024-09-17T04:33:37.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.644+0000 7f16a2c00640 1 -- 172.21.15.100:0/4206126056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f169c15dab0 con 0x7f169c0af170 2024-09-17T04:33:37.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.645+0000 7f1699600640 1 -- 172.21.15.100:0/4206126056 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5245 (secure 0 0 0) 0x7f1688083b60 con 0x7f169c0af170 2024-09-17T04:33:37.645 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:37.645 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":112,"btime":"2024-09-17T04:33:25:922164+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7295,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1131376012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1131376012},{"type":"v1","addr":"172.21.15.184:6801","nonce":1131376012}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":8230,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/3930702908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3930702908},{"type":"v1","addr":"172.21.15.100:6833","nonce":3930702908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":8808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2238795078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2238795078},{"type":"v1","addr":"172.21.15.100:6835","nonce":2238795078}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:14.363111+0000","modified":"2024-09-17T04:33:17.378067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8814":{"gid":8814,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/2965326201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2965326201},{"type":"v1","addr":"172.21.15.184:6835","nonce":2965326201}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8814,"qdb_cluster":[8814]},"id":12}]} 2024-09-17T04:33:37.645 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 112 2024-09-17T04:33:37.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.647+0000 7f16a2c00640 1 -- 172.21.15.100:0/4206126056 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f167c0795e0 msgr2=0x7f167c07baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:37.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.647+0000 7f16a2c00640 1 --2- 172.21.15.100:0/4206126056 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f167c0795e0 0x7f167c07baf0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1690001120 tx=0x7f1690009040 comp rx=0 tx=0).stop 2024-09-17T04:33:37.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.647+0000 7f16a2c00640 1 -- 172.21.15.100:0/4206126056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f169c0af170 msgr2=0x7f169c0ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:37.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.647+0000 7f16a2c00640 1 --2- 172.21.15.100:0/4206126056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f169c0af170 0x7f169c0ce290 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f168802ab20 tx=0x7f1688033c20 comp rx=0 tx=0).stop 2024-09-17T04:33:37.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.647+0000 7f16a2c00640 1 -- 172.21.15.100:0/4206126056 shutdown_connections 2024-09-17T04:33:37.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.647+0000 7f16a2c00640 1 --2- 172.21.15.100:0/4206126056 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f167c0795e0 0x7f167c07baf0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:37.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.647+0000 7f16a2c00640 1 --2- 172.21.15.100:0/4206126056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f169c15a0f0 0x7f169c09c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:37.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.647+0000 7f16a2c00640 1 --2- 172.21.15.100:0/4206126056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f169c0af170 0x7f169c0ce290 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:37.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.647+0000 7f16a2c00640 1 --2- 172.21.15.100:0/4206126056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f169c0ad9c0 0x7f169c1f5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:37.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.647+0000 7f16a2c00640 1 -- 172.21.15.100:0/4206126056 >> 172.21.15.100:0/4206126056 conn(0x7f169c0b1e40 msgr2=0x7f169c09dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:37.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.647+0000 7f16a2c00640 1 -- 172.21.15.100:0/4206126056 shutdown_connections 2024-09-17T04:33:37.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.647+0000 7f16a2c00640 1 -- 172.21.15.100:0/4206126056 wait complete. 2024-09-17T04:33:37.673 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:33:37.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.896+0000 7f0c3e400640 1 -- 172.21.15.100:0/2765914491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c380d2860 msgr2=0x7f0c380d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:37.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.896+0000 7f0c3e400640 1 --2- 172.21.15.100:0/2765914491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c380d2860 0x7f0c380d2c70 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f0c380d08e0 tx=0x7f0c3002a8b0 comp rx=0 tx=0).stop 2024-09-17T04:33:37.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.896+0000 7f0c3e400640 1 -- 172.21.15.100:0/2765914491 shutdown_connections 2024-09-17T04:33:37.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.896+0000 7f0c3e400640 1 --2- 172.21.15.100:0/2765914491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c380d2860 0x7f0c380d2c70 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:37.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.896+0000 7f0c3e400640 1 --2- 172.21.15.100:0/2765914491 >> v2:172.21.15.184:3300/0 conn(0x7f0c380cf8c0 0x7f0c380d2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:37.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.896+0000 7f0c3e400640 1 -- 172.21.15.100:0/2765914491 >> 172.21.15.100:0/2765914491 conn(0x7f0c3809d8a0 msgr2=0x7f0c3808bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:37.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.897+0000 7f0c3e400640 1 -- 172.21.15.100:0/2765914491 shutdown_connections 2024-09-17T04:33:37.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.897+0000 7f0c3e400640 1 -- 172.21.15.100:0/2765914491 wait complete. 2024-09-17T04:33:37.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.899+0000 7f0c3e400640 1 Processor -- start 2024-09-17T04:33:37.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.899+0000 7f0c3e400640 1 -- start start 2024-09-17T04:33:37.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.901+0000 7f0c3e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c380b3820 0x7f0c38161390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:37.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.901+0000 7f0c3e400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c380d2860 0x7f0c380ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:37.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.901+0000 7f0c3e400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c380cde10 0x7f0c380caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:37.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.901+0000 7f0c3e400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0c380b4070 con 0x7f0c380b3820 2024-09-17T04:33:37.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.901+0000 7f0c3e400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0c380b3ef0 con 0x7f0c380cde10 2024-09-17T04:33:37.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.901+0000 7f0c3e400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0c380d31b0 con 0x7f0c380d2860 2024-09-17T04:33:37.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.901+0000 7f0c3d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c380b3820 0x7f0c38161390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:37.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.901+0000 7f0c3d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c380b3820 0x7f0c38161390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55630/0 (socket says 172.21.15.100:55630) 2024-09-17T04:33:37.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.901+0000 7f0c3d000640 1 -- 172.21.15.100:0/2391319783 learned_addr learned my addr 172.21.15.100:0/2391319783 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:37.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.901+0000 7f0c37e00640 1 --2- 172.21.15.100:0/2391319783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c380d2860 0x7f0c380ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:37.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.901+0000 7f0c3da00640 1 --2- 172.21.15.100:0/2391319783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c380cde10 0x7f0c380caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:37.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.901+0000 7f0c3d000640 1 -- 172.21.15.100:0/2391319783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c380d2860 msgr2=0x7f0c380ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:37.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.901+0000 7f0c3d000640 1 --2- 172.21.15.100:0/2391319783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c380d2860 0x7f0c380ca510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:37.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.901+0000 7f0c3d000640 1 -- 172.21.15.100:0/2391319783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c380cde10 msgr2=0x7f0c380caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:37.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.901+0000 7f0c3d000640 1 --2- 172.21.15.100:0/2391319783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c380cde10 0x7f0c380caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:37.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.902+0000 7f0c3d000640 1 -- 172.21.15.100:0/2391319783 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c380c8740 con 0x7f0c380b3820 2024-09-17T04:33:37.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.902+0000 7f0c3d000640 1 --2- 172.21.15.100:0/2391319783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c380b3820 0x7f0c38161390 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00c3c0 tx=0x7f0c2c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:37.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.902+0000 7f0c34c00640 1 -- 172.21.15.100:0/2391319783 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c017070 con 0x7f0c380b3820 2024-09-17T04:33:37.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.902+0000 7f0c3e400640 1 -- 172.21.15.100:0/2391319783 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c380cb200 con 0x7f0c380b3820 2024-09-17T04:33:37.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.902+0000 7f0c34c00640 1 -- 172.21.15.100:0/2391319783 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c2c0035f0 con 0x7f0c380b3820 2024-09-17T04:33:37.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.902+0000 7f0c34c00640 1 -- 172.21.15.100:0/2391319783 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c003a80 con 0x7f0c380b3820 2024-09-17T04:33:37.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.902+0000 7f0c3e400640 1 -- 172.21.15.100:0/2391319783 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c380c8ce0 con 0x7f0c380b3820 2024-09-17T04:33:37.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.903+0000 7f0c3e400640 1 -- 172.21.15.100:0/2391319783 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c3815a190 con 0x7f0c380b3820 2024-09-17T04:33:37.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.904+0000 7f0c34c00640 1 -- 172.21.15.100:0/2391319783 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f0c2c01a070 con 0x7f0c380b3820 2024-09-17T04:33:37.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.905+0000 7f0c34c00640 1 --2- 172.21.15.100:0/2391319783 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0c180795e0 0x7f0c1807baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:37.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.905+0000 7f0c34c00640 1 -- 172.21.15.100:0/2391319783 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7f0c2c09b6c0 con 0x7f0c380b3820 2024-09-17T04:33:37.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.906+0000 7f0c37e00640 1 --2- 172.21.15.100:0/2391319783 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0c180795e0 0x7f0c1807baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:37.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.906+0000 7f0c37e00640 1 --2- 172.21.15.100:0/2391319783 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0c180795e0 0x7f0c1807baf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0c24009b30 tx=0x7f0c24005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:37.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:37.910+0000 7f0c34c00640 1 -- 172.21.15.100:0/2391319783 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0c3815a190 con 0x7f0c380b3820 2024-09-17T04:33:38.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.095+0000 7f0c3e400640 1 -- 172.21.15.100:0/2391319783 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c380d2320 con 0x7f0c380b3820 2024-09-17T04:33:38.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.096+0000 7f0c34c00640 1 -- 172.21.15.100:0/2391319783 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+16876 (secure 0 0 0) 0x7f0c2c0620a0 con 0x7f0c380b3820 2024-09-17T04:33:38.097 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:38.097 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":156,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:33:15.880224+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:33:12.271566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:13.334445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:33:38.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.099+0000 7f0c3e400640 1 -- 172.21.15.100:0/2391319783 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0c180795e0 msgr2=0x7f0c1807baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:38.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.099+0000 7f0c3e400640 1 --2- 172.21.15.100:0/2391319783 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0c180795e0 0x7f0c1807baf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0c24009b30 tx=0x7f0c24005f90 comp rx=0 tx=0).stop 2024-09-17T04:33:38.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.099+0000 7f0c3e400640 1 -- 172.21.15.100:0/2391319783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c380b3820 msgr2=0x7f0c38161390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:38.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.099+0000 7f0c3e400640 1 --2- 172.21.15.100:0/2391319783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c380b3820 0x7f0c38161390 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00c3c0 tx=0x7f0c2c00c880 comp rx=0 tx=0).stop 2024-09-17T04:33:38.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.099+0000 7f0c3e400640 1 -- 172.21.15.100:0/2391319783 shutdown_connections 2024-09-17T04:33:38.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.099+0000 7f0c3e400640 1 --2- 172.21.15.100:0/2391319783 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0c180795e0 0x7f0c1807baf0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:38.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.099+0000 7f0c3e400640 1 --2- 172.21.15.100:0/2391319783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c380cde10 0x7f0c380caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:38.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.099+0000 7f0c3e400640 1 --2- 172.21.15.100:0/2391319783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c380d2860 0x7f0c380ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:38.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.099+0000 7f0c3e400640 1 --2- 172.21.15.100:0/2391319783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c380b3820 0x7f0c38161390 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:38.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.099+0000 7f0c3e400640 1 -- 172.21.15.100:0/2391319783 >> 172.21.15.100:0/2391319783 conn(0x7f0c3809d8a0 msgr2=0x7f0c38162a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:38.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.099+0000 7f0c3e400640 1 -- 172.21.15.100:0/2391319783 shutdown_connections 2024-09-17T04:33:38.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.099+0000 7f0c3e400640 1 -- 172.21.15.100:0/2391319783 wait complete. 2024-09-17T04:33:38.122 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T04:33:38.123 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:33:38.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.344+0000 7f236da00640 1 -- 172.21.15.100:0/3512292211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f234c005350 msgr2=0x7f234c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:38.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.344+0000 7f236da00640 1 --2- 172.21.15.100:0/3512292211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f234c005350 0x7f234c00b9c0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f236000c630 tx=0x7f234c005830 comp rx=0 tx=0).stop 2024-09-17T04:33:38.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.344+0000 7f236da00640 1 -- 172.21.15.100:0/3512292211 shutdown_connections 2024-09-17T04:33:38.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.344+0000 7f236da00640 1 --2- 172.21.15.100:0/3512292211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f234c005350 0x7f234c00b9c0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:38.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.344+0000 7f236da00640 1 -- 172.21.15.100:0/3512292211 >> 172.21.15.100:0/3512292211 conn(0x7f236806d730 msgr2=0x7f23680b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:38.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.344+0000 7f236da00640 1 -- 172.21.15.100:0/3512292211 shutdown_connections 2024-09-17T04:33:38.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.344+0000 7f236da00640 1 -- 172.21.15.100:0/3512292211 wait complete. 2024-09-17T04:33:38.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.346+0000 7f236da00640 1 Processor -- start 2024-09-17T04:33:38.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.347+0000 7f236da00640 1 -- start start 2024-09-17T04:33:38.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.348+0000 7f236da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f234c005350 0x7f2368099a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:38.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.348+0000 7f236da00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23681f6900 0x7f23681f6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:38.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.348+0000 7f236da00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23680cfb40 0x7f23680cff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:38.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.348+0000 7f236da00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f23680ad4f0 con 0x7f234c005350 2024-09-17T04:33:38.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.348+0000 7f236da00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f23680b4e30 con 0x7f23681f6900 2024-09-17T04:33:38.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.348+0000 7f236da00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f23680ad670 con 0x7f23680cfb40 2024-09-17T04:33:38.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.348+0000 7f2366a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f234c005350 0x7f2368099a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:38.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.348+0000 7f2366a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f234c005350 0x7f2368099a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55652/0 (socket says 172.21.15.100:55652) 2024-09-17T04:33:38.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.348+0000 7f2366a00640 1 -- 172.21.15.100:0/3058235950 learned_addr learned my addr 172.21.15.100:0/3058235950 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:38.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.349+0000 7f235e000640 1 --2- 172.21.15.100:0/3058235950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23681f6900 0x7f23681f6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:38.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.349+0000 7f2367400640 1 --2- 172.21.15.100:0/3058235950 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23680cfb40 0x7f23680cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:38.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.349+0000 7f2366a00640 1 -- 172.21.15.100:0/3058235950 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23680cfb40 msgr2=0x7f23680cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:38.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.349+0000 7f2366a00640 1 --2- 172.21.15.100:0/3058235950 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23680cfb40 0x7f23680cff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:38.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.349+0000 7f2366a00640 1 -- 172.21.15.100:0/3058235950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23681f6900 msgr2=0x7f23681f6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:38.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.349+0000 7f2366a00640 1 --2- 172.21.15.100:0/3058235950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23681f6900 0x7f23681f6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:38.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.349+0000 7f2366a00640 1 -- 172.21.15.100:0/3058235950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23680d0490 con 0x7f234c005350 2024-09-17T04:33:38.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.349+0000 7f2367400640 1 --2- 172.21.15.100:0/3058235950 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23680cfb40 0x7f23680cff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:38.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.349+0000 7f235e000640 1 --2- 172.21.15.100:0/3058235950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23681f6900 0x7f23681f6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:38.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.349+0000 7f2366a00640 1 --2- 172.21.15.100:0/3058235950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f234c005350 0x7f2368099a80 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f2358011ae0 tx=0x7f2358009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:38.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.349+0000 7f235f400640 1 -- 172.21.15.100:0/3058235950 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235801e070 con 0x7f234c005350 2024-09-17T04:33:38.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.349+0000 7f235f400640 1 -- 172.21.15.100:0/3058235950 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2358004460 con 0x7f234c005350 2024-09-17T04:33:38.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.349+0000 7f236da00640 1 -- 172.21.15.100:0/3058235950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23681f76c0 con 0x7f234c005350 2024-09-17T04:33:38.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.349+0000 7f235f400640 1 -- 172.21.15.100:0/3058235950 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235800fc60 con 0x7f234c005350 2024-09-17T04:33:38.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.349+0000 7f236da00640 1 -- 172.21.15.100:0/3058235950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23681f38b0 con 0x7f234c005350 2024-09-17T04:33:38.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.350+0000 7f236da00640 1 -- 172.21.15.100:0/3058235950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23681535c0 con 0x7f234c005350 2024-09-17T04:33:38.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.352+0000 7f235f400640 1 -- 172.21.15.100:0/3058235950 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f2358021070 con 0x7f234c005350 2024-09-17T04:33:38.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.352+0000 7f235f400640 1 --2- 172.21.15.100:0/3058235950 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f23440795b0 0x7f234407bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:38.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.353+0000 7f235f400640 1 -- 172.21.15.100:0/3058235950 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7f235809a070 con 0x7f234c005350 2024-09-17T04:33:38.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.353+0000 7f235e000640 1 --2- 172.21.15.100:0/3058235950 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f23440795b0 0x7f234407bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:38.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.353+0000 7f235e000640 1 --2- 172.21.15.100:0/3058235950 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f23440795b0 0x7f234407bac0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2368161220 tx=0x7f2350006b70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:38.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.357+0000 7f235f400640 1 -- 172.21.15.100:0/3058235950 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f23680d0490 con 0x7f234c005350 2024-09-17T04:33:38.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.574+0000 7f236da00640 1 -- 172.21.15.100:0/3058235950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f236815e7e0 con 0x7f234c005350 2024-09-17T04:33:38.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.574+0000 7f235f400640 1 -- 172.21.15.100:0/3058235950 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5245 (secure 0 0 0) 0x7f23580638b0 con 0x7f234c005350 2024-09-17T04:33:38.575 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:38.575 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":112,"btime":"2024-09-17T04:33:25:922164+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7295,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1131376012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1131376012},{"type":"v1","addr":"172.21.15.184:6801","nonce":1131376012}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":8230,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/3930702908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3930702908},{"type":"v1","addr":"172.21.15.100:6833","nonce":3930702908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":8808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2238795078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2238795078},{"type":"v1","addr":"172.21.15.100:6835","nonce":2238795078}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:14.363111+0000","modified":"2024-09-17T04:33:17.378067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8814":{"gid":8814,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/2965326201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2965326201},{"type":"v1","addr":"172.21.15.184:6835","nonce":2965326201}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8814,"qdb_cluster":[8814]},"id":12}]} 2024-09-17T04:33:38.575 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 112 2024-09-17T04:33:38.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.577+0000 7f236da00640 1 -- 172.21.15.100:0/3058235950 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f23440795b0 msgr2=0x7f234407bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:38.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.577+0000 7f236da00640 1 --2- 172.21.15.100:0/3058235950 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f23440795b0 0x7f234407bac0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2368161220 tx=0x7f2350006b70 comp rx=0 tx=0).stop 2024-09-17T04:33:38.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.577+0000 7f236da00640 1 -- 172.21.15.100:0/3058235950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f234c005350 msgr2=0x7f2368099a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:38.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.577+0000 7f236da00640 1 --2- 172.21.15.100:0/3058235950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f234c005350 0x7f2368099a80 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f2358011ae0 tx=0x7f2358009e40 comp rx=0 tx=0).stop 2024-09-17T04:33:38.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.577+0000 7f236da00640 1 -- 172.21.15.100:0/3058235950 shutdown_connections 2024-09-17T04:33:38.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.577+0000 7f236da00640 1 --2- 172.21.15.100:0/3058235950 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f23440795b0 0x7f234407bac0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:38.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.577+0000 7f236da00640 1 --2- 172.21.15.100:0/3058235950 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23680cfb40 0x7f23680cff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:38.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.577+0000 7f236da00640 1 --2- 172.21.15.100:0/3058235950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23681f6900 0x7f23681f6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:38.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.577+0000 7f236da00640 1 --2- 172.21.15.100:0/3058235950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f234c005350 0x7f2368099a80 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:38.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.577+0000 7f236da00640 1 -- 172.21.15.100:0/3058235950 >> 172.21.15.100:0/3058235950 conn(0x7f236806d730 msgr2=0x7f2368152a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:38.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.577+0000 7f236da00640 1 -- 172.21.15.100:0/3058235950 shutdown_connections 2024-09-17T04:33:38.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.577+0000 7f236da00640 1 -- 172.21.15.100:0/3058235950 wait complete. 2024-09-17T04:33:38.599 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:33:38.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.818+0000 7f1b6d000640 1 -- 172.21.15.100:0/1723567580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b680b9e00 msgr2=0x7f1b68162a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:38.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.818+0000 7f1b6d000640 1 --2- 172.21.15.100:0/1723567580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b680b9e00 0x7f1b68162a50 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f1b58005fd0 tx=0x7f1b5802a540 comp rx=0 tx=0).stop 2024-09-17T04:33:38.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.818+0000 7f1b6d000640 1 -- 172.21.15.100:0/1723567580 shutdown_connections 2024-09-17T04:33:38.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.818+0000 7f1b6d000640 1 --2- 172.21.15.100:0/1723567580 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b680b3160 0x7f1b680d24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:38.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.818+0000 7f1b6d000640 1 --2- 172.21.15.100:0/1723567580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b680b9e00 0x7f1b68162a50 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:38.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.818+0000 7f1b6d000640 1 -- 172.21.15.100:0/1723567580 >> 172.21.15.100:0/1723567580 conn(0x7f1b6808bfe0 msgr2=0x7f1b6809d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:38.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.818+0000 7f1b6d000640 1 -- 172.21.15.100:0/1723567580 shutdown_connections 2024-09-17T04:33:38.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.818+0000 7f1b6d000640 1 -- 172.21.15.100:0/1723567580 wait complete. 2024-09-17T04:33:38.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.820+0000 7f1b6d000640 1 Processor -- start 2024-09-17T04:33:38.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.820+0000 7f1b6d000640 1 -- start start 2024-09-17T04:33:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.822+0000 7f1b6d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b680b3160 0x7f1b681f8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.822+0000 7f1b6d000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b680b9e00 0x7f1b681f93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.822+0000 7f1b6d000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b680d29e0 0x7f1b681613b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.822+0000 7f1b6d000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1b680d44b0 con 0x7f1b680b3160 2024-09-17T04:33:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.822+0000 7f1b6d000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1b680d4330 con 0x7f1b680b9e00 2024-09-17T04:33:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.822+0000 7f1b6d000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1b680d4630 con 0x7f1b680d29e0 2024-09-17T04:33:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.822+0000 7f1b67400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b680b3160 0x7f1b681f8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.822+0000 7f1b67400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b680b3160 0x7f1b681f8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55666/0 (socket says 172.21.15.100:55666) 2024-09-17T04:33:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.822+0000 7f1b67400640 1 -- 172.21.15.100:0/2313230247 learned_addr learned my addr 172.21.15.100:0/2313230247 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.822+0000 7f1b5fe00640 1 --2- 172.21.15.100:0/2313230247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b680b9e00 0x7f1b681f93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.822+0000 7f1b67e00640 1 --2- 172.21.15.100:0/2313230247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b680d29e0 0x7f1b681613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.822+0000 7f1b67400640 1 -- 172.21.15.100:0/2313230247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b680d29e0 msgr2=0x7f1b681613b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.822+0000 7f1b67400640 1 --2- 172.21.15.100:0/2313230247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b680d29e0 0x7f1b681613b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.822+0000 7f1b67400640 1 -- 172.21.15.100:0/2313230247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b680b9e00 msgr2=0x7f1b681f93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.822+0000 7f1b67400640 1 --2- 172.21.15.100:0/2313230247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b680b9e00 0x7f1b681f93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.822+0000 7f1b67400640 1 -- 172.21.15.100:0/2313230247 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b681517a0 con 0x7f1b680b3160 2024-09-17T04:33:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.822+0000 7f1b5fe00640 1 --2- 172.21.15.100:0/2313230247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b680b9e00 0x7f1b681f93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.823+0000 7f1b67400640 1 --2- 172.21.15.100:0/2313230247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b680b3160 0x7f1b681f8eb0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f1b58006010 tx=0x7f1b580038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.823+0000 7f1b5f400640 1 -- 172.21.15.100:0/2313230247 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5803e030 con 0x7f1b680b3160 2024-09-17T04:33:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.823+0000 7f1b5f400640 1 -- 172.21.15.100:0/2313230247 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b58003d90 con 0x7f1b680b3160 2024-09-17T04:33:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.823+0000 7f1b5f400640 1 -- 172.21.15.100:0/2313230247 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b58035040 con 0x7f1b680b3160 2024-09-17T04:33:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.823+0000 7f1b6d000640 1 -- 172.21.15.100:0/2313230247 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b681e9f90 con 0x7f1b680b3160 2024-09-17T04:33:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.823+0000 7f1b6d000640 1 -- 172.21.15.100:0/2313230247 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b6814e9a0 con 0x7f1b680b3160 2024-09-17T04:33:38.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.824+0000 7f1b6d000640 1 -- 172.21.15.100:0/2313230247 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b680d4b00 con 0x7f1b680b3160 2024-09-17T04:33:38.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.825+0000 7f1b5f400640 1 -- 172.21.15.100:0/2313230247 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f1b58041030 con 0x7f1b680b3160 2024-09-17T04:33:38.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.826+0000 7f1b5f400640 1 --2- 172.21.15.100:0/2313230247 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1b440795b0 0x7f1b4407bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:38.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.826+0000 7f1b5f400640 1 -- 172.21.15.100:0/2313230247 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7f1b580b9ff0 con 0x7f1b680b3160 2024-09-17T04:33:38.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.827+0000 7f1b5fe00640 1 --2- 172.21.15.100:0/2313230247 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1b440795b0 0x7f1b4407bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:38.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.827+0000 7f1b5fe00640 1 --2- 172.21.15.100:0/2313230247 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1b440795b0 0x7f1b4407bac0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1b680b2510 tx=0x7f1b50005f10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:38.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:38.831+0000 7f1b5f400640 1 -- 172.21.15.100:0/2313230247 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f1b58083cf0 con 0x7f1b680b3160 2024-09-17T04:33:39.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.054+0000 7f1b6d000640 1 -- 172.21.15.100:0/2313230247 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b680ce210 con 0x7f1b680b3160 2024-09-17T04:33:39.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.054+0000 7f1b5f400640 1 -- 172.21.15.100:0/2313230247 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5245 (secure 0 0 0) 0x7f1b580835f0 con 0x7f1b680b3160 2024-09-17T04:33:39.055 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:39.055 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":112,"btime":"2024-09-17T04:33:25:922164+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7295,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1131376012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1131376012},{"type":"v1","addr":"172.21.15.184:6801","nonce":1131376012}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":8230,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/3930702908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3930702908},{"type":"v1","addr":"172.21.15.100:6833","nonce":3930702908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":8808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2238795078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2238795078},{"type":"v1","addr":"172.21.15.100:6835","nonce":2238795078}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:14.363111+0000","modified":"2024-09-17T04:33:17.378067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8814":{"gid":8814,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/2965326201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2965326201},{"type":"v1","addr":"172.21.15.184:6835","nonce":2965326201}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8814,"qdb_cluster":[8814]},"id":12}]} 2024-09-17T04:33:39.055 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 112 2024-09-17T04:33:39.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.057+0000 7f1b6d000640 1 -- 172.21.15.100:0/2313230247 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1b440795b0 msgr2=0x7f1b4407bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:39.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.057+0000 7f1b6d000640 1 --2- 172.21.15.100:0/2313230247 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1b440795b0 0x7f1b4407bac0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1b680b2510 tx=0x7f1b50005f10 comp rx=0 tx=0).stop 2024-09-17T04:33:39.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.057+0000 7f1b6d000640 1 -- 172.21.15.100:0/2313230247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b680b3160 msgr2=0x7f1b681f8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:39.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.057+0000 7f1b6d000640 1 --2- 172.21.15.100:0/2313230247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b680b3160 0x7f1b681f8eb0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f1b58006010 tx=0x7f1b580038a0 comp rx=0 tx=0).stop 2024-09-17T04:33:39.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.057+0000 7f1b6d000640 1 -- 172.21.15.100:0/2313230247 shutdown_connections 2024-09-17T04:33:39.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.057+0000 7f1b6d000640 1 --2- 172.21.15.100:0/2313230247 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1b440795b0 0x7f1b4407bac0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:39.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.057+0000 7f1b6d000640 1 --2- 172.21.15.100:0/2313230247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b680d29e0 0x7f1b681613b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:39.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.057+0000 7f1b6d000640 1 --2- 172.21.15.100:0/2313230247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b680b9e00 0x7f1b681f93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:39.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.057+0000 7f1b6d000640 1 --2- 172.21.15.100:0/2313230247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b680b3160 0x7f1b681f8eb0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:39.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.057+0000 7f1b6d000640 1 -- 172.21.15.100:0/2313230247 >> 172.21.15.100:0/2313230247 conn(0x7f1b6808bfe0 msgr2=0x7f1b68068f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:39.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.057+0000 7f1b6d000640 1 -- 172.21.15.100:0/2313230247 shutdown_connections 2024-09-17T04:33:39.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.057+0000 7f1b6d000640 1 -- 172.21.15.100:0/2313230247 wait complete. 2024-09-17T04:33:39.080 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:33:39.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.299+0000 7f4dc0000640 1 -- 172.21.15.100:0/3364117946 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4db815d760 msgr2=0x7f4db81535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:39.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.299+0000 7f4dc0000640 1 --2- 172.21.15.100:0/3364117946 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4db815d760 0x7f4db81535c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4dac005fd0 tx=0x7f4dac02aa10 comp rx=0 tx=0).stop 2024-09-17T04:33:39.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.299+0000 7f4dc0000640 1 -- 172.21.15.100:0/3364117946 shutdown_connections 2024-09-17T04:33:39.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.299+0000 7f4dc0000640 1 --2- 172.21.15.100:0/3364117946 >> v2:172.21.15.184:3300/0 conn(0x7f4db80b3550 0x7f4db80b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:39.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.299+0000 7f4dc0000640 1 --2- 172.21.15.100:0/3364117946 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4db815d760 0x7f4db81535c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:39.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.300+0000 7f4dc0000640 1 -- 172.21.15.100:0/3364117946 >> 172.21.15.100:0/3364117946 conn(0x7f4db806d730 msgr2=0x7f4db80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:39.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.300+0000 7f4dc0000640 1 -- 172.21.15.100:0/3364117946 shutdown_connections 2024-09-17T04:33:39.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.300+0000 7f4dc0000640 1 -- 172.21.15.100:0/3364117946 wait complete. 2024-09-17T04:33:39.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.302+0000 7f4dc0000640 1 Processor -- start 2024-09-17T04:33:39.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.302+0000 7f4dc0000640 1 -- start start 2024-09-17T04:33:39.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.303+0000 7f4dc0000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4db80ad0e0 0x7f4db81f6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:39.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.303+0000 7f4dc0000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4db80b3550 0x7f4db8155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:39.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.303+0000 7f4dc0000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4db815d760 0x7f4db8155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:39.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.303+0000 7f4dc0000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4db80b4da0 con 0x7f4db80b3550 2024-09-17T04:33:39.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.303+0000 7f4dc0000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4db80b4c20 con 0x7f4db815d760 2024-09-17T04:33:39.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.303+0000 7f4dc0000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4db80b3ea0 con 0x7f4db80ad0e0 2024-09-17T04:33:39.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.304+0000 7f4dbcc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4db80b3550 0x7f4db8155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:39.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.304+0000 7f4dbcc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4db80b3550 0x7f4db8155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55680/0 (socket says 172.21.15.100:55680) 2024-09-17T04:33:39.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.304+0000 7f4dbcc00640 1 -- 172.21.15.100:0/2052546534 learned_addr learned my addr 172.21.15.100:0/2052546534 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:39.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.304+0000 7f4dbd600640 1 --2- 172.21.15.100:0/2052546534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4db80ad0e0 0x7f4db81f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:39.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.304+0000 7f4dbe000640 1 --2- 172.21.15.100:0/2052546534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4db815d760 0x7f4db8155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:39.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.304+0000 7f4dbcc00640 1 -- 172.21.15.100:0/2052546534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4db80ad0e0 msgr2=0x7f4db81f6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:39.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.304+0000 7f4dbcc00640 1 --2- 172.21.15.100:0/2052546534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4db80ad0e0 0x7f4db81f6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:39.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.304+0000 7f4dbcc00640 1 -- 172.21.15.100:0/2052546534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4db815d760 msgr2=0x7f4db8155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:39.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.304+0000 7f4dbcc00640 1 --2- 172.21.15.100:0/2052546534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4db815d760 0x7f4db8155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:39.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.304+0000 7f4dbcc00640 1 -- 172.21.15.100:0/2052546534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db814ab90 con 0x7f4db80b3550 2024-09-17T04:33:39.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.304+0000 7f4dbcc00640 1 --2- 172.21.15.100:0/2052546534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4db80b3550 0x7f4db8155aa0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f4da4001eb0 tx=0x7f4da400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:39.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.305+0000 7f4db5600640 1 -- 172.21.15.100:0/2052546534 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4017070 con 0x7f4db80b3550 2024-09-17T04:33:39.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.305+0000 7f4db5600640 1 -- 172.21.15.100:0/2052546534 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4da40053c0 con 0x7f4db80b3550 2024-09-17T04:33:39.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.305+0000 7f4dc0000640 1 -- 172.21.15.100:0/2052546534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db8154fc0 con 0x7f4db80b3550 2024-09-17T04:33:39.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.305+0000 7f4db5600640 1 -- 172.21.15.100:0/2052546534 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4003830 con 0x7f4db80b3550 2024-09-17T04:33:39.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.305+0000 7f4dc0000640 1 -- 172.21.15.100:0/2052546534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db8155580 con 0x7f4db80b3550 2024-09-17T04:33:39.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.306+0000 7f4dc0000640 1 -- 172.21.15.100:0/2052546534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4db80b4330 con 0x7f4db80b3550 2024-09-17T04:33:39.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.307+0000 7f4db5600640 1 -- 172.21.15.100:0/2052546534 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f4da401a070 con 0x7f4db80b3550 2024-09-17T04:33:39.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.308+0000 7f4db5600640 1 --2- 172.21.15.100:0/2052546534 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4d980795b0 0x7f4d9807bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:39.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.308+0000 7f4db5600640 1 -- 172.21.15.100:0/2052546534 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7f4da409b6b0 con 0x7f4db80b3550 2024-09-17T04:33:39.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.308+0000 7f4dbd600640 1 --2- 172.21.15.100:0/2052546534 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4d980795b0 0x7f4d9807bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:39.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.309+0000 7f4dbd600640 1 --2- 172.21.15.100:0/2052546534 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4d980795b0 0x7f4d9807bac0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4dac035000 tx=0x7f4dac03b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:39.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.313+0000 7f4db5600640 1 -- 172.21.15.100:0/2052546534 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4da40621b0 con 0x7f4db80b3550 2024-09-17T04:33:39.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.497+0000 7f4dc0000640 1 -- 172.21.15.100:0/2052546534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4db8148880 con 0x7f4db80b3550 2024-09-17T04:33:39.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.499+0000 7f4db5600640 1 -- 172.21.15.100:0/2052546534 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+16876 (secure 0 0 0) 0x7f4da4066be0 con 0x7f4db80b3550 2024-09-17T04:33:39.499 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:39.500 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":156,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:33:15.880224+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:33:12.271566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:13.334445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:33:39.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.501+0000 7f4dc0000640 1 -- 172.21.15.100:0/2052546534 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4d980795b0 msgr2=0x7f4d9807bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:39.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.501+0000 7f4dc0000640 1 --2- 172.21.15.100:0/2052546534 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4d980795b0 0x7f4d9807bac0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4dac035000 tx=0x7f4dac03b040 comp rx=0 tx=0).stop 2024-09-17T04:33:39.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.501+0000 7f4dc0000640 1 -- 172.21.15.100:0/2052546534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4db80b3550 msgr2=0x7f4db8155aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:39.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.501+0000 7f4dc0000640 1 --2- 172.21.15.100:0/2052546534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4db80b3550 0x7f4db8155aa0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f4da4001eb0 tx=0x7f4da400c890 comp rx=0 tx=0).stop 2024-09-17T04:33:39.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.501+0000 7f4dc0000640 1 -- 172.21.15.100:0/2052546534 shutdown_connections 2024-09-17T04:33:39.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.501+0000 7f4dc0000640 1 --2- 172.21.15.100:0/2052546534 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4d980795b0 0x7f4d9807bac0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:39.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.501+0000 7f4dc0000640 1 --2- 172.21.15.100:0/2052546534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4db815d760 0x7f4db8155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:39.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.501+0000 7f4dc0000640 1 --2- 172.21.15.100:0/2052546534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4db80b3550 0x7f4db8155aa0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:39.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.501+0000 7f4dc0000640 1 --2- 172.21.15.100:0/2052546534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4db80ad0e0 0x7f4db81f6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:39.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.501+0000 7f4dc0000640 1 -- 172.21.15.100:0/2052546534 >> 172.21.15.100:0/2052546534 conn(0x7f4db806d730 msgr2=0x7f4db815dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:39.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.501+0000 7f4dc0000640 1 -- 172.21.15.100:0/2052546534 shutdown_connections 2024-09-17T04:33:39.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.501+0000 7f4dc0000640 1 -- 172.21.15.100:0/2052546534 wait complete. 2024-09-17T04:33:39.524 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T04:33:39.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.744+0000 7faa26200640 1 -- 172.21.15.100:0/289671485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa2015e770 msgr2=0x7faa200b7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:39.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.744+0000 7faa26200640 1 --2- 172.21.15.100:0/289671485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa2015e770 0x7faa200b7a00 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7faa200d0cc0 tx=0x7faa0c02a560 comp rx=0 tx=0).stop 2024-09-17T04:33:39.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.744+0000 7faa26200640 1 -- 172.21.15.100:0/289671485 shutdown_connections 2024-09-17T04:33:39.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.744+0000 7faa26200640 1 --2- 172.21.15.100:0/289671485 >> v2:172.21.15.184:3300/0 conn(0x7faa200b7f40 0x7faa200b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:39.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.744+0000 7faa26200640 1 --2- 172.21.15.100:0/289671485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa2015e770 0x7faa200b7a00 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:39.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.744+0000 7faa26200640 1 -- 172.21.15.100:0/289671485 >> 172.21.15.100:0/289671485 conn(0x7faa200d44f0 msgr2=0x7faa200d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:39.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.744+0000 7faa26200640 1 -- 172.21.15.100:0/289671485 shutdown_connections 2024-09-17T04:33:39.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.744+0000 7faa26200640 1 -- 172.21.15.100:0/289671485 wait complete. 2024-09-17T04:33:39.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.746+0000 7faa26200640 1 Processor -- start 2024-09-17T04:33:39.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.747+0000 7faa26200640 1 -- start start 2024-09-17T04:33:39.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.748+0000 7faa26200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa200b7f40 0x7faa200aadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:39.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.748+0000 7faa26200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa2015e770 0x7faa2014c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:39.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.748+0000 7faa26200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa2014c9c0 0x7faa2014cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:39.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.748+0000 7faa26200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faa200d39a0 con 0x7faa200b7f40 2024-09-17T04:33:39.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.748+0000 7faa26200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faa200d3820 con 0x7faa2014c9c0 2024-09-17T04:33:39.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.748+0000 7faa26200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faa200d3b20 con 0x7faa2015e770 2024-09-17T04:33:39.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.748+0000 7faa24e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa200b7f40 0x7faa200aadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:39.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.748+0000 7faa24e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa200b7f40 0x7faa200aadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55696/0 (socket says 172.21.15.100:55696) 2024-09-17T04:33:39.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.748+0000 7faa24e00640 1 -- 172.21.15.100:0/377236051 learned_addr learned my addr 172.21.15.100:0/377236051 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.749+0000 7faa24e00640 1 -- 172.21.15.100:0/377236051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa2015e770 msgr2=0x7faa2014c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.749+0000 7faa1fe00640 1 --2- 172.21.15.100:0/377236051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa2015e770 0x7faa2014c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.749+0000 7faa25800640 1 --2- 172.21.15.100:0/377236051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa2014c9c0 0x7faa2014cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.749+0000 7faa24e00640 1 --2- 172.21.15.100:0/377236051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa2015e770 0x7faa2014c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.749+0000 7faa24e00640 1 -- 172.21.15.100:0/377236051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa2014c9c0 msgr2=0x7faa2014cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.749+0000 7faa24e00640 1 --2- 172.21.15.100:0/377236051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa2014c9c0 0x7faa2014cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.749+0000 7faa24e00640 1 -- 172.21.15.100:0/377236051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa200ac9d0 con 0x7faa200b7f40 2024-09-17T04:33:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.749+0000 7faa25800640 1 --2- 172.21.15.100:0/377236051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa2014c9c0 0x7faa2014cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.749+0000 7faa24e00640 1 --2- 172.21.15.100:0/377236051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa200b7f40 0x7faa200aadc0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7faa1400c3f0 tx=0x7faa1400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:39.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.749+0000 7faa1cc00640 1 -- 172.21.15.100:0/377236051 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa14017070 con 0x7faa200b7f40 2024-09-17T04:33:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.749+0000 7faa1cc00640 1 -- 172.21.15.100:0/377236051 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa140035f0 con 0x7faa200b7f40 2024-09-17T04:33:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.749+0000 7faa26200640 1 -- 172.21.15.100:0/377236051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa200ab930 con 0x7faa200b7f40 2024-09-17T04:33:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.749+0000 7faa1cc00640 1 -- 172.21.15.100:0/377236051 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa14003a40 con 0x7faa200b7f40 2024-09-17T04:33:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.749+0000 7faa26200640 1 -- 172.21.15.100:0/377236051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa200abef0 con 0x7faa200b7f40 2024-09-17T04:33:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.751+0000 7faa26200640 1 -- 172.21.15.100:0/377236051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa200d2460 con 0x7faa200b7f40 2024-09-17T04:33:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.752+0000 7faa1cc00640 1 -- 172.21.15.100:0/377236051 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7faa1401a070 con 0x7faa200b7f40 2024-09-17T04:33:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.753+0000 7faa1cc00640 1 --2- 172.21.15.100:0/377236051 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7faa000795b0 0x7faa0007bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.753+0000 7faa1cc00640 1 -- 172.21.15.100:0/377236051 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5765+0+0 (secure 0 0 0) 0x7faa14098b80 con 0x7faa200b7f40 2024-09-17T04:33:39.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.753+0000 7faa1fe00640 1 --2- 172.21.15.100:0/377236051 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7faa000795b0 0x7faa0007bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.754+0000 7faa1fe00640 1 --2- 172.21.15.100:0/377236051 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7faa000795b0 0x7faa0007bac0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7faa0c033000 tx=0x7faa0c004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:39.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.757+0000 7faa1cc00640 1 -- 172.21.15.100:0/377236051 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7faa14062880 con 0x7faa200b7f40 2024-09-17T04:33:39.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.980+0000 7faa26200640 1 -- 172.21.15.100:0/377236051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7faa200cf6c0 con 0x7faa200b7f40 2024-09-17T04:33:39.981 INFO:tasks.ceph.mon.a.smithi100.stderr:2024-09-17T04:33:39.980+0000 7fd3d0a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T04:33:39.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.982+0000 7faa1cc00640 1 -- 172.21.15.100:0/377236051 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v113) ==== 161+0+0 (secure 0 0 0) 0x7faa14062180 con 0x7faa200b7f40 2024-09-17T04:33:39.983 INFO:teuthology.orchestra.run.smithi100.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T04:33:39.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.985+0000 7faa26200640 1 -- 172.21.15.100:0/377236051 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7faa000795b0 msgr2=0x7faa0007bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:39.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.985+0000 7faa26200640 1 --2- 172.21.15.100:0/377236051 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7faa000795b0 0x7faa0007bac0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7faa0c033000 tx=0x7faa0c004730 comp rx=0 tx=0).stop 2024-09-17T04:33:39.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.985+0000 7faa26200640 1 -- 172.21.15.100:0/377236051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa200b7f40 msgr2=0x7faa200aadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:39.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.985+0000 7faa26200640 1 --2- 172.21.15.100:0/377236051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa200b7f40 0x7faa200aadc0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7faa1400c3f0 tx=0x7faa1400c8b0 comp rx=0 tx=0).stop 2024-09-17T04:33:39.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.986+0000 7faa26200640 1 -- 172.21.15.100:0/377236051 shutdown_connections 2024-09-17T04:33:39.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.986+0000 7faa26200640 1 --2- 172.21.15.100:0/377236051 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7faa000795b0 0x7faa0007bac0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:39.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.986+0000 7faa26200640 1 --2- 172.21.15.100:0/377236051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa2014c9c0 0x7faa2014cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:39.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.986+0000 7faa26200640 1 --2- 172.21.15.100:0/377236051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa2015e770 0x7faa2014c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:39.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.986+0000 7faa26200640 1 --2- 172.21.15.100:0/377236051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa200b7f40 0x7faa200aadc0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:39.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.986+0000 7faa26200640 1 -- 172.21.15.100:0/377236051 >> 172.21.15.100:0/377236051 conn(0x7faa200d44f0 msgr2=0x7faa20161a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:39.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.986+0000 7faa26200640 1 -- 172.21.15.100:0/377236051 shutdown_connections 2024-09-17T04:33:39.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:39.986+0000 7faa26200640 1 -- 172.21.15.100:0/377236051 wait complete. 2024-09-17T04:33:40.009 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T04:33:40.125 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-09-17T04:33:40.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.233+0000 7f46ad400640 1 -- 172.21.15.100:0/1841622785 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46a80742b0 msgr2=0x7f468c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:40.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.233+0000 7f46ad400640 1 --2- 172.21.15.100:0/1841622785 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46a80742b0 0x7f468c004800 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f46940016c0 tx=0x7f468c005830 comp rx=0 tx=0).stop 2024-09-17T04:33:40.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.233+0000 7f46aca00640 1 -- 172.21.15.100:0/1841622785 reap_dead start 2024-09-17T04:33:40.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.233+0000 7f46ad400640 1 -- 172.21.15.100:0/1841622785 shutdown_connections 2024-09-17T04:33:40.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.233+0000 7f46ad400640 1 -- 172.21.15.100:0/1841622785 >> 172.21.15.100:0/1841622785 conn(0x7f46a806d620 msgr2=0x7f46a80b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:40.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.233+0000 7f46ad400640 1 -- 172.21.15.100:0/1841622785 shutdown_connections 2024-09-17T04:33:40.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.233+0000 7f46ad400640 1 -- 172.21.15.100:0/1841622785 wait complete. 2024-09-17T04:33:40.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.235+0000 7f46ad400640 1 Processor -- start 2024-09-17T04:33:40.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.235+0000 7f46ad400640 1 -- start start 2024-09-17T04:33:40.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.237+0000 7f46ad400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46a80742b0 0x7f46a81f9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:40.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.237+0000 7f46ad400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46a8154250 0x7f46a80cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:40.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.237+0000 7f46ad400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46a8154660 0x7f46a80cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:40.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.237+0000 7f46ad400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f46a8072eb0 con 0x7f46a80742b0 2024-09-17T04:33:40.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.237+0000 7f46ad400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f46a80b4560 con 0x7f46a8154250 2024-09-17T04:33:40.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.237+0000 7f46ad400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f46a8073030 con 0x7f46a8154660 2024-09-17T04:33:40.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.237+0000 7f46a7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46a80742b0 0x7f46a81f9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:40.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.237+0000 7f46a7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46a80742b0 0x7f46a81f9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55718/0 (socket says 172.21.15.100:55718) 2024-09-17T04:33:40.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.237+0000 7f46a7e00640 1 -- 172.21.15.100:0/3242659214 learned_addr learned my addr 172.21.15.100:0/3242659214 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:40.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.237+0000 7f46aca00640 1 --2- 172.21.15.100:0/3242659214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46a8154660 0x7f46a80cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:40.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.237+0000 7f46a7400640 1 --2- 172.21.15.100:0/3242659214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46a8154250 0x7f46a80cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:40.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.238+0000 7f46a7e00640 1 -- 172.21.15.100:0/3242659214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46a8154660 msgr2=0x7f46a80cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:40.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.238+0000 7f46a7e00640 1 --2- 172.21.15.100:0/3242659214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46a8154660 0x7f46a80cc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:40.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.238+0000 7f46a7e00640 1 -- 172.21.15.100:0/3242659214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46a8154250 msgr2=0x7f46a80cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:40.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.238+0000 7f46a7e00640 1 --2- 172.21.15.100:0/3242659214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46a8154250 0x7f46a80cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:40.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.238+0000 7f46a7e00640 1 -- 172.21.15.100:0/3242659214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a81f5ef0 con 0x7f46a80742b0 2024-09-17T04:33:40.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.238+0000 7f46a7e00640 1 --2- 172.21.15.100:0/3242659214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46a80742b0 0x7f46a81f9ae0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f4698011d40 tx=0x7f469800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:40.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.238+0000 7f469fe00640 1 -- 172.21.15.100:0/3242659214 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4698003d80 con 0x7f46a80742b0 2024-09-17T04:33:40.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.238+0000 7f469fe00640 1 -- 172.21.15.100:0/3242659214 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4698004b20 con 0x7f46a80742b0 2024-09-17T04:33:40.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.238+0000 7f46ad400640 1 -- 172.21.15.100:0/3242659214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a81f61d0 con 0x7f46a80742b0 2024-09-17T04:33:40.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.238+0000 7f469fe00640 1 -- 172.21.15.100:0/3242659214 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4698004050 con 0x7f46a80742b0 2024-09-17T04:33:40.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.238+0000 7f46ad400640 1 -- 172.21.15.100:0/3242659214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a81f6790 con 0x7f46a80742b0 2024-09-17T04:33:40.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.240+0000 7f46ad400640 1 -- 172.21.15.100:0/3242659214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46a8073550 con 0x7f46a80742b0 2024-09-17T04:33:40.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.241+0000 7f469fe00640 1 -- 172.21.15.100:0/3242659214 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f46980041f0 con 0x7f46a80742b0 2024-09-17T04:33:40.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.242+0000 7f469fe00640 1 --2- 172.21.15.100:0/3242659214 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f46880795b0 0x7f468807bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:40.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.242+0000 7f469fe00640 1 -- 172.21.15.100:0/3242659214 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 5851+0+0 (secure 0 0 0) 0x7f469809a8e0 con 0x7f46a80742b0 2024-09-17T04:33:40.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.243+0000 7f46a7400640 1 --2- 172.21.15.100:0/3242659214 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f46880795b0 0x7f468807bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:40.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.243+0000 7f46a7400640 1 --2- 172.21.15.100:0/3242659214 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f46880795b0 0x7f468807bac0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4694032000 tx=0x7f4694001190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:40.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.247+0000 7f469fe00640 1 -- 172.21.15.100:0/3242659214 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4698064800 con 0x7f46a80742b0 2024-09-17T04:33:40.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.464+0000 7f46ad400640 1 -- 172.21.15.100:0/3242659214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f46a80b2ab0 con 0x7f46a80742b0 2024-09-17T04:33:40.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.984+0000 7f469fe00640 1 -- 172.21.15.100:0/3242659214 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v114) ==== 104+0+0 (secure 0 0 0) 0x7f4698064100 con 0x7f46a80742b0 2024-09-17T04:33:40.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.986+0000 7f46ad400640 1 -- 172.21.15.100:0/3242659214 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f46880795b0 msgr2=0x7f468807bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:40.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.986+0000 7f46ad400640 1 --2- 172.21.15.100:0/3242659214 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f46880795b0 0x7f468807bac0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4694032000 tx=0x7f4694001190 comp rx=0 tx=0).stop 2024-09-17T04:33:40.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.986+0000 7f46ad400640 1 -- 172.21.15.100:0/3242659214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46a80742b0 msgr2=0x7f46a81f9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:40.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.986+0000 7f46ad400640 1 --2- 172.21.15.100:0/3242659214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46a80742b0 0x7f46a81f9ae0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f4698011d40 tx=0x7f469800f790 comp rx=0 tx=0).stop 2024-09-17T04:33:40.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.987+0000 7f46ad400640 1 -- 172.21.15.100:0/3242659214 shutdown_connections 2024-09-17T04:33:40.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.987+0000 7f46ad400640 1 --2- 172.21.15.100:0/3242659214 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f46880795b0 0x7f468807bac0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:40.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.987+0000 7f46ad400640 1 --2- 172.21.15.100:0/3242659214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46a8154660 0x7f46a80cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:40.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.987+0000 7f46ad400640 1 --2- 172.21.15.100:0/3242659214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46a8154250 0x7f46a80cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:40.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.987+0000 7f46ad400640 1 --2- 172.21.15.100:0/3242659214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46a80742b0 0x7f46a81f9ae0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:40.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.987+0000 7f46ad400640 1 -- 172.21.15.100:0/3242659214 >> 172.21.15.100:0/3242659214 conn(0x7f46a806d620 msgr2=0x7f46a81604f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:40.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.987+0000 7f46ad400640 1 -- 172.21.15.100:0/3242659214 shutdown_connections 2024-09-17T04:33:40.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:40.987+0000 7f46ad400640 1 -- 172.21.15.100:0/3242659214 wait complete. 2024-09-17T04:33:41.010 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T04:33:41.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.235+0000 7fdfb1a00640 1 -- 172.21.15.100:0/988030981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfac06e5b0 msgr2=0x7fdfac06e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:41.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.235+0000 7fdfb1a00640 1 --2- 172.21.15.100:0/988030981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfac06e5b0 0x7fdfac06e9c0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4005fd0 tx=0x7fdfa402aa90 comp rx=0 tx=0).stop 2024-09-17T04:33:41.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.235+0000 7fdfb1a00640 1 -- 172.21.15.100:0/988030981 shutdown_connections 2024-09-17T04:33:41.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.235+0000 7fdfb1a00640 1 --2- 172.21.15.100:0/988030981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfac06e5b0 0x7fdfac06e9c0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.235+0000 7fdfb1a00640 1 --2- 172.21.15.100:0/988030981 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfac0d03b0 0x7fdfac0d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.235+0000 7fdfb1a00640 1 -- 172.21.15.100:0/988030981 >> 172.21.15.100:0/988030981 conn(0x7fdfac0cef30 msgr2=0x7fdfac0cf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:41.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.235+0000 7fdfb1a00640 1 -- 172.21.15.100:0/988030981 shutdown_connections 2024-09-17T04:33:41.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.235+0000 7fdfb1a00640 1 -- 172.21.15.100:0/988030981 wait complete. 2024-09-17T04:33:41.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.237+0000 7fdfb1a00640 1 Processor -- start 2024-09-17T04:33:41.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.237+0000 7fdfb1a00640 1 -- start start 2024-09-17T04:33:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.239+0000 7fdfb1a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfac06e5b0 0x7fdfac15bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.239+0000 7fdfb1a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfac0b7c90 0x7fdfac1622e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.239+0000 7fdfb1a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfac0d03b0 0x7fdfac0b0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.239+0000 7fdfb1a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac0d14f0 con 0x7fdfac06e5b0 2024-09-17T04:33:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.239+0000 7fdfb1a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac0d1370 con 0x7fdfac0b7c90 2024-09-17T04:33:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.239+0000 7fdfb1a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdfac0b75b0 con 0x7fdfac0d03b0 2024-09-17T04:33:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.239+0000 7fdfaaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfac06e5b0 0x7fdfac15bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.239+0000 7fdfaaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfac06e5b0 0x7fdfac15bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55744/0 (socket says 172.21.15.100:55744) 2024-09-17T04:33:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.239+0000 7fdfaaa00640 1 -- 172.21.15.100:0/2536449916 learned_addr learned my addr 172.21.15.100:0/2536449916 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.239+0000 7fdfa2000640 1 --2- 172.21.15.100:0/2536449916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfac0b7c90 0x7fdfac1622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:41.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.239+0000 7fdfab400640 1 --2- 172.21.15.100:0/2536449916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfac0d03b0 0x7fdfac0b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:41.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.240+0000 7fdfaaa00640 1 -- 172.21.15.100:0/2536449916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfac0d03b0 msgr2=0x7fdfac0b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:41.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.240+0000 7fdfaaa00640 1 --2- 172.21.15.100:0/2536449916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfac0d03b0 0x7fdfac0b0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.240+0000 7fdfaaa00640 1 -- 172.21.15.100:0/2536449916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfac0b7c90 msgr2=0x7fdfac1622e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:41.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.240+0000 7fdfaaa00640 1 --2- 172.21.15.100:0/2536449916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfac0b7c90 0x7fdfac1622e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.240+0000 7fdfaaa00640 1 -- 172.21.15.100:0/2536449916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfac1f9650 con 0x7fdfac06e5b0 2024-09-17T04:33:41.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.240+0000 7fdfaaa00640 1 --2- 172.21.15.100:0/2536449916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfac06e5b0 0x7fdfac15bab0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c00c3a0 tx=0x7fdf9c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:41.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.240+0000 7fdfa3400640 1 -- 172.21.15.100:0/2536449916 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c017070 con 0x7fdfac06e5b0 2024-09-17T04:33:41.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.240+0000 7fdfa3400640 1 -- 172.21.15.100:0/2536449916 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf9c0053c0 con 0x7fdfac06e5b0 2024-09-17T04:33:41.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.240+0000 7fdfb1a00640 1 -- 172.21.15.100:0/2536449916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfac1f9ac0 con 0x7fdfac06e5b0 2024-09-17T04:33:41.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.240+0000 7fdfa3400640 1 -- 172.21.15.100:0/2536449916 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c003850 con 0x7fdfac06e5b0 2024-09-17T04:33:41.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.240+0000 7fdfb1a00640 1 -- 172.21.15.100:0/2536449916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfac1fa080 con 0x7fdfac06e5b0 2024-09-17T04:33:41.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.241+0000 7fdfb1a00640 1 -- 172.21.15.100:0/2536449916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfac0d0930 con 0x7fdfac06e5b0 2024-09-17T04:33:41.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.243+0000 7fdfa3400640 1 -- 172.21.15.100:0/2536449916 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7fdf9c01a070 con 0x7fdfac06e5b0 2024-09-17T04:33:41.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.244+0000 7fdfa3400640 1 --2- 172.21.15.100:0/2536449916 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdf880795e0 0x7fdf8807baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:41.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.244+0000 7fdfa3400640 1 -- 172.21.15.100:0/2536449916 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 5851+0+0 (secure 0 0 0) 0x7fdf9c098ad0 con 0x7fdfac06e5b0 2024-09-17T04:33:41.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.245+0000 7fdfa2000640 1 --2- 172.21.15.100:0/2536449916 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdf880795e0 0x7fdf8807baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:41.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.245+0000 7fdfa2000640 1 --2- 172.21.15.100:0/2536449916 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdf880795e0 0x7fdf8807baf0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdfac06f2d0 tx=0x7fdf94005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:41.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.249+0000 7fdfa3400640 1 -- 172.21.15.100:0/2536449916 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fdf9c062140 con 0x7fdfac06e5b0 2024-09-17T04:33:41.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.430+0000 7fdfb1a00640 1 -- 172.21.15.100:0/2536449916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdfac06f5e0 con 0x7fdfac06e5b0 2024-09-17T04:33:41.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.990+0000 7fdfa3400640 1 -- 172.21.15.100:0/2536449916 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v158) ==== 181+0+0 (secure 0 0 0) 0x7fdf9c066b70 con 0x7fdfac06e5b0 2024-09-17T04:33:41.991 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' removed 2024-09-17T04:33:41.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.993+0000 7fdfb1a00640 1 -- 172.21.15.100:0/2536449916 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdf880795e0 msgr2=0x7fdf8807baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:41.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.993+0000 7fdfb1a00640 1 --2- 172.21.15.100:0/2536449916 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdf880795e0 0x7fdf8807baf0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdfac06f2d0 tx=0x7fdf94005f90 comp rx=0 tx=0).stop 2024-09-17T04:33:41.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.993+0000 7fdfb1a00640 1 -- 172.21.15.100:0/2536449916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfac06e5b0 msgr2=0x7fdfac15bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:41.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.993+0000 7fdfb1a00640 1 --2- 172.21.15.100:0/2536449916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfac06e5b0 0x7fdfac15bab0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c00c3a0 tx=0x7fdf9c00c860 comp rx=0 tx=0).stop 2024-09-17T04:33:41.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.993+0000 7fdfb1a00640 1 -- 172.21.15.100:0/2536449916 shutdown_connections 2024-09-17T04:33:41.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.993+0000 7fdfb1a00640 1 --2- 172.21.15.100:0/2536449916 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdf880795e0 0x7fdf8807baf0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.993+0000 7fdfb1a00640 1 --2- 172.21.15.100:0/2536449916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfac0d03b0 0x7fdfac0b0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.993+0000 7fdfb1a00640 1 --2- 172.21.15.100:0/2536449916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfac0b7c90 0x7fdfac1622e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.993+0000 7fdfb1a00640 1 --2- 172.21.15.100:0/2536449916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfac06e5b0 0x7fdfac15bab0 secure :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c00c3a0 tx=0x7fdf9c00c860 comp rx=0 tx=0).stop 2024-09-17T04:33:41.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.993+0000 7fdfb1a00640 1 -- 172.21.15.100:0/2536449916 >> 172.21.15.100:0/2536449916 conn(0x7fdfac0cef30 msgr2=0x7fdfac0cf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:41.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.994+0000 7fdfb1a00640 1 -- 172.21.15.100:0/2536449916 shutdown_connections 2024-09-17T04:33:41.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:41.994+0000 7fdfb1a00640 1 -- 172.21.15.100:0/2536449916 wait complete. 2024-09-17T04:33:42.021 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T04:33:42.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.246+0000 7fbc9b200640 1 -- 172.21.15.100:0/537295946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc78003a50 msgr2=0x7fbc78003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:42.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.246+0000 7fbc9b200640 1 --2- 172.21.15.100:0/537295946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc78003a50 0x7fbc78003ec0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009b50 tx=0x7fbc78005830 comp rx=0 tx=0).stop 2024-09-17T04:33:42.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.246+0000 7fbc9b200640 1 -- 172.21.15.100:0/537295946 shutdown_connections 2024-09-17T04:33:42.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.246+0000 7fbc9b200640 1 --2- 172.21.15.100:0/537295946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc78003a50 0x7fbc78003ec0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:42.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.246+0000 7fbc9b200640 1 -- 172.21.15.100:0/537295946 >> 172.21.15.100:0/537295946 conn(0x7fbc940b2540 msgr2=0x7fbc940d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:42.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.247+0000 7fbc9b200640 1 -- 172.21.15.100:0/537295946 shutdown_connections 2024-09-17T04:33:42.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.247+0000 7fbc9b200640 1 -- 172.21.15.100:0/537295946 wait complete. 2024-09-17T04:33:42.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.249+0000 7fbc9b200640 1 Processor -- start 2024-09-17T04:33:42.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.249+0000 7fbc9b200640 1 -- start start 2024-09-17T04:33:42.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.250+0000 7fbc9b200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc78003a50 0x7fbc941f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:42.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.250+0000 7fbc9b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc9409de90 0x7fbc94158180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:42.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.250+0000 7fbc9b200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc941f6870 0x7fbc941f6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:42.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.250+0000 7fbc9b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9406d1b0 con 0x7fbc9409de90 2024-09-17T04:33:42.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.250+0000 7fbc9b200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbc940b49f0 con 0x7fbc941f6870 2024-09-17T04:33:42.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.250+0000 7fbc9b200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbc9406d330 con 0x7fbc78003a50 2024-09-17T04:33:42.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.250+0000 7fbc93400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc9409de90 0x7fbc94158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:42.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.251+0000 7fbc93400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc9409de90 0x7fbc94158180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55756/0 (socket says 172.21.15.100:55756) 2024-09-17T04:33:42.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.251+0000 7fbc93400640 1 -- 172.21.15.100:0/2930105566 learned_addr learned my addr 172.21.15.100:0/2930105566 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:42.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.251+0000 7fbc93e00640 1 --2- 172.21.15.100:0/2930105566 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc78003a50 0x7fbc941f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:42.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.251+0000 7fbc99200640 1 --2- 172.21.15.100:0/2930105566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc941f6870 0x7fbc941f6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:42.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.251+0000 7fbc93400640 1 -- 172.21.15.100:0/2930105566 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc78003a50 msgr2=0x7fbc941f56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:42.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.251+0000 7fbc93400640 1 --2- 172.21.15.100:0/2930105566 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc78003a50 0x7fbc941f56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:42.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.251+0000 7fbc93400640 1 -- 172.21.15.100:0/2930105566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc941f6870 msgr2=0x7fbc941f6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:42.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.251+0000 7fbc93400640 1 --2- 172.21.15.100:0/2930105566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc941f6870 0x7fbc941f6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:42.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.251+0000 7fbc93400640 1 -- 172.21.15.100:0/2930105566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc941f71c0 con 0x7fbc9409de90 2024-09-17T04:33:42.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.251+0000 7fbc99200640 1 --2- 172.21.15.100:0/2930105566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc941f6870 0x7fbc941f6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:42.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.251+0000 7fbc93400640 1 --2- 172.21.15.100:0/2930105566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc9409de90 0x7fbc94158180 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fbc80006de0 tx=0x7fbc800049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:42.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.251+0000 7fbc8be00640 1 -- 172.21.15.100:0/2930105566 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc80015020 con 0x7fbc9409de90 2024-09-17T04:33:42.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.252+0000 7fbc8be00640 1 -- 172.21.15.100:0/2930105566 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc80011980 con 0x7fbc9409de90 2024-09-17T04:33:42.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.252+0000 7fbc9b200640 1 -- 172.21.15.100:0/2930105566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc9409cdf0 con 0x7fbc9409de90 2024-09-17T04:33:42.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.252+0000 7fbc8be00640 1 -- 172.21.15.100:0/2930105566 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc80011dd0 con 0x7fbc9409de90 2024-09-17T04:33:42.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.252+0000 7fbc9b200640 1 -- 172.21.15.100:0/2930105566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc9409d3b0 con 0x7fbc9409de90 2024-09-17T04:33:42.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.253+0000 7fbc9b200640 1 -- 172.21.15.100:0/2930105566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc94158e70 con 0x7fbc9409de90 2024-09-17T04:33:42.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.254+0000 7fbc8be00640 1 -- 172.21.15.100:0/2930105566 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7fbc80018020 con 0x7fbc9409de90 2024-09-17T04:33:42.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.255+0000 7fbc8be00640 1 --2- 172.21.15.100:0/2930105566 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbc740795b0 0x7fbc7407bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:42.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.255+0000 7fbc8be00640 1 -- 172.21.15.100:0/2930105566 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5413+0+0 (secure 0 0 0) 0x7fbc8009a280 con 0x7fbc9409de90 2024-09-17T04:33:42.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.255+0000 7fbc93e00640 1 --2- 172.21.15.100:0/2930105566 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbc740795b0 0x7fbc7407bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:42.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.256+0000 7fbc93e00640 1 --2- 172.21.15.100:0/2930105566 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbc740795b0 0x7fbc7407bac0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbc84035000 tx=0x7fbc840046c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.260+0000 7fbc8be00640 1 -- 172.21.15.100:0/2930105566 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fbc8001f020 con 0x7fbc9409de90 2024-09-17T04:33:42.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.448+0000 7fbc9b200640 1 -- 172.21.15.100:0/2930105566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbc9409cb60 con 0x7fbc9409de90 2024-09-17T04:33:42.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.993+0000 7fbc8be00640 1 -- 172.21.15.100:0/2930105566 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v159) ==== 169+0+0 (secure 0 0 0) 0x7fbc9415efd0 con 0x7fbc9409de90 2024-09-17T04:33:42.994 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' removed 2024-09-17T04:33:42.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.997+0000 7fbc9b200640 1 -- 172.21.15.100:0/2930105566 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbc740795b0 msgr2=0x7fbc7407bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:42.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.997+0000 7fbc9b200640 1 --2- 172.21.15.100:0/2930105566 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbc740795b0 0x7fbc7407bac0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbc84035000 tx=0x7fbc840046c0 comp rx=0 tx=0).stop 2024-09-17T04:33:42.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.997+0000 7fbc9b200640 1 -- 172.21.15.100:0/2930105566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc9409de90 msgr2=0x7fbc94158180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:42.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.997+0000 7fbc9b200640 1 --2- 172.21.15.100:0/2930105566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc9409de90 0x7fbc94158180 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fbc80006de0 tx=0x7fbc800049d0 comp rx=0 tx=0).stop 2024-09-17T04:33:42.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.997+0000 7fbc9b200640 1 -- 172.21.15.100:0/2930105566 shutdown_connections 2024-09-17T04:33:42.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.997+0000 7fbc9b200640 1 --2- 172.21.15.100:0/2930105566 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbc740795b0 0x7fbc7407bac0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:42.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.997+0000 7fbc9b200640 1 --2- 172.21.15.100:0/2930105566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc941f6870 0x7fbc941f6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:42.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.997+0000 7fbc9b200640 1 --2- 172.21.15.100:0/2930105566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc9409de90 0x7fbc94158180 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:42.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.997+0000 7fbc9b200640 1 --2- 172.21.15.100:0/2930105566 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc78003a50 0x7fbc941f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:42.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.997+0000 7fbc9b200640 1 -- 172.21.15.100:0/2930105566 >> 172.21.15.100:0/2930105566 conn(0x7fbc940b2540 msgr2=0x7fbc9415d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:42.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.997+0000 7fbc9b200640 1 -- 172.21.15.100:0/2930105566 shutdown_connections 2024-09-17T04:33:42.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:42.997+0000 7fbc9b200640 1 -- 172.21.15.100:0/2930105566 wait complete. 2024-09-17T04:33:43.022 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T04:33:43.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.246+0000 7f0fb7a00640 1 -- 172.21.15.100:0/994544831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fb015ce30 msgr2=0x7f0fb015d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:43.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.246+0000 7f0fb7a00640 1 --2- 172.21.15.100:0/994544831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fb015ce30 0x7f0fb015d240 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8005fd0 tx=0x7f0fa802aa70 comp rx=0 tx=0).stop 2024-09-17T04:33:43.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.249+0000 7f0fb7a00640 1 -- 172.21.15.100:0/994544831 shutdown_connections 2024-09-17T04:33:43.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.249+0000 7f0fb7a00640 1 --2- 172.21.15.100:0/994544831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fb015ce30 0x7f0fb015d240 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.249+0000 7f0fb7a00640 1 --2- 172.21.15.100:0/994544831 >> v2:172.21.15.184:3300/0 conn(0x7f0fb0070cc0 0x7f0fb0156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.249+0000 7f0fb7a00640 1 -- 172.21.15.100:0/994544831 >> 172.21.15.100:0/994544831 conn(0x7f0fb006d3d0 msgr2=0x7f0fb00b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:43.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.249+0000 7f0fb7a00640 1 -- 172.21.15.100:0/994544831 shutdown_connections 2024-09-17T04:33:43.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.250+0000 7f0fb7a00640 1 -- 172.21.15.100:0/994544831 wait complete. 2024-09-17T04:33:43.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.254+0000 7f0fb7a00640 1 Processor -- start 2024-09-17T04:33:43.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.255+0000 7f0fb7a00640 1 -- start start 2024-09-17T04:33:43.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.257+0000 7f0fb7a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fb00cfc80 0x7f0fb01f9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:43.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.257+0000 7f0fb7a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fb015ce30 0x7f0fb009d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:43.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.257+0000 7f0fb7a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fb009dbd0 0x7f0fb0073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:43.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.257+0000 7f0fb7a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb00cd050 con 0x7f0fb009dbd0 2024-09-17T04:33:43.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.257+0000 7f0fb7a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb00cced0 con 0x7f0fb015ce30 2024-09-17T04:33:43.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.257+0000 7f0fb7a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0fb00cd1d0 con 0x7f0fb00cfc80 2024-09-17T04:33:43.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.258+0000 7f0fb5a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fb009dbd0 0x7f0fb0073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:43.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.258+0000 7f0fb5000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fb00cfc80 0x7f0fb01f9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:43.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.258+0000 7f0fb5a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fb009dbd0 0x7f0fb0073cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55780/0 (socket says 172.21.15.100:55780) 2024-09-17T04:33:43.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.258+0000 7f0fafe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fb015ce30 0x7f0fb009d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:43.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.258+0000 7f0fb5a00640 1 -- 172.21.15.100:0/4196074024 learned_addr learned my addr 172.21.15.100:0/4196074024 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:43.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.258+0000 7f0fafe00640 1 -- 172.21.15.100:0/4196074024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fb00cfc80 msgr2=0x7f0fb01f9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:43.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.258+0000 7f0fafe00640 1 --2- 172.21.15.100:0/4196074024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fb00cfc80 0x7f0fb01f9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.258+0000 7f0fafe00640 1 -- 172.21.15.100:0/4196074024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fb009dbd0 msgr2=0x7f0fb0073cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:43.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.258+0000 7f0fafe00640 1 --2- 172.21.15.100:0/4196074024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fb009dbd0 0x7f0fb0073cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.258+0000 7f0fafe00640 1 -- 172.21.15.100:0/4196074024 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb00b0b90 con 0x7f0fb015ce30 2024-09-17T04:33:43.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.258+0000 7f0fb5a00640 1 --2- 172.21.15.100:0/4196074024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fb009dbd0 0x7f0fb0073cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:33:43.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.259+0000 7f0fafe00640 1 --2- 172.21.15.100:0/4196074024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fb015ce30 0x7f0fb009d690 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00e750 tx=0x7f0f9c00ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:43.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.259+0000 7f0facc00640 1 -- 172.21.15.100:0/4196074024 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c00ccc0 con 0x7f0fb015ce30 2024-09-17T04:33:43.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.259+0000 7f0facc00640 1 -- 172.21.15.100:0/4196074024 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f9c00ee60 con 0x7f0fb015ce30 2024-09-17T04:33:43.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.259+0000 7f0facc00640 1 -- 172.21.15.100:0/4196074024 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c004260 con 0x7f0fb015ce30 2024-09-17T04:33:43.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.259+0000 7f0fb7a00640 1 -- 172.21.15.100:0/4196074024 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fb01f9f70 con 0x7f0fb015ce30 2024-09-17T04:33:43.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.259+0000 7f0fb7a00640 1 -- 172.21.15.100:0/4196074024 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fb00b10e0 con 0x7f0fb015ce30 2024-09-17T04:33:43.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.261+0000 7f0fb7a00640 1 -- 172.21.15.100:0/4196074024 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fb00cd6f0 con 0x7f0fb015ce30 2024-09-17T04:33:43.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.262+0000 7f0facc00640 1 -- 172.21.15.100:0/4196074024 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f0f9c0035f0 con 0x7f0fb015ce30 2024-09-17T04:33:43.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.263+0000 7f0facc00640 1 --2- 172.21.15.100:0/4196074024 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0f900795b0 0x7f0f9007bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:43.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.263+0000 7f0facc00640 1 -- 172.21.15.100:0/4196074024 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4999+0+0 (secure 0 0 0) 0x7f0f9c099df0 con 0x7f0fb015ce30 2024-09-17T04:33:43.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.264+0000 7f0fb5000640 1 --2- 172.21.15.100:0/4196074024 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0f900795b0 0x7f0f9007bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:43.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.264+0000 7f0fb5000640 1 --2- 172.21.15.100:0/4196074024 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0f900795b0 0x7f0f9007bac0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0fb009e510 tx=0x7f0fa4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:43.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.269+0000 7f0facc00640 1 -- 172.21.15.100:0/4196074024 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0f9c0634a0 con 0x7f0fb015ce30 2024-09-17T04:33:43.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.453+0000 7f0fb7a00640 1 -- 172.21.15.100:0/4196074024 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f0fb00753d0 con 0x7f0fb015ce30 2024-09-17T04:33:43.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.461+0000 7f0facc00640 1 -- 172.21.15.100:0/4196074024 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v51) ==== 71+0+0 (secure 0 0 0) 0x7f0f9c067ed0 con 0x7f0fb015ce30 2024-09-17T04:33:43.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.464+0000 7f0fb7a00640 1 -- 172.21.15.100:0/4196074024 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0f900795b0 msgr2=0x7f0f9007bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:43.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.464+0000 7f0fb7a00640 1 --2- 172.21.15.100:0/4196074024 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0f900795b0 0x7f0f9007bac0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0fb009e510 tx=0x7f0fa4009040 comp rx=0 tx=0).stop 2024-09-17T04:33:43.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.464+0000 7f0fb7a00640 1 -- 172.21.15.100:0/4196074024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fb015ce30 msgr2=0x7f0fb009d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:43.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.464+0000 7f0fb7a00640 1 --2- 172.21.15.100:0/4196074024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fb015ce30 0x7f0fb009d690 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00e750 tx=0x7f0f9c00ec10 comp rx=0 tx=0).stop 2024-09-17T04:33:43.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.464+0000 7f0fb7a00640 1 -- 172.21.15.100:0/4196074024 shutdown_connections 2024-09-17T04:33:43.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.464+0000 7f0fb7a00640 1 --2- 172.21.15.100:0/4196074024 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f0f900795b0 0x7f0f9007bac0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.464+0000 7f0fb7a00640 1 --2- 172.21.15.100:0/4196074024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fb009dbd0 0x7f0fb0073cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.464+0000 7f0fb7a00640 1 --2- 172.21.15.100:0/4196074024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fb015ce30 0x7f0fb009d690 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.464+0000 7f0fb7a00640 1 --2- 172.21.15.100:0/4196074024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fb00cfc80 0x7f0fb01f9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.464+0000 7f0fb7a00640 1 -- 172.21.15.100:0/4196074024 >> 172.21.15.100:0/4196074024 conn(0x7f0fb006d3d0 msgr2=0x7f0fb006f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:43.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.464+0000 7f0fb7a00640 1 -- 172.21.15.100:0/4196074024 shutdown_connections 2024-09-17T04:33:43.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.464+0000 7f0fb7a00640 1 -- 172.21.15.100:0/4196074024 wait complete. 2024-09-17T04:33:43.486 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2' 2024-09-17T04:33:43.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.704+0000 7f1c17000640 1 -- 172.21.15.100:0/2026971532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c1015a0f0 msgr2=0x7f1c1015d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:43.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.704+0000 7f1c17000640 1 --2- 172.21.15.100:0/2026971532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c1015a0f0 0x7f1c1015d670 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f1c1006e790 tx=0x7f1c0802a9a0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.704+0000 7f1c17000640 1 -- 172.21.15.100:0/2026971532 shutdown_connections 2024-09-17T04:33:43.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.704+0000 7f1c17000640 1 --2- 172.21.15.100:0/2026971532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c1015a0f0 0x7f1c1015d670 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.704+0000 7f1c17000640 1 --2- 172.21.15.100:0/2026971532 >> v2:172.21.15.184:3300/0 conn(0x7f1c100af170 0x7f1c100b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.704+0000 7f1c17000640 1 -- 172.21.15.100:0/2026971532 >> 172.21.15.100:0/2026971532 conn(0x7f1c100b1e40 msgr2=0x7f1c100b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:43.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.704+0000 7f1c17000640 1 -- 172.21.15.100:0/2026971532 shutdown_connections 2024-09-17T04:33:43.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.705+0000 7f1c17000640 1 -- 172.21.15.100:0/2026971532 wait complete. 2024-09-17T04:33:43.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.707+0000 7f1c17000640 1 Processor -- start 2024-09-17T04:33:43.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.707+0000 7f1c17000640 1 -- start start 2024-09-17T04:33:43.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.708+0000 7f1c17000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c100ad9c0 0x7f1c101f5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:43.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.708+0000 7f1c17000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c100af170 0x7f1c10153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:43.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.708+0000 7f1c17000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c1015a0f0 0x7f1c10158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:43.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.708+0000 7f1c17000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1006f290 con 0x7f1c1015a0f0 2024-09-17T04:33:43.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.708+0000 7f1c17000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1c10070c80 con 0x7f1c100ad9c0 2024-09-17T04:33:43.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.708+0000 7f1c17000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1c1006f410 con 0x7f1c100af170 2024-09-17T04:33:43.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.708+0000 7f1c16600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c1015a0f0 0x7f1c10158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:43.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.709+0000 7f1c16600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c1015a0f0 0x7f1c10158a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55808/0 (socket says 172.21.15.100:55808) 2024-09-17T04:33:43.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.709+0000 7f1c16600640 1 -- 172.21.15.100:0/2588494243 learned_addr learned my addr 172.21.15.100:0/2588494243 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:43.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.709+0000 7f1c15c00640 1 --2- 172.21.15.100:0/2588494243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c100ad9c0 0x7f1c101f5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:43.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.709+0000 7f1c15200640 1 --2- 172.21.15.100:0/2588494243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c100af170 0x7f1c10153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:43.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.709+0000 7f1c16600640 1 -- 172.21.15.100:0/2588494243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c100af170 msgr2=0x7f1c10153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:43.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.709+0000 7f1c16600640 1 --2- 172.21.15.100:0/2588494243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c100af170 0x7f1c10153ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.709+0000 7f1c16600640 1 -- 172.21.15.100:0/2588494243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c100ad9c0 msgr2=0x7f1c101f5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:43.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.709+0000 7f1c16600640 1 --2- 172.21.15.100:0/2588494243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c100ad9c0 0x7f1c101f5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.709+0000 7f1c16600640 1 -- 172.21.15.100:0/2588494243 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c10156730 con 0x7f1c1015a0f0 2024-09-17T04:33:43.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.709+0000 7f1c15c00640 1 --2- 172.21.15.100:0/2588494243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c100ad9c0 0x7f1c101f5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:43.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.709+0000 7f1c16600640 1 --2- 172.21.15.100:0/2588494243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c1015a0f0 0x7f1c10158a70 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009d60 tx=0x7f1c08003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:43.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.710+0000 7f1c0d600640 1 -- 172.21.15.100:0/2588494243 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c08033050 con 0x7f1c1015a0f0 2024-09-17T04:33:43.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.710+0000 7f1c0d600640 1 -- 172.21.15.100:0/2588494243 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c08004450 con 0x7f1c1015a0f0 2024-09-17T04:33:43.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.710+0000 7f1c17000640 1 -- 172.21.15.100:0/2588494243 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c10156a10 con 0x7f1c1015a0f0 2024-09-17T04:33:43.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.710+0000 7f1c0d600640 1 -- 172.21.15.100:0/2588494243 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c08038450 con 0x7f1c1015a0f0 2024-09-17T04:33:43.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.710+0000 7f1c17000640 1 -- 172.21.15.100:0/2588494243 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c10154430 con 0x7f1c1015a0f0 2024-09-17T04:33:43.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.711+0000 7f1c17000640 1 -- 172.21.15.100:0/2588494243 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c10156c30 con 0x7f1c1015a0f0 2024-09-17T04:33:43.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.712+0000 7f1c0d600640 1 -- 172.21.15.100:0/2588494243 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f1c08036070 con 0x7f1c1015a0f0 2024-09-17T04:33:43.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.713+0000 7f1c0d600640 1 --2- 172.21.15.100:0/2588494243 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1bf00795e0 0x7f1bf007baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:43.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.713+0000 7f1c0d600640 1 -- 172.21.15.100:0/2588494243 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4999+0+0 (secure 0 0 0) 0x7f1c080ba970 con 0x7f1c1015a0f0 2024-09-17T04:33:43.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.713+0000 7f1c15c00640 1 --2- 172.21.15.100:0/2588494243 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1bf00795e0 0x7f1bf007baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:43.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.714+0000 7f1c15c00640 1 --2- 172.21.15.100:0/2588494243 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1bf00795e0 0x7f1bf007baf0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1c04007fd0 tx=0x7f1c04009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:43.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.717+0000 7f1c0d600640 1 -- 172.21.15.100:0/2588494243 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f1c10156c30 con 0x7f1c1015a0f0 2024-09-17T04:33:43.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.968+0000 7f1c17000640 1 -- 172.21.15.100:0/2588494243 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]} v 0) -- 0x7f1c1006f930 con 0x7f1c1015a0f0 2024-09-17T04:33:43.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.995+0000 7f1c0d600640 1 -- 172.21.15.100:0/2588494243 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]}]=0 v536) ==== 145+0+0 (secure 0 0 0) 0x7f1c08084800 con 0x7f1c1015a0f0 2024-09-17T04:33:43.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.998+0000 7f1c17000640 1 -- 172.21.15.100:0/2588494243 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1bf00795e0 msgr2=0x7f1bf007baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:43.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.998+0000 7f1c17000640 1 --2- 172.21.15.100:0/2588494243 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1bf00795e0 0x7f1bf007baf0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1c04007fd0 tx=0x7f1c04009040 comp rx=0 tx=0).stop 2024-09-17T04:33:43.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.998+0000 7f1c17000640 1 -- 172.21.15.100:0/2588494243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c1015a0f0 msgr2=0x7f1c10158a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:43.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.998+0000 7f1c17000640 1 --2- 172.21.15.100:0/2588494243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c1015a0f0 0x7f1c10158a70 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009d60 tx=0x7f1c08003f10 comp rx=0 tx=0).stop 2024-09-17T04:33:43.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.998+0000 7f1c17000640 1 -- 172.21.15.100:0/2588494243 shutdown_connections 2024-09-17T04:33:43.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.998+0000 7f1c17000640 1 --2- 172.21.15.100:0/2588494243 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1bf00795e0 0x7f1bf007baf0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.998+0000 7f1c17000640 1 --2- 172.21.15.100:0/2588494243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c1015a0f0 0x7f1c10158a70 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.998+0000 7f1c17000640 1 --2- 172.21.15.100:0/2588494243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c100af170 0x7f1c10153ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.998+0000 7f1c17000640 1 --2- 172.21.15.100:0/2588494243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c100ad9c0 0x7f1c101f5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.998+0000 7f1c17000640 1 -- 172.21.15.100:0/2588494243 >> 172.21.15.100:0/2588494243 conn(0x7f1c100b1e40 msgr2=0x7f1c100b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:43.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.998+0000 7f1c17000640 1 -- 172.21.15.100:0/2588494243 shutdown_connections 2024-09-17T04:33:43.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:43.998+0000 7f1c17000640 1 -- 172.21.15.100:0/2588494243 wait complete. 2024-09-17T04:33:44.023 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-09-17T04:33:44.023 INFO:tasks.cephfs_test_runner:Starting test: test_unparseable_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate unparseable text .meta file on legacy subvol root doesn't break the system 2024-09-17T04:33:44.024 INFO:tasks.cephfs_test_runner:test_unparseable_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-09-17T04:33:44.024 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:33:44.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.243+0000 7fd625e00640 1 -- 172.21.15.100:0/1517892927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd604003a00 msgr2=0x7fd604003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:44.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.243+0000 7fd625e00640 1 --2- 172.21.15.100:0/1517892927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd604003a00 0x7fd604003e70 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fd61000c4a0 tx=0x7fd604005830 comp rx=0 tx=0).stop 2024-09-17T04:33:44.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.243+0000 7fd61f400640 1 -- 172.21.15.100:0/1517892927 reap_dead start 2024-09-17T04:33:44.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.243+0000 7fd625e00640 1 -- 172.21.15.100:0/1517892927 shutdown_connections 2024-09-17T04:33:44.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.243+0000 7fd625e00640 1 -- 172.21.15.100:0/1517892927 >> 172.21.15.100:0/1517892927 conn(0x7fd62006d730 msgr2=0x7fd6200b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:44.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.243+0000 7fd625e00640 1 -- 172.21.15.100:0/1517892927 shutdown_connections 2024-09-17T04:33:44.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.243+0000 7fd625e00640 1 -- 172.21.15.100:0/1517892927 wait complete. 2024-09-17T04:33:44.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.245+0000 7fd625e00640 1 Processor -- start 2024-09-17T04:33:44.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.245+0000 7fd625e00640 1 -- start start 2024-09-17T04:33:44.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.247+0000 7fd625e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd604003a00 0x7fd620147870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:44.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.247+0000 7fd625e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6201f53b0 0x7fd620158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:44.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.247+0000 7fd625e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd62014f4e0 0x7fd62014f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:44.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.247+0000 7fd625e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd6200b4da0 con 0x7fd62014f4e0 2024-09-17T04:33:44.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.247+0000 7fd625e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd6200b4c20 con 0x7fd604003a00 2024-09-17T04:33:44.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.247+0000 7fd625e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd6200ad4f0 con 0x7fd6201f53b0 2024-09-17T04:33:44.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.247+0000 7fd61f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd62014f4e0 0x7fd62014f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:44.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.247+0000 7fd61f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd62014f4e0 0x7fd62014f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55834/0 (socket says 172.21.15.100:55834) 2024-09-17T04:33:44.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.247+0000 7fd61f400640 1 -- 172.21.15.100:0/1829094527 learned_addr learned my addr 172.21.15.100:0/1829094527 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:44.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.247+0000 7fd61e000640 1 --2- 172.21.15.100:0/1829094527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6201f53b0 0x7fd620158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:44.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.247+0000 7fd61ea00640 1 --2- 172.21.15.100:0/1829094527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd604003a00 0x7fd620147870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:44.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.247+0000 7fd61f400640 1 -- 172.21.15.100:0/1829094527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6201f53b0 msgr2=0x7fd620158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:44.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.247+0000 7fd61f400640 1 --2- 172.21.15.100:0/1829094527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6201f53b0 0x7fd620158bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.247+0000 7fd61f400640 1 -- 172.21.15.100:0/1829094527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd604003a00 msgr2=0x7fd620147870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:44.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.247+0000 7fd61f400640 1 --2- 172.21.15.100:0/1829094527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd604003a00 0x7fd620147870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.248+0000 7fd61f400640 1 -- 172.21.15.100:0/1829094527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd620147db0 con 0x7fd62014f4e0 2024-09-17T04:33:44.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.248+0000 7fd61e000640 1 --2- 172.21.15.100:0/1829094527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6201f53b0 0x7fd620158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:44.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.248+0000 7fd61f400640 1 --2- 172.21.15.100:0/1829094527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd62014f4e0 0x7fd62014f8f0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fd6200aea60 tx=0x7fd61800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:44.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.248+0000 7fd617400640 1 -- 172.21.15.100:0/1829094527 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd618003d80 con 0x7fd62014f4e0 2024-09-17T04:33:44.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.248+0000 7fd617400640 1 -- 172.21.15.100:0/1829094527 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd618004b20 con 0x7fd62014f4e0 2024-09-17T04:33:44.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.248+0000 7fd617400640 1 -- 172.21.15.100:0/1829094527 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd618004030 con 0x7fd62014f4e0 2024-09-17T04:33:44.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.248+0000 7fd625e00640 1 -- 172.21.15.100:0/1829094527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6201f5de0 con 0x7fd62014f4e0 2024-09-17T04:33:44.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.248+0000 7fd625e00640 1 -- 172.21.15.100:0/1829094527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6201583d0 con 0x7fd62014f4e0 2024-09-17T04:33:44.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.249+0000 7fd625e00640 1 -- 172.21.15.100:0/1829094527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6201f6300 con 0x7fd62014f4e0 2024-09-17T04:33:44.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.251+0000 7fd617400640 1 -- 172.21.15.100:0/1829094527 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7fd6180041d0 con 0x7fd62014f4e0 2024-09-17T04:33:44.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.251+0000 7fd617400640 1 --2- 172.21.15.100:0/1829094527 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd6000795b0 0x7fd60007bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:44.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.252+0000 7fd617400640 1 -- 172.21.15.100:0/1829094527 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4999+0+0 (secure 0 0 0) 0x7fd61809a3c0 con 0x7fd62014f4e0 2024-09-17T04:33:44.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.252+0000 7fd61ea00640 1 --2- 172.21.15.100:0/1829094527 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd6000795b0 0x7fd60007bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:44.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.252+0000 7fd61ea00640 1 --2- 172.21.15.100:0/1829094527 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd6000795b0 0x7fd60007bac0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd61000c4a0 tx=0x7fd610038040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:44.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.256+0000 7fd617400640 1 -- 172.21.15.100:0/1829094527 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd6201f6300 con 0x7fd62014f4e0 2024-09-17T04:33:44.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.438+0000 7fd625e00640 1 -- 172.21.15.100:0/1829094527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6201535c0 con 0x7fd62014f4e0 2024-09-17T04:33:44.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.439+0000 7fd617400640 1 -- 172.21.15.100:0/1829094527 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) ==== 74+0+13181 (secure 0 0 0) 0x7fd6180642e0 con 0x7fd62014f4e0 2024-09-17T04:33:44.440 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:44.440 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":159,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:33:42.991472+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.184:6835/2965326201":"2024-09-18T04:33:39.981873+0000","172.21.15.184:6834/2965326201":"2024-09-18T04:33:39.981873+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:33:44.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.442+0000 7fd625e00640 1 -- 172.21.15.100:0/1829094527 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd6000795b0 msgr2=0x7fd60007bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:44.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.442+0000 7fd625e00640 1 --2- 172.21.15.100:0/1829094527 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd6000795b0 0x7fd60007bac0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd61000c4a0 tx=0x7fd610038040 comp rx=0 tx=0).stop 2024-09-17T04:33:44.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.442+0000 7fd625e00640 1 -- 172.21.15.100:0/1829094527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd62014f4e0 msgr2=0x7fd62014f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:44.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.442+0000 7fd625e00640 1 --2- 172.21.15.100:0/1829094527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd62014f4e0 0x7fd62014f8f0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fd6200aea60 tx=0x7fd61800f790 comp rx=0 tx=0).stop 2024-09-17T04:33:44.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.442+0000 7fd625e00640 1 -- 172.21.15.100:0/1829094527 shutdown_connections 2024-09-17T04:33:44.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.442+0000 7fd625e00640 1 --2- 172.21.15.100:0/1829094527 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd6000795b0 0x7fd60007bac0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.442+0000 7fd625e00640 1 --2- 172.21.15.100:0/1829094527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd62014f4e0 0x7fd62014f8f0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.442+0000 7fd625e00640 1 --2- 172.21.15.100:0/1829094527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6201f53b0 0x7fd620158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.442+0000 7fd625e00640 1 --2- 172.21.15.100:0/1829094527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd604003a00 0x7fd620147870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.442+0000 7fd625e00640 1 -- 172.21.15.100:0/1829094527 >> 172.21.15.100:0/1829094527 conn(0x7fd62006d730 msgr2=0x7fd62015dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:44.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.442+0000 7fd625e00640 1 -- 172.21.15.100:0/1829094527 shutdown_connections 2024-09-17T04:33:44.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.442+0000 7fd625e00640 1 -- 172.21.15.100:0/1829094527 wait complete. 2024-09-17T04:33:44.466 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T04:24:03.995355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T04:33:44.466 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T04:33:44.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.688+0000 7f7204e00640 1 -- 172.21.15.100:0/3531528917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72000b4f00 msgr2=0x7f72000704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:44.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.688+0000 7f7204e00640 1 --2- 172.21.15.100:0/3531528917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72000b4f00 0x7f72000704a0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f7200071450 tx=0x7f71f002a5f0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.688+0000 7f7204e00640 1 -- 172.21.15.100:0/3531528917 shutdown_connections 2024-09-17T04:33:44.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.688+0000 7f7204e00640 1 --2- 172.21.15.100:0/3531528917 >> v2:172.21.15.184:3300/0 conn(0x7f7200160490 0x7f720015d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.688+0000 7f7204e00640 1 --2- 172.21.15.100:0/3531528917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72000b4f00 0x7f72000704a0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.688+0000 7f7204e00640 1 -- 172.21.15.100:0/3531528917 >> 172.21.15.100:0/3531528917 conn(0x7f720006d710 msgr2=0x7f72000b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:44.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.688+0000 7f7204e00640 1 -- 172.21.15.100:0/3531528917 shutdown_connections 2024-09-17T04:33:44.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.688+0000 7f7204e00640 1 -- 172.21.15.100:0/3531528917 wait complete. 2024-09-17T04:33:44.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.690+0000 7f7204e00640 1 Processor -- start 2024-09-17T04:33:44.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.690+0000 7f7204e00640 1 -- start start 2024-09-17T04:33:44.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.692+0000 7f7204e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72000ad3d0 0x7f72001546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:44.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.692+0000 7f7204e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72000b4f00 0x7f72000994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:44.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.692+0000 7f7204e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7200160490 0x7f72001f90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:44.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.692+0000 7f7204e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f720006f0f0 con 0x7f72000b4f00 2024-09-17T04:33:44.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.692+0000 7f7204e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f720006ef70 con 0x7f72000ad3d0 2024-09-17T04:33:44.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.692+0000 7f7204e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f720006f270 con 0x7f7200160490 2024-09-17T04:33:44.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.692+0000 7f71f5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72000b4f00 0x7f72000994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:44.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.692+0000 7f71f5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72000b4f00 0x7f72000994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55838/0 (socket says 172.21.15.100:55838) 2024-09-17T04:33:44.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.692+0000 7f71f5400640 1 -- 172.21.15.100:0/270624126 learned_addr learned my addr 172.21.15.100:0/270624126 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:44.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.692+0000 7f71fde00640 1 --2- 172.21.15.100:0/270624126 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72000ad3d0 0x7f72001546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:44.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.692+0000 7f71fe800640 1 --2- 172.21.15.100:0/270624126 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7200160490 0x7f72001f90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:44.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.692+0000 7f71f5400640 1 -- 172.21.15.100:0/270624126 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7200160490 msgr2=0x7f72001f90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:44.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.692+0000 7f71f5400640 1 --2- 172.21.15.100:0/270624126 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7200160490 0x7f72001f90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.693+0000 7f71f5400640 1 -- 172.21.15.100:0/270624126 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72000ad3d0 msgr2=0x7f72001546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:44.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.693+0000 7f71f5400640 1 --2- 172.21.15.100:0/270624126 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72000ad3d0 0x7f72001546f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.693+0000 7f71f5400640 1 -- 172.21.15.100:0/270624126 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7200155020 con 0x7f72000b4f00 2024-09-17T04:33:44.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.693+0000 7f71f5400640 1 --2- 172.21.15.100:0/270624126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72000b4f00 0x7f72000994c0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f71e800d770 tx=0x7f71e800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:44.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.693+0000 7f71f6a00640 1 -- 172.21.15.100:0/270624126 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e8017070 con 0x7f72000b4f00 2024-09-17T04:33:44.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.693+0000 7f71f6a00640 1 -- 172.21.15.100:0/270624126 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71e8009d20 con 0x7f72000b4f00 2024-09-17T04:33:44.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.693+0000 7f71f6a00640 1 -- 172.21.15.100:0/270624126 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e80106c0 con 0x7f72000b4f00 2024-09-17T04:33:44.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.693+0000 7f7204e00640 1 -- 172.21.15.100:0/270624126 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7200155300 con 0x7f72000b4f00 2024-09-17T04:33:44.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.693+0000 7f7204e00640 1 -- 172.21.15.100:0/270624126 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72001558c0 con 0x7f72000b4f00 2024-09-17T04:33:44.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.694+0000 7f7204e00640 1 -- 172.21.15.100:0/270624126 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f720006f720 con 0x7f72000b4f00 2024-09-17T04:33:44.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.696+0000 7f71f6a00640 1 -- 172.21.15.100:0/270624126 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f71e801a070 con 0x7f72000b4f00 2024-09-17T04:33:44.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.696+0000 7f71f6a00640 1 --2- 172.21.15.100:0/270624126 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f71dc0795b0 0x7f71dc07bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:44.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.697+0000 7f71f6a00640 1 -- 172.21.15.100:0/270624126 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4999+0+0 (secure 0 0 0) 0x7f71e8099900 con 0x7f72000b4f00 2024-09-17T04:33:44.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.697+0000 7f71fde00640 1 --2- 172.21.15.100:0/270624126 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f71dc0795b0 0x7f71dc07bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:44.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.697+0000 7f71fde00640 1 --2- 172.21.15.100:0/270624126 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f71dc0795b0 0x7f71dc07bac0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f71f0033000 tx=0x7f71f0004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:44.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.701+0000 7f71f6a00640 1 -- 172.21.15.100:0/270624126 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f71e8063010 con 0x7f72000b4f00 2024-09-17T04:33:44.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.882+0000 7f7204e00640 1 -- 172.21.15.100:0/270624126 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7200154db0 con 0x7f72000b4f00 2024-09-17T04:33:44.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.883+0000 7f71f6a00640 1 -- 172.21.15.100:0/270624126 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v159) ==== 93+0+10 (secure 0 0 0) 0x7f71e8067a40 con 0x7f72000b4f00 2024-09-17T04:33:44.883 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-09-17T04:33:44.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.885+0000 7f7204e00640 1 -- 172.21.15.100:0/270624126 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f71dc0795b0 msgr2=0x7f71dc07bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:44.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.885+0000 7f7204e00640 1 --2- 172.21.15.100:0/270624126 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f71dc0795b0 0x7f71dc07bac0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f71f0033000 tx=0x7f71f0004730 comp rx=0 tx=0).stop 2024-09-17T04:33:44.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.885+0000 7f7204e00640 1 -- 172.21.15.100:0/270624126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72000b4f00 msgr2=0x7f72000994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:44.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.885+0000 7f7204e00640 1 --2- 172.21.15.100:0/270624126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72000b4f00 0x7f72000994c0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f71e800d770 tx=0x7f71e800dc30 comp rx=0 tx=0).stop 2024-09-17T04:33:44.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.885+0000 7f7204e00640 1 -- 172.21.15.100:0/270624126 shutdown_connections 2024-09-17T04:33:44.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.885+0000 7f7204e00640 1 --2- 172.21.15.100:0/270624126 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f71dc0795b0 0x7f71dc07bac0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.885+0000 7f7204e00640 1 --2- 172.21.15.100:0/270624126 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7200160490 0x7f72001f90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.885+0000 7f7204e00640 1 --2- 172.21.15.100:0/270624126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72000b4f00 0x7f72000994c0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.885+0000 7f7204e00640 1 --2- 172.21.15.100:0/270624126 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72000ad3d0 0x7f72001546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.885+0000 7f7204e00640 1 -- 172.21.15.100:0/270624126 >> 172.21.15.100:0/270624126 conn(0x7f720006d710 msgr2=0x7f72000b1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:44.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.885+0000 7f7204e00640 1 -- 172.21.15.100:0/270624126 shutdown_connections 2024-09-17T04:33:44.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:44.885+0000 7f7204e00640 1 -- 172.21.15.100:0/270624126 wait complete. 2024-09-17T04:33:44.909 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade' 2024-09-17T04:33:45.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.132+0000 7f6f14e00640 1 -- 172.21.15.100:0/2055500307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f1015d670 msgr2=0x7f6f10070530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:45.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.132+0000 7f6f14e00640 1 --2- 172.21.15.100:0/2055500307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f1015d670 0x7f6f10070530 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f6f100d00a0 tx=0x7f6f0802a980 comp rx=0 tx=0).stop 2024-09-17T04:33:45.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.132+0000 7f6f14e00640 1 -- 172.21.15.100:0/2055500307 shutdown_connections 2024-09-17T04:33:45.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.132+0000 7f6f14e00640 1 --2- 172.21.15.100:0/2055500307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f1015d670 0x7f6f10070530 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:45.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.132+0000 7f6f14e00640 1 --2- 172.21.15.100:0/2055500307 >> v2:172.21.15.184:3300/0 conn(0x7f6f100b2f60 0x7f6f1015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:45.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.132+0000 7f6f14e00640 1 --2- 172.21.15.100:0/2055500307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f100ad9c0 0x7f6f100af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:45.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.132+0000 7f6f14e00640 1 -- 172.21.15.100:0/2055500307 >> 172.21.15.100:0/2055500307 conn(0x7f6f100b1e40 msgr2=0x7f6f100b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:45.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.132+0000 7f6f14e00640 1 -- 172.21.15.100:0/2055500307 shutdown_connections 2024-09-17T04:33:45.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.132+0000 7f6f14e00640 1 -- 172.21.15.100:0/2055500307 wait complete. 2024-09-17T04:33:45.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.134+0000 7f6f14e00640 1 Processor -- start 2024-09-17T04:33:45.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.135+0000 7f6f14e00640 1 -- start start 2024-09-17T04:33:45.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.136+0000 7f6f14e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f100ad9c0 0x7f6f10150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:45.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.136+0000 7f6f14e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f100b2f60 0x7f6f10072e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:45.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.136+0000 7f6f14e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f1015d670 0x7f6f10158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:45.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.136+0000 7f6f14e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1006f290 con 0x7f6f100b2f60 2024-09-17T04:33:45.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.136+0000 7f6f14e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6f10070c80 con 0x7f6f100ad9c0 2024-09-17T04:33:45.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.136+0000 7f6f14e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6f1006f410 con 0x7f6f1015d670 2024-09-17T04:33:45.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.136+0000 7f6f06a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f100b2f60 0x7f6f10072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:45.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.136+0000 7f6f06a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f100b2f60 0x7f6f10072e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39272/0 (socket says 172.21.15.100:39272) 2024-09-17T04:33:45.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.136+0000 7f6f06a00640 1 -- 172.21.15.100:0/2548357636 learned_addr learned my addr 172.21.15.100:0/2548357636 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:45.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.137+0000 7f6f0fe00640 1 --2- 172.21.15.100:0/2548357636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f1015d670 0x7f6f10158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:45.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.137+0000 7f6f0f400640 1 --2- 172.21.15.100:0/2548357636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f100ad9c0 0x7f6f10150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:45.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.137+0000 7f6f06a00640 1 -- 172.21.15.100:0/2548357636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f1015d670 msgr2=0x7f6f10158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:45.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.137+0000 7f6f06a00640 1 --2- 172.21.15.100:0/2548357636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f1015d670 0x7f6f10158670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:45.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.137+0000 7f6f06a00640 1 -- 172.21.15.100:0/2548357636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f100ad9c0 msgr2=0x7f6f10150d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:45.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.137+0000 7f6f06a00640 1 --2- 172.21.15.100:0/2548357636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f100ad9c0 0x7f6f10150d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:45.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.137+0000 7f6f06a00640 1 -- 172.21.15.100:0/2548357636 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f100c94c0 con 0x7f6f100b2f60 2024-09-17T04:33:45.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.137+0000 7f6f0f400640 1 --2- 172.21.15.100:0/2548357636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f100ad9c0 0x7f6f10150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:45.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.137+0000 7f6f06a00640 1 --2- 172.21.15.100:0/2548357636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f100b2f60 0x7f6f10072e90 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800c970 tx=0x7f6ef800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:45.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.137+0000 7f6f07e00640 1 -- 172.21.15.100:0/2548357636 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8007bb0 con 0x7f6f100b2f60 2024-09-17T04:33:45.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.137+0000 7f6f07e00640 1 -- 172.21.15.100:0/2548357636 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ef8007e40 con 0x7f6f100b2f60 2024-09-17T04:33:45.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.137+0000 7f6f07e00640 1 -- 172.21.15.100:0/2548357636 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef800f6b0 con 0x7f6f100b2f60 2024-09-17T04:33:45.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.137+0000 7f6f14e00640 1 -- 172.21.15.100:0/2548357636 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f100c97a0 con 0x7f6f100b2f60 2024-09-17T04:33:45.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.138+0000 7f6f14e00640 1 -- 172.21.15.100:0/2548357636 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f100c9d60 con 0x7f6f100b2f60 2024-09-17T04:33:45.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.139+0000 7f6f14e00640 1 -- 172.21.15.100:0/2548357636 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f1006f930 con 0x7f6f100b2f60 2024-09-17T04:33:45.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.140+0000 7f6f07e00640 1 -- 172.21.15.100:0/2548357636 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f6ef800f850 con 0x7f6f100b2f60 2024-09-17T04:33:45.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.141+0000 7f6f07e00640 1 --2- 172.21.15.100:0/2548357636 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6eec0795b0 0x7f6eec07bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:45.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.141+0000 7f6f07e00640 1 -- 172.21.15.100:0/2548357636 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4999+0+0 (secure 0 0 0) 0x7f6ef8099f90 con 0x7f6f100b2f60 2024-09-17T04:33:45.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.141+0000 7f6f0f400640 1 --2- 172.21.15.100:0/2548357636 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6eec0795b0 0x7f6eec07bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:45.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.142+0000 7f6f0f400640 1 --2- 172.21.15.100:0/2548357636 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6eec0795b0 0x7f6eec07bac0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000ba60 tx=0x7f6f00005cf0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:45.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.145+0000 7f6f07e00640 1 -- 172.21.15.100:0/2548357636 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6ef8014030 con 0x7f6f100b2f60 2024-09-17T04:33:45.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:45.396+0000 7f6f14e00640 1 -- 172.21.15.100:0/2548357636 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f6f1015b140 con 0x7f6f100b2f60 2024-09-17T04:33:46.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.004+0000 7f6f07e00640 1 -- 172.21.15.100:0/2548357636 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]}]=0 v538) ==== 166+0+0 (secure 0 0 0) 0x7f6ef8063c90 con 0x7f6f100b2f60 2024-09-17T04:33:46.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.006+0000 7f6f14e00640 1 -- 172.21.15.100:0/2548357636 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6eec0795b0 msgr2=0x7f6eec07bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:46.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.006+0000 7f6f14e00640 1 --2- 172.21.15.100:0/2548357636 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6eec0795b0 0x7f6eec07bac0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000ba60 tx=0x7f6f00005cf0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.006+0000 7f6f14e00640 1 -- 172.21.15.100:0/2548357636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f100b2f60 msgr2=0x7f6f10072e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:46.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.007+0000 7f6f14e00640 1 --2- 172.21.15.100:0/2548357636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f100b2f60 0x7f6f10072e90 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800c970 tx=0x7f6ef800ce30 comp rx=0 tx=0).stop 2024-09-17T04:33:46.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.007+0000 7f6f14e00640 1 -- 172.21.15.100:0/2548357636 shutdown_connections 2024-09-17T04:33:46.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.007+0000 7f6f14e00640 1 --2- 172.21.15.100:0/2548357636 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6eec0795b0 0x7f6eec07bac0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.007+0000 7f6f14e00640 1 --2- 172.21.15.100:0/2548357636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f1015d670 0x7f6f10158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.007+0000 7f6f14e00640 1 --2- 172.21.15.100:0/2548357636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f100b2f60 0x7f6f10072e90 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.007+0000 7f6f14e00640 1 --2- 172.21.15.100:0/2548357636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f100ad9c0 0x7f6f10150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.007+0000 7f6f14e00640 1 -- 172.21.15.100:0/2548357636 >> 172.21.15.100:0/2548357636 conn(0x7f6f100b1e40 msgr2=0x7f6f100b4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:46.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.007+0000 7f6f14e00640 1 -- 172.21.15.100:0/2548357636 shutdown_connections 2024-09-17T04:33:46.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.007+0000 7f6f14e00640 1 -- 172.21.15.100:0/2548357636 wait complete. 2024-09-17T04:33:46.032 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T04:33:46.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.252+0000 7f71dea00640 1 -- 172.21.15.100:0/3412517780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71d80ad3c0 msgr2=0x7f71d80ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:46.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.252+0000 7f71dea00640 1 --2- 172.21.15.100:0/3412517780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71d80ad3c0 0x7f71d80ad7d0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f71d8074580 tx=0x7f71c402a4d0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.252+0000 7f71dea00640 1 -- 172.21.15.100:0/3412517780 shutdown_connections 2024-09-17T04:33:46.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.252+0000 7f71dea00640 1 --2- 172.21.15.100:0/3412517780 >> v2:172.21.15.184:3300/0 conn(0x7f71d80add10 0x7f71d81646e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.252+0000 7f71dea00640 1 --2- 172.21.15.100:0/3412517780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71d80ad3c0 0x7f71d80ad7d0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.252+0000 7f71dea00640 1 -- 172.21.15.100:0/3412517780 >> 172.21.15.100:0/3412517780 conn(0x7f71d806d730 msgr2=0x7f71d80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:46.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.252+0000 7f71dea00640 1 -- 172.21.15.100:0/3412517780 shutdown_connections 2024-09-17T04:33:46.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.253+0000 7f71dea00640 1 -- 172.21.15.100:0/3412517780 wait complete. 2024-09-17T04:33:46.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.255+0000 7f71dea00640 1 Processor -- start 2024-09-17T04:33:46.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.255+0000 7f71dea00640 1 -- start start 2024-09-17T04:33:46.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.256+0000 7f71dea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71d80704b0 0x7f71d8099a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:46.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.256+0000 7f71dea00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71d80ad3c0 0x7f71d8159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:46.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.256+0000 7f71dea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71d80add10 0x7f71d8162a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:46.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.256+0000 7f71dea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f71d80712b0 con 0x7f71d80704b0 2024-09-17T04:33:46.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.256+0000 7f71dea00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f71d8071130 con 0x7f71d80add10 2024-09-17T04:33:46.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.256+0000 7f71dea00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f71d8164c20 con 0x7f71d80ad3c0 2024-09-17T04:33:46.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.257+0000 7f71d7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71d80704b0 0x7f71d8099a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:46.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.257+0000 7f71d7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71d80704b0 0x7f71d8099a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39294/0 (socket says 172.21.15.100:39294) 2024-09-17T04:33:46.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.257+0000 7f71d7e00640 1 -- 172.21.15.100:0/3890740725 learned_addr learned my addr 172.21.15.100:0/3890740725 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:46.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.257+0000 7f71dca00640 1 --2- 172.21.15.100:0/3890740725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71d80add10 0x7f71d8162a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:46.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.257+0000 7f71d7400640 1 --2- 172.21.15.100:0/3890740725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71d80ad3c0 0x7f71d8159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:46.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.257+0000 7f71d7e00640 1 -- 172.21.15.100:0/3890740725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71d80ad3c0 msgr2=0x7f71d8159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:46.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.257+0000 7f71d7e00640 1 --2- 172.21.15.100:0/3890740725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71d80ad3c0 0x7f71d8159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.257+0000 7f71d7e00640 1 -- 172.21.15.100:0/3890740725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71d80add10 msgr2=0x7f71d8162a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:46.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.257+0000 7f71d7e00640 1 --2- 172.21.15.100:0/3890740725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71d80add10 0x7f71d8162a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.257+0000 7f71d7e00640 1 -- 172.21.15.100:0/3890740725 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d814e110 con 0x7f71d80704b0 2024-09-17T04:33:46.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.257+0000 7f71d7e00640 1 --2- 172.21.15.100:0/3890740725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71d80704b0 0x7f71d8099a30 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f71c800b420 tx=0x7f71c800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:46.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.258+0000 7f71cfe00640 1 -- 172.21.15.100:0/3890740725 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8011830 con 0x7f71d80704b0 2024-09-17T04:33:46.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.258+0000 7f71cfe00640 1 -- 172.21.15.100:0/3890740725 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71c8016070 con 0x7f71d80704b0 2024-09-17T04:33:46.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.258+0000 7f71cfe00640 1 -- 172.21.15.100:0/3890740725 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8011d60 con 0x7f71d80704b0 2024-09-17T04:33:46.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.258+0000 7f71dea00640 1 -- 172.21.15.100:0/3890740725 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d815a220 con 0x7f71d80704b0 2024-09-17T04:33:46.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.258+0000 7f71dea00640 1 -- 172.21.15.100:0/3890740725 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d814de30 con 0x7f71d80704b0 2024-09-17T04:33:46.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.259+0000 7f71dea00640 1 -- 172.21.15.100:0/3890740725 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71d80ad7d0 con 0x7f71d80704b0 2024-09-17T04:33:46.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.260+0000 7f71cfe00640 1 -- 172.21.15.100:0/3890740725 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f71c801b460 con 0x7f71d80704b0 2024-09-17T04:33:46.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.261+0000 7f71cfe00640 1 --2- 172.21.15.100:0/3890740725 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f71b80795b0 0x7f71b807bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:46.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.261+0000 7f71cfe00640 1 -- 172.21.15.100:0/3890740725 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4999+0+0 (secure 0 0 0) 0x7f71c8099400 con 0x7f71d80704b0 2024-09-17T04:33:46.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.261+0000 7f71d7400640 1 --2- 172.21.15.100:0/3890740725 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f71b80795b0 0x7f71b807bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:46.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.262+0000 7f71d7400640 1 --2- 172.21.15.100:0/3890740725 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f71b80795b0 0x7f71b807bac0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f71c4033000 tx=0x7f71c4004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:46.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.265+0000 7f71cfe00640 1 -- 172.21.15.100:0/3890740725 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f71c8020020 con 0x7f71d80704b0 2024-09-17T04:33:46.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.448+0000 7f71dea00640 1 -- 172.21.15.100:0/3890740725 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f71d815e420 con 0x7f71d80704b0 2024-09-17T04:33:46.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.449+0000 7f71cfe00640 1 -- 172.21.15.100:0/3890740725 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v51) ==== 59+0+2294 (secure 0 0 0) 0x7f71c8063100 con 0x7f71d80704b0 2024-09-17T04:33:46.449 INFO:teuthology.orchestra.run.smithi100.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T04:33:46.449 INFO:teuthology.orchestra.run.smithi100.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T04:33:46.449 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced client_mount_timeout 30 2024-09-17T04:33:46.450 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_client 20/20 2024-09-17T04:33:46.450 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_mgr 20/20 2024-09-17T04:33:46.450 INFO:teuthology.orchestra.run.smithi100.stdout:mgr advanced debug_ms 1/1 2024-09-17T04:33:46.450 INFO:teuthology.orchestra.run.smithi100.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T04:33:46.450 INFO:teuthology.orchestra.run.smithi100.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T04:33:46.450 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3938.598654 2024-09-17T04:33:46.450 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4008.640154 2024-09-17T04:33:46.450 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3706.588546 2024-09-17T04:33:46.450 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 6299.338665 2024-09-17T04:33:46.450 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3633.876699 2024-09-17T04:33:46.451 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3674.821841 2024-09-17T04:33:46.451 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3789.849984 2024-09-17T04:33:46.451 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3467.457298 2024-09-17T04:33:46.451 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_mds 20/20 2024-09-17T04:33:46.451 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T04:33:46.451 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced debug_ms 2/2 2024-09-17T04:33:46.451 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_debug_frag true 2024-09-17T04:33:46.451 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_debug_scatterstat true 2024-09-17T04:33:46.451 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T04:33:46.451 INFO:teuthology.orchestra.run.smithi100.stdout:mds dev mds_verify_scatter true 2024-09-17T04:33:46.451 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T04:33:46.452 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T04:33:46.452 INFO:teuthology.orchestra.run.smithi100.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T04:33:46.452 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced client_mount_timeout 600 2024-09-17T04:33:46.452 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced debug_client 20/20 2024-09-17T04:33:46.452 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced debug_ms 1/1 2024-09-17T04:33:46.452 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T04:33:46.452 INFO:teuthology.orchestra.run.smithi100.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T04:33:46.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.451+0000 7f71dea00640 1 -- 172.21.15.100:0/3890740725 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f71b80795b0 msgr2=0x7f71b807bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:46.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.451+0000 7f71dea00640 1 --2- 172.21.15.100:0/3890740725 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f71b80795b0 0x7f71b807bac0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f71c4033000 tx=0x7f71c4004730 comp rx=0 tx=0).stop 2024-09-17T04:33:46.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.451+0000 7f71dea00640 1 -- 172.21.15.100:0/3890740725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71d80704b0 msgr2=0x7f71d8099a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:46.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.451+0000 7f71dea00640 1 --2- 172.21.15.100:0/3890740725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71d80704b0 0x7f71d8099a30 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f71c800b420 tx=0x7f71c800b8e0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.451+0000 7f71dea00640 1 -- 172.21.15.100:0/3890740725 shutdown_connections 2024-09-17T04:33:46.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.451+0000 7f71dea00640 1 --2- 172.21.15.100:0/3890740725 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f71b80795b0 0x7f71b807bac0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.451+0000 7f71dea00640 1 --2- 172.21.15.100:0/3890740725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71d80add10 0x7f71d8162a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.451+0000 7f71dea00640 1 --2- 172.21.15.100:0/3890740725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71d80ad3c0 0x7f71d8159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.451+0000 7f71dea00640 1 --2- 172.21.15.100:0/3890740725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71d80704b0 0x7f71d8099a30 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.451+0000 7f71dea00640 1 -- 172.21.15.100:0/3890740725 >> 172.21.15.100:0/3890740725 conn(0x7f71d806d730 msgr2=0x7f71d80ae260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:46.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.451+0000 7f71dea00640 1 -- 172.21.15.100:0/3890740725 shutdown_connections 2024-09-17T04:33:46.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.451+0000 7f71dea00640 1 -- 172.21.15.100:0/3890740725 wait complete. 2024-09-17T04:33:46.476 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T04:33:46.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.698+0000 7f6e71a00640 1 -- 172.21.15.100:0/917318201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e6c0cd000 msgr2=0x7f6e6c0af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:46.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.698+0000 7f6e71a00640 1 --2- 172.21.15.100:0/917318201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e6c0cd000 0x7f6e6c0af0d0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0b3160 tx=0x7f6e6402aac0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.698+0000 7f6e71a00640 1 -- 172.21.15.100:0/917318201 shutdown_connections 2024-09-17T04:33:46.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.699+0000 7f6e71a00640 1 --2- 172.21.15.100:0/917318201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e6c0cd000 0x7f6e6c0af0d0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.699+0000 7f6e71a00640 1 --2- 172.21.15.100:0/917318201 >> v2:172.21.15.184:3300/0 conn(0x7f6e6c156ca0 0x7f6e6c0ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.699+0000 7f6e71a00640 1 -- 172.21.15.100:0/917318201 >> 172.21.15.100:0/917318201 conn(0x7f6e6c06d620 msgr2=0x7f6e6c0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:46.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.699+0000 7f6e71a00640 1 -- 172.21.15.100:0/917318201 shutdown_connections 2024-09-17T04:33:46.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.699+0000 7f6e71a00640 1 -- 172.21.15.100:0/917318201 wait complete. 2024-09-17T04:33:46.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.701+0000 7f6e71a00640 1 Processor -- start 2024-09-17T04:33:46.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.701+0000 7f6e71a00640 1 -- start start 2024-09-17T04:33:46.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.703+0000 7f6e71a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e6c0b3f30 0x7f6e6c074450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:46.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.703+0000 7f6e71a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e6c0cd000 0x7f6e6c1445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:46.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.703+0000 7f6e71a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e6c156ca0 0x7f6e6c144af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:46.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.703+0000 7f6e71a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c0af9a0 con 0x7f6e6c0cd000 2024-09-17T04:33:46.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.703+0000 7f6e71a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c0af820 con 0x7f6e6c0b3f30 2024-09-17T04:33:46.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.703+0000 7f6e71a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6e6c0afb20 con 0x7f6e6c156ca0 2024-09-17T04:33:46.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.703+0000 7f6e6b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e6c0cd000 0x7f6e6c1445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:46.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.703+0000 7f6e6b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e6c0cd000 0x7f6e6c1445b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39314/0 (socket says 172.21.15.100:39314) 2024-09-17T04:33:46.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.703+0000 7f6e6b400640 1 -- 172.21.15.100:0/1047743144 learned_addr learned my addr 172.21.15.100:0/1047743144 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:46.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.703+0000 7f6e6be00640 1 --2- 172.21.15.100:0/1047743144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e6c0b3f30 0x7f6e6c074450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:46.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.703+0000 7f6e71000640 1 --2- 172.21.15.100:0/1047743144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e6c156ca0 0x7f6e6c144af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:46.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.703+0000 7f6e6b400640 1 -- 172.21.15.100:0/1047743144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e6c156ca0 msgr2=0x7f6e6c144af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:46.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.703+0000 7f6e6b400640 1 --2- 172.21.15.100:0/1047743144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e6c156ca0 0x7f6e6c144af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.703+0000 7f6e6b400640 1 -- 172.21.15.100:0/1047743144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e6c0b3f30 msgr2=0x7f6e6c074450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:46.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.703+0000 7f6e6b400640 1 --2- 172.21.15.100:0/1047743144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e6c0b3f30 0x7f6e6c074450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.703+0000 7f6e6b400640 1 -- 172.21.15.100:0/1047743144 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e6c072eb0 con 0x7f6e6c0cd000 2024-09-17T04:33:46.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.703+0000 7f6e71000640 1 --2- 172.21.15.100:0/1047743144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e6c156ca0 0x7f6e6c144af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:46.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.704+0000 7f6e6b400640 1 --2- 172.21.15.100:0/1047743144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e6c0cd000 0x7f6e6c1445b0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f6e5800d6a0 tx=0x7f6e5800db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:46.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.704+0000 7f6e63e00640 1 -- 172.21.15.100:0/1047743144 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e58017070 con 0x7f6e6c0cd000 2024-09-17T04:33:46.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.704+0000 7f6e63e00640 1 -- 172.21.15.100:0/1047743144 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e58005310 con 0x7f6e6c0cd000 2024-09-17T04:33:46.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.704+0000 7f6e63e00640 1 -- 172.21.15.100:0/1047743144 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e58009820 con 0x7f6e6c0cd000 2024-09-17T04:33:46.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.704+0000 7f6e71a00640 1 -- 172.21.15.100:0/1047743144 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e6c0733b0 con 0x7f6e6c0cd000 2024-09-17T04:33:46.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.704+0000 7f6e71a00640 1 -- 172.21.15.100:0/1047743144 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e6c073970 con 0x7f6e6c0cd000 2024-09-17T04:33:46.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.705+0000 7f6e71a00640 1 -- 172.21.15.100:0/1047743144 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e6c0af0d0 con 0x7f6e6c0cd000 2024-09-17T04:33:46.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.706+0000 7f6e63e00640 1 -- 172.21.15.100:0/1047743144 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f6e580099c0 con 0x7f6e6c0cd000 2024-09-17T04:33:46.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.707+0000 7f6e63e00640 1 --2- 172.21.15.100:0/1047743144 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6e4c0795e0 0x7f6e4c07baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:46.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.707+0000 7f6e63e00640 1 -- 172.21.15.100:0/1047743144 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4999+0+0 (secure 0 0 0) 0x7f6e580998e0 con 0x7f6e6c0cd000 2024-09-17T04:33:46.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.707+0000 7f6e6be00640 1 --2- 172.21.15.100:0/1047743144 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6e4c0795e0 0x7f6e4c07baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:46.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.708+0000 7f6e6be00640 1 --2- 172.21.15.100:0/1047743144 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6e4c0795e0 0x7f6e4c07baf0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0cf590 tx=0x7f6e5c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:46.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.712+0000 7f6e63e00640 1 -- 172.21.15.100:0/1047743144 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6e580635e0 con 0x7f6e6c0cd000 2024-09-17T04:33:46.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.894+0000 7f6e71a00640 1 -- 172.21.15.100:0/1047743144 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6e6c156660 con 0x7f6e6c0cd000 2024-09-17T04:33:46.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.898+0000 7f6e63e00640 1 -- 172.21.15.100:0/1047743144 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v52)=0 v52) ==== 122+0+0 (secure 0 0 0) 0x7f6e5809ae20 con 0x7f6e6c0cd000 2024-09-17T04:33:46.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.900+0000 7f6e71a00640 1 -- 172.21.15.100:0/1047743144 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6e4c0795e0 msgr2=0x7f6e4c07baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:46.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.900+0000 7f6e71a00640 1 --2- 172.21.15.100:0/1047743144 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6e4c0795e0 0x7f6e4c07baf0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0cf590 tx=0x7f6e5c009040 comp rx=0 tx=0).stop 2024-09-17T04:33:46.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.900+0000 7f6e71a00640 1 -- 172.21.15.100:0/1047743144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e6c0cd000 msgr2=0x7f6e6c1445b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:46.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.900+0000 7f6e71a00640 1 --2- 172.21.15.100:0/1047743144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e6c0cd000 0x7f6e6c1445b0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f6e5800d6a0 tx=0x7f6e5800db60 comp rx=0 tx=0).stop 2024-09-17T04:33:46.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.900+0000 7f6e71a00640 1 -- 172.21.15.100:0/1047743144 shutdown_connections 2024-09-17T04:33:46.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.900+0000 7f6e71a00640 1 --2- 172.21.15.100:0/1047743144 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6e4c0795e0 0x7f6e4c07baf0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.900+0000 7f6e71a00640 1 --2- 172.21.15.100:0/1047743144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e6c156ca0 0x7f6e6c144af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.900+0000 7f6e71a00640 1 --2- 172.21.15.100:0/1047743144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e6c0cd000 0x7f6e6c1445b0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.900+0000 7f6e71a00640 1 --2- 172.21.15.100:0/1047743144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e6c0b3f30 0x7f6e6c074450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.900+0000 7f6e71a00640 1 -- 172.21.15.100:0/1047743144 >> 172.21.15.100:0/1047743144 conn(0x7f6e6c06d620 msgr2=0x7f6e6c1550a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:46.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.900+0000 7f6e71a00640 1 -- 172.21.15.100:0/1047743144 shutdown_connections 2024-09-17T04:33:46.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:46.900+0000 7f6e71a00640 1 -- 172.21.15.100:0/1047743144 wait complete. 2024-09-17T04:33:46.928 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T04:33:46.928 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:33:47.016 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:33:47.108 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T04:33:47.108 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:33:47.108 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:33:47.143 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:33:47.144 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:33:47.144 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:33:47.181 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:33:47.181 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:33:47.181 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:33:47.218 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:33:47.218 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:33:47.219 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:33:47.257 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:33:47.257 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T04:33:47.258 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:33:47.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.477+0000 7f6fdfe00640 1 -- 172.21.15.100:0/1944176560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd80b3550 msgr2=0x7f6fd80b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:47.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.477+0000 7f6fdfe00640 1 --2- 172.21.15.100:0/1944176560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd80b3550 0x7f6fd80b3960 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0005fd0 tx=0x7f6fd002a540 comp rx=0 tx=0).stop 2024-09-17T04:33:47.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.477+0000 7f6fdfe00640 1 -- 172.21.15.100:0/1944176560 shutdown_connections 2024-09-17T04:33:47.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.477+0000 7f6fdfe00640 1 --2- 172.21.15.100:0/1944176560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd80b3550 0x7f6fd80b3960 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.477+0000 7f6fdfe00640 1 --2- 172.21.15.100:0/1944176560 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fd815d760 0x7f6fd81535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.477+0000 7f6fdfe00640 1 -- 172.21.15.100:0/1944176560 >> 172.21.15.100:0/1944176560 conn(0x7f6fd806d730 msgr2=0x7f6fd80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:47.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.477+0000 7f6fdfe00640 1 -- 172.21.15.100:0/1944176560 shutdown_connections 2024-09-17T04:33:47.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.477+0000 7f6fdfe00640 1 -- 172.21.15.100:0/1944176560 wait complete. 2024-09-17T04:33:47.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.479+0000 7f6fdfe00640 1 Processor -- start 2024-09-17T04:33:47.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.479+0000 7f6fdfe00640 1 -- start start 2024-09-17T04:33:47.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.481+0000 7f6fdfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd80ad0e0 0x7f6fd81f6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:47.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.481+0000 7f6fdfe00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fd80b3550 0x7f6fd8158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:47.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.481+0000 7f6fdfe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fd815d760 0x7f6fd8073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:47.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.481+0000 7f6fdfe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd80b4da0 con 0x7f6fd80ad0e0 2024-09-17T04:33:47.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.481+0000 7f6fdfe00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd80b4c20 con 0x7f6fd815d760 2024-09-17T04:33:47.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.481+0000 7f6fdfe00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6fd80b3ea0 con 0x7f6fd80b3550 2024-09-17T04:33:47.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.481+0000 7f6fdd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd80ad0e0 0x7f6fd81f6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:47.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.481+0000 7f6fdd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd80ad0e0 0x7f6fd81f6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39332/0 (socket says 172.21.15.100:39332) 2024-09-17T04:33:47.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.481+0000 7f6fdd400640 1 -- 172.21.15.100:0/2541168511 learned_addr learned my addr 172.21.15.100:0/2541168511 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:47.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.481+0000 7f6fdca00640 1 --2- 172.21.15.100:0/2541168511 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fd80b3550 0x7f6fd8158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:47.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.481+0000 7f6fdde00640 1 --2- 172.21.15.100:0/2541168511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fd815d760 0x7f6fd8073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:47.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.481+0000 7f6fdd400640 1 -- 172.21.15.100:0/2541168511 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fd80b3550 msgr2=0x7f6fd8158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:47.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.481+0000 7f6fdd400640 1 --2- 172.21.15.100:0/2541168511 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fd80b3550 0x7f6fd8158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.481+0000 7f6fdd400640 1 -- 172.21.15.100:0/2541168511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fd815d760 msgr2=0x7f6fd8073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:47.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.481+0000 7f6fdd400640 1 --2- 172.21.15.100:0/2541168511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fd815d760 0x7f6fd8073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.481+0000 7f6fdd400640 1 -- 172.21.15.100:0/2541168511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd81591e0 con 0x7f6fd80ad0e0 2024-09-17T04:33:47.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.481+0000 7f6fdde00640 1 --2- 172.21.15.100:0/2541168511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fd815d760 0x7f6fd8073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:47.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.482+0000 7f6fdd400640 1 --2- 172.21.15.100:0/2541168511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd80ad0e0 0x7f6fd81f6560 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00eb30 tx=0x7f6fcc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:47.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.482+0000 7f6fd5600640 1 -- 172.21.15.100:0/2541168511 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc010430 con 0x7f6fd80ad0e0 2024-09-17T04:33:47.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.482+0000 7f6fd5600640 1 -- 172.21.15.100:0/2541168511 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fcc0105d0 con 0x7f6fd80ad0e0 2024-09-17T04:33:47.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.482+0000 7f6fd5600640 1 -- 172.21.15.100:0/2541168511 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc010a20 con 0x7f6fd80ad0e0 2024-09-17T04:33:47.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.482+0000 7f6fdfe00640 1 -- 172.21.15.100:0/2541168511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fd81594c0 con 0x7f6fd80ad0e0 2024-09-17T04:33:47.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.482+0000 7f6fdfe00640 1 -- 172.21.15.100:0/2541168511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fd81f5d60 con 0x7f6fd80ad0e0 2024-09-17T04:33:47.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.483+0000 7f6fdfe00640 1 -- 172.21.15.100:0/2541168511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fd80b4330 con 0x7f6fd80ad0e0 2024-09-17T04:33:47.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.484+0000 7f6fd5600640 1 -- 172.21.15.100:0/2541168511 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f6fcc010bc0 con 0x7f6fd80ad0e0 2024-09-17T04:33:47.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.485+0000 7f6fd5600640 1 --2- 172.21.15.100:0/2541168511 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6fb80795b0 0x7f6fb807bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:47.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.485+0000 7f6fd5600640 1 -- 172.21.15.100:0/2541168511 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4999+0+0 (secure 0 0 0) 0x7f6fcc09a600 con 0x7f6fd80ad0e0 2024-09-17T04:33:47.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.486+0000 7f6fdca00640 1 --2- 172.21.15.100:0/2541168511 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6fb80795b0 0x7f6fb807bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:47.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.486+0000 7f6fdca00640 1 --2- 172.21.15.100:0/2541168511 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6fb80795b0 0x7f6fb807bac0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6fd80b3d30 tx=0x7f6fc4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:47.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.490+0000 7f6fd5600640 1 -- 172.21.15.100:0/2541168511 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6fcc0a1070 con 0x7f6fd80ad0e0 2024-09-17T04:33:47.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.710+0000 7f6fdfe00640 1 -- 172.21.15.100:0/2541168511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6fd815b1f0 con 0x7f6fd80ad0e0 2024-09-17T04:33:47.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.710+0000 7f6fd5600640 1 -- 172.21.15.100:0/2541168511 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+3925 (secure 0 0 0) 0x7f6fcc064300 con 0x7f6fd80ad0e0 2024-09-17T04:33:47.711 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:47.711 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":114,"btime":"2024-09-17T04:33:40:984065+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7295,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6801/1131376012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1131376012},{"type":"v1","addr":"172.21.15.184:6801","nonce":1131376012}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":8230,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/3930702908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3930702908},{"type":"v1","addr":"172.21.15.100:6833","nonce":3930702908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":8479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/789814004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":789814004},{"type":"v1","addr":"172.21.15.184:6835","nonce":789814004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":114},{"gid":8808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2238795078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2238795078},{"type":"v1","addr":"172.21.15.100:6835","nonce":2238795078}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105}],"filesystems":[]} 2024-09-17T04:33:47.711 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 114 2024-09-17T04:33:47.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.713+0000 7f6fdfe00640 1 -- 172.21.15.100:0/2541168511 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6fb80795b0 msgr2=0x7f6fb807bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:47.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.713+0000 7f6fdfe00640 1 --2- 172.21.15.100:0/2541168511 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6fb80795b0 0x7f6fb807bac0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6fd80b3d30 tx=0x7f6fc4009040 comp rx=0 tx=0).stop 2024-09-17T04:33:47.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.713+0000 7f6fdfe00640 1 -- 172.21.15.100:0/2541168511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd80ad0e0 msgr2=0x7f6fd81f6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:47.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.713+0000 7f6fdfe00640 1 --2- 172.21.15.100:0/2541168511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd80ad0e0 0x7f6fd81f6560 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00eb30 tx=0x7f6fcc00c2d0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.713+0000 7f6fdfe00640 1 -- 172.21.15.100:0/2541168511 shutdown_connections 2024-09-17T04:33:47.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.713+0000 7f6fdfe00640 1 --2- 172.21.15.100:0/2541168511 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6fb80795b0 0x7f6fb807bac0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.713+0000 7f6fdfe00640 1 --2- 172.21.15.100:0/2541168511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fd815d760 0x7f6fd8073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.713+0000 7f6fdfe00640 1 --2- 172.21.15.100:0/2541168511 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fd80b3550 0x7f6fd8158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.713+0000 7f6fdfe00640 1 --2- 172.21.15.100:0/2541168511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd80ad0e0 0x7f6fd81f6560 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.713+0000 7f6fdfe00640 1 -- 172.21.15.100:0/2541168511 >> 172.21.15.100:0/2541168511 conn(0x7f6fd806d730 msgr2=0x7f6fd80cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:47.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.713+0000 7f6fdfe00640 1 -- 172.21.15.100:0/2541168511 shutdown_connections 2024-09-17T04:33:47.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:47.713+0000 7f6fdfe00640 1 -- 172.21.15.100:0/2541168511 wait complete. 2024-09-17T04:33:47.737 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T04:33:47.737 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T04:33:47.737 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T04:33:47.738 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:33:47.738 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T04:33:47.738 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T04:33:47.738 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T04:33:47.738 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:33:47.738 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T04:33:47.738 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T04:33:47.739 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T04:33:47.739 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:33:47.739 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T04:33:47.739 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T04:33:47.739 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T04:33:47.739 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:33:47.763 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T04:33:47.763 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T04:33:47.764 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T04:33:47.764 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T04:33:47.766 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T04:33:47.766 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T04:33:47.766 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T04:33:47.767 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T04:33:47.808 INFO:tasks.ceph.mds.b:Started 2024-09-17T04:33:47.808 DEBUG:teuthology.parallel:result is None 2024-09-17T04:33:47.810 INFO:tasks.ceph.mds.d:Started 2024-09-17T04:33:47.810 DEBUG:teuthology.parallel:result is None 2024-09-17T04:33:47.812 INFO:tasks.ceph.mds.a:Started 2024-09-17T04:33:47.812 DEBUG:teuthology.parallel:result is None 2024-09-17T04:33:47.814 INFO:tasks.ceph.mds.c:Started 2024-09-17T04:33:47.815 DEBUG:teuthology.parallel:result is None 2024-09-17T04:33:47.815 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T04:33:48.004 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-09-17T04:33:48.005 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-09-17T04:33:48.008 INFO:tasks.ceph.mds.a.smithi100.stdout:starting mds.a at 2024-09-17T04:33:48.014 INFO:tasks.ceph.mds.c.smithi100.stdout:starting mds.c at 2024-09-17T04:33:48.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.048+0000 7f5fa5200640 1 -- 172.21.15.100:0/1856170495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fa015e770 msgr2=0x7f5fa00b7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:48.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.048+0000 7f5fa5200640 1 --2- 172.21.15.100:0/1856170495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fa015e770 0x7f5fa00b7a00 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c005fd0 tx=0x7f5f8c02a440 comp rx=0 tx=0).stop 2024-09-17T04:33:48.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.048+0000 7f5fa5200640 1 -- 172.21.15.100:0/1856170495 shutdown_connections 2024-09-17T04:33:48.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.048+0000 7f5fa5200640 1 --2- 172.21.15.100:0/1856170495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fa015e770 0x7f5fa00b7a00 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.048+0000 7f5fa5200640 1 --2- 172.21.15.100:0/1856170495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fa00cf6c0 0x7f5fa0162e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.048+0000 7f5fa5200640 1 -- 172.21.15.100:0/1856170495 >> 172.21.15.100:0/1856170495 conn(0x7f5fa00d44f0 msgr2=0x7f5fa00d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:48.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.048+0000 7f5fa5200640 1 -- 172.21.15.100:0/1856170495 shutdown_connections 2024-09-17T04:33:48.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.048+0000 7f5fa5200640 1 -- 172.21.15.100:0/1856170495 wait complete. 2024-09-17T04:33:48.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.050+0000 7f5fa5200640 1 Processor -- start 2024-09-17T04:33:48.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.051+0000 7f5fa5200640 1 -- start start 2024-09-17T04:33:48.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.052+0000 7f5fa5200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fa00b7f40 0x7f5fa014cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:48.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.052+0000 7f5fa5200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fa015e770 0x7f5fa00b0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:48.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.052+0000 7f5fa5200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fa01f6080 0x7f5fa01f8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:48.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.052+0000 7f5fa5200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa00d39a0 con 0x7f5fa00b7f40 2024-09-17T04:33:48.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.052+0000 7f5fa5200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa00d3820 con 0x7f5fa015e770 2024-09-17T04:33:48.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.052+0000 7f5fa5200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5fa00d3b20 con 0x7f5fa01f6080 2024-09-17T04:33:48.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.052+0000 7f5f9f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fa00b7f40 0x7f5fa014cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:48.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.052+0000 7f5f9f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fa00b7f40 0x7f5fa014cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39370/0 (socket says 172.21.15.100:39370) 2024-09-17T04:33:48.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.052+0000 7f5f9f400640 1 -- 172.21.15.100:0/1409980533 learned_addr learned my addr 172.21.15.100:0/1409980533 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:48.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.053+0000 7f5f9f400640 1 -- 172.21.15.100:0/1409980533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fa01f6080 msgr2=0x7f5fa01f8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:48.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.053+0000 7f5f9f400640 1 --2- 172.21.15.100:0/1409980533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fa01f6080 0x7f5fa01f8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.053+0000 7f5f9f400640 1 -- 172.21.15.100:0/1409980533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fa015e770 msgr2=0x7f5fa00b0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:48.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.053+0000 7f5f9f400640 1 --2- 172.21.15.100:0/1409980533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fa015e770 0x7f5fa00b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.053+0000 7f5f9f400640 1 -- 172.21.15.100:0/1409980533 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa00aadc0 con 0x7f5fa00b7f40 2024-09-17T04:33:48.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.053+0000 7f5f9f400640 1 --2- 172.21.15.100:0/1409980533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fa00b7f40 0x7f5fa014cc70 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000e790 tx=0x7f5f9000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:48.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.053+0000 7f5f97400640 1 -- 172.21.15.100:0/1409980533 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9000eea0 con 0x7f5fa00b7f40 2024-09-17T04:33:48.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.053+0000 7f5f97400640 1 -- 172.21.15.100:0/1409980533 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f9000cd50 con 0x7f5fa00b7f40 2024-09-17T04:33:48.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.053+0000 7f5fa5200640 1 -- 172.21.15.100:0/1409980533 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fa00ab0a0 con 0x7f5fa00b7f40 2024-09-17T04:33:48.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.053+0000 7f5f97400640 1 -- 172.21.15.100:0/1409980533 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90010690 con 0x7f5fa00b7f40 2024-09-17T04:33:48.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.053+0000 7f5fa5200640 1 -- 172.21.15.100:0/1409980533 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fa00ab660 con 0x7f5fa00b7f40 2024-09-17T04:33:48.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.055+0000 7f5fa5200640 1 -- 172.21.15.100:0/1409980533 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fa00cf6c0 con 0x7f5fa00b7f40 2024-09-17T04:33:48.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.056+0000 7f5f97400640 1 -- 172.21.15.100:0/1409980533 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f5f90010830 con 0x7f5fa00b7f40 2024-09-17T04:33:48.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.057+0000 7f5f97400640 1 --2- 172.21.15.100:0/1409980533 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5f800795b0 0x7f5f8007bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:48.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.057+0000 7f5f97400640 1 -- 172.21.15.100:0/1409980533 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4999+0+0 (secure 0 0 0) 0x7f5f9009a220 con 0x7f5fa00b7f40 2024-09-17T04:33:48.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.057+0000 7f5f9ea00640 1 --2- 172.21.15.100:0/1409980533 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5f800795b0 0x7f5f8007bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:48.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.058+0000 7f5f9ea00640 1 --2- 172.21.15.100:0/1409980533 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5f800795b0 0x7f5f8007bac0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c033000 tx=0x7f5f8c0046c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:48.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.062+0000 7f5f97400640 1 -- 172.21.15.100:0/1409980533 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f5f90063ef0 con 0x7f5fa00b7f40 2024-09-17T04:33:48.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:48.257+0000 7f5fa5200640 1 -- 172.21.15.100:0/1409980533 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f5fa0162180 con 0x7f5fa00b7f40 2024-09-17T04:33:49.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.025+0000 7f5f97400640 1 -- 172.21.15.100:0/1409980533 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v161) ==== 97+0+0 (secure 0 0 0) 0x7f5f900637f0 con 0x7f5fa00b7f40 2024-09-17T04:33:49.025 INFO:teuthology.orchestra.run.smithi100.stderr: removed all blocklist entries 2024-09-17T04:33:49.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.027+0000 7f5fa5200640 1 -- 172.21.15.100:0/1409980533 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5f800795b0 msgr2=0x7f5f8007bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:49.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.027+0000 7f5fa5200640 1 --2- 172.21.15.100:0/1409980533 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5f800795b0 0x7f5f8007bac0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c033000 tx=0x7f5f8c0046c0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.028+0000 7f5fa5200640 1 -- 172.21.15.100:0/1409980533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fa00b7f40 msgr2=0x7f5fa014cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:49.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.028+0000 7f5fa5200640 1 --2- 172.21.15.100:0/1409980533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fa00b7f40 0x7f5fa014cc70 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000e790 tx=0x7f5f9000ec50 comp rx=0 tx=0).stop 2024-09-17T04:33:49.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.028+0000 7f5fa5200640 1 -- 172.21.15.100:0/1409980533 shutdown_connections 2024-09-17T04:33:49.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.028+0000 7f5fa5200640 1 --2- 172.21.15.100:0/1409980533 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f5f800795b0 0x7f5f8007bac0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.028+0000 7f5fa5200640 1 --2- 172.21.15.100:0/1409980533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fa01f6080 0x7f5fa01f8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.028+0000 7f5fa5200640 1 --2- 172.21.15.100:0/1409980533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fa015e770 0x7f5fa00b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.028+0000 7f5fa5200640 1 --2- 172.21.15.100:0/1409980533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fa00b7f40 0x7f5fa014cc70 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.028+0000 7f5fa5200640 1 -- 172.21.15.100:0/1409980533 >> 172.21.15.100:0/1409980533 conn(0x7f5fa00d44f0 msgr2=0x7f5fa00d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:49.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.028+0000 7f5fa5200640 1 -- 172.21.15.100:0/1409980533 shutdown_connections 2024-09-17T04:33:49.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.028+0000 7f5fa5200640 1 -- 172.21.15.100:0/1409980533 wait complete. 2024-09-17T04:33:49.052 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T04:33:49.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.273+0000 7f104fe00640 1 -- 172.21.15.100:0/4254321675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10500b2f60 msgr2=0x7f105015a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:49.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.273+0000 7f104fe00640 1 --2- 172.21.15.100:0/4254321675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10500b2f60 0x7f105015a0f0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f1050051590 tx=0x7f103802a560 comp rx=0 tx=0).stop 2024-09-17T04:33:49.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.273+0000 7f104fe00640 1 -- 172.21.15.100:0/4254321675 shutdown_connections 2024-09-17T04:33:49.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.273+0000 7f104fe00640 1 --2- 172.21.15.100:0/4254321675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10500b2f60 0x7f105015a0f0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.273+0000 7f104fe00640 1 --2- 172.21.15.100:0/4254321675 >> v2:172.21.15.184:3300/0 conn(0x7f10500ad9c0 0x7f10500af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.273+0000 7f104fe00640 1 -- 172.21.15.100:0/4254321675 >> 172.21.15.100:0/4254321675 conn(0x7f10500b1e40 msgr2=0x7f10500b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:49.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.273+0000 7f104fe00640 1 -- 172.21.15.100:0/4254321675 shutdown_connections 2024-09-17T04:33:49.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.273+0000 7f104fe00640 1 -- 172.21.15.100:0/4254321675 wait complete. 2024-09-17T04:33:49.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.276+0000 7f104fe00640 1 Processor -- start 2024-09-17T04:33:49.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.276+0000 7f104fe00640 1 -- start start 2024-09-17T04:33:49.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.277+0000 7f104fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10500ad9c0 0x7f1050157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:49.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.278+0000 7f104fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10500b2f60 0x7f10501f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:49.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.278+0000 7f104fe00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f105015d670 0x7f1050074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:49.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.278+0000 7f104fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f105006f290 con 0x7f10500ad9c0 2024-09-17T04:33:49.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.278+0000 7f104fe00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1050070d50 con 0x7f10500b2f60 2024-09-17T04:33:49.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.278+0000 7f104fe00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f105006f410 con 0x7f105015d670 2024-09-17T04:33:49.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.278+0000 7f104ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10500ad9c0 0x7f1050157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:49.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.278+0000 7f104ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10500ad9c0 0x7f1050157670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39392/0 (socket says 172.21.15.100:39392) 2024-09-17T04:33:49.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.278+0000 7f104ea00640 1 -- 172.21.15.100:0/3587266601 learned_addr learned my addr 172.21.15.100:0/3587266601 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:49.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.278+0000 7f1046000640 1 --2- 172.21.15.100:0/3587266601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10500b2f60 0x7f10501f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:49.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.278+0000 7f104f400640 1 --2- 172.21.15.100:0/3587266601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f105015d670 0x7f1050074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:49.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.278+0000 7f104ea00640 1 -- 172.21.15.100:0/3587266601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f105015d670 msgr2=0x7f1050074550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:49.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.278+0000 7f104ea00640 1 --2- 172.21.15.100:0/3587266601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f105015d670 0x7f1050074550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.278+0000 7f104ea00640 1 -- 172.21.15.100:0/3587266601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10500b2f60 msgr2=0x7f10501f5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:49.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.278+0000 7f104ea00640 1 --2- 172.21.15.100:0/3587266601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10500b2f60 0x7f10501f5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.278+0000 7f104ea00640 1 -- 172.21.15.100:0/3587266601 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10501f6150 con 0x7f10500ad9c0 2024-09-17T04:33:49.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.279+0000 7f104ea00640 1 --2- 172.21.15.100:0/3587266601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10500ad9c0 0x7f1050157670 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f104000d770 tx=0x7f104000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:49.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.279+0000 7f1047400640 1 -- 172.21.15.100:0/3587266601 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040017070 con 0x7f10500ad9c0 2024-09-17T04:33:49.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.279+0000 7f1047400640 1 -- 172.21.15.100:0/3587266601 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1040009d20 con 0x7f10500ad9c0 2024-09-17T04:33:49.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.279+0000 7f104fe00640 1 -- 172.21.15.100:0/3587266601 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10500c9690 con 0x7f10500ad9c0 2024-09-17T04:33:49.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.279+0000 7f1047400640 1 -- 172.21.15.100:0/3587266601 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10400106c0 con 0x7f10500ad9c0 2024-09-17T04:33:49.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.279+0000 7f104fe00640 1 -- 172.21.15.100:0/3587266601 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1050073d50 con 0x7f10500ad9c0 2024-09-17T04:33:49.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.280+0000 7f104fe00640 1 -- 172.21.15.100:0/3587266601 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f105006f8c0 con 0x7f10500ad9c0 2024-09-17T04:33:49.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.282+0000 7f1047400640 1 -- 172.21.15.100:0/3587266601 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f104001a070 con 0x7f10500ad9c0 2024-09-17T04:33:49.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.282+0000 7f1047400640 1 --2- 172.21.15.100:0/3587266601 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f102c0795b0 0x7f102c07bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:49.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.283+0000 7f1047400640 1 -- 172.21.15.100:0/3587266601 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4913+0+0 (secure 0 0 0) 0x7f1040099870 con 0x7f10500ad9c0 2024-09-17T04:33:49.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.283+0000 7f1046000640 1 --2- 172.21.15.100:0/3587266601 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f102c0795b0 0x7f102c07bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:49.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.283+0000 7f1046000640 1 --2- 172.21.15.100:0/3587266601 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f102c0795b0 0x7f102c07bac0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1038033000 tx=0x7f1038004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:49.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.287+0000 7f1047400640 1 -- 172.21.15.100:0/3587266601 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f10501f6150 con 0x7f10500ad9c0 2024-09-17T04:33:49.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.537+0000 7f104fe00640 1 -- 172.21.15.100:0/3587266601 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f105015a290 con 0x7f10500ad9c0 2024-09-17T04:33:49.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.537+0000 7f1047400640 1 -- 172.21.15.100:0/3587266601 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) ==== 80+0+5304 (secure 0 0 0) 0x7f1040062f80 con 0x7f10500ad9c0 2024-09-17T04:33:49.538 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:49.538 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:33:49.538 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_dump": [ 2024-09-17T04:33:49.538 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:49.538 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.a", 2024-09-17T04:33:49.538 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmV8asJRAAkSgIWqGc1OeSRbs9PEOT8w==", 2024-09-17T04:33:49.538 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:49.538 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:33:49.538 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:33:49.538 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:33:49.538 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:49.539 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:49.539 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:49.539 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:49.539 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.b", 2024-09-17T04:33:49.539 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmpM7ZBRAA1Og785tJwa6NA9zBdmNNQA==", 2024-09-17T04:33:49.539 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:49.539 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:33:49.539 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:33:49.539 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:33:49.539 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:49.539 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:49.539 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:49.539 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:49.539 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.c", 2024-09-17T04:33:49.540 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmXaOGMxAAT+96g/5fAzPW8Cc2HtQUyA==", 2024-09-17T04:33:49.540 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:49.540 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:33:49.540 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:33:49.540 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:33:49.540 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:49.540 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:49.540 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:49.540 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:49.540 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mds.d", 2024-09-17T04:33:49.540 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmRqC9ExAA23kZ0Ks1lGN2k4GXL8ASrg==", 2024-09-17T04:33:49.540 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:49.540 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:33:49.540 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:33:49.541 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:33:49.541 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:49.541 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:49.541 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:49.541 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:49.541 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.0", 2024-09-17T04:33:49.541 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAwBOlmofFmAxAARerYkU9B2a9MM9vHa3o4LQ==", 2024-09-17T04:33:49.541 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:49.541 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:33:49.541 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:33:49.541 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:49.541 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:49.541 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:49.541 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:49.542 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.1", 2024-09-17T04:33:49.542 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAzBOlmbz6VExAAoVqSqXTc1wArKhdA6+NxnA==", 2024-09-17T04:33:49.542 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:49.542 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:33:49.542 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:33:49.542 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:49.542 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:49.542 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:49.542 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:49.542 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.2", 2024-09-17T04:33:49.542 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA2BOlmfeSAJBAAchY6XhyDRusc+zqwQ6v6wQ==", 2024-09-17T04:33:49.542 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:49.542 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:33:49.542 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:33:49.542 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:49.543 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:49.543 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:49.543 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:49.543 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.3", 2024-09-17T04:33:49.543 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQA6BOlm/TQiBBAAHhbs2+RCjAenGR1frhbTmQ==", 2024-09-17T04:33:49.543 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:49.543 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:33:49.543 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:33:49.543 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:49.543 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:49.543 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:49.543 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:49.543 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.4", 2024-09-17T04:33:49.543 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBABOlm7OKvBhAALkaAKEUYN5ChwGejoSF8yA==", 2024-09-17T04:33:49.544 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:49.544 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:33:49.544 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:33:49.544 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:49.544 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:49.544 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:49.544 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:49.544 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.5", 2024-09-17T04:33:49.544 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBDBOlmQpi1FhAAbA47dg6ckK1PFoCZL0SBEQ==", 2024-09-17T04:33:49.544 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:49.544 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:33:49.544 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:33:49.544 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:49.544 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:49.545 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:49.545 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:49.545 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.6", 2024-09-17T04:33:49.545 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBGBOlmx/w0JhAAWcPvZkMYtXD/EMKrg/W/4Q==", 2024-09-17T04:33:49.545 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:49.545 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:33:49.545 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:33:49.545 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:49.545 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:49.545 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:49.545 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:49.545 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "osd.7", 2024-09-17T04:33:49.545 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQBJBOlmWHjWNBAAgdul2UKY9g5p7EVi0ZdrFw==", 2024-09-17T04:33:49.545 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:49.546 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow profile osd", 2024-09-17T04:33:49.546 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile osd", 2024-09-17T04:33:49.546 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:49.546 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:49.546 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:49.546 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:49.546 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.0", 2024-09-17T04:33:49.546 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmfYg9IBAABP1uRKGOugrA47VxwRLkjA==", 2024-09-17T04:33:49.546 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:49.546 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:33:49.546 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow r", 2024-09-17T04:33:49.546 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T04:33:49.546 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:49.546 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:49.547 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:49.547 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.1", 2024-09-17T04:33:49.547 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmByhhNRAAPGaxFCDd8ywFVwwhprrPbQ==", 2024-09-17T04:33:49.547 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:49.547 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:33:49.547 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:33:49.547 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:33:49.547 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:33:49.547 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:49.547 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:49.547 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:49.547 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.2", 2024-09-17T04:33:49.547 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAsBOlmgcm0KhAAGF06VlHjBM/AGxafN6/3aA==", 2024-09-17T04:33:49.548 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:49.548 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:33:49.548 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:33:49.548 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:33:49.548 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:33:49.548 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:49.548 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:49.548 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:49.548 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.3", 2024-09-17T04:33:49.548 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAtBOlm9VQ6BBAAXcahu35/uDm8Z+f+3SF0Gg==", 2024-09-17T04:33:49.548 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:49.548 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow", 2024-09-17T04:33:49.548 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow r", 2024-09-17T04:33:49.548 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow rw", 2024-09-17T04:33:49.549 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow rwx" 2024-09-17T04:33:49.549 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:49.549 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:49.549 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:49.549 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "client.admin", 2024-09-17T04:33:49.549 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQAqBOlmZ+bIJBAAG3oaMNlYWNakywGDvAGbGw==", 2024-09-17T04:33:49.549 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:49.549 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:33:49.549 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": "allow *", 2024-09-17T04:33:49.549 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow *", 2024-09-17T04:33:49.549 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:49.549 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:49.549 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:49.549 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:49.550 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mgr.x", 2024-09-17T04:33:49.550 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmfaL7GhAArqLH66RkbBsz+TNOKtPHOA==", 2024-09-17T04:33:49.550 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:49.550 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:33:49.550 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile mgr", 2024-09-17T04:33:49.550 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:49.550 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:49.550 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-17T04:33:49.550 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:49.550 INFO:teuthology.orchestra.run.smithi100.stdout: "entity": "mgr.y", 2024-09-17T04:33:49.550 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "AQArBOlmM4D1DhAA5uUWgRVOMujWqbYzOjFp9A==", 2024-09-17T04:33:49.550 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": { 2024-09-17T04:33:49.550 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": "allow *", 2024-09-17T04:33:49.550 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": "allow profile mgr", 2024-09-17T04:33:49.551 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": "allow *" 2024-09-17T04:33:49.551 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:49.551 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:49.551 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-17T04:33:49.551 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:33:49.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.540+0000 7f104fe00640 1 -- 172.21.15.100:0/3587266601 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f102c0795b0 msgr2=0x7f102c07bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:49.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.540+0000 7f104fe00640 1 --2- 172.21.15.100:0/3587266601 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f102c0795b0 0x7f102c07bac0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1038033000 tx=0x7f1038004730 comp rx=0 tx=0).stop 2024-09-17T04:33:49.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.540+0000 7f104fe00640 1 -- 172.21.15.100:0/3587266601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10500ad9c0 msgr2=0x7f1050157670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:49.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.540+0000 7f104fe00640 1 --2- 172.21.15.100:0/3587266601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10500ad9c0 0x7f1050157670 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f104000d770 tx=0x7f104000dc30 comp rx=0 tx=0).stop 2024-09-17T04:33:49.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.540+0000 7f104fe00640 1 -- 172.21.15.100:0/3587266601 shutdown_connections 2024-09-17T04:33:49.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.540+0000 7f104fe00640 1 --2- 172.21.15.100:0/3587266601 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f102c0795b0 0x7f102c07bac0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.540+0000 7f104fe00640 1 --2- 172.21.15.100:0/3587266601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f105015d670 0x7f1050074550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.540+0000 7f104fe00640 1 --2- 172.21.15.100:0/3587266601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10500b2f60 0x7f10501f5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.540+0000 7f104fe00640 1 --2- 172.21.15.100:0/3587266601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10500ad9c0 0x7f1050157670 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.540+0000 7f104fe00640 1 -- 172.21.15.100:0/3587266601 >> 172.21.15.100:0/3587266601 conn(0x7f10500b1e40 msgr2=0x7f105015c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:49.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.540+0000 7f104fe00640 1 -- 172.21.15.100:0/3587266601 shutdown_connections 2024-09-17T04:33:49.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.540+0000 7f104fe00640 1 -- 172.21.15.100:0/3587266601 wait complete. 2024-09-17T04:33:49.562 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T04:33:49.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.782+0000 7fda6be00640 1 -- 172.21.15.100:0/1315954302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda6c0ad9c0 msgr2=0x7fda6c0af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:49.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.782+0000 7fda6be00640 1 --2- 172.21.15.100:0/1315954302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda6c0ad9c0 0x7fda6c0af170 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fda6c0af6b0 tx=0x7fda5c02a4d0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.782+0000 7fda6be00640 1 -- 172.21.15.100:0/1315954302 shutdown_connections 2024-09-17T04:33:49.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.782+0000 7fda6be00640 1 --2- 172.21.15.100:0/1315954302 >> v2:172.21.15.184:3300/0 conn(0x7fda6c15a0f0 0x7fda6c15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.782+0000 7fda6be00640 1 --2- 172.21.15.100:0/1315954302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda6c0ad9c0 0x7fda6c0af170 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.782+0000 7fda6be00640 1 -- 172.21.15.100:0/1315954302 >> 172.21.15.100:0/1315954302 conn(0x7fda6c0b1e40 msgr2=0x7fda6c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:49.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.782+0000 7fda6be00640 1 -- 172.21.15.100:0/1315954302 shutdown_connections 2024-09-17T04:33:49.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.782+0000 7fda6be00640 1 -- 172.21.15.100:0/1315954302 wait complete. 2024-09-17T04:33:49.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.784+0000 7fda6be00640 1 Processor -- start 2024-09-17T04:33:49.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.784+0000 7fda6be00640 1 -- start start 2024-09-17T04:33:49.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.786+0000 7fda6be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda6c0ad9c0 0x7fda6c14d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:49.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.786+0000 7fda6be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda6c0b2f60 0x7fda6c09c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:49.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.786+0000 7fda6be00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda6c15a0f0 0x7fda6c0cec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:49.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.786+0000 7fda6be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c06f290 con 0x7fda6c0b2f60 2024-09-17T04:33:49.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.786+0000 7fda6be00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c15a5e0 con 0x7fda6c0ad9c0 2024-09-17T04:33:49.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.786+0000 7fda6be00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fda6c06f410 con 0x7fda6c15a0f0 2024-09-17T04:33:49.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.786+0000 7fda6a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda6c0b2f60 0x7fda6c09c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:49.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.786+0000 7fda6a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda6c0b2f60 0x7fda6c09c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39410/0 (socket says 172.21.15.100:39410) 2024-09-17T04:33:49.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.786+0000 7fda6a000640 1 -- 172.21.15.100:0/2086634615 learned_addr learned my addr 172.21.15.100:0/2086634615 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:49.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.786+0000 7fda6aa00640 1 --2- 172.21.15.100:0/2086634615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda6c0ad9c0 0x7fda6c14d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:49.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.786+0000 7fda6b400640 1 --2- 172.21.15.100:0/2086634615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda6c15a0f0 0x7fda6c0cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:49.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.786+0000 7fda6a000640 1 -- 172.21.15.100:0/2086634615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda6c15a0f0 msgr2=0x7fda6c0cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:49.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.786+0000 7fda6a000640 1 --2- 172.21.15.100:0/2086634615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda6c15a0f0 0x7fda6c0cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.786+0000 7fda6a000640 1 -- 172.21.15.100:0/2086634615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda6c0ad9c0 msgr2=0x7fda6c14d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:49.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.786+0000 7fda6a000640 1 --2- 172.21.15.100:0/2086634615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda6c0ad9c0 0x7fda6c14d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.786+0000 7fda6a000640 1 -- 172.21.15.100:0/2086634615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda6c1f69c0 con 0x7fda6c0b2f60 2024-09-17T04:33:49.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.786+0000 7fda6a000640 1 --2- 172.21.15.100:0/2086634615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda6c0b2f60 0x7fda6c09c090 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fda5800c3a0 tx=0x7fda5800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:49.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.787+0000 7fda62a00640 1 -- 172.21.15.100:0/2086634615 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda58011790 con 0x7fda6c0b2f60 2024-09-17T04:33:49.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.787+0000 7fda62a00640 1 -- 172.21.15.100:0/2086634615 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda58017070 con 0x7fda6c0b2f60 2024-09-17T04:33:49.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.787+0000 7fda6be00640 1 -- 172.21.15.100:0/2086634615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda6c1f4250 con 0x7fda6c0b2f60 2024-09-17T04:33:49.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.787+0000 7fda62a00640 1 -- 172.21.15.100:0/2086634615 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda58011cc0 con 0x7fda6c0b2f60 2024-09-17T04:33:49.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.787+0000 7fda6be00640 1 -- 172.21.15.100:0/2086634615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda6c1f6f40 con 0x7fda6c0b2f60 2024-09-17T04:33:49.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.788+0000 7fda6be00640 1 -- 172.21.15.100:0/2086634615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda6c1f3fc0 con 0x7fda6c0b2f60 2024-09-17T04:33:49.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.789+0000 7fda62a00640 1 -- 172.21.15.100:0/2086634615 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7fda5801c460 con 0x7fda6c0b2f60 2024-09-17T04:33:49.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.790+0000 7fda62a00640 1 --2- 172.21.15.100:0/2086634615 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fda4c0795b0 0x7fda4c07bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:49.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.790+0000 7fda62a00640 1 -- 172.21.15.100:0/2086634615 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4913+0+0 (secure 0 0 0) 0x7fda58099690 con 0x7fda6c0b2f60 2024-09-17T04:33:49.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.790+0000 7fda6aa00640 1 --2- 172.21.15.100:0/2086634615 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fda4c0795b0 0x7fda4c07bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:49.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.791+0000 7fda6aa00640 1 --2- 172.21.15.100:0/2086634615 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fda4c0795b0 0x7fda4c07bac0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fda5c034000 tx=0x7fda5c004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:49.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.794+0000 7fda62a00640 1 -- 172.21.15.100:0/2086634615 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fda58063300 con 0x7fda6c0b2f60 2024-09-17T04:33:49.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.980+0000 7fda6be00640 1 -- 172.21.15.100:0/2086634615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fda6c06f950 con 0x7fda6c0b2f60 2024-09-17T04:33:49.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.980+0000 7fda62a00640 1 -- 172.21.15.100:0/2086634615 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v163) ==== 84+0+65 (secure 0 0 0) 0x7fda58062c00 con 0x7fda6c0b2f60 2024-09-17T04:33:49.981 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:49.981 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-09-17T04:33:49.981 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:33:49.981 INFO:teuthology.orchestra.run.smithi100.stdout: "poolnum": 1, 2024-09-17T04:33:49.981 INFO:teuthology.orchestra.run.smithi100.stdout: "poolname": ".mgr" 2024-09-17T04:33:49.981 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:33:49.981 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-09-17T04:33:49.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.982+0000 7fda6be00640 1 -- 172.21.15.100:0/2086634615 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fda4c0795b0 msgr2=0x7fda4c07bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:49.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.982+0000 7fda6be00640 1 --2- 172.21.15.100:0/2086634615 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fda4c0795b0 0x7fda4c07bac0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fda5c034000 tx=0x7fda5c004730 comp rx=0 tx=0).stop 2024-09-17T04:33:49.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.982+0000 7fda6be00640 1 -- 172.21.15.100:0/2086634615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda6c0b2f60 msgr2=0x7fda6c09c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:49.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.982+0000 7fda6be00640 1 --2- 172.21.15.100:0/2086634615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda6c0b2f60 0x7fda6c09c090 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fda5800c3a0 tx=0x7fda5800c860 comp rx=0 tx=0).stop 2024-09-17T04:33:49.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.982+0000 7fda6be00640 1 -- 172.21.15.100:0/2086634615 shutdown_connections 2024-09-17T04:33:49.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.982+0000 7fda6be00640 1 --2- 172.21.15.100:0/2086634615 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fda4c0795b0 0x7fda4c07bac0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.982+0000 7fda6be00640 1 --2- 172.21.15.100:0/2086634615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda6c15a0f0 0x7fda6c0cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.982+0000 7fda6be00640 1 --2- 172.21.15.100:0/2086634615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda6c0b2f60 0x7fda6c09c090 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.982+0000 7fda6be00640 1 --2- 172.21.15.100:0/2086634615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda6c0ad9c0 0x7fda6c14d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.983+0000 7fda6be00640 1 -- 172.21.15.100:0/2086634615 >> 172.21.15.100:0/2086634615 conn(0x7fda6c0b1e40 msgr2=0x7fda6c0b4080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:49.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.983+0000 7fda6be00640 1 -- 172.21.15.100:0/2086634615 shutdown_connections 2024-09-17T04:33:49.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:49.983+0000 7fda6be00640 1 -- 172.21.15.100:0/2086634615 wait complete. 2024-09-17T04:33:50.009 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T04:33:50.009 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T04:33:50.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.235+0000 7f43f6200640 1 -- 172.21.15.100:0/456681674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43f015fd30 msgr2=0x7f43f00b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:50.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.235+0000 7f43f6200640 1 --2- 172.21.15.100:0/456681674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43f015fd30 0x7f43f00b4620 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f43e0005fd0 tx=0x7f43e002a5d0 comp rx=0 tx=0).stop 2024-09-17T04:33:50.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.235+0000 7f43f6200640 1 -- 172.21.15.100:0/456681674 shutdown_connections 2024-09-17T04:33:50.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.235+0000 7f43f6200640 1 --2- 172.21.15.100:0/456681674 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43f0162ff0 0x7f43f0158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:50.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.235+0000 7f43f6200640 1 --2- 172.21.15.100:0/456681674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43f015fd30 0x7f43f00b4620 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:50.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.235+0000 7f43f6200640 1 -- 172.21.15.100:0/456681674 >> 172.21.15.100:0/456681674 conn(0x7f43f00b2540 msgr2=0x7f43f00d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:50.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.235+0000 7f43f6200640 1 -- 172.21.15.100:0/456681674 shutdown_connections 2024-09-17T04:33:50.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.235+0000 7f43f6200640 1 -- 172.21.15.100:0/456681674 wait complete. 2024-09-17T04:33:50.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.237+0000 7f43f6200640 1 Processor -- start 2024-09-17T04:33:50.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.237+0000 7f43f6200640 1 -- start start 2024-09-17T04:33:50.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.238+0000 7f43f6200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43f006cda0 0x7f43f0074c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:50.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.238+0000 7f43f6200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43f015fd30 0x7f43f0157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:50.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.238+0000 7f43f6200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43f0162ff0 0x7f43f00cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:50.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.238+0000 7f43f6200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f43f00af490 con 0x7f43f015fd30 2024-09-17T04:33:50.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.238+0000 7f43f6200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f43f00af310 con 0x7f43f0162ff0 2024-09-17T04:33:50.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.238+0000 7f43f6200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f43f00af610 con 0x7f43f006cda0 2024-09-17T04:33:50.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.239+0000 7f43eea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43f015fd30 0x7f43f0157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:50.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.239+0000 7f43eea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43f015fd30 0x7f43f0157430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39428/0 (socket says 172.21.15.100:39428) 2024-09-17T04:33:50.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.239+0000 7f43eea00640 1 -- 172.21.15.100:0/1385793794 learned_addr learned my addr 172.21.15.100:0/1385793794 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:50.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.239+0000 7f43ef400640 1 --2- 172.21.15.100:0/1385793794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43f006cda0 0x7f43f0074c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:50.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.239+0000 7f43efe00640 1 --2- 172.21.15.100:0/1385793794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43f0162ff0 0x7f43f00cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:50.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.239+0000 7f43eea00640 1 -- 172.21.15.100:0/1385793794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43f006cda0 msgr2=0x7f43f0074c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:50.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.239+0000 7f43eea00640 1 --2- 172.21.15.100:0/1385793794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43f006cda0 0x7f43f0074c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:50.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.239+0000 7f43eea00640 1 -- 172.21.15.100:0/1385793794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43f0162ff0 msgr2=0x7f43f00cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:50.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.239+0000 7f43eea00640 1 --2- 172.21.15.100:0/1385793794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43f0162ff0 0x7f43f00cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:50.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.239+0000 7f43eea00640 1 -- 172.21.15.100:0/1385793794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f015ad20 con 0x7f43f015fd30 2024-09-17T04:33:50.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.240+0000 7f43eea00640 1 --2- 172.21.15.100:0/1385793794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43f015fd30 0x7f43f0157430 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00d6d0 tx=0x7f43dc00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:50.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.240+0000 7f43e7400640 1 -- 172.21.15.100:0/1385793794 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc017070 con 0x7f43f015fd30 2024-09-17T04:33:50.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.240+0000 7f43e7400640 1 -- 172.21.15.100:0/1385793794 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43dc009b10 con 0x7f43f015fd30 2024-09-17T04:33:50.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.240+0000 7f43e7400640 1 -- 172.21.15.100:0/1385793794 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc010460 con 0x7f43f015fd30 2024-09-17T04:33:50.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.240+0000 7f43f6200640 1 -- 172.21.15.100:0/1385793794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f00aff40 con 0x7f43f015fd30 2024-09-17T04:33:50.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.240+0000 7f43f6200640 1 -- 172.21.15.100:0/1385793794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f00b0500 con 0x7f43f015fd30 2024-09-17T04:33:50.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.241+0000 7f43f6200640 1 -- 172.21.15.100:0/1385793794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43f00d07a0 con 0x7f43f015fd30 2024-09-17T04:33:50.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.242+0000 7f43e7400640 1 -- 172.21.15.100:0/1385793794 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f43dc01a070 con 0x7f43f015fd30 2024-09-17T04:33:50.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.243+0000 7f43e7400640 1 --2- 172.21.15.100:0/1385793794 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f43d00795b0 0x7f43d007bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:50.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.244+0000 7f43e7400640 1 -- 172.21.15.100:0/1385793794 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4913+0+0 (secure 0 0 0) 0x7f43dc0996e0 con 0x7f43f015fd30 2024-09-17T04:33:50.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.244+0000 7f43ef400640 1 --2- 172.21.15.100:0/1385793794 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f43d00795b0 0x7f43d007bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:50.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.244+0000 7f43ef400640 1 --2- 172.21.15.100:0/1385793794 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f43d00795b0 0x7f43d007bac0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f43e0033000 tx=0x7f43e0004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:50.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.248+0000 7f43e7400640 1 -- 172.21.15.100:0/1385793794 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f43dc062f20 con 0x7f43f015fd30 2024-09-17T04:33:50.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:50.434+0000 7f43f6200640 1 -- 172.21.15.100:0/1385793794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f43f015ec00 con 0x7f43f015fd30 2024-09-17T04:33:51.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.042+0000 7f43e7400640 1 -- 172.21.15.100:0/1385793794 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v164) ==== 138+0+0 (secure 0 0 0) 0x7f43dc09c0c0 con 0x7f43f015fd30 2024-09-17T04:33:51.042 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' created 2024-09-17T04:33:51.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.045+0000 7f43f6200640 1 -- 172.21.15.100:0/1385793794 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f43d00795b0 msgr2=0x7f43d007bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:51.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.045+0000 7f43f6200640 1 --2- 172.21.15.100:0/1385793794 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f43d00795b0 0x7f43d007bac0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f43e0033000 tx=0x7f43e0004730 comp rx=0 tx=0).stop 2024-09-17T04:33:51.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.045+0000 7f43f6200640 1 -- 172.21.15.100:0/1385793794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43f015fd30 msgr2=0x7f43f0157430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:51.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.045+0000 7f43f6200640 1 --2- 172.21.15.100:0/1385793794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43f015fd30 0x7f43f0157430 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00d6d0 tx=0x7f43dc00db90 comp rx=0 tx=0).stop 2024-09-17T04:33:51.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.045+0000 7f43f6200640 1 -- 172.21.15.100:0/1385793794 shutdown_connections 2024-09-17T04:33:51.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.045+0000 7f43f6200640 1 --2- 172.21.15.100:0/1385793794 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f43d00795b0 0x7f43d007bac0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:51.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.045+0000 7f43f6200640 1 --2- 172.21.15.100:0/1385793794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43f0162ff0 0x7f43f00cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:51.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.045+0000 7f43f6200640 1 --2- 172.21.15.100:0/1385793794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43f015fd30 0x7f43f0157430 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:51.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.045+0000 7f43f6200640 1 --2- 172.21.15.100:0/1385793794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43f006cda0 0x7f43f0074c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:51.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.045+0000 7f43f6200640 1 -- 172.21.15.100:0/1385793794 >> 172.21.15.100:0/1385793794 conn(0x7f43f00b2540 msgr2=0x7f43f00ae540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:51.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.045+0000 7f43f6200640 1 -- 172.21.15.100:0/1385793794 shutdown_connections 2024-09-17T04:33:51.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.045+0000 7f43f6200640 1 -- 172.21.15.100:0/1385793794 wait complete. 2024-09-17T04:33:51.074 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T04:33:51.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.301+0000 7f2223200640 1 -- 172.21.15.100:0/3806157328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2224160780 msgr2=0x7f22240cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:51.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.301+0000 7f2223200640 1 --2- 172.21.15.100:0/3806157328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2224160780 0x7f22240cd070 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f221c005fd0 tx=0x7f221c02aa10 comp rx=0 tx=0).stop 2024-09-17T04:33:51.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.301+0000 7f2223200640 1 -- 172.21.15.100:0/3806157328 shutdown_connections 2024-09-17T04:33:51.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.301+0000 7f2223200640 1 --2- 172.21.15.100:0/3806157328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2224160780 0x7f22240cd070 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:51.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.301+0000 7f2223200640 1 --2- 172.21.15.100:0/3806157328 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2224160370 0x7f22240ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:51.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.301+0000 7f2223200640 1 -- 172.21.15.100:0/3806157328 >> 172.21.15.100:0/3806157328 conn(0x7f22240b1dd0 msgr2=0x7f22240b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:51.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.302+0000 7f2223200640 1 -- 172.21.15.100:0/3806157328 shutdown_connections 2024-09-17T04:33:51.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.302+0000 7f2223200640 1 -- 172.21.15.100:0/3806157328 wait complete. 2024-09-17T04:33:51.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.304+0000 7f2223200640 1 Processor -- start 2024-09-17T04:33:51.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.304+0000 7f2223200640 1 -- start start 2024-09-17T04:33:51.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.305+0000 7f2223200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22240af4e0 0x7f222413fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.305+0000 7f2223200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2224160370 0x7f222415e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.306+0000 7f2223200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2224160780 0x7f222415ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.306+0000 7f2223200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f22240b4350 con 0x7f2224160370 2024-09-17T04:33:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.306+0000 7f2223200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f22240b41d0 con 0x7f2224160780 2024-09-17T04:33:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.306+0000 7f2223200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f22240b44d0 con 0x7f22240af4e0 2024-09-17T04:33:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.306+0000 7f2221400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2224160370 0x7f222415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.306+0000 7f2221400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2224160370 0x7f222415e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39454/0 (socket says 172.21.15.100:39454) 2024-09-17T04:33:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.306+0000 7f2221400640 1 -- 172.21.15.100:0/1893244779 learned_addr learned my addr 172.21.15.100:0/1893244779 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.306+0000 7f2222800640 1 --2- 172.21.15.100:0/1893244779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2224160780 0x7f222415ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.306+0000 7f2221e00640 1 --2- 172.21.15.100:0/1893244779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22240af4e0 0x7f222413fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.306+0000 7f2221400640 1 -- 172.21.15.100:0/1893244779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22240af4e0 msgr2=0x7f222413fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.306+0000 7f2221400640 1 --2- 172.21.15.100:0/1893244779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22240af4e0 0x7f222413fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:51.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.306+0000 7f2221400640 1 -- 172.21.15.100:0/1893244779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2224160780 msgr2=0x7f222415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:51.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.306+0000 7f2221400640 1 --2- 172.21.15.100:0/1893244779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2224160780 0x7f222415ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:51.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.306+0000 7f2221400640 1 -- 172.21.15.100:0/1893244779 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2224140660 con 0x7f2224160370 2024-09-17T04:33:51.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.307+0000 7f2221400640 1 --2- 172.21.15.100:0/1893244779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2224160370 0x7f222415e750 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f2210009fd0 tx=0x7f221000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:51.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.307+0000 7f221a000640 1 -- 172.21.15.100:0/1893244779 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221000ccc0 con 0x7f2224160370 2024-09-17T04:33:51.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.307+0000 7f221a000640 1 -- 172.21.15.100:0/1893244779 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f221000ee70 con 0x7f2224160370 2024-09-17T04:33:51.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.307+0000 7f2223200640 1 -- 172.21.15.100:0/1893244779 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2224140940 con 0x7f2224160370 2024-09-17T04:33:51.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.307+0000 7f221a000640 1 -- 172.21.15.100:0/1893244779 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221000ce60 con 0x7f2224160370 2024-09-17T04:33:51.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.307+0000 7f2223200640 1 -- 172.21.15.100:0/1893244779 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2224140f00 con 0x7f2224160370 2024-09-17T04:33:51.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.308+0000 7f2223200640 1 -- 172.21.15.100:0/1893244779 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22240b49b0 con 0x7f2224160370 2024-09-17T04:33:51.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.310+0000 7f221a000640 1 -- 172.21.15.100:0/1893244779 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f2210005260 con 0x7f2224160370 2024-09-17T04:33:51.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.311+0000 7f221a000640 1 --2- 172.21.15.100:0/1893244779 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f22040795e0 0x7f220407baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:51.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.311+0000 7f221a000640 1 -- 172.21.15.100:0/1893244779 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 5283+0+0 (secure 0 0 0) 0x7f221009a530 con 0x7f2224160370 2024-09-17T04:33:51.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.311+0000 7f2221e00640 1 --2- 172.21.15.100:0/1893244779 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f22040795e0 0x7f220407baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:51.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.312+0000 7f2221e00640 1 --2- 172.21.15.100:0/1893244779 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f22040795e0 0x7f220407baf0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2214001720 tx=0x7f2214009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:51.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.314+0000 7f221a000640 1 -- 172.21.15.100:0/1893244779 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f2210063be0 con 0x7f2224160370 2024-09-17T04:33:51.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:51.497+0000 7f2223200640 1 -- 172.21.15.100:0/1893244779 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2224156670 con 0x7f2224160370 2024-09-17T04:33:52.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.048+0000 7f221a000640 1 -- 172.21.15.100:0/1893244779 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v165) ==== 170+0+0 (secure 0 0 0) 0x7f2210068610 con 0x7f2224160370 2024-09-17T04:33:52.049 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' created 2024-09-17T04:33:52.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.052+0000 7f2223200640 1 -- 172.21.15.100:0/1893244779 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f22040795e0 msgr2=0x7f220407baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:52.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.052+0000 7f2223200640 1 --2- 172.21.15.100:0/1893244779 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f22040795e0 0x7f220407baf0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2214001720 tx=0x7f2214009040 comp rx=0 tx=0).stop 2024-09-17T04:33:52.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.052+0000 7f2223200640 1 -- 172.21.15.100:0/1893244779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2224160370 msgr2=0x7f222415e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:52.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.052+0000 7f2223200640 1 --2- 172.21.15.100:0/1893244779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2224160370 0x7f222415e750 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f2210009fd0 tx=0x7f221000ec20 comp rx=0 tx=0).stop 2024-09-17T04:33:52.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.052+0000 7f2223200640 1 -- 172.21.15.100:0/1893244779 shutdown_connections 2024-09-17T04:33:52.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.052+0000 7f2223200640 1 --2- 172.21.15.100:0/1893244779 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f22040795e0 0x7f220407baf0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:52.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.052+0000 7f2223200640 1 --2- 172.21.15.100:0/1893244779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2224160780 0x7f222415ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:52.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.052+0000 7f2223200640 1 --2- 172.21.15.100:0/1893244779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2224160370 0x7f222415e750 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:52.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.052+0000 7f2223200640 1 --2- 172.21.15.100:0/1893244779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22240af4e0 0x7f222413fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:52.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.052+0000 7f2223200640 1 -- 172.21.15.100:0/1893244779 >> 172.21.15.100:0/1893244779 conn(0x7f22240b1dd0 msgr2=0x7f222406ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:52.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.052+0000 7f2223200640 1 -- 172.21.15.100:0/1893244779 shutdown_connections 2024-09-17T04:33:52.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.052+0000 7f2223200640 1 -- 172.21.15.100:0/1893244779 wait complete. 2024-09-17T04:33:52.097 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T04:33:52.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.321+0000 7fac7de00640 1 -- 172.21.15.100:0/1578302892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac780742b0 msgr2=0x7fac780b42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:52.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.321+0000 7fac7de00640 1 --2- 172.21.15.100:0/1578302892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac780742b0 0x7fac780b42a0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fac780b52e0 tx=0x7fac6802a4d0 comp rx=0 tx=0).stop 2024-09-17T04:33:52.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.321+0000 7fac7de00640 1 -- 172.21.15.100:0/1578302892 shutdown_connections 2024-09-17T04:33:52.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.321+0000 7fac7de00640 1 --2- 172.21.15.100:0/1578302892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac7815fbe0 0x7fac780cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:52.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.321+0000 7fac7de00640 1 --2- 172.21.15.100:0/1578302892 >> v2:172.21.15.184:3300/0 conn(0x7fac781624b0 0x7fac7815f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:52.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.321+0000 7fac7de00640 1 --2- 172.21.15.100:0/1578302892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac780742b0 0x7fac780b42a0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:52.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.321+0000 7fac7de00640 1 -- 172.21.15.100:0/1578302892 >> 172.21.15.100:0/1578302892 conn(0x7fac7806d620 msgr2=0x7fac780b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:52.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.321+0000 7fac7de00640 1 -- 172.21.15.100:0/1578302892 shutdown_connections 2024-09-17T04:33:52.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.321+0000 7fac7de00640 1 -- 172.21.15.100:0/1578302892 wait complete. 2024-09-17T04:33:52.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.324+0000 7fac7de00640 1 Processor -- start 2024-09-17T04:33:52.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.324+0000 7fac7de00640 1 -- start start 2024-09-17T04:33:52.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.325+0000 7fac7de00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac780742b0 0x7fac781f9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:52.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.325+0000 7fac7de00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac7815fbe0 0x7fac78145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:52.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.325+0000 7fac7de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac781624b0 0x7fac780ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:52.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.325+0000 7fac7de00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fac78072eb0 con 0x7fac781624b0 2024-09-17T04:33:52.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.325+0000 7fac7de00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fac780cfb60 con 0x7fac780742b0 2024-09-17T04:33:52.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.325+0000 7fac7de00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fac78073030 con 0x7fac7815fbe0 2024-09-17T04:33:52.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.325+0000 7fac7d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac781624b0 0x7fac780ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:52.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.326+0000 7fac7d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac781624b0 0x7fac780ae230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39472/0 (socket says 172.21.15.100:39472) 2024-09-17T04:33:52.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.326+0000 7fac7d400640 1 -- 172.21.15.100:0/1020929637 learned_addr learned my addr 172.21.15.100:0/1020929637 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:52.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.326+0000 7fac7ca00640 1 --2- 172.21.15.100:0/1020929637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac780742b0 0x7fac781f9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:52.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.326+0000 7fac6fe00640 1 --2- 172.21.15.100:0/1020929637 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac7815fbe0 0x7fac78145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:52.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.326+0000 7fac7d400640 1 -- 172.21.15.100:0/1020929637 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac7815fbe0 msgr2=0x7fac78145ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:52.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.326+0000 7fac7d400640 1 --2- 172.21.15.100:0/1020929637 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac7815fbe0 0x7fac78145ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:52.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.326+0000 7fac7d400640 1 -- 172.21.15.100:0/1020929637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac780742b0 msgr2=0x7fac781f9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:52.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.326+0000 7fac7d400640 1 --2- 172.21.15.100:0/1020929637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac780742b0 0x7fac781f9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:52.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.326+0000 7fac7d400640 1 -- 172.21.15.100:0/1020929637 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac78149020 con 0x7fac781624b0 2024-09-17T04:33:52.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.326+0000 7fac7d400640 1 --2- 172.21.15.100:0/1020929637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac781624b0 0x7fac780ae230 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fac7000e9c0 tx=0x7fac7000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:52.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.327+0000 7fac75600640 1 -- 172.21.15.100:0/1020929637 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70004d20 con 0x7fac781624b0 2024-09-17T04:33:52.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.327+0000 7fac75600640 1 -- 172.21.15.100:0/1020929637 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac7000cac0 con 0x7fac781624b0 2024-09-17T04:33:52.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.327+0000 7fac75600640 1 -- 172.21.15.100:0/1020929637 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70005020 con 0x7fac781624b0 2024-09-17T04:33:52.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.327+0000 7fac7de00640 1 -- 172.21.15.100:0/1020929637 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac78149300 con 0x7fac781624b0 2024-09-17T04:33:52.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.327+0000 7fac7de00640 1 -- 172.21.15.100:0/1020929637 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac78147f60 con 0x7fac781624b0 2024-09-17T04:33:52.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.328+0000 7fac7de00640 1 -- 172.21.15.100:0/1020929637 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac78148d90 con 0x7fac781624b0 2024-09-17T04:33:52.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.329+0000 7fac75600640 1 -- 172.21.15.100:0/1020929637 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7fac700051c0 con 0x7fac781624b0 2024-09-17T04:33:52.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.330+0000 7fac75600640 1 --2- 172.21.15.100:0/1020929637 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fac540795e0 0x7fac5407baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:52.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.330+0000 7fac75600640 1 -- 172.21.15.100:0/1020929637 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5665+0+0 (secure 0 0 0) 0x7fac7009a420 con 0x7fac781624b0 2024-09-17T04:33:52.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.330+0000 7fac7ca00640 1 --2- 172.21.15.100:0/1020929637 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fac540795e0 0x7fac5407baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:52.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.331+0000 7fac7ca00640 1 --2- 172.21.15.100:0/1020929637 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fac540795e0 0x7fac5407baf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fac68033000 tx=0x7fac68004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:52.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.334+0000 7fac75600640 1 -- 172.21.15.100:0/1020929637 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fac70016030 con 0x7fac781624b0 2024-09-17T04:33:52.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:52.554+0000 7fac7de00640 1 -- 172.21.15.100:0/1020929637 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fac781481c0 con 0x7fac781624b0 2024-09-17T04:33:53.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.050+0000 7fac75600640 1 -- 172.21.15.100:0/1020929637 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 26 and data pool 27 v117) ==== 174+0+0 (secure 0 0 0) 0x7fac70063a40 con 0x7fac781624b0 2024-09-17T04:33:53.051 INFO:teuthology.orchestra.run.smithi100.stderr:new fs with metadata pool 26 and data pool 27 2024-09-17T04:33:53.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.056+0000 7fac7de00640 1 -- 172.21.15.100:0/1020929637 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fac540795e0 msgr2=0x7fac5407baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:53.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.056+0000 7fac7de00640 1 --2- 172.21.15.100:0/1020929637 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fac540795e0 0x7fac5407baf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fac68033000 tx=0x7fac68004730 comp rx=0 tx=0).stop 2024-09-17T04:33:53.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.056+0000 7fac7de00640 1 -- 172.21.15.100:0/1020929637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac781624b0 msgr2=0x7fac780ae230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:53.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.056+0000 7fac7de00640 1 --2- 172.21.15.100:0/1020929637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac781624b0 0x7fac780ae230 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fac7000e9c0 tx=0x7fac7000ee80 comp rx=0 tx=0).stop 2024-09-17T04:33:53.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.056+0000 7fac7de00640 1 -- 172.21.15.100:0/1020929637 shutdown_connections 2024-09-17T04:33:53.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.056+0000 7fac7de00640 1 --2- 172.21.15.100:0/1020929637 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fac540795e0 0x7fac5407baf0 secure :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fac68033000 tx=0x7fac68004730 comp rx=0 tx=0).stop 2024-09-17T04:33:53.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.056+0000 7fac7de00640 1 --2- 172.21.15.100:0/1020929637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac781624b0 0x7fac780ae230 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:53.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.056+0000 7fac7de00640 1 --2- 172.21.15.100:0/1020929637 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac7815fbe0 0x7fac78145ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:53.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.056+0000 7fac7de00640 1 --2- 172.21.15.100:0/1020929637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac780742b0 0x7fac781f9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:53.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.056+0000 7fac7de00640 1 -- 172.21.15.100:0/1020929637 >> 172.21.15.100:0/1020929637 conn(0x7fac7806d620 msgr2=0x7fac7815fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:53.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.057+0000 7fac7de00640 1 -- 172.21.15.100:0/1020929637 shutdown_connections 2024-09-17T04:33:53.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.057+0000 7fac7de00640 1 -- 172.21.15.100:0/1020929637 wait complete. 2024-09-17T04:33:53.102 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:33:53.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.322+0000 7ff993e00640 1 -- 172.21.15.100:0/4283727320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9940d2860 msgr2=0x7ff9940d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:53.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.322+0000 7ff993e00640 1 --2- 172.21.15.100:0/4283727320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9940d2860 0x7ff9940d2c70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff98c005fd0 tx=0x7ff98c02aa90 comp rx=0 tx=0).stop 2024-09-17T04:33:53.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.322+0000 7ff993e00640 1 -- 172.21.15.100:0/4283727320 shutdown_connections 2024-09-17T04:33:53.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.322+0000 7ff993e00640 1 --2- 172.21.15.100:0/4283727320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9940d2860 0x7ff9940d2c70 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:53.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.322+0000 7ff993e00640 1 --2- 172.21.15.100:0/4283727320 >> v2:172.21.15.184:3300/0 conn(0x7ff9940cf8c0 0x7ff9940d2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:53.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.322+0000 7ff993e00640 1 -- 172.21.15.100:0/4283727320 >> 172.21.15.100:0/4283727320 conn(0x7ff99409d8a0 msgr2=0x7ff99408bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:53.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.322+0000 7ff993e00640 1 -- 172.21.15.100:0/4283727320 shutdown_connections 2024-09-17T04:33:53.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.322+0000 7ff993e00640 1 -- 172.21.15.100:0/4283727320 wait complete. 2024-09-17T04:33:53.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.324+0000 7ff993e00640 1 Processor -- start 2024-09-17T04:33:53.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.325+0000 7ff993e00640 1 -- start start 2024-09-17T04:33:53.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.326+0000 7ff993e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9940b3820 0x7ff99415ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:53.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.326+0000 7ff993e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9940d2860 0x7ff994160660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:53.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.326+0000 7ff993e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff99415b650 0x7ff994160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:53.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.326+0000 7ff993e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff9940b4070 con 0x7ff9940d2860 2024-09-17T04:33:53.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.326+0000 7ff993e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff9940b3ef0 con 0x7ff9940b3820 2024-09-17T04:33:53.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.326+0000 7ff993e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff9940d31b0 con 0x7ff99415b650 2024-09-17T04:33:53.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.326+0000 7ff992000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9940d2860 0x7ff994160660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:53.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.327+0000 7ff992a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9940b3820 0x7ff99415ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:53.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.327+0000 7ff992000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9940d2860 0x7ff994160660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39480/0 (socket says 172.21.15.100:39480) 2024-09-17T04:33:53.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.327+0000 7ff992000640 1 -- 172.21.15.100:0/3773133790 learned_addr learned my addr 172.21.15.100:0/3773133790 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:53.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.327+0000 7ff993400640 1 --2- 172.21.15.100:0/3773133790 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff99415b650 0x7ff994160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:53.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.327+0000 7ff992000640 1 -- 172.21.15.100:0/3773133790 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff99415b650 msgr2=0x7ff994160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:53.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.327+0000 7ff992000640 1 --2- 172.21.15.100:0/3773133790 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff99415b650 0x7ff994160ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:53.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.327+0000 7ff992000640 1 -- 172.21.15.100:0/3773133790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9940b3820 msgr2=0x7ff99415ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:53.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.327+0000 7ff992000640 1 --2- 172.21.15.100:0/3773133790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9940b3820 0x7ff99415ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:53.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.327+0000 7ff992000640 1 -- 172.21.15.100:0/3773133790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9941e84a0 con 0x7ff9940d2860 2024-09-17T04:33:53.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.327+0000 7ff992000640 1 --2- 172.21.15.100:0/3773133790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9940d2860 0x7ff994160660 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7ff98000e9f0 tx=0x7ff98000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:53.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.328+0000 7ff98b400640 1 -- 172.21.15.100:0/3773133790 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98000cd80 con 0x7ff9940d2860 2024-09-17T04:33:53.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.328+0000 7ff98b400640 1 -- 172.21.15.100:0/3773133790 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff980004020 con 0x7ff9940d2860 2024-09-17T04:33:53.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.328+0000 7ff993e00640 1 -- 172.21.15.100:0/3773133790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9941e8740 con 0x7ff9940d2860 2024-09-17T04:33:53.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.328+0000 7ff98b400640 1 -- 172.21.15.100:0/3773133790 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff980004470 con 0x7ff9940d2860 2024-09-17T04:33:53.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.328+0000 7ff993e00640 1 -- 172.21.15.100:0/3773133790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9941e8d00 con 0x7ff9940d2860 2024-09-17T04:33:53.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.329+0000 7ff993e00640 1 -- 172.21.15.100:0/3773133790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff99406a3e0 con 0x7ff9940d2860 2024-09-17T04:33:53.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.330+0000 7ff98b400640 1 -- 172.21.15.100:0/3773133790 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7ff9800035f0 con 0x7ff9940d2860 2024-09-17T04:33:53.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.331+0000 7ff98b400640 1 --2- 172.21.15.100:0/3773133790 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff9740795b0 0x7ff97407bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:53.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.331+0000 7ff98b400640 1 -- 172.21.15.100:0/3773133790 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5765+0+0 (secure 0 0 0) 0x7ff98009a120 con 0x7ff9940d2860 2024-09-17T04:33:53.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.331+0000 7ff992a00640 1 --2- 172.21.15.100:0/3773133790 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff9740795b0 0x7ff97407bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:53.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.332+0000 7ff992a00640 1 --2- 172.21.15.100:0/3773133790 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff9740795b0 0x7ff97407bac0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff984007fd0 tx=0x7ff984009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:53.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.336+0000 7ff98b400640 1 -- 172.21.15.100:0/3773133790 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff99406a3e0 con 0x7ff9940d2860 2024-09-17T04:33:53.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.519+0000 7ff993e00640 1 -- 172.21.15.100:0/3773133790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9940d2320 con 0x7ff9940d2860 2024-09-17T04:33:53.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.520+0000 7ff98b400640 1 -- 172.21.15.100:0/3773133790 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) ==== 74+0+16893 (secure 0 0 0) 0x7ff9800636c0 con 0x7ff9940d2860 2024-09-17T04:33:53.521 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:53.522 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":166,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:33:52.555891+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:33:50.447902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:51.510699+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:33:53.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.523+0000 7ff993e00640 1 -- 172.21.15.100:0/3773133790 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff9740795b0 msgr2=0x7ff97407bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:53.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.523+0000 7ff993e00640 1 --2- 172.21.15.100:0/3773133790 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff9740795b0 0x7ff97407bac0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff984007fd0 tx=0x7ff984009040 comp rx=0 tx=0).stop 2024-09-17T04:33:53.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.523+0000 7ff993e00640 1 -- 172.21.15.100:0/3773133790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9940d2860 msgr2=0x7ff994160660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:53.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.523+0000 7ff993e00640 1 --2- 172.21.15.100:0/3773133790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9940d2860 0x7ff994160660 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7ff98000e9f0 tx=0x7ff98000eeb0 comp rx=0 tx=0).stop 2024-09-17T04:33:53.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.523+0000 7ff993e00640 1 -- 172.21.15.100:0/3773133790 shutdown_connections 2024-09-17T04:33:53.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.523+0000 7ff993e00640 1 --2- 172.21.15.100:0/3773133790 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff9740795b0 0x7ff97407bac0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:53.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.523+0000 7ff993e00640 1 --2- 172.21.15.100:0/3773133790 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff99415b650 0x7ff994160ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:53.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.523+0000 7ff993e00640 1 --2- 172.21.15.100:0/3773133790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9940d2860 0x7ff994160660 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:53.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.523+0000 7ff993e00640 1 --2- 172.21.15.100:0/3773133790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9940b3820 0x7ff99415ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:53.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.523+0000 7ff993e00640 1 -- 172.21.15.100:0/3773133790 >> 172.21.15.100:0/3773133790 conn(0x7ff99409d8a0 msgr2=0x7ff9941ea550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:53.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.523+0000 7ff993e00640 1 -- 172.21.15.100:0/3773133790 shutdown_connections 2024-09-17T04:33:53.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.523+0000 7ff993e00640 1 -- 172.21.15.100:0/3773133790 wait complete. 2024-09-17T04:33:53.546 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:33:53.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.764+0000 7f8f46c00640 1 -- 172.21.15.100:0/1180713049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f400ad0e0 msgr2=0x7f8f400ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:53.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.764+0000 7f8f46c00640 1 --2- 172.21.15.100:0/1180713049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f400ad0e0 0x7f8f400ad4f0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f8f28005fd0 tx=0x7f8f2802a540 comp rx=0 tx=0).stop 2024-09-17T04:33:53.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.764+0000 7f8f46c00640 1 -- 172.21.15.100:0/1180713049 shutdown_connections 2024-09-17T04:33:53.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.764+0000 7f8f46c00640 1 --2- 172.21.15.100:0/1180713049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f400ad0e0 0x7f8f400ad4f0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:53.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.764+0000 7f8f46c00640 1 --2- 172.21.15.100:0/1180713049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f4015d760 0x7f8f401535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:53.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.764+0000 7f8f46c00640 1 -- 172.21.15.100:0/1180713049 >> 172.21.15.100:0/1180713049 conn(0x7f8f4006d730 msgr2=0x7f8f400b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:53.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.764+0000 7f8f46c00640 1 -- 172.21.15.100:0/1180713049 shutdown_connections 2024-09-17T04:33:53.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.764+0000 7f8f46c00640 1 -- 172.21.15.100:0/1180713049 wait complete. 2024-09-17T04:33:53.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.766+0000 7f8f46c00640 1 Processor -- start 2024-09-17T04:33:53.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.766+0000 7f8f46c00640 1 -- start start 2024-09-17T04:33:53.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.768+0000 7f8f46c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f400ad0e0 0x7f8f401567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:53.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.768+0000 7f8f46c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f4015d760 0x7f8f4015baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:53.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.768+0000 7f8f46c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f40160310 0x7f8f4015bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:53.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.768+0000 7f8f46c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4006f220 con 0x7f8f40160310 2024-09-17T04:33:53.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.768+0000 7f8f46c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4006f0a0 con 0x7f8f400ad0e0 2024-09-17T04:33:53.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.768+0000 7f8f46c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8f400b4c20 con 0x7f8f4015d760 2024-09-17T04:33:53.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.768+0000 7f8f44c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f40160310 0x7f8f4015bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:53.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.768+0000 7f8f3fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f400ad0e0 0x7f8f401567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:53.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.768+0000 7f8f44c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f40160310 0x7f8f4015bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39500/0 (socket says 172.21.15.100:39500) 2024-09-17T04:33:53.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.768+0000 7f8f44c00640 1 -- 172.21.15.100:0/2634306852 learned_addr learned my addr 172.21.15.100:0/2634306852 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:53.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.768+0000 7f8f37400640 1 --2- 172.21.15.100:0/2634306852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f4015d760 0x7f8f4015baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:53.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.769+0000 7f8f44c00640 1 -- 172.21.15.100:0/2634306852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f4015d760 msgr2=0x7f8f4015baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:53.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.769+0000 7f8f44c00640 1 --2- 172.21.15.100:0/2634306852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f4015d760 0x7f8f4015baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:53.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.769+0000 7f8f44c00640 1 -- 172.21.15.100:0/2634306852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f400ad0e0 msgr2=0x7f8f401567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:53.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.769+0000 7f8f44c00640 1 --2- 172.21.15.100:0/2634306852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f400ad0e0 0x7f8f401567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:53.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.769+0000 7f8f44c00640 1 -- 172.21.15.100:0/2634306852 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f401f5680 con 0x7f8f40160310 2024-09-17T04:33:53.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.769+0000 7f8f44c00640 1 --2- 172.21.15.100:0/2634306852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f40160310 0x7f8f4015bfe0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f8f3800c760 tx=0x7f8f3800cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:53.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.769+0000 7f8f3cc00640 1 -- 172.21.15.100:0/2634306852 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38003b70 con 0x7f8f40160310 2024-09-17T04:33:53.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.769+0000 7f8f3cc00640 1 -- 172.21.15.100:0/2634306852 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f38003d10 con 0x7f8f40160310 2024-09-17T04:33:53.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.769+0000 7f8f46c00640 1 -- 172.21.15.100:0/2634306852 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f401f5940 con 0x7f8f40160310 2024-09-17T04:33:53.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.769+0000 7f8f3cc00640 1 -- 172.21.15.100:0/2634306852 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38005330 con 0x7f8f40160310 2024-09-17T04:33:53.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.769+0000 7f8f46c00640 1 -- 172.21.15.100:0/2634306852 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f401f5f00 con 0x7f8f40160310 2024-09-17T04:33:53.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.770+0000 7f8f46c00640 1 -- 172.21.15.100:0/2634306852 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f40160fa0 con 0x7f8f40160310 2024-09-17T04:33:53.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.772+0000 7f8f3cc00640 1 -- 172.21.15.100:0/2634306852 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f8f380054d0 con 0x7f8f40160310 2024-09-17T04:33:53.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.773+0000 7f8f3cc00640 1 --2- 172.21.15.100:0/2634306852 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8f1c0795b0 0x7f8f1c07bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:53.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.773+0000 7f8f3cc00640 1 -- 172.21.15.100:0/2634306852 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5765+0+0 (secure 0 0 0) 0x7f8f38099420 con 0x7f8f40160310 2024-09-17T04:33:53.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.774+0000 7f8f3fe00640 1 --2- 172.21.15.100:0/2634306852 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8f1c0795b0 0x7f8f1c07bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:53.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.774+0000 7f8f3fe00640 1 --2- 172.21.15.100:0/2634306852 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8f1c0795b0 0x7f8f1c07bac0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009b30 tx=0x7f8f30005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:53.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.778+0000 7f8f3cc00640 1 -- 172.21.15.100:0/2634306852 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8f400ada30 con 0x7f8f40160310 2024-09-17T04:33:53.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.962+0000 7f8f46c00640 1 -- 172.21.15.100:0/2634306852 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f401543b0 con 0x7f8f40160310 2024-09-17T04:33:53.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.963+0000 7f8f3cc00640 1 -- 172.21.15.100:0/2634306852 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) ==== 74+0+16893 (secure 0 0 0) 0x7f8f38062ad0 con 0x7f8f40160310 2024-09-17T04:33:53.964 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:53.964 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":166,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:33:52.555891+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:33:50.447902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:51.510699+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:33:53.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.966+0000 7f8f46c00640 1 -- 172.21.15.100:0/2634306852 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8f1c0795b0 msgr2=0x7f8f1c07bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:53.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.966+0000 7f8f46c00640 1 --2- 172.21.15.100:0/2634306852 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8f1c0795b0 0x7f8f1c07bac0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009b30 tx=0x7f8f30005f90 comp rx=0 tx=0).stop 2024-09-17T04:33:53.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.966+0000 7f8f46c00640 1 -- 172.21.15.100:0/2634306852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f40160310 msgr2=0x7f8f4015bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:53.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.966+0000 7f8f46c00640 1 --2- 172.21.15.100:0/2634306852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f40160310 0x7f8f4015bfe0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f8f3800c760 tx=0x7f8f3800cb60 comp rx=0 tx=0).stop 2024-09-17T04:33:53.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.966+0000 7f8f46c00640 1 -- 172.21.15.100:0/2634306852 shutdown_connections 2024-09-17T04:33:53.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.966+0000 7f8f46c00640 1 --2- 172.21.15.100:0/2634306852 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8f1c0795b0 0x7f8f1c07bac0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:53.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.966+0000 7f8f46c00640 1 --2- 172.21.15.100:0/2634306852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f40160310 0x7f8f4015bfe0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:53.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.966+0000 7f8f46c00640 1 --2- 172.21.15.100:0/2634306852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f4015d760 0x7f8f4015baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:53.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.966+0000 7f8f46c00640 1 --2- 172.21.15.100:0/2634306852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f400ad0e0 0x7f8f401567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:53.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.966+0000 7f8f46c00640 1 -- 172.21.15.100:0/2634306852 >> 172.21.15.100:0/2634306852 conn(0x7f8f4006d730 msgr2=0x7f8f400b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:53.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.966+0000 7f8f46c00640 1 -- 172.21.15.100:0/2634306852 shutdown_connections 2024-09-17T04:33:53.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:53.966+0000 7f8f46c00640 1 -- 172.21.15.100:0/2634306852 wait complete. 2024-09-17T04:33:53.993 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T04:33:54.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.218+0000 7f468d000640 1 -- 172.21.15.100:0/3991517264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4688073c50 msgr2=0x7f46880cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:54.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.218+0000 7f468d000640 1 --2- 172.21.15.100:0/3991517264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4688073c50 0x7f46880cd610 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f4680005fd0 tx=0x7f468002ab20 comp rx=0 tx=0).stop 2024-09-17T04:33:54.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.218+0000 7f468d000640 1 -- 172.21.15.100:0/3991517264 shutdown_connections 2024-09-17T04:33:54.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.218+0000 7f468d000640 1 --2- 172.21.15.100:0/3991517264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4688073c50 0x7f46880cd610 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:54.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.218+0000 7f468d000640 1 --2- 172.21.15.100:0/3991517264 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4688145550 0x7f468806fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:54.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.218+0000 7f468d000640 1 -- 172.21.15.100:0/3991517264 >> 172.21.15.100:0/3991517264 conn(0x7f468806d1c0 msgr2=0x7f468806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:54.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.218+0000 7f468d000640 1 -- 172.21.15.100:0/3991517264 shutdown_connections 2024-09-17T04:33:54.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.218+0000 7f468d000640 1 -- 172.21.15.100:0/3991517264 wait complete. 2024-09-17T04:33:54.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.220+0000 7f468d000640 1 Processor -- start 2024-09-17T04:33:54.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.221+0000 7f468d000640 1 -- start start 2024-09-17T04:33:54.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.222+0000 7f468d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4688145550 0x7f46880b3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:54.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.222+0000 7f468d000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4688145a40 0x7f468813dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:54.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.222+0000 7f468d000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46880af250 0x7f46881f5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:54.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.222+0000 7f468d000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f46880ce190 con 0x7f4688145550 2024-09-17T04:33:54.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.222+0000 7f468d000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f46880ce010 con 0x7f46880af250 2024-09-17T04:33:54.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.222+0000 7f468d000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f46880ce310 con 0x7f4688145a40 2024-09-17T04:33:54.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.222+0000 7f4685e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4688145550 0x7f46880b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:54.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.222+0000 7f4685e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4688145550 0x7f46880b3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39522/0 (socket says 172.21.15.100:39522) 2024-09-17T04:33:54.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.222+0000 7f4685e00640 1 -- 172.21.15.100:0/4247030455 learned_addr learned my addr 172.21.15.100:0/4247030455 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:54.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.222+0000 7f467d400640 1 --2- 172.21.15.100:0/4247030455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4688145a40 0x7f468813dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:54.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.223+0000 7f4686800640 1 --2- 172.21.15.100:0/4247030455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46880af250 0x7f46881f5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:54.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.223+0000 7f4685e00640 1 -- 172.21.15.100:0/4247030455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4688145a40 msgr2=0x7f468813dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:54.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.223+0000 7f4685e00640 1 --2- 172.21.15.100:0/4247030455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4688145a40 0x7f468813dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:54.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.223+0000 7f4685e00640 1 -- 172.21.15.100:0/4247030455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46880af250 msgr2=0x7f46881f5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:54.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.223+0000 7f4685e00640 1 --2- 172.21.15.100:0/4247030455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46880af250 0x7f46881f5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:54.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.223+0000 7f4685e00640 1 -- 172.21.15.100:0/4247030455 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46880ae1b0 con 0x7f4688145550 2024-09-17T04:33:54.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.223+0000 7f4685e00640 1 --2- 172.21.15.100:0/4247030455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4688145550 0x7f46880b3580 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f467800d750 tx=0x7f467800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:54.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.223+0000 7f467ea00640 1 -- 172.21.15.100:0/4247030455 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4678017070 con 0x7f4688145550 2024-09-17T04:33:54.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.223+0000 7f467ea00640 1 -- 172.21.15.100:0/4247030455 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4678004310 con 0x7f4688145550 2024-09-17T04:33:54.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.223+0000 7f468d000640 1 -- 172.21.15.100:0/4247030455 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46880ae490 con 0x7f4688145550 2024-09-17T04:33:54.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.223+0000 7f467ea00640 1 -- 172.21.15.100:0/4247030455 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467800de60 con 0x7f4688145550 2024-09-17T04:33:54.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.224+0000 7f468d000640 1 -- 172.21.15.100:0/4247030455 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46880aea50 con 0x7f4688145550 2024-09-17T04:33:54.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.224+0000 7f468d000640 1 -- 172.21.15.100:0/4247030455 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46880ce870 con 0x7f4688145550 2024-09-17T04:33:54.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.226+0000 7f467ea00640 1 -- 172.21.15.100:0/4247030455 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f4678009c30 con 0x7f4688145550 2024-09-17T04:33:54.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.227+0000 7f467ea00640 1 --2- 172.21.15.100:0/4247030455 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f46640795e0 0x7f466407baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:54.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.227+0000 7f467ea00640 1 -- 172.21.15.100:0/4247030455 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7f467809c6c0 con 0x7f4688145550 2024-09-17T04:33:54.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.227+0000 7f467d400640 1 --2- 172.21.15.100:0/4247030455 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f46640795e0 0x7f466407baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:54.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.228+0000 7f467d400640 1 --2- 172.21.15.100:0/4247030455 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f46640795e0 0x7f466407baf0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f46881f8c60 tx=0x7f4670009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:54.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.231+0000 7f467ea00640 1 -- 172.21.15.100:0/4247030455 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4678020020 con 0x7f4688145550 2024-09-17T04:33:54.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:54.451+0000 7f468d000640 1 -- 172.21.15.100:0/4247030455 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f468806fe60 con 0x7f4688145550 2024-09-17T04:33:55.033 INFO:tasks.ceph.mds.c.smithi100.stderr:2024-09-17T04:33:55.033+0000 7fcaae200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T04:33:55.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.083+0000 7f467ea00640 1 -- 172.21.15.100:0/4247030455 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v120) ==== 118+0+0 (secure 0 0 0) 0x7f4678063130 con 0x7f4688145550 2024-09-17T04:33:55.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.085+0000 7f468d000640 1 -- 172.21.15.100:0/4247030455 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f46640795e0 msgr2=0x7f466407baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.085+0000 7f468d000640 1 --2- 172.21.15.100:0/4247030455 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f46640795e0 0x7f466407baf0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f46881f8c60 tx=0x7f4670009040 comp rx=0 tx=0).stop 2024-09-17T04:33:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.085+0000 7f468d000640 1 -- 172.21.15.100:0/4247030455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4688145550 msgr2=0x7f46880b3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.085+0000 7f468d000640 1 --2- 172.21.15.100:0/4247030455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4688145550 0x7f46880b3580 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f467800d750 tx=0x7f467800dc10 comp rx=0 tx=0).stop 2024-09-17T04:33:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.086+0000 7f468d000640 1 -- 172.21.15.100:0/4247030455 shutdown_connections 2024-09-17T04:33:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.086+0000 7f468d000640 1 --2- 172.21.15.100:0/4247030455 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f46640795e0 0x7f466407baf0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.086+0000 7f468d000640 1 --2- 172.21.15.100:0/4247030455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46880af250 0x7f46881f5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.086+0000 7f468d000640 1 --2- 172.21.15.100:0/4247030455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4688145a40 0x7f468813dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.086+0000 7f468d000640 1 --2- 172.21.15.100:0/4247030455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4688145550 0x7f46880b3580 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.086+0000 7f468d000640 1 -- 172.21.15.100:0/4247030455 >> 172.21.15.100:0/4247030455 conn(0x7f468806d1c0 msgr2=0x7f4688068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.086+0000 7f468d000640 1 -- 172.21.15.100:0/4247030455 shutdown_connections 2024-09-17T04:33:55.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.086+0000 7f468d000640 1 -- 172.21.15.100:0/4247030455 wait complete. 2024-09-17T04:33:55.114 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T04:33:55.114 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T04:33:55.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.336+0000 7fc18be00640 1 -- 172.21.15.100:0/1561073306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc18c0cd1d0 msgr2=0x7fc18c0af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:55.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.336+0000 7fc18be00640 1 --2- 172.21.15.100:0/1561073306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc18c0cd1d0 0x7fc18c0af0d0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fc184005fd0 tx=0x7fc18402aa90 comp rx=0 tx=0).stop 2024-09-17T04:33:55.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.336+0000 7fc18be00640 1 -- 172.21.15.100:0/1561073306 shutdown_connections 2024-09-17T04:33:55.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.336+0000 7fc18be00640 1 --2- 172.21.15.100:0/1561073306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc18c0cd1d0 0x7fc18c0af0d0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:55.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.336+0000 7fc18be00640 1 --2- 172.21.15.100:0/1561073306 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc18c0b3f30 0x7fc18c156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:55.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.336+0000 7fc18be00640 1 -- 172.21.15.100:0/1561073306 >> 172.21.15.100:0/1561073306 conn(0x7fc18c06d620 msgr2=0x7fc18c0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:55.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.336+0000 7fc18be00640 1 -- 172.21.15.100:0/1561073306 shutdown_connections 2024-09-17T04:33:55.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.337+0000 7fc18be00640 1 -- 172.21.15.100:0/1561073306 wait complete. 2024-09-17T04:33:55.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.339+0000 7fc18be00640 1 Processor -- start 2024-09-17T04:33:55.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.339+0000 7fc18be00640 1 -- start start 2024-09-17T04:33:55.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.340+0000 7fc18be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc18c0b3f30 0x7fc18c099650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:55.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.340+0000 7fc18be00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc18c0cd1d0 0x7fc18c155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:55.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.340+0000 7fc18be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc18c15d6e0 0x7fc18c1e9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:55.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.340+0000 7fc18be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c0af9a0 con 0x7fc18c15d6e0 2024-09-17T04:33:55.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.340+0000 7fc18be00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c0af820 con 0x7fc18c0b3f30 2024-09-17T04:33:55.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.340+0000 7fc18be00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc18c0afb20 con 0x7fc18c0cd1d0 2024-09-17T04:33:55.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.340+0000 7fc18b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc18c15d6e0 0x7fc18c1e9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:55.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.340+0000 7fc18b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc18c15d6e0 0x7fc18c1e9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37790/0 (socket says 172.21.15.100:37790) 2024-09-17T04:33:55.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.340+0000 7fc18b400640 1 -- 172.21.15.100:0/3811917229 learned_addr learned my addr 172.21.15.100:0/3811917229 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:55.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.341+0000 7fc18aa00640 1 --2- 172.21.15.100:0/3811917229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc18c0b3f30 0x7fc18c099650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:55.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.341+0000 7fc18b400640 1 -- 172.21.15.100:0/3811917229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc18c0cd1d0 msgr2=0x7fc18c155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:55.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.341+0000 7fc18a000640 1 --2- 172.21.15.100:0/3811917229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc18c0cd1d0 0x7fc18c155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:55.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.341+0000 7fc18b400640 1 --2- 172.21.15.100:0/3811917229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc18c0cd1d0 0x7fc18c155400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:55.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.341+0000 7fc18b400640 1 -- 172.21.15.100:0/3811917229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc18c0b3f30 msgr2=0x7fc18c099650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:55.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.341+0000 7fc18b400640 1 --2- 172.21.15.100:0/3811917229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc18c0b3f30 0x7fc18c099650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:55.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.341+0000 7fc18b400640 1 -- 172.21.15.100:0/3811917229 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc18c1f6080 con 0x7fc18c15d6e0 2024-09-17T04:33:55.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.341+0000 7fc18aa00640 1 --2- 172.21.15.100:0/3811917229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc18c0b3f30 0x7fc18c099650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:55.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.341+0000 7fc18a000640 1 --2- 172.21.15.100:0/3811917229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc18c0cd1d0 0x7fc18c155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:55.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.341+0000 7fc18b400640 1 --2- 172.21.15.100:0/3811917229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc18c15d6e0 0x7fc18c1e9350 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fc184009bf0 tx=0x7fc184003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:55.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.342+0000 7fc182a00640 1 -- 172.21.15.100:0/3811917229 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184044070 con 0x7fc18c15d6e0 2024-09-17T04:33:55.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.342+0000 7fc182a00640 1 -- 172.21.15.100:0/3811917229 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc184003e00 con 0x7fc18c15d6e0 2024-09-17T04:33:55.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.342+0000 7fc182a00640 1 -- 172.21.15.100:0/3811917229 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1840044b0 con 0x7fc18c15d6e0 2024-09-17T04:33:55.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.342+0000 7fc18be00640 1 -- 172.21.15.100:0/3811917229 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc18c1f6c00 con 0x7fc18c15d6e0 2024-09-17T04:33:55.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.342+0000 7fc18be00640 1 -- 172.21.15.100:0/3811917229 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc18c1f6640 con 0x7fc18c15d6e0 2024-09-17T04:33:55.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.343+0000 7fc18be00640 1 -- 172.21.15.100:0/3811917229 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc18c0cf960 con 0x7fc18c15d6e0 2024-09-17T04:33:55.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.344+0000 7fc182a00640 1 -- 172.21.15.100:0/3811917229 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7fc184047070 con 0x7fc18c15d6e0 2024-09-17T04:33:55.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.345+0000 7fc182a00640 1 --2- 172.21.15.100:0/3811917229 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc16c0795b0 0x7fc16c07bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:55.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.345+0000 7fc182a00640 1 -- 172.21.15.100:0/3811917229 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7fc1840ba790 con 0x7fc18c15d6e0 2024-09-17T04:33:55.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.345+0000 7fc18aa00640 1 --2- 172.21.15.100:0/3811917229 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc16c0795b0 0x7fc16c07bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:55.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.346+0000 7fc18aa00640 1 --2- 172.21.15.100:0/3811917229 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc16c0795b0 0x7fc16c07bac0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009ab0 tx=0x7fc17c005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:55.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.349+0000 7fc182a00640 1 -- 172.21.15.100:0/3811917229 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc18c0cf960 con 0x7fc18c15d6e0 2024-09-17T04:33:55.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:55.565+0000 7fc18be00640 1 -- 172.21.15.100:0/3811917229 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc18c1567e0 con 0x7fc18c15d6e0 2024-09-17T04:33:56.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.089+0000 7fc182a00640 1 -- 172.21.15.100:0/3811917229 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v121) ==== 122+0+0 (secure 0 0 0) 0x7fc184032030 con 0x7fc18c15d6e0 2024-09-17T04:33:56.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.092+0000 7fc18be00640 1 -- 172.21.15.100:0/3811917229 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc16c0795b0 msgr2=0x7fc16c07bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:56.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.092+0000 7fc18be00640 1 --2- 172.21.15.100:0/3811917229 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc16c0795b0 0x7fc16c07bac0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009ab0 tx=0x7fc17c005f90 comp rx=0 tx=0).stop 2024-09-17T04:33:56.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.092+0000 7fc18be00640 1 -- 172.21.15.100:0/3811917229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc18c15d6e0 msgr2=0x7fc18c1e9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:56.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.092+0000 7fc18be00640 1 --2- 172.21.15.100:0/3811917229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc18c15d6e0 0x7fc18c1e9350 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fc184009bf0 tx=0x7fc184003c10 comp rx=0 tx=0).stop 2024-09-17T04:33:56.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.093+0000 7fc18be00640 1 -- 172.21.15.100:0/3811917229 shutdown_connections 2024-09-17T04:33:56.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.093+0000 7fc18be00640 1 --2- 172.21.15.100:0/3811917229 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc16c0795b0 0x7fc16c07bac0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.093+0000 7fc18be00640 1 --2- 172.21.15.100:0/3811917229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc18c15d6e0 0x7fc18c1e9350 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.093+0000 7fc18be00640 1 --2- 172.21.15.100:0/3811917229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc18c0cd1d0 0x7fc18c155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.093+0000 7fc18be00640 1 --2- 172.21.15.100:0/3811917229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc18c0b3f30 0x7fc18c099650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.093+0000 7fc18be00640 1 -- 172.21.15.100:0/3811917229 >> 172.21.15.100:0/3811917229 conn(0x7fc18c06d620 msgr2=0x7fc18c15e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:56.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.093+0000 7fc18be00640 1 -- 172.21.15.100:0/3811917229 shutdown_connections 2024-09-17T04:33:56.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.093+0000 7fc18be00640 1 -- 172.21.15.100:0/3811917229 wait complete. 2024-09-17T04:33:56.115 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T04:33:56.115 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:33:56.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.335+0000 7f5021c00640 1 -- 172.21.15.100:0/29666282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f501c0ad0e0 msgr2=0x7f501c0ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:56.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.335+0000 7f5021c00640 1 --2- 172.21.15.100:0/29666282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f501c0ad0e0 0x7f501c0ad4f0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f501c075250 tx=0x7f500402a580 comp rx=0 tx=0).stop 2024-09-17T04:33:56.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.335+0000 7f5021c00640 1 -- 172.21.15.100:0/29666282 shutdown_connections 2024-09-17T04:33:56.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.335+0000 7f5021c00640 1 --2- 172.21.15.100:0/29666282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f501c0ad0e0 0x7f501c0ad4f0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.335+0000 7f5021c00640 1 --2- 172.21.15.100:0/29666282 >> v2:172.21.15.184:3300/0 conn(0x7f501c15d760 0x7f501c1535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.335+0000 7f5021c00640 1 -- 172.21.15.100:0/29666282 >> 172.21.15.100:0/29666282 conn(0x7f501c06d730 msgr2=0x7f501c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:56.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.335+0000 7f5021c00640 1 -- 172.21.15.100:0/29666282 shutdown_connections 2024-09-17T04:33:56.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.335+0000 7f5021c00640 1 -- 172.21.15.100:0/29666282 wait complete. 2024-09-17T04:33:56.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.337+0000 7f5021c00640 1 Processor -- start 2024-09-17T04:33:56.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.338+0000 7f5021c00640 1 -- start start 2024-09-17T04:33:56.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.339+0000 7f5021c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f501c0ad0e0 0x7f501c158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:56.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.339+0000 7f5021c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f501c15d760 0x7f501c158840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:56.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.339+0000 7f5021c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f501c160310 0x7f501c15baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:56.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.339+0000 7f5021c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f501c0b4c20 con 0x7f501c160310 2024-09-17T04:33:56.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.339+0000 7f5021c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f501c153c70 con 0x7f501c15d760 2024-09-17T04:33:56.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.339+0000 7f5021c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f501c0b4da0 con 0x7f501c0ad0e0 2024-09-17T04:33:56.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.339+0000 7f501b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f501c160310 0x7f501c15baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:56.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.339+0000 7f501b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f501c160310 0x7f501c15baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37804/0 (socket says 172.21.15.100:37804) 2024-09-17T04:33:56.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.339+0000 7f501b400640 1 -- 172.21.15.100:0/75820745 learned_addr learned my addr 172.21.15.100:0/75820745 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:56.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.339+0000 7f5013e00640 1 --2- 172.21.15.100:0/75820745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f501c15d760 0x7f501c158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:56.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.340+0000 7f501aa00640 1 --2- 172.21.15.100:0/75820745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f501c0ad0e0 0x7f501c158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:56.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.340+0000 7f501b400640 1 -- 172.21.15.100:0/75820745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f501c0ad0e0 msgr2=0x7f501c158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:56.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.340+0000 7f501b400640 1 --2- 172.21.15.100:0/75820745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f501c0ad0e0 0x7f501c158300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.340+0000 7f501b400640 1 -- 172.21.15.100:0/75820745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f501c15d760 msgr2=0x7f501c158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:56.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.340+0000 7f501b400640 1 --2- 172.21.15.100:0/75820745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f501c15d760 0x7f501c158840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.340+0000 7f501b400640 1 -- 172.21.15.100:0/75820745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f501c156930 con 0x7f501c160310 2024-09-17T04:33:56.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.340+0000 7f501b400640 1 --2- 172.21.15.100:0/75820745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f501c160310 0x7f501c15baa0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f501400c790 tx=0x7f501400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:56.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.340+0000 7f5012a00640 1 -- 172.21.15.100:0/75820745 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5014017070 con 0x7f501c160310 2024-09-17T04:33:56.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.340+0000 7f5012a00640 1 -- 172.21.15.100:0/75820745 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f501400ce40 con 0x7f501c160310 2024-09-17T04:33:56.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.340+0000 7f5012a00640 1 -- 172.21.15.100:0/75820745 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50140052b0 con 0x7f501c160310 2024-09-17T04:33:56.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.340+0000 7f5021c00640 1 -- 172.21.15.100:0/75820745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f501c154200 con 0x7f501c160310 2024-09-17T04:33:56.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.341+0000 7f5021c00640 1 -- 172.21.15.100:0/75820745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f501c1547c0 con 0x7f501c160310 2024-09-17T04:33:56.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.342+0000 7f5021c00640 1 -- 172.21.15.100:0/75820745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f501c160ff0 con 0x7f501c160310 2024-09-17T04:33:56.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.343+0000 7f5012a00640 1 -- 172.21.15.100:0/75820745 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f501401a070 con 0x7f501c160310 2024-09-17T04:33:56.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.344+0000 7f5012a00640 1 --2- 172.21.15.100:0/75820745 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4ff80795e0 0x7f4ff807baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:56.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.345+0000 7f5012a00640 1 -- 172.21.15.100:0/75820745 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7f5014098e10 con 0x7f501c160310 2024-09-17T04:33:56.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.345+0000 7f501aa00640 1 --2- 172.21.15.100:0/75820745 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4ff80795e0 0x7f4ff807baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:56.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.345+0000 7f501aa00640 1 --2- 172.21.15.100:0/75820745 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4ff80795e0 0x7f4ff807baf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f500c00bb50 tx=0x7f500c00b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:56.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.349+0000 7f5012a00640 1 -- 172.21.15.100:0/75820745 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f501c0ada30 con 0x7f501c160310 2024-09-17T04:33:56.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.566+0000 7f5021c00640 1 -- 172.21.15.100:0/75820745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f501c152dd0 con 0x7f501c160310 2024-09-17T04:33:56.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.566+0000 7f5012a00640 1 -- 172.21.15.100:0/75820745 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) ==== 95+0+5242 (secure 0 0 0) 0x7f5014062430 con 0x7f501c160310 2024-09-17T04:33:56.567 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:56.567 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":121,"btime":"2024-09-17T04:33:56:084891+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/613141426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":613141426},{"type":"v1","addr":"172.21.15.184:6835","nonce":613141426}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":8554,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/1797529407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6801","nonce":1797529407},{"type":"v1","addr":"172.21.15.184:6834","nonce":1797529407}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":8560,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2263752731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2263752731},{"type":"v1","addr":"172.21.15.100:6833","nonce":2263752731}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:52.555851+0000","modified":"2024-09-17T04:33:55.567693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9153},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9153":{"gid":9153,"name":"c","rank":0,"incarnation":117,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/1207785711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1207785711},{"type":"v1","addr":"172.21.15.100:6835","nonce":1207785711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9153,"qdb_cluster":[9153]},"id":13}]} 2024-09-17T04:33:56.567 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 121 2024-09-17T04:33:56.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.569+0000 7f5021c00640 1 -- 172.21.15.100:0/75820745 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4ff80795e0 msgr2=0x7f4ff807baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:56.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.569+0000 7f5021c00640 1 --2- 172.21.15.100:0/75820745 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4ff80795e0 0x7f4ff807baf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f500c00bb50 tx=0x7f500c00b2e0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.569+0000 7f5021c00640 1 -- 172.21.15.100:0/75820745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f501c160310 msgr2=0x7f501c15baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:56.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.569+0000 7f5021c00640 1 --2- 172.21.15.100:0/75820745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f501c160310 0x7f501c15baa0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f501400c790 tx=0x7f501400cc50 comp rx=0 tx=0).stop 2024-09-17T04:33:56.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.569+0000 7f5021c00640 1 -- 172.21.15.100:0/75820745 shutdown_connections 2024-09-17T04:33:56.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.569+0000 7f5021c00640 1 --2- 172.21.15.100:0/75820745 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4ff80795e0 0x7f4ff807baf0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.569+0000 7f5021c00640 1 --2- 172.21.15.100:0/75820745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f501c160310 0x7f501c15baa0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.569+0000 7f5021c00640 1 --2- 172.21.15.100:0/75820745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f501c15d760 0x7f501c158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.569+0000 7f5021c00640 1 --2- 172.21.15.100:0/75820745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f501c0ad0e0 0x7f501c158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.569+0000 7f5021c00640 1 -- 172.21.15.100:0/75820745 >> 172.21.15.100:0/75820745 conn(0x7f501c06d730 msgr2=0x7f501c06f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:56.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.569+0000 7f5021c00640 1 -- 172.21.15.100:0/75820745 shutdown_connections 2024-09-17T04:33:56.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.569+0000 7f5021c00640 1 -- 172.21.15.100:0/75820745 wait complete. 2024-09-17T04:33:56.595 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:33:56.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.815+0000 7ffadb400640 1 -- 172.21.15.100:0/1526092049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad40ad0e0 msgr2=0x7ffad40ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:56.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.815+0000 7ffadb400640 1 --2- 172.21.15.100:0/1526092049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad40ad0e0 0x7ffad40ad4f0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7ffad40b3100 tx=0x7ffabc02a580 comp rx=0 tx=0).stop 2024-09-17T04:33:56.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.815+0000 7ffadb400640 1 -- 172.21.15.100:0/1526092049 shutdown_connections 2024-09-17T04:33:56.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.815+0000 7ffadb400640 1 --2- 172.21.15.100:0/1526092049 >> v2:172.21.15.184:3300/0 conn(0x7ffad40b3550 0x7ffad40b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.815+0000 7ffadb400640 1 --2- 172.21.15.100:0/1526092049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad40ad0e0 0x7ffad40ad4f0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.815+0000 7ffadb400640 1 -- 172.21.15.100:0/1526092049 >> 172.21.15.100:0/1526092049 conn(0x7ffad406d730 msgr2=0x7ffad40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:56.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.815+0000 7ffadb400640 1 -- 172.21.15.100:0/1526092049 shutdown_connections 2024-09-17T04:33:56.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.815+0000 7ffadb400640 1 -- 172.21.15.100:0/1526092049 wait complete. 2024-09-17T04:33:56.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.817+0000 7ffadb400640 1 Processor -- start 2024-09-17T04:33:56.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.817+0000 7ffadb400640 1 -- start start 2024-09-17T04:33:56.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.819+0000 7ffadb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad40ad0e0 0x7ffad41577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:56.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.819+0000 7ffadb400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffad40b3550 0x7ffad4073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:56.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.819+0000 7ffadb400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffad415d760 0x7ffad4073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:56.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.819+0000 7ffadb400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ffad40b4da0 con 0x7ffad40ad0e0 2024-09-17T04:33:56.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.819+0000 7ffadb400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffad40b4c20 con 0x7ffad415d760 2024-09-17T04:33:56.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.819+0000 7ffadb400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ffad40b3ea0 con 0x7ffad40b3550 2024-09-17T04:33:56.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.819+0000 7ffad8a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad40ad0e0 0x7ffad41577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:56.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.819+0000 7ffad8a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad40ad0e0 0x7ffad41577f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37832/0 (socket says 172.21.15.100:37832) 2024-09-17T04:33:56.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.819+0000 7ffad8a00640 1 -- 172.21.15.100:0/1204854831 learned_addr learned my addr 172.21.15.100:0/1204854831 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:56.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.820+0000 7ffad9400640 1 --2- 172.21.15.100:0/1204854831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffad415d760 0x7ffad4073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:56.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.820+0000 7ffacbe00640 1 --2- 172.21.15.100:0/1204854831 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffad40b3550 0x7ffad4073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:56.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.820+0000 7ffad8a00640 1 -- 172.21.15.100:0/1204854831 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffad40b3550 msgr2=0x7ffad4073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:56.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.820+0000 7ffad8a00640 1 --2- 172.21.15.100:0/1204854831 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffad40b3550 0x7ffad4073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.820+0000 7ffad8a00640 1 -- 172.21.15.100:0/1204854831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffad415d760 msgr2=0x7ffad4073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:56.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.820+0000 7ffad8a00640 1 --2- 172.21.15.100:0/1204854831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffad415d760 0x7ffad4073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.820+0000 7ffad8a00640 1 -- 172.21.15.100:0/1204854831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad4156b60 con 0x7ffad40ad0e0 2024-09-17T04:33:56.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.820+0000 7ffad8a00640 1 --2- 172.21.15.100:0/1204854831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad40ad0e0 0x7ffad41577f0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7ffac400c3a0 tx=0x7ffac400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:56.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.820+0000 7ffad1600640 1 -- 172.21.15.100:0/1204854831 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac4017070 con 0x7ffad40ad0e0 2024-09-17T04:33:56.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.820+0000 7ffad1600640 1 -- 172.21.15.100:0/1204854831 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffac40053c0 con 0x7ffad40ad0e0 2024-09-17T04:33:56.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.820+0000 7ffadb400640 1 -- 172.21.15.100:0/1204854831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad41595a0 con 0x7ffad40ad0e0 2024-09-17T04:33:56.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.820+0000 7ffad1600640 1 -- 172.21.15.100:0/1204854831 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac4003850 con 0x7ffad40ad0e0 2024-09-17T04:33:56.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.820+0000 7ffadb400640 1 -- 172.21.15.100:0/1204854831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad4157d30 con 0x7ffad40ad0e0 2024-09-17T04:33:56.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.821+0000 7ffadb400640 1 -- 172.21.15.100:0/1204854831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad415cb90 con 0x7ffad40ad0e0 2024-09-17T04:33:56.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.823+0000 7ffad1600640 1 -- 172.21.15.100:0/1204854831 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7ffac401a070 con 0x7ffad40ad0e0 2024-09-17T04:33:56.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.824+0000 7ffad1600640 1 --2- 172.21.15.100:0/1204854831 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ffab00795e0 0x7ffab007baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:56.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.824+0000 7ffad1600640 1 -- 172.21.15.100:0/1204854831 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7ffac4098b50 con 0x7ffad40ad0e0 2024-09-17T04:33:56.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.824+0000 7ffacbe00640 1 --2- 172.21.15.100:0/1204854831 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ffab00795e0 0x7ffab007baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:56.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.825+0000 7ffacbe00640 1 --2- 172.21.15.100:0/1204854831 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ffab00795e0 0x7ffab007baf0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ffabc033000 tx=0x7ffabc004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:56.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:56.828+0000 7ffad1600640 1 -- 172.21.15.100:0/1204854831 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ffad415cb90 con 0x7ffad40ad0e0 2024-09-17T04:33:57.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.013+0000 7ffadb400640 1 -- 172.21.15.100:0/1204854831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffad40b43a0 con 0x7ffad40ad0e0 2024-09-17T04:33:57.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.015+0000 7ffad1600640 1 -- 172.21.15.100:0/1204854831 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) ==== 74+0+16880 (secure 0 0 0) 0x7ffac4067e20 con 0x7ffad40ad0e0 2024-09-17T04:33:57.016 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:57.016 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":167,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:33:54.064660+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:33:50.447902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:51.510699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:33:57.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.017+0000 7ffadb400640 1 -- 172.21.15.100:0/1204854831 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ffab00795e0 msgr2=0x7ffab007baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:57.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.017+0000 7ffadb400640 1 --2- 172.21.15.100:0/1204854831 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ffab00795e0 0x7ffab007baf0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ffabc033000 tx=0x7ffabc004730 comp rx=0 tx=0).stop 2024-09-17T04:33:57.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.017+0000 7ffadb400640 1 -- 172.21.15.100:0/1204854831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad40ad0e0 msgr2=0x7ffad41577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:57.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.017+0000 7ffadb400640 1 --2- 172.21.15.100:0/1204854831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad40ad0e0 0x7ffad41577f0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7ffac400c3a0 tx=0x7ffac400c860 comp rx=0 tx=0).stop 2024-09-17T04:33:57.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.017+0000 7ffadb400640 1 -- 172.21.15.100:0/1204854831 shutdown_connections 2024-09-17T04:33:57.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.017+0000 7ffadb400640 1 --2- 172.21.15.100:0/1204854831 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ffab00795e0 0x7ffab007baf0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.017+0000 7ffadb400640 1 --2- 172.21.15.100:0/1204854831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffad415d760 0x7ffad4073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.017+0000 7ffadb400640 1 --2- 172.21.15.100:0/1204854831 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffad40b3550 0x7ffad4073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.017+0000 7ffadb400640 1 --2- 172.21.15.100:0/1204854831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad40ad0e0 0x7ffad41577f0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.017+0000 7ffadb400640 1 -- 172.21.15.100:0/1204854831 >> 172.21.15.100:0/1204854831 conn(0x7ffad406d730 msgr2=0x7ffad40cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:57.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.017+0000 7ffadb400640 1 -- 172.21.15.100:0/1204854831 shutdown_connections 2024-09-17T04:33:57.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.017+0000 7ffadb400640 1 -- 172.21.15.100:0/1204854831 wait complete. 2024-09-17T04:33:57.042 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T04:33:57.042 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:33:57.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.263+0000 7faffb000640 1 -- 172.21.15.100:0/3907625006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafdc003fd0 msgr2=0x7fafdc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:57.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.263+0000 7faffb000640 1 --2- 172.21.15.100:0/3907625006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafdc003fd0 0x7fafdc004800 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fafe000c260 tx=0x7fafdc005830 comp rx=0 tx=0).stop 2024-09-17T04:33:57.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.263+0000 7faffa600640 1 -- 172.21.15.100:0/3907625006 reap_dead start 2024-09-17T04:33:57.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.263+0000 7faffb000640 1 -- 172.21.15.100:0/3907625006 shutdown_connections 2024-09-17T04:33:57.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.263+0000 7faffb000640 1 -- 172.21.15.100:0/3907625006 >> 172.21.15.100:0/3907625006 conn(0x7faff40b1e40 msgr2=0x7faff40b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:57.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.263+0000 7faffb000640 1 -- 172.21.15.100:0/3907625006 shutdown_connections 2024-09-17T04:33:57.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.263+0000 7faffb000640 1 -- 172.21.15.100:0/3907625006 wait complete. 2024-09-17T04:33:57.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.265+0000 7faffb000640 1 Processor -- start 2024-09-17T04:33:57.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.265+0000 7faffb000640 1 -- start start 2024-09-17T04:33:57.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.267+0000 7faffb000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafdc003fd0 0x7faff41f02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:57.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.267+0000 7faffb000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faff41f4180 0x7faff41f4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:57.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.267+0000 7faffb000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faff41f4ad0 0x7faff4157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:57.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.267+0000 7faffb000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faff40b3a70 con 0x7faff41f4180 2024-09-17T04:33:57.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.267+0000 7faffb000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faff40b38f0 con 0x7fafdc003fd0 2024-09-17T04:33:57.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.267+0000 7faffb000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faff40b3bf0 con 0x7faff41f4ad0 2024-09-17T04:33:57.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.267+0000 7faff9200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faff41f4180 0x7faff41f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:57.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.267+0000 7faff9200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faff41f4180 0x7faff41f4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37842/0 (socket says 172.21.15.100:37842) 2024-09-17T04:33:57.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.267+0000 7faff9200640 1 -- 172.21.15.100:0/2978047420 learned_addr learned my addr 172.21.15.100:0/2978047420 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:57.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.267+0000 7faff9c00640 1 --2- 172.21.15.100:0/2978047420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafdc003fd0 0x7faff41f02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:57.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.267+0000 7faffa600640 1 --2- 172.21.15.100:0/2978047420 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faff41f4ad0 0x7faff4157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:57.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.267+0000 7faff9200640 1 -- 172.21.15.100:0/2978047420 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faff41f4ad0 msgr2=0x7faff4157760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:57.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.267+0000 7faff9200640 1 --2- 172.21.15.100:0/2978047420 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faff41f4ad0 0x7faff4157760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.267+0000 7faff9200640 1 -- 172.21.15.100:0/2978047420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafdc003fd0 msgr2=0x7faff41f02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:57.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.267+0000 7faff9200640 1 --2- 172.21.15.100:0/2978047420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafdc003fd0 0x7faff41f02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.267+0000 7faff9200640 1 -- 172.21.15.100:0/2978047420 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff4157260 con 0x7faff41f4180 2024-09-17T04:33:57.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.268+0000 7faff9200640 1 --2- 172.21.15.100:0/2978047420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faff41f4180 0x7faff41f4590 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fafe002b110 tx=0x7fafe0006c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:57.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.268+0000 7faff1600640 1 -- 172.21.15.100:0/2978047420 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe0031040 con 0x7faff41f4180 2024-09-17T04:33:57.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.268+0000 7faff1600640 1 -- 172.21.15.100:0/2978047420 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafe00035f0 con 0x7faff41f4180 2024-09-17T04:33:57.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.268+0000 7faff1600640 1 -- 172.21.15.100:0/2978047420 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe0003a40 con 0x7faff41f4180 2024-09-17T04:33:57.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.268+0000 7faffb000640 1 -- 172.21.15.100:0/2978047420 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff415e000 con 0x7faff41f4180 2024-09-17T04:33:57.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.268+0000 7faffb000640 1 -- 172.21.15.100:0/2978047420 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff415e5c0 con 0x7faff41f4180 2024-09-17T04:33:57.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.269+0000 7faffb000640 1 -- 172.21.15.100:0/2978047420 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faff4073270 con 0x7faff41f4180 2024-09-17T04:33:57.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.270+0000 7faff1600640 1 -- 172.21.15.100:0/2978047420 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7fafe0034070 con 0x7faff41f4180 2024-09-17T04:33:57.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.271+0000 7faff1600640 1 --2- 172.21.15.100:0/2978047420 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fafd40795b0 0x7fafd407bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:57.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.271+0000 7faff1600640 1 -- 172.21.15.100:0/2978047420 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7fafe00b9a10 con 0x7faff41f4180 2024-09-17T04:33:57.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.272+0000 7faff9c00640 1 --2- 172.21.15.100:0/2978047420 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fafd40795b0 0x7fafd407bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:57.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.272+0000 7faff9c00640 1 --2- 172.21.15.100:0/2978047420 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fafd40795b0 0x7fafd407bac0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fafe8009bb0 tx=0x7fafe8005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:57.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.276+0000 7faff1600640 1 -- 172.21.15.100:0/2978047420 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fafe0083680 con 0x7faff41f4180 2024-09-17T04:33:57.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.454+0000 7faffb000640 1 -- 172.21.15.100:0/2978047420 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faff415dd90 con 0x7fafd40795b0 2024-09-17T04:33:57.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.458+0000 7faff1600640 1 -- 172.21.15.100:0/2978047420 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240262 (secure 0 0 0) 0x7fafe805baa0 con 0x7fafd40795b0 2024-09-17T04:33:57.462 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:57.463 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:33:57.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.463+0000 7faffb000640 1 -- 172.21.15.100:0/2978047420 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fafd40795b0 msgr2=0x7fafd407bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:57.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.463+0000 7faffb000640 1 --2- 172.21.15.100:0/2978047420 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fafd40795b0 0x7fafd407bac0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fafe8009bb0 tx=0x7fafe8005f90 comp rx=0 tx=0).stop 2024-09-17T04:33:57.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.463+0000 7faffb000640 1 -- 172.21.15.100:0/2978047420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faff41f4180 msgr2=0x7faff41f4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:57.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.463+0000 7faffb000640 1 --2- 172.21.15.100:0/2978047420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faff41f4180 0x7faff41f4590 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fafe002b110 tx=0x7fafe0006c50 comp rx=0 tx=0).stop 2024-09-17T04:33:57.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.463+0000 7faffb000640 1 -- 172.21.15.100:0/2978047420 shutdown_connections 2024-09-17T04:33:57.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.463+0000 7faffb000640 1 --2- 172.21.15.100:0/2978047420 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fafd40795b0 0x7fafd407bac0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.463+0000 7faffb000640 1 --2- 172.21.15.100:0/2978047420 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faff41f4ad0 0x7faff4157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.463+0000 7faffb000640 1 --2- 172.21.15.100:0/2978047420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faff41f4180 0x7faff41f4590 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.463+0000 7faffb000640 1 --2- 172.21.15.100:0/2978047420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafdc003fd0 0x7faff41f02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.463+0000 7faffb000640 1 -- 172.21.15.100:0/2978047420 >> 172.21.15.100:0/2978047420 conn(0x7faff40b1e40 msgr2=0x7faff4154cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:57.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.463+0000 7faffb000640 1 -- 172.21.15.100:0/2978047420 shutdown_connections 2024-09-17T04:33:57.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.463+0000 7faffb000640 1 -- 172.21.15.100:0/2978047420 wait complete. 2024-09-17T04:33:57.485 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-09-17T04:33:55.725054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":266780,"kb_used_data":17244,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754707940,"statfs":{"total":773094113280,"available":772820930560,"internally_reserved":0,"allocated":17657856,"data_stored":8434095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.680875"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.055065+0000","last_change":"2024-09-17T04:33:51.055065+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.055065+0000","last_undegraded":"2024-09-17T04:33:51.055065+0000","last_fullsized":"2024-09-17T04:33:51.055065+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.051542+0000","last_change":"2024-09-17T04:33:51.051542+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.051542+0000","last_undegraded":"2024-09-17T04:33:51.051542+0000","last_fullsized":"2024-09-17T04:33:51.051542+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.048451+0000","last_change":"2024-09-17T04:33:51.048451+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.048451+0000","last_undegraded":"2024-09-17T04:33:51.048451+0000","last_fullsized":"2024-09-17T04:33:51.048451+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.052671+0000","last_change":"2024-09-17T04:33:51.052671+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.052671+0000","last_undegraded":"2024-09-17T04:33:51.052671+0000","last_fullsized":"2024-09-17T04:33:51.052671+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.051067+0000","last_change":"2024-09-17T04:33:51.051067+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.051067+0000","last_undegraded":"2024-09-17T04:33:51.051067+0000","last_fullsized":"2024-09-17T04:33:51.051067+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.050539+0000","last_change":"2024-09-17T04:33:51.050539+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.050539+0000","last_undegraded":"2024-09-17T04:33:51.050539+0000","last_fullsized":"2024-09-17T04:33:51.050539+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.057359+0000","last_change":"2024-09-17T04:33:51.057359+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.057359+0000","last_undegraded":"2024-09-17T04:33:51.057359+0000","last_fullsized":"2024-09-17T04:33:51.057359+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.051073+0000","last_change":"2024-09-17T04:33:51.051073+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.051073+0000","last_undegraded":"2024-09-17T04:33:51.051073+0000","last_fullsized":"2024-09-17T04:33:51.051073+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.055941+0000","last_change":"2024-09-17T04:33:51.055941+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.055941+0000","last_undegraded":"2024-09-17T04:33:51.055941+0000","last_fullsized":"2024-09-17T04:33:51.055941+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.054789+0000","last_change":"2024-09-17T04:33:51.054789+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.054789+0000","last_undegraded":"2024-09-17T04:33:51.054789+0000","last_fullsized":"2024-09-17T04:33:51.054789+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.050954+0000","last_change":"2024-09-17T04:33:51.050954+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.050954+0000","last_undegraded":"2024-09-17T04:33:51.050954+0000","last_fullsized":"2024-09-17T04:33:51.050954+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.056398+0000","last_change":"2024-09-17T04:33:51.056398+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.056398+0000","last_undegraded":"2024-09-17T04:33:51.056398+0000","last_fullsized":"2024-09-17T04:33:51.056398+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.055617+0000","last_change":"2024-09-17T04:33:51.055617+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.055617+0000","last_undegraded":"2024-09-17T04:33:51.055617+0000","last_fullsized":"2024-09-17T04:33:51.055617+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.053826+0000","last_change":"2024-09-17T04:33:51.053826+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.053826+0000","last_undegraded":"2024-09-17T04:33:51.053826+0000","last_fullsized":"2024-09-17T04:33:51.053826+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.048438+0000","last_change":"2024-09-17T04:33:51.048438+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.048438+0000","last_undegraded":"2024-09-17T04:33:51.048438+0000","last_fullsized":"2024-09-17T04:33:51.048438+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.050586+0000","last_change":"2024-09-17T04:33:51.050586+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.050586+0000","last_undegraded":"2024-09-17T04:33:51.050586+0000","last_fullsized":"2024-09-17T04:33:51.050586+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.051011+0000","last_change":"2024-09-17T04:33:51.051011+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.051011+0000","last_undegraded":"2024-09-17T04:33:51.051011+0000","last_fullsized":"2024-09-17T04:33:51.051011+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.053998+0000","last_change":"2024-09-17T04:33:51.053998+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.053998+0000","last_undegraded":"2024-09-17T04:33:51.053998+0000","last_fullsized":"2024-09-17T04:33:51.053998+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.048441+0000","last_change":"2024-09-17T04:33:51.048441+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.048441+0000","last_undegraded":"2024-09-17T04:33:51.048441+0000","last_fullsized":"2024-09-17T04:33:51.048441+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.054124+0000","last_change":"2024-09-17T04:33:51.054124+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.054124+0000","last_undegraded":"2024-09-17T04:33:51.054124+0000","last_fullsized":"2024-09-17T04:33:51.054124+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.056286+0000","last_change":"2024-09-17T04:33:51.056286+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.056286+0000","last_undegraded":"2024-09-17T04:33:51.056286+0000","last_fullsized":"2024-09-17T04:33:51.056286+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.056358+0000","last_change":"2024-09-17T04:33:51.056358+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.056358+0000","last_undegraded":"2024-09-17T04:33:51.056358+0000","last_fullsized":"2024-09-17T04:33:51.056358+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.050539+0000","last_change":"2024-09-17T04:33:51.050539+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.050539+0000","last_undegraded":"2024-09-17T04:33:51.050539+0000","last_fullsized":"2024-09-17T04:33:51.050539+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.047709+0000","last_change":"2024-09-17T04:33:51.047709+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.047709+0000","last_undegraded":"2024-09-17T04:33:51.047709+0000","last_fullsized":"2024-09-17T04:33:51.047709+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.051252+0000","last_change":"2024-09-17T04:33:51.051252+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.051252+0000","last_undegraded":"2024-09-17T04:33:51.051252+0000","last_fullsized":"2024-09-17T04:33:51.051252+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.050583+0000","last_change":"2024-09-17T04:33:51.050583+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.050583+0000","last_undegraded":"2024-09-17T04:33:51.050583+0000","last_fullsized":"2024-09-17T04:33:51.050583+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.047709+0000","last_change":"2024-09-17T04:33:51.047709+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.047709+0000","last_undegraded":"2024-09-17T04:33:51.047709+0000","last_fullsized":"2024-09-17T04:33:51.047709+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.050959+0000","last_change":"2024-09-17T04:33:51.050959+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.050959+0000","last_undegraded":"2024-09-17T04:33:51.050959+0000","last_fullsized":"2024-09-17T04:33:51.050959+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.050520+0000","last_change":"2024-09-17T04:33:51.050520+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.050520+0000","last_undegraded":"2024-09-17T04:33:51.050520+0000","last_fullsized":"2024-09-17T04:33:51.050520+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.056916+0000","last_change":"2024-09-17T04:33:51.056916+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.056916+0000","last_undegraded":"2024-09-17T04:33:51.056916+0000","last_fullsized":"2024-09-17T04:33:51.056916+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.054415+0000","last_change":"2024-09-17T04:33:51.054415+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.054415+0000","last_undegraded":"2024-09-17T04:33:51.054415+0000","last_fullsized":"2024-09-17T04:33:51.054415+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:33:52.043794+0000","last_change":"2024-09-17T04:33:52.043794+0000","last_active":"2024-09-17T04:33:52.043794+0000","last_peered":"2024-09-17T04:33:52.043794+0000","last_clean":"2024-09-17T04:33:52.043794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:52.043794+0000","last_undegraded":"2024-09-17T04:33:52.043794+0000","last_fullsized":"2024-09-17T04:33:52.043794+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":1,"reported_epoch":164,"state":"creating+peering","last_fresh":"2024-09-17T04:33:51.054433+0000","last_change":"2024-09-17T04:33:51.054433+0000","last_active":"2024-09-17T04:33:51.041003+0000","last_peered":"2024-09-17T04:33:51.041003+0000","last_clean":"2024-09-17T04:33:51.041003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:33:51.054433+0000","last_undegraded":"2024-09-17T04:33:51.054433+0000","last_fullsized":"2024-09-17T04:33:51.054433+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"76'79","reported_seq":398,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-17T04:33:51.053292+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:33:51.053292+0000","last_peered":"2024-09-17T04:33:51.053292+0000","last_clean":"2024-09-17T04:33:51.053292+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:33:51.053292+0000","last_undegraded":"2024-09-17T04:33:51.053292+0000","last_fullsized":"2024-09-17T04:33:51.053292+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476859,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33784,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338056,"statfs":{"total":96636764160,"available":96602169344,"internally_reserved":0,"allocated":2654208,"data_stored":1497772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476858,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33204,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338636,"statfs":{"total":96636764160,"available":96602763264,"internally_reserved":0,"allocated":2060288,"data_stored":907404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476859,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33216,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338624,"statfs":{"total":96636764160,"available":96602750976,"internally_reserved":0,"allocated":2060288,"data_stored":907404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476859,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33200,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338640,"statfs":{"total":96636764160,"available":96602767360,"internally_reserved":0,"allocated":2060288,"data_stored":907404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542266,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33200,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338640,"statfs":{"total":96636764160,"available":96602767360,"internally_reserved":0,"allocated":2060288,"data_stored":907404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509562,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33196,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338644,"statfs":{"total":96636764160,"available":96602771456,"internally_reserved":0,"allocated":2048000,"data_stored":901531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509561,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33200,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338640,"statfs":{"total":96636764160,"available":96602767360,"internally_reserved":0,"allocated":2060288,"data_stored":907404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509561,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33780,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338060,"statfs":{"total":96636764160,"available":96602173440,"internally_reserved":0,"allocated":2654208,"data_stored":1497772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:33:57.490 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:33:57.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.745+0000 7f991d600640 1 -- 172.21.15.100:0/2992179522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f991815d760 msgr2=0x7f99181535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:57.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.745+0000 7f991d600640 1 --2- 172.21.15.100:0/2992179522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f991815d760 0x7f99181535c0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f9908005fd0 tx=0x7f990802a4a0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.745+0000 7f991d600640 1 -- 172.21.15.100:0/2992179522 shutdown_connections 2024-09-17T04:33:57.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.745+0000 7f991d600640 1 --2- 172.21.15.100:0/2992179522 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99180ad0e0 0x7f99180ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.745+0000 7f991d600640 1 --2- 172.21.15.100:0/2992179522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f991815d760 0x7f99181535c0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.745+0000 7f991d600640 1 -- 172.21.15.100:0/2992179522 >> 172.21.15.100:0/2992179522 conn(0x7f991806d730 msgr2=0x7f99180b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:57.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.745+0000 7f991d600640 1 -- 172.21.15.100:0/2992179522 shutdown_connections 2024-09-17T04:33:57.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.745+0000 7f991d600640 1 -- 172.21.15.100:0/2992179522 wait complete. 2024-09-17T04:33:57.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.748+0000 7f991d600640 1 Processor -- start 2024-09-17T04:33:57.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.748+0000 7f991d600640 1 -- start start 2024-09-17T04:33:57.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.749+0000 7f991d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99180ad0e0 0x7f99181f9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:57.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.749+0000 7f991d600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f991815d760 0x7f99181f54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:57.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.749+0000 7f991d600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9918160310 0x7f991815b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:57.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.749+0000 7f991d600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f99180b4c20 con 0x7f99180ad0e0 2024-09-17T04:33:57.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.749+0000 7f991d600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9918153c70 con 0x7f991815d760 2024-09-17T04:33:57.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.749+0000 7f991d600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f99180b4da0 con 0x7f9918160310 2024-09-17T04:33:57.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.749+0000 7f9916a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99180ad0e0 0x7f99181f9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:57.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.750+0000 7f9916a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99180ad0e0 0x7f99181f9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37850/0 (socket says 172.21.15.100:37850) 2024-09-17T04:33:57.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.750+0000 7f9916a00640 1 -- 172.21.15.100:0/3166533295 learned_addr learned my addr 172.21.15.100:0/3166533295 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:57.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.750+0000 7f9916000640 1 --2- 172.21.15.100:0/3166533295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f991815d760 0x7f99181f54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:57.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.750+0000 7f9917400640 1 --2- 172.21.15.100:0/3166533295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9918160310 0x7f991815b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:57.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.750+0000 7f9916a00640 1 -- 172.21.15.100:0/3166533295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9918160310 msgr2=0x7f991815b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:57.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.750+0000 7f9916a00640 1 --2- 172.21.15.100:0/3166533295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9918160310 0x7f991815b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.750+0000 7f9916a00640 1 -- 172.21.15.100:0/3166533295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f991815d760 msgr2=0x7f99181f54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:57.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.750+0000 7f9916a00640 1 --2- 172.21.15.100:0/3166533295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f991815d760 0x7f99181f54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.750+0000 7f9916a00640 1 -- 172.21.15.100:0/3166533295 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99181fa140 con 0x7f99180ad0e0 2024-09-17T04:33:57.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.750+0000 7f9917400640 1 --2- 172.21.15.100:0/3166533295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9918160310 0x7f991815b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:57.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.750+0000 7f9916a00640 1 --2- 172.21.15.100:0/3166533295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99180ad0e0 0x7f99181f9c00 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f9908009850 tx=0x7f99080039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:57.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.751+0000 7f990ea00640 1 -- 172.21.15.100:0/3166533295 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990803e030 con 0x7f99180ad0e0 2024-09-17T04:33:57.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.751+0000 7f990ea00640 1 -- 172.21.15.100:0/3166533295 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99080047a0 con 0x7f99180ad0e0 2024-09-17T04:33:57.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.751+0000 7f990ea00640 1 -- 172.21.15.100:0/3166533295 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9908035040 con 0x7f99180ad0e0 2024-09-17T04:33:57.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.751+0000 7f991d600640 1 -- 172.21.15.100:0/3166533295 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f991809a8b0 con 0x7f99180ad0e0 2024-09-17T04:33:57.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.751+0000 7f991d600640 1 -- 172.21.15.100:0/3166533295 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f991809c580 con 0x7f99180ad0e0 2024-09-17T04:33:57.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.752+0000 7f991d600640 1 -- 172.21.15.100:0/3166533295 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f991815d3a0 con 0x7f99180ad0e0 2024-09-17T04:33:57.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.753+0000 7f990ea00640 1 -- 172.21.15.100:0/3166533295 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f9908041030 con 0x7f99180ad0e0 2024-09-17T04:33:57.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.754+0000 7f990ea00640 1 --2- 172.21.15.100:0/3166533295 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f98f80795b0 0x7f98f807bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:57.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.754+0000 7f990ea00640 1 -- 172.21.15.100:0/3166533295 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7f99080ba190 con 0x7f99180ad0e0 2024-09-17T04:33:57.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.754+0000 7f9916000640 1 --2- 172.21.15.100:0/3166533295 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f98f80795b0 0x7f98f807bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:57.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.755+0000 7f9916000640 1 --2- 172.21.15.100:0/3166533295 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f98f80795b0 0x7f98f807bac0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f991809e240 tx=0x7f9904008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:57.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.759+0000 7f990ea00640 1 -- 172.21.15.100:0/3166533295 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f991815d3a0 con 0x7f99180ad0e0 2024-09-17T04:33:57.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.937+0000 7f991d600640 1 -- 172.21.15.100:0/3166533295 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9918160fa0 con 0x7f98f80795b0 2024-09-17T04:33:57.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.942+0000 7f990ea00640 1 -- 172.21.15.100:0/3166533295 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252482 (secure 0 0 0) 0x7f9918160fa0 con 0x7f98f80795b0 2024-09-17T04:33:57.945 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:57.946 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:33:57.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.945+0000 7f991d600640 1 -- 172.21.15.100:0/3166533295 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f98f80795b0 msgr2=0x7f98f807bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:57.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.945+0000 7f991d600640 1 --2- 172.21.15.100:0/3166533295 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f98f80795b0 0x7f98f807bac0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f991809e240 tx=0x7f9904008040 comp rx=0 tx=0).stop 2024-09-17T04:33:57.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.945+0000 7f991d600640 1 -- 172.21.15.100:0/3166533295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99180ad0e0 msgr2=0x7f99181f9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:57.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.945+0000 7f991d600640 1 --2- 172.21.15.100:0/3166533295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99180ad0e0 0x7f99181f9c00 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f9908009850 tx=0x7f99080039e0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.945+0000 7f991d600640 1 -- 172.21.15.100:0/3166533295 shutdown_connections 2024-09-17T04:33:57.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.945+0000 7f991d600640 1 --2- 172.21.15.100:0/3166533295 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f98f80795b0 0x7f98f807bac0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.945+0000 7f991d600640 1 --2- 172.21.15.100:0/3166533295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9918160310 0x7f991815b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.945+0000 7f991d600640 1 --2- 172.21.15.100:0/3166533295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f991815d760 0x7f99181f54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.945+0000 7f991d600640 1 --2- 172.21.15.100:0/3166533295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99180ad0e0 0x7f99181f9c00 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.945+0000 7f991d600640 1 -- 172.21.15.100:0/3166533295 >> 172.21.15.100:0/3166533295 conn(0x7f991806d730 msgr2=0x7f9918160720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:57.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.946+0000 7f991d600640 1 -- 172.21.15.100:0/3166533295 shutdown_connections 2024-09-17T04:33:57.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:57.946+0000 7f991d600640 1 -- 172.21.15.100:0/3166533295 wait complete. 2024-09-17T04:33:57.969 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":241,"stamp":"2024-09-17T04:33:57.725506+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":163,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":275460,"kb_used_data":17608,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754699260,"statfs":{"total":773094113280,"available":772812042240,"internally_reserved":0,"allocated":18030592,"data_stored":8600932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":263835188},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001318"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.265522+0000","last_change":"2024-09-17T04:33:53.090566+0000","last_active":"2024-09-17T04:33:54.265522+0000","last_peered":"2024-09-17T04:33:54.265522+0000","last_clean":"2024-09-17T04:33:54.265522+0000","last_became_active":"2024-09-17T04:33:53.090275+0000","last_became_peered":"2024-09-17T04:33:53.090275+0000","last_unstale":"2024-09-17T04:33:54.265522+0000","last_undegraded":"2024-09-17T04:33:54.265522+0000","last_fullsized":"2024-09-17T04:33:54.265522+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:22.641900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.080860+0000","last_change":"2024-09-17T04:33:53.064621+0000","last_active":"2024-09-17T04:33:54.080860+0000","last_peered":"2024-09-17T04:33:54.080860+0000","last_clean":"2024-09-17T04:33:54.080860+0000","last_became_active":"2024-09-17T04:33:53.064252+0000","last_became_peered":"2024-09-17T04:33:53.064252+0000","last_unstale":"2024-09-17T04:33:54.080860+0000","last_undegraded":"2024-09-17T04:33:54.080860+0000","last_fullsized":"2024-09-17T04:33:54.080860+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:10:24.788312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.076690+0000","last_change":"2024-09-17T04:33:53.085781+0000","last_active":"2024-09-17T04:33:54.076690+0000","last_peered":"2024-09-17T04:33:54.076690+0000","last_clean":"2024-09-17T04:33:54.076690+0000","last_became_active":"2024-09-17T04:33:53.079708+0000","last_became_peered":"2024-09-17T04:33:53.079708+0000","last_unstale":"2024-09-17T04:33:54.076690+0000","last_undegraded":"2024-09-17T04:33:54.076690+0000","last_fullsized":"2024-09-17T04:33:54.076690+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:47.163389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.245383+0000","last_change":"2024-09-17T04:33:53.083676+0000","last_active":"2024-09-17T04:33:54.245383+0000","last_peered":"2024-09-17T04:33:54.245383+0000","last_clean":"2024-09-17T04:33:54.245383+0000","last_became_active":"2024-09-17T04:33:53.083365+0000","last_became_peered":"2024-09-17T04:33:53.083365+0000","last_unstale":"2024-09-17T04:33:54.245383+0000","last_undegraded":"2024-09-17T04:33:54.245383+0000","last_fullsized":"2024-09-17T04:33:54.245383+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:44.202028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.265673+0000","last_change":"2024-09-17T04:33:53.082909+0000","last_active":"2024-09-17T04:33:54.265673+0000","last_peered":"2024-09-17T04:33:54.265673+0000","last_clean":"2024-09-17T04:33:54.265673+0000","last_became_active":"2024-09-17T04:33:53.082611+0000","last_became_peered":"2024-09-17T04:33:53.082611+0000","last_unstale":"2024-09-17T04:33:54.265673+0000","last_undegraded":"2024-09-17T04:33:54.265673+0000","last_fullsized":"2024-09-17T04:33:54.265673+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:35.003560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.264630+0000","last_change":"2024-09-17T04:33:53.085085+0000","last_active":"2024-09-17T04:33:54.264630+0000","last_peered":"2024-09-17T04:33:54.264630+0000","last_clean":"2024-09-17T04:33:54.264630+0000","last_became_active":"2024-09-17T04:33:53.079090+0000","last_became_peered":"2024-09-17T04:33:53.079090+0000","last_unstale":"2024-09-17T04:33:54.264630+0000","last_undegraded":"2024-09-17T04:33:54.264630+0000","last_fullsized":"2024-09-17T04:33:54.264630+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:24:35.834511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.264425+0000","last_change":"2024-09-17T04:33:53.085621+0000","last_active":"2024-09-17T04:33:54.264425+0000","last_peered":"2024-09-17T04:33:54.264425+0000","last_clean":"2024-09-17T04:33:54.264425+0000","last_became_active":"2024-09-17T04:33:53.084889+0000","last_became_peered":"2024-09-17T04:33:53.084889+0000","last_unstale":"2024-09-17T04:33:54.264425+0000","last_undegraded":"2024-09-17T04:33:54.264425+0000","last_fullsized":"2024-09-17T04:33:54.264425+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:57.268387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.264165+0000","last_change":"2024-09-17T04:33:53.097195+0000","last_active":"2024-09-17T04:33:54.264165+0000","last_peered":"2024-09-17T04:33:54.264165+0000","last_clean":"2024-09-17T04:33:54.264165+0000","last_became_active":"2024-09-17T04:33:53.096895+0000","last_became_peered":"2024-09-17T04:33:53.096895+0000","last_unstale":"2024-09-17T04:33:54.264165+0000","last_undegraded":"2024-09-17T04:33:54.264165+0000","last_fullsized":"2024-09-17T04:33:54.264165+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:05:24.427757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.081294+0000","last_change":"2024-09-17T04:33:53.075201+0000","last_active":"2024-09-17T04:33:54.081294+0000","last_peered":"2024-09-17T04:33:54.081294+0000","last_clean":"2024-09-17T04:33:54.081294+0000","last_became_active":"2024-09-17T04:33:53.074879+0000","last_became_peered":"2024-09-17T04:33:53.074879+0000","last_unstale":"2024-09-17T04:33:54.081294+0000","last_undegraded":"2024-09-17T04:33:54.081294+0000","last_fullsized":"2024-09-17T04:33:54.081294+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:17.021787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.265984+0000","last_change":"2024-09-17T04:33:53.093896+0000","last_active":"2024-09-17T04:33:54.265984+0000","last_peered":"2024-09-17T04:33:54.265984+0000","last_clean":"2024-09-17T04:33:54.265984+0000","last_became_active":"2024-09-17T04:33:53.093567+0000","last_became_peered":"2024-09-17T04:33:53.093567+0000","last_unstale":"2024-09-17T04:33:54.265984+0000","last_undegraded":"2024-09-17T04:33:54.265984+0000","last_fullsized":"2024-09-17T04:33:54.265984+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:47.484416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.076717+0000","last_change":"2024-09-17T04:33:53.079303+0000","last_active":"2024-09-17T04:33:54.076717+0000","last_peered":"2024-09-17T04:33:54.076717+0000","last_clean":"2024-09-17T04:33:54.076717+0000","last_became_active":"2024-09-17T04:33:53.078345+0000","last_became_peered":"2024-09-17T04:33:53.078345+0000","last_unstale":"2024-09-17T04:33:54.076717+0000","last_undegraded":"2024-09-17T04:33:54.076717+0000","last_fullsized":"2024-09-17T04:33:54.076717+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:42.106270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.265060+0000","last_change":"2024-09-17T04:33:53.088640+0000","last_active":"2024-09-17T04:33:54.265060+0000","last_peered":"2024-09-17T04:33:54.265060+0000","last_clean":"2024-09-17T04:33:54.265060+0000","last_became_active":"2024-09-17T04:33:53.087503+0000","last_became_peered":"2024-09-17T04:33:53.087503+0000","last_unstale":"2024-09-17T04:33:54.265060+0000","last_undegraded":"2024-09-17T04:33:54.265060+0000","last_fullsized":"2024-09-17T04:33:54.265060+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:49.266912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.265226+0000","last_change":"2024-09-17T04:33:53.091913+0000","last_active":"2024-09-17T04:33:54.265226+0000","last_peered":"2024-09-17T04:33:54.265226+0000","last_clean":"2024-09-17T04:33:54.265226+0000","last_became_active":"2024-09-17T04:33:53.091297+0000","last_became_peered":"2024-09-17T04:33:53.091297+0000","last_unstale":"2024-09-17T04:33:54.265226+0000","last_undegraded":"2024-09-17T04:33:54.265226+0000","last_fullsized":"2024-09-17T04:33:54.265226+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:41.612703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.264501+0000","last_change":"2024-09-17T04:33:53.097690+0000","last_active":"2024-09-17T04:33:54.264501+0000","last_peered":"2024-09-17T04:33:54.264501+0000","last_clean":"2024-09-17T04:33:54.264501+0000","last_became_active":"2024-09-17T04:33:53.097388+0000","last_became_peered":"2024-09-17T04:33:53.097388+0000","last_unstale":"2024-09-17T04:33:54.264501+0000","last_undegraded":"2024-09-17T04:33:54.264501+0000","last_fullsized":"2024-09-17T04:33:54.264501+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:59.476505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.264451+0000","last_change":"2024-09-17T04:33:53.085244+0000","last_active":"2024-09-17T04:33:54.264451+0000","last_peered":"2024-09-17T04:33:54.264451+0000","last_clean":"2024-09-17T04:33:54.264451+0000","last_became_active":"2024-09-17T04:33:53.084457+0000","last_became_peered":"2024-09-17T04:33:53.084457+0000","last_unstale":"2024-09-17T04:33:54.264451+0000","last_undegraded":"2024-09-17T04:33:54.264451+0000","last_fullsized":"2024-09-17T04:33:54.264451+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:40:27.971272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.264996+0000","last_change":"2024-09-17T04:33:53.079723+0000","last_active":"2024-09-17T04:33:54.264996+0000","last_peered":"2024-09-17T04:33:54.264996+0000","last_clean":"2024-09-17T04:33:54.264996+0000","last_became_active":"2024-09-17T04:33:53.079442+0000","last_became_peered":"2024-09-17T04:33:53.079442+0000","last_unstale":"2024-09-17T04:33:54.264996+0000","last_undegraded":"2024-09-17T04:33:54.264996+0000","last_fullsized":"2024-09-17T04:33:54.264996+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:26:53.024711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.265718+0000","last_change":"2024-09-17T04:33:53.091531+0000","last_active":"2024-09-17T04:33:54.265718+0000","last_peered":"2024-09-17T04:33:54.265718+0000","last_clean":"2024-09-17T04:33:54.265718+0000","last_became_active":"2024-09-17T04:33:53.091205+0000","last_became_peered":"2024-09-17T04:33:53.091205+0000","last_unstale":"2024-09-17T04:33:54.265718+0000","last_undegraded":"2024-09-17T04:33:54.265718+0000","last_fullsized":"2024-09-17T04:33:54.265718+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:53.132668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.267158+0000","last_change":"2024-09-17T04:33:53.077820+0000","last_active":"2024-09-17T04:33:54.267158+0000","last_peered":"2024-09-17T04:33:54.267158+0000","last_clean":"2024-09-17T04:33:54.267158+0000","last_became_active":"2024-09-17T04:33:53.077375+0000","last_became_peered":"2024-09-17T04:33:53.077375+0000","last_unstale":"2024-09-17T04:33:54.267158+0000","last_undegraded":"2024-09-17T04:33:54.267158+0000","last_fullsized":"2024-09-17T04:33:54.267158+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:55.805853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.266467+0000","last_change":"2024-09-17T04:33:53.094883+0000","last_active":"2024-09-17T04:33:54.266467+0000","last_peered":"2024-09-17T04:33:54.266467+0000","last_clean":"2024-09-17T04:33:54.266467+0000","last_became_active":"2024-09-17T04:33:53.093711+0000","last_became_peered":"2024-09-17T04:33:53.093711+0000","last_unstale":"2024-09-17T04:33:54.266467+0000","last_undegraded":"2024-09-17T04:33:54.266467+0000","last_fullsized":"2024-09-17T04:33:54.266467+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:52:28.816731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.265157+0000","last_change":"2024-09-17T04:33:53.083128+0000","last_active":"2024-09-17T04:33:54.265157+0000","last_peered":"2024-09-17T04:33:54.265157+0000","last_clean":"2024-09-17T04:33:54.265157+0000","last_became_active":"2024-09-17T04:33:53.082766+0000","last_became_peered":"2024-09-17T04:33:53.082766+0000","last_unstale":"2024-09-17T04:33:54.265157+0000","last_undegraded":"2024-09-17T04:33:54.265157+0000","last_fullsized":"2024-09-17T04:33:54.265157+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:43:26.015365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.245790+0000","last_change":"2024-09-17T04:33:53.088884+0000","last_active":"2024-09-17T04:33:54.245790+0000","last_peered":"2024-09-17T04:33:54.245790+0000","last_clean":"2024-09-17T04:33:54.245790+0000","last_became_active":"2024-09-17T04:33:53.088518+0000","last_became_peered":"2024-09-17T04:33:53.088518+0000","last_unstale":"2024-09-17T04:33:54.245790+0000","last_undegraded":"2024-09-17T04:33:54.245790+0000","last_fullsized":"2024-09-17T04:33:54.245790+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:16:52.499944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.080536+0000","last_change":"2024-09-17T04:33:53.064231+0000","last_active":"2024-09-17T04:33:54.080536+0000","last_peered":"2024-09-17T04:33:54.080536+0000","last_clean":"2024-09-17T04:33:54.080536+0000","last_became_active":"2024-09-17T04:33:53.063250+0000","last_became_peered":"2024-09-17T04:33:53.063250+0000","last_unstale":"2024-09-17T04:33:54.080536+0000","last_undegraded":"2024-09-17T04:33:54.080536+0000","last_fullsized":"2024-09-17T04:33:54.080536+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:30.174645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.079931+0000","last_change":"2024-09-17T04:33:53.091440+0000","last_active":"2024-09-17T04:33:54.079931+0000","last_peered":"2024-09-17T04:33:54.079931+0000","last_clean":"2024-09-17T04:33:54.079931+0000","last_became_active":"2024-09-17T04:33:53.090725+0000","last_became_peered":"2024-09-17T04:33:53.090725+0000","last_unstale":"2024-09-17T04:33:54.079931+0000","last_undegraded":"2024-09-17T04:33:54.079931+0000","last_fullsized":"2024-09-17T04:33:54.079931+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:04.632130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.245283+0000","last_change":"2024-09-17T04:33:53.080472+0000","last_active":"2024-09-17T04:33:54.245283+0000","last_peered":"2024-09-17T04:33:54.245283+0000","last_clean":"2024-09-17T04:33:54.245283+0000","last_became_active":"2024-09-17T04:33:53.079847+0000","last_became_peered":"2024-09-17T04:33:53.079847+0000","last_unstale":"2024-09-17T04:33:54.245283+0000","last_undegraded":"2024-09-17T04:33:54.245283+0000","last_fullsized":"2024-09-17T04:33:54.245283+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:42:11.571082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.080994+0000","last_change":"2024-09-17T04:33:53.070934+0000","last_active":"2024-09-17T04:33:54.080994+0000","last_peered":"2024-09-17T04:33:54.080994+0000","last_clean":"2024-09-17T04:33:54.080994+0000","last_became_active":"2024-09-17T04:33:53.070051+0000","last_became_peered":"2024-09-17T04:33:53.070051+0000","last_unstale":"2024-09-17T04:33:54.080994+0000","last_undegraded":"2024-09-17T04:33:54.080994+0000","last_fullsized":"2024-09-17T04:33:54.080994+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:16:30.611610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.265473+0000","last_change":"2024-09-17T04:33:53.085807+0000","last_active":"2024-09-17T04:33:54.265473+0000","last_peered":"2024-09-17T04:33:54.265473+0000","last_clean":"2024-09-17T04:33:54.265473+0000","last_became_active":"2024-09-17T04:33:53.085521+0000","last_became_peered":"2024-09-17T04:33:53.085521+0000","last_unstale":"2024-09-17T04:33:54.265473+0000","last_undegraded":"2024-09-17T04:33:54.265473+0000","last_fullsized":"2024-09-17T04:33:54.265473+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:01.047446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.265146+0000","last_change":"2024-09-17T04:33:53.090093+0000","last_active":"2024-09-17T04:33:54.265146+0000","last_peered":"2024-09-17T04:33:54.265146+0000","last_clean":"2024-09-17T04:33:54.265146+0000","last_became_active":"2024-09-17T04:33:53.089343+0000","last_became_peered":"2024-09-17T04:33:53.089343+0000","last_unstale":"2024-09-17T04:33:54.265146+0000","last_undegraded":"2024-09-17T04:33:54.265146+0000","last_fullsized":"2024-09-17T04:33:54.265146+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:42.565011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.076304+0000","last_change":"2024-09-17T04:33:53.095666+0000","last_active":"2024-09-17T04:33:54.076304+0000","last_peered":"2024-09-17T04:33:54.076304+0000","last_clean":"2024-09-17T04:33:54.076304+0000","last_became_active":"2024-09-17T04:33:53.094497+0000","last_became_peered":"2024-09-17T04:33:53.094497+0000","last_unstale":"2024-09-17T04:33:54.076304+0000","last_undegraded":"2024-09-17T04:33:54.076304+0000","last_fullsized":"2024-09-17T04:33:54.076304+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:06.670977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.264783+0000","last_change":"2024-09-17T04:33:53.088214+0000","last_active":"2024-09-17T04:33:54.264783+0000","last_peered":"2024-09-17T04:33:54.264783+0000","last_clean":"2024-09-17T04:33:54.264783+0000","last_became_active":"2024-09-17T04:33:53.087868+0000","last_became_peered":"2024-09-17T04:33:53.087868+0000","last_unstale":"2024-09-17T04:33:54.264783+0000","last_undegraded":"2024-09-17T04:33:54.264783+0000","last_fullsized":"2024-09-17T04:33:54.264783+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:35:37.917120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.079731+0000","last_change":"2024-09-17T04:33:53.083812+0000","last_active":"2024-09-17T04:33:54.079731+0000","last_peered":"2024-09-17T04:33:54.079731+0000","last_clean":"2024-09-17T04:33:54.079731+0000","last_became_active":"2024-09-17T04:33:53.083439+0000","last_became_peered":"2024-09-17T04:33:53.083439+0000","last_unstale":"2024-09-17T04:33:54.079731+0000","last_undegraded":"2024-09-17T04:33:54.079731+0000","last_fullsized":"2024-09-17T04:33:54.079731+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:12:26.729194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.265840+0000","last_change":"2024-09-17T04:33:53.069895+0000","last_active":"2024-09-17T04:33:54.265840+0000","last_peered":"2024-09-17T04:33:54.265840+0000","last_clean":"2024-09-17T04:33:54.265840+0000","last_became_active":"2024-09-17T04:33:53.069466+0000","last_became_peered":"2024-09-17T04:33:53.069466+0000","last_unstale":"2024-09-17T04:33:54.265840+0000","last_undegraded":"2024-09-17T04:33:54.265840+0000","last_fullsized":"2024-09-17T04:33:54.265840+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:08.838520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.264280+0000","last_change":"2024-09-17T04:33:53.093862+0000","last_active":"2024-09-17T04:33:54.264280+0000","last_peered":"2024-09-17T04:33:54.264280+0000","last_clean":"2024-09-17T04:33:54.264280+0000","last_became_active":"2024-09-17T04:33:53.086361+0000","last_became_peered":"2024-09-17T04:33:53.086361+0000","last_unstale":"2024-09-17T04:33:54.264280+0000","last_undegraded":"2024-09-17T04:33:54.264280+0000","last_fullsized":"2024-09-17T04:33:54.264280+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:22.488407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.081405+0000","last_change":"2024-09-17T04:33:53.077246+0000","last_active":"2024-09-17T04:33:54.081405+0000","last_peered":"2024-09-17T04:33:54.081405+0000","last_clean":"2024-09-17T04:33:54.081405+0000","last_became_active":"2024-09-17T04:33:53.075727+0000","last_became_peered":"2024-09-17T04:33:53.075727+0000","last_unstale":"2024-09-17T04:33:54.081405+0000","last_undegraded":"2024-09-17T04:33:54.081405+0000","last_fullsized":"2024-09-17T04:33:54.081405+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:37.175879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"166'4","reported_seq":19,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.265147+0000","last_change":"2024-09-17T04:33:52.070858+0000","last_active":"2024-09-17T04:33:54.265147+0000","last_peered":"2024-09-17T04:33:54.265147+0000","last_clean":"2024-09-17T04:33:54.265147+0000","last_became_active":"2024-09-17T04:33:52.070531+0000","last_became_peered":"2024-09-17T04:33:52.070531+0000","last_unstale":"2024-09-17T04:33:54.265147+0000","last_undegraded":"2024-09-17T04:33:54.265147+0000","last_fullsized":"2024-09-17T04:33:54.265147+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:24:13.910475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.079812+0000","last_change":"2024-09-17T04:33:53.077915+0000","last_active":"2024-09-17T04:33:54.079812+0000","last_peered":"2024-09-17T04:33:54.079812+0000","last_clean":"2024-09-17T04:33:54.079812+0000","last_became_active":"2024-09-17T04:33:53.077589+0000","last_became_peered":"2024-09-17T04:33:53.077589+0000","last_unstale":"2024-09-17T04:33:54.079812+0000","last_undegraded":"2024-09-17T04:33:54.079812+0000","last_fullsized":"2024-09-17T04:33:54.079812+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:51:09.820513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"166'1","reported_seq":16,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.081196+0000","last_change":"2024-09-17T04:33:52.071769+0000","last_active":"2024-09-17T04:33:54.081196+0000","last_peered":"2024-09-17T04:33:54.081196+0000","last_clean":"2024-09-17T04:33:54.081196+0000","last_became_active":"2024-09-17T04:33:52.071291+0000","last_became_peered":"2024-09-17T04:33:52.071291+0000","last_unstale":"2024-09-17T04:33:54.081196+0000","last_undegraded":"2024-09-17T04:33:54.081196+0000","last_fullsized":"2024-09-17T04:33:54.081196+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:53.622152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.265457+0000","last_change":"2024-09-17T04:33:53.085165+0000","last_active":"2024-09-17T04:33:54.265457+0000","last_peered":"2024-09-17T04:33:54.265457+0000","last_clean":"2024-09-17T04:33:54.265457+0000","last_became_active":"2024-09-17T04:33:53.084872+0000","last_became_peered":"2024-09-17T04:33:53.084872+0000","last_unstale":"2024-09-17T04:33:54.265457+0000","last_undegraded":"2024-09-17T04:33:54.265457+0000","last_fullsized":"2024-09-17T04:33:54.265457+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:34:29.682850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"166'2","reported_seq":17,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.076752+0000","last_change":"2024-09-17T04:33:52.075035+0000","last_active":"2024-09-17T04:33:54.076752+0000","last_peered":"2024-09-17T04:33:54.076752+0000","last_clean":"2024-09-17T04:33:54.076752+0000","last_became_active":"2024-09-17T04:33:52.074280+0000","last_became_peered":"2024-09-17T04:33:52.074280+0000","last_unstale":"2024-09-17T04:33:54.076752+0000","last_undegraded":"2024-09-17T04:33:54.076752+0000","last_fullsized":"2024-09-17T04:33:54.076752+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:25:52.288780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.266678+0000","last_change":"2024-09-17T04:33:53.075823+0000","last_active":"2024-09-17T04:33:54.266678+0000","last_peered":"2024-09-17T04:33:54.266678+0000","last_clean":"2024-09-17T04:33:54.266678+0000","last_became_active":"2024-09-17T04:33:53.075479+0000","last_became_peered":"2024-09-17T04:33:53.075479+0000","last_unstale":"2024-09-17T04:33:54.266678+0000","last_undegraded":"2024-09-17T04:33:54.266678+0000","last_fullsized":"2024-09-17T04:33:54.266678+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:41:51.777749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"166'2","reported_seq":17,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.245495+0000","last_change":"2024-09-17T04:33:52.076560+0000","last_active":"2024-09-17T04:33:54.245495+0000","last_peered":"2024-09-17T04:33:54.245495+0000","last_clean":"2024-09-17T04:33:54.245495+0000","last_became_active":"2024-09-17T04:33:52.076104+0000","last_became_peered":"2024-09-17T04:33:52.076104+0000","last_unstale":"2024-09-17T04:33:54.245495+0000","last_undegraded":"2024-09-17T04:33:54.245495+0000","last_fullsized":"2024-09-17T04:33:54.245495+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:25:38.292881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.245605+0000","last_change":"2024-09-17T04:33:53.081681+0000","last_active":"2024-09-17T04:33:54.245605+0000","last_peered":"2024-09-17T04:33:54.245605+0000","last_clean":"2024-09-17T04:33:54.245605+0000","last_became_active":"2024-09-17T04:33:53.081319+0000","last_became_peered":"2024-09-17T04:33:53.081319+0000","last_unstale":"2024-09-17T04:33:54.245605+0000","last_undegraded":"2024-09-17T04:33:54.245605+0000","last_fullsized":"2024-09-17T04:33:54.245605+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:33:42.273632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"166'1","reported_seq":16,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.264757+0000","last_change":"2024-09-17T04:33:52.067069+0000","last_active":"2024-09-17T04:33:54.264757+0000","last_peered":"2024-09-17T04:33:54.264757+0000","last_clean":"2024-09-17T04:33:54.264757+0000","last_became_active":"2024-09-17T04:33:52.066808+0000","last_became_peered":"2024-09-17T04:33:52.066808+0000","last_unstale":"2024-09-17T04:33:54.264757+0000","last_undegraded":"2024-09-17T04:33:54.264757+0000","last_fullsized":"2024-09-17T04:33:54.264757+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:12.166454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.265061+0000","last_change":"2024-09-17T04:33:53.080495+0000","last_active":"2024-09-17T04:33:54.265061+0000","last_peered":"2024-09-17T04:33:54.265061+0000","last_clean":"2024-09-17T04:33:54.265061+0000","last_became_active":"2024-09-17T04:33:53.080168+0000","last_became_peered":"2024-09-17T04:33:53.080168+0000","last_unstale":"2024-09-17T04:33:54.265061+0000","last_undegraded":"2024-09-17T04:33:54.265061+0000","last_fullsized":"2024-09-17T04:33:54.265061+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:49.526976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.264963+0000","last_change":"2024-09-17T04:33:52.073121+0000","last_active":"2024-09-17T04:33:54.264963+0000","last_peered":"2024-09-17T04:33:54.264963+0000","last_clean":"2024-09-17T04:33:54.264963+0000","last_became_active":"2024-09-17T04:33:52.072707+0000","last_became_peered":"2024-09-17T04:33:52.072707+0000","last_unstale":"2024-09-17T04:33:54.264963+0000","last_undegraded":"2024-09-17T04:33:54.264963+0000","last_fullsized":"2024-09-17T04:33:54.264963+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:44:12.462165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.245878+0000","last_change":"2024-09-17T04:33:53.073180+0000","last_active":"2024-09-17T04:33:54.245878+0000","last_peered":"2024-09-17T04:33:54.245878+0000","last_clean":"2024-09-17T04:33:54.245878+0000","last_became_active":"2024-09-17T04:33:53.072904+0000","last_became_peered":"2024-09-17T04:33:53.072904+0000","last_unstale":"2024-09-17T04:33:54.245878+0000","last_undegraded":"2024-09-17T04:33:54.245878+0000","last_fullsized":"2024-09-17T04:33:54.245878+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:46.018098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.079376+0000","last_change":"2024-09-17T04:33:52.074862+0000","last_active":"2024-09-17T04:33:54.079376+0000","last_peered":"2024-09-17T04:33:54.079376+0000","last_clean":"2024-09-17T04:33:54.079376+0000","last_became_active":"2024-09-17T04:33:52.074599+0000","last_became_peered":"2024-09-17T04:33:52.074599+0000","last_unstale":"2024-09-17T04:33:54.079376+0000","last_undegraded":"2024-09-17T04:33:54.079376+0000","last_fullsized":"2024-09-17T04:33:54.079376+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:36:01.875578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.245653+0000","last_change":"2024-09-17T04:33:53.085897+0000","last_active":"2024-09-17T04:33:54.245653+0000","last_peered":"2024-09-17T04:33:54.245653+0000","last_clean":"2024-09-17T04:33:54.245653+0000","last_became_active":"2024-09-17T04:33:53.085497+0000","last_became_peered":"2024-09-17T04:33:53.085497+0000","last_unstale":"2024-09-17T04:33:54.245653+0000","last_undegraded":"2024-09-17T04:33:54.245653+0000","last_fullsized":"2024-09-17T04:33:54.245653+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:13.601401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.265521+0000","last_change":"2024-09-17T04:33:52.069125+0000","last_active":"2024-09-17T04:33:54.265521+0000","last_peered":"2024-09-17T04:33:54.265521+0000","last_clean":"2024-09-17T04:33:54.265521+0000","last_became_active":"2024-09-17T04:33:52.068663+0000","last_became_peered":"2024-09-17T04:33:52.068663+0000","last_unstale":"2024-09-17T04:33:54.265521+0000","last_undegraded":"2024-09-17T04:33:54.265521+0000","last_fullsized":"2024-09-17T04:33:54.265521+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:48:18.196110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.076335+0000","last_change":"2024-09-17T04:33:53.074274+0000","last_active":"2024-09-17T04:33:54.076335+0000","last_peered":"2024-09-17T04:33:54.076335+0000","last_clean":"2024-09-17T04:33:54.076335+0000","last_became_active":"2024-09-17T04:33:53.073805+0000","last_became_peered":"2024-09-17T04:33:53.073805+0000","last_unstale":"2024-09-17T04:33:54.076335+0000","last_undegraded":"2024-09-17T04:33:54.076335+0000","last_fullsized":"2024-09-17T04:33:54.076335+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:15.481878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.080689+0000","last_change":"2024-09-17T04:33:52.068300+0000","last_active":"2024-09-17T04:33:54.080689+0000","last_peered":"2024-09-17T04:33:54.080689+0000","last_clean":"2024-09-17T04:33:54.080689+0000","last_became_active":"2024-09-17T04:33:52.067983+0000","last_became_peered":"2024-09-17T04:33:52.067983+0000","last_unstale":"2024-09-17T04:33:54.080689+0000","last_undegraded":"2024-09-17T04:33:54.080689+0000","last_fullsized":"2024-09-17T04:33:54.080689+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:36.173088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.245971+0000","last_change":"2024-09-17T04:33:53.079220+0000","last_active":"2024-09-17T04:33:54.245971+0000","last_peered":"2024-09-17T04:33:54.245971+0000","last_clean":"2024-09-17T04:33:54.245971+0000","last_became_active":"2024-09-17T04:33:53.077868+0000","last_became_peered":"2024-09-17T04:33:53.077868+0000","last_unstale":"2024-09-17T04:33:54.245971+0000","last_undegraded":"2024-09-17T04:33:54.245971+0000","last_fullsized":"2024-09-17T04:33:54.245971+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:17.493801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"166'3","reported_seq":18,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.266501+0000","last_change":"2024-09-17T04:33:52.070705+0000","last_active":"2024-09-17T04:33:54.266501+0000","last_peered":"2024-09-17T04:33:54.266501+0000","last_clean":"2024-09-17T04:33:54.266501+0000","last_became_active":"2024-09-17T04:33:52.068177+0000","last_became_peered":"2024-09-17T04:33:52.068177+0000","last_unstale":"2024-09-17T04:33:54.266501+0000","last_undegraded":"2024-09-17T04:33:54.266501+0000","last_fullsized":"2024-09-17T04:33:54.266501+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:13.995128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.265472+0000","last_change":"2024-09-17T04:33:53.096282+0000","last_active":"2024-09-17T04:33:54.265472+0000","last_peered":"2024-09-17T04:33:54.265472+0000","last_clean":"2024-09-17T04:33:54.265472+0000","last_became_active":"2024-09-17T04:33:53.095970+0000","last_became_peered":"2024-09-17T04:33:53.095970+0000","last_unstale":"2024-09-17T04:33:54.265472+0000","last_undegraded":"2024-09-17T04:33:54.265472+0000","last_fullsized":"2024-09-17T04:33:54.265472+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:30:41.866927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"166'6","reported_seq":21,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.245624+0000","last_change":"2024-09-17T04:33:52.076550+0000","last_active":"2024-09-17T04:33:54.245624+0000","last_peered":"2024-09-17T04:33:54.245624+0000","last_clean":"2024-09-17T04:33:54.245624+0000","last_became_active":"2024-09-17T04:33:52.073604+0000","last_became_peered":"2024-09-17T04:33:52.073604+0000","last_unstale":"2024-09-17T04:33:54.245624+0000","last_undegraded":"2024-09-17T04:33:54.245624+0000","last_fullsized":"2024-09-17T04:33:54.245624+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:29.171853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.265924+0000","last_change":"2024-09-17T04:33:53.073224+0000","last_active":"2024-09-17T04:33:54.265924+0000","last_peered":"2024-09-17T04:33:54.265924+0000","last_clean":"2024-09-17T04:33:54.265924+0000","last_became_active":"2024-09-17T04:33:53.072940+0000","last_became_peered":"2024-09-17T04:33:53.072940+0000","last_unstale":"2024-09-17T04:33:54.265924+0000","last_undegraded":"2024-09-17T04:33:54.265924+0000","last_fullsized":"2024-09-17T04:33:54.265924+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:50.149920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"166'2","reported_seq":17,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.080419+0000","last_change":"2024-09-17T04:33:52.073611+0000","last_active":"2024-09-17T04:33:54.080419+0000","last_peered":"2024-09-17T04:33:54.080419+0000","last_clean":"2024-09-17T04:33:54.080419+0000","last_became_active":"2024-09-17T04:33:52.073282+0000","last_became_peered":"2024-09-17T04:33:52.073282+0000","last_unstale":"2024-09-17T04:33:54.080419+0000","last_undegraded":"2024-09-17T04:33:54.080419+0000","last_fullsized":"2024-09-17T04:33:54.080419+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:16.018995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.266223+0000","last_change":"2024-09-17T04:33:53.065033+0000","last_active":"2024-09-17T04:33:54.266223+0000","last_peered":"2024-09-17T04:33:54.266223+0000","last_clean":"2024-09-17T04:33:54.266223+0000","last_became_active":"2024-09-17T04:33:53.064731+0000","last_became_peered":"2024-09-17T04:33:53.064731+0000","last_unstale":"2024-09-17T04:33:54.266223+0000","last_undegraded":"2024-09-17T04:33:54.266223+0000","last_fullsized":"2024-09-17T04:33:54.266223+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:23.330650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.080399+0000","last_change":"2024-09-17T04:33:52.077007+0000","last_active":"2024-09-17T04:33:54.080399+0000","last_peered":"2024-09-17T04:33:54.080399+0000","last_clean":"2024-09-17T04:33:54.080399+0000","last_became_active":"2024-09-17T04:33:52.075600+0000","last_became_peered":"2024-09-17T04:33:52.075600+0000","last_unstale":"2024-09-17T04:33:54.080399+0000","last_undegraded":"2024-09-17T04:33:54.080399+0000","last_fullsized":"2024-09-17T04:33:54.080399+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:18.430477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.076008+0000","last_change":"2024-09-17T04:33:53.088715+0000","last_active":"2024-09-17T04:33:54.076008+0000","last_peered":"2024-09-17T04:33:54.076008+0000","last_clean":"2024-09-17T04:33:54.076008+0000","last_became_active":"2024-09-17T04:33:53.081407+0000","last_became_peered":"2024-09-17T04:33:53.081407+0000","last_unstale":"2024-09-17T04:33:54.076008+0000","last_undegraded":"2024-09-17T04:33:54.076008+0000","last_fullsized":"2024-09-17T04:33:54.076008+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:27:58.542200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.265905+0000","last_change":"2024-09-17T04:33:52.075175+0000","last_active":"2024-09-17T04:33:54.265905+0000","last_peered":"2024-09-17T04:33:54.265905+0000","last_clean":"2024-09-17T04:33:54.265905+0000","last_became_active":"2024-09-17T04:33:52.074460+0000","last_became_peered":"2024-09-17T04:33:52.074460+0000","last_unstale":"2024-09-17T04:33:54.265905+0000","last_undegraded":"2024-09-17T04:33:54.265905+0000","last_fullsized":"2024-09-17T04:33:54.265905+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:00:29.393704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.264656+0000","last_change":"2024-09-17T04:33:53.083407+0000","last_active":"2024-09-17T04:33:54.264656+0000","last_peered":"2024-09-17T04:33:54.264656+0000","last_clean":"2024-09-17T04:33:54.264656+0000","last_became_active":"2024-09-17T04:33:53.082934+0000","last_became_peered":"2024-09-17T04:33:53.082934+0000","last_unstale":"2024-09-17T04:33:54.264656+0000","last_undegraded":"2024-09-17T04:33:54.264656+0000","last_fullsized":"2024-09-17T04:33:54.264656+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:18:59.846653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"166'8","reported_seq":23,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.076087+0000","last_change":"2024-09-17T04:33:52.077352+0000","last_active":"2024-09-17T04:33:54.076087+0000","last_peered":"2024-09-17T04:33:54.076087+0000","last_clean":"2024-09-17T04:33:54.076087+0000","last_became_active":"2024-09-17T04:33:52.077004+0000","last_became_peered":"2024-09-17T04:33:52.077004+0000","last_unstale":"2024-09-17T04:33:54.076087+0000","last_undegraded":"2024-09-17T04:33:54.076087+0000","last_fullsized":"2024-09-17T04:33:54.076087+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:52.892282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.081112+0000","last_change":"2024-09-17T04:33:53.080022+0000","last_active":"2024-09-17T04:33:54.081112+0000","last_peered":"2024-09-17T04:33:54.081112+0000","last_clean":"2024-09-17T04:33:54.081112+0000","last_became_active":"2024-09-17T04:33:53.076827+0000","last_became_peered":"2024-09-17T04:33:53.076827+0000","last_unstale":"2024-09-17T04:33:54.081112+0000","last_undegraded":"2024-09-17T04:33:54.081112+0000","last_fullsized":"2024-09-17T04:33:54.081112+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:27.289230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"166'1","reported_seq":16,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.265667+0000","last_change":"2024-09-17T04:33:52.079089+0000","last_active":"2024-09-17T04:33:54.265667+0000","last_peered":"2024-09-17T04:33:54.265667+0000","last_clean":"2024-09-17T04:33:54.265667+0000","last_became_active":"2024-09-17T04:33:52.078678+0000","last_became_peered":"2024-09-17T04:33:52.078678+0000","last_unstale":"2024-09-17T04:33:54.265667+0000","last_undegraded":"2024-09-17T04:33:54.265667+0000","last_fullsized":"2024-09-17T04:33:54.265667+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:35:36.914329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.081171+0000","last_change":"2024-09-17T04:33:53.071430+0000","last_active":"2024-09-17T04:33:54.081171+0000","last_peered":"2024-09-17T04:33:54.081171+0000","last_clean":"2024-09-17T04:33:54.081171+0000","last_became_active":"2024-09-17T04:33:53.070540+0000","last_became_peered":"2024-09-17T04:33:53.070540+0000","last_unstale":"2024-09-17T04:33:54.081171+0000","last_undegraded":"2024-09-17T04:33:54.081171+0000","last_fullsized":"2024-09-17T04:33:54.081171+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:36:02.878369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"166'2","reported_seq":17,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.081315+0000","last_change":"2024-09-17T04:33:52.069688+0000","last_active":"2024-09-17T04:33:54.081315+0000","last_peered":"2024-09-17T04:33:54.081315+0000","last_clean":"2024-09-17T04:33:54.081315+0000","last_became_active":"2024-09-17T04:33:52.068489+0000","last_became_peered":"2024-09-17T04:33:52.068489+0000","last_unstale":"2024-09-17T04:33:54.081315+0000","last_undegraded":"2024-09-17T04:33:54.081315+0000","last_fullsized":"2024-09-17T04:33:54.081315+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:51.882108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.076402+0000","last_change":"2024-09-17T04:33:53.077063+0000","last_active":"2024-09-17T04:33:54.076402+0000","last_peered":"2024-09-17T04:33:54.076402+0000","last_clean":"2024-09-17T04:33:54.076402+0000","last_became_active":"2024-09-17T04:33:53.076765+0000","last_became_peered":"2024-09-17T04:33:53.076765+0000","last_unstale":"2024-09-17T04:33:54.076402+0000","last_undegraded":"2024-09-17T04:33:54.076402+0000","last_fullsized":"2024-09-17T04:33:54.076402+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:52.553565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"166'1","reported_seq":16,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.266556+0000","last_change":"2024-09-17T04:33:52.072986+0000","last_active":"2024-09-17T04:33:54.266556+0000","last_peered":"2024-09-17T04:33:54.266556+0000","last_clean":"2024-09-17T04:33:54.266556+0000","last_became_active":"2024-09-17T04:33:52.072270+0000","last_became_peered":"2024-09-17T04:33:52.072270+0000","last_unstale":"2024-09-17T04:33:54.266556+0000","last_undegraded":"2024-09-17T04:33:54.266556+0000","last_fullsized":"2024-09-17T04:33:54.266556+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:30:33.427812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.266355+0000","last_change":"2024-09-17T04:33:53.068453+0000","last_active":"2024-09-17T04:33:54.266355+0000","last_peered":"2024-09-17T04:33:54.266355+0000","last_clean":"2024-09-17T04:33:54.266355+0000","last_became_active":"2024-09-17T04:33:53.067455+0000","last_became_peered":"2024-09-17T04:33:53.067455+0000","last_unstale":"2024-09-17T04:33:54.266355+0000","last_undegraded":"2024-09-17T04:33:54.266355+0000","last_fullsized":"2024-09-17T04:33:54.266355+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:21:12.318935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.076373+0000","last_change":"2024-09-17T04:33:52.073971+0000","last_active":"2024-09-17T04:33:54.076373+0000","last_peered":"2024-09-17T04:33:54.076373+0000","last_clean":"2024-09-17T04:33:54.076373+0000","last_became_active":"2024-09-17T04:33:52.073464+0000","last_became_peered":"2024-09-17T04:33:52.073464+0000","last_unstale":"2024-09-17T04:33:54.076373+0000","last_undegraded":"2024-09-17T04:33:54.076373+0000","last_fullsized":"2024-09-17T04:33:54.076373+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:45.690759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.266101+0000","last_change":"2024-09-17T04:33:53.081131+0000","last_active":"2024-09-17T04:33:54.266101+0000","last_peered":"2024-09-17T04:33:54.266101+0000","last_clean":"2024-09-17T04:33:54.266101+0000","last_became_active":"2024-09-17T04:33:53.080754+0000","last_became_peered":"2024-09-17T04:33:53.080754+0000","last_unstale":"2024-09-17T04:33:54.266101+0000","last_undegraded":"2024-09-17T04:33:54.266101+0000","last_fullsized":"2024-09-17T04:33:54.266101+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:16.004374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"166'2","reported_seq":17,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.266184+0000","last_change":"2024-09-17T04:33:52.074998+0000","last_active":"2024-09-17T04:33:54.266184+0000","last_peered":"2024-09-17T04:33:54.266184+0000","last_clean":"2024-09-17T04:33:54.266184+0000","last_became_active":"2024-09-17T04:33:52.074713+0000","last_became_peered":"2024-09-17T04:33:52.074713+0000","last_unstale":"2024-09-17T04:33:54.266184+0000","last_undegraded":"2024-09-17T04:33:54.266184+0000","last_fullsized":"2024-09-17T04:33:54.266184+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:15:06.261507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.076364+0000","last_change":"2024-09-17T04:33:53.084062+0000","last_active":"2024-09-17T04:33:54.076364+0000","last_peered":"2024-09-17T04:33:54.076364+0000","last_clean":"2024-09-17T04:33:54.076364+0000","last_became_active":"2024-09-17T04:33:53.083811+0000","last_became_peered":"2024-09-17T04:33:53.083811+0000","last_unstale":"2024-09-17T04:33:54.076364+0000","last_undegraded":"2024-09-17T04:33:54.076364+0000","last_fullsized":"2024-09-17T04:33:54.076364+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:24:44.766361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.264829+0000","last_change":"2024-09-17T04:33:52.071769+0000","last_active":"2024-09-17T04:33:54.264829+0000","last_peered":"2024-09-17T04:33:54.264829+0000","last_clean":"2024-09-17T04:33:54.264829+0000","last_became_active":"2024-09-17T04:33:52.070557+0000","last_became_peered":"2024-09-17T04:33:52.070557+0000","last_unstale":"2024-09-17T04:33:54.264829+0000","last_undegraded":"2024-09-17T04:33:54.264829+0000","last_fullsized":"2024-09-17T04:33:54.264829+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:20.321145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.079881+0000","last_change":"2024-09-17T04:33:53.061551+0000","last_active":"2024-09-17T04:33:54.079881+0000","last_peered":"2024-09-17T04:33:54.079881+0000","last_clean":"2024-09-17T04:33:54.079881+0000","last_became_active":"2024-09-17T04:33:53.061165+0000","last_became_peered":"2024-09-17T04:33:53.061165+0000","last_unstale":"2024-09-17T04:33:54.079881+0000","last_undegraded":"2024-09-17T04:33:54.079881+0000","last_fullsized":"2024-09-17T04:33:54.079881+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:46:00.490244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.264524+0000","last_change":"2024-09-17T04:33:52.070461+0000","last_active":"2024-09-17T04:33:54.264524+0000","last_peered":"2024-09-17T04:33:54.264524+0000","last_clean":"2024-09-17T04:33:54.264524+0000","last_became_active":"2024-09-17T04:33:52.070139+0000","last_became_peered":"2024-09-17T04:33:52.070139+0000","last_unstale":"2024-09-17T04:33:54.264524+0000","last_undegraded":"2024-09-17T04:33:54.264524+0000","last_fullsized":"2024-09-17T04:33:54.264524+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:49:03.742868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.080175+0000","last_change":"2024-09-17T04:33:53.071070+0000","last_active":"2024-09-17T04:33:54.080175+0000","last_peered":"2024-09-17T04:33:54.080175+0000","last_clean":"2024-09-17T04:33:54.080175+0000","last_became_active":"2024-09-17T04:33:53.070697+0000","last_became_peered":"2024-09-17T04:33:53.070697+0000","last_unstale":"2024-09-17T04:33:54.080175+0000","last_undegraded":"2024-09-17T04:33:54.080175+0000","last_fullsized":"2024-09-17T04:33:54.080175+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:06.367266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.265125+0000","last_change":"2024-09-17T04:33:52.075993+0000","last_active":"2024-09-17T04:33:54.265125+0000","last_peered":"2024-09-17T04:33:54.265125+0000","last_clean":"2024-09-17T04:33:54.265125+0000","last_became_active":"2024-09-17T04:33:52.075674+0000","last_became_peered":"2024-09-17T04:33:52.075674+0000","last_unstale":"2024-09-17T04:33:54.265125+0000","last_undegraded":"2024-09-17T04:33:54.265125+0000","last_fullsized":"2024-09-17T04:33:54.265125+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:52.601739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.265221+0000","last_change":"2024-09-17T04:33:53.089227+0000","last_active":"2024-09-17T04:33:54.265221+0000","last_peered":"2024-09-17T04:33:54.265221+0000","last_clean":"2024-09-17T04:33:54.265221+0000","last_became_active":"2024-09-17T04:33:53.087807+0000","last_became_peered":"2024-09-17T04:33:53.087807+0000","last_unstale":"2024-09-17T04:33:54.265221+0000","last_undegraded":"2024-09-17T04:33:54.265221+0000","last_fullsized":"2024-09-17T04:33:54.265221+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:45:43.199925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"166'2","reported_seq":17,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.076378+0000","last_change":"2024-09-17T04:33:52.061884+0000","last_active":"2024-09-17T04:33:54.076378+0000","last_peered":"2024-09-17T04:33:54.076378+0000","last_clean":"2024-09-17T04:33:54.076378+0000","last_became_active":"2024-09-17T04:33:52.061556+0000","last_became_peered":"2024-09-17T04:33:52.061556+0000","last_unstale":"2024-09-17T04:33:54.076378+0000","last_undegraded":"2024-09-17T04:33:54.076378+0000","last_fullsized":"2024-09-17T04:33:54.076378+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:55:25.950159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.266112+0000","last_change":"2024-09-17T04:33:53.091303+0000","last_active":"2024-09-17T04:33:54.266112+0000","last_peered":"2024-09-17T04:33:54.266112+0000","last_clean":"2024-09-17T04:33:54.266112+0000","last_became_active":"2024-09-17T04:33:53.090988+0000","last_became_peered":"2024-09-17T04:33:53.090988+0000","last_unstale":"2024-09-17T04:33:54.266112+0000","last_undegraded":"2024-09-17T04:33:54.266112+0000","last_fullsized":"2024-09-17T04:33:54.266112+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:25:32.145082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"167'1","reported_seq":16,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.245562+0000","last_change":"2024-09-17T04:33:52.075365+0000","last_active":"2024-09-17T04:33:54.245562+0000","last_peered":"2024-09-17T04:33:54.245562+0000","last_clean":"2024-09-17T04:33:54.245562+0000","last_became_active":"2024-09-17T04:33:52.075057+0000","last_became_peered":"2024-09-17T04:33:52.075057+0000","last_unstale":"2024-09-17T04:33:54.245562+0000","last_undegraded":"2024-09-17T04:33:54.245562+0000","last_fullsized":"2024-09-17T04:33:54.245562+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:03.629339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.080212+0000","last_change":"2024-09-17T04:33:53.085835+0000","last_active":"2024-09-17T04:33:54.080212+0000","last_peered":"2024-09-17T04:33:54.080212+0000","last_clean":"2024-09-17T04:33:54.080212+0000","last_became_active":"2024-09-17T04:33:53.085505+0000","last_became_peered":"2024-09-17T04:33:53.085505+0000","last_unstale":"2024-09-17T04:33:54.080212+0000","last_undegraded":"2024-09-17T04:33:54.080212+0000","last_fullsized":"2024-09-17T04:33:54.080212+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:44.459712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.265101+0000","last_change":"2024-09-17T04:33:52.073026+0000","last_active":"2024-09-17T04:33:54.265101+0000","last_peered":"2024-09-17T04:33:54.265101+0000","last_clean":"2024-09-17T04:33:54.265101+0000","last_became_active":"2024-09-17T04:33:52.072693+0000","last_became_peered":"2024-09-17T04:33:52.072693+0000","last_unstale":"2024-09-17T04:33:54.265101+0000","last_undegraded":"2024-09-17T04:33:54.265101+0000","last_fullsized":"2024-09-17T04:33:54.265101+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:49:27.707416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.079879+0000","last_change":"2024-09-17T04:33:53.092847+0000","last_active":"2024-09-17T04:33:54.079879+0000","last_peered":"2024-09-17T04:33:54.079879+0000","last_clean":"2024-09-17T04:33:54.079879+0000","last_became_active":"2024-09-17T04:33:53.092074+0000","last_became_peered":"2024-09-17T04:33:53.092074+0000","last_unstale":"2024-09-17T04:33:54.079879+0000","last_undegraded":"2024-09-17T04:33:54.079879+0000","last_fullsized":"2024-09-17T04:33:54.079879+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:18:56.851015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.076479+0000","last_change":"2024-09-17T04:33:52.060331+0000","last_active":"2024-09-17T04:33:54.076479+0000","last_peered":"2024-09-17T04:33:54.076479+0000","last_clean":"2024-09-17T04:33:54.076479+0000","last_became_active":"2024-09-17T04:33:52.060023+0000","last_became_peered":"2024-09-17T04:33:52.060023+0000","last_unstale":"2024-09-17T04:33:54.076479+0000","last_undegraded":"2024-09-17T04:33:54.076479+0000","last_fullsized":"2024-09-17T04:33:54.076479+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:50.893292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.264867+0000","last_change":"2024-09-17T04:33:53.087281+0000","last_active":"2024-09-17T04:33:54.264867+0000","last_peered":"2024-09-17T04:33:54.264867+0000","last_clean":"2024-09-17T04:33:54.264867+0000","last_became_active":"2024-09-17T04:33:53.086968+0000","last_became_peered":"2024-09-17T04:33:53.086968+0000","last_unstale":"2024-09-17T04:33:54.264867+0000","last_undegraded":"2024-09-17T04:33:54.264867+0000","last_fullsized":"2024-09-17T04:33:54.264867+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:34:46.321268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.245687+0000","last_change":"2024-09-17T04:33:52.074159+0000","last_active":"2024-09-17T04:33:54.245687+0000","last_peered":"2024-09-17T04:33:54.245687+0000","last_clean":"2024-09-17T04:33:54.245687+0000","last_became_active":"2024-09-17T04:33:52.073282+0000","last_became_peered":"2024-09-17T04:33:52.073282+0000","last_unstale":"2024-09-17T04:33:54.245687+0000","last_undegraded":"2024-09-17T04:33:54.245687+0000","last_fullsized":"2024-09-17T04:33:54.245687+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:05.364475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.265021+0000","last_change":"2024-09-17T04:33:53.074501+0000","last_active":"2024-09-17T04:33:54.265021+0000","last_peered":"2024-09-17T04:33:54.265021+0000","last_clean":"2024-09-17T04:33:54.265021+0000","last_became_active":"2024-09-17T04:33:53.074172+0000","last_became_peered":"2024-09-17T04:33:53.074172+0000","last_unstale":"2024-09-17T04:33:54.265021+0000","last_undegraded":"2024-09-17T04:33:54.265021+0000","last_fullsized":"2024-09-17T04:33:54.265021+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:53.604531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.264895+0000","last_change":"2024-09-17T04:33:52.078845+0000","last_active":"2024-09-17T04:33:54.264895+0000","last_peered":"2024-09-17T04:33:54.264895+0000","last_clean":"2024-09-17T04:33:54.264895+0000","last_became_active":"2024-09-17T04:33:52.078528+0000","last_became_peered":"2024-09-17T04:33:52.078528+0000","last_unstale":"2024-09-17T04:33:54.264895+0000","last_undegraded":"2024-09-17T04:33:54.264895+0000","last_fullsized":"2024-09-17T04:33:54.264895+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:15:03.636092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.076023+0000","last_change":"2024-09-17T04:33:53.089079+0000","last_active":"2024-09-17T04:33:54.076023+0000","last_peered":"2024-09-17T04:33:54.076023+0000","last_clean":"2024-09-17T04:33:54.076023+0000","last_became_active":"2024-09-17T04:33:53.081793+0000","last_became_peered":"2024-09-17T04:33:53.081793+0000","last_unstale":"2024-09-17T04:33:54.076023+0000","last_undegraded":"2024-09-17T04:33:54.076023+0000","last_fullsized":"2024-09-17T04:33:54.076023+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:39.492930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.080064+0000","last_change":"2024-09-17T04:33:52.072151+0000","last_active":"2024-09-17T04:33:54.080064+0000","last_peered":"2024-09-17T04:33:54.080064+0000","last_clean":"2024-09-17T04:33:54.080064+0000","last_became_active":"2024-09-17T04:33:52.070810+0000","last_became_peered":"2024-09-17T04:33:52.070810+0000","last_unstale":"2024-09-17T04:33:54.080064+0000","last_undegraded":"2024-09-17T04:33:54.080064+0000","last_fullsized":"2024-09-17T04:33:54.080064+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:11:37.348558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.079588+0000","last_change":"2024-09-17T04:33:53.072124+0000","last_active":"2024-09-17T04:33:54.079588+0000","last_peered":"2024-09-17T04:33:54.079588+0000","last_clean":"2024-09-17T04:33:54.079588+0000","last_became_active":"2024-09-17T04:33:53.070819+0000","last_became_peered":"2024-09-17T04:33:53.070819+0000","last_unstale":"2024-09-17T04:33:54.079588+0000","last_undegraded":"2024-09-17T04:33:54.079588+0000","last_fullsized":"2024-09-17T04:33:54.079588+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:05:08.747454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.264606+0000","last_change":"2024-09-17T04:33:52.073059+0000","last_active":"2024-09-17T04:33:54.264606+0000","last_peered":"2024-09-17T04:33:54.264606+0000","last_clean":"2024-09-17T04:33:54.264606+0000","last_became_active":"2024-09-17T04:33:52.072757+0000","last_became_peered":"2024-09-17T04:33:52.072757+0000","last_unstale":"2024-09-17T04:33:54.264606+0000","last_undegraded":"2024-09-17T04:33:54.264606+0000","last_fullsized":"2024-09-17T04:33:54.264606+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:48.723973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.264760+0000","last_change":"2024-09-17T04:33:53.083982+0000","last_active":"2024-09-17T04:33:54.264760+0000","last_peered":"2024-09-17T04:33:54.264760+0000","last_clean":"2024-09-17T04:33:54.264760+0000","last_became_active":"2024-09-17T04:33:53.083699+0000","last_became_peered":"2024-09-17T04:33:53.083699+0000","last_unstale":"2024-09-17T04:33:54.264760+0000","last_undegraded":"2024-09-17T04:33:54.264760+0000","last_fullsized":"2024-09-17T04:33:54.264760+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:52.043794+0000","last_clean_scrub_stamp":"2024-09-17T04:33:52.043794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:00.355038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"166'1","reported_seq":16,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.265125+0000","last_change":"2024-09-17T04:33:52.075188+0000","last_active":"2024-09-17T04:33:54.265125+0000","last_peered":"2024-09-17T04:33:54.265125+0000","last_clean":"2024-09-17T04:33:54.265125+0000","last_became_active":"2024-09-17T04:33:52.074874+0000","last_became_peered":"2024-09-17T04:33:52.074874+0000","last_unstale":"2024-09-17T04:33:54.265125+0000","last_undegraded":"2024-09-17T04:33:54.265125+0000","last_fullsized":"2024-09-17T04:33:54.265125+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:51.041003+0000","last_clean_scrub_stamp":"2024-09-17T04:33:51.041003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:50:03.803314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"76'79","reported_seq":404,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-17T04:33:54.245353+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:33:54.245353+0000","last_peered":"2024-09-17T04:33:54.245353+0000","last_clean":"2024-09-17T04:33:54.245353+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:33:54.245353+0000","last_undegraded":"2024-09-17T04:33:54.245353+0000","last_fullsized":"2024-09-17T04:33:54.245353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476860,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33836,"kb_used_data":2640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338004,"statfs":{"total":96636764160,"available":96602116096,"internally_reserved":0,"allocated":2703360,"data_stored":1518902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33248,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338592,"statfs":{"total":96636764160,"available":96602718208,"internally_reserved":0,"allocated":2101248,"data_stored":927504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476860,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33248,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338592,"statfs":{"total":96636764160,"available":96602718208,"internally_reserved":0,"allocated":2101248,"data_stored":927504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476860,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33256,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338584,"statfs":{"total":96636764160,"available":96602710016,"internally_reserved":0,"allocated":2117632,"data_stored":928638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542267,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37408,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334432,"statfs":{"total":96636764160,"available":96598458368,"internally_reserved":0,"allocated":2109440,"data_stored":927104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509563,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2097152,"data_stored":926922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509562,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33248,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338592,"statfs":{"total":96636764160,"available":96602718208,"internally_reserved":0,"allocated":2105344,"data_stored":926978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509562,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37980,"kb_used_data":2632,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333860,"statfs":{"total":96636764160,"available":96597872640,"internally_reserved":0,"allocated":2695168,"data_stored":1517380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:33:57.973 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T04:33:57.973 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T04:33:58.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.240+0000 7f2335200640 1 -- 172.21.15.100:0/1380607258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2330145550 msgr2=0x7f233006fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:58.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.240+0000 7f2335200640 1 --2- 172.21.15.100:0/1380607258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2330145550 0x7f233006fb40 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f2320009830 tx=0x7f232002a5a0 comp rx=0 tx=0).stop 2024-09-17T04:33:58.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.240+0000 7f2335200640 1 -- 172.21.15.100:0/1380607258 shutdown_connections 2024-09-17T04:33:58.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.240+0000 7f2335200640 1 --2- 172.21.15.100:0/1380607258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2330145a40 0x7f2330073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:58.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.240+0000 7f2335200640 1 --2- 172.21.15.100:0/1380607258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2330145550 0x7f233006fb40 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:58.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.240+0000 7f2335200640 1 -- 172.21.15.100:0/1380607258 >> 172.21.15.100:0/1380607258 conn(0x7f233006d1c0 msgr2=0x7f233006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:58.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.240+0000 7f2335200640 1 -- 172.21.15.100:0/1380607258 shutdown_connections 2024-09-17T04:33:58.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.240+0000 7f2335200640 1 -- 172.21.15.100:0/1380607258 wait complete. 2024-09-17T04:33:58.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.243+0000 7f2335200640 1 Processor -- start 2024-09-17T04:33:58.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.243+0000 7f2335200640 1 -- start start 2024-09-17T04:33:58.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.244+0000 7f2335200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2330073a80 0x7f233015ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:58.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.244+0000 7f2335200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2330145550 0x7f23300a99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:58.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.244+0000 7f2335200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2330145a40 0x7f23301f3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:58.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.244+0000 7f2335200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2330160a30 con 0x7f2330073a80 2024-09-17T04:33:58.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.244+0000 7f2335200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f23300ce940 con 0x7f2330145a40 2024-09-17T04:33:58.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.244+0000 7f2335200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2330160bb0 con 0x7f2330145550 2024-09-17T04:33:58.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.245+0000 7f232de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2330073a80 0x7f233015ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:58.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.245+0000 7f232de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2330073a80 0x7f233015ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37870/0 (socket says 172.21.15.100:37870) 2024-09-17T04:33:58.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.245+0000 7f232de00640 1 -- 172.21.15.100:0/3152887375 learned_addr learned my addr 172.21.15.100:0/3152887375 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:58.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.245+0000 7f2325400640 1 --2- 172.21.15.100:0/3152887375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2330145550 0x7f23300a99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:58.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.245+0000 7f232e800640 1 --2- 172.21.15.100:0/3152887375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2330145a40 0x7f23301f3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:58.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.245+0000 7f232de00640 1 -- 172.21.15.100:0/3152887375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2330145550 msgr2=0x7f23300a99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:58.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.245+0000 7f232de00640 1 --2- 172.21.15.100:0/3152887375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2330145550 0x7f23300a99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:58.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.245+0000 7f232de00640 1 -- 172.21.15.100:0/3152887375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2330145a40 msgr2=0x7f23301f3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:58.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.245+0000 7f232de00640 1 --2- 172.21.15.100:0/3152887375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2330145a40 0x7f23301f3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:58.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.245+0000 7f232de00640 1 -- 172.21.15.100:0/3152887375 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2330153800 con 0x7f2330073a80 2024-09-17T04:33:58.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.245+0000 7f232e800640 1 --2- 172.21.15.100:0/3152887375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2330145a40 0x7f23301f3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:58.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.245+0000 7f232de00640 1 --2- 172.21.15.100:0/3152887375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2330073a80 0x7f233015ed60 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f2320009950 tx=0x7f2320033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:58.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.246+0000 7f2326a00640 1 -- 172.21.15.100:0/3152887375 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2320031d60 con 0x7f2330073a80 2024-09-17T04:33:58.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.246+0000 7f2326a00640 1 -- 172.21.15.100:0/3152887375 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2320033e10 con 0x7f2330073a80 2024-09-17T04:33:58.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.246+0000 7f2326a00640 1 -- 172.21.15.100:0/3152887375 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2320005260 con 0x7f2330073a80 2024-09-17T04:33:58.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.246+0000 7f2335200640 1 -- 172.21.15.100:0/3152887375 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f233015f940 con 0x7f2330073a80 2024-09-17T04:33:58.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.246+0000 7f2335200640 1 -- 172.21.15.100:0/3152887375 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f233013c320 con 0x7f2330073a80 2024-09-17T04:33:58.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.247+0000 7f2335200640 1 -- 172.21.15.100:0/3152887375 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23301462e0 con 0x7f2330073a80 2024-09-17T04:33:58.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.248+0000 7f2326a00640 1 -- 172.21.15.100:0/3152887375 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f2320005400 con 0x7f2330073a80 2024-09-17T04:33:58.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.250+0000 7f2326a00640 1 --2- 172.21.15.100:0/3152887375 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f230c0795b0 0x7f230c07bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:58.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.250+0000 7f2326a00640 1 -- 172.21.15.100:0/3152887375 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7f23200ba9e0 con 0x7f2330073a80 2024-09-17T04:33:58.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.250+0000 7f2325400640 1 --2- 172.21.15.100:0/3152887375 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f230c0795b0 0x7f230c07bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:58.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.251+0000 7f2325400640 1 --2- 172.21.15.100:0/3152887375 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f230c0795b0 0x7f230c07bac0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f23301534b0 tx=0x7f231800b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:58.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.254+0000 7f2326a00640 1 -- 172.21.15.100:0/3152887375 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f23301462e0 con 0x7f2330073a80 2024-09-17T04:33:58.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.504+0000 7f2335200640 1 -- 172.21.15.100:0/3152887375 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f233006fb40 con 0x7f2330073a80 2024-09-17T04:33:58.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.507+0000 7f2326a00640 1 -- 172.21.15.100:0/3152887375 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) ==== 190+0+147 (secure 0 0 0) 0x7f2330153800 con 0x7f2330073a80 2024-09-17T04:33:58.508 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-09-17T04:33:58.508 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAsBOlmfYg9IBAABP1uRKGOugrA47VxwRLkjA== 2024-09-17T04:33:58.508 INFO:teuthology.orchestra.run.smithi100.stdout: caps mds = "allow" 2024-09-17T04:33:58.509 INFO:teuthology.orchestra.run.smithi100.stdout: caps mon = "allow r" 2024-09-17T04:33:58.509 INFO:teuthology.orchestra.run.smithi100.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T04:33:58.509 INFO:teuthology.orchestra.run.smithi100.stderr:updated caps for client.0 2024-09-17T04:33:58.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.510+0000 7f2335200640 1 -- 172.21.15.100:0/3152887375 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f230c0795b0 msgr2=0x7f230c07bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.510+0000 7f2335200640 1 --2- 172.21.15.100:0/3152887375 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f230c0795b0 0x7f230c07bac0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f23301534b0 tx=0x7f231800b2e0 comp rx=0 tx=0).stop 2024-09-17T04:33:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.510+0000 7f2335200640 1 -- 172.21.15.100:0/3152887375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2330073a80 msgr2=0x7f233015ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.510+0000 7f2335200640 1 --2- 172.21.15.100:0/3152887375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2330073a80 0x7f233015ed60 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f2320009950 tx=0x7f2320033c20 comp rx=0 tx=0).stop 2024-09-17T04:33:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.510+0000 7f2335200640 1 -- 172.21.15.100:0/3152887375 shutdown_connections 2024-09-17T04:33:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.510+0000 7f2335200640 1 --2- 172.21.15.100:0/3152887375 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f230c0795b0 0x7f230c07bac0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.510+0000 7f2335200640 1 --2- 172.21.15.100:0/3152887375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2330145a40 0x7f23301f3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.510+0000 7f2335200640 1 --2- 172.21.15.100:0/3152887375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2330145550 0x7f23300a99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.510+0000 7f2335200640 1 --2- 172.21.15.100:0/3152887375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2330073a80 0x7f233015ed60 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.510+0000 7f2335200640 1 -- 172.21.15.100:0/3152887375 >> 172.21.15.100:0/3152887375 conn(0x7f233006d1c0 msgr2=0x7f2330141aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.510+0000 7f2335200640 1 -- 172.21.15.100:0/3152887375 shutdown_connections 2024-09-17T04:33:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.510+0000 7f2335200640 1 -- 172.21.15.100:0/3152887375 wait complete. 2024-09-17T04:33:58.531 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:33:58.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.752+0000 7fda45200640 1 -- 172.21.15.100:0/2455511584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda24004800 msgr2=0x7fda24004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:58.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.752+0000 7fda45200640 1 --2- 172.21.15.100:0/2455511584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda24004800 0x7fda24004c10 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fda28006fc0 tx=0x7fda24005830 comp rx=0 tx=0).stop 2024-09-17T04:33:58.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.752+0000 7fda3fe00640 1 -- 172.21.15.100:0/2455511584 reap_dead start 2024-09-17T04:33:58.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.752+0000 7fda45200640 1 -- 172.21.15.100:0/2455511584 shutdown_connections 2024-09-17T04:33:58.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.752+0000 7fda45200640 1 -- 172.21.15.100:0/2455511584 >> 172.21.15.100:0/2455511584 conn(0x7fda400b1e40 msgr2=0x7fda400b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:58.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.752+0000 7fda45200640 1 -- 172.21.15.100:0/2455511584 shutdown_connections 2024-09-17T04:33:58.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.752+0000 7fda45200640 1 -- 172.21.15.100:0/2455511584 wait complete. 2024-09-17T04:33:58.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.754+0000 7fda45200640 1 Processor -- start 2024-09-17T04:33:58.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.754+0000 7fda45200640 1 -- start start 2024-09-17T04:33:58.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.756+0000 7fda45200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda24004800 0x7fda401551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:58.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.756+0000 7fda45200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda40068e80 0x7fda40069290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:58.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.756+0000 7fda45200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda400cd400 0x7fda400cd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:58.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.756+0000 7fda45200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fda400b30e0 con 0x7fda400cd400 2024-09-17T04:33:58.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.756+0000 7fda45200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fda400b2f60 con 0x7fda40068e80 2024-09-17T04:33:58.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.756+0000 7fda45200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fda400b3260 con 0x7fda24004800 2024-09-17T04:33:58.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.756+0000 7fda3fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda400cd400 0x7fda400cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:58.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.756+0000 7fda3fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda400cd400 0x7fda400cd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37894/0 (socket says 172.21.15.100:37894) 2024-09-17T04:33:58.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.756+0000 7fda3fe00640 1 -- 172.21.15.100:0/2299409008 learned_addr learned my addr 172.21.15.100:0/2299409008 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:58.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.756+0000 7fda37e00640 1 --2- 172.21.15.100:0/2299409008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda40068e80 0x7fda40069290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:58.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.756+0000 7fda3f400640 1 --2- 172.21.15.100:0/2299409008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda24004800 0x7fda401551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:58.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.757+0000 7fda3fe00640 1 -- 172.21.15.100:0/2299409008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda24004800 msgr2=0x7fda401551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:58.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.757+0000 7fda3fe00640 1 --2- 172.21.15.100:0/2299409008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda24004800 0x7fda401551d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:58.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.757+0000 7fda3fe00640 1 -- 172.21.15.100:0/2299409008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda40068e80 msgr2=0x7fda40069290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:58.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.757+0000 7fda3fe00640 1 --2- 172.21.15.100:0/2299409008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda40068e80 0x7fda40069290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:58.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.757+0000 7fda3fe00640 1 -- 172.21.15.100:0/2299409008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda400697d0 con 0x7fda400cd400 2024-09-17T04:33:58.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.757+0000 7fda3fe00640 1 --2- 172.21.15.100:0/2299409008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda400cd400 0x7fda400cd810 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fda4015d640 tx=0x7fda3800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:58.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.757+0000 7fda37400640 1 -- 172.21.15.100:0/2299409008 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38004b20 con 0x7fda400cd400 2024-09-17T04:33:58.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.757+0000 7fda37400640 1 -- 172.21.15.100:0/2299409008 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda38004cc0 con 0x7fda400cd400 2024-09-17T04:33:58.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.757+0000 7fda37400640 1 -- 172.21.15.100:0/2299409008 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38003bd0 con 0x7fda400cd400 2024-09-17T04:33:58.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.757+0000 7fda45200640 1 -- 172.21.15.100:0/2299409008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda401f83d0 con 0x7fda400cd400 2024-09-17T04:33:58.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.758+0000 7fda45200640 1 -- 172.21.15.100:0/2299409008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda401f8990 con 0x7fda400cd400 2024-09-17T04:33:58.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.759+0000 7fda45200640 1 -- 172.21.15.100:0/2299409008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda4006a610 con 0x7fda400cd400 2024-09-17T04:33:58.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.760+0000 7fda37400640 1 -- 172.21.15.100:0/2299409008 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7fda38004030 con 0x7fda400cd400 2024-09-17T04:33:58.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.761+0000 7fda37400640 1 --2- 172.21.15.100:0/2299409008 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fda1c0795e0 0x7fda1c07baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:58.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.761+0000 7fda37400640 1 -- 172.21.15.100:0/2299409008 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7fda38099960 con 0x7fda400cd400 2024-09-17T04:33:58.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.761+0000 7fda3f400640 1 --2- 172.21.15.100:0/2299409008 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fda1c0795e0 0x7fda1c07baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:58.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.762+0000 7fda3f400640 1 --2- 172.21.15.100:0/2299409008 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fda1c0795e0 0x7fda1c07baf0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fda30000c00 tx=0x7fda30006b70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:58.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.766+0000 7fda37400640 1 -- 172.21.15.100:0/2299409008 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fda4006a610 con 0x7fda400cd400 2024-09-17T04:33:58.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.981+0000 7fda45200640 1 -- 172.21.15.100:0/2299409008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda401ebd20 con 0x7fda400cd400 2024-09-17T04:33:58.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.982+0000 7fda37400640 1 -- 172.21.15.100:0/2299409008 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) ==== 95+0+5242 (secure 0 0 0) 0x7fda401ebd20 con 0x7fda400cd400 2024-09-17T04:33:58.983 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:33:58.983 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":121,"btime":"2024-09-17T04:33:56:084891+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/613141426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":613141426},{"type":"v1","addr":"172.21.15.184:6835","nonce":613141426}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":8554,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/1797529407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6801","nonce":1797529407},{"type":"v1","addr":"172.21.15.184:6834","nonce":1797529407}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":8560,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2263752731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2263752731},{"type":"v1","addr":"172.21.15.100:6833","nonce":2263752731}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:52.555851+0000","modified":"2024-09-17T04:33:55.567693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9153},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9153":{"gid":9153,"name":"c","rank":0,"incarnation":117,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/1207785711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1207785711},{"type":"v1","addr":"172.21.15.100:6835","nonce":1207785711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9153,"qdb_cluster":[9153]},"id":13}]} 2024-09-17T04:33:58.983 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 121 2024-09-17T04:33:58.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.985+0000 7fda45200640 1 -- 172.21.15.100:0/2299409008 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fda1c0795e0 msgr2=0x7fda1c07baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:58.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.985+0000 7fda45200640 1 --2- 172.21.15.100:0/2299409008 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fda1c0795e0 0x7fda1c07baf0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fda30000c00 tx=0x7fda30006b70 comp rx=0 tx=0).stop 2024-09-17T04:33:58.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.985+0000 7fda45200640 1 -- 172.21.15.100:0/2299409008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda400cd400 msgr2=0x7fda400cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:58.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.985+0000 7fda45200640 1 --2- 172.21.15.100:0/2299409008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda400cd400 0x7fda400cd810 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fda4015d640 tx=0x7fda3800f3c0 comp rx=0 tx=0).stop 2024-09-17T04:33:58.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.985+0000 7fda45200640 1 -- 172.21.15.100:0/2299409008 shutdown_connections 2024-09-17T04:33:58.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.985+0000 7fda45200640 1 --2- 172.21.15.100:0/2299409008 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fda1c0795e0 0x7fda1c07baf0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:58.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.985+0000 7fda45200640 1 --2- 172.21.15.100:0/2299409008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda400cd400 0x7fda400cd810 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:58.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.985+0000 7fda45200640 1 --2- 172.21.15.100:0/2299409008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda40068e80 0x7fda40069290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:58.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.985+0000 7fda45200640 1 --2- 172.21.15.100:0/2299409008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda24004800 0x7fda401551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:58.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.985+0000 7fda45200640 1 -- 172.21.15.100:0/2299409008 >> 172.21.15.100:0/2299409008 conn(0x7fda400b1e40 msgr2=0x7fda400b1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:58.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.985+0000 7fda45200640 1 -- 172.21.15.100:0/2299409008 shutdown_connections 2024-09-17T04:33:58.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:58.985+0000 7fda45200640 1 -- 172.21.15.100:0/2299409008 wait complete. 2024-09-17T04:33:59.412 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 121, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T04:33:52.555851+0000', 'modified': '2024-09-17T04:33:55.567693+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9153}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9153': {'gid': 9153, 'name': 'c', 'rank': 0, 'incarnation': 117, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6835/1207785711', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6834', 'nonce': 1207785711}, {'type': 'v1', 'addr': '172.21.15.100:6835', 'nonce': 1207785711}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9153, 'qdb_cluster': [9153]} 2024-09-17T04:33:59.412 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T04:33:59.413 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T04:33:59.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.632+0000 7fd1a4a00640 1 -- 172.21.15.100:0/3535812354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1a00705b0 msgr2=0x7fd1a00709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:59.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.632+0000 7fd1a4a00640 1 --2- 172.21.15.100:0/3535812354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1a00705b0 0x7fd1a00709c0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fd18c005fd0 tx=0x7fd18c02a540 comp rx=0 tx=0).stop 2024-09-17T04:33:59.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.632+0000 7fd1a4a00640 1 -- 172.21.15.100:0/3535812354 shutdown_connections 2024-09-17T04:33:59.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.633+0000 7fd1a4a00640 1 --2- 172.21.15.100:0/3535812354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1a00705b0 0x7fd1a00709c0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.633+0000 7fd1a4a00640 1 --2- 172.21.15.100:0/3535812354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1a00cfc80 0x7fd1a0156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.633+0000 7fd1a4a00640 1 -- 172.21.15.100:0/3535812354 >> 172.21.15.100:0/3535812354 conn(0x7fd1a006d3d0 msgr2=0x7fd1a00b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:59.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.633+0000 7fd1a4a00640 1 -- 172.21.15.100:0/3535812354 shutdown_connections 2024-09-17T04:33:59.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.633+0000 7fd1a4a00640 1 -- 172.21.15.100:0/3535812354 wait complete. 2024-09-17T04:33:59.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.635+0000 7fd1a4a00640 1 Processor -- start 2024-09-17T04:33:59.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.635+0000 7fd1a4a00640 1 -- start start 2024-09-17T04:33:59.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.636+0000 7fd1a4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1a00705b0 0x7fd1a00c6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:59.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.636+0000 7fd1a4a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1a00cfc80 0x7fd1a00c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:59.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.636+0000 7fd1a4a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1a015ce30 0x7fd1a00c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:59.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.636+0000 7fd1a4a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a00cd050 con 0x7fd1a00705b0 2024-09-17T04:33:59.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.636+0000 7fd1a4a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a00cced0 con 0x7fd1a00cfc80 2024-09-17T04:33:59.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.636+0000 7fd1a4a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd1a00cd1d0 con 0x7fd1a015ce30 2024-09-17T04:33:59.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.637+0000 7fd19de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1a00705b0 0x7fd1a00c6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:59.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.637+0000 7fd19de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1a00705b0 0x7fd1a00c6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37902/0 (socket says 172.21.15.100:37902) 2024-09-17T04:33:59.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.637+0000 7fd19de00640 1 -- 172.21.15.100:0/1500088027 learned_addr learned my addr 172.21.15.100:0/1500088027 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:59.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.637+0000 7fd19d400640 1 --2- 172.21.15.100:0/1500088027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1a00cfc80 0x7fd1a00c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:59.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.637+0000 7fd19e800640 1 --2- 172.21.15.100:0/1500088027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1a015ce30 0x7fd1a00c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:59.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.637+0000 7fd19de00640 1 -- 172.21.15.100:0/1500088027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1a015ce30 msgr2=0x7fd1a00c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:59.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.637+0000 7fd19de00640 1 --2- 172.21.15.100:0/1500088027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1a015ce30 0x7fd1a00c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.637+0000 7fd19de00640 1 -- 172.21.15.100:0/1500088027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1a00cfc80 msgr2=0x7fd1a00c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:59.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.637+0000 7fd19de00640 1 --2- 172.21.15.100:0/1500088027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1a00cfc80 0x7fd1a00c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.637+0000 7fd19de00640 1 -- 172.21.15.100:0/1500088027 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1a0074590 con 0x7fd1a00705b0 2024-09-17T04:33:59.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.637+0000 7fd19e800640 1 --2- 172.21.15.100:0/1500088027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1a015ce30 0x7fd1a00c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:59.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.637+0000 7fd19de00640 1 --2- 172.21.15.100:0/1500088027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1a00705b0 0x7fd1a00c6c60 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fd190009fd0 tx=0x7fd19000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:59.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.638+0000 7fd196000640 1 -- 172.21.15.100:0/1500088027 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19000ccc0 con 0x7fd1a00705b0 2024-09-17T04:33:59.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.638+0000 7fd196000640 1 -- 172.21.15.100:0/1500088027 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd19000ee70 con 0x7fd1a00705b0 2024-09-17T04:33:59.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.638+0000 7fd1a4a00640 1 -- 172.21.15.100:0/1500088027 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1a0074870 con 0x7fd1a00705b0 2024-09-17T04:33:59.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.638+0000 7fd196000640 1 -- 172.21.15.100:0/1500088027 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19000ce60 con 0x7fd1a00705b0 2024-09-17T04:33:59.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.638+0000 7fd1a4a00640 1 -- 172.21.15.100:0/1500088027 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1a01f72c0 con 0x7fd1a00705b0 2024-09-17T04:33:59.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.640+0000 7fd196000640 1 -- 172.21.15.100:0/1500088027 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7fd190004280 con 0x7fd1a00705b0 2024-09-17T04:33:59.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.641+0000 7fd196000640 1 --2- 172.21.15.100:0/1500088027 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd1800795e0 0x7fd18007baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:59.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.641+0000 7fd196000640 1 -- 172.21.15.100:0/1500088027 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7fd19009a1e0 con 0x7fd1a00705b0 2024-09-17T04:33:59.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.641+0000 7fd19d400640 1 --2- 172.21.15.100:0/1500088027 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd1800795e0 0x7fd18007baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:59.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.642+0000 7fd1aba18740 1 Processor -- start 2024-09-17T04:33:59.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.642+0000 7fd1aba18740 1 -- start start 2024-09-17T04:33:59.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.642+0000 7fd19d400640 1 --2- 172.21.15.100:0/1500088027 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd1800795e0 0x7fd18007baf0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd18c033000 tx=0x7fd18c004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:59.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.643+0000 7fd1aba18740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556961ce0270 0x5569620cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:59.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.643+0000 7fd1aba18740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556961cb73b0 0x556962080f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:59.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.643+0000 7fd1aba18740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556961cbf590 0x5569620814a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:59.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.643+0000 7fd1aba18740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55696200a430 con 0x556961cb73b0 2024-09-17T04:33:59.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.643+0000 7fd1aba18740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556961ffa5d0 con 0x556961ce0270 2024-09-17T04:33:59.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.643+0000 7fd1aba18740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x556961fe9a00 con 0x556961cbf590 2024-09-17T04:33:59.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.643+0000 7fd19d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556961cb73b0 0x556962080f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:59.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.643+0000 7fd19d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556961cb73b0 0x556962080f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37918/0 (socket says 172.21.15.100:37918) 2024-09-17T04:33:59.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.643+0000 7fd19d400640 1 -- 172.21.15.100:0/1897037400 learned_addr learned my addr 172.21.15.100:0/1897037400 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:59.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.644+0000 7fd19de00640 1 --2- 172.21.15.100:0/1897037400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556961ce0270 0x5569620cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:59.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.644+0000 7fd19e800640 1 --2- 172.21.15.100:0/1897037400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556961cbf590 0x5569620814a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:59.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.644+0000 7fd19d400640 1 -- 172.21.15.100:0/1897037400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556961cbf590 msgr2=0x5569620814a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:33:59.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.644+0000 7fd19d400640 1 --2- 172.21.15.100:0/1897037400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556961cbf590 0x5569620814a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.644+0000 7fd19d400640 1 -- 172.21.15.100:0/1897037400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556961ce0270 msgr2=0x5569620cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:33:59.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.644+0000 7fd19d400640 1 --2- 172.21.15.100:0/1897037400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556961ce0270 0x5569620cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.644+0000 7fd19d400640 1 -- 172.21.15.100:0/1897037400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556961fb2520 con 0x556961cb73b0 2024-09-17T04:33:59.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.644+0000 7fd19d400640 1 --2- 172.21.15.100:0/1897037400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556961cb73b0 0x556962080f60 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fd18c006010 tx=0x7fd18c03c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ef354e7f706298 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:59.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.644+0000 7fd18a600640 1 -- 172.21.15.100:0/1897037400 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c035040 con 0x556961cb73b0 2024-09-17T04:33:59.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.645+0000 7fd18a600640 1 -- 172.21.15.100:0/1897037400 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd18c04c070 con 0x556961cb73b0 2024-09-17T04:33:59.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.645+0000 7fd18a600640 1 -- 172.21.15.100:0/1897037400 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c003ae0 con 0x556961cb73b0 2024-09-17T04:33:59.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.645+0000 7fd1aba18740 1 -- 172.21.15.100:0/1897037400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556961cb73b0 msgr2=0x556962080f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:59.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.645+0000 7fd1aba18740 1 --2- 172.21.15.100:0/1897037400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556961cb73b0 0x556962080f60 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fd18c006010 tx=0x7fd18c03c000 comp rx=0 tx=0).stop 2024-09-17T04:33:59.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.645+0000 7fd1aba18740 1 -- 172.21.15.100:0/1897037400 shutdown_connections 2024-09-17T04:33:59.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.645+0000 7fd1aba18740 1 --2- 172.21.15.100:0/1897037400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556961cbf590 0x5569620814a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.645+0000 7fd1aba18740 1 --2- 172.21.15.100:0/1897037400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556961cb73b0 0x556962080f60 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.645+0000 7fd1aba18740 1 --2- 172.21.15.100:0/1897037400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556961ce0270 0x5569620cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.645+0000 7fd1aba18740 1 -- 172.21.15.100:0/1897037400 >> 172.21.15.100:0/1897037400 conn(0x556961cd43c0 msgr2=0x556962086970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:59.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.646+0000 7fd1aba18740 1 -- 172.21.15.100:0/1897037400 shutdown_connections 2024-09-17T04:33:59.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.646+0000 7fd1aba18740 1 -- 172.21.15.100:0/1897037400 wait complete. 2024-09-17T04:33:59.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.647+0000 7fd1aba18740 1 -- start start 2024-09-17T04:33:59.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.647+0000 7fd1aba18740 1 Processor -- start 2024-09-17T04:33:59.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.648+0000 7fd1aba18740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556961cb73b0 0x5569620c4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:59.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.648+0000 7fd1aba18740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556961cbf590 0x5569620e2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:59.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.648+0000 7fd1aba18740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556961ce0270 0x5569620c7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:59.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.648+0000 7fd1aba18740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556961f90330 con 0x556961cbf590 2024-09-17T04:33:59.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.648+0000 7fd1aba18740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556961fed810 con 0x556961ce0270 2024-09-17T04:33:59.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.648+0000 7fd1aba18740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x556961d5a620 con 0x556961cb73b0 2024-09-17T04:33:59.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.648+0000 7fd1aba18740 10 client.0 _ll_register_callbacks cb 0x556961f95420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:33:59.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.648+0000 7fd19d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556961cbf590 0x5569620e2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:59.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.649+0000 7fd19d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556961cbf590 0x5569620e2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37928/0 (socket says 172.21.15.100:37928) 2024-09-17T04:33:59.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.649+0000 7fd19d400640 1 -- 172.21.15.100:0/3336740488 learned_addr learned my addr 172.21.15.100:0/3336740488 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:59.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.649+0000 7fd19de00640 1 --2- 172.21.15.100:0/3336740488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556961cb73b0 0x5569620c4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:59.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.649+0000 7fd19e800640 1 --2- 172.21.15.100:0/3336740488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556961ce0270 0x5569620c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:59.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.649+0000 7fd19d400640 1 -- 172.21.15.100:0/3336740488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556961cb73b0 msgr2=0x5569620c4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:33:59.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.649+0000 7fd19d400640 1 --2- 172.21.15.100:0/3336740488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556961cb73b0 0x5569620c4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.649+0000 7fd19d400640 1 -- 172.21.15.100:0/3336740488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556961ce0270 msgr2=0x5569620c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:33:59.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.649+0000 7fd19d400640 1 --2- 172.21.15.100:0/3336740488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556961ce0270 0x5569620c7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.649+0000 7fd19d400640 1 -- 172.21.15.100:0/3336740488 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556961d4b850 con 0x556961cbf590 2024-09-17T04:33:59.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.649+0000 7fd19d400640 1 --2- 172.21.15.100:0/3336740488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556961cbf590 0x5569620e2850 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fd18c005f70 tx=0x7fd18c02a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1cb0d58ecc32f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:59.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.649+0000 7fd17ea00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-17T04:33:59.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.650+0000 7fd17ea00640 1 -- 172.21.15.100:0/3336740488 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c0049b0 con 0x556961cbf590 2024-09-17T04:33:59.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.650+0000 7fd17ea00640 1 -- 172.21.15.100:0/3336740488 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7fd18c003e40 con 0x556961cbf590 2024-09-17T04:33:59.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.650+0000 7fd1aba18740 1 -- 172.21.15.100:0/3336740488 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd18c04c070 con 0x556961cbf590 2024-09-17T04:33:59.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.650+0000 7fd17ea00640 1 -- 172.21.15.100:0/3336740488 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=121) ==== 24+0+0 (secure 0 0 0) 0x7fd18c063050 con 0x556961cbf590 2024-09-17T04:33:59.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.650+0000 7fd1aba18740 10 client.9279 fetch_fsmap learned FSMap version 121 2024-09-17T04:33:59.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.650+0000 7fd1aba18740 1 -- 172.21.15.100:0/3336740488 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=121,monmap=2+}) -- 0x556961dc26b0 con 0x556961cbf590 2024-09-17T04:33:59.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.650+0000 7fd17ea00640 1 -- 172.21.15.100:0/3336740488 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd18c04c070 con 0x556961cbf590 2024-09-17T04:33:59.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.650+0000 7fd17ea00640 1 -- 172.21.15.100:0/3336740488 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 121) ==== 3502+0+0 (secure 0 0 0) 0x7fd18c066020 con 0x556961cbf590 2024-09-17T04:33:59.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.650+0000 7fd1aba18740 10 client.9279 fetch_fsmap finished waiting for FSMap version 121 2024-09-17T04:33:59.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.650+0000 7fd1aba18740 10 client.9279 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T04:33:59.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.650+0000 7fd1aba18740 20 client.9279 populate_metadata read hostname 'smithi100' 2024-09-17T04:33:59.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.650+0000 7fd1aba18740 1 --2- 172.21.15.100:0/3336740488 >> [v2:172.21.15.100:6834/1207785711,v1:172.21.15.100:6835/1207785711] conn(0x556961c9ee40 0x5569620c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:59.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.650+0000 7fd1aba18740 4 client.9279 mds_command: new command op to 9153 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T04:33:59.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.650+0000 7fd1aba18740 1 -- 172.21.15.100:0/3336740488 --> [v2:172.21.15.100:6834/1207785711,v1:172.21.15.100:6835/1207785711] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556961ffc520 con 0x556961c9ee40 2024-09-17T04:33:59.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.651+0000 7fd19de00640 1 --2- 172.21.15.100:0/3336740488 >> [v2:172.21.15.100:6834/1207785711,v1:172.21.15.100:6835/1207785711] conn(0x556961c9ee40 0x5569620c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:59.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.651+0000 7fd17ea00640 10 client.9279 ms_handle_connect on v2:172.21.15.100:6834/1207785711 2024-09-17T04:33:59.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.651+0000 7fd19de00640 1 --2- 172.21.15.100:0/3336740488 >> [v2:172.21.15.100:6834/1207785711,v1:172.21.15.100:6835/1207785711] conn(0x556961c9ee40 0x5569620c5e10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ed79c319f4462d server_cookie=8329a8583b320385 in_seq=0 out_seq=0 2024-09-17T04:33:59.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.652+0000 7fd17ea00640 1 -- 172.21.15.100:0/3336740488 <== mds.0 v2:172.21.15.100:6834/1207785711 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556961ffc520 con 0x556961c9ee40 2024-09-17T04:33:59.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.652+0000 7fd17ea00640 10 client.9279 handle_command_reply: tid=0 2024-09-17T04:33:59.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.652+0000 7fd1aba18740 1 client.9279 shutdown 2024-09-17T04:33:59.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.652+0000 7fd1aba18740 1 -- 172.21.15.100:0/3336740488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556961cbf590 msgr2=0x5569620e2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:59.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.652+0000 7fd1aba18740 1 --2- 172.21.15.100:0/3336740488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556961cbf590 0x5569620e2850 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fd18c005f70 tx=0x7fd18c02a540 comp rx=0 tx=0).stop 2024-09-17T04:33:59.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.652+0000 7fd1aba18740 1 -- 172.21.15.100:0/3336740488 shutdown_connections 2024-09-17T04:33:59.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.652+0000 7fd1aba18740 1 --2- 172.21.15.100:0/3336740488 >> [v2:172.21.15.100:6834/1207785711,v1:172.21.15.100:6835/1207785711] conn(0x556961c9ee40 0x5569620c5e10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.652+0000 7fd1aba18740 1 --2- 172.21.15.100:0/3336740488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556961ce0270 0x5569620c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.652+0000 7fd1aba18740 1 --2- 172.21.15.100:0/3336740488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556961cbf590 0x5569620e2850 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.652+0000 7fd17ea00640 0 client.9279 ms_handle_reset on v2:172.21.15.100:6834/1207785711 2024-09-17T04:33:59.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.652+0000 7fd1aba18740 1 --2- 172.21.15.100:0/3336740488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556961cb73b0 0x5569620c4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.652+0000 7fd1aba18740 1 -- 172.21.15.100:0/3336740488 >> 172.21.15.100:0/3336740488 conn(0x556961cd43c0 msgr2=0x5569620802b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:59.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.653+0000 7fd1aba18740 1 -- 172.21.15.100:0/3336740488 shutdown_connections 2024-09-17T04:33:59.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.653+0000 7fd1aba18740 1 -- 172.21.15.100:0/3336740488 wait complete. 2024-09-17T04:33:59.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.653+0000 7fd1aba18740 20 client.9279 trim_cache size 0 max 16384 2024-09-17T04:33:59.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.671+0000 7fd1aba18740 1 Processor -- start 2024-09-17T04:33:59.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.672+0000 7fd1aba18740 1 -- start start 2024-09-17T04:33:59.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.673+0000 7fd1aba18740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556961cd43c0 0x5569620f3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:59.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.673+0000 7fd1aba18740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556961cb73b0 0x5569620ad480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:59.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.673+0000 7fd1aba18740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556961cbf590 0x5569620ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:59.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.673+0000 7fd1aba18740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a00cc640 con 0x556961cd43c0 2024-09-17T04:33:59.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.673+0000 7fd1aba18740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a01f81a0 con 0x556961cb73b0 2024-09-17T04:33:59.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.673+0000 7fd1aba18740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd1a00cbff0 con 0x556961cbf590 2024-09-17T04:33:59.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.673+0000 7fd19de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556961cd43c0 0x5569620f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:59.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.673+0000 7fd19de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556961cd43c0 0x5569620f3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37944/0 (socket says 172.21.15.100:37944) 2024-09-17T04:33:59.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.673+0000 7fd19de00640 1 -- 172.21.15.100:0/11256602 learned_addr learned my addr 172.21.15.100:0/11256602 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:59.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.673+0000 7fd19e800640 1 --2- 172.21.15.100:0/11256602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556961cbf590 0x5569620ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:59.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.673+0000 7fd19d400640 1 --2- 172.21.15.100:0/11256602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556961cb73b0 0x5569620ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:59.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.673+0000 7fd19de00640 1 -- 172.21.15.100:0/11256602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556961cbf590 msgr2=0x5569620ad9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:33:59.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.674+0000 7fd19de00640 1 --2- 172.21.15.100:0/11256602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556961cbf590 0x5569620ad9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.674+0000 7fd19de00640 1 -- 172.21.15.100:0/11256602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556961cb73b0 msgr2=0x5569620ad480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:33:59.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.674+0000 7fd19de00640 1 --2- 172.21.15.100:0/11256602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556961cb73b0 0x5569620ad480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.674+0000 7fd19de00640 1 -- 172.21.15.100:0/11256602 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556962087d20 con 0x556961cd43c0 2024-09-17T04:33:59.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.674+0000 7fd19e800640 1 --2- 172.21.15.100:0/11256602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556961cbf590 0x5569620ad9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:59.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.674+0000 7fd19d400640 1 --2- 172.21.15.100:0/11256602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556961cb73b0 0x5569620ad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:59.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.674+0000 7fd19de00640 1 --2- 172.21.15.100:0/11256602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556961cd43c0 0x5569620f3430 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fd1900096b0 tx=0x7fd1900631d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b85ceefedc8563 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:59.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.674+0000 7fd189c00640 1 -- 172.21.15.100:0/11256602 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1900637e0 con 0x556961cd43c0 2024-09-17T04:33:59.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.674+0000 7fd189c00640 1 -- 172.21.15.100:0/11256602 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd19007d030 con 0x556961cd43c0 2024-09-17T04:33:59.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.674+0000 7fd189c00640 1 -- 172.21.15.100:0/11256602 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19006cf40 con 0x556961cd43c0 2024-09-17T04:33:59.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.675+0000 7fd1aba18740 1 -- 172.21.15.100:0/11256602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556961cd43c0 msgr2=0x5569620f3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:59.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.675+0000 7fd1aba18740 1 --2- 172.21.15.100:0/11256602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556961cd43c0 0x5569620f3430 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fd1900096b0 tx=0x7fd1900631d0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.675+0000 7fd1aba18740 1 -- 172.21.15.100:0/11256602 shutdown_connections 2024-09-17T04:33:59.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.675+0000 7fd1aba18740 1 --2- 172.21.15.100:0/11256602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556961cbf590 0x5569620ad9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.675+0000 7fd1aba18740 1 --2- 172.21.15.100:0/11256602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556961cb73b0 0x5569620ad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.675+0000 7fd1aba18740 1 --2- 172.21.15.100:0/11256602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556961cd43c0 0x5569620f3430 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.675+0000 7fd1aba18740 1 -- 172.21.15.100:0/11256602 >> 172.21.15.100:0/11256602 conn(0x5569620e20b0 msgr2=0x556962080100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:59.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.675+0000 7fd1aba18740 1 -- 172.21.15.100:0/11256602 shutdown_connections 2024-09-17T04:33:59.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.675+0000 7fd1aba18740 1 -- 172.21.15.100:0/11256602 wait complete. 2024-09-17T04:33:59.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.676+0000 7fd1aba18740 1 -- start start 2024-09-17T04:33:59.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.676+0000 7fd1aba18740 1 Processor -- start 2024-09-17T04:33:59.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.677+0000 7fd1aba18740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556961cb73b0 0x556962146290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:59.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.677+0000 7fd1aba18740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556961cbf590 0x5569620ce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:59.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.677+0000 7fd1aba18740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556961cd43c0 0x5569620ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:59.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.677+0000 7fd1aba18740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5569620c4db0 con 0x556961cd43c0 2024-09-17T04:33:59.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.677+0000 7fd1aba18740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5569620af400 con 0x556961cbf590 2024-09-17T04:33:59.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.677+0000 7fd1aba18740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5569620ae790 con 0x556961cb73b0 2024-09-17T04:33:59.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.677+0000 7fd1aba18740 10 client.0 _ll_register_callbacks cb 0x7fd18c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:33:59.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.678+0000 7fd19e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556961cd43c0 0x5569620ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:59.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.678+0000 7fd19e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556961cd43c0 0x5569620ce540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37952/0 (socket says 172.21.15.100:37952) 2024-09-17T04:33:59.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.678+0000 7fd19e800640 1 -- 172.21.15.100:0/1723477496 learned_addr learned my addr 172.21.15.100:0/1723477496 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:59.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.678+0000 7fd19d400640 1 --2- 172.21.15.100:0/1723477496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556961cbf590 0x5569620ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:59.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.678+0000 7fd19de00640 1 --2- 172.21.15.100:0/1723477496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556961cb73b0 0x556962146290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:59.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.678+0000 7fd19e800640 1 -- 172.21.15.100:0/1723477496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556961cb73b0 msgr2=0x556962146290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:33:59.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.678+0000 7fd19e800640 1 --2- 172.21.15.100:0/1723477496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556961cb73b0 0x556962146290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.678+0000 7fd19e800640 1 -- 172.21.15.100:0/1723477496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556961cbf590 msgr2=0x5569620ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:33:59.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.678+0000 7fd19e800640 1 --2- 172.21.15.100:0/1723477496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556961cbf590 0x5569620ce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.678+0000 7fd19e800640 1 -- 172.21.15.100:0/1723477496 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569620f3430 con 0x556961cd43c0 2024-09-17T04:33:59.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.678+0000 7fd19e800640 1 --2- 172.21.15.100:0/1723477496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556961cd43c0 0x5569620ce540 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fd198001fd0 tx=0x7fd198005420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8e833a0c5c5437 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:59.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.678+0000 7fd17ea00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-17T04:33:59.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.679+0000 7fd17ea00640 1 -- 172.21.15.100:0/1723477496 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19801a070 con 0x556961cd43c0 2024-09-17T04:33:59.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.679+0000 7fd17ea00640 1 -- 172.21.15.100:0/1723477496 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7fd1980168f0 con 0x556961cd43c0 2024-09-17T04:33:59.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.679+0000 7fd1aba18740 1 -- 172.21.15.100:0/1723477496 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd19007d030 con 0x556961cd43c0 2024-09-17T04:33:59.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.679+0000 7fd17ea00640 1 -- 172.21.15.100:0/1723477496 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=121) ==== 24+0+0 (secure 0 0 0) 0x7fd198025050 con 0x556961cd43c0 2024-09-17T04:33:59.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.679+0000 7fd1aba18740 10 client.9285 fetch_fsmap learned FSMap version 121 2024-09-17T04:33:59.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.679+0000 7fd1aba18740 1 -- 172.21.15.100:0/1723477496 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=121,monmap=2+}) -- 0x556962152f70 con 0x556961cd43c0 2024-09-17T04:33:59.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.679+0000 7fd17ea00640 1 -- 172.21.15.100:0/1723477496 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd19007d030 con 0x556961cd43c0 2024-09-17T04:33:59.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.679+0000 7fd17ea00640 1 -- 172.21.15.100:0/1723477496 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 121) ==== 3502+0+0 (secure 0 0 0) 0x7fd198028020 con 0x556961cd43c0 2024-09-17T04:33:59.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.679+0000 7fd1aba18740 10 client.9285 fetch_fsmap finished waiting for FSMap version 121 2024-09-17T04:33:59.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.679+0000 7fd1aba18740 10 client.9285 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T04:33:59.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.679+0000 7fd1aba18740 20 client.9285 populate_metadata read hostname 'smithi100' 2024-09-17T04:33:59.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.679+0000 7fd1aba18740 1 --2- 172.21.15.100:0/1723477496 >> [v2:172.21.15.100:6834/1207785711,v1:172.21.15.100:6835/1207785711] conn(0x556961c74280 0x556962150340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:59.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.679+0000 7fd1aba18740 4 client.9285 mds_command: new command op to 9153 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T04:33:59.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.679+0000 7fd1aba18740 1 -- 172.21.15.100:0/1723477496 --> [v2:172.21.15.100:6834/1207785711,v1:172.21.15.100:6835/1207785711] -- command(tid 0: {"prefix": "status"}) -- 0x556961ff7fd0 con 0x556961c74280 2024-09-17T04:33:59.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.679+0000 7fd19de00640 1 --2- 172.21.15.100:0/1723477496 >> [v2:172.21.15.100:6834/1207785711,v1:172.21.15.100:6835/1207785711] conn(0x556961c74280 0x556962150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:59.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.680+0000 7fd19de00640 1 --2- 172.21.15.100:0/1723477496 >> [v2:172.21.15.100:6834/1207785711,v1:172.21.15.100:6835/1207785711] conn(0x556961c74280 0x556962150340 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccae77cbf1bf6769 server_cookie=f7f1247a91630b5f in_seq=0 out_seq=0 2024-09-17T04:33:59.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.680+0000 7fd17ea00640 10 client.9285 ms_handle_connect on v2:172.21.15.100:6834/1207785711 2024-09-17T04:33:59.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.680+0000 7fd17ea00640 1 -- 172.21.15.100:0/1723477496 <== mds.0 v2:172.21.15.100:6834/1207785711 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x556961ff7fd0 con 0x556961c74280 2024-09-17T04:33:59.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.680+0000 7fd17ea00640 10 client.9285 handle_command_reply: tid=0 2024-09-17T04:33:59.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.680+0000 7fd1aba18740 1 client.9285 shutdown 2024-09-17T04:33:59.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.680+0000 7fd1aba18740 1 -- 172.21.15.100:0/1723477496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556961cd43c0 msgr2=0x5569620ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:59.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.680+0000 7fd1aba18740 1 --2- 172.21.15.100:0/1723477496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556961cd43c0 0x5569620ce540 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fd198001fd0 tx=0x7fd198005420 comp rx=0 tx=0).stop 2024-09-17T04:33:59.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.680+0000 7fd1aba18740 1 -- 172.21.15.100:0/1723477496 shutdown_connections 2024-09-17T04:33:59.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.680+0000 7fd1aba18740 1 --2- 172.21.15.100:0/1723477496 >> [v2:172.21.15.100:6834/1207785711,v1:172.21.15.100:6835/1207785711] conn(0x556961c74280 0x556962150340 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.680+0000 7fd1aba18740 1 --2- 172.21.15.100:0/1723477496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556961cd43c0 0x5569620ce540 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.680+0000 7fd1aba18740 1 --2- 172.21.15.100:0/1723477496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556961cbf590 0x5569620ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.680+0000 7fd17ea00640 0 client.9285 ms_handle_reset on v2:172.21.15.100:6834/1207785711 2024-09-17T04:33:59.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.680+0000 7fd1aba18740 1 --2- 172.21.15.100:0/1723477496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556961cb73b0 0x556962146290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.680+0000 7fd1aba18740 1 -- 172.21.15.100:0/1723477496 >> 172.21.15.100:0/1723477496 conn(0x5569620e20b0 msgr2=0x55696207ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:59.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.681+0000 7fd1aba18740 1 -- 172.21.15.100:0/1723477496 shutdown_connections 2024-09-17T04:33:59.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.681+0000 7fd1aba18740 1 -- 172.21.15.100:0/1723477496 wait complete. 2024-09-17T04:33:59.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.681+0000 7fd1aba18740 20 client.9285 trim_cache size 0 max 16384 2024-09-17T04:33:59.688 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:33:59.688 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "8a399842-3a5c-4ea2-9afb-b2704332e59f", 2024-09-17T04:33:59.688 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 0, 2024-09-17T04:33:59.688 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 9153, 2024-09-17T04:33:59.688 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-09-17T04:33:59.688 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-09-17T04:33:59.688 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-09-17T04:33:59.688 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 6.6287385170000004, 2024-09-17T04:33:59.688 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 121, 2024-09-17T04:33:59.688 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 167, 2024-09-17T04:33:59.688 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 167, 2024-09-17T04:33:59.688 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 11.660478384999999 2024-09-17T04:33:59.689 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:33:59.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.681+0000 7fd1a4a00640 1 -- 172.21.15.100:0/1500088027 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd1800795e0 msgr2=0x7fd18007baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:59.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.681+0000 7fd1a4a00640 1 --2- 172.21.15.100:0/1500088027 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd1800795e0 0x7fd18007baf0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd18c033000 tx=0x7fd18c004730 comp rx=0 tx=0).stop 2024-09-17T04:33:59.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.681+0000 7fd1a4a00640 1 -- 172.21.15.100:0/1500088027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1a00705b0 msgr2=0x7fd1a00c6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:59.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.681+0000 7fd1a4a00640 1 --2- 172.21.15.100:0/1500088027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1a00705b0 0x7fd1a00c6c60 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fd190009fd0 tx=0x7fd19000ec20 comp rx=0 tx=0).stop 2024-09-17T04:33:59.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.681+0000 7fd1a4a00640 1 -- 172.21.15.100:0/1500088027 shutdown_connections 2024-09-17T04:33:59.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.681+0000 7fd1a4a00640 1 --2- 172.21.15.100:0/1500088027 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd1800795e0 0x7fd18007baf0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.681+0000 7fd1a4a00640 1 --2- 172.21.15.100:0/1500088027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1a015ce30 0x7fd1a00c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.681+0000 7fd1a4a00640 1 --2- 172.21.15.100:0/1500088027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1a00cfc80 0x7fd1a00c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.681+0000 7fd1a4a00640 1 --2- 172.21.15.100:0/1500088027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1a00705b0 0x7fd1a00c6c60 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.681+0000 7fd1a4a00640 1 -- 172.21.15.100:0/1500088027 >> 172.21.15.100:0/1500088027 conn(0x7fd1a006d3d0 msgr2=0x7fd1a00afd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:59.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.682+0000 7fd1a4a00640 1 -- 172.21.15.100:0/1500088027 shutdown_connections 2024-09-17T04:33:59.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.682+0000 7fd1a4a00640 1 -- 172.21.15.100:0/1500088027 wait complete. 2024-09-17T04:33:59.705 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:33:59.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.925+0000 7f10c9c00640 1 -- 172.21.15.100:0/3776568564 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10c40b9b80 msgr2=0x7f10c40cfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:59.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.925+0000 7f10c9c00640 1 --2- 172.21.15.100:0/3776568564 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10c40b9b80 0x7f10c40cfd30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009860 tx=0x7f10b402a5a0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.925+0000 7f10c9c00640 1 -- 172.21.15.100:0/3776568564 shutdown_connections 2024-09-17T04:33:59.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.925+0000 7f10c9c00640 1 --2- 172.21.15.100:0/3776568564 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10c40b9b80 0x7f10c40cfd30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.925+0000 7f10c9c00640 1 -- 172.21.15.100:0/3776568564 >> 172.21.15.100:0/3776568564 conn(0x7f10c409d820 msgr2=0x7f10c408bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:59.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.925+0000 7f10c9c00640 1 -- 172.21.15.100:0/3776568564 shutdown_connections 2024-09-17T04:33:59.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.925+0000 7f10c9c00640 1 -- 172.21.15.100:0/3776568564 wait complete. 2024-09-17T04:33:59.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.927+0000 7f10c9c00640 1 Processor -- start 2024-09-17T04:33:59.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.927+0000 7f10c9c00640 1 -- start start 2024-09-17T04:33:59.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.929+0000 7f10c9c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10c40b9b80 0x7f10c40c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:59.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.929+0000 7f10c9c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10c41620e0 0x7f10c40ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:59.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.929+0000 7f10c9c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10c415cc90 0x7f10c415b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:59.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.929+0000 7f10c9c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f10c40b9610 con 0x7f10c415cc90 2024-09-17T04:33:59.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.929+0000 7f10c9c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f10c40b43f0 con 0x7f10c40b9b80 2024-09-17T04:33:59.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.929+0000 7f10c9c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f10c40b9790 con 0x7f10c41620e0 2024-09-17T04:33:59.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.929+0000 7f10c9200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10c415cc90 0x7f10c415b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:59.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.929+0000 7f10c9200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10c415cc90 0x7f10c415b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37966/0 (socket says 172.21.15.100:37966) 2024-09-17T04:33:59.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.929+0000 7f10c9200640 1 -- 172.21.15.100:0/3324329139 learned_addr learned my addr 172.21.15.100:0/3324329139 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:33:59.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.929+0000 7f10c3e00640 1 --2- 172.21.15.100:0/3324329139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10c40b9b80 0x7f10c40c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:59.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.929+0000 7f10bb400640 1 --2- 172.21.15.100:0/3324329139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10c41620e0 0x7f10c40ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:59.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.929+0000 7f10c9200640 1 -- 172.21.15.100:0/3324329139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10c41620e0 msgr2=0x7f10c40ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:59.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.929+0000 7f10c9200640 1 --2- 172.21.15.100:0/3324329139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10c41620e0 0x7f10c40ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.929+0000 7f10c9200640 1 -- 172.21.15.100:0/3324329139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10c40b9b80 msgr2=0x7f10c40c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:59.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.929+0000 7f10c9200640 1 --2- 172.21.15.100:0/3324329139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10c40b9b80 0x7f10c40c8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.930+0000 7f10c9200640 1 -- 172.21.15.100:0/3324329139 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c40cae20 con 0x7f10c415cc90 2024-09-17T04:33:59.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.930+0000 7f10c9200640 1 --2- 172.21.15.100:0/3324329139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10c415cc90 0x7f10c415b860 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00c730 tx=0x7f10bc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:59.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.930+0000 7f10c0c00640 1 -- 172.21.15.100:0/3324329139 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc003b70 con 0x7f10c415cc90 2024-09-17T04:33:59.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.930+0000 7f10c0c00640 1 -- 172.21.15.100:0/3324329139 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10bc003d10 con 0x7f10c415cc90 2024-09-17T04:33:59.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.930+0000 7f10c9c00640 1 -- 172.21.15.100:0/3324329139 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c40c9450 con 0x7f10c415cc90 2024-09-17T04:33:59.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.930+0000 7f10c0c00640 1 -- 172.21.15.100:0/3324329139 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc00aab0 con 0x7f10c415cc90 2024-09-17T04:33:59.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.930+0000 7f10c9c00640 1 -- 172.21.15.100:0/3324329139 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c40c9a10 con 0x7f10c415cc90 2024-09-17T04:33:59.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.931+0000 7f10c9c00640 1 -- 172.21.15.100:0/3324329139 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10c40cab90 con 0x7f10c415cc90 2024-09-17T04:33:59.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.933+0000 7f10c0c00640 1 -- 172.21.15.100:0/3324329139 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f10bc00ac50 con 0x7f10c415cc90 2024-09-17T04:33:59.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.933+0000 7f10c0c00640 1 --2- 172.21.15.100:0/3324329139 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f10a00795e0 0x7f10a007baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:59.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.934+0000 7f10c0c00640 1 -- 172.21.15.100:0/3324329139 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7f10bc0994d0 con 0x7f10c415cc90 2024-09-17T04:33:59.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.934+0000 7f10c3e00640 1 --2- 172.21.15.100:0/3324329139 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f10a00795e0 0x7f10a007baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:59.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.934+0000 7f10c3e00640 1 --2- 172.21.15.100:0/3324329139 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f10a00795e0 0x7f10a007baf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f10b4034000 tx=0x7f10b4004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:33:59.938+0000 7f10c0c00640 1 -- 172.21.15.100:0/3324329139 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f10bc062b80 con 0x7f10c415cc90 2024-09-17T04:34:00.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:00.163+0000 7f10c9c00640 1 -- 172.21.15.100:0/3324329139 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f10c40cfd30 con 0x7f10c415cc90 2024-09-17T04:34:00.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:00.163+0000 7f10c0c00640 1 -- 172.21.15.100:0/3324329139 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v121) ==== 53+0+73 (secure 0 0 0) 0x7f10bc0675b0 con 0x7f10c415cc90 2024-09-17T04:34:00.164 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:34:00.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:00.166+0000 7f10c9c00640 1 -- 172.21.15.100:0/3324329139 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f10a00795e0 msgr2=0x7f10a007baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:00.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:00.166+0000 7f10c9c00640 1 --2- 172.21.15.100:0/3324329139 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f10a00795e0 0x7f10a007baf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f10b4034000 tx=0x7f10b4004730 comp rx=0 tx=0).stop 2024-09-17T04:34:00.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:00.166+0000 7f10c9c00640 1 -- 172.21.15.100:0/3324329139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10c415cc90 msgr2=0x7f10c415b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:00.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:00.166+0000 7f10c9c00640 1 --2- 172.21.15.100:0/3324329139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10c415cc90 0x7f10c415b860 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00c730 tx=0x7f10bc00cbf0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:00.166+0000 7f10c9c00640 1 -- 172.21.15.100:0/3324329139 shutdown_connections 2024-09-17T04:34:00.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:00.166+0000 7f10c9c00640 1 --2- 172.21.15.100:0/3324329139 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f10a00795e0 0x7f10a007baf0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:00.166+0000 7f10c9c00640 1 --2- 172.21.15.100:0/3324329139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10c415cc90 0x7f10c415b860 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:00.166+0000 7f10c9c00640 1 --2- 172.21.15.100:0/3324329139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10c41620e0 0x7f10c40ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:00.166+0000 7f10c9c00640 1 --2- 172.21.15.100:0/3324329139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10c40b9b80 0x7f10c40c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:00.166+0000 7f10c9c00640 1 -- 172.21.15.100:0/3324329139 >> 172.21.15.100:0/3324329139 conn(0x7f10c409d820 msgr2=0x7f10c415ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:00.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:00.166+0000 7f10c9c00640 1 -- 172.21.15.100:0/3324329139 shutdown_connections 2024-09-17T04:34:00.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:00.166+0000 7f10c9c00640 1 -- 172.21.15.100:0/3324329139 wait complete. 2024-09-17T04:34:00.193 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T04:34:00.193 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:34:00.193 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-09-17T04:34:00.193 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T04:34:00.193 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T04:34:00.193 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T04:34:00.193 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T04:34:00.194 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:34:00.194 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:34:00.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:00.194 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-09-17T04:34:00.230 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:34:00.230 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:34:00.230 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:34:00.230 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:34:00.230 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-17T04:34:00.230 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:34:00.230 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:34:00.230 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:34:00.230 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-09-17T04:34:00.230 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:34:00.231 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:34:00.231 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-09-17T04:34:00.231 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:34:00.231 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:34:00.231 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:34:00.231 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:34:00.231 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T04:34:00.231 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42120sec preferred_lft 42120sec 2024-09-17T04:34:00.231 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link noprefixroute 2024-09-17T04:34:00.231 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:34:00.231 INFO:teuthology.orchestra.run.smithi100.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:34:00.231 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:2e:de:8e:a4:7c brd ff:ff:ff:ff:ff:ff 2024-09-17T04:34:00.231 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T04:34:00.232 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:34:00.232 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::cc78:e4ff:fe70:7fbe/64 scope link 2024-09-17T04:34:00.232 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:34:00.232 INFO:teuthology.orchestra.run.smithi100.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:34:00.232 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether ea:8c:45:9f:34:87 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:34:00.232 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::e88c:45ff:fe9f:3487/64 scope link 2024-09-17T04:34:00.232 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:34:00.232 INFO:teuthology.orchestra.run.smithi100.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:34:00.232 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:2e:de:8e:a4:7c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:34:00.232 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::c2e:deff:fe8e:a47c/64 scope link 2024-09-17T04:34:00.232 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:34:00.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:00.233 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:34:00.323 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-09-17T04:34:00.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:00.326 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-09-17T04:34:00.362 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-09-17T04:34:00.362 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:34:00.362 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T04:34:00.362 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T04:34:00.362 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:34:00.363 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:34:00.363 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-17T04:34:00.363 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T04:34:00.363 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T04:34:00.363 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T04:34:00.363 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-17T04:34:00.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:34:00.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:34:00.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:00.558 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-17T04:34:00.594 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T04:34:00.594 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T04:34:00.594 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:34:00.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:00.595 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-09-17T04:34:00.691 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:34:00.691 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T04:34:00.726 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:34:00.727 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:34:00.727 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:34:00.803 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T04:34:00.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:00.803 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:34:00.997 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:34:00.997 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:34:00.997 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:34:01.017 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:34:01.017 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:34:01.017 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:34:01.017 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:34:01.018 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:34:01.046 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T04:34:01.046 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T04:34:01.046 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-09-17T04:34:01.046 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:34:01.046 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:34:01.046 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-09-17T04:34:01.046 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-09-17T04:34:01.046 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8a399842-3a5c-4ea2-9afb-b2704332e59f" will pass to kernel 2024-09-17T04:34:01.047 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-17T04:34:01.047 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-09-17T04:34:01.048 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-17T04:34:01.048 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-09-17T04:34:01.048 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-17T04:34:01.050 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T04:34:01.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:01.051 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:34:01.109 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:34:01.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:34:01.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:34:01.205 INFO:teuthology.orchestra.run.smithi100.stdout:client9294 2024-09-17T04:34:01.206 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:34:01.206 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:34:01.241 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:34:01.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.502+0000 7f14a9800640 1 -- 172.21.15.100:0/2450679532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14a415d760 msgr2=0x7f14a41535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:01.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.502+0000 7f14a9800640 1 --2- 172.21.15.100:0/2450679532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14a415d760 0x7f14a41535c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f1494005fd0 tx=0x7f149402a540 comp rx=0 tx=0).stop 2024-09-17T04:34:01.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.502+0000 7f14a9800640 1 -- 172.21.15.100:0/2450679532 shutdown_connections 2024-09-17T04:34:01.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.502+0000 7f14a9800640 1 --2- 172.21.15.100:0/2450679532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14a40b3550 0x7f14a40b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:01.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.502+0000 7f14a9800640 1 --2- 172.21.15.100:0/2450679532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14a415d760 0x7f14a41535c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:01.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.502+0000 7f14a9800640 1 -- 172.21.15.100:0/2450679532 >> 172.21.15.100:0/2450679532 conn(0x7f14a406d730 msgr2=0x7f14a40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:01.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.503+0000 7f14a9800640 1 -- 172.21.15.100:0/2450679532 shutdown_connections 2024-09-17T04:34:01.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.503+0000 7f14a9800640 1 -- 172.21.15.100:0/2450679532 wait complete. 2024-09-17T04:34:01.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.505+0000 7f14a9800640 1 Processor -- start 2024-09-17T04:34:01.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.505+0000 7f14a9800640 1 -- start start 2024-09-17T04:34:01.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.506+0000 7f14a9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14a40ad0e0 0x7f14a414f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:01.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.506+0000 7f14a9800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14a40b3550 0x7f14a413b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:01.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.506+0000 7f14a9800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14a415d760 0x7f14a40b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:01.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.506+0000 7f14a9800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f14a406f240 con 0x7f14a40ad0e0 2024-09-17T04:34:01.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.506+0000 7f14a9800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f14a40b4e30 con 0x7f14a415d760 2024-09-17T04:34:01.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.506+0000 7f14a9800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f14a40b3ea0 con 0x7f14a40b3550 2024-09-17T04:34:01.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.507+0000 7f14a2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14a40ad0e0 0x7f14a414f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:01.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.507+0000 7f14a2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14a40ad0e0 0x7f14a414f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37976/0 (socket says 172.21.15.100:37976) 2024-09-17T04:34:01.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.507+0000 7f14a2a00640 1 -- 172.21.15.100:0/398891114 learned_addr learned my addr 172.21.15.100:0/398891114 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:01.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.507+0000 7f14a3400640 1 --2- 172.21.15.100:0/398891114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14a415d760 0x7f14a40b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:01.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.507+0000 7f14a2000640 1 --2- 172.21.15.100:0/398891114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14a40b3550 0x7f14a413b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:01.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.507+0000 7f14a2a00640 1 -- 172.21.15.100:0/398891114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14a40b3550 msgr2=0x7f14a413b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:01.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.507+0000 7f14a2a00640 1 --2- 172.21.15.100:0/398891114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14a40b3550 0x7f14a413b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:01.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.507+0000 7f14a2a00640 1 -- 172.21.15.100:0/398891114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14a415d760 msgr2=0x7f14a40b0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:01.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.507+0000 7f14a2a00640 1 --2- 172.21.15.100:0/398891114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14a415d760 0x7f14a40b0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:01.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.507+0000 7f14a2a00640 1 -- 172.21.15.100:0/398891114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a40afe80 con 0x7f14a40ad0e0 2024-09-17T04:34:01.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.507+0000 7f14a3400640 1 --2- 172.21.15.100:0/398891114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14a415d760 0x7f14a40b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:01.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.507+0000 7f14a2000640 1 --2- 172.21.15.100:0/398891114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14a40b3550 0x7f14a413b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:01.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.507+0000 7f14a2a00640 1 --2- 172.21.15.100:0/398891114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14a40ad0e0 0x7f14a414f0b0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f1494034040 tx=0x7f14940038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:01.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.508+0000 7f149aa00640 1 -- 172.21.15.100:0/398891114 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149403e030 con 0x7f14a40ad0e0 2024-09-17T04:34:01.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.508+0000 7f14a9800640 1 -- 172.21.15.100:0/398891114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14a40b0160 con 0x7f14a40ad0e0 2024-09-17T04:34:01.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.508+0000 7f149aa00640 1 -- 172.21.15.100:0/398891114 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1494003d90 con 0x7f14a40ad0e0 2024-09-17T04:34:01.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.508+0000 7f149aa00640 1 -- 172.21.15.100:0/398891114 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1494035040 con 0x7f14a40ad0e0 2024-09-17T04:34:01.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.508+0000 7f14a9800640 1 -- 172.21.15.100:0/398891114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14a40b0720 con 0x7f14a40ad0e0 2024-09-17T04:34:01.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.509+0000 7f14a9800640 1 -- 172.21.15.100:0/398891114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14a40b42f0 con 0x7f14a40ad0e0 2024-09-17T04:34:01.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.510+0000 7f149aa00640 1 -- 172.21.15.100:0/398891114 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f1494041030 con 0x7f14a40ad0e0 2024-09-17T04:34:01.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.511+0000 7f149aa00640 1 --2- 172.21.15.100:0/398891114 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f14840795b0 0x7f148407bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:01.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.511+0000 7f149aa00640 1 -- 172.21.15.100:0/398891114 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7f14940ba420 con 0x7f14a40ad0e0 2024-09-17T04:34:01.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.512+0000 7f14a2000640 1 --2- 172.21.15.100:0/398891114 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f14840795b0 0x7f148407bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:01.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.512+0000 7f14a2000640 1 --2- 172.21.15.100:0/398891114 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f14840795b0 0x7f148407bac0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f14a406ec60 tx=0x7f1490009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:01.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.516+0000 7f149aa00640 1 -- 172.21.15.100:0/398891114 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f1494084010 con 0x7f14a40ad0e0 2024-09-17T04:34:01.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.736+0000 7f14a9800640 1 -- 172.21.15.100:0/398891114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f14a414ee70 con 0x7f14a40ad0e0 2024-09-17T04:34:01.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.737+0000 7f149aa00640 1 -- 172.21.15.100:0/398891114 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v121) ==== 53+0+73 (secure 0 0 0) 0x7f1494083910 con 0x7f14a40ad0e0 2024-09-17T04:34:01.736 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:34:01.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.739+0000 7f14a9800640 1 -- 172.21.15.100:0/398891114 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f14840795b0 msgr2=0x7f148407bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:01.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.739+0000 7f14a9800640 1 --2- 172.21.15.100:0/398891114 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f14840795b0 0x7f148407bac0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f14a406ec60 tx=0x7f1490009040 comp rx=0 tx=0).stop 2024-09-17T04:34:01.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.739+0000 7f14a9800640 1 -- 172.21.15.100:0/398891114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14a40ad0e0 msgr2=0x7f14a414f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:01.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.739+0000 7f14a9800640 1 --2- 172.21.15.100:0/398891114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14a40ad0e0 0x7f14a414f0b0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f1494034040 tx=0x7f14940038a0 comp rx=0 tx=0).stop 2024-09-17T04:34:01.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.739+0000 7f14a9800640 1 -- 172.21.15.100:0/398891114 shutdown_connections 2024-09-17T04:34:01.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.739+0000 7f14a9800640 1 --2- 172.21.15.100:0/398891114 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f14840795b0 0x7f148407bac0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:01.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.739+0000 7f14a9800640 1 --2- 172.21.15.100:0/398891114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14a415d760 0x7f14a40b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:01.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.739+0000 7f14a9800640 1 --2- 172.21.15.100:0/398891114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14a40b3550 0x7f14a413b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:01.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.739+0000 7f14a9800640 1 --2- 172.21.15.100:0/398891114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14a40ad0e0 0x7f14a414f0b0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:01.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.739+0000 7f14a9800640 1 -- 172.21.15.100:0/398891114 >> 172.21.15.100:0/398891114 conn(0x7f14a406d730 msgr2=0x7f14a415c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:01.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.739+0000 7f14a9800640 1 -- 172.21.15.100:0/398891114 shutdown_connections 2024-09-17T04:34:01.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:01.739+0000 7f14a9800640 1 -- 172.21.15.100:0/398891114 wait complete. 2024-09-17T04:34:01.764 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T04:34:01.764 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:34:01.765 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2024-09-17T04:34:01.765 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T04:34:01.765 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T04:34:01.765 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T04:34:01.765 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T04:34:01.765 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:34:01.765 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:34:01.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:01.765 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2024-09-17T04:34:01.807 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:34:01.807 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:34:01.807 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:34:01.807 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:34:01.807 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-09-17T04:34:01.807 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:34:01.807 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:34:01.807 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:34:01.807 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2024-09-17T04:34:01.808 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:34:01.808 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:34:01.808 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2024-09-17T04:34:01.808 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:34:01.808 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2024-09-17T04:34:01.808 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:34:01.808 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2024-09-17T04:34:01.808 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T04:34:01.808 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42102sec preferred_lft 42102sec 2024-09-17T04:34:01.808 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ec4:7aff:fe88:7b8f/64 scope link noprefixroute 2024-09-17T04:34:01.808 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:34:01.808 INFO:teuthology.orchestra.run.smithi184.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:34:01.808 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 7e:67:e5:69:96:69 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:34:01.809 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T04:34:01.809 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:34:01.809 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::3c9d:aff:fe6e:8fcb/64 scope link 2024-09-17T04:34:01.809 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:34:01.809 INFO:teuthology.orchestra.run.smithi184.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:34:01.809 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether a6:a9:1b:a9:61:ec brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T04:34:01.809 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::a4a9:1bff:fea9:61ec/64 scope link 2024-09-17T04:34:01.809 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:34:01.809 INFO:teuthology.orchestra.run.smithi184.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:34:01.809 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 7e:67:e5:69:96:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T04:34:01.809 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::7c67:e5ff:fe69:9669/64 scope link 2024-09-17T04:34:01.809 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:34:01.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:01.810 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:34:01.906 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-09-17T04:34:01.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:01.909 DEBUG:teuthology.orchestra.run.smithi184:> route 2024-09-17T04:34:01.946 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2024-09-17T04:34:01.946 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:34:01.946 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T04:34:01.947 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T04:34:01.947 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:34:01.947 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:34:01.947 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-09-17T04:34:01.947 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T04:34:01.947 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T04:34:01.947 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T04:34:01.947 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-09-17T04:34:02.065 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:34:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:34:02.138 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:34:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:34:02.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:02.147 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-09-17T04:34:02.182 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T04:34:02.182 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T04:34:02.182 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:34:02.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:02.183 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.0 up 2024-09-17T04:34:02.279 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:34:02.279 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T04:34:02.314 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:34:02.314 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:34:02.314 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T04:34:02.389 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T04:34:02.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:02.389 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:34:02.583 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:34:02.583 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:34:02.583 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:34:02.603 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:34:02.603 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:34:02.603 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:34:02.603 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:34:02.603 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:34:02.633 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T04:34:02.633 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T04:34:02.633 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-09-17T04:34:02.633 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-09-17T04:34:02.633 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T04:34:02.633 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-09-17T04:34:02.633 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.100:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-09-17T04:34:02.633 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8a399842-3a5c-4ea2-9afb-b2704332e59f" will pass to kernel 2024-09-17T04:34:02.634 INFO:teuthology.orchestra.run.smithi184.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-17T04:34:02.634 INFO:teuthology.orchestra.run.smithi184.stdout: You just mounted a file system that supports labels which does not 2024-09-17T04:34:02.634 INFO:teuthology.orchestra.run.smithi184.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-17T04:34:02.634 INFO:teuthology.orchestra.run.smithi184.stdout: applications will generate AVC messages and not be allowed access to 2024-09-17T04:34:02.634 INFO:teuthology.orchestra.run.smithi184.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-17T04:34:02.637 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T04:34:02.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:02.638 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T04:34:02.693 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T04:34:02.771 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:34:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:34:02.774 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-17T04:34:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:34:02.782 INFO:teuthology.orchestra.run.smithi184.stdout:client8671 2024-09-17T04:34:02.782 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:34:02.782 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:34:02.818 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T04:34:03.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.044+0000 7f9439400640 1 -- 172.21.15.100:0/390640160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94340aec80 msgr2=0x7f94340b56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:03.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.044+0000 7f9439400640 1 --2- 172.21.15.100:0/390640160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94340aec80 0x7f94340b56e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f9424009830 tx=0x7f942402a480 comp rx=0 tx=0).stop 2024-09-17T04:34:03.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.045+0000 7f9439400640 1 -- 172.21.15.100:0/390640160 shutdown_connections 2024-09-17T04:34:03.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.045+0000 7f9439400640 1 --2- 172.21.15.100:0/390640160 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94340ad970 0x7f94340add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.045+0000 7f9439400640 1 --2- 172.21.15.100:0/390640160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94340aec80 0x7f94340b56e0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.045+0000 7f9439400640 1 -- 172.21.15.100:0/390640160 >> 172.21.15.100:0/390640160 conn(0x7f94340b2220 msgr2=0x7f94340d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:03.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.045+0000 7f9439400640 1 -- 172.21.15.100:0/390640160 shutdown_connections 2024-09-17T04:34:03.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.045+0000 7f9439400640 1 -- 172.21.15.100:0/390640160 wait complete. 2024-09-17T04:34:03.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.047+0000 7f9439400640 1 Processor -- start 2024-09-17T04:34:03.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.047+0000 7f9439400640 1 -- start start 2024-09-17T04:34:03.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.048+0000 7f9439400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94340ad970 0x7f943414d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:03.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.048+0000 7f9439400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94340ae2c0 0x7f943415eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:03.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.048+0000 7f9439400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94340aec80 0x7f94341f8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:03.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.048+0000 7f9439400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f94340705f0 con 0x7f94340ae2c0 2024-09-17T04:34:03.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.048+0000 7f9439400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9434070470 con 0x7f94340aec80 2024-09-17T04:34:03.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.048+0000 7f9439400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f94340ae6d0 con 0x7f94340ad970 2024-09-17T04:34:03.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.049+0000 7f942be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94340ae2c0 0x7f943415eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:03.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.049+0000 7f942be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94340ae2c0 0x7f943415eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37984/0 (socket says 172.21.15.100:37984) 2024-09-17T04:34:03.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.049+0000 7f942be00640 1 -- 172.21.15.100:0/3013061532 learned_addr learned my addr 172.21.15.100:0/3013061532 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:03.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.049+0000 7f9433e00640 1 --2- 172.21.15.100:0/3013061532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94340ad970 0x7f943414d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:03.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.049+0000 7f9438a00640 1 --2- 172.21.15.100:0/3013061532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94340aec80 0x7f94341f8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:03.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.049+0000 7f942be00640 1 -- 172.21.15.100:0/3013061532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94340ad970 msgr2=0x7f943414d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:03.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.049+0000 7f942be00640 1 --2- 172.21.15.100:0/3013061532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94340ad970 0x7f943414d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.049+0000 7f942be00640 1 -- 172.21.15.100:0/3013061532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94340aec80 msgr2=0x7f94341f8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:03.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.049+0000 7f942be00640 1 --2- 172.21.15.100:0/3013061532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94340aec80 0x7f94341f8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.049+0000 7f942be00640 1 -- 172.21.15.100:0/3013061532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94341f6850 con 0x7f94340ae2c0 2024-09-17T04:34:03.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.050+0000 7f942be00640 1 --2- 172.21.15.100:0/3013061532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94340ae2c0 0x7f943415eaf0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f941c00e790 tx=0x7f941c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:03.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.050+0000 7f9430c00640 1 -- 172.21.15.100:0/3013061532 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c00eea0 con 0x7f94340ae2c0 2024-09-17T04:34:03.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.050+0000 7f9430c00640 1 -- 172.21.15.100:0/3013061532 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f941c00cdb0 con 0x7f94340ae2c0 2024-09-17T04:34:03.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.050+0000 7f9430c00640 1 -- 172.21.15.100:0/3013061532 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c010690 con 0x7f94340ae2c0 2024-09-17T04:34:03.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.050+0000 7f9439400640 1 -- 172.21.15.100:0/3013061532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f943415f6d0 con 0x7f94340ae2c0 2024-09-17T04:34:03.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.050+0000 7f9439400640 1 -- 172.21.15.100:0/3013061532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f943409ae90 con 0x7f94340ae2c0 2024-09-17T04:34:03.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.051+0000 7f9439400640 1 -- 172.21.15.100:0/3013061532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f943415f440 con 0x7f94340ae2c0 2024-09-17T04:34:03.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.052+0000 7f9430c00640 1 -- 172.21.15.100:0/3013061532 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f941c010830 con 0x7f94340ae2c0 2024-09-17T04:34:03.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.053+0000 7f9430c00640 1 --2- 172.21.15.100:0/3013061532 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f94100795b0 0x7f941007bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:03.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.053+0000 7f9430c00640 1 -- 172.21.15.100:0/3013061532 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7f941c09a1e0 con 0x7f94340ae2c0 2024-09-17T04:34:03.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.053+0000 7f9433e00640 1 --2- 172.21.15.100:0/3013061532 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f94100795b0 0x7f941007bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:03.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.054+0000 7f9433e00640 1 --2- 172.21.15.100:0/3013061532 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f94100795b0 0x7f941007bac0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9424034000 tx=0x7f9424004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:03.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.058+0000 7f9430c00640 1 -- 172.21.15.100:0/3013061532 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f941c063860 con 0x7f94340ae2c0 2024-09-17T04:34:03.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.278+0000 7f9439400640 1 -- 172.21.15.100:0/3013061532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f94340add80 con 0x7f94340ae2c0 2024-09-17T04:34:03.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.331+0000 7f9430c00640 1 -- 172.21.15.100:0/3013061532 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v122) ==== 136+0+0 (secure 0 0 0) 0x7f941c068290 con 0x7f94340ae2c0 2024-09-17T04:34:03.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.333+0000 7f9439400640 1 -- 172.21.15.100:0/3013061532 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f94100795b0 msgr2=0x7f941007bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:03.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.333+0000 7f9439400640 1 --2- 172.21.15.100:0/3013061532 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f94100795b0 0x7f941007bac0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9424034000 tx=0x7f9424004730 comp rx=0 tx=0).stop 2024-09-17T04:34:03.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.333+0000 7f9439400640 1 -- 172.21.15.100:0/3013061532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94340ae2c0 msgr2=0x7f943415eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:03.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.333+0000 7f9439400640 1 --2- 172.21.15.100:0/3013061532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94340ae2c0 0x7f943415eaf0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f941c00e790 tx=0x7f941c00ec50 comp rx=0 tx=0).stop 2024-09-17T04:34:03.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.333+0000 7f9439400640 1 -- 172.21.15.100:0/3013061532 shutdown_connections 2024-09-17T04:34:03.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.333+0000 7f9439400640 1 --2- 172.21.15.100:0/3013061532 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f94100795b0 0x7f941007bac0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.333+0000 7f9439400640 1 --2- 172.21.15.100:0/3013061532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94340aec80 0x7f94341f8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.333+0000 7f9439400640 1 --2- 172.21.15.100:0/3013061532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94340ae2c0 0x7f943415eaf0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.333+0000 7f9439400640 1 --2- 172.21.15.100:0/3013061532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94340ad970 0x7f943414d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.333+0000 7f9439400640 1 -- 172.21.15.100:0/3013061532 >> 172.21.15.100:0/3013061532 conn(0x7f94340b2220 msgr2=0x7f94340cdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:03.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.333+0000 7f9439400640 1 -- 172.21.15.100:0/3013061532 shutdown_connections 2024-09-17T04:34:03.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.334+0000 7f9439400640 1 -- 172.21.15.100:0/3013061532 wait complete. 2024-09-17T04:34:03.360 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T04:34:03.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.581+0000 7f24d1a00640 1 -- 172.21.15.100:0/1616923710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24cc0ad9c0 msgr2=0x7f24cc0af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:03.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.581+0000 7f24d1a00640 1 --2- 172.21.15.100:0/1616923710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24cc0ad9c0 0x7f24cc0af170 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f24cc0af6b0 tx=0x7f24bc02a560 comp rx=0 tx=0).stop 2024-09-17T04:34:03.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.581+0000 7f24d1a00640 1 -- 172.21.15.100:0/1616923710 shutdown_connections 2024-09-17T04:34:03.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.581+0000 7f24d1a00640 1 --2- 172.21.15.100:0/1616923710 >> v2:172.21.15.184:3300/0 conn(0x7f24cc15a0f0 0x7f24cc15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.581+0000 7f24d1a00640 1 --2- 172.21.15.100:0/1616923710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24cc0ad9c0 0x7f24cc0af170 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.581+0000 7f24d1a00640 1 -- 172.21.15.100:0/1616923710 >> 172.21.15.100:0/1616923710 conn(0x7f24cc0b1e40 msgr2=0x7f24cc0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:03.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.581+0000 7f24d1a00640 1 -- 172.21.15.100:0/1616923710 shutdown_connections 2024-09-17T04:34:03.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.581+0000 7f24d1a00640 1 -- 172.21.15.100:0/1616923710 wait complete. 2024-09-17T04:34:03.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.584+0000 7f24d1a00640 1 Processor -- start 2024-09-17T04:34:03.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.584+0000 7f24d1a00640 1 -- start start 2024-09-17T04:34:03.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.585+0000 7f24d1a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24cc0ad9c0 0x7f24cc1f8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:03.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.585+0000 7f24d1a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24cc0b2f60 0x7f24cc14b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:03.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.585+0000 7f24d1a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24cc15a0f0 0x7f24cc14b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:03.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.585+0000 7f24d1a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc06f410 con 0x7f24cc0b2f60 2024-09-17T04:34:03.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.585+0000 7f24d1a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc06f290 con 0x7f24cc0ad9c0 2024-09-17T04:34:03.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.585+0000 7f24d1a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f24cc06f590 con 0x7f24cc15a0f0 2024-09-17T04:34:03.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.585+0000 7f24c3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24cc0b2f60 0x7f24cc14b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:03.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.585+0000 7f24c3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24cc0b2f60 0x7f24cc14b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37994/0 (socket says 172.21.15.100:37994) 2024-09-17T04:34:03.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.585+0000 7f24c3400640 1 -- 172.21.15.100:0/477806196 learned_addr learned my addr 172.21.15.100:0/477806196 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:03.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.586+0000 7f24cbe00640 1 --2- 172.21.15.100:0/477806196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24cc0ad9c0 0x7f24cc1f8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:03.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.586+0000 7f24d1000640 1 --2- 172.21.15.100:0/477806196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24cc15a0f0 0x7f24cc14b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:03.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.586+0000 7f24c3400640 1 -- 172.21.15.100:0/477806196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24cc15a0f0 msgr2=0x7f24cc14b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:03.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.586+0000 7f24c3400640 1 --2- 172.21.15.100:0/477806196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24cc15a0f0 0x7f24cc14b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.586+0000 7f24c3400640 1 -- 172.21.15.100:0/477806196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24cc0ad9c0 msgr2=0x7f24cc1f8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:03.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.586+0000 7f24c3400640 1 --2- 172.21.15.100:0/477806196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24cc0ad9c0 0x7f24cc1f8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.586+0000 7f24c3400640 1 -- 172.21.15.100:0/477806196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24cc14bd50 con 0x7f24cc0b2f60 2024-09-17T04:34:03.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.586+0000 7f24c3400640 1 --2- 172.21.15.100:0/477806196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24cc0b2f60 0x7f24cc14b2d0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f24b400c400 tx=0x7f24b400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:03.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.586+0000 7f24c8c00640 1 -- 172.21.15.100:0/477806196 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4017070 con 0x7f24cc0b2f60 2024-09-17T04:34:03.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.586+0000 7f24c8c00640 1 -- 172.21.15.100:0/477806196 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24b40053c0 con 0x7f24cc0b2f60 2024-09-17T04:34:03.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.586+0000 7f24d1a00640 1 -- 172.21.15.100:0/477806196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24cc14c030 con 0x7f24cc0b2f60 2024-09-17T04:34:03.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.587+0000 7f24c8c00640 1 -- 172.21.15.100:0/477806196 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4003850 con 0x7f24cc0b2f60 2024-09-17T04:34:03.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.587+0000 7f24d1a00640 1 -- 172.21.15.100:0/477806196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24cc14edf0 con 0x7f24cc0b2f60 2024-09-17T04:34:03.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.588+0000 7f24d1a00640 1 -- 172.21.15.100:0/477806196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24cc14e880 con 0x7f24cc0b2f60 2024-09-17T04:34:03.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.589+0000 7f24c8c00640 1 -- 172.21.15.100:0/477806196 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f24b401a070 con 0x7f24cc0b2f60 2024-09-17T04:34:03.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.590+0000 7f24c8c00640 1 --2- 172.21.15.100:0/477806196 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f24a80795b0 0x7f24a807bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:03.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.590+0000 7f24c8c00640 1 -- 172.21.15.100:0/477806196 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7f24b4098c50 con 0x7f24cc0b2f60 2024-09-17T04:34:03.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.590+0000 7f24cbe00640 1 --2- 172.21.15.100:0/477806196 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f24a80795b0 0x7f24a807bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:03.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.591+0000 7f24cbe00640 1 --2- 172.21.15.100:0/477806196 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f24a80795b0 0x7f24a807bac0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f24bc033000 tx=0x7f24bc004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:03.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.595+0000 7f24c8c00640 1 -- 172.21.15.100:0/477806196 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f24cc14bd50 con 0x7f24cc0b2f60 2024-09-17T04:34:03.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.800+0000 7f24d1a00640 1 -- 172.21.15.100:0/477806196 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f24cc14efc0 con 0x7f24a80795b0 2024-09-17T04:34:03.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.802+0000 7f24c8c00640 1 -- 172.21.15.100:0/477806196 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f24cc14efc0 con 0x7f24a80795b0 2024-09-17T04:34:03.801 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-09-17T04:34:03.801 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-17T04:34:03.801 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephfs" 2024-09-17T04:34:03.801 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-17T04:34:03.802 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-09-17T04:34:03.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.804+0000 7f24d1a00640 1 -- 172.21.15.100:0/477806196 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f24a80795b0 msgr2=0x7f24a807bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:03.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.804+0000 7f24d1a00640 1 --2- 172.21.15.100:0/477806196 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f24a80795b0 0x7f24a807bac0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f24bc033000 tx=0x7f24bc004730 comp rx=0 tx=0).stop 2024-09-17T04:34:03.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.804+0000 7f24d1a00640 1 -- 172.21.15.100:0/477806196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24cc0b2f60 msgr2=0x7f24cc14b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:03.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.804+0000 7f24d1a00640 1 --2- 172.21.15.100:0/477806196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24cc0b2f60 0x7f24cc14b2d0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f24b400c400 tx=0x7f24b400c8c0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.804+0000 7f24d1a00640 1 -- 172.21.15.100:0/477806196 shutdown_connections 2024-09-17T04:34:03.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.804+0000 7f24d1a00640 1 --2- 172.21.15.100:0/477806196 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f24a80795b0 0x7f24a807bac0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.804+0000 7f24d1a00640 1 --2- 172.21.15.100:0/477806196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24cc15a0f0 0x7f24cc14b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.804+0000 7f24d1a00640 1 --2- 172.21.15.100:0/477806196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24cc0b2f60 0x7f24cc14b2d0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.804+0000 7f24d1a00640 1 --2- 172.21.15.100:0/477806196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24cc0ad9c0 0x7f24cc1f8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.804+0000 7f24d1a00640 1 -- 172.21.15.100:0/477806196 >> 172.21.15.100:0/477806196 conn(0x7f24cc0b1e40 msgr2=0x7f24cc0b10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:03.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.804+0000 7f24d1a00640 1 -- 172.21.15.100:0/477806196 shutdown_connections 2024-09-17T04:34:03.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:03.804+0000 7f24d1a00640 1 -- 172.21.15.100:0/477806196 wait complete. 2024-09-17T04:34:03.831 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T04:34:04.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.055+0000 7f125ae00640 1 -- 172.21.15.100:0/2422572980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1254145a40 msgr2=0x7f1254073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:04.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.055+0000 7f125ae00640 1 --2- 172.21.15.100:0/2422572980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1254145a40 0x7f1254073540 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f12541464e0 tx=0x7f124002a5f0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.055+0000 7f125ae00640 1 -- 172.21.15.100:0/2422572980 shutdown_connections 2024-09-17T04:34:04.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.055+0000 7f125ae00640 1 --2- 172.21.15.100:0/2422572980 >> v2:172.21.15.184:3300/0 conn(0x7f1254073a80 0x7f1254160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.055+0000 7f125ae00640 1 --2- 172.21.15.100:0/2422572980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1254145a40 0x7f1254073540 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.055+0000 7f125ae00640 1 --2- 172.21.15.100:0/2422572980 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1254145550 0x7f125406fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.055+0000 7f125ae00640 1 -- 172.21.15.100:0/2422572980 >> 172.21.15.100:0/2422572980 conn(0x7f125406d1c0 msgr2=0x7f125406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:04.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.055+0000 7f125ae00640 1 -- 172.21.15.100:0/2422572980 shutdown_connections 2024-09-17T04:34:04.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.055+0000 7f125ae00640 1 -- 172.21.15.100:0/2422572980 wait complete. 2024-09-17T04:34:04.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.057+0000 7f125ae00640 1 Processor -- start 2024-09-17T04:34:04.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.058+0000 7f125ae00640 1 -- start start 2024-09-17T04:34:04.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.059+0000 7f125ae00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1254073a80 0x7f12541f6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:04.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.059+0000 7f125ae00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1254145550 0x7f12540adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:04.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.059+0000 7f125ae00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1254145a40 0x7f125415bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:04.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.059+0000 7f125ae00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1254160c20 con 0x7f1254073a80 2024-09-17T04:34:04.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.059+0000 7f125ae00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1254160aa0 con 0x7f1254145550 2024-09-17T04:34:04.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.059+0000 7f125ae00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1254160da0 con 0x7f1254145a40 2024-09-17T04:34:04.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.059+0000 7f1253e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1254073a80 0x7f12541f6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:04.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.059+0000 7f1253e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1254073a80 0x7f12541f6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38014/0 (socket says 172.21.15.100:38014) 2024-09-17T04:34:04.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.059+0000 7f1253e00640 1 -- 172.21.15.100:0/1127912000 learned_addr learned my addr 172.21.15.100:0/1127912000 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:04.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.060+0000 7f1258e00640 1 --2- 172.21.15.100:0/1127912000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1254145a40 0x7f125415bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:04.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.060+0000 7f1253400640 1 --2- 172.21.15.100:0/1127912000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1254145550 0x7f12540adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:04.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.060+0000 7f1253e00640 1 -- 172.21.15.100:0/1127912000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1254145a40 msgr2=0x7f125415bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:04.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.060+0000 7f1253e00640 1 --2- 172.21.15.100:0/1127912000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1254145a40 0x7f125415bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.060+0000 7f1253e00640 1 -- 172.21.15.100:0/1127912000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1254145550 msgr2=0x7f12540adbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:04.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.060+0000 7f1253e00640 1 --2- 172.21.15.100:0/1127912000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1254145550 0x7f12540adbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.060+0000 7f1253e00640 1 -- 172.21.15.100:0/1127912000 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12540b3620 con 0x7f1254073a80 2024-09-17T04:34:04.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.060+0000 7f1253e00640 1 --2- 172.21.15.100:0/1127912000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1254073a80 0x7f12541f6930 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f124400d740 tx=0x7f124400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:04.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.060+0000 7f124be00640 1 -- 172.21.15.100:0/1127912000 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1244017070 con 0x7f1254073a80 2024-09-17T04:34:04.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.060+0000 7f124be00640 1 -- 172.21.15.100:0/1127912000 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1244009cc0 con 0x7f1254073a80 2024-09-17T04:34:04.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.060+0000 7f124be00640 1 -- 172.21.15.100:0/1127912000 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12440106c0 con 0x7f1254073a80 2024-09-17T04:34:04.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.060+0000 7f125ae00640 1 -- 172.21.15.100:0/1127912000 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12540b3900 con 0x7f1254073a80 2024-09-17T04:34:04.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.061+0000 7f125ae00640 1 -- 172.21.15.100:0/1127912000 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12540b3ec0 con 0x7f1254073a80 2024-09-17T04:34:04.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.062+0000 7f125ae00640 1 -- 172.21.15.100:0/1127912000 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f125406fb40 con 0x7f1254073a80 2024-09-17T04:34:04.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.063+0000 7f124be00640 1 -- 172.21.15.100:0/1127912000 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f124401a070 con 0x7f1254073a80 2024-09-17T04:34:04.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.064+0000 7f124be00640 1 --2- 172.21.15.100:0/1127912000 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f12340795e0 0x7f123407baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:04.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.064+0000 7f124be00640 1 -- 172.21.15.100:0/1127912000 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7f12440998e0 con 0x7f1254073a80 2024-09-17T04:34:04.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.064+0000 7f1253400640 1 --2- 172.21.15.100:0/1127912000 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f12340795e0 0x7f123407baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:04.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.065+0000 7f1253400640 1 --2- 172.21.15.100:0/1127912000 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f12340795e0 0x7f123407baf0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1240033000 tx=0x7f1240004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:04.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.069+0000 7f124be00640 1 -- 172.21.15.100:0/1127912000 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f1244062f00 con 0x7f1254073a80 2024-09-17T04:34:04.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.252+0000 7f125ae00640 1 -- 172.21.15.100:0/1127912000 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f12540cc690 con 0x7f1254073a80 2024-09-17T04:34:04.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.252+0000 7f124be00640 1 -- 172.21.15.100:0/1127912000 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v52)=0 v52) ==== 122+0+0 (secure 0 0 0) 0x7f1244067930 con 0x7f1254073a80 2024-09-17T04:34:04.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.255+0000 7f125ae00640 1 -- 172.21.15.100:0/1127912000 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f12340795e0 msgr2=0x7f123407baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:04.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.255+0000 7f125ae00640 1 --2- 172.21.15.100:0/1127912000 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f12340795e0 0x7f123407baf0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1240033000 tx=0x7f1240004730 comp rx=0 tx=0).stop 2024-09-17T04:34:04.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.255+0000 7f125ae00640 1 -- 172.21.15.100:0/1127912000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1254073a80 msgr2=0x7f12541f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:04.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.255+0000 7f125ae00640 1 --2- 172.21.15.100:0/1127912000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1254073a80 0x7f12541f6930 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f124400d740 tx=0x7f124400dc00 comp rx=0 tx=0).stop 2024-09-17T04:34:04.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.255+0000 7f125ae00640 1 -- 172.21.15.100:0/1127912000 shutdown_connections 2024-09-17T04:34:04.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.255+0000 7f125ae00640 1 --2- 172.21.15.100:0/1127912000 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f12340795e0 0x7f123407baf0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.255+0000 7f125ae00640 1 --2- 172.21.15.100:0/1127912000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1254145a40 0x7f125415bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.255+0000 7f125ae00640 1 --2- 172.21.15.100:0/1127912000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1254145550 0x7f12540adbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.255+0000 7f125ae00640 1 --2- 172.21.15.100:0/1127912000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1254073a80 0x7f12541f6930 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.255+0000 7f125ae00640 1 -- 172.21.15.100:0/1127912000 >> 172.21.15.100:0/1127912000 conn(0x7f125406d1c0 msgr2=0x7f12540b29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:04.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.255+0000 7f125ae00640 1 -- 172.21.15.100:0/1127912000 shutdown_connections 2024-09-17T04:34:04.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.255+0000 7f125ae00640 1 -- 172.21.15.100:0/1127912000 wait complete. 2024-09-17T04:34:04.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:04.276 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp53780/subvol98520) 2024-09-17T04:34:04.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:04.340 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-17T04:34:04.377 INFO:teuthology.orchestra.run.smithi100.stdout:cephfs_data 2024-09-17T04:34:04.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:04.378 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp53780/subvol98520) 2024-09-17T04:34:04.479 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:34:04.479 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp53780/subvol98520/.meta 2024-09-17T04:34:04.538 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol98520 subvol_grp53780 2024-09-17T04:34:04.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.806+0000 7fc297200640 1 -- 172.21.15.100:0/1741336852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2980af3c0 msgr2=0x7fc2981602b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:04.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.806+0000 7fc297200640 1 --2- 172.21.15.100:0/1741336852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2980af3c0 0x7fc2981602b0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fc29806e750 tx=0x7fc29002a820 comp rx=0 tx=0).stop 2024-09-17T04:34:04.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.806+0000 7fc297200640 1 -- 172.21.15.100:0/1741336852 shutdown_connections 2024-09-17T04:34:04.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.806+0000 7fc297200640 1 --2- 172.21.15.100:0/1741336852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2980af3c0 0x7fc2981602b0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.806+0000 7fc297200640 1 --2- 172.21.15.100:0/1741336852 >> v2:172.21.15.184:3300/0 conn(0x7fc2980705b0 0x7fc2980b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.806+0000 7fc297200640 1 -- 172.21.15.100:0/1741336852 >> 172.21.15.100:0/1741336852 conn(0x7fc2980b1dd0 msgr2=0x7fc2980b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:04.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.807+0000 7fc297200640 1 -- 172.21.15.100:0/1741336852 shutdown_connections 2024-09-17T04:34:04.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.807+0000 7fc297200640 1 -- 172.21.15.100:0/1741336852 wait complete. 2024-09-17T04:34:04.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.809+0000 7fc297200640 1 Processor -- start 2024-09-17T04:34:04.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.809+0000 7fc297200640 1 -- start start 2024-09-17T04:34:04.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.810+0000 7fc297200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2980705b0 0x7fc2981f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:04.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.810+0000 7fc297200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2980af3c0 0x7fc2981e5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:04.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.810+0000 7fc297200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2980b3ec0 0x7fc2981e6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:04.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.810+0000 7fc297200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc298160b80 con 0x7fc2980705b0 2024-09-17T04:34:04.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.810+0000 7fc297200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc298160a00 con 0x7fc2980b3ec0 2024-09-17T04:34:04.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.810+0000 7fc297200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc298153c40 con 0x7fc2980af3c0 2024-09-17T04:34:04.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.810+0000 7fc295e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2980705b0 0x7fc2981f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:04.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.811+0000 7fc295e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2980705b0 0x7fc2981f27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43234/0 (socket says 172.21.15.100:43234) 2024-09-17T04:34:04.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.811+0000 7fc295e00640 1 -- 172.21.15.100:0/3900490951 learned_addr learned my addr 172.21.15.100:0/3900490951 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:04.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.811+0000 7fc296800640 1 --2- 172.21.15.100:0/3900490951 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2980b3ec0 0x7fc2981e6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:04.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.811+0000 7fc28d400640 1 --2- 172.21.15.100:0/3900490951 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2980af3c0 0x7fc2981e5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:04.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.811+0000 7fc295e00640 1 -- 172.21.15.100:0/3900490951 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2980af3c0 msgr2=0x7fc2981e5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:04.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.811+0000 7fc295e00640 1 --2- 172.21.15.100:0/3900490951 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2980af3c0 0x7fc2981e5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.811+0000 7fc295e00640 1 -- 172.21.15.100:0/3900490951 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2980b3ec0 msgr2=0x7fc2981e6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:04.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.811+0000 7fc295e00640 1 --2- 172.21.15.100:0/3900490951 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2980b3ec0 0x7fc2981e6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.811+0000 7fc295e00640 1 -- 172.21.15.100:0/3900490951 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29813dcb0 con 0x7fc2980705b0 2024-09-17T04:34:04.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.811+0000 7fc28d400640 1 --2- 172.21.15.100:0/3900490951 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2980af3c0 0x7fc2981e5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:04.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.811+0000 7fc295e00640 1 --2- 172.21.15.100:0/3900490951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2980705b0 0x7fc2981f27e0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fc28800c760 tx=0x7fc28800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:04.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.812+0000 7fc28ea00640 1 -- 172.21.15.100:0/3900490951 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc288016020 con 0x7fc2980705b0 2024-09-17T04:34:04.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.812+0000 7fc28ea00640 1 -- 172.21.15.100:0/3900490951 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc28800a950 con 0x7fc2980705b0 2024-09-17T04:34:04.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.812+0000 7fc297200640 1 -- 172.21.15.100:0/3900490951 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc29813b300 con 0x7fc2980705b0 2024-09-17T04:34:04.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.812+0000 7fc28ea00640 1 -- 172.21.15.100:0/3900490951 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28800ada0 con 0x7fc2980705b0 2024-09-17T04:34:04.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.812+0000 7fc297200640 1 -- 172.21.15.100:0/3900490951 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc29813b8c0 con 0x7fc2980705b0 2024-09-17T04:34:04.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.813+0000 7fc297200640 1 -- 172.21.15.100:0/3900490951 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2980b47c0 con 0x7fc2980705b0 2024-09-17T04:34:04.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.814+0000 7fc28ea00640 1 -- 172.21.15.100:0/3900490951 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7fc288019020 con 0x7fc2980705b0 2024-09-17T04:34:04.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.815+0000 7fc28ea00640 1 --2- 172.21.15.100:0/3900490951 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc2740795b0 0x7fc27407bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:04.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.815+0000 7fc28ea00640 1 -- 172.21.15.100:0/3900490951 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7fc2880996a0 con 0x7fc2980705b0 2024-09-17T04:34:04.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.815+0000 7fc28d400640 1 --2- 172.21.15.100:0/3900490951 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc2740795b0 0x7fc27407bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:04.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.816+0000 7fc28d400640 1 --2- 172.21.15.100:0/3900490951 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc2740795b0 0x7fc27407bac0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc2981f6340 tx=0x7fc28000a520 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:04.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:04.819+0000 7fc28ea00640 1 -- 172.21.15.100:0/3900490951 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc288020020 con 0x7fc2980705b0 2024-09-17T04:34:05.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.031+0000 7fc297200640 1 -- 172.21.15.100:0/3900490951 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol98520", "group_name": "subvol_grp53780", "target": ["mon-mgr", ""]}) -- 0x7fc2981f2d20 con 0x7fc2740795b0 2024-09-17T04:34:05.037 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:34:05.038+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:34:05.037 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:34:05.038+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:34:05.037 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:34:05.038+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:34:05.037 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:34:05.038+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:34:05.038 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:34:05.038+0000 7f30b3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:34:05.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.079+0000 7fc28ea00640 1 -- 172.21.15.100:0/3900490951 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc28000a730 con 0x7fc2740795b0 2024-09-17T04:34:05.079 INFO:teuthology.orchestra.run.smithi100.stdout:/volumes/subvol_grp53780/subvol98520 2024-09-17T04:34:05.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.082+0000 7fc297200640 1 -- 172.21.15.100:0/3900490951 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc2740795b0 msgr2=0x7fc27407bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:05.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.082+0000 7fc297200640 1 --2- 172.21.15.100:0/3900490951 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc2740795b0 0x7fc27407bac0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc2981f6340 tx=0x7fc28000a520 comp rx=0 tx=0).stop 2024-09-17T04:34:05.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.082+0000 7fc297200640 1 -- 172.21.15.100:0/3900490951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2980705b0 msgr2=0x7fc2981f27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:05.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.082+0000 7fc297200640 1 --2- 172.21.15.100:0/3900490951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2980705b0 0x7fc2981f27e0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fc28800c760 tx=0x7fc28800cc20 comp rx=0 tx=0).stop 2024-09-17T04:34:05.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.082+0000 7fc297200640 1 -- 172.21.15.100:0/3900490951 shutdown_connections 2024-09-17T04:34:05.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.082+0000 7fc297200640 1 --2- 172.21.15.100:0/3900490951 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc2740795b0 0x7fc27407bac0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.082+0000 7fc297200640 1 --2- 172.21.15.100:0/3900490951 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2980b3ec0 0x7fc2981e6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.082+0000 7fc297200640 1 --2- 172.21.15.100:0/3900490951 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2980af3c0 0x7fc2981e5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.082+0000 7fc297200640 1 --2- 172.21.15.100:0/3900490951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2980705b0 0x7fc2981f27e0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.082+0000 7fc297200640 1 -- 172.21.15.100:0/3900490951 >> 172.21.15.100:0/3900490951 conn(0x7fc2980b1dd0 msgr2=0x7fc29815f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:05.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.082+0000 7fc297200640 1 -- 172.21.15.100:0/3900490951 shutdown_connections 2024-09-17T04:34:05.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.082+0000 7fc297200640 1 -- 172.21.15.100:0/3900490951 wait complete. 2024-09-17T04:34:05.103 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol98520 subvol_grp53780 2024-09-17T04:34:05.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.328+0000 7f24bbc00640 1 -- 172.21.15.100:0/3083014731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24b40716b0 msgr2=0x7f24b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:05.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.328+0000 7f24bbc00640 1 --2- 172.21.15.100:0/3083014731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24b40716b0 0x7f24b406e2f0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f24b40d07b0 tx=0x7f24ac02a980 comp rx=0 tx=0).stop 2024-09-17T04:34:05.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.328+0000 7f24bbc00640 1 -- 172.21.15.100:0/3083014731 shutdown_connections 2024-09-17T04:34:05.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.328+0000 7f24bbc00640 1 --2- 172.21.15.100:0/3083014731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24b40716b0 0x7f24b406e2f0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.328+0000 7f24bbc00640 1 --2- 172.21.15.100:0/3083014731 >> v2:172.21.15.184:3300/0 conn(0x7f24b40712a0 0x7f24b406ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.328+0000 7f24bbc00640 1 --2- 172.21.15.100:0/3083014731 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24b4152990 0x7f24b415c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.328+0000 7f24bbc00640 1 -- 172.21.15.100:0/3083014731 >> 172.21.15.100:0/3083014731 conn(0x7f24b4073890 msgr2=0x7f24b4073ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:05.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.328+0000 7f24bbc00640 1 -- 172.21.15.100:0/3083014731 shutdown_connections 2024-09-17T04:34:05.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.328+0000 7f24bbc00640 1 -- 172.21.15.100:0/3083014731 wait complete. 2024-09-17T04:34:05.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.330+0000 7f24bbc00640 1 Processor -- start 2024-09-17T04:34:05.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.330+0000 7f24bbc00640 1 -- start start 2024-09-17T04:34:05.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.332+0000 7f24bbc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24b40712a0 0x7f24b414cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.332+0000 7f24bbc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24b40716b0 0x7f24b414d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.332+0000 7f24bbc00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24b4152990 0x7f24b414d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.332+0000 7f24bbc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f24b40b51b0 con 0x7f24b40716b0 2024-09-17T04:34:05.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.332+0000 7f24bbc00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f24b40b5030 con 0x7f24b40712a0 2024-09-17T04:34:05.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.332+0000 7f24bbc00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f24b40b5330 con 0x7f24b4152990 2024-09-17T04:34:05.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.332+0000 7f24abe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24b40716b0 0x7f24b414d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.332+0000 7f24abe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24b40716b0 0x7f24b414d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43254/0 (socket says 172.21.15.100:43254) 2024-09-17T04:34:05.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.332+0000 7f24abe00640 1 -- 172.21.15.100:0/3601653713 learned_addr learned my addr 172.21.15.100:0/3601653713 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:05.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.332+0000 7f24b9200640 1 --2- 172.21.15.100:0/3601653713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24b40712a0 0x7f24b414cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.332+0000 7f24b9c00640 1 --2- 172.21.15.100:0/3601653713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24b4152990 0x7f24b414d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.332+0000 7f24abe00640 1 -- 172.21.15.100:0/3601653713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24b4152990 msgr2=0x7f24b414d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:05.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.332+0000 7f24abe00640 1 --2- 172.21.15.100:0/3601653713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24b4152990 0x7f24b414d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.332+0000 7f24abe00640 1 -- 172.21.15.100:0/3601653713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24b40712a0 msgr2=0x7f24b414cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:05.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.332+0000 7f24abe00640 1 --2- 172.21.15.100:0/3601653713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24b40712a0 0x7f24b414cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.332+0000 7f24abe00640 1 -- 172.21.15.100:0/3601653713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b414b290 con 0x7f24b40716b0 2024-09-17T04:34:05.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.333+0000 7f24abe00640 1 --2- 172.21.15.100:0/3601653713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24b40716b0 0x7f24b414d1c0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f249c00d650 tx=0x7f249c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:05.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.333+0000 7f24b1600640 1 -- 172.21.15.100:0/3601653713 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c017070 con 0x7f24b40716b0 2024-09-17T04:34:05.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.333+0000 7f24b1600640 1 -- 172.21.15.100:0/3601653713 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f249c004310 con 0x7f24b40716b0 2024-09-17T04:34:05.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.333+0000 7f24b1600640 1 -- 172.21.15.100:0/3601653713 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c005280 con 0x7f24b40716b0 2024-09-17T04:34:05.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.333+0000 7f24bbc00640 1 -- 172.21.15.100:0/3601653713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b413a540 con 0x7f24b40716b0 2024-09-17T04:34:05.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.333+0000 7f24bbc00640 1 -- 172.21.15.100:0/3601653713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b413ab00 con 0x7f24b40716b0 2024-09-17T04:34:05.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.334+0000 7f24bbc00640 1 -- 172.21.15.100:0/3601653713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24b414b000 con 0x7f24b40716b0 2024-09-17T04:34:05.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.335+0000 7f24b1600640 1 -- 172.21.15.100:0/3601653713 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f249c005420 con 0x7f24b40716b0 2024-09-17T04:34:05.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.336+0000 7f24b1600640 1 --2- 172.21.15.100:0/3601653713 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f24900795b0 0x7f249007bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.336+0000 7f24b1600640 1 -- 172.21.15.100:0/3601653713 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7f249c099af0 con 0x7f24b40716b0 2024-09-17T04:34:05.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.337+0000 7f24b9200640 1 --2- 172.21.15.100:0/3601653713 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f24900795b0 0x7f249007bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.337+0000 7f24b9200640 1 --2- 172.21.15.100:0/3601653713 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f24900795b0 0x7f249007bac0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009b30 tx=0x7f24a4005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:05.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.341+0000 7f24b1600640 1 -- 172.21.15.100:0/3601653713 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f249c020020 con 0x7f24b40716b0 2024-09-17T04:34:05.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.550+0000 7f24bbc00640 1 -- 172.21.15.100:0/3601653713 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol98520", "group_name": "subvol_grp53780", "target": ["mon-mgr", ""]}) -- 0x7f24b40b5800 con 0x7f24900795b0 2024-09-17T04:34:05.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.554+0000 7f24b1600640 1 -- 172.21.15.100:0/3601653713 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f24b40b5800 con 0x7f24900795b0 2024-09-17T04:34:05.554 INFO:teuthology.orchestra.run.smithi100.stdout:/volumes/subvol_grp53780/subvol98520 2024-09-17T04:34:05.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.557+0000 7f24bbc00640 1 -- 172.21.15.100:0/3601653713 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f24900795b0 msgr2=0x7f249007bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:05.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.557+0000 7f24bbc00640 1 --2- 172.21.15.100:0/3601653713 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f24900795b0 0x7f249007bac0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009b30 tx=0x7f24a4005f90 comp rx=0 tx=0).stop 2024-09-17T04:34:05.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.557+0000 7f24bbc00640 1 -- 172.21.15.100:0/3601653713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24b40716b0 msgr2=0x7f24b414d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:05.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.557+0000 7f24bbc00640 1 --2- 172.21.15.100:0/3601653713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24b40716b0 0x7f24b414d1c0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f249c00d650 tx=0x7f249c00db10 comp rx=0 tx=0).stop 2024-09-17T04:34:05.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.557+0000 7f24bbc00640 1 -- 172.21.15.100:0/3601653713 shutdown_connections 2024-09-17T04:34:05.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.557+0000 7f24bbc00640 1 --2- 172.21.15.100:0/3601653713 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f24900795b0 0x7f249007bac0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.557+0000 7f24bbc00640 1 --2- 172.21.15.100:0/3601653713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24b4152990 0x7f24b414d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.557+0000 7f24bbc00640 1 --2- 172.21.15.100:0/3601653713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24b40716b0 0x7f24b414d1c0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.557+0000 7f24bbc00640 1 --2- 172.21.15.100:0/3601653713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24b40712a0 0x7f24b414cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.557+0000 7f24bbc00640 1 -- 172.21.15.100:0/3601653713 >> 172.21.15.100:0/3601653713 conn(0x7f24b4073890 msgr2=0x7f24b40b5a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:05.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.557+0000 7f24bbc00640 1 -- 172.21.15.100:0/3601653713 shutdown_connections 2024-09-17T04:34:05.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.557+0000 7f24bbc00640 1 -- 172.21.15.100:0/3601653713 wait complete. 2024-09-17T04:34:05.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:05.579 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/f1c557792efa359341e5735fb961e74d.meta) 2024-09-17T04:34:05.639 INFO:teuthology.orchestra.run.smithi100.stdout:[GLOBAL] 2024-09-17T04:34:05.640 INFO:teuthology.orchestra.run.smithi100.stdout:version = 1 2024-09-17T04:34:05.640 INFO:teuthology.orchestra.run.smithi100.stdout:type = subvolume 2024-09-17T04:34:05.640 INFO:teuthology.orchestra.run.smithi100.stdout:path = /volumes/subvol_grp53780/subvol98520 2024-09-17T04:34:05.640 INFO:teuthology.orchestra.run.smithi100.stdout:state = complete 2024-09-17T04:34:05.640 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:34:05.642 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol98520 subvol_grp53780 2024-09-17T04:34:05.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.870+0000 7fef00000640 1 -- 172.21.15.100:0/1578998852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feee0003a30 msgr2=0x7feee0003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:05.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.870+0000 7fef00000640 1 --2- 172.21.15.100:0/1578998852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feee0003a30 0x7feee0003ea0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7feeec00c630 tx=0x7feee0005830 comp rx=0 tx=0).stop 2024-09-17T04:34:05.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.870+0000 7feefe000640 1 -- 172.21.15.100:0/1578998852 reap_dead start 2024-09-17T04:34:05.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.870+0000 7fef00000640 1 -- 172.21.15.100:0/1578998852 shutdown_connections 2024-09-17T04:34:05.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.870+0000 7fef00000640 1 -- 172.21.15.100:0/1578998852 >> 172.21.15.100:0/1578998852 conn(0x7feef80b2540 msgr2=0x7feef80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:05.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.870+0000 7fef00000640 1 -- 172.21.15.100:0/1578998852 shutdown_connections 2024-09-17T04:34:05.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.870+0000 7fef00000640 1 -- 172.21.15.100:0/1578998852 wait complete. 2024-09-17T04:34:05.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.872+0000 7fef00000640 1 Processor -- start 2024-09-17T04:34:05.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.872+0000 7fef00000640 1 -- start start 2024-09-17T04:34:05.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.874+0000 7fef00000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feee0003a30 0x7feef81f9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.874+0000 7fef00000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feef8099010 0x7feef809b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.874+0000 7fef00000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feef809b860 0x7feef809d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.874+0000 7fef00000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7feef80b32a0 con 0x7feee0003a30 2024-09-17T04:34:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.874+0000 7fef00000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feef80b3120 con 0x7feef8099010 2024-09-17T04:34:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.874+0000 7fef00000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7feef80b3420 con 0x7feef809b860 2024-09-17T04:34:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.874+0000 7feefd600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feee0003a30 0x7feef81f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.874+0000 7feefd600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feee0003a30 0x7feef81f9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43272/0 (socket says 172.21.15.100:43272) 2024-09-17T04:34:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.874+0000 7feefd600640 1 -- 172.21.15.100:0/3243988138 learned_addr learned my addr 172.21.15.100:0/3243988138 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.874+0000 7feefe000640 1 --2- 172.21.15.100:0/3243988138 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feef809b860 0x7feef809d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.874+0000 7feefcc00640 1 --2- 172.21.15.100:0/3243988138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feef8099010 0x7feef809b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.874+0000 7feefd600640 1 -- 172.21.15.100:0/3243988138 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feef809b860 msgr2=0x7feef809d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.874+0000 7feefd600640 1 --2- 172.21.15.100:0/3243988138 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feef809b860 0x7feef809d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.874+0000 7feefd600640 1 -- 172.21.15.100:0/3243988138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feef8099010 msgr2=0x7feef809b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:05.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.874+0000 7feefd600640 1 --2- 172.21.15.100:0/3243988138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feef8099010 0x7feef809b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.874+0000 7feefd600640 1 -- 172.21.15.100:0/3243988138 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef809bc70 con 0x7feee0003a30 2024-09-17T04:34:05.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.875+0000 7feefd600640 1 --2- 172.21.15.100:0/3243988138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feee0003a30 0x7feef81f9080 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7feeec02b4e0 tx=0x7feeec004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:05.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.875+0000 7feef5600640 1 -- 172.21.15.100:0/3243988138 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec042070 con 0x7feee0003a30 2024-09-17T04:34:05.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.875+0000 7feef5600640 1 -- 172.21.15.100:0/3243988138 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feeec02bcf0 con 0x7feee0003a30 2024-09-17T04:34:05.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.875+0000 7fef00000640 1 -- 172.21.15.100:0/3243988138 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef80c8920 con 0x7feee0003a30 2024-09-17T04:34:05.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.875+0000 7feef5600640 1 -- 172.21.15.100:0/3243988138 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec039070 con 0x7feee0003a30 2024-09-17T04:34:05.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.875+0000 7fef00000640 1 -- 172.21.15.100:0/3243988138 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef80c8ea0 con 0x7feee0003a30 2024-09-17T04:34:05.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.876+0000 7fef00000640 1 -- 172.21.15.100:0/3243988138 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feef80d1030 con 0x7feee0003a30 2024-09-17T04:34:05.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.879+0000 7feef5600640 1 -- 172.21.15.100:0/3243988138 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7feeec0062b0 con 0x7feee0003a30 2024-09-17T04:34:05.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.879+0000 7feef5600640 1 --2- 172.21.15.100:0/3243988138 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7feed80795e0 0x7feed807baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.880+0000 7feef5600640 1 -- 172.21.15.100:0/3243988138 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7feeec0b9c80 con 0x7feee0003a30 2024-09-17T04:34:05.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.880+0000 7feefcc00640 1 --2- 172.21.15.100:0/3243988138 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7feed80795e0 0x7feed807baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.880+0000 7feefcc00640 1 --2- 172.21.15.100:0/3243988138 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7feed80795e0 0x7feed807baf0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7feef809e350 tx=0x7feee4006b70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:05.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:05.882+0000 7feef5600640 1 -- 172.21.15.100:0/3243988138 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7feeec004e20 con 0x7feee0003a30 2024-09-17T04:34:06.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.094+0000 7fef00000640 1 -- 172.21.15.100:0/3243988138 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol98520", "group_name": "subvol_grp53780", "target": ["mon-mgr", ""]}) -- 0x7feef80702a0 con 0x7feed80795e0 2024-09-17T04:34:06.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.125+0000 7feef5600640 1 -- 172.21.15.100:0/3243988138 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7feef80702a0 con 0x7feed80795e0 2024-09-17T04:34:06.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.128+0000 7fef00000640 1 -- 172.21.15.100:0/3243988138 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7feed80795e0 msgr2=0x7feed807baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:06.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.128+0000 7fef00000640 1 --2- 172.21.15.100:0/3243988138 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7feed80795e0 0x7feed807baf0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7feef809e350 tx=0x7feee4006b70 comp rx=0 tx=0).stop 2024-09-17T04:34:06.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.128+0000 7fef00000640 1 -- 172.21.15.100:0/3243988138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feee0003a30 msgr2=0x7feef81f9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:06.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.128+0000 7fef00000640 1 --2- 172.21.15.100:0/3243988138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feee0003a30 0x7feef81f9080 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7feeec02b4e0 tx=0x7feeec004c30 comp rx=0 tx=0).stop 2024-09-17T04:34:06.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.128+0000 7fef00000640 1 -- 172.21.15.100:0/3243988138 shutdown_connections 2024-09-17T04:34:06.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.128+0000 7fef00000640 1 --2- 172.21.15.100:0/3243988138 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7feed80795e0 0x7feed807baf0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:06.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.128+0000 7fef00000640 1 --2- 172.21.15.100:0/3243988138 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feef809b860 0x7feef809d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:06.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.128+0000 7fef00000640 1 --2- 172.21.15.100:0/3243988138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feef8099010 0x7feef809b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:06.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.128+0000 7fef00000640 1 --2- 172.21.15.100:0/3243988138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feee0003a30 0x7feef81f9080 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:06.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.128+0000 7fef00000640 1 -- 172.21.15.100:0/3243988138 >> 172.21.15.100:0/3243988138 conn(0x7feef80b2540 msgr2=0x7feef806f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:06.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.128+0000 7fef00000640 1 -- 172.21.15.100:0/3243988138 shutdown_connections 2024-09-17T04:34:06.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.128+0000 7fef00000640 1 -- 172.21.15.100:0/3243988138 wait complete. 2024-09-17T04:34:06.131 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:34:06.132+0000 7f30b6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:34:06.132 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:34:06.132+0000 7f30b6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:34:06.132 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:34:06.132+0000 7f30b6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:34:06.132 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:34:06.132+0000 7f30b6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:34:06.132 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:34:06.132+0000 7f30b6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:34:06.151 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-17T04:34:06.152 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:34:06.153+0000 7f30b6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:34:06.152 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:34:06.153+0000 7f30b6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:34:06.153 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:34:06.153+0000 7f30b6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:34:06.153 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:34:06.153+0000 7f30b6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:34:06.153 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-17T04:34:06.153+0000 7f30b6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T04:34:06.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T04:34:06.239 INFO:teuthology.orchestra.run.smithi100.stdout:2 2024-09-17T04:34:06.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:34:06.248 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp53780 2024-09-17T04:34:06.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.472+0000 7fada4800640 1 -- 172.21.15.100:0/345168289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad9c0aee00 msgr2=0x7fad9c0af210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:06.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.472+0000 7fada4800640 1 --2- 172.21.15.100:0/345168289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad9c0aee00 0x7fad9c0af210 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fad8c005fd0 tx=0x7fad8c02a4b0 comp rx=0 tx=0).stop 2024-09-17T04:34:06.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.472+0000 7fada4800640 1 -- 172.21.15.100:0/345168289 shutdown_connections 2024-09-17T04:34:06.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.472+0000 7fada4800640 1 --2- 172.21.15.100:0/345168289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad9c0aee00 0x7fad9c0af210 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:06.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.472+0000 7fada4800640 1 --2- 172.21.15.100:0/345168289 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad9c15f9e0 0x7fad9c147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:06.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.472+0000 7fada4800640 1 -- 172.21.15.100:0/345168289 >> 172.21.15.100:0/345168289 conn(0x7fad9c06d1c0 msgr2=0x7fad9c06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:06.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.472+0000 7fada4800640 1 -- 172.21.15.100:0/345168289 shutdown_connections 2024-09-17T04:34:06.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.472+0000 7fada4800640 1 -- 172.21.15.100:0/345168289 wait complete. 2024-09-17T04:34:06.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.475+0000 7fada4800640 1 Processor -- start 2024-09-17T04:34:06.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.475+0000 7fada4800640 1 -- start start 2024-09-17T04:34:06.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.476+0000 7fada4800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad9c073540 0x7fad9c0cede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:06.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.476+0000 7fada4800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad9c0aee00 0x7fad9c1f6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.476+0000 7fada4800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad9c15f9e0 0x7fad9c142620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.476+0000 7fada4800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c06fb60 con 0x7fad9c15f9e0 2024-09-17T04:34:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.476+0000 7fada4800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c06f9e0 con 0x7fad9c0aee00 2024-09-17T04:34:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.476+0000 7fada4800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fad9c06fce0 con 0x7fad9c073540 2024-09-17T04:34:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.477+0000 7fada2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad9c15f9e0 0x7fad9c142620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.477+0000 7fada2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad9c15f9e0 0x7fad9c142620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43314/0 (socket says 172.21.15.100:43314) 2024-09-17T04:34:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.477+0000 7fada2800640 1 -- 172.21.15.100:0/3147669909 learned_addr learned my addr 172.21.15.100:0/3147669909 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.477+0000 7fada1e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad9c073540 0x7fad9c0cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.477+0000 7fada1400640 1 --2- 172.21.15.100:0/3147669909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad9c0aee00 0x7fad9c1f6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.477+0000 7fada2800640 1 -- 172.21.15.100:0/3147669909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad9c073540 msgr2=0x7fad9c0cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.477+0000 7fada2800640 1 --2- 172.21.15.100:0/3147669909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad9c073540 0x7fad9c0cede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.477+0000 7fada2800640 1 -- 172.21.15.100:0/3147669909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad9c0aee00 msgr2=0x7fad9c1f6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.477+0000 7fada2800640 1 --2- 172.21.15.100:0/3147669909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad9c0aee00 0x7fad9c1f6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:06.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.477+0000 7fada2800640 1 -- 172.21.15.100:0/3147669909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad9c144600 con 0x7fad9c15f9e0 2024-09-17T04:34:06.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.477+0000 7fada2800640 1 --2- 172.21.15.100:0/3147669909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad9c15f9e0 0x7fad9c142620 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fad9800c730 tx=0x7fad9800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:06.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.478+0000 7fad96000640 1 -- 172.21.15.100:0/3147669909 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98003b70 con 0x7fad9c15f9e0 2024-09-17T04:34:06.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.478+0000 7fad96000640 1 -- 172.21.15.100:0/3147669909 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad98003d10 con 0x7fad9c15f9e0 2024-09-17T04:34:06.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.478+0000 7fada4800640 1 -- 172.21.15.100:0/3147669909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad9c144b30 con 0x7fad9c15f9e0 2024-09-17T04:34:06.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.478+0000 7fad96000640 1 -- 172.21.15.100:0/3147669909 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9800aab0 con 0x7fad9c15f9e0 2024-09-17T04:34:06.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.478+0000 7fada4800640 1 -- 172.21.15.100:0/3147669909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad9c1f6ef0 con 0x7fad9c15f9e0 2024-09-17T04:34:06.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.479+0000 7fada4800640 1 -- 172.21.15.100:0/3147669909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad9c0af210 con 0x7fad9c15f9e0 2024-09-17T04:34:06.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.480+0000 7fad96000640 1 -- 172.21.15.100:0/3147669909 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7fad9800ac50 con 0x7fad9c15f9e0 2024-09-17T04:34:06.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.481+0000 7fad96000640 1 --2- 172.21.15.100:0/3147669909 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fad800795b0 0x7fad8007bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:06.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.481+0000 7fad96000640 1 -- 172.21.15.100:0/3147669909 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7fad980994b0 con 0x7fad9c15f9e0 2024-09-17T04:34:06.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.481+0000 7fada1e00640 1 --2- 172.21.15.100:0/3147669909 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fad800795b0 0x7fad8007bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:06.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.482+0000 7fada1e00640 1 --2- 172.21.15.100:0/3147669909 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fad800795b0 0x7fad8007bac0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0b3240 tx=0x7fad90005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:06.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.486+0000 7fad96000640 1 -- 172.21.15.100:0/3147669909 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fad98062b40 con 0x7fad9c15f9e0 2024-09-17T04:34:06.526 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:34:06.530 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:34:06.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.700+0000 7fada4800640 1 -- 172.21.15.100:0/3147669909 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp53780", "target": ["mon-mgr", ""]}) -- 0x7fad9c1f6760 con 0x7fad800795b0 2024-09-17T04:34:06.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.704+0000 7fad96000640 1 -- 172.21.15.100:0/3147669909 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fad9c1f6760 con 0x7fad800795b0 2024-09-17T04:34:06.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.707+0000 7fada4800640 1 -- 172.21.15.100:0/3147669909 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fad800795b0 msgr2=0x7fad8007bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:06.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.707+0000 7fada4800640 1 --2- 172.21.15.100:0/3147669909 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fad800795b0 0x7fad8007bac0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0b3240 tx=0x7fad90005f90 comp rx=0 tx=0).stop 2024-09-17T04:34:06.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.707+0000 7fada4800640 1 -- 172.21.15.100:0/3147669909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad9c15f9e0 msgr2=0x7fad9c142620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:06.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.707+0000 7fada4800640 1 --2- 172.21.15.100:0/3147669909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad9c15f9e0 0x7fad9c142620 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fad9800c730 tx=0x7fad9800cbf0 comp rx=0 tx=0).stop 2024-09-17T04:34:06.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.707+0000 7fada4800640 1 -- 172.21.15.100:0/3147669909 shutdown_connections 2024-09-17T04:34:06.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.707+0000 7fada4800640 1 --2- 172.21.15.100:0/3147669909 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fad800795b0 0x7fad8007bac0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:06.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.707+0000 7fada4800640 1 --2- 172.21.15.100:0/3147669909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad9c15f9e0 0x7fad9c142620 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:06.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.707+0000 7fada4800640 1 --2- 172.21.15.100:0/3147669909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad9c0aee00 0x7fad9c1f6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:06.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.707+0000 7fada4800640 1 --2- 172.21.15.100:0/3147669909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad9c073540 0x7fad9c0cede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:06.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.707+0000 7fada4800640 1 -- 172.21.15.100:0/3147669909 >> 172.21.15.100:0/3147669909 conn(0x7fad9c06d1c0 msgr2=0x7fad9c0b3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:06.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.707+0000 7fada4800640 1 -- 172.21.15.100:0/3147669909 shutdown_connections 2024-09-17T04:34:06.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:06.707+0000 7fada4800640 1 -- 172.21.15.100:0/3147669909 wait complete. 2024-09-17T04:34:06.732 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T04:34:06.732 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:34:06.817 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:34:06.903 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T04:34:06.903 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:34:06.903 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:34:06.903 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:34:06.939 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:34:06.939 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:34:07.015 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:34:07.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.275+0000 7f4a35600640 1 -- 172.21.15.100:0/2739240167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a3015ce30 msgr2=0x7f4a3015d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:07.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.275+0000 7f4a35600640 1 --2- 172.21.15.100:0/2739240167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a3015ce30 0x7f4a3015d240 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f4a300b4fa0 tx=0x7f4a2802a800 comp rx=0 tx=0).stop 2024-09-17T04:34:07.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.275+0000 7f4a35600640 1 -- 172.21.15.100:0/2739240167 shutdown_connections 2024-09-17T04:34:07.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.275+0000 7f4a35600640 1 --2- 172.21.15.100:0/2739240167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a3015ce30 0x7f4a3015d240 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:07.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.275+0000 7f4a35600640 1 --2- 172.21.15.100:0/2739240167 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a300cfc80 0x7f4a30156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:07.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.275+0000 7f4a35600640 1 -- 172.21.15.100:0/2739240167 >> 172.21.15.100:0/2739240167 conn(0x7f4a3006d3d0 msgr2=0x7f4a300b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:07.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.276+0000 7f4a35600640 1 -- 172.21.15.100:0/2739240167 shutdown_connections 2024-09-17T04:34:07.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.276+0000 7f4a35600640 1 -- 172.21.15.100:0/2739240167 wait complete. 2024-09-17T04:34:07.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.278+0000 7f4a35600640 1 Processor -- start 2024-09-17T04:34:07.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.278+0000 7f4a35600640 1 -- start start 2024-09-17T04:34:07.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.279+0000 7f4a35600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a300705b0 0x7f4a300cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:07.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.279+0000 7f4a35600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a300cfc80 0x7f4a300c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:07.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.279+0000 7f4a35600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a3015ce30 0x7f4a300ca940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:07.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.279+0000 7f4a35600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4a300cd050 con 0x7f4a300705b0 2024-09-17T04:34:07.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.279+0000 7f4a35600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4a300cced0 con 0x7f4a300cfc80 2024-09-17T04:34:07.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.279+0000 7f4a35600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4a300cd1d0 con 0x7f4a3015ce30 2024-09-17T04:34:07.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.279+0000 7f4a2ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a300705b0 0x7f4a300cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:07.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.280+0000 7f4a2ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a300705b0 0x7f4a300cbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43320/0 (socket says 172.21.15.100:43320) 2024-09-17T04:34:07.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.280+0000 7f4a2ea00640 1 -- 172.21.15.100:0/3482213011 learned_addr learned my addr 172.21.15.100:0/3482213011 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:07.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.280+0000 7f4a2f400640 1 --2- 172.21.15.100:0/3482213011 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a3015ce30 0x7f4a300ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.280+0000 7f4a2e000640 1 --2- 172.21.15.100:0/3482213011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a300cfc80 0x7f4a300c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.280+0000 7f4a2ea00640 1 -- 172.21.15.100:0/3482213011 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a3015ce30 msgr2=0x7f4a300ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.280+0000 7f4a2ea00640 1 --2- 172.21.15.100:0/3482213011 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a3015ce30 0x7f4a300ca940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.280+0000 7f4a2ea00640 1 -- 172.21.15.100:0/3482213011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a300cfc80 msgr2=0x7f4a300c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.280+0000 7f4a2ea00640 1 --2- 172.21.15.100:0/3482213011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a300cfc80 0x7f4a300c60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.280+0000 7f4a2ea00640 1 -- 172.21.15.100:0/3482213011 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a300b1f10 con 0x7f4a300705b0 2024-09-17T04:34:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.280+0000 7f4a2f400640 1 --2- 172.21.15.100:0/3482213011 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a3015ce30 0x7f4a300ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.280+0000 7f4a2e000640 1 --2- 172.21.15.100:0/3482213011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a300cfc80 0x7f4a300c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.280+0000 7f4a2ea00640 1 --2- 172.21.15.100:0/3482213011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a300705b0 0x7f4a300cbf00 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f4a2000e790 tx=0x7f4a2000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.280+0000 7f4a26a00640 1 -- 172.21.15.100:0/3482213011 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2000eea0 con 0x7f4a300705b0 2024-09-17T04:34:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.280+0000 7f4a26a00640 1 -- 172.21.15.100:0/3482213011 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a2000ccc0 con 0x7f4a300705b0 2024-09-17T04:34:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.281+0000 7f4a35600640 1 -- 172.21.15.100:0/3482213011 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a300b21f0 con 0x7f4a300705b0 2024-09-17T04:34:07.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.281+0000 7f4a26a00640 1 -- 172.21.15.100:0/3482213011 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a20004280 con 0x7f4a300705b0 2024-09-17T04:34:07.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.281+0000 7f4a35600640 1 -- 172.21.15.100:0/3482213011 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a300c73a0 con 0x7f4a300705b0 2024-09-17T04:34:07.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.282+0000 7f4a35600640 1 -- 172.21.15.100:0/3482213011 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a300b4420 con 0x7f4a300705b0 2024-09-17T04:34:07.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.283+0000 7f4a26a00640 1 -- 172.21.15.100:0/3482213011 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f4a200035f0 con 0x7f4a300705b0 2024-09-17T04:34:07.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.284+0000 7f4a26a00640 1 --2- 172.21.15.100:0/3482213011 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4a100795b0 0x7f4a1007bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:07.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.284+0000 7f4a26a00640 1 -- 172.21.15.100:0/3482213011 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7f4a2009a2f0 con 0x7f4a300705b0 2024-09-17T04:34:07.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.284+0000 7f4a2e000640 1 --2- 172.21.15.100:0/3482213011 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4a100795b0 0x7f4a1007bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:07.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.285+0000 7f4a2e000640 1 --2- 172.21.15.100:0/3482213011 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4a100795b0 0x7f4a1007bac0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4a3015d670 tx=0x7f4a1c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:07.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.288+0000 7f4a26a00640 1 -- 172.21.15.100:0/3482213011 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4a300b4420 con 0x7f4a300705b0 2024-09-17T04:34:07.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.484+0000 7f4a35600640 1 -- 172.21.15.100:0/3482213011 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4a300cd6f0 con 0x7f4a300705b0 2024-09-17T04:34:07.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.485+0000 7f4a26a00640 1 -- 172.21.15.100:0/3482213011 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v167) ==== 80+0+0 (secure 0 0 0) 0x7f4a20063ee0 con 0x7f4a300705b0 2024-09-17T04:34:07.484 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-09-17T04:34:07.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.487+0000 7f4a35600640 1 -- 172.21.15.100:0/3482213011 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4a100795b0 msgr2=0x7f4a1007bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:07.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.487+0000 7f4a35600640 1 --2- 172.21.15.100:0/3482213011 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4a100795b0 0x7f4a1007bac0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4a3015d670 tx=0x7f4a1c009040 comp rx=0 tx=0).stop 2024-09-17T04:34:07.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.487+0000 7f4a35600640 1 -- 172.21.15.100:0/3482213011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a300705b0 msgr2=0x7f4a300cbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:07.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.487+0000 7f4a35600640 1 --2- 172.21.15.100:0/3482213011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a300705b0 0x7f4a300cbf00 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f4a2000e790 tx=0x7f4a2000ec50 comp rx=0 tx=0).stop 2024-09-17T04:34:07.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.487+0000 7f4a35600640 1 -- 172.21.15.100:0/3482213011 shutdown_connections 2024-09-17T04:34:07.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.487+0000 7f4a35600640 1 --2- 172.21.15.100:0/3482213011 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4a100795b0 0x7f4a1007bac0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:07.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.487+0000 7f4a35600640 1 --2- 172.21.15.100:0/3482213011 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a3015ce30 0x7f4a300ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:07.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.487+0000 7f4a35600640 1 --2- 172.21.15.100:0/3482213011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a300cfc80 0x7f4a300c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:07.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.487+0000 7f4a35600640 1 --2- 172.21.15.100:0/3482213011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a300705b0 0x7f4a300cbf00 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:07.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.487+0000 7f4a35600640 1 -- 172.21.15.100:0/3482213011 >> 172.21.15.100:0/3482213011 conn(0x7f4a3006d3d0 msgr2=0x7f4a3015ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:07.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.487+0000 7f4a35600640 1 -- 172.21.15.100:0/3482213011 shutdown_connections 2024-09-17T04:34:07.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.487+0000 7f4a35600640 1 -- 172.21.15.100:0/3482213011 wait complete. 2024-09-17T04:34:07.512 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T04:34:07.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:07.512 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T04:34:07.602 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-09-17T04:34:07.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:07.602 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T04:34:07.657 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:34:07.657 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:34:07.657 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:34:07.695 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:34:07.695 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:34:07.772 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:34:07.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.998+0000 7fc237200640 1 -- 172.21.15.100:0/3710677212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc23815c510 msgr2=0x7fc238160ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:07.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.998+0000 7fc237200640 1 --2- 172.21.15.100:0/3710677212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc23815c510 0x7fc238160ac0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc228009830 tx=0x7fc22802a530 comp rx=0 tx=0).stop 2024-09-17T04:34:07.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.998+0000 7fc237200640 1 -- 172.21.15.100:0/3710677212 shutdown_connections 2024-09-17T04:34:07.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.998+0000 7fc237200640 1 --2- 172.21.15.100:0/3710677212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc23815c510 0x7fc238160ac0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:07.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.998+0000 7fc237200640 1 -- 172.21.15.100:0/3710677212 >> 172.21.15.100:0/3710677212 conn(0x7fc2380d1470 msgr2=0x7fc238073670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:07.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.998+0000 7fc237200640 1 -- 172.21.15.100:0/3710677212 shutdown_connections 2024-09-17T04:34:07.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:07.998+0000 7fc237200640 1 -- 172.21.15.100:0/3710677212 wait complete. 2024-09-17T04:34:07.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.000+0000 7fc237200640 1 Processor -- start 2024-09-17T04:34:07.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.000+0000 7fc237200640 1 -- start start 2024-09-17T04:34:08.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.002+0000 7fc237200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2380cdcf0 0x7fc23815f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:08.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.002+0000 7fc237200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc23815c510 0x7fc238159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:08.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.002+0000 7fc237200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc23815a0d0 0x7fc23815a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:08.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.002+0000 7fc237200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc2380ce280 con 0x7fc2380cdcf0 2024-09-17T04:34:08.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.002+0000 7fc237200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc2380ce100 con 0x7fc23815c510 2024-09-17T04:34:08.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.002+0000 7fc237200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc2380b3f80 con 0x7fc23815a0d0 2024-09-17T04:34:08.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.002+0000 7fc235e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2380cdcf0 0x7fc23815f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:08.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.002+0000 7fc235e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2380cdcf0 0x7fc23815f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43332/0 (socket says 172.21.15.100:43332) 2024-09-17T04:34:08.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.002+0000 7fc235e00640 1 -- 172.21.15.100:0/2770651273 learned_addr learned my addr 172.21.15.100:0/2770651273 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:08.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.002+0000 7fc235400640 1 --2- 172.21.15.100:0/2770651273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc23815c510 0x7fc238159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:08.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.002+0000 7fc236800640 1 --2- 172.21.15.100:0/2770651273 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc23815a0d0 0x7fc23815a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:08.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.002+0000 7fc235e00640 1 -- 172.21.15.100:0/2770651273 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc23815a0d0 msgr2=0x7fc23815a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:08.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.003+0000 7fc235e00640 1 --2- 172.21.15.100:0/2770651273 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc23815a0d0 0x7fc23815a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.003+0000 7fc235e00640 1 -- 172.21.15.100:0/2770651273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc23815c510 msgr2=0x7fc238159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:08.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.003+0000 7fc235e00640 1 --2- 172.21.15.100:0/2770651273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc23815c510 0x7fc238159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.003+0000 7fc235e00640 1 -- 172.21.15.100:0/2770651273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23815fa80 con 0x7fc2380cdcf0 2024-09-17T04:34:08.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.003+0000 7fc235400640 1 --2- 172.21.15.100:0/2770651273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc23815c510 0x7fc238159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:08.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.003+0000 7fc236800640 1 --2- 172.21.15.100:0/2770651273 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc23815a0d0 0x7fc23815a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:08.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.003+0000 7fc235e00640 1 --2- 172.21.15.100:0/2770651273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2380cdcf0 0x7fc23815f540 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fc228009950 tx=0x7fc228033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:08.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.003+0000 7fc22e000640 1 -- 172.21.15.100:0/2770651273 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc228031b50 con 0x7fc2380cdcf0 2024-09-17T04:34:08.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.003+0000 7fc22e000640 1 -- 172.21.15.100:0/2770651273 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc228031de0 con 0x7fc2380cdcf0 2024-09-17T04:34:08.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.003+0000 7fc237200640 1 -- 172.21.15.100:0/2770651273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc23814ea20 con 0x7fc2380cdcf0 2024-09-17T04:34:08.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.003+0000 7fc22e000640 1 -- 172.21.15.100:0/2770651273 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc228005280 con 0x7fc2380cdcf0 2024-09-17T04:34:08.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.003+0000 7fc237200640 1 -- 172.21.15.100:0/2770651273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2381510f0 con 0x7fc2380cdcf0 2024-09-17T04:34:08.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.004+0000 7fc237200640 1 -- 172.21.15.100:0/2770651273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc238160ac0 con 0x7fc2380cdcf0 2024-09-17T04:34:08.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.006+0000 7fc22e000640 1 -- 172.21.15.100:0/2770651273 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7fc228005420 con 0x7fc2380cdcf0 2024-09-17T04:34:08.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.007+0000 7fc22e000640 1 --2- 172.21.15.100:0/2770651273 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc218079790 0x7fc21807bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:08.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.007+0000 7fc22e000640 1 -- 172.21.15.100:0/2770651273 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7fc2280ba980 con 0x7fc2380cdcf0 2024-09-17T04:34:08.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.007+0000 7fc235400640 1 --2- 172.21.15.100:0/2770651273 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc218079790 0x7fc21807bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:08.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.008+0000 7fc235400640 1 --2- 172.21.15.100:0/2770651273 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc218079790 0x7fc21807bca0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc2381f8650 tx=0x7fc224009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:08.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.012+0000 7fc22e000640 1 -- 172.21.15.100:0/2770651273 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc22803f030 con 0x7fc2380cdcf0 2024-09-17T04:34:08.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.213+0000 7fc237200640 1 -- 172.21.15.100:0/2770651273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc2380b45d0 con 0x7fc2380cdcf0 2024-09-17T04:34:08.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.213+0000 7fc22e000640 1 -- 172.21.15.100:0/2770651273 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v167) ==== 80+0+0 (secure 0 0 0) 0x7fc228084640 con 0x7fc2380cdcf0 2024-09-17T04:34:08.213 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-09-17T04:34:08.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.216+0000 7fc237200640 1 -- 172.21.15.100:0/2770651273 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc218079790 msgr2=0x7fc21807bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:08.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.216+0000 7fc237200640 1 --2- 172.21.15.100:0/2770651273 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc218079790 0x7fc21807bca0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc2381f8650 tx=0x7fc224009040 comp rx=0 tx=0).stop 2024-09-17T04:34:08.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.216+0000 7fc237200640 1 -- 172.21.15.100:0/2770651273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2380cdcf0 msgr2=0x7fc23815f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:08.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.216+0000 7fc237200640 1 --2- 172.21.15.100:0/2770651273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2380cdcf0 0x7fc23815f540 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fc228009950 tx=0x7fc228033ba0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.216+0000 7fc237200640 1 -- 172.21.15.100:0/2770651273 shutdown_connections 2024-09-17T04:34:08.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.216+0000 7fc237200640 1 --2- 172.21.15.100:0/2770651273 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc218079790 0x7fc21807bca0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.216+0000 7fc237200640 1 --2- 172.21.15.100:0/2770651273 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc23815a0d0 0x7fc23815a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.216+0000 7fc237200640 1 --2- 172.21.15.100:0/2770651273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc23815c510 0x7fc238159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.216+0000 7fc237200640 1 --2- 172.21.15.100:0/2770651273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2380cdcf0 0x7fc23815f540 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.216+0000 7fc237200640 1 -- 172.21.15.100:0/2770651273 >> 172.21.15.100:0/2770651273 conn(0x7fc2380d1470 msgr2=0x7fc2380b3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:08.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.216+0000 7fc237200640 1 -- 172.21.15.100:0/2770651273 shutdown_connections 2024-09-17T04:34:08.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.216+0000 7fc237200640 1 -- 172.21.15.100:0/2770651273 wait complete. 2024-09-17T04:34:08.237 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-17T04:34:08.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:08.237 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-17T04:34:08.302 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2024-09-17T04:34:08.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:08.302 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T04:34:08.338 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:34:08.338 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:34:08.338 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:34:08.375 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:34:08.376 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:34:08.376 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:34:08.376 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:34:08.414 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:34:08.414 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:34:08.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.635+0000 7f2dd2a00640 1 -- 172.21.15.100:0/816992383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dcc0aebc0 msgr2=0x7f2dcc152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:08.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.635+0000 7f2dd2a00640 1 --2- 172.21.15.100:0/816992383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dcc0aebc0 0x7f2dcc152920 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc06f3c0 tx=0x7f2dbc02a820 comp rx=0 tx=0).stop 2024-09-17T04:34:08.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.635+0000 7f2dd2a00640 1 -- 172.21.15.100:0/816992383 shutdown_connections 2024-09-17T04:34:08.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.635+0000 7f2dd2a00640 1 --2- 172.21.15.100:0/816992383 >> v2:172.21.15.184:3300/0 conn(0x7f2dcc0b2750 0x7f2dcc15f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.635+0000 7f2dd2a00640 1 --2- 172.21.15.100:0/816992383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dcc0aebc0 0x7f2dcc152920 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.635+0000 7f2dd2a00640 1 -- 172.21.15.100:0/816992383 >> 172.21.15.100:0/816992383 conn(0x7f2dcc06d730 msgr2=0x7f2dcc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:08.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.635+0000 7f2dd2a00640 1 -- 172.21.15.100:0/816992383 shutdown_connections 2024-09-17T04:34:08.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.635+0000 7f2dd2a00640 1 -- 172.21.15.100:0/816992383 wait complete. 2024-09-17T04:34:08.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.637+0000 7f2dd2a00640 1 Processor -- start 2024-09-17T04:34:08.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.638+0000 7f2dd2a00640 1 -- start start 2024-09-17T04:34:08.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.639+0000 7f2dd2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dcc0aebc0 0x7f2dcc09a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:08.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.639+0000 7f2dd2a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2dcc0b2750 0x7f2dcc14a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:08.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.639+0000 7f2dd2a00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2dcc15f910 0x7f2dcc155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:08.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.639+0000 7f2dd2a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2dcc15c4a0 con 0x7f2dcc0aebc0 2024-09-17T04:34:08.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.639+0000 7f2dd2a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2dcc0b4970 con 0x7f2dcc0b2750 2024-09-17T04:34:08.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.639+0000 7f2dd2a00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2dcc15c620 con 0x7f2dcc15f910 2024-09-17T04:34:08.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.639+0000 7f2dcbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dcc0aebc0 0x7f2dcc09a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:08.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.639+0000 7f2dcbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dcc0aebc0 0x7f2dcc09a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43350/0 (socket says 172.21.15.100:43350) 2024-09-17T04:34:08.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.639+0000 7f2dcbe00640 1 -- 172.21.15.100:0/2119899413 learned_addr learned my addr 172.21.15.100:0/2119899413 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:08.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.640+0000 7f2dd0a00640 1 --2- 172.21.15.100:0/2119899413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2dcc15f910 0x7f2dcc155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:08.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.640+0000 7f2dcb400640 1 --2- 172.21.15.100:0/2119899413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2dcc0b2750 0x7f2dcc14a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:08.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.640+0000 7f2dcbe00640 1 -- 172.21.15.100:0/2119899413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2dcc15f910 msgr2=0x7f2dcc155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:08.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.640+0000 7f2dcbe00640 1 --2- 172.21.15.100:0/2119899413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2dcc15f910 0x7f2dcc155150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.640+0000 7f2dcbe00640 1 -- 172.21.15.100:0/2119899413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2dcc0b2750 msgr2=0x7f2dcc14a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:08.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.640+0000 7f2dcbe00640 1 --2- 172.21.15.100:0/2119899413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2dcc0b2750 0x7f2dcc14a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.640+0000 7f2dcbe00640 1 -- 172.21.15.100:0/2119899413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dcc14adc0 con 0x7f2dcc0aebc0 2024-09-17T04:34:08.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.640+0000 7f2dd0a00640 1 --2- 172.21.15.100:0/2119899413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2dcc15f910 0x7f2dcc155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:08.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.640+0000 7f2dcb400640 1 --2- 172.21.15.100:0/2119899413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2dcc0b2750 0x7f2dcc14a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:08.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.640+0000 7f2dcbe00640 1 --2- 172.21.15.100:0/2119899413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dcc0aebc0 0x7f2dcc09a2d0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc02ae10 tx=0x7f2dbc032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:08.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.640+0000 7f2dc3e00640 1 -- 172.21.15.100:0/2119899413 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc037040 con 0x7f2dcc0aebc0 2024-09-17T04:34:08.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.640+0000 7f2dc3e00640 1 -- 172.21.15.100:0/2119899413 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2dbc042070 con 0x7f2dcc0aebc0 2024-09-17T04:34:08.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.640+0000 7f2dd2a00640 1 -- 172.21.15.100:0/2119899413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dcc0b41d0 con 0x7f2dcc0aebc0 2024-09-17T04:34:08.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.640+0000 7f2dc3e00640 1 -- 172.21.15.100:0/2119899413 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc003e60 con 0x7f2dcc0aebc0 2024-09-17T04:34:08.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.640+0000 7f2dd2a00640 1 -- 172.21.15.100:0/2119899413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dcc154390 con 0x7f2dcc0aebc0 2024-09-17T04:34:08.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.641+0000 7f2dd2a00640 1 -- 172.21.15.100:0/2119899413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dcc15cbd0 con 0x7f2dcc0aebc0 2024-09-17T04:34:08.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.643+0000 7f2dc3e00640 1 -- 172.21.15.100:0/2119899413 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7f2dbc0042a0 con 0x7f2dcc0aebc0 2024-09-17T04:34:08.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.644+0000 7f2dc3e00640 1 --2- 172.21.15.100:0/2119899413 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2dac079790 0x7f2dac07bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:08.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.644+0000 7f2dc3e00640 1 -- 172.21.15.100:0/2119899413 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7f2dbc0ba7a0 con 0x7f2dcc0aebc0 2024-09-17T04:34:08.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.644+0000 7f2dcb400640 1 --2- 172.21.15.100:0/2119899413 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2dac079790 0x7f2dac07bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:08.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.645+0000 7f2dcb400640 1 --2- 172.21.15.100:0/2119899413 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2dac079790 0x7f2dac07bca0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2db8005fd0 tx=0x7f2db8005d40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:08.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.648+0000 7f2dc3e00640 1 -- 172.21.15.100:0/2119899413 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f2dbc084460 con 0x7f2dcc0aebc0 2024-09-17T04:34:08.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.868+0000 7f2dd2a00640 1 -- 172.21.15.100:0/2119899413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2dcc0b4da0 con 0x7f2dcc0aebc0 2024-09-17T04:34:08.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.869+0000 7f2dc3e00640 1 -- 172.21.15.100:0/2119899413 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) ==== 95+0+5242 (secure 0 0 0) 0x7f2dbc083d60 con 0x7f2dcc0aebc0 2024-09-17T04:34:08.868 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:34:08.869 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":122,"btime":"2024-09-17T04:34:03:329513+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/613141426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":613141426},{"type":"v1","addr":"172.21.15.184:6835","nonce":613141426}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":8554,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/1797529407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6801","nonce":1797529407},{"type":"v1","addr":"172.21.15.184:6834","nonce":1797529407}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":8560,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2263752731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2263752731},{"type":"v1","addr":"172.21.15.100:6833","nonce":2263752731}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:52.555851+0000","modified":"2024-09-17T04:33:55.567693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9153},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9153":{"gid":9153,"name":"c","rank":0,"incarnation":117,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/1207785711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1207785711},{"type":"v1","addr":"172.21.15.100:6835","nonce":1207785711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9153,"qdb_cluster":[9153]},"id":13}]} 2024-09-17T04:34:08.869 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 122 2024-09-17T04:34:08.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.871+0000 7f2dd2a00640 1 -- 172.21.15.100:0/2119899413 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2dac079790 msgr2=0x7f2dac07bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:08.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.871+0000 7f2dd2a00640 1 --2- 172.21.15.100:0/2119899413 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2dac079790 0x7f2dac07bca0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2db8005fd0 tx=0x7f2db8005d40 comp rx=0 tx=0).stop 2024-09-17T04:34:08.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.871+0000 7f2dd2a00640 1 -- 172.21.15.100:0/2119899413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dcc0aebc0 msgr2=0x7f2dcc09a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:08.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.871+0000 7f2dd2a00640 1 --2- 172.21.15.100:0/2119899413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dcc0aebc0 0x7f2dcc09a2d0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc02ae10 tx=0x7f2dbc032d10 comp rx=0 tx=0).stop 2024-09-17T04:34:08.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.871+0000 7f2dd2a00640 1 -- 172.21.15.100:0/2119899413 shutdown_connections 2024-09-17T04:34:08.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.871+0000 7f2dd2a00640 1 --2- 172.21.15.100:0/2119899413 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2dac079790 0x7f2dac07bca0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.871+0000 7f2dd2a00640 1 --2- 172.21.15.100:0/2119899413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2dcc15f910 0x7f2dcc155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.871+0000 7f2dd2a00640 1 --2- 172.21.15.100:0/2119899413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2dcc0b2750 0x7f2dcc14a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.871+0000 7f2dd2a00640 1 --2- 172.21.15.100:0/2119899413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dcc0aebc0 0x7f2dcc09a2d0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.871+0000 7f2dd2a00640 1 -- 172.21.15.100:0/2119899413 >> 172.21.15.100:0/2119899413 conn(0x7f2dcc06d730 msgr2=0x7f2dcc15d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:08.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.872+0000 7f2dd2a00640 1 -- 172.21.15.100:0/2119899413 shutdown_connections 2024-09-17T04:34:08.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:08.872+0000 7f2dd2a00640 1 -- 172.21.15.100:0/2119899413 wait complete. 2024-09-17T04:34:08.893 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:34:09.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.120+0000 7ffaada00640 1 -- 172.21.15.100:0/3370133559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa8c004800 msgr2=0x7ffa8c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:09.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.120+0000 7ffaada00640 1 --2- 172.21.15.100:0/3370133559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa8c004800 0x7ffa8c004c10 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7ffa94009b20 tx=0x7ffa8c005830 comp rx=0 tx=0).stop 2024-09-17T04:34:09.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.120+0000 7ffaada00640 1 -- 172.21.15.100:0/3370133559 shutdown_connections 2024-09-17T04:34:09.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.120+0000 7ffaada00640 1 --2- 172.21.15.100:0/3370133559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa8c004800 0x7ffa8c004c10 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:09.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.120+0000 7ffaada00640 1 -- 172.21.15.100:0/3370133559 >> 172.21.15.100:0/3370133559 conn(0x7ffaa806d730 msgr2=0x7ffaa80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:09.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.120+0000 7ffaada00640 1 -- 172.21.15.100:0/3370133559 shutdown_connections 2024-09-17T04:34:09.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.120+0000 7ffaada00640 1 -- 172.21.15.100:0/3370133559 wait complete. 2024-09-17T04:34:09.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.122+0000 7ffaada00640 1 Processor -- start 2024-09-17T04:34:09.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.123+0000 7ffaada00640 1 -- start start 2024-09-17T04:34:09.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.124+0000 7ffaada00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa8c004800 0x7ffaa8155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:09.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.124+0000 7ffaada00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaa814c440 0x7ffaa81f7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:09.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.124+0000 7ffaada00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffaa81f7bd0 0x7ffaa814a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:09.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.124+0000 7ffaada00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa806f260 con 0x7ffaa81f7bd0 2024-09-17T04:34:09.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.124+0000 7ffaada00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa80b4e30 con 0x7ffaa814c440 2024-09-17T04:34:09.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.124+0000 7ffaada00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ffaa80ad4f0 con 0x7ffa8c004800 2024-09-17T04:34:09.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.124+0000 7ffaa7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffaa81f7bd0 0x7ffaa814a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:09.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.124+0000 7ffaa7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffaa81f7bd0 0x7ffaa814a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43368/0 (socket says 172.21.15.100:43368) 2024-09-17T04:34:09.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.124+0000 7ffaa7400640 1 -- 172.21.15.100:0/284403728 learned_addr learned my addr 172.21.15.100:0/284403728 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:09.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.124+0000 7ffaa6a00640 1 --2- 172.21.15.100:0/284403728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa8c004800 0x7ffaa8155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:09.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.124+0000 7ffaa6000640 1 --2- 172.21.15.100:0/284403728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaa814c440 0x7ffaa81f7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:09.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.125+0000 7ffaa7400640 1 -- 172.21.15.100:0/284403728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa8c004800 msgr2=0x7ffaa8155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:09.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.125+0000 7ffaa7400640 1 --2- 172.21.15.100:0/284403728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa8c004800 0x7ffaa8155a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:09.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.125+0000 7ffaa7400640 1 -- 172.21.15.100:0/284403728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaa814c440 msgr2=0x7ffaa81f7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:09.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.125+0000 7ffaa7400640 1 --2- 172.21.15.100:0/284403728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaa814c440 0x7ffaa81f7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:09.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.125+0000 7ffaa7400640 1 -- 172.21.15.100:0/284403728 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa8156250 con 0x7ffaa81f7bd0 2024-09-17T04:34:09.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.125+0000 7ffaa7400640 1 --2- 172.21.15.100:0/284403728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffaa81f7bd0 0x7ffaa814a0f0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7ffaa80ae780 tx=0x7ffaa0006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:09.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.125+0000 7ffa9ea00640 1 -- 172.21.15.100:0/284403728 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0015070 con 0x7ffaa81f7bd0 2024-09-17T04:34:09.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.125+0000 7ffa9ea00640 1 -- 172.21.15.100:0/284403728 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffaa0011900 con 0x7ffaa81f7bd0 2024-09-17T04:34:09.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.125+0000 7ffaada00640 1 -- 172.21.15.100:0/284403728 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa8156530 con 0x7ffaa81f7bd0 2024-09-17T04:34:09.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.125+0000 7ffa9ea00640 1 -- 172.21.15.100:0/284403728 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0011d50 con 0x7ffaa81f7bd0 2024-09-17T04:34:09.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.126+0000 7ffaada00640 1 -- 172.21.15.100:0/284403728 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa814b170 con 0x7ffaa81f7bd0 2024-09-17T04:34:09.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.127+0000 7ffaada00640 1 -- 172.21.15.100:0/284403728 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaa81535c0 con 0x7ffaa81f7bd0 2024-09-17T04:34:09.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.128+0000 7ffa9ea00640 1 -- 172.21.15.100:0/284403728 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7ffaa0018070 con 0x7ffaa81f7bd0 2024-09-17T04:34:09.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.129+0000 7ffa9ea00640 1 --2- 172.21.15.100:0/284403728 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ffa88079760 0x7ffa8807bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:09.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.129+0000 7ffa9ea00640 1 -- 172.21.15.100:0/284403728 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7ffaa009a570 con 0x7ffaa81f7bd0 2024-09-17T04:34:09.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.129+0000 7ffaa6a00640 1 --2- 172.21.15.100:0/284403728 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ffa88079760 0x7ffa8807bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:09.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.130+0000 7ffaa6a00640 1 --2- 172.21.15.100:0/284403728 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ffa88079760 0x7ffa8807bc70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ffa98006fd0 tx=0x7ffa98006970 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:09.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.133+0000 7ffa9ea00640 1 -- 172.21.15.100:0/284403728 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ffaa0063be0 con 0x7ffaa81f7bd0 2024-09-17T04:34:09.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.351+0000 7ffaada00640 1 -- 172.21.15.100:0/284403728 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffaa8155fe0 con 0x7ffaa81f7bd0 2024-09-17T04:34:09.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.352+0000 7ffa9ea00640 1 -- 172.21.15.100:0/284403728 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) ==== 95+0+5242 (secure 0 0 0) 0x7ffaa0068610 con 0x7ffaa81f7bd0 2024-09-17T04:34:09.351 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:34:09.351 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":122,"btime":"2024-09-17T04:34:03:329513+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/613141426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":613141426},{"type":"v1","addr":"172.21.15.184:6835","nonce":613141426}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":8554,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/1797529407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6801","nonce":1797529407},{"type":"v1","addr":"172.21.15.184:6834","nonce":1797529407}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":8560,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2263752731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2263752731},{"type":"v1","addr":"172.21.15.100:6833","nonce":2263752731}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:52.555851+0000","modified":"2024-09-17T04:33:55.567693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9153},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9153":{"gid":9153,"name":"c","rank":0,"incarnation":117,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/1207785711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1207785711},{"type":"v1","addr":"172.21.15.100:6835","nonce":1207785711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9153,"qdb_cluster":[9153]},"id":13}]} 2024-09-17T04:34:09.352 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 122 2024-09-17T04:34:09.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.354+0000 7ffaada00640 1 -- 172.21.15.100:0/284403728 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ffa88079760 msgr2=0x7ffa8807bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:09.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.354+0000 7ffaada00640 1 --2- 172.21.15.100:0/284403728 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ffa88079760 0x7ffa8807bc70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ffa98006fd0 tx=0x7ffa98006970 comp rx=0 tx=0).stop 2024-09-17T04:34:09.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.354+0000 7ffaada00640 1 -- 172.21.15.100:0/284403728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffaa81f7bd0 msgr2=0x7ffaa814a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:09.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.354+0000 7ffaada00640 1 --2- 172.21.15.100:0/284403728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffaa81f7bd0 0x7ffaa814a0f0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7ffaa80ae780 tx=0x7ffaa0006430 comp rx=0 tx=0).stop 2024-09-17T04:34:09.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.354+0000 7ffaada00640 1 -- 172.21.15.100:0/284403728 shutdown_connections 2024-09-17T04:34:09.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.354+0000 7ffaada00640 1 --2- 172.21.15.100:0/284403728 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ffa88079760 0x7ffa8807bc70 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:09.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.354+0000 7ffaada00640 1 --2- 172.21.15.100:0/284403728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffaa81f7bd0 0x7ffaa814a0f0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:09.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.354+0000 7ffaada00640 1 --2- 172.21.15.100:0/284403728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaa814c440 0x7ffaa81f7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:09.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.354+0000 7ffaada00640 1 --2- 172.21.15.100:0/284403728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa8c004800 0x7ffaa8155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:09.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.354+0000 7ffaada00640 1 -- 172.21.15.100:0/284403728 >> 172.21.15.100:0/284403728 conn(0x7ffaa806d730 msgr2=0x7ffaa80d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:09.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.354+0000 7ffaada00640 1 -- 172.21.15.100:0/284403728 shutdown_connections 2024-09-17T04:34:09.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.354+0000 7ffaada00640 1 -- 172.21.15.100:0/284403728 wait complete. 2024-09-17T04:34:09.378 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:34:09.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.598+0000 7fd84e600640 1 -- 172.21.15.100:0/2206841852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd84815d760 msgr2=0x7fd8481535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:09.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.598+0000 7fd84e600640 1 --2- 172.21.15.100:0/2206841852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd84815d760 0x7fd8481535c0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fd848072d60 tx=0x7fd83802a4d0 comp rx=0 tx=0).stop 2024-09-17T04:34:09.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.598+0000 7fd84e600640 1 -- 172.21.15.100:0/2206841852 shutdown_connections 2024-09-17T04:34:09.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.598+0000 7fd84e600640 1 --2- 172.21.15.100:0/2206841852 >> v2:172.21.15.184:3300/0 conn(0x7fd8480ad0e0 0x7fd8480ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:09.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.598+0000 7fd84e600640 1 --2- 172.21.15.100:0/2206841852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd84815d760 0x7fd8481535c0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:09.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.598+0000 7fd84e600640 1 -- 172.21.15.100:0/2206841852 >> 172.21.15.100:0/2206841852 conn(0x7fd84806d730 msgr2=0x7fd8480b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:09.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.598+0000 7fd84e600640 1 -- 172.21.15.100:0/2206841852 shutdown_connections 2024-09-17T04:34:09.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.598+0000 7fd84e600640 1 -- 172.21.15.100:0/2206841852 wait complete. 2024-09-17T04:34:09.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.600+0000 7fd84e600640 1 Processor -- start 2024-09-17T04:34:09.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.601+0000 7fd84e600640 1 -- start start 2024-09-17T04:34:09.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.602+0000 7fd84e600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8480ad0e0 0x7fd848156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:09.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.602+0000 7fd84e600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd84815d760 0x7fd8481f90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:09.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.602+0000 7fd84e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd848160310 0x7fd8481f9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:09.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.602+0000 7fd84e600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd8480b4c20 con 0x7fd848160310 2024-09-17T04:34:09.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.602+0000 7fd84e600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd848153c70 con 0x7fd84815d760 2024-09-17T04:34:09.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.602+0000 7fd84e600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd8480b4da0 con 0x7fd8480ad0e0 2024-09-17T04:34:09.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.602+0000 7fd847e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd848160310 0x7fd8481f9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:09.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.602+0000 7fd847e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd848160310 0x7fd8481f9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43388/0 (socket says 172.21.15.100:43388) 2024-09-17T04:34:09.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.602+0000 7fd847e00640 1 -- 172.21.15.100:0/601003363 learned_addr learned my addr 172.21.15.100:0/601003363 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:09.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.603+0000 7fd83ea00640 1 --2- 172.21.15.100:0/601003363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd84815d760 0x7fd8481f90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:09.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.603+0000 7fd847400640 1 --2- 172.21.15.100:0/601003363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8480ad0e0 0x7fd848156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:09.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.603+0000 7fd847e00640 1 -- 172.21.15.100:0/601003363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8480ad0e0 msgr2=0x7fd848156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:09.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.603+0000 7fd847e00640 1 --2- 172.21.15.100:0/601003363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8480ad0e0 0x7fd848156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:09.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.603+0000 7fd847e00640 1 -- 172.21.15.100:0/601003363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd84815d760 msgr2=0x7fd8481f90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:09.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.603+0000 7fd847e00640 1 --2- 172.21.15.100:0/601003363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd84815d760 0x7fd8481f90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:09.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.603+0000 7fd847e00640 1 -- 172.21.15.100:0/601003363 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8480b0800 con 0x7fd848160310 2024-09-17T04:34:09.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.603+0000 7fd83ea00640 1 --2- 172.21.15.100:0/601003363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd84815d760 0x7fd8481f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:09.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.603+0000 7fd847e00640 1 --2- 172.21.15.100:0/601003363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd848160310 0x7fd8481f9630 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fd84000b9e0 tx=0x7fd84000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:09.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.603+0000 7fd83fe00640 1 -- 172.21.15.100:0/601003363 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840003b70 con 0x7fd848160310 2024-09-17T04:34:09.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.603+0000 7fd83fe00640 1 -- 172.21.15.100:0/601003363 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd840003e00 con 0x7fd848160310 2024-09-17T04:34:09.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.603+0000 7fd84e600640 1 -- 172.21.15.100:0/601003363 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8480b0ae0 con 0x7fd848160310 2024-09-17T04:34:09.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.603+0000 7fd83fe00640 1 -- 172.21.15.100:0/601003363 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84000cb20 con 0x7fd848160310 2024-09-17T04:34:09.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.603+0000 7fd84e600640 1 -- 172.21.15.100:0/601003363 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8480b10a0 con 0x7fd848160310 2024-09-17T04:34:09.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.605+0000 7fd84e600640 1 -- 172.21.15.100:0/601003363 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8480b4620 con 0x7fd848160310 2024-09-17T04:34:09.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.606+0000 7fd83fe00640 1 -- 172.21.15.100:0/601003363 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7fd84000ccc0 con 0x7fd848160310 2024-09-17T04:34:09.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.607+0000 7fd83fe00640 1 --2- 172.21.15.100:0/601003363 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd824079760 0x7fd82407bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:09.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.607+0000 7fd83fe00640 1 -- 172.21.15.100:0/601003363 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7fd840099710 con 0x7fd848160310 2024-09-17T04:34:09.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.607+0000 7fd847400640 1 --2- 172.21.15.100:0/601003363 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd824079760 0x7fd82407bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:09.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.608+0000 7fd847400640 1 --2- 172.21.15.100:0/601003363 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd824079760 0x7fd82407bc70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd838033000 tx=0x7fd838004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:09.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.611+0000 7fd83fe00640 1 -- 172.21.15.100:0/601003363 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd8480b4620 con 0x7fd848160310 2024-09-17T04:34:09.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.797+0000 7fd84e600640 1 -- 172.21.15.100:0/601003363 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd848160fa0 con 0x7fd848160310 2024-09-17T04:34:09.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.798+0000 7fd83fe00640 1 -- 172.21.15.100:0/601003363 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) ==== 74+0+16880 (secure 0 0 0) 0x7fd8400633a0 con 0x7fd848160310 2024-09-17T04:34:09.798 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:34:09.798 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":167,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:33:54.064660+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:33:50.447902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:51.510699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:34:09.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.800+0000 7fd84e600640 1 -- 172.21.15.100:0/601003363 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd824079760 msgr2=0x7fd82407bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:09.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.800+0000 7fd84e600640 1 --2- 172.21.15.100:0/601003363 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd824079760 0x7fd82407bc70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd838033000 tx=0x7fd838004730 comp rx=0 tx=0).stop 2024-09-17T04:34:09.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.800+0000 7fd84e600640 1 -- 172.21.15.100:0/601003363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd848160310 msgr2=0x7fd8481f9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:09.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.800+0000 7fd84e600640 1 --2- 172.21.15.100:0/601003363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd848160310 0x7fd8481f9630 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fd84000b9e0 tx=0x7fd84000bea0 comp rx=0 tx=0).stop 2024-09-17T04:34:09.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.801+0000 7fd84e600640 1 -- 172.21.15.100:0/601003363 shutdown_connections 2024-09-17T04:34:09.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.801+0000 7fd84e600640 1 --2- 172.21.15.100:0/601003363 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd824079760 0x7fd82407bc70 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:09.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.801+0000 7fd84e600640 1 --2- 172.21.15.100:0/601003363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd848160310 0x7fd8481f9630 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:09.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.801+0000 7fd84e600640 1 --2- 172.21.15.100:0/601003363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd84815d760 0x7fd8481f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:09.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.801+0000 7fd84e600640 1 --2- 172.21.15.100:0/601003363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8480ad0e0 0x7fd848156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:09.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.801+0000 7fd84e600640 1 -- 172.21.15.100:0/601003363 >> 172.21.15.100:0/601003363 conn(0x7fd84806d730 msgr2=0x7fd84806fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:09.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.801+0000 7fd84e600640 1 -- 172.21.15.100:0/601003363 shutdown_connections 2024-09-17T04:34:09.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:09.801+0000 7fd84e600640 1 -- 172.21.15.100:0/601003363 wait complete. 2024-09-17T04:34:09.823 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T04:34:09.824 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:34:10.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.046+0000 7ff145c00640 1 -- 172.21.15.100:0/786321666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1400ad0e0 msgr2=0x7ff1400ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:10.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.046+0000 7ff145c00640 1 --2- 172.21.15.100:0/786321666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1400ad0e0 0x7ff1400ad4f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009830 tx=0x7ff12c02a410 comp rx=0 tx=0).stop 2024-09-17T04:34:10.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.046+0000 7ff145c00640 1 -- 172.21.15.100:0/786321666 shutdown_connections 2024-09-17T04:34:10.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.046+0000 7ff145c00640 1 --2- 172.21.15.100:0/786321666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1400b3550 0x7ff1400b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:10.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.046+0000 7ff145c00640 1 --2- 172.21.15.100:0/786321666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1400ad0e0 0x7ff1400ad4f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:10.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.046+0000 7ff145c00640 1 -- 172.21.15.100:0/786321666 >> 172.21.15.100:0/786321666 conn(0x7ff14006d730 msgr2=0x7ff1400b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:10.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.046+0000 7ff145c00640 1 -- 172.21.15.100:0/786321666 shutdown_connections 2024-09-17T04:34:10.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.046+0000 7ff145c00640 1 -- 172.21.15.100:0/786321666 wait complete. 2024-09-17T04:34:10.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.048+0000 7ff145c00640 1 Processor -- start 2024-09-17T04:34:10.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.048+0000 7ff145c00640 1 -- start start 2024-09-17T04:34:10.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.050+0000 7ff145c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1400ad0e0 0x7ff140146630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:10.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.050+0000 7ff145c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1400b3550 0x7ff1400b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:10.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.050+0000 7ff145c00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff14015d760 0x7ff1400b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:10.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.050+0000 7ff145c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff1400b4da0 con 0x7ff1400b3550 2024-09-17T04:34:10.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.050+0000 7ff145c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff1400b4c20 con 0x7ff1400ad0e0 2024-09-17T04:34:10.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.050+0000 7ff145c00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff1400b3ea0 con 0x7ff14015d760 2024-09-17T04:34:10.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.050+0000 7ff13e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1400b3550 0x7ff1400b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:10.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.050+0000 7ff13e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1400b3550 0x7ff1400b0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43404/0 (socket says 172.21.15.100:43404) 2024-09-17T04:34:10.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.050+0000 7ff13e000640 1 -- 172.21.15.100:0/4265798896 learned_addr learned my addr 172.21.15.100:0/4265798896 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:10.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.050+0000 7ff13f400640 1 --2- 172.21.15.100:0/4265798896 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff14015d760 0x7ff1400b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:10.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.050+0000 7ff13ea00640 1 --2- 172.21.15.100:0/4265798896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1400ad0e0 0x7ff140146630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:10.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.050+0000 7ff13e000640 1 -- 172.21.15.100:0/4265798896 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff14015d760 msgr2=0x7ff1400b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:10.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.050+0000 7ff13e000640 1 --2- 172.21.15.100:0/4265798896 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff14015d760 0x7ff1400b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:10.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.050+0000 7ff13e000640 1 -- 172.21.15.100:0/4265798896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1400ad0e0 msgr2=0x7ff140146630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:10.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.050+0000 7ff13e000640 1 --2- 172.21.15.100:0/4265798896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1400ad0e0 0x7ff140146630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:10.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.050+0000 7ff13e000640 1 -- 172.21.15.100:0/4265798896 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff140158870 con 0x7ff1400b3550 2024-09-17T04:34:10.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.051+0000 7ff13e000640 1 --2- 172.21.15.100:0/4265798896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1400b3550 0x7ff1400b0da0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009950 tx=0x7ff12c003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:10.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.051+0000 7ff136a00640 1 -- 172.21.15.100:0/4265798896 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c03f030 con 0x7ff1400b3550 2024-09-17T04:34:10.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.051+0000 7ff136a00640 1 -- 172.21.15.100:0/4265798896 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff12c004510 con 0x7ff1400b3550 2024-09-17T04:34:10.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.051+0000 7ff145c00640 1 -- 172.21.15.100:0/4265798896 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff140158b50 con 0x7ff1400b3550 2024-09-17T04:34:10.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.051+0000 7ff136a00640 1 -- 172.21.15.100:0/4265798896 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c037040 con 0x7ff1400b3550 2024-09-17T04:34:10.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.051+0000 7ff145c00640 1 -- 172.21.15.100:0/4265798896 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1401f3b20 con 0x7ff1400b3550 2024-09-17T04:34:10.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.052+0000 7ff145c00640 1 -- 172.21.15.100:0/4265798896 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1400b4380 con 0x7ff1400b3550 2024-09-17T04:34:10.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.053+0000 7ff136a00640 1 -- 172.21.15.100:0/4265798896 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7ff12c042070 con 0x7ff1400b3550 2024-09-17T04:34:10.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.055+0000 7ff136a00640 1 --2- 172.21.15.100:0/4265798896 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff120079760 0x7ff12007bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:10.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.055+0000 7ff136a00640 1 -- 172.21.15.100:0/4265798896 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7ff12c0b9410 con 0x7ff1400b3550 2024-09-17T04:34:10.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.055+0000 7ff13ea00640 1 --2- 172.21.15.100:0/4265798896 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff120079760 0x7ff12007bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:10.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.056+0000 7ff13ea00640 1 --2- 172.21.15.100:0/4265798896 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff120079760 0x7ff12007bc70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff130007960 tx=0x7ff130001290 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:10.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.059+0000 7ff136a00640 1 -- 172.21.15.100:0/4265798896 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff12c049070 con 0x7ff1400b3550 2024-09-17T04:34:10.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.279+0000 7ff145c00640 1 -- 172.21.15.100:0/4265798896 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff14015fa80 con 0x7ff1400b3550 2024-09-17T04:34:10.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.280+0000 7ff136a00640 1 -- 172.21.15.100:0/4265798896 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) ==== 95+0+5242 (secure 0 0 0) 0x7ff12c082a80 con 0x7ff1400b3550 2024-09-17T04:34:10.279 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:34:10.279 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":122,"btime":"2024-09-17T04:34:03:329513+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/613141426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":613141426},{"type":"v1","addr":"172.21.15.184:6835","nonce":613141426}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":8554,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/1797529407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6801","nonce":1797529407},{"type":"v1","addr":"172.21.15.184:6834","nonce":1797529407}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":8560,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2263752731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2263752731},{"type":"v1","addr":"172.21.15.100:6833","nonce":2263752731}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:52.555851+0000","modified":"2024-09-17T04:33:55.567693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9153},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9153":{"gid":9153,"name":"c","rank":0,"incarnation":117,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/1207785711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1207785711},{"type":"v1","addr":"172.21.15.100:6835","nonce":1207785711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9153,"qdb_cluster":[9153]},"id":13}]} 2024-09-17T04:34:10.280 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 122 2024-09-17T04:34:10.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.282+0000 7ff145c00640 1 -- 172.21.15.100:0/4265798896 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff120079760 msgr2=0x7ff12007bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:10.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.282+0000 7ff145c00640 1 --2- 172.21.15.100:0/4265798896 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff120079760 0x7ff12007bc70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff130007960 tx=0x7ff130001290 comp rx=0 tx=0).stop 2024-09-17T04:34:10.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.282+0000 7ff145c00640 1 -- 172.21.15.100:0/4265798896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1400b3550 msgr2=0x7ff1400b0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:10.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.282+0000 7ff145c00640 1 --2- 172.21.15.100:0/4265798896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1400b3550 0x7ff1400b0da0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009950 tx=0x7ff12c003b60 comp rx=0 tx=0).stop 2024-09-17T04:34:10.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.282+0000 7ff145c00640 1 -- 172.21.15.100:0/4265798896 shutdown_connections 2024-09-17T04:34:10.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.282+0000 7ff145c00640 1 --2- 172.21.15.100:0/4265798896 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff120079760 0x7ff12007bc70 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:10.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.282+0000 7ff145c00640 1 --2- 172.21.15.100:0/4265798896 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff14015d760 0x7ff1400b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:10.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.282+0000 7ff145c00640 1 --2- 172.21.15.100:0/4265798896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1400b3550 0x7ff1400b0da0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:10.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.282+0000 7ff145c00640 1 --2- 172.21.15.100:0/4265798896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1400ad0e0 0x7ff140146630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:10.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.282+0000 7ff145c00640 1 -- 172.21.15.100:0/4265798896 >> 172.21.15.100:0/4265798896 conn(0x7ff14006d730 msgr2=0x7ff1400cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:10.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.282+0000 7ff145c00640 1 -- 172.21.15.100:0/4265798896 shutdown_connections 2024-09-17T04:34:10.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.282+0000 7ff145c00640 1 -- 172.21.15.100:0/4265798896 wait complete. 2024-09-17T04:34:10.303 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:34:10.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.523+0000 7fd338400640 1 -- 172.21.15.100:0/1883268631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd33006f7b0 msgr2=0x7fd3300b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:10.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.523+0000 7fd338400640 1 --2- 172.21.15.100:0/1883268631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd33006f7b0 0x7fd3300b31e0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fd328005fd0 tx=0x7fd32802aa90 comp rx=0 tx=0).stop 2024-09-17T04:34:10.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.523+0000 7fd338400640 1 -- 172.21.15.100:0/1883268631 shutdown_connections 2024-09-17T04:34:10.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.523+0000 7fd338400640 1 --2- 172.21.15.100:0/1883268631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd33006f7b0 0x7fd3300b31e0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:10.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.523+0000 7fd338400640 1 --2- 172.21.15.100:0/1883268631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3300af630 0x7fd330073500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:10.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.523+0000 7fd338400640 1 -- 172.21.15.100:0/1883268631 >> 172.21.15.100:0/1883268631 conn(0x7fd33006d730 msgr2=0x7fd3300b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:10.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.523+0000 7fd338400640 1 -- 172.21.15.100:0/1883268631 shutdown_connections 2024-09-17T04:34:10.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.523+0000 7fd338400640 1 -- 172.21.15.100:0/1883268631 wait complete. 2024-09-17T04:34:10.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.525+0000 7fd338400640 1 Processor -- start 2024-09-17T04:34:10.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.525+0000 7fd338400640 1 -- start start 2024-09-17T04:34:10.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.527+0000 7fd338400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd33006f7b0 0x7fd33009b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:10.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.527+0000 7fd338400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd330073a40 0x7fd33009dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:10.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.527+0000 7fd338400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3300af630 0x7fd33009c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:10.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.527+0000 7fd338400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd3300b3ca0 con 0x7fd33006f7b0 2024-09-17T04:34:10.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.527+0000 7fd338400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd3300b3b20 con 0x7fd330073a40 2024-09-17T04:34:10.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.527+0000 7fd338400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd3300b3e20 con 0x7fd3300af630 2024-09-17T04:34:10.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.527+0000 7fd335a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd33006f7b0 0x7fd33009b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:10.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.527+0000 7fd335a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd33006f7b0 0x7fd33009b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43424/0 (socket says 172.21.15.100:43424) 2024-09-17T04:34:10.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.527+0000 7fd335a00640 1 -- 172.21.15.100:0/2947366143 learned_addr learned my addr 172.21.15.100:0/2947366143 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:10.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.527+0000 7fd335000640 1 --2- 172.21.15.100:0/2947366143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd330073a40 0x7fd33009dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:10.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.527+0000 7fd336400640 1 --2- 172.21.15.100:0/2947366143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3300af630 0x7fd33009c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:10.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.528+0000 7fd335a00640 1 -- 172.21.15.100:0/2947366143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3300af630 msgr2=0x7fd33009c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:10.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.528+0000 7fd335a00640 1 --2- 172.21.15.100:0/2947366143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3300af630 0x7fd33009c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:10.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.528+0000 7fd335a00640 1 -- 172.21.15.100:0/2947366143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd330073a40 msgr2=0x7fd33009dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:10.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.528+0000 7fd335a00640 1 --2- 172.21.15.100:0/2947366143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd330073a40 0x7fd33009dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:10.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.528+0000 7fd335a00640 1 -- 172.21.15.100:0/2947366143 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3300ca2b0 con 0x7fd33006f7b0 2024-09-17T04:34:10.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.528+0000 7fd335a00640 1 --2- 172.21.15.100:0/2947366143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd33006f7b0 0x7fd33009b8d0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fd32400d740 tx=0x7fd32400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:10.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.528+0000 7fd32d600640 1 -- 172.21.15.100:0/2947366143 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd324017070 con 0x7fd33006f7b0 2024-09-17T04:34:10.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.528+0000 7fd32d600640 1 -- 172.21.15.100:0/2947366143 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd324009cc0 con 0x7fd33006f7b0 2024-09-17T04:34:10.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.528+0000 7fd338400640 1 -- 172.21.15.100:0/2947366143 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd33009adf0 con 0x7fd33006f7b0 2024-09-17T04:34:10.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.528+0000 7fd32d600640 1 -- 172.21.15.100:0/2947366143 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3240106c0 con 0x7fd33006f7b0 2024-09-17T04:34:10.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.528+0000 7fd338400640 1 -- 172.21.15.100:0/2947366143 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd33009b3b0 con 0x7fd33006f7b0 2024-09-17T04:34:10.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.529+0000 7fd338400640 1 -- 172.21.15.100:0/2947366143 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3300b4340 con 0x7fd33006f7b0 2024-09-17T04:34:10.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.531+0000 7fd32d600640 1 -- 172.21.15.100:0/2947366143 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7fd32401a070 con 0x7fd33006f7b0 2024-09-17T04:34:10.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.532+0000 7fd32d600640 1 --2- 172.21.15.100:0/2947366143 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd310079760 0x7fd31007bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:10.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.532+0000 7fd32d600640 1 -- 172.21.15.100:0/2947366143 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7fd3240999b0 con 0x7fd33006f7b0 2024-09-17T04:34:10.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.532+0000 7fd335000640 1 --2- 172.21.15.100:0/2947366143 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd310079760 0x7fd31007bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:10.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.533+0000 7fd335000640 1 --2- 172.21.15.100:0/2947366143 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd310079760 0x7fd31007bc70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd330157fc0 tx=0x7fd31c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:10.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.536+0000 7fd32d600640 1 -- 172.21.15.100:0/2947366143 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd324062ff0 con 0x7fd33006f7b0 2024-09-17T04:34:10.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.755+0000 7fd338400640 1 -- 172.21.15.100:0/2947366143 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3300aeca0 con 0x7fd33006f7b0 2024-09-17T04:34:10.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.755+0000 7fd32d600640 1 -- 172.21.15.100:0/2947366143 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) ==== 95+0+5242 (secure 0 0 0) 0x7fd324067a20 con 0x7fd33006f7b0 2024-09-17T04:34:10.755 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:34:10.755 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":122,"btime":"2024-09-17T04:34:03:329513+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/613141426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":613141426},{"type":"v1","addr":"172.21.15.184:6835","nonce":613141426}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":8554,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/1797529407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6801","nonce":1797529407},{"type":"v1","addr":"172.21.15.184:6834","nonce":1797529407}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":8560,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2263752731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2263752731},{"type":"v1","addr":"172.21.15.100:6833","nonce":2263752731}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:52.555851+0000","modified":"2024-09-17T04:33:55.567693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9153},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9153":{"gid":9153,"name":"c","rank":0,"incarnation":117,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/1207785711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1207785711},{"type":"v1","addr":"172.21.15.100:6835","nonce":1207785711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9153,"qdb_cluster":[9153]},"id":13}]} 2024-09-17T04:34:10.755 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 122 2024-09-17T04:34:10.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.758+0000 7fd338400640 1 -- 172.21.15.100:0/2947366143 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd310079760 msgr2=0x7fd31007bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:10.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.758+0000 7fd338400640 1 --2- 172.21.15.100:0/2947366143 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd310079760 0x7fd31007bc70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd330157fc0 tx=0x7fd31c009040 comp rx=0 tx=0).stop 2024-09-17T04:34:10.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.758+0000 7fd338400640 1 -- 172.21.15.100:0/2947366143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd33006f7b0 msgr2=0x7fd33009b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:10.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.758+0000 7fd338400640 1 --2- 172.21.15.100:0/2947366143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd33006f7b0 0x7fd33009b8d0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fd32400d740 tx=0x7fd32400dc00 comp rx=0 tx=0).stop 2024-09-17T04:34:10.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.758+0000 7fd338400640 1 -- 172.21.15.100:0/2947366143 shutdown_connections 2024-09-17T04:34:10.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.758+0000 7fd338400640 1 --2- 172.21.15.100:0/2947366143 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd310079760 0x7fd31007bc70 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:10.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.758+0000 7fd338400640 1 --2- 172.21.15.100:0/2947366143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3300af630 0x7fd33009c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:10.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.758+0000 7fd338400640 1 --2- 172.21.15.100:0/2947366143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd330073a40 0x7fd33009dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:10.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.758+0000 7fd338400640 1 --2- 172.21.15.100:0/2947366143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd33006f7b0 0x7fd33009b8d0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:10.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.758+0000 7fd338400640 1 -- 172.21.15.100:0/2947366143 >> 172.21.15.100:0/2947366143 conn(0x7fd33006d730 msgr2=0x7fd3300b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:10.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.758+0000 7fd338400640 1 -- 172.21.15.100:0/2947366143 shutdown_connections 2024-09-17T04:34:10.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:10.758+0000 7fd338400640 1 -- 172.21.15.100:0/2947366143 wait complete. 2024-09-17T04:34:10.784 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:34:11.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.003+0000 7f699c200640 1 -- 172.21.15.100:0/4021033492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6994152920 msgr2=0x7f699415f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:11.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.003+0000 7f699c200640 1 --2- 172.21.15.100:0/4021033492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6994152920 0x7f699415f3d0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f6980005fd0 tx=0x7f698002a540 comp rx=0 tx=0).stop 2024-09-17T04:34:11.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.003+0000 7f699c200640 1 -- 172.21.15.100:0/4021033492 shutdown_connections 2024-09-17T04:34:11.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.003+0000 7f699c200640 1 --2- 172.21.15.100:0/4021033492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f699415f910 0x7f69940b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:11.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.003+0000 7f699c200640 1 --2- 172.21.15.100:0/4021033492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6994152920 0x7f699415f3d0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:11.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.003+0000 7f699c200640 1 -- 172.21.15.100:0/4021033492 >> 172.21.15.100:0/4021033492 conn(0x7f699406d730 msgr2=0x7f69940b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:11.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.003+0000 7f699c200640 1 -- 172.21.15.100:0/4021033492 shutdown_connections 2024-09-17T04:34:11.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.003+0000 7f699c200640 1 -- 172.21.15.100:0/4021033492 wait complete. 2024-09-17T04:34:11.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.006+0000 7f699c200640 1 Processor -- start 2024-09-17T04:34:11.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.006+0000 7f699c200640 1 -- start start 2024-09-17T04:34:11.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.007+0000 7f699c200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69940aebc0 0x7f6994148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:11.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.007+0000 7f699c200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6994152920 0x7f69940ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:11.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.007+0000 7f699c200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f699415f910 0x7f69941f9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:11.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.007+0000 7f699c200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f69940b4190 con 0x7f6994152920 2024-09-17T04:34:11.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.007+0000 7f699c200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f69940b4010 con 0x7f69940aebc0 2024-09-17T04:34:11.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.007+0000 7f699c200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f69940b4310 con 0x7f699415f910 2024-09-17T04:34:11.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.007+0000 7f6998e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6994152920 0x7f69940ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:11.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.007+0000 7f6998e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6994152920 0x7f69940ceea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43430/0 (socket says 172.21.15.100:43430) 2024-09-17T04:34:11.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.007+0000 7f6998e00640 1 -- 172.21.15.100:0/2026320248 learned_addr learned my addr 172.21.15.100:0/2026320248 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:11.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.007+0000 7f6999800640 1 --2- 172.21.15.100:0/2026320248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69940aebc0 0x7f6994148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:11.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.008+0000 7f699a200640 1 --2- 172.21.15.100:0/2026320248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f699415f910 0x7f69941f9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:11.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.008+0000 7f6998e00640 1 -- 172.21.15.100:0/2026320248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f699415f910 msgr2=0x7f69941f9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:11.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.008+0000 7f6998e00640 1 --2- 172.21.15.100:0/2026320248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f699415f910 0x7f69941f9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:11.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.008+0000 7f6998e00640 1 -- 172.21.15.100:0/2026320248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69940aebc0 msgr2=0x7f6994148aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:11.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.008+0000 7f6998e00640 1 --2- 172.21.15.100:0/2026320248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69940aebc0 0x7f6994148aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:11.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.008+0000 7f6998e00640 1 -- 172.21.15.100:0/2026320248 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69941fa280 con 0x7f6994152920 2024-09-17T04:34:11.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.008+0000 7f6998e00640 1 --2- 172.21.15.100:0/2026320248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6994152920 0x7f69940ceea0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f6980034040 tx=0x7f69800038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:11.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.008+0000 7f6991600640 1 -- 172.21.15.100:0/2026320248 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698003e030 con 0x7f6994152920 2024-09-17T04:34:11.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.009+0000 7f6991600640 1 -- 172.21.15.100:0/2026320248 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6980003d90 con 0x7f6994152920 2024-09-17T04:34:11.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.009+0000 7f6991600640 1 -- 172.21.15.100:0/2026320248 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980035040 con 0x7f6994152920 2024-09-17T04:34:11.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.009+0000 7f699c200640 1 -- 172.21.15.100:0/2026320248 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6994155be0 con 0x7f6994152920 2024-09-17T04:34:11.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.009+0000 7f699c200640 1 -- 172.21.15.100:0/2026320248 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69940b00f0 con 0x7f6994152920 2024-09-17T04:34:11.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.010+0000 7f699c200640 1 -- 172.21.15.100:0/2026320248 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f699415b070 con 0x7f6994152920 2024-09-17T04:34:11.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.011+0000 7f6991600640 1 -- 172.21.15.100:0/2026320248 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7f6980041030 con 0x7f6994152920 2024-09-17T04:34:11.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.012+0000 7f6991600640 1 --2- 172.21.15.100:0/2026320248 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6974079790 0x7f697407bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:11.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.012+0000 7f6991600640 1 -- 172.21.15.100:0/2026320248 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7f69800ba3a0 con 0x7f6994152920 2024-09-17T04:34:11.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.012+0000 7f6999800640 1 --2- 172.21.15.100:0/2026320248 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6974079790 0x7f697407bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:11.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.013+0000 7f6999800640 1 --2- 172.21.15.100:0/2026320248 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6974079790 0x7f697407bca0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6988001720 tx=0x7f6988009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:11.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.016+0000 7f6991600640 1 -- 172.21.15.100:0/2026320248 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f699415fd20 con 0x7f6994152920 2024-09-17T04:34:11.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.201+0000 7f699c200640 1 -- 172.21.15.100:0/2026320248 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69940b4d90 con 0x7f6994152920 2024-09-17T04:34:11.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.202+0000 7f6991600640 1 -- 172.21.15.100:0/2026320248 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) ==== 74+0+16880 (secure 0 0 0) 0x7f69800839e0 con 0x7f6994152920 2024-09-17T04:34:11.202 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:34:11.202 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":167,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:33:54.064660+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:33:50.447902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:51.510699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:34:11.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.205+0000 7f699c200640 1 -- 172.21.15.100:0/2026320248 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6974079790 msgr2=0x7f697407bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:11.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.205+0000 7f699c200640 1 --2- 172.21.15.100:0/2026320248 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6974079790 0x7f697407bca0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6988001720 tx=0x7f6988009040 comp rx=0 tx=0).stop 2024-09-17T04:34:11.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.205+0000 7f699c200640 1 -- 172.21.15.100:0/2026320248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6994152920 msgr2=0x7f69940ceea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:11.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.205+0000 7f699c200640 1 --2- 172.21.15.100:0/2026320248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6994152920 0x7f69940ceea0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f6980034040 tx=0x7f69800038a0 comp rx=0 tx=0).stop 2024-09-17T04:34:11.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.205+0000 7f699c200640 1 -- 172.21.15.100:0/2026320248 shutdown_connections 2024-09-17T04:34:11.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.205+0000 7f699c200640 1 --2- 172.21.15.100:0/2026320248 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6974079790 0x7f697407bca0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:11.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.205+0000 7f699c200640 1 --2- 172.21.15.100:0/2026320248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f699415f910 0x7f69941f9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:11.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.205+0000 7f699c200640 1 --2- 172.21.15.100:0/2026320248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6994152920 0x7f69940ceea0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:11.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.205+0000 7f699c200640 1 --2- 172.21.15.100:0/2026320248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69940aebc0 0x7f6994148aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:11.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.205+0000 7f699c200640 1 -- 172.21.15.100:0/2026320248 >> 172.21.15.100:0/2026320248 conn(0x7f699406d730 msgr2=0x7f69940cb830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:11.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.205+0000 7f699c200640 1 -- 172.21.15.100:0/2026320248 shutdown_connections 2024-09-17T04:34:11.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.205+0000 7f699c200640 1 -- 172.21.15.100:0/2026320248 wait complete. 2024-09-17T04:34:11.230 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T04:34:11.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.454+0000 7fd504200640 1 -- 172.21.15.100:0/3945863240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4e40041c0 msgr2=0x7fd4e4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:11.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.454+0000 7fd504200640 1 --2- 172.21.15.100:0/3945863240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4e40041c0 0x7fd4e4004800 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8009b50 tx=0x7fd4e4005830 comp rx=0 tx=0).stop 2024-09-17T04:34:11.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.454+0000 7fd502200640 1 -- 172.21.15.100:0/3945863240 reap_dead start 2024-09-17T04:34:11.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.455+0000 7fd504200640 1 -- 172.21.15.100:0/3945863240 shutdown_connections 2024-09-17T04:34:11.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.455+0000 7fd504200640 1 -- 172.21.15.100:0/3945863240 >> 172.21.15.100:0/3945863240 conn(0x7fd4fc06d3d0 msgr2=0x7fd4fc0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:11.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.455+0000 7fd504200640 1 -- 172.21.15.100:0/3945863240 shutdown_connections 2024-09-17T04:34:11.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.455+0000 7fd504200640 1 -- 172.21.15.100:0/3945863240 wait complete. 2024-09-17T04:34:11.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.457+0000 7fd504200640 1 Processor -- start 2024-09-17T04:34:11.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.457+0000 7fd504200640 1 -- start start 2024-09-17T04:34:11.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.458+0000 7fd504200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4e40041c0 0x7fd4fc155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:11.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.459+0000 7fd504200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4fc156480 0x7fd4fc0ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:11.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.459+0000 7fd504200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4fc155f30 0x7fd4fc0cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:11.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.459+0000 7fd504200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd4fc0cb950 con 0x7fd4fc156480 2024-09-17T04:34:11.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.459+0000 7fd504200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd4fc0afab0 con 0x7fd4fc155f30 2024-09-17T04:34:11.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.459+0000 7fd504200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd4fc0cbad0 con 0x7fd4e40041c0 2024-09-17T04:34:11.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.459+0000 7fd500e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4fc156480 0x7fd4fc0ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:11.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.459+0000 7fd500e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4fc156480 0x7fd4fc0ccd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43454/0 (socket says 172.21.15.100:43454) 2024-09-17T04:34:11.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.459+0000 7fd500e00640 1 -- 172.21.15.100:0/995697740 learned_addr learned my addr 172.21.15.100:0/995697740 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:11.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.459+0000 7fd502200640 1 --2- 172.21.15.100:0/995697740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4fc155f30 0x7fd4fc0cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:11.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.459+0000 7fd501800640 1 --2- 172.21.15.100:0/995697740 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4e40041c0 0x7fd4fc155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:11.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.459+0000 7fd500e00640 1 -- 172.21.15.100:0/995697740 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4e40041c0 msgr2=0x7fd4fc155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:11.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.459+0000 7fd500e00640 1 --2- 172.21.15.100:0/995697740 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4e40041c0 0x7fd4fc155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:11.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.459+0000 7fd500e00640 1 -- 172.21.15.100:0/995697740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4fc155f30 msgr2=0x7fd4fc0cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:11.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.459+0000 7fd500e00640 1 --2- 172.21.15.100:0/995697740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4fc155f30 0x7fd4fc0cd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:11.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.459+0000 7fd500e00640 1 -- 172.21.15.100:0/995697740 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4fc157e50 con 0x7fd4fc156480 2024-09-17T04:34:11.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.459+0000 7fd501800640 1 --2- 172.21.15.100:0/995697740 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4e40041c0 0x7fd4fc155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:11.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.459+0000 7fd502200640 1 --2- 172.21.15.100:0/995697740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4fc155f30 0x7fd4fc0cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:11.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.460+0000 7fd500e00640 1 --2- 172.21.15.100:0/995697740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4fc156480 0x7fd4fc0ccd70 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8036040 tx=0x7fd4e8033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:11.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.460+0000 7fd4f9600640 1 -- 172.21.15.100:0/995697740 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8038040 con 0x7fd4fc156480 2024-09-17T04:34:11.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.460+0000 7fd4f9600640 1 -- 172.21.15.100:0/995697740 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4e8041070 con 0x7fd4fc156480 2024-09-17T04:34:11.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.460+0000 7fd504200640 1 -- 172.21.15.100:0/995697740 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4fc158130 con 0x7fd4fc156480 2024-09-17T04:34:11.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.460+0000 7fd4f9600640 1 -- 172.21.15.100:0/995697740 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8033d50 con 0x7fd4fc156480 2024-09-17T04:34:11.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.460+0000 7fd504200640 1 -- 172.21.15.100:0/995697740 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4fc0c52d0 con 0x7fd4fc156480 2024-09-17T04:34:11.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.461+0000 7fd504200640 1 -- 172.21.15.100:0/995697740 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4fc157bc0 con 0x7fd4fc156480 2024-09-17T04:34:11.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.462+0000 7fd4f9600640 1 -- 172.21.15.100:0/995697740 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7fd4e8046460 con 0x7fd4fc156480 2024-09-17T04:34:11.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.463+0000 7fd4f9600640 1 --2- 172.21.15.100:0/995697740 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd4dc079790 0x7fd4dc07bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:11.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.463+0000 7fd4f9600640 1 -- 172.21.15.100:0/995697740 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5765+0+0 (secure 0 0 0) 0x7fd4e80ba200 con 0x7fd4fc156480 2024-09-17T04:34:11.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.463+0000 7fd501800640 1 --2- 172.21.15.100:0/995697740 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd4dc079790 0x7fd4dc07bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:11.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.464+0000 7fd501800640 1 --2- 172.21.15.100:0/995697740 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd4dc079790 0x7fd4dc07bca0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc072000 tx=0x7fd4f0006720 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:11.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.468+0000 7fd4f9600640 1 -- 172.21.15.100:0/995697740 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd4e8083ec0 con 0x7fd4fc156480 2024-09-17T04:34:11.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.689+0000 7fd504200640 1 -- 172.21.15.100:0/995697740 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd4fc0ad440 con 0x7fd4fc156480 2024-09-17T04:34:11.689 INFO:tasks.ceph.mon.a.smithi100.stderr:2024-09-17T04:34:11.690+0000 7fd3d0a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T04:34:11.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.691+0000 7fd4f9600640 1 -- 172.21.15.100:0/995697740 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v123) ==== 161+0+0 (secure 0 0 0) 0x7fd4e80837c0 con 0x7fd4fc156480 2024-09-17T04:34:11.691 INFO:teuthology.orchestra.run.smithi100.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T04:34:11.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.695+0000 7fd504200640 1 -- 172.21.15.100:0/995697740 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd4dc079790 msgr2=0x7fd4dc07bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:11.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.695+0000 7fd504200640 1 --2- 172.21.15.100:0/995697740 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd4dc079790 0x7fd4dc07bca0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc072000 tx=0x7fd4f0006720 comp rx=0 tx=0).stop 2024-09-17T04:34:11.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.695+0000 7fd504200640 1 -- 172.21.15.100:0/995697740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4fc156480 msgr2=0x7fd4fc0ccd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:11.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.695+0000 7fd504200640 1 --2- 172.21.15.100:0/995697740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4fc156480 0x7fd4fc0ccd70 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8036040 tx=0x7fd4e8033670 comp rx=0 tx=0).stop 2024-09-17T04:34:11.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.695+0000 7fd504200640 1 -- 172.21.15.100:0/995697740 shutdown_connections 2024-09-17T04:34:11.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.695+0000 7fd504200640 1 --2- 172.21.15.100:0/995697740 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fd4dc079790 0x7fd4dc07bca0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:11.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.695+0000 7fd504200640 1 --2- 172.21.15.100:0/995697740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4fc155f30 0x7fd4fc0cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:11.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.695+0000 7fd504200640 1 --2- 172.21.15.100:0/995697740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4fc156480 0x7fd4fc0ccd70 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:11.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.695+0000 7fd504200640 1 --2- 172.21.15.100:0/995697740 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4e40041c0 0x7fd4fc155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:11.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.695+0000 7fd504200640 1 -- 172.21.15.100:0/995697740 >> 172.21.15.100:0/995697740 conn(0x7fd4fc06d3d0 msgr2=0x7fd4fc0aea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:11.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.695+0000 7fd504200640 1 -- 172.21.15.100:0/995697740 shutdown_connections 2024-09-17T04:34:11.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.696+0000 7fd504200640 1 -- 172.21.15.100:0/995697740 wait complete. 2024-09-17T04:34:11.718 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T04:34:11.836 INFO:tasks.ceph.mds.c.smithi100.stdout:starting mds.c at 2024-09-17T04:34:11.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.944+0000 7f354fe00640 1 -- 172.21.15.100:0/717791147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35480d0ee0 msgr2=0x7f35480cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:11.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.944+0000 7f354fe00640 1 --2- 172.21.15.100:0/717791147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35480d0ee0 0x7f35480cef70 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3534009830 tx=0x7f353402a520 comp rx=0 tx=0).stop 2024-09-17T04:34:11.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.944+0000 7f354fe00640 1 -- 172.21.15.100:0/717791147 shutdown_connections 2024-09-17T04:34:11.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.944+0000 7f354fe00640 1 --2- 172.21.15.100:0/717791147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35480b6b80 0x7f35480cf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:11.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.944+0000 7f354fe00640 1 --2- 172.21.15.100:0/717791147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35480d0ee0 0x7f35480cef70 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:11.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.944+0000 7f354fe00640 1 -- 172.21.15.100:0/717791147 >> 172.21.15.100:0/717791147 conn(0x7f35480b2ff0 msgr2=0x7f35480b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:11.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.944+0000 7f354fe00640 1 -- 172.21.15.100:0/717791147 shutdown_connections 2024-09-17T04:34:11.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.944+0000 7f354fe00640 1 -- 172.21.15.100:0/717791147 wait complete. 2024-09-17T04:34:11.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.946+0000 7f354fe00640 1 Processor -- start 2024-09-17T04:34:11.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.946+0000 7f354fe00640 1 -- start start 2024-09-17T04:34:11.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.948+0000 7f354fe00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35480b6b80 0x7f354815f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:11.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.948+0000 7f354fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35480d0ee0 0x7f35480afe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:11.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.948+0000 7f354fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f354815ee80 0x7f35480b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:11.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.948+0000 7f354fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f35480c7fc0 con 0x7f354815ee80 2024-09-17T04:34:11.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.948+0000 7f354fe00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f35480c7e40 con 0x7f35480d0ee0 2024-09-17T04:34:11.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.948+0000 7f354fe00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f35480c8140 con 0x7f35480b6b80 2024-09-17T04:34:11.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.948+0000 7f354de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f354815ee80 0x7f35480b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:11.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.948+0000 7f354de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f354815ee80 0x7f35480b0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43478/0 (socket says 172.21.15.100:43478) 2024-09-17T04:34:11.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.948+0000 7f354de00640 1 -- 172.21.15.100:0/2356282547 learned_addr learned my addr 172.21.15.100:0/2356282547 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:11.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.948+0000 7f354ca00640 1 --2- 172.21.15.100:0/2356282547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35480d0ee0 0x7f35480afe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:11.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.948+0000 7f354d400640 1 --2- 172.21.15.100:0/2356282547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35480b6b80 0x7f354815f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:11.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.948+0000 7f354de00640 1 -- 172.21.15.100:0/2356282547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35480b6b80 msgr2=0x7f354815f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:11.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.949+0000 7f354de00640 1 --2- 172.21.15.100:0/2356282547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35480b6b80 0x7f354815f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:11.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.949+0000 7f354de00640 1 -- 172.21.15.100:0/2356282547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35480d0ee0 msgr2=0x7f35480afe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:11.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.949+0000 7f354de00640 1 --2- 172.21.15.100:0/2356282547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35480d0ee0 0x7f35480afe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:11.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.949+0000 7f354de00640 1 -- 172.21.15.100:0/2356282547 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35480b03a0 con 0x7f354815ee80 2024-09-17T04:34:11.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.949+0000 7f354d400640 1 --2- 172.21.15.100:0/2356282547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35480b6b80 0x7f354815f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:11.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.949+0000 7f354ca00640 1 --2- 172.21.15.100:0/2356282547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35480d0ee0 0x7f35480afe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:11.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.949+0000 7f354de00640 1 --2- 172.21.15.100:0/2356282547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f354815ee80 0x7f35480b0f70 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f354000ea10 tx=0x7f354000eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:11.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.949+0000 7f3545600640 1 -- 172.21.15.100:0/2356282547 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354000cde0 con 0x7f354815ee80 2024-09-17T04:34:11.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.949+0000 7f3545600640 1 -- 172.21.15.100:0/2356282547 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3540004c10 con 0x7f354815ee80 2024-09-17T04:34:11.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.949+0000 7f3545600640 1 -- 172.21.15.100:0/2356282547 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3540005020 con 0x7f354815ee80 2024-09-17T04:34:11.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.949+0000 7f354fe00640 1 -- 172.21.15.100:0/2356282547 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35480ae450 con 0x7f354815ee80 2024-09-17T04:34:11.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.949+0000 7f354fe00640 1 -- 172.21.15.100:0/2356282547 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f354814d0f0 con 0x7f354815ee80 2024-09-17T04:34:11.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.950+0000 7f354fe00640 1 -- 172.21.15.100:0/2356282547 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35480cb6b0 con 0x7f354815ee80 2024-09-17T04:34:11.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.951+0000 7f3545600640 1 -- 172.21.15.100:0/2356282547 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7f35400051c0 con 0x7f354815ee80 2024-09-17T04:34:11.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.952+0000 7f3545600640 1 --2- 172.21.15.100:0/2356282547 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3528079760 0x7f352807bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:11.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.952+0000 7f3545600640 1 -- 172.21.15.100:0/2356282547 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5851+0+0 (secure 0 0 0) 0x7f354009a480 con 0x7f354815ee80 2024-09-17T04:34:11.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.953+0000 7f354d400640 1 --2- 172.21.15.100:0/2356282547 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3528079760 0x7f352807bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:11.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.953+0000 7f354d400640 1 --2- 172.21.15.100:0/2356282547 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3528079760 0x7f352807bc70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f353c007a50 tx=0x7f353c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:11.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:11.957+0000 7f3545600640 1 -- 172.21.15.100:0/2356282547 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f35480cb6b0 con 0x7f354815ee80 2024-09-17T04:34:12.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:12.176+0000 7f354fe00640 1 -- 172.21.15.100:0/2356282547 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f3548076300 con 0x7f354815ee80 2024-09-17T04:34:12.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:12.803+0000 7f3545600640 1 -- 172.21.15.100:0/2356282547 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v124) ==== 104+0+0 (secure 0 0 0) 0x7f3548076300 con 0x7f354815ee80 2024-09-17T04:34:12.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:12.806+0000 7f354fe00640 1 -- 172.21.15.100:0/2356282547 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3528079760 msgr2=0x7f352807bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:12.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:12.806+0000 7f354fe00640 1 --2- 172.21.15.100:0/2356282547 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3528079760 0x7f352807bc70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f353c007a50 tx=0x7f353c009040 comp rx=0 tx=0).stop 2024-09-17T04:34:12.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:12.806+0000 7f354fe00640 1 -- 172.21.15.100:0/2356282547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f354815ee80 msgr2=0x7f35480b0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:12.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:12.806+0000 7f354fe00640 1 --2- 172.21.15.100:0/2356282547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f354815ee80 0x7f35480b0f70 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f354000ea10 tx=0x7f354000eed0 comp rx=0 tx=0).stop 2024-09-17T04:34:12.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:12.806+0000 7f354fe00640 1 -- 172.21.15.100:0/2356282547 shutdown_connections 2024-09-17T04:34:12.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:12.806+0000 7f354fe00640 1 --2- 172.21.15.100:0/2356282547 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f3528079760 0x7f352807bc70 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:12.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:12.806+0000 7f354fe00640 1 --2- 172.21.15.100:0/2356282547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f354815ee80 0x7f35480b0f70 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:12.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:12.806+0000 7f354fe00640 1 --2- 172.21.15.100:0/2356282547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35480d0ee0 0x7f35480afe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:12.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:12.806+0000 7f354fe00640 1 --2- 172.21.15.100:0/2356282547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35480b6b80 0x7f354815f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:12.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:12.806+0000 7f354fe00640 1 -- 172.21.15.100:0/2356282547 >> 172.21.15.100:0/2356282547 conn(0x7f35480b2ff0 msgr2=0x7f35480d1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:12.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:12.806+0000 7f354fe00640 1 -- 172.21.15.100:0/2356282547 shutdown_connections 2024-09-17T04:34:12.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:12.806+0000 7f354fe00640 1 -- 172.21.15.100:0/2356282547 wait complete. 2024-09-17T04:34:12.833 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T04:34:13.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.059+0000 7f8cd7200640 1 -- 172.21.15.100:0/3798934419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cd006cda0 msgr2=0x7f8cd006d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:13.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.059+0000 7f8cd7200640 1 --2- 172.21.15.100:0/3798934419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cd006cda0 0x7f8cd006d1b0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f8cd006d8a0 tx=0x7f8cb802a580 comp rx=0 tx=0).stop 2024-09-17T04:34:13.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.060+0000 7f8cd7200640 1 -- 172.21.15.100:0/3798934419 shutdown_connections 2024-09-17T04:34:13.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.060+0000 7f8cd7200640 1 --2- 172.21.15.100:0/3798934419 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8cd00af100 0x7f8cd00af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:13.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.060+0000 7f8cd7200640 1 --2- 172.21.15.100:0/3798934419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cd006cda0 0x7f8cd006d1b0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:13.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.060+0000 7f8cd7200640 1 --2- 172.21.15.100:0/3798934419 >> v2:172.21.15.184:3300/0 conn(0x7f8cd00cc830 0x7f8cd00b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:13.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.060+0000 7f8cd7200640 1 -- 172.21.15.100:0/3798934419 >> 172.21.15.100:0/3798934419 conn(0x7f8cd00b2540 msgr2=0x7f8cd00d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:13.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.060+0000 7f8cd7200640 1 -- 172.21.15.100:0/3798934419 shutdown_connections 2024-09-17T04:34:13.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.060+0000 7f8cd7200640 1 -- 172.21.15.100:0/3798934419 wait complete. 2024-09-17T04:34:13.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.062+0000 7f8cd7200640 1 Processor -- start 2024-09-17T04:34:13.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.062+0000 7f8cd7200640 1 -- start start 2024-09-17T04:34:13.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.063+0000 7f8cd7200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8cd006cda0 0x7f8cd01f4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:13.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.063+0000 7f8cd7200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cd00af100 0x7f8cd00c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:13.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.063+0000 7f8cd7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cd00cc830 0x7f8cd01f6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:13.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.063+0000 7f8cd7200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd00b34b0 con 0x7f8cd00cc830 2024-09-17T04:34:13.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.063+0000 7f8cd7200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd00b3330 con 0x7f8cd00af100 2024-09-17T04:34:13.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.063+0000 7f8cd7200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8cd00b3630 con 0x7f8cd006cda0 2024-09-17T04:34:13.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.064+0000 7f8cd5200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cd00cc830 0x7f8cd01f6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:13.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.064+0000 7f8cd5200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cd00cc830 0x7f8cd01f6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43492/0 (socket says 172.21.15.100:43492) 2024-09-17T04:34:13.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.064+0000 7f8cd5200640 1 -- 172.21.15.100:0/3005361262 learned_addr learned my addr 172.21.15.100:0/3005361262 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:13.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.064+0000 7f8cc7e00640 1 --2- 172.21.15.100:0/3005361262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cd00af100 0x7f8cd00c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:13.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.064+0000 7f8ccfe00640 1 --2- 172.21.15.100:0/3005361262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8cd006cda0 0x7f8cd01f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:13.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.064+0000 7f8cd5200640 1 -- 172.21.15.100:0/3005361262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8cd006cda0 msgr2=0x7f8cd01f4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:13.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.064+0000 7f8cd5200640 1 --2- 172.21.15.100:0/3005361262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8cd006cda0 0x7f8cd01f4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:13.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.064+0000 7f8cd5200640 1 -- 172.21.15.100:0/3005361262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cd00af100 msgr2=0x7f8cd00c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:13.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.064+0000 7f8cd5200640 1 --2- 172.21.15.100:0/3005361262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cd00af100 0x7f8cd00c9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:13.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.064+0000 7f8cd5200640 1 -- 172.21.15.100:0/3005361262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd0074780 con 0x7f8cd00cc830 2024-09-17T04:34:13.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.064+0000 7f8ccfe00640 1 --2- 172.21.15.100:0/3005361262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8cd006cda0 0x7f8cd01f4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:13.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.065+0000 7f8cd5200640 1 --2- 172.21.15.100:0/3005361262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cd00cc830 0x7f8cd01f6bc0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f8cc800d9b0 tx=0x7f8cc800de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:13.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.065+0000 7f8cccc00640 1 -- 172.21.15.100:0/3005361262 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8016070 con 0x7f8cd00cc830 2024-09-17T04:34:13.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.065+0000 7f8cccc00640 1 -- 172.21.15.100:0/3005361262 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8cc800bda0 con 0x7f8cd00cc830 2024-09-17T04:34:13.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.065+0000 7f8cd7200640 1 -- 172.21.15.100:0/3005361262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd01f4540 con 0x7f8cd00cc830 2024-09-17T04:34:13.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.065+0000 7f8cccc00640 1 -- 172.21.15.100:0/3005361262 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8003bd0 con 0x7f8cd00cc830 2024-09-17T04:34:13.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.065+0000 7f8cd7200640 1 -- 172.21.15.100:0/3005361262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd0074d00 con 0x7f8cd00cc830 2024-09-17T04:34:13.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.066+0000 7f8cd7200640 1 -- 172.21.15.100:0/3005361262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cd00af510 con 0x7f8cd00cc830 2024-09-17T04:34:13.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.067+0000 7f8cccc00640 1 -- 172.21.15.100:0/3005361262 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7f8cc8010460 con 0x7f8cd00cc830 2024-09-17T04:34:13.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.068+0000 7f8cccc00640 1 --2- 172.21.15.100:0/3005361262 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8cac079790 0x7f8cac07bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:13.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.068+0000 7f8cccc00640 1 -- 172.21.15.100:0/3005361262 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5851+0+0 (secure 0 0 0) 0x7f8cc8099c40 con 0x7f8cd00cc830 2024-09-17T04:34:13.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.068+0000 7f8ccfe00640 1 --2- 172.21.15.100:0/3005361262 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8cac079790 0x7f8cac07bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:13.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.069+0000 7f8ccfe00640 1 --2- 172.21.15.100:0/3005361262 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8cac079790 0x7f8cac07bca0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0009ab0 tx=0x7f8cc0005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:13.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.072+0000 7f8cccc00640 1 -- 172.21.15.100:0/3005361262 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8cd0074780 con 0x7f8cd00cc830 2024-09-17T04:34:13.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.255+0000 7f8cd7200640 1 -- 172.21.15.100:0/3005361262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8cd00d1030 con 0x7f8cd00cc830 2024-09-17T04:34:13.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.817+0000 7f8cccc00640 1 -- 172.21.15.100:0/3005361262 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v169) ==== 181+0+0 (secure 0 0 0) 0x7f8cc8063bc0 con 0x7f8cd00cc830 2024-09-17T04:34:13.817 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' removed 2024-09-17T04:34:13.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.820+0000 7f8cd7200640 1 -- 172.21.15.100:0/3005361262 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8cac079790 msgr2=0x7f8cac07bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:13.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.820+0000 7f8cd7200640 1 --2- 172.21.15.100:0/3005361262 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8cac079790 0x7f8cac07bca0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0009ab0 tx=0x7f8cc0005f90 comp rx=0 tx=0).stop 2024-09-17T04:34:13.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.820+0000 7f8cd7200640 1 -- 172.21.15.100:0/3005361262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cd00cc830 msgr2=0x7f8cd01f6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:13.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.820+0000 7f8cd7200640 1 --2- 172.21.15.100:0/3005361262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cd00cc830 0x7f8cd01f6bc0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f8cc800d9b0 tx=0x7f8cc800de70 comp rx=0 tx=0).stop 2024-09-17T04:34:13.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.821+0000 7f8cd7200640 1 -- 172.21.15.100:0/3005361262 shutdown_connections 2024-09-17T04:34:13.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.821+0000 7f8cd7200640 1 --2- 172.21.15.100:0/3005361262 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8cac079790 0x7f8cac07bca0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:13.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.821+0000 7f8cd7200640 1 --2- 172.21.15.100:0/3005361262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cd00cc830 0x7f8cd01f6bc0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:13.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.821+0000 7f8cd7200640 1 --2- 172.21.15.100:0/3005361262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cd00af100 0x7f8cd00c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:13.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.821+0000 7f8cd7200640 1 --2- 172.21.15.100:0/3005361262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8cd006cda0 0x7f8cd01f4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:13.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.821+0000 7f8cd7200640 1 -- 172.21.15.100:0/3005361262 >> 172.21.15.100:0/3005361262 conn(0x7f8cd00b2540 msgr2=0x7f8cd00b0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:13.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.821+0000 7f8cd7200640 1 -- 172.21.15.100:0/3005361262 shutdown_connections 2024-09-17T04:34:13.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:13.821+0000 7f8cd7200640 1 -- 172.21.15.100:0/3005361262 wait complete. 2024-09-17T04:34:13.851 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T04:34:14.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.075+0000 7f182cc00640 1 -- 172.21.15.100:0/391737577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18280af810 msgr2=0x7f18280b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:14.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.075+0000 7f182cc00640 1 --2- 172.21.15.100:0/391737577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18280af810 0x7f18280b4620 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f1820005fd0 tx=0x7f182002a940 comp rx=0 tx=0).stop 2024-09-17T04:34:14.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.075+0000 7f182cc00640 1 -- 172.21.15.100:0/391737577 shutdown_connections 2024-09-17T04:34:14.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.075+0000 7f182cc00640 1 --2- 172.21.15.100:0/391737577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18280af810 0x7f18280b4620 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:14.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.075+0000 7f182cc00640 1 -- 172.21.15.100:0/391737577 >> 172.21.15.100:0/391737577 conn(0x7f18280b2540 msgr2=0x7f18280d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:14.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.075+0000 7f182cc00640 1 -- 172.21.15.100:0/391737577 shutdown_connections 2024-09-17T04:34:14.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.075+0000 7f182cc00640 1 -- 172.21.15.100:0/391737577 wait complete. 2024-09-17T04:34:14.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.077+0000 7f182cc00640 1 Processor -- start 2024-09-17T04:34:14.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.077+0000 7f182cc00640 1 -- start start 2024-09-17T04:34:14.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.079+0000 7f182cc00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18280af810 0x7f18281e7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:14.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.079+0000 7f182cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18280cc830 0x7f18280c0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:14.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.079+0000 7f182cc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18281535b0 0x7f18280c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:14.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.079+0000 7f182cc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f18280b3780 con 0x7f18280cc830 2024-09-17T04:34:14.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.079+0000 7f182cc00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f18280b3600 con 0x7f18281535b0 2024-09-17T04:34:14.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.079+0000 7f182cc00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f18280b3900 con 0x7f18280af810 2024-09-17T04:34:14.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.079+0000 7f181d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18280cc830 0x7f18280c0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:14.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.079+0000 7f181d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18280cc830 0x7f18280c0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43516/0 (socket says 172.21.15.100:43516) 2024-09-17T04:34:14.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.079+0000 7f181d400640 1 -- 172.21.15.100:0/2854824651 learned_addr learned my addr 172.21.15.100:0/2854824651 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:14.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.079+0000 7f1826800640 1 --2- 172.21.15.100:0/2854824651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18281535b0 0x7f18280c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:14.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.079+0000 7f1825e00640 1 --2- 172.21.15.100:0/2854824651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18280af810 0x7f18281e7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:14.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.079+0000 7f181d400640 1 -- 172.21.15.100:0/2854824651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18280af810 msgr2=0x7f18281e7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:14.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.079+0000 7f181d400640 1 --2- 172.21.15.100:0/2854824651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18280af810 0x7f18281e7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:14.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.079+0000 7f181d400640 1 -- 172.21.15.100:0/2854824651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18281535b0 msgr2=0x7f18280c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:14.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.079+0000 7f181d400640 1 --2- 172.21.15.100:0/2854824651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18281535b0 0x7f18280c12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:14.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.079+0000 7f181d400640 1 -- 172.21.15.100:0/2854824651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18281e7720 con 0x7f18280cc830 2024-09-17T04:34:14.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.079+0000 7f1825e00640 1 --2- 172.21.15.100:0/2854824651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18280af810 0x7f18281e7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:14.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.080+0000 7f181d400640 1 --2- 172.21.15.100:0/2854824651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18280cc830 0x7f18280c0d70 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f181000c370 tx=0x7f181000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:14.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.080+0000 7f181ea00640 1 -- 172.21.15.100:0/2854824651 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1810017070 con 0x7f18280cc830 2024-09-17T04:34:14.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.080+0000 7f181ea00640 1 -- 172.21.15.100:0/2854824651 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18100035f0 con 0x7f18280cc830 2024-09-17T04:34:14.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.080+0000 7f182cc00640 1 -- 172.21.15.100:0/2854824651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18280b18b0 con 0x7f18280cc830 2024-09-17T04:34:14.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.080+0000 7f181ea00640 1 -- 172.21.15.100:0/2854824651 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1810003a40 con 0x7f18280cc830 2024-09-17T04:34:14.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.080+0000 7f182cc00640 1 -- 172.21.15.100:0/2854824651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18281e52a0 con 0x7f18280cc830 2024-09-17T04:34:14.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.081+0000 7f182cc00640 1 -- 172.21.15.100:0/2854824651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18280ca240 con 0x7f18280cc830 2024-09-17T04:34:14.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.082+0000 7f181ea00640 1 -- 172.21.15.100:0/2854824651 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7f181001a070 con 0x7f18280cc830 2024-09-17T04:34:14.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.083+0000 7f181ea00640 1 --2- 172.21.15.100:0/2854824651 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1804079790 0x7f180407bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:14.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.083+0000 7f181ea00640 1 -- 172.21.15.100:0/2854824651 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5413+0+0 (secure 0 0 0) 0x7f1810098b50 con 0x7f18280cc830 2024-09-17T04:34:14.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.083+0000 7f1825e00640 1 --2- 172.21.15.100:0/2854824651 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1804079790 0x7f180407bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:14.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.084+0000 7f1825e00640 1 --2- 172.21.15.100:0/2854824651 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1804079790 0x7f180407bca0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1818007fd0 tx=0x7f1818009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:14.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.088+0000 7f181ea00640 1 -- 172.21.15.100:0/2854824651 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f18280ca240 con 0x7f18280cc830 2024-09-17T04:34:14.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.271+0000 7f182cc00640 1 -- 172.21.15.100:0/2854824651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f18280b3dd0 con 0x7f18280cc830 2024-09-17T04:34:14.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.825+0000 7f181ea00640 1 -- 172.21.15.100:0/2854824651 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v170) ==== 169+0+0 (secure 0 0 0) 0x7f18281e7720 con 0x7f18280cc830 2024-09-17T04:34:14.826 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' removed 2024-09-17T04:34:14.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.830+0000 7f182cc00640 1 -- 172.21.15.100:0/2854824651 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1804079790 msgr2=0x7f180407bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:14.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.830+0000 7f182cc00640 1 --2- 172.21.15.100:0/2854824651 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1804079790 0x7f180407bca0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1818007fd0 tx=0x7f1818009040 comp rx=0 tx=0).stop 2024-09-17T04:34:14.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.830+0000 7f182cc00640 1 -- 172.21.15.100:0/2854824651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18280cc830 msgr2=0x7f18280c0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:14.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.830+0000 7f182cc00640 1 --2- 172.21.15.100:0/2854824651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18280cc830 0x7f18280c0d70 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f181000c370 tx=0x7f181000c830 comp rx=0 tx=0).stop 2024-09-17T04:34:14.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.831+0000 7f182cc00640 1 -- 172.21.15.100:0/2854824651 shutdown_connections 2024-09-17T04:34:14.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.831+0000 7f182cc00640 1 --2- 172.21.15.100:0/2854824651 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1804079790 0x7f180407bca0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:14.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.831+0000 7f182cc00640 1 --2- 172.21.15.100:0/2854824651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18281535b0 0x7f18280c12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:14.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.831+0000 7f182cc00640 1 --2- 172.21.15.100:0/2854824651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18280cc830 0x7f18280c0d70 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:14.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.831+0000 7f182cc00640 1 --2- 172.21.15.100:0/2854824651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18280af810 0x7f18281e7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:14.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.831+0000 7f182cc00640 1 -- 172.21.15.100:0/2854824651 >> 172.21.15.100:0/2854824651 conn(0x7f18280b2540 msgr2=0x7f1828099890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:14.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.831+0000 7f182cc00640 1 -- 172.21.15.100:0/2854824651 shutdown_connections 2024-09-17T04:34:14.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:14.831+0000 7f182cc00640 1 -- 172.21.15.100:0/2854824651 wait complete. 2024-09-17T04:34:14.861 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T04:34:15.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.087+0000 7ff579e00640 1 -- 172.21.15.100:0/3607768883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5740cf6b0 msgr2=0x7ff5740d2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:15.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.087+0000 7ff579e00640 1 --2- 172.21.15.100:0/3607768883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5740cf6b0 0x7ff5740d2280 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7ff564009830 tx=0x7ff56402a510 comp rx=0 tx=0).stop 2024-09-17T04:34:15.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.087+0000 7ff579e00640 1 -- 172.21.15.100:0/3607768883 shutdown_connections 2024-09-17T04:34:15.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.087+0000 7ff579e00640 1 --2- 172.21.15.100:0/3607768883 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff57415d0d0 0x7ff5740d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.087+0000 7ff579e00640 1 --2- 172.21.15.100:0/3607768883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5740cf6b0 0x7ff5740d2280 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.087+0000 7ff579e00640 1 -- 172.21.15.100:0/3607768883 >> 172.21.15.100:0/3607768883 conn(0x7ff57406d1c0 msgr2=0x7ff57406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:15.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.087+0000 7ff579e00640 1 -- 172.21.15.100:0/3607768883 shutdown_connections 2024-09-17T04:34:15.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.087+0000 7ff579e00640 1 -- 172.21.15.100:0/3607768883 wait complete. 2024-09-17T04:34:15.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.089+0000 7ff579e00640 1 Processor -- start 2024-09-17T04:34:15.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.089+0000 7ff579e00640 1 -- start start 2024-09-17T04:34:15.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.091+0000 7ff579e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5740cf6b0 0x7ff5741588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:15.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.091+0000 7ff579e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5740d27c0 0x7ff5741f9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:15.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.091+0000 7ff579e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff57415d0d0 0x7ff5741f9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:15.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.091+0000 7ff579e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff5740d0b10 con 0x7ff5740cf6b0 2024-09-17T04:34:15.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.091+0000 7ff579e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff5740d0990 con 0x7ff57415d0d0 2024-09-17T04:34:15.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.091+0000 7ff579e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff5740d0c90 con 0x7ff5740d27c0 2024-09-17T04:34:15.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.091+0000 7ff578a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5740cf6b0 0x7ff5741588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:15.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.091+0000 7ff578a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5740cf6b0 0x7ff5741588f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59596/0 (socket says 172.21.15.100:59596) 2024-09-17T04:34:15.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.091+0000 7ff578a00640 1 -- 172.21.15.100:0/2495350213 learned_addr learned my addr 172.21.15.100:0/2495350213 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:15.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.092+0000 7ff56be00640 1 --2- 172.21.15.100:0/2495350213 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5740d27c0 0x7ff5741f9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:15.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.092+0000 7ff579400640 1 --2- 172.21.15.100:0/2495350213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff57415d0d0 0x7ff5741f9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:15.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.092+0000 7ff578a00640 1 -- 172.21.15.100:0/2495350213 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5740d27c0 msgr2=0x7ff5741f9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:15.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.092+0000 7ff578a00640 1 --2- 172.21.15.100:0/2495350213 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5740d27c0 0x7ff5741f9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.092+0000 7ff578a00640 1 -- 172.21.15.100:0/2495350213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff57415d0d0 msgr2=0x7ff5741f9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:15.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.092+0000 7ff578a00640 1 --2- 172.21.15.100:0/2495350213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff57415d0d0 0x7ff5741f9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.092+0000 7ff578a00640 1 -- 172.21.15.100:0/2495350213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5741f5aa0 con 0x7ff5740cf6b0 2024-09-17T04:34:15.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.092+0000 7ff56be00640 1 --2- 172.21.15.100:0/2495350213 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5740d27c0 0x7ff5741f9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:15.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.092+0000 7ff579400640 1 --2- 172.21.15.100:0/2495350213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff57415d0d0 0x7ff5741f9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:15.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.092+0000 7ff578a00640 1 --2- 172.21.15.100:0/2495350213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5740cf6b0 0x7ff5741588f0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7ff56402ab00 tx=0x7ff564033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:15.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.092+0000 7ff571600640 1 -- 172.21.15.100:0/2495350213 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff564005c90 con 0x7ff5740cf6b0 2024-09-17T04:34:15.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.092+0000 7ff571600640 1 -- 172.21.15.100:0/2495350213 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff564031e50 con 0x7ff5740cf6b0 2024-09-17T04:34:15.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.092+0000 7ff571600640 1 -- 172.21.15.100:0/2495350213 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff564005260 con 0x7ff5740cf6b0 2024-09-17T04:34:15.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.092+0000 7ff579e00640 1 -- 172.21.15.100:0/2495350213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5741f9ed0 con 0x7ff5740cf6b0 2024-09-17T04:34:15.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.092+0000 7ff579e00640 1 -- 172.21.15.100:0/2495350213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5740b5540 con 0x7ff5740cf6b0 2024-09-17T04:34:15.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.093+0000 7ff579e00640 1 -- 172.21.15.100:0/2495350213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5740d2280 con 0x7ff5740cf6b0 2024-09-17T04:34:15.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.095+0000 7ff571600640 1 -- 172.21.15.100:0/2495350213 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7ff564005400 con 0x7ff5740cf6b0 2024-09-17T04:34:15.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.095+0000 7ff571600640 1 --2- 172.21.15.100:0/2495350213 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff550079760 0x7ff55007bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:15.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.096+0000 7ff571600640 1 -- 172.21.15.100:0/2495350213 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4999+0+0 (secure 0 0 0) 0x7ff5640baa50 con 0x7ff5740cf6b0 2024-09-17T04:34:15.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.096+0000 7ff56be00640 1 --2- 172.21.15.100:0/2495350213 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff550079760 0x7ff55007bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:15.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.096+0000 7ff56be00640 1 --2- 172.21.15.100:0/2495350213 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff550079760 0x7ff55007bc70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0013f0 tx=0x7ff55c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:15.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.100+0000 7ff571600640 1 -- 172.21.15.100:0/2495350213 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff56403f030 con 0x7ff5740cf6b0 2024-09-17T04:34:15.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.285+0000 7ff579e00640 1 -- 172.21.15.100:0/2495350213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7ff57406eec0 con 0x7ff5740cf6b0 2024-09-17T04:34:15.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.290+0000 7ff571600640 1 -- 172.21.15.100:0/2495350213 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v53) ==== 71+0+0 (secure 0 0 0) 0x7ff564084660 con 0x7ff5740cf6b0 2024-09-17T04:34:15.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.292+0000 7ff579e00640 1 -- 172.21.15.100:0/2495350213 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff550079760 msgr2=0x7ff55007bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:15.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.292+0000 7ff579e00640 1 --2- 172.21.15.100:0/2495350213 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff550079760 0x7ff55007bc70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0013f0 tx=0x7ff55c009040 comp rx=0 tx=0).stop 2024-09-17T04:34:15.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.292+0000 7ff579e00640 1 -- 172.21.15.100:0/2495350213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5740cf6b0 msgr2=0x7ff5741588f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:15.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.292+0000 7ff579e00640 1 --2- 172.21.15.100:0/2495350213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5740cf6b0 0x7ff5741588f0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7ff56402ab00 tx=0x7ff564033c20 comp rx=0 tx=0).stop 2024-09-17T04:34:15.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.292+0000 7ff579e00640 1 -- 172.21.15.100:0/2495350213 shutdown_connections 2024-09-17T04:34:15.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.292+0000 7ff579e00640 1 --2- 172.21.15.100:0/2495350213 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff550079760 0x7ff55007bc70 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.292+0000 7ff579e00640 1 --2- 172.21.15.100:0/2495350213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff57415d0d0 0x7ff5741f9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.292+0000 7ff579e00640 1 --2- 172.21.15.100:0/2495350213 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5740d27c0 0x7ff5741f9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.292+0000 7ff579e00640 1 --2- 172.21.15.100:0/2495350213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5740cf6b0 0x7ff5741588f0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.292+0000 7ff579e00640 1 -- 172.21.15.100:0/2495350213 >> 172.21.15.100:0/2495350213 conn(0x7ff57406d1c0 msgr2=0x7ff5740cb430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:15.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.293+0000 7ff579e00640 1 -- 172.21.15.100:0/2495350213 shutdown_connections 2024-09-17T04:34:15.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.293+0000 7ff579e00640 1 -- 172.21.15.100:0/2495350213 wait complete. 2024-09-17T04:34:15.320 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade' 2024-09-17T04:34:15.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.544+0000 7fa630400640 1 -- 172.21.15.100:0/2964346983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa610003a50 msgr2=0x7fa610003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:15.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.544+0000 7fa630400640 1 --2- 172.21.15.100:0/2964346983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa610003a50 0x7fa610003ec0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00c3f0 tx=0x7fa610005830 comp rx=0 tx=0).stop 2024-09-17T04:34:15.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.544+0000 7fa630400640 1 -- 172.21.15.100:0/2964346983 shutdown_connections 2024-09-17T04:34:15.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.544+0000 7fa630400640 1 --2- 172.21.15.100:0/2964346983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa610003a50 0x7fa610003ec0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.544+0000 7fa630400640 1 -- 172.21.15.100:0/2964346983 >> 172.21.15.100:0/2964346983 conn(0x7fa62806d730 msgr2=0x7fa6280b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:15.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.544+0000 7fa630400640 1 -- 172.21.15.100:0/2964346983 shutdown_connections 2024-09-17T04:34:15.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.544+0000 7fa630400640 1 -- 172.21.15.100:0/2964346983 wait complete. 2024-09-17T04:34:15.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.546+0000 7fa630400640 1 Processor -- start 2024-09-17T04:34:15.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.547+0000 7fa630400640 1 -- start start 2024-09-17T04:34:15.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.548+0000 7fa630400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa610003a50 0x7fa628151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:15.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.548+0000 7fa630400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa62809af50 0x7fa628149320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:15.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.548+0000 7fa630400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6281537d0 0x7fa628153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:15.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.548+0000 7fa630400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa62806f260 con 0x7fa6281537d0 2024-09-17T04:34:15.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.548+0000 7fa630400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa6280b4e30 con 0x7fa62809af50 2024-09-17T04:34:15.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.548+0000 7fa630400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa6280ad4f0 con 0x7fa610003a50 2024-09-17T04:34:15.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.548+0000 7fa62e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6281537d0 0x7fa628153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:15.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.548+0000 7fa62d000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa62809af50 0x7fa628149320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:15.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.548+0000 7fa62e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6281537d0 0x7fa628153be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59606/0 (socket says 172.21.15.100:59606) 2024-09-17T04:34:15.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.548+0000 7fa62e400640 1 -- 172.21.15.100:0/306547092 learned_addr learned my addr 172.21.15.100:0/306547092 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:15.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.548+0000 7fa62da00640 1 --2- 172.21.15.100:0/306547092 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa610003a50 0x7fa628151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:15.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.549+0000 7fa62e400640 1 -- 172.21.15.100:0/306547092 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa610003a50 msgr2=0x7fa628151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:15.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.549+0000 7fa62e400640 1 --2- 172.21.15.100:0/306547092 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa610003a50 0x7fa628151a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.549+0000 7fa62e400640 1 -- 172.21.15.100:0/306547092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa62809af50 msgr2=0x7fa628149320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:15.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.549+0000 7fa62e400640 1 --2- 172.21.15.100:0/306547092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa62809af50 0x7fa628149320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.549+0000 7fa62e400640 1 -- 172.21.15.100:0/306547092 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62809dfb0 con 0x7fa6281537d0 2024-09-17T04:34:15.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.549+0000 7fa62e400640 1 --2- 172.21.15.100:0/306547092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6281537d0 0x7fa628153be0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fa6280afd20 tx=0x7fa6200019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:15.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.549+0000 7fa625600640 1 -- 172.21.15.100:0/306547092 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa620011870 con 0x7fa6281537d0 2024-09-17T04:34:15.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.549+0000 7fa625600640 1 -- 172.21.15.100:0/306547092 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa620017070 con 0x7fa6281537d0 2024-09-17T04:34:15.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.549+0000 7fa630400640 1 -- 172.21.15.100:0/306547092 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa62809a470 con 0x7fa6281537d0 2024-09-17T04:34:15.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.549+0000 7fa625600640 1 -- 172.21.15.100:0/306547092 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa620011da0 con 0x7fa6281537d0 2024-09-17T04:34:15.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.549+0000 7fa630400640 1 -- 172.21.15.100:0/306547092 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa62809b360 con 0x7fa6281537d0 2024-09-17T04:34:15.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.551+0000 7fa630400640 1 -- 172.21.15.100:0/306547092 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6281535c0 con 0x7fa6281537d0 2024-09-17T04:34:15.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.552+0000 7fa625600640 1 -- 172.21.15.100:0/306547092 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7fa62001c460 con 0x7fa6281537d0 2024-09-17T04:34:15.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.553+0000 7fa625600640 1 --2- 172.21.15.100:0/306547092 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa608079790 0x7fa60807bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:15.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.553+0000 7fa625600640 1 -- 172.21.15.100:0/306547092 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4999+0+0 (secure 0 0 0) 0x7fa620099ed0 con 0x7fa6281537d0 2024-09-17T04:34:15.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.553+0000 7fa62da00640 1 --2- 172.21.15.100:0/306547092 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa608079790 0x7fa60807bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:15.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.554+0000 7fa62da00640 1 --2- 172.21.15.100:0/306547092 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa608079790 0x7fa60807bca0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa61c033000 tx=0x7fa61c000fe0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:15.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.557+0000 7fa625600640 1 -- 172.21.15.100:0/306547092 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa620063bf0 con 0x7fa6281537d0 2024-09-17T04:34:15.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.810+0000 7fa630400640 1 -- 172.21.15.100:0/306547092 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7fa628156e90 con 0x7fa6281537d0 2024-09-17T04:34:15.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.828+0000 7fa625600640 1 -- 172.21.15.100:0/306547092 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]}]=0 v566) ==== 163+0+0 (secure 0 0 0) 0x7fa6200634f0 con 0x7fa6281537d0 2024-09-17T04:34:15.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.830+0000 7fa630400640 1 -- 172.21.15.100:0/306547092 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa608079790 msgr2=0x7fa60807bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:15.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.830+0000 7fa630400640 1 --2- 172.21.15.100:0/306547092 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa608079790 0x7fa60807bca0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa61c033000 tx=0x7fa61c000fe0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.830+0000 7fa630400640 1 -- 172.21.15.100:0/306547092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6281537d0 msgr2=0x7fa628153be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:15.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.830+0000 7fa630400640 1 --2- 172.21.15.100:0/306547092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6281537d0 0x7fa628153be0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fa6280afd20 tx=0x7fa6200019b0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.830+0000 7fa630400640 1 -- 172.21.15.100:0/306547092 shutdown_connections 2024-09-17T04:34:15.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.830+0000 7fa630400640 1 --2- 172.21.15.100:0/306547092 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa608079790 0x7fa60807bca0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.830+0000 7fa630400640 1 --2- 172.21.15.100:0/306547092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6281537d0 0x7fa628153be0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.830+0000 7fa630400640 1 --2- 172.21.15.100:0/306547092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa62809af50 0x7fa628149320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.830+0000 7fa630400640 1 --2- 172.21.15.100:0/306547092 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa610003a50 0x7fa628151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.830+0000 7fa630400640 1 -- 172.21.15.100:0/306547092 >> 172.21.15.100:0/306547092 conn(0x7fa62806d730 msgr2=0x7fa6281602a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:15.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.830+0000 7fa630400640 1 -- 172.21.15.100:0/306547092 shutdown_connections 2024-09-17T04:34:15.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:15.830+0000 7fa630400640 1 -- 172.21.15.100:0/306547092 wait complete. 2024-09-17T04:34:15.854 INFO:tasks.cephfs_test_runner:Validate unparseable text .meta file on legacy subvol root doesn't break the system ... ok 2024-09-17T04:34:15.854 INFO:tasks.cephfs_test_runner: 2024-09-17T04:34:15.855 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-17T04:34:15.855 INFO:tasks.cephfs_test_runner:Ran 11 tests in 568.794s 2024-09-17T04:34:15.855 INFO:tasks.cephfs_test_runner: 2024-09-17T04:34:15.855 INFO:tasks.cephfs_test_runner:OK 2024-09-17T04:34:15.855 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-17T04:34:15.869 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-17T04:34:15.938 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-17T04:34:15.938 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:34:15.938 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:34:15.973 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:34:15.974 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-17T04:34:15.974 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:34:16.049 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:34:16.050 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:34:16.050 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:34:16.090 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:34:16.090 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-17T04:34:16.090 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:34:16.168 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:34:16.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:16.168 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-17T04:34:16.204 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T04:34:16.204 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T04:34:16.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:16.205 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:34:16.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:16.302 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:34:16.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:16.567 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-09-17T04:34:16.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:16.875 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-17T04:34:16.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:16.911 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-09-17T04:34:17.011 INFO:teuthology.orchestra.run.smithi100.stderr:Cannot find device "ceph-brx" 2024-09-17T04:34:17.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T04:34:17.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:17.014 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-09-17T04:34:17.055 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T04:34:17.055 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T04:34:17.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:17.055 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T04:34:17.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:17.158 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T04:34:17.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:17.423 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link delete ceph-brx 2024-09-17T04:34:17.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:17.728 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-09-17T04:34:17.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:17.764 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link delete ceph-brx 2024-09-17T04:34:17.864 INFO:teuthology.orchestra.run.smithi184.stderr:Cannot find device "ceph-brx" 2024-09-17T04:34:17.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T04:34:17.867 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-17T04:34:17.877 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T04:34:17.877 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:34:18.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.102+0000 7f28ea200640 1 -- 172.21.15.100:0/1495130121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28e40ad0e0 msgr2=0x7f28e40ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:18.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.102+0000 7f28ea200640 1 --2- 172.21.15.100:0/1495130121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28e40ad0e0 0x7f28e40ad4f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f28d0009860 tx=0x7f28d002a510 comp rx=0 tx=0).stop 2024-09-17T04:34:18.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.102+0000 7f28ea200640 1 -- 172.21.15.100:0/1495130121 shutdown_connections 2024-09-17T04:34:18.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.102+0000 7f28ea200640 1 --2- 172.21.15.100:0/1495130121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28e40ad0e0 0x7f28e40ad4f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.102+0000 7f28ea200640 1 -- 172.21.15.100:0/1495130121 >> 172.21.15.100:0/1495130121 conn(0x7f28e406d730 msgr2=0x7f28e40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:18.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.102+0000 7f28ea200640 1 -- 172.21.15.100:0/1495130121 shutdown_connections 2024-09-17T04:34:18.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.102+0000 7f28ea200640 1 -- 172.21.15.100:0/1495130121 wait complete. 2024-09-17T04:34:18.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.104+0000 7f28ea200640 1 Processor -- start 2024-09-17T04:34:18.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.104+0000 7f28ea200640 1 -- start start 2024-09-17T04:34:18.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.106+0000 7f28ea200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e40ad0e0 0x7f28e4149ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:18.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.106+0000 7f28ea200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28e40b3550 0x7f28e4150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:18.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.106+0000 7f28ea200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28e415d760 0x7f28e415b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:18.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.106+0000 7f28ea200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f28e40b4da0 con 0x7f28e40ad0e0 2024-09-17T04:34:18.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.106+0000 7f28ea200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f28e40b4c20 con 0x7f28e415d760 2024-09-17T04:34:18.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.106+0000 7f28ea200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f28e40b4070 con 0x7f28e40b3550 2024-09-17T04:34:18.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.106+0000 7f28e3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e40ad0e0 0x7f28e4149ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:18.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.106+0000 7f28e3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e40ad0e0 0x7f28e4149ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59618/0 (socket says 172.21.15.100:59618) 2024-09-17T04:34:18.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.106+0000 7f28e3400640 1 -- 172.21.15.100:0/1148663350 learned_addr learned my addr 172.21.15.100:0/1148663350 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:18.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.106+0000 7f28e2a00640 1 --2- 172.21.15.100:0/1148663350 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28e40b3550 0x7f28e4150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:18.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.106+0000 7f28e3e00640 1 --2- 172.21.15.100:0/1148663350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28e415d760 0x7f28e415b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:18.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.106+0000 7f28e3400640 1 -- 172.21.15.100:0/1148663350 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28e40b3550 msgr2=0x7f28e4150340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:18.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.106+0000 7f28e3400640 1 --2- 172.21.15.100:0/1148663350 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28e40b3550 0x7f28e4150340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.106+0000 7f28e3400640 1 -- 172.21.15.100:0/1148663350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28e415d760 msgr2=0x7f28e415b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:18.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.106+0000 7f28e3400640 1 --2- 172.21.15.100:0/1148663350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28e415d760 0x7f28e415b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.106+0000 7f28e3400640 1 -- 172.21.15.100:0/1148663350 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e41f6df0 con 0x7f28e40ad0e0 2024-09-17T04:34:18.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.107+0000 7f28e3400640 1 --2- 172.21.15.100:0/1148663350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e40ad0e0 0x7f28e4149ab0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f28d400c730 tx=0x7f28d400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:18.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.107+0000 7f28db400640 1 -- 172.21.15.100:0/1148663350 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d4016020 con 0x7f28e40ad0e0 2024-09-17T04:34:18.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.107+0000 7f28db400640 1 -- 172.21.15.100:0/1148663350 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28d400a950 con 0x7f28e40ad0e0 2024-09-17T04:34:18.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.107+0000 7f28ea200640 1 -- 172.21.15.100:0/1148663350 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28e41f6270 con 0x7f28e40ad0e0 2024-09-17T04:34:18.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.107+0000 7f28db400640 1 -- 172.21.15.100:0/1148663350 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d400ade0 con 0x7f28e40ad0e0 2024-09-17T04:34:18.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.107+0000 7f28ea200640 1 -- 172.21.15.100:0/1148663350 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28e41f6830 con 0x7f28e40ad0e0 2024-09-17T04:34:18.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.108+0000 7f28ea200640 1 -- 172.21.15.100:0/1148663350 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28e40ad4f0 con 0x7f28e40ad0e0 2024-09-17T04:34:18.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.109+0000 7f28db400640 1 -- 172.21.15.100:0/1148663350 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7f28d4019020 con 0x7f28e40ad0e0 2024-09-17T04:34:18.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.110+0000 7f28db400640 1 --2- 172.21.15.100:0/1148663350 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f28c4079760 0x7f28c407bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:18.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.110+0000 7f28db400640 1 -- 172.21.15.100:0/1148663350 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4999+0+0 (secure 0 0 0) 0x7f28d40998b0 con 0x7f28e40ad0e0 2024-09-17T04:34:18.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.110+0000 7f28e2a00640 1 --2- 172.21.15.100:0/1148663350 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f28c4079760 0x7f28c407bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:18.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.111+0000 7f28e2a00640 1 --2- 172.21.15.100:0/1148663350 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f28c4079760 0x7f28c407bc70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f28d0034000 tx=0x7f28d0004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:18.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.115+0000 7f28db400640 1 -- 172.21.15.100:0/1148663350 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f28d4020030 con 0x7f28e40ad0e0 2024-09-17T04:34:18.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.295+0000 7f28ea200640 1 -- 172.21.15.100:0/1148663350 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f28e41f5e00 con 0x7f28c4079760 2024-09-17T04:34:18.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.296+0000 7f28db400640 1 -- 172.21.15.100:0/1148663350 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13498 (secure 0 0 0) 0x7f28e41f5e00 con 0x7f28c4079760 2024-09-17T04:34:18.297 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:34:18.297 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:34:18.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.298+0000 7f28ea200640 1 -- 172.21.15.100:0/1148663350 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f28c4079760 msgr2=0x7f28c407bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:18.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.298+0000 7f28ea200640 1 --2- 172.21.15.100:0/1148663350 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f28c4079760 0x7f28c407bc70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f28d0034000 tx=0x7f28d0004730 comp rx=0 tx=0).stop 2024-09-17T04:34:18.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.298+0000 7f28ea200640 1 -- 172.21.15.100:0/1148663350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e40ad0e0 msgr2=0x7f28e4149ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:18.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.298+0000 7f28ea200640 1 --2- 172.21.15.100:0/1148663350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e40ad0e0 0x7f28e4149ab0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f28d400c730 tx=0x7f28d400cbf0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.298+0000 7f28ea200640 1 -- 172.21.15.100:0/1148663350 shutdown_connections 2024-09-17T04:34:18.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.298+0000 7f28ea200640 1 --2- 172.21.15.100:0/1148663350 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f28c4079760 0x7f28c407bc70 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.298+0000 7f28ea200640 1 --2- 172.21.15.100:0/1148663350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28e415d760 0x7f28e415b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.298+0000 7f28ea200640 1 --2- 172.21.15.100:0/1148663350 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28e40b3550 0x7f28e4150340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.298+0000 7f28ea200640 1 --2- 172.21.15.100:0/1148663350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e40ad0e0 0x7f28e4149ab0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.298+0000 7f28ea200640 1 -- 172.21.15.100:0/1148663350 >> 172.21.15.100:0/1148663350 conn(0x7f28e406d730 msgr2=0x7f28e41535c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:18.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.299+0000 7f28ea200640 1 -- 172.21.15.100:0/1148663350 shutdown_connections 2024-09-17T04:34:18.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.299+0000 7f28ea200640 1 -- 172.21.15.100:0/1148663350 wait complete. 2024-09-17T04:34:18.322 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2024-09-17T04:34:17.730073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":275736,"kb_used_data":17896,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754698984,"statfs":{"total":773094113280,"available":772811759616,"internally_reserved":0,"allocated":18325504,"data_stored":8738648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":263835188},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52459,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-5,"num_read_kb":-5,"num_write":-72,"num_write_kb":-80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.036950"},"pg_stats":[{"pgid":"1.0","version":"76'79","reported_seq":411,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-17T04:34:15.149713+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:34:15.149713+0000","last_peered":"2024-09-17T04:34:15.149713+0000","last_clean":"2024-09-17T04:34:15.149713+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:34:15.149713+0000","last_undegraded":"2024-09-17T04:34:15.149713+0000","last_fullsized":"2024-09-17T04:34:15.149713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33864,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337976,"statfs":{"total":96636764160,"available":96602087424,"internally_reserved":0,"allocated":2736128,"data_stored":1535107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33288,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476864,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33280,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476864,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33280,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542271,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37440,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334400,"statfs":{"total":96636764160,"available":96598425600,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509567,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33280,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509566,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33284,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338556,"statfs":{"total":96636764160,"available":96602681344,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509566,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38020,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333820,"statfs":{"total":96636764160,"available":96597831680,"internally_reserved":0,"allocated":2736128,"data_stored":1535107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:34:18.323 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:34:18.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.542+0000 7fc255200640 1 -- 172.21.15.100:0/185090559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2500b3550 msgr2=0x7fc2500b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:18.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.542+0000 7fc255200640 1 --2- 172.21.15.100:0/185090559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2500b3550 0x7fc2500b3960 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fc248005fd0 tx=0x7fc24802a970 comp rx=0 tx=0).stop 2024-09-17T04:34:18.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.542+0000 7fc255200640 1 -- 172.21.15.100:0/185090559 shutdown_connections 2024-09-17T04:34:18.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.543+0000 7fc255200640 1 --2- 172.21.15.100:0/185090559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2500b3550 0x7fc2500b3960 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.543+0000 7fc255200640 1 --2- 172.21.15.100:0/185090559 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc25015d760 0x7fc2501535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.543+0000 7fc255200640 1 -- 172.21.15.100:0/185090559 >> 172.21.15.100:0/185090559 conn(0x7fc25006d730 msgr2=0x7fc2500b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:18.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.543+0000 7fc255200640 1 -- 172.21.15.100:0/185090559 shutdown_connections 2024-09-17T04:34:18.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.543+0000 7fc255200640 1 -- 172.21.15.100:0/185090559 wait complete. 2024-09-17T04:34:18.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.545+0000 7fc255200640 1 Processor -- start 2024-09-17T04:34:18.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.545+0000 7fc255200640 1 -- start start 2024-09-17T04:34:18.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.546+0000 7fc255200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2500ad0e0 0x7fc250148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:18.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.546+0000 7fc255200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2500b3550 0x7fc250158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:18.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.546+0000 7fc255200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc25015d760 0x7fc2501511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:18.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.546+0000 7fc255200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc2500b4da0 con 0x7fc25015d760 2024-09-17T04:34:18.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.546+0000 7fc255200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc2500b4c20 con 0x7fc2500ad0e0 2024-09-17T04:34:18.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.547+0000 7fc255200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc2500b3ea0 con 0x7fc2500b3550 2024-09-17T04:34:18.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.547+0000 7fc24e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc25015d760 0x7fc2501511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:18.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.547+0000 7fc24de00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2500ad0e0 0x7fc250148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:18.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.547+0000 7fc24e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc25015d760 0x7fc2501511a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59636/0 (socket says 172.21.15.100:59636) 2024-09-17T04:34:18.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.547+0000 7fc24e800640 1 -- 172.21.15.100:0/4106848997 learned_addr learned my addr 172.21.15.100:0/4106848997 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:18.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.547+0000 7fc24d400640 1 --2- 172.21.15.100:0/4106848997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2500b3550 0x7fc250158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:18.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.547+0000 7fc24e800640 1 -- 172.21.15.100:0/4106848997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2500b3550 msgr2=0x7fc250158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:18.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.547+0000 7fc24e800640 1 --2- 172.21.15.100:0/4106848997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2500b3550 0x7fc250158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.547+0000 7fc24e800640 1 -- 172.21.15.100:0/4106848997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2500ad0e0 msgr2=0x7fc250148aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:18.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.547+0000 7fc24e800640 1 --2- 172.21.15.100:0/4106848997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2500ad0e0 0x7fc250148aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.547+0000 7fc24e800640 1 -- 172.21.15.100:0/4106848997 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc250157580 con 0x7fc25015d760 2024-09-17T04:34:18.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.548+0000 7fc24e800640 1 --2- 172.21.15.100:0/4106848997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc25015d760 0x7fc2501511a0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fc248033040 tx=0x7fc248003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:18.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.548+0000 7fc246000640 1 -- 172.21.15.100:0/4106848997 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc248034040 con 0x7fc25015d760 2024-09-17T04:34:18.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.548+0000 7fc246000640 1 -- 172.21.15.100:0/4106848997 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc248004540 con 0x7fc25015d760 2024-09-17T04:34:18.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.548+0000 7fc255200640 1 -- 172.21.15.100:0/4106848997 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc250157860 con 0x7fc25015d760 2024-09-17T04:34:18.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.548+0000 7fc246000640 1 -- 172.21.15.100:0/4106848997 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc248045480 con 0x7fc25015d760 2024-09-17T04:34:18.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.548+0000 7fc255200640 1 -- 172.21.15.100:0/4106848997 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc250157e20 con 0x7fc25015d760 2024-09-17T04:34:18.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.549+0000 7fc255200640 1 -- 172.21.15.100:0/4106848997 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2500b4350 con 0x7fc25015d760 2024-09-17T04:34:18.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.550+0000 7fc246000640 1 -- 172.21.15.100:0/4106848997 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7fc248037020 con 0x7fc25015d760 2024-09-17T04:34:18.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.551+0000 7fc246000640 1 --2- 172.21.15.100:0/4106848997 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc230079760 0x7fc23007bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:18.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.551+0000 7fc246000640 1 -- 172.21.15.100:0/4106848997 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4999+0+0 (secure 0 0 0) 0x7fc2480bae70 con 0x7fc25015d760 2024-09-17T04:34:18.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.552+0000 7fc24de00640 1 --2- 172.21.15.100:0/4106848997 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc230079760 0x7fc23007bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:18.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.552+0000 7fc24de00640 1 --2- 172.21.15.100:0/4106848997 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc230079760 0x7fc23007bc70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc24000bb50 tx=0x7fc24000b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:18.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.555+0000 7fc246000640 1 -- 172.21.15.100:0/4106848997 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc2480846b0 con 0x7fc25015d760 2024-09-17T04:34:18.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.735+0000 7fc255200640 1 -- 172.21.15.100:0/4106848997 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc250148fe0 con 0x7fc230079760 2024-09-17T04:34:18.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.736+0000 7fc246000640 1 -- 172.21.15.100:0/4106848997 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13498 (secure 0 0 0) 0x7fc250148fe0 con 0x7fc230079760 2024-09-17T04:34:18.737 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:34:18.737 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:34:18.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.738+0000 7fc255200640 1 -- 172.21.15.100:0/4106848997 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc230079760 msgr2=0x7fc23007bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:18.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.738+0000 7fc255200640 1 --2- 172.21.15.100:0/4106848997 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc230079760 0x7fc23007bc70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc24000bb50 tx=0x7fc24000b2e0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.738+0000 7fc255200640 1 -- 172.21.15.100:0/4106848997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc25015d760 msgr2=0x7fc2501511a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:18.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.738+0000 7fc255200640 1 --2- 172.21.15.100:0/4106848997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc25015d760 0x7fc2501511a0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fc248033040 tx=0x7fc248003f10 comp rx=0 tx=0).stop 2024-09-17T04:34:18.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.738+0000 7fc255200640 1 -- 172.21.15.100:0/4106848997 shutdown_connections 2024-09-17T04:34:18.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.738+0000 7fc255200640 1 --2- 172.21.15.100:0/4106848997 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fc230079760 0x7fc23007bc70 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.738+0000 7fc255200640 1 --2- 172.21.15.100:0/4106848997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc25015d760 0x7fc2501511a0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.738+0000 7fc255200640 1 --2- 172.21.15.100:0/4106848997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2500b3550 0x7fc250158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.738+0000 7fc255200640 1 --2- 172.21.15.100:0/4106848997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2500ad0e0 0x7fc250148aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.738+0000 7fc255200640 1 -- 172.21.15.100:0/4106848997 >> 172.21.15.100:0/4106848997 conn(0x7fc25006d730 msgr2=0x7fc2500cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:18.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.739+0000 7fc255200640 1 -- 172.21.15.100:0/4106848997 shutdown_connections 2024-09-17T04:34:18.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.739+0000 7fc255200640 1 -- 172.21.15.100:0/4106848997 wait complete. 2024-09-17T04:34:18.761 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2024-09-17T04:34:17.730073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":275736,"kb_used_data":17896,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754698984,"statfs":{"total":773094113280,"available":772811759616,"internally_reserved":0,"allocated":18325504,"data_stored":8738648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":263835188},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52459,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-5,"num_read_kb":-5,"num_write":-72,"num_write_kb":-80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.036950"},"pg_stats":[{"pgid":"1.0","version":"76'79","reported_seq":411,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-17T04:34:15.149713+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:34:15.149713+0000","last_peered":"2024-09-17T04:34:15.149713+0000","last_clean":"2024-09-17T04:34:15.149713+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:34:15.149713+0000","last_undegraded":"2024-09-17T04:34:15.149713+0000","last_fullsized":"2024-09-17T04:34:15.149713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33864,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337976,"statfs":{"total":96636764160,"available":96602087424,"internally_reserved":0,"allocated":2736128,"data_stored":1535107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33288,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476864,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33280,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476864,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33280,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542271,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37440,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334400,"statfs":{"total":96636764160,"available":96598425600,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509567,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33280,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509566,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33284,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338556,"statfs":{"total":96636764160,"available":96602681344,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509566,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38020,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333820,"statfs":{"total":96636764160,"available":96597831680,"internally_reserved":0,"allocated":2736128,"data_stored":1535107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:34:18.761 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T04:34:18.762 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:34:18.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.981+0000 7f74bf200640 1 -- 172.21.15.100:0/3521790133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74c00af050 msgr2=0x7f74c00ae080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:18.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.981+0000 7f74bf200640 1 --2- 172.21.15.100:0/3521790133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74c00af050 0x7f74c00ae080 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f74b0005fd0 tx=0x7f74b002a530 comp rx=0 tx=0).stop 2024-09-17T04:34:18.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.981+0000 7f74bf200640 1 -- 172.21.15.100:0/3521790133 shutdown_connections 2024-09-17T04:34:18.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.981+0000 7f74bf200640 1 --2- 172.21.15.100:0/3521790133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74c0070710 0x7f74c0073270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.981+0000 7f74bf200640 1 --2- 172.21.15.100:0/3521790133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74c00af050 0x7f74c00ae080 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.981+0000 7f74bf200640 1 -- 172.21.15.100:0/3521790133 >> 172.21.15.100:0/3521790133 conn(0x7f74c00b1e40 msgr2=0x7f74c00b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:18.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.981+0000 7f74bf200640 1 -- 172.21.15.100:0/3521790133 shutdown_connections 2024-09-17T04:34:18.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.981+0000 7f74bf200640 1 -- 172.21.15.100:0/3521790133 wait complete. 2024-09-17T04:34:18.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.983+0000 7f74bf200640 1 Processor -- start 2024-09-17T04:34:18.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.983+0000 7f74bf200640 1 -- start start 2024-09-17T04:34:18.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.985+0000 7f74bf200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74c0070710 0x7f74c00c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:18.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.985+0000 7f74bf200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74c0072e60 0x7f74c014c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:18.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.985+0000 7f74bf200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74c00af050 0x7f74c00d0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:18.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.985+0000 7f74bf200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f74c01658a0 con 0x7f74c0070710 2024-09-17T04:34:18.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.985+0000 7f74bf200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f74c0165720 con 0x7f74c0072e60 2024-09-17T04:34:18.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.985+0000 7f74bf200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f74c0165a20 con 0x7f74c00af050 2024-09-17T04:34:18.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.985+0000 7f74bde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74c0070710 0x7f74c00c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:18.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.985+0000 7f74bde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74c0070710 0x7f74c00c9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59644/0 (socket says 172.21.15.100:59644) 2024-09-17T04:34:18.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.985+0000 7f74bde00640 1 -- 172.21.15.100:0/3372149371 learned_addr learned my addr 172.21.15.100:0/3372149371 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:18.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.985+0000 7f74b5400640 1 --2- 172.21.15.100:0/3372149371 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74c0072e60 0x7f74c014c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:18.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.985+0000 7f74be800640 1 --2- 172.21.15.100:0/3372149371 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74c00af050 0x7f74c00d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:18.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.985+0000 7f74bde00640 1 -- 172.21.15.100:0/3372149371 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74c00af050 msgr2=0x7f74c00d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:18.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.985+0000 7f74bde00640 1 --2- 172.21.15.100:0/3372149371 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74c00af050 0x7f74c00d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.985+0000 7f74bde00640 1 -- 172.21.15.100:0/3372149371 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74c0072e60 msgr2=0x7f74c014c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:18.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.985+0000 7f74bde00640 1 --2- 172.21.15.100:0/3372149371 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74c0072e60 0x7f74c014c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.986+0000 7f74bde00640 1 -- 172.21.15.100:0/3372149371 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c014c820 con 0x7f74c0070710 2024-09-17T04:34:18.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.986+0000 7f74be800640 1 --2- 172.21.15.100:0/3372149371 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74c00af050 0x7f74c00d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:18.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.986+0000 7f74bde00640 1 --2- 172.21.15.100:0/3372149371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74c0070710 0x7f74c00c9320 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f74b0034040 tx=0x7f74b00039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:18.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.986+0000 7f74b6a00640 1 -- 172.21.15.100:0/3372149371 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b003e030 con 0x7f74c0070710 2024-09-17T04:34:18.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.986+0000 7f74b6a00640 1 -- 172.21.15.100:0/3372149371 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74b00047a0 con 0x7f74c0070710 2024-09-17T04:34:18.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.986+0000 7f74bf200640 1 -- 172.21.15.100:0/3372149371 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74c01f5ad0 con 0x7f74c0070710 2024-09-17T04:34:18.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.986+0000 7f74b6a00640 1 -- 172.21.15.100:0/3372149371 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b0035040 con 0x7f74c0070710 2024-09-17T04:34:18.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.986+0000 7f74bf200640 1 -- 172.21.15.100:0/3372149371 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74c01f6080 con 0x7f74c0070710 2024-09-17T04:34:18.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.988+0000 7f74bf200640 1 -- 172.21.15.100:0/3372149371 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74c0051080 con 0x7f74c0070710 2024-09-17T04:34:18.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.988+0000 7f74b6a00640 1 -- 172.21.15.100:0/3372149371 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7f74b0041030 con 0x7f74c0070710 2024-09-17T04:34:18.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.989+0000 7f74b6a00640 1 --2- 172.21.15.100:0/3372149371 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f749c079760 0x7f749c07bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:18.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.990+0000 7f74b6a00640 1 -- 172.21.15.100:0/3372149371 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4999+0+0 (secure 0 0 0) 0x7f74b00ba170 con 0x7f74c0070710 2024-09-17T04:34:18.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.990+0000 7f74b5400640 1 --2- 172.21.15.100:0/3372149371 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f749c079760 0x7f749c07bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:18.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.990+0000 7f74b5400640 1 --2- 172.21.15.100:0/3372149371 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f749c079760 0x7f749c07bc70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f74c0150ae0 tx=0x7f74a8009060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:18.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:18.994+0000 7f74b6a00640 1 -- 172.21.15.100:0/3372149371 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f74c0051080 con 0x7f74c0070710 2024-09-17T04:34:19.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.174+0000 7f74bf200640 1 -- 172.21.15.100:0/3372149371 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f74c0073270 con 0x7f749c079760 2024-09-17T04:34:19.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.174+0000 7f74b6a00640 1 -- 172.21.15.100:0/3372149371 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13498 (secure 0 0 0) 0x7f74c0073270 con 0x7f749c079760 2024-09-17T04:34:19.175 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:34:19.175 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:34:19.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.177+0000 7f74bf200640 1 -- 172.21.15.100:0/3372149371 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f749c079760 msgr2=0x7f749c07bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:19.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.177+0000 7f74bf200640 1 --2- 172.21.15.100:0/3372149371 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f749c079760 0x7f749c07bc70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f74c0150ae0 tx=0x7f74a8009060 comp rx=0 tx=0).stop 2024-09-17T04:34:19.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.177+0000 7f74bf200640 1 -- 172.21.15.100:0/3372149371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74c0070710 msgr2=0x7f74c00c9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:19.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.177+0000 7f74bf200640 1 --2- 172.21.15.100:0/3372149371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74c0070710 0x7f74c00c9320 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f74b0034040 tx=0x7f74b00039e0 comp rx=0 tx=0).stop 2024-09-17T04:34:19.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.177+0000 7f74bf200640 1 -- 172.21.15.100:0/3372149371 shutdown_connections 2024-09-17T04:34:19.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.177+0000 7f74bf200640 1 --2- 172.21.15.100:0/3372149371 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f749c079760 0x7f749c07bc70 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:19.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.177+0000 7f74bf200640 1 --2- 172.21.15.100:0/3372149371 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74c00af050 0x7f74c00d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:19.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.177+0000 7f74bf200640 1 --2- 172.21.15.100:0/3372149371 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74c0072e60 0x7f74c014c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:19.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.177+0000 7f74bf200640 1 --2- 172.21.15.100:0/3372149371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74c0070710 0x7f74c00c9320 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:19.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.177+0000 7f74bf200640 1 -- 172.21.15.100:0/3372149371 >> 172.21.15.100:0/3372149371 conn(0x7f74c00b1e40 msgr2=0x7f74c01518d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:19.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.177+0000 7f74bf200640 1 -- 172.21.15.100:0/3372149371 shutdown_connections 2024-09-17T04:34:19.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.177+0000 7f74bf200640 1 -- 172.21.15.100:0/3372149371 wait complete. 2024-09-17T04:34:19.199 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2024-09-17T04:34:17.730073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":275736,"kb_used_data":17896,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754698984,"statfs":{"total":773094113280,"available":772811759616,"internally_reserved":0,"allocated":18325504,"data_stored":8738648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":263835188},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52459,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-5,"num_read_kb":-5,"num_write":-72,"num_write_kb":-80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.036950"},"pg_stats":[{"pgid":"1.0","version":"76'79","reported_seq":411,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-17T04:34:15.149713+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:34:15.149713+0000","last_peered":"2024-09-17T04:34:15.149713+0000","last_clean":"2024-09-17T04:34:15.149713+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:34:15.149713+0000","last_undegraded":"2024-09-17T04:34:15.149713+0000","last_fullsized":"2024-09-17T04:34:15.149713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33864,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337976,"statfs":{"total":96636764160,"available":96602087424,"internally_reserved":0,"allocated":2736128,"data_stored":1535107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33288,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476864,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33280,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476864,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33280,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542271,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37440,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334400,"statfs":{"total":96636764160,"available":96598425600,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509567,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33280,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509566,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33284,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338556,"statfs":{"total":96636764160,"available":96602681344,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509566,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38020,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333820,"statfs":{"total":96636764160,"available":96597831680,"internally_reserved":0,"allocated":2736128,"data_stored":1535107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:34:19.200 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:34:19.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.419+0000 7feee3600640 1 -- 172.21.15.100:0/3363809114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feec4004d40 msgr2=0x7feec4005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:19.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.419+0000 7feee3600640 1 --2- 172.21.15.100:0/3363809114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feec4004d40 0x7feec4005150 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7feedc0aea60 tx=0x7feec4005830 comp rx=0 tx=0).stop 2024-09-17T04:34:19.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.419+0000 7feee1600640 1 -- 172.21.15.100:0/3363809114 reap_dead start 2024-09-17T04:34:19.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.419+0000 7feee3600640 1 -- 172.21.15.100:0/3363809114 shutdown_connections 2024-09-17T04:34:19.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.419+0000 7feee3600640 1 -- 172.21.15.100:0/3363809114 >> 172.21.15.100:0/3363809114 conn(0x7feedc06d730 msgr2=0x7feedc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:19.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.420+0000 7feee3600640 1 -- 172.21.15.100:0/3363809114 shutdown_connections 2024-09-17T04:34:19.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.420+0000 7feee3600640 1 -- 172.21.15.100:0/3363809114 wait complete. 2024-09-17T04:34:19.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.422+0000 7feee3600640 1 Processor -- start 2024-09-17T04:34:19.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.422+0000 7feee3600640 1 -- start start 2024-09-17T04:34:19.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.423+0000 7feee3600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feec4004d40 0x7feedc150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:19.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.423+0000 7feee3600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feedc151680 0x7feedc06faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:19.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.423+0000 7feee3600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feedc070030 0x7feedc0743c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:19.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.423+0000 7feee3600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7feedc0b4da0 con 0x7feedc070030 2024-09-17T04:34:19.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.423+0000 7feee3600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feedc0b4c20 con 0x7feedc151680 2024-09-17T04:34:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.423+0000 7feee3600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7feedc0ad4f0 con 0x7feec4004d40 2024-09-17T04:34:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.424+0000 7feee1600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feedc070030 0x7feedc0743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.424+0000 7feee1600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feedc070030 0x7feedc0743c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59654/0 (socket says 172.21.15.100:59654) 2024-09-17T04:34:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.424+0000 7feee1600640 1 -- 172.21.15.100:0/2260182351 learned_addr learned my addr 172.21.15.100:0/2260182351 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.424+0000 7feee0c00640 1 --2- 172.21.15.100:0/2260182351 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feec4004d40 0x7feedc150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.424+0000 7feedbe00640 1 --2- 172.21.15.100:0/2260182351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feedc151680 0x7feedc06faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.424+0000 7feee1600640 1 -- 172.21.15.100:0/2260182351 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feec4004d40 msgr2=0x7feedc150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.424+0000 7feee1600640 1 --2- 172.21.15.100:0/2260182351 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feec4004d40 0x7feedc150f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.424+0000 7feee1600640 1 -- 172.21.15.100:0/2260182351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feedc151680 msgr2=0x7feedc06faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.424+0000 7feee1600640 1 --2- 172.21.15.100:0/2260182351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feedc151680 0x7feedc06faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.424+0000 7feee1600640 1 -- 172.21.15.100:0/2260182351 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feedc070440 con 0x7feedc070030 2024-09-17T04:34:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.424+0000 7feedbe00640 1 --2- 172.21.15.100:0/2260182351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feedc151680 0x7feedc06faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:19.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.424+0000 7feee1600640 1 --2- 172.21.15.100:0/2260182351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feedc070030 0x7feedc0743c0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7feed4036040 tx=0x7feed4033b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:19.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.425+0000 7feed8c00640 1 -- 172.21.15.100:0/2260182351 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed4043070 con 0x7feedc070030 2024-09-17T04:34:19.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.425+0000 7feed8c00640 1 -- 172.21.15.100:0/2260182351 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feed4033cf0 con 0x7feedc070030 2024-09-17T04:34:19.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.425+0000 7feee3600640 1 -- 172.21.15.100:0/2260182351 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feedc15a640 con 0x7feedc070030 2024-09-17T04:34:19.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.425+0000 7feed8c00640 1 -- 172.21.15.100:0/2260182351 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed4038040 con 0x7feedc070030 2024-09-17T04:34:19.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.425+0000 7feee3600640 1 -- 172.21.15.100:0/2260182351 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feedc1584c0 con 0x7feedc070030 2024-09-17T04:34:19.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.426+0000 7feee3600640 1 -- 172.21.15.100:0/2260182351 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feedc15a3b0 con 0x7feedc070030 2024-09-17T04:34:19.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.427+0000 7feed8c00640 1 -- 172.21.15.100:0/2260182351 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7feed4003bd0 con 0x7feedc070030 2024-09-17T04:34:19.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.428+0000 7feed8c00640 1 --2- 172.21.15.100:0/2260182351 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7feebc079790 0x7feebc07bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:19.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.429+0000 7feed8c00640 1 -- 172.21.15.100:0/2260182351 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4999+0+0 (secure 0 0 0) 0x7feed40b9a00 con 0x7feedc070030 2024-09-17T04:34:19.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.429+0000 7feee0c00640 1 --2- 172.21.15.100:0/2260182351 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7feebc079790 0x7feebc07bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:19.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.429+0000 7feee0c00640 1 --2- 172.21.15.100:0/2260182351 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7feebc079790 0x7feebc07bca0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7feed00013f0 tx=0x7feed0001d60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:19.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.433+0000 7feed8c00640 1 -- 172.21.15.100:0/2260182351 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7feedc070440 con 0x7feedc070030 2024-09-17T04:34:19.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.616+0000 7feee3600640 1 -- 172.21.15.100:0/2260182351 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feedc1535c0 con 0x7feedc070030 2024-09-17T04:34:19.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.617+0000 7feed8c00640 1 -- 172.21.15.100:0/2260182351 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) ==== 74+0+13181 (secure 0 0 0) 0x7feed4083040 con 0x7feedc070030 2024-09-17T04:34:19.618 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:34:19.618 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":170,"fsid":"8a399842-3a5c-4ea2-9afb-b2704332e59f","created":"2024-09-17T04:23:46.835777+0000","modified":"2024-09-17T04:34:14.818687+0000","last_up_change":"2024-09-17T04:24:03.497005+0000","last_in_change":"2024-09-17T04:23:52.610465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:24:03.995355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30913101-9146-4dd0-8dd1-fcf99fc3489f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6817","nonce":1292500299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6819","nonce":1292500299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6823","nonce":1292500299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1292500299},{"type":"v1","addr":"172.21.15.100:6821","nonce":1292500299}]},"public_addr":"172.21.15.100:6817/1292500299","cluster_addr":"172.21.15.100:6819/1292500299","heartbeat_back_addr":"172.21.15.100:6823/1292500299","heartbeat_front_addr":"172.21.15.100:6821/1292500299","state":["exists","up"]},{"osd":1,"uuid":"26225b84-4d11-4e1d-bf54-c91e7899f413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6809","nonce":1254254975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6811","nonce":1254254975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6815","nonce":1254254975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1254254975},{"type":"v1","addr":"172.21.15.100:6813","nonce":1254254975}]},"public_addr":"172.21.15.100:6809/1254254975","cluster_addr":"172.21.15.100:6811/1254254975","heartbeat_back_addr":"172.21.15.100:6815/1254254975","heartbeat_front_addr":"172.21.15.100:6813/1254254975","state":["exists","up"]},{"osd":2,"uuid":"f12f76eb-7482-4c5b-847d-99b9746807c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6825","nonce":2894635016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6827","nonce":2894635016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6831","nonce":2894635016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2894635016},{"type":"v1","addr":"172.21.15.100:6829","nonce":2894635016}]},"public_addr":"172.21.15.100:6825/2894635016","cluster_addr":"172.21.15.100:6827/2894635016","heartbeat_back_addr":"172.21.15.100:6831/2894635016","heartbeat_front_addr":"172.21.15.100:6829/2894635016","state":["exists","up"]},{"osd":3,"uuid":"c2694cdd-d620-4025-96d9-9aebbb2cedce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6801","nonce":2935840419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6803","nonce":2935840419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6807","nonce":2935840419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2935840419},{"type":"v1","addr":"172.21.15.100:6805","nonce":2935840419}]},"public_addr":"172.21.15.100:6801/2935840419","cluster_addr":"172.21.15.100:6803/2935840419","heartbeat_back_addr":"172.21.15.100:6807/2935840419","heartbeat_front_addr":"172.21.15.100:6805/2935840419","state":["exists","up"]},{"osd":4,"uuid":"e5b4d175-f95f-4dc1-b00e-75bd8604cbd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6811","nonce":985001108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6813","nonce":985001108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6817","nonce":985001108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":985001108},{"type":"v1","addr":"172.21.15.184:6815","nonce":985001108}]},"public_addr":"172.21.15.184:6811/985001108","cluster_addr":"172.21.15.184:6813/985001108","heartbeat_back_addr":"172.21.15.184:6817/985001108","heartbeat_front_addr":"172.21.15.184:6815/985001108","state":["exists","up"]},{"osd":5,"uuid":"c38f68c8-5099-440f-981c-01e5710475ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6803","nonce":3708129583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6805","nonce":3708129583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6809","nonce":3708129583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3708129583},{"type":"v1","addr":"172.21.15.184:6807","nonce":3708129583}]},"public_addr":"172.21.15.184:6803/3708129583","cluster_addr":"172.21.15.184:6805/3708129583","heartbeat_back_addr":"172.21.15.184:6809/3708129583","heartbeat_front_addr":"172.21.15.184:6807/3708129583","state":["exists","up"]},{"osd":6,"uuid":"d7791716-3d17-43ea-a427-31dffe31ecd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6825","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6827","nonce":3908254181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6829","nonce":3908254181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6833","nonce":3908254181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3908254181},{"type":"v1","addr":"172.21.15.184:6831","nonce":3908254181}]},"public_addr":"172.21.15.184:6827/3908254181","cluster_addr":"172.21.15.184:6829/3908254181","heartbeat_back_addr":"172.21.15.184:6833/3908254181","heartbeat_front_addr":"172.21.15.184:6831/3908254181","state":["exists","up"]},{"osd":7,"uuid":"fb888ad0-76e4-4352-8067-37ec89059f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6819","nonce":366777084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6821","nonce":366777084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6826","nonce":366777084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":366777084},{"type":"v1","addr":"172.21.15.184:6823","nonce":366777084}]},"public_addr":"172.21.15.184:6819/366777084","cluster_addr":"172.21.15.184:6821/366777084","heartbeat_back_addr":"172.21.15.184:6826/366777084","heartbeat_front_addr":"172.21.15.184:6823/366777084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.270689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.255642+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.654194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.024449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.735781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.286768+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.760821+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:23:59.445678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6835/1207785711":"2024-09-18T04:34:11.689984+0000","172.21.15.100:6834/1207785711":"2024-09-18T04:34:11.689984+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:34:19.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.619+0000 7feee3600640 1 -- 172.21.15.100:0/2260182351 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7feebc079790 msgr2=0x7feebc07bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:19.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.619+0000 7feee3600640 1 --2- 172.21.15.100:0/2260182351 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7feebc079790 0x7feebc07bca0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7feed00013f0 tx=0x7feed0001d60 comp rx=0 tx=0).stop 2024-09-17T04:34:19.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.619+0000 7feee3600640 1 -- 172.21.15.100:0/2260182351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feedc070030 msgr2=0x7feedc0743c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:19.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.619+0000 7feee3600640 1 --2- 172.21.15.100:0/2260182351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feedc070030 0x7feedc0743c0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7feed4036040 tx=0x7feed4033b80 comp rx=0 tx=0).stop 2024-09-17T04:34:19.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.619+0000 7feee3600640 1 -- 172.21.15.100:0/2260182351 shutdown_connections 2024-09-17T04:34:19.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.619+0000 7feee3600640 1 --2- 172.21.15.100:0/2260182351 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7feebc079790 0x7feebc07bca0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:19.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.619+0000 7feee3600640 1 --2- 172.21.15.100:0/2260182351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feedc070030 0x7feedc0743c0 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:19.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.619+0000 7feee3600640 1 --2- 172.21.15.100:0/2260182351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feedc151680 0x7feedc06faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:19.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.619+0000 7feee3600640 1 --2- 172.21.15.100:0/2260182351 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feec4004d40 0x7feedc150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:19.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.619+0000 7feee3600640 1 -- 172.21.15.100:0/2260182351 >> 172.21.15.100:0/2260182351 conn(0x7feedc06d730 msgr2=0x7feedc1602a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:19.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.620+0000 7feee3600640 1 -- 172.21.15.100:0/2260182351 shutdown_connections 2024-09-17T04:34:19.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:19.620+0000 7feee3600640 1 -- 172.21.15.100:0/2260182351 wait complete. 2024-09-17T04:34:20.644 INFO:tasks.ceph:Scrubbing osd.0 2024-09-17T04:34:20.644 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-17T04:34:20.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.875+0000 7feabcc00640 1 -- 172.21.15.100:0/2303969479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feab80af100 msgr2=0x7feab80af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:20.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.875+0000 7feabcc00640 1 --2- 172.21.15.100:0/2303969479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feab80af100 0x7feab80af510 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7feab806d8a0 tx=0x7feab002aa30 comp rx=0 tx=0).stop 2024-09-17T04:34:20.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.875+0000 7feabcc00640 1 -- 172.21.15.100:0/2303969479 shutdown_connections 2024-09-17T04:34:20.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.875+0000 7feabcc00640 1 --2- 172.21.15.100:0/2303969479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feab80af100 0x7feab80af510 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:20.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.875+0000 7feabcc00640 1 --2- 172.21.15.100:0/2303969479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feab806cda0 0x7feab806d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:20.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.875+0000 7feabcc00640 1 --2- 172.21.15.100:0/2303969479 >> v2:172.21.15.184:3300/0 conn(0x7feab80cc830 0x7feab80b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:20.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.875+0000 7feabcc00640 1 -- 172.21.15.100:0/2303969479 >> 172.21.15.100:0/2303969479 conn(0x7feab80b2540 msgr2=0x7feab80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:20.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.875+0000 7feabcc00640 1 -- 172.21.15.100:0/2303969479 shutdown_connections 2024-09-17T04:34:20.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.875+0000 7feabcc00640 1 -- 172.21.15.100:0/2303969479 wait complete. 2024-09-17T04:34:20.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.878+0000 7feabcc00640 1 Processor -- start 2024-09-17T04:34:20.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.878+0000 7feabcc00640 1 -- start start 2024-09-17T04:34:20.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.879+0000 7feabcc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feab806cda0 0x7feab809d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:20.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.879+0000 7feabcc00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feab80af100 0x7feab81f3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:20.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.879+0000 7feabcc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feab80cc830 0x7feab8154830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:20.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.879+0000 7feabcc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7feab80b34b0 con 0x7feab806cda0 2024-09-17T04:34:20.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.879+0000 7feabcc00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feab80b3330 con 0x7feab80cc830 2024-09-17T04:34:20.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.879+0000 7feabcc00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7feab80b3630 con 0x7feab80af100 2024-09-17T04:34:20.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.879+0000 7feab5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feab806cda0 0x7feab809d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:20.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.879+0000 7feab5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feab806cda0 0x7feab809d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59684/0 (socket says 172.21.15.100:59684) 2024-09-17T04:34:20.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.879+0000 7feab5e00640 1 -- 172.21.15.100:0/420175150 learned_addr learned my addr 172.21.15.100:0/420175150 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:20.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.880+0000 7feab6800640 1 --2- 172.21.15.100:0/420175150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feab80cc830 0x7feab8154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:20.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.880+0000 7feaafe00640 1 --2- 172.21.15.100:0/420175150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feab80af100 0x7feab81f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:20.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.880+0000 7feab5e00640 1 -- 172.21.15.100:0/420175150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feab80af100 msgr2=0x7feab81f3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:20.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.880+0000 7feab5e00640 1 --2- 172.21.15.100:0/420175150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feab80af100 0x7feab81f3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:20.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.880+0000 7feab5e00640 1 -- 172.21.15.100:0/420175150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feab80cc830 msgr2=0x7feab8154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:20.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.880+0000 7feab5e00640 1 --2- 172.21.15.100:0/420175150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feab80cc830 0x7feab8154830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:20.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.880+0000 7feab5e00640 1 -- 172.21.15.100:0/420175150 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab809d730 con 0x7feab806cda0 2024-09-17T04:34:20.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.880+0000 7feab6800640 1 --2- 172.21.15.100:0/420175150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feab80cc830 0x7feab8154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:20.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.880+0000 7feaafe00640 1 --2- 172.21.15.100:0/420175150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feab80af100 0x7feab81f3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:20.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.880+0000 7feab5e00640 1 --2- 172.21.15.100:0/420175150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feab806cda0 0x7feab809d1f0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7feaa800d740 tx=0x7feaa800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:20.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.881+0000 7feaae000640 1 -- 172.21.15.100:0/420175150 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8017070 con 0x7feab806cda0 2024-09-17T04:34:20.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.881+0000 7feaae000640 1 -- 172.21.15.100:0/420175150 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feaa8009d20 con 0x7feab806cda0 2024-09-17T04:34:20.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.881+0000 7feaae000640 1 -- 172.21.15.100:0/420175150 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa80106c0 con 0x7feab806cda0 2024-09-17T04:34:20.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.881+0000 7feabcc00640 1 -- 172.21.15.100:0/420175150 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feab81565d0 con 0x7feab806cda0 2024-09-17T04:34:20.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.881+0000 7feabcc00640 1 -- 172.21.15.100:0/420175150 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feab81562f0 con 0x7feab806cda0 2024-09-17T04:34:20.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.882+0000 7feabcc00640 1 -- 172.21.15.100:0/420175150 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feab80af510 con 0x7feab806cda0 2024-09-17T04:34:20.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.883+0000 7feaae000640 1 -- 172.21.15.100:0/420175150 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7feaa801a070 con 0x7feab806cda0 2024-09-17T04:34:20.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.884+0000 7feaae000640 1 --2- 172.21.15.100:0/420175150 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fea94079760 0x7fea9407bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:20.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.884+0000 7feaae000640 1 -- 172.21.15.100:0/420175150 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4999+0+0 (secure 0 0 0) 0x7feaa809a150 con 0x7feab806cda0 2024-09-17T04:34:20.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.884+0000 7feaae000640 1 --2- 172.21.15.100:0/420175150 >> [v2:172.21.15.100:6816/1292500299,v1:172.21.15.100:6817/1292500299] conn(0x7fea9407f670 0x7fea94081b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:20.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.884+0000 7feaae000640 1 -- 172.21.15.100:0/420175150 --> [v2:172.21.15.100:6816/1292500299,v1:172.21.15.100:6817/1292500299] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feaa8026e50 con 0x7fea9407f670 2024-09-17T04:34:20.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.884+0000 7feaae000640 1 -- 172.21.15.100:0/420175150 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=170) ==== 24+0+0 (secure 0 0 0) 0x7feaa809ca60 con 0x7feab806cda0 2024-09-17T04:34:20.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.884+0000 7feaafe00640 1 --2- 172.21.15.100:0/420175150 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fea94079760 0x7fea9407bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:20.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.885+0000 7feab6800640 1 --2- 172.21.15.100:0/420175150 >> [v2:172.21.15.100:6816/1292500299,v1:172.21.15.100:6817/1292500299] conn(0x7fea9407f670 0x7fea94081b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:20.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.885+0000 7feaafe00640 1 --2- 172.21.15.100:0/420175150 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fea94079760 0x7fea9407bc70 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7feaa0005fd0 tx=0x7feaa0005d60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:20.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.885+0000 7feab6800640 1 --2- 172.21.15.100:0/420175150 >> [v2:172.21.15.100:6816/1292500299,v1:172.21.15.100:6817/1292500299] conn(0x7fea9407f670 0x7fea94081b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:20.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.886+0000 7feaae000640 1 -- 172.21.15.100:0/420175150 <== osd.0 v2:172.21.15.100:6816/1292500299 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feaa8026e50 con 0x7fea9407f670 2024-09-17T04:34:20.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.899+0000 7feabcc00640 1 -- 172.21.15.100:0/420175150 --> [v2:172.21.15.100:6816/1292500299,v1:172.21.15.100:6817/1292500299] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7feab8070f00 con 0x7fea9407f670 2024-09-17T04:34:20.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.899+0000 7feaae000640 1 -- 172.21.15.100:0/420175150 <== osd.0 v2:172.21.15.100:6816/1292500299 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7feab8070f00 con 0x7fea9407f670 2024-09-17T04:34:20.900 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:34:20.900 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T04:34:20.900 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:34:20.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.900+0000 7feabcc00640 1 -- 172.21.15.100:0/420175150 >> [v2:172.21.15.100:6816/1292500299,v1:172.21.15.100:6817/1292500299] conn(0x7fea9407f670 msgr2=0x7fea94081b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:20.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.900+0000 7feabcc00640 1 --2- 172.21.15.100:0/420175150 >> [v2:172.21.15.100:6816/1292500299,v1:172.21.15.100:6817/1292500299] conn(0x7fea9407f670 0x7fea94081b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:20.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.900+0000 7feabcc00640 1 -- 172.21.15.100:0/420175150 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fea94079760 msgr2=0x7fea9407bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:20.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.900+0000 7feabcc00640 1 --2- 172.21.15.100:0/420175150 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fea94079760 0x7fea9407bc70 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7feaa0005fd0 tx=0x7feaa0005d60 comp rx=0 tx=0).stop 2024-09-17T04:34:20.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.900+0000 7feabcc00640 1 -- 172.21.15.100:0/420175150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feab806cda0 msgr2=0x7feab809d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:20.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.900+0000 7feabcc00640 1 --2- 172.21.15.100:0/420175150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feab806cda0 0x7feab809d1f0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7feaa800d740 tx=0x7feaa800dc00 comp rx=0 tx=0).stop 2024-09-17T04:34:20.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.900+0000 7feab6800640 1 -- 172.21.15.100:0/420175150 reap_dead start 2024-09-17T04:34:20.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.900+0000 7feabcc00640 1 -- 172.21.15.100:0/420175150 shutdown_connections 2024-09-17T04:34:20.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.900+0000 7feabcc00640 1 -- 172.21.15.100:0/420175150 >> 172.21.15.100:0/420175150 conn(0x7feab80b2540 msgr2=0x7feab8070520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:20.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.900+0000 7feabcc00640 1 -- 172.21.15.100:0/420175150 shutdown_connections 2024-09-17T04:34:20.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:20.900+0000 7feabcc00640 1 -- 172.21.15.100:0/420175150 wait complete. 2024-09-17T04:34:20.923 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-17T04:34:21.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.143+0000 7ff1a1800640 1 -- 172.21.15.100:0/2110443277 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff19c06f9a0 msgr2=0x7ff19c0b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:21.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.143+0000 7ff1a1800640 1 --2- 172.21.15.100:0/2110443277 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff19c06f9a0 0x7ff19c0b2f60 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff194005fd0 tx=0x7ff19402a820 comp rx=0 tx=0).stop 2024-09-17T04:34:21.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.143+0000 7ff1a1800640 1 -- 172.21.15.100:0/2110443277 shutdown_connections 2024-09-17T04:34:21.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.143+0000 7ff1a1800640 1 --2- 172.21.15.100:0/2110443277 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff19c06f9a0 0x7ff19c0b2f60 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.143+0000 7ff1a1800640 1 -- 172.21.15.100:0/2110443277 >> 172.21.15.100:0/2110443277 conn(0x7ff19c0b1e40 msgr2=0x7ff19c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:21.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.143+0000 7ff1a1800640 1 -- 172.21.15.100:0/2110443277 shutdown_connections 2024-09-17T04:34:21.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.144+0000 7ff1a1800640 1 -- 172.21.15.100:0/2110443277 wait complete. 2024-09-17T04:34:21.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.146+0000 7ff1a1800640 1 Processor -- start 2024-09-17T04:34:21.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.146+0000 7ff1a1800640 1 -- start start 2024-09-17T04:34:21.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.147+0000 7ff1a1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff19c0ad9c0 0x7ff19c151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:21.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.147+0000 7ff1a1800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff19c0af170 0x7ff19c158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:21.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.147+0000 7ff1a1800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff19c158790 0x7ff19c0caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:21.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.147+0000 7ff1a1800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c15a4c0 con 0x7ff19c0ad9c0 2024-09-17T04:34:21.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.147+0000 7ff1a1800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c15a340 con 0x7ff19c0af170 2024-09-17T04:34:21.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.147+0000 7ff1a1800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff19c15d670 con 0x7ff19c158790 2024-09-17T04:34:21.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.147+0000 7ff19be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff19c0ad9c0 0x7ff19c151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:21.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.148+0000 7ff19be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff19c0ad9c0 0x7ff19c151330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59700/0 (socket says 172.21.15.100:59700) 2024-09-17T04:34:21.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.148+0000 7ff19be00640 1 -- 172.21.15.100:0/4252593607 learned_addr learned my addr 172.21.15.100:0/4252593607 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:21.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.148+0000 7ff19be00640 1 -- 172.21.15.100:0/4252593607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff19c158790 msgr2=0x7ff19c0caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:21.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.148+0000 7ff19be00640 1 --2- 172.21.15.100:0/4252593607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff19c158790 0x7ff19c0caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.148+0000 7ff19be00640 1 -- 172.21.15.100:0/4252593607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff19c0af170 msgr2=0x7ff19c158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:21.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.148+0000 7ff19be00640 1 --2- 172.21.15.100:0/4252593607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff19c0af170 0x7ff19c158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.148+0000 7ff19be00640 1 -- 172.21.15.100:0/4252593607 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff19c156620 con 0x7ff19c0ad9c0 2024-09-17T04:34:21.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.148+0000 7ff19be00640 1 --2- 172.21.15.100:0/4252593607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff19c0ad9c0 0x7ff19c151330 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00c430 tx=0x7ff18c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:21.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.148+0000 7ff198c00640 1 -- 172.21.15.100:0/4252593607 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c017070 con 0x7ff19c0ad9c0 2024-09-17T04:34:21.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.149+0000 7ff198c00640 1 -- 172.21.15.100:0/4252593607 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff18c0053c0 con 0x7ff19c0ad9c0 2024-09-17T04:34:21.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.149+0000 7ff1a1800640 1 -- 172.21.15.100:0/4252593607 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff19c1f52b0 con 0x7ff19c0ad9c0 2024-09-17T04:34:21.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.149+0000 7ff198c00640 1 -- 172.21.15.100:0/4252593607 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c003850 con 0x7ff19c0ad9c0 2024-09-17T04:34:21.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.149+0000 7ff1a1800640 1 -- 172.21.15.100:0/4252593607 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff19c1f5870 con 0x7ff19c0ad9c0 2024-09-17T04:34:21.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.150+0000 7ff1a1800640 1 -- 172.21.15.100:0/4252593607 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff19c0b2f60 con 0x7ff19c0ad9c0 2024-09-17T04:34:21.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.151+0000 7ff198c00640 1 -- 172.21.15.100:0/4252593607 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7ff18c01a070 con 0x7ff19c0ad9c0 2024-09-17T04:34:21.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.152+0000 7ff198c00640 1 --2- 172.21.15.100:0/4252593607 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff178079760 0x7ff17807bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:21.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.152+0000 7ff198c00640 1 -- 172.21.15.100:0/4252593607 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4999+0+0 (secure 0 0 0) 0x7ff18c098d00 con 0x7ff19c0ad9c0 2024-09-17T04:34:21.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.152+0000 7ff193e00640 1 --2- 172.21.15.100:0/4252593607 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff178079760 0x7ff17807bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:21.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.153+0000 7ff193e00640 1 --2- 172.21.15.100:0/4252593607 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff178079760 0x7ff17807bc70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff184007fd0 tx=0x7ff1840078c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:21.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.156+0000 7ff198c00640 1 -- 172.21.15.100:0/4252593607 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff18c062970 con 0x7ff19c0ad9c0 2024-09-17T04:34:21.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.339+0000 7ff1a1800640 1 -- 172.21.15.100:0/4252593607 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7ff19c0aff00 con 0x7ff178079760 2024-09-17T04:34:21.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.340+0000 7ff198c00640 1 -- 172.21.15.100:0/4252593607 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff19c0aff00 con 0x7ff178079760 2024-09-17T04:34:21.340 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 0 to deep-scrub 2024-09-17T04:34:21.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.342+0000 7ff1a1800640 1 -- 172.21.15.100:0/4252593607 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff178079760 msgr2=0x7ff17807bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:21.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.342+0000 7ff1a1800640 1 --2- 172.21.15.100:0/4252593607 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff178079760 0x7ff17807bc70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff184007fd0 tx=0x7ff1840078c0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.342+0000 7ff1a1800640 1 -- 172.21.15.100:0/4252593607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff19c0ad9c0 msgr2=0x7ff19c151330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:21.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.342+0000 7ff1a1800640 1 --2- 172.21.15.100:0/4252593607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff19c0ad9c0 0x7ff19c151330 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00c430 tx=0x7ff18c00c8f0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.342+0000 7ff1a1800640 1 -- 172.21.15.100:0/4252593607 shutdown_connections 2024-09-17T04:34:21.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.342+0000 7ff1a1800640 1 --2- 172.21.15.100:0/4252593607 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7ff178079760 0x7ff17807bc70 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.342+0000 7ff1a1800640 1 --2- 172.21.15.100:0/4252593607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff19c158790 0x7ff19c0caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.342+0000 7ff1a1800640 1 --2- 172.21.15.100:0/4252593607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff19c0af170 0x7ff19c158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.342+0000 7ff1a1800640 1 --2- 172.21.15.100:0/4252593607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff19c0ad9c0 0x7ff19c151330 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.342+0000 7ff1a1800640 1 -- 172.21.15.100:0/4252593607 >> 172.21.15.100:0/4252593607 conn(0x7ff19c0b1e40 msgr2=0x7ff19c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:21.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.342+0000 7ff1a1800640 1 -- 172.21.15.100:0/4252593607 shutdown_connections 2024-09-17T04:34:21.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.342+0000 7ff1a1800640 1 -- 172.21.15.100:0/4252593607 wait complete. 2024-09-17T04:34:21.367 INFO:tasks.ceph:Scrubbing osd.1 2024-09-17T04:34:21.368 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-17T04:34:21.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.591+0000 7f7a49800640 1 -- 172.21.15.100:0/578044572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a441609c0 msgr2=0x7f7a440b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:21.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.591+0000 7f7a49800640 1 --2- 172.21.15.100:0/578044572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a441609c0 0x7f7a440b47c0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c005fd0 tx=0x7f7a3c02a540 comp rx=0 tx=0).stop 2024-09-17T04:34:21.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.591+0000 7f7a49800640 1 -- 172.21.15.100:0/578044572 shutdown_connections 2024-09-17T04:34:21.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.591+0000 7f7a49800640 1 --2- 172.21.15.100:0/578044572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a441609c0 0x7f7a440b47c0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.591+0000 7f7a49800640 1 -- 172.21.15.100:0/578044572 >> 172.21.15.100:0/578044572 conn(0x7f7a440b1dd0 msgr2=0x7f7a440b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:21.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.591+0000 7f7a49800640 1 -- 172.21.15.100:0/578044572 shutdown_connections 2024-09-17T04:34:21.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.591+0000 7f7a49800640 1 -- 172.21.15.100:0/578044572 wait complete. 2024-09-17T04:34:21.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.594+0000 7f7a49800640 1 Processor -- start 2024-09-17T04:34:21.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.594+0000 7f7a49800640 1 -- start start 2024-09-17T04:34:21.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.595+0000 7f7a49800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a440705b0 0x7f7a44074970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:21.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.595+0000 7f7a49800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a440b3ec0 0x7f7a44152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:21.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.595+0000 7f7a49800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a441534b0 0x7f7a4413acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:21.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.595+0000 7f7a49800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7a440af860 con 0x7f7a440b3ec0 2024-09-17T04:34:21.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.595+0000 7f7a49800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7a440af6e0 con 0x7f7a441534b0 2024-09-17T04:34:21.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.595+0000 7f7a49800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7a44167e20 con 0x7f7a440705b0 2024-09-17T04:34:21.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.596+0000 7f7a43400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a440b3ec0 0x7f7a44152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:21.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.596+0000 7f7a43400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a440b3ec0 0x7f7a44152f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59706/0 (socket says 172.21.15.100:59706) 2024-09-17T04:34:21.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.596+0000 7f7a43400640 1 -- 172.21.15.100:0/528884205 learned_addr learned my addr 172.21.15.100:0/528884205 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:21.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.596+0000 7f7a48e00640 1 --2- 172.21.15.100:0/528884205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a441534b0 0x7f7a4413acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:21.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.596+0000 7f7a43e00640 1 --2- 172.21.15.100:0/528884205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a440705b0 0x7f7a44074970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:21.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.596+0000 7f7a43400640 1 -- 172.21.15.100:0/528884205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a440705b0 msgr2=0x7f7a44074970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:21.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.596+0000 7f7a43400640 1 --2- 172.21.15.100:0/528884205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a440705b0 0x7f7a44074970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.596+0000 7f7a43400640 1 -- 172.21.15.100:0/528884205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a441534b0 msgr2=0x7f7a4413acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:21.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.596+0000 7f7a43400640 1 --2- 172.21.15.100:0/528884205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a441534b0 0x7f7a4413acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.596+0000 7f7a43400640 1 -- 172.21.15.100:0/528884205 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a441502e0 con 0x7f7a440b3ec0 2024-09-17T04:34:21.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.596+0000 7f7a43e00640 1 --2- 172.21.15.100:0/528884205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a440705b0 0x7f7a44074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:21.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.596+0000 7f7a43400640 1 --2- 172.21.15.100:0/528884205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a440b3ec0 0x7f7a44152f70 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f7a3000b3f0 tx=0x7f7a3000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:21.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.597+0000 7f7a3be00640 1 -- 172.21.15.100:0/528884205 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a30015020 con 0x7f7a440b3ec0 2024-09-17T04:34:21.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.597+0000 7f7a3be00640 1 -- 172.21.15.100:0/528884205 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a30011920 con 0x7f7a440b3ec0 2024-09-17T04:34:21.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.597+0000 7f7a3be00640 1 -- 172.21.15.100:0/528884205 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a30011d70 con 0x7f7a440b3ec0 2024-09-17T04:34:21.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.597+0000 7f7a49800640 1 -- 172.21.15.100:0/528884205 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a44073da0 con 0x7f7a440b3ec0 2024-09-17T04:34:21.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.597+0000 7f7a49800640 1 -- 172.21.15.100:0/528884205 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a4409add0 con 0x7f7a440b3ec0 2024-09-17T04:34:21.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.598+0000 7f7a49800640 1 -- 172.21.15.100:0/528884205 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7a440008d0 con 0x7f7a440b3ec0 2024-09-17T04:34:21.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.599+0000 7f7a3be00640 1 -- 172.21.15.100:0/528884205 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7f7a30018020 con 0x7f7a440b3ec0 2024-09-17T04:34:21.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.600+0000 7f7a3be00640 1 --2- 172.21.15.100:0/528884205 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7a24079760 0x7f7a2407bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:21.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.600+0000 7f7a3be00640 1 -- 172.21.15.100:0/528884205 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4999+0+0 (secure 0 0 0) 0x7f7a300994d0 con 0x7f7a440b3ec0 2024-09-17T04:34:21.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.600+0000 7f7a43e00640 1 --2- 172.21.15.100:0/528884205 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7a24079760 0x7f7a2407bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:21.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.600+0000 7f7a3be00640 1 --2- 172.21.15.100:0/528884205 >> [v2:172.21.15.100:6808/1254254975,v1:172.21.15.100:6809/1254254975] conn(0x7f7a2407f670 0x7f7a24081b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:21.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.600+0000 7f7a3be00640 1 -- 172.21.15.100:0/528884205 --> [v2:172.21.15.100:6808/1254254975,v1:172.21.15.100:6809/1254254975] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7a3000de50 con 0x7f7a2407f670 2024-09-17T04:34:21.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.601+0000 7f7a3be00640 1 -- 172.21.15.100:0/528884205 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=170) ==== 24+0+0 (secure 0 0 0) 0x7f7a300a13f0 con 0x7f7a440b3ec0 2024-09-17T04:34:21.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.601+0000 7f7a48e00640 1 --2- 172.21.15.100:0/528884205 >> [v2:172.21.15.100:6808/1254254975,v1:172.21.15.100:6809/1254254975] conn(0x7f7a2407f670 0x7f7a24081b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:21.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.601+0000 7f7a43e00640 1 --2- 172.21.15.100:0/528884205 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7a24079760 0x7f7a2407bc70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7a34007fd0 tx=0x7f7a34009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:21.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.601+0000 7f7a48e00640 1 --2- 172.21.15.100:0/528884205 >> [v2:172.21.15.100:6808/1254254975,v1:172.21.15.100:6809/1254254975] conn(0x7f7a2407f670 0x7f7a24081b00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:21.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.602+0000 7f7a3be00640 1 -- 172.21.15.100:0/528884205 <== osd.1 v2:172.21.15.100:6808/1254254975 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7a3000de50 con 0x7f7a2407f670 2024-09-17T04:34:21.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.615+0000 7f7a49800640 1 -- 172.21.15.100:0/528884205 --> [v2:172.21.15.100:6808/1254254975,v1:172.21.15.100:6809/1254254975] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7a44073b50 con 0x7f7a2407f670 2024-09-17T04:34:21.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.615+0000 7f7a3be00640 1 -- 172.21.15.100:0/528884205 <== osd.1 v2:172.21.15.100:6808/1254254975 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7a44073b50 con 0x7f7a2407f670 2024-09-17T04:34:21.616 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:34:21.616 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T04:34:21.616 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:34:21.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.616+0000 7f7a49800640 1 -- 172.21.15.100:0/528884205 >> [v2:172.21.15.100:6808/1254254975,v1:172.21.15.100:6809/1254254975] conn(0x7f7a2407f670 msgr2=0x7f7a24081b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:21.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.616+0000 7f7a49800640 1 --2- 172.21.15.100:0/528884205 >> [v2:172.21.15.100:6808/1254254975,v1:172.21.15.100:6809/1254254975] conn(0x7f7a2407f670 0x7f7a24081b00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.616+0000 7f7a49800640 1 -- 172.21.15.100:0/528884205 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7a24079760 msgr2=0x7f7a2407bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:21.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.616+0000 7f7a49800640 1 --2- 172.21.15.100:0/528884205 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7a24079760 0x7f7a2407bc70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7a34007fd0 tx=0x7f7a34009040 comp rx=0 tx=0).stop 2024-09-17T04:34:21.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.616+0000 7f7a49800640 1 -- 172.21.15.100:0/528884205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a440b3ec0 msgr2=0x7f7a44152f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:21.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.616+0000 7f7a49800640 1 --2- 172.21.15.100:0/528884205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a440b3ec0 0x7f7a44152f70 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f7a3000b3f0 tx=0x7f7a3000b8b0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.616+0000 7f7a48e00640 1 -- 172.21.15.100:0/528884205 reap_dead start 2024-09-17T04:34:21.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.616+0000 7f7a49800640 1 -- 172.21.15.100:0/528884205 shutdown_connections 2024-09-17T04:34:21.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.616+0000 7f7a49800640 1 -- 172.21.15.100:0/528884205 >> 172.21.15.100:0/528884205 conn(0x7f7a440b1dd0 msgr2=0x7f7a440b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:21.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.616+0000 7f7a49800640 1 -- 172.21.15.100:0/528884205 shutdown_connections 2024-09-17T04:34:21.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.616+0000 7f7a49800640 1 -- 172.21.15.100:0/528884205 wait complete. 2024-09-17T04:34:21.641 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-17T04:34:21.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.864+0000 7f1eb6600640 1 -- 172.21.15.100:0/2601372966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1eb0070710 msgr2=0x7f1eb01661c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:21.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.864+0000 7f1eb6600640 1 --2- 172.21.15.100:0/2601372966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1eb0070710 0x7f1eb01661c0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8005fd0 tx=0x7f1ea802a940 comp rx=0 tx=0).stop 2024-09-17T04:34:21.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.864+0000 7f1eb6600640 1 -- 172.21.15.100:0/2601372966 shutdown_connections 2024-09-17T04:34:21.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.864+0000 7f1eb6600640 1 --2- 172.21.15.100:0/2601372966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1eb0070710 0x7f1eb01661c0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.864+0000 7f1eb6600640 1 -- 172.21.15.100:0/2601372966 >> 172.21.15.100:0/2601372966 conn(0x7f1eb00b1e40 msgr2=0x7f1eb00b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:21.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.864+0000 7f1eb6600640 1 -- 172.21.15.100:0/2601372966 shutdown_connections 2024-09-17T04:34:21.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.864+0000 7f1eb6600640 1 -- 172.21.15.100:0/2601372966 wait complete. 2024-09-17T04:34:21.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.866+0000 7f1eb6600640 1 Processor -- start 2024-09-17T04:34:21.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.866+0000 7f1eb6600640 1 -- start start 2024-09-17T04:34:21.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.868+0000 7f1eb6600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1eb0070710 0x7f1eb0145360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:21.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.868+0000 7f1eb6600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1eb00ae080 0x7f1eb009ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:21.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.868+0000 7f1eb6600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1eb00af050 0x7f1eb009cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:21.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.868+0000 7f1eb6600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb0166ac0 con 0x7f1eb00af050 2024-09-17T04:34:21.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.868+0000 7f1eb6600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb0166940 con 0x7f1eb0070710 2024-09-17T04:34:21.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.868+0000 7f1eb6600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1eb0166c40 con 0x7f1eb00ae080 2024-09-17T04:34:21.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.868+0000 7f1eb5c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1eb00af050 0x7f1eb009cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:21.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.868+0000 7f1eb5c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1eb00af050 0x7f1eb009cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59724/0 (socket says 172.21.15.100:59724) 2024-09-17T04:34:21.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.868+0000 7f1eb5c00640 1 -- 172.21.15.100:0/811571419 learned_addr learned my addr 172.21.15.100:0/811571419 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:21.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.868+0000 7f1ea7e00640 1 --2- 172.21.15.100:0/811571419 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1eb00ae080 0x7f1eb009ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:21.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.868+0000 7f1eb5200640 1 --2- 172.21.15.100:0/811571419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1eb0070710 0x7f1eb0145360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:21.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.868+0000 7f1eb5c00640 1 -- 172.21.15.100:0/811571419 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1eb00ae080 msgr2=0x7f1eb009ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:21.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.868+0000 7f1eb5c00640 1 --2- 172.21.15.100:0/811571419 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1eb00ae080 0x7f1eb009ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.868+0000 7f1eb5c00640 1 -- 172.21.15.100:0/811571419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1eb0070710 msgr2=0x7f1eb0145360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:21.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.868+0000 7f1eb5c00640 1 --2- 172.21.15.100:0/811571419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1eb0070710 0x7f1eb0145360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.868+0000 7f1eb5c00640 1 -- 172.21.15.100:0/811571419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb009d7a0 con 0x7f1eb00af050 2024-09-17T04:34:21.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.869+0000 7f1eb5c00640 1 --2- 172.21.15.100:0/811571419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1eb00af050 0x7f1eb009cf90 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8009cd0 tx=0x7f1ea8031ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:21.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.869+0000 7f1ead600640 1 -- 172.21.15.100:0/811571419 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea803e070 con 0x7f1eb00af050 2024-09-17T04:34:21.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.869+0000 7f1ead600640 1 -- 172.21.15.100:0/811571419 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ea8004560 con 0x7f1eb00af050 2024-09-17T04:34:21.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.869+0000 7f1ead600640 1 -- 172.21.15.100:0/811571419 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8043460 con 0x7f1eb00af050 2024-09-17T04:34:21.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.869+0000 7f1eb6600640 1 -- 172.21.15.100:0/811571419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb0133c20 con 0x7f1eb00af050 2024-09-17T04:34:21.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.869+0000 7f1eb6600640 1 -- 172.21.15.100:0/811571419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb01341e0 con 0x7f1eb00af050 2024-09-17T04:34:21.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.870+0000 7f1eb6600640 1 -- 172.21.15.100:0/811571419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb0070000 con 0x7f1eb00af050 2024-09-17T04:34:21.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.871+0000 7f1ead600640 1 -- 172.21.15.100:0/811571419 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7f1ea803a440 con 0x7f1eb00af050 2024-09-17T04:34:21.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.872+0000 7f1ead600640 1 --2- 172.21.15.100:0/811571419 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1e8c079760 0x7f1e8c07bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:21.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.872+0000 7f1ead600640 1 -- 172.21.15.100:0/811571419 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4999+0+0 (secure 0 0 0) 0x7f1ea80bb0a0 con 0x7f1eb00af050 2024-09-17T04:34:21.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.873+0000 7f1eb5200640 1 --2- 172.21.15.100:0/811571419 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1e8c079760 0x7f1e8c07bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:21.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.873+0000 7f1eb5200640 1 --2- 172.21.15.100:0/811571419 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1e8c079760 0x7f1e8c07bc70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0001610 tx=0x7f1ea0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:21.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:21.877+0000 7f1ead600640 1 -- 172.21.15.100:0/811571419 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f1ea804a030 con 0x7f1eb00af050 2024-09-17T04:34:22.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.059+0000 7f1eb6600640 1 -- 172.21.15.100:0/811571419 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f1eb009d4d0 con 0x7f1e8c079760 2024-09-17T04:34:22.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.060+0000 7f1ead600640 1 -- 172.21.15.100:0/811571419 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1eb009d4d0 con 0x7f1e8c079760 2024-09-17T04:34:22.060 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 1 to deep-scrub 2024-09-17T04:34:22.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.062+0000 7f1eb6600640 1 -- 172.21.15.100:0/811571419 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1e8c079760 msgr2=0x7f1e8c07bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:22.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.062+0000 7f1eb6600640 1 --2- 172.21.15.100:0/811571419 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1e8c079760 0x7f1e8c07bc70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0001610 tx=0x7f1ea0009040 comp rx=0 tx=0).stop 2024-09-17T04:34:22.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.062+0000 7f1eb6600640 1 -- 172.21.15.100:0/811571419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1eb00af050 msgr2=0x7f1eb009cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:22.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.062+0000 7f1eb6600640 1 --2- 172.21.15.100:0/811571419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1eb00af050 0x7f1eb009cf90 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8009cd0 tx=0x7f1ea8031ea0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.062+0000 7f1eb6600640 1 -- 172.21.15.100:0/811571419 shutdown_connections 2024-09-17T04:34:22.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.062+0000 7f1eb6600640 1 --2- 172.21.15.100:0/811571419 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f1e8c079760 0x7f1e8c07bc70 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.062+0000 7f1eb6600640 1 --2- 172.21.15.100:0/811571419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1eb00af050 0x7f1eb009cf90 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.062+0000 7f1eb6600640 1 --2- 172.21.15.100:0/811571419 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1eb00ae080 0x7f1eb009ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.062+0000 7f1eb6600640 1 --2- 172.21.15.100:0/811571419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1eb0070710 0x7f1eb0145360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.062+0000 7f1eb6600640 1 -- 172.21.15.100:0/811571419 >> 172.21.15.100:0/811571419 conn(0x7f1eb00b1e40 msgr2=0x7f1eb01486f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:22.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.062+0000 7f1eb6600640 1 -- 172.21.15.100:0/811571419 shutdown_connections 2024-09-17T04:34:22.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.062+0000 7f1eb6600640 1 -- 172.21.15.100:0/811571419 wait complete. 2024-09-17T04:34:22.085 INFO:tasks.ceph:Scrubbing osd.2 2024-09-17T04:34:22.086 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-17T04:34:22.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.310+0000 7f000ee00640 1 -- 172.21.15.100:0/3989913674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f000806f010 msgr2=0x7f000806f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:22.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.310+0000 7f000ee00640 1 --2- 172.21.15.100:0/3989913674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f000806f010 0x7f000806f420 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f00080b2e80 tx=0x7f000002a8f0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.310+0000 7f000ee00640 1 -- 172.21.15.100:0/3989913674 shutdown_connections 2024-09-17T04:34:22.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.310+0000 7f000ee00640 1 --2- 172.21.15.100:0/3989913674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f000806f010 0x7f000806f420 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.310+0000 7f000ee00640 1 --2- 172.21.15.100:0/3989913674 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f000815ba50 0x7f00080b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.310+0000 7f000ee00640 1 --2- 172.21.15.100:0/3989913674 >> v2:172.21.15.184:3300/0 conn(0x7f00080cc7d0 0x7f00080b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.310+0000 7f000ee00640 1 -- 172.21.15.100:0/3989913674 >> 172.21.15.100:0/3989913674 conn(0x7f00080b2540 msgr2=0x7f00080d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:22.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.310+0000 7f000ee00640 1 -- 172.21.15.100:0/3989913674 shutdown_connections 2024-09-17T04:34:22.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.310+0000 7f000ee00640 1 -- 172.21.15.100:0/3989913674 wait complete. 2024-09-17T04:34:22.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.312+0000 7f000ee00640 1 Processor -- start 2024-09-17T04:34:22.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.312+0000 7f000ee00640 1 -- start start 2024-09-17T04:34:22.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.314+0000 7f000ee00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f000806f010 0x7f0008074860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:22.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.314+0000 7f000ee00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00080cc7d0 0x7f0008155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:22.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.314+0000 7f000ee00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f000815ba50 0x7f0008072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:22.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.314+0000 7f000ee00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f000806fe90 con 0x7f00080cc7d0 2024-09-17T04:34:22.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.314+0000 7f000ee00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f000806fd10 con 0x7f000815ba50 2024-09-17T04:34:22.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.314+0000 7f000ee00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0008070010 con 0x7f000806f010 2024-09-17T04:34:22.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.314+0000 7f0007400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00080cc7d0 0x7f0008155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:22.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.314+0000 7f0007400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00080cc7d0 0x7f0008155a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59730/0 (socket says 172.21.15.100:59730) 2024-09-17T04:34:22.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.314+0000 7f0007400640 1 -- 172.21.15.100:0/1898164054 learned_addr learned my addr 172.21.15.100:0/1898164054 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:22.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.314+0000 7f0007e00640 1 --2- 172.21.15.100:0/1898164054 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f000806f010 0x7f0008074860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:22.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.314+0000 7f000ce00640 1 --2- 172.21.15.100:0/1898164054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f000815ba50 0x7f0008072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:22.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.314+0000 7f0007400640 1 -- 172.21.15.100:0/1898164054 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f000806f010 msgr2=0x7f0008074860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:22.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.314+0000 7f0007400640 1 --2- 172.21.15.100:0/1898164054 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f000806f010 0x7f0008074860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.314+0000 7f0007400640 1 -- 172.21.15.100:0/1898164054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f000815ba50 msgr2=0x7f0008072e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:22.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.314+0000 7f0007400640 1 --2- 172.21.15.100:0/1898164054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f000815ba50 0x7f0008072e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.314+0000 7f0007400640 1 -- 172.21.15.100:0/1898164054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0008074da0 con 0x7f00080cc7d0 2024-09-17T04:34:22.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.315+0000 7f0007400640 1 --2- 172.21.15.100:0/1898164054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00080cc7d0 0x7f0008155a30 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7efff400d6d0 tx=0x7efff400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:22.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.315+0000 7effffe00640 1 -- 172.21.15.100:0/1898164054 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4017070 con 0x7f00080cc7d0 2024-09-17T04:34:22.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.315+0000 7effffe00640 1 -- 172.21.15.100:0/1898164054 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efff4005330 con 0x7f00080cc7d0 2024-09-17T04:34:22.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.315+0000 7f000ee00640 1 -- 172.21.15.100:0/1898164054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00080aeb10 con 0x7f00080cc7d0 2024-09-17T04:34:22.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.315+0000 7effffe00640 1 -- 172.21.15.100:0/1898164054 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4009820 con 0x7f00080cc7d0 2024-09-17T04:34:22.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.315+0000 7f000ee00640 1 -- 172.21.15.100:0/1898164054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0008074060 con 0x7f00080cc7d0 2024-09-17T04:34:22.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.316+0000 7f000ee00640 1 -- 172.21.15.100:0/1898164054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f000806f420 con 0x7f00080cc7d0 2024-09-17T04:34:22.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.317+0000 7effffe00640 1 -- 172.21.15.100:0/1898164054 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7efff4009bd0 con 0x7f00080cc7d0 2024-09-17T04:34:22.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.318+0000 7effffe00640 1 --2- 172.21.15.100:0/1898164054 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7effe8079790 0x7effe807bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:22.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.318+0000 7effffe00640 1 -- 172.21.15.100:0/1898164054 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4999+0+0 (secure 0 0 0) 0x7efff409a080 con 0x7f00080cc7d0 2024-09-17T04:34:22.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.319+0000 7f0007e00640 1 --2- 172.21.15.100:0/1898164054 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7effe8079790 0x7effe807bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:22.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.319+0000 7effffe00640 1 --2- 172.21.15.100:0/1898164054 >> [v2:172.21.15.100:6824/2894635016,v1:172.21.15.100:6825/2894635016] conn(0x7effe807f6a0 0x7effe8081b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:22.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.319+0000 7effffe00640 1 -- 172.21.15.100:0/1898164054 --> [v2:172.21.15.100:6824/2894635016,v1:172.21.15.100:6825/2894635016] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7effe8082200 con 0x7effe807f6a0 2024-09-17T04:34:22.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.319+0000 7effffe00640 1 -- 172.21.15.100:0/1898164054 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=170) ==== 24+0+0 (secure 0 0 0) 0x7efff409c990 con 0x7f00080cc7d0 2024-09-17T04:34:22.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.319+0000 7f000ce00640 1 --2- 172.21.15.100:0/1898164054 >> [v2:172.21.15.100:6824/2894635016,v1:172.21.15.100:6825/2894635016] conn(0x7effe807f6a0 0x7effe8081b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:22.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.319+0000 7f0007e00640 1 --2- 172.21.15.100:0/1898164054 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7effe8079790 0x7effe807bca0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7efff8005fd0 tx=0x7efff800b260 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:22.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.319+0000 7f000ce00640 1 --2- 172.21.15.100:0/1898164054 >> [v2:172.21.15.100:6824/2894635016,v1:172.21.15.100:6825/2894635016] conn(0x7effe807f6a0 0x7effe8081b30 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:22.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.320+0000 7effffe00640 1 -- 172.21.15.100:0/1898164054 <== osd.2 v2:172.21.15.100:6824/2894635016 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7effe8082200 con 0x7effe807f6a0 2024-09-17T04:34:22.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.333+0000 7f000ee00640 1 -- 172.21.15.100:0/1898164054 --> [v2:172.21.15.100:6824/2894635016,v1:172.21.15.100:6825/2894635016] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0008168420 con 0x7effe807f6a0 2024-09-17T04:34:22.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.334+0000 7effffe00640 1 -- 172.21.15.100:0/1898164054 <== osd.2 v2:172.21.15.100:6824/2894635016 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0008168420 con 0x7effe807f6a0 2024-09-17T04:34:22.335 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:34:22.335 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T04:34:22.335 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:34:22.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.334+0000 7f000ee00640 1 -- 172.21.15.100:0/1898164054 >> [v2:172.21.15.100:6824/2894635016,v1:172.21.15.100:6825/2894635016] conn(0x7effe807f6a0 msgr2=0x7effe8081b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:22.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.334+0000 7f000ee00640 1 --2- 172.21.15.100:0/1898164054 >> [v2:172.21.15.100:6824/2894635016,v1:172.21.15.100:6825/2894635016] conn(0x7effe807f6a0 0x7effe8081b30 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.334+0000 7f000ee00640 1 -- 172.21.15.100:0/1898164054 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7effe8079790 msgr2=0x7effe807bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:22.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.334+0000 7f000ee00640 1 --2- 172.21.15.100:0/1898164054 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7effe8079790 0x7effe807bca0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7efff8005fd0 tx=0x7efff800b260 comp rx=0 tx=0).stop 2024-09-17T04:34:22.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.334+0000 7f000ee00640 1 -- 172.21.15.100:0/1898164054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00080cc7d0 msgr2=0x7f0008155a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:22.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.334+0000 7f000ee00640 1 --2- 172.21.15.100:0/1898164054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00080cc7d0 0x7f0008155a30 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7efff400d6d0 tx=0x7efff400db90 comp rx=0 tx=0).stop 2024-09-17T04:34:22.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.334+0000 7f000ce00640 1 -- 172.21.15.100:0/1898164054 reap_dead start 2024-09-17T04:34:22.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.334+0000 7f000ee00640 1 -- 172.21.15.100:0/1898164054 shutdown_connections 2024-09-17T04:34:22.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.334+0000 7f000ee00640 1 -- 172.21.15.100:0/1898164054 >> 172.21.15.100:0/1898164054 conn(0x7f00080b2540 msgr2=0x7f0008152ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:22.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.334+0000 7f000ee00640 1 -- 172.21.15.100:0/1898164054 shutdown_connections 2024-09-17T04:34:22.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.334+0000 7f000ee00640 1 -- 172.21.15.100:0/1898164054 wait complete. 2024-09-17T04:34:22.356 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-17T04:34:22.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.575+0000 7fcfaf200640 1 -- 172.21.15.100:0/2249059263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfb00ad9c0 msgr2=0x7fcfb00af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:22.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.575+0000 7fcfaf200640 1 --2- 172.21.15.100:0/2249059263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfb00ad9c0 0x7fcfb00af170 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0005fd0 tx=0x7fcfa002a540 comp rx=0 tx=0).stop 2024-09-17T04:34:22.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.575+0000 7fcfaf200640 1 -- 172.21.15.100:0/2249059263 shutdown_connections 2024-09-17T04:34:22.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.575+0000 7fcfaf200640 1 --2- 172.21.15.100:0/2249059263 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfb00b2f60 0x7fcfb015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.575+0000 7fcfaf200640 1 --2- 172.21.15.100:0/2249059263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfb00ad9c0 0x7fcfb00af170 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.575+0000 7fcfaf200640 1 -- 172.21.15.100:0/2249059263 >> 172.21.15.100:0/2249059263 conn(0x7fcfb00b1e40 msgr2=0x7fcfb00b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:22.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.575+0000 7fcfaf200640 1 -- 172.21.15.100:0/2249059263 shutdown_connections 2024-09-17T04:34:22.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.575+0000 7fcfaf200640 1 -- 172.21.15.100:0/2249059263 wait complete. 2024-09-17T04:34:22.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.577+0000 7fcfaf200640 1 Processor -- start 2024-09-17T04:34:22.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.577+0000 7fcfaf200640 1 -- start start 2024-09-17T04:34:22.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.579+0000 7fcfaf200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfb00ad9c0 0x7fcfb009aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:22.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.579+0000 7fcfaf200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfb00b2f60 0x7fcfb009da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:22.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.579+0000 7fcfaf200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfb015d670 0x7fcfb009b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:22.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.579+0000 7fcfaf200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb006f410 con 0x7fcfb015d670 2024-09-17T04:34:22.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.579+0000 7fcfaf200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb006f290 con 0x7fcfb00b2f60 2024-09-17T04:34:22.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.579+0000 7fcfaf200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcfb006f590 con 0x7fcfb00ad9c0 2024-09-17T04:34:22.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.579+0000 7fcfae800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfb015d670 0x7fcfb009b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:22.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.579+0000 7fcfae800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfb015d670 0x7fcfb009b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59750/0 (socket says 172.21.15.100:59750) 2024-09-17T04:34:22.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.579+0000 7fcfade00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfb00ad9c0 0x7fcfb009aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:22.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.579+0000 7fcfa5400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfb00b2f60 0x7fcfb009da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:22.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.579+0000 7fcfae800640 1 -- 172.21.15.100:0/3743451469 learned_addr learned my addr 172.21.15.100:0/3743451469 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:22.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.580+0000 7fcfae800640 1 -- 172.21.15.100:0/3743451469 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfb00ad9c0 msgr2=0x7fcfb009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:22.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.580+0000 7fcfae800640 1 --2- 172.21.15.100:0/3743451469 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfb00ad9c0 0x7fcfb009aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.580+0000 7fcfae800640 1 -- 172.21.15.100:0/3743451469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfb00b2f60 msgr2=0x7fcfb009da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:22.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.580+0000 7fcfae800640 1 --2- 172.21.15.100:0/3743451469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfb00b2f60 0x7fcfb009da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.580+0000 7fcfae800640 1 -- 172.21.15.100:0/3743451469 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb00ccb60 con 0x7fcfb015d670 2024-09-17T04:34:22.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.580+0000 7fcfae800640 1 --2- 172.21.15.100:0/3743451469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfb015d670 0x7fcfb009b2f0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800c700 tx=0x7fcfa800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:22.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.580+0000 7fcfa6a00640 1 -- 172.21.15.100:0/3743451469 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8003b70 con 0x7fcfb015d670 2024-09-17T04:34:22.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.580+0000 7fcfa6a00640 1 -- 172.21.15.100:0/3743451469 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfa8003d10 con 0x7fcfb015d670 2024-09-17T04:34:22.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.580+0000 7fcfaf200640 1 -- 172.21.15.100:0/3743451469 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb009df80 con 0x7fcfb015d670 2024-09-17T04:34:22.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.580+0000 7fcfa6a00640 1 -- 172.21.15.100:0/3743451469 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa800aab0 con 0x7fcfb015d670 2024-09-17T04:34:22.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.580+0000 7fcfaf200640 1 -- 172.21.15.100:0/3743451469 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb0074570 con 0x7fcfb015d670 2024-09-17T04:34:22.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.581+0000 7fcfaf200640 1 -- 172.21.15.100:0/3743451469 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfb006fa20 con 0x7fcfb015d670 2024-09-17T04:34:22.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.583+0000 7fcfa6a00640 1 -- 172.21.15.100:0/3743451469 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7fcfa800ac50 con 0x7fcfb015d670 2024-09-17T04:34:22.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.584+0000 7fcfa6a00640 1 --2- 172.21.15.100:0/3743451469 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcf8c079790 0x7fcf8c07bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:22.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.584+0000 7fcfa6a00640 1 -- 172.21.15.100:0/3743451469 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4999+0+0 (secure 0 0 0) 0x7fcfa8099660 con 0x7fcfb015d670 2024-09-17T04:34:22.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.584+0000 7fcfade00640 1 --2- 172.21.15.100:0/3743451469 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcf8c079790 0x7fcf8c07bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:22.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.585+0000 7fcfade00640 1 --2- 172.21.15.100:0/3743451469 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcf8c079790 0x7fcf8c07bca0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0033000 tx=0x7fcfa0004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:22.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.588+0000 7fcfa6a00640 1 -- 172.21.15.100:0/3743451469 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fcfa8062c20 con 0x7fcfb015d670 2024-09-17T04:34:22.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.773+0000 7fcfaf200640 1 -- 172.21.15.100:0/3743451469 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fcfb015ac00 con 0x7fcf8c079790 2024-09-17T04:34:22.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.773+0000 7fcfa6a00640 1 -- 172.21.15.100:0/3743451469 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fcfb015ac00 con 0x7fcf8c079790 2024-09-17T04:34:22.774 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 2 to deep-scrub 2024-09-17T04:34:22.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.775+0000 7fcfaf200640 1 -- 172.21.15.100:0/3743451469 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcf8c079790 msgr2=0x7fcf8c07bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:22.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.775+0000 7fcfaf200640 1 --2- 172.21.15.100:0/3743451469 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcf8c079790 0x7fcf8c07bca0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0033000 tx=0x7fcfa0004730 comp rx=0 tx=0).stop 2024-09-17T04:34:22.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.775+0000 7fcfaf200640 1 -- 172.21.15.100:0/3743451469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfb015d670 msgr2=0x7fcfb009b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:22.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.775+0000 7fcfaf200640 1 --2- 172.21.15.100:0/3743451469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfb015d670 0x7fcfb009b2f0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800c700 tx=0x7fcfa800cbc0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.776+0000 7fcfaf200640 1 -- 172.21.15.100:0/3743451469 shutdown_connections 2024-09-17T04:34:22.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.776+0000 7fcfaf200640 1 --2- 172.21.15.100:0/3743451469 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fcf8c079790 0x7fcf8c07bca0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.776+0000 7fcfaf200640 1 --2- 172.21.15.100:0/3743451469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfb015d670 0x7fcfb009b2f0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.776+0000 7fcfaf200640 1 --2- 172.21.15.100:0/3743451469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfb00b2f60 0x7fcfb009da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.776+0000 7fcfaf200640 1 --2- 172.21.15.100:0/3743451469 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfb00ad9c0 0x7fcfb009aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.776+0000 7fcfaf200640 1 -- 172.21.15.100:0/3743451469 >> 172.21.15.100:0/3743451469 conn(0x7fcfb00b1e40 msgr2=0x7fcfb00afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:22.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.776+0000 7fcfaf200640 1 -- 172.21.15.100:0/3743451469 shutdown_connections 2024-09-17T04:34:22.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:22.776+0000 7fcfaf200640 1 -- 172.21.15.100:0/3743451469 wait complete. 2024-09-17T04:34:22.799 INFO:tasks.ceph:Scrubbing osd.3 2024-09-17T04:34:22.800 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-17T04:34:23.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.022+0000 7f63d3e00640 1 -- 172.21.15.100:0/1712893287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63d40af3c0 msgr2=0x7f63d41602b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.022+0000 7f63d3e00640 1 --2- 172.21.15.100:0/1712893287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63d40af3c0 0x7f63d41602b0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f63cc005fd0 tx=0x7f63cc02a540 comp rx=0 tx=0).stop 2024-09-17T04:34:23.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.022+0000 7f63d3e00640 1 -- 172.21.15.100:0/1712893287 shutdown_connections 2024-09-17T04:34:23.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.022+0000 7f63d3e00640 1 --2- 172.21.15.100:0/1712893287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63d40af3c0 0x7f63d41602b0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.022+0000 7f63d3e00640 1 --2- 172.21.15.100:0/1712893287 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63d40b3ec0 0x7f63d40b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.022+0000 7f63d3e00640 1 -- 172.21.15.100:0/1712893287 >> 172.21.15.100:0/1712893287 conn(0x7f63d40b1dd0 msgr2=0x7f63d40b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.023+0000 7f63d3e00640 1 -- 172.21.15.100:0/1712893287 shutdown_connections 2024-09-17T04:34:23.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.023+0000 7f63d3e00640 1 -- 172.21.15.100:0/1712893287 wait complete. 2024-09-17T04:34:23.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.025+0000 7f63d3e00640 1 Processor -- start 2024-09-17T04:34:23.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.025+0000 7f63d3e00640 1 -- start start 2024-09-17T04:34:23.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.026+0000 7f63d3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63d40705b0 0x7f63d40749e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.026+0000 7f63d3e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63d40af3c0 0x7f63d41f5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.026+0000 7f63d3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63d40b3ec0 0x7f63d41e5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.026+0000 7f63d3e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f63d4160b80 con 0x7f63d40b3ec0 2024-09-17T04:34:23.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.026+0000 7f63d3e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f63d4160a00 con 0x7f63d40705b0 2024-09-17T04:34:23.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.026+0000 7f63d3e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f63d415cc60 con 0x7f63d40af3c0 2024-09-17T04:34:23.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.027+0000 7f63d3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63d40b3ec0 0x7f63d41e5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.027+0000 7f63d3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63d40b3ec0 0x7f63d41e5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59776/0 (socket says 172.21.15.100:59776) 2024-09-17T04:34:23.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.027+0000 7f63d3400640 1 -- 172.21.15.100:0/3120919298 learned_addr learned my addr 172.21.15.100:0/3120919298 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:23.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.027+0000 7f63d2a00640 1 --2- 172.21.15.100:0/3120919298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63d40705b0 0x7f63d40749e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.027+0000 7f63ca000640 1 --2- 172.21.15.100:0/3120919298 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63d40af3c0 0x7f63d41f5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.027+0000 7f63d3400640 1 -- 172.21.15.100:0/3120919298 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63d40af3c0 msgr2=0x7f63d41f5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.027+0000 7f63d3400640 1 --2- 172.21.15.100:0/3120919298 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63d40af3c0 0x7f63d41f5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.027+0000 7f63d3400640 1 -- 172.21.15.100:0/3120919298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63d40705b0 msgr2=0x7f63d40749e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.027+0000 7f63d3400640 1 --2- 172.21.15.100:0/3120919298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63d40705b0 0x7f63d40749e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.027+0000 7f63d3400640 1 -- 172.21.15.100:0/3120919298 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63d4072e20 con 0x7f63d40b3ec0 2024-09-17T04:34:23.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.027+0000 7f63d3400640 1 --2- 172.21.15.100:0/3120919298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63d40b3ec0 0x7f63d41e5a10 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f63cc006010 tx=0x7f63cc003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.028+0000 7f63cb400640 1 -- 172.21.15.100:0/3120919298 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc03e030 con 0x7f63d40b3ec0 2024-09-17T04:34:23.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.028+0000 7f63cb400640 1 -- 172.21.15.100:0/3120919298 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63cc003d10 con 0x7f63d40b3ec0 2024-09-17T04:34:23.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.028+0000 7f63d3e00640 1 -- 172.21.15.100:0/3120919298 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63d4073320 con 0x7f63d40b3ec0 2024-09-17T04:34:23.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.028+0000 7f63cb400640 1 -- 172.21.15.100:0/3120919298 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc035040 con 0x7f63d40b3ec0 2024-09-17T04:34:23.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.028+0000 7f63d3e00640 1 -- 172.21.15.100:0/3120919298 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63d40738e0 con 0x7f63d40b3ec0 2024-09-17T04:34:23.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.029+0000 7f63d3e00640 1 -- 172.21.15.100:0/3120919298 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f63d4077140 con 0x7f63d40b3ec0 2024-09-17T04:34:23.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.030+0000 7f63cb400640 1 -- 172.21.15.100:0/3120919298 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7f63cc041030 con 0x7f63d40b3ec0 2024-09-17T04:34:23.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.031+0000 7f63cb400640 1 --2- 172.21.15.100:0/3120919298 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f63b0079760 0x7f63b007bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.031+0000 7f63cb400640 1 -- 172.21.15.100:0/3120919298 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4999+0+0 (secure 0 0 0) 0x7f63cc0b9ee0 con 0x7f63d40b3ec0 2024-09-17T04:34:23.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.031+0000 7f63d2a00640 1 --2- 172.21.15.100:0/3120919298 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f63b0079760 0x7f63b007bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.031+0000 7f63cb400640 1 --2- 172.21.15.100:0/3120919298 >> [v2:172.21.15.100:6800/2935840419,v1:172.21.15.100:6801/2935840419] conn(0x7f63b007f670 0x7f63b0081b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.031+0000 7f63cb400640 1 -- 172.21.15.100:0/3120919298 --> [v2:172.21.15.100:6800/2935840419,v1:172.21.15.100:6801/2935840419] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f63cc04ae50 con 0x7f63b007f670 2024-09-17T04:34:23.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.031+0000 7f63cb400640 1 -- 172.21.15.100:0/3120919298 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=170) ==== 24+0+0 (secure 0 0 0) 0x7f63cc0c12a0 con 0x7f63d40b3ec0 2024-09-17T04:34:23.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.032+0000 7f63ca000640 1 --2- 172.21.15.100:0/3120919298 >> [v2:172.21.15.100:6800/2935840419,v1:172.21.15.100:6801/2935840419] conn(0x7f63b007f670 0x7f63b0081b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.032+0000 7f63d2a00640 1 --2- 172.21.15.100:0/3120919298 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f63b0079760 0x7f63b007bc70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f63c4005fd0 tx=0x7f63c4009500 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.032+0000 7f63ca000640 1 --2- 172.21.15.100:0/3120919298 >> [v2:172.21.15.100:6800/2935840419,v1:172.21.15.100:6801/2935840419] conn(0x7f63b007f670 0x7f63b0081b00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.033+0000 7f63cb400640 1 -- 172.21.15.100:0/3120919298 <== osd.3 v2:172.21.15.100:6800/2935840419 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f63cc04ae50 con 0x7f63b007f670 2024-09-17T04:34:23.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.046+0000 7f63d3e00640 1 -- 172.21.15.100:0/3120919298 --> [v2:172.21.15.100:6800/2935840419,v1:172.21.15.100:6801/2935840419] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f63d40730d0 con 0x7f63b007f670 2024-09-17T04:34:23.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.046+0000 7f63cb400640 1 -- 172.21.15.100:0/3120919298 <== osd.3 v2:172.21.15.100:6800/2935840419 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f63d40730d0 con 0x7f63b007f670 2024-09-17T04:34:23.047 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:34:23.047 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T04:34:23.047 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:34:23.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.047+0000 7f63d3e00640 1 -- 172.21.15.100:0/3120919298 >> [v2:172.21.15.100:6800/2935840419,v1:172.21.15.100:6801/2935840419] conn(0x7f63b007f670 msgr2=0x7f63b0081b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.047+0000 7f63d3e00640 1 --2- 172.21.15.100:0/3120919298 >> [v2:172.21.15.100:6800/2935840419,v1:172.21.15.100:6801/2935840419] conn(0x7f63b007f670 0x7f63b0081b00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.047+0000 7f63d3e00640 1 -- 172.21.15.100:0/3120919298 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f63b0079760 msgr2=0x7f63b007bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.047+0000 7f63d3e00640 1 --2- 172.21.15.100:0/3120919298 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f63b0079760 0x7f63b007bc70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f63c4005fd0 tx=0x7f63c4009500 comp rx=0 tx=0).stop 2024-09-17T04:34:23.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.047+0000 7f63d3e00640 1 -- 172.21.15.100:0/3120919298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63d40b3ec0 msgr2=0x7f63d41e5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.047+0000 7f63d3e00640 1 --2- 172.21.15.100:0/3120919298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63d40b3ec0 0x7f63d41e5a10 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f63cc006010 tx=0x7f63cc003820 comp rx=0 tx=0).stop 2024-09-17T04:34:23.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.047+0000 7f63d3400640 1 -- 172.21.15.100:0/3120919298 reap_dead start 2024-09-17T04:34:23.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.047+0000 7f63d3e00640 1 -- 172.21.15.100:0/3120919298 shutdown_connections 2024-09-17T04:34:23.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.047+0000 7f63d3e00640 1 -- 172.21.15.100:0/3120919298 >> 172.21.15.100:0/3120919298 conn(0x7f63d40b1dd0 msgr2=0x7f63d40cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.047+0000 7f63d3e00640 1 -- 172.21.15.100:0/3120919298 shutdown_connections 2024-09-17T04:34:23.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.047+0000 7f63d3e00640 1 -- 172.21.15.100:0/3120919298 wait complete. 2024-09-17T04:34:23.068 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-17T04:34:23.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.288+0000 7f0566e00640 1 -- 172.21.15.100:0/712169881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05600af6a0 msgr2=0x7f05600ad1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.288+0000 7f0566e00640 1 --2- 172.21.15.100:0/712169881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05600af6a0 0x7f05600ad1c0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f0560072ea0 tx=0x7f054802a4d0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.288+0000 7f0566e00640 1 -- 172.21.15.100:0/712169881 shutdown_connections 2024-09-17T04:34:23.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.288+0000 7f0566e00640 1 --2- 172.21.15.100:0/712169881 >> v2:172.21.15.184:3300/0 conn(0x7f05600ad700 0x7f05600adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.288+0000 7f0566e00640 1 --2- 172.21.15.100:0/712169881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05600af6a0 0x7f05600ad1c0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.288+0000 7f0566e00640 1 -- 172.21.15.100:0/712169881 >> 172.21.15.100:0/712169881 conn(0x7f056006d3d0 msgr2=0x7f05600b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.289+0000 7f0566e00640 1 -- 172.21.15.100:0/712169881 shutdown_connections 2024-09-17T04:34:23.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.289+0000 7f0566e00640 1 -- 172.21.15.100:0/712169881 wait complete. 2024-09-17T04:34:23.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.291+0000 7f0566e00640 1 Processor -- start 2024-09-17T04:34:23.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.291+0000 7f0566e00640 1 -- start start 2024-09-17T04:34:23.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.292+0000 7f0566e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0560070660 0x7f0560155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.292+0000 7f0566e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05600ad700 0x7f05601f5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.292+0000 7f0566e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05600af6a0 0x7f05601f9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.292+0000 7f0566e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f05600cb950 con 0x7f0560070660 2024-09-17T04:34:23.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.292+0000 7f0566e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f05600afab0 con 0x7f05600af6a0 2024-09-17T04:34:23.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.292+0000 7f0566e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f05600cbad0 con 0x7f05600ad700 2024-09-17T04:34:23.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.293+0000 7f055fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0560070660 0x7f0560155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.293+0000 7f055fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0560070660 0x7f0560155390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59792/0 (socket says 172.21.15.100:59792) 2024-09-17T04:34:23.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.293+0000 7f055fe00640 1 -- 172.21.15.100:0/1744033666 learned_addr learned my addr 172.21.15.100:0/1744033666 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:23.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.293+0000 7f055fe00640 1 -- 172.21.15.100:0/1744033666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05600ad700 msgr2=0x7f05601f5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.293+0000 7f055fe00640 1 --2- 172.21.15.100:0/1744033666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05600ad700 0x7f05601f5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.293+0000 7f055fe00640 1 -- 172.21.15.100:0/1744033666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05600af6a0 msgr2=0x7f05601f9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.293+0000 7f0564e00640 1 --2- 172.21.15.100:0/1744033666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05600af6a0 0x7f05601f9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.293+0000 7f055fe00640 1 --2- 172.21.15.100:0/1744033666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05600af6a0 0x7f05601f9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.293+0000 7f055fe00640 1 -- 172.21.15.100:0/1744033666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05600c6b00 con 0x7f0560070660 2024-09-17T04:34:23.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.293+0000 7f0564e00640 1 --2- 172.21.15.100:0/1744033666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05600af6a0 0x7f05601f9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:23.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.293+0000 7f055fe00640 1 --2- 172.21.15.100:0/1744033666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0560070660 0x7f0560155390 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f055000c3d0 tx=0x7f055000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.294+0000 7f055cc00640 1 -- 172.21.15.100:0/1744033666 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0550017070 con 0x7f0560070660 2024-09-17T04:34:23.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.294+0000 7f055cc00640 1 -- 172.21.15.100:0/1744033666 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05500053c0 con 0x7f0560070660 2024-09-17T04:34:23.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.294+0000 7f0566e00640 1 -- 172.21.15.100:0/1744033666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05600c6de0 con 0x7f0560070660 2024-09-17T04:34:23.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.294+0000 7f055cc00640 1 -- 172.21.15.100:0/1744033666 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0550003850 con 0x7f0560070660 2024-09-17T04:34:23.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.294+0000 7f0566e00640 1 -- 172.21.15.100:0/1744033666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05601f5fd0 con 0x7f0560070660 2024-09-17T04:34:23.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.295+0000 7f0566e00640 1 -- 172.21.15.100:0/1744033666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05600adb10 con 0x7f0560070660 2024-09-17T04:34:23.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.296+0000 7f055cc00640 1 -- 172.21.15.100:0/1744033666 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7f055001a070 con 0x7f0560070660 2024-09-17T04:34:23.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.297+0000 7f055cc00640 1 --2- 172.21.15.100:0/1744033666 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f053c079790 0x7f053c07bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.297+0000 7f055cc00640 1 -- 172.21.15.100:0/1744033666 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4999+0+0 (secure 0 0 0) 0x7f0550098c30 con 0x7f0560070660 2024-09-17T04:34:23.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.297+0000 7f0557400640 1 --2- 172.21.15.100:0/1744033666 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f053c079790 0x7f053c07bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.298+0000 7f0557400640 1 --2- 172.21.15.100:0/1744033666 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f053c079790 0x7f053c07bca0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0548033000 tx=0x7f05480046c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.301+0000 7f055cc00640 1 -- 172.21.15.100:0/1744033666 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f05500628f0 con 0x7f0560070660 2024-09-17T04:34:23.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.485+0000 7f0566e00640 1 -- 172.21.15.100:0/1744033666 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f05600cd7d0 con 0x7f053c079790 2024-09-17T04:34:23.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.486+0000 7f055cc00640 1 -- 172.21.15.100:0/1744033666 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f05600cd7d0 con 0x7f053c079790 2024-09-17T04:34:23.486 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 3 to deep-scrub 2024-09-17T04:34:23.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.488+0000 7f0566e00640 1 -- 172.21.15.100:0/1744033666 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f053c079790 msgr2=0x7f053c07bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.488+0000 7f0566e00640 1 --2- 172.21.15.100:0/1744033666 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f053c079790 0x7f053c07bca0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0548033000 tx=0x7f05480046c0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.488+0000 7f0566e00640 1 -- 172.21.15.100:0/1744033666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0560070660 msgr2=0x7f0560155390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.488+0000 7f0566e00640 1 --2- 172.21.15.100:0/1744033666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0560070660 0x7f0560155390 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f055000c3d0 tx=0x7f055000c890 comp rx=0 tx=0).stop 2024-09-17T04:34:23.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.488+0000 7f0566e00640 1 -- 172.21.15.100:0/1744033666 shutdown_connections 2024-09-17T04:34:23.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.488+0000 7f0566e00640 1 --2- 172.21.15.100:0/1744033666 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f053c079790 0x7f053c07bca0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.488+0000 7f0566e00640 1 --2- 172.21.15.100:0/1744033666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05600af6a0 0x7f05601f9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.488+0000 7f0566e00640 1 --2- 172.21.15.100:0/1744033666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05600ad700 0x7f05601f5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.488+0000 7f0566e00640 1 --2- 172.21.15.100:0/1744033666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0560070660 0x7f0560155390 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.488+0000 7f0566e00640 1 -- 172.21.15.100:0/1744033666 >> 172.21.15.100:0/1744033666 conn(0x7f056006d3d0 msgr2=0x7f05600d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.488+0000 7f0566e00640 1 -- 172.21.15.100:0/1744033666 shutdown_connections 2024-09-17T04:34:23.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.488+0000 7f0566e00640 1 -- 172.21.15.100:0/1744033666 wait complete. 2024-09-17T04:34:23.510 INFO:tasks.ceph:Scrubbing osd.4 2024-09-17T04:34:23.511 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-17T04:34:23.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.733+0000 7fa839200640 1 -- 172.21.15.100:0/1359644475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8340af100 msgr2=0x7fa8340af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.733+0000 7fa839200640 1 --2- 172.21.15.100:0/1359644475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8340af100 0x7fa8340af510 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa82c005fd0 tx=0x7fa82c02a8b0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.733+0000 7fa839200640 1 -- 172.21.15.100:0/1359644475 shutdown_connections 2024-09-17T04:34:23.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.733+0000 7fa839200640 1 --2- 172.21.15.100:0/1359644475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8340af100 0x7fa8340af510 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.733+0000 7fa839200640 1 --2- 172.21.15.100:0/1359644475 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8340cc830 0x7fa8340b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.733+0000 7fa839200640 1 -- 172.21.15.100:0/1359644475 >> 172.21.15.100:0/1359644475 conn(0x7fa8340b2540 msgr2=0x7fa8340d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.733+0000 7fa839200640 1 -- 172.21.15.100:0/1359644475 shutdown_connections 2024-09-17T04:34:23.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.733+0000 7fa839200640 1 -- 172.21.15.100:0/1359644475 wait complete. 2024-09-17T04:34:23.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.735+0000 7fa839200640 1 Processor -- start 2024-09-17T04:34:23.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.735+0000 7fa839200640 1 -- start start 2024-09-17T04:34:23.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.737+0000 7fa839200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa83406cda0 0x7fa834098250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.737+0000 7fa839200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8340af100 0x7fa8340ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.737+0000 7fa839200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8340cc830 0x7fa8340ad620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.737+0000 7fa839200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa8340b3650 con 0x7fa8340af100 2024-09-17T04:34:23.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.737+0000 7fa839200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa8340b34d0 con 0x7fa8340cc830 2024-09-17T04:34:23.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.737+0000 7fa839200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa8340b37d0 con 0x7fa83406cda0 2024-09-17T04:34:23.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.737+0000 7fa831400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8340af100 0x7fa8340ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.737+0000 7fa831400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8340af100 0x7fa8340ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59808/0 (socket says 172.21.15.100:59808) 2024-09-17T04:34:23.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.737+0000 7fa831400640 1 -- 172.21.15.100:0/1819920416 learned_addr learned my addr 172.21.15.100:0/1819920416 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:23.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.737+0000 7fa831e00640 1 --2- 172.21.15.100:0/1819920416 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa83406cda0 0x7fa834098250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.737+0000 7fa832800640 1 --2- 172.21.15.100:0/1819920416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8340cc830 0x7fa8340ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.738+0000 7fa831400640 1 -- 172.21.15.100:0/1819920416 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa83406cda0 msgr2=0x7fa834098250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.738+0000 7fa831400640 1 --2- 172.21.15.100:0/1819920416 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa83406cda0 0x7fa834098250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.738+0000 7fa831400640 1 -- 172.21.15.100:0/1819920416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8340cc830 msgr2=0x7fa8340ad620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.738+0000 7fa831400640 1 --2- 172.21.15.100:0/1819920416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8340cc830 0x7fa8340ad620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.738+0000 7fa831400640 1 -- 172.21.15.100:0/1819920416 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8341f5b30 con 0x7fa8340af100 2024-09-17T04:34:23.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.738+0000 7fa831400640 1 --2- 172.21.15.100:0/1819920416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8340af100 0x7fa8340ad0e0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fa82000c430 tx=0x7fa82000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.738+0000 7fa82a000640 1 -- 172.21.15.100:0/1819920416 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820011790 con 0x7fa8340af100 2024-09-17T04:34:23.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.738+0000 7fa82a000640 1 -- 172.21.15.100:0/1819920416 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa820017070 con 0x7fa8340af100 2024-09-17T04:34:23.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.738+0000 7fa839200640 1 -- 172.21.15.100:0/1819920416 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8341f6060 con 0x7fa8340af100 2024-09-17T04:34:23.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.738+0000 7fa82a000640 1 -- 172.21.15.100:0/1819920416 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820011ce0 con 0x7fa8340af100 2024-09-17T04:34:23.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.738+0000 7fa839200640 1 -- 172.21.15.100:0/1819920416 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8341f6620 con 0x7fa8340af100 2024-09-17T04:34:23.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.739+0000 7fa839200640 1 -- 172.21.15.100:0/1819920416 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa8340b3cf0 con 0x7fa8340af100 2024-09-17T04:34:23.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.741+0000 7fa82a000640 1 -- 172.21.15.100:0/1819920416 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7fa82001c460 con 0x7fa8340af100 2024-09-17T04:34:23.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.742+0000 7fa82a000640 1 --2- 172.21.15.100:0/1819920416 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa814079790 0x7fa81407bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.742+0000 7fa82a000640 1 -- 172.21.15.100:0/1819920416 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4999+0+0 (secure 0 0 0) 0x7fa820099f30 con 0x7fa8340af100 2024-09-17T04:34:23.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.742+0000 7fa831e00640 1 --2- 172.21.15.100:0/1819920416 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa814079790 0x7fa81407bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.743+0000 7fa82a000640 1 --2- 172.21.15.100:0/1819920416 >> [v2:172.21.15.184:6810/985001108,v1:172.21.15.184:6811/985001108] conn(0x7fa81407f6a0 0x7fa814081b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.743+0000 7fa82a000640 1 -- 172.21.15.100:0/1819920416 --> [v2:172.21.15.184:6810/985001108,v1:172.21.15.184:6811/985001108] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa82000ae50 con 0x7fa81407f6a0 2024-09-17T04:34:23.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.743+0000 7fa82a000640 1 -- 172.21.15.100:0/1819920416 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=170) ==== 24+0+0 (secure 0 0 0) 0x7fa82009cc50 con 0x7fa8340af100 2024-09-17T04:34:23.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.743+0000 7fa831e00640 1 --2- 172.21.15.100:0/1819920416 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa814079790 0x7fa81407bca0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa824009b30 tx=0x7fa824005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.743+0000 7fa832800640 1 --2- 172.21.15.100:0/1819920416 >> [v2:172.21.15.184:6810/985001108,v1:172.21.15.184:6811/985001108] conn(0x7fa81407f6a0 0x7fa814081b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.744+0000 7fa832800640 1 --2- 172.21.15.100:0/1819920416 >> [v2:172.21.15.184:6810/985001108,v1:172.21.15.184:6811/985001108] conn(0x7fa81407f6a0 0x7fa814081b30 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.745+0000 7fa82a000640 1 -- 172.21.15.100:0/1819920416 <== osd.4 v2:172.21.15.184:6810/985001108 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa82000ae50 con 0x7fa81407f6a0 2024-09-17T04:34:23.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.758+0000 7fa839200640 1 -- 172.21.15.100:0/1819920416 --> [v2:172.21.15.184:6810/985001108,v1:172.21.15.184:6811/985001108] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa8341f5e10 con 0x7fa81407f6a0 2024-09-17T04:34:23.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.758+0000 7fa82a000640 1 -- 172.21.15.100:0/1819920416 <== osd.4 v2:172.21.15.184:6810/985001108 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa8341f5e10 con 0x7fa81407f6a0 2024-09-17T04:34:23.759 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:34:23.759 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T04:34:23.759 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:34:23.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.759+0000 7fa839200640 1 -- 172.21.15.100:0/1819920416 >> [v2:172.21.15.184:6810/985001108,v1:172.21.15.184:6811/985001108] conn(0x7fa81407f6a0 msgr2=0x7fa814081b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.759+0000 7fa839200640 1 --2- 172.21.15.100:0/1819920416 >> [v2:172.21.15.184:6810/985001108,v1:172.21.15.184:6811/985001108] conn(0x7fa81407f6a0 0x7fa814081b30 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.759+0000 7fa839200640 1 -- 172.21.15.100:0/1819920416 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa814079790 msgr2=0x7fa81407bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.759+0000 7fa839200640 1 --2- 172.21.15.100:0/1819920416 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fa814079790 0x7fa81407bca0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa824009b30 tx=0x7fa824005f90 comp rx=0 tx=0).stop 2024-09-17T04:34:23.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.759+0000 7fa839200640 1 -- 172.21.15.100:0/1819920416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8340af100 msgr2=0x7fa8340ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.759+0000 7fa839200640 1 --2- 172.21.15.100:0/1819920416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8340af100 0x7fa8340ad0e0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fa82000c430 tx=0x7fa82000c8f0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.759+0000 7fa832800640 1 -- 172.21.15.100:0/1819920416 reap_dead start 2024-09-17T04:34:23.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.759+0000 7fa839200640 1 -- 172.21.15.100:0/1819920416 shutdown_connections 2024-09-17T04:34:23.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.759+0000 7fa839200640 1 -- 172.21.15.100:0/1819920416 >> 172.21.15.100:0/1819920416 conn(0x7fa8340b2540 msgr2=0x7fa83415fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.759+0000 7fa839200640 1 -- 172.21.15.100:0/1819920416 shutdown_connections 2024-09-17T04:34:23.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:23.759+0000 7fa839200640 1 -- 172.21.15.100:0/1819920416 wait complete. 2024-09-17T04:34:23.783 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-17T04:34:24.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.004+0000 7fdf4a400640 1 -- 172.21.15.100:0/221150296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf4415ce30 msgr2=0x7fdf4415d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:24.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.004+0000 7fdf4a400640 1 --2- 172.21.15.100:0/221150296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf4415ce30 0x7fdf4415d240 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fdf440b4fa0 tx=0x7fdf3c02aa00 comp rx=0 tx=0).stop 2024-09-17T04:34:24.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.004+0000 7fdf4a400640 1 -- 172.21.15.100:0/221150296 shutdown_connections 2024-09-17T04:34:24.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.004+0000 7fdf4a400640 1 --2- 172.21.15.100:0/221150296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf4415ce30 0x7fdf4415d240 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:24.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.004+0000 7fdf4a400640 1 --2- 172.21.15.100:0/221150296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf44070cc0 0x7fdf44156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:24.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.004+0000 7fdf4a400640 1 -- 172.21.15.100:0/221150296 >> 172.21.15.100:0/221150296 conn(0x7fdf4406d3d0 msgr2=0x7fdf440b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:24.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.004+0000 7fdf4a400640 1 -- 172.21.15.100:0/221150296 shutdown_connections 2024-09-17T04:34:24.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.004+0000 7fdf4a400640 1 -- 172.21.15.100:0/221150296 wait complete. 2024-09-17T04:34:24.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.007+0000 7fdf4a400640 1 Processor -- start 2024-09-17T04:34:24.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.007+0000 7fdf4a400640 1 -- start start 2024-09-17T04:34:24.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.008+0000 7fdf4a400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf440cfc80 0x7fdf44155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:24.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.008+0000 7fdf4a400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf4415ce30 0x7fdf440c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:24.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.008+0000 7fdf4a400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf440c45b0 0x7fdf440c6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:24.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.008+0000 7fdf4a400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdf440cd050 con 0x7fdf440c45b0 2024-09-17T04:34:24.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.008+0000 7fdf4a400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdf440cced0 con 0x7fdf4415ce30 2024-09-17T04:34:24.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.008+0000 7fdf4a400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdf440cd1d0 con 0x7fdf440cfc80 2024-09-17T04:34:24.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.008+0000 7fdf43e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf440c45b0 0x7fdf440c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:24.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.008+0000 7fdf43e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf440c45b0 0x7fdf440c6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59820/0 (socket says 172.21.15.100:59820) 2024-09-17T04:34:24.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.009+0000 7fdf42a00640 1 --2- 172.21.15.100:0/180673077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf4415ce30 0x7fdf440c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:24.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.008+0000 7fdf43e00640 1 -- 172.21.15.100:0/180673077 learned_addr learned my addr 172.21.15.100:0/180673077 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:24.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.009+0000 7fdf43400640 1 --2- 172.21.15.100:0/180673077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf440cfc80 0x7fdf44155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:24.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.009+0000 7fdf43e00640 1 -- 172.21.15.100:0/180673077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf440cfc80 msgr2=0x7fdf44155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:24.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.009+0000 7fdf43e00640 1 --2- 172.21.15.100:0/180673077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf440cfc80 0x7fdf44155e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:24.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.009+0000 7fdf43e00640 1 -- 172.21.15.100:0/180673077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf4415ce30 msgr2=0x7fdf440c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:24.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.009+0000 7fdf43e00640 1 --2- 172.21.15.100:0/180673077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf4415ce30 0x7fdf440c60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:24.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.009+0000 7fdf43e00640 1 -- 172.21.15.100:0/180673077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf441f6ad0 con 0x7fdf440c45b0 2024-09-17T04:34:24.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.009+0000 7fdf43e00640 1 --2- 172.21.15.100:0/180673077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf440c45b0 0x7fdf440c6600 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c02aff0 tx=0x7fdf3c003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:24.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.010+0000 7fdf3b400640 1 -- 172.21.15.100:0/180673077 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c033050 con 0x7fdf440c45b0 2024-09-17T04:34:24.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.010+0000 7fdf3b400640 1 -- 172.21.15.100:0/180673077 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf3c004450 con 0x7fdf440c45b0 2024-09-17T04:34:24.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.010+0000 7fdf4a400640 1 -- 172.21.15.100:0/180673077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf441f6db0 con 0x7fdf440c45b0 2024-09-17T04:34:24.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.010+0000 7fdf3b400640 1 -- 172.21.15.100:0/180673077 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c038450 con 0x7fdf440c45b0 2024-09-17T04:34:24.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.010+0000 7fdf4a400640 1 -- 172.21.15.100:0/180673077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf441f7370 con 0x7fdf440c45b0 2024-09-17T04:34:24.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.011+0000 7fdf4a400640 1 -- 172.21.15.100:0/180673077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf441f6840 con 0x7fdf440c45b0 2024-09-17T04:34:24.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.012+0000 7fdf3b400640 1 -- 172.21.15.100:0/180673077 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7fdf3c036070 con 0x7fdf440c45b0 2024-09-17T04:34:24.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.014+0000 7fdf3b400640 1 --2- 172.21.15.100:0/180673077 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdf24079760 0x7fdf2407bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:24.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.014+0000 7fdf3b400640 1 -- 172.21.15.100:0/180673077 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4999+0+0 (secure 0 0 0) 0x7fdf3c0bad00 con 0x7fdf440c45b0 2024-09-17T04:34:24.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.014+0000 7fdf43400640 1 --2- 172.21.15.100:0/180673077 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdf24079760 0x7fdf2407bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:24.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.014+0000 7fdf43400640 1 --2- 172.21.15.100:0/180673077 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdf24079760 0x7fdf2407bc70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fdf4409e510 tx=0x7fdf34009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:24.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.018+0000 7fdf3b400640 1 -- 172.21.15.100:0/180673077 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fdf3c0842b0 con 0x7fdf440c45b0 2024-09-17T04:34:24.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.205+0000 7fdf4a400640 1 -- 172.21.15.100:0/180673077 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fdf440cd6f0 con 0x7fdf24079760 2024-09-17T04:34:24.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.205+0000 7fdf3b400640 1 -- 172.21.15.100:0/180673077 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fdf440cd6f0 con 0x7fdf24079760 2024-09-17T04:34:24.205 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 4 to deep-scrub 2024-09-17T04:34:24.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.208+0000 7fdf4a400640 1 -- 172.21.15.100:0/180673077 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdf24079760 msgr2=0x7fdf2407bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:24.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.208+0000 7fdf4a400640 1 --2- 172.21.15.100:0/180673077 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdf24079760 0x7fdf2407bc70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fdf4409e510 tx=0x7fdf34009040 comp rx=0 tx=0).stop 2024-09-17T04:34:24.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.208+0000 7fdf4a400640 1 -- 172.21.15.100:0/180673077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf440c45b0 msgr2=0x7fdf440c6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:24.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.208+0000 7fdf4a400640 1 --2- 172.21.15.100:0/180673077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf440c45b0 0x7fdf440c6600 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c02aff0 tx=0x7fdf3c003f90 comp rx=0 tx=0).stop 2024-09-17T04:34:24.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.208+0000 7fdf4a400640 1 -- 172.21.15.100:0/180673077 shutdown_connections 2024-09-17T04:34:24.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.208+0000 7fdf4a400640 1 --2- 172.21.15.100:0/180673077 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdf24079760 0x7fdf2407bc70 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:24.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.208+0000 7fdf4a400640 1 --2- 172.21.15.100:0/180673077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf440c45b0 0x7fdf440c6600 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:24.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.208+0000 7fdf4a400640 1 --2- 172.21.15.100:0/180673077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf4415ce30 0x7fdf440c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:24.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.208+0000 7fdf4a400640 1 --2- 172.21.15.100:0/180673077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf440cfc80 0x7fdf44155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:24.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.208+0000 7fdf4a400640 1 -- 172.21.15.100:0/180673077 >> 172.21.15.100:0/180673077 conn(0x7fdf4406d3d0 msgr2=0x7fdf440b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:24.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.208+0000 7fdf4a400640 1 -- 172.21.15.100:0/180673077 shutdown_connections 2024-09-17T04:34:24.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.208+0000 7fdf4a400640 1 -- 172.21.15.100:0/180673077 wait complete. 2024-09-17T04:34:24.234 INFO:tasks.ceph:Scrubbing osd.5 2024-09-17T04:34:24.234 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-17T04:34:24.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.456+0000 7fdce6e00640 1 -- 172.21.15.100:0/2118253514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdce00af100 msgr2=0x7fdce00af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:24.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.456+0000 7fdce6e00640 1 --2- 172.21.15.100:0/2118253514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdce00af100 0x7fdce00af510 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fdce00d0d00 tx=0x7fdcd802aac0 comp rx=0 tx=0).stop 2024-09-17T04:34:24.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.456+0000 7fdce6e00640 1 -- 172.21.15.100:0/2118253514 shutdown_connections 2024-09-17T04:34:24.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.456+0000 7fdce6e00640 1 --2- 172.21.15.100:0/2118253514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdce00af100 0x7fdce00af510 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:24.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.456+0000 7fdce6e00640 1 --2- 172.21.15.100:0/2118253514 >> v2:172.21.15.184:3300/0 conn(0x7fdce006d4b0 0x7fdce00b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:24.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.456+0000 7fdce6e00640 1 -- 172.21.15.100:0/2118253514 >> 172.21.15.100:0/2118253514 conn(0x7fdce00b2540 msgr2=0x7fdce00d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:24.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.456+0000 7fdce6e00640 1 -- 172.21.15.100:0/2118253514 shutdown_connections 2024-09-17T04:34:24.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.457+0000 7fdce6e00640 1 -- 172.21.15.100:0/2118253514 wait complete. 2024-09-17T04:34:24.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.459+0000 7fdce6e00640 1 Processor -- start 2024-09-17T04:34:24.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.459+0000 7fdce6e00640 1 -- start start 2024-09-17T04:34:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.460+0000 7fdce6e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdce00af100 0x7fdce0156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.460+0000 7fdce6e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdce00cc830 0x7fdce01f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.460+0000 7fdce6e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdce0155240 0x7fdce009d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.460+0000 7fdce6e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdce00b35f0 con 0x7fdce00cc830 2024-09-17T04:34:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.460+0000 7fdce6e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdce00b3470 con 0x7fdce00af100 2024-09-17T04:34:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.460+0000 7fdce6e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdce00b3770 con 0x7fdce0155240 2024-09-17T04:34:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.461+0000 7fdcdf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdce00cc830 0x7fdce01f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.461+0000 7fdcdf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdce00cc830 0x7fdce01f6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59838/0 (socket says 172.21.15.100:59838) 2024-09-17T04:34:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.461+0000 7fdcdf400640 1 -- 172.21.15.100:0/817500016 learned_addr learned my addr 172.21.15.100:0/817500016 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.461+0000 7fdcdfe00640 1 --2- 172.21.15.100:0/817500016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdce00af100 0x7fdce0156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.461+0000 7fdce4e00640 1 --2- 172.21.15.100:0/817500016 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdce0155240 0x7fdce009d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.461+0000 7fdcdf400640 1 -- 172.21.15.100:0/817500016 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdce0155240 msgr2=0x7fdce009d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.461+0000 7fdcdf400640 1 --2- 172.21.15.100:0/817500016 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdce0155240 0x7fdce009d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.461+0000 7fdcdf400640 1 -- 172.21.15.100:0/817500016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdce00af100 msgr2=0x7fdce0156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.461+0000 7fdcdf400640 1 --2- 172.21.15.100:0/817500016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdce00af100 0x7fdce0156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.461+0000 7fdcdf400640 1 -- 172.21.15.100:0/817500016 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce00c9d30 con 0x7fdce00cc830 2024-09-17T04:34:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.461+0000 7fdcdf400640 1 --2- 172.21.15.100:0/817500016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdce00cc830 0x7fdce01f6f70 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fdccc009fd0 tx=0x7fdccc00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.462+0000 7fdcd7e00640 1 -- 172.21.15.100:0/817500016 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc00ccc0 con 0x7fdce00cc830 2024-09-17T04:34:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.462+0000 7fdcd7e00640 1 -- 172.21.15.100:0/817500016 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdccc00ee70 con 0x7fdce00cc830 2024-09-17T04:34:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.462+0000 7fdcd7e00640 1 -- 172.21.15.100:0/817500016 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc00ce60 con 0x7fdce00cc830 2024-09-17T04:34:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.462+0000 7fdce6e00640 1 -- 172.21.15.100:0/817500016 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdce0153580 con 0x7fdce00cc830 2024-09-17T04:34:24.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.462+0000 7fdce6e00640 1 -- 172.21.15.100:0/817500016 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdce01f7990 con 0x7fdce00cc830 2024-09-17T04:34:24.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.463+0000 7fdce6e00640 1 -- 172.21.15.100:0/817500016 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdce00b3c90 con 0x7fdce00cc830 2024-09-17T04:34:24.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.464+0000 7fdcd7e00640 1 -- 172.21.15.100:0/817500016 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7fdccc005260 con 0x7fdce00cc830 2024-09-17T04:34:24.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.465+0000 7fdcd7e00640 1 --2- 172.21.15.100:0/817500016 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdcc0079790 0x7fdcc007bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:24.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.465+0000 7fdcd7e00640 1 -- 172.21.15.100:0/817500016 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4999+0+0 (secure 0 0 0) 0x7fdccc09a460 con 0x7fdce00cc830 2024-09-17T04:34:24.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.465+0000 7fdcdfe00640 1 --2- 172.21.15.100:0/817500016 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdcc0079790 0x7fdcc007bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:24.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.465+0000 7fdcd7e00640 1 --2- 172.21.15.100:0/817500016 >> [v2:172.21.15.184:6802/3708129583,v1:172.21.15.184:6803/3708129583] conn(0x7fdcc007f6a0 0x7fdcc0081b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:24.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.466+0000 7fdcd7e00640 1 -- 172.21.15.100:0/817500016 --> [v2:172.21.15.184:6802/3708129583,v1:172.21.15.184:6803/3708129583] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdccc016ec0 con 0x7fdcc007f6a0 2024-09-17T04:34:24.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.466+0000 7fdcd7e00640 1 -- 172.21.15.100:0/817500016 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=170) ==== 24+0+0 (secure 0 0 0) 0x7fdccc09d180 con 0x7fdce00cc830 2024-09-17T04:34:24.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.466+0000 7fdcdfe00640 1 --2- 172.21.15.100:0/817500016 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdcc0079790 0x7fdcc007bca0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0001720 tx=0x7fdcd0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:24.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.466+0000 7fdce4e00640 1 --2- 172.21.15.100:0/817500016 >> [v2:172.21.15.184:6802/3708129583,v1:172.21.15.184:6803/3708129583] conn(0x7fdcc007f6a0 0x7fdcc0081b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:24.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.467+0000 7fdce4e00640 1 --2- 172.21.15.100:0/817500016 >> [v2:172.21.15.184:6802/3708129583,v1:172.21.15.184:6803/3708129583] conn(0x7fdcc007f6a0 0x7fdcc0081b30 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:24.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.468+0000 7fdcd7e00640 1 -- 172.21.15.100:0/817500016 <== osd.5 v2:172.21.15.184:6802/3708129583 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdccc016ec0 con 0x7fdcc007f6a0 2024-09-17T04:34:24.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.481+0000 7fdce6e00640 1 -- 172.21.15.100:0/817500016 --> [v2:172.21.15.184:6802/3708129583,v1:172.21.15.184:6803/3708129583] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fdce0153330 con 0x7fdcc007f6a0 2024-09-17T04:34:24.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.482+0000 7fdcd7e00640 1 -- 172.21.15.100:0/817500016 <== osd.5 v2:172.21.15.184:6802/3708129583 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fdce0153330 con 0x7fdcc007f6a0 2024-09-17T04:34:24.483 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:34:24.483 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T04:34:24.483 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:34:24.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.483+0000 7fdce6e00640 1 -- 172.21.15.100:0/817500016 >> [v2:172.21.15.184:6802/3708129583,v1:172.21.15.184:6803/3708129583] conn(0x7fdcc007f6a0 msgr2=0x7fdcc0081b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:24.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.483+0000 7fdce6e00640 1 --2- 172.21.15.100:0/817500016 >> [v2:172.21.15.184:6802/3708129583,v1:172.21.15.184:6803/3708129583] conn(0x7fdcc007f6a0 0x7fdcc0081b30 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:24.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.483+0000 7fdce6e00640 1 -- 172.21.15.100:0/817500016 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdcc0079790 msgr2=0x7fdcc007bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:24.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.483+0000 7fdce6e00640 1 --2- 172.21.15.100:0/817500016 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fdcc0079790 0x7fdcc007bca0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0001720 tx=0x7fdcd0009040 comp rx=0 tx=0).stop 2024-09-17T04:34:24.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.483+0000 7fdce6e00640 1 -- 172.21.15.100:0/817500016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdce00cc830 msgr2=0x7fdce01f6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:24.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.483+0000 7fdce6e00640 1 --2- 172.21.15.100:0/817500016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdce00cc830 0x7fdce01f6f70 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fdccc009fd0 tx=0x7fdccc00ec20 comp rx=0 tx=0).stop 2024-09-17T04:34:24.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.483+0000 7fdce4e00640 1 -- 172.21.15.100:0/817500016 reap_dead start 2024-09-17T04:34:24.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.483+0000 7fdce6e00640 1 -- 172.21.15.100:0/817500016 shutdown_connections 2024-09-17T04:34:24.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.483+0000 7fdce6e00640 1 -- 172.21.15.100:0/817500016 >> 172.21.15.100:0/817500016 conn(0x7fdce00b2540 msgr2=0x7fdce0099690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:24.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.483+0000 7fdce6e00640 1 -- 172.21.15.100:0/817500016 shutdown_connections 2024-09-17T04:34:24.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.483+0000 7fdce6e00640 1 -- 172.21.15.100:0/817500016 wait complete. 2024-09-17T04:34:24.504 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-17T04:34:24.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.723+0000 7fad7c400640 1 -- 172.21.15.100:0/3750851391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad7415ce30 msgr2=0x7fad7415d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:24.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.723+0000 7fad7c400640 1 --2- 172.21.15.100:0/3750851391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad7415ce30 0x7fad7415d240 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fad740b4fa0 tx=0x7fad6c02a780 comp rx=0 tx=0).stop 2024-09-17T04:34:24.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.724+0000 7fad7c400640 1 -- 172.21.15.100:0/3750851391 shutdown_connections 2024-09-17T04:34:24.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.724+0000 7fad7c400640 1 --2- 172.21.15.100:0/3750851391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad7415ce30 0x7fad7415d240 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:24.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.724+0000 7fad7c400640 1 --2- 172.21.15.100:0/3750851391 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad740cfc80 0x7fad74156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:24.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.724+0000 7fad7c400640 1 -- 172.21.15.100:0/3750851391 >> 172.21.15.100:0/3750851391 conn(0x7fad7406d3d0 msgr2=0x7fad740b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:24.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.724+0000 7fad7c400640 1 -- 172.21.15.100:0/3750851391 shutdown_connections 2024-09-17T04:34:24.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.724+0000 7fad7c400640 1 -- 172.21.15.100:0/3750851391 wait complete. 2024-09-17T04:34:24.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.726+0000 7fad7c400640 1 Processor -- start 2024-09-17T04:34:24.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.726+0000 7fad7c400640 1 -- start start 2024-09-17T04:34:24.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.727+0000 7fad7c400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad740705b0 0x7fad741f4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:24.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.727+0000 7fad7c400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad740cfc80 0x7fad740b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:24.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.727+0000 7fad7c400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad7415ce30 0x7fad740b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:24.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.727+0000 7fad7c400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fad740cd050 con 0x7fad740cfc80 2024-09-17T04:34:24.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.727+0000 7fad7c400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fad740cced0 con 0x7fad740705b0 2024-09-17T04:34:24.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.727+0000 7fad7c400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fad740cd1d0 con 0x7fad7415ce30 2024-09-17T04:34:24.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.728+0000 7fad79000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad740cfc80 0x7fad740b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:24.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.728+0000 7fad79000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad740cfc80 0x7fad740b0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59842/0 (socket says 172.21.15.100:59842) 2024-09-17T04:34:24.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.728+0000 7fad79000640 1 -- 172.21.15.100:0/1149821994 learned_addr learned my addr 172.21.15.100:0/1149821994 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:24.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.728+0000 7fad79a00640 1 --2- 172.21.15.100:0/1149821994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad740705b0 0x7fad741f4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:24.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.728+0000 7fad7a400640 1 --2- 172.21.15.100:0/1149821994 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad7415ce30 0x7fad740b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:24.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.728+0000 7fad79000640 1 -- 172.21.15.100:0/1149821994 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad7415ce30 msgr2=0x7fad740b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:24.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.728+0000 7fad79000640 1 --2- 172.21.15.100:0/1149821994 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad7415ce30 0x7fad740b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:24.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.728+0000 7fad79000640 1 -- 172.21.15.100:0/1149821994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad740705b0 msgr2=0x7fad741f4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:24.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.728+0000 7fad79000640 1 --2- 172.21.15.100:0/1149821994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad740705b0 0x7fad741f4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:24.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.728+0000 7fad79000640 1 -- 172.21.15.100:0/1149821994 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad741f57d0 con 0x7fad740cfc80 2024-09-17T04:34:24.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.728+0000 7fad79000640 1 --2- 172.21.15.100:0/1149821994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad740cfc80 0x7fad740b0da0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fad6000c530 tx=0x7fad6000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:24.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.729+0000 7fad71600640 1 -- 172.21.15.100:0/1149821994 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad60016020 con 0x7fad740cfc80 2024-09-17T04:34:24.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.729+0000 7fad71600640 1 -- 172.21.15.100:0/1149821994 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad60012940 con 0x7fad740cfc80 2024-09-17T04:34:24.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.729+0000 7fad7c400640 1 -- 172.21.15.100:0/1149821994 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad741f5ab0 con 0x7fad740cfc80 2024-09-17T04:34:24.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.729+0000 7fad71600640 1 -- 172.21.15.100:0/1149821994 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad60012d90 con 0x7fad740cfc80 2024-09-17T04:34:24.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.729+0000 7fad7c400640 1 -- 172.21.15.100:0/1149821994 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad741f6070 con 0x7fad740cfc80 2024-09-17T04:34:24.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.730+0000 7fad7c400640 1 -- 172.21.15.100:0/1149821994 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad740cd710 con 0x7fad740cfc80 2024-09-17T04:34:24.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.731+0000 7fad71600640 1 -- 172.21.15.100:0/1149821994 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7fad60019020 con 0x7fad740cfc80 2024-09-17T04:34:24.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.732+0000 7fad71600640 1 --2- 172.21.15.100:0/1149821994 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fad54079790 0x7fad5407bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:24.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.732+0000 7fad71600640 1 -- 172.21.15.100:0/1149821994 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4999+0+0 (secure 0 0 0) 0x7fad600991b0 con 0x7fad740cfc80 2024-09-17T04:34:24.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.732+0000 7fad79a00640 1 --2- 172.21.15.100:0/1149821994 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fad54079790 0x7fad5407bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:24.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.733+0000 7fad79a00640 1 --2- 172.21.15.100:0/1149821994 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fad54079790 0x7fad5407bca0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fad680095f0 tx=0x7fad68005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:24.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.737+0000 7fad71600640 1 -- 172.21.15.100:0/1149821994 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fad60020070 con 0x7fad740cfc80 2024-09-17T04:34:24.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.921+0000 7fad7c400640 1 -- 172.21.15.100:0/1149821994 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fad740afa70 con 0x7fad54079790 2024-09-17T04:34:24.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.922+0000 7fad71600640 1 -- 172.21.15.100:0/1149821994 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fad740afa70 con 0x7fad54079790 2024-09-17T04:34:24.922 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 5 to deep-scrub 2024-09-17T04:34:24.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.924+0000 7fad7c400640 1 -- 172.21.15.100:0/1149821994 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fad54079790 msgr2=0x7fad5407bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:24.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.924+0000 7fad7c400640 1 --2- 172.21.15.100:0/1149821994 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fad54079790 0x7fad5407bca0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fad680095f0 tx=0x7fad68005f90 comp rx=0 tx=0).stop 2024-09-17T04:34:24.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.924+0000 7fad7c400640 1 -- 172.21.15.100:0/1149821994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad740cfc80 msgr2=0x7fad740b0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:24.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.924+0000 7fad7c400640 1 --2- 172.21.15.100:0/1149821994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad740cfc80 0x7fad740b0da0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fad6000c530 tx=0x7fad6000c9f0 comp rx=0 tx=0).stop 2024-09-17T04:34:24.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.924+0000 7fad7c400640 1 -- 172.21.15.100:0/1149821994 shutdown_connections 2024-09-17T04:34:24.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.924+0000 7fad7c400640 1 --2- 172.21.15.100:0/1149821994 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fad54079790 0x7fad5407bca0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:24.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.924+0000 7fad7c400640 1 --2- 172.21.15.100:0/1149821994 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad7415ce30 0x7fad740b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:24.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.924+0000 7fad7c400640 1 --2- 172.21.15.100:0/1149821994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad740cfc80 0x7fad740b0da0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:24.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.924+0000 7fad7c400640 1 --2- 172.21.15.100:0/1149821994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad740705b0 0x7fad741f4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:24.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.924+0000 7fad7c400640 1 -- 172.21.15.100:0/1149821994 >> 172.21.15.100:0/1149821994 conn(0x7fad7406d3d0 msgr2=0x7fad7415ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:24.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.924+0000 7fad7c400640 1 -- 172.21.15.100:0/1149821994 shutdown_connections 2024-09-17T04:34:24.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:24.924+0000 7fad7c400640 1 -- 172.21.15.100:0/1149821994 wait complete. 2024-09-17T04:34:24.946 INFO:tasks.ceph:Scrubbing osd.6 2024-09-17T04:34:24.946 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-17T04:34:25.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.170+0000 7f423f800640 1 -- 172.21.15.100:0/1827703193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f423815fd30 msgr2=0x7f42380b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:25.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.170+0000 7f423f800640 1 --2- 172.21.15.100:0/1827703193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f423815fd30 0x7f42380b4620 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f42380d02a0 tx=0x7f422802a560 comp rx=0 tx=0).stop 2024-09-17T04:34:25.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.170+0000 7f423f800640 1 -- 172.21.15.100:0/1827703193 shutdown_connections 2024-09-17T04:34:25.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.170+0000 7f423f800640 1 --2- 172.21.15.100:0/1827703193 >> v2:172.21.15.184:3300/0 conn(0x7f4238162ff0 0x7f4238158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.170+0000 7f423f800640 1 --2- 172.21.15.100:0/1827703193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f423815fd30 0x7f42380b4620 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.170+0000 7f423f800640 1 -- 172.21.15.100:0/1827703193 >> 172.21.15.100:0/1827703193 conn(0x7f42380b2540 msgr2=0x7f42380d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:25.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.170+0000 7f423f800640 1 -- 172.21.15.100:0/1827703193 shutdown_connections 2024-09-17T04:34:25.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.170+0000 7f423f800640 1 -- 172.21.15.100:0/1827703193 wait complete. 2024-09-17T04:34:25.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.172+0000 7f423f800640 1 Processor -- start 2024-09-17T04:34:25.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.172+0000 7f423f800640 1 -- start start 2024-09-17T04:34:25.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.174+0000 7f423f800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f423806cda0 0x7f423814e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:25.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.174+0000 7f423f800640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f423815fd30 0x7f42380cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:25.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.174+0000 7f423f800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4238162ff0 0x7f42380cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:25.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.174+0000 7f423f800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f42380af490 con 0x7f4238162ff0 2024-09-17T04:34:25.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.174+0000 7f423f800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f42380af310 con 0x7f423806cda0 2024-09-17T04:34:25.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.174+0000 7f423f800640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f42380af610 con 0x7f423815fd30 2024-09-17T04:34:25.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.174+0000 7f423d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4238162ff0 0x7f42380cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:25.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.174+0000 7f423d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4238162ff0 0x7f42380cc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34470/0 (socket says 172.21.15.100:34470) 2024-09-17T04:34:25.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.174+0000 7f423d800640 1 -- 172.21.15.100:0/3199802329 learned_addr learned my addr 172.21.15.100:0/3199802329 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:25.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.175+0000 7f423ce00640 1 --2- 172.21.15.100:0/3199802329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f423806cda0 0x7f423814e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:25.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.175+0000 7f422fe00640 1 --2- 172.21.15.100:0/3199802329 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f423815fd30 0x7f42380cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:25.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.175+0000 7f423d800640 1 -- 172.21.15.100:0/3199802329 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f423815fd30 msgr2=0x7f42380cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:25.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.175+0000 7f423d800640 1 --2- 172.21.15.100:0/3199802329 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f423815fd30 0x7f42380cc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.175+0000 7f423d800640 1 -- 172.21.15.100:0/3199802329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f423806cda0 msgr2=0x7f423814e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:25.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.175+0000 7f423d800640 1 --2- 172.21.15.100:0/3199802329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f423806cda0 0x7f423814e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.175+0000 7f423d800640 1 -- 172.21.15.100:0/3199802329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4238074140 con 0x7f4238162ff0 2024-09-17T04:34:25.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.175+0000 7f423d800640 1 --2- 172.21.15.100:0/3199802329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4238162ff0 0x7f42380cc9b0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f423000b9e0 tx=0x7f423000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:25.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.175+0000 7f4235600640 1 -- 172.21.15.100:0/3199802329 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4230003b70 con 0x7f4238162ff0 2024-09-17T04:34:25.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.176+0000 7f4235600640 1 -- 172.21.15.100:0/3199802329 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4230003e00 con 0x7f4238162ff0 2024-09-17T04:34:25.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.176+0000 7f4235600640 1 -- 172.21.15.100:0/3199802329 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423000cb80 con 0x7f4238162ff0 2024-09-17T04:34:25.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.176+0000 7f423f800640 1 -- 172.21.15.100:0/3199802329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4238074420 con 0x7f4238162ff0 2024-09-17T04:34:25.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.176+0000 7f423f800640 1 -- 172.21.15.100:0/3199802329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42380cbf50 con 0x7f4238162ff0 2024-09-17T04:34:25.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.176+0000 7f423f800640 1 -- 172.21.15.100:0/3199802329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f42380cd7b0 con 0x7f4238162ff0 2024-09-17T04:34:25.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.178+0000 7f4235600640 1 -- 172.21.15.100:0/3199802329 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7f4230004000 con 0x7f4238162ff0 2024-09-17T04:34:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.179+0000 7f4235600640 1 --2- 172.21.15.100:0/3199802329 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4214079790 0x7f421407bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.179+0000 7f4235600640 1 -- 172.21.15.100:0/3199802329 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4999+0+0 (secure 0 0 0) 0x7f4230099720 con 0x7f4238162ff0 2024-09-17T04:34:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.179+0000 7f423ce00640 1 --2- 172.21.15.100:0/3199802329 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4214079790 0x7f421407bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:25.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.179+0000 7f4235600640 1 --2- 172.21.15.100:0/3199802329 >> [v2:172.21.15.184:6825/3908254181,v1:172.21.15.184:6827/3908254181] conn(0x7f421407f6a0 0x7f4214081b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:25.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.179+0000 7f4235600640 1 -- 172.21.15.100:0/3199802329 --> [v2:172.21.15.184:6825/3908254181,v1:172.21.15.184:6827/3908254181] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4230012e50 con 0x7f421407f6a0 2024-09-17T04:34:25.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.179+0000 7f4235600640 1 -- 172.21.15.100:0/3199802329 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=170) ==== 24+0+0 (secure 0 0 0) 0x7f42300a3050 con 0x7f4238162ff0 2024-09-17T04:34:25.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.180+0000 7f423ce00640 1 --2- 172.21.15.100:0/3199802329 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4214079790 0x7f421407bca0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4228033000 tx=0x7f4228004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:25.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.180+0000 7f422fe00640 1 --2- 172.21.15.100:0/3199802329 >> [v2:172.21.15.184:6825/3908254181,v1:172.21.15.184:6827/3908254181] conn(0x7f421407f6a0 0x7f4214081b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:25.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.180+0000 7f422fe00640 1 --2- 172.21.15.100:0/3199802329 >> [v2:172.21.15.184:6825/3908254181,v1:172.21.15.184:6827/3908254181] conn(0x7f421407f6a0 0x7f4214081b30 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:25.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.181+0000 7f4235600640 1 -- 172.21.15.100:0/3199802329 <== osd.6 v2:172.21.15.184:6825/3908254181 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4230012e50 con 0x7f421407f6a0 2024-09-17T04:34:25.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.194+0000 7f423f800640 1 -- 172.21.15.100:0/3199802329 --> [v2:172.21.15.184:6825/3908254181,v1:172.21.15.184:6827/3908254181] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f42380afac0 con 0x7f421407f6a0 2024-09-17T04:34:25.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.195+0000 7f4235600640 1 -- 172.21.15.100:0/3199802329 <== osd.6 v2:172.21.15.184:6825/3908254181 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f42380afac0 con 0x7f421407f6a0 2024-09-17T04:34:25.196 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:34:25.196 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T04:34:25.196 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:34:25.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.196+0000 7f423f800640 1 -- 172.21.15.100:0/3199802329 >> [v2:172.21.15.184:6825/3908254181,v1:172.21.15.184:6827/3908254181] conn(0x7f421407f6a0 msgr2=0x7f4214081b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:25.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.196+0000 7f423f800640 1 --2- 172.21.15.100:0/3199802329 >> [v2:172.21.15.184:6825/3908254181,v1:172.21.15.184:6827/3908254181] conn(0x7f421407f6a0 0x7f4214081b30 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.196+0000 7f423f800640 1 -- 172.21.15.100:0/3199802329 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4214079790 msgr2=0x7f421407bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:25.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.196+0000 7f423f800640 1 --2- 172.21.15.100:0/3199802329 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f4214079790 0x7f421407bca0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4228033000 tx=0x7f4228004730 comp rx=0 tx=0).stop 2024-09-17T04:34:25.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.196+0000 7f423f800640 1 -- 172.21.15.100:0/3199802329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4238162ff0 msgr2=0x7f42380cc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:25.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.196+0000 7f423f800640 1 --2- 172.21.15.100:0/3199802329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4238162ff0 0x7f42380cc9b0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f423000b9e0 tx=0x7f423000bea0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.196+0000 7f423d800640 1 -- 172.21.15.100:0/3199802329 reap_dead start 2024-09-17T04:34:25.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.196+0000 7f423f800640 1 -- 172.21.15.100:0/3199802329 shutdown_connections 2024-09-17T04:34:25.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.196+0000 7f423f800640 1 -- 172.21.15.100:0/3199802329 >> 172.21.15.100:0/3199802329 conn(0x7f42380b2540 msgr2=0x7f42380adad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:25.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.196+0000 7f423f800640 1 -- 172.21.15.100:0/3199802329 shutdown_connections 2024-09-17T04:34:25.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.196+0000 7f423f800640 1 -- 172.21.15.100:0/3199802329 wait complete. 2024-09-17T04:34:25.218 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-17T04:34:25.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.442+0000 7f8332400640 1 -- 172.21.15.100:0/1878358960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8310004d40 msgr2=0x7f8310005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:25.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.442+0000 7f8332400640 1 --2- 172.21.15.100:0/1878358960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8310004d40 0x7f8310005150 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f8324005fd0 tx=0x7f8310005830 comp rx=0 tx=0).stop 2024-09-17T04:34:25.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.442+0000 7f8331a00640 1 -- 172.21.15.100:0/1878358960 reap_dead start 2024-09-17T04:34:25.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.442+0000 7f8332400640 1 -- 172.21.15.100:0/1878358960 shutdown_connections 2024-09-17T04:34:25.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.442+0000 7f8332400640 1 -- 172.21.15.100:0/1878358960 >> 172.21.15.100:0/1878358960 conn(0x7f832c0b1e40 msgr2=0x7f832c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:25.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.442+0000 7f8332400640 1 -- 172.21.15.100:0/1878358960 shutdown_connections 2024-09-17T04:34:25.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.442+0000 7f8332400640 1 -- 172.21.15.100:0/1878358960 wait complete. 2024-09-17T04:34:25.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.444+0000 7f8332400640 1 Processor -- start 2024-09-17T04:34:25.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.445+0000 7f8332400640 1 -- start start 2024-09-17T04:34:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.446+0000 7f8332400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8310004d40 0x7f832c09bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.446+0000 7f8332400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f832c09aa50 0x7f832c1592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.446+0000 7f8332400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f832c1f6430 0x7f832c1f6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.446+0000 7f8332400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f832c15d7f0 con 0x7f832c09aa50 2024-09-17T04:34:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.446+0000 7f8332400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f832c15d670 con 0x7f832c1f6430 2024-09-17T04:34:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.446+0000 7f8332400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f832c15d970 con 0x7f8310004d40 2024-09-17T04:34:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.446+0000 7f8323e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f832c09aa50 0x7f832c1592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.446+0000 7f8323e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f832c09aa50 0x7f832c1592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34488/0 (socket says 172.21.15.100:34488) 2024-09-17T04:34:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.446+0000 7f8323e00640 1 -- 172.21.15.100:0/1936840339 learned_addr learned my addr 172.21.15.100:0/1936840339 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.447+0000 7f8323e00640 1 -- 172.21.15.100:0/1936840339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8310004d40 msgr2=0x7f832c09bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.447+0000 7f8323e00640 1 --2- 172.21.15.100:0/1936840339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8310004d40 0x7f832c09bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.447+0000 7f8323e00640 1 -- 172.21.15.100:0/1936840339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f832c1f6430 msgr2=0x7f832c1f6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.447+0000 7f8323e00640 1 --2- 172.21.15.100:0/1936840339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f832c1f6430 0x7f832c1f6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.447+0000 7f8323e00640 1 -- 172.21.15.100:0/1936840339 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832c1f6d80 con 0x7f832c09aa50 2024-09-17T04:34:25.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.447+0000 7f8323e00640 1 --2- 172.21.15.100:0/1936840339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f832c09aa50 0x7f832c1592d0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f831400e6d0 tx=0x7f831400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:25.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.447+0000 7f8329600640 1 -- 172.21.15.100:0/1936840339 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314015020 con 0x7f832c09aa50 2024-09-17T04:34:25.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.447+0000 7f8329600640 1 -- 172.21.15.100:0/1936840339 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8314004320 con 0x7f832c09aa50 2024-09-17T04:34:25.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.447+0000 7f8332400640 1 -- 172.21.15.100:0/1936840339 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f832c156c00 con 0x7f832c09aa50 2024-09-17T04:34:25.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.447+0000 7f8329600640 1 -- 172.21.15.100:0/1936840339 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831400ca40 con 0x7f832c09aa50 2024-09-17T04:34:25.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.447+0000 7f8332400640 1 -- 172.21.15.100:0/1936840339 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f832c157340 con 0x7f832c09aa50 2024-09-17T04:34:25.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.449+0000 7f8332400640 1 -- 172.21.15.100:0/1936840339 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f832c156970 con 0x7f832c09aa50 2024-09-17T04:34:25.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.450+0000 7f8329600640 1 -- 172.21.15.100:0/1936840339 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7f8314018020 con 0x7f832c09aa50 2024-09-17T04:34:25.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.451+0000 7f8329600640 1 --2- 172.21.15.100:0/1936840339 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8308079760 0x7f830807bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:25.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.451+0000 7f8329600640 1 -- 172.21.15.100:0/1936840339 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4999+0+0 (secure 0 0 0) 0x7f831409a0a0 con 0x7f832c09aa50 2024-09-17T04:34:25.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.451+0000 7f8331000640 1 --2- 172.21.15.100:0/1936840339 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8308079760 0x7f830807bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:25.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.452+0000 7f8331000640 1 --2- 172.21.15.100:0/1936840339 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8308079760 0x7f830807bc70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f831c009b50 tx=0x7f831c009f50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:25.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.456+0000 7f8329600640 1 -- 172.21.15.100:0/1936840339 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f831401f020 con 0x7f832c09aa50 2024-09-17T04:34:25.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.637+0000 7f8332400640 1 -- 172.21.15.100:0/1936840339 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f832c0af170 con 0x7f8308079760 2024-09-17T04:34:25.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.637+0000 7f8329600640 1 -- 172.21.15.100:0/1936840339 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f832c0af170 con 0x7f8308079760 2024-09-17T04:34:25.638 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 6 to deep-scrub 2024-09-17T04:34:25.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.639+0000 7f8332400640 1 -- 172.21.15.100:0/1936840339 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8308079760 msgr2=0x7f830807bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:25.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.639+0000 7f8332400640 1 --2- 172.21.15.100:0/1936840339 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8308079760 0x7f830807bc70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f831c009b50 tx=0x7f831c009f50 comp rx=0 tx=0).stop 2024-09-17T04:34:25.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.639+0000 7f8332400640 1 -- 172.21.15.100:0/1936840339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f832c09aa50 msgr2=0x7f832c1592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:25.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.639+0000 7f8332400640 1 --2- 172.21.15.100:0/1936840339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f832c09aa50 0x7f832c1592d0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f831400e6d0 tx=0x7f831400eb90 comp rx=0 tx=0).stop 2024-09-17T04:34:25.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.640+0000 7f8332400640 1 -- 172.21.15.100:0/1936840339 shutdown_connections 2024-09-17T04:34:25.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.640+0000 7f8332400640 1 --2- 172.21.15.100:0/1936840339 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f8308079760 0x7f830807bc70 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.640+0000 7f8332400640 1 --2- 172.21.15.100:0/1936840339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f832c1f6430 0x7f832c1f6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.640+0000 7f8332400640 1 --2- 172.21.15.100:0/1936840339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f832c09aa50 0x7f832c1592d0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.640+0000 7f8332400640 1 --2- 172.21.15.100:0/1936840339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8310004d40 0x7f832c09bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.640+0000 7f8332400640 1 -- 172.21.15.100:0/1936840339 >> 172.21.15.100:0/1936840339 conn(0x7f832c0b1e40 msgr2=0x7f832c070530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:25.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.640+0000 7f8332400640 1 -- 172.21.15.100:0/1936840339 shutdown_connections 2024-09-17T04:34:25.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.640+0000 7f8332400640 1 -- 172.21.15.100:0/1936840339 wait complete. 2024-09-17T04:34:25.662 INFO:tasks.ceph:Scrubbing osd.7 2024-09-17T04:34:25.662 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-17T04:34:25.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.886+0000 7f2b99000640 1 -- 172.21.15.100:0/3874176196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b9406d4b0 msgr2=0x7f2b940b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:25.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.886+0000 7f2b99000640 1 --2- 172.21.15.100:0/3874176196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b9406d4b0 0x7f2b940b4620 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2b80009830 tx=0x7f2b8002a520 comp rx=0 tx=0).stop 2024-09-17T04:34:25.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.886+0000 7f2b99000640 1 -- 172.21.15.100:0/3874176196 shutdown_connections 2024-09-17T04:34:25.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.886+0000 7f2b99000640 1 --2- 172.21.15.100:0/3874176196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b940af100 0x7f2b940af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.886+0000 7f2b99000640 1 --2- 172.21.15.100:0/3874176196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b9406d4b0 0x7f2b940b4620 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.886+0000 7f2b99000640 1 -- 172.21.15.100:0/3874176196 >> 172.21.15.100:0/3874176196 conn(0x7f2b940b2540 msgr2=0x7f2b940d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:25.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.886+0000 7f2b99000640 1 -- 172.21.15.100:0/3874176196 shutdown_connections 2024-09-17T04:34:25.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.886+0000 7f2b99000640 1 -- 172.21.15.100:0/3874176196 wait complete. 2024-09-17T04:34:25.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.888+0000 7f2b99000640 1 Processor -- start 2024-09-17T04:34:25.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.888+0000 7f2b99000640 1 -- start start 2024-09-17T04:34:25.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.890+0000 7f2b99000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b940af100 0x7f2b9409bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:25.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.890+0000 7f2b99000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b940cc830 0x7f2b941559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:25.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.890+0000 7f2b99000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b940c32c0 0x7f2b940ad680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:25.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.890+0000 7f2b99000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2b940b36a0 con 0x7f2b940c32c0 2024-09-17T04:34:25.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.890+0000 7f2b99000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2b940b3520 con 0x7f2b940cc830 2024-09-17T04:34:25.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.890+0000 7f2b99000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2b940b3820 con 0x7f2b940af100 2024-09-17T04:34:25.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.890+0000 7f2b92800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b940c32c0 0x7f2b940ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:25.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.890+0000 7f2b92800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b940c32c0 0x7f2b940ad680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34500/0 (socket says 172.21.15.100:34500) 2024-09-17T04:34:25.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.890+0000 7f2b92800640 1 -- 172.21.15.100:0/1710442620 learned_addr learned my addr 172.21.15.100:0/1710442620 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:25.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.890+0000 7f2b91e00640 1 --2- 172.21.15.100:0/1710442620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b940af100 0x7f2b9409bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:25.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.890+0000 7f2b91400640 1 --2- 172.21.15.100:0/1710442620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b940cc830 0x7f2b941559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:25.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.890+0000 7f2b92800640 1 -- 172.21.15.100:0/1710442620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b940af100 msgr2=0x7f2b9409bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:25.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.890+0000 7f2b92800640 1 --2- 172.21.15.100:0/1710442620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b940af100 0x7f2b9409bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.890+0000 7f2b92800640 1 -- 172.21.15.100:0/1710442620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b940cc830 msgr2=0x7f2b941559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:25.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.890+0000 7f2b92800640 1 --2- 172.21.15.100:0/1710442620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b940cc830 0x7f2b941559b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.890+0000 7f2b92800640 1 -- 172.21.15.100:0/1710442620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b94152fa0 con 0x7f2b940c32c0 2024-09-17T04:34:25.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.890+0000 7f2b91400640 1 --2- 172.21.15.100:0/1710442620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b940cc830 0x7f2b941559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:25.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.891+0000 7f2b92800640 1 --2- 172.21.15.100:0/1710442620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b940c32c0 0x7f2b940ad680 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c00ebf0 tx=0x7f2b8c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:25.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.891+0000 7f2b8a000640 1 -- 172.21.15.100:0/1710442620 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c003b70 con 0x7f2b940c32c0 2024-09-17T04:34:25.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.891+0000 7f2b8a000640 1 -- 172.21.15.100:0/1710442620 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b8c003d10 con 0x7f2b940c32c0 2024-09-17T04:34:25.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.891+0000 7f2b99000640 1 -- 172.21.15.100:0/1710442620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b9409c250 con 0x7f2b940c32c0 2024-09-17T04:34:25.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.891+0000 7f2b8a000640 1 -- 172.21.15.100:0/1710442620 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c004280 con 0x7f2b940c32c0 2024-09-17T04:34:25.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.891+0000 7f2b99000640 1 -- 172.21.15.100:0/1710442620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b9406f630 con 0x7f2b940c32c0 2024-09-17T04:34:25.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.892+0000 7f2b99000640 1 -- 172.21.15.100:0/1710442620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2b94156f90 con 0x7f2b940c32c0 2024-09-17T04:34:25.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.893+0000 7f2b8a000640 1 -- 172.21.15.100:0/1710442620 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7f2b8c004420 con 0x7f2b940c32c0 2024-09-17T04:34:25.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.894+0000 7f2b8a000640 1 --2- 172.21.15.100:0/1710442620 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2b74079760 0x7f2b7407bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:25.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.894+0000 7f2b8a000640 1 -- 172.21.15.100:0/1710442620 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4999+0+0 (secure 0 0 0) 0x7f2b8c09a310 con 0x7f2b940c32c0 2024-09-17T04:34:25.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.894+0000 7f2b91e00640 1 --2- 172.21.15.100:0/1710442620 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2b74079760 0x7f2b7407bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:25.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.894+0000 7f2b8a000640 1 --2- 172.21.15.100:0/1710442620 >> [v2:172.21.15.184:6818/366777084,v1:172.21.15.184:6819/366777084] conn(0x7f2b7407f670 0x7f2b74081b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:25.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.894+0000 7f2b8a000640 1 -- 172.21.15.100:0/1710442620 --> [v2:172.21.15.184:6818/366777084,v1:172.21.15.184:6819/366777084] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2b8c01de50 con 0x7f2b7407f670 2024-09-17T04:34:25.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.894+0000 7f2b8a000640 1 -- 172.21.15.100:0/1710442620 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=170) ==== 24+0+0 (secure 0 0 0) 0x7f2b8c0a12a0 con 0x7f2b940c32c0 2024-09-17T04:34:25.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.895+0000 7f2b91e00640 1 --2- 172.21.15.100:0/1710442620 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2b74079760 0x7f2b7407bc70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2b84007fd0 tx=0x7f2b84001370 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:25.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.895+0000 7f2b91400640 1 --2- 172.21.15.100:0/1710442620 >> [v2:172.21.15.184:6818/366777084,v1:172.21.15.184:6819/366777084] conn(0x7f2b7407f670 0x7f2b74081b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:25.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.896+0000 7f2b91400640 1 --2- 172.21.15.100:0/1710442620 >> [v2:172.21.15.184:6818/366777084,v1:172.21.15.184:6819/366777084] conn(0x7f2b7407f670 0x7f2b74081b00 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:25.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.897+0000 7f2b8a000640 1 -- 172.21.15.100:0/1710442620 <== osd.7 v2:172.21.15.184:6818/366777084 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2b8c01de50 con 0x7f2b7407f670 2024-09-17T04:34:25.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.910+0000 7f2b99000640 1 -- 172.21.15.100:0/1710442620 --> [v2:172.21.15.184:6818/366777084,v1:172.21.15.184:6819/366777084] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f2b940b3d40 con 0x7f2b7407f670 2024-09-17T04:34:25.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.910+0000 7f2b8a000640 1 -- 172.21.15.100:0/1710442620 <== osd.7 v2:172.21.15.184:6818/366777084 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f2b940b3d40 con 0x7f2b7407f670 2024-09-17T04:34:25.911 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-17T04:34:25.911 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T04:34:25.911 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-17T04:34:25.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.911+0000 7f2b99000640 1 -- 172.21.15.100:0/1710442620 >> [v2:172.21.15.184:6818/366777084,v1:172.21.15.184:6819/366777084] conn(0x7f2b7407f670 msgr2=0x7f2b74081b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:25.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.911+0000 7f2b99000640 1 --2- 172.21.15.100:0/1710442620 >> [v2:172.21.15.184:6818/366777084,v1:172.21.15.184:6819/366777084] conn(0x7f2b7407f670 0x7f2b74081b00 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.911+0000 7f2b99000640 1 -- 172.21.15.100:0/1710442620 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2b74079760 msgr2=0x7f2b7407bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:25.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.911+0000 7f2b99000640 1 --2- 172.21.15.100:0/1710442620 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f2b74079760 0x7f2b7407bc70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2b84007fd0 tx=0x7f2b84001370 comp rx=0 tx=0).stop 2024-09-17T04:34:25.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.911+0000 7f2b99000640 1 -- 172.21.15.100:0/1710442620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b940c32c0 msgr2=0x7f2b940ad680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:25.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.911+0000 7f2b99000640 1 --2- 172.21.15.100:0/1710442620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b940c32c0 0x7f2b940ad680 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c00ebf0 tx=0x7f2b8c00c6a0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.911+0000 7f2b92800640 1 -- 172.21.15.100:0/1710442620 reap_dead start 2024-09-17T04:34:25.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.911+0000 7f2b99000640 1 -- 172.21.15.100:0/1710442620 shutdown_connections 2024-09-17T04:34:25.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.911+0000 7f2b99000640 1 -- 172.21.15.100:0/1710442620 >> 172.21.15.100:0/1710442620 conn(0x7f2b940b2540 msgr2=0x7f2b940997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:25.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.911+0000 7f2b99000640 1 -- 172.21.15.100:0/1710442620 shutdown_connections 2024-09-17T04:34:25.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:25.911+0000 7f2b99000640 1 -- 172.21.15.100:0/1710442620 wait complete. 2024-09-17T04:34:25.937 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-17T04:34:26.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.156+0000 7f7f23e00640 1 -- 172.21.15.100:0/4173939466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f1c0af8d0 msgr2=0x7f7f1c070570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:26.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.156+0000 7f7f23e00640 1 --2- 172.21.15.100:0/4173939466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f1c0af8d0 0x7f7f1c070570 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f7f08005fd0 tx=0x7f7f0802a540 comp rx=0 tx=0).stop 2024-09-17T04:34:26.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.156+0000 7f7f23e00640 1 -- 172.21.15.100:0/4173939466 shutdown_connections 2024-09-17T04:34:26.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.156+0000 7f7f23e00640 1 --2- 172.21.15.100:0/4173939466 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f1c070ab0 0x7f7f1c0b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:26.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.156+0000 7f7f23e00640 1 --2- 172.21.15.100:0/4173939466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f1c0af8d0 0x7f7f1c070570 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:26.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.156+0000 7f7f23e00640 1 -- 172.21.15.100:0/4173939466 >> 172.21.15.100:0/4173939466 conn(0x7f7f1c06d3d0 msgr2=0x7f7f1c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:26.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.156+0000 7f7f23e00640 1 -- 172.21.15.100:0/4173939466 shutdown_connections 2024-09-17T04:34:26.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.157+0000 7f7f23e00640 1 -- 172.21.15.100:0/4173939466 wait complete. 2024-09-17T04:34:26.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.159+0000 7f7f23e00640 1 Processor -- start 2024-09-17T04:34:26.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.159+0000 7f7f23e00640 1 -- start start 2024-09-17T04:34:26.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.160+0000 7f7f23e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f1c070ab0 0x7f7f1c0b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:26.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.160+0000 7f7f23e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f1c0af8d0 0x7f7f1c0984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:26.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.160+0000 7f7f23e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f1c0cbfc0 0x7f7f1c0ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:26.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.160+0000 7f7f23e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c0b3e20 con 0x7f7f1c070ab0 2024-09-17T04:34:26.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.160+0000 7f7f23e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c0b3ca0 con 0x7f7f1c0af8d0 2024-09-17T04:34:26.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.160+0000 7f7f23e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7f1c0b3fa0 con 0x7f7f1c0cbfc0 2024-09-17T04:34:26.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.160+0000 7f7f21400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f1c070ab0 0x7f7f1c0b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:26.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.160+0000 7f7f21400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f1c070ab0 0x7f7f1c0b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34522/0 (socket says 172.21.15.100:34522) 2024-09-17T04:34:26.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.161+0000 7f7f21400640 1 -- 172.21.15.100:0/2018226143 learned_addr learned my addr 172.21.15.100:0/2018226143 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:26.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.161+0000 7f7f20a00640 1 --2- 172.21.15.100:0/2018226143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f1c0af8d0 0x7f7f1c0984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:26.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.161+0000 7f7f21e00640 1 --2- 172.21.15.100:0/2018226143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f1c0cbfc0 0x7f7f1c0ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:26.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.161+0000 7f7f21400640 1 -- 172.21.15.100:0/2018226143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f1c0cbfc0 msgr2=0x7f7f1c0ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:26.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.161+0000 7f7f21400640 1 --2- 172.21.15.100:0/2018226143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f1c0cbfc0 0x7f7f1c0ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:26.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.161+0000 7f7f21400640 1 -- 172.21.15.100:0/2018226143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f1c0af8d0 msgr2=0x7f7f1c0984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:26.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.161+0000 7f7f21400640 1 --2- 172.21.15.100:0/2018226143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f1c0af8d0 0x7f7f1c0984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:26.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.161+0000 7f7f21400640 1 -- 172.21.15.100:0/2018226143 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1c0989f0 con 0x7f7f1c070ab0 2024-09-17T04:34:26.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.161+0000 7f7f21400640 1 --2- 172.21.15.100:0/2018226143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f1c070ab0 0x7f7f1c0b20f0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000c3d0 tx=0x7f7f1000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:26.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.161+0000 7f7f19600640 1 -- 172.21.15.100:0/2018226143 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10017070 con 0x7f7f1c070ab0 2024-09-17T04:34:26.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.162+0000 7f7f19600640 1 -- 172.21.15.100:0/2018226143 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f100053c0 con 0x7f7f1c070ab0 2024-09-17T04:34:26.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.162+0000 7f7f23e00640 1 -- 172.21.15.100:0/2018226143 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f1c098ce0 con 0x7f7f1c070ab0 2024-09-17T04:34:26.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.162+0000 7f7f19600640 1 -- 172.21.15.100:0/2018226143 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10003850 con 0x7f7f1c070ab0 2024-09-17T04:34:26.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.162+0000 7f7f23e00640 1 -- 172.21.15.100:0/2018226143 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f1c0992a0 con 0x7f7f1c070ab0 2024-09-17T04:34:26.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.163+0000 7f7f23e00640 1 -- 172.21.15.100:0/2018226143 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f1c0b3550 con 0x7f7f1c070ab0 2024-09-17T04:34:26.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.164+0000 7f7f19600640 1 -- 172.21.15.100:0/2018226143 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7f7f1001a070 con 0x7f7f1c070ab0 2024-09-17T04:34:26.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.165+0000 7f7f19600640 1 --2- 172.21.15.100:0/2018226143 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7efc079760 0x7f7efc07bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:26.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.165+0000 7f7f19600640 1 -- 172.21.15.100:0/2018226143 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4999+0+0 (secure 0 0 0) 0x7f7f10098cf0 con 0x7f7f1c070ab0 2024-09-17T04:34:26.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.165+0000 7f7f20a00640 1 --2- 172.21.15.100:0/2018226143 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7efc079760 0x7f7efc07bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:26.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.166+0000 7f7f20a00640 1 --2- 172.21.15.100:0/2018226143 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7efc079760 0x7f7efc07bc70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7f08033000 tx=0x7f7f08004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:26.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.169+0000 7f7f19600640 1 -- 172.21.15.100:0/2018226143 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f7f1c070ec0 con 0x7f7f1c070ab0 2024-09-17T04:34:26.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.354+0000 7f7f23e00640 1 -- 172.21.15.100:0/2018226143 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f7f1c0b4c80 con 0x7f7efc079760 2024-09-17T04:34:26.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.354+0000 7f7f19600640 1 -- 172.21.15.100:0/2018226143 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f7f1c0b4c80 con 0x7f7efc079760 2024-09-17T04:34:26.355 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 7 to deep-scrub 2024-09-17T04:34:26.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.356+0000 7f7f23e00640 1 -- 172.21.15.100:0/2018226143 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7efc079760 msgr2=0x7f7efc07bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:26.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.356+0000 7f7f23e00640 1 --2- 172.21.15.100:0/2018226143 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7efc079760 0x7f7efc07bc70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7f08033000 tx=0x7f7f08004730 comp rx=0 tx=0).stop 2024-09-17T04:34:26.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.356+0000 7f7f23e00640 1 -- 172.21.15.100:0/2018226143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f1c070ab0 msgr2=0x7f7f1c0b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:26.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.356+0000 7f7f23e00640 1 --2- 172.21.15.100:0/2018226143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f1c070ab0 0x7f7f1c0b20f0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000c3d0 tx=0x7f7f1000c890 comp rx=0 tx=0).stop 2024-09-17T04:34:26.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.356+0000 7f7f23e00640 1 -- 172.21.15.100:0/2018226143 shutdown_connections 2024-09-17T04:34:26.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.356+0000 7f7f23e00640 1 --2- 172.21.15.100:0/2018226143 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f7efc079760 0x7f7efc07bc70 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:26.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.357+0000 7f7f23e00640 1 --2- 172.21.15.100:0/2018226143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f1c0cbfc0 0x7f7f1c0ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:26.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.357+0000 7f7f23e00640 1 --2- 172.21.15.100:0/2018226143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f1c0af8d0 0x7f7f1c0984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:26.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.357+0000 7f7f23e00640 1 --2- 172.21.15.100:0/2018226143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f1c070ab0 0x7f7f1c0b20f0 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:26.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.357+0000 7f7f23e00640 1 -- 172.21.15.100:0/2018226143 >> 172.21.15.100:0/2018226143 conn(0x7f7f1c06d3d0 msgr2=0x7f7f1c074b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:26.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.357+0000 7f7f23e00640 1 -- 172.21.15.100:0/2018226143 shutdown_connections 2024-09-17T04:34:26.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.357+0000 7f7f23e00640 1 -- 172.21.15.100:0/2018226143 wait complete. 2024-09-17T04:34:26.380 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:34:26.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.598+0000 7f9564e00640 1 -- 172.21.15.100:0/4109089718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95600b2f60 msgr2=0x7f956015a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:26.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.598+0000 7f9564e00640 1 --2- 172.21.15.100:0/4109089718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95600b2f60 0x7f956015a0f0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f95600aea00 tx=0x7f954c02a4d0 comp rx=0 tx=0).stop 2024-09-17T04:34:26.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.598+0000 7f9564e00640 1 -- 172.21.15.100:0/4109089718 shutdown_connections 2024-09-17T04:34:26.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.598+0000 7f9564e00640 1 --2- 172.21.15.100:0/4109089718 >> v2:172.21.15.184:3300/0 conn(0x7f956015d670 0x7f9560070530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:26.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.598+0000 7f9564e00640 1 --2- 172.21.15.100:0/4109089718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95600b2f60 0x7f956015a0f0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:26.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.598+0000 7f9564e00640 1 --2- 172.21.15.100:0/4109089718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95600ad9c0 0x7f95600af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:26.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.598+0000 7f9564e00640 1 -- 172.21.15.100:0/4109089718 >> 172.21.15.100:0/4109089718 conn(0x7f95600b1e40 msgr2=0x7f95600b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:26.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.598+0000 7f9564e00640 1 -- 172.21.15.100:0/4109089718 shutdown_connections 2024-09-17T04:34:26.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.598+0000 7f9564e00640 1 -- 172.21.15.100:0/4109089718 wait complete. 2024-09-17T04:34:26.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.601+0000 7f9564e00640 1 Processor -- start 2024-09-17T04:34:26.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.601+0000 7f9564e00640 1 -- start start 2024-09-17T04:34:26.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.602+0000 7f9564e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95600ad9c0 0x7f9560145b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:26.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.602+0000 7f9564e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95600b2f60 0x7f95600cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:26.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.602+0000 7f9564e00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f956015d670 0x7f95600ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:26.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.602+0000 7f9564e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f956006f290 con 0x7f95600b2f60 2024-09-17T04:34:26.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.602+0000 7f9564e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9560070c80 con 0x7f95600ad9c0 2024-09-17T04:34:26.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.602+0000 7f9564e00640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f956006f410 con 0x7f956015d670 2024-09-17T04:34:26.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.602+0000 7f955ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95600b2f60 0x7f95600cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:26.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.603+0000 7f955ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95600b2f60 0x7f95600cdb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34534/0 (socket says 172.21.15.100:34534) 2024-09-17T04:34:26.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.603+0000 7f955ea00640 1 -- 172.21.15.100:0/4264966265 learned_addr learned my addr 172.21.15.100:0/4264966265 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:26.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.603+0000 7f955f400640 1 --2- 172.21.15.100:0/4264966265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95600ad9c0 0x7f9560145b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:26.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.603+0000 7f955fe00640 1 --2- 172.21.15.100:0/4264966265 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f956015d670 0x7f95600ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:26.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.603+0000 7f955ea00640 1 -- 172.21.15.100:0/4264966265 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f956015d670 msgr2=0x7f95600ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:26.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.603+0000 7f955ea00640 1 --2- 172.21.15.100:0/4264966265 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f956015d670 0x7f95600ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:26.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.603+0000 7f955ea00640 1 -- 172.21.15.100:0/4264966265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95600ad9c0 msgr2=0x7f9560145b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:26.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.603+0000 7f955ea00640 1 --2- 172.21.15.100:0/4264966265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95600ad9c0 0x7f9560145b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:26.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.603+0000 7f955ea00640 1 -- 172.21.15.100:0/4264966265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9560146080 con 0x7f95600b2f60 2024-09-17T04:34:26.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.603+0000 7f955fe00640 1 --2- 172.21.15.100:0/4264966265 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f956015d670 0x7f95600ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:26.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.603+0000 7f955ea00640 1 --2- 172.21.15.100:0/4264966265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95600b2f60 0x7f95600cdb10 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f954c034040 tx=0x7f954c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:26.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.603+0000 7f9557400640 1 -- 172.21.15.100:0/4264966265 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c03e030 con 0x7f95600b2f60 2024-09-17T04:34:26.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.604+0000 7f9557400640 1 -- 172.21.15.100:0/4264966265 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f954c003d10 con 0x7f95600b2f60 2024-09-17T04:34:26.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.604+0000 7f9564e00640 1 -- 172.21.15.100:0/4264966265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95601ebd40 con 0x7f95600b2f60 2024-09-17T04:34:26.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.604+0000 7f9557400640 1 -- 172.21.15.100:0/4264966265 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c035040 con 0x7f95600b2f60 2024-09-17T04:34:26.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.604+0000 7f9564e00640 1 -- 172.21.15.100:0/4264966265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9560158250 con 0x7f95600b2f60 2024-09-17T04:34:26.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.605+0000 7f9564e00640 1 -- 172.21.15.100:0/4264966265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95600acf40 con 0x7f95600b2f60 2024-09-17T04:34:26.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.606+0000 7f9557400640 1 -- 172.21.15.100:0/4264966265 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7f954c041030 con 0x7f95600b2f60 2024-09-17T04:34:26.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.607+0000 7f9557400640 1 --2- 172.21.15.100:0/4264966265 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9540079790 0x7f954007bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:26.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.607+0000 7f9557400640 1 -- 172.21.15.100:0/4264966265 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4999+0+0 (secure 0 0 0) 0x7f954c0ba070 con 0x7f95600b2f60 2024-09-17T04:34:26.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.607+0000 7f955f400640 1 --2- 172.21.15.100:0/4264966265 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9540079790 0x7f954007bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:26.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.608+0000 7f955f400640 1 --2- 172.21.15.100:0/4264966265 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9540079790 0x7f954007bca0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9550005fd0 tx=0x7f9550009500 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:26.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.611+0000 7f9557400640 1 -- 172.21.15.100:0/4264966265 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f95600acf40 con 0x7f95600b2f60 2024-09-17T04:34:26.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.791+0000 7f9564e00640 1 -- 172.21.15.100:0/4264966265 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f95601ebad0 con 0x7f9540079790 2024-09-17T04:34:26.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.792+0000 7f9557400640 1 -- 172.21.15.100:0/4264966265 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13363 (secure 0 0 0) 0x7f95601ebad0 con 0x7f9540079790 2024-09-17T04:34:26.793 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:34:26.793 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:34:26.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.794+0000 7f9564e00640 1 -- 172.21.15.100:0/4264966265 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9540079790 msgr2=0x7f954007bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:26.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.794+0000 7f9564e00640 1 --2- 172.21.15.100:0/4264966265 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9540079790 0x7f954007bca0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9550005fd0 tx=0x7f9550009500 comp rx=0 tx=0).stop 2024-09-17T04:34:26.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.794+0000 7f9564e00640 1 -- 172.21.15.100:0/4264966265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95600b2f60 msgr2=0x7f95600cdb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:26.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.794+0000 7f9564e00640 1 --2- 172.21.15.100:0/4264966265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95600b2f60 0x7f95600cdb10 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f954c034040 tx=0x7f954c003820 comp rx=0 tx=0).stop 2024-09-17T04:34:26.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.794+0000 7f9564e00640 1 -- 172.21.15.100:0/4264966265 shutdown_connections 2024-09-17T04:34:26.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.794+0000 7f9564e00640 1 --2- 172.21.15.100:0/4264966265 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f9540079790 0x7f954007bca0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:26.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.794+0000 7f9564e00640 1 --2- 172.21.15.100:0/4264966265 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f956015d670 0x7f95600ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:26.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.794+0000 7f9564e00640 1 --2- 172.21.15.100:0/4264966265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95600b2f60 0x7f95600cdb10 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:26.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.794+0000 7f9564e00640 1 --2- 172.21.15.100:0/4264966265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95600ad9c0 0x7f9560145b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:26.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.794+0000 7f9564e00640 1 -- 172.21.15.100:0/4264966265 >> 172.21.15.100:0/4264966265 conn(0x7f95600b1e40 msgr2=0x7f95600b4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:26.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.795+0000 7f9564e00640 1 -- 172.21.15.100:0/4264966265 shutdown_connections 2024-09-17T04:34:26.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:26.795+0000 7f9564e00640 1 -- 172.21.15.100:0/4264966265 wait complete. 2024-09-17T04:34:26.818 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":258,"stamp":"2024-09-17T04:34:25.731806+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":275736,"kb_used_data":17896,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754698984,"statfs":{"total":773094113280,"available":772811759616,"internally_reserved":0,"allocated":18325504,"data_stored":8738648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":263835188},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.904129"},"pg_stats":[{"pgid":"1.0","version":"76'79","reported_seq":411,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-17T04:34:15.149713+0000","last_change":"2024-09-17T04:24:06.226784+0000","last_active":"2024-09-17T04:34:15.149713+0000","last_peered":"2024-09-17T04:34:15.149713+0000","last_clean":"2024-09-17T04:34:15.149713+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:34:15.149713+0000","last_undegraded":"2024-09-17T04:34:15.149713+0000","last_fullsized":"2024-09-17T04:34:15.149713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:24:04.501352+0000","last_clean_scrub_stamp":"2024-09-17T04:24:04.501352+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:21.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33864,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337976,"statfs":{"total":96636764160,"available":96602087424,"internally_reserved":0,"allocated":2736128,"data_stored":1535107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33288,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33280,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33280,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542272,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37440,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334400,"statfs":{"total":96636764160,"available":96598425600,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509568,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33280,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509567,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33284,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338556,"statfs":{"total":96636764160,"available":96602681344,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509567,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38020,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333820,"statfs":{"total":96636764160,"available":96597831680,"internally_reserved":0,"allocated":2736128,"data_stored":1535107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:34:26.822 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-17T04:24:04.501352+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=4, tm_min=24, tm_sec=4, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=4, tm_min=34, tm_sec=19, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T04:34:26.822 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-17T04:34:36.624 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:34:36.629 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:34:46.822 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:34:47.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.058+0000 7f65b6600640 1 -- 172.21.15.100:0/4148427676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65b00ad3c0 msgr2=0x7f65b00ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:47.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.058+0000 7f65b6600640 1 --2- 172.21.15.100:0/4148427676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65b00ad3c0 0x7f65b00ad7d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6598009830 tx=0x7f659802a410 comp rx=0 tx=0).stop 2024-09-17T04:34:47.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.058+0000 7f65b6600640 1 -- 172.21.15.100:0/4148427676 shutdown_connections 2024-09-17T04:34:47.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.058+0000 7f65b6600640 1 --2- 172.21.15.100:0/4148427676 >> v2:172.21.15.184:3300/0 conn(0x7f65b00add10 0x7f65b01646e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:47.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.058+0000 7f65b6600640 1 --2- 172.21.15.100:0/4148427676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65b00ad3c0 0x7f65b00ad7d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:47.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.058+0000 7f65b6600640 1 -- 172.21.15.100:0/4148427676 >> 172.21.15.100:0/4148427676 conn(0x7f65b006d730 msgr2=0x7f65b00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:47.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.058+0000 7f65b6600640 1 -- 172.21.15.100:0/4148427676 shutdown_connections 2024-09-17T04:34:47.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.058+0000 7f65b6600640 1 -- 172.21.15.100:0/4148427676 wait complete. 2024-09-17T04:34:47.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.060+0000 7f65b6600640 1 Processor -- start 2024-09-17T04:34:47.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.060+0000 7f65b6600640 1 -- start start 2024-09-17T04:34:47.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.062+0000 7f65b6600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65b00704b0 0x7f65b0151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:47.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.062+0000 7f65b6600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65b00ad3c0 0x7f65b014fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:47.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.062+0000 7f65b6600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65b00add10 0x7f65b014bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:47.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.062+0000 7f65b6600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f65b00712b0 con 0x7f65b00704b0 2024-09-17T04:34:47.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.062+0000 7f65b6600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f65b0071130 con 0x7f65b00ad3c0 2024-09-17T04:34:47.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.062+0000 7f65b6600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f65b0164c20 con 0x7f65b00add10 2024-09-17T04:34:47.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.062+0000 7f65af400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65b00704b0 0x7f65b0151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:47.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.062+0000 7f65af400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65b00704b0 0x7f65b0151600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36916/0 (socket says 172.21.15.100:36916) 2024-09-17T04:34:47.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.062+0000 7f65af400640 1 -- 172.21.15.100:0/3515619195 learned_addr learned my addr 172.21.15.100:0/3515619195 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:47.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.062+0000 7f65a6a00640 1 --2- 172.21.15.100:0/3515619195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65b00ad3c0 0x7f65b014fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:47.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.062+0000 7f65afe00640 1 --2- 172.21.15.100:0/3515619195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65b00add10 0x7f65b014bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:47.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.063+0000 7f65af400640 1 -- 172.21.15.100:0/3515619195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65b00add10 msgr2=0x7f65b014bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:47.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.063+0000 7f65af400640 1 --2- 172.21.15.100:0/3515619195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65b00add10 0x7f65b014bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:47.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.063+0000 7f65af400640 1 -- 172.21.15.100:0/3515619195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65b00ad3c0 msgr2=0x7f65b014fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:47.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.063+0000 7f65af400640 1 --2- 172.21.15.100:0/3515619195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65b00ad3c0 0x7f65b014fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:47.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.063+0000 7f65af400640 1 -- 172.21.15.100:0/3515619195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b014c010 con 0x7f65b00704b0 2024-09-17T04:34:47.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.063+0000 7f65af400640 1 --2- 172.21.15.100:0/3515619195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65b00704b0 0x7f65b0151600 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f65a000c530 tx=0x7f65a000c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:47.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.063+0000 7f65a7e00640 1 -- 172.21.15.100:0/3515619195 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a0011790 con 0x7f65b00704b0 2024-09-17T04:34:47.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.063+0000 7f65b6600640 1 -- 172.21.15.100:0/3515619195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b0150560 con 0x7f65b00704b0 2024-09-17T04:34:47.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.063+0000 7f65a7e00640 1 -- 172.21.15.100:0/3515619195 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65a0017070 con 0x7f65b00704b0 2024-09-17T04:34:47.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.063+0000 7f65a7e00640 1 -- 172.21.15.100:0/3515619195 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a0011ce0 con 0x7f65b00704b0 2024-09-17T04:34:47.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.063+0000 7f65b6600640 1 -- 172.21.15.100:0/3515619195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b0150b20 con 0x7f65b00704b0 2024-09-17T04:34:47.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.065+0000 7f65b6600640 1 -- 172.21.15.100:0/3515619195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65b0163bd0 con 0x7f65b00704b0 2024-09-17T04:34:47.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.066+0000 7f65a7e00640 1 -- 172.21.15.100:0/3515619195 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7f65a001c460 con 0x7f65b00704b0 2024-09-17T04:34:47.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.067+0000 7f65a7e00640 1 --2- 172.21.15.100:0/3515619195 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f658c079760 0x7f658c07bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:47.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.067+0000 7f65a7e00640 1 -- 172.21.15.100:0/3515619195 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4999+0+0 (secure 0 0 0) 0x7f65a0099780 con 0x7f65b00704b0 2024-09-17T04:34:47.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.067+0000 7f65a6a00640 1 --2- 172.21.15.100:0/3515619195 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f658c079760 0x7f658c07bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:47.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.068+0000 7f65a6a00640 1 --2- 172.21.15.100:0/3515619195 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f658c079760 0x7f658c07bc70 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6598034000 tx=0x7f6598004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:47.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.072+0000 7f65a7e00640 1 -- 172.21.15.100:0/3515619195 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f65b0163bd0 con 0x7f65b00704b0 2024-09-17T04:34:47.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.253+0000 7f65b6600640 1 -- 172.21.15.100:0/3515619195 --> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f65b00ad7d0 con 0x7f658c079760 2024-09-17T04:34:47.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.254+0000 7f65a7e00640 1 -- 172.21.15.100:0/3515619195 <== mgr.4108 v2:172.21.15.100:6836/516807857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13370 (secure 0 0 0) 0x7f65b00ad7d0 con 0x7f658c079760 2024-09-17T04:34:47.253 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:34:47.254 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-17T04:34:47.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.256+0000 7f65b6600640 1 -- 172.21.15.100:0/3515619195 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f658c079760 msgr2=0x7f658c07bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:47.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.256+0000 7f65b6600640 1 --2- 172.21.15.100:0/3515619195 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f658c079760 0x7f658c07bc70 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6598034000 tx=0x7f6598004730 comp rx=0 tx=0).stop 2024-09-17T04:34:47.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.256+0000 7f65b6600640 1 -- 172.21.15.100:0/3515619195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65b00704b0 msgr2=0x7f65b0151600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:47.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.256+0000 7f65b6600640 1 --2- 172.21.15.100:0/3515619195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65b00704b0 0x7f65b0151600 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f65a000c530 tx=0x7f65a000c930 comp rx=0 tx=0).stop 2024-09-17T04:34:47.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.256+0000 7f65b6600640 1 -- 172.21.15.100:0/3515619195 shutdown_connections 2024-09-17T04:34:47.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.256+0000 7f65b6600640 1 --2- 172.21.15.100:0/3515619195 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f658c079760 0x7f658c07bc70 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:47.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.256+0000 7f65b6600640 1 --2- 172.21.15.100:0/3515619195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65b00add10 0x7f65b014bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:47.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.256+0000 7f65b6600640 1 --2- 172.21.15.100:0/3515619195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65b00ad3c0 0x7f65b014fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:47.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.256+0000 7f65b6600640 1 --2- 172.21.15.100:0/3515619195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65b00704b0 0x7f65b0151600 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:47.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.256+0000 7f65b6600640 1 -- 172.21.15.100:0/3515619195 >> 172.21.15.100:0/3515619195 conn(0x7f65b006d730 msgr2=0x7f65b0165010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:47.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.256+0000 7f65b6600640 1 -- 172.21.15.100:0/3515619195 shutdown_connections 2024-09-17T04:34:47.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.256+0000 7f65b6600640 1 -- 172.21.15.100:0/3515619195 wait complete. 2024-09-17T04:34:47.277 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2024-09-17T04:34:45.736414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":275724,"kb_used_data":17896,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754698996,"statfs":{"total":773094113280,"available":772811771904,"internally_reserved":0,"allocated":18325504,"data_stored":8738648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":263835188},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002969"},"pg_stats":[{"pgid":"1.0","version":"170'81","reported_seq":421,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-17T04:34:27.183714+0000","last_change":"2024-09-17T04:34:27.183714+0000","last_active":"2024-09-17T04:34:27.183714+0000","last_peered":"2024-09-17T04:34:27.183714+0000","last_clean":"2024-09-17T04:34:27.183714+0000","last_became_active":"2024-09-17T04:24:06.225385+0000","last_became_peered":"2024-09-17T04:24:06.225385+0000","last_unstale":"2024-09-17T04:34:27.183714+0000","last_undegraded":"2024-09-17T04:34:27.183714+0000","last_fullsized":"2024-09-17T04:34:27.183714+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"170'81","last_scrub_stamp":"2024-09-17T04:34:27.183590+0000","last_deep_scrub":"170'81","last_deep_scrub_stamp":"2024-09-17T04:34:27.183590+0000","last_clean_scrub_stamp":"2024-09-17T04:34:27.183590+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:20:40.069785+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33856,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337984,"statfs":{"total":96636764160,"available":96602095616,"internally_reserved":0,"allocated":2736128,"data_stored":1535107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33288,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33280,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33280,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542276,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37440,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334400,"statfs":{"total":96636764160,"available":96598425600,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509572,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33280,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509571,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33284,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338556,"statfs":{"total":96636764160,"available":96602681344,"internally_reserved":0,"allocated":2142208,"data_stored":944739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509571,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38016,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333824,"statfs":{"total":96636764160,"available":96597835776,"internally_reserved":0,"allocated":2736128,"data_stored":1535107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:34:47.278 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-17T04:34:47.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.495+0000 7f65bb200640 1 -- 172.21.15.100:0/1013058936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65b4156ca0 msgr2=0x7f65b40ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:47.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.495+0000 7f65bb200640 1 --2- 172.21.15.100:0/1013058936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65b4156ca0 0x7f65b40ccac0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f65b415de20 tx=0x7f65a402a5f0 comp rx=0 tx=0).stop 2024-09-17T04:34:47.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.495+0000 7f65bb200640 1 -- 172.21.15.100:0/1013058936 shutdown_connections 2024-09-17T04:34:47.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.495+0000 7f65bb200640 1 --2- 172.21.15.100:0/1013058936 >> v2:172.21.15.184:3300/0 conn(0x7f65b40cd000 0x7f65b40af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:47.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.495+0000 7f65bb200640 1 --2- 172.21.15.100:0/1013058936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65b4156ca0 0x7f65b40ccac0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:47.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.495+0000 7f65bb200640 1 -- 172.21.15.100:0/1013058936 >> 172.21.15.100:0/1013058936 conn(0x7f65b406d620 msgr2=0x7f65b40b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:47.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.495+0000 7f65bb200640 1 -- 172.21.15.100:0/1013058936 shutdown_connections 2024-09-17T04:34:47.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.495+0000 7f65bb200640 1 -- 172.21.15.100:0/1013058936 wait complete. 2024-09-17T04:34:47.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.498+0000 7f65bb200640 1 Processor -- start 2024-09-17T04:34:47.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.498+0000 7f65bb200640 1 -- start start 2024-09-17T04:34:47.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.499+0000 7f65bb200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65b40b3f30 0x7f65b41f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:47.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.499+0000 7f65bb200640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65b40cd000 0x7f65b40ad790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:47.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.499+0000 7f65bb200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65b4156ca0 0x7f65b40adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:47.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.499+0000 7f65bb200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f65b40af9a0 con 0x7f65b4156ca0 2024-09-17T04:34:47.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.499+0000 7f65bb200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f65b40af820 con 0x7f65b40b3f30 2024-09-17T04:34:47.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.499+0000 7f65bb200640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f65b40afb20 con 0x7f65b40cd000 2024-09-17T04:34:47.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.499+0000 7f65ba800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65b4156ca0 0x7f65b40adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:47.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.499+0000 7f65ba800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65b4156ca0 0x7f65b40adcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36930/0 (socket says 172.21.15.100:36930) 2024-09-17T04:34:47.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.500+0000 7f65ba800640 1 -- 172.21.15.100:0/4258958775 learned_addr learned my addr 172.21.15.100:0/4258958775 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:47.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.500+0000 7f65b9e00640 1 --2- 172.21.15.100:0/4258958775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65b40b3f30 0x7f65b41f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:47.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.500+0000 7f65b9400640 1 --2- 172.21.15.100:0/4258958775 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65b40cd000 0x7f65b40ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:47.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.500+0000 7f65ba800640 1 -- 172.21.15.100:0/4258958775 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65b40cd000 msgr2=0x7f65b40ad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:47.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.500+0000 7f65ba800640 1 --2- 172.21.15.100:0/4258958775 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65b40cd000 0x7f65b40ad790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:47.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.500+0000 7f65ba800640 1 -- 172.21.15.100:0/4258958775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65b40b3f30 msgr2=0x7f65b41f4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:47.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.500+0000 7f65ba800640 1 --2- 172.21.15.100:0/4258958775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65b40b3f30 0x7f65b41f4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:47.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.500+0000 7f65ba800640 1 -- 172.21.15.100:0/4258958775 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b41f54b0 con 0x7f65b4156ca0 2024-09-17T04:34:47.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.500+0000 7f65ba800640 1 --2- 172.21.15.100:0/4258958775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65b4156ca0 0x7f65b40adcd0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f65b000ea40 tx=0x7f65b000ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:47.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.500+0000 7f65ae000640 1 -- 172.21.15.100:0/4258958775 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b000ce40 con 0x7f65b4156ca0 2024-09-17T04:34:47.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.501+0000 7f65ae000640 1 -- 172.21.15.100:0/4258958775 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65b0004c10 con 0x7f65b4156ca0 2024-09-17T04:34:47.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.501+0000 7f65ae000640 1 -- 172.21.15.100:0/4258958775 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0005020 con 0x7f65b4156ca0 2024-09-17T04:34:47.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.501+0000 7f65bb200640 1 -- 172.21.15.100:0/4258958775 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b41f5980 con 0x7f65b4156ca0 2024-09-17T04:34:47.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.501+0000 7f65bb200640 1 -- 172.21.15.100:0/4258958775 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b41f5f40 con 0x7f65b4156ca0 2024-09-17T04:34:47.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.502+0000 7f65bb200640 1 -- 172.21.15.100:0/4258958775 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65b41f56d0 con 0x7f65b4156ca0 2024-09-17T04:34:47.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.503+0000 7f65ae000640 1 -- 172.21.15.100:0/4258958775 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7f65b00051c0 con 0x7f65b4156ca0 2024-09-17T04:34:47.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.504+0000 7f65ae000640 1 --2- 172.21.15.100:0/4258958775 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6598079760 0x7f659807bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:47.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.504+0000 7f65ae000640 1 -- 172.21.15.100:0/4258958775 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4999+0+0 (secure 0 0 0) 0x7f65b009a3c0 con 0x7f65b4156ca0 2024-09-17T04:34:47.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.504+0000 7f65b9e00640 1 --2- 172.21.15.100:0/4258958775 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6598079760 0x7f659807bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:47.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.505+0000 7f65b9e00640 1 --2- 172.21.15.100:0/4258958775 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6598079760 0x7f659807bc70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f65b40cf590 tx=0x7f65a8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:47.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.508+0000 7f65ae000640 1 -- 172.21.15.100:0/4258958775 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f65b0063c50 con 0x7f65b4156ca0 2024-09-17T04:34:47.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.691+0000 7f65bb200640 1 -- 172.21.15.100:0/4258958775 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f65b40af0d0 con 0x7f65b4156ca0 2024-09-17T04:34:47.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.695+0000 7f65ae000640 1 -- 172.21.15.100:0/4258958775 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v54)=0 v54) ==== 123+0+0 (secure 0 0 0) 0x7f65b0068680 con 0x7f65b4156ca0 2024-09-17T04:34:47.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.696+0000 7f65ae000640 1 -- 172.21.15.100:0/4258958775 <== mon.0 v2:172.21.15.100:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f65b0063760 con 0x7f65b4156ca0 2024-09-17T04:34:47.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.698+0000 7f65bb200640 1 -- 172.21.15.100:0/4258958775 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6598079760 msgr2=0x7f659807bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:47.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.698+0000 7f65bb200640 1 --2- 172.21.15.100:0/4258958775 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6598079760 0x7f659807bc70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f65b40cf590 tx=0x7f65a8009040 comp rx=0 tx=0).stop 2024-09-17T04:34:47.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.698+0000 7f65bb200640 1 -- 172.21.15.100:0/4258958775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65b4156ca0 msgr2=0x7f65b40adcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:47.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.698+0000 7f65bb200640 1 --2- 172.21.15.100:0/4258958775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65b4156ca0 0x7f65b40adcd0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f65b000ea40 tx=0x7f65b000ef00 comp rx=0 tx=0).stop 2024-09-17T04:34:47.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.698+0000 7f65bb200640 1 -- 172.21.15.100:0/4258958775 shutdown_connections 2024-09-17T04:34:47.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.698+0000 7f65bb200640 1 --2- 172.21.15.100:0/4258958775 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7f6598079760 0x7f659807bc70 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:47.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.698+0000 7f65bb200640 1 --2- 172.21.15.100:0/4258958775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65b4156ca0 0x7f65b40adcd0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:47.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.698+0000 7f65bb200640 1 --2- 172.21.15.100:0/4258958775 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65b40cd000 0x7f65b40ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:47.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.698+0000 7f65bb200640 1 --2- 172.21.15.100:0/4258958775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65b40b3f30 0x7f65b41f4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:47.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.698+0000 7f65bb200640 1 -- 172.21.15.100:0/4258958775 >> 172.21.15.100:0/4258958775 conn(0x7f65b406d620 msgr2=0x7f65b40ae750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:47.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.699+0000 7f65bb200640 1 -- 172.21.15.100:0/4258958775 shutdown_connections 2024-09-17T04:34:47.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.699+0000 7f65bb200640 1 -- 172.21.15.100:0/4258958775 wait complete. 2024-09-17T04:34:47.724 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T04:34:47.724 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:34:47.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.947+0000 7fbe71400640 1 -- 172.21.15.100:0/2972109912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe6c06d7b0 msgr2=0x7fbe6c0b67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:47.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.947+0000 7fbe71400640 1 --2- 172.21.15.100:0/2972109912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe6c06d7b0 0x7fbe6c0b67d0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c162540 tx=0x7fbe5c02a4a0 comp rx=0 tx=0).stop 2024-09-17T04:34:47.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.947+0000 7fbe71400640 1 -- 172.21.15.100:0/2972109912 shutdown_connections 2024-09-17T04:34:47.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.947+0000 7fbe71400640 1 --2- 172.21.15.100:0/2972109912 >> v2:172.21.15.184:3300/0 conn(0x7fbe6c0cc0d0 0x7fbe6c0cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:47.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.947+0000 7fbe71400640 1 --2- 172.21.15.100:0/2972109912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe6c06d7b0 0x7fbe6c0b67d0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:47.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.947+0000 7fbe71400640 1 -- 172.21.15.100:0/2972109912 >> 172.21.15.100:0/2972109912 conn(0x7fbe6c0b2fd0 msgr2=0x7fbe6c0b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:47.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.947+0000 7fbe71400640 1 -- 172.21.15.100:0/2972109912 shutdown_connections 2024-09-17T04:34:47.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.947+0000 7fbe71400640 1 -- 172.21.15.100:0/2972109912 wait complete. 2024-09-17T04:34:47.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.950+0000 7fbe71400640 1 Processor -- start 2024-09-17T04:34:47.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.950+0000 7fbe71400640 1 -- start start 2024-09-17T04:34:47.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.951+0000 7fbe71400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe6c06d7b0 0x7fbe6c1538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:47.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.951+0000 7fbe71400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe6c0cc0d0 0x7fbe6c1f9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:47.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.951+0000 7fbe71400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe6c0d0560 0x7fbe6c0b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:47.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.951+0000 7fbe71400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c0ccdb0 con 0x7fbe6c06d7b0 2024-09-17T04:34:47.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.951+0000 7fbe71400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c0ccc30 con 0x7fbe6c0d0560 2024-09-17T04:34:47.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.951+0000 7fbe71400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbe6c0ccf30 con 0x7fbe6c0cc0d0 2024-09-17T04:34:47.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.952+0000 7fbe6a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe6c06d7b0 0x7fbe6c1538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:47.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.952+0000 7fbe6a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe6c06d7b0 0x7fbe6c1538a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36958/0 (socket says 172.21.15.100:36958) 2024-09-17T04:34:47.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.952+0000 7fbe6a800640 1 -- 172.21.15.100:0/310528741 learned_addr learned my addr 172.21.15.100:0/310528741 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-17T04:34:47.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.952+0000 7fbe6b200640 1 --2- 172.21.15.100:0/310528741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe6c0d0560 0x7fbe6c0b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:47.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.952+0000 7fbe63e00640 1 --2- 172.21.15.100:0/310528741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe6c0cc0d0 0x7fbe6c1f9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:47.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.952+0000 7fbe6a800640 1 -- 172.21.15.100:0/310528741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe6c0cc0d0 msgr2=0x7fbe6c1f9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:47.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.952+0000 7fbe6a800640 1 --2- 172.21.15.100:0/310528741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe6c0cc0d0 0x7fbe6c1f9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:47.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.952+0000 7fbe6a800640 1 -- 172.21.15.100:0/310528741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe6c0d0560 msgr2=0x7fbe6c0b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:47.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.952+0000 7fbe6a800640 1 --2- 172.21.15.100:0/310528741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe6c0d0560 0x7fbe6c0b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:47.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.952+0000 7fbe6a800640 1 -- 172.21.15.100:0/310528741 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6c153de0 con 0x7fbe6c06d7b0 2024-09-17T04:34:47.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.953+0000 7fbe6a800640 1 --2- 172.21.15.100:0/310528741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe6c06d7b0 0x7fbe6c1538a0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c02aa90 tx=0x7fbe5c005430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:47.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.953+0000 7fbe62a00640 1 -- 172.21.15.100:0/310528741 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c03f070 con 0x7fbe6c06d7b0 2024-09-17T04:34:47.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.953+0000 7fbe62a00640 1 -- 172.21.15.100:0/310528741 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7fbe5c005620 con 0x7fbe6c06d7b0 2024-09-17T04:34:47.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.953+0000 7fbe62a00640 1 -- 172.21.15.100:0/310528741 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c0042b0 con 0x7fbe6c06d7b0 2024-09-17T04:34:47.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.953+0000 7fbe71400640 1 -- 172.21.15.100:0/310528741 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe6c0c4e80 con 0x7fbe6c06d7b0 2024-09-17T04:34:47.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.953+0000 7fbe71400640 1 -- 172.21.15.100:0/310528741 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe6c0ca430 con 0x7fbe6c06d7b0 2024-09-17T04:34:47.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.954+0000 7fbe71400640 1 -- 172.21.15.100:0/310528741 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe6c0b67d0 con 0x7fbe6c06d7b0 2024-09-17T04:34:47.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.957+0000 7fbe62a00640 1 -- 172.21.15.100:0/310528741 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) ==== 102512+0+0 (secure 0 0 0) 0x7fbe5c042070 con 0x7fbe6c06d7b0 2024-09-17T04:34:47.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.958+0000 7fbe62a00640 1 --2- 172.21.15.100:0/310528741 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbe48079760 0x7fbe4807bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:47.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.958+0000 7fbe62a00640 1 -- 172.21.15.100:0/310528741 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 4999+0+0 (secure 0 0 0) 0x7fbe5c0ba920 con 0x7fbe6c06d7b0 2024-09-17T04:34:47.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.958+0000 7fbe63e00640 1 --2- 172.21.15.100:0/310528741 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbe48079760 0x7fbe4807bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:47.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.959+0000 7fbe63e00640 1 --2- 172.21.15.100:0/310528741 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbe48079760 0x7fbe4807bc70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c15bc00 tx=0x7fbe54007eb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:47.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:47.964+0000 7fbe62a00640 1 -- 172.21.15.100:0/310528741 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fbe5c083f90 con 0x7fbe6c06d7b0 2024-09-17T04:34:48.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:48.187+0000 7fbe71400640 1 -- 172.21.15.100:0/310528741 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe6c0c5050 con 0x7fbe6c06d7b0 2024-09-17T04:34:48.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:48.188+0000 7fbe62a00640 1 -- 172.21.15.100:0/310528741 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+3925 (secure 0 0 0) 0x7fbe5c0889c0 con 0x7fbe6c06d7b0 2024-09-17T04:34:48.188 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-17T04:34:48.188 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":124,"btime":"2024-09-17T04:34:12:802235+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/613141426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":613141426},{"type":"v1","addr":"172.21.15.184:6835","nonce":613141426}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":8554,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/1797529407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6801","nonce":1797529407},{"type":"v1","addr":"172.21.15.184:6834","nonce":1797529407}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":8560,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2263752731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2263752731},{"type":"v1","addr":"172.21.15.100:6833","nonce":2263752731}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":9426,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1616090511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1616090511},{"type":"v1","addr":"172.21.15.100:6835","nonce":1616090511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":124}],"filesystems":[]} 2024-09-17T04:34:48.188 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 124 2024-09-17T04:34:48.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:48.190+0000 7fbe71400640 1 -- 172.21.15.100:0/310528741 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbe48079760 msgr2=0x7fbe4807bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:48.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:48.190+0000 7fbe71400640 1 --2- 172.21.15.100:0/310528741 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbe48079760 0x7fbe4807bc70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c15bc00 tx=0x7fbe54007eb0 comp rx=0 tx=0).stop 2024-09-17T04:34:48.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:48.190+0000 7fbe71400640 1 -- 172.21.15.100:0/310528741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe6c06d7b0 msgr2=0x7fbe6c1538a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:48.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:48.190+0000 7fbe71400640 1 --2- 172.21.15.100:0/310528741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe6c06d7b0 0x7fbe6c1538a0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c02aa90 tx=0x7fbe5c005430 comp rx=0 tx=0).stop 2024-09-17T04:34:48.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:48.191+0000 7fbe71400640 1 -- 172.21.15.100:0/310528741 shutdown_connections 2024-09-17T04:34:48.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:48.191+0000 7fbe71400640 1 --2- 172.21.15.100:0/310528741 >> [v2:172.21.15.100:6836/516807857,v1:172.21.15.100:6837/516807857] conn(0x7fbe48079760 0x7fbe4807bc70 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:48.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:48.191+0000 7fbe71400640 1 --2- 172.21.15.100:0/310528741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe6c0d0560 0x7fbe6c0b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:48.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:48.191+0000 7fbe71400640 1 --2- 172.21.15.100:0/310528741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe6c0cc0d0 0x7fbe6c1f9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:48.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:48.191+0000 7fbe71400640 1 --2- 172.21.15.100:0/310528741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe6c06d7b0 0x7fbe6c1538a0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:48.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:48.191+0000 7fbe71400640 1 -- 172.21.15.100:0/310528741 >> 172.21.15.100:0/310528741 conn(0x7fbe6c0b2fd0 msgr2=0x7fbe6c0b51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:48.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:48.191+0000 7fbe71400640 1 -- 172.21.15.100:0/310528741 shutdown_connections 2024-09-17T04:34:48.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-17T04:34:48.191+0000 7fbe71400640 1 -- 172.21.15.100:0/310528741 wait complete. 2024-09-17T04:34:48.215 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-17T04:34:48.215 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-17T04:34:48.215 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T04:34:48.215 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:34:48.240 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T04:34:48.240 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T04:34:48.240 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:34:48.304 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T04:34:48.304 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T04:34:48.305 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:34:48.329 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T04:34:48.329 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T04:34:48.329 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:34:48.393 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T04:34:48.393 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-17T04:34:48.393 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-17T04:34:48.394 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:34:48.446 INFO:tasks.ceph.osd.0:Stopped 2024-09-17T04:34:48.446 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-17T04:34:48.447 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:34:48.510 INFO:tasks.ceph.osd.1:Stopped 2024-09-17T04:34:48.511 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-17T04:34:48.511 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:34:48.573 INFO:tasks.ceph.osd.2:Stopped 2024-09-17T04:34:48.573 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-17T04:34:48.573 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:34:48.634 INFO:tasks.ceph.osd.3:Stopped 2024-09-17T04:34:48.634 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-17T04:34:48.634 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:34:48.682 INFO:tasks.ceph.osd.4:Stopped 2024-09-17T04:34:48.682 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-17T04:34:48.682 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:34:48.734 INFO:tasks.ceph.osd.5:Stopped 2024-09-17T04:34:48.735 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-17T04:34:48.735 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:34:48.783 INFO:tasks.ceph.osd.6:Stopped 2024-09-17T04:34:48.783 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-17T04:34:48.784 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:34:48.831 INFO:tasks.ceph.osd.7:Stopped 2024-09-17T04:34:48.832 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-17T04:34:48.832 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-17T04:34:48.832 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:34:48.914 INFO:tasks.ceph.mgr.y:Stopped 2024-09-17T04:34:48.914 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-17T04:34:48.915 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:34:48.987 INFO:tasks.ceph.mgr.x:Stopped 2024-09-17T04:34:48.987 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-17T04:34:48.988 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-17T04:34:48.988 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:34:49.018 INFO:tasks.ceph.mon.a:Stopped 2024-09-17T04:34:49.018 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-17T04:34:49.018 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:34:49.045 INFO:tasks.ceph.mon.b:Stopped 2024-09-17T04:34:49.045 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-17T04:34:49.045 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:34:49.112 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-09-17T04:34:49.112 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-09-17T04:34:49.112 INFO:tasks.ceph.mon.c:Stopped 2024-09-17T04:34:49.112 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-17T04:34:49.113 DEBUG:teuthology.orchestra.run.smithi100:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-09-17T04:34:49.198 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-17T04:34:49.198 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-17T04:34:49.399 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-17T04:34:49.400 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-17T04:34:49.544 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-17T04:34:49.544 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-17T04:34:49.686 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-17T04:34:49.686 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-17T04:34:49.785 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi184.front.sepia.ceph.com 2024-09-17T04:34:49.785 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-17T04:34:50.009 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi184.front.sepia.ceph.com 2024-09-17T04:34:50.010 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-17T04:34:50.103 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi184.front.sepia.ceph.com 2024-09-17T04:34:50.104 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-17T04:34:50.227 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi184.front.sepia.ceph.com 2024-09-17T04:34:50.227 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-17T04:34:50.320 INFO:tasks.ceph:Archiving mon data... 2024-09-17T04:34:50.321 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908519/data/mon.a.tgz 2024-09-17T04:34:50.321 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-09-17T04:34:50.356 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.4HMTmuQYCH 2024-09-17T04:34:50.357 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.4HMTmuQYCH 2024-09-17T04:34:50.668 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.4HMTmuQYCH 2024-09-17T04:34:50.763 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.4HMTmuQYCH is 1MB 2024-09-17T04:34:50.871 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.4HMTmuQYCH 2024-09-17T04:34:50.906 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908519/data/mon.b.tgz 2024-09-17T04:34:50.906 DEBUG:teuthology.orchestra.run.smithi184:> mktemp 2024-09-17T04:34:50.943 INFO:teuthology.orchestra.run.smithi184.stdout:/tmp/tmp.oYrqUTnJ76 2024-09-17T04:34:50.943 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.oYrqUTnJ76 2024-09-17T04:34:51.280 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0666 /tmp/tmp.oYrqUTnJ76 2024-09-17T04:34:51.374 DEBUG:teuthology.orchestra.remote:smithi184:/tmp/tmp.oYrqUTnJ76 is 1MB 2024-09-17T04:34:51.476 DEBUG:teuthology.orchestra.run.smithi184:> rm -fr /tmp/tmp.oYrqUTnJ76 2024-09-17T04:34:51.511 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908519/data/mon.c.tgz 2024-09-17T04:34:51.511 DEBUG:teuthology.orchestra.run.smithi184:> mktemp 2024-09-17T04:34:51.585 INFO:teuthology.orchestra.run.smithi184.stdout:/tmp/tmp.TshZ0M6YzV 2024-09-17T04:34:51.586 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.TshZ0M6YzV 2024-09-17T04:34:51.899 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0666 /tmp/tmp.TshZ0M6YzV 2024-09-17T04:34:51.992 DEBUG:teuthology.orchestra.remote:smithi184:/tmp/tmp.TshZ0M6YzV is 1MB 2024-09-17T04:34:52.095 DEBUG:teuthology.orchestra.run.smithi184:> rm -fr /tmp/tmp.TshZ0M6YzV 2024-09-17T04:34:52.130 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-17T04:34:52.130 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T04:34:52.135 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T04:34:52.262 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T04:34:52.297 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T04:34:52.297 INFO:tasks.ceph:Archiving crash dumps... 2024-09-17T04:34:52.298 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908519/remote/smithi100/crash 2024-09-17T04:34:52.299 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T04:34:52.358 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908519/remote/smithi184/crash 2024-09-17T04:34:52.359 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T04:34:52.413 INFO:tasks.ceph:Shutting down logrotate 2024-09-17T04:34:52.413 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:34:52.417 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:34:52.506 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-17T04:34:52.561 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-17T04:34:52.616 INFO:tasks.ceph:Compressing logs... 2024-09-17T04:34:52.616 DEBUG:teuthology.orchestra.run.smithi100:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T04:34:52.620 DEBUG:teuthology.orchestra.run.smithi184:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T04:34:52.671 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64797.log 2024-09-17T04:34:52.672 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-17T04:34:52.673 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.64797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42196.log 2024-09-17T04:34:52.673 INFO:teuthology.orchestra.run.smithi100.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.64797.log.gz 2024-09-17T04:34:52.673 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71938.log 2024-09-17T04:34:52.674 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.47355.log 2024-09-17T04:34:52.674 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42196.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42196.log.gz 2024-09-17T04:34:52.675 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75848.log 2024-09-17T04:34:52.679 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.46858.log 2024-09-17T04:34:52.679 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.47355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36169.log 2024-09-17T04:34:52.679 INFO:teuthology.orchestra.run.smithi184.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.47355.log.gz 2024-09-17T04:34:52.679 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.46858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43177.log 2024-09-17T04:34:52.679 INFO:teuthology.orchestra.run.smithi184.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.46858.log.gz 2024-09-17T04:34:52.679 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.36169.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-17T04:34:52.679 INFO:teuthology.orchestra.run.smithi184.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36169.log.gz 2024-09-17T04:34:52.679 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.43177.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43177.log.gzgzip 2024-09-17T04:34:52.680 INFO:teuthology.orchestra.run.smithi184.stderr: -5 --verbose -- /var/log/ceph/ceph-client.1.46862.log 2024-09-17T04:34:52.680 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36369.log 2024-09-17T04:34:52.681 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.71938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60256.log 2024-09-17T04:34:52.681 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71938.log.gz 2024-09-17T04:34:52.681 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.75848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81331.log 2024-09-17T04:34:52.682 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75848.log.gz 2024-09-17T04:34:52.682 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.60256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67052.log 2024-09-17T04:34:52.682 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60256.log.gz 2024-09-17T04:34:52.682 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.81331.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.81331.log.gz 2024-09-17T04:34:52.682 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58659.log 2024-09-17T04:34:52.682 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.67052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53010.log 2024-09-17T04:34:52.682 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67052.log.gz 2024-09-17T04:34:52.682 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.58659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57049.log 2024-09-17T04:34:52.682 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58659.log.gz 2024-09-17T04:34:52.682 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.53010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55950.log 2024-09-17T04:34:52.682 INFO:teuthology.orchestra.run.smithi100.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53010.log.gz 2024-09-17T04:34:52.682 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.57049.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57049.log.gz 2024-09-17T04:34:52.683 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46911.log 2024-09-17T04:34:52.692 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.46862.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.46862.log.gz 2024-09-17T04:34:52.692 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-17T04:34:52.692 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.36369.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36369.log.gz 2024-09-17T04:34:52.692 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36209.log 2024-09-17T04:34:52.692 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-17T04:34:52.692 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.36209.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36209.log.gz 2024-09-17T04:34:52.692 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T04:34:52.693 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.47857.log 2024-09-17T04:34:52.693 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44277.log 2024-09-17T04:34:52.693 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.47857.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43741.log 2024-09-17T04:34:52.693 INFO:teuthology.orchestra.run.smithi184.stderr: -- replaced with /var/log/ceph/ceph-client.1.47857.log.gz 2024-09-17T04:34:52.693 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.44277.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44277.log.gz 2024-09-17T04:34:52.693 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41621.log 2024-09-17T04:34:52.693 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.43741.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43741.log.gz 2024-09-17T04:34:52.693 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36529.log 2024-09-17T04:34:52.693 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.41621.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41621.log.gz 2024-09-17T04:34:52.693 INFO:teuthology.orchestra.run.smithi184.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T04:34:52.693 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.46361.log 2024-09-17T04:34:52.693 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.36529.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-17T04:34:52.694 INFO:teuthology.orchestra.run.smithi184.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36529.log.gz 2024-09-17T04:34:52.694 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.46361.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.46361.log.gz 2024-09-17T04:34:52.694 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36329.log 2024-09-17T04:34:52.698 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51920.log 2024-09-17T04:34:52.698 INFO:teuthology.orchestra.run.smithi100.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55950.log.gz 2024-09-17T04:34:52.698 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80070.log 2024-09-17T04:34:52.698 INFO:teuthology.orchestra.run.smithi100.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46911.log.gz 2024-09-17T04:34:52.699 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.51920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47415.log 2024-09-17T04:34:52.699 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51920.log.gz 2024-09-17T04:34:52.699 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.80070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46121.log 2024-09-17T04:34:52.699 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.80070.log.gz 2024-09-17T04:34:52.699 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.47415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45246.log 2024-09-17T04:34:52.699 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47415.log.gz 2024-09-17T04:34:52.699 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66405.log 2024-09-17T04:34:52.699 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46121.log.gz 2024-09-17T04:34:52.699 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.45246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54631.log 2024-09-17T04:34:52.699 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45246.log.gz 2024-09-17T04:34:52.699 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.66405.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66405.log.gz 2024-09-17T04:34:52.700 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66493.log 2024-09-17T04:34:52.700 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47914.log 2024-09-17T04:34:52.700 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54631.log.gz 2024-09-17T04:34:52.700 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.66493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54087.log 2024-09-17T04:34:52.700 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66493.log.gz 2024-09-17T04:34:52.700 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.47914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70692.log 2024-09-17T04:34:52.700 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47914.log.gz 2024-09-17T04:34:52.700 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54087.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54087.log.gz 2024-09-17T04:34:52.700 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60317.log 2024-09-17T04:34:52.700 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.70692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75764.log 2024-09-17T04:34:52.700 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70692.log.gz 2024-09-17T04:34:52.701 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.60317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72545.log 2024-09-17T04:34:52.701 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60317.log.gz 2024-09-17T04:34:52.701 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.75764.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.75764.log.gz 2024-09-17T04:34:52.701 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48134.log 2024-09-17T04:34:52.701 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.72545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74626.log 2024-09-17T04:34:52.701 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72545.log.gz 2024-09-17T04:34:52.701 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.48134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42364.log 2024-09-17T04:34:52.701 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48134.log.gz 2024-09-17T04:34:52.701 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.74626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75890.log 2024-09-17T04:34:52.701 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74626.log.gz 2024-09-17T04:34:52.701 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76058.log 2024-09-17T04:34:52.701 INFO:teuthology.orchestra.run.smithi100.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.42364.log.gz 2024-09-17T04:34:52.704 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.46357.log 2024-09-17T04:34:52.704 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.36329.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36329.log.gz 2024-09-17T04:34:52.704 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43737.log 2024-09-17T04:34:52.704 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.46357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45866.log 2024-09-17T04:34:52.704 INFO:teuthology.orchestra.run.smithi184.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.46357.log.gz 2024-09-17T04:34:52.704 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.43737.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43737.log.gz 2024-09-17T04:34:52.704 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.47359.log 2024-09-17T04:34:52.705 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.45866.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.45866.log.gz 2024-09-17T04:34:52.705 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43173.log 2024-09-17T04:34:52.705 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.47359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42039.log 2024-09-17T04:34:52.705 INFO:teuthology.orchestra.run.smithi184.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.47359.log.gz 2024-09-17T04:34:52.705 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.43173.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43173.log.gz 2024-09-17T04:34:52.705 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45357.log 2024-09-17T04:34:52.705 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-17T04:34:52.705 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.3.42039.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.42039.log.gz 2024-09-17T04:34:52.705 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.45357.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.45357.log.gz 2024-09-17T04:34:52.705 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36289.log 2024-09-17T04:34:52.705 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-17T04:34:52.705 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.36289.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T04:34:52.706 INFO:teuthology.orchestra.run.smithi184.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36289.log.gz 2024-09-17T04:34:52.709 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-17T04:34:52.709 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-17T04:34:52.714 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.75890.log: gzip -5 87.4% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.75890.log.gz /var/log/ceph/ceph-client.admin.75510.log 2024-09-17T04:34:52.715 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-09-17T04:34:52.715 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.76058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56286.log 2024-09-17T04:34:52.715 INFO:teuthology.orchestra.run.smithi100.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.76058.log.gz 2024-09-17T04:34:52.715 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.75510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72022.log 2024-09-17T04:34:52.715 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75510.log.gz 2024-09-17T04:34:52.715 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70608.log 2024-09-17T04:34:52.715 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.72022.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.56286.log.gz 2024-09-17T04:34:52.715 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.72022.log.gz 2024-09-17T04:34:52.715 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81289.log 2024-09-17T04:34:52.715 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.70608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49434.log 2024-09-17T04:34:52.715 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70608.log.gz 2024-09-17T04:34:52.715 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.81289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44973.log 2024-09-17T04:34:52.715 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.81289.log.gz 2024-09-17T04:34:52.716 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.49434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74208.log 2024-09-17T04:34:52.716 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49434.log.gz 2024-09-17T04:34:52.716 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.44973.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44973.log.gz 2024-09-17T04:34:52.716 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42448.log 2024-09-17T04:34:52.716 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.74208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68719.log 2024-09-17T04:34:52.716 INFO:teuthology.orchestra.run.smithi100.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.74208.log.gz 2024-09-17T04:34:52.716 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76562.log 2024-09-17T04:34:52.716 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42448.log.gz 2024-09-17T04:34:52.716 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.68719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54171.log 2024-09-17T04:34:52.716 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68719.log.gz 2024-09-17T04:34:52.716 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.76562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45913.log 2024-09-17T04:34:52.717 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.76562.log.gz 2024-09-17T04:34:52.717 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54171.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54171.log.gz 2024-09-17T04:34:52.717 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52754.log 2024-09-17T04:34:52.717 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.45913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47541.log 2024-09-17T04:34:52.717 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45913.log.gz 2024-09-17T04:34:52.717 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.52754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57571.log 2024-09-17T04:34:52.717 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52754.log.gz 2024-09-17T04:34:52.717 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.47541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56540.log 2024-09-17T04:34:52.717 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47541.log.gz 2024-09-17T04:34:52.717 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.57571.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57571.log.gz 2024-09-17T04:34:52.717 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77637.log 2024-09-17T04:34:52.717 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.56540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73887.log 2024-09-17T04:34:52.718 INFO:teuthology.orchestra.run.smithi100.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56540.log.gz 2024-09-17T04:34:52.718 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.77637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56664.log 2024-09-17T04:34:52.718 INFO:teuthology.orchestra.run.smithi100.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.77637.log.gz 2024-09-17T04:34:52.720 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42035.log 2024-09-17T04:34:52.720 INFO:teuthology.orchestra.run.smithi184.stderr: 92.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T04:34:52.720 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45862.log 2024-09-17T04:34:52.720 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.3.42035.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.42035.log.gz 2024-09-17T04:34:52.720 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36489.log 2024-09-17T04:34:52.723 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.73887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77469.log 2024-09-17T04:34:52.723 INFO:teuthology.orchestra.run.smithi100.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.73887.log.gz 2024-09-17T04:34:52.723 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56942.log 2024-09-17T04:34:52.724 INFO:teuthology.orchestra.run.smithi100.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56664.log.gz 2024-09-17T04:34:52.724 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.77469.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77469.log.gz 2024-09-17T04:34:52.724 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77189.log 2024-09-17T04:34:52.724 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43088.log 2024-09-17T04:34:52.724 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56942.log.gz 2024-09-17T04:34:52.724 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.77189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52530.log 2024-09-17T04:34:52.724 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77189.log.gz 2024-09-17T04:34:52.724 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69894.log 2024-09-17T04:34:52.724 INFO:teuthology.orchestra.run.smithi100.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43088.log.gz 2024-09-17T04:34:52.724 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.52530.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52530.log.gz 2024-09-17T04:34:52.724 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62227.log 2024-09-17T04:34:52.725 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.69894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41772.log 2024-09-17T04:34:52.725 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69894.log.gz 2024-09-17T04:34:52.725 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.62227.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.62227.log.gz 2024-09-17T04:34:52.725 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61643.log 2024-09-17T04:34:52.725 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54129.log 2024-09-17T04:34:52.725 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41772.log.gz 2024-09-17T04:34:52.727 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.45862.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.45862.log.gz 2024-09-17T04:34:52.727 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.36489.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36489.log.gz 2024-09-17T04:34:52.727 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36249.log 2024-09-17T04:34:52.727 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.47861.log 2024-09-17T04:34:52.729 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.61643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60828.log 2024-09-17T04:34:52.729 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61643.log.gz 2024-09-17T04:34:52.729 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41814.log 2024-09-17T04:34:52.729 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54129.log.gz 2024-09-17T04:34:52.730 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.60828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77721.log 2024-09-17T04:34:52.730 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60828.log.gz 2024-09-17T04:34:52.730 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74836.log 2024-09-17T04:34:52.730 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41814.log.gz 2024-09-17T04:34:52.730 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.77721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56748.log 2024-09-17T04:34:52.730 INFO:teuthology.orchestra.run.smithi100.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.77721.log.gz 2024-09-17T04:34:52.730 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.74836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71420.log 2024-09-17T04:34:52.730 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74836.log.gz 2024-09-17T04:34:52.730 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56748.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56748.log.gz 2024-09-17T04:34:52.730 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76484.log 2024-09-17T04:34:52.732 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.36249.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36249.log.gz 2024-09-17T04:34:52.732 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-17T04:34:52.732 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.47861.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.47861.log.gz 2024-09-17T04:34:52.733 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.71420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76226.log 2024-09-17T04:34:52.733 INFO:teuthology.orchestra.run.smithi100.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.71420.log.gz 2024-09-17T04:34:52.734 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.76484.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.76484.log.gz 2024-09-17T04:34:52.734 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62949.log 2024-09-17T04:34:52.734 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.76226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69978.log 2024-09-17T04:34:52.734 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.62949.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.76226.log.gz 2024-09-17T04:34:52.734 INFO:teuthology.orchestra.run.smithi100.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.62949.log.gz 2024-09-17T04:34:52.734 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48260.log 2024-09-17T04:34:52.734 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.69978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70230.log 2024-09-17T04:34:52.734 INFO:teuthology.orchestra.run.smithi100.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.69978.log.gz 2024-09-17T04:34:52.736 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.48260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50608.log 2024-09-17T04:34:52.736 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48260.log.gz 2024-09-17T04:34:52.736 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.70230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60214.log 2024-09-17T04:34:52.736 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.50608.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.70230.log.gz 2024-09-17T04:34:52.736 INFO:teuthology.orchestra.run.smithi100.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50608.log.gz 2024-09-17T04:34:52.736 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79173.log 2024-09-17T04:34:52.737 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44825.log 2024-09-17T04:34:52.738 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.60214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69667.log 2024-09-17T04:34:52.738 INFO:teuthology.orchestra.run.smithi100.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.60214.log.gz 2024-09-17T04:34:52.738 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.79173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50258.log 2024-09-17T04:34:52.738 INFO:teuthology.orchestra.run.smithi100.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.79173.log.gz 2024-09-17T04:34:52.739 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.69667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55387.log 2024-09-17T04:34:52.739 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.69667.log.gz 2024-09-17T04:34:52.740 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41640.log 2024-09-17T04:34:52.741 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.50258.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50258.log.gz 2024-09-17T04:34:52.741 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67951.log 2024-09-17T04:34:52.741 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69074.log 2024-09-17T04:34:52.741 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55387.log.gz 2024-09-17T04:34:52.742 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.67951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69326.log 2024-09-17T04:34:52.742 INFO:teuthology.orchestra.run.smithi100.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.67951.log.gz 2024-09-17T04:34:52.742 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.69074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51607.log 2024-09-17T04:34:52.742 INFO:teuthology.orchestra.run.smithi100.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.69074.log.gz 2024-09-17T04:34:52.743 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.69326.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69326.log.gz 2024-09-17T04:34:52.743 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57476.log 2024-09-17T04:34:52.744 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.51607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70104.log 2024-09-17T04:34:52.747 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.44825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45361.log 2024-09-17T04:34:52.747 INFO:teuthology.orchestra.run.smithi184.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44825.log.gz 2024-09-17T04:34:52.747 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.41640.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41640.log.gz 2024-09-17T04:34:52.747 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42638.log 2024-09-17T04:34:52.748 INFO:teuthology.orchestra.run.smithi100.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51607.log.gz 2024-09-17T04:34:52.749 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.57476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55555.log 2024-09-17T04:34:52.749 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57476.log.gz 2024-09-17T04:34:52.749 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.70104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55471.log 2024-09-17T04:34:52.749 INFO:teuthology.orchestra.run.smithi100.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.70104.log.gz 2024-09-17T04:34:52.749 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55555.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55555.log.gz 2024-09-17T04:34:52.749 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66758.log 2024-09-17T04:34:52.749 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52615.log 2024-09-17T04:34:52.749 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55471.log.gz 2024-09-17T04:34:52.749 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.66758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59825.log 2024-09-17T04:34:52.749 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.66758.log.gz 2024-09-17T04:34:52.749 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.52615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64923.log 2024-09-17T04:34:52.751 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.45361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44821.log 2024-09-17T04:34:52.751 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.42638.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.45361.log.gz 2024-09-17T04:34:52.751 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52615.log.gz 2024-09-17T04:34:52.751 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.59825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65792.log 2024-09-17T04:34:52.752 INFO:teuthology.orchestra.run.smithi100.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.59825.log.gz 2024-09-17T04:34:52.752 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.64923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58094.log 2024-09-17T04:34:52.752 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.64923.log.gz 2024-09-17T04:34:52.752 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.65792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75065.log 2024-09-17T04:34:52.752 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65792.log.gz 2024-09-17T04:34:52.753 INFO:teuthology.orchestra.run.smithi184.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42638.log.gz 2024-09-17T04:34:52.753 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42634.log 2024-09-17T04:34:52.754 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.58094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42741.log 2024-09-17T04:34:52.754 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58094.log.gz 2024-09-17T04:34:52.754 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.75065.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.75065.log.gz 2024-09-17T04:34:52.755 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67394.log 2024-09-17T04:34:52.755 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42741.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-17T04:34:52.755 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42741.log.gz 2024-09-17T04:34:52.756 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.44821.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44821.log.gz 2024-09-17T04:34:52.757 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.67394.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67394.log.gz 2024-09-17T04:34:52.757 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58052.log 2024-09-17T04:34:52.757 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73509.log 2024-09-17T04:34:52.757 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.58052.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.58052.log.gz 2024-09-17T04:34:52.757 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62907.log 2024-09-17T04:34:52.758 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44273.log 2024-09-17T04:34:52.759 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.42634.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42634.log.gz 2024-09-17T04:34:52.759 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.73509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70146.log 2024-09-17T04:34:52.759 INFO:teuthology.orchestra.run.smithi100.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.73509.log.gz 2024-09-17T04:34:52.760 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.62907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51122.log 2024-09-17T04:34:52.760 INFO:teuthology.orchestra.run.smithi100.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.62907.log.gz 2024-09-17T04:34:52.760 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.70146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54902.log 2024-09-17T04:34:52.760 INFO:teuthology.orchestra.run.smithi100.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.70146.log.gz 2024-09-17T04:34:52.761 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.44273.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44273.log.gz 2024-09-17T04:34:52.761 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.51122.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51122.log.gz 2024-09-17T04:34:52.761 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61265.log 2024-09-17T04:34:52.762 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66239.log 2024-09-17T04:34:52.762 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54902.log.gz 2024-09-17T04:34:52.762 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.61265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81205.log 2024-09-17T04:34:52.762 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61265.log.gz 2024-09-17T04:34:52.763 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.66239.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66239.log.gz 2024-09-17T04:34:52.763 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65918.log 2024-09-17T04:34:52.764 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.81205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53724.log 2024-09-17T04:34:52.765 INFO:teuthology.orchestra.run.smithi100.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.81205.log.gz 2024-09-17T04:34:52.765 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.65918.log: gzip -5 --verbose -- 87.7% /var/log/ceph/ceph-client.admin.78190.log -- replaced with /var/log/ceph/ceph-client.admin.65918.log.gz 2024-09-17T04:34:52.765 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-09-17T04:34:52.766 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.53724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61307.log 2024-09-17T04:34:52.767 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53724.log.gz 2024-09-17T04:34:52.767 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.78190.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.78190.log.gz 2024-09-17T04:34:52.767 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54505.log 2024-09-17T04:34:52.768 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.61307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75195.log 2024-09-17T04:34:52.769 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61307.log.gz 2024-09-17T04:34:52.769 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58478.log 2024-09-17T04:34:52.770 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54505.log.gz 2024-09-17T04:34:52.770 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.75195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71121.log 2024-09-17T04:34:52.771 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.75195.log.gz 2024-09-17T04:34:52.771 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.58478.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58478.log.gz 2024-09-17T04:34:52.771 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70875.log 2024-09-17T04:34:52.772 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.71121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73763.log 2024-09-17T04:34:52.773 INFO:teuthology.orchestra.run.smithi100.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.71121.log.gz 2024-09-17T04:34:52.773 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.70875.log: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.70875.log.gz 2024-09-17T04:34:52.773 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70957.log 2024-09-17T04:34:52.774 INFO:teuthology.orchestra.run.smithi184.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-17T04:34:52.774 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.73763.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.73763.log.gz 2024-09-17T04:34:52.775 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57111.log 2024-09-17T04:34:52.775 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.70957.log: gzip -5 --verbose 84.8% -- -- replaced with /var/log/ceph/ceph-client.admin.70957.log.gz 2024-09-17T04:34:52.775 INFO:teuthology.orchestra.run.smithi100.stderr: /var/log/ceph/ceph-client.admin.71039.log 2024-09-17T04:34:52.776 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.57111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68761.log 2024-09-17T04:34:52.777 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57111.log.gz 2024-09-17T04:34:52.777 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.71039.log: gzip -5 --verbose 85.4% -- -- replaced with /var/log/ceph/ceph-client.admin.71039.log.gz 2024-09-17T04:34:52.777 INFO:teuthology.orchestra.run.smithi100.stderr: /var/log/ceph/ceph-client.admin.60443.log 2024-09-17T04:34:52.778 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.68761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81121.log 2024-09-17T04:34:52.778 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68761.log.gz 2024-09-17T04:34:52.779 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.60443.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60443.log.gz 2024-09-17T04:34:52.779 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62173.log 2024-09-17T04:34:52.780 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.81121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48386.log 2024-09-17T04:34:52.781 INFO:teuthology.orchestra.run.smithi100.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.81121.log.gz 2024-09-17T04:34:52.781 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.62173.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.62173.log.gz 2024-09-17T04:34:52.781 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50732.log 2024-09-17T04:34:52.782 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.48386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74668.log 2024-09-17T04:34:52.783 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48386.log.gz 2024-09-17T04:34:52.783 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.50732.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50732.log.gz 2024-09-17T04:34:52.784 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48698.log 2024-09-17T04:34:52.784 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.74668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45413.log 2024-09-17T04:34:52.784 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74668.log.gz 2024-09-17T04:34:52.785 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.48698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69625.log 2024-09-17T04:34:52.785 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48698.log.gz 2024-09-17T04:34:52.787 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.45413.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45413.log.gz 2024-09-17T04:34:52.787 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44889.log 2024-09-17T04:34:52.787 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.69625.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69625.log.gz 2024-09-17T04:34:52.787 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65647.log 2024-09-17T04:34:52.788 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.44889.log: gzip 87.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.44889.log.gz 2024-09-17T04:34:52.788 INFO:teuthology.orchestra.run.smithi100.stderr: --verbose -- /var/log/ceph/ceph-client.admin.76930.log 2024-09-17T04:34:52.789 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.65647.log: gzip 87.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.65647.log.gz 2024-09-17T04:34:52.789 INFO:teuthology.orchestra.run.smithi100.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51691.log 2024-09-17T04:34:52.790 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.76930.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.76930.log.gz 2024-09-17T04:34:52.790 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60994.log 2024-09-17T04:34:52.791 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.51691.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51691.log.gz 2024-09-17T04:34:52.791 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46533.log 2024-09-17T04:34:52.792 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.60994.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.60994.log.gz 2024-09-17T04:34:52.792 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65708.log 2024-09-17T04:34:52.793 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46533.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46533.log.gz 2024-09-17T04:34:52.793 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45168.log 2024-09-17T04:34:52.794 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.65708.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.65708.log.gz 2024-09-17T04:34:52.794 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46869.log 2024-09-17T04:34:52.795 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.45168.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45168.log.gz 2024-09-17T04:34:52.795 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67472.log 2024-09-17T04:34:52.796 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46869.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46869.log.gz 2024-09-17T04:34:52.796 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67094.log 2024-09-17T04:34:52.797 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.67472.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67472.log.gz 2024-09-17T04:34:52.797 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44763.log 2024-09-17T04:34:52.797 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.67094.log: gzip 87.3% -- replaced with /var/log/ceph/ceph-client.admin.67094.log.gz -5 2024-09-17T04:34:52.798 INFO:teuthology.orchestra.run.smithi100.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44280.log 2024-09-17T04:34:52.798 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.44763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41986.log 2024-09-17T04:34:52.799 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44763.log.gz 2024-09-17T04:34:52.799 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.44280.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44280.log.gzgzip 2024-09-17T04:34:52.799 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50022.log 2024-09-17T04:34:52.800 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48428.log 2024-09-17T04:34:52.800 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41986.log.gz 2024-09-17T04:34:52.801 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.50022.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50022.log.gz 2024-09-17T04:34:52.801 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81415.log 2024-09-17T04:34:52.802 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.48428.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48428.log.gz 2024-09-17T04:34:52.802 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68077.log 2024-09-17T04:34:52.804 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.81415.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.81415.log.gz 2024-09-17T04:34:52.804 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78793.log 2024-09-17T04:34:52.804 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.68077.log: gzip 87.1% -- replaced with /var/log/ceph/ceph-client.admin.68077.log.gz 2024-09-17T04:34:52.805 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.66320.log 2024-09-17T04:34:52.805 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.78793.log: gzip -5 --verbose -- 87.2% /var/log/ceph/ceph-client.admin.78110.log -- replaced with /var/log/ceph/ceph-client.admin.78793.log.gz 2024-09-17T04:34:52.805 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-09-17T04:34:52.807 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.66320.log: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.66320.log.gz 2024-09-17T04:34:52.807 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.63075.log 2024-09-17T04:34:52.807 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.78110.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78110.log.gz 2024-09-17T04:34:52.808 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65605.log 2024-09-17T04:34:52.808 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.63075.log: 87.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.63075.log.gz 2024-09-17T04:34:52.809 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.74335.log 2024-09-17T04:34:52.809 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.65605.log: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.65605.log.gz 2024-09-17T04:34:52.809 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51184.log 2024-09-17T04:34:52.810 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.74335.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.74335.log.gz 2024-09-17T04:34:52.810 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65295.log 2024-09-17T04:34:52.810 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.51184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52703.log 2024-09-17T04:34:52.811 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51184.log.gz 2024-09-17T04:34:52.811 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.65295.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65295.log.gz 2024-09-17T04:34:52.811 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74269.log 2024-09-17T04:34:52.812 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.52703.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52703.log.gz 2024-09-17T04:34:52.812 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64984.log 2024-09-17T04:34:52.813 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.74269.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.74269.log.gzgzip 2024-09-17T04:34:52.813 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.81611.log 2024-09-17T04:34:52.814 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.64984.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.64984.log.gz 2024-09-17T04:34:52.814 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47145.log 2024-09-17T04:34:52.815 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.81611.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.81611.log.gz 2024-09-17T04:34:52.815 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.29981.log 2024-09-17T04:34:52.816 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.47145.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47145.log.gz 2024-09-17T04:34:52.816 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48218.log 2024-09-17T04:34:52.817 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph.tmp-client.admin.29981.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.29981.log.gz 2024-09-17T04:34:52.817 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72762.log 2024-09-17T04:34:52.818 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.48218.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48218.log.gz 2024-09-17T04:34:52.818 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49304.log 2024-09-17T04:34:52.819 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.72762.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72762.log.gz 2024-09-17T04:34:52.819 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66449.log 2024-09-17T04:34:52.820 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.49304.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49304.log.gz 2024-09-17T04:34:52.820 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61349.log 2024-09-17T04:34:52.821 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.66449.log: 87.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.66449.log.gz 2024-09-17T04:34:52.821 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.73575.log 2024-09-17T04:34:52.822 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.61349.log: gzip 87.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.61349.log.gz --verbose 2024-09-17T04:34:52.822 INFO:teuthology.orchestra.run.smithi100.stderr: -- /var/log/ceph/ceph-client.admin.71162.log 2024-09-17T04:34:52.823 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.73575.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73575.log.gz 2024-09-17T04:34:52.823 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74794.log 2024-09-17T04:34:52.824 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.71162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58232.log 2024-09-17T04:34:52.824 INFO:teuthology.orchestra.run.smithi100.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.71162.log.gz 2024-09-17T04:34:52.824 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.74794.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74794.log.gz 2024-09-17T04:34:52.825 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53682.log 2024-09-17T04:34:52.825 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.58232.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58232.log.gz 2024-09-17T04:34:52.825 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57926.log 2024-09-17T04:34:52.826 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.53682.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53682.log.gz 2024-09-17T04:34:52.826 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53094.log 2024-09-17T04:34:52.827 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.57926.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.57926.log.gz 2024-09-17T04:34:52.827 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42742.log 2024-09-17T04:34:52.828 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.53094.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.53094.log.gz 2024-09-17T04:34:52.828 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49518.log 2024-09-17T04:34:52.829 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42742.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42742.log.gz 2024-09-17T04:34:52.829 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75293.log 2024-09-17T04:34:52.830 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.49518.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49518.log.gz 2024-09-17T04:34:52.830 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36142.log 2024-09-17T04:34:52.831 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.75293.log: gzip 87.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.75293.log.gz 2024-09-17T04:34:52.831 INFO:teuthology.orchestra.run.smithi100.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52046.log 2024-09-17T04:34:52.831 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36142.log: 76.8% -- replaced with /var/log/ceph/ceph-client.admin.36142.log.gz 2024-09-17T04:34:52.832 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46743.log 2024-09-17T04:34:52.833 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.52046.log: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.52046.log.gz -5 2024-09-17T04:34:52.833 INFO:teuthology.orchestra.run.smithi100.stderr: --verbose -- /var/log/ceph/ceph-client.admin.80154.log 2024-09-17T04:34:52.834 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46743.log: 87.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.46743.log.gz 2024-09-17T04:34:52.834 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48997.log 2024-09-17T04:34:52.835 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.80154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46036.log 2024-09-17T04:34:52.835 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.80154.log.gz 2024-09-17T04:34:52.836 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.48997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74105.log 2024-09-17T04:34:52.836 INFO:teuthology.orchestra.run.smithi100.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48997.log.gz 2024-09-17T04:34:52.837 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72461.log 2024-09-17T04:34:52.837 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46036.log.gz 2024-09-17T04:34:52.838 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.74105.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.74105.log.gz 2024-09-17T04:34:52.838 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63033.log 2024-09-17T04:34:52.839 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.72461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49039.log 2024-09-17T04:34:52.839 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.72461.log.gz 2024-09-17T04:34:52.840 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.63033.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.63033.log.gz 2024-09-17T04:34:52.840 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77679.log 2024-09-17T04:34:52.841 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.49039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37290.log 2024-09-17T04:34:52.841 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49039.log.gz 2024-09-17T04:34:52.842 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.77679.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77679.log.gz 2024-09-17T04:34:52.842 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51859.log 2024-09-17T04:34:52.843 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66586.log 2024-09-17T04:34:52.843 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37290.log.gz 2024-09-17T04:34:52.844 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.51859.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51859.log.gz 2024-09-17T04:34:52.844 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45539.log 2024-09-17T04:34:52.846 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.66586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69116.log 2024-09-17T04:34:52.846 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66586.log.gz 2024-09-17T04:34:52.846 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.45539.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45539.log.gz 2024-09-17T04:34:52.846 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43092.log 2024-09-17T04:34:52.847 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.69116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77343.log 2024-09-17T04:34:52.847 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69116.log.gz 2024-09-17T04:34:52.848 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43092.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.43092.log.gz 2024-09-17T04:34:52.848 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.70998.log 2024-09-17T04:34:52.849 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.77343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60004.log 2024-09-17T04:34:52.849 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77343.log.gz 2024-09-17T04:34:52.850 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.70998.log: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.70998.log.gz 2024-09-17T04:34:52.850 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72335.log 2024-09-17T04:34:52.851 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.60004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43330.log 2024-09-17T04:34:52.851 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60004.log.gz 2024-09-17T04:34:52.852 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.72335.log: gzip -5 87.4% --verbose -- replaced with /var/log/ceph/ceph-client.admin.72335.log.gz -- 2024-09-17T04:34:52.852 INFO:teuthology.orchestra.run.smithi100.stderr: /var/log/ceph/ceph-client.admin.36058.log 2024-09-17T04:34:52.853 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64694.log 2024-09-17T04:34:52.853 INFO:teuthology.orchestra.run.smithi100.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43330.log.gz 2024-09-17T04:34:52.854 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36058.log: 77.5% -- replaced with /var/log/ceph/ceph-client.admin.36058.log.gz 2024-09-17T04:34:52.854 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79986.log 2024-09-17T04:34:52.855 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.64694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46701.log 2024-09-17T04:34:52.855 INFO:teuthology.orchestra.run.smithi100.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.64694.log.gz 2024-09-17T04:34:52.856 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.79986.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.79986.log.gz 2024-09-17T04:34:52.856 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47037.log 2024-09-17T04:34:52.857 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53262.log 2024-09-17T04:34:52.857 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46701.log.gz 2024-09-17T04:34:52.858 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.47037.log: gzip -5 87.5% --verbose -- replaced with /var/log/ceph/ceph-client.admin.47037.log.gz 2024-09-17T04:34:52.858 INFO:teuthology.orchestra.run.smithi100.stderr: -- /var/log/ceph/ceph-client.admin.46995.log 2024-09-17T04:34:52.859 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.53262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36650.log 2024-09-17T04:34:52.859 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53262.log.gz 2024-09-17T04:34:52.860 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46995.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46995.log.gz 2024-09-17T04:34:52.860 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46316.log 2024-09-17T04:34:52.861 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41858.log 2024-09-17T04:34:52.861 INFO:teuthology.orchestra.run.smithi100.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36650.log.gz 2024-09-17T04:34:52.862 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46316.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46316.log.gz 2024-09-17T04:34:52.862 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45581.log 2024-09-17T04:34:52.863 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58873.log 2024-09-17T04:34:52.863 INFO:teuthology.orchestra.run.smithi100.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41858.log.gz 2024-09-17T04:34:52.864 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.45581.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45581.log.gz 2024-09-17T04:34:52.864 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61835.log 2024-09-17T04:34:52.865 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.58873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61577.log 2024-09-17T04:34:52.865 INFO:teuthology.orchestra.run.smithi100.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.58873.log.gz 2024-09-17T04:34:52.865 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.61835.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.61835.log.gz 2024-09-17T04:34:52.866 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46953.log 2024-09-17T04:34:52.867 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.61577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80238.log 2024-09-17T04:34:52.867 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46953.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.61577.log.gz 2024-09-17T04:34:52.868 INFO:teuthology.orchestra.run.smithi100.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46953.log.gz 2024-09-17T04:34:52.868 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58915.log 2024-09-17T04:34:52.869 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.80238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36126.log 2024-09-17T04:34:52.869 INFO:teuthology.orchestra.run.smithi100.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.80238.log.gz 2024-09-17T04:34:52.870 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.58915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52926.log 2024-09-17T04:34:52.870 INFO:teuthology.orchestra.run.smithi100.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58915.log.gz 2024-09-17T04:34:52.870 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36126.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36126.log.gz 2024-09-17T04:34:52.871 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57655.log 2024-09-17T04:34:52.872 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.52926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62744.log 2024-09-17T04:34:52.872 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52926.log.gz 2024-09-17T04:34:52.873 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.57655.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57655.log.gz 2024-09-17T04:34:52.873 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58789.log 2024-09-17T04:34:52.874 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.62744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62311.log 2024-09-17T04:34:52.874 INFO:teuthology.orchestra.run.smithi100.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.62744.log.gz 2024-09-17T04:34:52.875 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.58789.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.58789.log.gz 2024-09-17T04:34:52.875 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60088.log 2024-09-17T04:34:52.876 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.62311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36639.log 2024-09-17T04:34:52.876 INFO:teuthology.orchestra.run.smithi100.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.62311.log.gz 2024-09-17T04:34:52.877 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.60088.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60088.log.gz 2024-09-17T04:34:52.877 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46785.log 2024-09-17T04:34:52.878 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56706.log 2024-09-17T04:34:52.878 INFO:teuthology.orchestra.run.smithi100.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.36639.log.gz 2024-09-17T04:34:52.879 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46785.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46785.log.gz 2024-09-17T04:34:52.879 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64839.log 2024-09-17T04:34:52.880 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75720.log 2024-09-17T04:34:52.880 INFO:teuthology.orchestra.run.smithi100.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56706.log.gz 2024-09-17T04:34:52.881 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.64839.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.64839.log.gz 2024-09-17T04:34:52.881 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42754.log 2024-09-17T04:34:52.882 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.75720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81247.log 2024-09-17T04:34:52.882 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75720.log.gz 2024-09-17T04:34:52.883 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54673.log 2024-09-17T04:34:52.883 INFO:teuthology.orchestra.run.smithi100.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.42754.log.gz 2024-09-17T04:34:52.884 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.81247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71203.log 2024-09-17T04:34:52.884 INFO:teuthology.orchestra.run.smithi100.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.81247.log.gz 2024-09-17T04:34:52.885 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54673.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54673.log.gz 2024-09-17T04:34:52.885 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36587.log 2024-09-17T04:34:52.886 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.71203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63117.log 2024-09-17T04:34:52.886 INFO:teuthology.orchestra.run.smithi100.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.71203.log.gz 2024-09-17T04:34:52.887 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69242.log 2024-09-17T04:34:52.887 INFO:teuthology.orchestra.run.smithi100.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36587.log.gz 2024-09-17T04:34:52.888 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.63117.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.63117.log.gz 2024-09-17T04:34:52.888 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52106.log 2024-09-17T04:34:52.889 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.69242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80785.log 2024-09-17T04:34:52.889 INFO:teuthology.orchestra.run.smithi100.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.69242.log.gz 2024-09-17T04:34:52.890 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.52106.log: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.52106.log.gz 2024-09-17T04:34:52.890 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.66926.log 2024-09-17T04:34:52.891 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.80785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51388.log 2024-09-17T04:34:52.891 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80785.log.gz 2024-09-17T04:34:52.892 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.66926.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66926.log.gz 2024-09-17T04:34:52.892 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65960.log 2024-09-17T04:34:52.893 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.51388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42406.log 2024-09-17T04:34:52.893 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51388.log.gz 2024-09-17T04:34:52.894 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.65960.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.65960.log.gz 2024-09-17T04:34:52.894 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79808.log 2024-09-17T04:34:52.895 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53966.log 2024-09-17T04:34:52.895 INFO:teuthology.orchestra.run.smithi100.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42406.log.gz 2024-09-17T04:34:52.896 INFO:teuthology.orchestra.run.smithi184.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-17T04:34:52.897 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.79808.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79808.log.gz 2024-09-17T04:34:52.897 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43084.log 2024-09-17T04:34:52.897 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.53966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64649.log 2024-09-17T04:34:52.897 INFO:teuthology.orchestra.run.smithi100.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.53966.log.gz 2024-09-17T04:34:52.898 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43084.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43084.log.gz 2024-09-17T04:34:52.898 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41453.log 2024-09-17T04:34:52.899 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.64649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70734.log 2024-09-17T04:34:52.899 INFO:teuthology.orchestra.run.smithi100.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.64649.log.gz 2024-09-17T04:34:52.900 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41453.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41453.log.gz 2024-09-17T04:34:52.900 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43456.log 2024-09-17T04:34:52.901 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.70734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42756.log 2024-09-17T04:34:52.901 INFO:teuthology.orchestra.run.smithi100.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.70734.log.gz 2024-09-17T04:34:52.902 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43456.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43456.log.gz 2024-09-17T04:34:52.902 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57613.log 2024-09-17T04:34:52.903 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47337.log 2024-09-17T04:34:52.903 INFO:teuthology.orchestra.run.smithi100.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42756.log.gz 2024-09-17T04:34:52.904 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.57613.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57613.log.gz 2024-09-17T04:34:52.904 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55599.log 2024-09-17T04:34:52.905 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.47337.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47337.log.gz 2024-09-17T04:34:52.905 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71666.log 2024-09-17T04:34:52.906 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68481.log 2024-09-17T04:34:52.906 INFO:teuthology.orchestra.run.smithi100.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55599.log.gz 2024-09-17T04:34:52.907 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.71666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61831.log 2024-09-17T04:34:52.907 INFO:teuthology.orchestra.run.smithi100.stderr: 83.5% -- replaced with /var/log/ceph/ceph-client.admin.71666.log.gz 2024-09-17T04:34:52.908 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.68481.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68481.log.gz 2024-09-17T04:34:52.908 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72928.log 2024-09-17T04:34:52.909 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.61831.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.0.61831.log.gz 2024-09-17T04:34:52.909 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.68223.log 2024-09-17T04:34:52.910 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.72928.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.72928.log.gz 2024-09-17T04:34:52.910 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70296.log 2024-09-17T04:34:52.911 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.68223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50180.log 2024-09-17T04:34:52.911 INFO:teuthology.orchestra.run.smithi100.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.68223.log.gz 2024-09-17T04:34:52.912 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.70296.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.70296.log.gz 2024-09-17T04:34:52.912 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51775.log 2024-09-17T04:34:52.913 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.50180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73929.log 2024-09-17T04:34:52.913 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50180.log.gz 2024-09-17T04:34:52.914 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.51775.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51775.log.gz 2024-09-17T04:34:52.914 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78312.log 2024-09-17T04:34:52.915 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.73929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78667.log 2024-09-17T04:34:52.915 INFO:teuthology.orchestra.run.smithi100.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.73929.log.gz 2024-09-17T04:34:52.916 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.78312.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78312.log.gz 2024-09-17T04:34:52.916 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80995.log 2024-09-17T04:34:52.917 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.78667.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T04:34:52.917 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78667.log.gz 2024-09-17T04:34:52.918 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.80995.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80995.log.gz 2024-09-17T04:34:52.918 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53766.log 2024-09-17T04:34:52.919 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52796.log 2024-09-17T04:34:52.920 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.53766.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53766.log.gz 2024-09-17T04:34:52.920 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50546.log 2024-09-17T04:34:52.921 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55070.log 2024-09-17T04:34:52.922 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.52796.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52796.log.gz 2024-09-17T04:34:52.922 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.50546.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50546.log.gz 2024-09-17T04:34:52.923 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55736.log 2024-09-17T04:34:52.923 INFO:teuthology.orchestra.run.smithi100.stderr: 89.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T04:34:52.923 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54715.log 2024-09-17T04:34:52.924 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55070.log: /var/log/ceph/ceph-client.admin.55736.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55070.log.gz 2024-09-17T04:34:52.924 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74568.log 2024-09-17T04:34:52.924 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55736.log.gz 2024-09-17T04:34:52.925 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75932.log 2024-09-17T04:34:52.925 INFO:teuthology.orchestra.run.smithi100.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54715.log.gz 2024-09-17T04:34:52.926 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.74568.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.74568.log.gz 2024-09-17T04:34:52.926 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46405.log 2024-09-17T04:34:52.927 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.75932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73971.log 2024-09-17T04:34:52.927 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75932.log.gz 2024-09-17T04:34:52.928 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46405.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46405.log.gz 2024-09-17T04:34:52.928 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69711.log 2024-09-17T04:34:52.929 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.73971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73341.log 2024-09-17T04:34:52.929 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73971.log.gz 2024-09-17T04:34:52.930 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.69711.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.69711.log.gz 2024-09-17T04:34:52.930 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67598.log 2024-09-17T04:34:52.931 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.73341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58999.log 2024-09-17T04:34:52.931 INFO:teuthology.orchestra.run.smithi100.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73341.log.gz 2024-09-17T04:34:52.932 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.67598.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67598.log.gz 2024-09-17T04:34:52.932 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48344.log 2024-09-17T04:34:52.933 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.58999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48915.log 2024-09-17T04:34:52.933 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58999.log.gz 2024-09-17T04:34:52.934 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.48344.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48344.log.gz 2024-09-17T04:34:52.934 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49125.log 2024-09-17T04:34:52.935 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.48915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55694.log 2024-09-17T04:34:52.935 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48915.log.gz 2024-09-17T04:34:52.936 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.49125.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49125.log.gz 2024-09-17T04:34:52.936 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53808.log 2024-09-17T04:34:52.937 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71502.log 2024-09-17T04:34:52.937 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55694.log.gz 2024-09-17T04:34:52.938 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.53808.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53808.log.gz 2024-09-17T04:34:52.938 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77010.log 2024-09-17T04:34:52.939 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.71502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80299.log 2024-09-17T04:34:52.939 INFO:teuthology.orchestra.run.smithi100.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.71502.log.gz 2024-09-17T04:34:52.940 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.77010.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.77010.log.gz 2024-09-17T04:34:52.940 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61997.log 2024-09-17T04:34:52.941 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.80299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56034.log 2024-09-17T04:34:52.941 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.80299.log.gz 2024-09-17T04:34:52.942 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.61997.log: 87.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.61997.log.gz 2024-09-17T04:34:52.942 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41552.log 2024-09-17T04:34:52.943 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78709.log 2024-09-17T04:34:52.943 INFO:teuthology.orchestra.run.smithi100.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56034.log.gz 2024-09-17T04:34:52.944 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41552.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41552.log.gz 2024-09-17T04:34:52.944 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41902.log 2024-09-17T04:34:52.946 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.78709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50466.log 2024-09-17T04:34:52.946 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78709.log.gz 2024-09-17T04:34:52.946 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41902.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41902.log.gz 2024-09-17T04:34:52.946 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44128.log 2024-09-17T04:34:52.947 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.50466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77763.log 2024-09-17T04:34:52.947 INFO:teuthology.orchestra.run.smithi100.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50466.log.gz 2024-09-17T04:34:52.948 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.44128.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44128.log.gz 2024-09-17T04:34:52.948 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52004.log 2024-09-17T04:34:52.949 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.77763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45833.log 2024-09-17T04:34:52.949 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.77763.log.gz 2024-09-17T04:34:52.950 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.52004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79297.log 2024-09-17T04:34:52.950 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52004.log.gz 2024-09-17T04:34:52.951 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.45833.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45833.log.gz 2024-09-17T04:34:52.951 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60172.log 2024-09-17T04:34:52.952 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.79297.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.79297.log.gzgzip 2024-09-17T04:34:52.952 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.59420.log 2024-09-17T04:34:52.953 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.60172.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60172.log.gz 2024-09-17T04:34:52.953 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76646.log 2024-09-17T04:34:52.954 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.59420.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59420.log.gz 2024-09-17T04:34:52.954 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79944.log 2024-09-17T04:34:52.954 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.76646.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.76646.log.gz 2024-09-17T04:34:52.955 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51649.log 2024-09-17T04:34:52.955 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.79944.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79944.log.gz 2024-09-17T04:34:52.955 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42280.log 2024-09-17T04:34:52.956 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.51649.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51649.log.gz 2024-09-17T04:34:52.956 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71896.log 2024-09-17T04:34:52.957 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42280.log: gzip -5 --verbose -- 87.3% /var/log/ceph/ceph-client.admin.62396.log -- replaced with /var/log/ceph/ceph-client.admin.42280.log.gz 2024-09-17T04:34:52.957 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-09-17T04:34:52.958 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.71896.log: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.71896.log.gz 2024-09-17T04:34:52.958 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49220.log 2024-09-17T04:34:52.959 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.62396.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.62396.log.gz 2024-09-17T04:34:52.959 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73299.log 2024-09-17T04:34:52.960 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.49220.log: gzip -5 87.9% --verbose -- replaced with /var/log/ceph/ceph-client.admin.49220.log.gz -- 2024-09-17T04:34:52.960 INFO:teuthology.orchestra.run.smithi100.stderr: /var/log/ceph/ceph-client.admin.48656.log 2024-09-17T04:34:52.961 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.73299.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73299.log.gz 2024-09-17T04:34:52.961 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61038.log 2024-09-17T04:34:52.962 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.48656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65437.log 2024-09-17T04:34:52.962 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48656.log.gz 2024-09-17T04:34:52.963 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.61038.log: 87.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.61038.log.gz 2024-09-17T04:34:52.963 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52369.log 2024-09-17T04:34:52.963 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.65437.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.65437.log.gz 2024-09-17T04:34:52.964 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59650.log 2024-09-17T04:34:52.964 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.52369.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52369.log.gz 2024-09-17T04:34:52.965 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80383.log 2024-09-17T04:34:52.965 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.59650.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.59650.log.gz 2024-09-17T04:34:52.966 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44931.log 2024-09-17T04:34:52.966 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.80383.log: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.80383.log.gz 2024-09-17T04:34:52.966 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46491.log 2024-09-17T04:34:52.967 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.44931.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44931.log.gz 2024-09-17T04:34:52.967 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75806.log 2024-09-17T04:34:52.968 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46491.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46491.log.gz 2024-09-17T04:34:52.968 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81079.log 2024-09-17T04:34:52.969 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.75806.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.75806.log.gz 2024-09-17T04:34:52.969 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47079.log 2024-09-17T04:34:52.970 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.81079.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.81079.log.gz 2024-09-17T04:34:52.970 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46449.log 2024-09-17T04:34:52.971 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.47079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81037.log 2024-09-17T04:34:52.972 INFO:teuthology.orchestra.run.smithi100.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.47079.log.gz 2024-09-17T04:34:52.972 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46449.log: gzip -5 86.5% --verbose -- replaced with /var/log/ceph/ceph-client.admin.46449.log.gz -- 2024-09-17T04:34:52.972 INFO:teuthology.orchestra.run.smithi100.stderr: /var/log/ceph/ceph-client.admin.36458.log 2024-09-17T04:34:52.973 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.81037.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.81037.log.gz -5 2024-09-17T04:34:52.973 INFO:teuthology.orchestra.run.smithi100.stderr: --verbose -- /var/log/ceph/ceph-client.admin.78985.log 2024-09-17T04:34:52.974 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36458.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36458.log.gz 2024-09-17T04:34:52.974 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36351.log 2024-09-17T04:34:52.975 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.78985.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78985.log.gz 2024-09-17T04:34:52.975 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46659.log 2024-09-17T04:34:52.976 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36351.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36351.log.gz 2024-09-17T04:34:52.976 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57289.log 2024-09-17T04:34:52.977 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46659.log: gzip 87.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.46659.log.gz --verbose 2024-09-17T04:34:52.977 INFO:teuthology.orchestra.run.smithi100.stderr: -- /var/log/ceph/ceph-client.admin.57153.log 2024-09-17T04:34:52.977 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.57289.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.57289.log.gz 2024-09-17T04:34:52.978 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75251.log 2024-09-17T04:34:52.978 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.57153.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57153.log.gz 2024-09-17T04:34:52.978 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48302.log 2024-09-17T04:34:52.979 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.75251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53604.log 2024-09-17T04:34:52.979 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.75251.log.gz 2024-09-17T04:34:52.980 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.48302.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48302.log.gz 2024-09-17T04:34:52.981 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78495.log 2024-09-17T04:34:52.981 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.53604.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53604.log.gz 2024-09-17T04:34:52.982 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68577.log 2024-09-17T04:34:52.982 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.78495.log: gzip 87.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.78495.log.gz 2024-09-17T04:34:52.982 INFO:teuthology.orchestra.run.smithi100.stderr: --verbose -- /var/log/ceph/ceph-client.admin.73067.log 2024-09-17T04:34:52.983 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.68577.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68577.log.gz 2024-09-17T04:34:52.983 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65750.log 2024-09-17T04:34:52.984 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.73067.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.73067.log.gz 2024-09-17T04:34:52.984 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46827.log 2024-09-17T04:34:52.985 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.65750.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.65750.log.gz 2024-09-17T04:34:52.985 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60951.log 2024-09-17T04:34:52.986 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46827.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46827.log.gz 2024-09-17T04:34:52.986 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51450.log 2024-09-17T04:34:52.987 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.60951.log: gzip -5 --verbose -- 87.7% /var/log/ceph/ceph-client.admin.72148.log -- replaced with /var/log/ceph/ceph-client.admin.60951.log.gz 2024-09-17T04:34:52.987 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-09-17T04:34:52.987 INFO:teuthology.orchestra.run.smithi184.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-17T04:34:52.988 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.51450.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51450.log.gz 2024-09-17T04:34:52.988 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80425.log 2024-09-17T04:34:52.988 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.72148.log: gzip -5 87.3% --verbose -- replaced with /var/log/ceph/ceph-client.admin.72148.log.gz -- 2024-09-17T04:34:52.988 INFO:teuthology.orchestra.run.smithi100.stderr: /var/log/ceph/ceph-client.admin.51060.log 2024-09-17T04:34:52.989 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.80425.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.80425.log.gz 2024-09-17T04:34:52.990 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67514.log 2024-09-17T04:34:52.990 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.51060.log: gzip -5 87.5% --verbose -- replaced with /var/log/ceph/ceph-client.admin.51060.log.gz -- 2024-09-17T04:34:52.990 INFO:teuthology.orchestra.run.smithi100.stderr: /var/log/ceph/ceph-client.admin.45994.log 2024-09-17T04:34:52.991 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.67514.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67514.log.gz 2024-09-17T04:34:52.991 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36197.log 2024-09-17T04:34:52.992 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.45994.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45994.log.gz 2024-09-17T04:34:52.992 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70834.log 2024-09-17T04:34:52.993 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36197.log: 76.0% -- replaced with /var/log/ceph/ceph-client.admin.36197.log.gz 2024-09-17T04:34:52.993 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65395.log 2024-09-17T04:34:52.994 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.70834.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.70834.log.gz 2024-09-17T04:34:52.994 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77385.log 2024-09-17T04:34:52.995 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.65395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81457.log 2024-09-17T04:34:52.995 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65395.log.gz 2024-09-17T04:34:52.996 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.77385.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77385.log.gz 2024-09-17T04:34:52.996 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79550.log 2024-09-17T04:34:52.997 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.81457.log: gzip 87.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.81457.log.gz 2024-09-17T04:34:52.997 INFO:teuthology.orchestra.run.smithi100.stderr: --verbose -- /var/log/ceph/ceph-client.admin.75023.log 2024-09-17T04:34:52.998 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.79550.log: gzip 87.3% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.79550.log.gz 2024-09-17T04:34:52.998 INFO:teuthology.orchestra.run.smithi100.stderr: -- /var/log/ceph/ceph-client.admin.59041.log 2024-09-17T04:34:52.999 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.75023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60359.log 2024-09-17T04:34:52.999 INFO:teuthology.orchestra.run.smithi100.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.75023.log.gz 2024-09-17T04:34:53.000 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.59041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50794.log 2024-09-17T04:34:53.000 INFO:teuthology.orchestra.run.smithi100.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59041.log.gz 2024-09-17T04:34:53.001 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.60359.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60359.log.gz 2024-09-17T04:34:53.001 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43083.log 2024-09-17T04:34:53.002 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.50794.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50794.log.gz 2024-09-17T04:34:53.002 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77267.log 2024-09-17T04:34:53.003 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43083.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43083.log.gz 2024-09-17T04:34:53.003 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59586.log 2024-09-17T04:34:53.003 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.77267.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77267.log.gz 2024-09-17T04:34:53.004 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78270.log 2024-09-17T04:34:53.005 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.59586.log: 87.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.59586.log.gz 2024-09-17T04:34:53.005 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.68803.log 2024-09-17T04:34:53.005 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.78270.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78270.log.gz 2024-09-17T04:34:53.006 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80701.log 2024-09-17T04:34:53.006 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.68803.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68803.log.gz 2024-09-17T04:34:53.006 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69421.log 2024-09-17T04:34:53.007 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.80701.log: gzip 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80701.log.gz -5 2024-09-17T04:34:53.007 INFO:teuthology.orchestra.run.smithi100.stderr: --verbose -- /var/log/ceph/ceph-client.admin.79886.log 2024-09-17T04:34:53.008 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.69421.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.69421.log.gz 2024-09-17T04:34:53.008 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49476.log 2024-09-17T04:34:53.009 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.79886.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79886.log.gz 2024-09-17T04:34:53.009 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45707.log 2024-09-17T04:34:53.010 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.49476.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49476.log.gz 2024-09-17T04:34:53.010 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54463.log 2024-09-17T04:34:53.011 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.45707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45665.log 2024-09-17T04:34:53.011 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45707.log.gz 2024-09-17T04:34:53.012 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79381.log 2024-09-17T04:34:53.012 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54463.log.gz 2024-09-17T04:34:53.013 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.45665.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45665.log.gz 2024-09-17T04:34:53.013 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69501.log 2024-09-17T04:34:53.014 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.79381.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79381.log.gz 2024-09-17T04:34:53.014 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36623.log 2024-09-17T04:34:53.014 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.69501.log: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.69501.log.gz 2024-09-17T04:34:53.014 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.80953.log 2024-09-17T04:34:53.015 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36623.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36623.log.gz 2024-09-17T04:34:53.016 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36252.log 2024-09-17T04:34:53.016 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.80953.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.80953.log.gz 2024-09-17T04:34:53.016 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79177.log 2024-09-17T04:34:53.017 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36252.log: 79.8% -- replaced with /var/log/ceph/ceph-client.admin.36252.log.gz 2024-09-17T04:34:53.017 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49644.log 2024-09-17T04:34:53.018 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.79177.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.79177.log.gz 2024-09-17T04:34:53.018 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68326.log 2024-09-17T04:34:53.019 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.49644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61134.log 2024-09-17T04:34:53.019 INFO:teuthology.orchestra.run.smithi100.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.49644.log.gz 2024-09-17T04:34:53.020 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.68326.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.68326.log.gz 2024-09-17T04:34:53.020 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80028.log 2024-09-17T04:34:53.021 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.61134.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61134.log.gz 2024-09-17T04:34:53.021 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62991.log 2024-09-17T04:34:53.022 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.80028.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.80028.log.gz 2024-09-17T04:34:53.022 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60485.log 2024-09-17T04:34:53.023 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.62991.log: 87.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.62991.log.gz 2024-09-17T04:34:53.023 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.72251.log 2024-09-17T04:34:53.024 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.60485.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60485.log.gz 2024-09-17T04:34:53.024 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55908.log 2024-09-17T04:34:53.025 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.72251.log: gzip -5 --verbose 87.3% -- -- replaced with /var/log/ceph/ceph-client.admin.72251.log.gz /var/log/ceph/ceph-client.admin.65521.log 2024-09-17T04:34:53.025 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-09-17T04:34:53.026 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55908.log: gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55908.log.gz 2024-09-17T04:34:53.026 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.72972.log 2024-09-17T04:34:53.027 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.65521.log: gzip -5 --verbose -- 87.4% /var/log/ceph/ceph-client.admin.61913.log -- replaced with /var/log/ceph/ceph-client.admin.65521.log.gz 2024-09-17T04:34:53.027 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-09-17T04:34:53.028 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.72972.log: gzip 87.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.72972.log.gz --verbose 2024-09-17T04:34:53.028 INFO:teuthology.orchestra.run.smithi100.stderr: -- /var/log/ceph/ceph-client.admin.47499.log 2024-09-17T04:34:53.028 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.61913.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61913.log.gz 2024-09-17T04:34:53.029 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50670.log 2024-09-17T04:34:53.029 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.47499.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47499.log.gz 2024-09-17T04:34:53.029 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36304.log 2024-09-17T04:34:53.030 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.50670.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50670.log.gz 2024-09-17T04:34:53.030 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57739.log 2024-09-17T04:34:53.031 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36304.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.36304.log.gz 2024-09-17T04:34:53.031 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73151.log 2024-09-17T04:34:53.032 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.57739.log: gzip 87.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.57739.log.gz --verbose 2024-09-17T04:34:53.032 INFO:teuthology.orchestra.run.smithi100.stderr: -- /var/log/ceph/ceph-client.admin.74395.log 2024-09-17T04:34:53.033 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.73151.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73151.log.gz 2024-09-17T04:34:53.033 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43540.log 2024-09-17T04:34:53.034 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.74395.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.74395.log.gz 2024-09-17T04:34:53.034 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80112.log 2024-09-17T04:34:53.035 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43540.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43540.log.gz 2024-09-17T04:34:53.035 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68368.log 2024-09-17T04:34:53.036 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.80112.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.80112.log.gz 2024-09-17T04:34:53.036 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49902.log 2024-09-17T04:34:53.037 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.68368.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.68368.log.gz 2024-09-17T04:34:53.037 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58617.log 2024-09-17T04:34:53.038 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.49902.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49902.log.gz 2024-09-17T04:34:53.038 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62438.log 2024-09-17T04:34:53.039 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.58617.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58617.log.gz 2024-09-17T04:34:53.039 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-17T04:34:53.040 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.62438.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.62438.log.gz 2024-09-17T04:34:53.040 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55992.log 2024-09-17T04:34:53.040 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54326.log 2024-09-17T04:34:53.042 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55992.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.55992.log.gz 2024-09-17T04:34:53.042 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53136.log 2024-09-17T04:34:53.043 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54326.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54326.log.gz 2024-09-17T04:34:53.043 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48740.log 2024-09-17T04:34:53.044 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.53136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53905.log 2024-09-17T04:34:53.044 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53136.log.gz 2024-09-17T04:34:53.045 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.48740.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48740.log.gz -5 2024-09-17T04:34:53.045 INFO:teuthology.orchestra.run.smithi100.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42490.log 2024-09-17T04:34:53.046 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.53905.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53905.log.gz 2024-09-17T04:34:53.046 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61391.log 2024-09-17T04:34:53.047 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52572.log 2024-09-17T04:34:53.048 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42490.log.gz 2024-09-17T04:34:53.048 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.61391.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.61391.log.gz 2024-09-17T04:34:53.048 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70488.log 2024-09-17T04:34:53.050 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.52572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63220.log 2024-09-17T04:34:53.050 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52572.log.gz 2024-09-17T04:34:53.050 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.70488.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.70488.log.gz 2024-09-17T04:34:53.051 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68887.log 2024-09-17T04:34:53.052 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.63220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44584.log 2024-09-17T04:34:53.052 INFO:teuthology.orchestra.run.smithi100.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.63220.log.gz 2024-09-17T04:34:53.053 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.68887.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68887.log.gz 2024-09-17T04:34:53.053 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59299.log 2024-09-17T04:34:53.054 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.2.44584.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.44584.log.gz 2024-09-17T04:34:53.054 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44805.log 2024-09-17T04:34:53.055 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.59299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37199.log 2024-09-17T04:34:53.055 INFO:teuthology.orchestra.run.smithi100.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59299.log.gz 2024-09-17T04:34:53.057 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.44805.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44805.log.gz 2024-09-17T04:34:53.057 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58435.log 2024-09-17T04:34:53.057 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.37199.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37199.log.gz 2024-09-17T04:34:53.058 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53304.log 2024-09-17T04:34:53.058 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.58435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59462.log 2024-09-17T04:34:53.059 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58435.log.gz 2024-09-17T04:34:53.060 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.53304.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53304.log.gz 2024-09-17T04:34:53.060 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57434.log 2024-09-17T04:34:53.061 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.59462.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59462.log.gz 2024-09-17T04:34:53.061 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61082.log 2024-09-17T04:34:53.062 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.57434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41944.log 2024-09-17T04:34:53.062 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57434.log.gz 2024-09-17T04:34:53.063 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.61082.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.61082.log.gz 2024-09-17T04:34:53.063 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55782.log 2024-09-17T04:34:53.065 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50874.log 2024-09-17T04:34:53.065 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41944.log.gz 2024-09-17T04:34:53.069 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55782.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55782.log.gz 2024-09-17T04:34:53.069 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55249.log 2024-09-17T04:34:53.069 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.50874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42746.log 2024-09-17T04:34:53.069 INFO:teuthology.orchestra.run.smithi100.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.50874.log.gz 2024-09-17T04:34:53.069 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55249.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55249.log.gz 2024-09-17T04:34:53.069 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60401.log 2024-09-17T04:34:53.070 INFO:teuthology.orchestra.run.smithi184.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-17T04:34:53.070 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70566.log 2024-09-17T04:34:53.071 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42746.log.gz 2024-09-17T04:34:53.071 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.60401.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60401.log.gz 2024-09-17T04:34:53.071 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75107.log 2024-09-17T04:34:53.071 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.70566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62354.log 2024-09-17T04:34:53.072 INFO:teuthology.orchestra.run.smithi100.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70566.log.gz 2024-09-17T04:34:53.073 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.75107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77847.log 2024-09-17T04:34:53.073 INFO:teuthology.orchestra.run.smithi100.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.75107.log.gz 2024-09-17T04:34:53.074 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.62354.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.62354.log.gz 2024-09-17T04:34:53.074 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42070.log 2024-09-17T04:34:53.074 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.77847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68157.log 2024-09-17T04:34:53.075 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77847.log.gz 2024-09-17T04:34:53.076 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55643.log 2024-09-17T04:34:53.076 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42070.log.gz 2024-09-17T04:34:53.077 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.68157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73257.log 2024-09-17T04:34:53.077 INFO:teuthology.orchestra.run.smithi100.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.68157.log.gz 2024-09-17T04:34:53.078 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55643.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55643.log.gz 2024-09-17T04:34:53.078 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42112.log 2024-09-17T04:34:53.079 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.73257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57697.log 2024-09-17T04:34:53.079 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73257.log.gz 2024-09-17T04:34:53.080 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42112.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42112.log.gz 2024-09-17T04:34:53.080 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49169.log 2024-09-17T04:34:53.081 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.57697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71379.log 2024-09-17T04:34:53.082 INFO:teuthology.orchestra.run.smithi100.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57697.log.gz 2024-09-17T04:34:53.083 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.49169.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49169.log.gzgzip 2024-09-17T04:34:53.083 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.73109.log 2024-09-17T04:34:53.084 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.71379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56544.log 2024-09-17T04:34:53.084 INFO:teuthology.orchestra.run.smithi100.stderr: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.71379.log.gz 2024-09-17T04:34:53.085 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.73109.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73109.log.gz 2024-09-17T04:34:53.085 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71244.log 2024-09-17T04:34:53.085 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.56544.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56544.log.gz 2024-09-17T04:34:53.086 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53346.log 2024-09-17T04:34:53.087 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.71244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72885.log 2024-09-17T04:34:53.087 INFO:teuthology.orchestra.run.smithi100.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.71244.log.gz 2024-09-17T04:34:53.088 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.53346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46163.log 2024-09-17T04:34:53.088 INFO:teuthology.orchestra.run.smithi100.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53346.log.gz 2024-09-17T04:34:53.089 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.72885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42322.log 2024-09-17T04:34:53.089 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.72885.log.gz 2024-09-17T04:34:53.090 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76688.log 2024-09-17T04:34:53.091 INFO:teuthology.orchestra.run.smithi100.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46163.log.gz 2024-09-17T04:34:53.091 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42322.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42322.log.gz 2024-09-17T04:34:53.091 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79255.log 2024-09-17T04:34:53.092 INFO:teuthology.orchestra.run.smithi100.stderr:gzip/var/log/ceph/ceph-client.admin.76688.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.58010.log 2024-09-17T04:34:53.093 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76688.log.gz 2024-09-17T04:34:53.093 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.79255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58136.log 2024-09-17T04:34:53.094 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.79255.log.gz 2024-09-17T04:34:53.095 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.58010.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58010.log.gz 2024-09-17T04:34:53.095 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80869.log 2024-09-17T04:34:53.096 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.58136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52274.log 2024-09-17T04:34:53.096 INFO:teuthology.orchestra.run.smithi100.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.58136.log.gz 2024-09-17T04:34:53.097 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.80869.log: gzip 87.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.80869.log.gz --verbose 2024-09-17T04:34:53.097 INFO:teuthology.orchestra.run.smithi100.stderr: -- /var/log/ceph/ceph-client.admin.44721.log 2024-09-17T04:34:53.098 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.52274.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52274.log.gz 2024-09-17T04:34:53.098 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45057.log 2024-09-17T04:34:53.099 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.44721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72377.log 2024-09-17T04:34:53.099 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44721.log.gz 2024-09-17T04:34:53.100 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.45057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72106.log 2024-09-17T04:34:53.100 INFO:teuthology.orchestra.run.smithi100.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45057.log.gz 2024-09-17T04:34:53.101 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.72377.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72377.log.gz 2024-09-17T04:34:53.101 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81569.log 2024-09-17T04:34:53.102 INFO:teuthology.orchestra.run.smithi100.stderr:gzip/var/log/ceph/ceph-client.admin.72106.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.62269.log 2024-09-17T04:34:53.103 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.72106.log.gz 2024-09-17T04:34:53.103 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.81569.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.81569.log.gz 2024-09-17T04:34:53.103 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65563.log 2024-09-17T04:34:53.105 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.62269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61177.log 2024-09-17T04:34:53.105 INFO:teuthology.orchestra.run.smithi100.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.62269.log.gz 2024-09-17T04:34:53.106 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.65563.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.65563.log.gz 2024-09-17T04:34:53.106 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44147.log 2024-09-17T04:34:53.107 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.61177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57007.log 2024-09-17T04:34:53.107 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61177.log.gz 2024-09-17T04:34:53.107 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.44147.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44147.log.gz 2024-09-17T04:34:53.108 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59546.log 2024-09-17T04:34:53.109 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.57007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47819.log 2024-09-17T04:34:53.109 INFO:teuthology.orchestra.run.smithi100.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.57007.log.gz 2024-09-17T04:34:53.110 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.59546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58957.log 2024-09-17T04:34:53.110 INFO:teuthology.orchestra.run.smithi100.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.59546.log.gz 2024-09-17T04:34:53.111 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.47819.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47819.log.gz 2024-09-17T04:34:53.112 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70188.log 2024-09-17T04:34:53.112 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.58957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48614.log 2024-09-17T04:34:53.113 INFO:teuthology.orchestra.run.smithi100.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58957.log.gz 2024-09-17T04:34:53.113 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.70188.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70188.log.gz 2024-09-17T04:34:53.114 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48530.log 2024-09-17T04:34:53.114 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.48614.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48614.log.gz 2024-09-17T04:34:53.115 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36015.log 2024-09-17T04:34:53.116 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.48530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77805.log 2024-09-17T04:34:53.116 INFO:teuthology.orchestra.run.smithi100.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48530.log.gz 2024-09-17T04:34:53.117 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36015.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36015.log.gz 2024-09-17T04:34:53.117 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72064.log 2024-09-17T04:34:53.118 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.77805.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-17T04:34:53.118 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.77805.log.gz 2024-09-17T04:34:53.119 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.72064.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.72064.log.gz 2024-09-17T04:34:53.119 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80743.log 2024-09-17T04:34:53.120 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69583.log 2024-09-17T04:34:53.121 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.80743.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.80743.log.gz 2024-09-17T04:34:53.122 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55866.log 2024-09-17T04:34:53.123 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.69583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43288.log 2024-09-17T04:34:53.123 INFO:teuthology.orchestra.run.smithi100.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.69583.log.gz 2024-09-17T04:34:53.124 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55866.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55866.log.gz 2024-09-17T04:34:53.124 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77076.log 2024-09-17T04:34:53.125 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46360.log 2024-09-17T04:34:53.125 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43288.log.gz 2024-09-17T04:34:53.126 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.77076.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.77076.log.gz 2024-09-17T04:34:53.126 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65834.log 2024-09-17T04:34:53.127 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46360.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46360.log.gz 2024-09-17T04:34:53.127 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36412.log 2024-09-17T04:34:53.128 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.65834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57800.log 2024-09-17T04:34:53.129 INFO:teuthology.orchestra.run.smithi100.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.65834.log.gz 2024-09-17T04:34:53.129 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36412.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36412.log.gz 2024-09-17T04:34:53.129 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54986.log 2024-09-17T04:34:53.130 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47759.log/var/log/ceph/ceph-client.admin.57800.log: 2024-09-17T04:34:53.131 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57800.log.gz 2024-09-17T04:34:53.131 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54986.log: 87.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71980.log 2024-09-17T04:34:53.131 INFO:teuthology.orchestra.run.smithi100.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54986.log.gz 2024-09-17T04:34:53.132 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.47759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77427.log 2024-09-17T04:34:53.133 INFO:teuthology.orchestra.run.smithi100.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47759.log.gz 2024-09-17T04:34:53.133 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.71980.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71980.log.gz 2024-09-17T04:34:53.133 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45015.log 2024-09-17T04:34:53.134 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54589.log 2024-09-17T04:34:53.135 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.77427.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77427.log.gz 2024-09-17T04:34:53.136 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.45015.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45015.log.gz 2024-09-17T04:34:53.136 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67897.log 2024-09-17T04:34:53.136 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36664.log 2024-09-17T04:34:53.137 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54589.log.gz 2024-09-17T04:34:53.138 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.67897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71080.log 2024-09-17T04:34:53.138 INFO:teuthology.orchestra.run.smithi100.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.67897.log.gz 2024-09-17T04:34:53.139 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36664.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.36664.log.gz 2024-09-17T04:34:53.139 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73767.log 2024-09-17T04:34:53.140 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71760.log 2024-09-17T04:34:53.141 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.71080.log: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.71080.log.gz 2024-09-17T04:34:53.141 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.73767.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.73767.log.gz 2024-09-17T04:34:53.141 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62039.log 2024-09-17T04:34:53.142 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70020.log 2024-09-17T04:34:53.142 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.71760.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71760.log.gz 2024-09-17T04:34:53.143 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.62039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64736.log 2024-09-17T04:34:53.143 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62039.log.gz 2024-09-17T04:34:53.144 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.70020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77553.log 2024-09-17T04:34:53.144 INFO:teuthology.orchestra.run.smithi100.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.70020.log.gz 2024-09-17T04:34:53.145 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.64736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43762.log 2024-09-17T04:34:53.146 INFO:teuthology.orchestra.run.smithi100.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.64736.log.gz 2024-09-17T04:34:53.146 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.77553.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77553.log.gz 2024-09-17T04:34:53.146 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36645.log 2024-09-17T04:34:53.147 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74981.log 2024-09-17T04:34:53.148 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43762.log.gz 2024-09-17T04:34:53.149 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51565.log 2024-09-17T04:34:53.149 INFO:teuthology.orchestra.run.smithi100.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36645.log.gz 2024-09-17T04:34:53.150 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.74981.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.74981.log.gz 2024-09-17T04:34:53.150 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74752.log 2024-09-17T04:34:53.151 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.51565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63314.log 2024-09-17T04:34:53.151 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51565.log.gz 2024-09-17T04:34:53.152 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.74752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73016.log 2024-09-17T04:34:53.152 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74752.log.gz 2024-09-17T04:34:53.153 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.63314.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.63314.log.gz 2024-09-17T04:34:53.153 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68035.log 2024-09-17T04:34:53.154 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.73016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46248.log 2024-09-17T04:34:53.155 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73016.log.gz 2024-09-17T04:34:53.155 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.68035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36477.log 2024-09-17T04:34:53.156 INFO:teuthology.orchestra.run.smithi100.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.68035.log.gz 2024-09-17T04:34:53.156 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46248.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46248.log.gz 2024-09-17T04:34:53.156 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76604.log 2024-09-17T04:34:53.157 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61955.log 2024-09-17T04:34:53.158 INFO:teuthology.orchestra.run.smithi100.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.36477.log.gz 2024-09-17T04:34:53.159 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.76604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58701.log 2024-09-17T04:34:53.159 INFO:teuthology.orchestra.run.smithi100.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.76604.log.gz 2024-09-17T04:34:53.160 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.61955.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61955.log.gz 2024-09-17T04:34:53.160 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46617.log 2024-09-17T04:34:53.161 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.58701.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-17T04:34:53.161 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58701.log.gz 2024-09-17T04:34:53.162 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56352.log 2024-09-17T04:34:53.162 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46617.log.gz 2024-09-17T04:34:53.163 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68845.log 2024-09-17T04:34:53.165 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59690.log 2024-09-17T04:34:53.165 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56352.log.gz 2024-09-17T04:34:53.166 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.68845.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68845.log.gz 2024-09-17T04:34:53.166 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45497.log 2024-09-17T04:34:53.167 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.59690.log: 85.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60653.log 2024-09-17T04:34:53.167 INFO:teuthology.orchestra.run.smithi100.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59690.log.gz 2024-09-17T04:34:53.168 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.45497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59378.log 2024-09-17T04:34:53.168 INFO:teuthology.orchestra.run.smithi100.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45497.log.gz 2024-09-17T04:34:53.170 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.60653.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.60653.log.gz 2024-09-17T04:34:53.170 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78625.log 2024-09-17T04:34:53.171 INFO:teuthology.orchestra.run.smithi184.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-17T04:34:53.172 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.59378.log: gzip 87.9% -5 --verbose -- /var/log/ceph/ceph-client.admin.45455.log 2024-09-17T04:34:53.172 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.78625.log: -- replaced with /var/log/ceph/ceph-client.admin.59378.log.gz 2024-09-17T04:34:53.173 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.78625.log.gz 2024-09-17T04:34:53.173 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76184.log 2024-09-17T04:34:53.174 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.45455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42532.log 2024-09-17T04:34:53.175 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45455.log.gz 2024-09-17T04:34:53.176 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.76184.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.76184.log.gz 2024-09-17T04:34:53.176 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50064.log 2024-09-17T04:34:53.177 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42532.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42532.log.gz 2024-09-17T04:34:53.177 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54860.log 2024-09-17T04:34:53.178 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.50064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55112.log 2024-09-17T04:34:53.178 INFO:teuthology.orchestra.run.smithi100.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50064.log.gz 2024-09-17T04:34:53.180 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54860.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54860.log.gz 2024-09-17T04:34:53.180 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74920.log 2024-09-17T04:34:53.181 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55112.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55112.log.gz 2024-09-17T04:34:53.181 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57842.log 2024-09-17T04:34:53.183 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.74920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71543.log 2024-09-17T04:34:53.183 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74920.log.gz 2024-09-17T04:34:53.184 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.57842.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57842.log.gz 2024-09-17T04:34:53.185 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.71543.log: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.71543.log.gz 2024-09-17T04:34:53.185 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47656.log 2024-09-17T04:34:53.186 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51326.log 2024-09-17T04:34:53.188 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.47656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60130.log 2024-09-17T04:34:53.188 INFO:teuthology.orchestra.run.smithi100.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47656.log.gz 2024-09-17T04:34:53.189 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.51326.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51326.log.gz 2024-09-17T04:34:53.190 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.60130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72682.log 2024-09-17T04:34:53.190 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60130.log.gz 2024-09-17T04:34:53.191 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72190.log 2024-09-17T04:34:53.192 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60527.log 2024-09-17T04:34:53.194 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.72682.log: /var/log/ceph/ceph-client.admin.72190.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72682.log.gz 2024-09-17T04:34:53.194 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.72190.log.gz 2024-09-17T04:34:53.195 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54008.log/var/log/ceph/ceph-client.admin.60527.log: 2024-09-17T04:34:53.195 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60527.log.gz 2024-09-17T04:34:53.196 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76142.log 2024-09-17T04:34:53.197 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58522.log 2024-09-17T04:34:53.197 INFO:teuthology.orchestra.run.smithi100.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54008.log.gz 2024-09-17T04:34:53.198 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.76142.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.76142.log.gz 2024-09-17T04:34:53.199 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.58522.log: -- /var/log/ceph/ceph-client.admin.77511.log 2024-09-17T04:34:53.199 INFO:teuthology.orchestra.run.smithi100.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58522.log.gz 2024-09-17T04:34:53.201 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51962.log 2024-09-17T04:34:53.202 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.77511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46205.log 2024-09-17T04:34:53.202 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.77511.log.gz 2024-09-17T04:34:53.204 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.51962.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51962.log.gz 2024-09-17T04:34:53.204 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5/var/log/ceph/ceph-client.admin.46205.log: --verbose -- /var/log/ceph/ceph-client.admin.45791.log 2024-09-17T04:34:53.205 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46205.log.gz 2024-09-17T04:34:53.205 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76100.log 2024-09-17T04:34:53.207 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.45791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51817.log 2024-09-17T04:34:53.207 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45791.log.gz 2024-09-17T04:34:53.208 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.76100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77595.log 2024-09-17T04:34:53.208 INFO:teuthology.orchestra.run.smithi100.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.76100.log.gz 2024-09-17T04:34:53.209 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.51817.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51817.log.gz 2024-09-17T04:34:53.210 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.77595.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.77595.log.gz 2024-09-17T04:34:53.211 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44588.log 2024-09-17T04:34:53.212 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69200.log 2024-09-17T04:34:53.213 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.2.44588.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.44588.log.gz 2024-09-17T04:34:53.214 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.69200.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69200.log.gz 2024-09-17T04:34:53.215 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67556.log 2024-09-17T04:34:53.216 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79695.log 2024-09-17T04:34:53.218 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.67556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75335.log 2024-09-17T04:34:53.218 INFO:teuthology.orchestra.run.smithi100.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.67556.log.gz 2024-09-17T04:34:53.219 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.79695.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.79695.log.gz 2024-09-17T04:34:53.219 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-17T04:34:53.221 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.75335.log: gzip -5 87.5% --verbose -- replaced with /var/log/ceph/ceph-client.admin.75335.log.gz -- 2024-09-17T04:34:53.221 INFO:teuthology.orchestra.run.smithi100.stderr: /var/log/ceph/ceph-client.admin.56822.log 2024-09-17T04:34:53.222 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67136.log 2024-09-17T04:34:53.223 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56822.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56822.log.gz 2024-09-17T04:34:53.224 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60569.log 2024-09-17T04:34:53.225 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.67136.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.67136.log.gz 2024-09-17T04:34:53.226 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.60569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49392.log 2024-09-17T04:34:53.227 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60569.log.gz 2024-09-17T04:34:53.228 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52884.log 2024-09-17T04:34:53.229 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.49392.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49392.log.gz 2024-09-17T04:34:53.230 INFO:teuthology.orchestra.run.smithi100.stderr:gzip/var/log/ceph/ceph-client.admin.52884.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.52659.log 2024-09-17T04:34:53.230 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52884.log.gz 2024-09-17T04:34:53.231 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41730.log 2024-09-17T04:34:53.233 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.52659.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52659.log.gz 2024-09-17T04:34:53.234 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.41730.log: /var/log/ceph/ceph-client.admin.42755.log 2024-09-17T04:34:53.234 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41730.log.gz 2024-09-17T04:34:53.235 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53052.log 2024-09-17T04:34:53.236 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42755.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42755.log.gz 2024-09-17T04:34:53.237 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64881.log 2024-09-17T04:34:53.237 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.53052.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.53052.log.gz 2024-09-17T04:34:53.238 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66362.log 2024-09-17T04:34:53.239 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.64881.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.64881.log.gz 2024-09-17T04:34:53.240 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58566.log 2024-09-17T04:34:53.241 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.66362.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66362.log.gz 2024-09-17T04:34:53.242 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43414.log 2024-09-17T04:34:53.243 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.58566.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58566.log.gz 2024-09-17T04:34:53.244 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69806.log 2024-09-17T04:34:53.244 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43414.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43414.log.gz 2024-09-17T04:34:53.245 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67390.log 2024-09-17T04:34:53.246 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.69806.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69806.log.gz 2024-09-17T04:34:53.247 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65216.log 2024-09-17T04:34:53.248 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.67390.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67390.log.gz 2024-09-17T04:34:53.249 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66544.log 2024-09-17T04:34:53.250 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.65216.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65216.log.gz 2024-09-17T04:34:53.251 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71461.log/var/log/ceph/ceph-client.admin.66544.log: 2024-09-17T04:34:53.251 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66544.log.gz 2024-09-17T04:34:53.252 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66046.log 2024-09-17T04:34:53.253 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.71461.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.71461.log.gz 2024-09-17T04:34:53.255 INFO:teuthology.orchestra.run.smithi100.stderr:gzip/var/log/ceph/ceph-client.admin.66046.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.68266.log 2024-09-17T04:34:53.255 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66046.log.gz 2024-09-17T04:34:53.256 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52190.log 2024-09-17T04:34:53.257 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.68266.log: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.68266.log.gz 2024-09-17T04:34:53.258 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T04:34:53.258 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.52190.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52190.log.gz 2024-09-17T04:34:53.259 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60909.log 2024-09-17T04:34:53.261 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.60909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47333.log 2024-09-17T04:34:53.263 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60909.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36230.log 2024-09-17T04:34:53.263 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-09-17T04:34:53.264 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.47333.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47333.log.gz 2024-09-17T04:34:53.265 INFO:teuthology.orchestra.run.smithi100.stderr: 86.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T04:34:53.265 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78444.log 2024-09-17T04:34:53.266 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36230.log: 77.4% -- replaced with /var/log/ceph/ceph-client.admin.36230.log.gz 2024-09-17T04:34:53.267 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49980.log 2024-09-17T04:34:53.268 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.78444.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78444.log.gz 2024-09-17T04:34:53.269 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66884.log 2024-09-17T04:34:53.270 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.49980.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49980.log.gz 2024-09-17T04:34:53.271 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80196.log 2024-09-17T04:34:53.272 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.66884.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.66884.log.gz 2024-09-17T04:34:53.272 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42574.log 2024-09-17T04:34:53.273 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.80196.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.80196.log.gz 2024-09-17T04:34:53.274 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58393.log 2024-09-17T04:34:53.275 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42574.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42574.log.gz 2024-09-17T04:34:53.275 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78356.log 2024-09-17T04:34:53.276 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.58393.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58393.log.gz 2024-09-17T04:34:53.277 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71338.log 2024-09-17T04:34:53.278 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.78356.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78356.log.gz 2024-09-17T04:34:53.279 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54547.log 2024-09-17T04:34:53.280 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.71338.log: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.71338.log.gz 2024-09-17T04:34:53.281 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76876.log 2024-09-17T04:34:53.282 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54547.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54547.log.gz 2024-09-17T04:34:53.283 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70916.log 2024-09-17T04:34:53.284 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.76876.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.76876.log.gz 2024-09-17T04:34:53.284 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69755.log 2024-09-17T04:34:53.285 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.70916.log: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.70916.log.gz 2024-09-17T04:34:53.286 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78579.log 2024-09-17T04:34:53.287 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.69755.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.69755.log.gz 2024-09-17T04:34:53.287 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47698.log/var/log/ceph/ceph-client.admin.78579.log: 2024-09-17T04:34:53.288 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78579.log.gz 2024-09-17T04:34:53.289 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63178.log 2024-09-17T04:34:53.290 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.47698.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47698.log.gz 2024-09-17T04:34:53.291 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69158.log 2024-09-17T04:34:53.292 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.63178.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.63178.log.gz 2024-09-17T04:34:53.293 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45623.log 2024-09-17T04:34:53.294 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.69158.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69158.log.gz 2024-09-17T04:34:53.295 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53600.log 2024-09-17T04:34:53.295 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.45623.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45623.log.gz 2024-09-17T04:34:53.296 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76016.log 2024-09-17T04:34:53.297 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.53600.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53600.log.gz 2024-09-17T04:34:53.298 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56202.log 2024-09-17T04:34:53.299 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.76016.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76016.log.gz 2024-09-17T04:34:53.300 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36667.log 2024-09-17T04:34:53.300 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56202.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56202.log.gz 2024-09-17T04:34:53.301 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50404.log 2024-09-17T04:34:53.302 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36667.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.36667.log.gz 2024-09-17T04:34:53.303 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.50404.log: -- /var/log/ceph/ceph-client.admin.67993.log 2024-09-17T04:34:53.303 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50404.log.gz 2024-09-17T04:34:53.305 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.67993.log: -- /var/log/ceph/ceph-client.admin.50300.log 2024-09-17T04:34:53.306 INFO:teuthology.orchestra.run.smithi100.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.67993.log.gz 2024-09-17T04:34:53.306 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79611.log 2024-09-17T04:34:53.308 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.50300.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50300.log.gz 2024-09-17T04:34:53.308 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54213.log 2024-09-17T04:34:53.309 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.79611.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.79611.log.gz 2024-09-17T04:34:53.310 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75676.log 2024-09-17T04:34:53.311 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54213.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54213.log.gz 2024-09-17T04:34:53.312 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80827.log 2024-09-17T04:34:53.313 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.75676.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.75676.log.gz 2024-09-17T04:34:53.313 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73845.log 2024-09-17T04:34:53.314 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.80827.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.80827.log.gz 2024-09-17T04:34:53.315 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60611.log/var/log/ceph/ceph-client.admin.73845.log: 2024-09-17T04:34:53.316 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73845.log.gz 2024-09-17T04:34:53.317 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69284.log 2024-09-17T04:34:53.318 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.60611.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60611.log.gz 2024-09-17T04:34:53.319 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70650.log 2024-09-17T04:34:53.319 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.69284.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.69284.log.gz 2024-09-17T04:34:53.320 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51264.log 2024-09-17T04:34:53.322 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.70650.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.70650.log.gz 2024-09-17T04:34:53.322 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.51264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69936.log 2024-09-17T04:34:53.323 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51264.log.gz 2024-09-17T04:34:53.324 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78537.log 2024-09-17T04:34:53.325 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.69936.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69936.log.gz 2024-09-17T04:34:53.326 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36423.log 2024-09-17T04:34:53.327 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.78537.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.78537.log.gz 2024-09-17T04:34:53.327 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71838.log 2024-09-17T04:34:53.329 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36423.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.36423.log.gz 2024-09-17T04:34:53.330 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5/var/log/ceph/ceph-client.admin.71838.log: --verbose -- /var/log/ceph/ceph-client.admin.66800.log 2024-09-17T04:34:53.330 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71838.log.gz 2024-09-17T04:34:53.331 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77973.log 2024-09-17T04:34:53.332 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.66800.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.66800.log.gz 2024-09-17T04:34:53.333 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52838.log 2024-09-17T04:34:53.333 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.77973.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77973.log.gz 2024-09-17T04:34:53.334 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76480.log 2024-09-17T04:34:53.335 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.52838.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52838.log.gz 2024-09-17T04:34:53.336 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68929.log 2024-09-17T04:34:53.337 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.76480.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.76480.log.gz 2024-09-17T04:34:53.337 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53178.log 2024-09-17T04:34:53.339 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.68929.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.68929.log.gz 2024-09-17T04:34:53.340 INFO:teuthology.orchestra.run.smithi100.stderr:gzip/var/log/ceph/ceph-client.admin.53178.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50998.log 2024-09-17T04:34:53.340 INFO:teuthology.orchestra.run.smithi100.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.53178.log.gz 2024-09-17T04:34:53.341 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43372.log 2024-09-17T04:34:53.342 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.50998.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50998.log.gz 2024-09-17T04:34:53.343 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59295.log 2024-09-17T04:34:53.344 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43372.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43372.log.gz 2024-09-17T04:34:53.345 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67010.log 2024-09-17T04:34:53.345 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.59295.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59295.log.gz 2024-09-17T04:34:53.346 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71285.log 2024-09-17T04:34:53.347 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.67010.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.67010.log.gz 2024-09-17T04:34:53.348 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55824.log 2024-09-17T04:34:53.349 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.71285.log: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.71285.log.gz 2024-09-17T04:34:53.350 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75430.log 2024-09-17T04:34:53.351 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55824.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55824.log.gz 2024-09-17T04:34:53.351 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78835.log 2024-09-17T04:34:53.352 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.75430.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.75430.log.gz 2024-09-17T04:34:53.353 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66159.log 2024-09-17T04:34:53.354 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.78835.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78835.log.gz 2024-09-17T04:34:53.355 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55154.log 2024-09-17T04:34:53.356 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.66159.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.66159.log.gz 2024-09-17T04:34:53.356 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77889.log 2024-09-17T04:34:53.358 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55154.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55154.log.gz 2024-09-17T04:34:53.358 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48835.log 2024-09-17T04:34:53.359 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.77889.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.77889.log.gz 2024-09-17T04:34:53.360 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48050.log 2024-09-17T04:34:53.361 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.48835.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48835.log.gz 2024-09-17T04:34:53.362 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56622.log 2024-09-17T04:34:53.363 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.48050.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48050.log.gz 2024-09-17T04:34:53.363 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70793.log 2024-09-17T04:34:53.364 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56622.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56622.log.gz 2024-09-17T04:34:53.365 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56160.log 2024-09-17T04:34:53.366 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.70793.log: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.70793.log.gz 2024-09-17T04:34:53.367 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60748.log 2024-09-17T04:34:53.368 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56160.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56160.log.gz 2024-09-17T04:34:53.369 INFO:teuthology.orchestra.run.smithi100.stderr:gzip/var/log/ceph/ceph-client.admin.60748.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.58312.log 2024-09-17T04:34:53.369 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60748.log.gz 2024-09-17T04:34:53.370 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41648.log 2024-09-17T04:34:53.371 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.58312.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58312.log.gz 2024-09-17T04:34:53.372 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67202.log 2024-09-17T04:34:53.373 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41648.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41648.log.gz 2024-09-17T04:34:53.374 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61223.log 2024-09-17T04:34:53.375 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.67202.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67202.log.gz 2024-09-17T04:34:53.376 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42154.log 2024-09-17T04:34:53.377 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.61223.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61223.log.gz 2024-09-17T04:34:53.377 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44847.log 2024-09-17T04:34:53.378 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42154.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42154.log.gz 2024-09-17T04:34:53.380 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.44847.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44847.log.gz 2024-09-17T04:34:53.380 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70062.log 2024-09-17T04:34:53.382 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36996.log 2024-09-17T04:34:53.383 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.70062.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70062.log.gz 2024-09-17T04:34:53.384 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73467.log 2024-09-17T04:34:53.384 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36996.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36996.log.gz 2024-09-17T04:34:53.385 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76292.log 2024-09-17T04:34:53.386 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.73467.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.73467.log.gz 2024-09-17T04:34:53.387 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48572.log 2024-09-17T04:34:53.388 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.76292.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76292.log.gz 2024-09-17T04:34:53.389 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48092.log 2024-09-17T04:34:53.389 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.48572.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48572.log.gz 2024-09-17T04:34:53.390 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42977.log 2024-09-17T04:34:53.391 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.48092.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48092.log.gz 2024-09-17T04:34:53.392 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81163.log 2024-09-17T04:34:53.393 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42977.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42977.log.gz 2024-09-17T04:34:53.394 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56244.log 2024-09-17T04:34:53.395 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.81163.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.81163.log.gz 2024-09-17T04:34:53.395 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50106.log 2024-09-17T04:34:53.397 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56244.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56244.log.gz 2024-09-17T04:34:53.397 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.50106.log: -- /var/log/ceph/ceph-client.admin.52968.log 2024-09-17T04:34:53.398 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50106.log.gz 2024-09-17T04:34:53.398 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50362.log 2024-09-17T04:34:53.400 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.52968.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52968.log.gz 2024-09-17T04:34:53.401 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49560.log 2024-09-17T04:34:53.402 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.50362.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50362.log.gz 2024-09-17T04:34:53.402 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61433.log 2024-09-17T04:34:53.403 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.49560.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49560.log.gz 2024-09-17T04:34:53.404 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49602.log 2024-09-17T04:34:53.405 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.61433.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.61433.log.gz 2024-09-17T04:34:53.406 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55513.log 2024-09-17T04:34:53.406 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.49602.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49602.log.gz 2024-09-17T04:34:53.407 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75634.log 2024-09-17T04:34:53.408 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55513.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55513.log.gz 2024-09-17T04:34:53.409 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81529.log 2024-09-17T04:34:53.410 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.75634.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.75634.log.gz 2024-09-17T04:34:53.411 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62786.log 2024-09-17T04:34:53.412 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.81529.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.81529.log.gz 2024-09-17T04:34:53.412 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54818.log 2024-09-17T04:34:53.413 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.62786.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.62786.log.gz 2024-09-17T04:34:53.414 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53220.log 2024-09-17T04:34:53.416 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54818.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54818.log.gz 2024-09-17T04:34:53.416 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.53220.log: /var/log/ceph/ceph-client.admin.65479.log 2024-09-17T04:34:53.417 INFO:teuthology.orchestra.run.smithi100.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.53220.log.gz 2024-09-17T04:34:53.417 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56118.log 2024-09-17T04:34:53.419 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.65479.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65479.log.gz 2024-09-17T04:34:53.420 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59962.log 2024-09-17T04:34:53.420 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56118.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.56118.log.gz 2024-09-17T04:34:53.421 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52449.log 2024-09-17T04:34:53.422 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.59962.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59962.log.gz 2024-09-17T04:34:53.423 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81373.log 2024-09-17T04:34:53.424 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.52449.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52449.log.gz 2024-09-17T04:34:53.424 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43624.log 2024-09-17T04:34:53.426 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.81373.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.81373.log.gz 2024-09-17T04:34:53.426 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57529.log 2024-09-17T04:34:53.428 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43624.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.43624.log.gz 2024-09-17T04:34:53.428 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.57529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42969.log 2024-09-17T04:34:53.429 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57529.log.gz 2024-09-17T04:34:53.430 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66842.log 2024-09-17T04:34:53.431 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42969.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42969.log.gz 2024-09-17T04:34:53.432 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55429.log 2024-09-17T04:34:53.433 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.66842.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.66842.log.gz 2024-09-17T04:34:53.434 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60046.log 2024-09-17T04:34:53.434 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55429.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55429.log.gz 2024-09-17T04:34:53.435 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65876.log 2024-09-17T04:34:53.436 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.60046.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60046.log.gz 2024-09-17T04:34:53.437 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.65876.log: /var/log/ceph/ceph-client.admin.66716.log 2024-09-17T04:34:53.438 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65876.log.gz 2024-09-17T04:34:53.438 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80341.log 2024-09-17T04:34:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.66716.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.66716.log.gz 2024-09-17T04:34:53.440 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58831.log 2024-09-17T04:34:53.441 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.80341.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.80341.log.gz 2024-09-17T04:34:53.442 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65353.log 2024-09-17T04:34:53.443 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.58831.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58831.log.gz 2024-09-17T04:34:53.444 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61517.log 2024-09-17T04:34:53.444 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.65353.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65353.log.gz 2024-09-17T04:34:53.445 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70484.log 2024-09-17T04:34:53.446 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.61517.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61517.log.gz 2024-09-17T04:34:53.448 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43582.log 2024-09-17T04:34:53.448 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.70484.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.70484.log.gz 2024-09-17T04:34:53.449 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68677.log 2024-09-17T04:34:53.450 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43582.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43582.log.gz 2024-09-17T04:34:53.451 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73383.log 2024-09-17T04:34:53.451 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.68677.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68677.log.gz 2024-09-17T04:34:53.452 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80911.log 2024-09-17T04:34:53.453 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.73383.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.73383.log.gz 2024-09-17T04:34:53.454 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54944.log 2024-09-17T04:34:53.455 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.80911.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80911.log.gz 2024-09-17T04:34:53.455 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49710.log 2024-09-17T04:34:53.457 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54944.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54944.log.gz 2024-09-17T04:34:53.457 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49350.log 2024-09-17T04:34:53.458 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.49710.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49710.log.gz 2024-09-17T04:34:53.459 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72587.log 2024-09-17T04:34:53.460 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.49350.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49350.log.gz 2024-09-17T04:34:53.460 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54757.log 2024-09-17T04:34:53.461 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.72587.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.72587.log.gz 2024-09-17T04:34:53.463 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54757.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54757.log.gz 2024-09-17T04:34:53.463 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56076.log 2024-09-17T04:34:53.465 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54405.log 2024-09-17T04:34:53.466 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56076.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56076.log.gz 2024-09-17T04:34:53.466 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47457.log 2024-09-17T04:34:53.467 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54405.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54405.log.gz 2024-09-17T04:34:53.468 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49898.log 2024-09-17T04:34:53.469 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.47457.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47457.log.gz 2024-09-17T04:34:53.469 INFO:teuthology.orchestra.run.smithi184.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-17T04:34:53.470 INFO:teuthology.orchestra.run.smithi184.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-17T04:34:53.471 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.49898.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49898.log.gz 2024-09-17T04:34:53.471 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71584.log 2024-09-17T04:34:53.472 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48176.log 2024-09-17T04:34:53.474 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.71584.log: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.71584.log.gz 2024-09-17T04:34:53.474 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46079.log 2024-09-17T04:34:53.475 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.48176.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48176.log.gz 2024-09-17T04:34:53.476 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59504.log 2024-09-17T04:34:53.477 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46079.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46079.log.gz 2024-09-17T04:34:53.481 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51733.log 2024-09-17T04:34:53.482 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.59504.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59504.log.gz 2024-09-17T04:34:53.482 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80617.log 2024-09-17T04:34:53.482 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.51733.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51733.log.gz 2024-09-17T04:34:53.482 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.80617.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80617.log.gz 2024-09-17T04:34:53.483 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72293.log 2024-09-17T04:34:53.484 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59107.log 2024-09-17T04:34:53.484 INFO:teuthology.orchestra.run.smithi184.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-17T04:34:53.485 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.72293.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.72293.log.gz 2024-09-17T04:34:53.486 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66628.log 2024-09-17T04:34:53.486 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.59107.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59107.log.gz 2024-09-17T04:34:53.487 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69848.log 2024-09-17T04:34:53.488 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.66628.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66628.log.gz 2024-09-17T04:34:53.490 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.69848.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69848.log.gz 2024-09-17T04:34:53.490 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56900.log 2024-09-17T04:34:53.492 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65050.log 2024-09-17T04:34:53.493 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56900.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56900.log.gz 2024-09-17T04:34:53.493 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42238.log 2024-09-17T04:34:53.495 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.65050.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.65050.log.gz 2024-09-17T04:34:53.495 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74051.log 2024-09-17T04:34:53.496 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42238.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42238.log.gz 2024-09-17T04:34:53.497 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47992.log 2024-09-17T04:34:53.498 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.74051.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.74051.log.gz 2024-09-17T04:34:53.499 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74710.log 2024-09-17T04:34:53.499 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.47992.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47992.log.gz 2024-09-17T04:34:53.500 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71625.log 2024-09-17T04:34:53.501 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.74710.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.74710.log.gz 2024-09-17T04:34:53.503 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52232.log/var/log/ceph/ceph-client.admin.71625.log: 2024-09-17T04:34:53.503 INFO:teuthology.orchestra.run.smithi100.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.71625.log.gz 2024-09-17T04:34:53.504 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36956.log 2024-09-17T04:34:53.505 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.52232.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52232.log.gz 2024-09-17T04:34:53.506 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57884.log 2024-09-17T04:34:53.506 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36956.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36956.log.gz 2024-09-17T04:34:53.507 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79496.log 2024-09-17T04:34:53.508 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.57884.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57884.log.gz 2024-09-17T04:34:53.509 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43498.log 2024-09-17T04:34:53.510 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.79496.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.79496.log.gz 2024-09-17T04:34:53.511 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69032.log 2024-09-17T04:34:53.512 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43498.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43498.log.gz 2024-09-17T04:34:53.513 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45749.log 2024-09-17T04:34:53.513 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.69032.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.69032.log.gz 2024-09-17T04:34:53.515 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.45749.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45749.log.gz 2024-09-17T04:34:53.515 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50936.log 2024-09-17T04:34:53.517 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43081.log 2024-09-17T04:34:53.518 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.50936.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50936.log.gz 2024-09-17T04:34:53.519 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59904.log 2024-09-17T04:34:53.520 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43081.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43081.log.gz 2024-09-17T04:34:53.520 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75974.log 2024-09-17T04:34:53.521 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.59904.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59904.log.gz 2024-09-17T04:34:53.523 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.75974.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.75974.log.gz 2024-09-17T04:34:53.523 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36597.log 2024-09-17T04:34:53.525 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74490.log 2024-09-17T04:34:53.526 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36597.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36597.log.gz 2024-09-17T04:34:53.527 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57968.log 2024-09-17T04:34:53.527 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.74490.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74490.log.gz 2024-09-17T04:34:53.528 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53412.log 2024-09-17T04:34:53.529 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.57968.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57968.log.gz 2024-09-17T04:34:53.530 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66674.log 2024-09-17T04:34:53.531 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.53412.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53412.log.gz 2024-09-17T04:34:53.532 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43086.log 2024-09-17T04:34:53.533 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.66674.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.66674.log.gz 2024-09-17T04:34:53.534 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73425.log 2024-09-17T04:34:53.534 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43086.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43086.log.gz 2024-09-17T04:34:53.535 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74878.log 2024-09-17T04:34:53.536 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.73425.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73425.log.gz 2024-09-17T04:34:53.538 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.74878.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74878.log.gz 2024-09-17T04:34:53.538 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78877.log 2024-09-17T04:34:53.540 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62853.log 2024-09-17T04:34:53.541 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.78877.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78877.log.gz 2024-09-17T04:34:53.542 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78919.log 2024-09-17T04:34:53.542 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.62853.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.62853.log.gz 2024-09-17T04:34:53.543 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42753.log 2024-09-17T04:34:53.545 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.78919.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.78919.log.gz 2024-09-17T04:34:53.546 INFO:teuthology.orchestra.run.smithi184.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-17T04:34:53.546 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42753.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42753.log.gz 2024-09-17T04:34:53.546 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65093.log 2024-09-17T04:34:53.548 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59730.log 2024-09-17T04:34:53.549 INFO:teuthology.orchestra.run.smithi184.stderr: 2024-09-17T04:34:53.549 INFO:teuthology.orchestra.run.smithi184.stderr:real 0m0.896s 2024-09-17T04:34:53.549 INFO:teuthology.orchestra.run.smithi184.stderr:user 0m4.151s 2024-09-17T04:34:53.549 INFO:teuthology.orchestra.run.smithi184.stderr:sys 0m0.499s 2024-09-17T04:34:53.549 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.65093.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.65093.log.gz 2024-09-17T04:34:53.551 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77931.log 2024-09-17T04:34:53.551 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.59730.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.59730.log.gz 2024-09-17T04:34:53.551 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55028.log 2024-09-17T04:34:53.552 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.77931.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.77931.log.gz 2024-09-17T04:34:53.553 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78015.log 2024-09-17T04:34:53.554 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55028.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55028.log.gz 2024-09-17T04:34:53.555 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49262.log 2024-09-17T04:34:53.557 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.78015.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78015.log.gz 2024-09-17T04:34:53.557 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75592.log 2024-09-17T04:34:53.558 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.49262.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49262.log.gz 2024-09-17T04:34:53.559 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48488.log 2024-09-17T04:34:53.559 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.75592.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75592.log.gz 2024-09-17T04:34:53.560 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46575.log 2024-09-17T04:34:53.561 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.48488.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48488.log.gz 2024-09-17T04:34:53.562 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68635.log 2024-09-17T04:34:53.563 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46575.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46575.log.gz 2024-09-17T04:34:53.564 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58747.log 2024-09-17T04:34:53.565 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.68635.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68635.log.gz 2024-09-17T04:34:53.566 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78400.log 2024-09-17T04:34:53.567 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.58747.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58747.log.gz 2024-09-17T04:34:53.567 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36037.log 2024-09-17T04:34:53.568 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.78400.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78400.log.gz 2024-09-17T04:34:53.569 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-17T04:34:53.570 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36037.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36037.log.gz 2024-09-17T04:34:53.571 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78751.log 2024-09-17T04:34:53.573 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79339.log 2024-09-17T04:34:53.574 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75149.log 2024-09-17T04:34:53.575 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.78751.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.78751.log.gz 2024-09-17T04:34:53.576 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73215.log 2024-09-17T04:34:53.579 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.75149.log: 87.6%/var/log/ceph/ceph-client.admin.73215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66004.log 2024-09-17T04:34:53.580 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.79339.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.79339.log.gz 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-17T04:34:53.580 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-09-17T04:34:53.580 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73215.log.gz 2024-09-17T04:34:53.580 INFO:teuthology.orchestra.run.smithi100.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75149.log.gz 2024-09-17T04:34:53.582 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74147.log 2024-09-17T04:34:53.583 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.66004.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66004.log.gz 2024-09-17T04:34:53.583 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-17T04:34:53.584 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.74147.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.74147.log.gz 2024-09-17T04:34:53.585 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72843.log 2024-09-17T04:34:53.587 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.72843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61475.log 2024-09-17T04:34:53.589 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.72843.log.gz 2024-09-17T04:34:53.590 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68990.log 2024-09-17T04:34:53.592 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.61475.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61475.log.gz 2024-09-17T04:34:53.594 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.68990.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68990.log.gz 2024-09-17T04:34:53.595 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49081.log 2024-09-17T04:34:53.597 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.49081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52148.log 2024-09-17T04:34:53.598 INFO:teuthology.orchestra.run.smithi100.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49081.log.gz 2024-09-17T04:34:53.600 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72503.log 2024-09-17T04:34:53.602 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.52148.log: 87.2%gzip -5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.52148.log.gz /var/log/ceph/ceph-client.admin.80659.log 2024-09-17T04:34:53.603 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-09-17T04:34:53.603 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.72503.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.72503.log.gz 2024-09-17T04:34:53.604 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42745.log 2024-09-17T04:34:53.606 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.80659.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80659.log.gz 2024-09-17T04:34:53.607 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42028.log 2024-09-17T04:34:53.608 INFO:teuthology.orchestra.run.smithi100.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.42745.log.gz 2024-09-17T04:34:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66968.log 2024-09-17T04:34:53.611 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42028.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42028.log.gz 2024-09-17T04:34:53.612 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45336.log 2024-09-17T04:34:53.615 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.66968.log: 87.2%/var/log/ceph/ceph-client.admin.45336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72419.log 2024-09-17T04:34:53.616 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45336.log.gz 2024-09-17T04:34:53.616 INFO:teuthology.orchestra.run.smithi100.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66968.log.gz 2024-09-17T04:34:53.617 INFO:teuthology.orchestra.run.smithi100.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-17T04:34:53.618 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.72419.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72419.log.gz 2024-09-17T04:34:53.873 INFO:teuthology.orchestra.run.smithi100.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-17T04:34:54.005 INFO:teuthology.orchestra.run.smithi100.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-17T04:34:54.008 INFO:teuthology.orchestra.run.smithi100.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-17T04:34:54.053 INFO:teuthology.orchestra.run.smithi100.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-17T04:34:54.113 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-09-17T04:34:54.113 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-09-17T04:34:54.147 INFO:teuthology.orchestra.run.smithi100.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-17T04:34:54.290 INFO:teuthology.orchestra.run.smithi100.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-17T04:34:54.294 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-09-17T04:34:54.294 INFO:teuthology.orchestra.run.smithi100.stderr:real 0m1.645s 2024-09-17T04:34:54.294 INFO:teuthology.orchestra.run.smithi100.stderr:user 0m6.092s 2024-09-17T04:34:54.294 INFO:teuthology.orchestra.run.smithi100.stderr:sys 0m2.285s 2024-09-17T04:34:54.294 INFO:tasks.ceph:Archiving logs... 2024-09-17T04:34:54.295 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908519/remote/smithi100/log 2024-09-17T04:34:54.296 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T04:34:55.634 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908519/remote/smithi184/log 2024-09-17T04:34:55.635 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T04:34:56.203 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-17T04:34:56.213 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-17T04:34:56.213 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T04:34:56.217 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T04:34:56.333 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-17T04:34:56.342 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-17T04:34:56.342 DEBUG:teuthology.orchestra.run.smithi100:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T04:34:56.348 DEBUG:teuthology.orchestra.run.smithi184:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T04:34:56.377 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-09-17T04:34:56.382 INFO:teuthology.orchestra.run.smithi184.stderr:bash: line 1: ntpq: command not found 2024-09-17T04:34:56.384 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T04:34:56.385 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-09-17T04:34:56.385 INFO:teuthology.orchestra.run.smithi100.stdout:^* hv01.front.sepia.ceph.com 3 6 377 59 +1523us[+1520us] +/- 58ms 2024-09-17T04:34:56.385 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 59 +214us[ +214us] +/- 73ms 2024-09-17T04:34:56.385 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 -2657us[-2660us] +/- 86ms 2024-09-17T04:34:56.385 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T04:34:56.389 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T04:34:56.389 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2024-09-17T04:34:56.389 INFO:teuthology.orchestra.run.smithi184.stdout:^* hv01.front.sepia.ceph.com 3 6 377 57 +1554us[+1563us] +/- 58ms 2024-09-17T04:34:56.389 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 62 +240us[ +249us] +/- 73ms 2024-09-17T04:34:56.389 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 58 -2626us[-2617us] +/- 86ms 2024-09-17T04:34:56.389 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T04:34:56.390 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-17T04:34:56.400 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-17T04:34:56.400 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-17T04:34:56.462 DEBUG:teuthology.orchestra.run.smithi100:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T04:34:56.602 DEBUG:teuthology.orchestra.run.smithi184:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T04:34:56.721 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T04:34:56.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T04:34:56.784 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T04:34:56.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T04:34:56.844 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-17T04:34:56.854 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-17T04:34:56.918 INFO:teuthology.task.internal:Duration was 1127.322545 seconds 2024-09-17T04:34:56.918 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-17T04:34:56.979 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-17T04:34:56.980 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T04:34:56.984 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T04:34:57.066 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T04:34:57.070 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T04:34:57.565 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-17T04:34:57.565 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi100.front.sepia.ceph.com 2024-09-17T04:34:57.565 DEBUG:teuthology.orchestra.run.smithi100:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T04:34:57.614 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi184.front.sepia.ceph.com 2024-09-17T04:34:57.615 DEBUG:teuthology.orchestra.run.smithi184:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T04:34:57.665 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-17T04:34:57.665 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T04:34:57.669 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T04:34:57.728 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-17T04:34:57.728 DEBUG:teuthology.orchestra.run.smithi100:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T04:34:57.732 DEBUG:teuthology.orchestra.run.smithi184:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T04:34:57.907 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-17T04:34:57.917 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-17T04:34:57.917 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T04:34:57.972 DEBUG:teuthology.orchestra.run.smithi184:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T04:34:58.028 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-17T04:34:58.038 DEBUG:teuthology.orchestra.run.smithi100:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T04:34:58.042 DEBUG:teuthology.orchestra.run.smithi184:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T04:34:58.109 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = core 2024-09-17T04:34:58.113 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = core 2024-09-17T04:34:58.142 DEBUG:teuthology.orchestra.run.smithi100:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T04:34:58.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T04:34:58.175 DEBUG:teuthology.orchestra.run.smithi184:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T04:34:58.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T04:34:58.209 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-17T04:34:58.220 INFO:teuthology.task.internal:Transferring archived files... 2024-09-17T04:34:58.220 DEBUG:teuthology.misc:Transferring archived files from smithi100:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908519/remote/smithi100 2024-09-17T04:34:58.221 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T04:34:58.287 DEBUG:teuthology.misc:Transferring archived files from smithi184:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908519/remote/smithi184 2024-09-17T04:34:58.287 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T04:34:58.350 INFO:teuthology.task.internal:Removing archive directory... 2024-09-17T04:34:58.350 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T04:34:58.354 DEBUG:teuthology.orchestra.run.smithi184:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T04:34:58.392 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-17T04:34:58.404 INFO:teuthology.task.internal:Not uploading archives. 2024-09-17T04:34:58.404 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-17T04:34:58.457 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-17T04:34:58.457 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T04:34:58.461 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T04:34:58.492 INFO:teuthology.orchestra.run.smithi100.stdout: 393731 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 04:34 /home/ubuntu/cephtest 2024-09-17T04:34:58.496 INFO:teuthology.orchestra.run.smithi184.stdout: 393233 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 04:34 /home/ubuntu/cephtest 2024-09-17T04:34:58.498 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-17T04:34:58.553 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-17T04:34:58.670 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a client.1-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}} duration: 1127.3225452899933 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-17T04:34:58.671 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T04:34:58.749 INFO:teuthology.run:pass